Case Studies
Metrics Monitoring
Ingesting a firehose of time-series data and making long-term retention viable through hierarchical downsampling.
Designing a system that ingests time-series metrics from thousands of hosts and services, stores them efficiently for both live dashboards and long-term trend queries, evaluates alerts reliably even when the rest of the pipeline lags, and drives notifications — the concrete design behind the "how would you actually know" question Reliability's Observability section raises but doesn't build out.
What this lesson covers
Host agents push batched points through Kafka, partitioned by series, into a write-optimized time-series store. Keep raw resolution for a short window and downsample for long retention, and evaluate alerts on a path that stays reliable when the rest of the pipeline lags.
- Live dashboards want the last few minutes instantly; trend queries want months of history; alerts must fire even when the pipeline lags.
- Ingesting a firehose of tagged time-series points is a write-path problem, shaped nothing like the read path that queries it.
- Storing every raw point forever is not viable; retention hinges on downsampling and routing each query to the right resolution.
- Alert on a sustained window rather than a single noisy point, and never let alerting depend on the query path staying caught up.
- Staff answers treat cardinality as its own growth axis and reason about late-arriving data and where a grace period belongs.
Included in Plus and Pro
Continue reading “Metrics Monitoring”
The full lesson works the design through end to end — diagrams, trade-offs, and what interviewers expect at each level. About 18 minutes.
- Understanding the Problem
- Functional Requirements
- Non-Functional Requirements
- Capacity Estimation
- Core Entities
- API Interface
- High-Level Design
- Trade-offs
- Final Design
- Operations & Observability
- Level Expectations
- Follow-Up Questions
- Try It Yourself