Putting together an observability stack

Every capability covered in this domain is available both as self-hosted open-source software and as a paid managed platform — the underlying concepts (a counter is a counter, a trace is a trace) don't change based on which path a team takes, but the operational trade-offs are genuinely different.

A common, well-established open-source combination, each piece handling one of this domain's pillars:

Prometheus scrapes and stores metrics (lessons 2–3's counters, gauges, histograms). Loki stores logs, designed to integrate tightly with the same labels Prometheus uses for metrics. Tempo stores traces, ingesting OpenTelemetry data (lesson 6) directly. Grafana sits on top of all three as the shared dashboard and alerting layer — one place to build the dashboards from lesson 9 and the alerts from lesson 8, querying across metrics, logs, and traces together.

A managed observability platform bundles equivalent capabilities — metrics, logs, traces, dashboards, alerting, synthetic checks (lesson 10), RUM (lesson 11) — as one integrated product, typically ingesting data via OpenTelemetry (lesson 6) the same way the open-source stack does, so instrumentation code is often largely portable between the two paths.

Self-hosting avoids per-metric or per-host licensing costs but means a team is now responsible for running, scaling, and maintaining the observability stack itself — genuinely nontrivial at real data volume, since Prometheus, Loki, and Tempo all have their own operational characteristics and failure modes to learn and manage. A managed platform removes that operational burden entirely, at a real, often usage-scaling cost that can become substantial at high volume — the same "who operates this" trade-off System Design's discussion of managed versus self-hosted databases makes for storage, just applied here to the observability stack itself.

Every concept in this domain — RED and USE (lesson 3), SLOs and error budgets (lesson 5), trace context propagation (lesson 6), sampling (lesson 7), alert design (lesson 8), dashboard structure (lesson 9), synthetic and real-user monitoring (lessons 10–11) — applies identically regardless of which specific tool ends up implementing it. A team that understands why an SLO-based alert beats a raw threshold, or why tail-based sampling exists, can evaluate and configure any tool correctly; a team that only knows "click here to make a Datadog alert" is stuck the moment the underlying tool changes, without ever having learned what the tool was actually supposed to accomplish.

Because OpenTelemetry (lesson 6) has become the common instrumentation standard both self-hosted and managed backends increasingly accept directly, a team's actual application instrumentation is largely portable between the two paths — the collector's export configuration (lesson 6) is often the main thing that changes when switching, not the instrumentation scattered throughout application code. This is precisely the decoupling OpenTelemetry was built to provide, and it's what makes "start simple, reconsider later" a genuinely viable strategy rather than a decision made once and never revisited.

Observability isn't a single tool or a checklist to complete once — it's the ongoing discipline of knowing what's actually happening inside a system, at the right level of detail, fast enough to act on it. Every lesson in this domain is one piece of that discipline: what to measure, how good it needs to be, how to trace a request across services, how to alert without exhausting the people who respond to those alerts, and how to see what's true for users a synthetic check alone would never catch.