Reading Program
Observability, SRE and chaos
Postmortems, structured events and chaos experiments for weeks 14 to 25.
This page covers weeks 14 to 25, when you build Shiplane and Faultline and start the ops season. Read the four canonical posts in build order. Add one rotating blog on the free days.
Read before the incident
Before your first page
Read the postmortem post before you deploy Shiplane, not after. You write a clearer timeline while the decision is fresh, and the template is easier to follow before you have something to hide.
Canonical posts
| # | Week | Post | What to take |
|---|---|---|---|
| 1 | 14 | Google SRE Book: Postmortem Culture | The template for postmortem-001.md: blameless, timeline, impact, action items. |
| 2 | 19 | incident.io: SRE postmortem best practices | Practical formatting and action-item tracking. |
| 3 | 16 | Honeycomb: Observability, a Manifesto | Why metrics need structured events, not loose log lines. |
| 4 | 16 | Charity Majors: Live your best life with structured events | Read before you design /metrics scraping. |
Chaos stack
- Netflix Chaos Monkey docs: the tool that started the discipline. Scope your kill-tick design to this, not to cron.
- Netflix TechBlog: Chaos Engineering Upgraded (2015): how "kill things at random" became a hypothesis-driven experiment. Model Faultline's report on it.
- Google Cloud: Getting started with chaos engineering: a modern primer. Use it to keep a solo project in scope.
Rotating blogs
- Julia Evans: debugging and Linux deep dives. Read from week 14.
- Brendan Gregg: performance method, USE and FlameGraphs, from the person who named them.
- Grafana Labs blog: metrics, logs and traces, plus honest postmortems.
- Datadog Engineering blog: SRE practice at scale.
- Last9 blog: high-cardinality observability from an Indian team solving the problems Shiplane invents.
- Marc Brooker's blog: short notes on reliability and randomness.
Common mistakes
- Writing the postmortem as a defence. Write the timeline first, blame nobody.
- Adding dashboards before you have a question. Fix one p99 number instead.
- Running chaos experiments with no hypothesis or stop condition.