Run a Production Readiness Review
Review a service before launch across ownership, SLOs, capacity, security, observability, failure handling, data safety, and rollback.
Objective
Decide with evidence whether a service is safe to expose to production traffic, and record the gaps that must close before launch.
Environment
- A service with a named owner and an on-call rotation
- Defined SLOs and a dashboard with request, error, latency, and saturation metrics
- Load-test results at expected peak
- A tested rollback or feature-flag disable path
Steps
1. Confirm ownership and escalation
Name the service owner, the on-call rotation, the escalation path, and the runbook location.
Expected state: An engineer paged at 3 a.m. knows who owns the service and what to do first.
2. Review SLOs and alerts
Check that user-facing SLOs exist, that alerts fire on symptoms rather than every cause, and that each alert links to a runbook.
Expected state: Every page has an owner, a threshold, and an action.
3. Verify capacity and failure behavior
Run a load test at expected peak plus headroom, review dependency limits, and exercise the degradation path for each critical dependency.
Expected state: The service meets its latency target at peak and degrades predictably when a dependency fails.
4. Review security and data handling
Check authentication, authorization, secret management, input limits, dependency vulnerabilities, and data retention.
Expected state: No known critical vulnerability, no secret in the image or repository, and a documented data lifecycle.
5. Exercise rollback and disaster recovery
Deploy a canary, roll it back, and restore from a backup in a non-production environment.
Expected state: Rollback and restore are measured, documented, and within the recovery objectives.
6. Record the decision
List blockers, accepted risks with owners, and a launch decision. Do not mark ready with an unowned critical gap.
Expected state: The launch decision and its evidence are stored with the service.
Decision points
Can a service launch with a known high-severity security finding?
- No, close it or mitigate before launch
- Yes, with a compensating control and an owner
Block launch for an exploitable high-severity finding unless a documented compensating control reduces the risk to acceptable.
Is a manual rollback acceptable?
- Yes, if it is tested and timed
- No, require an automated switch
Accept a manual rollback only when it is rehearsed and fits the recovery objective; otherwise automate it.
Expected outcome
The service launches only after ownership, SLOs, capacity, security, rollback, and recovery are demonstrated, with accepted risks explicitly recorded.
Verification
Re-run the checklist against the evidence links; confirm each blocker is closed or has a named owner and deadline, and that the on-call engineer can execute the rollback from the runbook.
Cleanup
Archive the review record, schedule the next review, and convert accepted risks into tracked work.
A launch decision needs evidence
A production readiness review is not a meeting where everyone says the service looks fine. It is an evidence review. For each area, the team points to a dashboard, a test result, a runbook, or a recorded exercise. Missing evidence is a gap, not a reason to proceed faster.
Owner: payments-api
SLO: 99.9% successful requests, p95 < 300 ms
Peak test: 2x expected traffic, p95 240 ms, error rate 0.05%
Rollback: feature flag off in 40 seconds, tested 2026-10-12
Backup restore: 18 minutes, tested 2026-10-10
Open risks: none criticalReview failure, not only success
The happy-path demo is the least informative part of the review. Ask what happens when the database is slow, the queue is full, a dependency returns 500, or a deploy is bad. Review the timeouts, retries, circuit breakers, and fallbacks, and verify they are exercised. A service without a tested degradation path is relying on its dependencies never failing.
Record accepted risks
Some gaps are acceptable for a launch if the risk is understood and owned. Write them down with severity, mitigation, owner, and deadline. An accepted risk without an owner is an unmanaged risk. Revisit the list at the next review; a growing list of expired mitigations is a signal that the review is ceremonial.
Keep it proportionate
A small internal tool does not need the same review as a payment service. Scale the depth to the blast radius and data sensitivity. The invariant is the same: the decision must be based on evidence, and the people who will be paged must be able to operate the service.