Workflows

Repeatable engineering workflows with explicit verification.

364 published

Page 10 of 11

Restore a Production Backup and Prove It Works

Restore a production backup into an isolated database, verify row counts and application queries, and measure the real recovery time before an incident demands it.

intermediate~45 min
database · backup · reliability

Review a Third-Party Vendor Before Contract Renewal

Assess a vendor's security evidence, data flows, reliability history and exit options before renewal, then record a decision with named owners and dated follow-ups.

intermediate~45 min
security · vendor · governance

Rotate Database Application Credentials Without Downtime

Replace an application database password in controlled phases: create a successor role, migrate every consumer, confirm zero old connections, then disable the old login.

intermediate~45 min
database · security · operations

Run a Tabletop Exercise for a Critical Incident

Rehearse a realistic outage with responders and a facilitator, inject pressure over ninety minutes, and convert observations into owned, dated remediation work.

beginner~45 min
incident · training · operations

Upgrade PostgreSQL a Major Version with a Rehearsed Cutover

Move a production PostgreSQL cluster to the next major version with a rehearsed pg_upgrade run, a verified backup and a cutover that can fall back to the old cluster.

advanced~45 min
database · operations · reliability

Backfill a Large Table Safely

Backfill data in small idempotent batches, track a watermark, throttle writes, verify coverage, and avoid long locks or replication lag.

intermediate~60 min
database · migration · operations

Blue-Green Deployment with a Reversible Switch

Run two identical environments, deploy to the idle color, verify it against production checks, switch traffic atomically, and keep a tested rollback path.

intermediate~45 min
deployment · reliability · operations

Rotate TLS Certificates Without Downtime

Inventory certificates, issue replacements, deploy them with overlap, verify the served chain and expiry, and automate renewal before expiration becomes an outage.

intermediate~45 min
security · tls · operations

Run a Controlled Chaos Experiment

State a falsifiable resilience hypothesis, define steady state and blast radius, inject one failure, and abort automatically if the guardrail trips.

advanced~60 min
reliability · chaos-engineering · operations

Configure Log Retention and Archival

Classify logs by purpose and sensitivity, set hot/warm/cold retention, archive to object storage, and prove deletion without losing audit evidence.

intermediate~45 min
logging · privacy · operations

Create a Database Index Without Downtime

Build an index concurrently, detect invalid builds, watch replication lag, and avoid locking writes on a large production table.

intermediate~45 min
database · performance · operations

Implement a Data Retention Cleanup

Inventory data by class, define retention and deletion rules, dry-run the cleanup, delete in bounded batches, and verify while preserving audit evidence.

intermediate~75 min
data-governance · privacy · operations

Run a Database Failover Drill

Practice promoting a replica, reconnecting the application, verifying writes, and switching back before a real outage forces an untested failover.

advanced~90 min
database · reliability · drills

Define Incident Severity Levels

Classify incidents by customer impact and urgency, assign paging and communication rules, and avoid every alert becoming a SEV1.

beginner~20 min
operations · incident-response · reliability

Plan a Major Dependency Upgrade

Read the migration guide, isolate the upgrade in a branch, run codemods and compatibility tests, canary the change, and keep a tested rollback.

intermediate~60 min
dependencies · upgrades · operations

Run a Multi-Region Failover Drill

Simulate loss of a primary region, shift traffic to a secondary, verify data freshness and writes, then fail back without leaving split-brain risk.

advanced~120 min
reliability · disaster-recovery · operations

Run an On-Call Handoff

Transfer on-call responsibility with current incidents, risky changes, capacity notes, alert noise, and explicit ownership so nothing depends on memory.

beginner~20 min
operations · on-call · reliability

Triage a Performance Regression

Confirm a latency or throughput regression against a baseline, narrow it to a change or dependency, mitigate safely, and add a guardrail that catches the next one.

advanced~75 min
performance · operations · debugging

Run a Production Access Review

Inventory production identities and permissions, remove stale or excessive access, verify MFA and break-glass paths, and schedule the next review.

intermediate~90 min
security · access-control · compliance

Run a Production Readiness Review

Review a service before launch across ownership, SLOs, capacity, security, observability, failure handling, data safety, and rollback.

intermediate~90 min
operations · reliability · launch

Retire a Feature Flag Safely

Confirm full rollout, remove the old code path and flag checks, deploy in the safe order, and delete the flag configuration without leaving dead branches.

beginner~20 min
delivery · feature-flags · operations

Review API Contract Changes Before Merge

Diff the OpenAPI document, classify changes as breaking or compatible, collect consumer impact, and require a migration plan for breaking changes.

intermediate~30 min
api · contracts · ci-cd

Rotate API Keys Without Breaking Clients

Issue a new key, accept old and new during a migration window, track client usage, and revoke the old key only after every caller has moved.

intermediate~40 min
security · api · operations

Plan a Schema Migration Rollback

Design a database migration that can be rolled back without data loss, rehearse it on a copy, measure lock impact, and keep old and new application versions compatible.

advanced~90 min
database · migration · operations

Security Incident Response Runbook

Contain a suspected credential compromise, preserve evidence, rotate affected secrets, eradicate persistence, and recover with verification and clear communication.

advanced~120 min
security · incident-response · operations

Accessibility Audit Workflow

Combine automated checks with keyboard and screen-reader testing, fix findings by severity, and add regression checks for critical journeys.

intermediate~90 min
accessibility · testing · quality

API Versioning and Sunset Release

Introduce a new API version, migrate consumers with measured adoption, announce deprecation with standard headers, and remove the old version only after traffic reaches zero.

intermediate~60 min
api-design · versioning · operations

Backup and Restore Drill

Prove that backups are usable by restoring them into an isolated environment, verifying data integrity, measuring recovery time, and documenting gaps.

intermediate~90 min
backup · disaster-recovery · databases

Blameless Postmortem Workflow

Reconstruct an incident timeline, identify contributing system causes, assign verifiable action items, and review completion without blaming individuals.

intermediate~60 min
operations · postmortem · reliability

Canary Release with Metric Gates

Route a small slice of traffic to a new version, compare it against the stable baseline on explicit metrics, and promote or roll back based on evidence.

advanced~60 min
deployment · canary · reliability

Dependency Upgrade Workflow

Upgrade dependencies in small, reversible batches with lockfile verification, vulnerability checks, tests, staging validation, and a rollback plan.

intermediate~45 min
dependencies · security · maintenance

Expand-Contract Database Migration

Ship a schema change in backward-compatible phases so old and new application versions can run simultaneously during a rolling deployment.

advanced~90 min
database · migration · deployment

Feature Flag Staged Rollout

Release a feature behind a flag, expose it to progressively larger audiences, monitor impact, and remove the flag after full rollout or rollback.

intermediate~40 min
deployment · feature-flags · product

Incident Response Runbook

Detect, declare, mitigate, communicate, and review an incident with explicit roles and evidence, prioritizing restoration over root-cause analysis during the outage.

intermediate~60 min
operations · incident-response · reliability

Load Test Before Launch

Define realistic scenarios, measure baseline and peak behavior, identify saturation points, and decide capacity with evidence instead of guesses.

intermediate~60 min
performance · testing · capacity

Scheduled Secret Rotation Workflow

Rotate credentials with overlapping validity, update consumers in a controlled order, verify access, and revoke the old version without downtime.

intermediate~60 min
security · secrets · operations