Detect Anomalies Against a Rolling Baseline
Compute a 7-day rolling median and alert when the current value exceeds 3 sigma so a gradual regression is caught not just spikes.
Task
Instrument const median = rollingMedian(values, 7); const anomaly = Math.abs(v - median) > 3 * std; so on-call engineers can diagnose an incident without guessing at log lines.
When to use this
An outage lasts 47 minutes because the team cannot correlate const median = rollingMedian(values, 7); const anomaly = Math.abs(v - median) > 3 * std; across services without structured traces.
Prerequisites
- An application with structured logging available
- A metrics or tracing backend configured
Steps
1. Configure const median = rollingMedian(values, 7); const anomaly = Math.abs(v - median) > 3 * std; in the project configuration file with the recommended defaults.
const median = rollingMedian(values, 7); const anomaly = Math.abs(v - median) > 3 * std;
Expected output: The configuration is accepted and const median = rollingMedian(values, 7); const anomaly = Math.abs(v - median) > 3 * std; runs without errors on the first invocation.
2. Run the check locally to verify const median = rollingMedian(values, 7); const anomaly = Math.abs(v - median) > 3 * std; produces the expected result before pushing.
pnpm test
Expected output: Local run matches CI behavior; no surprises after push.
3. Add the gate to the CI pipeline so const median = rollingMedian(values, 7); const anomaly = Math.abs(v - median) > 3 * std; blocks merge on failure.
- name: Run add anomaly detection baseline
run: pnpm test
Expected output: Pull requests with failing const median = rollingMedian(values, 7); const anomaly = Math.abs(v - median) > 3 * std; are blocked from merging.
Troubleshooting
Problem
The check passes locally but fails in CI
Solution
Pin dependency versions and the runtime to match CI exactly; reproduce with the same Docker image or act.
Problem
Configuration is rejected with a schema error
Solution
Check the tool version matches the config schema; older versions may not support newer fields.
Limitations
- The gate only runs on events that trigger the workflow; locally pushed commits bypass it.
- Configuration drift is possible if someone edits settings outside version control.
Make the system observable
Start by measuring the problem. Compute a 7-day rolling median and alert when the current value exceeds 3 sigma so a gradual regression is caught not just spikes. The most common mistake is adding the gate without first confirming it catches the issue you care about. Run const median = rollingMedian(values, 7); const anomaly = Math.abs(v - median) > 3 * std; against the current codebase and inspect the output before wiring it into CI.
const median = rollingMedian(values, 7); const anomaly = Math.abs(v - median) > 3 * std;Correlate across services
Once the gate is in place, make failures actionable. A failing const median = rollingMedian(values, 7); const anomaly = Math.abs(v - median) > 3 * std; should point to the exact file and line, not a generic error. Pin the tool version in the lockfile so a upstream release does not silently change behavior. Review the gate output in every pull request just like a test result.