Load Shedding Workflow
Classify requests by criticality, shed the lowest tier first when saturation rises, and shed progressively so the system stays available for the most important traffic.
Objective
Keep the system available for critical traffic during overload by shedding less important requests in priority order.
Environment
- A classifier that tags each request with a criticality tier
- A saturation signal like queue depth or CPU
- A load shedder that can drop a tier at a threshold
Steps
1. Classify requests
Tag every request with a criticality tier, like checkout above browse above bot, and make the tag visible to the shedder.
Expected state: Every request carries a tier the shedder can act on without re-parsing.
2. Set shed thresholds
For each tier set a saturation threshold, with the lowest tier shed first and the highest shed last or never.
Expected state: A threshold table where tier 1 sheds at 70 percent CPU, tier 2 at 85, tier 3 never.
3. Shed progressively
As saturation rises, shed one tier at a time and watch saturation fall before shedding the next.
Expected state: The system sheds the minimum tier to hold saturation and recovers as load drops.
4. Report what was shed
Count shed requests per tier and export the metric so the team sees what was dropped and when.
Expected state: A dashboard with shed counts per tier so an overload event is visible after the fact.
Decision points
Which tier never sheds?
- Checkout and write paths
- All read paths
Write paths that lose data on a shed; a dropped browse is recoverable, a dropped checkout is not.
Shed by error or by delay?
- Return 503 immediately
- Queue and delay
Return 503 immediately; a queued request under overload becomes a timeout.
Expected outcome
The system stays available for critical traffic during overload, sheds the minimum tier needed, and reports what was dropped.
Verification
Confirm the shedder drops the lowest tier first, the saturation holds, and the dashboard shows shed counts per tier.
Cleanup
Reset the thresholds after the overload, archive the shed counts, and review the tier boundaries for the next event.
Classify before you shed
You cannot shed what you cannot classify. Tag every request with a criticality tier at the edge, and make the tag visible to the shedder. Checkout is more critical than browse; browse is more critical than a bot crawl.
shedd.shedIf(cpu > 0.70 && tier === 'browse');Shed in order
Set a threshold per tier, lowest first. As saturation rises, shed one tier at a time and watch the saturation fall before shedding the next. The goal is to shed the minimum tier that holds the system.
Report what was dropped
Count shed requests per tier and export the metric. An overload that is invisible after the fact cannot be prevented next time. Review the tier boundaries after every event.
Verification
The shedder drops the lowest tier first, saturation holds, and the dashboard shows shed counts per tier. Return 503 immediately, not a queue; a queued request under overload becomes a timeout. Never shed write paths that lose data.