← Debugging Guides

阅读中文版本

AWS SigV4 — RequestTimeTooSkewed 403 After Clock Drift

Fix AWS SignatureDoesNotMatch and RequestTimeTooSkewed errors by checking NTP synchronization, host clocks in VMs and containers, and credential scope correctness.

beginnerdebuggingawssigv4time

Symptoms

  • API calls fail with RequestTimeTooSkewed: The difference between the request time and the server time is too large
  • SignatureDoesNotMatch appears after a snapshot restore, VM pause, or container migration
  • Errors start on one host while identical code on other hosts succeeds

Possible causes

  • The host clock drifted beyond the SigV4 tolerance window (about 15 minutes) because NTP is stopped or blocked
  • A VM was paused or a container live-migrated, and the clock jumped on resume
  • The request timestamp is taken from an application cache instead of the current time
  • Credential scope date or region mismatches the request (signed for yesterday region or wrong service)

Diagnosis steps

Compare local and AWS time

date -u && curl -sI https://sts.amazonaws.com | grep -i ^date

Interpretation

A difference beyond a few seconds between your clock and the AWS Date header is the skew, measured directly against the authority.

Check NTP synchronization state

timedatectl status; chronyc tracking 2>/dev/null || ntpq -p 2>/dev/null

Interpretation

System clock synchronized: no, or a large offset value, identifies the host whose drift produced the 403.

Verify the signed scope

grep -Rn "new Date\|Date.now" src/aws-client.ts | head; check SDK version pinning

Interpretation

Timestamps cached at module load or a wrong region/service in the credential scope produce SignatureDoesNotMatch rather than skew errors.

Root cause

SigV4 signs each request with the request timestamp and rejects timestamps outside a tolerance window to limit replay exposure. A host whose clock drifts past that window fails every signed call with RequestTimeTooSkewed — the signature is valid, only the clock is wrong.

Fix

Restore NTP synchronization on the host (chronyd or systemd-timesyncd with reachable servers), correct the clock, take timestamps at request time rather than caching them, and pin the SDK credential scope to the correct region and service.

Risk note

Manually setting the clock forward can break TLS validation and log ordering. Sync with NTP, then verify, rather than hand-setting.

Verify the fix

After resync, signed calls succeed immediately; monitoring reports clock offset within tolerance on all hosts; a deliberate pause-resume test recovers without errors.

Symptom walkthrough

The error text states the whole mechanism:

RequestTimeTooSkewed: The difference between the request time
and the server time is too large.

The signature itself is fine — the clock is not. Measure the drift against the authority directly:

date -u
curl -sI https://sts.amazonaws.com | grep -i ^date
# local:  Thu, 22 Oct 2026 08:41:03 GMT
# aws:    Thu, 22 Oct 2026 09:02:11 GMT   <- 21 minutes ahead

The single-host pattern is the fingerprint: identical code, same credentials, one host failing — the credential is not the variable, the clock is. The pause-resume history (VM snapshot restore, live migration) explains jumps that NTP would never produce gradually.

Fix in practice

Restore and keep synchronization, then verify against the authority:

# enable and start NTP (chronyd or systemd-timesyncd):
timedatectl set-ntp true
timedatectl status                 # "System clock synchronized: yes"

# measure the drift against AWS directly:
date -u
curl -sI https://sts.amazonaws.com | grep -i ^date

In containers, confirm the host clock is the sync source — a paused VM or a live-migrated pod can jump minutes on resume. Take request timestamps at call time (Date.now() per request, never a value cached at module load), pin the SDK region and service in the credential scope, and add clock-offset monitoring per host so the next drift page you before the 403s do.