← Debugging Guides

阅读中文版本

Node.js: child process killed with SIGKILL

Diagnose Node child processes killed with SIGKILL by distinguishing OOM kills, timeouts, and signal handling gaps in spawned services.

intermediatedebuggingnodeprocessessignals

Symptoms

  • child process exited with signal SIGKILL
  • A spawned worker dies without running its cleanup handler
  • kill(pid, "SIGTERM") has no effect and SIGKILL follows

Possible causes

  • The kernel OOM killer targeted the child because it exceeded memory
  • A container or supervisor sent SIGKILL after a timeout
  • The child ignored SIGTERM, forcing SIGKILL to terminate it
  • A deployment or CI timed out and force-killed the process tree

Diagnosis steps

Log the child exit code and signal

child.on('exit', (code, signal) => console.log({ code, signal }))

Interpretation

signal === "SIGKILL" with code null means it was killed, not that it exited on its own.

Check dmesg for OOM kills

dmesg | grep -i "killed process"

Interpretation

An OOM entry naming the process proves the kernel killed it for memory, not a timeout.

Review container or supervisor timeouts

Interpretation

A stop timeout shorter than the graceful shutdown window leads to SIGKILL after SIGTERM.

Root cause

SIGKILL is not catchable, so the process ends immediately with no cleanup. Something external decided to kill it: the kernel OOM killer for memory pressure, a supervisor or container runtime after a grace period elapsed, or an operator. Because SIGKILL cannot be handled, the only remedies are preventing the kill or handling SIGTERM fast enough to exit cleanly beforehand.

Fix

Handle SIGTERM promptly so graceful shutdown finishes within the grace window, raise memory limits or fix the leak that triggers the OOM killer, and align container stop-grace-period with the shutdown duration. Detect SIGKILL after the fact through exit signals and dmesg, since it cannot be trapped, and log enough before exit to explain the failure.

Verify the fix

Send SIGTERM to the parent and observe the child; it exits cleanly with a code rather than a signal, and dmesg shows no new OOM kill, confirming graceful shutdown now completes in time.

Symptoms

A spawned child terminates with signal SIGKILL and no cleanup runs. The parent sees code null and signal SIGKILL in the exit event, which means the process was killed rather than exiting.

Diagnosis

Observe the exit event and check the kernel log:

const child = spawn('worker');
child.on('exit', (code, signal) => console.log({ code, signal }));
process.on('SIGTERM', () => { /* graceful shutdown */ });
dmesg | grep -i 'killed process'

An OOM entry identifies a memory kill; its absence points to a timeout or manual kill.

Fix

Exit fast on SIGTERM and align the grace window:

process.on('SIGTERM', async () => {
  server.close();
  await drain();
  process.exit(0);
});

Set the container stop timeout longer than the drain time so SIGKILL is not needed.

Verification

Send SIGTERM and watch the child exit with a code instead of a signal; dmesg shows no new OOM kill, confirming the shutdown completes within the grace period and SIGKILL is no longer required.