Linux: Segmentation fault (core dumped) debugging

Debug Linux segmentation faults with core dumps, gdb, and ulimit settings to locate the faulty instruction and memory access.

advanceddebugginglinuxnativecrash

Symptoms

Segmentation fault (core dumped)
A native program crashes intermittently
The process exits with signal 11 (SIGSEGV)

Possible causes

A null or wild pointer dereference
A buffer overflow writing past allocated memory
Use-after-free or double-free of heap memory
Stack overflow from unbounded recursion in a native program

Diagnosis steps

Enable core dumps and locate the file

ulimit -c unlimited; cat /proc/sys/kernel/core_pattern

Interpretation

The pattern shows where cores are written so you can load them in gdb.

Load the core and get a backtrace

gdb ./program core -ex bt -ex quit

Interpretation

The backtrace shows the exact function and line of the faulting access.

Run under sanitizers for earlier detection

compile with -fsanitize=address,undefined

Interpretation

ASan catches use-after-free and overflows closer to the first bad access.

Root cause

SIGSEGV means the CPU accessed memory the process does not own: a null or invalid pointer, an out-of-bounds write, or use of freed memory. The kernel sends the signal and, if configured, writes a core file capturing the process memory and register state for post-mortem analysis.

Fix

Reproduce under a debugger or with AddressSanitizer to find the first bad access, then fix the pointer arithmetic, bounds, or lifetime. Validate pointers before dereference, use safe string and buffer functions, and free memory exactly once. Raise stack size for deep recursion or convert it to iteration.

Verify the fix

Run the fixed program under load and under ASan; it exits cleanly with no SIGSEGV, and the previously failing input completes with correct output.

Symptoms

The program dies with Segmentation fault (core dumped) and exits on signal 11. It may be deterministic for particular input or intermittent under concurrency.

Diagnosis

Enable cores and analyse the backtrace:

ulimit -c unlimited
./program bad-input
gdb ./program core -ex bt -ex 'info registers' -ex quit

Rebuild with sanitizers to catch the first invalid access:

gcc -g -fsanitize=address,undefined -o program main.c

Fix

Correct the pointer, bound, or lifetime that the backtrace points at, then guard dereferences and prefer sized copy functions over unbounded ones.

Verification

Run under ASan and with the original input; ASan reports no errors and the program exits 0, confirming the invalid memory access is fixed.

Cores Are Only Useful If You Keep Them

A core dump is the most direct window into a crash, but only if the system is configured to write one and you keep the binary that produced it. The kernel core_pattern decides the path, and many distributions pipe cores to a handler or disable them entirely. Cores must be read with the exact executable and matching debug symbols, because a rebuilt binary changes addresses and makes the backtrace meaningless. For production, enable cores deliberately, version the binaries alongside them, and consider systemd-coredump or a crash reporter so the evidence survives the container that died.