How Debuggers Work: Breakpoints, Single-Stepping and Debug Symbols

A debugger feels like a special power. You set a breakpoint, run the program, and it stops exactly where you asked, showing the values of your variables. Behind that experience is a small set of operating system and processor features, combined with a table that maps machine addresses back to source lines. Understanding these pieces makes debugging faster, explains why some breakpoints fail, and shows what a debugger can and cannot see. This post walks through the mechanisms on a typical Unix-like system and on x86-64 processors.

Debuggers are controllers

A debugger is a separate process that controls another process, called the target. The operating system provides the link. On Linux, the classic interface is the ptrace system call, which lets one process observe and modify another. With it, the debugger can stop the target, read and write its memory and registers, and resume it. macOS and Windows offer equivalent interfaces, such as Mach exceptions and the Windows debugging API. The principle is the same everywhere: the operating system gives the debugger control that the target cannot refuse.

Because the debugger is a different process, the target does not need to be recompiled to be inspected, although debug information makes the experience far better. Some debuggers attach to a process that is already running, while others launch it under their control from the start.

Software breakpoints

The most common breakpoint is a software breakpoint. On x86 processors, the instruction INT3 is a one-byte opcode, hexadecimal 0xCC, that raises a trap when executed. To set a breakpoint, the debugger saves the original byte at the target address and writes 0xCC in its place. When the program reaches that address, the processor traps into the operating system, which notifies the debugger. The debugger sees that the program stopped at a breakpoint, restores the original byte, and sets the program counter back so the real instruction can run when resumed.

To keep going past the breakpoint, the debugger must put the original instruction back, single-step over it, and then re-insert the trap. This sequence explains why a breakpoint on code that is modified at runtime can behave strangely.

Hardware breakpoints and watchpoints

Processors also have debug registers. On x86-64 there are four address registers, each of which can trigger on execution, on writes or on reads and writes, for a chosen length. Hardware breakpoints do not change the code bytes, so they work on read-only memory and avoid side effects. They are limited in number, which is why a debugger may refuse a fifth hardware breakpoint.

Watchpoints use the same mechanism to stop when a variable changes. This is one of the most useful tools for finding memory corruption, because it reports the exact instruction that wrote the bad value. Without it, a bug may show up far away from its cause.

Single-stepping

Single-stepping runs one instruction and then stops. On x86 processors, the trap flag in the flags register causes a debug exception after each instruction. Debuggers use this for "step into" and "step over" commands. The difference is what the debugger does after a call instruction. Step into stops at the first instruction of the called function, while step over places a temporary breakpoint at the return address and runs until it is reached.

Single-stepping is slow, because each step involves a round trip through the kernel. Debuggers avoid it where possible, for example by using temporary breakpoints for step over and by stepping through whole source lines rather than individual instructions.

Mapping machines back to source

Machine code has no variable names and no line numbers. The compiler records the mapping in debug information. The most common format on Unix-like systems is DWARF, which is stored in sections of the executable or in a separate file. A line table maps address ranges to source files and line numbers. A symbol table maps function names to addresses. Type descriptions tell the debugger how to interpret bytes as integers, pointers or structures.

Variable locations are more complex. DWARF describes where a variable lives at each point in the program, which may be a register, a stack offset, or nothing at all after optimization. This is why debugging optimized code often shows "optimized out" for some variables. The compiler discarded or moved the value, and the debug information honestly reports that.

Stripped binaries have no symbols, so the debugger can show only addresses. Keeping a separate symbol file lets you debug a release binary later without shipping debug information in the product.

Stack frames

When the program stops, the debugger needs to show the call stack. On many platforms, it walks frames using unwind information, which describes how to recover the caller's registers at each address. Without that information, the stack trace may be incomplete. Frame pointers, when kept, make walking simpler but are not required on modern compilers that emit unwind tables.

The stack trace is a snapshot of how the program got here. Reading it from the bottom up shows the sequence of calls, and reading the top frame shows where the stop happened. Combined with the local variables of each frame, it often answers the question "how did we arrive in this state."

A comparison of techniques

The table below compares the main ways to observe a running program. Each has a place, and experienced developers combine them.

Technique	What it does	Typical cost	Best for
Software breakpoint	Replaces an instruction with a trap	Low, but modifies code	Stopping at a known line
Hardware breakpoint	Uses CPU debug registers	Very low, limited count	Read-only code and quick checks
Watchpoint	Stops when memory is written	Low to medium	Finding corruption
Single-step	Stops after every instruction	High	Tracing tricky control flow
Logging	Prints chosen values	Medium, changes timing	Long runs and production

Practical habits

Build with debug information while debugging, and check that the binary you run matches the source you are reading. A common confusion is that the debugger shows lines from an old build because the symbols belong to a different file. Prefer conditional breakpoints for loops, so the debugger stops only on the iteration that matters. Use watchpoints when a value changes unexpectedly, instead of guessing where the write happens.

Remember that a debugger changes timing. A race condition may disappear when you step through it, because the threads no longer overlap in the same way. In those cases, logging with timestamps or tracing tools is often more informative than stopping the program.

Key takeaways

Debuggers rely on operating system interfaces that let one process control another. Software breakpoints replace an instruction with a trap, and hardware breakpoints and watchpoints use CPU debug registers to stop without changing code. Single-stepping relies on the processor's trap behavior and is slow but precise. Debug information in formats such as DWARF maps addresses to lines, functions and variables. Understanding these layers helps you choose the right tool, read stack traces with confidence, and recognize when the debugger itself is changing what you observe.