rr Deterministic Debugger
rr provides deterministic record-replay debugging with full reverse execution capabilities.
Core Workflow
- Record:
rr record <program> [args]
- Replay:
rr replay (enters gdb interface with reverse execution)
Reverse Execution Commands
All standard gdb commands work, plus reverse variants:
reverse-next / rn: Step back over function calls
reverse-step / rs: Step back into functions
reverse-continue / rc: Continue backward to previous breakpoint
reverse-stepi / rsi: Step back one instruction
reverse-nexti / rni: Step back over one instruction
Crash Trace Extraction
Regular Crashes
After rr record <crashing-program>:
rr replay
# In gdb:
reverse-next 100 # Go back 100 steps (adjust N as needed)
# Now step forward to see execution leading to crash:
next
next
...
ASAN Crashes
After rr record <asan-program>:
rr replay
# In gdb:
bt # View stack trace
up # Issue "up" commands until last app frame (before ASAN runtime)
break *$pc # Set breakpoint at that location
reverse-continue # Go back to last app instruction before ASAN
# Now step forward to see execution leading to fault:
next
next
...
Inspecting Variables and Memory
Standard gdb commands work at any point:
print <var>: Print variable value
print *<ptr>: Dereference pointer
x/<format> <address>: Examine memory
x/10xb <addr>: 10 bytes in hex
x/s <addr>: String at address
info locals: Show local variables
info args: Show function arguments
Source vs Assembly View
list: Show source code around current location
disassemble: Show assembly around current location
layout src: TUI source view
layout asm: TUI assembly view
set disassemble-next-line on: Show assembly with each step
Automation Script
Use scripts/crash_trace.py to automatically extract execution trace before crash.