Valgrind — Memory Debugging:… | OneNoughtOne

0/107

Operating SystemsMemory Debugging

Memory Debugging: Detecting and Resolving Memory Errors

LevelAdvanced

Duration120 mins

TopicMemory Debugging

4 / 5

Valgrind

The Memory Detective

In the early 2000s, Julian Seward faced a common frustration: tracking down memory bugs in complex C programs was tedious, error-prone, and often incomplete. Commercial tools existed but were expensive and proprietary. In 2002, he released the first version of Valgrind —a tool that would revolutionize how developers find and fix memory errors. Valgrind's approach was novel: instead of modifying the program's source or using special compiler instrumentation, it would run the program on a synthetic CPU, intercepting every memory access and checking it against a shadow memory map that tracked the validity of each byte. This dynamic binary instrumentation allowed Valgrind to detect errors that static analysis and simple testing would miss. Today, Valgrind is the de facto standard for memory debugging on Linux and macOS. It has detected countless bugs, prevented security vulnerabilities, and saved developers millions of hours of debugging time. Understanding Valgrind is essential for any systems programmer writing C or C++.

What You Will Learn

By the end of this page, you will understand: (1) Valgrind's architecture and how dynamic binary instrumentation works, (2) Memcheck—the primary memory error detection tool, (3) How to interpret Valgrind's error reports, (4) Detecting memory leaks with precise source locations, (5) Advanced usage patterns for complex debugging scenarios, and (6) Integrating Valgrind into development workflows.

Understanding Valgrind's Architecture

Valgrind is not a single tool but a framework for building dynamic analysis tools . Its architecture enables deep inspection of program behavior at runtime with minimal source code changes. Core Architecture:

valgrind_architecture.txt
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
Valgrind Architecture Overview
 
┌────────────────────────────────────────────────────────────────┐
│                    User Program (Binary)                        │
│         (compiled normally - no special instrumentation)        │
└────────────────────────────────────────────────────────────────┘
                              │
                              ▼
┌────────────────────────────────────────────────────────────────┐
│                    Valgrind Core                                │
│  ┌─────────────────────────────────────────────────────────┐   │
│  │              Dynamic Binary Translation                  │   │
│  │   • Disassembles native code to Valgrind IR (VEX IR)    │   │
│  │   • Allows tools to instrument at IR level              │   │
│  │   • Recompiles IR to native code for execution          │   │
│  └─────────────────────────────────────────────────────────┘   │
│                              │                                  │
│                              ▼                                  │
│  ┌─────────────────────────────────────────────────────────┐   │
│  │                     Tool Plugin                          │   │
│  │   • Memcheck (memory errors)                             │   │
│  │   • Helgrind (thread errors)                             │   │
│  │   • Cachegrind (cache profiling)                         │   │
│  │   • Callgrind (call graph profiling)                     │   │
│  │   • Massif (heap profiling)                              │   │
│  │   • DRD (thread debugging)                               │   │
│  │   • DHAT (dynamic heap analysis)                         │   │
│  └─────────────────────────────────────────────────────────┘   │
│                              │                                  │
│                              ▼                                  │
│  ┌─────────────────────────────────────────────────────────┐   │
│  │                   Synthetic CPU                          │   │
│  │   • Executes instrumented code                           │   │
│  │   • Manages shadow memory                                │   │
│  │   • Intercepts system calls                              │   │
│  └─────────────────────────────────────────────────────────┘   │
└────────────────────────────────────────────────────────────────┘
                              │
                              ▼
┌────────────────────────────────────────────────────────────────┐
│                   Host Operating System                         │
│            (Linux, macOS, FreeBSD, Android)                     │
└────────────────────────────────────────────────────────────────┘

Dynamic Binary Instrumentation (DBI): Valgrind's power comes from Dynamic Binary Instrumentation. Instead of running your program directly on the CPU: Disassembly — Valgrind reads the executable, disassembling machine code into an intermediate representation (VEX IR) Instrumentation — The selected tool (e.g., Memcheck) adds instrumentation code to the IR. For Memcheck, this means adding checks before every memory access. Recompilation — The instrumented IR is compiled back to native machine code and executed Execution — The instrumented code runs, with checks executing alongside the original logic This approach works on any compiled binary—no recompilation, no special flags, no source access required.

Valgrind's Built-in Tools
Tool	Purpose	Common Use Cases
Memcheck	Memory error detection	Finding leaks, overflows, UAF, uninitialized reads
Helgrind	Thread error detection	Finding data races, lock order violations
DRD	Data race detection	Alternative to Helgrind with different tradeoffs
Cachegrind	Cache profiling	Analyzing cache miss rates, I1/D1/LL cache behavior
Callgrind	Call graph profiling	Function-level performance analysis, pairs with KCachegrind
Massif	Heap profiler	Understanding memory consumption over time
DHAT	Dynamic heap analysis	Finding inefficient heap usage patterns

The Performance Cost

Valgrind's thoroughness comes at a cost: programs run 10-50x slower under Memcheck. This is because every memory access is checked against shadow memory. For most debugging scenarios this is acceptable—you run specific test cases, not the full application. For continuous integration, consider using Valgrind on targeted test suites.

Memcheck: The Primary Memory Debugging Tool

Memcheck is Valgrind's flagship tool—the reason most developers use Valgrind. It detects a comprehensive range of memory errors by maintaining shadow memory that tracks the state of every byte in your program's address space. What Memcheck Tracks:

Shadow Memory State

•Addressability (A-bits) — Is each byte legally accessible? Memcheck knows which memory is allocated, freed, or outside valid regions. Accessing non-addressable memory is an error.
•Definedness (V-bits) — Is each byte's value defined? Memory from malloc() is undefined until written. Reading undefined memory is an error—even if the address is valid.
•Heap Block Origins — For each allocated block: where it was allocated, its size, and whether it's been freed. Essential for reporting leaks and UAF with context.
•Free Status — Whether memory has been freed. Accessing freed memory is immediately flagged as use-after-free.

Errors Detected by Memcheck:

Memcheck Error Detection Categories
Error Type	Description	Example	Severity
Invalid Read	Reading from unallocated/freed memory	Accessing `buffer[-1]` or freed pointer	Critical
Invalid Write	Writing to unallocated/freed memory	Buffer overflow, UAF write	Critical
Invalid Free	Freeing invalid pointer	Double free, freeing stack memory	Critical
Mismatched Free	Wrong deallocation function	malloc/delete mismatch	High
Uninitialized Value	Using undefined memory	Reading malloc'd memory before writing	Medium-High
Uninitialized Condition	Branch on undefined value	if(uninitialized_var)	Medium-High
Syscall Param Uninitialized	Passing undefined to syscall	write() with uninitialized buffer	Medium
Memory Leak	Unreachable heap memory at exit	Lost malloc without free	Medium
Overlap in memcpy	Source and dest overlap	memcpy with overlapping regions	Medium

Shadow Memory Implementation: Memcheck maintains a compressed shadow memory map. For every 8 bytes of your program's memory, Memcheck tracks: 8 A-bits (1 per byte): Is byte addressable? 8 V-bits (1 per bit of data): Is bit defined? This means Memcheck uses roughly 25% additional memory for shadow data, plus the overhead of its synthetic CPU.

Why Uninitialized Checks Matter

Reading uninitialized memory might 'work' because the bytes happen to contain benign values. But the bug is latent—different runs or different systems may have different garbage, causing intermittent failures. Memcheck catches these before they manifest as production bugs.

Running Valgrind: Complete Usage Guide

Using Valgrind effectively requires understanding its invocation options and how to compile programs for optimal debugging. Compiling for Valgrind:

compilation_flags.sh
Bash
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
# Optimal compilation for Valgrind debugging
 
# Essential: Include debug symbols (-g)
# Valgrind works without them, but error reports won't show source lines
gcc -g program.c -o program
 
# Better: Debug symbols + no optimization
# Optimization can confuse line number mapping
gcc -g -O0 program.c -o program
 
# Best: Debug symbols, no optimization, frame pointers
# Frame pointers improve stack traces
gcc -g -O0 -fno-omit-frame-pointer program.c -o program
 
# For production debugging (when you must use optimization):
# Debug info with optimization - line numbers may be approximate
gcc -g -O2 program.c -o program
 
# Important: Valgrind understands standard library debug symbols
# On Ubuntu/Debian, install: libc6-dbg
# This gives you meaningful stack traces through glibc functions

Basic Valgrind Invocation:

valgrind_basics.sh
Bash
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
# Basic invocation (uses Memcheck by default)
valgrind ./program arg1 arg2
 
# Explicit tool selection
valgrind --tool=memcheck ./program
 
# Enable leak checking (essential for finding memory leaks)
valgrind --leak-check=full ./program
 
# Full recommended options for debugging
valgrind \
    --leak-check=full \
    --show-leak-kinds=all \
    --track-origins=yes \
    --verbose \
    ./program
 
# Explanation of options:
# --leak-check=full    : Detailed leak report with stack traces
# --show-leak-kinds=all: Report definite, indirect, possible, reachable leaks
# --track-origins=yes  : Track where uninitialized values came from
# --verbose            : More detail about Valgrind operation
 
# Writing output to file (preserves terminal for program output)
valgrind --log-file=valgrind.log --leak-check=full ./program
 
# Generate suppressions for known/acceptable issues
valgrind --gen-suppressions=all --leak-check=full ./program 2>&1 | tee supp.txt
 
# Apply suppressions file
valgrind --suppressions=myproject.supp --leak-check=full ./program

Key Command-Line Options:

Essential Valgrind/Memcheck Options
Option	Values	Purpose
`--leak-check`	no/summary/full	Level of leak checking detail
`--show-leak-kinds`	definite/indirect/possible/reachable/all	Which leak types to report
`--track-origins`	yes/no	Track source of undefined values (slower)
`--undef-value-errors`	yes/no	Report undefined value errors
`--log-file=<file>`	path	Write output to file instead of stderr
`--suppressions=<file>`	path	Load suppressions from file
`--gen-suppressions`	no/yes/all	Generate suppression entries for errors
`--error-exitcode=<N>`	number	Exit with N if errors found (for CI)
`--trace-children`	yes/no	Also trace child processes
`--num-callers=<N>`	number	Stack trace depth (default 12)

The track-origins Trade-off

The --track-origins=yes option significantly helps debugging uninitialized value errors by showing where the undefined value originated. However, it roughly doubles Valgrind's slowdown. Use it when debugging specific uninitialized issues; omit it for routine leak checking.

Interpreting Valgrind's Error Reports

Valgrind's reports can seem overwhelming at first. Understanding their structure helps you quickly identify and fix issues. Anatomy of an Error Report:

error_report_anatomy.txt
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
==12345== Invalid read of size 4
==12345==    at 0x4005E7: main (example.c:15)
==12345==    by 0x7FFFFFFF: (below main)
==12345==  Address 0x5204050 is 0 bytes after a block of size 40 alloc'd
==12345==    at 0x4C2AB80: malloc (vg_replace_malloc.c:299)
==12345==    by 0x4005C7: main (example.c:10)
 
Breakdown:
─────────────────────────────────────────────────────────────────
 
==12345==                    Process ID (same for all lines in this run)
 
Invalid read of size 4       Error type and size
                             - Read/Write: direction of access
                             - Size: 1/2/4/8 bytes typically
 
at 0x4005E7: main (example.c:15)    WHERE the error occurred
                                     - Address in executable
                                     - Function name
                                     - Source file and line number
 
by 0x7FFFFFFF: (below main)  Stack trace of calling functions
 
Address 0x5204050 is...      WHAT the problematic address is
                             - Relationship to known blocks
                             - How it was allocated
 
0 bytes after a block        Position relative to valid memory
                             - "0 bytes after" = immediately past end
                             - "4 bytes inside" = within valid block (shouldn't error)
                             - "16 bytes before" = underflow
 
of size 40 alloc'd           The valid block's size
 
at 0x4C2AB80: malloc...      WHERE the block was allocated
                             - Critical for understanding the bug

Common Error Patterns and Fixes:

error_examples.c
C
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
// ERROR: Invalid read of size 4
// Address 0x... is 0 bytes after a block of size 40
int* arr = malloc(10 * sizeof(int));  // 40 bytes
printf("%d", arr[10]);                 // Off-by-one! Index 10 is byte 40-43
// FIX: Use arr[9] or allocate 11 elements
 
// ERROR: Invalid write of size 1  
// Address 0x... is 0 bytes after a block of size 10
char* str = malloc(10);
strcpy(str, "0123456789");  // 11 chars including null terminator!
// FIX: malloc(11) or use strncpy
 
// ERROR: Use of uninitialized value of size 4
int x;
if (x > 0) { ... }  // x was never assigned
// FIX: int x = 0; or ensure assignment before use
 
// ERROR: Invalid free()
// Address 0x... is 0 bytes inside a block of size 100 free'd
char* p = malloc(100);
free(p);
free(p);  // Double free!
// FIX: Set p = NULL after free; check before freeing
 
// ERROR: Mismatched free() / delete / delete[]
// at 0x...: operator delete(void*)
// Address 0x... is 0 bytes inside a block of size 40 alloc'd
// at 0x...: operator new[](unsigned long)
int* arr = new int[10];
delete arr;    // Should be delete[]
// FIX: delete[] arr;
 
// ERROR: Conditional jump depends on uninitialised value
void process(int* data, int count) {
    int sum;  // Uninitialized!
    for (int i = 0; sum < 100 && i < count; i++) {  // Using sum before init
        sum += data[i];
    }
}
// FIX: int sum = 0;

Reading the Address Context

The 'Address is X bytes after/before a block' line is crucial. 'After' suggests buffer overflow. 'Before' suggests underflow. 'Inside ... free'd' indicates use-after-free. 'Not stack'd, malloc'd or (recently) free'd' suggests stack corruption or corrupted pointer.

Comprehensive Memory Leak Analysis

Valgrind's leak detection is one of its most valuable features. At program exit, it analyzes remaining heap allocations and categorizes them by reachability. Leak Classification:

Valgrind Leak Categories
Category	Meaning	Action Required	Example
Definitely Lost	No pointer to block exists anywhere	Must fix - true leak	Pointer overwritten without free
Indirectly Lost	Only reachable via definitely lost block	Will be fixed when parent fixed	Linked list where head is lost
Possibly Lost	Only interior pointer exists	Investigate - may be intentional	Pointer arithmetic on allocated block
Still Reachable	Pointer exists at exit (not freed)	Usually acceptable	Global cache freed at exit by OS

leak_report_example.txt
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
# Run with full leak checking
$ valgrind --leak-check=full --show-leak-kinds=all ./program
 
==12345== HEAP SUMMARY:
==12345==     in use at exit: 1,024 bytes in 4 blocks
==12345==   total heap usage: 100 allocs, 96 frees, 10,240 bytes allocated
 
==12345== 256 bytes in 1 blocks are definitely lost in loss record 3 of 4
==12345==    at 0x4C2AB80: malloc (vg_replace_malloc.c:299)
==12345==    by 0x400547: create_buffer (mylib.c:42)
==12345==    by 0x400623: process_data (main.c:87)
==12345==    by 0x4006B2: main (main.c:112)
 
==12345== 512 bytes in 2 blocks are indirectly lost in loss record 4 of 4
==12345==    at 0x4C2AB80: malloc (vg_replace_malloc.c:299)
==12345==    by 0x400589: add_node (mylib.c:56)
==12345==    by 0x400547: create_buffer (mylib.c:48)
==12345==    by 0x400623: process_data (main.c:87)
==12345==    by 0x4006B2: main (main.c:112)
 
==12345== 256 bytes in 1 blocks are still reachable in loss record 2 of 4
==12345==    at 0x4C2AB80: malloc (vg_replace_malloc.c:299)
==12345==    by 0x4007A1: init_global_cache (cache.c:15)
==12345==    by 0x4006A2: main (main.c:105)
 
==12345== LEAK SUMMARY:
==12345==    definitely lost: 256 bytes in 1 blocks
==12345==    indirectly lost: 512 bytes in 2 blocks
==12345==      possibly lost: 0 bytes in 0 blocks
==12345==    still reachable: 256 bytes in 1 blocks
==12345==         suppressed: 0 bytes in 0 blocks

Interpreting the Leak Report: In the example above: Definitely lost (256 bytes) — A buffer allocated in create_buffer() is truly leaked. The stack trace shows it was called from process_data() . This is the primary bug to fix. Indirectly lost (512 bytes) — These are nodes added to the leaked buffer. Once you fix the definitely lost block, these will also be fixed (assuming proper cleanup of the buffer's contents). Still reachable (256 bytes) — A global cache allocated at startup. This isn't freed before exit, but it's intentional—the OS reclaims it. This is typically acceptable for global resources. Prioritization: Fix definitely lost first, then indirectly lost resolves automatically. Still reachable is usually informational unless you're looking for strict leak-free code.

Possibly Lost Explained

Possibly lost occurs when the only pointer to a block points to its interior, not its start. This can be legitimate (pointer arithmetic for efficiency) or a bug (original pointer was corrupted). Investigate each case. Common legitimate patterns: pointing to the 'data' portion of a struct starting with a header, or custom allocators.

Advanced Valgrind Techniques

Beyond basic usage, Valgrind offers powerful features for complex debugging scenarios. Suppressions: Managing Known Issues Libraries may have benign issues you can't fix. Suppressions let you ignore specific errors:

suppression_file.supp

Text

# myproject.supp - Suppression file example
 
# Suppress a specific leak in third-party library
{
   thirdparty_lib_leak
   Memcheck:Leak
   match-leak-kinds: definite
   fun:malloc
   fun:third_party_init
   fun:main
}
 
# Suppress all leaks in a specific library  
{
   libfoo_all_leaks
   Memcheck:Leak
   ...
   obj:*/libfoo.so*
}
 
# Suppress uninitialized value in system library
{
   glibc_uninitialized
   Memcheck:Cond
   fun:*
   obj:/lib/x86_64-linux-gnu/libc-2.*.so
}
 
# Pattern matching:
# fun:  - function name (can use wildcards: fun:*alloc*)
# obj:  - shared library path
# src:  - source file
# ...   - match any number of frames
 
# Generate suppressions automatically:
# valgrind --gen-suppressions=all ./program 2>&1 | grep -A20 "{"

Client Requests: Valgrind-Aware Code Your program can communicate with Valgrind using client requests:

client_requests.c
C
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
#include <valgrind/memcheck.h>
#include <valgrind/valgrind.h>
 
void custom_allocator_example() {
    // For custom memory pools: tell Valgrind about allocations
    
    char* pool = malloc(1024);
    
    // Mark pool as not accessible initially
    VALGRIND_MAKE_MEM_NOACCESS(pool, 1024);
    
    // When allocating from pool, mark as undefined (allocated but unwritten)
    char* obj = pool;
    VALGRIND_MAKE_MEM_UNDEFINED(obj, 64);
    
    // When freeing back to pool, mark as inaccessible
    VALGRIND_MAKE_MEM_NOACCESS(obj, 64);
    
    // Custom allocation tracking
    VALGRIND_MALLOCLIKE_BLOCK(obj, 64, 0, 0);
    // ... use obj ...
    VALGRIND_FREELIKE_BLOCK(obj, 0);
}
 
void conditional_debugging() {
    // Check if running under Valgrind
    if (RUNNING_ON_VALGRIND) {
        printf("Running under Valgrind - enabling extra checks
");
        enable_debug_mode();
    }
    
    // Request garbage collection (for Valgrind's internal state)
    // Useful before checking for leaks mid-execution
    VALGRIND_DO_ADDED_LEAK_CHECK;
    
    // Count errors so far
    int errors = VALGRIND_COUNT_ERRORS;
    printf("Valgrind errors so far: %d
", errors);
}
 
// Note: These macros are no-ops when not running under Valgrind
// They add zero overhead to production builds

Debugging Under Valgrind with GDB:

valgrind_gdb.sh
Bash
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
# Run Valgrind with GDB server
valgrind --vgdb=yes --vgdb-error=0 ./program
 
# In another terminal, connect GDB
gdb ./program
(gdb) target remote | vgdb
 
# Now you can:
# - Set breakpoints
# - Step through execution  
# - Inspect memory at each Valgrind error
# - Use GDB commands alongside Valgrind's analysis
 
# Useful GDB commands when connected to Valgrind:
(gdb) monitor leak_check full reachable any
(gdb) monitor block_list 
(gdb) monitor who_points_at <address>
 
# The --vgdb-error=0 option stops at first error
# Set to higher number to stop at Nth error

Memory Pool Debugging

If you use custom allocators or memory pools, use VALGRIND_MALLOCLIKE_BLOCK and VALGRIND_FREELIKE_BLOCK macros. Without these, Valgrind can't track allocations from your pool, and you'll miss leaks/errors within it.

Integrating Valgrind into Development Workflows

Valgrind is most effective when integrated into regular development practices, not just used for emergency debugging. CI/CD Integration:

ci_integration.yaml
YAML
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
# GitHub Actions example for Valgrind integration
 
name: Memory Safety Check
 
on: [push, pull_request]
 
jobs:
  valgrind:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
      
      - name: Install Valgrind
        run: sudo apt-get install -y valgrind libc6-dbg
      
      - name: Build with Debug Symbols
        run: |
          mkdir build && cd build
          cmake -DCMAKE_BUILD_TYPE=Debug ..
          make
      
      - name: Run Tests Under Valgrind
        run: |
          cd build
          valgrind \
            --leak-check=full \
            --show-leak-kinds=definite,indirect \
            --error-exitcode=1 \
            --errors-for-leak-kinds=definite \
            --suppressions=../valgrind.supp \
            ./test_suite
      
      - name: Upload Valgrind Log on Failure
        if: failure()
        uses: actions/upload-artifact@v3
        with:
          name: valgrind-log
          path: valgrind.log

Development Workflow Best Practices:

Valgrind Workflow Recommendations

•Run Early, Run Often — Don't wait until bugs manifest. Run Valgrind on new code before merging. Catching issues early is much cheaper than debugging production problems.
•Maintain a Suppression File — Create and version-control a project suppression file for known third-party issues. Review suppressions periodically to ensure they're still needed.
•Use in Pre-Commit Hooks — For critical code paths, run targeted Valgrind tests before allowing commits. Focus on recently changed modules.
•Combine with Unit Tests — Your unit tests are ideal Valgrind targets. They exercise specific functionality and are designed to be fast. Run valgrind ./test_module_x as part of test routine.
•Profile Before Optimizing — Use Callgrind/Cachegrind to profile before making performance changes. 'Optimization' without profiling often makes things worse.
•Document False Positives — When adding suppressions, document why. This helps future developers understand whether a suppression is still valid.

Valgrind vs. AddressSanitizer Trade-offs:

Valgrind vs. AddressSanitizer Comparison
Aspect	Valgrind Memcheck	AddressSanitizer
Slowdown	10-50x	~2x
Preparation	No recompilation needed	Requires recompilation with flags
Memory Overhead	~2x	~2-3x
Leak Detection	Comprehensive at exit	Optional, less detailed
Uninitialized Memory	Excellent detection	Requires MemorySanitizer (separate)
Binary Analysis	Works on any binary	Requires source + compiler support
Platform Support	Linux, macOS, FreeBSD	Linux, macOS (better), Windows (limited)
CI Suitability	Slower, best for targeted tests	Fast enough for full test suites

Complementary Tools

Use both Valgrind and ASan in your workflow. ASan for fast, comprehensive coverage in CI (every build). Valgrind for deep analysis when ASan finds issues, for binary-only debugging, and for detailed leak analysis. They catch different things and complement each other well.

Operating SystemsMemory Debugging

Memory Debugging: Detecting and Resolving Memory Errors

LevelAdvanced

Duration120 mins

TopicMemory Debugging

4 / 5

Valgrind

The Memory Detective

In the early 2000s, Julian Seward faced a common frustration: tracking down memory bugs in complex C programs was tedious, error-prone, and often incomplete. Commercial tools existed but were expensive and proprietary. In 2002, he released the first version of Valgrind —a tool that would revolutionize how developers find and fix memory errors. Valgrind's approach was novel: instead of modifying the program's source or using special compiler instrumentation, it would run the program on a synthetic CPU, intercepting every memory access and checking it against a shadow memory map that tracked the validity of each byte. This dynamic binary instrumentation allowed Valgrind to detect errors that static analysis and simple testing would miss. Today, Valgrind is the de facto standard for memory debugging on Linux and macOS. It has detected countless bugs, prevented security vulnerabilities, and saved developers millions of hours of debugging time. Understanding Valgrind is essential for any systems programmer writing C or C++.

What You Will Learn

By the end of this page, you will understand: (1) Valgrind's architecture and how dynamic binary instrumentation works, (2) Memcheck—the primary memory error detection tool, (3) How to interpret Valgrind's error reports, (4) Detecting memory leaks with precise source locations, (5) Advanced usage patterns for complex debugging scenarios, and (6) Integrating Valgrind into development workflows.

Understanding Valgrind's Architecture

Valgrind is not a single tool but a framework for building dynamic analysis tools . Its architecture enables deep inspection of program behavior at runtime with minimal source code changes. Core Architecture:

valgrind_architecture.txt
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
Valgrind Architecture Overview
 
┌────────────────────────────────────────────────────────────────┐
│                    User Program (Binary)                        │
│         (compiled normally - no special instrumentation)        │
└────────────────────────────────────────────────────────────────┘
                              │
                              ▼
┌────────────────────────────────────────────────────────────────┐
│                    Valgrind Core                                │
│  ┌─────────────────────────────────────────────────────────┐   │
│  │              Dynamic Binary Translation                  │   │
│  │   • Disassembles native code to Valgrind IR (VEX IR)    │   │
│  │   • Allows tools to instrument at IR level              │   │
│  │   • Recompiles IR to native code for execution          │   │
│  └─────────────────────────────────────────────────────────┘   │
│                              │                                  │
│                              ▼                                  │
│  ┌─────────────────────────────────────────────────────────┐   │
│  │                     Tool Plugin                          │   │
│  │   • Memcheck (memory errors)                             │   │
│  │   • Helgrind (thread errors)                             │   │
│  │   • Cachegrind (cache profiling)                         │   │
│  │   • Callgrind (call graph profiling)                     │   │
│  │   • Massif (heap profiling)                              │   │
│  │   • DRD (thread debugging)                               │   │
│  │   • DHAT (dynamic heap analysis)                         │   │
│  └─────────────────────────────────────────────────────────┘   │
│                              │                                  │
│                              ▼                                  │
│  ┌─────────────────────────────────────────────────────────┐   │
│  │                   Synthetic CPU                          │   │
│  │   • Executes instrumented code                           │   │
│  │   • Manages shadow memory                                │   │
│  │   • Intercepts system calls                              │   │
│  └─────────────────────────────────────────────────────────┘   │
└────────────────────────────────────────────────────────────────┘
                              │
                              ▼
┌────────────────────────────────────────────────────────────────┐
│                   Host Operating System                         │
│            (Linux, macOS, FreeBSD, Android)                     │
└────────────────────────────────────────────────────────────────┘

Dynamic Binary Instrumentation (DBI): Valgrind's power comes from Dynamic Binary Instrumentation. Instead of running your program directly on the CPU: Disassembly — Valgrind reads the executable, disassembling machine code into an intermediate representation (VEX IR) Instrumentation — The selected tool (e.g., Memcheck) adds instrumentation code to the IR. For Memcheck, this means adding checks before every memory access. Recompilation — The instrumented IR is compiled back to native machine code and executed Execution — The instrumented code runs, with checks executing alongside the original logic This approach works on any compiled binary—no recompilation, no special flags, no source access required.

Valgrind's Built-in Tools
Tool	Purpose	Common Use Cases
Memcheck	Memory error detection	Finding leaks, overflows, UAF, uninitialized reads
Helgrind	Thread error detection	Finding data races, lock order violations
DRD	Data race detection	Alternative to Helgrind with different tradeoffs
Cachegrind	Cache profiling	Analyzing cache miss rates, I1/D1/LL cache behavior
Callgrind	Call graph profiling	Function-level performance analysis, pairs with KCachegrind
Massif	Heap profiler	Understanding memory consumption over time
DHAT	Dynamic heap analysis	Finding inefficient heap usage patterns

The Performance Cost

Valgrind's thoroughness comes at a cost: programs run 10-50x slower under Memcheck. This is because every memory access is checked against shadow memory. For most debugging scenarios this is acceptable—you run specific test cases, not the full application. For continuous integration, consider using Valgrind on targeted test suites.

Memcheck: The Primary Memory Debugging Tool

Memcheck is Valgrind's flagship tool—the reason most developers use Valgrind. It detects a comprehensive range of memory errors by maintaining shadow memory that tracks the state of every byte in your program's address space. What Memcheck Tracks:

Shadow Memory State

•Addressability (A-bits) — Is each byte legally accessible? Memcheck knows which memory is allocated, freed, or outside valid regions. Accessing non-addressable memory is an error.
•Definedness (V-bits) — Is each byte's value defined? Memory from malloc() is undefined until written. Reading undefined memory is an error—even if the address is valid.
•Heap Block Origins — For each allocated block: where it was allocated, its size, and whether it's been freed. Essential for reporting leaks and UAF with context.
•Free Status — Whether memory has been freed. Accessing freed memory is immediately flagged as use-after-free.

Errors Detected by Memcheck:

Memcheck Error Detection Categories
Error Type	Description	Example	Severity
Invalid Read	Reading from unallocated/freed memory	Accessing `buffer[-1]` or freed pointer	Critical
Invalid Write	Writing to unallocated/freed memory	Buffer overflow, UAF write	Critical
Invalid Free	Freeing invalid pointer	Double free, freeing stack memory	Critical
Mismatched Free	Wrong deallocation function	malloc/delete mismatch	High
Uninitialized Value	Using undefined memory	Reading malloc'd memory before writing	Medium-High
Uninitialized Condition	Branch on undefined value	if(uninitialized_var)	Medium-High
Syscall Param Uninitialized	Passing undefined to syscall	write() with uninitialized buffer	Medium
Memory Leak	Unreachable heap memory at exit	Lost malloc without free	Medium
Overlap in memcpy	Source and dest overlap	memcpy with overlapping regions	Medium

Shadow Memory Implementation: Memcheck maintains a compressed shadow memory map. For every 8 bytes of your program's memory, Memcheck tracks: 8 A-bits (1 per byte): Is byte addressable? 8 V-bits (1 per bit of data): Is bit defined? This means Memcheck uses roughly 25% additional memory for shadow data, plus the overhead of its synthetic CPU.

Why Uninitialized Checks Matter

Reading uninitialized memory might 'work' because the bytes happen to contain benign values. But the bug is latent—different runs or different systems may have different garbage, causing intermittent failures. Memcheck catches these before they manifest as production bugs.

Running Valgrind: Complete Usage Guide

Using Valgrind effectively requires understanding its invocation options and how to compile programs for optimal debugging. Compiling for Valgrind:

compilation_flags.sh
Bash
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
# Optimal compilation for Valgrind debugging
 
# Essential: Include debug symbols (-g)
# Valgrind works without them, but error reports won't show source lines
gcc -g program.c -o program
 
# Better: Debug symbols + no optimization
# Optimization can confuse line number mapping
gcc -g -O0 program.c -o program
 
# Best: Debug symbols, no optimization, frame pointers
# Frame pointers improve stack traces
gcc -g -O0 -fno-omit-frame-pointer program.c -o program
 
# For production debugging (when you must use optimization):
# Debug info with optimization - line numbers may be approximate
gcc -g -O2 program.c -o program
 
# Important: Valgrind understands standard library debug symbols
# On Ubuntu/Debian, install: libc6-dbg
# This gives you meaningful stack traces through glibc functions

Basic Valgrind Invocation:

valgrind_basics.sh
Bash
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
# Basic invocation (uses Memcheck by default)
valgrind ./program arg1 arg2
 
# Explicit tool selection
valgrind --tool=memcheck ./program
 
# Enable leak checking (essential for finding memory leaks)
valgrind --leak-check=full ./program
 
# Full recommended options for debugging
valgrind \
    --leak-check=full \
    --show-leak-kinds=all \
    --track-origins=yes \
    --verbose \
    ./program
 
# Explanation of options:
# --leak-check=full    : Detailed leak report with stack traces
# --show-leak-kinds=all: Report definite, indirect, possible, reachable leaks
# --track-origins=yes  : Track where uninitialized values came from
# --verbose            : More detail about Valgrind operation
 
# Writing output to file (preserves terminal for program output)
valgrind --log-file=valgrind.log --leak-check=full ./program
 
# Generate suppressions for known/acceptable issues
valgrind --gen-suppressions=all --leak-check=full ./program 2>&1 | tee supp.txt
 
# Apply suppressions file
valgrind --suppressions=myproject.supp --leak-check=full ./program

Key Command-Line Options:

Essential Valgrind/Memcheck Options
Option	Values	Purpose
`--leak-check`	no/summary/full	Level of leak checking detail
`--show-leak-kinds`	definite/indirect/possible/reachable/all	Which leak types to report
`--track-origins`	yes/no	Track source of undefined values (slower)
`--undef-value-errors`	yes/no	Report undefined value errors
`--log-file=<file>`	path	Write output to file instead of stderr
`--suppressions=<file>`	path	Load suppressions from file
`--gen-suppressions`	no/yes/all	Generate suppression entries for errors
`--error-exitcode=<N>`	number	Exit with N if errors found (for CI)
`--trace-children`	yes/no	Also trace child processes
`--num-callers=<N>`	number	Stack trace depth (default 12)

The track-origins Trade-off

The --track-origins=yes option significantly helps debugging uninitialized value errors by showing where the undefined value originated. However, it roughly doubles Valgrind's slowdown. Use it when debugging specific uninitialized issues; omit it for routine leak checking.

Interpreting Valgrind's Error Reports

Valgrind's reports can seem overwhelming at first. Understanding their structure helps you quickly identify and fix issues. Anatomy of an Error Report:

error_report_anatomy.txt
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
==12345== Invalid read of size 4
==12345==    at 0x4005E7: main (example.c:15)
==12345==    by 0x7FFFFFFF: (below main)
==12345==  Address 0x5204050 is 0 bytes after a block of size 40 alloc'd
==12345==    at 0x4C2AB80: malloc (vg_replace_malloc.c:299)
==12345==    by 0x4005C7: main (example.c:10)
 
Breakdown:
─────────────────────────────────────────────────────────────────
 
==12345==                    Process ID (same for all lines in this run)
 
Invalid read of size 4       Error type and size
                             - Read/Write: direction of access
                             - Size: 1/2/4/8 bytes typically
 
at 0x4005E7: main (example.c:15)    WHERE the error occurred
                                     - Address in executable
                                     - Function name
                                     - Source file and line number
 
by 0x7FFFFFFF: (below main)  Stack trace of calling functions
 
Address 0x5204050 is...      WHAT the problematic address is
                             - Relationship to known blocks
                             - How it was allocated
 
0 bytes after a block        Position relative to valid memory
                             - "0 bytes after" = immediately past end
                             - "4 bytes inside" = within valid block (shouldn't error)
                             - "16 bytes before" = underflow
 
of size 40 alloc'd           The valid block's size
 
at 0x4C2AB80: malloc...      WHERE the block was allocated
                             - Critical for understanding the bug

Common Error Patterns and Fixes:

error_examples.c
C
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
// ERROR: Invalid read of size 4
// Address 0x... is 0 bytes after a block of size 40
int* arr = malloc(10 * sizeof(int));  // 40 bytes
printf("%d", arr[10]);                 // Off-by-one! Index 10 is byte 40-43
// FIX: Use arr[9] or allocate 11 elements
 
// ERROR: Invalid write of size 1  
// Address 0x... is 0 bytes after a block of size 10
char* str = malloc(10);
strcpy(str, "0123456789");  // 11 chars including null terminator!
// FIX: malloc(11) or use strncpy
 
// ERROR: Use of uninitialized value of size 4
int x;
if (x > 0) { ... }  // x was never assigned
// FIX: int x = 0; or ensure assignment before use
 
// ERROR: Invalid free()
// Address 0x... is 0 bytes inside a block of size 100 free'd
char* p = malloc(100);
free(p);
free(p);  // Double free!
// FIX: Set p = NULL after free; check before freeing
 
// ERROR: Mismatched free() / delete / delete[]
// at 0x...: operator delete(void*)
// Address 0x... is 0 bytes inside a block of size 40 alloc'd
// at 0x...: operator new[](unsigned long)
int* arr = new int[10];
delete arr;    // Should be delete[]
// FIX: delete[] arr;
 
// ERROR: Conditional jump depends on uninitialised value
void process(int* data, int count) {
    int sum;  // Uninitialized!
    for (int i = 0; sum < 100 && i < count; i++) {  // Using sum before init
        sum += data[i];
    }
}
// FIX: int sum = 0;

Reading the Address Context

The 'Address is X bytes after/before a block' line is crucial. 'After' suggests buffer overflow. 'Before' suggests underflow. 'Inside ... free'd' indicates use-after-free. 'Not stack'd, malloc'd or (recently) free'd' suggests stack corruption or corrupted pointer.

Comprehensive Memory Leak Analysis

Valgrind's leak detection is one of its most valuable features. At program exit, it analyzes remaining heap allocations and categorizes them by reachability. Leak Classification:

Valgrind Leak Categories
Category	Meaning	Action Required	Example
Definitely Lost	No pointer to block exists anywhere	Must fix - true leak	Pointer overwritten without free
Indirectly Lost	Only reachable via definitely lost block	Will be fixed when parent fixed	Linked list where head is lost
Possibly Lost	Only interior pointer exists	Investigate - may be intentional	Pointer arithmetic on allocated block
Still Reachable	Pointer exists at exit (not freed)	Usually acceptable	Global cache freed at exit by OS

leak_report_example.txt
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
# Run with full leak checking
$ valgrind --leak-check=full --show-leak-kinds=all ./program
 
==12345== HEAP SUMMARY:
==12345==     in use at exit: 1,024 bytes in 4 blocks
==12345==   total heap usage: 100 allocs, 96 frees, 10,240 bytes allocated
 
==12345== 256 bytes in 1 blocks are definitely lost in loss record 3 of 4
==12345==    at 0x4C2AB80: malloc (vg_replace_malloc.c:299)
==12345==    by 0x400547: create_buffer (mylib.c:42)
==12345==    by 0x400623: process_data (main.c:87)
==12345==    by 0x4006B2: main (main.c:112)
 
==12345== 512 bytes in 2 blocks are indirectly lost in loss record 4 of 4
==12345==    at 0x4C2AB80: malloc (vg_replace_malloc.c:299)
==12345==    by 0x400589: add_node (mylib.c:56)
==12345==    by 0x400547: create_buffer (mylib.c:48)
==12345==    by 0x400623: process_data (main.c:87)
==12345==    by 0x4006B2: main (main.c:112)
 
==12345== 256 bytes in 1 blocks are still reachable in loss record 2 of 4
==12345==    at 0x4C2AB80: malloc (vg_replace_malloc.c:299)
==12345==    by 0x4007A1: init_global_cache (cache.c:15)
==12345==    by 0x4006A2: main (main.c:105)
 
==12345== LEAK SUMMARY:
==12345==    definitely lost: 256 bytes in 1 blocks
==12345==    indirectly lost: 512 bytes in 2 blocks
==12345==      possibly lost: 0 bytes in 0 blocks
==12345==    still reachable: 256 bytes in 1 blocks
==12345==         suppressed: 0 bytes in 0 blocks

Interpreting the Leak Report: In the example above: Definitely lost (256 bytes) — A buffer allocated in create_buffer() is truly leaked. The stack trace shows it was called from process_data() . This is the primary bug to fix. Indirectly lost (512 bytes) — These are nodes added to the leaked buffer. Once you fix the definitely lost block, these will also be fixed (assuming proper cleanup of the buffer's contents). Still reachable (256 bytes) — A global cache allocated at startup. This isn't freed before exit, but it's intentional—the OS reclaims it. This is typically acceptable for global resources. Prioritization: Fix definitely lost first, then indirectly lost resolves automatically. Still reachable is usually informational unless you're looking for strict leak-free code.

Possibly Lost Explained

Possibly lost occurs when the only pointer to a block points to its interior, not its start. This can be legitimate (pointer arithmetic for efficiency) or a bug (original pointer was corrupted). Investigate each case. Common legitimate patterns: pointing to the 'data' portion of a struct starting with a header, or custom allocators.

Advanced Valgrind Techniques

Beyond basic usage, Valgrind offers powerful features for complex debugging scenarios. Suppressions: Managing Known Issues Libraries may have benign issues you can't fix. Suppressions let you ignore specific errors:

suppression_file.supp

Text

# myproject.supp - Suppression file example
 
# Suppress a specific leak in third-party library
{
   thirdparty_lib_leak
   Memcheck:Leak
   match-leak-kinds: definite
   fun:malloc
   fun:third_party_init
   fun:main
}
 
# Suppress all leaks in a specific library  
{
   libfoo_all_leaks
   Memcheck:Leak
   ...
   obj:*/libfoo.so*
}
 
# Suppress uninitialized value in system library
{
   glibc_uninitialized
   Memcheck:Cond
   fun:*
   obj:/lib/x86_64-linux-gnu/libc-2.*.so
}
 
# Pattern matching:
# fun:  - function name (can use wildcards: fun:*alloc*)
# obj:  - shared library path
# src:  - source file
# ...   - match any number of frames
 
# Generate suppressions automatically:
# valgrind --gen-suppressions=all ./program 2>&1 | grep -A20 "{"

Client Requests: Valgrind-Aware Code Your program can communicate with Valgrind using client requests:

client_requests.c
C
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
#include <valgrind/memcheck.h>
#include <valgrind/valgrind.h>
 
void custom_allocator_example() {
    // For custom memory pools: tell Valgrind about allocations
    
    char* pool = malloc(1024);
    
    // Mark pool as not accessible initially
    VALGRIND_MAKE_MEM_NOACCESS(pool, 1024);
    
    // When allocating from pool, mark as undefined (allocated but unwritten)
    char* obj = pool;
    VALGRIND_MAKE_MEM_UNDEFINED(obj, 64);
    
    // When freeing back to pool, mark as inaccessible
    VALGRIND_MAKE_MEM_NOACCESS(obj, 64);
    
    // Custom allocation tracking
    VALGRIND_MALLOCLIKE_BLOCK(obj, 64, 0, 0);
    // ... use obj ...
    VALGRIND_FREELIKE_BLOCK(obj, 0);
}
 
void conditional_debugging() {
    // Check if running under Valgrind
    if (RUNNING_ON_VALGRIND) {
        printf("Running under Valgrind - enabling extra checks
");
        enable_debug_mode();
    }
    
    // Request garbage collection (for Valgrind's internal state)
    // Useful before checking for leaks mid-execution
    VALGRIND_DO_ADDED_LEAK_CHECK;
    
    // Count errors so far
    int errors = VALGRIND_COUNT_ERRORS;
    printf("Valgrind errors so far: %d
", errors);
}
 
// Note: These macros are no-ops when not running under Valgrind
// They add zero overhead to production builds

Debugging Under Valgrind with GDB:

valgrind_gdb.sh
Bash
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
# Run Valgrind with GDB server
valgrind --vgdb=yes --vgdb-error=0 ./program
 
# In another terminal, connect GDB
gdb ./program
(gdb) target remote | vgdb
 
# Now you can:
# - Set breakpoints
# - Step through execution  
# - Inspect memory at each Valgrind error
# - Use GDB commands alongside Valgrind's analysis
 
# Useful GDB commands when connected to Valgrind:
(gdb) monitor leak_check full reachable any
(gdb) monitor block_list 
(gdb) monitor who_points_at <address>
 
# The --vgdb-error=0 option stops at first error
# Set to higher number to stop at Nth error

Memory Pool Debugging

If you use custom allocators or memory pools, use VALGRIND_MALLOCLIKE_BLOCK and VALGRIND_FREELIKE_BLOCK macros. Without these, Valgrind can't track allocations from your pool, and you'll miss leaks/errors within it.

Integrating Valgrind into Development Workflows

Valgrind is most effective when integrated into regular development practices, not just used for emergency debugging. CI/CD Integration:

ci_integration.yaml
YAML
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
# GitHub Actions example for Valgrind integration
 
name: Memory Safety Check
 
on: [push, pull_request]
 
jobs:
  valgrind:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
      
      - name: Install Valgrind
        run: sudo apt-get install -y valgrind libc6-dbg
      
      - name: Build with Debug Symbols
        run: |
          mkdir build && cd build
          cmake -DCMAKE_BUILD_TYPE=Debug ..
          make
      
      - name: Run Tests Under Valgrind
        run: |
          cd build
          valgrind \
            --leak-check=full \
            --show-leak-kinds=definite,indirect \
            --error-exitcode=1 \
            --errors-for-leak-kinds=definite \
            --suppressions=../valgrind.supp \
            ./test_suite
      
      - name: Upload Valgrind Log on Failure
        if: failure()
        uses: actions/upload-artifact@v3
        with:
          name: valgrind-log
          path: valgrind.log

Development Workflow Best Practices:

Valgrind Workflow Recommendations

•Run Early, Run Often — Don't wait until bugs manifest. Run Valgrind on new code before merging. Catching issues early is much cheaper than debugging production problems.
•Maintain a Suppression File — Create and version-control a project suppression file for known third-party issues. Review suppressions periodically to ensure they're still needed.
•Use in Pre-Commit Hooks — For critical code paths, run targeted Valgrind tests before allowing commits. Focus on recently changed modules.
•Combine with Unit Tests — Your unit tests are ideal Valgrind targets. They exercise specific functionality and are designed to be fast. Run valgrind ./test_module_x as part of test routine.
•Profile Before Optimizing — Use Callgrind/Cachegrind to profile before making performance changes. 'Optimization' without profiling often makes things worse.
•Document False Positives — When adding suppressions, document why. This helps future developers understand whether a suppression is still valid.

Valgrind vs. AddressSanitizer Trade-offs:

Valgrind vs. AddressSanitizer Comparison
Aspect	Valgrind Memcheck	AddressSanitizer
Slowdown	10-50x	~2x
Preparation	No recompilation needed	Requires recompilation with flags
Memory Overhead	~2x	~2-3x
Leak Detection	Comprehensive at exit	Optional, less detailed
Uninitialized Memory	Excellent detection	Requires MemorySanitizer (separate)
Binary Analysis	Works on any binary	Requires source + compiler support
Platform Support	Linux, macOS, FreeBSD	Linux, macOS (better), Windows (limited)
CI Suitability	Slower, best for targeted tests	Fast enough for full test suites

Complementary Tools

Use both Valgrind and ASan in your workflow. ASan for fast, comprehensive coverage in CI (every build). Valgrind for deep analysis when ASan finds issues, for binary-only debugging, and for detailed leak analysis. They catch different things and complement each other well.

Valgrind

The Memory Detective

Understanding Valgrind's Architecture

Memcheck: The Primary Memory Debugging Tool

Running Valgrind: Complete Usage Guide

Interpreting Valgrind's Error Reports

Comprehensive Memory Leak Analysis

Advanced Valgrind Techniques

Integrating Valgrind into Development Workflows

There’s more to understand.

Valgrind

The Memory Detective

Understanding Valgrind's Architecture

Memcheck: The Primary Memory Debugging Tool

Running Valgrind: Complete Usage Guide

Interpreting Valgrind's Error Reports

Comprehensive Memory Leak Analysis

Advanced Valgrind Techniques

Integrating Valgrind into Development Workflows

There’s more to understand.