Loading content...
In 2011, Google engineers faced a dilemma. Valgrind was excellent at finding memory bugs, but its 20x slowdown made it impractical for continuous testing of large codebases like Chrome. Running Chrome's full test suite under Valgrind would take days instead of hours. They needed something faster—something that could run with every build, not just special debugging sessions.
The result was AddressSanitizer (ASan)—a compiler-based memory error detector that achieves only ~2x slowdown while catching buffer overflows, use-after-free, and other critical memory bugs. Since its introduction, ASan has found over 25,000 bugs in Chrome alone, and countless more across Firefox, the Linux kernel, LLVM, and thousands of open-source projects.
ASan's key innovation was moving detection logic from runtime interpretation (like Valgrind) to compile-time instrumentation. By having the compiler insert checks directly into the generated code, and using highly optimized shadow memory schemes, ASan achieves unprecedented efficiency for memory safety checking.
By the end of this page, you will understand: (1) ASan's architecture and how compile-time instrumentation differs from runtime analysis, (2) The shadow memory scheme that enables fast checking, (3) What memory errors ASan detects (and doesn't), (4) How to compile, run, and interpret ASan output, (5) The sanitizer family (MSan, TSan, UBSan) and when to use each, and (6) Best practices for integrating sanitizers into your development workflow.
ASan's efficiency comes from its compile-time approach. Instead of interpreting code at runtime, the compiler instruments the source code directly, adding checks that execute as native machine code.
Compile-Time Instrumentation:
123456789101112131415161718192021222324252627
// What you write:int array[10];array[i] = 42; // What ASan-compiled code effectively becomes:int array[10];// ... plus redzone padding on stack ... // Before every memory access:if (!is_addressable(&array[i], sizeof(int))) { report_error("stack-buffer-overflow", &array[i]);}array[i] = 42; // The is_addressable check is HIGHLY optimized:// 1. Uses shadow memory for O(1) lookup// 2. Inlined as few instructions (not a function call)// 3. Shadow computation uses bit shifts, not division // Assembly-level view of the check:// mov rdi, address// shr rdi, 3 ; Divide by 8 to get shadow byte address// add rdi, shadow_base// movzbl al, [rdi] ; Load shadow byte// test al, al ; Is it zero (fully addressable)?// jne slow_path ; If not, do detailed check// [original memory access]Shadow Memory Scheme:
ASan maps every 8 bytes of application memory to 1 byte of shadow memory. This shadow byte encodes the addressability status:
12345678910111213141516171819202122232425262728293031323334
ASan Shadow Memory Encoding════════════════════════════════════════════════════════════════ Application Memory Shadow Byte Meaning────────────────────────────────────────────────────────────────[8 bytes valid] 0x00 All 8 bytes addressable[7 bytes valid] 0x07 First 7 bytes addressable [6 bytes valid] 0x06 First 6 bytes addressable[5 bytes valid] 0x05 First 5 bytes addressable[4 bytes valid] 0x04 First 4 bytes addressable[3 bytes valid] 0x03 First 3 bytes addressable[2 bytes valid] 0x02 First 2 bytes addressable[1 byte valid] 0x01 First 1 byte addressable Special Shadow Values (Negative):────────────────────────────────────────────────────────────────Heap left redzone 0xfa Padding before heap allocationHeap right redzone 0xfb Padding after heap allocationFreed heap 0xfd Memory was freedStack left redzone 0xf1 Before stack variableStack mid redzone 0xf2 Between stack variables Stack right redzone 0xf3 After stack variablesGlobal redzone 0xf9 Padding around global variablesStack use after return 0xf5 Stack frame after function returned Shadow Memory Layout (64-bit Linux):────────────────────────────────────────────────────────────────Shadow address = (ApplicationAddress >> 3) + 0x7fff8000 Application: 0x10000000000 - 0x7fffffffffff (High Memory) ↓ >> 3 + offset ↓Shadow: 0x2000000000 - 0x10007fff7fff This gives ~1/8 of address space to shadow memoryRedzones: The Detection Mechanism
ASan surrounds every allocation with "redzones"—poisoned memory regions that should never be accessed. Any access to a redzone is an overflow:
1234567891011121314151617181920212223242526
Heap Allocation Layout with ASan═════════════════════════════════════════════════════════════ malloc(16): ┌─────────────┬─────────────────────────────┬─────────────┐│ Left Redzone│ User Memory (16) │Right Redzone││ (poisoned) │ (valid) │ (poisoned) │└─────────────┴─────────────────────────────┴─────────────┘ │ │ │ ▼ ▼ ▼ Access = ERROR Access = OK Access = ERROR Stack Variable Layout: Function with: int a; int b[10]; int c; ┌────────┬──────┬────────┬───────────┬────────┬──────┬────────┐│Redzone │ a │Redzone │ b[10] │Redzone │ c │Redzone ││ (F1) │ (4B) │ (F2) │ (40B) │ (F2) │ (4B) │ (F3) │└────────┴──────┴────────┴───────────┴────────┴──────┴────────┘ This catches:- b[-1] → hits redzone between a and b- b[10] → hits redzone between b and c - Overflow from a into b's memoryThe shadow memory check compiles to just 3-4 instructions in the common case (when access is valid). Only when a potential error is detected does the slower path execute. Since most accesses are valid, the overhead stays low. Additionally, modern CPUs cache shadow memory effectively, reducing memory bandwidth impact.
ASan is highly effective but not omniscient. Understanding its detection capabilities and limitations is essential for effective use.
Errors ASan Detects:
| Error Type | Detection Quality | Example |
|---|---|---|
| Heap buffer overflow | Excellent | malloc(10); p[10] = 'x'; |
| Stack buffer overflow | Excellent | int a[10]; a[10] = 1; |
| Global buffer overflow | Excellent | int g[10]; g[15] = 1; |
| Heap use-after-free | Excellent (with quarantine) | free(p); *p = 0; |
| Stack use after return | With ASAN_OPTIONS=detect_stack_use_after_return=1 | return &local_var; |
| Stack use after scope | With -fsanitize-address-use-after-scope | { int x; } use(x); |
| Double free | Excellent | free(p); free(p); |
| Invalid free | Excellent | free(stack_var_ptr); |
| Memory leaks | Optional with LeakSanitizer | Unreachable heap at exit |
| Initialization order bugs | With -fsanitize=init-order | Static init dependencies |
What ASan Does NOT Detect:
123456789101112131415161718192021222324252627282930
// Example: What ASan DOESN'T catch #include <string.h> struct User { char name[16]; // Offset 0-15 int is_admin; // Offset 16-19}; void vulnerable(struct User* u, const char* input) { // If input is "AAAAAAAAAAAAAAAA1" (17 chars) // name overflows into is_admin strcpy(u->name, input); // ASan sees valid access within struct! // is_admin is now corrupted if (u->is_admin) { grant_admin_access(); // Security vulnerability! }} // ASan doesn't detect this because:// 1. The struct is one allocation// 2. name and is_admin are within the same 24-byte block// 3. Access is to valid (allocated) memory// 4. No redzone exists BETWEEN struct members // Detection requires:// - Bounds checking at the field level (some compilers can do this)// - Stack buffer overflow protection (partial)// - Careful code reviewA clean ASan run doesn't mean your code is bug-free. It means ASan didn't find bugs during that particular execution. Different inputs, different code paths, or bugs in ASan's blind spots might still exist. Use ASan as one layer of defense, not the only one.
ASan requires compiler support. It's built into Clang, GCC, and MSVC (Windows). Here's how to use it effectively.
Basic Compilation:
1234567891011121314151617181920212223242526272829
# Clang (recommended - most complete implementation)clang -fsanitize=address -g -O1 program.c -o program # GCCgcc -fsanitize=address -g -O1 program.c -o program # C++clang++ -fsanitize=address -g -O1 program.cpp -o program # Explanation of flags:# -fsanitize=address : Enable AddressSanitizer# -g : Include debug symbols (essential for readable reports)# -O1 : Optimization level 1 (recommended)# -O0 works but slightly slower# -O2/-O3 work but may complicate stack traces # Additional useful flags:clang -fsanitize=address \ -fno-omit-frame-pointer \ # Better stack traces -fno-optimize-sibling-calls \ # Clearer call stacks -g \ program.c -o program # For shared libraries (all must be ASan-compiled):clang -fsanitize=address -shared -fPIC lib.c -o libfoo.soclang -fsanitize=address program.c -L. -lfoo -o program # Static linking of ASan runtime (for distribution):clang -fsanitize=address -static-libasan program.c -o programCMake Integration:
12345678910111213141516171819202122232425
# CMake ASan integration # Option to enable sanitizersoption(ENABLE_ASAN "Enable AddressSanitizer" OFF)option(ENABLE_UBSAN "Enable UndefinedBehaviorSanitizer" OFF) if(ENABLE_ASAN) add_compile_options(-fsanitize=address -fno-omit-frame-pointer) add_link_options(-fsanitize=address)endif() if(ENABLE_UBSAN) add_compile_options(-fsanitize=undefined) add_link_options(-fsanitize=undefined)endif() # ASan + UBSan together (recommended combination)if(ENABLE_ASAN AND ENABLE_UBSAN) add_compile_options(-fsanitize=address,undefined -fno-omit-frame-pointer) add_link_options(-fsanitize=address,undefined)endif() # Usage:# cmake -DENABLE_ASAN=ON -DCMAKE_BUILD_TYPE=Debug ..# makeRuntime Configuration:
1234567891011121314151617181920212223242526272829303132333435363738
# ASan behavior is controlled via ASAN_OPTIONS environment variable # Increase verbosityASAN_OPTIONS=verbosity=1 ./program # Enable leak detection (on by default on Linux, off on macOS)ASAN_OPTIONS=detect_leaks=1 ./program # Disable leak detection (faster, when leaks aren't the focus)ASAN_OPTIONS=detect_leaks=0 ./program # Abort on first error (default: continue and show all errors)ASAN_OPTIONS=halt_on_error=1 ./program # Continue on errors, don't abortASAN_OPTIONS=halt_on_error=0 ./program # Detect stack-use-after-return (extra overhead)ASAN_OPTIONS=detect_stack_use_after_return=1 ./program # Increase heap quarantine (better UAF detection, more memory)ASAN_OPTIONS=quarantine_size_mb=256 ./program # Larger redzones (catch larger overflows, more memory)ASAN_OPTIONS=redzone=128 ./program # Multiple options combinedASAN_OPTIONS=detect_leaks=1:halt_on_error=0:verbosity=1 ./program # Write output to fileASAN_OPTIONS=log_path=/tmp/asan ./program# Creates /tmp/asan.<pid> # Suppress specific errors (like Valgrind)ASAN_OPTIONS=suppressions=asan.supp ./program # Fast stack unwinding (less accurate but faster)ASAN_OPTIONS=fast_unwind_on_malloc=1 ./programASan includes LeakSanitizer (LSan) for memory leak detection. On Linux, it's enabled by default. On macOS, use ASAN_OPTIONS=detect_leaks=1 to enable it. LSan is lightweight and adds minimal overhead to ASan's runtime cost.
ASan provides detailed, colorized error reports. Understanding their structure helps you quickly identify and fix bugs.
Anatomy of an ASan Report:
12345678910111213141516171819202122232425
==12345==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x614000000050 at pc 0x00000050f8b2 bp 0x7ffc6d6d0e50 sp 0x7ffc6d6d0e48WRITE of size 4 at 0x614000000050 thread T0 #0 0x50f8b1 in main /home/user/demo.c:8:14 #1 0x7f3d8c0e1c86 in __libc_start_main /build/glibc-2.27/csu/../csu/libc-start.c:310 #2 0x41c499 in _start (/home/user/demo+0x41c499) 0x614000000050 is located 0 bytes to the right of 80-byte region [0x614000000000,0x614000000050)allocated by thread T0 here: #0 0x4de3a8 in malloc /llvm/projects/compiler-rt/lib/asan/asan_malloc_linux.cc:88:3 #1 0x50f871 in main /home/user/demo.c:6:20 #2 0x7f3d8c0e1c86 in __libc_start_main /build/glibc-2.27/csu/../csu/libc-start.c:310 SUMMARY: AddressSanitizer: heap-buffer-overflow /home/user/demo.c:8:14 in mainShadow bytes around the buggy address: 0x0c287fff7fb0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 0x0c287fff7fc0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 0x0c287fff7fd0: fa fa fa fa 00 00 00 00 00 00 00 00 00 00 00 00=>0x0c287fff7fe0: 00 00 00 00 00 00 00 00 00 00[fa]fa fa fa fa fa 0x0c287fff7ff0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa faShadow byte legend (one byte per 8 bytes of mem): Addressable: 00 Heap left redzone: fa Heap right redzone: fb Freed heap region: fd ...Decoding the Report:
heap-buffer-overflow tells you exactly what kind of bug it is. Other types: stack-buffer-overflow, heap-use-after-free, double-free, etc.WRITE of size 4 means a 4-byte write caused the error. Could also be READ.0 bytes to the right of 80-byte region means we wrote exactly at the end of an 80-byte allocation. Off-by-one error![fa] shows the shadow byte we hit. fa = heap right redzone = overflow detected.Common Error Types and Their Meaning:
| Error | Shadow Byte | Meaning | Typical Cause |
|---|---|---|---|
heap-buffer-overflow | fa/fb | Write/read past heap allocation | Off-by-one, wrong size calculation |
stack-buffer-overflow | f1/f2/f3 | Write/read past stack variable | Local array overflow |
global-buffer-overflow | f9 | Write/read past global variable | Global array overflow |
heap-use-after-free | fd | Access after free() | Dangling pointer, UAF |
stack-use-after-return | f5 | Return and use of local address | Returning pointer to local |
double-free | fd | free() called twice | Ownership confusion |
alloc-dealloc-mismatch | — | malloc/delete or new/free | Wrong deallocation function |
For meaningful stack traces, compile with -g and ensure llvm-symbolizer or addr2line is in your PATH. On macOS, use dsymutil to generate dSYM bundles. Without symbols, you'll only see addresses like 0x50f8b1 without file:line information.
ASan is part of a family of sanitizers, each targeting different bug classes. Understanding when to use each helps maximize your testing effectiveness.
The Sanitizer Family:
| Sanitizer | Abbreviation | Detects | Overhead | Compatible With |
|---|---|---|---|---|
| AddressSanitizer | ASan | Buffer overflows, UAF, leaks | ~2x | UBSan |
| MemorySanitizer | MSan | Uninitialized memory reads | ~3x | Not ASan |
| ThreadSanitizer | TSan | Data races, deadlocks | ~5-15x | Not ASan/MSan |
| UndefinedBehaviorSanitizer | UBSan | Undefined behavior | Minimal | ASan, MSan, TSan |
| LeakSanitizer | LSan | Memory leaks | Minimal | Part of ASan, standalone |
| HWAddressSanitizer | HWASan | Like ASan, hardware-assisted | ~2x | ARM64 only |
MemorySanitizer (MSan):
123456789101112131415161718192021222324252627
// MSan detects uninitialized memory reads #include <stdio.h>#include <stdlib.h> int main() { int* p = malloc(sizeof(int)); // p points to allocated but UNINITIALIZED memory if (*p > 0) { // MSan ERROR: use of uninitialized value printf("positive\n"); } free(p); return 0;} // Compile: clang -fsanitize=memory -g program.c// Run: ./a.out// // Output:// ==12345==WARNING: MemorySanitizer: use-of-uninitialized-value// #0 0x... in main program.c:8:9 // IMPORTANT: MSan requires ALL code to be MSan-compiled// Including libc! Use MemorySanitizer-instrumented libc// or many false positives from library codeThreadSanitizer (TSan):
12345678910111213141516171819202122232425262728293031
// TSan detects data races #include <pthread.h> int counter = 0; void* increment(void* arg) { for (int i = 0; i < 1000000; i++) { counter++; // DATA RACE: unsynchronized access } return NULL;} int main() { pthread_t t1, t2; pthread_create(&t1, NULL, increment, NULL); pthread_create(&t2, NULL, increment, NULL); pthread_join(t1, NULL); pthread_join(t2, NULL); printf("Counter: %d\n", counter); return 0;} // Compile: clang -fsanitize=thread -g program.c -pthread//// Output:// WARNING: ThreadSanitizer: data race (pid=12345)// Write of size 4 at 0x... by thread T2:// #0 increment program.c:8// Previous write of size 4 at 0x... by thread T1:// #0 increment program.c:8UndefinedBehaviorSanitizer (UBSan):
1234567891011121314151617181920212223242526272829303132
// UBSan detects undefined behavior #include <stdint.h>#include <stdio.h> int main() { int x = INT32_MAX; x = x + 1; // Signed integer overflow - UNDEFINED! printf("%d\n", x); int arr[10]; int y = arr[20]; // Array OOB (UBSan also catches this) int* p = NULL; int z = *p; // Null pointer dereference return 0;} // Compile: clang -fsanitize=undefined -g program.c// // Output:// program.c:7:9: runtime error: signed integer overflow:// 2147483647 + 1 cannot be represented in type 'int'//// UBSan sub-sanitizers (can enable individually):// -fsanitize=signed-integer-overflow// -fsanitize=null // -fsanitize=alignment// -fsanitize=shift// -fsanitize=vptr (C++ virtual calls)// ... many moreASan, MSan, and TSan are mutually exclusive—they use incompatible shadow memory schemes. You can combine ASan+UBSan or TSan+UBSan, but not ASan+TSan or ASan+MSan. Run separate builds/tests for each sanitizer you want to use.
Maximizing ASan's effectiveness requires thoughtful integration into your development process.
Development Workflow:
ASAN_OPTIONS=quarantine_size_mb=512 to hold freed memory longer. This improves UAF detection at the cost of memory.detect_leaks=1, stack-use-after-return, and scope detection for comprehensive coverage during testing.CI/CD Example:
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354
# GitHub Actions: Comprehensive Sanitizer Testing name: Sanitizer Tests on: [push, pull_request] jobs: asan: name: AddressSanitizer + UBSan runs-on: ubuntu-latest env: ASAN_OPTIONS: detect_leaks=1:halt_on_error=1:abort_on_error=1 UBSAN_OPTIONS: halt_on_error=1:print_stacktrace=1 steps: - uses: actions/checkout@v3 - name: Build with ASan + UBSan run: | cmake -B build \ -DCMAKE_C_COMPILER=clang \ -DCMAKE_CXX_COMPILER=clang++ \ -DCMAKE_C_FLAGS="-fsanitize=address,undefined -g" \ -DCMAKE_CXX_FLAGS="-fsanitize=address,undefined -g" cmake --build build - name: Test run: ctest --test-dir build --output-on-failure tsan: name: ThreadSanitizer runs-on: ubuntu-latest env: TSAN_OPTIONS: halt_on_error=1 steps: - uses: actions/checkout@v3 - name: Build with TSan run: | cmake -B build \ -DCMAKE_C_FLAGS="-fsanitize=thread -g" \ -DCMAKE_CXX_FLAGS="-fsanitize=thread -g" cmake --build build - name: Test run: ctest --test-dir build --output-on-failure msan: name: MemorySanitizer runs-on: ubuntu-latest steps: - uses: actions/checkout@v3 - name: Build with MSan # Note: MSan requires MSan-instrumented libraries run: | cmake -B build \ -DCMAKE_C_FLAGS="-fsanitize=memory -g" \ -DCMAKE_CXX_FLAGS="-fsanitize=memory -g" cmake --build build || echo "MSan build may need special setup"Fuzzing with ASan:
123456789101112131415161718192021222324252627282930313233
# LibFuzzer + ASan = Powerful bug finding # Create a fuzz targetcat > fuzz_target.c << 'EOF'#include <stdint.h>#include <stddef.h> // Your code under testvoid parse_input(const uint8_t* data, size_t size); // LibFuzzer entry pointint LLVMFuzzerTestOneInput(const uint8_t *data, size_t size) { parse_input(data, size); return 0;}EOF # Compile with fuzzing + ASanclang -fsanitize=fuzzer,address -g fuzz_target.c parse.c -o fuzzer # Create seed corpus directorymkdir corpusecho "normal input" > corpus/seed1 # Run fuzzer (will run indefinitely, Ctrl-C to stop)./fuzzer corpus/ -max_len=1024 # If a crash is found, fuzzer saves reproducer to crash-*# Run again to see ASan report:./fuzzer crash-abc123 # LibFuzzer automatically stops on ASan errors# Minimizes crashing input to smallest reproducerFuzzing generates millions of inputs; sanitizers catch bugs triggered by those inputs. Together, they find bugs no amount of manual testing would discover. Google's ClusterFuzz has found 20,000+ bugs using this combination. Most serious open-source projects now use continuous fuzzing.
Understanding when to use ASan versus alternatives helps build an effective testing strategy.
Comprehensive Comparison:
| Aspect | AddressSanitizer | Valgrind Memcheck | Static Analysis |
|---|---|---|---|
| Type | Compile-time instrumentation | Runtime interpretation | Source analysis |
| Overhead | ~2x CPU, ~2-3x memory | 10-50x CPU, ~2x memory | None at runtime |
| Setup | Recompile with flags | Run any binary under valgrind | Integrate into build |
| Buffer overflow | Excellent | Excellent | Good (obvious cases) |
| Use-after-free | Excellent (with quarantine) | Excellent | Limited |
| Memory leaks | Good (via LSan) | Excellent (detailed) | Limited |
| Uninitialized reads | No (use MSan) | Excellent | Good |
| CI/CD suitability | Excellent (fast) | Limited (slow) | Excellent |
| Production use | Possible (with overhead) | No (too slow) | N/A (pre-release) |
| False positives | Very low | Low | Can be higher |
Recommended Testing Strategy:
12345678910111213141516171819202122232425262728293031323334353637383940414243
Comprehensive Memory Safety Testing Strategy═══════════════════════════════════════════════════════════════ Development Phase:──────────────────□ ASan + UBSan as default debug build - Fast enough for regular development - Catches most bugs immediately □ TSan build for concurrent code - Separate build, run periodically - Essential for multithreaded code Continuous Integration:───────────────────────□ ASan + UBSan on every PR - Full test suite with sanitizers - Block merge on any sanitizer error □ TSan weekly/nightly - More overhead, but catches data races - Run on dedicated CI resources □ Fuzzing with ASan continuously - ClusterFuzz, OSS-Fuzz, or own setup - 24/7 bug hunting Periodic Deep Analysis:───────────────────────□ Valgrind monthly or on major releases - Comprehensive leak analysis - Catches some things ASan misses - Good for uninitialized reads if MSan is hard to set up □ Static analysis (Clang Static Analyzer, Coverity) - Catches bugs statically - Complement to dynamic analysis Release Testing:────────────────□ Full sanitizer test suite pass required□ Clean Valgrind run on critical paths□ Fuzz testing milestone (N hours without new bugs)No single tool finds all bugs. Static analysis catches what dynamic misses (unexplored code paths). Valgrind's detailed uninitialized tracking catches what ASan doesn't. TSan finds races invisible to memory sanitizers. Fuzzing explores input space humans wouldn't imagine. Use all layers.
AddressSanitizer has transformed memory safety testing from an occasional deep-dive activity into a continuous practice integrated into every build. Its efficiency makes comprehensive testing practical at scale.
Key Takeaways:
-fsanitize=address to your build. CMake integration is straightforward. Works with GCC, Clang, and MSVC.Module Complete:
This concludes our comprehensive exploration of memory debugging. You now understand:
With these tools and concepts, you're equipped to write memory-safe code and debug the memory errors that inevitably occur in complex systems.
You now have comprehensive knowledge of memory debugging—from understanding memory error classes to using industry-standard tools for detection and prevention. Apply these techniques consistently: use sanitizers in development, integrate them into CI/CD, and combine with fuzzing for maximum bug-finding power. Memory safety requires vigilance, but the tools are now powerful enough to catch bugs before they reach production.