Loading learning content...
Every memory access in a segmented system must pass through a critical checkpoint: bounds checking. Before any data is read or written, before any instruction is fetched, the hardware asks a simple but vital question: Is this access within the legal range for this segment?
Bounds checking is the enforcement mechanism that transforms segmentation from a mere addressing scheme into a robust protection system. Without it, segments would be meaningless lines on a memory map—easily crossed, trivially violated. With it, segments become fortress walls that confine programs to their allocated territory, protecting the operating system, other processes, and the data integrity of the system as a whole.
By the end of this page, you will understand the precise mechanics of bounds checking, how it's implemented in hardware, the different checking strategies used across architectures, what happens when bounds are violated, and how this mechanism prevents entire classes of security vulnerabilities.
At its core, bounds checking is a comparison operation that validates the offset against the segment's limit before allowing memory access to proceed.
The Core Validation:
Given:
The bounds check must verify:
d + s ≤ L
Or equivalently, to avoid overflow:
d ≤ L - s
This ensures that every byte being accessed falls within the segment's boundaries. For a single-byte access, this simplifies to d < L (or d ≤ L depending on whether the limit is inclusive).
The exact comparison depends on architectural convention. Some systems define the limit as the segment SIZE (check: offset < limit), others as the LAST VALID OFFSET (check: offset ≤ limit). Intel x86 protected mode uses the latter—a segment of 4096 bytes has limit = 4095. Always verify the architecture's specification!
Bounds checking must be blazingly fast—it happens on every memory access, and any delay directly impacts system performance. Hardware implementations achieve this through parallel comparison circuits that execute simultaneously with other translation steps.
Parallel Execution:
In a typical MMU, bounds checking occurs in parallel with the base address lookup:
Steps 3 and 4 happen simultaneously, so bounds checking adds no latency to the critical path when implemented correctly.
1234567891011121314151617181920212223242526272829303132333435363738394041424344454647484950515253545556
Bounds Checking Hardware Architecture: ┌────────────────────────────────────────────────────────────────┐│ Segment Translation Unit │├────────────────────────────────────────────────────────────────┤│ ││ Logical Address ─────┬────────────────────┐ ││ [seg | offset] │ │ ││ ▼ ▼ ││ ┌─────────────────┐ ┌───────────────┐ ││ │ Segment Number │ │ Offset │ ││ │ Extraction │ │ Extraction │ ││ └────────┬────────┘ └───────┬───────┘ ││ │ │ ││ ▼ │ ││ ┌─────────────────────────────────┐ │ ││ │ Segment Table │ │ ││ │ ┌─────┬───────┬────────┐ │ │ ││ │ │ Seg │ Base │ Limit │ │ │ ││ │ ├─────┼───────┼────────┤ │ │ ││ │ │ 0 │0x1000 │ 4096 │ │ │ ││ │ │ 1 │0x5000 │ 8192 │ │ │ ││ │ │ 2 │0x9000 │ 2048 │ │ │ ││ │ └─────┴───────┴────────┘ │ │ ││ └────────────┬──────────┬────────┘ │ ││ │ │ │ ││ ▼ ▼ │ ││ ┌─────────────────┐ ┌────────────────┐ │ ││ │ Base Value │ │ Limit Value │ │ ││ └────────┬────────┘ └───────┬────────┘ │ ││ │ │ │ ││ │ ▼ ▼ ││ │ ┌───────────────────────┐ ││ │ │ COMPARATOR │ ││ │ │ offset + size ≤ L? │ ││ │ └──────────┬────────────┘ ││ │ │ ││ │ ┌──────────────┼──────────────┐ ││ │ │ │ │ ││ ▼ ▼ ▼ │ ││ ┌─────────────────┐ ┌──────────┐ │ ││ │ ADDER │ │ Valid? │ │ ││ │ base + offset │ │ Gate │ │ ││ └────────┬────────┘ └────┬─────┘ │ ││ │ │ │ ││ └────────┬────────┘ │ ││ ▼ ▼ ││ ┌────────────────┐ ┌────────────────┐ ││ │Physical Address│ │ Fault Signal │ ││ │ (if valid) │ │ (if invalid) │ ││ └────────────────┘ └────────────────┘ ││ │└──────────────────────────────────────────────────────────────┘ Timing: Comparison and addition happen IN PARALLEL.The "valid gate" blocks the physical address if bounds check fails.The Comparator Circuit:
A binary magnitude comparator determines if offset ≤ limit. For n-bit values, this requires O(n) gate delays in a ripple implementation, or O(log n) delays with carry-lookahead techniques. With segment limits typically 16-32 bits, this comparison completes in nanoseconds.
Multi-Byte Access Handling:
For accesses larger than one byte, the hardware must verify that the entire access falls within bounds. Two approaches exist:
offset + size - 1 ≤ limit (risks overflow)offset ≤ limit - size + 1 (requires subtraction)offset ≥ 0 AND offset + size ≤ limitMost implementations handle common access sizes (1, 2, 4, 8 bytes) with dedicated comparison logic.
Modern processors cache recently-used segment descriptors. When a segment's base and limit are cached, the bounds check uses the cached limit, avoiding a segment table lookup. This makes repeated accesses to the same segment extremely fast—the comparison is pure combinational logic with no memory access needed.
When bounds checking fails, the hardware initiates a well-defined exception handling sequence. This sequence ensures that violations are caught, reported, and handled appropriately by the operating system.
The Fault Sequence:
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118
#include <stdio.h>#include <signal.h>#include <setjmp.h>#include <stdlib.h> /* * Segment Fault Handling Demonstration * * On Unix-like systems, segment violations manifest as SIGSEGV. * This shows how the OS notifies user processes of such faults. */ static sigjmp_buf jump_buffer;static volatile sig_atomic_t fault_occurred = 0; /** * Signal handler for segmentation violations * In a real OS, similar logic runs in kernel space */void segfault_handler(int signum) { fault_occurred = 1; printf("\n!!! SEGMENT VIOLATION DETECTED !!!\n"); printf(" Signal received: %d (SIGSEGV)\n", signum); printf(" Cause: Memory access outside valid bounds\n"); printf(" Action: Longjmp to recovery point\n\n"); // In real scenarios without recovery, process would terminate siglongjmp(jump_buffer, 1);} /** * Attempt an out-of-bounds access * Simulates what happens when offset exceeds limit */void trigger_bounds_violation() { // This simulates what happens in segmented systems when // offset >= limit: the access is blocked and fault raised int* null_ptr = NULL; printf("Attempting to access address 0x0 (invalid)...\n"); // This will trigger SIGSEGV on most systems // In a segmented system, this would be caught by bounds checking int value = *null_ptr; // FAULT! // Never reached printf("Value: %d\n", value);} /** * The OS's perspective on handling segment faults */void explain_os_response() { printf("=== OS Response to Segment Violation ===\n\n"); printf("When bounds checking fails, the OS must decide:\n\n"); printf("1. Is this a legitimate fault?\n"); printf(" - Bug in program (buffer overflow, null pointer)\n"); printf(" - Malicious access attempt\n"); printf(" - Stack growth beyond current allocation\n\n"); printf("2. Can it be resolved?\n"); printf(" - Demand paging: page in the needed data\n"); printf(" - Stack expansion: allocate more stack space\n"); printf(" - Copy-on-write: create private copy\n\n"); printf("3. If unresolvable:\n"); printf(" - Deliver SIGSEGV to process (Unix)\n"); printf(" - Default action: terminate with core dump\n"); printf(" - Or process handles signal (if handler installed)\n\n");} int main() { printf("=== Bounds Violation Handling Demo ===\n\n"); explain_os_response(); // Install signal handler struct sigaction sa; sa.sa_handler = segfault_handler; sigemptyset(&sa.sa_mask); sa.sa_flags = 0; if (sigaction(SIGSEGV, &sa, NULL) == -1) { perror("sigaction"); return 1; } printf("Signal handler installed.\n\n"); // Set up recovery point if (sigsetjmp(jump_buffer, 1) == 0) { // First time through - trigger the fault trigger_bounds_violation(); } else { // Returned via longjmp - fault was caught printf("Recovered from segment violation.\n"); printf("In production: process would typically be terminated.\n"); } return fault_occurred ? 1 : 0;} /* * Hardware/OS Interaction on Fault: * * 1. MMU detects: offset > limit * 2. MMU asserts: fault signal * 3. CPU: aborts instruction, saves state * 4. CPU: looks up exception handler in IDT/IVT * 5. CPU: jumps to kernel fault handler * 6. Kernel: examines CR2 (faulting address on x86) * 7. Kernel: checks if fault is resolvable * 8. Kernel: terminates process or delivers signal */| Scenario | Resolution | Outcome |
|---|---|---|
| Stack growth needed | Extend stack segment, map new pages | Instruction restarted, execution continues |
| Demand paging fault | Load page from disk (in paged systems) | Instruction restarted after page loaded |
| Copy-on-write trigger | Create private copy of shared page | Instruction restarted with writable copy |
| Null pointer dereference | Unresolvable | SIGSEGV, process terminated |
| Buffer overflow | Unresolvable | SIGSEGV, process terminated |
| Malicious access attempt | Unresolvable | SIGSEGV, security log entry |
For fault resolution to work, the processor must be able to restart the faulting instruction. This requires saving precise state before the instruction commits any changes. In segmented systems, the fault occurs before any memory access, making restart straightforward—the instruction simply never completed.
Bounds checking is a fundamental security mechanism. It prevents entire classes of attacks that exploit unchecked memory access to corrupt data, hijack control flow, or leak sensitive information.
The Buffer Overflow Example:
Consider a classic buffer overflow attack:
char buffer[64];
strcpy(buffer, user_input); // No length check!
In a flat address space without bounds checking, a 100-byte input overwrites 36 bytes beyond the buffer, potentially corrupting return addresses or function pointers.
In a segmented system with bounds checking:
The effectiveness depends on segment granularity—finer segments provide stronger protection.
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104
#include <stdio.h>#include <string.h>#include <stdint.h> /* * Security Analysis: Bounds Checking vs Buffer Overflow * * Demonstrates how segment boundaries affect exploit possibilities. */ // Simulated segment layouttypedef struct { char buffer[64]; // Offsets 0-63 int important_value; // Offsets 64-67 void (*callback)(void); // Offsets 68-75} VulnerableData; // Simulated segment limits for different scenarios#define LIMIT_TIGHT 64 // Segment covers only buffer#define LIMIT_STRUCT 76 // Segment covers entire struct #define LIMIT_LOOSE 1024 // Segment has extra space void check_overflow_scenario(const char* name, uint32_t limit, uint32_t write_offset, uint32_t write_size) { printf("Scenario: %s\n", name); printf(" Segment limit: %u bytes\n", limit); printf(" Write offset: %u, size: %u\n", write_offset, write_size); uint32_t write_end = write_offset + write_size - 1; if (write_end >= limit) { printf(" Result: BOUNDS VIOLATION - Attack blocked!\n"); printf(" Write end (%u) exceeds limit (%u)\n", write_end, limit); } else { printf(" Result: Write permitted within segment bounds\n"); if (write_end >= 64) { printf(" WARNING: Intra-segment overflow corrupts struct fields!\n"); } } printf("\n");} int main() { printf("=== Bounds Checking Security Analysis ===\n\n"); printf("Vulnerable struct layout:\n"); printf(" Bytes 0-63: char buffer[64]\n"); printf(" Bytes 64-67: int important_value\n"); printf(" Bytes 68-75: void (*callback)(void)\n\n"); printf("Attack: Overflow buffer with 100 bytes starting at offset 0\n\n"); // Scenario 1: Tight segment around buffer only check_overflow_scenario( "Tight Segmentation (buffer in its own segment)", LIMIT_TIGHT, 0, 100 // Attempt to write 100 bytes starting at offset 0 ); // Scenario 2: Segment contains entire struct check_overflow_scenario( "Struct-level Segmentation", LIMIT_STRUCT, 0, 100 // 100 bytes > 76 byte segment ); // Scenario 3: Loose segment with extra space check_overflow_scenario( "Loose Segmentation (large segment)", LIMIT_LOOSE, 0, 100 // 100 bytes < 1024 byte segment ); printf("=== Key Insight ===\n"); printf("Bounds checking effectiveness depends on segmentation granularity.\n"); printf("Finer segments = more protection, but more overhead.\n"); printf("Coarse segments = less protection, but simpler management.\n"); return 0;} /* * Output: * * Attack: Overflow buffer with 100 bytes starting at offset 0 * * Scenario: Tight Segmentation (buffer in its own segment) * Segment limit: 64 bytes * Write offset: 0, size: 100 * Result: BOUNDS VIOLATION - Attack blocked! * Write end (99) exceeds limit (64) * * Scenario: Struct-level Segmentation * Segment limit: 76 bytes * Write offset: 0, size: 100 * Result: BOUNDS VIOLATION - Attack blocked! * Write end (99) exceeds limit (76) * * Scenario: Loose Segmentation (large segment) * Segment limit: 1024 bytes * Write offset: 0, size: 100 * Result: Write permitted within segment bounds * WARNING: Intra-segment overflow corrupts struct fields! */Bounds checking prevents segment-boundary violations but doesn't stop all attacks. Intra-segment overflows, type confusion, time-of-check-time-of-use races, and many other vulnerabilities require additional defenses. Modern systems combine bounds checking with ASLR, stack canaries, DEP/NX, and other mitigations for defense in depth.
Different processor architectures implement bounds checking with varying strategies, reflecting different design philosophies and performance/security tradeoffs.
| Architecture | Checking Method | Limit Interpretation | Notes |
|---|---|---|---|
| Intel 8086 | Software (near), none (far) | N/A in real mode | No hardware protection in real mode |
| Intel 80286+ | Hardware comparator | Last valid offset | Limit is size-1; 20-bit limit field |
| Intel 80386+ | Hardware + granularity | Byte or 4KB units | G bit: limit × 4096 if set |
| MIPS (segments) | Base + bound registers | Size in bytes | Separate registers per segment |
| Multics | Hardware per segment | Segment size | Very fine-grained segments |
| ARM (legacy) | Domain-based | Regions defined in CP15 | Coarse protection domains |
Intel x86 Protected Mode Bounds Checking:
The x86 architecture provides sophisticated bounds checking in protected mode:
1234567891011121314151617181920212223242526272829303132333435363738
Intel x86 Segment Descriptor and Bounds Checking: Segment Descriptor (8 bytes):┌───────────────────────────────────────────────────────────────┐│ 63 56 55 52 51 48 47 40 39 32 ││ Base[31:24] │ Flags │ Limit │ Access │ Base[23:16] ││ │ G D L │[19:16]│ Rights │ │├───────────────────────────────────────────────────────────────┤│ 31 16 15 0││ Base[15:0] │ Limit[15:0] │└───────────────────────────────────────────────────────────────┘ Flags: G (Granularity): 0 = limit in bytes, 1 = limit in 4KB pages D (Default): 0 = 16-bit, 1 = 32-bit L (Long mode): 1 = 64-bit code segment Limit Calculation: If G = 0: Effective Limit = Limit (up to 1 MB) If G = 1: Effective Limit = (Limit × 4096) + 4095 (up to 4 GB) Bounds Check (Normal Segment): IF offset > Effective_Limit THEN #GP(0) // General Protection Fault Bounds Check (Expand-Down Stack Segment): IF offset ≤ Effective_Limit THEN #SS(0) // Stack Fault (Valid range is ABOVE the limit, not below) Example: Limit = 0x7FFFF, G = 1 Effective Limit = (0x7FFFF × 4096) + 4095 = 2,147,483,647 Maximum addressable offset = 2 GB - 1 Access Rights byte includes: - Present bit: segment must be in memory - DPL: Descriptor Privilege Level (0-3) - S: System vs code/data segment - Type: Readable, writable, executable, conforming, etc.Traditional segment-based bounds checking has largely been replaced by paging in modern systems. However, the need for bounds checking led to new approaches: Intel MPX attempted hardware-assisted pointer bounds (discontinued), while ARM MTE and CHERI represent current efforts to bring fine-grained bounds checking back in new forms.
Bounds checking transforms segmentation from an addressing convenience into a protection mechanism. Every offset is validated against the segment's limit before any memory access proceeds, catching violations at the hardware level.
What's Next:
With bounds checking passed, the offset is confirmed to be within legal range. Now we can proceed to the final step of address translation: forming the physical address. By adding the validated offset to the segment's base address, we compute the actual memory location that the hardware will access.
You now understand bounds checking—the guardian that validates every memory access against segment limits. You can explain its hardware implementation, describe what happens on violation, and analyze its security implications. Next, we'll see how the physical address is finally computed.