Every buffer overflow, every array out-of-bounds access, every wild pointer dereference represents unauthorized memory access. In the relentless battle between software bugs and system integrity, the segment limit stands as a hardware-enforced line of defense.

The segment limit defines the maximum extent of a segment—the boundary beyond which access is forbidden. Unlike software-based bounds checking that can be bypassed or forgotten, limit checking happens in silicon, on every memory access, without exception.

This is not a suggestion; it is a law of physics for that CPU.

When a program attempts to access an offset beyond the segment limit, the CPU stops the access in its tracks, generates a processor exception, and hands control to the operating system's exception handler. The rogue access never reaches memory. Data integrity is preserved. The system survives.

Understanding segment limits at a deep level reveals how hardware and software cooperate to create protected execution environments—and why modern systems still employ similar concepts through paging.

The segment limit field exists to answer a fundamental question: How large is this segment?

But the implications of this simple question are profound. By defining segment size, the limit enables:

1. Memory Protection:

Each segment has well-defined boundaries. A data segment cannot accidentally (or maliciously) access code, stack, or kernel memory. Even if a program has a buffer overflow bug, the corruption is contained within the segment's bounds.

2. Resource Allocation:

The limit represents committed memory. The OS knows exactly how much physical memory is allocated to each segment, enabling accurate resource accounting and enforcement of memory quotas.

3. Sparse Address Spaces:

Not all logical addresses need to be valid. By setting segment limits appropriately, large address space ranges can be marked as inaccessible, catching null pointer dereferences and uninitialized pointer usage.

4. Growth Control:

Dynamic segments like heaps and stacks need room to grow. The limit defines the current size; the OS can increase it as the segment grows, maintaining control over memory consumption.

The limit field occupies a fixed number of bits in the segment descriptor. In x86 protected mode, the limit is a 20-bit value, split across two locations in the descriptor (just like the base address).

x86 Limit Field Layout:

Segment Descriptor (8 bytes):

Bytes 0-1:  Limit[15:0]           (lower 16 bits of limit)
Byte 6:     Limit[19:16] | Flags  (upper 4 bits + G, D/B, L, AVL flags)

Extracting the Limit:

uint32_t limit = (descriptor[0] | (descriptor[1] << 8)) |   // Limit[15:0]
                 ((descriptor[6] & 0x0F) << 16);              // Limit[19:16]

Limit Interpretation:

The raw limit value represents the last valid offset within the segment, not the size. For a segment containing bytes 0 through N, the limit is N.

• Limit = 0: Only offset 0 is valid (1 byte)
• Limit = 0xFFFF: Offsets 0 through 65535 valid (64 KB)
• Limit = 0xFFFFF: Offsets 0 through 1,048,575 valid (1 MB with byte granularity)

The 1MB Limitation Problem:

A 20-bit limit can represent at most 1,048,575—meaning segments cannot exceed 1 MB with byte granularity. For 32-bit systems where segments up to 4 GB are needed, this is unacceptable. The solution is the granularity flag.

The granularity (G) flag in the segment descriptor determines how the limit value is interpreted. This single bit expands the range of possible segment sizes from 1 MB to 4 GB.

Granularity Flag Semantics:

• G = 0 (Byte Granularity): The limit is interpreted in bytes. The maximum segment size is 2²⁰ = 1,048,576 bytes (1 MB).

• G = 1 (Page Granularity): The limit is interpreted in 4 KB pages. The actual byte limit = (limit + 1) × 4096 - 1. Maximum segment size is 2²⁰ × 4096 = 4,294,967,296 bytes (4 GB).

Mathematical Transformation:

Effective Limit (G=0): raw_limit
Effective Limit (G=1): (raw_limit + 1) × 4096 - 1
                     = raw_limit × 4096 + 4095
                     = (raw_limit << 12) | 0xFFF

Example Calculations:

Granularity and Precision Trade-off:

With page granularity, you lose fine-grained control over segment size. You cannot create a segment of exactly 5000 bytes with G=1; the closest you get is either 4 KB (4096 bytes) or 8 KB (8192 bytes).

This trade-off is acceptable for most use cases:

• Code segments are typically page-aligned anyway
• Data segments are usually large enough that 4 KB granularity doesn't matter
• The OS manages memory in page units, making page-aligned limits natural

Common Configurations:

// Flat 4GB data segment (typical for modern OS)
Base  = 0x00000000
Limit = 0xFFFFF
G     = 1
Effective range: 0 to 4,294,967,295 (entire 32-bit space)

// 64KB code segment (embedded system)
Base  = 0x00010000
Limit = 0xFFFF
G     = 0
Effective range: 0 to 65535 (64 KB)

// 128KB data segment
Base  = 0x00020000
Limit = 0x1FFFF
G     = 0
Effective range: 0 to 131071 (128 KB with byte precision)

// 1MB stack segment
Base  = 0x00100000
Limit = 0x0FF
G     = 1
Effective range: 0 to 1048575 (1 MB with page granularity)

Most segments are expand-up—valid offsets start at 0 and extend upward to the limit. But stacks present a unique challenge: they grow downward from high addresses to low addresses. Expand-down segments address this by inverting how the limit is interpreted.

Expand-Up Semantics (E=0):

Valid offset range: 0 to limit (inclusive)

    Segment Start (Base)        Segment End (Base + Limit)
           │                              │
           ▼                              ▼
    ┌──────┴──────────────────────────────┴──────┐
    │         Valid Address Range                │
    │    Offsets 0 to Limit are accessible       │
    └────────────────────────────────────────────┘
           ▲                              ▲
           │                              │
    Lowest Valid                  Highest Valid
      Offset = 0                   Offset = Limit

Expand-Down Semantics (E=1):

Valid offset range: (limit + 1) to (max_offset) where max_offset depends on D/B flag

    ┌────────────────────────────────────────────────┐
    │   Invalid Range (0 to Limit)                  │
    ├────────────────────────────────────────────────┤
    │   Valid Range (Limit+1 to MaxOffset)           │
    │   Stack pointer starts high, grows down        │
    └────────────────────────────────────────────────┘
         ▲                                    ▲
         │                                    │
    Stack can grow              Stack top
    down to Limit+1             (high address)

Why Expand-Down for Stacks?

Stacks traditionally grow from high memory addresses toward low addresses. When a function is called, the stack pointer decreases to allocate stack frame space. When the function returns, the stack pointer increases.

With an expand-down segment:

1. Initial stack pointer is at the maximum offset (segment top)
2. As the stack grows, the pointer decreases
3. If the stack grows too large, the pointer drops below (limit + 1), triggering a fault
4. The OS can then grow the stack by decreasing the limit value

Stack Growth Scenario:

Initial State:
  Limit = 0xFFFF0000 (with G=1, allowing ~64KB stack)
  Valid range: 0xFFFF0001 to 0xFFFFFFFF
  Stack pointer (ESP) = 0xFFFFFFF0
  
After Heavy Function Calls:
  Stack pointer (ESP) = 0xFFFF1000
  Still within valid range—OK
  
Stack Overflow Attempt:
  Stack pointer tries to go to 0xFFFEFFFF
  Offset 0xFFFEFFFF < limit (0xFFFF0000) + 1
  FAULT! General Protection Exception triggered
  
OS Grows Stack:
  OS allocates more memory, decreases limit to 0xFFFE0000
  Valid range now: 0xFFFE0001 to 0xFFFFFFFF
  Fault handler returns, instruction retries successfully

The CPU performs bounds checking on every memory access through a segment. This checking happens in hardware, adds negligible overhead (it's part of the address translation pipeline), and cannot be bypassed by software.

Bounds Check Logic:

For expand-up segments (most common):

if (offset > effective_limit)
    raise GeneralProtectionFault(#GP)
else
    proceed with memory access

For expand-down segments (stacks):

if (offset <= effective_limit || offset > max_offset)
    raise GeneralProtectionFault(#GP)
else
    proceed with memory access

Operand Size Considerations:

The check must account for the size of the data being accessed. A 4-byte read at offset 0xFFFC in a segment with limit 0xFFFF would access bytes 0xFFFC, 0xFFFD, 0xFFFE, 0xFFFF—all valid. But a 4-byte read at offset 0xFFFD would try to access 0xFFFD through 0x10000, with 0x10000 exceeding the limit.

The Check:

if (offset + operand_size - 1 > effective_limit)
    raise #GP

Checking at Multiple Levels:

Modern x86 systems often have both segmentation and paging enabled. Bounds checking happens at both levels:

1. Segment Check: Does the offset exceed the segment limit?
2. Page Check: Is the linear address (after segment translation) mapped to a valid page with appropriate permissions?

Both checks must pass. A segment might allow access to linear address 0x40001000, but if that page is marked not-present, a page fault occurs—and vice versa.

Performance of Bounds Checking:

Bounds checking is pipelined into address translation. The comparison between offset and limit happens in parallel with other address calculation steps. On modern CPUs, it adds zero additional cycles to memory access latency—the check is truly free.

This is a fundamental advantage of hardware protection: unlike software bounds checking (which adds instructions), hardware checking is invisible to performance.

When a bound check fails, the CPU raises an exception. The specific exception depends on the context, but most commonly it's a General Protection Fault (#GP, Interrupt 13) or a Stack Fault (#SS, Interrupt 12).

Exception Classifications:

Exception Frame Information:

When a limit exception occurs, the CPU pushes the following onto the kernel stack:

┌─────────────────────────────────────┐  Higher Addresses
│           SS (if privilege change) │
├─────────────────────────────────────┤
│          ESP (if privilege change) │
├─────────────────────────────────────┤
│               EFLAGS                │
├─────────────────────────────────────┤
│                CS                   │
├─────────────────────────────────────┤
│               EIP (return address)  │
├─────────────────────────────────────┤
│     Error Code (selector or 0)      │  ← Pushed for #GP, #SS
└─────────────────────────────────────┘  Stack Pointer (ESP) after push

The EIP points to the instruction that caused the fault, allowing the handler to analyze or retry.

Handler Responsibilities:

Unlike code segments which are typically fixed-size, data segments (heaps) and stack segments often need to grow during program execution. The limit field enables this growth—the OS simply increases (or for expand-down, decreases) the limit value.

Heap Growth (Expand-Up):

When a program calls malloc() and the current heap is full, the runtime requests more memory from the OS (via sbrk() or mmap()):

1. Program calls sbrk(increment) or mmap() for more heap
2. OS checks if segment can grow (enough physical memory, no collision)
3. OS allocates physical pages for the new region
4. OS increases the segment limit: new_limit = old_limit + increment
5. Return to program; malloc() now has more space

Stack Growth (Expand-Down):

Stack growth is often demand-driven—the stack faults when it needs to grow:

1. Function call pushes data, stack pointer decreases
2. Access to new stack location faults (offset <= limit for expand-down)
3. Stack fault handler (#SS) invoked
4. Handler checks if growth is within stack limit (ulimit)
5. Handler allocates physical pages for new stack region
6. Handler decreases the expand-down limit to allow new accesses
7. Return from handler; faulting instruction retries successfully

Protection During Growth:

The OS must carefully validate growth requests:

1. Resource Limits: Is the process within its memory quota (ulimit)?
2. Physical Availability: Is there enough RAM or swap space?
3. Collision Detection: Would growth overlap with another segment?
4. Security Checks: Is this a legitimate growth or an attack?

Automatic vs. Explicit Growth:

• Stack Growth: Usually automatic via fault handling (on-demand)
• Heap Growth: Explicit via system calls (sbrk, mmap)
• Code Growth: Typically not grown dynamically (fixed at load time)

Segment limits are a powerful security mechanism, providing containment of memory errors and preventing certain classes of exploits. Understanding their security implications is crucial for systems security.

Buffer Overflow Containment:

In a properly segmented system, a buffer overflow in a data segment cannot:

• Overwrite code (code is in a different segment)
• Corrupt the stack (stack is in a separate segment)
• Access kernel memory (kernel segments have different privilege)

The damage is contained within the single segment, greatly limiting exploit potential.

Attack Prevention:

Limitations of Segment-Based Protection:

While segment limits are powerful, they have limitations:

1. Intra-Segment Corruption: Limits don't prevent corrupting other data within the same segment. A heap buffer overflow can corrupt other heap objects.

2. Coarse Granularity: Segments are large. Fine-grained protection (per-object) requires additional mechanisms.

3. Flat Model Bypass: Modern systems using flat memory models (all segments base=0, limit=4GB) get minimal protection from segment limits—paging provides the actual protection.

4. No Use-After-Free Protection: Limits don't detect accessing memory that's been freed but is still within the segment.

Modern Security Layers:

Segment limits are now just one layer in a defense-in-depth stack:

• Page-level protection (NX, SMEP, SMAP)
• ASLR (randomizes memory layout)
• Stack Canaries (detect stack smashing)
• CFI (Control Flow Integrity)
• Memory Sanitizers (debug-time detection)
• Hardware enforcement (MTE, PAC on ARM64)

When the kernel needs to access user-space memory (e.g., reading syscall arguments), it must verify that user-provided pointers are within valid segment bounds. This prevents confused deputy attacks where user code tricks the kernel into accessing its own protected memory.

The Confused Deputy Problem:

A user provides a pointer to read(fd, buf, count). If the kernel blindly accesses that pointer:

• User could provide a pointer to kernel memory
• Kernel (running at Ring 0) would be able to access it
• User thus reads kernel data through the kernel

The solution: verify the pointer is within user segment limits before access.

User Pointer Verification:

Kernel/User Split:

Historically, on x86 32-bit systems, the address space was often split:

• User space: 0x00000000 - 0xBFFFFFFF (3 GB)
• Kernel space: 0xC0000000 - 0xFFFFFFFF (1 GB)

The user data segment would have:

• Base = 0
• Limit = 0xBFFFFFFF (or corresponding page-granular value)

Any attempt by user code to access kernel addresses would exceed the limit and fault. The kernel could access anywhere (flat 4GB segment with Ring 0).

SMAP/SMEP (Modern x86):

On modern x86 systems, hardware features like SMAP (Supervisor Mode Access Prevention) and SMEP (Supervisor Mode Execution Prevention) add another layer: they prevent the kernel from accidentally accessing user memory unless explicit override is set. This protects against confused deputy attacks even in flat memory models.

The segment limit field is a cornerstone of memory protection. Let's consolidate the key concepts we've explored:

What's Next:

With base and limit covered, we'll now explore protection bits—the segment descriptor fields that control read, write, and execute permissions. Protection bits determine not just where you can access, but what operations you can perform, completing the trifecta of segment-based memory protection.