Loading content...
Imagine a parking lot with 100 spaces. Fifty cars are parked, scattered throughout the lot, leaving 50 empty spaces. A tour bus needing 20 contiguous spaces arrives—but the 50 empty spaces are dispersed in groups of 3, 5, 7, and smaller clusters. Despite having 50% vacancy, the bus cannot park. This is external fragmentation.
In memory management, external fragmentation occurs when free memory is distributed in non-contiguous holes throughout the address space. The system may have plenty of total free memory, but no single contiguous region large enough to satisfy a request. The memory is free, it's just in the wrong places.
External fragmentation is the defining challenge of variable-partition memory allocation. Unlike internal fragmentation (waste within allocations), external fragmentation represents waste in the free pool—memory that exists but cannot be used. Understanding and managing external fragmentation is essential for any system that allocates variable-sized memory regions.
By the end of this page, you will understand external fragmentation's causes and mechanics, why it's inevitable in variable-partition systems, how to measure and quantify fragmentation severity, and strategies including compaction and allocation algorithms that combat its effects.
External fragmentation occurs when the total free memory is sufficient to satisfy a memory request, but no single contiguous block is large enough. Free memory exists external to allocated regions—in the gaps between them—but is fragmented into unusable pieces.
Formal Definition:
External Fragmentation exists when:
Σ(Free Blocks) ≥ Request SizebutMax(Free Block Size) < Request Size
In other words:
The memory request fails despite adequate total resources.
Why 'External'?
The fragmentation is external because the wasted space lies outside of any allocation—in the free list, between allocated regions. Contrast this with internal fragmentation, where waste is inside allocations.
The Fundamental Problem:
External fragmentation is intrinsic to variable-partition allocation. When processes of different sizes arrive and depart in unpredictable order:
Unlike internal fragmentation (bounded by partition/page size), external fragmentation can grow unboundedly. In pathological cases, a system could fail to allocate 1MB despite having 100MB free—if that 100MB is scattered across thousands of tiny holes.
External fragmentation is a consequence of good allocation—matching process sizes exactly. Systems that avoid internal fragmentation (through exact-fit allocation) inevitably suffer external fragmentation. Systems that avoid external fragmentation (through fixed-size allocation) inevitably suffer internal fragmentation. This trade-off is fundamental.
External fragmentation accumulates through the natural lifecycle of memory allocation. Let's trace how a pristine memory space degrades into fragmented chaos:
| Step | Action | Memory State | Holes | Largest Hole |
|---|---|---|---|---|
| 1 | Initial state | 100 KB free | 1 hole: 100 KB | 100 KB |
| 2 | Allocate P1 (30 KB) | P1(30) + 70 free | 1 hole: 70 KB | 70 KB |
| 3 | Allocate P2 (20 KB) | P1(30) + P2(20) + 50 free | 1 hole: 50 KB | 50 KB |
| 4 | Allocate P3 (25 KB) | P1 + P2 + P3(25) + 25 free | 1 hole: 25 KB | 25 KB |
| 5 | Free P2 | P1 + [20 free] + P3 + 25 free | 2 holes: 20+25 KB | 25 KB |
| 6 | Allocate P4 (15 KB) | P1 + P4(15) + [5 free] + P3 + 25 free | 2 holes: 5+25 KB | 25 KB |
| 7 | Free P1 | [30 free] + P4 + [5 free] + P3 + 25 free | 3 holes: 30+5+25 KB | 30 KB |
| 8 | Allocate P5 (28 KB) | P5(28) + [2 free] + P4 + [5 free] + P3 + 25 free | 3 holes: 2+5+25 KB | 25 KB |
After step 8, we have 32 KB free total, but cannot allocate 26 KB because our largest hole is only 25 KB. This is external fragmentation in action.
Key Observations:
The Fragmentation Ratchet:
With typical workloads, fragmentation tends to worsen over time:
Without intervention (compaction, special allocation strategies), external fragmentation approaches a steady-state that wastes substantial memory.
In severe fragmentation, small allocations succeed (filling tiny holes), but large allocations fail. This biases the remaining holes toward being even smaller. The system can reach a state where significant memory is free but only tiny allocations can succeed—a self-reinforcing spiral.
Understanding external fragmentation requires visualizing how memory looks after extended operation. Let's examine the structure of fragmented memory in detail.
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134135136137138139140141142143144145146147148149150151152153154155156157158159160161162163164165166167168169170171172173174175176177178179180181182183184185186187188189190191192193194195196197198199200201
#include <stdio.h>#include <stdlib.h>#include <stdbool.h>#include <string.h> #define MEMORY_SIZE 1024 // 1024 units of memory#define MAX_PROCESSES 100 typedef struct { int start; int size; bool allocated; int process_id;} MemoryBlock; typedef struct { MemoryBlock blocks[MAX_PROCESSES * 2]; int block_count; int total_memory;} MemoryManager; void init_memory(MemoryManager* mm) { mm->block_count = 1; mm->total_memory = MEMORY_SIZE; mm->blocks[0] = (MemoryBlock){0, MEMORY_SIZE, false, -1};} // Visualize current memory statevoid visualize_memory(MemoryManager* mm) { printf("|"); for (int i = 0; i < mm->block_count; i++) { char fill = mm->blocks[i].allocated ? '#' : '.'; int display_size = mm->blocks[i].size / 16; // Scale down if (display_size < 1) display_size = 1; for (int j = 0; j < display_size; j++) { printf("%c", fill); } printf("|"); } printf("");} // Calculate fragmentation statisticsvoid analyze_fragmentation(MemoryManager* mm) { int total_free = 0; int hole_count = 0; int largest_hole = 0; int smallest_hole = MEMORY_SIZE; for (int i = 0; i < mm->block_count; i++) { if (!mm->blocks[i].allocated) { total_free += mm->blocks[i].size; hole_count++; if (mm->blocks[i].size > largest_hole) largest_hole = mm->blocks[i].size; if (mm->blocks[i].size < smallest_hole) smallest_hole = mm->blocks[i].size; } } printf("Fragmentation Analysis:"); printf(" Total Free Memory: %d units", total_free); printf(" Number of Holes: %d", hole_count); printf(" Largest Hole: %d units", largest_hole); printf(" Smallest Hole: %d units", hole_count > 0 ? smallest_hole : 0); printf(" Average Hole Size: %.1f units", hole_count > 0 ? (float)total_free / hole_count : 0); // Calculate external fragmentation ratio if (total_free > 0) { float frag_ratio = 1.0 - (float)largest_hole / total_free; printf(" External Fragmentation Ratio: %.2f%%", frag_ratio * 100); printf(" (Unusable free memory due to fragmentation)"); }} // First-fit allocationbool allocate_first_fit(MemoryManager* mm, int size, int pid) { for (int i = 0; i < mm->block_count; i++) { if (!mm->blocks[i].allocated && mm->blocks[i].size >= size) { // Split block if necessary if (mm->blocks[i].size > size) { // Shift blocks to make room for new one for (int j = mm->block_count; j > i + 1; j--) { mm->blocks[j] = mm->blocks[j-1]; } mm->block_count++; // Create new hole after allocation mm->blocks[i+1] = (MemoryBlock){ mm->blocks[i].start + size, mm->blocks[i].size - size, false, -1 }; mm->blocks[i].size = size; } mm->blocks[i].allocated = true; mm->blocks[i].process_id = pid; return true; } } return false; // Allocation failed} // Free memory and coalesce adjacent holesvoid free_memory(MemoryManager* mm, int pid) { for (int i = 0; i < mm->block_count; i++) { if (mm->blocks[i].process_id == pid) { mm->blocks[i].allocated = false; mm->blocks[i].process_id = -1; // Coalesce with next block if also free if (i + 1 < mm->block_count && !mm->blocks[i+1].allocated) { mm->blocks[i].size += mm->blocks[i+1].size; // Shift blocks down for (int j = i + 1; j < mm->block_count - 1; j++) { mm->blocks[j] = mm->blocks[j+1]; } mm->block_count--; } // Coalesce with previous block if also free if (i > 0 && !mm->blocks[i-1].allocated) { mm->blocks[i-1].size += mm->blocks[i].size; // Shift blocks down for (int j = i; j < mm->block_count - 1; j++) { mm->blocks[j] = mm->blocks[j+1]; } mm->block_count--; } break; } }} int main() { MemoryManager mm; init_memory(&mm); printf("=== External Fragmentation Demonstration ==="); printf("Memory Size: %d units", MEMORY_SIZE); printf("Legend: # = allocated, . = free"); // Simulate allocations printf("Initial state:"); visualize_memory(&mm); allocate_first_fit(&mm, 200, 1); allocate_first_fit(&mm, 150, 2); allocate_first_fit(&mm, 100, 3); allocate_first_fit(&mm, 180, 4); allocate_first_fit(&mm, 120, 5); printf("After allocating P1(200), P2(150), P3(100), P4(180), P5(120):"); visualize_memory(&mm); // Deallocate some processes (creating holes) free_memory(&mm, 2); // Free P2 (150 units) free_memory(&mm, 4); // Free P4 (180 units) printf("After freeing P2 and P4:"); visualize_memory(&mm); // Try to allocate something that doesn't fit any single hole bool success = allocate_first_fit(&mm, 250, 6); printf("Attempt to allocate P6(250): %s", success ? "SUCCESS" : "FAILED"); visualize_memory(&mm); analyze_fragmentation(&mm); // Continue fragmentation allocate_first_fit(&mm, 80, 7); allocate_first_fit(&mm, 50, 8); free_memory(&mm, 3); free_memory(&mm, 5); allocate_first_fit(&mm, 60, 9); printf("After more operations:"); visualize_memory(&mm); analyze_fragmentation(&mm); return 0;}The Checkerboard Effect:
After extended operation, memory resembles a checkerboard:
|####|..|#####|...|##|.|#######|....|###|..|#|
Each . segment is a free hole; each # segment is an allocated region. The holes are:
Coalescing:
When adjacent allocated regions both become free, their holes merge:
Before: |###|...|###| (two 3-unit holes)
After P2 frees: |...|...|###| → |......|###| (one 6-unit hole)
Automatic coalescing helps slow fragmentation but cannot eliminate it when non-adjacent regions free.
Proper memory allocators always coalesce adjacent free blocks during deallocation. This requires maintaining metadata (size, allocation status) for each block, typically in a header preceding the data area. Without coalescing, fragmentation accelerates dramatically.
Quantifying external fragmentation helps assess system health and make allocation policy decisions. Several metrics capture different aspects of fragmentation:
1. Basic Fragmentation Ratio:
Fragmentation Ratio = 1 - (Largest Free Block / Total Free Memory)
2. Fragmentation Index (Alternative):
Fragmentation Index = 1 - (Largest Allocatable Size / Total Free Memory)
This is more practical—it measures what can actually be allocated versus total free.
3. Hole Count and Distribution:
| Fragmentation Ratio | Interpretation | Likely Symptoms | Recommended Action |
|---|---|---|---|
| 0.00 - 0.20 | Minimal | System healthy, large allocations succeed | Normal operation |
| 0.20 - 0.40 | Low | Occasional large allocation delays | Monitor, consider preventive compaction |
| 0.40 - 0.60 | Moderate | Large allocations may fail, system slowing | Schedule compaction, review allocation patterns |
| 0.60 - 0.80 | High | Significant requests failing, memory under-utilized | Compaction needed, consider algorithm change |
| 0.80 - 1.00 | Severe | Most allocations fail despite free memory | Immediate compaction, system redesign needed |
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134135136137138139140141142143144145146
#include <stdio.h>#include <stdlib.h>#include <math.h> typedef struct { size_t* hole_sizes; size_t hole_count; size_t total_memory; size_t allocated_memory;} MemoryState; typedef struct { double basic_ratio; double fragmentation_index; size_t largest_hole; size_t smallest_hole; double average_hole; double std_deviation; size_t hole_count; size_t total_free; double usable_percent; // Percent of free memory actually usable} FragmentationMetrics; FragmentationMetrics analyze_fragmentation(MemoryState* state) { FragmentationMetrics m = {0}; if (state->hole_count == 0) { m.fragmentation_index = 0; // All memory allocated (or none) return m; } // Basic statistics m.hole_count = state->hole_count; m.total_free = 0; m.largest_hole = 0; m.smallest_hole = (size_t)-1; for (size_t i = 0; i < state->hole_count; i++) { size_t hole = state->hole_sizes[i]; m.total_free += hole; if (hole > m.largest_hole) m.largest_hole = hole; if (hole < m.smallest_hole) m.smallest_hole = hole; } m.average_hole = (double)m.total_free / m.hole_count; // Standard deviation double variance = 0; for (size_t i = 0; i < state->hole_count; i++) { double diff = state->hole_sizes[i] - m.average_hole; variance += diff * diff; } m.std_deviation = sqrt(variance / m.hole_count); // Fragmentation ratios m.basic_ratio = 1.0 - ((double)m.largest_hole / m.total_free); m.fragmentation_index = m.basic_ratio; // Same for external frag // Usable percentage (how much can we actually allocate?) // This depends on request distribution, but largest hole is key m.usable_percent = 100.0 * m.largest_hole / m.total_free; return m;} void print_metrics(FragmentationMetrics* m) { printf("=== External Fragmentation Metrics ==="); printf("Total Free Memory: %zu bytes", m->total_free); printf("Number of Holes: %zu", m->hole_count); printf("Largest Hole: %zu bytes", m->largest_hole); printf("Smallest Hole: %zu bytes", m->smallest_hole); printf("Average Hole: %.1f bytes", m->average_hole); printf("Std Deviation: %.1f bytes", m->std_deviation); printf("Fragmentation Ratio: %.2f (%.0f%%)", m->basic_ratio, m->basic_ratio * 100); printf("Usable Free Memory: %.1f%%", m->usable_percent); printf("Wasted (Fragmented): %.1f%%", 100 - m->usable_percent); // Severity assessment printf("Severity: "); if (m->basic_ratio < 0.2) printf("MINIMAL - System healthy"); else if (m->basic_ratio < 0.4) printf("LOW - Monitor situation"); else if (m->basic_ratio < 0.6) printf("MODERATE - Consider compaction"); else if (m->basic_ratio < 0.8) printf("HIGH - Compaction recommended"); else printf("SEVERE - Immediate action required");} int main() { // Example: A fragmented memory system size_t holes[] = {150, 32, 85, 12, 200, 45, 8, 18}; size_t num_holes = sizeof(holes) / sizeof(holes[0]); MemoryState state = { .hole_sizes = holes, .hole_count = num_holes, .total_memory = 2048, .allocated_memory = 1498 // 2048 - sum(holes) }; printf("Memory Configuration:"); printf("Total Memory: %zu bytes", state.total_memory); printf("Allocated: %zu bytes", state.allocated_memory); printf("Holes: "); for (size_t i = 0; i < num_holes; i++) { printf("%zu ", holes[i]); } printf(""); FragmentationMetrics m = analyze_fragmentation(&state); print_metrics(&m); // Demonstrate what allocations would succeed/fail printf("=== Allocation Feasibility ==="); size_t test_sizes[] = {50, 100, 150, 200, 250, 300}; for (size_t i = 0; i < sizeof(test_sizes)/sizeof(test_sizes[0]); i++) { printf("Request %zu bytes: %s", test_sizes[i], test_sizes[i] <= m.largest_hole ? "POSSIBLE" : "BLOCKED by fragmentation"); } return 0;}Production systems monitor fragmentation continuously. Linux exposes fragmentation info through /proc/buddyinfo and /proc/pagetypeinfo. Database systems track page fragmentation percentages. When thresholds are crossed, automated compaction or defragmentation triggers.
External fragmentation emerges from fundamental characteristics of dynamic memory allocation. Understanding the causes helps design systems that minimize fragmentation.
The Entropy Analogy:
External fragmentation resembles entropy in thermodynamics:
Just as heat naturally disperses and never spontaneously reconcentrates, memory holes naturally scatter and rarely spontaneously merge into large contiguous regions. Compaction is the 'work' required to fight memory entropy.
Studies of real workloads show that fragmentation typically stabilizes at 20-50% of free memory being unusable under steady-state conditions. Pathological workloads (e.g., many small allocations interspersed with occasional large ones) can push this above 80%.
Several strategies address external fragmentation, each with distinct trade-offs:
| Strategy | Description | Pros | Cons |
|---|---|---|---|
| Compaction | Move allocations to collect free space into one contiguous hole | Eliminates fragmentation completely | Expensive (CPU, I/O), requires relocatable addresses |
| Paging | Divide memory into fixed-size pages; any page can satisfy any request | Eliminates external fragmentation entirely | Introduces internal fragmentation, page table overhead |
| Best-Fit Allocation | Choose the smallest hole that fits, minimizing leftover fragments | Reduces remnant sizes | May create many tiny unusable holes |
| Buddy System | Allocate in power-of-2 blocks; easy merging of adjacent buddies | Efficient coalescing, limited fragmentation | High internal fragmentation (power-of-2 rounding) |
| Pool Allocation | Separate pools for different size classes | Excellent for known patterns | Complex management, may waste memory if pools imbalanced |
| Slab Allocation | Pre-allocate object caches for common sizes | Near-zero fragmentation for cached objects | Only works for known, repeated allocations |
Compaction in Detail:
Compaction physically moves allocated regions to consolidate free space. This is conceptually simple but operationally complex:
Compaction Algorithm:
Requirements:
Why Paging Solves External Fragmentation:
Paging divides both physical memory (into frames) and virtual memory (into pages) into equal-sized units. Because every frame is interchangeable:
This is why paging became the dominant memory management technique. The cost (internal fragmentation in the last page) is bounded and predictable, unlike external fragmentation which can grow arbitrarily.
Modern operating systems use paging for process memory (avoiding external fragmentation there) but still face external fragmentation in kernel memory allocators. The Linux kernel uses the buddy system combined with slab allocators to manage this—a hybrid approach that handles both large and small allocations efficiently.
Understanding the relationship between external and internal fragmentation is crucial for memory system design. They represent different manifestations of allocation inefficiency and often trade off against each other.
| Aspect | External Fragmentation | Internal Fragmentation |
|---|---|---|
| Location of Waste | Between allocations (in free pool) | Within allocations (in allocated blocks) |
| Cause | Variable-sized allocations leave holes | Fixed-size blocks exceed actual need |
| Visibility | Free memory exists but unusable | Allocated memory partially unused |
| Measurement | 1 - (largest_hole / total_free) | (allocated - used) / allocated |
| Boundedness | Unbounded (can approach 100%) | Bounded by allocation unit size |
| Recovery | Compaction or clever allocation | Cannot recover without reallocation |
| Allocation Scheme | Variable partitions, segments | Fixed partitions, paging |
| Eliminated By | Paging (fixed-size allocation) | Perfect-fit allocation |
The Fundamental Trade-Off:
Variable-Size Allocation:
✓ Zero internal fragmentation (exact fit)
✗ External fragmentation accumulates
Fixed-Size Allocation:
✓ Zero external fragmentation (any slot works)
✗ Internal fragmentation inevitable
Why This Trade-Off Exists:
With variable-sized allocation:
With fixed-sized allocation:
The Paging Compromise:
Paging represents the industry's answer to this dilemma:
No allocation scheme eliminates both internal and external fragmentation. Attempts to minimize one typically increase the other. The art of memory management is choosing which fragmentation type is acceptable for your workload and constraints.
We've explored the mechanics, measurement, and mitigation of external fragmentation. Let's consolidate the key insights:
What's Next:
Having understood both internal and external fragmentation, we now examine a classic result that quantifies expected fragmentation: the 50-percent rule. This empirical observation provides insight into how much memory is wasted under typical allocation patterns and helps explain why fragmentation is an inevitable cost of dynamic memory management.
You now understand external fragmentation—the challenge of scattered free memory that plagues variable-partition systems. This knowledge explains why paging became dominant despite its internal fragmentation cost, and why systems needing variable-sized allocations must employ sophisticated strategies to remain usable.