Main Memory (RAM) — The Primary System Memory

Main memory, commonly referred to as RAM (Random Access Memory), is the primary workspace of a computer system. It holds the operating system kernel, running applications, and their data during execution. While caches provide speed and storage provides capacity, main memory occupies the critical middle ground—large enough to hold active workloads, fast enough to keep the CPU reasonably fed.

From an operating system perspective, main memory is one of the most precious and carefully managed resources. The OS must:

• Track which memory regions are free and which are in use
• Allocate memory to processes and reclaim it when no longer needed
• Implement virtual memory to present a uniform address space to each process
• Balance memory pressure across competing applications

Understanding how RAM works at the hardware level is foundational to understanding all of these OS functions.

Main memory in nearly all modern computers uses DRAM (Dynamic Random Access Memory), a technology that stores each bit as a charge in a tiny capacitor. Understanding DRAM's fundamental operation explains many of its performance characteristics and why it behaves differently from caches.

The DRAM cell:

Each DRAM cell consists of:

1. One transistor: Acts as a switch, connecting the capacitor to a bitline
2. One capacitor: Stores charge representing a bit value (charged = 1, discharged = 0)

This "1T1C" design is extremely compact—the smallest practical memory cell—which is why DRAM achieves high densities at low cost. Compare this to SRAM (cache), which uses 6 transistors per bit.

The problem with capacitors:

Capacitors leak charge over time. Left alone, a DRAM cell would lose its stored value within milliseconds. This creates two critical consequences:

Sensing the bit:

DRAM cells are arranged in a 2D grid of rows and columns. Reading a cell involves:

1. Precharge: The bitline (column wire) is set to a reference voltage (typically VDD/2)
2. Row activation: The wordline for the target row is raised, connecting all cells in that row to their bitlines. The tiny charge difference tips the bitline voltage up or down.
3. Sensing: A sense amplifier detects the tiny voltage difference and amplifies it to a full logic level
4. Restoration: The amplified value is written back to the cell, restoring its charge

This process takes tens of nanoseconds—orders of magnitude slower than an SRAM read. The sense amplifiers act as a row buffer, holding the entire activated row (typically 8KB) for subsequent column accesses.

Modern memory systems are hierarchically organized to maximize bandwidth and parallelism while managing physical constraints. Understanding this organization is essential for understanding memory performance characteristics.

Memory hierarchy (from CPU outward):

Address mapping example:

When the memory controller receives a physical address, it decodes it into:

• Channel bits: Select which memory channel
• Rank bits: Select which rank on that channel
• Bank bits: Select which bank within the rank
• Row bits: Select which row within the bank
• Column bits: Select which column within the activated row
• Byte offset: Select which byte within the burst transfer

The exact bit positions vary by system and can significantly impact performance. Interleaving lower address bits across channels/banks improves parallelism for sequential accesses.

DDR SDRAM (Double Data Rate Synchronous DRAM) is the dominant memory technology in computers. "Double data rate" means data is transferred on both the rising and falling edges of the clock signal, effectively doubling bandwidth compared to single data rate (SDR) memory at the same clock frequency.

Each DDR generation roughly doubles bandwidth through higher clock speeds and architectural improvements, while the underlying DRAM cell technology remains similar.

Key evolutionary improvements:

Prefetch architecture: Each DDR generation increases the prefetch width—the number of bits fetched from the memory array in a single internal access. DDR5 prefetches 16n bits (16 bits per data pin per access). Higher prefetch enables higher external data rates while keeping internal array speeds manageable.

Bank groups: DDR4 introduced bank groups—clusters of banks that can serve back-to-back requests faster than banks in different groups. This helps maintain high bandwidth for interleaved access patterns.

DDR5 innovations:

• Dual 32-bit channels per DIMM: Each DIMM now has two independent channels, doubling command/address bandwidth
• Decision feedback equalization (DFE): Enables higher data rates by correcting signal distortion
• Same-bank refresh: Allows other banks to operate while one bank refreshes
• On-DIMM power management ICs (PMICs): More precise voltage regulation

DRAM operations are governed by precise timing requirements. Understanding these timings helps explain why memory access patterns dramatically affect performance and why the memory controller is so complex.

The fundamental timing parameters:

Access latency scenarios:

Row buffer hit (best case):

• The requested row is already activated
• Latency = tCL (just column access)
• Typical: ~14 ns

Row buffer miss (row already activated, needs different row):

• Must precharge current row, then activate new row
• Latency = tRP + tRCD + tCL
• Typical: ~40-50 ns

Row buffer closed (no row activated):

• Must activate the row, then access column
• Latency = tRCD + tCL
• Typical: ~28-35 ns

Impact on programming:

Sequential memory access achieves row buffer hits—only paying tCL for each cache line after the first. Random access within a large region pays full tRP + tRCD + tCL for most accesses. This 3-4× latency difference is why access patterns matter so much.

The memory controller is the hardware unit that translates CPU memory requests into DRAM commands (activate, precharge, read, write, refresh). Modern memory controllers are integrated into the CPU die and implement sophisticated scheduling algorithms to maximize performance.

Memory controller responsibilities:

Command scheduling policies:

FCFS (First Come First Served): Process requests in arrival order. Simple but poor performance—ignores row buffer locality.

FR-FCFS (First Ready - First Come First Served): Prioritize row buffer hits over misses, using FCFS as a tiebreaker. Much better performance but can starve requests to different rows if one row is hot.

ATLAS (Adaptive per-Thread Least-Attained-Service): Tracks how much service each thread has received; prioritizes under-served threads. Improves fairness in multi-core systems.

BLISS (Blacklisting: Throttle memory-intensive threads): Identifies and temporarily de-prioritizes memory-hogging threads to prevent them from blocking others. Improves quality of service for latency-sensitive workloads.

Row buffer policies:

The operating system must manage physical memory as a precious, finite resource. The OS doesn't see DRAM cells and timing parameters—it sees a contiguous range of physical addresses that must be partitioned, tracked, and allocated efficiently.

Physical address space layout:

Not all physical addresses correspond to RAM. The physical address space includes:

• Main memory regions: Actual installed DRAM
• Memory-mapped I/O: Regions mapped to device registers
• Reserved regions: BIOS, ACPI tables, system management RAM
• PCI holes: Address ranges assigned to PCI devices

The BIOS/UEFI provides a memory map to the OS at boot time, describing which regions are usable RAM, reserved, or memory-mapped I/O.

Page-based memory management:

The OS manages physical memory in fixed-size units called page frames (typically 4 KB on x86). The page frame allocator tracks which frames are:

• Free: Available for allocation
• Allocated to kernel: Used by the OS itself
• Allocated to processes: Mapped into user virtual address spaces
• Used by page cache: Caching file data
• Reserved: Hardware-reserved or unavailable

Key data structures:

Memory pressure and reclamation:

When physical memory runs low, the OS must reclaim pages. Strategies include:

1. Page cache eviction: Discard cached file data (can be re-read from disk)
2. Swap out: Write anonymous pages (heap, stack) to swap device, freeing physical frames
3. OOM killer: As a last resort, terminate processes to free memory

The OS continuously balances page cache size (for I/O performance) against free memory (for allocation headroom). The kswapd daemon in Linux proactively reclaims pages when free memory drops below thresholds.

NUMA (Non-Uniform Memory Access) describes architectures where memory access time depends on which processor accesses which memory. In NUMA systems, each processor (or group of processors) has "local" memory that it can access faster than "remote" memory attached to other processors.

Why NUMA exists:

As core counts increased, memory bandwidth became a bottleneck. If all cores shared a single memory controller, that controller becomes a chokepoint. NUMA distributes memory controllers across sockets, providing:

• Scalable bandwidth: Each socket has dedicated memory bandwidth
• Reduced interconnect traffic: Local accesses don't traverse inter-socket links
• But: Software must be aware of locality to benefit

OS NUMA support:

Operating systems expose NUMA topology to applications and implement NUMA-aware policies:

Allocation policies:

• Local allocation: Allocate memory on the same node as the requesting CPU (default)
• Interleaved: Round-robin across nodes (useful for shared data structures)
• Bind: Force allocation to a specific node
• Preferred: Try one node first, fall back to others

Process scheduling:

• Keep processes on the same NUMA node as their memory
• Migrate pages if a process moves to a different node (memory migration)
• Balance load while respecting locality constraints

Linux tools:

• numactl: Run programs with specific NUMA policies
• numastat: Display NUMA memory statistics
• /sys/devices/system/node/: NUMA topology information
• mbind(), set_mempolicy(): System calls for memory policies