Access Times - The Physics of Storage Performance

When you execute a database query and wait for results, that wait time is composed of countless smaller delays—each governed by the laws of physics and the constraints of engineering. Understanding access time at a fundamental level transforms vague notions of "fast" and "slow" into precise, actionable knowledge.

Every nanosecond of storage access is accounted for. Light traveling through fiber optic cable covers about 20 centimeters per nanosecond. Electrons propagating through copper move at roughly 60-70% of light speed. A magnetic disk platter spinning at 7,200 RPM completes one rotation in 8.33 milliseconds. These physical constants are immutable—no amount of software optimization can transcend them.

This page dissects access time at each tier of the memory hierarchy, revealing the physics behind the numbers. When you understand why an HDD access takes 10 milliseconds while a cache access takes 1 nanosecond, you can predict performance behavior, identify bottlenecks, and make informed architectural decisions.

Storage access time is not a single monolithic quantity—it's the sum of multiple components, each with its own physical origin. Understanding these components is essential for diagnosing performance issues and optimizing system behavior.

The Fundamental Equation:

Total Access Time = Command Overhead + Seek/Address Time + Transfer Time + Protocol Overhead

Let's define each component:

Latency vs. Throughput:

Two distinct metrics characterize storage performance:

• Latency: Time from request to response. Measures how long you wait for a single operation. Critical for interactive workloads.
• Throughput: Data volume per unit time. Measures aggregate bandwidth. Critical for batch processing.

These metrics don't always correlate:

Queue Depth and Parallelism:

Modern storage systems can process multiple requests concurrently. The queue depth is the number of outstanding requests. Higher queue depths can mask latency by keeping the device busy:

Effective Throughput = IOPS × Transfer Size
IOPS = 1 / Latency (for QD=1)
IOPS = Queue Depth / Latency (for higher QD, up to device limits)

An SSD with 100μs latency achieves 10,000 IOPS at queue depth 1. At queue depth 32, it can achieve 300,000+ IOPS if the internal parallelism supports it.

At the top of the memory hierarchy, CPU registers and caches operate at speeds difficult to comprehend. Understanding their timing illuminates why databases invest heavily in cache-efficient algorithms.

CPU Registers:

Registers are the fastest possible storage—integral parts of the CPU execution pipeline:

• Location: On the CPU die, within the execution core
• Size: ~1KB total (typically 16-32 general-purpose registers of 64 bits each)
• Access time: 0 cycles (registers are part of the instruction's operands)
• Technology: Made of SRAM flip-flops, no separate memory access needed

Registers do not have "access time" in the traditional sense—they are inputs and outputs of CPU instructions, not addressable memory.

L1 Cache:

The L1 cache is the first true memory level, sitting closest to the CPU cores:

L2 Cache:

The L2 cache serves as a larger, slightly slower backing store for L1:

L3 Cache (Last-Level Cache):

Shared across all cores, L3 is the last line of defense before main memory:

Why Cache Latency Matters for Databases:

Query processing involves billions of memory references per second. Consider a hash join probing a hash table:

• Table fits in L1: ~1ns per probe → 1 billion probes/second
• Table fits in L3: ~15ns per probe → 65 million probes/second
• Table in RAM: ~100ns per probe → 10 million probes/second
• Table on SSD: ~100μs per probe → 10,000 probes/second

The difference between L1-resident and RAM-resident is 100x. Cache-conscious algorithms can achieve dramatic speedups by ensuring that hot data structures fit in cache.

Main memory access involves a complex choreography between the CPU's memory controller and the DRAM modules. Understanding this process reveals why DRAM latency has improved slowly compared to CPU speeds.

The Memory Access Sequence:

1. Address Decode: The memory controller decodes the address to identify channel, rank, bank, row, and column.

2. Row Activation (tRCD): If the target row is not already active, it must be opened:

   • The row address is sent to the bank
   • The entire row (8KB) is read into the row buffer
   • This takes 13-18ns (tRCD: Row-to-Column Delay)

3. Column Read (tCAS): The column address selects the target bytes:

   • Bytes are read from the row buffer
   • Takes 13-19ns (CAS latency, or CL)

4. Data Transfer: Data travels from DRAM to CPU:

   • DDR transfers data on both clock edges
   • 64 bytes (burst length of 8 × 8 byte bus width)

5. Precharge (tRP): To access a different row in the same bank:

   • Current row must be closed and precharged
   • Takes 13-18ns (tRP: Row Precharge Time)

*Absolute latency = CL × clock period. Higher CL at higher frequencies results in similar absolute latencies.

Access Time Scenarios:

Why DRAM Is "Slow":

DRAM uses capacitors, not transistors, to store data. This has consequences:

1. Charge sensing is slow: Reading a capacitor requires sensing tiny voltage differences
2. Refresh is mandatory: Capacitors leak; every cell must be refreshed every 64ms
3. Destructive reads: Reading a DRAM cell destroys its contents; it must immediately be rewritten
4. Row buffer width: The entire row loads on activation—wasteful for random access

Implications for Databases:

DRAM access patterns significantly affect the buffer pool and query processing:

• Sequential scans benefit from row buffer locality
• Random index probes cause row buffer conflicts
• NUMA systems add complexity—memory local to a CPU is faster than remote memory
• Memory channel interleaving distributes accesses but complicates locality analysis

SSD access times are dominated by the characteristics of NAND flash memory and the overhead of the Flash Translation Layer (FTL). Unlike RAM's relatively predictable timing, SSD latency varies significantly based on the operation type, queue depth, and device state.

NAND Flash Operation Times:

Why Read Latency Varies:

NAND flash reads involve:

1. Address Decode: ~1μs to decode the target page
2. Charge Sensing: Reading voltage levels from floating gates
   • SLC: 1 bit, 2 levels → fast, simple threshold
   • QLC: 4 bits, 16 levels → complex, multiple read voltages needed
3. Error Correction: ECC decodes the raw data (longer for worn cells)
4. Data Transfer: Moving page data to controller buffer

Controller Overhead:

Beyond raw NAND timing, SSD controllers add latency:

Total observed latency = NAND time + controller overhead = 25-100μs for reads

Read vs. Write Asymmetry:

Writes are inherently slower than reads on NAND flash:

• Read: Sense voltage levels—passive operation
• Write (Program): Inject electrons into floating gates—requires high voltage, multiple verify cycles
• Erase: Remove electrons from entire block—requires even higher voltage, very slow

Queue Depth Effects:

SSDs contain multiple NAND dies operating in parallel. Higher queue depths exploit this parallelism:

Database workloads should target the sweet spot—enough concurrency to achieve high throughput without latency degradation.

NVMe vs. SATA:

NVMe SSDs are strictly superior for database workloads, especially OLTP with high concurrency.

Hard disk drive access times are dominated by mechanical motion. Understanding the physics of Head movement and platter rotation explains why HDD latency is orders of magnitude higher than electronic storage.

The Three Components of HDD Access Time:

HDD Access Time = Seek Time + Rotational Latency + Transfer Time

Let's analyze each in detail.

Seek Time Details:

The actuator arm uses a voice coil motor—essentially a speaker coil—to move the heads. The seek has three phases:

1. Acceleration: Apply current to accelerate the arm toward the target
2. Coast/Travel: The arm moves across tracks (may coast with minimal power)
3. Deceleration/Settle: Precisely position the head and wait for vibrations to dampen

Seek distance affects time non-linearly:

Sequential vs. Random Performance:

For sequential access, seek and rotational latency are incurred only once. Subsequent sectors stream continuously:

Database Design Implications:

• Append-only logs: Redo/undo logs should be on separate spindles for sequential writes
• Tablespace layout: Separate filegroups for tables and indexes, placed on different drives
• Defragmentation: Fragmented files cause random seeks; regular maintenance helps
• RAID stripe size: Match stripe size to typical I/O size for sequential benefit

Modern database deployments increasingly involve network-attached storage, cloud block storage, and distributed storage systems. Understanding network latency components is essential for architecting distributed databases.

Components of Network Storage Latency:

Network Storage Latency = Software Stack + Network Transit + Remote Storage Access

Storage Area Network (SAN) Latency:

Enterprise databases often use Fibre Channel or iSCSI SANs:

Cloud Block Storage:

Cloud providers offer block storage with varying characteristics:

Cloud latency includes:

• Virtualization overhead (hypervisor)
• Virtual network overhead
• Storage service processing
• Multi-tenant interference

RDMA and Kernel Bypass:

For ultra-low-latency storage access, modern systems use:

RDMA (Remote Direct Memory Access):

• Bypasses CPU and OS kernel
• Direct NIC-to-memory transfers
• Latency: 2-5 μs for small reads
• Used in high-performance databases and distributed storage

NVMe-oF (NVMe over Fabrics):

• NVMe protocol over RDMA or TCP
• Near-local SSD latency over network
• Latency: 20-50 μs (RDMA), 100-200 μs (TCP)

Database Architecture Impact:

• Synchronous replication: Add 2× network latency to commit time
• Distributed transactions: Two-phase commit adds multiple round-trips
• Read replicas: Query routing can add network latency
• Placement optimization: Co-locate compute with storage, minimize network hops

Understanding theoretical access times is valuable, but production systems require measurement. Monitoring storage latency reveals bottlenecks, validates configurations, and enables capacity planning.

Key Metrics to Monitor:

Measurement Tools:

Operating System Level:

Example iostat Output Interpretation:

Device  r/s    w/s   rMB/s  wMB/s  avgrq-sz  avgqu-sz  await  svctm  %util
nvme0  12500  8000      49     31      8.0       1.2    0.06   0.04   82%

• await (0.06ms): Average total request time including queue wait
• svctm (0.04ms): Average service time (device busy time)
• %util (82%): Approaching saturation—latency will increase with more load

Database-Level Monitoring: