Encryption at Rest - Learning Module

Loading content...

0/273

Performance Considerations

The Security-Performance Tradeoff (And Why It's Often Overstated)

"We can't enable encryption—it will kill our performance."

This concern, raised in countless architecture reviews, reflects a persistent misconception. While encryption does have performance implications, modern hardware and optimized implementations have reduced this overhead to the point where encryption is rarely a bottleneck in well-designed systems.

The reality:

Modern CPUs include hardware acceleration for AES encryption (AES-NI)
Self-encrypting drives (SEDs) provide encryption with zero CPU overhead
Cloud providers encrypt by default with minimal measurable impact
For most workloads, I/O latency and network time dwarf encryption overhead

That said, performance does matter, and encryption is not entirely free. This page provides the knowledge to accurately measure encryption overhead, optimize where necessary, and make informed tradeoffs between security and performance.

What You Will Learn

By the end of this page, you will understand where encryption overhead comes from, how to accurately measure it, hardware acceleration technologies that minimize impact, optimization strategies for high-performance systems, and how to make principled tradeoffs when performance constraints genuinely conflict with encryption requirements.

Understanding Encryption Overhead

Encryption overhead manifests in several ways. Understanding each component helps target optimization efforts effectively.

Components of encryption overhead:

Encryption Overhead Components
Component	Description	Typical Impact	Optimization
CPU computation	Cycles to encrypt/decrypt data	1-10% CPU with AES-NI	Hardware acceleration
Memory bandwidth	Moving data through CPU for encryption	Minimal with AES-NI	Streaming encryption
Key operations	Fetching, deriving, caching keys	< 1ms per operation	Key caching
KMS latency	Network round-trip to KMS for key operations	5-50ms per call	Envelope encryption, DEK caching
I/O amplification	Encrypted data may be larger (IVs, auth tags)	< 1% size increase	Minimize metadata overhead
Initialization	Setting up cipher contexts, sessions	Microseconds	Context reuse

Where encryption overhead is noticeable:

High-throughput data pipelines — Streaming terabytes per hour, encryption CPU adds up
Low-latency applications — Every microsecond matters; encryption adds some
Key-intensive operations — Encrypting many small files/records each with unique keys
Legacy systems without AES-NI — Older CPUs without hardware acceleration
Application-level encryption — Encrypting/decrypting in application code rather than infrastructure

Where encryption overhead is negligible:

I/O-bound workloads — Disk/network are the bottleneck, not CPU
Web applications — Network latency dwarfs encryption time
Infrequent large operations — Encrypting one large backup is fast
Hardware-accelerated infrastructure — Modern CPUs, SEDs, cloud native encryption

AES-NI Performance Reality

Intel's AES-NI instructions, present in nearly all modern x86 CPUs since 2010, can encrypt data at over 10 GB/s per core. For comparison, enterprise NVMe SSDs max out around 7 GB/s. This means AES-NI can encrypt data faster than most storage can provide it. In practice, encryption is almost never the bottleneck on modern hardware.

Measuring Encryption Performance

Accurate measurement is essential before optimizing. Many teams assume encryption is a problem without measuring, or measure incorrectly and draw wrong conclusions.

Principles for accurate measurement:

Measurement Best Practices

•Measure in production-like conditions — Synthetic benchmarks often show worst-case or unrealistic scenarios. Test with your actual data patterns.
•Isolate the variable — Run the same workload with and without encryption. Change nothing else.
•Measure what matters — Throughput, latency, CPU utilization, not just raw encryption speed.
•Include warm-up — Key caches populate, JIT compiles; initial performance differs from steady-state.
•Test at scale — Overhead may manifest differently at 100 ops/sec vs. 100,000 ops/sec.
•Measure percentiles, not averages — P99 latency often reveals overhead hidden by averages.

Encryption Benchmarking
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
# Quick encryption performance check using OpenSSL
 
# Check if AES-NI is available
openssl version -a | grep -i aes
grep -o aes /proc/cpuinfo | head -1
 
# Benchmark AES-256-GCM (recommended for most uses)
openssl speed -evp aes-256-gcm
 
# Sample output on modern CPU:
# type             16 bytes     64 bytes    256 bytes   1024 bytes   8192 bytes  16384 bytes
# aes-256-gcm     615.8M/s    1985.6M/s    5026.8M/s    8267.4M/s   10521.2M/s  10714.9M/s
 
# That's ~10.7 GB/s for large blocks - faster than most SSDs!
 
# Compare with non-accelerated (disable AES-NI)
OPENSSL_ia32cap="~0x200000200000000" openssl speed -evp aes-256-gcm
# You'll see 10-50x slower without hardware acceleration
 
# Benchmark ChaCha20-Poly1305 (ARM-friendly alternative)
openssl speed -evp chacha20-poly1305
 
# Benchmark RSA operations (for asymmetric key operations)
openssl speed rsa2048 rsa4096
 
# === Database encryption benchmarks ===
 
# PostgreSQL with pgbench (compare encrypted vs. unencrypted)
# Create test database on encrypted volume
pgbench -i -s 100 testdb_encrypted
 
# Run benchmark
pgbench -c 10 -j 4 -t 10000 testdb_encrypted
 
# Compare with same test on unencrypted volume
# Measure: TPS, latency, CPU usage
 
# === File encryption benchmarks ===
 
# Create test file
dd if=/dev/urandom of=testfile bs=1M count=1000
 
# Time file encryption with AES-256-GCM
time openssl enc -aes-256-gcm -in testfile -out testfile.enc -k "password"
 
# Time decryption
time openssl enc -d -aes-256-gcm -in testfile.enc -out testfile.dec -k "password"
 
# Clean up
rm testfile testfile.enc testfile.dec

The KMS Latency Reality

KMS API calls typically take 5-50ms per call. If you call KMS for every record, this becomes the dominant overhead—not encryption itself. This is why envelope encryption (generate DEK once, cache it, use for many operations) is essential for performance. A cached DEK means encryption is local and fast; KMS calls are rare.

Hardware Acceleration Technologies

Modern hardware includes specialized features that make encryption dramatically faster. Understanding and leveraging these is key to high-performance encrypted systems.

Major hardware acceleration technologies:

Hardware Encryption Acceleration
Technology	Platform	Description	Performance Gain
AES-NI	Intel/AMD x86	Dedicated AES instructions in CPU	10-50x faster than software AES
ARMv8 Crypto Extensions	ARM (mobile, AWS Graviton)	AES, SHA in ARM processors	10-20x faster than software
VAES	Intel (10th gen+), AMD Zen 3+	Vectorized AES for AVX-512	2x improvement over AES-NI
Self-Encrypting Drives (SED)	Storage	Encryption in drive controller	Zero CPU overhead
Intel QAT	Intel (server)	Dedicated crypto accelerator card	High-throughput bulk encryption
NVIDIA GPUs	NVIDIA	GPU-accelerated cryptography	Parallel encryption workloads

Ensuring hardware acceleration is active:

On Linux:

Verifying Hardware Acceleration
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
# Check for AES-NI support in CPU
grep -o aes /proc/cpuinfo | head -1
# Output: aes  (if supported)
 
# Check feature flags (more detail)
cat /proc/cpuinfo | grep -m1 flags | tr ' ' '\n' | grep -E 'aes|sha'
 
# Verify OpenSSL is using AES-NI
openssl engine
# Look for "aesni" engine
 
# Check dm-crypt is using hardware acceleration
sudo cryptsetup benchmark
# AES-XTS with 256-bit key should show multi-GB/s speeds
 
# Check if kernel crypto modules are using AES-NI
lsmod | grep aesni
# Output should show aesni_intel
 
# Test with and without AES-NI (for comparison)
# With AES-NI:
openssl speed -evp aes-256-gcm
 
# Without AES-NI (for comparison only):
OPENSSL_ia32cap="~0x200000200000000" openssl speed -evp aes-256-gcm
 
# === ARM systems ===
 
# Check ARMv8 crypto extensions
cat /proc/cpuinfo | grep -i features
# Look for: aes pmull sha1 sha2
 
# === Cloud instance types with best crypto performance ===
# AWS: Any instance with "Intel Xeon" or "AWS Graviton2/3"
# GCP: N2, N2D, C2, C2D instances (Ice Lake/Milan CPUs)
# Azure: Dv4, Ev4, and newer series

VMs and Hardware Passthrough

In virtualized environments, CPU feature flags might be hidden from VMs. Ensure your hypervisor passes through AES-NI (most modern ones do by default). In Docker/containers, CPU features are typically inherited from the host—no special configuration needed. Check CPU flags inside your actual runtime environment, not just the host.

Optimization Strategies

When encryption performance matters, these strategies can minimize overhead while maintaining security.

Critical optimizations:

High-Impact Optimizations

•Cache Data Encryption Keys (DEKs) — Generate DEKs via KMS once, cache locally (in memory or secure storage), reuse for many encryption operations. Drastically reduces KMS calls.
•Use envelope encryption — Encrypt data locally with cached DEKs; only use KMS for DEK management. Local encryption is ~1000x faster than KMS round-trips.
•Batch operations — When encrypting many items, amortize setup costs by processing in batches rather than individually.
•Stream large data — Don't load entire files into memory. Use streaming ciphers for large data to avoid memory pressure.
•Choose appropriate cipher modes — AES-GCM is fast with authentication. AES-CTR is fast without (but you need separate HMAC). Avoid CBC for new designs.
•Parallelize encryption — AES-GCM can be parallelized for multi-core benefits on large data.

Performance Optimization Examples
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
import { LRUCache } from 'lru-cache';
import { KMSClient, GenerateDataKeyCommand, DecryptCommand } from '@aws-sdk/client-kms';
 
interface CachedDEK {
  plaintextKey: Buffer;
  encryptedKey: Buffer;
  createdAt: number;
}
 
class DEKCache {
  private cache: LRUCache<string, CachedDEK>;
  private kms: KMSClient;
  private keyId: string;
  private maxAgeMs: number;
  
  constructor(kmsKeyId: string, maxAgeMs: number = 3600000) {  // 1 hour default
    this.kms = new KMSClient({ region: 'us-east-1' });
    this.keyId = kmsKeyId;
    this.maxAgeMs = maxAgeMs;
    
    this.cache = new LRUCache({
      max: 100,  // Max cached DEKs
      ttl: maxAgeMs,
      dispose: (value) => {
        // Zero out key on eviction
        value.plaintextKey.fill(0);
      },
    });
  }
  
  async getDEK(scope: string): Promise<{ key: Buffer; encryptedKey: Buffer }> {
    const cacheKey = `dek:${scope}`;
    
    // Check cache first
    const cached = this.cache.get(cacheKey);
    if (cached) {
      return {
        key: cached.plaintextKey,
        encryptedKey: cached.encryptedKey,
      };
    }
    
    // Generate new DEK via KMS
    const response = await this.kms.send(new GenerateDataKeyCommand({
      KeyId: this.keyId,
      KeySpec: 'AES_256',
      EncryptionContext: { scope },  // Bind to scope
    }));
    
    if (!response.Plaintext || !response.CiphertextBlob) {
      throw new Error('Failed to generate DEK');
    }
    
    const dek: CachedDEK = {
      plaintextKey: Buffer.from(response.Plaintext),
      encryptedKey: Buffer.from(response.CiphertextBlob),
      createdAt: Date.now(),
    };
    
    this.cache.set(cacheKey, dek);
    
    return {
      key: dek.plaintextKey,
      encryptedKey: dek.encryptedKey,
    };
  }
  
  // Call this on graceful shutdown
  clear(): void {
    this.cache.clear();  // dispose() will zero keys
  }
}
 
// Usage: Create cache once, reuse for all operations
const dekCache = new DEKCache(process.env.KMS_KEY_ID!);
 
async function encryptRecord(data: Buffer): Promise<EncryptedData> {
  // DEK is cached - no KMS call in most cases
  const { key, encryptedKey } = await dekCache.getDEK('records');
  
  // Local encryption is fast
  const iv = randomBytes(12);
  const cipher = createCipheriv('aes-256-gcm', key, iv);
  const ciphertext = Buffer.concat([cipher.update(data), cipher.final()]);
  const authTag = cipher.getAuthTag();
  
  return {
    ciphertext,
    iv,
    authTag,
    encryptedDek: encryptedKey,  // Store for future decryption via KMS
  };
}

The 80/20 Rule

DEK caching and envelope encryption typically eliminate 80%+ of encryption-related latency by removing KMS round-trips. If you do nothing else, implement DEK caching. It transforms encryption from a network-bound operation (10-50ms) to a CPU-bound operation (microseconds).

Database Encryption Performance

Database encryption (TDE) is a common concern for performance. Here's what to expect and how to optimize.

Typical TDE overhead:

TDE Performance Impact by Database
Database	Typical Overhead	CPU Impact	Notes
SQL Server TDE	0-5% on reads, 2-10% on writes	Low with AES-NI	Backup compression affected
Oracle TDE	2-8% typical	Minimal with HW accel	Column encryption higher overhead
PostgreSQL (volume encryption)	0-5%	Depends on storage	No native TDE; use volume/disk
MySQL InnoDB Encryption	3-10%	Low with AES-NI	Buffer pool caches decrypted
AWS RDS Encrypted	< 5% typically	Managed by AWS	AWS claims ~5% latency increase
Azure SQL TDE	2-5%	Hardware accelerated	Enabled by default

Factors affecting TDE performance:

Buffer pool/cache effectiveness — Decrypted pages are cached in memory. High cache hit rates mean fewer decrypt operations.
I/O patterns — Random I/O means more decrypt operations. Sequential scans amortize overhead.
Write-heavy vs. read-heavy — Writes always require encryption. Reads may hit cache (no encryption needed).
Column encryption vs. full TDE — Column-level encryption operates on individual values, potentially many times per query. TDE encrypts/decrypts pages, which is more efficient.
Query patterns — Queries that can use indexes effectively touch less encrypted data than full table scans.

TDE Performance Tips

•Maximize buffer pool/cache size
•Use SSDs to minimize I/O latency
•Optimize queries to minimize page reads
•Consider read replicas for read-heavy workloads
•Monitor CPU—TDE is CPU-bound once I/O is fast

TDE Performance Traps

•Undersized buffer pools force more decryption
•Column encryption on frequently accessed columns
•Encrypted columns in WHERE clauses (can't use indexes)
•Backup without compression (encrypted data doesn't compress)
•Inadequate CPU for encryption-heavy workloads

Real-World Observation

In most production deployments, TDE overhead is lost in the noise of normal performance variation. I/O latency, query optimization, and index effectiveness typically have 10-100x more impact on database performance than encryption. Enable TDE; then optimize the things that actually matter.

Storage Encryption Performance

Full-disk and volume encryption are commonly questioned for performance impact. Here's the reality across different implementations.

Disk/Volume encryption overhead by implementation:

Storage Encryption Performance
Technology	Overhead	CPU Impact	Best For
Self-Encrypting Drive (SED)	0%	None	Maximum performance, any workload
BitLocker (hardware mode)	< 1%	Minimal	Windows with compatible hardware
BitLocker (software mode)	5-10%	Moderate	Windows without SED
LUKS/dm-crypt (AES-NI)	2-5%	Low	Linux servers and desktops
FileVault 2	3-7%	Low	macOS with T2/M1 chip
AWS EBS Encryption	< 5%	Managed	AWS workloads
eCryptfs (file-level)	10-25%	High per-file	Home directory encryption

Understanding storage encryption performance:

Why SEDs have zero CPU overhead: Self-encrypting drives perform encryption in the drive controller. The CPU never sees unencrypted data (for purposes of encryption—it sees unencrypted data for actual processing). The drive handles everything.

Why software encryption overhead varies:

CPU/AES-NI — With hardware acceleration, encryption is fast enough that I/O becomes the bottleneck
Cipher mode — Some modes (XTS) are optimal for disk encryption; others add overhead
Key derivation — Initial unlock may be slow (intentionally, for security); ongoing operation is fast
Implementation quality — LUKS/dm-crypt are highly optimized; other solutions may not be

NVMe SSD performance with encryption: Modern NVMe SSDs can exceed 7 GB/s read and 5 GB/s write. AES-NI can encrypt at 10+ GB/s. This means encryption is not the bottleneck even for the fastest consumer storage. For typical enterprise SSDs (3-4 GB/s), the gap is even larger.

Storage Benchmark Comparison
Linux Benchmarks
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
#!/bin/bash
# Compare encrypted vs. unencrypted storage performance
 
# Variables
TEST_FILE="/path/to/testfile"
TEST_SIZE="4G"
ITERATIONS=3
 
echo "=== Storage Encryption Performance Test ==="
echo ""
 
# Test 1: Unencrypted partition
echo "Testing unencrypted partition..."
dd if=/dev/zero of=/mnt/unencrypted/test bs=1M count=4096 conv=fdatasync 2>&1 | tail -1
rm /mnt/unencrypted/test
 
# Test 2: LUKS encrypted partition
echo "Testing LUKS encrypted partition..."
dd if=/dev/zero of=/mnt/encrypted/test bs=1M count=4096 conv=fdatasync 2>&1 | tail -1
rm /mnt/encrypted/test
 
# Test 3: fio benchmark (more comprehensive)
echo ""
echo "=== fio Sequential Write Test ==="
echo ""
 
# Unencrypted
fio --name=seqwrite --rw=write --bs=1M --size=4G \
    --directory=/mnt/unencrypted --runtime=30 --output-format=json | \
    jq '.jobs[0].write.bw_bytes / 1024 / 1024 | "Unencrypted: \(.) MB/s"'
 
# Encrypted
fio --name=seqwrite --rw=write --bs=1M --size=4G \
    --directory=/mnt/encrypted --runtime=30 --output-format=json | \
    jq '.jobs[0].write.bw_bytes / 1024 / 1024 | "Encrypted: \(.) MB/s"'
 
echo ""
echo "=== fio Random Read Test (IOPS) ==="
echo ""
 
# Unencrypted
fio --name=randread --rw=randread --bs=4k --size=1G --numjobs=4 \
    --directory=/mnt/unencrypted --runtime=30 --output-format=json | \
    jq '.jobs[].read.iops | "Unencrypted IOPS: \(.)"'
 
# Encrypted
fio --name=randread --rw=randread --bs=4k --size=1G --numjobs=4 \
    --directory=/mnt/encrypted --runtime=30 --output-format=json | \
    jq '.jobs[].read.iops | "Encrypted IOPS: \(.)"'
 
# Test 4: Check dm-crypt stats
echo ""
echo "=== dm-crypt Statistics ==="
dmsetup status /dev/mapper/encrypted_volume
cat /sys/block/dm-0/stat

SSD Trim with Encryption

TRIM/discard support improves SSD performance and longevity by informing the drive which blocks are no longer in use. LUKS 2 supports TRIM passthrough (allow-discards option), but be aware this leaks some information about which blocks are used. For most use cases, the performance benefit outweighs the minimal information leakage.

Making Principled Performance Tradeoffs

Despite the generally low overhead of modern encryption, there are cases where tradeoffs must be made. Here's how to think about them.

When encryption overhead genuinely matters:

•Ultra-low-latency trading systems — Microseconds matter; even AES-NI adds some latency
•Video encoding/transcoding — CPU-intensive; encryption competes for cycles
•Real-time data streaming — Multi-gigabit streams with strict latency requirements
•Resource-constrained embedded systems — Limited CPU, no hardware acceleration
•Very small data volumes at high frequency — Encryption setup overhead dominates

Decision framework for tradeoffs:

Measure first — Don't assume. Benchmark with encryption on and off.
Quantify the requirement — 'Performance matters' is not actionable. 'We need < 1ms P99 latency' is.
Assess the sensitivity — Not all data is equal. Public data needs less protection than PII.
Consider alternatives:
- Hardware encryption (SEDs) for zero-overhead full disk
- Offload encryption to dedicated hardware (QAT, HSM)
- Selective encryption (encrypt only sensitive fields)
- Architecture changes (encrypt at ingest, serve unencrypted from cache)
Document the decision — If you choose reduced encryption for performance, document the risk acceptance.

Performance-Sensitive Encryption Strategies
Performance Requirement	Strategy	Security Trade-off
Low latency (< 1ms)	SED for storage; cache decrypted in memory	Cache hit = unencrypted access from RAM
High throughput (GB/s)	Hardware acceleration; parallel encryption	Minimal trade-off with modern hardware
CPU constrained	Selective encryption; encrypt only sensitive data	Less data protected
Real-time streaming	ChaCha20-Poly1305 (faster on some platforms)	Equivalent security; different algorithm
Legacy systems	Encrypt at rest only; skip in-transit for internal	Internal network exposure

Avoiding False Trade-offs

Many 'performance concerns' about encryption are solved by proper implementation, not by reducing security. Before accepting any security trade-off: (1) Verify hardware acceleration is active, (2) Implement DEK caching, (3) Benchmark accurately, (4) Consider hardware upgrades. Often, a modest hardware investment eliminates the perceived trade-off entirely.

Summary: Performance Considerations

We've examined the performance implications of encryption at rest and strategies to minimize overhead. Let's consolidate the key insights:

Key Takeaways

•Encryption overhead is often overstated — With AES-NI, encryption is faster than most storage. For typical workloads, overhead is < 5%.
•Hardware acceleration is essential — Verify AES-NI/ARMv8 crypto is active. Without it, overhead is 10-50x higher.
•KMS latency is the real bottleneck — DEK caching and envelope encryption eliminate this by making operations local.
•Measure before optimizing — Benchmark in production-like conditions. Many 'performance issues' are assumptions, not facts.
•Self-encrypting drives offer zero CPU overhead — For performance-critical storage, SEDs provide encryption 'for free.'
•Database TDE overhead is typically 2-10% — Buffer pool effectiveness matters more than encryption speed.
•Optimize the right things — I/O, queries, and architecture typically have 10-100x more performance impact than encryption.
•Document tradeoffs carefully — If reduced security is accepted for performance, document the risk acceptance formally.

Module complete:

You've now completed the comprehensive exploration of encryption at rest. You understand why encryption at rest is critical, how to encrypt databases and file systems, how to manage encryption keys properly, and how to optimize for performance without compromising security.

These skills form the foundation for protecting data wherever it persists—in databases, on disks, in cloud storage, and in backup systems. As you design systems, remember: encryption at rest is not optional—it's a fundamental requirement for any system handling sensitive data.

Module Complete

Congratulations! You've completed the Encryption at Rest module. You now understand the full spectrum of data protection at rest—from the foundational 'why' through database and file system encryption, key management, and performance optimization. You're equipped to design and implement encryption strategies appropriate for any security and compliance requirement.