Thin Provisioning — Storage Virtualization | OneNoughtOne

0/131

Operating SystemsStorage Management

Storage Virtualization

LevelAdvanced

Duration90 mins

TopicStorage Management

5 / 5

Thin Provisioning

Storage on Demand

Traditional storage allocation follows a simple rule: you request 100GB, you consume 100GB—whether you use it or not. This thick provisioning model wastes storage when volumes are larger than their actual data, which is almost always the case. The average enterprise server uses only 40-60% of its allocated storage capacity. Thin provisioning revolutionizes this model by decoupling virtual capacity from physical allocation. You can create a 1TB thin volume that initially consumes only kilobytes of actual storage. Space is allocated automatically, on-demand, as data is actually written. A 1TB thin volume containing 100GB of data uses only ~100GB of pool capacity—not 1TB. This enables over-commitment : you can provision more virtual capacity than you have physical storage, betting that not all volumes will fill simultaneously. A 500GB thin pool might serve 2TB of virtual volumes, provided actual data usage stays within the physical limit. This mirrors how airlines oversell flights, banks lend more than their reserves, and clouds over-provision compute resources.

What You Will Learn

By the end of this page, you will understand thin pool architecture and design, thin volume creation and management, over-commitment strategies and risk management, thin pool auto-extension and monitoring, discard/TRIM support for space reclamation, the relationship between thin volumes and snapshots, production deployment best practices, and failure modes and recovery procedures.

Thick vs. Thin Provisioning

Understanding the fundamental difference between provisioning models is essential for making informed storage design decisions. Thick Provisioning (Traditional): When you create a 100GB thick logical volume: 100GB of physical extents are immediately allocated from the VG VG free space decreases by 100GB The volume can use up to 100GB Even if you write only 1GB of data, 100GB remains allocated Other volumes cannot use the 99GB of empty space Thin Provisioning: When you create a 100GB thin volume: Only metadata is allocated (kilobytes) Pool free space barely changes The volume appears as 100GB to the file system As you write data, blocks are allocated from the pool on-demand Actual pool usage matches actual data written Other volumes can use pool space not yet claimed

Converting Mermaid diagram...

Thick vs. Thin Provisioning Comparison
Aspect	Thick Provisioning	Thin Provisioning
Allocation timing	Immediate, at creation	On-demand, at first write
Space efficiency	Low (empty space wasted)	High (only used space consumed)
Over-commitment	Not possible	Core capability
Performance	No allocation overhead	Minor allocation overhead
Snapshots	CoW with pre-allocated space	Pool-shared, highly efficient
Complexity	Simple	Requires pool management
Risk	None (space guaranteed)	Pool exhaustion possible
Best for	Predictable, dense workloads	Variable, sparse workloads

The Over-Commitment Risk

Thin provisioning trades guaranteed capacity for efficiency. If all thin volumes fill simultaneously and exhaust the pool, writes will fail—potentially causing application errors, data corruption, or system crashes. Thin provisioning requires active monitoring and capacity management that thick provisioning does not.

Thin Pool Architecture

A thin pool is a special logical volume that serves as a container for thin volumes. Understanding its internal structure is essential for proper sizing and management. Thin Pool Components: A thin pool consists of two sub-LVs: Data LV (tdata) : Stores the actual data blocks for all thin volumes in the pool Metadata LV (tmeta) : Stores the mapping between thin volume blocks and data LV blocks When you create a thin pool, LVM automatically creates these hidden sub-LVs. The metadata LV is critical—its corruption or exhaustion can make the entire pool inaccessible.

Thin Pool Structure
Bash
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
#!/bin/bash
# Understanding Thin Pool Structure
 
# Create a thin pool
lvcreate --type thin-pool -L 100G -n thin_pool vg_storage
 
# View the pool and its components
lvs -a vg_storage
 
# Output:
#   LV                  VG         Attr       LSize   Pool   ...
#   thin_pool           vg_storage twi-a-t--- 100.00g
#   [thin_pool_tdata]   vg_storage Twi-ao---- 100.00g
#   [thin_pool_tmeta]   vg_storage ewi-ao----   1.00g
 
# The [brackets] indicate hidden LVs
# thin_pool is the visible pool
# thin_pool_tdata is the data storage (100GB)
# thin_pool_tmeta is the metadata (auto-sized, ~1GB)
 
# === METADATA SIZE REQUIREMENTS ===
 
# Metadata LV size depends on:
# - Pool size
# - Block size (chunk size)
# - Number of thin volumes
# - Snapshot chains
 
# Rule of thumb: 1GB metadata per ~1TB of pool data
# Minimum: 2MB
# Maximum: 16GB
 
# === EXPLICIT METADATA SIZING ===
 
# Create pool with specific metadata size
lvcreate --type thin-pool -L 500G \
    --poolmetadatasize 4G \
    -n large_pool vg_storage
 
# Or create data and metadata LVs separately
lvcreate -L 500G -n pool_data vg_storage
lvcreate -L 4G -n pool_meta vg_storage
lvconvert --type thin-pool \
    --poolmetadata vg_storage/pool_meta \
    vg_storage/pool_data
 
# === POOL CHUNK SIZE ===
 
# Chunk size determines allocation granularity
# Smaller = finer allocation, more metadata
# Larger = coarser allocation, less metadata
 
# Default: auto-selected based on pool size
# Range: 64KB to 1GB
 
# Create with explicit chunk size
lvcreate --type thin-pool -L 100G \
    --chunksize 256K \
    -n custom_pool vg_storage
 
# Check chunk size
lvs -o lv_name,chunk_size vg_storage/thin_pool
 
# === POOL PROFILE ===
 
# Use a profile for common configurations
lvcreate --type thin-pool -L 100G \
    --config 'allocation/thin_pool_chunk_size=128' \
    -n profiled_pool vg_storage

Block Allocation Mechanics: When a thin volume writes data: Write request arrives for a block not yet allocated Pool allocates a new chunk from the data LV (tdata) Metadata updated in tmeta to record the mapping Write completes to the newly allocated chunk Subsequent writes to the same block go directly to the already-allocated chunk. This is why thin provisioning is sometimes called "just-in-time" or "on-demand" allocation.

Metadata LV Importance

The metadata LV is the most critical component. If metadata is corrupted or the metadata LV fills up (separate from data LV), the entire pool becomes unusable. LVM automatically sizes metadata conservatively, but for very large pools with many snapshots, you may need to explicitly specify larger metadata. Always monitor metadata usage alongside data usage.

Creating Thin Volumes

Thin volume creation differs from regular volume creation. You create volumes within a pool, specifying virtual size independently of actual pool capacity.

Creating and Managing Thin Volumes
Bash
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
#!/bin/bash
# Creating and Managing Thin Volumes
 
# === PREREQUISITES: CREATE THIN POOL ===
 
# Create a 200GB thin pool
lvcreate --type thin-pool -L 200G -n thin_pool vg_storage
 
# === CREATING THIN VOLUMES ===
 
# Basic thin volume creation
# -V = Virtual size (can exceed pool size!)
# --thinpool = Which pool to use
lvcreate --type thin -V 500G \
    --thinpool thin_pool \
    -n thin_vol1 vg_storage
 
# Create multiple thin volumes that over-commit
lvcreate --type thin -V 200G --thinpool thin_pool -n vm_disk1 vg_storage
lvcreate --type thin -V 200G --thinpool thin_pool -n vm_disk2 vg_storage
lvcreate --type thin -V 200G --thinpool thin_pool -n vm_disk3 vg_storage
# Total virtual: 600GB in 200GB pool - 3x overcommit
 
# === ALTERNATIVE SYNTAX ===
 
# Shorter form using -T
lvcreate -T vg_storage/thin_pool -V 100G -n quick_vol
 
# Create pool and first volume in one command
lvcreate -T vg_storage/new_pool -L 100G -V 500G -n first_vol
 
# === THIN VOLUME SIZING ===
 
# Virtual size can be any value
lvcreate -T vg_storage/thin_pool -V 1T -n large_virtual   # 1TB virtual
lvcreate -T vg_storage/thin_pool -V 10T -n huge_virtual   # 10TB virtual
 
# Initially uses almost no space
lvs -o lv_name,lv_size,data_percent vg_storage
# Output:
#   LV            LSize    Data%
#   thin_pool     200.00g  0.00
#   large_virtual   1.00t  0.00
 
# === FILE SYSTEM ON THIN VOLUME ===
 
# Create filesystem
mkfs.ext4 /dev/vg_storage/thin_vol1
 
# Check actual usage after filesystem creation
lvs -o lv_name,data_percent vg_storage/thin_vol1
# Output shows ~0.1-0.5% (filesystem metadata only)
 
# Mount and use
mount /dev/vg_storage/thin_vol1 /mnt/thin1
 
# === THIN VOLUME EXTENSION ===
 
# Extend virtual size (no pool space needed!)
lvextend -L +100G /dev/vg_storage/thin_vol1
 
# Then extend filesystem
resize2fs /dev/vg_storage/thin_vol1
 
# Or combined
lvextend -L +100G -r /dev/vg_storage/thin_vol1
 
# === LISTING THIN VOLUMES ===
 
# Show thin volumes with pool information
lvs -o lv_name,lv_size,pool_lv,data_percent vg_storage
 
# Output:
#   LV         LSize   Pool       Data%
#   thin_pool  200.00g            45.50
#   thin_vol1  500.00g thin_pool  12.30
#   vm_disk1   200.00g thin_pool  66.00
#   vm_disk2   200.00g thin_pool  23.00
#   vm_disk3   200.00g thin_pool   5.00
 
# === CONVERTING EXISTING LV TO THIN ===
 
# Convert a regular LV to thin (requires pool)
# Warning: This copies all data!
lvconvert --type thin \
    --thinpool vg_storage/thin_pool \
    vg_storage/existing_regular_lv

Virtual Size Strategy

Set virtual sizes generously. Since thin volumes don't consume space until used, oversizing them provides growth room without wasting storage. A VM that might need 100GB over its lifetime can be given 500GB virtual—it costs nothing until used, and you avoid resize operations later.

Pool Monitoring and Auto-Extension

Thin pool monitoring is mandatory for production use. Pool exhaustion causes write failures, potential data corruption, and application crashes. LVM provides both monitoring tools and automatic extension capabilities. Pool Status Indicators:

Pool Monitoring and Auto-Extension
Bash
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
#!/bin/bash
# Thin Pool Monitoring and Auto-Extension
 
# === CHECKING POOL USAGE ===
 
# Basic pool status
lvs vg_storage/thin_pool
# Shows data and metadata usage percentage
 
# Detailed pool information
lvs -o lv_name,lv_size,data_percent,metadata_percent \
    vg_storage/thin_pool
 
# Output:
#   LV        LSize    Data%  Meta%
#   thin_pool 200.00g  78.50  12.30
 
# === MONITORING COMMANDS ===
 
# Continuous monitoring
watch -n 5 'lvs -o lv_name,data_percent,metadata_percent \
    vg_storage/thin_pool'
 
# All thin pool details
lvs -a -o+pool_lv,data_percent,metadata_percent vg_storage
 
# Show transaction ID (for debugging)
lvs -o lv_name,transaction_id vg_storage/thin_pool
 
# === ALERTING SCRIPT ===
 
#!/bin/bash
# thin_pool_monitor.sh - Add to cron
 
DATA_THRESHOLD=80
META_THRESHOLD=75
POOL="/dev/vg_storage/thin_pool"
 
# Get current usage
DATA_PCT=$(lvs --noheadings -o data_percent $POOL | tr -d ' ')
META_PCT=$(lvs --noheadings -o metadata_percent $POOL | tr -d ' ')
 
# Check data usage
if (( $(echo "$DATA_PCT > $DATA_THRESHOLD" | bc -l) )); then
    echo "CRITICAL: Thin pool data at ${DATA_PCT}%"
    # Send alert: mail, Slack, PagerDuty, etc.
fi
 
# Check metadata usage (often overlooked!)
if (( $(echo "$META_PCT > $META_THRESHOLD" | bc -l) )); then
    echo "CRITICAL: Thin pool metadata at ${META_PCT}%"
    # Metadata exhaustion is MORE dangerous than data exhaustion
fi
 
# === AUTO-EXTENSION CONFIGURATION ===
 
# Configure in /etc/lvm/lvm.conf:
 
# thin_pool_autoextend_threshold = 80
#   When pool reaches 80% usage, try to extend
#
# thin_pool_autoextend_percent = 20
#   Extend by 20% of current size
#
# monitoring = 1
#   Enable dmeventd monitoring (required for auto-extend)
 
# Verify dmeventd is running
systemctl status lvm2-monitor
 
# Enable monitoring for a pool explicitly
lvchange --monitor y vg_storage/thin_pool
 
# Check if monitoring is enabled
lvs -o lv_name,lv_health_status vg_storage/thin_pool
 
# === MANUAL EXTENSION ===
 
# Extend pool data (most common need)
lvextend -L +50G vg_storage/thin_pool
 
# Extend pool metadata (when metadata_percent is high)
lvextend --poolmetadatasize +1G vg_storage/thin_pool
 
# === POOL EXHAUSTION RESPONSE ===
 
# If pool is 100% and writes are failing:
 
# 1. Immediate: Extend the pool (if VG has space)
lvextend -L +50G vg_storage/thin_pool
 
# 2. If VG has no space: Add a new PV
pvcreate /dev/new_disk
vgextend vg_storage /dev/new_disk
lvextend -L +50G vg_storage/thin_pool
 
# 3. Delete/reduce thin volumes to reclaim space
lvremove vg_storage/unused_thin_vol
 
# 4. Run fstrim/discard to reclaim deleted file space
fstrim /mount/point

Pool Full = Write Failure

When a thin pool reaches 100% capacity, all writes to all thin volumes in that pool fail. Applications may crash, filesystems may become corrupted, and databases may lose transactions. Unlike classic snapshots that only affect themselves when full, a full thin pool is a system-wide emergency. Configure auto-extension and monitoring BEFORE deploying thin provisioning in production.

Space Reclamation with Discards

Thin provisioning allocates space on write, but what happens when data is deleted? By default, nothing—the pool retains allocated blocks even after files are deleted. Discard (also called TRIM) enables space reclamation, returning unused blocks to the pool. The Reclamation Problem: Consider this scenario: Create 100GB thin volume, write 80GB of data Pool usage: 80GB Delete 40GB of files Pool usage: still 80GB! The filesystem knows those blocks are free, but the thin pool doesn't. Discards bridge this gap by informing the pool which blocks are no longer needed.

Discard Configuration and Usage
Bash
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
#!/bin/bash
# Thin Pool Discard/TRIM Configuration and Usage
 
# === POOL DISCARD MODE ===
 
# Check current discard mode
lvs -o lv_name,discards vg_storage/thin_pool
 
# Discard modes:
# - ignore:   Discards are ignored (default in older versions)
# - nopassdown: Pool processes discards but doesn't pass to underlying devices
# - passdown:   Pool processes and passes to underlying devices (SSDs)
 
# Set discard mode
lvchange --discards passdown vg_storage/thin_pool
 
# Create pool with discard mode
lvcreate --type thin-pool -L 200G \
    --discards passdown \
    -n thin_pool vg_storage
 
# === MANUAL DISCARD (FSTRIM) ===
 
# Run fstrim on mounted filesystem to reclaim space
fstrim -v /mnt/thin_volume
 
# Output: /mnt/thin_volume: 45 GiB (48318382080 bytes) trimmed
 
# Verify space was reclaimed
lvs -o lv_name,data_percent vg_storage/thin_pool
 
# === AUTOMATIC DISCARD (MOUNT OPTION) ===
 
# Mount with discard option for real-time space reclamation
mount -o discard /dev/vg_storage/thin_vol1 /mnt/data
 
# In /etc/fstab:
# /dev/vg_storage/thin_vol1 /mnt/data ext4 defaults,discard 0 2
 
# WARNING: Online discard can impact write performance
# Periodic fstrim is often preferred for busy systems
 
# === PERIODIC FSTRIM SERVICE ===
 
# Most distributions include fstrim.timer
systemctl enable fstrim.timer
systemctl start fstrim.timer
 
# Check timer status
systemctl status fstrim.timer
systemctl list-timers | grep fstrim
 
# Default runs weekly; adjust if needed
 
# === DISCARD GRANULARITY ===
 
# Check minimum discard granularity
blockdev --getdiscardgranularity /dev/vg_storage/thin_vol1
 
# Should match or exceed pool chunk size for efficiency
 
# === ZEROING AND DISCARD ===
 
# Thin pool zero behavior
# When a new block is allocated, should it be zeroed?
 
# Check current setting
lvs -o lv_name,zero vg_storage/thin_pool
 
# Enable zeroing (security, but slower allocation)
lvchange --zero y vg_storage/thin_pool
 
# Disable zeroing (faster, for trusted environments)
lvchange --zero n vg_storage/thin_pool
 
# === VERIFYING DISCARD IS WORKING ===
 
# Create test file
dd if=/dev/urandom of=/mnt/thin_vol/test bs=1M count=1000
 
# Check pool usage
lvs -o data_percent vg_storage/thin_pool
# Example: 50.00
 
# Delete file
rm /mnt/thin_vol/test
 
# Without discard, usage unchanged
# With discard...
sync
sleep 2
lvs -o data_percent vg_storage/thin_pool
# Example: 45.00 (if online discard) or unchanged until fstrim

Periodic fstrim vs. Mount Discard

For most workloads, running fstrim weekly via systemd timer is preferable to the 'discard' mount option. Online discard adds latency to every delete operation, which can impact performance on busy systems. Weekly fstrim batches all discards into a single maintenance window, minimizing impact on normal operations.

Thin Provisioning with Snapshots

Thin provisioning and snapshots are deeply integrated in LVM. Thin snapshots are far more efficient than classic snapshots and offer capabilities not possible with thick provisioning. Thin Snapshot Advantages: No pre-allocation required : Snapshot size is unlimited (within pool capacity) Shared pool efficiency : Multiple snapshots share the same pool Snapshot chains : Create snapshots of snapshots to any depth Writable by default : Each snapshot can diverge independently No origin penalty : No CoW overhead on the origin volume

Thin Snapshots
Bash
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
#!/bin/bash
# Thin Snapshots - Advanced Usage
 
# === BASIC THIN SNAPSHOT ===
 
# Create snapshot of thin volume (instant, no size needed)
lvcreate -s -n snap1 vg_storage/thin_vol1
 
# Verify it's a thin snapshot
lvs -o lv_name,pool_lv,origin vg_storage/snap1
# Shows: snap1  thin_pool  thin_vol1
 
# === SNAPSHOT CHAINS ===
 
# Create snapshot of a snapshot
lvcreate -s -n snap2 vg_storage/snap1
 
# And another level
lvcreate -s -n snap3 vg_storage/snap2
 
# Check the chain
lvs -o lv_name,origin vg_storage | grep snap
# snap1  thin_vol1
# snap2  snap1
# snap3  snap2
 
# Can continue indefinitely (limited only by metadata)
 
# === WRITABLE THIN SNAPSHOTS ===
 
# Thin snapshots are writable by default
mount /dev/vg_storage/snap1 /mnt/snap1
 
# Make changes
echo "Modified in snapshot" > /mnt/snap1/test.txt
 
# Original is unaffected
mount /dev/vg_storage/thin_vol1 /mnt/original
cat /mnt/original/test.txt
# (shows original content)
 
# === EXTERNAL ORIGIN SNAPSHOTS ===
 
# Create thin snapshot of a NON-thin (thick) volume
# Origin becomes "external origin" - read-only reference
 
# First, have a regular LV
lvcreate -L 50G -n thick_vol vg_storage
 
# Create thin snapshot of it
lvcreate -s --thinpool vg_storage/thin_pool \
    -n thin_snap_of_thick \
    vg_storage/thick_vol
 
# The thick volume is now a read-only external origin
# All changes go to the thin pool
 
# === VM TEMPLATE DEPLOYMENT ===
 
# Create golden template
lvcreate -T vg_storage/thin_pool -V 100G -n vm_template
 
# Install and configure OS on template
# ...
 
# Deploy VMs as thin snapshots
for i in {1..10}; do
    lvcreate -s -n vm_instance_$i vg_storage/vm_template
done
 
# 10 VMs sharing template, each storing only differences
# Much faster than copying 100GB x 10!
 
# Check actual storage usage
lvs -o lv_name,lv_size,data_percent vg_storage
# Each vm_instance shows individual data_percent
 
# === SNAPSHOT DELETION ===
 
# Removing middle of chain
# If: original → snap1 → snap2
 
lvremove vg_storage/snap1
 
# LVM handles this - snap2 now references original
# No data loss, but reorganization happens
 
# === THIN TIME MACHINE PATTERN ===
 
# Create rotating snapshots for point-in-time recovery
#!/bin/bash
# thin_rotation.sh
 
ORIGIN="vg_storage/thin_data"
MAX_SNAPS=24  # Keep 24 hourly snapshots
 
# Create new snapshot with timestamp
SNAP_NAME="hourly_$(date +%Y%m%d_%H%M)"
lvcreate -s -n $SNAP_NAME $ORIGIN
 
# Remove old snapshots beyond retention
SNAP_COUNT=$(lvs --noheadings -o lv_name $ORIGIN 2>/dev/null | \
    grep "^  hourly_" | wc -l)
 
if [ $SNAP_COUNT -gt $MAX_SNAPS ]; then
    # Remove oldest
    OLDEST=$(lvs --noheadings --sort -lv_time -o lv_name \
        vg_storage 2>/dev/null | grep "hourly_" | tail -1 | tr -d ' ')
    lvremove -f vg_storage/$OLDEST
fi

Snapshot Chain Depth

While thin snapshot chains have no hard limit, very deep chains (50+ levels) can impact read performance as LVM must traverse the chain to find data. For production use, consider periodic 'flattening' by creating a new base thin volume from accumulated snapshots. Also note that chain depth affects metadata consumption.

Production Best Practices

Thin provisioning requires more careful planning and monitoring than thick provisioning. These practices derive from production deployments and failure analysis.

Essential Practices

•Enable dmeventd monitoring: Required for auto-extension to work
•Set auto-extend thresholds: Configure thin_pool_autoextend_threshold at 80% in lvm.conf
•Monitor metadata separately: Metadata exhaustion is more dangerous than data exhaustion
•Leave VG headroom: Auto-extend needs VG free space; reserve 20%+
•Enable discards: Configure passdown mode and run periodic fstrim
•Limit overcommit ratio: 2:1 to 4:1 max depending on workload predictability
•Alert before crisis: Set alerts at 70% to allow response time
•Document virtual vs actual: Track total virtual allocation vs pool capacity

Common Failures

•No monitoring: Pool exhausts silently, discovered when applications crash
•Ignoring metadata: Pool metadata fills before data, total failure
•Infinite overcommit: Provisioning without tracking leads to sudden capacity crises
•Disabled discards: Pool never reclaims space, perpetual growth
•No auto-extend space: VG is full, auto-extend fails, pool fails
•Relying on auto-extend alone: Burst writes can exhaust faster than extend runs
•Snapshot accumulation: Old snapshots consume pool silently
•Missing from backup scope: Pool metadata not backed up, recovery impossible

Production Configuration
Configuration
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
# /etc/lvm/lvm.conf - Production thin provisioning settings
 
activation {
    # Enable dmeventd monitoring (CRITICAL for auto-extend)
    monitoring = 1
    
    # Auto-extend threshold (percent full before extending)
    thin_pool_autoextend_threshold = 80
    
    # Auto-extend amount (percent of current size)
    thin_pool_autoextend_percent = 20
}
 
allocation {
    # Default chunk size for new thin pools (KB)
    thin_pool_chunk_size = 256
    
    # Zero new blocks (set to 0 for performance, 1 for security)
    thin_pool_zero = 0
    
    # Discard mode for new pools
    thin_pool_discards = "passdown"
}
 
# After modifying, restart LVM monitoring
# systemctl restart lvm2-monitor

Capacity Planning Formula: Recommended Pool Size = (Sum of Expected Actual Usage) \times 1.3 + (Snapshot Retention \times Daily Change Rate \times Days) + (Auto-extend Buffer: 20% of pool) Example: 5 VMs expected to use 50GB actual each = 250GB Daily change rate: 5GB/day, 7-day snapshot retention = 35GB Pool size: (250 \times 1.3) + 35 + 20% buffer = 325GB + 57GB + 76GB \approx 460GB pool With 4:1 overcommit, can provision 1.8TB virtual

Recovery Preparation

Prepare for pool exhaustion BEFORE it happens. Document the emergency procedure: which volumes can be deleted, which can be migrated to other pools, and how to quickly extend the pool. Test the procedure during maintenance windows. When a pool fills at 3 AM, you don't want to be reading documentation for the first time.

Operating SystemsStorage Management

Storage Virtualization

LevelAdvanced

Duration90 mins

TopicStorage Management

5 / 5

Thin Provisioning

Storage on Demand

Traditional storage allocation follows a simple rule: you request 100GB, you consume 100GB—whether you use it or not. This thick provisioning model wastes storage when volumes are larger than their actual data, which is almost always the case. The average enterprise server uses only 40-60% of its allocated storage capacity. Thin provisioning revolutionizes this model by decoupling virtual capacity from physical allocation. You can create a 1TB thin volume that initially consumes only kilobytes of actual storage. Space is allocated automatically, on-demand, as data is actually written. A 1TB thin volume containing 100GB of data uses only ~100GB of pool capacity—not 1TB. This enables over-commitment : you can provision more virtual capacity than you have physical storage, betting that not all volumes will fill simultaneously. A 500GB thin pool might serve 2TB of virtual volumes, provided actual data usage stays within the physical limit. This mirrors how airlines oversell flights, banks lend more than their reserves, and clouds over-provision compute resources.

What You Will Learn

By the end of this page, you will understand thin pool architecture and design, thin volume creation and management, over-commitment strategies and risk management, thin pool auto-extension and monitoring, discard/TRIM support for space reclamation, the relationship between thin volumes and snapshots, production deployment best practices, and failure modes and recovery procedures.

Thick vs. Thin Provisioning

Understanding the fundamental difference between provisioning models is essential for making informed storage design decisions. Thick Provisioning (Traditional): When you create a 100GB thick logical volume: 100GB of physical extents are immediately allocated from the VG VG free space decreases by 100GB The volume can use up to 100GB Even if you write only 1GB of data, 100GB remains allocated Other volumes cannot use the 99GB of empty space Thin Provisioning: When you create a 100GB thin volume: Only metadata is allocated (kilobytes) Pool free space barely changes The volume appears as 100GB to the file system As you write data, blocks are allocated from the pool on-demand Actual pool usage matches actual data written Other volumes can use pool space not yet claimed

Converting Mermaid diagram...

Thick vs. Thin Provisioning Comparison
Aspect	Thick Provisioning	Thin Provisioning
Allocation timing	Immediate, at creation	On-demand, at first write
Space efficiency	Low (empty space wasted)	High (only used space consumed)
Over-commitment	Not possible	Core capability
Performance	No allocation overhead	Minor allocation overhead
Snapshots	CoW with pre-allocated space	Pool-shared, highly efficient
Complexity	Simple	Requires pool management
Risk	None (space guaranteed)	Pool exhaustion possible
Best for	Predictable, dense workloads	Variable, sparse workloads

The Over-Commitment Risk

Thin provisioning trades guaranteed capacity for efficiency. If all thin volumes fill simultaneously and exhaust the pool, writes will fail—potentially causing application errors, data corruption, or system crashes. Thin provisioning requires active monitoring and capacity management that thick provisioning does not.

Thin Pool Architecture

A thin pool is a special logical volume that serves as a container for thin volumes. Understanding its internal structure is essential for proper sizing and management. Thin Pool Components: A thin pool consists of two sub-LVs: Data LV (tdata) : Stores the actual data blocks for all thin volumes in the pool Metadata LV (tmeta) : Stores the mapping between thin volume blocks and data LV blocks When you create a thin pool, LVM automatically creates these hidden sub-LVs. The metadata LV is critical—its corruption or exhaustion can make the entire pool inaccessible.

Thin Pool Structure
Bash
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
#!/bin/bash
# Understanding Thin Pool Structure
 
# Create a thin pool
lvcreate --type thin-pool -L 100G -n thin_pool vg_storage
 
# View the pool and its components
lvs -a vg_storage
 
# Output:
#   LV                  VG         Attr       LSize   Pool   ...
#   thin_pool           vg_storage twi-a-t--- 100.00g
#   [thin_pool_tdata]   vg_storage Twi-ao---- 100.00g
#   [thin_pool_tmeta]   vg_storage ewi-ao----   1.00g
 
# The [brackets] indicate hidden LVs
# thin_pool is the visible pool
# thin_pool_tdata is the data storage (100GB)
# thin_pool_tmeta is the metadata (auto-sized, ~1GB)
 
# === METADATA SIZE REQUIREMENTS ===
 
# Metadata LV size depends on:
# - Pool size
# - Block size (chunk size)
# - Number of thin volumes
# - Snapshot chains
 
# Rule of thumb: 1GB metadata per ~1TB of pool data
# Minimum: 2MB
# Maximum: 16GB
 
# === EXPLICIT METADATA SIZING ===
 
# Create pool with specific metadata size
lvcreate --type thin-pool -L 500G \
    --poolmetadatasize 4G \
    -n large_pool vg_storage
 
# Or create data and metadata LVs separately
lvcreate -L 500G -n pool_data vg_storage
lvcreate -L 4G -n pool_meta vg_storage
lvconvert --type thin-pool \
    --poolmetadata vg_storage/pool_meta \
    vg_storage/pool_data
 
# === POOL CHUNK SIZE ===
 
# Chunk size determines allocation granularity
# Smaller = finer allocation, more metadata
# Larger = coarser allocation, less metadata
 
# Default: auto-selected based on pool size
# Range: 64KB to 1GB
 
# Create with explicit chunk size
lvcreate --type thin-pool -L 100G \
    --chunksize 256K \
    -n custom_pool vg_storage
 
# Check chunk size
lvs -o lv_name,chunk_size vg_storage/thin_pool
 
# === POOL PROFILE ===
 
# Use a profile for common configurations
lvcreate --type thin-pool -L 100G \
    --config 'allocation/thin_pool_chunk_size=128' \
    -n profiled_pool vg_storage

Block Allocation Mechanics: When a thin volume writes data: Write request arrives for a block not yet allocated Pool allocates a new chunk from the data LV (tdata) Metadata updated in tmeta to record the mapping Write completes to the newly allocated chunk Subsequent writes to the same block go directly to the already-allocated chunk. This is why thin provisioning is sometimes called "just-in-time" or "on-demand" allocation.

Metadata LV Importance

The metadata LV is the most critical component. If metadata is corrupted or the metadata LV fills up (separate from data LV), the entire pool becomes unusable. LVM automatically sizes metadata conservatively, but for very large pools with many snapshots, you may need to explicitly specify larger metadata. Always monitor metadata usage alongside data usage.

Creating Thin Volumes

Thin volume creation differs from regular volume creation. You create volumes within a pool, specifying virtual size independently of actual pool capacity.

Creating and Managing Thin Volumes
Bash
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
#!/bin/bash
# Creating and Managing Thin Volumes
 
# === PREREQUISITES: CREATE THIN POOL ===
 
# Create a 200GB thin pool
lvcreate --type thin-pool -L 200G -n thin_pool vg_storage
 
# === CREATING THIN VOLUMES ===
 
# Basic thin volume creation
# -V = Virtual size (can exceed pool size!)
# --thinpool = Which pool to use
lvcreate --type thin -V 500G \
    --thinpool thin_pool \
    -n thin_vol1 vg_storage
 
# Create multiple thin volumes that over-commit
lvcreate --type thin -V 200G --thinpool thin_pool -n vm_disk1 vg_storage
lvcreate --type thin -V 200G --thinpool thin_pool -n vm_disk2 vg_storage
lvcreate --type thin -V 200G --thinpool thin_pool -n vm_disk3 vg_storage
# Total virtual: 600GB in 200GB pool - 3x overcommit
 
# === ALTERNATIVE SYNTAX ===
 
# Shorter form using -T
lvcreate -T vg_storage/thin_pool -V 100G -n quick_vol
 
# Create pool and first volume in one command
lvcreate -T vg_storage/new_pool -L 100G -V 500G -n first_vol
 
# === THIN VOLUME SIZING ===
 
# Virtual size can be any value
lvcreate -T vg_storage/thin_pool -V 1T -n large_virtual   # 1TB virtual
lvcreate -T vg_storage/thin_pool -V 10T -n huge_virtual   # 10TB virtual
 
# Initially uses almost no space
lvs -o lv_name,lv_size,data_percent vg_storage
# Output:
#   LV            LSize    Data%
#   thin_pool     200.00g  0.00
#   large_virtual   1.00t  0.00
 
# === FILE SYSTEM ON THIN VOLUME ===
 
# Create filesystem
mkfs.ext4 /dev/vg_storage/thin_vol1
 
# Check actual usage after filesystem creation
lvs -o lv_name,data_percent vg_storage/thin_vol1
# Output shows ~0.1-0.5% (filesystem metadata only)
 
# Mount and use
mount /dev/vg_storage/thin_vol1 /mnt/thin1
 
# === THIN VOLUME EXTENSION ===
 
# Extend virtual size (no pool space needed!)
lvextend -L +100G /dev/vg_storage/thin_vol1
 
# Then extend filesystem
resize2fs /dev/vg_storage/thin_vol1
 
# Or combined
lvextend -L +100G -r /dev/vg_storage/thin_vol1
 
# === LISTING THIN VOLUMES ===
 
# Show thin volumes with pool information
lvs -o lv_name,lv_size,pool_lv,data_percent vg_storage
 
# Output:
#   LV         LSize   Pool       Data%
#   thin_pool  200.00g            45.50
#   thin_vol1  500.00g thin_pool  12.30
#   vm_disk1   200.00g thin_pool  66.00
#   vm_disk2   200.00g thin_pool  23.00
#   vm_disk3   200.00g thin_pool   5.00
 
# === CONVERTING EXISTING LV TO THIN ===
 
# Convert a regular LV to thin (requires pool)
# Warning: This copies all data!
lvconvert --type thin \
    --thinpool vg_storage/thin_pool \
    vg_storage/existing_regular_lv

Virtual Size Strategy

Set virtual sizes generously. Since thin volumes don't consume space until used, oversizing them provides growth room without wasting storage. A VM that might need 100GB over its lifetime can be given 500GB virtual—it costs nothing until used, and you avoid resize operations later.

Pool Monitoring and Auto-Extension

Thin pool monitoring is mandatory for production use. Pool exhaustion causes write failures, potential data corruption, and application crashes. LVM provides both monitoring tools and automatic extension capabilities. Pool Status Indicators:

Pool Monitoring and Auto-Extension
Bash
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
#!/bin/bash
# Thin Pool Monitoring and Auto-Extension
 
# === CHECKING POOL USAGE ===
 
# Basic pool status
lvs vg_storage/thin_pool
# Shows data and metadata usage percentage
 
# Detailed pool information
lvs -o lv_name,lv_size,data_percent,metadata_percent \
    vg_storage/thin_pool
 
# Output:
#   LV        LSize    Data%  Meta%
#   thin_pool 200.00g  78.50  12.30
 
# === MONITORING COMMANDS ===
 
# Continuous monitoring
watch -n 5 'lvs -o lv_name,data_percent,metadata_percent \
    vg_storage/thin_pool'
 
# All thin pool details
lvs -a -o+pool_lv,data_percent,metadata_percent vg_storage
 
# Show transaction ID (for debugging)
lvs -o lv_name,transaction_id vg_storage/thin_pool
 
# === ALERTING SCRIPT ===
 
#!/bin/bash
# thin_pool_monitor.sh - Add to cron
 
DATA_THRESHOLD=80
META_THRESHOLD=75
POOL="/dev/vg_storage/thin_pool"
 
# Get current usage
DATA_PCT=$(lvs --noheadings -o data_percent $POOL | tr -d ' ')
META_PCT=$(lvs --noheadings -o metadata_percent $POOL | tr -d ' ')
 
# Check data usage
if (( $(echo "$DATA_PCT > $DATA_THRESHOLD" | bc -l) )); then
    echo "CRITICAL: Thin pool data at ${DATA_PCT}%"
    # Send alert: mail, Slack, PagerDuty, etc.
fi
 
# Check metadata usage (often overlooked!)
if (( $(echo "$META_PCT > $META_THRESHOLD" | bc -l) )); then
    echo "CRITICAL: Thin pool metadata at ${META_PCT}%"
    # Metadata exhaustion is MORE dangerous than data exhaustion
fi
 
# === AUTO-EXTENSION CONFIGURATION ===
 
# Configure in /etc/lvm/lvm.conf:
 
# thin_pool_autoextend_threshold = 80
#   When pool reaches 80% usage, try to extend
#
# thin_pool_autoextend_percent = 20
#   Extend by 20% of current size
#
# monitoring = 1
#   Enable dmeventd monitoring (required for auto-extend)
 
# Verify dmeventd is running
systemctl status lvm2-monitor
 
# Enable monitoring for a pool explicitly
lvchange --monitor y vg_storage/thin_pool
 
# Check if monitoring is enabled
lvs -o lv_name,lv_health_status vg_storage/thin_pool
 
# === MANUAL EXTENSION ===
 
# Extend pool data (most common need)
lvextend -L +50G vg_storage/thin_pool
 
# Extend pool metadata (when metadata_percent is high)
lvextend --poolmetadatasize +1G vg_storage/thin_pool
 
# === POOL EXHAUSTION RESPONSE ===
 
# If pool is 100% and writes are failing:
 
# 1. Immediate: Extend the pool (if VG has space)
lvextend -L +50G vg_storage/thin_pool
 
# 2. If VG has no space: Add a new PV
pvcreate /dev/new_disk
vgextend vg_storage /dev/new_disk
lvextend -L +50G vg_storage/thin_pool
 
# 3. Delete/reduce thin volumes to reclaim space
lvremove vg_storage/unused_thin_vol
 
# 4. Run fstrim/discard to reclaim deleted file space
fstrim /mount/point

Pool Full = Write Failure

When a thin pool reaches 100% capacity, all writes to all thin volumes in that pool fail. Applications may crash, filesystems may become corrupted, and databases may lose transactions. Unlike classic snapshots that only affect themselves when full, a full thin pool is a system-wide emergency. Configure auto-extension and monitoring BEFORE deploying thin provisioning in production.

Space Reclamation with Discards

Thin provisioning allocates space on write, but what happens when data is deleted? By default, nothing—the pool retains allocated blocks even after files are deleted. Discard (also called TRIM) enables space reclamation, returning unused blocks to the pool. The Reclamation Problem: Consider this scenario: Create 100GB thin volume, write 80GB of data Pool usage: 80GB Delete 40GB of files Pool usage: still 80GB! The filesystem knows those blocks are free, but the thin pool doesn't. Discards bridge this gap by informing the pool which blocks are no longer needed.

Discard Configuration and Usage
Bash
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
#!/bin/bash
# Thin Pool Discard/TRIM Configuration and Usage
 
# === POOL DISCARD MODE ===
 
# Check current discard mode
lvs -o lv_name,discards vg_storage/thin_pool
 
# Discard modes:
# - ignore:   Discards are ignored (default in older versions)
# - nopassdown: Pool processes discards but doesn't pass to underlying devices
# - passdown:   Pool processes and passes to underlying devices (SSDs)
 
# Set discard mode
lvchange --discards passdown vg_storage/thin_pool
 
# Create pool with discard mode
lvcreate --type thin-pool -L 200G \
    --discards passdown \
    -n thin_pool vg_storage
 
# === MANUAL DISCARD (FSTRIM) ===
 
# Run fstrim on mounted filesystem to reclaim space
fstrim -v /mnt/thin_volume
 
# Output: /mnt/thin_volume: 45 GiB (48318382080 bytes) trimmed
 
# Verify space was reclaimed
lvs -o lv_name,data_percent vg_storage/thin_pool
 
# === AUTOMATIC DISCARD (MOUNT OPTION) ===
 
# Mount with discard option for real-time space reclamation
mount -o discard /dev/vg_storage/thin_vol1 /mnt/data
 
# In /etc/fstab:
# /dev/vg_storage/thin_vol1 /mnt/data ext4 defaults,discard 0 2
 
# WARNING: Online discard can impact write performance
# Periodic fstrim is often preferred for busy systems
 
# === PERIODIC FSTRIM SERVICE ===
 
# Most distributions include fstrim.timer
systemctl enable fstrim.timer
systemctl start fstrim.timer
 
# Check timer status
systemctl status fstrim.timer
systemctl list-timers | grep fstrim
 
# Default runs weekly; adjust if needed
 
# === DISCARD GRANULARITY ===
 
# Check minimum discard granularity
blockdev --getdiscardgranularity /dev/vg_storage/thin_vol1
 
# Should match or exceed pool chunk size for efficiency
 
# === ZEROING AND DISCARD ===
 
# Thin pool zero behavior
# When a new block is allocated, should it be zeroed?
 
# Check current setting
lvs -o lv_name,zero vg_storage/thin_pool
 
# Enable zeroing (security, but slower allocation)
lvchange --zero y vg_storage/thin_pool
 
# Disable zeroing (faster, for trusted environments)
lvchange --zero n vg_storage/thin_pool
 
# === VERIFYING DISCARD IS WORKING ===
 
# Create test file
dd if=/dev/urandom of=/mnt/thin_vol/test bs=1M count=1000
 
# Check pool usage
lvs -o data_percent vg_storage/thin_pool
# Example: 50.00
 
# Delete file
rm /mnt/thin_vol/test
 
# Without discard, usage unchanged
# With discard...
sync
sleep 2
lvs -o data_percent vg_storage/thin_pool
# Example: 45.00 (if online discard) or unchanged until fstrim

Periodic fstrim vs. Mount Discard

For most workloads, running fstrim weekly via systemd timer is preferable to the 'discard' mount option. Online discard adds latency to every delete operation, which can impact performance on busy systems. Weekly fstrim batches all discards into a single maintenance window, minimizing impact on normal operations.

Thin Provisioning with Snapshots

Thin provisioning and snapshots are deeply integrated in LVM. Thin snapshots are far more efficient than classic snapshots and offer capabilities not possible with thick provisioning. Thin Snapshot Advantages: No pre-allocation required : Snapshot size is unlimited (within pool capacity) Shared pool efficiency : Multiple snapshots share the same pool Snapshot chains : Create snapshots of snapshots to any depth Writable by default : Each snapshot can diverge independently No origin penalty : No CoW overhead on the origin volume

Thin Snapshots
Bash
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
#!/bin/bash
# Thin Snapshots - Advanced Usage
 
# === BASIC THIN SNAPSHOT ===
 
# Create snapshot of thin volume (instant, no size needed)
lvcreate -s -n snap1 vg_storage/thin_vol1
 
# Verify it's a thin snapshot
lvs -o lv_name,pool_lv,origin vg_storage/snap1
# Shows: snap1  thin_pool  thin_vol1
 
# === SNAPSHOT CHAINS ===
 
# Create snapshot of a snapshot
lvcreate -s -n snap2 vg_storage/snap1
 
# And another level
lvcreate -s -n snap3 vg_storage/snap2
 
# Check the chain
lvs -o lv_name,origin vg_storage | grep snap
# snap1  thin_vol1
# snap2  snap1
# snap3  snap2
 
# Can continue indefinitely (limited only by metadata)
 
# === WRITABLE THIN SNAPSHOTS ===
 
# Thin snapshots are writable by default
mount /dev/vg_storage/snap1 /mnt/snap1
 
# Make changes
echo "Modified in snapshot" > /mnt/snap1/test.txt
 
# Original is unaffected
mount /dev/vg_storage/thin_vol1 /mnt/original
cat /mnt/original/test.txt
# (shows original content)
 
# === EXTERNAL ORIGIN SNAPSHOTS ===
 
# Create thin snapshot of a NON-thin (thick) volume
# Origin becomes "external origin" - read-only reference
 
# First, have a regular LV
lvcreate -L 50G -n thick_vol vg_storage
 
# Create thin snapshot of it
lvcreate -s --thinpool vg_storage/thin_pool \
    -n thin_snap_of_thick \
    vg_storage/thick_vol
 
# The thick volume is now a read-only external origin
# All changes go to the thin pool
 
# === VM TEMPLATE DEPLOYMENT ===
 
# Create golden template
lvcreate -T vg_storage/thin_pool -V 100G -n vm_template
 
# Install and configure OS on template
# ...
 
# Deploy VMs as thin snapshots
for i in {1..10}; do
    lvcreate -s -n vm_instance_$i vg_storage/vm_template
done
 
# 10 VMs sharing template, each storing only differences
# Much faster than copying 100GB x 10!
 
# Check actual storage usage
lvs -o lv_name,lv_size,data_percent vg_storage
# Each vm_instance shows individual data_percent
 
# === SNAPSHOT DELETION ===
 
# Removing middle of chain
# If: original → snap1 → snap2
 
lvremove vg_storage/snap1
 
# LVM handles this - snap2 now references original
# No data loss, but reorganization happens
 
# === THIN TIME MACHINE PATTERN ===
 
# Create rotating snapshots for point-in-time recovery
#!/bin/bash
# thin_rotation.sh
 
ORIGIN="vg_storage/thin_data"
MAX_SNAPS=24  # Keep 24 hourly snapshots
 
# Create new snapshot with timestamp
SNAP_NAME="hourly_$(date +%Y%m%d_%H%M)"
lvcreate -s -n $SNAP_NAME $ORIGIN
 
# Remove old snapshots beyond retention
SNAP_COUNT=$(lvs --noheadings -o lv_name $ORIGIN 2>/dev/null | \
    grep "^  hourly_" | wc -l)
 
if [ $SNAP_COUNT -gt $MAX_SNAPS ]; then
    # Remove oldest
    OLDEST=$(lvs --noheadings --sort -lv_time -o lv_name \
        vg_storage 2>/dev/null | grep "hourly_" | tail -1 | tr -d ' ')
    lvremove -f vg_storage/$OLDEST
fi

Snapshot Chain Depth

While thin snapshot chains have no hard limit, very deep chains (50+ levels) can impact read performance as LVM must traverse the chain to find data. For production use, consider periodic 'flattening' by creating a new base thin volume from accumulated snapshots. Also note that chain depth affects metadata consumption.

Production Best Practices

Thin provisioning requires more careful planning and monitoring than thick provisioning. These practices derive from production deployments and failure analysis.

Essential Practices

•Enable dmeventd monitoring: Required for auto-extension to work
•Set auto-extend thresholds: Configure thin_pool_autoextend_threshold at 80% in lvm.conf
•Monitor metadata separately: Metadata exhaustion is more dangerous than data exhaustion
•Leave VG headroom: Auto-extend needs VG free space; reserve 20%+
•Enable discards: Configure passdown mode and run periodic fstrim
•Limit overcommit ratio: 2:1 to 4:1 max depending on workload predictability
•Alert before crisis: Set alerts at 70% to allow response time
•Document virtual vs actual: Track total virtual allocation vs pool capacity

Common Failures

•No monitoring: Pool exhausts silently, discovered when applications crash
•Ignoring metadata: Pool metadata fills before data, total failure
•Infinite overcommit: Provisioning without tracking leads to sudden capacity crises
•Disabled discards: Pool never reclaims space, perpetual growth
•No auto-extend space: VG is full, auto-extend fails, pool fails
•Relying on auto-extend alone: Burst writes can exhaust faster than extend runs
•Snapshot accumulation: Old snapshots consume pool silently
•Missing from backup scope: Pool metadata not backed up, recovery impossible

Production Configuration
Configuration
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
# /etc/lvm/lvm.conf - Production thin provisioning settings
 
activation {
    # Enable dmeventd monitoring (CRITICAL for auto-extend)
    monitoring = 1
    
    # Auto-extend threshold (percent full before extending)
    thin_pool_autoextend_threshold = 80
    
    # Auto-extend amount (percent of current size)
    thin_pool_autoextend_percent = 20
}
 
allocation {
    # Default chunk size for new thin pools (KB)
    thin_pool_chunk_size = 256
    
    # Zero new blocks (set to 0 for performance, 1 for security)
    thin_pool_zero = 0
    
    # Discard mode for new pools
    thin_pool_discards = "passdown"
}
 
# After modifying, restart LVM monitoring
# systemctl restart lvm2-monitor

Capacity Planning Formula: Recommended Pool Size = (Sum of Expected Actual Usage) \times 1.3 + (Snapshot Retention \times Daily Change Rate \times Days) + (Auto-extend Buffer: 20% of pool) Example: 5 VMs expected to use 50GB actual each = 250GB Daily change rate: 5GB/day, 7-day snapshot retention = 35GB Pool size: (250 \times 1.3) + 35 + 20% buffer = 325GB + 57GB + 76GB \approx 460GB pool With 4:1 overcommit, can provision 1.8TB virtual

Recovery Preparation

Prepare for pool exhaustion BEFORE it happens. Document the emergency procedure: which volumes can be deleted, which can be migrated to other pools, and how to quickly extend the pool. Test the procedure during maintenance windows. When a pool fills at 3 AM, you don't want to be reading documentation for the first time.

Thin Provisioning

Storage on Demand

Thick vs. Thin Provisioning

Thin Pool Architecture

Creating Thin Volumes

Pool Monitoring and Auto-Extension

Space Reclamation with Discards

Thin Provisioning with Snapshots

Production Best Practices

There’s more to understand.

Thin Provisioning

Storage on Demand

Thick vs. Thin Provisioning

Thin Pool Architecture

Creating Thin Volumes

Pool Monitoring and Auto-Extension

Space Reclamation with Discards

Thin Provisioning with Snapshots

Production Best Practices

There’s more to understand.