System Design (HLD)Secrets & Compliance

Key Management

LevelAdvanced

Duration90 mins

TopicSecrets & Compliance

5 / 5

Key Hierarchy

Organizing Keys at Scale

A mature system might use thousands—even millions—of cryptographic keys. Every encrypted database record could have its own key. Each user session generates ephemeral keys. Every service holds API signing keys. Managing this complexity without structure leads to chaos, security gaps, and operational nightmares.

Key hierarchy is the discipline of organizing keys in layered structures where high-value, rarely-used master keys protect lower-level operational keys. This approach balances security (critical keys are highly protected), manageability (operational keys can be rotated independently), and performance (data encryption uses local keys).

This page explores key hierarchy design patterns, from the root keys at the top through operational keys that touch your data.

What You Will Learn

By the end of this page, you will understand key hierarchy levels, the relationships between root keys, key encryption keys, and data encryption keys, design patterns for enterprise key hierarchies, and how to implement hierarchies using cloud KMS services.

Why Key Hierarchies

A flat key structure—where every piece of data is encrypted directly by a master key—creates severe problems at scale:

Without hierarchy:

Rotating the master key requires re-encrypting ALL data
One compromised key exposes everything
Every encryption operation needs KMS/HSM access (slow, expensive)
No way to delegate key access to specific services

With hierarchy:

Only immediate child keys need re-wrapping on rotation
Compromise is contained to keys at that level
Data encryption uses local keys (fast)
Each level can have different access policies

Converting Mermaid diagram...

The Power of Layers

Key hierarchies follow the same principle as organizational hierarchies: each level has authority over the level below. Rotating a mid-level key affects only its children, not siblings or the root. This isolation is what makes large-scale key management tractable.

Key Hierarchy Levels

A typical enterprise key hierarchy has 3-4 levels, each with distinct properties and protection requirements:

Key Hierarchy Levels
Level	Name	Protection	Rotation	Purpose
1	Root Key (Master Key)	HSM, never extracted, offline for highest security	Rarely (years) or never	Protects all downstream keys
2	Key Encryption Keys (KEKs)	KMS or HSM, strict access policies	Annually	Wrap service/regional master keys
3	Service Master Keys	KMS, per-service policies	Quarterly to annually	Wrap data encryption keys for a service
4	Data Encryption Keys (DEKs)	Encrypted by parent, stored with data	Per-data-element or frequently	Directly encrypt data

Properties by Level

•Root Key — Highest security, lowest volume. May exist only in HSM, accessed only for key ceremonies. Some organizations keep root keys offline entirely, bringing them online only to sign subordinate keys.
•Key Encryption Keys — Balance security and accessibility. Protected by KMS/HSM but available for automated operations. Group keys by region, compliance boundary, or organizational unit.
•Service Master Keys — Per-service isolation. Each service/application has its own key, enabling independent rotation and access control. Compromise of one service doesn't expose others.
•Data Encryption Keys — High volume, short lifespan. Often unique per record or session. Stored encrypted (wrapped) alongside the data they protect. Fast local operations.

Hierarchy Design Patterns

Different organizational structures and requirements lead to different hierarchy designs. Here are common patterns:

Multi-Tenant SaaS Pattern

•Root → Regional KEKs → Tenant Master Keys → Data DEKs
•Each tenant has isolated master key
•Tenant key deletion enables crypto-shredding
•Cross-tenant data access prevented by design
•Tenant can bring their own key (BYOK)

Microservices Pattern

•Root → Environment KEKs → Service Keys → DEKs
•Separate hierarchies per environment
•Services can't access each other's keys
•Enables zero-trust service isolation
•CI/CD has limited key scope per service

key_hierarchy_implementation
Python
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
import boto3
import os
from dataclasses import dataclass
from typing import Dict, Optional
 
@dataclass
class KeyHierarchyConfig:
    """Configuration for a multi-level key hierarchy."""
    root_key_alias: str
    region: str
    service_name: str
    tenant_id: Optional[str] = None
 
class KeyHierarchyManager:
    """
    Implements a 4-level key hierarchy:
    Root Key → Regional KEK → Service Key → DEKs
    """
    
    def __init__(self, config: KeyHierarchyConfig):
        self.kms = boto3.client('kms', region_name=config.region)
        self.config = config
        
        # Key aliases following naming convention
        self.regional_kek_alias = f"alias/{config.region}-kek"
        self.service_key_alias = f"alias/{config.service_name}-master"
        if config.tenant_id:
            self.service_key_alias += f"-{config.tenant_id}"
    
    def initialize_hierarchy(self) -> Dict[str, str]:
        """
        Initialize key hierarchy for this service.
        Creates keys at each level if they don't exist.
        """
        results = {}
        
        # Level 2: Regional KEK (created once per region)
        regional_kek = self._ensure_key_exists(
            self.regional_kek_alias,
            f"Regional KEK for {self.config.region}",
            self.config.root_key_alias  # Wrapped by root
        )
        results['regional_kek'] = regional_kek
        
        # Level 3: Service Master Key
        service_key = self._ensure_key_exists(
            self.service_key_alias,
            f"Master key for {self.config.service_name}",
            self.regional_kek_alias  # Wrapped by regional KEK
        )
        results['service_key'] = service_key
        
        return results
    
    def generate_dek(self, context: Dict[str, str]) -> Dict[str, bytes]:
        """
        Generate a Data Encryption Key (Level 4).
        DEK is wrapped by the service master key.
        """
        response = self.kms.generate_data_key(
            KeyId=self.service_key_alias,
            KeySpec='AES_256',
            EncryptionContext=context
        )
        
        return {
            'plaintext': response['Plaintext'],
            'wrapped': response['CiphertextBlob'],
            'context': context
        }
    
    def rewrap_dek(self, old_wrapped_dek: bytes, 
                   old_context: Dict[str, str],
                   new_service_key_alias: str) -> bytes:
        """
        Re-wrap a DEK with a new service key during rotation.
        This is the key operation for hierarchy rotation.
        """
        # Decrypt with old key
        decrypt_response = self.kms.decrypt(
            CiphertextBlob=old_wrapped_dek,
            EncryptionContext=old_context
        )
        plaintext_dek = decrypt_response['Plaintext']
        
        # Encrypt with new key
        encrypt_response = self.kms.encrypt(
            KeyId=new_service_key_alias,
            Plaintext=plaintext_dek,
            EncryptionContext=old_context
        )
        
        # Zero plaintext DEK
        del plaintext_dek
        
        return encrypt_response['CiphertextBlob']
    
    def _ensure_key_exists(self, alias: str, description: str,
                           parent_alias: str) -> str:
        """Create key if it doesn't exist, return key ID."""
        try:
            response = self.kms.describe_key(KeyId=alias)
            return response['KeyMetadata']['KeyId']
        except self.kms.exceptions.NotFoundException:
            # Create new key
            key = self.kms.create_key(Description=description)
            key_id = key['KeyMetadata']['KeyId']
            
            # Create alias
            self.kms.create_alias(AliasName=alias, TargetKeyId=key_id)
            
            # Enable rotation
            self.kms.enable_key_rotation(KeyId=key_id)
            
            return key_id
 
# Usage for a multi-tenant SaaS
config = KeyHierarchyConfig(
    root_key_alias='alias/global-root-key',
    region='us-east-1',
    service_name='user-service',
    tenant_id='tenant-acme-corp'
)
 
hierarchy = KeyHierarchyManager(config)
hierarchy.initialize_hierarchy()
 
# Generate DEK for encrypting user data
dek = hierarchy.generate_dek({
    'tenant': 'acme-corp',
    'data_type': 'user_profile',
    'user_id': 'user-12345'
})

Rotating Keys in a Hierarchy

Key rotation in a hierarchy affects different levels differently:

DEK Rotation: Simplest case. Generate new DEK, re-encrypt data, delete old DEK. Affects only one data element.

Service Key Rotation: Must re-wrap all DEKs encrypted by the old service key. Schedule as a background job. Both old and new keys remain active during transition.

KEK Rotation: Must re-wrap all service keys. Cascading effect but doesn't require re-encrypting data.

Root Key Rotation: Extremely rare. Requires re-wrapping entire key tree from root. Often involves key ceremonies with multiple authorized parties.

Rotation Impact by Level
Key Level	Rotation Triggers	Affected Keys	Typical Approach
DEK	Data update, access event	None (leaf node)	Generate new, re-encrypt, discard old
Service Key	Schedule, personnel change	All DEKs for this service	Background re-wrap job
Regional KEK	Annual, compliance	All service keys in region	Scheduled maintenance window
Root Key	Compromise, HSM migration	Entire hierarchy	Planned key ceremony event

Cascading Rotation Risk

Rotating a high-level key triggers cascading re-wrapping down the tree. For large hierarchies, this can take hours or days. Plan with: staged rollout, progress monitoring, rollback procedures, and tested automation. Never rotate high-level keys without extensive planning.

Crypto-Shredding with Hierarchies

Key hierarchies enable crypto-shredding: making data permanently unreadable by destroying its encryption key rather than locating and overwriting every copy of the data.

Why crypto-shredding matters:

Data exists in multiple locations: primary storage, backups, replicas, logs, caches
Finding and securely erasing all copies is nearly impossible
Destroying the key renders ALL copies useless simultaneously
Essential for GDPR 'right to erasure' compliance

Implementing Crypto-Shredding

•Per-subject key design — Each user/tenant/entity gets their own key in the hierarchy. Deleting one key doesn't affect others.
•Key deletion workflow — Verify deletion scope, schedule key destruction, confirm key is unrecoverable, document for compliance.
•Backup considerations — Backups contain data encrypted with the deleted key. They become garbage (undecryptable) automatically.
•Audit trail — Log key destruction with timestamps, authorized parties, and scope for regulatory proof of deletion.

Tenant Offboarding Example

When a SaaS tenant churns:

Revoke tenant's access permissions
Schedule their tenant master key for deletion
After waiting period, confirm permanent deletion
All tenant data across all systems is now unreadable
Provide deletion certificate to tenant for their records

Summary: Key Hierarchy Best Practices

Key Takeaways

•Layer keys by protection level — Root keys in HSMs, KEKs in KMS with strict policies, DEKs stored encrypted with data. Each level protects the next.
•Design for isolation — Per-tenant, per-service, per-region key branches prevent compromise propagation and enable independent lifecycle management.
•Plan rotation for each level — DEKs rotate frequently (auto). Service keys rotate periodically (scheduled). Root keys rarely (ceremony). Document procedures for each.
•Enable crypto-shredding — Design key-per-subject to support GDPR deletion via key destruction rather than data hunting.
•Use naming conventions — Consistent key aliases (alias/region-service-purpose) make hierarchy visible and manageable.
•Document key relationships — Maintain diagrams showing which keys wrap which. Essential for incident response and rotation planning.

Module Complete

Congratulations! You've completed the Key Management module. You now understand key lifecycle, rotation strategies, cloud KMS services, HSMs, and key hierarchy design. These skills are essential for building systems that protect sensitive data at scale.

5 / 5

Loading learning content...

System Design (HLD)Secrets & Compliance

Key Management

LevelAdvanced

Duration90 mins

TopicSecrets & Compliance

5 / 5

Key Hierarchy

Organizing Keys at Scale

This page explores key hierarchy design patterns, from the root keys at the top through operational keys that touch your data.

What You Will Learn

Why Key Hierarchies

A flat key structure—where every piece of data is encrypted directly by a master key—creates severe problems at scale:

Without hierarchy:

Rotating the master key requires re-encrypting ALL data
One compromised key exposes everything
Every encryption operation needs KMS/HSM access (slow, expensive)
No way to delegate key access to specific services

With hierarchy:

Only immediate child keys need re-wrapping on rotation
Compromise is contained to keys at that level
Data encryption uses local keys (fast)
Each level can have different access policies

Converting Mermaid diagram...

The Power of Layers

Key Hierarchy Levels

A typical enterprise key hierarchy has 3-4 levels, each with distinct properties and protection requirements:

Key Hierarchy Levels
Level	Name	Protection	Rotation	Purpose
1	Root Key (Master Key)	HSM, never extracted, offline for highest security	Rarely (years) or never	Protects all downstream keys
2	Key Encryption Keys (KEKs)	KMS or HSM, strict access policies	Annually	Wrap service/regional master keys
3	Service Master Keys	KMS, per-service policies	Quarterly to annually	Wrap data encryption keys for a service
4	Data Encryption Keys (DEKs)	Encrypted by parent, stored with data	Per-data-element or frequently	Directly encrypt data

Properties by Level

•Root Key — Highest security, lowest volume. May exist only in HSM, accessed only for key ceremonies. Some organizations keep root keys offline entirely, bringing them online only to sign subordinate keys.
•Key Encryption Keys — Balance security and accessibility. Protected by KMS/HSM but available for automated operations. Group keys by region, compliance boundary, or organizational unit.
•Service Master Keys — Per-service isolation. Each service/application has its own key, enabling independent rotation and access control. Compromise of one service doesn't expose others.
•Data Encryption Keys — High volume, short lifespan. Often unique per record or session. Stored encrypted (wrapped) alongside the data they protect. Fast local operations.

Hierarchy Design Patterns

Different organizational structures and requirements lead to different hierarchy designs. Here are common patterns:

Multi-Tenant SaaS Pattern

•Root → Regional KEKs → Tenant Master Keys → Data DEKs
•Each tenant has isolated master key
•Tenant key deletion enables crypto-shredding
•Cross-tenant data access prevented by design
•Tenant can bring their own key (BYOK)

Microservices Pattern

•Root → Environment KEKs → Service Keys → DEKs
•Separate hierarchies per environment
•Services can't access each other's keys
•Enables zero-trust service isolation
•CI/CD has limited key scope per service

key_hierarchy_implementation
Python
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
import boto3
import os
from dataclasses import dataclass
from typing import Dict, Optional
 
@dataclass
class KeyHierarchyConfig:
    """Configuration for a multi-level key hierarchy."""
    root_key_alias: str
    region: str
    service_name: str
    tenant_id: Optional[str] = None
 
class KeyHierarchyManager:
    """
    Implements a 4-level key hierarchy:
    Root Key → Regional KEK → Service Key → DEKs
    """
    
    def __init__(self, config: KeyHierarchyConfig):
        self.kms = boto3.client('kms', region_name=config.region)
        self.config = config
        
        # Key aliases following naming convention
        self.regional_kek_alias = f"alias/{config.region}-kek"
        self.service_key_alias = f"alias/{config.service_name}-master"
        if config.tenant_id:
            self.service_key_alias += f"-{config.tenant_id}"
    
    def initialize_hierarchy(self) -> Dict[str, str]:
        """
        Initialize key hierarchy for this service.
        Creates keys at each level if they don't exist.
        """
        results = {}
        
        # Level 2: Regional KEK (created once per region)
        regional_kek = self._ensure_key_exists(
            self.regional_kek_alias,
            f"Regional KEK for {self.config.region}",
            self.config.root_key_alias  # Wrapped by root
        )
        results['regional_kek'] = regional_kek
        
        # Level 3: Service Master Key
        service_key = self._ensure_key_exists(
            self.service_key_alias,
            f"Master key for {self.config.service_name}",
            self.regional_kek_alias  # Wrapped by regional KEK
        )
        results['service_key'] = service_key
        
        return results
    
    def generate_dek(self, context: Dict[str, str]) -> Dict[str, bytes]:
        """
        Generate a Data Encryption Key (Level 4).
        DEK is wrapped by the service master key.
        """
        response = self.kms.generate_data_key(
            KeyId=self.service_key_alias,
            KeySpec='AES_256',
            EncryptionContext=context
        )
        
        return {
            'plaintext': response['Plaintext'],
            'wrapped': response['CiphertextBlob'],
            'context': context
        }
    
    def rewrap_dek(self, old_wrapped_dek: bytes, 
                   old_context: Dict[str, str],
                   new_service_key_alias: str) -> bytes:
        """
        Re-wrap a DEK with a new service key during rotation.
        This is the key operation for hierarchy rotation.
        """
        # Decrypt with old key
        decrypt_response = self.kms.decrypt(
            CiphertextBlob=old_wrapped_dek,
            EncryptionContext=old_context
        )
        plaintext_dek = decrypt_response['Plaintext']
        
        # Encrypt with new key
        encrypt_response = self.kms.encrypt(
            KeyId=new_service_key_alias,
            Plaintext=plaintext_dek,
            EncryptionContext=old_context
        )
        
        # Zero plaintext DEK
        del plaintext_dek
        
        return encrypt_response['CiphertextBlob']
    
    def _ensure_key_exists(self, alias: str, description: str,
                           parent_alias: str) -> str:
        """Create key if it doesn't exist, return key ID."""
        try:
            response = self.kms.describe_key(KeyId=alias)
            return response['KeyMetadata']['KeyId']
        except self.kms.exceptions.NotFoundException:
            # Create new key
            key = self.kms.create_key(Description=description)
            key_id = key['KeyMetadata']['KeyId']
            
            # Create alias
            self.kms.create_alias(AliasName=alias, TargetKeyId=key_id)
            
            # Enable rotation
            self.kms.enable_key_rotation(KeyId=key_id)
            
            return key_id
 
# Usage for a multi-tenant SaaS
config = KeyHierarchyConfig(
    root_key_alias='alias/global-root-key',
    region='us-east-1',
    service_name='user-service',
    tenant_id='tenant-acme-corp'
)
 
hierarchy = KeyHierarchyManager(config)
hierarchy.initialize_hierarchy()
 
# Generate DEK for encrypting user data
dek = hierarchy.generate_dek({
    'tenant': 'acme-corp',
    'data_type': 'user_profile',
    'user_id': 'user-12345'
})

Rotating Keys in a Hierarchy

Key rotation in a hierarchy affects different levels differently:

DEK Rotation: Simplest case. Generate new DEK, re-encrypt data, delete old DEK. Affects only one data element.

Service Key Rotation: Must re-wrap all DEKs encrypted by the old service key. Schedule as a background job. Both old and new keys remain active during transition.

KEK Rotation: Must re-wrap all service keys. Cascading effect but doesn't require re-encrypting data.

Root Key Rotation: Extremely rare. Requires re-wrapping entire key tree from root. Often involves key ceremonies with multiple authorized parties.

Rotation Impact by Level
Key Level	Rotation Triggers	Affected Keys	Typical Approach
DEK	Data update, access event	None (leaf node)	Generate new, re-encrypt, discard old
Service Key	Schedule, personnel change	All DEKs for this service	Background re-wrap job
Regional KEK	Annual, compliance	All service keys in region	Scheduled maintenance window
Root Key	Compromise, HSM migration	Entire hierarchy	Planned key ceremony event

Cascading Rotation Risk

Crypto-Shredding with Hierarchies

Key hierarchies enable crypto-shredding: making data permanently unreadable by destroying its encryption key rather than locating and overwriting every copy of the data.

Why crypto-shredding matters:

Data exists in multiple locations: primary storage, backups, replicas, logs, caches
Finding and securely erasing all copies is nearly impossible
Destroying the key renders ALL copies useless simultaneously
Essential for GDPR 'right to erasure' compliance

Implementing Crypto-Shredding

•Per-subject key design — Each user/tenant/entity gets their own key in the hierarchy. Deleting one key doesn't affect others.
•Key deletion workflow — Verify deletion scope, schedule key destruction, confirm key is unrecoverable, document for compliance.
•Backup considerations — Backups contain data encrypted with the deleted key. They become garbage (undecryptable) automatically.
•Audit trail — Log key destruction with timestamps, authorized parties, and scope for regulatory proof of deletion.

Tenant Offboarding Example

When a SaaS tenant churns:

Revoke tenant's access permissions
Schedule their tenant master key for deletion
After waiting period, confirm permanent deletion
All tenant data across all systems is now unreadable
Provide deletion certificate to tenant for their records

Summary: Key Hierarchy Best Practices

Key Takeaways

•Layer keys by protection level — Root keys in HSMs, KEKs in KMS with strict policies, DEKs stored encrypted with data. Each level protects the next.
•Design for isolation — Per-tenant, per-service, per-region key branches prevent compromise propagation and enable independent lifecycle management.
•Plan rotation for each level — DEKs rotate frequently (auto). Service keys rotate periodically (scheduled). Root keys rarely (ceremony). Document procedures for each.
•Enable crypto-shredding — Design key-per-subject to support GDPR deletion via key destruction rather than data hunting.
•Use naming conventions — Consistent key aliases (alias/region-service-purpose) make hierarchy visible and manageable.
•Document key relationships — Maintain diagrams showing which keys wrap which. Essential for incident response and rotation planning.

Module Complete

5 / 5