Key Management - Learning Module

Loading content...

0/273

KMS Services (AWS KMS, GCP KMS)

Managed Key Management at Scale

Building and operating your own key management infrastructure is extraordinarily complex. You need FIPS-validated cryptographic modules, physically secure facilities, redundant storage, access controls, audit logging, disaster recovery, and 24/7 operations—all while maintaining the security expertise to do it correctly.

Cloud Key Management Services (KMS) solve this by providing managed cryptographic infrastructure. AWS KMS, GCP Cloud KMS, and Azure Key Vault offer enterprise-grade key management without the operational burden. Keys are generated and stored in hardware security modules, protected by the cloud provider's physical and logical security, and accessible via simple APIs.

This page explores how cloud KMS services work, their security properties, and practical patterns for integrating them into your systems.

What You Will Learn

By the end of this page, you will understand cloud KMS architectures, key types and policies, envelope encryption implementation, cross-region and cross-account patterns, and how to choose between AWS KMS and GCP Cloud KMS for your use case.

Cloud KMS Architecture

Cloud KMS services share a common architectural pattern: cryptographic operations happen inside hardened, isolated environments where key material never exists in extractable form.

Core components:

Hardware Security Modules (HSMs): FIPS 140-2 Level 2 or 3 validated hardware that generates and stores keys
Key hierarchy: Root keys (never leave HSM) protect customer master keys (CMKs)
API layer: RESTful APIs for cryptographic operations (encrypt, decrypt, sign, verify)
Policy engine: Fine-grained access control determining who can use which keys for what
Audit logging: Complete record of all key operations for compliance and forensics

Cloud KMS Service Comparison
Feature	AWS KMS	GCP Cloud KMS	Azure Key Vault
HSM backing	FIPS 140-2 L2 (L3 with CloudHSM)	FIPS 140-2 L3	FIPS 140-2 L2 (L3 with Dedicated HSM)
Symmetric algorithms	AES-256-GCM	AES-128/256-GCM, AES-CBC	AES-128/256
Asymmetric algorithms	RSA, ECC (P-256, P-384, P-521)	RSA, EC (P-256, P-384, secp256k1)	RSA, EC
Automatic rotation	Annual (symmetric only)	Configurable (any period)	Configurable
Multi-region keys	Yes (replica keys)	Yes (global keys)	No (geo-redundancy)
Pricing model	Per key/month + per request	Per key version + per operation	Per key + per operation

Key Material Security

In standard KMS operation, plaintext key material NEVER leaves the HSM. When you call Encrypt, your plaintext travels to KMS, is encrypted inside the HSM, and ciphertext returns. The key itself stays locked inside secure hardware throughout its entire lifecycle.

AWS KMS Deep Dive

AWS Key Management Service is the most widely adopted cloud KMS, deeply integrated with AWS services. Understanding its key types, policies, and patterns is essential for AWS-based systems.

Key types in AWS KMS:

AWS managed keys: Created and managed by AWS for integrated services (S3-SSE, EBS, RDS). Free, no management required, limited control.
Customer managed keys (CMKs): You create, control policies, enable rotation. Full audit trail. Primary choice for application encryption.
AWS owned keys: Internal AWS keys. You don't see or manage them.

aws_kms_operations
Python
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
import boto3
import base64
from typing import Optional
 
class AWSKMSManager:
    """
    AWS KMS operations for key management and cryptography.
    """
    
    def __init__(self, region: str = 'us-east-1'):
        self.kms = boto3.client('kms', region_name=region)
        self.region = region
    
    def create_key(self, description: str, 
                   key_usage: str = 'ENCRYPT_DECRYPT') -> dict:
        """
        Create a customer managed key (CMK).
        
        key_usage options:
        - ENCRYPT_DECRYPT: Symmetric encryption key
        - SIGN_VERIFY: Asymmetric signing key
        """
        response = self.kms.create_key(
            Description=description,
            KeyUsage=key_usage,
            Origin='AWS_KMS',  # Key generated by KMS HSM
            Tags=[
                {'TagKey': 'Environment', 'TagValue': 'production'},
                {'TagKey': 'ManagedBy', 'TagValue': 'application-team'}
            ]
        )
        
        key_id = response['KeyMetadata']['KeyId']
        
        # Create an alias for easier reference
        alias = f"alias/{description.lower().replace(' ', '-')}"
        self.kms.create_alias(AliasName=alias, TargetKeyId=key_id)
        
        # Enable automatic rotation (recommended for symmetric keys)
        if key_usage == 'ENCRYPT_DECRYPT':
            self.kms.enable_key_rotation(KeyId=key_id)
        
        return {
            'key_id': key_id,
            'alias': alias,
            'arn': response['KeyMetadata']['Arn']
        }
    
    def encrypt_with_context(self, key_id: str, plaintext: bytes,
                             context: dict) -> bytes:
        """
        Encrypt with encryption context for additional security.
        Context must match exactly on decrypt.
        """
        response = self.kms.encrypt(
            KeyId=key_id,
            Plaintext=plaintext,
            EncryptionContext=context,
            EncryptionAlgorithm='SYMMETRIC_DEFAULT'  # AES-256-GCM
        )
        return response['CiphertextBlob']
    
    def generate_data_key(self, key_id: str, 
                          context: Optional[dict] = None) -> dict:
        """
        Generate a data key for envelope encryption.
        Returns both plaintext and encrypted versions.
        """
        params = {
            'KeyId': key_id,
            'KeySpec': 'AES_256'
        }
        if context:
            params['EncryptionContext'] = context
            
        response = self.kms.generate_data_key(**params)
        
        return {
            'plaintext_key': response['Plaintext'],
            'encrypted_key': response['CiphertextBlob'],
            'key_id': response['KeyId']
        }
    
    def set_key_policy(self, key_id: str, policy: dict):
        """
        Set a key policy to control access.
        """
        import json
        self.kms.put_key_policy(
            KeyId=key_id,
            PolicyName='default',  # AWS KMS only supports 'default'
            Policy=json.dumps(policy)
        )
 
# Example key policy enabling cross-account access
CROSS_ACCOUNT_POLICY = {
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "Enable IAM policies",
            "Effect": "Allow",
            "Principal": {"AWS": "arn:aws:iam::111111111111:root"},
            "Action": "kms:*",
            "Resource": "*"
        },
        {
            "Sid": "Allow use from other account",
            "Effect": "Allow",
            "Principal": {"AWS": "arn:aws:iam::222222222222:role/DataProcessing"},
            "Action": ["kms:Encrypt", "kms:Decrypt", "kms:GenerateDataKey"],
            "Resource": "*"
        }
    ]
}

GCP Cloud KMS Deep Dive

Google Cloud KMS offers a hierarchical key organization model and strong integration with GCP services. Its key ring and key version concepts provide natural organization for complex key estates.

GCP KMS hierarchy:

Project: Top-level container (GCP project)
Location: Geographic region for key storage
Key Ring: Logical grouping of keys (cannot be deleted once created)
Crypto Key: The key itself (symmetric or asymmetric)
Crypto Key Version: Specific version of key material (enables rotation)

gcp_kms_operations
Python
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
from google.cloud import kms
from google.cloud.kms_v1 import types
import base64
 
class GCPKMSManager:
    """
    GCP Cloud KMS operations for key management and cryptography.
    """
    
    def __init__(self, project_id: str, location: str = 'us-east1'):
        self.client = kms.KeyManagementServiceClient()
        self.project_id = project_id
        self.location = location
        self.location_path = f"projects/{project_id}/locations/{location}"
    
    def create_key_ring(self, ring_id: str) -> str:
        """Create a key ring to organize related keys."""
        key_ring = {}
        parent = self.location_path
        
        created = self.client.create_key_ring(
            request={
                "parent": parent,
                "key_ring_id": ring_id,
                "key_ring": key_ring
            }
        )
        return created.name
    
    def create_symmetric_key(self, ring_id: str, key_id: str,
                             rotation_days: int = 90) -> str:
        """
        Create a symmetric encryption key with automatic rotation.
        """
        key_ring_path = f"{self.location_path}/keyRings/{ring_id}"
        
        # Define key with rotation schedule
        from datetime import datetime, timedelta
        from google.protobuf import duration_pb2, timestamp_pb2
        
        crypto_key = {
            "purpose": types.CryptoKey.CryptoKeyPurpose.ENCRYPT_DECRYPT,
            "version_template": {
                "algorithm": types.CryptoKeyVersion.CryptoKeyVersionAlgorithm.GOOGLE_SYMMETRIC_ENCRYPTION
            },
            "rotation_period": duration_pb2.Duration(
                seconds=rotation_days * 24 * 60 * 60
            ),
            "next_rotation_time": timestamp_pb2.Timestamp(
                seconds=int((datetime.utcnow() + timedelta(days=rotation_days)).timestamp())
            )
        }
        
        created = self.client.create_crypto_key(
            request={
                "parent": key_ring_path,
                "crypto_key_id": key_id,
                "crypto_key": crypto_key
            }
        )
        return created.name
    
    def encrypt(self, key_path: str, plaintext: bytes) -> bytes:
        """Encrypt data using the primary key version."""
        response = self.client.encrypt(
            request={
                "name": key_path,
                "plaintext": plaintext
            }
        )
        return response.ciphertext
    
    def decrypt(self, key_path: str, ciphertext: bytes) -> bytes:
        """Decrypt data (automatically uses correct key version)."""
        response = self.client.decrypt(
            request={
                "name": key_path,
                "ciphertext": ciphertext
            }
        )
        return response.plaintext
 
# Usage
# manager = GCPKMSManager('my-project-id', 'us-east1')
# manager.create_key_ring('production-keys')
# key_path = manager.create_symmetric_key('production-keys', 'database-key')
# ciphertext = manager.encrypt(key_path, b'sensitive data')

Envelope Encryption with KMS

Both AWS KMS and GCP Cloud KMS are optimized for envelope encryption: using a KMS-managed key (KEK) to protect locally-generated data keys (DEKs). This pattern combines the security of HSM-backed keys with the performance of local encryption.

Why envelope encryption?

Performance: Local encryption is orders of magnitude faster than API calls to KMS
Cost: Fewer KMS API calls reduces cost (especially at high volume)
Availability: Local encryption works even during brief KMS outages
Flexibility: Each data element can have unique DEK without KMS key explosion

Converting Mermaid diagram...

Caching Data Keys

For high-throughput applications, cache plaintext DEKs in memory (with TTL) to reduce KMS calls. AWS Encryption SDK and Google Tink libraries implement secure caching. Balance cache TTL against security—longer caches mean more data encrypted with one DEK.

Multi-Region and Disaster Recovery

Global applications require keys available in multiple regions. Both AWS and GCP offer solutions, with different trade-offs.

AWS Multi-Region Keys

•Create primary key in one region
•Replicate to other regions as replica keys
•Same key ID works across all regions
•Encrypt in one region, decrypt in another
•Automatic sync of key metadata

GCP Global Keys

•Use 'global' location for key ring
•Key accessible from all regions
•Automatic geo-routing to nearest endpoint
•Simpler setup than AWS approach
•Consider data residency requirements

Disaster Recovery Planning

KMS is a dependency for decryption. Plan for: • Region outages: Multi-region keys or cross-region key replication • Account lockout: Break-glass procedures, escrow keys in separate account • Accidental deletion: Enable deletion protection, appropriate waiting periods

Summary: Cloud KMS Best Practices

Key Takeaways

•Use customer managed keys — AWS managed keys are convenient but limited. CMKs provide full control, audit, and cross-account capabilities.
•Implement envelope encryption — Use KMS for key protection, local libraries for data encryption. This optimizes performance and cost.
•Apply least-privilege policies — Key policies should grant minimum necessary permissions. Separate encryption and decryption where possible.
•Enable rotation — Automatic rotation (AWS) or versioning (GCP) limits exposure. Configure based on risk assessment.
•Use encryption context — Bind ciphertext to its purpose. Prevents misuse of encrypted data in unintended contexts.
•Plan for disaster — Multi-region keys, cross-account access, and documented recovery procedures prevent key loss from causing data loss.

Page Complete

You now understand cloud KMS architectures, AWS KMS and GCP Cloud KMS specifics, and practical implementation patterns. Next, we'll explore Hardware Security Modules (HSMs) for scenarios requiring the highest level of key protection.