Auditing and Logging

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113

# NIST 800-53 Rev 5 - Audit and Accountability Family (AU)
# Key controls for compliance mapping
 
controls:
  AU-2:
    name: "Audit Events"
    description: "Identify auditable events"
    requirements:
      - "Coordinate audit event selection with other entities"
      - "Include rationale for why audit events are deemed adequate"
      - "Include auditable events listed in 10.2.1 (if PCI applicable)"
    implementation: |
      Document auditable event categories:
      - Authentication events (success/failure)
      - Authorization decisions
      - System administrator actions
      - Data access and modifications
      - Security policy changes
 
  AU-3:
    name: "Content of Audit Records"
    description: "What information to include in each record"
    requirements:
      - "Type of event"
      - "Time of event"
      - "Source of event"
      - "Outcome (success/failure)"
      - "Identity of subject"
      - "Objects accessed or modified"
    implementation: |
      Minimum audit record fields:
      - timestamp (UTC, microsecond precision)
      - event_type (authentication, authorization, etc.)
      - outcome (success, failure, error)
      - subject (user ID, process ID, service account)
      - source (hostname, IP address, terminal)
      - target (file path, database table, API endpoint)
      - details (additional context as JSON)
 
  AU-4:
    name: "Audit Storage Capacity"
    description: "Allocate sufficient audit log storage"
    requirements:
      - "Allocate storage based on audit record size and retention"
      - "Transfer to alternate storage if primary fills"
      - "Alert when storage reaches threshold"
    implementation: |
      - Calculate: daily_events * record_size * retention_days * 1.5
      - Set alerts at 70%, 80%, 90% capacity
      - Automatic transfer to archive at threshold
 
  AU-5:
    name: "Response to Audit Processing Failures"
    description: "Define behavior when audit fails"
    requirements:
      - "Alert personnel when audit fails"
      - "Take defined actions (shutdown, switch destination)"
    implementation: |
      failure_actions:
        disk_full: "halt_non_essential_services"
        daemon_crash: "restart_attempt_then_alert"
        network_unavailable: "local_buffer_then_retry"
 
  AU-6:
    name: "Audit Review, Analysis, and Reporting"
    description: "Regularly review audit records"
    requirements:
      - "Review and analyze audit records at defined frequency"
      - "Report findings to appropriate personnel"
      - "Integrate audit review with incident response"
    implementation: |
      review_schedule:
        security_events: "real-time SIEM alerting"
        admin_actions: "daily manual review"
        access_patterns: "weekly statistical analysis"
        full_audit: "monthly compliance review"
 
  AU-9:
    name: "Protection of Audit Information"
    description: "Protect audit records from unauthorized access/modification"
    requirements:
      - "Protect audit information from unauthorized access"
      - "Protect audit tools from unauthorized modification"
      - "Alert on unauthorized access attempts"
    implementation: |
      - Separate log storage with restricted access
      - Immutable attributes on log files
      - Cryptographic signing of audit records
      - Network segmentation for log infrastructure
 
  AU-11:
    name: "Audit Record Retention"
    description: "Retain audit records per retention schedule"
    requirements:
      - "Retain audit records for defined period"
      - "Ensure records are retrievable for investigation"
    implementation: |
      retention_tiers:
        online: "90 days in primary SIEM"
        warm: "1 year in searchable archive"
        cold: "7 years in compliance archive"
        
  AU-12:
    name: "Audit Generation"
    description: "Generate audit records for defined events"
    requirements:
      - "Provide audit record generation capability"
      - "Allow authorized personnel to configure auditable events"
      - "Generate audit records with required content"
    implementation: |
      - Deploy audit agents on all systems
      - Centralized audit policy management
      - Verify audit generation with periodic tests

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231

# Terraform configuration for compliant audit infrastructure
# Satisfies PCI DSS, HIPAA, and SOC 2 requirements
 
# ================================================
# ENCRYPTED, IMMUTABLE LOG STORAGE
# ================================================
 
# S3 bucket with compliance features
resource "aws_s3_bucket" "audit_logs" {
  bucket = "company-security-audit-logs"
  
  # Prevent accidental deletion
  force_destroy = false
  
  tags = {
    Purpose     = "Compliance Audit Logs"
    Compliance  = "PCI-DSS,HIPAA,SOC2"
    DataClass   = "Confidential"
  }
}
 
# Enable versioning (protects against accidental deletion/modification)
resource "aws_s3_bucket_versioning" "audit_logs" {
  bucket = aws_s3_bucket.audit_logs.id
  
  versioning_configuration {
    status = "Enabled"
  }
}
 
# Server-side encryption
resource "aws_s3_bucket_server_side_encryption_configuration" "audit_logs" {
  bucket = aws_s3_bucket.audit_logs.id
 
  rule {
    apply_server_side_encryption_by_default {
      sse_algorithm     = "aws:kms"
      kms_master_key_id = aws_kms_key.audit_logs.arn
    }
    bucket_key_enabled = true
  }
}
 
# Object Lock for immutability (WORM)
resource "aws_s3_bucket_object_lock_configuration" "audit_logs" {
  bucket = aws_s3_bucket.audit_logs.id
 
  rule {
    default_retention {
      mode = "COMPLIANCE"  # Cannot be removed, even by root
      years = 7
    }
  }
}
 
# Lifecycle policy for tiered storage
resource "aws_s3_bucket_lifecycle_configuration" "audit_logs" {
  bucket = aws_s3_bucket.audit_logs.id
 
  rule {
    id     = "compliance-tiering"
    status = "Enabled"
 
    transition {
      days          = 90
      storage_class = "STANDARD_IA"
    }
 
    transition {
      days          = 365
      storage_class = "GLACIER"
    }
    
    # No expiration - compliance storage is permanent
    # Deletion handled by separate compliance process
  }
}
 
# Block all public access
resource "aws_s3_bucket_public_access_block" "audit_logs" {
  bucket = aws_s3_bucket.audit_logs.id
 
  block_public_acls       = true
  block_public_policy     = true
  ignore_public_acls      = true
  restrict_public_buckets = true
}
 
# ================================================
# ENCRYPTION KEY MANAGEMENT
# ================================================
 
resource "aws_kms_key" "audit_logs" {
  description             = "KMS key for audit log encryption"
  deletion_window_in_days = 30
  enable_key_rotation     = true
  
  # Key policy restricting access
  policy = jsonencode({
    Version = "2012-10-17"
    Statement = [
      {
        Sid    = "Allow administration"
        Effect = "Allow"
        Principal = {
          AWS = "arn:aws:iam::${data.aws_caller_identity.current.account_id}:role/SecurityAdmin"
        }
        Action   = ["kms:*"]
        Resource = "*"
      },
      {
        Sid    = "Allow audit log service usage"
        Effect = "Allow"
        Principal = {
          AWS = "arn:aws:iam::${data.aws_caller_identity.current.account_id}:role/AuditLogService"
        }
        Action = [
          "kms:Encrypt",
          "kms:GenerateDataKey*"
        ]
        Resource = "*"
      }
    ]
  })
  
  tags = {
    Purpose = "Audit Log Encryption"
  }
}
 
# ================================================
# ACCESS CONTROL
# ================================================
 
# IAM policy for log writers (minimal permissions)
resource "aws_iam_policy" "audit_log_writer" {
  name        = "AuditLogWriter"
  description = "Allows writing audit logs only"
  
  policy = jsonencode({
    Version = "2012-10-17"
    Statement = [
      {
        Effect = "Allow"
        Action = [
          "s3:PutObject"
        ]
        Resource = "${aws_s3_bucket.audit_logs.arn}/*"
        Condition = {
          StringEquals = {
            "s3:x-amz-server-side-encryption" = "aws:kms"
          }
        }
      }
    ]
  })
}
 
# IAM policy for log readers (SOC analysts)
resource "aws_iam_policy" "audit_log_reader" {
  name        = "AuditLogReader"
  description = "Allows reading audit logs for investigation"
  
  policy = jsonencode({
    Version = "2012-10-17"
    Statement = [
      {
        Effect = "Allow"
        Action = [
          "s3:GetObject",
          "s3:ListBucket"
        ]
        Resource = [
          aws_s3_bucket.audit_logs.arn,
          "${aws_s3_bucket.audit_logs.arn}/*"
        ]
      },
      {
        Effect = "Allow"
        Action = [
          "kms:Decrypt"
        ]
        Resource = aws_kms_key.audit_logs.arn
      }
    ]
  })
}
 
# ================================================
# MONITORING AND ALERTING
# ================================================
 
# CloudWatch alarm for audit destination health
resource "aws_cloudwatch_metric_alarm" "audit_destination_failures" {
  alarm_name          = "AuditDestinationFailures"
  comparison_operator = "GreaterThanThreshold"
  evaluation_periods  = 1
  metric_name         = "AuditLogDeliveryFailures"
  namespace           = "Security/AuditLogs"
  period              = 300
  statistic           = "Sum"
  threshold           = 0
  alarm_description   = "Alert if any audit log delivery fails"
  
  alarm_actions = [
    aws_sns_topic.security_alerts.arn
  ]
  
  ok_actions = [
    aws_sns_topic.security_alerts.arn
  ]
}
 
# CloudTrail for log access auditing (audit the audits)
resource "aws_cloudtrail" "audit_log_access" {
  name                          = "audit-log-access-trail"
  s3_bucket_name                = aws_s3_bucket.cloudtrail_logs.id
  include_global_service_events = true
  is_multi_region_trail         = true
  enable_log_file_validation    = true  # Detect tampering
  
  event_selector {
    read_write_type           = "All"
    include_management_events = true
 
    data_resource {
      type   = "AWS::S3::Object"
      values = ["${aws_s3_bucket.audit_logs.arn}/"]
    }
  }
}

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157

# Security Audit Evidence Package
 
## 1. Audit and Logging Policy Document
 
**Document:** POL-SEC-003 - Audit and Logging Policy
**Version:** 2.4
**Last Review:** 2024-01-15
**Next Review:** 2025-01-15
**Approved By:** CISO
 
### Policy Statements
 
1.1 All systems processing [sensitive data type] SHALL log authentication events.
1.2 All administrative actions SHALL be logged with timestamp, user, and action detail.
1.3 Logs SHALL be retained for minimum [X] years per regulatory requirement.
1.4 Log integrity SHALL be protected using cryptographic controls.
1.5 Log review SHALL occur [frequency] by qualified personnel.
 
---
 
## 2. System Architecture Documentation
 
**Diagram:** audit-architecture-diagram.pdf
**Systems in Scope:**
- Web servers (AWS EC2)
- Database servers (AWS RDS)
- Application servers (ECS)
- Network devices (Palo Alto)
- Cloud infrastructure (AWS CloudTrail, GuardDuty)
 
**Log Flow:**
Sources → Fluentd Agents → Kafka → Logstash → Elasticsearch → S3 Archive
 
---
 
## 3. Configuration Evidence
 
### 3.1 Linux Auditd Configuration
```
# Output of: cat /etc/audit/auditd.conf
# Date captured: 2024-01-15
# System: app-server-prod-01
log_file = /var/log/audit/audit.log
max_log_file = 50
num_logs = 10
max_log_file_action = ROTATE
space_left = 75
space_left_action = email
admin_space_left = 50
admin_space_left_action = single
disk_full_action = HALT
[... full output ...]
```
 
### 3.2 Windows Audit Policy
```
# Output of: auditpol /get /category:*
# Date captured: 2024-01-15
# System: dc-prod-01
 
System audit policy for DC-PROD-01:
  Account Logon
    Credential Validation              Success and Failure
    Kerberos Authentication Service    Success and Failure
  Logon/Logoff
    Logon                              Success and Failure
    Logoff                             Success
    Special Logon                      Success
[... full output ...]
```
 
---
 
## 4. Retention Evidence
 
### 4.1 Retention Policy
| Log Type | Online Retention | Archive Retention | Verification Date |
|----------|------------------|-------------------|-------------------|
| Authentication | 90 days | 7 years | 2024-01-15 |
| Privileged Access | 180 days | 7 years | 2024-01-15 |
| File Access | 30 days | 1 year | 2024-01-15 |
 
### 4.2 Retrieval Test
**Test Date:** 2024-01-15
**Tested By:** jsmith@security.com
 
**Test 1:** Retrieve authentication logs from 18 months ago
- Date requested: 2022-07-15
- Search executed: [query]
- Result: 4,532 events retrieved successfully
- Screenshot: retrieval-test-001.png
 
**Test 2:** Retrieve privileged access logs from 3 years ago
- Date requested: 2021-01-15
- Search executed: [query]
- Result: 12,847 events retrieved successfully
- Screenshot: retrieval-test-002.png
 
---
 
## 5. Log Review Evidence
 
### 5.1 Review Schedule
| Review Type | Frequency | Responsible Party | Documentation |
|-------------|-----------|-------------------|---------------|
| Security alerts | Real-time | SOC Analysts | SIEM tickets |
| Admin activity | Daily | Security Engineer | Daily report |
| Access patterns | Weekly | Security Manager | Weekly report |
| Full audit | Monthly | Security Director | Monthly report |
 
### 5.2 Sample Review Documentation
**Week of:** 2024-01-08 to 2024-01-14
**Reviewed By:** analyst@security.com
 
**Summary:**
- Total alerts reviewed: 342
- True positives: 28
- False positives: 314
- Escalated to incident: 3
 
**Notable Findings:**
1. [Ticket SEC-2024-0042] Brute force attempt from [IP] - blocked
2. [Ticket SEC-2024-0045] Unusual file access by service account - investigated, authorized
3. [Ticket SEC-2024-0048] Potential data exfiltration - escalated to IR
 
---
 
## 6. Integrity Evidence
 
### 6.1 S3 Object Lock Configuration
```json
{
  "ObjectLockConfiguration": {
    "ObjectLockEnabled": "Enabled",
    "Rule": {
      "DefaultRetention": {
        "Mode": "COMPLIANCE",
        "Years": 7
      }
    }
  }
}
```
 
### 6.2 CloudTrail Log File Validation
**Test Date:** 2024-01-15
```
$ aws cloudtrail validate-logs --trail-arn arn:aws:cloudtrail:us-east-1:123456789:trail/audit-trail --start-time 2024-01-01
 
Validating log files for trail arn:aws:cloudtrail:us-east-1:123456789:trail/audit-trail between 2024-01-01T00:00:00Z and 2024-01-15T23:59:59Z
 
Results:
Log files validated: 15
Log files not validated: 0
Digest files validated: 15
Digest files not validated: 0
```

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119

# Chef InSpec profile for audit logging compliance
# Satisfies PCI DSS 10.x, HIPAA, and SOC 2 requirements
 
# Control: Audit daemon must be running
control 'audit-1.1' do
  impact 1.0
  title 'Audit daemon is running'
  desc 'The auditd service must be running to capture security events'
  tag 'pci-dss': '10.1'
  tag 'hipaa': '164.312(b)'
  
  describe service('auditd') do
    it { should be_installed }
    it { should be_enabled }
    it { should be_running }
  end
end
 
# Control: Audit configuration is immutable
control 'audit-1.2' do
  impact 1.0
  title 'Audit configuration is immutable'
  desc 'Audit rules cannot be modified without reboot'
  tag 'pci-dss': '10.5'
  
  describe auditd_conf do
    its('enabled') { should eq 2 }  # Immutable mode
  end
end
 
# Control: Required events are being audited
control 'audit-2.1' do
  impact 1.0
  title 'Authentication events are audited'
  desc 'All login/logout events must be captured'
  tag 'pci-dss': '10.2.1'
  tag 'hipaa': '164.308(a)(5)(ii)(C)'
  
  describe auditd_rules do
    its('lines') { should include('-w /var/log/lastlog -p wa -k logins') }
    its('lines') { should include('-w /var/run/faillock/ -p wa -k logins') }
  end
  
  describe file('/etc/audit/rules.d/security.rules') do
    its('content') { should match(/identity/) }
    its('content') { should match(/logins/) }
    its('content') { should match(/privileged/) }
  end
end
 
# Control: Log files are protected
control 'audit-3.1' do
  impact 1.0
  title 'Audit log files have correct permissions'
  desc 'Only root/audit group can access audit logs'
  tag 'pci-dss': '10.5.1'
  
  describe file('/var/log/audit/audit.log') do
    it { should exist }
    its('mode') { should cmp '0600' }
    its('owner') { should eq 'root' }
    its('group') { should eq 'root' }
  end
end
 
# Control: Time synchronization is configured
control 'audit-4.1' do
  impact 0.7
  title 'Time synchronization is enabled'
  desc 'NTP must be configured for accurate timestamps'
  tag 'pci-dss': '10.4'
  
  describe service('chronyd') do
    it { should be_running }
    it { should be_enabled }
  end
  
  describe command('chronyc tracking') do
    its('stdout') { should match(/Leap statuss+:s+Normal/) }
  end
end
 
# Control: Logs are being forwarded
control 'audit-5.1' do
  impact 1.0
  title 'Logs are forwarded to central collector'
  desc 'Logs must be sent to SIEM for review and long-term storage'
  tag 'pci-dss': '10.5.3'
  
  describe service('fluent-bit') do
    it { should be_running }
    it { should be_enabled }
  end
  
  # Verify recent successful forwarding
  describe command('journalctl -u fluent-bit --since "1 hour ago" | grep -c "output:es"') do
    its('stdout.to_i') { should be > 0 }
  end
end
 
# Control: Logs are retained appropriately
control 'audit-6.1' do
  impact 1.0
  title 'Log retention meets requirements'
  desc 'Logs must be available for minimum retention period'
  tag 'pci-dss': '10.7'
  tag 'hipaa': '164.530(j)'
  
  describe auditd_conf do
    its('max_log_file') { should be >= 50 }
    its('num_logs') { should be >= 10 }
    its('max_log_file_action') { should eq 'ROTATE' }
  end
  
  # Verify we have logs from 90+ days ago (online retention)
  describe command('find /var/log/audit -name "audit.log.*" -mtime +90 | wc -l') do
    its('stdout.to_i') { should be > 0 }
  end
end

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79

# Incident Logging Procedure
 
## 1. Evidence Preservation (First 1 Hour)
 
### Immediate Actions
- [ ] Activate log preservation hold (prevent rotation/deletion)
- [ ] Capture current log files with timestamps
- [ ] Document chain of custody
- [ ] Take memory snapshots if malware suspected
- [ ] Preserve network traffic captures
 
### Commands for Evidence Preservation
```bash
# Prevent log rotation
chattr +a /var/log/audit/audit.log
systemctl stop logrotate
 
# Create forensic copy with verification
DATE=$(date +%Y%m%d_%H%M%S)
INCIDENT_ID="INC-2024-0042"
 
mkdir -p /evidence/$INCIDENT_ID
cp -a /var/log/audit/ /evidence/$INCIDENT_ID/audit_logs/
sha256sum /evidence/$INCIDENT_ID/audit_logs/* > /evidence/$INCIDENT_ID/hashes.txt
 
# Document collection
cat << EOF > /evidence/$INCIDENT_ID/collection_log.txt
Incident: $INCIDENT_ID
Collected By: $(whoami)
Collected At: $(date -u)
System: $(hostname)
Hashes: See hashes.txt
EOF
```
 
## 2. Incident Timeline Logging
 
### Required Documentation
Every action during incident response must be logged:
 
| Timestamp (UTC) | Actor | Action | Result | Reference |
|-----------------|-------|--------|--------|-----------|
| 2024-01-15T14:32:00Z | SOC Analyst | Detected anomaly in SIEM | Alert SEC-9876 | [SIEM screenshot] |
| 2024-01-15T14:35:00Z | Incident Lead | Declared incident | INC-2024-0042 | [Incident ticket] |
| 2024-01-15T14:40:00Z | IR Team | Isolated affected host | Network segment quarantined | [Firewall change record] |
 
## 3. Regulatory Notification Log
 
### GDPR 72-Hour Timeline
- [ ] Hour 0: Breach confirmed
- [ ] Hour 4: Scope assessment complete
- [ ] Hour 12: Impact assessment complete
- [ ] Hour 24: Draft notification prepared
- [ ] Hour 48: Legal review complete
- [ ] Hour 72: Submission to supervisory authority
 
### Notification Record
```
Notification Date: 2024-01-17
Notified Authority: ICO (UK)
Reference: [submission number]
Submitted By: DPO Name
Backup Confirmation: [screenshot/email]
```
 
## 4. forensic Evidence Chain of Custody
 
### Chain of Custody Form
| Evidence ID | Description | Collected By | Date/Time | Transferred To | Date/Time | Storage Location |
|-------------|-------------|--------------|-----------|----------------|-----------|------------------|
| EVD-001 | Audit logs Jan 1-15 | jsmith | 2024-01-15 15:00 UTC | forensics team | 2024-01-15 16:00 UTC | Secure evidence locker |
 
### Evidence Integrity Verification
```bash
# Verify evidence integrity before analysis
cd /evidence/$INCIDENT_ID
sha256sum -c hashes.txt
# Expected output: All files OK
```