Auditing And Logging - Learning Module

Loading content...

0/227

Intrusion Detection

The First Line of Active Defense

In March 2023, a hospital discovered that attackers had been inside their network for 247 days before being detected—far longer than the global average of 197 days. During that time, the attackers had exfiltrated patient records, installed backdoors in multiple systems, and positioned themselves for a ransomware attack. The breach was finally detected not by their security team, but by an FBI notification based on dark web monitoring.

Intrusion detection is the discipline of identifying malicious activity as early as possible in the attack lifecycle. Every hour an attacker remains undetected expands the scope of damage and cost of remediation. The difference between detecting an attack in minutes versus months is often the difference between a minor incident and a catastrophic breach.

This page explores the complete landscape of intrusion detection—from theoretical foundations to practical implementation, from signature-based detection to behavioral analytics, from host-based sensors to network monitoring. You'll learn how to build detection capabilities that find attackers hiding in the noise of normal operations.

What You Will Learn

By the end of this page, you will understand: (1) The fundamental principles and types of intrusion detection, (2) Signature-based detection with pattern matching, (3) Anomaly-based detection with baselines and deviations, (4) Host-based intrusion detection (HIDS), (5) Network-based intrusion detection (NIDS), (6) Modern approaches including UEBA and threat hunting, and (7) Detection engineering best practices.

Intrusion Detection Fundamentals

An Intrusion Detection System (IDS) monitors systems or networks for signs of malicious activity, policy violations, or security threats. Unlike firewalls that prevent certain traffic, IDS analyzes activity after it has occurred (or is occurring) to identify attacks.

Detection Taxonomy

Intrusion Detection System Classification
Classification	Types	Description
By Location	HIDS, NIDS, Cloud IDS	Where sensors are deployed (host, network, cloud)
By Method	Signature, Anomaly, Hybrid	How detection is performed
By Response	Passive (IDS), Active (IPS)	Whether system takes action
By Analysis Time	Real-time, Batch/Historical	When analysis occurs

The Detection Problem

Intrusion detection is fundamentally a classification problem with four possible outcomes:

Detection Outcome Matrix
	Actual Attack	No Attack
Alert Raised	True Positive (TP) ✓	False Positive (FP) - Alert fatigue
No Alert	False Negative (FN) - Missed attack!	True Negative (TN) ✓

Critical Metrics

Detection Performance Metrics

•True Positive Rate (Recall) = TP / (TP + FN) — Percentage of real attacks detected. Higher is better.
•False Positive Rate = FP / (FP + TN) — Percentage of benign activity flagged as attacks. Lower is better.
•Precision = TP / (TP + FP) — When we alert, how often is it a real attack. Higher is better.
•Mean Time to Detect (MTTD) — Average time from attack start to detection. Lower is better.
•Detection Coverage — Percentage of ATT&CK techniques with at least one detection rule.

The False Positive Crisis

A typical enterprise SOC processes 10,000+ alerts per day, of which 99%+ are false positives. This alert fatigue causes analysts to ignore or dismiss alerts, creating an environment where real attacks are lost in the noise. Reducing false positives while maintaining detection coverage is the central challenge of intrusion detection. A detection that generates 100 alerts per day with 1% true positive rate wastes 99 analyst reviews to find 1 real issue.

The Base Rate Fallacy

Even a highly accurate detection system can be impractical due to the base rate of attacks. Consider:

base-rate-example.py
Python
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
"""
The base rate fallacy in intrusion detection.
Even 99% accurate detections can be mostly false positives.
"""
 
# Scenario: Enterprise with 1,000,000 events per day
total_events = 1_000_000
 
# Attack base rate: 0.01% of events are malicious
attack_rate = 0.0001
actual_attacks = int(total_events * attack_rate)  # 100 attacks
normal_events = total_events - actual_attacks     # 999,900 normal
 
# Detection system with:
# - 99% True Positive Rate (catches 99% of attacks)
# - 1% False Positive Rate (flags 1% of normal events)
true_positive_rate = 0.99
false_positive_rate = 0.01
 
# Calculate alerts
true_positives = int(actual_attacks * true_positive_rate)     # 99 real attacks caught
false_positives = int(normal_events * false_positive_rate)    # 9,999 false alarms!
 
total_alerts = true_positives + false_positives  # 10,098 alerts
 
# The problem:
precision = true_positives / total_alerts  # 0.98% !!
 
print(f"Total alerts: {total_alerts:,}")
print(f"True positives: {true_positives}")
print(f"False positives: {false_positives:,}")
print(f"Precision: {precision:.2%}")
print(f"\nResult: Only ~1% of alerts are real attacks!")
print(f"Analysts must review {false_positives:,} false alarms to find {true_positives} real attacks")
 
# To achieve 50% precision (half of alerts are real):
# FP rate must be: attacks / normal_events = 100 / 999,900 = 0.01%
# This means 99.99% accuracy on normal events is required!

Context is Everything

The base rate fallacy shows why context matters so much. A failed login isn't concerning; 50 failed logins from the same IP in one minute is. Detections based on patterns, sequences, and context have much better signal-to-noise than single-event rules.

Signature-Based Detection

Signature-based detection (also called misuse detection) identifies known attack patterns by matching observed activity against a database of attack signatures. It's the most common and reliable form of intrusion detection.

How Signatures Work

A signature defines the characteristics of a known attack:

•Network signatures — Specific byte patterns in packets, protocol anomalies, malicious payloads
•Host signatures — Suspicious process behaviors, file hashes, registry modifications
•Log signatures — Specific event patterns, field value combinations, sequences

Network Signature Example: Suricata Rules

Suricata (and Snort) use a powerful rule language for network detection:

suricata.rules

Suricata Rules

# Suricata IDS rules for common attacks
 
# ================================================
# SQL Injection Detection
# ================================================
# Detect SQL injection attempts in HTTP requests
alert http any any -> any any (
    msg:"ATTACK SQL Injection Attempt - UNION SELECT";
    flow:established,to_server;
    http.uri;
    content:"UNION"; nocase;
    content:"SELECT"; nocase; distance:0;
    pcre:"/UNION\s+(ALL\s+)?SELECT/i";
    classtype:web-application-attack;
    sid:1000001; rev:3;
)
 
# SQL injection in POST body
alert http any any -> any any (
    msg:"ATTACK SQL Injection in POST Body";
    flow:established,to_server;
    http.method; content:"POST";
    http.request_body;
    pcre:"/('|%27)\s*(OR|AND)\s*('|%27)?\d*\s*[=<>]/i";
    classtype:web-application-attack;
    sid:1000002; rev:2;
)
 
# ================================================
# Command Injection Detection
# ================================================
# Detect command injection via semicolon or pipe
alert http any any -> any any (
    msg:"ATTACK Command Injection Attempt";
    flow:established,to_server;
    http.uri;
    pcre:"/[;|&]\s*(cat|ls|id|whoami|uname|pwd|nc|wget|curl)/i";
    classtype:web-application-attack;
    sid:1000003; rev:1;
)
 
# ================================================
# Reconnaissance Detection
# ================================================
# Aggressive scanning behavior (many SYN to different ports)
alert tcp any any -> $HOME_NET any (
    msg:"RECON Possible Port Scan - Multiple Ports";
    flags:S;
    threshold: type both, track by_src, count 50, seconds 10;
    classtype:attempted-recon;
    sid:1000010; rev:1;
)
 
# ================================================
# Malware C2 Detection
# ================================================
# Known C2 beacon pattern (Cobalt Strike default)
alert http any any -> any any (
    msg:"MALWARE Cobalt Strike Beacon C2 Traffic";
    flow:established,to_server;
    content:"/submit.php?id="; http.uri;
    pcre:"/\/submit\.php\?id=[0-9]{5,10}$/";
    http.user_agent; content:"Mozilla/"; startswith;
    classtype:trojan-activity;
    priority:1;
    sid:1000020; rev:2;
)
 
# ================================================
# Lateral Movement Detection
# ================================================
# PsExec-style remote execution
alert tcp any any -> any 445 (
    msg:"LATERAL PsExec-style Remote Service Install";
    flow:established,to_server;
    content:"|FF|SMB"; depth:4;
    content:"|25 00|"; distance:0; within:2;  # Tree Connect
    content:"IPC$"; distance:0; within:100;
    content:"PSEXESVC"; distance:0; within:200;
    classtype:trojan-activity;
    sid:1000030; rev:1;
)
 
# ================================================
# DNS Tunneling Detection
# ================================================
# Unusually long DNS queries (data exfiltration)
alert dns any any -> any any (
    msg:"EXFIL Potential DNS Tunneling - Long Query";
    dns.query;
    pcre:"/^[a-z0-9]{30,}/i";  # 30+ char subdomain
    threshold: type threshold, track by_src, count 10, seconds 60;
    classtype:bad-unknown;
    sid:1000040; rev:1;
)

Log-Based Signature Detection: Sigma Rules

Sigma is the open signature format for log-based detection, allowing platform-agnostic rule sharing:

mimikatz-detection.yaml
YAML (Sigma)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
# Sigma rule for detecting Mimikatz usage
# Can be converted to Splunk, Elastic, QRadar, etc.
 
title: Mimikatz Credential Dumping
id: d9dcbe87-0e5e-4a21-bb5d-3f1ea4e36c73
status: stable
description: |
    Detects usage of Mimikatz for credential theft through
    multiple detection methods including process creation,
    command line arguments, and loaded modules.
author: Security Team
date: 2024/01/15
modified: 2024/01/15
references:
    - https://attack.mitre.org/techniques/T1003/001/
    - https://github.com/gentilkiwi/mimikatz
tags:
    - attack.credential_access
    - attack.t1003.001
logsource:
    category: process_creation
    product: windows
detection:
    # Method 1: Known process names
    selection_name:
        Image|endswith:
            - '\mimikatz.exe'
            - '\mimi.exe'
            - '\m.exe'
    
    # Method 2: Command line indicators
    selection_cmdline:
        CommandLine|contains:
            - 'sekurlsa::logonpasswords'
            - 'sekurlsa::wdigest'
            - 'sekurlsa::kerberos'
            - 'lsadump::sam'
            - 'lsadump::dcsync'
            - 'token::elevate'
            - 'privilege::debug'
    
    # Method 3: Original filename in PE header
    selection_original:
        OriginalFileName: 'mimikatz.exe'
    
    # Method 4: Known hashes (updates frequently)
    selection_hash:
        Hashes|contains:
            - 'SHA256=...'  # Add known hashes
    
    condition: selection_name or selection_cmdline or selection_original or selection_hash
 
fields:
    - User
    - CommandLine
    - ParentImage
    - ParentCommandLine
    - Computer
    
falsepositives:
    - Legitimate credential management tools
    - Security testing with authorized tools
    
level: critical

Signature-Based Advantages

•Low false positive rate for known attacks
•Fast processing — pattern matching is efficient
•Easy to understand — human-readable rules
•Actionable alerts — clear attack identification
•Community sharing — open rule repositories

Signature-Based Limitations

•Cannot detect novel attacks — zero-days missed
•Evasion through modification — slight changes bypass
•Signature maintenance burden — constant updates
•Encrypted traffic blind spots — can't inspect TLS
•Polymorphic malware — self-modifying code evades

Anomaly-Based Detection

Anomaly-based detection (also called behavioral detection) learns what "normal" looks like and alerts on deviations. Unlike signatures, it can detect previously unknown attacks—but at the cost of higher false positive rates.

The Anomaly Detection Process

Converting Mermaid diagram...

Types of Anomaly Detection

1. Statistical Anomaly Detection

Simplest approach: flag values that deviate significantly from historical norms.

statistical-anomaly.py
Python
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
"""
Statistical anomaly detection for security metrics.
Uses z-score and IQR methods for detecting outliers.
"""
 
import numpy as np
from collections import deque
from typing import Tuple, Optional
 
class StatisticalAnomalyDetector:
    """
    Detects anomalies using statistical methods.
    Maintains rolling baselines for adaptive detection.
    """
    
    def __init__(self, window_size: int = 1000, z_threshold: float = 3.0):
        """
        Initialize detector with rolling window size and threshold.
        
        Args:
            window_size: Number of historical values to consider
            z_threshold: Number of standard deviations for anomaly
        """
        self.window_size = window_size
        self.z_threshold = z_threshold
        self.values = deque(maxlen=window_size)
        
    def add_value(self, value: float) -> Tuple[bool, Optional[float]]:
        """
        Add a new value and check if it's anomalous.
        
        Returns:
            (is_anomaly, z_score)
        """
        if len(self.values) < 30:  # Need minimum data for baseline
            self.values.append(value)
            return (False, None)
        
        # Calculate baseline statistics
        mean = np.mean(self.values)
        std = np.std(self.values)
        
        if std == 0:  # Avoid division by zero
            std = 0.0001
        
        # Calculate z-score
        z_score = (value - mean) / std
        
        # Check for anomaly
        is_anomaly = abs(z_score) > self.z_threshold
        
        # Add to rolling window (even if anomaly)
        self.values.append(value)
        
        return (is_anomaly, z_score)
 
 
class MultiMetricAnomalyDetector:
    """
    Detects anomalies across multiple security metrics.
    """
    
    def __init__(self):
        self.metrics = {
            'failed_logins': StatisticalAnomalyDetector(window_size=1440, z_threshold=3.0),
            'bytes_transferred': StatisticalAnomalyDetector(window_size=1440, z_threshold=4.0),
            'unique_destinations': StatisticalAnomalyDetector(window_size=1440, z_threshold=3.5),
            'after_hours_activity': StatisticalAnomalyDetector(window_size=720, z_threshold=2.5),
        }
        
    def check(self, metric_name: str, value: float) -> dict:
        """Check a metric value for anomalies."""
        if metric_name not in self.metrics:
            raise ValueError(f"Unknown metric: {metric_name}")
        
        is_anomaly, z_score = self.metrics[metric_name].add_value(value)
        
        return {
            'metric': metric_name,
            'value': value,
            'is_anomaly': is_anomaly,
            'z_score': z_score,
            'severity': self._calculate_severity(z_score) if z_score else 'unknown'
        }
    
    def _calculate_severity(self, z_score: float) -> str:
        """Map z-score to severity level."""
        abs_z = abs(z_score)
        if abs_z > 5:
            return 'critical'
        elif abs_z > 4:
            return 'high'
        elif abs_z > 3:
            return 'medium'
        else:
            return 'low'
 
 
# Example usage
if __name__ == "__main__":
    detector = MultiMetricAnomalyDetector()
    
    # Simulate normal traffic
    import random
    for _ in range(100):
        detector.check('failed_logins', random.gauss(10, 2))
    
    # Simulate brute force attack
    result = detector.check('failed_logins', 150)  # 150 failures suddenly
    print(f"Result: {result}")
    # Output: {'metric': 'failed_logins', 'value': 150, 'is_anomaly': True, 
    #          'z_score': 35.2, 'severity': 'critical'}

2. Behavioral Baselines

More sophisticated approach: learn typical patterns for each entity (user, host, application).

Behavioral Baseline Examples
Entity	Baseline Features	Anomaly Example
User	Login times, locations, accessed resources	Admin logging in from foreign country at 3 AM
Host	Normal processes, network connections, resource usage	Web server making outbound connections to unknown IPs
Application	API call patterns, data volumes, error rates	Database reading 10x normal data volume
Network	Traffic volumes, protocol mix, destination frequency	Internal host suddenly contacting 1000 external IPs

3. Machine Learning Approaches

Modern IDS increasingly use ML for anomaly detection:

•Clustering — Group similar behaviors; outliers are anomalies (K-means, DBSCAN)
•Classification — Train on labeled attack/normal data (Random Forest, XGBoost)
•Autoencoders — Learn to compress/decompress normal data; high reconstruction error = anomaly
•LSTM/Sequence Models — Learn temporal patterns; detect unusual event sequences
•Graph Neural Networks — Model entity relationships; detect unusual graph patterns

The Baseline Poisoning Problem

If an attacker gains access and operates slowly ('low and slow' attack), their malicious behavior becomes part of the baseline over time. After weeks of small data exfiltration, the anomaly detector considers it 'normal.' This is why anomaly detection must be paired with periodic baseline review and signature-based detection.

Host-Based Intrusion Detection (HIDS)

Host-based Intrusion Detection Systems (HIDS) run on individual hosts and monitor local activity. They have visibility into encrypted traffic (after decryption), host-level events, and file system changes that network-based systems cannot see.

HIDS Monitoring Capabilities

HIDS Monitoring Domains
Domain	What's Monitored	Attack Examples Detected
File Integrity	File hashes, permissions, ownership	Rootkits, webshells, config tampering
Process Activity	Process creation, termination, commands	Malware execution, suspicious tools
System Calls	Kernel API calls, syscall sequences	Exploitation, privilege escalation
Log Analysis	Local log files, journal entries	Authentication attacks, privilege abuse
Registry (Windows)	Registry modifications	Persistence, configuration changes
Network Connections	Outbound connections per process	C2 communication, data exfiltration

File Integrity Monitoring (FIM)

FIM is a foundational HIDS capability that detects unauthorized changes to critical files:

ossec-fim.conf
XML (OSSEC)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
<!-- OSSEC HIDS File Integrity Monitoring Configuration -->
<!-- /var/ossec/etc/ossec.conf -->
 
<ossec_config>
  <syscheck>
    <!-- How often to run FIM scans (in seconds) -->
    <frequency>7200</frequency>
    
    <!-- Enable real-time monitoring (requires inotify) -->
    <scan_on_start>yes</scan_on_start>
    
    <!-- ========================================== -->
    <!-- Critical system directories to monitor -->
    <!-- ========================================== -->
    
    <!-- Binary directories - any change is suspicious -->
    <directories check_all="yes" realtime="yes">/bin</directories>
    <directories check_all="yes" realtime="yes">/sbin</directories>
    <directories check_all="yes" realtime="yes">/usr/bin</directories>
    <directories check_all="yes" realtime="yes">/usr/sbin</directories>
    
    <!-- System configuration -->
    <directories check_all="yes" realtime="yes">/etc</directories>
    
    <!-- Startup scripts -->
    <directories check_all="yes" realtime="yes">/etc/init.d</directories>
    <directories check_all="yes" realtime="yes">/etc/systemd</directories>
    
    <!-- Kernel and boot -->
    <directories check_all="yes" realtime="yes">/boot</directories>
    
    <!-- Library directories -->
    <directories check_all="yes">/lib</directories>
    <directories check_all="yes">/lib64</directories>
    <directories check_all="yes">/usr/lib</directories>
    
    <!-- Web server directories (webshell detection) -->
    <directories check_all="yes" realtime="yes">/var/www</directories>
    <directories check_all="yes" realtime="yes">/var/www/html</directories>
    
    <!-- SSH configuration -->
    <directories check_all="yes" realtime="yes">/root/.ssh</directories>
    <directories check_all="yes" realtime="yes"
        report_changes="yes">/etc/ssh</directories>
    
    <!-- Cron directories (persistence detection) -->
    <directories check_all="yes" realtime="yes">/etc/cron.d</directories>
    <directories check_all="yes" realtime="yes">/etc/cron.daily</directories>
    <directories check_all="yes" realtime="yes">/var/spool/cron</directories>
    
    <!-- ========================================== -->
    <!-- Directories to ignore -->
    <!-- ========================================== -->
    
    <ignore>/etc/mtab</ignore>
    <ignore>/etc/hosts.deny</ignore>
    <ignore>/etc/adjtime</ignore>
    <ignore>/etc/resolv.conf</ignore>
    
    <!-- Log files change frequently, don't monitor -->
    <ignore type="sregex">/var/log/*</ignore>
    
    <!-- Ignore backup files -->
    <ignore type="sregex">.*.swp$</ignore>
    <ignore type="sregex">.*~$</ignore>
    
    <!-- ========================================== -->
    <!-- What attributes to check -->
    <!-- ========================================== -->
    <!-- 
      check_all: permissions, owner, group, size, hash
      realtime: Use inotify for immediate detection
      report_changes: Log file content differences
    -->
    
  </syscheck>
</ossec_config>

Process and Command Monitoring

Monitoring process execution captures attacker actions in real-time:

process-monitor.yaml
YAML
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
# Detection rules for suspicious process activity
# Format compatible with Elastic Endpoint Security, Carbon Black, etc.
 
rules:
  # ========================================
  # Reconnaissance Tools
  # ========================================
  - name: "Suspicious Network Discovery"
    description: "Detect common network reconnaissance tools"
    condition:
      process.name in:
        - nmap
        - masscan
        - netdiscover
        - arp-scan
      OR
      process.command_line contains_any:
        - "net view"
        - "net group"
        - "net user /domain"
        - "nltest /dclist"
    severity: medium
    mitre: T1018
    
  # ========================================
  # Credential Access
  # ========================================
  - name: "Credential Dumping Tool Execution"
    description: "Detect execution of common credential theft tools"
    condition:
      process.name in:
        - mimikatz.exe
        - procdump.exe
        - sekurlsa.exe
      OR
      process.command_line contains_any:
        - "sekurlsa::logonpasswords"
        - "-ma lsass"
        - "comsvcs.dll MiniDump"
        - "copy \\ntds.dit"
    severity: critical
    mitre: T1003
    
  - name: "LSASS Memory Access"
    description: "Detect processes accessing LSASS memory"
    condition:
      target.process.name == "lsass.exe"
      AND
      access_type in:
        - READ
        - ALL_ACCESS
      AND
      source.process.name not in:
        - csrss.exe
        - wininit.exe
        - wmiprvse.exe
    severity: critical
    mitre: T1003.001
    
  # ========================================
  # Persistence
  # ========================================
  - name: "Scheduled Task Creation"
    description: "Monitor for persistence via scheduled tasks"
    condition:
      process.name == "schtasks.exe"
      AND
      process.command_line contains "/create"
    severity: medium
    mitre: T1053.005
    
  - name: "Registry Run Key Modification"
    description: "Detect persistence via registry run keys"
    condition:
      registry.path matches_any:
        - "*\Software\Microsoft\Windows\CurrentVersion\Run*"
        - "*\Software\Microsoft\Windows\CurrentVersion\RunOnce*"
      AND
      event.type == "modification"
    severity: high
    mitre: T1547.001
 
  # ========================================
  # Defense Evasion  
  # ========================================
  - name: "Security Tool Tampering"
    description: "Detect attempts to disable security software"
    condition:
      (
        process.command_line contains_any:
          - "stop windefend"
          - "Set-MpPreference -DisableRealtimeMonitoring"
          - "uninstall antivirus"
          - "taskkill /IM"
        AND
        process.command_line contains_any:
          - "defender"
          - "antivirus"
          - "security"
      )
    severity: critical
    mitre: T1562.001

HIDS Agent Selection

Popular HIDS solutions include: OSSEC/Wazuh (open source, comprehensive), osquery (SQL interface to OS state), Elastic Endpoint Security (commercial, ML capabilities), CrowdStrike Falcon (commercial, cloud-native), and Carbon Black (commercial, endpoint detection and response). Choose based on your budget, scale, and integration requirements.

Network-Based Intrusion Detection (NIDS)

Network-based Intrusion Detection Systems (NIDS) analyze network traffic flowing through a network segment. They see all traffic on that segment, regardless of the number of hosts, making them efficient for detecting network-borne attacks.

NIDS Deployment Architectures

Converting Mermaid diagram...

Traffic Capture Methods

NIDS Traffic Capture Options
Method	Description	Pros	Cons
SPAN/Mirror Port	Switch copies traffic to monitoring port	No additional hardware, easy setup	Can drop traffic under load, affects switch
Network TAP	Passive hardware device copies traffic	No packet loss, fail-open/safe	Additional cost, physical deployment
Inline (IPS mode)	Traffic flows through sensor	Can block attacks in real-time	Latency, single point of failure
pcap/BPF	Software capture on host	Flexible, per-host visibility	CPU intensive, limited scale

Suricata NIDS Configuration

suricata.yaml
YAML
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
# Suricata NIDS configuration
# /etc/suricata/suricata.yaml
 
%YAML 1.1
---
 
# Global settings
vars:
  address-groups:
    HOME_NET: "[192.168.0.0/16,10.0.0.0/8,172.16.0.0/12]"
    EXTERNAL_NET: "!$HOME_NET"
    DNS_SERVERS: "$HOME_NET"
    HTTP_SERVERS: "$HOME_NET"
    
  port-groups:
    HTTP_PORTS: "80,8080,8000,8888"
    HTTPS_PORTS: "443,8443"
    SSH_PORTS: "22"
    DNS_PORTS: "53"
 
# Network interface configuration
af-packet:
  - interface: eth0
    threads: auto
    cluster-id: 99
    cluster-type: cluster_flow
    defrag: yes
    use-mmap: yes
    ring-size: 200000
    buffer-size: 32768
    
# Detection engine tuning
detect:
  profile: high
  sgh-mpm-context: auto
  inspection-recursion-limit: 3000
 
# Multi-pattern matching algorithm
mpm-algo: hs  # Hyperscan for maximum performance
 
# Stream reassembly (critical for evasion prevention)
stream:
  memcap: 256mb
  checksum-validation: no
  reassembly:
    memcap: 512mb
    depth: 1mb
    toserver-chunk-size: 2560
    toclient-chunk-size: 2560
    randomize-chunk-size: yes
 
# Protocol detection
app-layer:
  protocols:
    http:
      enabled: yes
      libhtp:
        request-body-limit: 100kb
        response-body-limit: 100kb
        request-body-minimal-inspect-size: 32kb
        request-body-inspect-window: 4kb
        response-body-minimal-inspect-size: 40kb
        response-body-inspect-window: 16kb
        
    tls:
      enabled: yes
      detection-ports:
        dp: 443, 8443, 993, 995
      ja3-fingerprints: yes  # TLS fingerprinting
      
    dns:
      enabled: yes
      tcp:
        enabled: yes
        detection-ports:
          dp: 53
    
    ssh:
      enabled: yes
      hassh: yes  # SSH fingerprinting
 
# Logging configuration
outputs:
  - eve-log:
      enabled: yes
      filetype: regular
      filename: eve.json
      types:
        - alert:
            tagged-packets: yes
            metadata: yes
        - http:
            extended: yes
        - dns:
            query: yes
            answer: yes
        - tls:
            extended: yes
            session-resumption: yes
        - files:
            force-magic: yes
            force-hash: [md5, sha256]
        - flow
        - netflow
 
# Rule management
default-rule-path: /var/lib/suricata/rules
rule-files:
  - suricata.rules
  - emerging-threats.rules
  - custom-rules.rules
 
# Classification and priority
classification-file: /etc/suricata/classification.config
reference-config-file: /etc/suricata/reference.config

Network Detection Use Cases

NIDS Detection Capabilities
Attack Type	Detection Method	Example Indicators
Reconnaissance	Scan patterns, unusual port access	Port sweep, version probes
Exploitation	Known exploit payloads, overflow patterns	EternalBlue SMB, Log4Shell
C2 Communication	Beacon patterns, known C2 protocols	Cobalt Strike, Metasploit
Data Exfiltration	Large outbound transfers, DNS tunneling	Long DNS queries, icmp payloads
Lateral Movement	Internal scanning, credential attacks	Pass-the-hash, PsExec
Malware Download	Known malware hashes, suspicious downloads	PE files from pastebin

The TLS Visibility Challenge

Modern networks encrypt 80-90% of traffic with TLS. NIDS cannot inspect encrypted payloads, creating significant blind spots. Solutions include TLS inspection (controversial for privacy), JA3/JA3S fingerprinting (identifies TLS implementations), and relying on metadata (connection patterns, certificate details) rather than payload.

Modern Detection: UEBA and Threat Hunting

Traditional IDS (signature + anomaly) struggles against sophisticated adversaries who blend in with normal activity. Modern approaches add behavioral analytics and proactive hunting.

User and Entity Behavior Analytics (UEBA)

UEBA creates behavioral baselines for every entity (users, hosts, applications) and detects deviations across multiple dimensions:

UEBA Behavioral Dimensions
Dimension	Normal Baseline	Anomaly Indicator
Temporal	Work hours, access patterns	3 AM database access
Geospatial	Usual locations, VPN patterns	Login from new country
Peer Group	Similar role behavior	Developer accessing financial systems
Resource	Usual applications, data	First-time access to sensitive share
Velocity	Rate of actions	100x normal file downloads
Sequence	Order of operations	Password reset before access

Risk Scoring

UEBA systems aggregate multiple weak signals into a risk score:

ueba-risk-scoring.py
Python
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
"""
UEBA Risk Scoring Engine
Aggregates multiple behavioral signals into actionable risk scores.
"""
 
from dataclasses import dataclass
from typing import List, Dict
from datetime import datetime, timedelta
 
@dataclass
class RiskSignal:
    """Individual risk indicator."""
    category: str  # temporal, geo, peer, resource, velocity
    severity: float  # 0.0 to 1.0
    description: str
    timestamp: datetime
    decay_hours: int = 24  # Signal loses relevance over time
 
@dataclass
class EntityRiskProfile:
    """Risk profile for a user or entity."""
    entity_id: str
    entity_type: str  # user, host, service_account
    signals: List[RiskSignal]
    base_risk: float  # Historical risk level
    
    def calculate_current_risk(self) -> float:
        """
        Calculate current risk score with time decay.
        Recent signals weighted more heavily.
        """
        now = datetime.now()
        total_risk = self.base_risk
        
        for signal in self.signals:
            age_hours = (now - signal.timestamp).total_seconds() / 3600
            
            if age_hours < signal.decay_hours:
                # Apply exponential decay
                decay_factor = 0.5 ** (age_hours / signal.decay_hours)
                total_risk += signal.severity * decay_factor
        
        # Normalize to 0-100 scale
        return min(100, total_risk * 100)
    
    def get_active_signals(self) -> List[RiskSignal]:
        """Get signals that haven't fully decayed."""
        now = datetime.now()
        return [
            s for s in self.signals
            if (now - s.timestamp).total_seconds() / 3600 < s.decay_hours * 2
        ]
 
 
class UEBARiskEngine:
    """Main UEBA risk calculation engine."""
    
    def __init__(self):
        self.entity_profiles: Dict[str, EntityRiskProfile] = {}
        self.alert_threshold = 75  # Risk score that triggers alert
        
    def add_signal(self, entity_id: str, signal: RiskSignal):
        """Add a risk signal to an entity's profile."""
        if entity_id not in self.entity_profiles:
            self.entity_profiles[entity_id] = EntityRiskProfile(
                entity_id=entity_id,
                entity_type="user",
                signals=[],
                base_risk=0.1
            )
        
        self.entity_profiles[entity_id].signals.append(signal)
        
        # Check for alert condition
        risk = self.entity_profiles[entity_id].calculate_current_risk()
        if risk >= self.alert_threshold:
            self._generate_alert(entity_id, risk)
    
    def _generate_alert(self, entity_id: str, risk_score: float):
        """Generate alert for high-risk entity."""
        profile = self.entity_profiles[entity_id]
        signals = profile.get_active_signals()
        
        print(f"=== HIGH RISK ALERT ===")
        print(f"Entity: {entity_id}")
        print(f"Risk Score: {risk_score:.1f}")
        print(f"Active Signals:")
        for s in sorted(signals, key=lambda x: x.severity, reverse=True):
            print(f"  [{s.category}] {s.description} (severity: {s.severity})")
 
 
# Example usage
if __name__ == "__main__":
    engine = UEBARiskEngine()
    
    # User jsmith has several subtle risk signals
    now = datetime.now()
    
    # Signal 1: Off-hours access
    engine.add_signal("jsmith", RiskSignal(
        category="temporal",
        severity=0.3,
        description="Login at 2:47 AM (outside normal hours)",
        timestamp=now - timedelta(hours=2)
    ))
    
    # Signal 2: New geographic location
    engine.add_signal("jsmith", RiskSignal(
        category="geo",
        severity=0.4,
        description="VPN connection from new country (Romania)",
        timestamp=now - timedelta(hours=1)
    ))
    
    # Signal 3: Unusual resource access
    engine.add_signal("jsmith", RiskSignal(
        category="resource",
        severity=0.3,
        description="First access to HR file share",
        timestamp=now - timedelta(minutes=30)
    ))
    
    # Signal 4: High velocity downloads
    engine.add_signal("jsmith", RiskSignal(
        category="velocity",
        severity=0.5,
        description="Downloaded 500 files in 10 minutes (10x normal)",
        timestamp=now
    ))
    
    # Cumulative signals exceed threshold - ALERT triggered

Threat Hunting

Threat hunting is proactive detection—assuming attackers are already present and actively searching for evidence, rather than waiting for alerts.

Threat Hunting Process

•Hypothesis Formation — Form a theory about how attackers might operate (e.g., 'Attackers are using PowerShell for lateral movement')
•Data Collection — Gather relevant logs, telemetry, and artifacts
•Investigation — Search for indicators supporting or refuting the hypothesis
•Discovery — Document findings, whether or not hypothesis confirmed
•Response — If malicious activity found, escalate to incident response
•Automation — Convert successful hunts into automated detections

Hunt Example: PowerShell-Based Attacks

powershell-hunt.kql
KQL
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
// Threat Hunt: Suspicious PowerShell Activity
// Platform: Microsoft Sentinel (KQL)
// Hypothesis: Attackers are using encoded PowerShell commands
 
// Hunt 1: Base64 encoded commands (common obfuscation)
DeviceProcessEvents
| where TimeGenerated > ago(7d)
| where FileName =~ "powershell.exe" or FileName =~ "pwsh.exe"
| where ProcessCommandLine has_any (
    "-encodedcommand",
    "-enc",
    "-e ",
    "FromBase64String"
)
| project TimeGenerated, DeviceName, AccountName, ProcessCommandLine
| order by TimeGenerated desc
 
// Hunt 2: PowerShell downloading content
DeviceProcessEvents
| where TimeGenerated > ago(7d)
| where FileName =~ "powershell.exe"
| where ProcessCommandLine has_any (
    "WebClient",
    "DownloadString",
    "DownloadFile",
    "Invoke-WebRequest",
    "IWR",
    "wget",
    "curl",
    "Net.WebClient",
    "Start-BitsTransfer"
)
| project TimeGenerated, DeviceName, AccountName, ProcessCommandLine
| order by TimeGenerated desc
 
// Hunt 3: PowerShell execution from unusual parents
DeviceProcessEvents
| where TimeGenerated > ago(7d)
| where FileName =~ "powershell.exe"
| where InitiatingProcessFileName in~ (
    "winword.exe",
    "excel.exe",
    "outlook.exe",
    "wmiprvse.exe",
    "mshta.exe",
    "regsvr32.exe",
    "rundll32.exe"
)
| project TimeGenerated, DeviceName, AccountName, 
          InitiatingProcessFileName, ProcessCommandLine
| order by TimeGenerated desc
 
// Hunt 4: PowerShell making network connections
DeviceNetworkEvents
| where TimeGenerated > ago(7d)
| where InitiatingProcessFileName =~ "powershell.exe"
| where RemoteIPType == "Public"
| summarize 
    ConnectionCount = count(),
    UniqueDestinations = dcount(RemoteIP),
    Destinations = make_set(RemoteIP, 10)
    by DeviceName, AccountName
| where ConnectionCount > 5 or UniqueDestinations > 3
| order by ConnectionCount desc
 
// Hunt 5: AMSI bypass attempts
DeviceProcessEvents
| where TimeGenerated > ago(7d)
| where FileName =~ "powershell.exe"
| where ProcessCommandLine has_any (
    "AmsiInitFailed",
    "amsiContext",
    "AmsiUtils",
    "[Ref].Assembly.GetType",
    "NonPublic,Static"
)
| project TimeGenerated, DeviceName, AccountName, ProcessCommandLine

The MITRE ATT&CK Framework

Structure your detection and hunting efforts around the MITRE ATT&CK framework. It catalogs known adversary techniques across the attack lifecycle. Use it to: (1) Identify detection gaps (which techniques can't you detect?), (2) Prioritize detection development (cover common techniques first), (3) Map alerts to attacker behavior for context.

Summary: Intrusion Detection

Intrusion detection is the practice of finding attackers in your environment—the earlier, the better. Let's consolidate the key concepts:

Key Takeaways

•Detection is classification — Every detection has four outcomes (TP, TN, FP, FN); optimize for low FP while maintaining coverage.
•Signature-based detection — Known attack patterns, low false positives, but cannot detect novel attacks or evasion.
•Anomaly-based detection — Learns normal behavior, can detect unknown attacks, but higher false positives.
•HIDS sees the host — File integrity, process execution, local logs; sees decrypted traffic and host-level attacks.
•NIDS sees the network — Traffic patterns, payloads, protocol abuse; efficient coverage but TLS blindness.
•UEBA adds behavior — Entity baselines, risk scoring, correlation of weak signals into strong indicators.
•Threat hunting is proactive — Don't wait for alerts; actively search for adversaries using hypotheses and investigation.

What's Next:

With detection capabilities in place, we now turn to log analysis—the techniques for efficiently searching, correlating, and investigating the massive volumes of data that detection systems generate.

Page Complete

You now understand the full landscape of intrusion detection—from signatures to behavior analytics, from host sensors to network monitors, from passive alerting to active hunting. These capabilities form the core of any security operations program.

Intrusion Detection

The First Line of Active Defense

What You Will Learn

Intrusion Detection Fundamentals

Detection Taxonomy

Intrusion Detection System Classification
Classification	Types	Description
By Location	HIDS, NIDS, Cloud IDS	Where sensors are deployed (host, network, cloud)
By Method	Signature, Anomaly, Hybrid	How detection is performed
By Response	Passive (IDS), Active (IPS)	Whether system takes action
By Analysis Time	Real-time, Batch/Historical	When analysis occurs

The Detection Problem

Intrusion detection is fundamentally a classification problem with four possible outcomes:

Detection Outcome Matrix
	Actual Attack	No Attack
Alert Raised	True Positive (TP) ✓	False Positive (FP) - Alert fatigue
No Alert	False Negative (FN) - Missed attack!	True Negative (TN) ✓

Critical Metrics

Detection Performance Metrics

•True Positive Rate (Recall) = TP / (TP + FN) — Percentage of real attacks detected. Higher is better.
•False Positive Rate = FP / (FP + TN) — Percentage of benign activity flagged as attacks. Lower is better.
•Precision = TP / (TP + FP) — When we alert, how often is it a real attack. Higher is better.
•Mean Time to Detect (MTTD) — Average time from attack start to detection. Lower is better.
•Detection Coverage — Percentage of ATT&CK techniques with at least one detection rule.

The False Positive Crisis

The Base Rate Fallacy

Even a highly accurate detection system can be impractical due to the base rate of attacks. Consider:

base-rate-example.py
Python
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
"""
The base rate fallacy in intrusion detection.
Even 99% accurate detections can be mostly false positives.
"""
 
# Scenario: Enterprise with 1,000,000 events per day
total_events = 1_000_000
 
# Attack base rate: 0.01% of events are malicious
attack_rate = 0.0001
actual_attacks = int(total_events * attack_rate)  # 100 attacks
normal_events = total_events - actual_attacks     # 999,900 normal
 
# Detection system with:
# - 99% True Positive Rate (catches 99% of attacks)
# - 1% False Positive Rate (flags 1% of normal events)
true_positive_rate = 0.99
false_positive_rate = 0.01
 
# Calculate alerts
true_positives = int(actual_attacks * true_positive_rate)     # 99 real attacks caught
false_positives = int(normal_events * false_positive_rate)    # 9,999 false alarms!
 
total_alerts = true_positives + false_positives  # 10,098 alerts
 
# The problem:
precision = true_positives / total_alerts  # 0.98% !!
 
print(f"Total alerts: {total_alerts:,}")
print(f"True positives: {true_positives}")
print(f"False positives: {false_positives:,}")
print(f"Precision: {precision:.2%}")
print(f"\nResult: Only ~1% of alerts are real attacks!")
print(f"Analysts must review {false_positives:,} false alarms to find {true_positives} real attacks")
 
# To achieve 50% precision (half of alerts are real):
# FP rate must be: attacks / normal_events = 100 / 999,900 = 0.01%
# This means 99.99% accuracy on normal events is required!

Context is Everything

Signature-Based Detection

How Signatures Work

A signature defines the characteristics of a known attack:

•Network signatures — Specific byte patterns in packets, protocol anomalies, malicious payloads
•Host signatures — Suspicious process behaviors, file hashes, registry modifications
•Log signatures — Specific event patterns, field value combinations, sequences

Network Signature Example: Suricata Rules

Suricata (and Snort) use a powerful rule language for network detection:

suricata.rules

Suricata Rules

# Suricata IDS rules for common attacks
 
# ================================================
# SQL Injection Detection
# ================================================
# Detect SQL injection attempts in HTTP requests
alert http any any -> any any (
    msg:"ATTACK SQL Injection Attempt - UNION SELECT";
    flow:established,to_server;
    http.uri;
    content:"UNION"; nocase;
    content:"SELECT"; nocase; distance:0;
    pcre:"/UNION\s+(ALL\s+)?SELECT/i";
    classtype:web-application-attack;
    sid:1000001; rev:3;
)
 
# SQL injection in POST body
alert http any any -> any any (
    msg:"ATTACK SQL Injection in POST Body";
    flow:established,to_server;
    http.method; content:"POST";
    http.request_body;
    pcre:"/('|%27)\s*(OR|AND)\s*('|%27)?\d*\s*[=<>]/i";
    classtype:web-application-attack;
    sid:1000002; rev:2;
)
 
# ================================================
# Command Injection Detection
# ================================================
# Detect command injection via semicolon or pipe
alert http any any -> any any (
    msg:"ATTACK Command Injection Attempt";
    flow:established,to_server;
    http.uri;
    pcre:"/[;|&]\s*(cat|ls|id|whoami|uname|pwd|nc|wget|curl)/i";
    classtype:web-application-attack;
    sid:1000003; rev:1;
)
 
# ================================================
# Reconnaissance Detection
# ================================================
# Aggressive scanning behavior (many SYN to different ports)
alert tcp any any -> $HOME_NET any (
    msg:"RECON Possible Port Scan - Multiple Ports";
    flags:S;
    threshold: type both, track by_src, count 50, seconds 10;
    classtype:attempted-recon;
    sid:1000010; rev:1;
)
 
# ================================================
# Malware C2 Detection
# ================================================
# Known C2 beacon pattern (Cobalt Strike default)
alert http any any -> any any (
    msg:"MALWARE Cobalt Strike Beacon C2 Traffic";
    flow:established,to_server;
    content:"/submit.php?id="; http.uri;
    pcre:"/\/submit\.php\?id=[0-9]{5,10}$/";
    http.user_agent; content:"Mozilla/"; startswith;
    classtype:trojan-activity;
    priority:1;
    sid:1000020; rev:2;
)
 
# ================================================
# Lateral Movement Detection
# ================================================
# PsExec-style remote execution
alert tcp any any -> any 445 (
    msg:"LATERAL PsExec-style Remote Service Install";
    flow:established,to_server;
    content:"|FF|SMB"; depth:4;
    content:"|25 00|"; distance:0; within:2;  # Tree Connect
    content:"IPC$"; distance:0; within:100;
    content:"PSEXESVC"; distance:0; within:200;
    classtype:trojan-activity;
    sid:1000030; rev:1;
)
 
# ================================================
# DNS Tunneling Detection
# ================================================
# Unusually long DNS queries (data exfiltration)
alert dns any any -> any any (
    msg:"EXFIL Potential DNS Tunneling - Long Query";
    dns.query;
    pcre:"/^[a-z0-9]{30,}/i";  # 30+ char subdomain
    threshold: type threshold, track by_src, count 10, seconds 60;
    classtype:bad-unknown;
    sid:1000040; rev:1;
)

Log-Based Signature Detection: Sigma Rules

Sigma is the open signature format for log-based detection, allowing platform-agnostic rule sharing:

mimikatz-detection.yaml
YAML (Sigma)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
# Sigma rule for detecting Mimikatz usage
# Can be converted to Splunk, Elastic, QRadar, etc.
 
title: Mimikatz Credential Dumping
id: d9dcbe87-0e5e-4a21-bb5d-3f1ea4e36c73
status: stable
description: |
    Detects usage of Mimikatz for credential theft through
    multiple detection methods including process creation,
    command line arguments, and loaded modules.
author: Security Team
date: 2024/01/15
modified: 2024/01/15
references:
    - https://attack.mitre.org/techniques/T1003/001/
    - https://github.com/gentilkiwi/mimikatz
tags:
    - attack.credential_access
    - attack.t1003.001
logsource:
    category: process_creation
    product: windows
detection:
    # Method 1: Known process names
    selection_name:
        Image|endswith:
            - '\mimikatz.exe'
            - '\mimi.exe'
            - '\m.exe'
    
    # Method 2: Command line indicators
    selection_cmdline:
        CommandLine|contains:
            - 'sekurlsa::logonpasswords'
            - 'sekurlsa::wdigest'
            - 'sekurlsa::kerberos'
            - 'lsadump::sam'
            - 'lsadump::dcsync'
            - 'token::elevate'
            - 'privilege::debug'
    
    # Method 3: Original filename in PE header
    selection_original:
        OriginalFileName: 'mimikatz.exe'
    
    # Method 4: Known hashes (updates frequently)
    selection_hash:
        Hashes|contains:
            - 'SHA256=...'  # Add known hashes
    
    condition: selection_name or selection_cmdline or selection_original or selection_hash
 
fields:
    - User
    - CommandLine
    - ParentImage
    - ParentCommandLine
    - Computer
    
falsepositives:
    - Legitimate credential management tools
    - Security testing with authorized tools
    
level: critical

Signature-Based Advantages

•Low false positive rate for known attacks
•Fast processing — pattern matching is efficient
•Easy to understand — human-readable rules
•Actionable alerts — clear attack identification
•Community sharing — open rule repositories

Signature-Based Limitations

•Cannot detect novel attacks — zero-days missed
•Evasion through modification — slight changes bypass
•Signature maintenance burden — constant updates
•Encrypted traffic blind spots — can't inspect TLS
•Polymorphic malware — self-modifying code evades

Anomaly-Based Detection

The Anomaly Detection Process

Converting Mermaid diagram...

Types of Anomaly Detection

1. Statistical Anomaly Detection

Simplest approach: flag values that deviate significantly from historical norms.

statistical-anomaly.py
Python
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
"""
Statistical anomaly detection for security metrics.
Uses z-score and IQR methods for detecting outliers.
"""
 
import numpy as np
from collections import deque
from typing import Tuple, Optional
 
class StatisticalAnomalyDetector:
    """
    Detects anomalies using statistical methods.
    Maintains rolling baselines for adaptive detection.
    """
    
    def __init__(self, window_size: int = 1000, z_threshold: float = 3.0):
        """
        Initialize detector with rolling window size and threshold.
        
        Args:
            window_size: Number of historical values to consider
            z_threshold: Number of standard deviations for anomaly
        """
        self.window_size = window_size
        self.z_threshold = z_threshold
        self.values = deque(maxlen=window_size)
        
    def add_value(self, value: float) -> Tuple[bool, Optional[float]]:
        """
        Add a new value and check if it's anomalous.
        
        Returns:
            (is_anomaly, z_score)
        """
        if len(self.values) < 30:  # Need minimum data for baseline
            self.values.append(value)
            return (False, None)
        
        # Calculate baseline statistics
        mean = np.mean(self.values)
        std = np.std(self.values)
        
        if std == 0:  # Avoid division by zero
            std = 0.0001
        
        # Calculate z-score
        z_score = (value - mean) / std
        
        # Check for anomaly
        is_anomaly = abs(z_score) > self.z_threshold
        
        # Add to rolling window (even if anomaly)
        self.values.append(value)
        
        return (is_anomaly, z_score)
 
 
class MultiMetricAnomalyDetector:
    """
    Detects anomalies across multiple security metrics.
    """
    
    def __init__(self):
        self.metrics = {
            'failed_logins': StatisticalAnomalyDetector(window_size=1440, z_threshold=3.0),
            'bytes_transferred': StatisticalAnomalyDetector(window_size=1440, z_threshold=4.0),
            'unique_destinations': StatisticalAnomalyDetector(window_size=1440, z_threshold=3.5),
            'after_hours_activity': StatisticalAnomalyDetector(window_size=720, z_threshold=2.5),
        }
        
    def check(self, metric_name: str, value: float) -> dict:
        """Check a metric value for anomalies."""
        if metric_name not in self.metrics:
            raise ValueError(f"Unknown metric: {metric_name}")
        
        is_anomaly, z_score = self.metrics[metric_name].add_value(value)
        
        return {
            'metric': metric_name,
            'value': value,
            'is_anomaly': is_anomaly,
            'z_score': z_score,
            'severity': self._calculate_severity(z_score) if z_score else 'unknown'
        }
    
    def _calculate_severity(self, z_score: float) -> str:
        """Map z-score to severity level."""
        abs_z = abs(z_score)
        if abs_z > 5:
            return 'critical'
        elif abs_z > 4:
            return 'high'
        elif abs_z > 3:
            return 'medium'
        else:
            return 'low'
 
 
# Example usage
if __name__ == "__main__":
    detector = MultiMetricAnomalyDetector()
    
    # Simulate normal traffic
    import random
    for _ in range(100):
        detector.check('failed_logins', random.gauss(10, 2))
    
    # Simulate brute force attack
    result = detector.check('failed_logins', 150)  # 150 failures suddenly
    print(f"Result: {result}")
    # Output: {'metric': 'failed_logins', 'value': 150, 'is_anomaly': True, 
    #          'z_score': 35.2, 'severity': 'critical'}

2. Behavioral Baselines

More sophisticated approach: learn typical patterns for each entity (user, host, application).

Behavioral Baseline Examples
Entity	Baseline Features	Anomaly Example
User	Login times, locations, accessed resources	Admin logging in from foreign country at 3 AM
Host	Normal processes, network connections, resource usage	Web server making outbound connections to unknown IPs
Application	API call patterns, data volumes, error rates	Database reading 10x normal data volume
Network	Traffic volumes, protocol mix, destination frequency	Internal host suddenly contacting 1000 external IPs

3. Machine Learning Approaches

Modern IDS increasingly use ML for anomaly detection:

•Clustering — Group similar behaviors; outliers are anomalies (K-means, DBSCAN)
•Classification — Train on labeled attack/normal data (Random Forest, XGBoost)
•Autoencoders — Learn to compress/decompress normal data; high reconstruction error = anomaly
•LSTM/Sequence Models — Learn temporal patterns; detect unusual event sequences
•Graph Neural Networks — Model entity relationships; detect unusual graph patterns

The Baseline Poisoning Problem

Host-Based Intrusion Detection (HIDS)

HIDS Monitoring Capabilities

HIDS Monitoring Domains
Domain	What's Monitored	Attack Examples Detected
File Integrity	File hashes, permissions, ownership	Rootkits, webshells, config tampering
Process Activity	Process creation, termination, commands	Malware execution, suspicious tools
System Calls	Kernel API calls, syscall sequences	Exploitation, privilege escalation
Log Analysis	Local log files, journal entries	Authentication attacks, privilege abuse
Registry (Windows)	Registry modifications	Persistence, configuration changes
Network Connections	Outbound connections per process	C2 communication, data exfiltration

File Integrity Monitoring (FIM)

FIM is a foundational HIDS capability that detects unauthorized changes to critical files:

ossec-fim.conf
XML (OSSEC)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
<!-- OSSEC HIDS File Integrity Monitoring Configuration -->
<!-- /var/ossec/etc/ossec.conf -->
 
<ossec_config>
  <syscheck>
    <!-- How often to run FIM scans (in seconds) -->
    <frequency>7200</frequency>
    
    <!-- Enable real-time monitoring (requires inotify) -->
    <scan_on_start>yes</scan_on_start>
    
    <!-- ========================================== -->
    <!-- Critical system directories to monitor -->
    <!-- ========================================== -->
    
    <!-- Binary directories - any change is suspicious -->
    <directories check_all="yes" realtime="yes">/bin</directories>
    <directories check_all="yes" realtime="yes">/sbin</directories>
    <directories check_all="yes" realtime="yes">/usr/bin</directories>
    <directories check_all="yes" realtime="yes">/usr/sbin</directories>
    
    <!-- System configuration -->
    <directories check_all="yes" realtime="yes">/etc</directories>
    
    <!-- Startup scripts -->
    <directories check_all="yes" realtime="yes">/etc/init.d</directories>
    <directories check_all="yes" realtime="yes">/etc/systemd</directories>
    
    <!-- Kernel and boot -->
    <directories check_all="yes" realtime="yes">/boot</directories>
    
    <!-- Library directories -->
    <directories check_all="yes">/lib</directories>
    <directories check_all="yes">/lib64</directories>
    <directories check_all="yes">/usr/lib</directories>
    
    <!-- Web server directories (webshell detection) -->
    <directories check_all="yes" realtime="yes">/var/www</directories>
    <directories check_all="yes" realtime="yes">/var/www/html</directories>
    
    <!-- SSH configuration -->
    <directories check_all="yes" realtime="yes">/root/.ssh</directories>
    <directories check_all="yes" realtime="yes"
        report_changes="yes">/etc/ssh</directories>
    
    <!-- Cron directories (persistence detection) -->
    <directories check_all="yes" realtime="yes">/etc/cron.d</directories>
    <directories check_all="yes" realtime="yes">/etc/cron.daily</directories>
    <directories check_all="yes" realtime="yes">/var/spool/cron</directories>
    
    <!-- ========================================== -->
    <!-- Directories to ignore -->
    <!-- ========================================== -->
    
    <ignore>/etc/mtab</ignore>
    <ignore>/etc/hosts.deny</ignore>
    <ignore>/etc/adjtime</ignore>
    <ignore>/etc/resolv.conf</ignore>
    
    <!-- Log files change frequently, don't monitor -->
    <ignore type="sregex">/var/log/*</ignore>
    
    <!-- Ignore backup files -->
    <ignore type="sregex">.*.swp$</ignore>
    <ignore type="sregex">.*~$</ignore>
    
    <!-- ========================================== -->
    <!-- What attributes to check -->
    <!-- ========================================== -->
    <!-- 
      check_all: permissions, owner, group, size, hash
      realtime: Use inotify for immediate detection
      report_changes: Log file content differences
    -->
    
  </syscheck>
</ossec_config>

Process and Command Monitoring

Monitoring process execution captures attacker actions in real-time:

process-monitor.yaml
YAML
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
# Detection rules for suspicious process activity
# Format compatible with Elastic Endpoint Security, Carbon Black, etc.
 
rules:
  # ========================================
  # Reconnaissance Tools
  # ========================================
  - name: "Suspicious Network Discovery"
    description: "Detect common network reconnaissance tools"
    condition:
      process.name in:
        - nmap
        - masscan
        - netdiscover
        - arp-scan
      OR
      process.command_line contains_any:
        - "net view"
        - "net group"
        - "net user /domain"
        - "nltest /dclist"
    severity: medium
    mitre: T1018
    
  # ========================================
  # Credential Access
  # ========================================
  - name: "Credential Dumping Tool Execution"
    description: "Detect execution of common credential theft tools"
    condition:
      process.name in:
        - mimikatz.exe
        - procdump.exe
        - sekurlsa.exe
      OR
      process.command_line contains_any:
        - "sekurlsa::logonpasswords"
        - "-ma lsass"
        - "comsvcs.dll MiniDump"
        - "copy \\ntds.dit"
    severity: critical
    mitre: T1003
    
  - name: "LSASS Memory Access"
    description: "Detect processes accessing LSASS memory"
    condition:
      target.process.name == "lsass.exe"
      AND
      access_type in:
        - READ
        - ALL_ACCESS
      AND
      source.process.name not in:
        - csrss.exe
        - wininit.exe
        - wmiprvse.exe
    severity: critical
    mitre: T1003.001
    
  # ========================================
  # Persistence
  # ========================================
  - name: "Scheduled Task Creation"
    description: "Monitor for persistence via scheduled tasks"
    condition:
      process.name == "schtasks.exe"
      AND
      process.command_line contains "/create"
    severity: medium
    mitre: T1053.005
    
  - name: "Registry Run Key Modification"
    description: "Detect persistence via registry run keys"
    condition:
      registry.path matches_any:
        - "*\Software\Microsoft\Windows\CurrentVersion\Run*"
        - "*\Software\Microsoft\Windows\CurrentVersion\RunOnce*"
      AND
      event.type == "modification"
    severity: high
    mitre: T1547.001
 
  # ========================================
  # Defense Evasion  
  # ========================================
  - name: "Security Tool Tampering"
    description: "Detect attempts to disable security software"
    condition:
      (
        process.command_line contains_any:
          - "stop windefend"
          - "Set-MpPreference -DisableRealtimeMonitoring"
          - "uninstall antivirus"
          - "taskkill /IM"
        AND
        process.command_line contains_any:
          - "defender"
          - "antivirus"
          - "security"
      )
    severity: critical
    mitre: T1562.001

HIDS Agent Selection

Network-Based Intrusion Detection (NIDS)

NIDS Deployment Architectures

Converting Mermaid diagram...

Traffic Capture Methods

NIDS Traffic Capture Options
Method	Description	Pros	Cons
SPAN/Mirror Port	Switch copies traffic to monitoring port	No additional hardware, easy setup	Can drop traffic under load, affects switch
Network TAP	Passive hardware device copies traffic	No packet loss, fail-open/safe	Additional cost, physical deployment
Inline (IPS mode)	Traffic flows through sensor	Can block attacks in real-time	Latency, single point of failure
pcap/BPF	Software capture on host	Flexible, per-host visibility	CPU intensive, limited scale

Suricata NIDS Configuration

suricata.yaml
YAML
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
# Suricata NIDS configuration
# /etc/suricata/suricata.yaml
 
%YAML 1.1
---
 
# Global settings
vars:
  address-groups:
    HOME_NET: "[192.168.0.0/16,10.0.0.0/8,172.16.0.0/12]"
    EXTERNAL_NET: "!$HOME_NET"
    DNS_SERVERS: "$HOME_NET"
    HTTP_SERVERS: "$HOME_NET"
    
  port-groups:
    HTTP_PORTS: "80,8080,8000,8888"
    HTTPS_PORTS: "443,8443"
    SSH_PORTS: "22"
    DNS_PORTS: "53"
 
# Network interface configuration
af-packet:
  - interface: eth0
    threads: auto
    cluster-id: 99
    cluster-type: cluster_flow
    defrag: yes
    use-mmap: yes
    ring-size: 200000
    buffer-size: 32768
    
# Detection engine tuning
detect:
  profile: high
  sgh-mpm-context: auto
  inspection-recursion-limit: 3000
 
# Multi-pattern matching algorithm
mpm-algo: hs  # Hyperscan for maximum performance
 
# Stream reassembly (critical for evasion prevention)
stream:
  memcap: 256mb
  checksum-validation: no
  reassembly:
    memcap: 512mb
    depth: 1mb
    toserver-chunk-size: 2560
    toclient-chunk-size: 2560
    randomize-chunk-size: yes
 
# Protocol detection
app-layer:
  protocols:
    http:
      enabled: yes
      libhtp:
        request-body-limit: 100kb
        response-body-limit: 100kb
        request-body-minimal-inspect-size: 32kb
        request-body-inspect-window: 4kb
        response-body-minimal-inspect-size: 40kb
        response-body-inspect-window: 16kb
        
    tls:
      enabled: yes
      detection-ports:
        dp: 443, 8443, 993, 995
      ja3-fingerprints: yes  # TLS fingerprinting
      
    dns:
      enabled: yes
      tcp:
        enabled: yes
        detection-ports:
          dp: 53
    
    ssh:
      enabled: yes
      hassh: yes  # SSH fingerprinting
 
# Logging configuration
outputs:
  - eve-log:
      enabled: yes
      filetype: regular
      filename: eve.json
      types:
        - alert:
            tagged-packets: yes
            metadata: yes
        - http:
            extended: yes
        - dns:
            query: yes
            answer: yes
        - tls:
            extended: yes
            session-resumption: yes
        - files:
            force-magic: yes
            force-hash: [md5, sha256]
        - flow
        - netflow
 
# Rule management
default-rule-path: /var/lib/suricata/rules
rule-files:
  - suricata.rules
  - emerging-threats.rules
  - custom-rules.rules
 
# Classification and priority
classification-file: /etc/suricata/classification.config
reference-config-file: /etc/suricata/reference.config

Network Detection Use Cases

NIDS Detection Capabilities
Attack Type	Detection Method	Example Indicators
Reconnaissance	Scan patterns, unusual port access	Port sweep, version probes
Exploitation	Known exploit payloads, overflow patterns	EternalBlue SMB, Log4Shell
C2 Communication	Beacon patterns, known C2 protocols	Cobalt Strike, Metasploit
Data Exfiltration	Large outbound transfers, DNS tunneling	Long DNS queries, icmp payloads
Lateral Movement	Internal scanning, credential attacks	Pass-the-hash, PsExec
Malware Download	Known malware hashes, suspicious downloads	PE files from pastebin

The TLS Visibility Challenge

Modern Detection: UEBA and Threat Hunting

Traditional IDS (signature + anomaly) struggles against sophisticated adversaries who blend in with normal activity. Modern approaches add behavioral analytics and proactive hunting.

User and Entity Behavior Analytics (UEBA)

UEBA creates behavioral baselines for every entity (users, hosts, applications) and detects deviations across multiple dimensions:

UEBA Behavioral Dimensions
Dimension	Normal Baseline	Anomaly Indicator
Temporal	Work hours, access patterns	3 AM database access
Geospatial	Usual locations, VPN patterns	Login from new country
Peer Group	Similar role behavior	Developer accessing financial systems
Resource	Usual applications, data	First-time access to sensitive share
Velocity	Rate of actions	100x normal file downloads
Sequence	Order of operations	Password reset before access

Risk Scoring

UEBA systems aggregate multiple weak signals into a risk score:

ueba-risk-scoring.py
Python
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
"""
UEBA Risk Scoring Engine
Aggregates multiple behavioral signals into actionable risk scores.
"""
 
from dataclasses import dataclass
from typing import List, Dict
from datetime import datetime, timedelta
 
@dataclass
class RiskSignal:
    """Individual risk indicator."""
    category: str  # temporal, geo, peer, resource, velocity
    severity: float  # 0.0 to 1.0
    description: str
    timestamp: datetime
    decay_hours: int = 24  # Signal loses relevance over time
 
@dataclass
class EntityRiskProfile:
    """Risk profile for a user or entity."""
    entity_id: str
    entity_type: str  # user, host, service_account
    signals: List[RiskSignal]
    base_risk: float  # Historical risk level
    
    def calculate_current_risk(self) -> float:
        """
        Calculate current risk score with time decay.
        Recent signals weighted more heavily.
        """
        now = datetime.now()
        total_risk = self.base_risk
        
        for signal in self.signals:
            age_hours = (now - signal.timestamp).total_seconds() / 3600
            
            if age_hours < signal.decay_hours:
                # Apply exponential decay
                decay_factor = 0.5 ** (age_hours / signal.decay_hours)
                total_risk += signal.severity * decay_factor
        
        # Normalize to 0-100 scale
        return min(100, total_risk * 100)
    
    def get_active_signals(self) -> List[RiskSignal]:
        """Get signals that haven't fully decayed."""
        now = datetime.now()
        return [
            s for s in self.signals
            if (now - s.timestamp).total_seconds() / 3600 < s.decay_hours * 2
        ]
 
 
class UEBARiskEngine:
    """Main UEBA risk calculation engine."""
    
    def __init__(self):
        self.entity_profiles: Dict[str, EntityRiskProfile] = {}
        self.alert_threshold = 75  # Risk score that triggers alert
        
    def add_signal(self, entity_id: str, signal: RiskSignal):
        """Add a risk signal to an entity's profile."""
        if entity_id not in self.entity_profiles:
            self.entity_profiles[entity_id] = EntityRiskProfile(
                entity_id=entity_id,
                entity_type="user",
                signals=[],
                base_risk=0.1
            )
        
        self.entity_profiles[entity_id].signals.append(signal)
        
        # Check for alert condition
        risk = self.entity_profiles[entity_id].calculate_current_risk()
        if risk >= self.alert_threshold:
            self._generate_alert(entity_id, risk)
    
    def _generate_alert(self, entity_id: str, risk_score: float):
        """Generate alert for high-risk entity."""
        profile = self.entity_profiles[entity_id]
        signals = profile.get_active_signals()
        
        print(f"=== HIGH RISK ALERT ===")
        print(f"Entity: {entity_id}")
        print(f"Risk Score: {risk_score:.1f}")
        print(f"Active Signals:")
        for s in sorted(signals, key=lambda x: x.severity, reverse=True):
            print(f"  [{s.category}] {s.description} (severity: {s.severity})")
 
 
# Example usage
if __name__ == "__main__":
    engine = UEBARiskEngine()
    
    # User jsmith has several subtle risk signals
    now = datetime.now()
    
    # Signal 1: Off-hours access
    engine.add_signal("jsmith", RiskSignal(
        category="temporal",
        severity=0.3,
        description="Login at 2:47 AM (outside normal hours)",
        timestamp=now - timedelta(hours=2)
    ))
    
    # Signal 2: New geographic location
    engine.add_signal("jsmith", RiskSignal(
        category="geo",
        severity=0.4,
        description="VPN connection from new country (Romania)",
        timestamp=now - timedelta(hours=1)
    ))
    
    # Signal 3: Unusual resource access
    engine.add_signal("jsmith", RiskSignal(
        category="resource",
        severity=0.3,
        description="First access to HR file share",
        timestamp=now - timedelta(minutes=30)
    ))
    
    # Signal 4: High velocity downloads
    engine.add_signal("jsmith", RiskSignal(
        category="velocity",
        severity=0.5,
        description="Downloaded 500 files in 10 minutes (10x normal)",
        timestamp=now
    ))
    
    # Cumulative signals exceed threshold - ALERT triggered

Threat Hunting

Threat hunting is proactive detection—assuming attackers are already present and actively searching for evidence, rather than waiting for alerts.

Threat Hunting Process

•Hypothesis Formation — Form a theory about how attackers might operate (e.g., 'Attackers are using PowerShell for lateral movement')
•Data Collection — Gather relevant logs, telemetry, and artifacts
•Investigation — Search for indicators supporting or refuting the hypothesis
•Discovery — Document findings, whether or not hypothesis confirmed
•Response — If malicious activity found, escalate to incident response
•Automation — Convert successful hunts into automated detections

Hunt Example: PowerShell-Based Attacks

powershell-hunt.kql
KQL
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
// Threat Hunt: Suspicious PowerShell Activity
// Platform: Microsoft Sentinel (KQL)
// Hypothesis: Attackers are using encoded PowerShell commands
 
// Hunt 1: Base64 encoded commands (common obfuscation)
DeviceProcessEvents
| where TimeGenerated > ago(7d)
| where FileName =~ "powershell.exe" or FileName =~ "pwsh.exe"
| where ProcessCommandLine has_any (
    "-encodedcommand",
    "-enc",
    "-e ",
    "FromBase64String"
)
| project TimeGenerated, DeviceName, AccountName, ProcessCommandLine
| order by TimeGenerated desc
 
// Hunt 2: PowerShell downloading content
DeviceProcessEvents
| where TimeGenerated > ago(7d)
| where FileName =~ "powershell.exe"
| where ProcessCommandLine has_any (
    "WebClient",
    "DownloadString",
    "DownloadFile",
    "Invoke-WebRequest",
    "IWR",
    "wget",
    "curl",
    "Net.WebClient",
    "Start-BitsTransfer"
)
| project TimeGenerated, DeviceName, AccountName, ProcessCommandLine
| order by TimeGenerated desc
 
// Hunt 3: PowerShell execution from unusual parents
DeviceProcessEvents
| where TimeGenerated > ago(7d)
| where FileName =~ "powershell.exe"
| where InitiatingProcessFileName in~ (
    "winword.exe",
    "excel.exe",
    "outlook.exe",
    "wmiprvse.exe",
    "mshta.exe",
    "regsvr32.exe",
    "rundll32.exe"
)
| project TimeGenerated, DeviceName, AccountName, 
          InitiatingProcessFileName, ProcessCommandLine
| order by TimeGenerated desc
 
// Hunt 4: PowerShell making network connections
DeviceNetworkEvents
| where TimeGenerated > ago(7d)
| where InitiatingProcessFileName =~ "powershell.exe"
| where RemoteIPType == "Public"
| summarize 
    ConnectionCount = count(),
    UniqueDestinations = dcount(RemoteIP),
    Destinations = make_set(RemoteIP, 10)
    by DeviceName, AccountName
| where ConnectionCount > 5 or UniqueDestinations > 3
| order by ConnectionCount desc
 
// Hunt 5: AMSI bypass attempts
DeviceProcessEvents
| where TimeGenerated > ago(7d)
| where FileName =~ "powershell.exe"
| where ProcessCommandLine has_any (
    "AmsiInitFailed",
    "amsiContext",
    "AmsiUtils",
    "[Ref].Assembly.GetType",
    "NonPublic,Static"
)
| project TimeGenerated, DeviceName, AccountName, ProcessCommandLine

The MITRE ATT&CK Framework

Summary: Intrusion Detection

Intrusion detection is the practice of finding attackers in your environment—the earlier, the better. Let's consolidate the key concepts:

Key Takeaways

•Detection is classification — Every detection has four outcomes (TP, TN, FP, FN); optimize for low FP while maintaining coverage.
•Signature-based detection — Known attack patterns, low false positives, but cannot detect novel attacks or evasion.
•Anomaly-based detection — Learns normal behavior, can detect unknown attacks, but higher false positives.
•HIDS sees the host — File integrity, process execution, local logs; sees decrypted traffic and host-level attacks.
•NIDS sees the network — Traffic patterns, payloads, protocol abuse; efficient coverage but TLS blindness.
•UEBA adds behavior — Entity baselines, risk scoring, correlation of weak signals into strong indicators.
•Threat hunting is proactive — Don't wait for alerts; actively search for adversaries using hypotheses and investigation.

What's Next:

Page Complete