Operating SystemsCPU Execution Modes

CPU Execution Modes

LevelBeginner

Duration60 mins

TopicCPU Execution Modes

4 / 5

Privileged Instructions

The Forbidden Operations

Not all CPU instructions are created equal. Among the hundreds of operations a processor can perform, a select subset is inherently dangerous—operations that, if available to any program, would undermine the entire security model of the operating system.

These are Privileged Instructions: CPU operations that only execute when the processor is in Kernel Mode. Attempt them in User Mode, and the CPU immediately generates an exception, halting the offending code and transferring control to the operating system.

Examples of privileged operations:

Disabling interrupts (could freeze the system indefinitely)
Loading new page tables (could access any memory)
Communicating directly with hardware (could bypass all access control)
Halting the CPU (could denial-of-service the machine)

Every privileged instruction represents a "what if untrusted code could do this?" thought experiment with catastrophic answers.

What You Will Learn

By the end of this page, you will understand: (1) Why certain instructions must be restricted to Kernel Mode, (2) The complete categories of privileged instructions and their purposes, (3) What happens when User Mode code attempts a privileged operation, (4) How sensitive (not privileged) instructions differ from privileged ones, and (5) Real examples of privileged instructions on x86 and ARM architectures.

Why Privileged Instructions Exist

The concept of privileged instructions emerges directly from the need to maintain system security and stability. Without restrictions, any program could:

1. Compromise Memory Protection

Instructions that modify page tables or segment descriptors control which memory each process can access. If unrestricted:

A malicious program could map another process's memory into its own space
Passwords, encryption keys, and private data would be exposed
The entire premise of process isolation would collapse

2. Monopolize Resources

Instructions that disable interrupts or halt the CPU would let a program:

Prevent the scheduler from ever running (infinite CPU monopoly)
Block keyboard/mouse input (denial-of-service attacks)
Prevent the system from responding to any event

3. Corrupt System State

Instructions that modify control registers, descriptor tables, or system configuration could:

Disable memory protection entirely
Change the interrupt handling behavior
Put the CPU into an undefined or crashed state

Threat Analysis: Instructions Without Privilege Restriction
If This Instruction Were Unrestricted	An Attacker Could
CLI (Clear Interrupt Flag)	Disable all interrupts, freeze the system, prevent preemption forever
MOV to CR3 (Page Table Base)	Swap in their own page tables, access ANY memory address
IN/OUT (Port I/O)	Read any disk sector directly, bypass filesystem permissions
WRMSR (Write MSR)	Modify CPU features, disable security protections like SMEP/SMAP
HLT (Halt CPU)	Stop the processor, crash the system with one instruction
LGDT (Load GDT)	Install new segment descriptors, redefine memory layout

The Minimal Privilege Principle

The set of privileged instructions is designed to be the minimal set that MUST be restricted. Regular computation (arithmetic, logic, control flow) is left unrestricted because it poses no threat to system integrity. Only operations that affect system-wide state or could violate process isolation are privileged.

Categories of Privileged Instructions

Privileged instructions fall into several logical categories, each protecting a different aspect of system security:

Category 1: Interrupt and Exception Control

These instructions control whether and how the CPU responds to interrupts:

CLI (Clear Interrupt Flag): Disables maskable interrupts
STI (Set Interrupt Flag): Enables maskable interrupts
LIDT (Load IDT Register): Changes the Interrupt Descriptor Table pointer
IRET (Interrupt Return): Returns from interrupt (indirectly affects interrupt state)

Why Privileged? If user code could disable interrupts, the timer interrupt would never fire, the scheduler would never run, and the user process would monopolize the CPU forever.

Category 2: Memory Management Instructions

•MOV to CR3 — Load Page Directory Base Register. Changes which page tables are active, controlling all memory access permissions.
•INVLPG — Invalidate TLB entry. Forces re-translation of a virtual address; could be used to cause consistency issues.
•LGDT — Load Global Descriptor Table Register. Defines segment base/limit for all segments.
•LLDT — Load Local Descriptor Table Register. Defines per-process segment descriptors.
•INVPCID — Invalidate Process-Context Identifier cache entries. TLB management instruction.

Category 3: I/O Instructions

•IN — Read from I/O port. Direct hardware register access.
•OUT — Write to I/O port. Direct hardware control.
•INS/OUTS — String versions for block I/O transfer.
•Note: These can be selectively allowed via the I/O Permission Bitmap in the TSS, but default to privileged.

Category 4: Control Register and MSR Access

•MOV to CRn — Write to control registers CR0, CR2, CR3, CR4, CR8. These control paging, protection, and core CPU features.
•WRMSR — Write Model-Specific Register. Configure CPU features, performance counters, security settings.
•RDMSR — Read Model-Specific Register (often privileged to prevent information leakage).
•XSETBV — Write to Extended Control Register for AVX/extended state.

Category 5: System State Instructions

•HLT — Halt processor until next interrupt. Would freeze system if used with interrupts disabled.
•LMSW — Load Machine Status Word (legacy CR0 load).
•CLTS — Clear Task-Switched flag in CR0.
•LTR — Load Task Register (hardware task switching).
•WBINVD — Write-back and invalidate all caches. Severe performance impact.

x86/x64 Privileged Instructions Reference

The x86 architecture has a well-defined set of privileged instructions. Here's a comprehensive reference with their purpose and why each requires Kernel Mode.

x86_privileged_instructions.asm
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
; ========== INTERRUPT CONTROL ==========
CLI         ; Clear Interrupt Flag — disables maskable interrupts
STI         ; Set Interrupt Flag — enables maskable interrupts
 
; ========== DESCRIPTOR TABLE OPERATIONS ==========
LGDT [mem]  ; Load Global Descriptor Table Register
SGDT [mem]  ; Store GDT Register (was privileged pre-Pentium, now allowed)
LIDT [mem]  ; Load Interrupt Descriptor Table Register
SIDT [mem]  ; Store IDT Register (was privileged pre-Pentium, now allowed)
LLDT ax     ; Load Local Descriptor Table from selector
SLDT [mem]  ; Store LDT selector
LTR ax      ; Load Task Register
STR [mem]   ; Store Task Register
 
; ========== CONTROL REGISTER ACCESS ==========
MOV CR0, rax  ; Load Control Register 0 (paging, protection enable)
MOV CR2, rax  ; Load CR2 (normally read-only page fault address)
MOV CR3, rax  ; Load CR3 (page table base address)
MOV CR4, rax  ; Load CR4 (extended control features)
MOV CR8, rax  ; Load CR8 (task priority for interrupt masking)
MOV rax, CRn  ; Reading CRs is also privileged
 
; ========== MODEL-SPECIFIC REGISTERS ==========
RDMSR       ; Read MSR (ECX = MSR index, output EDX:EAX)
WRMSR       ; Write MSR (ECX = MSR index, input EDx:EAX)
 
; ========== I/O OPERATIONS ==========
IN al, dx   ; Read byte from port DX
IN ax, dx   ; Read word from port DX
IN eax, dx  ; Read dword from port DX
OUT dx, al  ; Write byte to port DX
INS/OUTS    ; String I/O operations
 
; ========== CACHE CONTROL ==========
INVD        ; Invalidate caches (no writeback!)
WBINVD      ; Write-back and invalidate caches
INVLPG [mem]; Invalidate TLB entry for address
 
; ========== SYSTEM CONTROL ==========
HLT         ; Halt processor until interrupt
LMSW ax     ; Load Machine Status Word (legacy)
CLTS        ; Clear Task-Switched flag in CR0

Ring 0, Ring 3, and the I/O Permission Bitmap

IN/OUT instructions can be selectively allowed for User Mode processes via the I/O Permission Bitmap (IOPB) in the TSS. The kernel can grant specific port ranges to specific processes. However, this is rarely used in modern systems—typically only for legacy applications or specialized drivers running in user space.

ARM Architecture Privileged Operations

ARM uses a different privilege model with Exception Levels (EL0-EL3). Operations are restricted based on the minimum EL required:

EL1+ Required (OS Kernel Privilege):

ARM System Register Access by Exception Level
System Register	Minimum EL	Purpose
TTBR0_EL1	EL1	User space page table base
TTBR1_EL1	EL1	Kernel space page table base
SCTLR_EL1	EL1	System control (MMU enable, caches)
TCR_EL1	EL1	Translation control (page size, etc.)
MAIR_EL1	EL1	Memory attribute encoding
VBAR_EL1	EL1	Exception vector base address
ESR_EL1	EL1	Exception syndrome (fault reason)
FAR_EL1	EL1	Fault address register
SPSR_EL1	EL1	Saved process status (for exception return)
ELR_EL1	EL1	Exception link register (return address)

arm_privileged_operations.asm
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
; ARM (AArch64) Privileged Operations
 
; === SYSTEM REGISTER ACCESS ===
; Only accessible at EL1 or higher:
 
MSR SCTLR_EL1, x0   ; Write System Control Register
MRS x0, SCTLR_EL1   ; Read System Control Register
 
MSR TTBR0_EL1, x0   ; Write Translation Table Base 0
MRS x0, TTBR0_EL1   ; Read Translation Table Base 0
 
MSR VBAR_EL1, x0    ; Write Vector Base Address Register
MRS x0, ESR_EL1     ; Read Exception Syndrome Register
 
; === CACHE AND TLB OPERATIONS ===
DC IVAC, x0         ; Data Cache Invalidate by VA (requires privilege)
DC CIVAC, x0        ; Data Cache Clean and Invalidate by VA
IC IALLU            ; Instruction Cache Invalidate All (to Point of Unification)
TLBI VMALLE1        ; TLB Invalidate All, EL1
TLBI VAE1, x0       ; TLB Invalidate by VA, EL1
 
; === EXCEPTION RETURN ===
ERET                ; Exception Return — restores SPSR and jumps to ELR
                    ; This changes exception level (privilege level)
 
; === SUPERVISOR CALL (goes UP to EL1) ===
SVC #imm            ; Supervisor Call — triggers exception to EL1
                    ; This is how user code requests kernel services
 
; === WAIT FOR INTERRUPT/EVENT ===
WFI                 ; Wait For Interrupt (privileged in some configs)
WFE                 ; Wait For Event (may be privileged)

ARM's Cleaner Model

ARM's system register naming convention includes the exception level: SCTLR_EL1 is only accessible at EL1+, SCTLR_EL2 at EL2+. This makes privilege requirements self-documenting. There's no equivalent of x86's scattered privilege rules where you have to consult a manual for each instruction.

What Happens When User Mode Attempts Privileged Instructions

When User Mode code attempts to execute a privileged instruction, the CPU immediately generates an exception. This is a hardware-enforced response that cannot be bypassed or ignored. Let's trace exactly what happens.

The Exception Flow:

privileged_violation_flow.txt
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
User Program                  CPU Hardware                    Kernel
     |                              |                              |
     | Execute CLI instruction      |                              |
     |----------------------------->|                              |
     |                              | Check CPL: CPL=3 (User Mode) |
     |                              | CLI requires CPL=0           |
     |                              | CPL > Required: VIOLATION    |
     |                              |                              |
     |                              | === GENERATE #GP EXCEPTION ===
     |                              |                              |
     |                              | 1. Save SS:RSP to internal   |
     |                              | 2. Load SS:RSP from TSS      |
     |                              | 3. Push SS, RSP, RFLAGS,     |
     |                              |    CS, RIP to kernel stack   |
     |                              | 4. Push error code (0)       |
     |                              | 5. Set CPL = 0               |
     |                              | 6. Load new CS:RIP from IDT  |
     |                              |----------------------------->|
     |                              |                              | #GP Handler Runs
     |                              |                              | - Examine error code
     |                              |                              | - Examine faulting RIP
     |                              |                              | - Process attempted CLI
     |                              |                              | - Decision: TERMINATE
     |                              |                              |
     X  SIGILL/SIGSEGV/Access Violation                           |
        Process Terminated                                         |

Exception Types for Privilege Violations:

Violation Type	x86 Exception	ARM Exception	Signal
Privileged instruction	#GP (13)	Undefined Instruction	SIGILL
I/O instruction (if IOPL < CPL)	#GP (13)	N/A	SIGSEGV
Supervisor memory access	#PF (14)	Data Abort	SIGSEGV
Supervisor code execution	#PF (14)	Prefetch Abort	SIGSEGV

The Kernel's Response:

When the kernel's exception handler receives a #GP caused by a privileged instruction:

Identify the faulting instruction by examining RIP (instruction pointer)
Determine the cause — was it a privileged instruction or something else?
Deliver a signal to the process (SIGILL for illegal instruction)
If no handler, terminate the process with a core dump

User-Visible Results:

privileged_instruction_crash.c
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
// Example: Attempting a privileged instruction in C
// This program will crash immediately
 
#include <stdio.h>
 
int main() {
    printf("About to attempt CLI (clear interrupts)...\n");
    
    // Inline assembly to attempt a privileged instruction
    __asm__ volatile ("cli");  // Clear Interrupt Flag
    
    // This line will NEVER execute
    printf("CLI succeeded!\n");
    return 0;
}
 
/*
$ gcc -o priv_test priv_test.c
$ ./priv_test
About to attempt CLI (clear interrupts)...
Segmentation fault (core dumped)
 
$ dmesg | tail -1
[12345.678] traps: priv_test[1234] general protection...
 
$ gdb ./priv_test core
(gdb) bt
#0 0x0000555555555149 in main () at priv_test.c:8
(gdb) x/i $rip
=> 0x555555555149: cli
*/

No Second Chances

Unlike some errors (e.g., page faults for demand paging), privilege violations are fatal. There's no scenario where the kernel allows the privileged instruction to proceed for a User Mode process. The only exception would be if a debugger is controlling the process and the kernel emulates the instruction for debugging purposes.

Sensitive vs. Privileged Instructions

An important distinction exists between privileged instructions and sensitive instructions. This distinction becomes critical for virtualization.

Definitions:

Term	Definition
Privileged Instruction	Causes a trap when executed in User Mode (always Ring 0 required)
Sensitive Instruction	Affects system state or behaves differently based on CPL
Innocuous Instruction	Same behavior regardless of privilege level

The key insight: All privileged instructions are sensitive, but not all sensitive instructions are privileged.

Examples of Sensitive-but-Unprivileged Instructions on x86:

Sensitive but Unprivileged Instructions (Virtualization Traps)
Instruction	Why Sensitive	Problem for Virtualization
SGDT/SIDT	Reveals system table addresses	Can detect if running in a VM
POPF	Can modify the interrupt flag	In Ring 3, POPF silently ignores IF modification
SMSW	Reads CR0 (protection/paging status)	Reveals system state
STR	Returns current task register	Can reveal VM context
CPUID	Returns processor information	Can reveal hypervisor presence

Why This Matters for Virtualization:

The Popek and Goldberg virtualization requirements (1974) state that for efficient virtualization:

All sensitive instructions must be privileged.

If a sensitive instruction doesn't trap, the hypervisor can't intercept and emulate it. The guest OS executes the real instruction and observes real hardware state, breaking the virtualization illusion.

x86's Historical Problem:

x86 originally had several sensitive-but-unprivileged instructions, making pure software virtualization difficult. Solutions included:

Binary translation (VMware): Scan and rewrite sensitive instructions
Paravirtualization (Xen): Modify guest OS to not use these instructions
Hardware virtualization (Intel VT-x, AMD-V): Add new CPU mode that traps all sensitive instructions

Modern x86 with VT-x:

Hardware virtualization extensions solve this by adding a new privilege level (VMX root mode) and configuration bits that control which instructions trap to the hypervisor. Now all sensitive instructions can be made to trap, enabling efficient virtualization.

RISC-V Gets It Right

RISC-V, designed in the 2010s with virtualization in mind, has no sensitive-but-unprivileged instructions. All sensitive operations are privileged and trap properly. This clean-slate design avoids the complexity legacy x86 needed to work around its historical gaps.

How the Kernel Uses Privileged Instructions

Let's examine real scenarios where the kernel executes privileged instructions, illustrating why these operations must be protected.

Scenario 1: Context Switch

When switching between processes, the kernel must:

context_switch_privileged.txt
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
// Simplified context switch flow
 
void switch_to(struct task *next) {
    // Save current process state (not privileged)
    save_registers(&current->context);
    save_fpu_state(&current->fpu);
    
    // PRIVILEGED: Switch address space
    // MOV CR3, next->mm->pgd_phys
    load_cr3(next->mm->pgd);  // Changes page tables!
    
    // PRIVILEGED: Update kernel stack in TSS
    // For next process's kernel stack
    set_tss_rsp0(next->kernel_stack + STACK_SIZE);
    
    // PRIVILEGED: Potentially update LDT
    if (next->mm->ldt != current->mm->ldt) {
        lldt(next->mm->ldt);  // LLDT instruction
    }
    
    // Restore next process state (not privileged)
    restore_registers(&next->context);
    
    // PRIVILEGED: Return to user mode
    // IRET or SYSRET instruction, changing CPL from 0 to 3
}

Scenario 2: Interrupt Handling

When a hardware interrupt arrives (e.g., keyboard press):

interrupt_handling_privileged.txt
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
// Hardware automatically executes (via microcode):
// 1. Saves RSP, RFLAGS, CS, RIP to kernel stack
// 2. Sets CPL = 0 (PRIVILEGED transition)
// 3. Jumps to handler from IDT
 
void keyboard_interrupt_handler(struct pt_regs *regs) {
    // PRIVILEGED: Read scan code from hardware
    uint8_t scancode = inb(0x60);  // IN instruction
    
    // Process the key (not privileged)
    process_scancode(scancode);
    
    // PRIVILEGED: Acknowledge interrupt to PIC
    outb(0x20, 0x20);  // OUT instruction to EOI
    
    // PRIVILEGED: Return from interrupt
    // IRET: restores RFLAGS, CS:RIP, SS:RSP
    // Changes CPL back to 3 (user mode)
}

Scenario 3: Enabling a New CPU Feature

When the kernel wants to enable AVX-512 for user processes:

enable_cpu_feature.txt
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
void enable_avx512(void) {
    uint64_t cr4;
    
    // PRIVILEGED: Read current CR4
    cr4 = read_cr4();
    
    // PRIVILEGED: Enable OSXSAVE (bit 18) in CR4
    cr4 |= (1 << 18);  // X86_CR4_OSXSAVE
    write_cr4(cr4);
    
    // PRIVILEGED: Configure XSAVE mask via XCR0
    // XSETBV instruction (extended control register)
    wrxcr(0, XFEATURE_MASK_AVX512);
    
    // Now user mode can use AVX-512 instructions
    // (but can't control which features are enabled)
}

Observing Privileged Instruction Usage

You can observe kernel use of privileged instructions with 'sudo perf stat -e instructions:k sleep 1' to count kernel-mode instructions, or trace specific instructions with eBPF. On Windows, the Windows Performance Toolkit can capture detailed kernel activity including privileged operations.

Summary: Privileged Instructions

Privileged instructions form the instruction-level foundation of OS security—specific CPU operations that would be catastrophic if available to untrusted code. Let's consolidate our understanding:

Key Takeaways

•Privileged instructions require Kernel Mode — They trap immediately if attempted in User Mode, generating a #GP (x86) or Undefined Instruction exception (ARM).
•Categories include interrupt control, memory management, I/O, and system registers — Each protects a different aspect of system integrity.
•Regular computation is unrestricted — Arithmetic, logic, and control flow don't affect system state and can run in User Mode.
•Violations cause immediate process termination — The kernel delivers SIGILL or similar, and the process dies with no second chance.
•Sensitive ≠ Privileged — Some instructions affect state or reveal information but don't trap; this was a problem for early virtualization.
•The kernel uses privileged instructions for fundamental operations — Context switching, interrupt handling, memory management all require operations that cannot safely be exposed to applications.
•Hardware virtualization addresses sensitive-but-unprivileged instructions — VT-x/AMD-V add new modes where all sensitive instructions can trap.

Looking ahead:

We've seen what User Mode and Kernel Mode are, how the Mode Bit tracks privilege, and which instructions are restricted. But how does code actually transition between modes? The next page examines Mode Switching—the mechanisms by which User Mode code enters Kernel Mode (system calls, interrupts) and how control returns to User Mode.

Page Complete

You now understand privileged instructions: the CPU operations that are forbidden to User Mode code because they could compromise system security or stability. These instruction-level restrictions, enforced by hardware, ensure that only the trusted kernel can perform operations that affect system-wide state.

4 / 5

Loading learning content...

Operating SystemsCPU Execution Modes

CPU Execution Modes

LevelBeginner

Duration60 mins

TopicCPU Execution Modes

4 / 5

Privileged Instructions

The Forbidden Operations

Examples of privileged operations:

Disabling interrupts (could freeze the system indefinitely)
Loading new page tables (could access any memory)
Communicating directly with hardware (could bypass all access control)
Halting the CPU (could denial-of-service the machine)

Every privileged instruction represents a "what if untrusted code could do this?" thought experiment with catastrophic answers.

What You Will Learn

Why Privileged Instructions Exist

The concept of privileged instructions emerges directly from the need to maintain system security and stability. Without restrictions, any program could:

1. Compromise Memory Protection

Instructions that modify page tables or segment descriptors control which memory each process can access. If unrestricted:

A malicious program could map another process's memory into its own space
Passwords, encryption keys, and private data would be exposed
The entire premise of process isolation would collapse

2. Monopolize Resources

Instructions that disable interrupts or halt the CPU would let a program:

Prevent the scheduler from ever running (infinite CPU monopoly)
Block keyboard/mouse input (denial-of-service attacks)
Prevent the system from responding to any event

3. Corrupt System State

Instructions that modify control registers, descriptor tables, or system configuration could:

Disable memory protection entirely
Change the interrupt handling behavior
Put the CPU into an undefined or crashed state

Threat Analysis: Instructions Without Privilege Restriction
If This Instruction Were Unrestricted	An Attacker Could
CLI (Clear Interrupt Flag)	Disable all interrupts, freeze the system, prevent preemption forever
MOV to CR3 (Page Table Base)	Swap in their own page tables, access ANY memory address
IN/OUT (Port I/O)	Read any disk sector directly, bypass filesystem permissions
WRMSR (Write MSR)	Modify CPU features, disable security protections like SMEP/SMAP
HLT (Halt CPU)	Stop the processor, crash the system with one instruction
LGDT (Load GDT)	Install new segment descriptors, redefine memory layout

The Minimal Privilege Principle

Categories of Privileged Instructions

Privileged instructions fall into several logical categories, each protecting a different aspect of system security:

Category 1: Interrupt and Exception Control

These instructions control whether and how the CPU responds to interrupts:

CLI (Clear Interrupt Flag): Disables maskable interrupts
STI (Set Interrupt Flag): Enables maskable interrupts
LIDT (Load IDT Register): Changes the Interrupt Descriptor Table pointer
IRET (Interrupt Return): Returns from interrupt (indirectly affects interrupt state)

Why Privileged? If user code could disable interrupts, the timer interrupt would never fire, the scheduler would never run, and the user process would monopolize the CPU forever.

Category 2: Memory Management Instructions

•MOV to CR3 — Load Page Directory Base Register. Changes which page tables are active, controlling all memory access permissions.
•INVLPG — Invalidate TLB entry. Forces re-translation of a virtual address; could be used to cause consistency issues.
•LGDT — Load Global Descriptor Table Register. Defines segment base/limit for all segments.
•LLDT — Load Local Descriptor Table Register. Defines per-process segment descriptors.
•INVPCID — Invalidate Process-Context Identifier cache entries. TLB management instruction.

Category 3: I/O Instructions

•IN — Read from I/O port. Direct hardware register access.
•OUT — Write to I/O port. Direct hardware control.
•INS/OUTS — String versions for block I/O transfer.
•Note: These can be selectively allowed via the I/O Permission Bitmap in the TSS, but default to privileged.

Category 4: Control Register and MSR Access

•MOV to CRn — Write to control registers CR0, CR2, CR3, CR4, CR8. These control paging, protection, and core CPU features.
•WRMSR — Write Model-Specific Register. Configure CPU features, performance counters, security settings.
•RDMSR — Read Model-Specific Register (often privileged to prevent information leakage).
•XSETBV — Write to Extended Control Register for AVX/extended state.

Category 5: System State Instructions

•HLT — Halt processor until next interrupt. Would freeze system if used with interrupts disabled.
•LMSW — Load Machine Status Word (legacy CR0 load).
•CLTS — Clear Task-Switched flag in CR0.
•LTR — Load Task Register (hardware task switching).
•WBINVD — Write-back and invalidate all caches. Severe performance impact.

x86/x64 Privileged Instructions Reference

The x86 architecture has a well-defined set of privileged instructions. Here's a comprehensive reference with their purpose and why each requires Kernel Mode.

x86_privileged_instructions.asm
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
; ========== INTERRUPT CONTROL ==========
CLI         ; Clear Interrupt Flag — disables maskable interrupts
STI         ; Set Interrupt Flag — enables maskable interrupts
 
; ========== DESCRIPTOR TABLE OPERATIONS ==========
LGDT [mem]  ; Load Global Descriptor Table Register
SGDT [mem]  ; Store GDT Register (was privileged pre-Pentium, now allowed)
LIDT [mem]  ; Load Interrupt Descriptor Table Register
SIDT [mem]  ; Store IDT Register (was privileged pre-Pentium, now allowed)
LLDT ax     ; Load Local Descriptor Table from selector
SLDT [mem]  ; Store LDT selector
LTR ax      ; Load Task Register
STR [mem]   ; Store Task Register
 
; ========== CONTROL REGISTER ACCESS ==========
MOV CR0, rax  ; Load Control Register 0 (paging, protection enable)
MOV CR2, rax  ; Load CR2 (normally read-only page fault address)
MOV CR3, rax  ; Load CR3 (page table base address)
MOV CR4, rax  ; Load CR4 (extended control features)
MOV CR8, rax  ; Load CR8 (task priority for interrupt masking)
MOV rax, CRn  ; Reading CRs is also privileged
 
; ========== MODEL-SPECIFIC REGISTERS ==========
RDMSR       ; Read MSR (ECX = MSR index, output EDX:EAX)
WRMSR       ; Write MSR (ECX = MSR index, input EDx:EAX)
 
; ========== I/O OPERATIONS ==========
IN al, dx   ; Read byte from port DX
IN ax, dx   ; Read word from port DX
IN eax, dx  ; Read dword from port DX
OUT dx, al  ; Write byte to port DX
INS/OUTS    ; String I/O operations
 
; ========== CACHE CONTROL ==========
INVD        ; Invalidate caches (no writeback!)
WBINVD      ; Write-back and invalidate caches
INVLPG [mem]; Invalidate TLB entry for address
 
; ========== SYSTEM CONTROL ==========
HLT         ; Halt processor until interrupt
LMSW ax     ; Load Machine Status Word (legacy)
CLTS        ; Clear Task-Switched flag in CR0

Ring 0, Ring 3, and the I/O Permission Bitmap

ARM Architecture Privileged Operations

ARM uses a different privilege model with Exception Levels (EL0-EL3). Operations are restricted based on the minimum EL required:

EL1+ Required (OS Kernel Privilege):

ARM System Register Access by Exception Level
System Register	Minimum EL	Purpose
TTBR0_EL1	EL1	User space page table base
TTBR1_EL1	EL1	Kernel space page table base
SCTLR_EL1	EL1	System control (MMU enable, caches)
TCR_EL1	EL1	Translation control (page size, etc.)
MAIR_EL1	EL1	Memory attribute encoding
VBAR_EL1	EL1	Exception vector base address
ESR_EL1	EL1	Exception syndrome (fault reason)
FAR_EL1	EL1	Fault address register
SPSR_EL1	EL1	Saved process status (for exception return)
ELR_EL1	EL1	Exception link register (return address)

arm_privileged_operations.asm
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
; ARM (AArch64) Privileged Operations
 
; === SYSTEM REGISTER ACCESS ===
; Only accessible at EL1 or higher:
 
MSR SCTLR_EL1, x0   ; Write System Control Register
MRS x0, SCTLR_EL1   ; Read System Control Register
 
MSR TTBR0_EL1, x0   ; Write Translation Table Base 0
MRS x0, TTBR0_EL1   ; Read Translation Table Base 0
 
MSR VBAR_EL1, x0    ; Write Vector Base Address Register
MRS x0, ESR_EL1     ; Read Exception Syndrome Register
 
; === CACHE AND TLB OPERATIONS ===
DC IVAC, x0         ; Data Cache Invalidate by VA (requires privilege)
DC CIVAC, x0        ; Data Cache Clean and Invalidate by VA
IC IALLU            ; Instruction Cache Invalidate All (to Point of Unification)
TLBI VMALLE1        ; TLB Invalidate All, EL1
TLBI VAE1, x0       ; TLB Invalidate by VA, EL1
 
; === EXCEPTION RETURN ===
ERET                ; Exception Return — restores SPSR and jumps to ELR
                    ; This changes exception level (privilege level)
 
; === SUPERVISOR CALL (goes UP to EL1) ===
SVC #imm            ; Supervisor Call — triggers exception to EL1
                    ; This is how user code requests kernel services
 
; === WAIT FOR INTERRUPT/EVENT ===
WFI                 ; Wait For Interrupt (privileged in some configs)
WFE                 ; Wait For Event (may be privileged)

ARM's Cleaner Model

What Happens When User Mode Attempts Privileged Instructions

The Exception Flow:

privileged_violation_flow.txt
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
User Program                  CPU Hardware                    Kernel
     |                              |                              |
     | Execute CLI instruction      |                              |
     |----------------------------->|                              |
     |                              | Check CPL: CPL=3 (User Mode) |
     |                              | CLI requires CPL=0           |
     |                              | CPL > Required: VIOLATION    |
     |                              |                              |
     |                              | === GENERATE #GP EXCEPTION ===
     |                              |                              |
     |                              | 1. Save SS:RSP to internal   |
     |                              | 2. Load SS:RSP from TSS      |
     |                              | 3. Push SS, RSP, RFLAGS,     |
     |                              |    CS, RIP to kernel stack   |
     |                              | 4. Push error code (0)       |
     |                              | 5. Set CPL = 0               |
     |                              | 6. Load new CS:RIP from IDT  |
     |                              |----------------------------->|
     |                              |                              | #GP Handler Runs
     |                              |                              | - Examine error code
     |                              |                              | - Examine faulting RIP
     |                              |                              | - Process attempted CLI
     |                              |                              | - Decision: TERMINATE
     |                              |                              |
     X  SIGILL/SIGSEGV/Access Violation                           |
        Process Terminated                                         |

Exception Types for Privilege Violations:

Violation Type	x86 Exception	ARM Exception	Signal
Privileged instruction	#GP (13)	Undefined Instruction	SIGILL
I/O instruction (if IOPL < CPL)	#GP (13)	N/A	SIGSEGV
Supervisor memory access	#PF (14)	Data Abort	SIGSEGV
Supervisor code execution	#PF (14)	Prefetch Abort	SIGSEGV

The Kernel's Response:

When the kernel's exception handler receives a #GP caused by a privileged instruction:

Identify the faulting instruction by examining RIP (instruction pointer)
Determine the cause — was it a privileged instruction or something else?
Deliver a signal to the process (SIGILL for illegal instruction)
If no handler, terminate the process with a core dump

User-Visible Results:

privileged_instruction_crash.c
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
// Example: Attempting a privileged instruction in C
// This program will crash immediately
 
#include <stdio.h>
 
int main() {
    printf("About to attempt CLI (clear interrupts)...\n");
    
    // Inline assembly to attempt a privileged instruction
    __asm__ volatile ("cli");  // Clear Interrupt Flag
    
    // This line will NEVER execute
    printf("CLI succeeded!\n");
    return 0;
}
 
/*
$ gcc -o priv_test priv_test.c
$ ./priv_test
About to attempt CLI (clear interrupts)...
Segmentation fault (core dumped)
 
$ dmesg | tail -1
[12345.678] traps: priv_test[1234] general protection...
 
$ gdb ./priv_test core
(gdb) bt
#0 0x0000555555555149 in main () at priv_test.c:8
(gdb) x/i $rip
=> 0x555555555149: cli
*/

No Second Chances

Sensitive vs. Privileged Instructions

An important distinction exists between privileged instructions and sensitive instructions. This distinction becomes critical for virtualization.

Definitions:

Term	Definition
Privileged Instruction	Causes a trap when executed in User Mode (always Ring 0 required)
Sensitive Instruction	Affects system state or behaves differently based on CPL
Innocuous Instruction	Same behavior regardless of privilege level

The key insight: All privileged instructions are sensitive, but not all sensitive instructions are privileged.

Examples of Sensitive-but-Unprivileged Instructions on x86:

Sensitive but Unprivileged Instructions (Virtualization Traps)
Instruction	Why Sensitive	Problem for Virtualization
SGDT/SIDT	Reveals system table addresses	Can detect if running in a VM
POPF	Can modify the interrupt flag	In Ring 3, POPF silently ignores IF modification
SMSW	Reads CR0 (protection/paging status)	Reveals system state
STR	Returns current task register	Can reveal VM context
CPUID	Returns processor information	Can reveal hypervisor presence

Why This Matters for Virtualization:

The Popek and Goldberg virtualization requirements (1974) state that for efficient virtualization:

All sensitive instructions must be privileged.

x86's Historical Problem:

x86 originally had several sensitive-but-unprivileged instructions, making pure software virtualization difficult. Solutions included:

Binary translation (VMware): Scan and rewrite sensitive instructions
Paravirtualization (Xen): Modify guest OS to not use these instructions
Hardware virtualization (Intel VT-x, AMD-V): Add new CPU mode that traps all sensitive instructions

Modern x86 with VT-x:

RISC-V Gets It Right

How the Kernel Uses Privileged Instructions

Let's examine real scenarios where the kernel executes privileged instructions, illustrating why these operations must be protected.

Scenario 1: Context Switch

When switching between processes, the kernel must:

context_switch_privileged.txt
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
// Simplified context switch flow
 
void switch_to(struct task *next) {
    // Save current process state (not privileged)
    save_registers(&current->context);
    save_fpu_state(&current->fpu);
    
    // PRIVILEGED: Switch address space
    // MOV CR3, next->mm->pgd_phys
    load_cr3(next->mm->pgd);  // Changes page tables!
    
    // PRIVILEGED: Update kernel stack in TSS
    // For next process's kernel stack
    set_tss_rsp0(next->kernel_stack + STACK_SIZE);
    
    // PRIVILEGED: Potentially update LDT
    if (next->mm->ldt != current->mm->ldt) {
        lldt(next->mm->ldt);  // LLDT instruction
    }
    
    // Restore next process state (not privileged)
    restore_registers(&next->context);
    
    // PRIVILEGED: Return to user mode
    // IRET or SYSRET instruction, changing CPL from 0 to 3
}

Scenario 2: Interrupt Handling

When a hardware interrupt arrives (e.g., keyboard press):

interrupt_handling_privileged.txt
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
// Hardware automatically executes (via microcode):
// 1. Saves RSP, RFLAGS, CS, RIP to kernel stack
// 2. Sets CPL = 0 (PRIVILEGED transition)
// 3. Jumps to handler from IDT
 
void keyboard_interrupt_handler(struct pt_regs *regs) {
    // PRIVILEGED: Read scan code from hardware
    uint8_t scancode = inb(0x60);  // IN instruction
    
    // Process the key (not privileged)
    process_scancode(scancode);
    
    // PRIVILEGED: Acknowledge interrupt to PIC
    outb(0x20, 0x20);  // OUT instruction to EOI
    
    // PRIVILEGED: Return from interrupt
    // IRET: restores RFLAGS, CS:RIP, SS:RSP
    // Changes CPL back to 3 (user mode)
}

Scenario 3: Enabling a New CPU Feature

When the kernel wants to enable AVX-512 for user processes:

enable_cpu_feature.txt
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
void enable_avx512(void) {
    uint64_t cr4;
    
    // PRIVILEGED: Read current CR4
    cr4 = read_cr4();
    
    // PRIVILEGED: Enable OSXSAVE (bit 18) in CR4
    cr4 |= (1 << 18);  // X86_CR4_OSXSAVE
    write_cr4(cr4);
    
    // PRIVILEGED: Configure XSAVE mask via XCR0
    // XSETBV instruction (extended control register)
    wrxcr(0, XFEATURE_MASK_AVX512);
    
    // Now user mode can use AVX-512 instructions
    // (but can't control which features are enabled)
}

Observing Privileged Instruction Usage

Summary: Privileged Instructions

Privileged instructions form the instruction-level foundation of OS security—specific CPU operations that would be catastrophic if available to untrusted code. Let's consolidate our understanding:

Key Takeaways

•Privileged instructions require Kernel Mode — They trap immediately if attempted in User Mode, generating a #GP (x86) or Undefined Instruction exception (ARM).
•Categories include interrupt control, memory management, I/O, and system registers — Each protects a different aspect of system integrity.
•Regular computation is unrestricted — Arithmetic, logic, and control flow don't affect system state and can run in User Mode.
•Violations cause immediate process termination — The kernel delivers SIGILL or similar, and the process dies with no second chance.
•Sensitive ≠ Privileged — Some instructions affect state or reveal information but don't trap; this was a problem for early virtualization.
•The kernel uses privileged instructions for fundamental operations — Context switching, interrupt handling, memory management all require operations that cannot safely be exposed to applications.
•Hardware virtualization addresses sensitive-but-unprivileged instructions — VT-x/AMD-V add new modes where all sensitive instructions can trap.

Looking ahead:

Page Complete

4 / 5