Operating SystemsLinux Scheduling

Linux Scheduling

LevelAdvanced

Duration90 mins

TopicLinux Scheduling

4 / 5

Real-Time Policies

When Fairness Is Not Enough

CFS excels at providing proportionally fair CPU access across competing workloads. But some applications have requirements that fairness cannot satisfy:

Audio processing: A buffer underrun causes audible glitches—latency must be bounded, not averaged
Industrial control: A missed deadline can cause physical damage or safety hazards
High-frequency trading: Microseconds of latency directly translate to financial loss
Telecommunications: Packet processing must meet strict timing to maintain connection quality

For these workloads, Linux provides real-time scheduling policies that operate outside the CFS framework. Real-time tasks receive absolute priority over normal tasks—they preempt immediately and run until they voluntarily yield or block.

This page explores Linux's real-time capabilities: the SCHED_FIFO and SCHED_RR policies for static priority real-time, the newer SCHED_DEADLINE for deadline-based scheduling, and the trade-offs and configurations that make real-time work on a general-purpose operating system.

Learning Objectives

By the end of this page, you will understand: (1) The distinction between hard and soft real-time, (2) SCHED_FIFO and SCHED_RR policy semantics, (3) Real-time priority levels and their relationship to CFS, (4) The SCHED_DEADLINE policy and EDF scheduling, and (5) Configuration, safety mechanisms, and practical considerations.

Real-Time Scheduling Concepts

Real-time systems are characterized by correctness depending not just on computational results but on when those results are produced. Understanding the taxonomy of real-time systems is essential before diving into Linux's implementation.

Hard vs. Soft Real-Time

Hard real-time systems have absolute deadlines. Missing a deadline constitutes system failure, potentially with catastrophic consequences:

Aircraft flight controls
Nuclear reactor monitoring
Automotive airbag deployment
Pacemaker timing

Soft real-time systems have deadlines where occasional misses are tolerable, typically degrading quality rather than causing failure:

Video playback (dropped frames cause visual glitches)
Audio processing (buffer underruns cause pops)
Interactive gaming (frame timing affects smoothness)
Voice over IP (late packets cause audio artifacts)

Linux Is Not a Hard Real-Time OS

Standard Linux is designed for soft real-time. While it provides real-time scheduling policies, it cannot guarantee microsecond-level deadline bounds due to kernel preemption points, interrupt handling latency, and driver behavior. For hard real-time, specialized systems like PREEMPT_RT patches, Xenomai, or dedicated RTOSes are required.

Priority-Based vs. Deadline-Based Scheduling

Linux offers two paradigms for real-time scheduling:

Static Priority (Rate-Monotonic): Tasks are assigned fixed priority levels. Higher-priority tasks always preempt lower-priority ones. The programmer/administrator determines priorities at design time.

SCHED_FIFO: Run until block or yield
SCHED_RR: Run with time-slicing among equal priorities

Dynamic Priority (Deadline-Based): Tasks specify their timing requirements (period, deadline, execution time). The scheduler dynamically orders tasks by deadline urgency.

SCHED_DEADLINE: Earliest Deadline First (EDF) scheduling

The Scheduling Class Hierarchy

Linux organizes schedulers into classes with strict priority:

STOP class (highest)  → kernel threads (migration, watchdog)
    ↓
DL class              → SCHED_DEADLINE tasks
    ↓
RT class              → SCHED_FIFO, SCHED_RR tasks (priorities 1-99)
    ↓
FAIR class            → SCHED_NORMAL (CFS) tasks
    ↓
IDLE class (lowest)   → SCHED_IDLE tasks

A higher class always preempts lower classes. Within each class, the class-specific algorithm determines scheduling.

Key Real-Time Terminology

•Period (T): Time between successive releases of a periodic task. Audio at 48kHz has T = 20.83μs periods.
•Deadline (D): Time by which task must complete after being released. Often D = T for periodic tasks.
•Worst-Case Execution Time (WCET): Maximum time the task might need. Must be known for schedulability.
•Utilization (U): WCET / Period. Total utilization bounds schedulability.
•Jitter: Variation in timing. Low jitter indicates consistent, predictable behavior.
•Latency: Delay from event (interrupt, wake) to response. Critical for responsiveness.

SCHED_FIFO: First-In-First-Out Real-Time

SCHED_FIFO is the simplest real-time policy. A SCHED_FIFO task runs until:

It voluntarily yields (sched_yield)
It blocks (I/O, sleep, mutex)
A higher-priority real-time task becomes runnable

Key Characteristics:

No time slicing: Unlike SCHED_RR, SCHED_FIFO tasks don't have time quantums
Static priority: Priority (1-99) is fixed at creation, doesn't change dynamically
FIFO ordering within priority: If multiple tasks share the same priority, they're queued in arrival order
Immediate preemption: Preempts CFS tasks instantly, preempts lower-priority RT tasks instantly

When to Use SCHED_FIFO:

Tasks with known, bounded execution times
Non-interactive, compute-bound real-time work
When you need deterministic ordering among equal-priority tasks
Latency-critical interrupt handling threads

sched_fifo_example.c
C
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
/*
 * SCHED_FIFO Real-Time Task Example
 *
 * This example creates a high-priority real-time task
 * that processes audio buffers with minimal latency.
 */
 
#include <pthread.h>
#include <sched.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <unistd.h>
#include <sys/mman.h>
 
#define AUDIO_PRIORITY 80  /* High RT priority (1-99 scale) */
 
void *audio_processing_thread(void *arg) {
    printf("Audio thread running with SCHED_FIFO priority %d\n", 
           AUDIO_PRIORITY);
    
    /* Simulate periodic audio processing */
    while (1) {
        /* 
         * In real code: 
         * 1. Wait for audio buffer ready (blocking)
         * 2. Process audio samples
         * 3. Output processed samples
         */
        process_audio_buffer();
        
        /* 
         * Key behavior: This thread will NOT be preempted by 
         * ANY CFS (normal) task, regardless of their nice level.
         * Only higher-priority RT tasks can preempt.
         */
    }
    
    return NULL;
}
 
int main() {
    pthread_t audio_thread;
    pthread_attr_t attr;
    struct sched_param param;
    int ret;
    
    /*
     * Lock memory to prevent page faults during RT operation.
     * Page faults cause unpredictable latency spikes.
     */
    if (mlockall(MCL_CURRENT | MCL_FUTURE) == -1) {
        perror("mlockall failed (need CAP_IPC_LOCK)");
    }
    
    /* Initialize thread attributes */
    pthread_attr_init(&attr);
    
    /* Set SCHED_FIFO policy */
    pthread_attr_setschedpolicy(&attr, SCHED_FIFO);
    
    /* Set priority (1-99, higher is higher priority) */
    param.sched_priority = AUDIO_PRIORITY;
    pthread_attr_setschedparam(&attr, &param);
    
    /* Ensure policy/priority are used (vs inheriting from parent) */
    pthread_attr_setinheritsched(&attr, PTHREAD_EXPLICIT_SCHED);
    
    /* Create the real-time thread */
    ret = pthread_create(&audio_thread, &attr, 
                         audio_processing_thread, NULL);
    if (ret != 0) {
        fprintf(stderr, "pthread_create failed: %s\n", strerror(ret));
        fprintf(stderr, "(Need CAP_SYS_NICE or root for RT scheduling)\n");
        return 1;
    }
    
    /* Main thread continues with normal scheduling... */
    pthread_join(audio_thread, NULL);
    
    return 0;
}
 
/*
 * Running this requires privileges:
 * 
 * Option 1: Run as root
 *   sudo ./audio_rt
 *
 * Option 2: Set capabilities
 *   sudo setcap cap_sys_nice,cap_ipc_lock+ep ./audio_rt
 *
 * Option 3: Configure rtprio limit in /etc/security/limits.conf
 *   @audio - rtprio 99
 */

SCHED_FIFO Behavior Nuances

Preemption Rules:

FIFO task A (priority 50) is running
FIFO task B (priority 60) becomes runnable
A is immediately preempted, B runs
When B blocks, A resumes

Same-Priority Ordering:

FIFO task C (priority 50) is running
FIFO task D (priority 50) becomes runnable
C continues until it blocks or yields
D runs next (FIFO order)
When D blocks, C runs if runnable

Yield Behavior:

sched_yield() moves the calling task to the end of its priority queue
Does NOT allow lower-priority tasks to run
Used for cooperative multitasking among equal-priority RT tasks

The Runaway SCHED_FIFO Danger

A SCHED_FIFO task that loops without blocking will completely starve all CFS tasks, including critical system processes like SSH. The system becomes unresponsive; only higher-priority RT tasks or a reboot can recover. Always ensure RT tasks have bounded execution and block appropriately. Linux provides sched_rt_runtime_us as a safety throttle.

SCHED_RR: Round-Robin Real-Time

SCHED_RR is identical to SCHED_FIFO with one crucial addition: time slicing among equal-priority tasks. When multiple SCHED_RR tasks share the same priority level, they round-robin using a configurable time quantum.

Key Differences from SCHED_FIFO:

Aspect	SCHED_FIFO	SCHED_RR
Time slice	None	Yes (default 100ms)
Equal-priority behavior	Run until yield/block	Round-robin
Preemption by higher	Immediate	Immediate
Use case	Single RT task per priority	Multiple RT tasks at same priority

When to Use SCHED_RR:

Multiple real-time tasks that need to share a priority level
When you want fairness within a real-time priority
Interactive real-time applications where responsiveness among RT tasks matters
Defensive scheduling to prevent any single RT task from monopolizing

sched_rr_example.c
C
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
/*
 * SCHED_RR Real-Time Task Example
 *
 * Creates multiple worker threads that share CPU time
 * fairly within the real-time class.
 */
 
#include <pthread.h>
#include <sched.h>
#include <stdio.h>
#include <unistd.h>
 
#define RT_PRIORITY 50
#define NUM_WORKERS 4
 
void *worker_thread(void *arg) {
    int id = *(int *)arg;
    int iterations = 0;
    
    printf("Worker %d started with SCHED_RR priority %d\n", 
           id, RT_PRIORITY);
    
    while (iterations < 10) {
        /* Simulate computational work */
        volatile long i;
        for (i = 0; i < 100000000; i++) {
            /* Busy work */
        }
        
        printf("Worker %d completed iteration %d\n", id, ++iterations);
        
        /*
         * With SCHED_RR:
         * - Each worker runs for the time quantum (default 100ms)
         * - Then is preempted for the next equal-priority worker
         * - All workers make progress concurrently
         *
         * With SCHED_FIFO (same priority):
         * - First worker would run all 10 iterations
         * - Then second worker, etc.
         * - No interleaving without explicit yields
         */
    }
    
    return NULL;
}
 
int main() {
    pthread_t threads[NUM_WORKERS];
    pthread_attr_t attr;
    struct sched_param param;
    int worker_ids[NUM_WORKERS];
    
    pthread_attr_init(&attr);
    pthread_attr_setschedpolicy(&attr, SCHED_RR);
    param.sched_priority = RT_PRIORITY;
    pthread_attr_setschedparam(&attr, &param);
    pthread_attr_setinheritsched(&attr, PTHREAD_EXPLICIT_SCHED);
    
    /* Create multiple RR workers at the same priority */
    for (int i = 0; i < NUM_WORKERS; i++) {
        worker_ids[i] = i;
        pthread_create(&threads[i], &attr, worker_thread, &worker_ids[i]);
    }
    
    /*
     * Query the RR time quantum for informational purposes
     */
    struct timespec ts;
    if (sched_rr_get_interval(0, &ts) == 0) {
        printf("RR time quantum: %ld.%09ld seconds\n", 
               ts.tv_sec, ts.tv_nsec);
    }
    
    for (int i = 0; i < NUM_WORKERS; i++) {
        pthread_join(threads[i], NULL);
    }
    
    return 0;
}
 
/*
 * Output pattern with SCHED_RR (interleaved):
 *   Worker 0 completed iteration 1
 *   Worker 1 completed iteration 1
 *   Worker 2 completed iteration 1
 *   Worker 3 completed iteration 1
 *   Worker 0 completed iteration 2
 *   ... (interleaved progress)
 *
 * Output pattern with SCHED_FIFO (sequential):
 *   Worker 0 completed iteration 1
 *   Worker 0 completed iteration 2
 *   ... Worker 0 all 10 ...
 *   Worker 1 completed iteration 1
 *   ... Worker 1 all 10 ...
 *   (etc.)
 */

SCHED_RR Time Quantum Details

•Default quantum: 100ms (configurable via /proc/sys/kernel/sched_rr_timeslice_ms)
•Independent of CFS: RR time slice is separate from CFS's sched_latency calculations
•Priority still dominates: Higher-priority tasks preempt regardless of quantum
•Quantum reset on wakeup: After blocking, a task's quantum restarts from full
•sched_rr_get_interval(): API to query the current time quantum

Choosing Between FIFO and RR

Use SCHED_FIFO when you have one task per priority level or need explicit control over ordering. Use SCHED_RR when multiple tasks at the same priority need to share CPU time fairly. In practice, careful priority assignment often means SCHED_FIFO is sufficient—RR is a convenience for grouping equivalent-importance RT tasks.

SCHED_DEADLINE: Deadline-Based Scheduling

SCHED_DEADLINE, introduced in Linux 3.14 (2014), implements the Earliest Deadline First (EDF) algorithm combined with the Constant Bandwidth Server (CBS) algorithm. This is the most sophisticated real-time policy in Linux.

Key Concepts:

Runtime (WCET): Maximum CPU time the task needs per period Deadline: Time by which the runtime must be consumed Period: How often the task repeats

The scheduler always runs the task with the earliest absolute deadline. This is dynamic—deadlines change as time passes and tasks complete their periods.

Why EDF Over Static Priority?

Optimal utilization: EDF can achieve 100% CPU utilization for schedulable task sets (vs ~69% theoretical limit for rate-monotonic)
No priority assignment needed: The scheduler determines urgency automatically from deadlines
Natural fit for periodic tasks: Matches how real-time work is often structured
Admission control: Kernel verifies schedulability before accepting new DEADLINE tasks

sched_deadline_example.c
C
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
/*
 * SCHED_DEADLINE Task Example
 *
 * Configure a task to receive guaranteed CPU bandwidth
 * with deadline-based scheduling.
 */
 
#define _GNU_SOURCE
#include <sched.h>
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <linux/sched.h>
#include <sys/syscall.h>
 
/*
 * SCHED_DEADLINE parameters:
 * 
 * This task needs 10ms of computation every 100ms
 * Deadline is also 100ms (must finish before next period)
 */
#define SCHED_DEADLINE_RUNTIME   (10 * 1000000)   /* 10ms in ns */
#define SCHED_DEADLINE_DEADLINE  (100 * 1000000)  /* 100ms in ns */
#define SCHED_DEADLINE_PERIOD    (100 * 1000000)  /* 100ms in ns */
 
/*
 * sched_attr structure for SCHED_DEADLINE
 */
struct sched_attr {
    uint32_t size;
    uint32_t sched_policy;
    uint64_t sched_flags;
    int32_t  sched_nice;
    uint32_t sched_priority;
    uint64_t sched_runtime;
    uint64_t sched_deadline;
    uint64_t sched_period;
};
 
static int sched_setattr(pid_t pid, const struct sched_attr *attr,
                         unsigned int flags) {
    return syscall(__NR_sched_setattr, pid, attr, flags);
}
 
int main() {
    struct sched_attr attr;
    int ret;
 
    /* Initialize sched_attr */
    memset(&attr, 0, sizeof(attr));
    attr.size = sizeof(attr);
    attr.sched_policy = SCHED_DEADLINE;
    attr.sched_runtime = SCHED_DEADLINE_RUNTIME;
    attr.sched_deadline = SCHED_DEADLINE_DEADLINE;
    attr.sched_period = SCHED_DEADLINE_PERIOD;
 
    /*
     * Set the scheduling parameters
     * Kernel will reject if the new task would make the system
     * unschedulable (admission control)
     */
    ret = sched_setattr(0, &attr, 0);
    if (ret < 0) {
        perror("sched_setattr failed");
        fprintf(stderr, "Possible causes:\n"
                "  - Not running as root\n"
                "  - Utilization would exceed system capacity\n"
                "  - Invalid parameters (runtime > deadline > period)\n");
        return 1;
    }
 
    printf("Running with SCHED_DEADLINE:\n"
           "  Runtime:  %llu ms\n"
           "  Deadline: %llu ms\n"
           "  Period:   %llu ms\n",
           (unsigned long long)attr.sched_runtime / 1000000,
           (unsigned long long)attr.sched_deadline / 1000000,
           (unsigned long long)attr.sched_period / 1000000);
 
    /*
     * Main work loop - periodic task pattern
     */
    while (1) {
        /* Do computation (should complete within runtime budget) */
        do_periodic_work();
 
        /*
         * sched_yield() has special meaning for SCHED_DEADLINE:
         * - Signals end of current period's computation
         * - Task blocks until next period begins
         * - Runtime budget resets for next period
         */
        sched_yield();
    }
 
    return 0;
}
 
/*
 * SCHED_DEADLINE admission control:
 *
 * Before accepting a new DEADLINE task, kernel checks:
 * 
 *   Σ (runtime_i / period_i) ≤ total_available_bandwidth
 *
 * Default total bandwidth: 95% of each CPU (5% reserved for non-RT)
 * Configurable via: /proc/sys/kernel/sched_rt_runtime_us
 *
 * This prevents oversubscription that would cause missed deadlines.
 */

Comparison of Real-Time Scheduling Policies
Aspect	SCHED_FIFO	SCHED_RR	SCHED_DEADLINE
Priority model	Static (1-99)	Static (1-99)	Dynamic (deadline-based)
Time slicing	None	Yes (100ms default)	Implicit by runtime budget
Admission control	None	None	Yes (utilization check)
Utilization limit	~69% optimal	~69% optimal	~100% optimal (EDF)
Configuration complexity	Low (assign priority)	Low (assign priority)	Medium (R, D, P parameters)
Class priority	Below DEADLINE	Below DEADLINE	Highest RT class
Best for	Known-priority tasks	Equal-importance tasks	Periodic, bounded workloads

SCHED_DEADLINE Priority

SCHED_DEADLINE tasks always preempt SCHED_FIFO and SCHED_RR tasks, regardless of their priorities. This is because deadline scheduling correctly identifies which task is most urgent. A DEADLINE task with an imminent deadline needs CPU immediately, regardless of any static priority assignments.

Configuration and Safety Mechanisms

Real-time scheduling can easily render a system unresponsive if misconfigured. Linux provides several safety mechanisms and configuration points.

RT Throttling: The Safety Net

By default, Linux reserves CPU time for non-real-time tasks. Even if a runaway RT task consumes its full budget, CFS tasks still get some CPU.

/proc/sys/kernel/sched_rt_period_us   = 1000000 (1 second)
/proc/sys/kernel/sched_rt_runtime_us  = 950000  (950ms)

This means: In any 1-second period, RT tasks can only use 950ms total. The remaining 50ms (5%) is guaranteed to non-RT tasks.

To disable throttling (dangerous in production):

echo -1 > /proc/sys/kernel/sched_rt_runtime_us

rt_configuration.sh
Bash
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
#!/bin/bash
# Real-Time Scheduling Configuration for Linux
 
# ========================================
# View current RT scheduling parameters
# ========================================
 
echo "=== RT Throttling Configuration ==="
echo "Period (us): $(cat /proc/sys/kernel/sched_rt_period_us)"
echo "Runtime (us): $(cat /proc/sys/kernel/sched_rt_runtime_us)"
echo "RT utilization: $(($(cat /proc/sys/kernel/sched_rt_runtime_us) * 100 / $(cat /proc/sys/kernel/sched_rt_period_us)))%"
 
echo ""
echo "=== SCHED_RR Time Quantum ==="
echo "RR timeslice (ms): $(cat /proc/sys/kernel/sched_rr_timeslice_ms)"
 
# ========================================
# Recommended production configuration
# ========================================
 
# Allow RT tasks to use 95% of CPU (default)
echo 950000 > /proc/sys/kernel/sched_rt_runtime_us
 
# Set RR time slice to 50ms (more responsive than default 100ms)
echo 50 > /proc/sys/kernel/sched_rr_timeslice_ms
 
# ========================================
# Per-user RT limits (/etc/security/limits.conf)
# ========================================
 
# Allow 'audio' group to use RT priorities up to 99
# @audio - rtprio 99
 
# Allow 'audio' group to lock memory
# @audio - memlock unlimited
 
# Allow specific user to use RT
# alice - rtprio 50
# alice - memlock 512000
 
# ========================================
# Capabilities for non-root RT scheduling
# ========================================
 
# Grant RT scheduling capability to a binary
# sudo setcap cap_sys_nice+ep /path/to/binary
 
# Check capabilities on a binary
# getcap /path/to/binary
 
# ========================================
# Monitoring RT task behavior
# ========================================
 
# View RT tasks with their priorities
ps -eo pid,class,rtprio,ni,comm --sort=-rtprio | head -20
 
# Real-time statistics from /proc/sched_debug
cat /proc/sched_debug | grep -A5 "cfs_rq\|rt_rq"
 
# Trace scheduling events (requires root, uses ftrace)
echo 1 > /sys/kernel/debug/tracing/events/sched/sched_switch/enable
cat /sys/kernel/debug/tracing/trace_pipe
 
# ========================================
# Check for priority inversion issues
# ========================================
 
# Show tasks waiting on mutexes held by RT tasks
# (This would require specific debugging tools like lockdep)

Real-Time Safety Best Practices

•Always keep RT throttling enabled: The 5% CFS guarantee can save you from locked-out systems.
•Lock memory with mlockall(): Prevent page faults during RT execution. Page faults cause unpredictable latency.
•Use capabilities instead of root: Grant CAP_SYS_NICE and CAP_IPC_LOCK to specific binaries rather than running as root.
•Test thoroughly before production: RT bugs cause system hangs. Test in VMs or on dedicated hardware.
•Implement watchdogs: High-priority threads should monitor lower-priority RT tasks for hangs.
•Profile worst-case behavior: Stress test under load to find actual WCET, not just average case.
•Document priority assignments: Priority inversion bugs are easier to diagnose with clear priority documentation.

PREEMPT_RT for Lower Latency

For applications requiring sub-millisecond latency guarantees, consider the PREEMPT_RT patchset (being mainlined into Linux). It makes more kernel code preemptible, converts spinlocks to mutexes, and provides significantly lower interrupt-to-process latency—often under 100μs vs 500μs+ on standard kernels.

Real-World Real-Time Applications

Real-time scheduling in Linux enables diverse applications that require predictable timing. Here are common use cases and their scheduling configurations.

Professional Audio (JACK/PipeWire)

Digital audio workstations need to process audio samples without glitches:

Priority: SCHED_FIFO, priority 80-95
Configuration: Memory locked, CPU affinity often set
Latency target: < 5ms end-to-end
Tools: JACK uses RT threads; PipeWire has adaptive RT scheduling

Industrial Automation (EtherCAT, CAN)

Real-time Ethernet and CAN bus protocols need precise packet timing:

Priority: SCHED_FIFO, priority 90+ or SCHED_DEADLINE
Configuration: Dedicated core (isolcpus), interrupt affinity
Latency target: < 1ms cycle time
Tools: LinuxCNC, Machinekit, EtherLab

Appropriate RT Use Cases

•Audio/video capture and processing
•Industrial control loops
•High-frequency trading (latency)
•Software-defined radio
•Robotics control systems
•Network packet processing
•Some database commit paths

Inappropriate RT Use Cases

•Web servers (use CFS nice)
•Batch processing jobs
•General desktop applications
•Background system services
•Database query processing
•File serving (NFS, SMB)
•'More important' but not latency-critical

audio_rt_setup.sh
Bash
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
#!/bin/bash
# Professional Audio Real-Time Setup
# Configures a Linux system for low-latency audio production
 
# 1. Add user to audio group for RT privileges
sudo usermod -aG audio $USER
 
# 2. Configure limits for audio group (/etc/security/limits.d/audio.conf)
cat << 'EOF' | sudo tee /etc/security/limits.d/audio.conf
@audio   -  rtprio     95
@audio   -  memlock    unlimited
@audio   -  nice       -19
EOF
 
# 3. Configure CPU governor for consistent timing
echo performance | sudo tee /sys/devices/system/cpu/cpu*/cpufreq/scaling_governor
 
# 4. Disable CPU frequency scaling (optional, for minimal jitter)
for cpu in /sys/devices/system/cpu/cpu*/cpufreq/scaling_min_freq; do
    cat "${cpu / min / max}" | sudo tee "$cpu"
done
 
# 5. Isolate a CPU core for audio (kernel boot parameter)
# Add to GRUB_CMDLINE_LINUX in /etc/default/grub:
#   isolcpus=3 nohz_full=3 rcu_nocbs=3
 
# 6. Set RT scheduling for the audio server
# JACK automatically uses RT when properly configured
# PipeWire: copy /usr/share/pipewire/pipewire.conf to ~/.config/pipewire/
#           modify to enable RT with nice/rt settings
 
# 7. Verify configuration
echo ""
echo "Current user groups:"
groups
 
echo ""
echo "RT limits for current user:"
ulimit -r   # Max RT priority
 
# After relogin and starting JACK:
# jack_lsp -l  # Check JACK latency
# Should show latency in frames, e.g., 256 frames @ 48kHz = 5.3ms

The Linux RT Advantage

Linux's real-time capabilities enable running soft real-time workloads on commodity hardware without dedicated RTOSes. Combined with low costs and rich ecosystems, this makes Linux the dominant platform for audio production, broadcast, industrial automation, and telecommunications—applications that once required specialized (and expensive) operating systems.

Summary: Real-Time Scheduling in Linux

Linux's real-time scheduling policies provide essential capabilities for latency-sensitive applications, complementing CFS's fairness-oriented approach with strict priority-based scheduling.

Key Takeaways

•Real-time tasks preempt normal tasks — SCHED_FIFO and SCHED_RR tasks at any priority (1-99) run before any CFS task.
•SCHED_FIFO provides non-preemptive RT — Runs until yield or block; simplest RT policy for single-task-per-priority workloads.
•SCHED_RR adds time-slicing within priorities — Multiple RT tasks at the same priority share CPU time fairly.
•SCHED_DEADLINE implements EDF — Most advanced policy; specifies runtime, deadline, and period for optimal utilization.
•RT throttling provides safety — Default 5% reservation prevents RT tasks from completely starving the system.
•Proper configuration requires privileges — CAP_SYS_NICE, limits.conf, or root access needed for RT scheduling.

What's Next

The final page explores nice values in depth—the user-facing mechanism for influencing CFS scheduling without real-time privileges. We'll see how nice values map to weights, their practical impact on CPU allocation, and guidelines for effective use.

Page Complete

You now understand Linux's real-time scheduling policies—SCHED_FIFO for strict priority ordering, SCHED_RR for time-sliced RT scheduling, and SCHED_DEADLINE for deadline-based optimal scheduling. This knowledge enables building latency-sensitive applications while understanding their system-wide implications.

4 / 5

Loading learning content...

Operating SystemsLinux Scheduling

Linux Scheduling

LevelAdvanced

Duration90 mins

TopicLinux Scheduling

4 / 5

Real-Time Policies

When Fairness Is Not Enough

CFS excels at providing proportionally fair CPU access across competing workloads. But some applications have requirements that fairness cannot satisfy:

Audio processing: A buffer underrun causes audible glitches—latency must be bounded, not averaged
Industrial control: A missed deadline can cause physical damage or safety hazards
High-frequency trading: Microseconds of latency directly translate to financial loss
Telecommunications: Packet processing must meet strict timing to maintain connection quality

Learning Objectives

Real-Time Scheduling Concepts

Hard vs. Soft Real-Time

Hard real-time systems have absolute deadlines. Missing a deadline constitutes system failure, potentially with catastrophic consequences:

Aircraft flight controls
Nuclear reactor monitoring
Automotive airbag deployment
Pacemaker timing

Soft real-time systems have deadlines where occasional misses are tolerable, typically degrading quality rather than causing failure:

Video playback (dropped frames cause visual glitches)
Audio processing (buffer underruns cause pops)
Interactive gaming (frame timing affects smoothness)
Voice over IP (late packets cause audio artifacts)

Linux Is Not a Hard Real-Time OS

Priority-Based vs. Deadline-Based Scheduling

Linux offers two paradigms for real-time scheduling:

SCHED_FIFO: Run until block or yield
SCHED_RR: Run with time-slicing among equal priorities

Dynamic Priority (Deadline-Based): Tasks specify their timing requirements (period, deadline, execution time). The scheduler dynamically orders tasks by deadline urgency.

SCHED_DEADLINE: Earliest Deadline First (EDF) scheduling

The Scheduling Class Hierarchy

Linux organizes schedulers into classes with strict priority:

STOP class (highest)  → kernel threads (migration, watchdog)
    ↓
DL class              → SCHED_DEADLINE tasks
    ↓
RT class              → SCHED_FIFO, SCHED_RR tasks (priorities 1-99)
    ↓
FAIR class            → SCHED_NORMAL (CFS) tasks
    ↓
IDLE class (lowest)   → SCHED_IDLE tasks

A higher class always preempts lower classes. Within each class, the class-specific algorithm determines scheduling.

Key Real-Time Terminology

•Period (T): Time between successive releases of a periodic task. Audio at 48kHz has T = 20.83μs periods.
•Deadline (D): Time by which task must complete after being released. Often D = T for periodic tasks.
•Worst-Case Execution Time (WCET): Maximum time the task might need. Must be known for schedulability.
•Utilization (U): WCET / Period. Total utilization bounds schedulability.
•Jitter: Variation in timing. Low jitter indicates consistent, predictable behavior.
•Latency: Delay from event (interrupt, wake) to response. Critical for responsiveness.

SCHED_FIFO: First-In-First-Out Real-Time

SCHED_FIFO is the simplest real-time policy. A SCHED_FIFO task runs until:

It voluntarily yields (sched_yield)
It blocks (I/O, sleep, mutex)
A higher-priority real-time task becomes runnable

Key Characteristics:

No time slicing: Unlike SCHED_RR, SCHED_FIFO tasks don't have time quantums
Static priority: Priority (1-99) is fixed at creation, doesn't change dynamically
FIFO ordering within priority: If multiple tasks share the same priority, they're queued in arrival order
Immediate preemption: Preempts CFS tasks instantly, preempts lower-priority RT tasks instantly

When to Use SCHED_FIFO:

Tasks with known, bounded execution times
Non-interactive, compute-bound real-time work
When you need deterministic ordering among equal-priority tasks
Latency-critical interrupt handling threads

sched_fifo_example.c
C
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
/*
 * SCHED_FIFO Real-Time Task Example
 *
 * This example creates a high-priority real-time task
 * that processes audio buffers with minimal latency.
 */
 
#include <pthread.h>
#include <sched.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <unistd.h>
#include <sys/mman.h>
 
#define AUDIO_PRIORITY 80  /* High RT priority (1-99 scale) */
 
void *audio_processing_thread(void *arg) {
    printf("Audio thread running with SCHED_FIFO priority %d\n", 
           AUDIO_PRIORITY);
    
    /* Simulate periodic audio processing */
    while (1) {
        /* 
         * In real code: 
         * 1. Wait for audio buffer ready (blocking)
         * 2. Process audio samples
         * 3. Output processed samples
         */
        process_audio_buffer();
        
        /* 
         * Key behavior: This thread will NOT be preempted by 
         * ANY CFS (normal) task, regardless of their nice level.
         * Only higher-priority RT tasks can preempt.
         */
    }
    
    return NULL;
}
 
int main() {
    pthread_t audio_thread;
    pthread_attr_t attr;
    struct sched_param param;
    int ret;
    
    /*
     * Lock memory to prevent page faults during RT operation.
     * Page faults cause unpredictable latency spikes.
     */
    if (mlockall(MCL_CURRENT | MCL_FUTURE) == -1) {
        perror("mlockall failed (need CAP_IPC_LOCK)");
    }
    
    /* Initialize thread attributes */
    pthread_attr_init(&attr);
    
    /* Set SCHED_FIFO policy */
    pthread_attr_setschedpolicy(&attr, SCHED_FIFO);
    
    /* Set priority (1-99, higher is higher priority) */
    param.sched_priority = AUDIO_PRIORITY;
    pthread_attr_setschedparam(&attr, &param);
    
    /* Ensure policy/priority are used (vs inheriting from parent) */
    pthread_attr_setinheritsched(&attr, PTHREAD_EXPLICIT_SCHED);
    
    /* Create the real-time thread */
    ret = pthread_create(&audio_thread, &attr, 
                         audio_processing_thread, NULL);
    if (ret != 0) {
        fprintf(stderr, "pthread_create failed: %s\n", strerror(ret));
        fprintf(stderr, "(Need CAP_SYS_NICE or root for RT scheduling)\n");
        return 1;
    }
    
    /* Main thread continues with normal scheduling... */
    pthread_join(audio_thread, NULL);
    
    return 0;
}
 
/*
 * Running this requires privileges:
 * 
 * Option 1: Run as root
 *   sudo ./audio_rt
 *
 * Option 2: Set capabilities
 *   sudo setcap cap_sys_nice,cap_ipc_lock+ep ./audio_rt
 *
 * Option 3: Configure rtprio limit in /etc/security/limits.conf
 *   @audio - rtprio 99
 */

SCHED_FIFO Behavior Nuances

Preemption Rules:

FIFO task A (priority 50) is running
FIFO task B (priority 60) becomes runnable
A is immediately preempted, B runs
When B blocks, A resumes

Same-Priority Ordering:

FIFO task C (priority 50) is running
FIFO task D (priority 50) becomes runnable
C continues until it blocks or yields
D runs next (FIFO order)
When D blocks, C runs if runnable

Yield Behavior:

sched_yield() moves the calling task to the end of its priority queue
Does NOT allow lower-priority tasks to run
Used for cooperative multitasking among equal-priority RT tasks

The Runaway SCHED_FIFO Danger

SCHED_RR: Round-Robin Real-Time

Key Differences from SCHED_FIFO:

Aspect	SCHED_FIFO	SCHED_RR
Time slice	None	Yes (default 100ms)
Equal-priority behavior	Run until yield/block	Round-robin
Preemption by higher	Immediate	Immediate
Use case	Single RT task per priority	Multiple RT tasks at same priority

When to Use SCHED_RR:

Multiple real-time tasks that need to share a priority level
When you want fairness within a real-time priority
Interactive real-time applications where responsiveness among RT tasks matters
Defensive scheduling to prevent any single RT task from monopolizing

sched_rr_example.c
C
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
/*
 * SCHED_RR Real-Time Task Example
 *
 * Creates multiple worker threads that share CPU time
 * fairly within the real-time class.
 */
 
#include <pthread.h>
#include <sched.h>
#include <stdio.h>
#include <unistd.h>
 
#define RT_PRIORITY 50
#define NUM_WORKERS 4
 
void *worker_thread(void *arg) {
    int id = *(int *)arg;
    int iterations = 0;
    
    printf("Worker %d started with SCHED_RR priority %d\n", 
           id, RT_PRIORITY);
    
    while (iterations < 10) {
        /* Simulate computational work */
        volatile long i;
        for (i = 0; i < 100000000; i++) {
            /* Busy work */
        }
        
        printf("Worker %d completed iteration %d\n", id, ++iterations);
        
        /*
         * With SCHED_RR:
         * - Each worker runs for the time quantum (default 100ms)
         * - Then is preempted for the next equal-priority worker
         * - All workers make progress concurrently
         *
         * With SCHED_FIFO (same priority):
         * - First worker would run all 10 iterations
         * - Then second worker, etc.
         * - No interleaving without explicit yields
         */
    }
    
    return NULL;
}
 
int main() {
    pthread_t threads[NUM_WORKERS];
    pthread_attr_t attr;
    struct sched_param param;
    int worker_ids[NUM_WORKERS];
    
    pthread_attr_init(&attr);
    pthread_attr_setschedpolicy(&attr, SCHED_RR);
    param.sched_priority = RT_PRIORITY;
    pthread_attr_setschedparam(&attr, &param);
    pthread_attr_setinheritsched(&attr, PTHREAD_EXPLICIT_SCHED);
    
    /* Create multiple RR workers at the same priority */
    for (int i = 0; i < NUM_WORKERS; i++) {
        worker_ids[i] = i;
        pthread_create(&threads[i], &attr, worker_thread, &worker_ids[i]);
    }
    
    /*
     * Query the RR time quantum for informational purposes
     */
    struct timespec ts;
    if (sched_rr_get_interval(0, &ts) == 0) {
        printf("RR time quantum: %ld.%09ld seconds\n", 
               ts.tv_sec, ts.tv_nsec);
    }
    
    for (int i = 0; i < NUM_WORKERS; i++) {
        pthread_join(threads[i], NULL);
    }
    
    return 0;
}
 
/*
 * Output pattern with SCHED_RR (interleaved):
 *   Worker 0 completed iteration 1
 *   Worker 1 completed iteration 1
 *   Worker 2 completed iteration 1
 *   Worker 3 completed iteration 1
 *   Worker 0 completed iteration 2
 *   ... (interleaved progress)
 *
 * Output pattern with SCHED_FIFO (sequential):
 *   Worker 0 completed iteration 1
 *   Worker 0 completed iteration 2
 *   ... Worker 0 all 10 ...
 *   Worker 1 completed iteration 1
 *   ... Worker 1 all 10 ...
 *   (etc.)
 */

SCHED_RR Time Quantum Details

•Default quantum: 100ms (configurable via /proc/sys/kernel/sched_rr_timeslice_ms)
•Independent of CFS: RR time slice is separate from CFS's sched_latency calculations
•Priority still dominates: Higher-priority tasks preempt regardless of quantum
•Quantum reset on wakeup: After blocking, a task's quantum restarts from full
•sched_rr_get_interval(): API to query the current time quantum

Choosing Between FIFO and RR

SCHED_DEADLINE: Deadline-Based Scheduling

Key Concepts:

Runtime (WCET): Maximum CPU time the task needs per period Deadline: Time by which the runtime must be consumed Period: How often the task repeats

The scheduler always runs the task with the earliest absolute deadline. This is dynamic—deadlines change as time passes and tasks complete their periods.

Why EDF Over Static Priority?

Optimal utilization: EDF can achieve 100% CPU utilization for schedulable task sets (vs ~69% theoretical limit for rate-monotonic)
No priority assignment needed: The scheduler determines urgency automatically from deadlines
Natural fit for periodic tasks: Matches how real-time work is often structured
Admission control: Kernel verifies schedulability before accepting new DEADLINE tasks

sched_deadline_example.c
C
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
/*
 * SCHED_DEADLINE Task Example
 *
 * Configure a task to receive guaranteed CPU bandwidth
 * with deadline-based scheduling.
 */
 
#define _GNU_SOURCE
#include <sched.h>
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <linux/sched.h>
#include <sys/syscall.h>
 
/*
 * SCHED_DEADLINE parameters:
 * 
 * This task needs 10ms of computation every 100ms
 * Deadline is also 100ms (must finish before next period)
 */
#define SCHED_DEADLINE_RUNTIME   (10 * 1000000)   /* 10ms in ns */
#define SCHED_DEADLINE_DEADLINE  (100 * 1000000)  /* 100ms in ns */
#define SCHED_DEADLINE_PERIOD    (100 * 1000000)  /* 100ms in ns */
 
/*
 * sched_attr structure for SCHED_DEADLINE
 */
struct sched_attr {
    uint32_t size;
    uint32_t sched_policy;
    uint64_t sched_flags;
    int32_t  sched_nice;
    uint32_t sched_priority;
    uint64_t sched_runtime;
    uint64_t sched_deadline;
    uint64_t sched_period;
};
 
static int sched_setattr(pid_t pid, const struct sched_attr *attr,
                         unsigned int flags) {
    return syscall(__NR_sched_setattr, pid, attr, flags);
}
 
int main() {
    struct sched_attr attr;
    int ret;
 
    /* Initialize sched_attr */
    memset(&attr, 0, sizeof(attr));
    attr.size = sizeof(attr);
    attr.sched_policy = SCHED_DEADLINE;
    attr.sched_runtime = SCHED_DEADLINE_RUNTIME;
    attr.sched_deadline = SCHED_DEADLINE_DEADLINE;
    attr.sched_period = SCHED_DEADLINE_PERIOD;
 
    /*
     * Set the scheduling parameters
     * Kernel will reject if the new task would make the system
     * unschedulable (admission control)
     */
    ret = sched_setattr(0, &attr, 0);
    if (ret < 0) {
        perror("sched_setattr failed");
        fprintf(stderr, "Possible causes:\n"
                "  - Not running as root\n"
                "  - Utilization would exceed system capacity\n"
                "  - Invalid parameters (runtime > deadline > period)\n");
        return 1;
    }
 
    printf("Running with SCHED_DEADLINE:\n"
           "  Runtime:  %llu ms\n"
           "  Deadline: %llu ms\n"
           "  Period:   %llu ms\n",
           (unsigned long long)attr.sched_runtime / 1000000,
           (unsigned long long)attr.sched_deadline / 1000000,
           (unsigned long long)attr.sched_period / 1000000);
 
    /*
     * Main work loop - periodic task pattern
     */
    while (1) {
        /* Do computation (should complete within runtime budget) */
        do_periodic_work();
 
        /*
         * sched_yield() has special meaning for SCHED_DEADLINE:
         * - Signals end of current period's computation
         * - Task blocks until next period begins
         * - Runtime budget resets for next period
         */
        sched_yield();
    }
 
    return 0;
}
 
/*
 * SCHED_DEADLINE admission control:
 *
 * Before accepting a new DEADLINE task, kernel checks:
 * 
 *   Σ (runtime_i / period_i) ≤ total_available_bandwidth
 *
 * Default total bandwidth: 95% of each CPU (5% reserved for non-RT)
 * Configurable via: /proc/sys/kernel/sched_rt_runtime_us
 *
 * This prevents oversubscription that would cause missed deadlines.
 */

Comparison of Real-Time Scheduling Policies
Aspect	SCHED_FIFO	SCHED_RR	SCHED_DEADLINE
Priority model	Static (1-99)	Static (1-99)	Dynamic (deadline-based)
Time slicing	None	Yes (100ms default)	Implicit by runtime budget
Admission control	None	None	Yes (utilization check)
Utilization limit	~69% optimal	~69% optimal	~100% optimal (EDF)
Configuration complexity	Low (assign priority)	Low (assign priority)	Medium (R, D, P parameters)
Class priority	Below DEADLINE	Below DEADLINE	Highest RT class
Best for	Known-priority tasks	Equal-importance tasks	Periodic, bounded workloads

SCHED_DEADLINE Priority

Configuration and Safety Mechanisms

Real-time scheduling can easily render a system unresponsive if misconfigured. Linux provides several safety mechanisms and configuration points.

RT Throttling: The Safety Net

By default, Linux reserves CPU time for non-real-time tasks. Even if a runaway RT task consumes its full budget, CFS tasks still get some CPU.

/proc/sys/kernel/sched_rt_period_us   = 1000000 (1 second)
/proc/sys/kernel/sched_rt_runtime_us  = 950000  (950ms)

This means: In any 1-second period, RT tasks can only use 950ms total. The remaining 50ms (5%) is guaranteed to non-RT tasks.

To disable throttling (dangerous in production):

echo -1 > /proc/sys/kernel/sched_rt_runtime_us

rt_configuration.sh
Bash
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
#!/bin/bash
# Real-Time Scheduling Configuration for Linux
 
# ========================================
# View current RT scheduling parameters
# ========================================
 
echo "=== RT Throttling Configuration ==="
echo "Period (us): $(cat /proc/sys/kernel/sched_rt_period_us)"
echo "Runtime (us): $(cat /proc/sys/kernel/sched_rt_runtime_us)"
echo "RT utilization: $(($(cat /proc/sys/kernel/sched_rt_runtime_us) * 100 / $(cat /proc/sys/kernel/sched_rt_period_us)))%"
 
echo ""
echo "=== SCHED_RR Time Quantum ==="
echo "RR timeslice (ms): $(cat /proc/sys/kernel/sched_rr_timeslice_ms)"
 
# ========================================
# Recommended production configuration
# ========================================
 
# Allow RT tasks to use 95% of CPU (default)
echo 950000 > /proc/sys/kernel/sched_rt_runtime_us
 
# Set RR time slice to 50ms (more responsive than default 100ms)
echo 50 > /proc/sys/kernel/sched_rr_timeslice_ms
 
# ========================================
# Per-user RT limits (/etc/security/limits.conf)
# ========================================
 
# Allow 'audio' group to use RT priorities up to 99
# @audio - rtprio 99
 
# Allow 'audio' group to lock memory
# @audio - memlock unlimited
 
# Allow specific user to use RT
# alice - rtprio 50
# alice - memlock 512000
 
# ========================================
# Capabilities for non-root RT scheduling
# ========================================
 
# Grant RT scheduling capability to a binary
# sudo setcap cap_sys_nice+ep /path/to/binary
 
# Check capabilities on a binary
# getcap /path/to/binary
 
# ========================================
# Monitoring RT task behavior
# ========================================
 
# View RT tasks with their priorities
ps -eo pid,class,rtprio,ni,comm --sort=-rtprio | head -20
 
# Real-time statistics from /proc/sched_debug
cat /proc/sched_debug | grep -A5 "cfs_rq\|rt_rq"
 
# Trace scheduling events (requires root, uses ftrace)
echo 1 > /sys/kernel/debug/tracing/events/sched/sched_switch/enable
cat /sys/kernel/debug/tracing/trace_pipe
 
# ========================================
# Check for priority inversion issues
# ========================================
 
# Show tasks waiting on mutexes held by RT tasks
# (This would require specific debugging tools like lockdep)

Real-Time Safety Best Practices

•Always keep RT throttling enabled: The 5% CFS guarantee can save you from locked-out systems.
•Lock memory with mlockall(): Prevent page faults during RT execution. Page faults cause unpredictable latency.
•Use capabilities instead of root: Grant CAP_SYS_NICE and CAP_IPC_LOCK to specific binaries rather than running as root.
•Test thoroughly before production: RT bugs cause system hangs. Test in VMs or on dedicated hardware.
•Implement watchdogs: High-priority threads should monitor lower-priority RT tasks for hangs.
•Profile worst-case behavior: Stress test under load to find actual WCET, not just average case.
•Document priority assignments: Priority inversion bugs are easier to diagnose with clear priority documentation.

PREEMPT_RT for Lower Latency

Real-World Real-Time Applications

Real-time scheduling in Linux enables diverse applications that require predictable timing. Here are common use cases and their scheduling configurations.

Professional Audio (JACK/PipeWire)

Digital audio workstations need to process audio samples without glitches:

Priority: SCHED_FIFO, priority 80-95
Configuration: Memory locked, CPU affinity often set
Latency target: < 5ms end-to-end
Tools: JACK uses RT threads; PipeWire has adaptive RT scheduling

Industrial Automation (EtherCAT, CAN)

Real-time Ethernet and CAN bus protocols need precise packet timing:

Priority: SCHED_FIFO, priority 90+ or SCHED_DEADLINE
Configuration: Dedicated core (isolcpus), interrupt affinity
Latency target: < 1ms cycle time
Tools: LinuxCNC, Machinekit, EtherLab

Appropriate RT Use Cases

•Audio/video capture and processing
•Industrial control loops
•High-frequency trading (latency)
•Software-defined radio
•Robotics control systems
•Network packet processing
•Some database commit paths

Inappropriate RT Use Cases

•Web servers (use CFS nice)
•Batch processing jobs
•General desktop applications
•Background system services
•Database query processing
•File serving (NFS, SMB)
•'More important' but not latency-critical

audio_rt_setup.sh
Bash
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
#!/bin/bash
# Professional Audio Real-Time Setup
# Configures a Linux system for low-latency audio production
 
# 1. Add user to audio group for RT privileges
sudo usermod -aG audio $USER
 
# 2. Configure limits for audio group (/etc/security/limits.d/audio.conf)
cat << 'EOF' | sudo tee /etc/security/limits.d/audio.conf
@audio   -  rtprio     95
@audio   -  memlock    unlimited
@audio   -  nice       -19
EOF
 
# 3. Configure CPU governor for consistent timing
echo performance | sudo tee /sys/devices/system/cpu/cpu*/cpufreq/scaling_governor
 
# 4. Disable CPU frequency scaling (optional, for minimal jitter)
for cpu in /sys/devices/system/cpu/cpu*/cpufreq/scaling_min_freq; do
    cat "${cpu / min / max}" | sudo tee "$cpu"
done
 
# 5. Isolate a CPU core for audio (kernel boot parameter)
# Add to GRUB_CMDLINE_LINUX in /etc/default/grub:
#   isolcpus=3 nohz_full=3 rcu_nocbs=3
 
# 6. Set RT scheduling for the audio server
# JACK automatically uses RT when properly configured
# PipeWire: copy /usr/share/pipewire/pipewire.conf to ~/.config/pipewire/
#           modify to enable RT with nice/rt settings
 
# 7. Verify configuration
echo ""
echo "Current user groups:"
groups
 
echo ""
echo "RT limits for current user:"
ulimit -r   # Max RT priority
 
# After relogin and starting JACK:
# jack_lsp -l  # Check JACK latency
# Should show latency in frames, e.g., 256 frames @ 48kHz = 5.3ms

The Linux RT Advantage

Summary: Real-Time Scheduling in Linux

Linux's real-time scheduling policies provide essential capabilities for latency-sensitive applications, complementing CFS's fairness-oriented approach with strict priority-based scheduling.

Key Takeaways

•Real-time tasks preempt normal tasks — SCHED_FIFO and SCHED_RR tasks at any priority (1-99) run before any CFS task.
•SCHED_FIFO provides non-preemptive RT — Runs until yield or block; simplest RT policy for single-task-per-priority workloads.
•SCHED_RR adds time-slicing within priorities — Multiple RT tasks at the same priority share CPU time fairly.
•SCHED_DEADLINE implements EDF — Most advanced policy; specifies runtime, deadline, and period for optimal utilization.
•RT throttling provides safety — Default 5% reservation prevents RT tasks from completely starving the system.
•Proper configuration requires privileges — CAP_SYS_NICE, limits.conf, or root access needed for RT scheduling.

What's Next

Page Complete

4 / 5