Loading learning content...
Signals and threads are both mechanisms for handling concurrent events—and combining them creates one of the most challenging areas in systems programming. Signals are inherently process-wide, designed long before threads existed. Threads are execution contexts within a process, each with their own stack and execution flow. When a signal arrives at a threaded process, fundamental questions arise:
The answers to these questions are subtle and, historically, have been a source of countless bugs in production systems. This page provides the deep understanding necessary to navigate this complexity correctly.
By the end of this page, you will understand: (1) the fundamental signal delivery model in threaded processes, (2) per-thread signal masks and their role, (3) synchronous vs. asynchronous signal handling strategies, (4) the pthread_sigmask() and sigwait() functions, (5) the dedicated signal-handling thread pattern, and (6) critical pitfalls and how to avoid them.
Before diving into the complexities, let's establish the foundational model for how signals and threads interact in POSIX systems.
Process-Wide vs. Thread-Specific Properties:
In a POSIX system, some signal-related properties belong to the entire process, while others are per-thread:
Process-Wide (Shared by All Threads):
sigaction())Per-Thread:
| Property | Scope | Implications |
|---|---|---|
| Signal Handler | Process-wide | All threads share the same handler function for a given signal |
| Signal Disposition | Process-wide | Setting SIG_IGN affects all threads immediately |
| Signal Mask | Per-thread | Each thread independently controls which signals it blocks |
| Pending Process Signals | Process-wide | Process-directed signals wait for any eligible thread |
| Pending Thread Signals | Per-thread | Thread-directed signals are delivered only to that specific thread |
| Alternate Signal Stack | Per-thread | Each thread can define its own signal stack via sigaltstack() |
Because signal handlers are process-wide, if one thread installs a handler for SIGTERM, ALL threads in the process will use that handler. This means signal handlers must be thread-safe and cannot assume they run in any particular thread's context. Handler installation should typically happen once, early in the program's startup, before creating additional threads.
Signal Delivery in Threaded Processes:
When a signal is generated for a threaded process, the kernel must decide which thread receives it. The rules depend on the signal's source:
Thread-Directed Signals:
Some signals are directed at a specific thread (via pthread_kill() or tgkill()):
Process-Directed Signals:
General signals directed at the process (via kill() to the process ID):
Each thread has its own signal mask—a set of signals that are blocked from delivery to that thread. Blocked signals remain pending until unblocked; they aren't lost, just deferred.
The pthread_sigmask() Function:
In threaded programs, you must use pthread_sigmask() rather than sigprocmask(). While sigprocmask() might work on some implementations, POSIX specifies that its behavior is undefined in multithreaded programs.
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778
#include <pthread.h>#include <signal.h>#include <stdio.h>#include <stdlib.h>#include <unistd.h> // pthread_sigmask() - modify the calling thread's signal mask// // int pthread_sigmask(int how, const sigset_t *set, sigset_t *oldset);//// 'how' determines the operation:// SIG_BLOCK - Add signals in 'set' to current mask// SIG_UNBLOCK - Remove signals in 'set' from current mask// SIG_SETMASK - Replace mask entirely with 'set' void *worker_thread(void *arg) { sigset_t mask, oldmask; // Get the current thread's signal mask pthread_sigmask(SIG_SETMASK, NULL, &oldmask); // Block SIGUSR1 and SIGUSR2 for this thread sigemptyset(&mask); sigaddset(&mask, SIGUSR1); sigaddset(&mask, SIGUSR2); pthread_sigmask(SIG_BLOCK, &mask, NULL); printf("Thread %lu: SIGUSR1 and SIGUSR2 are now blocked\n", (unsigned long)pthread_self()); // Do work... // SIGUSR1/SIGUSR2 will be pending if received, but not delivered for (int i = 0; i < 10; i++) { printf("Thread working: %d\n", i); sleep(1); // Signals would normally interrupt sleep // But blocked signals don't interrupt } // Unblock signals pthread_sigmask(SIG_UNBLOCK, &mask, NULL); // If SIGUSR1 or SIGUSR2 were pending, they're delivered NOW printf("Thread %lu: Signals unblocked, any pending signals delivered\n", (unsigned long)pthread_self()); return NULL;} // Pattern: Block signals before thread creation// so new threads inherit the blocked stateint main() { sigset_t mask; pthread_t threads[5]; // Block signals in main thread BEFORE creating worker threads // All child threads inherit this signal mask sigemptyset(&mask); sigaddset(&mask, SIGINT); sigaddset(&mask, SIGTERM); sigaddset(&mask, SIGUSR1); pthread_sigmask(SIG_BLOCK, &mask, NULL); // Create threads - each inherits the blocked signals for (int i = 0; i < 5; i++) { pthread_create(&threads[i], NULL, worker_thread, NULL); } // Main thread can unblock to handle signals, // or a dedicated signal thread can be created (see later) for (int i = 0; i < 5; i++) { pthread_join(threads[i], NULL); } return 0;}When a new thread is created with pthread_create(), it inherits the signal mask of its creating thread. This makes the recommended pattern clear: block all signals you want to handle specially in the main thread BEFORE creating any other threads. Then, create a dedicated signal-handling thread that unblocks and processes these signals.
Signal Sets:
Signal sets (sigset_t) are manipulated using these functions:
sigemptyset(&set); // Initialize to empty (no signals)
sigfillset(&set); // Initialize to full (all signals)
sigaddset(&set, sig); // Add a signal to the set
sigdelset(&set, sig); // Remove a signal from the set
sigismember(&set, sig); // Test if signal is in the set
Always initialize signal sets before use—uninitialized sets contain undefined contents.
Understanding exactly when and how signals are delivered is crucial for writing correct threaded code. The delivery model has several important properties:
Delivery to Arbitrary Eligible Thread:
For process-directed signals, if multiple threads have the signal unblocked, the kernel arbitrarily chooses one. You cannot predict which thread will receive it. This non-determinism is a fundamental property—do not design systems that depend on a specific thread receiving process-directed signals.
Thread-Directed Signals:
Signals can be sent directly to a specific thread using pthread_kill():
12345678910111213141516171819202122232425262728293031323334353637383940414243444546474849505152535455565758596061
#include <pthread.h>#include <signal.h>#include <stdio.h>#include <unistd.h> // Thread-specific signal handlervolatile sig_atomic_t received_signal = 0; void handler(int sig) { received_signal = sig; // Note: printf is not async-signal-safe, but used here for demonstration write(STDOUT_FILENO, "Signal received in thread!\n", 28);} void *target_thread(void *arg) { // Set up handler struct sigaction sa; sa.sa_handler = handler; sigemptyset(&sa.sa_mask); sa.sa_flags = 0; sigaction(SIGUSR1, &sa, NULL); // Make sure SIGUSR1 is unblocked in this thread sigset_t mask; sigemptyset(&mask); sigaddset(&mask, SIGUSR1); pthread_sigmask(SIG_UNBLOCK, &mask, NULL); printf("Target thread running, PID=%d, TID=%lu\n", getpid(), (unsigned long)pthread_self()); while (!received_signal) { pause(); // Wait for signal } printf("Target thread exiting after signal\n"); return NULL;} int main() { pthread_t tid; // Block SIGUSR1 in main thread sigset_t mask; sigemptyset(&mask); sigaddset(&mask, SIGUSR1); pthread_sigmask(SIG_BLOCK, &mask, NULL); pthread_create(&tid, NULL, target_thread, NULL); sleep(1); // Let target thread set up // Send signal to specific thread printf("Main: Sending SIGUSR1 to thread %lu\n", (unsigned long)tid); pthread_kill(tid, SIGUSR1); pthread_join(tid, NULL); printf("Main: Thread joined\n"); return 0;}pthread_kill(thread, sig) sends a signal to a specific thread. kill(pid, sig) sends a signal to a process (which the kernel delivers to an arbitrary eligible thread). Always use pthread_kill() when you need to signal a specific thread, and be aware that the target thread must have the signal unblocked to receive it.
The canonical solution to signal handling complexity in threaded programs is the dedicated signal-handling thread. This pattern centralizes all signal processing in a single thread, eliminating the uncertainty of asynchronous signal delivery.
The Pattern:
sigwait() to synchronously wait for themsigwait() returns with the signal number123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132
#include <pthread.h>#include <signal.h>#include <stdio.h>#include <stdlib.h>#include <unistd.h>#include <stdbool.h>#include <stdatomic.h> // Global state for coordinating shutdownatomic_bool shutdown_requested = false;pthread_mutex_t shutdown_mutex = PTHREAD_MUTEX_INITIALIZER;pthread_cond_t shutdown_cond = PTHREAD_COND_INITIALIZER; // Dedicated signal handling threadvoid *signal_handler_thread(void *arg) { sigset_t *mask = (sigset_t *)arg; int sig; printf("Signal handler thread started\n"); while (1) { // sigwait() SYNCHRONOUSLY waits for a signal // The signal is removed from pending and its number is returned // This is NOT async - we're just waiting like any blocking call int err = sigwait(mask, &sig); if (err != 0) { fprintf(stderr, "sigwait error: %d\n", err); continue; } printf("Signal handler thread received signal %d\n", sig); switch (sig) { case SIGINT: printf("Received SIGINT (Ctrl+C), initiating graceful shutdown\n"); atomic_store(&shutdown_requested, true); pthread_cond_broadcast(&shutdown_cond); return NULL; // Exit the signal thread case SIGTERM: printf("Received SIGTERM, initiating graceful shutdown\n"); atomic_store(&shutdown_requested, true); pthread_cond_broadcast(&shutdown_cond); return NULL; case SIGHUP: printf("Received SIGHUP, reloading configuration\n"); reload_configuration(); // Safe to call any function here! break; case SIGUSR1: printf("Received SIGUSR1, dumping statistics\n"); dump_statistics(); break; default: printf("Unhandled signal: %d\n", sig); } } return NULL;} // Worker threads - signal handling is NOT their concernvoid *worker_thread(void *arg) { int id = *(int *)arg; printf("Worker %d started\n", id); while (!atomic_load(&shutdown_requested)) { // Do real work... do_work(id); // Check for shutdown with timeout struct timespec timeout; clock_gettime(CLOCK_REALTIME, &timeout); timeout.tv_sec += 1; pthread_mutex_lock(&shutdown_mutex); pthread_cond_timedwait(&shutdown_cond, &shutdown_mutex, &timeout); pthread_mutex_unlock(&shutdown_mutex); } printf("Worker %d shutting down gracefully\n", id); cleanup_worker(id); return NULL;} int main() { pthread_t sig_thread; pthread_t workers[4]; int worker_ids[4] = {0, 1, 2, 3}; sigset_t mask; // Step 1: Block signals BEFORE creating any threads sigemptyset(&mask); sigaddset(&mask, SIGINT); sigaddset(&mask, SIGTERM); sigaddset(&mask, SIGHUP); sigaddset(&mask, SIGUSR1); // Use pthread_sigmask in main thread if (pthread_sigmask(SIG_BLOCK, &mask, NULL) != 0) { perror("pthread_sigmask"); exit(1); } // Step 2: Create worker threads // They inherit the blocked signal mask for (int i = 0; i < 4; i++) { pthread_create(&workers[i], NULL, worker_thread, &worker_ids[i]); } // Step 3: Create dedicated signal handling thread // Pass the mask so it knows which signals to wait for pthread_create(&sig_thread, NULL, signal_handler_thread, &mask); // Step 4: Wait for signal thread to exit (happens on SIGINT/SIGTERM) pthread_join(sig_thread, NULL); printf("Signal thread exited, waiting for workers...\n"); // Step 5: Wait for all workers to finish for (int i = 0; i < 4; i++) { pthread_join(workers[i], NULL); } printf("All threads exited, cleanup complete\n"); return 0;}The key insight is that sigwait() converts asynchronous signal delivery into synchronous processing. Instead of a signal interrupting your code at an arbitrary point and running a handler, your signal thread explicitly waits for signals like it would wait for any I/O. When sigwait() returns, you're in normal code—you can call any function, acquire locks, and use non-async-signal-safe functions safely.
When signals are handled via traditional handlers (not sigwait), extreme care is needed because the handler can interrupt any code at any point. A signal might arrive while a thread is in the middle of malloc(), printf(), or acquiring a mutex. If the signal handler calls any of these functions, deadlock or corruption is likely.
Async-signal-safe functions are the only functions guaranteed safe to call from signal handlers. POSIX explicitly lists these functions; everything else is unsafe.
| Category | Functions |
|---|---|
| Process Control | _exit, abort, getpid, fork, execve |
| Signal Operations | signal, sigaction, sigprocmask, kill, raise |
| I/O (Low-level) | read, write, open, close, dup, pipe |
| Time | time, clock_gettime, nanosleep, alarm |
| File Info | stat, fstat, lstat, access, chmod |
| Error | errno (accessing, not calling functions using it) |
These commonly-used functions are NOT async-signal-safe and must NEVER be called from signal handlers: printf(), sprintf(), malloc(), free(), new/delete, pthread_mutex_lock(), mutex operations of any kind, most of the C standard library, C++ STL functions, logging libraries, most third-party code. If you use a signal handler (not sigwait), you are limited to the explicit POSIX list.
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899
#include <signal.h>#include <unistd.h>#include <string.h>#include <errno.h> // CORRECT: Async-signal-safe signal handlervolatile sig_atomic_t signal_received = 0; void safe_handler(int sig) { // sig_atomic_t is guaranteed safe for signal handler access signal_received = sig; // write() is async-signal-safe const char msg[] = "Signal received\n"; write(STDOUT_FILENO, msg, sizeof(msg) - 1);} // INCORRECT: This handler is DANGEROUSvoid dangerous_handler(int sig) { // printf is NOT async-signal-safe // If the signal interrupts printf(), this causes deadlock or corruption printf("Received signal %d\n", sig); // WRONG! // malloc is NOT async-signal-safe char *buf = malloc(100); // WRONG! // Mutex operations are NOT async-signal-safe pthread_mutex_lock(&some_mutex); // WRONG! Deadlock if already held // free is NOT async-signal-safe free(buf); // WRONG!} // Pattern: Minimal handler, defer work to main loopvoid minimal_handler(int sig) { // Just set a flag signal_received = sig; // Let the main loop do the real work} void main_loop() { while (1) { if (signal_received != 0) { int sig = signal_received; signal_received = 0; // Now we're NOT in the handler context // All functions are safe to call printf("Processing signal %d\n", sig); handle_signal_safely(sig); } do_normal_work(); }} // Advanced: Using self-pipe trick for event loop integrationint signal_pipe[2]; void signal_to_pipe(int sig) { // write() is async-signal-safe // write the signal number to a pipe char c = (char)sig; write(signal_pipe[1], &c, 1);} void init_signal_pipe() { pipe(signal_pipe); // Make write end non-blocking fcntl(signal_pipe[1], F_SETFL, O_NONBLOCK); // Set up handler struct sigaction sa; sa.sa_handler = signal_to_pipe; sigemptyset(&sa.sa_mask); sa.sa_flags = SA_RESTART; sigaction(SIGINT, &sa, NULL);} void event_loop() { struct pollfd fds[2]; fds[0].fd = signal_pipe[0]; fds[0].events = POLLIN; // ... other fds ... while (1) { poll(fds, 2, -1); if (fds[0].revents & POLLIN) { char sig; read(signal_pipe[0], &sig, 1); // Handle signal safely in main context handle_signal_safely((int)sig); } // Handle other events... }}When a signal is delivered to a thread that's blocked in a system call, the system call is typically interrupted. This creates a critical consideration for robust code: handling EINTR.
The Interruption Sequence:
read(), accept(), sem_wait())The SA_RESTART Flag:
When installing a signal handler with sigaction(), the SA_RESTART flag requests that certain system calls be automatically restarted after the handler returns, rather than failing with EINTR. However, not all system calls are restartable.
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596
#include <stdio.h>#include <errno.h>#include <unistd.h>#include <signal.h>#include <sys/socket.h> // Pattern 1: Manual EINTR retry loopssize_t read_with_retry(int fd, void *buf, size_t count) { ssize_t result; do { result = read(fd, buf, count); } while (result < 0 && errno == EINTR); return result;} // Pattern 2: Accept with EINTR handlingint accept_with_retry(int sockfd, struct sockaddr *addr, socklen_t *len) { int client; while (1) { client = accept(sockfd, addr, len); if (client >= 0) { return client; // Success } if (errno == EINTR) { // Interrupted by signal, retry continue; } // Real error return -1; }} // More realistic: Check for shutdown between retriesatomic_bool shutdown_flag = false; ssize_t read_interruptible(int fd, void *buf, size_t count) { ssize_t result; while (1) { result = read(fd, buf, count); if (result >= 0) { return result; // Success } if (errno == EINTR) { // Interrupted - check if we should stop if (shutdown_flag) { errno = ECANCELED; // Indicate controlled shutdown return -1; } continue; // Retry } // Real error return -1; }} // Using SA_RESTARTvoid setup_handler_with_restart() { struct sigaction sa; sa.sa_handler = my_handler; sigemptyset(&sa.sa_mask); sa.sa_flags = SA_RESTART; // Request automatic restart sigaction(SIGUSR1, &sa, NULL); // With SA_RESTART, many syscalls automatically retry after handler // BUT: This is not guaranteed for all functions! // Non-restartable: poll(), select(), sem_wait(), nanosleep(), ...} // Table of restartable vs non-restartable calls (depends on OS)/* * Typically restartable with SA_RESTART: * read(), write(), recv(), send() * open(), close() * wait(), waitpid() * ioctl() (some) * flock() * * Typically NOT restartable even with SA_RESTART: * poll(), select(), epoll_wait() * msgsnd(), msgrcv() * sem_wait(), sem_timedwait() * nanosleep(), clock_nanosleep() * connect() (with timeout) * futex() operations */Even with SA_RESTART, some system calls cannot be restarted and will return EINTR. In production code, always wrap blocking calls in retry loops that check for EINTR. Libraries like glibc provide TEMP_FAILURE_RETRY() macro for this purpose. Never assume a successful installation of SA_RESTART means your code is safe from EINTR.
Signal handling in threaded programs is notoriously difficult to get right. Here are the most common mistakes and how to avoid them:
1234567891011121314151617181920212223242526272829303132333435363738394041424344454647484950515253545556575859606162
// PITFALL 1: Handler set after threads existvoid bad_setup() { pthread_t t1, t2; pthread_create(&t1, NULL, worker, NULL); pthread_create(&t2, NULL, worker, NULL); // WRONG: Handler set after threads created // Behavior is unpredictable signal(SIGINT, handler);} void good_setup() { sigset_t mask; sigemptyset(&mask); sigaddset(&mask, SIGINT); pthread_sigmask(SIG_BLOCK, &mask, NULL); // Block FIRST pthread_t t1, t2; pthread_create(&t1, NULL, worker, NULL); // Inherits blocked mask pthread_create(&t2, NULL, worker, NULL); pthread_t sig_thread; pthread_create(&sig_thread, NULL, signal_handler, &mask);} // PITFALL 2: Race between flag check and waitvolatile sig_atomic_t stop = 0; void bad_signal_loop(sigset_t *mask) { while (!stop) { // Check flag // RACE: Signal could arrive HERE // After we check but before we wait sigwait(mask, &sig); // Missing signal! // ... }} // Better: Use sigwait's blocking nature correctlyvoid good_signal_loop(sigset_t *mask) { int sig; while (1) { sigwait(mask, &sig); // Atomically waits if (sig == SIGTERM) { break; // Clean exit } handle_signal(sig); }} // PITFALL 3: Assuming signal delivery to specific threadvoid bad_assumption() { // Main thread sets up for SIGINT // Developer assumes main thread will receive it // WRONG: Any thread with SIGINT unblocked might get it while (1) { pause(); // Wait for signal // Might never see SIGINT if another thread gets it! }}Signal handling in multithreaded programs is one of the most complex areas in systems programming. Let's consolidate the essential concepts:
You now understand the intricate relationship between signals and threads. The dedicated signal thread pattern is your primary tool for robust signal handling in production threaded code. Next, we'll explore thread-local storage—a mechanism for giving threads their own private data within shared address spaces.