Pipes - Learning Module

Loading content...

0/227

Parent-Child Communication

The Family Connection

Anonymous pipes exist in a peculiar situation: they have no name, no path, no way for unrelated processes to discover them. So how can they connect processes? The answer lies in inheritance—the fundamental mechanism by which a child process receives its execution environment from its parent.

When a process calls fork(), the child inherits virtually everything from the parent: memory contents, register state, open file descriptors—including pipe file descriptors. This inheritance is the sole mechanism by which anonymous pipes propagate between processes. It's why pipes are sometimes called 'hereditary IPC'—they can only travel through the family tree.

This page explores parent-child communication in depth: how inheritance works, the patterns for redirecting standard streams, how shells implement pipelines, the fork-exec pattern, and sophisticated coordination scenarios that leverage the unique properties of this relationship.

What You Will Learn

By the end of this page, you will master file descriptor inheritance through fork(), know how to redirect stdin/stdout to pipes using dup2(), understand how shells implement pipelines, effectively use the fork-exec pattern for launching programs, and coordinate complex parent-child workflows.

File Descriptor Inheritance Through fork()

The fork() system call creates a new process by duplicating the calling process. The child is an almost-exact copy of the parent, differing only in:

Process ID (PID)
Parent process ID (PPID)
Return value from fork() (0 in child, child's PID in parent)
A few other process-specific attributes (pending signals, locks, etc.)

Critically, the file descriptor table is duplicated. Every open file descriptor in the parent exists in the child, pointing to the same underlying kernel objects. For pipes, this means:

Parent creates pipe: pipe(pipefd) → gets fd[0] and fd[1]
Parent forks: child inherits copies of fd[0] and fd[1]
Both processes now have access to the same pipe
The kernel's reference counts double (readers=2, writers=2)

fd_inheritance.txt
┌─────────────────────────────────────────────────────────────────────────┐
│              FILE DESCRIPTOR INHERITANCE THROUGH FORK()                  │
└─────────────────────────────────────────────────────────────────────────┘
 
BEFORE FORK - Only Parent Exists
════════════════════════════════════════════════════════════════════════════
 
  PARENT PROCESS (PID 100)                    KERNEL PIPE OBJECT
  ┌─────────────────────────┐                ┌──────────────────┐
  │ File Descriptor Table   │                │ readers = 1      │
  │ ───────────────────── │                │ writers = 1      │
  │ 0 → stdin              │                │ buffer = [...]   │
  │ 1 → stdout             │                └────────▲─────────┘
  │ 2 → stderr             │                         │
  │ 3 → pipe read end  ────┼─────────────────────────┤
  │ 4 → pipe write end ────┼─────────────────────────┘
  └─────────────────────────┘
 
 
AFTER FORK - Parent and Child
════════════════════════════════════════════════════════════════════════════
 
  PARENT PROCESS (PID 100)                    KERNEL PIPE OBJECT
  ┌─────────────────────────┐                ┌──────────────────┐
  │ File Descriptor Table   │                │ readers = 2  ←──┐│
  │ ───────────────────── │                │ writers = 2  ←──┼┤
  │ 0 → stdin              │                │ buffer = [...]   ││
  │ 1 → stdout             │                └────────▲─────────┘│
  │ 2 → stderr             │                         │          │
  │ 3 → pipe read end  ────┼─────────────────────────┤          │
  │ 4 → pipe write end ────┼─────────────────────────┤          │
  └─────────────────────────┘                        │          │
                                                     │          │
  CHILD PROCESS (PID 101)                            │          │
  ┌─────────────────────────┐                        │          │
  │ File Descriptor Table   │                        │          │
  │ ───────────────────── │  (inherited from       │          │
  │ 0 → stdin              │   parent, same FD #s)  │          │
  │ 1 → stdout             │                        │          │
  │ 2 → stderr             │                        │          │
  │ 3 → pipe read end  ────┼────────────────────────┘          │
  │ 4 → pipe write end ────┼───────────────────────────────────┘
  └─────────────────────────┘
 
  Key Insight: fd[3] in parent and fd[3] in child point to
  the SAME kernel pipe object. Changes by one are visible
  to the other. Closing in one doesn't close in the other.

Critical Understanding: Shared Kernel Objects

The file descriptors themselves are duplicated (each process has its own table), but they point to the same kernel objects. This means:

Data written by the parent appears for the child to read
Data written by the child appears for the parent to read
Buffer state is shared—both see the same bytes
Closing in one process decrements the reference count but doesn't close the underlying object until all references are gone

The Two-Phase Pattern:

Effective parent-child pipe communication follows a consistent pattern:

Before fork: Create the pipe
After fork: Each process closes the end it won't use
Communication: Use the remaining ends
Cleanup: Close the used ends and wait for child

This pattern establishes clear roles (one writes, one reads) and ensures proper EOF signaling.

The Inheritance Trap

Because both processes inherit both ends, failing to close unused ends prevents EOF detection. If the child keeps the write end open (even if it never writes), the reader will never see EOF. This is the most common parent-child pipe bug.

Redirecting Standard Streams with dup2()

While you can use pipe file descriptors directly with read/write, the most powerful pattern involves redirecting standard streams (stdin, stdout, stderr). This allows:

Programs that read from stdin to read from the pipe instead
Programs that write to stdout to write to the pipe instead
Existing programs to participate in pipelines without modification

The dup2() system call is the key:

int dup2(int oldfd, int newfd);

dup2(oldfd, newfd) makes newfd refer to the same kernel object as oldfd. If newfd was already open, it's closed first. After the call, operations on newfd are indistinguishable from operations on oldfd.

dup2_redirection.c
C
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
#include <unistd.h>
#include <stdio.h>
#include <stdlib.h>
#include <sys/wait.h>
 
/**
 * Demonstrates redirecting stdout to a pipe.
 * 
 * Child: printf() → stdout → pipe → Parent: read()
 */
int main(void) {
    int pipefd[2];
    pid_t pid;
    
    if (pipe(pipefd) == -1) {
        perror("pipe");
        exit(EXIT_FAILURE);
    }
    
    pid = fork();
    if (pid == -1) {
        perror("fork");
        exit(EXIT_FAILURE);
    }
    
    if (pid == 0) {
        // ═══════════════════════════════════════════════════════════
        // CHILD: Redirect stdout to pipe write end
        // ═══════════════════════════════════════════════════════════
        
        close(pipefd[0]);  // Close read end (child only writes)
        
        // MAGIC STEP: Make stdout (fd 1) point to the pipe write end
        // After this, printf() writes to the pipe!
        if (dup2(pipefd[1], STDOUT_FILENO) == -1) {
            perror("dup2");
            exit(EXIT_FAILURE);
        }
        
        // Close original write end (stdout is now our write end)
        close(pipefd[1]);
        
        // Now printf() goes to the pipe!
        printf("Hello from child process!\n");
        printf("This goes through the pipe.\n");
        printf("No modifications to standard library needed.\n");
        
        // Important: flush before exiting
        fflush(stdout);
        
        exit(EXIT_SUCCESS);
    }
    
    // ═══════════════════════════════════════════════════════════
    // PARENT: Read from pipe
    // ═══════════════════════════════════════════════════════════
    
    close(pipefd[1]);  // Close write end
    
    char buffer[256];
    ssize_t n;
    
    printf("[Parent] Reading from pipe:\n");
    printf("─────────────────────────────\n");
    
    while ((n = read(pipefd[0], buffer, sizeof(buffer) - 1)) > 0) {
        buffer[n] = '\0';
        printf("%s", buffer);
    }
    
    printf("─────────────────────────────\n");
    printf("[Parent] Child finished.\n");
    
    close(pipefd[0]);
    waitpid(pid, NULL, 0);
    
    return 0;
}

The dup2() Dance:

The sequence for redirection is:

close(unused_end) — Close the pipe end we won't use
dup2(pipe_end, target_fd) — Make target_fd (0 or 1) point to pipe
close(pipe_end) — Close original pipe fd (target_fd is now our handle)

Why close the original pipe fd after dup2()?

After dup2(pipefd[1], STDOUT_FILENO), we have two file descriptors pointing to the write end:

pipefd[1] (the original)
STDOUT_FILENO (the copy)

Calling close(pipefd[1]) removes the original reference, but STDOUT_FILENO still works. This is clean because:

We only have one fd to track (stdout)
When we close stdout or exit, the reference count goes to 0
No confusion about which fd to use

redirect_stdin.c
C
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
#include <unistd.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <sys/wait.h>
 
/**
 * Demonstrates redirecting stdin from a pipe.
 * 
 * Parent: write() → pipe → Child: scanf()/getchar()/fgets()
 */
int main(void) {
    int pipefd[2];
    pid_t pid;
    
    if (pipe(pipefd) == -1) {
        perror("pipe");
        exit(EXIT_FAILURE);
    }
    
    pid = fork();
    if (pid == -1) {
        perror("fork");
        exit(EXIT_FAILURE);
    }
    
    if (pid == 0) {
        // ═══════════════════════════════════════════════════════════
        // CHILD: Redirect stdin to pipe read end
        // ═══════════════════════════════════════════════════════════
        
        close(pipefd[1]);  // Close write end (child only reads)
        
        // Make stdin read from the pipe
        if (dup2(pipefd[0], STDIN_FILENO) == -1) {
            perror("dup2");
            exit(EXIT_FAILURE);
        }
        
        close(pipefd[0]);  // Close original
        
        // Now standard input functions read from the pipe!
        char line[256];
        int line_num = 1;
        
        fprintf(stderr, "[Child] Reading lines from 'stdin':\n");
        
        while (fgets(line, sizeof(line), stdin) != NULL) {
            fprintf(stderr, "[Child] Line %d: %s", line_num++, line);
        }
        
        fprintf(stderr, "[Child] EOF on stdin.\n");
        exit(EXIT_SUCCESS);
    }
    
    // ═══════════════════════════════════════════════════════════
    // PARENT: Write to pipe
    // ═══════════════════════════════════════════════════════════
    
    close(pipefd[0]);  // Close read end
    
    const char *lines[] = {
        "First line of input\n",
        "Second line of input\n",
        "Third and final line\n"
    };
    
    for (int i = 0; i < 3; i++) {
        write(pipefd[1], lines[i], strlen(lines[i]));
    }
    
    // Close write end to signal EOF
    close(pipefd[1]);
    printf("[Parent] Sent data and closed pipe.\n");
    
    waitpid(pid, NULL, 0);
    printf("[Parent] Child exited.\n");
    
    return 0;
}

dup2() Is Atomic

The close(newfd) and dup(oldfd) in dup2() happen atomically in a single system call. This prevents races where another thread might grab the fd between close and dup. Always prefer dup2() over close() + dup() for this reason.

Implementing Shell Pipelines

When you type ls | grep txt | wc -l in a shell, the shell orchestrates a complex dance of process creation, pipe setup, and stream redirection. Understanding this implementation reveals the full power of parent-child pipe communication.

The Algorithm:

Parse the command line into separate commands
For N commands, create N-1 pipes
Fork N children
Each child redirects its stdin/stdout to appropriate pipe ends
Each child exec()s its command
Parent closes all pipe ends and waits

Let's implement a two-command pipeline: cmd1 | cmd2

simple_pipeline.c
C
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
#include <unistd.h>
#include <stdio.h>
#include <stdlib.h>
#include <sys/wait.h>
 
/**
 * Implements: ls -la | grep ".c"
 * 
 * This is how a shell executes simple pipelines.
 */
int main(void) {
    int pipefd[2];
    pid_t pid1, pid2;
    
    // Create the pipe connecting the two commands
    if (pipe(pipefd) == -1) {
        perror("pipe");
        exit(EXIT_FAILURE);
    }
    
    // ═══════════════════════════════════════════════════════════
    // Fork first child: ls -la (writes to pipe)
    // ═══════════════════════════════════════════════════════════
    pid1 = fork();
    if (pid1 == -1) {
        perror("fork");
        exit(EXIT_FAILURE);
    }
    
    if (pid1 == 0) {
        // Child 1: ls -la
        
        // Redirect stdout to pipe write end
        close(pipefd[0]);  // Don't need read end
        dup2(pipefd[1], STDOUT_FILENO);
        close(pipefd[1]);  // Close original
        
        // Execute ls
        execlp("ls", "ls", "-la", NULL);
        
        // If exec fails
        perror("execlp ls");
        exit(EXIT_FAILURE);
    }
    
    // ═══════════════════════════════════════════════════════════
    // Fork second child: grep ".c" (reads from pipe)
    // ═══════════════════════════════════════════════════════════
    pid2 = fork();
    if (pid2 == -1) {
        perror("fork");
        exit(EXIT_FAILURE);
    }
    
    if (pid2 == 0) {
        // Child 2: grep ".c"
        
        // Redirect stdin to pipe read end
        close(pipefd[1]);  // Don't need write end
        dup2(pipefd[0], STDIN_FILENO);
        close(pipefd[0]);  // Close original
        
        // Execute grep
        execlp("grep", "grep", ".c", NULL);
        
        // If exec fails
        perror("execlp grep");
        exit(EXIT_FAILURE);
    }
    
    // ═══════════════════════════════════════════════════════════
    // Parent: Close all pipe ends and wait
    // ═══════════════════════════════════════════════════════════
    
    // CRITICAL: Parent must close BOTH pipe ends!
    // Otherwise children may not see EOF (grep would block forever)
    close(pipefd[0]);
    close(pipefd[1]);
    
    // Wait for both children
    int status1, status2;
    waitpid(pid1, &status1, 0);
    waitpid(pid2, &status2, 0);
    
    return WEXITSTATUS(status2);  // Return grep's exit status
}

The Critical Parent Close:

Notice that the parent closes both pipe ends after forking both children. This is essential:

Parent holds fd copies from before any fork
If parent keeps pipefd[1] open, grep (child 2) never sees EOF
If parent keeps pipefd[0] open, ls (child 1) gets SIGPIPE if grep exits

The parent's job is to set up the plumbing and get out of the way.

Longer Pipelines:

For cmd1 | cmd2 | cmd3 | cmd4, the pattern extends:

multi_stage_pipeline.c
C
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
#include <unistd.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <sys/wait.h>
 
/**
 * Generalized N-stage pipeline executor.
 * 
 * Implements: cat file | grep pattern | sort | uniq
 */
 
typedef struct {
    char *cmd;
    char **args;
} Command;
 
void execute_pipeline(Command *cmds, int num_cmds) {
    // Need num_cmds - 1 pipes
    int pipes[num_cmds - 1][2];
    
    // Create all pipes first
    for (int i = 0; i < num_cmds - 1; i++) {
        if (pipe(pipes[i]) == -1) {
            perror("pipe");
            exit(EXIT_FAILURE);
        }
    }
    
    // Fork each command
    pid_t pids[num_cmds];
    
    for (int i = 0; i < num_cmds; i++) {
        pids[i] = fork();
        
        if (pids[i] == -1) {
            perror("fork");
            exit(EXIT_FAILURE);
        }
        
        if (pids[i] == 0) {
            // ═══════════════════════════════════════════════════
            // CHILD i
            // ═══════════════════════════════════════════════════
            
            // Redirect stdin (except first command)
            if (i > 0) {
                dup2(pipes[i - 1][0], STDIN_FILENO);
            }
            
            // Redirect stdout (except last command)
            if (i < num_cmds - 1) {
                dup2(pipes[i][1], STDOUT_FILENO);
            }
            
            // Close ALL pipe fds in child
            // (we only need stdin/stdout now)
            for (int j = 0; j < num_cmds - 1; j++) {
                close(pipes[j][0]);
                close(pipes[j][1]);
            }
            
            // Execute
            execvp(cmds[i].cmd, cmds[i].args);
            perror("execvp");
            exit(EXIT_FAILURE);
        }
    }
    
    // ═══════════════════════════════════════════════════
    // PARENT: Close all pipes and wait
    // ═══════════════════════════════════════════════════
    
    for (int i = 0; i < num_cmds - 1; i++) {
        close(pipes[i][0]);
        close(pipes[i][1]);
    }
    
    for (int i = 0; i < num_cmds; i++) {
        waitpid(pids[i], NULL, 0);
    }
}
 
int main(void) {
    // Simulate: echo "hello\nworld\nhello" | sort | uniq
    
    char *echo_args[] = {"echo", "-e", "hello\nworld\nhello", NULL};
    char *sort_args[] = {"sort", NULL};
    char *uniq_args[] = {"uniq", NULL};
    
    Command pipeline[] = {
        { "echo", echo_args },
        { "sort", sort_args },
        { "uniq", uniq_args }
    };
    
    execute_pipeline(pipeline, 3);
    
    return 0;
}

Real Shells Are More Complex

Actual shells handle many additional concerns: here-documents, process substitution, background execution, signal handling, job control, and error recovery. But the core pipe-based communication follows exactly this pattern.

The Fork-Exec Pattern in Depth

The fork-exec pattern is the standard Unix method for launching new programs while establishing communication channels. It separates two concerns:

fork() — Create a new process (with pipe access)
exec() — Replace the process image with a new program

Between fork and exec, the child can set up its environment—redirecting pipes, changing working directories, setting environment variables—before the new program starts.

Why This Separation Matters:

The new program inherits the pipe connections without knowing they exist
Standard programs (grep, sort, cat) work in pipelines without modification
Environment setup is flexible and composable
The pattern is applicable to any resource setup, not just pipes

fork_exec_detailed.c
C
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
#include <unistd.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <sys/wait.h>
#include <errno.h>
 
/**
 * Detailed fork-exec pattern with error pipe for reliable error reporting.
 * 
 * Problem: If exec() fails after fork(), how does the parent know?
 * Solution: Use an error-reporting pipe with O_CLOEXEC.
 */
int spawn_with_pipes(char *const argv[], 
                     int *child_stdin,   // NULL to not redirect
                     int *child_stdout,  // NULL to not redirect
                     pid_t *child_pid) {
    
    int in_pipe[2] = {-1, -1};
    int out_pipe[2] = {-1, -1};
    int err_pipe[2];
    
    // Create error-reporting pipe with O_CLOEXEC
    // If exec succeeds, it auto-closes and parent sees EOF
    // If exec fails, child writes error before closing
    if (pipe2(err_pipe, O_CLOEXEC) == -1) {
        return -1;
    }
    
    // Create input pipe if requested
    if (child_stdin && pipe(in_pipe) == -1) {
        close(err_pipe[0]); close(err_pipe[1]);
        return -1;
    }
    
    // Create output pipe if requested
    if (child_stdout && pipe(out_pipe) == -1) {
        close(err_pipe[0]); close(err_pipe[1]);
        if (child_stdin) { close(in_pipe[0]); close(in_pipe[1]); }
        return -1;
    }
    
    pid_t pid = fork();
    if (pid == -1) {
        // Fork failed, clean up
        close(err_pipe[0]); close(err_pipe[1]);
        if (child_stdin) { close(in_pipe[0]); close(in_pipe[1]); }
        if (child_stdout) { close(out_pipe[0]); close(out_pipe[1]); }
        return -1;
    }
    
    if (pid == 0) {
        // ═══════════════════════════════════════════════════════════
        // CHILD PROCESS
        // ═══════════════════════════════════════════════════════════
        
        // Close parent's ends of the pipes
        close(err_pipe[0]);
        if (child_stdin) close(in_pipe[1]);
        if (child_stdout) close(out_pipe[0]);
        
        // Redirect stdin if requested
        if (child_stdin) {
            dup2(in_pipe[0], STDIN_FILENO);
            close(in_pipe[0]);
        }
        
        // Redirect stdout if requested
        if (child_stdout) {
            dup2(out_pipe[1], STDOUT_FILENO);
            close(out_pipe[1]);
        }
        
        // Execute the program
        execvp(argv[0], argv);
        
        // If we get here, exec failed! Report via error pipe
        int saved_errno = errno;
        write(err_pipe[1], &saved_errno, sizeof(saved_errno));
        close(err_pipe[1]);
        _exit(127);  // Convention for "command not found"
    }
    
    // ═══════════════════════════════════════════════════════════
    // PARENT PROCESS
    // ═══════════════════════════════════════════════════════════
    
    // Close child's ends
    close(err_pipe[1]);
    if (child_stdin) close(in_pipe[0]);
    if (child_stdout) close(out_pipe[1]);
    
    // Check if exec succeeded
    int exec_errno;
    ssize_t n = read(err_pipe[0], &exec_errno, sizeof(exec_errno));
    close(err_pipe[0]);
    
    if (n > 0) {
        // exec() failed! Child sent us the errno
        if (child_stdin) close(in_pipe[1]);
        if (child_stdout) close(out_pipe[0]);
        waitpid(pid, NULL, 0);  // Reap the child
        errno = exec_errno;
        return -1;
    }
    
    // Success! Return the fds to caller
    if (child_stdin) *child_stdin = in_pipe[1];    // Parent writes here
    if (child_stdout) *child_stdout = out_pipe[0]; // Parent reads here
    if (child_pid) *child_pid = pid;
    
    return 0;
}
 
int main(void) {
    int child_in, child_out;
    pid_t pid;
    
    char *cat_args[] = {"cat", "-n", NULL};  // Number lines
    
    if (spawn_with_pipes(cat_args, &child_in, &child_out, &pid) == -1) {
        perror("spawn");
        return 1;
    }
    
    printf("[Parent] Spawned cat with PID %d\n", pid);
    
    // Write to child's stdin
    const char *input = "Line one\nLine two\nLine three\n";
    write(child_in, input, strlen(input));
    close(child_in);  // Signal EOF
    
    // Read from child's stdout
    char buffer[256];
    ssize_t n;
    printf("[Parent] Output from cat -n:\n");
    while ((n = read(child_out, buffer, sizeof(buffer) - 1)) > 0) {
        buffer[n] = '\0';
        printf("%s", buffer);
    }
    close(child_out);
    
    waitpid(pid, NULL, 0);
    return 0;
}

The Error Pipe Trick

The O_CLOEXEC error pipe is a clever technique: if exec() succeeds, the pipe closes automatically (because of CLOEXEC), and the parent sees EOF on read(). If exec() fails, the child writes the error before exiting, and the parent receives it. This distinguishes exec failures from runtime failures of the new program.

Communicating with Multiple Children

Complex parent processes often manage multiple children simultaneously. This introduces challenges in pipe management, multiplexing, and coordination.

Pattern 1: Pool of Workers

Parent distributes work to a pool of child workers, each with its own pipe pair:

worker_pool.c
C
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
#include <unistd.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <sys/wait.h>
#include <poll.h>
 
#define NUM_WORKERS 4
 
typedef struct {
    pid_t pid;
    int to_worker;    // Parent writes here
    int from_worker;  // Parent reads here
    int busy;
} Worker;
 
void worker_process(int read_fd, int write_fd, int worker_id) {
    char task[256];
    ssize_t n;
    
    while ((n = read(read_fd, task, sizeof(task) - 1)) > 0) {
        task[n] = '\0';
        
        // Simulate work
        usleep(100000);  // 100ms
        
        // Send result
        char result[256];
        int len = snprintf(result, sizeof(result), 
                          "Worker %d completed: %s", worker_id, task);
        write(write_fd, result, len);
    }
    
    close(read_fd);
    close(write_fd);
}
 
int main(void) {
    Worker workers[NUM_WORKERS];
    
    // Create worker pool
    for (int i = 0; i < NUM_WORKERS; i++) {
        int parent_to_child[2], child_to_parent[2];
        pipe(parent_to_child);
        pipe(child_to_parent);
        
        pid_t pid = fork();
        if (pid == 0) {
            // Child: worker process
            close(parent_to_child[1]);
            close(child_to_parent[0]);
            worker_process(parent_to_child[0], child_to_parent[1], i);
            exit(0);
        }
        
        // Parent: store handles
        close(parent_to_child[0]);
        close(child_to_parent[1]);
        
        workers[i].pid = pid;
        workers[i].to_worker = parent_to_child[1];
        workers[i].from_worker = child_to_parent[0];
        workers[i].busy = 0;
    }
    
    printf("Created %d workers\n", NUM_WORKERS);
    
    // Distribute tasks
    const char *tasks[] = {
        "Task_A", "Task_B", "Task_C", "Task_D",
        "Task_E", "Task_F", "Task_G", "Task_H"
    };
    int num_tasks = 8;
    int next_task = 0;
    int completed = 0;
    
    while (completed < num_tasks) {
        // Send tasks to available workers
        while (next_task < num_tasks) {
            int found = 0;
            for (int i = 0; i < NUM_WORKERS && !found; i++) {
                if (!workers[i].busy) {
                    char msg[64];
                    snprintf(msg, sizeof(msg), "%s", tasks[next_task]);
                    write(workers[i].to_worker, msg, strlen(msg));
                    workers[i].busy = 1;
                    printf("Sent %s to worker %d\n", tasks[next_task], i);
                    next_task++;
                    found = 1;
                }
            }
            if (!found) break;  // All workers busy
        }
        
        // Collect results using poll()
        struct pollfd pfds[NUM_WORKERS];
        for (int i = 0; i < NUM_WORKERS; i++) {
            pfds[i].fd = workers[i].busy ? workers[i].from_worker : -1;
            pfds[i].events = POLLIN;
        }
        
        if (poll(pfds, NUM_WORKERS, 100) > 0) {
            for (int i = 0; i < NUM_WORKERS; i++) {
                if (pfds[i].revents & POLLIN) {
                    char result[256];
                    ssize_t n = read(workers[i].from_worker, 
                                     result, sizeof(result) - 1);
                    if (n > 0) {
                        result[n] = '\0';
                        printf("Result: %s\n", result);
                        workers[i].busy = 0;
                        completed++;
                    }
                }
            }
        }
    }
    
    // Cleanup
    for (int i = 0; i < NUM_WORKERS; i++) {
        close(workers[i].to_worker);
        close(workers[i].from_worker);
        waitpid(workers[i].pid, NULL, 0);
    }
    
    printf("All tasks completed!\n");
    return 0;
}

Key Insights from the Worker Pool Pattern:

Separate pipes per worker — Each worker gets its own pipes. This prevents message interleaving and simplifies routing.
Multiplexing with poll() — The parent uses poll() to efficiently wait on multiple workers simultaneously without spinning.
Work distribution — Round-robin, first-available, or load-balanced distribution strategies are all possible.
Reply routing is implicit — Because each worker has its own reply pipe, the parent knows which worker sent each result.

Pattern 2: Parent as Coordinator

In more complex scenarios, the parent orchestrates communication between children through itself:

coordinator_pattern.txt
┌─────────────────────────────────────────────────────────────────────────┐
│                  PARENT AS COORDINATOR PATTERN                           │
└─────────────────────────────────────────────────────────────────────────┘
 
                              ┌──────────────┐
                              │    PARENT    │
                              │ (Coordinator)│
                              └──────┬───────┘
                                     │
           ┌──────────────┬──────────┼──────────┬──────────────┐
           │              │          │          │              │
           ▼              ▼          ▼          ▼              ▼
     ┌─────────┐    ┌─────────┐    ···   ┌─────────┐    ┌─────────┐
     │ Child 1 │    │ Child 2 │          │ Child N │    │ Child M │
     │ (Reader)│    │(Compute)│          │(Compute)│    │ (Writer)│
     └─────────┘    └─────────┘          └─────────┘    └─────────┘
 
  Data Flow:
  1. Child 1 reads input, sends to Parent
  2. Parent distributes to compute children (2..N)
  3. Compute children process and return results
  4. Parent aggregates and sends to Child M (Writer)
  5. Child M writes final output
 
  Why Go Through Parent?
  - Centralized control and monitoring
  - Dynamic routing decisions
  - Error handling and recovery
  - Rate limiting and flow control
  - Children remain simple and focused

When Direct Child-to-Child Is Needed

If children need to communicate directly without going through the parent, anonymous pipes become awkward (each child would need to inherit pipes to all siblings). Named pipes or sockets are better choices for complex many-to-many communication topologies.

Error Handling and Recovery

Robust parent-child pipe communication requires careful error handling. Children can die unexpectedly, pipes can break, and the system can run out of resources. Here's how to handle these scenarios:

Child Death Detection:

When a child terminates, the parent needs to know. Two mechanisms:

SIGCHLD signal — Sent to parent when child state changes
Pipe behavior — Write fails with EPIPE; read returns 0 (EOF)

Handling SIGCHLD:

error_handling.c
C
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
#include <unistd.h>
#include <stdio.h>
#include <stdlib.h>
#include <signal.h>
#include <errno.h>
#include <sys/wait.h>
#include <string.h>
 
/**
 * Robust error handling for parent-child communication.
 */
 
volatile sig_atomic_t child_exited = 0;
volatile pid_t exited_pid = 0;
volatile int exit_status = 0;
 
void sigchld_handler(int sig) {
    (void)sig;
    int saved_errno = errno;
    
    pid_t pid;
    int status;
    
    // Reap all dead children (might be multiple)
    while ((pid = waitpid(-1, &status, WNOHANG)) > 0) {
        child_exited = 1;
        exited_pid = pid;
        exit_status = status;
    }
    
    errno = saved_errno;
}
 
void setup_sigchld(void) {
    struct sigaction sa;
    sa.sa_handler = sigchld_handler;
    sigemptyset(&sa.sa_mask);
    sa.sa_flags = SA_RESTART | SA_NOCLDSTOP;  // Don't notify on stop
    sigaction(SIGCHLD, &sa, NULL);
}
 
/**
 * Write all bytes, handling signals and partial writes.
 */
ssize_t safe_write_all(int fd, const void *buf, size_t count) {
    const char *ptr = buf;
    size_t remaining = count;
    
    while (remaining > 0) {
        ssize_t n = write(fd, ptr, remaining);
        
        if (n == -1) {
            if (errno == EINTR) {
                // Interrupted by signal, check for dead child
                if (child_exited) {
                    fprintf(stderr, "Child %d exited during write\n", 
                            exited_pid);
                    // Decide: abort or continue?
                }
                continue;  // Retry the write
            }
            if (errno == EPIPE) {
                // Reader is gone
                fprintf(stderr, "Pipe closed by reader\n");
                return count - remaining;  // Return bytes written so far
            }
            return -1;  // Real error
        }
        
        ptr += n;
        remaining -= n;
    }
    
    return count;
}
 
/**
 * Read with timeout using poll().
 */
#include <poll.h>
 
ssize_t read_with_timeout(int fd, void *buf, size_t count, int timeout_ms) {
    struct pollfd pfd = { .fd = fd, .events = POLLIN };
    
    int ret = poll(&pfd, 1, timeout_ms);
    
    if (ret == -1) {
        if (errno == EINTR) {
            // Interrupted, could be SIGCHLD
            if (child_exited) {
                errno = ECHILD;  // Custom: child died
                return -1;
            }
        }
        return -1;
    }
    
    if (ret == 0) {
        errno = ETIMEDOUT;  // Timeout
        return -1;
    }
    
    if (pfd.revents & POLLERR) {
        errno = EIO;
        return -1;
    }
    
    if (pfd.revents & POLLHUP) {
        // Write end closed, but might still have data
    }
    
    return read(fd, buf, count);
}
 
int main(void) {
    // Ignore SIGPIPE, handle EPIPE programmatically
    signal(SIGPIPE, SIG_IGN);
    setup_sigchld();
    
    int pipefd[2];
    pipe(pipefd);
    
    pid_t pid = fork();
    if (pid == 0) {
        // Child: read a bit then exit unexpectedly
        close(pipefd[1]);
        
        char buf[100];
        read(pipefd[0], buf, 50);
        
        fprintf(stderr, "[Child] Exiting unexpectedly!\n");
        exit(42);
    }
    
    close(pipefd[0]);
    
    // Parent: keep writing until child dies
    char data[1000];
    memset(data, 'X', sizeof(data));
    
    while (1) {
        ssize_t n = safe_write_all(pipefd[1], data, sizeof(data));
        
        if (n < (ssize_t)sizeof(data)) {
            printf("[Parent] Write interrupted. Bytes written: %zd\n", n);
            break;
        }
        
        if (child_exited) {
            printf("[Parent] Detected child exit (pid=%d, status=%d)\n",
                   exited_pid, WEXITSTATUS(exit_status));
            break;
        }
    }
    
    close(pipefd[1]);
    waitpid(pid, NULL, 0);  // Cleanup
    
    return 0;
}

Signal Handler Restrictions

Signal handlers can only use async-signal-safe functions. You cannot safely use printf, malloc, or most library functions in handlers. Set flags and handle them in main code. The handler above is carefully written to only set volatile variables and call waitpid (which is async-signal-safe).

Best Practices for Parent-Child Pipes

Over decades of Unix programming, patterns have emerged for reliable parent-child pipe communication. Here are the most important best practices:

Essential Best Practices

•Create pipes before fork() — The pipe must exist before the fork so both processes inherit it. Creating after fork means only one process has access.
•Close unused ends immediately after fork() — Both parent and child should close the end they won't use as the first operation after fork. This ensures proper EOF and SIGPIPE signaling.
•Use O_CLOEXEC when appropriate — If the child calls exec(), use pipe2() with O_CLOEXEC to prevent pipe leakage to the new program (unless you want the new program to inherit it).
•Handle SIGPIPE or check for EPIPE — Either set up a handler, ignore SIGPIPE globally, or check every write for EPIPE. Don't let unexpected termination surprise you.
•Always waitpid() for children — Even if you don't care about exit status, call waitpid() to prevent zombie accumulation. Use SIGCHLD handlers for async notification.
•Loop on partial reads/writes — Never assume read() or write() transfers the full amount. Always loop until you've transferred everything.
•Use poll()/epoll for multiple children — Blocking on one child means missing events from others. Multiplex for responsive handling.
•Flush stdio before fork() — Call fflush(NULL) before fork() to prevent duplicate output from buffered data inherited by both processes.

idiomatic_pattern.c
C
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
#include <unistd.h>
#include <stdio.h>
#include <stdlib.h>
#include <sys/wait.h>
 
/**
 * Idiomatic parent-child pipe communication template.
 * Copy and modify for your specific use case.
 */
int main(void) {
    int pipefd[2];
    pid_t pid;
    
    // 1. Flush all stdio buffers
    fflush(NULL);
    
    // 2. Create pipe BEFORE forking
    if (pipe(pipefd) == -1) {
        perror("pipe");
        exit(EXIT_FAILURE);
    }
    
    // 3. Fork
    pid = fork();
    if (pid == -1) {
        perror("fork");
        close(pipefd[0]);
        close(pipefd[1]);
        exit(EXIT_FAILURE);
    }
    
    if (pid == 0) {
        // ═══════════ CHILD ═══════════
        
        // 4. Immediately close unused end
        close(pipefd[1]);  // Child reads, close write end
        
        // 5. Do your work
        // ... read from pipefd[0] ...
        
        // 6. Close your end when done
        close(pipefd[0]);
        
        exit(EXIT_SUCCESS);
    }
    
    // ═══════════ PARENT ═══════════
    
    // 4. Immediately close unused end
    close(pipefd[0]);  // Parent writes, close read end
    
    // 5. Do your work
    // ... write to pipefd[1] ...
    
    // 6. Close your end to signal EOF to child
    close(pipefd[1]);
    
    // 7. Wait for child
    int status;
    if (waitpid(pid, &status, 0) == -1) {
        perror("waitpid");
        exit(EXIT_FAILURE);
    }
    
    // 8. Check child's exit status
    if (WIFEXITED(status)) {
        printf("Child exited with status %d\n", WEXITSTATUS(status));
    } else if (WIFSIGNALED(status)) {
        printf("Child killed by signal %d\n", WTERMSIG(status));
    }
    
    return 0;
}

Summary: Parent-Child Pipe Mastery

We've explored parent-child pipe communication in depth—the primary use case that makes anonymous pipes so powerful. Let's consolidate the key insights:

Key Takeaways

•Inheritance is the access mechanism — Anonymous pipes propagate only through fork(). The child inherits the parent's file descriptors, including pipe ends.
•dup2() enables stdin/stdout redirection — By making standard streams point to pipe ends, existing programs can participate in pipelines without modification.
•Shell pipelines are fork + pipe + redirect + exec — Understanding this sequence reveals how complex command lines become orchestrated process networks.
•The fork-exec pattern separates concerns — Fork creates the process; the window before exec allows pipe setup; exec replaces the program image.
•Multiple children require multiplexing — Use poll()/epoll to efficiently monitor many pipes without blocking on any single one.
•Robust error handling is essential — Handle SIGCHLD, SIGPIPE, child death, and partial I/O for production-grade code.
•Best practices prevent common bugs — Create before fork, close unused ends, flush buffers, waitpid for all children.

What's Next:

Now that we understand the power of pipes for communication, the final page explores their limitations. Every powerful tool has constraints, and understanding where pipes fall short guides you toward choosing the right IPC mechanism for each situation.

Page Complete

You now have deep mastery of parent-child pipe communication—inheritance mechanics, stream redirection, shell pipeline implementation, the fork-exec pattern, and multi-child coordination. This knowledge enables you to build sophisticated process pipelines and understand how shells work internally.