Consider a simple program that writes one million characters to a file, one at a time. With unbuffered I/O, this requires one million system calls—each one crossing the user-kernel boundary, saving registers, executing kernel code, and returning. This might take 15-30 seconds.

With buffered I/O, those million characters are accumulated in a user-space buffer. Only when the buffer fills (typically every 4-8KB) does a syscall occur. Instead of a million syscalls, you execute perhaps 125-250 write calls. The same operation completes in under a second—a 100x performance improvement.

This is the power of buffering—one of libc's most important contributions to application performance. Yet buffering is often misunderstood, leading to confusing bugs: output appearing in the wrong order, data lost on program crashes, or mysterious performance variations between terminal and file output.

This page provides a complete understanding of how libc buffering works, when it helps, when it hurts, and how to control it precisely.

To understand buffering, we must first understand the cost of system calls. Every syscall is expensive—far more expensive than you might expect.

The Anatomy of a System Call

When you call write(fd, buf, 1) to write a single byte:

1. User-space preparation: Load syscall number and arguments into registers
2. Trap instruction: Execute syscall (x86-64) or equivalent, switching to kernel mode
3. Mode switch overhead: CPU saves user-space state, changes privilege level
4. Kernel dispatch: Kernel looks up syscall handler, validates arguments
5. Actual work: Kernel performs the write (often very fast)
6. Return path: Switch back to user mode, restore registers
7. User-space resume: Return to caller with result

The actual "write to device" is often the smallest part. The context switching overhead dominates for small operations.

The Mathematics of Buffering

Let's do the math for writing 1 million bytes:

Without buffering (1 byte at a time):

• 1,000,000 syscalls × 300 ns/syscall = 300 ms of syscall overhead alone
• Actual disk write time adds more

With 8KB buffer:

• 1,000,000 / 8,192 ≈ 122 syscalls
• 122 syscalls × 300 ns/syscall = 0.04 ms of syscall overhead
• 7,500x reduction in syscall overhead

This isn't a micro-optimization—it's a fundamental change in program behavior.

The C standard defines three buffering modes for stdio streams. Understanding these modes is essential for predicting I/O behavior.

Mode 1: Unbuffered (_IONBF)

Unbuffered streams write data immediately to the underlying file descriptor without any user-space buffering. Every write operation results in a syscall.

// stderr is unbuffered by default
fprintf(stderr, "Error: connection failed");  // Immediately written

Characteristics:

• Output appears immediately
• No data lost on crash (already sent to kernel)
• Poor performance for frequent small writes
• Useful for error messages that must appear instantly

Default assignment: stderr is unbuffered by default on most systems.

Mode 2: Line-Buffered (_IOLBF)

Line-buffered streams accumulate data in a buffer until:

1. A newline character ( ) is written
2. The buffer becomes full
3. Input is requested from an unbuffered or line-buffered stream (causes flush)

// stdout to a terminal is typically line-buffered
printf("Processing...");    // May be buffered
printf("done!
");          // Newline triggers flush; both lines appear

Characteristics:

• Interactive programs see output at end of each line
• Good balance between performance and responsiveness
• Progress messages may not appear until newline
• Reading stdin implicitly flushes stdout for prompts

Default assignment: stdout is line-buffered when connected to a terminal.

Mode 3: Fully-Buffered (_IOFBF)

Fully-buffered streams accumulate data until:

1. The buffer becomes full (typically 4-8 KB)
2. fflush() is explicitly called
3. The stream is closed

// File streams are fully-buffered
FILE *fp = fopen("data.txt", "w");
for (int i = 0; i < 10000; i++) {
    fprintf(fp, "Line %d
", i);  // Accumulates in buffer
}
// Only ~2-3 write() syscalls for entire loop!
fclose(fp);  // Final flush on close

Characteristics:

• Maximum performance for bulk I/O
• Output may not appear for thousands of operations
• Data can be lost if program crashes before flush
• Appropriate for file I/O, not interactive use

Default assignment:

• stdout when redirected to a file or pipe
• All streams opened with fopen() (except to terminals)

One of the most confusing aspects of stdio buffering is that stdout behaves differently depending on what it's connected to. This leads to programs that work correctly in one context but mysteriously fail in another.

The Terminal vs. File/Pipe Distinction

When a program starts, libc checks whether stdout is connected to a terminal device (using the isatty() function):

• If stdout is a terminal: Line-buffered mode
• If stdout is a file or pipe: Fully-buffered mode

This heuristic makes sense:

• Terminal users want to see output interactively → line-buffered
• File/pipe output prioritizes throughput → fully-buffered

The Classic Progress Display Problem

This buffering behavior causes a notorious issue with progress displays:

#include <stdio.h>
#include <unistd.h>

int main() {
    printf("Downloading...");  // No newline!
    // ... lengthy operation ...
    sleep(5);  // Simulate download
    printf(" done!
");
    return 0;
}

Expected behavior: See "Downloading..." immediately, then " done!" after 5 seconds.

Actual behavior when piped or redirected:

• Nothing appears for 5 seconds
• Then "Downloading... done!" appears all at once

The first printf() is buffered, waiting for a newline or buffer fill.

The setvbuf() function allows explicit control over stream buffering. It must be called before any I/O operations on the stream.

int setvbuf(FILE *stream, char *buf, int mode, size_t size);

Parameters:

• stream: The FILE pointer to modify
• buf: User-provided buffer, or NULL to let libc allocate
• mode: One of _IONBF, _IOLBF, or _IOFBF
• size: Buffer size in bytes (ignored for _IONBF)

Returns: 0 on success, non-zero on failure

Alternative Functions: setbuf() and setbuffer()

Simpler interfaces exist for common cases:

// setbuf() - Simple interface (standard C)
void setbuf(FILE *stream, char *buf);
// Equivalent to:
//   setvbuf(stream, buf, buf ? _IOFBF : _IONBF, BUFSIZ);

// If buf is NULL: unbuffered
// If buf is non-NULL: fully-buffered with BUFSIZ buffer

setbuf(stdout, NULL);      // Make stdout unbuffered
char mybuf[BUFSIZ];
setbuf(stdout, mybuf);     // Make stdout fully-buffered with custom buffer

// setbuffer() - BSD extension, allows size specification
void setbuffer(FILE *stream, char *buf, size_t size);

// setlinebuf() - BSD extension, force line-buffered
void setlinebuf(FILE *stream);
// Equivalent to:
//   setvbuf(stream, NULL, _IOLBF, 0);

Flushing forces buffered data to be written to the underlying file descriptor. Understanding when flushing occurs—automatically and manually—is crucial.

Manual Flushing with fflush()

int fflush(FILE *stream);

Behavior:

• Writes all buffered output data to the kernel (via write() syscall)
• Returns 0 on success, EOF on error
• If stream is NULL, flushes all open output streams

printf("Connecting to server...");
fflush(stdout);  // Force immediate display
// ... connection code ...
printf(" connected!
");

Automatic Flushing Scenarios

libc automatically flushes buffers in several situations:

The stdin-stdout coordination:

printf("Enter your name: ");  // No newline, might be buffered
// Before scanf blocks waiting for input, stdout is flushed
char name[100];
scanf("%99s", name);  // This causes printf output to appear first

This automatic flush before input is why interactive prompts work without explicit fflush()—but it only applies when stdin and stdout are both line-buffered and connected to a terminal.

The interaction between stdio buffering and fork() is a classic source of bugs. When a process forks, the child inherits an exact copy of the parent's memory—including any unflushed stdio buffers.

The Duplicated Buffer Problem

#include <stdio.h>
#include <unistd.h>

int main() {
    printf("Hello");  // Written to buffer, NOT to stdout fd
    
    pid_t pid = fork();  // Child gets copy of buffer with "Hello"
    
    if (pid == 0) {
        printf(" from child
");  // Adds to copied buffer
    } else {
        printf(" from parent
"); // Adds to original buffer
    }
    
    return 0;  // exit() flushes buffers
}

Expected output:

Hello from parent
Hello from child

Actual output (both buffers flushed at exit):

Hello from parent
Hello from child

This looks correct, but what if the output is fully-buffered (redirected to file)? You might see:

Hello from child
Hello from parent

Or with more buffered data, the "Hello" prefix appears twice—once from each process flushing its copy of the buffer!

The Solution: Flush Before Fork

#include <stdio.h>
#include <unistd.h>

int main() {
    printf("Starting... ");
    fflush(stdout);  // ← CRITICAL: Flush before fork
    
    pid_t pid = fork();
    
    if (pid == 0) {
        printf("child
");
    } else {
        printf("parent
");
    }
    
    return 0;
}

Now both processes start with empty buffers. The "Starting... " appears exactly once.

Best Practice: Always call fflush(NULL) immediately before fork() to flush all output streams. This prevents duplicated output across process boundaries.

While output buffering gets more attention, input buffering is equally important. When you call fread() or fgetc(), libc may read more data than requested into an internal buffer, serving subsequent reads from that buffer.

How Input Buffering Works

// You request 1 character:
int ch = fgetc(fp);

// Internally, libc might do:
// 1. Check if data exists in input buffer
// 2. If buffer empty: read(fd, buffer, BUFSIZ) - read 4-8KB!
// 3. Return one character from buffer
// 4. Next fgetc() reads from buffer, no syscall needed

This means a file read of 1 million characters might result in only ~125 read() syscalls (at 8KB each), not a million.

The Problem: Mixing stdio and Raw I/O

Input buffering creates issues when you mix stdio (FILE*) functions with raw read() syscalls on the same file descriptor:

#include <stdio.h>
#include <unistd.h>

int main() {
    // stdin is FILE* with buffering
    char line[100];
    fgets(line, sizeof(line), stdin);  // Reads buffered
    printf("fgets got: %s", line);
    
    // PROBLEM: Direct read on same fd
    char buf[100];
    ssize_t n = read(STDIN_FILENO, buf, sizeof(buf));  // Raw read
    // This might return nothing because fgets() already read-ahead!
    buf[n] = '\0';
    printf("read() got: %s
", buf);
    
    return 0;
}

What happens:

1. fgets() needs to find a newline
2. It calls read(0, buffer, BUFSIZ) - reads perhaps 1000 bytes
3. It returns the first line, keeping remaining bytes in stdio buffer
4. You call read(0, ...) directly on fd 0
5. That data is already consumed by stdio! read() sees different data or blocks

Rule: Never mix stdio functions and raw I/O on the same underlying file descriptor.

Buffering is one of libc's most impactful features—providing 100x performance improvements while introducing subtle behavioral complexities. Understanding buffering is essential for debugging I/O issues and writing correct, efficient programs.

What's Next:

Now that we understand buffering, we'll explore the performance considerations that determine when to use library functions versus direct system calls. The next page examines the trade-offs between convenience and raw performance, and how to make informed decisions for different application requirements.