System Call Types - Learning Module

Loading content...

0/240

File Management System Calls

Files as the Universal Abstraction

In the UNIX philosophy, everything is a file. This deceptively simple design principle has profound implications: regular files, directories, devices, pipes, network sockets, and even process information are accessed through a unified interface—file descriptors and file operations. This abstraction creates both power and responsibility.

File management system calls represent the second major category in our taxonomy, responsible for mediating every interaction between user programs and persistent storage. Every database write, every log entry, every configuration read, every saved document—all flow through these fundamental operations. Understanding file management system calls means understanding how programs interact with the most essential abstraction in computing: persistent, named data.

These system calls must balance competing concerns: performance (minimizing context switches and copying), safety (preventing data corruption), security (enforcing access permissions), and correctness (maintaining file system invariants). The design decisions embedded in these interfaces shape everything from how we structure our programs to how entire storage systems are architected.

What You Will Learn

By the end of this page, you will understand the complete lifecycle of file operations: how files are opened and file descriptors allocated, how data is read and written at the byte level, how the file position is managed, how files are created and removed, and how file metadata is accessed and modified. You'll grasp the semantics that make reliable I/O programming possible.

Opening Files — The Gateway to I/O

Before any read, write, or manipulation, a file must be opened. The open() system call (and its variants) establishes the connection between a pathname in the filesystem and a file descriptor in the process—an integer handle that identifies the open file for all subsequent operations.

The open() System Call

int open(const char *pathname, int flags);
int open(const char *pathname, int flags, mode_t mode);

The call returns a file descriptor (non-negative integer) on success or -1 on error. The three-argument form is required when creating files to specify permissions.

File Descriptor Semantics

File descriptors are per-process integers that index into the process's file descriptor table. Several critical semantics govern their behavior:

Lowest available: open() always returns the lowest-numbered available descriptor
Three standard descriptors: 0 (stdin), 1 (stdout), 2 (stderr) are pre-opened
Shared with children: Child processes inherit file descriptors across fork()
Reference counted: Multiple descriptors can reference the same underlying file
Per-process: The same descriptor number in different processes refers to different things

Essential open() Flags
Flag	Purpose	Required Context
O_RDONLY	Open for reading only	Read access needed
O_WRONLY	Open for writing only	Write access needed
O_RDWR	Open for reading and writing	Both accesses needed
O_CREAT	Create file if it doesn't exist	Requires mode argument
O_EXCL	Fail if O_CREAT and file exists	Atomic file creation
O_TRUNC	Truncate existing file to zero length	Writing existing file fresh
O_APPEND	Writes append to end atomically	Log files, concurrent writes
O_CLOEXEC	Close descriptor on exec()	Security, leak prevention
O_SYNC	Synchronous writes (wait for disk)	Durability requirements
O_NONBLOCK	Non-blocking I/O mode	Async I/O patterns

open_examples.c
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
#include <stdio.h>
#include <fcntl.h>      // open(), O_* flags
#include <unistd.h>     // close()
#include <sys/stat.h>   // mode constants
#include <errno.h>
#include <string.h>
 
void demonstrate_open_patterns() {
    int fd;
    
    // ========================================
    // Pattern 1: Open existing file for reading
    // ========================================
    fd = open("/etc/passwd", O_RDONLY);
    if (fd == -1) {
        // Always check for errors!
        fprintf(stderr, "open /etc/passwd failed: %s
", 
                strerror(errno));
    } else {
        printf("Opened /etc/passwd as fd %d
", fd);
        close(fd);
    }
    
    // ========================================
    // Pattern 2: Create new file (fails if exists)
    // ========================================
    // Mode: 0644 = rw-r--r-- (owner read/write, group/other read)
    fd = open("new_file.txt", O_WRONLY | O_CREAT | O_EXCL, 0644);
    if (fd == -1) {
        if (errno == EEXIST) {
            printf("new_file.txt already exists
");
        } else {
            fprintf(stderr, "create failed: %s
", strerror(errno));
        }
    } else {
        printf("Created new_file.txt as fd %d
", fd);
        close(fd);
        unlink("new_file.txt");  // Clean up for demo
    }
    
    // ========================================
    // Pattern 3: Create or truncate (typical overwrite)
    // ========================================
    fd = open("output.log", 
              O_WRONLY | O_CREAT | O_TRUNC | O_CLOEXEC, 
              0600);  // 0600 = rw------- (private)
    if (fd == -1) {
        perror("open output.log");
    } else {
        printf("Ready to write output.log (fd %d)
", fd);
        // O_CLOEXEC ensures this fd won't leak to child processes
        close(fd);
        unlink("output.log");
    }
    
    // ========================================
    // Pattern 4: Append mode for log files
    // ========================================
    fd = open("app.log", 
              O_WRONLY | O_CREAT | O_APPEND, 
              0644);
    if (fd != -1) {
        // Each write() atomically appends to end of file
        // Safe for multiple processes writing to same log
        const char *msg = "Log entry
";
        write(fd, msg, strlen(msg));
        close(fd);
        unlink("app.log");
    }
    
    // ========================================
    // Pattern 5: Read-write access
    // ========================================
    // Create a file we can both read and write
    fd = open("data.bin", O_RDWR | O_CREAT | O_TRUNC, 0600);
    if (fd != -1) {
        // Can both read and write, seek back and forth
        write(fd, "Hello", 5);
        lseek(fd, 0, SEEK_SET);  // Rewind
        char buf[10];
        read(fd, buf, 5);
        buf[5] = '\0';
        printf("Read back: %s
", buf);
        close(fd);
        unlink("data.bin");
    }
}
 
int main() {
    demonstrate_open_patterns();
    return 0;
}

Race Condition Warning

Never check if a file exists and then open it in separate operations—this creates a TOCTOU (time-of-check-to-time-of-use) race condition where another process could create/delete the file between your check and open. Use O_CREAT | O_EXCL to atomically create and fail if exists, or simply attempt the open and handle errors appropriately.

openat() and Directory File Descriptors

Modern systems provide openat() which opens relative to a directory file descriptor:

int openat(int dirfd, const char *pathname, int flags);
int openat(int dirfd, const char *pathname, int flags, mode_t mode);

This addresses several problems:

Race conditions: Directory can't be renamed/moved while we hold the fd
Long paths: Avoid repeated directory traversal
Security: Confine operations to a subtree
Thread safety: Each thread can have its own 'current directory'

The special value AT_FDCWD makes openat() behave like open() using the process's current working directory.

Reading Data — From Kernel to User Space

The read() system call transfers data from an open file descriptor into user-space memory. Despite its apparent simplicity, read() embodies subtle semantics that are essential for correct I/O programming.

ssize_t read(int fd, void *buf, size_t count);

Return Value Semantics

The return value of read() requires careful interpretation:

Positive value: Number of bytes actually read (may be less than requested)
Zero: End-of-file reached (no more data available)
-1: Error occurred (check errno)

The fact that read() may return fewer bytes than requested is fundamental—not an error. This occurs when:

Fewer bytes are available than requested
Reading from a terminal or pipe (line-at-a-time or available data)
Interrupted by a signal (depending on SA_RESTART)
Approaching end of file

read_examples.c
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
#include <stdio.h>
#include <fcntl.h>
#include <unistd.h>
#include <errno.h>
#include <string.h>
 
// Correct pattern: read exactly n bytes (or until EOF)
ssize_t read_exact(int fd, void *buf, size_t count) {
    size_t total = 0;
    char *ptr = buf;
    
    while (total < count) {
        ssize_t n = read(fd, ptr + total, count - total);
        
        if (n == 0) {
            // EOF reached before reading all bytes
            break;
        }
        
        if (n == -1) {
            if (errno == EINTR) {
                // Interrupted by signal, retry
                continue;
            }
            // Real error
            return -1;
        }
        
        total += n;
    }
    
    return total;
}
 
// Read entire file into memory (for small files)
char *read_entire_file(const char *path, size_t *out_size) {
    int fd = open(path, O_RDONLY);
    if (fd == -1) return NULL;
    
    // Get file size
    off_t size = lseek(fd, 0, SEEK_END);
    if (size == -1) {
        close(fd);
        return NULL;
    }
    lseek(fd, 0, SEEK_SET);  // Rewind
    
    // Allocate buffer
    char *buffer = malloc(size + 1);  // +1 for null terminator
    if (!buffer) {
        close(fd);
        return NULL;
    }
    
    // Read entire file
    ssize_t bytes_read = read_exact(fd, buffer, size);
    close(fd);
    
    if (bytes_read == -1) {
        free(buffer);
        return NULL;
    }
    
    buffer[bytes_read] = '\0';
    if (out_size) *out_size = bytes_read;
    
    return buffer;
}
 
// Buffered reading with explicit buffer management
void demonstrate_buffered_reading() {
    int fd = open("/etc/passwd", O_RDONLY);
    if (fd == -1) {
        perror("open");
        return;
    }
    
    // Use a reasonable buffer size (often 4KB, 8KB, or page-aligned)
    char buffer[8192];
    ssize_t bytes_read;
    size_t total = 0;
    
    // Read in chunks until EOF
    while ((bytes_read = read(fd, buffer, sizeof(buffer))) > 0) {
        // Process chunk...
        total += bytes_read;
        printf("Read chunk of %zd bytes (total: %zu)
", 
               bytes_read, total);
    }
    
    if (bytes_read == -1) {
        perror("read");
    } else {
        printf("Finished reading. Total: %zu bytes
", total);
    }
    
    close(fd);
}

Buffer Size Selection

Buffer size significantly impacts performance. Too small means too many system calls (context switch overhead). Too large wastes memory and may exceed cache size. Common choices: 4KB (page size on many systems), 8KB (common block size), or BUFSIZ (defined in stdio.h, typically 8KB). For maximum throughput, align buffers to page boundaries and use sizes that are multiples of the underlying block size.

pread() — Positional Read Without Seeking

For multi-threaded programs or situations requiring atomic read-at-offset semantics:

ssize_t pread(int fd, void *buf, size_t count, off_t offset);

Advantages of pread():

Thread-safe: Multiple threads can read different parts of the same file simultaneously without interfering with each other's file position
Atomic: The seek and read are combined into one kernel operation
Efficiency: One system call instead of two (lseek + read)

Note that pread() does not modify the file offset—subsequent regular reads still occur at the previous position.

readv() — Scatter Read

For reading into multiple non-contiguous buffers:

ssize_t readv(int fd, const struct iovec *iov, int iovcnt);

This is useful when data has structure (e.g., header in one buffer, payload in another) and can be more efficient than multiple read() calls or post-read copying.

Writing Data — From User Space to Persistence

The write() system call transfers data from user-space memory to an open file descriptor. While conceptually symmetric with read(), write has its own critical semantics, particularly regarding durability and atomicity.

ssize_t write(int fd, const void *buf, size_t count);

Return Value Semantics

Positive value: Number of bytes actually written
Zero: Theoretically possible but practically rare for regular files
-1: Error occurred (check errno)

Like read(), write() may write fewer bytes than requested. This occurs for:

Writing to pipes/sockets with limited buffer space
Signals interrupting the call
Approaching storage limits

write_examples.c
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
#include <stdio.h>
#include <fcntl.h>
#include <unistd.h>
#include <errno.h>
#include <string.h>
 
// Correct pattern: write exactly n bytes
ssize_t write_exact(int fd, const void *buf, size_t count) {
    size_t total = 0;
    const char *ptr = buf;
    
    while (total < count) {
        ssize_t n = write(fd, ptr + total, count - total);
        
        if (n == -1) {
            if (errno == EINTR) {
                // Interrupted by signal, retry
                continue;
            }
            // Real error
            return -1;
        }
        
        // Note: n == 0 could indicate a problem, but is rare
        
        total += n;
    }
    
    return total;
}
 
// Write with durability guarantees
int write_durable(int fd, const void *buf, size_t count) {
    // Write all data
    if (write_exact(fd, buf, count) != (ssize_t)count) {
        return -1;
    }
    
    // fdatasync: ensure data (but not necessarily metadata) on disk
    // Use fsync() if metadata (size, timestamps) must also be durable
    if (fdatasync(fd) == -1) {
        return -1;
    }
    
    return 0;
}
 
// Atomic append pattern for log files
void append_to_log(const char *path, const char *message) {
    // O_APPEND makes writes atomic at end of file
    int fd = open(path, O_WRONLY | O_APPEND | O_CREAT, 0644);
    if (fd == -1) {
        perror("open log");
        return;
    }
    
    // Even with multiple processes writing, entries won't interleave
    // (up to PIPE_BUF bytes, or filesystem-specific limit)
    size_t len = strlen(message);
    if (write(fd, message, len) != (ssize_t)len) {
        perror("write log");
    }
    
    close(fd);
}
 
// Write-then-rename pattern for crash safety
int safe_write_file(const char *path, const void *data, size_t size) {
    char tmp_path[256];
    snprintf(tmp_path, sizeof(tmp_path), "%s.tmp", path);
    
    // Write to temporary file
    int fd = open(tmp_path, O_WRONLY | O_CREAT | O_TRUNC, 0644);
    if (fd == -1) return -1;
    
    ssize_t written = write_exact(fd, data, size);
    
    // Sync before rename to ensure data is durable
    if (fsync(fd) == -1) {
        close(fd);
        unlink(tmp_path);
        return -1;
    }
    
    close(fd);
    
    if (written != (ssize_t)size) {
        unlink(tmp_path);
        return -1;
    }
    
    // Atomic rename: either old content or new, never partial
    if (rename(tmp_path, path) == -1) {
        unlink(tmp_path);
        return -1;
    }
    
    return 0;
}

Durability vs. Performance

write() returning success does NOT mean data is on disk! Data may be in kernel buffer cache awaiting writeback. Use fsync(fd) to wait for data AND metadata, fdatasync(fd) for data only (faster), or open with O_SYNC for synchronous writes. However, these significantly impact performance—fsync can be 100-1000x slower than buffered writes. Design your durability requirements carefully.

pwrite() and writev() — Positional and Scatter-Gather Writes

Mirroring the read variants:

pwrite() writes at a specific offset without modifying file position:

ssize_t pwrite(int fd, const void *buf, size_t count, off_t offset);

writev() writes from multiple buffers (gather write):

ssize_t writev(int fd, const struct iovec *iov, int iovcnt);

The O_APPEND Guarantee

When a file is opened with O_APPEND, every write atomically:

Seeks to the end of file
Writes the data

This atomicity is crucial for multi-process logging. Without O_APPEND, two processes might both seek to the end, then both write—overwriting each other. With O_APPEND, writes are serialized correctly.

Note: Atomicity for writes is typically guaranteed only up to PIPE_BUF bytes (usually 4096) for pipes, and varies for regular files depending on the filesystem.

Durability Levels and Their Costs
Method	Data in Kernel Buffer	Data on Disk	Metadata on Disk	Typical Latency
write() alone	✓	Eventually	Eventually	~1μs
write() + fdatasync()	✓	✓	Partially	~1-5ms
write() + fsync()	✓	✓	✓	~3-10ms
O_SYNC per write	✓	✓	✓	~3-10ms per write
O_DIRECT (bypass cache)	N/A	✓	No	~0.1-1ms

Seeking — Navigating File Positions

Every open file descriptor maintains a file offset (or file position)—the location where the next read or write will occur. The lseek() system call allows explicit manipulation of this position.

off_t lseek(int fd, off_t offset, int whence);

The whence Parameter

SEEK_SET: Set position to offset bytes from beginning of file
SEEK_CUR: Set position to current location plus offset (can be negative)
SEEK_END: Set position to file size plus offset (often negative to seek backward)

Return Value

Returns the new offset position (from beginning), or -1 on error. A common idiom to get current position:

off_t current = lseek(fd, 0, SEEK_CUR);  // No movement, just query

seek_examples.c
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
#include <stdio.h>
#include <fcntl.h>
#include <unistd.h>
#include <sys/stat.h>
 
void demonstrate_seeking() {
    // Create a test file
    int fd = open("test.dat", O_RDWR | O_CREAT | O_TRUNC, 0644);
    if (fd == -1) {
        perror("open");
        return;
    }
    
    // Write some data
    const char *message = "Hello, World!";  // 13 bytes
    write(fd, message, 13);
    
    printf("After write, position: %lld
", 
           (long long)lseek(fd, 0, SEEK_CUR));
    
    // ========================================
    // SEEK_SET: Absolute positioning
    // ========================================
    lseek(fd, 0, SEEK_SET);  // Beginning of file
    printf("After SEEK_SET 0: %lld
", 
           (long long)lseek(fd, 0, SEEK_CUR));
    
    // Read and verify
    char buf[20];
    read(fd, buf, 5);
    buf[5] = '\0';
    printf("First 5 bytes: '%s'
", buf);  // "Hello"
    
    // ========================================
    // SEEK_CUR: Relative positioning
    // ========================================
    lseek(fd, 2, SEEK_CUR);  // Skip 2 bytes forward
    read(fd, buf, 5);
    buf[5] = '\0';
    printf("After skipping 2: '%s'
", buf);  // "World"
    
    lseek(fd, -5, SEEK_CUR);  // Go back 5 bytes
    read(fd, buf, 5);
    buf[5] = '\0';
    printf("After -5: '%s'
", buf);  // "World"
    
    // ========================================
    // SEEK_END: From end of file
    // ========================================
    lseek(fd, -6, SEEK_END);  // 6 bytes before end
    read(fd, buf, 6);
    buf[6] = '\0';
    printf("Last 6 bytes: '%s'
", buf);  // "World!"
    
    // ========================================
    // Get file size using seek
    // ========================================
    off_t size = lseek(fd, 0, SEEK_END);
    printf("File size: %lld bytes
", (long long)size);
    
    // ========================================
    // Creating sparse file with seek past end
    // ========================================
    lseek(fd, 1000000, SEEK_SET);  // Seek beyond file size
    write(fd, "X", 1);  // Write one byte at position 1000000
    
    struct stat st;
    fstat(fd, &st);
    printf("After sparse write:
");
    printf("  Logical size: %lld bytes
", (long long)st.st_size);
    printf("  Blocks used: %lld (x512 = %lld bytes)
", 
           (long long)st.st_blocks,
           (long long)st.st_blocks * 512);
    // Note: much less disk space actually used!
    
    close(fd);
    unlink("test.dat");
}
 
int main() {
    demonstrate_seeking();
    return 0;
}

Sparse Files

When you seek past the end of file and write, a sparse file is created. The 'hole' doesn't consume disk blocks—only the actual data regions. Reading from holes returns zeros. This is useful for virtual machine disk images and databases. Use lseek(..., SEEK_HOLE) and SEEK_DATA (Linux-specific) to detect and traverse sparse regions efficiently.

File Offset Sharing Semantics

Understanding how file offsets are shared (or not) is crucial:

After fork(): Parent and child share the same file offset (changes in one affect the other). This is because they share the underlying kernel file description.
After separate open(): Two opens of the same file create independent offsets. Each process has its own file description.
After dup()/dup2(): The new descriptor shares the offset with the original (same file description).
Threads in same process: Naturally share file descriptors and offsets (unless using thread-local descriptors).

This sharing behavior explains when pread()/pwrite() are necessary for thread safety, and why forked children's writes can unexpectedly interleave.

File Offset Sharing After Various Operations
Operation	Offset Relationship	Implication
fork()	Shared (same file description)	Concurrent access needs coordination
open() same file	Independent	Each process has own view
dup() / dup2()	Shared (same file description)	Used for I/O redirection
O_APPEND flag	Seek before each write	Atomically appends regardless of shared offset

Closing Files — Resource Release and Guarantees

The close() system call releases a file descriptor, freeing the kernel resources associated with the open file. While seemingly trivial, close() carries subtle semantics that matter for correctness.

int close(int fd);

What close() Does

Releases file descriptor: The integer becomes available for reuse
Decrements reference count: On the underlying kernel file description
Triggers writeback: Any unwritten cached data may be scheduled for writing
Releases locks: Any file locks held by this process through this fd
Potential inode release: If this was the last reference to an unlinked file, free space

What close() Does NOT Guarantee

Critically, close() returning success does not mean data is on disk. Buffered writes may still be pending. If durability is required, call fsync() or fdatasync() before closing.

close_examples.c
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
#include <stdio.h>
#include <fcntl.h>
#include <unistd.h>
#include <errno.h>
 
// Proper error handling on close
int safe_close(int fd) {
    // close() can fail!
    // For non-network filesystems, failure is rare but possible
    // For NFS, close() can report delayed write errors
    if (close(fd) == -1) {
        // Log or handle error
        // Note: fd is still closed, don't retry close()!
        return -1;
    }
    return 0;
}
 
// Pattern: close with durability requirement
int close_durable(int fd, int require_sync) {
    int result = 0;
    
    if (require_sync) {
        // Ensure data is on disk before closing
        if (fsync(fd) == -1) {
            // Save errno before close potentially changes it
            int saved_errno = errno;
            close(fd);  // Still close, but report the sync error
            errno = saved_errno;
            return -1;
        }
    }
    
    if (close(fd) == -1) {
        return -1;
    }
    
    return 0;
}
 
// Demonstrating reference counting and close
void demonstrate_close_semantics() {
    // Create a file
    int fd1 = open("demo.txt", O_RDWR | O_CREAT | O_TRUNC, 0600);
    write(fd1, "Hello", 5);
    
    // dup() creates second reference to same file description
    int fd2 = dup(fd1);
    
    printf("fd1=%d, fd2=%d (both point to same file)
", fd1, fd2);
    
    // Close one - file stays open via the other
    close(fd1);
    
    // Can still read/write through fd2
    lseek(fd2, 0, SEEK_SET);
    char buf[10];
    read(fd2, buf, 5);
    buf[5] = '\0';
    printf("After closing fd1, read via fd2: %s
", buf);
    
    // Unlink while file is open - file stays accessible
    unlink("demo.txt");
    // File doesn't appear in directory anymore, but...
    
    write(fd2, " World", 6);
    lseek(fd2, 0, SEEK_SET);
    read(fd2, buf, 11);
    buf[11] = '\0';
    printf("After unlink, still accessible: %s
", buf);
    
    // When fd2 is closed, file's space is finally freed
    close(fd2);
    printf("File space freed after final close
");
}
 
// RAII-style pattern for C (using GCC attribute)
#ifdef __GNUC__
static void auto_close(int *fd) {
    if (*fd != -1) {
        close(*fd);
    }
}
#define AUTO_CLOSE __attribute__((cleanup(auto_close)))
 
void raii_style_example() {
    // fd automatically closed when function returns
    AUTO_CLOSE int fd = open("/etc/passwd", O_RDONLY);
    if (fd == -1) return;
    
    char buf[100];
    read(fd, buf, sizeof(buf));
    // No need for explicit close() - cleanup attribute handles it
}
#endif

Double-Close Bug

Closing a file descriptor that's already closed, or closing an invalid descriptor, is undefined behavior in practice (though close() will return EBADF). Worse, if another thread/code path has opened a new file between your first close and second close, you'll close the wrong file! Always set fd to -1 after closing or use a wrapper that tracks state.

File Metadata — stat, chmod, and Ownership

Beyond reading and writing file content, programs often need to examine and modify file metadata: size, permissions, ownership, timestamps, and other attributes. Several system calls provide this functionality.

stat() Family — Retrieving File Information

int stat(const char *pathname, struct stat *statbuf);
int fstat(int fd, struct stat *statbuf);  // By file descriptor
int lstat(const char *pathname, struct stat *statbuf);  // Don't follow symlinks

The stat structure contains rich information:

Key Fields in struct stat
Field	Type	Description
st_mode	mode_t	File type (reg/dir/symlink/etc) and permissions
st_ino	ino_t	Inode number (unique within filesystem)
st_dev	dev_t	Device ID of filesystem
st_nlink	nlink_t	Number of hard links
st_uid / st_gid	uid_t / gid_t	Owner user ID and group ID
st_size	off_t	File size in bytes
st_blocks	blkcnt_t	Number of 512-byte blocks allocated
st_atime	time_t	Last access time
st_mtime	time_t	Last modification time (content)
st_ctime	time_t	Last status change time (metadata)

stat_examples.c
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
#include <stdio.h>
#include <sys/stat.h>
#include <time.h>
#include <pwd.h>     // getpwuid
#include <grp.h>     // getgrgid
#include <unistd.h>
 
void print_file_info(const char *path) {
    struct stat st;
    
    if (stat(path, &st) == -1) {
        perror("stat");
        return;
    }
    
    printf("File: %s
", path);
    
    // File type
    printf("Type: ");
    switch (st.st_mode & S_IFMT) {
        case S_IFREG:  printf("regular file
"); break;
        case S_IFDIR:  printf("directory
"); break;
        case S_IFLNK:  printf("symbolic link
"); break;
        case S_IFCHR:  printf("character device
"); break;
        case S_IFBLK:  printf("block device
"); break;
        case S_IFIFO:  printf("FIFO/pipe
"); break;
        case S_IFSOCK: printf("socket
"); break;
        default:       printf("unknown
");
    }
    
    // Permissions in octal and symbolic
    printf("Mode: %04o (", st.st_mode & 0777);
    printf("%c", (st.st_mode & S_IRUSR) ? 'r' : '-');
    printf("%c", (st.st_mode & S_IWUSR) ? 'w' : '-');
    printf("%c", (st.st_mode & S_IXUSR) ? 'x' : '-');
    printf("%c", (st.st_mode & S_IRGRP) ? 'r' : '-');
    printf("%c", (st.st_mode & S_IWGRP) ? 'w' : '-');
    printf("%c", (st.st_mode & S_IXGRP) ? 'x' : '-');
    printf("%c", (st.st_mode & S_IROTH) ? 'r' : '-');
    printf("%c", (st.st_mode & S_IWOTH) ? 'w' : '-');
    printf("%c", (st.st_mode & S_IXOTH) ? 'x' : '-');
    printf(")
");
    
    // Owner info
    struct passwd *pw = getpwuid(st.st_uid);
    struct group *gr = getgrgid(st.st_gid);
    printf("Owner: %s (%d)
", pw ? pw->pw_name : "unknown", st.st_uid);
    printf("Group: %s (%d)
", gr ? gr->gr_name : "unknown", st.st_gid);
    
    // Size and blocks
    printf("Size: %lld bytes
", (long long)st.st_size);
    printf("Blocks: %lld (actual disk: %lld bytes)
", 
           (long long)st.st_blocks, (long long)st.st_blocks * 512);
    
    // Hard link count
    printf("Hard links: %d
", (int)st.st_nlink);
    
    // Inode
    printf("Inode: %llu
", (unsigned long long)st.st_ino);
    
    // Timestamps
    printf("Access time: %s", ctime(&st.st_atime));
    printf("Modify time: %s", ctime(&st.st_mtime));
    printf("Change time: %s", ctime(&st.st_ctime));
}
 
// Check specific conditions
int is_regular_file(const char *path) {
    struct stat st;
    return (stat(path, &st) == 0 && S_ISREG(st.st_mode));
}
 
int is_directory(const char *path) {
    struct stat st;
    return (stat(path, &st) == 0 && S_ISDIR(st.st_mode));
}
 
int is_file_newer(const char *file1, const char *file2) {
    struct stat st1, st2;
    if (stat(file1, &st1) == -1 || stat(file2, &st2) == -1) {
        return -1;  // Error
    }
    return st1.st_mtime > st2.st_mtime;
}

Modifying Metadata

Several system calls modify file attributes:

chmod() / fchmod() — Change permissions:

int chmod(const char *pathname, mode_t mode);
int fchmod(int fd, mode_t mode);

chown() / fchown() / lchown() — Change ownership:

int chown(const char *pathname, uid_t owner, gid_t group);
int fchown(int fd, uid_t owner, gid_t group);
int lchown(const char *pathname, uid_t owner, gid_t group);  // Symlink itself

utimensat() / futimens() — Change timestamps:

int utimensat(int dirfd, const char *pathname, 
              const struct timespec times[2], int flags);

truncate() / ftruncate() — Change file size:

int truncate(const char *path, off_t length);
int ftruncate(int fd, off_t length);

Security Implications

chmod() and chown() have significant security implications. Only the file owner (or root) can change permissions. Only root can change ownership arbitrarily. The setuid/setgid bits (S_ISUID/S_ISGID) allow executable files to run with the file owner's privileges—a powerful but dangerous feature. Many systems clear these bits when files are written to prevent privilege escalation attacks.

Summary: File Management System Calls

File management system calls mediate every interaction between programs and persistent storage. We've explored the complete landscape:

Key Takeaways

•open() creates the connection between pathnames and file descriptors, with flags controlling access mode, creation behavior, and performance characteristics. The openat() variant enables race-free operations relative to directory file descriptors.
•read() transfers data from kernel to user space, with return values that demand careful handling—partial reads are normal, not errors. Variants like pread() and readv() address thread-safety and efficiency concerns.
•write() transfers data from user space toward persistent storage, but buffering means success doesn't imply durability. fsync() and O_SYNC provide guarantees when needed, at significant performance cost.
•lseek() navigates file positions, enabling random access. Understanding offset sharing semantics across fork() and dup() is crucial for concurrent access patterns.
•close() releases resources but requires checking for errors, especially on network filesystems. Reference counting means files persist until all descriptors and links are released.
•stat() family retrieves comprehensive file metadata, while chmod(), chown(), and truncate() modify it. These operations form the basis of file system utilities and security mechanisms.

Looking Ahead

With process control and file management mastered, we turn to device management—how operating systems abstract hardware through system calls. Then we'll examine information maintenance and communication system calls, completing our taxonomy of the interfaces through which programs access operating system services.

Page Complete

You now understand the file management system calls that underpin all persistent I/O in UNIX-like systems. From opening files through closing them, from reading bytes to modifying metadata, these primitives form the vocabulary through which programs interact with storage. Next, we'll explore how similar abstractions apply to devices and hardware resources.

File Management System Calls

Files as the Universal Abstraction

What You Will Learn

Opening Files — The Gateway to I/O

The open() System Call

int open(const char *pathname, int flags);
int open(const char *pathname, int flags, mode_t mode);

The call returns a file descriptor (non-negative integer) on success or -1 on error. The three-argument form is required when creating files to specify permissions.

File Descriptor Semantics

File descriptors are per-process integers that index into the process's file descriptor table. Several critical semantics govern their behavior:

Lowest available: open() always returns the lowest-numbered available descriptor
Three standard descriptors: 0 (stdin), 1 (stdout), 2 (stderr) are pre-opened
Shared with children: Child processes inherit file descriptors across fork()
Reference counted: Multiple descriptors can reference the same underlying file
Per-process: The same descriptor number in different processes refers to different things

Essential open() Flags
Flag	Purpose	Required Context
O_RDONLY	Open for reading only	Read access needed
O_WRONLY	Open for writing only	Write access needed
O_RDWR	Open for reading and writing	Both accesses needed
O_CREAT	Create file if it doesn't exist	Requires mode argument
O_EXCL	Fail if O_CREAT and file exists	Atomic file creation
O_TRUNC	Truncate existing file to zero length	Writing existing file fresh
O_APPEND	Writes append to end atomically	Log files, concurrent writes
O_CLOEXEC	Close descriptor on exec()	Security, leak prevention
O_SYNC	Synchronous writes (wait for disk)	Durability requirements
O_NONBLOCK	Non-blocking I/O mode	Async I/O patterns

open_examples.c
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
#include <stdio.h>
#include <fcntl.h>      // open(), O_* flags
#include <unistd.h>     // close()
#include <sys/stat.h>   // mode constants
#include <errno.h>
#include <string.h>
 
void demonstrate_open_patterns() {
    int fd;
    
    // ========================================
    // Pattern 1: Open existing file for reading
    // ========================================
    fd = open("/etc/passwd", O_RDONLY);
    if (fd == -1) {
        // Always check for errors!
        fprintf(stderr, "open /etc/passwd failed: %s
", 
                strerror(errno));
    } else {
        printf("Opened /etc/passwd as fd %d
", fd);
        close(fd);
    }
    
    // ========================================
    // Pattern 2: Create new file (fails if exists)
    // ========================================
    // Mode: 0644 = rw-r--r-- (owner read/write, group/other read)
    fd = open("new_file.txt", O_WRONLY | O_CREAT | O_EXCL, 0644);
    if (fd == -1) {
        if (errno == EEXIST) {
            printf("new_file.txt already exists
");
        } else {
            fprintf(stderr, "create failed: %s
", strerror(errno));
        }
    } else {
        printf("Created new_file.txt as fd %d
", fd);
        close(fd);
        unlink("new_file.txt");  // Clean up for demo
    }
    
    // ========================================
    // Pattern 3: Create or truncate (typical overwrite)
    // ========================================
    fd = open("output.log", 
              O_WRONLY | O_CREAT | O_TRUNC | O_CLOEXEC, 
              0600);  // 0600 = rw------- (private)
    if (fd == -1) {
        perror("open output.log");
    } else {
        printf("Ready to write output.log (fd %d)
", fd);
        // O_CLOEXEC ensures this fd won't leak to child processes
        close(fd);
        unlink("output.log");
    }
    
    // ========================================
    // Pattern 4: Append mode for log files
    // ========================================
    fd = open("app.log", 
              O_WRONLY | O_CREAT | O_APPEND, 
              0644);
    if (fd != -1) {
        // Each write() atomically appends to end of file
        // Safe for multiple processes writing to same log
        const char *msg = "Log entry
";
        write(fd, msg, strlen(msg));
        close(fd);
        unlink("app.log");
    }
    
    // ========================================
    // Pattern 5: Read-write access
    // ========================================
    // Create a file we can both read and write
    fd = open("data.bin", O_RDWR | O_CREAT | O_TRUNC, 0600);
    if (fd != -1) {
        // Can both read and write, seek back and forth
        write(fd, "Hello", 5);
        lseek(fd, 0, SEEK_SET);  // Rewind
        char buf[10];
        read(fd, buf, 5);
        buf[5] = '\0';
        printf("Read back: %s
", buf);
        close(fd);
        unlink("data.bin");
    }
}
 
int main() {
    demonstrate_open_patterns();
    return 0;
}

Race Condition Warning

openat() and Directory File Descriptors

Modern systems provide openat() which opens relative to a directory file descriptor:

int openat(int dirfd, const char *pathname, int flags);
int openat(int dirfd, const char *pathname, int flags, mode_t mode);

This addresses several problems:

Race conditions: Directory can't be renamed/moved while we hold the fd
Long paths: Avoid repeated directory traversal
Security: Confine operations to a subtree
Thread safety: Each thread can have its own 'current directory'

The special value AT_FDCWD makes openat() behave like open() using the process's current working directory.

Reading Data — From Kernel to User Space

ssize_t read(int fd, void *buf, size_t count);

Return Value Semantics

The return value of read() requires careful interpretation:

Positive value: Number of bytes actually read (may be less than requested)
Zero: End-of-file reached (no more data available)
-1: Error occurred (check errno)

The fact that read() may return fewer bytes than requested is fundamental—not an error. This occurs when:

Fewer bytes are available than requested
Reading from a terminal or pipe (line-at-a-time or available data)
Interrupted by a signal (depending on SA_RESTART)
Approaching end of file

read_examples.c
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
#include <stdio.h>
#include <fcntl.h>
#include <unistd.h>
#include <errno.h>
#include <string.h>
 
// Correct pattern: read exactly n bytes (or until EOF)
ssize_t read_exact(int fd, void *buf, size_t count) {
    size_t total = 0;
    char *ptr = buf;
    
    while (total < count) {
        ssize_t n = read(fd, ptr + total, count - total);
        
        if (n == 0) {
            // EOF reached before reading all bytes
            break;
        }
        
        if (n == -1) {
            if (errno == EINTR) {
                // Interrupted by signal, retry
                continue;
            }
            // Real error
            return -1;
        }
        
        total += n;
    }
    
    return total;
}
 
// Read entire file into memory (for small files)
char *read_entire_file(const char *path, size_t *out_size) {
    int fd = open(path, O_RDONLY);
    if (fd == -1) return NULL;
    
    // Get file size
    off_t size = lseek(fd, 0, SEEK_END);
    if (size == -1) {
        close(fd);
        return NULL;
    }
    lseek(fd, 0, SEEK_SET);  // Rewind
    
    // Allocate buffer
    char *buffer = malloc(size + 1);  // +1 for null terminator
    if (!buffer) {
        close(fd);
        return NULL;
    }
    
    // Read entire file
    ssize_t bytes_read = read_exact(fd, buffer, size);
    close(fd);
    
    if (bytes_read == -1) {
        free(buffer);
        return NULL;
    }
    
    buffer[bytes_read] = '\0';
    if (out_size) *out_size = bytes_read;
    
    return buffer;
}
 
// Buffered reading with explicit buffer management
void demonstrate_buffered_reading() {
    int fd = open("/etc/passwd", O_RDONLY);
    if (fd == -1) {
        perror("open");
        return;
    }
    
    // Use a reasonable buffer size (often 4KB, 8KB, or page-aligned)
    char buffer[8192];
    ssize_t bytes_read;
    size_t total = 0;
    
    // Read in chunks until EOF
    while ((bytes_read = read(fd, buffer, sizeof(buffer))) > 0) {
        // Process chunk...
        total += bytes_read;
        printf("Read chunk of %zd bytes (total: %zu)
", 
               bytes_read, total);
    }
    
    if (bytes_read == -1) {
        perror("read");
    } else {
        printf("Finished reading. Total: %zu bytes
", total);
    }
    
    close(fd);
}

Buffer Size Selection

pread() — Positional Read Without Seeking

For multi-threaded programs or situations requiring atomic read-at-offset semantics:

ssize_t pread(int fd, void *buf, size_t count, off_t offset);

Advantages of pread():

Thread-safe: Multiple threads can read different parts of the same file simultaneously without interfering with each other's file position
Atomic: The seek and read are combined into one kernel operation
Efficiency: One system call instead of two (lseek + read)

Note that pread() does not modify the file offset—subsequent regular reads still occur at the previous position.

readv() — Scatter Read

For reading into multiple non-contiguous buffers:

ssize_t readv(int fd, const struct iovec *iov, int iovcnt);

This is useful when data has structure (e.g., header in one buffer, payload in another) and can be more efficient than multiple read() calls or post-read copying.

Writing Data — From User Space to Persistence

ssize_t write(int fd, const void *buf, size_t count);

Return Value Semantics

Positive value: Number of bytes actually written
Zero: Theoretically possible but practically rare for regular files
-1: Error occurred (check errno)

Like read(), write() may write fewer bytes than requested. This occurs for:

Writing to pipes/sockets with limited buffer space
Signals interrupting the call
Approaching storage limits

write_examples.c
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
#include <stdio.h>
#include <fcntl.h>
#include <unistd.h>
#include <errno.h>
#include <string.h>
 
// Correct pattern: write exactly n bytes
ssize_t write_exact(int fd, const void *buf, size_t count) {
    size_t total = 0;
    const char *ptr = buf;
    
    while (total < count) {
        ssize_t n = write(fd, ptr + total, count - total);
        
        if (n == -1) {
            if (errno == EINTR) {
                // Interrupted by signal, retry
                continue;
            }
            // Real error
            return -1;
        }
        
        // Note: n == 0 could indicate a problem, but is rare
        
        total += n;
    }
    
    return total;
}
 
// Write with durability guarantees
int write_durable(int fd, const void *buf, size_t count) {
    // Write all data
    if (write_exact(fd, buf, count) != (ssize_t)count) {
        return -1;
    }
    
    // fdatasync: ensure data (but not necessarily metadata) on disk
    // Use fsync() if metadata (size, timestamps) must also be durable
    if (fdatasync(fd) == -1) {
        return -1;
    }
    
    return 0;
}
 
// Atomic append pattern for log files
void append_to_log(const char *path, const char *message) {
    // O_APPEND makes writes atomic at end of file
    int fd = open(path, O_WRONLY | O_APPEND | O_CREAT, 0644);
    if (fd == -1) {
        perror("open log");
        return;
    }
    
    // Even with multiple processes writing, entries won't interleave
    // (up to PIPE_BUF bytes, or filesystem-specific limit)
    size_t len = strlen(message);
    if (write(fd, message, len) != (ssize_t)len) {
        perror("write log");
    }
    
    close(fd);
}
 
// Write-then-rename pattern for crash safety
int safe_write_file(const char *path, const void *data, size_t size) {
    char tmp_path[256];
    snprintf(tmp_path, sizeof(tmp_path), "%s.tmp", path);
    
    // Write to temporary file
    int fd = open(tmp_path, O_WRONLY | O_CREAT | O_TRUNC, 0644);
    if (fd == -1) return -1;
    
    ssize_t written = write_exact(fd, data, size);
    
    // Sync before rename to ensure data is durable
    if (fsync(fd) == -1) {
        close(fd);
        unlink(tmp_path);
        return -1;
    }
    
    close(fd);
    
    if (written != (ssize_t)size) {
        unlink(tmp_path);
        return -1;
    }
    
    // Atomic rename: either old content or new, never partial
    if (rename(tmp_path, path) == -1) {
        unlink(tmp_path);
        return -1;
    }
    
    return 0;
}

Durability vs. Performance

pwrite() and writev() — Positional and Scatter-Gather Writes

Mirroring the read variants:

pwrite() writes at a specific offset without modifying file position:

ssize_t pwrite(int fd, const void *buf, size_t count, off_t offset);

writev() writes from multiple buffers (gather write):

ssize_t writev(int fd, const struct iovec *iov, int iovcnt);

The O_APPEND Guarantee

When a file is opened with O_APPEND, every write atomically:

Seeks to the end of file
Writes the data

Note: Atomicity for writes is typically guaranteed only up to PIPE_BUF bytes (usually 4096) for pipes, and varies for regular files depending on the filesystem.

Durability Levels and Their Costs
Method	Data in Kernel Buffer	Data on Disk	Metadata on Disk	Typical Latency
write() alone	✓	Eventually	Eventually	~1μs
write() + fdatasync()	✓	✓	Partially	~1-5ms
write() + fsync()	✓	✓	✓	~3-10ms
O_SYNC per write	✓	✓	✓	~3-10ms per write
O_DIRECT (bypass cache)	N/A	✓	No	~0.1-1ms

Seeking — Navigating File Positions

off_t lseek(int fd, off_t offset, int whence);

The whence Parameter

SEEK_SET: Set position to offset bytes from beginning of file
SEEK_CUR: Set position to current location plus offset (can be negative)
SEEK_END: Set position to file size plus offset (often negative to seek backward)

Return Value

Returns the new offset position (from beginning), or -1 on error. A common idiom to get current position:

off_t current = lseek(fd, 0, SEEK_CUR);  // No movement, just query

seek_examples.c
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
#include <stdio.h>
#include <fcntl.h>
#include <unistd.h>
#include <sys/stat.h>
 
void demonstrate_seeking() {
    // Create a test file
    int fd = open("test.dat", O_RDWR | O_CREAT | O_TRUNC, 0644);
    if (fd == -1) {
        perror("open");
        return;
    }
    
    // Write some data
    const char *message = "Hello, World!";  // 13 bytes
    write(fd, message, 13);
    
    printf("After write, position: %lld
", 
           (long long)lseek(fd, 0, SEEK_CUR));
    
    // ========================================
    // SEEK_SET: Absolute positioning
    // ========================================
    lseek(fd, 0, SEEK_SET);  // Beginning of file
    printf("After SEEK_SET 0: %lld
", 
           (long long)lseek(fd, 0, SEEK_CUR));
    
    // Read and verify
    char buf[20];
    read(fd, buf, 5);
    buf[5] = '\0';
    printf("First 5 bytes: '%s'
", buf);  // "Hello"
    
    // ========================================
    // SEEK_CUR: Relative positioning
    // ========================================
    lseek(fd, 2, SEEK_CUR);  // Skip 2 bytes forward
    read(fd, buf, 5);
    buf[5] = '\0';
    printf("After skipping 2: '%s'
", buf);  // "World"
    
    lseek(fd, -5, SEEK_CUR);  // Go back 5 bytes
    read(fd, buf, 5);
    buf[5] = '\0';
    printf("After -5: '%s'
", buf);  // "World"
    
    // ========================================
    // SEEK_END: From end of file
    // ========================================
    lseek(fd, -6, SEEK_END);  // 6 bytes before end
    read(fd, buf, 6);
    buf[6] = '\0';
    printf("Last 6 bytes: '%s'
", buf);  // "World!"
    
    // ========================================
    // Get file size using seek
    // ========================================
    off_t size = lseek(fd, 0, SEEK_END);
    printf("File size: %lld bytes
", (long long)size);
    
    // ========================================
    // Creating sparse file with seek past end
    // ========================================
    lseek(fd, 1000000, SEEK_SET);  // Seek beyond file size
    write(fd, "X", 1);  // Write one byte at position 1000000
    
    struct stat st;
    fstat(fd, &st);
    printf("After sparse write:
");
    printf("  Logical size: %lld bytes
", (long long)st.st_size);
    printf("  Blocks used: %lld (x512 = %lld bytes)
", 
           (long long)st.st_blocks,
           (long long)st.st_blocks * 512);
    // Note: much less disk space actually used!
    
    close(fd);
    unlink("test.dat");
}
 
int main() {
    demonstrate_seeking();
    return 0;
}

Sparse Files

File Offset Sharing Semantics

Understanding how file offsets are shared (or not) is crucial:

After fork(): Parent and child share the same file offset (changes in one affect the other). This is because they share the underlying kernel file description.
After separate open(): Two opens of the same file create independent offsets. Each process has its own file description.
After dup()/dup2(): The new descriptor shares the offset with the original (same file description).
Threads in same process: Naturally share file descriptors and offsets (unless using thread-local descriptors).

This sharing behavior explains when pread()/pwrite() are necessary for thread safety, and why forked children's writes can unexpectedly interleave.

File Offset Sharing After Various Operations
Operation	Offset Relationship	Implication
fork()	Shared (same file description)	Concurrent access needs coordination
open() same file	Independent	Each process has own view
dup() / dup2()	Shared (same file description)	Used for I/O redirection
O_APPEND flag	Seek before each write	Atomically appends regardless of shared offset

Closing Files — Resource Release and Guarantees

int close(int fd);

What close() Does

Releases file descriptor: The integer becomes available for reuse
Decrements reference count: On the underlying kernel file description
Triggers writeback: Any unwritten cached data may be scheduled for writing
Releases locks: Any file locks held by this process through this fd
Potential inode release: If this was the last reference to an unlinked file, free space

What close() Does NOT Guarantee

Critically, close() returning success does not mean data is on disk. Buffered writes may still be pending. If durability is required, call fsync() or fdatasync() before closing.

close_examples.c
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
#include <stdio.h>
#include <fcntl.h>
#include <unistd.h>
#include <errno.h>
 
// Proper error handling on close
int safe_close(int fd) {
    // close() can fail!
    // For non-network filesystems, failure is rare but possible
    // For NFS, close() can report delayed write errors
    if (close(fd) == -1) {
        // Log or handle error
        // Note: fd is still closed, don't retry close()!
        return -1;
    }
    return 0;
}
 
// Pattern: close with durability requirement
int close_durable(int fd, int require_sync) {
    int result = 0;
    
    if (require_sync) {
        // Ensure data is on disk before closing
        if (fsync(fd) == -1) {
            // Save errno before close potentially changes it
            int saved_errno = errno;
            close(fd);  // Still close, but report the sync error
            errno = saved_errno;
            return -1;
        }
    }
    
    if (close(fd) == -1) {
        return -1;
    }
    
    return 0;
}
 
// Demonstrating reference counting and close
void demonstrate_close_semantics() {
    // Create a file
    int fd1 = open("demo.txt", O_RDWR | O_CREAT | O_TRUNC, 0600);
    write(fd1, "Hello", 5);
    
    // dup() creates second reference to same file description
    int fd2 = dup(fd1);
    
    printf("fd1=%d, fd2=%d (both point to same file)
", fd1, fd2);
    
    // Close one - file stays open via the other
    close(fd1);
    
    // Can still read/write through fd2
    lseek(fd2, 0, SEEK_SET);
    char buf[10];
    read(fd2, buf, 5);
    buf[5] = '\0';
    printf("After closing fd1, read via fd2: %s
", buf);
    
    // Unlink while file is open - file stays accessible
    unlink("demo.txt");
    // File doesn't appear in directory anymore, but...
    
    write(fd2, " World", 6);
    lseek(fd2, 0, SEEK_SET);
    read(fd2, buf, 11);
    buf[11] = '\0';
    printf("After unlink, still accessible: %s
", buf);
    
    // When fd2 is closed, file's space is finally freed
    close(fd2);
    printf("File space freed after final close
");
}
 
// RAII-style pattern for C (using GCC attribute)
#ifdef __GNUC__
static void auto_close(int *fd) {
    if (*fd != -1) {
        close(*fd);
    }
}
#define AUTO_CLOSE __attribute__((cleanup(auto_close)))
 
void raii_style_example() {
    // fd automatically closed when function returns
    AUTO_CLOSE int fd = open("/etc/passwd", O_RDONLY);
    if (fd == -1) return;
    
    char buf[100];
    read(fd, buf, sizeof(buf));
    // No need for explicit close() - cleanup attribute handles it
}
#endif

Double-Close Bug

File Metadata — stat, chmod, and Ownership

stat() Family — Retrieving File Information

int stat(const char *pathname, struct stat *statbuf);
int fstat(int fd, struct stat *statbuf);  // By file descriptor
int lstat(const char *pathname, struct stat *statbuf);  // Don't follow symlinks

The stat structure contains rich information:

Key Fields in struct stat
Field	Type	Description
st_mode	mode_t	File type (reg/dir/symlink/etc) and permissions
st_ino	ino_t	Inode number (unique within filesystem)
st_dev	dev_t	Device ID of filesystem
st_nlink	nlink_t	Number of hard links
st_uid / st_gid	uid_t / gid_t	Owner user ID and group ID
st_size	off_t	File size in bytes
st_blocks	blkcnt_t	Number of 512-byte blocks allocated
st_atime	time_t	Last access time
st_mtime	time_t	Last modification time (content)
st_ctime	time_t	Last status change time (metadata)

stat_examples.c
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
#include <stdio.h>
#include <sys/stat.h>
#include <time.h>
#include <pwd.h>     // getpwuid
#include <grp.h>     // getgrgid
#include <unistd.h>
 
void print_file_info(const char *path) {
    struct stat st;
    
    if (stat(path, &st) == -1) {
        perror("stat");
        return;
    }
    
    printf("File: %s
", path);
    
    // File type
    printf("Type: ");
    switch (st.st_mode & S_IFMT) {
        case S_IFREG:  printf("regular file
"); break;
        case S_IFDIR:  printf("directory
"); break;
        case S_IFLNK:  printf("symbolic link
"); break;
        case S_IFCHR:  printf("character device
"); break;
        case S_IFBLK:  printf("block device
"); break;
        case S_IFIFO:  printf("FIFO/pipe
"); break;
        case S_IFSOCK: printf("socket
"); break;
        default:       printf("unknown
");
    }
    
    // Permissions in octal and symbolic
    printf("Mode: %04o (", st.st_mode & 0777);
    printf("%c", (st.st_mode & S_IRUSR) ? 'r' : '-');
    printf("%c", (st.st_mode & S_IWUSR) ? 'w' : '-');
    printf("%c", (st.st_mode & S_IXUSR) ? 'x' : '-');
    printf("%c", (st.st_mode & S_IRGRP) ? 'r' : '-');
    printf("%c", (st.st_mode & S_IWGRP) ? 'w' : '-');
    printf("%c", (st.st_mode & S_IXGRP) ? 'x' : '-');
    printf("%c", (st.st_mode & S_IROTH) ? 'r' : '-');
    printf("%c", (st.st_mode & S_IWOTH) ? 'w' : '-');
    printf("%c", (st.st_mode & S_IXOTH) ? 'x' : '-');
    printf(")
");
    
    // Owner info
    struct passwd *pw = getpwuid(st.st_uid);
    struct group *gr = getgrgid(st.st_gid);
    printf("Owner: %s (%d)
", pw ? pw->pw_name : "unknown", st.st_uid);
    printf("Group: %s (%d)
", gr ? gr->gr_name : "unknown", st.st_gid);
    
    // Size and blocks
    printf("Size: %lld bytes
", (long long)st.st_size);
    printf("Blocks: %lld (actual disk: %lld bytes)
", 
           (long long)st.st_blocks, (long long)st.st_blocks * 512);
    
    // Hard link count
    printf("Hard links: %d
", (int)st.st_nlink);
    
    // Inode
    printf("Inode: %llu
", (unsigned long long)st.st_ino);
    
    // Timestamps
    printf("Access time: %s", ctime(&st.st_atime));
    printf("Modify time: %s", ctime(&st.st_mtime));
    printf("Change time: %s", ctime(&st.st_ctime));
}
 
// Check specific conditions
int is_regular_file(const char *path) {
    struct stat st;
    return (stat(path, &st) == 0 && S_ISREG(st.st_mode));
}
 
int is_directory(const char *path) {
    struct stat st;
    return (stat(path, &st) == 0 && S_ISDIR(st.st_mode));
}
 
int is_file_newer(const char *file1, const char *file2) {
    struct stat st1, st2;
    if (stat(file1, &st1) == -1 || stat(file2, &st2) == -1) {
        return -1;  // Error
    }
    return st1.st_mtime > st2.st_mtime;
}

Modifying Metadata

Several system calls modify file attributes:

chmod() / fchmod() — Change permissions:

int chmod(const char *pathname, mode_t mode);
int fchmod(int fd, mode_t mode);

chown() / fchown() / lchown() — Change ownership:

int chown(const char *pathname, uid_t owner, gid_t group);
int fchown(int fd, uid_t owner, gid_t group);
int lchown(const char *pathname, uid_t owner, gid_t group);  // Symlink itself

utimensat() / futimens() — Change timestamps:

int utimensat(int dirfd, const char *pathname, 
              const struct timespec times[2], int flags);

truncate() / ftruncate() — Change file size:

int truncate(const char *path, off_t length);
int ftruncate(int fd, off_t length);

Security Implications

Summary: File Management System Calls

File management system calls mediate every interaction between programs and persistent storage. We've explored the complete landscape:

Key Takeaways

•open() creates the connection between pathnames and file descriptors, with flags controlling access mode, creation behavior, and performance characteristics. The openat() variant enables race-free operations relative to directory file descriptors.
•read() transfers data from kernel to user space, with return values that demand careful handling—partial reads are normal, not errors. Variants like pread() and readv() address thread-safety and efficiency concerns.
•write() transfers data from user space toward persistent storage, but buffering means success doesn't imply durability. fsync() and O_SYNC provide guarantees when needed, at significant performance cost.
•lseek() navigates file positions, enabling random access. Understanding offset sharing semantics across fork() and dup() is crucial for concurrent access patterns.
•close() releases resources but requires checking for errors, especially on network filesystems. Reference counting means files persist until all descriptors and links are released.
•stat() family retrieves comprehensive file metadata, while chmod(), chown(), and truncate() modify it. These operations form the basis of file system utilities and security mechanisms.

Looking Ahead

Page Complete