Network File Systems - Learning Module

Loading content...

0/227

NFS Architecture

Anatomy of a Distributed File System

The Network File System's elegant transparency conceals a sophisticated multi-layered architecture. When a user opens a file, they experience a seamless local operation, but beneath the surface, an intricate dance of protocols, daemons, and kernel modules orchestrates communication between client and server.

Understanding NFS architecture is crucial for anyone deploying, troubleshooting, or optimizing NFS environments. It reveals why NFS behaves the way it does—its performance characteristics, failure modes, and tuning options all follow directly from architectural decisions.

This page dissects the NFS architecture from top to bottom, examining each layer and how they interact to create the illusion of local file access across the network.

What You Will Learn

By the end of this page, you will understand the complete NFS protocol stack, the role of each daemon and kernel component, how the mount protocol establishes connections, and how the client and server interact at each architectural layer. This knowledge is essential for debugging NFS issues and optimizing performance.

The NFS Protocol Stack

NFS doesn't exist in isolation—it's built upon a stack of protocols, each providing services to the layer above. Understanding this stack is fundamental to understanding NFS behavior.

The Complete Protocol Stack

From application to network, the layers are:

Converting Mermaid diagram...

Layer-by-Layer Analysis

1. Application Layer Applications use standard POSIX system calls (open, read, write, stat, etc.). They have no awareness of whether files are local or remote—the same code works for both.

2. Virtual File System (VFS) Layer The VFS provides a uniform interface abstracting different file system implementations. When an application performs a file operation, VFS:

Determines which file system handles the path (local ext4? NFS mount? tmpfs?)
Translates the operation into file-system-specific calls
Manages common operations like path resolution and permission checking

3. NFS Client Layer The NFS client translates VFS operations into NFS remote procedure calls. It:

Manages file handle mappings
Implements caching for attributes, data, and directory entries
Handles network failures, timeouts, and retries
Performs credential mapping for access control

4. RPC Layer Sun RPC (ONC RPC) provides the remote procedure call abstraction:

Marshals arguments into XDR format
Sends requests and waits for responses
Handles retransmission for UDP
Manages authentication (AUTH_SYS, AUTH_KERBEROS)

5. XDR Layer External Data Representation (XDR) serializes data types into a canonical network format:

Handles byte ordering differences between architectures
Provides fixed-size encodings for integers, strings, arrays
Ensures interoperability between different systems

6. Transport Layer UDP or TCP carries RPC messages:

UDP: lower latency, NFS/RPC handles reliability
TCP: built-in reliability, required for NFSv4

Why So Many Layers?

Each layer provides a clean abstraction that simplifies the layer above. VFS means NFS doesn't worry about POSIX details. RPC means NFS doesn't handle networking. XDR means NFS doesn't worry about byte ordering. This separation allows each layer to be developed, tested, and optimized independently.

Server-Side Architecture

The NFS server comprises several components working together to handle client requests. Understanding this architecture is essential for server configuration and performance tuning.

Converting Mermaid diagram...

Core Server Daemons

rpcbind (portmapper)

The rpcbind daemon is the RPC service registry. It runs on the well-known port 111 and maps RPC program numbers to actual port numbers.

When an NFS service starts, it:

Binds to a port (often dynamic)
Registers with rpcbind: "I am program 100003, version 3, on TCP port 2049"

When a client wants to contact NFS, it:

Queries rpcbind on port 111: "What port is program 100003 version 3 on?"
Gets the response: "TCP port 2049"
Connects directly to that port

# Show registered RPC services
$ rpcinfo -p localhost
   program vers proto   port  service
    100000    4   tcp    111  portmapper
    100000    4   udp    111  portmapper
    100003    3   tcp   2049  nfs
    100003    3   udp   2049  nfs
    100003    4   tcp   2049  nfs
    100005    3   tcp  20048  mountd
    100005    3   udp  20048  mountd
    100021    4   tcp  46245  nlockmgr
    100024    1   tcp  56952  status

mountd (Mount Daemon)

The mount daemon handles the mount protocol—the handshake that occurs before NFS access begins. When a client wants to mount an export:

Client contacts portmapper to find mountd's port
Client sends MOUNT request with the export path
mountd checks /etc/exports to verify access permission
If permitted, mountd returns the root file handle for the export
Client uses this handle for all subsequent NFS operations

mountd responsibilities:

Parse and cache /etc/exports configuration
Authenticate client machines (IP-based or hostname)
Log mount/unmount activities
Report currently mounted clients via showmount command

Mount Protocol Exchange

Protocol Flow

Mount Protocol Sequence:
 
Client                                    Server
   |                                         |
   |------ PORTMAP: Lookup mountd ---------->|  (port 111)
   |<----- mountd is on port 20048 ----------|
   |                                         |
   |------ MNT: Mount /export/home --------->|  (port 20048)
   |       (includes client hostname/IP)     |
   |                                         |  Server checks /etc/exports
   |                                         |  Server logs mount event
   |<----- MNT Reply: file_handle + status --|
   |       (root file handle for export)     |
   |                                         |
   |------ PORTMAP: Lookup nfs ------------->|  (port 111)
   |<----- nfs is on port 2049 --------------|
   |                                         |
   |========= NFS Operations Begin ==========>|  (port 2049)
   |       (using root file handle)          |
   |                                         |
 
Unmount Protocol:
   |------ UMNT: Unmount /export/home ------>|
   |<----- UMNT Reply: OK -------------------|
   |       (server logs unmount)             |

nfsd (NFS Daemon)

The nfsd is the heart of the NFS server. In modern Linux implementations, nfsd is actually implemented in kernel space for performance, with a user-space component that starts and configures the kernel threads.

Kernel-mode nfsd (knfsd):

Most production NFS servers use kernel-mode NFS (knfsd) for performance reasons:

Avoids user/kernel context switches for each request
Can directly access kernel caches and file system interfaces
Runs as multiple kernel threads handling requests in parallel

# Start 8 NFS server threads
$ sudo systemctl start nfs-server
# Or manually specify thread count
$ sudo rpc.nfsd 8

# View nfsd threads
$ ps aux | grep nfsd
root      1234  0.0  0.0      0     0 ?   S    10:00 [nfsd]
root      1235  0.0  0.0      0     0 ?   S    10:00 [nfsd]
... (8 threads)

The number of nfsd threads affects concurrency—more threads handle more simultaneous requests but consume more resources.

Additional Server Daemons

•lockd (rpc.lockd) — Implements the Network Lock Manager (NLM) protocol for file locking. Since NFS core is stateless, locking requires a separate stateful protocol. lockd manages advisory locks across the network.
•statd (rpc.statd) — The Network Status Monitor (NSM) protocol. Tracks which machines have locked files. When a server reboots, statd notifies clients so they can reclaim their locks.
•idmapd (rpc.idmapd) — NFSv4 ID mapping daemon. Translates between numeric UIDs/GIDs and user@domain strings. Essential for NFSv4's identity model.
•rquotad — Manages and reports disk quotas for NFS clients. Allows quota enforcement to extend across the network.
•gssd (rpc.gssd) — Handles GSS-API/Kerberos authentication for secure NFS. Required for NFSv4 with sec=krb5 mounts.

Firewall Considerations

NFS traditionally uses dynamically assigned ports for mountd, lockd, and statd, making firewall configuration challenging. Modern NFS installations typically configure fixed ports for these services. NFSv4 simplifies this by multiplexing all operations through port 2049.

Client-Side Architecture

The NFS client is equally sophisticated, responsible for translating local file operations into network requests and managing the complexity of remote access transparently.

Converting Mermaid diagram...

Client Components in Detail

VFS Integration

The NFS client registers with the VFS layer as a file system type. When a mount command specifies NFS:

mount -t nfs server:/export/home /home

VFS invokes the NFS module's mount function, which:

Parses mount options (version, security, caching parameters)
Contacts the server's mount protocol to obtain the root file handle
Creates internal data structures representing the mounted file system
Registers the mount point with VFS

File Handle Management

When an application opens a file, the client must translate the path to a file handle:

Start with the root file handle (from mount)
For each path component, issue LOOKUP to get the next handle
Cache intermediate handles to avoid repeated lookups

The file handle cache is critical for performance—without it, accessing /home/alice/projects/code/main.c would require 5 LOOKUP RPCs every time.

Path Resolution and Caching
C (Pseudocode)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
/* Simplified path resolution with caching */
 
struct nfs_fh* nfs_lookup_path(const char* path) {
    struct nfs_fh* current_fh = get_mount_root_fh();
    char* component;
    char* path_copy = strdup(path);
    
    /* Split path and resolve component by component */
    for (component = strtok(path_copy, "/"); 
         component != NULL; 
         component = strtok(NULL, "/")) {
        
        /* Check directory cache first */
        struct nfs_fh* cached = dir_cache_lookup(current_fh, component);
        if (cached != NULL) {
            current_fh = cached;
            continue;  /* Cache hit - no RPC needed */
        }
        
        /* Cache miss - must contact server */
        struct nfs_lookup_result result;
        int status = nfs_rpc_lookup(current_fh, component, &result);
        
        if (status != NFS_OK) {
            free(path_copy);
            return NULL;
        }
        
        /* Cache the result for future lookups */
        dir_cache_insert(current_fh, component, &result.file_handle);
        
        /* Also cache attributes to avoid GETATTR calls */
        attr_cache_insert(&result.file_handle, &result.attributes);
        
        current_fh = copy_fh(&result.file_handle);
    }
    
    free(path_copy);
    return current_fh;
}

The Caching Subsystem

Aggressive caching is essential for NFS performance. The client maintains several caches:

1. Attribute Cache File attributes (size, mtime, permissions) are cached to avoid GETATTR calls. This is controlled by the ac mount option and acmin/acmax timeouts.

acregmin/acregmax — Attribute cache timeout for regular files (default: 3-60 seconds)
acdirmin/acdirmax — Attribute cache timeout for directories (default: 30-60 seconds)
noac — Disable attribute caching entirely (poor performance but strict consistency)

2. Directory Cache (DNLC/dcache) Maps (parent_handle, name) → child_handle. Eliminates repeated LOOKUP calls for frequently accessed paths.

3. Data Cache (Page Cache) File contents are cached in the kernel's page cache, just like local files. This enables:

Read-ahead for sequential access
Write buffering for better performance
Multiple reads from cache without RPCs

4. Reply Cache (Client) Retained replies for duplicate detection. If retransmitting a request, the client can avoid re-sending if it already has the answer.

Cache Coherence Challenge

Client-side caching creates a fundamental trade-off: better performance versus data consistency. A cached value might be stale if another client modified the file. We'll explore NFS's weak consistency semantics and cache invalidation strategies in the Performance Considerations page.

NFS and Mount Protocol Details

Let's examine the actual protocol messages exchanged between client and server. Understanding these details is invaluable for debugging NFS issues with tools like Wireshark or tcpdump.

RPC Message Structure

Every NFS operation is wrapped in an RPC message. The structure includes:

RPC Call (Request):

┌─────────────────────────────────────────────────────┐
│ Fragment Header (4 bytes) - TCP only                │
├─────────────────────────────────────────────────────┤
│ XID (4 bytes) - Transaction ID for matching replies │
├─────────────────────────────────────────────────────┤
│ Message Type (4 bytes) - 0 = Call                   │
├─────────────────────────────────────────────────────┤
│ RPC Version (4 bytes) - Always 2                    │
├─────────────────────────────────────────────────────┤
│ Program (4 bytes) - 100003 for NFS                  │
├─────────────────────────────────────────────────────┤
│ Version (4 bytes) - 2, 3, or 4                      │
├─────────────────────────────────────────────────────┤
│ Procedure (4 bytes) - Operation number              │
├─────────────────────────────────────────────────────┤
│ Credentials (variable) - Auth info                  │
├─────────────────────────────────────────────────────┤
│ Verifier (variable) - Auth verification             │
├─────────────────────────────────────────────────────┤
│ Procedure Arguments (variable) - XDR encoded        │
└─────────────────────────────────────────────────────┘

NFS READ Request/Response Example

Protocol Analysis

NFSv3 READ Operation Analysis
 
=== READ Request ===
RPC Header:
  XID:       0x12345678 (transaction identifier)
  MsgType:   CALL (0)
  RPCVers:   2
  Program:   100003 (NFS)
  ProgVers:  3 (NFSv3)
  Procedure: 6 (READ)
 
Credentials (AUTH_SYS):
  Stamp:     0x00000001
  Machine:   "client.example.com"
  UID:       1000 (alice)
  GID:       1000 (alice)
  GIDs:      [1000, 100, 27]  (supplementary groups)
 
READ Arguments:
  File Handle: 
    0x0000000a 00000001 40000000 00000010
    00000000 00000002 00000000 00000000
    (32 bytes, opaque to client)
  Offset:    0 (start of file)
  Count:     32768 (32KB requested)
 
=== READ Response ===
RPC Header:
  XID:       0x12345678 (matches request)
  MsgType:   REPLY (1)
  Status:    MSG_ACCEPTED
 
READ Result:
  Status:    NFS3_OK (0)
  File Attributes (post-op):
    Type:    Regular File
    Mode:    0644
    Size:    45231 bytes
    Mtime:   2024-01-15 10:30:00
    ...
  Count:     32768 (bytes read)
  EOF:       FALSE (more data available)
  Data:      [32768 bytes of file content]

Common NFSv3 Procedures
Proc #	Name	Request Args	Response
0	NULL	None	None (ping/test)
1	GETATTR	file_handle	attributes
2	SETATTR	file_handle, new_attrs, guard	wcc_data
3	LOOKUP	dir_handle, name	file_handle, attributes
6	READ	file_handle, offset, count	attributes, count, eof, data
7	WRITE	file_handle, offset, count, stable, data	wcc_data, count, committed, verf
8	CREATE	dir_handle, name, how, attrs	file_handle, attributes, wcc_data
12	REMOVE	dir_handle, name	wcc_data
14	RENAME	from_dir, from_name, to_dir, to_name	wcc_data (both dirs)
16	READDIR	dir_handle, cookie, cookieverf, count	entries, eof
17	READDIRPLUS	dir_handle, cookie, verifier, dircount, maxcount	entries with attributes

Weak Cache Consistency (WCC)

NFSv3 introduced Weak Cache Consistency (WCC) data to help clients maintain cache coherence. For operations that modify files, the response includes:

pre-op attributes — File attributes before the operation
post-op attributes — File attributes after the operation

The client can compare pre-op attributes to its cached version:

If they match: cache was valid, update with post-op attributes
If they don't match: another client changed the file, invalidate cache

This isn't perfect consistency (changes between operations aren't detected), but it catches common cases without requiring server-side state.

WCC Data Structure
C
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
/* NFSv3 Weak Cache Consistency Data */
 
/* Pre-operation attributes - lightweight, just for comparison */
struct wcc_attr {
    uint64_t  size;     /* File size before operation */
    nfstime3  mtime;    /* Modification time before */
    nfstime3  ctime;    /* Change time before */
};
 
/* Optional pre-op attributes (may not be available) */
union pre_op_attr switch (bool attributes_follow) {
    case TRUE:  wcc_attr attributes;
    case FALSE: void;
};
 
/* Full post-operation attributes */
union post_op_attr switch (bool attributes_follow) {
    case TRUE:  fattr3 attributes;  /* Complete attribute set */
    case FALSE: void;
};
 
/* WCC data returned with modifying operations */
struct wcc_data {
    pre_op_attr   before;   /* Attributes before change */
    post_op_attr  after;    /* Attributes after change */
};
 
/* Example: Client cache coherence check */
void update_cache_from_wcc(struct inode* inode, struct wcc_data* wcc) {
    if (!wcc->before.attributes_follow)
        return;  /* No pre-op data, can't validate */
    
    /* Compare cached attributes with pre-op */
    if (inode->cached_size != wcc->before.attributes.size ||
        !times_equal(&inode->cached_mtime, &wcc->before.attributes.mtime)) {
        /* Cache is stale - someone else modified the file */
        invalidate_data_cache(inode);
        invalidate_attr_cache(inode);
    }
    
    /* Update cache with new attributes */
    if (wcc->after.attributes_follow) {
        update_cached_attrs(inode, &wcc->after.attributes);
    }
}

File Handle Architecture

File handles are the cornerstone of NFS's stateless design. Let's examine their architecture in depth—how they're created, validated, and used to enable seamless file access without server-side session state.

File Handle Generation

When a client performs a LOOKUP, CREATE, or other operation that returns a file handle, the server constructs a handle that encodes enough information to identify the file in future requests.

The handle typically includes:

File System ID — Identifies which mounted file system contains the file
Inode/Object Number — The file's unique identifier within that file system
Generation Number — Incremented when inodes are reused, detecting stale handles
Export Verification — Information to verify the file is within an exported path

File Handle Implementation (Linux KNFSD)
C
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
/* Linux NFS file handle structure (simplified from knfsd) */
 
struct knfsd_fh {
    uint8_t  fh_size;           /* Actual handle size (varies) */
    uint8_t  fh_auth_type;      /* Authentication/export type */
    uint8_t  fh_fsid_type;      /* Type of fsid encoding */
    uint8_t  fh_fileid_type;    /* Type of fileid encoding */
    
    /* Variable-size payload follows, contains: */
    union {
        struct {
            /* For most local file systems */
            uint32_t  fsid_major;     /* File system major number */
            uint32_t  fsid_minor;     /* File system minor number */
            uint64_t  inode;          /* Inode number */
            uint32_t  generation;     /* Inode generation count */
            uint32_t  parent_inode;   /* Parent inode (for verification) */
            uint32_t  parent_gen;     /* Parent generation */
        } local;
        
        struct {
            /* For UUID-identified file systems */
            uuid_t    fsid_uuid;      /* File system UUID */
            uint64_t  inode;
            uint32_t  generation;
        } uuid;
    } fh_payload;
};
 
/* Maximum file handle sizes */
#define NFS2_FHSIZE    32    /* NFSv2: exactly 32 bytes */
#define NFS3_FHSIZE    64    /* NFSv3: up to 64 bytes */
#define NFS4_FHSIZE   128    /* NFSv4: up to 128 bytes */
 
/* Handle validation on server */
int nfsd_validate_fh(struct knfsd_fh* fh, struct inode** result) {
    struct super_block* sb;
    struct dentry* dentry;
    struct inode* inode;
    
    /* Step 1: Find the file system */
    sb = nfsd_find_filesystem(fh->fh_payload.local.fsid_major,
                              fh->fh_payload.local.fsid_minor);
    if (!sb)
        return nfserr_stale;
    
    /* Step 2: Look up the inode */
    inode = nfsd_iget(sb, fh->fh_payload.local.inode);
    if (!inode || IS_ERR(inode))
        return nfserr_stale;
    
    /* Step 3: Validate generation number */
    if (inode->i_generation != fh->fh_payload.local.generation) {
        iput(inode);
        return nfserr_stale;  /* Inode reused - stale handle! */
    }
    
    /* Step 4: Verify file is within exported subtree */
    if (!nfsd_check_export_path(fh, inode)) {
        iput(inode);
        return nfserr_stale;
    }
    
    *result = inode;
    return 0;
}

Stale File Handle Detection

The generation number mechanism prevents a dangerous scenario:

Client opens file with inode #1234, gets handle containing (inode=1234, gen=5)
File is deleted on server, inode #1234 is freed
New file is created, assigned inode #1234 (with gen=6)
Client tries to read using old handle
Server checks: inode 1234 exists, but generation is 6, not 5
Server returns NFS3ERR_STALE
Client knows handle is invalid, can report error to application

Without generation numbers, step 4-6 would access the wrong file silently—a catastrophic data corruption scenario.

Handle Persistence Across Reboots

File handles must remain valid across server reboots. This means the handle cannot contain volatile information (like memory addresses). The handle's components (fsid, inode, generation) must be reconstructable from persistent on-disk data. Some file systems (like XFS with changing UUIDs) can cause issues with handle persistence.

Security Concerns with File Handles

File handles in NFSv2/v3 present security challenges:

Handle Guessing — If handles are predictable (sequential inodes, no generation), attackers might guess valid handles
Handle Leakage — If an attacker obtains a handle, they can access the file directly, bypassing path-based security
Export Verification — Handles must ensure the file is within an exported directory, not just any file the server can access

NFSv4 addresses some of these concerns with volatile file handles, stronger authentication, and pseudofs for export isolation.

Authentication and Security Architecture

Security in NFS has evolved significantly from its original trusted-network design to modern mechanisms supporting secure deployment across untrusted networks.

NFS Authentication Mechanisms
Mechanism	Security Level	Description	Use Case
AUTH_NULL	None	No authentication	Testing only
AUTH_SYS (AUTH_UNIX)	Weak	Client-asserted UID/GID	Trusted LANs
AUTH_DH (Diffie-Hellman)	Medium	Cryptographic (rarely used)	Legacy secure NFS
RPCSEC_GSS / Kerberos	Strong	Full authentication, optional encryption	Enterprise/untrusted networks

AUTH_SYS: The Default (and Weak) Scenario

The default NFS authentication is AUTH_SYS (historically called AUTH_UNIX). With this mechanism:

Client sends its claimed UID, GID, and supplementary groups
Server trusts this completely
Server checks file permissions against these claimed credentials

The security implications are severe:

Anyone with root on a client can claim to be any UID
No cryptographic verification occurs
Network traffic (including credentials) is unencrypted

This is acceptable only in controlled, trusted network environments.

AUTH_SYS Credential Structure
C
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
/* AUTH_SYS (AUTH_UNIX) credentials */
struct authsys_parms {
    uint32_t  aup_time;           /* Timestamp (not verified) */
    char     *aup_machname;       /* Hostname of client */
    uint32_t  aup_uid;            /* User ID (trusted blindly!) */
    uint32_t  aup_gid;            /* Primary group ID */
    uint32_t  aup_len;            /* Number of supplementary groups */
    uint32_t  aup_gids[16];       /* Supplementary group IDs (max 16) */
};
 
/* The server receives this and simply believes it */
/* Example: impersonating root is trivial */
 
/* On malicious client: */
struct authsys_parms fake_creds = {
    .aup_time = time(NULL),
    .aup_machname = "legit-client.example.com",
    .aup_uid = 0,     /* Root! */
    .aup_gid = 0,     /* Root group! */
    .aup_len = 0,
};
 
/* Server cannot distinguish this from legitimate root request
 * unless root_squash is enabled on the export */

RPCSEC_GSS with Kerberos

For secure NFS deployment, RPCSEC_GSS with Kerberos provides:

Authentication Levels:

sec=krb5 — Authentication only. Client identity is cryptographically verified, but data is unencrypted.
sec=krb5i — Authentication + Integrity. Data is signed with cryptographic checksums to detect tampering.
sec=krb5p — Authentication + Integrity + Privacy. Full encryption of all data traffic.

How Kerberos Integration Works:

Users obtain Kerberos tickets from the Key Distribution Center (KDC)
NFS client's rpc.gssd daemon uses tickets to authenticate requests
Server's rpc.svcgssd validates tickets against the KDC
UIDs are mapped via rpc.idmapd using principal names (user@REALM → UID)

# Kerberos-secured NFS mount
mount -t nfs4 -o sec=krb5p server.example.com:/export /mnt

# Client needs valid Kerberos credentials
klist  # Show current tickets
kinit alice@EXAMPLE.COM  # Obtain ticket if needed

Kerberos Deployment Complexity

While Kerberos provides strong security, it requires significant infrastructure: a KDC, synchronized clocks, DNS entries, and key management. For many internal deployments, the complexity is justified. For simpler scenarios, network segmentation and firewall rules with AUTH_SYS may be acceptable.

Linux NFS Implementation

Linux provides one of the most widely-used NFS implementations. Understanding its specific architecture helps with deployment and troubleshooting.

Kernel Module Structure

Linux NFS is implemented primarily in kernel modules:

$ lsmod | grep nfs
nfs                   [NFS client module]
nfsv3                 [NFSv3 support]
nfsv4                 [NFSv4 support]
nfsd                  [NFS server module]
auth_rpcgss           [GSS authentication]
sunrpc                [RPC infrastructure]

Server Implementation (knfsd):

The Linux NFS server uses a kernel-space implementation for performance:

nfsd module handles NFS protocol
exportfs module manages exports
lockd module handles file locking
Multiple kernel threads ([nfsd]) handle concurrent requests

Configuration flows from user-space tools (exportfs, rpc.nfsd) to kernel via /proc/fs/nfsd/.

NFS Server Configuration and Monitoring
Bash
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
#!/bin/bash
# Comprehensive NFS server setup and monitoring
 
# === Server Configuration ===
 
# Configure /etc/exports
cat > /etc/exports << 'EOF'
# Home directories - authenticated access
/export/home    192.168.1.0/24(rw,sync,no_subtree_check,root_squash)
 
# Shared project data
/export/projects  *(rw,sync,no_subtree_check,all_squash,anonuid=1000,anongid=1000)
 
# Read-only repository
/export/repo    *(ro,sync,no_subtree_check)
EOF
 
# Apply export configuration
exportfs -ra
 
# View current exports
exportfs -v
 
# Start NFS services
systemctl start rpcbind
systemctl start nfs-server
 
# Configure number of server threads (/etc/nfs.conf)
# [nfsd]
# threads=16
 
# === Monitoring and Statistics ===
 
# View NFS statistics
nfsstat -s   # Server statistics
nfsstat -c   # Client statistics
 
# Detailed RPC info
cat /proc/net/rpc/nfsd
 
# Current NFS sessions (NFSv4)
cat /proc/fs/nfsd/clients/*/info
 
# === Kernel Parameters ===
 
# View NFS-related kernel params
sysctl -a | grep nfs
 
# Important parameters:
# sunrpc.tcp_slot_table_entries   - Max concurrent RPC requests
# fs.nfs.nfs_congestion_kb        - Congestion control writeback threshold
# fs.nfs.nfs_acl_strict           - Strict ACL enforcement
 
# === Log Analysis ===
 
# Enable NFS debug logging
rpcdebug -m nfsd -s all    # Server debug
rpcdebug -m nfs -s all     # Client debug
 
# Debug messages go to kernel log
dmesg | grep -i nfs
 
# Journal logs
journalctl -u nfs-server -f

The /proc/fs/nfsd Interface

Linux exposes NFS server configuration and status via the /proc filesystem:

File	Purpose
`/proc/fs/nfsd/threads`	Number of nfsd threads (read/write)
`/proc/fs/nfsd/exports`	Currently active exports
`/proc/fs/nfsd/pool_stats`	Statistics per CPU pool
`/proc/fs/nfsd/clients/`	NFSv4 client sessions
`/proc/fs/nfsd/versions`	Enabled NFS versions
`/proc/fs/nfsd/max_block_size`	Maximum read/write block size

NFS Version Selection

Linux can simultaneously support NFSv3 and NFSv4. By default, clients negotiate the highest version supported by both sides. You can force specific versions with mount options (vers=3, vers=4, vers=4.1, vers=4.2) for testing or compatibility.

Summary: NFS Architecture Mastery

We've thoroughly examined the multi-layered architecture that makes NFS work. This understanding is foundational for deploying, troubleshooting, and optimizing NFS systems. Let's consolidate the key points:

Key Takeaways

•NFS is built on a layered protocol stack — VFS, NFS client/server, RPC, XDR, and TCP/UDP each provide clean abstractions. Understanding each layer helps diagnose issues at the right level.
•Server architecture centers on multiple daemons — rpcbind for service discovery, mountd for access control, nfsd for core operations, and auxiliary services for locking and status monitoring.
•Client architecture emphasizes aggressive caching — Attribute, directory, data, and file handle caches reduce network round-trips but introduce consistency challenges.
•File handles are the stateless key — Generation numbers prevent stale handle attacks. Handle format is opaque to clients but must persist across server reboots.
•Security ranges from weak to strong — AUTH_SYS trusts clients completely; RPCSEC_GSS with Kerberos provides authentication, integrity, and encryption.
•Linux implementation uses kernel-space performance — knfsd runs as kernel threads, with configuration via /proc/fs/nfsd and standard tools.

What's Next

With the architecture understood, we'll explore the Stateless Protocol Design in depth. We'll examine why statelessness was chosen, how it enables crash recovery, the challenges it creates for certain operations (especially locking), and how NFS accommodates operations that inherently require state. This understanding is crucial for predicting NFS behavior in failure scenarios.

Page Complete

You now understand the complete NFS architecture—from the VFS layer through RPC to the wire protocol. You can trace the path of a file operation from application system call to disk blocks on a remote server. This architectural foundation enables effective deployment, debugging, and optimization of NFS systems.

NFS Architecture

Anatomy of a Distributed File System

This page dissects the NFS architecture from top to bottom, examining each layer and how they interact to create the illusion of local file access across the network.

What You Will Learn

The NFS Protocol Stack

NFS doesn't exist in isolation—it's built upon a stack of protocols, each providing services to the layer above. Understanding this stack is fundamental to understanding NFS behavior.

The Complete Protocol Stack

From application to network, the layers are:

Converting Mermaid diagram...

Layer-by-Layer Analysis

2. Virtual File System (VFS) Layer The VFS provides a uniform interface abstracting different file system implementations. When an application performs a file operation, VFS:

Determines which file system handles the path (local ext4? NFS mount? tmpfs?)
Translates the operation into file-system-specific calls
Manages common operations like path resolution and permission checking

3. NFS Client Layer The NFS client translates VFS operations into NFS remote procedure calls. It:

Manages file handle mappings
Implements caching for attributes, data, and directory entries
Handles network failures, timeouts, and retries
Performs credential mapping for access control

4. RPC Layer Sun RPC (ONC RPC) provides the remote procedure call abstraction:

Marshals arguments into XDR format
Sends requests and waits for responses
Handles retransmission for UDP
Manages authentication (AUTH_SYS, AUTH_KERBEROS)

5. XDR Layer External Data Representation (XDR) serializes data types into a canonical network format:

Handles byte ordering differences between architectures
Provides fixed-size encodings for integers, strings, arrays
Ensures interoperability between different systems

6. Transport Layer UDP or TCP carries RPC messages:

UDP: lower latency, NFS/RPC handles reliability
TCP: built-in reliability, required for NFSv4

Why So Many Layers?

Server-Side Architecture

The NFS server comprises several components working together to handle client requests. Understanding this architecture is essential for server configuration and performance tuning.

Converting Mermaid diagram...

Core Server Daemons

rpcbind (portmapper)

The rpcbind daemon is the RPC service registry. It runs on the well-known port 111 and maps RPC program numbers to actual port numbers.

When an NFS service starts, it:

Binds to a port (often dynamic)
Registers with rpcbind: "I am program 100003, version 3, on TCP port 2049"

When a client wants to contact NFS, it:

Queries rpcbind on port 111: "What port is program 100003 version 3 on?"
Gets the response: "TCP port 2049"
Connects directly to that port

# Show registered RPC services
$ rpcinfo -p localhost
   program vers proto   port  service
    100000    4   tcp    111  portmapper
    100000    4   udp    111  portmapper
    100003    3   tcp   2049  nfs
    100003    3   udp   2049  nfs
    100003    4   tcp   2049  nfs
    100005    3   tcp  20048  mountd
    100005    3   udp  20048  mountd
    100021    4   tcp  46245  nlockmgr
    100024    1   tcp  56952  status

mountd (Mount Daemon)

The mount daemon handles the mount protocol—the handshake that occurs before NFS access begins. When a client wants to mount an export:

Client contacts portmapper to find mountd's port
Client sends MOUNT request with the export path
mountd checks /etc/exports to verify access permission
If permitted, mountd returns the root file handle for the export
Client uses this handle for all subsequent NFS operations

mountd responsibilities:

Parse and cache /etc/exports configuration
Authenticate client machines (IP-based or hostname)
Log mount/unmount activities
Report currently mounted clients via showmount command

Mount Protocol Exchange

Protocol Flow

Mount Protocol Sequence:
 
Client                                    Server
   |                                         |
   |------ PORTMAP: Lookup mountd ---------->|  (port 111)
   |<----- mountd is on port 20048 ----------|
   |                                         |
   |------ MNT: Mount /export/home --------->|  (port 20048)
   |       (includes client hostname/IP)     |
   |                                         |  Server checks /etc/exports
   |                                         |  Server logs mount event
   |<----- MNT Reply: file_handle + status --|
   |       (root file handle for export)     |
   |                                         |
   |------ PORTMAP: Lookup nfs ------------->|  (port 111)
   |<----- nfs is on port 2049 --------------|
   |                                         |
   |========= NFS Operations Begin ==========>|  (port 2049)
   |       (using root file handle)          |
   |                                         |
 
Unmount Protocol:
   |------ UMNT: Unmount /export/home ------>|
   |<----- UMNT Reply: OK -------------------|
   |       (server logs unmount)             |

nfsd (NFS Daemon)

Kernel-mode nfsd (knfsd):

Most production NFS servers use kernel-mode NFS (knfsd) for performance reasons:

Avoids user/kernel context switches for each request
Can directly access kernel caches and file system interfaces
Runs as multiple kernel threads handling requests in parallel

# Start 8 NFS server threads
$ sudo systemctl start nfs-server
# Or manually specify thread count
$ sudo rpc.nfsd 8

# View nfsd threads
$ ps aux | grep nfsd
root      1234  0.0  0.0      0     0 ?   S    10:00 [nfsd]
root      1235  0.0  0.0      0     0 ?   S    10:00 [nfsd]
... (8 threads)

The number of nfsd threads affects concurrency—more threads handle more simultaneous requests but consume more resources.

Additional Server Daemons

•lockd (rpc.lockd) — Implements the Network Lock Manager (NLM) protocol for file locking. Since NFS core is stateless, locking requires a separate stateful protocol. lockd manages advisory locks across the network.
•statd (rpc.statd) — The Network Status Monitor (NSM) protocol. Tracks which machines have locked files. When a server reboots, statd notifies clients so they can reclaim their locks.
•idmapd (rpc.idmapd) — NFSv4 ID mapping daemon. Translates between numeric UIDs/GIDs and user@domain strings. Essential for NFSv4's identity model.
•rquotad — Manages and reports disk quotas for NFS clients. Allows quota enforcement to extend across the network.
•gssd (rpc.gssd) — Handles GSS-API/Kerberos authentication for secure NFS. Required for NFSv4 with sec=krb5 mounts.

Firewall Considerations

Client-Side Architecture

The NFS client is equally sophisticated, responsible for translating local file operations into network requests and managing the complexity of remote access transparently.

Converting Mermaid diagram...

Client Components in Detail

VFS Integration

The NFS client registers with the VFS layer as a file system type. When a mount command specifies NFS:

mount -t nfs server:/export/home /home

VFS invokes the NFS module's mount function, which:

Parses mount options (version, security, caching parameters)
Contacts the server's mount protocol to obtain the root file handle
Creates internal data structures representing the mounted file system
Registers the mount point with VFS

File Handle Management

When an application opens a file, the client must translate the path to a file handle:

Start with the root file handle (from mount)
For each path component, issue LOOKUP to get the next handle
Cache intermediate handles to avoid repeated lookups

The file handle cache is critical for performance—without it, accessing /home/alice/projects/code/main.c would require 5 LOOKUP RPCs every time.

Path Resolution and Caching
C (Pseudocode)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
/* Simplified path resolution with caching */
 
struct nfs_fh* nfs_lookup_path(const char* path) {
    struct nfs_fh* current_fh = get_mount_root_fh();
    char* component;
    char* path_copy = strdup(path);
    
    /* Split path and resolve component by component */
    for (component = strtok(path_copy, "/"); 
         component != NULL; 
         component = strtok(NULL, "/")) {
        
        /* Check directory cache first */
        struct nfs_fh* cached = dir_cache_lookup(current_fh, component);
        if (cached != NULL) {
            current_fh = cached;
            continue;  /* Cache hit - no RPC needed */
        }
        
        /* Cache miss - must contact server */
        struct nfs_lookup_result result;
        int status = nfs_rpc_lookup(current_fh, component, &result);
        
        if (status != NFS_OK) {
            free(path_copy);
            return NULL;
        }
        
        /* Cache the result for future lookups */
        dir_cache_insert(current_fh, component, &result.file_handle);
        
        /* Also cache attributes to avoid GETATTR calls */
        attr_cache_insert(&result.file_handle, &result.attributes);
        
        current_fh = copy_fh(&result.file_handle);
    }
    
    free(path_copy);
    return current_fh;
}

The Caching Subsystem

Aggressive caching is essential for NFS performance. The client maintains several caches:

1. Attribute Cache File attributes (size, mtime, permissions) are cached to avoid GETATTR calls. This is controlled by the ac mount option and acmin/acmax timeouts.

acregmin/acregmax — Attribute cache timeout for regular files (default: 3-60 seconds)
acdirmin/acdirmax — Attribute cache timeout for directories (default: 30-60 seconds)
noac — Disable attribute caching entirely (poor performance but strict consistency)

2. Directory Cache (DNLC/dcache) Maps (parent_handle, name) → child_handle. Eliminates repeated LOOKUP calls for frequently accessed paths.

3. Data Cache (Page Cache) File contents are cached in the kernel's page cache, just like local files. This enables:

Read-ahead for sequential access
Write buffering for better performance
Multiple reads from cache without RPCs

4. Reply Cache (Client) Retained replies for duplicate detection. If retransmitting a request, the client can avoid re-sending if it already has the answer.

Cache Coherence Challenge

NFS and Mount Protocol Details

Let's examine the actual protocol messages exchanged between client and server. Understanding these details is invaluable for debugging NFS issues with tools like Wireshark or tcpdump.

RPC Message Structure

Every NFS operation is wrapped in an RPC message. The structure includes:

RPC Call (Request):

┌─────────────────────────────────────────────────────┐
│ Fragment Header (4 bytes) - TCP only                │
├─────────────────────────────────────────────────────┤
│ XID (4 bytes) - Transaction ID for matching replies │
├─────────────────────────────────────────────────────┤
│ Message Type (4 bytes) - 0 = Call                   │
├─────────────────────────────────────────────────────┤
│ RPC Version (4 bytes) - Always 2                    │
├─────────────────────────────────────────────────────┤
│ Program (4 bytes) - 100003 for NFS                  │
├─────────────────────────────────────────────────────┤
│ Version (4 bytes) - 2, 3, or 4                      │
├─────────────────────────────────────────────────────┤
│ Procedure (4 bytes) - Operation number              │
├─────────────────────────────────────────────────────┤
│ Credentials (variable) - Auth info                  │
├─────────────────────────────────────────────────────┤
│ Verifier (variable) - Auth verification             │
├─────────────────────────────────────────────────────┤
│ Procedure Arguments (variable) - XDR encoded        │
└─────────────────────────────────────────────────────┘

NFS READ Request/Response Example

Protocol Analysis

NFSv3 READ Operation Analysis
 
=== READ Request ===
RPC Header:
  XID:       0x12345678 (transaction identifier)
  MsgType:   CALL (0)
  RPCVers:   2
  Program:   100003 (NFS)
  ProgVers:  3 (NFSv3)
  Procedure: 6 (READ)
 
Credentials (AUTH_SYS):
  Stamp:     0x00000001
  Machine:   "client.example.com"
  UID:       1000 (alice)
  GID:       1000 (alice)
  GIDs:      [1000, 100, 27]  (supplementary groups)
 
READ Arguments:
  File Handle: 
    0x0000000a 00000001 40000000 00000010
    00000000 00000002 00000000 00000000
    (32 bytes, opaque to client)
  Offset:    0 (start of file)
  Count:     32768 (32KB requested)
 
=== READ Response ===
RPC Header:
  XID:       0x12345678 (matches request)
  MsgType:   REPLY (1)
  Status:    MSG_ACCEPTED
 
READ Result:
  Status:    NFS3_OK (0)
  File Attributes (post-op):
    Type:    Regular File
    Mode:    0644
    Size:    45231 bytes
    Mtime:   2024-01-15 10:30:00
    ...
  Count:     32768 (bytes read)
  EOF:       FALSE (more data available)
  Data:      [32768 bytes of file content]

Common NFSv3 Procedures
Proc #	Name	Request Args	Response
0	NULL	None	None (ping/test)
1	GETATTR	file_handle	attributes
2	SETATTR	file_handle, new_attrs, guard	wcc_data
3	LOOKUP	dir_handle, name	file_handle, attributes
6	READ	file_handle, offset, count	attributes, count, eof, data
7	WRITE	file_handle, offset, count, stable, data	wcc_data, count, committed, verf
8	CREATE	dir_handle, name, how, attrs	file_handle, attributes, wcc_data
12	REMOVE	dir_handle, name	wcc_data
14	RENAME	from_dir, from_name, to_dir, to_name	wcc_data (both dirs)
16	READDIR	dir_handle, cookie, cookieverf, count	entries, eof
17	READDIRPLUS	dir_handle, cookie, verifier, dircount, maxcount	entries with attributes

Weak Cache Consistency (WCC)

NFSv3 introduced Weak Cache Consistency (WCC) data to help clients maintain cache coherence. For operations that modify files, the response includes:

pre-op attributes — File attributes before the operation
post-op attributes — File attributes after the operation

The client can compare pre-op attributes to its cached version:

If they match: cache was valid, update with post-op attributes
If they don't match: another client changed the file, invalidate cache

This isn't perfect consistency (changes between operations aren't detected), but it catches common cases without requiring server-side state.

WCC Data Structure
C
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
/* NFSv3 Weak Cache Consistency Data */
 
/* Pre-operation attributes - lightweight, just for comparison */
struct wcc_attr {
    uint64_t  size;     /* File size before operation */
    nfstime3  mtime;    /* Modification time before */
    nfstime3  ctime;    /* Change time before */
};
 
/* Optional pre-op attributes (may not be available) */
union pre_op_attr switch (bool attributes_follow) {
    case TRUE:  wcc_attr attributes;
    case FALSE: void;
};
 
/* Full post-operation attributes */
union post_op_attr switch (bool attributes_follow) {
    case TRUE:  fattr3 attributes;  /* Complete attribute set */
    case FALSE: void;
};
 
/* WCC data returned with modifying operations */
struct wcc_data {
    pre_op_attr   before;   /* Attributes before change */
    post_op_attr  after;    /* Attributes after change */
};
 
/* Example: Client cache coherence check */
void update_cache_from_wcc(struct inode* inode, struct wcc_data* wcc) {
    if (!wcc->before.attributes_follow)
        return;  /* No pre-op data, can't validate */
    
    /* Compare cached attributes with pre-op */
    if (inode->cached_size != wcc->before.attributes.size ||
        !times_equal(&inode->cached_mtime, &wcc->before.attributes.mtime)) {
        /* Cache is stale - someone else modified the file */
        invalidate_data_cache(inode);
        invalidate_attr_cache(inode);
    }
    
    /* Update cache with new attributes */
    if (wcc->after.attributes_follow) {
        update_cached_attrs(inode, &wcc->after.attributes);
    }
}

File Handle Architecture

File Handle Generation

When a client performs a LOOKUP, CREATE, or other operation that returns a file handle, the server constructs a handle that encodes enough information to identify the file in future requests.

The handle typically includes:

File System ID — Identifies which mounted file system contains the file
Inode/Object Number — The file's unique identifier within that file system
Generation Number — Incremented when inodes are reused, detecting stale handles
Export Verification — Information to verify the file is within an exported path

File Handle Implementation (Linux KNFSD)
C
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
/* Linux NFS file handle structure (simplified from knfsd) */
 
struct knfsd_fh {
    uint8_t  fh_size;           /* Actual handle size (varies) */
    uint8_t  fh_auth_type;      /* Authentication/export type */
    uint8_t  fh_fsid_type;      /* Type of fsid encoding */
    uint8_t  fh_fileid_type;    /* Type of fileid encoding */
    
    /* Variable-size payload follows, contains: */
    union {
        struct {
            /* For most local file systems */
            uint32_t  fsid_major;     /* File system major number */
            uint32_t  fsid_minor;     /* File system minor number */
            uint64_t  inode;          /* Inode number */
            uint32_t  generation;     /* Inode generation count */
            uint32_t  parent_inode;   /* Parent inode (for verification) */
            uint32_t  parent_gen;     /* Parent generation */
        } local;
        
        struct {
            /* For UUID-identified file systems */
            uuid_t    fsid_uuid;      /* File system UUID */
            uint64_t  inode;
            uint32_t  generation;
        } uuid;
    } fh_payload;
};
 
/* Maximum file handle sizes */
#define NFS2_FHSIZE    32    /* NFSv2: exactly 32 bytes */
#define NFS3_FHSIZE    64    /* NFSv3: up to 64 bytes */
#define NFS4_FHSIZE   128    /* NFSv4: up to 128 bytes */
 
/* Handle validation on server */
int nfsd_validate_fh(struct knfsd_fh* fh, struct inode** result) {
    struct super_block* sb;
    struct dentry* dentry;
    struct inode* inode;
    
    /* Step 1: Find the file system */
    sb = nfsd_find_filesystem(fh->fh_payload.local.fsid_major,
                              fh->fh_payload.local.fsid_minor);
    if (!sb)
        return nfserr_stale;
    
    /* Step 2: Look up the inode */
    inode = nfsd_iget(sb, fh->fh_payload.local.inode);
    if (!inode || IS_ERR(inode))
        return nfserr_stale;
    
    /* Step 3: Validate generation number */
    if (inode->i_generation != fh->fh_payload.local.generation) {
        iput(inode);
        return nfserr_stale;  /* Inode reused - stale handle! */
    }
    
    /* Step 4: Verify file is within exported subtree */
    if (!nfsd_check_export_path(fh, inode)) {
        iput(inode);
        return nfserr_stale;
    }
    
    *result = inode;
    return 0;
}

Stale File Handle Detection

The generation number mechanism prevents a dangerous scenario:

Client opens file with inode #1234, gets handle containing (inode=1234, gen=5)
File is deleted on server, inode #1234 is freed
New file is created, assigned inode #1234 (with gen=6)
Client tries to read using old handle
Server checks: inode 1234 exists, but generation is 6, not 5
Server returns NFS3ERR_STALE
Client knows handle is invalid, can report error to application

Without generation numbers, step 4-6 would access the wrong file silently—a catastrophic data corruption scenario.

Handle Persistence Across Reboots

Security Concerns with File Handles

File handles in NFSv2/v3 present security challenges:

Handle Guessing — If handles are predictable (sequential inodes, no generation), attackers might guess valid handles
Handle Leakage — If an attacker obtains a handle, they can access the file directly, bypassing path-based security
Export Verification — Handles must ensure the file is within an exported directory, not just any file the server can access

NFSv4 addresses some of these concerns with volatile file handles, stronger authentication, and pseudofs for export isolation.

Authentication and Security Architecture

Security in NFS has evolved significantly from its original trusted-network design to modern mechanisms supporting secure deployment across untrusted networks.

NFS Authentication Mechanisms
Mechanism	Security Level	Description	Use Case
AUTH_NULL	None	No authentication	Testing only
AUTH_SYS (AUTH_UNIX)	Weak	Client-asserted UID/GID	Trusted LANs
AUTH_DH (Diffie-Hellman)	Medium	Cryptographic (rarely used)	Legacy secure NFS
RPCSEC_GSS / Kerberos	Strong	Full authentication, optional encryption	Enterprise/untrusted networks

AUTH_SYS: The Default (and Weak) Scenario

The default NFS authentication is AUTH_SYS (historically called AUTH_UNIX). With this mechanism:

Client sends its claimed UID, GID, and supplementary groups
Server trusts this completely
Server checks file permissions against these claimed credentials

The security implications are severe:

Anyone with root on a client can claim to be any UID
No cryptographic verification occurs
Network traffic (including credentials) is unencrypted

This is acceptable only in controlled, trusted network environments.

AUTH_SYS Credential Structure
C
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
/* AUTH_SYS (AUTH_UNIX) credentials */
struct authsys_parms {
    uint32_t  aup_time;           /* Timestamp (not verified) */
    char     *aup_machname;       /* Hostname of client */
    uint32_t  aup_uid;            /* User ID (trusted blindly!) */
    uint32_t  aup_gid;            /* Primary group ID */
    uint32_t  aup_len;            /* Number of supplementary groups */
    uint32_t  aup_gids[16];       /* Supplementary group IDs (max 16) */
};
 
/* The server receives this and simply believes it */
/* Example: impersonating root is trivial */
 
/* On malicious client: */
struct authsys_parms fake_creds = {
    .aup_time = time(NULL),
    .aup_machname = "legit-client.example.com",
    .aup_uid = 0,     /* Root! */
    .aup_gid = 0,     /* Root group! */
    .aup_len = 0,
};
 
/* Server cannot distinguish this from legitimate root request
 * unless root_squash is enabled on the export */

RPCSEC_GSS with Kerberos

For secure NFS deployment, RPCSEC_GSS with Kerberos provides:

Authentication Levels:

sec=krb5 — Authentication only. Client identity is cryptographically verified, but data is unencrypted.
sec=krb5i — Authentication + Integrity. Data is signed with cryptographic checksums to detect tampering.
sec=krb5p — Authentication + Integrity + Privacy. Full encryption of all data traffic.

How Kerberos Integration Works:

Users obtain Kerberos tickets from the Key Distribution Center (KDC)
NFS client's rpc.gssd daemon uses tickets to authenticate requests
Server's rpc.svcgssd validates tickets against the KDC
UIDs are mapped via rpc.idmapd using principal names (user@REALM → UID)

# Kerberos-secured NFS mount
mount -t nfs4 -o sec=krb5p server.example.com:/export /mnt

# Client needs valid Kerberos credentials
klist  # Show current tickets
kinit alice@EXAMPLE.COM  # Obtain ticket if needed

Kerberos Deployment Complexity

Linux NFS Implementation

Linux provides one of the most widely-used NFS implementations. Understanding its specific architecture helps with deployment and troubleshooting.

Kernel Module Structure

Linux NFS is implemented primarily in kernel modules:

$ lsmod | grep nfs
nfs                   [NFS client module]
nfsv3                 [NFSv3 support]
nfsv4                 [NFSv4 support]
nfsd                  [NFS server module]
auth_rpcgss           [GSS authentication]
sunrpc                [RPC infrastructure]

Server Implementation (knfsd):

The Linux NFS server uses a kernel-space implementation for performance:

nfsd module handles NFS protocol
exportfs module manages exports
lockd module handles file locking
Multiple kernel threads ([nfsd]) handle concurrent requests

Configuration flows from user-space tools (exportfs, rpc.nfsd) to kernel via /proc/fs/nfsd/.

NFS Server Configuration and Monitoring
Bash
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
#!/bin/bash
# Comprehensive NFS server setup and monitoring
 
# === Server Configuration ===
 
# Configure /etc/exports
cat > /etc/exports << 'EOF'
# Home directories - authenticated access
/export/home    192.168.1.0/24(rw,sync,no_subtree_check,root_squash)
 
# Shared project data
/export/projects  *(rw,sync,no_subtree_check,all_squash,anonuid=1000,anongid=1000)
 
# Read-only repository
/export/repo    *(ro,sync,no_subtree_check)
EOF
 
# Apply export configuration
exportfs -ra
 
# View current exports
exportfs -v
 
# Start NFS services
systemctl start rpcbind
systemctl start nfs-server
 
# Configure number of server threads (/etc/nfs.conf)
# [nfsd]
# threads=16
 
# === Monitoring and Statistics ===
 
# View NFS statistics
nfsstat -s   # Server statistics
nfsstat -c   # Client statistics
 
# Detailed RPC info
cat /proc/net/rpc/nfsd
 
# Current NFS sessions (NFSv4)
cat /proc/fs/nfsd/clients/*/info
 
# === Kernel Parameters ===
 
# View NFS-related kernel params
sysctl -a | grep nfs
 
# Important parameters:
# sunrpc.tcp_slot_table_entries   - Max concurrent RPC requests
# fs.nfs.nfs_congestion_kb        - Congestion control writeback threshold
# fs.nfs.nfs_acl_strict           - Strict ACL enforcement
 
# === Log Analysis ===
 
# Enable NFS debug logging
rpcdebug -m nfsd -s all    # Server debug
rpcdebug -m nfs -s all     # Client debug
 
# Debug messages go to kernel log
dmesg | grep -i nfs
 
# Journal logs
journalctl -u nfs-server -f

The /proc/fs/nfsd Interface

Linux exposes NFS server configuration and status via the /proc filesystem:

File	Purpose
`/proc/fs/nfsd/threads`	Number of nfsd threads (read/write)
`/proc/fs/nfsd/exports`	Currently active exports
`/proc/fs/nfsd/pool_stats`	Statistics per CPU pool
`/proc/fs/nfsd/clients/`	NFSv4 client sessions
`/proc/fs/nfsd/versions`	Enabled NFS versions
`/proc/fs/nfsd/max_block_size`	Maximum read/write block size

NFS Version Selection

Summary: NFS Architecture Mastery

Key Takeaways

•NFS is built on a layered protocol stack — VFS, NFS client/server, RPC, XDR, and TCP/UDP each provide clean abstractions. Understanding each layer helps diagnose issues at the right level.
•Server architecture centers on multiple daemons — rpcbind for service discovery, mountd for access control, nfsd for core operations, and auxiliary services for locking and status monitoring.
•Client architecture emphasizes aggressive caching — Attribute, directory, data, and file handle caches reduce network round-trips but introduce consistency challenges.
•File handles are the stateless key — Generation numbers prevent stale handle attacks. Handle format is opaque to clients but must persist across server reboots.
•Security ranges from weak to strong — AUTH_SYS trusts clients completely; RPCSEC_GSS with Kerberos provides authentication, integrity, and encryption.
•Linux implementation uses kernel-space performance — knfsd runs as kernel threads, with configuration via /proc/fs/nfsd and standard tools.

What's Next

Page Complete