Loading learning content...
The SSH File Transfer Protocol (SFTP) represents the complete reimagining of file transfer for the modern network security landscape. Unlike adaptations that bolt encryption onto legacy protocols, SFTP was designed from its foundation to operate within SSH's secure channel while providing comprehensive filesystem functionality.
SFTP is not FTP over SSH—a common misconception that confuses many engineers. SFTP is an entirely different protocol with its own packet format, command structure, and operational semantics. The 'FTP' in the name reflects its purpose (file transfer), not its lineage. Understanding this distinction is crucial for correctly implementing and troubleshooting SFTP deployments.
This page examines the SFTP protocol in complete technical depth: its architecture, packet structure, the full command repertoire, session lifecycle, and the operational patterns that enable reliable, secure file transfer at enterprise scale.
By the end of this page, you will understand SFTP's subsystem architecture within SSH, the binary packet format and protocol version negotiation, all major SFTP operations and their request/response patterns, file handle management and session state, error handling mechanisms, and the design decisions that make SFTP both powerful and portable across platforms.
SFTP operates as a subsystem within the SSH connection layer. Understanding this architectural position explains SFTP's design choices and operational characteristics.
The SSH Subsystem Model:
SSH provides a multiplexed connection framework where multiple logical channels can operate over a single encrypted TCP connection. Each channel can run different services:
A subsystem is a pre-defined server-side process that the client can invoke by name. When a client requests the 'sftp' subsystem, the SSH server spawns the SFTP server process and connects its standard I/O to the SSH channel.
1234567891011121314151617181920212223242526272829303132333435363738
┌─────────────────────────────────────────────────────────────────────────┐│ CLIENT SYSTEM ││ ┌─────────────────────────────────────────────────────────────────┐ ││ │ SFTP Client Application (FileZilla, WinSCP, sftp command) │ ││ │ └─── Translates user actions to SFTP protocol messages │ ││ └─────────────────────────────────────────────────────────────────┘ ││ │ ││ SFTP Protocol Messages ││ ▼ ││ ┌─────────────────────────────────────────────────────────────────┐ ││ │ SSH Client Library (libssh, OpenSSH, Paramiko) │ ││ │ └─── SSH Connection Layer: Channel for SFTP subsystem │ ││ │ └─── SSH Transport Layer: Encryption, MAC, compression │ ││ └─────────────────────────────────────────────────────────────────┘ ││ │ ││ Encrypted SSH Packets over TCP │└──────────────────────────────────┼──────────────────────────────────────┘ │ Port 22 ▼┌─────────────────────────────────────────────────────────────────────────┐│ SERVER SYSTEM ││ ┌─────────────────────────────────────────────────────────────────┐ ││ │ SSH Server (sshd) │ ││ │ └─── Decrypts/validates packets │ ││ │ └─── Routes channel data to subsystem │ ││ └─────────────────────────────────────────────────────────────────┘ ││ │ ││ Channel stdin/stdout forwarded to subsystem ││ ▼ ││ ┌─────────────────────────────────────────────────────────────────┐ ││ │ SFTP Server Process (sftp-server, internal-sftp) │ ││ │ └─── Parses SFTP protocol messages │ ││ │ └─── Executes filesystem operations │ ││ │ └─── Returns responses through SSH channel │ ││ └─────────────────────────────────────────────────────────────────┘ ││ │ ││ Local Filesystem │└─────────────────────────────────────────────────────────────────────────┘Key Architectural Implications:
This subsystem architecture creates important operational characteristics:
OpenSSH provides two SFTP server implementations. 'sftp-server' is an external binary process. 'internal-sftp' runs within the sshd process itself. The internal version is required for ChrootDirectory functionality (jailing users to specific directories) because external processes cannot access paths outside the chroot. For security-hardened deployments, internal-sftp with chroot is the recommended configuration.
SFTP has evolved through multiple protocol versions, each adding capabilities while maintaining backward compatibility. Understanding version differences is essential for interoperability troubleshooting.
Version History:
| Version | Key Additions | Specification Status | Common Support |
|---|---|---|---|
| Version 1 | Basic operations only | Obsolete | Legacy systems only |
| Version 2 | Rename, file attributes | Obsolete | Rare |
| Version 3 | Symlinks, extended attributes | Draft (2001) | Widest support (de facto standard) |
| Version 4 | UTF-8 filenames, ACLs, improved error codes | Draft (2005) | Growing support |
| Version 5 | Blocking lock, fsync, extended operations | Draft (2006) | Limited support |
| Version 6 | Additional file operations, text hints | Draft (2006) | Limited support |
The Version 3 Dominance:
Despite later versions existing, SFTP version 3 remains the most widely implemented and interoperable choice. This happened because:
Most modern SFTP implementations advertise version 3 for maximum compatibility, regardless of internal capabilities.
Version Negotiation Process:
When the SFTP subsystem starts, client and server exchange version information to establish the protocol version for the session:
123456789101112131415161718192021222324
Client Server │ │ │ ─────────── SSH_FXP_INIT ─────────> │ │ uint32 version = 3 │ Client's highest supported version │ [extension data pairs] │ Optional capability extensions │ │ │ <────────── SSH_FXP_VERSION ─────── │ │ uint32 version = 3 │ Negotiated version (min of both) │ [extension data pairs] │ Server's supported extensions │ │ │ [SFTP operations proceed at v3] │ │ │ Extension Data Format:┌──────────────────┬──────────────────┐│ extension-name │ extension-data ││ (string) │ (string) │└──────────────────┴──────────────────┘ Common Extensions:• "posix-rename@openssh.com" - Atomic rename operation• "statvfs@openssh.com" - Filesystem statistics• "hardlink@openssh.com" - Hard link creation• "fsync@openssh.com" - Synchronize file to diskThe extension mechanism allows servers to advertise capabilities beyond the base protocol. Clients should check for extensions like 'posix-rename@openssh.com' before using advanced features. The OpenSSH extensions in particular are widely supported and enable atomic operations that improve reliability.
SFTP uses a binary packet protocol optimized for efficiency and reliable parsing. Understanding this format is essential for debugging, protocol analysis, and implementing SFTP libraries.
Basic Packet Structure:
Every SFTP packet follows this structure:
123456789101112131415161718192021222324252627282930
SFTP Packet Layout (Network Byte Order / Big Endian): 0 1 2 3 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+| packet_length |+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+| type | |+-+-+-+-+-+-+-+-+ +| |~ data payload ~| |+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ Field Descriptions:┌─────────────────┬──────────┬────────────────────────────────────┐│ Field │ Size │ Description │├─────────────────┼──────────┼────────────────────────────────────┤│ packet_length │ 4 bytes │ Length of type + data (not itself) ││ type │ 1 byte │ Message type code (see table) ││ data │ variable │ Type-specific payload │└─────────────────┴──────────┴────────────────────────────────────┘ Data Types Used in Payloads:• byte - Single byte value• boolean - byte (0 = false, non-zero = true) • uint32 - 4-byte unsigned integer (big-endian)• uint64 - 8-byte unsigned integer (big-endian)• string - uint32 length prefix + byte sequence• ATTRS - File attribute structure (see attributes section)Message Types:
SFTP defines specific message types for requests (client to server), responses (server to client), and shared types:
| Code | Name | Direction | Purpose |
|---|---|---|---|
| 1 | SSH_FXP_INIT | C→S | Initialize protocol, send client version |
| 2 | SSH_FXP_VERSION | S→C | Protocol version response |
| 3 | SSH_FXP_OPEN | C→S | Open or create file |
| 4 | SSH_FXP_CLOSE | C→S | Close file or directory handle |
| 5 | SSH_FXP_READ | C→S | Read data from open file |
| 6 | SSH_FXP_WRITE | C→S | Write data to open file |
| 7 | SSH_FXP_LSTAT | C→S | Get file attributes (no symlink follow) |
| 8 | SSH_FXP_FSTAT | C→S | Get attributes from handle |
| 9 | SSH_FXP_SETSTAT | C→S | Set file attributes by path |
| 10 | SSH_FXP_FSETSTAT | C→S | Set attributes via handle |
| 11 | SSH_FXP_OPENDIR | C→S | Open directory for reading |
| 12 | SSH_FXP_READDIR | C→S | Read directory entries |
| 13 | SSH_FXP_REMOVE | C→S | Delete file |
| 14 | SSH_FXP_MKDIR | C→S | Create directory |
| 15 | SSH_FXP_RMDIR | C→S | Remove directory |
| 16 | SSH_FXP_REALPATH | C→S | Canonicalize path |
| 17 | SSH_FXP_STAT | C→S | Get file attributes (follow symlinks) |
| 18 | SSH_FXP_RENAME | C→S | Rename file or directory |
| 19 | SSH_FXP_READLINK | C→S | Read symlink target |
| 20 | SSH_FXP_SYMLINK | C→S | Create symbolic link |
| 101 | SSH_FXP_STATUS | S→C | Status/error response |
| 102 | SSH_FXP_HANDLE | S→C | Return file/directory handle |
| 103 | SSH_FXP_DATA | S→C | File data response |
| 104 | SSH_FXP_NAME | S→C | Name/path response (directory entries) |
| 105 | SSH_FXP_ATTRS | S→C | File attributes response |
| 200 | SSH_FXP_EXTENDED | C→S | Extended operation request |
| 201 | SSH_FXP_EXTENDED_REPLY | S→C | Extended operation response |
Most request messages include a 'request-id' (uint32) as the first field after the type byte. The server echoes this ID in the response, allowing clients to match responses to requests. This enables pipelining: clients can send multiple requests without waiting for responses, improving throughput on high-latency connections.
SFTP provides a rich set of filesystem operations that go far beyond simple file transfer. Understanding each operation's semantics is crucial for robust client implementations.
File Operations:
The fundamental file transfer workflow involves opening, reading/writing, and closing files:
12345678910111213141516171819202122232425262728293031323334353637383940
File Download (Client → Server → Client): 1. SSH_FXP_OPEN Request: ┌──────────────┬──────────────┬───────────────────────────────┐ │ request-id │ filename │ pflags │ attrs │ │ (uint32) │ (string) │ (uint32) │ (ATTRS structure) │ └──────────────┴──────────────┴───────────┴───────────────────┘ pflags = SSH_FXF_READ (0x00000001) 2. SSH_FXP_HANDLE Response: ┌──────────────┬───────────────────────────────┐ │ request-id │ handle (opaque string) │ └──────────────┴───────────────────────────────┘ Handle is server-generated, opaque to client 3. SSH_FXP_READ Request (repeated): ┌──────────────┬──────────────┬──────────────┬──────────────┐ │ request-id │ handle │ offset │ len │ │ (uint32) │ (string) │ (uint64) │ (uint32) │ └──────────────┴──────────────┴──────────────┴──────────────┘ 4. SSH_FXP_DATA Response: ┌──────────────┬───────────────────────────────┐ │ request-id │ data (string with file bytes) │ └──────────────┴───────────────────────────────┘ Returns less than 'len' bytes if near EOF 5. SSH_FXP_STATUS Response (on EOF): ┌──────────────┬──────────────┬───────────────────────────────┐ │ request-id │ error-code │ error-message │ language │ │ │ SSH_FX_EOF │ (string) │ (string) │ └──────────────┴──────────────┴───────────────────────────────┘ 6. SSH_FXP_CLOSE Request: ┌──────────────┬──────────────┐ │ request-id │ handle │ └──────────────┴──────────────┘Directory Operations:
SFTP provides complete directory manipulation capabilities:
1234567891011121314151617181920212223242526
Directory Listing Workflow: 1. SSH_FXP_OPENDIR("/home/user/documents") → SSH_FXP_HANDLE(handle="xyz123") 2. SSH_FXP_READDIR(handle="xyz123") → SSH_FXP_NAME containing: [ { filename: ".", longname: "drwxr-xr-x 2 user user 4096 ...", attrs: {...} }, { filename: "..", longname: "drwxr-xr-x 8 user user 4096 ...", attrs: {...} }, { filename: "report.pdf", longname: "-rw-r--r-- 1 user user 102400 ...", attrs: {...} }, { filename: "notes.txt", longname: "-rw-r--r-- 1 user user 2048 ...", attrs: {...} }, ] 3. SSH_FXP_READDIR(handle="xyz123") [continue reading] → SSH_FXP_NAME containing more entries... OR → SSH_FXP_STATUS(SSH_FX_EOF) [no more entries] 4. SSH_FXP_CLOSE(handle="xyz123") → SSH_FXP_STATUS(SSH_FX_OK) Note: Each READDIR returns multiple entries (implementation-defined batch size). Client must handle partial batches and continue until EOF.When recursively traversing directories, pipeline OPENDIR requests for child directories while still reading the parent. The request-id mechanism lets you track responses. This parallelization dramatically improves performance on high-latency connections, reducing round-trip delays.
SFTP transmits file metadata through the ATTRS structure, enabling preservation of permissions, timestamps, and ownership during transfers. Understanding this structure is essential for maintaining file integrity across systems.
The ATTRS Structure:
12345678910111213141516171819202122232425262728293031323334353637383940414243
ATTRS Structure (Protocol Version 3): ┌──────────────────────────────────────────────────────────────────────┐│ uint32 flags - Indicates which fields are present │├──────────────────────────────────────────────────────────────────────┤│ uint64 size - File size in bytes (if SSH_FILEXFER_ATTR_SIZE) │├──────────────────────────────────────────────────────────────────────┤│ uint32 uid - Owner user ID (if SSH_FILEXFER_ATTR_UIDGID) ││ uint32 gid - Owner group ID │├──────────────────────────────────────────────────────────────────────┤│ uint32 permissions - POSIX permission bits (if ATTR_PERMISSIONS) │├──────────────────────────────────────────────────────────────────────┤│ uint32 atime - Access time (if SSH_FILEXFER_ATTR_ACMODTIME) ││ uint32 mtime - Modification time │├──────────────────────────────────────────────────────────────────────┤│ uint32 extended_count - Number of extended attribute pairs ││ [extended pairs] - name/value string pairs │└──────────────────────────────────────────────────────────────────────┘ Attribute Flags:┌───────────────────────────────┬────────────┬──────────────────────────┐│ Flag │ Value │ Fields Present │├───────────────────────────────┼────────────┼──────────────────────────┤│ SSH_FILEXFER_ATTR_SIZE │ 0x00000001 │ size ││ SSH_FILEXFER_ATTR_UIDGID │ 0x00000002 │ uid, gid ││ SSH_FILEXFER_ATTR_PERMISSIONS │ 0x00000004 │ permissions ││ SSH_FILEXFER_ATTR_ACMODTIME │ 0x00000008 │ atime, mtime ││ SSH_FILEXFER_ATTR_EXTENDED │ 0x80000000 │ extended_count + pairs │└───────────────────────────────┴────────────┴──────────────────────────┘ Permission Bits (POSIX compatible):┌─────────────┬────────────┬────────────────────────────────────────────┐│ Bits │ Octal │ Meaning │├─────────────┼────────────┼────────────────────────────────────────────┤│ 15-12 │ (type) │ S_IFDIR=0040000 (dir), S_IFREG=0100000 (file)││ 11-9 │ (special) │ setuid, setgid, sticky ││ 8-6 │ 0700 │ Owner: rwx ││ 5-3 │ 0070 │ Group: rwx ││ 2-0 │ 0007 │ Others: rwx │└─────────────┴────────────┴────────────────────────────────────────────┘ Example: Regular file, owner rw, group r, others r permissions = 0100644 (S_IFREG | 0644)Attribute Retrieval Operations:
SFTP provides three ways to retrieve file attributes:
| Operation | Input | Symlink Behavior | Use Case |
|---|---|---|---|
| SSH_FXP_STAT | Path string | Follows symlinks | Get info about target file/directory |
| SSH_FXP_LSTAT | Path string | Returns symlink attributes | Detect symlinks, get link metadata |
| SSH_FXP_FSTAT | Open file handle | N/A (handle to actual file) | Get attributes of already-opened file |
Modifying Attributes:
SFTP allows changing file attributes through SETSTAT and FSETSTAT operations. Common use cases include:
SFTP attributes follow POSIX semantics. Windows clients must translate: NTFS ACLs don't map cleanly to POSIX rwx bits; Windows lacks uid/gid concepts; symbolic link handling differs. Robust implementations should gracefully handle attribute preservation failures when crossing platform boundaries.
SFTP uses structured error responses to communicate operation outcomes. Proper error handling is essential for robust client implementations.
The SSH_FXP_STATUS Response:
Most SFTP operations return SSH_FXP_STATUS on completion (or partial failure). The structure includes:
123456789101112
SSH_FXP_STATUS Response: ┌──────────────────────────────────────────────────────────────────────┐│ byte type = SSH_FXP_STATUS (101) ││ uint32 request-id = Matches original request ││ uint32 error-code = Status code (see table) ││ string error-message = Human-readable description ││ string language = ISO-639 language tag (e.g., "en") │└──────────────────────────────────────────────────────────────────────┘ Note: error-message is for human consumption only. Applications should switch on error-code, not message text.| Code | Name | Meaning | Common Causes |
|---|---|---|---|
| 0 | SSH_FX_OK | Success | Operation completed normally |
| 1 | SSH_FX_EOF | End of file | Read past file end, no more directory entries |
| 2 | SSH_FX_NO_SUCH_FILE | File not found | Path doesn't exist |
| 3 | SSH_FX_PERMISSION_DENIED | Permission denied | Insufficient privileges for operation |
| 4 | SSH_FX_FAILURE | Generic failure | Server error, check message for details |
| 5 | SSH_FX_BAD_MESSAGE | Malformed request | Protocol error, corrupt packet |
| 6 | SSH_FX_NO_CONNECTION | No connection | Session disconnected |
| 7 | SSH_FX_CONNECTION_LOST | Connection lost | Network failure during operation |
| 8 | SSH_FX_OP_UNSUPPORTED | Not supported | Server doesn't support this operation |
| 9 | SSH_FX_INVALID_HANDLE | Invalid handle | Handle expired or never existed |
| 10 | SSH_FX_NO_SUCH_PATH | Path not found | Parent directory doesn't exist |
| 11 | SSH_FX_FILE_ALREADY_EXISTS | File exists | EXCL flag with existing file |
| 12 | SSH_FX_WRITE_PROTECT | Write protected | Read-only filesystem |
| 13 | SSH_FX_NO_MEDIA | No media | Removable media not present |
Error Handling Patterns:
Robust SFTP clients implement several error handling strategies:
While error codes are standardized, error messages vary wildly between implementations. OpenSSH provides helpful messages; other servers may return generic text. Never parse error messages programmatically—use the numeric code.
SFTP's request-response protocol introduces inherent latency challenges. On high-latency links (WAN, satellite, intercontinental), naive implementations perform poorly. Understanding optimization techniques is crucial for production deployments.
The Latency Problem:
Consider downloading a 100MB file with 50ms round-trip latency:
123456789101112131415161718192021222324
Scenario: 100MB file, 50ms RTT, 64KB read chunks Naive Implementation (one request, wait for response, repeat): Chunks needed: 100MB / 64KB = 1,600 chunks Time per chunk: 50ms (RTT) + transfer time Assuming 100Mbps bandwidth: Transfer time per 64KB: 64KB / 100Mbps = ~5ms Total per chunk: 50ms + 5ms = 55ms Total time: 1,600 × 55ms = 88 seconds Effective throughput: 100MB / 88s ≈ 9 Mbps (9% of bandwidth!) Pipelined Implementation (multiple requests in flight): With 16 requests in parallel: Waiting for first response: 50ms Subsequent responses arrive continuously Effective time: ~50ms + (100MB / 100Mbps) = 50ms + 8s ≈ 8 seconds Effective throughput: 100MB / 8s ≈ 100 Mbps (full bandwidth!) Key Insight: Pipelining amortizes latency across multiple requestsPipelining Implementation:
Effective pipelining requires careful request management:
OpenSSH's sftp command supports '-R num_requests' to set the outstanding request limit (default: 64). For high-latency links, increase this value. For batch operations, '-b' flag reads commands from a file, enabling further optimization. The '-l limit' flag can throttle bandwidth if needed.
We've comprehensively examined the SFTP protocol architecture and operations. Let's consolidate the essential knowledge.
What's Next:
Having mastered SFTP's architecture and protocol, the next page examines SCP (Secure Copy Protocol)—its simpler design philosophy, protocol mechanics, current deprecation status, and scenarios where understanding SCP remains valuable despite its supersession by SFTP.
You now possess deep understanding of the SFTP protocol—from subsystem architecture through binary packet format to performance optimization. This knowledge enables you to effectively deploy, troubleshoot, and optimize SFTP in production environments.