Ftp Operation - Learning Module

Loading content...

0/240

File Transfer

The Core Purpose: Moving Files

Everything in FTP—login processes, directory navigation, mode settings—serves a single ultimate purpose: transferring files between client and server. This is where the File Transfer Protocol earns its name, where the dual-channel architecture becomes essential, and where understanding the nuances of data connections separates reliable implementations from fragile ones.

File transfer in FTP involves coordinating two TCP connections: the control channel we've been using for commands, and a separate data channel that carries the actual file content. This separation, revolutionary when FTP was designed in the 1970s, enables the protocol to transfer files of any size while maintaining independent command interaction.

What You Will Learn

By the end of this page, you will understand FTP file transfer from the ground up: data connection establishment in active and passive modes, file retrieval (RETR) and storage (STOR) commands, progress monitoring, resumable transfers (REST), and practical patterns for reliable file operations.

The Dual-Channel Architecture

FTP uniquely separates control signaling from data transfer using two distinct TCP connections. This architecture, while more complex than single-connection protocols, provides significant advantages for file transfer scenarios.

Control Connection:

Established on port 21
Persists for entire session
Carries all FTP commands and responses
Limited bandwidth usage (text-based commands)

Data Connection:

Established on-demand for transfers
Closed after each transfer completes
Carries file contents or directory listings
Can use different ports per operation

Converting Mermaid diagram...

Benefits of Separation:

Out-of-Band Control: Send ABOR (abort) command during transfer without waiting for data to complete
Multiple Transfers: Negotiate next transfer while current one completes
Flexible Data Routing: Data can flow through different network paths than control
Stateless Data Channel: Each transfer starts fresh; no accumulated state issues

The Price of Complexity:

Firewall Challenges: Multiple ports to open and track
NAT Issues: Address/port translation breaks embedded IP information
Connection Overhead: New TCP connection for each transfer
State Synchronization: Must coordinate between control and data channels

Historical Context

The dual-channel design made sense in 1971 when FTP was designed. Networks were simpler, NAT didn't exist, and firewalls were minimal. Modern protocols like HTTP wrap everything in a single connection. FTP's architecture, while elegant, creates challenges in today's network environments.

Active Mode Data Connections (PORT)

In active mode, the client opens a listening port and instructs the server to connect to it. The client is 'active' in the sense of receiving the incoming connection. This was the original FTP data connection method.

PORT Command Syntax:

PORT <h1>,<h2>,<h3>,<h4>,<p1>,<p2><CRLF>

Where:

h1,h2,h3,h4 form the IPv4 address (e.g., 192,168,1,100)
p1,p2 encode the port number as: (p1 × 256) + p2

active-mode-sequence.txt
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
# Active Mode File Transfer Sequence
 
# Step 1: Client creates listening socket on local port (e.g., 50000)
# Client IP: 192.168.1.100
# Port calculation: 50000 = 195 × 256 + 80 = (195, 80)
 
→ PORT 192,168,1,100,195,80
← 200 PORT command successful
 
# Step 2: Client initiates transfer
→ RETR document.pdf
← 150 Opening BINARY mode data connection for document.pdf (1234567 bytes)
 
# Step 3: Server connects FROM port 20 TO client port 50000
# [Server 203.0.113.50:20 → Client 192.168.1.100:50000]
 
# Step 4: Data flows from server to client over data connection
# [1234567 bytes transferred...]
 
# Step 5: Server closes data connection
# Step 6: Server sends completion status on control channel
← 226 Transfer complete
 
# ============================================
# PORT Command Port Calculation Examples
 
# Port 1024 = 4 × 256 + 0 = 4,0
PORT 192,168,1,100,4,0     # Port 1024
 
# Port 21000 = 82 × 256 + 8 = 82,8  (82×256=20992, +8=21000)
PORT 192,168,1,100,82,8    # Port 21000
 
# Port 65535 = 255 × 256 + 255 = 255,255
PORT 192,168,1,100,255,255 # Port 65535

Active Mode Challenges:

Active mode, while conceptually simple, faces significant problems in modern networks:

Active Mode Problems

•Client Firewall Blocking — Inbound connections to client ports are typically blocked by firewalls. The server's connection attempt fails at the client's firewall.
•NAT Incompatibility — If client is behind NAT, the private IP (192.168.x.x) in PORT command is unreachable from the server. Server connects to wrong address.
•Port Conflicts — Client must open specific ports; may conflict with other services or security policies.
•Server Port 20 Requirement — Traditional servers use source port 20, but some firewalls may still block incoming connections regardless of source port.
•Dynamic Port Allocation — Client must choose and manage ephemeral ports, adding implementation complexity.

Avoid Active Mode

Active mode is effectively deprecated for most use cases. It fails behind NAT, struggles with firewalls, and creates security concerns (servers initiating connections to clients). Use passive mode unless your specific network topology requires active mode.

Passive Mode Data Connections (PASV/EPSV)

In passive mode, the server opens a listening port and tells the client to connect to it. The server is 'passive' in accepting the incoming connection. This mode works far better with firewalls and NAT.

PASV Command:

PASV<CRLF>

PASV Response Format:

227 Entering Passive Mode (h1,h2,h3,h4,p1,p2)

The address and port encoding matches the PORT command format.

passive-mode-sequence.txt
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
# Passive Mode File Transfer Sequence
 
# Step 1: Request passive mode
→ PASV
← 227 Entering Passive Mode (203,0,113,50,195,88)
 
# Decode: IP = 203.0.113.50, Port = 195×256 + 88 = 50008
 
# Step 2: Client connects to server's passive port
# [Client → Server 203.0.113.50:50008]
# Connection established
 
# Step 3: Client requests file (on control channel)
→ RETR largefile.zip
← 150 Opening BINARY mode data connection for largefile.zip (524288000 bytes)
 
# Step 4: Data flows from server to client over data connection
# [524288000 bytes transferred...]
 
# Step 5: Data connection closes
# Step 6: Completion notification on control channel  
← 226 Transfer complete
 
# ============================================
# For uploads (STOR), same process but data flows client→server
 
→ PASV
← 227 Entering Passive Mode (203,0,113,50,196,44)
# Port = 196×256 + 44 = 50220
 
→ STOR upload.zip
← 150 Opening BINARY mode data connection for upload.zip
 
# Client sends file data over data connection
# [Client → Server: file bytes]
 
← 226 Transfer complete

Extended Passive Mode (EPSV):

For IPv6 support and simpler parsing, RFC 2428 introduced EPSV:

→ EPSV
← 229 Entering Extended Passive Mode (|||50008|)

EPSV returns only the port number; the client connects to the same IP as the control connection. This avoids address encoding issues and works seamlessly with IPv6.

PASV vs EPSV Comparison
Feature	PASV	EPSV
RFC	RFC 959	RFC 2428
IPv6 Support	No (IPv4 only)	Yes
Response Format	IP and port encoded	Port only in delimiters
Server IP	Explicit in response	Same as control connection
Parsing	Six comma-separated values	Port between \|\|\|delimiters\|
NAT Transparency	May return wrong IP	No IP to be wrong

Best Practice

Try EPSV first; fall back to PASV if the server doesn't support it. EPSV avoids IP address issues (critical when servers are behind NAT that rewrites addresses incorrectly) and is required for IPv6 FTP connections.

RETR: Retrieving Files

The RETR (Retrieve) command downloads a file from the server to the client. Before issuing RETR, you must have an established data connection (via PASV/PORT).

Command Syntax:

RETR <SP> <pathname> <CRLF>

The pathname can be relative to current directory or absolute.

RETR Command Reply Codes
Reply Code	Meaning	When Occurs
125	Data connection already open	Server reusing existing connection (rare)
150	File status okay; opening data connection	Normal success; transfer beginning
226	Closing data connection; transfer complete	Transfer finished successfully
250	Requested file action okay, completed	Alternative success code
421	Service not available	Server problem
425	Can't open data connection	Firewall/NAT blocking
426	Connection closed; transfer aborted	Network error during transfer
450	File unavailable (busy)	File locked by another process
451	Local error in processing	Server filesystem error
500	Syntax error	Command malformed
501	Syntax error in parameters	Invalid filename
530	Not logged in	Session expired
550	File not found or permission denied	File doesn't exist or no read access

retr-complete-example.txt
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
# Complete File Download Sequence
 
# 1. Navigate to desired directory
→ CWD /projects/reports
← 250 Directory changed
 
# 2. Set binary mode for non-text files
→ TYPE I
← 200 Switching to Binary mode
 
# 3. Check file size (optional, for progress reporting)
→ SIZE quarterly-report.pdf
← 213 2548731
 
# 4. Enter passive mode
→ PASV
← 227 Entering Passive Mode (203,0,113,50,196,180)
 
# 5. Client connects to 203.0.113.50:50356 (196×256+180)
# [TCP connection established]
 
# 6. Request file
→ RETR quarterly-report.pdf
← 150 Opening BINARY mode data connection for quarterly-report.pdf (2548731 bytes)
 
# 7. Server sends file over data connection
# Client reads: [2,548,731 bytes in chunks...]
 
# 8. Server closes data connection when complete
# [TCP FIN/ACK exchange]
 
# 9. Server confirms completion
← 226 Transfer complete
 
# ============================================
# Error Examples
 
# File not found
→ RETR nonexistent.pdf
← 550 nonexistent.pdf: No such file or directory
 
# Permission denied
→ RETR /etc/shadow
← 550 /etc/shadow: Permission denied
 
# No data connection established
→ RETR file.txt
← 425 Use PASV or PORT first

Transfer Mode Matters

Always set appropriate transfer type (TYPE I for binary, TYPE A for ASCII) before RETR. Binary mode transfers exact bytes; ASCII mode converts line endings between platforms. Using wrong mode corrupts files—binary files become unusable, text files get mangled line endings.

STOR: Storing (Uploading) Files

The STOR (Store) command uploads a file from the client to the server. Like RETR, it requires a data connection to be established first.

Command Syntax:

STOR <SP> <pathname> <CRLF>

Related Commands:

STOU (Store Unique): Server generates unique filename, preventing overwrites
APPE (Append): Adds data to existing file instead of replacing

STOR Command Reply Codes
Reply Code	Meaning	When Occurs
125	Data connection already open	Transfer can begin
150	File status okay; opening connection	Normal; server ready to receive
226	Transfer complete	File successfully written
250	Requested file action okay	Alternative success
421	Service not available	Server problem
425	Can't open data connection	Connection issue
426	Connection closed; transfer aborted	Network error during upload
450	File unavailable (locked)	Destination locked
451	Local error; action aborted	Server write error
452	Insufficient storage space	Disk full or quota exceeded
532	Need account for storing files	ACCT required
550	Permission denied	No write permission to directory
552	Exceeded storage allocation	User quota exceeded
553	File name not allowed	Invalid characters or path

stor-complete-example.txt
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
# Complete File Upload Sequence
 
# 1. Navigate to target directory
→ CWD /incoming/uploads
← 250 Directory changed
 
# 2. Set binary mode
→ TYPE I
← 200 Switching to Binary mode
 
# 3. Enter passive mode
→ PASV  
← 227 Entering Passive Mode (203,0,113,50,197,22)
 
# 4. Client connects to 203.0.113.50:50454
# [TCP connection established]
 
# 5. Initiate upload
→ STOR project-backup.tar.gz
← 150 Opening BINARY mode data connection for project-backup.tar.gz
 
# 6. Client sends file over data connection
# [15,728,640 bytes sent in chunks...]
 
# 7. Client closes data connection when complete
# [TCP FIN sent]
 
# 8. Server confirms receipt
← 226 Transfer complete; 15728640 bytes received
 
# ============================================
# Store Unique Example
→ PASV
← 227 Entering Passive Mode (203,0,113,50,197,50)
→ STOU report.pdf
← 150 Opening BINARY mode data connection for report.pdf.1
# [data transfer]
← 226 Transfer complete; unique filename is report.pdf.1
 
# ============================================
# Append Example (add to existing file)
→ PASV
← 227 Entering Passive Mode (203,0,113,50,197,78)
→ APPE logfile.txt
← 150 Opening ASCII mode data connection for logfile.txt
# [new log entries sent]
← 226 Transfer complete
 
# ============================================
# Error: Quota Exceeded
→ STOR massive-dataset.zip
← 552 Disk quota exceeded

Atomic Uploads

STOR overwrites existing files without warning. For atomic updates, upload to a temporary filename then rename (RNFR/RNTO commands). This prevents serving partial files if the upload fails midway. Many production systems use this pattern: STOR file.tmp → RNFR file.tmp → RNTO file.final.

Resumable Transfers (REST)

Large file transfers over unreliable networks often fail partway through. The REST (Restart) command enables resuming transfers from a specific byte offset, avoiding the need to re-transfer already-received data.

Command Syntax:

REST <SP> <byte-offset> <CRLF>

REST sets a marker for the next transfer command. It doesn't transfer anything itself—it configures where the next RETR or STOR will start.

REST Command Reply Codes
Reply Code	Meaning	Notes
350	Requested action pending	Offset accepted; ready for RETR/STOR
500	Syntax error	Command not recognized
501	Syntax error in parameters	Invalid offset format
502	Command not implemented	Server doesn't support REST
503	Bad sequence of commands	REST after data connection? (varies)
530	Not logged in	Session expired

resumable-download.txt
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
# Resumable Download Scenario
 
# Original attempt - Failed at 50MB of 100MB file
# Client has 52,428,800 bytes saved locally
 
# Step 1: Verify server has REST support
→ FEAT
← 211-Features:
←  SIZE
←  REST STREAM
←  MDTM
←  PASV
← 211 End
 
# Step 2: Check current file size matches expected
→ SIZE bigfile.iso
← 213 104857600    # 100 MB
 
# Step 3: Verify local file size
# Client checks: local_file_size = 52428800
 
# Step 4: Set restart offset
→ REST 52428800
← 350 Restart position accepted (52428800)
 
# Step 5: Enter passive mode
→ PASV
← 227 Entering Passive Mode (203,0,113,50,198,100)
# Connect to data port...
 
# Step 6: RETR resumes from offset
→ RETR bigfile.iso
← 150 Opening BINARY mode data connection for bigfile.iso (104857600 bytes)
# NOTE: Server may report full size or remaining size
 
# Step 7: Server sends bytes starting from 52428800
# Client appends to local file: [52,428,800 additional bytes]
 
← 226 Transfer complete
 
# ============================================
# Resumable Upload Example
 
# Upload interrupted at 25 MB of 50 MB file
# Server has 26,214,400 bytes of partial upload
 
# Check partial file on server
→ SIZE partial-upload.zip
← 213 26214400
 
# Set restart offset
→ REST 26214400
← 350 Restart position accepted
 
# Passive mode and upload
→ PASV
← 227 Entering Passive Mode (203,0,113,50,198,128)
→ STOR partial-upload.zip
← 150 Opening BINARY mode data connection
 
# Client seeks to offset in local file
# Client sends bytes from position 26214400 onward
 
← 226 Transfer complete

Implementation Considerations:

For Downloads:

Check if local partial file exists
Get remote file size with SIZE command
Verify local size < remote size (otherwise file is complete or corrupted)
Set REST to local file size
Open local file in append mode
RETR and write received bytes

For Uploads:

Check if remote partial file exists (SIZE command)
Verify remote size < local size
Set REST to remote file size
Seek local file to that offset
STOR and send remaining bytes

Integrity Verification:

REST/resume doesn't verify that the partial transfer is correct—just that sizes match. For critical transfers, compute checksums after completion:

→ XMD5 bigfile.iso    # Non-standard but common
← 213 d41d8cd98f00b204e9800998ecf8427e

→ XSHA256 bigfile.iso  # Even less standard

Most servers don't support these; verify checksums through other means (companion .md5 files, etc.).

REST Must Be Binary

REST byte offsets only make sense in binary (TYPE I) mode. In ASCII mode, line ending conversions change byte positions unpredictably. Always use TYPE I before REST-based resume operations.

Progress Monitoring and File Information

Good user experience requires progress indication during transfers. FTP provides commands to gather file information before transfer begins, enabling accurate progress bars and time estimates.

SIZE Command:

Returns the file size in bytes (when using binary mode):

SIZE <SP> <pathname> <CRLF>

progress-monitoring.txt
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
# Getting File Information
 
# SIZE command
→ SIZE largefile.zip
← 213 1073741824    # 1 GB in bytes
 
# SIZE requires TYPE I for accurate byte count
→ TYPE A
← 200 Switching to ASCII mode
→ SIZE textfile.txt
← 550 SIZE command could not complete.  # Many servers fail in ASCII mode
 
→ TYPE I
← 200 Switching to Binary mode
→ SIZE textfile.txt
← 213 15234
 
# ============================================
# MDTM - Modification Time
 
→ MDTM report.pdf
← 213 20250115143022    # YYYYMMDDhhmmss format
# = January 15, 2025 at 14:30:22
 
→ MDTM /archive/old-backup.tar
← 213 20230601000000
 
# ============================================
# Combining for Download Preparation
 
→ TYPE I
← 200 Switching to Binary mode
 
→ SIZE project.tar.gz  
← 213 536870912    # 512 MB
 
→ MDTM project.tar.gz
← 213 20250114120000    # Check if newer than local copy
 
→ PASV
← 227 Entering Passive Mode (203,0,113,50,199,50)
 
# Client now knows:
# - File is 536,870,912 bytes
# - Modified Jan 14, 2025 at 12:00:00
# - Can compute ETA based on bandwidth
# - Can check if local copy is current
 
→ RETR project.tar.gz
← 150 Opening BINARY mode data connection

Progress Calculation:

With file size known, calculate progress during transfer:

total_size = 536870912  # From SIZE command
bytes_received = 0
start_time = time.time()

while data := read_from_data_connection():
    bytes_received += len(data)
    write_to_file(data)
    
    # Progress percentage
    progress = (bytes_received / total_size) * 100
    
    # Speed calculation
    elapsed = time.time() - start_time
    speed = bytes_received / elapsed  # bytes/second
    
    # ETA
    remaining_bytes = total_size - bytes_received
    eta_seconds = remaining_bytes / speed if speed > 0 else 0
    
    print(f"{progress:.1f}% | {speed/1024/1024:.1f} MB/s | ETA: {eta_seconds:.0f}s")

File Information Commands
Command	Purpose	Response Format
SIZE	Get file size in bytes	213 <byte-count>
MDTM	Get modification time	213 YYYYMMDDhhmmss
MLST	Machine-parseable file facts	250- facts;facts;... filename
STAT <file>	Status of file/transfer	211/212/213 multi-line status

When SIZE Isn't Available

Some servers don't support SIZE, or SIZE fails for certain file types. In these cases, the 150 response often includes the file size: '150 Opening BINARY mode data connection for file (12345 bytes)'. Parse this as fallback. If even that's unavailable, show an indeterminate progress indicator.

Aborting Transfers and Handling Timeouts

Transfers don't always complete successfully. Users cancel downloads, networks fail, servers become unresponsive. Handling these scenarios gracefully is essential for robust FTP clients.

ABOR Command:

The ABOR (Abort) command interrupts an in-progress transfer:

ABOR<CRLF>

Because the data channel may be blocking, FTP specifies a special sequence for sending ABOR:

abort-sequences.txt
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
# Standard ABOR Sequence (when control channel is responsive)
 
# Transfer in progress...
← 150 Opening BINARY mode data connection
# [data flowing on data channel]
 
# User cancels - send ABOR on control channel
→ ABOR
← 426 Transfer aborted.
← 226 Abort successful.
 
# OR
→ ABOR
← 225 Abort successful but file transfer was complete.
← 226 Transfer complete.
 
# ============================================
# Telnet Interrupt Signals (when control might be blocked)
 
# If data transfer is consuming all I/O, send:
# 1. Telnet IP (Interrupt Process): 0xFF 0xF4
# 2. Telnet DM (Data Mark): 0xFF 0xF2 (in TCP urgent data)
# 3. ABOR command
 
# This ensures ABOR is processed even if buffers are full
[TCP URG] 0xFF 0xF4 0xFF 0xF2 ABOR
← 426 Connection closed; transfer aborted
 
# ============================================
# Handling Hung Transfers
 
# No response for > 60 seconds during data transfer
# Options:
# 1. Close data connection unilaterally
# 2. Send ABOR and wait briefly  
# 3. Close control connection if ABOR doesn't respond
# 4. Full disconnect and reconnect
 
# Example timeout handling:
→ RETR stuck-file.bin
← 150 Opening connection
# [no data arrives for 120 seconds]
 
→ ABOR
← [no response for 30 seconds]
 
# Give up - close sockets
[TCP RST sent]
# Reconnect and start new session

Timeout Strategies:

Timeout Type	Typical Value	Action on Expiry
Connection timeout	30-60s	Cancel connection attempt
Command response	60-120s	Retry or abort operation
Data transfer idle	120-300s	Abort and resume or fail
Session idle	300-900s	Server may disconnect

Keepalive for Long Transfers:

During very long transfers, the control connection sits idle. Some servers/firewalls may close idle connections. Solutions:

NOOP Command: Send NOOP periodically to keep control channel active
TCP Keepalives: Enable OS-level TCP keepalives on control socket
Data Progress: Rely on data channel activity to signal overall session health

Incomplete Abort

If ABOR fails to cleanly terminate a transfer, the server may leave partial files. For uploads, this means corrupt destination files. Always verify file integrity after interrupted transfers, and consider using temporary filenames with post-transfer rename for atomicity.

Practical Transfer Implementation

Putting together everything we've learned, here's a production-quality file transfer implementation that handles the complexities we've discussed:

robust-transfer.py
Python
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
import socket
import time
from dataclasses import dataclass
from typing import Optional, Callable, BinaryIO
from pathlib import Path
 
@dataclass
class TransferProgress:
    bytes_transferred: int
    total_bytes: Optional[int]
    elapsed_seconds: float
    
    @property
    def percentage(self) -> Optional[float]:
        if self.total_bytes and self.total_bytes > 0:
            return (self.bytes_transferred / self.total_bytes) * 100
        return None
    
    @property
    def speed_bps(self) -> float:
        if self.elapsed_seconds > 0:
            return self.bytes_transferred / self.elapsed_seconds
        return 0
    
    @property
    def eta_seconds(self) -> Optional[float]:
        if self.total_bytes and self.speed_bps > 0:
            remaining = self.total_bytes - self.bytes_transferred
            return remaining / self.speed_bps
        return None
 
ProgressCallback = Callable[[TransferProgress], None]
 
class FTPTransfer:
    """
    Robust FTP file transfer with resume, progress, and error handling.
    """
    
    def __init__(self, control_socket, buffer_size: int = 65536):
        self.control = control_socket
        self.buffer_size = buffer_size
        self.data_socket: Optional[socket.socket] = None
    
    def _send_command(self, cmd: str) -> str:
        """Send command and return response."""
        self.control.sendall(f"{cmd}\r
".encode())
        return self._receive_response()
    
    def _receive_response(self) -> str:
        """Receive complete FTP response."""
        response = ""
        while True:
            chunk = self.control.recv(4096).decode()
            response += chunk
            # Check for end of response
            lines = response.split('\r
')
            for line in lines:
                if len(line) >= 4 and line[3] == ' ':
                    return response.strip()
        return response.strip()
    
    def _enter_passive_mode(self) -> tuple:
        """Enter passive mode, return (host, port)."""
        # Try EPSV first
        response = self._send_command("EPSV")
        if response.startswith("229"):
            # Parse |||port|
            start = response.find("|||")
            end = response.find("|", start + 3)
            port = int(response[start+3:end])
            # Use same host as control connection
            host = self.control.getpeername()[0]
            return (host, port)
        
        # Fall back to PASV
        response = self._send_command("PASV")
        if not response.startswith("227"):
            raise RuntimeError(f"PASV failed: {response}")
        
        # Parse (h1,h2,h3,h4,p1,p2)
        start = response.find("(")
        end = response.find(")")
        numbers = response[start+1:end].split(",")
        host = ".".join(numbers[:4])
        port = int(numbers[4]) * 256 + int(numbers[5])
        return (host, port)
    
    def _get_file_size(self, remote_path: str) -> Optional[int]:
        """Get remote file size, None if unavailable."""
        response = self._send_command(f"SIZE {remote_path}")
        if response.startswith("213"):
            return int(response.split()[1])
        return None
    
    def download(
        self,
        remote_path: str,
        local_path: Path,
        progress_callback: Optional[ProgressCallback] = None,
        resume: bool = True
    ) -> bool:
        """
        Download file with optional resume and progress reporting.
        Returns True on success, False on failure.
        """
        # Set binary mode
        self._send_command("TYPE I")
        
        # Get remote file size for progress
        remote_size = self._get_file_size(remote_path)
        
        # Check for resume
        start_offset = 0
        mode = "wb"
        
        if resume and local_path.exists():
            local_size = local_path.stat().st_size
            if remote_size and local_size < remote_size:
                start_offset = local_size
                mode = "ab"
                resp = self._send_command(f"REST {start_offset}")
                if not resp.startswith("350"):
                    # REST not supported, start over
                    start_offset = 0
                    mode = "wb"
        
        # Enter passive mode and connect
        host, port = self._enter_passive_mode()
        self.data_socket = socket.create_connection((host, port), timeout=60)
        
        # Request file
        response = self._send_command(f"RETR {remote_path}")
        if not (response.startswith("150") or response.startswith("125")):
            self.data_socket.close()
            return False
        
        # If we didn't get size from SIZE, try parsing 150 response
        if remote_size is None:
            # Try to parse "150 ... (12345 bytes)"
            if "(" in response and "bytes)" in response:
                try:
                    size_str = response.split("(")[1].split()[0]
                    remote_size = int(size_str)
                except:
                    pass
        
        # Transfer data
        bytes_received = start_offset
        start_time = time.time()
        
        try:
            with open(local_path, mode) as f:
                while True:
                    data = self.data_socket.recv(self.buffer_size)
                    if not data:
                        break
                    
                    f.write(data)
                    bytes_received += len(data)
                    
                    if progress_callback:
                        progress = TransferProgress(
                            bytes_transferred=bytes_received,
                            total_bytes=remote_size,
                            elapsed_seconds=time.time() - start_time
                        )
                        progress_callback(progress)
        
        finally:
            self.data_socket.close()
        
        # Get completion response
        response = self._receive_response()
        return response.startswith("226") or response.startswith("250")
    
    def upload(
        self,
        local_path: Path,
        remote_path: str,
        progress_callback: Optional[ProgressCallback] = None,
        resume: bool = True,
        atomic: bool = True
    ) -> bool:
        """
        Upload file with optional resume, progress, and atomic writes.
        atomic=True writes to temp file then renames on completion.
        """
        if not local_path.exists():
            raise FileNotFoundError(f"Local file not found: {local_path}")
        
        local_size = local_path.stat().st_size
        
        # Set binary mode
        self._send_command("TYPE I")
        
        # Determine target path (temp for atomic)
        actual_remote = f"{remote_path}.tmp" if atomic else remote_path
        
        # Check for resume
        start_offset = 0
        if resume:
            remote_size = self._get_file_size(actual_remote)
            if remote_size and remote_size < local_size:
                resp = self._send_command(f"REST {remote_size}")
                if resp.startswith("350"):
                    start_offset = remote_size
        
        # Enter passive mode
        host, port = self._enter_passive_mode()
        self.data_socket = socket.create_connection((host, port), timeout=60)
        
        # Initiate upload
        response = self._send_command(f"STOR {actual_remote}")
        if not (response.startswith("150") or response.startswith("125")):
            self.data_socket.close()
            return False
        
        # Transfer data
        bytes_sent = start_offset
        start_time = time.time()
        
        try:
            with open(local_path, "rb") as f:
                if start_offset > 0:
                    f.seek(start_offset)
                
                while True:
                    data = f.read(self.buffer_size)
                    if not data:
                        break
                    
                    self.data_socket.sendall(data)
                    bytes_sent += len(data)
                    
                    if progress_callback:
                        progress = TransferProgress(
                            bytes_transferred=bytes_sent,
                            total_bytes=local_size,
                            elapsed_seconds=time.time() - start_time
                        )
                        progress_callback(progress)
        
        finally:
            self.data_socket.close()
        
        # Get completion response
        response = self._receive_response()
        success = response.startswith("226") or response.startswith("250")
        
        # Atomic rename if requested
        if success and atomic:
            self._send_command(f"RNFR {actual_remote}")
            response = self._send_command(f"RNTO {remote_path}")
            success = response.startswith("250")
        
        return success
 
 
# Usage example
def print_progress(p: TransferProgress):
    if p.percentage is not None:
        eta = f"ETA: {p.eta_seconds:.0f}s" if p.eta_seconds else ""
        print(f"\r{p.percentage:.1f}% | "
              f"{p.speed_bps/1024/1024:.1f} MB/s | "
              f"{eta}", end="")
    else:
        print(f"\r{p.bytes_transferred:,} bytes | "
              f"{p.speed_bps/1024:.1f} KB/s", end="")

Production Usage

This implementation demonstrates concepts. For production, use battle-tested libraries like ftplib (Python), basic-ftp (Node.js), or Apache Commons Net (Java). They handle protocol edge cases, encoding issues, and server quirks accumulated over years of real-world usage.

Summary: File Transfer

We've covered the complete file transfer mechanism in FTP, from dual-channel architecture through practical implementation patterns. Let's consolidate the key takeaways:

Key Takeaways

•Dual-channel design — FTP uses separate control (commands) and data (file contents) connections, enabling out-of-band control during transfers.
•Avoid active mode — Active mode (PORT) fails with NAT and firewalls; always prefer passive mode (PASV/EPSV).
•EPSV over PASV — Use extended passive mode first for IPv6 support and to avoid NAT-related IP address issues.
•RETR downloads, STOR uploads — Both require prior data connection setup; verify success with 226/250 responses.
•REST enables resume — Set byte offset before transfer to resume interrupted operations; requires binary mode.
•SIZE before transfer — Get file size for progress calculation and resume validation.
•ABOR for cancellation — Cleanly abort in-progress transfers; may require telnet interrupt sequences.
•Atomic uploads — Use temporary filename + rename (RNFR/RNTO) to prevent serving partial files.

What's Next:

File transfer works differently depending on the data type. The next page explores ASCII vs Binary transfer modes—when to use each, how they differ, and the consequences of choosing incorrectly.

Page Complete

You now understand FTP file transfer mechanics including dual-channel architecture, active/passive modes, RETR/STOR commands, resumable transfers, and progress monitoring. Next, we'll explore the critical difference between ASCII and binary transfer modes.

File Transfer

The Core Purpose: Moving Files

What You Will Learn

The Dual-Channel Architecture

Control Connection:

Established on port 21
Persists for entire session
Carries all FTP commands and responses
Limited bandwidth usage (text-based commands)

Data Connection:

Established on-demand for transfers
Closed after each transfer completes
Carries file contents or directory listings
Can use different ports per operation

Converting Mermaid diagram...

Benefits of Separation:

Out-of-Band Control: Send ABOR (abort) command during transfer without waiting for data to complete
Multiple Transfers: Negotiate next transfer while current one completes
Flexible Data Routing: Data can flow through different network paths than control
Stateless Data Channel: Each transfer starts fresh; no accumulated state issues

The Price of Complexity:

Firewall Challenges: Multiple ports to open and track
NAT Issues: Address/port translation breaks embedded IP information
Connection Overhead: New TCP connection for each transfer
State Synchronization: Must coordinate between control and data channels

Historical Context

Active Mode Data Connections (PORT)

PORT Command Syntax:

PORT <h1>,<h2>,<h3>,<h4>,<p1>,<p2><CRLF>

Where:

h1,h2,h3,h4 form the IPv4 address (e.g., 192,168,1,100)
p1,p2 encode the port number as: (p1 × 256) + p2

active-mode-sequence.txt
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
# Active Mode File Transfer Sequence
 
# Step 1: Client creates listening socket on local port (e.g., 50000)
# Client IP: 192.168.1.100
# Port calculation: 50000 = 195 × 256 + 80 = (195, 80)
 
→ PORT 192,168,1,100,195,80
← 200 PORT command successful
 
# Step 2: Client initiates transfer
→ RETR document.pdf
← 150 Opening BINARY mode data connection for document.pdf (1234567 bytes)
 
# Step 3: Server connects FROM port 20 TO client port 50000
# [Server 203.0.113.50:20 → Client 192.168.1.100:50000]
 
# Step 4: Data flows from server to client over data connection
# [1234567 bytes transferred...]
 
# Step 5: Server closes data connection
# Step 6: Server sends completion status on control channel
← 226 Transfer complete
 
# ============================================
# PORT Command Port Calculation Examples
 
# Port 1024 = 4 × 256 + 0 = 4,0
PORT 192,168,1,100,4,0     # Port 1024
 
# Port 21000 = 82 × 256 + 8 = 82,8  (82×256=20992, +8=21000)
PORT 192,168,1,100,82,8    # Port 21000
 
# Port 65535 = 255 × 256 + 255 = 255,255
PORT 192,168,1,100,255,255 # Port 65535

Active Mode Challenges:

Active mode, while conceptually simple, faces significant problems in modern networks:

Active Mode Problems

•Client Firewall Blocking — Inbound connections to client ports are typically blocked by firewalls. The server's connection attempt fails at the client's firewall.
•NAT Incompatibility — If client is behind NAT, the private IP (192.168.x.x) in PORT command is unreachable from the server. Server connects to wrong address.
•Port Conflicts — Client must open specific ports; may conflict with other services or security policies.
•Server Port 20 Requirement — Traditional servers use source port 20, but some firewalls may still block incoming connections regardless of source port.
•Dynamic Port Allocation — Client must choose and manage ephemeral ports, adding implementation complexity.

Avoid Active Mode

Passive Mode Data Connections (PASV/EPSV)

PASV Command:

PASV<CRLF>

PASV Response Format:

227 Entering Passive Mode (h1,h2,h3,h4,p1,p2)

The address and port encoding matches the PORT command format.

passive-mode-sequence.txt
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
# Passive Mode File Transfer Sequence
 
# Step 1: Request passive mode
→ PASV
← 227 Entering Passive Mode (203,0,113,50,195,88)
 
# Decode: IP = 203.0.113.50, Port = 195×256 + 88 = 50008
 
# Step 2: Client connects to server's passive port
# [Client → Server 203.0.113.50:50008]
# Connection established
 
# Step 3: Client requests file (on control channel)
→ RETR largefile.zip
← 150 Opening BINARY mode data connection for largefile.zip (524288000 bytes)
 
# Step 4: Data flows from server to client over data connection
# [524288000 bytes transferred...]
 
# Step 5: Data connection closes
# Step 6: Completion notification on control channel  
← 226 Transfer complete
 
# ============================================
# For uploads (STOR), same process but data flows client→server
 
→ PASV
← 227 Entering Passive Mode (203,0,113,50,196,44)
# Port = 196×256 + 44 = 50220
 
→ STOR upload.zip
← 150 Opening BINARY mode data connection for upload.zip
 
# Client sends file data over data connection
# [Client → Server: file bytes]
 
← 226 Transfer complete

Extended Passive Mode (EPSV):

For IPv6 support and simpler parsing, RFC 2428 introduced EPSV:

→ EPSV
← 229 Entering Extended Passive Mode (|||50008|)

EPSV returns only the port number; the client connects to the same IP as the control connection. This avoids address encoding issues and works seamlessly with IPv6.

PASV vs EPSV Comparison
Feature	PASV	EPSV
RFC	RFC 959	RFC 2428
IPv6 Support	No (IPv4 only)	Yes
Response Format	IP and port encoded	Port only in delimiters
Server IP	Explicit in response	Same as control connection
Parsing	Six comma-separated values	Port between \|\|\|delimiters\|
NAT Transparency	May return wrong IP	No IP to be wrong

Best Practice

RETR: Retrieving Files

The RETR (Retrieve) command downloads a file from the server to the client. Before issuing RETR, you must have an established data connection (via PASV/PORT).

Command Syntax:

RETR <SP> <pathname> <CRLF>

The pathname can be relative to current directory or absolute.

RETR Command Reply Codes
Reply Code	Meaning	When Occurs
125	Data connection already open	Server reusing existing connection (rare)
150	File status okay; opening data connection	Normal success; transfer beginning
226	Closing data connection; transfer complete	Transfer finished successfully
250	Requested file action okay, completed	Alternative success code
421	Service not available	Server problem
425	Can't open data connection	Firewall/NAT blocking
426	Connection closed; transfer aborted	Network error during transfer
450	File unavailable (busy)	File locked by another process
451	Local error in processing	Server filesystem error
500	Syntax error	Command malformed
501	Syntax error in parameters	Invalid filename
530	Not logged in	Session expired
550	File not found or permission denied	File doesn't exist or no read access

retr-complete-example.txt
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
# Complete File Download Sequence
 
# 1. Navigate to desired directory
→ CWD /projects/reports
← 250 Directory changed
 
# 2. Set binary mode for non-text files
→ TYPE I
← 200 Switching to Binary mode
 
# 3. Check file size (optional, for progress reporting)
→ SIZE quarterly-report.pdf
← 213 2548731
 
# 4. Enter passive mode
→ PASV
← 227 Entering Passive Mode (203,0,113,50,196,180)
 
# 5. Client connects to 203.0.113.50:50356 (196×256+180)
# [TCP connection established]
 
# 6. Request file
→ RETR quarterly-report.pdf
← 150 Opening BINARY mode data connection for quarterly-report.pdf (2548731 bytes)
 
# 7. Server sends file over data connection
# Client reads: [2,548,731 bytes in chunks...]
 
# 8. Server closes data connection when complete
# [TCP FIN/ACK exchange]
 
# 9. Server confirms completion
← 226 Transfer complete
 
# ============================================
# Error Examples
 
# File not found
→ RETR nonexistent.pdf
← 550 nonexistent.pdf: No such file or directory
 
# Permission denied
→ RETR /etc/shadow
← 550 /etc/shadow: Permission denied
 
# No data connection established
→ RETR file.txt
← 425 Use PASV or PORT first

Transfer Mode Matters

STOR: Storing (Uploading) Files

The STOR (Store) command uploads a file from the client to the server. Like RETR, it requires a data connection to be established first.

Command Syntax:

STOR <SP> <pathname> <CRLF>

Related Commands:

STOU (Store Unique): Server generates unique filename, preventing overwrites
APPE (Append): Adds data to existing file instead of replacing

STOR Command Reply Codes
Reply Code	Meaning	When Occurs
125	Data connection already open	Transfer can begin
150	File status okay; opening connection	Normal; server ready to receive
226	Transfer complete	File successfully written
250	Requested file action okay	Alternative success
421	Service not available	Server problem
425	Can't open data connection	Connection issue
426	Connection closed; transfer aborted	Network error during upload
450	File unavailable (locked)	Destination locked
451	Local error; action aborted	Server write error
452	Insufficient storage space	Disk full or quota exceeded
532	Need account for storing files	ACCT required
550	Permission denied	No write permission to directory
552	Exceeded storage allocation	User quota exceeded
553	File name not allowed	Invalid characters or path

stor-complete-example.txt
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
# Complete File Upload Sequence
 
# 1. Navigate to target directory
→ CWD /incoming/uploads
← 250 Directory changed
 
# 2. Set binary mode
→ TYPE I
← 200 Switching to Binary mode
 
# 3. Enter passive mode
→ PASV  
← 227 Entering Passive Mode (203,0,113,50,197,22)
 
# 4. Client connects to 203.0.113.50:50454
# [TCP connection established]
 
# 5. Initiate upload
→ STOR project-backup.tar.gz
← 150 Opening BINARY mode data connection for project-backup.tar.gz
 
# 6. Client sends file over data connection
# [15,728,640 bytes sent in chunks...]
 
# 7. Client closes data connection when complete
# [TCP FIN sent]
 
# 8. Server confirms receipt
← 226 Transfer complete; 15728640 bytes received
 
# ============================================
# Store Unique Example
→ PASV
← 227 Entering Passive Mode (203,0,113,50,197,50)
→ STOU report.pdf
← 150 Opening BINARY mode data connection for report.pdf.1
# [data transfer]
← 226 Transfer complete; unique filename is report.pdf.1
 
# ============================================
# Append Example (add to existing file)
→ PASV
← 227 Entering Passive Mode (203,0,113,50,197,78)
→ APPE logfile.txt
← 150 Opening ASCII mode data connection for logfile.txt
# [new log entries sent]
← 226 Transfer complete
 
# ============================================
# Error: Quota Exceeded
→ STOR massive-dataset.zip
← 552 Disk quota exceeded

Atomic Uploads

Resumable Transfers (REST)

Command Syntax:

REST <SP> <byte-offset> <CRLF>

REST sets a marker for the next transfer command. It doesn't transfer anything itself—it configures where the next RETR or STOR will start.

REST Command Reply Codes
Reply Code	Meaning	Notes
350	Requested action pending	Offset accepted; ready for RETR/STOR
500	Syntax error	Command not recognized
501	Syntax error in parameters	Invalid offset format
502	Command not implemented	Server doesn't support REST
503	Bad sequence of commands	REST after data connection? (varies)
530	Not logged in	Session expired

resumable-download.txt
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
# Resumable Download Scenario
 
# Original attempt - Failed at 50MB of 100MB file
# Client has 52,428,800 bytes saved locally
 
# Step 1: Verify server has REST support
→ FEAT
← 211-Features:
←  SIZE
←  REST STREAM
←  MDTM
←  PASV
← 211 End
 
# Step 2: Check current file size matches expected
→ SIZE bigfile.iso
← 213 104857600    # 100 MB
 
# Step 3: Verify local file size
# Client checks: local_file_size = 52428800
 
# Step 4: Set restart offset
→ REST 52428800
← 350 Restart position accepted (52428800)
 
# Step 5: Enter passive mode
→ PASV
← 227 Entering Passive Mode (203,0,113,50,198,100)
# Connect to data port...
 
# Step 6: RETR resumes from offset
→ RETR bigfile.iso
← 150 Opening BINARY mode data connection for bigfile.iso (104857600 bytes)
# NOTE: Server may report full size or remaining size
 
# Step 7: Server sends bytes starting from 52428800
# Client appends to local file: [52,428,800 additional bytes]
 
← 226 Transfer complete
 
# ============================================
# Resumable Upload Example
 
# Upload interrupted at 25 MB of 50 MB file
# Server has 26,214,400 bytes of partial upload
 
# Check partial file on server
→ SIZE partial-upload.zip
← 213 26214400
 
# Set restart offset
→ REST 26214400
← 350 Restart position accepted
 
# Passive mode and upload
→ PASV
← 227 Entering Passive Mode (203,0,113,50,198,128)
→ STOR partial-upload.zip
← 150 Opening BINARY mode data connection
 
# Client seeks to offset in local file
# Client sends bytes from position 26214400 onward
 
← 226 Transfer complete

Implementation Considerations:

For Downloads:

Check if local partial file exists
Get remote file size with SIZE command
Verify local size < remote size (otherwise file is complete or corrupted)
Set REST to local file size
Open local file in append mode
RETR and write received bytes

For Uploads:

Check if remote partial file exists (SIZE command)
Verify remote size < local size
Set REST to remote file size
Seek local file to that offset
STOR and send remaining bytes

Integrity Verification:

REST/resume doesn't verify that the partial transfer is correct—just that sizes match. For critical transfers, compute checksums after completion:

→ XMD5 bigfile.iso    # Non-standard but common
← 213 d41d8cd98f00b204e9800998ecf8427e

→ XSHA256 bigfile.iso  # Even less standard

Most servers don't support these; verify checksums through other means (companion .md5 files, etc.).

REST Must Be Binary

REST byte offsets only make sense in binary (TYPE I) mode. In ASCII mode, line ending conversions change byte positions unpredictably. Always use TYPE I before REST-based resume operations.

Progress Monitoring and File Information

Good user experience requires progress indication during transfers. FTP provides commands to gather file information before transfer begins, enabling accurate progress bars and time estimates.

SIZE Command:

Returns the file size in bytes (when using binary mode):

SIZE <SP> <pathname> <CRLF>

progress-monitoring.txt
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
# Getting File Information
 
# SIZE command
→ SIZE largefile.zip
← 213 1073741824    # 1 GB in bytes
 
# SIZE requires TYPE I for accurate byte count
→ TYPE A
← 200 Switching to ASCII mode
→ SIZE textfile.txt
← 550 SIZE command could not complete.  # Many servers fail in ASCII mode
 
→ TYPE I
← 200 Switching to Binary mode
→ SIZE textfile.txt
← 213 15234
 
# ============================================
# MDTM - Modification Time
 
→ MDTM report.pdf
← 213 20250115143022    # YYYYMMDDhhmmss format
# = January 15, 2025 at 14:30:22
 
→ MDTM /archive/old-backup.tar
← 213 20230601000000
 
# ============================================
# Combining for Download Preparation
 
→ TYPE I
← 200 Switching to Binary mode
 
→ SIZE project.tar.gz  
← 213 536870912    # 512 MB
 
→ MDTM project.tar.gz
← 213 20250114120000    # Check if newer than local copy
 
→ PASV
← 227 Entering Passive Mode (203,0,113,50,199,50)
 
# Client now knows:
# - File is 536,870,912 bytes
# - Modified Jan 14, 2025 at 12:00:00
# - Can compute ETA based on bandwidth
# - Can check if local copy is current
 
→ RETR project.tar.gz
← 150 Opening BINARY mode data connection

Progress Calculation:

With file size known, calculate progress during transfer:

total_size = 536870912  # From SIZE command
bytes_received = 0
start_time = time.time()

while data := read_from_data_connection():
    bytes_received += len(data)
    write_to_file(data)
    
    # Progress percentage
    progress = (bytes_received / total_size) * 100
    
    # Speed calculation
    elapsed = time.time() - start_time
    speed = bytes_received / elapsed  # bytes/second
    
    # ETA
    remaining_bytes = total_size - bytes_received
    eta_seconds = remaining_bytes / speed if speed > 0 else 0
    
    print(f"{progress:.1f}% | {speed/1024/1024:.1f} MB/s | ETA: {eta_seconds:.0f}s")

File Information Commands
Command	Purpose	Response Format
SIZE	Get file size in bytes	213 <byte-count>
MDTM	Get modification time	213 YYYYMMDDhhmmss
MLST	Machine-parseable file facts	250- facts;facts;... filename
STAT <file>	Status of file/transfer	211/212/213 multi-line status

When SIZE Isn't Available

Aborting Transfers and Handling Timeouts

Transfers don't always complete successfully. Users cancel downloads, networks fail, servers become unresponsive. Handling these scenarios gracefully is essential for robust FTP clients.

ABOR Command:

The ABOR (Abort) command interrupts an in-progress transfer:

ABOR<CRLF>

Because the data channel may be blocking, FTP specifies a special sequence for sending ABOR:

abort-sequences.txt
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
# Standard ABOR Sequence (when control channel is responsive)
 
# Transfer in progress...
← 150 Opening BINARY mode data connection
# [data flowing on data channel]
 
# User cancels - send ABOR on control channel
→ ABOR
← 426 Transfer aborted.
← 226 Abort successful.
 
# OR
→ ABOR
← 225 Abort successful but file transfer was complete.
← 226 Transfer complete.
 
# ============================================
# Telnet Interrupt Signals (when control might be blocked)
 
# If data transfer is consuming all I/O, send:
# 1. Telnet IP (Interrupt Process): 0xFF 0xF4
# 2. Telnet DM (Data Mark): 0xFF 0xF2 (in TCP urgent data)
# 3. ABOR command
 
# This ensures ABOR is processed even if buffers are full
[TCP URG] 0xFF 0xF4 0xFF 0xF2 ABOR
← 426 Connection closed; transfer aborted
 
# ============================================
# Handling Hung Transfers
 
# No response for > 60 seconds during data transfer
# Options:
# 1. Close data connection unilaterally
# 2. Send ABOR and wait briefly  
# 3. Close control connection if ABOR doesn't respond
# 4. Full disconnect and reconnect
 
# Example timeout handling:
→ RETR stuck-file.bin
← 150 Opening connection
# [no data arrives for 120 seconds]
 
→ ABOR
← [no response for 30 seconds]
 
# Give up - close sockets
[TCP RST sent]
# Reconnect and start new session

Timeout Strategies:

Timeout Type	Typical Value	Action on Expiry
Connection timeout	30-60s	Cancel connection attempt
Command response	60-120s	Retry or abort operation
Data transfer idle	120-300s	Abort and resume or fail
Session idle	300-900s	Server may disconnect

Keepalive for Long Transfers:

During very long transfers, the control connection sits idle. Some servers/firewalls may close idle connections. Solutions:

NOOP Command: Send NOOP periodically to keep control channel active
TCP Keepalives: Enable OS-level TCP keepalives on control socket
Data Progress: Rely on data channel activity to signal overall session health

Incomplete Abort

Practical Transfer Implementation

Putting together everything we've learned, here's a production-quality file transfer implementation that handles the complexities we've discussed:

robust-transfer.py
Python
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
import socket
import time
from dataclasses import dataclass
from typing import Optional, Callable, BinaryIO
from pathlib import Path
 
@dataclass
class TransferProgress:
    bytes_transferred: int
    total_bytes: Optional[int]
    elapsed_seconds: float
    
    @property
    def percentage(self) -> Optional[float]:
        if self.total_bytes and self.total_bytes > 0:
            return (self.bytes_transferred / self.total_bytes) * 100
        return None
    
    @property
    def speed_bps(self) -> float:
        if self.elapsed_seconds > 0:
            return self.bytes_transferred / self.elapsed_seconds
        return 0
    
    @property
    def eta_seconds(self) -> Optional[float]:
        if self.total_bytes and self.speed_bps > 0:
            remaining = self.total_bytes - self.bytes_transferred
            return remaining / self.speed_bps
        return None
 
ProgressCallback = Callable[[TransferProgress], None]
 
class FTPTransfer:
    """
    Robust FTP file transfer with resume, progress, and error handling.
    """
    
    def __init__(self, control_socket, buffer_size: int = 65536):
        self.control = control_socket
        self.buffer_size = buffer_size
        self.data_socket: Optional[socket.socket] = None
    
    def _send_command(self, cmd: str) -> str:
        """Send command and return response."""
        self.control.sendall(f"{cmd}\r
".encode())
        return self._receive_response()
    
    def _receive_response(self) -> str:
        """Receive complete FTP response."""
        response = ""
        while True:
            chunk = self.control.recv(4096).decode()
            response += chunk
            # Check for end of response
            lines = response.split('\r
')
            for line in lines:
                if len(line) >= 4 and line[3] == ' ':
                    return response.strip()
        return response.strip()
    
    def _enter_passive_mode(self) -> tuple:
        """Enter passive mode, return (host, port)."""
        # Try EPSV first
        response = self._send_command("EPSV")
        if response.startswith("229"):
            # Parse |||port|
            start = response.find("|||")
            end = response.find("|", start + 3)
            port = int(response[start+3:end])
            # Use same host as control connection
            host = self.control.getpeername()[0]
            return (host, port)
        
        # Fall back to PASV
        response = self._send_command("PASV")
        if not response.startswith("227"):
            raise RuntimeError(f"PASV failed: {response}")
        
        # Parse (h1,h2,h3,h4,p1,p2)
        start = response.find("(")
        end = response.find(")")
        numbers = response[start+1:end].split(",")
        host = ".".join(numbers[:4])
        port = int(numbers[4]) * 256 + int(numbers[5])
        return (host, port)
    
    def _get_file_size(self, remote_path: str) -> Optional[int]:
        """Get remote file size, None if unavailable."""
        response = self._send_command(f"SIZE {remote_path}")
        if response.startswith("213"):
            return int(response.split()[1])
        return None
    
    def download(
        self,
        remote_path: str,
        local_path: Path,
        progress_callback: Optional[ProgressCallback] = None,
        resume: bool = True
    ) -> bool:
        """
        Download file with optional resume and progress reporting.
        Returns True on success, False on failure.
        """
        # Set binary mode
        self._send_command("TYPE I")
        
        # Get remote file size for progress
        remote_size = self._get_file_size(remote_path)
        
        # Check for resume
        start_offset = 0
        mode = "wb"
        
        if resume and local_path.exists():
            local_size = local_path.stat().st_size
            if remote_size and local_size < remote_size:
                start_offset = local_size
                mode = "ab"
                resp = self._send_command(f"REST {start_offset}")
                if not resp.startswith("350"):
                    # REST not supported, start over
                    start_offset = 0
                    mode = "wb"
        
        # Enter passive mode and connect
        host, port = self._enter_passive_mode()
        self.data_socket = socket.create_connection((host, port), timeout=60)
        
        # Request file
        response = self._send_command(f"RETR {remote_path}")
        if not (response.startswith("150") or response.startswith("125")):
            self.data_socket.close()
            return False
        
        # If we didn't get size from SIZE, try parsing 150 response
        if remote_size is None:
            # Try to parse "150 ... (12345 bytes)"
            if "(" in response and "bytes)" in response:
                try:
                    size_str = response.split("(")[1].split()[0]
                    remote_size = int(size_str)
                except:
                    pass
        
        # Transfer data
        bytes_received = start_offset
        start_time = time.time()
        
        try:
            with open(local_path, mode) as f:
                while True:
                    data = self.data_socket.recv(self.buffer_size)
                    if not data:
                        break
                    
                    f.write(data)
                    bytes_received += len(data)
                    
                    if progress_callback:
                        progress = TransferProgress(
                            bytes_transferred=bytes_received,
                            total_bytes=remote_size,
                            elapsed_seconds=time.time() - start_time
                        )
                        progress_callback(progress)
        
        finally:
            self.data_socket.close()
        
        # Get completion response
        response = self._receive_response()
        return response.startswith("226") or response.startswith("250")
    
    def upload(
        self,
        local_path: Path,
        remote_path: str,
        progress_callback: Optional[ProgressCallback] = None,
        resume: bool = True,
        atomic: bool = True
    ) -> bool:
        """
        Upload file with optional resume, progress, and atomic writes.
        atomic=True writes to temp file then renames on completion.
        """
        if not local_path.exists():
            raise FileNotFoundError(f"Local file not found: {local_path}")
        
        local_size = local_path.stat().st_size
        
        # Set binary mode
        self._send_command("TYPE I")
        
        # Determine target path (temp for atomic)
        actual_remote = f"{remote_path}.tmp" if atomic else remote_path
        
        # Check for resume
        start_offset = 0
        if resume:
            remote_size = self._get_file_size(actual_remote)
            if remote_size and remote_size < local_size:
                resp = self._send_command(f"REST {remote_size}")
                if resp.startswith("350"):
                    start_offset = remote_size
        
        # Enter passive mode
        host, port = self._enter_passive_mode()
        self.data_socket = socket.create_connection((host, port), timeout=60)
        
        # Initiate upload
        response = self._send_command(f"STOR {actual_remote}")
        if not (response.startswith("150") or response.startswith("125")):
            self.data_socket.close()
            return False
        
        # Transfer data
        bytes_sent = start_offset
        start_time = time.time()
        
        try:
            with open(local_path, "rb") as f:
                if start_offset > 0:
                    f.seek(start_offset)
                
                while True:
                    data = f.read(self.buffer_size)
                    if not data:
                        break
                    
                    self.data_socket.sendall(data)
                    bytes_sent += len(data)
                    
                    if progress_callback:
                        progress = TransferProgress(
                            bytes_transferred=bytes_sent,
                            total_bytes=local_size,
                            elapsed_seconds=time.time() - start_time
                        )
                        progress_callback(progress)
        
        finally:
            self.data_socket.close()
        
        # Get completion response
        response = self._receive_response()
        success = response.startswith("226") or response.startswith("250")
        
        # Atomic rename if requested
        if success and atomic:
            self._send_command(f"RNFR {actual_remote}")
            response = self._send_command(f"RNTO {remote_path}")
            success = response.startswith("250")
        
        return success
 
 
# Usage example
def print_progress(p: TransferProgress):
    if p.percentage is not None:
        eta = f"ETA: {p.eta_seconds:.0f}s" if p.eta_seconds else ""
        print(f"\r{p.percentage:.1f}% | "
              f"{p.speed_bps/1024/1024:.1f} MB/s | "
              f"{eta}", end="")
    else:
        print(f"\r{p.bytes_transferred:,} bytes | "
              f"{p.speed_bps/1024:.1f} KB/s", end="")

Production Usage

Summary: File Transfer

We've covered the complete file transfer mechanism in FTP, from dual-channel architecture through practical implementation patterns. Let's consolidate the key takeaways:

Key Takeaways

•Dual-channel design — FTP uses separate control (commands) and data (file contents) connections, enabling out-of-band control during transfers.
•Avoid active mode — Active mode (PORT) fails with NAT and firewalls; always prefer passive mode (PASV/EPSV).
•EPSV over PASV — Use extended passive mode first for IPv6 support and to avoid NAT-related IP address issues.
•RETR downloads, STOR uploads — Both require prior data connection setup; verify success with 226/250 responses.
•REST enables resume — Set byte offset before transfer to resume interrupted operations; requires binary mode.
•SIZE before transfer — Get file size for progress calculation and resume validation.
•ABOR for cancellation — Cleanly abort in-progress transfers; may require telnet interrupt sequences.
•Atomic uploads — Use temporary filename + rename (RNFR/RNTO) to prevent serving partial files.

What's Next:

Page Complete