Loading content...
The Simple Mail Transfer Protocol was designed in 1982 for a very different Internet—a network of a few thousand research computers exchanging plain ASCII text among trusted academic users. Today, SMTP carries billions of messages containing attachments in every conceivable format, international characters, encrypted content, and authenticated transactions across a hostile Internet filled with spam and malware.
How has a 40-year-old protocol remained relevant? The answer is ESMTP (Extended SMTP)—a brilliant extension mechanism that allows the protocol to evolve while maintaining backward compatibility. Understanding ESMTP extensions is essential for anyone implementing, configuring, or troubleshooting modern email systems.
By the end of this page, you will master the ESMTP extension negotiation mechanism, understand critical extensions including AUTH, STARTTLS, SIZE, 8BITMIME, PIPELINING, CHUNKING, and SMTPUTF8, learn how to configure and troubleshoot extensions, and appreciate the elegant design that enables ongoing protocol evolution.
RFC 1869 (1995) introduced Extended SMTP, solving a fundamental protocol design problem: how do you add new features to a protocol while maintaining compatibility with older implementations?
The EHLO Solution:
ESMTP replaces the original HELO command with EHLO (Extended HELO). When a client uses EHLO, it signals ESMTP support and the server responds with a list of supported extensions:
C: EHLO client.example.org
S: 250-mail.example.com Hello client.example.org
S: 250-SIZE 52428800
S: 250-8BITMIME
S: 250-STARTTLS
S: 250-AUTH LOGIN PLAIN XOAUTH2
S: 250-PIPELINING
S: 250-ENHANCEDSTATUSCODES
S: 250-CHUNKING
S: 250-SMTPUTF8
S: 250 DSN
STARTTLS, starttls, and StartTLS are equivalentBackward Compatibility:
The genius of ESMTP is its graceful degradation:
# Modern server receiving HELO from legacy client
C: HELO old-client.example.org
S: 250 Hello old-client.example.org
# Server enters basic SMTP mode; no extensions available
# Modern client connecting to legacy server
C: EHLO new-client.example.org
S: 500 Command not recognized
C: HELO new-client.example.org
S: 250 Hello new-client.example.org
# Client falls back to basic SMTP
This ensures a 2024 client can communicate with a 1985 server (if one still exists), and vice versa. Extensions enhance capability without breaking interoperability.
After upgrading to TLS, the client MUST issue EHLO again. Extension availability may change after encryption—for example, AUTH often only appears after TLS is established. Always treat the post-STARTTLS extension list as authoritative.
The STARTTLS extension (RFC 3207) enables upgrading a plaintext SMTP connection to TLS encryption. It's the primary mechanism for protecting email content and credentials in transit.
Extension Keyword: STARTTLS
How STARTTLS Works:
C: EHLO client.example.org
S: 250-mail.example.com
S: 250-STARTTLS
S: 250 OK
C: STARTTLS
S: 220 Ready to start TLS
[TLS handshake begins]
[All subsequent traffic is encrypted]
C: EHLO client.example.org <-- Must re-issue after TLS!
S: 250-mail.example.com
S: 250-AUTH PLAIN LOGIN
S: 250 OK
| Code | Meaning | Client Action |
|---|---|---|
| 220 | Ready to start TLS | Begin TLS handshake immediately |
| 454 | TLS not available (temporary) | Retry later or proceed without TLS |
| 501 | Syntax error (unexpected parameter) | Fix command syntax |
| 503 | Bad sequence (TLS already active) | Already using TLS; continue normally |
Security Considerations:
1. Opportunistic vs. Mandatory:
For submission (port 587), TLS should be mandatory. For relay (port 25), opportunistic TLS is the compromise between security and deliverability.
2. The STARTTLS Stripping Attack:
Since STARTTLS is negotiated over plaintext, an attacker can remove the STARTTLS keyword from the server's response, forcing a downgrade to plaintext. Countermeasures:
3. Certificate Validation:
Traditionally, SMTP servers often didn't validate TLS certificates strictly (any encryption better than none). Modern configurations should verify certificates for known destinations.
123456789101112131415
# Enable STARTTLS for inbound connectionssmtpd_tls_security_level = may # Offer TLS, don't require# For submission port, use "encrypt" to require TLS smtpd_tls_cert_file = /etc/ssl/certs/mail.example.com.crtsmtpd_tls_key_file = /etc/ssl/private/mail.example.com.keysmtpd_tls_CAfile = /etc/ssl/certs/ca-certificates.crt # Modern TLS only (disable SSLv3, TLSv1.0, TLSv1.1)smtpd_tls_mandatory_protocols = >=TLSv1.2smtpd_tls_protocols = >=TLSv1.2 # Strong cipherssmtpd_tls_mandatory_ciphers = highsmtpd_tls_exclude_ciphers = aNULL, MD5, DES, RC4STARTTLS encrypts the connection between two adjacent servers (hop-to-hop), not the entire email path. Each hop decrypts and re-encrypts. For true end-to-end encryption, use S/MIME or PGP. STARTTLS protects against passive eavesdropping, not compromised intermediate servers.
The AUTH extension (RFC 4954) enables SMTP clients to authenticate with servers, proving their identity before sending mail. This is essential for mail submission and prevents unauthorized use of mail servers.
Extension Keyword: AUTH mechanism [mechanism ...]
250-AUTH LOGIN PLAIN CRAM-MD5 XOAUTH2
The server advertises supported authentication mechanisms. The client selects one and initiates the authentication exchange.
| Mechanism | RFC | Security | Description |
|---|---|---|---|
| PLAIN | RFC 4616 | Weak (needs TLS) | NUL-separated username and password in Base64 |
| LOGIN | Draft | Weak (needs TLS) | Server prompts for Base64 username, then password |
| CRAM-MD5 | RFC 2195 | Medium | Challenge-response using MD5-HMAC; password not sent |
| SCRAM-SHA-256 | RFC 7677 | Strong | Salted challenge-response with SHA-256 |
| XOAUTH2 | Google/MS | Strong | OAuth 2.0 bearer token authentication |
| OAUTHBEARER | RFC 7628 | Strong | Standardized OAuth 2.0 bearer token |
AUTH PLAIN Deep Dive:
PLAIN is the most common mechanism due to simplicity. It requires TLS for security.
Step 1: Construct credentials string
[authzid] NUL authcid NUL password
Where authzid is optional authorization identity (usually empty), authcid is authentication identity (username), and password is the password.
Step 2: Base64 encode
import base64
auth_string = '\\x00alice@example.com\\x00mypassword'
encoded = base64.b64encode(auth_string.encode()).decode()
# Result: AGFsaWNlQGV4YW1wbGUuY29tAG15cGFzc3dvcmQ=
Step 3: Send AUTH command
C: AUTH PLAIN AGFsaWNlQGV4YW1wbGUuY29tAG15cGFzc3dvcmQ=
S: 235 2.7.0 Authentication successful
Alternative (two-step):
C: AUTH PLAIN
S: 334 (empty challenge, client should send credentials)
C: AGFsaWNlQGV4YW1wbGUuY29tAG15cGFzc3dvcmQ=
S: 235 2.7.0 Authentication successful
1234567891011121314151617181920212223242526272829303132333435
import base64import hmacimport hashlib def auth_plain(username: str, password: str) -> str: """Generate AUTH PLAIN credentials string.""" auth_string = f'\x00{username}\x00{password}' return base64.b64encode(auth_string.encode()).decode() def auth_cram_md5(username: str, password: str, challenge: bytes) -> str: """ Generate AUTH CRAM-MD5 response. Challenge is Base64-decoded server challenge. """ # HMAC-MD5 of the challenge using password as key digest = hmac.new( password.encode(), challenge, hashlib.md5 ).hexdigest() # Response format: username SPACE digest response = f'{username} {digest}' return base64.b64encode(response.encode()).decode() def auth_xoauth2(email: str, access_token: str) -> str: """Generate XOAUTH2 authentication string.""" auth_string = f'user={email}\x01auth=Bearer {access_token}\x01\x01' return base64.b64encode(auth_string.encode()).decode() # Examplesprint("PLAIN:", auth_plain("alice@example.com", "secret"))# decode challenge from server's 334 response first# print("CRAM-MD5:", auth_cram_md5("alice", "secret", decoded_challenge))print("XOAUTH2:", auth_xoauth2("alice@gmail.com", "ya29.access_token"))Major providers (Google, Microsoft) now require or strongly prefer OAuth (XOAUTH2/OAUTHBEARER) over password authentication. This provides better security through token scoping, easier revocation, and MFA support. Implement OAuth for any serious email-sending application.
The SIZE extension (RFC 1870) allows clients to declare message size before transmission and servers to advertise maximum acceptable size. This prevents wasted bandwidth when a message would be rejected anyway.
Extension Keyword: SIZE [maximum]
Server Advertisement:
250-SIZE 52428800
The server indicates it will accept messages up to 52,428,800 bytes (50 MB).
Client Declaration:
MAIL FROM:<alice@example.org> SIZE=1048576
The client declares the message is approximately 1,048,576 bytes (1 MB).
| Scenario | Response | Client Action |
|---|---|---|
| Size acceptable | 250 OK | Proceed with RCPT TO and DATA |
| Size exceeds limit | 552 5.3.4 Message too large | Abort; notify user; don't retry |
| Size acceptable per-recipient | 250 OK | Recipient may have different limit |
| Server has no limit | 250-SIZE (no value) | Any size theoretically accepted |
| Client doesn't declare | N/A | Server may reject after DATA |
Size Calculation Considerations:
1. Overhead Matters: The SIZE should represent the complete message as transmitted, including:
2. Dot Stuffing:
Lines beginning with . are escaped by prepending another ., slightly increasing size. For safety, estimate slightly high.
3. Per-Recipient Limits:
Some systems have per-recipient size limits (mailbox quotas). A message may be rejected for RCPT TO even if the server's global limit is higher.
Practical Implications:
Without SIZE, a client might transmit a 100 MB message only to have it rejected after full transfer—wasting bandwidth and time. With SIZE:
C: MAIL FROM:<alice@example.org> SIZE=104857600
S: 552 5.3.4 Message size exceeds fixed maximum message size (52428800)
# Rejection happens immediately; no wasted transfer
Email size limits range from 10 MB (many ISPs) to 150 MB (GMail) to unlimited (some enterprise systems). When building applications that send attachments, check limits before assuming large files will be accepted. Consider using file-sharing services for large content with email notifications containing links.
Original SMTP was designed for 7-bit ASCII text only. The 8BITMIME extension (RFC 6152) enables transmission of 8-bit data, essential for international characters and binary content.
Extension Keyword: 8BITMIME
Historical Context:
RFC 821 specified that SMTP transmits lines of no more than 1000 characters of ASCII (codes 0-127, 7-bit). International characters (accented letters, Cyrillic, Chinese, etc.) and binary attachments exceed this range.
The Pre-8BITMIME Solution:
Before 8BITMIME, content was encoded:
=XX (hex). Efficient for mostly-ASCII text with occasional special charactersThese encodings are still used, but 8BITMIME can avoid encoding overhead when both client and server support it.
Using 8BITMIME:
When server advertises 8BITMIME, client can declare its message body encoding:
C: EHLO client.example.org
S: 250-mail.example.com
S: 250-8BITMIME
S: 250 OK
C: MAIL FROM:<alice@example.org> BODY=8BITMIME
S: 250 OK
C: DATA
S: 354 Start input
C: Subject: Café résumé
C: Content-Type: text/plain; charset=UTF-8
C: Content-Transfer-Encoding: 8bit
C:
C: Voilà! Non-ASCII: é, ñ, 中文, Русский
C: .
S: 250 OK
BODY Parameter Values:
BODY=7BIT — Traditional 7-bit ASCII (default if omitted)BODY=8BITMIME — 8-bit clean transmission allowedBODY=BINARYMIME — Binary data with BDAT/CHUNKING| Encoding | Use Case | 8BITMIME Required | Overhead |
|---|---|---|---|
| 7bit | Plain ASCII text | No | None |
| 8bit | UTF-8 text, high-bit characters | Yes | None (raw) |
| quoted-printable | Mostly ASCII with some special chars | No | Low (~5-20%) |
| base64 | Binary files, attachments | No | 33% |
| binary | Raw binary (with BINARYMIME/CHUNKING) | Advanced | None |
As of 2024, virtually all SMTP servers support 8BITMIME. If building an email client, you can generally assume 8-bit support is available. However, defensive encoding (Base64 for attachments, Quoted-Printable for headers) ensures compatibility with the rare legacy system.
The PIPELINING extension (RFC 2920) allows clients to send multiple commands without waiting for individual responses, dramatically reducing latency on high-latency connections.
Extension Keyword: PIPELINING
Without Pipelining:
Each command requires a round-trip:
C: MAIL FROM:<alice@example.org>
[100ms network latency]
S: 250 OK
C: RCPT TO:<bob@example.com>
[100ms]
S: 250 OK
C: RCPT TO:<carol@example.com>
[100ms]
S: 250 OK
C: DATA
[100ms]
S: 354 Start input
# 400ms just for envelope!
With Pipelining:
Commands are batched; one round-trip covers multiple commands:
C: MAIL FROM:<alice@example.org>
C: RCPT TO:<bob@example.com>
C: RCPT TO:<carol@example.com>
C: DATA
[100ms for all]
S: 250 OK
S: 250 OK
S: 250 OK
S: 354 Start input
# 100ms for entire envelope!
Performance Impact:
For a message with 10 recipients on a 100ms latency connection:
That's a 6x improvement!
1234567891011121314151617181920212223242526272829303132333435363738394041424344454647484950515253545556575859606162636465
import socket def send_email_pipelined(server: str, port: int, sender: str, recipients: list[str], message: str): """ Send email using PIPELINING for efficient envelope transfer. """ with socket.create_connection((server, port), timeout=30) as sock: fh = sock.makefile('rw', encoding='utf-8', newline='\r') # Read greeting greeting = fh.readline() assert greeting.startswith('220') # EHLO - cannot pipeline, must wait fh.write('EHLO client.example.org\r') fh.flush() extensions = [] while True: line = fh.readline() extensions.append(line) if line[3] == ' ': # Last line has space, not hyphen break # Check for PIPELINING support pipelining = any('PIPELINING' in ext for ext in extensions) if pipelining: # Send envelope commands all at once envelope = f'MAIL FROM:<{sender}>\r' for rcpt in recipients: envelope += f'RCPT TO:<{rcpt}>\r' envelope += 'DATA\r' fh.write(envelope) fh.flush() # Now read all responses mail_resp = fh.readline() # MAIL FROM response for rcpt in recipients: rcpt_resp = fh.readline() # Each RCPT TO response data_resp = fh.readline() # DATA response (354) else: # Fallback: sequential commands # ... traditional send-wait-send-wait pattern pass # Send message body (cannot pipeline) fh.write(message) fh.write('\r.\r') fh.flush() final = fh.readline() # 250 OK fh.write('QUIT\r') fh.flush()When pipelining, you must handle errors for commands sent before you knew they'd fail. For example, if MAIL FROM fails, subsequent RCPT TO responses will be 503 (bad sequence). Robust clients parse all pipelined responses and use RSET to recover from failures.
The CHUNKING extension (RFC 3030) introduces the BDAT command as an alternative to DATA, enabling efficient binary message transfer without dot-stuffing.
Extension Keyword: CHUNKING
The Problem with DATA:
The traditional DATA command has inefficiencies:
. must be escaped (dot-stuffed)<CRLF>.<CRLF>BDAT Solution:
BDAT chunk-size [LAST]
BDAT transfers a specified number of raw bytes:
C: BDAT 1000
C: [exactly 1000 bytes of message content]
S: 250 1000 bytes received
C: BDAT 5000 LAST
C: [exactly 5000 bytes, completing the message]
S: 250 Message complete, 6000 bytes total
Advantages:
123456789101112131415161718192021222324252627
C: EHLO client.example.orgS: 250-mail.example.comS: 250-CHUNKINGS: 250-8BITMIMES: 250 OK C: MAIL FROM:<alice@example.org> BODY=BINARYMIMES: 250 OK C: RCPT TO:<bob@example.com>S: 250 OK # Send message in chunksC: BDAT 500C: [500 bytes: headers]S: 250 500 bytes received C: BDAT 2048C: [2048 bytes: first part of body]S: 250 2048 bytes received C: BDAT 10240 LASTC: [10240 bytes: attachment data, completing message]S: 250 2.0.0 Message complete, 12788 bytes C: QUITS: 221 Bye| Aspect | DATA | BDAT (CHUNKING) |
|---|---|---|
| Termination | <CRLF>.<CRLF> | Explicit byte count + LAST flag |
| Dot stuffing | Required | Not needed |
| Binary data | Must be encoded | Native support (BINARYMIME) |
| Progress tracking | Difficult | Chunk-by-chunk confirmation |
| Error at end | Only after full message | After any chunk |
| Adoption | Universal | Common but not universal |
BDAT is particularly valuable for large messages with binary attachments. By chunking the transfer, both sides can manage memory more efficiently, and errors are detected earlier. If sending large emails programmatically, prefer BDAT when available.
The SMTPUTF8 extension (RFC 6531) enables internationalized email addresses—addresses containing non-ASCII characters in the local part, domain, or both.
Extension Keyword: SMTPUTF8
The Historic Limitation:
Traditional email addresses were restricted to ASCII:
alice@example.com ✓münchen@example.de ✗用户@例子.中国 ✗This excluded billions of potential users whose names contain characters outside the ASCII range.
Internationalized Email Addresses (EAI):
SMTPUTF8 enables:
UTF-8 Local Part:
münchen@example.de — German city name用户@example.com — Chinese charactersпользователь@example.ru — CyrillicInternationalized Domain Names (IDN):
user@例え.jp — Japanese domainuser@مثال.مصر — Arabic domainFull Internationalization:
用户@例子.中国 — Both parts non-ASCIIHow SMTPUTF8 Works:
C: EHLO client.example.org
S: 250-mail.example.com
S: 250-SMTPUTF8
S: 250 OK
C: MAIL FROM:<münchen@example.de> SMTPUTF8
S: 250 OK
C: RCPT TO:<用户@例子.中国>
S: 250 OK
C: DATA
S: 354 Start input
C: From: Benutzer <münchen@example.de>
C: To: 用户 <用户@例子.中国>
C: Subject: =?UTF-8?B?...?= or UTF-8 directly with SMTPUTF8
C: MIME-Version: 1.0
C: Content-Type: text/plain; charset=UTF-8
C: ...
C: .
S: 250 OK
| Consideration | Details |
|---|---|
| Server support | Must support SMTPUTF8 end-to-end; can't downgrade |
| Downgrade path | RFC 6857 defines ASCII-only downgrades where possible |
| Bounces | Bounces must also support SMTPUTF8 if original did |
| DNS | Requires IDN support (Punycode: 例子.中国 → xn--fsqu00a.xn--fiqs8s) |
| Headers | All headers may contain UTF-8 when SMTPUTF8 is used |
| Display | Email clients must render UTF-8 correctly |
SMTPUTF8 is supported by major providers (Gmail, Outlook.com) but not universally deployed. Organizations can send internationalized email to SMTPUTF8-capable recipients, but fallback addresses may be needed for legacy systems. The technology is mature; adoption is the limiting factor.
Beyond the core extensions covered in depth, several other ESMTP extensions provide important functionality. Here's a comprehensive overview:
| Extension | RFC | Purpose | Common Use |
|---|---|---|---|
| DSN | RFC 3461 | Delivery Status Notifications | Request delivery/failure/delay reports |
| ENHANCEDSTATUSCODES | RFC 2034 | Extended status codes (X.Y.Z format) | Detailed error information |
| DELIVERBY | RFC 2852 | Delivery deadline specification | Time-sensitive messages |
| ETRN | RFC 1985 | Remote queue start | Trigger deferred delivery |
| ATRN | RFC 2645 | Authenticated ETRN | Secure queue initiation |
| NO-SOLICITING | RFC 3865 | Spam blocking hint | Indicate no spam tolerance |
| MTRK | RFC 3885 | Message tracking | Trace message status |
| SUBMITTER | RFC 4405 | Responsible submitter ID | Sender policy framework support |
| REQUIRETLS | RFC 8689 | Mandatory TLS for message | Force encryption end-to-end |
| LIMITS | RFC 9422 | Server limit advertisement | Inform clients of specific limits |
DSN (Delivery Status Notification) Deep Dive:
DSN provides reliable notification about message delivery:
C: MAIL FROM:<alice@example.org> RET=FULL ENVID=ABC123
S: 250 OK
C: RCPT TO:<bob@example.com> NOTIFY=SUCCESS,FAILURE,DELAY
S: 250 OK
NOTIFY options:
NEVER — No notificationsSUCCESS — Notify on successful deliveryFAILURE — Notify on permanent failureDELAY — Notify if message is delayedRET options:
FULL — Return entire message in DSNHDRS — Return headers onlyENVID: Client-specified identifier for tracking
ENHANCEDSTATUSCODES Example:
550 5.1.1 <unknown@example.com>: Recipient address rejected
The 5.1.1 enhanced code means:
RFC 8689's REQUIRETLS extension allows senders to mandate TLS for the entire delivery path. If any hop can't provide TLS, the message bounces rather than being transmitted insecurely. Use this for highly sensitive communications where plaintext exposure is unacceptable.
We've comprehensively explored SMTP extensions—the mechanism that has kept email's core protocol relevant for over four decades. Let's consolidate the essential knowledge:
| Need | Extension | Key Parameter |
|---|---|---|
| Encryption | STARTTLS | — |
| Authentication | AUTH | mechanism name |
| Size declaration | SIZE | byte count |
| 8-bit content | 8BITMIME | BODY=8BITMIME |
| Performance | PIPELINING | — |
| Binary transfer | CHUNKING | BDAT byte-count |
| Int'l addresses | SMTPUTF8 | SMTPUTF8 |
| Delivery receipts | DSN | NOTIFY=... |
Module Complete:
With this final page, you've completed the comprehensive SMTP module. You now understand SMTP's purpose, command vocabulary, mail submission and relay processes, and the extension mechanisms that enable modern email functionality. This knowledge forms the foundation for implementing, configuring, and securing email infrastructure.
Congratulations! You've mastered SMTP—from fundamental purpose through advanced extensions. You can now implement SMTP clients, configure mail servers, troubleshoot delivery issues, and appreciate the elegant design that has kept email functioning for over four decades. Next, explore POP3 and IMAP for mailbox access, or dive into email security with SPF, DKIM, and DMARC.