SMTP & Email Protocols
SMTP sends email between servers, IMAP/POP3 retrieves it for clients, and SPF/DKIM/DMARC authenticate everything to fight spam.
The Problem
Email is the oldest internet communication protocol still in daily use. How does it actually work end-to-end, and why is reliably delivering email to an inbox — not spam — so surprisingly difficult?
Mental Model
Like the physical postal system - the local post office (SMTP server) routes to the recipient's post office, which holds it in a mailbox (IMAP)
Architecture Diagram
How It Works
Email is deceptively simple on the surface but remarkably complex underneath. Sending a message from alice@company-a.com to bob@company-b.com involves DNS lookups, multi-hop server routing, cryptographic authentication, and reputation scoring — all before the recipient sees anything.
The Journey of an Email
Step 1: Submission. Alice's email client (Gmail, Outlook, Thunderbird) connects to her outgoing mail server via SMTP on port 587 (submission port) with TLS encryption. She authenticates with her credentials.
Step 2: DNS MX Lookup. Alice's mail server queries DNS for the MX (Mail Exchange) records of company-b.com to find where to deliver the message.
Step 3: Server-to-Server Transfer. Alice's MTA connects to Bob's MTA on port 25 (the standard SMTP relay port) and delivers the message. If the primary MX server is unavailable, it tries the backup MX (lower priority number = higher priority).
Step 4: Authentication. Bob's MTA checks SPF, DKIM, and DMARC before accepting the message.
Step 5: Delivery. The message is stored in Bob's mailbox on the server.
Step 6: Retrieval. Bob's email client connects via IMAP (port 993 with TLS) or POP3 (port 995 with TLS) to download or sync the message.
The SMTP Conversation
SMTP is a text-based protocol with a simple command-response structure:
S: 220 mail.company-b.com ESMTP Postfix
C: EHLO mail.company-a.com
S: 250-mail.company-b.com Hello
S: 250-STARTTLS
S: 250-AUTH PLAIN LOGIN
S: 250 OK
C: STARTTLS
S: 220 Ready to start TLS
[TLS handshake]
C: MAIL FROM:<alice@company-a.com>
S: 250 OK
C: RCPT TO:<bob@company-b.com>
S: 250 OK
C: DATA
S: 354 Start mail input
C: From: Alice <alice@company-a.com>
C: To: Bob <bob@company-b.com>
C: Subject: Meeting tomorrow
C: Date: Mon, 6 Apr 2026 10:00:00 +0000
C: DKIM-Signature: v=1; a=rsa-sha256; d=company-a.com; ...
C:
C: Hey Bob, are we still on for tomorrow?
C: .
S: 250 OK: queued as ABC123
C: QUIT
S: 221 Bye
Each command gets a three-digit response code: 2xx (success), 3xx (continue), 4xx (temporary failure, retry later), 5xx (permanent failure, don't retry).
SPF, DKIM, and DMARC — The Authentication Trifecta
Email has no built-in sender verification. Anyone can claim to be ceo@any-company.com in the From header. SPF, DKIM, and DMARC were created to fix this, and all three are required.
SPF (Sender Policy Framework)
SPF declares which IP addresses are allowed to send email for a domain. It's a DNS TXT record:
company-a.com. IN TXT "v=spf1 ip4:203.0.113.0/24 include:_spf.google.com include:sendgrid.net ~all"
This says: email from company-a.com should come from the 203.0.113.0/24 network, Google's mail servers, or SendGrid. Everything else is suspicious (~all = soft fail, -all = hard fail).
Limitation: SPF only validates the envelope sender (MAIL FROM), not the From header the user sees. A phisher can pass SPF while showing a different From address.
DKIM (DomainKeys Identified Mail)
DKIM cryptographically signs the email content. The sending server adds a signature header, and the receiving server verifies it using a public key published in DNS:
# DNS record
selector._domainkey.company-a.com. IN TXT "v=DKIM1; k=rsa; p=MIGfMA0GCS..."
# Email header
DKIM-Signature: v=1; a=rsa-sha256; d=company-a.com; s=selector;
h=from:to:subject:date:message-id;
bh=2jUSOH9NhtVGCQWNr9BrIAPreKQjO6Sn7XIkfJVOzv8=;
b=AuUoFEfDxTDkHlLXSZEpZj79LICEps6eda7W3deTVFOk2G...
DKIM proves that the message content hasn't been tampered with and was signed by the domain's private key. It survives forwarding (unlike SPF).
DMARC (Domain-based Message Authentication, Reporting & Conformance)
DMARC ties SPF and DKIM together and tells receiving servers what to do when authentication fails:
_dmarc.company-a.com. IN TXT "v=DMARC1; p=reject; rua=mailto:dmarc@company-a.com; pct=100"
p=none— Monitor only, deliver everything (use during initial setup)p=quarantine— Send failures to spamp=reject— Reject failures entirely
The rua tag specifies where to send aggregate reports — these show who is sending email as the domain, both legitimate and fraudulent.
The deployment order: Start with p=none to collect data, then p=quarantine at 10%, gradually increase to 100%, then switch to p=reject.
IMAP vs POP3 — How Recipients Get Their Email
IMAP (Internet Message Access Protocol) keeps messages on the server and synchronizes state across all devices. When an email is read on the phone, it's marked as read on the laptop too. IMAP supports folders, flags, and search.
POP3 (Post Office Protocol v3) downloads messages to the client and optionally deletes them from the server. It's simpler but doesn't sync across devices. POP3 is essentially "check the mailbox and take the letters home."
For modern use: always use IMAP. POP3 is only relevant for legacy systems or specific compliance scenarios where messages must be removed from the server immediately.
Why Email Deliverability Is Hard
Getting email into the inbox — not the spam folder — is an ongoing battle. Here's what affects deliverability:
IP Reputation: Email providers track the sending history of every IP address. A new IP with no history is treated with suspicion. An IP that has sent spam is blacklisted. New IPs must be "warmed up" by gradually increasing sending volume over 2-4 weeks.
Domain Reputation: Independent of IP reputation. Gmail, Microsoft, and others track reputation per domain. New domains have no reputation and face throttling.
Content Quality: Spam filters analyze subject lines, body content, link URLs, image-to-text ratio, and HTML structure. Avoid spam trigger words, include plain-text alternatives, and use a consistent sending identity.
Engagement: Gmail and others use recipient engagement (opens, clicks, replies, marks-as-spam) to determine inbox placement. Sending to users who never open the emails damages sender reputation.
List Hygiene: Remove hard bounces immediately. Suppress unsubscribes. Regularly prune inactive subscribers. High bounce rates signal spam-like behavior.
# Check if an IP is blacklisted
# Use mxtoolbox.com or check manually:
dig +short 34.0.113.203.zen.spamhaus.org
# If it returns 127.0.0.x, the IP is blacklisted
# Check the domain's email authentication
dig TXT example.com # SPF
dig TXT _dmarc.example.com # DMARC
dig TXT s1._domainkey.example.com # DKIM
Best Practices for Production Email
-
Use a dedicated sending service (SendGrid, SES, Mailgun) for transactional email. They manage IP reputation, handle bounces, and provide analytics.
-
Separate transactional and marketing email on different IPs/subdomains. A marketing campaign that gets marked as spam shouldn't affect the password reset emails.
-
Implement all three authentication records — SPF, DKIM, and DMARC with
p=reject. This is non-negotiable in 2026. -
Monitor DMARC reports — use a service like dmarcian or Valimail to visualize who is sending email as the domain.
-
Handle bounces programmatically — suppress hard bounces (5xx) immediately, retry soft bounces (4xx) with exponential backoff, and monitor bounce rates.
Email is 50+ years old and will outlive most modern messaging platforms. Understanding its internals — from SMTP handshakes to DMARC policies — is essential for any engineer building systems that send email at scale.
Key Points
- •Email delivery is a multi-hop process: sender client → sender MTA → DNS MX lookup → recipient MTA → recipient IMAP server → client
- •SPF, DKIM, and DMARC work together — SPF validates the sending server, DKIM signs the message content, DMARC sets the policy
- •SMTP uses a store-and-forward model — each server accepts the message and takes responsibility for delivery or bounce
- •Email deliverability depends on IP reputation, authentication records, content quality, and recipient engagement
- •IMAP keeps mail on the server and syncs state across devices; POP3 downloads and (optionally) deletes from server
Key Components
| Component | Role |
|---|---|
| SMTP (Simple Mail Transfer Protocol) | The protocol for sending email between mail servers and from clients to their outgoing mail server |
| IMAP (Internet Message Access Protocol) | Allows clients to read, organize, and sync email from the server while keeping messages stored server-side |
| MX Records | DNS records that specify which mail servers accept email for a domain, with priority ordering |
| SPF / DKIM / DMARC | Email authentication trifecta that proves sender identity and prevents spoofing and phishing |
| MTA (Mail Transfer Agent) | The server software (Postfix, Sendmail, Exchange) that routes email between organizations |
When to Use
Use managed email services (SendGrid, SES, Mailgun) for transactional email (receipts, password resets, notifications). Use self-hosted SMTP only when specific compliance or volume requirements exist and the team has the expertise to manage deliverability.
Tool Comparison
| Tool | Type | Best For | Scale |
|---|---|---|---|
| Postfix | Open Source | Self-hosted MTA with excellent performance and security defaults | Handles millions of messages per day on modest hardware |
| SendGrid | Managed | Transactional and marketing email with deliverability optimization and analytics | Sends 100+ billion emails per month across all customers |
| AWS SES | Managed | Cost-effective email sending integrated with AWS infrastructure | Pay-per-email with dedicated IPs available |
| Mailgun | Managed | Developer-focused email API with powerful log search and email validation | Startup to enterprise transactional email |
Debug Checklist
- Check SPF record: dig TXT example.com — look for v=spf1 include:_spf.google.com ~all
- Verify DKIM: dig TXT selector._domainkey.example.com — should return a public key
- Check DMARC policy: dig TXT _dmarc.example.com — should return v=DMARC1; p=reject or p=quarantine
- Test SMTP directly: openssl s_client -connect mail.example.com:587 -starttls smtp
- Check MX records: dig MX example.com — verify priority ordering and reachability
Common Mistakes
- Not setting up SPF, DKIM, and DMARC records — without all three, email will land in spam
- Using a shared IP for transactional email — one bad neighbor's spam can tank the sender's IP reputation
- Sending from a new domain/IP without warming up — ISPs throttle unknown senders aggressively
- Not handling SMTP bounce codes correctly — soft bounces (4xx) should retry, hard bounces (5xx) should remove the address
- Assuming email delivery is instant — SMTP allows servers to queue and retry for up to 5 days
Real World Usage
- •Gmail processes over 1.8 billion active accounts and uses machine learning to filter 99.9% of spam
- •Stripe sends transactional receipts via SES/SendGrid with SPF, DKIM, and custom return-path for deliverability
- •GitHub notification emails include List-Unsubscribe headers and proper DKIM signatures for inbox placement
- •Marketing platforms like Mailchimp manage IP warming, bounce handling, and reputation across thousands of senders
- •Enterprise Exchange/O365 deployments handle internal email routing and external delivery with DLP and compliance scanning