Email Service: requirements, scale and the protocols

"Design a distributed email service." An unusual shape: enormous storage, a per-user search problem, and a mandatory forty-year-old protocol you do not control.

This lesson scopes the problem, runs the numbers that make storage the leading constraint, and covers just enough protocol to design around it. The second lesson designs the storage and search; the third covers outbound delivery, spam and the follow-up questions.

What makes email different from everything before it

Three things.

You do not own the other end. A feed, a chat system, and a video platform all talk to clients you wrote. Email must exchange messages with tens of thousands of independent servers running software you have never seen, using a protocol standardised long before you started. Interoperability is not a feature you can descope.

Storage dominates. Most systems in this course are constrained by queries per second. This one is constrained by petabytes. The arithmetic later in this lesson shows the write rate is modest and the storage figure is enormous, which inverts the usual order of design decisions.

Search is per-user and mandatory. A shared inverted index over everyone's mail is a privacy catastrophe. Every user needs their own searchable index over their own mail, and building millions of small indexes is a different problem from building one big one.

The questions that shape everything after

1. What is in scope beyond send and receive? Folders, labels, threading, search, attachments, spam filtering, and calendar are each substantial. Narrow it and say so.
2. Must it interoperate with the outside world? The answer is yes, and you should say so before the interviewer does. An email system that only talks to itself is a chat system.
3. How many users and how much storage each? This is the leading number.
4. What retention and deletion guarantees? "Deleted" that must be genuinely unrecoverable changes the storage design, especially with deduplication.
5. Web, mobile, and third-party clients? Third-party clients mean supporting the retrieval protocols, not only your own API.
6. What availability and durability targets? Losing mail is worse than being briefly unable to read it, which is a hierarchy worth stating.

The assumptions this section uses

Requirements and scale

Functional requirements

• Receive mail from any internet server and place it in the right mailbox.
• Send mail to any internet server, with retry on temporary failure.
• List, read, search, label, move, and delete messages.
• Handle attachments up to a stated size limit.
• Present conversations as threads.

Non-functional requirements

• Accepted mail is never lost. Durability is the top requirement.
• Mailbox listing and message read under 200 ms at the 99th percentile.
• Search under one second for a typical mailbox.
• Available across a datacentre failure.

The arithmetic

Users and storage. 100 million active users at 2 GB average = 200 PB. Replicated three ways for durability, 600 PB. That is the number that leads the design, and it is larger than anything else in this course.

Message rate. Assume 20 delivered messages per user per day: 100 million × 20 = 2 billion messages per day, or 2 billion ÷ 100,000 s = 20,000 messages per second average, and 60,000 per second at a 3× peak.

Message size. Most mail is small — a few kilobytes of text and headers. Attachments skew the average hard. Take a mean of 100 KB, understanding the distribution is extremely long-tailed: perhaps 90% of messages under 20 KB and 1% over 5 MB.

Ingest bandwidth. 20,000/s × 100 KB = 2 GB/s, or 16 Gbit/s sustained. Peak triples it.

Daily growth. 2 billion × 100 KB = 200 TB per day before deduplication and before spam rejection. Two mitigations, both large:

• Spam rejection at the edge removes a majority of connections before a message body is ever stored. If 60% is rejected, the stored figure drops to 80 TB per day.
• Attachment deduplication. A 4 MB deck mailed to 300 colleagues is stored once, not 300 times. Storage design quantifies this; savings of tens of per cent on total bytes are plausible.

Metadata volume. Per message: message ID, mailbox ID, thread ID, sender, recipients, subject, timestamp, flags, size, and blob pointers. Call it 1 KB. 2 billion per day = 2 TB of metadata per day, and 730 TB per year. Metadata is 1% of total bytes and 100% of the query load — which is exactly why Storage design separates it.

What the numbers rule out

• One database for everything. 200 PB will not sit in a database, and metadata queries should not compete with 2 GB/s of body writes.
• A single global search index. 100 million users × their own mail is not one index. Search explains why per-user indexes are required rather than preferable.
• Storing everything on fast media. Mail older than a year is read almost never. Paying fast-storage prices for 90% of 200 PB is the single largest avoidable cost in the system.

The protocols, briefly

Before the high-level design, enough protocol to be credible. This is not a protocol lecture, and an interviewer who wanted one would have asked a different question.

Receiving a message, step by step

1. A sending server looks up the MX records for example.com and picks one by priority.
2. It opens a connection to port 25 and speaks SMTP: a greeting, the envelope sender, the envelope recipients, then the message data.
3. Your edge server may reject at any point — a bad sender reputation, an unknown recipient, a rate limit — and rejecting early is cheap, which is the whole reason spam defence lives at the edge.
4. On acceptance, your server takes responsibility for the message. That is a promise: once you have said "250 OK", losing the message is a data-loss incident, not a delivery failure.
5. The message is written durably, then processed: spam scoring, virus scanning, rule application, threading, indexing, and finally mailbox placement.

Step 4 is the one to emphasise. The acceptance boundary is where durability requirements begin, and any design that acknowledges before the message is durably stored is wrong.

Sending a message

1. The client submits over your HTTP API, or over SMTP on the submission port with authentication.
2. The message is queued.
3. A delivery worker resolves the recipient domain's MX records and attempts SMTP delivery.
4. A temporary failure — the receiving server is busy or greylisting you — means retry with backoff over hours or days. A permanent failure means generating a bounce message back to the sender.

The modern HTTP API on top

Your own web and mobile clients should not speak IMAP. A purpose-built HTTP API returns exactly what a mailbox view needs in one round trip, supports push notification of new mail, and evolves without a standards process.

Keep IMAP available for third-party clients, and be honest that it is expensive to support: IMAP is stateful, holds long-lived connections, and its synchronisation model — sequence numbers and unique identifiers per folder — does not map cleanly onto a label-based mailbox where one message appears in several places. Real providers maintain a compatibility layer that translates, and it is a persistent source of complexity.