Inside our Go SMTP server: how authentication works without a database lookup

Every message that reaches a Mailclutch inbox arrives the same way: an application somewhere authenticates against smtp.mailclutch.com, sends a message, and gets a 250. The thing on the other end of that socket is not Postfix with a plugin. It is a small SMTP server written in Go, built for exactly one job — accept mail from authenticated clients and hand it to the application — and this article is about how it does the first half of that job.

We are writing it up because the design question turned out to be more interesting than "run an SMTP daemon". A sandbox receives credentials from anyone on the internet, all day, and most of them are wrong: typos, stale .env files, scanners walking through admin/admin. Asking the application to look up each one in the database is the obvious design, and it is the one we replaced.

What the server is

The server is a Go program built on emersion/go-smtp, which handles the protocol state machine, and a Backend we wrote that decides who may log in and what happens to a message. It listens on four ports — 25, 465, 587 and 2525 — and speaks to the Laravel application over HTTP for the two things only the application knows: whether a set of credentials belongs to a live inbox, and where a captured message should go.

There is no mail queue, no local delivery, no relaying to the outside world by default. A message is accepted, handed over, and forgotten. That narrowness is the point: the fewer things an internet-facing SMTP daemon can do, the fewer things it can be made to do.

Listening: four ports, two TLS modes

The four listeners share one configuration except for how they do TLS:

// implicitTLSPort is SMTPS: TLS from the first byte, no STARTTLS.
const implicitTLSPort = ":465"

ports := []string{":25", implicitTLSPort, ":587", ":2525"}

for _, p := range ports {
    go func(p string) {
        s := newSMTPServer(be, tlsConfig, p)
        if p == implicitTLSPort {
            log.Fatal(s.ListenAndServeTLS())
        }
        log.Fatal(s.ListenAndServe())
    }(p)
}

Port 465 wraps the connection in TLS before a single SMTP byte is exchanged. Ports 25, 587 and 2525 start in clear text and advertise STARTTLS; the client upgrades if it wants to. Both modes end up with the same server configuration, the same certificate — rotated by ACME and hot-reloaded without a restart — and a TLS 1.2 floor.

Why four ports? Because clients are not uniform. 587 is the standard submission port and what every framework guide on this site defaults to. 465 exists for clients that only do implicit TLS — some .NET mailers, some managed platforms. 2525 exists because a surprising number of cloud providers block outbound 25 and 587 from their compute. And 25 exists so that a misconfigured relay pointed at us still lands somewhere you can see it. The full reasoning is in which SMTP port to use.

Two limits are set on every listener:

s.MaxMessageBytes = int(GetMaxMessageBytes()) // 36 MiB by default
s.MaxRecipients   = 50

The message cap is derived from the application's per-attachment limit plus base64 and header overhead, so a message the application would refuse is refused at the socket instead of after a full upload.

Authentication: PLAIN and LOGIN, one code path

go-smtp enables AUTH PLAIN by default. That covers modern libraries, but not everything: Outlook, older MTAs and swaks --auth LOGIN speak only AUTH LOGIN, a slightly older mechanism that sends the username and password as two separate base64 challenges instead of one. The upstream SASL library provides a LOGIN client but no server, so we wrote one — about forty lines — and pointed both mechanisms at the same Backend.Login:

type loginServer struct {
    authenticate func(username, password string) error
    username     string
    state        int
}

func (s *loginServer) Next(response []byte) (challenge []byte, done bool, err error) {
    switch s.state {
    case loginStateInit:
        // Clients may send the username as the initial response.
        if len(response) == 0 {
            s.state = loginStateUsername
            return []byte("Username:"), false, nil
        }
        s.username = string(response)
        s.state = loginStatePassword
        return []byte("Password:"), false, nil
    case loginStateUsername:
        s.username = string(response)
        s.state = loginStatePassword
        return []byte("Password:"), false, nil
    case loginStatePassword:
        s.state = loginStateDone
        return nil, true, s.authenticate(s.username, string(response))
    default:
        return nil, false, sasl.ErrUnexpectedClientResponse
    }
}

The mechanism is only transport. What matters is what authenticate does with the pair — and the interesting decision is what it does before asking the application.

If you want the protocol-level view of PLAIN versus LOGIN — the base64, the transcripts, what a 535 means — that is its own article: SMTP AUTH explained.

The problem with looking credentials up

The first version of the server did the obvious thing. Every AUTH attempt became a signed POST /api/smtp-auth to the application, which booted, queried the inbox table, and answered yes or no. Successful pairs were cached for a while. Failures were not.

That is fine at the volume of legitimate clients. It is not fine at the volume of the internet. An SMTP port that is open to the world receives credential-stuffing traffic continuously, and every guess cost us one HMAC-signed HTTP request, one framework boot and one database query — to learn that admin/admin is not an inbox. The application was doing the sandbox's bouncer work, and the bouncer was the expensive part.

The usual answer is rate limiting, and we do have caches. But rate limiting is a throttle on a cost you are still paying. We wanted the cost to be zero for a credential that could not possibly be valid.

Self-certifying credentials

The fix is that a Mailclutch credential pair is derived, not random. The username carries a small random seed; the password is a MAC of the username under a secret key:

username = "mc_" + base32(seed)
password = "pw_" + base32(HMAC-SHA256(key, username)[0:10])

Both are 19 characters, lowercase alphanumeric, in an alphabet chosen so nothing an SMTP client does to them — case folding, double-click selection, copying out of a terminal — can change them.

The consequence is that anyone holding the key can verify a pair offline, and nobody else can produce one that verifies. The SMTP server holds the key. So before it talks to anyone, it recomputes the MAC:

func (bkd *Backend) Login(state *smtp.ConnectionState, username, password string) (smtp.Session, error) {
    if !verifyCredentials(username, password) {
        // Not minted under our keys. Nothing upstream could accept it,
        // so nothing upstream is asked.
        return nil, &smtp.SMTPError{Code: 535, EnhancedCode: smtp.EnhancedCode{5, 7, 8}, Message: "Authentication credentials invalid"}
    }
    // ...positive/negative cache, then the application.
}

A guessed pair fails here, at the cost of one SHA-256 — on the order of a microsecond — with no network traffic, no application boot, no database. A scanner can run through a million guesses and the application never hears about a single one.

A pair that does verify is, at that moment, only known to be well-formed. It still has to correspond to a live inbox, which is the application's decision: the server forwards it, the application re-verifies the derivation (defence in depth — the server is not trusted to have done it), then checks the row, its expiry, its owner. That call is cached, positively and negatively, in bounded in-memory maps so a valid client's second message costs nothing and a stale-but-well-formed pair does not hammer the application either.

What the key protects, and what it does not

It is worth being precise about the threat model, because "HMAC" can sound like more than it is.

• The key does not protect existing inboxes. Access to an inbox requires the pair the application stored for it; a forged pair that verifies under the key still fails the application's stored-password check.
• The key does protect the application from spending resources on garbage. If it leaked, an attacker could mint well-formed pairs and make the application evaluate them — one cheap call each — and could self-activate sandboxes at volume. That is a nuisance, not a breach.
• A blank key fails closed on every side: nothing can be minted, every login is refused. There is no mode in which "no key configured" means "accept everything".

The scheme is a filter, in other words — the same way SPF is a filter. Its job is to make the definite "no" cheap so the expensive "maybe" is only asked of candidates worth asking about.

Rotation

Keys will need to be rotated, so the server does not hold one key; it holds a ring, each entry with a small id that the username encodes. The first entry mints, every entry verifies. Rotation is therefore two deploys: append the new key to the ring everywhere (verify-only), then move it to the front of the ring on the components that mint. Revoking a leaked key is removing it — every pair minted under it stops verifying, at the socket, immediately.

Three separate systems hold the ring: the SMTP server, the application, and this website. That last one matters more than it sounds.

Why the marketing site holds the key

Every framework guide on this site shows credentials that work. Not YOUR_USERNAME_HERE — a real mc_…/pw_… pair, minted for your visit, that you can paste into a .env and send to.

Before derived credentials, that meant the website had to ask the application to create a sandbox row on every page view, just to have something to display. Most of those rows were never used. Now the site mints a pair itself, with no backend call and no database write. The sandbox comes into existence the first time the pair is actually used — the first authenticated message, or the first time someone opens its URL — because the application recognises the pair as well-formed and creates the row on demand.

That is the same self-certification property doing a second job: the credential is the proof of its own validity, so whoever holds the key can hand one out without coordinating with anyone.

The application boundary

The server talks to the application over two endpoints: authenticate this pair, and store this message. Both are signed. Every request carries a timestamp and an HMAC-SHA256 of {timestamp}.{body} under a shared secret, and the application refuses anything unsigned, mis-signed, or more than five minutes old. A blank secret disables the endpoints outright with a 503 rather than leaving them open.

This exists because "these routes are only called from the SMTP server's IP" was, at one point, a thing we believed and could not enforce — the routes are served by the same ingress as everything else. Signing makes the assumption true regardless of network topology.

The only deliberate omission is a replay cache. Replaying a captured store this message request re-delivers a message that was already delivered — to the same inbox, where it is already visible. Paying a cache round trip on every captured message, and adding a hard dependency on that cache being up, was the wrong side of that trade.

What we would tell someone building one

• Own the protocol layer, not the protocol. A well-maintained library for the SMTP state machine, and your own backend for the decisions. The interesting code was never in parsing RCPT TO.
• Make the definite "no" free. Anything that reaches your application should already have passed a check that no attacker can pass by guessing.
• Support LOGIN. Not because it is good — it is not — but because the clients that need it will not tell you why they cannot connect.
• Sign the internal API even if it is internal. Especially if it is internal.
• Fail closed on missing configuration. A server that starts with no keys and accepts everyone is a server that will, eventually, be deployed that way.

If you want to see the result from the other side: point your staging at it, send a message, and read back the headers.