IMAP

Features

Commands are lexed incrementally with the shared push-parser style (tag / verb / arguments / literals) — synchronizing literals emit + continuations, and LITERAL- non-synchronizing literals are accepted. The APPEND message literal is never buffered whole: the codec emits it as a series of chunk events as wire bytes arrive, and the control layer streams each chunk straight to a bounded temp file, which is then streamed into the mailbox on proceed and removed — memory use stays flat regardless of message size.

Handler SPI

Gumdrop-style staged traits:

• ClientConnected → greeting (ImapConnectionMetadata, including optional traceparent)
• NotAuthenticatedHandler → LOGIN / AUTHENTICATE policy (proceed opens the store)
• AuthenticatedHandler → SELECT/LIST/STATUS/APPEND policy
• SelectedHandler → FETCH/STORE/SEARCH/COPY/MOVE/EXPUNGE policy

State traits constrain which replies are legal at each stage. The default handler (DefaultImapHandlerFactory) accepts everything the config allows and delegates storage I/O to the protocol via proceed. Storage work always runs on the Runtime storage pool, never on the reactor; while an operation is in flight the connection queues pipelined commands and answers them in order.

Quick start

use std::sync::Arc;
use hopf_auth::PasswordStore;
use hopf_core::{Runtime, RuntimeConfig};
use hopf_imap::{ImapConfig, ImapService};
use hopf_mailbox::MaildirFactory;

let rt = Arc::new(Runtime::start(RuntimeConfig::default())?);
let factory = Arc::new(MaildirFactory::new("./mail"));
let store = Arc::new(PasswordStore::new().with_user("alice", "secret"));
let config = ImapConfig::new("127.0.0.1:1143".parse()?, "localhost", store, factory);
let svc = ImapService::new(config, Arc::clone(&rt));
svc.start()?;

ImapConfig knobs: with_tls(acceptor) (+ implicit_tls() for IMAPS), with_greeting, with_quota_manager, with_server_id, and per-extension enable_* flags (idle, namespace, quota, move, condstore, qresync, enable, compress, utf8_accept, sort, thread_ordered_subject, thread_references, status_size, objectid, metadata, notify) — all default to enabled.

Example binary: cargo run -p imap -- 127.0.0.1:1143 localhost ./mail.

Client

The high-level ImapFetch auto-pilot drives greeting → CAPABILITY → (STARTTLS → CAPABILITY) → LOGIN / AUTHENTICATE PLAIN → SELECT → FETCH → LOGOUT. ImapIdle drives the same preamble then IDLE, delivering unsolicited EXISTS/EXPUNGE/FLAGS through a MailboxEventListener until DONE.

use std::sync::Arc;
use hopf_core::{Runtime, RuntimeConfig};
use hopf_imap::{ImapClient, ImapFetch};

let rt = Arc::new(Runtime::start(RuntimeConfig::default())?);

ImapClient::new("mail.example.com", 143)
    .connect(
        &rt,
        Arc::new(
            ImapFetch::new()
                .credentials("alice", "secret")
                .mailbox("INBOX")
                .on_message(Box::new(|seq, uid, body| {
                    println!("message {seq} uid={uid:?} ({} bytes)", body.len());
                }))
                .on_complete(Box::new(|ok| {
                    println!("fetch complete: {ok}");
                })),
        ),
    )?;

For full control implement ImapClientDriver: every server event surfaces as a callback (on_greeting, on_capability, on_authenticated, on_selected, on_fetch_data, on_status_data, on_list_entry, on_idle_started, …), and each callback receives the staged state reference (ImapClientNotAuthenticated → ImapClientAuthenticated → ImapClientSelected / ImapClientIdle) whose methods issue the commands that are legal in that state.

on_fetch_data delivers an ImapFetchData with FLAGS, UID, RFC822.SIZE, MODSEQ, EMAILID/THREADID parsed, and ENVELOPE, BODYSTRUCTURE (or BODY) and INTERNALDATE captured verbatim — any literal the server used inside them (a non-ASCII subject, say) is re-encoded as a quoted string, so each is one line of IMAP syntax. Parse them with ImapEnvelope::parse and ImapBodyStructure::parse; ImapBodyStructure::parts() lists every leaf with the section number BODY[section] would use for it (RFC 3501 §6.4.5, encapsulated message/rfc822 parts included), and display_candidates() picks the first non-attachment text/plain and text/html parts, which is what a mail client fetches instead of the whole message. Literal-bearing items (BODY[section], RFC822*) are streamed through on_fetch_literal* as before.

list_extended(reference, pattern, &ImapListOptions) issues an RFC 5258 LIST with selection options (SUBSCRIBED, REMOTE, RECURSIVEMATCH, SPECIAL-USE) and RETURN items (SUBSCRIBED, CHILDREN, SPECIAL-USE, and STATUS (…) per RFC 5819); the * STATUS replies a RETURN (STATUS …) provokes are routed to on_status_data interleaved with the on_list_entry calls, which is the one round trip a mail client needs for a folder list with unread counts. Extended data items after a LIST name (CHILDINFO, OLDNAME) are accepted and skipped. ImapCapabilities reports list_extended, list_status, special_use and children.

Commands that do not originate in a reply — a mail client whose user just clicked a folder, say — reach a live session through ImapClientDriver::on_wake. It runs at the start of every receive, so in particular after a hopf_core::ConnHandle::poke() from any thread (which re-enters receive with no data), and it is handed an ImapClientWakeState: the staged state reference for the session's current state (NotAuthenticated / Authenticated / Selected / Idle), or Busy while nothing can be issued (connecting, mid-STARTTLS, IDLE sent but not yet acknowledged). Whatever the driver issues is flushed when it returns. The pattern is: stash the ConnHandle from ep.handle() in an early callback, queue work somewhere the driver can see it, poke, and drain the queue in on_wake — putting back anything the current state does not allow. During IDLE the only legal command is done(); issue the real work from on_idle_complete, which hands over ImapClientSelected.

fn on_wake(&mut self, state: ImapClientWakeState<'_>, _ep: &mut dyn Endpoint) {
    let Some(job) = self.queue.lock().unwrap().pop_front() else { return };
    match (job, state) {
        (Job::Fetch(set), ImapClientWakeState::Selected(s)) => s.uid_fetch(&set, "(FLAGS BODY.PEEK[HEADER])"),
        (Job::Fetch(_), ImapClientWakeState::Idle(i)) => { i.done(); /* re-queue; see on_idle_complete */ }
        (job, _) => self.queue.lock().unwrap().push_front(job), // not now
    }
}
// elsewhere, on the UI thread:
queue.lock().unwrap().push_back(Job::Fetch("1:*".into()));
conn_handle.poke();

Client options

Two things the options table below does not cover. max_net_out(bytes) (and max_net_in) set this connection's buffer limits; hopf-core's 4 MiB outbound default closes the connection on a send that would overflow it, and ImapClientAppend::send_literal writes a whole APPEND literal in one call, so a client that appends messages must raise it above its largest message. And a dial that never produces a connection — DNS failure, no addresses, a connect that cannot start — reaches ImapClientHandlerFactory::connect_failed(host, error), which prints to stderr unless overridden; ImapFetch and ImapIdle override it to call their on_complete(false).

Example binary: cargo run -p imap-fetch -- 127.0.0.1 1143 alice secret.

Pipelining

The client keeps a tag-correlated PendingMap of outstanding commands (up to max_pipeline). Untagged replies are classified by prefix (STATUS, LIST, FETCH, SEARCH, …) and routed to the oldest compatible pending command, so tagged completions may arrive in any order. pipeline_status_and_list issues STATUS and LIST back-to-back as a demonstration; the Hopf server serialises storage-backed commands per connection and answers in order, but the client also handles out-of-order completion from other servers.

Timeouts

ImapClientTimeouts phases:

On expiry the driver’s on_timeout fires and the connection closes.

Testing

Unit tests run with the workspace default (cargo test -p hopf-imap --lib). Loopback TCP / TLS / filesystem integration tests are opt-in:

cargo test -p hopf-imap --features integration

They cover raw-socket LOGIN/SELECT/FETCH/APPEND, pipelined STATUS+LIST (in-order against the Hopf server and synthetic out-of-order against a scripted server), STARTTLS and implicit-TLS fetch, hostname dial, and IDLE (server DONE handling plus client EXISTS→DONE).

Metrics

Process-local ImapServerMetrics (in hopf-imap): connections, auth_ok / auth_fail, commands, and starttls via ImapService::metrics().

OTLP/JSONL export (when ImapService::with_telemetry(&pipeline) is used): hopf_otel::ImapServerMetrics emits imap.server.connections, imap.server.active_connections, imap.server.auth, imap.server.starttls, imap.server.commands (verb/outcome), and imap.server.command.duration (ms).

With traces enabled on the pipeline, ImapConnectionMetadata::traceparent carries the active W3C traceparent for the connection or current tagged command. Handler SPI implementers can pass it to outbound HTTP with hopf_otel::with_traceparent.

Limitations

• The ImapFetch/ImapIdle auto-pilots only drive AUTHENTICATE PLAIN themselves (same scope limit as hopf-pop3's auto-pilot) — a custom driver is needed to exercise LOGIN/CRAM-MD5/DIGEST-MD5/SCRAM-SHA-256/OAUTHBEARER/EXTERNAL from the client side, though the underlying authenticate(mechanism, initial) + ImapClientAuthExchange API is mechanism-agnostic. EXTERNAL isn't reachable end-to-end yet on either side: the TLS layer doesn't plumb a peer certificate through to SaslServerOptions.
• QRESYNC does not fabricate VANISHED (EARLIER) history; clients fall back to full resync.
• mbox backend: no hierarchy / COPY / MOVE (see Mailbox); IDLE refresh is a no-op for mbox (Maildir rescans new//cur/ each poll).
• BODYSTRUCTURE body size/line counts reflect rmimeparser's decoded body bytes; for base64/quoted-printable-encoded parts this differs from the RFC-mandated wire-encoded octet count (7bit/8bit/binary parts, including message/rfc822, are unaffected since decoded and wire bytes are identical there).
• BODYSTRUCTURE extension data always reports Content-Language/Content-Location as NIL (not parsed by rmimeparser's handler trait) and Content-Disposition parameters other than filename are omitted (rmimeparser only supports named parameter lookup, not enumeration).
• NOTIFY (RFC 5465) is a deliberately partial implementation: the personal/inboxes/subtree/mailboxes mailbox-selectors (real support needs polling every mailbox in the store, not just the selected one), the AnnotationChange/MailboxName/SubscriptionChange/MailboxMetadataChange/ServerMetadataChange event groups, MessageNew's optional status-item sublist, and SET STATUS's immediate catch-up response are all rejected with a tagged BAD rather than silently ignored.
• THREAD REFERENCES doesn't fabricate a new dummy parent to join two root-level messages that share a base subject when neither is itself a reply/forward (RFC 5256 §2.2 step 5C) — there is no thread-list wire form for a parent-less placeholder there, so the two threads are left separate instead of merged.
• OBJECTID (MAILBOXID/EMAILID) and per-mailbox/server METADATA storage are implemented against the Maildir backend only; the mbox backend returns no id/entries for either (the underlying Mailbox/MailboxStore trait methods default to None/empty).

GitHub