Architecture

Purpose

The default topology is a graph of endpoints on one Runtime: N listeners and M dialers in one process are normal (peer / P2P compositions), not a third stack. Clients routinely host multiple protocols together — at minimum DNS under dial.

Many Internet application protocols share the same transport model: TLS/QUIC where appropriate, transport-level backpressure, worker/storage pools independent of connection count, and plain buffers (&[u8] / pooled owned buffers) rather than a proprietary buffer world.

Hopf is a re-implementation of Gumdrop’s techniques, protocol work, and architectural contracts, with Rust ownership and TPC execution — not a line-by-line port of Java types.

Thread-per-core

• Each connection is owned by one reactor thread for its lifetime (affinity).
• I/O, TLS, and protocol byte handling stay on that thread.
• Application / blocking / storage work runs on a separate storage pool.
• Cross-core coordination is explicit (ConnHandle, execute, message passing) — not implicit task migration.

Optional future: io_uring completion paths for Linux behind the same Endpoint / handler traits — evolution, not a day-one requirement.

I/O substrate (mio)

Build on mio (epoll / kqueue readiness) with an in-tree executor, timer, and registration model. Hopf uses mio only, not the Tokio runtime.

This preserves the readiness mental model (interest ops, fill buffer, parse, write queue, backpressure), portable development (including macOS), and a near-zero dependency surface for the reactor.

Accept loops, per-core reactors, timers, cross-thread queues, buffer pools, and listen/dial APIs are implemented in-tree. Reactor/callback pattern for all I/O.

Storage and DNS

mio covers sockets, not arbitrary filesystem I/O. Mail, FTP, WebDAV, and large body paths use an explicit storage executor: blocking or off-loop file work, never on the reactor thread.

Name resolution is Runtime substrate, not a bolt-on product:

• Per-reactor DnsResolver (sockets, pending queries, timers on that core).
• Process-shared config + TTL cache where appropriate.
• Dial-by-name resolves on the target reactor; callbacks run there, then dial TCP/QUIC.
• No blocking getaddrinfo on reactor threads.

See DNS, runtime options.

Layering

Transport vs HTTP Stream

H1 adapts a TCP Endpoint into serialised Streams; H2 multiplexes Streams on one TCP Endpoint; H3 maps each request to a QUIC stream Endpoint. Bind vs dial only affects how the transport was born; server/client role is a separate axis. Terminology: HTTP Stream ≠ H2 stream id ≠ QUIC stream endpoint.

Endpoint vs ProtocolHandler

HTTP adapters (H1Endpoint, H2Endpoint, CleartextHttpEndpoint, AlpnHttpEndpoint, H3 handlers) are ProtocolHandlers that drive Stream handlers. receive uses NIO compact semantics: advance the &mut &[u8] cursor; unconsumed suffix is preserved.

Listen and dial are peers

Both reduce to registering a stream on a reactor with lifetime affinity.

Module layout: server and client as peers

Gumdrop assumes server functionality by default, with client as a submodule. Hopf deliberately diverges: within a protocol crate, server (bind) and client (dial) are equal, not a default role plus an add-on. This is a module-layout axis distinct from the listen/dial API axis above — it governs where code lives, not how a socket was born. Three layouts are legitimate, chosen per crate by how tightly client and server share wire state:

Anti-pattern: server code flat at crate root with no server/ name, plus a client/ submodule bolted on — server-as-unmarked-default is exactly the Gumdrop shape this axis exists to avoid. Crates in this shape get a server/ module carved out of their root files, mirroring their existing client/.

Dynamic bindings and composition

All bindings are dynamic. Listen and dial are added/removed through Runtime APIs while the process is alive. Configuration is a script over those APIs (Rust main / builder, or XML). Service orchestrates lifecycle; it does not own an immutable listener list as source of truth.

• Canonical: Rust Composition builder.
• Declarative: XML via tractrix → same builder; closed registry of handler / trust names — no reflective DI.
• TOML/YAML are not primary composition formats.

Details: composition, services, clients.

Auth vocabulary

Prefer TrustPolicy + IdentityMaterial over Realm-as-root naming. Attach to both listen and dial. SASL client/server are wire roles, independent of listen/dial. See auth.

Codec style

Grammar-driven incremental codecs: each protocol owns a self-contained parser (H1Scanner, Pop3ServerLexer, SmtpServerLexer, FtpServerLexer, ImapServerLexer, …) with its own token alphabet and bounded in-progress-token scratch state; update the parse FSM as each production completes; CRLF is one production, not the only event boundary. A parser consumes every byte it's handed in a single call and never asks the caller to retain or re-supply anything. Early parse state ≠ early app callbacks. Sibling incremental parsers (tractrix, rjsonparser, rmimeparser, rprotobuf) follow the same push model for XML/JSON/MIME/protobuf.

Security substrate

Encrypted transport and shared authentication crypto are part of hopf-core's contract — not a bolt-on crate graph. Gumdrop treated TLS and DTLS as framework-transparent; Hopf follows the same idea with Rust ownership and TPC reactors.

Today

• AWS-LC (BoringSSL lineage) is the native crypto library, reached through aws-lc-rs / aws-lc-sys.
• TCP TLS / STARTTLS: in-tree TLS 1.2/1.3 in hopf-core::tls — TlsVariant (V13 / V12 / Negotiating) driven by TcpConnection through TlsRecordSink; PEM helpers return SharedTlsAcceptor / SharedTlsConnector wired with TcpListenerConfig::with_tls, with_starttls_acceptor, or TcpConnectorConfig::with_tls. Handlers see plaintext; metadata in SecurityInfo after handshake.
• QUIC / HTTP/3: in-tree RFC 9000 transport and its TLS 1.3 handshake for RFC 9001, both in hopf-quic on AWS-LC directly — no quinn-proto or other external QUIC/TLS library dependency anywhere in the workspace; H3/QPACK in-tree in hopf-http.
• Mail/DNS auth crypto: DKIM and DNSSEC verification call aws-lc-rs directly from protocol crates.
• UDP: datagram handlers in core are cleartext; DTLS 1.2/1.3 sessions (hopf-core::dtls/dtls12) have a reactor-driven UDP driver and per-peer 1.3-vs-1.2 version negotiation (DtlsVersionPolicy, mirroring TCP's TcpTlsVersionPolicy). No protocol crate dials or listens over DTLS yet — see Direction.

Direction

The AWS-LC consolidation and in-tree TCP TLS / QUIC work below are done. What remains:

• DTLS: expose sessions through the same generic Endpoint contract TCP TLS uses, so protocol crates can dial/listen over DTLS without bespoke wiring — the handshake/record engine and version negotiation are already shipped; only this integration layer is outstanding.
• Primitives: one trust/identity surface for X.509 (TLS/mTLS), DKIM/ARC signing, DNSSEC, and future host-key models (SSH).
• Auth: optional GSSAPI/Kerberos SASL on hopf-auth, once the native foundation is settled — KDC work stays on StorageExecutor.

Other transports (SSH/SFTP, DNSCrypt, CoAP/OSCORE) share the primitive layer but bring their own framing and key exchange — they are not TLS handshakes.

Status table: Conformance → Security substrate.

Dependencies

Allowed today (hard)

Sibling parsers (when features need them)

Explicit non-dependencies

Async app runtimes (Tokio et al.), Hyper, Axum, Tower, serde as architecture, the quinn/quinn-proto crates (fully replaced by hopf-quic's in-tree RFC 9000 transport), servlet/JSP/Java EE APIs, reflective XML DI, TOML/YAML as primary composition formats.

Non-goals

• Replacing AWS-LC with multiple competing pure-Rust crypto stacks for TLS, DTLS, and QUIC.
• Servlet container or Java EE APIs (Gumdrop had these; Hopf does not).

Further reading

• Services · Clients · Runtime options
• HTTP overview · QUIC / H3 · TLS · AMQP
• Auth · Access control · Telemetry

GitHub