Gumdrop

Gumdrop Mailbox API

← Back to Main Page

Gumdrop provides an abstraction layer for mailbox access that decouples mail servers from storage implementation details. The IMAP, POP3, and SMTP servers can all access mailboxes through this API, allowing the storage backend to be changed or extended without modifying the protocol implementations.

Contents

The Abstraction Layer

The mailbox abstraction provides several important benefits:

The abstraction is organised into three tiers:

Mailbox Name Encoding

Mailbox names may contain Unicode characters (e.g., "Données", "日本語") and special characters that are problematic on certain filesystems. Gumdrop uses a modified Quoted-Printable encoding based on RFC 2047 to ensure mailbox names are safely stored on all platforms.

Encoding Format

The MailboxNameCodec encodes characters as =XX hex sequences (where XX are uppercase hex digits) for:

Examples: 

"Données/été"   → "Donn=C3=A9es=2F=C3=A9t=C3=A9"
"Reports:2025"  → "Reports=3A2025"
"日本語"        → "=E6=97=A5=E6=9C=AC=E8=AA=9E"

Filesystem Compatibility

The encoding produces output containing only safe characters: A-Z a-z 0-9 . _ - =. This is compatible with:

Unlike full RFC 2047 which uses the =?charset?Q?...?= wrapper format, this encoding omits the wrapper because the ? character is forbidden in Windows filenames. The encoding is always UTF-8.

Core Interfaces

MailboxFactory

The factory interface creates store instances. Each authenticated session receives its own store to ensure thread safety: 

public interface MailboxFactory {
    MailboxStore createStore();
}

Factories are configured in gumdroprc and referenced by servers that require mailbox access.

MailboxStore

A store represents a user's complete mail hierarchy. After authentication, the server opens the store for the user: 

MailboxStore store = factory.createStore();
store.open(username);

The store provides:

Mailbox

A mailbox is a single folder containing messages. For POP3, only INBOX is used; IMAP accesses the full hierarchy.

Message operations:

IMAP-specific operations:

MessageDescriptor

Message metadata without loading content:

Configuration

Mailbox factories are configured in gumdroprc and referenced by servers: 

<!-- Mbox mailbox factory -->
<mailbox-factory id="mboxFactory" 
                 class="org.bluezoo.gumdrop.mailbox.mbox.MboxMailboxFactory">
    <property name="base-directory">/var/mail</property>
    <property name="extension">.mbox</property>
</mailbox-factory>

<!-- Maildir mailbox factory -->
<mailbox-factory id="maildirFactory"
                 class="org.bluezoo.gumdrop.mailbox.maildir.MaildirMailboxFactory">
    <property name="base-directory">/var/mail</property>
</mailbox-factory>

<!-- IMAP service using Maildir -->
<service class="org.bluezoo.gumdrop.imap.DefaultIMAPService">
    <property name="mailbox-factory" ref="#maildirFactory"/>
    <property name="realm" ref="#mailRealm"/>
    <listener class="org.bluezoo.gumdrop.imap.IMAPListener" port="143"/>
    <listener class="org.bluezoo.gumdrop.imap.IMAPListener" port="993"
            secure="true"/>
</service>

<!-- POP3 service using mbox -->
<service class="org.bluezoo.gumdrop.pop3.DefaultPOP3Service">
    <property name="mailbox-factory" ref="#mboxFactory"/>
    <property name="realm" ref="#mailRealm"/>
    <listener class="org.bluezoo.gumdrop.pop3.POP3Listener" port="110"/>
    <listener class="org.bluezoo.gumdrop.pop3.POP3Listener" port="995"
            secure="true"/>
</service>

SMTP services can also reference a mailbox factory for local delivery: 

<service class="org.bluezoo.gumdrop.smtp.LocalDeliveryService">
    <property name="mailbox-factory" ref="#mboxFactory"/>
    <property name="local-domain">example.com</property>
    <listener class="org.bluezoo.gumdrop.smtp.SMTPListener" port="25"/>
</service>

The LocalDeliveryService uses the mailbox factory to deliver messages to local recipients.

Mbox Implementation

The mbox format stores all messages in a single file, separated by "From " envelope lines. This format originated in Unix systems and remains widely used due to its simplicity.

Format

Each message begins with a "From " line containing the sender and timestamp: 

From sender@example.com Mon Jan  1 00:00:00 2025
[RFC 822 headers]

[message body]

From another@example.com Tue Jan  2 00:00:00 2025
[next message...]

Lines in the message body beginning with "From " are escaped by prepending ">" to prevent confusion with message boundaries.

Implementation Details

Configuration

<mailbox-factory id="mbox" 
                 class="org.bluezoo.gumdrop.mailbox.mbox.MboxMailboxFactory">
    <property name="base-directory">/var/mail</property>
    <property name="extension">.mbox</property>
</mailbox-factory>

User mailboxes are located at {base-directory}/{username}/INBOX{extension}. Subfolders follow the same pattern with folder names as subdirectories.

Strengths

Limitations

Use Cases

Mbox is suitable for:

Maildir Implementation

Maildir stores each message as a separate file within a directory structure. Developed by Daniel J. Bernstein for qmail, Maildir is designed for reliable delivery and concurrent access.

Directory Structure

Each mailbox consists of three subdirectories:

Subfolders follow the Maildir++ convention, represented as directories starting with a dot (e.g., .Sent, .Drafts).

Filename Format

Message metadata is encoded in the filename: 

{timestamp}.{unique}.{hostname},S={size}:2,{flags}

Example: 1704067200.12345.mail.example.com,S=4096:2,S

Flag characters: S=Seen, R=Replied, F=Flagged, T=Trashed, D=Draft. This allows flag changes without modifying file contents—only a rename is required.

Implementation Details

Configuration

<mailbox-factory id="maildir"
                 class="org.bluezoo.gumdrop.mailbox.maildir.MaildirMailboxFactory">
    <property name="base-directory">/var/mail</property>
</mailbox-factory>

User mailboxes are located at {base-directory}/{username}/ with the standard cur/, new/, tmp/ subdirectories.

Strengths

Limitations

Use Cases

Maildir is recommended for:

Search Indexing

Both mbox and Maildir implementations include a search indexing facility that dramatically accelerates IMAP SEARCH operations. Without indexing, each search requires parsing every message in the mailbox—a prohibitively slow operation for large mailboxes. The index pre-extracts searchable metadata, enabling most searches to complete without touching the underlying message files.

What the Index Stores

The search index (.gidx file) stores the following for each message:

All string values are stored in lowercase for case-insensitive searching. Structured data is extracted using the RFC 5322 message parser—for example, EmailAddress objects provide just the address without display names or angle brackets.

What the Index Does NOT Store

To keep the index compact, the following are not indexed:

When a search requires body content (e.g., SEARCH BODY "invoice"), the implementation falls back to the default parsing-based search for those messages. However, the index can still accelerate compound searches by first narrowing candidates using indexed criteria.

Sub-Indexes for Fast Lookups

Beyond the primary entry list, the index maintains auxiliary data structures for O(1) and O(log n) lookups:

Index File Format

The index is stored as a binary file with the .gidx extension:

The file format includes: 

HEADER (32 bytes):
  magic: 4 bytes ("GIDX")
  version: 2 bytes
  flags: 2 bytes (reserved)
  uidValidity: 8 bytes
  uidNext: 8 bytes
  entryCount: 4 bytes
  headerChecksum: 4 bytes (CRC32)

ENTRIES SECTION:
  For each message: [fixed header + property descriptors + variable data]
  sectionChecksum: 4 bytes (CRC32)

Each entry uses "property descriptors" (offset + length pairs) for variable-length strings, allowing fast random access without parsing the entire entry. All strings are UTF-8 encoded.

Index Lifecycle

The index is managed automatically:

Corruption Detection

The index includes multiple layers of corruption detection:

If corruption is detected, the index is discarded and rebuilt from the mailbox contents. No data is lost—the index is purely a cache of derived information.

Performance Characteristics

Search TypeWithout IndexWith Index
Flag search (e.g., UNSEEN)O(n) message parsesO(1) BitSet lookup
Date range (SINCE/BEFORE)O(n) message parsesO(log n) TreeMap
Size (LARGER/SMALLER)O(n) stat callsO(log n) TreeMap
Address (FROM/TO/CC)O(n) header parsesO(n) scan, no parsing
SubjectO(n) header parsesO(n) scan, no parsing
Body text (BODY/TEXT)O(n) full parsesO(n) full parses*

* Body searches cannot be accelerated by indexing and always require parsing. However, compound searches like UNSEEN BODY "invoice" first filter by flag, reducing the messages that need parsing.

Design Considerations

The index is designed with these principles:

Future Enhancements

Potential improvements noted for future development:

Custom Implementations

The mailbox API can be implemented for alternative storage backends. Potential implementations include:

Implementation Guidelines

When implementing a custom backend:

Example Skeleton

public class DatabaseMailboxFactory implements MailboxFactory {
    
    private final DataSource dataSource;
    
    public DatabaseMailboxFactory(DataSource dataSource) {
        this.dataSource = dataSource;
    }
    
    @Override
    public MailboxStore createStore() {
        return new DatabaseMailboxStore(dataSource);
    }
}

public class DatabaseMailboxStore implements MailboxStore {
    
    private final DataSource dataSource;
    private Connection connection;
    private String username;
    
    @Override
    public void open(String username) throws IOException {
        this.username = username;
        try {
            this.connection = dataSource.getConnection();
        } catch (SQLException e) {
            throw new IOException("Database connection failed", e);
        }
    }
    
    @Override
    public Mailbox openMailbox(String name, boolean readOnly) throws IOException {
        return new DatabaseMailbox(connection, username, name, readOnly);
    }
    
    // ... implement remaining methods
}

← Back to Main Page | SMTP Server & Client | IMAP Server | POP3 Server | DNS Server | Telemetry

Gumdrop Mailbox API