Skip to main content

​
Motivation

A mailbox has several identities. The visible address can change, a message can carry several Gmail labels, and a Graph message can move between folders. If a connector treats the label or filename as the identity, ordinary mailbox changes can duplicate sources, lose attachments or repeat a paid classification. Gmail and Microsoft Mail are separate acquisition packages. They delegate indexing and triage to the same AskMyDocs host that already processes IMAP. This separation keeps API transport and OAuth inside the provider package while profiles, configuration assistance, immutable originals, Decisions usage and versioned indexing remain in one email module.

​
Install the released packages

Since 2026-10-16, AskMyDocs includes both published v1.0.0 packages as production dependencies with ^1.0 constraints. The committed lockfile fixes the installed versions; the production manifest declares their VCS release sources: Install the dependencies from the existing manifest and lockfile, then refresh Laravel package discovery and apply package migrations:
The normal Composer registry and Laravel service providers expose both sources, their configuration and their migrations. Connecting a mailbox remains a separate OAuth action in the authenticated Connectors page.

​
Configure OAuth in the server

Use dedicated Google and Microsoft Web applications, with these server variables:
The URLs above are the existing HTTPS Herd routes. Register the exact callback accepted by each provider; deployments use their own actual domain. Google may reject a .test Web callback, in which case use an approved HTTPS callback domain/proxy routed to the same host callback while retaining Herd’s APP_URL. After changing environment variables with cached configuration, run php artisan config:clear before reconnecting. Enable Gmail API and configure the Google consent audience and testing users. The Gmail grant requests https://www.googleapis.com/auth/gmail.readonly and openid, with offline access. Microsoft requests delegated Mail.Read, User.Read, offline_access and openid, plus Mail.Read.Shared when a shared mailbox is selected. Match MICROSOFT_MAIL_AUTHORITY (common, organizations, consumers or a directory GUID) to the Entra application’s supported account types. A shared mailbox requires its persistent target GUID and delegated access. Begin OAuth with the intended AskMyDocs tenant active, complete provider consent, then verify native health and folder/label discovery before selecting sources and syncing. Successful package installation does not certify a real mailbox grant.

​
Theory: identity, content and delivery are different state

The API parent key contains tenant, installation, provider, stable mailbox ID and native message ID. Labels, folder names and sender address do not participate. Two installations of the same mailbox remain isolated; moving a message within the same mailbox preserves its logical source path. IMAP retains its existing mailbox/UIDVALIDITY/UID identity and imap_doc_key. A SHA256 body hash identifies the final acquired Markdown, including the host’s required source redaction, before adding the automatic card. A separate revision hash includes the semantic attachment manifest. Receipt transitions such as pending to saved are delivery state and do not change the content revision. This distinction permits replay after a crash without reopening completed files or paying for an unchanged decision.

​
Design

The prepared listener runs inline through Laravel’s ordinary event dispatcher. It is not a queued listener. This is essential: asynchronous preregistration could race a fast attachment receipt or allow a cursor commit before the manifest exists. A listener exception stops the provider batch. An installation with triage off continues ordinary ingestion and does not stage a decision. The body waits for its own durable receipt and each accepted attachment’s terminal receipt. Explicit exclusion or extraction/OCR refusal is visible coverage. An unexpected fetch, write or dispatch failure prevents successful batch completion. The Flow persistence boundary retains the native API connector key for final revision, provenance and ACL checks. Attachments inherit the current parent scope/grants, retain existing denials and manual tags, and cannot add access the parent does not allow. They use the existing normal ingestion path independently of body classification. The download worker never executes AI analysis.

​
Data model and contract

The common v1 metadata on body and attachments includes: Attachments add email_attachment_id and email_attachment_slot, plus the existing attachment_filename and attachment_of_message_id linkage. Two files named invoice.pdf remain distinct because receipts match ID, slot and hash. Provider fields remain source-aware converter_hints; generated search tags use the existing _derived propagation. An OAuth identity does not increase source trust: these sources default to untrusted external provenance. The key uses SHA256 over JSON [tenantId, installationId, connectorKey, mailboxId, nativeMessageId], with unescaped Unicode/slashes and throwing JSON errors. All entries are strings except the integer installation ID. The revision hashes JSON of [emailBodyHash, manifestTuples]; a tuple is [slot, id, name, mime, hash, excluded ? reason : null].

​
Decision rationale

Connector extraction and ADR 0009 place acquisition in Composer packages and dispatch/redaction/audit in the public ingestion bridge. The prepared event extends that boundary without patching the installed connector-base or coupling a package to App classes. Schema-driven settings and ADR 0022 keep editing and discovery common to credential and OAuth installations. Gmail and Microsoft Mail advertise their native ID selectors and safe filters. The shared UI uses their schema; it does not translate IDs into unstable folder slugs. One host module also preserves the established cost behavior: changing read state or membership alone does not require a new paid decision when the body, semantic manifest and profile are unchanged. Disabling triage suspends queued AI and pauses campaigns while downloads continue. Index retries reuse the saved card.

​
Worked example: a bounded retrospective campaign

An administrator opens Connectors → Edit → Triage email, saves reviewed rules and explicitly enables the profile after readiness checks. For a Gmail label Label_42, the campaign preview request uses the existing route and shape:
The legacy field name mailbox denotes a folder selector: an IMAP path, Gmail label ID or Graph folder ID obtained through discovery. A Gmail message with several labels is still one parent document. Preview returns the exact selected count, availability checked on at most 100 originals and an explicit unchecked count. Estimates use recent observed decision costs or configured rates; absent evidence yields unavailable estimates. Starting is a separate request carrying the previewed profile_id. The campaign pins its profile revision and maximum selected document ID. New mail cannot extend the campaign indefinitely. Pause stops new work at job boundaries; resume retains cursor and snapshots. Missing originals are reported per item, and recovery never reconstructs a body from retrieval chunks or refetches mail.

​
Operations and concrete limits

Gmail uses delegated gmail.readonly, offline OAuth and a verified OIDC subject for its stable mailbox. Microsoft Mail uses delegated Graph read permission and immutable message IDs; shared-mailbox access requires explicit target and consent. Neither connector mutates upstream mail to perform sync or triage. Sync settings and the Folders action are available on OAuth accounts when the package advertises the corresponding schema/discovery. Folder/label discovery and package health do not ingest messages, change committed cursors or run AI. The existing test-fetch sample download and email-triage:pilot command are IMAP-only. API email sampling has no neutral host capability yet; an explicitly requested triage preview of an already imported original remains available and can incur Decisions usage. Run the dedicated worker separately from downloads:
The scheduler’s email-triage:pump recovers committed outbox work and advances campaigns in finite batches. Immutable .artifacts/email-triage snapshots consume tenant disk space and intentionally survive retries and retrospective analysis. Google consent/restricted-scope verification and Microsoft organizational policies remain deployment checks. Synthetic fixture tests prove contract behavior; they do not certify a real OAuth grant or a live mailbox.

​
Optional local package development without production paths

For unreleased provider changes, test integration in an isolated worktree with a separate dependency directory. The standard released-package workflow above remains the deployment path. The host test helper generates an alternate Composer manifest from explicitly supplied local repositories:
Run package discovery explicitly after --no-scripts so an existing Laravel manifest cache includes both providers, their configuration and migrations. After changing OAuth environment variables with cached configuration, run php artisan config:clear in this local worktree before testing the connection. The machine-local manifest and lock are ignored; the production composer.json and composer.lock keep their existing dependency configuration. The temporary 0.1.0 aliases identify local test copies and do not claim a remote release. Composer mirrors each package into the isolated vendor directory, so repeat the targeted update after provider changes. EmailApiPackageDiscovery supplements the base registry’s production-lockfile discovery using actually installed Composer metadata. It retains the registry’s FQCN, interface and duplicate-key checks; an absent package does not appear in the connector roster. Normal Laravel package discovery registers the providers in the application. Testbench smoke tests register them explicitly because they bypass the application’s package-discovery cache. The smoke exercises both providers’ native discovery, health and full acquisition through the real host bridge, with triage both on and off. HTTP, Storage and queues use synthetic fixtures; no mailbox is read and no AI is paid. The production integration suite requires both packages and fails explicitly if either is missing. This check is distinct from provider consent/live-mailbox verification.