build.allanninal.dev

Part 7 of 7 · Form intake router series ~8 min read

Engineering reference: the form intake router architecture

Same system, drawn for engineers. Region, service names, resource identifiers, Bedrock model IDs, Lambda inventory, IAM scopes, the Function URL setup, the SQS and dead-letter queue config, the DynamoDB schemas, and the idempotency design. Read alongside the previous six posts; this one’s the build sheet.

Region and account shape

Default region: ap-southeast-1 (Singapore). Lambda Function URLs, SQS, SES, Bedrock cross-Region inference, and DynamoDB are all in good shape there. A second region for multi-region resilience isn’t worth the extra setup work at SMB volume — the failure mode for an SMB is a downstream tool being briefly unreachable, which the queue already handles, not a regional outage. One AWS account dedicated to the router (separate from your other workloads) keeps the IAM blast radius small and lets a single AWS Budgets alarm cover the whole system.

Topology

AWS topology of the form intake router A topology diagram with three bands stacked vertically inside one AWS account boundary. Top band: ingress. Three boxes show the three capture lanes — the intake Function URL on the intake-door Lambda that receives the browser POST, an SES inbound rule set with action S3 PUT to s3://fir-raw-mime/ plus the email-adapter Lambda for the email fallback, and the intake-door's idempotent write keyed by submission_key that absorbs duplicate re-posts. All converge on the saved record. Middle band: scheduled and per-submission processing. The intake-door Lambda writes the raw payload to s3://fir-raw/ and a record to the DynamoDB submissions table, then returns the acknowledgment; it then invokes the checker Lambda, which validates required fields, runs the spam rules from s3://fir-rules-source/, optionally calls Bedrock Haiku 4.5 for a borderline-spam second-opinion or a category guess, looks up the routing rule, and on routed-ready sends fan-out jobs to the SQS deliveries queue. A sheet-sync Lambda triggered every 15 minutes by EventBridge Scheduler mirrors the routing sheet and rules to s3://fir-rules-source/. Bottom band: dispatch and confirmation. The delivery-worker Lambda consumes the SQS deliveries queue with batch size one and partial-batch responses; per job it sends the team email or customer reply via SES outbound, writes to the CRM API or the Sheet via the Sheets API, and records the result to DynamoDB fir-deliveries with a conditional write for idempotency; jobs that exhaust the redrive policy land in the fir-deliveries-dlq dead-letter queue, which raises a CloudWatch alarm. CloudWatch Logs collects from every Lambda at 7-day retention. Across the right edge: a small box labelled AWS Budgets alarm at $15 monthly threshold, posting to SNS topic fir-cost-alarm. A note at the bottom: every submission is saved and acknowledged first — and every delivery is logged to fir-audit. Ingress Function URL · intake AuthType: NONE browser POST → s3://fir-raw/ + submissions record SES inbound rule set fir-inbound-rules action: S3 PUT s3://fir-raw-mime/ trigger: email-adapter Idempotent write key: submission_key conditional PutItem absorbs duplicate re-posts → one lead DynamoDB submissions saved record · status: received Per-submission processing EventBridge Scheduler rate(15 minutes) sheet-sync mirrors routing + rules → s3://fir-rules-source/ Lambda · checker required fields, spam rules, route; Haiku only on borderline + category SQS deliveries queue team_email job crm + sheet jobs customer_reply job (held → no jobs) Dispatch & confirmation Lambda · delivery-worker consumes SQS, SES / CRM / Sheets; retry with backoff, partial-batch responses Dead-letter queue fir-deliveries-dlq past redrive budget → CloudWatch alarm, operator replay DynamoDB fir-deliveries conditional write per delivery type de-dup guard; also writes fir-audit Every submission is saved and acknowledged first — and every delivery is logged to fir-audit.
Fig 7. AWS topology, in three bands: ingress (three capture lanes into the saved record), per-submission processing (the checker emitting fan-out jobs), dispatch and confirmation (delivery workers with retries, a dead-letter queue, and an idempotent log). Every step is queue- or event-driven; the only synchronous hop is intake-door → acknowledgment.

Lambda functions

All Lambdas use the arm64 architecture, the smallest memory size that meets latency targets (typically 256 MB), Python 3.14 runtime, and CloudWatch Logs at 7-day retention. Each function has its own least-privilege IAM role. None run inside a VPC.

  • intake-door — Lambda Function URL, AuthType: NONE, CORS locked to your site origins. On each POST it parses the body, writes the raw payload to s3://fir-raw/<submission_key>, performs a conditional PutItem on the submissions table keyed by submission_key (so a duplicate re-post is a no-op that returns the same 200), and returns the acknowledgment. It then invokes checker asynchronously (InvocationType: Event) so the customer round-trip never waits on validation. Memory: 256 MB. Timeout: 10 s.
  • email-adapter — S3 PUT trigger on s3://fir-raw-mime/. Parses the inbound MIME from the email-fallback lane, maps the email’s fields to the canonical submission shape with a submission_key derived from the message id, writes to s3://fir-raw/ and the submissions table, and invokes checker. Falls back to a permissive field parser for plain-text bodies. Memory: 256 MB. Timeout: 30 s.
  • checker — invoked by intake-door / email-adapter. Reads the saved submission and s3://fir-rules-source/rules.json. Validates required fields and formats; runs the deterministic spam stack (honeypot, time-on-page, per-address rate limit via the fir-rate-limit table, banned patterns). Calls Bedrock Haiku 4.5 only on a borderline spam score or an unknown category. Resolves the routing rule from s3://fir-rules-source/routing.json. On routed-ready, emits one SQS message per delivery to the fir-deliveries queue; on held, updates status and stops. Memory: 512 MB. Timeout: 30 s.
  • delivery-worker — SQS event source on fir-deliveries with BatchSize: 1 and ReportBatchItemFailures enabled (partial-batch responses). Per job, switches on delivery_type: team_email / customer_reply via SES SendEmail; crm via the CRM API; sheet via the Sheets API. Before sending it checks fir-deliveries for an existing row keyed by (submission_key, delivery_type); if present it acks the message without re-sending. On success it writes that row with a conditional put and an fir-audit entry. On failure it raises, letting SQS redrive per the queue’s maxReceiveCount. Memory: 256 MB. Timeout: 30 s.
  • sheet-sync — EventBridge Scheduler target, every 15 minutes. Uses the Google Sheets API (service-account credentials in Secrets Manager under fir/google/sa) to export the routing sheet and the rules tab as JSON and write to s3://fir-rules-source/ only if changed since the last sync. Memory: 256 MB. Timeout: 30 s.
  • dlq-alarm — SQS event source on fir-deliveries-dlq. On any message, marks the affected submission’s delivery as dead-lettered in fir-deliveries, writes fir-audit, and publishes to the fir-cost-alarm SNS topic’s sibling fir-ops-alarm for the on-call admin. Does not delete the DLQ message; an operator replays after the downstream tool recovers. Memory: 256 MB.
  • replay — manual / admin Function URL (IAM-auth). Redrives messages from fir-deliveries-dlq back to fir-deliveries in batches, or re-drives a specific submission from its saved record. Used after an outage is resolved. Memory: 256 MB. Timeout: 60 s.
  • held-review — Function URL behind the internal held-queue UI. Lists held submissions with their held_reason; on release, re-runs checker with the spam stack bypassed for that one submission so a false-positive lead routes normally. Memory: 256 MB.

Storage and queues

  • DynamoDB · submissions — one row per submission. PK submission_key; attributes: form_id, category, fields, status (received/checking/held/routed-ready), held_reason, received_at. On-demand. TTL on a copy of the raw fields at 90 days (the S3 payload is the long-term store).
  • DynamoDB · fir-deliveries — one row per completed or dead-lettered delivery. PK (submission_key, delivery_type); attributes: result (sent/dead-lettered/duplicate-skipped), target, sent_at, attempts. On-demand. The conditional write on this table is the de-dup guard.
  • DynamoDB · fir-audit — one row per action of any kind. PK (submission_key, ts); attributes: action, delivery_type, result, actor. On-demand. No TTL — long-term audit trail.
  • DynamoDB · fir-rate-limit — sliding-window counters for the spam rate check. PK source_ip; attribute count with a short TTL so windows self-expire. On-demand.
  • SQS · fir-deliveries — standard queue, visibility timeout 6× the worker timeout, maxReceiveCount: 6 in the redrive policy to fir-deliveries-dlq. Backoff is the natural product of visibility-timeout redelivery.
  • SQS · fir-deliveries-dlq — dead-letter queue, 14-day retention, triggers dlq-alarm.
  • S3 · fir-raw — canonical raw payload per submission. Versioning enabled. Lifecycle to Glacier at 90 days; expiry at 7 years.
  • S3 · fir-raw-mime — raw inbound MIME from the email-fallback lane. Lifecycle to Glacier at 30 days; expiry at 7 years.
  • S3 · fir-rules-source — mirrored routing table and rules as JSON. Versioning enabled so a bad sheet edit rolls back in one click.

Bedrock

  • Foundation model. anthropic.claude-haiku-4-5-20251001-v1:0 via the Global cross-Region inference profile global.anthropic.claude-haiku-4-5-20251001-v1:0. Two callsites, both in checker: the borderline-spam second-opinion and the category guess for generic contact forms. Heavier reasoning (Sonnet 4.6) is not used — neither call justifies it.
  • Embeddings. Not used. Routing is a deterministic table lookup keyed by form_id and category; spam is rule-driven. No Knowledge Base, no S3 Vectors.
  • Quotas. Default account quotas are more than enough at SMB volume. The hot path doesn’t call Bedrock; only the minority of submissions that are borderline or uncategorized do.

Function URL and ingress

  • The intake-door Function URL is AuthType: NONE (public, by design — it’s a form endpoint) with CORS AllowOrigins pinned to your site domains and AllowMethods: POST. The function rejects bodies over a small size cap and requires the form_id and submission_key fields.
  • Abuse control is layered, not at the edge: the spam stack’s rate limit (the fir-rate-limit table) plus the honeypot and time-on-page checks. A reserved-concurrency cap on intake-door bounds a flood’s blast radius.
  • For the email-fallback lane, set the MX record on a dedicated subdomain (e.g. forms.your-company.com) to inbound-smtp.ap-southeast-1.amazonaws.com; the SES inbound rule set fir-inbound-rules does spam-scan → S3 PUT to s3://fir-raw-mime/<message-id> → stop.

IAM (least privilege per Lambda)

Each Lambda has its own role with policies scoped to exact ARNs. Sketch:

  • intake-door role: s3:PutObject on fir-raw; dynamodb:PutItem (conditional) on submissions; lambda:InvokeFunction on checker. No bedrock:*, no SES, no SQS.
  • checker role: s3:GetObject on fir-raw and fir-rules-source; dynamodb:GetItem + UpdateItem on submissions and fir-rate-limit; sqs:SendMessage on fir-deliveries; bedrock:InvokeModel on the Haiku ARN.
  • delivery-worker role: sqs:ReceiveMessage + DeleteMessage on fir-deliveries; ses:SendEmail from the verified sender identity; dynamodb:PutItem (conditional) on fir-deliveries and PutItem on fir-audit; secretsmanager:GetSecretValue on the CRM and Sheets secrets; outbound network access to the CRM API host and sheets.googleapis.com.
  • dlq-alarm / replay roles: sqs:* scoped to fir-deliveries-dlq and (replay only) sqs:SendMessage on fir-deliveries; sns:Publish on fir-ops-alarm; dynamodb:UpdateItem on fir-deliveries and fir-audit.
  • sheet-sync role: secretsmanager:GetSecretValue on fir/google/sa; s3:PutObject on fir-rules-source; outbound network to sheets.googleapis.com.

Idempotency and exactly-once-effect

The system is “at-least-once” end to end and leans on two conditional writes to get exactly-once effect. At ingress, the submissions PutItem is conditional on attribute_not_exists(submission_key) — a duplicate re-post finds the row present, skips creation, and returns the original acknowledgment. At egress, the fir-deliveries PutItem is conditional on attribute_not_exists((submission_key, delivery_type)) — a redelivered SQS message whose work already completed is detected and acked without a second send. Both keys are derived deterministically (the browser-generated submission_key, and the fixed delivery_type enum), so retries always collide with their own prior success rather than creating a new effect.

SES outbound

  • Verify a sender identity at forms@your-company.com with DKIM and SPF on the parent domain; request production access out of sandbox.
  • Two template families in the rules doc: the team-notification template (full submission, link to the held-review UI) and the per-form customer auto-reply template. Both rendered by delivery-worker; no model in the rendering path.
  • Set the SES configuration set to publish bounce and complaint events to an SNS topic so a customer-reply address that hard-bounces is flagged rather than silently retried.

Observability and cost gates

  • CloudWatch Logs: all Lambdas, 7-day retention, structured JSON. Subscription filter on "error" + "throttle" + "timeout" to a CloudWatch metric for alerting.
  • Alarms: fir-deliveries-dlq depth > 0 (a delivery type is failing); intake-door 5xx rate > 1% in 5 min (the door is the one piece that must always answer); checker error rate > 1% in 24h.
  • X-Ray: off by default. Not worth the cost at SMB volume.
  • AWS Budgets: $15/month threshold, alarm at 80% and 100%, posts to SNS topic fir-cost-alarm subscribed to the on-call admin’s email and Slack.

Config and secrets

Service-account credentials for the Sheets API live in Secrets Manager under fir/google/sa. CRM API tokens live under fir/crm/token. The SES sender identity lives in IAM and the verified-domain config. The allowed CORS origins, the spam thresholds, the rate-limit window, the category list, and the admin fallback address all live in Parameter Store under /fir/config/. Lambdas fetch config on cold start and cache for the lifetime of the execution environment.

Deploy

GitHub Actions with OIDC into a deploy role (no long-lived keys) running AWS SAM. The opinionated bits: deploy the SES rule set as a separate stack (rule-set changes affect mail flow), turn on S3 versioning for fir-raw and fir-rules-source so a bad payload or sheet edit can be rolled back, and keep the SQS redrive policy and DLQ in the same stack as the worker so maxReceiveCount and the queue are versioned together. Total deployable surface: around eight Lambdas, four DynamoDB tables, two SQS queues (main + DLQ), three S3 buckets, one EventBridge Scheduler rule, one SES rule set, and one Budgets alarm.

That’s the full system. Six narrative posts and this engineering reference. If you want to talk about adapting it for your business, see Work with me.

All posts