build.allanninal.dev

Part 7 of 7 · Newsletter composer series ~8 min read

Engineering reference: the newsletter composer architecture

Same system, drawn for engineers. Region, service names, resource identifiers, Bedrock model IDs, Lambda inventory, IAM scopes, the SES inbound rule set, EventBridge Scheduler config, the DynamoDB schemas, and the Slack interactive flow. Read alongside the previous six posts; this one’s the build sheet.

Region and account shape

Default region: ap-southeast-1 (Singapore). SES inbound and outbound, Bedrock Global cross-Region inference, and EventBridge Scheduler are all in good shape there. A second region for multi-region resilience isn’t worth the extra setup at SMB volume — the failure mode for an SMB is one weekly issue going out a day late, not a regional outage. One AWS account dedicated to the composer (separate from your other workloads) keeps the IAM blast radius small and lets a single AWS Budgets alarm cover the whole system.

Topology

AWS topology of the newsletter composer A topology diagram with three regions stacked vertically inside one AWS account boundary. Top region: ingress. Three boxes show the three intake lanes — a Drive sheet sync via the drive-sync Lambda triggered every 15 minutes by EventBridge Scheduler that mirrors the item-pool CSV to s3://nc-items-source/, a feed-sync Lambda triggered hourly by EventBridge Scheduler that fetches the blog's RSS feed and proposes a row per new post for Slack approval, and an SES inbound rule set with action S3 PUT to s3://nc-raw-mime/ plus the parser Lambda intake-ses-parser that calls Bedrock Haiku 4.5 to tidy a forwarded update into a proposed item for Slack approval. Middle region: scheduled processing. The composer Lambda is triggered weekly by EventBridge Scheduler the morning before the send day; it reads s3://nc-items-source/items.csv, counts fresh items against the threshold, loads the voice and rules docs, calls Bedrock Sonnet 4.6 once to draft the issue grounded in the items, runs a self-check, stores the draft to S3, and emits one event to the EventBridge default bus per run that needs review: nc.ready, or nc.skipped when there are too few items. Bottom region: review and send. The sender Lambda is triggered by an EventBridge rule on nc.ready; it resolves the reviewer, checks quiet hours, runs the grounding check, and posts the review message to Slack via chat.postMessage with Approve, Edit, and Skip buttons or sends it via SES outbound, writing a row to DynamoDB nc-issues. Slack interactive button clicks land on a Function URL Lambda action-handler that updates nc-issues and nc-audit; on approve it starts a cooling-off one-off in EventBridge Scheduler, then the send Lambda mails the issue to the subscriber list via SES outbound and marks the items used in nc-items-state. CloudWatch Logs collects from every Lambda at 7-day retention. Across the right edge: a small box labelled AWS Budgets alarm at $15 monthly threshold, posting to SNS topic nc-cost-alarm. A note at the bottom: every issue is drafted only from gathered items — and nothing sends until the owner approves. Ingress Lambda · drive-sync every 15 min Sheets API → s3://nc-items-source/ items.csv Lambda · feed-sync hourly poll fetch blog RSS, new post → proposal → Slack approval SES inbound rule set nc-inbound-rules action: S3 PUT s3://nc-raw-mime/ trigger: intake-ses-parser Drive item-pool sheet canonical store · mirrored to S3 Scheduled processing EventBridge Scheduler cron(0 7 ? * THU *) in TZ_NAME target: composer Lambda + deferred one-offs Lambda · composer reads CSV from S3 + voice.txt + rules.txt Sonnet 4.6 drafts, picks one of four moves EventBridge default bus nc.ready nc.skipped (redraft → in-process) draft stored in S3 Review & send Lambda · sender resolves reviewer, quiet hours, grounding; Slack postMessage or SES outbound review Slack interactive DM with [Approve] [Edit] [Skip] button clicks → Function URL Lambda · action-handler writes nc-issues, nc-audit; on approve cooling-off → send to list via SES Every issue is drafted only from gathered items — nothing sends without sign-off.
Fig 7. AWS topology, in three regions of the diagram: ingress (three lanes into the item pool), scheduled processing (the weekly composer run drafting and emitting events), review and send (the draft is reviewed and only an approved issue is mailed). Every Lambda is event- or schedule-driven; nothing is synchronous-chained.

Lambda functions

All Lambdas use the arm64 architecture, the smallest memory size that meets latency targets (typically 256 MB), Python 3.14 runtime, and CloudWatch Logs at 7-day retention. Each function has its own least-privilege IAM role. None run inside a VPC.

  • drive-sync — EventBridge Scheduler target, fires every 15 minutes. Uses the Google Drive API + Sheets API (service-account credentials in Secrets Manager under nc/drive/sa) to export the item-pool sheet as CSV and write to s3://nc-items-source/items.csv only if the sheet has changed since the last sync. Same pattern syncs the voice and rules docs to s3://nc-rules-source/. Memory: 256 MB. Timeout: 30 s.
  • feed-sync — EventBridge Scheduler target, hourly. Fetches the configured blog RSS/Atom feed, diffs entry ids against the nc-feed-seen DynamoDB table, and for each new entry creates a Slack interactive proposal (title, link, summary). For lower latency you can switch to a WebSub/PubSubHubbub subscription that pushes to a Function URL instead of polling, at the cost of managing the subscription lease. Memory: 256 MB. Timeout: 30 s.
  • intake-ses-parser — S3 PUT trigger on s3://nc-raw-mime/. Parses MIME, extracts the text body (and any forwarded-quote text), and calls Bedrock Haiku 4.5 (anthropic.claude-haiku-4-5-20251001-v1:0 via global.anthropic.claude-haiku-4-5-20251001-v1:0) to tidy the update into a proposed item (title, one-line note, category, link). Posts the proposal to Slack via the bot token with Approve/Edit/Discard buttons. Memory: 512 MB. Timeout: 30 s. No Sonnet calls.
  • composer — EventBridge Scheduler target, weekly the morning before the send day (the schedule runs in TZ_NAME set to the SMB’s timezone, e.g. Asia/Singapore). Reads s3://nc-items-source/items.csv, the voice and rules docs, and nc-items-state to determine fresh items. If the fresh count is below threshold, emits nc.skipped. Otherwise calls Bedrock Sonnet 4.6 (anthropic.claude-sonnet-4-6-20250115-v1:0 via the Global cross-Region profile) once to draft the issue grounded in the items, runs the self-check, stores the draft to s3://nc-drafts/<issue-id>.json, and emits nc.ready. Memory: 1024 MB. Timeout: 120 s.
  • sender — EventBridge rule on nc.ready. Resolves the reviewer, checks quiet hours, runs the grounding check (attaches the source item behind each paragraph, flags any unsourced line), formats the review message from the draft, and ships via Slack chat.postMessage (nc/slack/bot-token in Secrets Manager) or SES SendRawEmail. On a quiet-hours defer, creates a one-off EventBridge Scheduler rule that re-invokes sender at the next business minute. Writes a row to nc-issues marking the issue awaiting review. Memory: 256 MB. Timeout: 30 s. No Bedrock calls.
  • action-handler — Lambda Function URL, public with AuthType: NONE; verifies a Slack signature on the request body. Triggered by Slack interactive button clicks (Approve/Edit/Skip, plus Pull-it-back) and by email-link clicks. Writes to nc-issues and nc-audit. On edit, copies the draft to a Google Doc and replies with the link; on re-submit, re-runs the grounding check. On approve, creates a cooling-off one-off EventBridge Scheduler rule (default 30 min) targeting send-issue. Memory: 256 MB. Timeout: 15 s.
  • send-issue — EventBridge Scheduler target, fired once by the cooling-off rule. Renders the approved issue, sends it to the subscriber list via SES SendBulkEmail in batches that respect the account’s send rate, marks every used item as used in nc-items-state, and writes a sent row to nc-audit with the recipient count and a snapshot of exactly what went out. Memory: 512 MB. Timeout: 120 s.
  • summary — EventBridge Scheduler target, monthly on the first Monday at 9am. Reads the past month’s nc-issues and nc-audit (issues sent, skips, open rate if a webhook feeds it back); calls Bedrock Haiku 4.5 to write a short narrative; emails it via SES to the configured stakeholder list. Memory: 512 MB.

Storage

  • DynamoDB · nc-items-state — one row per item. PK item_id; attributes: used_in_issue, used_date, category, added_by. On-demand. Drives the “fresh” count.
  • DynamoDB · nc-issues — one row per weekly run. PK issue_id; attributes: move (skip/draft/redraft/ready/approved/sent/skipped-by-owner), item_ids, draft_s3_key, reviewer, state. On-demand.
  • DynamoDB · nc-audit — one row per write action of any kind. PK (issue_id, ts); attributes: action (approve/edit/skip/pull-back/sent), by_user, before, after. On-demand. No TTL — this is the long-term audit trail.
  • DynamoDB · nc-feed-seen — one row per blog feed entry already seen. PK entry_id; attribute: first_seen. On-demand. TTL at 180 days.
  • S3 · nc-items-source — mirrored CSV from the Drive item-pool sheet. Versioning enabled. Lifecycle to Glacier at 90 days; expiry at 7 years.
  • S3 · nc-rules-source — mirrored voice and rules docs as plain text. Versioning enabled.
  • S3 · nc-drafts — each issue draft as JSON (subject, body, per-paragraph sources). Versioning enabled, so an edited issue keeps its draft history.
  • S3 · nc-raw-mime — raw inbound MIME from forwarded updates. Lifecycle to Glacier at 30 days; expiry at 7 years.

Bedrock

  • Foundation models. anthropic.claude-sonnet-4-6-20250115-v1:0 via the Global cross-Region inference profile global.anthropic.claude-sonnet-4-6-20250115-v1:0 for the one weekly issue draft (the only genuinely hard writing task). anthropic.claude-haiku-4-5-20251001-v1:0 via global.anthropic.claude-haiku-4-5-20251001-v1:0 for the forwarded-update tidy-ups in intake-ses-parser and the monthly summary.
  • Embeddings. Not used. The item pool is a short structured list passed in full to the draft prompt; there’s nothing to retrieve. No Knowledge Base, no S3 Vectors. (If a future version drew on a large archive of past issues for style, Titan Text Embeddings V2 at 1024-dim into S3 Vectors would be the path.)
  • Quotas. Default account quotas are more than enough at SMB volume. The composer fires one Sonnet call per issue; Haiku fires a few times a month.

EventBridge Scheduler config

  • nc-weekly-compose — cron(0 7 ? * THU *) in the SMB’s timezone (Thursday 7am, the morning before a Friday send). Target: composer Lambda.
  • nc-drive-sync — rate(15 minutes). Target: drive-sync Lambda.
  • nc-feed-sync — rate(1 hour). Target: feed-sync Lambda.
  • nc-monthly-summary — cron(0 9 ? * 2#1 *) (first Monday at 9am) in TZ. Target: summary Lambda.
  • One-off rules — created on the fly by action-handler for the cooling-off window, and by sender for quiet-hours defers. Use at(YYYY-MM-DDTHH:MM:SS) expressions with --action-after-completion DELETE so the rule self-cleans.

SES inbound and outbound

  • Set the MX record on a dedicated subdomain (e.g. updates.your-company.com) to inbound-smtp.ap-southeast-1.amazonaws.com.
  • SES inbound rule set nc-inbound-rules: one rule with recipient updates@your-company.com → spam scan → S3 PUT to s3://nc-raw-mime/<message-id> → stop. The S3 PUT triggers intake-ses-parser.
  • SES outbound for the review messages and the issue sends: verify a sender identity at news@your-company.com with DKIM and SPF on the parent domain, plus a custom MAIL FROM and a List-Unsubscribe header on every issue. Out of sandbox by request; a configuration set with event publishing feeds bounces and complaints back to an SNS topic so a hard bounce removes a subscriber.

IAM (least privilege per Lambda)

Each Lambda has its own role with policies scoped to exact ARNs. Sketch:

  • composer role: s3:GetObject on the items, voice, and rules keys; s3:PutObject on nc-drafts; dynamodb:Query + GetItem on nc-items-state; bedrock:InvokeModel on the Sonnet ARN; events:PutEvents on the default bus.
  • sender role: s3:GetObject on nc-drafts; events:CreateSchedule for quiet-hours one-offs; secretsmanager:GetSecretValue on the Slack bot token; ses:SendRawEmail from the verified sender; dynamodb:PutItem on nc-issues; outbound network access to slack.com.
  • action-handler role: dynamodb:PutItem on nc-issues and nc-audit; events:CreateSchedule for the cooling-off one-off; secretsmanager:GetSecretValue on the Sheets and Slack secrets; outbound network access to sheets.googleapis.com and slack.com.
  • send-issue role: s3:GetObject on nc-drafts; ses:SendBulkEmail from the verified sender; dynamodb:UpdateItem on nc-items-state; dynamodb:PutItem on nc-audit; secretsmanager:GetSecretValue on the subscriber-list secret if the list lives outside DynamoDB.
  • intake-ses-parser role: s3:GetObject on nc-raw-mime; bedrock:InvokeModel on the Haiku ARN; secretsmanager:GetSecretValue on the Slack bot token.
  • drive-sync and feed-sync roles: secretsmanager:GetSecretValue on the Google service-account secret; s3:PutObject on the items and rules buckets; dynamodb:*Item on nc-feed-seen (feed-sync only); outbound network to www.googleapis.com and the blog host.

Slack interactive flow

Review messages and item proposals are posted via the chat.postMessage Web API with Block Kit blocks containing the action buttons, because the simpler incoming-webhook surface doesn’t support interactive responses. Button clicks are sent by Slack to the configured Interactivity request URL, which is the action-handler Function URL. action-handler verifies the Slack signing secret on the inbound request, parses the action_id (approve, edit, skip, pull_back, and the proposal actions item_approve, item_edit, item_discard), opens a modal where needed, and processes the response on submit.

The Slack app needs chat:write and im:write, plus the Interactivity URL configured. The bot token lives in Secrets Manager under nc/slack/bot-token. The signing secret is nc/slack/signing-secret.

Observability and cost gates

  • CloudWatch Logs: all Lambdas, 7-day retention, structured JSON. Subscription filter on "error" + "throttle" + "timeout" to a CloudWatch metric for alerting.
  • Alarms: composer Lambda failures > 0 on a run day (the weekly run is the piece that has to work); send-issue failure rate > 1% in a send; action-handler signature-verification failures > 5/hour (might mean the Slack secret rotated); SES complaint rate above the SES threshold.
  • X-Ray: off by default. Not worth the cost at SMB volume.
  • AWS Budgets: $15/month threshold, alarm at 80% and 100%, posts to SNS topic nc-cost-alarm subscribed to the on-call admin’s email and Slack.

Config and secrets

Service-account credentials for Drive, Sheets, and Docs APIs all live in Secrets Manager under nc/drive/sa (one service account with scopes for all three APIs). Slack bot token and signing secret under nc/slack/*. SES sender identity lives in IAM and the verified-domain config. The configured timezone, item threshold, send day, reviewer mapping, quiet-hours window, cooling-off minutes, and admin fallback all live in Parameter Store under /nc/config/. Lambdas fetch config on cold start and cache for the lifetime of the execution environment.

Deploy

GitHub Actions with OIDC into a deploy role (no long-lived keys) and AWS SAM. The opinionated bits: deploy the SES rule set as a separate stack (rule-set changes affect mail flow), turn on S3 versioning for nc-items-source, nc-rules-source, and nc-drafts so a bad edit can be rolled back in one click, and version the EventBridge Scheduler timezone setting so you don’t accidentally start drafting in UTC after a CI rotation. Keep the cooling-off window and the subscriber-list handling in their own reviewable config so a change to either is a visible PR. Total deployable surface: around eight Lambdas, four DDB tables, four S3 buckets, one EventBridge rule on the default bus (plus the Scheduler rules), one SES rule set, and one Budgets alarm.

That’s the full system. Six narrative posts and this engineering reference. If you want to talk about adapting it for your business, see Work with me.

All posts