build.allanninal.dev

Part 7 of 7 · Sentiment monitor series ~8 min read

Engineering reference: the sentiment monitor architecture

Same system, drawn for engineers. Region, service names, resource identifiers, Bedrock model IDs, Lambda inventory, IAM scopes, the SQS and dead-letter-queue config, EventBridge Scheduler config, the DynamoDB schemas, and the alert delivery path. Read alongside the previous six posts; this one’s the build sheet.

Region and account shape

Default region: ap-southeast-1 (Singapore). Bedrock cross-Region inference, SQS, SNS, SES, and EventBridge Scheduler are all in good shape there. A second region for multi-region resilience isn’t worth the extra setup work at SMB volume — the failure mode for an SMB is a missed mood swing, not a regional outage, and a stalled poll simply resumes on the next tick. One AWS account dedicated to the monitor (separate from your other workloads) keeps the IAM blast radius small and lets a single AWS Budgets alarm cover the whole system. The monitor has no write path to any external account by design: it reads sources and sends to your own inbox, nothing else.

Topology

AWS topology of the sentiment monitor A topology diagram with three regions stacked vertically inside one AWS account boundary. Top region: ingress. Three boxes show the three intake lanes — a poller Lambda triggered every 15 minutes by EventBridge Scheduler that reads each configured review feed and de-duplicates against the sm-seen table; an export-parser Lambda triggered by an S3 PUT on s3://sm-listening-export/ that reads the social-listening export and de-duplicates the same way; and a webhook-handler on a Lambda Function URL that validates a shared secret on inbound comments and de-duplicates by id. All three send new mentions to the SQS queue sm-mentions, which has a dead-letter queue behind it. Middle region: scheduled and queued processing. The reader Lambda has an SQS event source on sm-mentions; for each mention it pre-checks, calls Bedrock Haiku 4.5 for a mood score and one-line reason, writes the score to DynamoDB sm-scores, and recomputes the rolling trend; failures land in the dead-letter queue. Bottom region: report and alert. The reporter Lambda is triggered after each batch and on a weekly EventBridge Scheduler rule; it reads sm-scores, checks the two alert triggers, de-bounces against sm-alerts, and either fires an instant alert through SNS topic sm-alerts-topic to email and optional SMS, or assembles the weekly pulse and sends it via SES. A separate pulse path uses one Bedrock Haiku 4.5 call for the summary line. CloudWatch Logs collects from every Lambda at 7-day retention. Across the right edge: a small box labelled AWS Budgets alarm at $15 monthly threshold, posting to SNS topic sm-cost-alarm. A note at the bottom: the monitor only reads and reports — there is no write path back to any source. Ingress Lambda · poller every 15 min reads review feeds de-dupes vs sm-seen → sm-mentions queue Lambda · export-parser S3 PUT trigger s3://sm-listening-export/ de-dupes each row → sm-mentions queue Lambda · webhook-handler Function URL verifies shared secret de-dupes by id → sm-mentions queue SQS sm-mentions de-duped · DLQ behind it Queued processing SQS event source batch of mentions to the reader DLQ on repeat fail maxReceiveCount 3 Lambda · reader pre-check, then Bedrock Haiku 4.5 writes sm-scores, recomputes trend DynamoDB sm-scores score per mention source · posted-at reason line (trend reads from here) Report & alert Lambda · reporter checks 2 triggers, de-bounces, quiet hours, ranks worst first → SNS SNS + SES out instant alert via sm-alerts-topic weekly pulse via SES (no reply path) Lambda · pulse weekly Scheduler, reads sm-scores, one Haiku 4.5 line, SES to recipients The monitor only reads and reports — there is no write path back to any source.
Fig 7. AWS topology, in three regions of the diagram: ingress (three lanes into one queue), queued processing (the reader scoring each mention into sm-scores), and report and alert (the reporter and pulse fanning out through SNS and SES). Every Lambda is event-, queue-, or schedule-driven; nothing is synchronous-chained, and nothing writes back to a source.

Lambda functions

All Lambdas use the arm64 architecture, the smallest memory size that meets latency targets (typically 256 MB), Python 3.14 runtime, and CloudWatch Logs at 7-day retention. Each function has its own least-privilege IAM role. None run inside a VPC.

  • poller — EventBridge Scheduler target, fires every 15 minutes. Reads each review feed listed in /sm/config/sources (Parameter Store), computes a stable id per item (sha256(platform + native_id)), checks it against sm-seen with a conditional write, and sends genuinely new items to the sm-mentions SQS queue. Feed credentials, where needed, live in Secrets Manager under sm/sources/*. Memory: 256 MB. Timeout: 30 s.
  • export-parser — S3 PUT trigger on s3://sm-listening-export/. Reads the dropped export (CSV or JSON Lines), iterates rows, computes the stable id, de-duplicates against sm-seen, and enqueues new mentions to sm-mentions. Tolerant of partial files: a malformed row is logged and skipped, never fatal. Memory: 256 MB. Timeout: 60 s.
  • webhook-handler — Lambda Function URL, AuthType: NONE, verifies an HMAC shared secret (sm/webhook/secret in Secrets Manager) on the raw request body before doing anything. Parses the comment payload, computes the id, de-duplicates, and enqueues to sm-mentions. Returns 200 quickly; all work after signature check is minimal. Memory: 256 MB. Timeout: 15 s.
  • reader — SQS event source on sm-mentions, batch size 5, with partial-batch-response enabled so one bad mention doesn’t fail the batch. For each mention: clean/pre-check; if worth a call, invoke Bedrock Haiku 4.5 (anthropic.claude-haiku-4-5-20251001-v1:0 via global.anthropic.claude-haiku-4-5-20251001-v1:0) for a mood score (−2..+2) and a one-line reason, JSON only; validate; write to sm-scores; recompute the trailing-window rolling average and write it to sm-trend. Heavier reasoning is not justified here, so Sonnet is not used. Memory: 512 MB. Timeout: 60 s. Mentions that fail 3 times land in the DLQ.
  • reporter — invoked after a reader batch (via an internal event) to evaluate the two alert triggers. Reads sm-trend and the relevant sm-scores rows; applies the single-mention floor and the slope check from /sm/config/rules; de-bounces against sm-alerts within the cool-down window; applies quiet hours (deferring routine alerts via a one-off EventBridge Scheduler rule, letting urgent ones through); ranks worst-first; composes from the voice template; and publishes to SNS topic sm-alerts-topic. Writes the alert to sm-alerts. No Bedrock calls. Memory: 256 MB. Timeout: 30 s.
  • pulse — EventBridge Scheduler target, weekly Monday 8am local. Reads the past week of sm-scores and last week’s stored summary; builds the three blocks (trend, standout mentions, what-changed) in plain Python; makes one Bedrock Haiku 4.5 call for the summary line; renders the email and sends via SES to the recipients in /sm/config/recipients; writes this week’s figures to sm-summary for next week’s comparison. Memory: 512 MB. Timeout: 60 s.
  • housekeeping — EventBridge Scheduler target, hourly. Expires old sm-seen rows past the de-dup horizon (TTL-driven, this is a safety sweep), and re-drives any DLQ messages that look transient. Memory: 256 MB. Timeout: 30 s.

Storage

  • DynamoDB · sm-scores — one row per scored mention. PK mention_id; sort key posted_at; attributes: source, score (−2..+2), reason, author, link. On-demand. GSI on (source, posted_at) for per-source weekly rollups. No TTL — this is the long-term record.
  • DynamoDB · sm-seen — de-duplication ledger. PK mention_id; attribute first_seen. TTL on a 90-day horizon so the table self-trims. On-demand. Written with a conditional put so concurrent lanes can’t double-enqueue.
  • DynamoDB · sm-alerts — one row per instant alert. PK event_key (a hash of the trigger type + window bucket); attributes: fired_at, count, worst_mention_id, severity. On-demand. The de-bounce reads this; count grows as repeat firings fold in.
  • DynamoDB · sm-trend — the current rolling average and slope. PK window (e.g. rolling_14d); attributes: avg, prev_avg, slope, updated_at. On-demand. Tiny; effectively a single hot row.
  • DynamoDB · sm-summary — one row per weekly pulse for week-over-week deltas. PK week_start; attributes: per-source counts, weekly average, standout ids. On-demand.
  • S3 · sm-listening-export — drop zone for the social-listening tool’s exports. Versioning enabled. Lifecycle to Glacier at 30 days; expiry at 1 year.
  • S3 · sm-config-source — the rules and voice docs as plain text, mirrored to Parameter Store keys on change. Versioning enabled so a bad edit rolls back in one click.

SQS and the dead-letter queue

  • sm-mentions — standard queue, visibility timeout 90 s (six times the reader timeout headroom is not needed; the reader is fast). Redrive policy: maxReceiveCount: 3 to the DLQ. Absorbs ingest bursts from a viral thread so the reader processes at a steady, cost-predictable rate.
  • sm-mentions-dlq — dead-letter queue for mentions that fail to process three times (bad payload, transient Bedrock error that didn’t clear). A CloudWatch alarm on DLQ depth > 0 pages the admin; housekeeping re-drives anything that looks transient.
  • Partial-batch response — the reader returns batchItemFailures so a single poison mention is retried/dead-lettered without re-processing (and re-charging Bedrock for) the rest of its batch.

Bedrock

  • Foundation model. anthropic.claude-haiku-4-5-20251001-v1:0 via the Global cross-Region inference profile global.anthropic.claude-haiku-4-5-20251001-v1:0. Two callsites: reader for the per-mention mood read, and pulse for the weekly summary line. Sonnet 4.6 is deliberately not used — mood labeling and a one-line summary don’t justify the heavier path.
  • Prompts. Both prompts demand JSON-only output with a fixed schema; the reader pins the score to the −2..+2 range and instructs the model to return 0 with a note when text is ambiguous rather than guessing.
  • Embeddings. Not used. The monitor scores and trends; it doesn’t retrieve past mentions by meaning. No Titan embeddings, no S3 Vectors, no Knowledge Base.
  • Quotas. Default account quotas are more than enough at SMB volume. The per-mention calls are short; bursts are smoothed by the SQS queue and the reader’s batch size, so Bedrock is never hit by the raw ingest spike.

EventBridge Scheduler config

  • sm-poll — rate(15 minutes). Target: poller Lambda.
  • sm-housekeeping — rate(1 hour). Target: housekeeping Lambda.
  • sm-weekly-pulse — cron(0 8 ? * MON *) in TZ_NAME. Target: pulse Lambda.
  • One-off rules — created on the fly by reporter when a quiet-hours defer is needed. Use at(YYYY-MM-DDTHH:MM:SS) expressions with --action-after-completion DELETE so the rule self-cleans.

SNS and SES (outbound only)

  • SNS topic sm-alerts-topic carries the instant alerts. Subscriptions: the admin’s email (always) and an optional SMS endpoint for urgent severity. The topic is the only fan-out point for alerts.
  • SES outbound sends the weekly pulse. Verify a sender identity at monitor@your-company.com with DKIM and SPF on the parent domain. Out of sandbox by request. There is no SES inbound rule set — the monitor never receives mail and never replies.
  • No outbound path posts to any review site, social platform, or comment thread. The IAM roles below contain no permission that could.

IAM (least privilege per Lambda)

Each Lambda has its own role with policies scoped to exact ARNs. Sketch:

  • poller role: dynamodb:PutItem (conditional) on sm-seen; sqs:SendMessage on sm-mentions; ssm:GetParameter on /sm/config/*; secretsmanager:GetSecretValue on sm/sources/*; outbound network to the feed hosts. No bedrock:*.
  • export-parser role: s3:GetObject on sm-listening-export; dynamodb:PutItem on sm-seen; sqs:SendMessage on sm-mentions.
  • webhook-handler role: secretsmanager:GetSecretValue on sm/webhook/secret; dynamodb:PutItem on sm-seen; sqs:SendMessage on sm-mentions.
  • reader role: sqs:ReceiveMessage + DeleteMessage on sm-mentions; bedrock:InvokeModel on the Haiku ARN; dynamodb:PutItem + UpdateItem on sm-scores and sm-trend; dynamodb:Query on sm-scores for the trend recompute.
  • reporter role: dynamodb:Query on sm-scores and GetItem on sm-trend; dynamodb:PutItem + UpdateItem on sm-alerts; sns:Publish on sm-alerts-topic; scheduler:CreateSchedule for the quiet-hours one-offs; ssm:GetParameter on /sm/config/*. No bedrock:*.
  • pulse role: dynamodb:Query on sm-scores and sm-summary; dynamodb:PutItem on sm-summary; bedrock:InvokeModel on the Haiku ARN; ses:SendEmail from the verified sender identity; ssm:GetParameter on /sm/config/*.

Trend math and thresholds

The rolling average is a trailing-window mean over sm-scores, default 14 days, recomputed incrementally on each reader batch (subtract the rows now outside the window, add the new ones) so the cost is constant per batch rather than a full table scan. The slope is avg − prev_avg over the batch. Two configurable thresholds gate alerts, both in /sm/config/rules: mood_floor (default −2, the single-mention trigger) and slope_drop (default a drop steeper than 0.4 of a point over the window). A third value, urgent_floor, marks a mention or slope severe enough to bypass quiet hours. All three are plain numbers an owner can edit; no deploy.

Observability and cost gates

  • CloudWatch Logs: all Lambdas, 7-day retention, structured JSON. Subscription filter on "error" + "throttle" + "timeout" to a metric for alerting.
  • Alarms: DLQ depth > 0 (a mention is stuck); reader Bedrock error rate > 2% in 1h (the model path is degraded); poller failures > 0 in an hour (ingest stalled); webhook signature-verification failures > 10/hour (a misconfigured or hostile caller).
  • X-Ray: off by default. Not worth the cost at SMB volume.
  • AWS Budgets: $15/month threshold, alarm at 80% and 100%, posts to SNS topic sm-cost-alarm subscribed to the admin’s email. A runaway re-queue loop (a feed re-listing the same items) shows up here first.

Config and secrets

The source list, the rules thresholds, the quiet-hours window, the timezone, and the recipient list all live in Parameter Store under /sm/config/, mirrored from the plain-text docs in sm-config-source. Feed credentials live in Secrets Manager under sm/sources/*; the webhook HMAC secret under sm/webhook/secret. Lambdas fetch config on cold start and cache it for the lifetime of the execution environment. There are no credentials anywhere that grant write access to an external platform — the monitor has nothing to post with.

Deploy

GitHub Actions with OIDC into a deploy role (no long-lived keys), AWS SAM for the stack. The opinionated bits: deploy the SQS queue and its DLQ together with the redrive policy in one template so they can’t drift apart; turn on S3 versioning for both sm-listening-export and sm-config-source; enable partial-batch-response on the reader’s event source mapping so a poison mention never re-charges Bedrock for its whole batch; and pin the EventBridge Scheduler timezone so the weekly pulse doesn’t silently move to UTC after a CI rotation. Total deployable surface: around seven Lambdas, five DynamoDB tables, two S3 buckets, one SQS queue plus DLQ, two SNS topics, one SES sender identity, the Scheduler rules, and one Budgets alarm.

That’s the full system. Six narrative posts and this engineering reference. If you want to talk about adapting it for your business, see Work with me.

All posts