build.allanninal.dev

Part 1 of 7 · Sentiment monitor series ~5 min read

A sentiment monitor on AWS for a few dollars a month

A small business is talked about in more places than anyone keeps up with. The one-star review that landed at 11pm and is already showing on the search result. The comment thread on last week’s post that quietly soured. The handful of replies under an ad that went from curious to annoyed. By the time someone notices, the mood has already moved — and the first you hear of it is a customer asking why everyone seems upset. This post walks through the design of a small monitor that watches the mentions you point it at, reads the mood of each, tracks whether tone is rising or falling, and tells you — clearly, and before it gets worse. It never replies. That part stays human.

Key takeaways

  • Three sources for mentions: review-site feeds, a social-listening export, and your own comment webhooks.
  • Every mention gets a mood score and a one-line reason from one cheap model call.
  • A rolling trend tracks whether tone is rising or falling, in plain Python — no model on the math.
  • Two instant-alert triggers: a single very angry mention, or a sharp drop in the average.
  • It only listens and reports. It never posts a reply. Designed on AWS for about $2/month at SMB volume.

The whole system on one page

Before any code, here’s the shape of what we’re designing.

System architecture: three sources, three pieces inside AWS At the top, three external boxes in a row. Far left, "Sources" — the review-site feeds, social-listening export, and comment webhooks you point the monitor at; new mentions flow from here into AWS. Centre, "Rules and voice" — a config doc with the alert thresholds, the mood floor, the trend slope, the trailing window, plus a voice doc with the weekly-pulse layout and the alert wording. Far right, "Your inbox" — where the weekly pulse and the instant alerts land for a human to read. Each connects via an arrow to the AWS account container below. Sources have an outgoing arrow into AWS. Rules and voice feed in to ground every alert. Your inbox receives the weekly pulse and the instant alert with the angriest mention listed first and a link to the original. Inside the AWS account are three components in a row, mirroring the layout above. On the left, the Collector — polls each source, de-duplicates, and queues each new mention. In the middle, the Reader — reads the mood of each mention with one Bedrock call, stores the score, and updates the rolling trend. On the right, the Reporter — checks the alert triggers, sends an instant alert if the mood turns bad, and emails the weekly pulse. Internal arrows flow left to right. A note at the bottom reads: the monitor only listens and reports — it never posts a reply. Sources reviews, posts, comments Rules and voice thresholds, templates Your inbox where reports land mentions in grounds pulse and alerts AWS account Collector poll, de-duplicate, queue each mention Reader score the mood, update the rolling trend Reporter instant alert + pulse, never replies mention mood The monitor only listens and reports — it never posts a reply, and a human reads the worst first.
Fig 1. Three sources outside, three pieces inside AWS. Mentions flow in from review feeds, a social-listening export, and comment webhooks. The Reader scores each mention’s mood and updates the trend. The Reporter sends the weekly pulse and the instant alert — and never replies.

What you set up once (the outside)

  • Sources. A short list of the places you want watched: a review-site feed for each platform you care about, an export from whatever social-listening tool you already pay for, and a webhook for the comments on your own site or app. Each source has a type and a way to reach it — a feed URL, an export bucket, or a webhook secret. You point the monitor at the handful that matter and ignore the rest; it only reads what you list. New sources can be added later without a deploy — covered in Part 2.
  • A rules folder. Two short docs in a folder you control. The rules doc holds the numbers: the mood floor that makes a single mention an instant alert, the trend slope that makes a sharp drop an instant alert, the trailing window the average is measured over (default 14 days), and quiet hours so a routine alert doesn’t ping at 3am. The voice doc holds the layout of the weekly pulse and the wording of the alert — what the email actually says. Both are plain text; a non-technical owner can edit either one.
  • Your inbox. Where reports land. The weekly pulse is an email every Monday morning. The instant alert is an email (and an optional text) the moment a trigger fires, with the angriest mention quoted at the top and a link straight to the original so a human can decide what to do. There is no button that replies for you — on purpose.

What runs on each cycle (the inside)

  • The collector. A poller checks each source on a schedule (every 15 minutes by default). For each source it pulls anything new, and de-duplicates — it keeps a record of every mention it has already seen, so the same review never gets read twice even if a feed re-lists it. Each genuinely new mention is dropped into a queue, which evens out bursts (a viral post can produce hundreds of comments in an hour) so the rest of the system isn’t overwhelmed.
  • The reader. Pulls mentions off the queue and reads the mood of each one with a single Bedrock Haiku 4.5 call. The call returns a mood score (how positive or negative, on a fixed scale) and a one-line reason (“upset about a late delivery”). The score is stored against the mention. Then plain Python updates a rolling average — the trend — so the system always knows whether tone over the trailing window is rising or falling. The model only labels mood; it never decides whether to alert, and it never writes a reply.
  • The reporter. After each batch is read, plain Python checks the two alert triggers. If a single mention scored at or below the mood floor, or if the rolling average dropped faster than the configured slope, it fires an instant alert through SNS — an email, optionally a text — with the angriest item listed first. Separately, once a week it emails the pulse: the trend line, counts by source, the standout good and bad mentions, and what changed since last week. One Bedrock call writes the pulse’s summary sentence; the rest is layout.

In plain words

On Thursday afternoon a delivery goes wrong and three customers say so — one in a review, two in comments under the same post. None is the worst review you’ve ever had, but together they pull the trailing-14-day average down sharply in a few hours. The collector picks all three up within fifteen minutes of each appearing. The reader scores them — two clearly negative, one furious — and the rolling average drops past the slope you set. The reporter fires an instant alert to your phone: “Mood falling fast — 3 negative mentions in 4 hours, worst one quoted below, all about delivery.” You read the furious one, see it’s the same root cause, and handle it yourself — reply to the customer, fix the delivery, post an update. The monitor did none of that. It only made sure you knew while there was still time to act.

The cost of running this is about $2 a month at SMB volume. The cost of not running it is the slow week where the mood quietly turned and nobody saw it until a prospect mentioned “the reviews lately.”

Design rules that shaped every decision

  • It only listens and reports. It never posts a reply or touches your public accounts. Replies stay human.
  • The angriest item is always surfaced first, so a human reads the worst before deciding anything.
  • One cheap model call per mention, for mood only. The trend math and the alert logic are plain Python.
  • Two alert triggers, both deterministic: a single very angry mention, or a sharp drop in the average.
  • Thresholds live in a doc you edit. Changing the mood floor or the trend slope doesn’t need a deploy.
  • Every mention and score is logged. Look back next quarter and you can see exactly what moved the trend.

Why this shape

Most owners track their reputation in one of three ways: a tab they open when they remember, an alert from one platform that misses the other five, or a gut feeling. The open-tab approach works until a busy week, when nobody opens it. The single-platform alert is loud about one place and silent about everywhere else. And the gut feeling is always a few days behind the actual mood — by the time it “feels off,” the bad week already happened.

The setup above watches every source you name, reads the mood of each mention the same way, and turns that into one number you can trust: is tone rising or falling right now. It is quiet on the normal days. It speaks up the moment a single mention is furious or the average drops sharply. And it never, ever replies for you — because a wrong public reply from a bot is far worse than a slow one from a human. The monitor’s whole job is to make sure the human is never the last to know.

The next four posts walk through each piece in turn: how a mention gets collected, how the mood of a mention gets read, how an angry mention reaches a human, and how the weekly pulse gets sent. One diagram per post. A cost breakdown and a final engineering reference at the end.

All posts