build.allanninal.dev

Part 3 of 7 · Survey analyzer series ~5 min read

How answers get grouped into themes

Once a week, an EventBridge Scheduler rule fires the grouper Lambda. The Lambda reads the week’s answers, turns each one into a vector, groups the close ones together, and lands on a handful of real themes — each with a count and one real quote. This is the one place in the whole system where AI genuinely earns its keep: plain code can’t tell that “took forever to pay” and “the line at the till was endless” are the same complaint. Embeddings can. But the counting is still plain math, and the model only ever names and quotes — it never decides the numbers.

Key takeaways

  • The grouper runs once a week via EventBridge Scheduler — not on every answer.
  • Each answer becomes a vector via Amazon Titan Text Embeddings V2 (1024 numbers per answer).
  • Close vectors are clustered with plain code; the cluster sizes are the theme counts.
  • Claude Haiku 4.5 names each cluster and picks one real answer as the representative quote.
  • A cluster has to clear a minimum size to count as a theme; small ones fall into a long tail.

The grouping flow, per weekly run

Grouping flow per weekly run A vertical flow diagram. At the top, an input box "Week's answers from store" with each answer's id, text, date, and rating. Below that, a step "Embed each answer" — Amazon Titan Text Embeddings V2 turns each cleaned answer into a 1024-number vector, stored in the S3 Vectors index. Below that, a step "Cluster the vectors" — plain-code clustering groups vectors that sit close together so answers that mean the same thing land in the same group. The next step "Drop tiny clusters" — any group smaller than the minimum theme size from the rules doc is set aside into a long tail rather than reported as a theme. The next step "Name each theme" — a Claude Haiku 4.5 call reads a sample from each cluster and writes a short plain-English name like Slow checkout or Friendly staff. The next step "Pick a representative quote" — for each named theme, the answer closest to the cluster centre is chosen as one real verbatim quote. Four terminal boxes — Long tail set aside, Quote chosen, Counts recorded, and Named theme — feed the weekly theme table that the summary reads. A note at the bottom: the model only names and quotes; the cluster sizes are the counts, computed in plain code, never guessed. Week’s answers from store id · text · date · rating Step 1 Embed each answer Titan V2 → 1024-number vector Step 2 Cluster the vectors plain-code, close ones group Step 3 Drop tiny clusters below min size → long tail Step 4 Name each theme Haiku 4.5 reads a sample writes a short plain name Step 5 Pick a representative quote answer nearest the centre Long tail small clusters set aside Quote chosen one real answer Counts recorded cluster size = count Named theme plain-English label too small tail name quote count The model only names and quotes — the cluster sizes are the counts, in plain code.
Fig 3. The grouper’s flow, per weekly run. Five steps turn a week of raw answers into a handful of named, counted, quoted themes. The model labels and quotes; the clustering and the counting are plain code.

Step 1: turn each answer into a vector

The grouper reads the week’s cleaned answers from the store. For each one, it calls Amazon Titan Text Embeddings V2 and gets back a vector — a list of 1024 numbers that captures what the answer is about. The useful property is that two answers with the same meaning produce vectors that sit close together, even when they share no words. “Took forever to pay” and “the line at the till was endless” end up near each other; “loved the staff” ends up somewhere else entirely. The vectors are saved in an Amazon S3 Vectors index so the grouper can search and compare them cheaply without running a database server.

This is the step plain code simply cannot do. Keyword matching would put “till” and “pay” in different buckets and miss that they’re the same gripe. Embeddings are what make grouping by meaning possible at all — which is exactly why this is one of the few places the system reaches for a model.

Step 2: cluster the close vectors

With every answer now a point in space, grouping the close ones is a plain-math job — no model needed. The grouper runs a standard clustering routine in Python that finds dense groups of nearby points and leaves the scattered ones ungrouped. Answers that mean roughly the same thing fall into the same cluster; genuinely different answers form their own. Crucially, the number of answers in each cluster is just a count — a real, exact number, computed by counting rows. When the summary later says “61 people raised slow checkout,” that 61 is a count of actual answers, not a model’s estimate.

Step 3: set aside the tiny clusters

Not every cluster is a theme. Three people mentioning the same oddly specific thing is interesting, but it isn’t a trend, and a summary that lists fifteen “themes” of two answers each is just noise in a nicer font. The rules doc sets a min_theme_size (default around 1% of the week’s answers, with a floor of five). Any cluster below that is folded into a long tail that the summary mentions in one line (“plus a scattering of one-off comments”) but doesn’t break out. This keeps the summary to the handful of themes that actually matter.

Step 4: name each theme

Now the model earns its second keep. For each surviving cluster, the grouper sends Claude Haiku 4.5 a small sample of the answers in it and asks for a short, plain-English name: “Slow checkout,” “Friendly staff,” “Parking is hard.” The prompt is tight: “Name the shared topic in three or four words. Don’t add a topic that isn’t in these answers. Return the name only.” The model never sees the counts and never decides which answers belong to which group — the clustering already did that. It’s purely putting a readable label on a group plain code already formed.

Step 5: pick one real quote

A count and a name tell you what people said and how many; a quote tells you how it felt. For each theme, the grouper picks the answer sitting closest to the centre of its cluster — the most representative single response — and stores it verbatim as the theme’s quote. It’s a real thing a real person wrote, shown word for word, never a paraphrase the model invented. If the most central answer is unusually long, Haiku is allowed to trim it to a clean sentence, but only by cutting — never by rewording.

Why weekly, and why this split of work

The grouper runs weekly rather than on every answer for two reasons. First, grouping only makes sense in bulk — you can’t cluster one answer. Second, embedding and clustering a whole week at once is far cheaper than re-running the math every time a single answer lands. The urgent lane in Part 5 is what handles the “can’t wait” case; the grouper is deliberately the slow, thorough, weekly read.

And the split of work — model for embeddings, naming, and quote-trimming; plain code for clustering and counting — is the whole reason the numbers are trustworthy. The expensive, fuzzy judgment (what does this mean?) goes to the model. The exact, checkable arithmetic (how many?) stays in code where it can’t drift. That’s what lets the summary say a number and mean it.

Next post: how the summary gets written from these themes and reaches the owner’s inbox — and the guardrails that keep it honest.

All posts