build.allanninal.dev

Part 1 of 7 · FAQ builder series ~5 min read

A self-updating FAQ builder on AWS for a few dollars a month

Every small business answers the same questions over and over. “Do you ship to Canada?” “How do I reset my password?” “What’s your refund window?” “Can I change my plan mid-month?” The answers live in someone’s head, or in a help doc nobody links to, or in a hundred past replies that all said roughly the same thing. Your FAQ page, meanwhile, was last touched a year ago. This post walks through the design of a small builder that reads the questions people actually ask, spots the ones that keep coming back, drafts a clear answer from your own help docs, and hands you a proposed FAQ entry to approve before anything goes live.

Key takeaways

  • Three sources of questions: a support-inbox lane, a chat-export lane, and a manual lane.
  • Questions that keep repeating get grouped into clusters; a cluster that crosses a threshold earns a FAQ entry.
  • Answers are drafted from your own help docs and must cite a source — no made-up facts.
  • Nothing publishes on its own. Every entry is approved, edited, or rejected by a person first.
  • Designed on AWS for about $3/month at typical small-business volume.

The whole system on one page

Before any code, here’s the shape of what we’re designing.

System: three sources, three pieces inside AWS At the top, three external boxes in a row. Far left, "Customer questions" — the questions people actually ask, arriving via a support-inbox forwarding lane, a chat-export lane, and a manual lane where a rep drops in a question worth answering. Centre, "Help docs and voice" — a Drive folder with your existing help docs that answers are grounded in, plus a voice doc with the tone and format the FAQ should follow. Far right, "Reviewers" — the team members who approve entries; proposals land in their review queue. Each connects via an arrow to the AWS account container below. Customer questions have an outgoing arrow into AWS. Help docs and voice feed in to ground every draft. Reviewers receive each proposed FAQ entry with the question, the drafted answer, the cited source passage, and approve, edit, and reject buttons. Inside the AWS account are three components in a row, mirroring the layout above. On the left, the Question intake — receives questions from each source, cleans them, embeds each one, and adds it to the question pile. In the middle, the Grouper — runs daily; reads the new questions; groups near-duplicates into clusters; flags clusters that cross a repeat threshold as candidates for a FAQ entry. On the right, the Drafter — pulls matching help-doc passages for a candidate cluster, asks the model for a short grounded answer that cites a source, and posts the proposed entry to the review queue. Internal arrows flow left to right. A note at the bottom reads: every answer is grounded and cited — the builder never invents a fact, and a human approves before it publishes. Customer questions inbox, chat, manual Help docs and voice sources, tone, format Reviewers where entries get approved questions in grounds proposed entry with source AWS account Question intake clean, embed, add to the pile Grouper groups repeats, flags candidate clusters Drafter grounded answer, cites a source pile cluster Every answer is grounded and cited — the builder never invents a fact, and a human approves it.
Fig 1. Three sources outside, three pieces inside AWS. Questions flow in from a support inbox, a chat export, and a manual lane. The Grouper runs daily and finds the repeats. The Drafter writes a grounded answer for each cluster that earns one, and a reviewer approves it.

What you set up once (the outside)

  • Customer questions. The raw material. Three lanes feed the pile, covered in Part 2: a support-inbox forwarding lane (forward or auto-route support email to a dedicated address and the builder reads the question out of it), a chat-export lane (drop your chat transcripts in a Drive folder and the builder picks them up), and a manual lane (a rep types in a question they keep getting asked). You don’t have to wire up all three; one is enough to start.
  • A help-docs folder. Your existing help content in a Drive folder — setup guides, policy pages, product notes, whatever you already wrote. The builder reads these to ground every answer, so the FAQ says what your docs say, not what a model guessed. A short voice doc sits alongside them: the tone and length the FAQ should follow (“friendly, two or three sentences, no jargon”) plus any phrases to avoid.
  • Reviewers. The people who approve entries. Each proposed FAQ entry lands in a review queue — a Slack message or a simple web list — with the question, the drafted answer, the source passage it was grounded in, and three buttons: approve, edit, reject. Nothing reaches the live FAQ doc until somebody taps one.

What runs each day (the inside)

  • The question intake. As questions arrive, a small Lambda cleans each one (strips signatures, greetings, and order numbers so the question stands on its own), turns it into a vector with Titan Text Embeddings V2 (a vector is just a list of numbers that places similar questions near each other), and writes it to the question pile. A vector means the grouper can tell “do you ship to Canada?” and “can you deliver to Toronto?” are the same question without any exact word match.
  • The grouper. Runs once a day. Reads the new questions, compares each against the existing question vectors, and groups the near-duplicates into clusters. A cluster that crosses a repeat threshold — say, asked five times this month and not already covered by an approved FAQ entry — becomes a candidate. The grouping itself is plain Python over the vectors; no model writes anything here.
  • The drafter. For each candidate cluster, a Lambda pulls the help-doc passages that best match the question, then asks Claude Haiku 4.5 for a short answer using only those passages, with an instruction to cite the source passage and to say “not covered in the docs” rather than guess. The proposed entry — question, answer, source link — goes to the review queue. A weekly digest lists what published, what’s waiting, and which clusters had no grounded answer.

In plain words

Over two weeks, eleven different customers email some version of “can I switch from monthly to annual without losing my current discount?” The intake cleans and embeds each one as it arrives. On the daily pass, the grouper notices they’re all the same question and the cluster crosses the threshold. The drafter pulls the two paragraphs from your billing help doc that cover plan changes and proration, and writes: “Yes — you can switch to annual at any time. Your current discount carries over, and we prorate the difference on your next invoice. (Source: Billing & Plans → Changing your plan.)” That proposal lands in your reviewer’s Slack. They read it, see it matches the doc, and tap Approve. The entry is written to the live FAQ doc, and the next customer who searches your help center finds the answer without emailing anyone.

The cost of running this is about $3 a month at SMB volume. The cost of not running it is the same question answered by hand a hundred more times, a help center that quietly goes stale, and the support queue that never quite gets shorter.

Design rules that shaped every decision

  • Answers are grounded in your own docs and cite a source. The builder never speaks for facts it can’t point to.
  • If nothing grounds an answer, the builder says so — it does not fill the gap with a guess.
  • Nothing publishes on its own. A person approves, edits, or rejects every entry.
  • Repeats earn an entry; one-offs don’t. The threshold keeps the FAQ about what people actually ask.
  • The FAQ and help docs live in Drive. Editing an answer doesn’t need a deploy.
  • Every action is logged. Audit any entry later and you can see who approved it and from which source.

Why this shape

Most teams keep a FAQ in one of three states: missing, stale, or guessed-at. The missing one sends every question to a human. The stale one answers questions nobody asks anymore and misses the ones they do. The guessed-at one — the tempting shortcut of pointing a chatbot at your site and letting it answer freely — is the most dangerous, because it will confidently tell a customer something that isn’t true, and you won’t find out until they act on it.

The setup above keeps the source of truth in docs your team already edits, but adds a small system that watches what people ask, only writes answers it can ground, and never publishes without a human. The FAQ stays current because it’s fed by real questions. It stays correct because every answer points at a source. And it stays yours because a person signs off on every word before it goes live.

The next four posts walk through each piece in turn: how the question pile gets built, how repeat questions get grouped, how the answer gets drafted and grounded, and how an entry gets approved and published. One diagram per post. A cost breakdown and a final engineering reference at the end.

All posts