build.allanninal.dev

Part 4 of 7 · Receipt organizer series ~5 min read

How a receipt gets categorized

The reader pulled clean fields off a receipt — vendor, date, total, tax. Now the organizer has to decide which expense category it belongs to: is this lunch a “meal,” the petrol “fuel,” the software subscription “software”? Get it wrong and the books are misleading and the tax return is off. So the category isn’t a wild guess from an AI model. Four small gates sit between the read fields and the filed record, and the AI only weighs in once it’s grounded in your own rules.

Key takeaways

  • Categories come from your own chart of accounts in the rules doc — not a generic list.
  • A vendor-hint lookup handles the easy cases first, with no AI call at all.
  • Bedrock Haiku 4.5 picks the category for the rest, choosing only from your account list.
  • A sanity check makes sure the model returned a real category and a confidence score.
  • Low-confidence picks go to the bookkeeper’s review queue, not straight to the books.

Four gates on every category decision

Four gates between a read receipt and a filed expense category A horizontal flow diagram. On the far left, a "Fields read" box: the reader passed clean fields — vendor, date, total, tax — for a receipt that was confident enough to categorize. Four gates sit in a row to the right, each drawn as a vertical bar. Gate 1: Vendor hint — checks the vendor name against the per-vendor hints in the rules doc; if there's a match, like Shell mapping to fuel, the category is set with no AI call at all. Gate 2: Ask the model — for vendors with no hint, Bedrock Haiku 4.5 is given the read fields and the chart of accounts and asked to pick exactly one category from that list, with a short reason and a confidence score; it cannot invent a category that isn't in the chart. Gate 3: Sanity check — confirms the model returned a real category that exists in the chart, that the confidence is a number, and that the tax-to-total ratio is plausible for that category; anything off is treated as low confidence. Gate 4: Review gate — compares the category confidence against the threshold in the rules doc; above it, the receipt is cleared to file with its category; below it, the receipt is sent to the bookkeeper's review queue with the proposed category, the reason, and an Approve button. After all four gates pass, the categorized record moves on to filing. A note at the bottom: the chart of accounts is the only menu — the model picks from it, never beyond it. Fields read vendor, date, total, tax (confident) Gate 1 Vendor hint known vendor? e.g. Shell → fuel set, no AI call no hint? pass to Gate 2 Gate 2 Ask the model Haiku 4.5 picks from the chart returns reason + confidence, never invents Gate 3 Sanity check real category? score a number? tax ratio plausible? else low conf. Gate 4 Review gate conf vs threshold above → file below → send to review queue Categorized — cleared to file (confident) or queued for the bookkeeper (unsure) category + reason + confidence travel with the record every category choice is logged — a corrected one teaches the vendor hints The chart of accounts is the only menu — the model picks from it, never beyond it.
Fig 4. Four gates between the read fields and the category. Try a vendor hint first. Ask the model only when needed. Sanity-check the answer. Then either file with confidence or send to the bookkeeper. The chosen category and its reason travel with the record.

Gate 1: the vendor hint (the free, instant case)

Most categorization is boring and repetitive. Anything from Shell or BP is fuel. Anything from your one software vendor is software. Anything from the office-supply shop down the road is office supplies. The rules doc holds a short list of these per-vendor hints — vendor name on the left, category on the right. Gate 1 checks the read vendor against that list first. If there’s a match, the category is set right there, with no AI call at all. Over a few weeks of use, this list grows to cover the vendors you see most, and the share of receipts that even reach the model shrinks.

This is the cheapest possible path: a lookup in a short list. The hints are plain text a bookkeeper can edit in Drive, so adding “the new café near the office goes to meals” is a thirty-second edit, not a code change.

Gate 2: ask the model (only when there’s no hint)

For a vendor with no hint — a one-off supplier, a shop nobody’s seen before — the organizer asks Bedrock Haiku 4.5. The prompt is short and tightly framed: here are the read fields, here is the full chart of accounts, pick exactly one category from this list, give a one-line reason, and rate your confidence. The crucial constraint is that the model chooses from your list. It can’t invent “travel-meals-misc” that your bookkeeper has never heard of; it has to return one of the categories that already exists in your books.

Haiku is the right model here: this is a small, well-defined classification job, not heavy reasoning, so the cheap, fast model is plenty. It runs only on the receipts that got past Gate 1, which over time is the minority. The reason it returns matters too — “restaurant name and a tip line suggest a meal” — because if a human ends up reviewing it, that one line saves them re-reading the whole receipt.

Gate 3: the sanity check

An AI answer needs checking before it’s trusted. Gate 3 confirms three things. The category the model returned actually exists in the chart of accounts (if it somehow returned something off-list, that’s treated as no answer). The confidence is a real number, not missing. And the numbers are plausible for the category — a receipt where the tax is larger than the total, or a “fuel” receipt for $4,000, is suspicious enough to flag regardless of how confident the model was. Anything that fails a sanity check is downgraded to low confidence, which means it heads to a human at the next gate.

Gate 4: the review gate (the human check)

The last gate is the one that decides whether a person ever sees the receipt. It compares the category confidence against the threshold in the rules doc. Above the line — a clear vendor hint, or a confident model pick that passed the sanity check — and the receipt is cleared to file with its category attached. Below the line — an unsure model pick, a sanity-check downgrade, a category the business has flagged as “always review” (some businesses want every “entertainment” receipt eyeballed) — and it’s sent to the bookkeeper’s review queue with the proposed category, the model’s reason, and an Approve button.

When the bookkeeper corrects a category — “no, that vendor is office supplies, not software” — the system offers to remember it as a new vendor hint. Accept, and the next receipt from that vendor skips the model entirely and files on its own. The system gets quieter the more it’s used.

Why the gates exist

None of these gates is clever on its own. They’re the steps a careful bookkeeper already takes — recognize the regulars on sight, look up the unfamiliar ones, double-check anything that smells wrong, and ask when genuinely unsure. Putting them in code as four small gates makes the AI a junior assistant that proposes, not a black box that decides. The chart of accounts is the only menu it gets to choose from, every choice comes with a reason, and the unsure ones always reach a human before they touch the books.

Next post: how a categorized receipt actually reaches the books — the clean record written to the expense sheet, the image filed by month, and what happens when the bookkeeper approves or corrects a reviewed one.

All posts