build.allanninal.dev

Part 3 of 7 · Refund handler series ~5 min read

How a refund gets checked against policy

This is the heart of the system. A request comes off the queue, and the checker has to decide what your policy says about it. The trick is keeping that decision honest: the checker only acts on what the policy actually says, and it says so by quoting the line it used. To do that it pulls the few policy passages that apply to this exact case, decides only from them, and refuses to guess when nothing fits. The result is always one of four outcomes.

Key takeaways

  • The policy lives in a Drive doc; a sync job indexes it into Amazon S3 Vectors for fast lookup.
  • For each request, the checker pulls only the policy passages that apply — not the whole doc.
  • It decides only from those passages and cites the exact line behind every decision.
  • Four outcomes per request: in policy, out of policy, high-value, or not covered.
  • If nothing in the policy fits, the request is marked not-covered and sent to a human — never guessed.

The decision flow, per request

Decision flow per refund request A vertical decision flow diagram. At the top, an input box "Request from queue" with the customer, item, order reference, amount, and what they asked for. Below that, a step "Pull policy passages" — the request is embedded with Titan Text Embeddings V2 and the closest policy passages are fetched from the S3 Vectors index. Below that, a check "Any passage covers this?" — if no passage is a real match, route to "Not covered" (send to a human to decide). If yes, continue. The next step "Ask the model: allowed by these lines?" — the model is given only the pulled passages and told to decide solely from them and cite the line it used. The next step "Over the dollar cap?" — compares the refund amount against the cap named in the policy. If the policy allows the refund and it's under the cap, route to "In policy." If the policy allows it but it's over the cap, route to "High-value." If the policy clearly disallows it, route to "Out of policy." Each terminal box — In policy, Out of policy, High-value, Not covered — carries the cited policy line forward to the drafter. A note at the bottom: the policy doc holds every rule; the checker only reads it — change a line in the doc and the next request uses the new rule. Request from queue customer · item · amount · ask Step 1 Pull policy passages embed → S3 Vectors search Step 2 Any passage covers this? real match, not a guess Step 3 Allowed by these lines? decide only from passages Step 4 Allowed, and under cap? disallowed → out of policy over cap → high-value Step 5 Cite the policy line used carried to the drafter Not covered human decides In policy one-tap approve Out of policy kind decline draft High-value named approver if none unclear over cap allowed disallowed The policy doc holds every rule — change a line and the next request uses the new rule.
Fig 3. The checker’s decision tree, per request. Five steps decide which of four outcomes applies. The policy doc holds every rule; the checker only reads it — and cites the line behind every decision.

Grounding: the checker reads your policy, not its own memory

The danger with any model deciding refunds is that it answers from a general idea of what refund policies usually look like, rather than from yours. A “14-day window” policy gets treated like the “30-day” one the model saw more often in training, and a customer gets the wrong answer. Grounding fixes this. Your policy lives in a Drive doc you write in plain prose. A small policy-sync job mirrors it to S3 every fifteen minutes and breaks it into short passages — one per rule or exception. Each passage is turned into a row of numbers (an “embedding,” a math fingerprint of the meaning) by Titan Text Embeddings V2 and stored in Amazon S3 Vectors, a cheap store built for exactly this kind of lookup.

When a request comes in, the checker turns the request into the same kind of fingerprint and asks S3 Vectors for the closest few policy passages. Those — and only those — are handed to the model. So the model is never deciding from memory; it’s deciding from the exact lines of your policy that match this case. Change a window from 30 days to 14 in the doc, and within fifteen minutes the next request is judged by 14.

Four outcomes, always

Every request, every time, lands in exactly one of four buckets. The names are plain on purpose.

  • In policy. The pulled passages clearly allow the refund and the amount is under the cap. This is the easy lane — route a one-tap approve card with the draft attached. Most requests, most days, land here.
  • Out of policy. A passage clearly disallows it — the item is past the window, or it’s a final-sale item. Draft a kind decline that quotes the exact rule, and route it to a person to confirm before it goes out. A “no” still gets a human tap, because a wrongly-refused refund is its own kind of damage.
  • High-value. The policy allows it, but the amount is over the dollar cap (“refunds over $200 need a manager”). Route to the named approver in the policy, not the one-tap lane. Same draft, higher bar.
  • Not covered. No passage really matches — the case is genuinely new (a goodwill ask, a shipping mishap the policy never anticipated). Mark it not-covered and send it straight to a human. The checker never invents a rule to cover a gap.

Cheap on the easy cases, careful on the hard ones

Most requests are clear — in the window, normal item, normal amount. Those run on Claude Haiku 4.5, a small fast model, because the decision is easy once the right policy passages are in front of it. A few requests are genuinely tricky: conflicting passages, a partial-refund judgment call, an exception inside an exception. For those, the checker escalates the single decision to Claude Sonnet 4.6, a stronger model, and pays a few cents more for a better read. The split keeps the bill low while still giving the hard cases the care they need. Either way the rule is the same: decide only from the pulled passages, and cite the line.

Why every decision carries a citation

The cited policy line isn’t decoration. It does three jobs. It lets the approver sanity-check the decision in two seconds — “yes, that’s the right rule.” It becomes the “why” the drafter quotes to the customer, so the reply explains itself. And it goes into the audit log, so a refund questioned six months later can be traced back to the exact policy line that justified it. A decision without a citation is just an opinion; a decision with one is something the whole team can stand behind.

Next post: how the reply gets drafted — kind, plain, in your voice, quoting the policy line — and the four guardrails between the draft and the person who approves it.

All posts