build.allanninal.dev

Part 5 of 7 · Lead intake bot series ~5 min read

How a reply stays on policy

A first-touch reply to a real lead is one of the easiest places for an AI to make an expensive mistake. A discount that doesn’t exist. An SLA the team can’t meet. A “yes, we offer that” on a feature you don’t. The composer in this system can’t do any of those things, by design. Three Drive docs ground every reply. Four guardrails sit between the model and the send button.

Key takeaways

  • Three Drive docs ground every reply: voice, pricing-and-promos, and ICP-and-disqualifiers.
  • Four guardrails sit between the model and the send button.
  • Citation required: every factual claim must point to a retrieved passage.
  • No fabricated specifics, no commit on availability, no PII in subject lines.
  • Hot leads never auto-send. Warm follow-ups can — but only when explicitly enabled per source-and-intent.

Three docs in, four guardrails out

Composer grounded by three Drive docs, gated by four guardrails A diagram with a left column of three input docs, a centre column with the composer, and a right column of four guardrails before the send button. Left column: three Drive docs labelled "Voice" (warm, brief, never pushy; signature lines per source; opening templates), "Pricing and promos" (your tiers, what's included, the active campaign codes with end dates, things you can and can't commit to in a first-touch reply), and "ICP and disqualifiers" (the kinds of buyers you sell to, the kinds you don't, with the reasons for each). Each doc is mirrored from Drive into S3 every five minutes by a small sync Lambda; a Bedrock Knowledge Base reads from the S3 mirror and produces vector embeddings used by the composer at retrieval time. Centre column: the composer Lambda receives the structured lead and the chosen move (warm draft or hot first-touch), calls Bedrock Retrieve to fetch the top grounded chunks from the Knowledge Base, then calls Bedrock ConverseStream with those chunks plus strict tool_use over four tool definitions answer, draft, escalate, and ignore, and produces the candidate reply with cited passages attached. Right column: four guardrails in series. Guardrail 1 is citation required — every factual claim about pricing, availability, or features must reference a retrieved passage by ID; if the model cites a passage that wasn't in the retrieved set, the reply is downgraded from auto to draft. Guardrail 2 is no fabricated specifics — a regex sweep for prices, percentages, and timeframes that don't appear in the cited passages causes a downgrade. Guardrail 3 is no commit on availability — a small block-list of phrases like "we can have someone on a call tomorrow at 2pm" is rejected; the composer is required to use scheduling-link language instead. Guardrail 4 is no PII in subject lines — outbound subject lines are stripped of any token that looks like an email or a phone number, since notification systems often log subjects in cleartext. Beyond the four guardrails, the candidate reply is written to the dispatch queue. A note at the bottom: a reply that fails any guardrail is downgraded, never sent, and audited. drive (mirrored to s3, embedded) Voice tone, signature, openers Pricing & promos tiers, codes, end dates ICP & disqualifiers who you sell to, who not composer (lambda) Bedrock Retrieve + ConverseStream • Reads structured lead + chosen move • Strict tool_use over 4 tools: answer, draft, escalate, ignore • Retrieves top passages from the Knowledge Base • Produces candidate reply + citation IDs • Hot move → Slack ping draft • Warm move → team draft queue guardrails 1. Citation required every claim → passage ID 2. No fabricated specifics prices, %, timeframes 3. No commit on availability use scheduling link, not time 4. No PII in subject strip emails, phones Dispatch queue passes all 4 → route by move Fail any guardrail → downgrade to draft, audit, never auto-send A reply that fails any guardrail is downgraded, never sent, and audited.
Fig 5. Three docs ground the composer; four guardrails gate the send button. The model is required to cite a retrieved passage for every factual claim — if it can’t, the reply downgrades to a draft for human review.

Three Drive docs ground every reply

Voice is the smallest of the three. A page of how you sound: warm, brief, never pushy. The sentence templates you use to open and close. A signature line per source (a Meta Lead Ads response opens differently than a website-form response). The voice doc is the part most likely to change month to month as you tune what works. Keeping it small means edits are easy, and the model has nowhere to hide if a phrase slipped past you.

Pricing and promos is where the real risk lives. Your tier names. What each one includes. What it doesn’t. The active campaign codes and their end dates (so a January discount doesn’t get quoted in May). And an explicit list of things you can and can’t commit to in a first reply. “We can offer a free 30-day trial” goes here. “We can match any competitor’s price” never goes here, even if you sometimes do, because the bot can’t make case-by-case judgement calls. Edit this doc the moment a campaign ends. The composer reads the update on the next refresh. No deploy.

ICP and disqualifiers is the same doc the move-picker reads. Including it in the composer’s grounding lets the reply gracefully de-escalate when the lead isn’t a fit. Instead of pushing on a sale that won’t happen, the bot can write “we focus on companies in [these industries]; we may not be the best fit, but here’s a partner who might be” from a passage you wrote. Disqualifying a lead politely is a feature the disqualifiers doc enables.

All three docs are mirrored from Drive to an S3 prefix every five minutes by a small sync Lambda. Bedrock Knowledge Bases don’t have a native Drive connector, so the mirror is the bridge. The Knowledge Base reads from S3, chunks each doc by heading, embeds each chunk with Titan Embeddings, and stores the vectors in the managed S3 Vectors index. At composer time, the model retrieves the top passages relevant to the lead before it writes a single word.

Four guardrails between the model and the send button

Guardrail 1 — citation required. The composer runs with strict tool_use. The answer tool requires a citation_passages array referencing one or more retrieved passages by ID. Every factual claim about pricing, availability, or features must point to a passage. If the model emits an answer with a citation that wasn’t in the retrieved set (a hallucinated reference), the runtime downgrades the move from auto-send to draft. That’s the safer-by-default failure mode, not a silent error. The rep gets the proposed reply with a flag “citation verification failed” and decides what to do.

Guardrail 2 — no fabricated specifics. A regex sweep over the candidate reply looks for prices, percentages, and timeframes: “$X,” “X%,” “within X days,” “by tomorrow,” specific dates. Every match is checked against the cited passages. The price has to actually appear in pricing.gdoc. The “within 30 days” SLA has to actually appear in a passage. Anything specific that isn’t in the source downgrades the reply. This catches a class of failures that pure citation enforcement misses, because models occasionally cite a passage and then write a number that isn’t in it.

Guardrail 3 — no commit on availability. A small block-list of phrases like “we can have someone on a call tomorrow at 2pm” or “Friday at noon works” is rejected outright. The bot doesn’t know the team’s actual availability and shouldn’t pretend to. The composer is required to use scheduling-link language instead: “here’s a link to book a time that works for you,” not “does Friday at 2pm work for you?” This guardrail is the one most likely to false-positive on a friendly “happy to chat soon.” The block-list is tuned for specificity, not vagueness — it rejects committed times, not warm intent.

Guardrail 4 — no PII in subject lines. Outbound email subjects get a final pass that strips any token looking like an email address or a phone number. Subject lines often end up logged in cleartext by mail providers, monitoring tools, and Slack notifications. A subject like “Re: lead from jane@example.com” is a quiet PII leak across all of those. The bot uses opaque references (“Re: your inquiry,” “Re: about your demo request”) and the lead’s email address only inside the body.

What the composer doesn’t do

The composer never sends an auto-reply for a hot lead. Hot moves always go to the human first, with the proposed first reply as a draft inside the Slack ping. The bot writes; the human chooses to send. Auto-send is reserved for warm follow-ups, and only where the team has explicitly enabled it for a specific source-and-intent combination. A typical setup might auto-send for warm info-intent leads on the website-form lane — where the message is “send me your pricing sheet” and the right answer is “here’s the pricing page link, and a person will follow up.” Warm quote-intent leads never auto-send. They wait for a human glance.

The composer also never invents a feature, a partnership, a customer logo, or a case study. If the lead asks “do you integrate with HubSpot?” and the pricing doc doesn’t mention HubSpot in the integrations passage, the reply has two options: “great question, let me get the right person to confirm,” or escalate to a human. There is no third option that involves the bot guessing.

What the next post handles

The composer is the last AI step in the pipeline. The next post is a cost breakdown — what all of this actually adds up to per month at the volumes a small business sees, and where the dollars go when the volume grows.

All posts