build.allanninal.dev

Part 4 of 7 · Document expiry watcher series ~5 min read

How an alert finds the right person

The watcher picked a move — first alert, reminder, or escalate. Now the dispatch Lambda has to figure out who to send it to, on what channel, at what time of day, and with what context attached. Get any of those wrong and the alert is worse than no alert: a 2am Slack ping, a generic “something is expiring,” a notification to somebody who left the company three months ago. Four small guardrails sit between the move and the actual ping.

Key takeaways

  • Owner resolution: per-item override beats per-category default beats fallback to the configured admin.
  • Slack DMs are the default; email is the fallback if no Slack ID is configured.
  • Quiet hours and holiday calendars defer pings to the next available business hour.
  • Every alert ships with the item, days remaining, renewal cost, link to the source, and an Acknowledge button.
  • Escalation pings the named target instead of (or alongside) the owner; the owner stays in the loop.

Four guardrails on every dispatch

Four guardrails between the watcher's chosen move and the dispatched alert A horizontal flow diagram. On the far left, a "Move chosen" box: the watcher emitted an event with the item, the category, and one of three sending moves — first alert, reminder, or escalate. Four guardrail gates sit in a row to the right, each drawn as a vertical bar. Gate 1: Resolve owner — looks up the per-item owner column first; if blank, falls back to the per-category owner from the rules doc; if still blank, falls back to the configured admin. Returns a Slack member ID if one is set, otherwise an email address. Gate 2: Quiet hours — checks the local time against the rules-doc quiet-hours window (default 6pm to 8am); if outside business hours, defers the dispatch to the next available business minute via EventBridge Scheduler one-off rule. Gate 3: Holiday calendar — checks the date against the configured holiday list; if it's a holiday, defers to the next non-holiday business day. Gate 4: Final compose — formats the alert message from the voice doc template for the chosen move and category, attaches the item context (name, days remaining, renewal cost, contract value, link to the source document), adds an Acknowledge button. After all four gates pass, the alert ships via the Slack incoming webhook for the resolved Slack ID, or via SES outbound for the resolved email. A note at the bottom: every dispatch is logged to DynamoDB so the next tick knows the ping has fired. Move chosen first alert, reminder, or escalate Gate 1 Resolve owner item override? category default? admin fallback? prefer Slack ID, fall back to email Gate 2 Quiet hours local time in business window? if not, defer via Scheduler Gate 3 Holiday calendar date in holiday list? if yes, defer to next business day Gate 4 Compose + context voice template for category + item, days, cost, link, Ack button Dispatch — Slack DM (preferred) or SES outbound email (fallback) incoming webhook for Slack · SES SendRawEmail for inbox every dispatch logged to DDB ew-pings — the next tick won’t duplicate Every gate is a deterministic check — no model calls, no surprise behavior on a Tuesday in April.
Fig 4. Four guardrails between the move and the dispatched alert. Resolve the owner. Honor quiet hours. Skip holidays. Compose with full context. Then ship via Slack or email and log the dispatch so the next tick doesn’t duplicate.

Gate 1: resolve the owner

Three places the dispatch Lambda looks for the owner of an item, in order. First, the registry sheet’s per-item owner_email column — if a row has a specific person assigned, that person owns it regardless of the category default. Second, the per-category default in the rules doc (“all insurance items default to the office manager”). Third, the configured admin fallback — the person who set up the watcher and gets every unowned ping. The fallback should never fire in steady state; if it does, the weekly digest names every item that hit the fallback so the rules doc can be updated.

Once the dispatch knows which person to ping, it looks up their delivery preference. The voice doc maps each owner to a Slack member ID if one is set, otherwise to an email address. Slack is preferred because alerts feel like work-context messages, and a Slack DM with action buttons is more useful than an email link. Email is the fallback so nobody falls through the cracks.

Gate 2: quiet hours

The watcher itself runs at 8am local time, so the first time a move fires it’s already in business hours. But escalations and reminders that result from a tick can fire later in the day. And one-off computed dispatches (the second-window ping that takes effect at the same time as the first) can land outside the configured window.

Gate 2 reads the rules doc’s quiet-hours setting (default 6pm to 8am, configurable per business). If the current local time is in the quiet window, the dispatch creates a one-off EventBridge Scheduler rule that fires at the next business-hour minute and exits without sending. The Scheduler invokes the same dispatch Lambda with the same payload at the deferred time, where Gate 2 will let it through.

Gate 3: holiday calendar

The rules doc lists the holidays you observe — either a static list (“Christmas Day, New Year’s Day, Independence Day...”) or a reference to a Google Calendar that holds them. Gate 3 checks the current local date against that list and, if it’s a configured holiday, defers the dispatch to the next non-holiday business day.

The list is on purpose — the watcher won’t auto-detect a country’s public holidays for you. The failure modes are very different. A holiday you forgot to add fires a ping that lands on a closed laptop. A holiday in the list that’s no longer observed just delays a ping by one business day, which is fine. The trade-off favors keeping the list explicit.

Gate 4: compose with full context, then ship

The voice doc has one Slack message template per category: a short message with placeholders for the item name, days remaining, renewal cost, contract value, and link to the source document. The dispatch Lambda fills the placeholders, attaches an “Acknowledge” button, and ships the message via the Slack incoming webhook. The webhook URL itself lives in Secrets Manager.

For email fallback, the same template is wrapped in a small HTML email with the same fields and a link that, when clicked, hits a Function URL that records the acknowledgment — the email equivalent of the Slack button.

An escalate move adds a second recipient: the escalation target named in the rules doc for that category. The owner is still pinged (the escalation isn’t a substitute for the original owner’s ping — both go out), but the manager now sees it too. The escalate template is slightly different: it includes the previous ping dates and the cumulative days the item has been overdue, so the manager has the audit trail at hand.

Every dispatch — Slack or email, owner or escalate — writes a row to ew-pings in DynamoDB. The next day’s tick reads that row and knows not to ping the same window again.

Why the guardrails exist

None of these gates are exotic. They’re the kind of small care a thoughtful human would take if they were sending the alerts themselves — check who actually owns this, don’t ping at 11pm, skip the day everyone’s off, include enough context that the recipient doesn’t have to ask a follow-up question. Putting them in code as four small sequential gates makes them part of the design, not a feature you’re trusting the writer of any one alert to remember.

Next post: how an item gets renewed once an owner has acted on the alert — how the watcher captures the new expiry, archives the old chain, and starts a fresh one for next year.

All posts