build.allanninal.dev

Part 3 of 7 · Daily briefing bot series ~5 min read

How the bot picks what matters

Most items in the morning’s haul aren’t worth your time. The ranker’s job is to throw out the noise as cheaply as possible and only pay for AI on the items that survive. Three filters in order, cheapest first.

The ranking pipeline: three filters in order, cheapest first A vertical pipeline. At the top, “Today’s items” — everything the ingestor brought back. An arrow leads down to Filter 1 — rules. To its left, the annotation “free”; to its right, an arrow labeled “drop” pointing off-page for items that fail the rules. Items that pass continue down to Filter 2 — similarity. Left annotation: “a fraction of a cent each”. Filter 2 has two outputs: items that confidently match continue straight down, items that are borderline branch off to the right to Filter 3 — smart AI — with annotation “a tenth of a cent each, only borderline”. Filter 3 either drops or sends items back into the main flow. All keepers converge on the bottom box, “Today’s keepers” — usually three to eight items. A bottom note reads: most items die at the free filter. The smart AI only sees the few borderline cases. Today’s items whatever the ingestor brought Filter 1 · rules duplicates, too-short, wrong language free drop Filter 2 · similarity does it sound like your topic? cheap embed confident keep borderline Filter 3 · smart AI final call on borderline items small cost judged keep Today’s keepers usually 3–8 items Most items die at the free filter. The smart AI only sees the few borderline cases.
Fig 3. The ranker: three filters in order. Cheap rules first; smart AI last; most items never reach it.

Three filters, cheapest first

If you handed every fetched item to a smart AI, your bill would explode — and most of the AI’s time would be spent looking at things that obviously don’t fit. So the ranker runs three filters in order. An item only moves to the next filter if it survived the previous one.

Filter 1 — is this even worth looking at?

Free, runs in milliseconds. Throws out:

  • Items already in your digest archive (duplicate)
  • Items too short to carry meaning (a tweet-length blurb with no body)
  • Items in the wrong language for your topic
  • Items from sources you’ve quietly muted

This usually clears out roughly half the morning’s haul without any AI involved.

Filter 2 — does it sound like your topic?

A small AI converts both your topic description and each item into a list of numbers. The closer the lists, the more related the item is to your topic.

Items well above the threshold pass through as confident keepers. Items well below it are dropped. Items in the borderline middle get one more look.

The cost is a fraction of a cent per item, even on a heavy news morning.

Filter 3 — is it actually relevant?

For the borderline items only, a smarter AI reads the title and summary and makes the final call. It returns one of:

  • Keep — relevant, include in digest.
  • Skip — not actually a fit on closer reading.
  • Skip but explain — borderline, log the reason so you can tune your topic later.

Only borderline items hit this filter. On a typical morning that’s a handful, not a hundred — which is the entire point.

What survives

Whatever made it through becomes the digest — usually three to eight items, occasionally zero. Zero is fine. The next post explains why empty digests are a feature, not a bug.

In plain words

Most items die at the free filter. A small AI sorts most of the rest. The smart AI only sees the few borderline cases. The whole pipeline costs pennies a day, even when the news cycle is loud. Cheap rules first, smart AI last — the same shape used by serious content guardrails everywhere, applied to a different job.

All posts