build.allanninal.dev

Part 5 of 7 · Content moderator series ~5 min read

How the moderator makes the final call

A review card lands in Sam’s Slack at 8:03am. A held comment, a reason, the exact rule it may break, three buttons. What happens when Sam taps one? The honest answer is “it depends which button.” This post walks through the three things a moderator can do — publish, remove, or edit-and-publish — how the item, the queue, and the audit trail stay in sync, and how a moderator’s correction teaches the system the better call for next time.

Key takeaways

  • Three actions per card: publish (release a held item), remove (take it down), edit-and-publish.
  • Removal is the only action that takes content down — and it is always a human’s call.
  • Each action updates the item, clears it from the queue, and writes an audit row.
  • An overturn — publishing a held item, or removing a passed one — becomes a worked example.
  • Those examples are fed back into the model prompt so the next similar item gets the better call.

Three actions on the review card

Three actions on the review card A diagram showing one input on the left flowing through an interactive Slack review card, then branching into three action paths. Far left: a "Card in Slack DM" box showing a typical card — the held item's text, the verdict, the confidence, the rule cited, a link to source — with three Slack-button placeholders below: Publish, Remove, Edit. The moderator taps one button. The middle column shows the three branches. Branch one, Publish: a Function URL Lambda releases the held item so it goes public on the platform via the platform API, clears it from the cm-queue, and writes a published event to the audit trail; if the system had held or flagged the item, this counts as an overturn and is saved as a worked example. Branch two, Remove: the Function URL Lambda takes the item down via the platform API, sends the poster the polite house reply from the voice doc explaining which rule it broke, clears the queue entry, and writes a removed event with the rule and the moderator's note. Branch three, Edit: opens a Slack modal with the item text pre-filled so the moderator can trim a link or a slur and post the cleaned version; on save, the edited text is published and both the original and the edit are kept. The right side shows the convergence: every action writes a row to the cm-audit DynamoDB table with timestamp, item id, action, by-user, the rule, and a before-and-after snapshot, and any overturn is appended to the worked-examples set the checker loads next time. A note at the bottom: removal is the only action that takes content down — and it is always a person's decision, never the system's. Card in Slack DM item, rule, confidence [Publish] [Remove] [Edit] Action 1 Publish • Release the held item live via platform API • Clear the queue entry • Overturn? save example Action 2 Remove • Take it down via API (a human’s call) • Send house reply with the rule it broke Action 3 Edit • Modal: trim a link or a slur, then post • Keep original and the edited version Audit trail DynamoDB cm-audit timestamp · item_id action · by-user · rule + worked examples Removal is the only action that takes content down — and it is always a person’s decision.
Fig 5. Three actions per card, three different effects. Publish releases a held item. Remove takes it down with the reason logged. Edit-and-publish cleans and posts. Every action writes to the audit trail, and every overturn becomes a worked example.

Action 1: publish (the “this is fine”)

Sam reads the held comment — “this is the dumbest thing I’ve read all week” — and decides it’s rude but within the line. Sam taps Publish. The button submits to a Function URL Lambda. Three things happen, in order. First, the held item is released so it goes public on the platform through the platform’s API. Second, the item is cleared from the cm-queue so it won’t resurface in a later digest. Third, a published row is written to cm-audit with Sam, the timestamp, and the rule the checker had cited.

Because the system had held this item and a human published it, this counts as an overturn — the system was more cautious than the house actually wants. That correction is saved (more on that below), so the next comment that reads like this one is more likely to pass on its own.

Action 2: remove (the only way content comes down)

The spam comment with the blocked link is the easy case. Sam taps Remove. The Function URL Lambda takes the item down through the platform API, sends the poster the polite house reply from the voice doc — “Your comment was removed because it broke our rule on promotional links from unknown sites” — clears the queue entry, and writes a removed row to cm-audit with the rule and any note Sam added.

This is the one action in the whole system that actually takes content down, and it is always a person tapping the button. The system never reaches this state on its own; the strongest thing it can do unattended is hold an item out of view, which is reversible. Removal is not reversible in the same way, so it stays a human decision — with the reason recorded, so anyone asking “why did this come down?” six months later gets a straight answer.

Action 3: edit-and-publish (the salvage)

Some items are mostly fine with one bad part — a genuinely useful comment that happens to end with a spam link, or a good point wrapped around one slur. Removing the whole thing loses the good part; publishing it as-is breaks a rule. Edit opens a Slack modal with the item text pre-filled. Sam trims the link or the slur, hits Save, and the cleaned version is published. Both the original and the edited text are kept — the original in S3, the edit in the item record — so the change is never a mystery and can be undone.

Edit is used sparingly and only on your own surfaces; on platforms that don’t allow editing a member’s words, this button is simply hidden and the moderator chooses publish or remove instead.

Every action is logged, every overturn teaches

The cm-audit table records every publish, remove, and edit with the moderator, the timestamp, the rule, and a before-and-after snapshot. That alone makes the system auditable. But the audit trail does double duty: whenever a moderator overturns a call — publishes something the system held, or removes something it had passed — that item, the verdict the system gave, and the human’s decision are appended to a small set of worked examples.

Those examples are exactly what the checker in Part 3 loads alongside the house rules and feeds into the model prompt. So the learning loop is simple and contained: the model doesn’t retrain, and the rules doc stays the source of truth, but the next borderline item that looks like a past correction gets nudged toward the call your moderators actually made. The set is capped and curated — the most recent and most representative overturns per area — so it sharpens the edges without ballooning the prompt or letting one odd call skew everything.

Next post: the cost breakdown. The whole pipeline above runs in coffee-money territory at SMB volume; Part 6 explains exactly where the dollars go and why the cheap rule pass keeps the model bill small.

All posts