Trickle.news

Problem

Every week, US agencies publish rules that change what people pay, what they’re allowed to do, and what treatment they can get. Some become front-page stories; many are never covered at all. Which ones fall through, and is there a pattern?

Answering that honestly needs two things that are easy to fake: a judgment of what matters made before anyone knows what the news did with it, and a coverage count you can trust, including its misses.

Approach

• One pipeline, a second product. Trickle.news grew out of a private research pipeline that reads public signals. Labelling regulatory signals kept turning up rules that were real news but not business opportunities, so the same sources, ledger, and decision cascade now feed a public digest with its own question: would the people affected want to know this?
• Models only where they earn it. Code rules decide about half the items, the cheap judge answers narrow questions about the rest, and Claude writes the summary. Code, not a model, supplies every date and every proposed-or-final status.
• Check the checker. On October 9 a fact-checker held back 19 of 61 leads, nearly all false positives: “not yet peer-reviewed” read as “peer-reviewed,” a date phrase read as a name. A comparison of two writer models looked like a model problem until the checker was fixed; then the smaller, cheaper model held back two leads and wrote just as accurately.
• Measure the gap, then measure the measurement. Each published rule gets a news-coverage count from Media Cloud’s open archive, counts only. Zero counts are tested, not trusted. The first independent recall check caught 7 of 12 covered rules, and the causes were specific: the window opened on the publication date though agencies announce days earlier, and official agency names aren’t how reporters write. The revised method opens a week early and searches names the way reporters do (“HHS”). The rerun on 58 rules lifted recall to 60% and precision from 58% to 71%. What’s left was mostly formal titles no reporter would type, so a third method searches the subject words reporters use, requires the agency, and lets a companion rule share its main rule’s coverage. Its recall check is next. Research coverage is now counted too, labelled experimental until a recall check measures it.
• Show every step, including the misses. The public method page lists every step’s accuracy against its target, re-scored weekly on hand-labelled sets, with the metrics still below target marked as such; a weekly job opens an issue when one slips. A run dashboard shows every job’s counts from source to lead.
• Protect people. Officials and authors can be named in their public role; private individuals, minors, and contact details never are, and a test caught a name leaking through one register’s scrub.