A CVE triage pipeline that keeps asking, because day one tells you nothing

The pipeline is called Rappaport, after a scientist in Lem’s His Master’s Voice, a novel in which a team of researchers reads meaning into a signal from the stars and slowly learns that the meaning tells them more about themselves than about the signal. That is the failure I kept meeting in the field. A detection stack returns a confident verdict, and the confidence comes from how little it has seen. When partial signal can corrupt the verdict, priority cannot be read off whatever a record claims about its own severity. It has to be computed from signal that actually predicts exploitation, before anything a model could be talked into gets near it.

So I built a CVE triage pipeline on one rule: the language model is the least-trusted component in it. Priority is decided in plain code before the model runs. The model has no tools, no write access, and no authority to change a ranking. It writes the summary and nothing else. The code is open source, on GitHub.

Then I left it running against live feeds, first once a day and then twice, and watched. The architecture held; what I had misjudged was the problem itself. I had assumed the hard part of triage was ranking what sits in front of you, but on the day a CVE is published there is almost nothing in front of you to rank it by. The design problem turned out to be asking again on later runs.

Update, 1 November 2026. The three Splunk figures are now screenshots of the dashboard itself, with Splunk’s own labels. The earlier versions were PDF exports, one with labels added by hand. All three, and every number in the text, stop at 23:00 on 31 October. One more item has reached act-today since then, through the review lane: CVE-2026-120194, on the morning run of 1 November. That makes 22: eighteen through the KEV lane, four through the review lane, still none from the day’s new publications.

Terms used below

Seven questions, one label

Anyone who does vulnerability management for a living knows the questions that arrive with every newly published CVE. Is it one of these?

• critical and trivial to exploit, with no preconditions;
• critical with no public exploit yet, though one may land tomorrow or next month;
• scored critical, but is it really? the number inflated;
• scored low or medium but critical in the deployments that happen to be common;
• critical only in a non-default configuration, which is sometimes the configuration everyone runs;
• an undisclosed zero-day that someone has simply published;
• and, underneath all of it, the question the rest depends on: do we even run the affected thing.

People who do this full time get lost in that list. The person handling security part time at a small company has no chance at all, and the volume only ever grows: more tooling, more scanners, more model-written advisories, all of it adding to the pile without making the decision any easier. The list is seven different questions wearing one label, and most tools answer whichever is cheapest and quietly drop the rest.

That is the problem I wanted to take seriously: which of those seven questions a machine can actually answer, and how to hand the rest to a person on purpose.

What signals can answer, and what they cannot

Here are the seven questions against the signals that actually exist.

Two of those rows have no machine answer: the configuration case and the undisclosed zero-day. Both need a person, so the pipeline flags the item and steps back.

The rest can be computed before anything that reasons in prose is involved. That severity is the wrong axis has held up as a finding for more than a decade. A case-control study of CVSS scores against real-world exploits found that ranking on a high CVSS score alone is not an economically effective patching policy, cutting real risk only marginally. So CVSS enters the ranking only as impact context, scaled by exploitation signal. And the one signal that can lift an item on its own, confirmed exploitation, is read from a catalogue. A model never infers it, because the moment a model is allowed to decide, the text it reads becomes an attack surface.

The model doesn’t get a vote

The thesis of the whole thing is that a prompt is not a control but a wish. The real controls are order and scope: what is computed before the model runs is out of its reach, and what it is never handed, it cannot use.

So priority is decided first, in plain code, and only then does the pipeline decide whether to spend a model on an item at all. Act-today items get a written summary; everything below gets a deterministic one-liner, so the lower tier costs no tokens and offers no attack surface. When the model does run, the prompt has two channels that never mix: trusted facts assembled by the code, and untrusted data copied from the feed, held apart so that a sentence inside a CVE description cannot pose as an instruction. The output is validated against a schema before anything downstream reads it. And if a known-exploited item carries language in its own fields that argues for its own downgrade, it is pulled out of the automated path and handed to a human, with the event logged.

The strongest design decision in the project was a subtraction. Early on, a read-only tool layer let the model look things up; I took it out and moved all bounded agency into Pirx, a separate project with its own threat model and a versioned contract between the two. “The model decides nothing, ever” is a claim you can check by reading one function. “The model decides almost nothing” cannot be checked that way, whatever guardrails surround it.

The last piece is choosing how each part fails, one boundary at a time. The estate index fails open: an unusable lookup can rank an item too high, and the next run corrects it. The ticket-deduplication index fails closed: the cost of getting that one wrong is an irreversible burst of duplicates in someone else’s system of record, so in doubt it does nothing. When a write budget is exceeded the whole batch aborts rather than writing the first few and stopping halfway. Each choice is written down with its reason.

None of this is proof against prompt injection. The guardrails are threat-modelled and mapped to the usual catalogues, not yet proven by a published adversarial evaluation; that is being built, and until it exists the honest word is “threat-modelled”, not “tested”. What the pipeline can prove today is narrower and, to an auditor, more useful. The ranking is deterministic, and every run can print its own derivation rule by rule, including the rules that were evaluated and did not fire.

On day one, there’s nothing to rank by

The literature is clear about what to rank on. Jacobs and colleagues showed the size of the gap: a CVSS 7-and-above threshold flags about 58% of all CVEs, while ranking by the probability of exploitation reaches around 82% of what actually gets exploited at roughly 7% of that effort. I built the ranker to rank by that probability.

The first full day I ran it, one pass on my laptop before any of this went unattended, looked like the paper come to life. One 24-hour window from NVD, 878 freshly published CVEs. A CVSS 7 threshold would have handed an analyst 505 of them. The pipeline handed over one. But that one was promoted by its presence in CISA’s known-exploited catalogue, and its exploitation score played no part in it.

Then I stopped curating days and let it run on a systemd timer in a home lab. Ten unattended days, 3,846 records ranked, and the act-today tier was empty across all ten. Not one record reached the top two priorities. EPSS had not scored nine records in ten, and the highest probability it returned over the whole stretch was 0.0126. The queue that actually formed was a review lane of records with no usable signal, because on the day of publication there was none to carry. It averaged about a hundred records a day and swung from over four hundred on one day to a single record on another. The effort-and-coverage curve from the literature is measured on mature CVEs. An independent five-year evaluation reports the same thing from the other side: score EPSS on the day of publication rather than a month later, and the queue inflates several-fold while its measured efficiency collapses.

That reframed the whole problem, and it exposed three ways the pipeline was quietly losing items. Each one became a lane: on every run the pipeline reads more than the new window, and routes what it cannot yet decide back to a store the next run will ask about again.

The first was a vulnerability scored CVSS 9.4, ranked P3 one morning, entered into KEV the same day, and never seen again, because the window reads by publication date and that item had already been read. CISA adds most entries days after a CVE appears. Barracuda, measuring the catalogue, puts the median gap from publication to a KEV entry at fourteen days in 2026, up from six in 2023. That is a vendor’s figure, but the lag it reports is exactly the failure this lane exists to catch. So the catalogue is now re-read on every run for entries added since the last one, whatever their publication date. The lab’s first run with that lane ranked eight such entries, all P1, in under four seconds.

The second miss was quieter. Two items scored CVSS 9.2 were ranked P3 with EPSS still pending. Days later their EPSS scores were 0.9998 and 1.0000, about as close to certain as that model goes. They carried a CVSS, so the review lane never held them, and nothing re-scored them. Now any P3 item is scored again on every run for fourteen days and reported once if it climbs. In the end, those two items reached P1 through the KEV lane before the watch could lift them.

The third was the estate question, is this even ours, and every way I got it wrong is worth stating. The join runs through package URLs against OSV, an exact key with no lossy name translation. It also read the wrong field for months, so Debian and Ubuntu packages could never register as present until I fixed it. Binary package names match nothing; only source names do. The per-run coverage-gap count measures how much of the window OSV can speak to, not how complete your inventory is, and treating the one as the other is an easy and expensive mistake. And an item is only ever declared absent on the strength of an operator declaring the inventory complete, never inferred from OSV staying silent. Not knowing because a lookup failed is a different fact from not knowing because no source recognises the item. The design weighs them the same, because neither is evidence of absence.

What it costs, and where it fits

Two worries usually come up here: that something this small is just a script, and that something this careful is one more server to keep alive.

The triage runs in seconds. A full day of 878 published CVEs finished in about forty seconds on a laptop, most of it spent waiting on public APIs rather than computing anything. Smaller days finish faster. The only expensive part is optional: the estate index, the layer that answers whether a vulnerability is even yours, takes around seven and a half minutes to build cold. It is cached for eighteen hours, so it rebuilds roughly once a day, and a warm run reads it back in under forty seconds. Switch the estate layer off and that cost leaves with it.

It runs as one Python process and a SQLite file, with no agent, scanner, daemon or database server. It sits after whatever produces your inventory and before whatever cuts your tickets. It emits a digest, a verdict and a versioned record, and takes no action itself.

Is this any good, then

Two answers, because it is two different things depending on what you ask it to be.

As a product that saves you measured hours, it has nothing to show yet. Nobody has timed an analyst against it, and it runs in one lab under one operator.

It is a reference implementation of refusing to let a model decide, and on that one axis it does exactly what it claims and can prove it in a single function. The ranking is deterministic, the model writes prose and nothing else, and both of those are things you check by reading code. The architecture is proven and the payoff is not.

How to disbelieve this in fifteen minutes

Take any twenty-four hour window from the last month, point the pipeline at it, and read three numbers off the compression event: how many CVEs were published, how many it put in the act-today tier, and how many a plain CVSS-7 threshold would have handed you instead. The gap between the last two is the argument, measured on your own data. Then run --explain on any CVE in that window and read the derivation rule by rule, including the rules that were evaluated and did nothing.

One caveat: the demo reads live feeds, so it works only while NVD does.

And if you are the one running it, be clear about the division of labour. It will tell you what is exploited, what is likely to be, and what is probably yours. It will not tell you whether a vulnerability bites only in a configuration you happen to run, and it will not invent a zero-day that has no CVE.

What it doesn’t do

Estate coverage is whatever OSV can speak to and no further, which today leaves most KEV entries without a matching record; broader identity is designed, not built. And the failures that worry me are the quiet ones: a lookup that fails in a way that marks an item absent rather than unknown scores it at 39.6, a notch below the 40 that would have lifted it into view, so the mistake never reaches the digest to be caught. Loud tripwires for exactly that case are the next thing being built.

What comes next

This is the first of three pieces. The second goes back to the data once there is a month of it rather than two weeks: a scorecard against a plain CVSS threshold, how far ahead of CISA the exploitation score moves, where the estate layer promoted items it should have left alone, and how the pipeline compares with the tools people already run. It lands in the second half of November. The third is about process: auditing my own claims, using the lab to test my own assumptions, and the release discipline that ties every number to the command that produced it. That one follows in December.

Update, 3 November 2026. The second part is out, earlier than planned. How far ahead of CISA the exploitation score moves turned out to be answerable from public history, so it did not need to wait for a month of lab data. The harder half of the scorecard, how much of what the pipeline ranked low later turned out to matter, needs that month, and moves to the third.

Back to the signal

Ten days with an empty act-today tier is where this piece started. Here are the two weeks around it, straight from the lab’s Splunk: nine days before the lanes went in, six after.

Since 26 October, 21 items have reached act-today. Eighteen came through the KEV lane, read again because the catalogue moved. The other three came through the review lane: CVEs that carried no signal when they were published, asked about again until a score arrived. Not one came from the day’s new publications. Take away the first run, which swept eight backlog entries the moment the lane switched on, and it is still thirteen to none: every urgent item came from a re-read.

The scientists in Lem’s His Master’s Voice, the novel Rappaport takes its name from, could not read their signal, so they read themselves into it. A CVE feed on day one is the same: there is not enough there yet, and a tool that ranks it anyway is reading its own assumptions. What helps is telling a signal apart from a blank you cannot read yet, and coming back to it.

Day one tells you nothing. Day fourteen tells you whether you were still asking.

Raw data

The close rests on two tables, and both are here as exported from Splunk, so the chart can be checked.

kev_added is the KEV lane and recheck is the review lane. Every KEV-lane row can be checked against the dateAdded field in CISA’s catalogue. CVSS is empty on most KEV-lane rows. That is what the verdict carried, not the CVE’s real score: CVE-2026-109618 and CVE-2026-109619 are the two items from the second miss above, scored 9.2 when they were first ranked. The lane ranks on the catalogue entry alone, so the empty field changes nothing in the ranking. The same rows as JSON: rappaport-act-today-2026-10.json.

One run a day until 25 October, two a day after, none missed. Every run’s verdicts arrived in full except one, at 23:46 on 26 October, whose own count never reached Splunk, so the 334 verdicts that did arrive cannot be checked against it. “unwired” means no model was connected yet; the ranking never depends on it.

The query behind the table, with the index name left out:

sourcetype="cve-digest:verdict" priority IN ("P1","P2")
| stats min(_time) as first_seen earliest(priority) as priority
        earliest(cvss) as cvss earliest(origin) as origin by cve_id
| sort 0 first_seen

Sources

• EPSS v3: Jacobs, Romanosky, Suciu, Edwards, Sarabi, “Enhancing Vulnerability Prioritization: Data-Driven Exploit Predictions with Community-Driven Insights”, 2023, arXiv:2302.14172 (Figure 5). The 58 / 7 / 82 figures are from that paper, not measured here.
• CVSS as a predictor of exploitation: Allodi and Massacci, “My Software has a Vulnerability, Should I Worry?”, arXiv:1301.1275.
• EPSS scored at publication versus later: Paul, “How Good Is EPSS, Really? A Five-Year Empirical Evaluation Correcting for Look-Ahead Bias”, 2025. An independent preprint, not peer-reviewed.
• The KEV lag and its trend: Barracuda, “The KEV gap: how fast do exploited bugs get flagged?”, July 2026. A vendor analysis; the median gap from publication to a KEV entry has risen from six days in 2023 to fourteen in 2026.