Building kameHunt: LLM-Assisted Variant Analysis for CVE Research

When a vulnerability is found in one project, the same mistake is usually repeated in others. Finding those repeats is called variant analysis, and the bottleneck is not the search - it is the triage. I built kameHunt opens in a new tab to hunt the shape of a known bug across public source code and then validate and rank the matches, so a human reviews a short list of plausible leads instead of a wall of pattern-matcher output.

This article covers the design at the audited project revision opens in a new tab, including the benchmark it ships with, a field test against real published CVEs, and the limits I would work on next.

KameHunt produces unverified leads. It reads publicly available source code only. It never sends a request to a live system, contains no exploitation capability, and never contacts a maintainer. A human verifies every lead by hand before any coordinated disclosure.

Why the triage is the hard part

A lookup of a record by an identifier is only a vulnerability if nothing checks that the caller is allowed to see that record. A route guarded by a login check is only a problem if it also needs an authorization check and does not have one. A plain text search - with grep, or with a broad Semgrep rule - cannot tell those cases apart, so it returns hundreds of matches and most of them are safe.

Sorting the real risks from that noise, by hand, is where the time goes. That sorting is the entire reason kameHunt exists. The detection step is deliberately commodity; the validation step is the product.

How it works

KameHunt runs a match through four stages: acquire, detect, validate, report.

Acquire points the tool at code. The default and safest mode reads local folders you already have. An optional mode uses the GitHub code search API to pull a candidate subset of public code - a biased slice, not an exhaustive sweep, and the tool says so.

Detect casts a wide net. Broad Semgrep patterns find every candidate for a bug class: a database lookup by a request-supplied id, a route with only an authentication middleware, an unsafe deserialization call. High recall is the goal here, not precision.

Validate is the core. Fast textual heuristics score each match on whether the input is attacker-controlled, whether the sink is reachable from a request, and whether an ownership or tenant-scope guard appears to be missing. The heuristics are provenance-aware: a query scoped by an organization id taken from the session is safe, while the same key taken from the request is not a scope at all. Matches that are clearly safe are dropped for free; clearly unsafe ones are kept; the ambiguous middle is handed to an optional language-model pass that reasons about the snippet the way a reviewer would.

Report hands over a ranked, short list. Each item is marked UNVERIFIED, states why it might be exploitable, and gives the exact manual test to confirm it - for example, requesting a resource as one user with an identifier owned by another and checking whether the record comes back.

The measured claim

The reason a tool like this can be trusted is that its central claim is measured, not asserted. kameHunt ships a benchmark that runs the same set of matches through three levels of validation and reports the false-positive rate at each:

false-positive rate:  raw pattern 100%  ->  + heuristics 42%  ->  + model 11%
recall:               100%              ->  95%               ->  100%

The heuristics remove most of the noise on their own. The model pass removes more of it and recovers the small amount of recall the heuristics gave up. The corpus is a deliberately adversarial synthetic set - every safe sample is written to trip the raw pattern - so the number stress-tests the validator rather than describing a real-world base rate.

Tested against real CVEs

A synthetic benchmark is not the same as real code, so I ran kameHunt against seven real, published access-control CVEs at their pre-patch commits. The first version caught none of them: its patterns assumed Express and Mongoose, while the real code used Next.js route handlers, Hono, custom repositories, and raw SQL. Detection never fired.

After broadening detection to those modern shapes, kameHunt caught six of the seven. The seventh is a fail-open tenant check whose query already carries the scope column - the bug is a bad value, not a bad sink, so no pattern can see it without dataflow analysis. I report that miss plainly rather than hiding it behind the headline, and I did not hardcode the repository's own function names to force a seventh catch. Two of the six were caught at a real precision cost that the model pass is meant to absorb.

Using it

Installation is a single dependency step, and a hunt is one command against code you are authorized to review:

pip install -r requirements.txt
kamehunt hunt --pattern authz_idor --repos ./target --out leads.md
kamehunt list-patterns      # the detection library and the CVEs it maps to
kamehunt benchmark          # measure the false-positive reduction yourself

The model pass is optional and uses your own API key. Without it, kameHunt falls back to heuristics - more leads to skim, no key required.

Limits and what is next

KameHunt is a research accelerator, not an oracle. It emits unverified leads, and pattern-based detection tops out around six of seven on real CVEs because some bugs are logic flaws inside a check that already exists, not a bad call the tool can match. The broad patterns that reach the harder cases lower precision, and it is the model stage that is designed to bring that back - so the next work is wiring that stage into the detection loop and proving the precision claim on real code, rather than adding more patterns.