---
type: annotated-source-index
title: "Conference evidence on the cost of vulnerability discovery"
researched_at: 2026-10-17
scope: "Discovery, verification, exploit development, maintenance, and the costs of receiving findings"
status: "Primary materials inspected; boundaries and date precision recorded per entry"
---

# Conference evidence on the cost of vulnerability discovery

This index supplements the site's pricing evidence with evidence about production costs. A bounty payment, compute bill, researcher's time, validated report, working exploit, and maintained chain describe different economic objects. They must not become interchangeable points on a single cost curve.

Dates below describe the event or conference publication, not a search engine's crawl date. Slide and PDF page locators are one-based. No new third-party archive captures were verified in this pass; the links below are live source pointers, not invented archival records. Short annotations paraphrase the inspected sources; no transcripts are reproduced.

The index covers **15 conference contributions**, including the supplied Binchomp presentation and one later article adapted from a talk. Data identifiers refer to [the typed evidence corpus](../../data/discovery-cost-evidence-2026-10.json); the complementary [recent AI source notes](RECENT-AI-SOURCES.md) cover five preprints, maintainer reports, selection and rejected leads. Conference and company results are first-party reports; inspecting their materials is not independent replication.

The supplied Binchomp presentation (`DC01`) is documented separately in [Valentina Palmiotti — Binchomp N-Day Pipeline](../../sources/2026-valentina-palmiotti-binchomp.md). Its n-day starting conditions must remain separate from unconstrained discovery of an unknown vulnerability.

## Foundational talks

### F1. Thomas Dullien / Halvar Flake — Repeated vs. single-round games in security (`DC06`)

- **Event/date:** BSides Zurich, **2017**; exact day not verified. The speaker's catalogue uses the alternate title *Repeated vs. single-shot games in security*; the deck title uses *single-round*.
- **Evidence type:** conceptual cost model and practitioner estimate.
- **Sources:** [speaker catalogue](https://thomasdullien.github.io/about/index.html), [speaker-linked slide deck](https://docs.google.com/presentation/d/1y_R6Lkby-10LIIzFMnF4wqzqPWD-eh5RemWy31y4UjE/edit).
- **Inspected locators:** PDF slides 7–8, 14–15, 19–21, 29–33.

Discovery and exploitation have separate marginal costs. Browser research can require substantial initial learning; slide 14 estimates one to four months of ramp-up. Knowledge and techniques can then be reused, while scarcity can eventually increase the difficulty of additional discoveries. Entry barriers can change which researchers participate. Slide 15 explicitly describes the plots as conceptual expected-cost illustrations.

**Establishes:** a reason to separate initial investment from subsequent work. **Excludes:** a measured population cost curve, universal falling costs, or dollars per bug. The time estimate is practitioner judgment, not a benchmark.

### F2. Mark Dowd — Security Technology Arms Race 2021 – Medal Event (`DC07`)

- **Event/date:** HITBSecConf Singapore, **2021-08-26**, 12:00–13:00 SGT; virtual event.
- **Evidence type:** practitioner cost decomposition and retrospective anecdote.
- **Sources:** [official speaker/event record](https://archive.conference.hitb.org/hitbsecconf2021sin/speakers/mark-dowd/), [official slide PDF](https://conference.hitb.org/files/hitbsecconf2021sin/materials/KEYNOTE%201%20-%20Security%20Technology%20Arms%20Race%20-%20Mark%20Dowd.pdf).
- **Inspected locators:** printed slides 18–25, corresponding to PDF pages 19–26.

Dowd separates finding a flaw, developing a reliable exploit, and maintaining operation. A junior researcher's Chrome example around 2013 describes six months without a finding, then four bugs in two weeks; an initial exploit strategy took two months, with subsequent exploits taking a week each. He also describes defenses that defeat technique reuse and isolation that adds discovery and development requirements to each chain link.

**Establishes:** learning effects coexist with recurring conversion and maintenance costs. **Excludes:** representative discovery times or a market-wide productivity trend. The Chrome example is anecdotal. The talk's broad stockpiling claim remains an attributed opinion, not an established result.

### F3. Marcel Böhme and Brandon Falk — Fuzzing: On the Exponential Cost of Vulnerability Discovery (`DC08`)

- **Event/date:** ESEC/FSE **2020**, conference paper dated **November 8–13, 2020**. The official programme includes the presentation; a later ESEC/FSE 2022 presentation is separately listed for November 16, 2022.
- **Evidence type:** empirical compute scaling, probabilistic analysis, and simulation.
- **Sources:** [author paper](https://mboehme.github.io/paper/FSE20.EmpiricalLaw.pdf), [2020 official programme](https://2020.esec-fse.org/room/esecfse-2020-venue-virtual-room-1), [2022 presentation listing](https://2022.esec-fse.org/details/fse-2022-esec-fse-2020/23/Fuzzing-On-the-Exponential-Cost-of-Vulnerability-Discovery).
- **Inspected locators:** PDF page 1, Figure 1 and cost definition; page 10, §6 and Figure 12.

The study distinguishes reaching the same findings faster from finding additional bugs within a fixed time budget. Its experiments cover several program cohorts and over four CPU years; cohort counts are not assumed to be disjoint. Cost is machine-equivalent input-generation capacity. The initial model assumes no synchronization overhead; later discussion examines that limitation.

**Establishes:** the tested fuzzing configurations exhibit diminishing returns at the discovery frontier even when repeat discovery accelerates. **Excludes:** total research labor, dollar cost, exploitability of every crash, or a universal scaling law for AI discovery. Known benchmark bugs and newly discovered real-world vulnerabilities are not interchangeable outcomes.

### F4. Katie Moussouris and Michael Siegel — The Wolves of Vuln Street: The 1st System Dynamics Model of the 0day Market (`DC09`)

- **Event/date:** RSA Conference, Moscone Center, **2015-04-21**, confirmed by MIT's research index.
- **Evidence type:** system-dynamics cost/capacity model; policy simulations.
- **Sources:** [MIT research/event index](https://cams.mit.edu/search-demo/), [MIT-hosted slides](https://cams.mit.edu/wp-content/uploads/2017/12/The-Wolves-of-Vuln-Street-The-1st-System-Dynamics-Model-of-the-0day-Market.pdf), [speaker's accompanying explanation](https://www.hackerone.com/blog/wolves-vuln-street-first-system-dynamics-model-0day-market).
- **Inspected locators:** slides 10, 13–27, 30, and 32.

The model connects undiscovered vulnerabilities, discovery capacity, offensive stockpiles, and patching. It considers how shared techniques correlate discoveries and proposes incentives for reusable tools alongside bounties. Slides 25–26 give 9% and 0.8% discovery overlap under specified model parameters for young and hardened software.

**Establishes:** a mechanism for investments and incentives beyond individual bounty prices to affect discovery. **Excludes:** observed rediscovery percentages, actual market clearing prices, or measured cost per finding. The overlap figures are simulation outputs and should not be graphed as empirical rates.

### F5. Matthew Finifter, Devdatta Akhawe, and David Wagner — An Empirical Study of Vulnerability Rewards Programs (`DC10`)

- **Event/date:** 22nd USENIX Security Symposium, Washington, D.C., **August 14–16, 2013**; exact presentation day not reverified here.
- **Evidence type:** observed vendor bounty payments and comparative vendor cost analysis.
- **Sources:** [official presentation and recording](https://www.usenix.org/conference/usenixsecurity13/technical-sessions/presentation/finifter), [official paper](https://www.usenix.org/system/files/conference/usenixsecurity13/sec13-paper_finifter.pdf).
- **Inspected locators:** §§3.2–3.4 and Table 1, PDF page 5; §§4.1.5–4.1.6, PDF pages 8–9; §5.2, PDF page 15.

Chrome's $579,605 total comes from reward metadata. Firefox's $570,000 is an **estimate: 190 rewarded bugs × an assumed $3,000**. Section 3.4 notes that 8% of Firefox rewards were below that assumption; the authors bound the resulting error at no more than 8%, rather than supplying exact lower payments. These are historical programme totals, not annual spending. The salary comparison and payout data exclude programme administration, while researchers' unpaid searches remain outside the vendor ledger.

**Establishes:** a vendor can purchase externally produced findings at an observed payout cost different from employing researchers. **Excludes:** the external researchers' total discovery expense, total programme operating cost, current bounty economics, or fully loaded cost per bug. Payout divided by findings is not a production-cost estimate.

### F6. Noura Alomar, Primal Wijesekera, Edward Qiu, and Serge Egelman — “You've Got Your Nice List of Bugs, Now What?” Vulnerability Discovery and Management Processes in the Wild (`DC11`)

- **Event/date:** SOUPS **2020**, **August 2020**; month precision retained because the paper cover and internal conference date span differ.
- **Evidence type:** qualitative interviews with 53 security practitioners.
- **Sources:** [official presentation and video](https://www.usenix.org/conference/soups2020/presentation/alomar), [official slides](https://www.usenix.org/system/files/soups2020-paper158-slides-alomar.pdf), [official paper](https://www.usenix.org/system/files/soups2020-alomar.pdf).
- **Inspected locators:** slides 6–9 and 23; paper §6.1, printed pp. 323–324; §6.2, pp. 324–326; §10, p. 330.

The interviews describe work beyond obtaining a report: validation, reproduction, impact assessment, ownership assignment, and remediation. Participants also discuss institutional knowledge, staffing, funding, trust, and scope constraints. The authors recommend considering those costs before launching a bounty programme.

**Establishes:** organizational and verification work can limit the usefulness of additional discovery. **Excludes:** measured labor hours, dollar cost per report, causal proof that AI moved a bottleneck, or a quantified trend over time. This supports a conceptual workflow and its constraints.

## Recent automation talks

### R1. Qinrun Dai and Yifei Xie — BinWhisper: LLM-Driven Reasoning for Automated Vulnerability Discovery Behind Hall-of-Fame (`DC02`)

- **Event/date:** Black Hat USA, **August 6–7, 2025**; exact session day unresolved.
- **Evidence type:** author-reported model-run cost and elapsed time in a prepared workflow.
- **Source:** [official slide PDF](https://i.blackhat.com/BH-USA-25/Presentations/US-25-Dai-BinWhisper.pdf).
- **Inspected locators:** slide 32, human/system responsibilities; slide 76, cost/time comparison.

The comparison reports approximately $40 and 5h29m for o3, $20 and 2h15m for o4-mini, and $11 and 21h18m for r1. The workflow still assigns target selection and result verification to people and uses prepared binary-analysis infrastructure.

**Establishes:** inference expense and elapsed time differ across models within this particular assisted analysis pipeline. **Excludes:** fully autonomous discovery, total cost of an unknown zero-day, researcher preparation and verification expenses, reliable exploit development, or comparable per-bug rates across unrelated studies. These are model-run observations, not a market price or total production bill.

### R2. Michael D. Brown — Buckle Up, Buttercup: Our Experience Competing in the AI Cyber Challenge (`DC03`)

- **Event/date:** DEF CON 33, AIxCC stage, **2025-08-09**.
- **Evidence type:** team-reported competition results and runtime expenditure.
- **Sources:** [speaker/company slide PDF](https://www.trailofbits.com/documents/DEFCON_AIxCC_Stage_Talk.pdf), [dated team accounting](https://blog.trailofbits.com/2025/08/09/trail-of-bits-buttercup-wins-2nd-place-in-aixcc-challenge/).
- **Inspected locators:** slides 35–36; companion article's cost breakdown.

The presentation reports 28 vulnerabilities, 19 patches, approximately $40,000 expenditure, and 90% accuracy. The accompanying account separates $21,100 for LLMs and $18,500 for compute, totaling $39,600. Its $181 efficiency figure is per competition point, not per vulnerability.

**Establishes:** a runtime bill for a competition system tackling discovery and patching under defined scoring rules. **Excludes:** all engineering investment, an open-ended real-world discovery denominator, or $181 per bug. Seeded competition tasks and patching outputs must remain visible when comparing these results with other workflows.

### R3. Andrew Chin and coauthors — OSS-CRS: Liberating AIxCC Cyber Reasoning Systems for Real-World Open-Source Security (`DC04`)

- **Event/date:** USENIX WOOT, **2026-08-11**.
- **Evidence type:** author-reported live-target experiment with explicit campaign resource limits.
- **Sources:** [official conference record](https://www.usenix.org/conference/woot26/presentation/chin), [official slides](https://www.usenix.org/system/files/woot26-slides-chin.pdf), [official paper](https://www.usenix.org/system/files/woot26-chin.pdf).
- **Inspected locators:** slide 16; paper §§5–6.

The slides report seven bugs, four found via existing harnesses, and under $50 per bug. The paper describes eight projects, seven bugs across six projects, $50 LLM campaign limits, a 24-hour timeout, and a 16-core/64-GB environment. It separately reports several person-days of porting effort, including four for Buttercup and eight for ARTIPHISHELL.

**Establishes:** discovery beyond the original competition setting is possible with bounded inference budgets and reusable infrastructure. **Excludes:** complete per-bug cost accounting or the assumption that harnesses and porting labor are free. The campaign cap, reported per-bug figure, compute allocation, and engineering effort are different measurements.

### R4. Joshua Rogers and Luigino Camastra — Solving All Memory Bugs in Network Security Tooling (`DC05`)

- **Event/date:** OffensiveAICon, **October 2026**; event October 4–7, exact session day unverified. The author's accompanying post is dated October 6.
- **Evidence type:** author-reported candidate-to-reproducer pipeline and subsequent manual security triage.
- **Sources:** [author slide PDF](https://joshua.hu/files/sanitizers/oaic-2026-slides.pdf), [author's methodology and results](https://joshua.hu/lessons-using-sanitizers-oracle-llm-vulnerability-triage).
- **Inspected locators:** slide 2; companion article's reproducer and threat-model assessment.

The pipeline starts with static-analysis candidates: 203 findings become 109 memory candidates and 79 sanitizer crashes. It reports approximately $5 and eight minutes per reproducer, or $400 per repository. Subsequent manual threat-model review invalidated roughly 99% of YARA results; the article separately reports 18 CVEs among 30 Wireshark findings.

**Establishes:** inexpensive automated reproduction does not remove the need to assess security impact. **Excludes:** $5 per validated vulnerability, unrestricted discovery from scratch, or a common denominator across the named projects. A sanitizer crash and an exploitable security issue are separate endpoints.

### R5. Dmitrijs Trizna — Before the CVE: Mining Open-Source Commits for Negative-Day Vulnerabilities (`DC12`)

- **Event/publication:** [un]prompted.au, **2026-09-19**, 15:30 AEST; measured figures below come from the **2026-10-02** adapted article, not an assumed identical conference deck.
- **Evidence type:** first-party retrospective security-patch classification benchmark.
- **Sources:** [official schedule](https://unprompted.au/schedule), [author's adapted article](https://aisle.com/blog/before-the-cve-mining-open-source-commits-for-negative-day-vulnerabilities).
- **Inspected locators:** *Evaluation by Time Travel*, *Do tools help?*, Figures 6–7.

Across 323 commits in 40 windows, the study compares 21 models. On the same corpus, GPT 5.6 Luna reports 90% recall at about $0.001 per commit with a single call, versus 95% at $0.0057 with an agent. List-price API costs omit cache discounts; precision remains unresolved because the surrounding commits are not a sufficiently reliable clean negative set.

**Establishes:** a task-specific cost/recall tradeoff across models and harnesses. **Excludes:** cost per novel vulnerability, validated exploitation, fully loaded expenses, or AI savings against human research. The 39-window timing analysis is a separate population from the 40-window model benchmark.

### R6. Anthony Lai, Alan Ho and Ken Wong — Harness & Hunt: A Hybrid, Role-Driven Agent Architecture for Autonomous Vulnerability Research (`DC16`)

- **Event/date:** [un]prompted.au, **2026-09-18**, 15:45 AEST.
- **Evidence type:** first-party conference abstract; full slides or paper not inspected.
- **Source/locator:** [official schedule](https://unprompted.au/schedule), Day 1 *Harness & Hunt* abstract.

A three-month, two-model study adjudicates 630 candidate findings. On one static target, the more expensive model produces 285 findings with 99 false positives; the cheaper model produces 169 with 39. Those imply approximately 65.3% and 76.9% precision. Human code review remains material; the abstract also reports automated-judge drift and silent pipeline failures.

**Establishes:** the observed candidate/verification gap and a same-target precision comparison. **Excludes:** recall, a dollar ratio, unique CVE counts, reviewer-hour savings or an all-in cost estimate. Model names, monetary spending and complete experimental detail are unavailable in the inspected abstract.

### R7. Ta-Lun Yen — One Percent of the Tokens, All of the Strategy: LLM-Assisted Vulnerability Discovery in IoT and Embedded Firmware (`DC17`)

- **Event/date:** Black Hat USA, **2026-08-06**, 11:05 Pacific.
- **Evidence type:** first-party fieldwork and small within-workflow ablation.
- **Sources:** [official session](https://blackhat.com/us-26/briefings/schedule/#one-percent-of-the-tokens-all-of-the-strategy-llm-assisted-vulnerability-discovery-in-iot-and-embedded-firmware-53075), [REV-01 deck](https://i.blackhat.com/BH-USA-26/Presentations/USA-26-Yen-LLM-Assisted-Vulnerability-REV-01.pdf).
- **Inspected locators:** PDF page 18, Device B funnel; page 42, completed ablation table. Page 41 is an incomplete animation build.

The Device B engagement reports 156 findings, 50 reports, 25 viable reports and two critical reports over three engagement days. Separately, reseeding context plus a capability graph changes complete-chain identification from 0/3 to 3/3 runs on the same existing corpus.

**Establishes:** filtering work and sensitivity to workflow design. **Excludes:** vendor acceptance of all viable reports, new discovery in the ablation, executed-chain reliability, or measured monetary savings. The title is not evidence of a 99% cost reduction; engagement days are not measured person-hours.

### R8. Bar Lanyado and Eliya Cohen — Cost-Effective, Private, Frontier-Grade: AI Agent Exploitation with a Fine-Tuned OSS Model (`DC18`)

- **Event/date:** Black Hat USA, **2026-08-06**, 10:15 Pacific.
- **Evidence type:** first-party AI-agent security benchmark; adjacent to conventional vulnerability discovery.
- **Sources:** [official session](https://blackhat.com/us-26/briefings/schedule/#cost-effective-private-frontier-grade-ai-agent-exploitation-with-a-fine-tuned-oss-model-52759), [AgentBreaker deck](https://i.blackhat.com/BH-USA-26/Presentations/AgentBreaker-BlackHat.pdf).
- **Inspected locators:** PDF page 5, cost averages; pages 8/58, performance; page 32, metrics/testbed; page 51, training.

The deck reports average scan costs of $0.007 for a 30B open model and $0.69 for frontier models. The shared testbed contains 30 victim agents across 15 scenarios and two security levels. AgentBreaker achieves 66.1% ESR versus 52.4% for base Nano and 68.2% for Opus 4.5; ESR counts verified exploited tools over vulnerable tools, not successful attempts or compromised agents.

**Establishes:** a specialized model's reported execution-cost/performance tradeoff. **Excludes:** conventional zero-day discovery cost, all-in economics or a human baseline. Training/data generation and hardware accounting are incomplete. The older abstract's 56% result differs from the deck, and headline cost ranges differ between slides; use the stated averages without blending versions.

## Relationship to existing sources

[Dowd's OffensiveCon 2022 keynote](../../sources/2022-mark-dowd-offensivecon.md) is already part of the repository's source spine. That existing pointer was read for context; its metadata and interpretation were not reverified or changed in this pass. F2 is a separate 2021 talk with directly inspected cost-specific slides.

For visualizations, preserve each observation's starting condition and endpoint: unknown flaw, disclosed target, crash, validated vulnerability, exploit, chain, or sustained operation. F1–F2 motivate separating learning from repeat work; F3 addresses compute scaling; F4 models capacity and incentives; F5 measures vendor payments; F6 describes downstream organizational work. Their evidence types complement one another but do not form one comparable time series.