{
  "$type": "site.standard.document",
  "bskyPostRef": {
    "cid": "bafyreic62dzqsix6fjfgpxj5oed6v7yobpuh663toq42qpw3jmrerh7goe",
    "uri": "at://did:plc:5opbpi2nomj4y3d5kpwamkrd/app.bsky.feed.post/3mgmbzhqz24j2"
  },
  "coverImage": {
    "$type": "blob",
    "ref": {
      "$link": "bafkreidqh3ayjozf7aag25nso27aslywcb3bq3l2prf4pdyuushz3vluqm"
    },
    "mimeType": "image/jpeg",
    "size": 150846
  },
  "description": "Better models matter, but better harnesses may matter more. The future of AI-assisted security is evidence, validation, and human-guided judgment.\n\nA year ago, a team at Microsoft explored an idea that felt promising but still a little early: using an AI agent to go beyond vulnerability scanning and perform deeper CVE analysis, including generating VEX documents. The goal was not just to detect that a vulnerable package existed somewhere in a dependency tree, but to reason about whether a given ",
  "path": "/from-scanners-to-reasoning-how-llms-and-agent-harnesses-can-improve-code-security/",
  "publishedAt": "2026-03-09T07:03:06.000Z",
  "site": "https://corti.com",
  "tags": [
    "GitHub repo",
    "collaboration between Mozilla and Anthropic",
    "security-hardened open-source projects"
  ],
  "textContent": "_Better models matter, but better harnesses may matter more. The future of AI-assisted security is evidence, validation, and human-guided judgment._\n\nA year ago, a team at Microsoft explored an idea that felt promising but still a little early: using an AI agent to go beyond vulnerability scanning and perform deeper CVE analysis, including generating VEX documents. The goal was not just to detect that a vulnerable package existed somewhere in a dependency tree, but to reason about whether a given vulnerability actually mattered in a specific deployment.\n\nThe results can be found in this GitHub repo.\n\nAt the time, the conclusion was measured and practical. The direction looked right, but the models were not yet reliable enough for high-accuracy security work without heavy expert tuning and close human oversight.\n\nThat assessment made sense then. It looks different now.\n\n## What changed\n\nThe shift is not explained by model quality alone. The real change is that both the foundation models and the harnesses around them have improved materially.\n\nA strong signal came from the recent collaboration between Mozilla and Anthropic. In that work, Claude Opus 4.6 reportedly discovered 22 Firefox vulnerabilities over a two-week period, 14 of them classified as high severity, and Mozilla addressed them in Firefox 148. Just as important as the count was the quality of the output: the reports included reproducible test cases, which made them actionable for engineers rather than merely interesting.\n\nThat matters because Firefox is not an easy codebase to analyze. It is one of the most heavily scrutinized and security-hardened open-source projects on the web, backed by years of fuzzing, static analysis, secure engineering practices, and review. Finding meaningful vulnerabilities there is a higher bar than generating plausible-looking bug reports against toy repositories or lightly maintained projects.\n\nThe implication is significant: AI-assisted security analysis is starting to show value not only on under-defended code, but on mature and deeply examined systems as well.\n\n## Why agent harnesses matter\n\nThe most important lesson here is that this is not simply a story of “the model got smarter.”\n\nThe harness is doing real work.\n\nA strong agent harness gives the model a structured environment in which it can search, test hypotheses, validate findings, and iterate. Instead of producing a one-shot answer, the system operates in a loop:\n\n  1. Form a hypothesis about a vulnerability or unsafe behavior.\n  2. Inspect code, runtime behavior, and surrounding context.\n  3. Run checks or test cases to validate the hypothesis.\n  4. Propose a fix or mitigation.\n  5. Verify that the fix removes the issue without breaking intended functionality.\n\n\n\nThis is a very different pattern from autocomplete or chat-based code assistance. The critical capability is not only generation, but verification.\n\nOne especially interesting idea highlighted in recent work is the use of task verifiers. These are trusted checks that help confirm whether the agent truly found a real bug, whether a proposed patch actually resolves it, and whether the resulting system still behaves correctly. That moves the workflow away from ungrounded suggestion and toward evidence-based engineering.\n\nFor security teams, that distinction is everything.\n\n## From vulnerability scanning to exploitability reasoning\n\nThis also aligns closely with what many teams have been trying to do with VEX and deeper CVE workflows.\n\nA good security workflow cannot stop at “scanner found package X with CVE-Y.”\n\nThat is useful as input, but not as a final decision.\n\nTo assess the real impact of a CVE, teams need answers to questions like these:\n\n  * Is the vulnerable component actually present in the deployed environment?\n  * Is the vulnerable code path reachable in this application?\n  * Are relevant mitigations already in place?\n  * Are the exploit preconditions realistic in this operating context?\n  * Is the component used in a way that makes the vulnerability relevant?\n  * Can we justify a disposition such as **affected** , **not affected** , or **fixed** with defensible evidence?\n\n\n\nThis is exactly where LLMs paired with strong agent harnesses become interesting.\n\nA well-designed harness can help gather dependency data, inspect code paths, review configuration, compare runtime assumptions, look for mitigations, and assemble evidence that supports a VEX decision. Instead of merely reporting a CVE, the system can assist with reasoning about exploitability.\n\nThat is a much higher-value outcome than raw scanner output.\n\n## The emerging workflow\n\nWhat is starting to emerge is a new operating model for secure software engineering.\n\nAgents can help with:\n\n  * wide search across large codebases\n  * triage support for vulnerability findings\n  * hypothesis generation for exploitability\n  * evidence gathering across source, configuration, and build artifacts\n  * patch proposal and remediation suggestions\n  * regression-aware validation of those patches\n  * structured output for downstream security processes such as VEX\n\n\n\nHumans still own the trust boundaries.\n\nThat part does not change. Security engineers remain responsible for final judgment, risk acceptance, production decisions, and the governance around what evidence is sufficient. The human role becomes less about manually doing every first-pass investigation and more about supervising, validating, and making high-consequence decisions.\n\nThat is not replacement. It is amplification.\n\n## Product direction is following the same pattern\n\nWe are also seeing product momentum in this direction. New security-focused coding agents are being framed not as scanner replacements, but as systems that find, validate, and propose fixes while using project context and reducing false-positive noise.\n\nThat framing is important.\n\nSecurity teams do not need more unactionable findings. They already have too many. What they need are workflows that improve signal quality:\n\n  * fewer false positives\n  * stronger reproducibility\n  * better contextual reasoning\n  * clearer remediation guidance\n  * more defensible decisions\n\n\n\nThe value of these systems is not in dumping more alerts on engineers. The value is in helping turn raw findings into validated and prioritized engineering work.\n\n## The caution still matters\n\nNone of this means “trust the bot.”\n\nThat would be the wrong lesson.\n\nSecurity teams have good reasons to be skeptical. AI-generated bug reports have had a mixed track record, and false positives impose real costs on maintainers and responders. Every low-quality report consumes time, attention, and trust. In security, those costs add up quickly.\n\nSo the correct pattern is not autonomous trust. It is bounded autonomy.\n\nUse agents for exploration, synthesis, and validation support. Require evidence. Keep human review in the loop. Build workflows where the system earns trust through reproducibility, verification, and traceability rather than persuasive language.\n\nThat is the difference between a demo and an operational capability.\n\n## Why this matters for VEX\n\nVEX is a particularly strong candidate for this model.\n\nGenerating a high-quality VEX statement is not just a documentation task. It is a reasoning task. It requires the system to connect vulnerability metadata with the reality of a specific codebase and deployment model.\n\nA useful AI-assisted VEX pipeline would need to:\n\n  * ingest CVE and advisory information\n  * map affected components to the actual software bill of materials\n  * inspect source and configuration for feature usage and reachability\n  * identify environment-specific mitigations\n  * evaluate exploit preconditions\n  * collect evidence for the final status\n  * produce a human-reviewable explanation for why the status is justified\n\n\n\nThat is exactly the kind of multi-step workflow where agent harnesses shine.\n\nThe model provides reasoning and synthesis. The harness provides structure, tools, validation, and repeatability. Together, they can reduce manual effort while improving the quality of the final security assessment.\n\n## The real inflection point\n\nTwelve months ago, using agents for deep CVE reasoning and VEX generation felt early but directionally correct.\n\nIn 2026, it looks much more like an emerging operating model.\n\nThe pattern is becoming clearer:\n\n  * let agents perform broad search across code and security context\n  * let them generate and test hypotheses\n  * let them gather evidence and propose patches\n  * let them support exploitability reasoning and VEX authoring\n  * let humans make the final call\n\n\n\nThat is the real story. The interesting development is not just that language models have improved. It is that we are getting better at surrounding them with the right harnesses: verifiers, tools, evidence loops, and guardrails.\n\nThat combination is what makes the output useful for serious security work.\n\n## Final thoughts\n\nFor engineering organizations, this opens up a practical and high-value set of opportunities. The most promising areas are the ones where the cost of manual analysis is high, the amount of contextual reasoning is substantial, and the final decision still benefits from expert human review.\n\nCVE triage, VEX generation, exploitability assessment, and patch validation all fit that model extremely well.\n\nThe opportunity is not to remove security engineers from the loop. It is to give them better leverage.\n\nAnd for teams building internal platforms and developer tooling, this may be one of the most interesting places to invest next: security workflows where LLMs provide breadth and speed, agent harnesses provide discipline and validation, and humans provide judgment.\n\nThat is a much stronger foundation than either models or scanners alone.",
  "title": "From scanners to reasoning: how LLMs and agent harnesses can improve code security",
  "updatedAt": "2026-03-09T07:03:06.513Z"
}