AI vulnerability research now has an orchestration problem. One magic model will not carry the work. Security engineering still comes down to plumbing under stress: scoped jobs, state, validation, reachability, and review gates that survive contact with real code.
This is a revisit of Anthropic Published the Missing Manual for AI-Assisted Exploits and Cloudflare Made Agent Sandboxing an Edge Runtime Problem. The new development is specific: Cloudflare has now published a concrete guide to building a vulnerability harness out of its Project Glasswing work. The earlier pieces were about AI exploit capability and agent execution substrate. This one is about the machine that makes those capabilities usable without turning every security team into a helpdesk for hallucinated CVEs.
Cloudflare’s core claim is the right one: “the harness is the bit that lasts.” Models rotate. Context windows grow. Benchmarks get farmed. Vendor mythology mutates every quarter like a startup with a head wound. The durable artifact is the system around the model: how it scopes work, stores state, forks investigations, validates findings, deduplicates root causes, proves reachability, writes fixes, and refuses to merge patches just because a language model wrote something confident.
Unglamorous, yes. Also the whole product.
The obvious failure mode for AI security is asking a model to “find vulnerabilities” in a repo and then pretending the resulting pile of markdown is research. That workflow is seductive because the first demo looks sick. The model names files, invents threat language, quotes code, and emits a finding with a severity label. Then humans discover that half the findings are unreachable, duplicated, stale, based on impossible input paths, or written in that awful LLM tone where “potentially” is doing the work of evidence. Congratulations, you built a bug-report spam cannon.
Cloudflare’s harness exists because the one-shot agent shape collapses under real codebases. Their post describes a vulnerability discovery harness that starts with recon, writes repo-specific architecture notes and threat models, then runs scoped hunt agents for specific bug classes. A hunter is not supposed to wander the whole cathedral sniffing vibes. It gets a narrow assignment, a target surface, and enough context to stay under control. If it sees a promising side path outside scope, it can fork a sibling instead of derailing itself. This is basic agent hygiene, but basic hygiene is exactly what the AI security hype market keeps skipping because choreography does not fit in a keynote slide.
The validation stage is where the thing gets serious. Cloudflare describes a two-pass validation flow: mechanical checks over schema, paths, and evidence, then an adversarial agent trying to disprove the finding. That second model should often be different from the discovery model. Good. If the same statistical priesthood finds the bug and blesses the bug, you have not validated anything. You have built a confidence laundering machine.
The post’s numbers make the point. Cloudflare says better recon dropped initial validation rejection from 40 percent to 11 percent and increased high-integrity findings from 35 percent to 58 percent. Across the lifetime funnel, it reports 20,799 raw candidates, 12,057 surviving initial validation, 5,442 folded as duplicates, and 7,245 final actionable findings. Those are industrial process numbers. The signal comes from compression, filtering, proof, and state, not from model intelligence as theater.
State is the unsexy center. Cloudflare recommends externalizing state into a database keyed by run, repo, and stage, instead of letting long chat sessions become the memory layer. This should be obvious to anyone who has run real systems, but AI tooling discourse keeps rediscovering databases like a cargo cult finds agriculture. Context windows make terrible databases. Agent transcripts make terrible asset inventories. A model remembering something in the middle of a 90,000-token swamp does not give you resumability. Persistent state gives you deduplication, retries, audits, scheduling, and survival after partial failure.
The reachability split matters even more. A code flaw only becomes a vulnerability when attacker-controlled input can reach it in production. That boring boundary question decides whether the finding matters. Cloudflare’s trace stage walks dependency graphs and consumer repos to decide whether a suspected issue is exploitable from a real boundary. Visa makes the same point in its open-sourced Vulnerability Agentic Harness, where it argues for mean time to adapt over mere time to detect: current inventory, remaining attack paths after a release, and proof that a fix works in production.
That is the correct metric shift. Finding more bugs is not automatically progress if the human triage queue becomes the new denial-of-service target. Anthropic’s Project Glasswing update said partners had identified more than 10,000 high- or critical-severity vulnerabilities in the first month and that Mythos scanned over 1,000 open-source projects, surfacing 23,019 issues with 6,202 estimated high or critical. It also named the bottleneck plainly: human capacity to verify, disclose, and patch.
That bottleneck is where the hype becomes dangerous. If AI drops ten thousand findings into the world and maintainers are still patching by hand under disclosure pressure, then the system has not solved software security. It has shifted scarcity from discovery to verification and remediation. The vuln economy does not become safer because both sides get faster at finding sharp objects.
Cloudflare’s automated fixing stage is appropriately constrained. The fixer writes a patch and a unit test, and it needs a fail-to-pass test flip before clearing the gate. It does not merge. A human signs off on the dry run. That is not timidity. That is operational sanity. Model-written fixes can make a unit test green while violating an invariant two call paths away, changing a timeout assumption, breaking a downstream consumer, or turning one exploitable edge into a different one. Anyone demanding autonomous merge-to-prod security patches because “the SLA is two hours” deserves to spend a weekend debugging their own bravery.
The more subtle Cloudflare point is model agnosticism. Use one model for discovery, another for validation, maybe another for fix generation. Treat models as stateless compute engines. Do not build the harness around the emotional support object of this week’s leaderboard. This is the same lesson that keeps appearing in agent infrastructure generally: the durable layer is scheduling, isolation, state, permissions, logs, rollback, and human accountability. The model is a worker inside the operating system.
There is a cynical reading here. Big platforms are turning AI security into another enterprise control plane. Cloudflare, Anthropic, Visa, Microsoft, and the rest all have incentives to make the answer look like a managed harness with dashboards, policy gates, model routing, audit trails, and paid capacity. Fine. That is what serious systems become when the toy phase ends. The alternative is every company running a prompt zoo against production code and filing nonsense into Jira until maintainers start treating every AI-assisted report like spam.
The useful historical analogy is fuzzing. Fuzzers became powerful not because someone said “random input go brrr” and walked away. They became powerful because the ecosystem accumulated harnesses, sanitizers, corpora, minimizers, crash deduplication, coverage guidance, CI integration, reproducers, triage workflows, and bug-class literacy. AI vuln research is entering the same harness era. The model is AFL with a law degree and a tendency to bullshit when unsupervised. You still need the harness.
Cloudflare’s post matters because it moves the conversation away from model mysticism and toward operational shape: narrow agents, adversarial review, proof-of-concept gates, reachability analysis, duplicate collapse, database-backed state, budget caps per repo, sandboxed execution, and human-owned merge decisions.
The future of AI security looks like a swarm of constrained idiots with receipts: narrow workers doing one job, checked by other narrow workers, all feeding a state machine that humans can audit.
Less magical. Better. Magic is what vendors call missing architecture.