Google Open Sourced Mantis: Security Review Skills For Coding Agents, With Warnings Attached

AI code scanners have a credibility problem, and Google says so in the opening argument of its own release. Language models trained to recognise the shape of a vulnerability class are good at flagging code that resembles a bug, and bad at confirming that the code path is reachable, exploitable or even real. The result is a flood of plausible findings that costs security teams more attention than it saves, and it erodes trust in the whole category.

Mantis, published as an open-source project under Google's GitHub organisation, is a different shape of answer: a set of security review skills plus an ADK reference harness that walks a coding agent through the vulnerability lifecycle, from threat modelling and hypothesis building through triage, reproduction, chaining, patching and calibration. It is Apache-2.0 licensed, explicitly a starting point rather than a product, and it arrives with two warnings in capital letters that are as informative as the feature list. This article works from the repository, read in full, and covers what the toolkit actually does, how the harness coordinates it, and why the cautions matter more than the capabilities.

What Mantis is, precisely

The repository describes Mantis as a set of skills along with an ADK reference harness for building secure software in the current era of AI-assisted development. Two halves, doing different jobs.

The Mantis skills are modular, stack-agnostic instructions for security review: the knowledge and procedure a coding agent needs to perform a specific kind of work. Google notes they can be adapted to specialised domains such as hardware and RTL, infrastructure as code, machine learning pipelines or compiled firmware, which is the tell that these are procedures rather than prompt snippets. The Mantis harness is a reference implementation on Google's Agent Development Kit, with automated configuration and launcher tooling, a workflow definition showing how a deep review runs, and a research graph synthesis mode that designs a new agent topology for a stated objective.

The positioning is deliberately modest, and the modesty is load-bearing: a starting point, adaptable and tunable to an organisation's own stack, convertible to a framework of choice by any competent coding agent. What Google is shipping is a well-specified pattern for agentic security work, plus one working instance of it.

The loop, stage by stage

The repository lists the intended sequence for a Mantis campaign, and reading it as a pipeline is the clearest way to understand the design.

Historical review. Read the codebase history for vulnerabilities the project has already fixed, so the agent does not rediscover old bugs or, worse, reintroduce them. That step is unusual and sensible: a project's own patch history is a map of where its developers have been careless before.

Semantic indexing. Build an index and summaries of the codebase so the agent can navigate it efficiently rather than rereading files until the context window fills.

Threat modelling. Generate a threat model automatically, which gives every later hypothesis a scope and a rationale instead of an open-ended hunt.

Hypothesis generation. Build a set of hypotheses for individual agents to research, or scan every file when the target is small enough that breadth beats prioritisation.

Execution and deduplication. Run the research plans and collapse duplicate findings, which is the least glamorous step and the one that determines whether a human can read the output.

Triage and critique. Critique findings specifically to combat hallucination and to statically verify that a reported issue could plausibly exist in production. This is the step that addresses Google's stated motivation directly.

Reproduction. Attempt to demonstrate the vulnerability to varying degrees depending on available environments: a static guess at what an exploit would look like, a unit test, or a mock server stood up to attempt the exploit for real. The gradient matters, because it lets a team trade infrastructure cost against confidence.

Chaining. Look across known findings to build more impactful chains, which is how real incidents escalate and how single-issue scanners miss the story.

Patching with adversarial verification. Fix the vulnerability, then run an adversarial loop to verify the fix actually closes it, rather than trusting that a plausible diff resolved the issue.

Calibration. Score every finding against an established rubric specifically to combat severity inflation, on the stated goal of surfacing the most critical risks to humans instead of producing thousands of criticals. Anyone who has triaged scanner output knows exactly which failure this targets.

Reflection, and advise. Reflect on each completed round to capture what was learned, and use a dedicated advise skill during development so the same mistakes and removed guards are not reintroduced in the next change.

That final feedback into authoring, rather than only reviewing, is the most interesting structural idea in Mantis. Most security tooling is a gate at the end of the pipeline; Mantis's advise skill is an attempt to move some of the learning to the point where code is written.

The harness, and what research graph synthesis does

The Mantis reference harness runs on ADK and ships with a workflow definition showing how a deep review is composed. The configuration story is unusually practical for a research release: an install script, an automatic configuration mode with capability detection, a preflight validation that also probes live reachability in about a second, and a launcher that takes a file or a directory.

Two Mantis invocation options reveal the intended range. A focus flag lets an operator aim the campaign in plain language, for example asking for IDOR review specifically. An objective flag triggers research graph synthesis, in which the harness designs a new agent graph topology for the stated goal, such as auditing webhook handlers for server-side request forgery. The second is the more ambitious feature: rather than running a fixed pipeline, the system composes a review strategy matching the objective, which is the same idea this site covered in the agent orchestration survey and in the harness economics work, applied to security review rather than to general task execution.

There is also a budget mechanism: spend limits by default, with a flag to remove them for long unattended runs and a parallelism setting. Naming a budget as a first-class concept is a small sign of operational maturity, because unattended agentic review is exactly the workload where cost runs away quietly.

What a first campaign looks like

The repository's own quickstart is worth walking through, because it shows how much of the operational burden Mantis removes and how much it deliberately leaves to the operator.

Installation is a script in the reference directory, following a system package for the Python virtual environment. Configuration has two modes: an automatic path that detects capabilities, or an interactive wizard for operators who want to see the choices. A preflight step then validates the configuration and probes live reachability in roughly a second, which is the sort of detail that separates a harness someone will actually use from one that fails ten minutes into a run with a permissions error.

From there, a campaign takes a path: a single file or a directory. An optional focus flag expresses the hunt in plain language, which is how an operator translates a security priority, such as a specific access-control class, into Mantis's plan. An optional objective flag goes further and asks Mantis to synthesise a research graph topology for that objective, which is where the harness stops being a pipeline and becomes a small strategy generator. Budget limits apply by default, and removing them is an explicit choice with a parallelism setting, so an unattended overnight run is possible without being accidental.

The operator's remaining responsibilities are the ones that cannot be automated: choosing a target that is safe to attack, providing an isolated environment, and staffing the human verification that Mantis's own documentation requires before anything is reported. A first campaign run in shadow mode against a codebase with known issues is the sensible smoke test, because it produces both a capability check and a calibration check in one exercise.

The warnings are the most important part

Two blocks at the top of the repository deserve more attention than any feature.

The first is a caution: use at your own risk, be extremely careful, because the suite is designed to generate and execute autonomously generated code that may be unstable or perform unexpected actions. Run it only in isolated, restricted environments, and never on a machine with access to production systems, sensitive data or internal networks. Google then goes further than most vendors would, recommending that teams using the most capable frontier models add a second sandboxing layer with strong monitoring for escape attempts.

That advice is the correct reading of the threat model. A security agent asked to reproduce vulnerabilities will write code, run it, and attempt exploits; an agent doing that on a machine with credentials is a liability by design, not by accident. The same isolation logic runs through the OpenCode MCP flaw and the Unsloth Studio research: the tool that executes code owns the blast radius.

The second is a responsible-use notice, and it is the most socially valuable sentence in the release. AI models are non-deterministic and can hallucinate findings or generate incorrect patches, so all findings must be manually verified by a security expert before being reported, and unverified AI-generated reports must not be mass-filed to open-source maintainers. Google also notes that a failure to reproduce does not definitively mean a false positive, and a successful reproducer does not guarantee exploitability in every context.

That caveat matters because the failure mode already exists in the wild. Maintainers of open-source projects have spent the past two years absorbing machine-generated vulnerability reports that waste hours and occasionally pressure them into publishing advisories for non-issues. A toolkit that ships with an explicit instruction not to do that, plus a calibration stage designed to reduce the number of criticals it emits, is at least aimed at the right problem. This site argued the same point from the maintainer's side in the Rank Math backdoor analysis: tooling that produces findings faster than humans can verify them does not improve security, it just moves the queue.

Where Mantis sits relative to static analysis

The instructive comparison is not against other AI scanners but against the deterministic tools security teams already trust, because the two answer different questions.

A static analysis tool encodes rules, dataflow and type information, and its findings are reproducible by construction: run it twice, get the same list. Its weakness is coverage of semantic bugs, business-logic flaws and cross-component issues that no rule expresses well, which is precisely the space where an agent with a threat model can contribute. Mantis occupies that space, and its calibration stage exists because the space is also where confident nonsense lives.

The sensible composition, and the one the release's framing implies, is layered. Deterministic tooling supplies a floor of reproducible findings, Mantis adds hypothesis-driven review of the semantic cases, and human experts verify before anything leaves the building. What Mantis should not become is the sole gate: an agent that hallucinates a vulnerability wastes an hour, while an agent that hallucinates a patch and is trusted can remove a guard that was doing real work, which is exactly the failure the advise skill is designed to prevent in later development.

That is also why the adversarial patch-verification stage deserves attention disproportionate to its one-line description. Verifying that a fix actually closes the reported issue is the step that separates a review assistant from a change generator, and it is the step most likely to fail quietly, because a plausible diff looks the same as a correct one until something re-tests it. Mantis at least names the requirement, which is more than most agentic security tooling does.

Where the claims stop

• The Mantis repository does not publish benchmarks. There are no measured false-positive rates, no comparison against conventional scanners or other agentic security tools, and no data on how often the reproduction or adversarial patch-verification stages succeed. Google's central claim, that this loop reduces hallucinated findings, is a design argument supported by the pipeline's structure rather than by published measurement.
• Which underlying models the harness uses by default, whether third-party models are supported beyond the Vertex AI authentication path shown, and what compute the reproduction stage demands at scale are not specified in the repository.
• The release describes itself as a starting point and as demonstration-only, explicitly not an officially supported Google product and not eligible for the company's vulnerability rewards programme. Treat its maturity accordingly.
• The skills are described as adaptable to hardware, infrastructure, ML pipelines and firmware, but the shipped examples and the reference harness target software review. Adaptation is an invitation, not a delivered capability.
• I have not run Mantis. Every capability described here is cited from the repository, and no effectiveness claim in this article rests on independent testing.

FAQ

What is Google Mantis and what does it do?

It is an open-source, Apache-2.0 licensed toolkit from Google for AI-assisted security review: a set of modular skills plus a reference harness built on Google's Agent Development Kit. It guides a coding agent through a vulnerability review lifecycle including historical review, semantic indexing, threat modelling, hypothesis generation, triage against hallucination, reproduction, chaining, patching with adversarial verification and severity calibration. Google describes it as a starting point for adaptation, not a supported product.

How is it different from an AI code scanner?

A conventional AI scanner runs a model over source code and reports what looks suspicious, which is why false positives and hallucinated findings dominate the output. Mantis is a harness rather than a single pass: it builds a threat model first, generates hypotheses, critiques findings to filter hallucination, attempts reproduction at whatever depth the environment allows, and then calibrates severity against a rubric. The intent is to surface fewer, better-supported findings instead of a high volume of plausible ones.

Is it safe to run?

Only in an isolated, restricted environment, and Google says so in capital letters. The suite is designed to generate and execute autonomously generated code that may be unstable or perform unexpected actions, and it must never run on a machine with access to production systems, sensitive data or internal networks. For the most capable models, Google recommends an additional sandboxing layer with monitoring for escape attempts. Findings also require manual verification by a security expert before anyone reports them.

The short version

Google's Mantis is a portable security review toolkit: skills plus an ADK harness that turns a coding agent into a multi-stage vulnerability reviewer, with threat modelling, anti-hallucination triage, graded reproduction, exploit chaining, adversarially verified patching and severity calibration. It is Apache-2.0, explicitly a starting point, and its most valuable content may be its warnings: run it only in isolation, and never mass-file unverified machine findings to open-source maintainers. The pipeline is a credible answer to the false-positive problem that plagues AI scanning, and the absence of published benchmarks means the claim that it works better remains an argument rather than a measurement.

Sources

• Google, Mantis repository, the skills, the ADK reference harness, the pipeline description, the safety cautions and the responsible-use notice: https://github.com/google/mantis