A Coding Agent Ran Overnight on a 200,000-Line Codebase: What Gemini 4 Argon Actually Changed

Posted on by
Contents show
A coding agent working overnight across a large codebase, with modules loading only when needed

Most model launches arrive with a demo built from an empty folder. Google's developer blog tried something harder for Gemini 4 Argon: it pointed a coding agent at a large existing repository and let it work through the night with nobody watching.

The repository was google/adk-python, Google's open source Agent Development Kit for building and deploying enterprise agents. It spans roughly 205,000 lines of Python across nearly 800 source files, with almost 17,000 unit tests. The run lasted 11 hours, starting at 9:11 in the evening. By morning the coding agent had opened a pull request proposing six separate optimizations across 32 files. Google's maintainers shipped two of them, rejected four during review, and merged a 22-file change whose commit note reads “Argon generated these changes.”

The measured results are specific rather than dramatic. Session preparation time in long conversations fell by about 40 percent, cold package imports got 21 percent faster, and a 158-turn tool calling workload dropped from 65.6 seconds to 56.1 seconds, which is nearly ten seconds of pure framework overhead removed.

What the experiment actually involved

Google did not hand the coding agent a single hot function to fix. The starting prompt in the first round described only the outcome the team wanted: make the Agent Development Kit add no noticeable overhead to any agent, at startup, in long sessions, and in multi-agent and workflow setups, keep the public API and what the model receives unchanged, and back every change with a test that would catch a regression.

Round one ran for about two hours and revealed both the promise and the problem. The coding agent wrote a benchmark script within seven minutes, then spent 70 minutes measuring and reading code before editing a single line. The coding agent found genuine bottlenecks in session copying and package imports, and parts of its work survived into the final commit. It also broke maintainer conventions: it attached a custom hook to the authentication model that subtly altered a public JSON schema, missed mock event edge cases in a custom copier, ran only 216 nearby unit tests instead of the full suite, and left benchmark numbers out of the pull request description.

Round two fixed the process rather than the model. Google added two things. First, the team's maintainer guidelines went into the repository's .agents/skills directory, structured like an onboarding guide, with a short SKILL.md index pointing to focused reference files that load on demand so the coding agent's context window is not flooded on every turn. Second, the run used goal mode in Google Antigravity, which keeps the objective pinned across context compactions so the coding agent keeps working until the task is finished. That is the same context management problem we examined in Claude's long context window, applied to a task that runs for hours instead of minutes. The prompt also asked for one unmerged pull request with before and after numbers in its description.

With those in place, the coding agent ran unattended for 11 hours. Before touching source code it read 12 guideline files and ran a baseline benchmark across four workloads: package startup, a 200-step conversation, a test app where 20 sub-agents hand tasks to each other, and a 12-node workflow, each run 30 times. Three minutes before the end the coding agent was still auditing its own diff, catching a bug in its own multi-agent cache and rerunning all 16,892 unit tests before stopping on its own.

The two changes that shipped

Filtering session history before copying. Every step in an Agent Development Kit run begins by loading the conversation history. The in-memory session service deep-copied the entire stored conversation on every read, before filtering down to the events the caller actually asked for, and listing sessions without their events still copied every event first. The coding agent changed the service to filter before copying and replaced Pydantic's generic deep-copy path on sessions and events with a faster cycle-safe copier that preserves shared state dictionaries and test mocks.

Loading heavy packages only when needed. Running import google.adk previously pulled in modules for live audio and video streaming, OpenTelemetry tracing, and web servers such as FastAPI and Starlette, even for a plain text agent. Deleting the top-level imports outright breaks downstream tests that patch attributes directly on modules, which the run proved the hard way: when the second round removed three internal flow imports from one file, all 16,892 open source tests passed while 25 internal Google test targets failed, because they patched those exact attributes. The shipped fix uses lazy module accessors instead, so heavy packages load on first use while every public attribute remains patchable where tests expect it.

The four changes the maintainers rejected

Google's post is unusually candid about the work it threw away, and the reasons are the most instructive part for anyone running a coding agent against a real repository.

  • A multi-agent event cache sped up a synthetic benchmark by 17 percent, but relied on a weak memory reference that could return stale data across sessions if Python reused a memory address after garbage collection.
  • A telemetry check read a private attribute on OpenTelemetry's tracer implementation, which could break silently on a minor library upgrade.
  • A rewrite of how the framework formats conversation history before each model call made a core code path harder to read without moving end to end latency on the independent holdout benchmark.
  • A lookup index for workflow event replays added state-tracking complexity that only helped synthetic replay tests rather than everyday runs.

Google summarised the division of labour in one line: the agent explores, the maintainer subtracts. That sentence is the most useful thing in the post, because it describes a workflow rather than a model capability.

Why cross-file work is the hard part

A profiler will tell you that copying sessions and importing packages take time. It will not trace every caller, mock target, and shared reference across 40 files to confirm that speeding up those paths leaves existing applications working. That gap is where a coding agent earns its place, because the work is reading and mapping rather than editing.

The two shipped changes show the shape of it. Filtering events before copying sounds like a one line edit until you account for every place that reads session history and every test that asserts on the copied objects. Replacing a framework wide deep copy with a custom copier means preserving shared state dictionaries and the mocks that tests attach to them.

The lazy import change is the sharper example. Removing three top level imports passed all 16,892 open source tests and broke 25 internal Google test targets that patched those exact attributes, which is a dependency that no single file reveals. The fix that shipped keeps every public attribute patchable while deferring the heavy modules until first use.

That is the argument for giving a coding agent a goal rather than a task. A goal forces the cross-module reading a profiler cannot do, and it also produces the changes a reviewer then has to subtract.

Before and after at a glance

The coding agent's results, in one place, with the caveat that every figure comes from the vendor run described above.

MeasureBeforeAfter
Cold import google.adkbaseline21 percent faster
In-memory session preparation in long sessionsbaselineabout 40 percent less
Full 158 turn tool calling run65.6 seconds56.1 seconds
New process to first model request3.07 seconds3.09 seconds, effectively flat
Unit tests run by the coding agent216 nearby tests in round oneall 16,892 in round two
Optimizations proposed and shipped6 proposed2 shipped, 4 rejected in review

The numbers, and the one definition you need

To read the benchmark results properly, it helps to know what the Agent Development Kit calls a turn. When a person types a prompt and receives a final answer, that is one invocation. A turn is an internal step underneath, each time the agent calls a tool, runs a batch of parallel tool calls, or queries an MCP server. Multi-turn tool calling is the reason to use an agent framework at all, and the framework overhead lives in those internal turns.

Google wrote an independent holdout benchmark that the coding agent never saw. A session grows turn by turn against an instant model and instant tools, so every millisecond measured is framework overhead rather than model thinking. Each turn runs three rounds of four parallel tool calls, and each number is the median of 6 to 12 runs.

Three results matter for anyone deploying these frameworks:

  • Import time. Cold import google.adk is about 21 percent faster, which helps command line tools, local development, and test discovery immediately. Because deferred modules still load on the first turn, time from a brand new process to its first model request stays flat, at 3.07 seconds before and 3.09 seconds after. On a production server, send a warm-up request before taking live traffic so the first real user does not pay that cost.
  • Session preparation. The 40 percent reduction applies to the in-memory session service, which powers the in-memory runner, local evaluation loops, continuous integration test suites, and default Cloud Run or GKE deployments when no external database is configured. Across 158 turns, the saving compounds into that ten second drop.
  • What does not change. External database, SQLite, Gemini platform, Firestore, and Redis session stores deserialize from storage rather than deep-copying in memory, so their read times are unaffected by this particular fix.

How to set up the same overnight run

The post reads as a workflow more than a product announcement, and the reusable parts are worth separating from the model.

Put a hard limit on what it can touch. The merged change stayed inside the framework and never altered the public API, because the prompt made that a constraint rather than a hope. An unattended run is only safe when its boundaries are written down in advance, and the first round shows why: the coding agent changed a public JSON schema through a custom hook, and nobody noticed until review.

Write the repository rules down. The .agents/skills pattern works because it gives the coding agent the knowledge a new engineer would receive in their first week: naming rules, visibility conventions, and checks the team runs. Keeping the top level file short and pushing detail into reference files is what keeps context usage under control across an 11 hour run.

Ask for the outcome, not the fix. The prompt in this experiment described a performance goal and two constraints, and the coding agent spent most of its first hour measuring rather than editing. Naming a specific function to optimize would have skipped the cross-module analysis that produced the useful changes.

Keep a benchmark the agent cannot see. Google's holdout benchmark is the reason the rejected rewrite was caught as cosmetic. A coding agent optimizing against its own measurements will happily improve the measurement. The same instinct that makes inference caching pay off in production applies inside a framework: measure the thing the user feels, not the number that is easiest to move.

Demand the full test suite and the numbers in writing. Round one ran 216 nearby tests and reported no figures. Round two ran all 16,892 tests and put before and after numbers in the pull request, which is what made review fast.

Keep a human on the merge button. Four of six proposals were dropped, and every rejection was correct. The autopilot is in the exploration, not in the approval.

What this means if you use AI coding tools

Most readers of this site will not run an 11 hour unattended session on their own repository, but the pattern behind the experiment is already arriving in the tools they do use.

The models are being trained and evaluated on real repositories now rather than on toy examples, which is why coding agents keep getting better at the boring parts of a large codebase. A 205,000-line project with 17,000 tests is a statement about the intended workload, and it builds on the same groundwork that production grade agentic systems depend on.

The workflow around the agent matters more than the agent. Two of the six proposals in this run came from a first attempt that ignored every repository convention, and the difference between round one and round two was not a better model. It was written conventions, a persistent goal, and a review gate.

Guardrails are still the point of failure. The four rejected changes are the honest picture: a coding agent will produce plausible optimizations, and several of these would have introduced stale caches, private attribute dependencies, and unreadable code without measurable benefit.

Where the claims stop

Several limits are worth stating plainly rather than repeating the headline.

  • It is a vendor run. The experiment comes from Google, on Google's own repository, with Google's model and Google's agent tooling. Nobody has replicated it independently.
  • The sample is one repository. One codebase, one language, one set of maintainer conventions. A JavaScript monorepo or a codebase with fewer tests would likely produce different results.
  • The averages come from medians of a small number of runs. Each figure is the median of 6 to 12 runs, which is enough for the published numbers and not a formal benchmark suite.
  • The performance wins are narrow. The 40 percent reduction applies to in-memory session handling, and the import improvement is offset at first request by deferred loading. Neither claim is a general statement about model speed.
  • The human work is invisible in the headline. The merged change used 22 files and 17 new test functions, and the review effort that separated those from the rejected four is not measured anywhere in the post.

FAQ

What is Gemini 4 Argon?

It is Google's frontier model released on 30 September 2026, available through the Gemini API under the model identifier gemini-4-argon, in Google AI Studio, and inside Google Antigravity. This experiment used it as the driver of an unattended coding agent rather than as a chat model.

What is the Agent Development Kit?

It is Google's open source framework for building and deploying agents, which we covered when Google launched its agent development kits in 2025 for Python and Java. The overnight run optimized the framework itself, not an application built with it.

Do I need Google Antigravity to try this?

Antigravity supplied the goal mode that kept the task alive across context compactions, so it is part of this specific setup. The transferable parts, which are written conventions, a held out benchmark, and a human review gate, work with any coding agent that can open a pull request.

Why did so many proposed optimizations fail review?

Because they were true only under conditions the benchmark created, or they depended on internals that a library update could break. A coding agent can optimize for measurable speed within a narrow harness, which is not the same as a change a maintainer should merge.

The short version

Google let a coding agent driven by Gemini 4 Argon run unattended for 11 hours against a 205,000-line Python repository. It proposed six optimizations, two shipped, and the merged change cut long session overhead by about 40 percent and cold import time by 21 percent. The more valuable output is the process around it: write your repository rules where the agent can read them, ask for the outcome rather than the fix, keep a benchmark the agent never sees, and keep a human on the merge button. The agent explores, the maintainer subtracts.

Sources
Subscription Form