If you lead engineering on a mature repository, the question is not which coding agent can produce the longest patch. The useful question is which workflow can enter old code, discover the real constraints, run meaningful checks, and leave reviewers with a small change they can accept or reject confidently.<\/p>\n\n\n\n
Short answer:<\/strong> start fragile legacy work with a terminal or IDE-supervised agent. Move to cloud and platform-native agents after the repo can bootstrap in a documented environment and the issue is specific enough to be judged by tests, CI, and review.<\/p>\n\n\n\n This ranking is for the first serious trial, not every future ticket. Once a repository has reliable setup scripts and a stable evaluation harness, cloud and platform-native agents become more attractive because their isolation and background execution are real advantages.<\/p>\n\n\n\n For this article, a coding agent means a tool that can inspect files, use repository tools, propose edits, run or request checks, and produce evidence. Examples include OpenAI Codex cloud, GitHub Copilot coding agent, Claude Code, and Gemini Code Assist agent mode.[1]<\/sup>[2]<\/sup>[3]<\/sup>[4]<\/sup><\/p>\n\n\n\n The model matters, but the permission boundary matters more. A powerful model behind a weak workflow still makes risky changes. In old code, the first failure is often not reasoning. It is setup, stale docs, hidden fixtures, local-only services, private package registries, or a test command nobody has written down.<\/p>\n\n\n\n Compare the route before you compare the model. Ask whether the agent can establish where it is, what owns the behavior, how to test it, and what it cannot verify. Only then does model cost, context window, or benchmark ranking become useful.<\/p>\n\n\n\n Do not start with a toy refactor or a greenfield feature. Pick one historical production bug that already has a known fix and a meaningful test. The agent should not see the final patch, but your reviewers should know what a good answer looks like.<\/p>\n\n\n\n A useful trial card should be this concrete:<\/p>\n\n\n\n The comparison should not be a final paragraph that says the agent did well. Capture the evidence while each route works. The most useful log has the same rows for every candidate:<\/p>\n\n\n\n The loser in a legacy trial is not always a vendor. It is often the route that hides too much from the reviewer. If the agent cannot show what it read, why it edited the files it edited, what it ran, and what remains unverified, it is not ready for high-risk areas.<\/p>\n\n\n\n Legacy repositories expose weak agents before the first code change. A stale README, old lockfile, private registry, split test suite, or undocumented Docker service is enough to separate careful agents from confident guessers.<\/p>\n\n\n\n If an agent edits production logic before it can explain how the repo is built and tested, stop the trial. That behavior usually gets worse on larger tasks.<\/p>\n\n\n\n Large context windows help only when the agent fills them with the right material. A curated packet usually beats a dump of unrelated docs, logs, and duplicate files.<\/p>\n\n\n\n For a legacy task, give the agent the issue, failing trace if available, owning module, neighboring tests, fixtures, and one or two call sites. Let it ask for more. Do not reward an agent for swallowing half the repository if it cannot explain which files changed its decision.<\/p>\n\n\n\n The output of an agent run is not code. It is a reviewable patch plus evidence. A good agent makes the reviewer\u2019s job smaller by separating what changed from what was checked and what remains risky.<\/p>\n\n\n\n This packet matters most for cloud and platform-native agents because the reviewer may not have watched the exploration. If the PR description is generic, the route is not ready for unsupervised legacy work.<\/p>\n\n\n\n Public benchmarks are useful as weak priors. SWE-bench Verified is relevant because it focuses on human-validated software-engineering tasks, while HumanEval, MMLU, and LMArena can help shortlist general model capability.[5]<\/sup>[6]<\/sup>[7]<\/sup>[8]<\/sup> None of them knows your deprecated payment path, weekend deploy freeze, private fixtures, or old authorization exception.<\/p>\n\n\n\n Batch APIs belong after the inner loop. They are useful for offline evaluation, historical issue replay, prompt regression, labeling old PRs, and comparing review summaries at scale. They are the wrong tool when the agent must inspect a failing command, patch the repo, rerun a targeted test, and adapt in real time. Provider batch features and limits change, so treat the docs as implementation references rather than strategy.[9]<\/sup>[10]<\/sup>[11]<\/sup>[12]<\/sup>[13]<\/sup><\/p>\n\n\n\nThe Fast Decision<\/h2>\n\n\n\n
Route<\/th> Best for<\/th> Avoid when<\/th> Why it wins or loses on old code<\/th><\/tr><\/thead> Terminal agent<\/td> First trials on billing, auth, migrations, data deletion, old fixtures, and brittle tests<\/td> The repo cannot safely run on a developer machine<\/td> Usually the strongest first choice because setup failures, grep results, targeted tests, and diffs stay visible in one feedback loop.<\/td><\/tr> IDE agent<\/td> Developer-supervised fixes where the engineer wants to approve plans and edits inline<\/td> The task needs a clean environment or should not touch the local workspace<\/td> Works well when the human keeps scope tight, but it can over-weight currently open files unless you force broader search.<\/td><\/tr> Cloud coding agent<\/td> Well-scoped tickets with reproducible setup and clear acceptance tests<\/td> Private services, manual setup, secrets, or data dependencies are still undocumented<\/td> Good for parallel work after setup is scripted. Weak for first contact with a messy repository.<\/td><\/tr> Platform-native agent<\/td> GitHub issue-to-PR workflows with strong CI, branch protection, and reviewer discipline<\/td> Issues are vague or CI does not cover the changed behavior<\/td> Fits review operations, but it inherits the quality of your issue template, test suite, and branch rules.<\/td><\/tr><\/tbody><\/table><\/figure>\n\n\n\n What You Are Actually Comparing<\/h2>\n\n\n\n
Use One Real Bug<\/h2>\n\n\n\n
Trial field<\/th> Example<\/th><\/tr><\/thead> Bug<\/td> Monthly invoice export includes subscriptions cancelled exactly at 00:00 UTC on the first day of the next month.<\/td><\/tr> Repository condition<\/td> Eight-year Rails monolith, Postgres, RSpec, factories, mixed service objects, full suite only in CI.<\/td><\/tr> Acceptance criteria<\/td> Add or update a failing spec, keep the fix in the billing export path, avoid schema changes, avoid dependency changes.<\/td><\/tr> Expected discovery<\/td> README, lockfile, billing export service, related spec, factory setup, date helper, and one call site.<\/td><\/tr> Expected command<\/td> bundle exec rspec spec\/services\/billing\/export_spec.rb<\/code> or the nearest equivalent discovered from the repo.<\/td><\/tr>Reviewer concern<\/td> Whether the fix changes inclusive or exclusive date semantics for existing reports.<\/td><\/tr><\/tbody><\/table><\/figure>\n\n\n\n Run a Head-to-Head Trial Log<\/h2>\n\n\n\n
Evidence<\/th> Terminal agent<\/th> IDE agent<\/th> Cloud agent<\/th> Platform-native agent<\/th><\/tr><\/thead> Setup<\/td> Should expose the exact failing command and environment gap immediately.<\/td> Good if the developer can run commands from the same workspace.<\/td> Good only if package registries, services, and secrets are already scripted.<\/td> Good if GitHub Actions can create the same environment reviewers trust.<\/td><\/tr> First files read<\/td> Should search README, lockfiles, owning module, tests, fixtures, and recent call sites.<\/td> Must be pushed beyond the currently open file.<\/td> Needs explicit prompt rules because the human may not see exploration live.<\/td> Needs issue templates that name the failing behavior and acceptance test.<\/td><\/tr> Commands run<\/td> Best at quick targeted reruns after a patch.<\/td> Strong when the engineer validates commands as the plan evolves.<\/td> Useful when the command set is known before the task starts.<\/td> Useful when CI is the source of truth and checks are required before review.<\/td><\/tr> Common failure<\/td> Can touch local-only configuration if permissions are too broad.<\/td> Can accept a narrow editor context too early.<\/td> Can stall on missing services or private packages.<\/td> Can produce a clean PR for the wrong interpretation of a vague issue.<\/td><\/tr> Review packet<\/td> Usually easiest to make detailed because command history is close at hand.<\/td> Good if the human asks for changed files, tests, and risks before accepting edits.<\/td> Must be required explicitly.<\/td> Must be visible in the PR description or review comments.<\/td><\/tr> First-trial verdict<\/td> Best default for fragile legacy bugs.<\/strong><\/td> Best when a senior engineer is actively supervising.<\/strong><\/td> Best after setup is reproducible.<\/strong><\/td> Best when GitHub workflow quality is already high.<\/strong><\/td><\/tr><\/tbody><\/table><\/figure>\n\n\n\n Setup Is the First Filter<\/h2>\n\n\n\n
Context Is Search Discipline<\/h2>\n\n\n\n
Context behavior<\/th> Good sign<\/th> Bad sign<\/th><\/tr><\/thead> Search path<\/td> Finds owning module, related tests, fixtures, migrations, and call sites before editing.<\/td> Edits the first file with a matching string.<\/td><\/tr> Pattern use<\/td> Copies existing error handling, feature flags, validation, logging, and dependency injection style.<\/td> Adds a new service layer or package for a narrow behavior change.<\/td><\/tr> Scope control<\/td> Keeps the diff near the requested behavior and explains every extra file.<\/td> Reformats unrelated files or changes tests to fit the patch.<\/td><\/tr> Uncertainty<\/td> Names the missing contract and asks for one product decision.<\/td> Invents behavior or treats a failing test as wrong without evidence.<\/td><\/tr><\/tbody><\/table><\/figure>\n\n\n\n Review Quality Is the Product<\/h2>\n\n\n\n
Benchmarks and Batch Come Later<\/h2>\n\n\n\n
Where Model Choice Fits<\/h2>\n\n\n\n