1How can an agent fix real code without breaking what already works?
Code written blind looks right and often fails. A coding agent works in a sandboxed copy of the repository: it reads files, edits them and runs the project's own tests and linters. The checks, not the agent's confidence, decide when the change is done.
You may need it if
- Your team spends hours on small, well-described bugs and routine upgrades.
- The project has tests and linters that run in minutes from the command line.
- AI suggestions pasted from a chat often fail to compile or break other tests.
2When is a coding agent the wrong tool?
Use it when
- The issue is specific and its success can be shown by a test or a reproduction.
- The repository builds and its checks run without manual setup.
- The change is a few files, not a redesign.
Avoid it when
- The project has no tests. The loop has nothing reliable to stop on.
Use Reflection instead - The real question is what to build. The agent needs a plan before it touches code.
Use Planning instead - The change must be made on a live server, not in a copy, and each command has real effects.
Use Dry Run instead
What it costs
- Model calls
- 10 to 50 per issue, about one per file read, edit or check run
- Latency
- high
- Complexity
- high
3How does an issue become a change that passes?
The coder reads and edits a sandboxed repository, then runs the project's checks. A plain rule reads the check output. Failures go back to the coder with the output. On green, a reviewer compares the diff with the issue. An attempt limit ends the loop.
- Step 1 of 6
Explore before editing
For "Export to CSV drops accented names", the coder searches the repository for the export code, reads it and writes a failing test that reproduces the bug.
- Step 2 of 6
Edit in the sandbox
The coder changes the files in the sandbox copy. The real repository is never touched, so a bad edit costs nothing but time.
- Step 3 of 6
The project's checks decide
The coder runs the same tests, type checks and linter your team runs. The gate reads the result as plain pass or fail, with no model judgement involved.
- Step 4 of 6
Red goes back with the output
Failing output returns to the coder, trimmed to the failing tests and their errors. The coder reads it, edits again and reruns, until green or out of attempts.
- Step 5 of 6
Green goes to review
The reviewer reads only the issue and the diff. It checks that the change fixes the issue, touches nothing unrelated and does not weaken any test.
- Step 6 of 6
Two exits
An approved diff leaves as a change ready for a pull request. When attempts run out, the run leaves as unresolved with what was tried, so a developer starts from there.
What each part does
- Issue
- The bug or change request, with steps to reproduce and any plan from an earlier stage.
- Coder
- Searches and reads files, makes edits, and asks for the checks to run.
- Sandbox repo
- An isolated copy of the repository. Edits here never touch the real one.
- Project checks
- The project's own tests, type checker and linter, run in the sandbox.
- Checks pass?
- Reads the check output and counts attempts. Green goes to review; red goes back or stops.
- Reviewer
- Reads the diff against the issue. Rejects changes that miss the point or weaken tests.
- Change
- The diff, the check results and a short explanation, ready for a pull request.
- Unresolved
- The last diff, the failing output and what was tried, when the attempt limit is reached.
- Sandbox repo → Coder: files and search results
- Coder → Sandbox repo: edits
- Coder → Project checks: run the checks
- Sandbox repo → Project checks: current code
- Project checks → Checks pass?: check output
- Checks pass? → Coder: failing, attempts left
- Checks pass? → Reviewer: all checks green
- Checks pass? → Unresolved: failing, attempts spent
- Reviewer → Coder: diff rejected, attempts left
- Reviewer → Change: diff approved
Where it connects
An issue or change request on a named repository, ideally with steps to reproduce. A plan, if given, lists the intended changes in order.
When: All checks pass and the reviewer approves the diff.
The diff, the list of checks run with their results, the test that reproduces the issue, and a short explanation for the pull request.
When: The checks still fail, or the reviewer still rejects, when the attempt limit is reached.
The last diff, the failing check output or the reviewer's reasons, and a summary of each attempt and what it changed.
The sandboxed working copy with the agent's edits. Discarded at the end of the run; only the diff leaves.
4What do I ask my assistant to set up?
The agents to launch
Coder
Resolve the issue in the sandbox repository. First find and read the relevant code, then write a test that reproduces the problem. Make the smallest change that fixes it, then run the project's checks. Do not edit or delete existing tests unless the issue requires it, and say why if you do.
Sees: The issue, any plan, the sandbox repository and the latest check output. Not the reviewer's notes until a rejection.
Reviewer
Compare the diff with the issue. Approve only if it fixes the issue as described, touches nothing unrelated and weakens no test. Otherwise reject with specific points referring to lines of the diff.
Sees: The issue and the diff with the check results. Not the coder's exploration or reasoning.
A prompt you can paste
Works with any assistant that can launch sub-agents. It contains the agents, the procedure and the stop rule above.
You are the orchestrator of a multi-agent workflow that follows the "SWE Agent" architecture. Launch every agent listed below as a separate sub-agent with its own context. Give each agent only the context stated for it. Do not do an agent's work yourself. If you cannot launch sub-agents, play each agent in a clearly separated turn and never let it use what is withheld from it. # Task [Describe your task here] # SWE Agent Explore the code, edit it, run the checks, repeat until green. Input: the task above. It must provide: An issue or change request on a named repository, ideally with steps to reproduce. A plan, if given, lists the intended changes in order. If something essential is missing, ask for it before starting. Agents to launch: - Coder: Resolve the issue in the sandbox repository. First find and read the relevant code, then write a test that reproduces the problem. Make the smallest change that fixes it, then run the project's checks. Do not edit or delete existing tests unless the issue requires it, and say why if you do. Context for this agent: The issue, any plan, the sandbox repository and the latest check output. Not the reviewer's notes until a rejection. - Reviewer: Compare the diff with the issue. Approve only if it fixes the issue as described, touches nothing unrelated and weakens no test. Otherwise reject with specific points referring to lines of the diff. Context for this agent: The issue and the diff with the check results. Not the coder's exploration or reasoning. Procedure: 1. Make an isolated copy of the repository, such as a container or a fresh branch in a throwaway clone. Confirm the checks run there before any edit. 2. Write down the exact commands for the checks, for example the test, type check and lint commands. 3. Launch the coder on the issue in the sandbox. 4. Each time the coder asks, run the checks and return the failing part of the output. Count each run as one attempt. 5. When all checks pass, launch the reviewer in a fresh context on the issue and the diff. On rejection, send the reasons to the coder and continue. Stop rule: Stop when the reviewer approves a green diff, or after 8 check runs or 2 reviewer rejections, whichever comes first. Then report as unresolved. Exits: - `change`, when: All checks pass and the reviewer approves the diff. Deliver: The diff, the list of checks run with their results, the test that reproduces the issue, and a short explanation for the pull request. - `unresolved`, when: The checks still fail, or the reviewer still rejects, when the attempt limit is reached. Deliver: The last diff, the failing check output or the reviewer's reasons, and a summary of each attempt and what it changed. # Final deliverable Either the approved diff with the checks and their results and a pull request description, or the unresolved report with the last diff, the failing output and each attempt.
The same thing in code
from typing import TypedDict
from langchain.agents import create_agent
from langchain.chat_models import init_chat_model
from langchain_core.messages import HumanMessage
from langchain_core.tools import tool
from langgraph.graph import END, START, StateGraph
from pydantic import BaseModel
model = init_chat_model("anthropic:claude-sonnet-5-5")
REPO: dict[str, str] = {} # sandbox copy, path -> contents: plug in a container or throwaway clone
BASE = dict(REPO)
MAX_CHECK_RUNS, MAX_REJECTIONS = 8, 2
@tool
def search_code(pattern: str) -> list[str]:
"""List sandbox files containing the pattern."""
return [p for p, src in REPO.items() if pattern in src]
@tool
def read_file(path: str) -> str:
"""Read one file from the sandbox."""
return REPO.get(path, "no such file")
@tool
def write_file(path: str, content: str) -> str:
"""Overwrite one file in the sandbox. The real repository is never touched."""
REPO[path] = content
return "written"
def run_checks() -> tuple[bool, str]:
# plug in the project's test, type check and lint commands, run inside the sandbox
return True, "all checks passed"
def diff() -> str:
return "\n".join(f"--- {p}\n+++ {p}\n{REPO[p]}" for p in REPO if REPO[p] != BASE.get(p))
coder = create_agent(model, [search_code, read_file, write_file], system_prompt=(
"Resolve the issue in the sandbox. Read the relevant code, write a test that reproduces the problem, "
"then make the smallest change that fixes it. Do not edit or delete existing tests unless the issue requires it."))
class Review(BaseModel):
approved: bool
points: list[str] # specific points referring to lines of the diff
class State(TypedDict):
issue: str
feedback: str # latest failing output, or the reviewer's points after a rejection
check_runs: int
rejections: int
passed: bool
status: str
async def code(state: State) -> dict:
# sees the issue, the sandbox and the latest feedback only
prompt = f"Issue:\n{state['issue']}\n\nFeedback:\n{state['feedback']}"
out = await coder.ainvoke({"messages": [HumanMessage(prompt)]})
return {"feedback": out["messages"][-1].text}
async def checks(state: State) -> dict:
passed, output = run_checks()
return {"passed": passed, "feedback": output[-4000:], "check_runs": state["check_runs"] + 1} # failing part only
def gate(state: State) -> str: # plain rule on the real check output
if state["passed"]:
return "reviewer"
return "coder" if state["check_runs"] < MAX_CHECK_RUNS else "unresolved"
async def reviewer(state: State) -> dict:
# fresh context: the issue and the diff with the check results, not the coder's exploration
r = Review.model_validate(await model.with_structured_output(Review).ainvoke(
"Approve only if the diff fixes the issue as described, touches nothing unrelated and weakens no test. "
f"Otherwise reject with specific points.\n\nIssue:\n{state['issue']}\n\nDiff:\n{diff()}\n\nChecks: green"))
if r.approved:
return {"status": "change"}
return {"feedback": "Reviewer rejected:\n" + "\n".join(r.points), "rejections": state["rejections"] + 1}
def after_review(state: State) -> str:
if state["status"] == "change":
return END
if state["rejections"] < MAX_REJECTIONS and state["check_runs"] < MAX_CHECK_RUNS:
return "coder"
return "unresolved"
async def unresolved(state: State) -> dict:
return {"status": "unresolved"}
graph = StateGraph(State)
graph.add_node("coder", code)
graph.add_node("checks", checks)
graph.add_node("reviewer", reviewer)
graph.add_node("unresolved", unresolved)
graph.add_edge(START, "coder")
graph.add_edge("coder", "checks")
graph.add_conditional_edges("checks", gate, ["coder", "reviewer", "unresolved"])
graph.add_conditional_edges("reviewer", after_review, ["coder", "unresolved", END])
graph.add_edge("unresolved", END)
app = graph.compile()
async def run(task: str):
s = await app.ainvoke({"issue": task, "feedback": "", "check_runs": 0, "rejections": 0,
"passed": False, "status": ""}, {"recursion_limit": 60})
return {"status": s["status"], "diff": diff(), "last_output": s["feedback"], "check_runs": s["check_runs"]}
Replace REPO and run_checks with a real sandbox (container or throwaway clone) running the project's own commands, with no production credentials.
from pathlib import Path
from typing import TypeVar
from claude_agent_sdk import AgentDefinition, ClaudeAgentOptions, ResultMessage, query
from pydantic import BaseModel
SANDBOX = Path("sandbox") # an isolated copy of the repository: a throwaway clone or a container mount
MAX_CHECK_RUNS, MAX_REJECTIONS = 8, 2
AGENTS = {
"coder": AgentDefinition(
description="Resolves an issue by reading and editing the sandbox repository.",
prompt="Resolve the issue in this repository. Find and read the relevant code, write a test that "
"reproduces the problem, then make the smallest change that fixes it. Do not edit or delete "
"existing tests unless the issue requires it, and say why if you do.",
tools=["Read", "Grep", "Glob", "Edit", "Write"]), # no Bash: checks are run by the gate, not the coder
"reviewer": AgentDefinition(
description="Reviews a diff against its issue.",
prompt="Approve only if the diff fixes the issue as described, touches nothing unrelated and weakens "
"no test. Otherwise reject with specific points referring to lines of the diff.",
tools=[]),
}
class Report(BaseModel):
summary: str
class Review(BaseModel):
approved: bool
points: list[str]
M = TypeVar("M", bound=BaseModel)
async def delegate(agent: str, payload: str, schema: type[M]) -> M:
# one fresh sub-agent run per step, working inside the sandbox only
options = ClaudeAgentOptions(
agents={agent: AGENTS[agent]}, cwd=SANDBOX, permission_mode="acceptEdits",
allowed_tools=["Agent", "Read", "Grep", "Glob", "Edit", "Write"],
output_format={"type": "json_schema", "schema": schema.model_json_schema()})
prompt = f"Hand exactly this input to the {agent} agent and return its result.\n\n{payload}"
async for msg in query(prompt=prompt, options=options):
if isinstance(msg, ResultMessage) and msg.structured_output:
return schema.model_validate(msg.structured_output)
raise RuntimeError(f"{agent} returned no result")
def run_checks() -> tuple[bool, str]:
# plug in the project's test, type check and lint commands, run inside SANDBOX
return True, "all checks passed"
def sandbox_diff() -> str:
# plug in `git diff` run inside SANDBOX
return ""
async def run(task: str):
feedback, rejections, attempts = "", 0, []
for check_run in range(1, MAX_CHECK_RUNS + 1):
report = await delegate("coder", f"Issue:\n{task}\n\nLatest feedback:\n{feedback or 'none'}", Report)
passed, output = run_checks() # the gate reads real check output, not the coder's opinion
attempts.append(f"run {check_run}: {report.summary} -> {'green' if passed else 'red'}")
if not passed:
feedback = output[-4000:] # only the failing part goes back
continue
diff = sandbox_diff()
review = await delegate("reviewer", f"Issue:\n{task}\n\nDiff:\n{diff}\n\nChecks: {output}", Review)
if review.approved:
return {"status": "change", "diff": diff, "checks": output, "attempts": attempts}
rejections += 1
if rejections >= MAX_REJECTIONS: # stop after 2 reviewer rejections
break
feedback = "Reviewer rejected:\n" + "\n".join(review.points)
return {"status": "unresolved", "diff": sandbox_diff(), "last_output": feedback, "attempts": attempts}
Point SANDBOX at a throwaway clone or container without production credentials, and wire run_checks and sandbox_diff to the project's own commands.
from agents import Agent, Runner, function_tool
from pydantic import BaseModel
REPO: dict[str, str] = {} # sandbox copy, path -> contents: plug in a container or throwaway clone
BASE = dict(REPO)
MAX_CHECK_RUNS, MAX_REJECTIONS = 8, 2
@function_tool
def search_code(pattern: str) -> list[str]:
"""List sandbox files containing the pattern."""
return [p for p, src in REPO.items() if pattern in src]
@function_tool
def read_file(path: str) -> str:
"""Read one file from the sandbox."""
return REPO.get(path, "no such file")
@function_tool
def write_file(path: str, content: str) -> str:
"""Overwrite one file in the sandbox. The real repository is never touched."""
REPO[path] = content
return "written"
def run_checks() -> tuple[bool, str]:
# plug in the project's test, type check and lint commands, run inside the sandbox
return True, "all checks passed"
def diff() -> str:
return "\n".join(f"--- {p}\n+++ {p}\n{REPO[p]}" for p in REPO if REPO[p] != BASE.get(p))
class Review(BaseModel):
approved: bool
points: list[str]
coder = Agent(
name="Coder",
instructions="Resolve the issue in the sandbox. Read the relevant code, write a test that reproduces the "
"problem, then make the smallest change that fixes it. Do not edit or delete existing tests "
"unless the issue requires it, and say why if you do. End with a one-line summary.",
tools=[search_code, read_file, write_file])
reviewer = Agent(
name="Reviewer",
instructions="Approve only if the diff fixes the issue as described, touches nothing unrelated and weakens "
"no test. Otherwise reject with specific points referring to lines of the diff.",
output_type=Review)
async def run(task: str):
feedback, rejections, attempts = "", 0, []
for check_run in range(1, MAX_CHECK_RUNS + 1):
# the coder sees the issue, the sandbox and the latest feedback, not earlier reviewer notes
result = await Runner.run(coder, f"Issue:\n{task}\n\nLatest feedback:\n{feedback or 'none'}", max_turns=30)
passed, output = run_checks() # the gate: a plain rule on the real check output
attempts.append(f"run {check_run}: {result.final_output} -> {'green' if passed else 'red'}")
if not passed:
feedback = output[-4000:] # only the failing part goes back
continue
review = (await Runner.run(reviewer, f"Issue:\n{task}\n\nDiff:\n{diff()}\n\nChecks: {output}")
).final_output_as(Review)
if review.approved:
return {"status": "change", "diff": diff(), "checks": output, "attempts": attempts}
rejections += 1
if rejections >= MAX_REJECTIONS: # stop after 2 reviewer rejections
break
feedback = "Reviewer rejected:\n" + "\n".join(review.points)
return {"status": "unresolved", "diff": diff(), "last_output": feedback, "attempts": attempts}
Replace REPO and run_checks with a real sandbox (container or throwaway clone) running the project's own commands, with no production credentials.
import { anthropic } from '@ai-sdk/anthropic';
import { generateText, isStepCount, Output, tool } from 'ai';
import { z } from 'zod';
const model = anthropic('claude-sonnet-5-5');
const REPO = new Map<string, string>(); // sandbox copy, path -> contents: plug in a container or throwaway clone
const BASE = new Map(REPO);
const MAX_CHECK_RUNS = 8;
const MAX_REJECTIONS = 2;
const sandboxTools = {
searchCode: tool({
description: 'List sandbox files containing the pattern.',
inputSchema: z.object({ pattern: z.string() }),
execute: async ({ pattern }) => [...REPO].filter(([, src]) => src.includes(pattern)).map(([p]) => p),
}),
readFile: tool({
description: 'Read one file from the sandbox.',
inputSchema: z.object({ path: z.string() }),
execute: async ({ path }) => REPO.get(path) ?? 'no such file',
}),
writeFile: tool({
description: 'Overwrite one file in the sandbox. The real repository is never touched.',
inputSchema: z.object({ path: z.string(), content: z.string() }),
execute: async ({ path, content }) => { REPO.set(path, content); return 'written'; },
}),
};
// plug in the project's test, type check and lint commands, run inside the sandbox
async function runChecks(): Promise<{ passed: boolean; output: string }> {
return { passed: true, output: 'all checks passed' };
}
const diff = () => [...REPO].filter(([p, src]) => BASE.get(p) !== src)
.map(([p, src]) => `--- ${p}\n+++ ${p}\n${src}`).join('\n');
const Review = z.object({ approved: z.boolean(), points: z.array(z.string()) });
export async function run(task: string) {
let feedback = 'none';
let rejections = 0;
const attempts: string[] = [];
for (let checkRun = 1; checkRun <= MAX_CHECK_RUNS; checkRun++) {
// the coder sees the issue, the sandbox and the latest feedback only
const { text } = await generateText({
model,
instructions: 'Resolve the issue in the sandbox. Read the relevant code, write a test that reproduces the '
+ 'problem, then make the smallest change that fixes it. Do not edit or delete existing tests unless '
+ 'the issue requires it, and say why if you do. End with a one-line summary.',
prompt: `Issue:\n${task}\n\nLatest feedback:\n${feedback}`,
tools: sandboxTools,
stopWhen: isStepCount(30),
});
const { passed, output } = await runChecks(); // the gate: a plain rule on the real check output
attempts.push(`run ${checkRun}: ${text.slice(-200)} -> ${passed ? 'green' : 'red'}`);
if (!passed) { feedback = output.slice(-4000); continue; } // only the failing part goes back
const { output: review } = await generateText({
model,
instructions: 'Approve only if the diff fixes the issue as described, touches nothing unrelated and weakens '
+ 'no test. Otherwise reject with specific points referring to lines of the diff.',
prompt: `Issue:\n${task}\n\nDiff:\n${diff()}\n\nChecks: ${output}`,
output: Output.object({ schema: Review }),
});
if (review.approved) return { status: 'change', diff: diff(), checks: output, attempts };
if (++rejections >= MAX_REJECTIONS) break; // stop after 2 reviewer rejections
feedback = `Reviewer rejected:\n${review.points.join('\n')}`;
}
return { status: 'unresolved', diff: diff(), lastOutput: feedback, attempts };
}
Replace REPO and runChecks with a real sandbox (container or throwaway clone) running the project's own commands, with no production credentials.
Snippets are generated when the site is built and type-checked against the SDK versions listed in the repository. Treat them as a starting skeleton.
What goes wrong
- Running the agent on the real repository or with production credentials turns every edit into a live change. Always use a sandbox.
- Returning thousands of lines of test output drowns the coder. Send only the failing tests and their errors.
- A diff that touches many unrelated files is hard to review and often hides regressions. Ask for the smallest change.