Agentic Architectures

Catalogue / Sampling & Search

Language Agent Tree Search

Try moves in a sandbox, score them, and steer the search by scores

Shape
tree-search
Model calls
iterations × (1 expansion + k evaluations); e.g. 10 × (1 + 3) = 40
Latency
high
Complexity
high

Also known as LATS, Monte-Carlo tree search for agents

1

What if an agent needs many tries to find the sequence of actions that works?

A step-by-step agent follows one line of actions and gets stuck when an early move was wrong. LATS keeps a tree of everything tried. Each round it picks the most promising point, tries a few next moves in a sandbox, scores the results and passes the scores up the tree, so effort goes where progress is.

You may need it if

  • An agent works in a system that answers back: a test suite, a staging API, a spreadsheet model.
  • It often gets close, then fails because of a choice made several steps earlier.
  • You can score how close a state is to done, not only whether it is done.
2

When is a full tree search too much machinery?

Use it when

  • The agent acts in a sandbox where moves can be tried, observed and thrown away.
  • The task needs several moves in a row, and many different sequences are possible.
  • Progress can be scored at each step: tests passing, rows matching, a page closer to the target.

Avoid it when

  • Moves change real systems and cannot be rolled back. Exploring would mean doing damage on every branch.
    Use Dry Run instead
  • There is no environment to act in, only reasoning steps to compare.
    Use Tree of Thoughts instead
  • The path is usually straightforward and a step-by-step agent succeeds most of the time.
    Use ReAct instead
  • Users wait for a reply in seconds. A search runs for minutes and dozens of calls.

What it costs

Model calls
iterations × (1 expansion + k evaluations); e.g. 10 × (1 + 3) = 40
Latency
high
Complexity
high
3

How does one round of the search work?

A loop around a stored tree. A selection rule picks the most promising node. The expander proposes a few next moves, the sandbox runs them, the evaluator scores the new states and the scores are added up the path to the root. Rounds repeat until a move solves the task or the budget runs out.

not solved,budget leftsolved orbudget spentTask: The goal, the sandbox the agent may act in, and the test that says the task is solved.TaskSelect: A formula that picks the node with the best balance of high score and few visits.SelectSearch tree: Every state reached, the move that led to it, its visit count, its average score and notes on failures.Search treeExpander: Proposes a few different next moves from the selected state, avoiding moves already tried there.ExpanderSandbox: Runs each move in an isolated copy of the environment and returns what happened.SandboxEvaluator: Scores each new state for progress toward the goal and writes a short lesson when a move fails.EvaluatorSolved or out?: Checks whether a state passed the success test and counts the rounds left in the budget.Solved orout?Best trajectory: The sequence of moves that solved the task, or the best one found, with what each move did.Besttrajectory
  1. Step 1 of 5

    Pick where to dig

    The selector reads the tree and picks a state. It favours states with high average scores, but gives a bonus to states rarely visited, so a lucky early branch does not take all the effort.

  2. Step 2 of 5

    Propose next moves

    The expander reads the path to that state and the notes left by failed branches, then proposes three different next moves, for example three fixes for a failing data import.

  3. Step 3 of 5

    Try them for real, safely

    The sandbox runs each move on a throwaway copy and returns the observation, such as an error message or the number of rows that now match.

  4. Step 4 of 5

    Score and pass it up

    The evaluator scores each new state for progress and writes a one-line lesson when a move fails. The scores are added to every state on the path back to the root, so good branches rise.

  5. Step 5 of 5

    Another round, or stop

    If no state passes the success test and rounds remain, the loop returns to selection. Otherwise the best trajectory leaves with the record of what each move did.

What each part does
Task
The goal, the sandbox the agent may act in, and the test that says the task is solved.
Select
A formula that picks the node with the best balance of high score and few visits.
Search tree
Every state reached, the move that led to it, its visit count, its average score and notes on failures.
Expander
Proposes a few different next moves from the selected state, avoiding moves already tried there.
Sandbox
Runs each move in an isolated copy of the environment and returns what happened.
Evaluator
Scores each new state for progress toward the goal and writes a short lesson when a move fails.
Solved or out?
Checks whether a state passed the success test and counts the rounds left in the budget.
Best trajectory
The sequence of moves that solved the task, or the best one found, with what each move did.
  • Task → Select: root state
  • Search tree → Select: scores and visits
  • Select → Expander: chosen state
  • Search tree → Expander: path and failure notes
  • Expander → Sandbox: k candidate moves
  • Sandbox → Evaluator: observations
  • Evaluator → Search tree: scores passed up to root
  • Solved or out? → Select: not solved, budget left
  • Solved or out? → Best trajectory: solved or budget spent

Where it connects

Entry
task
Task

A goal reachable through a sequence of moves, the sandbox the agent may act in, and a success test it can run. A round budget is optional, the stage sets one when absent.

Exit
result
AnswerOutcome

When: A state passes the success test, or the round budget is spent.

The trajectory of moves from the root, what each move did in the sandbox, its score, whether the success test passed, and the final answer or state reached.

Working memory for one run
tree
Outcome

Each state reached, the move that led to it, its parent, its visit count, its average score and any failure note. Discarded when the run ends.

4

What do I ask my assistant to set up?

The agents to launch

Expander

You receive the goal, the path of moves that led to the current state, and lessons from failed branches. Propose 3 different next moves, each one concrete enough to run in the sandbox. Do not repeat a move already tried from this state. Do not predict the result.

Sees: The goal, the path from the root to the selected state with its observations, and failure notes. Not the scores.

Evaluator

Score the new state from 0 to 10 for progress toward the goal, using hard signals first: tests passing, errors gone, values matching. Say whether the success test passed. If the move made things worse or failed, write one line saying why, as a lesson for later moves.

Sees: The goal, the success test, the move and the sandbox observation. Not the expander's reasoning, not other scores.

A prompt you can paste

Works with any assistant that can launch sub-agents. It contains the agents, the procedure and the stop rule above.

Framework-agnostic prompt
You are the orchestrator of a multi-agent workflow that follows the "Language Agent Tree Search" architecture.
Launch every agent listed below as a separate sub-agent with its own context. Give each agent only the context stated for it. Do not do an agent's work yourself. If you cannot launch sub-agents, play each agent in a clearly separated turn and never let it use what is withheld from it.

# Task
[Describe your task here]

# Language Agent Tree Search
Try moves in a sandbox, score them, and steer the search by scores.
Input: the task above. It must provide: A goal reachable through a sequence of moves, the sandbox the agent may act in, and a success test it can run. A round budget is optional, the stage sets one when absent. If something essential is missing, ask for it before starting.

Agents to launch:
- Expander: You receive the goal, the path of moves that led to the current state, and lessons from failed branches. Propose 3 different next moves, each one concrete enough to run in the sandbox. Do not repeat a move already tried from this state. Do not predict the result.
  Context for this agent: The goal, the path from the root to the selected state with its observations, and failure notes. Not the scores.
- Evaluator: Score the new state from 0 to 10 for progress toward the goal, using hard signals first: tests passing, errors gone, values matching. Say whether the success test passed. If the move made things worse or failed, write one line saying why, as a lesson for later moves.
  Context for this agent: The goal, the success test, the move and the sandbox observation. Not the expander's reasoning, not other scores.

Procedure:
1. Set up the sandbox so every move runs on a disposable copy. Write the success test and set the budget, for example 10 rounds of 3 moves.
2. Create the tree with the starting state as root. Keep for each state its parent, move, observation, visits, total score and failure note.
3. Select: starting from the root, repeatedly go to the child with the best average score plus a bonus for few visits, until reaching a state without children.
4. Launch the expander on the selected state, then run each proposed move in the sandbox.
5. Launch the evaluator in a fresh context on each new state. Add its score and one visit to that state and to every state on its path to the root.
6. If a state passes the success test, stop. Otherwise, if rounds remain, return to selection.

Stop rule: Stop as soon as a state passes the success test, or after 10 rounds, whichever comes first. Never extend the budget during the run.

Exit:
- `result`, when: A state passes the success test, or the round budget is spent. Deliver: The trajectory of moves from the root, what each move did in the sandbox, its score, whether the success test passed, and the final answer or state reached.

# Final deliverable
The trajectory from the root to the best state, what each move did, its score, whether the success test passed, the rounds used, and the main lessons recorded from failed branches.

The same thing in code

LangGraph, python, package langgraph
import asyncio
import math
from dataclasses import dataclass, field
from typing import TypedDict

from langchain.chat_models import init_chat_model
from langgraph.graph import END, START, StateGraph
from pydantic import BaseModel

llm = init_chat_model("anthropic:claude-sonnet-5-5")
ROUNDS, K, C = 10, 3, 1.4  # budget: 10 rounds of 3 moves
EXPANDER = ("Given the goal, the path of moves to the current state and lessons from failed branches, "
            "propose 3 different next moves, each concrete enough to run in the sandbox. "
            "Do not repeat a move already tried from this state. Do not predict the result.")
EVALUATOR = ("Score the new state from 0 to 10 for progress toward the goal, hard signals first: tests "
             "passing, errors gone, values matching. Say whether the success test passed. If the move "
             "failed or made things worse, write one line saying why; otherwise leave the lesson empty.")

class Moves(BaseModel):
    moves: list[str]

class Score(BaseModel):
    score: float
    solved: bool
    lesson: str

@dataclass
class Node:  # one state of the search tree (the store)
    parent: int | None
    move: str
    observation: str
    children: list[int] = field(default_factory=list)
    visits: int = 0
    total: float = 0.0
    note: str = ""

class State(TypedDict):
    task: str
    tree: dict[int, Node]
    selected: int
    moves: list[str]
    new: list[int]
    rounds: int
    solved: int

def run_in_sandbox(moves: list[str]) -> str:
    """Stub: replay the moves on a disposable copy of the environment (container, VM, scratch repo)."""
    return f"ran {moves[-1]!r}"

def path(tree: dict[int, Node], i: int | None) -> list[int]:
    ids: list[int] = []
    while i is not None:
        ids, i = [i] + ids, tree[i].parent
    return ids

def select(state: State) -> dict:
    tree, i = state["tree"], 0
    def uct(c: int) -> float:  # average score plus a bonus for few visits
        n = tree[c]
        return math.inf if n.visits == 0 else n.total / n.visits + C * math.sqrt(math.log(tree[i].visits) / n.visits)
    while tree[i].children:
        i = max(tree[i].children, key=uct)
    return {"selected": i}

async def expand(state: State) -> dict:
    # the expander sees the path and failure notes, not the scores or the whole tree
    tree = state["tree"]
    steps = "\n".join(f"- {tree[j].move} -> {tree[j].observation}" for j in path(tree, state["selected"]))
    lessons = "\n".join(n.note for n in tree.values() if n.note)
    out = await llm.with_structured_output(Moves).ainvoke(
        [("system", EXPANDER), ("user", f"Goal: {state['task']}\nPath:\n{steps}\nLessons:\n{lessons}")])
    return {"moves": Moves.model_validate(out).moves[:K]}

def sandbox(state: State) -> dict:
    tree, parent = state["tree"], state["selected"]
    prefix = [tree[j].move for j in path(tree, parent)[1:]]
    for move in state["moves"]:
        tree[len(tree)] = Node(parent=parent, move=move, observation=run_in_sandbox(prefix + [move]))
        tree[parent].children.append(len(tree) - 1)
    return {"tree": tree, "new": tree[parent].children}

async def evaluate(state: State) -> dict:
    tree, solved = state["tree"], -1
    async def score(k: int) -> Score:  # fresh context: goal, move and observation only
        prompt = f"Goal and success test: {state['task']}\nMove: {tree[k].move}\nObservation: {tree[k].observation}"
        return Score.model_validate(await llm.with_structured_output(Score).ainvoke([("system", EVALUATOR), ("user", prompt)]))
    for k, s in zip(state["new"], await asyncio.gather(*(score(k) for k in state["new"]))):
        tree[k].note = s.lesson
        for j in path(tree, k):  # pass the score up to the root
            tree[j].visits, tree[j].total = tree[j].visits + 1, tree[j].total + s.score
        solved = k if s.solved else solved
    return {"tree": tree, "solved": solved, "rounds": state["rounds"] + 1}

def solved_or_out(state: State) -> str:
    # the gate: stop on success or after 10 rounds, never extend the budget
    return END if state["solved"] >= 0 or state["rounds"] >= ROUNDS else "select"

builder = StateGraph(State)
builder.add_node("select", select)
builder.add_node("expand", expand)
builder.add_node("sandbox", sandbox)
builder.add_node("evaluate", evaluate)
builder.add_edge(START, "select")
builder.add_edge("select", "expand")
builder.add_edge("expand", "sandbox")
builder.add_edge("sandbox", "evaluate")
builder.add_conditional_edges("evaluate", solved_or_out, ["select", END])
graph = builder.compile()

async def run(task: str):
    root = Node(parent=None, move="(start)", observation="initial state")
    s = await graph.ainvoke({"task": task, "tree": {0: root}, "selected": 0, "moves": [], "new": [], "rounds": 0, "solved": -1})
    tree: dict[int, Node] = s["tree"]
    avg = {k: n.total / n.visits for k, n in tree.items() if k and n.visits}
    best = s["solved"] if s["solved"] >= 0 else max(avg, key=lambda k: avg[k])
    return {"trajectory": [(tree[j].move, tree[j].observation) for j in path(tree, best)[1:]],
            "score": avg.get(best), "solved": s["solved"] >= 0, "rounds": s["rounds"],
            "lessons": [n.note for n in tree.values() if n.note]}

Replace run_in_sandbox with a truly disposable environment and a real success test; the search is only as safe as that isolation.

Claude Agent SDK, python, package claude-agent-sdk
import math
from dataclasses import dataclass, field
from typing import Any

from claude_agent_sdk import AgentDefinition, ClaudeAgentOptions, ResultMessage, query

ROUNDS, K, C = 10, 3, 1.4  # budget: 10 rounds of 3 moves

AGENTS = {
    "expander": AgentDefinition(
        description="Proposes 3 different next moves from the current state.",
        prompt=("Given the goal, the path of moves to the current state and lessons from failed branches, "
                "propose 3 different next moves, each concrete enough to run in the sandbox. "
                "Do not repeat a move already tried from this state. Do not predict the result."),
        tools=[]),
    "evaluator": AgentDefinition(
        description="Scores one new state from 0 to 10 and says whether the success test passed.",
        prompt=("Score the new state from 0 to 10 for progress toward the goal, hard signals first: tests "
                "passing, errors gone, values matching. Say whether the success test passed. If the move "
                "failed or made things worse, write one line saying why; otherwise leave the lesson empty."),
        tools=[]),
}
MOVES = {"type": "object", "required": ["moves"], "properties": {"moves": {"type": "array", "items": {"type": "string"}}}}
SCORES = {"type": "object", "required": ["scores"], "properties": {"scores": {"type": "array", "items": {
    "type": "object", "required": ["score", "solved", "lesson"],
    "properties": {"score": {"type": "number"}, "solved": {"type": "boolean"}, "lesson": {"type": "string"}}}}}}

@dataclass
class Node:  # one state of the search tree (the store)
    parent: int | None
    move: str
    observation: str
    children: list[int] = field(default_factory=list)
    visits: int = 0
    total: float = 0.0
    note: str = ""

def run_in_sandbox(moves: list[str]) -> str:
    """Stub: replay the moves on a disposable copy of the environment (container, VM, scratch repo)."""
    return f"ran {moves[-1]!r}"

async def ask(prompt: str, schema: dict[str, Any]) -> Any:
    options = ClaudeAgentOptions(agents=AGENTS, tools=["Agent"], allowed_tools=["Agent"],
                                 output_format={"type": "json_schema", "schema": schema})
    async for message in query(prompt=prompt, options=options):
        if isinstance(message, ResultMessage):
            return message.structured_output
    raise RuntimeError("no result")

def path(tree: dict[int, Node], i: int | None) -> list[int]:
    ids: list[int] = []
    while i is not None:
        ids, i = [i] + ids, tree[i].parent
    return ids

def select(tree: dict[int, Node]) -> int:
    i = 0
    def uct(c: int) -> float:  # average score plus a bonus for few visits
        n = tree[c]
        return math.inf if n.visits == 0 else n.total / n.visits + C * math.sqrt(math.log(tree[i].visits) / n.visits)
    while tree[i].children:
        i = max(tree[i].children, key=uct)
    return i

async def run(task: str):
    tree = {0: Node(parent=None, move="(start)", observation="initial state")}
    solved, rounds = -1, 0
    while solved < 0 and rounds < ROUNDS:  # the gate: success or 10 rounds, never extended
        rounds += 1
        leaf = select(tree)
        steps = "\n".join(f"- {tree[j].move} -> {tree[j].observation}" for j in path(tree, leaf))
        lessons = "\n".join(n.note for n in tree.values() if n.note)
        # the expander sees the path and failure notes, not the scores
        out = await ask(f"Use the expander sub-agent. Give it only this:\nGoal: {task}\nPath:\n{steps}\n"
                        f"Lessons:\n{lessons}\nReturn its moves.", MOVES)
        prefix = [tree[j].move for j in path(tree, leaf)[1:]]
        for move in out["moves"][:K]:
            tree[len(tree)] = Node(parent=leaf, move=move, observation=run_in_sandbox(prefix + [move]))
            tree[leaf].children.append(len(tree) - 1)
        new = tree[leaf].children
        cases = "\n".join(f"{n}. Move: {tree[k].move}\n   Observation: {tree[k].observation}" for n, k in enumerate(new))
        graded = await ask(f"Goal and success test: {task}\nLaunch one evaluator sub-agent per case below, in parallel. "
                           f"Give each only the goal, its move and its observation.\n{cases}\nReturn scores in case order.", SCORES)
        for k, s in zip(new, graded["scores"]):
            tree[k].note = s["lesson"]
            for j in path(tree, k):  # pass the score up to the root
                tree[j].visits, tree[j].total = tree[j].visits + 1, tree[j].total + s["score"]
            solved = k if s["solved"] else solved
    avg = {k: n.total / n.visits for k, n in tree.items() if k and n.visits}
    best = solved if solved >= 0 else max(avg, key=lambda k: avg[k])
    return {"trajectory": [(tree[j].move, tree[j].observation) for j in path(tree, best)[1:]],
            "score": avg.get(best), "solved": solved >= 0, "rounds": rounds,
            "lessons": [n.note for n in tree.values() if n.note]}

Replace run_in_sandbox with a truly disposable environment and a real success test; the search is only as safe as that isolation.

OpenAI Agents SDK, python, package openai-agents
import asyncio
import math
from dataclasses import dataclass, field

from agents import Agent, Runner
from pydantic import BaseModel

ROUNDS, K, C = 10, 3, 1.4  # budget: 10 rounds of 3 moves

class Moves(BaseModel):
    moves: list[str]

class Score(BaseModel):
    score: float
    solved: bool
    lesson: str

expander = Agent(
    name="Expander",
    instructions=("Given the goal, the path of moves to the current state and lessons from failed branches, "
                  "propose 3 different next moves, each concrete enough to run in the sandbox. "
                  "Do not repeat a move already tried from this state. Do not predict the result."),
    output_type=Moves)
evaluator = Agent(
    name="Evaluator",
    instructions=("Score the new state from 0 to 10 for progress toward the goal, hard signals first: tests "
                  "passing, errors gone, values matching. Say whether the success test passed. If the move "
                  "failed or made things worse, write one line saying why; otherwise leave the lesson empty."),
    output_type=Score)

@dataclass
class Node:  # one state of the search tree (the store)
    parent: int | None
    move: str
    observation: str
    children: list[int] = field(default_factory=list)
    visits: int = 0
    total: float = 0.0
    note: str = ""

def run_in_sandbox(moves: list[str]) -> str:
    """Stub: replay the moves on a disposable copy of the environment (container, VM, scratch repo)."""
    return f"ran {moves[-1]!r}"

def path(tree: dict[int, Node], i: int | None) -> list[int]:
    ids: list[int] = []
    while i is not None:
        ids, i = [i] + ids, tree[i].parent
    return ids

def select(tree: dict[int, Node]) -> int:
    i = 0
    def uct(c: int) -> float:  # average score plus a bonus for few visits
        n = tree[c]
        return math.inf if n.visits == 0 else n.total / n.visits + C * math.sqrt(math.log(tree[i].visits) / n.visits)
    while tree[i].children:
        i = max(tree[i].children, key=uct)
    return i

async def score(task: str, node: Node) -> Score:
    # fresh run per state: goal, move and observation only
    r = await Runner.run(evaluator, f"Goal and success test: {task}\nMove: {node.move}\nObservation: {node.observation}")
    return r.final_output_as(Score)

async def run(task: str):
    tree = {0: Node(parent=None, move="(start)", observation="initial state")}
    solved, rounds = -1, 0
    while solved < 0 and rounds < ROUNDS:  # the gate: success or 10 rounds, never extended
        rounds += 1
        leaf = select(tree)
        steps = "\n".join(f"- {tree[j].move} -> {tree[j].observation}" for j in path(tree, leaf))
        lessons = "\n".join(n.note for n in tree.values() if n.note)
        # the expander sees the path and failure notes, not the scores
        r = await Runner.run(expander, f"Goal: {task}\nPath:\n{steps}\nLessons:\n{lessons}")
        prefix = [tree[j].move for j in path(tree, leaf)[1:]]
        for move in r.final_output_as(Moves).moves[:K]:
            tree[len(tree)] = Node(parent=leaf, move=move, observation=run_in_sandbox(prefix + [move]))
            tree[leaf].children.append(len(tree) - 1)
        new = tree[leaf].children
        for k, s in zip(new, await asyncio.gather(*(score(task, tree[k]) for k in new))):
            tree[k].note = s.lesson
            for j in path(tree, k):  # pass the score up to the root
                tree[j].visits, tree[j].total = tree[j].visits + 1, tree[j].total + s.score
            solved = k if s.solved else solved
    avg = {k: n.total / n.visits for k, n in tree.items() if k and n.visits}
    best = solved if solved >= 0 else max(avg, key=lambda k: avg[k])
    return {"trajectory": [(tree[j].move, tree[j].observation) for j in path(tree, best)[1:]],
            "score": avg.get(best), "solved": solved >= 0, "rounds": rounds,
            "lessons": [n.note for n in tree.values() if n.note]}

Replace run_in_sandbox with a truly disposable environment and a real success test; the search is only as safe as that isolation.

Vercel AI SDK, typescript, package ai
import { anthropic } from '@ai-sdk/anthropic';
import { generateText, Output } from 'ai';
import { z } from 'zod';

const model = anthropic('claude-sonnet-5-5');
const ROUNDS = 10, K = 3, C = 1.4; // budget: 10 rounds of 3 moves

const EXPANDER =
  'Given the goal, the path of moves to the current state and lessons from failed branches, ' +
  'propose 3 different next moves, each concrete enough to run in the sandbox. ' +
  'Do not repeat a move already tried from this state. Do not predict the result.';
const EVALUATOR =
  'Score the new state from 0 to 10 for progress toward the goal, hard signals first: tests passing, ' +
  'errors gone, values matching. Say whether the success test passed. If the move failed or made ' +
  'things worse, write one line saying why; otherwise leave the lesson empty.';
const scoreSchema = z.object({ score: z.number(), solved: z.boolean(), lesson: z.string() });

type TreeNode = { parent: number | null; move: string; observation: string; children: number[]; visits: number; total: number; note: string };

// Stub: replay the moves on a disposable copy of the environment (container, VM, scratch repo).
async function runInSandbox(moves: string[]): Promise<string> {
  return `ran ${JSON.stringify(moves.at(-1))}`;
}

function path(tree: TreeNode[], i: number | null): number[] {
  const ids: number[] = [];
  for (; i !== null; i = tree[i].parent) ids.unshift(i);
  return ids;
}

function select(tree: TreeNode[]): number {
  let i = 0;
  const uct = (c: number) => { // average score plus a bonus for few visits
    const n = tree[c];
    return n.visits === 0 ? Infinity : n.total / n.visits + C * Math.sqrt(Math.log(tree[i].visits) / n.visits);
  };
  while (tree[i].children.length) i = tree[i].children.reduce((a, b) => (uct(b) > uct(a) ? b : a));
  return i;
}

export async function run(task: string) {
  const tree: TreeNode[] = [{ parent: null, move: '(start)', observation: 'initial state', children: [], visits: 0, total: 0, note: '' }];
  let solved = -1, rounds = 0;
  while (solved < 0 && rounds < ROUNDS) { // the gate: success or 10 rounds, never extended
    rounds++;
    const leaf = select(tree);
    const steps = path(tree, leaf).map((j) => `- ${tree[j].move} -> ${tree[j].observation}`).join('\n');
    const lessons = tree.filter((n) => n.note).map((n) => n.note).join('\n');
    // the expander sees the path and failure notes, not the scores
    const { output } = await generateText({
      model, instructions: EXPANDER,
      prompt: `Goal: ${task}\nPath:\n${steps}\nLessons:\n${lessons}`,
      output: Output.object({ schema: z.object({ moves: z.array(z.string()) }) }),
    });
    const prefix = path(tree, leaf).slice(1).map((j) => tree[j].move);
    for (const move of output.moves.slice(0, K)) {
      tree.push({ parent: leaf, move, observation: await runInSandbox([...prefix, move]), children: [], visits: 0, total: 0, note: '' });
      tree[leaf].children.push(tree.length - 1);
    }
    const scores = await Promise.all(tree[leaf].children.map(async (k) => (await generateText({
      model, instructions: EVALUATOR, // fresh call per state: goal, move and observation only
      prompt: `Goal and success test: ${task}\nMove: ${tree[k].move}\nObservation: ${tree[k].observation}`,
      output: Output.object({ schema: scoreSchema }),
    })).output));
    tree[leaf].children.forEach((k, n) => {
      tree[k].note = scores[n].lesson;
      for (const j of path(tree, k)) { tree[j].visits++; tree[j].total += scores[n].score; } // up to the root
      if (scores[n].solved) solved = k;
    });
  }
  const avg = (k: number) => tree[k].total / Math.max(tree[k].visits, 1);
  const best = solved >= 0 ? solved : tree.slice(1).reduce((b, _, k) => (avg(k + 1) > avg(b) ? k + 1 : b), 1);
  return {
    trajectory: path(tree, best).slice(1).map((j) => ({ move: tree[j].move, observation: tree[j].observation })),
    score: avg(best), solved: solved >= 0, rounds,
    lessons: tree.filter((n) => n.note).map((n) => n.note),
  };
}

Replace runInSandbox with a truly disposable environment and a real success test; the search is only as safe as that isolation.

Snippets are generated when the site is built and type-checked against the SDK versions listed in the repository. Treat them as a starting skeleton.

What goes wrong

  • A sandbox that leaks into real systems turns every explored branch into a real change. Check isolation before the first run.
  • Without the bonus for rarely visited states, the search locks onto the first decent branch. Without scores, it wanders.
  • Large trees fill the expander's context. Give it only the path to the selected state, never the whole tree.

Chain it

Its result can feed

Source