1What if the agent spends most of its time waiting between steps?
This changes how long a run takes, not what it returns. While the slow step runs, a fast model guesses its outcome and the call that would follow is started early. If the real outcome matches a guess, that call's result is already there. If not, the early work is thrown away.
You may need it if
- Each round, the tools sit idle for seconds while a strong model decides what to call.
- The next call is often predictable, such as a lookup that nearly always follows the previous one.
- Users wait on a multi-step task whose steps are right but slow.
2When does guessing ahead cost more than it saves?
Use it when
- The loop already returns the right results and the only complaint is the wait.
- The calls that would start early only read data, run in a sandbox or can be undone.
- A fast model can often predict the slow step, because the next move is usually routine.
- You accept paying for calls whose results are thrown away, in exchange for a shorter wait.
- The slow step is a strong model, a slow tool or a person's reply. The method is the same for each.
Avoid it when
- You hope for better answers. The result is the one the plain loop would give. A guess never replaces the real decision.
- The next calls send messages, change records or spend money, and cannot be undone or sandboxed. Simulate them and have them approved.
Use Dry Run instead - The calls of the task do not depend on each other. Plan them once and run them in parallel. Nothing needs guessing.
Use LLMCompiler instead - A fast model rarely predicts the next step, or calls are metered. Most early calls are then paid for and discarded.
Use Tool Use instead
What it costs
- Model calls
- per round: 1 agent call + 1 fast guess call, plus up to 2 tool calls started early
- Latency
- low
- Complexity
- medium
3How can a call start before the agent has asked for it?
A tool loop with a second, faster path. Each round the agent and a speculator read the same state. The speculator's guessed calls start early if they are safe to discard. When the agent's real call arrives, a matching guess supplies its result. Otherwise the guesses are dropped and the real call runs.
- Step 1 of 7
Two readers get the same state
The task is "Is customer 812 eligible for a refund?". The strong agent starts working out its first call, which takes a few seconds. At the same moment a small fast model reads the same task.
- Step 2 of 7
The speculator guesses the next call
Well before the agent is done, the speculator replies that the agent will probably look up the orders of customer 812, or else search the refund policy. Plain code checks that both are read-only lookups.
- Step 3 of 7
The guessed calls start early
Both lookups are launched now. Their results are held aside. The agent does not see them, and nothing is added to the record of the run.
- Step 4 of 7
The real decision arrives
The agent asks for the orders of customer 812. This real call is compared with the early ones on the tool and on the exact arguments. It is the first guess.
- Step 5 of 7
A match removes the wait
The order lookup is already finished or well under way, so its result becomes the result of the agent's call. The policy search is cancelled. The next round starts from the state the plain loop would have reached.
- Step 6 of 7
A miss costs only the wasted calls
Had the agent asked for the customer's plan instead, both early calls would be dropped and the plan lookup would run in the normal way. The run goes on as if no guess had been made.
- Step 7 of 7
The same answer, sooner
The agent answers when it has enough, or the round limit ends the run. Had the speculator guessed "send the refund email", that call would not have started early. The safety check holds back anything with a lasting effect.
What each part does
- Task
- The request or question, plus the tool list with each tool marked safe to start early or not.
- Rounds left?
- Plain code. Counts rounds and gives the same state, the task and the results so far, to the agent and the speculator at once.
- Agent
- The slow model whose decisions count. Decides the real next tool call or the final answer. It never sees a guess.
- Speculator
- A fast model. Guesses up to two calls the agent is likely to ask for next, and answers before the agent has finished.
- Safe to discard?
- Plain code. Lets a guessed call start only if it reads data, runs in a sandbox or can be undone.
- Calls started early
- Guessed calls already running. Their results stay pending, outside the agent's view, until a guess is confirmed.
- Guess matched?
- Compares the agent's real call with the early ones. Same tool and same arguments is a match. All other early calls are cancelled.
- Tools
- The real call, run in the normal way when no guess matched it.
- Answer
- The final reply and the record of real calls. The same as the loop without speculation would return.
- Rounds left? → Agent: state
- Rounds left? → Speculator: same state
- Speculator → Safe to discard?: guessed calls
- Safe to discard? → Calls started early: no lasting effect
- Agent → Guess matched?: real tool call
- Calls started early → Guess matched?: pending results
- Guess matched? → Rounds left?: match: early result used
- Guess matched? → Tools: no match: guesses dropped
- Tools → Rounds left?: tool result
- Agent → Answer: final answer
- Rounds left? → Answer: round limit reached
Where it connects
A request or question that one agent can handle with the listed tools. Each tool is marked safe to start early (read-only, sandboxed or undoable) or not.
When: The agent gives a final answer, or the round limit is reached.
The reply and the record of real tool calls that the same loop without speculation would have produced, with the round-limit flag. Discarded guesses leave no trace in it.
4What do I ask my assistant to set up?
The agents to launch
Tool agent
Answer the task using the listed tools. Each turn, either call exactly one tool with complete arguments, or give your final answer. Do not guess a fact a tool can give you. When you answer, list the calls you relied on.
Sees: The task, the tool descriptions, and every real call and result so far. Not the speculator's guesses or the calls started early.
Speculator
Predict the next tool call the main agent will make from this state. Give at most two guesses, most likely first, each with the exact tool name and complete arguments. Be quick. Do not solve the task and do not explain. If the agent will probably give its final answer next, return no guess.
Sees: The task, the tool descriptions and the real calls and results so far, which is the state the agent has. Not the agent's reasoning in progress.
A prompt you can paste
Works with any assistant that can launch sub-agents. It contains the agents, the procedure and the stop rule above.
You are the orchestrator of a multi-agent workflow that follows the "Speculative Actions" architecture. Launch every agent listed below as a separate sub-agent with its own context. Give each agent only the context stated for it. Do not do an agent's work yourself. If you cannot launch sub-agents, play each agent in a clearly separated turn and never let it use what is withheld from it. # Task [Describe your task here] # Speculative Actions Guess the slow step, start the next call early, keep it if right. Input: the task above. It must provide: A request or question that one agent can handle with the listed tools. Each tool is marked safe to start early (read-only, sandboxed or undoable) or not. If something essential is missing, ask for it before starting. Agents to launch: - Tool agent: Answer the task using the listed tools. Each turn, either call exactly one tool with complete arguments, or give your final answer. Do not guess a fact a tool can give you. When you answer, list the calls you relied on. Context for this agent: The task, the tool descriptions, and every real call and result so far. Not the speculator's guesses or the calls started early. - Speculator: Predict the next tool call the main agent will make from this state. Give at most two guesses, most likely first, each with the exact tool name and complete arguments. Be quick. Do not solve the task and do not explain. If the agent will probably give its final answer next, return no guess. Context for this agent: The task, the tool descriptions and the real calls and results so far, which is the state the agent has. Not the agent's reasoning in progress. Procedure: 1. Build and test the plain tool loop first. Speculation must not change what it returns. 2. Mark each tool as safe to start early or not. Safe means it only reads, it runs in a sandbox, or it can be undone. When unsure, mark it not safe. 3. At the start of each round, give the same state to the tool agent and to the speculator, which runs on a small fast model. Do not wait for one before starting the other. 4. Start each guessed call that is marked safe, at most 2 per round. Keep its pending result under a key made of the tool name and the exact arguments. 5. When the agent's real call arrives, look up its key. On a match, use the pending result as the result of the call. Otherwise run the call in the normal way. 6. In both cases, cancel the other early calls and undo any that needs it. Put only real calls in the record of the run. Count matches and discarded calls in a separate log. 7. Count rounds. After 8 real tool calls, stop running tools and ask the agent for its best final answer with what it has. Stop rule: Stop when the agent gives a final answer, or after 8 real tool calls, whichever comes first. Start at most 2 guessed calls per round. When the run ends, cancel every early call still pending. Exit: - `answer`, when: The agent gives a final answer, or the round limit is reached. Deliver: The reply and the record of real tool calls that the same loop without speculation would have produced, with the round-limit flag. Discarded guesses leave no trace in it. # Final deliverable The final answer, then each real tool call with a one-line summary of its result, and whether the round limit was reached. Separately, for tuning only, how many guesses matched and how many early calls were discarded.
The same thing in code
import asyncio
from typing import Annotated, TypedDict
from langchain.chat_models import init_chat_model
from langchain_core.messages import AIMessage, AnyMessage, HumanMessage, SystemMessage, ToolMessage
from langchain_core.tools import tool
from langgraph.graph import END, START, StateGraph
from langgraph.graph.message import add_messages
from pydantic import BaseModel
MAX_CALLS = 8 # real tool calls per run
MAX_GUESSES = 2 # calls started early per round
async def search_docs(query: str) -> str:
"""Search the product docs and return the top 3 matches. Example: search_docs(query="refund policy")"""
return "stub: 3 matches" # plug in your search index
async def get_order(order_id: str) -> str:
"""Fetch one order by id. Example: get_order(order_id="A-1042")"""
return "stub: order summary" # plug in your database, read-only
TOOLS = {"search_docs": search_docs, "get_order": get_order} # both stubs take one string argument
SAFE_EARLY = {"search_docs", "get_order"} # read-only. Never add a tool that sends, writes or pays.
TOOL_RULES = "search_docs(query): search the docs\nget_order(order_id): fetch one order"
AGENT = ("Answer the task using the listed tools. Each turn, call exactly one tool with complete "
"arguments, or give your final answer. Do not guess a fact a tool can give you.")
SPECULATOR = ("Predict the next tool call the main agent will make from this state. Give at most two "
"guesses, most likely first, each with the exact tool name and its argument. Be quick and "
"do not solve the task. If the agent will probably answer next, return no guess.")
class Guess(BaseModel):
tool: str
argument: str
class Guesses(BaseModel):
calls: list[Guess]
model = init_chat_model("anthropic:claude-sonnet-5-5")
agent_model = model.bind_tools([tool(f) for f in TOOLS.values()])
fast_model = init_chat_model("anthropic:claude-haiku-5-5").with_structured_output(Guesses)
async def ask_speculator(task: str, calls: list[str]) -> list[Guess]:
prompt = f"Task: {task}\n\nTools:\n{TOOL_RULES}\n\nReal calls so far:\n" + "\n".join(calls)
return Guesses.model_validate(await fast_model.ainvoke([SystemMessage(SPECULATOR), HumanMessage(prompt)])).calls
class Speculation:
"""The calls started early in one run, keyed by tool and argument, and the record of real calls."""
def __init__(self, task: str) -> None:
self.task = task
self.calls: list[str] = [] # real calls only: what the plain loop would record
self.pending: dict[tuple[str, str], asyncio.Task[str]] = {}
self.guessing: asyncio.Task[None] | None = None
self.matched = self.discarded = 0 # for your metrics, never part of the result
self.limit_hit = False
def start_round(self) -> None:
self.guessing = asyncio.create_task(self._guess()) # runs while the slow agent is deciding
async def _guess(self) -> None:
for g in (await ask_speculator(self.task, self.calls))[:MAX_GUESSES]:
if g.tool in SAFE_EARLY and (g.tool, g.argument) not in self.pending: # the safety gate
self.pending[(g.tool, g.argument)] = asyncio.create_task(TOOLS[g.tool](g.argument))
def drop(self) -> None:
if self.guessing:
self.guessing.cancel() # a guess that comes after the real decision is useless
for early in self.pending.values():
early.cancel()
self.discarded += len(self.pending)
self.pending.clear()
async def real_call(self, name: str, argument: str) -> str:
if len(self.calls) >= MAX_CALLS: # the round limit
self.limit_hit = True
self.drop()
return "Not run: round limit reached. Give your best final answer now."
early = self.pending.pop((name, argument), None) # a match: same tool, same argument
self.drop() # every other early call is discarded
if early:
self.matched += 1
result = await early if early else await TOOLS[name](argument)
self.calls.append(f"{name}({argument!r}) -> {result[:120]}")
self.start_round()
return result
class State(TypedDict):
messages: Annotated[list[AnyMessage], add_messages]
def build(spec: Speculation):
async def agent(state: State) -> dict:
return {"messages": [await agent_model.ainvoke([SystemMessage(AGENT), *state["messages"]])]}
async def tools(state: State) -> dict:
last = state["messages"][-1]
assert isinstance(last, AIMessage)
results = []
for call in last.tool_calls:
argument = str(next(iter(call["args"].values()), ""))
results.append(ToolMessage(await spec.real_call(call["name"], argument), tool_call_id=call["id"] or ""))
return {"messages": results}
def route(state: State) -> str:
last = state["messages"][-1]
return "tools" if isinstance(last, AIMessage) and last.tool_calls else END
builder = StateGraph(State)
builder.add_node("agent", agent)
builder.add_node("tools", tools)
builder.add_edge(START, "agent")
builder.add_conditional_edges("agent", route, ["tools", END])
builder.add_edge("tools", "agent")
return builder.compile()
async def run(task: str) -> dict:
spec = Speculation(task)
spec.start_round()
state = await build(spec).ainvoke({"messages": [HumanMessage(task)]}, {"recursion_limit": 40})
spec.drop() # cancel whatever is still pending
return {"answer": state["messages"][-1].text, "calls": spec.calls, "incomplete": spec.limit_hit}
The agent's tools go through a cache of calls started early. Guessed calls are cancelled, not undone, so list in SAFE_EARLY only tools that read. The stubs take one string argument each.
import asyncio
from typing import Any
from claude_agent_sdk import (
AgentDefinition, ClaudeAgentOptions, ResultMessage, create_sdk_mcp_server, query, tool,
)
from pydantic import BaseModel
MAX_CALLS = 8 # real tool calls per run
MAX_GUESSES = 2 # calls started early per round
async def search_docs(query: str) -> str:
return "stub: 3 matches" # plug in your search index
async def get_order(order_id: str) -> str:
return "stub: order summary" # plug in your database, read-only
TOOLS = {"search_docs": search_docs, "get_order": get_order} # both stubs take one string argument
SAFE_EARLY = {"search_docs", "get_order"} # read-only. Never add a tool that sends, writes or pays.
TOOL_NAMES = ["mcp__tools__search_docs", "mcp__tools__get_order"]
TOOL_RULES = "search_docs(query): search the docs\nget_order(order_id): fetch one order"
SPECULATOR = ("Predict the next tool call the main agent will make from this state. Give at most two "
"guesses, most likely first, each with the exact tool name and its argument. Be quick and "
"do not solve the task. If the agent will probably answer next, return no guess.")
AGENTS = {
"tool-agent": AgentDefinition(
description="Answers the task with the listed tools.",
prompt="Answer the task using the listed tools. Each turn, call exactly one tool with complete "
"arguments, or give your final answer. Do not guess a fact a tool can give you. "
"When you answer, list the calls you relied on.",
tools=TOOL_NAMES,
),
}
class Guess(BaseModel):
tool: str
argument: str
class Guesses(BaseModel):
calls: list[Guess]
async def ask_speculator(task: str, calls: list[str]) -> list[Guess]:
# the speculator is its own short session on a small fast model
options = ClaudeAgentOptions(system_prompt=SPECULATOR, model="haiku",
output_format={"type": "json_schema", "schema": Guesses.model_json_schema()})
prompt = f"Task: {task}\n\nTools:\n{TOOL_RULES}\n\nReal calls so far:\n" + "\n".join(calls)
async for msg in query(prompt=prompt, options=options):
if isinstance(msg, ResultMessage):
return Guesses.model_validate(msg.structured_output).calls
return []
class Speculation:
"""The calls started early in one run, keyed by tool and argument, and the record of real calls."""
def __init__(self, task: str) -> None:
self.task = task
self.calls: list[str] = [] # real calls only: what the plain loop would record
self.pending: dict[tuple[str, str], asyncio.Task[str]] = {}
self.guessing: asyncio.Task[None] | None = None
self.matched = self.discarded = 0 # for your metrics, never part of the result
self.limit_hit = False
def start_round(self) -> None:
self.guessing = asyncio.create_task(self._guess()) # runs while the slow agent is deciding
async def _guess(self) -> None:
for g in (await ask_speculator(self.task, self.calls))[:MAX_GUESSES]:
if g.tool in SAFE_EARLY and (g.tool, g.argument) not in self.pending: # the safety gate
self.pending[(g.tool, g.argument)] = asyncio.create_task(TOOLS[g.tool](g.argument))
def drop(self) -> None:
if self.guessing:
self.guessing.cancel() # a guess that comes after the real decision is useless
for early in self.pending.values():
early.cancel()
self.discarded += len(self.pending)
self.pending.clear()
async def real_call(self, name: str, argument: str) -> dict[str, Any]:
if len(self.calls) >= MAX_CALLS: # the round limit
self.limit_hit = True
self.drop()
result = "Not run: round limit reached. Give your best final answer now."
else:
early = self.pending.pop((name, argument), None) # a match: same tool, same argument
self.drop() # every other early call is discarded
if early:
self.matched += 1
result = await early if early else await TOOLS[name](argument)
self.calls.append(f"{name}({argument!r}) -> {result[:120]}")
self.start_round()
return {"content": [{"type": "text", "text": result}]}
async def run(task: str) -> dict:
spec = Speculation(task)
@tool("search_docs", 'Search the product docs, top 3 matches. Example: {"query": "refund policy"}', {"query": str})
async def search_docs_tool(args: dict[str, Any]) -> dict[str, Any]:
return await spec.real_call("search_docs", args["query"])
@tool("get_order", 'Fetch one order by id. Example: {"order_id": "A-1042"}', {"order_id": str})
async def get_order_tool(args: dict[str, Any]) -> dict[str, Any]:
return await spec.real_call("get_order", args["order_id"])
options = ClaudeAgentOptions(
agents=AGENTS,
mcp_servers={"tools": create_sdk_mcp_server("tools", tools=[search_docs_tool, get_order_tool])},
allowed_tools=["Agent", *TOOL_NAMES],
)
spec.start_round()
answer = ""
async for msg in query(prompt=f"Use the tool-agent agent on this task and return its answer:\n\n{task}", options=options):
if isinstance(msg, ResultMessage):
answer = msg.result or ""
spec.drop() # cancel whatever is still pending
return {"answer": answer, "calls": spec.calls, "incomplete": spec.limit_hit}
The agent's tools go through a cache of calls started early. Guessed calls are cancelled, not undone, so list in SAFE_EARLY only tools that read. The stubs take one string argument each.
import asyncio
from agents import Agent, RunContextWrapper, Runner, function_tool
from pydantic import BaseModel
MAX_CALLS = 8 # real tool calls per run
MAX_GUESSES = 2 # calls started early per round
async def lookup_docs(query: str) -> str:
return "stub: 3 matches" # plug in your search index
async def lookup_order(order_id: str) -> str:
return "stub: order summary" # plug in your database, read-only
TOOLS = {"search_docs": lookup_docs, "get_order": lookup_order} # both stubs take one string argument
SAFE_EARLY = {"search_docs", "get_order"} # read-only. Never add a tool that sends, writes or pays.
TOOL_RULES = "search_docs(query): search the docs\nget_order(order_id): fetch one order"
class Guess(BaseModel):
tool: str
argument: str
class Guesses(BaseModel):
calls: list[Guess]
speculator = Agent( # set model= to a small fast model: the guess must arrive before the real decision
name="Speculator",
instructions="Predict the next tool call the main agent will make from this state. Give at most two "
"guesses, most likely first, each with the exact tool name and its argument. Be quick and "
"do not solve the task. If the agent will probably answer next, return no guess.",
output_type=Guesses,
)
class Speculation:
"""The calls started early in one run, keyed by tool and argument, and the record of real calls."""
def __init__(self, task: str) -> None:
self.task = task
self.calls: list[str] = [] # real calls only: what the plain loop would record
self.pending: dict[tuple[str, str], asyncio.Task[str]] = {}
self.guessing: asyncio.Task[None] | None = None
self.matched = self.discarded = 0 # for your metrics, never part of the result
self.limit_hit = False
def start_round(self) -> None:
self.guessing = asyncio.create_task(self._guess()) # runs while the slow agent is deciding
async def _guess(self) -> None:
prompt = f"Task: {self.task}\n\nTools:\n{TOOL_RULES}\n\nReal calls so far:\n" + "\n".join(self.calls)
guesses = (await Runner.run(speculator, prompt)).final_output_as(Guesses).calls
for g in guesses[:MAX_GUESSES]:
if g.tool in SAFE_EARLY and (g.tool, g.argument) not in self.pending: # the safety gate
self.pending[(g.tool, g.argument)] = asyncio.create_task(TOOLS[g.tool](g.argument))
def drop(self) -> None:
if self.guessing:
self.guessing.cancel() # a guess that comes after the real decision is useless
for early in self.pending.values():
early.cancel()
self.discarded += len(self.pending)
self.pending.clear()
async def real_call(self, name: str, argument: str) -> str:
if len(self.calls) >= MAX_CALLS: # the round limit
self.limit_hit = True
self.drop()
return "Not run: round limit reached. Give your best final answer now."
early = self.pending.pop((name, argument), None) # a match: same tool, same argument
self.drop() # every other early call is discarded
if early:
self.matched += 1
result = await early if early else await TOOLS[name](argument)
self.calls.append(f"{name}({argument!r}) -> {result[:120]}")
self.start_round()
return result
@function_tool
async def search_docs(ctx: RunContextWrapper[Speculation], query: str) -> str:
"""Search the product docs and return the top 3 matches. Example: query="refund policy"."""
return await ctx.context.real_call("search_docs", query)
@function_tool
async def get_order(ctx: RunContextWrapper[Speculation], order_id: str) -> str:
"""Fetch one order by id. Example: order_id="A-1042"."""
return await ctx.context.real_call("get_order", order_id)
tool_agent = Agent[Speculation](
name="Tool agent",
instructions="Answer the task using the listed tools. Each turn, call exactly one tool with complete "
"arguments, or give your final answer. Do not guess a fact a tool can give you. "
"When you answer, list the calls you relied on.",
tools=[search_docs, get_order],
)
async def run(task: str) -> dict:
spec = Speculation(task)
spec.start_round()
result = await Runner.run(tool_agent, task, context=spec, max_turns=MAX_CALLS + 6)
spec.drop() # cancel whatever is still pending
return {"answer": result.final_output, "calls": spec.calls, "incomplete": spec.limit_hit}
The agent's tools go through a cache of calls started early. Guessed calls are cancelled, not undone, so list in SAFE_EARLY only tools that read. The stubs take one string argument each.
import { anthropic } from '@ai-sdk/anthropic';
import { isStepCount, Output, tool, ToolLoopAgent } from 'ai';
import { z } from 'zod';
const MAX_CALLS = 8; // real tool calls per run
const MAX_GUESSES = 2; // calls started early per round
const model = anthropic('claude-sonnet-5-5');
// stubs: plug in your search index and your database. Both take one string argument.
const TOOLS: Record<string, (argument: string) => Promise<string>> = {
searchDocs: async (query) => `stub: 3 matches for ${query}`,
getOrder: async (orderId) => `stub: summary of ${orderId}`,
};
const SAFE_EARLY = new Set(['searchDocs', 'getOrder']); // read-only. Never add a tool that sends, writes or pays.
const TOOL_RULES = 'searchDocs(query): search the docs\ngetOrder(orderId): fetch one order';
const speculator = new ToolLoopAgent({
model: anthropic('claude-haiku-5-5'), // small and fast: the guess must arrive before the real decision
instructions:
'Predict the next tool call the main agent will make from this state. Give at most two guesses, ' +
'most likely first, each with the exact tool name and its argument. Be quick and do not solve ' +
'the task. If the agent will probably answer next, return no guess.',
output: Output.object({ schema: z.object({ calls: z.array(z.object({ tool: z.string(), argument: z.string() })) }) }),
});
export async function run(task: string) {
const calls: string[] = []; // real calls only: what the plain loop would record
const pending = new Map<string, Promise<string>>(); // calls started early, keyed by tool and argument
const stats = { matched: 0, discarded: 0 }; // for your metrics, never part of the result
let round = 0;
let limitHit = false;
const drop = () => {
round++; // a guess that comes after the real decision is ignored
stats.discarded += pending.size; // pass an AbortSignal to your tools to stop them for real
pending.clear();
};
async function guess() {
const mine = round;
const { output } = await speculator.generate({
prompt: `Task: ${task}\n\nTools:\n${TOOL_RULES}\n\nReal calls so far:\n${calls.join('\n')}`,
});
if (mine !== round) return;
for (const g of output.calls.slice(0, MAX_GUESSES)) {
const key = `${g.tool}:${g.argument}`;
if (!SAFE_EARLY.has(g.tool) || pending.has(key)) continue; // the safety gate
const early = TOOLS[g.tool](g.argument);
early.catch(() => {}); // a failed guess must not break the run
pending.set(key, early);
}
}
const startRound = () => void guess().catch(() => {}); // runs while the slow agent is deciding
async function realCall(name: string, argument: string) {
if (calls.length >= MAX_CALLS) { // the round limit
limitHit = true;
drop();
return 'Not run: round limit reached. Give your best final answer now.';
}
const key = `${name}:${argument}`;
const early = pending.get(key); // a match: same tool, same argument
pending.delete(key);
drop(); // every other early call is discarded
if (early) stats.matched++;
const result = await (early ?? TOOLS[name](argument));
calls.push(`${name}(${argument}) -> ${result.slice(0, 120)}`);
startRound();
return result;
}
const agent = new ToolLoopAgent({
model,
instructions:
'Answer the task using the listed tools. Each turn, call exactly one tool with complete ' +
'arguments, or give your final answer. Do not guess a fact a tool can give you. ' +
'When you answer, list the calls you relied on.',
tools: {
searchDocs: tool({
description: 'Search the product docs, top 3 matches. Example: { query: "refund policy" }',
inputSchema: z.object({ query: z.string() }),
execute: ({ query }) => realCall('searchDocs', query),
}),
getOrder: tool({
description: 'Fetch one order by id. Example: { orderId: "A-1042" }',
inputSchema: z.object({ orderId: z.string() }),
execute: ({ orderId }) => realCall('getOrder', orderId),
}),
},
prepareStep: async () => (limitHit ? { toolChoice: 'none' as const } : {}), // at the limit, answer only
stopWhen: isStepCount(MAX_CALLS + 6),
});
startRound();
const { text } = await agent.generate({ prompt: task });
drop();
console.info('speculation', stats);
return { answer: text, calls, incomplete: limitHit };
}
The agent's tools go through a cache of calls started early. A dropped promise still runs to its end here, so list in SAFE_EARLY only tools that read. The stubs take one string argument.
Snippets are generated when the site is built and type-checked against the SDK versions listed in the repository. Treat them as a starting skeleton.
What goes wrong
- Starting a guessed call that sends, writes or pays is unsafe, because a wrong guess has already acted. Speculate only on calls that are read-only, sandboxed or that you can undo.
- Matching on the tool name alone hands the agent the result of a different call. Require the same tool and the same arguments.
- If the agent sees guesses or early results, they change its decisions, and the run is no longer the one the plain loop would give.
- Every miss is a call you paid for. Cap the guesses per round, watch the share that is discarded, and switch speculation off where it rarely matches.
- A read whose result changes from one moment to the next, such as a live price, may not equal the call the agent would have made. Mark such tools not safe.
Chain it
Its result can feed
- Reflection conditional
Condition: Use only when A's payload is a written informational reply, not a short value and not the outcome of tool calls that changed state.
- Chain of Verification conditional
Condition: Use only when A's payload is a reply to an informational question, not the outcome of tool calls that changed state.
- Debate conditional
Condition: Use only when A's payload is an answer to a question or decision, not the outcome of tool calls that changed state.
- Constitutional Critique conditional
Condition: Use only when A's payload is a textual reply, not the outcome of tool calls that changed state.
- Textual Backpropagation conditional
Condition: Use only when A's payload is a written informational reply, not a short value and not the outcome of tool calls that changed state.