Agentic Architectures

Catalogue / Tools & Action

Speculative Actions

Guess the slow step, start the next call early, keep it if right

Shape
loop
Model calls
per round: 1 agent call + 1 fast guess call, plus up to 2 tool calls started early
Latency
low
Complexity
medium

Also known as Speculative execution for agents, Interactive speculative planning, Speculative tool calling

1

What if the agent spends most of its time waiting between steps?

This changes how long a run takes, not what it returns. While the slow step runs, a fast model guesses its outcome and the call that would follow is started early. If the real outcome matches a guess, that call's result is already there. If not, the early work is thrown away.

You may need it if

  • Each round, the tools sit idle for seconds while a strong model decides what to call.
  • The next call is often predictable, such as a lookup that nearly always follows the previous one.
  • Users wait on a multi-step task whose steps are right but slow.
2

When does guessing ahead cost more than it saves?

Use it when

  • The loop already returns the right results and the only complaint is the wait.
  • The calls that would start early only read data, run in a sandbox or can be undone.
  • A fast model can often predict the slow step, because the next move is usually routine.
  • You accept paying for calls whose results are thrown away, in exchange for a shorter wait.
  • The slow step is a strong model, a slow tool or a person's reply. The method is the same for each.

Avoid it when

  • You hope for better answers. The result is the one the plain loop would give. A guess never replaces the real decision.
  • The next calls send messages, change records or spend money, and cannot be undone or sandboxed. Simulate them and have them approved.
    Use Dry Run instead
  • The calls of the task do not depend on each other. Plan them once and run them in parallel. Nothing needs guessing.
    Use LLMCompiler instead
  • A fast model rarely predicts the next step, or calls are metered. Most early calls are then paid for and discarded.
    Use Tool Use instead

What it costs

Model calls
per round: 1 agent call + 1 fast guess call, plus up to 2 tool calls started early
Latency
low
Complexity
medium
3

How can a call start before the agent has asked for it?

A tool loop with a second, faster path. Each round the agent and a speculator read the same state. The speculator's guessed calls start early if they are safe to discard. When the agent's real call arrives, a matching guess supplies its result. Otherwise the guesses are dropped and the real call runs.

no lastingeffectreal tool callmatch: earlyresult usedno match:guesses droppedtool resultfinal answerround limitreachedTask: The request or question, plus the tool list with each tool marked safe to start early or not.TaskRounds left?: Plain code. Counts rounds and gives the same state, the task and the results so far, to the agent and the speculator at once.Rounds left?Agent: The slow model whose decisions count. Decides the real next tool call or the final answer. It never sees a guess.AgentSpeculator: A fast model. Guesses up to two calls the agent is likely to ask for next, and answers before the agent has finished.SpeculatorSafe to discard?: Plain code. Lets a guessed call start only if it reads data, runs in a sandbox or can be undone.Safe todiscard?Calls started early: Guessed calls already running. Their results stay pending, outside the agent's view, until a guess is confirmed.Calls startedearlyGuess matched?: Compares the agent's real call with the early ones. Same tool and same arguments is a match. All other early calls are cancelled.Guessmatched?Tools: The real call, run in the normal way when no guess matched it.ToolsAnswer: The final reply and the record of real calls. The same as the loop without speculation would return.Answer
  1. Step 1 of 7

    Two readers get the same state

    The task is "Is customer 812 eligible for a refund?". The strong agent starts working out its first call, which takes a few seconds. At the same moment a small fast model reads the same task.

  2. Step 2 of 7

    The speculator guesses the next call

    Well before the agent is done, the speculator replies that the agent will probably look up the orders of customer 812, or else search the refund policy. Plain code checks that both are read-only lookups.

  3. Step 3 of 7

    The guessed calls start early

    Both lookups are launched now. Their results are held aside. The agent does not see them, and nothing is added to the record of the run.

  4. Step 4 of 7

    The real decision arrives

    The agent asks for the orders of customer 812. This real call is compared with the early ones on the tool and on the exact arguments. It is the first guess.

  5. Step 5 of 7

    A match removes the wait

    The order lookup is already finished or well under way, so its result becomes the result of the agent's call. The policy search is cancelled. The next round starts from the state the plain loop would have reached.

  6. Step 6 of 7

    A miss costs only the wasted calls

    Had the agent asked for the customer's plan instead, both early calls would be dropped and the plan lookup would run in the normal way. The run goes on as if no guess had been made.

  7. Step 7 of 7

    The same answer, sooner

    The agent answers when it has enough, or the round limit ends the run. Had the speculator guessed "send the refund email", that call would not have started early. The safety check holds back anything with a lasting effect.

What each part does
Task
The request or question, plus the tool list with each tool marked safe to start early or not.
Rounds left?
Plain code. Counts rounds and gives the same state, the task and the results so far, to the agent and the speculator at once.
Agent
The slow model whose decisions count. Decides the real next tool call or the final answer. It never sees a guess.
Speculator
A fast model. Guesses up to two calls the agent is likely to ask for next, and answers before the agent has finished.
Safe to discard?
Plain code. Lets a guessed call start only if it reads data, runs in a sandbox or can be undone.
Calls started early
Guessed calls already running. Their results stay pending, outside the agent's view, until a guess is confirmed.
Guess matched?
Compares the agent's real call with the early ones. Same tool and same arguments is a match. All other early calls are cancelled.
Tools
The real call, run in the normal way when no guess matched it.
Answer
The final reply and the record of real calls. The same as the loop without speculation would return.
  • Rounds left? → Agent: state
  • Rounds left? → Speculator: same state
  • Speculator → Safe to discard?: guessed calls
  • Safe to discard? → Calls started early: no lasting effect
  • Agent → Guess matched?: real tool call
  • Calls started early → Guess matched?: pending results
  • Guess matched? → Rounds left?: match: early result used
  • Guess matched? → Tools: no match: guesses dropped
  • Tools → Rounds left?: tool result
  • Agent → Answer: final answer
  • Rounds left? → Answer: round limit reached

Where it connects

Entry
task
TaskQuestion

A request or question that one agent can handle with the listed tools. Each tool is marked safe to start early (read-only, sandboxed or undoable) or not.

Exit
answer
AnswerOutcome

When: The agent gives a final answer, or the round limit is reached.

The reply and the record of real tool calls that the same loop without speculation would have produced, with the round-limit flag. Discarded guesses leave no trace in it.

4

What do I ask my assistant to set up?

The agents to launch

Tool agent

Answer the task using the listed tools. Each turn, either call exactly one tool with complete arguments, or give your final answer. Do not guess a fact a tool can give you. When you answer, list the calls you relied on.

Sees: The task, the tool descriptions, and every real call and result so far. Not the speculator's guesses or the calls started early.

Speculator

Predict the next tool call the main agent will make from this state. Give at most two guesses, most likely first, each with the exact tool name and complete arguments. Be quick. Do not solve the task and do not explain. If the agent will probably give its final answer next, return no guess.

Sees: The task, the tool descriptions and the real calls and results so far, which is the state the agent has. Not the agent's reasoning in progress.

A prompt you can paste

Works with any assistant that can launch sub-agents. It contains the agents, the procedure and the stop rule above.

Framework-agnostic prompt
You are the orchestrator of a multi-agent workflow that follows the "Speculative Actions" architecture.
Launch every agent listed below as a separate sub-agent with its own context. Give each agent only the context stated for it. Do not do an agent's work yourself. If you cannot launch sub-agents, play each agent in a clearly separated turn and never let it use what is withheld from it.

# Task
[Describe your task here]

# Speculative Actions
Guess the slow step, start the next call early, keep it if right.
Input: the task above. It must provide: A request or question that one agent can handle with the listed tools. Each tool is marked safe to start early (read-only, sandboxed or undoable) or not. If something essential is missing, ask for it before starting.

Agents to launch:
- Tool agent: Answer the task using the listed tools. Each turn, either call exactly one tool with complete arguments, or give your final answer. Do not guess a fact a tool can give you. When you answer, list the calls you relied on.
  Context for this agent: The task, the tool descriptions, and every real call and result so far. Not the speculator's guesses or the calls started early.
- Speculator: Predict the next tool call the main agent will make from this state. Give at most two guesses, most likely first, each with the exact tool name and complete arguments. Be quick. Do not solve the task and do not explain. If the agent will probably give its final answer next, return no guess.
  Context for this agent: The task, the tool descriptions and the real calls and results so far, which is the state the agent has. Not the agent's reasoning in progress.

Procedure:
1. Build and test the plain tool loop first. Speculation must not change what it returns.
2. Mark each tool as safe to start early or not. Safe means it only reads, it runs in a sandbox, or it can be undone. When unsure, mark it not safe.
3. At the start of each round, give the same state to the tool agent and to the speculator, which runs on a small fast model. Do not wait for one before starting the other.
4. Start each guessed call that is marked safe, at most 2 per round. Keep its pending result under a key made of the tool name and the exact arguments.
5. When the agent's real call arrives, look up its key. On a match, use the pending result as the result of the call. Otherwise run the call in the normal way.
6. In both cases, cancel the other early calls and undo any that needs it. Put only real calls in the record of the run. Count matches and discarded calls in a separate log.
7. Count rounds. After 8 real tool calls, stop running tools and ask the agent for its best final answer with what it has.

Stop rule: Stop when the agent gives a final answer, or after 8 real tool calls, whichever comes first. Start at most 2 guessed calls per round. When the run ends, cancel every early call still pending.

Exit:
- `answer`, when: The agent gives a final answer, or the round limit is reached. Deliver: The reply and the record of real tool calls that the same loop without speculation would have produced, with the round-limit flag. Discarded guesses leave no trace in it.

# Final deliverable
The final answer, then each real tool call with a one-line summary of its result, and whether the round limit was reached. Separately, for tuning only, how many guesses matched and how many early calls were discarded.

The same thing in code

LangGraph, python, package langgraph
import asyncio
from typing import Annotated, TypedDict

from langchain.chat_models import init_chat_model
from langchain_core.messages import AIMessage, AnyMessage, HumanMessage, SystemMessage, ToolMessage
from langchain_core.tools import tool
from langgraph.graph import END, START, StateGraph
from langgraph.graph.message import add_messages
from pydantic import BaseModel

MAX_CALLS = 8  # real tool calls per run
MAX_GUESSES = 2  # calls started early per round


async def search_docs(query: str) -> str:
    """Search the product docs and return the top 3 matches. Example: search_docs(query="refund policy")"""
    return "stub: 3 matches"  # plug in your search index


async def get_order(order_id: str) -> str:
    """Fetch one order by id. Example: get_order(order_id="A-1042")"""
    return "stub: order summary"  # plug in your database, read-only


TOOLS = {"search_docs": search_docs, "get_order": get_order}  # both stubs take one string argument
SAFE_EARLY = {"search_docs", "get_order"}  # read-only. Never add a tool that sends, writes or pays.
TOOL_RULES = "search_docs(query): search the docs\nget_order(order_id): fetch one order"
AGENT = ("Answer the task using the listed tools. Each turn, call exactly one tool with complete "
         "arguments, or give your final answer. Do not guess a fact a tool can give you.")
SPECULATOR = ("Predict the next tool call the main agent will make from this state. Give at most two "
              "guesses, most likely first, each with the exact tool name and its argument. Be quick and "
              "do not solve the task. If the agent will probably answer next, return no guess.")


class Guess(BaseModel):
    tool: str
    argument: str


class Guesses(BaseModel):
    calls: list[Guess]


model = init_chat_model("anthropic:claude-sonnet-5-5")
agent_model = model.bind_tools([tool(f) for f in TOOLS.values()])
fast_model = init_chat_model("anthropic:claude-haiku-5-5").with_structured_output(Guesses)


async def ask_speculator(task: str, calls: list[str]) -> list[Guess]:
    prompt = f"Task: {task}\n\nTools:\n{TOOL_RULES}\n\nReal calls so far:\n" + "\n".join(calls)
    return Guesses.model_validate(await fast_model.ainvoke([SystemMessage(SPECULATOR), HumanMessage(prompt)])).calls


class Speculation:
    """The calls started early in one run, keyed by tool and argument, and the record of real calls."""

    def __init__(self, task: str) -> None:
        self.task = task
        self.calls: list[str] = []  # real calls only: what the plain loop would record
        self.pending: dict[tuple[str, str], asyncio.Task[str]] = {}
        self.guessing: asyncio.Task[None] | None = None
        self.matched = self.discarded = 0  # for your metrics, never part of the result
        self.limit_hit = False

    def start_round(self) -> None:
        self.guessing = asyncio.create_task(self._guess())  # runs while the slow agent is deciding

    async def _guess(self) -> None:
        for g in (await ask_speculator(self.task, self.calls))[:MAX_GUESSES]:
            if g.tool in SAFE_EARLY and (g.tool, g.argument) not in self.pending:  # the safety gate
                self.pending[(g.tool, g.argument)] = asyncio.create_task(TOOLS[g.tool](g.argument))

    def drop(self) -> None:
        if self.guessing:
            self.guessing.cancel()  # a guess that comes after the real decision is useless
        for early in self.pending.values():
            early.cancel()
        self.discarded += len(self.pending)
        self.pending.clear()

    async def real_call(self, name: str, argument: str) -> str:
        if len(self.calls) >= MAX_CALLS:  # the round limit
            self.limit_hit = True
            self.drop()
            return "Not run: round limit reached. Give your best final answer now."
        early = self.pending.pop((name, argument), None)  # a match: same tool, same argument
        self.drop()  # every other early call is discarded
        if early:
            self.matched += 1
        result = await early if early else await TOOLS[name](argument)
        self.calls.append(f"{name}({argument!r}) -> {result[:120]}")
        self.start_round()
        return result


class State(TypedDict):
    messages: Annotated[list[AnyMessage], add_messages]


def build(spec: Speculation):
    async def agent(state: State) -> dict:
        return {"messages": [await agent_model.ainvoke([SystemMessage(AGENT), *state["messages"]])]}

    async def tools(state: State) -> dict:
        last = state["messages"][-1]
        assert isinstance(last, AIMessage)
        results = []
        for call in last.tool_calls:
            argument = str(next(iter(call["args"].values()), ""))
            results.append(ToolMessage(await spec.real_call(call["name"], argument), tool_call_id=call["id"] or ""))
        return {"messages": results}

    def route(state: State) -> str:
        last = state["messages"][-1]
        return "tools" if isinstance(last, AIMessage) and last.tool_calls else END

    builder = StateGraph(State)
    builder.add_node("agent", agent)
    builder.add_node("tools", tools)
    builder.add_edge(START, "agent")
    builder.add_conditional_edges("agent", route, ["tools", END])
    builder.add_edge("tools", "agent")
    return builder.compile()


async def run(task: str) -> dict:
    spec = Speculation(task)
    spec.start_round()
    state = await build(spec).ainvoke({"messages": [HumanMessage(task)]}, {"recursion_limit": 40})
    spec.drop()  # cancel whatever is still pending
    return {"answer": state["messages"][-1].text, "calls": spec.calls, "incomplete": spec.limit_hit}

The agent's tools go through a cache of calls started early. Guessed calls are cancelled, not undone, so list in SAFE_EARLY only tools that read. The stubs take one string argument each.

Claude Agent SDK, python, package claude-agent-sdk
import asyncio
from typing import Any

from claude_agent_sdk import (
    AgentDefinition, ClaudeAgentOptions, ResultMessage, create_sdk_mcp_server, query, tool,
)
from pydantic import BaseModel

MAX_CALLS = 8  # real tool calls per run
MAX_GUESSES = 2  # calls started early per round


async def search_docs(query: str) -> str:
    return "stub: 3 matches"  # plug in your search index


async def get_order(order_id: str) -> str:
    return "stub: order summary"  # plug in your database, read-only


TOOLS = {"search_docs": search_docs, "get_order": get_order}  # both stubs take one string argument
SAFE_EARLY = {"search_docs", "get_order"}  # read-only. Never add a tool that sends, writes or pays.
TOOL_NAMES = ["mcp__tools__search_docs", "mcp__tools__get_order"]
TOOL_RULES = "search_docs(query): search the docs\nget_order(order_id): fetch one order"
SPECULATOR = ("Predict the next tool call the main agent will make from this state. Give at most two "
              "guesses, most likely first, each with the exact tool name and its argument. Be quick and "
              "do not solve the task. If the agent will probably answer next, return no guess.")
AGENTS = {
    "tool-agent": AgentDefinition(
        description="Answers the task with the listed tools.",
        prompt="Answer the task using the listed tools. Each turn, call exactly one tool with complete "
               "arguments, or give your final answer. Do not guess a fact a tool can give you. "
               "When you answer, list the calls you relied on.",
        tools=TOOL_NAMES,
    ),
}


class Guess(BaseModel):
    tool: str
    argument: str


class Guesses(BaseModel):
    calls: list[Guess]


async def ask_speculator(task: str, calls: list[str]) -> list[Guess]:
    # the speculator is its own short session on a small fast model
    options = ClaudeAgentOptions(system_prompt=SPECULATOR, model="haiku",
                                 output_format={"type": "json_schema", "schema": Guesses.model_json_schema()})
    prompt = f"Task: {task}\n\nTools:\n{TOOL_RULES}\n\nReal calls so far:\n" + "\n".join(calls)
    async for msg in query(prompt=prompt, options=options):
        if isinstance(msg, ResultMessage):
            return Guesses.model_validate(msg.structured_output).calls
    return []


class Speculation:
    """The calls started early in one run, keyed by tool and argument, and the record of real calls."""

    def __init__(self, task: str) -> None:
        self.task = task
        self.calls: list[str] = []  # real calls only: what the plain loop would record
        self.pending: dict[tuple[str, str], asyncio.Task[str]] = {}
        self.guessing: asyncio.Task[None] | None = None
        self.matched = self.discarded = 0  # for your metrics, never part of the result
        self.limit_hit = False

    def start_round(self) -> None:
        self.guessing = asyncio.create_task(self._guess())  # runs while the slow agent is deciding

    async def _guess(self) -> None:
        for g in (await ask_speculator(self.task, self.calls))[:MAX_GUESSES]:
            if g.tool in SAFE_EARLY and (g.tool, g.argument) not in self.pending:  # the safety gate
                self.pending[(g.tool, g.argument)] = asyncio.create_task(TOOLS[g.tool](g.argument))

    def drop(self) -> None:
        if self.guessing:
            self.guessing.cancel()  # a guess that comes after the real decision is useless
        for early in self.pending.values():
            early.cancel()
        self.discarded += len(self.pending)
        self.pending.clear()

    async def real_call(self, name: str, argument: str) -> dict[str, Any]:
        if len(self.calls) >= MAX_CALLS:  # the round limit
            self.limit_hit = True
            self.drop()
            result = "Not run: round limit reached. Give your best final answer now."
        else:
            early = self.pending.pop((name, argument), None)  # a match: same tool, same argument
            self.drop()  # every other early call is discarded
            if early:
                self.matched += 1
            result = await early if early else await TOOLS[name](argument)
            self.calls.append(f"{name}({argument!r}) -> {result[:120]}")
            self.start_round()
        return {"content": [{"type": "text", "text": result}]}


async def run(task: str) -> dict:
    spec = Speculation(task)

    @tool("search_docs", 'Search the product docs, top 3 matches. Example: {"query": "refund policy"}', {"query": str})
    async def search_docs_tool(args: dict[str, Any]) -> dict[str, Any]:
        return await spec.real_call("search_docs", args["query"])

    @tool("get_order", 'Fetch one order by id. Example: {"order_id": "A-1042"}', {"order_id": str})
    async def get_order_tool(args: dict[str, Any]) -> dict[str, Any]:
        return await spec.real_call("get_order", args["order_id"])

    options = ClaudeAgentOptions(
        agents=AGENTS,
        mcp_servers={"tools": create_sdk_mcp_server("tools", tools=[search_docs_tool, get_order_tool])},
        allowed_tools=["Agent", *TOOL_NAMES],
    )
    spec.start_round()
    answer = ""
    async for msg in query(prompt=f"Use the tool-agent agent on this task and return its answer:\n\n{task}", options=options):
        if isinstance(msg, ResultMessage):
            answer = msg.result or ""
    spec.drop()  # cancel whatever is still pending
    return {"answer": answer, "calls": spec.calls, "incomplete": spec.limit_hit}

The agent's tools go through a cache of calls started early. Guessed calls are cancelled, not undone, so list in SAFE_EARLY only tools that read. The stubs take one string argument each.

OpenAI Agents SDK, python, package openai-agents
import asyncio

from agents import Agent, RunContextWrapper, Runner, function_tool
from pydantic import BaseModel

MAX_CALLS = 8  # real tool calls per run
MAX_GUESSES = 2  # calls started early per round


async def lookup_docs(query: str) -> str:
    return "stub: 3 matches"  # plug in your search index


async def lookup_order(order_id: str) -> str:
    return "stub: order summary"  # plug in your database, read-only


TOOLS = {"search_docs": lookup_docs, "get_order": lookup_order}  # both stubs take one string argument
SAFE_EARLY = {"search_docs", "get_order"}  # read-only. Never add a tool that sends, writes or pays.
TOOL_RULES = "search_docs(query): search the docs\nget_order(order_id): fetch one order"


class Guess(BaseModel):
    tool: str
    argument: str


class Guesses(BaseModel):
    calls: list[Guess]


speculator = Agent(  # set model= to a small fast model: the guess must arrive before the real decision
    name="Speculator",
    instructions="Predict the next tool call the main agent will make from this state. Give at most two "
                 "guesses, most likely first, each with the exact tool name and its argument. Be quick and "
                 "do not solve the task. If the agent will probably answer next, return no guess.",
    output_type=Guesses,
)


class Speculation:
    """The calls started early in one run, keyed by tool and argument, and the record of real calls."""

    def __init__(self, task: str) -> None:
        self.task = task
        self.calls: list[str] = []  # real calls only: what the plain loop would record
        self.pending: dict[tuple[str, str], asyncio.Task[str]] = {}
        self.guessing: asyncio.Task[None] | None = None
        self.matched = self.discarded = 0  # for your metrics, never part of the result
        self.limit_hit = False

    def start_round(self) -> None:
        self.guessing = asyncio.create_task(self._guess())  # runs while the slow agent is deciding

    async def _guess(self) -> None:
        prompt = f"Task: {self.task}\n\nTools:\n{TOOL_RULES}\n\nReal calls so far:\n" + "\n".join(self.calls)
        guesses = (await Runner.run(speculator, prompt)).final_output_as(Guesses).calls
        for g in guesses[:MAX_GUESSES]:
            if g.tool in SAFE_EARLY and (g.tool, g.argument) not in self.pending:  # the safety gate
                self.pending[(g.tool, g.argument)] = asyncio.create_task(TOOLS[g.tool](g.argument))

    def drop(self) -> None:
        if self.guessing:
            self.guessing.cancel()  # a guess that comes after the real decision is useless
        for early in self.pending.values():
            early.cancel()
        self.discarded += len(self.pending)
        self.pending.clear()

    async def real_call(self, name: str, argument: str) -> str:
        if len(self.calls) >= MAX_CALLS:  # the round limit
            self.limit_hit = True
            self.drop()
            return "Not run: round limit reached. Give your best final answer now."
        early = self.pending.pop((name, argument), None)  # a match: same tool, same argument
        self.drop()  # every other early call is discarded
        if early:
            self.matched += 1
        result = await early if early else await TOOLS[name](argument)
        self.calls.append(f"{name}({argument!r}) -> {result[:120]}")
        self.start_round()
        return result


@function_tool
async def search_docs(ctx: RunContextWrapper[Speculation], query: str) -> str:
    """Search the product docs and return the top 3 matches. Example: query="refund policy"."""
    return await ctx.context.real_call("search_docs", query)


@function_tool
async def get_order(ctx: RunContextWrapper[Speculation], order_id: str) -> str:
    """Fetch one order by id. Example: order_id="A-1042"."""
    return await ctx.context.real_call("get_order", order_id)


tool_agent = Agent[Speculation](
    name="Tool agent",
    instructions="Answer the task using the listed tools. Each turn, call exactly one tool with complete "
                 "arguments, or give your final answer. Do not guess a fact a tool can give you. "
                 "When you answer, list the calls you relied on.",
    tools=[search_docs, get_order],
)


async def run(task: str) -> dict:
    spec = Speculation(task)
    spec.start_round()
    result = await Runner.run(tool_agent, task, context=spec, max_turns=MAX_CALLS + 6)
    spec.drop()  # cancel whatever is still pending
    return {"answer": result.final_output, "calls": spec.calls, "incomplete": spec.limit_hit}

The agent's tools go through a cache of calls started early. Guessed calls are cancelled, not undone, so list in SAFE_EARLY only tools that read. The stubs take one string argument each.

Vercel AI SDK, typescript, package ai
import { anthropic } from '@ai-sdk/anthropic';
import { isStepCount, Output, tool, ToolLoopAgent } from 'ai';
import { z } from 'zod';

const MAX_CALLS = 8; // real tool calls per run
const MAX_GUESSES = 2; // calls started early per round
const model = anthropic('claude-sonnet-5-5');

// stubs: plug in your search index and your database. Both take one string argument.
const TOOLS: Record<string, (argument: string) => Promise<string>> = {
  searchDocs: async (query) => `stub: 3 matches for ${query}`,
  getOrder: async (orderId) => `stub: summary of ${orderId}`,
};
const SAFE_EARLY = new Set(['searchDocs', 'getOrder']); // read-only. Never add a tool that sends, writes or pays.
const TOOL_RULES = 'searchDocs(query): search the docs\ngetOrder(orderId): fetch one order';

const speculator = new ToolLoopAgent({
  model: anthropic('claude-haiku-5-5'), // small and fast: the guess must arrive before the real decision
  instructions:
    'Predict the next tool call the main agent will make from this state. Give at most two guesses, ' +
    'most likely first, each with the exact tool name and its argument. Be quick and do not solve ' +
    'the task. If the agent will probably answer next, return no guess.',
  output: Output.object({ schema: z.object({ calls: z.array(z.object({ tool: z.string(), argument: z.string() })) }) }),
});

export async function run(task: string) {
  const calls: string[] = []; // real calls only: what the plain loop would record
  const pending = new Map<string, Promise<string>>(); // calls started early, keyed by tool and argument
  const stats = { matched: 0, discarded: 0 }; // for your metrics, never part of the result
  let round = 0;
  let limitHit = false;

  const drop = () => {
    round++; // a guess that comes after the real decision is ignored
    stats.discarded += pending.size; // pass an AbortSignal to your tools to stop them for real
    pending.clear();
  };

  async function guess() {
    const mine = round;
    const { output } = await speculator.generate({
      prompt: `Task: ${task}\n\nTools:\n${TOOL_RULES}\n\nReal calls so far:\n${calls.join('\n')}`,
    });
    if (mine !== round) return;
    for (const g of output.calls.slice(0, MAX_GUESSES)) {
      const key = `${g.tool}:${g.argument}`;
      if (!SAFE_EARLY.has(g.tool) || pending.has(key)) continue; // the safety gate
      const early = TOOLS[g.tool](g.argument);
      early.catch(() => {}); // a failed guess must not break the run
      pending.set(key, early);
    }
  }
  const startRound = () => void guess().catch(() => {}); // runs while the slow agent is deciding

  async function realCall(name: string, argument: string) {
    if (calls.length >= MAX_CALLS) { // the round limit
      limitHit = true;
      drop();
      return 'Not run: round limit reached. Give your best final answer now.';
    }
    const key = `${name}:${argument}`;
    const early = pending.get(key); // a match: same tool, same argument
    pending.delete(key);
    drop(); // every other early call is discarded
    if (early) stats.matched++;
    const result = await (early ?? TOOLS[name](argument));
    calls.push(`${name}(${argument}) -> ${result.slice(0, 120)}`);
    startRound();
    return result;
  }

  const agent = new ToolLoopAgent({
    model,
    instructions:
      'Answer the task using the listed tools. Each turn, call exactly one tool with complete ' +
      'arguments, or give your final answer. Do not guess a fact a tool can give you. ' +
      'When you answer, list the calls you relied on.',
    tools: {
      searchDocs: tool({
        description: 'Search the product docs, top 3 matches. Example: { query: "refund policy" }',
        inputSchema: z.object({ query: z.string() }),
        execute: ({ query }) => realCall('searchDocs', query),
      }),
      getOrder: tool({
        description: 'Fetch one order by id. Example: { orderId: "A-1042" }',
        inputSchema: z.object({ orderId: z.string() }),
        execute: ({ orderId }) => realCall('getOrder', orderId),
      }),
    },
    prepareStep: async () => (limitHit ? { toolChoice: 'none' as const } : {}), // at the limit, answer only
    stopWhen: isStepCount(MAX_CALLS + 6),
  });

  startRound();
  const { text } = await agent.generate({ prompt: task });
  drop();
  console.info('speculation', stats);
  return { answer: text, calls, incomplete: limitHit };
}

The agent's tools go through a cache of calls started early. A dropped promise still runs to its end here, so list in SAFE_EARLY only tools that read. The stubs take one string argument.

Snippets are generated when the site is built and type-checked against the SDK versions listed in the repository. Treat them as a starting skeleton.

What goes wrong

  • Starting a guessed call that sends, writes or pays is unsafe, because a wrong guess has already acted. Speculate only on calls that are read-only, sandboxed or that you can undo.
  • Matching on the tool name alone hands the agent the result of a different call. Require the same tool and the same arguments.
  • If the agent sees guesses or early results, they change its decisions, and the run is no longer the one the plain loop would give.
  • Every miss is a call you paid for. Cap the guesses per round, watch the share that is discarded, and switch speculation off where it rarely matches.
  • A read whose result changes from one moment to the next, such as a live price, may not equal the call the agent would have made. Mark such tools not safe.

Chain it

Its result can feed

Source