1Why run every request through the same expensive machinery?
Requests differ. A greeting needs one call, a policy question needs retrieval, a data migration needs a plan with checks. A meta-controller reads each request, picks the best-fitting architecture from a fixed roster, runs it and returns its result.
You may need it if
- Simple requests take as long and cost as much as hard ones.
- You have built two or three architectures and users do not know which one to pick.
- One pipeline handles some request types well and others badly.
2When is a router one layer too many?
Use it when
- Incoming requests fall into a few kinds that clearly need different handling.
- The cost or latency gap between your architectures is large.
- Each architecture in the roster can be described in two or three lines that a classifier can apply.
Avoid it when
- Nearly every request is of the same kind. Use that one architecture directly.
- The real question is whether to answer at all, given the risk, not how to answer.
Use Metacognitive Triage instead - The only choice is how much retrieval a question needs.
Use Adaptive RAG instead - You want several approaches to run and their results to be combined, not one picked.
Use Ensemble instead
What it costs
- Model calls
- 1 classification + the chosen architecture's own calls
- Latency
- low
- Complexity
- medium
3How does a request find its architecture?
A hub with a few spokes. The controller reads the request and the roster, picks one route and dispatches. The chosen architecture runs with its own rules. A check confirms the result is usable, with one reroute allowed before the answer leaves.
- Step 1 of 5
Read the request against the roster
The controller sees the request and the roster entries, each a few lines on what that architecture is for and what it costs. It does not try to answer the request itself.
- Step 2 of 5
Pick exactly one route
It names one route, a confidence and a one-line reason. When unsure between two, it picks the more thorough one. A wrong cheap answer costs more than a slow right one.
- Step 3 of 5
The route runs on its own terms
The chosen architecture receives the request unchanged and runs with its own agents and stop rule. The controller does not interfere.
- Step 4 of 5
A simple check on what came back
A rule checks that the route finished normally and returned the expected kind of result. If not, the controller may pick a different route, once.
- Step 5 of 5
The answer leaves with its route
The result goes out with the route name and reason attached, so misroutes can be found later in the logs.
What each part does
- Request
- The incoming task or question, with any context about the user.
- Controller
- Classifies the request against the roster and picks one route, with a confidence and a reason.
- Roster
- The fixed list of available architectures: what each is for, what it costs, and examples.
- Quick answer
- Example route. A single call for simple, low-stakes requests.
- Research route
- Example route. A retrieval architecture for questions that need your documents.
- Multi-step route
- Example route. A plan-and-verify architecture for tasks with several steps.
- Usable result?
- Checks that the route met its own stop rule and returned a result of the expected kind.
- Answer
- The chosen route's result, with the route name and the reason it was chosen.
- Controller → Quick answer: simple request
- Controller → Research route: needs documents
- Controller → Multi-step route: several steps
- Usable result? → Controller: unusable, reroute not used
- Usable result? → Answer: usable, or reroute spent
Where it connects
A task or question in plain language, with any user context that affects which route fits.
When: A route returned a usable result, or the single reroute has been used.
The result of the chosen route, its name, the controller's reason and confidence, and whether a reroute happened.
One entry per architecture, with its purpose, typical cost, the request kinds it handles and two example requests. Edited by people, not by the run.
4What do I ask my assistant to set up?
The agents to launch
Controller
Read the request and the roster. Pick exactly one route by name, give a confidence from 1 to 5 and one line of reasons. If your confidence is below 4, pick the more thorough of the routes you hesitate between. Do not answer the request.
Sees: The request and the roster. On a reroute, also the failed route and why it failed. Never the routes' internal work.
Quick responder
Answer the request directly in one reply. If it turns out to need documents or several steps, say so in one line instead of guessing.
Sees: The request only. Not the roster, the controller's reasoning or any other route.
Route runner, one launch per dispatched request, following that route's own procedure
Run the named architecture on the request, following its own procedure, agents and stop rule exactly. Return its deliverable and state whether its stop rule was met normally.
Sees: The request and the chosen architecture's procedure. Not the controller's reasoning or the other routes.
A prompt you can paste
Works with any assistant that can launch sub-agents. It contains the agents, the procedure and the stop rule above.
You are the orchestrator of a multi-agent workflow that follows the "Meta-Controller" architecture. Launch every agent listed below as a separate sub-agent with its own context. Give each agent only the context stated for it. Do not do an agent's work yourself. If you cannot launch sub-agents, play each agent in a clearly separated turn and never let it use what is withheld from it. # Task [Describe your task here] # Meta-Controller Classify each request and send it to the architecture that fits. Input: the task above. It must provide: A task or question in plain language, with any user context that affects which route fits. If something essential is missing, ask for it before starting. Agents to launch: - Controller: Read the request and the roster. Pick exactly one route by name, give a confidence from 1 to 5 and one line of reasons. If your confidence is below 4, pick the more thorough of the routes you hesitate between. Do not answer the request. Context for this agent: The request and the roster. On a reroute, also the failed route and why it failed. Never the routes' internal work. - Quick responder: Answer the request directly in one reply. If it turns out to need documents or several steps, say so in one line instead of guessing. Context for this agent: The request only. Not the roster, the controller's reasoning or any other route. - Route runner (one launch per dispatched request, following that route's own procedure): Run the named architecture on the request, following its own procedure, agents and stop rule exactly. Return its deliverable and state whether its stop rule was met normally. Context for this agent: The request and the chosen architecture's procedure. Not the controller's reasoning or the other routes. Procedure: 1. Write the roster with two to five routes. For each, state its purpose, its cost, the request kinds it handles and two example requests. 2. Launch the controller on the request and the roster. Record the route, confidence and reason. 3. Launch the chosen route as a separate agent with the request unchanged. For the quick route, use the quick responder. For the others, use the route runner with that architecture's procedure. 4. Check the result. It is usable if the route finished normally and returned the expected kind of result. 5. If it is unusable and no reroute has happened, return to the controller with the failure and let it pick a different route. Otherwise return the result. Stop rule: Stop when a route returns a usable result, or after one reroute. Never run more than two routes for one request. Exit: - `answer`, when: A route returned a usable result, or the single reroute has been used. Deliver: The result of the chosen route, its name, the controller's reason and confidence, and whether a reroute happened. # Final deliverable The result, then one line naming the route, the controller's confidence and reason, and whether a reroute was needed.
The same thing in code
from typing import Literal
from langchain.chat_models import init_chat_model
from langgraph.graph import END, START, StateGraph
from pydantic import BaseModel
llm = init_chat_model("anthropic:claude-sonnet-5-5")
RouteName = Literal["quick", "research", "build"]
# the roster: edited by people, never by the run
ROSTER = """- quick: single call. Greetings, definitions, low-stakes questions. Never for our documents.
- research: retrieval over our documents. Policy or product questions. About 5 calls.
- build: plan, execute, verify. Tasks with several steps such as migrations. 10+ calls."""
PROCEDURES = {
"research": "Search the documents, read the top passages, answer with citations.",
"build": "Write a numbered plan, do each step, verify each step, stop at 8 steps.",
}
class Route(BaseModel):
route: RouteName
confidence: int # 1 to 5
reason: str
class RouteResult(BaseModel):
deliverable: str = ""
stop_rule_met: bool = False
class State(BaseModel):
task: str
route: Route | None = None
tried: list[str] = []
result: RouteResult = RouteResult()
def search_docs(query: str) -> str:
return "" # plug in your document search
async def controller(state: State) -> dict:
failed = f"\nRoute {state.tried[-1]} failed: {state.result.deliverable[:200]}. Pick another." if state.tried else ""
prompt = (
"Pick exactly one route for the request, a confidence from 1 to 5 and one line of reason. "
"Below 4, pick the more thorough route. Do not answer the request.\n"
f"Roster:\n{ROSTER}{failed}\n\nRequest: {state.task}"
)
route = Route.model_validate(await llm.with_structured_output(Route).ainvoke(prompt))
return {"route": route, "tried": state.tried + [route.route]}
async def quick(state: State) -> dict:
# sees the request only: not the roster, not the controller's reason
reply = await llm.ainvoke("Answer in one reply. If this needs documents or several steps, "
"reply 'ESCALATE: <why>' instead.\n\n" + state.task)
return {"result": RouteResult(deliverable=reply.text, stop_rule_met=not reply.text.startswith("ESCALATE"))}
def route_runner(name: str):
async def node(state: State) -> dict:
docs = search_docs(state.task) if name == "research" else ""
prompt = (f"Follow this procedure exactly: {PROCEDURES[name]}\nReport whether its stop rule was met.\n"
f"Documents: {docs}\n\nRequest: {state.task}")
return {"result": RouteResult.model_validate(await llm.with_structured_output(RouteResult).ainvoke(prompt))}
return node
def usable(state: State) -> str:
if state.result.stop_rule_met and state.result.deliverable.strip():
return END
return "controller" if len(state.tried) < 2 else END # the gate: one reroute, two routes at most
graph = StateGraph(State)
graph.add_node("controller", controller)
graph.add_node("quick", quick)
graph.add_node("research", route_runner("research"))
graph.add_node("build", route_runner("build"))
graph.add_edge(START, "controller")
graph.add_conditional_edges("controller", lambda s: s.route.route if s.route else "quick", ["quick", "research", "build"])
for name in ("quick", "research", "build"):
graph.add_conditional_edges(name, usable, ["controller", END])
app = graph.compile()
async def run(task: str):
final = State.model_validate(await app.ainvoke(State(task=task)))
r = final.route
why = f"{r.route} (confidence {r.confidence}): {r.reason}" if r else "none"
return f"{final.result.deliverable}\n\nRoute: {why}. Routes tried: {final.tried}"
Replace the roster text and the two procedures with your real architectures, and log every route and outcome so misroutes can be reviewed.
from typing import Literal, TypeVar
from claude_agent_sdk import AgentDefinition, ClaudeAgentOptions, ResultMessage, query
from pydantic import BaseModel
M = TypeVar("M", bound=BaseModel)
ROSTER = """- quick: single call. Greetings, definitions, low-stakes questions.
- research: retrieval over our documents. Policy or product questions.
- build: plan, execute, verify. Tasks with several steps."""
class Route(BaseModel):
route: Literal["quick", "research", "build"]
confidence: int
reason: str
class RouteResult(BaseModel):
deliverable: str
stop_rule_met: bool
AGENTS = {
"controller": AgentDefinition(
description="Picks one route from the roster. Never answers.",
prompt="Pick exactly one route by name, a confidence from 1 to 5 and one line of reason. "
"Below 4, pick the more thorough route. Do not answer the request.",
tools=[],
),
"quick": AgentDefinition(
description="Answers simple requests in one reply.",
prompt="Answer directly in one reply. If it needs documents or several steps, say so in one line.",
tools=[],
),
"research": AgentDefinition(
description="Route runner for the retrieval architecture.",
prompt="Search the documents, answer with citations, and state whether the stop rule was met.",
tools=["Grep", "Read"], # point it at your document folder or a search MCP server
),
"build": AgentDefinition(
description="Route runner for the plan-and-verify architecture.",
prompt="Plan, execute and verify each step, at most 8 steps. State whether the stop rule was met.",
tools=["Read", "Grep"],
),
}
async def delegate(agent: str, payload: str, schema: type[M]) -> M:
# one query per step, so each sub-agent sees only the payload it is handed
options = ClaudeAgentOptions(
agents={agent: AGENTS[agent]},
tools=["Agent"],
allowed_tools=["Agent"],
output_format={"type": "json_schema", "schema": schema.model_json_schema()},
max_turns=6,
)
prompt = f"Hand this input unchanged to the {agent} agent and return its result:\n\n{payload}"
async for message in query(prompt=prompt, options=options):
if isinstance(message, ResultMessage):
return schema.model_validate(message.structured_output)
raise RuntimeError(f"{agent} returned nothing")
async def run(task: str):
tried: list[str] = []
failure = ""
route = Route(route="quick", confidence=0, reason="")
result = RouteResult(deliverable="", stop_rule_met=False)
while len(tried) < 2: # the gate: never more than two routes
retry = f"\nRoute {tried[-1]} failed: {failure}. Pick a different one." if tried else ""
route = await delegate("controller", f"Roster:\n{ROSTER}{retry}\n\nRequest: {task}", Route)
tried.append(route.route)
result = await delegate(route.route, task, RouteResult)
if result.stop_rule_met and result.deliverable.strip():
break
failure = result.deliverable[:200]
return f"{result.deliverable}\n\nRoute: {route.route} ({route.confidence}/5): {route.reason}. Tried: {tried}"
Give each route runner the tools its architecture really needs, and keep the controller tool-free so it can only choose.
from typing import Literal
from agents import Agent, Runner, function_tool
from pydantic import BaseModel
ROSTER = """- quick: single call. Greetings, definitions, low-stakes questions.
- research: retrieval over our documents. Policy or product questions.
- build: plan, execute, verify. Tasks with several steps."""
class Route(BaseModel):
route: Literal["quick", "research", "build"]
confidence: int
reason: str
class RouteResult(BaseModel):
deliverable: str
stop_rule_met: bool
@function_tool
def search_docs(query: str) -> str:
"""Search the organisation's documents."""
return "" # plug in your document search
controller = Agent(
name="Controller",
instructions="Read the request and the roster. Pick exactly one route, a confidence from 1 to 5 and one "
"line of reason. Below 4, pick the more thorough route. Do not answer the request.",
output_type=Route,
)
routes: dict[str, Agent] = {
"quick": Agent(
name="Quick responder",
instructions="Answer in one reply. If it needs documents or several steps, say so in one line "
"and set stop_rule_met to false.",
output_type=RouteResult,
),
"research": Agent(
name="Route runner: research",
instructions="Search the documents, answer with citations, report whether the stop rule was met.",
tools=[search_docs],
output_type=RouteResult,
),
"build": Agent(
name="Route runner: multi-step",
instructions="Plan, do and verify each step, at most 8 steps. Report whether the stop rule was met.",
output_type=RouteResult,
),
}
async def run(task: str):
tried: list[str] = []
note = ""
route = Route(route="quick", confidence=0, reason="")
result = RouteResult(deliverable="", stop_rule_met=False)
while len(tried) < 2: # the gate: one reroute at most
picked = await Runner.run(controller, f"Roster:\n{ROSTER}{note}\n\nRequest: {task}")
route = picked.final_output_as(Route)
tried.append(route.route)
# the route sees the request unchanged, not the controller's reasoning
ran = await Runner.run(routes[route.route], task, max_turns=12)
result = ran.final_output_as(RouteResult)
if result.stop_rule_met and result.deliverable.strip():
break
note = f"\nRoute {route.route} failed: {result.deliverable[:200]}. Pick a different one."
return f"{result.deliverable}\n\nRoute: {route.route} ({route.confidence}/5): {route.reason}. Tried: {tried}"
Replace the search stub and the two route runners with your real architectures, and log every route decision for later review.
import { anthropic } from '@ai-sdk/anthropic';
import { generateText, isStepCount, Output, tool } from 'ai';
import { z } from 'zod';
const model = anthropic('claude-sonnet-5-5');
const ROSTER = `- quick: single call. Greetings, definitions, low-stakes questions.
- research: retrieval over our documents. Policy or product questions.
- build: plan, execute, verify. Tasks with several steps.`;
const routeSchema = z.object({
route: z.enum(['quick', 'research', 'build']),
confidence: z.number().int(),
reason: z.string(),
});
const resultSchema = z.object({ deliverable: z.string(), stopRuleMet: z.boolean() });
type RouteName = z.infer<typeof routeSchema>['route'];
const searchDocs = tool({
description: "Search the organisation's documents.",
inputSchema: z.object({ query: z.string() }),
execute: async ({ query }) => ({ query, passages: [] as string[] }), // plug in your document search
});
const routes: Record<RouteName, { instructions: string; tools?: { searchDocs: typeof searchDocs } }> = {
quick: { instructions: 'Answer in one reply. If it needs documents or several steps, say so and set stopRuleMet false.' },
research: { instructions: 'Search the documents, answer with citations, report whether the stop rule was met.', tools: { searchDocs } },
build: { instructions: 'Plan, do and verify each step, at most 8 steps. Report whether the stop rule was met.' },
};
export async function run(task: string) {
const tried: RouteName[] = [];
let note = '';
let route: z.infer<typeof routeSchema> = { route: 'quick', confidence: 1, reason: '' };
let result: z.infer<typeof resultSchema> = { deliverable: '', stopRuleMet: false };
while (tried.length < 2) { // the gate: never more than two routes
({ output: route } = await generateText({
model,
instructions: 'Pick exactly one route, a confidence from 1 to 5 and one line of reason. ' +
'Below 4, pick the more thorough route. Do not answer the request.',
prompt: `Roster:\n${ROSTER}${note}\n\nRequest: ${task}`,
output: Output.object({ schema: routeSchema }),
}));
tried.push(route.route);
const spec = routes[route.route];
// the route sees the request unchanged, not the controller's reasoning
({ output: result } = await generateText({
model,
instructions: spec.instructions,
prompt: task,
tools: spec.tools,
stopWhen: isStepCount(10),
output: Output.object({ schema: resultSchema }),
}));
if (result.stopRuleMet && result.deliverable.trim()) break;
note = `\nRoute ${route.route} failed: ${result.deliverable.slice(0, 200)}. Pick a different one.`;
}
return `${result.deliverable}\n\nRoute: ${route.route} (${route.confidence}/5): ${route.reason}. Tried: ${tried.join(', ')}`;
}
Replace the search stub and route instructions with your real architectures, and log each route, confidence and outcome.
Snippets are generated when the site is built and type-checked against the SDK versions listed in the repository. Treat them as a starting skeleton.
What goes wrong
- Vague roster entries cause random routing. Write each entry with concrete example requests and what it must not handle.
- A controller that also tries to answer leaks its guess into the route. Keep it limited to choosing.
- Without a log of routes and outcomes, misroutes go unseen. Record every decision and review a sample each week.
Chain it
Its result can feed
- Reflection conditional
Condition: Use only when the route named in A's payload returned written text (an answer or document) and not an executed action, a passed test run, an approval or a rule-compliance pass.
- Chain of Verification conditional
Condition: Use only when the route named in A's payload returned a factual answer in text that no hard gate certified and that the chain-of-verification route did not itself produce.
- Debate conditional
Condition: Use only when the route named in A's payload answered a question or decision that could be wrong, and did not return an executed action or a result certified by a hard gate.
- Constitutional Critique conditional
Condition: Use only when the route named in A's payload returned a text answer or document, not an executed action or a result already certified by tests, approval or a compliance pass.
- Textual Backpropagation conditional
Condition: Build the link only when the route named in A's payload returned a substantive answer or document that no hard gate certified, so not an executed action, a test-passed or approved result, and not a one-line conversational reply.