Agentic Architectures

Catalogue / Retrieval

Adaptive RAG

Judge how hard the question is, then spend only what it needs

Shape
router
Model calls
1 to classify + 1 direct, 1 to 2 single-step, or up to 6 multi-step
Latency
medium
Complexity
medium

Also known as Complexity-routed retrieval

1

Why pay for a research process when most questions need one lookup or none?

Questions differ in difficulty, but most systems treat them all the same way: either too lightly for the hard ones or too heavily for the easy ones. Adaptive RAG classifies each question first and routes it to no retrieval, one retrieval, or several rounds of retrieval.

You may need it if

  • Your assistant answers "what does SSO stand for?" with the same slow search as a question comparing three contracts.
  • Your model bill grows with traffic, but most questions are short and simple.
  • Hard questions get shallow answers because the system only ever runs one search.
2

When is a classifier in front just an extra step?

Use it when

  • Your traffic mixes trivial questions with questions that need several documents combined.
  • Cost or response time matters at your volume.
  • You can give examples of easy, medium and hard questions from your own logs.

Avoid it when

  • All questions are of the same kind, such as order status lookups. Pick the one route that fits and drop the classifier.
  • Most questions are hard and need several rounds of searching anyway.
    Use Agentic RAG instead
  • Questions are about the whole collection, such as the main themes across all customer interviews.
    Use GraphRAG instead
  • The risk is not effort but accuracy, and every claim must be checked against a source.
    Use Self-RAG instead

What it costs

Model calls
1 to classify + 1 direct, 1 to 2 single-step, or up to 6 multi-step
Latency
medium
Complexity
medium
3

How does a question find its route?

The classifier sorts the question into one of three routes. Simple questions are answered directly. Single-fact questions get one search and a grounded answer. Complex questions go to a researcher that searches in several steps, with a hop limit, before the same writer answers.

simplesingle-stepmulti-stepsteps open,hops leftall steps done,or hops spentQuestion: The user's question in plain language.QuestionHow complex?: Sorts the question into simple, single-step or multi-step, rounding up when unsure.How complex?Direct answer: Answers simple questions without searching, such as definitions or rewording.Direct answerOne search: Runs a single search on your documents and returns the top passages.One searchResearcher: Breaks a complex question into steps and runs one search per step, using earlier findings.ResearcherEnough found?: Checks whether every step is answered, and counts the hops used.Enough found?Writer: Writes the answer from the passages found, citing them.WriterAnswer: The answer, the route that was taken and the sources when any were used.Answer
  1. Step 1 of 5

    Sort before spending

    A small, cheap classifier reads the question and picks a route. It is given examples of each kind from your own traffic. When in doubt, it picks the more thorough route.

  2. Step 2 of 5

    Simple questions, no search

    "What does SSO stand for?" needs no documents. The direct route answers in one call and is the fastest path through the system.

  3. Step 3 of 5

    One fact, one search

    "What is the refund window?" needs one passage. One search, then the writer answers from what it returned. This is classic retrieval, used only where it fits.

  4. Step 4 of 5

    Several facts, several hops

    "Which of our plans include SSO and cost less than the competitor's?" needs facts that build on each other. The researcher searches step by step until each step is answered or the hop limit is hit.

  5. Step 5 of 5

    One answer format, whatever the route

    Both retrieval routes end with the same writer, and every answer states which route it took. That label is what lets you check the classifier later.

What each part does
Question
The user's question in plain language.
How complex?
Sorts the question into simple, single-step or multi-step, rounding up when unsure.
Direct answer
Answers simple questions without searching, such as definitions or rewording.
One search
Runs a single search on your documents and returns the top passages.
Researcher
Breaks a complex question into steps and runs one search per step, using earlier findings.
Enough found?
Checks whether every step is answered, and counts the hops used.
Writer
Writes the answer from the passages found, citing them.
Answer
The answer, the route that was taken and the sources when any were used.
  • How complex? → Direct answer: simple
  • How complex? → One search: single-step
  • How complex? → Researcher: multi-step
  • Researcher → Enough found?: findings so far
  • Enough found? → Researcher: steps open, hops left
  • Enough found? → Writer: all steps done, or hops spent
  • One search → Writer: passages

Where it connects

Entry
question
Question

A question in plain language. Optionally, a forced route for testing or for questions known to be hard.

Exit
answer
Answer

When: The chosen route has produced an answer, or the multi-step route has spent its hops.

The answer, the route taken (simple, single-step or multi-step), and the sources used when the route involved a search.

4

What do I ask my assistant to set up?

The agents to launch

Classifier

Sort the question into one route. Simple: answerable without our documents. Single-step: one fact from our documents. Multi-step: several facts, where one depends on another. If unsure between two routes, choose the more thorough one. Reply with the route and one line of reason.

Sees: The question and a few labelled example questions for each route. Not the documents.

Direct answerer

Answer the question briefly from general knowledge. If you find it needs facts specific to our company or product, say so instead of guessing.

Sees: The question only. No documents.

Researcher

Break the question into the steps it depends on. For each open step, write one search query, using what earlier steps found. After each search, mark which steps are answered. Do not write the final answer.

Sees: The question, the search results so far and the list of open steps. Not the classifier's reasoning.

Writer

Answer the question from the passages provided and cite each one you use. If the passages leave part of the question open, say which part.

Sees: The question and the passages from the search or the researcher. Not the researcher's notes.

A prompt you can paste

Works with any assistant that can launch sub-agents. It contains the agents, the procedure and the stop rule above.

Framework-agnostic prompt
You are the orchestrator of a multi-agent workflow that follows the "Adaptive RAG" architecture.
Launch every agent listed below as a separate sub-agent with its own context. Give each agent only the context stated for it. Do not do an agent's work yourself. If you cannot launch sub-agents, play each agent in a clearly separated turn and never let it use what is withheld from it.

# Task
[Describe your task here]

# Adaptive RAG
Judge how hard the question is, then spend only what it needs.
Input: the task above. It must provide: A question in plain language. Optionally, a forced route for testing or for questions known to be hard. If something essential is missing, ask for it before starting.

Agents to launch:
- Classifier: Sort the question into one route. Simple: answerable without our documents. Single-step: one fact from our documents. Multi-step: several facts, where one depends on another. If unsure between two routes, choose the more thorough one. Reply with the route and one line of reason.
  Context for this agent: The question and a few labelled example questions for each route. Not the documents.
- Direct answerer: Answer the question briefly from general knowledge. If you find it needs facts specific to our company or product, say so instead of guessing.
  Context for this agent: The question only. No documents.
- Researcher: Break the question into the steps it depends on. For each open step, write one search query, using what earlier steps found. After each search, mark which steps are answered. Do not write the final answer.
  Context for this agent: The question, the search results so far and the list of open steps. Not the classifier's reasoning.
- Writer: Answer the question from the passages provided and cite each one you use. If the passages leave part of the question open, say which part.
  Context for this agent: The question and the passages from the search or the researcher. Not the researcher's notes.

Procedure:
1. Collect ten to twenty real questions from your logs and label each as simple, single-step or multi-step. Give them to the classifier as examples.
2. Launch the classifier on the question.
3. If simple, launch the direct answerer. If it reports that company facts are needed, reroute to single-step.
4. If single-step, run one search on your documents and launch the writer with the top five passages.
5. If multi-step, launch the researcher and run each query it writes. Repeat while steps are open, up to 4 hops, then launch the writer with all useful passages.
6. Label the answer with the route taken.

Stop rule: Every route runs once. The multi-step route stops when all steps are answered or after 4 hops. A direct answer may be rerouted to single-step once, never more.

Exit:
- `answer`, when: The chosen route has produced an answer, or the multi-step route has spent its hops. Deliver: The answer, the route taken (simple, single-step or multi-step), and the sources used when the route involved a search.

# Final deliverable
The answer, the route taken with the classifier's one-line reason, and the sources used when a search ran.

The same thing in code

LangGraph, python, package langgraph
import operator
from typing import Annotated, Literal, TypedDict

from langchain.chat_models import init_chat_model
from langgraph.graph import END, START, StateGraph
from pydantic import BaseModel

llm = init_chat_model("anthropic:claude-sonnet-5-5")
MAX_HOPS = 4
EXAMPLES = ("simple: What does SSO stand for?\n"
            "single-step: What is our refund window?\n"
            "multi-step: Which plans include SSO and cost less than the competitor's?")


def search_docs(query: str, k: int = 5) -> list[str]:
    return []  # plug in your document search (vector store, BM25, ...)


class Route(BaseModel):
    route: Literal["simple", "single-step", "multi-step"]
    reason: str


class Direct(BaseModel):
    answer: str
    needs_company_facts: bool


class Step(BaseModel):
    open_steps: list[str]
    query: str | None


class State(TypedDict):
    question: str
    route: str
    reason: str
    answer: str
    passages: Annotated[list[str], operator.add]
    open_steps: list[str]
    hops: int


async def ask[T: BaseModel](schema: type[T], prompt: str) -> T:
    out = await llm.with_structured_output(schema).ainvoke(prompt)
    assert isinstance(out, schema)
    return out


async def classify(state: State) -> dict:
    # the classifier sees labelled examples, never the documents
    r = await ask(Route, f"Sort the question into simple, single-step or multi-step. "
                  f"If unsure, choose the more thorough route.\nExamples:\n{EXAMPLES}\n\nQuestion: {state['question']}")
    return {"route": r.route, "reason": r.reason}


async def direct(state: State) -> dict:
    d = await ask(Direct, "Answer briefly from general knowledge. If it needs facts about our company "
                  f"or product, set needs_company_facts instead of guessing.\n\n{state['question']}")
    if d.needs_company_facts:  # reroute to single-step, once
        return {"route": "single-step", "reason": state["reason"] + " (rerouted from simple)"}
    return {"answer": d.answer}


async def search(state: State) -> dict:
    return {"passages": search_docs(state["question"])}


async def researcher(state: State) -> dict:
    step = await ask(Step, "Break the question into the steps it depends on. List the steps still open "
                     "and write one search query for the next one (none if all are answered). Do not answer.\n\n"
                     f"Question: {state['question']}\nOpen steps: {state['open_steps']}\nResults so far: {state['passages']}")
    found = search_docs(step.query) if step.query else []
    return {"open_steps": step.open_steps, "passages": found, "hops": state["hops"] + 1}


async def writer(state: State) -> dict:
    msg = await llm.ainvoke("Answer from these passages and cite each one you use. Say which part stays open.\n\n"
                            f"Question: {state['question']}\nPassages: {state['passages']}")
    return {"answer": msg.text}


def by_route(state: State) -> Literal["direct", "search", "researcher"]:
    if state["route"] == "simple":
        return "direct"
    return "search" if state["route"] == "single-step" else "researcher"


def after_direct(state: State) -> str:
    return "search" if state["route"] == "single-step" else END


def enough_found(state: State) -> Literal["researcher", "writer"]:
    # the hop gate: steps open and hops left, else write
    return "researcher" if state["open_steps"] and state["hops"] < MAX_HOPS else "writer"


g = StateGraph(State)
g.add_node("classify", classify)
g.add_node("direct", direct)
g.add_node("search", search)
g.add_node("researcher", researcher)
g.add_node("writer", writer)
g.add_edge(START, "classify")
g.add_conditional_edges("classify", by_route)
g.add_conditional_edges("direct", after_direct, ["search", END])
g.add_edge("search", "writer")
g.add_conditional_edges("researcher", enough_found)
g.add_edge("writer", END)
app = g.compile()


async def run(task: str):
    state = await app.ainvoke({"question": task, "route": "", "reason": "", "answer": "",
                           "passages": [], "open_steps": [], "hops": 0})
    return {"answer": state["answer"], "route": state["route"], "reason": state["reason"], "sources": state["passages"]}

Replace search_docs with your retriever and give the classifier ten to twenty labelled questions from your own logs.

Claude Agent SDK, python, package claude-agent-sdk
from typing import Any

from claude_agent_sdk import (AgentDefinition, ClaudeAgentOptions, ResultMessage,
                              create_sdk_mcp_server, query, tool)

MAX_HOPS = 4
passages: list[str] = []
hops = [0]


def search_docs(q: str, k: int = 5) -> list[str]:
    return []  # plug in your document search (vector store, BM25, ...)


@tool("search", "Search our documents and return the top passages", {"query": str})
async def search(args: dict[str, Any]) -> dict[str, Any]:
    if hops[0] >= MAX_HOPS:  # the hop gate: 4 searches, then the researcher must stop
        return {"content": [{"type": "text", "text": "Hop limit reached. Stop searching."}], "is_error": True}
    hops[0] += 1
    found = search_docs(args["query"])
    passages.extend(found)
    return {"content": [{"type": "text", "text": "\n\n".join(found) or "No results."}]}


DOCS = create_sdk_mcp_server("docs", tools=[search])
AGENTS = {
    "classifier": AgentDefinition(
        description="Sorts a question into simple, single-step or multi-step.",
        prompt="Sort the question into one route. Simple: no documents needed. Single-step: one fact "
               "from our documents. Multi-step: several dependent facts. If unsure, choose the more "
               "thorough route. Give the route and one line of reason.", tools=[]),
    "direct": AgentDefinition(
        description="Answers simple questions without documents.",
        prompt="Answer briefly from general knowledge. If it needs facts about our company or product, "
               "say so instead of guessing.", tools=[]),
    "researcher": AgentDefinition(
        description="Searches step by step for multi-step questions.",
        prompt="Break the question into the steps it depends on. For each open step, run one search, "
               "using what earlier steps found. Stop when every step is answered or search refuses. "
               "Do not write the final answer.", tools=["mcp__docs__search"]),
    "writer": AgentDefinition(
        description="Writes the cited answer from passages.",
        prompt="Answer from the passages provided and cite each one you use. Say which part stays open.",
        tools=[]),
}


async def delegate(agent: str, payload: str, schema: dict[str, Any] | None = None) -> Any:
    options = ClaudeAgentOptions(
        agents=AGENTS, mcp_servers={"docs": DOCS}, allowed_tools=["Agent", "mcp__docs__search"],
        output_format={"type": "json_schema", "schema": schema} if schema else None)
    prompt = f"Use the {agent} agent on the input below and return its reply unchanged.\n\n{payload}"
    async for msg in query(prompt=prompt, options=options):
        if isinstance(msg, ResultMessage):
            return msg.structured_output if schema else msg.result


ROUTE = {"type": "object", "required": ["route", "reason"], "properties": {
    "route": {"enum": ["simple", "single-step", "multi-step"]}, "reason": {"type": "string"}}}
DIRECT = {"type": "object", "required": ["answer", "needs_company_facts"], "properties": {
    "answer": {"type": "string"}, "needs_company_facts": {"type": "boolean"}}}


async def run(task: str):
    passages.clear()
    hops[0] = 0
    r = await delegate("classifier", task, ROUTE)
    route, reason = r["route"], r["reason"]
    if route == "simple":
        d = await delegate("direct", task, DIRECT)
        if not d["needs_company_facts"]:
            return {"answer": d["answer"], "route": route, "reason": reason, "sources": []}
        route = "single-step"  # rerouted once, never more
    if route == "single-step":
        passages.extend(search_docs(task))
    else:
        await delegate("researcher", task)  # its notes stay inside the sub-agent
    # the writer sees the question and passages only
    answer = await delegate("writer", f"Question: {task}\n\nPassages:\n" + "\n\n".join(passages))
    return {"answer": answer, "route": route, "reason": reason, "sources": list(passages)}

Replace search_docs with your retriever; the module-level passage list and hop counter assume one run at a time.

OpenAI Agents SDK, python, package openai-agents
from typing import Literal

from agents import Agent, Runner
from pydantic import BaseModel

MAX_HOPS = 4


def search_docs(query: str, k: int = 5) -> list[str]:
    return []  # plug in your document search (vector store, BM25, ...)


class Route(BaseModel):
    route: Literal["simple", "single-step", "multi-step"]
    reason: str


class Direct(BaseModel):
    answer: str
    needs_company_facts: bool


class Step(BaseModel):
    open_steps: list[str]
    query: str | None


classifier = Agent(
    name="Classifier", output_type=Route,
    instructions="Sort the question into simple (no documents needed), single-step (one fact from our "
                 "documents) or multi-step (several dependent facts). If unsure, choose the more thorough "
                 "route. Give one line of reason. Examples: 'What does SSO stand for?' is simple.")
direct = Agent(
    name="Direct answerer", output_type=Direct,
    instructions="Answer briefly from general knowledge. If it needs facts about our company or product, "
                 "set needs_company_facts instead of guessing.")
researcher = Agent(
    name="Researcher", output_type=Step,
    instructions="Break the question into the steps it depends on. List the steps still open and write one "
                 "search query for the next one, using earlier results. Leave query empty when all are "
                 "answered. Do not write the final answer.")
writer = Agent(
    name="Writer",
    instructions="Answer from the passages provided and cite each one you use. Say which part stays open.")


async def run(task: str):
    r = (await Runner.run(classifier, task)).final_output_as(Route)
    route = r.route
    passages: list[str] = []
    if route == "simple":
        d = (await Runner.run(direct, task)).final_output_as(Direct)
        if not d.needs_company_facts:
            return {"answer": d.answer, "route": route, "reason": r.reason, "sources": []}
        route = "single-step"  # rerouted once, never more
    if route == "single-step":
        passages = search_docs(task)
    else:
        open_steps: list[str] = []
        for _ in range(MAX_HOPS):  # the hop gate: at most 4 searches
            # the researcher sees the question, results and open steps, not the classifier's reasoning
            step = (await Runner.run(researcher, f"Question: {task}\nOpen steps: {open_steps}\n"
                                                 f"Results so far: {passages}")).final_output_as(Step)
            open_steps = step.open_steps
            if not step.query or not open_steps:
                break
            passages += search_docs(step.query)
    # the writer sees passages, not the researcher's notes
    out = await Runner.run(writer, f"Question: {task}\n\nPassages:\n" + "\n\n".join(passages))
    return {"answer": out.final_output, "route": route, "reason": r.reason, "sources": passages}

Replace search_docs with your retriever and put real labelled examples from your logs in the classifier's instructions.

Vercel AI SDK, typescript, package ai
import { anthropic } from '@ai-sdk/anthropic';
import { generateText, Output } from 'ai';
import { z } from 'zod';

const model = anthropic('claude-sonnet-5-5');
const MAX_HOPS = 4;

async function searchDocs(query: string, k = 5): Promise<string[]> {
  return []; // plug in your document search (vector store, BM25, ...)
}

const Route = z.object({ route: z.enum(['simple', 'single-step', 'multi-step']), reason: z.string() });
const Direct = z.object({ answer: z.string(), needsCompanyFacts: z.boolean() });
const Step = z.object({ openSteps: z.array(z.string()), query: z.string().nullable() });

export async function run(task: string) {
  // the classifier sees the question and labelled examples, never the documents
  const { output: r } = await generateText({
    model,
    output: Output.object({ schema: Route }),
    instructions:
      'Sort the question into simple (no documents needed), single-step (one fact from our documents) ' +
      'or multi-step (several dependent facts). If unsure, choose the more thorough route. ' +
      "Give one line of reason. Example: 'What does SSO stand for?' is simple.",
    prompt: task,
  });
  let route = r.route;
  let passages: string[] = [];

  if (route === 'simple') {
    const { output: d } = await generateText({
      model,
      output: Output.object({ schema: Direct }),
      instructions:
        'Answer briefly from general knowledge. If it needs facts about our company or product, ' +
        'set needsCompanyFacts instead of guessing.',
      prompt: task,
    });
    if (!d.needsCompanyFacts) return { answer: d.answer, route, reason: r.reason, sources: [] };
    route = 'single-step'; // rerouted once, never more
  }

  if (route === 'single-step') {
    passages = await searchDocs(task);
  } else {
    let openSteps: string[] = [];
    for (let hop = 0; hop < MAX_HOPS; hop++) { // the hop gate: at most 4 searches
      const { output: step } = await generateText({
        model,
        output: Output.object({ schema: Step }),
        instructions:
          'Break the question into the steps it depends on. List the steps still open and write one ' +
          'search query for the next one, using earlier results. Null query when all are answered. ' +
          'Do not write the final answer.',
        prompt: `Question: ${task}\nOpen steps: ${JSON.stringify(openSteps)}\nResults so far: ${passages.join('\n')}`,
      });
      openSteps = step.openSteps;
      if (!step.query || openSteps.length === 0) break;
      passages.push(...(await searchDocs(step.query)));
    }
  }

  // the writer sees the passages, not the researcher's notes
  const { text } = await generateText({
    model,
    instructions: 'Answer from the passages provided and cite each one you use. Say which part stays open.',
    prompt: `Question: ${task}\n\nPassages:\n${passages.join('\n\n')}`,
  });
  return { answer: text, route, reason: r.reason, sources: passages };
}

Replace searchDocs with your retriever and put real labelled examples from your logs in the classifier's instructions.

Snippets are generated when the site is built and type-checked against the SDK versions listed in the repository. Treat them as a starting skeleton.

What goes wrong

  • A classifier with no examples from your traffic guesses by question length. Short questions can be the hardest ones.
  • Without a route label on each answer you cannot audit the classifier. Always record it.
  • If the direct route is allowed to answer company-specific questions, it invents policies. Make it hand those back.

Chain it

Its result can feed

Source