Agentic Architectures

Catalogue / Retrieval

Agentic RAG

The agent decides what to look up, and looks again until it knows

Shape
loop
Model calls
1 per search round + 1 to write (rounds typically 1 to 4)
Latency
medium
Complexity
medium

Also known as Iterative retrieval, Research agent

1

Why does a single search so often miss what the question needs?

Classic retrieval runs one search with the user's words and answers from whatever comes back. Real questions often need a different wording, or two facts found in two places. Here an agent chooses whether to search, what to search for, and whether to search again, before anyone writes the answer.

You may need it if

  • Your support bot answers well when the customer uses the manual's wording, and badly otherwise.
  • Questions need two facts from different documents, such as a plan's price and the refund rule.
  • Your team often rephrases a search two or three times before they find the right page.
2

When is a searching agent more than I need?

Use it when

  • Answers live in your own documents, such as help articles, contracts or internal wikis.
  • Questions vary in wording and often need several pieces combined.
  • A few seconds of extra wait is acceptable in exchange for fewer wrong answers.

Avoid it when

  • Most questions are simple lookups and only a few need several searches. Paying for the loop on every question is waste.
    Use Adaptive RAG instead
  • Questions are about the whole collection, such as the main complaints across a year of reviews. No search returns that.
    Use GraphRAG instead
  • Every sentence of the answer must be backed by a passage you can show, and unsupported claims are unacceptable.
    Use Self-RAG instead
  • The first search usually finds the right documents and the risk is that some of them are off-topic.
    Use Corrective RAG instead

What it costs

Model calls
1 per search round + 1 to write (rounds typically 1 to 4)
Latency
medium
Complexity
medium
3

How does a question turn into searches and then an answer?

The researcher reads the question and either answers without documents or writes a query. The search tool pulls passages from the index. The researcher checks what is still missing and searches again with a new query, or hands the passages to the writer. A round limit ends the loop.

needsdocumentsno lookupneededgaps remain,rounds leftcovered, orrounds spentQuestion: The user's question, or a task that needs facts from your documents.QuestionResearcher: Decides whether documents are needed, and writes the next query aimed at what is still missing.ResearcherSearch: Runs one query against the index and returns the top passages with their sources.SearchDocument index: Your documents, split into passages and indexed for search. Built ahead of time and reused.DocumentindexEnough to answer?: Checks each part of the question against the passages found so far, and counts the rounds.Enough toanswer?Writer: Writes the answer from the collected passages only, citing each one it uses.WriterAnswer: The answer with its sources, or a plain statement of what the documents do not cover.Answer
  1. Step 1 of 5

    Is a lookup needed at all?

    The researcher reads the question. A greeting or a request to reword a sentence needs no documents and goes straight to the writer. Anything that depends on your facts gets a first query.

  2. Step 2 of 5

    A targeted search

    The query is the researcher's own wording, not a copy of the question. The search tool returns a handful of passages from the index, each with its source.

  3. Step 3 of 5

    What is still missing?

    The researcher splits the question into the facts it needs and marks which ones the passages cover. Any uncovered fact becomes the next query, worded differently.

  4. Step 4 of 5

    Hand over the evidence

    When every part is covered, or the round limit is reached, the collected passages go to the writer. Gaps that remain are passed on too, so they are stated, not hidden.

  5. Step 5 of 5

    Answer with sources

    The writer answers from the passages alone and cites them. If a part could not be found, the answer says so instead of filling it in from memory.

What each part does
Question
The user's question, or a task that needs facts from your documents.
Researcher
Decides whether documents are needed, and writes the next query aimed at what is still missing.
Search
Runs one query against the index and returns the top passages with their sources.
Document index
Your documents, split into passages and indexed for search. Built ahead of time and reused.
Enough to answer?
Checks each part of the question against the passages found so far, and counts the rounds.
Writer
Writes the answer from the collected passages only, citing each one it uses.
Answer
The answer with its sources, or a plain statement of what the documents do not cover.
  • Researcher → Search: needs documents
  • Researcher → Writer: no lookup needed
  • Search → Enough to answer?: passages
  • Enough to answer? → Researcher: gaps remain, rounds left
  • Enough to answer? → Writer: covered, or rounds spent

Where it connects

Entry
question
QuestionTask

A question or task in plain language, with any context that narrows the search, such as the customer's plan or the product version.

Exit
answer
AnswerEvidence

When: Every part of the question is covered, or the round limit is spent.

The answer, the passages it relies on with their sources, the queries that were run, and the parts the documents did not cover.

Memory kept between runs
index
Evidence

Passages from your documents with their source and date, searchable by query. Read during the run, updated outside it when documents change.

4

What do I ask my assistant to set up?

The agents to launch

Researcher

Decide whether the question needs your documents. If it does, list the facts it depends on and write one search query for the first missing fact. After each search, mark which facts are now covered by a passage and write a new, differently worded query for the next gap. Never answer the question yourself.

Sees: The question, the passages returned so far and the queries already run. Not the writer's draft.

Writer

Answer the question using only the passages provided. Cite the source of each claim. For any part the passages do not cover, say plainly that the documents do not answer it. Do not add facts from general knowledge.

Sees: The question, the collected passages with their sources and the list of uncovered facts. Not the researcher's reasoning or rejected queries.

A prompt you can paste

Works with any assistant that can launch sub-agents. It contains the agents, the procedure and the stop rule above.

Framework-agnostic prompt
You are the orchestrator of a multi-agent workflow that follows the "Agentic RAG" architecture.
Launch every agent listed below as a separate sub-agent with its own context. Give each agent only the context stated for it. Do not do an agent's work yourself. If you cannot launch sub-agents, play each agent in a clearly separated turn and never let it use what is withheld from it.

# Task
[Describe your task here]

# Agentic RAG
The agent decides what to look up, and looks again until it knows.
Input: the task above. It must provide: A question or task in plain language, with any context that narrows the search, such as the customer's plan or the product version. If something essential is missing, ask for it before starting.

Agents to launch:
- Researcher: Decide whether the question needs your documents. If it does, list the facts it depends on and write one search query for the first missing fact. After each search, mark which facts are now covered by a passage and write a new, differently worded query for the next gap. Never answer the question yourself.
  Context for this agent: The question, the passages returned so far and the queries already run. Not the writer's draft.
- Writer: Answer the question using only the passages provided. Cite the source of each claim. For any part the passages do not cover, say plainly that the documents do not answer it. Do not add facts from general knowledge.
  Context for this agent: The question, the collected passages with their sources and the list of uncovered facts. Not the researcher's reasoning or rejected queries.

Procedure:
1. Make sure the documents are indexed for search, split into passages with their source and date. Build the index once, not per question.
2. Launch the researcher on the question. If it says no documents are needed, send the question straight to the writer.
3. Run each query the researcher writes against the index and return the top five passages to it.
4. After each round, read the researcher's coverage list. If facts are still uncovered and fewer than 4 rounds were run, ask it for the next query and search again.
5. Launch the writer in a fresh context with the question, the passages the researcher marked as useful and the list of uncovered facts.

Stop rule: Stop searching when every fact the question needs is covered by a passage, or after 4 search rounds, whichever comes first. Then write once.

Exit:
- `answer`, when: Every part of the question is covered, or the round limit is spent. Deliver: The answer, the passages it relies on with their sources, the queries that were run, and the parts the documents did not cover.

# Final deliverable
The answer with a source for each claim, the queries that were run, and a line listing any part of the question the documents did not cover.

The same thing in code

LangGraph, python, package langgraph
import operator
from typing import Annotated, TypedDict

from langchain.chat_models import init_chat_model
from langgraph.graph import END, START, StateGraph
from pydantic import BaseModel

llm = init_chat_model("anthropic:claude-sonnet-5-5")
MAX_ROUNDS = 4
RESEARCHER = ("Decide whether the question needs the documents. If it does, list the facts it depends on, "
              "mark which are covered by a passage, list the useful passage numbers and write one new, "
              "differently worded query for the next missing fact (empty if none). Never answer the question.")
WRITER = ("Answer the question using only the passages provided. Cite the source of each claim. For any part "
          "the passages do not cover, say plainly that the documents do not answer it. No general knowledge.")

class Research(BaseModel):
    needs_documents: bool
    facts: list[str]
    covered: list[str]
    useful: list[int]
    next_query: str

class Input(TypedDict):
    task: str

class State(Input):
    passages: Annotated[list[str], operator.add]
    queries: Annotated[list[str], operator.add]
    research: Research
    answer: str

def search_index(query: str, k: int = 5) -> list[str]:
    """Stub: query your document index (vector or keyword); return passages with source and date."""
    return [f"(handbook.pdf, 2026-01-10) passage about {query}"][:k]

async def researcher(state: State) -> dict:
    # sees the question, passages so far and queries run; never the writer's draft
    found = "\n".join(f"[{i}] {p}" for i, p in enumerate(state["passages"]))
    out = await llm.with_structured_output(Research).ainvoke([("system", RESEARCHER), (
        "user", f"Question: {state['task']}\nQueries run: {state['queries']}\nPassages:\n{found or '(none yet)'}")])
    return {"research": Research.model_validate(out)}

def search(state: State) -> dict:
    q = state["research"].next_query
    return {"queries": [q], "passages": search_index(q)}

def enough(state: State) -> str:
    # the gate: every fact covered, or 4 search rounds spent
    r = state["research"]
    gaps = set(r.facts) - set(r.covered)
    if not r.needs_documents or not gaps or not r.next_query or len(state["queries"]) >= MAX_ROUNDS:
        return "writer"
    return "search"

async def writer(state: State) -> dict:
    # fresh context: question, useful passages and uncovered facts, not the researcher's reasoning
    r = state["research"]
    kept = "\n".join(state["passages"][i] for i in r.useful if i < len(state["passages"]))
    gaps = [f for f in r.facts if f not in r.covered]
    msg = await llm.ainvoke([("system", WRITER), (
        "user", f"Question: {state['task']}\nPassages:\n{kept or '(none)'}\nNot covered: {gaps}")])
    return {"answer": msg.text}

builder = StateGraph(State, input_schema=Input)
builder.add_node("researcher", researcher)
builder.add_node("search", search)
builder.add_node("writer", writer)
builder.add_edge(START, "researcher")
builder.add_conditional_edges("researcher", enough, ["search", "writer"])
builder.add_edge("search", "researcher")
builder.add_edge("writer", END)
graph = builder.compile()

async def run(task: str):
    s = await graph.ainvoke({"task": task})
    r: Research = s["research"]
    return {"answer": s["answer"], "queries": s["queries"], "not_covered": [f for f in r.facts if f not in r.covered]}

Replace search_index with your real index of dated, sourced passages, built once outside the run.

Claude Agent SDK, python, package claude-agent-sdk
from typing import Any

from claude_agent_sdk import AgentDefinition, ClaudeAgentOptions, ResultMessage, query

MAX_ROUNDS = 4
AGENTS = {
    "researcher": AgentDefinition(
        description="Decides whether documents are needed and writes the next search query.",
        prompt=("Decide whether the question needs the documents. If it does, list the facts it depends on, "
                "mark which are covered by a passage, list the useful passage numbers and write one new, "
                "differently worded query for the next missing fact (empty if none). Never answer the question."),
        tools=[]),
    "writer": AgentDefinition(
        description="Writes the answer from the collected passages only, with citations.",
        prompt=("Answer the question using only the passages provided. Cite the source of each claim. For any "
                "part the passages do not cover, say plainly that the documents do not answer it. "
                "Do not add facts from general knowledge."),
        tools=[]),
}
RESEARCH = {"type": "object", "required": ["needs_documents", "facts", "covered", "useful", "next_query"], "properties": {
    "needs_documents": {"type": "boolean"}, "facts": {"type": "array", "items": {"type": "string"}},
    "covered": {"type": "array", "items": {"type": "string"}},
    "useful": {"type": "array", "items": {"type": "integer"}}, "next_query": {"type": "string"}}}

def search_index(query: str, k: int = 5) -> list[str]:
    """Stub: query your document index (vector or keyword); return passages with source and date."""
    return [f"(handbook.pdf, 2026-01-10) passage about {query}"][:k]

async def ask(prompt: str, schema: dict[str, Any] | None = None) -> Any:
    fmt = {"type": "json_schema", "schema": schema} if schema else None
    options = ClaudeAgentOptions(agents=AGENTS, tools=["Agent"], allowed_tools=["Agent"], output_format=fmt)
    async for message in query(prompt=prompt, options=options):
        if isinstance(message, ResultMessage):
            return message.structured_output if schema else message.result
    raise RuntimeError("no result")

async def run(task: str):
    passages: list[str] = []
    queries: list[str] = []
    while True:
        found = "\n".join(f"[{i}] {p}" for i, p in enumerate(passages)) or "(none yet)"
        # the researcher sees the question, the passages and the queries; never the writer's draft
        r = await ask(f"Use the researcher sub-agent. Give it only this:\nQuestion: {task}\n"
                      f"Queries run: {queries}\nPassages:\n{found}\nReturn its assessment.", RESEARCH)
        gaps = [f for f in r["facts"] if f not in r["covered"]]
        # the gate: every fact covered, or 4 search rounds spent
        if not r["needs_documents"] or not gaps or not r["next_query"] or len(queries) >= MAX_ROUNDS:
            break
        queries.append(r["next_query"])
        passages += search_index(r["next_query"])
    kept = "\n".join(passages[i] for i in r["useful"] if i < len(passages)) or "(none)"
    answer = await ask(f"Use the writer sub-agent in a fresh context. Give it only this:\nQuestion: {task}\n"
                       f"Passages:\n{kept}\nNot covered: {gaps}\nReturn its answer verbatim.")
    return {"answer": answer, "queries": queries, "not_covered": gaps}

Replace search_index with your real index of dated, sourced passages, built once outside the run.

OpenAI Agents SDK, python, package openai-agents
from agents import Agent, Runner
from pydantic import BaseModel

MAX_ROUNDS = 4

class Research(BaseModel):
    needs_documents: bool
    facts: list[str]
    covered: list[str]
    useful: list[int]
    next_query: str

researcher = Agent(
    name="Researcher",
    instructions=("Decide whether the question needs the documents. If it does, list the facts it depends on, "
                  "mark which are covered by a passage, list the useful passage numbers and write one new, "
                  "differently worded query for the next missing fact (empty if none). Never answer the question."),
    output_type=Research)
writer = Agent(
    name="Writer",
    instructions=("Answer the question using only the passages provided. Cite the source of each claim. For any "
                  "part the passages do not cover, say plainly that the documents do not answer it. "
                  "Do not add facts from general knowledge."))

def search_index(query: str, k: int = 5) -> list[str]:
    """Stub: query your document index (vector or keyword); return passages with source and date."""
    return [f"(handbook.pdf, 2026-01-10) passage about {query}"][:k]

async def run(task: str):
    passages: list[str] = []
    queries: list[str] = []
    while True:
        found = "\n".join(f"[{i}] {p}" for i, p in enumerate(passages)) or "(none yet)"
        # the researcher sees the question, the passages and the queries; never the writer's draft
        res = await Runner.run(researcher, f"Question: {task}\nQueries run: {queries}\nPassages:\n{found}")
        r = res.final_output_as(Research)
        gaps = [f for f in r.facts if f not in r.covered]
        # the gate: every fact covered, or 4 search rounds spent
        if not r.needs_documents or not gaps or not r.next_query or len(queries) >= MAX_ROUNDS:
            break
        queries.append(r.next_query)
        passages += search_index(r.next_query)
    kept = "\n".join(passages[i] for i in r.useful if i < len(passages)) or "(none)"
    # a fresh run: the writer never sees the researcher's reasoning or rejected queries
    out = await Runner.run(writer, f"Question: {task}\nPassages:\n{kept}\nNot covered: {gaps}")
    return {"answer": str(out.final_output), "queries": queries, "not_covered": gaps}

Replace search_index with your real index of dated, sourced passages, built once outside the run.

Vercel AI SDK, typescript, package ai
import { anthropic } from '@ai-sdk/anthropic';
import { generateText, Output } from 'ai';
import { z } from 'zod';

const model = anthropic('claude-sonnet-5-5');
const MAX_ROUNDS = 4;

const RESEARCHER =
  'Decide whether the question needs the documents. If it does, list the facts it depends on, mark which ' +
  'are covered by a passage, list the useful passage numbers and write one new, differently worded query ' +
  'for the next missing fact (empty if none). Never answer the question.';
const WRITER =
  'Answer the question using only the passages provided. Cite the source of each claim. For any part the ' +
  'passages do not cover, say plainly that the documents do not answer it. Do not add facts from general knowledge.';
const researchSchema = z.object({
  needsDocuments: z.boolean(),
  facts: z.array(z.string()),
  covered: z.array(z.string()),
  useful: z.array(z.number().int()),
  nextQuery: z.string(),
});

// Stub: query your document index (vector or keyword); return passages with source and date.
async function searchIndex(query: string, k = 5): Promise<string[]> {
  return [`(handbook.pdf, 2026-01-10) passage about ${query}`].slice(0, k);
}

export async function run(task: string) {
  const passages: string[] = [];
  const queries: string[] = [];
  let r: z.infer<typeof researchSchema>;
  let gaps: string[];
  while (true) {
    const found = passages.map((p, i) => `[${i}] ${p}`).join('\n') || '(none yet)';
    // the researcher sees the question, the passages and the queries; never the writer's draft
    ({ output: r } = await generateText({
      model, instructions: RESEARCHER,
      prompt: `Question: ${task}\nQueries run: ${JSON.stringify(queries)}\nPassages:\n${found}`,
      output: Output.object({ schema: researchSchema }),
    }));
    gaps = r.facts.filter((f) => !r.covered.includes(f));
    // the gate: every fact covered, or 4 search rounds spent
    if (!r.needsDocuments || !gaps.length || !r.nextQuery || queries.length >= MAX_ROUNDS) break;
    queries.push(r.nextQuery);
    passages.push(...(await searchIndex(r.nextQuery)));
  }
  const kept = r.useful.filter((i) => i < passages.length).map((i) => passages[i]).join('\n') || '(none)';
  // a fresh call: the writer never sees the researcher's reasoning or rejected queries
  const { text } = await generateText({
    model, instructions: WRITER,
    prompt: `Question: ${task}\nPassages:\n${kept}\nNot covered: ${JSON.stringify(gaps)}`,
  });
  return { answer: text, queries, notCovered: gaps };
}

Replace searchIndex with your real index of dated, sourced passages, built once outside the run.

Snippets are generated when the site is built and type-checked against the SDK versions listed in the repository. Treat them as a starting skeleton.

What goes wrong

  • Re-running the same query with small word changes finds the same passages. Each new query must target a named missing fact.
  • A writer allowed to use general knowledge fills gaps with plausible guesses. Restrict it to the passages and make it name gaps.
  • An index built from outdated documents gives confident wrong answers. Store a date with each passage and refresh the index when documents change.

Chain it

Its result can feed