1How do I catch the confident little errors inside an otherwise good answer?
A fluent answer can hide a wrong date, a made-up feature or a misquoted figure. Chain of Verification turns each claim into a short question, answers every question separately without the original text in view, and rewrites the answer keeping only the claims that the checks confirmed.
You may need it if
- Answers read well but contain the odd invented name, number or product feature.
- Reviewers spend their time fact-checking lists, not improving the writing.
- Errors are in individual claims, not in the overall structure of the answer.
2When is checking every claim not worth it?
Use it when
- The answer is a list of facts, such as names, dates, figures, features or quotes.
- Each claim can be checked on its own with a short question.
- A single wrong fact costs trust, for example in a sales brief, a support reply or a report.
Avoid it when
- The model simply does not know the facts. Checking from the same memory confirms nothing; fetch sources first.
Use Corrective RAG instead - The weakness is structure, tone or argument, not individual facts.
Use Reflection instead - There is one short answer, like a number or a category, and you want the most reliable version.
Use Self-Consistency instead - The answer must arrive in a live chat within a second or two.
What it costs
- Model calls
- 3 + number of verification questions (often 8 to 15)
- Latency
- medium
- Complexity
- low
3How does a draft get checked claim by claim?
A straight line with one fan-out. A drafter writes a baseline answer. A planner turns its claims into verification questions. Several verifiers answer those questions independently, without the draft. A reviser compares and rewrites, dropping or fixing what failed.
- Step 1 of 5
A baseline answer
The drafter answers as a single prompt would. For example, a brief on a prospect listing its founders, funding and main products. If the input is already a draft, it is used as is.
- Step 2 of 5
Claims become questions
The planner extracts every checkable claim and writes one question per claim, such as "Who founded this company?". Questions are neutral: they do not contain the draft's answer.
- Step 3 of 5
Answered blind
Each verifier gets one question and nothing else. It cannot see the draft, so it cannot simply agree with it. Verifiers may use the same tools or sources as the drafter.
- Step 4 of 5
Compare and rewrite
The reviser lines up each claim with its independent answer. Matching claims stay. Contradicted claims are corrected or removed. Uncertain ones are removed or marked as unconfirmed.
- Step 5 of 5
Release with a change log
The final answer leaves with the list of claims that changed, so a reader can see what the checks caught and what was taken out.
What each part does
- Question
- The question or task, or an existing draft answer to check.
- Drafter
- Writes a baseline answer in one pass. Skipped when the input is already a draft.
- Question planner
- Lists each factual claim in the draft and turns it into one short, neutral question.
- Verifiers
- Each answers one verification question on its own, with no access to the draft.
- Reviser
- Compares each claim with its independent answer, then rewrites keeping only confirmed claims.
- Verified answer
- The rewritten answer, with a list of claims removed or corrected.
- Drafter → Question planner: baseline draft
- Question planner → Verifiers: one question each
- Verifiers → Reviser: independent answers
Where it connects
A question or task to answer, or an existing answer whose factual claims should be checked.
When: Every verification question has been answered and the reviser has rewritten the draft.
The rewritten answer, then each claim checked with its status: confirmed, corrected or removed.
4What do I ask my assistant to set up?
The agents to launch
Drafter
Answer the question fully in one pass, as you normally would. State facts plainly so each one can be checked later.
Sees: The question or task only.
Question planner
List every factual claim in the draft: names, dates, figures, features, quotes. For each, write one short question that asks for the fact without stating it. Skip opinions and wording. Return a numbered list of claim and question pairs.
Sees: The question and the draft.
Verifier, one per verification question, each in a fresh context
Answer this one question as accurately as you can, in one or two sentences. If you are not sure, say so plainly instead of guessing.
Sees: A single verification question, plus any sources or tools allowed. Never the draft, the other questions or the original task.
Reviser
Compare each claim with its independent answer. Keep claims that match. Correct claims that were contradicted. Remove or mark claims the verifier was unsure about. Rewrite the answer with these changes and keep everything else as it was.
Sees: The question, the draft, the claim and question list and every verifier answer.
A prompt you can paste
Works with any assistant that can launch sub-agents. It contains the agents, the procedure and the stop rule above.
You are the orchestrator of a multi-agent workflow that follows the "Chain of Verification" architecture. Launch every agent listed below as a separate sub-agent with its own context. Give each agent only the context stated for it. Do not do an agent's work yourself. If you cannot launch sub-agents, play each agent in a clearly separated turn and never let it use what is withheld from it. # Task [Describe your task here] # Chain of Verification Check each claim on its own, then rewrite with only what survived. Input: the task above. It must provide: A question or task to answer, or an existing answer whose factual claims should be checked. If something essential is missing, ask for it before starting. Agents to launch: - Drafter: Answer the question fully in one pass, as you normally would. State facts plainly so each one can be checked later. Context for this agent: The question or task only. - Question planner: List every factual claim in the draft: names, dates, figures, features, quotes. For each, write one short question that asks for the fact without stating it. Skip opinions and wording. Return a numbered list of claim and question pairs. Context for this agent: The question and the draft. - Verifier (one per verification question, each in a fresh context): Answer this one question as accurately as you can, in one or two sentences. If you are not sure, say so plainly instead of guessing. Context for this agent: A single verification question, plus any sources or tools allowed. Never the draft, the other questions or the original task. - Reviser: Compare each claim with its independent answer. Keep claims that match. Correct claims that were contradicted. Remove or mark claims the verifier was unsure about. Rewrite the answer with these changes and keep everything else as it was. Context for this agent: The question, the draft, the claim and question list and every verifier answer. Procedure: 1. If the input is already an answer or a draft, use it as the baseline. Otherwise launch the drafter on the question. 2. Launch the planner on the draft. Cap the list at 15 questions, keeping the claims that matter most to the reader. 3. Launch one verifier per question, each in a separate fresh context with only its question. They can run in parallel. 4. Launch the reviser with the draft, the claim list and all verifier answers. 5. Check that no claim marked contradicted remains in the final text unchanged. Stop rule: Stop after one pass. Do not verify the rewritten answer again unless a person asks; a second pass rarely finds new errors and doubles the cost. Exit: - `verified`, when: Every verification question has been answered and the reviser has rewritten the draft. Deliver: The rewritten answer, then each claim checked with its status: confirmed, corrected or removed. # Final deliverable The verified answer, then a table of claims with their verification question, the independent answer and the status: confirmed, corrected or removed.
The same thing in code
import operator
from typing import Annotated, TypedDict, cast
from langchain.chat_models import init_chat_model
from langgraph.graph import END, START, StateGraph
from langgraph.types import Send
from pydantic import BaseModel
MAX_QUESTIONS = 15
model = init_chat_model("anthropic:claude-sonnet-5-5")
class Check(BaseModel):
claim: str
question: str
class Checks(BaseModel):
checks: list[Check]
class Input(TypedDict):
question: str
class State(TypedDict):
question: str
draft: str
checks: list[Check]
answers: Annotated[list[tuple[int, str]], operator.add] # fan-in from the verifiers
verified: str
class Blind(TypedDict):
index: int
question: str
async def drafter(state: State) -> dict:
msg = await model.ainvoke("Answer fully in one pass. State facts plainly so each one can "
f"be checked later.\n\n{state['question']}")
return {"draft": msg.text}
async def planner(state: State) -> dict:
out = cast(Checks, await model.with_structured_output(Checks).ainvoke(
"List every factual claim in the draft: names, dates, figures, features, quotes. For "
"each, write one short question that asks for the fact without stating it. Skip "
f"opinions and wording.\nQuestion: {state['question']}\nDraft:\n{state['draft']}"))
return {"checks": out.checks[:MAX_QUESTIONS]} # cap at 15 questions
def fan_out(state: State) -> list[Send]:
# each verifier gets one question and nothing else: no draft, no task
return [Send("verifier", {"index": i, "question": c.question}) for i, c in enumerate(state["checks"])]
async def verifier(state: Blind) -> dict:
msg = await model.ainvoke("Answer this one question accurately in one or two sentences. If "
f"you are not sure, say so plainly instead of guessing.\n\n{state['question']}")
return {"answers": [(state["index"], msg.text)]}
async def reviser(state: State) -> dict:
answers = dict(state["answers"])
table = "\n".join(f"{i + 1}. Claim: {c.claim}\n Q: {c.question}\n A: {answers.get(i, '')}"
for i, c in enumerate(state["checks"]))
msg = await model.ainvoke(
"Compare each claim with its independent answer. Keep claims that match, correct "
"contradicted ones, remove or mark unsure ones. Rewrite the answer with only these "
"changes, then list each claim as confirmed, corrected or removed.\n"
f"Question: {state['question']}\nDraft:\n{state['draft']}\nChecks:\n{table}")
return {"verified": msg.text}
graph = StateGraph(State, input_schema=Input)
graph.add_node("drafter", drafter)
graph.add_node("planner", planner)
graph.add_node("verifier", verifier)
graph.add_node("reviser", reviser)
graph.add_edge(START, "drafter")
graph.add_edge("drafter", "planner")
graph.add_conditional_edges("planner", fan_out, ["verifier"])
graph.add_edge("verifier", "reviser")
graph.add_edge("reviser", END) # one pass only, the revision is not re-verified
app = graph.compile()
async def run(task: str):
return await app.ainvoke({"question": task})
Give the verifiers a search tool or your documents; without sources they share the drafter's memory and confirm its errors.
import json
from typing import Any
from claude_agent_sdk import AgentDefinition, ClaudeAgentOptions, ResultMessage, query
MAX_QUESTIONS = 15
AGENTS = {
"drafter": AgentDefinition(
description="Writes a baseline answer in one pass.",
prompt="Answer the question fully in one pass. State facts plainly so each one can be "
"checked later."),
"planner": AgentDefinition(
description="Turns each factual claim in a draft into a neutral question.",
prompt="List every factual claim in the draft: names, dates, figures, features, quotes. "
"For each, write one short question that asks for the fact without stating it. "
"Skip opinions and wording."),
"verifier": AgentDefinition(
description="Answers one verification question blind.",
prompt="Answer this one question as accurately as you can, in one or two sentences. If "
"you are not sure, say so plainly instead of guessing.",
tools=["WebSearch"]), # the sources a verifier may use; swap in your own documents
"reviser": AgentDefinition(
description="Rewrites a draft keeping only confirmed claims.",
prompt="Compare each claim with its independent answer. Keep claims that match, correct "
"contradicted ones, remove or mark unsure ones. Rewrite the answer with only these "
"changes, then list each claim as confirmed, corrected or removed."),
}
CHECKS = {"type": "object", "required": ["checks"], "properties": {"checks": {"type": "array",
"items": {"type": "object", "required": ["claim", "question"],
"properties": {"claim": {"type": "string"}, "question": {"type": "string"}}}}}}
ANSWERS = {"type": "object", "required": ["answers"],
"properties": {"answers": {"type": "array", "items": {"type": "string"}}}}
async def ask(prompt: str, schema: dict[str, Any] | None = None) -> ResultMessage:
options = ClaudeAgentOptions(
agents=AGENTS, allowed_tools=["Agent", "WebSearch"],
output_format={"type": "json_schema", "schema": schema} if schema else None)
async for msg in query(prompt=prompt, options=options):
if isinstance(msg, ResultMessage):
return msg
raise RuntimeError("no result")
async def run(task: str):
draft = (await ask(f"Launch the drafter agent on this question and return its answer:\n{task}")).result
plan = await ask(f"Launch the planner agent with this question and draft.\nQuestion: {task}\n"
f"Draft:\n{draft}", CHECKS)
checks = plan.structured_output["checks"][:MAX_QUESTIONS] # cap at 15 questions
# the fan-out: this session holds only the questions, never the draft or the task
questions = "\n".join(f"{i + 1}. {c['question']}" for i, c in enumerate(checks))
blind = await ask("For each question below, launch a separate verifier agent in parallel, "
"passing it that one question only. Return their answers in order.\n"
+ questions, ANSWERS)
answers = blind.structured_output["answers"]
table = [{**c, "answer": a} for c, a in zip(checks, answers)]
final = await ask("Launch the reviser agent with this input and return its reply:\n"
f"Question: {task}\nDraft:\n{draft}\nChecks: {json.dumps(table)}")
return {"verified": final.result, "checks": table} # one pass, no re-verification
Replace WebSearch with your own document source if answers must come from internal material, and keep the verifier session free of the draft.
import asyncio
from agents import Agent, Runner
from pydantic import BaseModel
MAX_QUESTIONS = 15
class Check(BaseModel):
claim: str
question: str
class Checks(BaseModel):
checks: list[Check]
drafter = Agent(
name="Drafter",
instructions="Answer the question fully in one pass. State facts plainly so each one can be "
"checked later.")
planner = Agent(
name="Question planner", output_type=Checks,
instructions="List every factual claim in the draft: names, dates, figures, features, "
"quotes. For each, write one short question that asks for the fact without "
"stating it. Skip opinions and wording.")
verifier = Agent(
name="Verifier",
instructions="Answer this one question as accurately as you can, in one or two sentences. "
"If you are not sure, say so plainly instead of guessing.")
reviser = Agent(
name="Reviser",
instructions="Compare each claim with its independent answer. Keep claims that match, "
"correct contradicted ones, remove or mark unsure ones. Rewrite the answer with "
"only these changes, then list each claim as confirmed, corrected or removed.")
async def verify(question: str) -> str:
# a fresh run with one question: the verifier never sees the draft or the task
return str((await Runner.run(verifier, question)).final_output)
async def run(task: str):
draft = str((await Runner.run(drafter, task)).final_output)
plan = await Runner.run(planner, f"Question: {task}\nDraft:\n{draft}")
checks = plan.final_output_as(Checks).checks[:MAX_QUESTIONS] # cap at 15 questions
answers = await asyncio.gather(*(verify(c.question) for c in checks)) # the fan-out
table = "\n".join(f"{i + 1}. Claim: {c.claim}\n Q: {c.question}\n A: {a}"
for i, (c, a) in enumerate(zip(checks, answers)))
final = await Runner.run(reviser, f"Question: {task}\nDraft:\n{draft}\nChecks:\n{table}")
# one pass only: the revised answer is not verified again
return {"verified": final.final_output,
"checks": [{"claim": c.claim, "question": c.question, "answer": a}
for c, a in zip(checks, answers)]}
Give the verifiers a search tool or your documents; without sources they share the drafter's memory and confirm its errors.
import { generateText, Output } from 'ai';
import { anthropic } from '@ai-sdk/anthropic';
import { z } from 'zod';
const model = anthropic('claude-sonnet-5-5');
const MAX_QUESTIONS = 15;
const Checks = z.object({
checks: z.array(z.object({ claim: z.string(), question: z.string() })),
});
// a fresh call with one question: the verifier never sees the draft or the task
async function verify(question: string) {
const { text } = await generateText({
model,
instructions:
'Answer this one question as accurately as you can, in one or two sentences. ' +
'If you are not sure, say so plainly instead of guessing.',
prompt: question,
});
return text;
}
export async function run(task: string) {
const { text: draft } = await generateText({
model,
instructions: 'Answer the question fully in one pass. State facts plainly so each one can be checked later.',
prompt: task,
});
const { output } = await generateText({
model,
instructions:
'List every factual claim in the draft: names, dates, figures, features, quotes. For each, ' +
'write one short question that asks for the fact without stating it. Skip opinions and wording.',
prompt: `Question: ${task}\nDraft:\n${draft}`,
output: Output.object({ schema: Checks }),
});
const checks = output.checks.slice(0, MAX_QUESTIONS); // cap at 15 questions
const answers = await Promise.all(checks.map((c) => verify(c.question))); // the fan-out
const table = checks.map((c, i) => ({ ...c, answer: answers[i] }));
const { text: verified } = await generateText({
model,
instructions:
'Compare each claim with its independent answer. Keep claims that match, correct contradicted ' +
'ones, remove or mark unsure ones. Rewrite the answer with only these changes, then list each ' +
'claim as confirmed, corrected or removed.',
prompt: `Question: ${task}\nDraft:\n${draft}\nChecks: ${JSON.stringify(table)}`,
});
return { verified, checks: table }; // one pass, no re-verification
}
Give the verifiers a search tool or your documents; without sources they share the drafter's memory and confirm its errors.
Snippets are generated when the site is built and type-checked against the SDK versions listed in the repository. Treat them as a starting skeleton.
What goes wrong
- Showing the draft to verifiers defeats the whole pattern. They must answer blind.
- Questions that restate the claim invite agreement. Ask for the fact, not for confirmation.
- Without sources, verifiers share the drafter's memory. For facts the model may never have learned, give verifiers search or documents.
- A reviser that rewrites freely can add new, unchecked claims. Limit it to fixing and removing.
Chain it
Its result can feed
- Reflection direct
Treat the verified answer as an existing result to improve against quality criteria, keeping confirmed and corrected claims intact and not reintroducing removed ones.
- Debate direct
Treat the verified answer as an existing answer to challenge on the original task, using its claim statuses as evidence the debaters may contest.
- Constitutional Critique direct
Check the verified answer rule by rule against the constitution and fix only failed rules, without altering the factual content of confirmed or corrected claims.
- Textual Backpropagation direct
Give the verified answer and its claim-status list to the first step as the existing result to improve. Keep the confirmed claims and do not bring back claims marked removed.
It can take the result of
- Reflection
- Tool Use
- ReAct
- Planning
- Supervisor & Specialists
- Plan–Execute–Verify
- Blackboard
- Episodic + Semantic Memory
- Tree of Thoughts
- Meta-Controller
- Ensemble
- Scored Archive Self-improvement
- Self-Discover
- Self-Consistency
- Language Agent Tree Search
- Agentic RAG
- Corrective RAG
- Self-RAG
- GraphRAG
- Debate
- Tiered Memory
- Computer Use
- Workflow Memory
- Magentic Orchestration
- Map-Reduce
- ReWOO
- LLMCompiler
- Code as Action
- Dual LLM
- LLM Cascade
- Shared Task List
- Graph of Thoughts
- As-Needed Decomposition
- Dynamic Agent Generation
- Tool Retrieval
- Trusted Monitoring
- Textual Backpropagation
- Recursive Agent Harness
- Recoverable Execution
- Speculative Actions