Agentic Architectures

Catalogue / Sampling & Search

Self-Consistency

Ask the same question several times and keep the majority answer

Shape
fan-out
Model calls
N samples + 1 normalisation (N = 5, topped up to 10 if needed)
Latency
low
Complexity
low

Also known as Majority vote over samples, Sample-and-vote

1

Why does the same prompt sometimes give a wrong answer it would not give twice?

A model that reasons step by step can slip on one step and land on a wrong answer. Slips are random, so they rarely repeat. Self-consistency asks the same prompt several times with some randomness, compares only the final answers and keeps the most frequent one. Sound reasoning converges, slips scatter.

You may need it if

  • The same question, asked twice, sometimes gets two different answers.
  • The answer is short and comparable, such as a number, a date, a category or yes or no.
  • Your prompt is right most of the time, and the errors look random rather than systematic.
2

When does voting over samples not help?

Use it when

  • The question has one checkable final answer: an invoice total, an eligibility decision, a ticket category.
  • The prompt is already good and the remaining errors come from occasional slips.
  • Accuracy matters more than paying five to ten times for each answer.

Avoid it when

  • The prompt is wrong in a consistent way, such as a misread policy. Every sample repeats the error and the vote confirms it.
    Use Ensemble instead
  • The answer is a long text, such as an email or a proposal. There is no majority among texts that all differ.
    Use Reflection instead
  • The answer rests on facts the model does not have. Samples will agree on a guess.
    Use Corrective RAG instead
  • Errors come from a wrong approach chosen early, not from slips along the way.
    Use Tree of Thoughts instead

What it costs

Model calls
N samples + 1 normalisation (N = 5, topped up to 10 if needed)
Latency
low
Complexity
low
3

How do the samples become one answer?

A fan-out and a count. The same prompt runs N times in parallel with some randomness. A normaliser reduces each run to its final answer in a standard form. A plain count picks the majority, and a weak majority triggers one top-up batch of samples.

belowthreshold, notop-up yetmajority foundor top-up spentQuestion: The question or task, with the form the final answer must take.QuestionSamplers: The same prompt, run N times with some randomness. Each run reasons and ends with a final answer.SamplersNormaliser: Reduces each run to its final answer in a standard form, so that 1,200 and 1200.00 count as one.NormaliserClear majority?: Counts the normalised answers and compares the top share with the threshold.Clearmajority?Majority answer: The most frequent answer, with its share of the votes and one reasoning that reached it.Majorityanswer
  1. Step 1 of 5

    Same prompt, several runs

    The question, for example "which refund rule applies to this order?", is sent five times with the same prompt and some randomness. Every run reasons in its own way and ends with an answer.

  2. Step 2 of 5

    Only the final answers count

    The normaliser strips the reasoning and writes each final answer in one standard form. The reasoning is ignored on purpose, so a convincing argument gets one vote like any other.

  3. Step 3 of 5

    Count

    A plain count, not a model, gives each answer's share. If the top answer holds at least 60% of the votes, it wins.

  4. Step 4 of 5

    One top-up when unsure

    Below the threshold, five more samples are drawn once and counted with the first ones. There is no second top-up.

  5. Step 5 of 5

    The majority and its share

    The winning answer leaves with its share of votes. A reader can tell an answer that won 9 to 1 from one that scraped through.

What each part does
Question
The question or task, with the form the final answer must take.
Samplers
The same prompt, run N times with some randomness. Each run reasons and ends with a final answer.
Normaliser
Reduces each run to its final answer in a standard form, so that 1,200 and 1200.00 count as one.
Clear majority?
Counts the normalised answers and compares the top share with the threshold.
Majority answer
The most frequent answer, with its share of the votes and one reasoning that reached it.
  • Samplers → Normaliser: N reasonings
  • Normaliser → Clear majority?: N final answers
  • Clear majority? → Samplers: below threshold, no top-up yet
  • Clear majority? → Majority answer: majority found or top-up spent

Where it connects

Entry
question
QuestionTask

A question or task with one short final answer that can be compared across runs. If the answer form is not stated, the stage fixes one.

Exit
answer
Answer

When: The top answer reaches the threshold, or the single top-up batch has been counted.

The majority answer, its share of the votes, the full tally, one reasoning that reached it, and a flag when the share stayed below the threshold.

4

What do I ask my assistant to set up?

The agents to launch

Sampler, 5 in parallel, plus 5 more once if the majority is weak

Answer the question. Reason step by step, then end with one line that starts with "Final answer:" followed by the answer in the required form. Use the same instructions for every run, with moderate randomness, not the most predictable setting.

Sees: The question and the required answer form. Never another run's output.

Normaliser

For each run, extract the final answer and rewrite it in the standard form: same units, same spelling of categories, numbers without formatting. If two answers mean the same thing, write them identically. If a run has no final answer, mark it as invalid. Do not judge which answer is right.

Sees: The required answer form and the final lines of every run. Not the reasoning that led to them.

A prompt you can paste

Works with any assistant that can launch sub-agents. It contains the agents, the procedure and the stop rule above.

Framework-agnostic prompt
You are the orchestrator of a multi-agent workflow that follows the "Self-Consistency" architecture.
Launch every agent listed below as a separate sub-agent with its own context. Give each agent only the context stated for it. Do not do an agent's work yourself. If you cannot launch sub-agents, play each agent in a clearly separated turn and never let it use what is withheld from it.

# Task
[Describe your task here]

# Self-Consistency
Ask the same question several times and keep the majority answer.
Input: the task above. It must provide: A question or task with one short final answer that can be compared across runs. If the answer form is not stated, the stage fixes one. If something essential is missing, ask for it before starting.

Agents to launch:
- Sampler (5 in parallel, plus 5 more once if the majority is weak): Answer the question. Reason step by step, then end with one line that starts with "Final answer:" followed by the answer in the required form. Use the same instructions for every run, with moderate randomness, not the most predictable setting.
  Context for this agent: The question and the required answer form. Never another run's output.
- Normaliser: For each run, extract the final answer and rewrite it in the standard form: same units, same spelling of categories, numbers without formatting. If two answers mean the same thing, write them identically. If a run has no final answer, mark it as invalid. Do not judge which answer is right.
  Context for this agent: The required answer form and the final lines of every run. Not the reasoning that led to them.

Procedure:
1. Fix the form of the final answer, such as one category from a list, a number with its unit, or yes or no.
2. Launch 5 samplers at the same time, each in a fresh context, with the same prompt and moderate randomness.
3. Launch the normaliser on the final lines of all runs.
4. Count the normalised answers, leaving out invalid ones. If the top answer has at least 60% of the valid votes, it wins.
5. If it does not and no top-up was drawn, launch 5 more samplers, normalise their answers and count all 10 together.

Stop rule: Stop when the top answer holds 60% of the valid votes, or after one top-up of 5 samples. Never draw a third batch.

Exit:
- `answer`, when: The top answer reaches the threshold, or the single top-up batch has been counted. Deliver: The majority answer, its share of the votes, the full tally, one reasoning that reached it, and a flag when the share stayed below the threshold.

# Final deliverable
The majority answer, the full tally, its share of the votes, one reasoning that reached it, and a clear flag if the share stayed below 60%.

The same thing in code

LangGraph, python, package langgraph
import operator
from collections import Counter
from typing import Annotated, TypedDict

from langchain.chat_models import init_chat_model
from langgraph.graph import END, START, StateGraph
from langgraph.types import Send
from pydantic import BaseModel

# moderate randomness for the samplers, so that slips scatter instead of repeating
sampler_llm = init_chat_model("anthropic:claude-sonnet-5-5", temperature=0.8)
normaliser_llm = init_chat_model("anthropic:claude-sonnet-5-5", temperature=0)

SAMPLER = ("Answer the question. Reason step by step, then end with one line that starts with "
           "'Final answer:' followed by the answer in the required form.")
NORMALISER = ("For each final line, rewrite the answer in the standard form: same units, same spelling "
              "of categories, numbers without formatting. Write INVALID when a run has no final answer. "
              "Do not judge which answer is right.")
BATCH, THRESHOLD, MAX_BATCHES = 5, 0.6, 2

class Normalised(BaseModel):
    answers: list[str]  # one per run, same order

class State(TypedDict):
    task: str
    runs: Annotated[list[str], operator.add]
    answers: list[str]
    batches: int
    winner: str
    share: float

class SamplerInput(TypedDict):
    task: str

def launch_batch(state: State) -> list[Send]:
    return [Send("sampler", {"task": state["task"]}) for _ in range(BATCH)]

async def sampler(state: SamplerInput) -> dict:
    # each run gets only the question, never another run's output
    msg = await sampler_llm.ainvoke([("system", SAMPLER), ("user", state["task"])])
    return {"runs": [msg.text]}

async def normaliser(state: State) -> dict:
    # the normaliser sees the final lines only, not the reasoning
    finals = [r.rsplit("Final answer:", 1)[1].strip() if "Final answer:" in r else "" for r in state["runs"]]
    lines = "\n".join(f"{i}. {f}" for i, f in enumerate(finals))
    out = await normaliser_llm.with_structured_output(Normalised).ainvoke(
        [("system", NORMALISER), ("user", f"Question and answer form: {state['task']}\nFinal lines:\n{lines}")])
    return {"answers": Normalised.model_validate(out).answers, "batches": state["batches"] + 1}

def tally(state: State) -> dict:
    # a plain count, not a model
    valid = [a for a in state["answers"] if a != "INVALID"]
    if not valid:
        return {"winner": "", "share": 0.0}
    winner, votes = Counter(valid).most_common(1)[0]
    return {"winner": winner, "share": votes / len(valid)}

def clear_majority(state: State) -> list[Send] | str:
    # the gate: 60% of valid votes, or stop after one top-up batch
    if state["share"] >= THRESHOLD or state["batches"] >= MAX_BATCHES:
        return END
    return launch_batch(state)

builder = StateGraph(State)
builder.add_node("sampler", sampler)
builder.add_node("normaliser", normaliser)
builder.add_node("tally", tally)
builder.add_conditional_edges(START, launch_batch, ["sampler"])
builder.add_edge("sampler", "normaliser")
builder.add_edge("normaliser", "tally")
builder.add_conditional_edges("tally", clear_majority, ["sampler", END])
graph = builder.compile()

async def run(task: str):
    s = await graph.ainvoke({"task": task, "runs": [], "answers": [], "batches": 0, "winner": "", "share": 0.0})
    pairs = list(zip(s["runs"], s["answers"]))
    return {
        "answer": s["winner"],
        "share": s["share"],
        "tally": Counter(a for _, a in pairs if a != "INVALID"),
        "reasoning": next((r for r, a in pairs if a == s["winner"]), ""),
        "weak_majority": s["share"] < THRESHOLD,
    }

Calibrate the 60% threshold and the sampler temperature on a labelled set of your own questions before trusting the vote.

Claude Agent SDK, python, package claude-agent-sdk
from collections import Counter
from typing import Any

from claude_agent_sdk import AgentDefinition, ClaudeAgentOptions, ResultMessage, query

BATCH, THRESHOLD, MAX_BATCHES = 5, 0.6, 2

AGENTS = {
    "sampler": AgentDefinition(
        description="Answers the question once, reasoning step by step.",
        prompt=("Answer the question. Reason step by step, then end with one line that starts with "
                "'Final answer:' followed by the answer in the required form."),
        tools=[],
    ),
    "normaliser": AgentDefinition(
        description="Rewrites a list of final answers in one standard form.",
        prompt=("For each final line, rewrite the answer in the standard form: same units, same spelling, "
                "numbers without formatting. Write INVALID when a line has no answer. Do not judge which is right."),
        tools=[],
    ),
}

BATCH_SCHEMA = {"type": "json_schema", "schema": {
    "type": "object", "required": ["runs"],
    "properties": {"runs": {"type": "array", "items": {
        "type": "object", "required": ["reasoning", "answer"],
        "properties": {"reasoning": {"type": "string"}, "answer": {"type": "string"}}}}}}}

OPTIONS = ClaudeAgentOptions(agents=AGENTS, tools=["Agent"], allowed_tools=["Agent"], output_format=BATCH_SCHEMA)

async def ask(prompt: str) -> Any:
    async for message in query(prompt=prompt, options=OPTIONS):
        if isinstance(message, ResultMessage):
            return message.structured_output
    raise RuntimeError("no result")

async def sample_batch(task: str) -> list[dict[str, str]]:
    # one batch: N sampler sub-agents in parallel, each in its own context, then the normaliser
    out = await ask(
        f"Launch the sampler sub-agent {BATCH} times in parallel, giving each only this question:\n{task}\n"
        "Then give the normaliser sub-agent the question and only the 'Final answer:' lines, not the reasoning. "
        "Return each run's full text as reasoning and its normalised answer.")
    return out["runs"]

async def run(task: str):
    runs: list[dict[str, str]] = []
    winner, share = "", 0.0
    for _ in range(MAX_BATCHES):  # the gate: one top-up at most, never a third batch
        runs += await sample_batch(task)
        votes = Counter(r["answer"] for r in runs if r["answer"] != "INVALID")
        if votes:
            winner, n = votes.most_common(1)[0]
            share = n / sum(votes.values())
        if share >= THRESHOLD:
            break
    return {
        "answer": winner,
        "share": share,
        "tally": Counter(r["answer"] for r in runs if r["answer"] != "INVALID"),
        "reasoning": next((r["reasoning"] for r in runs if r["answer"] == winner), ""),
        "weak_majority": share < THRESHOLD,
    }

The Agent SDK sets no temperature, so variety comes only from separate sub-agent contexts; check that runs really differ before trusting the vote.

OpenAI Agents SDK, python, package openai-agents
import asyncio
from collections import Counter

from agents import Agent, ModelSettings, Runner
from pydantic import BaseModel

BATCH, THRESHOLD, MAX_BATCHES = 5, 0.6, 2

class Normalised(BaseModel):
    answers: list[str]  # one per final line, same order; INVALID when missing

sampler = Agent(
    name="Sampler",
    instructions=("Answer the question. Reason step by step, then end with one line that starts with "
                  "'Final answer:' followed by the answer in the required form."),
    model_settings=ModelSettings(temperature=0.8),  # some randomness, not the most predictable setting
)

normaliser = Agent(
    name="Normaliser",
    instructions=("For each final line, rewrite the answer in the standard form: same units, same spelling "
                  "of categories, numbers without formatting. Write INVALID when a line has no answer. "
                  "Do not judge which answer is right."),
    output_type=Normalised,
)

def final_line(text: str) -> str:
    return text.rsplit("Final answer:", 1)[1].strip() if "Final answer:" in text else ""

async def sample_batch(task: str) -> list[str]:
    # fan-out: separate runs, none sees another's output
    results = await asyncio.gather(*(Runner.run(sampler, task) for _ in range(BATCH)))
    return [str(r.final_output) for r in results]

async def run(task: str):
    runs: list[str] = []
    answers: list[str] = []
    winner, share = "", 0.0
    for _ in range(MAX_BATCHES):  # the gate: one top-up at most
        runs += await sample_batch(task)
        lines = "\n".join(f"{i}. {final_line(r)}" for i, r in enumerate(runs))
        result = await Runner.run(normaliser, f"Question and answer form: {task}\nFinal lines:\n{lines}")
        answers = result.final_output_as(Normalised).answers
        votes = Counter(a for a in answers if a != "INVALID")
        if votes:
            winner, n = votes.most_common(1)[0]
            share = n / sum(votes.values())
        if share >= THRESHOLD:
            break
    return {
        "answer": winner,
        "share": share,
        "tally": Counter(a for a in answers if a != "INVALID"),
        "reasoning": next((r for r, a in zip(runs, answers) if a == winner), ""),
        "weak_majority": share < THRESHOLD,
    }

Confirm that the SDK's default model accepts a temperature setting, or pick one that does, so the five runs really vary.

Vercel AI SDK, typescript, package ai
import { anthropic } from '@ai-sdk/anthropic';
import { generateText, Output } from 'ai';
import { z } from 'zod';

const model = anthropic('claude-sonnet-5-5');
const BATCH = 5, THRESHOLD = 0.6, MAX_BATCHES = 2;

const SAMPLER =
  "Answer the question. Reason step by step, then end with one line that starts with " +
  "'Final answer:' followed by the answer in the required form.";
const NORMALISER =
  'For each final line, rewrite the answer in the standard form: same units, same spelling of ' +
  'categories, numbers without formatting. Write INVALID when a line has no answer. ' +
  'Do not judge which answer is right.';

function finalLine(text: string): string {
  const i = text.lastIndexOf('Final answer:');
  return i < 0 ? '' : text.slice(i + 'Final answer:'.length).trim();
}

async function sampleBatch(task: string): Promise<string[]> {
  // fan-out: each run sees only the question; moderate randomness
  const runs = await Promise.all(
    Array.from({ length: BATCH }, () =>
      generateText({ model, instructions: SAMPLER, prompt: task, temperature: 0.8 })),
  );
  return runs.map((r) => r.text);
}

async function normalise(task: string, runs: string[]): Promise<string[]> {
  // the normaliser gets the final lines only, not the reasoning
  const lines = runs.map((r, i) => `${i}. ${finalLine(r)}`).join('\n');
  const { output } = await generateText({
    model,
    instructions: NORMALISER,
    prompt: `Question and answer form: ${task}\nFinal lines:\n${lines}`,
    output: Output.object({ schema: z.object({ answers: z.array(z.string()) }) }),
  });
  return output.answers;
}

export async function run(task: string) {
  let runs: string[] = [];
  let answers: string[] = [];
  let winner = '', share = 0;
  const tally = new Map<string, number>();
  for (let batch = 0; batch < MAX_BATCHES; batch++) { // the gate: one top-up at most
    runs = runs.concat(await sampleBatch(task));
    answers = await normalise(task, runs);
    tally.clear();
    winner = '';
    share = 0;
    for (const a of answers) if (a !== 'INVALID') tally.set(a, (tally.get(a) ?? 0) + 1);
    const valid = [...tally.values()].reduce((s, n) => s + n, 0);
    for (const [a, n] of tally) if (valid && n / valid > share) { winner = a; share = n / valid; }
    if (share >= THRESHOLD) break;
  }
  return {
    answer: winner,
    share,
    tally: Object.fromEntries(tally),
    reasoning: runs[answers.indexOf(winner)] ?? '',
    weakMajority: share < THRESHOLD,
  };
}

Calibrate the 60% threshold and the sampler temperature on a labelled set of your own questions before trusting the vote.

Snippets are generated when the site is built and type-checked against the SDK versions listed in the repository. Treat them as a starting skeleton.

What goes wrong

  • At the most predictable setting all runs are identical and the vote is empty. Allow some randomness.
  • Counting before normalising splits one answer into several spellings and hands the win to a wrong one.
  • The vote cannot fix a wrong prompt. If samples agree on wrong answers, test the prompt, or switch to an ensemble.

Chain it

Its result can feed

Source