1What if the best result is a combination of several branches?
Each model output is a vertex in a graph, linked to the outputs it was made from. A plan written before the run says when to generate several attempts, score them and keep the best, merge several kept results into one new result, or improve a result in a loop. A tree can only extend and prune separate paths.
You may need it if
- The work splits into parts, and the parts' results must be combined into one result.
- Two candidates each get a different half right, and choosing one loses the other half.
- A single merge of all the parts comes out with duplicates, gaps or contradictions.
- You already know how the problem decomposes and what a good partial result looks like.
2When is a graph of thoughts more machinery than the task needs?
Use it when
- The task decomposes into parts whose results can be merged, such as several sources into one document.
- You can write the sequence of operations in advance: where to split, how many attempts, when to merge.
- A partial result can be scored, by a rubric or by code, before the whole is finished.
- A weak merge costs more than a few dozen extra model calls.
Avoid it when
- The problem is one line of reasoning where an early choice may be wrong. There is nothing to merge, only paths to extend and prune.
Use Tree of Thoughts instead - Each part is processed once and the results are merged once, with no competing attempts to score.
Use Map-Reduce instead - Nobody knows before the run how the problem splits. The plan of operations here is written by hand in advance.
Use As-Needed Decomposition instead - The question has one short final answer, and the risk is a random slip rather than a poor combination.
Use Self-Consistency instead - Partial results cannot be scored. Keeping the best attempt would then be a random choice.
What it costs
- Model calls
- per operation: attempts + scores; e.g. 4 parts × 6, merge 6, refine 4 = 34
- Latency
- high
- Complexity
- high
3Who decides when to split, score, merge and refine?
A controller in plain code follows a plan of operations fixed before the run. Each operation takes thoughts from the graph and sends them to a generator, an aggregator or a refiner. A scorer rates every new thought and the best are kept. The thought left when the plan is exhausted is the result.
- Step 1 of 6
A plan written before the run
A product team has four long interview transcripts and wants one list of the customers' main problems. The plan is fixed in advance: one part per transcript, three attempts per part, keep the best, merge, then refine twice at most.
- Step 2 of 6
Several attempts per part
The first operation is a generation. For each transcript, three generators each write a list of the problems it mentions. They work apart, so the three lists differ in what they catch.
- Step 3 of 6
Score, and keep the best
The scorer rates each list against the rubric: every problem traced to a quote, nothing invented, no duplicates. The controller records all twelve lists in the graph and keeps the best one for each transcript.
- Step 4 of 6
Several thoughts in, one out
The plan now says merge. The aggregator receives the four kept lists and writes a single list from them. The new thought has four incoming links. It is attempted three times, and the scorer picks the best merge.
- Step 5 of 6
Refine in a loop
The best merge scores 6: two problems are stated twice in different words. The refiner rewrites it with those defects in hand. The scorer rates the new version, and the controller keeps whichever of the two scores higher.
- Step 6 of 6
The plan is exhausted
The refined list scores 8, so the second refinement is skipped. No operation is left. The kept thought leaves as the result, with the graph showing which transcript lists it was built from.
What each part does
- Problem
- The task or question, with the material it applies to and the constraints on the result.
- Next operation?
- Plain code. Follows the operation plan, picks the thoughts each operation works on and keeps the best-scored ones.
- Thought graph
- Every thought produced so far, with its score and links to the thoughts it was made from.
- Generator
- Writes several new thoughts from one: the parts of a task, or several attempts at one part.
- Aggregator
- Writes one new thought from several kept thoughts. This is the step a tree does not have.
- Refiner
- Rewrites one thought to remove the defects found by the scorer, without changing what it covers.
- Scorer
- Rates each new thought against a fixed rubric and lists its defects. Can be plain code when the task allows.
- Result
- The thought kept after the last operation, with its score and the graph that led to it.
- Thought graph → Next operation?: thoughts to work on
- Next operation? → Generator: plan says: generate
- Next operation? → Aggregator: plan says: merge
- Next operation? → Refiner: plan says: refine
- Scorer → Next operation?: scored thoughts
- Next operation? → Thought graph: vertices, links, scores
- Next operation? → Result: plan exhausted
Where it connects
A problem that splits into parts whose results can be merged, with its constraints. The plan of operations and the scoring rubric are already configured.
When: The last operation of the plan has run, or the refined thought reached the target score.
The final thought as the answer, its score, and the list of thoughts with their scores and the thoughts each one was made from.
All thoughts of the run as vertices, each with its text, its score and links to its source thoughts. Dropped attempts stay recorded. Discarded when the run ends.
4What do I ask my assistant to set up?
The agents to launch
Generator, 3 per part, each in a fresh context; one more to split the task into parts
You receive a task and one part of it. Produce a complete result for that part only. Do not cover the other parts and do not refer to them. Work from the material given and add nothing that is not in it.
Sees: The task and its one part with the material for it. Not the other parts, not the other attempts, not any score.
Aggregator, 3 merge attempts, each in a fresh context
You receive several partial results for one task. Write a single result that keeps everything they contain, states each item once and resolves contradictions explicitly. Do not add content that is in none of them.
Sees: The task and the kept partial results. Not the attempts that were dropped, not the scores.
Refiner
You receive one result and the list of defects a reviewer found in it. Rewrite the result so that each defect is fixed. Keep everything that was not criticised. Do not shorten it and do not add new content.
Sees: The task, the one result to improve and its list of defects. Not the earlier attempts.
Scorer
Judge one result against the rubric. First list concrete defects, such as a missing item, a duplicate, an invented fact or a broken constraint. Then give a score from 1 to 10. Do not rewrite the result.
Sees: The goal of the part or of the whole task, the rubric and one result. Not who wrote it, not the other results.
A prompt you can paste
Works with any assistant that can launch sub-agents. It contains the agents, the procedure and the stop rule above.
You are the orchestrator of a multi-agent workflow that follows the "Graph of Thoughts" architecture. Launch every agent listed below as a separate sub-agent with its own context. Give each agent only the context stated for it. Do not do an agent's work yourself. If you cannot launch sub-agents, play each agent in a clearly separated turn and never let it use what is withheld from it. # Task [Describe your task here] # Graph of Thoughts Split, score and keep the best, then merge branches into one. Input: the task above. It must provide: A problem that splits into parts whose results can be merged, with its constraints. The plan of operations and the scoring rubric are already configured. If something essential is missing, ask for it before starting. Agents to launch: - Generator (3 per part, each in a fresh context; one more to split the task into parts): You receive a task and one part of it. Produce a complete result for that part only. Do not cover the other parts and do not refer to them. Work from the material given and add nothing that is not in it. Context for this agent: The task and its one part with the material for it. Not the other parts, not the other attempts, not any score. - Aggregator (3 merge attempts, each in a fresh context): You receive several partial results for one task. Write a single result that keeps everything they contain, states each item once and resolves contradictions explicitly. Do not add content that is in none of them. Context for this agent: The task and the kept partial results. Not the attempts that were dropped, not the scores. - Refiner: You receive one result and the list of defects a reviewer found in it. Rewrite the result so that each defect is fixed. Keep everything that was not criticised. Do not shorten it and do not add new content. Context for this agent: The task, the one result to improve and its list of defects. Not the earlier attempts. - Scorer: Judge one result against the rubric. First list concrete defects, such as a missing item, a duplicate, an invented fact or a broken constraint. Then give a score from 1 to 10. Do not rewrite the result. Context for this agent: The goal of the part or of the whole task, the rubric and one result. Not who wrote it, not the other results. Procedure: 1. Write the plan of operations before anything runs: how the task splits into 2 to 4 parts, 3 attempts per part, 3 merge attempts, at most 2 refinements. Do not change it during the run. 2. Write a rubric of three to five checks that apply to a partial result and to the merged one. 3. For each part, launch 3 generators in separate contexts. Launch the scorer on each attempt, one at a time. Keep the best attempt of each part. 4. Record every attempt in the thought graph with its score and the thought it came from, including the ones you drop. 5. Launch the aggregator 3 times on the kept attempts. Score the 3 merges and keep the best. Record it with links to all of its sources. 6. If the kept merge scores under 8, launch the refiner with the scorer's defects, score the new version and keep the higher of the two. Do this at most twice. 7. Return the kept thought when no operation is left. Stop rule: Stop when the plan is exhausted. With 4 parts that is 12 attempts, 3 merges and at most 2 refinements, each scored once. Refinement ends early when the kept thought scores 8 or more. Exit: - `answer`, when: The last operation of the plan has run, or the refined thought reached the target score. Deliver: The final thought as the answer, its score, and the list of thoughts with their scores and the thoughts each one was made from. # Final deliverable The final result and its score, then the thought graph as a list. For each thought, the operation that made it, its score and the thoughts it was made from.
The same thing in code
import asyncio
from typing import TypedDict
from langchain.chat_models import init_chat_model
from langgraph.graph import END, START, StateGraph
from pydantic import BaseModel
# the plan of operations, fixed before the run: split, attempt each part, merge, refine
MAX_PARTS, ATTEMPTS, MERGES, MAX_REFINES, GOOD_ENOUGH = 4, 3, 3, 2, 8
RUBRIC = "1. Covers everything in the material. 2. States each item once. 3. Adds nothing that is not in the material."
GENERATOR = "Produce a complete result for your part only. Work from the material given and add nothing to it."
AGGREGATOR = ("Write a single result from the partial results. Keep everything they contain, state each item "
"once and resolve contradictions explicitly. Add nothing that is in none of them.")
REFINER = "Rewrite the result so that each listed defect is fixed. Keep everything else. Add no new content."
SCORER = "Judge the result against the rubric. List concrete defects first, then score 1 to 10. Do not rewrite it."
llm = init_chat_model("anthropic:claude-sonnet-5-5")
class Parts(BaseModel):
parts: list[str] # independent parts whose results can be merged
class Score(BaseModel):
defects: list[str]
score: int # 1 to 10
class Thought(BaseModel): # a vertex; parents are its incoming links
id: int
op: str
text: str
parents: list[int]
score: int = 0
defects: list[str] = []
class State(TypedDict):
task: str
graph: list[Thought] # every thought of the run, kept or dropped
kept: list[int] # ids the next operation works on
refines: int
async def scored(next_id: int, goal: str, texts: list[str], parents: list[int], op: str) -> list[Thought]:
# a fresh scorer call per thought: the goal, the rubric and the text, nothing else
judge = llm.with_structured_output(Score)
raw = await asyncio.gather(*(judge.ainvoke(f"{SCORER}\nGoal: {goal}\nRubric: {RUBRIC}\n\nResult:\n{t}") for t in texts))
new = []
for text, r in zip(texts, raw):
s = Score.model_validate(r)
new.append(Thought(id=next_id + len(new), op=op, text=text, parents=parents, score=s.score, defects=s.defects))
return new
async def write(instructions: str, prompt: str) -> str:
return (await llm.ainvoke(f"{instructions}\n\n{prompt}")).text
async def generate(state: State) -> dict:
task = state["task"]
split = Parts.model_validate(await llm.with_structured_output(Parts).ainvoke(
f"Split this task into 2 to {MAX_PARTS} independent parts whose results can be merged.\n\n{task}"))
graph, kept = list(state["graph"]), []
for part_text in split.parts[:MAX_PARTS]:
part = Thought(id=len(graph), op="part", text=part_text, parents=[])
graph.append(part)
# several attempts per part, each in a fresh context; the best one survives
texts = await asyncio.gather(*(write(GENERATOR, f"Task: {task}\nYour part: {part_text}") for _ in range(ATTEMPTS)))
attempts = await scored(len(graph), part_text, list(texts), [part.id], "generate")
graph += attempts
kept.append(max(attempts, key=lambda t: t.score).id)
return {"graph": graph, "kept": kept}
async def aggregate(state: State) -> dict:
# several thoughts in, one out: the new vertex has one parent per kept attempt
material = "\n\n".join(f"Partial result {i}:\n{state['graph'][k].text}" for i, k in enumerate(state["kept"], 1))
texts = await asyncio.gather(*(write(AGGREGATOR, f"Task: {state['task']}\n\n{material}") for _ in range(MERGES)))
merges = await scored(len(state["graph"]), state["task"], list(texts), state["kept"], "aggregate")
return {"graph": state["graph"] + merges, "kept": [max(merges, key=lambda t: t.score).id]}
async def refine(state: State) -> dict:
best = state["graph"][state["kept"][0]]
text = await write(REFINER, f"Task: {state['task']}\nDefects:\n" + "\n".join(best.defects) + f"\n\nResult:\n{best.text}")
(new,) = await scored(len(state["graph"]), state["task"], [text], [best.id], "refine")
keep = new if new.score > best.score else best # refining can make it worse
return {"graph": state["graph"] + [new], "kept": [keep.id], "refines": state["refines"] + 1}
def more(state: State) -> str:
best = state["graph"][state["kept"][0]]
return END if best.score >= GOOD_ENOUGH or state["refines"] >= MAX_REFINES else "refine"
builder = StateGraph(State)
builder.add_node("generate", generate)
builder.add_node("aggregate", aggregate)
builder.add_node("refine", refine)
builder.add_edge(START, "generate")
builder.add_edge("generate", "aggregate")
builder.add_conditional_edges("aggregate", more, ["refine", END])
builder.add_conditional_edges("refine", more, ["refine", END])
app = builder.compile()
async def run(task: str) -> dict:
s = await app.ainvoke({"task": task, "graph": [], "kept": [], "refines": 0})
best = s["graph"][s["kept"][0]]
return {"answer": best.text, "score": best.score, "thoughts": [t.model_dump(exclude={"text"}) for t in s["graph"]]}
The wiring of the graph is the plan of operations: change it for your task, before the run. Write your own rubric, or score in code when the result can be checked.
import asyncio
from claude_agent_sdk import ClaudeAgentOptions, ResultMessage, query
from pydantic import BaseModel
# the plan of operations, fixed before the run: split, attempt each part, merge, refine
MAX_PARTS, ATTEMPTS, MERGES, MAX_REFINES, GOOD_ENOUGH = 4, 3, 3, 2, 8
RUBRIC = "1. Covers everything in the material. 2. States each item once. 3. Adds nothing that is not in the material."
SPLITTER = f"Split the task into 2 to {MAX_PARTS} independent parts whose results can be merged."
GENERATOR = "Produce a complete result for your part only. Work from the material given and add nothing to it."
AGGREGATOR = ("Write a single result from the partial results. Keep everything they contain, state each item "
"once and resolve contradictions explicitly. Add nothing that is in none of them.")
REFINER = "Rewrite the result so that each listed defect is fixed. Keep everything else. Add no new content."
SCORER = "Judge the result against the rubric. List concrete defects first, then score 1 to 10. Do not rewrite it."
class Parts(BaseModel):
parts: list[str]
class Score(BaseModel):
defects: list[str]
score: int # 1 to 10
class Thought(BaseModel): # a vertex; parents are its incoming links
id: int
op: str
text: str
parents: list[int]
score: int = 0
defects: list[str] = []
async def ask(system: str, prompt: str, schema: type[BaseModel] | None = None) -> ResultMessage:
# one query per call: a new session, so no agent sees another attempt or a score
options = ClaudeAgentOptions(
system_prompt=system,
output_format={"type": "json_schema", "schema": schema.model_json_schema()} if schema else None)
async for msg in query(prompt=prompt, options=options):
if isinstance(msg, ResultMessage):
return msg
raise RuntimeError("no result")
async def write(system: str, prompt: str) -> str:
return (await ask(system, prompt)).result or ""
async def run(task: str) -> dict:
graph: list[Thought] = [] # every thought of the run, kept or dropped
async def scored(goal: str, texts: list[str], parents: list[int], op: str) -> list[Thought]:
# a fresh scorer per thought: the goal, the rubric and the text, nothing else
raw = await asyncio.gather(*(ask(SCORER, f"Goal: {goal}\nRubric: {RUBRIC}\n\nResult:\n{t}", Score) for t in texts))
new = []
for text, msg in zip(texts, raw):
s = Score.model_validate(msg.structured_output)
new.append(Thought(id=len(graph) + len(new), op=op, text=text, parents=parents, score=s.score, defects=s.defects))
graph.extend(new)
return new
# operation 1, generate: the parts, then several attempts per part; the best one survives
split = Parts.model_validate((await ask(SPLITTER, task, Parts)).structured_output)
kept: list[Thought] = []
for part_text in split.parts[:MAX_PARTS]:
part = Thought(id=len(graph), op="part", text=part_text, parents=[])
graph.append(part)
texts = await asyncio.gather(*(write(GENERATOR, f"Task: {task}\nYour part: {part_text}") for _ in range(ATTEMPTS)))
kept.append(max(await scored(part_text, list(texts), [part.id], "generate"), key=lambda t: t.score))
# operation 2, aggregate: several thoughts in, one out, with one parent per kept attempt
material = "\n\n".join(f"Partial result {i}:\n{t.text}" for i, t in enumerate(kept, 1))
texts = await asyncio.gather(*(write(AGGREGATOR, f"Task: {task}\n\n{material}") for _ in range(MERGES)))
best = max(await scored(task, list(texts), [t.id for t in kept], "aggregate"), key=lambda t: t.score)
# operation 3, refine: a loop on one thought, at most twice
for _ in range(MAX_REFINES):
if best.score >= GOOD_ENOUGH:
break
text = await write(REFINER, f"Task: {task}\nDefects:\n" + "\n".join(best.defects) + f"\n\nResult:\n{best.text}")
(new,) = await scored(task, [text], [best.id], "refine")
best = new if new.score > best.score else best # refining can make it worse
# the plan is exhausted: the kept thought is the result
return {"answer": best.text, "score": best.score, "thoughts": [t.model_dump(exclude={"text"}) for t in graph]}
The body of run is the plan of operations: change it for your task, before the run. Write your own rubric, or score in code when the result can be checked.
import asyncio
from agents import Agent, Runner
from pydantic import BaseModel
# the plan of operations, fixed before the run: split, attempt each part, merge, refine
MAX_PARTS, ATTEMPTS, MERGES, MAX_REFINES, GOOD_ENOUGH = 4, 3, 3, 2, 8
RUBRIC = "1. Covers everything in the material. 2. States each item once. 3. Adds nothing that is not in the material."
class Parts(BaseModel):
parts: list[str]
class Score(BaseModel):
defects: list[str]
score: int # 1 to 10
class Thought(BaseModel): # a vertex; parents are its incoming links
id: int
op: str
text: str
parents: list[int]
score: int = 0
defects: list[str] = []
splitter = Agent(
name="Splitter",
instructions=f"Split the task into 2 to {MAX_PARTS} independent parts whose results can be merged.",
output_type=Parts,
)
generator = Agent(
name="Generator",
instructions="Produce a complete result for your part only. Work from the material given and add nothing to it.",
)
aggregator = Agent(
name="Aggregator",
instructions="Write a single result from the partial results. Keep everything they contain, state each item "
"once and resolve contradictions explicitly. Add nothing that is in none of them.",
)
refiner = Agent(
name="Refiner",
instructions="Rewrite the result so that each listed defect is fixed. Keep everything else. Add no new content.",
)
scorer = Agent(
name="Scorer",
instructions="Judge the result against the rubric. List concrete defects first, then score 1 to 10. Do not rewrite it.",
output_type=Score,
)
async def write(agent: Agent, prompt: str) -> str:
return str((await Runner.run(agent, prompt)).final_output) # a fresh run: this prompt only
async def run(task: str) -> dict:
graph: list[Thought] = [] # every thought of the run, kept or dropped
async def scored(goal: str, texts: list[str], parents: list[int], op: str) -> list[Thought]:
# a fresh scorer run per thought: the goal, the rubric and the text, nothing else
runs = await asyncio.gather(*(
Runner.run(scorer, f"Goal: {goal}\nRubric: {RUBRIC}\n\nResult:\n{t}") for t in texts))
new = []
for text, r in zip(texts, runs):
s = r.final_output_as(Score)
new.append(Thought(id=len(graph) + len(new), op=op, text=text, parents=parents, score=s.score, defects=s.defects))
graph.extend(new)
return new
# operation 1, generate: the parts, then several attempts per part; the best one survives
split = (await Runner.run(splitter, task)).final_output_as(Parts)
kept: list[Thought] = []
for part_text in split.parts[:MAX_PARTS]:
part = Thought(id=len(graph), op="part", text=part_text, parents=[])
graph.append(part)
texts = await asyncio.gather(*(write(generator, f"Task: {task}\nYour part: {part_text}") for _ in range(ATTEMPTS)))
kept.append(max(await scored(part_text, list(texts), [part.id], "generate"), key=lambda t: t.score))
# operation 2, aggregate: several thoughts in, one out, with one parent per kept attempt
material = "\n\n".join(f"Partial result {i}:\n{t.text}" for i, t in enumerate(kept, 1))
texts = await asyncio.gather(*(write(aggregator, f"Task: {task}\n\n{material}") for _ in range(MERGES)))
best = max(await scored(task, list(texts), [t.id for t in kept], "aggregate"), key=lambda t: t.score)
# operation 3, refine: a loop on one thought, at most twice
for _ in range(MAX_REFINES):
if best.score >= GOOD_ENOUGH:
break
text = await write(refiner, f"Task: {task}\nDefects:\n" + "\n".join(best.defects) + f"\n\nResult:\n{best.text}")
(new,) = await scored(task, [text], [best.id], "refine")
best = new if new.score > best.score else best # refining can make it worse
# the plan is exhausted: the kept thought is the result
return {"answer": best.text, "score": best.score, "thoughts": [t.model_dump(exclude={"text"}) for t in graph]}
The body of run is the plan of operations: change it for your task, before the run. Write your own rubric, or score in code when the result can be checked.
import { anthropic } from '@ai-sdk/anthropic';
import { generateText, Output } from 'ai';
import { z } from 'zod';
const model = anthropic('claude-sonnet-5-5');
// the plan of operations, fixed before the run: split, attempt each part, merge, refine
const [MAX_PARTS, ATTEMPTS, MERGES, MAX_REFINES, GOOD_ENOUGH] = [4, 3, 3, 2, 8];
const RUBRIC = '1. Covers everything in the material. 2. States each item once. 3. Adds nothing that is not in the material.';
const GENERATOR = 'Produce a complete result for your part only. Work from the material given and add nothing to it.';
const AGGREGATOR =
'Write a single result from the partial results. Keep everything they contain, state each item ' +
'once and resolve contradictions explicitly. Add nothing that is in none of them.';
const REFINER = 'Rewrite the result so that each listed defect is fixed. Keep everything else. Add no new content.';
const SCORER = 'Judge the result against the rubric. List concrete defects first, then score 1 to 10. Do not rewrite it.';
const partsSchema = z.object({ parts: z.array(z.string()) });
const scoreSchema = z.object({ defects: z.array(z.string()), score: z.number().int().min(1).max(10) });
// a vertex; parents are its incoming links
type Thought = { id: number; op: string; text: string; parents: number[]; score: number; defects: string[] };
const best = (thoughts: Thought[]) => thoughts.reduce((a, b) => (b.score > a.score ? b : a));
// a fresh call each time: these instructions and this prompt only
async function write(instructions: string, prompt: string): Promise<string> {
const { text } = await generateText({ model, instructions, prompt });
return text;
}
export async function run(task: string) {
const graph: Thought[] = []; // every thought of the run, kept or dropped
async function scored(goal: string, texts: string[], parents: number[], op: string): Promise<Thought[]> {
// a fresh scorer call per thought: the goal, the rubric and the text, nothing else
const scores = await Promise.all(texts.map(async (text) => {
const { output } = await generateText({
model,
instructions: SCORER,
prompt: `Goal: ${goal}\nRubric: ${RUBRIC}\n\nResult:\n${text}`,
output: Output.object({ schema: scoreSchema }),
});
return output;
}));
const fresh = texts.map((text, i) => ({ id: graph.length + i, op, text, parents, ...scores[i] }));
graph.push(...fresh);
return fresh;
}
// operation 1, generate: the parts, then several attempts per part; the best one survives
const { output: split } = await generateText({
model,
instructions: `Split the task into 2 to ${MAX_PARTS} independent parts whose results can be merged.`,
prompt: task,
output: Output.object({ schema: partsSchema }),
});
const kept: Thought[] = [];
for (const partText of split.parts.slice(0, MAX_PARTS)) {
const part: Thought = { id: graph.length, op: 'part', text: partText, parents: [], score: 0, defects: [] };
graph.push(part);
const texts = await Promise.all(
Array.from({ length: ATTEMPTS }, () => write(GENERATOR, `Task: ${task}\nYour part: ${partText}`)));
kept.push(best(await scored(partText, texts, [part.id], 'generate')));
}
// operation 2, aggregate: several thoughts in, one out, with one parent per kept attempt
const material = kept.map((t, i) => `Partial result ${i + 1}:\n${t.text}`).join('\n\n');
const merges = await Promise.all(
Array.from({ length: MERGES }, () => write(AGGREGATOR, `Task: ${task}\n\n${material}`)));
let result = best(await scored(task, merges, kept.map((t) => t.id), 'aggregate'));
// operation 3, refine: a loop on one thought, at most twice
for (let i = 0; i < MAX_REFINES && result.score < GOOD_ENOUGH; i++) {
const text = await write(REFINER, `Task: ${task}\nDefects:\n${result.defects.join('\n')}\n\nResult:\n${result.text}`);
const [next] = await scored(task, [text], [result.id], 'refine');
if (next.score > result.score) result = next; // refining can make it worse
}
// the plan is exhausted: the kept thought is the result
return {
answer: result.text,
score: result.score,
thoughts: graph.map(({ id, op, parents, score }) => ({ id, op, parents, score })),
};
}
The body of run is the plan of operations: change it for your task, before the run. Write your own rubric, or score in code when the result can be checked.
Snippets are generated when the site is built and type-checked against the SDK versions listed in the repository. Treat them as a starting skeleton.
What goes wrong
- An aggregator given too many thoughts drops content silently. Merge a few at a time, and have the rubric check that nothing was lost.
- Parts that overlap produce duplicates that the merge must then undo. Split so that each item belongs to one part.
- A scorer that rates every attempt 7 or 8 makes keeping the best a random pick. Ask for defects before the score.
- Refining can make a result worse. Score the new version and keep the old one when it rated higher.
- The plan fixes the number of calls. Count them before the run. Do not let the model add operations.
Chain it
Its result can feed
- Chain of Verification conditional
Condition: Use this link only when the final answer states factual claims about the world, such as dates, figures or named facts, rather than being a computed or merged result like a sorted list or a count.
- Debate direct
Treat the final thought as an existing answer to challenge, with the original task as the question and the high-scoring alternative thoughts as positions debaters may argue for.
- Constitutional Critique direct
Treat the final thought as the draft to check rule by rule against the constitution, and fix only the rules it fails.
- Textual Backpropagation direct
Give the final thought to the configured pipeline as an existing result to improve, using backward feedback to optimize that pipeline's prompts.