Agentic AI · Module 8 of 41 · 11 concepts · about 16 min

Reasoning Patterns

The named techniques agents use to think: chain-of-thought, ReAct, plan-and-execute, reflection and self-critique, iterative refinement, Tree of Thoughts and Graph of Thoughts; how each works, what it costs, and when it is worth it.

At work: The menu an engineer chooses from when a single model call is not accurate enough, and the vocabulary interviewers expect you to compare, not just recite.

Chain-of-thought

What it is

Chain-of-thought (CoT) means having the model work through a problem in steps before giving its answer, instead of answering at once.

The model writes text one piece at a time, and each piece is influenced by what it has already written.
Working written out first becomes material the final answer is built on.
Many current models do this internally, as extended thinking or reasoning, before they reply. You set how much with a parameter, not a prompt.

Why it matters

It improves accuracy on anything with several steps: arithmetic, logic, comparing options, choosing a tool.

It is the base every other pattern here is built on.
For an agent, a brief stated reason before each action also makes its behaviour explainable in a trace.
The cost is extra output tokens and time on every call.

In the real world: A pricing assistant that jumped straight to a discount figure was wrong one time in eight; asked to list the applicable rules first, it was wrong one time in forty.

Visual example

Chain-of-thought

The question: is order 88213, promised at 6:00 pm and delivered at 7:02 pm, eligible for a fee refund if the rule is 'more than 30 minutes late'?

Answering directly versus reasoning first

	direct answer	reasoning first
What the model writes	“Yes.”	“Promised 6:00, arrived 7:02: 62 minutes late. 62 > 30. Eligible.”
Checkable by a person	No	Yes
Output tokens	Few	More
Accuracy on multi-step questions	Lower	Higher

Treating the written reasoning as a faithful account of why the model decided. It is useful evidence, not a guarantee. Also, with a model that already reasons internally, adding 'think step by step' to the prompt adds cost and little else.

ReAct in brief

What it is

ReAct (reason and act) interleaves thinking with tool use, one step at a time:

Thought: what do I know, and what do I need?
Action: one tool request.
Observation: the tool's result.

Then another thought, and so on until the answer. It is chain-of-thought with real information arriving between the steps.

Why it matters

It is the default pattern for a tool-using agent and the one behind most agent loops.

Reasoning is grounded in real results, not in what the model imagines the results would be.
It adapts at every step.
Its weaknesses are cost (the history is resent each turn) and a tendency to lose the wider goal. Module 9 covers it in full.

In the real world: Nearly every 'tool-calling agent' in production is a ReAct loop under another name; the differences between products are in the tools, limits and context management around it.

Using ReAct for a task whose steps are fixed. If the same three tools are always called in the same order, a workflow does it in fewer calls with no chance of wandering.

Plan-and-execute as a reasoning pattern

What it is

Plan-and-execute reasons in two phases:

Plan: one call writes all the steps.
Execute: the steps are carried out, by cheaper models or plain code, with the planner called again only if replanning is needed.

Module 7 covered its mechanics; here it is set beside ReAct.

Why it matters

It moves the expensive thinking to the start.

Fewer calls to the large model, so lower cost on long tasks.
Independent steps can run in parallel.
The plan can be validated and approved before anything happens.
It is weaker when early results should change later steps, which is exactly ReAct's strength.

In the real world: A report-writing agent plans six sections once, then runs a small ReAct loop per section in parallel; the report arrives in a third of the time of a single long loop.

Visual example

Comparison

Nest them: a plan on the outside, ReAct inside each step.

ReAct versus plan-and-execute

	ReAct	plan-and-execute
Decides	One step at a time	All steps first
Calls to the large model	Every step	Once, plus replans
Adapts to surprises	Immediately	By replanning
Parallel steps	Limited	Natural
Reviewable before acting	No	Yes
Fails by	Looping, drifting	Following a stale plan

Presenting the two as rivals. The common production design nests them: a plan at the top, and a short ReAct loop inside each plan step.

Reflection, self-reflection and self-critique

What it is

Three closely related ideas, all about checking work before it is accepted:

Reflection: after producing something, examine it and improve it. The general term.
Self-critique: the same model is prompted to find faults in its own draft, usually against a checklist.
Self-reflection: after a failed attempt, the agent writes down what went wrong and carries that lesson into the next attempt. The lesson can be saved as memory.

The strongest form uses a separate critic: a different prompt, or a different model, that did not write the draft.

Why it matters

A first draft is rarely the best a model can do, and a model is better at spotting faults in text put in front of it than at avoiding them while writing.

Reflection measurably improves code, long writing and plans.
It is strongest when there is real feedback to reflect on: a failing test, a validation error, a rubric.
It costs at least one extra call per round.

In the real world: A coding agent's reflection step is simply 'run the tests and read the failures'; that grounded feedback beats any amount of the model re-reading its own code.

Visual example

Reflection

The critic has a checklist and the evidence. Rounds are capped.

Draft, critique, revise

DraftFirst reply from tool results
CritiqueReviewer lists rule violations
ReviseFix exactly what was listed
StopPASS, or two rounds used

CRITIC_PROMPT = """You are reviewing a draft reply to a customer. You did not write it.
Check it against each rule and list every violation. If there are none, say PASS.

Rules
1. Every factual claim is supported by the tool results below.
2. The refund amount matches the tool result exactly.
3. It says what was done, not what will be done.
4. No internal ids other than the order number.

Tool results: {evidence}
Draft: {draft}"""

def write_with_reflection(task, evidence, max_rounds: int = 2) -> str:
    draft = llm(WRITER_PROMPT.format(task=task, evidence=evidence))
    for _ in range(max_rounds):
        review = llm(CRITIC_PROMPT.format(evidence=evidence, draft=draft))
        if review.strip() == "PASS":
            break
        draft = llm(REVISE_PROMPT.format(draft=draft, review=review, evidence=evidence))
    return draft

Asking 'are you sure?' with no criteria and no evidence. The model either agrees with itself or changes a correct answer to please you. Reflection needs a checklist and something real to check against.

Tree of Thoughts

What it is

Tree of Thoughts (ToT) explores several lines of reasoning instead of one:

at each step the model proposes several possible next thoughts;
each is scored for how promising it is, by the model or by code;
the best are expanded further and the weak ones dropped;
the search can back up from a dead end and try another branch.

The result is a tree of partial solutions, searched like a game tree.

Why it matters

A single chain commits to its first idea and cannot recover if that idea is wrong.

ToT helps on problems where early choices matter and dead ends are common: puzzles, constrained planning, some design tasks.
It costs many times more: a tree three wide and three deep can need dozens of calls.
For most business tasks it is too expensive and too slow to use live.

In the real world: A scheduling tool uses a small tree search offline to build weekly rotas under many constraints, where one chain kept painting itself into a corner; nobody waits on it live.

Visual example

Tree of Thoughts

Breadth and depth multiply. Three options over three levels is up to 39 calls.

Searching several lines of reasoning

ProposeThree possible next steps
EvaluatePromising, uncertain, dead end
PruneKeep the best one or two
Expand or back upDeeper, or an earlier branch

Reaching for ToT because it sounds thorough. Try first: a better prompt, extended thinking, or generating three independent answers and picking the best. Those get most of the gain for a fraction of the cost.

Graph of Thoughts

What it is

Graph of Thoughts (GoT) generalises the tree. Thoughts are nodes in a graph, and besides branching they can be:

merged: several partial results combined into one;
refined: a thought improved in a loop;
reused: one result feeding several later thoughts.

A tree only splits. A graph can also join.

Why it matters

Some problems are naturally 'split, solve the parts, combine'.

Summarising twenty documents: summarise each, merge in pairs, refine the merged result.
Merging lets good parts of different attempts be kept together, which a tree cannot do.
In practice GoT is less a product feature than a way of thinking: it is what a graph-shaped workflow (module 14) already does.

In the real world: A contract-review pipeline extracts clauses from each document separately, merges duplicates across documents, and refines the merged list: a graph of thoughts, built as an ordinary workflow.

Visual example

Structures

Each adds one move the previous shape cannot make.

Chain, tree and graph

shape	moves available	calls	good for
Chain (CoT)	Continue	1	Most reasoning
Tree (ToT)	Continue, branch, back up	Tens	Search with dead ends
Graph (GoT)	Continue, branch, merge, refine	Tens to hundreds	Split, solve, combine

Citing Graph of Thoughts as something you would deploy as-is. Interviewers want to hear that you know it as a research idea and can name its practical form: a map-reduce or graph workflow.

Decomposition, hierarchical planning and replanning

What it is

Three patterns from module 7, named here so the catalogue is complete:

Decomposition: split the goal into subtasks.
Hierarchical planning: plan stages, then plan inside each stage when it is reached.
Dynamic replanning: revise the plan when a result invalidates it.

Why it matters

They are reasoning patterns in the same sense as the others: ways of arranging model calls so a hard problem becomes several easier ones.

Decomposition raises the success rate of each piece.
Hierarchy keeps each plan short enough to hold in view.
Replanning stops a wrong early guess from spoiling the whole task.

In the real world: An interviewer asks a candidate to 'name the reasoning patterns'; the strong answer groups them: think before answering, think between actions, plan before acting, check after acting, and search over alternatives.

Visual example

Catalogue

Cost is relative to a single direct call.

Every reasoning pattern, with its cost and its use

pattern	idea in one line	extra cost	use when	avoid when
Chain-of-thought	Reason, then answer	Low	Any multi-step question	Simple look-ups
ReAct	Reason between tool calls	Medium	The path depends on results	Steps are fixed
Plan-and-execute	Plan all, then do	Medium	Long, predictable tasks	Highly exploratory tasks
Reflection	Critique and revise	Medium	There is a checklist or test	No way to check
Iterative refinement	Loop until it passes	Medium to high	Output is testable	No pass condition
Tree of Thoughts	Search branches	Very high	Dead ends are common	Live, cost-sensitive use
Graph of Thoughts	Branch and merge	Very high	Split, solve, combine	Simple linear tasks
Hierarchical planning	Plan in levels	Medium	Hundreds of steps	Short tasks

Practise it: Design a Multi-Agent AI System

Choosing a pattern in production

What it is

A practical order for choosing:

Start with one call, with the model's built-in reasoning switched on.
If it needs live information or actions, use a ReAct loop with firm limits.
If the task is long and its shape is known, add a plan and a to-do list.
If the output has a test or a rubric, add one round of reflection.
Only if all that falls short, and the task is offline, consider search (trees or graphs).

Why it matters

Every pattern buys accuracy with money and time, and the returns shrink.

Measure each addition on a fixed set of test cases. Keep it only if the success rate rises enough to pay for it.
More reasoning also means more places to fail and longer traces to debug.
The simplest pattern that meets the target is the right one.

In the real world: A team adds reflection to its support agent and measures: success rises from 91 to 92 percent while cost per task rises 60 percent. They remove it and spend the effort on better tools.

Stacking patterns because each sounds like an improvement. A plan-and-execute agent with ReAct steps, reflection on every step and a critic on the plan can cost fifteen times a single call for a few points of accuracy.

Interview questions: reasoning patterns

What it is

The questions interviewers ask about reasoning patterns, from the first screening call to the senior round. Cover the answer, say yours aloud, then compare.

Why it matters

Reading an idea and explaining it are different skills.

Each answer below is the two or three sentences an interviewer is listening for.
The reasoning behind each one is in the cards above. If an answer surprises you, go back to that card.

In the real world: A candidate lists seven pattern names; the interviewer asks 'which would you remove first to halve the cost?', and the answer to that question is what gets scored.

Q: What is chain-of-thought?

Having the model reason step by step before answering. Each written step conditions the next, which improves multi-step accuracy. Modern models often do it internally as extended thinking.

Q: ReAct versus plan-and-execute?

ReAct decides one step at a time and adapts instantly but costs a large-model call per step and can drift. Plan-and-execute plans once, allows parallel and cheaper execution and review, but needs replanning when results surprise it. They are often nested.

Q: What is reflection, and when does it help?

A critique-and-revise pass on a draft. It helps when there is something concrete to check against: tests, validation errors, a rubric. Without that it mostly rewords.

Q: Self-critique versus a separate critic?

Self-critique reuses the writer and tends to approve its own work. A separate critic with its own prompt, or another model, finds more. Both need a cap on rounds.

Q: What is Tree of Thoughts?

A search over several lines of reasoning: propose options, score them, expand the best, back up from dead ends. Powerful for problems with dead ends, but many times the cost, so mostly offline.

Q: How does Graph of Thoughts differ?

It adds merging and refining: results from different branches can be combined. In practice it appears as map-reduce and graph workflows rather than as a named feature.

Q: How do you choose a pattern?

Start with the simplest, measure on a fixed test set, and add a pattern only when the gain in success rate justifies its cost and latency.

Q: What are the production concerns?

Cost and latency multiply with every extra call; loops need stop conditions; traces get longer; and reasoning text is evidence, not proof of the model's real process.

Describing Tree of Thoughts as your production design for a chatbot. It signals reading without building.

Exercise: measure what reflection buys

What it is

A short build that uses this module's ideas. Write it yourself before looking anything up; the checks at the end tell you whether it works.

Why it matters

An idea you have only read fades in a week.

Building it once, even in fifty lines, is what makes it yours.
Interviewers can tell the difference between someone who has read about a loop and someone who has debugged one.

In the real world: An engineering lead asks every new hire to run this comparison in their first week; it replaces opinions about patterns with a table.

Your task. Compare three patterns on the same small task set and report accuracy and cost for each.

Steps

Write ten word problems with a single correct numeric answer, each needing two or three steps (for example, a discount, then tax, then a delivery fee).
Solve each with (a) a direct answer with reasoning off, (b) reasoning on, (c) reasoning on plus one critique-and-revise round.
Record for every run whether it was correct and how many tokens it used.
Put the results in a table: accuracy and average tokens per pattern.

You are done when

You have a three-row table with real numbers.
You can state, with evidence, whether the reflection round was worth its cost on this task.
You found at least one case where reflection changed a correct answer to a wrong one, or can say that it never did.

Stretch. Add a fourth pattern: generate three independent answers and take the majority. Compare it with reflection for the same cost.

Reasoning Patterns

Chain-of-thought

ReAct in brief

Plan-and-execute as a reasoning pattern

Reflection, self-reflection and self-critique

Iterative reasoning and refinement

Tree of Thoughts

Graph of Thoughts

Decomposition, hierarchical planning and replanning

Choosing a pattern in production

Interview questions: reasoning patterns

Exercise: measure what reflection buys