Purpose
This article explains major reasoning-time methods used with LLMs, including Chain-of-Thought prompting, self-consistency, STaR, self-refinement, and search-based approaches such as Tree-of-Thoughts.
Overview
For many tasks, the raw next-token policy of an LLM is not enough to reliably solve the problem in one pass. This is especially true for tasks requiring:
- arithmetic decomposition
- symbolic manipulation
- structured planning
- multi-hop QA
- theorem-style reasoning
- code synthesis and debugging
Reasoning-time methods improve performance not by changing model weights directly, but by changing how inference is conducted or how intermediate reasoning traces are elicited, sampled, evaluated, and reused.
A useful mental model is:
- base model capability defines what reasoning trajectories are available
- test-time method changes how the system explores that trajectory space
This means two systems built on the same base model can behave very differently depending on inference-time scaffolding.
Chain-of-Thought (CoT)
Chain-of-Thought prompting encourages the model to emit intermediate reasoning steps before committing to a final answer.
Instead of compressing the entire solution into a short response, the model is guided to generate a structured trajectory:
- interpret the problem
- derive intermediate facts
- combine them
- produce the final answer
Why CoT can help
From a modeling perspective, CoT changes the conditional generation path. The model no longer has to jump directly from problem statement to answer tokens. It can generate intermediate states that reduce the difficulty of later prediction.
In effect, the rationale tokens act as a latent scratchpad exposed in natural language.
Ways to induce CoT
- few-shot exemplars with explicit worked solutions
- direct prompting for intermediate reasoning
- SFT on rationale-rich datasets
- verifier-guided or search-guided test-time generation
Important limitation
CoT is not automatically truthful or correct. A model can generate plausible but invalid reasoning. The existence of intermediate text does not guarantee actual logical consistency.
Self-consistency
Self-consistency replaces single-path greedy reasoning with multi-sample reasoning followed by answer aggregation.
Procedure:
- sample multiple CoT trajectories for the same question
- extract the final answer from each trajectory
- choose the most frequent or highest-confidence answer
This works because the reasoning trace itself may be noisy, but the correct solution can still be a stable attractor across multiple independent samples.
Why it helps
A greedy decode commits early to one trajectory. If an early step is wrong, the entire completion may collapse. Self-consistency reduces that variance by marginalizing over multiple sampled paths.
Practical tradeoff
The gain comes at increased inference cost:
- more forward passes
- more output tokens
- answer normalization / aggregation logic required
This method is often effective on math and structured reasoning tasks, but less useful when the bottleneck is factual knowledge rather than reasoning decomposition.
STaR
STaR (Self-Taught Reasoner) is a reasoning-bootstrapping framework. Note that STaR changes the model weights so it is effectively a training-time framework.
The core loop is:
- prompt the model to generate rationales and answers
- keep examples where the final answer is correct
- for failures, regenerate rationales conditioned on the known correct answer
- fine-tune the model on successful rationales
- repeat
The important insight is that reasoning traces do not need to come entirely from human annotation. A model can generate candidate rationales, and training can preferentially retain those associated with correct outcomes.
Why this matters
High-quality reasoning data is expensive. STaR offers a mechanism for self-improving reasoning performance by converting answer supervision into rationale supervision through iterative generation and filtering.
Failure mode
A rationale that reaches the correct answer is not always causally valid. If filtering only checks final correctness, the training data may still contain spurious or post-hoc rationales.
Self-correction and self-refinement
Self-correction asks the model to critique or revise its own output rather than stopping after the first pass.
Typical loop:
- generate initial answer
- ask the model to identify flaws, missing steps, or inconsistencies
- generate a revised answer
- optionally repeat under a stopping rule
This is useful when first-pass outputs are often near-correct but suffer from:
- arithmetic slips
- formatting mistakes
- missing edge cases
- shallow reasoning depth
Important detail
Self-correction only works if the model’s internal error detector is stronger than its first-pass generator on the relevant error class. In many tasks this is partly true, but not universally.
Tree-of-Thoughts (ToT)
Tree-of-Thoughts generalizes linear CoT into an explicit search process.
Instead of generating one next reasoning step at a time along a single path, the system:
- expands multiple candidate next thoughts
- scores or evaluates them
- keeps promising branches
- prunes weak branches
- continues until a solution is found or the budget is exhausted
A “thought” is not necessarily one token or one sentence; it is a higher-level reasoning unit used as a search node.
Why ToT helps
Many hard problems are sensitive to early branching choices. A single linear decode can get stuck in a dead end. ToT allows exploration of multiple partial plans.
This is particularly useful for:
- constrained search problems
- planning tasks
- mathematical derivations
- compositional puzzles
Design choices in ToT systems
Performance depends heavily on:
- branching factor
- thought granularity
- evaluation heuristic
- breadth-first vs depth-first expansion
- pruning budget
- whether evaluation is done by the same model, another model, or an external verifier
ToT is therefore better understood as an inference framework than as a single algorithm.
Search-based reasoning more broadly
Many reasoning-time systems can be framed as policy + search + evaluator.
Examples:
- Best-of-N
- beam search over rationale candidates
- verifier-guided sampling
- MCTS-style rollout methods
- self-refine loops
- debate / multi-agent critique systems
This is important because it connects LLM reasoning to classical search and decision-making ideas. The model proposes candidate continuations, but system performance depends on whether those candidates are explored and filtered intelligently.
Exchange-of-Thought and multi-agent critique
In multi-agent or debate-style methods, several model instances generate, critique, or improve one another’s outputs.
This can introduce:
- diverse initial proposals
- error discovery through disagreement
- role specialization (solver, critic, verifier)
The gain does not come from “more intelligence” in the abstract, but from inducing diversity and selection pressure over candidate solutions.
Continuous / latent reasoning directions
Some recent research directions aim to improve reasoning without forcing all intermediate structure into natural-language text.
Instead of explicit textual CoT, the model may use:
- latent intermediate states
- internal iterative computation
- hidden-space planning modules
This matters because textual CoT is bandwidth-heavy and may be an inefficient interface for all forms of reasoning. Not every useful computation needs to be verbalized.
For platform purposes, though, most deployed reasoning-time methods still operate through text-visible trajectories because they are easy to supervise, debug, and verify.
Why these methods matter in practice
Reasoning quality is often not solely a property of the base checkpoint. It is a property of the inference protocol.
A system may improve substantially by changing:
- prompt scaffolding
- number of samples
- search depth
- verifier usage
- critique/revision loops
without changing the underlying model weights.
This is especially relevant for engineering teams evaluating models for:
- math
- code generation
- planning
- multi-step QA
- tool-using agents
A weak inference protocol can hide the true capability of the model. A strong one can recover much better performance, at the cost of more compute and latency.
Key takeaways
- CoT improves many tasks by decomposing generation into intermediate reasoning steps.
- Self-consistency improves robustness by aggregating over multiple sampled reasoning traces.
- STaR bootstraps reasoning supervision through generate-filter-fine-tune loops.
- ToT converts reasoning from a single trajectory into an explicit search process.
- Reasoning-time performance is often a joint function of model quality, sampling strategy, search budget, and evaluator quality.
Comments
0 comments
Article is closed for comments.