Tree of Thoughts

A reasoning strategy where the model branches out to explore several possible next steps at each decision point, evaluates which paths look most promising, and backtracks from dead ends. Like solving a maze by mapping multiple routes simultaneously.

Tree of Thoughts (ToT) is a reasoning strategy where the model branches out to explore several possible next steps at each decision point, evaluates which paths look most promising, and backtracks from dead ends. This branching and pruning approach is especially powerful for problems where early choices constrain later options, like planning, scheduling, and constraint satisfaction. Consider how you might solve a crossword puzzle: you try a word, see if the crossing letters work, and if they do not, you backtrack and try a different word.

Builder example

ToT is computationally expensive because it multiplies generation and evaluation calls at each step. For most production tasks like summarization, extraction, classification, and drafting, simpler approaches like chain-of-thought or self-consistency deliver comparable results at a fraction of the cost. Reserve ToT for problems where the model genuinely needs to look ahead and backtrack, like multi-constraint planning or combinatorial puzzles.

Common confusion: ToT is a research technique for specific problem structures. Applying it to straightforward generation tasks adds cost and latency with no meaningful quality gain.