Tree of Thoughts

A reasoning strategy where the model branches out to explore several possible next steps at each decision point, evaluates which paths look most promising, and backtracks from dead ends. Like solving a maze by mapping multiple routes simultaneously.

Standard chain-of-thought follows one reasoning path from start to finish. Tree of Thoughts (ToT) lets the model consider several possible directions at each step, score them, and pursue the most promising ones while abandoning dead ends. Consider how you might solve a crossword puzzle: you try a word, see if the crossing letters work, and if they do not, you backtrack and try a different word. This branching and pruning approach is especially powerful for problems where early choices constrain later options, like planning, scheduling, and constraint satisfaction.

Builder example

ToT is computationally expensive because it multiplies generation and evaluation calls at each step. For most production tasks like summarization, extraction, classification, and drafting, simpler approaches like chain-of-thought or self-consistency deliver comparable results at a fraction of the cost. Reserve ToT for problems where the model genuinely needs to look ahead and backtrack, like multi-constraint planning or combinatorial puzzles.

Common confusion: ToT is a research technique for specific problem structures. Applying it to straightforward generation tasks adds cost and latency with no meaningful quality gain.