New Method Lets AI Reason in Hidden Vector Space Rather Than Words

Researchers have proposed a novel approach to improve how large language models tackle difficult reasoning tasks by bypassing the need to work within language itself. Rather than generating step-by-step explanations that humans can read, the method lets models reason using the underlying numerical representations—embeddings—that sit at the heart of how neural networks operate.

The Coconut Framework#

The technique, called Coconut (Chain of Continuous Thought), shifts reasoning away from the language space that conventional models rely on. Instead of outputting word tokens to explain each step, the approach directly manipulates the internal hidden states that the model produces. These hidden representations, which the researchers term "continuous thoughts," are fed back into the model as inputs without being converted into readable text. This allows the model to maintain and refine abstract reasoning patterns within the embedding space itself.

How It Works#

Traditional chain-of-thought prompting forces models to commit to a linear path: one word, then another, then another. Because that sequence must make grammatical sense, many tokens serve purely to maintain readability rather than contribute to actual problem-solving. Coconut sidesteps this constraint by allowing continuous thought vectors to encode multiple potential next steps simultaneously. This enables the model to perform a breadth-first search across different solution branches—exploring multiple promising directions in parallel rather than locking into a single predetermined route early on.

Advantages Over Language-Based Reasoning#

On logical reasoning tasks that demand significant computational search and planning, Coconut outperforms conventional chain-of-thought approaches. The method achieves better balance between answer accuracy and computational efficiency, suggesting that certain classes of problems become easier to solve once you remove the requirement that intermediate reasoning be expressible as coherent text. The findings suggest that language itself, while essential for communication, can sometimes constrain the problem-solving mechanisms inside LLMs.