Geometric Latent Reasoning Induces Shorter Generations in LLMs
| Type of publication: | Conference paper |
| Citation: | Kumar_NEURIPS2026_2026 |
| Publication status: | Accepted |
| Booktitle: | The Fortieth Annual Conference on Neural Information Processing Systems |
| Year: | 2026 |
| URL: | https://arxiv.org/abs/2606.022... |
| Abstract: | Large language models solve complex problems by generating lengthy chains of explicit reasoning tokens. While effective, this makes reasoning expensive, length-sensitive, and constrained to (discrete) natural language. While latent reasoning offers a continuous alternative, determining useful structures for intermediate latent states is an open challenge. In this paper, we formulate latent reasoning as a geometric path-approximation problem within the model's pretrained token-embedding space. We introduce Geometric Latent Reasoning (GLR), which uses a lightweight transition head to predict iterative direction updates in embedding space. Using textual chain-of-thought traces as anchors, GLR learns to approximate discrete reasoning trajectories while permitting continuous deviations from exact token embeddings. Evaluations on mathematical reasoning benchmarks using Qwen3 models reveal an emergent phenomenon: geometric latent reasoning induces substantially shorter generations without an explicit length objective. By replacing early explicit reasoning with continuous latent steps, models often reach correct answers using substantially fewer total generation steps. These findings suggest that continuous trajectories act as compact intermediate reasoning states, exposing a new tradeoff between latent computation budget, output length, and accuracy. |
| Main Research Program: | AI for Everyone |
| Additional Research Programs: |
AI for Life |
| Keywords: | |
| Projects: |
ELOQUENCE |
| Authors: | |
| Added by: | [UNK] |
| Total mark: | 0 |
|
Attachments
|
|
|
Notes
|
|
|
|
|