Early Stopping for Large Reasoning Models via Confidence Dynamics | AI Paper Digest

TL;DR Highlight

A method to save 25-50% of tokens by observing the pattern of changes in the model's confidence during inference and stopping unnecessary reasoning early.

Who Should Read

ML engineers who are deploying reasoning models like DeepSeek-R1 and Qwen3 into production and are struggling with high inference costs. Developers who want to reduce the inference cost of long chain-of-thought generation.

Core Mechanics

Correct reasoning trajectories show a rapid increase in confidence early in generation and stabilize quickly, while incorrect trajectories exhibit unstable and fluctuating confidence patterns.
Incorrect reasoning trajectories are more than twice as long as correct ones (12K vs 25K tokens) and account for a significant portion of the total computation.
CoDE-Stop combines two signals: (1) a ramping confidence threshold to stop when confidence is sufficiently high, and (2) a degeneration score to detect unstable reasoning.
The degeneration score gives higher weights (log weighting) to early reasoning steps, as early confidence patterns are more useful for distinguishing between correct and incorrect answers.
Even as inference continues, the confidence of incorrect trajectories tends to increase, making it easy to misjudge based on late-stage confidence alone – therefore, early signals are more important.
It requires no additional training and can be directly attached to existing models at inference time. It also has additive effects when used with prompting techniques like Chain-of-Draft.

Evidence

"Qwen3-4B achieved a 25% reduction in token usage while maintaining accuracy across 4 benchmarks (8344 → 5956 tokens).\nQwen3-14B maintained 93.0% accuracy on MATH500 while reducing tokens by 49.1% (4878 → 2529 tokens).\nCoDE-Stop achieved a better accuracy-compute tradeoff than DEER (the most similar baseline) across all models and benchmarks. For example: Qwen3-4B AIME achieved similar accuracy with 13400 tokens (DEER) vs 12800 tokens (CoDE-Stop).\nOn GSM8K, Qwen3-4B achieved a compression rate of 52.4% (Vanilla 2306 → CoDE-Stop 1233 tokens) with the highest compression rate, maintaining accuracy at 94.8% → 94.6%."

How to Apply

"If you are using reasoning models like Qwen3, DeepSeek-R1, and Nemotron on HuggingFace, you can wrap CoDE-Stop around the inference loop by importing the GitHub code. Start with max_new_tokens set to 32K, δ=0.55, and rmax=0.95 without any additional training.\nDuring inference, measure the confidence by generating intermediate answers each time a reasoning step delimiter token such as 'Wait' or '\\n\\n' appears, and force the final answer to be generated when the degeneration score exceeds the threshold τ.\nTo adjust the cost vs. accuracy tradeoff, simply change τ (the degeneration threshold). A smooth tradeoff curve was confirmed in the paper, so lowering τ will stop earlier and save tokens, while raising it will allow for longer inference and recover accuracy."

Code Example

snippet

Terminology

Chain-of-Thought (CoT)A method where the model writes out the intermediate reasoning process step-by-step before giving the final answer. It's like a person writing out the solution process when solving a math problem.

Confidence ScoreA number between 0 and 1 representing how confident the model is in its answer. It is calculated as the average probability of the generated tokens.

Degeneration ScoreAn accumulated score that detects whether the reasoning is going astray. The score increases as unstable confidence patterns accumulate, and inference is stopped when the threshold is exceeded.

Reasoning TrajectoryThe entire reasoning flow generated by the model until it reaches the final answer. It is divided into a path that reaches the correct answer (correct) and a path that wanders (incorrect).

OverthinkingA phenomenon where the model continues reasoning even after reaching the correct answer, wasting tokens and sometimes even degrading performance.

Inference-timeA method of applying to the model at the time of actual use (inference) without additional training. It is practical because it can be attached immediately without training.

Compression Rate (CR)The ratio of how much the inference is shortened. It is the ratio of the reduction to the original length, and the lower the value, the more compressed it is.

Related Papers

Related Resources

Original Abstract (Expand)

Large reasoning models rely on long chain-of-thought generation to solve complex problems, but extended reasoning often incurs substantial computational cost and can even degrade performance due to overthinking. A key challenge is determining when the model should stop reasoning and produce the final answer. In this work, we study the confidence of intermediate answers during reasoning and observe two characteristic behaviors: correct reasoning trajectories often reach high-confidence answers early, while incorrect rollouts tend to produce long, unproductive reasoning traces and exhibit less reliable confidence dynamics. Motivated by these observations, we propose CoDE-Stop (Confidence Dynamics Early Stop), an early stopping method that leverages the dynamics of intermediate answer confidence to decide when to terminate reasoning, requiring no additional training and easily integrating into existing models. We evaluate CoDE-Stop on diverse reasoning and science benchmarks across multiple models. Compared to prior early stopping methods, it achieves a more favorable accuracy-compute tradeoff and reduces total token usage by 25-50% compared to standard full-length reasoning. In addition, we provide analyses of confidence dynamics during reasoning, offering insights into how confidence changes in both correct and incorrect trajectories.