Reasoning Shift: How Context Silently Shortens LLM Reasoning | AI Paper Digest

TL;DR Highlight

When irrelevant context is present, reasoning models skip self-verification and cut reasoning tokens by up to 50%.

Who Should Read

Developers building LLM agents or multi-turn chatbots who wonder why accuracy drops on complex reasoning tasks. ML engineers designing systems that use reasoning models in long-context environments.

Core Mechanics

When irrelevant context is present (e.g., long documents, previous conversation history), reasoning models generate up to 50% fewer reasoning tokens for the same problem. Confirmed across all four models: Qwen3.5-27B, GPT-OSS-120B, Gemini 3 Flash Preview, and Kimi K2 Thinking.
The shortened reasoning is not due to the model misunderstanding the problem. In fact, the model immediately recognizes irrelevant context as 'not relevant' and moves on — yet reasoning still becomes shorter.
The key difference lies not in 'when the first candidate answer is found' but in 'the self-verification behavior that follows.' In the baseline, models continue double-checking after finding an answer, but under long-input conditions, models terminate reasoning immediately after producing an answer far more often (57% vs. 68% immediate termination).
Inserting just a few hundred tokens of irrelevant prefix reduces average reasoning length by 18%, and inserting 64k tokens reduces it by up to 53%.
For easy problems, this phenomenon acts as a reduction in overthinking and does not affect accuracy, but for difficult math problems (by IMOAnswerBench standards), it causes a 9–15% accuracy drop.
This phenomenon is far more pronounced in thinking mode. Non-thinking mode sees a 19% reduction in response length, while thinking mode sees a 53% reduction.

Evidence

"Accuracy drops under Subtask/Long input scenarios on IMOAnswerBench: Qwen-3.5-27B 12%, GPT-OSS-120B 9%, Gemini 3 Flash Preview 15%, Kimi K2 Thinking 9%. Average reasoning length for Qwen3.5-27B on MATH500: Baseline 8,003 tokens → Long input (64k prefix) 3,762 tokens, a 53% reduction. In resampling experiments starting from the same reasoning prefix, 46% of Long input conditions terminated reasoning immediately vs. only 21% in Baseline conditions; frequencies of self-verification keywords such as 'Wait', 'Alternatively', and 'But' also decreased. Inserting as few as 128 tokens reduces Qwen3.5-27B's average reasoning length by approximately 18%, with the reduction increasing monotonically as more tokens are inserted."

How to Apply

"When building multi-turn agents, be aware that longer conversation histories can degrade reasoning quality for the current subtask. Consider architectures that process subtasks in isolated new contexts (separate API calls) or apply context compaction. When using reasoning models for difficult reasoning tasks, avoid prepending unnecessary background information or prior outputs to the prompt. Extracting only the necessary information and constructing a minimal context helps preserve reasoning quality. When using reasoning models in a RAG pipeline, inserting retrieved documents wholesale can shorten reasoning. Consider precisely extracting only the relevant chunks, or decomposing the problem into independent subtasks and calling the model separately with shorter context for each."

Code Example

snippet

# Example patterns for maintaining reasoning quality in long-context environments
# Bad example: putting irrelevant context and the problem together
bad_prompt = """
[Previous conversation history or thousands of tokens of long document...]

Based on the above, please solve the following math problem:
{problem}
"""

# Good example 1: call subtask in an isolated new context
def solve_subtask_isolated(problem: str, client) -> str:
    """Isolate subtasks in a complex agent workflow to maintain reasoning quality"""
    response = client.chat.completions.create(
        model="qwen/qwen3.5-27b",  # or another reasoning model
        messages=[
            {
                "role": "user",
                "content": f"Please reason step-by-step and put the final answer within \\boxed{{}}\n\n{problem}"
            }
        ],
        # Pass only the problem — do not carry over the long agent context
    )
    return response.choices[0].message.content

# Good example 2: if long context is unavoidable, explicitly instruct the model to ignore it
def solve_with_context_hint(problem: str, context: str, client) -> str:
    response = client.chat.completions.create(
        model="qwen/qwen3.5-27b",
        messages=[
            {
                "role": "user",
                "content": f"[Background context - you may ignore this]:\n{context}\n\n[Task - focus entirely on this]:\nPlease reason step-by-step carefully, verify your answer, and put the final answer within \\boxed{{}}\n\n{problem}"
            }
        ]
    )
    return response.choices[0].message.content

Terminology

Test-time scalingA strategy for improving performance by using more computation at inference time (when generating answers) rather than during training. The principle is similar to how spending more time thinking during an exam leads to a higher correct answer rate.

Chain-of-Thought (CoT)A technique that prompts the model to write out intermediate reasoning steps before arriving at a final answer, rather than producing the answer directly. Like showing your work: '1+1=2, and 2+3=5, so the answer is 5.'

Reasoning traceThe long chain-of-thought text the model generates before producing an answer. It is the content inside `<think>...</think>` tags and includes review, revision, and self-verification steps.

Self-verificationThe behavior in which a model reviews its own answer and checks whether it is correct. Analogous to reviewing your answers on an exam after finishing.

Uncertainty managementThe behavior in which a model recognizes uncertain points during reasoning — such as 'Is this right?' or 'Could there be another approach?' — and explores alternatives.

IMOAnswerBenchAn LLM evaluation benchmark (reference dataset) composed of difficult math problems at the level of the International Mathematical Olympiad (IMO).

DPODirect Preference Optimization. A method for training a model by comparing responses that humans prefer against those they do not. A simpler alternative to RLHF.

Context compactionA technique for efficiently managing the context window in long-running agent conversations by summarizing and compressing older context.

Related Papers

Related Resources

Original Abstract (Expand)

Large language models (LLMs) exhibiting test-time scaling behavior, such as extended reasoning traces and self-verification, have demonstrated remarkable performance on complex, long-term reasoning tasks. However, the robustness of these reasoning behaviors remains underexplored. To investigate this, we conduct a systematic evaluation of multiple reasoning models across three scenarios: (1) problems augmented with lengthy, irrelevant context; (2) multi-turn conversational settings with independent tasks; and (3) problems presented as a subtask within a complex task. We observe an interesting phenomenon: reasoning models tend to produce much shorter reasoning traces (up to 50%) for the same problem under different context conditions compared to the traces produced when the problem is presented in isolation. A finer-grained analysis reveals that this compression is associated with a decrease in self-verification and uncertainty management behaviors, such as double-checking. While this behavioral shift does not compromise performance on straightforward problems, it might affect performance on more challenging tasks. We hope our findings draw additional attention to both the robustness of reasoning models and the problem of context management for LLMs and LLM-based agents.