The Reasoning Tax: When Thinking Too Much Costs More Than It Earns

Every LLM deployment today faces the same question: should reasoning be on or off? The industry default has been "on, always" — let the model think longer, reason harder, produce more tokens. But a new paper from Wani, Dholakia, and Ellison argues that this blanket approach is costing real money with diminishing returns.

The Reasoning Tax: Token Economics of LLM Reasoning Across Task Types and Deployment Contexts introduces the Token Economy Score (TES), a metric that measures accuracy gain normalized by token multiplier — essentially, the efficiency of reasoning spend. The core insight is brutal but intuitive: not all thinking is created equal, and more thinking doesn't always mean better answers.

What problem does this solve?

Benchmarking today compares accuracy numbers and calls it a day. But from a deployment perspective, accuracy gain per dollar is what actually matters. A model that achieves +5% accuracy at 10x the token cost might be a bad trade depending on the task. This paper is the first systematic attempt to quantify when the trade is worth making and when it isn't.

What's the method?

The authors define TES as the marginal accuracy improvement of a reasoning model over its non-reasoning baseline, divided by the token multiplier (how many more tokens the reasoning version generates). They introduce two variants:

• Paired TES — for model families where reasoning can be toggled on/off (e.g., the same base model with and without chain-of-thought)
• Approximated TES — for frontier models without direct non-reasoning counterparts, estimated via benchmark-specific baselines

They then run 151 model-benchmark evaluations across seven benchmarks: AIME 2025 (math), LiveCodeBench (code), GPQA (science reasoning), IFEval (instruction following), SimpleQA (knowledge recall), MMLU-Pro (expert knowledge), and research-level physics problems.

Three deployment-facing dimensions are examined:

• Which task structures yield positive marginal reasoning efficiency
• How increasing reasoning effort changes TES within model families
• How deployment context (cloud vs. on-premises) changes the economic calculus

Results

The findings are the kind that should live on a deployment checklist:

• Task structure predicts reasoning efficiency better than difficulty. Sequential inference-chain tasks (AIME 2025, LiveCodeBench) show high TES — each thinking token earns its keep. Knowledge-recall tasks (MMLU-Pro, SimpleQA) show low TES despite being subjectively difficult — the model is searching memory, not reasoning.
• Systematic diminishing returns at higher reasoning effort levels. Past a certain point, additional thinking tokens yield negligible accuracy gains — and in some cases, reduce accuracy as the model talks itself into wrong answers.
• Reasoning Cost Share (RCS): Inference spend is often dominated by internal "thinking" tokens, not the visible output.
• Deployment Cost Multiplier (DCM): On-premises deployment can radically change the economics — what's wasteful in the cloud may be viable on owned hardware, and vice versa.

mindmap
  root((Reasoning
Efficiency))
    High TES
      AIME 2025
      LiveCodeBench
      GPQA Science
    Low TES
      MMLU-Pro
      SimpleQA
      IFEval
    Diminishing Returns
      Token waste past threshold
      Accuracy degradation
    Deployment Context
      Cloud: cost-dominant
      On-prem: fixed-cost shift

Limitations

The study is about economics, not capability — it doesn't measure whether reasoning improves answer quality in absolute terms, only whether the improvement is worth the token cost. It also treats benchmarks as proxies for real workloads, which always introduces abstraction error. The analysis focuses on single-turn reasoning; multi-turn agentic contexts where reasoning compounds across steps aren't covered.

Why it matters

If you're deploying LLMs in production, this paper gives you a vocabulary for a decision you're already making implicitly. The recommendation is clean: enable reasoning selectively by task type, effort level, and deployment context. Don't treat "more thinking" as a universal good. For knowledge recall, a fast non-reasoning pass is cheaper and just as accurate. For multi-step inference chains, let the model burn tokens — they're actually productive.

The TES framework is immediately actionable. Any production team can adapt it: run your workload with and without reasoning, measure the token multiplier, divide the accuracy delta — and decide whether the tax is worth paying.