about
The Price Reversal Phenomenon: When Cheaper Reasoning Models Cost More (arxiv.org)
3 points by root-parent 119 days ago | hide | past | pdf | discuss on HN

In plain words: Checked whether the listed API price of eight reasoning models matches what users actually pay across twelve tasks, counting the extra thinking tokens and back-and-forth steps each answer needs. In 32% of pairs, the cheaper-looking model actually cost more, up to 28 times.

Abstract

Developers and consumers increasingly choose reasoning models (RMs) based on their listed API prices. However, how accurately do these prices reflect actual inference costs? We conduct the first systematic study of this question, evaluating 8 frontier RMs across 12 diverse tasks covering competition math, science QA, code generation, and multi-domain agents. We uncover the pricing reversal phenomenon: in 32% of model-pair comparisons, the model with a lower listed price actually incurs a higher total cost, with reversal magnitude reaching up to 28x. For example, Gemini 3 Flash's listed price is 80% cheaper than GPT-5.4's, yet its actual cost across all tasks is 38% higher. We build a formal cost attribution framework based on Shapley value, and leverage it to trace the dominating contributors to vast heterogeneity in thinking token consumption and number of interaction turns: on the same query, one model may use 900% more thinking tokens than another, or 10x more turns of environment interactions. We further show that per-query cost prediction is fundamentally difficult: repeated runs of the same query yield thinking token variation up to 9.7x, establishing an irreducible noise floor for any predictor. Thus, we propose cost distribution prediction as an open challenge. Our findings demonstrate that listed API pricing is an unreliable proxy for actual cost, calling for cost-aware model selection and transparent per-request cost monitoring.

Lingjiao Chen, Chi Zhang, Yeye He, Ion Stoica, Matei Zaharia, James Zou
arXiv:2603.23971 · cs.CL, cs.AI, cs.GT, cs.LG, cs.MA · submitted Mar 25, 2026 · updated May 28, 2026
abstract · pdf · html

add comment on HN