In plain words: A full year of real production traffic from one company was examined across many models and users, and the trace was released publicly. Traffic shifts over time and users stick with particular models, patterns that short or sampled studies miss.
Abstract
Large Language Model (LLM) serving has become a critical cloud workload, and realistic traces are essential for motivating and benchmarking serving systems. However, existing LLM serving workload studies remain limited in scale and scope. They often observe short time periods and provide limited visibility into how users interact with models in production. As a result, they do not fully capture how LLM serving workloads evolve over time or how user-model interactions shape production traffic. In this work, we further the understanding of real-world LLM serving workloads through both a global characterization and a longitudinal study of a one-year production trace from CompanyX. Unlike prior studies, our trace captures full production behavior across many models and users, including both popular and long-tail models. We analyze the workload from aggregate, temporal, model-level, and user-level perspectives, revealing workload evolution and user-model structure that are typically hidden behind aggregate views. To support future research, we publicly release the full one-year trace, enabling downstream studies of production behavior without relying on sampled or synthetically generated workloads. The trace is available at https://github.com/HarvardMadSys/chutes_workload.
William Nixon, Jon Durbin, Florian Standhartinger, Haryadi S. Gunawi, Juncheng Yang
arXiv:2608.13573 · cs.AI · submitted Jul 3, 2026 · updated Sep 7, 2026
abstract · pdf · html