In plain words: Instead of writing a special rulebook for each job, this system asks a pretrained model to pick the next action from the current state, goal, and choices. A small model met response-time targets using within 10% of the copies that hand-written scaling rules allocate.
Abstract
Resource managers typically encode what to do in a task-specific policy---a heuristic, an optimization procedure, or a model trained for the task---that must be developed before use and may need revision as the workload or system changes. We propose bypassing policy development and instead asking pretrained models to make resource-management decisions directly, given the current system state, desired outcome, and available choices. We evaluate this approach on microservice autoscaling, DRAM sizing, and video encoding, using a frontier LLM and two small decision models. On two microservice applications, a small decision model meets latency targets while using within 10% of the replicas allocated by policy-based autoscalers and within 13% of the frontier LLM. In DRAM sizing, a small decision model reduces DRAM usage by up to 27.5% relative to static provisioning while maintaining at least 95% of baseline performance. In video encoding, it completes encodes up to 4.3$\times$ faster than the frontier LLM, with an imperceptible 1.1-point decrease in Video Multi-Method Assessment Fusion (VMAF), a metric of perceptual video quality. Making individual resource-management decisions may be considerably simpler than developing a general strategy for making them.
Saarth Deshpande, Neeraja J. Yadwadkar, Venkat Arun
arXiv:2611.01463 · cs.PF, cs.AI, cs.CV, cs.DC, cs.OS · submitted Nov 1, 2026
abstract · pdf · html