In plain words: Teams often pick the fastest hardware for running AI models in hospitals, but a comparison weighing both speed and cost under real healthcare limits found a plain processor can beat a graphics card. Set your speed needs first.
Abstract · Impact of Inference Accelerators on hardware selection
As opportunities for AI-assisted healthcare grow steadily, model deployment faces challenges due to the specific characteristics of the industry. The configuration choice for a production device can impact model performance while influencing operational costs. Moreover, in healthcare some situations might require fast, but not real time, inference. We study different configurations and conduct a cost-performance analysis to determine the optimized hardware for the deployment of a model subject to healthcare domain constraints. We observe that a naive performance comparison may not lead to an optimal configuration selection. In fact, given realistic domain constraints, CPU execution might be preferable to GPU accelerators. Hence, defining beforehand precise expectations for model deployment is crucial.
Dibyajyoti Pati, Caroline Favart, Purujit Bahl, Vivek Soni, Yun-chan Tsai, Michael Potter, Jiahui Guan, Xiaomeng Dong, V. Ratna Saripalli
arXiv:1910.03060 · cs.DC, cs.LG · submitted Oct 7, 2019
abstract · pdf · html