about
Benchmarking Confidential Computing Performance on Nvidia Blackwell GPUs (arxiv.org)
1 point by Jimmc414 34 days ago | hide | past | pdf | discuss on HN

In plain words: They measured the slowdown from running AI training and inference inside encrypted, sealed-off GPU memory by comparing paired runs on one machine that differ only in the security switch. Properly configured, the cost was just 1-3%, while default setups lost 30-40% to fixable settings.

Abstract · Benchmarking Confidential Computing Performance on NVIDIA Blackwell GPUs

This paper measures the performance impact of running large language model inference and training inside a Trusted Execution Environment (TEE) on NVIDIA B200 GPUs, using Intel Trust Domain Extensions (TDX) confidential VMs together with NVIDIA Confidential Computing (CC) on Blackwell GPUs. The performance impact is derived from paired confidential versus non-confidential runs on a single physical host where the only variable is the GPU CC bit and the TDX guest object in the VM launch. The main result is that confidential inference on Blackwell achieves low single-digit throughput overhead when the stack is configured correctly, at about 1-3%. Stock inference stacks incur 30 to 40% penalties due to avoidable configurations rather than the achievable operating point. The cost is not fully represented by a single number because it is governed by two independent axes, a fixed per-host-operation cost that amortizes as batch size grows and a per-NVLink-traffic cost that tracks the share of the step spent in encrypted collectives, and which of the two dominates is set by the workload and the software. We localize each cost to a specific encrypted boundary, give a microbenchmark that predicts the serving penalty to within a submission count, and end with concrete deployment guidance. GPU compute, energy draw, and usable memory capacity are unaffected by CC.

Amean Asad, Ansgar Grunseid
arXiv:2608.26575 · cs.DC, cs.CR · submitted Aug 27, 2026 · updated Sep 1, 2026
abstract · pdf · html · 23 pages, 3 figures, 22 tables

add comment on HN