about
Evaluating CUDA Tile for AI Workloads on Hopper and Blackwell GPUs (arxiv.org)
2 points by matt_d 158 days ago | hide | past | pdf | discuss on HN

In plain words: CuTile is a Python way to write GPU kernels in small tiles that still uses the chip's matrix hardware. It beat the standard attention kernel by 2.5x on the biggest Blackwell chip but only 53% of its speed on another, so it needs retuning per chip.

Abstract

NVIDIA's CUDA Tile (CuTile) introduces a Python-based, tile-centric abstraction for GPU kernel development that aims to simplify programming while retaining Tensor Core and Tensor Memory Accelerator (TMA) efficiency on modern GPUs. We present the first independent, cross-architecture evaluation of CuTile against established approaches such as cuBLAS, Triton, WMMA, and raw SIMT on three NVIDIA GPUs spanning Hopper and Blackwell: H100 NVL, B200, and RTX PRO 6000 Blackwell Server Edition. We benchmark representative AI workloads, including GEMM, fused multi-head attention, and end-to-end LLM inference in BF16/FP16 precision, to assess both performance and portability. Our results show that CuTile effectiveness is strongly workload- and architecture-dependent. On datacenter-class Blackwell (B200), CuTile achieves up to 1007 TFLOP/s for fused attention, outperforming FlashAttention-2 by 2.5x while requiring only 60 lines of Python kernel code. For GEMM, CuTile reaches 52-79% of cuBLAS performance in 22 lines of code (versus 123 for WMMA), making it a practical replacement for hand-written CUDA kernels but not yet for vendor-optimized libraries. However, the same CuTile attention kernel achieves only 53% of FlashAttention-2 throughput on RTX PRO 6000 (sm_120), exposing significant cross-architecture optimization gaps. In contrast, Triton sustains 62-101% of cuBLAS performance across all tested platforms without architecture-specific tuning, demonstrating substantially stronger portability.

Divakar Kumar Yadav, Tian Zhao, Deepak Kumar
arXiv:2604.23466 · cs.LG, cs.AI, cs.AR · submitted Apr 25, 2026 · updated Jun 3, 2026
abstract · pdf · html

add comment on HN