about
Pushing Tensor Accelerators Beyond MatMul in a User-Schedulable Language (arxiv.org)
2 points by matt_d 234 days ago | hide | past | pdf | discuss on HN

In plain words: A compiler tool rewrites image-processing math—filters, resampling, denoising—into matrix operations so the fast matrix-multiplying hardware in CPUs and GPUs can run it, while programmers choose how the work is ordered. Downsampling alone ran 6.1× faster on an Nvidia GPU than without it.

Abstract

Tensor accelerators now represent a growing share of compute resources in modern CPUs and GPUs. However, they are hard to program, leading developers to use vendor-provided kernel libraries that support tensor accelerators. As a result, the usage of tensor accelerators is limited to the provided interface, mainly designed for traditional ML and scientific computing workloads. In this paper, we show that tensor accelerators can improve the performance of applications beyond simple variants of MatMul. For example, many image processing pipelines are linear transformations over matrices in disguise and can therefore utilize such specialized hardware. This is nonetheless hindered by the difficulties in programming tensor accelerators. We tackle this problem with compiler-based techniques. We use the Halide user-schedulable language and express operations as Halide algorithms succinctly. To this end, we implement a flexible tensor instruction selector based on equality saturation. The tensor instruction selector supports both CPU- and GPU-attached tensor accelerators and works with existing scheduling operations (e.g., producer-consumer fusion). Together, this enables developers to write diverse accelerator-leveraging applications in a few dozen lines. Using our system, we demonstrate the potential of tensor accelerators beyond their traditional domains. We implement several image processing pipelines (e.g., filtering, resampling, and denoising) in our system and evaluate them against non-accelerator-leveraging baselines. We show that these pipelines can achieve significant speedups. For example, a downsampling routine is sped up by $6.1\times$ by utilizing Tensor Cores on an Nvidia RTX 4070 GPU.

Yihong Zhang, Derek Gerstmann, Andrew Adams, Maaz Bin Safeer Ahmad
arXiv:2512.02371 · cs.PL · submitted Dec 2, 2025 · updated Feb 11, 2026
abstract · pdf · html · CGO 2026

add comment on HN