about
I designed a bfloat16/FP8 alternative in a week using LLMs (arxiv.org)
3 points by k1832 207 days ago | hide | past | pdf | 4 comments on HN

In plain words: A new number format for AI chips drops the usual hidden leading bit and base-2 scaling, storing its digits outright and scaling in base 4, so 8-bit values need no per-block rescaling hardware. In the multiply-accumulate unit it cut silicon area by 33%.

Abstract · The AetherFloat Family: Block-Scale-Free Quad-Radix Floating-Point Architectures for AI Accelerators

The IEEE 754 floating-point standard is the bedrock of modern computing, but its structural requirements -- a hidden leading bit, Base-2 bit-level normalization, and Sign-Magnitude encoding -- impose significant silicon area and power overhead in massively parallel Neural Processing Units (NPUs). Furthermore, the industry's recent shift to 8-bit formats (e.g., FP8 E4M3, OCP MX formats) has introduced a new hardware penalty: the strict necessity of Block-Scaling (AMAX) logic to prevent out-of-bound Large Language Model (LLM) activations from overflowing and degrading accuracy. The AetherFloat Family is a parameterizable architectural replacement designed from first principles for Hardware/Software Co-Design in AI acceleration. By synthesizing Lexicographic One's Complement Unpacking, Quad-Radix (Base-4) Scaling, and an Explicit Mantissa, AetherFloat achieves zero-cycle native integer comparability, branchless subnormal handling, and a verified 33.17% area, 21.99% total power, and 11.73% critical path delay reduction across the multiply-accumulate (MAC) unit. Instantiated as AetherFloat-8 (AF8), the architecture relies on a purely explicit 3-bit mantissa. Combined with Base-4 scaling, AF8 delivers a substantially wider dynamic range, acting as a ``Block-Scale-Free'' format for inference that circumvents dynamic scaling microarchitecture. Finally, a novel Vector-Shared 32-bit Galois Stochastic Rounding topology bounds precision variance while neutralizing the vanishing gradients that plague legacy formats. While AF16 serves as a near-lossless bfloat16 replacement via post-training quantization, AF8 is designed as a QAT-first inference format: its Block-Scale-Free property eliminates dynamic AMAX hardware at the cost of requiring quantization-aware fine-tuning for deployment.

Keita Morisaki
arXiv:2603.08741 · cs.AR, cs.LG · submitted Feb 26, 2026
abstract · pdf · html

add comment on HN

The "Block-Scale-Free" property is the most compelling part here. Anyone who's run quantized LLMs locally knows that dynamic scaling logic is a real pain point — it adds complexity and is often where things silently go wrong. Trading that for QAT-first deployment seems like a reasonable bargain, especially for edge inference where you want the simplest possible hardware path. Curious whether AF8 has been tested against GGUF Q8_0 on any standard benchmarks.
Thanks! Exactly, getting rid of that dynamic scaling hardware tax was the exact goal.

Regarding GGUF Q8_0: I haven't benchmarked against it yet. My focus so far was on proving the hardware thesis (RTL synthesis via SkyWater 130nm) and validating the numerics/convergence via PyTorch QAT.

Bridging this into the ggml/llama.cpp ecosystem to run standard LLM benchmarks is absolutely the next logical step. Getting this to run efficiently in software (simulating the hardware behavior) to compare against Q8_0 is something I'm looking into next.

If anyone in the local inference community is interested in exploring this or has pointers on the best way to integrate custom QAT formats into standard benchmarking pipelines, I'm all ears!

GP is a bot.
Oh.. thanks for letting me know