about
Reconsidering the energy efficiency of spiking neural networks (arxiv.org)
1 point by PaulHoule on Sep 26, 2024 | hide | past | pdf | discuss on HN

In plain words: They built an energy calculator that counts computation plus every data move and memory access, comparing spiking networks with ordinary low-precision ones of equal capacity. Spiking only saves energy when, on average, under 5.7% of its neurons fire per step, with 5 steps.

Abstract · Reconsidering the Energy Efficiency of Spiking Neural Networks Inference from Analytical Perspectives

Spiking Neural Networks (SNNs) promise higher energy efficiency over conventional Quantized Artificial Neural Networks (QNNs) due to their event-driven, spike-based computation. However, prevailing energy evaluations often oversimplify, focusing on computational aspects while neglecting critical overheads like comprehensive data movements and memory accesses. Such simplifications can lead to misleading conclusions regarding the true energy benefits of SNNs. This paper presents a rigorous re-evaluation. We establish a fair baseline by mapping rate-encoded SNNs with $T$ timesteps to capacity-matched QNNs with $\lceil \log_2(T+1) \rceil$ bits. This ensures both models have comparable representational capacities, as well as similar hardware requirements, enabling meaningful energy comparisons. We introduce a detailed analytical energy model encompassing core computation and data movements. Using this model, we systematically explore a wide parameter space, including intrinsic network characteristics (SNN time window size, spike rate, QNN sparsity, model size, weight bit-level) and hardware characteristics (memory system and network-on-chip). Our analysis identifies specific operational regimes where SNNs genuinely offer superior energy efficiency. For example, under typical neuromorphic hardware conditions, SNNs with moderate time windows ($T = 5$) require an average spike rate ($s_r$) below 5.7% to outperform equivalent QNNs These insights guide the design of truly energy-efficient neural network solutions.

Zhanglu Yan, Zhenyu Bai, Kaiwen Tang, Weng-Fai Wong
arXiv:2409.08290 · cs.NE, cs.AI, cs.LG · submitted Aug 29, 2024 · updated Aug 4, 2026
abstract · pdf · html · accepted by TCAD

add comment on HN