 |
ML Newsnew | past | best | rss | submit | about |
| PIM-LLM: A High-Throughput Hybrid PIM Architecture for 1-Bit LLMs (arxiv.org) | |
| 3 points by PaulHoule on Apr 18, 2025 | hide | past | pdf | discuss on HN | |
|
In plain words: A chip that runs language models storing each weight as one bit, doing matrix math inside memory cells: analog circuits handle 1-bit layers and digital arrays handle attention. It yields up to 80 times more words per second than standard chips, using less energy. Abstract · PIM-LLM: A High-Throughput Hybrid PIM Architecture for 1-bit LLMsIn this paper, we propose PIM-LLM, a hybrid architecture developed to accelerate 1-bit large language models (LLMs). PIM-LLM leverages analog processing-in-memory (PIM) architectures and digital systolic arrays to accelerate low-precision matrix multiplication (MatMul) operations in projection layers and high-precision MatMul operations in attention heads of 1-bit LLMs, respectively. Our design achieves up to roughly 80x improvement in tokens per second and a 70% increase in tokens per joule compared to conventional hardware accelerators. Additionally, PIM-LLM outperforms previous PIM-based LLM accelerators, setting a new benchmark with at least 2x and 5x improvement in GOPS and GOPS/W, respectively. | |
| add comment on HN |