about
Show HN: CellARC Measuring Intelligence with Cellular Automata (arxiv.org)
1 point by mireklzicar 326 days ago | hide | past | pdf | discuss on HN

In plain words: CellARC is a puzzle set where a model sees five examples of a hidden color-grid rule and must predict the next grid, with dials for difficulty like alphabet size. A plain tiny transformer beat recent recursive models, scoring 58% on rules from familiar settings.

Abstract · CellARC: Measuring Intelligence with Cellular Automata

We introduce CellARC, a synthetic benchmark for abstraction and reasoning built from multicolor 1D cellular automata (CA). Each episode has five support pairs and one query serialized in 256 tokens, enabling rapid iteration with small models while exposing a controllable task space with explicit knobs for alphabet size k, radius r, rule family, Langton's lambda, query coverage, and cell entropy. We release 95k training episodes plus two 1k test splits (interpolation/extrapolation) and evaluate symbolic, recurrent, convolutional, transformer, recursive, and LLM baselines. CellARC decouples generalization from anthropomorphic priors, supports unlimited difficulty-controlled sampling, and enables reproducible studies of how quickly models infer new rules under tight budgets. Our strongest small-model baseline (a 10M-parameter vanilla transformer) outperforms recent recursive models (TRM, HRM), reaching 58.0%/32.4% per-token accuracy on the interpolation/extrapolation splits, while a large closed model (GPT-5 High) attains 62.3%/48.1% on subsets of 100 test tasks. An ensemble that chooses per episode between the Transformer and the best symbolic baseline reaches 65.4%/35.5%, highlighting neuro-symbolic complementarity. Leaderboard: https://cellarc.mireklzicar.com

Miroslav Lžičař
arXiv:2511.07908 · cs.LG · submitted Nov 11, 2025
abstract · pdf · 22 pages, 11 figures. Working draft. Dataset and leaderboard available at https://cellarc.mireklzicar.com

add comment on HN