about
A Comparison of LLM and Human Performance on Random Number Generation Tasks[pdf] (arxiv.org)
1 point by bikenaga on Aug 21, 2024 | hide | past | pdf | discuss on HN

In plain words: A psychology task asking people to list random numbers was adapted so ChatGPT-3.5 could try it, testing whether the model shows the same habits humans do. Its sequences repeated numbers and used neighboring numbers less often than human ones.

Abstract · A Comparison of Large Language Model and Human Performance on Random Number Generation Tasks

Random Number Generation Tasks (RNGTs) are used in psychology for examining how humans generate sequences devoid of predictable patterns. By adapting an existing human RNGT for an LLM-compatible environment, this preliminary study tests whether ChatGPT-3.5, a large language model (LLM) trained on human-generated text, exhibits human-like cognitive biases when generating random number sequences. Initial findings indicate that ChatGPT-3.5 more effectively avoids repetitive and sequential patterns compared to humans, with notably lower repeat frequencies and adjacent number frequencies. Continued research into different models, parameters, and prompting methodologies will deepen our understanding of how LLMs can more closely mimic human random generation behaviors, while also broadening their applications in cognitive and behavioral science research.

Rachel M. Harrison
arXiv:2408.09656 · cs.AI, cs.CL, q-bio.NC · submitted Aug 19, 2024 · updated Aug 20, 2024
abstract · pdf · html

add comment on HN