Attributed metadata history
Up to 100 retained publisher events per page for 86e22b82300247d9575999252835fcd93d91e1bbe0145857fc5f39d928d736c8. Withdrawals and unavailable references remain attributed to their publishers.
intelligence_src_e53adb99e40ae66258
observed · 2026-10-12 16:25:04 UTC
ePrint Report: Language Modeling is Monotone Compression Noam Mazor, Andrew Morgan, Rafael Pass A long-standing hypothesis in artificial intelligence and neuroscience posits that intelligence is closely related to compression: the ability to compress information efficiently intuitively reflects capacities associated with intelligence and learning. Indeed, recent experimental works verify this intuition by showing connections between the capabilities of large language models (LLMs) and their ability as compressors: for instance, Deletang et al. (ICLR'24) demonstrate that LLMs can be used as powerful compressors, and Huang et al. (COLM'24) show that the compression ability of LLMs is highly correlated with their performance on benchmarks for knowledge and reasoning. In this work, we initiate a theoretical study of this connection. Our main result is that LLMs (formally modeled as next-token predictors) are equivalent to monotone (a.k.a. order-preserving) compression algorithms---namely, compression algorithms where the encoding process preserves the ordering of the inputs---in the sense that the one can be
Fingerprint: 8bb7f58275b72068b66ed717f235ff0cc4936fe11f5702538e172a59819f7912
intelligence_src_e53adb99e40ae66258
observed · 2026-10-12 16:25:03 UTC
available
Fingerprint: 366cb966e32414edc1db353849a1ee8d6b8462b83786e04001881ba6594765ad