LLMs Don’t Need More Parameters. They Need Loops.

Bystaff



A deep dive into how looped language models change the scaling game.

Our paper “Scaling Latent Reasoning via Looped Language Models” goes into a lot more depth and can be found on arXiv: https://arxiv.org/abs/2510.25741

Ouro Homepage: https://ouro-llm.github.io/

Video developed by Jason Eshraghian and Rui-Jie Zhu.

—

Papers cited in the video:

Scaling Laws for Neural Language Models: https://arxiv.org/abs/2001.08361

Will we run out of data? Limits of LLM scaling based on human-generated data: https://arxiv.org/abs/2211.04325

Does Reinforcement Learning Really Incentivize Reasoning Capacity in LLMs Beyond the Base Model?: https://arxiv.org/abs/2504.13837

Universal Transformers: https://arxiv.org/abs/1807.03819

PonderNet: Learning to Ponder: https://arxiv.org/abs/2107.05407

Physics of Language Models: https://physics.allen-zhu.com/

—

Timestamps

0:00 – Scaling Laws
1:42 – The Data Wall
2:40 – Reasoning and its Problems
6:50 – Looped LLMs
9:30 – Dynamic Termination
13:13 – Reward Hacking
15:09 – Entropy Regularization
16:42 – Looped KV Caching
19:41 – Training Pipeline
20:44 – Results
22:43 – Physics of LLMs

source

Similar Posts