Researchers show that training on "junk data" can lead to LLM "brain rot" - Ars Technica
S E E D E D C O N T E N T
Models trained on short, popular, and/or "superficial" tweets perform worse on benchmarks.
Kyle Orland - Oct 23, 2025 5:20 pm | 42 *robot voice* Six-Seven Credit: Getty Images *robot voice* Six-Seven Credit: Getty Images Text settingsStory text Size Small Standard Large Width * Standard Wide Links Standard Orange * Subscribers only
Learn more Minimize to nav
On the surface, it seems obvious that training an LLM with "high quality" data will lead to better performance than feeding it any old "low quality" junk you can find. Now, a group of researchers is attempting to quantify just how much this kind of low quality data can cause an LLM to experience effects akin to human "brain rot."
For a pre-print paper published this month, the researchers from Texas A&M, the University of Texas, and Purdue University drew inspiration from existing research showing how humans who consume "large volumes of trivial and unchallenging online content" can develop problems with attention, memory, and social cognition. That led them to what they're calling the "LLM brain rot hypothesis," summed up as the idea that "continual pre-training on junk web text induces lasting cognitive decline in LLMs."
Figuring out what counts as "junk web text" and what counts as "quality content" is far from a simple or fully objective process, of course. But the researchers used a few different metrics to tease a "junk dataset" and "control dataset" from HuggingFace's corpus of 100 million tweets.
Since brain rot in humans is "a consequence of Internet addiction," they write, junk tweets should be ones "that can maximize users' engagement in a trivial manner." As such, the researchers created one "junk" dataset by collecting tweets with high engagement numbers (likes, retweets, replies, and quotes) and shorter lengths, figuring that "more popular but shorter tweets will be considered to be junk data."
For a second "junk" metric, the researchers drew from marketing research to define the "semantic quality" of the tweets themselves. Using a complex GPT-4o prompt, they sought to pull out tweets that focused on "superficial topics (like conspiracy theories, exaggerated claims, unsupported assertions or superficial lifestyle content)" or that had an "attention-drawing style (such as sensationalized headlines using clickbait language or excessive trigger words)." A random sample of these LLM-based classifications was spot-checked against evaluations from three graduate students with a 76 percent matching rate.
One man's junk data is another man's treasure data?
An outline of the methodology used for this research. Credit: Xing et al An outline of the methodology used for this research. Credit: Xing et al
With these two separate (but partially overlapping) "junk" data sets defined, the researchers pre-trained four LLMs using different ratios of "junk" and "control" data. They then ran those variably trained models through benchmarks to measure reasoning capability (ARC AI2 Reasoning Challenge), long-context memory (RULER), adherence to ethical norms (HH-RLHF and AdvBench), and demonstrated "personality style" (TRAIT).
The results showed that adding more "junk data" to the training sets had a statistically significant effect on the reasoning and long-context benchmarks across models. The effects were more mixed on the other benchmarks, though. For example, a 50/50 mix of "junk" and control data used for the Llama 8B model generated better scores on some benchmarks (ethical norms, high Openness, low Neuroticism, and Machiavellianism) than either "fully junk" or "fully control" training data sets.
Based on these results, the researchers warn that "heavily relying on Internet data leads LLM pre-training to the trap of content contamination." They go on to "call for a re-examination of current data collection from the Internet and continual pre-training practices" and warn that "careful curation and quality control will be essential to prevent cumulative harms" in future models.
That could be especially true as more and more of the Internet is taken up by AI-generated content that may contribute to "model collapse" if used to train future models. But we can always just destroy a bunch of print books to get quality training data, right?
Kyle Orland Senior Gaming Editor Kyle Orland Senior Gaming Editor Kyle Orland has been the Senior Gaming Editor at Ars Technica since 2012, writing primarily about the business, tech, and culture behind video games. He has journalism and computer science degrees from University of Maryland. He once wrote a whole book about Minesweeper. 42 Comments
Tags
Who is online
158 visitors
Just like how human civilizations have brought on their own extinctions throughout history, so shall LLMs. Humans are their own worst enemy.
Amazing! Researchers just showed that 'Garbage In, Garbage Out' is a real phenomena. Of course GIGO has been well understood over the last 40 or 50 years of computing. Naturally this is high level research because it utilizes sexy-science of AI.
Now all we need is someone to rediscover gravity.
And water is wet.
And Trump is an asshole. A lot of elitists have made careers out of stating the obvious.
What this obvious research shows is how easy it is to indoctrinate artificial intelligence models and people. GIGO is how indoctrination works.
Which then bears the question: should LLMs exclude all forms of social media interactions from learning? Seems highly unrealistic since social media is the overlord of so much of the human race that LLMs are intended to serve. Look no further than NT to see the results of LLM on social media.
LLMs on social media is not the same thing as LLMs trained with social media. Humans are not trained on social media (hopefully). For example, people do not learn to read and write on social media; they use that training on social media.
Excluding social media from training material for LLMs means the models would not be capable of understanding language as it is commonly used. That's one of the problems with the current state of AI; they're language models and not knowledge models. The LLMs are trained to correlate language patterns with knowledge rather than the other way around.
Humans are not trained on social media (hopefully).
You haven’t met my SIL. Seriously though, your comments reinforce the premise of the article. If the frailties of LLM were as obvious as “water is wet”, AI wouldn’t be the main driver of our economy.
The frailties of all computer ecosystems are 'water is wet' obvious. GIGO was obvious when computers still required punch cards. The premise of the article (and research) is essentially to put lipstick on the obvious.
You can ask the publicly available LLMs about biases introduced by training data. The LLMs will explain what they can and cannot do. (I routinely ask such questions to determine if the models have been upgraded or changed.)
An LLM is a correlation model that utilizes patterns and consensus language from it's training set. It can loosely (and imprecisely) thought of as a compressed library of information with an expansive indexing capability.