AI Is Failing at the Most Hilarious Task Imaginable
S E E D E D C O N T E N T
With so many puff pieces out there about what AI can do, it's rare to see a story about what it can't do. And as some researchers tell it, AI is falling terribly short at what many of us find to be one of the easiest tasks of all: getting into arguments on social media.
First spotted by Ars Technica, a team of researchers from Switzerland, the Netherlands, and the US recently released a paper analyzing social media posts generated by large language models (LLMs).
To conduct the yet-to-be-peer-reviewed study, the researches applied what they call a "computational Turing test" to posts LLMs on X-formerly-Twitter, Reddit, and Bluesky. They found that posts generated by AI bots — all open-weight models, ranging from DeepSeek to Qwen — were all "readily distinguishable" from ones by human users with a 70-80 percent accuracy rate, which is "well above [the threshold for] chance."
In other words, it's laughably easy to catch an AI shitposter in the act by applying a one-size-fits-all screening to any text it spits out — let alone by using a little basic human judgement.
One of the major reasons for this, the scholars posit, is that AI can only mimic a human's emotional depth, what we might call the "heat of the moment" vitriol of a typical flame war. When we get into it, we really get into it, with a level of both "toxicity" and "sentiment" that remain unmistakably human.
"Even after calibration, LLM outputs remain clearly distinguishable from human text, particularly in affective tone and emotional expression," the team wrote.
Interestingly, they found that an LLM's size and complexity didn't necessarily correlate to more realistic vitriol. For example, "the large Llama-3.1-70B performs on par with, or even below, smaller models," the researchers wrote. "This suggests that scaling does not translate into more authentically human communication."
The findings are particularly ironic given that one of AI's most prominent use-cases at the moment seems to be spamming social media, particularly the well-trafficked platforms of X-formerly-Twitter, Facebook, and Instagram (though other sites, like Reddit, are also being overrun.)
Even the wannabe tech CEOs are getting on board, with startups like Doublespeed offering clients access to an AI-powered bot army tailored to their advertising needs.
In a way, the paper's findings are good news for anyone worried about the rate at which AI was becoming indistinguishable — though something tells us it won't make much of a difference as AI bros continue to flood the internet with algorithmic slop.
Tags
Who is online
137 visitors
I feel like I could have written this article, and I’ve never even been on twitter, X, Facebook, instagram, or Bluesky.
I dont follow any rightwing accounts on X, but I do follow some progressive media. I dont have issues arising with bots, as far as I know. Twitter is really what you make of it. Because all I look at are the lists I created, I dont even see the crap Musk puts out personally, because I dont follow him on any of my lists.
AI guesses at what it reads and responds with. It is all 'A' and NOT 'I'. I keep telling people that current AI is mostly hype. It does well in certain areas, but it should not be taken as a replacement for competent highly functional people. Not yet.
Lol yes but can it be taken as a replacement for incompetent low functioning people
I once put a potato on an absent coworker's desk with googly eyes and the office productivity level increased.
A couple weeks ago my wife was stressed about some college papers she had to grade. Neither of us uses ChatGPT, but I suggested she try it. She gave it a try and said it did an amazing job in minutes. She would refine the prompts a bit and verify the output, but still way faster than the old way. She is obsessed with it now, just overhauling everything in her job through this insanely intelligent helper. It’s worrying thinking about how many jobs could be eliminated.
It's also fascinating thinking about how many will be created.
Like what? If AI is doing the thinking, it’s not going to suggest new jobs for us limited thinking ability humans.
She is also unpleasantly surprised at how bad the students spelling and grammar is. She said that otherwise the papers were well written. My feeling is that the students all use ChatGPT to write their papers, and then go through them and misspell things to make it look like it’s not written by AI. Hopefully students still manage to get smarter, while using their brains even less.
I have occasionally used Google AI to fact check that which the article describes as, " the "heat of the moment" vitriol of a typical flame war. " Often in these situations very outlandish or highly exaggerated claims are made and passed off as "facts", when indeed they are anything but. IMHO, gathering the raw facts and making them clear without frantic emotion seems to me a better process for productive communication.
Yeah, no shit! But does that really help when it comes to having productive dialog and solving problems? My hope is that AI never learns to mimic that aspect of being human. I think Sarah Conner might agree with me on that!
I also use ChatGPT or Google AI for my job to help me research the many codes and standards that might apply to engineering problems I encounter in my line of consulting. I have found that to be a real time saver in making sure my solutions and designs are fully code compliant.
Productive communication? On social media? Don’t hold your breath.
I know, right? The internet was designed to bring people together and to make news and the exchange of information more readily accessible to all. Social media in many ways destroyed all that, brought out the very worst in people, and it happened fast. When some people realized that they could use social media to spread misinformation and disinformation and psychologically manipulate people from behind a curtain of anonymity, it was all over. The internet became a weapon, and by the looks of it so far, it will also corrupt AI if we let it.
Hi Freewill ( and Hal A. Lujah) social media did not create the savaging we experience online. . . people do. And yes, I hear 'echoes' of the guns do not kill-people do argument in my mind.
(Some) People are such a problem. . . to people, animals, and things. On the other-hand, people are the solution to every problem people have too!
For example, I am pretty sure (though I can not prove it) that there is a group of . . . for lack of a better word: "people" in our country using AI to try and start a second civil war between liberals and conservatives. On the other-hand, there should be groups, plural, working with AI to stop any second civil war from occurring in this country!