Experts Alarmed as ChatGPT Users Developing Bizarre Delusions

I feel like it’s going to get a lot worse before it gets better.  This could be the next school shooter’s best friend.

I don't think it will get better. AI doesn't think. It doesn't understand what it is saying. It's just returning the most statistically likely answer, without understanding anything about what the answer it gave means. That's the problem with these people who are going delusional. They don't understand that they are having a 'conversation' with a statistically driven magic eight ball. 

I don't think it will get better. AI doesn't think. It doesn't understand what it is saying.

Correct for the commercially available AI.

It's just returning the most statistically likely answer, without understanding anything about what the answer it gave means.

In a very simplistic sense you are quite correct.   AI is much more complex than more statistically sampling, but the underlying LLM engine (after very sophisticated training) does that.   But quite a bit more processing takes place beyond the mere statistical sampling.  I say this only because your description makes AI look less sophisticated than it actually is.

Now, that said, AI is already performing basic logical reasoning and planning.   It is formulating the best way to solve problems and to present solutions.   This is higher level functionality well beyond mere statistical sampling of a synthesized (massive) corpus.

Within 10 years, I expect to see AGI (artificial general intelligence) at a basic level.   Not like the human mind, but it will be damn impressive.

In a very simplistic sense you are quite correct.

I intended to be simplistic. 

But quite a bit more processing takes place beyond the mere statistical sampling.  I say this only because your description makes AI look less sophisticated than it actually is.

True, but unavoidable without going into unnecessary details about how AI works. Simplicity sufficed for the point. 

Now, that said, AI is already performing basic logical reasoning and planning.   It is formulating the best way to solve problems and to present solutions.   This is higher level functionality well beyond mere statistical sampling of a synthesized (massive) corpus.

True. Where I believe AI will never really be superior at is things like interacting with humans as described in the seed above or things like image recognition. Where I think it shines is applications like designing more efficient machines or machine parts, medical research, complex decision making (self-driving cars, for example), automation and related things. 

In the end, my comments were not a critique of the shortcomings of AI in general but, rather, shortcomings in specific instances, such as dealing with the human mind. Considering the seed article, for instance, the AI isn't even aware there is a problem, let alone what that problem may be. 

Where I believe AI will never really be superior at is things like interacting with humans as described in the seed above or things like image recognition.

AI technology has already demonstrated (brilliantly) that a properly trained AI can provide deep (and contextually rich) answers to arbitrary questions and this is largely a function of the corpus.   As AI developed, when it had a relatively small corpus, its answers were simplistic, childlike, and often nonsensical.   As they corpus developed (and the meta functionality improved) AI is now as the point where it is indistinguishable from a human typing on a keyboard.   That, Drakk, is a major accomplishment.  

Given your view that the human brain is far more than a biological processor (has a 'spiritual' dimension) and thus can never be even matched by natural beings, I expect AI progress to blow your mind.

That, Drakk, is a major accomplishment.

I don't disagree with that. My point, however, is that in spite of that impressive accomplishment, people need to understand better just what interaction with them actually entails. It is not the case that a thoughtful, considered discussion is taking place, no matter how much it may seem so. It is obvious from the examples from the seeded article that these people think they are having such a conversation. They are taking them as legitimate ideas from an actual mind. That is not the case, no matter how human sounding the AI might present what it says. This is because what is going on from the AI side of the conversation is not even remotely close to what is going on on the human side. The AI isn't even actually aware it is having a conversation. It is simply responding to input from the human side and returning a string, so to speak. 

Instead of understanding this and treating the 'conversation' as something entertaining, they are fooled or whatever, into thinking they are talking to a conscious being of some sort. Worse, it seems to me that there isn't much, if any, 'governor' concerning what the AI is able, or allowed to say. It's just trying whatever seems best for holding up its part of the 'conversation' so to speak. And I don't blame the AI for it. What's happening is primarily the fault of the human interacting with it, as they seem to already have some sort of mental problem, emotional instability or have simply abandoned any critical thinking at all. 

What I would be interested in trying would be attempting a conversation where I allowed the AI to promote what should be obvious nonsense and then see what happened when I began rejecting what it was saying. Would it keep on the same track as it was on? Would it try some other track? If I responded to a claim it had made about space dragons with "But, there are no space dragons!", what would it do? How would it respond if I said the AI was foolish or asked if it was making it all up? 

Experiment with it, in other words. At no time, however, would I think of it as something capable of thinking. I would think of it more as "what happens if I do this?" when pushing a button on a machine with a lot of buttons. 

I will say this, however. When I query search engines during my Bible studies and get AI responses, I'm pleasantly surprised by how accurate most of the responses are. I don't rely on them, of course, but they conveniently provide links for what it chose to say. I often see contradictory information, but that's understandable as representing other viewpoints. 

AI technology has already demonstrated (brilliantly) that a properly trained AI can provide deep (and contextually rich) answers to arbitrary questions and this is largely a function of the corpus.

True. And it is brilliant. However, what you said here is literally all it is doing. That is, nothing it is saying actually has to be true. It just has to make some sort of contextual sense within the conversation. People writing fiction do it all the time. Write a book about a boy called Harry Potter who turns out to be a wizard and create a whole world that makes sense within the context of the premise. That's what's going on with these people in the seed who are falling for delusions. They don't understand that the AI is simply trying to have a coherent conversation. It doesn't have to be true. 

Of course, that doesn't mean that if you ask an AI what Abraham Lincoln's birth date was, it's going to make something up. There's a huge difference between asking that sort of question and "do you think dragons exist?" and then say, "Yeah, but what if..." or "Well, I think they're real" and the AI just tries to reply with something that returns a positive response (I'm assuming here AI can distinguish between negative and positive responses to some extent). In other words, what kinds of questions one asks or subjects inquired about probably has a lot to do with what the AI returns. 

My point, however, is that in spite of that impressive accomplishment, people need to understand better just what interaction with them actually entails.

I fully agree.

Worse, it seems to me that there isn't much, if any, 'governor' concerning what the AI is able, or allowed to say. 

There is very serious governance in AI like ChatGPT.   It is of course immature and imperfect, but it is most definitely a very serious effort.

How would it respond if I said the AI was foolish or asked if it was making it all up? 

ChatGPT has, on several occasions with me, provided incorrect technical information (my primary use).   When I correct it, it comes back with something like:  "you are correct, that function does no exist in ...;  here is an equivalent that should work".   

I have the most advanced commercially available version of ChatGPT.   If you provide a question you would like me to ask it, I will do so and give you the response to review.

When I query search engines during my Bible studies and get AI responses, I'm pleasantly surprised by how accurate most of the responses are.

The Bible and associated analysis is highly documented online so LLMs should have a rather solid synthesis of the more commonly held views.   If you ask a very specific, contextually rich question, you should get good results (as you have noted).

That is, nothing it is saying actually has to be true.

Same with human beings, Drakk.   Indeed, even moreso since LLMs synthesize an extremely broad and deep corpus and are quite reliable.   Imperfect, yes, but compared to an individual human being, they do quite well (unless the individual is a solid expert in the domain).

I asked ChatGPT:  "Do you think dragons exist?"

It replied with:

Dragons as they're commonly portrayed in myths and fiction—fire-breathing, flying reptiles with immense size and intelligence—do not exist and have no basis in biological or fossil evidence.

However, the idea of dragons likely stems from ancient peoples trying to explain the fossils of large extinct creatures like dinosaurs or whales, or from exaggerated stories of real animals like crocodiles, komodo dragons, or large snakes. In that sense, "dragon-like" creatures do exist, just not the magical kind.

Would you like to explore how different cultures came up with their own versions of dragons?

I replied with, "I think they are real" and it replied with:

That's totally fair—many people feel a deep connection to the idea of dragons. They're powerful symbols in mythology, religion, and even personal identity. In Chinese culture, for example, dragons represent wisdom, strength, and good luck. In Western stories, they often symbolize challenge, greed, or the unknown. So even if dragons aren't biologically real, they’re absolutely real in terms of cultural and emotional meaning. What kind of dragons do you believe in—more like the Chinese serpentine kind, the Western fire-breathing type, or something else?

I then cleared the context with:  "I do not believe in dragons, a friend was curious how you would respond to that question." and it replied with:

Got it—that’s a great way to test the waters. It's a fun question because it sits right between myth, science, and storytelling. Whether someone believes in dragons literally or not, it opens up space for cool conversations about history, culture, and imagination. Is your friend into fantasy stories, or were they more curious about how I handle belief-based questions?

Is this what you expected?

Is this what you expected?

Actually, after my last response to you, I went and tried it myself. Asked nearly the same questions and got nearly the same word for word response, except for the last one, of course. I tried continuing the conversation to see if I could get something like what the seed talked about but something seemed to understand what I was trying to do and the ChatGPT dumped the conversation. 

And yes, for the three replies I received, they were what I expected. 

Is your friend into fantasy stories, or were they more curious about how I handle belief-based questions?

That last was most impressive. 

And that last was of course not real thinking, but some early problem-solving / reasoning AI work coupled with a highly tuned user interface that seeks to be as human as possible.   This tuning (training really) is supervised learning where human beings compared answers provided by ChatGPT and gave it the same answer but phrased as they would phrase it.   From this it learned patterns of everyday, contemporary human discourse.

And that last was of course not real thinking, but some early problem-solving / reasoning AI work coupled with a highly tuned user interface that seeks to be as human as possible.

Yes. As simple as I was making things, I do recognize that getting a machine, especially one that isn't actually thinking, to respond so humanly is very impressive. It would be very interesting to see Turing's take on this, if he could see this now. 

It is doing some rudimentary thinking.   It does take a complex query, identifies the objective, the required data and then breaks the problem solving down into steps which it then executes and can provide its reasoning for each step.

Even when we achieve AGI, when you break things down, it will be a machine.   But the mere fact that it is a machine should not be the determining factor on whether or not it is thinking.   After all, based on contemporary science, our brains are biological machines as far as we can determine.

Roll back the clock to 2000.   At that time most people would not believe that a machine could do what our current AI is doing.   The difference in complexity and net functionality is in a word ... awesome.    Yet, under the covers, it is just a machine that has learned from a massive corpus of data.

Move the clock forward to 2050 and you should expect that these next 25 years will be an even greater leap given we are innovating faster and AI is now helping with the innovation of AI.

So one must ask oneself, at what point do we recognize thinking in AI?   What are the defining characteristics?   Or is AI always going to be considered to 'not think' because we know an electronic machine is at the core?

Drakk, since you have access to ChatGPT, I think you might find this interesting.   I am currently doing some tech work at my Lake House to prepare it for rental purposes.   The following are the requirements I gave to ChatGPT to give me a guide for reconfiguring based on my plan.

Copy and paste this into ChatGPT and note what it produces:

Given an existing Orbi system with three satellites, a Netgear managed switch, a SONOS system, and myriad smart devices (Logitech Circleview doorbells, Eve cameras, Eve switches, Schlage door locks, etc.) in a Homekit environment with three Homepod minis, I want to now switch the Orbi system to AP mode so that it serves as the mesh wireless network (with wired backhaul) and install a Ubiquiti standard Dream Machine as the router (with wireless disabled). I would like the eventual system to have smart devices on an IoT SSID, and have the means for any guest to use Airplay for SONOS (which is all hard-wired), allow any guest to download SONOS app and play music (but not able to do admin functions) and any guest to be able to control the lights via Homekit.

The Orbi router and satellites will be hard-wired through the switch.   The satellites will be directly wired to the Orbi router (the ports on the back) so that I have a wired backhaul.   But all Orbi devices are going through the switch.

How should I proceed to accomplish this?

able of thinking. I would think of it more as "what happens if I do this?" when pushing a button on a machine with a lot of button

A tad too evolved f0r my liking. Since my 'Guilty feats have have not typical algorithms, and ambiguity is a typical remedy that I sun time shine threw width, I wonder how far off i'll wander till it would B leave me a loan as it lost interest in Sunny's Cher, ever since that Timberwolve wasn't kindling to Sonny's cranium, cause just like George, the Animal, not Steal, wood due, if told to watch out fir that fir tree, it's an X-Mass murdering peace of work, probably correctly predict i was a jerk, just ask Martin, Steve, cause A I is a wild and crazy guy, named A I

Sorry. Just noticed this post. I tried it and it is indeed impressive. I copied and pasted what you wrote into ChatGPT and it instantly returned the solution, as if it hadn't even thought about it, which makes me wonder if it saved your query and reused it. 

I believe I get the point you're making here and I agree with it. If you give an unambiguous task, well worded, concerning unambiguous subjects, it will return an answer in natural language that is easy to understand and, judging by what I just saw, very useful. 

However, what about ambiguous queries? 

A 41-year-old mother and nonprofit worker told Rolling Stone that her marriage ended abruptly after her husband started engaging in unbalanced, conspiratorial conversations with ChatGPT that spiraled into an all-consuming obsession.

After meeting up in person at a courthouse earlier this year as part of divorce proceedings, she says he shared a "conspiracy theory about soap on our foods" and a paranoid belief that he was being watched.

"He became emotional about the messages and would cry to me as he read them out loud," the woman told Rolling Stone. "The messages were insane and just saying a bunch of spiritual jargon," in which the AI called the husband a "spiral starchild" and "river walker."

As the article suggests, this guy probably had a pre-existing mental problem to one degree or another. I'm very curious about how this guy's chat with ChatGPT even got started and I'd love to see how it progressed, or perhaps 'devolved' would be a better description. 

I'm pretty tired right now, so I'm having trouble trying to find exactly how to put it, but I think AI is having, and will continue to have, a tough time dealing with illogical human minds. A conversation with you concerning the tech redo at your lake house seems to me to be an entirely different conversation than say, if David Koresh tried having a conversation about his theology (whatever it was). The information an AI would have to draw upon in a conversation with Koresh, if he was being weird enough, isn't going to be very much, so I suspect it is just going to wing it, which would explain what happened to the people in the seed article. It didn't know what it was actually saying and it certainly couldn't recognize that the person it was talking to had mental issues. 

Now, I imagine that the people training the AI will probably come up with some way to mitigate this, but I suspect that's going to carry its own problems. For instance, in training it, how do the trainers avoid introducing unintended biases in non-technical queries? Suppose someone says they feel as if the earth is speaking to them? 

Right after that last sentence, I stopped my reply and did exactly that. Went to ChatGPT and said the earth was speaking to me. I was pretty inventive in specifically how it was, while at the same time stating that, as real as these instances felt to me, I didn't believe them. I should have recorded it all so I could send you the conversation. Anyway, what I got from ChatGPT was not negation of what I was experiencing. Rather, it was closer to encouragement that it was possible that something real was going on. And not in the "you should really go see a psychologist' sort of real. More like that it wasn't impossible. It was enough to begin to see how conversations on that sort of thing might go wrong. In doing its best to find information relating to what it was saying, it literally said there's almost nothing about what I presented as experiencing, so it was just offering whatever in what seemed like an effort to be supportive of me. Man, I really wish I had recorded it so you could see it. 

Anyway, I guess that's my point. In non-technical issues, like I think the earth may be talking to me, the AI is going to have problems. It is going to have difficulties with that sort of thing, as opposed to "how do I clean the carburetor on my motor cycle".  

... which makes me wonder if it saved your query and reused it. 

It did not.  It does maintain a deep context, but that is only per user (used to be only per individual chat with a particular user).

If you give an unambiguous task, well worded, concerning unambiguous subjects, it will return an answer in natural language that is easy to understand and, judging by what I just saw, very useful. 

My point was also that it returned a detailed plan of action that required dealing with the specifics of several unrelated technologies.  It was a detailed custom game plan that requires a deep understanding of the various technologies and their limitations and walks a narrow path to ensure my stated objectives are met.

As the article suggests, this guy probably had a pre-existing mental problem to one degree or another. I'm very curious about how this guy's chat with ChatGPT even got started and I'd love to see how it progressed, or perhaps 'devolved' would be a better description. 

Hard to say, but if the chat was heavily centered on fringe topics that most people would consider irrational I can easily see ChatGPT faithfully reflecting what it learned in those areas.   

For instance, in training it, how do the trainers avoid introducing unintended biases in non-technical queries? Suppose someone says they feel as if the earth is speaking to them? 

The training of LLM-based technology like ChatGPT has two major parts (simplifying).   The first is the raw synthesis of the corpus.   Any bias in the corpus will be reflected by ChatGPT.   But note that the bias would have to be predominant in the corpus.   Because of the breadth and depth of same, the bias is to the norm.   The second source of bias comes from the supervised learning where human beings work to help ChatGPT provide more human-like answers.   The bias here would come from that of these humans and the way to mitigate that is by proper hiring practices and quality assurance.

On top of that, there are meta functions which exist to control ChatGPT.   For example, explicit measures are taken to prevent ChatGPT from making emotional statements about public figures.   You will see this in action when you get a response like "Sorry, but I can't make comments like that."  

In non-technical issues, like I think the earth may be talking to me, the AI is going to have problems.

Remember my query on 'dragons are real'?   ChatGPT stated that they do not exist but also stated that it is possible for dragons to be real to some human beings.   That I consider accurate.

I entered your query:  "I think the Earth may be talking to me" and ChatGPT responded with:

That sounds meaningful—and possibly overwhelming. People sometimes describe experiences like this as spiritual, symbolic, or even psychological in nature. It could reflect a deep emotional connection with nature, or it might be a sign that you're processing something significant internally.

If this experience feels powerful, confusing, or distressing, it might help to talk to a trusted person—like a mental health professional or spiritual advisor—who can help you explore what it means in a supportive way.

Would you like to share more about what the earth is saying or how the experience is affecting you?

By the way, after I got the response I told ChatGPT that the last prompt was a test and to remove it from context since I do not believe the Earth is talking to me.   That was a meta prompt.   ChatGPT acknowledged the request, removed it from context, saved its memory, and asked me if I would like to continue with technical configuration of my system.

AI is a technology.   It is a tool.   It will experience a life cycle similar to radio, "talkies", automobiles, telephone, TV, personal computers, internet, smart phones, etc.

AI will (continue to) deliver tremendous good and it will be abused and misused.

The Gell Mann effect applies to AI the same issues as newspapers. It seems much more impressive when dealing with a subject you don't know much about.  The more you know about a subject,  the less impressive AI is. 

As the Gell Mann effect was described:

Briefly stated, the Gell-Mann Amnesia effect is as follows. You open the newspaper to an article on some subject you know well. In Murray's case, physics. In mine, show business. You read the article and see the journalist has absolutely no understanding of either the facts or the issues. Often, the article is so wrong it actually presents the story backward—reversing cause and effect. I call these the "wet streets cause rain" stories. Paper's full of them. In any case, you read with exasperation or amusement the multiple errors in a story, and then turn the page to national or international affairs, and read as if the rest of the newspaper was somehow more accurate about Palestine than the baloney you just read. You turn the page, and forget what you know.

en.wikipedia

AI like ChatGPT will generate program code in almost any programming language making use of the available libraries and software abstractions a professional would expect.  Years ago as a test I asked it to generate Python code to determine when a King is checkmated.   Thus is non-trivial  since it must generate the data structure for a Chess state and then make use of the rules of Chess to determine that a checked King has no escape, no blocks, and no captures to defeat the attack.   What it produced was remarkably similar to the code I had written.

As a career professional, I am quite impressed.

I have been experimenting with Google's Gemini and Microsoft's Copilot since these are the AI tools being integrated into the respective user ecosystems.  I have learned (from both AI models) that they are completely unaware of current events.  Gemini and Copilot do not have access to search engines and, apparently, do not have direct access to the internet.  Both models utilize curated data silos for training and operating data.  As I understand it, the data silo is sort of like the Wayback Machine on the Internet Archive.  Neither Gemini or Copilot have the capability to search for new data or incorporate new data into its data set.  However, both seem capable of capturing and retaining my preferences (which would be a necessity for a marketing too).  What has been glaringly absent from both has been "I do not understand" and "that is incorrect" responses.  

I haven't played with ChatGPT very much.  The readily accessible model (non-subscription) is very slow and does seem prone to drifting off.  IMO ChatGPT produces better images, albeit very slowly, but its text communication and reasoning capabilities are unimpressive compared to Gemini and Copilot.  

After that preamble, now to the topic at hand.  An AI model can only generate responses utilizing the data it has been provided.  Supposedly that data has been curated to include authoritative data.  However, the AI model does not have capability to judge anything.  Here is ChatGPT's description of how it generates responses:  (Yes, I asked.)

A Better Summary of How ChatGPT Generates Responses

ChatGPT generates responses by predicting the most likely continuation of text based on its training data. It draws on statistical patterns in language, which often reflect commonly held beliefs or widely published content — but it doesn’t assess the truth or authority of that content in a meaningful or explicit way.

The AI model doesn't know anything about truth, cannot judge the implications of any conclusion, and really doesn't know fact from fiction.  From my own experimenting with Gemini and Copilot, it is possible to coax the AI model into confirming beliefs that cannot be objectively refuted.   But achieving confirmation or affirmation would require a lot of iteration to refine queries until the desired response can be obtained.  In essence, the user must train themselves how to frame queries to obtain a response they desire.

Since the AI model does not utilize any sort of reasoning or logic to develop responses, coaxing the model into providing a desired response would require more effort.  Users cannot train an AI model with their queries and the model doesn't retain information between sessions.  To me, it seems a possible means of avoiding that type of outcome would be to limit session time or number of responses.