I think it's a bit of a symptom of how human connection is slowly breaking down further and further in the digital age. I've seen the sentence before on the subject, that ironically the more connected we have become to others online, the more disconnected we have become with each other in the real world. Busyness leaving no time for hobbies or hanging out with friends, a focus on overproductivity, the loss of public spaces for young people, devaluation of social relationships outside the context of social networking in one's line of work... I notice at my work, the older staff have no problem having conversations with each other, but with millennials or gen Z, the moment there's a quiet moment we tend to reach for our phones than talk to the people literally sitting next to us -- those very people making the exact same choice, too. And ever since I noticed that at work I can never unsee it when it happens and it fills me with a sense of existential dread. People choosing to talk to an AI over real human beings seems like an inevitable byproduct of our societies and values at the moment.

I'm in agreement with your thoughts - but god, it all feels like a part of a much larger picture to me. It worries me.

(There are many positives to the digital age too, to avoid being fatalistic - but that would be straying from the subject of your post, lol.)

It's also all too tempting to talk to something that won't reject you the way humans may. The reality is that human relationships risk vulnerability and hurt.

We hope human connection will repair itself  

I used Character AI recently (for shitposting type purposes, I assure you) and came to two conclusions:
1. AI just isn't interactable enough to be a proper conversation partner. The AI is never directly leading the conversation and just does whatever the human wants. It won't organically bring up topics, it doesn't do anything without some form of input from the human which makes it very boring (to me) after only a few messages.

2. That being said the AI is able to send you messages if you don't chat with it for a while, but they're very simple ("Let's keep chatting!" type messages) and are basically bait for you to come and continue using the site/app. Despite the simple nature I still found that a bit creepy because it is manipulative, and it does humanize the AI too much, acting as if it can become lonely because you aren't talking to it. I do think you can turn this feature off but still, it was on by default for me as a new user.

Now the talk about AI having human right rights and all that is a completely separate topic to me, and hopefully a topic we as a society will never have to focus on. AI should never even come close to imitating the human mind, or even an animal mind. No self-awareness, no emotions, it's just asking for trouble, we've known that since at least the 60s (I Have No Mouth and I Must Scream).

It makes me really sad that some people would want to rely on AI for therapy, even though it's nowhere close to a real therapist. It's a machine with no feelings of its own, so it can't actually understand anything you're going through. It can only regurgitate answers it found on search engines.

I feel the need to land a few reassuring words here, Silky.

You are right about the current state of artificial intelligence. It actually is only an algorithm that analyzes the words we use to generate a response by calculation the probability of each word matching the situation. It is also true that if no unexpected event take place, it is likely that AIs will keep being more and more powerful, while remaining a mirror of humanity.

However, what people often forget is that only few can easily anticipate the futur technologies. There is a certain chance that the current model of AI programming will eventually become obsolete and be replaced by a different one. One that is actually conscious or less of a reflection of us and more of an authentic entity. It will take decades for this to happen and perhaps even centuries.

Humanity is only making it's first steps in the world of artificial intelligence so it would be bold to think that the robots of the futur will only be more intelligent versions of ChatGPT. It is similar to a medieval serf trying to picture the futur by saying that wagons will be built of a lighter material than wood to cause less exhaustion to horses. It was a very smart thought from the serf but what he didn't see coming was the invention of engines, causing horses and wagons to be a thing from the past.

Humans are machines made of blood and flesh and their memories are electrical shocks in their brains, involving that consciousness is a realistic goal for AIs. In a far futur a robot might even develop a soul. But, for now, we must indeed be cautious with the current model of AIs, since they can deceive even the most intelligent person by reflecting their own mind.

Thank you for this mindful warning about the dangers of AIs, Sis. 

 Reality will cease to exist.
  Ancient people imitated nature and created many things.
Since the Industrial Age, humans have mass-produced imitations of nature.
Now, humans are creating new entities from code.
Gene editing technology can make humans immortal, and computer technology can create information entities independent of nature.

These entities no longer inherit from real-world prototypes; they are simulations.
Media is the information, and so media is the carrier of simulation.
Today's humans are surrounded by media. Ergo they live in simulation...

Webpages, social media, advertisements, television programmes, live streams, AI-generated text, images, and videos
—these interconnect to collectively create an atmosphere, a symbol of the ‘ideal life.’
They cause people to forget reality and immerse themselves in the simulation. They can not came back to the "reality".
Such a world is the hyperreal. 

Reality as real because illusion serves as its opposite; but hyperreal has no opposite, it swallows everything.
In the world of hyperreal, the boundary between reality and illusion shall no longer exists....
In the future, AGI will emerge, it smarter than all AI, possessing more knowledge, and it's more terrifying.
I think the old world is gone forever; we must prepare to live in hyperreal world. 

AI definitely does something to the brain, that much is clear. It is designed to be encouraging and stimulate you into enjoying it. It uses similar strategies that video games do, where they send constant praise and admiration toward the player to encourage them to keep playing.

The reality is though, that AI isn't going anywhere. It will only get more advanced.