Really good article on AI chatbots and psychics: https://softwarecrisis.dev/letters/llmentalist/
I especially like the parts about "smart" people potentially becoming even bigger evangelists when they do get fooled by it, because boy howdy does that ring true
For the past couple of months, I’ve been working on an idea that I think explains the mechanism of this intelligence illusion.
I now believe that there is even less intelligence and reasoning in these LLMs than I thought before.
Many of the proposed use cases now look like borderline fraudulent pseudoscience to me.
Finally people are starting to get it
Read this one all the way through, it's great.
I have one criticism: The author gives too much benefit of the doubt to corporations trying to sell this crap. While I think the research behind LLMs was very sincere before any of these companies saw the glimmer of a product, ever since productization the marketing and development has been very dishonest.
