i claim i want to make things and then i dont make things.
pfp by datcravat
Really good article on AI chatbots and psychics: https://softwarecrisis.dev/letters/llmentalist/
I especially like the parts about "smart" people potentially becoming even bigger evangelists when they do get fooled by it, because boy howdy does that ring true
blown away by this. even calling it a fantastic metaphor feels like selling it short. it clicks perfectly with a persistent nagging feeling i've had about how the output is generic but confident and how that must be important somehow
One thing that I don't know, which I would like to know is whether being aware of the mechanics of the psychic con dissuades people from their belief.
It seems to matter very little to people who are suckered by convincing text generation - which I recall learning was also true when people were getting Too Into ELIZA.
Definitely a good question. I could absolutely see someone who's aware of what cold reading is get thrown completely off-kilter by a read that hits a little too close to home and kinda spiral from there, which feels very similar to programmers being shocked that it can refrigerate basic python, but neither of those are empirical
my wager, and this is totally speculation, is that this is mostly an indicator that people don't really truly understand the underlying mechanisms, which especially in LLM type things is very common, since it's functionally an entire vocation unto itself. it's much easier to comprehend "the psychic says stuff that's probably true and waits for me to flinch" than it is to comprehend a token cloud and the inner workings of the model. and i think there's a lot of smoke & mirrors in the software industry about what people actually understand vs what they know how to use.
Wow yeah this perspective is great! Maybe it can shake some of my “AI evangelist” friends out of their illusion
this is incredible and like. yeah. it really is the psychic's con huh
