(person with subject knowledge posts a detailed breakdown of the fact that they asked ChatGPT about something they know about, and ChatGPT gave them some text that scanned grammatically, but was not factually accurate. Person concludes that ChatGPT is "unfit for purpose".)
...look, it's a fucking text generator. It's like doing a deep dive, with fact-checked citations, on the fact that The Onion is posting stories that somebody — gasp! — just made up! The answer can only be a puzzled, then weary, "You missed the point." ChatGPT is a text generator. It generates text.
A lot of the things it's being used for are marketed in ways that are flat-out lies given that all it does is generate text, granted. But you can't really complain that a statistical text generator is unfit for purpose as a search engine, because — drumroll — it's simply not one! It's a fucking text generator!
These things have been around for literal decades. ChatGPT's improvement is, solely, in the verisimilitude of the output. And they achieved that principally by sheer brute-force scale of training data. (That's literally why all these "AI" companies murkily stole all the data they could lay hands on in the first place! Their One Weird Trick only works based on the sheer scale of training data. They couldn't get it to work as well as it does using only 100% uncontentious legitimate data sources! They have admitted as much in public!)