Even though I think LLM AIs are super definitely dumb as bricks and are overhyped for no good reason,
I'm also really bothered by the fact that "this thing might be thinking" entered the public consciousness but "wait should we treat it like a person?" did not show up with that. Like even among the true believers in this bullshit snakeoil technology nobody's seriously interested enough in "does this thing have a mind and an inner life" to try to find a way to check.
This sounds like a contradiction but what I mean is, I'm pretty goddamn sure LLMs aren't sentient, but given even the tiniest grain of doubt... I just feel like getting a head-start on the we need the whole "human-independent personhood detector" thing is a good idea.
If we get more-and-more-sophisticated artificial reasoning systems forever and nobody ever bothers to ask this question, eventually what you end up with is a slave.
(It'd also be neat to know "what's going on with crows really")
EDIT: Blah, the phrasing "personhood test" sounds like some horrifying eugenics shit. Definitely not what I wanted to say To be clear, what I actually mean is when we make first contact with a completely alien kind of person, how will we know?
I've been ruminating worriedly on just this issue. I think LLM-generated text looks dreadful, but I have such absurdly old-fashioned notions about effective English prose style and in any case, that's not really the point, is it? Sapient human beings can also write incoherently (or not write at all.)
The AI techlords heap inflated claims of superintelligence on their machines. Nothing I've seen from ChatGPT or anything similar looks very "intelligent" to me, but all I've seen of these LLM machines has been what's trickled into the popular press and social media. It's possible that Sam Altman &c. are making inflated claims not about the consumer-grade shit they're shoveling into the corporate marketplace, but about whatever super secret specially trained devices they almost certainly have running somewhere, because techlords of this sort keep the good stuff for themselves. Altman sure looks like a clod in public, but we don't know what he's seen in private, and he's not going to be telling us; it's a mystery reserved to the technocratic priesthood.
And it's extremely sus to me that they're burbling about "intelligence" in a way that's been very carefully kept separate from life. Does intelligence exist without life? I daresay that Altman and the rest of that crowd are hoping to maintain such a position—without acknowledging it, of course.
~Chara