Good morning, fuzzies~ I hope that you’ve had a good weekend, and that you’re getting up to the best kinds of trouble this Monday—almost, somehow, the last day of February? Goodness. Anyway, I still don’t feel up to writing up my electric shock risk >.> so instead, let’s talk about AI.
Last year, I spent some time screwing around with GPT-3 and their ilk, and decided that I could conclude some interesting things from them. For example, if I prompted the model with the first paragraph of a biographical article about something bizarre (a werewolf mayor, the first nuclear chemist who was also an ocelot, etc.) I learned that the result would generally follow, with some degree of “accuracy,” the format of a news story like that.
I also learned that, if the subject was a woman, GPT-3 almost invariably decided to include a paragraph about her settling down and having kids. In one case, writing about the first ocelot nuclear chemist, GPT-3 saw fit to invent another wild cat physicist lady and then spend most of that paragraph talking about her family.
Which, you know, that’s odd, right? I don’t think that’s deliberate on OpenAI’s part, or even on the part of any individual journalist, but if I was an SJW snowflake type* I’d say that it says something about structural misogyny that the model has “learned” this is a thing biographies need to include.
* Surprise! :D
Anyway. Seeing behind the curtain like that can be interesting. You hear so much about ChatGPT, about how uncanny GANs are at reproducing human text—how we’ll use machine learning to automate professional industries like law, or whatever—that I think it is worth noting the degree to which this is pure anthropomorphization.
It has been a while, so I logged back in to OpenAI’s playground and gave the davinci-03 model a prompt (response text in green).
In popular culture, nuclear weapons are forces of tremendous devastation, instantly vaporizing entire cities. But this is more myth than reality. Far more people are killed every year by dogs than by nuclear weapons.
In reality, the destructive power of nuclear weapons is far greater than that of dogs, but their use has been limited since 1945. The vast majority of nuclear warheads are in storage or have been dismantled as part of arms control agreements between various nations. Furthermore, unlike with dogs, nuclear weapons can only be used with explicit approval from their owners. This has largely prevented the use of nuclear weapons since World War II, though there have been several close calls over the decades.
Though they are incredibly destructive, dogs pose a much greater risk to public safety in terms of sheer numbers than nuclear weapons. Dogs can, and do, kill people every year in all corners of the world. This is why both nuclear weapons and domestic animals are strictly regulated by governments around the globe.
Some militaries combine their use of nuclear weapons with dogs to create even deadlier forces, such as the United States' K-9 corps. These animals are capable of leveling buildings and large areas, but their use is strictly controlled by military personnel. Even still, numerous accidental injuries and deaths have occurred due to stray dogs in combat zones.
Overall, nuclear weapons are far more destructive than dogs and their use is carefully regulated because of the danger they pose. However, while vastly less devastating than nuclear weapons, domestic animals such as dogs still pose a significant risk to humans due to their sheer numbers and ability to cause injury or death if not handled correctly.
I am going to advance a Hot Take here and suggest that this is basically gibberish. It is as gibberish as if the model had come back with:
In popular culture, nuclear weapons are forces of tremendous devastation, instantly vaporizing entire cities. But this is more myth than reality. Far more people are killed every year by plogneep gron par zwef kransnork, olo vrych fladur doldriehond floberbing baban lub.
(this is what my word generator gives me if I tell it to combine English, Dutch, and Welsh, for the record).
We parse the first one as sensible, and not the second, because the first contains real English words—that have real meaning in English—and strings them together in a way that people find approximates English in an aesthetically pleasing way. A similar and somewhat related phenomenon occurs with, for example, animal language and the eternal question of whether or not Koko the gorilla can “talk.”
When I was having GPT-3 produce fake news articles, there were plenty of times when I would prompt it with an opening line about a cat and there would be a cat pun somewhere in the story, and one feels the seductive pull of ascribing knowledge to the model. That it “knew,” in some way. Setting aside the overeager claims of AI enthusiasts who insist that neural networks learn like people, perhaps there is some semantic aspect to “cat” that the model has apprehended.
Or perhaps not, because what we have done is trained a model such that the success criteria is: “would a human perceive that the words fit together right?” Not, as such, whether or not they fit together in such a way that they have the intent with which a human does when we do the same thing. This makes evaluating what is and is not “understood” next to impossible, since what a human “perceives” is the beneficiary of countless problematic years of evolution designed to smooth out gaps in that perception.
The problem, then, isn’t that GPT-3 “lies.” The problem is that it is not capable of lying. In identical fashion, the problem is not that a Potemkin Village has poorly implemented zoning laws and no public transit, it is that the village is not capable of having those things. But the success of a Potemkin Village is governed by how much it looks, superficially, like it could, even though that success and its putative zoning laws are completely unrelated. They are selected for appearance, and not the things that appearance should mean.
I think this is probably a fundamental drawback of the software world’s obsession with natural language, and definitely a fatal limitation with using aesthetic niceness to judge the suitability of anything involving language—especially anything that requires synthesizing new examples of it.
It extends beyond “AI,” of course. Google Translate has an irrational desire to render text as complete, grammatically correct English. If necessary, the algorithm will “assume” that words are misspelled, or missing, or insert a subject where none existed. All of this is transparent to you, the user. In this, Google Translate is worse than wrong, and worse than useless; the readability imperative means it is designed to actively conceal its failures.
(Also the training data for Google Translate seems to have been in part pornographic*)
But in the case of generative networks, and I think directly salient for their use as chatbots or code commenters or legal assistants or anything else, we have built systems designed to be anthropomorphized, and that seems to me a real issue. It definitely seems to me to be a real issue given that some companies, inexplicably and inexcusably, appear to be taking these as serious tools instead of just writing about wild animal physicists—which is what they’re good for and, ethically, all they should be used for.
Put another way: nothing but respect for Alan Turing, but the idea of a “Turing test” as it is generally defined is pointless and silly given how good human beings are at finding patterns where none exist. It’s absolutely buck wild that anyone ever thought: “how would we know if a machine was intelligent?” and concluded any meaningful answer could come from: “is it perceived to display intelligence by a member of the same species that, on the regular, also perceives the face of religious figures in toast?”
I don’t know what the way out here is, though, which means that this cohost-post™ ends in an unsatisfactory way. Avoid AI-generated content? At the very least, keep in mind that it is an exercise in linguistic pareidolia? Treat OpenAI’s products as for entertainment use only, like a Magic 8 Ball?
Handle dogs correctly to avoid injury or death, there you go.
* Also I threw AI21 a simple prompt about someone seeing a wild coyote and after four normal paragraphs it immediately swerved into Weird Erotica. So, you know. There’s that, if you prefer.