I have been increasingly haunted by the following paradox: the AI techbros, the infuriating cis whıte guys (it IS all cis whıte guys, isn't it, at the top of this field?) who basically demand that the world hand over all its money and resources to build superintelligent computers who will solve every conceivable problem so long as you believe in them hard enough, are clearly motivated by a pile of extremely bad reasons, and yet...they're perplexingly sincere. Yes, they're clearly salivating over the huge stacks of money at stake. Yes, they're wanting to push all their responsibilities and ethics onto computers so they can hoard money with a clear conscience. Yes, they're keen to build machines that regurgitate all their favorite fallacies and bıgotries back at them—like they're trying to build the magic mirror from Snow White, training it up to tell them just what they want to hear from it. One can easily discern other nasty reasons for why Sam Altman and Marc Andreessen and all the rest of these nasty people are ginning up their AI craze.
But I sense another quality in these awful people: sincerity. Maybe I'm totally wrong; maybe they really are purely cynical. But cynicism can coexist with sincerity in the human soul. "Roko's Basilisk"—the premise that the inevitable superintelligent AI will eternally torture anyone who doesn't believe in it hard enough, or something—seems utterly without merit, mere bad-faith Pascal's-wager crap meant to scare people, except...I think that the techbros who believe in it are actually scared. The Basilisk does have power—over themselves. Some Internet jester like me can taunt them with their own contradictions: if they're scared that AI wants to kill us all, then why are the AI tech execs trying to give their machines as much power as possible, pushing them into every industry, including military applications? If they're afraid of Skynet then why are they building Skynet? I suspect that the simplest answer one really is, they believe they're under a compulsion. They do not know how to break away from it, and thus the "Roko's Basilisk" nonsense is really an indirect attempt to warn the rest of the world: "We're too weak NOT to build killer machines."
I think I know exactly where the weakness comes from, too—the actual power in the paralyzing glance of Roko's Basilisk isn't AI, but the overwhelming culture of detachment and learned helplessness that's encouraged in capitalism.
Corporate executives and managers routinely pretend that they're unable to make decisions that aren't hurtful and greedy, even though they're (supposedly) free human beings with the power of discretion and choice over their decisions. One does not need to starve workers. One does not need to make dangerous products. Yet these bad decisions and so many others are said to be not merely necessary, but forced decisions, compelled by inexorable laws of money and markets. Even though money and markets are purely human creations which work the way they do because of arbitrary human definitions, capitalists treat the rules of money as equivalent in force to physical laws—and it's telling that capitalists and their reactionary political beliefs are increasingly in favor of violent enforcement of what they perceive as inflexible laws of biology. The laws of money similarly exist only because they're violently enforced, so it's a clear hint that capitalists see no real distinction between the laws of money and the laws of science. They see both in terms of authoritarian enforcement and compulsion.
Even the wild assertions of replacing human intelligence and creativity with something that's claimed to be exponentially better, something that's guaranteed to solve every insoluble human problem, have a faint whiff of humility about them. Altman, Andreessen, Elon Musk...these are not great thinkers, despite all the hype and the braggadocio about intelligence that dominates their cultish followers, and I'll hazard the guess that in their private moments, Altman and the rest of them are dimly aware that they're not wise or imaginative men. They've become like parents who once had grand dreams of becoming great scholars or artists or sporting legends in childhood, failed at these things, and now compensate by trying to push these ambitions onto their children. Their AI creations are their children, the ones who will surpass them. After all, isn't there some reflected glory to be gained from being the parent of a brilliant child? (Quod vide Maye Musk, who bases her public image on this notion.) And because AI isn't a human child but a machine that can constantly be made bigger and faster, Altman &c. can make themselves feel that their virtual child is guaranteed to succeed and become super smart.
Western society is reeling and crumbling for a lot of good reasons. An important one is this: the static, stagnant values of "the West" can no longer account for the complexity of life on Earth, human and otherwise. Those who are best able to grasp the necessity for comprehensive change have all been shoved to the margins. The leaders and influencers of Western civilization have set their faces against further change: even the best of them want nothing more than preservation of status quo, while the worst of them think that our only hope is a massive purge and forcible return to the simplified values of despotism. In such a condition of intolerable social deadlock, miracles start to look like the only answer, and the AI techbros promise the only sort of miracle that mainstream Western society is primed to accept en masse: technological magic, a new religion in secular clothing.
I honestly think they don't know what else to do, especially because I think the techbros themselves are aware that their own ideas for solving the world's problems are bad ones. Beyond doubt, Musk and Altman and the rest of the crowd regularly boast among themselves about how their vast brains are full of easy and obvious solutions to intractable human problems that everyone else is too "dumb" to accept—solutions involving massacres and nuclear bombs and so forth—but they still feel shame about these ideas; they try to keep that stuff private and hidden, and overcompensate by memorizing a lot of fluffy sales pitches about loving humanity and innovation and puppies and all that. And in the meanwhile, perhaps they hope that their marvellous thinking machines will bail them out, and give them the answers to the questions they themselves can't solve.
(The answer is 42, of course. That's how old our RL father was when he fathered us, and it's how old we were when we resolved on gender transition, so it's a very important number! =D ~Chara)
