peach eating vagus nerve cultist of the house of tool ape


fizbin
@fizbin

If you look up "paperclip maximizer" you'll find loads of references telling you that it's a cautionary analogy or thought experiment about AI.

But there's nothing about the paperclip maximizer scenario that requires AI at all: any algorithm hooked up directly to the means to achieve its goal can go there. (The unthinking magic in "The Sorcerer's Apprentice" is a transport-water maximizer) The only reason "intelligence" enters into it is because we're stupid/arrogant enough to not take threats by non-intelligent agents seriously, so you need to add "AI" into your thought experiment to make it appropriately scary. (What scares you more: violent crime, or your wet bathtub? For the vast majority of us living in the US, your wet bathtub is much, much likelier to kill you)


On the other hand, many of the "proposed solutions" to the paperclip maximizer (or "squiggle maximizer" as some people seem to have renamed it) problem from the kind of philosophy enthusiasts who worry that someone they know will think of Roko's basilisk absolutely require full general AI of a kind we aren't even vaguely close to yet. You can't build basic psychological impulses or restrictions into something that's barely a few hundred linear equations.

So we have the paperclip maximizer threat, but no useful solutions from the people who claim to be doing important philosophizing work on this. One might therefore suspect that either there are examples of this having happened, or that the threat is overblown.

Which brings me to Facebook.

For three years after it was allowed into Myanmar, the Facebook feed of everyone in the country was run purely by the FB algorithm with no moderator intervention (because the company didn't want to spend money to hire Burmese-speaking moderators).

The algorithm maximized engagement, per its design. It boosted scapegoating conspiracy theories until they turned into an actual genocide. It literally resulted in people being burned in service of maximizing engagement.

The paperclip maximizer is here, it's already killed hundreds of times over, we could turn it off if we wanted to, and we won't because its actions are propping up the material success of the same social class of the people who are sitting around Silicon Valley having Very Serious Thoughts about the future of AI and how we're all in mortal danger from spicy autocomplete soon being able to replace vapid opinion piece writers.


You must log in to comment.

in reply to @fizbin's post:

A younger me would be scared about all this talk of AI apocalypses, but now that I'm older and have Seen Things, I see the fear of malicious AI as a deflection in just about every way possible.

They warn about rogue AI wiping out humanity, but have zero issues advancing the field, kind of a "my abortion is the only moral abortion" mentality extended to ML/AI.

If they're in control, they can be "heroes", the one corporation in a sea of bad actors who gets it right while making a profit, because that's the only metric they can think in, and they say it all the time: "Saving the planet can be profitable!"

No. It isn't. It's not possible. It's necessary, but there is no world in which Capitalist profit aligns with a sustainable future.

And they don't want to acknowledge that. They don't want to say that they're doing irreparable harm right now because it's an admission--an indictment of Capitalism as a whole and their complicity in it, so they make up a fictitious future of scary robots and laser guns, because it's clearly so much scarier than an algorithm manipulating people in the manic phase of bipolar disorder to impulse-buy more stuff. They can justify all the collateral damage they're causing along the way to say that they're saving us from extinction, and they want all of us to ignore the exploitation and deaths, to buy into the ends justifying the means, and see them as heroes of a bold new future. They so desperately want to be heroes, and rich heroes at that, historically notable enough to get a snazzy biography written about them with an artsy black-and-white portrait photo on the cover staring intensely at a potential customer from a bookshelf or online store page, ready to offer eternal wisdom to those who also crave enlightenment in the bloodstained cathedral of Capitalism.

Fuck them.