ceɑꞃᵹæꞅꞇ

@ceargaest

[tʃæɑ̯rˠɣæːst]

she/her
twitter.com/ceargaest

linguist & software engineer in Lenapehoking; jewish ancom trans woman.

since twitter's burning gonna try bringing my posts about language stuff and losing my shit over star wars and such here - hi!

username etymology

bosworthtoller.com/5952

ceɑꞃᵹæꞅꞇ @ceargaest11/20/2023, 4:57 PM

Laura @lmichet

Cliff 🦖 Pervocracy @pervocracy11/20/2023, 12:55 PM

wait holdon quick question

if OpenAI is so concerned with AI "alignment" above all else, why do they just shrug and patch in a quick "as a Large Language Model, I cannot answer that question" every time someone finds the thing giving bomb-making directions or being racist

shouldn't that be like a lights-n-sirens emergency for these people??? you devoted your life to making AI not be evil, but then when it is evil on the regular you're just like oopsie, little spot to polish up there?

I guess if your only understanding of "evil" is "trying to take over the world muahahaha", then bad medical advice hardly registers. But then again, if you seriously believe an AI can spontaneously develop the motivation and ability to take over the world with words alone, why are you letting it have unsupervised conversations with random members of the public???

it's a good thing I don't share these guys' religion because if I did, I would think at this rate it's a matter of weeks until I wake up as a large pile of paperclips

#openai #rationalism

Laura @lmichet11/20/2023, 4:49 PM

This page's posts are visible only to users who are logged in.

You must log in to comment.

in reply to @pervocracy's post:

REX @R-E-X11/20/2023, 2:02 PM

cannot be overstated how much "ai ethics" guys are mostly just tech bros trying to hype up their investments by making them seem worthy of alarm

Cliff 🦖 Pervocracy @pervocracy11/20/2023, 2:11 PM

I disagree, actually. I knew some LessWrongers before this was a thing and at least a good chunk of them are true believers. I don't know why, I don't know how someone gets that way, but it's not just financial (in fact a lot of them blew a ton of cash on donations to "AI alignment research"), they really do believe in Evil Computer God.

Not so sure that the people collecting those donations were all true believers, but the phenomenon exists, it isn't just "NFTs are going to the moon"-style hype.

Taylor @sepia-official11/20/2023, 2:19 PM

So I mean, there's different answers to that depending on who you ask: Some AI safety people think that the kind of safety they're concerned about is basically entirely unrelated to the things where they can't stop ChatGPT from being racist or giving bomb-making instructions or whatever, but I think they're a minority. Another answer is that OpenAI has been putting a lot of research effort into preventing these small-scale misalignments but they still don't know how to do it very well because it turns out to be very hard.

But most significantly at the moment, when you say "it doesn't look like OpenAI has been prioritizing safety very much" there are a lot of AI safety people who agree with you, including the OpenAI board of directors! That's why this happened! A very common and fair criticism of AI safety rhetoric up to this point has been that for all people talk about the imminent apocalypse, they seem to prioritize profit over anything safety-related in all situations, and none of the doomsaying actually affects what anyone's doing. But that's changed now, OpenAI has made a big decision on the basis of AI safety concerns that seems pretty clearly counter to their ability to profit. And I don't think anyone at the moment really knows what direction they're going to go from here.

Annie D @ann-arcana11/20/2023, 3:18 PM

The thing is ... people inside AI, with the exception of a handful of actual ethicists who keep getting fired for being right, seem to consistently prioritize alleged safety threats that just ... aren't fucking real.

ChatGPT is not going to take over the world. It IS going to spread insane misinformation and replace real people's jobs with useless noise generators ... but that's never what these "AI alignment" cultists are talking about, because of course they're not.

Even if "AGI" is even possible, it's so incredibly curious that the same people who claim it will kill us all, are the same people who keep fucking building it anyway.

At best, its the product of a fundamentally dissonant mind. At the worst, they're just fucking lying, and its marketing copy to exploit people's tendency to assume a middle ground into overestimating what they actually can do.

Brian Danger Hicks @ckape11/20/2023, 4:04 PM

The only thing that stops a bad guy with a super-intelligent, self-improving AI is a good guy with a super-intelligent self-improving AI.

Cliff 🦖 Pervocracy @pervocracy11/20/2023, 4:25 PM

this is unironically the justification

or more specially the idea is that once an AI hits the Singularity it'll self-improve to infinite power infinitely fast, so what matters is that the good AI is developed first

JP @vectorpoem11/20/2023, 4:54 PM

the short answer is that these people are not tethered to reality, live and work in a bubble of reality-denial sustained (for now) by capital, and they believe the fake dangers are real and the real dangers are fake / don't matter.

Rocket @Rocketknight11/20/2023, 5:09 PM

Hey, I work in AI! (don't yell at me I'll give you juicy tidbits)

Anyway, your other comment was totally correct - OpenAI are based in the Valley and they're absolutely true believers in AGI. Obviously, there are probably some people who are more starry-eyed and some people who are more, uhm, entrepreneurial within a company of that size, but broadly speaking the engineers are there to ascend humanity into a new era and the enormous compensation packages are just a nice bonus.

As for why they don't see ChatGPT being obviously "unaligned" as a huge emergency, the answer is that they basically treat the current generation of models as a trial run. ChatGPT and GPT-4 are not really self-improving; they were each "trained" in one big expensive training run, and modifications to them after that are more like finessing and minor corrections than real boosts to their brainpower. Even if you fully believe in AGI (and, again, they absolutely do) the current crop are very unlikely to spontaneously beget it.

As a result, they're pretty happy to charge subscriptions and run experiments and let users fuck around for now. Some bad medical advice might be dispensed, some race science might be espoused, but it's all hard currency and useful data that will be helpful when it comes time to wrangle God. Present-day harms are kind of beneath them.

Tom @tomotter11/20/2023, 5:10 PM

I think a lot of these guys in AI companies who talk about how dangerous AI is don’t actually buy into their own hype about general AI being just inches away, they’re doing it as a cynical thing to get more buzz. I think most of that variety believe superintelligent AI is possible but they recognize the fact that if it is, it’s not coming from language models.