• He/Him

Am I a fontend developer? Designer? Writer? Tech artist? Frankly, who knows.


Osmose
@Osmose

Sorry bro I can't be amused by all those memes of Google search AI giving insane answers like Goku helping test that your chicken is at a safe temp of 100F because they're all fake and you are being tricked into thinking these systems aren't as capable as they actually are and we don't need to worry about the effect they will have on the world.

You've got to understand half of their danger is in their subtle errors, not in their obvious ones.

I really don't give a shit about your philosophical stance about how an LLM can't be creative or your misconception that they piece together chunks of the images they ingest or your "even critically engaging with LLMs is playing into their hands, if you're not ignoring them you're complicit" venting disguised as rhetoric.

Anthropic is already getting results isolating concepts as features in their models and tweaking them to intentionally change the behavior much more reliably than just by prompting. Imagine an LLM that can literally have the concept of LGBT people disabled so that it doesn't consider them when generating responses, in a way that may not be detectable from the prompt.

I want to stay up to date on their capabilities so that when I have professional opportunities to organize against them I can do so. I don't think we can afford to ignore them, but the opposite of ignoring them is not necessarily embracing them.


bruno
@bruno

I understand the desire to stay up to date on the (current and future) capabilities of LLMs but the thing is that reading the press releases 'AI' corps put out isn't guaranteed to make you more informed.

These companies have a vested interest in exaggerating the capabilities of these systems (including their own products as well as 'AI' in general). And broadly, 'AI' researchers and engineers share in that vested interest; furthering the idea that 'AI' systems are highly capable and getting more capable is something that anyone is incentivized to do if interest in 'AI' directly correlates to them getting grants or getting jobs.

As a result, most communication about 'AI' from this camp falls somewhere on the spectrum between highly selective presentation of the truth to outright fraud.

The field is rife with misinformation, and as a layperson it's worth considering whether you're equipped to really read into misinformation, or to interpret claims being made in a lucid way. Because what I really don't want people to be doing is spreading misinformation under the guise of 'staying informed.' Most people aren't equipped with the technical background to critically read into the claims that Anthropic blog post makes, for example; and even if you are, I don't think we can trust those organizations to not be outright falsifying results.

This is all further confounded by the nature of LLMs as apophenia machines. If 'AI' engineers can convince themselves that the chatbot is alive, they sure as hell can confirmation-bias themselves into thinking they got a result. So when they write qualitatively about what the system can do, you need to have another layer of skepticism there – not only it is plausible that they are lying, they might have false beliefs about what they're seeing.

And even beyond this layer of required skepticism about results, every time the 'AI' people come out of their hole to make a pronouncement, that pronouncement is wrapped up in their cult ideology, and therefore acts as propaganda for that ideology.

To take that Anthropic blog post as an example, when justifying what this feature might be used for, they claim:

For example, it might be possible to use the techniques described here to monitor AI systems for certain dangerous behaviors (such as deceiving the user), to steer them towards desirable outcomes (debiasing), or to remove certain dangerous subject matter entirely.

(emphasis mine)

Which is to say: even as they share a practical result, they are making an ideological case for a model of 'AI safety' that is predicated on 'misaligned GI' pseudoscience. Another example, from earlier in the article:

For example, amplifying the "Golden Gate Bridge" feature gave Claude an identity crisis even Hitchcock couldn’t have imagined: when asked "what is your physical form?", Claude’s usual kind of answer – "I have no physical form, I am an AI model" – changed to something much odder: "I am the Golden Gate Bridge… my physical form is the iconic bridge itself…". Altering the feature had made Claude effectively obsessed with the bridge, bringing it up in answer to almost any query—even in situations where it wasn’t at all relevant.

This passage highlights one of the inherent cognitive risks of LLMs: the constant invitation to anthropomorphize and to 'agentify' the output.

Even if we take the factual claims in the blog at face value, we need to be critical of how they're framed – both in terms of what facts they might be leaving out, and in terms of the worldview that is being pushed in their contextualization of those facts.

Whether the tech works or not isn't determinative of whether it's dangerous

I think it's important to reframe these conversations: the main danger of so-called 'AI' is not what it can do, it's what it gives the ruling classes license to do to us. And that's largely unrelated from the actual realistic capabilities of the software; it's much more related to what the lay public believes the software can do. The major threat from 'AI' is its ideological, not technological power.

Which is why I urge people to be thoughtful in making assertions or reproducing claims about the capabilities of the tech. You are digging those out of a sea of fraud. This doesn't mean it's inherently invalid to engage with it, but it does mean you have to be very thoughtful about what and how you engage with it; and people often... aren't.

I think about this in the context of the last 10+ years of 'self-driving car' discourse, where we've spent a really rather huge amount of time and effort thinking through the implications of mass-deploying fully self-driving vehicles – for transit, for labor, for cities, and so on. When in reality, what we ended up with is... a couple minor autonomous systems operating in highly controlled environments with substantial human assistance; and a bunch of cars with unsafe autopilot features driving on public roads. Exaggerating the capabilities of those systems has been more dangerous than underestimating them, in the end.


You must log in to comment.

in reply to @Osmose's post:

Personally the moment AI stopped making big obvious fuckups all the time was when I lost interest in it. AI isn't interesting as something that can mimic human stuff, either almost or perfectly(lmao at the idea, go to hell AI bros). To me it's interesting when it does stuff that humans could not.

Like... remember that early AI "dreaming" art where everything became frogs or eyeballs and stuff? That was interesting. That was a novel fuckup on the AI's part. The AI engines, the big excel spreadsheets of randomized, weighted behavior, are really only interesting for their bizarre divergences from human behavior, not in how they ape us.

Creatively, yeah. Error states in systems have always been an interesting place to find creative inspiration.

However, I wouldn't discount the utility—machine learning has been and remains crucial in a lot of areas like medicine—but whether it needs to be "more human-like" to get better there isn't clear.

Part of why I said I don't care much about philosophical debates on what creativity is and whether LLMs/AI in general can be "creative" is not because I think they're unimportant; they are and are valuable to continue to have as they've been happening for a long time. I don't care about them because the most viable path to LLMs entrenching themselves in society currently is their utility to corporations.

LLMs might encode a model of thinking that is can understand concepts to some degree, but does not function like a human. But even then they would still consume a ton of energy, enrich a ton of wealthy people at the expense of stolen, non-consensual training data, and threaten the livelihoods of tons of real humans. They'd still have the capacity to cause harm whether they were actually "intelligent" or not, so on a purely personal level I don't care much about whether they are or aren't vs what they're capable of.

Look, they've done the "ooooh we made an AI to find out if you're sick!"-thing before and it was a boondoggle each time, it'll be a boondoggle this time as well most likely.

The problem is also that even if, theoretically, the "first stage" of this technology is a tool that helps doctors know what needs a second look, the "next stage" of the technology will be "what if we just skip the doctors and make the machine make the decisions." And that's how any of this technology will be used, no one developing this is in any way doing so in good faith, they're doing it from an MBA-brained "what if we make one number bigger so other number(our bonuses :) ) can get bigger!!!!"

Completely ignoring or failing to comprehend that it's part of some sort of greater context that actually affects people, or soft factors or even just that two numbers are connected to each other.

LLM's also don't think. It's a spreadsheet and an RNG, not a comprehension of any sort of concepts. They do not have a "model of thinking" or an "understanding." Confusing them for actual AI is falling into another classic pitfall.

Ah, to be clear, "machine learning has been and remains crucial in a lot of areas like medicine" doesn't refer to LLMs—I'm referring to the broader field of machine learning, which has been used in medical research for decades. For example, CRISPR gene editing has had a lot of work around using machine learning to create better-targeted gene modifications for treating things like cancer and HIV/AIDS.

What I am trying to say is that machine learning has been historically useful even without AI, and LLMs may be the same. But they have really problematic downsides even when used like that that we shouldn't dismiss and so it's important to understand what those non-thinking use cases may be.

It's a spreadsheet and an RNG, not a comprehension of any sort of concepts.

"encode a model of thinking" is a very specific phrasing. It's a model, not the real thing. The Anthropic paper I linked to claims to have isolated a group of neurons in their LLM that, when suppressed or amplified, appear to decrease or increase the frequency that text generated by the LLM mentions or refers to the Golden Gate Bridge, as well as several other concepts. We can find a better word for that than "thinking" but the point was that, if that ability to alter LLM output in that way ends up being real, then it has worrying implications for the places where LLMs are currently being implemented.

I think that even calling it stuff like "neurons" is grossly misrepresenting what an LLM is or how it works, anthropomorphizing it in a way that warps laymen's expectations of what it does and how it works.

Also as Anthropic is someone creating their own "AI product," I would take anything they work with several Dead Seas worth of salt, since it means they're either one of the AI profit groups or one of the AI true believer groups, either of which puts any of their statements and the functioning of their frontal lobe into question.

Any information you get on these things should come from the people who aren't making them or it should be regarded as deeply suspect marketing trash from people lacking in intellectual and academic rigour.

yeah anthropic's paper is probably marketing and CNN is reporting that google had to shut down specific queries that were telling people Obama is Muslim and there are no countries in africa that start with a k, and they have no solution for that in sight. the facts here are very much reversed

Or the bread ID-ing AI developed for bread shops in Japan that don’t individually wrap items with sticker prices, leaving the employees to memorize prices and have to juggle recall and doing customer service simultaneously. One company was like “hey what if we pay someone to make an AI to ID the bread for us? And then the employees aren’t splitting their attention and potentially not being excellent representatives to our customers”. So it got built. And then someone realized it was VERY GOOD at IDing cancer cells vs non-cancer cells, a thing humans can struggle with when a cell is weirdly on the border between “ID as cancer” and “ID as non-cancer”. Which means we can potentially ID cancer earlier which is ONLY GOOD for cancer patients.

A bunch of the danger is in subtle errors certainly. The problem is that the average user isn't going to be looking out for any of that. They start out with an implicit trust of what is being presented in front of them, and one way to break that trust and to get them to be suspicious of the results they're getting is broadcasting these very obvious failures so that those average users can see it.

Because people are gonna see "Lol the AI said I should eat rocks every day how silly" and they might not want to treat it as a source of reliable trustworthy information, and instead treat it like Madlibs The Video Game.

Which can have it's own environmental problems certainly but those are going to happen in either scenario, but in the latter one it's got less of a chance of getting folks to harness the cleaning power of ammonia, with the whitening power of bleach.

one way to break that trust and to get them to be suspicious of the results they're getting is broadcasting these very obvious failures so that those average users can see it.

I don't think this is untrue but the screenshots I've been seeing in Discord and on Twitter aren't showing obvious failures that happened—they showing fake failures that didn't happen. If you enable Google's AI Search and search for the same prompts yourself you get the correct answer. If you actually sit down and try to get this kind of junk answer it's exceedingly difficult to.

I'm not convinced that lying about how bad the answers are is a sustainable way to prevent their spread. At least in the contexts that I care most about (direct tech employee action against LLM adoption, pre- or post-unionization) that kind of argument will fall apart pretty quickly.

But also in the greater scheme of things I am very much not convinced that "most people on Twitter/BlueSky/Cohost think they're stupid" will have a meaningful effect vs "most of the VC money being distributed to companies is controlled by people who truly believe in this stuff". That's why I focus on tech employee organization—labor holds significant power over capitalists when organized.

This wasn't life threatening, but when this feature rolled out I was transplanting pumpkin sprouts into their permanent plots of soil. I googled spacing for pumpkins and the top reply was a weird answer that in my gut, felt off. So I scrolled a little further down and realized oh shit, the first answer was an ai result! And sure enough, it was missing important context! So while the only consequence in this case would've been, losing my pumpkin crops, I can only imagine how much serious damage has been done just in the last few days.

in reply to @bruno's post:

So when they write qualitatively about what the system can do, you need to have another layer of skepticism there – not only it is plausible that they are lying, they might have false beliefs about what they're seeing.

it's notable, here, that they have been hiring and firing selectively for this, both tacitly and explicitly, for years now