The legend goes like this:
As AI art floods the internet, AI will start to scrape its own outputs, becoming dumber and dumber over time.
I get that you hate learning things about the soulless robot threatening to take your job, but people were shitting on and nerd-emoji-ing any replies to the effect of ‘that’s not how AI works’ just because they wanted a victory lap. As for me, I think it’s cruel to give artists false hope and lull them into complacency. I’m writing this post firstly as a short, non-technical dialectic education about AI training, and secondly to explain what I think are the real weaknesses of AI art and the most disruptive direction it could take.
First, let’s have a look at the unspoken assumptions expressed by the OP. In order to blind-like the post I’m talking about, you need to have an imagined mental model of AI training that’s very common among artists, which I’ve taken to calling the Roomba Brain.
The Roomba Brain is a robot unleashed onto the internet by techbros. It (or a part of it) is at all times scraping the internet for art, and allowing itself to be influenced by the pictures that it sees. The techbros check up on it occasionally, but it’s mostly left to its own devices, leaving it free to suck up things that harm it. When prompters ask it to make pictures, they’re asking the latest and greatest Roomba Brain, the one that’s really been enjoying the taste of its own tail recently, just like the mythical serpent. If you’re really gullible, there’s maybe a single digit number of Roombas per Megacorp, and they’re costly to fix.
The Roomba Brain does not exist.
In truth, AI projects interact with data in separate stages:
- People scrape the data.
I’ve worded this very deliberately. Scrapers are written by humans, often bespoke to the project. Doing so means making decisions about where to look and what pictures to accept. If you want to generate art, you might look on your favourite art websites. At this point they can choose to include public datasets like LAION, or outsource tagging work to microworkers in the Global South. - The same people play various games with the data they’ve collected.
This can take as long as needed. Typical operations include:
- Tagging - Deciding what text goes along with each picture, which in turn decides what sort of prompt will summon aspects of the picture.
- Augmentation - ‘Making more data.’ At a very minimum they add a flipped copy of every picture.
- Filtering - Removing bad data. Key point: Any system that can tell AI-generated art apart from human art can be applied here at their leisure. Multiple projects exist that are quite reliable. More perversely, any website that automatically filters or deletes AI art has done this job for them.
They may also decide to go back to step 1.
- They freeze the dataset, then temporarily reserve the hardware to actually train the AI (i.e. update the weights) in a expensive orgy of heat and hardware known as a ‘training run’.
This step creates thousands of backups called ‘checkpoints’ at every stage of training (for example, once every 1000 pictures ‘seen’). It’s often set up to automatically restart from a previous checkpoint if ‘loss’ (a measure of conformity to the dataset) suddenly degrades. At no other point does the AI actually change. - They let people use the AI in its final, fixed form.
The copy served to people is effectively frozen and never learns anything new.
Apart from ensuring the quality of the data, there is an economic reason that training is only done at the third stage. Think of an AI as a large (multi-gigabyte) table of numbers called “weights.”
- To generate AI art means reading from the table, which needs fairly weak hardware.
- To train the AI means writing to the table, which for technical reasons requires more costly hardware, which can be rented hourly from a provider.
Because it’s such a transient and expensive event, some people even set up live public displays of their training runs using tools like Weights and Biases.
You should have a clearer understanding of the way people and data are involved in training image-generating AI. There are multiple really existing systemic defenses against an AI just whimsically lobotomizing itself in the manner suggested by the OP. Furthermore, even outside of Megacorps, there are uncountable individual customized copies of Stable Diffusion trained and stored by hobbyists. All of them are satisfactory to their owners and do not accept new data.
Speaking of hobbyists, the premise of AI outputs being useless for training is not even necessarily true. Around early 2023, hobbyists on Civitai selected thousands of the worst AI outputs to create a ~100kb textual embedding, an “essence of badness” that when applied in reverse caused a sudden leap in generation quality that trickled down to Twitter. That might sound like an edge case, but it emphasizes that the fact that data collection is the main point at which human intent enters the system. It’s too important to leave up to chance.
“Alright, that sucks. What are some actual weaknesses of AI art?”
- In my opinion, the main weakness of AI art is that only non-artists use it.
- Prompters can’t replace trained artists in professional settings due to poor taste and a lack of real control over their tools.
- It’s also detectable using automated tools, and many people can discern it by eye.
“How could things get suddenly worse?”
- In my opinion, the most pressing issue is workflow improvements that invite intermediate or advanced digital artists into the fold in a painless way. Digital paintings that seamlessly blend AI and human strokes could end up being both indistinguishable by automated tools and fit-for-purpose in a way that prompter-created art will never be. Worse come to worst they could sigh and do certain things manually while automating the parts that work.
- Adobe’s Generative Fill is one example of a workflow improvement. It’s an ‘inpainting’ system which regenerates only a specific part of an image.
- Another example is ControlNet, which allows generating images from human-created sketches or lineart, among other things. At the moment, ControlNet is extremely painful to use due to poor integration, but the situation could change fast if an upstart art app adds it to its featureset.
“That fucking sucks. What can I do about it?”
- Crap all over attempts to use AI art in production. This should be instinctive to you already.
- Create and enforce no AI spaces. Despite the Catch 22 that I mentioned above, you can at least keep the attention on human talent.
- Legislate against the use of AI art in commercial projects, and against non-consensual data collection for the purposes of training commercial AI.
- Don't be cowed by jeers of "banning linear algebra,” or “banning math”. Techbros want to make it seem unenforceable, but identifying fixed points such as non-consensual data collection may eventually form the seeds of a legal test.
- Remember this is a labor issue. Luddites do not hate looms, they hate their bosses.
