Alice Over Zero

I don't hate the game upscale stuff, but I'm really negative on it all in general. It's a bunch of rather flawed techniques that sure do look like they broke boundaries because of scale, but aren't doing anything fundamentally new.

the thing about use cases is that they always have a particular social-political context surrounding them. you can't evaluate the current wave of LLMs in a vacuum without looking at how OpenAI, Google, MS, Facebook, etc operate. everything about how they work is determined by how those companies built and deploy them, and what their larger strategic goals are.

so whenever people are trying to imagine a socially non-toxic version of them, i always want to know - is this still OpenAI et al that are creating this hypothetical product? it is never a question of "pure technology" because technology is always deployed in particular contexts for particular aims.

its very good at vomiting milquetoast garbage you can look at and go "I can do better".

genuinely one of the main reasons i hope openai crashes and burns as soon as possible is that all of the air is being sucked out of the room by this. this used to be a field where people attempted to make theoretical inroads into improving the performance of, like, enhancements over general statistical classification and prediction methods, essentially. now it's instead "what if google and microsoft each put 20 billion dollars into building a huge computer that can make an enormous black box and then we pretended that it solved every problem, especially the ones it's really bad at, because so much money was put into it that it's business critical to make fetch happen". research has been completely overshadowed by Product and the Product isn't even good for anything

that it additionally involves these companies doing things that would be illegal if i did them, openly marketing the results towards unsuitable and/or immoral goals, and suppressing any discussion of actual ethical problems in favour of "but what if my cool big computer suddenly gains the power of god, somehow, which is inevitable btw" certainly doesn't improve things

Utilizing public domain data from government entities to do scientific modeling is about the only legitimate use case I can think of that doesn't have ethical concerns. Machine learning is already being applied in weather, physics, and astronomy modeling, and they are using the exact same data you would use if you chose a different technique to do the modeling.

I think that there is a legitimate use for LLMs in finding bathroom faucets. https://www.dbreunig.com/2023/09/26/faucet-finder.html

Put another way, the “generative” part of generative AI is the part that is obviously plagiarism. But “take this all this data I have, compute embeddings for it, then let me query for similarity within my own corpus” seems legit to me.

I guess there’s a question whether or not one “needs” all of ChatGPT to do a good job on this kind of task, or whether some other tech might do just as good a job without having digested the entire internet — even some other tech that isn’t an LLM at all. Certainly for textual similarity on a very large corpus, algorithms like NNMF have been around for 2 decades and aren’t giant electricity and water hogs. But for computing embeddings from images? I dunno.