i wonder if i could fit GPT-2 into the 11gb of my current graphics card. it was the last deep learning text generation tool useful for creative purposes (as the last one you could download and finetune on a chosen body of text), and the last one useless for commercial purposes. it babbles and it takes a lot of work to get coherent text out of it, and even more work and curation to get interesting text out of it. no "prompt engineer" possesses the necessary skills to use it for anything.
its descendants may have spurred the AI takeover, but as an evolutionary ancestor, it is so much more primitive that it could never have taken off for anything on its own. i don't remember anyone calling it "AI" because the assertion is flatly ridiculous to anyone using it or looking at its output. you cannot talk to it.
in function it's the ultimate evolution of those "write text using autocorrect algorithms" things that were briefly popular in the late 2010s.
my last experiments in Colab look like they were set to download the 1558M model, but that couldn't be finetuned on the GPUs available in Colab at the time, and I know I only worked with a finetuned model. I think maybe I experimented with that, failed, and stopped.
i wonder if i could finetune a 1558M on the 24GB low-end Nvidia tensor core GPUs on an EC2 instance these days...
i will NEVER stop being pissed that i was doing ML art back when all you had were experimental RNNs and LSTMs and all these Fuckers soiled the entire medium by dumping billions of dollars into it for commercial reasons
