because i like owning my toys. i'm so used to giant corporations offering us free storage space and compuring power that it feels fake at this point. hearing my gpu fan running feels like it's actually doing something.
as a bonus, you're also not giving your data to OpenAI, which is very important for private/sensitive data
the big downside is obviously the hardware. unless you're super rich, you'll be stuck running relatively small models that are nowhere near something like chatgpt.
but honestly, i don't care. LLMs are toys anyways. as fun as they are, they're basically never practical. for the vast majority of use cases there are always better alternatives that don't require a fuckton of vram and a beefy GPU, like... using a search engine.
and LLMs have been getting better, even the smaller models too. the fact that mistral-based models can run on my pc is impressive. it's always fascinating how impressive these technologies are, yet no one stopped and think about how impractical the premise is in the first place.





