This post contains 18+ content. You can view it if you're over 18.
- she/her
Recently appeared on this plane. Last seen: Posting (in a serif font) and/or casting spells. my icon and header image are turned around on purpose actually its not like i dont know how to fix it or anything. my age is private information but if you feel the need to know it presume i'm somewhere between 18 and the age you are and treat me accordingly
We expose a surprising failure of generalization in auto-regressive large language
models (LLMs). If a model is trained on a sentence of the form “A is B”, it will
not automatically generalize to the reverse direction “B is A”. This is the Reversal Curse.
This is an interesting paper showing that, again, the same AI scaling problems that plagued us in the 60s affect modern systems too, but the bias and intent of the researchers shows through so plainly. A "surprising failure of generalization" instead of a more or less expected result of what LLMs actually do (ie, predict which language token could come next) and vague appeals to "well maybe humans have the same problem!!!1"
reminded of how -- iirc from artificial intelligence class in college -- part of what precipitated the first AI winter was a decline of interest in neural networks in favor of symbolic systems1 because it was realized that perceptrons (early, rudimentary precursors to the same AI techniques in favor today) were mathematically incapable of learning the exclusive-or function -- the logical formalization of the concept "A or B but not both"
-
which everyone eventually lost interest in because building a machine that can reason by hand requires too much work
Birds are fucked up. Here's a dinosaur that evolved to be an extremophile filter feeder. Here's a dinosaur with the lifestyle of a bee. Here's a dinosaur that deliberately starts wildfires.
When I'm feeding my chickens, especially live food like bugs off my plants, I am always consciously grateful that the size difference between us runs in my favor.
Relevant title text from a different comic: Whenever you see a video of birds doing something weird, remember: Birds are a small subset of dinosaurs, so the weirdness of birds is a small subset of the weirdness of dinosaurs.
