the core enabling technology for large-scale machine learning systems as they're currently designed is computer processors which can do matrix multiplication quickly. that's it.
GPUs make a useful computational substrate for machine learning solely because, before machine learning, the closest thing we had to a task that needed to be accomplished at the same scale was the matrix transforms required to produce real-time 3D graphics. the combination of rising polygon counts and the desire for more sophisticated visual effects led engineers in the late '90s to redesign graphics cards from a so-called fixed-function architecture which just did computer graphics, to an architecture which could execute general matrix arithmetic at a staggering degree of parallelism; if you've heard of "shaders", those are small computer programs which execute on one of the up to sixteen thousand individual processor cores on a modern graphics card.1
however, they're quite expensive and power-inefficient -- machine learning applications don't need the high precision mathematics of 3D graphics, and the graphics-oriented feature set can be pared back substantially -- and all of the large cloud computing providers are already getting rid of GPUs in their AI data centers in favor of purpose-built AI coprocessors.
and again, this is all just matrix arithmetic. while there are implementation specifics that surely amount to a significant body of trade secrets, there are no secret theoretical techniques involved here; an effective containment regime would basically be tantamount to a general trade ban on computer processors, and if the US tried that, China would immediately institute a crash development program to make up the deficit. they've already released homemade GPUs which are in the same realm of performance, so this whole thing would win the US maybe 24 months of head start, much of which it would spend building up its industrial base to actually produce physical semiconductors in the country instead of other countries where labor is cheaper.
you would think people who work in practical computing for a living and have access to nearly limitless resources for research and devising industrial strategy would be aware of this, but apparently not.
-
this is also why certain cryptocurrencies used GPUs: the core operation of cryptocurrencies is to run a random number generator seeded on a piece of garbage data and see if the number it spits out is below a threshold -- or, as theophite on twitter famously said, "idling your car 24/7 to produce solved sudokus you could trade for heroin". if you can get that random number generator to run on one core of a GPU, you can run 16,000 instances of it in parallel! but similarly, if the money is in it, and the cryptocurrency isn't explicitly designed to make this impractical, after GPUs stop being dirt cheap, people spin up production lines to make application-specific cryptocurrency mining chips, because GPUs are so power-inefficient.
If you’re a) a weird computer nerd or adjacent in mindset (hi, there’s a lot of us on this website), and b) not already familiar, then you deserve a warning: LessWrong, Effective Altruism, Eliezer Yudkowsky, Urbit, AI Alignment (“AI Ethics” is separate and good, the alignment folks dislike the ethics folks, listen to Timnit Gebru and co), the online “Rationalism” movement (note: “rational” here is the antonym of “empirical” not “irrational”), and everything else in their orbit are dangerous to you.
I cannot emphasize this strongly enough: DO NOT approach learning about this with an open mind. If you must read their stuff, go in shields all the way up, assuming everything you read is propaganda aimed at you specifically. Yes, most of you would probably be fine, but I can’t predict who won’t be, and I’ve already had to spend an evening talking someone in my communities back from this crap this year.
“Neoreaction” (aka “the dark enlightenment” aka NRX), the far right political movement driving these topics, is a basically-fascist cult, but it’s a fascist cult that looks very different from the mainstream ones, and LessWrong and friends are the entry funnels for the cult. Cognitohazards are real and this is one of them.
YOU ARE NOT IMMUNE TO PROPAGANDA
there seemed to be a pretty specific set of norms that took common leftist/queer Bay Area techie culture and turned it up to 11. everyone was polyamorous in a way that actually turned me off of polyamory for a while because 1 person would be dating between 10-40 people and it felt more like way for them to constantly get positive attention and less like they had genuine affection for all of their partners. especially noticeable was that they seemed to get bored of a relationship after a while and would focus more on newer more novel relationships.
there's also a strange relationship to drugs. some rationalists seem to think of their bodies as merely an inconvenient vessel for their brains that they can manipulate by applying the right substances and stimuli. one time at a party i saw someone wash down pills (not sure what they were) with half-and-half orange juice and Everclear.
the worst part though is that anyone who is absolutely certain that they are morally superior to you (in this case, because they donated a lot of their tech job salary to "efficient" causes) is that they essentially think they can do no wrong and that there's no need to treat the actual people around them with consideration and care. this way of thinking also rapidly leads to the conclusion that the richest people are the morally best because they give the most money to charity, and that sure doesn't lead anywhere good.
anyone who is genuinely convinced they've overcome the cognitive biases that bind all other humans, thinks that they're smarter and righter than everyone else, which does not lead them to be at all pleasant to interact with. it also lets them dismiss criticism from outsiders as being the product of less enlightened thinking.
in reality, though, you're actually filtering for people with the kind of ego who can believe that they're smarter than everyone else, which means you're actually filtering for people who may be intelligent but who are also incredibly lacking in self-reflection and wisdom, and who just kinda suck.