the core enabling technology for large-scale machine learning systems as they're currently designed is computer processors which can do matrix multiplication quickly. that's it.
GPUs make a useful computational substrate for machine learning solely because, before machine learning, the closest thing we had to a task that needed to be accomplished at the same scale was the matrix transforms required to produce real-time 3D graphics. the combination of rising polygon counts and the desire for more sophisticated visual effects led engineers in the late '90s to redesign graphics cards from a so-called fixed-function architecture which just did computer graphics, to an architecture which could execute general matrix arithmetic at a staggering degree of parallelism; if you've heard of "shaders", those are small computer programs which execute on one of the up to sixteen thousand individual processor cores on a modern graphics card.1
however, they're quite expensive and power-inefficient -- machine learning applications don't need the high precision mathematics of 3D graphics, and the graphics-oriented feature set can be pared back substantially -- and all of the large cloud computing providers are already getting rid of GPUs in their AI data centers in favor of purpose-built AI coprocessors.
and again, this is all just matrix arithmetic. while there are implementation specifics that surely amount to a significant body of trade secrets, there are no secret theoretical techniques involved here; an effective containment regime would basically be tantamount to a general trade ban on computer processors, and if the US tried that, China would immediately institute a crash development program to make up the deficit. they've already released homemade GPUs which are in the same realm of performance, so this whole thing would win the US maybe 24 months of head start, much of which it would spend building up its industrial base to actually produce physical semiconductors in the country instead of other countries where labor is cheaper.
you would think people who work in practical computing for a living and have access to nearly limitless resources for research and devising industrial strategy would be aware of this, but apparently not.
-
this is also why certain cryptocurrencies used GPUs: the core operation of cryptocurrencies is to run a random number generator seeded on a piece of garbage data and see if the number it spits out is below a threshold -- or, as theophite on twitter famously said, "idling your car 24/7 to produce solved sudokus you could trade for heroin". if you can get that random number generator to run on one core of a GPU, you can run 16,000 instances of it in parallel! but similarly, if the money is in it, and the cryptocurrency isn't explicitly designed to make this impractical, after GPUs stop being dirt cheap, people spin up production lines to make application-specific cryptocurrency mining chips, because GPUs are so power-inefficient.
If youāre a) a weird computer nerd or adjacent in mindset (hi, thereās a lot of us on this website), and b) not already familiar, then you deserve a warning: LessWrong, Effective Altruism, Eliezer Yudkowsky, Urbit, AI Alignment (āAI Ethicsā is separate and good, the alignment folks dislike the ethics folks, listen to Timnit Gebru and co), the online āRationalismā movement (note: ārationalā here is the antonym of āempiricalā not āirrationalā), and everything else in their orbit are dangerous to you.
I cannot emphasize this strongly enough: DO NOT approach learning about this with an open mind. If you must read their stuff, go in shields all the way up, assuming everything you read is propaganda aimed at you specifically. Yes, most of you would probably be fine, but I canāt predict who wonāt be, and Iāve already had to spend an evening talking someone in my communities back from this crap this year.
āNeoreactionā (aka āthe dark enlightenmentā aka NRX), the far right political movement driving these topics, is a basically-fascist cult, but itās a fascist cult that looks very different from the mainstream ones, and LessWrong and friends are the entry funnels for the cult. Cognitohazards are real and this is one of them.
YOU ARE NOT IMMUNE TO PROPAGANDA
I remember reading some of the stuff from these guys while I was looking for study resources for quantum mechanics. While they're correct about the algebra involved in Bayesian analysis, they were basically anywhere from misleading to flat-out wrong on everything else, including, notably, the Heisenberg uncertainty principle, which exists partly as a consequence of the tools with which we do particle physics experiments, and partly due to the mathematics of wavelet transforms (localizing a signal in time domain vs frequency domain). Obviously they're wrong about using Bayesian reasoning for your own decision-making process, because we don't actually do conditional probability in our decision making that way. You have never quantified your "priors". Brains don't actually work that way!
When I got reporting that suggested the real audience for these guys were specifically silicon valley finance guys, a lot of what was going on with that audience made a lot more sense. Not that they're the only people who'd fall for this stuff, but that a lot of real science education helps to inoculate against building a bigger worldview out of this stuff, and to a certain level making it easier to see the fascistic side of it all.
I've always found it suspicious that the guys asking me to cite sources treat it like a struggle session instead of me saying things on twitter that being with "I think"
