the core enabling technology for large-scale machine learning systems as they're currently designed is computer processors which can do matrix multiplication quickly. that's it.
GPUs make a useful computational substrate for machine learning solely because, before machine learning, the closest thing we had to a task that needed to be accomplished at the same scale was the matrix transforms required to produce real-time 3D graphics. the combination of rising polygon counts and the desire for more sophisticated visual effects led engineers in the late '90s to redesign graphics cards from a so-called fixed-function architecture which just did computer graphics, to an architecture which could execute general matrix arithmetic at a staggering degree of parallelism; if you've heard of "shaders", those are small computer programs which execute on one of the up to sixteen thousand individual processor cores on a modern graphics card.1
however, they're quite expensive and power-inefficient -- machine learning applications don't need the high precision mathematics of 3D graphics, and the graphics-oriented feature set can be pared back substantially -- and all of the large cloud computing providers are already getting rid of GPUs in their AI data centers in favor of purpose-built AI coprocessors.
and again, this is all just matrix arithmetic. while there are implementation specifics that surely amount to a significant body of trade secrets, there are no secret theoretical techniques involved here; an effective containment regime would basically be tantamount to a general trade ban on computer processors, and if the US tried that, China would immediately institute a crash development program to make up the deficit. they've already released homemade GPUs which are in the same realm of performance, so this whole thing would win the US maybe 24 months of head start, much of which it would spend building up its industrial base to actually produce physical semiconductors in the country instead of other countries where labor is cheaper.
you would think people who work in practical computing for a living and have access to nearly limitless resources for research and devising industrial strategy would be aware of this, but apparently not.
-
this is also why certain cryptocurrencies used GPUs: the core operation of cryptocurrencies is to run a random number generator seeded on a piece of garbage data and see if the number it spits out is below a threshold -- or, as theophite on twitter famously said, "idling your car 24/7 to produce solved sudokus you could trade for heroin". if you can get that random number generator to run on one core of a GPU, you can run 16,000 instances of it in parallel! but similarly, if the money is in it, and the cryptocurrency isn't explicitly designed to make this impractical, after GPUs stop being dirt cheap, people spin up production lines to make application-specific cryptocurrency mining chips, because GPUs are so power-inefficient.
If you’re a) a weird computer nerd or adjacent in mindset (hi, there’s a lot of us on this website), and b) not already familiar, then you deserve a warning: LessWrong, Effective Altruism, Eliezer Yudkowsky, Urbit, AI Alignment (“AI Ethics” is separate and good, the alignment folks dislike the ethics folks, listen to Timnit Gebru and co), the online “Rationalism” movement (note: “rational” here is the antonym of “empirical” not “irrational”), and everything else in their orbit are dangerous to you.
I cannot emphasize this strongly enough: DO NOT approach learning about this with an open mind. If you must read their stuff, go in shields all the way up, assuming everything you read is propaganda aimed at you specifically. Yes, most of you would probably be fine, but I can’t predict who won’t be, and I’ve already had to spend an evening talking someone in my communities back from this crap this year.
“Neoreaction” (aka “the dark enlightenment” aka NRX), the far right political movement driving these topics, is a basically-fascist cult, but it’s a fascist cult that looks very different from the mainstream ones, and LessWrong and friends are the entry funnels for the cult. Cognitohazards are real and this is one of them.
YOU ARE NOT IMMUNE TO PROPAGANDA
we believe, strongly, in always finding out what things are, in hearing people out in their own words, whether or not we expect to agree with them. that's a necessary defense against dogmatism, polarization, imperialism... all sorts of things. it's necessary.
with that said, yes, "cognitohazard" is a good term here. we are not immune to propaganda, and neither are you. be careful out there.
