mtrc
@mtrc

This is a post I've been trying to write for a while - like, years - and I've finally gotten it down. I want to stress that I'm not a sociologist, or a historian, this is not an academic treatise or anything like that. It's just a bunch of memories and thoughts, and I don't have a complete picture of all the political and social changes of the last few decades. (Update: thank you for the lovely responses! I will reply to every one, it just might take a little while.)

A few days ago someone sent me a clip of Elon Musk talking to Joe Rogan. In a wild act of self-hatred, I decided to play the clip. Here's a transcript of what he says:

Musk: If you start thinking that humans are bad, then the natural conclusion is that humans should die out. Now, I'm heading to an international AI safety conference later tonight, leaving in about three hours, and I'm gonna meet with the British Prime Minister and a number of other people. So you have to say, like, how could AI go wrong? Well, if the AI gets programmed by the extinctionists it will... its utility function will be the extinction of humanity.
Rogan: -pause- Well yeah... clearly.
Musk: They won't even think it's bad, like that guy. It's messed up.
Rogan: There's a lot of decisions that AI would make that would be similar to eugenics.

This is a blog post about the TV show QI, how the belligerent arrogance of a few people set an example for a whole generation, and why loving science is not enough.


meanderbot
@meanderbot

"I think that one of the healthiest things we could do as a society right now in the west is make everyone feel like they are good enough to learn about and participate in debates about science, technology and the future; while also acknowledging the limits of what 'knowing' things or being 'smart' actually are."


You must log in to comment.

in reply to @mtrc's post:

Really good post! I love reading in-depth stuff like this.

I had a similar revelation while studying philosophy: there are no hard and fast answers to anything, ever. You can use rhetoric to tell any story you want. Giving hard evidence is just one of many tools that you can use to convince people that you're right. So much of the public political and academic sphere is based around reputation, appearance, rhetorical tecnhique, framing, etc etc.

Believability is not a replacement for truth.

Yeah! In my day job people often ask me to define creativity, and I feel like they're often disappointed when I tell them I think it's something we collectively define. But that doesn't make it less real, and actually appreciating that there are very few hard and fast truths out there and working together to navigate that is part of life!

As a religious person, I think the stuff you described - of euphoric redditors slam-dunking on everyone - feels like its trending downwards these days. And even leftist spaces are never as hostile as I fear them to be.

Now please tell us about how BBT contributed to the Acting Smart problem!

P.S. from my limited understanding, wouldn't an AGI be grown rather than programmed, and thus be somewhat human-like unlikely to maximize paperclips into a Gray Goo scenario?

I'm really glad to hear the slam-dunking is on the downswing. I kind of had a similar sense. I think I'm right in saying that the generation younger than me seem to like chatting and sharing about spiritual stuff a bit more, I wonder if that's helping people be a bit more open-minded to faith-based beliefs that they don't share.

The AGI question is hard to answer - I don't really think AGI is a thing, no-one can really define it, they just dodge the question. But we can think of ChatGPT etc as 'grown' by learning from tasks. Even there though we do direct them - we give it rewards and tell it what to aim for.

Thanks for reading :)

My favorite person on QI will always be Alan Davies. The hapless man who is constantly set up to make himself look stupid but never stops trying anyway while we all laugh at the big moron who keeps coming back for more.

I've caught some episodes from later seasons and the funny thing is that through a combination of being the only person on every episode and his own personal work, he's now become a savant who occasionally just destroys all the other contestants, which is an amazing anime-ass character arc.

"I am wiser than this man, for neither of us appears to know anything great and good; but he fancies he knows something, although he knows nothing; whereas I, as I do not know anything, do not fancy I do. In this trifling particular, then, I appear to be wiser than he, because I do not fancy I know what I do not know." -- Socrates

The actual smartest people on earth, if such a thing even exists, are the people who have the most sense of doubt about their own knowledge, and this has been a disaster for humanity for nearly as long as there have been humans.

I did the tech conference circuit a lot for a while when I was still living in Europe, and I met some genuinely brilliant people who I'd be considered name dropping for bringing up ... and every one of them was among the most humble people in the room.

It was always the junior and mid-level devs in a conversation circle most presenting as if they knew a goddamn thing, and with no shame whatsoever to be doing it, even in front of "man who literally invented the programming language they're talking about".

ok i need to explore a thought here, i don't know if it's tangential or not to this post, but i still gotta get it out

for those who weren't immersed in the religious arguing that happened for a few years after the publication of the god delusion, tons and tons of theists (mostly, but not all, christian) would absolutely pull the kind of stuff this post complains about dawkins doing all the time. endless trite gotchas, people genuinely going "if humans evolved from monkeys why are there still monkeys". yes really, that wasn't a joke! it happened!

and in the realm of more formal debate, there were plenty of smarmy creationists happy to abuse the setting (a reason that a lot of people nowadays think debate is inherently flawed). famously, creationist duane gish ended up lending his name to the gish gallop, where you just ask as many pointless questions about different aspects of evolution as possible and demand your opponent explain them all. any decent evolutionary biologist would be able to give coherent and reasonable answers, with explanations of the current limits of our knowledge, to all gish's questions if given enough time, but the structure of debates meant that no matter how hard gish's opponent tried, they would never be able to answer every one of gish's questions sufficiently, allowing him to declare victory

and this kind of stuff still happens today! a couple years ago i got to see an atheist say something kinda dumb on twitter and then get pilloried by a bunch of equally-smug (and often way worse) theists, breaking out more shitty gotchas like 'if god isn't real, then why do oranges have peels to protect their fruit?" complain about richard dawkins all you want (which you should, he's a huge dick) but ray comfort still casts a gigantic, hideous banana-shaped shadow across the land.

i'm left thinking about how easily we focus on the anger and bitterness and assholishness of one group and not the other. i'm thinking about who is and is not allowed to be mean, how we interpret going too far, etc. i've lived my entire life in the american midwest, and i'm used to midwestern nice. politeness glorified, weaponized, turned into a means of manipulation: they were so polite to you, why aren't you being nice to them by doing what they want? (i'm not even from the upper midwest, the stories i hear from minnesota frighten me hard)

and that means i have to ask, what are we missing when we talk about this kind of smug, arrogant 'smartness'? obviously, something that leads to the popularity of useless, cruel shitheads like dawkins, fry, and musk deserves critique. but where did this come from? what is it a reaction to? are we focusing on the parts of it that annoy us the most while inadvertently giving other parts, maybe even worse parts, a pass? and most importantly, how do we make sure we do better than this smugness instead of just reverting to another terrible mode of thinking about the world?

yes i know those questions are leading, but i don't know how to phrase them better right now.

i think for the common practicioner this kind of shitty gotcha-ism is more an exercise in power than anything else. the power to victory, to render one's chosen opponent objectively Wrong by the authority of a higher, inexorable power: twitter screenshot of @widdr commenting on a person arguing about a PEMDAS engagement bait: "they feel like they're granted a deontological superiority by pure and immutable principles from a higher plane of being, so it's basically a religious fervor in side-taking over a pseudoscientific dispute. like this guy, who took a quick break from posting bluemaga shit for this"

if that's true i sure don't know what to do about it because it's basically a human nature thing, cultivated in large part by the social environments those kinds of arguments stereotypically thrive in.

I don't know how to explain it, but "if no God, then how come banana" creationist don't have the cultural cachet of Dawkins et al. Everybody [we care to align with culturally or gets prime spots on TV] thinks those guys are a joke. Meanwhile, Dawkins got major science respect points, Bill Maher still has a show, etc..

As a Catholic Lithuanian, I know those guys off-hand, but I had to google Ray Comfort and never heard of gish gallop's namesake.

Now, you'll find some of the worst theology possible in the right-wing circles, but, like, what do you expect to find there?

Politeness, on the other hand, extends outside of midwest. It's long been a weapon wielded against the left because you're not being angry about life and death questions like M4A or accepting that trans people are people - you're IRRATIONAL and TRIGGERED!

Thanks for adding this! I think this is useful context that I only briefly alluded to in the piece. Terrible people exist in all spaces, and I agree that we often do center ourselves too much on one group and then ignore the problems caused by the other. For example, as terrible as 'techbros' are, I can tell you I've met quite a few 'artbros' who are just as bad on the other side of the fence, and as someone trying to do research in both spaces it basically leaves you feeling pretty unhappy! Haha.

I think for my part, I focus on these people because they are 'my' people to some extent. I work in academia, I'm told I am 'smart' a lot, I compete for airspace with people like Musk when talking about AI and trying to discuss problems and issues for the future. So I think my post centers on this from a "speak about what you know" perspective - I haven't been near religious communities for a long time, and I wasn't really religious myself even when I was near them, and they were pretty milquetoast English ones anyway. So I just wouldn't feel as confident talking about those issues. But you're right to raise these points I think, and nothing comes out of a vacuum entirely.

Thank you for writing this Mike, it very neatly encapsulates a lot of problems I often think about. And especially growing up and doing some debating at school, how that kind of Objective Victory structure was very ingrained as part of education and in particular British public schools.

Thank you Lisa! And thanks for giving me feedback on it too ^_^ Debating at school, haha, god. I only did it like, twice, but it was a whole scene. The American version of it is genuinely terrifying.

I'm a big fan of science. :) But I've never cared for the whole emphasis on STEM, because it represents an effort toward economic hegemony via high technology products, and not any particular love of or respect for science on the part of the people who own and run our society. I also happen to think that the humanities and the arts are what make life worth living, and that it's a great mistake to devalue them.

As for Dawkins particularly, I always felt he's exactly the sort of person you wouldn't want on your rhetorical side, because he's just such a prick that he turns everyone off to your point of view... it's a little shocking to imagine people responding positively to his behavior. I suppose I don't really understand people.

Really enjoyed the article. :)

I think you're right, although thankfully a lot of the people who are drawn to teach STEM subjects have an infectious love for it that does often inspire people to apprecaite the good bits, even if the system as a whole is geared towards the bad bits. I think often about how lucky I am to be paid with public money to make the world better for the public, and how sad it is that my job often conflicts with that idea and encourages me to do other things instead.

Thanks so much for reading and commenting :)

I don't mean to impugn the motives of the people who are actually doing the work, of course, the people who are actually in the thick of it. Given the sheer grind that can be involved in study and research, you really have to believe in what you're doing. But the boosterism by people who aren't invested in those fields (usually administrators or politicians, what's the difference), as a means to an end... I could do without that.

But yes, it's a very thought-provoking article. Thanks for composing and sharing it. :)

Spot on. One of my great personal realizations is that being intelligent doesn't imply being right. "Smart" people are often (usually?) just better at rationalizing than other people. It takes an incredible amount of humility and discipline to evaluate arguments in a truly logical way, and in general our society does not know how to do it. We really need philosophy to be the core of our education system.

Yes. Admitting failure or being wrong is so hard, and we incentivise everyone against it (including scientists, which is where you get a lot of big academic scandals from). We actually tried to work against this a bit in my field by hosting things like workshops explicitly for papers about 'failed' work.

great essay! perhaps this puts a finger towards one of the things that has always bothered me. the "Aesthetics of science" are so far removed from the realities of science. especially in the context of media.

the "super smart person (tony starks) are just going through the vaguely convincing motions of just knowing things and making things. and being good at everything along the way.

real science is messy, slow, incremental, and highly collaborative. This disconnect i feel like gets in the way of communication, because it sets expectations too high.

It also bothers me that capitalists tend to exploit this aesthetic for the air of credibility from people who dont understand better. (its mostly this part haha)

Nothing makes me more antsy than when a 'smart' character appears in a movie. It's always so bad. It's even worse when what they're saying or doing doesn't make sense as being smart. Even the notion of having multiple PhDs is very very funny because it's supposed to make someone sound smart but in almost all circumstances would be the most confusing and weird flex ever.

wish folks understood science as something everyone is doing constantly, like looking at and puzzling over a bug on the street. if they knew the significance of their own behaviors then maybe they'd be less susceptible to the funny science mimics

as a former "logic guy", the worst thing you can do is convince yourself is that you are a rational actor. because you are a fleshy fallible person, it makes you believe that everything you say and do is "rational" and "logical", and that everyone who opposes you is "irrational“ and "illogical". it makes you a profoundly insufferable person that will alienate your friends and push you into the company of truly terrible people.

deeply interesting thoughts, thank you. i'm from the humanities field (now game design) and back when i was doing more academic stuff we faced a similar issue - which i noticed even more once i started studying game design and realized a lot of people talk about video games the same way Musk talks about anything.

back then, we called it the "people scan but don't read, people hear but don't listen" problem: it's extremely easy to fool someone into thinking you're an authority if the other person isn't willing to stop and actually think about what's being presented to them - or if they don't have the humility to ask them what they mean with the words they're using.

there's a media aspect to this as well. Rogan is pretty much slurry in podcast form, but the same thing happens whenever you turn on the news: a serious-looking journalist will keep nodding while someone (often, but not always, a politician) will spout out stuff that's either misleading, false, or impossible to verify or parse without more information. i've personally felt the impression, especially when i worked in journalism-adjacent fields, that going "could you clarify this" is both showing my lack of knowledge of a subject and extremely unprofessional. it's an instinct that needs to actively be fought, and it's not easy, especially when it's tied to your employment

Yeah absolutely. The culture of appearing smart and challenging people to question you is really self-perpetuating too because, as people have noted, the people who really should be interjecting and questioning often feel the least able to because of the power imbalances, which this performance is intensifying. And then in large groups its even harder - students often report feeling like they're the only one who doesn't understand something so they don't ask, in reality the entire room all think that so no-one asks and no-one learns haha.

Great thoughtful read as usual. Also I cannot believe that this is how I found out that QI was an abbreviation.

I think some of it is is just our education system teaching Science for exams that we must get right as opposed to a process that doesn't really end (cuts off tangent me pitching Science history as an A level). And things like STEM and later STEA(rts)M are still firmly in the education as preparing you for the workplace, which I get, we need to pay for a roof and food, as if there couldn't be space for education for the sake of learning around like minded folk being of value on it's own.

Thanks Andrew ^_^ Haha I don't remember where I learned it, I think reading the back of one of their books because it's their company name or something.

I agree with you on the preparation for work stuff. It reminds me a bit of research funding too, because the best ideas often come out of blue sky research, but if you ask to do blue sky research it's like, no, do something that'll result in results, we can't let you do that.

BTW the "Q" in "QI" is very clever. The word "quite" in UK English is a lovely double-edged sword. It can mean either "very" or "not at all" depending on the context. In the context of the title of the show... well, it begs you to decide, doesn't it?

You know how before "computers" were boxes of electronics, we had "computers" who were people (usually women) who... computed? Well every time someone says "what if the AI decides" I replace "AI" with "The RAND Corporation" which was the AI we used in the 50s and 60s.

Hahaha, yes. So much stuff hidden behind language. I saw the "we're going to use AI to fix [completely random political issue]" headlines beginning to spark up again as we near an election.

I recently saw a 538 video about how AI would affect the US election. A fair bit of eye-rolling stuff, but they did finally get down to the point that sophisticated election campaigns have always used a huge number of human beings to plan their strategies, and the only change if they use AI is that it will have the same output, it just will cost them slightly less. What it DOES do is elevate those who can't afford to hire a room full of writers to produce lies. I go back and forth over whether this is a good thing.

Really enjoyed this post. Lots of good food for thought; I paused often, finding myself thinking about various personal experiences in education (once upon a time I studied a science at university!) and life outside of academia. The final bit is a hope I share: “I think that one of the healthiest things we could do as a society right now in the west is make everyone feel like they are good enough to learn about and participate in debates about science, technology and the future; while also acknowledging the limits of what 'knowing' things or being 'smart' actually are.”
Thank you :)

Thank you so much Halima! I'm so glad you enjoyed reading it. I've felt a bit powerless to really do anything about this lately, but wanting to make everyone feel like science belongs to them, and that they have a stake and a voice in it, used to be one of the big things I wanted to do as a scientist. It feels harder than ever today but I still really do believe in it. I hope we get a chance to meet in person again soon and chat!

Oh my goodness thank you for writing this. It looks like it was a tonne of work and represents a lot of distilled experience and thought.

As someone who got caught up and hurt by this aesthetic in the 00's (should have been an engineer not a scientist, but had already shackled my identity to getting a PhD) it rings so, so, so true to me. It's the reason I find Big Bang Theory so distasteful, why The Imitation Game left me so cold. I'd had the damn feeling of working on a tough, science-y problem in a group but the popular depictions felt so off.

For me, it eventually boiled down to be kind, be inclusive, don't ever wield your knowledge as a cudgel or stand for someone putting themself down because the world tells them their knowledge is "softer". I found learning about topics like Linguistics to be a great balm for this: the first principle there seems to be "people who speak the language know it and if it doesn't fit in your system then your system is broken, not the person who knows less about linguistics and more about the language; yes, even if the language is Klingon"). This post gave me a much clearer understanding of why a more aggressive attitude is so wrong.

Thoughtful writing like this also helps keep away the ennui of trying to earn a living as an honest code monkey in an industry that seems to keep inventing dumber and dumber ways of building the torment nexus (often by throwing person-years of care and effort at problems that could be easily solved by "not surveilling users" or "using technology that isn't trendy").

So again, thanks <3.

Oh damn, a lot of stuff resonating here - there was originally a huge section on Big Bang Theory in this article, and I also hated The Imitation Game. The use of knowledge as a cudgel is such a big thing, I hate it so much, it really activates something in me when I see it happen haha.

I'm sorry your current job isn't making you so happy, but it sounds like you're a thoughtful person with a lot of great experiences, so I'm very glad you're one of the people in the mix of that, rather than people who wouldn't give it a second thought :)

Yes, thinking about the shifts in internet culture and technology in general... it's so fascinating, if I had another academic life I think I'd become a historian of technology and just analyse that all day long.

This is a great post Mike! Thank you for writing it! I've been thinking about this sort of thing a lot as I finish my PhD and talk more with people outside of academia. People get intimidated by the fact that I have 2 going on 3 grad degrees and worry about boring me or not being smart enough to talk to me.

Smartness being a quality people possess according to our society is so damaging to actually communicating knowledge. Like you say towards the end of your post, whether we in our field are the "smart people" or the know nothings depends on who is the majority in the room. And all of that, I guess I can call it hierarchy and posturing means actually talking about knowledge, science, research, etc. becomes impossible in a way that feels very new

I love this post overall, and the amount of context about the changing of science communication is great, but I really want to thank you for the hours of fun as the idea of mismatched facts sent me on the path to entirely replicating the core of consensus reality theory with absolutely no foreknowledge that it was already published in 1966. Accidentally doing epistemology is a great use of an evening, and I think it's a great model for this phenomenon broadly as well as a ton of other bullshit going on right now.

Our conception of "success" is a phenomenon that exists in consensus reality, and influences how we judge others' worth; it influences how we interpret material reality into subjective experience. When someone has a lot of money, or a large following on social media, or "wins" in the "debate", we judge them as "successful" because consensus reality teaches us to do so. As this judgement of success re-enters consensus reality it collides with meritocracy, which claims that if someone is successful, they must be competent. Now, the consensus reality is that they are competent, despite not having displayed competency. Meanwhile, those who display competency are only judged competent by those competent enough to recognize them, rather than benefiting from the mass appeal of generic meritocratic "success", thus being less likely to be judged the ideal candidate.

Can't guarantee it's Correct, but I wouldn't have been able to articulate that in a thousand words this morning, so thanks for the inspiration to do a little science of my own!

This sums up so many of the problems I had in high school - it was all about "memorise this thing that'll be on the test" rather than actually learning anything.

My ability to consume literature was judged not on my taking away anything from the the book, but that I agreed with the teacher's interpretation of it. We were told to watch out for "recurring themes" without actually being told what a "recurring theme" actually was, and were even asked if we'd spotted the recurring theme while the class was on chapter one and the theme in question was mentioned for the first time and therefore hadn't actually recurred yet. Oh, and actually going "screw it" and just reading the whole damn book over a weekend got us chastised if we brought anything up from it because "we haven't got to that chapter yet" - so we were supposed to somehow discuss foreshadowing without being allowed to bring up the events they foreshadowed.

I was terrible at maths in my very first years of primary school because it was all based on "memorise these times tables" and my apparent ability shot through the roof when we learned how to multiply arbitrary numbers instead of just memorising the answers. Likewise when I did high school physics, I had the equations of motion memorised for years before my maths class got onto the very first calc lesson and it suddenly clicked how they were all related to each other.

I suppose part of the issue is that your average layperson doesn't need to know the why for certain things and just needs to remember what it is, but the real problems come when people treat those "good enough for the layman" facts as the be all and end all of ultimate scientific knowledge that must not be questioned.

I remember despising Fahrenheit 451 because I had zero clue what I was looking for during my sophomore year summer assignment (which is bullshit in itself, but I digress). By comparison, when I read Homegoing (by Yaa Gyasi) during senior year, I was enthralled and read ahead of when I was supposed to for my assignment. One thing that helped was the relative lack of pressure. This was one book in the middle of the semester, as opposed to the thing that will determine my grade at the start of the semester. Also having teacher input is nice. I suspect my re-readings of Fahrenheit will be mired by that traumatic memory, while Homegoing will forever have pleasant memories associated with it (reading it, not the subject matter. The book is about the trauma of slavery through the centuries through the lens of one black family tree)

I need to give the Great Gatsby another go now that my reading of it isn't tainted by someone who's read the book a thousand times and is assigning marks based on the cliff's notes. I swear I've learned more about literature interpretation from twitter than I ever learned in that class.

To give you an idea of the kind of teacher I had, part of the scottish higher grade english course was a "review of personal reading" which was a fancy way of saying a book review where you had to turn in an essay about a novel, and she had this whole list of things about identifying an "appropriate" novel to review that included such hot tips as "be suspicious of anything where the author's name is in larger text than the title" and outright saying that certain genres like horror or fantasy don't count as literature.

You know the kind of person. The kind of literature snob who, when presented with literature that breaks their rules, will come up with some crap like "oh it transcends the genre" or some other high-handed way of saying that Discworld doesn't count as a fantasy novel because it's actually good, anything rather than admit that their rules about what counts or not are flawed.

One of the most frustrating things about this perception of "public intelligence" is that there are people who side with them because they look smart, but it takes far more intelligence to systematically dismantle these wannabe intellectuals' shitty arguments. It's a structure that favors weed-smoking, fast-talking con men over the actual intelligentsia of the internet. On an unrelated note, so many of them lean right. That's weird.

In response to the "facts don't care about your feelings" thing I've started saying "that may be true, but feelings don't care about your facts", as a way to say that your opinion is often dictated by your feelings on a certain subject rather than what we decided on as fact

After having had countless arguments in my life I've come to see that if you want to convince someone that your argument has merit, simply stating facts and citing studies is entirely ineffective. You need to argue with the emotions a person is feeling on said subject instead

Though acknowledging this and applying it in real life or online can be very difficult for me still

Excellent post and well written

It reminds me of recently giving the most barebones layman's explanation of ChatGPT to someone, which I had picked up from my reading and watching videos on the subject. Their response was to act like I must be an expert on it and I should go into studying it. Like what the fuck are you talking about, I don't know shit about AI, this was a bare surface level understanding of the subject. "It's a glorified Markov chain and shouldn't be trusted with anything important" is not Deep Knowledge. But since I had done ANY research, it sounded like expertise to them.

It was a very weird interaction and just. I wasn't even trying to give that impression, I was just ranting about how it isn't what people claim it is. If that's someone's reaction to irritated rambling, of course the general public believes someone who claims to know their shit, they have no basis to compare or question it.

Which is a problem when grifters are the ones with the charisma (...or whatever the fuck Musk has) to supplant actual experts. Which isn't to say experts can't be charismatic, but grifters are trying to make a career off it. The "I'd have a beer with him" effect.

Maybe if high school taught skills like how to vet sources and analyze information, instead of waiting until college, we would have less of this shit. Maybe it already is, since the younger generations seem pretty done with bullshit. The problem is with the majority of the population who would no longer benefit, anyway, but there might be some hope there for the future.

As someone born in 2000 who grew up as a gifted kid that developed a lot of anxiety and weird psychological dysfunction over my intelligence, Half of this is stuff I've been saying for years now and the other is stuff I really needed to hear, and I thank you very deeply for all of it. (Especially introducing me to the idea of a Utility Function! I can apply that to my life now!)