Fixed! As it turns out, the DOI I had intended to link to hits a paywall anyway, so this gives me the opportunity to use a link that actually leads to the PDF.

In the classic DK demonstration, people took a brief test of "logic puzzles" and were also asked how good they think they are at logic puzzles. Here, 4000 such participants are divided up by their "true score" on that brief test, and the average of everyone getting a particular score is shown in blue. These averages have error bars because we should be less confident declaring that we know that average when it's due to only a few observations. For example, very, very few participants got a score of 0 or a score of 19, so the errors bars on those averages are correspondingly huge, because the tiny handful of confidence reports we got from people with such extreme scores haven't taught us very much about how future participants with those scores might respond on the confidence scale.

If performance and confidence perfectly predicted one another, the blue averages would lie on the red dotted line. Clearly, the spread of the average confidence is much shallower than that, implying that the folks on the left side of the graph are overestimating and the folks on the right of the graph are underestimating. This is the "classic DK effect," but it's very misleading. Since the simple test people were given isn't a perfect measurement, and since people also vary in how confident they feel, both measures have a degree of error. Since both scales have a top and a bottom bound, if there's going to be an error in measurement, it's probably going to be an error that moves a score toward the average. This leads to "regression to the mean," a statistical phenomenon that's been with us since we first started thinking about averages, and that has been fooling us for just as long. What Jansen and colleagues have done (which isn't at all trivial) is make a good-faith effort to estimate how strong this effect is for this sort of data, and what they found is that regression to the mean dominates this pattern. This is reflected by the red squares: If we assume only that the logic test and corresponding self-report of ability are prone to error in their measures, with no true DK effect whatsoever, you get the red squares. That's the basis for saying the "true effect" doesn't exist - the famous pattern looks almost the same as what you would expect from noisy measurement alone.

However, Jansen and colleagues considered another possibility: Participants have some idea of how well they did on the logic test, right? Like, did it feel easy? Or hard? So they fit a second model that anticipated that perceived ability might depend both on their own beliefs about themselves and on how well they did on the test. This yields the yellow-green circles, which is the model that best describes the data: A curve that mostly follows the RttM pattern, but deviates from it a tiny bit. So their takeaway is that there's a small pattern of bias, but it's more complicated than the traditional "beginners overestimate, experts underestimate" narrative that DK normally predicts. Indeed, there's still a big of discrepancy between the yellow-green circles and the blue averages, so it's possible that the real relationship is even more complicated, due to some mixture of "kinds of people" that are being lumped together.

Hope that helps!

"Spaceship Cybertrukk Jackass" sounds like an amazing twist on Galaxy Trucker, particularly since it implies shoddy vehicles that are prone to falling apart and exploding.