• YE GUILTY

belle of the haters’ ball. colorado


joewintergreen
@joewintergreen

so i was working on a game. the game has haystacks sometimes. if you set fire to the haystack, there's a one in six chance that when it burns away, it turns out to contain a treasure chest! you can get some loot

i implement this. much later, QA reports a bug: the chance of a treasure chest spawning isn't one in six. uhhh ok. i have a look and can't repro. i look at the code and it's the same "random chance" code used everywhere, so if there was anything wrong with it, this wouldn't be where it came up. i have a bit more of a think and then send it back, CNR.

the bug bounces back and forth for 2 weeks as the tester keeps saying it's still not one in six. i am down a few rabbit holes thinking about, could we be not spawning the chest even though the result comes in true? is some other class destroying chests? are some levels containing instances of the haystack that are busted in some way? much to think about

eventually i'm like, ok, what exactly are you seeing in your repro. old mate says "i burn 6 haystacks. sometimes there's no treasure chest at all. sometimes i burn 6 haystacks and get more than one treasure chest. so that's not a 1 in 6 chance"

i now have a new, much more challenging task: explain this without embarassing anyone


lexyeevee
@lexyeevee

the slow dawning horror of where this story had to be going


You must log in to comment.

in reply to @joewintergreen's post:

thinking about how some games (don't remember which) will intentionally fudge their 'random' elements to cater to this perception of probability (e.g. if something is a 1 in 6 chance, and the first five rolls are false, then there's a 100% chance of the sixth roll being true). wondering if it was all maybe a big mistake.

I'm sure Path of Exile does that for evasion chance (i remember reading about that sometime in the past).
It feels extra fair in an environment where players are trying to min-max their build and compare different defensive measures (max life, resistances, armor, block, evasion etc.) and if any was pure rng prone to unlucky streaks, minmaxers would just not use it at all. Wouldn't be fun if every meta build was life+armor and 1/3 of skill tree was gathering dust.

different domain, but this reminds me of (on a database engine) having to have the conversation over and over again that "not enough memory to execute query" means... that there's not enough memory to execute the query. no, it is not a bug that you can't do a group-by on 2 billion records on a docker instance with less RAM than an average Minecraft server

to be fair a lot of games implement a "draw bag" like Tetris does, where 1 in 6 means exactly once every six times, rather than a probabilistic/random determination, but that's probably not enough of a sample for said QA person to completely flunk statistics over

Yup, it's like shuffling a deck with 6 times x cards (smallest 6-card deck means it's not possible to get the same result more than twice in a row) instead of using dice.
I know that at least one digital version of Catan lets you pick between both styles - roll based or shuffled results. Which is super nice to see, because i kinda prefer dice-based in that game and being forced to use the other one would feel off to me.

reminded of a bit in the game "Stories: The Path of Destinies" where in one path our protagonist gets his hands on a doomsday weapon which he's warned has a "1 in 128 chance" of destroying the world if used

having no grasp of probability, he blithely marches on the enemy wielding it freely, claiming that he'll be done with it long before he "enters the danger zone"

more than one person chews him out on his fallacious logic (it's a 1 in 128 chance every time he uses it; it doesn't work like a deck of cards), and sure enough, the last time he uses it is the last time he uses it; it rips a hole in reality and brings the world to an end

"1 in 128. What are the odds?"

Or practical examples calling out other possible expected behavior:

Ah, I see how “1 in 6” was ambiguous and could have been documented better. We don't enforce this event happening once each six tries* or periodically (i.e. “once every six hours”) so it should ideally behave like rolling a fair 6-sided die (so, rarely, you will roll two 1s in a row, or you will not get a 1 in 12 rolls. These should seem both rare but possible to the player)

* presumably this is the kind of thing where game developers are sometimes obliged to do an “iTunes shuffle random” to feel more correct to players?

as infuriating as this is! (it is) there's probably a fair bit of milage in conversations around RNG and so on in games and how they're presented to the player.

FOR EXAMPLE- monster rancher 2 (or one, if you lived in the PAL badlands) has a 'bit' where in battles the to-hit chance is displayed as a percentage onscreen. given that attack damage ranges from 'devastating' to 'tickled' it can feel extremely bullshit for your monster to miss five attacks in a row with a 90% hit chance for the enemy monster to one hit KO you with an attack with a 10% hit chance. Even if it is statistically possible.

I remember a Jake Solomon interview where he referenced the exact same thing. Players in XCOM would take a 50/50 shot, miss and expect the next 50/50 shot to hit. And were very frustrated when it didn't.
Firaxis ended up implementing an under the hood system where the more misses a player gets in a row the higher chance their next shot will hit, unbeknownst to the player.
Couldn't find the exact interview but this one mentions it: https://www.gamedeveloper.com/design/jake-solomon-explains-the-careful-use-of-randomness-in-i-xcom-2-i-

A friend has a similar story about a seat-neighbor on a plane trip talking about how weird it is that celebrity deaths seem to cluster together. He responded that it'd probably be weirder if they died on a regular schedule...

ohmygod Hhh I remembered a GMTK video that was all about randomness and there was a part where Mark showed a ton of angry players going to the Steam forums of XCOM and complaining that they missed 6 shots in a row that all had a 90% chance to hit or something like that.

Eventually, I'll end up making a game with random chances in it, and what I'm thinking I'll do is secretly roll two random numbers back to back and average them together. I swear I heard someone mention this strategy before, but I can't remember from where. It's reasonable to say that this should massage numbers towards the middle of the total number range, making it less likely to miss high chance shots or hit low chance ones (fitting the player's broken understanding of probability better).

I'd also be tempted to have a value in the background that tracks the player's luck and secretly gives them an advantage/disadvantage on their rolls depending on what kind of luck they've been having.

Another option for randomness is to roll a random number within a range and then decrement that value each time a specific event occurs, and only do the special thing whenever that value hits zero. Stops the special thing from happening too frequently, or being too spaced out.

Eventually, I'll end up making a game with random chances in it, and what I'm thinking I'll do is secretly roll two random numbers back to back and average them together. I swear I heard someone mention this strategy before, but I can't remember from where.

i think that's how fire emblem does it

"I'd also be tempted to have a value in the background that tracks the player's luck and secretly gives them an advantage/disadvantage on their rolls depending on what kind of luck they've been having"

Reminds me of the L4D AI Director which tracks per-player and per-squad "stress" via a bunch of factors like "how long since you've seen health" and figures the most dramatic still-winnable moment to throw in a boss

I've noticed that has RPG-style mechanics have progressed in computer games, the concept of the miss has been deprecated, in favor of the critical.

An old-school game might give an enemy 30 HP, and your attack has a 50% to hit to do 2-8 points of damage, so you can expect your enemy to go down in about six hits.

A modern game might give an enemy 2,000 HP, and your attack always hits to do 20-40 points of damage with a 50% chance to critical for 700-900 points of damage, so you can expect your enemy to go down in about six hits. Same effective results, but the player feels better that "there's no misses".

This is actually a pretty hard problem in probability! Like, just hard to think about. Probably the most respectful way to explain this is to show a normal distribution? Then you're not calling them an idjit or whatever you're just having a conversation about math.

I thought there was a thing called the "Chernoff bound" that was a specific way of measuring "the more trials you do, the more likely your 1/N probability is to get exactly N successes", but looking it up on Google now it might be something more complicated. Math Wikipedia used to be more readable than it is now.

By the way, in addition to Strawberry Fox's proposal above to make your random numbers seem subjectively "more random" by making the distribution more "lumpy", another interesting approach is to do something like the "7bag" approach in modern Tetris, which has the effective result of forcing probability to act like "intuitive" probability rather than "real" probability. For example, in the haystack case, instead of calculating a 1/6 probability, every 6 haystacks you generate you could pick a number between 1 and 6, and for example if your random number is 4 then you put a single treasure chest under the 4th haystack. I'm not sure this is as fun as true probability, but it would result in nice clean "works as expected" properties like "if there are 36 haystacks there are exactly 6 chests".

a friend elsewhere linked this post and suggested that video games may be partially responsible for teaching bad models of probability

we do find that plausible, though we feel that the primary blame rests with education policymakers, not with game studios

but... yeah. a video game is a collection of hyper-real simulacra, to a degree that Baudrillard must have been astonished by in his later years. lots of fake things masquerading as real things and taking a moment to congratulate the player on the fake thing

FFXIV is a great example of the devs using full randomness on chances. Just ask any gatherer that's heard the muffled noise of failing a 99% gather chance, or a melder who just had a 95% fail rate cause a stack of 100+ materia to explode into smoke because each one rolled a fail.

Deeply frustrating, but I appreciate that it really is random chance.

in reply to @lexyeevee's post:

i have been on BOTH SIDES of this story before and neither side is very fun to be on but everyone watching at least has a lot of fun lmfao

in fact i've also been in the version of this story where the 1 in 6 chance is ACTUALLY NOT 1 in 6 too. there was a point in the game Caves of Qud where the Menacing Stare ability had a bug where it literally couldn't fail and that was honestly pretty easy to repro once they debugged a creature who was built to resist it a meaningful amount of the time. i did get a lot of pushback for trying to report that one too. game dev is funny sometimes