Greetings fellow adventurers. I have heard a number of complains that D&Dbeyond dice are rigged. I was curious, so I decided to investigate further. I rolled 2000 dice, 100 at a time and recorded the results. The raw data can be found here if you are interested.
The summary however is that the average of 1042 was slightly below the expected average of 1050. However, this could have been due to chance variation (for example, if you roll a d20 and get 9 that doesn't mean it's rigged, you just rolled unlucky). There is a calculation to determine the plausibility of a result, called a t-test. Using this returned a p-value of 0.58, which basically means that if we assume the dice aren't rigged at all, there is a 58% chance that by random luck you get a value of 1042 or more extreme. This means that we can't make any conclusions. It is likely therefore that I got this value due to chance, given that it could have happened at a 58% probability (5% is usually required to have any plausible claim).
It wouldn't make sense for them to be rigged, anyways. Sure, the fact that they're simulated instead of just randomly generated means that some faces relative to the face that's on top at the start of the roll almost definitely have a higher chance to land, but the face that starts facing up is randomized. Even if the dice were just dropped and not rolled at all, the odds would still be fair because the system does use an actual random number generator to determine which face it's dropped on.
Rollback Post to RevisionRollBack
Look at what you've done. You spoiled it. You have nobody to blame but yourself. Go sit and think about your actions.
Don't be mean. Rudeness is a vicious cycle, and it has to stop somewhere. Exceptions for things that are funny. Go to the current Competition of the Finest 'Brews! It's a cool place where cool people make cool things.
How I'm posting based on text formatting: Mod Hat Off - Mod Hat Also Off (I'm not a mod)
Not a computer person, but I found anecdotally I get better results, generally, the bigger the screen is. Someone in a thread a while back said it may have to do with the screen size effecting the starting point for the random numbers, or something. I’m not saying this is true or not, just putting it out there.
Not a computer person, but I found anecdotally I get better results, generally, the bigger the screen is. Someone in a thread a while back said it may have to do with the screen size effecting the starting point for the random numbers, or something. I’m not saying this is true or not, just putting it out there.
It's highly unlikely to be true. We see patterns in random data, especially small quantities of it like the number of rolls in a D&D session. Once we've seen it a couple of times, or if we've been told to look for it, our brains are primed to notice it, and not so primed to notice when it doesn't happen.
If the die face is chosen by a good enough PRNG (and "good enough" is a pretty easy bar to clear for these purposes), the interactions of the physics model and screen size cannot introduce a bias.
It wouldn't make sense for them to be rigged, anyways. Sure, the fact that they're simulated instead of just randomly generated means that some faces relative to the face that's on top at the start of the roll almost definitely have a higher chance to land, but the face that starts facing up is randomized. Even if the dice were just dropped and not rolled at all, the odds would still be fair because the system does use an actual random number generator to determine which face it's dropped on.
The starting configuration of the roll, including initial orientation of the dice, is randomised. And they're simulated using physics models meaning their behaviour is pseudo random
It wouldn't make sense for them to be rigged, anyways. Sure, the fact that they're simulated instead of just randomly generated means that some faces relative to the face that's on top at the start of the roll almost definitely have a higher chance to land, but the face that starts facing up is randomized. Even if the dice were just dropped and not rolled at all, the odds would still be fair because the system does use an actual random number generator to determine which face it's dropped on.
That is true, and I agree. My expectation coming in was that it was random. I'm sure much more extensive studies have been done by the devs. However, I was curious and bored so I wanted to make sure. I guess if anyone complains you now have pretty solid evidence.
Edit: pRNG is also surprisingly hard to do well. I've messed around with it a little and my models have nearly always been biased towards certain numbers.
Not a computer person, but I found anecdotally I get better results, generally, the bigger the screen is. Someone in a thread a while back said it may have to do with the screen size effecting the starting point for the random numbers, or something. I’m not saying this is true or not, just putting it out there.
That is interesting. The screen size is actually a component of the rng. I may actually look into that as well.
"Rigged" would mean somebody decided to deliberately, intentionally, code up the system to give higher/lower probability to specific numbers than what they should have. This is actually a lot more work and would result in a very large and very noticeable skew of results for everyone who uses the dice. It would be very noticeable.
There is no sound reason for anyone to do this and there has not been a massive complaint from the many thousands of users... So, even without doing any statistics we can immediately deduce the likelihood of this scenario is virtually nil.
Rollback Post to RevisionRollBack
Click ✨ HERE ✨ For My Youtube Videos featuring Guides, Tips & Tricks for using D&D Beyond. Need help with Homebrew? Check out ✨ thisFAQ/Guide thread ✨ by IamSposta.
People often don't choose their words carefully, and "biased" in the random number sense is somewhat obscure usage, and is likely what was meant.
And it's certainly possible to implement a bad PRNG, or use it poorly in a way that biases the results. Without the randomization of the starting face, I'd be pretty skeptical of the "we're doing it with physics!" approach.
There was a dev update (a long time ago) where they showed a dice report one of the devs recieved on a regular basis. If any particular dice were falling outside of the normal probablity, then they would look to fix it. I don't know if they ever had to fix it, or if they're still monitoring a report, but at least the initial goal was to make sure it was functioning properly.
"Rigged" would mean somebody decided to deliberately, intentionally, code up the system to give higher/lower probability to specific numbers than what they should have. This is actually a lot more work and would result in a very large and very noticeable skew of results for everyone who uses the dice. It would be very noticeable.
There is no sound reason for anyone to do this and there has not been a massive complaint from the many thousands of users... So, even without doing any statistics we can immediately deduce the likelihood of this scenario is virtually nil.
That's true. I think a better choice of words may have been "biased." I certainly did not think that the dice were intentionally rigged, and did not give any substantial warrant to the claims that the dice were biased. I wanted to do a cool project. I confirmed that, indeed, the assumption that the dice are not biased is an accurate one.
I confirmed that, indeed, the assumption that the dice are not biased is an accurate one.
Strictly speaking, that's not quite what your result prove. You've shown that the average roll of the d20 is not substantially different to that which would be expected from fair dice.
However, it could still be biased - for example, if it never rolled a 1 or a 20, it would still produce similar results to yours but it would still be biased. Another dimension is association with other numbers. You might be more likely to roll a 20 after you've rolled a 5, say. Another might be that your first roll might be more or less likely to produce a 20 than after say 10 rolls.
There are a lot of these potential factors and relationships that you have to explore before you can declare them unbiased - unfortunately. Maths is a lot of work.
Rollback Post to RevisionRollBack
If you're not willing or able to to discuss in good faith, then don't be surprised if I don't respond, there are better things in life for me to do than humour you. This signature is that response.
I confirmed that, indeed, the assumption that the dice are not biased is an accurate one.
Strictly speaking, that's not quite what your result prove. You've shown that the average roll of the d20 is not substantially different to that which would be expected from fair dice.
True. And, strictly speaking, I didn't even prove that. In statistics its easy to prove something false (ie that the true average is 10.5), but very hard to prove something true (eg if the true average was 10.42 the probability would not be statistically significant (in fact the p-value would be larger)
However, it could still be biased - for example, if it never rolled a 1 or a 20, it would still produce similar results to yours but it would still be biased. Another dimension is association with other numbers. You might be more likely to roll a 20 after you've rolled a 5, say. Another might be that your first roll might be more or less likely to produce a 20 than after say 10 rolls.
Correct.
There are a lot of these potential factors and relationships that you have to explore before you can declare them unbiased - unfortunately. Maths is a lot of work.
Yeah. As said previously, it's really hard to prove something true using purely statistics. Heck, even with science it's a lot easier to prove something false than true (we can set up experiments but why they can tell us an explanation is wrong, they can't tell us if the explanation we think is true or if there's another explanation that could create similar results.)
Good data... the irony is however that the dice roller uses dice physics to determine the result. Meaning that by rolling more than one die, or having too large or too small of a screen can and will change the results.
ie, in a text based random calculation the screen matters not what ever the dice numbers are your results will be spread evenly across 1 ~ x (x being highest number)
Adding the physics of the graphical dice means the calculation will determine after the roll physics are calculated. Which is why sometimes there is dice lag. Also it has a method to prevent skewed numbers which doesn't always work. So that is why the numbers are slightly off of the predicted while still being in range.
Good data... the irony is however that the dice roller uses dice physics to determine the result. Meaning that by rolling more than one die, or having too large or too small of a screen can and will change the results.
As long as starting orientation is randomized (as Davyd says), it doesn't actually matter how the die rolling mechanic works, it could use no rolling at all (just set the dice down without any rolling) and it would still be random. I would note that many people's die rolling in the real world isn't all that random.
It's certainly still possible for DDBs dice to not be random, but it requires making some fairly easily avoidable errors, such as using a low quality prng (cryptographically secure randomizers should be available on any modern OS), randomizing starting position incorrectly (not totally trivial but not exactly hard), or having biased models (with engraved dice, the side with a 20 is lighter than the side with a 1 unless you correct for the weight difference, but DDBs virtual dice look to be painted).
It's certainly still possible for DDBs dice to not be random, but it requires making some fairly easily avoidable errors, such as using a low quality prng (cryptographically secure randomizers should be available on any modern OS),
I'd be shocked if they're not using javascript's math.random(), which is the Merseinne Twister, which is a perfectly fine PRNG. For the amount of rolls in a session and the granularity used, virtually anything would be good enough. Cryptographically strong is way, way overkill. Old-school linear congruential would probably be sufficient, but standard libraries exist, so why go to that much bother?
randomizing starting position incorrectly (not totally trivial but not exactly hard), or having biased models (with engraved dice, the side with a 20 is lighter than the side with a 1 unless you correct for the weight difference, but DDBs virtual dice look to be painted).
There's no way the die models are anything but flat Platonic solids with the numbers applied as textures.
I'd be shocked if they're not using javascript's math.random(), which is the Merseinne Twister, which is a perfectly fine PRNG.
Are the dice generated browser-side or server-side? The animations are obviously on the browser, but I confess to not having looked at the implementation on the web page side.
Not that it matters a lot. You could probably use a BSD rand(3) on a server-side generator and still get okay results because of the amount of noise inherent in multiple different clients using the service at the same time.
Greetings fellow adventurers. I have heard a number of complains that D&Dbeyond dice are rigged. I was curious, so I decided to investigate further. I rolled 2000 dice, 100 at a time and recorded the results. The raw data can be found here if you are interested.
The summary however is that the average of 1042 was slightly below the expected average of 1050. However, this could have been due to chance variation (for example, if you roll a d20 and get 9 that doesn't mean it's rigged, you just rolled unlucky). There is a calculation to determine the plausibility of a result, called a t-test. Using this returned a p-value of 0.58, which basically means that if we assume the dice aren't rigged at all, there is a 58% chance that by random luck you get a value of 1042 or more extreme. This means that we can't make any conclusions. It is likely therefore that I got this value due to chance, given that it could have happened at a 58% probability (5% is usually required to have any plausible claim).
So are the dice rigged? Probably not.
I am an average mathematics enjoyer.
>Extended Signature<
It wouldn't make sense for them to be rigged, anyways. Sure, the fact that they're simulated instead of just randomly generated means that some faces relative to the face that's on top at the start of the roll almost definitely have a higher chance to land, but the face that starts facing up is randomized. Even if the dice were just dropped and not rolled at all, the odds would still be fair because the system does use an actual random number generator to determine which face it's dropped on.
Look at what you've done. You spoiled it. You have nobody to blame but yourself. Go sit and think about your actions.
Don't be mean. Rudeness is a vicious cycle, and it has to stop somewhere. Exceptions for things that are funny.
Go to the current Competition of the Finest 'Brews! It's a cool place where cool people make cool things.
How I'm posting based on text formatting: Mod Hat Off - Mod Hat Also Off (I'm not a mod)
Not a computer person, but I found anecdotally I get better results, generally, the bigger the screen is. Someone in a thread a while back said it may have to do with the screen size effecting the starting point for the random numbers, or something. I’m not saying this is true or not, just putting it out there.
It's highly unlikely to be true. We see patterns in random data, especially small quantities of it like the number of rolls in a D&D session. Once we've seen it a couple of times, or if we've been told to look for it, our brains are primed to notice it, and not so primed to notice when it doesn't happen.
If the die face is chosen by a good enough PRNG (and "good enough" is a pretty easy bar to clear for these purposes), the interactions of the physics model and screen size cannot introduce a bias.
The starting configuration of the roll, including initial orientation of the dice, is randomised. And they're simulated using physics models meaning their behaviour is pseudo random
Find my D&D Beyond articles here
That is true, and I agree. My expectation coming in was that it was random. I'm sure much more extensive studies have been done by the devs. However, I was curious and bored so I wanted to make sure. I guess if anyone complains you now have pretty solid evidence.
Edit: pRNG is also surprisingly hard to do well. I've messed around with it a little and my models have nearly always been biased towards certain numbers.
I am an average mathematics enjoyer.
>Extended Signature<
That is interesting. The screen size is actually a component of the rng. I may actually look into that as well.
I am an average mathematics enjoyer.
>Extended Signature<
"Rigged" would mean somebody decided to deliberately, intentionally, code up the system to give higher/lower probability to specific numbers than what they should have. This is actually a lot more work and would result in a very large and very noticeable skew of results for everyone who uses the dice. It would be very noticeable.
There is no sound reason for anyone to do this and there has not been a massive complaint from the many thousands of users... So, even without doing any statistics we can immediately deduce the likelihood of this scenario is virtually nil.
Click ✨ HERE ✨ For My Youtube Videos featuring Guides, Tips & Tricks for using D&D Beyond.
Need help with Homebrew? Check out ✨ this FAQ/Guide thread ✨ by IamSposta.
People often don't choose their words carefully, and "biased" in the random number sense is somewhat obscure usage, and is likely what was meant.
And it's certainly possible to implement a bad PRNG, or use it poorly in a way that biases the results. Without the randomization of the starting face, I'd be pretty skeptical of the "we're doing it with physics!" approach.
There was a dev update (a long time ago) where they showed a dice report one of the devs recieved on a regular basis. If any particular dice were falling outside of the normal probablity, then they would look to fix it. I don't know if they ever had to fix it, or if they're still monitoring a report, but at least the initial goal was to make sure it was functioning properly.
This is very interesting. I usually get pretty random dice rolls on dndbeyond.
Rolling 1d20...
10
"Big sword, bigger brain"
-BigBrainGoblin
That's true. I think a better choice of words may have been "biased." I certainly did not think that the dice were intentionally rigged, and did not give any substantial warrant to the claims that the dice were biased. I wanted to do a cool project. I confirmed that, indeed, the assumption that the dice are not biased is an accurate one.
I am an average mathematics enjoyer.
>Extended Signature<
Strictly speaking, that's not quite what your result prove. You've shown that the average roll of the d20 is not substantially different to that which would be expected from fair dice.
However, it could still be biased - for example, if it never rolled a 1 or a 20, it would still produce similar results to yours but it would still be biased. Another dimension is association with other numbers. You might be more likely to roll a 20 after you've rolled a 5, say. Another might be that your first roll might be more or less likely to produce a 20 than after say 10 rolls.
There are a lot of these potential factors and relationships that you have to explore before you can declare them unbiased - unfortunately. Maths is a lot of work.
If you're not willing or able to to discuss in good faith, then don't be surprised if I don't respond, there are better things in life for me to do than humour you. This signature is that response.
True. And, strictly speaking, I didn't even prove that. In statistics its easy to prove something false (ie that the true average is 10.5), but very hard to prove something true (eg if the true average was 10.42 the probability would not be statistically significant (in fact the p-value would be larger)
Correct.
Yeah. As said previously, it's really hard to prove something true using purely statistics. Heck, even with science it's a lot easier to prove something false than true (we can set up experiments but why they can tell us an explanation is wrong, they can't tell us if the explanation we think is true or if there's another explanation that could create similar results.)
I am an average mathematics enjoyer.
>Extended Signature<
Good data... the irony is however that the dice roller uses dice physics to determine the result. Meaning that by rolling more than one die, or having too large or too small of a screen can and will change the results.
ie, in a text based random calculation the screen matters not what ever the dice numbers are your results will be spread evenly across 1 ~ x (x being highest number)
Adding the physics of the graphical dice means the calculation will determine after the roll physics are calculated. Which is why sometimes there is dice lag. Also it has a method to prevent skewed numbers which doesn't always work. So that is why the numbers are slightly off of the predicted while still being in range.
As long as starting orientation is randomized (as Davyd says), it doesn't actually matter how the die rolling mechanic works, it could use no rolling at all (just set the dice down without any rolling) and it would still be random. I would note that many people's die rolling in the real world isn't all that random.
It's certainly still possible for DDBs dice to not be random, but it requires making some fairly easily avoidable errors, such as using a low quality prng (cryptographically secure randomizers should be available on any modern OS), randomizing starting position incorrectly (not totally trivial but not exactly hard), or having biased models (with engraved dice, the side with a 20 is lighter than the side with a 1 unless you correct for the weight difference, but DDBs virtual dice look to be painted).
I'd be shocked if they're not using javascript's math.random(), which is the Merseinne Twister, which is a perfectly fine PRNG. For the amount of rolls in a session and the granularity used, virtually anything would be good enough. Cryptographically strong is way, way overkill. Old-school linear congruential would probably be sufficient, but standard libraries exist, so why go to that much bother?
There's no way the die models are anything but flat Platonic solids with the numbers applied as textures.
Are the dice generated browser-side or server-side? The animations are obviously on the browser, but I confess to not having looked at the implementation on the web page side.
Not that it matters a lot. You could probably use a BSD rand(3) on a server-side generator and still get okay results because of the amount of noise inherent in multiple different clients using the service at the same time.
It seems to be client-side; I turned off my wifi, and could still roll dice. It breaks the records in the game log, though.