Do Your Dice Roll True?

The founder of GameScience, Lou Zocchi, has long claimed that GameScience dice roll more true than other gaming dice. In a well-known GenCon video Zocchi explained why GameScience dice should roll more true.

His logic is that due to how dice are made, traditional RPG dice are actually put through a process similar to a rock tumbler as part of the painting and polishing, and this process causes the dice to have rounded edges. In theory the uneven rounding gives the dice an inconsistent shape that favors certain sides. GameScience dice are not put through this process, which is why they retain their sharp edges and is also why their dice come uninked.

While Zocchi’s makes a good argument about egg-shaped d20s, what was lacking was any kind of actual testing of how the dice roll. Nowhere were we able to find any tests of d20s — either GameScience or traditional d20 dice — to determine whether or not they roll true. As giant fans of dice and an impartial third party, we decided to run a test ourselves and see just how randomly RPG d20s really roll.

We pitted GameScience precision dice against Chessex dice (the largest RPG dice manufacturer) to see what science has to say.

Methodology

For the principle test we used one Chessex d20 and one GameScience d20, both brand new right out of the packaging. The GameScience d20 was inked with a Sharpe to make it easier to read the results, but the dice were not modified in any other way.

The dice were rolled by hand on a battlemat on a level table. For this experiment the dice were rolled on the surface for at least two feet and had to bounce off a flat backstop before coming to rest. This is similar to the requirements of craps tables in casinos. Our logic is that if this method successfully prevents cheating with six-sided dice, it will more than suffice for d20 dice being rolled without any intent to alter the results. (Since casinos are not losing money on gambling, we assume they know what they’re doing).

Each die was rolled 10,000 times, and the results recorded.

Test Results

After an insane amount of dice rolling, here is a quick look at the results for each die:

A casual analysis of the results suggests that neither die is rolling randomly.

If we had a d20 that rolled perfectly, each face would come up 500 times. But of course randomness isn’t perfect and we’d expect some deviation: over the course of 10,000 rolls we’d expect, with 85% confidence, that each face would be within about 33 of 500 — so anywhere from 467 to 533 is within the bounds of randomness. (At 95% confidence the margin of error is 45). Neither die falls within these bounds.

The Chessex d20 had a standard deviation of 78.04, and the GameScience d20 had a standard deviation of 60.89.

While neither die rolled true, it’s certain that the Chessex die rolled less true, with a greater degree of deviation from the expected range across more of the dice faces. Interestingly, the GameScience die actually rolled very close to true except for the number 14 which rolled vastly less often than it should have, farther off than any face of the Chessex d20. Applying the results to a Chi Squared test also confirms that neither die is rolling randomly (even if you ignore the 14/7 on the GameScience die).

GameScience 14 Theory:
We have a theory as to why the 14 rolled so infrequently on the GameScience d20. Every GameScience die has a small chunk of plastic that sticks out of one face. This flashing is from where the die was removed from the mold. It occurs on all dice, but in Chessex dice this flashing is removed in the polishing process.

On GameScience 20-sided dice this flashing is on the 7 face — directly opposite the 14.

It seems likely that it is more difficult for the d20 to land on the face with the flashing sticking out, pushing the GameScience die off that face. In other words, this flashing makes the 14 roll far less often than it should. Since the flashing position is set from the mold, all GameScience d20s should have the flash in the same position (and all in our inventory do).

Some Confirmation

Since this test was simply one d20 from both manufacturers, it’s possible we just happened to choose the only Chessex d20 that didn’t roll true, and the only GameScience d20 that rolled far fewer 14s. As a check on our results we took another new d20 from both Chessex and GameScience and rolled each under the same conditions.

After 1,600 rolls the same pattern emerged (incidentally, the standard deviation after 1,600 rolls was almost identical to the 10,000 roll test). The Chessex d20 still had more deviation from expected than GameScience, and the GameScience d20 rolled massively fewer 14 results. Both dice still rolled sufficiently out of true to be beyond the margin of error. So this quick (well, not so quick) double check is some confirmation of the 10,000 roll test.

So Which Dice Are Better?

It’s worth stressing that based on our tests you would need a lot of dice rolls before you saw a meaningful difference in any of these gaming dice — roll a thousand times and maybe you’ll see 5 or 10 less of a given number than you’d expect (or more). So for gaming purposes both dice will work just fine. Seriously.

But that said Chessex dice (and in theory any rounded-edged dice) are going to roll less close to true. Because of the randomness of the process that changes the shape of the dice, there’s no way to predict which faces are going to roll better or worse. Indeed this means that you could have dice that are “lucky” and roll high more often or crit more often, and “cursed” dice that seldom roll 20s and fumble more often.

With GameScience dice, on the other hand, you know that the 14 will roll substantially less than any other result — so technically the dice will roll low, but the 20 should roll just about as often as the one, or the 10. If you carefully cut off the bump on the GameScience dice with a sharp box cutter or Exacto knife you should get a result that is very close to being truly random.

Raw Data

Here is all of the data from the 10,000 roll test, so anyone who wants can subject the numbers to their own statistical analysis. We’re including in here the percentage that the rolls of any given number deviate from the expected number of 500 per face.

This is Just One Test

In the world of science, this is just one very small test. To have relatively certain results we’d need to replicate this test across many different Chessex and GameScience dice — if anyone is interested in running their own test to corroborate or contradict our results, we would love to hear about it!

Once our wrists recover from all the rolling, we may consider a second test ourselves — specifically to confirm the theory that the flash on the GameScience die is what is causing the 14 to roll so low: we want to carefully sand the flash down and retest the same die to see if it then rolls more true.

Disclaimer: we have made every effort to ensure that our testing methodology was as fair and accurate as possible; however, without much more testing we cannot say with certainty whether one kind of dice roll better or worse.