I'm poking around on anydice again, and I'm trying to figure out a set of things that works nicely with "reliability". GURPS kind of has this right where you raise your Attribute, and thus your target number, to roll under for 3d6. The Roll Under is why I'm working the way I am.
These are the averages for various dice for the following rolls: 2d6, 3d6 keep highest 2, 3d6 keep lowest 2.
d4: 5, 6, 4
d6: 7, 9, 5
d8: 9, 12, 6
d10: 11, 14, 8
d12: 13, 17, 9
d20: 21, 28, 14
I did d20s for fun, I don't want to use them because the step down from d20 to d12 felt too sharp.
I took the most probable outcome for each number just as a baseline. My thought for the game was this: If you are really bad at a skill, akin to a rank 0, you'd start at 3d12h2, to keep the highest 2. As you improved, you'd go to just 2d12, and then to 3d12L2, and after that, down to 3d10H2 and so on. The problem is that when you keep lowest 2, and you take that average, it seems like staying at the Keep Lowest 2 step is mathematically better than the going down a step.
Looking at the "At Most" option and picking an arbitrary DC of 8, the D12 and D10 spreads have the following chance of getting an 8 or less.
3d12H2: 6.1%
2d12: 19.44%
3d12L2: 40%
3d10H2: 10.6%
2d10: 28%
3d10L2: 52%
So this dice system isn't going to work.
Unless I'm misreading the outputs.
Rolling Under is the best way, to me, to capture that reliability. It's shrinking the margin for error. It's setting a baseline. Raising your floor. A good archer isn't someone that can hit the bullseye once, but miss the target with 4 other shots. A good archer hits near the bullseye with all 5. Shrink the possible outcomes, get better results. This also allows you to completely eschew dice rolls against opponents that are of a certain tier. If you're rolling 2d8, and you need to roll a 16 or lower, you don't need to check, period.
I'm going to have to figure out a good way to shrink the margin for error. I could just keep the d4-d12 system because going down in dice size feels fun, and damn the math, bu that's something I'd need to tinker with more.
Ultimately, I want to make a system that's fairly generic but aimed squarely at telling sports stories through a tabletop setting. Something like a Haikyuu or a Friday Night Lights, or anything like that. The skill list will be fairly generalized things and the goal is to get the feel of competing against some rivals and stuff like that. I need to watch more sports dramas though to get the feeling right.
Math sucks, game design hard 
