I am writing this with singleplayer games in mind, for various reasons this won't directly apply to multiplayer games. Keep that in mind.
Games are a process. As you play something and improve at it, your relationship changes. Both you and the game go through different phases which never linearly build on each other. Rather, different parts of the whole get emphasized or diminished. IMO, it's very useful to see these phases as somewhat distinct sub-games (& player types) which exist in a state of tension/contradiction.
Usually these phases are split into 3-ish categories that correspond to a player's skill level - beginner, intermediate and expert. Let's say that beginners are figuring out the game and cannot play with intentionality - they are concerned with learning. Intermediate players can play with intentionality and start applying what they've learned. Expert players start thinking of games in a more holistic, meta level and are concerned with creative solutions + optimization.
While these phases are already helpful because they let devs figure out how to make the best subgame for each of the phases, looking at how these stages conflict is even more interesting. To demonstrate this, let me focus on a particular contradiction between intermediate and high level players - that of RNG-driven decision making and optimization.
When looked at in the context of optimization (speedruns, scoreplay) game decision making is a bell curve. At low levels of optimization, all game states are viable but none of them "weigh" much - so while there is choice, there's not enough conflict to create meaningful decision making. As a game gets more optimized, the states start to fatten up - suddenly choices mean something and players have to think. When the game's optimization surpasses a certain threshold, though, some states become too thick and start breaking the scales - decision making gets diminished because the suboptimal choices are obvious.
This is all fine and expected - no game can last forever, and singleplayer depth is ultimately a type of obfuscation. Things get trickier when you add RNG into this process, though.
RNG has the potential to make that middle phase a lot more interesting - it shuffles around the fatness of the game states and makes you estimate and decide which state's fatter at very hihg speeds. Sometimes, it deprives you of desired peak fatness and forces you to quickly asses which chubby states you can go for instead. It's fuzzy, varied & unpredictable.
As a game gets optimized, the less viable, thinner states will get filtered out. But unlike in the example without RNG, you have a bigger problem than just your game getting potentially "maxed out". Now you've made the process of maxxing it out pure luck based trash, hell on earth. Players will now get arbitrarily locked out of the desired states at no fault of their own.
This problem might seem like it happens at the highest of levels, but this isn't true - depending on how much RNG is involved and how much it affects your outcome, this can even become a problem at an intermediate level, long before the player maxes a game out. If I'm doing a run and get screwed over cause I'm unlucky at the start, it often doesn't even matter if I could compensate for it later on because I'm thinking about my moment-to-moment performance, not the game as a whole.
Games like this tend to be at their best when the RNG and optimization are balanced in a very particular way - they have to be optimized enough so choices have meaning, but not enough to counter the averaging out effect of the player dealing with a shit ton of instances of RNG. I call this phase The Wiggle Level - the player's stuck in a box but they can wiggle around and it feels nice (note : I will never call it this again after I finish this article). This can either be achieved with capped, built in optimization challenges (S ranks, counter stops, etc), or with low levels of competiton cuz nobody's playing your shitty RNG driven game.
Thinking about The Wiggle Level as a distinct skill level/phase of play helps clarify a lot. Some games might be good at the Wiggle Level but not High Level, some games might suck at the Wiggle Level cuz the decision making dries up too fast, but become fantastic at High Level because the consistency allows them to focus purely on maxxing out a game through skill.
Realistically though, this doesn't matter too much. If players get to either level - you've won. The modes are fundamentally incompatible, but their conflict can be minimized a lot. With enough tuning and luck, you might be able to make a game that keeps players of both levels happy enough. Many players also recognize this conflict and accept that games will boil down to luck. But I think it can be useful to decide if you wanna focus on the wigglers or the maxxers during the conceptual or early design phases, because it stops you from wasting your time trying to reconcile the 2 by somehow making a game that's decision making heavy but also very optimization-friendly. Often it's not really worth it.
