I was chatting about this briefly on a discord, but I've been thinking about it since then and I'm going to post here to organize my thoughts. I'm not an expert though so don't take this as truth.
Many people talk about the cycles of innovation - the idea that, metaphorically, big trees eventually fall and then the undergrowth goes wild until some of them turn into big trees and the cycle begins again. Or dinosaurs and mammals, although that's less obviously cyclical.
But that metaphor suggests that there's an inevitability to this that's not really present, I don't think. There's more dependent on outside forces making something change that allows for this - the chainsaw cutting down the tree, or more often the slow shift of climate that makes the trees die off.
Anyway, here's what I think happened. There were a LOT of different things that coalesced at the same time to allow the rise of the experimental wave of games. A big one was the advent of Steam, and digital distribution more broadly. In the early to mid 2000s, it's bizarre to remember, but most games were sold on discs in stores. That costs a lot of money and there's huge gatekeepers blocking the way in. It took years for Steam to be seen as a viable store, and for them to start adding non-Valve games, but suddenly there was a viable path to players that did an end-run around the gates.
The internet was also happening, and this exposed more people to more types of games. Importantly, it also exposed people who did not think of themselves as game players to game makers, and people who did not think of themselves as game makers to the possibility of making games.
Another piece of the puzzle was similar processes happening in games journalism - thanks to the internet and other factors a bunch of journalists decided they wanted to write about games in new ways, new types of games for new players, and a lot of people also decided they could write about games. So this meant that the places that people looked to learn about games were suddenly as or more excited about weird experimental small games as they were the bigger ones from existing parties in staid genres.
Similarly in the big studios, more people were becoming excited by new ideas and contemptuous of the same old way of doing things - people who knew their craft but wanted to do something more experimental.
And then also at the same time another set of gates were being bypassed on the making side of the equation - cheaper, easier tools were there for making games (mostly Unity to start but as the wave grew also Twine, GameMaker, etc) and suddenly you didn't have to make your own tech from scratch or pay a company millions to license something.
And lastly I suspect that the Free Money era had something to do with this too - lots of dollars floating around looking for something to attach to, fueling publishers' and platforms' high risk tolerance. But I know little about that so I'll leave it there.
So, gates breaking left and right - cheap and easy tools, a viable path to players, a trusted cheering section, a diverse, angry, curious crowd of creators, an engaged and open-minded audience eager for New things! It was a really cool time that feels so far away now. And what happened is that the trees grew and the paths around the gates got new gates built on top of them.
That's not to say that indie games are dead, or creativity and experimentation aren't happening any more - they clearly are. But in every "how to sell your game on steam" article or Twitter thread you can see that there's rules and gates and people to be appeased - and less appetite for weird stuff because you need to fit into the right holes to make it through the gates. Steam isn't the hidden path any more, it's the main road. Unity isn't a secret weapon, it's the standard. Queer stories, narrative-focused experiences, experiments mechanics, audiences beyond the young white dude aren't a rising groundswell, they're the reality of gamers. This is great!! I'm glad the definition of games and gamers has expanded, but it is harder, in my opinion, to push the boundaries further now.
In order for this to happen again, we need a break in the dams somewhere, or really in multiple places at once. Valve and Steam are the new gatekeepers, using algorithms to decide what is relevant to players. Unity has become a tech company chasing the latest New Thing in hopes of boosting stock prices and getting acquired. Games journalism has had incredible wounds from multiple places. The internet is becoming a broken shopping channel.
Again, I hope this doesn't come across as me saying "no one is making weird cool commercial indie games like we used to!" From my perspective, though, that mountain has gotten steeper and harder to climb. That's my last inept metaphor.
I didn't really think about that when I first made this post - indie games struggle against various headwinds, for sure, but it seemed like the barn door was open and that horse had sailed. Now it's pretty clear that things are not fragile than that - people will not stop making brilliant new games, but if enough barriers are placed there will be fewer and fewer. A lot of my very favorite people will be unsure what their future looks like, and as a player and an appreciator of the art form it makes me sad to think of the things the world will miss out on.
