I feel like the main conclusion I've come to after 10+ years of working on games is that good tooling is secretly one of the most important things in basically any production of any scale
This is true of pretty much any software project. Mozilla has a ton of tools:
- Crash Stats to analyze and inspect crash dumps
- Searchfox to index the source code of Firefox for semantic searching
- Bugzilla for bug reporting and tracking
- Pontoon for managing localization of various projects
- Treeherder for tracking commits and the results of various test suites for Firefox
- A Phabricator instance for code review
- And a billion others
Interestingly, Mozilla's tools are often standalone / split into several tools that perform a single task.
Companies tend to push for unifying tooling as much as possible behind a single UI (often a mega-CLI command that has like 100+ subcommands) under the assumption that this will be 1. easier for users as they don't have to remember/relearn a bunch of different tools, and it will be 2. easier for developers since they don't have to manage separate deployments or handle a new build pipeline, etc. etc.
In my experience, this is often paired with under-staffed and centralized tooling teams trying to cope with way too much work—tooling teams are a dependency of basically every other project at the company and often have to staff an oncall rotation or a help desk to try and figure out issues for product developers. Management sees that tooling is often a form of automation and assume that means they don't need to staff the teams as much since they are making themselves redundant as they build, thereby making the team that is already critical to the rest of the company even less able to function.
Mozilla (at least when I worked there) invested way more heavily in tooling staff than anywhere I've worked since relative to their size, and there were multiple tooling teams working on different areas of responsibility. In addition, teams were empowered to set up and run their own tooling as needed, sometimes with support or infra resources provided by the full tooling teams (e.g. we had a Heroku org set up for a long time to let devs from any team spin up backend servers for whatever little projects or tools they needed).
It turns out that many standalone tools and services are more resilient in the long term than unified tooling systems. Replacing individual tools is much easier when they're standalone, and you don't get bogged down in trying to make the new tool you're making fit the "mental model" of the unified tool. If a webpage works best for your UI, make a webpage! CLI is better? Do that! A Slack bot? Go for it homie. Desktop application? Fancy.
Sure, the UIs tend to vary both in quality and style depending on who makes what. Some tools end up being unowned and causing issues when they need maintenance and no one is around who can do it. Tracking all the available tools can become a nightmare. These are all the problems that push people to propose unified tool systems, but I'm pretty convinced after working on tooling for like 10 years that the absolute most important function of tooling is resiliency over time. Tooling that is hard to use is a project; tooling that isn't working is an emergency. And systems distributed over several independently-operating nodes are always more resilient than centralized systems at the cost of efficiency.
