Oh god, yes. Yes yes yes.
Like... on one hand I kinda get what the general advice is about; it is easy to get wrapped up in your own details to the point where it gets in the way of the story you're trying to tell, or at the very least it may feel that way to the audience you're trying to rake in. But as you said, different people have different creative processes, and I too end up very much taking a bottom-up approach where I'll develop a reasonably fleshed out idea of the setting I want before I've actually written a single plot point; for me this creates an ever expanding repository of creative ideas I can put to use as needed, even if much ends up never being told at all, or only vaguely implied.
Sometimes the worldbuilding is the entire point! Sometimes, a lot of it is needed behind the scenes just to get things to come together in a believable manner; and sometimes, the amount of worldbuilding done and put on display is perfectly in line with the author's intent, but the real problem lies in the type of media chosen. You rarely see this come up with things like video games, visual novels, hell, even wikis as opposed to movies or literature, for a large part because one allows the audience to manage their own attention curves while for the other, the author has to do that all by themselves.
But then I guess it's also important to remember that no matter what you do, you're not going to please everyone, and sometimes people will simply complain because they're not part of the target audience.