cuz 1. you're privileged as a white person to get away with saying that shit without people getting mad at you and 2. there's always this implication that they're "one of the good ones", as if dumping on other white people somehow makes them better
the big problems are structures and behaviors that perpetuate existing inequality, such as nepotism and hiring people as "culture fits"; generational wealth; explicit attempts to disenfranchise people of color; etc.
i think some liberals fantasize about not having white guilt, while some people further to the right just really really want to be allowed to say the n word