It is problematic that benefits accrue to those who already have them—things like being approved for loans for because of having assets already. Being let go for a minor offense because of not having an arrest record. However, it is not be unfair just because of a particular "-ism" involved, but because of the underlying assumption that someone is more deserving or "should" be treated differently. There is a common trope that "X would never happen if we had a woman president", or "some [group] have natural rhythm/can dance", "you know you can't trust anything they say because those people always lie". Whoever is included in that group can change; we saw it with waves of immigration in the 1800's. Whichever was the largest group last off the boat were the scapegoats. "White-ness" (aside from being an invention) was ascribed to longer-standing immigrant groups and withheld from the more recent ones.
Here is the heart of the problem: as long as someone can be marginalized, anyone can be. If some negative stereotype can be given to a group, it can be given to another. As long as it is accepted that there are different "kinds" of people, that some are inherently better (or worse), and that there exists some sort of "natural order", we are at risk of this happening. The only solution is to do away with all of them altogether.
Once no one can be labelled as "the problem" or "those people", then we will have peace. Until then, while we may make progress, it will be a constant vigil that another group is not vilified.