I've never really asked women in "my circle" of friends if they wear matching underwear. So, I have no idea how many (when you say "so many"). However, maybe matching underwear isn't a big deal to the women who wear neutral colors.
Personally, I could not care less if my underwear matched.
If that's all you get to enjoy, have fun with it.