Friedrich Nietzsche said this quote

The vanity of others runs counter to our taste only when it runs counter to our vanity.