Ralph Waldo Emerson said this quote

The selfish man suffers more from his selfishness than he from whom that selfishness withholds some important benefit.