George Bancroft said this quote

The fears of one class of men are not the measure of the rights of another.