Seneca said this quote

It is regret for the absence of his loved one which causes a mourner to grieve: yet it is clear that this in itself is bearable enough; for we do not weep at their being absent or intending to be absent during their lifetime, although when they leave our sight we have no more pleasure in them. What tortures us, therefore, is an idea.