What to do with lillies when they finish flowering

I have lots of lillies in pots and in the ground. They have largely stopped flowering by now and I am wondering if I should let the stalks die back naturally or cut the stalks. Would appreciate your advice