The irreducibility of restrictions of unitary representations to lattices