I actually like playing with them on. The constant fear of having one of them showing up in a wave makes every incoming wave exciting.
Right now, I'm playing a 7/7 round without them and I know my home world is impenetrable. If I had beachheads on, I knew I should at least keep my fleet nearby just in case. The reduced amount of ships accompanying the wave makes it easier for a fleet to defend against it.