Is the possibility for a peaceful resolution to the game meant to be over if you wait more than... What is it, a year or two after getting a third Federation member? Either way, after that your hostile alliance should form quickly and automatically. According to
this comment by Keith seems to say that you cannot pry a species out of their alliance, as they won't join the Federation once they've joined the non-Federation alliance.
So once that alliance forms, are you reduced to destroying that species?