About the battlefields within battlefields thing, that is something that would not be so easy to balance in well.
However, I thought of something. The current "free everything if I am massively overwhelmed" logic is good, and smart play, but right now, planets are frequently too weak, so this logic gets triggered nearly all the time.
Something I thought about is possibly increasing the amount of ships AI planets initially seed with by a good amount (but not so much that every planet becomes a grind), making it harder for ships to be freed (like not free a group of ships if only one of them gets shot at, but maybe only when what they are guarding <50% or dies, whichever comes first, instead of the as soon it is shot like it is now; or possibly have a large percentage of the defenders be
permanent defenders that will never be freed until the planet is lost). Reinforcement rates may need to go down some to ensure that planets don't get grindy too fast, or to prevent early game border aggression.
The idea behind this change is to make each planet much better defended, thus making it harder to trip the "free everything you can" switch, but make it harder to put ships into threat in the first place, so you don't have to worry as much that the stronger planets will instantly kick your butt upon the first attack, and you don't have to micro quite as much to prevent your ships from freeing stuff unintentionally, as small amounts of aggression won't free as much.
This would require a large amount of balance work, but it may be something worth thinking about.
EDIT: Or if this is too much to balance and change, how about an easier variant of this idea? Like making it harder to free ships (like some of the ways suggested above), but increasing reinforcement rate some to compensate.