Edit: I took a long time to write this post. This was pretty much written with only the first post in mind. Oh, and please share your version. I don't know from the release notes whether this was relevant to CPAs, but the AI *was* ignoring unit caps in waves for one release.
First up, hunting before a CPA can actually be detrimental (or can allow you to focus where the push comes from and when). The size of the CPA is determined at the announcement, but the ships freed are determined when the CPA actually occurs. In other words, unless the AI runs out of ships in the entire galaxy (possible, perhaps), a 5000 ship CPA will always be a 5000 ship CPA no matter how many units you kill after the announcement, and before the timer hits zero.
The AI first tries to free ships at the appropriate mark, but if it unable to find enough, it then looks at lower marks for availability, only going for higher marks if any lower are not available, IIRC. This isn't generally something to be concerned about, but... it can matter, depending on what you're doing. Long story short, don't hunt units currently in guard mode. Turn on the threat filter on the galaxy view and hunt down existing threat, since that is ON TOP of everything that will be freed in the CPA. I've customized the threat filter to T and the Hybrid filter to H, but those aren't default settings, I just use them enough. You can access those filters through the galaxy map user interface, even without a keybinding. It's just too useful not to bind, though.
Specific advice would require a better idea of the configuration of the game. General advice:
Build a few lightning warheads at key locations. Place them under a forcefield (on your territory) next to a hostile wormhole where threat is likely to gather. Once you see threat gathering around the wormhole to attack, but before it moves into your system, send in the lightning warheads. Note that missile frigates are immune to area attacks. This does cost AI Progress, so avoid overusing it, but it certainly costs less than a game over. With this strategy it actually pays to stack as much firepower in a single frontline system as you can safely manage, in order to provide a bigger cluster of enemies to warhead.
Defensive depth, logistic stations, mines, spider turrets, and riot starships carefully kited can significantly reduce the danger of a CPA and give you a chance to defeat the enemy in detail - depending on the layout of the map.
It can actually pay to leave a system mostly undefended. If you have one chokepoint they can approach through and all your forces wait there, then the AI will stalk the system until it decides it has a decisive advantage, and lacking reserves, you're all out of luck. If you wait in a nearby system, or leave a good portion of your forces in reserve, then it will attack sooner and with smaller taskforces, that you'll ideally be able to handle once the reserves come to the rescue. That said, I second the sentiment that your CPAs are a bit higher than I expected in a typical singleplayer game, though I can imagine a few scenarios that would reasonable cause CPAs of that size.
I repeat, don't go hunting for inactive units and barracks EDIT: Unless it actually works, in which case, nevermind. Aim for loose threat and carriers.
You can establish a beach head of a fortress and various turrets in AI territory adjacent to yours. It's not something I usually do, but it is an option if you need to fight the AI a bit further out.
Try logistics stations stationed at the edges of systems. They'll help your force respond, as well as buy you time to react to ships that slip past your front lines. Try mark 3 military stations at your chokepoints. The damage multiplier is very nice in all situations, and the teleportation attacks can't hold off an entire task force, but can hold off many of the smaller attacks for a rather long time.
I'm sure I have some other tips, but I can't remember them at the moment. I'll post 'em later.
