Hi.
I play a lot like you do: taking a ton of territory. I'm currently generally trying to learn how to take less and play more "normally", but in this case...
You said you hadn't built the Spire Cities yet. That was your mistake, really. If you'd built them earlier, by now you'd have enough power from them to challenge the AI directly. I'm not sure you can do it at this point (the AI will launch attacks when you start building cities), but if you can then you could still win if you can get some cities up.
Aside from providing you with a strong defense from the cities themselves, the true power of the Spire Cities is that they let you build the Spire fleet. If built properly, you can get 2 (occasionally 3) shipyards per spire city. That means each city gives you two Spire destroyers and a Spire cruiser. Those ships have absurdly powerful photon lance guns (the Cruiser's is over 3 million a shot)... and they mount modules. Crazy modules. The Cruiser flies around with shields, laser batteries, sniper rifles... and heavy beam cannons. Yeah, that big planetary gun? That ship can mount two of them. It's awesome.
With the amount of territory you have, you'd have had no problem by now having the knowledge to unlock everything you need. If you get five cities, you can build the galactic capital and unlock Spire battleships & dreadnaughts. The dreadnaught can basically mount a plasma siege starship as a gun. It hits for millions, and does area effect damage. It's a one ship fleet. All these ships (even the destroyer) also mount shields, which means other ships can hide beneath them and fire from safety. If that's not enough, one more building after that unlocks an alternate win condition where if you survive for 30 minutes against crazy AI assault... well, something cool happens and you win.
This morning I beat a 6000 ship carrier fleet at a planet (I've got a game going with AIP 600, just to see how far I can go ignoring AIP) using the spire ships from two cities (5 destroyers, 2 cruisers, and a bunch of frigates), a couple of Flagship mk II's, a Botnet Golem, and an Armored Golem. Those weren't mk V ships in the carriers, but it does give you an idea of the firepower that Spire cities let you bring out. The idea behind the cities was to allow someone to play like you did here: to try to conquer the map and assault the AI directly.
You could try using some warheads to weaken the carriers and attack them with your fleet. If you win that, rebuild your fleet and start doing spire cities ASAP. That's the only way I can see to win this one. (Alternately chalk it up as a good learning experience and try again!)
Here's a couple of other tips that might help, though it might be too late for this game:
1. If you don't need to, don't put core worlds on alert. You can avoid that by not taking a planet next to one and using transports to attack i the end, or if you do take a nearby planet, build a Warp Jammer. You don't need to neuter a planet that's next to a warp jammer unless something else is putting it on alert.
2. If you're not going to build boatloads of Spire Cities, you need to take less stuff. You don't need to take every capturable. Focus on some key ones and ones that are easily defendable.
3. If you're trying to take as few planets as possible, turn Spire Civilian Leaders off. If you're taking tons of planets, leave them on. (With few planets they're a liability since you can't likely capture enough.)
4. Aside from using transports, you can also use the warp gates (on the CONST menu) to get ships where you need them to be. Those are special buildings that whenever a ship of the given type is built, it warps to the gate immediately. If you want to attack the AI from a world that's cut off from your territory, you can use transports to get there and use the warp gates to replenish any ships you lose by having them teleport straight over. Pretty handy when you launch a homeworld assault and you've got a beachhead planet near the AI but don't own anything else near it.
5. Using ships like leech ships and parasites work best if you bring other ships. Those ships themselves don't do high damage, and need the help. As Keith mentioned the reclamation damage "jumps" around if it doesn't do enough on one ship, so it adds up onto other ships and you'll reclaim more with fewer losses. The Flagship line is a particularly useful one, since it grants an attack boost to the other ships. (The Botnet Golem doesn't need the help, but it does tend to need an escort if reclamation immune ships show up.)
6.
All activated extras are on easy and 3/4 max with the exception of botnet golem which for some reason I put on medium.
Well, the Botnet on medium is worth 100 AIP by itself. So that's something to think about (as compared to easy). Also, 3/4 max setting depending on what it is could be good, or it could be bad. If it's something like the Fallen Spire setting, setting it up to 7/10 will make the AI's counterattack more difficult to deal with vs the default of 4/10. Same thing with hard golems, 1/10 is the easiest and 7/10 would be quite difficult. For helpful things, the higher numbers make the help more powerful.