Try this in difficulty 9+.
I play on 10/10, and based my suggestion off of that (the planet one).
On Diff 10, you can still be at 10 AIP and face 2,000 or more ships in wave.
About 400 triangle ships is the max I have seen; 2,000 would be swarmer ships.
With 2 DC raids, you can do four nebulae at 10 AIP. By the time you absolutely have to increase the AIFloor, you will have at least one modular fortress.
One further clarification: failed nebulae would not count toward this.
The 2,000 ships was fighters; however the game is a 10/10, 2HW, 8 Champion game.
The nebula in that game are poorly distributed for me; of 12, 9 are 9 or more hops from my homeworlds. 2 minutes of deepstrike produces close to 2000 threat anyway, meaning I am most certainly paying a price for attempting those nebula. So, because of distribution, I am already forced to either pop worlds, or rack up lots of threat. While similar to your proposal, it means I am in control of the punishment I get for trying the nebula, rather than the reverse.
As for the AIP being handled by AIP reducers, remember that you cannot rely on those. I just finished a 9/9 game that had only 3 Data Centers on the entire 120 system map. One of the Co-Processors was on an AI Homeworld (with Core Raid Engine, of course). Finding 2 DCs early in those circumstances? Way too hard to rely on.
Next, what do you do if you don't unlock a modular fortress in your first 2/4 nebula? As is, that's not too unusual. Fine under moderate difficulties, doom at Diff 10.
Overall, I think your suggestion simply gives too high a chance of the RNG saying 'gg', no matter what you do. So, I'd prefer something a little less drastic.
An additional question - Are you suggesting that the human player must control 1/2/3 non-HW planets? Or merely destroy that many AI command stations? What would happen if the player lost a planet while the nebula was ongoing? If all you had to do was destroy an AI Command Station, then I'd be less opposed.
Really, if there's a need to limit access to the nebula, I'd prefer that it be something that the player does to trigger it. Taking a planet is bad (in my mind) because of the extreme AIP punishment at high difficulties, but other than that I do like the idea of triggering nebula. Fallen Spire manages to handle the first few stages in an exciting manner without either (major) time or AIP costs... not sure how to replace shard retreival, though.
Have the ability to manufacture something that reveals the next nebula? Cost starts out low, but increases as you win more nebula. Combine with a 'drive calibration' timer to force a minimum delay, maybe. (Make the facility a perma-cloak invulnerable object, to mark where the Shadow ships spawn. Didn't the lore say something about a 'beacon' at the home command center?)
edit: I keep forgetting, since I don't do multiplayer, but should probably keep in mind that for some players, the Champion may be their only ship. Forcing them to wait around for other things to happen may not be a good idea.