Generally, for the AI to attack worlds behind your front lines, they have to travel through your front line planets first. Ships which are in Threat mode (as opposed to guarding whatever they spawned next to) will happily fly into your systems via wormhole if there's no defenses to deter them. Waves and CPAs put ships directly into threat mode. Exo strike forces are even more aggressive and will fly through wormholes to attack their chosen target with a reckless disregard for your defenses.
if the AI has unlocked ships with cloaking, then they may be able to travel through your planets without your defenses being able to see them. Tachyon Beam Emitters are very cheap to unlock and will decloak ships that come into range. Covering a wormhole with a force field will prevent most ships from using that wormhole to leave the system, as well. Note that Raid Starships are fast and can fly through forcefields; they may be able to slip past your defenses and get to your less-protected planets, but they're fairly vulnerable to Sniper Turrets.
The AI has a very limited number of ways to attack planets behind your front lines, most of which are disabled by default nowadays. Some examples:
* If you're playing at higher difficulties and the AI happens to unlock Warp Guardians, each warp guardian is effectively a walking warp gate.
* If you have Human Marauders on, they can attack any of your systems and approach from the edge of the system, but those aren't actually the AI, they're a minor faction.
* if you have Counterattack Guard Posts AI Plot enabled, the galaxy will be seeded with Warp Counterattack Guard posts. When destroyed, they'll send a wave to one of your systems. They can send that wave to any of your systems (though I think they have a 50% chance of going after the first human homeworld), and the ships will appear at the edge of the gravity well, not from a wormhole.
* The Warp Relay plot allows idle threat ships to build warp relays, which allow waves to be launched at planets within a certain number of hops of the relay.
* Mining golems appear from the edge of the system if you have that minor faction enabled. These guys are a pain as they can attack any system on the map.
So, in short, the AI generally has to travel via wormhole. If the wormhole connections are such that to access planets A, B, and C, you have to fly through planet D first, then yeah, you can take A, B, C, and D, and fortify the heck out of D, and the AI will have to fight through D to get to A, B, or C. This is generally an excellent strat.