As a frame of reference, whenever I place turrets, I use ctrl-shift-clicks (meaning: groups of five, repeated).
Before anything else: I generally play on difficulty 7. Sometimes, my turrets stop waves. Sometimes they don't. Usually, I need to bring in fleet ships for backup anyway. However, the setup I'll outline generally puts a stop to random small-scale AI incursions that occur between waves or as a result of my raid starships decimating guard posts on nearby worlds.
There are two types of turret setups in my mind - "casual" and "wave defense". The casual setup is meant to blow up the occasional AI stragglers that wander into that system (it'll happen occasionally, regardless of how badly you don't want it to). Put down some sniper turrets for really small groups, and maybe some missile turrets for medium sized groups.
With wave defense, it's all about repeated evaluation. Start with a blob of 25 of each turret. Missile Turrets to do damage, Basic/Laser/MLRS to take fire (they don't seem to do nearly as much damage as missile turrets but they do absorb fire), Sniper Turrets to shoot down long-range jerks. I've also had positive experiences with Heavy Beam turrets and heartily recommend a few of them for wave defense if you can spare the knowledge. As Keith said, put tractor turrets around the hostile wormhole, I recommend enough to capture half an average wave. If the bad guys have lots of Neinzul ships, plop down a bunch of flak turrets too. I occasionally bring in starships (light, maybe flag+) to power up my turrets as well. If you've got the resources, plop down some minefields around the wormhole, and maybe some lightning turrets next to your tractors.
Once you've set up, see if your setup stops a wave. If it doesn't and you have to bring your fleet in, double up on everything and re-evaluate.
Always keep a pair of remains rebuilders on hand to quickly rebuild lost turrets, and possibly a pair of mark 1 engineers, too.
Consider using a logistics command station to slow the bad guys down so your turrets can get more pot shots in, or a military station to translocate them around the system. The best defensive position I ever had in this game was next to a gravity drill with the hostile wormholes on the other side of the system. The AI would lose probably 50-75% of a wave getting across the system, only to be translocated as soon as they got close. And then the mining golem showed up. I couldn't get to it fast enough because of the gravity drill, and then I got so pissed off that I abandoned that game. I'll miss that planet.
You will also want to place specialized turrets in response to the type of ships the AI deploys. Ships that are immune to forcefields and/or tractors? Put down gravity turrets. Snipers? Anti-sniper turrets. Missile ships? Anti-missile turrets. Astro Trains or anything else that fires dark matter shots? Anti-dark matter turrets.