I rather like Chemart's distinction between tactical and strategic challenge, and I think I basically agree that making Homeworlds a challenge that you have to do a lot of micro to overcome is a weird shift from the rest of the game, which is more about figuring out where you need to put what ships than figuring out what to do with them once they're there. That said, I can see why this is hard to make into a boss fight --- if the answer is just "all of them, in a big group"; it's going to feel awfully same-y from game to game.
(By the way, I do tend to get tired of games once they get to the slog bit, and I have to admit I haven't actually made it to a homeworld since before Ancient Shadows --- I hope you'll forgive me if my ideas are wildly out of date).
But (@Chemart) I think the the idea of breaking the homeworlds into micro-battlefields could work. The point, if I understand it right, is to break up the homeworld into a few parts such that you can focus on one and mostly ignore the others, which is very much like approaching a few different planets.
I wonder if you could do the same thing more simply by moving some of the homeworld guard posts onto the core worlds, creating more of a multi-stage boss fight? To make it clear, you would still have to kill them all before you could attack the homeworld. So you wouldn't get the cluster[mess] effect of having ten homeworld guard posts overlapping ranges with each other and the brutals, but you would have the additional challenge of having to deal with them at the same time as the fairly nasty core worlds, with maybe one brutal pick apiece. If you got a core eye or wraith lance, at the most you would have to deal with it for a third of your overall fight, leading to more of a variety of tactics within the fight.
The other thing that occurs to me, is that as I understand it, what strategic challenge is is mostly about managing your resources --- that is, the Napoloen issue of facing pressures from a lot of directions, and trying to deal with most of them efficiently enough that you can still focus overwhelming force on your point of attack. So to get that kind of challenge, the problem isn't about calibrating the homeworld itself, so much as giving you reasons to want a fleet somewhere else when you're trying to attack the homeworld.
Do the Core Raid and Core CPA Engines do this well? Is there room for more of that kind of thing, combined with a weaker homeworld?