I have noticed that all ennemy ships at wormhole attack me as soon as I come in, is that a normal behaviour I remember it was not like that in the past
I forget when it was introduced (or, rather, fixed to actually work) but free ("threat") AI ships will typically blob up around the wormhole leading to the planet they want to attack until there are enough of them that they think they have a good chance of winning, and then they go through. The other ships guarding a wormhole guard post are typically in low-power-mode and generally don't come out of it unless you fire on them or their guard post or another guard of the same post fires on you. But the threat ships are not in low-power-mode and will fire on you immediately.
Either way, the difference shouldn't be significant; the guard post fires on you pretty quick too, which wakes up the guards.
It now impossible to end the game as well because I don't know how to counter large amount of core AI ships running at me when I attack the core AI homeworld.
If they're all running at you it's probably because you brought in so much firepower (2x theirs) that it "woke up" the entire planet. Starting with smaller attacks may help you.
Alternatively you can try various approaches to dealing with a lot of mkV ships; once they're all freed they'll probably chase you back through the wormhole you came through (unless there are multiple good paths back to your territory; they might even split up), and you can have a nice ambush waiting for them. There's a bunch of different units that are great for spreading out fleet ships (engine damage from riot starships, slow from grav turrets, immobilization from tractor turrets, physical displacement from forcefield generators, etc), and various ways to kill them. The guardians that come with them will still be very annoying, of course. And if you aren't able to hit the homeworld shortly after dealing with the freed ships it will have reinforced enough that you'll probably be back where you started. This is indeed one way to lose the game, but it doesn't sound that dire yet.