Author Topic: Multicore systems  (Read 3599 times)

Offline Kaptein

  • Jr. Member
  • **
  • Posts: 62
Multicore systems
« on: November 26, 2009, 02:36:38 pm »
Seeing as i host games internally, and I've had some recent lagginess due to fleet sizes (i dont mind)
Any current or planned support for multicore systems? I havent checked core usage yet, but I'm guessing AI War currently uses only a single core
(or perhaps a second thread for AI calculation (is it really that much?))

I have a quad core 2.4ghz btw, and 6GB of RAM (and no hardware bottlenecking to speak of)

Finally, seeing as most people have 64bit systems nowadays (it was out in 2001, thats almost a decade ago), any performance you can get from there?

Offline Vaos

  • Full Member
  • ***
  • Posts: 107
Re: Multicore systems
« Reply #1 on: November 26, 2009, 03:11:48 pm »
There is already some support for multicore systems of some sort, but it's more noticeable in multiplayer games : In multiplayer games, the host use two process, one for running the game itself, and one for running all the AI for all the players, each can use one core effectively. All the other players just run the first process. Coding for multicore support isn't easy at all, but I can't say I know more about that.

Offline Kaptein

  • Jr. Member
  • **
  • Posts: 62
Re: Multicore systems
« Reply #2 on: November 26, 2009, 03:31:14 pm »
Yeah, i know its not easy :)
I wonder how these guys had it when they had such massive fleets on (read some posts around here)
And all we got to was around 3k against a fleet of perhaps 1-2k

Offline x4000

  • Chris McElligott Park, Arcen Founder and Lead Dev
  • Arcen Staff
  • Zenith Council Member Mark III
  • *****
  • Posts: 31,651
Re: Multicore systems
« Reply #3 on: November 26, 2009, 09:04:08 pm »
The game does support multithreading, in fact it uses over 16 threads for things like sound and music playback, etc, as well as the AI thread on the host.  The AI thread can be extremely intensive, in some cases using a full core in order for the AI to make good decisions in cases where there are 80,000+ ships in the game at once, etc.

However, you are quite right that the core simulation is single-threaded.  AI War, like all multiplayer RTS games, is based on a synchronized execution process.  For more information about that from the developers of the original Age of Empires game, this is a really classic article that most game designers/programmers in the genre are familiar with:  http://www.gamasutra.com/view/feature/3094/1500_archers_on_a_288_network_.php

Taking that sort of approach, which is needed for any sort of reasonable performance over the Internet, the issue of multithreading in the main simulation becomes absolutely nightmarish because of the need for everything client machine to deterministically give the same result of the simulation.  In this sort of game, all operations are cumulative (the order of which ship shoots first, and when exactly what happens in the simulation has a subtle effect on the outcome in the short-term, and major effects in the long-term.

There are also a lot of issues with sharing and locking of in-memory assets in today's programming languages.  C# is better about that sort of thing than C++ in my opinion, but it is not ideal to the degree of some of these newer experimental programming languages (which are not yet suitable for game development for a variety of reasons).  These sorts of things may be sorted out in time, but for the moment they are a challenge for highly realtime, highly-interlocking applications.  For tools such as 3D rendering packages (Vue, for example), they are able to break up one larger task into smaller tasks and complete those independently, simply joining the tiny result at the end.  That's a beautiful example of when multithreading works, and in some ways it is the premise behind what multithreading is used in AI War. 

But for a single simulation that is order-of-operations-dependent, and which must give deterministic results, the short answer is that I don't believe that is a problem that any game developer has solved yet, AAA or otherwise.  Those FPS games and such that use multithreading either just offload AI and other secondary logic, as AI War does, or they use an entirely different network model that is only suitable for games with smaller numbers of active entities (most action games use that sort of model, most strategy games have no choice but to use our model).  With an action game you get players jumping from place to place during lag, which is evidence of a desync that is being corrected, but in an RTS title it doesn't work the same way because there is so much more going on in a simulation sense.

As to the question of 64bit, that really only helps insofar as giving you access to more memory -- using 64bit executables has actually proven to be slower in the past when testing AI War, so it runs as 32bit only.  The reason for this, near as I can tell, is that allocating so much bigger integers causes more allocation/deallocation overhead.  The performance boosts that are often touted with 64bit architectures are very real, don't get me wrong, but those are all well below the programming level for the most part -- motherboard bus pathways and general memory/CPU architecture is far better, etc.  You get the benefit of those things even on a 32bit application.  The other, more OS-specific speed boosts of a 64bit architecture are also still in place even with 32bit applications.

Performance is something I'm always tuning, and is something that gets progressively better over time.  At the moment I don't have any expectations of increasing it more significantly anytime soon, but you never know what I might come across.  The latest 2.001 prerelease versions should perform better than the vanilla 2.0 version, by the way; so if performance is a concern, you could try that out (you don't have to buy the expansion to do so).

The performance that other players have seen varies based on their hardware (clock speeds are often misleading, as CPU/motherboard architecture can make a huge difference in performance -- there are threads with players talking about that elsewhere on these forums), and the scenario as a whole that they are playing, as well as how many ships are active at the time.  If you have 70,000+ ships in the game, but are only directly controlling 3k at once, that is still going to lead to some lag on a machine with your specs.  On the other hand, one player has reported using a 3.5Ghz quad core to control a few thousand ships on a map with 201,000 ships total on it.  That's well above normal usage for the game, and it runs quite poorly on my own 2.4GHz quad (Q6600).  But I've not had significant lag with the numbers you're talking about, except sometimes if there are a lot of collisions going on at once with ships coming out of a wormhole or something.  Still, compared to other, smaller games with less-intensive AI (SupCom comes to mind), the performance of AI War is much better in the main.

Anyway, rest assured that continuous improvement of performance is something I always have an eye out for, but I'm not planning any major rearchitectings of the game at present, as it's quite well architected in general at the moment and beats the pants off of pretty much every competitor when it comes to the scale/performance of the simulation, even if it does drop speed during some of the more intensive battles on some hardware.

Thanks for your support, and you interest in the game!
Have ideas or bug reports for one of our games?  Mantis for Suggestions and Bug Reports. Thanks for helping to make our games better!

Offline eRe4s3r

  • Core Member Mark II
  • *****
  • Posts: 2,825
Re: Multicore systems
« Reply #4 on: November 26, 2009, 09:21:39 pm »
Mhh  :D

It might be a good idea to maybe reduce fire-rates so that they are twice as slow and do twice as much damage in return.. this would increase performance when fleets shoot at each other...

I guess it would be nice to have the game dedicate targeting/fire calculations and movement of player owned ships to a whole core in itself. And have AI calculations (and THEIR fire things) use another core only for that.

Obviously only for quad cores and above viable... not sure its possible coding wise.. i think this would cause a massive performance increase for quad cores (like doubling performance).. but it doesn't sound like an easy thing to code ;/

For quad cores, it might even be better to split the AI calculations to a single core each for 1 AI so that on quad core all 3 players (in SP) each run on their own dedicated core (including movement/collision/targeting) and important - AI's run on Core 3/4 while Players run on Core 2 and all the other stuff on Core 1.

I wonder if stuff like that is possible to code/compile and if it would actually be improving performance or not?  ;D

Speaking of that.. I just remembered that this is maybe 4 years to soon to suggest  :D No manycores out yet ;/
« Last Edit: November 26, 2009, 09:39:25 pm by eRe4s3r »
Proud member of the Initiative for Bigger Weapons EV. - Bringer of Additive Blended Doom - Vote for Lore, get free cookie

Offline x4000

  • Chris McElligott Park, Arcen Founder and Lead Dev
  • Arcen Staff
  • Zenith Council Member Mark III
  • *****
  • Posts: 31,651
Re: Multicore systems
« Reply #5 on: November 26, 2009, 09:41:29 pm »
It might be a good idea to maybe reduce fire-rates so that they are twice as slow and do twice as much damage in return.. this would increase performance when fleets shoot at each other...

I did that in late alpha, I don't care to do it again at the moment again.  I don't think this would save significant performance at the moment.  The collision-detection and target-selection logic accounts for the huge, vast majority of the workload, anyway.

I guess it would be nice to have the game dedicate targeting/fire calculations and movement of player owned ships to a whole core in itself. And have AI calculations (and THEIR fire things) use another core only for that.

Yes, AI calculations are on their own thread.  The targeting calculations are on the main thread with the render logic because of the issues with memory sharing, etc.

Obviously only for quad cores and above viable... not sure its possible coding wise.. i think this would cause a massive performance increase for quad cores (like doubling performance).. but it doesn't sound like an easy thing to code ;/

In the best case, moving the entire sim logic to another thread would lead to a 20% speed boost, since that's around how much the render logic takes when the sim weight is low.  In very intensive situations, where the game would normally start to lag, moving the entire sim logic to another core would lead to far less of a gain, as little as 1% to 5% or less.

For quad cores, it might even be better to split the AI calculations to a single core each for 1 AI so that on quad core all 3 players (in SP) each run on their own dedicated core (including movement/collision/targeting) and important - AI's run on Core 3/4 while Players run on Core 2 and all the other stuff on Core 1.

The AI thread causes no lag on the game whatsoever on multi-core systems, but feeding it simulation data can. Having a single thread that can be fed all the sim data is vastly more efficiency in terms of overall run speed, from what I've found.

I wonder if stuff like that is possible to code/compile and if it would actually be improving performance or not?  ;D

Having the AI logic on a separate thread is one of the main reasons AI War is able to work as a concept at all.  Really, I don't think people quite understand just how amazingly efficient this is.  It starts lagging a bit where similar games would lag with 1/10 or less of the assets in the simulation.  Granted, I'm always interested in continuous improvement, but at the moment there is no time nor any compelling reason to do a complete overhaul of the threading model, and it's not a problem that any other programmers have solved in a similar sort of simulation scenario to my knowledge, either.  Thanks, though!
Have ideas or bug reports for one of our games?  Mantis for Suggestions and Bug Reports. Thanks for helping to make our games better!

Offline RCIX

  • Core Member Mark II
  • *****
  • Posts: 2,808
  • Avatar credit goes to Spookypatrol on League forum
Re: Multicore systems
« Reply #6 on: November 26, 2009, 09:47:29 pm »
Something just occurred to me: i hope GPG buys your company so you can make Supreme Commander 4 capable of handling 10k units on a midrange computer :D
Avid League player and apparently back from the dead!

If we weren't going for your money, you wouldn't have gotten as much value for it!

Oh, wait... *causation loop detonates*

Offline eRe4s3r

  • Core Member Mark II
  • *****
  • Posts: 2,825
Re: Multicore systems
« Reply #7 on: November 26, 2009, 09:57:43 pm »
Mhhhhhh
I guess i meant to split the calculation of the sim logic to 2 or more cores (preferable to as many cores as it needs to still compute in time for the next step). I guess this isn't really possible unless you want to kick all the dual core and less than 2 core users from your game  ::) ;D

I am coming from a 3d rendering standpoint here so maybe that isn't possible given that you have to do all this each main loop step...

Mhh, to explain it even better , instead of threads i meant to split the sim to its own process (for each core 1) and balance load between them on-demand

Is that even possible?

AI war is running pretty well either way - just wondering really.
Proud member of the Initiative for Bigger Weapons EV. - Bringer of Additive Blended Doom - Vote for Lore, get free cookie

Offline x4000

  • Chris McElligott Park, Arcen Founder and Lead Dev
  • Arcen Staff
  • Zenith Council Member Mark III
  • *****
  • Posts: 31,651
Re: Multicore systems
« Reply #8 on: November 26, 2009, 10:24:42 pm »
Mhhhhhh
I guess i meant to split the calculation of the sim logic to 2 or more cores (preferable to as many cores as it needs to still compute in time for the next step). I guess this isn't really possible unless you want to kick all the dual core and less than 2 core users from your game  ::) ;D

I am coming from a 3d rendering standpoint here so maybe that isn't possible given that you have to do all this each main loop step...

Mhh, to explain it even better , instead of threads i meant to split the sim to its own process (for each core 1) and balance load between them on-demand

Is that even possible?

AI war is running pretty well either way - just wondering really.

It's really not possible with current programming languages for the reasons I mentioned above.  A recent update to the simulation exporter for AI War does just that, actually: it runs n number of processes where n is equal to the number of cores you have, and then combines the result at the end.  So that's very similar to the 3d rendering standpoint that you are coming from, I use those sorts of tools as well and know just what you mean.

The problems, in realtime, are twofold: 1. operations are dependent on one another, not mutually-exclusive, as I've noted above (because of the need for the simulation to be deterministic, it is also linear to a degree), and 2. the bandwidth for passing data back and forth between the two threads in a timely (realtime) manner is simply too low for most programming architectures.  A different sort of shared memory approach would really be needed for this to be feasible in the future, in my opinion.  Perhaps in later versions of .NET, or some other platform.

I see just where you're coming from, and believe me I have done a lot of headscratching trying to decide if that is feasible, but I just haven't come up with a workable solution with the current kind of tools, etc.  For the moment, given that I do want to support the dual-core and single-core computers as much as possible, I think it's generally most productive for me to be focusing on overall efficiency reduction on the existing simulation thread, anyway.

It's a very interseting topic, and I think the answer to this will be very different in another 5-10 years from now if not sooner.  But the software development tools and programming languages are still kind of clamoring to really catch up to the multicore explosion in many ways at the moment...
Have ideas or bug reports for one of our games?  Mantis for Suggestions and Bug Reports. Thanks for helping to make our games better!

Offline x4000

  • Chris McElligott Park, Arcen Founder and Lead Dev
  • Arcen Staff
  • Zenith Council Member Mark III
  • *****
  • Posts: 31,651
Re: Multicore systems
« Reply #9 on: November 26, 2009, 10:30:45 pm »
Something just occurred to me: i hope GPG buys your company so you can make Supreme Commander 4 capable of handling 10k units on a midrange computer :D

Haha, we're not for sale, but I sure do love SupCom and am a big fan of GPG.  It's interesting, too, as a number of people I know also know GPG staff including Chris Taylor; Arcen's lawyer, for instance, is the same one who sold SupCom to THQ, which is pretty cool.  But I don't know anyone there personally, and I imagine Chris Taylor has no clue who I am.  By the time SupCom 4 is coming around, I imagine they'll have the problems of 10k units on a midrange computer licked, anyway (or hardware will have evolved to the point where every game will include that). 

The novelty of the huge unit counts in AI War is going to be pretty shortlived in the grand scheme of things, I think, though I doubt anyone will pass us for a while.  And, hey, as hardware improves and techniques improve and so forth, maybe I'll just make AI War even bigger.  Who knows what sort of unit counts might be routine in AI War on midrange hardware 3 or 4 years from now. :)
Have ideas or bug reports for one of our games?  Mantis for Suggestions and Bug Reports. Thanks for helping to make our games better!

Offline RCIX

  • Core Member Mark II
  • *****
  • Posts: 2,808
  • Avatar credit goes to Spookypatrol on League forum
Re: Multicore systems
« Reply #10 on: November 26, 2009, 11:33:14 pm »
Something just occurred to me: i hope GPG buys your company so you can make Supreme Commander 4 capable of handling 10k units on a midrange computer :D

Haha, we're not for sale, but I sure do love SupCom and am a big fan of GPG.  It's interesting, too, as a number of people I know also know GPG staff including Chris Taylor; Arcen's lawyer, for instance, is the same one who sold SupCom to THQ, which is pretty cool.  But I don't know anyone there personally, and I imagine Chris Taylor has no clue who I am.  By the time SupCom 4 is coming around, I imagine they'll have the problems of 10k units on a midrange computer licked, anyway (or hardware will have evolved to the point where every game will include that). 

The novelty of the huge unit counts in AI War is going to be pretty shortlived in the grand scheme of things, I think, though I doubt anyone will pass us for a while.  And, hey, as hardware improves and techniques improve and so forth, maybe I'll just make AI War even bigger.  Who knows what sort of unit counts might be routine in AI War on midrange hardware 3 or 4 years from now. :)
:o i can see it now: The first 1 million unit game! Est. completion time: 1 year. :D

Not to get too far off on a tangent here, but what do you think of SC 2? It looks so good that i've already preordered a copy (assuming that i still have the money set aside by then :) )!
Avid League player and apparently back from the dead!

If we weren't going for your money, you wouldn't have gotten as much value for it!

Oh, wait... *causation loop detonates*

Offline Kaptein

  • Jr. Member
  • **
  • Posts: 62
Re: Multicore systems
« Reply #11 on: November 28, 2009, 04:40:25 am »
of course, youre absolutely right, my hopes were that you had still some heavy parts of the game left you could delegate to another core

i guess i can try to downcore to 2 speedier cores, if thats more in line with your coding?
AMD downcores to 2x2 sharing, so it will become ~5ghz + ~5ghz.. we'll see how that goes i suppose
« Last Edit: November 28, 2009, 04:44:45 am by Kaptein »

Offline Fiskbit

  • Arcen Games Contractor
  • Master Member Mark III
  • *****
  • Posts: 1,752
Re: Multicore systems
« Reply #12 on: November 28, 2009, 08:05:07 am »
If you can actually get your CPU to those speeds without dry ice or somesuch, I'd love to hear about what sort of machine and settings you're using. Those speeds are crazy. I hadn't heard of downcoring before, and looking online seemed to indicate that it doesn't do very much beyond lower heat (while the problem most people have with overclocking, from what I've seen, is pushing voltage too high, not having temperatures too high). Pretty amazing, though, if you can reach those speeds.
Have ideas or bug reports for one of our games?  Click here to get started with Mantis for Suggestions and Bug Reports.  Thanks for helping to make our games better!

Offline Kaptein

  • Jr. Member
  • **
  • Posts: 62
Re: Multicore systems
« Reply #13 on: November 29, 2009, 05:34:00 am »
If you can actually get your CPU to those speeds without dry ice or somesuch, I'd love to hear about what sort of machine and settings you're using. Those speeds are crazy. I hadn't heard of downcoring before, and looking online seemed to indicate that it doesn't do very much beyond lower heat (while the problem most people have with overclocking, from what I've seen, is pushing voltage too high, not having temperatures too high). Pretty amazing, though, if you can reach those speeds.

actually, it sounds more like i had an inflated opinion about downcoring considering what you just described
i thought downcoring on AMD systems meaning cores sharing as 1 or 2 cpus effectively

http://www.ocforums.com/showthread.php?t=612483
but my luck, it seems it just turns off individual cores :) what a shame

Offline Fiskbit

  • Arcen Games Contractor
  • Master Member Mark III
  • *****
  • Posts: 1,752
Re: Multicore systems
« Reply #14 on: November 29, 2009, 07:10:37 am »
If only core speeds could be added together to make a single super-fast core. Alas!
Have ideas or bug reports for one of our games?  Click here to get started with Mantis for Suggestions and Bug Reports.  Thanks for helping to make our games better!