Interesting tidbit: 2.8GB of memory successfully allocated in Unity.

[x4000]

Due to a bug in a working copy of AVWW, I wound up accidentally allocation 2.8GB of some non-heap data.  And Unity was just fine with that, which is a significant find.  Though when I passed that 900MB of actual heap data (as in, game objects and world data, not music, sounds, or textures), then it gave the too many heap sections error.

This was something of an accidental discovery, but it is notable.  This tells me that Unity is doing great as a 32bit process with the extended address paging on windows, so that it was able to allocate more than a standard 32bit application should be able to by default -- as advertised.  In terms of AVWW or AI War, this means we can have however many textures, etc, as your machine will support all cached in RAM, without ever having to worry about running out of room (within reason), assuming you have 3GB or more of free ram when running the games.

That means that the only hard memory limit we're running into is that 900MB of heap space.  AI War just saw a real boost in the right direction so that it's using only more like 500MB at most for most large games, and of course AVWW still is using only about 60MB at most, period, because of how we're caching stuff to disk so heavily rather than leaving quite so much memory-resident as in AI War.

All in all... this is quite good news, and a bit unexpected.

[TechSY730]

Any future plans for a 64bit version? I'm sure this game will keep getting bigger in scope as future expansions come out, and memory being the main limitation to larger games, this is great news!

Not to try to put words in his mouth, but I think Chris has said in the past that a 64-bit version would only come if it would be truly necessary. It's too messy otherwise, especially for maintaining consistency among 32-bit and 64-bit versions (as the 32-bit versions would need to be kept for a while, as 32-bit only systems are still quite widespread). The memory issues the current engine has is not due to the limitations of the 32-bit address space but more to do with a really poor garbage collector.

Besides, I would like to see Unity get their memory management straight (by upgrading the version of Mono they run) before they try to compile a 64-bit version, which presumably Chris and Kieth would need first before they could compile AI war to a 64-bit version.

[x4000]

Yep, yep, and yep. I think the chance of a native 64bit version of ai war is about zero. 32bit and 64bit players would not be able to play multiplayer together, the 64bit version would run slower due to he larger default integer sizes (defaulting to int64 instead of int32 leads to desyncs and slowness), and frankly we just don't need he memory. With extended addressing in 32bit, we can access about 3gb of memory. Currently we use under half that, and even if we doubled the number of ships in he game, all their textures would not need to be in memory at once, so we ought to be able to keep texture memory about equal to what it is now.

As techsy mentioned, the current limitations are due to the heap and the mono gc, and are something that he engine needs to fix separate of any 32bit/64bit concerns.

[Echo35]

With extended addressing in 32bit, we can access about 3gb of memory.

Yeah, but the OS itself won't address much more than that in a 32-Bit environment. Depends on your video card (Since they share memory space) but you're going to cap at about 3.5-4 GB of memory for the system, and Windows 7 and Vista take close to a gig to run anyway, so unless you have a good bit of Swap set up (Whatever it's called in Windows...) you'll be crashing the game addressing that much. Even 2.8 would be a tad unsafe. Speaking of which, did you test WHERE that 2.88 GB of heap space was going? Did it go to the RAM or to Swap space?

[x4000]

You're talking about two different things: a 32bit program and a 32bit OS.  I'm talking about a 32bit program on a 64bit OS.  People with 64bit OSes (like me), want to use as much RAM as they can, and the 32bit cap on a 32bit program is annoying.  With extended addressing, what I was talking about above, that makes 32bit programs use unsigned pointers and thus greatly increases the amount of space they can address.  Using signed pointers in the first place (and then only using the positive ones) was kind of boneheaded in the first place.

And yes, this was all in RAM, none in swap; I have 8gb on my machine, and there was a good amount free at the time.  The only time my machine would go into swap would be if my OS was low on RAM, not if a specific  32bit application was requesting more than the standard non-extended 32bit range.

[Echo35]

With extended addressing, what I was talking about above, that makes 32bit programs use unsigned pointers and thus greatly increases the amount of space they can address.

I know PAE extends the memory pool a bit, but I thought there was still a limit of some degree on 32 Bit software regardless.

[x4000]

With extended addressing, what I was talking about above, that makes 32bit programs use unsigned pointers and thus greatly increases the amount of space they can address.

I know PAE extends the memory pool a bit, but I thought there was still a limit of some degree on 32 Bit software regardless.

Well, sure, there's still a limit. But when the limit is hit, it doesn't go to swap -- it just can't allocate any more memory (and thus probably dies). Swap is only useful at the OS level, when one program asks for memory that the OS doesn't have to spare.

[Echo35]

With extended addressing, what I was talking about above, that makes 32bit programs use unsigned pointers and thus greatly increases the amount of space they can address.

I know PAE extends the memory pool a bit, but I thought there was still a limit of some degree on 32 Bit software regardless.

Well, sure, there's still a limit. But when the limit is hit, it doesn't go to swap -- it just can't allocate any more memory (and thus probably dies).

That's what I thought, because I definitely have run large memory footprint games on a 64 bit OS and had them die, especially when my swap was low. Sins of a Solar Empire comes to mind, since that game doesn't really HAVE a built in memory limit.

EDIT: Also that lovely screw up with Empire Total War that made the game unplayable (Literally) for many months before they decided they should actually enable PAE for the game.

[x4000]

I can tell you that everything has a built-in memory limit -- it's not something you can configure as a programmer, it's the nature of the type of compiler you use -- 16bit, 32bit, 64bit, etc.  If Sins is compiled as 64bit, then yeah its limit is so high as to be more than anyone has in a consumer-level PC at the moment.  But if it's 32bit, then even with extended addressing its limit is identical to that of AI War. 

Heap space usage is something that varies by language, and is a separate matter aside from available RAM, but if we're just talking total overall RAM use, the above is what I mean. 

You can also run into trouble with large games when they need a big chunk of memory (say, 50mb all of a sudden), but the OS's available memory is all fragmented into chunks smaller than that.  So even if you have 200mb free, but it's all in fragments no larger than 5mb, then you might run into a crash that is premature because of the OS being too fragmented in its current memory allocation.  That's part of why, back in the day, games always advised you to close all other programs.