Too Much Heap Sections ???

[Shadoz]

How to handle this

i get often around all 5-10 minutes this error ... no fun to play ...

but in other games with more enemys more action and on Multiplayer this happends very rarely ...

what to do
what does heap sections to much means

[x4000]

This means that you've run out of memory for the game's "game objects" to run with -- it generally happens if your savegame is absolutely enormous.  The amount of memory you have on your system, if it's more than a GB or two, doesn't matter for that.

It might be that you just need to play smaller games of AI War; or particularly to play with normal ship caps rather than high.  If you post your savegame, we can take a look at it, but generally in the rare cases where this happens there's not a lot we can do.

[TechSY730]

Yea, until the engine AI war is running on can handle memory better (specifically, the garbage collector, if you are interested), playing a game with high ship caps will almost always eventually end up with a game too big for the Unity engine to handle.

[Shadoz]

what is a "garbage collector"
sry techsy730 my english is not perfect ... i cannt understand what you posted ...

so is there no way to give ai war the possibility to handle more memory ... i mean my game runs most time on around 1,5 GB ... but i have 8 GB ram :-) ... i think most have today 4GB standard of RAM ...

the think what iam suprised about is, that the my game where this happends randomly so often is not with high caps only normal and i have only 1 start home planet ... on the other hand i have only 22 planets occupated and there happends not so much ... only an agression around 10k units ... in wave / attack / etc. ...
on other games i plaed in multiplayer of example there was so much more action an no heap section errors ... maybe all 1-2 hours but so rarely that it doesnt matter and there was so much more around 40k - 45k ships on agression at us ... this was no problem ...


so is there anythink i can do ?

[x4000]

If you can post the savegame in question, we can look to see what is using so much memory, at least.

[Shadoz]

here is the savegame ...
its randomly in some actions ... sometimes it takes 15 minutes sometimes only 5-10 ... sometimes take longer time ... dont know why or what happend ...

[TechSY730]

what is a "garbage collector"
sry techsy730 my english is not perfect ... i cannt understand what you posted ...

so is there no way to give ai war the possibility to handle more memory ... i mean my game runs most time on around 1,5 GB ... but i have 8 GB ram :-) ... i think most have today 4GB standard of RAM ...

It would be hard to explain it in detail unless you are already familiar with computer science, but I will give you the basic idea.

A garbage collector basically finds unused "stuff" and "cleans" it up so that space can be used by other "stuff".

Thanks to the other software AI War is running on (the Unity engine), it is force to use a specific garbage collector. Sadly, it has a bug where if it needs to clean up a lot  of "stuff", instead of taking the time to clean it, it will give up and throw that heap sections error you are getting.

And no, sadly, the Unity Engine (what AI War runs on) does not give a way to say that you want more than 1.5GB, even if your computer has much more than that.  

In other words, the devs are limited to 1.5GB or so, and if they get near that limit, there is a good chance the game will crash, thanks to a bug outside of their control.

There is basically nothing the devs can do at this point outside of moving to another engine (VERY unlikely), or finding ways to cut down memory usage even further (though from what I understand, there isn't much more they can shrink memory usage down, because this is such a MASSIVE scale game)

I'm hoping that the people who make Unity, the part of the program that has the bug you are running into, fix this soon.

[x4000]

Actually, the limit is nothing to do with 1.5GB.  The limit is about 900MB, for the heap specifically.  The 1.5GB thing is coincidental with how much non-heap memory the game uses (for things like textures).

[TechSY730]

Actually, the limit is nothing to do with 1.5GB.  The limit is about 900MB, for the heap specifically.  The 1.5GB thing is coincidental with how much non-heap memory the game uses (for things like textures).

900MB, that is even worse. 

Though now that I think about it, how much memory does 40,000+ objects really take up? From what I can tell, the ForegroundObject type is not that big.

I wonder how of the "dead" memory is taken up by local lists and arrays declared in functions, especially the more sophisticated loops that go over most of the ships. Even if your objects are lightweight, if the GC tends to crash, every object allocation matters.

I guess you could try to minimize object allocations as much as you can, going as far as to reuse lists, arrays, and even local function variables (aka, moving them outside the function) as much as you can. But that would make the game extremely susceptible to "state leak bugs", and make the code extremely thread hostile. Plus it doesn't help with allocations made by Unity and Mono themselves.

[Shadoz]

So a big "what a pitty" to the unity engine :-)

ai war is so nice cause it could be a nice big scale :-)

but i hate little thinks like out of memory :-(

but in this way ... one easy question ... iam surprised how to get work a 120 map with high caps maybe with 5-16 start homes ( ) heavy AI like Raid Engine / extreme raiders and all by +300% ressources ... so no way to get it long to work ? noe one can handle it


look at my savegame i get crashes like i explained "heap sections" ... and at this game nothink is happend ^^ so low agression ...


By The Way ... @x4000 ...
in this savegame you can see a big bug or dont know what it is ^^ ... i explained it in another topic ... at the netral planet next to my last owned planet where the 10k units are deployed ... my capital ships are shooting but they are in low power mode and be cloaked ^^^^ ... so they are unattackable / undestructable / untouchtable but can kill with there guns ... only ther main gun dont shoot but all you can build on it on extras ... so heavy deadline for enemys ... the direpower is not much for these short capitals but the ai DONT attack me ...
so is this a bug or should it be like this normaly in low powermode no one shoots like the other spire frigates do ... and by shooting normaly they are decloaked ... both dont happends :-)

[x4000]

techsy730 -- In terms of the memory use, a lot of it is things like strings for a LOT of descriptive text, etc.  And then a ton more for all the various indexes and arrays of various sorts.  Doing savegames is particularly non-heap-static because of the fact that it's doing compression as well as all those stringbuilder addendums.

Shadoz -- In your savegame, there's not any bug that I see, but it is simply a really huge savegame.  I suspect that your game is dying during the savegame process, since that would be what might push it over the edge, but I'm not sure.

From working on AVWW, I've had a couple of ideas on some ways we can reduce the memory footprint in these extreme cases, such that we might be able to avoid these sorts of issues.  I'll push out a new beta tonight with some of those ideas in place to see how they do.

[TechSY730]

I'll push out a new beta tonight...

Music to my ears.


And yes, keeping track of indexes is sort of annoying. (Is it for "reverse lookups", and/or for marking a subset of stuff of interest in an array or list?)

I can see how String concatenation would be very non memory static. If you use the built in string, then you gotta make a new string for each step, as strings are immutable. If you use StringBuilder, either you allocate it, concatenate using it, convert to String, and then let it fall out of scope (giving it to the GC, which is bad thanks to that dumb bug), or you use a "common instance" of a StringBuilder, clearing it each time, but of course, clearing it could cause it to free up references or resize or something, giving stuff to the garbage collector (again, what you are trying to avoid). Or you could make your own "mutable string for concatenation" object, designed to minimize allocations, but reimplementing functionality provided in the standard libraries is almost always a REALLY bad idea.

In other words, there are no really good heap-static solutions. I sort of feel sorry for what you have to deal with.

Thankfully, the GC can handle smaller loads pretty well, so low ship cap and mid ship cap games are unlikely to start hitting these limits.

EDIT: I just realized, I am telling you stuff in detail that you already know, again... I really should stop that bad habit.

[x4000]

Think of our indexes as a form of "quad trees," but not for pathfinding purposes.

[Shadoz]

x4000
btw do ya have had a look for the capital ships who shoot by being cloaked and in low power mode like i explained


but what make my savegame so big :-) i mean there are more than 100 planets to kill ^^ and there more stuff i could unlock and use  and the ai can be more powerful after recover the next shard for fallen spire :-)
i wonder me cause at example a multiplayer game works much more better than this one and there should me more thinks happend ...
in my single player save there are around 65k enemys at galaxy stats ... why ? is that anythink what must be processed by the PC what can cause crashes in multiplayer game i plaed around 18 hours and there are only 35k units all over in galaxy stats ... but the AI diff is both the same only 50% more resources in singleplayer

i upload the multiplayer save ... please have a look - it should be much more bigger than the single player ... we killed around 600k enemys and there were waves of very heavy big fights ... and the heap section out of memory bug happends sooooo rarely ....

so if i compare these two savegames i cant understand why in the singleplayer the heap section happend so often

[x4000]

I'm really not sure I understand on the capital ships thing.  Do you mean starships?  I'm not understanding the question, anyway.

In terms of the multiplayer save, note that the size of the battles is not relevant when it comes to save size.  It matters how many ships are in the galaxy in total at once.