Author Topic: [Now on doc] Keep the option to save games uncompressed?  (Read 620 times)

Offline TechSY730

  • Core Member Mark V
  • *****
  • Posts: 4,550
[Now on doc] Keep the option to save games uncompressed?
« on: September 16, 2016, 02:24:54 PM »
Something that was fun to tinker around with in AIWC was being able to turn off save game compression and thus get easy access to modifying the save game arbitrarily using any ol' text editor. You could pull of craziness that not even cheats let you do (AI difficulties > 10, renaming the AI players, playing as the AI (sort of, it was very broken, but you could), all sorts of fun stuff).

Could there be an option to keep that in AIW2?

Even if you decided to go for a binary save file format, letting it be saved uncompressed would make it easier to write some sort of program for it. ;)
« Last Edit: September 19, 2016, 11:21:34 AM by x4000 »

Offline kasnavada

  • Hero Member Mark II
  • *****
  • Posts: 911
Re: Keep the option to save games uncompressed?
« Reply #1 on: September 16, 2016, 02:30:34 PM »
Or enable transforming a "save" to a "scenario" for modding purpose.

Offline TechSY730

  • Core Member Mark V
  • *****
  • Posts: 4,550
Re: Keep the option to save games uncompressed?
« Reply #2 on: September 17, 2016, 02:05:35 AM »
Or enable transforming a "save" to a "scenario" for modding purpose.
Oh right, a "mission"/"map" editor. Probably should make a new thread for that idea.

Offline kasnavada

  • Hero Member Mark II
  • *****
  • Posts: 911
Re: Keep the option to save games uncompressed?
« Reply #3 on: September 17, 2016, 02:17:54 AM »
Well, if you can save un uncompressed format, then notepad++ (or whatever) becomes your user-unfriendly map editor =).

Offline x4000

  • Chris Park, Arcen Games Founder and Lead Designer
  • Administrator
  • Zenith Council Member Mark III
  • *****
  • Posts: 30,700
Re: Keep the option to save games uncompressed?
« Reply #4 on: September 19, 2016, 11:21:06 AM »
1.f.iv. Optional Uncompressed Savegames

It's on the list now. :)
Have ideas or bug reports for one of our games?  Mantis for Suggestions and Bug Reports. Thanks for helping to make our games better!

Offline NickAragua

  • Sr. Member
  • ****
  • Posts: 280
Re: [Now on doc] Keep the option to save games uncompressed?
« Reply #5 on: September 19, 2016, 07:24:45 PM »
Can't you "just" make the save a plain zip file or something so that it stores compactly but can be decompressed into normal files? There's like a thousand zip libraries out there. Is compact save storage even an issue when you've got machines with terabytes of hard drives space?

Offline x4000

  • Chris Park, Arcen Games Founder and Lead Designer
  • Administrator
  • Zenith Council Member Mark III
  • *****
  • Posts: 30,700
Re: [Now on doc] Keep the option to save games uncompressed?
« Reply #6 on: September 19, 2016, 07:44:55 PM »
Can't you "just" make the save a plain zip file or something so that it stores compactly but can be decompressed into normal files? There's like a thousand zip libraries out there. Is compact save storage even an issue when you've got machines with terabytes of hard drives space?

There are a number of problems:

1. People upload savegames to our server in mantis tickets, and there's a size limit on that (both practical and technical).

2. Saving using our efficient algorithm is memory-neutral, aka it doesn't generate new heap usage (much).  Saving using a more verbose format, and/or using compression, requires native code calls and a whole lot of RAM allocations that then have to be garbage collected, etc.

3. Saving the more verbose way is slower because more has to be written in general, and compressing it is then also slower because it has to do all that work also.  So there's more of a hitch when the saving happens (autosave is mainly the only place you care), and there's a spike in RAM which in the worst cases can cause out of memory exceptions even on machines with plenty of RAM just because of how much ram the internal mono runtime embedded in unity is allowed to use.  To get around that in AI War Classic we have to garbage collect at like 4 different points in the process if the managed memory in use is above certain numbers.  That makes the saving hitch even larger.

4. When we're talking about something like autosaves, why have a limit on them?  If they are only a few KB, just save an indefinite number of them since hard drives are so big.  Autosaves are then unobtrusive and extremely thorough.  We did something like this with the level editor in Release Raptor, and it was glorious.  (Just put all those autosaves in a subfolder so they don't clutter the main view, of course).  However, if we're suddenly talking tens of megs, we'd be getting into gigs of autosaves pretty fast if we were not having some sort of cap or cleanup process for older autosaves.  That's always a fraught process, because then you never know when we're accidentally deleting or overriding an autosave that someone really wanted.  Is it a huge issue?  No, definitely not.  But it is something that bothers me.


In general none of the above issues are likely to be much of an issue in this sequel in general; they were all worse in AI War Classic, and we solved them all with compression and such.  But frankly, if we're compressing the contents of the file (versus putting it in a zip file, which is itself even slower), you can't read it anyway.  And it's fairly hard to read even if you have it in an uncompressed, verbose format.  Lots of stuff is just strings of numbers, not actually labeled or anything.

Ultimately there's very little incentive to do it an inefficient way "just because." ;)  A much better approach would be for us to provide a public API from our in-game DLL that would allow you to load the data from the disk without having to actually load the game.  Then people could link against that and make a savegame editor without having to reverse-engineer our code (which would of course change over time anyway).
Have ideas or bug reports for one of our games?  Mantis for Suggestions and Bug Reports. Thanks for helping to make our games better!

Offline NickAragua

  • Sr. Member
  • ****
  • Posts: 280
Re: [Now on doc] Keep the option to save games uncompressed?
« Reply #7 on: September 19, 2016, 08:21:20 PM »
Ultimately there's very little incentive to do it an inefficient way "just because." ;)  A much better approach would be for us to provide a public API from our in-game DLL that would allow you to load the data from the disk without having to actually load the game.  Then people could link against that and make a savegame editor without having to reverse-engineer our code (which would of course change over time anyway).

Hey, if you want to do save games "the right way", you won't hear any complaints from me. :)

I'm sure the modding community will be thrilled to have an API.

Offline x4000

  • Chris Park, Arcen Games Founder and Lead Designer
  • Administrator
  • Zenith Council Member Mark III
  • *****
  • Posts: 30,700
Re: [Now on doc] Keep the option to save games uncompressed?
« Reply #8 on: September 20, 2016, 10:36:57 AM »
All good!
Have ideas or bug reports for one of our games?  Mantis for Suggestions and Bug Reports. Thanks for helping to make our games better!