Save File Generation

[TechSY730]

Is it OK if I try to generate some save files from scratch with a custom set state to test out various staged battles and all that?

Come to think of it, how are the save files stored anyways? Is it the standard C# serialization mechanism (when stored uncompressed)?

Also, no I am not thinking of making a full fledged program for this type of thing. At best, I will make some helper functions, though I will be glad to post those once done.

EDIT: Oh, and I know it is against the EULA to disassemble the main AI War dll, but is it OK to link against it?

[x4000]

Is it OK if I try to generate some save files from scratch with a custom set state to test out various staged battles and all that?

Sure, no problem at all.  We have another player who was working on a full editor in Java, but I'm not sure what happened with that.

Come to think of it, how are the save files stored anyways? Is it the standard C# serialization mechanism (when stored uncompressed)?

It's using custom string-based serialization.  The .NET stuff is bulky and can break when .NET versions change.

EDIT: Oh, and I know it is against the EULA to disassemble the main AI War dll, but is it OK to link against it?

Yep, not a problem.

[TechSY730]

Come to think of it, how are the save files stored anyways? Is it the standard C# serialization mechanism (when stored uncompressed)?

It's using custom string-based serialization.  The .NET stuff is bulky and can break when .NET versions change.

OK then. Is the format intuitive enough that I can sort of easily find a way to parse it?

EDIT: Oh, and I know it is against the EULA to disassemble the main AI War dll, but is it OK to link against it?

Yep, not a problem.

Excellent. This means I won't have to go fishing and experimenting for data structures and enum constants and stuff like that. I can just get the AI War engine itself to handle all that. 

Of course, I will not be expecting cooperation, since the API is not public (aka, not supported for 3rd party calls), and various methods, classes, and constants will be accessed outside the intended context of a fully initialized game. In fact, it may break my code from version to version. But hey, doesn't hurt to try.

[x4000]

OK then. Is the format intuitive enough that I can sort of easily find a way to parse it?

It's all digits and delimiters, so it depends on what kind of codebreaker you are. Heh.  It's meant to be efficiently stored and generated, not to be human-readable.  But it's also not encrypted or anything like that, so a lot of it can be followed pretty easily particularly if you start with a savegame that is small where you know where stuff is any then reverse engineer it off that.  Say, for instance, with one of the basic tutorials as a savegame would be pretty easy to then do.

Excellent. This means I won't have to go fishing and experimenting for data structures and enum constants and stuff like that. I can just get the AI War engine itself to handle all that. 

Of course, I will not be expecting cooperation, since the API is not public (aka, not supported for 3rd party calls), and various methods, classes, and constants will be accessed outside the intended context of a fully initialized game. In fact, it may break my code from version to version. But hey, doesn't hurt to try.

Yep, no worries at all.

[TechSY730]

Isn't there supposed to be a "Save without compression" option in the save menu when debug mode (toggled by F3) is on? It's not showing up for me. Do you just turn on debug mode and it will show up, or is there something else you have to do too?

Nevermind, I found it under advanced options.

[x4000]

I think that if you hit F3, it actually gives you a new button, too.

[TechSY730]

OK, if you don't mind, I am going to be posting my notes about the save file format here.

There are three types of delimiters, '|', ',', and ':'. Not sure what the difference between all of them are yet, but '|' seems to be the "top level" delimiter. As such, I will call a field to be a section separated by the '|' character, with the first field being field 1.
"," and ":" are apparently used for lists of stuff, like properties of an object and all that. As such, I will be calling each thing separated by them sub-fields.

For all values, assume a numeric type unless other specified

The save I am looking at is from the basic tutorial that was paused before the first game cycle was allowed to run. As such, some of the generalizations I am making may not be true.

Some fields have headers, which is always two numeric digits with no delimeter. The first actual piece of data comes after the header.

Anyways.

Field 1: Version number of AI War save was made in
Field 2: UNKNOWN
Field 3: UNKNOWN
Field 4: A large collection of numbers, blanks, and one boolean. Possible a description of the settings used for the galaxy. Currently, meanings of sub-fields unknown
(I'll break this down later)

Fields 5-?: Human Player 1
And so on for each human player, each player taking at least 7 fields, with the exception for player one, which has at least 8 fields
For each human player:

1st field for player:
Header: 00
7 sub-fields, separated by ':':
Sub-field 1: Always 1 (?)
Sub-field 2: Player number
Sub-field 5: Player name (string)
Sub-field 6: Something to do with player type? (-196 for active human player, -1 for player not present, -65536 for AI player)
Sub-field 7: Something to do with player type? (2 for active human player, 1 for player not present, 1 for AI player)
Others UNKNOWN

2nd field for player:
Header: 08
3 sub-fields separated by ':':
Sub-field 1: Current metal (Fixed integer, shift amount TBD)
Sub-field 2: Current knowledge
Sub-field 3: Current crystal (Fixed integer, shift amount TBD)

3rd field for player:
Header: 09
4 sub-fields separated by ':':
Others UNKNOWN

4th field for player:
Header: 11
4 sub-fields separated by ':':
Sub-field 1: UNKNOWN (boolean)
Sub-field 2: UNKNOWN (boolean)
Others UNKNOWN

5th field for player:
Header: 12
9 sub-fields separated by ':':
Sub-fields UNKNOWN

6th field for player:
Header: 13
33 sub-fields separated by ',':
Sub-field 23: UNKNOWN (unknown type, allowed to be blank)
Sub-field 29: Always 400?
Sub-field 30: UNKNOWN (boolean)
Sub-field 31: UNKNOWN (boolean)
Others UNKNOWN

7th field for player (player 1 only):
Header: 14
13 sub-fields separated by ',' (with exceptions):
Sub-field 1: UNKNOWN (ends with ':')
Sub-field 6: UNKNOWN (unknown type, allowed to be blank)
Others UNKNOWN
As only player 1 has this info, I'm guessing it has to do with host settings.

8th field for player:
22 sub-fields separated by ',':
Seems to be some encoding of past information. I guess for the graphs and all that when they were still in the game.
Sub-field 16: UNKNOWN (has even further sub-fields with different delimiters, '^', '@')
Sub-field 22: Encoding of past resource information (?) (has even further sub-fields a different delimiter, '%')
Others UNKNOWN

After that comes a description of the AI players
After that comes to what seems to be a description of the planets.
After that, the descriptions of each ship.
And then finally is a descriptor of lobby settings used.

I will break down these sections later.

I think that these long lists of parameters will make more sense after I cross check them with the constructors of their representative class.

[x4000]

Lots of good analysis.  Couple of notes:

1. All the ships are saved at the end, after all the planets.  They have a reference to the planet ID.

2. The different delimiters are used for different levels of sub-nesting.  When you have a list within a list, you have to switch delimiters to do so in this sort of flat file export format, so that's what's going on there.

3. As you play, a lot of the player data is going to get larger and larger, because things about stats and whatnot get put in some of those fields -- over the course of a long game, that gets to be hundreds of sub-fields long.

4. I suggest mostly just copying the human lines as-is and not worrying about their format.  You can easily set up the names and colors of the players in the game itself, and even their handicaps, then just copy those lines into your desired scenarios verbatim to have it work.  Unless you wanted to adjust something like the metal or crystal reserves, which of course you could also do.

5. That pretty much leaves you with setting up the planets themselves, and then the ships after that.

6. And at the very end of the save file is actually most of the lobby options all serialized to one line, as I recall.  The easiest way to set those up would just be to set those up how you want in the lobby in-game, and then copy that into your savegame format.  It's mostly the planets and ships that you'd really want to customize, I'd think, but do as you will.

[TechSY730]

OK, AI player follows a similar format to the human players, except there are some extra fields, presumably for AI type, AI difficulty, and AI specific stuff like that. I'll break this down in more detail if I feel I need it.

Anyways, here is what I know about the format for planets. To save some typing, I will only list fields that I know what they are or seem to have a constant pattern.

Each planet has 31 fields.

Field 1: Planet ID
Field 2: X coordinate in canonical galaxy map
Field 3: Y coordinate in canonical galaxy map
Field 4: UNKNOWN (tends to be a negative number of a high absolute value (aka, negative with many digits))
Field 7: Name of planet (string)
Field 9: Adjacency list of all connected planets, separated by ':'
Field 11: Player ID of current owner (-1 if no one owns it)
Field 26: Planet notes (?) (Blank value is AAAAAB+LCAB487NOAP8=)
(Base64 string of a GZip Compressed string)
Field 30: Always 2 (?)
Field 31: Always 020 (?), except for the last planet, which is 049

As there is only one AI planet in tutorial 1, I can't really figure out what field is the planet's tech level. I'll have to get that from another save.

After this, I'm going to see if I can make sense of what I really care about, ship data.

[TechSY730]

Just in case anyone else once to check out what I am looking at, here is the save file I am using.

[x4000]

Regarding that base64 string, that's indeed what it is, but then once you un-encode that you've got a compressed string.  To get the raw string, just pass that whole base64 stuff to Compressor.DecompressStringGZip since you're linking against the DLL, and that will give you the result.  In this instance, it's probably empty string -- those are likely the planet notes, but I wouldn't swear to it.

[TechSY730]

The ship data is going to be hard, as there is no textual data assigned to any instance of a ship (or rather, ForegroundObject). As such, I'll have to try to cross reference based off of the ship type enum value and go off from there.

EDIT: Oh, also, I have updated the planet detail post with your tip about the planet notes.

[TechSY730]

OK, strange. I know that metal and crystal deposits (NOT harvesters) are most likely not instances of ForegroundObject, as there is no special debug info when you hover over them while debug mode is on. As such, they shouldn't be in the ship data section. However, I do not see anything that would line up with the observed values for the deposits in the planet data.

[x4000]

Metal and crystal deposits -- and wormholes, too -- actually are not in the savegame format at all.  They get procedurally generated every time the savegame is loaded (hence some of the "the wormhole moved" bugs in the past, when the procedural gen got altered at all). 

The only way to affect where wormholes and crystal deposits go is to change the overall map seed; and that will affect the seeding on every planet.  It's not possible to predict where these things will go, either, based on the map seed; it's deterministic in that you'll always get the same result, but it's not predictive.  Also: those are all OtherObjects, not ForegroundObjects, so they use a really different and scaled-down data structure (same one used by shots, explosions, rubble, rocks, and other similar miscellany).

Everything else, all the other ships and so on, are seeded in like you would expect.  So generally you'd want to find the position of the wormholes and crystal deposits, and then orient your scenario around that.

[TechSY730]

Well, there goes my idea of a joke map with 100 each each resource.

Oh well, at least I can still make my 2 planet "duel" map. Or the map with the inaccessible planet joke map.

Also, I can't seem to find where AI progress is stored. I know it would be two values, AI progress up and AI progress down (and possibly, AI progress floor). It should start at 10 up, 0 down, but I can't find such a pair in the file.

EDIT: Oh wait, I can simulate 100 of each resource by just spawning 100 of each harvester on a single planet.