Measureent of what game options people actually use?

[Tridus]

One of the issues that I'm seeing in trying to figure out what to do in AIW2 is in figuring out what options are actually being used by players. Wouldn't it be great if that was just captured when people start a new game? Then Chris would *know* what people like and don't like.

It's too late to do that for AIW Classic at this point, but not too late to do it in AIW2. You could prompt people at some point asking if you can collect "anonymous game creation statistics" or some such, then on game creation, if they're online, send a message to a server with what was used.

There's libraries that can do this out there as well, but a lot of them are aimed at F2P games and you don't need anything so invasive.

[x4000]

Yeah, it would be good to do that.  There are a lot less invasive libraries out there now than there were back when I was doing original AI War.  Back then I talked to several companies that were pitching player metrics, and it was super sketchy stuff in my opinion, so I backed away carefully.

[Captain Jack]

Yeah, it would be good to do that.  There are a lot less invasive libraries out there now than there were back when I was doing original AI War.  Back then I talked to several companies that were pitching player metrics, and it was super sketchy stuff in my opinion, so I backed away carefully.

Just make sure it's opt IN instead of opt out!

[Tridus]

Yeah, it would be good to do that.  There are a lot less invasive libraries out there now than there were back when I was doing original AI War.  Back then I talked to several companies that were pitching player metrics, and it was super sketchy stuff in my opinion, so I backed away carefully.

Definitely a good call. It's useful information to have, but some of them go WAY overboard in what's collected. For something non-invasive, I'd opt in for sure.

[x4000]

Makes perfect sense!

[Draco18s]

Just make sure it's opt IN instead of opt out!

Prompt on first run, "Gather player metrics? [Yes/No] (This can be changed in the options at any time)"

[Captain Jack]

Just make sure it's opt IN instead of opt out!

Prompt on first run, "Gather player metrics? [Yes/No] (This can be changed in the options at any time)"

Metrics sounds a bit sinister, I think. I'd go really colloquial and say "Tell Arcen what options I'm playing the game with".

[Orelius]

For fairly nonintrusive (and kind of expected) metric gathering, it could just be tied to steam achievements.  Achievements like 'play X games for Y time with setting Z turned on' could be good for tracking some metrics on all steam players.  It would require a massive amount of achievements but given that AI War Classic already has so many, I don't think it would be a big problem for AI War 2.

Alternatively, there could be a 'submit a high score to the servers' mechanic that could submit the details of the game and the score, which could then be analyzed.

On that note, a 'weekly seed' mode would be a pretty cool option to have, lots of roguelikes I play do that to add more of a sense of community and competition.  Win rates and other assorted things could be collected from the high score and whatnot collected from these.

[Draco18s]

Metrics sounds a bit sinister, I think. I'd go really colloquial and say "Tell Arcen what options I'm playing the game with".

*Shrug*
Whatever you want to call it.

[tadrinth]

This sounds like it might be tremendously valuable, especially if it doesn't have too much dev overhead (either for sending the data or investigating the results). 

If you tie the stats to starting a new game, that can get skewed by sandbox games.  I sometimes make games just to investigate a mechanic, and abandon it after a few minutes of testing.

On the other hand, if you go by achievements or score at victory, you'd only get numbers on completed games... and lots of games don't get finished.

I think ideally you'd sample at random time intervals (real world, not game clock), as well as upon winning or losing a game. 

If something along the lines of ARS hacking sticks around, then I'd probably want to sample that also. If people are consistently hacking an ARS to get ship X instead of ship Y, that suggests candidates for balance reviews.  Might be hard to get much data here, and with mods that data might be difficult to interpret, though. 

You might want to include a heartfelt 'this really helps our small dev team devote our time wisely' bit in any explanation around the opt-in. 

[kasnavada]

Steam achievements can give some basic metrics .
Actually AI War already has some achievement which are basically simple metrics.

"Started a game with option XXX".
"Finished a game with option XXX".
"Played for 50 hours".

The results are scary though.
Rainy Day Savings: 50,000 Energy => that's basically the number of people that started a game : 28%.

Nearing The End? 10 Hour Campaign => Played for a bit ? 5.5%...

First Victory ? => 3.6%.

These numbers need a serious buff, the ratio between started a game and finished should be much higher...

[Pumpkin]

I don't think "starting a game with option X" would provide accurate data, because it can be done by players for testing, or out of curiosity. It wouldn't tell "N% players liked option X". Nor would do "winning a game with option X".

[kasnavada]

I don't think "starting a game with option X" would provide accurate data, because it can be done by players for testing, or out of curiosity. It wouldn't tell "N% players liked option X". Nor would do "winning a game with option X".

I'd agree on the principle, but the number of people that would do that out of curiosity is fairly minor, as the numbers of steam achievement show.

[Tridus]

I don't think "starting a game with option X" would provide accurate data, because it can be done by players for testing, or out of curiosity. It wouldn't tell "N% players liked option X". Nor would do "winning a game with option X".

The more accurate you want to get, the more invasive the data capture has to be. This is a very simple method that doesn't log what people are doing while playing at all, so it's simple and easy to explain to the privacy conscious exactly what its doing.

You're unlikely to get perfect numbers no matter what you do, but this would tell us a lot. Primarily stuff like "with all the talk about Champions we had, is anybody actually *using* them?" If it turns out that people always use Champions, it's a different discussion than if people never use Champions.

Right now? We have absolutely no idea what anyone not on the forum (aka: the majority) is doing.

[Misery]

Rainy Day Savings: 50,000 Energy => that's basically the number of people that started a game : 28%.

Okay, that number surprised me.  I mean, really, 28% is all that bothered to go that far?

Looked at the other achievements, the percentages really are just amazingly low, aren't they....  It's like people either go through the tutorial and then just give up, or buy it and then just never touch the game after that.

Just... ugh.