A possible systematic approach to balancing the fleet ship types

[keith.lamothe]

This is basically a copy-and-paste from an email I sent Chris the other day, with some questions from me afterward:

-----------------

One of the most useful pieces of feedback I think we've gotten on the shield-network thing thus far is that people really don't value most of the bonus ship types.  It's not really surprising, given the whole middle-of-a-beta thing, but regardless of what happens with the shield-network, I'd like to really nail this particular problem soon.  Some of it is undoubtedly just a matter of user education and/or tastes, but I think a lot of it is that much of the game still hasn't recovered from the various "Balance Destroyer" changes we've made over the past months.

I've been thinking about this topic for a while, and I'd like to approach it in a more systematic way than we (or at least I) have done so in the past.  It may be a harebrained scheme, dunno, but it seems like it would have more inherent sanity-check and get much closer to an "apples and apples" comparison between the different fleet ship types.  Not that any system could really make it apples-and-apples without stripping out practically all the special mechanics, etc.

Anyway, I've thought through and done numeric models of a few different approaches, and the best one I've tried is inspired by the World of Warcraft "itemization" system.  In that system, each discrete stat or property that an item can have (+health, +spellDamage, +hitRating, etc) has a value in "item levels", and each item has an iLvl that is the sum of its properties' values.  They then go on to say that you can only use a green item with iLvl < (your level + 10), or a blue item with iLvl < (your level + 20), etc.  I'm sure I'm off on the numbers there and of course the iLvls detach from player level when they get into the endgame content of each expansion (though the mechanism is maintained for establishing which "tier" an item is in).

So that's the kind of model I've been hacking at between excel and VS (to get the game spitting out the desired csv files, probably pretty similar to the old secondary exports) a good part of today.  I'm certain that the actual functions I'm using to compute the "point value" of a value for each stat is way the waaay off, the first thing is to try to nail down a framework that makes any kind of sense and then iterate refinements to the actual computations.  It may be that such a framework is not achievable or at least that mine is too flawed to seriously try, dunno.

Very important is that we don't need to go _nearly_ as far as WoW took things, since our main goal right now is to just get all the types into the same ballpark of usefulness.  So we can keep things really simple and ignore tons of stuff and still reach an acceptable level of balance.  The stats I'm currently tracking are:

Survivability
1) Cap Health.  Meaning MaxHealth * EffectiveShipCap (this is all on High Caps, since that's the numeric starting point).
2) Armor Rating

Ease of Positioning
3) Move Speed
4) Attack Range

Ease of Support
5) Cap Metal + Crystal Cost
6) Cap Energy Cost

Stopping Power
7) Armor Piercing
Cap DPS Vs Scout Hull Type
9) Cap DPS Vs Light Hull Type
10) Cap DPS Vs Medium Hull Type
11) Cap DPS Vs Heavy Hull Type
12) Cap DPS Vs Artillery Hull Type
13) Cap DPS Vs Neutron Hull Type
14) Cap DPS Vs Swarmer Hull Type
15) Cap DPS Vs UltraLight Hull Type
16) Cap DPS Vs CloseCombat Hull Type
17) Cap DPS Vs CommandGrade Hull Type
18) Cap DPS Vs Refractive Hull Type
19) Cap DPS Vs Composite Hull Type
20) Cap DPS Vs Turret Hull Type
21) Cap DPS Vs UltraHeavy Hull Type
22) Cap DPS Vs Structural Hull Type
23) Cap DPS Vs Polycrystal Hull Type

Having one dps stat per hull type is more cumbersome than I'd like, but I've been a through several approaches on that and it's the most transparent way I can think of handling it.  Certainly couldn't have pulled this off before hull types, wow.

So this leaves out tons of stuff, like:
- Specific values for attack power, seconds-per-salvo, shots-per-salvo, or attack bonuses.
- Ratio of metal to crystal.
- Engine health (infinite engine health would be a separate property).
- Whether ship cap is large or small; the relevant stats compute based on total-at-cap.

Not that those don't impact balance, we just don't need to be that fine-grained with this.

I'm also ignoring knowledge cost since all bonus types have the same cost for mk2 and mk3.  If we started comparing turrets and/or starships obviously knowledge cost comes into play.

Only mkI ships are compared, it's assumed that higher mark versions have a linear progression of the relevant stats and that if the mkIs are all balanced the higher ones will at least not be too far off.

Also, for my first pass I'm only balancing those fleet ships that don't have any serious special feature, they're just there for dps.  Meaning no cloaking, no teleporting, no debuff/buff, no self-attrition, no kamikaze, etc.  If we can sort this out for the "just dps" ships we can work out towards those more exotic cases.  The types I'm looking at now are:

AcidSprayer
AntiArmorShip
ArmorShip
Bomber
BulletproofFighter
Fighter
LazerGatling
MicroFighter
MissileShip
MLRS
Raider
SentinelFrigate
Sniper
SpaceTank
ZenithBombardment
ZenithChameleon
ZenithElectricBomber

Not even all those are pure vanilla (obviously the game isn't going to have many of those, it wouldn't be very interesting), but close enough for my purposes.

So, here's the current point cost computations in my working copy:

Code: [Select]

   private static int GetPickCostForStatValue( BalanceCategory BalanceCategory, int StatValue )
    {
        switch ( BalanceCategory )
        {
            case BalanceCategory.CapHealth:
                if ( StatValue <= 250000 )
                    return 0;
                return ( StatValue - 250000 ) / 25000;
            case BalanceCategory.ArmorRating:
                return StatValue / 10;
            case BalanceCategory.ArmorPiercing:
                if ( StatValue > 20000 )
                    return 250;
                if ( StatValue > 5000 )
                    return 125;
                return StatValue / 20;
            case BalanceCategory.MoveSpeed:
                if ( StatValue <= 22 )
                    return 0;
                return ( StatValue - 22 ) * 10;
            case BalanceCategory.CapMetalCrystalCost:
                if ( StatValue <= 4000 ) // 10 m + 10 c
                    return 300;
                else if ( StatValue <= 8000 )
                    return 260;
                else if ( StatValue <= 12000 )
                    return 240;
                else if ( StatValue <= 16000 )
                    return 230;
                else if ( StatValue <= 20000 )
                    return 220;
                else if ( StatValue <= 26000 )
                    return 210;
                else if ( StatValue <= 30000 )
                    return 200;
                else if ( StatValue <= 40000 ) // 100 m + 100 c
                    return 180;
                else if ( StatValue <= 200000 ) // 500 m + 500 c
                    return 100 + ( ( -( StatValue - 200000 ) ) / 2500 );
                else if ( StatValue <= 500000 ) // 1250 m + 1250 c
                    return ( ( -( StatValue - 500000 ) ) / 3100 );
                else
                    return 0;
            case BalanceCategory.CapEnergyCost:
                if ( StatValue <= 2000 ) // 10 e
                    return 300;
                else if ( StatValue <= 4000 ) // 20 e
                    return 250;
                else if ( StatValue <= 6000 ) // 30 e
                    return 200;
                else if ( StatValue <= 8000 ) // 40 e
                    return 170;
                else if ( StatValue <= 10000 ) // 50 e
                    return 140;
                else if ( StatValue <= 15000 ) // 75 e
                    return 120;
                else if ( StatValue <= 20000 ) // 100 e
                    return 100;
                else if ( StatValue <= 30000 ) // 150 e
                    return 80;
                else if ( StatValue <= 40000 ) // 200 e
                    return 60;
                else if ( StatValue <= 50000 )
                    return 50;
                else if ( StatValue <= 60000 )
                    return 40;
                else if ( StatValue <= 70000 )
                    return 30;
                else if ( StatValue <= 80000 )
                    return 20;
                else if ( StatValue <= 90000 )
                    return 10;
                else if ( StatValue <= 100000 )
                    return 0;
                else
                    return 0;
            case BalanceCategory.AttackRange:
                if ( StatValue <= 0 ) // melee
                    return -50;
                else if ( StatValue <= 2000 )
                    return 0;
                else if ( StatValue <= 3000 )
                    return 5;
                else if ( StatValue <= 4000 )
                    return 10;
                else if ( StatValue <= 5500 )
                    return 20;
                else if ( StatValue <= 7000 )
                    return 30;
                else if ( StatValue <= 8500 )
                    return 40;
                else if ( StatValue <= 10000 )
                    return 50;
                else if ( StatValue <= 20000 )
                    return 75;
                else if ( StatValue <= 30000 )
                    return 100;
                else if ( StatValue <= 40000 )
                    return 110;
                else // sniper
                    return 250;
            case BalanceCategory.CapDPSVs_Scout:
            case BalanceCategory.CapDPSVs_CommandGrade:
                return StatValue / 2000;
            case BalanceCategory.CapDPSVs_Light:
            case BalanceCategory.CapDPSVs_Medium:
            case BalanceCategory.CapDPSVs_Heavy:
            case BalanceCategory.CapDPSVs_Artillery:
            case BalanceCategory.CapDPSVs_Neutron:
            case BalanceCategory.CapDPSVs_Swarmer:
            case BalanceCategory.CapDPSVs_UltraLight:
            case BalanceCategory.CapDPSVs_CloseCombat:
            case BalanceCategory.CapDPSVs_Refractive:
            case BalanceCategory.CapDPSVs_Composite:
            case BalanceCategory.CapDPSVs_Turret:
            case BalanceCategory.CapDPSVs_UltraHeavy:
            case BalanceCategory.CapDPSVs_Structural:
            case BalanceCategory.CapDPSVs_Polycrystal:
                return StatValue / 1000;
        }
        return 0;
    }

Like I said, I'm sure those numbers are way the waaay off
    
And I've attached the spreadsheet of the data output from that for the aforementioned types (actually all the other types are in there too, but I've got it filtered). It's interesting to note that even as bad as my first-guess algorithms are, it seems pretty clear that the LazerGatling and the ZenithChameleon are not playing by the same rules balance wise

-----------------

Anyway, my questions for y'all are:

1) Do you think this framework has a chance of being useful in achieving all-bonus-types-in-the-same-ballpark-of-usefulness?  Obviously the more exotic ships require additional consideration, etc.
2) Assuming the answer to 1 is "yes", do you have suggestions on the computation of "pick cost" for each stat?  For instance:
- The various dps-vs-hull stats need different valuation because some are much more useful to be able to kill than others, etc, but some (like UltraHeavy) also have tons more hp in general.
- Speaking of that, the value of the health stat should probably vary by the ship's own hull type; having 10 million hp as an ultra-light is very different than 10 million hp as ultra-heavy.
- The energy and m+c cost stats could stand to be less step-wise, certainly.  We can consider logarithmic functions and other such fun stuff if it seems necessary; it's not like this is being computed in the middle of a battle
- Range perhaps should simply multiply the total cost of the damage stats rather than having its own stat.  This might apply to other things too (like armor rating multiplying the cost of health).
- In general, the relative weighting between stats is just way off
- Etc; these certainly aren't me saying "this is how important this stat is, period", they're just first guesses to get some kind of example output from the framework.

This certainly would not be our only tool for finding a good balance, but it is helpful for spotting obvious outliers (hello, ZenithChameleon?) and for having some kind of objective-scalar baseline when anecdotal feedback from players can vary from "it's massively OP" to "it's massively UP"

Note: the attached .xls file has the filter and sorting already applied and should be readable through either Office or OpenOffice; but if you want a truly opens-anywhere file the csv is provided though it doesn't have the filter or sort applied.

[Red Cossack]

Personally, my choices of bonus ships are almost always something that is good at nailing the teleporters and shield ignorers.  I'm relatively new at the game, and still having problems dealing with 200+ clumps that fly by my defenses (immune to tractor beams) and straight into my command stations bubble (immune to shields) and then burn into another system and repeat.  I quite often loose 2-3 systems if I don't have my main fleet sitting on the avenue of attack. 

So for me, I don't experiment with as many of the unusual ships as I should, but I do like what you laid out and I think when more of the number crunchers chime in, it should be embraced/discussed more.

[Wingflier]

Actually this is a wonderful idea, and I thought you guys already had something like this in place.

Without some kind of universal baseline like the one you are describing, it's hard to believe that the bonus ships will ever reach anything even close to balanced.

We as a community could help (I think) by giving you examples of what we think are the most "balanced" bonus ship types.  Using the "balanced" types as a baseline, you can then begin tweaking all the other ships to be, as you said, "within the ballpark" of those ships.  Obviously things like cloaking, tachyon emission, shield boosting, etc. will be harder to balance, but that's where trial and error comes in, and I certainly don't think finding a balance for those will be impossible.

So great job with this idea, and I'm excited to see how it works and help it grow!

One thing I think you guys could do to make your job a lot easier is to "standarize" all the bonus ships the way you did with fighters.  Having a ship type that is only mediocre against most things and ridiculously good against a few others is going to be hell to balance.  Much better would to make most all ships fairly good against everything, with a few bonuses against particular ship types.  In this way, you can more or less use "special damages" as a standard practice that each ship will possess, more or less eliminating it from the equation in most cases; while still retaining a "uniqueness" for each ship.

[keith.lamothe]

Actually this is a wonderful idea, and I thought you guys already had something like this in place.

Chris has a pretty hardcore simulation that he uses, but he's been really busy with other stuff and it's hard to express that model in simple terms (since it's literally simulating the game, which is great for sanity checks too).

Without some kind of universal baseline like the one you are describing, it's hard to believe that the bonus ships will ever reach anything even close to balanced.

Yes, that's exactly my point

Anyway, yea, if you can look at the example data... ah, I see, I forgot to attach that, will fix

[superking]

Without critisisng that proposal, I am an absolute disbeleiver in deriving balance using mathmatical calculation:

• it is impossible to realisticallly factor in everything
• it is very difficult to even get a formula that represents the reality of balance
• it results in less contrast between unit types
• it makes for dull balance
• it gives hugely inferior results to the process of incrementally balancing and nerfing based on player feedback

That oppinion is a product of 5 years involvement in the balance of total annihlation derivative mods via TA spring, and it is impossible to list how many balance models for multiplayer RTS balance have been tried in that enviroment (so many).

The most successful method in spring by a huge margin just followed a 'make units each really cool, then refine them between releases until everything is useful'

in RL, the balance of fleetships in last 3.0 version was pretty good, and I have played that last release to death. OK, zenith polarisers were dominating- but no unit besides acid sprayers & neinzul ships, raptors & microfighters were actually non-viable choices of starting unit.

imo, the best way to balance an RTS is just make every unit cool to use from a players perspective, then make incremental changes accordingly. I think acting upon the feedbac  in the ARS unlock voting thead, and making changes based upon your own intuition as the developers and people who know the game best would give infitiely better results than some equation you've made in excell.

So yeah: the objective imo is just balance based upon feedback and intuition in a way that every unit is cool, fun and effective to use in the hands of the player, which can be acheived by changes of shipcap and cost as well tweaking as unit stats and roles. every unit should be considered by the criteria: 'is this a cool idea? do I want to get out there and use this ship?'

the zenith charmelon for example, isnt cool. its fragile, weak attacker with an ability that is unsuited to to both its role- aggression- and the purpose of building ships instead of turrets as a player- aggression. imo, it should be rebalanced as a defensive ambusher unit that functions like a toad- slow moving, then stops to camoflage itself, and when somthing comes close, BAM
cool idea balance method gives much more interesting unit rosters than mathmatical balancing.

just my 2cents, I support any direction AI war goes, I just have alot of misgivings about balancing methods that involve excell.. I really think that by now you guys & this community knows what balance changes to be made without scrapping it in favour of impartial maths 

edit: I have an established history of being somthing of an opposition party on the AI war forum, and I just want to say, I realyl hope I dont come accross as too much of a jerk when I make these kind of posts... thats not my intention.

[x4000]

Well, bear in mind that I have a whole system for balancing bonus ships and so forth myself, and I'll keep using that while Keith uses his.  This provides sort of a different way of partitioning a complex set of data.

The other key benefit to Keith's approach is that it doesn't require 15+ minutes of simulation data to generate.  In the case of mine, doing balance is really tricky because I have to wait 15+ minutes, then look at the data and make changes, then wait another 15+ minutes, and so forth.  That makes it into a really big event for me to do balance.  

Consequently, I haven't even attempted to use those tools yet since 4.0 came out and we've added all these new ship types and redefined a number of existing ones, etc.  I wanted to get everything in and functional before I started trying to balance with that.

The benefit here is that Keith's stuff will let us get things balanced more easily and more quickly, and it's something we can do anytime without it being such a big event.  And then at the end I can double-check things with my existing balancing methods, and hopefully we arrive at something that is as fine-tuned as possible.

But no matter what methods we use, there's always an incompelete view of things, because mostly what we're evaluating is single-ship-type combat, which is not really the most valid thing.  It undervalues things like shield bearers or munitions boosters, for example, which have value mainly in terms of boosting other ships.  It also really undervalues things that have extreme range, and thus can be used in interesting tactical situations.  Or things that are excessively cheap/fast to build, and which thus can be used for churn-and-burn.  Or things that are cloaked, and thus can be used for stealth attacks that have abnormal value to the players.

So there's something of an art to it, though we try to inject as much science as possible into the process.  I'm very happy with the idea of Keith's stuff, and how it will improve the science-y side of this, but I don't think there's any model we can use other than simple playtesting and player feedback that will catch everything.  Sometimes it's good to have bonus ships that are underpowered alone, but which are amazing when used in combinations with other ships.  I think that's really valid, and balancing those so that they are also completely powerful on their own would be a bad thing.

Anywho -- should be good.

[keith.lamothe]

@Superking,
Yes, I wouldn't just be making changes because the spreadsheet told me to  To re-iterate:

This certainly would not be our only tool for finding a good balance, but it is helpful for spotting obvious outliers (hello, ZenithChameleon?) and for having some kind of objective-scalar baseline when anecdotal feedback from players can vary from "it's massively OP" to "it's massively UP"

But yes, I do appreciate the feedback that such models have not worked as primary balance tools in other cases.

[x4000]

And superking: I totally agree with pretty much everything you said.  I've argued with other game designers from exactly your position, and it's a strangely unpopular position (I guess since so many are math-heavy).  But I'm really glad to see someone other than me espousing that position.

That said, it's possible for us to use the more math-y approaches to get a better "first guess" that then gets iteratively refined through playtesting feedback.  That's the point of the math-y parts that I already have in place, and why I'm excited about how Keith's process will make that faster and easier to turn around.

EDIT: Basically ninja'd by Keith.  But anyhow.

[Suzera]

As long as ships are all surviveable enough to where they can get into range to fire at things for most of their lifetime, range and speed are almost a non issue since the change from shields -> armor. It only really matters if something is really really slow or immobile, like turrets. As an example, frigate range and speed doesn't matter too much because A) They move way more than fast enough that they can get in range quick enough that most of their combat life is spent inside optimal firing range and B) Fighters and other things that are comparatively as useful that cruisers blow up all move way more than fast enough that only a small portion of their combat life is spent out of firing range but inside frigate range. Sniper length range matters because you can hit and run, but it is relatively irrelevant in most cast because fleet blobs or turret blobs vaporize almost everything anyway, and sniper + spider turrets do the job fine on planets. If everything all automatically microed their movement to be exactly as far away from their preferred targets as they could get but still be in range, range and speed would matter far far more. But they don't, so unless you're pause/unpausing every 5 seconds to do the ideal triangle micro, range and speed don't matter all that much for mobile things right now, as long as they don't interfere with your logistics which puts the minimum speed for offense ships around 22, but that is still a more minor thing.

It would really help your mathematical model if the armor rating subtraction was before the damage multiplier instead of after and everything were rebalanced on that.

Now:
floor( Damage * multiplier - |armor - AP|, damage * multiplier * .2)

Proposed:
floor( (Damage - |armor - AP|) * multiplier, damage*multiplier*.2) )

This way, you won't have cases where doubling the base per shot damage of a fighter subtype causes more than twice the amount of damage to bombers, or vice versa, and it would be far far more predictable to balance, particularly with how the calculations scale across low/medium/high ship cap settings. They would be much closer together.

AP also needs to scale with ship cap settings. Fighters have more AP than bomber armor on low, but less AP than bomber armor on high.

If you do both of those, the time to kills across ship cap settings in a ship cap vs ship cap sense should be very very close (if not theoretically identical if they approached in parallel lines if I am doing my math right) and make it much less of a concern which ship cap scaling you are using and thus, less balance difference between the three.

I still need a bit to look at the sheet, but I figured I would post at least this for now before you get too far into this.

[Suzera]

I noticed you set 22 as you 0 cost for movement speed. Clever.

This is a good method for a balance starting point (obviously things like ship formations and ship combos with things like FFBs will skew effectiveness) but I would really like to see the formulas you used to determine cost laid out rather than have to rederive them.

[shugyosha]

As far as I can see this approach simply adds points together for unit stats.

With years of experience as tabletop gamer let me tell you this method won't work even as a framework. The stats of a unit are functions of one another. In order to make any meaning out of the attack power of a unit you need to consider how long it can survive. Higher values should also be valued exponentially insteadt of just increasing the values slightly.

Some thoughts:

There are three main areas, sometimes four: Movement, Defense, Offense and sometimes special abilities. All areas are tied together. Moving away helps defense. Slow movement speed paired with low range is bad for the offense. A strong defense means staying power and directly affects the offensive strength as the unit can dish out damage longer.

Unit cap and resource cost should not be included at first. Try to pitch triangle ships against each other and find a formula that represents the outcome roughly without specific damage types.

How man fighters do you need to kill one Missile Frigate if the Fighters move towards it and the Frigate away. Set the Point value of Fighters to 100 each and then try to make a formula just for determining the the points of the frigate. This will give you a lot of insight about the formula you need.

I firmly believe there is a formula that can give a broad overview about unit strength but it might very well be extremely time consuming to find it.

[keith.lamothe]

AP also needs to scale with ship cap settings. Fighters have more AP than bomber armor on low, but less AP than bomber armor on high.

Yea, of course that means I'll have to do the same thing I did with armor rating: have the "under the hood" calculation use the un-scaled value for shooting non-scaling targets (like guardians).

[Admiral]

My main suggestion is to take your piece of code, which is essentially a rule, and make it available for players to modify. Use a rules engine (not sure what engines are available for your development platform) or a domain-specific language (a little Lisp or Lua or something) or even just a text or XML file. This way people can play with the numbers for themselves and figure out where the sweet spots might be.

Thoughts?

Cheers!

[keith.lamothe]

but I would really like to see the formulas you used to determine cost laid out rather than have to rederive them.

I included that in my original post on this topic, or is that not showing up?  It's in a code block.

[keith.lamothe]

My main suggestion is to take your piece of code, which is essentially a rule, and make it available for players to modify. Use a rules engine (not sure what engines are available for your development platform) or a domain-specific language (a little Lisp or Lua or something) or even just a text or XML file. This way people can play with the numbers for themselves and figure out where the sweet spots might be.

That would be interesting, yes.  But there's no way I'm going to try to fold a scripting language into a unity app just for this   I could let you set the variables in a separate .dat file or whatever but even then you're going to run into problems of what-if-I-want-to-use-an-exponential-function or what-if-I-want-it-to-vary-by-this-other-stat, etc.

I think you could do what you wanted to in the excel sheet I provided, though I guess it would help if it was using formulas instead of literals for the cost cells.

The button I used to generate the excel sheet is on the reference tab so you'll be able to generate it yourself, I just haven't checked in my changes yet.