Author Topic: Any updates on the way?  (Read 9005 times)

Offline Shadou

  • Newbie
  • *
  • Posts: 8
Any updates on the way?
« on: May 16, 2016, 05:21:40 pm »
It's been a year and a half since the last patch. Is this game still being supported -at all-, or is this an out of development, finished, not fixing any more bugs release?

In december 2014, I tracked a problem with crippling framerate drops down to the spire gravity drain ship, and posted my findings on the mantis tracker. Since then, I've been wondering when spire gravity drain ships are ever going to be safe to allow to be in my games. Is this kind of issue ever going to be fixed? Or are we pretty much feature frozen and what you have is what you get?

Offline Aklyon

  • Core Member
  • *****
  • Posts: 2,089
Re: Any updates on the way?
« Reply #1 on: May 16, 2016, 05:47:31 pm »
They are busy and this has been asked before. Currently right below this thread in fact.

Offline x4000

  • Chris McElligott Park, Arcen Founder and Lead Dev
  • Arcen Staff
  • Zenith Council Member Mark III
  • *****
  • Posts: 31,651
Re: Any updates on the way?
« Reply #2 on: May 16, 2016, 07:16:02 pm »
Yep, that's more or less the state of things right now.  That doesn't mean that the current patch is the last one that the current game will ever see, by any stretch.  We've been really stretched thin, though.
Have ideas or bug reports for one of our games?  Mantis for Suggestions and Bug Reports. Thanks for helping to make our games better!

Offline tadrinth

  • Hero Member
  • *****
  • Posts: 507
Re: Any updates on the way?
« Reply #3 on: May 16, 2016, 09:25:45 pm »
What's the stance on volunteer work for AI War?  I think previously the answer was always 'code base too scary', yeah? There are bugs I would happily try to fix in my free time.  I might not be able to fix engine bugs, but I bet I could tweak unit definitions. 

The gravity drain framerate issue sounds like a real pain to fix, though.  You could bandaid it somewhat by changing them to a ship cap of 5, though. 

Last optimization to gravity effects I can find is 5.007. 

I will note that you should usually not see 5000 AI grav drains in the air anymore; a wave of 5000 grav drains will show up as 2000 loose grav drains and three carriers.  That will still probably wreck the CPU, though. 


Offline Aklyon

  • Core Member
  • *****
  • Posts: 2,089
Re: Any updates on the way?
« Reply #4 on: May 17, 2016, 01:34:57 am »
I believe the previous code analogy involved sawmills and terror, or something like that.

Offline x4000

  • Chris McElligott Park, Arcen Founder and Lead Dev
  • Arcen Staff
  • Zenith Council Member Mark III
  • *****
  • Posts: 31,651
Re: Any updates on the way?
« Reply #5 on: May 17, 2016, 09:32:20 am »
AI War volunteer fixes are particularly tricky because the ramifications of changes are not really going to be immediately obvious in all cases. 

Imagine, if you will, a car that is able to go incredibly fast because you took out all the safety equipment, modified the seats, stuck jet engines inside where the combustion engine should be (with VERY careful exhaust and airflow concerns or the car stops moving or incinerates you or both).

You might say "why not just add a bit of safety equipment in this one spot at least?"  But the problem is that at the speeds this car goes, that actually causes something like a 10% degradation in performance even just to put a seatbelt in.  The metaphor breaks down here. ;)

You might also say "why not move the jet engines OUTSIDE" the car?  But it has to conform to the general shape of a car for a whole host of regulatory and pragmatic issues that require you to know all the possible lane dimensions, parking dimensions, and automobile laws of six countries.  There are some things you can do to still optimize it further, such as perhaps letting fire jet out past the door, but there's a little-known clause that says you'll get a ticket if the fire is more than six inches from the bottom of the door.  Turns out that from testing we've discovered that 5.1 inches from the bottom of the door is the ideal location for the fire in that 0-6 inch range, and that mostly has to do with the precise placement of the engines in the chassis, and oddly enough where the air conditioner vents go (not that those are an air conditioner now).  If you want to adjust the position of the engine in the chassis, you'll also need to recalculate the ideal height of the flame jets on the side exhaust.

And for the love of all don't use the turn signal. ;)

It is very possible to work in an environment such as this, and you can make some incredibly performant stuff like this.  Once upon a time, pretty well all games were coded this way, if you go way on back.  But the number of external APIs you had to be familiar with was lower, while the general math you had to know was higher.

It's just not the sort of thing one tinkers with casually, generally speaking.  Ultimately I'd need to talk to Keith to get his feelings on some of this, but that's my initial thoughts.  If we start having a desync 6 months from now, are you going to be available to hunt it down, and is it fully your fault in the first place or did you uncover something else, etc, etc.
Have ideas or bug reports for one of our games?  Mantis for Suggestions and Bug Reports. Thanks for helping to make our games better!

Offline tadrinth

  • Hero Member
  • *****
  • Posts: 507
Re: Any updates on the way?
« Reply #6 on: May 17, 2016, 10:18:03 am »
Yeesh. I've worked with some poorly documented performance-oriented code in my day job, but that sounds an order of magnitude worse than what I'm used to. 

On the other hand, of the tickets I've submitted that are unresolved, two are tooltips, one is mapgen-related, one is a map filter, and two are (I think) unit definition tweaks.  Plus a sort that's backwards from intended. 

If even those are dangerous to the unwary, then yeah, not gonna be worth it.  But if those are things that a volunteer could work on, well, collectively that's around half the tickets I've filed.

Also depends a lot on the effort involved in getting volunteers set up, how many volunteers will be able to usefully contribute without breaking stuff, and how much bandwidth you guys have for reading pull requests and doing releases.

Realistically, if you're planning on a rework for improved moddability anyway, it's probably wiser to wait on that.

Offline x4000

  • Chris McElligott Park, Arcen Founder and Lead Dev
  • Arcen Staff
  • Zenith Council Member Mark III
  • *****
  • Posts: 31,651
Re: Any updates on the way?
« Reply #7 on: May 17, 2016, 10:34:18 am »
The rework would be a paid sequel, so I wouldn't want to wait on that one -- it's a complete reimagining.

Generally speaking, in the interface you're pretty safe on making changes so long as you don't call the wrong Rand() -- one that is part of the sim -- or mess with the state on any game-sim object.

Overall the code here could stand to be better organized, but its lack of clarity isn't really from lack of organization or comments (though those are sparse).  Rather it comes from the fact that you need to know the internal performance characteristics of a whole lot of things in order to know the full ramifications of doing x y or z, and that can involve tracing a lot of code since things get rather deeply nested (it outright breaks the profiler in unity as soon as you turn it on).  It also requires understanding the tradeoffs between RAM usage and cleanup and garbage collection caused by things like foreach or whatever, etc, etc.

Plus in general the overall desync-avoidance, making sure that things stay in their boxes and that the AI thread looks but does not touch, and the interface is the same, etc.  The AI thread and the interface can only touch the game objects by sending commands through the network, which then get synchronously executed on all clients.

So there's kind of a separation enforcement there that gets tricky.

Right now Keith is out on family leave, but once he's back we can see what he thinks.
Have ideas or bug reports for one of our games?  Mantis for Suggestions and Bug Reports. Thanks for helping to make our games better!

Offline chemical_art

  • Core Member Mark IV
  • *****
  • Posts: 3,952
  • Fabulous
Re: Any updates on the way?
« Reply #8 on: May 17, 2016, 11:06:09 am »
...making sure that things stay in their boxes and that the AI thread looks but does not touch...

As a non-programmer that is the most amusing analogy I have ever heard about the subject.
Life is short. Have fun.

Offline x4000

  • Chris McElligott Park, Arcen Founder and Lead Dev
  • Arcen Staff
  • Zenith Council Member Mark III
  • *****
  • Posts: 31,651
Re: Any updates on the way?
« Reply #9 on: May 17, 2016, 11:08:34 am »
Hahahaha. :D
Have ideas or bug reports for one of our games?  Mantis for Suggestions and Bug Reports. Thanks for helping to make our games better!

Offline tadrinth

  • Hero Member
  • *****
  • Posts: 507
Re: Any updates on the way?
« Reply #10 on: May 17, 2016, 01:25:58 pm »
Well, while I've got you talking about the engine, and on the topic of gravity drains... what kind of spatial data structures are you using for collision detection/range finding/gravity?  Would naively assume something like a PR quadtree.  Just curious; haven't looked at spatial data structures since undergrad, but thinking about the gravity drain issue reminded me of them.
« Last Edit: May 17, 2016, 02:04:21 pm by tadrinth »

Offline x4000

  • Chris McElligott Park, Arcen Founder and Lead Dev
  • Arcen Staff
  • Zenith Council Member Mark III
  • *****
  • Posts: 31,651
Re: Any updates on the way?
« Reply #11 on: May 17, 2016, 03:02:22 pm »
We don't use quadtrees or any sort of spatial separation in there, because unfortunately depending on the usage of a given search, we wind up having to search far too much of the planet to make it feasible.  The amount of space that is being discussed there is also too high, and there is nothing in most of them.  There are a lot of ships that can hit things that would be in a large number of buckets, and some that would hit them in all, so it becomes a very inefficient way of sorting them. 

In TLF and Starward Rogue we don't have those same restrictions, so we use quadtrees and that's a big help.  With AI War it would make the ram spiral out of control as well as it doesn't seem to be effective anyhow.

Instead we use what we call rollups, which are a lot of subsets of data based on uses.  I believe there is something like 30ish distinct rollups per player per planet, or something on that order.

The core goal of spatial partitioning is that you want to iterate over as few things as possible for something like collisions.  But spatial issues are not the core cause of iteration in AI War.  Only when ships are stopping do they collide with anything at all, anyway.  Otherwise they can fly right past one another.  Shot hits are precalculated, not calculated on the fly, unlike in TLF or SR.

Instead our biggest problem -- by many orders of magnitude, and I use that term literally -- is target selection.  It could be engineers deciding what to repair or build, ships deciding what to autotarget to shoot at, or who knows.  We use a variety of tricks to help make those iterations as few as possible (the rollups are a huge part of that), but ultimately it's just a huge amount of data to crunch and there's no way around that.  So we have a cross-frame way of breaking up that data and throttling it so that it doesn't completely overwhelm the framerate most of the time.

With something like the gravity stuff causing issues, it's probably either a matter of there needing to be a different lookup, or different uses of lookups, or else throttling to keep it from doing too many of those calculations in one frame.  Most likely that's what would need to be done.  It should already be using all the distance speedups like doing hacky general range checks first, then doing better approximations if it's even feasible that something is in range, etc.

If it proves to be absolutely a hog of a method, then it might be a candidate for some inlining of an existing method to avoid the cost of a method call that might happen a million times a second in the current iteration.  And cutting that back from a million to something like a throttled 10k per second, potentially.

Something along those lines, anyhow.
Have ideas or bug reports for one of our games?  Mantis for Suggestions and Bug Reports. Thanks for helping to make our games better!

Offline tadrinth

  • Hero Member
  • *****
  • Posts: 507
Re: Any updates on the way?
« Reply #12 on: May 17, 2016, 03:53:33 pm »
Awesome, thanks for the detailed response!

I was thinking quad trees might be a good fit for gravity effects (in contrast to targeting/shooting/etc) because:
1) grav effects are either global or cap out at 8000 range
2) grav effects don't stack

So I was imagining something like a quad-tree but where the quadtree nodes stop subdividing after 1000x1000 (or whatever size) and then just have lists, rather than a full quadtree.  You calculate the grav areas first, and then label nodes as either 'no gravity', 'fully covered by gravity' (and so everything in the list is affected, no need for a range check), and 'partially covered' (fall back to existing logic). Since you're not fully subdividing, the overhead of the nodes and movement should be smaller.

Since a wave of 5000 grav drains will all initially be in the same spot, they'll basically 'paint' the same parts of the quadtree over and over again (linear-ish operation), and then for all ships in nodes fully covered, there's no range check so that's linear, but then if you're on the edge of a pile of drains, you're still back to some horrible quadratic comparison.  You could throttle the quadratic comparisons per second to something reasonable though, and maybe also throttle the 'paint' step.  That's probably too complicated and/or too much overhead for the benefit though, considering you wouldn't use that structure for anything BUT gravity. 

Changing grav drains to a ship cap of 5 would be easier, though. And that's something a volunteer dev could do. =)

Wouldn't fix grav drones from NCCs or combat carrier guardians; I don't know if either of those cause lag in extreme cases.  If those are a problem, it's back to optimizing rollups/throttling calculations.
« Last Edit: May 17, 2016, 04:29:11 pm by tadrinth »