Author Topic: 0.201 "Bugfixes and Optimizations" released!  (Read 8293 times)

Offline BadgerBadger

  • Arcen Volunteer
  • Hero Member Mark III
  • *****
  • Posts: 1,229
  • BadgerBadgerBadgerBadger
Re: 0.201 "Bugfixes and Optimizations" released!
« Reply #15 on: May 26, 2017, 09:47:29 am »
So I'm guessing we'll be looking at .202 coming out sometime next week?

Offline x4000

  • Chris McElligott Park, Arcen Founder and Lead Dev
  • Arcen Staff
  • Zenith Council Member Mark III
  • *****
  • Posts: 31,651
Re: 0.201 "Bugfixes and Optimizations" released!
« Reply #16 on: May 26, 2017, 11:28:14 am »
Nah, I expect a version today.  No promises for obvious reasons (recent events), but it should be today.

One of the things that I've been spending some of my time on other than performance in huge battles is the space backgrounds and the ability for players to make their own.  This in turn is leading to an overhaul of the entire post-processing pipeline, and I was doing some stupid things with the camera field of view that previously made the spaceships feel way smaller than they really are, etc.
Have ideas or bug reports for one of our games?  Mantis for Suggestions and Bug Reports. Thanks for helping to make our games better!

Offline BadgerBadger

  • Arcen Volunteer
  • Hero Member Mark III
  • *****
  • Posts: 1,229
  • BadgerBadgerBadgerBadger
Re: 0.201 "Bugfixes and Optimizations" released!
« Reply #17 on: May 26, 2017, 11:32:34 am »
It's always nice when you start doing some work and that leads you to be able to find and fix previously unnoticed bugs!

Offline x4000

  • Chris McElligott Park, Arcen Founder and Lead Dev
  • Arcen Staff
  • Zenith Council Member Mark III
  • *****
  • Posts: 31,651
Re: 0.201 "Bugfixes and Optimizations" released!
« Reply #18 on: May 26, 2017, 01:15:40 pm »
Indeed!

In this case it wasn't a bug per se, it was just poor cinematography, really.  When you use camera lenses different ways, it makes things look larger or smaller or more distant or closer, etc.  I was using something that made the ships look incredibly toylike in order to avoid distortion on the planet, which is supposed to look different.

The problem at root is that it needs to be a composite shot (which it already was), where the planets and the starfield look waaay further off, but the ships look a lot closer and larger, with more perspective even to the fighters and such.

We already are using three composited cameras, so it's no extra load to fix this, thankfully. :)
Have ideas or bug reports for one of our games?  Mantis for Suggestions and Bug Reports. Thanks for helping to make our games better!

Offline tadrinth

  • Hero Member
  • *****
  • Posts: 507
Re: 0.201 "Bugfixes and Optimizations" released!
« Reply #19 on: May 26, 2017, 04:33:47 pm »
If you wanna nerd out about the details of the SwizzleList, I always like hearing about interesting data structures. 

Offline x4000

  • Chris McElligott Park, Arcen Founder and Lead Dev
  • Arcen Staff
  • Zenith Council Member Mark III
  • *****
  • Posts: 31,651
Re: 0.201 "Bugfixes and Optimizations" released!
« Reply #20 on: May 26, 2017, 05:36:59 pm »
So I didn't quite get there in terms of having this release ready.  But it's coming along quite well.  A few performance things linger, but I can get between 40-60 fps during fights where I was previously getting a choppy 5fps.  So it has come a really long way.

Unfortunately my trick with the camera FOV to make all the ships look larger doesn't play well with all the various circles and spheres the game uses for things like forcefields and selection circles, etc.  They get super distorted by a larger field of view, that sort of fish-eye effect like you get off a Go Pro camera.

The idea is that with a larger field of view, things look larger (which is true), and that works well for something like a racing game where you're down low, or even for a FPS game where you're looking out of the eyes of the character and the stuff at screen edges is your "peripheral vision."  It's one of the reasons why console games often feel so wrong on PC ports, because we sit a different distances from those screens.

At any rate, for a strategy game it just wasn't working out.  We're actually using a 40 degree field of view, rather than the default 60, but then with foreshortening on the camera zoom to fake a wider field of view in some ways.  This eliminates all edge distortion, which you could see even at 60 (we did this months ago), while letting you still "see as much stuff" as if the fov was wider.

But unfortunately it doesn't contribute to things feeling "large," or at least not "close and large," because the lines of convergence on the invisible horizon are not very sharply inward with that sort of fov.  Such is life.  It looks the same now as it did last version and the last few months.

I've had a breakthrough, finally, on the background starfields, though.  I was unhappy with some of the existing ones we had, and it's annoying that a skybox takes 6 setpass calls.  And that I couldn't really customize them much, even with my own hsv-enabled shader for them.

I wound up creating my own new shader that blends a number of textures together using a variety of methods, to come up with something that can be a lot prettier.  It takes a lot of fiddling to really get a good result, but then you can save that and use that from then on.  The resources are low in terms of what it saves in the game directory, and in ram, and it has a huge amount of flexibility.  It also only uses one setpass call.

It can't be randomized because most random values would look like utter nonsense (not just like bad space graphics, but just a riot of nonsensical stuff).  It takes a dedicated hand to tune a bunch of things with this and get something worth saving, and it's probably a 20ish minute process at the moment, minimum.  But it's worth it!  And other people will be able to add to that, not just me, which is exciting for sure.  Yay variety. :)

I'm in the process of switching all the graphics to use a new reflection map that is much more studio-quality for photography, and to using an HDR camera.  I needed some idea of how the space backgrounds were going to look in this new system so that ideally they would be dark enough not to get picked up by the bloom levels, which can now be set to only work on light levels above 1, or something along those lines.  I'm going to have to redo the entire post-processing visual stack in general, but the groundwork has all been laid for that.

If you wanna nerd out about the details of the SwizzleList, I always like hearing about interesting data structures.

It's stupidly simple, really.  I went looking into all sorts of data structures, things like AList and DList and B trees and Hash sets and even things like linked lists, but everything just had too many downsides. We had been using just a generic List<>, which worked well for iterating (which we do a ton of) and adding at the end (which we do a ton of), but not for removing from arbitrary positions (which we also do a ton of).  So we'd have spikes of huge amounts of ms every time it had to remove something from low in the list, because then all the other items had to move down.

I got really frustrated with the other data structures, because they're all so complicated and only sort of improve things.  And many of them waste GC resources or reduce the quality of the iteration time below O(n).  I needed speed on those three things, and just nothing around was really matching up with what I wanted. 

Then I started thinking about how I've used swapping lists of things in the past, when doing pathfinding and other logic, to save on GC.

Then I realized that the arrays themselves are incredibly tiny (in terms of data size for the array and their pointers themselves), and I only need like ten at most anyway, and I can predict really safe WAY upper bounds on them.

I also realized that since I'm pooling the data that goes in these, I don't have any need for things to be released for the GC.  If a reference gets held for a long time, it affects nothing.

So... I decided to make SwizzleList, which basically has two raw arrays in it, which you initialize to some super large size.  For shots, I chose half a million right now.

One list is active, the other is inactive.  When you add stuff, it goes in the active list at whatever the next slot is, then it increments a length integer.  That's "how much stuff is in the list," as far as the outside world is concerned.  You can't remove things from the list, but you can iterate over it in equal time to what you normally would.

At some point you hit a period where things have been flagged for removal outside of this list, and it's time to make a new, condensed list.  So basically what it does is it clears the inactive list, loops over all the items in the active list, runs some logic for each one (of my choosing), and then as it does, any that are "not flagged for removal" get put into the inactive list.  At the end of that one iteration, I swap which list is active and which list is not.  And boom, things are "deleted."  I had to do that loop for deletion checks anyway, so the only added cost is copying to the inactive list, which is super cheap.

I feel kinda embarrassed about it because this is not a sophisticated data structure at all, and in many ways it's just a "duh" sort of thing.  But, well, I haven't seen anyone else do this for this sort of purpose, and it's wicked fast.  If you needed the GC to work, needed to sort, or needed to realtime remove things during list processing, this would not work well.  Or if you needed lots of these, of unknown size.  But for a pooled set of things with a known upper bound that you can exceed by an order of magnitude without a real ram hit... works very well.

I attached the code for the list itself, and then this is the code that uses it:

Code: [Select]
#region HandleShotUpdates
    public void HandleShotUpdates()
    {
        float deltaTime = Engine_AIW2.DeltaTime;
        GameSettings_AIW2 settings = GameSettings_AIW2.Current;
        bool drawAllShotRadii = settings.Debug_DrawAllRadiiAtAllTimes;
        bool drawShotDebugging = settings.Debug_DrawShotDebuggingData;

        Engine_Universal.BeginProfilerSample( "ShotRemovalChecks" );
        int shotLength;
        ArcenVisualShot[] shots = ActiveShots.GetActiveList( out shotLength, true );

        #region Removal Checks
        {
            int maxShotsToHandle = Mathf.CeilToInt( settings.Performance_ShotRemovalChecksToAttemptPerSecond * deltaTime );
            int min = settings.Performance_ShotRemovalChecksMinPerFrame;
            if ( maxShotsToHandle < min )
                maxShotsToHandle = min;
            int shotsHandled = 0;
            try
            {
                ArcenVisualShot shot;
                for ( int i = 0; i < shotLength; i++ )
                {
                    shot = shots[i];
                    if ( shotsHandled < maxShotsToHandle && shot.RemovalChecks_LastAnimationCycle != this.CurrentShotRemovalChecksPass )
                    {
                        shot.RemovalChecks_LastAnimationCycle = this.CurrentShotRemovalChecksPass;
                        shotsHandled++;
                        shot.DoRemovalChecks( currentSimFrameVisualOnly );
                    }
                    if ( shot.IsConsideredActive )
                        ActiveShots.AddToInactiveListNoChecks( shot );
                }

                ActiveShots.SwitchActiveList( true );

                if ( shotsHandled < maxShotsToHandle )
                {
                    this.CurrentShotRemovalChecksPass++;
                    if ( this.CurrentShotRemovalChecksPass > 10000 )
                        this.CurrentShotRemovalChecksPass = 1;
                }

                if ( Engine_Universal.IsProfilerEnabled )
                    Engine_Universal.LodgeMessageInProfilerData( "Total: " + shotLength + ", Handled: " + shotsHandled );
            }
            finally
            {
                Engine_Universal.EndProfilerSample( "ShotRemovalChecks" );
            }
        }
        #endregion

        shots = ActiveShots.GetActiveList( out shotLength, false );

        Engine_Universal.BeginProfilerSample( "ShotMainUpdates" );
        if ( Engine_Universal.IsProfilerEnabled )
        {
            Engine_Universal.LodgeMessageInProfilerData( "Count: " + shotLength );
        }
        try
        {
            ArcenVisualShot shot;
            for ( int i = 0; i < shotLength; i++ )
            {
                shot = shots[i];
                shot.DoMainUpdate( this, drawShotDebugging, drawAllShotRadii );
            }
        }
        finally
        {
            Engine_Universal.EndProfilerSample( "ShotMainUpdates" );
        }

        {
            Engine_Universal.BeginProfilerSample( "ShotMovement" );
            int maxShotsToHandle = Mathf.CeilToInt( settings.Performance_ShotMovementsToAttemptPerSecond * deltaTime );
            int min = settings.Performance_ShotMovementsMinPerFrame;
            if ( maxShotsToHandle < min )
                maxShotsToHandle = min;
            int shotsHandled = 0;

            try
            {
                ArcenVisualShot shot;
                for ( int i = 0; i < shotLength; i++ )
                {
                    shot = shots[i];
                    if ( !shot.IsConsideredActive || !shot.DidMainUpdate )
                        continue;
                    shot.ShotMovement_AccumulatedDeltaTime += deltaTime;
                    if ( shotsHandled < maxShotsToHandle && shot.ShotMovement_LastAnimationCycle != this.CurrentShotMovementPass )
                    {
                        shot.ShotMovement_LastAnimationCycle = this.CurrentShotRemovalChecksPass;
                        shotsHandled++;
                        shot.DoShotMovement( drawShotDebugging );
                        shot.ShotMovement_AccumulatedDeltaTime = 0f;
                    }
                }

                if ( shotsHandled < maxShotsToHandle )
                {
                    this.CurrentShotMovementPass++;
                    if ( this.CurrentShotMovementPass > 10000 )
                        this.CurrentShotMovementPass = 1;
                }

                if ( Engine_Universal.IsProfilerEnabled )
                    Engine_Universal.LodgeMessageInProfilerData( "Total: " + shotLength + ", Handled: " + shotsHandled );
            }
            finally
            {
                Engine_Universal.EndProfilerSample( "ShotMovement" );
            }
        }
    }
    #endregion
Have ideas or bug reports for one of our games?  Mantis for Suggestions and Bug Reports. Thanks for helping to make our games better!

Offline x4000

  • Chris McElligott Park, Arcen Founder and Lead Dev
  • Arcen Staff
  • Zenith Council Member Mark III
  • *****
  • Posts: 31,651
Re: 0.201 "Bugfixes and Optimizations" released!
« Reply #21 on: May 26, 2017, 05:42:18 pm »
Bear in mind that both of these screenshots are with hdr on, but no postprocessing at all yet, and with the ship shaders all not yet accounting for the new reflection cubemap.  It's also relatively subtle of a background, but I can do more dramatic ones, too.  In general the idea is going with stuff that's not so darn bright, though, and this was my first go at that without it also looking tacky.
Have ideas or bug reports for one of our games?  Mantis for Suggestions and Bug Reports. Thanks for helping to make our games better!

Offline BadgerBadger

  • Arcen Volunteer
  • Hero Member Mark III
  • *****
  • Posts: 1,229
  • BadgerBadgerBadgerBadger
Re: 0.201 "Bugfixes and Optimizations" released!
« Reply #22 on: May 26, 2017, 06:26:15 pm »
I'm intrigued by the comment about new buttons and fonts! I look forward to the game getting a bit of polish.

Offline Cyborg

  • Master Member Mark III
  • *****
  • Posts: 1,957
Re: 0.201 "Bugfixes and Optimizations" released!
« Reply #23 on: May 27, 2017, 12:00:55 am »
About the camera angle part of the conversation, is there a way to get an Adm. Akbar camera angle easily? By selecting a fleet or something?
Kahuna strategy guide:
http://www.arcengames.com/forums/index.php/topic,13369.0.html

Suggestions, bugs? Don't be lazy, give back:
http://www.arcengames.com/mantisbt/

Planetcracker. Believe it.

The stigma of hunger. http://wayw.re/Vi12BK

Offline BadgerBadger

  • Arcen Volunteer
  • Hero Member Mark III
  • *****
  • Posts: 1,229
  • BadgerBadgerBadgerBadger
Re: 0.201 "Bugfixes and Optimizations" released!
« Reply #24 on: May 27, 2017, 11:02:46 am »
If you zoom in all the way it's not too dissimilar from an Ackbar-style view.

Offline x4000

  • Chris McElligott Park, Arcen Founder and Lead Dev
  • Arcen Staff
  • Zenith Council Member Mark III
  • *****
  • Posts: 31,651
Re: 0.201 "Bugfixes and Optimizations" released!
« Reply #25 on: May 29, 2017, 10:12:57 am »
About the camera angle part of the conversation, is there a way to get an Adm. Akbar camera angle easily? By selecting a fleet or something?

We'll have a free-cam at some point prior to launch as well as the current camera style.
Have ideas or bug reports for one of our games?  Mantis for Suggestions and Bug Reports. Thanks for helping to make our games better!

Offline drspikes

  • Newbie
  • *
  • Posts: 3
Re: 0.201 "Bugfixes and Optimizations" released!
« Reply #26 on: May 29, 2017, 11:22:52 am »
Hi Chris, will the early access backers still get their keys today? Looking forward to trying this, knowing there is some way to go.  Many thanks!

Offline x4000

  • Chris McElligott Park, Arcen Founder and Lead Dev
  • Arcen Staff
  • Zenith Council Member Mark III
  • *****
  • Posts: 31,651
Re: 0.201 "Bugfixes and Optimizations" released!
« Reply #27 on: May 29, 2017, 12:33:17 pm »
Hi Chris, will the early access backers still get their keys today? Looking forward to trying this, knowing there is some way to go.  Many thanks!

You bet! :D

I've posted some more details over here: https://forums.arcengames.com/ai-war-ii/ai-war-2-early-access-good-news-and-bad-news/msg215142/#msg215142

But mainly the most important thing is "today at 4pm EST or thereabouts."
Have ideas or bug reports for one of our games?  Mantis for Suggestions and Bug Reports. Thanks for helping to make our games better!

Offline BadgerBadger

  • Arcen Volunteer
  • Hero Member Mark III
  • *****
  • Posts: 1,229
  • BadgerBadgerBadgerBadger
Re: 0.201 "Bugfixes and Optimizations" released!
« Reply #28 on: May 29, 2017, 12:55:20 pm »
And the rollout will come with .202 being released as well, he asked hopefully?

Offline x4000

  • Chris McElligott Park, Arcen Founder and Lead Dev
  • Arcen Staff
  • Zenith Council Member Mark III
  • *****
  • Posts: 31,651
Re: 0.201 "Bugfixes and Optimizations" released!
« Reply #29 on: May 29, 2017, 01:03:04 pm »
And the rollout will come with .202 being released as well, he asked hopefully?

Called 0.300 at the time, but yes.  There's a lot of almost-finished stuff that's not in the release notes for that yet that make it worth the big version uptick. :)
Have ideas or bug reports for one of our games?  Mantis for Suggestions and Bug Reports. Thanks for helping to make our games better!