So here's my dual dilemma for alpha (with Skyward, or really any game).

[x4000]

We keep pushing back our private alpha for a day or two at a time, and it's because of mainly two reasons:

Feature-Completeness
This really is two sets of issues:

1. "Well, if feature X isn't there, then the experience is thrown off what the final game would be, and thus how valid is the testing?"

There's a lot of validity to this, because if players get started and learn the game without X feature, then they have to re-learn it to some extent when the game is played again.  "Okay, now Chess has bishops" really makes a huge difference.  With something like AI War it's different because there are so many features that are optional and that only show up circumstantially anyway.  Even so, there are a lot of core mechanics that, without them, would really change things.  What if energy wasn't fully in there yet?  Etc.

2. "Well, if feature X isn't there, are they going to complain or have odd expectations when later it is added?"

Our experience in the past has been that if something has "always been there," it's looked at in a different light compared to something that was added or removed after the player encountered it.  In other words, it's taken as part of the game rather than something that potentially impedes the experience they had before.  Often this is a feature that new players to the game love, but that alienates some prior players because they played the game before the feature.  I can't think of specific examples off the top of my head, but with Valley 1 and AI War there were tons of examples of this. 

UPDATE: Okay, the caliber system changes in Valley 2 are a great example of the above.  The new way is something that I think is vastly better in every way, and it leads to a certain style of gameplay.  But some folks are still going on about the way it used to be, which they preferred.   But I propose that if they had never played the old way, they would be just as happy with the new way as everyone else.  You can't miss what you never had.  In the case of the conflict with the old system, it boiled down to this: 1) I felt it was broken to the point of being unsalvageable, and overly complicated anyhow; 2) the players who liked it preferred the complexity and thought that with tuning it could be salvaged.  I've spent a great deal of time actually looking at it and running the numbers, and I don't see how it could logically work, though; so what I wind up with are people who gripe at me for something I don't think was ever tenable in the first place.  It was an idea that almost worked, and they liked where it seemed to be heading, but the reality was it was actually heading for a brick wall.  So disappointment ensues where happiness should have been.


What these two things push us (mainly me) to try to do is basically get the game itself 100% done, just without being fully balanced, before we go to any testing at all.  It creates huge anxiety in me of "no, we shouldn't show this yet because of 1 or 2 above" in various cases.  I'm not sure how to really get around that, or really if it's even possible.  It's possible that we just need to take it slower on starting the alpha and that's that.  Or it's possible that we need the right testers in the early phases, who fully appreciate "okay, don't get attached to the flow of Chess minus bishops, because those are coming and like it or not the game is different with guys that can move like that."

Lack Of Time For Reacting To Feedback
Whenever an alpha starts, this leads to an influx of feedback, which leads to further delays in the beta if they are material things that need to be immediately addressed.  If x y and z need to be made clearer, or bug x has been found, or whatever, that takes time from actually getting this feature-complete.  Having the game already feature-complete by the time alpha starts solves this problem, because then we're geared up for that sort of thing.

THAT said, there are three kinds of problems typically reported:

1. Game-breaker things.  Obviously we need to know about those and fix them asap in general, feature-complete or no.

2. Nice-to-have things.  These can be cataloged and gotten to either later in the alpha or in early beta, depending on time availability.  In other words I can finish getting it feature complete before even looking at these at all (Josh sorts out all the feedback before I see it, during alpha, to protect my productivity).

3. Balance issues.  These are one of the big things we're looking for in this game, but to some extent it's hard to judge balance when all the features aren't in place yet.  So something that seems out of balance might not actually be.  THAT said, this doesn't really impact our ability to pursue feature-completeness, because Josh is our guru on the statistical balance on this game, and so it doesn't impact me aside from occasional guidance when he has a question or feedback from me saying that something seems off or whatever.


Result
So that's kind of my dilemma.  It's why I keep pushing alpha back and back: I keep adding more and more to what "must happen before we go to alpha."  I'm still kind of mulling the implications of this, and what should be done.  A few options occur to me:

1. Go to alpha tonight with some folks, with a lot of provisos of "x and y and z aren't implemented yet, so ignore those and understand that balance will shift, etc."  But see my concerns above about that sort of thing.

2. Go to alpha tomorrow night with some folks, with a lot fewer provisos but still some.

3. Wait until we are practically beta-ready except for balance, which might be Monday night, and then start a faster alpha that lasts just a few days, then go to beta later next week rather than earlier.

It's hard for anyone to judge who hasn't seen the game yet, obviously.  The game is in a state where it is fun, the interface is polished, it's winnable and loseable, and so on.  The functionality for the human and mythological military is at 100%.  The economy is at 100% except for trade, which I hope to finish tonight.  All the land tile functionality is complete.  The embassy functions and trade functions are really the last human-level things I need to finish.

On the other hand, a few edicts are in place but don't do anything yet except internally set the difficulty modifiers for things like crime and bandit spawn rates.  1 out of the 64 god + mythological tokens is fully functional, though the remainder will follow in a day or two because they are quick to implement once designed (and they have been designed for some while now).  The main reason for them not being more implemented yet (and same with the edicts) has been the need to get the underlying structures more in-place first.  And obviously those are an enormous part of the actual flow of the game, so not having those on day one of alpha means that only the regular warfare can be tested fully.  Some of the other meta stuff also isn't in place, like the challenges or the propositions, both of which I think will actually change the feel and flow of the game quite a lot despite being meta aspects.

Conclusion?
Well... we do want feedback and testing on balance at this point, and we're itching to have someone other than just Josh and Blue and I look at this.  That said, we only get to make a first impression once.  And we also really don't need to have comments along the lines of "well this seems shallow because of x and y are missing" that are coming in a few days anyhow.  (Not that the game feels shallow to me, as an aside, even just with the human stuff.  A big requirement of mine was that the game had to feel fun without any of the gods stuff at all, or else the game just wasn't fun at all.  The gods stuff has always been a part of the design, but it doesn't come into play until rounds 2 and 3 of the game anyhow, so basically that was stating that round 1 had to be oodles of fun no matter what.  That means we've been mostly testing -- in effect -- round 1 types of gameplay even when we play rounds 2 and 3 so far).

So that takes a special breed of tester, who really understands what they are getting into and wants to help from that vantage point.  NOT someone who is just excited to get their hands on the final product ASAP, because volunteering now isn't going to get you to the final product any faster than waiting until beta would.  The testers we need for this early phase, if we're to do something before this is absolutely feature-complete in the normal flow of gameplay (aka, prior to Monday) are people who are okay with seeing "how the hotdog is made" and a bit of scaffolding and sawdust here and there.  These people need to be able to look past those aspects and see the actual form or flaws beyond what is just momentary detritus.

Are you such a one?  If so, shoot Josh a PM again.  I'd rather not wait until Monday to start any form of alpha testing at all, to be frank.

[Cinth]

Speaking from how I approach testing, I take each addition at face value.

Is it functional?  If there are issues, how can they be addressed and how much information can I give to the effect?
As things are fixed (or added), does it mesh well with what is there or cause problems with what is there?  Is it going to be problematic until the next fix (or feature) is added.  And of course there is the pure player experience at each stage.

All of that probably comes from 6 years of fixing electronics.  Test, fix, test, put back in operation , except I had a manual and my customers generally just wanted the equipment to work.

[x4000]

That's how I approach it also, and I think that's basically the engineering approach in general.  That's how most engineers seem to think in general, I mean.  So we need a pool of engineers for testing? 

[Oralordos]

Sounds similar to me too. I guess programming since I was 13 helps?

[Aklyon]

As for me (not being an engineer, unless you count engineering small chainmail thingies that looks neat but don't do much besides occasionally confusing people how I made a cube out of circles), it sounded like a new dwarf fortress release before it gets some fixes: Playable, full of new things, neato, and is going to take sometime to understand but may or maynot have unique happenings in the meantime.

[Cinth]

That's how I approach it also, and I think that's basically the engineering approach in general.  That's how most engineers seem to think in general, I mean.  So we need a pool of engineers for testing? 

It's how I learned how to troubleshoot really.  I think a mix of players is best so you get a variety of views, but always with the grain of salt.  I can point out bugs fairly easily, but balance feedback is going to be tougher for me, that feel part.

6 years electronics tech for the AF then I did take a few EE courses in college after I got out.  Humorously enough, the programming section of my EE courses is what made me decide to change my major to math
I could do the programming coursework, I just couldn't do it in a reasonable time frame.

Back on topic however, it sounds like you want eyes on this project.  You just need someone with no expectations and some objectivity.  Someone who takes what they have for what it is at each interval.  The problem is finding the right people for right now.  Not a position I would be envious of.  Like the rest of the forum, we'll wait to see what you do decide to do

[x4000]

I think that the comparisons to Dwarf Fortress are a bit apt, in many ways.  Neither game is remotely a replacement for the other, so I'm not trying to step on Bay 12's toes or anything at all like that.  But both games share a huge amount of freedom in what you do, and both are simulations where you are balancing enough stuff that not every thing shows up at the same time in every game.  DF is orders of magnitude more complex than Skyward in almost every respect, but in terms of how the player goes about learning the game I think it has some similarities.  SimCity has them, too.  I think it's common in simulation games.

Basically, I hand you this... thing.  It's a world that you can build and poke and prod in different ways.  You have goals, modest ones at first and then grander ones, and you learn how all the many parts work as you go.  In some respects, simulation games are a little more toy-like than most other genres of games, I've always felt.  That said, because of the heavy strategic component here, that counterbalances the toy-like nature of a simulation game unless you keep the difficulty really low and tame.

In those regards, it's pretty challenging to know what is balanced and what is not without months of feedback from hundreds of people.  There are obvious things that we've smoothed out, but I'm also trying to structure things such that -- as in Dwarf Fortress -- there's not ever some strategy you can just plunk down.  Aka, making it so that situations vary enough that if Unit A is overpowered 40% of the time, it is just average or underpowered the other 60% of the time.  And thus that's not really balance we have to "fix."  Having players recognize the right tool for the job in a given situation is then a big part of the game.

And really that's what we're already doing, and have been doing with the design from the start, but we're still in the process of ironing all that out.  The "propositions" idea actually occurred to me while re-reading boatmurdered, so that's definitely DF-inspired, though I don't think that steps on any toes either.

[Cinth]

All these DF references are lost on me.  I started it up once and was immediately lost.  I haven't found the occasion to try and play it again.

[x4000]

Try reading boatmurdered.  It starts a little slow, but is absolutely fricking hilarious.  I literally laugh out loud almost constantly through all of StarkRavingMad's part.

In terms of being lost with the game... yes, that describes me also.  I've played about 12 hours of it, and got very little accomplished in it.  It was fun, but I couldn't learn it at a pace that I felt satisfied with.  But boatmurdered is one of my favorite things on the internet ever.

[GigaClon]

Part of the fun of AVWW I and II were how the game changed overtime. It was just as entertaining as the game itself. I'm also an aspiring game design and so all this detail is really nice.

[Aklyon]

Theres three things important relative to Dwarf Fortress:
First, you have the wiki.
Secondly, you have Boatmurdered, the most well known of DF stories, and from the versions before DF had a Z-axis.
Thirdly, you have Nist Akath, which is Boatmurdered on the polar opposite end of the scale: It starts in a frozen terrifiying glacier (Read: Polar temperatures freezing all hope at aboveground water or fishing, no useful aboveground plants, minimal wood, and undead everywhere), and becomes the source of the best (if most feared) dwarven soldiers everywhere. From the version before the current 0.31.*, so the stats are simpler and theres no vampires or werecreatures or night creatures or necromancers.

Theres also the hall of legends on the bay12 forums (which collects stories worth reading, including both of these), and one of the problems a recent df version (the one I'd been thinking of when I wrote unique problems, actually) had pre-fix was dwarves were having their fat melted by rain. Not acid rain, not magma rain, water rain with nothing particularly special about except a typo somewhere that changed the dwarves burn point.

[Cinth]

Try reading boatmurdered.  It starts a little slow, but is absolutely fricking hilarious.  I literally laugh out loud almost constantly through all of StarkRavingMad's part.

In terms of being lost with the game... yes, that describes me also.  I've played about 12 hours of it, and got very little accomplished in it.  It was fun, but I couldn't learn it at a pace that I felt satisfied with.  But boatmurdered is one of my favorite things on the internet ever.

Cool, I'll give it a read.  As far as the game itself goes, I never got off the ground (well, in the ground).  I spent about an hour trying to wrap my head around the UI (what UI) and get things going.  After that, frustration set in.  I feel like DF is going to take a lot more effort than I want to put into a game and that is just to get started.  One of these days, maybe, but right now it isn't what I want in a game.

@Aklyon: Funny you should mention glaciers.. that was how my game started 

[x4000]

I spent literally four hours trying to choose a plot of land to start my fortress in.  Not joking, not even a little bit.  On my birthday.  I don't regret the time spent, though.

@GigaClon: Welcome to the forums!  And yeah, I know what you mean on that; a lot of folks seem to enjoy that meta-game as much as the game itself, and I must say I enjoy that too.

[Aklyon]

I didn't take nearly as long, all I wanted was no aquifer.

What took me ages in df though (if we're gonig to continue the derail) was trying (and being really bad at) building a tower instead of going the more sensible route and digging thy fortress instead.

[Cinth]

I just picked what looked like a good spot (they all looked the same to me). 

About the derail, it's a friendly diversion form the dilemma Chris is facing right now

And until we hear otherwise, I'll keep tabbing back and forth between the forum and my game of Torchlight 2 (mindless fun smashing lots o mobs).