New Tactics on the Alpha/Beta ?

[Teal_Blue]

This is just a thought, but one that came to mind and i thought needed to be said, though it is just my opinion.

But basically i was wondering if the Alpha/Beta group hasn't been a little bit un-related to the larger public audience, meaning... that Arcen gets a title ready, they tweak it and bend it and hammer and saw at it, all according to how the alpha testers think... And then we move into launch and crash.

I think we may have a very specific and biased group, that doesn't intentionally mislead, but never-the-less, the result is the same as they are group of such precise and studied and perhaps even acclimated audience. Perhaps so much that what may best serve Arcen is to throw the water out and get a fresh batch?

Please understand i am not saying we are mean or anything, or that we don't know what we are doing, because there are many very smart and experienced people here. But what i do mean, is we may not, in fact i believe, though i could be wrong, that we 'are' not the best group to represent the broader public audience.

To that end, where i am repeating myself, sorry about that, i think Arcen needs to get a new group in. Perhaps even go out and specifically recruit them, not us forumers, (though we love them!)   But someone else. Maybe, some of the Steam posters, or those that were harsh on Arcen games in the past? Send them a note and say... Hi, we are requesting your assistance in testing our new game, would you please like to tell us what you think?' sort of thing.   

I'm not trying to put us as forumers down, but i do think we are more than a little biased and that probably can be deadly, when the public thinks one way and we are blindly denying or naysaying it because we don't want Arcen's feelings hurt. All hurt feelings aside, it doesn't do Arcen any good if we give them the wrong impression about something they think is going to be fine, or even gangbusters, when the public feels something entirely different.

Anyway, that is my feelings, i hope i haven't offended or hurt anyone's feelings, but to be honest, i didn't want us to love Arcen and cause them problems because we either subconsciously or purposely try to pull our punches when it comes to saying not only what a game should be, but how it plays and how it interacts with the player. Our group might be 'too' specialized and not 'broad' enough.

Thanks for listening,
Sincerely,
-Teal

[x4000]

I think you'll find that the feedback is numerous and nitpicky (in the best sense).  The pre-1.0 list of issues for Bionic Dues reported in mantis was immense, and 1.0 was incredibly polished and has been received very well by press and players outside the core forums.  We've had to make very few alterations to it since that time, and even the most major of them (the customization screen) was more in nice-to-have territory than it-stops-me-having-fun territory (for most people).

Our problem with Bionic wasn't that people tried it and didn't like it, our problem is that people didn't hear about it at all.  I think that the Bionic Dues project actually shows just how effective our forum members are, as opposed to the opposite.

Put another way: most companies do not do a general release of their games prior to 1.0.  Well, until recently.  And a lot of the public alphas lately are getting kind of blasted, and have various other problems.  I feel like our current process is working as well as anyone else's or better, because not only do we have a corps of volunteers, they are all legitimately interested in the game (unlike people in a QA department) and generally not shy about their opinions.  The fact that they say it in a constructive rather than an abrasive way doesn't lessen that.

[Teal_Blue]

  I am glad to hear that. I was worried we might be leading you down the garden path, so to speak and thought that someone else, a different group could do a better job for you. But you certainly are in a place to know better, i am just guessing from a rather distant outsider sort of point of view.

But i was worried, so that is why i mentioned it. Thank you for the correction, it is good to know that we aren't too bad for you. Thank you for listening, sincerely,
-Teal

[x4000]

Not a problem, and it's always good to evaluate and re-evaluate everything!

[Arnos]

You can always do a mix of both, keep it inside the forums but allow people in over a few stages so they have a fresh mindset instead of in one big rush.  I know that the folks who made N put up a blog post about how they wished they had done this to a greater extent.

[x4000]

Incidentally, that's exactly what we do.

[chemical_art]

I'm actually going to echo OP, even despite the responses given. While the forum may be good at getting at iron at bugs within a game's structure, it provides little guidance about if a game is good to begin with with regard to a larger audience I.E. Shattered Heaven. And if the testing is fine but the game simply isn't known, that is a failure on the advertising, period (regardless of how much time said advertisers put into it: similar to how if I spent 3 months full time performing a service and the service wasn't delivered, that is my fault)

In other words, something is wrong when only one game out of the last...5 or 6? Has been successful enough to get a sequel.

[x4000]

I wouldn't be quite so doom and gloom.  As a company, our profits have gone up every year except last year, which took a dip for a few reasonable reasons.

In terms of shattered haven, we had an extraordinarily small alpha/beta there, which should have been a sign.  That wasn't the case with Skyward, and definitely not with Bionic.

In terms of sequels, I really try to avoid those as a general rule.  So the presence or lack thereof of them doesn't really mean much.  AI War is by far our most profitable title, without a sequel or any contemplation of it.  Valley made loads of money, but we spent too much money making it as well as miscalculated in giving away the sequel for free.  Skyward did well.  Bionic is doing okay but not stellar, but the market is getting harder at this point.

I would argue that the forumites were actually really good with Skyward about telling us how the game was not really all that fun at first, and then there was a real transformation during the alpha of that one.  It was really night and day.

It's a really complicated situation, basically, and we look really hard at the data quite often.  There's been a lot of internal debate about whether or not to do a public beta with TLF for instance, but there are some serious pros and cons to both.  Getting feedback from random people disinclined to like the game isn't really something that interests me; our audience is inherently niche, and trying to make something that is everything to everyone is one reason why the Valley games had some identity issues, to put it mildly.  Letting people self-select and then tell us what they think could be better seems to work quite well.

As for a marketing failure, did I ever really say otherwise?  It's not really a science, and there are a lot of variables there.  We try to do the best possible each time, but the market is ever changing, every game is different, and we're always still learning.

I'm not saying that there's no room for improvement with anything, but I am saying that we've had many hours-long skype calls about these sorts of subjects in the past month, and we continue to do so periodically.  Having too little feedback about game quality really hasn't made the list of top issues remotely lately.

[MouldyK]

I sometimes feel like rooting for Arcen to do well is like rooting for the Wii U to do well.


Both deliver great games, but the general public don't care. But in both cases, one game which latches onto the public is all which is needed.


Hopefully this one is it.

[x4000]

Heh.  I think if you're rooting for Nintendo to do well in the sense that it becomes Sony or Microsoft in its market position, then sure that's pretty hopeless.  Honestly I just hope Nintendo keeps being comfortable and making great games for the market that they do have.  Their giant success with the Wii didn't really benefit their core niche, so for that niche it was kind of irrelevant.

For us we don't have to become the next Terraria or whatever, although something like that would be nice.  What we're looking for is comfort and stability to keep doing what we're doing, and that's about it.

[Tridus]

The problem with the Wii U is that it's self-inflicted. Nintendo has one of the best software lineups of any company on the planet. They could put it on any hardware they want and make zillions of dollars.

Instead they go around making hardware that will be obsolete in two weeks, give it a gimmick most people don't get and a totally bungled marketing campaign, overcharging for it, and shackling their software to the Nintendo Hardware Tax.

That's why they stopped getting money form me. I'm sick of buying hardware that gets used for Mario and Zelda and then collects dust for years. If they want my money for their games, they can put them on hardware that I actually want.

[x4000]

Fair enough.

[Tridus]

As for the alpha group... I don't know. Bionic Dues is a good game. The alpha group didn't try to change it from the kind of game it was when we got it, but we did give a lot of feedback on bugs, polish, and "this gun can blow up the entire map so maybe that's a balance issue" problems.

I don't see how a wider testing group would have changed that, since if you believe that the game didn't catch on because of the game itself, those decisions were made before the testers got their hands on it. If you believe the game is good and just didn't get noticed in a crowded market, that's not the testers fault either (in some ways that's not even PR's fault, I saw coverage on some good sites).

In this case I just can't point to anything the testers did that made the game more "niche" than it started off as.

[keith.lamothe]

I think Bionic's alpha went absolutely great and fulfilled all its goals, with the one exception that I didn't really understand just how hard it was to slog through the customization stuff until post-release.  Even there, one of the testers was pretty blunt about the problem on mantis and I basically just figured "well, it's just one guy so it's probably not something I need to spend a lot of time fixing, and it would take a lot of time".  And then of course he was proven very right   So there the problem wasn't so much with the testing but that I didn't take the feedback seriously enough.  If I'd escalated it to a forum discussion to see if there was more than one person who felt that way, that might have become more evident.

Even there, we didn't really get dinged for the customization in the reviews I saw (some mentioned the tedium, but didn't list it in a summary of cons or whatever).

Nah, the operative problems with Bionic's sales don't really include the game or testing phase.

Personally, I'm in favor of like a 2-week public beta after the alpha, when we would normally just release it.  But there are significant tradeoffs to that approach, and I don't really have a strong argument that it's worth it.

[JAlfredGoodwin]

A public beta is fine, just make sure you advertise  a release date.

Seems like lots of "Betas\Alphas" seem to be releasing on a "when its done" release schedule, and that makes people squeemish