Author Topic: A guide for helpful alpha/beta testing -- all testers please read this!  (Read 900 times)

Offline x4000

  • Chris Park, Arcen Games Founder and Lead Designer
  • Arcen Staff
  • Zenith Council Member Mark III
  • *****
  • Posts: 31,403
Hey all,

A few things that I want to say.  Not to call anyone out, but to hopefully make a more productive and efficient relationship between us-the-developers and you-the-testers.  Overall this process has been going great, but there are some things to comment on to make it even better.

1. Please no ultimatums.  Things along the lines of "well, with X changes, this is probably the last I will test" is really unproductive.  I know that I can't please everyone, and it may be that some testers ultimately do not wind up liking the game because it just isn't their cup of tea.  But what IS helpful is if you tell me, in as much detail as you think is appropriate, what is causing such strong feelings in you.

2. Saying something like "I liked it the old way better" is also not productive, as we are rarely ever going to just go back to the old way something worked, unless we really were just trying a wacky experiment.  And we're past the point of doing that.  That said, you should know: we take the approach of "when we try something new, try an extreme version of it."  Somebody on the forums said it was was Sid Meir's approach to balance to "either double it or cut it in half."  That's more or less my approach, too.  If we make slight changes, you won't notice.  If we make an extreme change, you WILL notice, and we can then find the appropriate ratio in between the two values.  This is part of why #1 really isn't productive.  If you're reacting to an extreme change and going "well now I hate this," then you're prematurely reacting to part 1 of an ongoing process of tweaking.  That's why I want your feelings written out, about what bugs you and what bits you do like, rather than simple statements of preference.  It's not whether you like black or white, it's what shade of gray you like the best. ;)

3. If you are finding yourself lost and then just kind of ragequitting in frustration, that's okay!  I want to hear from you anyway!  This isn't the time to be silent, my goodness.  Instead I want to know exactly what tripped you up, and at what point you stopped in frustration.  This is KEY information for us.  Please don't just go "well I guess this game is too complex for me" or "I guess I don't like it after all."  This game (as of this writing) has almost no tutorials, and we rely on feedback from people who get confused or upset to help us find the trouble spots where more explanation is needed.

4. As a corollary to #3, if you find yourself confused, I don't want to actually answer your question directly.  I just want to know what your questions are.  I ALSO don't want you to "just keep plugging away" in a way that you would not if you were just playing a regular old game.  Neither of those things is realistic as far as a real non-tester player experience would go.  Instead, I want to be able to adjust the game based on your feedback so that IT tells you what you need to know, and have you give it another go after that point.  And then if there are still frustrations, then repeat, etc.  If your understanding of the game has to result from me telling you about it, then someone who just buys the game on steam will be lost, since they aren't likely to come talk to me.

5. Please don't just be a silent tester.  Even if your only feedback is "I don't have anything to say because this is great fun and seems fine to me," that is useful for me to know.  Sometimes drowning in a sea of (helpful!) criticism, it is really useful to me to hear those sorts of comments rather than just wondering if you are silently hating it even more. ;)  There have been a number of players who have commented basically to this effect of "this is great, don't have much else to say!" and that is not a wasted post -- believe me.

6. If you can avoid PMing me, that is preferred.  The forums or mantis are the best place for reports of what is going on, because then others can also join in the discussion.  I love talking to you guys, but my time to type out lengthy individual replies is... extraordinarily limited at this point.  I think most of you know what kind of deadline pressure we are under, and if we pull this off properly it's going to be a minor miracle.  I think that this sort of miracle can happen, though. ;)

Thanks for reading!
Have ideas or bug reports for one of our games?  Mantis for Suggestions and Bug Reports. Thanks for helping to make our games better!

Offline orzelek

  • Hero Member Mark III
  • *****
  • Posts: 1,096
Ad 5.

I admit I'm not playing as much as I thought I would - other things (and/or games) took my time.
And from what I read most of the feedback I would need to duplicate.
I'm also impressed by the way in which combat evolves :)

Star map.. is one place I don't find the hook in - last changes to scrolling tickers (spam was there all around) might help a bit with knowledge there.
Currently it's hard for me to feel that I made some difference on star map - dispatches seem like a lot of free time for AI's for small benefit for one AI. It's quite difficult for me to judge what is actually helpful for planet - I can see the various indicators of it's state but no idea how to help with ones that are in negative for example (like quite large negative order factor on planet).

Offline x4000

  • Chris Park, Arcen Games Founder and Lead Designer
  • Arcen Staff
  • Zenith Council Member Mark III
  • *****
  • Posts: 31,403
Got it -- yeah, the tutorializing on the solar map is in particularly bad shape at the moment.  It's been a matter of just trying to run around getting everything we can done!
Have ideas or bug reports for one of our games?  Mantis for Suggestions and Bug Reports. Thanks for helping to make our games better!