Ashen Foundry

Alpha Testing and What to Expect

Hello everyone! I wanted to share some insights and information on Alpha testing. As you all know, there will be various alphas and betas that different kickstarter and crowdfunding packages offer access to. This means there's a very high possibility you will find yourself in an Alpha or Beta test for Ashes of Creation. If you've alpha tested before, but especially if you haven't, I think the following information below will really help outline the realities and expectations.


If you are considering, preparing, or participating in a pre-alpha or alpha test for a game, please review this information. This information will better prepare you for the alpha testing experience, expectations, and methods to increase your effectiveness which all eventually result in a robust, engaging, and successful title. Please understand this information was collectively obtained through my personal experiences with alpha testing environments, experienced alpha tester feedback, and developer feedback & comments.  


First, let’s define and describe “Pre-alpha”, “alpha”, and “beta” testing. These stages of development make up the first three stages of the software release life cycle

Pre-AlphaThis stage is the first stage of the release cycle. Pre-Alpha is usually defined by the following characteristics:

  • Not usually available to the public
  • Frequent build updates
  • Content is largely subject to change before more final builds
  • Buggy
  • Vulnerable to weaknesses and exploitation
  • Not feature complete; missing major systems

AlphaSecond stage of the release cycle. At the end of this stage, the project is closer to being complete and will result in a mostly stable state (again, by the end of the process). The objective of this stage is to continue to develop and improve the quality of the game (systems, performance, stability, etc.)

  • Major focus on bugs
  • Some missing features
  • Lengthy process
  • Targets specific systems with the general consumer’s experience in mind
  • Buggy
  • Frequent patching
  • Frequent downtime

BetaOccurs after the Alpha stage. At this stage, the project is usually considered “feature complete” and the focus turns instead to identifying any lingering bugs or ineffective engineering. 

  • Focuses on improving user experience by reducing negative impacts
  • Used frequently in demonstration of the title
  • Integrates tester input on what at this stage considered the final product
  • Only critical changes are made to the product

Still with me? Awesome- In this next section we’ll discuss how you can apply that information to whatever Pre-Alpha project or title you’re getting involved with.


DODON'T
Gather background information on the systems, developers, and current test build objectivesJump into the test with no background information
Approach the test with an objective and critical eye- at this stage it’s your role to find bugs, broken tech, and provide feedback on sub-optimal systems.Ignore possible bugs, issues, imbalance, or negative aspects of the build. You are not betraying the developers by providing quality feedback.
Realize, at this stage, you are testing, not playing. This is NOT a complete game.Expect or demand the all the “promised” content or smooth testing experience.
Make a plan to communicate your feedback/observations to the developers via forums, emails, etc. and use/conform to any reporting formats provided by developers.Keep your feedback to yourself- contribute so the issues can be addressed!
Hold the developers accountable through mature communication of feedback and exploratory questioning.Harass, flame, or otherwise disrespect the developers.
Expect and be prepared for frustrating, possibly “game-breaking”, bugs.React explosively to an unintended bug which negatively impacts your experience.
Identify and communicate the existence of bugs and methods to exploit these bugs. Attempt and document steps taken to recreate the bug; reboot the game as an additional method to replicate the bug. (Include probabilities: recreated it with success 25% of the time, for example).Identify a bug and actively exploit it to gain an advantage or reduce the quality of testing for others.
Understand that systems, content, and technology may likely change between this stage and beta/release.Expect the game upon release to mimic the current test build.
Encourage, educate, and assist others participating in the Pre-Alpha process.Bully, harass, or demean other testers based on their feedback.
Take frequent breaks- both from specific tests and testing in generalTest exhaustively to the point of frustration and aggravation. This will result in burnout!

I hope this helps prepare you for the expectations and realities of pre-alpha and alpha testing. Remember, you will see a more polished product after this process is over- you just have to get through it first!

Other resources: 

Alpha vs. Beta Testing

Thank you to the developers and games testers who provided feedback and insight.

About Nivhawk:

Longtime MMORPG player, guild leader, and PvPer. Currently, the community leader of Resplendence Gaming Community and guild leader of Killjoy, a competitive PvP guild which will be represented in both Crowfall and Ashes of Creation. Not playing anything seriously at the moment, but dabbles in various steam games and released MMORPGS. 

12 Replies
  1. #1

    Isarii

    Moderator73 Posts

    Fantastic post @Nivhawk! As someone who ends up in his fair share of tests, I have to agree with just about everything here.

    Beyond properly setting one's expectations, I think your last "Do / Do Not" bears special emphasis. I think the great majority of potential testers should be planning on infrequent testing sessions and check-ins, and not be expecting to hop into their first alpha and play it full time like you would a fully released MMO (if the test is even persistent, which is super unlikely). That's the road to frustration, disappointment, and pre-launch burnout.

  2. #2

    Nivhawk

    Member12 Posts
    Isarii wrote on July 7, 2017, 12:17 p.m.

    Fantastic post @Nivhawk! As someone who ends up in his fair share of tests, I have to agree with just about everything here.

    Beyond properly setting one's expectations, I think your last "Do / Do Not" bears special emphasis. I think the great majority of potential testers should be planning on infrequent testing sessions and check-ins, and not be expecting to hop into their first alpha and play it full time like you would a fully released MMO (if the test is even persistent, which is super unlikely). That's the road to frustration, disappointment, and pre-launch burnout.

     Thanks Isarii! 

    One of the most disappointing things is seeing an active, thriving community take a hit because of mis-matched expectations. I'm hoping, with the experience of some veteran testers and an outline of Intrepid's development milestones we can avoid the "norm" for the upcoming tests for Ashes! And agreed, that bottom DO/DON'T is probably one of the most underrated but most important ones.

  3. #3

    CasNation

    Member15 Posts

    Great post!

    These days with the increasing frequency of the "open-beta" with many AAA games, I think the general populace has a skewed idea of what testing actually means, and instead treat it as early access. Of course, many developers exacerbate this problem by offering early beta access in pre-order packages, etc., and are really using the beta as a hype generator. It's a bit of a bad cycle at the moment between developers and consumers of big-name games. 

    With a game on the scope of Ashes, this won't be the case, so having the correct expectations going into testing is really key. Glad to see someone explicitly outline the kind of mindset needed to effectively test.

  4. #4

    Nivhawk

    Member12 Posts
    CasNation wrote on July 7, 2017, 1:29 p.m.

    Great post!

    These days with the increasing frequency of the "open-beta" with many AAA games, I think the general populace has a skewed idea of what testing actually means, and instead treat it as early access. Of course, many developers exacerbate this problem by offering early beta access in pre-order packages, etc., and are really using the beta as a hype generator. It's a bit of a bad cycle at the moment between developers and consumers of big-name games. 

    With a game on the scope of Ashes, this won't be the case, so having the correct expectations going into testing is really key. Glad to see someone explicitly outline the kind of mindset needed to effectively test.

    You are absolutely right about the current tendency of AAA studios and EA/Open Alpha, Beta titles, which are also popular with Indie studios (especially those who rely on crowdfunding) as a means to get valued mechanic/systems tests for low-no cost. In a few more years, if/when this becomes the norm, perhaps the gaming community in general will be a bit more conditioned or understanding of the process and the realities of it. But right now, you said it, it's a bit of a bad cycle. 

  5. #5

    Atropos

    Administrator121 Posts

    Great article @Nivhawk, I think it's really important that players go into this experience with as realistic as expectations as possible. Even given these preemptive warnings, lots of folks are going to get a bit disappointed when the alpha/beta experience that they "paid" for with their Kickstarter pledge doesn't live up to their hopeful expectations for what the final launched game experience will be like. The best we can try to do is educate and encourage reasonable discussion (at least on this website)!

    Creator of Ashen Foundry and Tamriel Foundry. Former guildmaster of Entropy Rising. Economist and MMO enthusiast.

  6. #6

    Nivhawk

    Member12 Posts

    Thank you @Atropos! I certainly agree- having discussions about topics like this will be really important for the AoC community going forward, and not just on a single platform. Podcasts, Youtube videos, as well as forum-based discussions I think can be an absolute benefit. The DCN podcast being an excellent example of a platform that provides a way for MMO players to stay engaged, aware, and involved in discussions (one of the reasons I personally enjoy it so much). 

  7. #7

    Glosterian

    Member8 Posts

    Thanks so much Nivhawk, that was really helpful. I've been an Alpha and Beta tester by invite on ESO previous but this is the first really concise definition of the stages that I've seen, and I'll have some bullet points of these handy when doing Ashes on Alpha/Beta.

    Every hoopy frood needs his towel!

  8. #8

    Nivhawk

    Member12 Posts
    Glosterian wrote on July 8, 2017, 9:53 a.m.

    Thanks so much Nivhawk, that was really helpful. I've been an Alpha and Beta tester by invite on ESO previous but this is the first really concise definition of the stages that I've seen, and I'll have some bullet points of these handy when doing Ashes on Alpha/Beta.

     Then I've reached my objective with making this post, and I'm so glad! How were your experiences in tests for ESO? 

  9. #9

    Roseeli

    Member16 Posts

    Beautiful post Niv <3 I shed a tear while reading it cause it was so amazing!

    I want to hope that the main thing that anyone that is Alpha testing a game takes with them prior to starting that test is: They are there to break the game. Not play it, but to find its flaws. It is not meant solely as a form of entertainment. It is, in actuality, a job. And the simplest of tasks can prove to be the most beneficial for the developing team when the time comes for them to fix any and all problems that will arise. 

    Once people come to that realization, the testing will be successful. Many times do I see people drooling over the possibility of getting an early access key. I feel that they they don't realize that they will in truth NOT actually be there to play the game.

  10. #10

    Qoggish

    Moderator10 Posts

    Thank you for the great post. As many have said, as someone who has tested games from alpha to beta in the past, I agree with all the points. It should be treated as taking part in the game development cycle and not just a free pass to play the game earlier - that is why I am against it being a reward for people, but I digress... Hopefully everyone will go into it with this article in mind. 

  11. #11

    Oxillion

    Member3 Posts

    Great post!

     

    This takes me back to when games actually tested for bugs.  Way too many IP’s call it testing, but it is a far cry from it.  No pun intended there.  The armies of people getting on at the early stage and not understanding what “Alpha” means.  Then there is the crowd that is there to play it and complain when things are not polished. Now with the added issues of streaming and people with the intent to release new content even under NDA’s is a real mess for gaming companies.  I digress, testing should be just that and those doing should be happy to be there and help and be constructive.   I will stop here…Bull rant inbound….

     

    Great post!!!

     

    Ox

  12. #12

    Atropos

    Administrator121 Posts

    This article from @Nivhawk is particularly timely this weekend and well worth a read for anyone involved in (or following along with) the Ashes testing process.

    Creator of Ashen Foundry and Tamriel Foundry. Former guildmaster of Entropy Rising. Economist and MMO enthusiast.

Respond

Sorry, you must first log in or register for a Ashen Foundry account before you may comment on articles!

Please respect the Ashen Foundry Code of Conduct when commenting on articles.