Playtesting Ideas Cheaply

Any of you who have played the prototype I posted a while ago can tell that there is a remarkable difference between it and the final product. Needless to say a lot of energy went in to improving Adlib but that by itself wasn’t enough. In order to make sure that my changes were making the game better I took advantage of Flash Game License’s First Impression service. Using First Impressions it was possible to play test new ideas and get a sense about whether it was the right direction to pursue. Today I thought I’d take a brief look at the service and how I used it when developing Adlib.

So what are “First Impressions”? In my opinion, the service is a way to cheaply get access to a slew of testers for your Flash game. For less than a dollar per a player you can instruct testers to play your game for a minimum of five minutes, rate it against five dimensions, and answer a custom question.

For Adlib I went through a number of rounds of testing to try out new features and improve the game. For example, let’s take the controls; in the original prototype Adlib was controlled via keyboard. Initial play testing revealed that people enjoyed the game but found it difficult to play. Oftentimes users explicitly requested the ability to play with a mouse. At first I was skeptical whether a new interface would improve the game. I had experimented with mouse controls earlier in development and had failed to find an intuitive setup. However, the overwhelming user feedback could only be ignored at my own peril.

Once I found a control scheme which I was satisfied with I submitted Adlib for another round of First Impressions. The service shines when you use it to quickly perform A/B testing in this fashion. In about 24 hours I had collected all of my impressions and was able to analyze the two sets. This time people didn’t complain about the controls, but even more interesting they generally spoke of and rated the game more favorably. This gave me confidence that my changes were solid

So, you might be saying, “Of course I want to perform A/B testing on my features, but why should I pay someone to do it?” The answer really boils down to a few advantages that the First Impression service has over traditional beta testers:

1. Impressions are usually from unique players who haven’t played your game
2. Players have to rate your game across multiple categories
3. You can cheaply get access to many players
4. You can reasonably expect your feedback within 48 hours of purchase
5. It’s easy to compare results from different versions of your game

Ultimately, the service shouldn’t replace traditional beta testing but can serve as a valuable complement. For Adlib I found the service to be indispensable. I hope you’ll try it out in your next project, and if you’ve used it already why not share your experiences?

Related Posts

• Elements of Progression in Games
• Adlib: Play it now!
• Adlib’s Story
• Introducing Adlib, a first look