Getting More From Usability Testing Your Game

What makes a usability test work?

Decades of usability testing have established a standardised process for how they run. The approach is adaptable enough for any genre of game, but is standardised enough to know what types of data can be expected, and how accurate that data will be.

Does this prove players will comprehend our game?

Does our design reliably nudge players toward the fun?

Can players use the controls and UI in every scenario?

Each aspect of the prototype design has some desired outcome in players’ comprehension of game scenarios and players’ behaviours. Afterwards, with all notes from all 6-to-12 sessions in hand, teams ask themselves:

Are we OK with our designs generating these observed outcomes in players? If not, what’s the design fix?

Each usability test is short, focused, and iterative, usually taking 3-to-6 workdays. These rapid quality-checks are repeated sprint-to-sprint to re-test redesigns and new content.

Perfecting your usability testing

While the logistics of a usability test can sound simple, there are tough strategic choices to make to get the most from each test:

Formalise your ‘success factors’ for the first-time player

Teams will have objectives for what a new player will quickly understand.

Usability testing helps make these learnability expectations explicit, and then tests against them.

Before your usability tests, forge a list of ‘Things the player is supposed to understand in the first minute’, then -5 minutes, then -15 minutes, and so on.

Identify which mechanics are unclear by design

Following on from above. Tutorials usually aren’t proposed to teach the player everything.

Learnability is ‘under the microscope’ in the usability test lab. We must have a sense of what’s a pass or a fail.

The prime example here is anything ‘metagame’ or associated with longer-term progression. Players need a more playtime to grok the ‘loop’.

The value of a newcomer

Playtesters are selected anew for each round from the general public. But they should never be just some random person or volunteer. Choosing the ‘profile’ of the playtester is paramount to avoiding wacky and misleading insights.

“Is it Fun?”

Of course this is the number one question, top of every team’s mind.

Less “was it fun?”, more “could players get to the fun?”

Have a ‘next test’ question bucket

Each usability testing session tends to open a Pandora’s Box of new questions, concerns, insights and opportunities.

With sessions booked back-to-back, every interview question becomes precious use of time.

There is always a request for the moderator to “ask this player [XYZ]!” in the moment. A skilled facilitator will strike a balance: mainly following the interview script to capture answers comparable across all playtesters’, versus formulating new questions specific to this player or exploring unforeseen issues: going off script.

Design Some Representative Scenarios

Usability tests are designed to demonstrate natural play. Some UI design challenges can, however, sometimes only occur in uncommon or extreme situations that don’t arise in the short playtest sessions.

Only letting the usability testers roam freely might not expose nuanced design weaknesses.

To ensure playtesters are exposed to sensitive designs, it’s common to generate some scenarios for them to complete. For example, during usability testing on a third-person building game, we asked the playtesters:

Build a basic research repository

If you can find the brainspace between iterations of your usability testing (!) spend some time capturing some metrics on how playtesters performed.

Summary

Getting new players in front of your video game prototypes is a powerful and inspiring exercise. Setting goals. Observing players at play. Iterating design.

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store