Webinar Recap: Designing Testable Games

Gaming today has expanded far past consoles and arcades, into mobile phone games and online games, some with incredible levels of complexity and photorealism. And the industry is still growing—Statista estimated the value of the video game industry to be U.S. $115 billion in 2018, and projects that value to grow to U.S. $ 138 billion by 2021. The growth of the industry means that the need for thorough game testing has also grown.

Releasing a game without fixing all its bugs can cause great frustration to its players and impact sales. And game testing is much more complex than having testers simply play the game through to the end. Games need to be built to allow for testing at all levels, from checking that all basic systems work to checking that the full playthrough is an engaging experience. Although automation can greatly improve testing capabilities, human testers are still integral to any complete testing process.

By making sure that your code is testable as you develop the game, you can ease the task of game testers and set yourself up for success when the game is released. In a recent webinar, Marcin Seredyński, SDET (Software Development Engineer in Test) Lead for Lionbridge Poland, outlined what makes a game testable, how game testing works and when automation can work better than human testing.

To hear the whole webinar, click here.

What makes a game testable?

A game is testable when a tester can enter an input into a system to create the desired outcome. Being able to control the game is key to being able to test it, whether that is through using the controller or creating commands within the game to speed up testing and teleport to certain levels or locations. By automating control, a developer can test for the best-case scenario version of a game, while using human testers more closely mimics the experience of an actual user.

Testing methods also need to account for observability, or the ability to create a record of the game state or bugs. Automated game logs are useful in determining what went wrong if a game crashes and other data cannot be retrieved. “The goal of observation is simply yielding a pass or fail result that can be aggregated into a test report,” Seredyński says.

Testing is only helpful if developers can understand what the tests mean, and why a test might have failed. Clear communication among the tester, developer and designer is crucial so the testers understand what they should be testing. Seredyński recommends creating test levels to test specific aspects of the game. By testing in isolation, particularly for more complex narrative games where a player must complete a sequence of activities before advancing, a test level allows a tester to only test whether a certain component in the game is functioning correctly. “When that fails, you know exactly what has failed,” Seredyński says.

Seredyński cautions against only testing in isolation. At some point, developers need to test the entire game from the beginning to the end, and to assess whether the game experience is as desirable as expected.

Which game features should be tested by automation?

Game testing should be done in stages. First, check if system-level functions work, then make sure basic assets load, then more complex functions, before conducting a full playthrough of the game.

“Whatever is repeatable or tiring or boring for humans should be automated,” Seredyński says, and these tasks should take place during earlier stages of testing. “This isn’t saying that automation should replace humans. This is saying that automation should support humans.”

The main task that automation cannot do is checking whether the game is actually fun to play. Automation can gather engagement metrics, like how long a player spends on the game and how frequently they return to it, but it cannot provide subjective answers. Black box testing, or testing the functionalities of a game without having access to the code itself, allows human testers to provide feedback on the quality of a game and its design.

How can you make sure your code is testable?

Writing testable code goes back to making sure your code enables control, observation and isolation of certain components, and can be understood. Some key questions to ask about your code include:

• Does each piece of code have a designated, single responsibility?
• Who can access the code?
• How can you limit the number of factors affecting why the code might fail?

To learn more about how to write testable game code and answer these questions in more detail, watch the webinar here.