Need to do some mobile usability testing? Here’s some more hints and tips…
In some previous posts I gave some advice on how to conduct a usability test of mobile applications. Well, after a recent series of tests had several setbacks caused by our old pal Murphy showing up (no, not Peter Weller’s character from Robocop, the guy who created the law named after him) I have some additional advice and tips for you
Get comfortable with the devices you are testing
Even if you are recruiting users that are familiar with and/or are “power users” of a particular type of phone, you need to spend time getting used to how the device works. You may encounter a person who are not accustomed to the eccentricities of how the phone operates in the space you are testing (for example, a downloadable app on a blackberry). Be ready to work through such bumps by knowing how to do
When possible, test no more than two devices in one round of testing
If you are going from one device to another, you tend to lose focus on what you should be doing which is observing how the participants uses the app you are testing – you will focus more on how the device works than you should. So, test two device types in each usability test round, and try and have all the participants of Device A test on Day one and all of Device B test on Day 2.
Recruit people who use the device, not just people who “own one”
Sometimes you will encounter a participant who got through the screening process and has NO IDEA how to do anything with his mobile device other than make or receive calls. You won’t get a lot of valuable data from this person, because a good half of your time with them will be spent explaining how to do things with his phone. Make sure your screener has questions to prevent these type of recruits from showing up.
Remember to separate the device’s usability issues from the application
On some phones you are going to see some very painful interactions that the participant has to go through to accomplish the task you have laid out for him or her. More often than not that is caused by the awkward data entry mechanism that the phone has and NOT the app itself. Always remove the usability problems the device causes from issues observed with what you are testing. If you’d like to catalog those issues and send them to LG or Nokia, fine, but that’s NOT your priority.
Make sure what your are testing supports the phone’s native controls
A couple of show-stoppers came up in a previous test when the participant used the native buttons/controls while accessing the app (the keypad’s “Back” button was one example). The app was not able to support this and logged the users out. If you have at least one participant do this, odds are that many more will being doing this as well. Make sure to note this as a MAJOR issue that the dev team needs to fix.
