Boot-strap usability testing: How to test your designs with little or no money

This article is a short version of my presentation on the topic, which I can present to anyone interested. I do Bar Mitzvahs and weddings, but kids birthday parties are off the table. Sorry.

I’ve tested a LOT of different designs with users over the years, working in varying environments – from one extreme to the other. I’ve ran usability tests for companies who paid tens of thousands of dollars just to rent the (high-tech) facilities we used, and I’ve walked through design concepts with customer service reps in their break room. I’ve also helped set up three permanent usability labs, and through that process I’ve seen a lot of really expensive gear that several consultants said you “have to have” to really run an affective usability test.

To which, I say – Poppycock. Balderdash.

(I’d add another archaic term of disrespect, but I can’t think of one right now – maybe later.)

Cost is not important – the testing is. “It’s not that important, we don’t need to test that,” is a phrase I have heard repeatedly, sometimes from peers (who should know better). What I’ve found in my experience, the moment you have a high degree of confidence in a UI and think that you have “nailed” some design problem – that’s the moment you HAVE to schedule a usability test, even if it’s just with friends and family.

You may be completely right, the design may be utterly appropriate to the user task – but as a certain ex-President once famously said, “trust, but verify.”

Usability testing is of vital importance to any design, and doing as many focused tests as you can afford results in a better end product. Of course, if you’re an interactive design professional, I’m preaching to the choir here. You know that focused usability testing is important, and you are probably promoting usability testing to your project’s stakeholders all the time.

What you DON’T have, in many cases, is the budget to do the testing you know should to get done. And even if you do have budget, in current economic conditions it may be – or has been – squeezed. And many outside of our domain look at usability testing as budget items that can be cut, not vital activities that help maintain quality.

I’ve been there, done that, got the t-shirt. So, how to get the same results you need, for less money? Here’s some tips on cost cuts and strategies that will help get your tests funded and done.

(NOTE: In this article I refer to new designs, but the approaches I recommend are equally affective if you are testing existing systems to get “baseline” usability results.)

Need users? Don’t pay a recruiter, use Craig’s List.
This one is somewhat obvious, but worth noting. Craig’s List is an awesome tool to get leads on ANYTHING, from used pool tables to replacement lamp shades. It can also help generate leads for potential participants for your testing. Keep in mind that, unlike a recruiter you pay to do the screening, this will mean that you need to screen the candidates yourself, so have a solid script/screener you can use.

I have actually had the misfortune of hiring recruiters who claimed they had an extensive database to pick candidate participants from – who used Craig’s List to fill their slots. If you are comfortable doing the work (and have the time to dedicate to screening recruits)- cut out the middleman, and do it yourself.

Build your own reusable participant list.
The more you test, the more opportunities you will have to meet different people in various professions that fits certain “niches” that you may need to revisit for future projects. If there are no “awkward moments” during the tests and the participants provide good feedback, then ask if they would be interested in coming back for future testing. If they are, add them to a “call-back” list you can go to later.

Test a lot of stuff at once.
Again, seems like a no-brainer, but worth considering. Several times I’ve “stacked up” a backlog of design concepts that do not need urgent consideration, some barely past the ideation stage. I don’t set up specific targeted tests for this work, but the functional prototypes are “at the ready” whenever we have the time to expose them to users. If the dry-run of the test protocol runs 45 minutes and you have the participants for an hour, bring them into the test.

Interns rock – get some.
If you have limited bodies at your disposal, contact a local university and get some more. Most colleges have intern programs that you can tap into, and you will often find some very passionate students that can help turn what was previously planned to be a short session with participants into a full test. Passionate, and cheap – a great combination.

The cool software is nice, but usually unnecessary.
Don’t have the cool eyeball-tracking software from Facelab or great broadcasting/note taking tools like Morae? Don’t sweat it. Get a cheap $300 mini-DV camcorder and a $20 tripod, position it next to the workstation (so the camera can pick up the screen or workspace as well as the participant) and hit record after you have the participant sign the video release (also, obviously, don’t forget to have them sign the release). Don’t take notes yourself, focus on facilitating the test and following the protocol – you can take notes from the tape later.

Can’t afford the camcorder? Get an intern to sit in the room and take notes. Again, they’re cheap.

When all else fails, use friends and family.
It’s never preferred, because you may have built up a personal relationship with the very individuals you may be walking through one of your designs, and biases will creep in to their reactions. Get around this by e-mailing colleagues at your company and asking for volunteers – and, if possible, get one of your peers to facilitate the session if you are too close to the participant(s).

Schedule the tests whenever the participants are available.
We need to accept that people have lives, and, while making sure that you get that design right under deadline (and, depending upon the project, under pain of torture) may be of MISSION-CRITICAL IMPORTANCE, the rest of the world doesn’t care. So, be flexible – work with the participant’s schedule. They are doing you a favor by coming out – never forget that.

Don’t cut the participant’s compensation
One last note: don’t try and save money on participant compensation. Pay them for their time, and pay them well – I think $100 for an hour of a person’s time is appropriate. Obviously, adjust based on your location, but make sure you pay better than minimum wage.

Hopefully, this has been helpful, and remember, TEST YOUR DESIGNS. It’s important to do, even if you have no funding to do it. Test even if you don’t have the approval – in my experience, it’s always been easier to ask forgiveness than acceptance – especially easier after your usability test results in a better design to go to market with.

Comments are closed.