This is a topic of discussion that comes up in conversations with my peers about every six months or so. Should you test your own designs? Many, perhaps even the majority, say no – designers need to not facilitate usability testing of their own designs because of the potential bias that the designers bring. Others say it depends on what is being tested – for example, early concepts could be tested by the designer (because that is often more of a conversation than a strict task-driven test). Here’s my opinion:
It depends.
I have tested my own designs in usability tests, both early formative tests and later summative tests, and I am well aware of the biases that a designer can bring to such an activity. Many times you see users struggling with something you designed and it’s hard to keep the emotions that produces in check (or, if the participant doesn’t “get it,” your frustration becomes a factor). This is where my training and background in journalism comes in handy – you have to remain impartial and unbiased.
In many ways, you have to become Spock.
(Yes, I snuck another Star Trek reference into an article on user experience. I gotta be me.)
Spock had no “ego to bruise” and he never let his “passions be his undoing.” When he encountered a situation he wasn’t expecting he was not upset, he was fascinated. That’s what I try to do – which is often a necessity, as budget and time constraints usually mean I have to design AND test for projects.
No, I’ve never been “perfect” and there have been some moments I “nudged” a participant, and I almost always recognize I’m doing that when it happens. Obviously, try and avoid that. As I’ve mentioned before, I try and avoid making my own mistakes and instead learn from other people’s mistakes, and working at another company years ago I learned plenty when a usability test went very very wrong.
The company had a really slick usability lab, with an audio/video control room that had multiple monitors that connected to the participants area. We were able to talk to the participant via a telecom system from the monitoring area as they interacted with designs we had installed on the computer set up for them. One day I was facilitating a test of a design that a colleague had created – he had a passion about what he had done and was proud of his work.
Well, participant after participant continued to have issues with his design – they just did not “get” the information architecture he had implemented. My colleague got more and more frustrated and angry at what was happening with each test. Towards the end of the day, the last test subject, as others before her, encountered issues with the design. The designer was in the control room with me, and as I was talking to her through the intercom system the designer turned to another colleague and shouted “She’s just stupid – they are all stupid. It’s obvious! Can’t they see it?!”
As my mike was STILL OPEN.
The participant, startled, asked “What?”
Needless to say, the test was over, and the designer never saw this design implemented (due to both the usability issues observed and the unprofessional behavior he demonstrated).
So, what’s the lesson? If you’re able to separate yourself from the design and can follow “the Vulcan way,” then facilitating usability tests of your own work is possible. If you are too “wrapped up” in the work, then ask a fellow UX colleague to do it. When it doubt, ask someone else to test your designs.