#UX101: Best practices for gesture interaction

Gestural interaction has been around for a while now. Inspired by movies such as Minority Report, engineers and designers have made the ideas seen in that film a reality for millions of users. From the simplest “shake to play another random track” to complicated game-based gestures, the way we engage with technology has advanced greatly from the traditional keyboard and mouse. The release of the Leap Motion controller in July 2013 has brought incredibly advanced and sensitive motion control tech into homes for less than $90, and the Xbox One game system promises an additional “leap” in the commercial application of this new way of doing things.

As this “new way of doing” looks to not be going away anytime soon, here’s some “best practices” on how to do gestural design for your application.

Decide how and when gesture control should be used… and choose wisely

One of my favorite pieces of movie dialogue is from Jurassic Park, when Malcolm (played by Jeff Goldblum) questions the whole premise of the theme park:  “You realized you could do it, you never thought if you should.” Applying gestural control is a great example of technology that could be applied… but should it? Don’t just decide to do gesture control because you can, apply and use it in thoughtful ways… Make it align with what you have designed and don’t make it a “bolt-on.”

Keep gestures simple

I’ve seen some examples of gesture control that, well… are too cute and complicated to work. People don’t want to look foolish, even if/when they are alone. Don’t have users doing the “chicken dance” from Arrested Development in order to accomplish a simple task. Be thoughtful and keep things simple and straightforward.

Gestures have to make sense

Don’t do a gesture that doesn’t align with how people do things in real life. An example: If you are paging through “cards” in a UI, use a swipe to the right to dismiss the top card on the stack. This is like dealing playing cards, and this gesture will align perfectly with how people do things in the real world. If you do the opposite (swiping to the left), it will be unnatural to people and will reduce the “learnability” of the app using such a gesture. Remember: The more natural and obvious the gesture, the easier the application that has these gestures is to use.

Use common gestural “affordances”

Pull, push, swipe, twist: we all use our hands every day to do things with gestures like these. This follows through on the point made above: Leverage how people do things to make the gestures make sense to users. Watch how people use their hands and then reflect that understanding in the gestures you apply in your designs. Don’t try and be “creative” when you have the right answer staring you in the face: Align with what people already do, and you will be successful.

People are lazy

I spoke about future UI concepts and gestural computing at a conference several months ago, and as part of that speech I asked everyone to raise their arm above their head and keep it raised until I said so. Within one minute many of the people in the audience were getting fatigues, and for good reason: Most of us are not fit enough to maintain a gesture for more than a few seconds.

In fact, when Tom Cruise was filming his gestural sequences for Minority Report, he was dripping with sweat within an hour of shooting the scenes… and he’s in a lot better shape than most people (like me). Good gestural computing design aligns with the ergonomics and limits of its users, and understands that people usually need “restful” moments where they are not gesticulating. I labeled this section “people are lazy” but that’s not really true – they are human. Understand and align to their human limitations.


Joseph Dickerson is a user experience professional and UX Lead for Microsoft based out of Atlanta, GA. He has implemented processes in user testing, design and ethnographic research and provided design and consulting services for many different projects and organizations.

Back to Top