What is the best way to create a UX roadmap?

First off, let’s discuss the term “UX Roadmap.” I’ve heard it used in a couple of different ways. Here’s one definition, from UX Game Changers:

The UX roadmap defines the stages of the user delivery. And while demands can change deliverables, the roadmap provides guidance and helps set priorities. The term “roadmap” is defined as being a course of action or a plan for future actions.  Roadmaps provide the underpinnings of what should eventually turn into efficiencies and revenue.

Here’s my take on it:

A UX roadmap details where your users are, where they want/desire to be, and when you will provide offerings that will take them where they want to go. It is a timeline of activities and offerings that is driven by user needs and aligned (and, optimally, influencing) product release schedules.

An example: You sell a widget that allows people to instantly see their blood sugar level. This widget does one thing – the blood sugar check – incredibly well. Your users like it because it is simple and effective. However, they want more – they want to be able to log this sugar over time, and they also want to not have to sync this information with their computers – they want to just have the information “beam” itself to there. And as these users get more exposed to similar technology, they will also want to have multi-function devices that supports more than one function.

They – and the world – are heading towards “multitasking enablers” that support  health monitoring. How do you evolve your product to keep up with that? You flesh out a roadmap based on research, understanding, technology, and society trends.

This is what a lot of product managers do, but their product roadmaps are often “keep up with the Jones” efforts, where they strive for feature parity with competitors. The secrets sauce is the UX roadmap, because it looks not at the competitive landscape but user needs and their mental models. If you don’t do that, don’t understand where people’s “heads” are at and where they are going… Well, it can lead to a failed product line and a dead company.

See question on Quora

Seven things your boss needs to know about UX

As someone who has been in the user experience domain for a LOOONG time, I have had lots of conversations with many different executives. And, most of the time, these execs knew how to spell “UX”, but they didn’t know much else about it. There were many misconceptions that needed correcting and while it was sometimes frustrating, I also enjoyed the opportunity to educate these key decision makers on the discipline and how it worked.

Here’s seven conversation topics that came up when I was discussing UX with managers, and all are worth pointing out to YOUR boss – especially if you are a UX consultant and your boss is your client:

UX is iterative

“What is taking so long?” One impatient CFO once asked me. He expected the design for the application my team was doing to be “one and done” and the idea that we were having iteration cycles confused him. I had to explain to him that the process we followed was iterative, and that we had to refine the designs to get to the point where it was at the appropriate level of quality. He resisted that idea, but the results convinced him that iteration produced results.

User research is vital

“We already talked to our customers, why do you need to?” A project manager was surprised at the request, and we had to explain to him that a proper UX project required insight and understanding about what people thought and felt about the domain, and we couldn’t just leverage marketing research. Having those one-on-one conversations let us build out personas, and these personas, representing what we learned about the users, gave us “targets” to align our designs too. The project manager grudgingly accepted the results.

There is no standard “UX process”

Some managers are very process focused, and they want to see a Six Sigma-like UX process chart, one that is approved by some central UX “signing authority.” Well… while UX is a mature discipline with many different defined processes, it’s more flexible than that. In fact, UX is more a series of tactics and approaches than a formalized process. Many design teams and consultancies have formalized their specific processes, but there’s on “one UX process to rule them all.” Which is a good thing, in my opinion. It allows us to apply a flexible toolkit to solve specific business and user problems in targeted ways – letting us use the right tool for the right job.

UX is not UI

Well, specifically, UX is not JUST UI. Many executives that I encountered think that UX work was just designing screens. It’s not. And in fact, as we move more towards a connected “Internet of things” world, user experience design is going to be less about screens and more about how different systems and processes interact with each other. And burgeoning UX specialties like service design and content strategy are leading the way to be more about process and content design than UI creation.

Even if you are designing an application or a web site, there’s still more to the jib than designing and documenting screens. There’s the aforementioned user research, scenarios, storyboards… and of course:

Usability testing is incredibly important

“You guys are experts, why do you need to test the designs you come up with?” That was the question one stakeholder asked me as we were planning out the engagement.

My response was as follows: “Even if you have confidence in the design you have done, you are not the user. You can apply best practices, use the right design patterns, and do all the necessary research… but you still will not know how people will react to the design until you test it with them. And the best way you have got the design right is to have the user engage with it and then TELL YOU how it works, with no instruction or demonstration before hand. If they can do that, then you know you’ve got the design right.”

That answer convinced him. Because it’s true.

Usability is not UX, either

“You’re the usability guys, you tell us what it should do.” I sometimes think I should start a consultancy called “The Usability Guys” because of that comment. Usability is not UX - it’s important, but UX is much bigger and broader than that. In fact, because people’s standards have increased as they are exposed to better and more usable apps, usability has become the “table stakes” that all solutions need to provide. Anything that isn’t usable, in the competitive marketplace of ideas… will not succeed.

UX “Unicorns” are rare

“Why do we need a lead, a designer and a graphic artist? Can’t we just hire one guy who does all of that?” That was an executive looking at a proposed job ladder that was created for my team. I had to respond that what he was looking for was a “UX Unicorn,” a rare breed that is nearly never found in the wild. UX is a broad domain, that encompasses many different disciplines. While I believe that, as Robert Heinlein famously stated, “specialization is for insects,” it’s incredibly difficult to become proficient at all the skills in UX. Sometimes, specialization is useful and necessary.

There you have it, seven things that bosses should know about the user experience discipline. Hopefully, you can use these points to educate your superiors and “manage up.”

Will big data be used increasingly in the UX field to answer user experience questions?

Yes and no.

Big Data provides… well, big data. Lots of data. This data needs to be parsed, analyzed and understood. It is (potentially) a source for user insights and understanding that is unparalleled. It can provide a lot of answers into what users are doing, where they are doing it, and how they are doing it. So yes, Big Data will increasingly be used by companies to understand user behavior (many tech companies like Google have been doing it long before the term “Big Data” even existed).

But Big Data cannot tell us WHY users do things with real certainty. What motivates users, what their psychographic and emotional landscape is… it can be inferred from Big Data but not (in my opinion) to any degree of certainty. It will still take ethnographic research and usability testing to gain those particular insights (or at least identify them with any real confidence).

And I don’t state this because I love user research – though I do. I state it because I have worked with Big Data and I have seen its limitations. And Big Data is only as “good” as the data points that are captured. If you do not have the specific data you need to inform design decisions, it’s useless.

Finally, there is the question of access. Big Data tends to be mostly used in Enterprises, and even then access to the data is very limited. Until Big Data becomes available to access by design teams at mid-level companies and agencies, Big Data will not replace or supplement good old fashioned user research.

See question on Quora

What are some design problems a young designer can work on to build he design “chops”? Here’s a couple of ideas:

Learning how to read and write. Adult literacy around the world is at an all-time high, but there are still thousands of adults who can’t read or write. Can an self-guided app help solve this problem?

Budgeting. People need to budget their finances, but when you look at the amount of credit card debt in the country you see that many of us live beyond their means. A simple practical/instructional app can help people learn how to budget.

Decluttering. People have a lot of stuff, and in many cases way too much stuff. Can an app help them inventory and then prioritize their possessions, to help them simplify their life?

See question on Quora

A great experience comes when what is designed aligns with the needs and desires of the users. This means that the offering “fits” the user, if not like a glove then… like a nice warm bathrobe. A great experience accommodates all the factors – shape, touch, mental models, emotional drivers, and the user’s expected behaviors and outcomes. Controls should work as anticipated, and respond quickly.

Sounds easy, right? Well, how of you get there? How do you ensure that the great experience takes place? The primary thing to do is know your user. Research, dig deep into what they need and expect and then use that understanding to inform your design. Iterate, taking the candidate solutions back to users to test them, and then refine based on the feedback received.

The process will vary, but the key are the two things noted above: Iteration and user research. You have to do both in order to ensure a great experience.

See question on Quora

#UX101: On usability testing

Once you have created an interface, you need to test it, and this is known as usability testing. How do you do that? Well, it’s easier than you may think.

First, let’s define what we mean by the term “usability test.” A usability test is a structured session that involves a person engaging with a design and providing feedback on said design to a facilitator. Usability testing allows for an open objective conversation about a design to identify what works and what doesn’t. Why perform usability tests? By testing designs early, you can quickly identify potential problems with the interface before it is coded and implemented. This saves a lot of time, money, rework… and potential embarrassment.

I’ve met several UX professionals that think you need to have a huge usability lab with thousands of dollars worth of equipment, a two-way mirror, specialized software, and more. While I’ve used (and set up) a couple of usability labs, and there’s several benefits from having such a location for user research and testing… you don’t need most of that stuff. What do you need? The only things you absolutely need are:

  • You
  • The Design
  • People to show the design to

That’s it. You can do usability testing almost anywhere, and you don’t even need a computer. Now I know that there’s an entire cottage industry out there of people who sell usability-testing equipment, and many of them would prefer I didn’t state this. Sorry, but I call it like I see it… though I will spend some time detailing some usability test tools I have liked and used later.

While you don’t need a lot of James-Bond style gadgets to do a usability test, you do need to plan things out so you can get the most out of the effort. So, let’s talk about how you can plan a usability test. 

There are five things you have to do beforehand to have an effective usability test session:

  • Define test goals
  • Formalize the test artifacts
  • Define the type of test and the test method
  • Identify who (and how many people) to test with
  • Write the test protocol

Define test goals

What do you want to get out of the testing? What do you want to find out or understand? Define clear goals to focus on, and then make sure that the test details don’t lose sight or “muddle” those goals. If you don’t have any clear goals, then focus on getting answers to three key questions:

  • Do they “get” it? If they can grasp the purpose and utility of the design, you’re in a good position.
  • Can they use it? When given tasks to accomplish using the design, can they do it.
  • Can they explain it to you? Can they describe what it is and how it would work?

Formalize the test artifacts

Where you are in the design process informs what design artifacts you create and use for usability testing. If you are doing early testing, the paper sketches or “prototypes” are fine. You may need to sketch out different “screen states” for some screens to represent the process you need to test, but that depends on how you are testing and what your goals are.

If you are testing more “mature” designs you are going to want to make the test artifacts more interactive, so the test participant can engage with the design more. Creating an interactive prototype is fairly straightforward if you can code, or have access to a team member who can develop it for you. If you don’t have those options, tools like Axure RP, Omnigraffle or iRise can help you make clickable prototypes (more on those tools later).

Define the type of test and testing method

There are generally three types of usability tests: Formative, Summative and A-B Testing.

Formative tests are used early in the design process to assess the effectiveness and usability of a preliminary design, as well as to capture users’ thought processes and conceptual understanding. It can be done with sketches or more “formalized” designs, and can be task-driven or an open-ended conversation.

Summative tests evaluate more detailed designs to determine the satisfaction, effectiveness, and overall usability. It usually takes place after an earlier formative test has occurred and is usually very structured and task-driven. You can perform these types of tests with existing systems to evaluate the current state of a design.

A-B testing compares two or more products or designs to identify and distinguishes the strengths and weaknesses of each. If you have two differing approaches to solve a design problem, this is a good way to evaluate them. This can be done at any time, and can also be used as a competitive analysis tool (test two existing sites with the same tasks to identify what site better supports the user). What you will want to do is have two sets of participants, with one set testing design A and the other set testing design B.

Now that you have identified the type of test you want to do, how do you want to execute the test? Again, there are three different ways you can do it.

In-person, facilitated tests provides the opportunity to gather the most comprehensive feedback from participants about a design. It allows for any type of testing, and provides the most flexibility of what and how you test. In my experience, I’ve found it’s the best method to test designs for mobile application

An in-person facilitated test takes about an hour and you’ll expect to pay participants at least $100 for their time (usually with a VISA or Amex gift card).

Remote usability testing allows you to get feedback on designs without having to facilitate any sessions. It will cost less than in-person testing, in that the services charge a flat fee of less than $50 per participant. However, it doesn’t provide for extensive conversations or in-depth testing of designs, and the results can be hit-or-miss. In my opinion, this method is good for getting feedback on early concepts and if you need really quick user feedback.

If you’re strapped for cash or time, consider in-person friends and family testing, AKA “guerrilla” testing. While not as useful as formal in-person testing, it’s still a good way to get “casual” feedback or do some simple A/B testing. It has costs besides people’s time (and maybe some snacks or a lunch).

Identify who (and how many people) to test with

Who do you test with? People who reflect the key characteristics of your personas, of course. Use the information that fleshes out these personas to inform the creation of a recruiting screener. Be sure to have some “disqualifying” questions in this screener to prevent the wrong type of people to get in the test group (an obvious disqualifying question: (Do you work for a competitor?”)

When you test a design, you need to get a lot of different people to look at it. The question, of course, is how many is “a lot”? Well, it depends on the type of test you are doing. If it’s an early test and you want to get high-level feedback, you may decide to have a larger group of people. If it’s a task-driven test that is focused on identifying potential design or usability issues, then you can “get away” with only five people.

Why five people? Because the law of diminishing returns applies. Jakob Nielsen, after his company spent years doing formal usability tests, looked at the data and identified that after the fourth person you will (on average) identify over 80% of the usability issues in the design tested. With 5 people, that percentage goes up to over 95%. So, unless you REALLY want to test with lots of users, five is enough (details here: http://www.nngroup.com/articles/why-you-only-need-to-test-with-5-users/).

When it comes to recruiting participants, there are usually local companies that do market research who can find people for you. Expect to pay $100 “finders fee” for each person (not counting the compensation you will need to pay participants for their time).

Writing the test protocol

When writing the test protocol, you need to include the goals you have defined earlier as well as the list of materials/designs you are going to review. A lot of people like to script out everything – the questions, the introduction, the whole thing. I leave this to you, but the key is to present the design artifacts in a consistent way to all participants.

If you defined user scenarios to inform your designs (through either a narrative story or a journey map) then you will already have a list of tasks to use in your test, making this job easier. Using these scenarios in a usability test will also allow you to “sanity-check” these design artifacts to make sure that you have correctly understood user needs and intent – if many of your test participants say “I’d never do that” in response to the task they are given, you may need to rethink some previous decisions.


A decision you need to make at some point is whether your or some one on your team will do the testing or whether you should “farm it out” to an independent UX consultancy. The answer to that depends on some key questions you should ask yourself. Can you be objective? Do you have the proper skills in-house? Do you have confidence in your team’s ability to do the testing? Consider doing both – maybe you do formative tests internally and then supplement that with an outside consultant to do the summative testing.

Now that you have fleshed out your plan, identified what you are testing, defined participants, and have anything ready, it’s time to test. Which is what we’ll cover next time…