A very interesting behind-the-scenes story about the recently completed US presidential election cropped up late this week. As we all know, President Obama was reelected, beating Republican challenger Mitt Romney. Looking at the election totals, it showed that while Obama won the popular and electoral vote, both candidates did worse than their parties did in the previous presidential election. Obama lost almost eight million votes off his 2008 total, and Romney had almost two million less votes than previous Republican candidate John McCain did. Many pundits responding to this declared this ended up being a “get out the base” election, and Obama won because he was able to get more of his party’s core voters to vote (where it counted) than Romney did.
How? How did Romney lose a significant percentage of votes that an (over) confident Republican team thought they had “in the bag”? Social issues and other factors aside, I think a core reason for this was a failure in user experience design, the result of over sure consultants in Team Romney, consultants who looked at technology as a “magic wand” to solve a host of real and imagined problems. The technological solution these consultants created was with a system called Orca, a “big-data” app that was to help track who had voted in order to support tens of thousands of volunteers in the Republican “get out the vote” effort.
Here’s a quick summary of what happened with Orca from Computerworld: The app was kept secret from the bulk of end users until the last minute, and was unveiled with less than two days for the volunteers to learn how to use it. As it was apparently not “stress tested” it crashed – repeatedly – throughout election day. And when it did work much the data was generic and untargeted, making the app almost completely useless to the volunteers. The result was that many people spent more time learning and figuring out the app than actually using it (additional technical details here).
Here’s a “man on the street” report from a volunteer who tried to use the Orca app:
“My location and voter lists didn’t load. I called their Boston HQ about 15 min before the polls opened. They said they would get back to me. To their credit, they did call back within an hour, and my location/lists started loading. ”
“The voter lists took FOREVER to scroll through. At regular updates when the precinct officials post the voter rolls, we had an avg of 200 voters to mark off. It took me 20 min to do this by hand with hardcopy lists. When I used Orca, there is no quick function to sort through names, say, by typing in the first letter of a voters last name. You have to hit arrow keys to thumb through pages. It took me AN HOUR sitting in my car (they don’t allow cell phones in our polling place) to mark everyone off. That took me out of watching what was going on.”
“The app crashed on me a few times. That means I had to start over at the beginning of the voter list, and thumb throw the list which took 10 minutes for example, if I left off at names beginning with H. The search function by voter name did not work.”
So, what lessons can we learn from this? Here’s a few that I can see from my perspective as a user experience professional:
If ain’t broke, don’t fix it.
The previous get out the vote efforts were all paper-based, with volunteers marking off who voted and the local campaign management taking the lists to review and then call registered voters who hadn’t yet voted. This was done by both parties for decades. Replacing this with a new web-based mobile app, rolled out to a group of volunteers whose level of tech savviness varied widely… well, this probably looked like a great idea at first. On paper.
Technology isn’t a magic bullet.
Any deficiencies that an organization has can’t be fixed with a gadget or an app, even if the app is specifically designed to fix a specific targeted problem. In order to execute a plan, you need smart people, good communication and active collaboration – the app is a tool that can’t make up for institutional deficiencies. That the instruction manual for Orca was a PDF e-mailed the day before the election to 30,000 volunteers… that doesn’t sound like good communication. Volunteers had to take to social media to ask EACH OTHER for help figuring out the app.
A mobile website isn’t an “app.”
The makers of Orca continued to call the mobile web site that was the primary UI an “app” and this led many of the volunteers to continuously look for the app in the Android or iTunes marketplace. And because it was a website many volunteers dealt with sporadic internet coverage and had to “learn” a UI that did not follow the conventions they had already learned from actual apps they had already used on their phones… precious moments of cognitive dissonance that could have better applied to, you know, getting out the vote.
Big data needs filters.
Instead of providing a targeted app for users based on their location, the app was pretty “dumb,” and the volunteers had to apply their own “filter” to the raw data to make sense of it. Not a big problem for experienced poll watchers used to interpreting data, but a huge obstacle to users who were new at this sort of thing.
Usability testing is vital.
As previously noted, the first time the GOTV volunteers saw the ORCA app was the day before, and there were many confused users out there. If they had done some simple due diligence and hired a usability professional to show the app to a select group of volunteers before election day, many of the issues that took place could have been prevented. The cost of doing this? A thorough round of usability tests can take place in less than a month and for less than $20,000. The cost of not doing this? A billion dollars spent, in a losing campaign.
“Trust but verify.”
Ronald Reagan famously stated this when it came to nuclear arms reduction, but unfortunately the Romney campaign didn’t follow this advice when it comes to the service level and capacity that Orca was expected to handle. There are many different ways that you can “stress test” a website or application before rolling it out. Because of the widespread outages that were reported as happening, I doubt this happened.
Final thoughts: I hope no one reads this and thinks that I am either mourning a Romney loss or celebrating an Obama victory… that is not my intent nor my point. I would write a similar article no matter what “side” this happened to, because I look at the evidence and see an inescapable truth: User experience design matters, and anyone who invests money in developing any software or service solution without spending time focused on foundational user-centered activities do so at their peril. So, if you are planning a big project or running for office, apply some UX design to what you are doing. You will be a lot more successful than if you don’t.
If you have any doubts, you can take them up with Team Romney.