Last month I was browsing Fark.com, a popular news and community site, and came upon a link to a Wall Street Journal story about how hard it was for college graduates to get a job. One if the main subjects of the article was a college graduate who had not worked for two years, because he was holding out for a management job. He actually passed over a $40,000 a year opportunity at one point. The Fark commentators, many of whom are un- or underemployed, ripped this guy to shreds. It was, as they call it, an “epic thread” – thousands of comments were posted.
As I was reading it, I started to learn something very interesting. All these different individuals, people from all walks of life, were revealing their attitudes and, to a large extent, themselves, through their comments. It was a user researcher’s dream – unsolicited raw data from users, unedited (save for automatic profanity filters) and pure. I’ve done research projects that had less data points to analyze than this ONE comment thread – and spent a lot of time and money to get it.
Now, this is not to say that this type of approach is a substitute for actual user interviews, but some data is better than no data… And if you are constrained by budgets (and who isn’t these days) then leveraging this open information can provide insights for very little money. Here’s some recommendations as to how to do some quick and dirty “crowd sourced” user research:
Target the right sites/sources
Always keep what you are trying to find out, and let that guide the sources you use. What is the design problem you are trying to solve? What do you need to find out about your users? You have to focus and find the right source of data for your research needs – if you are doing research into attitudes towards technology, then look towards the comments on news sites such as Gizmodo or Engadget. If you are looking at attitudes towards politics, look at sites throughout the political spectrum. And so on.
Fark.com, because it has different sections with a continuous flow of linked stories every day and an active community of users is a great place to start or supplement the message threads you target for analysis.
Each online community has a different personality
Some sites are dominated by personalities that are snarky, intemperate and rude. Others have commentators that post replies that are thoughtful and in many instances are more intelligent than the original article/link they are commenting on. Understand that so that when you analyze the data you gather you can properly frame the comments to inform greater understanding.
Be structured
Set up rules and follow them, even if they may come off to others as arbitrary. But the rules have to be driven by what you are trying to find out about users. If you decide that one of the rules is to only look at users who actively engage with the community, that’s fine… But that has to be driven by a research goal to study only active Internet users. Don’t decide to not look at any comments that contain profanity just because “you don’t like it” – the data is the data, whether you like it or not.
Analyze the data using an affinity exercise
Yes, it’s hardly “green”, but consider printing out the comments in the message thread(s) you have targeted for analysis and doing a card sort/affinity to arrange “like” data points to identify attitudes, potential trends and inform draft conclusions. There are plenty of resources on the net that describe this process, so I won’t repeat that here: Google is your friend.
Ignore online polls
You may be tempted to add online polls and their results in this type of “data mining” exercise. I would avoid it, because in many instances these polls are “self-selecting” and inherently biased by the audience of the site that presents it. Such polls are useful, however, in that it can give you a sense of the community that the site has (see above).
Understand the limits of this approach
This type of analysis is no replacement for good old fashioned ethnographic research or focus groups – it will give you better understanding of users and inform any design work you do, but you still need to “pound the pavement” to get the deep data.