Well…I’d love to raise my nose in the air and snootily say, “That is ridiculous. That sort of thing does not happen!” But I’m pretty sure many of us have witnessed times in which it indeed has been the case. It’s interesting really, because the fact is that nobody wants it to end up that way. Not the people who paid for it, not the people who conducted it…and you know, even participants like to feel that they’ve made a valuable contribution via their role.

This man has made a living by riding his bike around and dusting unused research reports.

If that’s the case, then why does this happen? I suggest several scenarios:

1. The information isn’t all that great. This could be because the firm doing the research didn’t do a good job conducting it, or maybe they stopped too soon – that is, maybe they did do good work conducting it, but didn’t take it to the next level – they didn’t look for insights and actions tucked away in all of that data and conversation.
2. The organization isn’t prepared to use it. Sometimes this can be fed by the fact that the research firm they’re working with didn’t take it to the next level, which makes it less actionable, I’ll admit. But other times, the organization paying for it wasn’t quite ready to listen or make changes in the first place…so it just sits there on a shelf. And let’s face it, even the best data typically has an expiration date on its usefulness.
3. The organization doesn’t like what they hear, so they disregard it. We’ve seen it happen. It can be very hard to hear negative things about your organization, so there’s a tendency to want to defend current products or processes if you’re receiving information that isn’t entirely complimentary.

I’m sure there are other reasons, and if you’d like to add to the list, feel free.

Much of this list seems to suggest that lazy, shelf-occupying research is the fault of the organization buying it, and not the firm conducting it. But that isn’t really the takeaway, here. What I want to emphasize most is that research isn’t meant to sit on a shelf collecting dust, and an engaged and insightful firm will help you make sure that it doesn’t. They’ll have a more intimate understanding of your organization and the landscape in which you function. This is the kind of research – and relationship – that helps you effectively use that research to your advantage.

I’m calling this one PLAUSIBLE, realizing that the fate of any research, no matter how good it is, still depends on the ability of the firm doing the research to provide actionable insights, and the willingness of the organization to heed what the research is telling them.

Has your research fallen victim to a weekly visit with a feather duster, or did you find ways to fold what you learned from it into your organization?

Here’s a graph from ESPN.com that I found interesting for today:

Nationwide, we see that ESPN.com users overwhelmingly feel that the University of Kentucky is the #1 seed most likely to win the national title.  With over 93,000 completed surveys across the country, we can probably assume that more people in your office pool are picking Kentucky than any other team.  That is, unless you live in North Carolina, Michigan, or New York.

Survey respondents in those three states—which are home to the other three #1 seeds—selected their home team as the one they thought would win it all.  Do they have a bias?  Sure, they’re cheering for their team to win!

This is perhaps an oversimplified way of showing how survey responses can vary by audience and how their responses can become swayed.  What’s important to note here, though, is that it’s not always this easy to spot these types of differences in survey responses.  Sometimes, little nuggets of information and even key points are overlooked.  That’s where having a team like Corona (no pun intended) on your side can come in handy.

As Meredith mentioned in a previous post, stay tuned for some stats from Corona’s March Madness office pool, and good luck to all!  Predictions are more than welcome in our “Comments” section.

As a gal who holds qual research so near and dear to my heart, my initial reaction to the first half of this myth is to lean back in my desk chair and yell dramatically, “Noooooooo!”

But in an effort to remain professional, I shall take a deep breath and propose that, as any good researcher would tell you, there is a time, a place, and a purpose for each of these approaches to understanding your various constituents. Google “qualitative research” and the first search result will send you to the Wikipedia page on the topic. It’s tough to talk qual without contrasting it with quantitative research, as the Wiki page does, but it boils down to the type of information you’re trying to get. The widely accepted rule is that quant research helps you learn what, where, when, how many, and qual research helps hone in on the how, and why – mostly related to decision-making behaviors.

So you can see why one might be preferable in one situation but not another.

I think I’ve noted the ‘quant is better’ school of thought (perhaps because I’m sensitive to the assumption) in situations when we are working with a client who must take the results of the research and present them to decision-makers – and really, most of our clients must do just that. And I understand.  It’s different going to your manager with numbers and statistics with margins of error than it is going with quotes from a focus group. You don’t have a way to prove the accuracy of your qualitative results. Even when you’re working with a smart, reputable firm, qual research is tricky…but it’s the only way to dig deeper than those survey responses allow.

The fact is that there are certain situations in which there’s very little substitute for qual research, nor is there for quant. They’re a good team to help you answer all sorts of questions. So next time, before you scoff at the suggestion of a focus group, or poo-poo the recommendation for in-depth interviews, just consider what you’re trying to learn. And for goodness sake, find someone bright to guide you to the right research methods, and conduct it well.

I’m thinking we can declare this Myth TROUNCED. What do you think?

A recent survey from Deloitte found that one in five U.S. residents say they have either cut the cord (i.e., cancelled cable or satellite service) or are thinking about doing it. Nine percent of survey respondents indicated they have recently “cut the cord” while another eleven percent are thinking about it. Cord cutters are turning to services like Netflix, Hulu, and Amazon Prime, to name a few.

Interesting and a considerable percentage, but probably not true.

The problem?  The TV industry reported flat subscriptions last year.  As this article notes, even with a large margin of error, the numbers don’t work.

What happened?  Assuming the TV industry is not misleading their subscribers (a 10% drop would be hard to hide in earnings reports) or there isn’t a flood of new subscribers hiding the cutters (though the article seems to discount that, too), the one culprit is likely poor sampling.

The survey was administered online, and while more and more Americans are online, biases are still likely.  While they didn’t reveal how they sampled respondents online, my initial guess is that they skewed towards early adopters and people with faster, more consistent access to the internet.  The exact population you would expect to be shunning cable in favor of online options.

If you’re not really surveying a representative sample of the population you’re interested in, then you’re not really surveying them.

It first asked the six digit store number and then had me verify it was correct.  This was followed by a question if I used the drive-thru or dined in.  Next, I was asked how many days ago I visited the restaurant, what time I placed my order, and if it was AM or PM.  After keying in this background information and confirming it was all correct, I got to the one and only experience-related question: “Would you recommend this restaurant based on your most recent experience?”  Yes, it’s essentially The Ultimate Question, but was that one question enough? Even if the results were cross-tabbed with the previous questions?

To be fair, I was given the option to leave a voicemail for management or staff at the end, but I still had to wonder if the company really maximized my call.  (I presume the register already tracked I was at the store that day, and drove-thru at 5:43 PM).  All in all, I’d say I spent about two minutes giving the store information it already had (or should have had), and about ten seconds about my experience.

While I appreciate a quick survey as much as the next guy, if a person is ready, willing, and able to complete a survey for you, stay mindful of the simple things like making every second count and asking questions that provide real value.

The premise is simple enough – people don’t always reveal what’s driving their decision making, and more importantly, people don’t always even know what is driving their decisions.  Since people either don’t want to or simply can’t communicate what’s driving their actions, we need to look and see what their brain is saying.

Recently, several us read Buyology as part of our firm’s book club.  After reading this book, it is easy to get excited about the technology and benefits it could provide (especially when presented in an oversimplified manner).

The technology does hold promise; however, it isn’t yet the magic bullet in market research.  The ARF’s NeuroStandards Collaboration Project is working to shed light on what is and what isn’t working in the field.  As the article reports, these technologies are raising as many questions as they’re answering.  The biggest challenge?  Figuring out what it is actually measuring and being able to pinpoint what is causing the reaction.

Neuroscience is quickly earning a place in the researcher’s toolbox, but no research tool is without its faults.  We must understand the limitations of our methods for us to get the most out of them.

While infographics are often creative masterpieces, the real power of infographics of course is to convey the impact of the data, not just the data itself.  I think too that there is a sense of discovery in a beautifully designed graphic – the more you look, the more you see and absorb.

So, how to get started?  Here are a few free websites that can create something other than a bar or pie chart:

• Many Eyes From traditional graphs to geographical maps, treemaps, and several more options.
• GapMinder While you can’t upload your own data, you can create animated graphics that show change over time based on world demographics.  (Here is an example…be sure to click play)
• Tableau A more complete solution (and in this case, not free, though there is a free trial) that allows more customization of your graphics.

Of course, step 1 is to determine how your audience will best absorb the information and gauge the appropriateness of the medium.  As you get more advanced, consider hiring a professional designer to assist in creating truly unique visuals.

Need some inspiration to get started?  Check out the site Information is Beautiful or one of David McCandless’s books.  Or do a Google Image Search for the term infographic.

What other means have you used to create interesting graphics?  Any exceptional infographics you’ve seen?  Please share in the comments.

(If you recently attended the Content Marketing Conference in Denver, Michael Pranikoff of PRNewswire also spoke to the importance of inforgraphics in crating compelling content.)

Remember to look beyond your own numbers to see the bigger story they tell.

Happy Friday.

Take for example a tracking study we did measuring recreation activities.  We saw a large drop in certain outdoor recreation between two studies – but the first one was conducted in spring and the other in winter.  If you measured the concern over the flu among the general public, the research during the time of the flu season would likely result in higher concern.

A recent example?  Toyota took a hard fall in J.D. Power and Associates’ Initial Quality Study this year.  While Toyota obviously has had a rough year, their ratings were likely hurt further by the timing of the survey which took place from February through May (The below graph shows Google Trends data on “Toyota Recall“, point C,  which peaked immediately before the survey).

While the goal is often to measure the impact of these events, other times it is not ideal to measure the impact of a short-lived event and you at least should be aware of the possible impact.  What to keep an eye out for?  Three big ones are:

• Seasonal and weather (e.g blizzards, heat waves, etc.)
• Political events
• News stories (yes, a catch-all, but even a brief news story on a topic can change the opinions of respondents)*

* You can use news monitoring services like Google Alerts to keep an eye on developing stories.

The Economist summed it up nicely in an article from last week’s edition.

When asked whether they supported a variety of issues, most people showed strong support.  However, when the downsides of those propositions were pointed out, support decreased significantly.  The strongest example was support for outfitting soldiers with the best equipment – 83% strongly supported.  When reminded that it would mean less spent on public services that they’d receive, support dropped to 46%.

Interestingly, alcohol bucked the trend.  When asked if the government should try to limit people’s boozing by taxing alcohol more 32% strongly agreed.  When reminded that it would mean they’d personally pay more for alcohol, there was no statistical change.  Presumably, people understood this issue well enough from the start and knew it meant they’d pay more, and therefore that was already calculated into their first response.

Many, if not all, issues can be framed in terms of gains or losses for those concerned.  Asking a question that only addresses one side of the coin is considered a research bias.  Pundits and politicians may rely on biased questioning to sway support for their beliefs, but researchers shouldn’t.

In the end, be careful what you’re asking.  People don’t make decisions in  a vacuum and your research shouldn’t be conducted in one either.