Have you ever taken a survey that asked you to rank a list?  Maybe it was just a few items.  Maybe it was many more.  I was recently asked to rank a LONG list of attributes on a survey which quickly became an exercise in futility.  When ranking nearly 20 items, can you really decipher between your 9th and 10th rank, let along your 17th and 18th rankings?

When asking respondents to rank items, it is important to keep in mind how many items they can meaningfully keep track of when ranking.

To keep things manageable, there are several methods you can employ:

  • Ask the respondent to only rank their top choices (this is probably all you really are concerned with anyway); likewise, if you really want to know, you can ask them to rank their bottom choices (if you do this, remove the choices they already listed as their top choices when later asking their bottom choices).
  • Ask respondents to first check all that apply from a list.  For example, “Which attributes do you consider when purchasing X?”  Then have them only rank those attributes that they care about.
  • Break the list up into logical segments to help respondents focus their thinking.  Don’t ask everything at once.  For example, for an automobile, you can sort the list by entertainment system, safety features and so on.
  • Randomly select list items to create a few smaller lists for ranking.  Lists can even be divided among participants so not every person ranks every item.

Creating ranking questions that are easy to decipher and answer from the respondents’ point of view creates a better survey that leads to better, more meaningful data.  For an analysis of the implications of using various ranking methods, check out this article in this month’s issue of Quirk’s Marketing Research Review (free registration required).