Language is simultaneously one of the most interesting and one of the most frustrating aspects of doing research with people. However, until we figure out a way to read people’s minds as they go about their daily lives, we have to rely on surveys to better understand their thoughts, attitudes, and beliefs. Unfortunately, language is very messy and, unlike other measurements such as centimeters or pounds, is constantly evolving.

There is already a ton of research on how slight changes to question wording can have a huge impact on how people respond to a question. This is one of the reasons why there is always some polling data to support a political position—by manipulating the question wording, you can increase the chances that people will be for or against certain policies. As Matt pointed out in a series this summer, there are even many different ways to ask about behavior, which can all lead to different types of responses. All the research, in addition to our own experience at Corona, highlights why word choice can be so important when designing a survey.

Recently, though, I’ve been thinking about the inverse of this: how word choice in naturally occurring language can lead to critical insights. For example, as the meaning of a wedding and marriage has changed in American culture, so too have the words associated with it. Understanding how people use language to describe their relationships can shed light onto how relationships are evolving in today’s culture. Similarly, some psychologists have been using big data to pull out language that is associated with different personality characteristics. The psychologists are finding that the language associated with certain personality traits is increasing their understanding of personality overall.

I think all of this hints at why open ended responses in a survey can be so enlightening. Although they are more difficult to analyze and sometimes add to survey costs, even just one or two well-crafted open ended questions in a survey can add a ton of information to your findings. And as technology makes it easier to pull out themes in open ended responses, the barriers to including these types of questions in a survey become smaller.

So the next time you are thinking about surveying supporters, customers, clients, etc., it might be worth thinking beyond multiple choice questions and including an open ended question or two. When people use their own language to answer a question, what insights might they share?