I attended the American Association of Public Opinion Research (AAPOR) annual conference a few weeks ago, and I was happy to see that many sessions delved into the details of the push-to-web survey approach. Presentations described how many large and long-running surveys have transitioned from traditional phone modes to address-based push-to-web modes.

Photo by Christin Hume on Unsplash

Potentially, push-to-web surveys have a lot of advantages, including saving time, saving money, and even reducing bias. However, this survey approach requires planning and nuance to execute correctly.  Without careful tending, a push-to-web survey can turn sour and result in inferior data at a greater cost, something we all want to avoid.

This blog will discuss push-to-web surveys by exploring a few questions:

  1. What is a push-to-web (P2W) survey? 
  2. When should you consider a P2W? and
  3. What are P2W best practices?

What is a Push-To-Web Survey?

A push-to-web survey contacts a sample of your population through one medium and then asks them to participate in the survey online. The most common P2W approach is to mail potential respondents a survey invitation letter that asks them to access the survey through a secure web link. The link may be unique to that mailing address, or the respondent may need to enter a personal identification number included on the survey invitation.  Either way, the survey link is secure and controlled.  Typically, a series of follow-up communications, such as mailing reminder postcards or letters, further encourages participation in the survey. In most cases, a hardcopy questionnaire should eventually be mailed to everyone who has not yet responded to the initial invitation.

You might be asking, if people can participate in my study online, why should I mail a hardcopy questionnaire? Although it’s expensive and time consuming, mailing the hardcopy is critical because lots of research has shown that people who reply online skew from the broader population that the survey was meant to represent. Online respondents tend to skew towards higher incomes and higher levels of education, making them less representative of the population overall. They also tend to be less racially and ethnically diverse.

Obviously, a P2W survey cannot be completed by people without access to computers, smartphones, or the internet. And among those with access, unfamiliarity with technology and negative attitudes towards online participation pose additional barriers.  While this is a small and shrinking proportion of most populations, the opinions, behaviors, and demographics of people who don’t respond online tend to differ notably from people who do respond online, and therefore, their exclusion would bias survey results.

When to consider a Push-To-Web Survey?

Although the idea is attractive, push-to-web surveys are only advisable in certain situations. Think 10 and 10.  The benefits of a push-to-web approach can outweigh the drawbacks when a large mailing is planned (e.g., a sample of 10,000+) and/or when the questionnaire is very long (e.g., 10+ pages). A P2W approach shaves costs on big sample projects by reducing the number of hardcopy questionnaires mailed, and it can decrease data capture time (and budgets) because respondents are entering their data online. In general, it is safe to assume that small sample projects and shorter questionnaires don’t benefit much from a P2W approach because they don’t reap many of the mailing and data capture efficiencies.  Since P2W can increase survey project complexity and length (and in some cases decrease response rates compared to a traditional mail survey), they should be executed with careful planning.

If you’d like to see if a P2W is a good option for your next survey project, give us a call. We love talking about this stuff.