The field of quantitative research is often perceived as strictly numeric and free from bias—we acquire data points through processes such as surveys or census reports, and we use the ones and zeros to decipher some objective meaning about the topic of interest. In reality, quantitative data is not objective, and there are many considerations we make during our research process to ensure that the data reflect the population accurately, and that DEI principles are still at play.

Sampling Methods

At the most basic level, we need to interrogate our data sources. What is the best way to gather data about a given population? Are people more likely to respond to certain kinds of research methods? Online survey responses depend on technology access, so trying to reach a remote population with limited internet access would be challenging via an online survey. To ensure this population is represented in the final data, we might instead opt for a paper survey. The opposite might be true for reaching a young urban population.

There are also tactics that we can use to boost the likelihood of a representative sample by oversampling areas or communities with expected lower response rates. We know, both through our own experiences and through published studies, that certain groups are more likely to respond to surveys. If, for example, we were to conduct a mail survey in a city that was notably divided between an affluent, highly educated community and a lower income, working-class community, we might opt to send more surveys to the working-class neighborhoods to potentially boost the size of a respondent pool to accurately match the population.

Survey Questions

The wording of questions we ask can ensure both diversity in our data and equity in our process. In our work, we attempt to ensure that our survey instruments are worded at a fourth-grade comprehension level. Additionally, we aim to be sensitive to diversity within populations by ensuring that participants feel represented in demographic questions. Typically, we give people options to self-describe when asking demographic questions. For instance, instead of asking a question like:

What is your gender?

  1. Male
  2. Female

We might instead say:

Are you…?

  1. Male
  2. Female
  3. Prefer to self-describe:_________

The purpose of this is two-fold. First, it gives people the space to specify how they identify in their own words. Second, we do not frame male and female as a strict binary, but instead make space for different ways to identify without suggesting that anything other than male or female is fundamentally “other.” An open-end option opens the door for people to use their own terms and words should they feel limited by only male and female categories.

Data Analysis

Finally, another tool we use to ensure equity in quantitative research is weighting data. Here is an example: If we are surveying residents of a town that we know to be diverse in educational attainments, we might look at census data to see what proportion of residents have certain levels of education. We might find that a quarter of residents have a high school diploma or GED, but that most survey respondents have a bachelor’s degree or higher. We can use statistical weights to boost the voices of survey respondents with a high school education to make the data better match the population as a whole. Though the process is imperfect, it allows us to correct for sampling error and ensure that our data is more representative of the population we are serving.

These are only three of many tactics we employ to ensure that our quantitative data is representative, inclusive, diverse, and equitable.  Of course, there are limitations to our attempts at these outcomes.  In order to accurately and fairly represent a population, we need to have knowledge of this population. Representative data does not appear by accident, it is predicated on a knowledge of and interest in the communities we are collecting data from. Care must be taken from the outset to try to understand the population as a whole, in order to tailor sampling methods, survey questions, and analysis techniques that can produce inclusive data sets.

Quantitative research is both a science and an art, and although it is ostensibly data driven, we can and should work to ensure that it yields results that are representative of all people and all voices.