Haunted by Old Survey Questions

“As the Corona team bravely entered the haunted project file, they heard a strange sound. They quickly turned to the right to see analyses covered in cobwebs. They shuddered. Suddenly a weird shadow crossed their faces. As they looked up, they could barely make out what it was…a report? An old invoice? No, it couldn’t be! But it was. It was the original survey, completely unchanged since 1997…..AAAAHHHH!”

The above might be a slight exaggeration. If anything, we do try to keep our screaming to a minimum at work. However, I do think that often organizations can become “haunted” by old survey questions that they cannot seem to escape.

Obviously, there can be value in tracking certain questions over time. This is especially true if you want to determine the effect of certain programming or events. However, we sometimes see organizations that are afraid to change or remove any questions at all year over year, even if it is unclear whether all the questions are useful anymore.

As pointed out previously, it is a good idea to think through the goals of your research before you start a project. If you have not seen any changes in a measure over time and/or you did not do anything intentionally to move a measure in the past year, you might consider updating your survey. And there are a ton of different ways to do this.

You can ask key measures every other year, and in between, ask totally new questions that allow you to dig deeper into a certain issue, test new marketing concepts, etc. You can ask a smaller subset of only the critical key measures every year but rotate out a subset of new questions. You can ask the same questions, but maybe ask them of a new audience . You can ask the same questions, but try out a new type of analysis. For example, instead of just reporting 55% of donors strongly believe X and 31% strongly believe Y, we could look at which belief predicts the amount that donors give. We might find out that belief X more strongly predicts the donation amount than belief Y and that might change your entire marketing campaign.

Again, tracking questions over time can definitely be important. But it might be worth considering whether you are asking certain questions because they are of use or if you are doing so just because you have always asked these questions in your survey. If the latter is your reason, it might be time to rethink your survey goals and questions.

Don’t be haunted by old survey questions. Let Corona help you clear out those ghosts and design a survey tailored to meet your current ghouls…er, goals.