As an evaluator, it’s really easy to draft a thousand lines of questioning to capture every nuance in every conceivable outcome that might result from a particular program.  I want to know everything, and I want to understand everything deeply, and so do the organizations I work with.

DataYet collecting too much data burdens both program participants and the evaluation team, (and in some cases can change how participants respond to particular items – see details here). The hard part of evaluation work is distilling the goals down to their essence and choosing the highest impact measures.

This is especially important in situations where frequent measures are needed because services are adapting to meet changing needs.  A recent NPR story illustrates this, also showing how technology makes continuous monitoring possible.  The story points out that organizations providing aid in disasters often decide what to do at the outset, and then don’t have much information about how it went until they evaluate it formally after the program ends.  But in a few recent cases, including the international response to the Ebola epidemic, a group has implemented a short survey administered weekly by text messages to cell phones of residents in affected communities.  The survey measures only around five key objectives of the Ebola response (e.g., impacts on travel, trust in communications, etc.), and because it is implemented weekly, it provides ongoing updates on progress toward the desired objectives.  The data helps steer the program activities to best meet the most pressing needs.

It is both useful and inspiring to learn about thoughtful, creative solutions that other evaluators have developed to help organizations reach their goals.  We’re always looking to learn and grow so that we can best serve organizations who are themselves continuously improving as they work to make the world a better place.