What we talk about when we talk about impact
Very often, we hear the words "evaluation" and "impact" used interchangeably. Impact evaluation is a type of evaluation, but it is not the only one. Impact evaluation looks to determine the changes that can be directly attributable to a program or intervention. And as we all know, in the complicated landscape of the kinds of social change work that we are typically looking to evaluate, it is very difficult – if not impossible -- to attribute behavioral, attitudinal, or other outcomes directly to a particular program.
What follows is an overview of evaluation models that are frequently referenced in evaluation literature. This list is not meant to be exhaustive. Rather, we hope it will offer a starting point to think about the different approaches you can take to evaluate your program, strategy, or intervention. This list is adapted from various sources, which are referenced at the end of this post.
Formative Evaluation or Needs Assessment Evaluation
When you might use it• During development of a new programWhat it can show• Identifies areas for improvementWhy it can be useful• Allows program to be modified before full implementation begins
Summative Evaluation or Outcomes Evaluation
When you might use it• After program implementation has begun• At pre-determined intervals of an existing program • At the conclusion of a program What it can show• Degree to which program is having effect on knowledge, attitudes, or behaviors of target populationWhy it can be useful• Effectiveness of program against its stated objectives (at particular milestones)
Process / Monitoring Evaluation
When you might use it• When program implementation begins• During operation of existing programWhat it can show• Extent to which program is being implemented as designedWhy it can be useful• Provides early warning if things are not progressing as planned• Distinguishes program design (theory of change, logic model) from implementation
When you might use it• During implementation of a particularly complex or innovative program• In conditions of high uncertaintyWhat it can show• Emergence – patterns that emerge from interactions from groups of participants• Dynamic adaptations – extent to which program is affected by interactions between and among participantsWhy it can be useful• Can incorporate "nontraditional" concepts such as non-linearity, uncertainty, rapid cycling, vision-driven (rather that metrics-driven)
When you might use it• To support a community in building evaluation capacityWhat it can show• Community knowledge and assetsWhy it can be useful• Is designed for inclusion, participation, increased capacity, and community ownership