Some Evaluation Principles

  1. There is no “right” or “wrong” way to evaluate communications campaigns. There should be both recognition and acceptance of the fact that different evaluation needs and capabilities require different evaluation designs. The focus and methods should fit the information needs and available resources of stakeholders in the communications effort.
  2. Assessing whether a campaign caused its intended impact is often important ” and that is the activity funders tend to focus on ” but evaluation for purposes of learning and continuous improvement is also important. At the same time, all sides should recognize that leverage to convince sponsors to invest in campaigns will be enhanced by evaluations that assess causation (which often require higher evaluation budgets).
  3. Evaluations, like communications campaigns, need to identify up front their purpose and intended audiences. For example, is the evaluation intended to measure the impact of a campaign? Or is it to provide feedback so the campaign can learn over time from experience? Also, is the target audience for the evaluation the sponsoring foundation, the nonprofit(s) implementing the campaign, or both?
  4. It is best to design the evaluation early and in conjunction with the campaign. This will maximize opportunities to use the evaluation for both learning and impact assessment.
  5. Campaign staff members should participate whenever possible in the evaluation’s design as well as its implementation. Campaigners and evaluators both need to understand the existing challenges and opportunities. For example, some campaigns seek to change public opinion and then induce action by policy-makers. Other campaigns build upon existing favorable opinion and then mobilize people to a particular action. Finally, some campaigns or initiatives simply seek to provide more information about a particular issue. Obviously, campaigns to change or mobilize public opinion are more difficult from the start.
  6. Evaluation should push for methodological rigor and innovation whenever possible. It should also acknowledge that more than one evaluation approach can capture useful information.
  7. Different evaluation designs have different interpretive boundaries. It is important to understand those boundaries and avoid the temptation to make broad claims of success based on limited data or designs that do not warrant such claims.
  8. It is important to be realistic about impact. In commercial marketing campaigns, attitude improvements of one-tenth of one percent are deemed important because they can represent millions of dollars. But sometimes funders of communications campaigns want to see attitudinal shifts of 10 to 30 percent. In response, nonprofits sometimes make promises to funders that they cannot possibly fulfill.
  9. Sometimes simple things like having a good press list or establishing ongoing professional relationships with key reporters are the most significant measures of success, especially for locality-specific or small-budget efforts.
  10. “Values” are important to both campaigns and their evaluation. Typically, nonprofit communications efforts put forth information to achieve either behavioral or societal change. However, widely held and deeply entrenched values can often trump useful information (for example, values about the meaning of “family,” “community,” “independence” or “self-sufficiency”). Successful communications campaigns must acknowledge the “values vs. information” dichotomy, and the evaluation must take this into account when judging impact.
  11. Evaluation should be based on sound (and where possible research-based) theory for predicting how the campaign will achieve social change or provide the public with understanding of the issue.
  12. Evaluation can respond to hard-to-answer questions about the value and effectiveness of communication campaigns (for example, whether information alone can lead to behavioral change or whether attention to the social and policy context is also a necessary ingredient; and whether media advocacy can contribute).