Getting to Impact: Beyond Population Level Indicators

Oct. 6, 2015 in Articles

The following has been reprinted from the September issue of Engage! magazine.

Collective Impact has increased our focus on moving the needle on population level changes. John Kania, Managing Partner of FSG Social Impact Consultants and author of many articles on collective impact has called this 'positive and consistent progress at scale'. But are population level indicators the only ones that we need to pay attention to when moving toward collective impact?

A number of resources stress the importance of also focusing on systems level indicators as a means of tracking progress and impact. ORSImpact.com presented a working paper at the 2014 American Evaluation Conference called,Impact, Influence, Leverage and Learning (I2L2) Outcomes Framework which is useful for collective impact initiatives to mine systems change and policy influence outcomes in four key indicator areas:

  • Impact outcomes refer to changes in people's lives and are often referred to as population level outcomes. Impact outcomes can also include changes in attitudes, knowledge, behavior, skills, perceptions, beliefs, practices, relationships or conditions.
  • Influence outcomes are changes in system-level practices between organizations, networks, partnerships, policies, practices and community norms. An example of an influence change is that a community focuses increased services in a targeted neighbourhood.
  • Leverage outcomes include an increased commitment of resources, including non-monetary and in-kind resources focused on the change being targeted.
  • Learning outcomes are about field-building and advancing knowledge.

The I2L2 Outcomes Framework provides outcomes statements for each of the four key indicator areas which are particularly useful for those engaging in collective impact efforts focused on policy and systems level changes. The resource also provides practical case studies where the I2L2 Outcomes Framework has been used to articulate the theory of change for initiatives and track a defined set of relevant outcomes.

Similarly, Grantmakers for Effective Organizations has recently developed a resource called Evaluating Community Change: A Framework for Grantmakers which also explores a variety of systems change level indicators. While written from a grantmaker or philanthropist perspective, GEO has identified indicators in eight different areas of community or place-based change:

  • Political, economic and cultural context
  • Baseline conditions
  • Funder levers of change
  • Immediate program outcomes and potential to scale
  • Capacity outcome and changes to systems condition
  • System-level change
  • Scale and sustainability
  • Population-level impacts or outcomes

The GEO framework is designed to help grantmakers determine baseline and community capacity as well as to identify a set of indicators or measures that will represent progress in community change efforts.

Both resources are useful references for communities engaged in collective impact efforts focused at policy and systems change. They complement the Guide to Evaluating Collective Impact developed by FSG Social Impact Consultants and are most useful in helping community change efforts identify the multiple outcome measures embedded in these processes.

Learn More:

Credit: Liz Weaver