Blog / Evaluation

New Frameworks for Evaluating Impact

by Teri Behrens
New Frameworks for Evaluating Impact
Drawing on our local, national, and international research, tool development, training, and data work, leaders from the Johnson Center have identified 11 key trends that will impact the work of both grantmakers or nonprofit leaders in the months and years to come. In this piece, we explore one of those trends in depth.

Download and read the full report, featuring all 11 trends, here.

Front cover of the “11 Trends in Philanthropy for 2018” reportAs philanthropy has evolved, particularly over the past 20 years, the field of evaluation has evolved along with it. Perhaps the most significant change has been the shift towards strategic philanthropy and its attendant focus on achieving explicit outcomes.

Simultaneously, we have seen a growing recognition that significant systems change is also needed to affect the end result. Systems change means altering how resources are allocated and decisions are made, and how relationships among people and organizations evolve to alter that final outcome. A clear goal — for example, increased graduation rates, decreased maternal death rates, or higher employment rates — is an important aspect of strategic philanthropy, but to achieve these ambitious outcomes requires a strategy that changes large-scale systems like healthcare and education.

Yet, such strategies often face significant challenges. Large public systems are comprised of many moving parts, with a variety of institutions and players who can influence outcomes. Reshaping those structures often requires policy change, which itself is influenced by the political climate. And in many cases, systems also “push back”: those who benefit from the current arrangement have no incentive to change.

This is why we are continuing to see an increased focus on evaluation as a necessary part of the systems change strategy. Michael Patton, one of the most influential people in the field of evaluation, wrote the first book on the use of developmental evaluation to provide feedback on complex systems change work (Patton, 2011). Since its publication, the use of developmental evaluation has grown significantly. Developmental evaluation is an approach that focuses on meeting the information needs of people who are seeking to be innovative in addressing highly complex situations — situations in which it is not known how to achieve desired results, where there is disagreement about how to proceed, and where cause and effect are not related in a linear fashion. More recently, the Collective Impact Forum has encouraged the use of developmental evaluation in collective impact efforts (Holmes, 2014).

A related shift in the evaluation world is the adoption of Emergent Learning tools to enable insights into systems. Emergent Learning frameworks provide tools to capture lessons learned and support ongoing reflection during a change process. Examples include tools such as Before Action Reviews, After Action Reviews, and Emergent Learning Tables as ways to gather information to promote not only individual foundation learning but learning on the part of networks and communities (Darling, 2016).

Even for funders who have not adopted a strategic philanthropy lens, there are new approaches to evaluation that can be valuable. Patton has just introduced the concept of principles-based evaluation, a framework for assessing how closely a donor or funder is adhering to the principles they say are driving their philanthropy (Patton, 2017). This may be a particularly helpful approach to evaluation for family foundations whose primary concern is honoring donor intent rather than impact.

As evaluation continues to grow as a profession, we expect to see continuing innovations in practice to match innovations in philanthropic giving approaches.

Teresa (Teri) Behrens, Ph.D.
Teresa (Teri) Behrens, Ph.D.
Executive Director, 2018–2022
Teri Behrens joined the Dorothy A. Johnson Center for Philanthropy in 2009, serving in several roles throughout her tenure at the organization. She was the executive director of the Johnson Center from 2018 until her retirement in 2022. Teri is...
References

Darling, M., Guber, H., Smith, J. and Stiles, J. (2016). Emergent learning: A framework for whole-system strategy, learning and adaptation. The Foundation Review, 8(1), 59–73. Retrieved from https://doi.org/10.9707/1944-5660.1284.

Holmes, K. (2014, July 13). Developmental evaluation in the context of collective impact. [Blog post]. Retrieved November 28, 2017 from http://collectiveimpactforum.org/blogs/18451/developmental-evaluation-context-collective-impact-part-1.

Patton, M. Q. (2011). Developmental evaluation: Applying complexity concepts to enhance innovation and use. New York: The Guilford Press.

Patton, M. Q. (2017). Principles-focused evaluation: The guide. New York: The Guilford Press.