By Scott Chazdon, University of Minnesota Extension
In 2014, my evaluation colleagues and I began gathering stories about the impact Extension programs have on individuals and communities. Based initially on the Most Significant Change method (Dart & Davies, 2003), the project aimed to promote ongoing dialogue about Extension programming and help staff and stakeholders explore the changes that occur because of Extension programming.
The Most Significant Change methodology is a participatory, story-based approach to evaluate Extension’s impact on participants and the public. We piloted the project across programming in the central region of Minnesota—15 counties that include the Twin Cities metropolitan area and surrounding suburban and rural counties.
The project uses a dialogical process; each submitted story is reviewed against a rubric. The project drew on Brinkerhoff’s Success Case Method (2002) to show evidence of program impact through rich and verifiable descriptions. This method does not replace other evaluation efforts, but stories can be powerful communication tools, especially when combined with other evaluation methods.
As a result of the central region project, we strengthened the rubric to ensure that public value is deeply engrained in impact narratives. The rubric is based on our learning from the project as well as my own work on Evaluation for Public Value (Chazdon & Paine, 2014).
The elements of the new rubric are:
- Story demonstrates behavior changes that resulted from Extension programming. A strong narrative must incorporate evidence that the program has achieved its intended behavioral outcomes.
- Story demonstrates the trust and respect Extension has established with key audiences. Extension has built trust through long-standing relationships with key stakeholders. These aspects of programs are often overlooked and need to be incorporated into impact narratives.
- Story demonstrates Extension programs, staff, and volunteers meeting the needs of underrepresented populations. Part of the public value of a program is determining the audiences that most need the programming. These should be audiences that cannot otherwise receive the content through private sources.
- Story demonstrates Extension adapting to meet the changing needs of its key audiences. Public value also resides in staying current with traditional Extension audiences (farmers, youth, and conservation professionals) by addressing changing needs. This can include changing content due to new economic, environmental, or political contexts.
- Story demonstrates ways that Extension leverages organizations or partnerships to expand the delivery of research and education beyond initial program participants. Public value resides in the way Extension leverages its partnerships and collaborations to reach beyond its direct participants.
- Story demonstrates ways that Extension programming led to positive social, economic, environmental, cultural, health, or civic effects for public-serving organizations or communities. Public value resides in the “so what?”—the positive things that happen in families, organizations, and communities that can be attributed at least, in part, to Extension education. It is challenging to quantify these types of impacts, but systematic qualitative methods, such as Ripple Effects Mapping, can be very useful to document these effects.
These six aspects of public value are easy to teach and provide a useful framework for thinking about the public value of Extension education.
As evaluators move into public value narratives, we must tread carefully with the communications staff in our organizations. Typically, they write impact stories, and they do it well! But they may not employ evaluative frameworks, such as this rubric, in doing so. To distinguish our work, we describe it as public value “narratives” rather than “stories.”
Moving forward, we continue to develop tools and training resources to support the writing of impact narratives. We have developed the following quick guide to composing a narrative:
- What was the presenting issue?
- Who was the target audience, and why?
- Why Extension? Credible information, research-based, trusted resource?
- What changes in behavior or action occurred as a result of the program? Include evaluation evidence.
- What were the broader impacts? Evidence of spillover, leveraging, ripples, return on investment, benefit-cost analysis?
We are also working to train new Extension educators in this narrative writing process. We hope to have a public value narrative contest as part of our annual professional development conference and use narratives for our reporting to the national land grant impacts database.
I am happy to share more information on our process and can be reached at email@example.com.
Brinkerhoff, R. O. (2002). The success case method. San Francisco: Berrett-Koehler.
Chazdon, S.A. & Paine, N. (2014). Evaluating for Public Value: Clarifying the Relationship Between Public Value and Program Evaluation. Journal of Human Sciences and Extension, 2(2), 100-119. Retrieved from http://media.wix.com/ugd/c8fe6e_8b2458db408640e580cfbeb5f8c339ca.pdf.
Dart, J. & Davies, R. (2003). A dialogical, story-based evaluation tool: the Most Significant Change Technique. American Journal of Evaluation, 21(2), 137-155.
Franz, N. (2013). Improving Extension programs: Putting public value stories and statements to work. Journal of Extension, 51(3). Retrieved from http://www.joe.org/joe/2013june/tt1.php
Kalambokidis, L. (2011). Spreading the word about Extension’s public value. Journal of Extension, 49(2). Retrieved from http://www.joe.org/joe/2011april/a1.php.
Knowledge Sharing Toolkit. (2014). Most Significant Change. Retrieved from http://www.kstoolkit.org/Most+Significant+Change