Flexible Systematic Approaches Build Evaluation Capacity for Program Staff

By Celeste CarmichaelProgram Development and Accountability Specialist, Cornell Cooperative Extension Administration

“Systematic approaches with flexibility built in to meet local needs”—that is how I would describe ideal program development resources for Extension programs.  Most of our Extension Educators are busy with field responsibilities.  In order to assist with implementation of best practices, resources need to be applicable to broad goals, easy to find, use, and adapt.

For Cornell Cooperative Extension (CCE), Qualtrics has proven to be a systematic yet flexible resource for supporting needs assessments and program evaluations.  There are other options for survey development, but Qualtrics is supported at Cornell for faculty, staff, students and CCE educators.  We have also found Qualtrics to be a good match for any job from very simple to highly complex surveys, and it provides substantive data protection for respondents through secure servers.  One of the other features that makes Qualtrics very attractive is the ability to create a library of sample Cooperative Extension evaluation forms and questions to help Extension Educators get started with survey development. 

Staff have reported that because of time limitations there are instances when evaluation measure development is done in haste just prior to a face to face event.  When created in a hurry questions might not reflect the intended program outcomes and the resulting responses may not be as useful as they could have been otherwise.  Staff also report that survey development can be frozen by simple details that might feel overwhelming when having to develop a survey in short order.  Challenges noted include:

  • Getting the right survey look and feel
  • Developing questions and question order
  • Pilot testing questions
  • Understanding the overall evaluation questions for the program

In order to give more common programs a leg up on building evaluation forms, draft surveys that ask questions connected to how programs reach statewide outcomes are being developed and shared in the Qualtrics Cornell Cooperative Extension survey library.   The draft surveys have a Cooperative Extension header and footer, an appropriate question logic for typical programs, questions and blocks of questions that have been piloted, and questions related to behavioral aspirations and outcomes.  Surveys from the library can be saved into a user’s personal library and adapted as needed.  Additionally survey questions can be individually found in the question bank library.

On using the libraries:

CCE Qualtrics

Qualtrics users will note that “Library” is a tab in the Qualtrics menu where surveys can be saved into a user’s personal account and adapted.  The data collected belong with a user’s personal account and not the library.  A benefit to Qualtrics is the online documentation about using the features including libraries.

Similar options for a systematic approach exist beyond Qualtrics, of course.  The idea is simple—provide a starting point to allow all staff a baseline set of questions to collect data around programs.  When the starting point is adaptable—it builds capacity for the program practitioner to grow into the evaluator, adapting the questions to meet to local needs.  Where Qualtrics or another survey tool is not available, a virtual folder of adaptable documents can help local educators who are doing similar types of programs build around common program outcomes and indicators.