34 Reasons for and Types of Evaluation

In the basket metaphor, the weaving of an inner and outer wall together gives the basket strength. To fully realize the power of Indigenous evaluation, we weave program implementation together with program evaluation. We start the weaving by describing the story of the program—the story we plan to tell as a result of doing activities that lead to outcomes. As the program is implemented (creating the inner wall), evaluation (as the outer wall), captures the story that emerges allowing for reflecting, learning, and improving the program.

Joan LaFrance and Richard Nichols[1]

Reasons to Evaluate

It is imperative for us all to remember that people’s lives and well-being are at the core of what we are doing: infants, children, families, parents, elders, veterans, neighborhoods, business owners, etc.Two smiling adolescent boys. One has his arm around the other. One has sideways baseball hat and the other has sweatshirt hood.

You have a responsibility to determine if you are achieving your intended goals from your work, causing harm as a result of your work, and if you are actually improving the quality of life. You have an obligation to your stakeholders to figure this out, as identified in the generalist change-making process shown in the graphic below.

Graphic of three colorful rectangles that are all nested inside of each other. Largest green rectangle labeled Macro with smaller light blue rectangle labeled Mezzo inside of the green rectangle. A smaller darker blue rectangle labeled Micro is inside of the light blue rectangle. Two white boxes within the green Mezzo rectangle that read Addressing systemic and policy issues and Working with organizations, neighborhoods and communities. Two white boxes within the light blue Mezzo box that read Engaging groups and Working with teams. Two white boxes within darker blue Micro rectangle that read Assisting individuals and Working directly with families

Your evaluation will also demonstrate accountability to:

  • Consumers of your services
  • Staff and board of directors
  • Community stakeholders
  • Funders, members, and other investors in your services.

Types of Evaluation

There are a few different types of evaluations, but we are going to focus on Outcome Evaluation because it is a systematic measure of the impact of a program or strategy. Other common types of evaluations are process evaluations and needs assessments.

An outcome evaluation is an assessment of two primary phenomena:

  • The anticipated impact, based upon our best understanding of the needs of the population you are focused on.
  • The unanticipated impact, based upon our best understanding of the changes that occurred as a result of your work.

Using and Sharing Evaluation Results

The results of your outcome evaluation need to be shared with the stakeholders who are invested in your work:  staff, clients, people directly impacted by the strategies you are engaged in, board of directors or advisory group, and funders. The results of the evaluation will inform them of the results of your work and will often assist you in attracting new resources and support (economic, human, political, etc.).

Necessity for Culturally Relevant Evaluation

Evaluation of historically marginalized groups has often been invasive, intrusive, harmful, manipulative, judgmental, and exploitative. [2]  It makes sense that there is significant mistrust in people conducting or requiring evaluations, along with their motivations. Therefore, it is imperative that the evaluation is crafted carefully to ensure that the results won’t further exploit the group or point out deficits rather than strengths.

Culturally relevant evaluation recognizes that “demographic, sociopolitical, and contextual dimensions, locations, perspectives, and characteristics of culture matter fundamentally in evaluation” (p. 283).[3]  Culturally relevant evaluation requires an understanding of historical and current context and should be grounded in the wisdom and values of the communities it is intended to serve.

Culturally relevant evaluation also needs to be designed by, with, and for the historically marginalized group. The evaluation methods, specific data collection tools, and implications should all be closely considered. The results of the evaluation should focus on gathering knowledge that will result in improvements in people’s lives that they are seeking.

Resistance to Evaluation

As mentioned, you have a responsibility to the stakeholders of your work to evaluate the effectiveness. This hopefully seems like common sense but I am amazed at how frequently there is resistance to this phase of community change work. The resistance seems to be primarily about:

  • fear of what will be learned during the evaluation
  • concern that the feedback will be difficult to address
  • lack of staff capacity to carry out evaluation

What if the evaluation demands significant change from the organization? My initial response is that this could be great! Receiving constructive feedback on the need for an organization to change can be intimidating but the outcome is likely to be very positive and, of course, helpful. That is the primary point of an evaluation. How effective are we? How can we be more effective?

Be open to the feedback you will receive and recognize that the stronger your evaluation methods, the more meaningful the results will be, and the stronger your program will become as a result.

 

Evaluation Resources

Amherst H. Wilder Research[4]

The Wilder Research division of the foundation provides extensive evaluation reports on numerous human service topics (early childhood, youth, homelessness, etc.) as well as useful evaluation tools. 

Center for Culturally Responsive Evaluation and Assessment [5]

The Center is a part of the University of Illinois Urbana-Champaign and is “grounded in the need for designing and conducting evaluations and assessments that embody cognitive, cultural, and interdisciplinary diversity”. 

Community Toolbox

[6]

The toolbox is created and supported by the Center for Community Health and Development at the University of Kansas to “help taking action, teaching, and training others in organizing for community development.”  

Urban Institute’s Outcome Indicator’s Project[7]

Provides ideas for outcomes and indicators for a number of common human service sectors. One of their goals is to establish common measures for nonprofit performance.

  1. LaFrance, J. and Nichols, R. (2009). Indigenous framework for evaluation: Creating our story. American Indian Higher Education Consortium. https://portalcentral.aihec.org/Indigeval/Pages/Document-Collections.aspx. ↵
  2. LaFrance, J.; Nichols, R. (2010). Reframing evaluation: Defining an Indigenous evaluation framework. Canadian Journal of Program Evaluation, v23 n2 p13-31. ↵
  3. Hood, S.; Hopson, R.K.; Kirkhart, K.E. (2015). Culturally responsive evaluation; Newcomer, K.E.; Hatry, H.P.; Sholey, J.S. In Handbook of Practical Program Evaluation (4th Edition). Wiley Publishing. ↵
  4. http://www.wilder.org/research.0.html ↵
  5. https://crea.education.illinois.edu/about ↵
  6. Center for Community Health and Development. (n.d.). Chapter 36, Section 5. Developing an Evaluation Plan. University of Kansas. Retrieved November 3, 2023, from the Community Tool Box: https://ctb.ku.edu/en/table-of-contents/evaluate/evaluation/evaluation-plan/main ↵
  7. https://www.urban.org/policy-centers/cross-center-initiatives/performance-management-measurement/projects/nonprofit-organizations/projects-focused-nonprofit-organizations/outcome-indicators-project ↵

License

Macro Practice for Community and Organizational Change Copyright © by Lynn Amerman Goerdt. All Rights Reserved.