38 Planning the Evaluation
Reasons to Evaluate
It is imperative for us all to remember that people’s lives and well-being are at the core of what we are doing: infants, children, families, parents, elders, veterans, neighborhoods, business owners, etc.
You have a responsibility to determine if you are achieving your intended goals from your work, causing harm as a result of your work, and if you are actually improving the quality of life. You have an obligation to your stakeholders to figure this out, as identified in the generalist change-making process shown in the graphic below.
Your evaluation will also demonstrate accountability to:
- Consumers of your services
- Staff and board of directors
- Community stakeholders
- Funders, members, and other investors in your services.
Types of Evaluation
There are a few different types of evaluations, but we are going to focus on Outcome Evaluation because it is a systematic measure of the impact of a program or strategy. Other common types of evaluations are process evaluations and needs assessments.
An outcome evaluation is an assessment of two primary phenomena:
- The anticipated impact, based upon our best understanding of the needs of the population you are focused on.
- The unanticipated impact, based upon our best understanding of the changes that occurred as a result of your work.
Using and Sharing Evaluation Results
The results of your outcome evaluation need to be shared with the stakeholders who are invested in your work: staff, clients, people directly impacted by the strategies you are engaged in, board of directors or advisory group, and funders. The results of the evaluation will inform them of the results of your work and will often assist you in attracting new resources and support (economic, human, political, etc.).
Necessity for Culturally Relevant Evaluation
Evaluation of historically marginalized groups has often been invasive, intrusive, harmful, manipulative, judgmental, and exploitative. [1] It makes sense that there is significant mistrust in people conducting or requiring evaluations, along with their motivations. Therefore, it is imperative that the evaluation is crafted carefully to ensure that the results won’t further exploit the group or point out deficits rather than strengths.
Culturally relevant evaluation recognizes that “demographic, sociopolitical, and contextual dimensions, locations, perspectives, and characteristics of culture matter fundamentally in evaluation” (p. 283).[2] Culturally relevant evaluation requires an understanding of historical and current context and should be grounded in the wisdom and values of the communities it is intended to serve.
Culturally relevant evaluation also needs to be designed by, with, and for the historically marginalized group. The evaluation methods, specific data collection tools, and implications should all be closely considered. The results of the evaluation should focus on gathering knowledge that will result in improvements in people’s lives that they are seeking.
Resistance to Evaluation
As mentioned, you have a responsibility to the stakeholders of your work to evaluate the effectiveness. This hopefully seems like common sense but I am amazed at how frequently there is resistance to this phase of community change work. The resistance seems to be primarily about:
- fear of what will be learned during the evaluation
- concern that the feedback will be difficult to address
- lack of staff capacity to carry out evaluation
What if the evaluation demands significant change from the organization? My initial response is that this could be great! Receiving constructive feedback on the need for an organization to change can be intimidating but the outcome is likely to be very positive and, of course, helpful. That is the primary point of an evaluation. How effective are we? How can we be more effective?
Be open to the feedback you will receive and recognize that the stronger your evaluation methods, the more meaningful the results will be, and the stronger your program will become as a result.
Evaluation Resources
Amherst H. Wilder Research[3]
The Wilder Research division of the foundation provides extensive evaluation reports on numerous human service topics (early childhood, youth, homelessness, etc.) as well as useful evaluation tools.
Center for Culturally Responsive Evaluation and Assessment [4]
The Center is a part of the University of Illinois Urbana-Champaign and is “grounded in the need for designing and conducting evaluations and assessments that embody cognitive, cultural, and interdisciplinary diversity”.
Community Toolbox
The toolbox is created and supported by the Center for Community Health and Development at the University of Kansas to “help taking action, teaching, and training others in organizing for community development.”
Urban Institute’s Outcome Indicator’s Project[6]
Provides ideas for outcomes and indicators for a number of common human service sectors. One of their goals is to establish common measures for nonprofit performance.
- LaFrance, J.; Nichols, R. (2010). Reframing evaluation: Defining an Indigenous evaluation framework. Canadian Journal of Program Evaluation, v23 n2 p13-31. ↵
- Hood, S.; Hopson, R.K.; Kirkhart, K.E. (2015). Culturally responsive evaluation; Newcomer, K.E.; Hatry, H.P.; Sholey, J.S. In Handbook of Practical Program Evaluation (4th Edition). Wiley Publishing. ↵
- http://www.wilder.org/research.0.html ↵
- https://crea.education.illinois.edu/about ↵
- Center for Community Health and Development. (n.d.). Chapter 36, Section 5. Developing an Evaluation Plan. University of Kansas. Retrieved November 3, 2023, from the Community Tool Box: https://ctb.ku.edu/en/table-of-contents/evaluate/evaluation/evaluation-plan/main ↵
- https://www.urban.org/policy-centers/cross-center-initiatives/performance-management-measurement/projects/nonprofit-organizations/projects-focused-nonprofit-organizations/outcome-indicators-project ↵