Difference between revisions of "Evaluation"

From Guidelines & Principles
Jump to: navigation, search
(Strongly Recommended)
(Assessment Items)
 
(30 intermediate revisions by 2 users not shown)
Line 1: Line 1:
 
As entities that serve the public, nonprofit organizations have an obligation to demonstrate their value to the public. The public has a stake in nonprofit performance and is entitled to information regarding organization results. Nonprofits should regularly measure their performance against a clear set of goals and should share such information with their constituents. Nonprofit evaluation should be appropriate to the size and purpose of the organization and evaluation data should be used to continually improve the quality of processes, programs and activities.   
 
As entities that serve the public, nonprofit organizations have an obligation to demonstrate their value to the public. The public has a stake in nonprofit performance and is entitled to information regarding organization results. Nonprofits should regularly measure their performance against a clear set of goals and should share such information with their constituents. Nonprofit evaluation should be appropriate to the size and purpose of the organization and evaluation data should be used to continually improve the quality of processes, programs and activities.   
  
==Infrastructure Checklist==
+
=Assessment Items=
<section begin=infrastructure />
+
<section begin=Evaluation />
===Strongly Recommended===
+
===Methods===
<section begin=infrastructure_strongly_recommended />
+
<section begin=Methods />
 
{| class="wikitable"
 
{| class="wikitable"
 
|-
 
|-
!  
+
!
! Yes     
+
! Legal
! No     
+
! Essential
! In Progress
+
! Recommended
! Not Applicable
+
! Not Sure
+
  
 
|-
 
|-
| [[Evaluation Plans & Systems | Accountability, data collection & monitoring systems]]
+
|Information collected from constituents is kept confidential, and we never identify specific persons served unless we have their written permission. '''[Legal]'''
|
+
|style="width:80px; text-align:center" | X
|
+
|style="width:80px" |
|
+
|style="width:80px" |
|
+
|
+
 
|-
 
|-
| Environmental scan and/or community needs/assets assessment
+
|We conduct, analyze, and interpret evaluation in ways that are culturally sensitive and appropriate to the communities we serve and aspire to serve.
|
+
|style="width:80px" |
|
+
|style="width:80px; text-align:center" | X
|
+
|style="width:80px" |
|
+
|
+
 
|-
 
|-
| Evaluation procedures
+
|We strive to use the most scientifically rigorous evaluation methods possible. When appropriate, we use reporting language that accurately describes correlation rather than causation between our efforts and outcomes.
|
+
|style="width:80px" |
|
+
|style="width:80px; text-align:center" | X
|
+
|style="width:80px" |
|
+
|
+
 
|-
 
|-
| Written contracts with external consultants
+
|We utilize external evaluators when appropriate and feasible and have a written memo of agreement/contract with every external evaluator or consultant.
|
+
|style="width:80px" |  
|
+
|style="width:80px" |
|
+
|style="width:80px; text-align:center" | X
|
+
|
+
|}
+
<section end=infrastructure_strongly_recommended />
+
 
+
===Recommended===
+
<section begin=infrastructure_recommended />
+
{| class="wikitable"
+
 
|-
 
|-
!
+
|We conduct ongoing environmental scans to understand our community, including the needs of constituents, services provided by the government and other nonprofits, and applicable trends (i.e., economic, demographic, etc).
! Yes
+
|style="width:80px" |
! No
+
|style="width:80px" |
! In Progress
+
|style="width:80px; text-align:center" | X
! Not Applicable
+
! Not Sure
+
 
+
 
|-
 
|-
| Formative evaluation plan & systems
+
|Our evaluation strategy includes input from a wide variety of stakeholders including staff, board, funders, community members, etc.
|
+
|style="width:80px" |  
|
+
|style="width:80px" |
|
+
|style="width:80px; text-align:center" | X
|
+
|
+
 
|-
 
|-
| Outcome evaluation plan & systems
+
|We have defined, ongoing and sustainable procedures in place for evaluating our programs, procedures, and outcomes in relation to our mission.
|
+
|style="width:80px" |  
|
+
|style="width:80px" |
|
+
|style="width:80px; text-align:center" | X
|
+
|
+
 
|-
 
|-
| Program evaluation plan & systems
+
|We employ appropriate data collection and analysis tools in order to produce accurate, timely, and useful results.
|
+
|style="width:80px" |  
|
+
|style="width:80px" |
|
+
|style="width:80px; text-align:center" | X
|
+
|
+
|-
+
| Assess evaluation practices & modify as needed
+
|
+
|
+
|
+
|
+
|
+
|-
+
| Process for sharing results with the public
+
|
+
|
+
|
+
|
+
|
+
|-
+
| Solicit stakeholder input/feedback
+
|
+
|
+
|
+
|
+
|
+
 
|-
 
|-
 +
|Performance measures are realistic, specific, measurable, and appropriate to the size and scope of the organization and its constituents. Key measures seek information on activities, effectiveness, and community input using both qualitative and quantitative methods.
 +
|style="width:80px" |
 +
|style="width:80px" |
 +
|style="width:80px; text-align:center" | X
 
|}
 
|}
<section end=infrastructure_recommended />
+
<section end=Methods />
<section end=infrastructure />
+
===Use of Results===
 
+
<section begin=Use of Results />
==Assessment Tool==
+
<section begin=assessment />
+
===Methodology & Measurement===
+
 
{| class="wikitable"
 
{| class="wikitable"
 
|-
 
|-
!  
+
!
! No/Not Begun
+
! Legal
! In Process
+
! Essential
! Yes/ Complete
+
! Recommended
! Not Applicable
+
! Not Sure
+
  
 
|-
 
|-
| We have defined, ongoing and sustainable procedures in place for evaluating our programs, procedures, and outcomes in relation to our mission.  
+
|Evaluation results are used to strengthen and improve our programs and activities, and performance measures inform our decisions to continue, refocus, or end programs or efforts.
|
+
|style="width:80px" |  
|
+
|style="width:80px; text-align:center" | X
|
+
|style="width:80px" |  
|
+
|
+
 
|-
 
|-
| Our evaluation strategy is ongoing and includes input from a wide variety of stakeholders including staff, board, funders, community members, etc.
+
|Evaluation results inform our operational plan and are used to evaluate organizational effectiveness.
|
+
|style="width:80px" |  
|
+
|style="width:80px; text-align:center" | X
|
+
|style="width:80px" |  
|
+
|
+
 
|-
 
|-
| Performance measures are realistic, specific, measurable and appropriate to the size and scope of the organization and its constituents. Measurement includes information on satisfaction, activities, results, and community input; both qualitative and quantitative data; and data on efficiency and effectiveness.
+
|Evaluation results are communicated to a broad range of constituents including staff, board, constituents, funders, and community members.
|
+
|style="width:80px" |  
|
+
|style="width:80px" |
|
+
|style="width:80px; text-align:center" | X
|
+
|
+
 
|-
 
|-
| We employ appropriate data collection and analysis tools in order to produce accurate, timely, and useful evaluation information.
+
|We train our personnel in evaluation methods to improve their understanding and utilization of data developed from evaluation activities.
|
+
|style="width:80px" |  
|
+
|style="width:80px" |
|
+
|style="width:80px; text-align:center" | X
|
+
|
+
 
|-
 
|-
| We conduct evaluations in ways that are culturally sensitive and appropriate to the communities we serve.
+
|We actively assess the usefulness and accuracy of our evaluation practices and modify them as needed.
|
+
|style="width:80px" |  
|
+
|style="width:80px" |
|
+
|style="width:80px; text-align:center" | X
|
+
|}
|
+
<section end=Use of Results />
 +
 
 +
=Resources=
 +
<section begin=Resources />
 +
{| class="wikitable"
 
|-
 
|-
| Information that is collected from persons served is kept confidential, and we never identify specific persons served unless we have their written permission.
+
| [[Evaluation Plans & Systems | Accountability, data collection & monitoring systems]]
|
+
|
+
|
+
|
+
|
+
 
|-
 
|-
| We have a thorough understanding of the community in which we operate, including the needs of constituents, services provided by the government and other nonprofits, and applicable trends (i.e., economic, demographic, etc.).
+
| [[Evaluation Plans & Systems | Environmental scan and/or community needs/assets assessment]]
|
+
|
+
|
+
|
+
|
+
 
|-
 
|-
| We regularly monitor the needs and satisfaction of stakeholders and provide a grievance procedure to address complaints.
+
| [[Evaluation Plans & Systems | Evaluation procedures]]
|
+
|
+
|
+
|
+
|
+
 
|-
 
|-
| We utilize external evaluators when appropriate and feasible.  These evaluators follow the Guiding Principles for Evaluators set forth by the American Evaluation Association. 
+
| Written contracts with external consultants
*http://www.eval.org/GPTraining/GP%20Training%20Final/gp.principles.pdf.
+
|
+
|
+
|
+
|
+
|
+
 
|-
 
|-
| We have a written memo of agreement/contract with every external evaluator and/or consultant.
+
| [[Evaluation Plans & Systems | Formative evaluation plan & systems]]
|
+
|
+
|
+
|
+
|
+
|}
+
 
+
===Use of Evaluation Results===
+
{| class="wikitable"
+
 
|-
 
|-
!
+
| [[Evaluation Plans & Systems | Outcome evaluation plan & systems]]
! No/Not Begun
+
! In Process
+
! Yes/ Complete
+
! Not Applicable
+
! Not Sure
+
 
+
 
|-
 
|-
| Evaluation results are communicated to a broad range of constituents including staff, board, constituents, funders, and community members.
+
| [[Evaluation Plans & Systems | Program evaluation plan & systems]]
|
+
|
+
|
+
|
+
|
+
 
|-
 
|-
| Measurement informs the operational plan and is used to evaluate organizational effectiveness.
+
| [[Evaluation Plans & Systems | Assess evaluation practices & modify as needed]]
|
+
|
+
|
+
|
+
|
+
 
|-
 
|-
| Evaluation results are used to strengthen and improve our programs and activities by incorporating evaluation findings into strategic planning processes.
+
| [[Evaluation Plans & Systems | Process for sharing results with the public]]
|
+
|
+
|
+
|
+
|
+
 
|-
 
|-
| Our programs take into account and respond to the experience, needs, and satisfaction of the constituents they serve.
+
| [[Evaluation Plans & Systems | Solicit stakeholder input/feedback]]
|
+
|
+
|
+
|
+
|
+
 
|-
 
|-
| We train our personnel in evaluation methods to improve their understanding and utilization of data developed from evaluation activities.
 
|
 
|
 
|
 
|
 
|
 
|-
 
| We actively assess the usefulness and accuracy of our evaluation practices and modify them as needed.
 
|
 
|
 
|
 
|
 
|
 
|-
 
| We share relevant lessons learned with other nonprofits and funding sources.
 
|
 
|
 
|
 
|
 
|
 
 
|}
 
|}
<section end=assessment />
+
<section end=Resources />
 +
 
 +
=Best Practices=
 +
'''Methods'''
 +
*When designing a new program or project, identify very specifically how it will impact your mission through measurable outcomes and by addressing identified needs.
 +
*Take time to develop the theory behind the expected relationship between your programmatic efforts and intended outcomes. Discuss the “how,” “why,” “who,” and “when” you expect to see changes. Be careful not to just jump right into tracking metrics that have face value. 
 +
*In conducting evaluation, strive to use the most scientifically rigorous methods possible, including external evaluation, while taking into account resource limitations and/or ethical constraints.
 +
*Client testimonials and stories, while potentially valuable for marketing and fund development, are not by themselves evaluation strategies. Similarly, service counts are necessary for reporting volume of service, but are not evaluative of impact or effectiveness on their own.
 +
*Plan on at least 5% of a program’s budget for evaluation costs. This is a commonly accepted range for funders, and includes the cost of staff time to develop and process evaluation tools, copying, postage, etc. If you use an external evaluator, the cost jumps to 10% or more. Research in advance what method/approach is best for each program. 
 +
*Evaluators should follow the national Guiding Principles for Evaluators set forth by the American Evaluation Association. https://www.eval.org/p/cm/ld/fid=51.
 +
*Consumer feedback should be a significant component of ongoing evaluation efforts. Develop an evaluation plan (including surveys, focus groups, post-service feedback and other tools) that includes regular consumer feedback. Have the board review results of the feedback and discuss both successes and challenges. Board recommendations should be carried back to staff (via the Executive Director) for program evaluation and adaptation as needed. 
 +
*Familiarize yourself with various methods of evaluation to determine which approach is best for individual programs and projects; one size does not fit all.  Establish with staff that evaluation planning is an up-front activity rather than a last-minute reaction. Solid goals and objectives lead to effective evaluation plans. 
  
==Walking the Talk==
+
'''Use of Results'''
* When designing a new program/project, identify very specifically how it will impact your mission through measurable outcomes addressing identified needs.
+
*Establish a process for periodic review of evaluation feedback. If you run frequent programs/projects, you may consider compiling results of like programs for a thumb-nail sketch to present to board members.   
* Consumer feedback should be a significant component of ongoing evaluation efforts. Develop an evaluation plan (including surveys, focus groups, post-service feedback and other tools) that includes regular consumer feedback.  Have the board review results of the feedback and discuss both successes and challenges. Board recommendations should be carried back to staff (via the Executive Director) for program evaluation and adaptation as needed.
+
*Review evaluation feedback with staff following each program/project. Generate a list of improvements or “lessons learned” and file it where it will be incorporated into planning upcoming projects.
* Familiarize yourself with various methods of evaluation to determine which approach is best for individual programs and projects; one size does not fit all.
+
*Develop a mechanism for sharing evaluation feedback with key stakeholders. Consider devoting a section of your newsletter to “recent feedback,” or including anecdotal and hard data in your annual report. You should be able to show your impact on the issue as effectively as you can demonstrate your passion for the cause.
* Establish with staff that evaluation planning is an up-front activity rather than a last-minute reaction. Solid goals and objectives lead to effective evaluation plans.
+
<section end=Evaluation />
* Plan on at least 5% of a program’s budget for evaluation costs. This is a commonly accepted range for funders, and includes the cost of staff time to develop and process evaluation tools, copying, postage, etc.  If you use an external evaluator, the cost jumps to 10% or more. Research in advance what method/approach is best for each program.
+
* Establish a process for periodic review of evaluation feedback. If you run frequent programs/projects, you may consider compiling results of like programs for a thumb-nail sketch to present to board members.   
+
* Review evaluation feedback with staff following each program/project. Generate a list of improvements or “lessons learned” and file it where it will be incorporated into planning upcoming projects.
+
* Develop a mechanism for sharing evaluation feedback with key stakeholders. Consider devoting a section of your newsletter to “recent feedback,” or including anecdotal and hard data in your annual report. You should be able to show your impact on the issue as effectively as you can demonstrate your passion for the cause.
+
* Evaluators should follow the national Guiding Principles for Evaluators set forth by the American Evaluation Association. 
+
**http://www.eval.org/GPTraining/GP%20Training%20Final/gp.principles.pdf.
+

Latest revision as of 21:09, 9 October 2019

As entities that serve the public, nonprofit organizations have an obligation to demonstrate their value to the public. The public has a stake in nonprofit performance and is entitled to information regarding organization results. Nonprofits should regularly measure their performance against a clear set of goals and should share such information with their constituents. Nonprofit evaluation should be appropriate to the size and purpose of the organization and evaluation data should be used to continually improve the quality of processes, programs and activities.

Assessment Items

Methods

Legal Essential Recommended
Information collected from constituents is kept confidential, and we never identify specific persons served unless we have their written permission. [Legal] X
We conduct, analyze, and interpret evaluation in ways that are culturally sensitive and appropriate to the communities we serve and aspire to serve. X
We strive to use the most scientifically rigorous evaluation methods possible. When appropriate, we use reporting language that accurately describes correlation rather than causation between our efforts and outcomes. X
We utilize external evaluators when appropriate and feasible and have a written memo of agreement/contract with every external evaluator or consultant. X
We conduct ongoing environmental scans to understand our community, including the needs of constituents, services provided by the government and other nonprofits, and applicable trends (i.e., economic, demographic, etc). X
Our evaluation strategy includes input from a wide variety of stakeholders including staff, board, funders, community members, etc. X
We have defined, ongoing and sustainable procedures in place for evaluating our programs, procedures, and outcomes in relation to our mission. X
We employ appropriate data collection and analysis tools in order to produce accurate, timely, and useful results. X
Performance measures are realistic, specific, measurable, and appropriate to the size and scope of the organization and its constituents. Key measures seek information on activities, effectiveness, and community input using both qualitative and quantitative methods. X

Use of Results

Legal Essential Recommended
Evaluation results are used to strengthen and improve our programs and activities, and performance measures inform our decisions to continue, refocus, or end programs or efforts. X
Evaluation results inform our operational plan and are used to evaluate organizational effectiveness. X
Evaluation results are communicated to a broad range of constituents including staff, board, constituents, funders, and community members. X
We train our personnel in evaluation methods to improve their understanding and utilization of data developed from evaluation activities. X
We actively assess the usefulness and accuracy of our evaluation practices and modify them as needed. X


Resources

Accountability, data collection & monitoring systems
Environmental scan and/or community needs/assets assessment
Evaluation procedures
Written contracts with external consultants
Formative evaluation plan & systems
Outcome evaluation plan & systems
Program evaluation plan & systems
Assess evaluation practices & modify as needed
Process for sharing results with the public
Solicit stakeholder input/feedback


Best Practices

Methods

  • When designing a new program or project, identify very specifically how it will impact your mission through measurable outcomes and by addressing identified needs.
  • Take time to develop the theory behind the expected relationship between your programmatic efforts and intended outcomes. Discuss the “how,” “why,” “who,” and “when” you expect to see changes. Be careful not to just jump right into tracking metrics that have face value.
  • In conducting evaluation, strive to use the most scientifically rigorous methods possible, including external evaluation, while taking into account resource limitations and/or ethical constraints.
  • Client testimonials and stories, while potentially valuable for marketing and fund development, are not by themselves evaluation strategies. Similarly, service counts are necessary for reporting volume of service, but are not evaluative of impact or effectiveness on their own.
  • Plan on at least 5% of a program’s budget for evaluation costs. This is a commonly accepted range for funders, and includes the cost of staff time to develop and process evaluation tools, copying, postage, etc. If you use an external evaluator, the cost jumps to 10% or more. Research in advance what method/approach is best for each program.
  • Evaluators should follow the national Guiding Principles for Evaluators set forth by the American Evaluation Association. https://www.eval.org/p/cm/ld/fid=51.
  • Consumer feedback should be a significant component of ongoing evaluation efforts. Develop an evaluation plan (including surveys, focus groups, post-service feedback and other tools) that includes regular consumer feedback. Have the board review results of the feedback and discuss both successes and challenges. Board recommendations should be carried back to staff (via the Executive Director) for program evaluation and adaptation as needed.
  • Familiarize yourself with various methods of evaluation to determine which approach is best for individual programs and projects; one size does not fit all. Establish with staff that evaluation planning is an up-front activity rather than a last-minute reaction. Solid goals and objectives lead to effective evaluation plans.

Use of Results

  • Establish a process for periodic review of evaluation feedback. If you run frequent programs/projects, you may consider compiling results of like programs for a thumb-nail sketch to present to board members.
  • Review evaluation feedback with staff following each program/project. Generate a list of improvements or “lessons learned” and file it where it will be incorporated into planning upcoming projects.
  • Develop a mechanism for sharing evaluation feedback with key stakeholders. Consider devoting a section of your newsletter to “recent feedback,” or including anecdotal and hard data in your annual report. You should be able to show your impact on the issue as effectively as you can demonstrate your passion for the cause.