Step 6: Implement, Evaluate, and Improve
In Step 6, you ensure that your activities are implemented as planned, document progress toward your long-term goals, and share your findings.
To see an example of what Step 6 involves, take a look at the case study in SPRC’s online course A Strategic Planning Approach to Suicide Prevention.
Ensure Your Activities Are Implemented as Planned
In Step 4, you created an action plan for your activities. The action plan highlighted the tasks you would need to do, the people who would do the tasks, and the timeline in which they would occur. In Step 6, you ensure that your activities occurred as planned. You do this by conducting a process evaluation.
A process evaluation is a type of evaluation that determines if your activities have been implemented as intended.
The data from a process evaluation can help you determine the following:
- Did the way it was delivered match the developer’s original design? This is also called “fidelity.”
- Who participated in the activity, and for how long?
- What changes were made to the activity? How did these changes impact the activity?
- Were the resources spent on implementation sufficient?
- What obstacles were encountered in carrying out the activity?
- Did the activity meet the culture and the needs of participants? Did participants enjoy the activity?
Process evaluations can help you identify any issues related to implementation that could influence your results and ultimately impact your ability to achieve your long-term goals. By documenting the implementation of your activities, process evaluations can help you identify and solve problems that occurred so that you can improve your activities moving forward.
Document Progress toward Your Long-Term Goals
In Step 5, you planned your evaluation by identifying your evaluation questions and design. In Step 6, you document progress toward your long-term goals by implementing an outcome evaluation.
An outcome evaluation documents short- and long-term changes in knowledge, attitudes, skills, and/or behavior as a result of an activity.
Outcome evaluation data can help you determine the following:
- What changes actually occurred?
- How do these changes compare to what the activity was expected to achieve?
- How do these changes compare with the results for the people who did not participate in the activity?
As you document progress toward your short- and long-term outcomes using the data that you’ve collected, consider the following:
- Did you draw conclusions that truly fit with your research design? For example, if you did not use an experimental design with a control group, you cannot make claims about causation.
- Do you have enough data to make confident claims? Consider whether you have enough participants to be able to generalize your findings to similar people and places.
- Did you identify alternative explanations for your findings? Be sure to scan the environment where your activity took place to identify other potential sources besides your activity that may have helped produce the changes seen.
- What parts of your intervention do you need to adjust or expand for your suicide prevention efforts to be more effective?
Share Your Findings
Once there is documented progress toward your goals, you will want to share these results with others. Before you begin promotion, consider creating a communication plan that includes tasks, a timeline, and staffing. You may also want to consider how you will evaluate your promotion efforts.
When making a communication plan, consider the following:
Who Is Your Audience?
Different audiences might be interested in different aspects of your evaluation. For example, a funder might be interested in the details about the costs and benefits of the activity. A stakeholder might be interested in learning how long the activity will continue to produce benefit after it is over. An outside prevention professional might be interested in understanding how the activity was implemented. You will need to frame your communication materials to match the needs of your audience.Different audiences might be interested in different aspects of your evaluation. For example, a funder might be interested in the details about the costs and benefits of the activity. A stakeholder might be interested in learning how long the activity will continue to produce benefit after it is over. An outside prevention professional might be interested in understanding how the activity was implemented. You will need to frame your communication materials to match the needs of your audience.
What Channels Will You Use to Share Your Findings?
There are many different ways to promote information about your findings. Journal articles, website publications, in-person presentations, and social media are all ways you might share your findings. Consider using multiple channels to reach the biggest audience.
What Content and Format Will You Use?
The content you choose to promote and the format you use can vary. Will you produce a lengthy evaluation report? A PowerPoint presentation? A blog? What types of information will you make public? What you choose will depend on who your audience is and the channels you are using to implement your communication plan.
Recommended Resources
RAND Suicide Prevention Program Evaluation Toolkit
This toolkit is designed to help program staff overcome common challenges to evaluating and planning improvements to their programs.
Challenges and recommendations for evaluating suicide prevention programs
This report shares lessons learned by GLS evaluators regarding the collection of surveillance and evaluation data.
Building the evaluation capacity of local programs serving American Indian/Alaska Native populations: Lessons learned
This report describes the activities of 34 tribal communities served by the CAPT under SAMHSA’s Science to Service Initiative (2010–2014).