Quality assurance and quality improvement measures frequently focus on maintaining a set standard of patient care and assessing patient satisfaction with care provision. While QA and QI are essential to the successful operation of a SANE program, there are additional ways to evaluate SANE program effectiveness.
Why Should SANEs Invest in Program Evaluation?
Program evaluation can be used by SANEs to—
The Two Main Types of Program Evaluation
There are many reasons for SANE programs to invest in evaluation beyond QA and QI, and many ways in which a SANE program can be evaluated (see section below, Choosing What to Evaluate).
There are two main types of program evaluation:
Both process and outcome evaluations can provide useful information for SANE program development and operation (see Sidebar A). SANE programs should identify what they want to learn from evaluation, and then determine which type of evaluation (process or outcome) will provide the most comprehensive answer. It is often useful for a SANE program to first conduct a process evaluation to understand what they are doing and how they are doing, then examine if that process is effective. For example, if a SANE program wants to know if their services have a positive effect on patients’ psychological well-being (i.e., an impact evaluation), it may be important to understand first exactly what and how services are being provided to patients (i.e., a process evaluation).
Choosing What to Evaluate
SANE programs address patients' psychological and physical health, conduct forensic evidence collection, participate in legal proceedings, and strive for community change, thus SANE program evaluation could focus on any of these domains. For a discussion on what is already known regarding SANE effectiveness in these domains, see Chapter 1: Are SANE Programs Effective?. It is not possible to evaluate it all in one effort.
For any evaluation, it is necessary to identify one or two key evaluation questions. Evaluation questions define the scope of the evaluation by making explicit what is to be learned. A SANE program’s logic model can be particularly useful in selecting evaluation questions as it maps out program resources, activities, and goals (see Program Goals and Objectives section 3.7). SANE programs can use their logic model to ask pointed evaluation questions about how select activities are operating (i.e., process evaluation) and if they are leading to their intended impacts (i.e., outcome evaluation). When developing evaluation questions, it is essential that they are specific enough to be measured. For example, it is not possible to measure “coordination,” but it is possible to measure how often prosecutors consult with a SANE. The table below provides example process and outcome evaluation questions for the many domains in which SANE programs operate.
Sample Evaluation Questions
| Domain | Process Evaluation Questions | Outcome Evaluation Questions |
|---|---|---|
| Psychological Health |
What percentage of patients believed the nurse was supportive? What percentage of patients trusted the nurse? |
Do patients seen by the SANE program report fewer posttraumatic stress symptoms than those in the ED? Do patients seen by the SANE program report fewer depressive symptoms than those in the ED? |
| Medical/ Physical Health |
How many patients received STI prophylaxis? How many patients received emergency contraception? |
Are patients seen by the SANE program more likely to receive followup treatment for STIs compared to those in the ED? Do patients seen by the SANE program report fewer gastrointestinal problems compared to those in the ED? |
| Forensic Evidence Collection |
How many wet swabs, dry swabs, and smears are used for each sample collected? What paperwork is included in the kit when turned over to police? |
Are SANEs completing evidence collection more completely and correctly than non-SANEs? Are SANEs more likely to maintain the chain of custody than non-SANEs? |
| Legal Proceedings |
What percentage of patients received information about the criminal justice process? How many times did SANEs provide witness or expert testimony in criminal case proceedings? |
Are SANE cases more likely to be prosecuted or result in conviction than non-SANE cases? Are cases in which SANEs testify more likely to result in a conviction than cases in which SANEs do not testify? |
| Community Change |
How many trainings were provided to other organizations engaged in the coordinated community response to sexual assault? How many trainings/informational sessions were provided to the general public/potential jurors? |
Are prosecutors more likely to consult a medical expert after training on the value of such consultation? Are police more likely to refer victims to the SANE program after formalization of the police-SANE relationship (e.g., via a memorandum of understanding)? |
Overcoming Obstacles in Evaluation
Many SANE programs are hesitant to conduct evaluation due to a lack of expertise in evaluation, prior bad experiences with researchers or evaluators, a lack of resources, or a fear of negative evaluation results; however, each of these obstacles can be overcome.
The SANE Evaluation Toolkit
The SANE Evaluation Toolkit is a step-by-step guide for SANE programs to examine their impact on prosecution outcomes. It has already been used successfully in several jurisdictions across the country, allowing each community to know how many sexual assault cases treated by the SANE program reach prosecution and result in a conviction. The Toolkit is built for SANE programs that do not have any prior experience or expertise in evaluation. You can find it here.
Lack of expertise in evaluation: There are resources available that breakdown the evaluation process in a way that SANE program staff can implement the evaluation without prior experience or expertise (for example, see Text Box: The SANE Evaluation Toolkit). Alternatively, SANE programs can partner with an outside evaluator or researcher. Local colleges or universities may be a place to start looking for these partners.
Prior bad experience with researchers or evaluators: A prior bad experience with an evaluator or researcher will likely discourage a SANE program from engaging in these partnerships in the future. Not all researchers or evaluators are the same. A good research or evaluation partner will ask for and value the SANE program’s input, understand the dynamics of sexual assault, protect confidentiality, and plan for utilization of the evaluation findings.
Lack of resources: Evaluations do not have to cost a lot of money or take up a lot of time. Build resources for evaluation into the SANE program budget and remind funders that evaluation can actually bring in more money as it can provide hard evidence that the program is working, prompting additional financial support for its operation. An external researcher or evaluator may be able to help the SANE program identify how to implement an evaluation without compromising the program's number one priority—patient care.
Fear of negative evaluation results: Not knowing what will be discovered in the course of an evaluation can be concerning; however, it is important to remember that unexpected results are not a bad thing—they are a first step in identifying opportunities for program improvement.
Evaluation should be incorporated into SANE program development from the beginning. Effective evaluation becomes a useful tool for improving care to sexual assault patients and for improving all aspects of SANE services.