What are Implementation Outcomes?
When we introduce new programs, practices, or interventions—whether it's a new health treatment, an educational tool, or a community service—we want to make sure they actually work in the real world. For example, in public health people will often measure health outcomes, which focus on the effects of an intervention on patients (so things like mortality rates or readmission rates). But it’s not enough just to know that something works. We also need to consider how it fits into everyday life and whether people will actually use it. This is where "implementation outcomes" come in. These are the key factors that help us measure how well an intervention is adopted, used, and sustained over time.
There are many different ways to think about implementation outcomes, but one of the most common is based on a paper published in 2010 by Proctor et al., which identifies eight different implementation outcomes: acceptability, adoption, appropriateness, feasibility, fidelity, implementation cost, penetration, and sustainability
The Implementation Outcomes
- Acceptability
Simplified Definition: How much people like the intervention.
Who we usually measure: Individual providers or consumers.
A recent survey found that 90% of middle school students in the US dislike wearing school uniforms. If you were considering implementing a school uniform policy in schools, then you would say it probably has low acceptability among students.
- Adoption (or Uptake)
Simplified Definition: Whether or not people start using the intervention.
Who we usually measure: Individual providers or organizations/settings.
Adoption isn’t only the decision or actions taken to start using the intervention, but it can also refer to the intention to start using the intervention. You’ll sometimes hear terms like “early adopters” or “laggards” used, which are references to names for different groups depending on how quickly they adopt something. The graphic below shows the different groups:
- Appropriateness
Simplified Definition: Whether the intervention is a good fit for the situation.
Who we usually measure: Individual providers, individual consumers or organizations/settings.
This can sometimes be confused with acceptability. Appropriateness focuses more on if it's suitable rather than whether people like it (two things which are, admittedly, often related). For example, if we think about school uniforms, the acceptability might be low because students don’t like uniforms, but it might still be appropriate for the setting of the school if they wanted to solve an issue of safety and identification of students.
- Feasibility
Simplified Definition: How easy or hard it is to put the intervention into action.
Who we usually measure: Individual providers or organizations/settings.
Feasibility in this case is feasibility within a specific setting. This can sometimes be confused with appropriateness as well, but feasibility normally focuses more on things like resources or training requirements. So maybe uniforms would be a good fit for a school’s mission (high appropriateness), but the school doesn’t have the funds to design new uniforms (low feasibility).
- Fidelity
Simplified Definition: Whether the intervention is done as planned.
Who we usually measure: Individual providers
People love to measure fidelity as an outcome. Fidelity in this situation is referring to fidelity to the implementation, such as whether it was delivered according to a defined protocol, or as intended by the people developing the intervention. It doesn’t necessarily refer to fidelity to the intervention itself (although that’s often very closely related).
- Cost
Simplified Definition: How much money and resources are needed to carry out the intervention.
Who we usually measure: Provider or providing organization
Cost here is the cost to the organization, not the cost to the consumer. This includes costs of delivering the intervention, costs of implementing the intervention, and the costs associated with where it's being delivered. Cost is a really important outcome, and people are always trying to figure out different ways to reduce costs.
- Penetration
Simplified Definition: How widespread or available the intervention is to those who need it.
Who we usually measure: Organization or setting
You can also think about penetration as being “reach”, or about how well an intervention is integrated into a particular setting. An easy way to measure penetration is by getting a proportion of the number of people who use the service to the number of people eligible. This is called service penetration. You can also use a similar process to figure out penetration in terms of providers (number of providers delivering an intervention divided by the number expected to).
- Sustainability
Simplified Definition: Whether the intervention continues to be used over time.
Who we usually measure: Organization or setting
Sustainability is whether the intervention continues to be used over time. Within academic research, it’s often the case that an intervention will be implemented, but then once the research project ends (and all the funding and support that comes with it), the organization is unable or unwilling to keep delivering the intervention. It’s really important to think about sustainability throughout the process of implementing so you can make sure an intervention will keep being used long-term.
Why worry about these implementation outcomes?
Measuring and tracking implementation outcomes is useful because they give us the data we need to understand how well an intervention is being applied and what changes need to be made for success. It’s not enough to just launch a program or practice—implementation outcomes help us track and refine the process. They help us know if the intervention is being delivered as intended, what might be stopping the intervention from being used effectively, and if it is going to have the impact that we want. These findings can also help inform future projects.
Because many of these are fundamental to implementation, it is important to think about outcomes not only after something has been implemented, but also while you’re planning for implementation. That way, you can do things like build in strategies that will achieve certain outcomes, or adapt the intervention to make it more likely to be successful, rather than just judging success or failure retrospectively.
For each of these outcomes, there is a wealth of literature explaining when it is appropriate to focus on them, what their theoretical basis is, and the best ways to measure them, depending on what you’re trying to achieve. If you are thinking about looking at certain outcomes, it’s worth it to delve into that literature to see what the best way to go about your project is.
Main Source:
✉Corresponding author.