|
Content
|
- Implementation Science
- Implementation Science is the study of factors that influence the full and effective use of innovations in practice.
- In implementation science, implementation factors are identified or developed and demonstrated in practice, to “influence the full and effective use of innovations.” Each factor and the factors in combination are subject to continued study and improvement to advance the science of implementation.
- The following content outlines Active Implementation Frameworks from the National Implementation Research Network (NIRN)
- Usable Innovations
- Before implementing an innovation (i.e., an evidence-based program), it’s vital to have a clear understanding of the program and its suitability for your State or agency.
- It’s necessary to have sufficient detail about the innovation that you can train staff and administrators to implement it with fidelity; that the innovation can be replicated across your agency; and that there is an assessment that allows you to measure the use of the innovation.
- In other words, the innovation needs to be teachable, learnable, doable, and be readily assessed in practice. The following criteria need to be in place to ensure that your innovation is usable:
- Clear description of the program
- Clear essential functions that define the program
- Operational definitions of essential functions
- Practical performance assessment
- The Stages of Implementation
- In the NIRN framework there are 4 stages to implementation. They are Exploration, Installation, Initial Implementation, and Full Implementation (Fixsen et al., 2010).
- The Stages are dynamic within organizations such as schools and clinics, moving back and forth among Stages as personnel and circumstances change.
- The functions of the Exploration Stage are a critical starting place for work with States, districts, and others. Taking the time for exploration saves time and money (Romney, 2011) and improves the chances for success (Saldana, Chamberlain, Wang, & Brown, 2011; Slavin, Madden, Chamberlain, & Cheung, 2010). During Exploration, readiness is assessed by an Implementation Team. To the extent an organization is not ready the Implementation Team is accountable for helping create readiness, an important function when the goal is to reach an entire population.
- The function of the Installation Stage is to acquire or repurpose the resources needed to do the work ahead. Selecting staff, identifying sources for training and coaching, providing initial training for staff, finding, or establishing performance assessment (fidelity) tools, locating office space, assuring access to materials and equipment, and so on are among the resources that need to be in place before the work can be done effectively (Fixsen et al. 2005; Saldana et al., 2012).
- During the Exploration Stage Implementation Teams help organizations recognize the need for these resources and during the Installation Stage Implementation Teams help organizations secure the needed resources to do the work ahead and prepare staff for the new practices.
- Initial Implementation is the time when the innovation is being used for the first time. During this Stage, practitioners and staff are attempting to use newly learned skills (e.g., the evidence-based program) in the context of a provider organization that is just learning how to change to accommodate and support the new ways of work. This is the most fragile stage where the awkwardness associated with trying new things and the difficulties associated with changing old ways of work are strong motivations for giving up and going back to comfortable routines (business as usual).
- The Initial Implementation Stage is a real challenge. Establishing and sustaining changes to the point of integration into daily work is not likely unless there is external support for change at the practice level (support from coaches, Joyce & Showers, 2002), organization level (support from Implementation Teams; Aladjem & Borman, 2006; Nord & Tucker, 1987), and system level (support from Implementation Teams; Schofield, 2004).
- Full Implementation is reached when 50% or more of the intended practitioners, staff, or team members are using an effective innovation with fidelity and good outcomes.
- In the Full Implementation Stage, the new ways of providing services are now the standard ways of work where practitioners and staff routinely provide high quality services, and the implementation supports are part of the way the provider organization carries out its work.
- Implementation Teams
- Implementation Teams are an essential factor to the ongoing success of using evidence-based programs. Practitioners, staff, administrators, and leaders come and go and each new person needs to develop the competencies to effectively carry out the innovation and its implementation supports and the practice-policy communication loop to inform ongoing system change processes.
- First, the Implementation Teams must have the knowledge, skills, and abilities to help practitioners and staff make full and effective uses of the innovations enabled by policy.
- This capacity to implement with fidelity and good outcomes is essential to the system change process. If the policies or innovations are not being used as intended or are being used as intended but not producing desired outcomes, those implementation and intervention issues need to be resolved at the practice level before asking the executive leadership to intervene in how the system functions.
- Second, the Implementation Team members should have firsthand experience with the facilitators and barriers to making full and effective uses of innovations in human service settings.
- Managers and administrators come and go and need to continually adjust organizational supports to find solutions that facilitate the work of practitioners and improve outcomes.
- Developing ways to embed implementation teams into your current meeting and teaming structures is an essential part of implementation work
- For a refresher on strategies for building internal capacity in leadership teams and giving work back to those who are best suited to find solutions, go the following link:
http://courses.ncwwi.org/course/view.php?id=65
- Implementation Drivers
- Implementation Drivers are the engine of change (Fixsen et al., 2005). As with the Stages, Drivers are dynamic and interact in interesting ways to produce consistent uses of innovations and reliable outcomes for children and others.
- Implementation Drivers have been categorized as Competency, Organization, and Leadership supports. Effective innovations are, by definition, new ways of work.
- For competency development, new ways of work need to be taught and learned through training and coaching with practitioners (teachers, district staff, implementation team members) who have been selected (mutual selection for individuals is like the Exploration Stage for organizations) to be the first to use the innovation. As coaches support practitioners in learning the innovation and as performance (fidelity) assessments are used to monitor the progress of teaching and learning, organization and system facilitators and barriers are identified.
- Organization supports are developed by facilitative administrators (directors, managers, superintendents, principals, non-teaching staff) who change organization practices and support systems interventions so they can establish a hospitable environment for the use of effective innovations and the use of effective implementation supports for practitioners. Having a decision support data system is an essential component for guiding the processes of establishing the innovation, the implementation supports for practitioners, and the assessments of immediate outcomes. For example, data may show low fidelity performance for a group of teachers being coached by a given individual while other teachers readily are meeting performance criteria. These data might support a decision to focus on the quality of coaching rather than the quality of teaching.
- Finally, implementation requires leadership that can help resolve adaptive issues (convening groups to identify problems, arriving at consensus regarding how to approach a solution, detecting progress toward resolution) and technical problems (setting goals, managing time and effort, solving problems of known dimensions) that arise in the course of initiating changes in the ways of work and managing change in organizations and systems.
- These interactive processes are integrated to maximize their influence on staff behavior and the organizational culture. The interactive implementation drivers also compensate for one another so that a weakness in one component can be overcome by strengths in other components. These are core implementation components (implementation drivers).
- Active Implementation
- In addition to the above frameworks: implementation stages, implementation teams, and implementation drivers; active implementation also relies on improvement cycles that incorporates and practice-policy communication loops. Essentially, the improvement cycles in implementation work are based on the plan-do-study-act (PDSA) cycle developed by Bell Labs in the 1920s to improve quality and reduce errors in design and manufacturing (Shewhart, 1931). The PDSA cycle has been used successfully in many applications in human services (Joyce & Showers, 2002; Varkey, Reller, & Resar, 2007; Weick, Sutcliffe, & Obstfeld, 1999).
- Plan Do Study Act (PDSA)
- The plan-do-study-act cycle involves a “trial-and-learning” approach in which the PDSA steps are conducted over iterative cycles designed to discover and solve problems, and eventually leads to achieving high standards while eliminating error. For example, the “plan” can be the innovation as it is intended to be used in practice. To carry out the plan, the plan needs to be operationalized (what to “do” and say to enact the plan). This compels attention to the core innovation components and provides an opportunity to a) begin to develop a training process (e.g. here is how to do the plan) and b) create a measure of fidelity (e.g. did we do the plan as intended). The budding fidelity measure can be used to interpret the outcomes in the “study” part of the PDSA cycle (e.g. did doing the plan produce desired results). If the plan was done as intended and there are poor results (an innovation problem), the innovation needs to be changed before the next cycle. If the plan was not done as intended (an implementation problem), the preparation of practitioners needs to be improved before the next cycle so they can “do” the innovation as planned.
- Usability Testing
- Usability testing consists of a series of tests of an innovation. Usability testing is a way to see how easy to use something is by testing it with real users. Users are asked to complete tasks, typically while they are being observed by a researcher, to see where they encounter problems and experience confusion.
- Practice-Policy Communication Loop
- Implementing effective innovations requires change at the practice, organization, and system levels. But what needs to change and how much change is needed to achieve desired outcomes? Answering this question is the purpose of the practice-policy communication loop.
- The goal is to create systems that "are able to learn from their own experience and to modify their structure and design to reflect what they have learned" (Morgan & Ramirez, 1983; p. 4). Part of that learning comes from the SEA soliciting, receiving, and responding to feedback regarding barriers and facilitators to implementation. The use of this practice-policy improvement cycle helps to create the ability of the system to monitor and question the context in which it is operating and to question the rules that underlie its own operation. The system has the capacity to search for errors and faulty operating assumptions, the capacity to learn from them, and the ability to make needed changes to improve intended outcomes.
- Increasingly, Federal and State governments are supporting and occasionally insisting upon the use of evidence-based programs and other reforms in human services. They are developing policies and funding structures to encourage organizations and staff to use evidence-based programs and other reforms. In our examination of the literature and discussions with system change agents globally, there are many examples where policy makers have mandated the use of evidence-based programs and other reforms with little impact on service delivery (e.g., Chapin Hall for Children, 2002; Jerald, 2005; Nutt, 2002; O’Donoghue, 2002; Stuit, 2011). There are some examples where evidence-based programs or reforms were used with good outcomes for a period of time then abandoned (e.g., Bryce, Gilroy, Jones, Hazel, Black, & Victora, 2010; Glennan, Bodilly, Galegher, & Kerr, 2004). There also are a few examples of success where policies encouraged the use of effective innovations or reforms at the practice level, the innovations were supported with effective implementation efforts, and systems changed to encourage widespread use of the innovation (e.g., Glennan et al., 2004; Khatri & Frieden, 2002; Ogden et al., 2005; Rhoades, Bumbarger, & Moore, 2012).
- What differentiated the large-scale successes from those with temporary or no outcomes? The successes had direct and frequent communication from the practice level to the executive level (the practice-policy communication loop). The policy-practice (top down) support was present in the successes and failures. What was missing in many cases is the practice-policy (bottom up) communication with the leaders who initiated the process of change. Our analyses suggest the combination of the practice to policy communication loop (detect problems, identify leverage points) and the policy to practice supports (solve real problems in real time) are critical features of successful efforts to implement evidence-based programs and other reforms on a socially significant scale (Fixsen et al. in press).
- In successful system change efforts, executive management teams meet frequently (at least monthly) to hear about what is helping or hindering efforts to make full and effective use of evidence-based programs at the practice level. The information may consist of descriptions of experiences and include data collected with reasonable precision. They are engaged in system change informed by the practice-policy communication loop.
- At this point in the process the Implementation Team (and others) must assess if an innovation can be used as intended and can produce desired outcomes or if there are some ways the system is not supporting the effective use of the innovation.
- Given the competence of the Implementation Teams, if they are encountering difficulties, their concerns will have credibility with the executive management team and the leaders will have sufficient information and confidence to change the system to better support improved outcomes.
- For a review of the Active Implementation Frameworks, go to: https://implementation.fpg.unc.edu/module-1
|
