A total of 24 participants representing six stakeholder roles including attending, hospitalist/attending, unit medical director, case manager, nurse manager, and social worker.

Participants reported a wide range of years worked at MSHS, 33% have worked for 0 – 1 year, 33% have worked 5 – 10 years, 25% have worked 2 – 4 years, and 9% have worked for more than 10 years (See Table 2).

The results are reported according to the RE-AIM dimensions with relevant CFIR constructs provided for each RE-AIM dimension.

The Reach domain of the RE-AIM framework examines the motivations for accepting or rejecting the use of the 48DPT. Most participants across stakeholder roles were aware of the existence of the 48DPT. When asked how often the results from the 48DPT are utilized by at least one member of the interdisciplinary team on a given weekday, responses varied by stakeholder role. Among unit-based leadership participants, social workers and case managers reported daily use of the 48DPT. Nurse managers and unit medical director participants rarely used the 48DPT results. Hospitalist/Attending and attending/unit medical director participants also reported minimal use of the 48DPT on a daily basis. The self-report measure of the daily use of the 48DPT aligns with the CFIR construct, Innovation Recipient Impact, which aims to assess the degree to which the innovation impacts the recipients  [51]. Since the frequency of use of the 48DPT varied by stakeholder role with social workers and case manager participants reporting daily use and limited use by nurse managers, unit medical director, hospitalists, and attendings, Innovation Recipient Impact was determined to be both a barrier and facilitator (Mixed) of the Reach domain (See Table 3).

When asked if the use of the 48DPT varied by patient factors (e.g., age, race, disease), most participants responded that patient factors had minimal or no influence. Age, disease, and patients with a complicated medical history were reported as factors that may affect the use of the 48DPT. The reported use of the 48DPT in consideration of patient factors aligns with the Innovation Adaptability CFIR construct which assesses the degree to which the intervention can be modified, tailored, or refined to fit local context or needs. Use of the 48DPT may differ based on age and disease as reported by a few participants indicating the presence of the Innovation Adaptability construct. However, since the patient factors did not influence the remainder of the participants, Innovation Adaptability was considered neither a barrier nor facilitator (Mixed) to the reach domain (See Table 3).

Unit-based leadership participants were asked if there are particular clinician groups or teams that are more or less likely to utilize the 48DPT. Responses reveal that social worker and case manager participants were more likely to use the 48DPT. Case managers often used the 48DPT as a reference point to initiate the discussion of a patient who was identified as ready for discharge during interdisciplinary rounds. For social workers, the 48DPT served as a prompt to start preparations for the discharge of a patient. Nursing was reportedly less likely to use the 48DPT and preferred to use Discharge Today, an existing discharge functionality embedded into Epic EHR. The implementation of the 48DPT differed between case managers and social workers as the information generated by them resulted in separate action steps. The varying uses of the 48DPT within different stakeholder roles indicate the presence of the Tailoring Strategies CFIR construct. This construct represents how individuals select and implement strategies to overcome barriers, utilize facilitators, and match context. Findings from the interviews indicate that the presence of the Tailoring Strategies constructs among the stakeholder roles was modest and only associated with case managers and social workers. The construct was absent in hospitalist, attending, unit medical directors, and nurse manager participants. Therefore, Tailoring Strategies was determined to be a weak facilitator to the reach of the 48DPT (See Table 3).

The Efficacy domain of the RE-AIM framework focuses on understanding the overall impact of the implementation process on primary and secondary outcomes. Participants were asked to share their thoughts about the impact of using the 48DPT process on specific outcomes. Assessing the impact of the 48DPT on specific outcomes aligns with the Reflecting & Evaluating CFIR construct. Reflecting & Evaluating is described as the extent to which individuals collect and discuss quantitative and qualitative data concerning implementation progress. The Reflecting & Evaluating construct was expanded to be assessed across five subconstructs; Perceived effectiveness, Personal judgement, Length of Stay, Initiation of discharge process, and Clinical/Patient outcomes.

Regarding how effective or accurate the 48DPT is in determining patients who are medically ready for discharge, unit-based leadership and clinician participants differed in their responses (Reflecting & Evaluating: Perceived effectiveness). Hospitalists responded that the 48DPT was accurate 60 – 70% of the time. Social workers stated that the 48DPT is effective but not consistently accurate and case managers reported that the 48DPT was accurate within an estimated range of 60%—80%. Additional responses included a lack of training or use of the 48DPT and a preference for the Discharge Today, the functionality that existed in Epic before 48DPT implementation.

When asked to compare the accuracy of the 48DPT to their personal/clinical judgment some participants (social workers, case managers, hospitalists) thought the 48DPT was about the same or slightly better at predicting discharge than their personal/clinical judgment (Reflecting & Evaluating: Personal judgment). One participant (unit medical director) thought clinical judgment was better at predicting discharge. A few social workers mentioned that the accuracy of the 48DPT result varies especially when a patient becomes medically active and the case is referred to the medical team for review. Participants were asked if the 48DPT helped decrease the Length of Stay (LOS) (Reflecting & Evaluating: Length of stay). Several participants (hospitalists, attendings, case managers, and nurse manager) responded favorably explaining that it prompts the team to initiate the discharge process earlier. Unit medical directors and social workers did not think the 48DPT helped decrease LOS as social work follow-up is needed for many patients.

Participants were asked if the 48DPT helps initiate the processes needed for discharge (e.g., scheduling appointments, and communication with outpatient providers) (Reflecting & Evaluating: Initiation of discharge process). Several participants (attendings, unit medical directors, case managers, and social workers) responded positively and thought that the 48DPT could help start the discharge process. Additionally, they reported seeing an improvement with the use of the 48DPT when asked if the 48DPT helps improve clinical care or patient outcomes (Reflecting & Evaluating: Clinical/Patient Outcomes). These participants attributed the improvement to the ability to discuss the discharge process and start preparation in a timely manner. In contrast, two attendings did not think the clinical care or patient outcomes would improve with the use of the 48DPT based on their experience.

The responses detailed above indicate the presence of each of the subconstructs for Reflecting & Evaluating as a facilitator or positive influence on the effectiveness of the 48DPT implementation process. However, since several hospitalists/attending participants reported having limited experience with the 48DPT, each of the five subconstructs was determined to be a weak facilitator of the efficacy domain (See Table 4).

The Relative Advantage construct aims to assess the performance of an intervention in comparison to current practices or alternate solutions [38]. The relative advantage construct was expanded into two subconstructs, Familiarity with Discharge Today and Perceived Value compared to Discharge Today, to examine the performance of the 48DPT compared to Discharge Today (DT), an embedded CDS tool available in Epic EHR.

Participants were asked about their awareness and experience with the DT (Relative advantage: Familiarity with Discharge Today). All participants reported being aware of the existence of the “Discharge Today” field in Epic before the study. Hospitalist, attending, unit medical director, and nurse manager participants preferred using DT in their clinical workflow while social worker and case manager participants preferred the 48DPT. These responses suggest that awareness of the DT tool is a negative influence, or barrier, to the implementation of the 48DPT. The Relative advantage: Familiarity with Discharge Today subconstruct was identified as a weak barrier because familiarity with the DT tool did not deter the adoption of the 48DPT by social workers and case managers.

When participants were asked which of the two tools was more beneficial, if the tools were redundant, or whether they each provided unique information, responses across the stakeholder roles were mixed (Relative advantage: Perceived value compared to Discharge Today). Attending, unit medical director, case manager, and nurse manager participants thought that information from the 48DPT differed from the DT tool. In contrast, Hospitalist and Social worker participants stated that information from 48DPT overlaps with information from DT and interdisciplinary rounds (IDR). Two hospitalists responded that interdisciplinary rounds (IDRs) are sufficient in determining which patients are medically ready for discharge and that both the DT tool and the 48DPT are redundant. These findings indicate several participants found the information generated by the 48DPT to be redundant with the information from the DT and IDRs which can negatively influence implementation. However, this did not deter participants from adopting the 48DPT. Therefore, the Relative advantage: Perceived value compared to the Discharge Today subconstruct was rated as a weak barrier to the implementation of the 48DPT (See Table 4).

The Adoption domain examines 48DPT usage patterns at the individual and unit levels, as well as the acceptance of participants across medical specialties and levels of clinical experience. Understanding the intended use of an intervention aligns with the Adapting construct which measures the degree to which individuals modify the innovation and/or the Inner Setting for optimal fit and integration into work processes [38]. The Adapting construct was expanded into two subconstructs to assess the Individual intended use and the Unit-based intended use of the 48DPT. When asked about the use of the 48DPT, most participants responded that the 48DPT is used as originally intended (Adapting: Individual Intended Use). A few social worker participants were unsure if the 48DPT was being used as intended. We asked hospitalists if the use of 48DPT varied by unit and one stated that the use of the 48DPT differed across hospital units depending on the role and integration with IDRs (Adapting: Unit-based intended use).

The findings discussed above indicate the presence of both Adapting subconstructs as a facilitator or positive influence on the implementation and adoption of the 48DPT. The Adapting: Individual Intended Use and Adapting: Unit-based intended use subconstructs were identified as weak facilitators of implementation because several hospitalists had awareness of the 48DPT but did not have experience using the tool (See Table 5).

The Knowledge & Beliefs CFIR construct aims to seek to examine the views and the value placed on the intervention including the familiarity with facts, truths, and principles [38]. The Knowledge & Beliefs were adapted to be assessed across four subconstructs; Clinician response to 48DPT, Response to 48DPT by clinician role, Response to 48DPT by specialty or roles, and Response to 48DPT by clinician's time in practice.

Participants in unit-based leadership roles (social workers, case managers, and unit medical directors) were asked to discuss how clinicians might react or respond if a team member announces that a discharge is predicted within the next 48 h (Knowledge & Beliefs: Clinician response to 48DPT). Most unit-based leadership participants reported that team members would be accepting and welcoming to the information. They mentioned that clinician reactions to a 48DPT result may vary depending on the clinician's role (Knowledge & Beliefs: Response to 48DPT by clinician role) and the medical specialty (Knowledge & Beliefs: Response to 48DPT by specialty or roles).

These findings suggest the presence of the Knowledge & Beliefs: Clinician response to 48DPT, Knowledge & Beliefs: Response to 48DPT by clinician role, and Knowledge & Beliefs: Response to 48DPT by specialty or role subconstructs as a facilitator or positive influence on the implementation and adoption of the 48DPT. These subconstructs were recognized as weak facilitators of implementation since some hospitalists and attendings did not use the 48DPT regularly despite having knowledge about the existence of the tool (See Table 5).

When asked if the response to the 48DPT varied according to clinician’s time in practice, two unit-based leadership participants mentioned that newer clinicians are more likely to not be aware of the 48DPT compared to more experienced clinicians (Knowledge & Beliefs: Response to 48DPT by clinician's time in practice). The lack of awareness about the 48DPT by clinicians with limited time in practice was determined to be a barrier to implementation and adoption. Since the limited knowledge did not deter the adoption of the 48DPT by less experienced clinicians, the Knowledge & Beliefs: Response to 48DPT by clinician's time in practice subconstruct was rated as a weak barrier (See Table 5).

The Implementation domain of the RE-AIM framework aims to understand the enabling and predisposing factors that support the implementation of the 48DPT. Assessing the enabling and predisposing factors aligns with the Engaging CFIR construct. This construct refers to the degree to which individuals attract and promote involvement in the implementation process and/or the intervention [38]. The Engaging construct was expanded to two subconstructs to examine its presence or absence in Individuals and Interdisciplinary Team Members. Participants were asked if they were involved in developing the process for the use of the 48DPT (Engaging: Individuals). Most participants responded that they were not involved in the development process. These responses reveal that the Engaging: Individuals subconstruct was largely absent. However, the lack of participation in the development did not hinder the adoption of the 48DPT by several participants. Therefore, the Engaging: Individuals subconstruct was rated as a weak barrier to the Implementation domain (See Table 6).

Participants were asked about the involvement of interdisciplinary team members (e.g., case managers, nurse managers, and clinicians) in the development process of the 48DPT (Engaging: Interdisciplinary Team Members). The majority of participants reported having no knowledge of the 48DPT development process or if other team members were involved. A few participants did not think that the interdisciplinary team was adequately involved. Case manager and nurse manager participants responded that the involvement of interdisciplinary team members in the development of the 48DPT was satisfactory. These findings suggest that stakeholder engagement was limited and the lack of involvement of interdisciplinary team members is a negative influence on adoption. Participants were either unaware of the development phase and the involvement of other team members or did not think the efforts to involve team members were sufficient. Therefore, the Engaging: Interdisciplinary Team Members subconstruct was determined to be a strong barrier to implementation (See Table 6).

The Compatibility CFIR construct looks at how effectively the intervention fits with current processes, systems, and practices [38]. When asked if the 48DPT was integrated well into their workflow, the responses across roles were mixed. Several participants (unit medical director, and nurse manager) responded that the 48DPT was not adopted into their workflow. These participants suggested providing training and reminders about the availability of the tool to improve integration. Some hospitalists and attendings thought that the 48DPT was somewhat integrated into their workflow due to a lack of follow-up education and efforts to increase awareness of the 48DPT after the initial rollout. A few case managers and social workers reported that the 48DPT was well-integrated. These varied responses indicate that the Compatibility construct did not have a positive or negative influence on the implementation domain and was determined to be neither a barrier nor a facilitator (Mixed) (See Table 6).

Participants were asked what support services and technical skills are needed by users of the 48DPT. This question corresponds with the Access to Knowledge & Information CFIR construct which seeks to assess the degree to which guidance and/or training are available for intervention implementation. The Access to Knowledge & Information construct was expanded into two subconstructs to evaluate the Training and Support and Computer skills required for the use of the 48DPT. In regard to support services, increasing awareness about the availability of the 48DPT was highlighted by several participants. Additionally, participants suggested providing training and demonstration for each member of the interdisciplinary team and improving the accuracy of the 48DPT. A few participants thought the 48DPT was intuitive to use and did not require additional training or support. These responses indicate the presence of the Access to Knowledge & Information: Training and Support subconstruct as a weak facilitator of the implementation domain (See Table 6).

When asked what computer skills were necessary to successfully use the 48DPT, most participants stated that basic computer skills are needed. Participants were also familiar with the general operation of the 48DPT because of their prior experience using similar programs and in their clinical workflow. These findings reveal the presence of Access to Knowledge & Information: Computer Skills subconstruct as a facilitator or positive influence on the implementation of the 48DPT. Because the majority of participants reported possessing the requisite technical abilities for the successful deployment of the 48DPT, this subconstruct was evaluated as a strong facilitator (See Table 6).

The Maintenance domain of the RE-AIM framework investigates the degree to which the 48DPT is assimilated into organizational culture and routine practice. Examining the factors related to the success of the implementation of an intervention aligns with the Assessing Context CFIR construct. The Assessing Context construct aims to assess the extent to which individuals gather information to recognize and evaluate obstacles and enablers for implementing and delivering the innovation [38]. This construct was broadened to be examined across four subconstructs; Perceived burden, Comparison of perceived benefits to burden, Perceived barriers, and Use of 48DPT over time.

Participants were asked how much of a burden using the 48DPT. Eight participants responded that the use of the 48DPT did not add a burden. When asked to compare the burden of using the 48DPT to the benefits, the majority of participants responded that the benefits outweigh any burden (Assessing Context: Comparison of perceived benefits to burden). Findings from these responses indicate the presence of the Assessing Context: Perceived burden and Assessing Context: Comparison of perceived benefits to burden subconstructs as facilitators or positive influences on the implementation and maintenance of the 48DPT. Overall, participants did not find the 48DPT to be burdensome and agreed that the benefits of using the tool overshadowed any difficulties. Therefore, the Assessing Context: Perceived burden and Assessing Context: Comparison of perceived benefits to burden subconstructs were rated as strong facilitators of 48DPT implementation (See Table 7).

When asked about the barriers to using the 48DPT, participants most frequently mentioned the lack of awareness of the tool (Assessing Context: Perceived barriers). Additional reported barriers were the lack of educational training, limited integration into IDRs, and difficulty communicating with providers. These results reveal the prevalence of the Assessing Context: Perceived barriers subconstruct and barriers such as lack of awareness about 48DPT pose a negative influence on adoption. Therefore, Assessing Context: Perceived barriers were identified as a strong barrier to the implementation of the 48DPT (See Table 7).

Participants were asked if the use of the 48DPT changed over time. Most participants responded that the use of the 48DPT has not changed since their initial use. This question was omitted for participants who reported having limited familiarity with the use of the 48DPT. These responses indicate the absence of the Assessing Context: Use of 48DPT over time subconstruct which is deemed as a barrier. Since a number of participants had minimal experience with the 48DPT, the Assessing Context: Use of 48DPT over time subconstruct was identified as a weak barrier to the implementation and maintenance of the 48DPT (See Table 7).

The Goals & Feedback CFIR construct refers to the degree to which objectives are explicitly stated, acted upon, and given back to team members, as well as the alignment of that feedback with goals [38]. When asked if any efforts were made to obtain feedback about the 48DPT in order to make changes, a few participants reported attempts from leaders. Most participants were not aware of any opportunities to share their feedback about the 48DPT with leaders. The findings from these responses reveal that the presence of the Goals and feedback construct was minimal. The limited efforts to gather feedback and address concerns about the use of 48DPT were determined to be a barrier or negative influence on the implementation process of the 48DPT. Therefore, the Goals & Feedback construct was recognized as a strong barrier (See Table 7).

The Available Resources CFIR construct examines the availability of resources to implement and deliver the innovation. We asked participants if any tools helped improve their use of the 48DPT (e.g., training sessions, individual feedback, unit-based metrics). The unit-based metrics and feedback and discussions with the clinical team about discharge were found to be helpful for some participants. Most participants did not report any tools that were beneficial or did not have extensive experience with the 48DPT. These results highlight the absence of the Available Resources construct which is deemed as a negative influence or barrier to implementation. However, the lack of accessibility to support the use of the 48DPT did not hinder the implementation and adoption. Therefore, the Available Resources construct was recognized as a weak barrier (See Table 7).

Participants were asked how the use of the 48DPT could be improved. This question corresponds with the Assessing Needs: Innovation Recipients construct which aims to examine the extent to which the individual(s) gather information about the priorities, preferences, and needs of recipients to guide the implementation and delivery of the intervention. Increasing awareness and education of the 48DPT was a frequent suggestion, especially among hospitalists, nurse managers, and social workers. Participants also suggested including consideration of social factors as part of the algorithm and improving the accuracy of the 48DPT. This question was omitted for participants who reported having minimal use of the 48DPT. Findings from these responses suggest the presence of the Assessing Needs: Innovation Recipients construct as a facilitator or positive influence on the implementation process. A few participants had minimal experience with the 48DPT, therefore the Assessing Needs: Innovation Recipients construct was rated as a weak facilitator to the implementation and maintenance.