Abstract

Background: Transition of care (ToC) is a critical time when pertinent patient information can be lost between providers, potentially effecting outcomes.

Objective: The aim of this research was to assess:
a) Anesthesiology residents’ completeness of ToC in a crisis situation, and
b) whether using a cognitive aid might improve the ToC.

Methods: Simulation scenarios were developed for anesthesiology residents to lead a team through a crisis situation, perform a ToC to the critical care team without a cognitive aid (Baseline), and a similar scenario 10 months later with a Cognitive Aid (CA). In the simulated scenario, the residents responded to a deteriorating patient, stabilized the patient, and transferred care to the critical care provider. Residents then completed a questionnaire to assess patient information retention. Faculty from two institutions reviewed the recordings to assess completeness and quality of the ToC.

Results: Out of 36 PGY2-4 anesthesiology residents, 14 participated (39%). Resident questionnaire results after the Baseline simulation indicated that residents were able to recall 65.7±17.9% of patient information. Use of the CA increased the percentage of correct answers to 91.4 ± 9.5% (p<0.01). During baseline assessment, approximately 30% of pertinent patient information was lost during the ToC. The use of a CA 10 months later was associated with a significant increase in the CS by 0.19±0.08 (p=0.02) and overall quality of the ToC.

Conclusion: Critical patient information transfer was more complete and higher quality in the ToC process of a simulated patient crisis when residents used a cognitive aid.

Keywords: Graduate Medical Education; Transition of Care; Simulation; Cognitive Aid; Objective Assessment; Milestones

Introduction

Transfer of patient care between providers is key to safe medical practice and loss of patient information during this process may contribute to undesirable outcomes [1]. With the increase in patient handovers, it is essential that we reduce information loss [2,3]. Research in Transition of Care (ToC) has focused on the handoff quality occurring at shift-ends, which is usually a censusbased process occurring in a structured fashion at a standardized time and location [4-11]. The quality of ToC in crisis situations has not been well assessed [12]. Simulation based research may be an ideal methodology to assess ToC and evaluate options to improve this process [13,14]. A recent study in graduating medical students indicated a significant information loss during handoff after simulated crisis [15].

It could be postulated that this loss of information is more common among people of less experience, yet recent publications in anesthesiology indicate that regardless of training level, frequency of intraoperative handoffs negatively correlates with patient outcome [16,17]. The impact of anesthesia handover frequency on patient outcomes might be related to information lost during critical times, such as during patient crisis situations [15-17]. Cognitive aids decrease loss of information during ToC [8-19]. The use of cognitive aids during crisis has been shown to assist providers in performing and prioritizing better in high stress, time sensitive situations [20-23]. However, compliance with using these cognitive aids has been low [24]. We hypothesized that significant information is lost during the ToC and that the use of a cognitive aid would improve the completeness of the ToC following a crisis simulation.

The aim of this research was to assess:
1. anesthesiology residents’ completeness of ToC in a crisis situation, and
2. whether use of a cognitive aid improves the ToC quality.

Methods

After Institutional Review Board approval, anesthesiology residents in Post-Graduate Year (PGY) 2-4 from the University of Kentucky (UK) were given the option to participate in the project. All participants provided written consent for data analysis.

Simulation Scenario

The authors developed, by consensus, three patient crisis simulation scenarios. The scenarios, similar in crisis complexity, were randomly assigned to the participants. In each scenario, the anesthesiology resident led a rapid response team through a patient code. After stabilization, the resident transported the patient to the critical care unit (ICU) and transferred care to the critical care physician (Figure 1).

Figure 1: The figure illustrates the individual distribution (n=14) of completeness score of the Transition of Care process without (pre) and with Cognitive Aid (with CA). The completeness score is expressed as a percentage of maximum score. Individuals with improved performance are shown with solid line (n=13), while participants with ‘pre’ lower than ‘with CA’ score are shown with a dashed line (n=1).

Prior to participation in the scenario, each resident reviewed the ToC process by viewing a 15-minute instructional video produced by the authors. The video was a “just-in-time” teaching method providing instruction immediately prior to application 25. It outlined ToC knowledge components, structure, and demonstrated a correctly performed ToC in a critical care setting. The simulation used a computer- controlled mannequin (Laerdal Sim Man 3GTM) and a monitor continuously showing the patient’s vital signs in a simulated hospital setting. The scenario began when the rapid response team, led by the anesthesiology resident, arrived at the bedside of a deteriorating patient. The resident-initiated communication with the bedside nurse (Confederate 1) to obtain patient information and delegated tasks to other team members. When the resident determined sufficient patient information had been received, a read-back from the confederate was performed. The standardized read-back included all pertinent information about the patient in an organized format, ensuring that all residents received identical, complete information. In the next stage of the scenario, the patient deteriorated into cardiac arrest requiring resuscitation. After resuscitation, the resident was then instructed to transport the patient to the ICU. At arrival the resident transferred care to the ICU physician (Confederate 2). The scenario was videotaped with voice recording for data analysis.

Evaluation

Questionnaire: After completing the simulation scenario, the resident completed a ten-question multiple-choice questionnaire about patient information obtained in the initial patient evaluation and confederate read-back. This format was chosen to assess the amount of information recalled from the scenario.

Expert anesthesiology faculty from two institutions (University of Kentucky; Lexington KY [n=4], University of Louisville; Louisville, KY [n=3]) reviewed the audio recordings from the scenario to assess the transfer of care process for both completeness and quality (Communication, Organization, Team Leadership) of professional interactions. The assessment was completed using a checklist with scenario-specific content and a modified Likert Scale to provide an overall handover rating. The faculty had participated in previous ToC simulation projects and had extensive experience in rating ToC quality using similar checklists. All seven faculty reviewed the voice recordings and evaluated the ToC performance independently. Four faculty members were familiar with the residents (UK), while the other three faculty members were not (UL).

Completeness Score (CS): All faculty used scenario-specific checklists to grade the completeness of the ToC report to the ICU provider. The CS indicated the fraction of complete and correct information transferred to the provider.

Cognitive Aid

The authors utilized a cognitive aid, validated in previous research, that was developed by faculty consensus. [15-27]. It was provided to the resident in the second encounter upon entrance into the simulation room.

Statistical Analysis

All scores are reported as mean ± SD. Analysis of variance was performed to identify statistical differences due to training level or assigned scenario. Data were analyzed independently for faculty evaluations from both institutions. For parametric data, the paired t-test was used to assess statistical significance. Statistical significance was assumed if p<0.05. Regression analysis was used to assess inter-rater reliability amongst each faculty evaluation of the ToC process.

Results

A total of 14 of 36 PGY2-4 anesthesiology residents (39%) participated in the project at both, Baseline and in ToC CA.

Questionnaire

The resident questionnaire results immediately after the Baseline encounter demonstrated that residents able to recall only 65.7±17.9% of patient information. In the second simulation [ToC CA], the use of a cognitive aid increased the percentage of correctly answered questions to 91.4 ± 9.5% (p<0.01).

Completeness Score (CS)

CS based on rating of faculty from the training institution (UK) and the non-training institution (UL) are shown in Figure 2. The faculty assessment for the Baseline scenario indicated a CS of 0.67 ± 0.16 (UK) or 0.72 ± 0.17 (UL). On average, UK faculty reported a lower CS by 0.04 than UL faculty. There was substantial information lost during the ToC. This overall assessment was unaffected by the observation that the training institution faculty scored the residents lower than the non-training institution faculty. CSs were improved during ToC when using the cognitive aid. The CS change from Baseline to ToC CA as rated by faculty of the training institution (UK) was not statistically different from the change reported by faculty from the non-training institution (UL)[p=0.87]. Averaging the scores from UK and UL faculty, the use of the cognitive aid was associated with a significant increase in the CS by 0.19±0.08 (p=0.02).

Figure 2: The figure illustrates the individual distribution (n=14) of the communication score of the Transition of Care process without (pre) and with Cognitive Aid (with CA). The score is expressed a percentage of maximum score. Individuals with improved or unchanged performance are shown with solid line (n=12), while participants with ‘pre’ lower than ‘with CA’ score are shown with a dashed line (n=2).

Communication, Information Organization and Team Leadership Skills Assessment

Scores for communication skills, information organization and team leadership skills during the scenario rated by faculty from the training institution (UK) and the non- training institution (UL). An analysis comparing score ratings between the faculty from UK and faculty from UL demonstrated significant differences between the assigned scores in all categories, with UK faculty reporting lower scores on average in all three skill categories. However, the overall assessment indicated all professional attributes were rated higher when a cognitive aid was used. Based on faculty ratings from both institutions, the average scores for communication increased significantly from Baseline to ToC CA by 0.08, and the information organization and team leadership scores were significantly higher in ToC CA than at Baseline.

Influence of Training Level

Combined CS for Baseline and ToC CA measurement were significantly influenced by the training level. Junior residents (PGY2) residents were less complete in ToC than more advanced residents (PGY3 and 4) (p=0.05). Scores for communication skills, information organization and team leadership were combined as professional competency. This global measurement of professional competency improved with training level indicating that senior residents are more professionally proficient than junior residents (p=0.03). The increase in CS and professional competency score from Baseline to ToC CA was not associated with resident training level. All resident training levels benefitted from using the cognitive aid.

Inter-Rater Reliability

The association between the professional competency scores. One set of ratings from the non-training institution (UL-2) was not related to any other rater (training or non-training institution); however, UL-1 and UL-3 were closely related (Figure 3).

Figure 3: The figure illustrates the individual distribution (n=14) of the leadership score of the Transition of Care process without (pre) and with Cognitive Aid (with CA). The score is expressed as a percentage of maximum score. Individuals with improved or unchanged performance are shown with solid line (n=13), while participants with ‘pre’ lower than ‘with CA’ score are shown with a dashed line (n=1).

Discussion

Accurate and complete information transfer during the ToC process is essential for safe patient care. With increased frequency in patient handovers and the link between communication failures and medical errors [2-29], there is a need for increased understanding of this process [30,31]. Several cognitive aids have been developed to improve the thoroughness of the handover process [6-32], yet the majority of these tools were designed for use during shift transition, not during critical scenarios 5. Failures of communication occur more frequently surrounding times of crisis [12-33], therefore, our goal was to assess the ToC in a crisis simulation and attempt to improve information transfer with a cognitive aid.

The main findings of our study are:
1. Anesthesiology residents with clinical experience and video-based training performed the ToC process following a crisis situation with significant loss of information, and
2. The use of a cognitive aid improved the ToC completeness and communication, organization, and team leadership following a crisis situation.
Our findings raise awareness that substantial information is lost during the ToC process. Previous studies indicate that the frequency of intraoperative handoff affects the morbidity and mortality of patients [16-17]. However, the underlying factors are not clear. Considering our findings, the information lost during ToC might contribute to poor outcomes. The Accreditation Council for Graduate Medical Education (ACGME) has recognized the importance of training residents in ToC [34]. Additionally, the ACGME Clinical Learning Environment Review (CLER) was also developed with a goal of promoting a healthy learning environment to provide safe patient care, with one of its six focus areas being “Transitions in Care” [35].
The use of a cognitive aid in our study was associated with improved completeness of information transfer from 65.7% when it wasn’t utilized, to 91.4% when it was. In the last several years, many cognitive aids have been developed attempting to improve handover quality [8-36]. Although there have been many cognitive aids drafted, no specific formatting has been universally adopted. With many formats available, it seems plausible that providers could choose any available cognitive aid suitable to their needs. As confirmed by our results, and other published improvements in ToC with cognitive aids, patient outcomes will likely benefit from their use [8-18].

There are several limitations in our study including a limited sample size within a single institution. Participants encountered similar simulations for both measurements (baseline and with cognitive aid) so some of the improvement observed may be related to retesting. The relative familiarity of a resident with the cognitive aid could have affected our results. However, residents were not involved in the development of the cognitive aid and it is not routinely used at our hospital. Timing could also be a limitation. Since a repeat study was performed with the same resident population ten months following the Baseline study, the residents’ ToC

knowledge and skills may have improved in the interim, confounding whether the improvement was due to the use of a cognitive aid. There was improvement in the overall ToC process in our single-center, simulation- based study when a cognitive aid was used. A cognitive aid is beneficial in providing more complete information transfer between care providers, decreasing communication breakdown, and potentially improving patient outcomes. Further educational research is clearly needed to determine the best evidence for improving ToC to enhance patient safety.

References

• (2007) Joint Commission on Accreditation of Healthcare Organizations: National Patient Safety Goals Hospital Version Manual Chapter, including Implementation Expectations.
• Desai SV, Feldman L, Brown L, Dezube R, Yeh HC, et al. (2013) Effect of the 2011 vs 2003 duty hour regulation–compliant models on sleep duration, trainee education, and continuity of patient care among internal medicine house staff: A randomized trial. JAMA Intern Med 173(8): 649-655.
• Van Eaton EG, Horvath KD, Lober WB, Rossini AJ, Pellegrini CA (2005) A randomized, controlled trial evaluating the impact of a computerized rounding and sign-out system on continuity of care and resident work hours. J Am Coll Surg 200(4): 538-545.
• Habicht R, Block L, Silva KN, Oliver N, Wu A, et al. (2015) Assessing Intern Handover Processes. Clin Teach 12: 1-5.
• Wohlauer M (2012) Fragmented care in the era of limited work hours: A plea for an explicit handover curriculum. BMJ Qual Saf 21: i16-i18.
• Starmer AJ, Sectish TC, Simon DW, Keohane C, McSweeney ME, et al. (2013) Rates of medical errors and preventable adverse events among hospitalized children following implementation of a resident handoff bundle. JAMA 310(21): 2262-2270.
• Abraham J, Kannampallil TG, Almoosa KF, Patel B, Patel V (2014) Comparative evaluation of the content and structure of communication using two handoff tools: Implications for patient safety. J Crit Care 29(2): 311.e1-7.
• Abraham J, Kannampallil T, Patel B, Almoosa K, Patel V (2012) Ensuring patient safety in care transitions: An empirical evaluation of a handoff intervention tool. AMIA Ann Symp Proc 2012: 17-26.
• Flanagan ME, Patterson ES, Frankel RM, Doebbeling BN (2009) Evaluation of a physician informatics tool to improve patient handoffs. J Am Med Inform Assoc 16(4): 509-515.
• Patterson ES, Roth EM, Woods DD, Chow R, Gomes JO (2004) Handoff strategies in settings with high consequences for failure: Lessons for health care operations. Int J Qual Health Care 16(2): 125-132.
• Arora V, Johnson J (2006) A model for building a standardized hand-off protocol. Jt Comm J Qual Patient Saf 32(11): 646-655.
• Greenberg CC, Regenbogen SE, Studdert DM, Lipsitz SR, Rogers SO, et al. (2007) Patterns of communication breakdowns resulting in injury to surgical patients. J Am Coll Surg 204(4): 533-540.
• Gaba DM (2010) Crisis resource management and teamwork training in anaesthesia. Br J Anaesth 105(1): 3-6.
• Lateef F (2010) Simulation-based learning: Just like the real thing. J Emerg Trauma Shock 3(4): 348-352.
• Bauer B, Rebel A, DiLorenzo A, Schell RM, Dority JS, et al. (2016) Cognitive Aid Use Improves Transition of Care by Graduating Medical Students During a Simulated Crisis. Med Educ Online 21: 32118.
• Saager L, Hesler BD, You J, Turan A, Mascha EJ, et al. (2014) Intraoperative Transitions of Anesthesia Care and Postoperative Adverse Outcomes. Anesthesiology 121: 695-706.
• Hudson CC, McDonald B, Hudson JK, Tran D, Boodhwani M (2015) Impact of anesthetic handover on mortality and morbidity in cardiac surgery: A cohort study. J Cardiothor Vasc Anesth 29(1): 11-16.
• Potestio C, Mottla J, Kelley E, DeGroot K (2015) Improving Post Anesthesia Care Unit (PACU) handoff by implementing a succinct checklist. APSF Newsletter 13-15.
• Abraham J, Kannampallil T, Patel VL (2014) A systematic review of the literature on the evaluation of handoff tools: Implications for research and practice. J Am Med Inform Assoc 21(1): 154-162.
• Marshall S (2013) The use of cognitive aids during emergencies in anesthesia: A review of the literature. Anesth Analg 117(5): 1162-1171.
• Marshall SD, Mehra R (2014) The effects of a displayed cognitive aid on non- technical skills in a simulated 'can't intubate, can't oxygenate' crisis. Anaesthesia 69(7): 669-677.
• Marshall S, Harrison J, Flanagan B (2009) The Teaching of a structured tool improves the clarity and content of interprofessional clinical communication. Qual Saf Health Care 18(2): 137-140.
• Shaffner DH, Heitmiller ES, Deshpande JK (2013) Pediatric perioperative life support. Anesth Analg 117(4): 960-979.
• Watkins SC, Anders S, Clebone A, Hughes E, Zeigler L, et al. (2016) Paper or plastic? Simulation based evaluation of two versions of a cognitive aid for managing pediatric peri-operative critical events by anesthesia trainees: evaluation of the society for pediatric anesthesia emergency checklist. J Clin Monit Comput 30(3): 275-283.
• Chueh H, Barnett GO (1997) “Just-in-time” clinical information. Acad Med 72(6): 512- 517.
• Rebel A, DiLorenzo A, Sloan PA, Fragneto R, Lukens F, et al. (2014) Development of a cognitive aid to improve the quality of transition of care in crisis situations. Anesth Analg 118(5S): S-245.
• Rebel A, DiLorenzo A, Sloan PA, Latif R, Christie KB, et al. (2015) Use of a cognitive aid improves resident transition of care process quality in simulated crisis situation. Anesth Analg 120(3S): S-131.
• (2012) Joint Commission Center for Transforming Healthcare Releases Targeted Solutions Tool for Hand-Off. Jt Comm Perspect 32(8): 1-3.
• (2012) Joint Commission Center for Transforming Healthcare Releases Targeted Solutions Tool for Hand-Off Communications. Jt Comm Perspect® 32: 8.
• (2008) NTOCC Measures Work Group. Transitions of Care Measures. National Transitions of Care Coalition.
• (2007) World Health Organization Collaborating Center for Patient Safety: Communication during patient handovers. Geneva, Switzerland, WHO Press.
• Blum RH, Raemer DB, Carroll JS, Dufresne RL, Cooper JB (2005) A method for measuring the effectiveness of simulation-based team training for improving communication skills. Anesth Analg100(5): 1375-1380.
• Johnston M, Arora S, King D, Stroman L, Darzi A (2014) Escalation of care and failure to rescue: A multicenter, multiprofessional qualitative study. Surgery 155(6): 989-994.
• (2016) Accreditation Council for Graduate Medical Education. Common Program Requirements.
• Weiss KB, Wagner R, Nasca TJ (2012) Development, testing and implementation of the ACGME Clinical Learning Environment Review (CLER) Program. J Grad Med Educ 4(3): 396-398.
• Fabila TS, Hee HI, Sultana R, Assam PN, Kiew A, et al. (2016) Improving Postoperative handover from anaesthetists to non-anaesthetists in a children’s intensive care unit: The receiver’s perception. Singapore Med J 57(5): 242-253.

Abstract

Background: Transition of care (ToC) is a critical time when pertinent patient information can be lost between providers, potentially effecting outcomes.

Objective: The aim of this research was to assess:
a) Anesthesiology residents’ completeness of ToC in a crisis situation, and
b) whether using a cognitive aid might improve the ToC.

Methods: Simulation scenarios were developed for anesthesiology residents to lead a team through a crisis situation, perform a ToC to the critical care team without a cognitive aid (Baseline), and a similar scenario 10 months later with a Cognitive Aid (CA). In the simulated scenario, the residents responded to a deteriorating patient, stabilized the patient, and transferred care to the critical care provider. Residents then completed a questionnaire to assess patient information retention. Faculty from two institutions reviewed the recordings to assess completeness and quality of the ToC.

Results: Out of 36 PGY2-4 anesthesiology residents, 14 participated (39%). Resident questionnaire results after the Baseline simulation indicated that residents were able to recall 65.7±17.9% of patient information. Use of the CA increased the percentage of correct answers to 91.4 ± 9.5% (p<0.01). During baseline assessment, approximately 30% of pertinent patient information was lost during the ToC. The use of a CA 10 months later was associated with a significant increase in the CS by 0.19±0.08 (p=0.02) and overall quality of the ToC.

Conclusion: Critical patient information transfer was more complete and higher quality in the ToC process of a simulated patient crisis when residents used a cognitive aid.

Keywords: Graduate Medical Education; Transition of Care; Simulation; Cognitive Aid; Objective Assessment; Milestones