info@biomedres.us   +1 (720) 414-3554
  One Westbrook Corporate Center, Suite 300, Westchester, IL 60154, USA

Biomedical Journal of Scientific & Technical Research

September, 2019, Volume 22, 1, pp 16356-16362

Research Article

Research Article

Use of a Cognitive Aid during Simulated Patient Crisis Improves the Quality of the Transition of Care Process

Annette Rebel1*, Brooke Bauer1, Amy DiLorenzo1, Paul A Sloan1, Regina Y Fragneto1, Faith Lukens1, KariBeth Christie2, Rana Latif2-4, Zaki Udin Hassan1, Mitzi Schumacher5, and Randall M Schell1

Author Affiliations

1Department of Anesthesiology, USA

2Department of Anesthesiology and Perioperative Medicine, USA

3Paris Simulation Center, Office of Undergraduate Medical Education, USA

4Outcomes Research Consortium, USA

5Department of Behavioral Science, USA

Received: September 27, 2019 | Published: October 16, 2019

Corresponding author: Annette Rebel, Department of Anesthesiology, USA

DOI: 10.26717/BJSTR.2019.22.003693

Abstract

Background: Transition of care (ToC) is a critical time when pertinent patient information can be lost between providers, potentially effecting outcomes.

Objective: The aim of this research was to assess:
a) Anesthesiology residents’ completeness of ToC in a crisis situation, and
b) whether using a cognitive aid might improve the ToC.

Methods: Simulation scenarios were developed for anesthesiology residents to lead a team through a crisis situation, perform a ToC to the critical care team without a cognitive aid (Baseline), and a similar scenario 10 months later with a Cognitive Aid (CA). In the simulated scenario, the residents responded to a deteriorating patient, stabilized the patient, and transferred care to the critical care provider. Residents then completed a questionnaire to assess patient information retention. Faculty from two institutions reviewed the recordings to assess completeness and quality of the ToC.

Results: Out of 36 PGY2-4 anesthesiology residents, 14 participated (39%). Resident questionnaire results after the Baseline simulation indicated that residents were able to recall 65.7±17.9% of patient information. Use of the CA increased the percentage of correct answers to 91.4 ± 9.5% (p<0.01). During baseline assessment, approximately 30% of pertinent patient information was lost during the ToC. The use of a CA 10 months later was associated with a significant increase in the CS by 0.19±0.08 (p=0.02) and overall quality of the ToC.

Conclusion: Critical patient information transfer was more complete and higher quality in the ToC process of a simulated patient crisis when residents used a cognitive aid.

Keywords: Graduate Medical Education; Transition of Care; Simulation; Cognitive Aid; Objective Assessment; Milestones

Introduction

Transfer of patient care between providers is key to safe medical practice and loss of patient information during this process may contribute to undesirable outcomes [1]. With the increase in patient handovers, it is essential that we reduce information loss [2,3]. Research in Transition of Care (ToC) has focused on the handoff quality occurring at shift-ends, which is usually a censusbased process occurring in a structured fashion at a standardized time and location [4-11]. The quality of ToC in crisis situations has not been well assessed [12]. Simulation based research may be an ideal methodology to assess ToC and evaluate options to improve this process [13,14]. A recent study in graduating medical students indicated a significant information loss during handoff after simulated crisis [15].

It could be postulated that this loss of information is more common among people of less experience, yet recent publications in anesthesiology indicate that regardless of training level, frequency of intraoperative handoffs negatively correlates with patient outcome [16,17]. The impact of anesthesia handover frequency on patient outcomes might be related to information lost during critical times, such as during patient crisis situations [15-17]. Cognitive aids decrease loss of information during ToC [8-19]. The use of cognitive aids during crisis has been shown to assist providers in performing and prioritizing better in high stress, time sensitive situations [20-23]. However, compliance with using these cognitive aids has been low [24]. We hypothesized that significant information is lost during the ToC and that the use of a cognitive aid would improve the completeness of the ToC following a crisis simulation.

The aim of this research was to assess:
1. anesthesiology residents’ completeness of ToC in a crisis situation, and
2. whether use of a cognitive aid improves the ToC quality.

Methods

After Institutional Review Board approval, anesthesiology residents in Post-Graduate Year (PGY) 2-4 from the University of Kentucky (UK) were given the option to participate in the project. All participants provided written consent for data analysis.

Simulation Scenario

The authors developed, by consensus, three patient crisis simulation scenarios. The scenarios, similar in crisis complexity, were randomly assigned to the participants. In each scenario, the anesthesiology resident led a rapid response team through a patient code. After stabilization, the resident transported the patient to the critical care unit (ICU) and transferred care to the critical care physician (Figure 1).

Figure 1: The figure illustrates the individual distribution (n=14) of completeness score of the Transition of Care process without (pre) and with Cognitive Aid (with CA). The completeness score is expressed as a percentage of maximum score. Individuals with improved performance are shown with solid line (n=13), while participants with ‘pre’ lower than ‘with CA’ score are shown with a dashed line (n=1).

Prior to participation in the scenario, each resident reviewed the ToC process by viewing a 15-minute instructional video produced by the authors. The video was a “just-in-time” teaching method providing instruction immediately prior to application 25. It outlined ToC knowledge components, structure, and demonstrated a correctly performed ToC in a critical care setting. The simulation used a computer- controlled mannequin (Laerdal Sim Man 3GTM) and a monitor continuously showing the patient’s vital signs in a simulated hospital setting. The scenario began when the rapid response team, led by the anesthesiology resident, arrived at the bedside of a deteriorating patient. The resident-initiated communication with the bedside nurse (Confederate 1) to obtain patient information and delegated tasks to other team members. When the resident determined sufficient patient information had been received, a read-back from the confederate was performed. The standardized read-back included all pertinent information about the patient in an organized format, ensuring that all residents received identical, complete information. In the next stage of the scenario, the patient deteriorated into cardiac arrest requiring resuscitation. After resuscitation, the resident was then instructed to transport the patient to the ICU. At arrival the resident transferred care to the ICU physician (Confederate 2). The scenario was videotaped with voice recording for data analysis.

Evaluation

Questionnaire: After completing the simulation scenario, the resident completed a ten-question multiple-choice questionnaire about patient information obtained in the initial patient evaluation and confederate read-back. This format was chosen to assess the amount of information recalled from the scenario.

Expert anesthesiology faculty from two institutions (University of Kentucky; Lexington KY [n=4], University of Louisville; Louisville, KY [n=3]) reviewed the audio recordings from the scenario to assess the transfer of care process for both completeness and quality (Communication, Organization, Team Leadership) of professional interactions. The assessment was completed using a checklist with scenario-specific content and a modified Likert Scale to provide an overall handover rating. The faculty had participated in previous ToC simulation projects and had extensive experience in rating ToC quality using similar checklists. All seven faculty reviewed the voice recordings and evaluated the ToC performance independently. Four faculty members were familiar with the residents (UK), while the other three faculty members were not (UL).

Completeness Score (CS): All faculty used scenario-specific checklists to grade the completeness of the ToC report to the ICU provider. The CS indicated the fraction of complete and correct information transferred to the provider.

Cognitive Aid

The authors utilized a cognitive aid, validated in previous research, that was developed by faculty consensus. [15-27]. It was provided to the resident in the second encounter upon entrance into the simulation room.

Statistical Analysis

All scores are reported as mean ± SD. Analysis of variance was performed to identify statistical differences due to training level or assigned scenario. Data were analyzed independently for faculty evaluations from both institutions. For parametric data, the paired t-test was used to assess statistical significance. Statistical significance was assumed if p<0.05. Regression analysis was used to assess inter-rater reliability amongst each faculty evaluation of the ToC process.

Results

A total of 14 of 36 PGY2-4 anesthesiology residents (39%) participated in the project at both, Baseline and in ToC CA.

Questionnaire

The resident questionnaire results immediately after the Baseline encounter demonstrated that residents able to recall only 65.7±17.9% of patient information. In the second simulation [ToC CA], the use of a cognitive aid increased the percentage of correctly answered questions to 91.4 ± 9.5% (p<0.01).

Completeness Score (CS)

CS based on rating of faculty from the training institution (UK) and the non-training institution (UL) are shown in Figure 2. The faculty assessment for the Baseline scenario indicated a CS of 0.67 ± 0.16 (UK) or 0.72 ± 0.17 (UL). On average, UK faculty reported a lower CS by 0.04 than UL faculty. There was substantial information lost during the ToC. This overall assessment was unaffected by the observation that the training institution faculty scored the residents lower than the non-training institution faculty. CSs were improved during ToC when using the cognitive aid. The CS change from Baseline to ToC CA as rated by faculty of the training institution (UK) was not statistically different from the change reported by faculty from the non-training institution (UL)[p=0.87]. Averaging the scores from UK and UL faculty, the use of the cognitive aid was associated with a significant increase in the CS by 0.19±0.08 (p=0.02).

Figure 2: The figure illustrates the individual distribution (n=14) of the communication score of the Transition of Care process without (pre) and with Cognitive Aid (with CA). The score is expressed a percentage of maximum score. Individuals with improved or unchanged performance are shown with solid line (n=12), while participants with ‘pre’ lower than ‘with CA’ score are shown with a dashed line (n=2).

Communication, Information Organization and Team Leadership Skills Assessment

Scores for communication skills, information organization and team leadership skills during the scenario rated by faculty from the training institution (UK) and the non- training institution (UL). An analysis comparing score ratings between the faculty from UK and faculty from UL demonstrated significant differences between the assigned scores in all categories, with UK faculty reporting lower scores on average in all three skill categories. However, the overall assessment indicated all professional attributes were rated higher when a cognitive aid was used. Based on faculty ratings from both institutions, the average scores for communication increased significantly from Baseline to ToC CA by 0.08, and the information organization and team leadership scores were significantly higher in ToC CA than at Baseline.

Influence of Training Level

Combined CS for Baseline and ToC CA measurement were significantly influenced by the training level. Junior residents (PGY2) residents were less complete in ToC than more advanced residents (PGY3 and 4) (p=0.05). Scores for communication skills, information organization and team leadership were combined as professional competency. This global measurement of professional competency improved with training level indicating that senior residents are more professionally proficient than junior residents (p=0.03). The increase in CS and professional competency score from Baseline to ToC CA was not associated with resident training level. All resident training levels benefitted from using the cognitive aid.

Inter-Rater Reliability

The association between the professional competency scores. One set of ratings from the non-training institution (UL-2) was not related to any other rater (training or non-training institution); however, UL-1 and UL-3 were closely related (Figure 3).

Figure 3: The figure illustrates the individual distribution (n=14) of the leadership score of the Transition of Care process without (pre) and with Cognitive Aid (with CA). The score is expressed as a percentage of maximum score. Individuals with improved or unchanged performance are shown with solid line (n=13), while participants with ‘pre’ lower than ‘with CA’ score are shown with a dashed line (n=1).

Discussion

Accurate and complete information transfer during the ToC process is essential for safe patient care. With increased frequency in patient handovers and the link between communication failures and medical errors [2-29], there is a need for increased understanding of this process [30,31]. Several cognitive aids have been developed to improve the thoroughness of the handover process [6-32], yet the majority of these tools were designed for use during shift transition, not during critical scenarios 5. Failures of communication occur more frequently surrounding times of crisis [12-33], therefore, our goal was to assess the ToC in a crisis simulation and attempt to improve information transfer with a cognitive aid.

The main findings of our study are:
1. Anesthesiology residents with clinical experience and video-based training performed the ToC process following a crisis situation with significant loss of information, and
2. The use of a cognitive aid improved the ToC completeness and communication, organization, and team leadership following a crisis situation.
Our findings raise awareness that substantial information is lost during the ToC process. Previous studies indicate that the frequency of intraoperative handoff affects the morbidity and mortality of patients [16-17]. However, the underlying factors are not clear. Considering our findings, the information lost during ToC might contribute to poor outcomes. The Accreditation Council for Graduate Medical Education (ACGME) has recognized the importance of training residents in ToC [34]. Additionally, the ACGME Clinical Learning Environment Review (CLER) was also developed with a goal of promoting a healthy learning environment to provide safe patient care, with one of its six focus areas being “Transitions in Care” [35].
The use of a cognitive aid in our study was associated with improved completeness of information transfer from 65.7% when it wasn’t utilized, to 91.4% when it was. In the last several years, many cognitive aids have been developed attempting to improve handover quality [8-36]. Although there have been many cognitive aids drafted, no specific formatting has been universally adopted. With many formats available, it seems plausible that providers could choose any available cognitive aid suitable to their needs. As confirmed by our results, and other published improvements in ToC with cognitive aids, patient outcomes will likely benefit from their use [8-18].

There are several limitations in our study including a limited sample size within a single institution. Participants encountered similar simulations for both measurements (baseline and with cognitive aid) so some of the improvement observed may be related to retesting. The relative familiarity of a resident with the cognitive aid could have affected our results. However, residents were not involved in the development of the cognitive aid and it is not routinely used at our hospital. Timing could also be a limitation. Since a repeat study was performed with the same resident population ten months following the Baseline study, the residents’ ToC

knowledge and skills may have improved in the interim, confounding whether the improvement was due to the use of a cognitive aid. There was improvement in the overall ToC process in our single-center, simulation- based study when a cognitive aid was used. A cognitive aid is beneficial in providing more complete information transfer between care providers, decreasing communication breakdown, and potentially improving patient outcomes. Further educational research is clearly needed to determine the best evidence for improving ToC to enhance patient safety.

References

Research Article

Use of a Cognitive Aid during Simulated Patient Crisis Improves the Quality of the Transition of Care Process

Annette Rebel1*, Brooke Bauer1, Amy DiLorenzo1, Paul A Sloan1, Regina Y Fragneto1, Faith Lukens1, KariBeth Christie2, Rana Latif2-4, Zaki Udin Hassan1, Mitzi Schumacher5, and Randall M Schell1

Author Affiliations

1Department of Anesthesiology, USA

2Department of Anesthesiology and Perioperative Medicine, USA

3Paris Simulation Center, Office of Undergraduate Medical Education, USA

4Outcomes Research Consortium, USA

5Department of Behavioral Science, USA

Received: September 27, 2019 | Published: October 16, 2019

Corresponding author: Annette Rebel, Department of Anesthesiology, USA

DOI: 10.26717/BJSTR.2019.22.003693

Abstract

Background: Transition of care (ToC) is a critical time when pertinent patient information can be lost between providers, potentially effecting outcomes.

Objective: The aim of this research was to assess:
a) Anesthesiology residents’ completeness of ToC in a crisis situation, and
b) whether using a cognitive aid might improve the ToC.

Methods: Simulation scenarios were developed for anesthesiology residents to lead a team through a crisis situation, perform a ToC to the critical care team without a cognitive aid (Baseline), and a similar scenario 10 months later with a Cognitive Aid (CA). In the simulated scenario, the residents responded to a deteriorating patient, stabilized the patient, and transferred care to the critical care provider. Residents then completed a questionnaire to assess patient information retention. Faculty from two institutions reviewed the recordings to assess completeness and quality of the ToC.

Results: Out of 36 PGY2-4 anesthesiology residents, 14 participated (39%). Resident questionnaire results after the Baseline simulation indicated that residents were able to recall 65.7±17.9% of patient information. Use of the CA increased the percentage of correct answers to 91.4 ± 9.5% (p<0.01). During baseline assessment, approximately 30% of pertinent patient information was lost during the ToC. The use of a CA 10 months later was associated with a significant increase in the CS by 0.19±0.08 (p=0.02) and overall quality of the ToC.

Conclusion: Critical patient information transfer was more complete and higher quality in the ToC process of a simulated patient crisis when residents used a cognitive aid.

Keywords: Graduate Medical Education; Transition of Care; Simulation; Cognitive Aid; Objective Assessment; Milestones