Review ArticleABSTRACT
Radiation therapy is a basic tool in cancer treatment and so maximising safety during these procedures is essential. Here we present the evolution and development of these safety protocols in Spain by leveraging the so-called risk matrix tool. We also tested its application to 3D radiotherapy treatments. A working group comprising members of the Spanish Society of Medical Physics (SEFM), Spanish Society of Radiation Oncology (SEOR), Spanish Association of Radiotherapy Technicians (AETR), and Spanish Society for Radiation Protection (SEPR) was created with the collaboration of the Nuclear Safety Council, all under the auspices of the Spanish Health Authority, to adapt risk matrix methodology to radiotherapy applications in Spain (the Risk Matrices in Radiotherapy, or MARR project). The application we developed was a very effective tool to integrate the entire methodology as well as previous user experiences into risk analysis. Among its main qualities was the ease with which it could be modified to allow each hospital to generate analysis elements and parameters specific to its own practice and to accommodate process changes as part of their continuous improvement plans. As a continuation of this work, the MARRTA project, which includes the latest application techniques in external radiotherapy (IMRT and IGRT, among others) is currently in its latter research stages.
Keywords: Patient Safety; Risk Matrices; Radiation Oncology
Abbreviations: SEFM: Society of Medical Physics; SEOR: Society of Radiation Oncology; AETR: Association of Radiotherapy Technicians; SEPR: Society for Radiation Protection; ROTs: Radiation Oncology Therapies; PSA: Probabilistic Safety Analysis; FTs: Fault Trees; FMEA: Failure Mode and Effect Analysis; MARR: Matrices in Radiotherapy; CT: Computed Tomography; CSN: Council Safety Nuclear; CAT: Computed Axial Tomography; TPS: Treatment Planning System
Introduction
Ionising radiation therapy encompasses a set of procedures that have proven to be an effective means of treating oncological disease. These methods have been modified since their discovery, both because of increased precision knowledge of the effects radiation has on different tissues and to incorporate the technologies developed to make these treatments possible. In addition to its growing use in benign pathologies, it is now estimated that more than 60% of cancer treatments require some form of radiotherapy to radically or palliatively treat the oncological disease. The objective and procedures used for radiotherapy are similar to those employed for surgery. That is, to achieve local disease control and thereby mitigate, depending on the pathology, systemic dissemination of the neoplasm. Radiotherapy is considered a safe treatment. Under normal conditions, the success or failure of radiotherapy is determined by the tumour volumes, their radiosensitivity, and the ability to administer an absorbed dose that does not produce significant detrimental effects in neighbouring tissues. However, in recent decades adverse situations have been reported because of failures in the use of increasingly sophisticated technologies whose execution involves more complex procedural protocols. Indeed, in Spain, an accident involving radiotherapy treatments occurred between 10 and 20 December 1990 [1] that began a new era in the field of ionising radiation safety. Thus, the Spanish authorities established legal regulations to guarantee the quality [2] of radiation oncology therapies (ROTs) for the first time in 1998.
Analysis of these situations, which have been collected and reviewed by international organisations including the IAEA [3], has shown that these deviations were mostly caused by human errors rather than the technology used. Moreover, they occurred in every country, regardless of its overall technological level and healthcare capacity. Nevertheless, these crises have all garnered extensive media attention. Accidents resulting in death or serious injuries have considerable social repercussions and these are probably more common than we may imagine if we consider the very high number of procedures carried out every day worldwide. In general, medical professionals are trying to minimise such situations by observing the procedures employed in industrial settings to reduce the risk involved in these activities. This is especially true for contexts with a greater social perception of risk such as the chemical industry, aviation, and nuclear industry, among others. In all these cases, risk analysis procedures have been applied since the 1950s to catalogue the impact of the adverse effects the production or use of their products and services has on clients. In medicine, and in radiology specialties in particular, this culture of safety has followed other paths but are now all converging upon risk analysis procedures like those used in industry, thereby leveraging this vast pool of experience.
In this context, in an attempt to apply risk-management methodologies common in the nuclear industry to the context of medical radiotherapy services, the Ibero-American Forum of Radiation and Nuclear Safety Regulatory Agencies (known as FORO) carried out a probabilistic safety analysis (PSA) on a generic facility with a linear accelerator and multileaf collimator. Thus, the operation of the accelerator was broken down in the same way the therapeutic procedures and start-up of the equipment had already been broken down into operating blocks. This type of study is designed to implement quantitative risk assessments using fault trees (FTs), sequences, and failure mode and effect analysis (FMEA). Among others, two crucial general conclusions were reached based on this exhaustive risk analysis. First, failures due to the accelerator had a minimal effect on the failure risk compared to human procedural failures. Second, this work also confirmed that it was impossible to conduct this type of analysis in the everyday practice of radiotherapy services because of the time and specialised personnel required and the fact that the risk analysis would imply continuous improvement and subsequent modification of the procedures, thereby requiring reassessment of the risk profiles. Thus, we searched for a methodology that did not require quantitative assessments and thereby had a limited risk profile. This would allow us to make decisions about how to use resources designed to eliminate or mitigate factors that could otherwise reduce the therapeutic practice procedure safely to unacceptable or very poor levels. Hence, a methodology emerged that combined the effects of the frequency of the adverse event, consequences, or damage it can cause, and probability of failure of the possible existing barriers. This combination led to the creation of the risk matrix which defines risk on a non-numerical scale of relative importance.
The results of the application of this matrix in different facilities were presented as communications at several Ibero-American conferences [4,5] and led to national and international publications [6,7]. We also developed a computer application known as SEVRRA [8-10] to facilitate the analysis of these services. Application of this matrix in the Spanish context gave rise to different discrepancies, primarily because the implementation of radiotherapy in Spain slightly differs to its use in the Ibero-American context. This is the result of their installation in a technological park, the availability of professionals, and aspects related to therapeutic culture and linguistic expressions, etc. Consequently, as described below, the so-called Risk Matrices in Radiotherapy (MARR) project was created to analyse the adaptation of this methodology to a group of hospitals in the national public network and to generate a tool to standardise the characteristics of the resulting risk profiles.
State of the Art
The fundamental objectives of the MARR project were to:
1. Offer professionals involved in radiotherapy treatments the opportunity to gain cutting edge knowledge regarding the use of risk matrices.
2. Try to standardise risk estimation criteria to obtain joint strategies for use by administration and regulatory bodies and to more generally allow compliance with the implementation of new regulations.
The process we analysed was a conventional 3D conformal external radiotherapy treatment performed with a multileaf linear accelerator, from the time of clinical prescription of the therapy until its end. This included the acquisition of anatomical data by computed tomography (CT), including the treatment planning, initial patient setting, and daily treatments. The project was carried out from 2013 to 2014 (but was extended for 2 more years) as a primary phase before expanding the work to cover the other radiotherapy techniques now studied in the MARRTA project.
The risk matrix methodology was chosen over other analysis procedures for several reasons, among which we would like to highlight the following:
Figure 1: The proposed working method for undertaking the risk analysis.
1. It is a simple, systematic, and semi-quantitative method.
2. It can be applied in radiotherapy services without experts in risk analysis and without consuming many resources.
3. It assigns a relative risk level to each potential error, thereby allowing us to focus on the highest-risk events.
4. A significant number of public and private centres in the Ibero- American environment have previous experience with this methodology.
In 2013, there were 111 ROT centres in Spain with a total of 125 external beam radiotherapy teams. This current project was developed by members of professional societies involved in radiation therapy, including the Spanish Society of Medical Physics (SEFM), Spanish Society of Radiation Oncology (SEOR), Spanish Association of Radiotherapy Technicians (AETR), Spanish Society for Radiation Protection (SEPR), and representatives of the public body that regulates this area, the Nuclear Safety Council (CSN). As advised, we took a brainstorming approach to the work (Figure 1), avoiding the conditioning factors of individual opinion versus group opinion. In the case of the former, the opinion was expressed anonymously, and an average result was established by the group coordinator according to the majority opinion (Delphi method). Next, we sequentially decomposed the steps or activities involved in the radiotherapy process (Figure 2), analysing the possible failures that could occur in each case. The therapeutic process studied was a 3D conformal external radiotherapy treatment performed with a multileaf accelerator, from the time of the clinical prescription of the therapy (including the planning CT) until the end of the therapeutic process. The equipment considered in the study was a multileaf linear accelerator and its control system (Record & Verify), computed axial tomography (CAT) equipment used to digitise the patient images, and the communication networks between the different equipment and treatment planning system (TPS). Finally, other accessory elements for beam shaping (electron blocks) or patient immobilisation were also considered. Once the activity was broken down, the risk analysis aimed to answer three fundamental questions
Figure 2: Decomposition of the therapeutic process.
1. What could go wrong? (hazard identification)
2. What would be the consequences? and
3. What was the estimated frequency of these failures or errors?
The preventive or mitigating measures (barriers) detected in the process that, in the event of the indicated situation, would prevent the potential damage from occurring were also identified. Therefore, when estimating the risk associated with each danger (risk analysis), not only its frequency, but also the probability of failure of the existing barriers to avoid the consequences were considered. Thus, we characterised what constituted each initiating event (Table 1), defined as any equipment failure, human error, or external event that could lead to undesired consequences. After defining the initiating events, we identified the defenses (Table 2), comprising the security measures that prevented escalation of errors or failures in the process, allowing an incident to be avoided, prevented, detected, controlled, or mitigated.
Table 1: Examples of initiating events.
Table 2: Examples of defenses: barriers and frequency and consequence reducers.
These were classified into,
a. Barriers that could make it possible to stop the evolution of an initiating event,
b. Frequency reducers comprising security measures designed to prevent the occurrence of initiating events by reducing their frequency but without preventing their occurrence; and finally,
c. Consequence reducers, which are security measures that mitigate the consequences of an initiator when the previous barriers have already failed, thereby reducing their consequences, either by reducing the severity of the damage or by decreasing the population affected by the incident.,
The probability of failure of a barrier mainly depends on its nature, with four types of barrier having been defined:,
i. Automatic interlocks or locks. For example, the Record & Verify system will prevent more sessions being delivered than those approved.,
ii. Alarms. For example, the Record & Verify system will warn if any fields are missing from the session before it is closed.,
iii. Tasks performed by different people and based on the work procedures [3].,
iv. Tasks performed by the same person at different stages or times and based on the work procedures.,
Similar to the description for the frequencies and consequences of initiating events, a four-level scale was defined for the probability of failure of each of the four individual barrier types described in the previous paragraph. This parameter was called the robustness of each of the barriers and was classified according to four established levels (Supplementary Table 1). The occurrence of a possible error or ‘quasi-error’ was assigned a frequency of the initiating event occurrence rating and a consequence was assigned to it. Examples of the types of errors considered were the incorrect identification of a patient or omission of the dose in one or more fields. Four frequency levels were defined, which ranged between high frequency: more than 50 events per year (F > 50); medium frequency: 1 to 50 events per year (1 < F < 50); low frequency: between 1 event per year and 1 event every 100 years (0.01 < F < 1); and very low frequency: less than 1 event every 100 years (F < 0.01). This frequency was estimated by considering the probability of the initiator occurring, number of times the task was performed, and number of patients attended.
Supplementary Table 1: The types of robustness of each barrier type.
Supplementary Table 2: Classification of the consequences of initiating events.
Supplementary Figure 1: Representation of a risk matrix.
The criteria used to define the undesired consequences to the patient (Supplementary Table 2) were adapted from the ICRP-86 [11] and those established by the AAPM [12]. In some situations, it is possible for these events to fall outside the range defined in the consequences. Thus, because these were considered as evets resulting in deep reductions in the defense measures, we decided to group them into a single definition which would be excluded from the analysis. These were events that would have had little impact on the risk, even if they had been considered in a continuous improvement quality plan. Sequence was defined as the combination of an initiator and all the defenses that exist to prevent the evolution of said initiator from ending in undesired consequences. In addition to the frequency of occurrence of the initial failure or error and the severity of its potential consequences, the risk assessment also considered the likelihood of all these elements failing to avoid or mitigate the consequences. Thus, risk = frequency of failure × probability of failure × magnitude of the consequences, or R = F × P × C. After analysing all the stages, we were able to classify the events, according to risk, into four categories, very high risk, high risk, medium risk, and low risk, which would allow physicians to devote more effort and resources towards controlling the riskiest initiating events. Finally, the risk matrix is a set of rules that combines the four categorisation levels of each of the three variables: the frequency of the initiator, probability of barrier failure, and the consequences. Thus, as explained in Vilaragut, et al. [13], by applying this set of rules, a level of risk was associated with each of the 64 different value combination possibilities for these three variables and is represented in a three-dimensional matrix (Supplementary Figure 1).
Supplementary Figure 2: Examples of an initiating event and user-selected parameters.
Supplementary Table 3: Probability levels of a given sequence.
The MARR project used the SEVRRA application with a model containing a fairly complete set of initiating events that had been analysed for a generic radiotherapy process in order to carry out the analysis in each hospital. Once the user establishes the organisation chart for the process being executed in the hospital, they need only establish whether the events proposed by the application are valid in their specific case or if they should be discarded. A value for the estimated frequency is also provided and the consequences that could occur if the damage occurred are set out. Likewise, a set of barriers, frequency reducers, and consequences are proposed that the user could choose to incorporate into the analysis, depending on whether or not they apply to their particular case. Finally, the SEVRRA application provides users with the risk level obtained for their installation for each of the events, defenses, and values of the selected parameters. (Supplementary Figure 2) shows an example how the application presented the analysis for each initiating event and the corresponding parameters the user had selected. During this process, a window appears in which the user selects an initiating event they believe could occur in their context. A second window describes the characteristics of the event, and a final window describes the defenses that can be applied to mitigate the problem. The user only needs to accept or reject the possibilities offered by the application. According to the selected parameters applicable to each case (frequency of the event without barriers), consequences, and probability of failure of the barriers (or effect of the reducers), the application provides a level of risk for each initiating event (Supplementary Table 3).
In this work a set of hospitals distributed throughout the country were selected and agreed to participate in the project. They each had sufficient infrastructure to support the analysis and staff with enough experience to evaluate the methodology itself as well as the information provided. Thus, a total of 12 hospitals participated and 40 professionals attended the initial training course. First, we implemented training in the methodology and use of the SEVRRA application. Thus, a radiation oncologist, radiation physicist, and radiation therapy technician from each hospital attended this presential preparation session. Communication with the coordinating group was subsequently conducted online. We established a period of one month to complete the analysis of each of the stages the radiotherapy process was divided into. The estimated time required to complete this analysis was 108 person hours per hospital, corresponding to a total of approximately 1 month.
The coordinating group received 881 comments related to different aspects of this process which they evaluated and responded to in every case. Hence, based on the comments received and subsequent discussion with each of the hospitals, the original SEVRRA application was modified and adapted to the Spanish context by considering the most common practices, vocabulary used, and definition of the processes analysed. In our analysis we considered any events that arose during the discussion (and had not been included in the forum proposals) that were a common practice in most hospitals or for which there was evidence of their relevance.
Supplementary Table 4: List of hospitals participating in the MARR project.
For the validation a multidisciplinary work team consisting of at least one radiation oncologist, a radiation physicist, and a senior radiotherapy technician was established at each of the 12 centres (Supplementary Table 4). This team acted freely to express their opinions, independently of any hierarchical opinions that could have conditioned their opinions. A total of 884 user comments were received. Most of the responses from each group of three professionals agreed that the methodology had increased their awareness of the global characteristics of the radiotherapy process and the need to encourage a culture of safety at work. However, the response was less enthusiastic regarding the criteria for applying the methodology in their services because the participants considered it too time consuming.
Supplementary Figure 3: New initiating events, barriers, and frequency reducers identified after model validation.
One of the most important contributions of the participating hospitals was the proposal to introduce new initiating events, barriers, and reducers. In cases in which the proposed initiating event, barrier, or reducer did not already appear in the original list or was already part of another initiating event, new wording, an estimated occurrence frequency, and corresponding consequences were agreed upon so that it could be introduced into the final revised model. (Supplementary Figure 3) shows the number of new events, barriers, and reducers in the new model compared to the original model. (Supplementary Figure 4) shows the assessment survey of the methodology used, in which the acceptance of all the professional groups exceeded 85%. Of note, there was unanimous agreement to use a common methodology to compare the different radiotherapy processes and risk profiles between the different units.
Supplementary Figure 4: Methodology evaluation survey implemented by the participating teams.
Conclusion
The MARR Project, under the auspices of the Spanish Health Authority, involved the SEFM, AETR, SEPR, SEOR, and CSN in improving patient safety in the field of radiation oncology by developing a national program for the analysis of risks. The main objective of using this tool was to carry out an exhaustive, organised, and realistic analysis of the specific processes carried out by each radiotherapy service to detect opportunities for improvement in patient safety. This work allowed us to identify weaknesses in safety protocols, thereby making the MARR project a useful risk management tool in every type of practice involving the administration of radiation doses to patients. The risk analysis undertaken in this work allowed us to take certain measures, including:
i. Prioritising resources by directing them towards improving safety and designing additional barriers (or strengthening existing ones) that help make the possible failures detected in this analysis less likely or less severe.
ii. Monitoring the measures implemented.
iii. Updating the analysis when implementing new practices, teams, or ways of working.
iv. Helping groups to learn from their own mistakes.
As a result of this work, we used the same components to start developing the MARRTA program to extend our work with MARR to the external radiotherapy techniques not included in the original work such as intensity modulated radiation therapy (IMRT) and extracranial stereotactic body radiation therapy (SBRT), which has now finished its validation period and will soon be put into use in clinical practice.
Acknowledgement
The authors would like to thank the professionals at the aforementioned participating hospitals who made the development of the MARR project possible. Ethical approval was not necessary for the preparation of this article.
References
- Escó R, Lopez P, Bellosta R, Baquedano JE, Mateo P (1993) Accidental overirradiation sí Radiotherapy and Oncology 28(2): 177-178.
- (1998) Real Decreto 1566/1998, de 17 de julio, por el que se establecen los criterios de calidad en radioterapia.
- (2000) Lessons learned from accidental exposures in radiotherapy. Safety reports series N: 17.
- Duménigo C, López R, Paz A, Manriquez M, Santos S, et al. (2004) Matrices de riesgo en Radioterapia: Experiencia en Hospitales de latinoamerica. Revista de medicina e Investigación 2(1): 47-48.
- Prieto C (2017) El Proyecto MARR. 5ºCongreso conjunto SEFM/SEPR Gerona.
- Vilaragut JJ, Duménigo C, Delgado JM, Morales J, McDonnell JD, et al. (2013) Prevention of accidental exposure in radiotherapy: the risk matrix approach. Health Phys 104(2): 139-150.
- Rot San Juan MJ, Hernández Martínez AC, Martínez Gómez LC, Fernandez PS, Grande AG, et al. (2018) Evaluación de riesgos radiológicos en tratamientos de tiroides con I-131 mediante la metodología de matrices de riesgo. Rev Esp Med Nucl Imagen Mol 37(Supl 1): 238.
- Mc Donnell J, Papadopulos S, Paz A, Morones RL, Perez FR (2013) Aplicación de SEVRRA para la evaluación de condiciones de riego en braquiterapia HDR, Foro Iberoamericano de Organismos Reguladores Radiológicos y Nucleares, IRPA, Rio de Janeiro, Brazil.
- (2016) Comisión Nacional de Seguridad y Salvaguardias. Gobierno de mexico. Sistema de evaluación del riesgo en radioterapia SEVRRA. Fecha de publicación 7 de septiembre de.
- (2012) Foro Iberoamericano de organismos reguladores radiológicos y nucleares. Software de evaluación del nivel de riesgo de los servicios de radioterapia: SEVRRA.
- (2000) Prevention of accidental exposure to patients undergoing radiation therapy. ICRP Publication 86 ICRP Ann 30(3): 7-70.
- Purdy JA, Biggs PJ, Bowers C, Dally E, Downs W, et al. (1993) Medical Accelerator Safety Considerations - REPORT OF AAPM NUCLEAR MEDICINE COMMITTEE TASK GROUP 35. AAPM (American Institute of Physics) 20(4): 1261-1275.
- Villarragut JJ, Dumenigo C, Delgado JM, Morales J, McDonnell JD, et al. (2013) Prevention of Accidental Exposure in Radiotherapy: The Risk Matrix Approach. Health Physics 104(2): 139-150.
