AoSEberg Journey: Analysis of Similar Events as an Important Tool of Signal Detection and Safety Surveillance of Investigational Medicinal Products Volume 55- Issue 5

Roman Lyakhovetsky^1*, Maria Skryma¹, Silvie Slepickova¹, Lisa Carlson², Robin Richardson², Derek Tom² and Maxim Belotserkovskiy³

• ¹Pharmacovigilance Unit, Medical Affairs Department
• ²Medical Writing Unit, Medical Affairs Department
• ³Medical Affairs Department

Received: March 21, 2024; Published: April 03, 2024

*Corresponding author: Roman Lyakhovetsky, Pharmacovigilance Unit, Medical Affairs Department, PSI CRO AG, Israel

DOI: 10.26717/BJSTR.2024.55.008773

Abstract PDF

Reporting of adverse events is mandatory in the clinical development of investigational medicinal products (IMPs). While most reporting requirements have been globally harmonized, some local ones are vaguely formulated. Analysis of Similar Events (AoSE) may serve as an example of such vague requirement. Here, we advocate the AoSE as a useful tool for signal detection and strongly recommend including both non-serious and serious treatment-emergent adverse events (TEAEs) in the analysis. We studied datasets of two separate and distinct clinical studies (designated Study 1 and 2). Non-serious TEAEs exceed the serious TEAEs by ~30-fold. Interestingly, the number of non-serious Grade 3-4 TEAEs exceed by 30-fold the serious TEAEs of same severity in Study 1 (with cytotoxic IMP) and only by slightly more than 3-fold in Study 2 (with immunooncologic IMP). We hypothesized that including non-serious TEAEs may enhance the safety signals detection, especially applicable to Grade 3-4 non-serious TEAEs, since seriousness assessment of those depends on the medical judgment and local practice (constituting a “grey zone”). To test this, we examined the emergence of TEAEs for confirmed risks associated with the IMP in Study 1. In addition to serious TEAEs, we detected a significant number of non-serious TEAEs related to confirmed risks (including Grade 3) emerging months before, or coincidentally with the first serious TEAEs. We promote the AoSE from a burdensome regulatory requirement to one of the main tools of signal detection and safety surveillance, especially during the early stages of clinical development, when safety data are limited.

Keywords: Analysis of Similar Events (AoSE); Periodic Safety Reporting; Pharmacovigilance; Safety Surveillance; Signal Detection

Abbreviations: AE: Adverse Event; AoSE: Analysis of Similar Events; CIOMS: Council for International Organizations of Medical Sciences; CTCAE: Common Terminology Criteria for Adverse Events; FDA: Food and Drug Administration; GCP: Good Clinical Practice; GVP: Good Pharmacovigilance Practice; ICH: International Council for Harmonization; ICSR: Individual Case Safety Report; IMP: Investigational Medicinal Product; IND: Investigational New Drug; MedDRA: Medical Dictionary for Regulatory Activities; SAE: Serious Adverse Event; SMQ: Standardized MedDRA Query; SAR: Serious Adverse Reaction; SUSAR: Serious Suspected Unexpected Adverse Reaction; TEAE: Treatment-Emergent Adverse Event

Safety surveillance, including reporting of adverse events (AEs), is mandatory in the clinical development of investigational medicine products (IMPs) to ensure the safety and well-being of study subjects [1-3]. This process starts from the very beginning of development (Phase I) and continues throughout the entire clinical program, including post-marketing approval studies (Phase IV). Reporting of AEs to regulatory authorities, ethics committees/institutional review boards, and Investigators is done in an expedited (individual case safety reports [ICSRs]) or aggregated (periodic reports) manner. While most reporting requirements have been well-specified and globally harmonized, certain local requirements are vaguely formulated and leave some room for interpretation [4,5]. Analysis of Similar Events (AoSE), required as a part of Investigational New Drug (IND) safety reporting to the US Food and Drug Administration (FDA), may serve as an example of such vague requirements. For an IND safety report of a suspected unexpected serious adverse reaction (SUSAR), the Sponsor must identify all similar serious adverse reactions reported previously to the FDA and analyze the significance of this SUSAR in light of previous, similar reports or any other relevant information [6]. Detection and assessment of new and previously unknown safety information about a drug-event combination (safety signal) is the main aim of the pharmacovigilance process [7]. Each SUSAR, being an unexpected AE, may represent a potential new signal. AoSE prepared at the time of SUSAR reporting provides the opportunity for early analysis and medical assessment of aggregate data. For relevant definitions, the exact language of the guidance, and the proposed content of an AoSE, see Table 1 [8].

Table 1: Definitions and relevant regulatory requirements.

Note: AoSE: Analysis of Similar Events; BA/BE: Bioavailability/Bioequivalence; CFR: Code of Federal Regulations; FDA: Food and Drug Administration; IND: Investigational New Drug; IMP: Investigational Medicinal Product.

Despite being required by the FDA since 1987, the specification of the AoSE content and sources of data is still far from being clear and uniform. AoSE is traditionally prepared based on data from safety databases with serious AEs (SAEs) [9]. However, there are other relevant sources, such as clinical databases containing SAEs as well as non-serious AEs. These sources might be especially important for non-serious AEs of Grade 3 4 severity (severe to life threatening, per Common Terminology Criteria for Adverse Events [CTCAE]) [10,11], as meeting seriousness criteria (e.g., hospitalization) may differ from country to country, or be affected by the subjective judgment of Investigators or treating physicians (assessment as a medically important event). Data relevant to AoSE might also be provided by non-clinical studies, scientific literature, manufacturer or co-development partner data, or post-market safety surveillance if the molecule is marketed [12]. In this article, we discuss the role of AoSE in safety surveillance and signal detection. We advocate for using AoSE as one of the main tools of this process, especially in the early stages of clinical development, when available data are still limited. Further, we discuss the data sources used for AoSE with a particular focus on non-serious AEs from clinical databases as a complementary, but not less important, source of data in addition to safety databases. We also touch on the technical aspects of data aggregation from different databases.

Trials

Safety data from two different clinical trials (further designated as Study 1 2) were collected and processed

Table 2: Concise description of studies, used in the analysis.

Note: IMP: Investigational Medicinal Product

in compliance with the regulatory guidelines and as per each study’s protocol. See selected details about the trials listed in Table 2.

Data Sources

Both non-serious AEs and SAEs were collected in case report forms and, therefore, in study clinical databases. SAE reports received from sites were processed and aggregated in safety databases in accordance with Good Pharmacovigilance Practice (GVP) module VI [13] and “Detailed guidance on the collection, verification, and presentation of adverse event/reaction reports arising from clinical trials on medicinal products for human use (CT-3)” [3]. For each study, clinical and safety databases were periodically reconciled to exclude duplication or discordant entries as per the study protocols. Extracts from both databases were aggregated into consolidated datasets per study by mapping the corresponding database fields. These datasets consisted of non-serious AEs from clinical databases and SAEs from safety databases. Further analysis was performed only on treatment-emergent AEs (TEAEs; i.e., AEs that occurred after exposure to the IMP).

Data Processing

Consolidated listings were categorized hierarchically by MedDRA System Organ Class and then by MedDRA Preferred Term (PT). They were classified by event seriousness as per the International Council for Harmonisation Good Clinical Practice (ICH GCP) [1] and ICH E2A criteria and then by severity Grades 1-5 as per CTCAE (version 4.03 or 5.0, as applicable by study protocol). Data were presented visually using pie and bar charts (Figure 1). TEAEs were classified by percentage of the total number of TEAEs per severity grade. For identification of confirmed risks and potential signals (based on the pattern of emergence of TEAEs; see Results, Section 3), TEAEs relevant to these risks/signals were retrieved based on MedDRA v24.1 PTs. Confirmed risks were based on the latest versions of the Investigator’s Brochures for the respective IMPs and identified by matching PTs. Potential signals were based on the identification of the proposed pattern of the emergence of TEAEs and medical judgment, as described in Results, Section 3.

Figure 1

Note: AEs per study were classified by seriousness (inset pie charts) and severity (bar charts). AEs of grade 3-4 CTCAE severity constitute substantial portion of overall safety data. AE: adverse event.

Serious Adverse Events Constitute only a Small Fraction (“tip of the iceberg”) of Safety Data Available for AoSE

The CIOMS Working Group VI report and recent studies pointed to the potential importance of non-serious AEs for the safety profile of the IMP [14,15]. Here we examined safety data from two independent clinical studies (see selected details about the studies in Table 2). To assess the relative abundance of different subsets in safety datasets, we visualized the data including both serious and non-serious AEs, as shown in Figure 1. For all studies analyzed, SAEs also constituted a minor part of the TEAEs sometimes dataset (2-3%), as shown in the pie chart inserts. Thus, in the trials examined in this study, non-serious exceeded the serious TEAEs by more than approximately 30-fold for both studies.

Inclusion of Non-Serious Adverse Events Augments the IMP Safety Profile

We then examined the abundance of non-serious TEAEs relative to serious TEAEs when classified by seriousness and severity. Interestingly, as shown in the Figure 1 bar charts, the number of CTCAE Grade 3 and 4 non-serious TEAEs exceeds 3- to 30-fold the number of CTCAE Grade 3 and 4 serious TEAEs, respectively. While this is not unexpected, considering the vast prevalence of non-serious AEs in the dataset, the non-serious TEAEs of CTCAE Grades 3 and 4 pose significant clinical importance under certain circumstances since these same events may meet seriousness criteria; e.g., following a therapy course that dictates a hospital stay >24 hours. Such events constitute a kind of “grey zone.” As a consequence, the identification of safety risks may be delayed based on the data collection and signal management. Non-serious grade 3-4 AEs normally are not included in the AoSE, thus potentially affecting safety conventions used by various organizations. Since, as discussed previously, AoSE in most cases is prepared using only SAEs, the impact of “grey zone” CTCAE Grades 3 and 4 non-serious AEs on the AoSE, and potentially on the safety profile of the IMP, might be systemically excluded from the analysis. Of note, while CTCAE Grades 1 and 2 non-serious AEs are even more abundant than SAEs of comparable severity, as shown in the Figure 1 bar charts, the clinical significance of these AEs might be minor. Thus, we do not attribute them with a similar impact on the safety of the IMP.

Inclusion of Non-Serious Events Facilitates Early Detection of the Safety Risks

Early detection of any safety risk is the key aspect of the signal detection process [7]. We hypothesized that including the data on the non-serious TEAEs (especially of Grades 3 and 4, thus belonging to the earlier mentioned “grey zone”) in the safety evaluation could enhance the early detection of potential signals. These signals could then be thoroughly assessed. To test this proposition, we examined the temporal pattern of the emergence of TEAEs in Study 1, classified by MedDRA PT. We identified the expected emergence of serious TEAEs, but also of non-serious TEAEs of grade 3-4 for some of the IMP confirmed risks, sometimes emerging months before the SAEs (See representative examples of “anonymized” confirmed risks in Figure 2). These data corroborate our suggestion that close monitoring of emerging non-serious TEAEs can promote early detection of potential signals.

Figure 2

Note: Representative time-to-onset plot of emergence of AEs during Study 1 (2017-2018) shows time since the first IMP dose to onset of serious/non-serious AEs. Confirmed risks are identified at MedDRA PT level. The exact PT terms are “anonymized” to preserve data confidentiality. IMP: investigational medicinal product; AE: adverse event; MedDRA: Medical Dictionary for Regulatory Activities; PT: preferred term.

In recent years, the process of safety surveillance has undergone a transition from a primarily reactive approach to safety issues to the proactive evaluation of accumulated safety data in order to identify and manage risks in a timely and efficient manner [15]. As revealed in surveys conducted by the Clinical Trials Transformation Initiative, preparation of the AoSE is considered by Sponsors to be a burden and one of the main challenges of the FDA’s Final Rule for Safety Reporting [5,12]. In this paper, we advocate for the AoSE as a useful tool for signal detection and safety data evaluation to be conducted for every SUSAR regardless of the reporting requirements under different jurisdictions. Such an analysis could highlight data with a potentially important impact on the safety profile of the IMP, within the early phases of clinical development. Further, we strongly recommend the inclusion of non-serious AEs (or at least the ones of CTCAE Grades 3 and 4 severity) in data used for AoSE, as these may significantly contribute to the emerging safety profile of the IMP. Further details to support our proposition are presented below.

Non-Serious Adverse Events May Significantly Enhance the Analysis

CIOMS Working Group VI emphasized the importance of non-serious AEs for assessing the safety profile of the IMP, stating, “Although it is appropriate to apply greater scrutiny to what appear to be serious adverse events, the true safety profile of a medicinal product throughout development can only be assessed by careful evaluation of all AEs/adverse drug reactions” [2]. CIOMS Working Group VI also emphasized the importance of the relationship between non-serious and SAEs and stated that “non-serious AEs could be precursors (prodromes) of more serious medical conditions” [2]. We demonstrated in both studies, that the number of non-serious TEAEs of Grades 3 and 4 significantly exceeded the number of serious TEAEs of Grade 3-4 severity. Importantly, while the data describing serious TEAEs were collected both in safety and clinical databases, the data in the safety database were more extensive, including the Sponsor’s causality and expectedness. The significance of the inclusion of safety database data in safety analyses was shown previously [13], as well as corroborated by our own experience. Based on these observations and recommendations, and taking into account our data, we strongly suggest that non-serious TEAEs in clinical trial databases be used along with the serious TEAEs from safety databases for the preparation of AoSE.

AoSE Preparation is Facilitated by Data Analysis/Visualization Software Suites

In surveys conducted by the Clinical Trials Transformation Initiative, the burden associated with the preparation of the AoSE was named by Sponsors as one of the main challenges of the FDA’s Final Rule for Safety Reporting [5,12]. This finding likely reflects the heavy workload and the specific resources needed for the AoSE. Depending on the clinical program complexity, preparation of the AoSE (as well as signal detection and other safety surveillance-related tasks) may require data collection from different vendors, rely on the availability of analytical software, and need experienced staff capable of performing these activities, including the data analysis and interpretation. To ensure straightforward and efficient preparation of the AoSE, data may be aggregated using a software package that can ensure correct mapping, aggregation, and visualization. Consolidated data could then be pivoted and classified based on the parameters needed for the analysis (e.g., MedDRA terms, Standardized MedDRA Queries (SMQs), seriousness, causality, severity, outcome, and action taken with IMP). Using such software could enable quick focus on data relevant to the pre-approved search criteria. We have successfully used validated tools such as R programming language (via R Studio®), TIBCO Spotfire®, and PowerBI®, all of which suit this purpose. A tentative workflow for AoSE preparation is presented in Figure 3. The process is initiated with the collection of relevant safety data from sources as per the organizational conventions, e.g., the Investigator’s Brochure and snapshots from safety and clinical databases (including the AEs, demography, exposure data, etc.). Data are integrated into a consolidated dataset, which is then filtered in accordance with pre-approved search criteria. If applicable, a search of scientific literature may be performed. Information from all sources is then assessed for safety profile evaluation. Finally, a narrative is written that contains descriptions of the data sources used, search strategy, search results, assessment of the impact of reported SUSARs on the safety profile of the IMP, and a description of safety action(s), as applicable.

Figure 3

Note: Mentioned are main sources of information (green boxes), preparation stages (blue boxes) and possible outcomes (orange gradient boxes). Risk minimization measure stringency is shown by the gradient with white being the least severe measure, and dark orange the most severe. IB: Investigator’s Brochure. *”Dear Investigator” letter - used for update of the investigators on a protocol amendment/IB (reference safety information) update. **Protocol modification e.g., inclusion/exclusion criteria, dose/cycle length modification. Follow-up period prolongation etc.

AoSE May Serve as an Important Tool for Safety Monitoring and Signal Detection, Especially in the Early Stages of Clinical Development

Signal detection of small datasets is mainly performed via case-by-case review or by analysis of case series [7]. Safety information available during Phase I/II studies may be scarce, and sometimes aggregated analysis of the data may be required even before the first periodic safety report preparation. Thus, performing an AoSE might be the initial attempt at an aggregated safety analysis for an IMP. Here we advocate for using the AoSE as an opportunity to perform signal detection, as the data obtained may provide indications of possible safety issues that can be used for subsequent optimization of the clinical development program. As we have shown in Figure 1, the detection of potential signals may be enhanced by including data on non-serious AEs (especially of “grey zone” Grades 3 4). These events may emerge before the serious ones in the course of the study, and analysis of these data may facilitate and speed up the correct evaluation of the safety profile of the IMP. The emergence of “grey zone” AEs, per se, should not necessarily be classified as a signal; however, close monitoring of the data over time can clarify whether the observed phenomena represent isolated events with no impact on the safety profile of the IMP or, with subsequent repetitious reporting of SAEs, confirm a potential signal. The model of our approach is presented in Figure 4. We refer here to all available safety data as an “iceberg,” with SAEs being the “tip” and the bulk of the non-serious-AEs being the “body,” which, being not “visible above the water,” is not routinely included in the analysis. Thus, our model states that the inclusion of non-serious AEs and the use of advanced technology and experienced staff all contribute to the preparation of the AoSE, which significantly improves the early detection of potential signals.

Figure 4

Note: Building structured process and mining deeper into the data facilitates performing meaningful AoSE with significant impact on IMP safety profile. AoSE: analysis of similar events.

Along with other fields that deal with medical information, pharmacovigilance undergoes conceptual shifts. Readily available, user-friendly technology and outsourcing specific functions to external vendors facilitate complex analyses that previously required heavy workloads and personnel investments. The use of these resources leads to improved assessment of the safety of IMPs. In our opinion, AoSE should be considered as one of the main tools of signal detection, particularly in the early stages of clinical development. It may provide the context necessary for correct and efficient assessment of safety data (scarce as it may be), and thus improve the safety of study subjects. In summary, rather than a regulatory burden to handle, we regard AoSE as an opportunity for valuable insight regarding the safety of the IMP.

MS, RL and SS contributed equally to this work. MB, MS, RL and SS devised the research. RL performed the research. RL, MS and SS wrote the manuscript. LS, DT and RR assisted with drafting the manuscript. All authors read and approved the final manuscript.

The authors declare that there are no competing interests.

Due to its proprietary nature and confidential agreements in place, supporting data cannot be made openly available.

The authors would like to thank Rochelle Ku for kind assistance with the preparation of the figures.

1. (2017) European Medicines Agency. Guideline for good clinical practice E6(R2) Addendum Step 5 version. European Medicines Agency.
2. (2005) CIOMS Working Group VI. Management of safety information from clinical trials: Report of CIOMS Working Group VI. CIOMS.
3. (2011) Commission E. Communication from the Commission—Detailed guidance on the collection, verification and presentation of adverse event/reaction reports arising from clinical trials on medicinal products for human use (‘CT-3’).
4. Ann Neuer (2019) A Fresh Take on the Adverse Event Landscape. Clinical ResearcherTM 33(2): 36-45.
5. Perez R, Archdeacon P, Roach N, Goodwin R, Jarow J, et al. (2017) Sponsors’ and investigative staffs’ perceptions of the current investigational new drug safety reporting process in oncology trials. Clinical Trials (London, England) 14(3): 225-233.
6. (2010) US. Food and Drug Administration. CFR - Code of Federal Regulations Title 21.
7. (2010) CIOMS Working Group VIII. Practical aspects of signal detection in pharmacovigilance: Report of CIOMS Working Group VIII. CIOMS.
8. (2001) European Parliament and the Council of the European Union. Directive 2001/20/EC on the approximation of the laws, regulations and administrative provisions of the Member States relating to the implementation of good clinical practice in the conduct of clinical trials on medicinal products for human use. Official Journal of the European Communities L 121: 34-44.
9. Sethi SS, Kramer JM, Gagnon S, Nadzan G, Vega JM (2014) Industry Practices for Expedited Reporting to Investigators Conducting Research Under an IND. Therapeutic Innovation & Regulatory Science 48(6): 741-748.
10. (2010) US Department of Health and Human Services. Common terminology criteria for adverse events (CTCAE).
11. (2017) US Department of Health and Human Sevices. Common terminology criteria for adverse events (CTCAE).
12. (2010) Improving the System of Reporting and Interpreting Unexpected SAEs to Investigators Conducting Research Under an IND Summary of WS1: IND Sponsor Practices 03-04 October 2010. WS1: IND Sponsor Practices.
13. (2017) Guideline on Good Pharmacovigilance Practices (GVP)Module VI –Collection, Management and Submission of Reports of Suspected Adverse Reactions to Medicinal Products (Rev 2).
14. James EC, Dunn D, Cook AD, Clamp AR, Sydes MR (2020) Overlap between adverse events (AEs) and serious adverse events (SAEs): A case study of a phase III cancer clinical trial. Trials 21(1): 802.
15. Ball G, Reblin T, Buchanan J, Hendrickson BA, Lewis E, et al. (2020) A Framework for Safety Evaluation Throughout the Product Development Lifecycle. Therapeutic Innovation & Regulatory Science 54(4): 821-830.