Real-world Evidence Requirements with Respect to External Control Arms of Single-arm Clinical Trials by Major Regulatory and HTA Agencies – Comparative Review of Guidelines and Identification of Trends Based on Seven Recent Case Studies Volume 58- Issue 1

Volker Schuster* and Anna Tena Marco

• SAI MedPartners, Neuhofstrasse 12, 6341 Baar, Switzerland

Received: July 26, 2024; Published: August 07, 2024

*Corresponding author: Volker Schuster, SAI MedPartners, Neuhofstrasse 12, 6341 Baar, Switzerland

DOI: 10.26717/BJSTR.2024.58.009096

Abstract PDF

Shifting R&D focus towards the (ultra-)rare diseases and precision medicine leads to a rising need to assess single-arm clinical trials for regulatory and HTA purposes. While numerous time-tested methods exist to utilize real-world evidence as external control groups while limiting the confounding risk, there is still limited international consensus on how to generate, curate and use real-world-data in order to serve as acceptable external control. Both regulatory and HTA bodies are very aware of the challenge, and many have been recently providing more elaborate guidance on the subject. However, clear and detailed guidance is not yet ubiquitously available on the subject. Many agencies are still in the process of refining or even reforming their guidance on the subject while industry decision-makers are trying to figure out how to bring breakthrough therapeutics to patients faster while providing regulators and HTA bodies and payers with the evidence needed for a responsible decision.
The aim of this research is to review, analyze, and compare recent guidelines of all major regulatory and HTA bodies on the use of real-world evidence (RWE) and in particular on RWD-derived external controls. In order to better interpret the practical meaning of the written guidance and obtain a more reliable understanding of recent RWE and RWD-derived external control trends with respect to recent regulatory and HTA decisions, we conducted a systematic literature review as well as a search of single-arm clinical trials on ClinicalTrials.gov, to identify recent case studies, where RWE and, in particular, RWD-derived external controls might have been used in recent regulatory or HTA-related submission. After reviewing written guidance from the FDA, EMA, MHRA, NMPA, CDSCO, PMDA, Swissmedic, ANVISA, ICER, NICE, G-BA, IQWiG, HAS, AIFA, AETS, SwissHTA, Chuikyo/JHEP, CNHDRC, Conitec, and CADTH, we identified and compared trends in both regulatory and HTA agencies. We then complemented these theoretical findings with seven recent case studies identified via systematic literature review and clinical trial register (ClinicalTrials.gov) search.

Keywords: Real-world Evidence (RWE); Real-world Data (RWD); External Control Arms; External Controls; Regulatory Affairs; Health Technology Assessment (HTA); Propensity Score Matching (PSM); Inverse Probability of Treatment Weighting (IPTW); Matching-Adjusted Indirect Comparison (MAIC); single-arm trials

Regulatory agencies and health technology assessment (HTA) organizations are experiencing a growing trend in submissions that utilize external controls derived from real-world data (RWD) [1]. The U.S. Food and Drug Administration (FDA) and European Medicines Agency (EMA) have defined RWD as data collected routinely on patient health status or healthcare delivery from various sources outside traditional clinical trials [2-4]. External control arms derived from RWD “can be used to contextualize efficacy findings for investigational therapies evaluated in uncontrolled trials” [1]. Propensity score matching (PSM) and inverse probability of treatment weighting (IPTW) are increasingly popular methods to safeguard against confounding when using RWD to generate external controls. Both methods have their “own practical advantages as well as limitations, which may be more or less suitable under different scenarios such as the availability of treatment and comparator patients and number of treatments being compared” [5]. Hence, specific guidance on these two important methods will be given special consideration. The increasing familiarity of regulators and HTA bodies with external controls has led to more detailed and specific guidance from regulatory and HTA decision-makers alike. The aim of this paper is to investigate if and to what extent the guidance from regulatory agencies across the globe differs, and to what extent the HTA bodies’ external control guidance differs among each other and from regulatory agencies. Last, but not least, we conducted both a SLR and a search on ClinicalTrials. gov to identify recent regulatory and HTA submissions with RWD-derived external controls or alternative RWE strategies in order to investigate how the guideline-related differences or nuances translate into the practice of regulatory and HTA decision making.

Regulatory Authorities

To understand the detailed guidance provided by the FDA, EMA, NMPA, CDSCO, PMDA, Swissmedic, and ANVISA on the use of real- world data (RWD) for external control arms, it’s important to review the specific methodological guidance each authority offers. This includes how they address confounding and ensure the comparability of populations.

Food and Drug Administration (FDA): The FDA has provided detailed guidance on the use of RWD and RWE [2,3], particularly for the purpose of generating external control arms in clinical trials [4]. While the FDA cautions that in many instances, externally controlled trials may not be appropriate and that “the likelihood of credibly demonstrating effectiveness with an external control is often low” [4], the agency also lists important considerations for the acceptability of RWD-derived external controls:

Sources of Data: The FDA emphasizes the need for high-quality data from reliable sources such as electronic health records (EHRs), claims databases, registries, and patient-generated data.

Eligibility Criteria: There is a focus on ensuring that the criteria used to select RWD cohorts are consistent with those of the clinical trial population to minimize bias.

Temporality: The timing of data collection is crucial. The FDA suggests that the RWD should be contemporaneous with the clinical trial data to ensure relevancy.

Population Representation: Ensuring that the RWD cohort is representative of the broader patient population is essential for generalizability.

Clinical Evaluation and Biostatistical Analysis: The FDA supports both Bayesian and frequentist approaches but emphasizes the importance of prespecifying the analytical plan. The FDA underscores the importance of high-quality, reliable RWD. Data should be complete, accurate, and appropriately curated. Also, the FDA specifically mentions the need for patient-level data for external control arms, rather than summary-level data. Methodological choices and analytic strategies should be clearly documented and justified. The FDA emphasizes the need for prespecification of the analytical plan. When submitting studies that use RWD for external control arms, detailed documentation on data sources, cohort selection, statistical methods, and sensitivity analyses should be included. It’s important to note that the FDA’s approach to external control arms and RWD is evolving. They encourage early communication with sponsors to discuss specific study designs and analytical approaches [3,4]. In the past. the FDA has considered external controls on a case-by-case basis, particularly in situations where the natural history of a disease is well-defined and known not to improve with available therapies.

European Medicines Agency (EMA): The EMA’s guidance on RWD and external control arms focuses on the integration of RWE into regulatory decision-making [6-8]:

Sources of Data: The EMA advocates for the use of well-curated databases and registries and stresses the importance of data quality and relevance.

Eligibility Criteria: Similar to the FDA, the EMA emphasizes harmonized eligibility criteria between RWD and clinical trial populations to reduce selection bias.

Temporality: EMA guidance advises that the RWD should be as current as possible to match the temporal context of the clinical trial. Population Representation: Ensuring a diverse and representative patient cohort is highlighted to improve the validity of findings.

Clinical Evaluation and Biostatistical Analysis: The EMA encourages the use of advanced statistical methods to address confounding, including both frequentist and Bayesian approaches. The agency highlights the need for transparency in the analytical methods and the assumptions made [6-8]. The EMA guideline for registry- based studies does not contain specific recommendations on propensity score matching (PSM) and inverse probability of treatment weighting (IPTW) [8]. The EFPIA comments on the EMA guideline for registry-based studies specifically ask whether propensity score matching should be considered as a technique for addressing selection bias. Recently, PSM was mentioned in the HTACG`s “Methodological Guideline for Quantitative Evidence Synthesis: Direct and Indirect Comparisons”. While the guideline doesn’t provide extensive details on PSM specifically, it acknowledges propensity score methods as a recognized approach for addressing confounding in non-randomized evidence, particularly in the context of indirect comparisons when dealing with individual patient-level data [9]. There might be more detailed guidance with respect to RWD external controls in the future. Generally, EMA encourages the use of Bayesian methods for their flexibility and ability to integrate diverse data sources [10]. It’s important to note that the EMA’s guidance emphasizes the complementary role of RWE to existing evidence, such as clinical trials. Similar to the FDA, the EMA encourages early engagement with regulatory agencies and other stakeholders when planning to conduct registry-based studies [11].

Medicines and Healthcare Products Regulatory Agency(MHRA): The UK’s Medicines and Healthcare products Regulatory Agency (MHRA) has increasingly recognized the value of RWE in the regulatory submissions of pharmaceuticals. In December 2021, MHRA published consultation guidance that underscores the significance of RWE to support regulatory decisions [12]. This guidance encompasses various trial designs, including simple and hybrid trials, and outlines critical factors to consider when collecting RWD as part of a clinical trial [12]. One of the central tenets of the MHRA’s guidance is the emphasis on data quality. The agency asserts that there are no barriers to using RWE for initial approval or for the approval of new indications, provided that the data quality is robust. Additionally, the trial must be designed in a manner that allows it to provide the evidence required to answer the regulatory question effectively. This approach reflects the MHRA’s commitment to ensuring that RWE can be a reliable and credible source of information in the regulatory decision- making process. The MHRA has published two specific guidance documents to elaborate on these principles [13,14].

The first, “MHRA Guidance on the use of Real-World Data in Clinical Studies to Support Regulatory Decisions”, serves as an introduction to the MHRA’s RWD guideline series. It highlights key considerations when evaluating whether an RWD source is of sufficient quality for the intended use [13]. The second document, “MHRA Guideline on Randomised Controlled Trials using Real-World Data to Support Regulatory Decisions”, delves into more detailed aspects such as clinical trial authorization, design, choice of endpoints, and safety data requirements for prospective randomized trials using RWD sources [12,13].

Sources of Data: The MHRA recognizes various sources of RWD, including electronic healthcare records (EHR), primary and secondary care records, disease registries, and administrative data on births and deaths [12]. They also acknowledge patient-reported outcomes (PRO) data and data collected through wearable devices, specialized websites, or tablets as valid RWD sources [12]. The agency emphasizes the importance of demonstrating that the data source is of sufficient quality for the intended use [12,14].

Eligibility Criteria: The MHRA stresses the importance of considering the source population prior to submitting the study protocol. Population Representation: The MHRA guidance highlights the importance of considering the source population and its appropriateness for the study. They also note that evidence derived from RWD may be more representative of the true effects of a treatment in the community and more generalizable than data from traditional clinical trials [12]. This suggests that the MHRA values RWE for its potential to provide insights into broader, more diverse populations. Clinical Evaluation and Biostatistical Analysis: MHRA emphasizes that the principles used in traditional clinical trials, such as the importance of randomization and blinding, remain applicable for studies using RWD. They state that RWE generated from randomized controlled trials (RCTs), including RWD, “is not generally considered of more or less value for regulatory decision-making than evidence from conventional RCTs provided the data quality is robust and the trial well-designed” [12]. The agency requires that processes for selection, extraction, transfer, and handling of data, as well as methods for their validation, should be described in the study protocol [12]. MHRA also stresses the importance of establishing processes to ensure data integrity from acquisition through to archiving [12]. Furthermore, MHRA guidance covers clinical trial design, including choice of endpoints and safety data requirements for prospective randomized trials using RWD sources [12]. This indicates that the MHRA expects rigorous clinical evaluation and biostatistical analysis in RWE studies, comparable to traditional clinical trials. The MHRA’s guidance on RWE in regulatory submissions highlights the agency’s openness to innovative approaches in clinical research, provided that the data quality is robust and the trial design is sound. This progressive stance aims to enhance the regulatory framework’s flexibility while maintaining rigorous standards for evidence-based decision-making [12-14].

National Medical Products Administration (NMPA): The Chinese regulatory authority National Medical Products Administration (NMPA)’s guidance on RWD use is emerging, with an increasing focus on leveraging RWE for regulatory decisions [15-17]:

Sources of Data: Emphasizes high-quality and reliable data sources including local registries and hospital databases.

Eligibility Criteria: Guidance is evolving but emphasizes the alignment of RWD cohort characteristics with the clinical trial population.

Temporality: NMPA stresses the need for contemporaneous data collection to ensure relevance.

Population Representation: Ensuring a representative sample that mirrors the clinical trial population.

Clinical Evaluation and Biostatistical Analysis: The guidance [15] mentions common types of real-world research designs and statistical analysis methods, but doesn’t provide specific details on Bayesian methods, propensity score matching, or inverse probability of treatment weighting. It’s important to note that while RWE is increasingly accepted, the NMPA currently views it primarily as a supplement to existing clinical evidence rather than a replacement for traditional clinical investigations. The guidance is still evolving, and future refinements may provide more specific details on study conduct and statistical methods.

Central Drugs Standard Control Organization (CDSCO): India’s regulatory authority Central Drugs Standard Control Organization (CDSCO) is in the process of developing comprehensive guidelines for RWD use, with some emerging principles [18-20]:

Sources of Data: Focuses on data from Indian registries, EHRs, and hospital records.

Eligibility Criteria: Importance of matching criteria to ensure comparable cohorts.

Temporality: Advocates for the use of contemporaneous data. Population Representation: Emphasis on capturing a representative patient population.

Clinical Evaluation and Biostatistical Analysis: Guidance on statistical methods is evolving, with a focus on frequentist methods and emerging interest in Bayesian approaches. India currently lacks a formal framework that defines different designs for RWE studies, unlike regulatory bodies in other countries such as the U.S. FDA [19]. The CDSCO has established Subject Expert Committees (SECs) to advise on the evaluation of various types of applications, including those related to new drugs and clinical trials [19]. While not explicitly addressing RWD-derived external controls, the CDSCO’s guidance documents mention the evaluation of safety and efficacy data for new drug approvals, which could potentially include RWE [19]. The CDSCO appears to be open to considering various types of evidence in drug development and approval processes but has not yet issued detailed guidelines specifically for RWE, RWD, or external control arms derived from RWD [19,20].

Pharmaceuticals and Medical Devices Agency (PMDA): The Pharmaceuticals and Medical Devices Agency (PMDA) of Japan has embraced a progressive approach towards the use of RWD and RWE in regulatory decision-making [21-23]. This stance aligns with global trends in drug development and regulatory science. PMDA encourages the use of RWD/RWE throughout a product’s life cycle, from pre-approval development to post-marketing surveillance. Since 2017, PMDA has offered consultation services on RWD/RWE use, expanding these services in 2019-2020 to include registry data and database studies for new drug applications. The PMDA has released several guidance documents to support the appropriate use of RWD/ RWE in regulatory submissions. In April 2021, the PMDA established an RWD working group to address regulatory issues related to RWD/ RWE and promote its utilization within the Japanese regulatory framework. The PMDA actively participates in international discussions and harmonization efforts regarding RWD/RWE use in regulatory decision-making.

Sources of Data: The PMDA recognizes various sources of RWD, including electronic health records (EHRs), claims databases, disease registries, patient-generated data (e.g., from wearable devices), and health surveys. The PMDA emphasizes the importance of data quality and reliability, regardless of the source. They encourage early consultation to discuss the appropriateness of specific data sources for regulatory purposes.

Eligibility Criteria: When considering RWD for regulatory submissions, the PMDA focuses on data quality, accuracy, completeness, consistency of the data, relevance (appropriateness of the data for the specific research question), timeliness, and accessibility. Temporality: Advocates for the use of contemporaneous data. Population Representation: The PMDA stresses the importance of ensuring that RWD adequately represents the target population. Key considerations include demographic representation, geographic representation (urban vs. rural, different regions of Japan), and clinical representation (e.g., disease severity, comorbidities, treatment history). Applicants are encouraged to discuss population representation with the PMDA during consultation services to ensure the external validity of their findings.

Clinical Evaluation and Biostatistical Analysis: The PMDA emphasizes the need for robust methodologies in analyzing RWD. While the PMDA has not released specific guidance on PSM and IPTW, they generally accept these methods when appropriately applied. Key considerations include careful selection of variables for propensity score models, evaluating the balance of covariates after matching or weighting, sensitivity analysis and transparency. Applicants are encouraged to consult with the PMDA on the appropriateness of these methods for their specific study. The PMDA supports the use of RWD/ RWE in regulatory decision-making throughout a product’s lifecycle. Data quality, reliability, and relevance are crucial for regulatory acceptance of RWD/RWE. The PMDA offers consultation services to guide appropriate use of RWD/RWE in regulatory submissions. Population representation and external validity are important considerations in RWD studies. Robust methodologies, including appropriate statistical techniques, are essential for analyzing RWD. Early consultation with PMDA is encouraged to ensure alignment on RWD/RWE utilization strategies. The PMDA continues to develop guidance and participate in international discussions on RWD/RWE use in regulatory science.

Swissmedic: Swissmedic, the Swiss regulatory authority for therapeutic products, is developing guidance on the use of RWE and RWD [24], including their application in external controls. While Swissmedic’s approach aligns with international standards, it has its own specific considerations:

Sources of Data: Swissmedic considers RWD to be all data other than those collected through clinical trials conducted according to ICH GCP guidelines. This includes registries, observational studies, electronic health records, medical claims, and patient-generated data [25].

Eligibility Criteria: Swissmedic emphasizes the need for careful use of appropriate control groups from RWD, considering quality, size, and time frame.

Temporality: The Swiss regulatory authority stresses the importance of critically discussing RWE in the context of all available evidence, which implies consideration of temporal aspects.

Population Representation: Swissmedic acknowledges that RWE could provide therapeutic insights into the use of medical products among underrepresented and vulnerable populations.

Clinical Evaluation and Biostatistical Analysis: Swissmedic requires that RWD-based studies or analyses be listed, and the sources of RWD must be described in detail. This suggests a focus on robust methodologies, though specific statistical approaches are not explicitly mentioned in the available guidance. It’s important to note that Swissmedic does not currently provide detailed recommendations on specific statistical methods for addressing confounding or reducing selection bias in their publicly available guidance [25]. The authority emphasizes the need for critical evaluation of RWE and thorough documentation of RWD sources and analyses [25].

National Health Regulatory Agency (ANVISA): The Brazilian Health Regulatory Agency (Agência Nacional de Vigilância Sanitária, or ANVISA) has recently released new guidance on RWE and RWD for regulatory decision-making in Brazil [26,27]. Sources of Data: ANVISA emphasizes the importance of high-quality data for regulatory purposes. The agency provides recommendations on evaluating RWD quality and sources for regulatory decisions.

Eligibility Criteria: It is crucial to ensure that the eligibility criteria for the RWD cohort align with those of the clinical trial population to minimize selection bias and enhance comparability. Temporality: The timing of data collection is vital. ANVISA stresses that RWD should be contemporaneous with clinical trial data to ensure that the control group is relevant and reflective of the same period.

Population Representation: ANVISA acknowledges the importance of ensuring that RWD is representative of the Brazilian population. This is crucial for the generalizability of findings and their applicability to the Brazilian healthcare context.

Clinical Evaluation and Biostatistical Analysis: While specific statistical methodologies are not detailed in the available information, ANVISA’s guidance likely includes recommendations for addressing challenges in generating RWE and steps to improve the quality, transparency, and acceptability of RWE studies.

ANVISA’s guidance covers various types of study designs and stresses the importance of choosing data sources based on solid methodological principles. The agency recognizes the use of RWD as supportive evidence of efficacy and safety, particularly for drugs indicated for rare diseases or serious conditions where conducting properly controlled clinical trials may be challenging. ANVISA views the role of RWE with caution due to challenges such as ensuring data quality, limitations on using RWD/RWE for efficacy claims, and potential confounding factors and biases. The agency recommends communication between ANVISA and applicants to discuss the relevance of evidence to be submitted, including aspects such as the use of primary or secondary RWD sources and national or international data sources. ANVISA has formed a new RWE group (GT-EMR) responsible for improving research quality in Brazil by evaluating RWE methodologies, issuing technical opinions, and collaborating on regulatory decision-making. The agency is working towards building understanding for the critical assessment of RWD and RWE in regulatory decision-making and collaborating with international institutes to work towards global harmonization in RWD and RWE terminology, execution, use, and study structure.

Summary of regulatory guidelines: All eight regulatory bodies (FDA, EMA, MHRA, NMPA, CDSCO, PMDA, Swissmedic, and ANVISA) emphasize the importance of high-quality data sources, consistent eligibility criteria, contemporaneous data, representative populations, and robust statistical methods to handle confounding and ensure comparability. Overall, the agencies seem to take a cautioning approach. Key commonalities include:

Sources of Data: High-quality, reliable sources such as EHRs, claims databases, and registries.

Eligibility Criteria: Harmonized criteria to match the clinical trial population and minimize selection bias.

Temporality: The importance of using contemporaneous data to ensure relevance.

Population Representation: Ensuring that the RWD cohort is representative of the broader patient population to enhance generalizability. Clinical Evaluation and Biostatistical Analysis: Very limited guidance on the use of advanced methods to avoid confounding while using RWE/RWD is given. Each regulatory body provides tailored guidance within the context of their healthcare system and regulatory framework, but they share a common goal of integrating RWE effectively into the regulatory process for in cases where only single arm trials are pragmatically or ethically feasible.

HTA Agencies

Institute for Clinical and Economic Review (ICER): The Institute for Clinical and Economic Review (ICER) has been progressively integrating RWE into its health technology assessments, recognizing its potential value alongside the gold standard of randomized controlled trials (RCTs) [28-30]. While RCTs remain the cornerstone for evaluating treatment efficacy, ICER acknowledges that RWE can play a crucial role, especially in scenarios where RCTs are impractical or unethical. ICER has been increasing its use of RWE, particularly in economic assessments, primarily to inform costs, resource use, and long-term outcomes [31,32]. Historically, ICER has rarely used RWE for assessing treatment effectiveness, consistent with policies of other health technology assessment agencies [31].

Sources of Data: ICER encourages the use of high-quality data from national health databases, registries, and electronic health records (EHRs). Registry data and claims data are the most frequently used types of RWD in ICER’s evaluations [31,32]. Emphasis is placed on the quality, completeness, and relevance of the data to ensure robust and reliable assessments.

Eligibility Criteria: ICER stresses that importance with broader population representation [31].

Temporality: ICER recommends using contemporaneous data to ensure that the RWE reflects current clinical practices and is comparable to the time period of the RCT [28]. This approach ensures that the RWE is comparable to the time period of the RCTs, providing a more accurate picture of real-world treatment effects.

Population Representation: ICER underscores the need for RWE to represent the broader patient population, which helps in generalizing the findings to a wider audience.

Clinical Evaluation and Biostatistical Analysis: ICER is open to the use of Bayesian methods. To ensure the robustness of findings, ICER requires thorough sensitivity analyses to test the stability of results against various assumptions and methods. While ICER’s approach to RWE is evolving, these guidelines reflect their commitment to enhancing the understanding of real-world treatment performance and improving the overall quality of health technology assessments. ICER has recently announced plans to generate “decision-grade” RWE with partners to improve understanding of real-world treatment performance. Also, ICER has initiated a pilot program to generate new RWE to enhance their understanding of how medical treatments work in the real world [33]. Overall, ICER tries to reflect the current scientific standards. ICER plans to assess the internal and external validity of RWE based on guidelines from the ISPOR RWD Task Force report.

National Institute for Health and Care Excellence (NICE): NICE acknowledges that while RCTs are the gold standard for evaluating interventions, RWE can be used to supplement RCTs, especially when RCTs are not feasible or ethical [34]. NICE has developed a framework for evaluating and incorporating RWE into their guidance, demonstrating their openness to using this type of evidence [35].

NICE has shown willingness to consider RWD-derived external controls, particularly in cases where RCTs are challenging to conduct, such as in rare diseases or specific cancer indications [36]. In some cases, NICE committees have referred to RWE as “the best available source of evidence”, particularly when evaluating treatments for rare conditions where traditional RCTs are difficult to conduct [36]. While accepting RWE, NICE maintains a cautious approach. They acknowledge the potential for residual confounding in RWD comparisons and seek to address uncertainties in the evidence [36].

Sources of Data: NICE encourages the use of high-quality data from registries, EHRs, and other real-world datasets. It emphasizes the importance of data quality and relevance.

Eligibility Criteria: NICE stresses the need for RWE studies to align with RCT eligibility criteria to ensure comparability. The patient populations should match the intervention and comparator conditions of the RCT [34].

Temporality: NICE recommends using contemporaneous data to reflect current clinical practice and ensure relevance to current healthcare settings [34].

Population Representation: NICE emphasizes that RWE should represent the target population for the intervention to ensure generalizability. They recommend assessing the representativeness of the RWE population by comparing demographic and clinical characteristics with those of the RCT population [34].

Statistical Methods: NICE has demonstrated a willingness to consider different types of evidence and statistical approaches in their evaluations, including both frequentist and Bayesian methods. NICE has demonstrated a willingness to consider different types of evidence and statistical approaches in their evaluations. This suggests they may be open to PSM and IPTW when appropriately applied. NICE has developed a Data Suitability Assessment Tool (DataSAT) to help assess the quality and relevance of RWD for specific research questions. NICE describes their RWE framework as a “living framework” that will be updated periodically to reflect user feedback, learnings from implementation, and developments in RWE methodology [35]. NICE encourages companies planning to use RWD in their submissions to engage early with NICE Scientific Advice on how to best use RWD in their evidence-generation plans [35].

Joint Federal Committee/Gemeinsamer Bundesausschuss (G-BA): The G-BA is famous for its “hierarchy of evidence” in its code of procedure, that strongly emphasizes the “superiority of systematic reviews of RCT (Grade Ia) and RCTs itself (Grade Ib)” [37]. G-BA maintains that RCTs are the gold standard for evaluating medical interventions but starts cautiously acknowledging the role of RWE under specific conditions.

Sources of Data: G-BA recommends using high-quality data from national health registries and insurance claims. They emphasize the importance of data quality and representativeness [37]. Eligibility Criteria: G-BA stresses the need for RWE to align with the eligibility criteria of RCTs. This includes ensuring that the patient populations are comparable in terms of demographics and clinical characteristics [37].

Temporality: G-BA emphasizes the use of contemporaneous data to ensure that the RWE reflects current clinical practices and is comparable to the time period of the RCT [37].

Population Representation: G-BA emphasizes that RWE should represent the broader patient population to ensure generalizability. They suggest thorough comparison of demographic and clinical characteristics between RWE and RCT populations [37].

Statistical Methods: With respect to Bayesian methods, G-BA is generally cautious, preferring frequentist methods and RCT data as the gold standard evidence [37].

Institut Für Qualität und Wirtschaftlichkeit im Gesundheitswesen (IQWiG): IQWiG is generally skeptical about using observational RWD for benefit assessments of drugs [38]. According to IQWiG, observational data are usually inappropriate for examining drug effects in benefit assessments because it may be difficult to establish causal relationships between treatments and outcomes due to unknown differences in patient characteristics and it may also be more generally impossible to eliminate confounders even if the confounders are known [39,40]. IQWiG strongly prefers randomized controlled trials (RCTs) as the gold standard for generating evidence, even in supposedly difficult areas or with small study populations. Sources of Data: IQWiG recommends using high-quality data from registries and national health databases. They emphasize the importance of data quality and have compiled criteria for data quality in registries for benefit assessments of drugs [39,40].

Eligibility Criteria: IQWiG stresses the need for RWE studies to align with RCT eligibility criteria to ensure comparability. This includes ensuring that the patient populations are comparable in terms of demographics and clinical characteristics [38].

Temporality: IQWiG recommends using contemporaneous data to reflect current clinical practice and ensure relevance to current healthcare settings [38].

Population Representation: IQWiG emphasizes that RWE should represent the broader patient population to ensure generalizability. They recommend assessing the representativeness of the RWE population by comparing demographic and clinical characteristics with those of the RCT population [38].

Statistical Methods: IQWiG is cautious about Bayesian methods, emphasizing the need for “high-quality evidence” [38]. Haute Autorité de Santé (HAS): HAS supports the use of RWE to complement RCT data, particularly when RCTs are not feasible. Specifically with respect to RWD used as external control arms, HAS points out: “The implementation of an external comparison must be anticipated and scheduled in advance in order to improve its robustness and ensure it is part of a deductive reasoning approach”.

Sources of Data: HAS recommends using high-quality data from national health databases and registries [41]. They emphasize the importance of data quality and relevance.

Eligibility Criteria: HAS stresses the need for RWE studies to align with RCT eligibility criteria to ensure comparability. This includes ensuring that the patient populations are comparable in terms of demographics and clinical characteristics [41].

Temporality: HAS recommends using contemporaneous data to reflect current clinical practice and ensure relevance to current healthcare settings [41].

Population Representation: HAS emphasizes that RWE should represent the broader patient population to ensure generalizability. They recommend assessing the representativeness of the RWE population by comparing demographic and clinical characteristics with those of the RCT population [41].

Statistical Methods: HAS supports the use of Bayesian methods to integrate evidence from RCTs and RWE [41].

Agenzia Italiana del Farmaco (AIFA): AIFA’s formal position on RWE and RWD use in HTA seems not to be fully developed or publicly articulated. AIFA appears to generally recognize the importance of RWE, particularly in supplementing RCT data when RCTs are not feasible. In Italy, like in all other countries, manufacturers seeking reimbursement can and have submitted a variety of study results leveraging different RWD sources for health technology assessments (HTAs).

However, there seem to be no apparent criteria for acceptability beyond very general and widely uncontroversial recommendations.

Sources of Data: AIFA recommends using high-quality data from national health databases and registries [42]. They emphasize the importance of data quality and relevance.

Eligibility Criteria: AIFA stresses the need for RWE studies to align with RCT eligibility criteria to ensure comparability. This includes ensuring that the patient populations are comparable in terms of demographics and clinical characteristics [42].

Temporality: AIFA recommends using contemporaneous data to reflect current clinical practice and ensure relevance to current healthcare settings [42].

Population Representation: AIFA emphasizes that RWE should represent the broader patient population to ensure generalizability. They recommend assessing the representativeness of the RWE population by comparing demographic and clinical characteristics with those of the RCT population [42].

Statistical Methods: AIFA supports the use of Bayesian methods to integrate evidence from RCTs and RWE [42,43].

Agencia de Evaluación de Tecnologías Sanitarias del Instituto de Salud Carlos III (AETS-ISCIII): AETS-ISCIII supports the use of RWE to compleent RCT data, particularly when RCTs are not feasible.

Sources of Data: AETS-ISCIII recommends using high-quality data from national health databases and registries. They emphasize the importance of data quality and relevance [44].

Eligibility Criteria: AETS-ISCIII stresses the need for RWE studies to align with RCT eligibility criteria to ensure comparability. This includes ensuring that the patient populations are comparable in terms of demographics and clinical characteristics [44].

Temporality: AETS-ISCIII recommends using contemporaneous data to reflect current clinical practice and ensure relevance to current healthcare settings [44].

Population Representation: AETS-ISCIII emphasizes that RWE should represent the broader patient population to ensure generalizability. They recommend assessing the representativeness of the RWE population by comparing demographic and clinical characteristics with those of the RCT population [44].

Statistical Methods: AETS-ISCIII recommends using propensity score matching (PSM) to balance treatment and control groups based on observed covariates [44]. AETS-ISCIII supports inverse probability of treatment weighting (IPTW) to balance covariates by applying weights based on the inverse of the treatment probabilities [44]. AETS-ISCIII advocates for regression models to adjust for differences in baseline characteristics between treatment groups [44]. AETS-ISCIII supports the use of Bayesian methods to integrate evidence from RCTs and RWE [44]. AETS-ISCIII requires sensitivity analyses to test the robustness of findings against different assumptions and methods [44].

Swiss Health Technology Assessment (SwissHTA): SwissHTA and other Swiss health authorities recognize the increasing importance of RWE and RWD in the context of health technology assessments (HTAs) for pharmaceuticals, but maintain a cautious and waitand- see approach [45,46]. Similar to G-BA, SwissHTA defines “levels of evidence” in line with evidence-based medicine principles, with randomized controlled trials ranking highest, followed by prospective cohort studies, retrospective studies, case series, and expert opinion [46].

Sources of Data: SwissHTA recommends using high-quality data from national health databases, registries, and EHRs.

Eligibility Criteria: SwissHTA stresses the need for RWE studies to align with RCT eligibility criteria to ensure comparability. Temporality: SwissHTA tends to favor contemporaneous data to reflect current clinical practice.

Population Representation: SwissHTA seems to acknowledge a need for representativeness of the RWE population by comparing demographic and clinical characteristics with those of the RCT population. Statistical Methods: SwissHTA tends to consider both frequentist and Bayesian methods. The Swiss Tropical and Public Health Institute (Swiss TPH) has a Bayesian Modelling and Analysis group that conducts research in advanced statistical modelling and Bayesian computation [47].

Chuikyo (Central Social Insurance Medical Council) and JHEP (Japan Health Economics and Outcomes Research Foundation):

The HTA framework in Japan is still evolving, with Chuikyo playing a central role in decision-making and JHEP contributing to health economic evaluations [48-50]. Japan has recently introduced HTA processes and is working towards incorporating RWE and RWD, but their use is still limited compared to some other countries [49,50]. The Center for Outcomes Research and Economic Evaluation for Health (C2H), established in 2018, plays a key role in the Japanese HTA process. It is responsible for reviewing economic data submitted by manufacturers for cost-effectiveness evaluations. The Central Social Insurance Medical Council (Chuikyo) is involved in the HTA process in Japan. Manufacturers must submit economic data to the Chuikyo committee within nine months if a product meets the selection criteria for cost-effectiveness evaluation. It appears that Japan is still in the early stages of integrating RWE and RWD into its HTA framework, and further development and guidelines may be forthcoming. There is a push for collaboration and involvement of all stakeholders, including patients, healthcare providers, and industry, in the HTA process.

China National Health Development Research Center (CNHDRC): CNHDRC, part of the National Health Commission, is responsible for HTA in China and has started incorporating RWE in evaluations. In 2019, the CNHDRC led the initiative to incorporate RWD into health technology assessments. This decision demonstrates the CNHDRC’s commitment to leveraging RWE from the early stages of HTA development in China. The CNHDRC aims to use RWD not only for HTA but also for broader drug policy reforms. This includes using RWD for various aspects of drug assessment such as safety monitoring, survival rates, prescription costs, and surveillance data. The implementation of HTA using real-world settings is expected to be either government-led or based on clinical and social needs, while adhering to scientific methods and quality control. By deciding to integrate RWD at the initial stages of HTA development, the CNHDRC demonstrates its commitment to staying relevant in this dynamic area and its ability to learn from global experiences.

Sources of Data: The CNHDRC plans to utilize data from insurance, procurement, and prescriptions to inform drug pricing, reimbursement, and procurement policy evaluations [51].

Eligibility Criteria: RWE studies should align with the eligibility criteria of RCTs to ensure comparability, with patient populations matched in terms of demographics and clinical characteristics [51].

Temporality: Contemporaneous data are preferred to ensure that RWE reflects current clinical practices and is comparable to the time period of the RCT [51].

Population Representation: RWE should represent the broader patient population to ensure generalizability. Comparison of demographic and clinical characteristics between RWE and RCT populations is recommended [51]. The overall approach suggests that the CNHDRC is open to innovative uses of real-world data in HTA. The center’s focus on incorporating RWD from various sources and its commitment to scientific methods indicate that it may be receptive to considering RWD-derived external controls as part of its comprehensive approach to HTA.

Comissão Nacional de Incorporação de Tecnologias no Sistema Único de Saúde (Conitec): Conitec oversees HTA in Brazil and supports the use of RWE, particularly when RCTs are not feasible. There is a growing interest in expanding the use of existing RWD sources, such as DATASUS (Brazil’s national health database). Conitec, the Brazilian HTA agency, is showing an increasing demand for RWE in their reports [52,53].

Canadian Agency for Drugs and Technologies in Health (CADTH): The Canadian Agency for Drugs and Technologies in Health (CADTH) has published detailed guidance on the use of RWE in healthcare decision-making, particularly focusing on the integration of RWD into regulatory submissions and HTAs [54-56]. CADTH defines RWE as evidence regarding the use, safety, efficacy, and cost of health products derived from the analysis of RWD. The agency acknowledges the growing importance of RWE in supplementing traditional clinical trials, particularly in areas where clinical trials may not be feasible, such as rare diseases or populations often excluded from clinical trials, like children, older adults, and expectant mothers. CADTH’s guidance promotes the use of high-quality RWD and emphasizes the need for transparent and comprehensive reporting of RWE studies to ensure their reliability and reproducibility. This aligns with Health Canada’s position.

Sources of Data: CADTH acknowledges a wide range of potential sources for RWD, which include electronic health records (EHRs), administrative data, patient registries, claims databases, and data collected from wearable devices and patient-reported outcomes [54-56]. The agency emphasizes the importance of ensuring that these data sources are of high quality and relevant to the research question at hand. The RWD must be reliable, valid, and collected in a manner that minimizes bias and errors to be useful in generating credible RWE.

Eligibility Criteria: In terms of eligibility criteria, CADTH’s guidance highlights the need for pragmatic approaches that reflect real- world clinical practice. Unlike traditional clinical trials, which often have stringent eligibility criteria, RWE studies may adopt broader criteria to include a more diverse patient population. This approach helps in capturing a wider range of patient experiences and outcomes, thereby enhancing the generalizability of the findings. However, the criteria must still be clearly defined and justified to ensure the study’s validity and relevance.

Temporality: While CADTH’s guidance does not explicitly detail temporality, it implicitly underscores the importance of considering the timing of data collection and the temporal relationship between exposure and outcomes.

Population Representation: CADTH stresses the importance of population representation in RWE studies. The agency encourages the inclusion of diverse patient populations that are representative of those seen in routine clinical practice.

Clinical Evaluation and Biostatistical Analysis: For clinical evaluation and biostatistical analysis, CADTH’s guidance emphasizes the need for rigorous methodological standards. This includes ensuring that the study design is robust and appropriate for addressing the research question. This may involve using advanced statistical techniques to control for confounding factors and biases. Selecting meaningful and clinically relevant endpoints that reflect real-world outcomes is crucial. Implementing stringent processes for data validation, handling missing data, and ensuring data integrity throughout the study are also emphasized. Conducting sensitivity analyses to assess the robustness of the findings and to account for uncertainties and potential biases is recommended. Furthermore, providing clear and transparent reporting of the study methods, data sources, and findings is vital to facilitate reproducibility and critical appraisal by stakeholders. CADTH’s guidance on RWE and RWD-derived external control arms reflects a commitment to integrating high-quality RWD into healthcare decision-making. The agency’s focus on data quality, methodological rigor, and transparent reporting aims to ensure that RWE can effectively complement traditional clinical trial data, thereby enhancing the evidence base for regulatory and HTA decisions.

Regulatory and HTA Guideline Analysis

All regulatory and HTA guidelines reviewed have in common that they emphasize the superiority of randomized clinical trial evidence for establishing casual relationships. When guidance is provided, the focus is on the justification for conducting a RWE study instead of a clinical trial rather than providing guidance on preferred designs of RWE studies. Another strong trend among both regulatory and HTA guidelines is that RWE is seen as complementary evidence. In general, regulatory guidelines recommend using RWE in clinical trial support and post-market surveillance. Another area of strong consensus seems to be the frequently mentioned requirement of a robust study design and reliable methods when using RWD. Some guidelines recommend frameworks for study design (e.g., PICO, PICOTS, or PIRO). HTA guidelines tend to provide more specific recommendations with respect to analytical methods and statistical analysis. Frequently, the guidelines require the rationale for using a given method to be provided. RWD sources primarily collected for non-research purposes, such as medical records and claims, are viewed as more challenging due to their heterogeneity across healthcare systems and providers. There seems to be a preference for prospectively designed RWD sources, particularly registries. Another area of consensus is the frequently emphasized need for comprehensive documentation of the data collection process. Unsurprisingly, another ubiquitous trend is that both regulators and HTA organizations highlight the need for protecting data integrity and quality throughout the extraction and curation processes as well as the importance of transparency in data handling practices. Presently, most guidelines do not provide detailed guidance on the use of AI in the extraction and analysis of RWD and generation of RWE [53].

Given the availability of promising methodology in this arena, the scarcity of clear recommendations seems problematic. Few guidelines provide clear definitions with respect to specific circumstances and requirements that, if met, would result in acceptance of RWE, in particular external controls derived from RWD. This striking lack of clarity and commitment is another strong tendency both among regulators and HTA organizations. Each HTA body has specific guidelines for incorporating RWE into health technology assessments, reflecting their priorities and methodological rigor. While they all recognize the value of RWE, the degree of reliance and the preferred statistical methods vary. ICER, CADTH, and NICE are particularly proactive in using advanced statistical techniques like Bayesian methods, while G-BA and IQWiG remain more conservative, emphasizing the gold standard status of RCTs and being cautious about newer methods. HAS, AIFA, AETS-ISCIII, SwissHTA, Chuikyo/JHEP, China’s CNHDRC, and Conitec all support the integration of RWE, but insist on high standards for data quality and methodological rigor to ensure robust and generalizable findings. In summary, while there is a consensus on the importance of high-quality data and among regulatory and HTA bodies worldwide, significant differences exist in the degree of prescriptiveness and acceptance of methods such as Bayesian analysis. The FDA and PMDA provide the most detailed and prescriptive guidance, reflecting a robust methodological framework. In contrast, agencies like G-BA and IQWiG remain more conservative, emphasizing RCTs as the gold standard and being cautious about newer methodologies.

The search strategy for this systematic literature review is designed to identify and analyze studies related to the use of external control arms in the context of RWD and RWE. The aim is to gather a comprehensive and relevant set of publications from the recent years, specifically from 2019-2024. The keywords used in the search strategy are focused on two main concepts: external control arms and RWE/ RWD. The search string employed is: (“external control” OR “external control arm”) AND (“RW” OR “RWE” OR “RWD” OR “real world data” OR “real world evidence”). This combination ensures that the search is broad enough to capture various terminologies used in the literature while being specific enough to filter out irrelevant studies. The inclusion criteria for this review are tailored to capture case studies and commentaries related to the regulatory or HTA use of RWD-derived external controls. This includes case studies that focus on regulatory or HTA applications, as well as commentaries from regulatory bodies, HTA bodies, and industry-related experiences regarding the use of RWD-derived controls for regulatory or HTA purposes. Additionally, value-based agreements, particularly those involving coverage with evidence development (CED), were included. To ensure the relevance and quality of the studies, several exclusion criteria are applied.

Methodology papers explaining analysis methods, systematic reviews on methodological topics, and case studies that do not cover regulatory or HTA-related usage of RWD-derived controls were excluded. Systematic reviews that do not address regulatory or HTA-related usage of RWD-derived controls, studies focused on pharmacovigilance without reference to RWD or RWE, and value-based agreements that do not mention RWD or RWE were excluded as well. The search was conducted on PubMed and Google Scholar. After initial screening based on titles and abstracts, a detailed review of the full texts of potentially relevant studies to confirm their inclusion based on the established criteria was conducted. Relevant data from the included studies were extracted, focusing on the application, outcomes, and commentary regarding RWD-derived external controls in regulatory and HTA contexts (Figure 1). In addition, a search for single-arm pivotal trials on ClinicalTrials.gov has been conducted. Identified compounds were reviewed with respect to RWE usage with respect to regulatory and HTA-related submission in their respective indications and selected if reasonable evidence on RWE usage or RWD-derived external controls with respect to regulatory and HTA-related submissions was publicly available, which resulted in the identification of the remaining compounds, belzutifan, sotorasib, and Zolgensma.

Figure 1

In total, seven recent drug approvals have been identified that utilized single-arm trials supported by RWD-derived external control arms to generate comparative evidence.

Rybrevant (amivantamab)

Amivantamab, approved by the FDA in May 2021, is used for the treatment of non-small cell lung cancer (NSCLC) with EGFR exon 20 insertion mutations. The approval was based on the CHRYSALIS study, a single-arm trial that demonstrated significant clinical activity. To provide comparative context, RWD was incorporated, which included historical patient data and real-world outcomes from similar patient populations who had not received amivantamab. This strategy was crucial due to the rarity of the mutation and the challenges in conducting traditional randomized controlled trials (RCTs). The FDA’s acceptance of this approach reflects a broader trend towards embracing non-traditional study designs, recognizing the potential of RWD to fill gaps where RCTs are not feasible. The FDA’s guidance on the use of externally controlled trials underscores the importance of early and strategic data planning, emphasizing the need for robust data capture and validation methods to ensure the reliability of RWD [57,58]. The European Medicines Agency (EMA) has granted conditional marketing authorization for the treatment of advanced non-small cell lung cancer (NSCLC) with activating EGFR exon 20 insertion mutations on October 14, 2021. As part of the conditional approval, the marketing authorization holder, Janssen-Cilag International N.V., was required to fulfill specific post-authorization commitments.

These included conducting a randomized, open-label Phase III study comparing amivantamab in combination with carboplatin and pemetrexed therapy against carboplatin and pemetrexed alone in the first-line treatment of advanced or metastatic NSCLC patients with activating EGFR exon 20 insertion mutations. This requirement underscores the EMA’s emphasis on gathering robust data to confirm the efficacy and safety of new therapies. The FDA’s and EMA’s decision-making processes regarding amivantamab illustrates a proactive approach to facilitating access to groundbreaking therapies while ensuring that ongoing studies provide the necessary evidence to support their continued use and expansion in clinical practice. The German Joint Federal Committee (G-BA) did not see “any additional benefit” of amivantamab vs. best supportive therapy, as G-BA considers the RWD-derived external controls to be “limited in quality” [59]. NICE states that amivantamab meets NICE’s criteria to be considered a life-extending treatment at the end of life, but does not recommend amivantamab for similar reasons: “Indirect comparisons using real-world evidence on immunotherapies, platinum-based chemotherapy, and docetaxel with or without nintedanib suggest that amivantamab increases how long people live, and how long they have before their cancer gets worse. But this is uncertain because there is no direct comparison, and because of the way the real-world evidence was chosen and presented” [60]. In contrast, CADTH recommended to “reimburse with clinical criteria and/or conditions” [61].

Welireg (belzutifan)

Belzutifan, approved by the FDA in August 2021, targets von Hippel-Lindau (VHL) disease-associated renal cell carcinoma. The approval was based on STUDY 004, a single-arm trial that showed promising results. To enhance the robustness of the data, RWD was used to create an external control arm, drawing from historical patient records and real-world clinical outcomes. This approach provided a necessary comparative context, highlighting the drug’s efficacy and safety profile. Other regulatory agencies, most notably EMA, have been much more cautious. CADTH has recommended reimbursement based on the available clinical trial evidence, which “suggested that treatment caused tumors to shrink or disappear for adult patients with VHL-associated nonmetastatic RCC” [62].

Lumykras (sotorasib)

Sotorasib, approved by the FDA in May 2021 for KRAS G12C-mutated NSCLC, is another example of a drug approved based on a single- arm trial supplemented with RWD. The CODEBREAK 100 trial provided the primary efficacy data, while RWD from historical and contemporary patient cohorts offered a comparative benchmark. This strategy was essential in demonstrating the drug’s benefit in a patient population with limited treatment options. The use of RWD in this case was instrumental in providing a comprehensive understanding of the drug’s impact, facilitating a faster and more informed regulatory review process. Based on the same data package, sotorasib has been approved as well by EMA, MHRA, and Swissmedic. G-BA considers this data package insufficient to demonstrate “any additional value” in favor of sotorasib, primarily because G-BA considers the “descriptive comparison of arms from different studies to be unadjusted and insufficient” [63]. HAS recommended reimbursement but came to a similar verdict with respect to additional value (ASMR 5) [64]. NICE recommended sotorasib “for use within the Cancer Drugs Fund as an option for treating KRAS G12C mutation-positive locally advanced or metastatic non-small-cell lung cancer in adults whose disease has progressed on, or who cannot tolerate, platinum-based chemotherapy or anti-PD-1/PD-L1 immunotherapy” as NICE considers the “indirect comparison” with current treatment to be “promising” [65].

Zolgensma (onasemnogene abeparvovec)

In the case of Zolgensma, real-world data from natural history studies of SMA patients were used to provide a comparative context for the single-arm trial results. These data helped to illustrate the significant improvement in motor function and survival rates in treated patients compared to the natural progression of the disease, thereby supporting the regulatory approval process. According to the FDA, “The primary evidence of effectiveness is based on results from the 21 patients treated with Zolgensma in the ongoing clinical trial. In this trial, there are 19 remaining patients, who range in age from 9.4- 18.5 months; 13 of these 19 patients are at least 14 months of age. Compared to the natural history of patients with infantile-onset SMA, patients treated with Zolgensma also demonstrated significant improvement in their ability to reach developmental motor milestones (e.g., head control and the ability to sit without support)” [66]. EMA “conditionally approved” Zolgensma as well [67]. NICE has considered natural history studies and recommended Zolgensma [68].

The history of G-BA’s verdict is more evolved. After the drug’s sales exceeded the “orphan drug sales” threshold, the drugs benefits have been reevaluated and “no additional benefit” could be discovered [69], mostly based on methodological shortcomings of indirect treatment comparisons based on clinical evidence. HAS expressed a “favorable opinion for reimbursement in the treatment of patients with 5q spinal muscular atrophy (SMA) with a bi-allelic mutation in the SMN1 gene and a clinical diagnosis of SMA Type 1 and 2 or pre-symptomatic SMA, with up to three copies of the SMN2 gene” because of “results that suggest a marked improvement compared to the natural course of the disease after two years in symptomatic patients but without recovery” [70].

Ayvakyt (avapritinib)

While the FDA approved avapritinib for the treatment of advanced systemic mastocytosis (AdvSM) based on the results from a single-arm pivotal trial [71], EMA approved considering a combination of data from a single-arm pivotal trial and RWE [72]. The single-arm trial provided the primary clinical data, while the RWE served as an external control to contextualize and support the efficacy findings from the trial. This RWE provided an external control arm, which helped to indirectly compare the efficacy of avapritinib against other treatments, thereby supporting the regulatory decision. G-BA refused to consider the same RWE in its assessment of the additional benefit in Germany, arguing that the “limited study documentation” and the methodology used for the indirect comparison made it impossible [42]. To compare the RWD-derived external control arm with the clinical trial treatment arm, the manufacturer had both used a “naïve historic comparison” and an “unanchored” matching-adjusted indirect comparison “based on aggregated data without bridge comparator” [73].

Tecartus (brexucabtagene autoleucel)

Brexucabtagene autoleucel was granted Accelerated Approval by the FDA for relapsed or refractory mantle cell lymphoma based on a single-arm trial with 74 patients [74]. The FDA granted Orphan Drug designation, Breakthrough Therapy designation, and Priority Review to brexucabtagene autoleucel for this indication [74]. There is no explicit mention in the search results of RWE or RWD-derived external controls being used in the FDA approval process for brexucabtagene autoleucel in MCL. NICE recommended usage within the Cancer Drug Fund based on the single-arm trial [75]. Half a year later, G-BA conditionally approved brexucabtagene autoleucel with unquantifiable additional benefit despite considering the matching-adjusted indirect comparison based on a meta-analysis summarizing eight external control studies to be inconclusive due to study population diversity [76]. CADTH recommended reimbursement based on the single-arm trial and the review committee argued “that the uncertainty in results of the sponsor-submitted unanchored MAIC were compounded be the inclusion of lower-quality comparator trials and clinical heterogeneity across studies” [77].

Enhertu (trastuzumab deruxtecan)

Trastuzumab deruxtecan is approved and widely reimbursed in multiple indications, partly based on single-arm trials or Phase II studies that compare two dosages of trastuzumab deruxtecan. RWE for efficacy evaluation has been included in the European public assessment report. With respect to HER2-positive NSCLC, G-BA considers the additional benefit of trastuzumab deruxtecan not demonstrated because no comparison with any therapeutic alternative has been submitted [78]. Recently, the FDA granted Accelerated Approval to trastuzumab deruxtecan (April 2024) for adult patients with unresectable or metastatic HER2-positive (IHC 3+) solid tumors who have received prior systemic therapy and have no satisfactory alternative treatment options based on single-arm trial evidence [79].

In conclusion, the integration of RWD-derived external control arms in single-arm clinical trials represents a significant advancement in drug development, particularly for rare diseases and in the arena of precision medicine. Regulatory and HTA guidelines uniformly emphasize the superiority of RCTs for establishing causal relationships. When guidance is provided, it focuses on justifying RWE studies over clinical trials rather than specifying preferred RWE study designs. Both regulatory and HTA guidelines consider RWE as complementary evidence. Regulatory guidelines typically recommend using RWE for clinical trial support and post-market surveillance, emphasizing robust study design and reliable methods when utilizing RWD. RWD sources primarily collected for non-research purposes, like medical records and claims, are viewed as more challenging due to their heterogeneity. There is a preference for prospectively designed RWD sources, particularly registries. Comprehensive documentation of the data collection process and maintaining data integrity and quality throughout extraction and curation are frequently emphasized. Transparency in data handling practices is also crucial. Few guidelines clearly define the specific circumstances and requirements for accepting RWE, particularly external controls derived from RWD. This lack of clarity and commitment is a notable trend among regulators and HTA organizations [80].

Each HTA body has specific guidelines for incorporating RWE into HTAs, reflecting their priorities and methodological rigor. While all recognize RWE’s value, the degree of reliance and preferred statistical methods vary. Organizations like ICER, CADTH, and NICE proactively use advanced statistical techniques such as Bayesian methods, whereas G-BA and IQWiG are more conservative, upholding RCTs as the gold standard and being cautious about newer methods. Other agencies like HAS, AIFA, AETS-ISCIII, SwissHTA, Chuikyo/JHEP, China’s CNHDRC, and Conitec support integrating RWE, but demand high standards for data quality and methodological rigor. Overall, there is consensus on the importance of high-quality data and advanced statistical methods among regulatory and HTA bodies worldwide, with differences in prescriptiveness and acceptance of methods like Bayesian analysis. This approach has facilitated approvals of innovative therapies like Rybrevant, Welireg, Lumykras, Zolgensma, Ayvakyt, Tecartus and Enhertu, providing robust evidence of efficacy and safety in the absence of traditional RCTs. The overall positive reception by regulatory agencies highlights the potential of RWD to transform clinical research and expedite access to new treatments for patients with unmet medical needs. With respect to the added benefit, HTA bodies in general seem to adopt a more cautious approach with respect to RWE usage and, in particular, RWD-derived external controls. While NICE and CADTH seem to be more open to RWE usage and RDW-derived modeling than most other HTA bodies, G-BA and particularly IQWiG tend to be a bit more cautious and conservative.

1. Sola Morales O, Curtis LH, Heidt J, Walsh L, Casso D, et al. (2023) Effectively Leveraging RWD for External Controls: A Systematic Literature Review of Regulatory and HTA Decisions. Clin Pharmacol Ther 114(2): 325-355.
2. (2024) FDA Guidance Considerations for the Use of Real-World Data and Real-World Evidence To Support Regulatory Decision-Making for Drug and Biological Products.
3. (2024) Use of Real-World Evidence to Support Regulatory Decision-Making for Medical Devices, Draft Guidance for Industry and Food and Drug Administration Staff.
4. (2024) Considerations for the Design and Conduct of Externally Controlled Trials for Drug and Biological Products Guidance for Industry, FDA Draft Guidance.
5. Allan V, Ramagopalan SV, Mardekian J, Jenkins A, Li X, et al. (2020) Propensity score matching and inverse probability of treatment weighting to address confounding by indication in comparative effectiveness research of oral anticoagulants. J Comp Eff Res 9(9): 603-614.
6. (2024) EMA guideline on registry-based studies.
7. (2024) EMA Reflection paper on use of real-world data in non-interventional studies to generate real-world evidence.
8. (2024) Real-world evidence framework to support EU regulatory decision-making.
9. HTA CG Methodological Guideline for Quantitative Evidence Synthesis: Direct and Indirect Comparisons, Adopted on 8 March by HTA CG.
10. (2024) EMA ICH Topic E 9 Statistical Principles for Clinical Trials.
11. EMA Guidance on Methodological Standards in Pharmacoepidemiology.
12. (2024) MHRA guidance on the use of real-world-data in clinical studies to support regulatory decisions.
13. https://www.biosliceblog.com/2022/05/use-of-real-world-evidence-in-the-uk/.
14. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9408280/.
15. NMPA Guidance on RWE.
16. Li J, Liu L, Cao H, Mei Yang , Xin Sun, et al. (2023) Use of real-world evidence to support regulatory decisions on medical devices in China and a unique opportunity to gain accelerated approval in “Boao Lecheng Pilot Zone”. Cost Eff Resour Alloc 21: 7.
17. Li P, Wang S, Chen Y (2023) Use of Real-World Evidence for Drug Regulatory Decisions in China: Current Status and Future Directions. Ther Innov Regul Sci 57: 1167-1179.
18. CDSCO Guidelines.
19. (2022) Indian Society for Clinical Research RWE Position paper.
20. CDSCO Guidance Document on procedures to be followed for evaluation of applications by the subject expert committees.
21. 7 PMDA RWE Guidance.
22. 7a Regulatory Utilization of Real-World Data and Real World Evidence in Japan, PMDA Chief Executive Perspective.
23. 7b Burns L, Le Roux N, Kalesnik Orszulak R, Christian J, Dudinak J, et al. (2023) Real-world evidence for regulatory decision-making: updated guidance from around the world. Front Med (Lausanne) 10: 1236462.
24. Swissmedic Guidelines.
25. (2023) Swissmedic Guideline: Swissmedic position paper on the use of realworld evidence.
26. ANVISA Guidelines on RWE.
27. (ANVISA) Brazilian Health Regulatory Agency: RWD and RWE in Brazil: Perspective of the Brazilian Health Regulatory Agency.
28. ICER Methods for Evaluating Evidence.
29. ICER’s Value Assessment Framework.
30. ICER’s Technical Briefs on Methods.
31. Jiao B, Veenstra DL, Lee W, Carlson JJ, Devine B (2020) The use of real-world evidence in ICER's scoping process and clinical evidence assessments. J Manag Care Spec Pharm 26(12): 1590-1595.
32. Lee W, Dayer V, Jiao B, Carlson JJ, Devine B, et al. (2021) Use of real-world evidence in economic assessments of pharmaceuticals in the United States. J Manag Care Spec Pharm 27(1): 5-14.
33. Jaksa A, Bloudek L, Carlson JJ, Shah K, Chen Y, et al. (2022) Key learnings from Institute for Clinical and Economic Review’s real-world evidence reassessment pilot. International Journal of Technology Assessment in Health Care 38(1): 1-6.
34. NICE Methods Guide.
35. (2022) NICE real-world evidence Framework.
36. Bray B, Ramagopalan SVR (2023) WE ready for reimbursement? A round up of developments in real-world evidence relating to health technology assessment: part 12. J Comp Eff Res 12(7): e230092.
37. G-BA Verfahrensordnung, in effect since February 2024.
38. IQWiG Methods Paper.
39. (2020) IQWiG Press Release: Registry-data – of sufficient quality -are suitable for the extended benefit assessment of drugs.
40. IQWiG Report Concepts for the generation of routine practice data and their analysis for the benefit assessment of drugs.
41. HAS Methodological Guide Real-world studies for the assessment of medicinal products and medical devices.
42. AIFA Guidelines.
43. Sievers H, Joos A, Hiligsmann M (2021) Real-world evidence: perspectives on challenges, value, and alignment of regulatory and national health technology assessment data collection requirements. International Journal of Technology Assessment in Health Care 37(1): e40.
44. AETS-ISCIII Guidelines.
45. SwissHTA HTA in Switzerland.
46. SwissHTA Evidence.
47. Swiss TPH.
48. Chuikyo Guidelines.
49. Zisis K, Pavi E, Geitona M, Athanasakis K (2024) Real-world data: a comprehensive literature review on the barriers, challenges, and opportunities associated with their inclusion in the health technology assessment process. J Pharm Pharm Sci 27: 12302.
50. Graili P, Guertin JR, Chan KKW, Tadrous M (2023) Integration of real-world evidence from different data sources in health technology assessment. J Pharm Pharm Sci 26: 11460.
51. CNHDRC Guidelines.
52. Conitec Guidelines.
53. Reason T, Rawlinson W, Langham J, Gimblett A, Malcolm B, et al. (2024) Artificial Intelligence to Automate Health Economic Modelling: A Case Study to Evaluate the Potential Application of Large Language Models. Pharmacoecon Open 8(2): 191-203.
54. CADTH Guidance on Reporting of Real-World-Evidence.
55. CADTH Guidance on Real-World-Evidence decision making.
56. (2022) Report on Best Brains Exchange: Optimizing the Use of Real-World-Evidence as part of decision-making for drugs for rare diseases.
57. https://www.cadth.ca/sites/default/files/RWE/pdf/MG0022_best_brains_exchange_optimizing_the_use_of_real_world_evidence_as_part_of_decision_making_for_drugs_for_rare_diseases.pdf.
58. https://www.fda.gov/regulatory-information/search-fda-guidance-documents/considerations-design-and-conduct-externally-controlled-trials-drug-and-biological-products.
59. https://www.appliedclinicaltrialsonline.com/view/reviewing-fda-s-latest-guidance-on-rwd.
60. G-BA Tragende Gründe zum Beschluss des Gemeinsamen Bundesausschusses über eine Änderung der Arzneimittel-Richtlinie: Anlage XII – Nutzenbewertung von Arzneimitteln mit neuen Wirkstoffen nach § 35a des Fünften Buches Sozialgesetzbuch (SGB V) Amivantamab (Lungenkarzinom, nicht-kleinzelliges, aktivierende EGFR-Exon-20-Insertionsmutationen, nach platinbasierter Chemotherapie) https://www.g-ba.de/downloads/40-268-8635/2022-07-07_AM-RL-XII_Amivantamab_D-788_TrG.pdf.
61. Amivantamab for treating EGFR exon 20 insertion mutation-positive advanced non-small-cell lung cancer after platinum-based chemotherapy.
62. CADTH Reimbursement Review Amivantamab.
63. CADTH Reimbursement Recommendation Belzutifan (Welireg).
64. (2022) G-BA Tragende Gründe zum Beschluss des Gemeinsamen Bundesausschusses über eine Änderung der Arzneimittel-Richtlinie: Anlage XII – Nutzenbewertung von Arzneimitteln mit neuen Wirkstoffen nach 35a des Fünften Buches Sozialgesetzbuch (SGB V) Sotorasib (Lungenkarzinom, nicht-kleinzelliges, KRAS G12CMutation, ≥ 1 Vortherapie).
65. (2022) HAS LUMYKRAS (sotorasib) - Cancer bronchique non à petites cellules (CBNPC).
66. (2022) Sotorasib for previously treated KRAS G12C mutation-positive advanced non-small-cell lung cancer.
67. FDA News Release, FDA approves innovative gene therapy to treat pediatric patients with spinal muscular atrophy, a rare disease and leading genetic cause of infant mortality.
68. EMA Zolgensma, https://www.ema.europa.eu/en/medicines/human/EPAR/zolgensma.
69. Onasemnogene abeparvovec for treating spinal muscular atrophy, July 2021, https://www.nice.org.uk/guidance/hst15/chapter/4-Consideration-of-the-evidence
70. G-BA Tragende Gründe zum Beschluss des Gemeinsamen Bundesausschusses über eine Änderung der Arzneimittel-Richtlinie: Anlage XII – Nutzenbewertung von Arzneimitteln mit neuen Wirkstoffen nach § 35a SGB V Onasemnogen-Abeparvovec (Überschreitung 50 Mio. € Grenze: Spinale Muskelatrophie), November 22021, https://www.g-ba.de/downloads/40-268-8006/2021-11-04_AM-RL-XII_Onasemnogen-Abeparvovec_D-679_TrG.pdf
71. HAS Zolgensam (onasemnogene abeparvovec), Dec 2020, https://www.has-sante.fr/jcms/p_3224937/en/zolgensma-onasemnogene-abeparvovec.
72. FDA approves avapritinib for advanced systemic mastocytosis, June 2021, https://www.fda.gov/drugs/resources-information-approved-drugs/fda-approves-avapritinib-advanced-systemic-mastocytosis.
73. https://www.analysisgroup.com/news-and-events/press-releases/analysis-group-announces-study-using-real-world-evidence-to-support-blueprint-medicines-ayvakyt-avapritinib-label-expansion-for-advanced-systemic-mastocytosis-in-europe/.
74. G-BA Bewertung von Arzneimitteln für seltene Leiden nach 35a Absatz 1 Satz 11 i. V. m. Kapitel 5 § 12 Nr. 1 Satz 2 VerfO Wirkstoff: Avapritinib (nAWG), June 2022. https://www.g-ba.de/downloads/92-975-5640/2022-04-01_Nutzenbewertung-G-BA_Avapritinib_D-798.pdf.
75. FDA approves brexucabtageen autoleucel for replapsed or refractory mantle cell lymphoma, https://www.fda.gov/drugs/resources-information-approved-drugs/fda-approves-brexucabtagene-autoleucel-relapsed-or-refractory-mantle-cell-lymphoma.
76. Brexucabtagene autoleucel for treating relapsed or refractory mantle cell lymphoma, February 2021, https://www.nice.org.uk/guidance/ta677.
77. G-BA Tragende Gründe zum Beschluss des Gemeinsamen Bundesausschusses über eine Änderung der Arzneimittel-Richtlinie (AM RL): Anlage XII – Nutzenbewertung von Arzneimitteln mit neuen Wirkstoffen nach 35a SGB V Autologe Anti-CD19-transduzierte CD3-positive Zellen (Mantelzell-Lymphom, vorbehandelte Patienten), August 2021, https://www.g-ba.de/downloads/40-268-7740/2021-08-05_AM-RL-XII_Autologe-Anti-CD19-transduzierte-CD3-positive-Zellen_D-633_TrG.pdf.
78. (2021) CADTH Reimbursement Review.
79. G-BA Tragende Gründe zum Beschluss des Gemeinsamen Bundesausschusses über eine Änderung der Arzneimittel-Richtlinie: Anlage XII – Nutzenbewertung von Arzneimitteln mit neuen Wirkstoffen nach 35a des Fünften Buches Sozialgesetzbuch (SGB V) Trastuzumab deruxtecan (Neues Anwendungsgebiet: nichtkleinzelliges Lungenkarzinom, HER2(ERBB2)-Mutation, vorbehandelt), May, 2024, https://www.g-ba.de/downloads/40-268-10501/2024-05-16_AM-RL-XII_Trastuzumab-deruxtecan_D-990_TrG.pdf.
80. FDA grants accelerated approval to fam-trastuzumab deruxtecan-nxki for unresectable or metastatic HER2-positive solid tumors.