How to Successfully Employ External Comparator Arm Studies Using Real World Data –

Although randomized clinical trials (RCTs) are the gold standard for drug registration trials, the shift toward precision medicine has increased the use of single-arm trials (SATs). SATs lack results from control patients. Therefore, external comparator arms (ECAs) that compile data from external sources such as patient registries and other medical records can be used to contextualize study results. However, methodological considerations need to be made to ensure best practice and minimize potential bias in ECA study designs.

The differences between RCTs and ECAs

By randomizing both the treatment and control groups, RCTs allow researchers to control for potential bias and the influence of unmeasured variables. In addition, RCTs do not have to rely on two different data sources that may have different operational definitions, assessment methods, and measurement timing. However, in certain cases RCTs are not feasible and in this case SATs can be used.

When considering trends in precision medicine targeting specific biomarkers in patient populations of the same disease category, SATs allow researchers to evaluate new treatment options for smaller patient cohorts. To increase the power of the SAT results, the researchers used real-world data (RWD) from patients with the same characteristics to act as an external comparison arm. However, several factors should be considered before conducting ECA studies to ensure the results are statistically valid.

Importance of sample size in ECAs

As with any research study, sample size plays a critical role in ensuring that study results can be used as valid evidence of treatment efficacy and safety. When it comes to sample size for ECAs, there are several unique considerations, including whether data are already available for a treatment arm, whether additional precautions should be taken when estimating the required sample sizes, and how to incorporate the use of causal inference methods.

Populations and Treatment Conditions

Documenting detailed descriptions of the ECA population is critical, including the mechanisms and conditions that led to the patients being included in the data source. RWD sources typically only document comparators who actually received treatment, not those who should have received treatment, leading to a safety analysis population rather than an intention-to-treat (ITT) population, and this is recommended Compare like with like also in relation to the analysis populations.

A second consideration is determining for which target population treatment should be standardized in terms of estimating marginal treatment effects. That means identifying the estimates that describe the differences in treatment effects in different target populations, such as focusing on treated versus untreated populations. Also, the average treatment effect in the overlap population (ATO) is a possible estimate that focuses on the internal validity of the treatment comparison.

As with the study populations, the treatment conditions must also be clearly described for both the treatment and comparison groups. The description of the treatment group will be available via the clinical study protocol and the conditions in the RWD comparator group must be similarly described (e.g., appropriate dosages, route of drug administration, and frequency of drug intake). Since there is a likelihood of greater exposure time in the treatment group versus the comparison group due to the controlled setting of the SAT, investigators need to assess whether implementing a minimum number of treatment cycles or exposure times for both groups is helpful in achieving baseline exposure differences in the data sources, possibly as a sensitivity analysis.

Baseline and Endpoint Considerations

Before data collection, basic information must be established to ensure that the variable definitions of both groups are as identical as possible. The index date, ie the date on which the study officially begins, should be defined by the start of treatment initiation and not by the enrollment date in order to provide consistent definitions across all data sources.

Inclusion and exclusion criteria should be defined as uniformly as possible between the treatment and comparison groups. For example, the definition of treatment lines in location-based RWD sources is based on the physician’s clinical judgment and may differ from physicians in other locations and the SAT algorithm. In order to create more consistency across all datasets, it is necessary to review and frequently reclassify treatment lines and possibly other baseline data.

The measurement of endpoints in RWD and SATs can also differ due to different time points. For example, oncology studies with the RWD endpoint of progression (ie disease progression) may not be assessed using established classification rules, and some RWD sources may not even allow the use of such classification rules. While RW assessments of progression are typically less stringent compared to SATs, a bias can be introduced, possibly in favor of the SAT drug. One consideration for validating RWD progression assessments is a blinded central review employing reviewers who do not know from which group the data are derived.

Analysis Considerations

Propensity Score (PS) models are often used to analyze non-randomized data, but many other alternative methods are possible, e.g. B. Double-Robust, g-computation, non-PS weighting, or non-PS matching methods. A combination of approaches is also possible, e.g. B. Performing a PS weighting or g calculation after an initial fitting step.

Missing values ​​and unmeasured covariates are at the heart of the ECA design type, and effectively dealing with missing baseline covariate data is essential. Various sophisticated methods are possible and sensitivity analysis should be applied to check the robustness of the results.

discussion

In general, RCTs are the optimal approach for clinical research and drug development. However, when an RCT is not feasible — for example, because sample populations are too small when studying ultra-rare diseases — SATs with ECAs using real-world data can be a powerful tool to identify better treatments for patients.

In order to obtain statistically valid results, consideration of factors such as sample size, population and treatment conditions, baseline measurements and endpoint comparisons are crucial in the design and analysis of a study. However, thorough review will be key to the successful use of ECAs, as each disease type and data source presents its own challenges in the research process.

About the author

dr Gerd Rippin, Director of Biostatistics at IQVIA, received his bachelor’s degree in statistics from the University of Dortmund, Germany in 1995 and his PhD in 1999 from the University of Mainz, Germany. He has spent most of his career in the CRO business, including as a contractor and directly in the pharmaceutical industry. dr Rippin is a highly experienced biostatistician with over 20 years of experience applying statistical methods to clinical studies. His experience spans various indications and phases of medical research and he has a particular interest in External Comparator Arm (ECA) studies and the application of complex real-world statistical methods in general.