Randomized non-comparative trials: an oxymoron?

Maybe this is how events play out (?):

• Researchers really don’t want to risk missing an efficacy signal for a new therapy in a rare disease;

• They know that the most reliable way to assess whether one therapy is superior to another is by conducting an RCT i.e., randomizing some patients to the new drug and other patients to the standard of care, then comparing their clinical trajectories;

• But researchers also know that a small RCT will be less capable of detecting a new therapy’s superiority over standard of care than a large RCT. Since few people will remain alive at each time point after randomization (due to the small number of trial participants), between-arm differences in survival rates at any given time point will be hard to interpret;

• Therefore, they worry that the small size of their trial could cause them to miss the signal for superiority that they are seeking from the new drug and which might actually be present;

• To address this fear, they propose a backup plan- an alternate way to search for a potential superiority signal for the new therapy, should the within -trial between-arm comparison fail to detect one;

• They decide, when designing their study, that if the between-arm comparison within their small RCT ends up being non-informative, they will instead look for an efficacy signal by comparing the clinical trajectory of patients treated with the new drug in their current trial with the clinical trajectory of patients treated with standard of care therapy in historical trials. But this type of comparison, with historical controls, is no longer consistent with an “RCT” design. Rather, it’s a type of observational cohort study (?) , with the progress of the two arms potentially being observed in different time periods and using patients who are members of different convenience samples ;

• Researchers settled on the term “Randomized Non-Comparative Trial” because they couldn’t justifiably call their study an “RCT” if they weren’t, realistically, planning to focus their statistical analysis on the randomized between-arm comparison within their single convenience sample. They would plan to “eyeball” the within -trial between-arm comparison behind the scenes , but then ignore it in their subsequent publication if they didn’t see any obvious separation of the trial arms. By randomizing, they would preserve their ability to 1) be happily surprised by unexpected detection of a superiority signal for the new drug; or 2) contribute their trial’s data to a future meta-analysis of RCTs involving the same treatment;

• The problem: If they don’t end up reporting the within -trial between-arm comparison (because they don’t see a superiority signal when they eyeball their results) and instead end up focusing their analysis on the comparison with a historical control, any failure to admit, explicitly, that their main treatment comparison has become observational in nature (rather than randomized) will mean that readers might not recognize that they haven’t met the standards required for a valid observational treatment comparison. Readers who don’t recognize that they are actually reading about an observational treatment comparison (because the word “randomized” is used in the study’s title) might not register that the authors haven’t tried to identify important between-trial , between-arm untreated prognostic distribution differences. These between-trial differences need to be addressed because the patients in the arms being compared will come from two different convenience samples (the current trial’s convenience sample and the historical study’s convenience sample)- not from a single convenience sample, as would have been the case if the authors had compared the trial’s two randomized arms;

RNCT authors need to take the precautions required for a valid observational treatment comparison- the type of comparison that will usually end up being the main focus of an "R"NCT.