{
  "$type": "site.standard.document",
  "bskyPostRef": {
    "cid": "bafyreiehcx3w7whl23qt2hukieolp2qau6p3pkruyssmy35q33aesnsn4a",
    "uri": "at://did:plc:wwyqal4cnqhuwyacdj7rqq3n/app.bsky.feed.post/3moq6wubm2tz2"
  },
  "path": "/t/randomized-non-comparative-trials-an-oxymoron/20863?page=5#post_83",
  "publishedAt": "2026-06-20T12:06:48.000Z",
  "site": "https://discourse.datamethods.org",
  "textContent": "Maybe this is how events play out (?):\n\n  * Researchers really don’t want to risk _missing_ an efficacy signal for a new therapy in a rare disease;\n\n  * They know that the most _reliable_ way to assess whether one therapy is superior to another is by conducting an RCT i.e., randomizing some patients to the new drug and other patients to the standard of care, then comparing their clinical trajectories;\n\n  * But researchers _also_ know that a _small_ RCT will be less capable of _detecting_ a new therapy’s superiority over standard of care than a _large_ RCT. Since few people will remain alive at each time point after randomization (due to the small number of trial participants), between-arm differences in survival rates at any given time point will be hard to interpret;\n\n  * Therefore, they worry that the small size of their trial could cause them to miss the signal for superiority that they are seeking from the new drug and which might actually be present;\n\n  * To address this fear, they propose a backup plan- an _alternate_ way to search for a potential superiority signal for the new therapy, should the _within_ -trial between-arm comparison fail to detect one;\n\n  * They decide, when designing their study, that if the between-arm comparison _within_ their small RCT ends up being non-informative, they will _instead_ look for an efficacy signal by comparing the clinical trajectory of patients treated with the new drug in their _current_ trial with the clinical trajectory of patients treated with standard of care therapy in _historical_ trials. **But this type of comparison, with historical controls, is no longer consistent with an “RCT” design. Rather, it’s a type of _observational cohort study (?)_ , with the progress of the two arms potentially being observed in _different time periods_ and using patients who are members of _different convenience samples_** ;\n\n  * Researchers settled on the term “Randomized Non-Comparative Trial” because they couldn’t justifiably call their study an “RCT” if they weren’t, realistically, planning to focus their statistical analysis on the randomized between-arm comparison _within_ their single convenience sample. They would plan to “eyeball” the _within_ -trial between-arm comparison _behind the scenes_ , but then _ignore_ it in their subsequent publication if they didn’t see any obvious separation of the trial arms. By randomizing, they would preserve their ability to 1) be happily surprised by unexpected detection of a superiority signal for the new drug; or 2) contribute their trial’s data to a future meta-analysis of RCTs involving the same treatment;\n\n  * **The problem:** If they don’t end up _reporting_ the _within_ -trial between-arm comparison (because they don’t see a superiority signal when they eyeball their results) and _instead_ end up focusing their analysis on the comparison with a _historical_ control, any failure to admit, _explicitly,_ that their main treatment comparison has become _observational_ in nature (rather than _randomized_) will mean that _readers_ might not recognize that they haven’t _met the standards_ required for a valid observational treatment comparison. Readers who don’t recognize that they are _actually_ reading about an _observational_ treatment comparison (because the word “randomized” is used in the study’s title) might not register that the authors haven’t tried to identify important between-**trial** , between-arm untreated prognostic distribution differences. These between-_trial_ differences need to be addressed because the patients in the arms being compared will come from _two different_ convenience samples (the _current_ trial’s convenience sample and the _historical_ study’s convenience sample)- not from a _single_ convenience sample, as would have been the case if the authors had compared the trial’s two _randomized_ arms;\n\n\n\n\n**RNCT authors need to take the precautions required for a valid _observational_ treatment comparison- the type of comparison that will usually end up being the _main_ focus of an \"R\"NCT.**",
  "title": "Randomized non-comparative trials: an oxymoron?"
}