Randomized non-comparative trials: an oxymoron?

The more I think about this thread, the more I wonder if the whole problem could simply reflect a failure to understand the advantages of concurrent versus historical control (?) I suspect the issue might be a lot more complicated than this though, since I have trouble believing that biostatisticians wouldn’t understand the importance of concurrent control (?)…

I don’t know whether this very crude graph could help to illustrate the problem or not (I realize this isn’t exactly how survival curves look).

The meaning of the legend terms is as follows:

CS1 4y SC - survival curve for C onvenience S ample #1 , from 4 y ears ago, for subjects treated with “S tandard of C are” therapy

CS2 4y SC - survival curve for C onvenience S ample #2 , from 4 y ears ago, for subjects treated with “S tandard of C are” therapy

CS3 T SC - survival curve for C onvenience S ample #3 , from T oday, for subjects treated with “S tandard of C are” therapy

CS4 T SC - survival curve for C onvenience S ample #4 , from T oday, for subjects treated with “S tandard of C are” therapy

CS5 T ND - survival curve for C onvenience S ample #5 , from T oday, for subjects treated with “N ew D rug” therapy

CS6 T SC - survival curve for C onvenience S ample #6 , from T oday, for subjects treated with “Standard of Care” therapy

CS6 T ND - survival curve for C onvenience S ample #6 , from T oday, for subjects treated with “New Drug” therapy

Key points conveyed by the graph:

• If concurrent control is available (as in an “RNCT” design), it doesn’t make sense to compare the survival trajectory for patients taking a New Drug (brown line) with the survival trajectory of patients treated with Standard of Care therapy in a non -concurrent treatment arm (i.e., patients from studies that used other convenience samples, either historical or current day- Blue, Grey, Green, or Yellow lines);
• Different convenience samples will generate different survival trajectories ; this will be true for patients treated with both Standard of Care therapy AND those treated with a New Drug. This phenomenon occurs because the manner in which the available “covariate space” defined by a trial’s inclusion criteria will end up being populated by patients who enrol in trial will differ from trial to trial, even when inclusion criteria are the same. For example, consider two trials with identical designs and inclusion criteria, each open to patients with a particular disease who are between the ages of 18 and 65. In the first trial, 75% of patients who enrol might end up being over the age of 50 and none might be under the age of 40, while in the other trial, 75% of patients enrolling might end up being under the age of 30 and none might be over the age of 50. If age is an important prognostic factor for the disease in question, then the survival trajectories for patients treated with Standard of Care therapy in these two trials could end up looking quite different;
• If researchers choose to compare the survival trajectory of patients treated with a New Drug in their present-day convenience sample (brown line) with the trajectory of patients treated with the Standard of Care therapy in another study (either past or present-day- Blue, Grey, Green, or Yellow lines), their inferences regarding the relative efficacy of the New Drug could be quite different than if they had relied on comparison with their concurrent control (pink line);
• In the absence of extensive efforts to identify, and adjust for, between-convenience-sample differences in untreated prognosis, the only way to reliably isolate the relative intrinsic efficacies of the therapies being compared is to perform the between-arm comparison within the same convenience sample (brown compared with pink). Only in this way can researchers eliminate the possibility that factors OTHER than superior New Drug efficacy might explain the differing trajectories in their between-arm comparison.