Dichotomization

“I don’t think you can conclude that…The irony of responder analysis is that it fails at its original goal.”

Yes - I’ve read Dr.Senn’s articles and know he’s been screaming into the void about all this for many years:

The article linked in post #75 above seems, conceptually, horribly muddled to me- yet I fear that it’s impact might have been substantial…

Senn’s response and related publications:

https://link.springer.com/article/10.1177/009286150303700103#citeas

https://pmc.ncbi.nlm.nih.gov/articles/PMC524113/

https://errorstatistics.com/wp-content/uploads/2016/07/senn-2003-pharmaceutical_statistics.pdf

If you were to re-run the hypothetical trial described in post #75, you might obtain the same between-arm difference of 6 points. But this time, drilling down to see what happened to each individual patient’s score, you might see a completely different distribution of point score changes over the course of the trial. If you were to conclude, as a believer in “responder analysis,” based on analysis of your first trial, that “10% of patients exposed to this drug will respond exceptionally well,” how will you react when you repeat the trial and obtain the same between-arm difference of 6 points, but this time observe that nearly all patients’ scores changed by the same number of points over the course of the trial? If you had run the second trial before the first trial, you would NOT have concluded that the “worked exceptionally well” in 10% of patients, but rather that all patients “respond similarly.” This simple example illustrates the folly of the responder analysis approach and the importance of acknowledging the stochasticity in patients’ ostensible “response” to treatment, from one treatment episode to another.

Most importantly, the fact that a patient’s score changed over the course of the trial does NOT allow us to infer that the treatment he received caused that change, EVEN IF the treatment is one with established group -level efficacy. It’s valid to infer that the new drug “caused” the between-group/arm difference of 6 points (i.e., that the new drug caused one group’s score to wind up 6 points different from the other group’s score; we can say that the drug has meaningful intrinsic efficacy). But it’s NOT valid to “translate” that established group -level inference of efficacy to the level of individual patients enrolled in the trial (for the purpose of labelling them as “responders” or “non-responders”). For diseases with waxing/waning natural histories, replication (otherwise known as “crossover” or “positive dechallenge and/or rechallenge”) at the level of the individual is needed to establish causality at the level of individual patients. And since dechallenge/rechallenge/crossover is NOT a feature of most parallel group RCT designs, most trials do NOT allow us to make inferences of causality at the level of individual patients. Unless this erroneous, highly pernicious, and deeply entrenched conflation of group-level and individual-level causality is acknowledged and loudly criticized by statisticians, “responder analysis” will persist- and so will the practice that serves it: dichotomization of continuous endpoints.