Trying to understand statistical methods through the lens of replication

HuwLlewelyn:

The definition of a P-value being the probability of an observed value or some other more extreme hypothetical values conditional on another single hypothetical value convinces many that statistics is impossible to truly understand.

This is a wonderful quote because of its perceptive double use of “hypothetical”. I hadn’t stopped to think of the first hypothetical. It’s important because the sampling distribution is a true hypothetical thought experiment that depends on (often unknown) intentions of the experimenter (such as timing and number of data looks).

There is a rich literature on calculating replication probabilities from initial p-values. Consider adding several references to your top post.

I’ll leave it to other to comment on your calculations. I’m not tempted to because I’m not that interested in replication probabilities and only care about the probability that the treatment works given whatever we currently know.