{
  "$type": "site.standard.document",
  "bskyPostRef": {
    "cid": "bafyreibl6c67ic566vqemntvte4zlg5s2zl5r7bwpkwtcqu2tx7lnnfnl4",
    "uri": "at://did:plc:wwyqal4cnqhuwyacdj7rqq3n/app.bsky.feed.post/3mmkcadolyum2"
  },
  "path": "/t/relaxing-assumptions-and-targeted-estimands-with-most/28755#post_2",
  "publishedAt": "2026-05-23T15:00:47.000Z",
  "site": "https://discourse.datamethods.org",
  "tags": [
    "this paper",
    "@philb",
    "here"
  ],
  "textContent": "A warm welcome to `datamethods` Michael. I’m glad you are here.\n\nThis is an extremely important topic for which a full discussion will lead to our developing better statistical analysis plans.\n\nI want to first state that the promise of targeted maximum likelihood estimation has not been fulfilled. Model uncertainty is always in play, and TMLE does not properly take it into account. TMLE is able to provide trustworthy point estimates, but as shown in this paper is not able to correctly estimate uncertainties in point estimates (it very much underestimates standard errors). The authors had to put the entire TMLE process inside a bootstrap loop to get valid SEs. One TMLE analysis is very computationally expensive, but doing it 200 times may be out of the question. This also points out a more fundamental problem with the usage of complex predictive methods based on adaptive (rather than pre-specified) methods when used in pivotal studies.\n\nIn addition to the challenge of TMLE in estimating uncertainties, it has not been adequately demonstrated that TMLE improves the mean squared error in estimating a targeted effect over what can be achieved by simpler methods. As an aside, the one time I tried to use the _super learner_ used by TMLE, the random forest component of the super learner overfitted the data by a catastrophic amount, and the overall super learning gave far too much weight to this one model, resulting in a badly overfitted super learner ensemble.\n\nWhen formulating an overall attack plan, it is usually important to consider all that is known about disease processes and their (possibly differential) effects on different outcome components. Sometimes a unified model (like the proportional odds model) works very well, and sometimes modeling is best done separately for different components. This is exemplified by the research of Phil Boonstra @philb discussed here. In his example, there is reason to model mortality separately from nonfatal outcome components. This is especially true if we expect risk factors for mortality to be much different from risk factors for nonfatal components.\n\nIn other cases, as shown here it may be better to have a unified model but to allow pre-specified exceptions, here using partial proportional odds for treatment. This approach has the advantage of letting you specify how much borrowing you want to allow for learning the treatment effect on mortality vs. its other effects. This borrowing, through a skeptical prior on an X\\\\\\times Y interaction effect, boosts the effective sample size (e.g., when it’s difficult to get a sample with enough deaths to learn about mortality effects of treatment on its own).\n\nA proportional odds model for which the PO assumption is strongly violated for treatment will still tell you which treatment is better, in terms of moving patients to better outcome levels. It may provide a wrong standard error of the log odds ratio—this needs to be studied. But the PO model will give you inaccurate estimates of outcome risks or treatment benefit for certain levels of the ordinal outcome Y.\n\nI am trying to avoid specific targeting and instead spending more time on overall model specification.",
  "title": "Relaxing Assumptions and Targeted Estimands with MOST"
}