{
  "$type": "site.standard.document",
  "bskyPostRef": {
    "cid": "bafyreicrgcwq555t3shqwdv25ssufemkfdl4vapzy4vnwbc22mqpqb5aei",
    "uri": "at://did:plc:wwyqal4cnqhuwyacdj7rqq3n/app.bsky.feed.post/3mm2tyy7stz32"
  },
  "path": "/t/advice-when-models-perform-similarly-but-would-treat-different-patients/28748#post_3",
  "publishedAt": "2026-05-17T12:01:53.000Z",
  "site": "https://discourse.datamethods.org",
  "tags": [
    "@f2harrell"
  ],
  "textContent": "Thanks @f2harrell. Unfortunately I’m stuck with the stepwise no matter what, although most of the variables were pre-selected so it is what it is but I will add the stepwise optimism corrected calibration curves to help provide another layer of criticism on top of the bootstrap selection/ranking uncertainty graphs I’ve done already. BART has pretty aggressive shrinkage/penalization in small data applications (eg, most coefficients in PD plots are ~1 and you can see in the plot that closed circles are being pulled closer to average).\n\nI am a little cautious about doing more work to prove what we already know. I was hoping main contribution of this work to the team would be something like:\n\n  1. Variable selection in these data is unstable (bootstrap plots)\n  2. That instability leads to (potentially?) meaningful differences in the cohort of these patients who would have received additional follow–up.\n\n\n\nI have found a lot of resources on #1 but having trouble finding papers about ways to show #2.",
  "title": "Advice when models perform similarly but would treat different patients?"
}