{
  "$type": "site.standard.document",
  "bskyPostRef": {
    "cid": "bafyreiejmudw5z2ozkhzbn2vfboexibkazqlph34osn5vwbblzpyt2vmu4",
    "uri": "at://did:plc:pgryn3ephfd2xgft23qokfzt/app.bsky.feed.post/3mpt5hvb2ipw2"
  },
  "path": "/t/evidence-saturation-k-retrieval-depth-should-be-calibrated-not-guessed/177363#post_3",
  "publishedAt": "2026-07-04T12:14:38.000Z",
  "site": "https://discuss.huggingface.co",
  "textContent": "Thank you — this is a very useful distinction.\n\nI agree that k should not be treated as one single thing. In a conventional RAG pipeline, it makes sense to distinguish between:\n\n  * raw retrieval candidate_k\n  * reranked or filtered rerank_k\n  * actually injected injected_k\n  * effective evidence depth after deduplication, compression, or prompt packing\n\n\n\nThat helps avoid an important misunderstanding: my k* is not meant to be the retriever’s raw top_k. It is closer to the final evidence depth that actually reaches the model, or even the effective evidence units that remain after filtering and consolidation.\n\nI also agree that sufficiency, correctness, grounding, and contamination / framing leakage should be treated as separate reliability axes. A context can be sufficient while the model still fails to use it. And a model can remain factually correct while additional context already changes framing, source emphasis, or assumptions.\n\nFor transparency, I probably cannot run a full expanded test matrix right now. This is a privately funded pilot, and adding model × corpus × reranker × prompt-position × rubric-validation experiments would quickly become too expensive. So I would treat the larger matrix as future work, not as something the current pilot already proves.\n\nBut I can incorporate your distinction into the write-up as terminology and as a limitation. The careful claim would be:\n\nUnder this specific corpus, chunking setup, prompt format, model set, and evaluation rubric, the measured reliability curves suggest that retrieval depth should be calibrated rather than guessed.\n\nLonger term, I also think this points beyond k as the primary control variable. In a more state-oriented architecture, the central question is not simply “how many chunks should we put into the prompt?”, but rather:\n\nWhich evidence is authorized, current, relevant, non-conflicting, and allowed to influence the answer?\n\nIn that setting, k becomes less of a steering knob and more of a diagnostic trace: an observable consequence of prior evidence selection, state governance, and consolidation.\n\nSo I agree with your distinction for today’s RAG-style pipelines. I would just frame my long-term goal differently: not to find a universal best k, but to make evidence depth measurable, auditable, and eventually less dependent on guesswork.",
  "title": "Evidence saturation k*: retrieval depth should be calibrated, not guessed"
}