Evidence saturation k*: retrieval depth should be calibrated, not guessed

Thank you — this is a very useful distinction.

I agree that k should not be treated as one single thing. In a conventional RAG pipeline, it makes sense to distinguish between:

• raw retrieval candidate_k
• reranked or filtered rerank_k
• actually injected injected_k
• effective evidence depth after deduplication, compression, or prompt packing

That helps avoid an important misunderstanding: my k* is not meant to be the retriever’s raw top_k. It is closer to the final evidence depth that actually reaches the model, or even the effective evidence units that remain after filtering and consolidation.

I also agree that sufficiency, correctness, grounding, and contamination / framing leakage should be treated as separate reliability axes. A context can be sufficient while the model still fails to use it. And a model can remain factually correct while additional context already changes framing, source emphasis, or assumptions.

For transparency, I probably cannot run a full expanded test matrix right now. This is a privately funded pilot, and adding model × corpus × reranker × prompt-position × rubric-validation experiments would quickly become too expensive. So I would treat the larger matrix as future work, not as something the current pilot already proves.

But I can incorporate your distinction into the write-up as terminology and as a limitation. The careful claim would be:

Under this specific corpus, chunking setup, prompt format, model set, and evaluation rubric, the measured reliability curves suggest that retrieval depth should be calibrated rather than guessed.

Longer term, I also think this points beyond k as the primary control variable. In a more state-oriented architecture, the central question is not simply “how many chunks should we put into the prompt?”, but rather:

Which evidence is authorized, current, relevant, non-conflicting, and allowed to influence the answer?

In that setting, k becomes less of a steering knob and more of a diagnostic trace: an observable consequence of prior evidence selection, state governance, and consolidation.

So I agree with your distinction for today’s RAG-style pipelines. I would just frame my long-term goal differently: not to find a universal best k, but to make evidence depth measurable, auditable, and eventually less dependent on guesswork.