{
  "$type": "site.standard.document",
  "bskyPostRef": {
    "cid": "bafyreibllfrlvkvbbiwexngxiur7trymsxp2xkwgqxqdi3karklwkoqvt4",
    "uri": "at://did:plc:pgryn3ephfd2xgft23qokfzt/app.bsky.feed.post/3mpte676wu2k2"
  },
  "path": "/t/presenting-tis-token-importance-scoring-a-new-way-to-compress-kv-cache/177429#post_8",
  "publishedAt": "2026-07-04T15:05:15.000Z",
  "site": "https://discuss.huggingface.co",
  "textContent": "Yes, that distinction makes sense to me.\n\nI would probably keep the layers strictly separated:\n\n  1. **Upstream admission / evidence selection**\nThis happens before prefill. The system decides which evidence, state, or memory should enter the model context at all. This is where I would place DESi-style routing or epistemic state selection.\n  2. **Downstream retention / cache eviction**\nThis happens after tokens have already entered the model. Here TIS can be useful as a learned signal for deciding which tokens should survive cache compression or compaction.\n\n\n\nSo I would not treat TIS as an epistemic “source of truth” in the strong sense. In my architecture, the source of truth remains the external evidence/state layer. But TIS could be a very useful local mechanism once long prefill is unavoidable, or when importance only becomes visible during generation.\n\nA clean benchmark would be interesting:\n\n  * upstream evidence/state compression only\n  * TIS / KV-cache compression only\n  * both combined\n  * full-context baseline\n\n\n\nThat would show where the savings actually come from: avoiding context admission in the first place, preserving important tokens after admission, or combining both.",
  "title": "Presenting TIS (Token Importance Scoring) - A new way to compress KV cache"
}