{
  "$type": "site.standard.document",
  "bskyPostRef": {
    "cid": "bafyreiacog6tvok3ko3u27jaamppgi4zs6pjydxj2p3ej7capgrl6aweai",
    "uri": "at://did:plc:pgryn3ephfd2xgft23qokfzt/app.bsky.feed.post/3mpt5hh5uq7r2"
  },
  "path": "/t/presenting-tis-token-importance-scoring-a-new-way-to-compress-kv-cache/177429#post_6",
  "publishedAt": "2026-07-04T13:08:46.000Z",
  "site": "https://discuss.huggingface.co",
  "textContent": "That makes sense for constrained local inference.\n\nThis is close to a distinction I am exploring in DESi: upstream context admission vs. downstream cache eviction. In DESi, a router/evidence selector runs before prefill and tries to decide which evidence/state should enter the model at all. That reduces token and cache pressure at the source.\n\nSo I would see TIS as complementary: useful when long prefill is unavoidable, or when importance is only discovered during generation. But for RAG/agent systems, I would benchmark it against an upstream router/evidence-selection baseline, because that changes the problem from “which tokens survive in cache?” to “which evidence should be admitted into context?”",
  "title": "Presenting TIS (Token Importance Scoring) - A new way to compress KV cache"
}