{
  "$type": "site.standard.document",
  "bskyPostRef": {
    "cid": "bafyreihmpjlwz6vdsxbnh7ufogdu6esnvdm3onzz2phml6jrh2jeggucqq",
    "uri": "at://did:plc:pgryn3ephfd2xgft23qokfzt/app.bsky.feed.post/3mhd65clubpw2"
  },
  "path": "/t/making-local-llms-more-reliable-with-a-deterministic-context-compiler/174368#post_1",
  "publishedAt": "2026-03-18T06:07:32.000Z",
  "site": "https://discuss.huggingface.co",
  "tags": [
    "github.com",
    "GitHub - rlippmann/context-compiler: Deterministic state engine for managing..."
  ],
  "textContent": "I’ve been experimenting with running LLMs locally, and kept running into a common issue:\n\n> constraints and corrections drift out of the prompt over time\n\nExample:\n\n  * User: “don’t use peanuts”\n\n  * …long conversation…\n\n  * Model suggests something with peanuts anyway\n\n\n\n\nThis gets worse with smaller models or limited context windows.\n\nSo I built a small deterministic tool called a _context compiler_.\n\nInstead of relying only on the transcript, it extracts structured state like:\n\n  * `facts.focus.primary = \"vegan curry\"`\n\n  * `policies.prohibit = [\"peanuts\"]`\n\n\n\n\nThen that state is injected into the prompt every turn, so important constraints don’t get lost.\n\nKey idea:\n\n  * prompt engineering helps\n\n  * **compiled state makes constraints persistent**\n\n\n\n\nI added a set of demos comparing:\n\n  * baseline prompting\n\n  * stronger prompt engineering\n\n  * prompt + compiled state\n\n\n\n\nThe interesting part is that better prompting improves things, but the compiled state is what actually guarantees invariants.\n\nRepo + demos:\n\ngithub.com\n\n### GitHub - rlippmann/context-compiler: Deterministic state engine for managing...\n\nDeterministic state engine for managing conversation state and constraints in LLM applications.",
  "title": "Making local LLMs more reliable with a deterministic “context compiler”"
}