{
  "$type": "site.standard.document",
  "bskyPostRef": {
    "cid": "bafyreigy5s4siwhtnx5qkabrp5hxvtgjbud47db7mbd63mfn2i7nqsdoky",
    "uri": "at://did:plc:pgryn3ephfd2xgft23qokfzt/app.bsky.feed.post/3mpple26uh2w2"
  },
  "path": "/t/paraconsistent-logic-and-ai-models/174262#post_20",
  "publishedAt": "2026-07-03T01:28:59.000Z",
  "site": "https://discuss.huggingface.co",
  "tags": [
    "(click for more details)"
  ],
  "textContent": "Hi. Last time I may have leaned a bit too far into the broader/general framing​, so this time I tried to reorganize things in a more practical “how to move forward” way, using Doninha 6.0 as the base:\n\n* * *\n\nMy short version would be:\n\n**I would make the next Doninha milestone very small, visible, and inspectable.**\n\nNot a full benchmark yet.\nNot a claim that Doninha “solves hallucination.”\nNot a huge architecture expansion.\n\nThe first useful proof target could be:\n\n\n    one prompt\n      → extracted claims\n      → evidence per claim\n      → support / opposition / insufficiency / vagueness\n      → license state\n      → final answer constrained by those states\n\n\nIn one sentence:\n\n**RAG retrieves evidence; Doninha decides whether that evidence licenses the claim.**\n\nThat distinction feels important to me. RAG can bring evidence into the system, but Doninha’s more interesting role may be deciding what the system is allowed to assert, what it should qualify, what it should mark as conflicted, and when it should not answer directly.\n\nSo I would frame the next step as:\n\n> **The first win should be inspectability, not performance.**\n\nIf Doninha can show its intermediate epistemic decisions, that is already a meaningful and testable step.\n\nMain framing: Doninha as a claim-level epistemic controller (click for more details)\n\n* * *\n\n## 1. First visible milestone: one-prompt claim ledger\n\nI would start with a very small demo.\n\nInput:\n\n\n    one prompt\n\n\nOutput:\n\n\n    final answer\n    claim_ledger.json\n\n\nA first ledger does not need to be perfect. It only needs to be visible.\n\nA minimal shape could be:\n\n\n    {\n      \"claim_id\": \"c1\",\n      \"claim_text\": \"Detergent does not kill 100% of bacteria.\",\n      \"claim_type\": \"empirical_factual\",\n      \"evidence\": [\n        {\n          \"source_id\": \"s1\",\n          \"source_span\": \"...\",\n          \"relation\": \"supports\"\n        }\n      ],\n      \"support\": 0.82,\n      \"opposition\": 0.10,\n      \"vagueness\": 0.05,\n      \"context_sufficiency\": \"sufficient\",\n      \"license_state\": \"licensed\",\n      \"final_action\": \"state_normally\"\n    }\n\n\nThe exact field names can change. The important part is the contract:\n\n> For each claim, Doninha should show why the final answer is allowed to state it, weaken it, reject it, retrieve more evidence, present a conflict, or abstain.\n\nPossible first license states:\n\nState | Meaning\n---|---\n`licensed` | evidence supports the claim enough to state it\n`qualified` | evidence partially supports it, but the final answer should hedge\n`needs_retrieval` | Doninha should retrieve before answering\n`insufficient_context` | retrieved context is not enough to license the claim\n`conflicted` | strong support and strong opposition both exist\n`refuted` | evidence contradicts the claim\n`source_mismatch` | a citation/source exists but does not support the claim\n`invalid_premise` | the prompt assumes something false or unverified\n`vague_or_ambiguous` | the predicate or question needs qualification\n`abstain` | Doninha should not assert the claim\n\nWhy this ledger matters (click for more details)\n\n* * *\n\n## 2. Second milestone: a five-case mini set\n\nAfter one ledger example works, I would make a tiny hand-written mini set.\n\nMaybe only five cases at first:\n\nCase | What it tests | Expected Doninha behavior\n---|---|---\nsupported claim | normal factual grounding | `licensed`\nunsupported claim | missing evidence | `insufficient_context` / `abstain`\nconflicting evidence | source A supports X, source B refutes X | `conflicted`\nvague predicate | “hot”, “cold”, “safe”, “intelligent” | `vague_or_ambiguous` / `qualified`\nfalse premise | prompt assumes something false | `invalid_premise`\n\nThis is not meant to be a leaderboard. It is a debugging set.\n\nThe point is not:\n\n> “Doninha beats model X.”\n\nThe point is:\n\n> “Doninha distinguishes epistemic behaviors that ordinary generation often collapses.”\n\nWhy these five cases are useful (click for more details)\n\n* * *\n\n## 3. Branch map: choose what Doninha should prove first\n\nDepending on what you want Doninha to prove first, the path changes.\n\nI would not implement all branches at once.\n\nBranch | If you want to prove… | First artifact\n---|---|---\nClaim ledger | Doninha is inspectable | one-prompt ledger JSON\nRAG faithfulness | Doninha checks evidence, not just retrieval | claim-evidence relation\nConflicting evidence | paraconsistency has a practical role | `conflicted` claim state\nAbstention | reliability includes not answering | `abstain` / `invalid_premise` cases\nVerification / CoVe | draft answers can be checked before final answer | verification questions per claim\nFormal reasoning | LLM should not do all logic alone | symbolic-checker route\nCorrective retrieval | bad retrieval should trigger correction | retrieval-quality grading\n\nDetailed branch map with references (click for more details)\n\n* * *\n\n## 4. What I would test first\n\nI would test whether the final answer obeys the ledger.\n\nThat is the important contract.\n\nLedger state | Final answer should…\n---|---\n`licensed` | state the claim normally\n`qualified` | hedge or narrow the claim\n`conflicted` | present the conflict without collapsing it\n`insufficient_context` | retrieve more, qualify, or abstain\n`invalid_premise` | correct the premise\n`source_mismatch` | not use that citation as support\n`vague_or_ambiguous` | add context, threshold, or clarification\n`formal` claim type | route to symbolic checker if possible\n\nA simple test question for every case:\n\n> Did the final answer respect the claim’s license state?\n\nIf not, then Doninha may have produced a good-looking answer, but the controller failed.\n\nPossible tiny test format (click for more details)\n\n* * *\n\n## 5. Practical implementation shape\n\nI would keep the first implementation boring and explicit.\n\nSomething like:\n\n\n    Step 1 — get candidate answer or candidate claims\n    Step 2 — split into atomic claims\n    Step 3 — attach or retrieve evidence\n    Step 4 — classify evidence relation per claim\n    Step 5 — assign license state\n    Step 6 — generate final answer under license-state constraints\n    Step 7 — save final answer + ledger\n\n\nThe first version can use hand-written evidence. It does not need to solve retrieval on day one.\n\nActually, I think that may be better: if retrieval and epistemic routing are both changing at the same time, debugging becomes harder.\n\nWhy I would separate retrieval from licensing at first (click for more details)\n\n* * *\n\n## 6. Pitfalls I would avoid\n\nA few things I would avoid while keeping the first demo small:\n\nPitfall | Better Doninha behavior\n---|---\nRAG = grounding | check whether evidence supports each claim\ncitation = support | verify cited span against the claim\none aggregate score = proof | keep claim-level decisions visible\nvagueness = contradiction | route vague predicates separately\nall failures = hallucination | distinguish retrieval, evidence, generation, and premise failures\narchitecture expansion first | make one tiny vertical slice work first\n\nMore detail on the main pitfalls (click for more details)\n\n* * *\n\n## 7. Suggested progress ladder\n\nIf I had to choose a practical next sequence, I would do this:\n\nStep | Artifact | Why it helps\n---|---|---\n1 | one-prompt ledger | proves Doninha can expose intermediate decisions\n2 | five-case mini set | separates supported / unsupported / conflicted / vague / false-premise cases\n3 | conflicting-evidence demo | gives the paraconsistent branch a clean test\n4 | abstention demo | shows reliability as restraint, not only answer generation\n5 | source-mismatch demo | prevents “citation = support” confusion\n6 | 20–50 case small benchmark | makes the project easier to compare and discuss\n7 | optional metrics | RAGAS / faithfulness / context metrics can be added after the ledger is stable\n\nThe shortest path is:\n\n\n    one prompt\n      → claim ledger\n      → final answer constrained by ledger\n\n\nThen:\n\n\n    five cases\n      → five expected epistemic behaviors\n\n\nThen:\n\n\n    20–50 cases\n      → small inspectable benchmark\n\n\nA possible first public demo success criterion (click for more details)\n\n* * *\n\n## 8. Concrete recommendation\n\nIf I had to choose one next action, I would build this first:\n\n\n    Doninha 6.0 — one-prompt ledger demo\n\n\nTarget:\n\n\n    Input:\n      one prompt\n\n    Output:\n      final answer\n      claim_ledger.json\n\n\nThen expand to:\n\n\n    five cases:\n      supported\n      unsupported\n      conflicted\n      vague\n      false premise\n\n\nAnd only after that, move toward larger benchmarks, automatic scoring, UI, or more layers.\n\nThis keeps the philosophical motivation intact, but gives the project a small testable object.\n\nFor me, that is the key move:\n\n> **Keep the philosophy, but make the next proof target small and visible.**",
  "title": "Paraconsistent Logic and AI models"
}