CDRAG: RAG with LLM-guided document retrieval — outperforms standard cosine retrieval on legal QA

Hi all,

I developed an addition on a CRAG (Clustered RAG) framework that uses LLM-guided cluster-aware retrieval. Standard RAG retrieves the top-K most similar documents from the entire corpus using cosine similarity. While effective, this approach is blind to the semantic structure of the document collection and may under-retrieve documents that are relevant at a higher level of abstraction.

CDRAG (Clustered Dynamic RAG) addresses this with a two-stage retrieval process:

1. Pre-cluster all (embedded) documents into semantically coherent groups

2. Extract LLM-generated keywords per cluster to summarise content

3. At query time, route the query through an LLM that selects relevant clusters and allocates a document budget across them

4. Perform cosine similarity retrieval within those clusters only

This allows the retrieval budget to be distributed intelligently across the corpus rather than spread blindly over all documents.

Evaluated on 100 legal questions from the legal RAG bench dataset ( isaacus/legal-rag-bench · Datasets at Hugging Face ), scored by an LLM judge:

• Faithfulness : +12% over standard RAG

• Overall quality : +8%

• Outperforms on 5/6 metrics

Code and full writeup available on GitHub. Interested to hear whether others have explored similar cluster-routing approaches. For more information, visit: GitHub - BartAmin/Clustered-Dynamic-RAG: CDRAG is a new retrieval framework that uses hierarchical document clustering and LLM-guided cluster selection to allocate the retrieval budget intelligently across a corpus, benchmarked against standard top-K RAG on 100 legal questions from the Legal RAG Bench dataset. · GitHub