Graph-Native Retrieval (GraphRAG/HiRAG): When a Vector Isn’t Enough—and How to Implement It

Vector RAG is excellent at local similarity. It struggles with global questions (themes, causality, policy lineage) and temporal reasoning. A thin knowledge graph (KG) plus hierarchical traversal fixes that—without turning your stack into a PhD thesis.

Start intentionally small

Schema (v0): 3–5 entity types + high-signal relations.

• Entities: Spec, Decision, CodeUnit, Dataset, Ticket
• Relations: implements, supersedes, tested_by, references, effective_from

Why small? Easier to validate, easier to evolve, easier to explain.

Ingest pipeline (minimal, auditable)

1. Extract entities/edges with an LLM + regex/AST assists for code.
2. Validate with deterministic rules (IDs present, paths exist, timestamps sane).
3. Attach provenance: file, commit, line range, hash, timestamp.
4. Dual-index: vectors for passages; graph for entities/edges.

Retrieval strategy: coarse → fine

flowchart LR
  Q[User Query] --> C[Topic Cluster]
  C --> COM[Community Traverse]
  COM --> N[Neighborhood Sample]
  N --> P[Passage Fetch (Vector)]
  P --> S[Summarize + Cite Provenance]

• Start with community/cluster selection (cheap, global).
• Sample neighborhoods around high-degree/temporal nodes.
• Only then pull passages with vector search.
• Produce a grounded summary with citations to nodes/edges and passages.

Temporal questions made easy

Model effective_from/effective_to edges and supersedes links. Now “what was true on 2024‑11‑01?” is a graph filter, not a prompt hack.

Evaluation that matters

Create query baskets:

• Local factual
• Global/causal
• Temporal lineage
• Code↔Policy traceability

Track: hit rate, groundedness, time-to-first-token, and hallucination rate (manual spot checks).

Pitfalls (and antidotes)

• Over‑graphing: don’t model everything; extend schema via usage signals.
• Stale edges: add aging jobs and supersession rules.
• Opaque provenance: store source + hash at edge creation; show it in answers.

Subtext: Teams that keep the KG thin, attach provenance, and traverse hierarchically get lineage answers that vanilla RAG can’t touch.