Retrieval

Research Question Can knowledge graph (KG) navigation bring a 7B parameter model to within 1 point of a frontier model on statutory reasoning tasks? Sub-questions: How much does KG navigation help small models vs. introduce noise? Which questions benefit most from graph evidence? What proportion of the gap is closeable via prompt engineering vs. graph quality vs. model capability? Is there a model-capability floor below which KG navigation is ineffective? Does natural language serialization of graph content help or hurt differently across model tiers? Experimental Setup Knowledge Graph A manually constructed knowledge graph covering six legal/technical domains: ...

This benchmark compares four answer-generation architectures across two model backends on 20 freshness questions designed to defeat parametric knowledge. Approaches B0: Vanilla LLM — one direct model answer with no tools or search. B1: LLM + Search — query generation, web search/fetch, and a single evidence-grounded answer. B2: LLM + Search + Reflection — search-based draft, self-critique, and revised answer. MA: Multi-Agent — Meta-Planner → Research → Synthesis → Validator → Skeptic → Final Answer with iterative convergence and deterministic grounding. Question design The 20-question freshness pack uses facts published within ~24-48 hours of the experiment (June 22-23, 2026), sourced from NASA releases, arXiv postings, a Google Developers post, and a GitHub release feed. Questions are designed so models cannot answer from training data — B0 scores ~2-3 across both models, confirming the freshness property. ...