Coding Knowledge Graph Agent Benchmark
Research Question Can a code knowledge graph (KG) with multi-agent navigation bring 7-8B parameter models closer to frontier-model performance on code generation, repair, and completion tasks? Sub-questions: Does KG navigation help small models write more correct code, or does it add noise they can’t filter? Does the multi-agent pipeline (draft → directional reasoning → test → refine) recover ground that single-agent KG loses? Which task types benefit most from graph evidence (generation vs. repair vs. completion)? Is there a model-capability floor below which KG navigation is ineffective — and does it differ from the legal KG floor? Can the pipeline enable small models to solve tasks that frontier B0 cannot? Experimental Setup Code Knowledge Graph A knowledge graph automatically extracted from a multi-module Python codebase (payments, user_service, notifications, email_service) using static analysis. ...