Research

Agentic systems need an objective An agent is not just a tool that produces output. In the canonical definitions, an agent is a temporally persistent system that observes, retains state or memory, chooses actions, and acts in service of an objective. That objective matters. Without it, the system is only reacting; with it, the system becomes agentic. This is consistent with the core agent literature: Russell & Norvig treat agents as systems that perceive and act in an environment, Sutton & Barto define the RL agent as the decision-making side of the loop, and control theory treats the analogous object as a controller acting toward a desired outcome. ...

Canonical terms These are the project definitions used across the research site. Agent A temporally persistent system that maps observations and internal state to actions in an environment according to a policy, in service of an objective. Russell & Norvig ; Sutton & Barto ; Åström & Murray Single agent One decision-making system acting on one environment. Sutton & Barto ; Åström & Murray Policy The rule or function that maps decision state to action. Sutton & Barto ; Åström & Murray ...

Abstract This paper studies whether a hierarchical evidence tree, coupled with local support/residual measurements, can improve code localization and downstream coding-agent repair. The work is organized as a single experimental program with three phases: (i) construction of a grounded repository-state tree for traceability and localization, (ii) reformulation of traceability drift as evidence-weighted innovation over observable state rather than hyperbolic transport geometry, and (iii) a frozen-edit Codex augmentation replay to test whether routing and evidence packets reduce wasted code and wrong-region edits. ...

Abstract This page collects notes on how to design multi-agent systems that are useful in practice: clear role boundaries, coordination patterns, shared state, task routing, evaluation, and failure handling. The focus is on systems that can explain what they are doing, route work intelligently, and remain measurable under real workloads. Focus areas role decomposition task routing and handoff shared memory and state coordination and control evaluation and failure modes guardrails and recovery separation of generation and evaluation Useful source Edward Y. Chang’s Multi-LLM Agent Collaborative Intelligence – The Path to Artificial General Intelligence is useful here because it treats multi-agent work as a pipeline where one set of agents generates or debates candidates and a separate set of judges evaluates them. That maps well to reliable multi-agent design: search first, judge later. ...

Research Question Can a code knowledge graph (KG) with multi-agent navigation bring 7-8B parameter models closer to frontier-model performance on code generation, repair, and completion tasks? Sub-questions: Does KG navigation help small models write more correct code, or does it add noise they can’t filter? Does the multi-agent pipeline (draft → directional reasoning → test → refine) recover ground that single-agent KG loses? Which task types benefit most from graph evidence (generation vs. repair vs. completion)? Is there a model-capability floor below which KG navigation is ineffective — and does it differ from the legal KG floor? Can the pipeline enable small models to solve tasks that frontier B0 cannot? Experimental Setup Code Knowledge Graph A knowledge graph automatically extracted from a multi-module Python codebase (payments, user_service, notifications, email_service) using static analysis. ...

Research Question Can knowledge graph (KG) navigation bring a 7B parameter model to within 1 point of a frontier model on statutory reasoning tasks? Sub-questions: How much does KG navigation help small models vs. introduce noise? Which questions benefit most from graph evidence? What proportion of the gap is closeable via prompt engineering vs. graph quality vs. model capability? Is there a model-capability floor below which KG navigation is ineffective? Does natural language serialization of graph content help or hurt differently across model tiers? Experimental Setup Knowledge Graph A manually constructed knowledge graph covering six legal/technical domains: ...

Companion document to the 00n.ai Multi-Agent Benchmark This document maps our benchmark findings to the published research literature. Each finding is evaluated against verified papers from arXiv, ACL, NeurIPS, ICML, and EMNLP. Papers were confirmed to resolve and titles verified against abstracts. Finding 1: MA Compresses the Capability Gap Between Small and Large Models A 7B model with multi-agent orchestration (Ollama qwen2.5:7b, avg 8.0–8.1) matches frontier-model B2 performance (GPT-4o search+reflection, avg 8.2–8.4) on evidence-grounded tasks. ...

This benchmark compares four answer-generation architectures across two model backends on 20 freshness questions designed to defeat parametric knowledge. Approaches B0: Vanilla LLM — one direct model answer with no tools or search. B1: LLM + Search — query generation, web search/fetch, and a single evidence-grounded answer. B2: LLM + Search + Reflection — search-based draft, self-critique, and revised answer. MA: Multi-Agent — Meta-Planner → Research → Synthesis → Validator → Skeptic → Final Answer with iterative convergence and deterministic grounding. Question design The 20-question freshness pack uses facts published within ~24-48 hours of the experiment (June 22-23, 2026), sourced from NASA releases, arXiv postings, a Google Developers post, and a GitHub release feed. Questions are designed so models cannot answer from training data — B0 scores ~2-3 across both models, confirming the freshness property. ...

This is the canonical repoloc-bench entry for the iTrust requirement-to-code traceability benchmark package. It records the package and examples without copying the raw corpus into multiple places. Source Build brief: projects/state-estimation-traceability/research/dataset-benchmarking-buildout.md Package: projects/state-estimation-traceability/data/derived/paper1-benchmark/ GitHub repo: 00n-ai/repoloc-bench Reproduction bundle: benchmark_package_review_100 Snapshot commit: 64ecaab (Add benchmark reproduction bundle) What is included 34 requirements 4,907 methods 718 classes 307 gold traces masked gold-edge suites at 10/20/30% Claim boundary Primary branch is original requirement text. LLM-imputed requirements are auxiliary sensitivity data only. Gold links use traces with goldfinal == 'T'. This is a method-localization benchmark foundation, not yet a test-localization benchmark. Canonical package files manifest.json README.md README.json integrity.json EXAMPLES.md splits/ mask_suites/ systems/itrust/ Example outputs { "task_id": "orig:itrust:R1", "branch": "original", "system": "itrust", "requirement_id": 1, "gold_method_ids": [154, 2851, 2986, 2992, 3125, 3126, 4655], "positive_trace_count": 7 } { "node_id": "requirement:itrust:1", "original_text": "Add a New Patient ...", "imputed_text": "The system shall allow the addition of a new patient with basic information and generate a unique MID for them.", "gold_method_count": 7 } { "mask_percent": 10, "gold_edges": 307, "held_out_gold_edges": 39, "observed_gold_edges": 268 } Why this stays small The benchmark package itself remains the source of truth. This page is only the registry and summary layer, so we do not duplicate the underlying dataset here. ...