Abstract

This paper studies whether a hierarchical evidence tree, coupled with local support/residual measurements, can improve code localization and downstream coding-agent repair. The work is organized as a single experimental program with three phases: (i) construction of a grounded repository-state tree for traceability and localization, (ii) reformulation of traceability drift as evidence-weighted innovation over observable state rather than hyperbolic transport geometry, and (iii) a frozen-edit Codex augmentation replay to test whether routing and evidence packets reduce wasted code and wrong-region edits.

The broader literature framing is now narrower and stronger: attention and embedding retrieval are good at finding candidate regions, but the evidence needed to justify a requirement-satisfaction claim is relational, so the final support belongs in a graph or evidence subgraph.

Across Methods2Test routing, JHotDraw traversal, leaf localization, requirement-variant routing, Phase 2 calibration/drift experiments, and a six-task frozen Codex replay, the most consistent finding is that evidence-rich routing improves candidate narrowing, while the Codex augmentation packet remains mixed on patch churn and does not improve end-to-end repair success. Hyperbolic/Poincaré scoring does not outperform simpler baselines.

TL;DR

The tree is the win. It preserves structure, improves localization, and gives us a better way to route evidence toward likely edit regions. When we add that evidence to Codex, the agent’s search path changes, but the packet still does not reliably produce a final fix. So the current contribution is a stronger routing and error-measurement layer, not a full repair system.

Purpose and scope

This paper is an exploratory methodological study for our website and research program. Its purpose is to learn how to work effectively with AI research assistants, document open questions, and develop a repeatable research workflow. We used GPT-5.4 and GPT-5.5 to support literature review, synthesis, drafting, and analysis under author direction. The paper is intentionally framed as a first pass, not a final answer.

What this paper is / is not

  • Is: an initial methodological study of AI-assisted research; a record of open questions, working assumptions, and early experiments; a first attempt to learn repeatable ways of working with GPT-5.4 and GPT-5.5 as research assistants.
  • Is not: a finished theory of AI-assisted research; a claim that the workflow is optimal or complete; a production guide or a substitute for human judgment and review.

Manuscript

Key results

  • hierarchical evidence routing is stronger than flat similarity search for localization
  • attention/embedding retrieval is a strong candidate-region finder, not the final proof layer
  • local support/residual signals are useful for judging uncertainty
  • hyperbolic/Poincaré geometry does not beat the simpler discrete routing baselines
  • Codex augmentation changes routing and patch shape, but does not yet reliably improve repair success

Reproducibility

This paper is reproducible from the GitHub-backed research repo. The repo holds the code, data, experiment outputs, and paper artifacts.

Where to reproduce

  • GitHub reproduction bundle: 00n-ai/repoloc-bench
  • GitHub bundle snapshot: research/reproduction/benchmark_package_review_100
  • GitHub source of truth: state-estimation-traceability
  • Key reproducibility note: research/reproducibility-and-release.md
  • Benchmark summary: experiments/benchmark_package_review_100/README.md
  • Benchmark metrics: experiments/benchmark_package_review_100/benchmark_package_summary.json
  • Codex replay evidence: research/experiment-runs/2026-06-04T232257Z-h2_18-d1/metrics_summary.json

Reproduction rule

Re-run from the exact commit, with the documented command line and seed, and compare the new outputs to the saved summary files before publishing any claim here.