This is the canonical repoloc-bench entry for the iTrust requirement-to-code traceability benchmark package.

It records the package and examples without copying the raw corpus into multiple places.

Source

  • Build brief: projects/state-estimation-traceability/research/dataset-benchmarking-buildout.md
  • Package: projects/state-estimation-traceability/data/derived/paper1-benchmark/
  • GitHub repo: 00n-ai/repoloc-bench
  • Reproduction bundle: benchmark_package_review_100
  • Snapshot commit: 64ecaab (Add benchmark reproduction bundle)

What is included

  • 34 requirements
  • 4,907 methods
  • 718 classes
  • 307 gold traces
  • masked gold-edge suites at 10/20/30%

Claim boundary

  • Primary branch is original requirement text.
  • LLM-imputed requirements are auxiliary sensitivity data only.
  • Gold links use traces with goldfinal == 'T'.
  • This is a method-localization benchmark foundation, not yet a test-localization benchmark.

Canonical package files

  • manifest.json
  • README.md
  • README.json
  • integrity.json
  • EXAMPLES.md
  • splits/
  • mask_suites/
  • systems/itrust/

Example outputs

{
  "task_id": "orig:itrust:R1",
  "branch": "original",
  "system": "itrust",
  "requirement_id": 1,
  "gold_method_ids": [154, 2851, 2986, 2992, 3125, 3126, 4655],
  "positive_trace_count": 7
}

{
  "node_id": "requirement:itrust:1",
  "original_text": "Add a New Patient ...",
  "imputed_text": "The system shall allow the addition of a new patient with basic information and generate a unique MID for them.",
  "gold_method_count": 7
}

{
  "mask_percent": 10,
  "gold_edges": 307,
  "held_out_gold_edges": 39,
  "observed_gold_edges": 268
}

Why this stays small

The benchmark package itself remains the source of truth. This page is only the registry and summary layer, so we do not duplicate the underlying dataset here.