iTrust Requirement-to-Code Traceability Benchmark
This is the canonical repoloc-bench entry for the iTrust requirement-to-code traceability benchmark package. It records the package and examples without copying the raw corpus into multiple places. Source Build brief: projects/state-estimation-traceability/research/dataset-benchmarking-buildout.md Package: projects/state-estimation-traceability/data/derived/paper1-benchmark/ GitHub repo: 00n-ai/repoloc-bench Reproduction bundle: benchmark_package_review_100 Snapshot commit: 64ecaab (Add benchmark reproduction bundle) What is included 34 requirements 4,907 methods 718 classes 307 gold traces masked gold-edge suites at 10/20/30% Claim boundary Primary branch is original requirement text. LLM-imputed requirements are auxiliary sensitivity data only. Gold links use traces with goldfinal == 'T'. This is a method-localization benchmark foundation, not yet a test-localization benchmark. Canonical package files manifest.json README.md README.json integrity.json EXAMPLES.md splits/ mask_suites/ systems/itrust/ Example outputs { "task_id": "orig:itrust:R1", "branch": "original", "system": "itrust", "requirement_id": 1, "gold_method_ids": [154, 2851, 2986, 2992, 3125, 3126, 4655], "positive_trace_count": 7 } { "node_id": "requirement:itrust:1", "original_text": "Add a New Patient ...", "imputed_text": "The system shall allow the addition of a new patient with basic information and generate a unique MID for them.", "gold_method_count": 7 } { "mask_percent": 10, "gold_edges": 307, "held_out_gold_edges": 39, "observed_gold_edges": 268 } Why this stays small The benchmark package itself remains the source of truth. This page is only the registry and summary layer, so we do not duplicate the underlying dataset here. ...