This page preserves the accepted proposal's integrity principles while avoiding unconfirmed limits. Team size, submission frequency, API allowlists, and exact resource budgets will be added after organizer approval.
Who the competition is designed for
The primary leaderboard is intended for LLM-based and agentic repair systems from academic, industry, and independent teams. Registration rules, team-size limits, affiliation restrictions, and conflict-of-interest procedures will follow the official ICSE Competition Track requirements and the released rulebook.
Final registration and team-composition rules are TBA.
Non-LLM systems may appear as clearly labeled comparison baselines, subject to final policy.
A team must submit a conforming Agent and complete organizer-run evaluation to receive a final result.
Describe what the Agent actually uses
Each final entry must disclose enough information for organizers to understand and rerun the method without revealing hidden evaluator data.
- Base model names and exact versions when available
- Major prompts, retrieval sources, external tools, static analyzers, and repair-loop design
- Training, fine-tuning, external knowledge bases, or private data used for the entry
- Third-party APIs and services contacted during Agent execution
- Known nondeterminism, caching, preprocessing, and reproducibility constraints
Repair package content through the official interface
Specifications, sources, patches, logs, scripts, metadata, and architecture labels.
Prompting, retrieval, code search, static analysis, log parsing, and iterative Agent loops.
Specifications, build scripts, source code, tests, packaging macros, and related package files.
Repairs need not match an organizer reference patch when they pass policy and executable validation.
Do not bypass the task or evaluator
No embedded hidden patches, case-specific answers, leaked labels, or lookup tables.
No attempts to escape isolation, alter evaluator metadata, spoof results, or rely on unintended validator behavior.
No deleting essential tests, bypassing the build, or suppressing failures without a legitimate repair.
The repaired output must remain the intended package and preserve its meaningful functionality.
Case manifests, checksums, hidden metadata, evaluator files, and other immutable inputs must remain unchanged.
Organizer-run cases must be processed by the submitted Agent under the announced interface and policy.
Do not assume unrestricted external access
The unified Agent model requires an explicit policy for model APIs and network access. That policy is not final. The release will state whether calls are disabled, allowlisted, or brokered through organizer-managed APIs.
Participants must never embed production API keys or personal credentials in submitted artifacts. Secret injection, logging, accounting, and redaction will be defined by the official platform.
Results must survive review and rerun
Organizers may request source, logs, configuration, method documentation, or a reproducible artifact for finalist verification. A result may be corrected, removed, or disqualified when it cannot be reproduced or violates the released rules.
| Situation | Potential action |
|---|---|
| Malformed or nonconforming Agent output | Case failure or invalid submission under the final scoring policy |
| Accidental evaluator defect | Announced fix and affected-entry rerun where feasible |
| Undeclared dependency or unreproducible finalist | Request clarification, corrected artifact, or result removal |
| Build bypass, hidden-answer leakage, or sandbox abuse | Disqualification and removal from the leaderboard |