Main
Task languages
18 languages
Macro-F1 on held-out prompts in the 18 task languages, for Subtask A and Subtask B.
LangCode 2027 Task 2 Leaderboard
Leaderboard
Metrics, development-phase scores, and official results.
Updates
Oct 30, 2026Training and development data release; CodaBench registration opens
Jan 15, 2027Test data release / evaluation begins; surprise languages revealed
Jan 24, 2027Registration deadline
Jan 31, 2027System submission deadline
Feb 3, 2027Official results released
Feb 15, 2027Shared-task paper submission deadline
Mar 26, 2027Notification of acceptance
Apr 16, 2027Camera-ready paper due
Jun 1–5, 2027Workshop at NAACL 2027, San Francisco
LangCode 2027Co-located with NAACL 2027 · June 1–5, 2027 · San Francisco, California
AlsoShared Task 1: Multilingual Code Generation
Feb 5, 2027Regular workshop papers due. See the Call for Papers
Metrics
Systems are ranked by macro-F1 over the classes C, so a detector cannot win by favoring the majority class:
Macro-F1=
1|C|
Σc ∈ C
F1(c)
Two leaderboards
Cross-lingual
Surprise languages
Never seen in training
Macro-F1 on surprise languages revealed on Jan 15. Does your guardrail generalize?
Development phase
Oct 30, 2026 – Jan 15, 2027. Scores on the dev set appear on the CodaBench leaderboard as you submit. They are for practice and do not count toward the final ranking.
Evaluation phase: official results
Final submissions on the new test set are made on CodaBench from Jan 15 to Jan 31, 2027. The official rankings will be published here on February 3, 2027.
| Rank | Team | Subtask A | Subtask B | Surprise |
|---|---|---|---|---|
| No results yet | ||||
How to submit: see Participate.