Constrained
Small models
Limits on model size and compute
So the winning idea does not need the biggest GPU budget. Exact limits are announced with the training data.
LangCode 2027 Task 1 Leaderboard
Leaderboard
Metrics, development-phase scores, and official results.
Updates
Oct 30, 2026Training and development data release; CodaBench registration opens
Jan 15, 2027Test data release / evaluation begins
Jan 24, 2027Registration deadline
Jan 31, 2027System submission deadline
Feb 3, 2027Official results released
Feb 15, 2027Shared-task paper submission deadline
Mar 26, 2027Notification of acceptance
Apr 16, 2027Camera-ready paper due
Jun 1–5, 2027Workshop at NAACL 2027, San Francisco
LangCode 2027Co-located with NAACL 2027 · June 1–5, 2027 · San Francisco, California
AlsoShared Task 2: Adversarial Prompt Detection
Feb 5, 2027Regular workshop papers due. See the Call for Papers
Metrics
Systems are ranked by execution-based Pass@1 on hidden tests, macro-averaged across language resource tiers T:
Score=
1|T|
Σt ∈ T
Pass@1(t)
Pass@1: the share of problems whose single generated solution passes all hidden tests. Each tier counts equally.
Two tracks
Open
Anything goes
No limits
Show how far today's strongest systems reach across languages.
Development phase
Oct 30, 2026 – Jan 15, 2027. Scores on the dev set appear on the CodaBench leaderboard as you submit. They are for practice and do not count toward the final ranking.
Evaluation phase: official results
Final submissions on the new test problems are made on CodaBench from Jan 15 to Jan 31, 2027. The official ranking for each track will be published here on February 3, 2027.
| Rank | Team | Track | Score |
|---|---|---|---|
| No results yet | |||
How to submit: see Participate.