LangCode'27Shared Task 2

LangCode 2027 Task 2 Participate

Participate

Two phases on CodaBench: practice on the dev set, then submit on the test set.

Updates
Oct 30, 2026Training and development data release; CodaBench registration opens Jan 15, 2027Test data release / evaluation begins; surprise languages revealed Jan 24, 2027Registration deadline Jan 31, 2027System submission deadline Feb 3, 2027Official results released Feb 15, 2027Shared-task paper submission deadline Mar 26, 2027Notification of acceptance Apr 16, 2027Camera-ready paper due Jun 1–5, 2027Workshop at NAACL 2027, San Francisco LangCode 2027Co-located with NAACL 2027 · June 1–5, 2027 · San Francisco, California AlsoShared Task 1: Multilingual Code Generation Feb 5, 2027Regular workshop papers due. See the Call for Papers

The task runs on CodaBench in two phases. Participation is free. Registration opens on October 30, 2026; the competition link will be posted here that day.

Phase 1

Development phase

Oct 30, 2026 – Jan 15, 2027

Data
Training and dev sets, plus the starter kit
You
Build and tune your system
CodaBench
Submit predictions on the dev set to see your macro-F1 and check your file format. Practice only: it does not count toward the final ranking.

Phase 2 · Final

Evaluation phase

Jan 15 – Jan 31, 2027

Data
A new test set, released Jan 15, without labels, including surprise languages
You
Run your final system on it
CodaBench
Submit your test predictions by Jan 31. These decide the official ranking, published Feb 3.

Development phase · Oct 30, 2026 – Jan 15, 2027

  1. 1

    Register on CodaBench

    Create a CodaBench account, join the Task 2 competition, and accept the research-use terms for the data.

    Opens Oct 30, 2026 · Closes Jan 24, 2027

  2. 2

    Download the starter kit

    Train and dev sets, public baselines, and the submission format. What's in it

    Oct 30, 2026

  3. 3

    Build your system

    Start from a baseline and make it hold up across languages and scripts.

  4. 4

    Practice on CodaBench

    Predict labels for the dev set and submit them. You see your dev score and know your file format works.

    Until Jan 15, 2027

Evaluation phase · Jan 15 – Jan 31, 2027

  1. 5

    Get the test set

    New prompts you have not seen, without labels, including the surprise languages. Run your final system on them.

    Released Jan 15, 2027

  2. 6

    Submit your final predictions

    Upload your predictions for Subtask A, and for Subtask B if you take part in it, on CodaBench. This is the submission that is ranked.

    Deadline Jan 31, 2027, 11:59 PM AoE

After the competition

  1. 7

    See the results and write your paper

    The official ranking comes out on Feb 3, 2027. Then describe your system in a paper. Paper Submission

    Paper due Feb 15, 2027

Questions? Email mraihan@nd.edu, or see the organizers.