ODESIA Challenge @ SEPLN 2024
(July 1 – September 14)
Registration open until July 30th, 2024
This challenge aims to promote the development and evaluation of language models in Spanish using the evaluation platform and datasets provided by the ODESIA project (Espacio de Observación del Desarrollo del Español en la Inteligencia Artificial).
The challenge consists of solving 10 discriminative tasks in Spanish, that belong to the ODESIA Leaderboard and are evaluated on private data. These tasks, with private evaluation data, belong to the ODESIA-CORE section of the ODESIA Leaderboard. The ODESIA Leaderboard is an application that provides an evaluation infrastructure for pretrained language models in English and Spanish that allows a direct comparison between the performance of models in one and the other language. Additionally, the leaderboard has an ODESIA-EXTENDED section with 4 tasks with pre-existing public evaluation data, but these are not part of the challenge. Although for all tasks ODESIA provides bilingual data (Spanish and English), this challenge focuses only on the Spanish tasks (Spanish portion of ODESIA-CORE).
PROGRAM: Wednesday 25-SEP-2024 17:30-18:30 [Room 1306]
ODESIA Challenge
17:30
- ODESIA project presentation: a project for the compared evaluation of the AI development in Spanish versus English. Presentation of the results and resources built along the two first years of the project: Portal ODESIA, ODESIA leaderboard y EvALL 2.0.
18:00
- Presentation of the ODESIA challenge, discussion of results and discusión de resultados presentation of the award for the best system.
The team submitting the best system will receive a cash prize of 3.000 euros, donated by the company Llorente y Cuenca Madrid, SL.