Preliminary WMT24 Ranking of General MT Systems and LLMs
Tom Kocmi
(1)
,
Eleftherios Avramidis
(2)
,
Rachel Bawden
(3)
,
Ondrej Bojar
(4)
,
Anton Dvorkovich
(5)
,
Christian Federmann
(6)
,
Mark Fishel
(7)
,
Markus Freitag
(8)
,
Thamme Gowda
(9)
,
Roman Grundkiewicz
(10)
,
Barry Haddow
(11)
,
Marzena Karpinska
(12)
,
Philipp Koehn
(13)
,
Benjamin Marie
(14)
,
Kenton Murray
(13)
,
Masaaki Nagata
(15)
,
Martin Popel
(16)
,
Maja Popovic
(17, 18)
,
Mariya Shmatova
(5)
,
Steinþór Steingrímsson
(19)
,
Vilém Zouhar
(20)
1
Microsoft [Munich]
2 DFKI - Deutsches Forschungszentrum für Künstliche Intelligenz GmbH = German Research Center for Artificial Intelligence [Saarbruecken]
3 ALMAnaCH - Automatic Language Modelling and ANAlysis & Computational Humanities
4 IUUK - Computer Science Institute of Charles University [Prague]
5 Dubformer
6 Microsoft Research [Redmond]
7 Institute of Computer Science [University of Tartu, Estonie]
8 Google Inc [Mountain View]
9 Microsoft Corporation [Redmond]
10 Microsoft [Edinburg]
11 The University of Edinburgh
12 UMass Amherst - University of Massachusetts [Amherst]
13 JHU - Johns Hopkins University [Baltimore]
14 Chercheur indépendant
15 NTT Communication Science Laboratories
16 UK - Univerzita Karlova [Praha, Česká republika] = Charles University [Prague, Czech Republic] = Université Charles [Prague, Republique tchèque]
17 DCU - Dublin City University [Dublin]
18 IU International University of Applied Sciences
19 The Árni Magnússon Institute
20 ETH Zürich - Eidgenössische Technische Hochschule - Swiss Federal Institute of Technology [Zürich]
2 DFKI - Deutsches Forschungszentrum für Künstliche Intelligenz GmbH = German Research Center for Artificial Intelligence [Saarbruecken]
3 ALMAnaCH - Automatic Language Modelling and ANAlysis & Computational Humanities
4 IUUK - Computer Science Institute of Charles University [Prague]
5 Dubformer
6 Microsoft Research [Redmond]
7 Institute of Computer Science [University of Tartu, Estonie]
8 Google Inc [Mountain View]
9 Microsoft Corporation [Redmond]
10 Microsoft [Edinburg]
11 The University of Edinburgh
12 UMass Amherst - University of Massachusetts [Amherst]
13 JHU - Johns Hopkins University [Baltimore]
14 Chercheur indépendant
15 NTT Communication Science Laboratories
16 UK - Univerzita Karlova [Praha, Česká republika] = Charles University [Prague, Czech Republic] = Université Charles [Prague, Republique tchèque]
17 DCU - Dublin City University [Dublin]
18 IU International University of Applied Sciences
19 The Árni Magnússon Institute
20 ETH Zürich - Eidgenössische Technische Hochschule - Swiss Federal Institute of Technology [Zürich]
Rachel Bawden
- Fonction : Auteur
- PersonId : 9441
- IdHAL : rachel-bawden
- ORCID : 0000-0001-9553-1768
- IdRef : 233174591
Résumé
This is the preliminary ranking of WMT24 General MT systems based on automatic metrics. The official ranking will be a human evaluation, which is superior to the automatic ranking and supersedes it. The purpose of this report is not to interpret any findings but only provide preliminary results to the participants of the General MT task that may be useful during the writing of the system submission.