BLOOM: A 176B-Parameter Open-Access Multilingual Language Model - Inria - Institut national de recherche en sciences et technologies du numérique
Preprints, Working Papers, ... Year : 2023

BLOOM: A 176B-Parameter Open-Access Multilingual Language Model

Teven Le Scao (1) , Angela Fan (1) , Christopher Akiki , Ellie Pavlick , Suzana Ilić , Daniel Hesslow , Roman Castagné (2) , Alexandra Sasha Luccioni , François Yvon (3, 4) , Matthias Gallé , Jonathan Tow , Alexander M. Rush , Stella Biderman , Albert Webson , Pawan Sasanka Ammanamanchi , Thomas Wang , Benoît Sagot (2) , Niklas Muennighoff , Albert Villanova del Moral , Olatunji Ruwase , Rachel Bawden (2) , Stas Bekman , Angelina Mcmillan-Major , Iz Beltagy , Huu Nguyen , Lucile Saulnier , Samson Tan , Pedro Ortiz Suarez (5) , Victor Sanh , Hugo Laurençon , Yacine Jernite , Julien Launay , Margaret Mitchell , Colin Raffel , Aaron Gokaslan , Adi Simhi , Aitor Soroa , Alham Fikri Aji , Amit Alfassy , Anna Rogers , Ariel Kreisberg Nitzav , Canwen Xu , Chenghao Mou , Chris Emezue , Christopher Klamm , Colin Leong , Daniel van Strien , David Ifeoluwa Adelani , Dragomir Radev , Eduardo González Ponferrada , Efrat Levkovizh , Ethan Kim , Eyal Bar Natan , Francesco de Toni , Gérard Dupont , Germán Kruszewski , Giada Pistilli , Hady Elsahar , Hamza Benyamina , Hieu Tran , Ian Yu , Idris Abdulmumin , Isaac Johnson , Itziar Gonzalez-Dios , Javier de la Rosa , Jenny Chim , Jesse Dodge , Jian Zhu , Jonathan Chang , Jörg Frohberg , Joseph Tobing , Joydeep Bhattacharjee , Khalid Almubarak , Kimbo Chen , Kyle Lo , Leandro von Werra , Leon Weber , Long Phan , Loubna Ben Allal , Ludovic Tanguy , Manan Dey , Manuel Romero Muñoz , Maraim Masoud , María Grandury , Mario Šaško , Max Huang , Maximin Coavoux , Mayank Singh , Mike Tian-Jian Jiang , Minh Chien Vu , Mohammad A. Jauhar , Mustafa Ghaleb , Nishant Subramani , Nora Kassner , Nurulaqilla Khamis , Olivier Nguyen , Omar Espejel , Ona de Gibert , Paulo Villegas , Peter Henderson , Pierre Colombo , Priscilla Amuok , Quentin Lhoest , Rheza Harliman , Rishi Bommasani , Roberto Luis López , Rui Ribeiro , Salomey Osei , Sampo Pyysalo , Sebastian Nagel , Shamik Bose , Shamsuddeen Hassan Muhammad , Shanya Sharma , Shayne Longpre , Somaieh Nikpoor , Stanislav Silberberg , Suhas Pai , Sydney Zink , Tiago Timponi Torrent , Timo Schick , Tristan Thrush , Valentin Danchev , Vassilina Nikoulina , Veronika Laippala , Violette Lepercq , Vrinda Prabhu , Zaid Alyafeai , Zeerak Talat , Arun Raja , Benjamin Heinzerling , Chenglei Si , Elizabeth Salesky , Sabrina J. Mielke , Wilson Y. Lee , Abheesht Sharma , Andrea Santilli , Antoine Chaffin , Arnaud Stiegler , Debajyoti Datta , Eliza Szczechla , Gunjan Chhablani , Han Wang , Harshit Pandey , Hendrik Strobelt , Jason Alan Fries , Jos Rozen , Leo Gao , Lintang Sutawika , M Saiful Bari , Maged S. Al-Shaibani , Matteo Manica , Nihal Nayak , Ryan Teehan , Samuel Albanie , Sheng Shen , Srulik Ben-David , Stephen H. Bach , Taewoon Kim , Tali Bers , Thibault Fevry , Trishala Neeraj , Urmish Thakker , Vikas Raunak , Xiangru Tang , Zheng-Xin Yong , Zhiqing Sun , Shaked Brody , Yallow Uri , Hadar Tojarieh , Adam Roberts , Hyung Won Chung , Jaesung Tae , Jason Phang , Ofir Press , Conglong Li , Deepak Narayanan , Hatim Bourfoune , Jared Casper , Jeff Rasley , Max Ryabinin , Mayank Mishra , Minjia Zhang , Mohammad Shoeybi , Myriam Peyrounette , Nicolas Patry , Nouamane Tazi , Omar Sanseviero , Patrick von Platen , Pierre Cornette , Pierre François Lavallée , Rémi Lacroix , Samyam Rajbhandari , Sanchit Gandhi , Shaden Smith , Stéphane Requena , Suraj Patil , Tim Dettmers , Ahmed Baruwa , Amanpreet Singh , Anastasia Cheveleva , Anne-Laure Ligozat (3, 6) , Arjun Subramonian , Aurélie Névéol (3, 6) , Charles Lovering , Dan Garrette , Deepak Tunuguntla , Ehud Reiter , Ekaterina Taktasheva , Ekaterina Voloshina , Eli Bogdanov , Genta Indra Winata , Hailey Schoelkopf , Jan-Christoph Kalo , Jekaterina Novikova , Jessica Zosa Forde , Jordan Clive , Jungo Kasai , Ken Kawamura , Liam Hazan , Marine Carpuat , Miruna Clinciu , Najoung Kim , Newton Cheng , Oleg Serikov , Omer Antverg , Oskar van der Wal , Rui Zhang , Ruochen Zhang , Sebastian Gehrmann , Shani Pais , Tatiana Shavrina , Thomas Scialom , Tian Yun , Tomasz Limisiewicz , Verena Rieser , Vitaly Protasov , Vladislav Mikhailov , Yada Pruksachatkun , Yonatan Belinkov , Zachary Bamberger , Zdeněk Kasner , Alice Rueda , Amanda Pestana , Amir Feizpour , Ammar Khan , Amy Faranak , Ana Santos , Anthony Hevia , Antigona Unldreaj , Arash Aghagol , Arezoo Abdollahi , Aycha Tammour , Azadeh Hajihosseini , Bahareh Behroozi , Benjamin Ajibade , Bharat Saxena , Carlos Muñoz Ferrandis , Danish Contractor , David Lansky , Davis David , Douwe Kiela , Duong A. Nguyen , Edward Tan , Emi Baylor , Ezinwanne Ozoani , Fatima Mirza , Frankline Ononiwu , Habib Rezanejad , Hessie Jones , Indrani Bhattacharya , Irene Solaiman , Irina Sedenko , Isar Nejadgholi , Jesse Passmore , Josh Seltzer , Julio Bonis Sanz , Livia Dutra , Mairon Samagaio , Maraim Elbadri , Margot Mieskes , Marissa Gerchick , Martha Akinlolu , Michael Mckenna , Mike Qiu , Muhammed Ghauri , Mykola Burynok , Nafis Abrar , Nazneen Rajani , Nour Elkott , Nour Fahmy , Olanrewaju Samuel , Ran An , Rasmus Kromann , Ryan Hao , Samira Alizadeh , Sarmad Shubber , Silas Wang , Sourav Roy , Sylvain Viguier , Thanh Le , Tobi Oyebade , Trieu Le , Yoyo Yang , Zach Nguyen , Abhinav Ramesh Kashyap , Alfredo Palasciano , Alison Callahan , Anima Shukla , Antonio Miranda-Escalada , Ayush Singh , Benjamin Beilharz , Bo Wang , Caio Brito , Chenxi Zhou , Chirag Jain , Chuxin Xu , Clémentine Fourrier (2) , Daniel León Periñán , Daniel Molano , Dian Yu , Enrique Manjavacas , Fabio Barth , Florian Fuhrimann , Gabriel Altay , Giyaseddin Bayrak , Gully Burns , Helena U. Vrabec , Imane Bello , Ishani Dash , Jihyun Kang , John Giorgi , Jonas Golde , Jose David Posada , Karthik Rangasai Sivaraman , Lokesh Bulchandani , Lu Liu , Luisa Shinzato , Madeleine Hahn de Bykhovetz , Maiko Takeuchi , Marc Pàmies , Maria A Castillo , Marianna Nezhurina , Mario Sänger , Matthias Samwald , Michael Cullan , Michael Weinberg , Michiel de Wolf , Mina Mihaljcic , Minna Liu , Moritz Freidank , Myungsun Kang , Natasha Seelam , Nathan Dahlberg , Nicholas Michio Broad , Nikolaus Muellner , Pascale Fung , Patrick Haller , Ramya Chandrasekhar , Renata Eisenberg , Robert Martin , Rodrigo Canalli , Rosaline Su , Ruisi Su , Samuel Cahyawijaya , Samuele Garda , Shlok S Deshmukh , Shubhanshu Mishra , Sid Kiblawi , Simon Ott , Sinee Sang-Aroonsiri , Srishti Kumar , Stefan Schweter , Sushil Bharati , Tanmay Laud , Théo Gigant , Tomoya Kainuma , Wojciech Kusa , Yanis Labrak (7, 8) , Yash Shailesh Bajaj , Yash Venkatraman , Yifan Xu , Yingxin Xu , Yu Xu , Zhe Tan , Zhongli Xie , Zifan Ye , Mathilde Bras , Younes Belkada , Thomas Wolf
Christopher Akiki
  • Function : Author
Ellie Pavlick
  • Function : Author
Suzana Ilić
  • Function : Author
Daniel Hesslow
  • Function : Author
Alexandra Sasha Luccioni
  • Function : Author
Matthias Gallé
  • Function : Author
Jonathan Tow
  • Function : Author
Alexander M. Rush
  • Function : Author
Stella Biderman
  • Function : Author
Albert Webson
  • Function : Author
Pawan Sasanka Ammanamanchi
  • Function : Author
Thomas Wang
  • Function : Author
Niklas Muennighoff
  • Function : Author
Albert Villanova del Moral
  • Function : Author
Olatunji Ruwase
  • Function : Author
Stas Bekman
  • Function : Author
Angelina Mcmillan-Major
  • Function : Author
Iz Beltagy
  • Function : Author
Huu Nguyen
  • Function : Author
Lucile Saulnier
  • Function : Author
Samson Tan
  • Function : Author
Victor Sanh
  • Function : Author
Hugo Laurençon
  • Function : Author
Yacine Jernite
  • Function : Author
Julien Launay
  • Function : Author
Margaret Mitchell
  • Function : Author
Colin Raffel
  • Function : Author
Aaron Gokaslan
  • Function : Author
Adi Simhi
  • Function : Author
Aitor Soroa
  • Function : Author
Alham Fikri Aji
  • Function : Author
Amit Alfassy
  • Function : Author
Anna Rogers
  • Function : Author
Ariel Kreisberg Nitzav
  • Function : Author
Canwen Xu
  • Function : Author
Chenghao Mou
  • Function : Author
Chris Emezue
  • Function : Author
Christopher Klamm
  • Function : Author
Colin Leong
  • Function : Author
Daniel van Strien
  • Function : Author
David Ifeoluwa Adelani
  • Function : Author
Dragomir Radev
  • Function : Author
Eduardo González Ponferrada
  • Function : Author
Efrat Levkovizh
  • Function : Author
Ethan Kim
  • Function : Author
Eyal Bar Natan
  • Function : Author
Francesco de Toni
  • Function : Author
Gérard Dupont
  • Function : Author
Germán Kruszewski
  • Function : Author
Giada Pistilli
Hady Elsahar
  • Function : Author
Hamza Benyamina
  • Function : Author
Hieu Tran
  • Function : Author
Ian Yu
  • Function : Author
Idris Abdulmumin
  • Function : Author
Isaac Johnson
  • Function : Author
Itziar Gonzalez-Dios
  • Function : Author
Javier de la Rosa
  • Function : Author
Jenny Chim
  • Function : Author
Jesse Dodge
  • Function : Author
Jian Zhu
  • Function : Author
Jonathan Chang
  • Function : Author
Jörg Frohberg
  • Function : Author
Joseph Tobing
  • Function : Author
Joydeep Bhattacharjee
  • Function : Author
Khalid Almubarak
  • Function : Author
Kimbo Chen
  • Function : Author
Kyle Lo
  • Function : Author
Leandro von Werra
  • Function : Author
Leon Weber
  • Function : Author
Long Phan
  • Function : Author
Loubna Ben Allal
  • Function : Author
Ludovic Tanguy
Manan Dey
  • Function : Author
Manuel Romero Muñoz
  • Function : Author
Maraim Masoud
  • Function : Author
María Grandury
  • Function : Author
Mario Šaško
  • Function : Author
Max Huang
  • Function : Author
Maximin Coavoux
  • Function : Author
Mayank Singh
  • Function : Author
Mike Tian-Jian Jiang
  • Function : Author
Minh Chien Vu
  • Function : Author
Mohammad A. Jauhar
  • Function : Author
Mustafa Ghaleb
  • Function : Author
Nishant Subramani
  • Function : Author
Nora Kassner
  • Function : Author
Nurulaqilla Khamis
  • Function : Author
Olivier Nguyen
  • Function : Author
Omar Espejel
  • Function : Author
Ona de Gibert
  • Function : Author
Paulo Villegas
  • Function : Author
Peter Henderson
  • Function : Author
Pierre Colombo
  • Function : Author
Priscilla Amuok
  • Function : Author
Quentin Lhoest
  • Function : Author
Rheza Harliman
  • Function : Author
Rishi Bommasani
  • Function : Author
Roberto Luis López
  • Function : Author
Rui Ribeiro
  • Function : Author
Salomey Osei
  • Function : Author
Sampo Pyysalo
  • Function : Author
Sebastian Nagel
  • Function : Author
Shamik Bose
  • Function : Author
Shamsuddeen Hassan Muhammad
  • Function : Author
Shanya Sharma
  • Function : Author
Shayne Longpre
  • Function : Author
Somaieh Nikpoor
  • Function : Author
Stanislav Silberberg
  • Function : Author
Suhas Pai
  • Function : Author
Sydney Zink
  • Function : Author
Tiago Timponi Torrent
  • Function : Author
Timo Schick
  • Function : Author
Tristan Thrush
  • Function : Author
Valentin Danchev
  • Function : Author
Vassilina Nikoulina
  • Function : Author
Veronika Laippala
  • Function : Author
Violette Lepercq
  • Function : Author
Vrinda Prabhu
  • Function : Author
Zaid Alyafeai
  • Function : Author
Zeerak Talat
  • Function : Author
Arun Raja
  • Function : Author
Benjamin Heinzerling
  • Function : Author
Chenglei Si
  • Function : Author
Elizabeth Salesky
  • Function : Author
Sabrina J. Mielke
  • Function : Author
Wilson Y. Lee
  • Function : Author
Abheesht Sharma
  • Function : Author
Andrea Santilli
  • Function : Author
Antoine Chaffin
  • Function : Author
Arnaud Stiegler
  • Function : Author
Debajyoti Datta
  • Function : Author
Eliza Szczechla
  • Function : Author
Gunjan Chhablani
  • Function : Author
Han Wang
  • Function : Author
Harshit Pandey
  • Function : Author
Hendrik Strobelt
  • Function : Author
Jason Alan Fries
  • Function : Author
Jos Rozen
  • Function : Author
Leo Gao
  • Function : Author
Lintang Sutawika
  • Function : Author
M Saiful Bari
  • Function : Author
Maged S. Al-Shaibani
  • Function : Author
Matteo Manica
Nihal Nayak
  • Function : Author
Ryan Teehan
  • Function : Author
Samuel Albanie
  • Function : Author
Sheng Shen
  • Function : Author
Srulik Ben-David
  • Function : Author
Stephen H. Bach
  • Function : Author
Taewoon Kim
  • Function : Author
Tali Bers
  • Function : Author
Thibault Fevry
  • Function : Author
Trishala Neeraj
  • Function : Author
Urmish Thakker
  • Function : Author
Vikas Raunak
  • Function : Author
Xiangru Tang
  • Function : Author
Zheng-Xin Yong
  • Function : Author
Zhiqing Sun
  • Function : Author
Shaked Brody
  • Function : Author
Yallow Uri
  • Function : Author
Hadar Tojarieh
  • Function : Author
Adam Roberts
  • Function : Author
Hyung Won Chung
  • Function : Author
Jaesung Tae
  • Function : Author
Jason Phang
  • Function : Author
Ofir Press
  • Function : Author
Conglong Li
  • Function : Author
Deepak Narayanan
  • Function : Author
Hatim Bourfoune
  • Function : Author
Jared Casper
  • Function : Author
Jeff Rasley
  • Function : Author
Max Ryabinin
  • Function : Author
Mayank Mishra
  • Function : Author
Minjia Zhang
  • Function : Author
Mohammad Shoeybi
  • Function : Author
Myriam Peyrounette
  • Function : Author
Nicolas Patry
  • Function : Author
Nouamane Tazi
  • Function : Author
Omar Sanseviero
  • Function : Author
Patrick von Platen
  • Function : Author
Pierre Cornette
  • Function : Author
Pierre François Lavallée
  • Function : Author
Rémi Lacroix
  • Function : Author
Samyam Rajbhandari
  • Function : Author
Sanchit Gandhi
  • Function : Author
Shaden Smith
  • Function : Author
Stéphane Requena
  • Function : Author
Suraj Patil
  • Function : Author
Tim Dettmers
  • Function : Author
Ahmed Baruwa
  • Function : Author
Amanpreet Singh
  • Function : Author
Anastasia Cheveleva
  • Function : Author
Arjun Subramonian
  • Function : Author
Charles Lovering
  • Function : Author
Dan Garrette
  • Function : Author
Deepak Tunuguntla
  • Function : Author
Ehud Reiter
  • Function : Author
Ekaterina Taktasheva
  • Function : Author
Ekaterina Voloshina
  • Function : Author
Eli Bogdanov
  • Function : Author
Genta Indra Winata
  • Function : Author
Hailey Schoelkopf
  • Function : Author
Jan-Christoph Kalo
  • Function : Author
Jekaterina Novikova
  • Function : Author
Jessica Zosa Forde
  • Function : Author
Jordan Clive
  • Function : Author
Jungo Kasai
  • Function : Author
Ken Kawamura
  • Function : Author
Liam Hazan
  • Function : Author
Marine Carpuat
  • Function : Author
Miruna Clinciu
  • Function : Author
Najoung Kim
  • Function : Author
Newton Cheng
  • Function : Author
Oleg Serikov
  • Function : Author
Omer Antverg
  • Function : Author
Oskar van der Wal
  • Function : Author
Rui Zhang
  • Function : Author
Ruochen Zhang
  • Function : Author
Sebastian Gehrmann
  • Function : Author
Shani Pais
  • Function : Author
Tatiana Shavrina
  • Function : Author
Thomas Scialom
  • Function : Author
Tian Yun
  • Function : Author
Tomasz Limisiewicz
  • Function : Author
Verena Rieser
  • Function : Author
Vitaly Protasov
  • Function : Author
Vladislav Mikhailov
  • Function : Author
Yada Pruksachatkun
  • Function : Author
Yonatan Belinkov
  • Function : Author
Zachary Bamberger
  • Function : Author
Zdeněk Kasner
  • Function : Author
Alice Rueda
  • Function : Author
Amanda Pestana
  • Function : Author
Amir Feizpour
  • Function : Author
Ammar Khan
  • Function : Author
Amy Faranak
  • Function : Author
Ana Santos
  • Function : Author
Anthony Hevia
  • Function : Author
Antigona Unldreaj
  • Function : Author
Arash Aghagol
  • Function : Author
Arezoo Abdollahi
  • Function : Author
Aycha Tammour
  • Function : Author
Azadeh Hajihosseini
  • Function : Author
Bahareh Behroozi
  • Function : Author
Benjamin Ajibade
  • Function : Author
Bharat Saxena
  • Function : Author
Carlos Muñoz Ferrandis
  • Function : Author
Danish Contractor
  • Function : Author
David Lansky
  • Function : Author
Davis David
  • Function : Author
Douwe Kiela
  • Function : Author
Duong A. Nguyen
  • Function : Author
Edward Tan
  • Function : Author
Emi Baylor
  • Function : Author
Ezinwanne Ozoani
  • Function : Author
Fatima Mirza
  • Function : Author
Frankline Ononiwu
  • Function : Author
Habib Rezanejad
  • Function : Author
Hessie Jones
  • Function : Author
Indrani Bhattacharya
  • Function : Author
Irene Solaiman
  • Function : Author
Irina Sedenko
  • Function : Author
Isar Nejadgholi
  • Function : Author
Jesse Passmore
  • Function : Author
Josh Seltzer
  • Function : Author
Julio Bonis Sanz
  • Function : Author
Livia Dutra
  • Function : Author
Mairon Samagaio
  • Function : Author
Maraim Elbadri
  • Function : Author
Margot Mieskes
  • Function : Author
Marissa Gerchick
  • Function : Author
Martha Akinlolu
  • Function : Author
Michael Mckenna
  • Function : Author
Mike Qiu
  • Function : Author
Muhammed Ghauri
  • Function : Author
Mykola Burynok
  • Function : Author
Nafis Abrar
  • Function : Author
Nazneen Rajani
  • Function : Author
Nour Elkott
  • Function : Author
Nour Fahmy
  • Function : Author
Olanrewaju Samuel
  • Function : Author
Ran An
  • Function : Author
Rasmus Kromann
  • Function : Author
Ryan Hao
  • Function : Author
Samira Alizadeh
  • Function : Author
Sarmad Shubber
  • Function : Author
Silas Wang
  • Function : Author
Sourav Roy
  • Function : Author
Sylvain Viguier
  • Function : Author
Thanh Le
  • Function : Author
Tobi Oyebade
  • Function : Author
Trieu Le
  • Function : Author
Yoyo Yang
  • Function : Author
Zach Nguyen
  • Function : Author
Abhinav Ramesh Kashyap
  • Function : Author
Alfredo Palasciano
  • Function : Author
Alison Callahan
  • Function : Author
Anima Shukla
  • Function : Author
Antonio Miranda-Escalada
  • Function : Author
Ayush Singh
  • Function : Author
Benjamin Beilharz
  • Function : Author
Bo Wang
  • Function : Author
Caio Brito
  • Function : Author
Chenxi Zhou
  • Function : Author
Chirag Jain
  • Function : Author
Chuxin Xu
  • Function : Author
Daniel León Periñán
  • Function : Author
Daniel Molano
  • Function : Author
Dian Yu
  • Function : Author
Enrique Manjavacas
  • Function : Author
Fabio Barth
  • Function : Author
Florian Fuhrimann
  • Function : Author
Gabriel Altay
  • Function : Author
Giyaseddin Bayrak
  • Function : Author
Gully Burns
  • Function : Author
Helena U. Vrabec
  • Function : Author
Imane Bello
  • Function : Author
Ishani Dash
  • Function : Author
Jihyun Kang
  • Function : Author
John Giorgi
  • Function : Author
Jonas Golde
  • Function : Author
Jose David Posada
  • Function : Author
Karthik Rangasai Sivaraman
  • Function : Author
Lokesh Bulchandani
  • Function : Author
Lu Liu
  • Function : Author
Luisa Shinzato
  • Function : Author
Madeleine Hahn de Bykhovetz
  • Function : Author
Maiko Takeuchi
  • Function : Author
Marc Pàmies
  • Function : Author
Maria A Castillo
  • Function : Author
Marianna Nezhurina
  • Function : Author
Mario Sänger
  • Function : Author
Matthias Samwald
  • Function : Author
Michael Cullan
  • Function : Author
Michael Weinberg
  • Function : Author
Michiel de Wolf
  • Function : Author
Mina Mihaljcic
  • Function : Author
Minna Liu
  • Function : Author
Moritz Freidank
  • Function : Author
Myungsun Kang
  • Function : Author
Natasha Seelam
  • Function : Author
Nathan Dahlberg
  • Function : Author
Nicholas Michio Broad
  • Function : Author
Nikolaus Muellner
  • Function : Author
Pascale Fung
  • Function : Author
Patrick Haller
  • Function : Author
Ramya Chandrasekhar
  • Function : Author
Renata Eisenberg
  • Function : Author
Robert Martin
  • Function : Author
Rodrigo Canalli
  • Function : Author
Rosaline Su
  • Function : Author
Ruisi Su
  • Function : Author
Samuel Cahyawijaya
  • Function : Author
Samuele Garda
  • Function : Author
Shlok S Deshmukh
  • Function : Author
Shubhanshu Mishra
  • Function : Author
Sid Kiblawi
  • Function : Author
Simon Ott
  • Function : Author
Sinee Sang-Aroonsiri
  • Function : Author
Srishti Kumar
  • Function : Author
Stefan Schweter
  • Function : Author
Sushil Bharati
  • Function : Author
Tanmay Laud
  • Function : Author
Théo Gigant
  • Function : Author
Tomoya Kainuma
  • Function : Author
Wojciech Kusa
  • Function : Author
Yash Shailesh Bajaj
  • Function : Author
Yash Venkatraman
  • Function : Author
Yifan Xu
  • Function : Author
Yingxin Xu
  • Function : Author
Yu Xu
  • Function : Author
Zhe Tan
  • Function : Author
Zhongli Xie
  • Function : Author
Zifan Ye
  • Function : Author
Mathilde Bras
  • Function : Author
Younes Belkada
  • Function : Author
Thomas Wolf
  • Function : Author

Abstract

Large language models (LLMs) have been shown to be able to perform new tasks based on a few demonstrations or natural language instructions. While these capabilities have led to widespread adoption, most LLMs are developed by resource-rich organizations and are frequently kept from the public. As a step towards democratizing this powerful technology, we present BLOOM, a 176B-parameter open-access language model designed and built thanks to a collaboration of hundreds of researchers. BLOOM is a decoder-only Transformer language model that was trained on the ROOTS corpus, a dataset comprising hundreds of sources in 46 natural and 13 programming languages (59 in total). We find that BLOOM achieves competitive performance on a wide variety of benchmarks, with stronger results after undergoing multitask prompted finetuning. To facilitate future research and applications using LLMs, we publicly release our models and code under the Responsible AI License.
Fichier principal
Vignette du fichier
2211.05100.pdf (2.89 Mo) Télécharger le fichier
Origin Files produced by the author(s)

Dates and versions

hal-03850124 , version 1 (20-11-2023)

Identifiers

Cite

Teven Le Scao, Angela Fan, Christopher Akiki, Ellie Pavlick, Suzana Ilić, et al.. BLOOM: A 176B-Parameter Open-Access Multilingual Language Model. 2023. ⟨hal-03850124⟩
4968 View
4055 Download

Altmetric

Share

More