A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2022; you can also visit the original URL.
The file type is application/pdf.
Filters
French CrowS-Pairs: Extending a challenge dataset for measuring social bias in masked language models to a language other than English
2022
Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)
unpublished
Preserved Fulltext
We seek to widen the scope of bias studies by creating material to measure social bias in language models (LMs) against specific demographic groups in France. ...
We build on the US-centered CrowS-pairs dataset to create a multilingual stereotypes dataset that allows for comparability across languages while also characterizing biases that are specific to each country ...
We thank James Fiumara and Christopher Cieri for their guidance in the use of the Language ARC platform. ...
doi:10.18653/v1/2022.acl-long.583
fatcat:i3fyb46ojbh3pc4iwgqkun6slu
Investigating Bias in Multilingual Language Models: Cross-Lingual Transfer of Debiasing Techniques
[article]
2023
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2023; you can also visit the original URL.
The file type is application/pdf.
These novel insights contribute to a deeper understanding of bias mitigation in multilingual language models and provide practical guidance for debiasing techniques in different language contexts. ...
Using translations of the CrowS-Pairs dataset, our analysis identifies SentenceDebias as the best technique across different languages, reducing bias in mBERT by an average of 13%. ...
The resources and services used in this work were provided by the VSC (Flemish Supercomputer Center). ...
arXiv:2310.10310v1
fatcat:pc5eemthpfdc3jrwcen5s7jkj4
BLOOM: A 176B-Parameter Open-Access Multilingual Language Model
[article]
2023
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2023; you can also visit the original URL.
The file type is application/pdf.
Other Versions
BLOOM is a decoder-only Transformer language model that was trained on the ROOTS corpus, a dataset comprising hundreds of sources in 46 natural and 13 programming languages (59 in total). ...
Large language models (LLMs) have been shown to be able to perform new tasks based on a few demonstrations or natural language instructions. ...
Model training ran on the Jean-Zay supercomputer of GENCI at IDRIS, and we thank the IDRIS team for their responsive support throughout the project, in particular Rémi Lacroix. ...
arXiv:2211.05100v3
fatcat:3q5i7btwyzd7pdqlkb5zf4f3fu
Gender Bias in Masked Language Models for Multiple Languages
[article]
2022
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2022; you can also visit the original URL.
The file type is application/pdf.
Other Versions
Manual annotation of evaluation data for languages other than English has been challenging due to the cost and difficulty in recruiting annotators. ...
Masked Language Models (MLMs) pre-trained by predicting masked tokens on large corpora have been used successfully in natural language processing tasks for a variety of languages. ...
Acknowledgements This paper is based on results obtained from a project, JPNP18002, commissioned by the New Energy and Industrial Technology Development Organization (NEDO). ...
arXiv:2205.00551v3
fatcat:pr3hx54aszfxjg3t32qfelh5gy
SeeGULL Multilingual: a Dataset of Geo-Culturally Situated Stereotypes
[article]
2024
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2024; you can also visit the original URL.
The file type is application/pdf.
multilingual dataset of social stereotypes, containing over 25K stereotypes, spanning 20 languages, with human annotations across 23 regions, and demonstrate its utility in identifying gaps in model evaluations ...
While generative multilingual models are rapidly being deployed, their safety and fairness evaluations are largely limited to resources collected in English. ...
Crows-pairs: A challenge dataset for measuring social biases in masked lan-guage models. In Proceedings of the 2020 Confer- Dirk Hovy and Diyi Yang. 2021. ...
arXiv:2403.05696v1
fatcat:ckcsjgpypbdw3b4o5sntiikqn4
Fairness in Language Models Beyond English: Gaps and Challenges
[article]
2023
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2023; you can also visit the original URL.
The file type is application/pdf.
Other Versions
This paper presents a survey of fairness in multilingual and non-English contexts, highlighting the shortcomings of current research and the difficulties faced by methods designed for English. ...
Thus, the measurement and mitigation of biases must evolve beyond the current dataset-driven practices that are narrowly focused on specific dimensions and types of biases and, therefore, impossible to ...
Kaneko et al. (2022) measures gender bias in masked language models and proposes a method to use parallel corpora to evaluate bias in languages shown to have high correlations with human bias annotations ...
arXiv:2302.12578v2
fatcat:l6vll6d2xjcndd2nh4mfgaghee
Bias and Fairness in Large Language Models: A Survey
[article]
2024
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2024; you can also visit the original URL.
The file type is application/pdf.
Other Versions
Despite this success, these models can learn, perpetuate, and amplify harmful social biases. In this paper, we present a comprehensive survey of bias evaluation and mitigation techniques for LLMs. ...
Our first taxonomy of metrics for bias evaluation disambiguates the relationship between metrics and evaluation datasets, and organizes metrics by the different levels at which they operate in a model: ...
than English. ...
arXiv:2309.00770v2
fatcat:idqaltzjdndnhnwcgd7dsqcejm
Few-shot Learning with Multilingual Language Models
[article]
2022
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2022; you can also visit the original URL.
The file type is application/pdf.
Other Versions
Finally, we evaluate our models in social value tasks such as hate speech detection in five languages and find it has limitations similar to comparable sized GPT-3 models. ...
While these models are known to be able to jointly represent many different languages, their training data is dominated by English, potentially limiting their cross-lingual generalization. ...
Crows-pairs: A chal-Findings of the Association for Computational Lin-lenge dataset for measuring social biases in masked guistics: ACL/IJCNLP 2021, Online Event, August language models. arXiv preprint ...
arXiv:2112.10668v3
fatcat:lllzazhlv5at3mqbcc43dboq3a
On Measures of Biases and Harms in NLP
[article]
2022
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2022; you can also visit the original URL.
The file type is application/pdf.
Other Versions
While existing works propose bias evaluation and mitigation methods for various tasks, there remains a need to cohesively understand the biases and the specific harms they measure, and how different measures ...
As a validation of our framework and documentation questions, we also present several case studies of how existing bias measures in NLP -- both intrinsic measures of bias in representations and extrinsic ...
to feasibly automatically label a large number of samples, there could also be biases from the classifier itself. ...
arXiv:2108.03362v2
fatcat:3e3ezwedyzc37hfm4nyblkbr7q
OPT: Open Pre-trained Transformer Language Models
[article]
2022
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2022; you can also visit the original URL.
The file type is application/pdf.
Other Versions
Large language models, which are often trained for hundreds of thousands of compute days, have shown remarkable capabilities for zero- and few-shot learning. ...
We are also releasing our logbook detailing the infrastructure challenges we faced, along with code for experimenting with all of the released models. ...
Acknowledgements We would like to thank Scott Jeschonek, Giri Anantharaman, Diego Sarina, Joaquin Colombo, Chris Bray, Stephen Roylance, Kalyan Saladi, Shubho Sengupta, and Brian O'Horo for helping to ...
arXiv:2205.01068v4
fatcat:arnzfthgmzfmzj5u6ut3bbct3q
Aya Model: An Instruction Finetuned Open-Access Multilingual Language Model
[article]
2024
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2024; you can also visit the original URL.
The file type is application/pdf.
Recent breakthroughs in large language models (LLMs) have centered around a handful of data-rich languages. What does it take to broaden access to breakthroughs beyond first-class citizen languages? ...
Our work introduces Aya, a massively multilingual generative language model that follows instructions in 101 languages of which over 50% are considered as lower-resourced. ...
We also thank the language experts who helped us understand the quality of model generations in their languages. We thank John Dang for helping to convert Aya T5x checkpoint to PyTorch. ...
arXiv:2402.07827v1
fatcat:dyu3fpzkgrfdjo54bu7yzx2bpm
LLaMA: Open and Efficient Foundation Language Models
[article]
2023
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2023; you can also visit the original URL.
The file type is application/pdf.
We introduce LLaMA, a collection of foundation language models ranging from 7B to 65B parameters. ...
We train our models on trillions of tokens, and show that it is possible to train state-of-the-art models using publicly available datasets exclusively, without resorting to proprietary and inaccessible ...
We thank Shubho Sengupta, Kalyan Saladi, and all the AI infra team for their support. We thank Jane Yu for her input on evaluation. We thank Yongyi Hu for his help on data collection. ...
arXiv:2302.13971v1
fatcat:ug5hspba4jggrospmn45jk6qpy
Multitask Prompted Training Enables Zero-Shot Task Generalization
[article]
2022
arXiv
pre-print
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2022; you can also visit the original URL.
The file type is application/pdf.
Other Versions
These prompted datasets allow for benchmarking the ability of a model to perform completely held-out tasks. ...
To test this question at scale, we develop a system for easily mapping any natural language tasks into a human-readable prompted form. ...
The goal of the project is to research language models in a public environment outside large technology companies. The project has 600 researchers from 50 countries and more than 250 institutions. ...
arXiv:2110.08207v3
fatcat:vvacmc2phfg7dpmdqiebyvxvei
Investigating Bias in Multilingual Language Models: Cross-Lingual Transfer of Debiasing Techniques
2023
Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing
unpublished
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2023; you can also visit the original URL.
The file type is application/pdf.
These novel insights contribute to a deeper understanding of bias mitigation in multilingual language models and provide practical guidance for debiasing techniques in different language contexts. ...
Using translations of the CrowS-Pairs dataset, our analysis identifies SentenceDebias as the best technique across different languages, reducing bias in mBERT by an average of 13%. ...
The resources and services used in this work were provided by the VSC (Flemish Supercomputer Center). ...
doi:10.18653/v1/2023.emnlp-main.175
fatcat:dpa2u65tazeyfmfblfmu6bvvt4
Measuring Fairness with Biased Rulers: A Comparative Study on Bias Metrics for Pre-trained Language Models
2022
Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies
unpublished
Preserved Fulltext
A copy of this work was available on the public web and has been preserved in the Wayback Machine. The capture dates from 2022; you can also visit the original URL.
The file type is application/pdf.
These results indicate that fairness or bias evaluation remains challenging for contextualized language models, among other reasons because these choices remain subjective. ...
We survey the literature on fairness metrics for pre-trained language models and experimentally evaluate compatibility, including both biases in language models and in their downstream tasks. ...
Acknowledgements We thank Luc De Raedt for his continued support, Jessa Bekker for her practical advice on writing a survey, and Eva Vanmassenhove for sharing her knowledge on gender bias. ...
doi:10.18653/v1/2022.naacl-main.122
fatcat:d6gqg2ozircbbagywexckkrn3u
« Previous
Showing results 1 — 15 out of 19 results