| 1 |
Sociolinguistically Driven Approaches for Just Natural Language Processing ...
|
|
|
|
BASE
|
|
Show details
|
|
|
| 3 |
Stereotyping Norwegian Salmon: An Inventory of Pitfalls in Fairness Benchmark Datasets ...
|
|
|
|
Abstract:
Read paper: https://www.aclanthology.org/2021.acl-long.81 Abstract: Auditing NLP systems for computational harms like surfacing stereotypes is an elusive goal. Several recent efforts have focused on benchmark datasets consisting of pairs of contrastive sentences, which are often accompanied by metrics that aggregate an NLP system's behavior on these pairs into measurements of harms. We examine four such benchmarks constructed for two NLP tasks: language modeling and coreference resolution. We apply a measurement modeling lens---originating from the social sciences---to inventory a range of pitfalls that threaten these benchmarks' validity as measurement models for stereotyping. We find that these benchmarks frequently lack clear articulations of what is being measured, and we highlight a range of ambiguities and unstated assumptions that affect how these benchmarks conceptualize and operationalize stereotyping. ...
|
|
URL: https://dx.doi.org/10.48448/8z6w-es68
https://underline.io/lecture/25963-stereotyping-norwegian-salmon-an-inventory-of-pitfalls-in-fairness-benchmark-datasets
|
|
BASE
|
|
Hide details
|
|
|
| 5 |
Sociolinguistically Driven Approaches for Just Natural Language Processing
|
|
|
|
In: Doctoral Dissertations (2021)
|
|
BASE
|
|
Show details
|
|
|
| 6 |
Demographic Dialectal Variation in Social Media: A Case Study of African-American English ...
|
|
|
|
BASE
|
|
Show details
|
|
|
|
|
