Introducing CAD: the Contextual Abuse Dataset

Bertie Vidgen; Dong Nguyen; Helen Margetts; Patricia Rossini; Rebekah Tromble

doi:10.18653/v1/2021.naacl-main.182

Introducing CAD: the Contextual Abuse Dataset

Bertie Vidgen, Dong Nguyen, Helen Margetts, Patricia Rossini, Rebekah Tromble

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › Academic › peer-review

Abstract

Online abuse can inflict harm on users and communities, making online spaces unsafe and toxic. Progress in automatically detecting and classifying abusive content is often held back by the lack of high quality and detailed datasets.We introduce a new dataset of primarily English Reddit entries which addresses several limitations of prior work. It (1) contains six conceptually distinct primary categories as well as secondary categories, (2) has labels annotated in the context of the conversation thread, (3) contains rationales and (4) uses an expert-driven group-adjudication process for high quality annotations. We report several baseline models to benchmark the work of future researchers. The annotated dataset, annotation guidelines, models and code are freely available.

Original language	English
Title of host publication	Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies
Editors	Kristina Toutanova, Anna Rumshisky, Luke Zettlemoyer, Dilek Hakkani-Tur, Iz Beltagy, Steven Bethard, Ryan Cotterell, Tanmoy Chakraborty, Yichao Zhou
Publisher	Association for Computational Linguistics
Pages	2289-2303
Number of pages	15
DOIs	https://doi.org/10.18653/v1/2021.naacl-main.182
Publication status	Published - Jun 2021

Access to Document

10.18653/v1/2021.naacl-main.182Licence: CC BY

2021.naacl-main.182Final published version, 311 KBLicence: CC BY

Cite this

Vidgen, B., Nguyen, D., Margetts, H., Rossini, P., & Tromble, R. (2021). Introducing CAD: the Contextual Abuse Dataset. In K. Toutanova, A. Rumshisky, L. Zettlemoyer, D. Hakkani-Tur, I. Beltagy, S. Bethard, R. Cotterell, T. Chakraborty, & Y. Zhou (Eds.), Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (pp. 2289-2303). Association for Computational Linguistics. https://doi.org/10.18653/v1/2021.naacl-main.182

Vidgen, Bertie ; Nguyen, Dong ; Margetts, Helen et al. / Introducing CAD: the Contextual Abuse Dataset. Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. editor / Kristina Toutanova ; Anna Rumshisky ; Luke Zettlemoyer ; Dilek Hakkani-Tur ; Iz Beltagy ; Steven Bethard ; Ryan Cotterell ; Tanmoy Chakraborty ; Yichao Zhou. Association for Computational Linguistics, 2021. pp. 2289-2303

@inproceedings{7aa75d0269a4485bb186ea93171213e3,

title = "Introducing CAD: the Contextual Abuse Dataset",

abstract = "Online abuse can inflict harm on users and communities, making online spaces unsafe and toxic. Progress in automatically detecting and classifying abusive content is often held back by the lack of high quality and detailed datasets.We introduce a new dataset of primarily English Reddit entries which addresses several limitations of prior work. It (1) contains six conceptually distinct primary categories as well as secondary categories, (2) has labels annotated in the context of the conversation thread, (3) contains rationales and (4) uses an expert-driven group-adjudication process for high quality annotations. We report several baseline models to benchmark the work of future researchers. The annotated dataset, annotation guidelines, models and code are freely available.",

author = "Bertie Vidgen and Dong Nguyen and Helen Margetts and Patricia Rossini and Rebekah Tromble",

year = "2021",

month = jun,

doi = "10.18653/v1/2021.naacl-main.182",

language = "English",

pages = "2289--2303",

editor = "Kristina Toutanova and Anna Rumshisky and Luke Zettlemoyer and Dilek Hakkani-Tur and Iz Beltagy and Steven Bethard and Ryan Cotterell and Tanmoy Chakraborty and Yichao Zhou",

booktitle = "Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies",

publisher = "Association for Computational Linguistics",

address = "United States",

}

Vidgen, B, Nguyen, D, Margetts, H, Rossini, P & Tromble, R 2021, Introducing CAD: the Contextual Abuse Dataset. in K Toutanova, A Rumshisky, L Zettlemoyer, D Hakkani-Tur, I Beltagy, S Bethard, R Cotterell, T Chakraborty & Y Zhou (eds), Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. Association for Computational Linguistics, pp. 2289-2303. https://doi.org/10.18653/v1/2021.naacl-main.182

Introducing CAD: the Contextual Abuse Dataset. / Vidgen, Bertie; Nguyen, Dong; Margetts, Helen et al.
Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. ed. / Kristina Toutanova; Anna Rumshisky; Luke Zettlemoyer; Dilek Hakkani-Tur; Iz Beltagy; Steven Bethard; Ryan Cotterell; Tanmoy Chakraborty; Yichao Zhou. Association for Computational Linguistics, 2021. p. 2289-2303.

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › Academic › peer-review

TY - GEN

T1 - Introducing CAD: the Contextual Abuse Dataset

AU - Vidgen, Bertie

AU - Nguyen, Dong

AU - Margetts, Helen

AU - Rossini, Patricia

AU - Tromble, Rebekah

PY - 2021/6

Y1 - 2021/6

N2 - Online abuse can inflict harm on users and communities, making online spaces unsafe and toxic. Progress in automatically detecting and classifying abusive content is often held back by the lack of high quality and detailed datasets.We introduce a new dataset of primarily English Reddit entries which addresses several limitations of prior work. It (1) contains six conceptually distinct primary categories as well as secondary categories, (2) has labels annotated in the context of the conversation thread, (3) contains rationales and (4) uses an expert-driven group-adjudication process for high quality annotations. We report several baseline models to benchmark the work of future researchers. The annotated dataset, annotation guidelines, models and code are freely available.

AB - Online abuse can inflict harm on users and communities, making online spaces unsafe and toxic. Progress in automatically detecting and classifying abusive content is often held back by the lack of high quality and detailed datasets.We introduce a new dataset of primarily English Reddit entries which addresses several limitations of prior work. It (1) contains six conceptually distinct primary categories as well as secondary categories, (2) has labels annotated in the context of the conversation thread, (3) contains rationales and (4) uses an expert-driven group-adjudication process for high quality annotations. We report several baseline models to benchmark the work of future researchers. The annotated dataset, annotation guidelines, models and code are freely available.

U2 - 10.18653/v1/2021.naacl-main.182

DO - 10.18653/v1/2021.naacl-main.182

M3 - Conference contribution

SP - 2289

EP - 2303

BT - Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies

A2 - Toutanova, Kristina

A2 - Rumshisky, Anna

A2 - Zettlemoyer, Luke

A2 - Hakkani-Tur, Dilek

A2 - Beltagy, Iz

A2 - Bethard, Steven

A2 - Cotterell, Ryan

A2 - Chakraborty, Tanmoy

A2 - Zhou, Yichao

PB - Association for Computational Linguistics

ER -

Vidgen B, Nguyen D, Margetts H, Rossini P, Tromble R. Introducing CAD: the Contextual Abuse Dataset. In Toutanova K, Rumshisky A, Zettlemoyer L, Hakkani-Tur D, Beltagy I, Bethard S, Cotterell R, Chakraborty T, Zhou Y, editors, Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. Association for Computational Linguistics. 2021. p. 2289-2303 doi: 10.18653/v1/2021.naacl-main.182

Introducing CAD: the Contextual Abuse Dataset

Abstract

Access to Document

Fingerprint

Cite this