Is there less agreement when the discourse is underspecified? Annotation of coherence relations in TED talks

Research output: Chapter in Book/Report/Conference proceedingConference contributionAcademicpeer-review

Abstract

When annotating coherence relations, interannotator agreement tends to be lower on implicit relations than on relations that are explicitly marked by means of a connective or
a cue phrase. This paper explores one possible explanation for this: the additional inferencing involved in interpreting implicit relations compared to explicit relations. If this is
the main source of disagreements, agreement
should be highly related to the specificity of
the connective. Using the CCR framework, we
annotated relations from TED talks that were
marked by a very specific marker, marked by
a highly ambiguous connective, or not marked
by means of a connective at all. We indeed
reached higher inter-annotator agreement on
explicit than on implicit relations. However,
agreement on underspecified relations was not
necessarily in between, which is what would
be expected if agreement on implicit relations
mainly suffers because annotators have less
specific instructions for inferring the relation.
Original languageEnglish
Title of host publicationProceedings of the First Workshop on Integrating Perspectives on Discourse Annotation
EditorsKordula De Kuthy, Detmar Meurers
PublisherAssociation for Computational Linguistics (ACL)
Pages1–6
Number of pages6
Publication statusPublished - 2021

Fingerprint

Dive into the research topics of 'Is there less agreement when the discourse is underspecified? Annotation of coherence relations in TED talks'. Together they form a unique fingerprint.

Cite this