Constructive Type-Logical Supertagging with Self-Attention Networks

K. Kogkalidis; M.J. Moortgat; T. Deoskar

Constructive Type-Logical Supertagging with Self-Attention Networks

K. Kogkalidis
, M.J. Moortgat
, T. Deoskar

Research output: Contribution to conference › Paper › Academic

Abstract

We propose a novel application of self-attention networks towards grammar induction. We present an attention-based supertagger for a refined type-logical grammar, trained on constructing types inductively. In addition to achieving a high overall type accuracy, our model is able to learn the syntax of the grammar's type system along with its denotational semantics. This lifts the closed world assumption commonly made by lexicalized grammar supertaggers, greatly enhancing its generalization potential. This is evidenced both by its adequate accuracy over sparse word types and its ability to correctly construct complex types never seen during training, which, to the best of our knowledge, was as of yet unaccomplished.

Original language	English
Pages	113-123
Publication status	Published - 24 May 2019
Event	Representation Learning For NLP: ACL Workshop - Florence, Italy Duration: 2 Aug 2019 → 2 Aug 2019 Conference number: 4 https://sites.google.com/view/repl4nlp2019

Workshop

Workshop	Representation Learning For NLP
Abbreviated title	REPL4NLP
Country/Territory	Italy
City	Florence
Period	2/08/19 → 2/08/19
Internet address	https://sites.google.com/view/repl4nlp2019

Bibliographical note

Proceedings of the 4th Workshop on Representation Learning for NLP (RepL4NLP-2019)

Access to Document

W19-4314Final published version, 320 KB

9 Citations
1 Paper

Neural Proof Nets
Kogkalidis, K., Moortgat, M. J. & Moot, R. C. A., 2020, p. 26–40.
Research output: Contribution to conference › Paper › Academic

Open Access
File

Cite this

@conference{416ab49f1bdd4e2ca8b2ded9ccd03ec9,

title = "Constructive Type-Logical Supertagging with Self-Attention Networks",

abstract = "We propose a novel application of self-attention networks towards grammar induction. We present an attention-based supertagger for a refined type-logical grammar, trained on constructing types inductively. In addition to achieving a high overall type accuracy, our model is able to learn the syntax of the grammar's type system along with its denotational semantics. This lifts the closed world assumption commonly made by lexicalized grammar supertaggers, greatly enhancing its generalization potential. This is evidenced both by its adequate accuracy over sparse word types and its ability to correctly construct complex types never seen during training, which, to the best of our knowledge, was as of yet unaccomplished. ",

author = "K. Kogkalidis and M.J. Moortgat and T. Deoskar",

note = "Proceedings of the 4th Workshop on Representation Learning for NLP (RepL4NLP-2019); Representation Learning For NLP : ACL Workshop, REPL4NLP ; Conference date: 02-08-2019 Through 02-08-2019",

year = "2019",

month = may,

day = "24",

language = "English",

pages = "113--123",

url = "https://sites.google.com/view/repl4nlp2019",

}

TY - CONF

T1 - Constructive Type-Logical Supertagging with Self-Attention Networks

AU - Kogkalidis, K.

AU - Moortgat, M.J.

AU - Deoskar, T.

N1 - Conference code: 4

PY - 2019/5/24

Y1 - 2019/5/24

N2 - We propose a novel application of self-attention networks towards grammar induction. We present an attention-based supertagger for a refined type-logical grammar, trained on constructing types inductively. In addition to achieving a high overall type accuracy, our model is able to learn the syntax of the grammar's type system along with its denotational semantics. This lifts the closed world assumption commonly made by lexicalized grammar supertaggers, greatly enhancing its generalization potential. This is evidenced both by its adequate accuracy over sparse word types and its ability to correctly construct complex types never seen during training, which, to the best of our knowledge, was as of yet unaccomplished.

AB - We propose a novel application of self-attention networks towards grammar induction. We present an attention-based supertagger for a refined type-logical grammar, trained on constructing types inductively. In addition to achieving a high overall type accuracy, our model is able to learn the syntax of the grammar's type system along with its denotational semantics. This lifts the closed world assumption commonly made by lexicalized grammar supertaggers, greatly enhancing its generalization potential. This is evidenced both by its adequate accuracy over sparse word types and its ability to correctly construct complex types never seen during training, which, to the best of our knowledge, was as of yet unaccomplished.

M3 - Paper

SP - 113

EP - 123

T2 - Representation Learning For NLP

Y2 - 2 August 2019 through 2 August 2019

ER -

Constructive Type-Logical Supertagging with Self-Attention Networks

Abstract

Workshop

Bibliographical note

Access to Document

Fingerprint

Research output

Neural Proof Nets

Cite this