On Learning Interpreted Languages with Recurrent Models

Denis Paperno

doi:10.1162/coli_a_00431

On Learning Interpreted Languages with Recurrent Models

Research output: Contribution to journal › Article › Academic › peer-review

Abstract

Can recurrent neural nets, inspired by human sequential data processing, learn to understand language? We construct simplified data sets reflecting core properties of natural language as modeled in formal syntax and semantics: recursive syntactic structure and compositionality. We find LSTM and GRU networks to generalize to compositional interpretation well, but only in the most favorable learning settings, with a well-paced curriculum, extensive training data, and left-to-right (but not right-to-left) composition.

Original language	English
Pages (from-to)	471-483
Number of pages	13
Journal	Computational Linguistics
Volume	48
Issue number	2
DOIs	https://doi.org/10.1162/coli_a_00431
Publication status	Published - Jun 2022

Access to Document

10.1162/coli_a_00431Licence: CC BY-NC-ND

coli_a_00431Final published version, 195 KBLicence: CC BY-NC-ND

Cite this

@article{1b14226e65ed4e718249850626c0140f,

title = "On Learning Interpreted Languages with Recurrent Models",

abstract = "Can recurrent neural nets, inspired by human sequential data processing, learn to understand language? We construct simplified data sets reflecting core properties of natural language as modeled in formal syntax and semantics: recursive syntactic structure and compositionality. We find LSTM and GRU networks to generalize to compositional interpretation well, but only in the most favorable learning settings, with a well-paced curriculum, extensive training data, and left-to-right (but not right-to-left) composition.",

author = "Denis Paperno",

year = "2022",

month = jun,

doi = "10.1162/coli_a_00431",

language = "English",

volume = "48",

pages = "471--483",

journal = "Computational Linguistics",

issn = "0891-2017",

publisher = "MIT Press Journals",

number = "2",

}

TY - JOUR

T1 - On Learning Interpreted Languages with Recurrent Models

AU - Paperno, Denis

PY - 2022/6

Y1 - 2022/6

N2 - Can recurrent neural nets, inspired by human sequential data processing, learn to understand language? We construct simplified data sets reflecting core properties of natural language as modeled in formal syntax and semantics: recursive syntactic structure and compositionality. We find LSTM and GRU networks to generalize to compositional interpretation well, but only in the most favorable learning settings, with a well-paced curriculum, extensive training data, and left-to-right (but not right-to-left) composition.

AB - Can recurrent neural nets, inspired by human sequential data processing, learn to understand language? We construct simplified data sets reflecting core properties of natural language as modeled in formal syntax and semantics: recursive syntactic structure and compositionality. We find LSTM and GRU networks to generalize to compositional interpretation well, but only in the most favorable learning settings, with a well-paced curriculum, extensive training data, and left-to-right (but not right-to-left) composition.

U2 - 10.1162/coli_a_00431

DO - 10.1162/coli_a_00431

M3 - Article

SN - 0891-2017

VL - 48

SP - 471

EP - 483

JO - Computational Linguistics

JF - Computational Linguistics

IS - 2

ER -

On Learning Interpreted Languages with Recurrent Models

Abstract

Access to Document

Fingerprint

Cite this