Highly Efficient Knowledge Graph Embedding Learning with Orthogonal Procrustes Analysis

Xutan Peng; Guanyi Chen; Chenghua Lin; Mark Stevenson

doi:10.18653/v1/2021.naacl-main.187

Highly Efficient Knowledge Graph Embedding Learning with Orthogonal Procrustes Analysis

Xutan Peng, Guanyi Chen, Chenghua Lin, Mark Stevenson

University of Sheffield

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › Academic › peer-review

Abstract

Knowledge Graph Embeddings (KGEs) have been intensively explored in recent years due to their promise for a wide range of applications. However, existing studies focus on improving the final model performance without acknowledging the computational cost of the proposed approaches, in terms of execution time and environmental impact. This paper proposes a simple yet effective KGE framework which can reduce the training time and carbon footprint by orders of magnitudes compared with state-of-the-art approaches, while producing competitive performance. We highlight three technical innovations: full batch learning via relational matrices, closed-form Orthogonal Procrustes Analysis for KGEs, and non-negative-sampling training. In addition, as the first KGE method whose entity embeddings also store full relation information, our trained models encode rich semantics and are highly interpretable. Comprehensive experiments and ablation studies involving 13 strong baselines and two standard datasets verify the effectiveness and efficiency of our algorithm.

Original language	English
Title of host publication	Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies
Editors	Kristina Toutanova, Anna Rumshisky, Luke Zettlemoyer, Dilek Hakkani-Tur, Iz Beltagy, Steven Bethard, Ryan Cotterell, Tanmoy Chakraborty, Yichao Zhou
Place of Publication	Online
Publisher	Association for Computational Linguistics
Pages	2364-2375
Number of pages	12
DOIs	https://doi.org/10.18653/v1/2021.naacl-main.187
Publication status	Published - 1 Jun 2021

Bibliographical note

DBLP License: DBLP's bibliographic metadata records provided through http://dblp.org/ are distributed under a Creative Commons CC0 1.0 Universal Public Domain Dedication. Although the bibliographic metadata records are provided consistent with CC0 1.0 Dedication, the content described by the metadata records is not. Content may be subject to copyright, rights of privacy, rights of publicity and other restrictions.

Access to Document

10.18653/v1/2021.naacl-main.187Licence: CC BY

2021.naacl-main.187Final published version, 2.62 MBLicence: CC BY

https://aclanthology.org/2021.naacl-main.187

Cite this

Peng, X., Chen, G., Lin, C., & Stevenson, M. (2021). Highly Efficient Knowledge Graph Embedding Learning with Orthogonal Procrustes Analysis. In K. Toutanova, A. Rumshisky, L. Zettlemoyer, D. Hakkani-Tur, I. Beltagy, S. Bethard, R. Cotterell, T. Chakraborty, & Y. Zhou (Eds.), Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (pp. 2364-2375). Association for Computational Linguistics. https://doi.org/10.18653/v1/2021.naacl-main.187

Peng, Xutan ; Chen, Guanyi ; Lin, Chenghua et al. / Highly Efficient Knowledge Graph Embedding Learning with Orthogonal Procrustes Analysis. Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. editor / Kristina Toutanova ; Anna Rumshisky ; Luke Zettlemoyer ; Dilek Hakkani-Tur ; Iz Beltagy ; Steven Bethard ; Ryan Cotterell ; Tanmoy Chakraborty ; Yichao Zhou. Online : Association for Computational Linguistics, 2021. pp. 2364-2375

@inproceedings{5390f85c957546f8b29ea9fef12084c2,

title = "Highly Efficient Knowledge Graph Embedding Learning with Orthogonal Procrustes Analysis",

abstract = "Knowledge Graph Embeddings (KGEs) have been intensively explored in recent years due to their promise for a wide range of applications. However, existing studies focus on improving the final model performance without acknowledging the computational cost of the proposed approaches, in terms of execution time and environmental impact. This paper proposes a simple yet effective KGE framework which can reduce the training time and carbon footprint by orders of magnitudes compared with state-of-the-art approaches, while producing competitive performance. We highlight three technical innovations: full batch learning via relational matrices, closed-form Orthogonal Procrustes Analysis for KGEs, and non-negative-sampling training. In addition, as the first KGE method whose entity embeddings also store full relation information, our trained models encode rich semantics and are highly interpretable. Comprehensive experiments and ablation studies involving 13 strong baselines and two standard datasets verify the effectiveness and efficiency of our algorithm.",

author = "Xutan Peng and Guanyi Chen and Chenghua Lin and Mark Stevenson",

note = "DBLP License: DBLP's bibliographic metadata records provided through http://dblp.org/ are distributed under a Creative Commons CC0 1.0 Universal Public Domain Dedication. Although the bibliographic metadata records are provided consistent with CC0 1.0 Dedication, the content described by the metadata records is not. Content may be subject to copyright, rights of privacy, rights of publicity and other restrictions.",

year = "2021",

month = jun,

day = "1",

doi = "10.18653/v1/2021.naacl-main.187",

language = "English",

pages = "2364--2375",

editor = "Kristina Toutanova and Anna Rumshisky and Luke Zettlemoyer and Dilek Hakkani-Tur and Iz Beltagy and Steven Bethard and Ryan Cotterell and Tanmoy Chakraborty and Yichao Zhou",

booktitle = "Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies",

publisher = "Association for Computational Linguistics",

address = "United States",

}

Peng, X, Chen, G, Lin, C & Stevenson, M 2021, Highly Efficient Knowledge Graph Embedding Learning with Orthogonal Procrustes Analysis. in K Toutanova, A Rumshisky, L Zettlemoyer, D Hakkani-Tur, I Beltagy, S Bethard, R Cotterell, T Chakraborty & Y Zhou (eds), Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. Association for Computational Linguistics, Online, pp. 2364-2375. https://doi.org/10.18653/v1/2021.naacl-main.187

Highly Efficient Knowledge Graph Embedding Learning with Orthogonal Procrustes Analysis. / Peng, Xutan; Chen, Guanyi; Lin, Chenghua et al.
Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. ed. / Kristina Toutanova; Anna Rumshisky; Luke Zettlemoyer; Dilek Hakkani-Tur; Iz Beltagy; Steven Bethard; Ryan Cotterell; Tanmoy Chakraborty; Yichao Zhou. Online: Association for Computational Linguistics, 2021. p. 2364-2375.

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › Academic › peer-review

TY - GEN

T1 - Highly Efficient Knowledge Graph Embedding Learning with Orthogonal Procrustes Analysis

AU - Peng, Xutan

AU - Chen, Guanyi

AU - Lin, Chenghua

AU - Stevenson, Mark

N1 - DBLP License: DBLP's bibliographic metadata records provided through http://dblp.org/ are distributed under a Creative Commons CC0 1.0 Universal Public Domain Dedication. Although the bibliographic metadata records are provided consistent with CC0 1.0 Dedication, the content described by the metadata records is not. Content may be subject to copyright, rights of privacy, rights of publicity and other restrictions.

PY - 2021/6/1

Y1 - 2021/6/1

N2 - Knowledge Graph Embeddings (KGEs) have been intensively explored in recent years due to their promise for a wide range of applications. However, existing studies focus on improving the final model performance without acknowledging the computational cost of the proposed approaches, in terms of execution time and environmental impact. This paper proposes a simple yet effective KGE framework which can reduce the training time and carbon footprint by orders of magnitudes compared with state-of-the-art approaches, while producing competitive performance. We highlight three technical innovations: full batch learning via relational matrices, closed-form Orthogonal Procrustes Analysis for KGEs, and non-negative-sampling training. In addition, as the first KGE method whose entity embeddings also store full relation information, our trained models encode rich semantics and are highly interpretable. Comprehensive experiments and ablation studies involving 13 strong baselines and two standard datasets verify the effectiveness and efficiency of our algorithm.

AB - Knowledge Graph Embeddings (KGEs) have been intensively explored in recent years due to their promise for a wide range of applications. However, existing studies focus on improving the final model performance without acknowledging the computational cost of the proposed approaches, in terms of execution time and environmental impact. This paper proposes a simple yet effective KGE framework which can reduce the training time and carbon footprint by orders of magnitudes compared with state-of-the-art approaches, while producing competitive performance. We highlight three technical innovations: full batch learning via relational matrices, closed-form Orthogonal Procrustes Analysis for KGEs, and non-negative-sampling training. In addition, as the first KGE method whose entity embeddings also store full relation information, our trained models encode rich semantics and are highly interpretable. Comprehensive experiments and ablation studies involving 13 strong baselines and two standard datasets verify the effectiveness and efficiency of our algorithm.

U2 - 10.18653/v1/2021.naacl-main.187

DO - 10.18653/v1/2021.naacl-main.187

M3 - Conference contribution

SP - 2364

EP - 2375

BT - Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies

A2 - Toutanova, Kristina

A2 - Rumshisky, Anna

A2 - Zettlemoyer, Luke

A2 - Hakkani-Tur, Dilek

A2 - Beltagy, Iz

A2 - Bethard, Steven

A2 - Cotterell, Ryan

A2 - Chakraborty, Tanmoy

A2 - Zhou, Yichao

PB - Association for Computational Linguistics

CY - Online

ER -

Peng X, Chen G, Lin C, Stevenson M. Highly Efficient Knowledge Graph Embedding Learning with Orthogonal Procrustes Analysis. In Toutanova K, Rumshisky A, Zettlemoyer L, Hakkani-Tur D, Beltagy I, Bethard S, Cotterell R, Chakraborty T, Zhou Y, editors, Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. Online: Association for Computational Linguistics. 2021. p. 2364-2375 doi: 10.18653/v1/2021.naacl-main.187

Highly Efficient Knowledge Graph Embedding Learning with Orthogonal Procrustes Analysis

Abstract

Bibliographical note

Access to Document

Fingerprint

Cite this