Leveraging Pre-trained Language Models for Time Interval Prediction in Text-Enhanced Temporal Knowledge Graphs

Duygu Sezen Islakoglu; Melisachew Wudage Chekol; Yannis Velegrakis

doi:10.1007/978-3-031-60626-7_4

Leveraging Pre-trained Language Models for Time Interval Prediction in Text-Enhanced Temporal Knowledge Graphs

Duygu Sezen Islakoglu^*, Melisachew Wudage Chekol, Yannis Velegrakis

^*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › Academic › peer-review

Abstract

Most knowledge graph completion (KGC) methods rely solely on structural information, even though a large number of publicly available KGs contain additional temporal (validity time intervals) and textual data (entity descriptions). While recent temporal KGC methods utilize time information to enhance link prediction, they do not leverage textual descriptions or support inductive inference (prediction for entities that have not been seen during training).

In this work, we propose a novel framework called TEMT that exploits the power of pre-trained language models (PLMs) for temporal KGC. TEMT predicts time intervals of facts by fusing their textual and temporal information. It also supports inductive inference by utilizing PLMs. In order to showcase the power of TEMT, we carry out several experiments including time interval prediction, both in transductive and inductive settings, and triple classification. The experimental results demonstrate that TEMT is competitive with the state-of-the-art, while also supporting inductiveness.

Original language	English
Title of host publication	The Semantic Web - 21st International Conference, ESWC 2024, Proceedings
Subtitle of host publication	21st International Conference, ESWC 2024 Hersonissos, Crete, Greece, May 26–30, 2024 Proceedings, Part I
Editors	Albert Meroño Peñuela, Anastasia Dimou, Raphaël Troncy, Pasquale Lisena, Olaf Hartig, Maribel Acosta, Mehwish Alam, Heiko Paulheim
Publisher	Springer
Pages	59–78
Number of pages	20
ISBN (Electronic)	978-3-031-60626-7
ISBN (Print)	978-3-031-60625-0
DOIs	https://doi.org/10.1007/978-3-031-60626-7_4
Publication status	Published - 2024

Publication series

Name	Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume	14664 LNCS
ISSN (Print)	0302-9743
ISSN (Electronic)	1611-3349

Keywords

Knowledge Graph Completion
Pre-trained Language Models
Temporal Knowledge Graphs

Access to Document

10.1007/978-3-031-60626-7_4

978-3-031-60626-7_4Final published version, 547 KBLicence: Taverne

Cite this

Islakoglu, D. S., Chekol, M. W., & Velegrakis, Y. (2024). Leveraging Pre-trained Language Models for Time Interval Prediction in Text-Enhanced Temporal Knowledge Graphs. In A. Meroño Peñuela, A. Dimou, R. Troncy, P. Lisena, O. Hartig, M. Acosta, M. Alam, & H. Paulheim (Eds.), The Semantic Web - 21st International Conference, ESWC 2024, Proceedings: 21st International Conference, ESWC 2024 Hersonissos, Crete, Greece, May 26–30, 2024 Proceedings, Part I (pp. 59–78). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 14664 LNCS). Springer. https://doi.org/10.1007/978-3-031-60626-7_4

Islakoglu, Duygu Sezen ; Chekol, Melisachew Wudage ; Velegrakis, Yannis. / Leveraging Pre-trained Language Models for Time Interval Prediction in Text-Enhanced Temporal Knowledge Graphs. The Semantic Web - 21st International Conference, ESWC 2024, Proceedings: 21st International Conference, ESWC 2024 Hersonissos, Crete, Greece, May 26–30, 2024 Proceedings, Part I. editor / Albert Meroño Peñuela ; Anastasia Dimou ; Raphaël Troncy ; Pasquale Lisena ; Olaf Hartig ; Maribel Acosta ; Mehwish Alam ; Heiko Paulheim. Springer, 2024. pp. 59–78 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).

@inproceedings{349b6ec7bb6b45cd8b9326741cb57f5f,

title = "Leveraging Pre-trained Language Models for Time Interval Prediction in Text-Enhanced Temporal Knowledge Graphs",

abstract = "Most knowledge graph completion (KGC) methods rely solely on structural information, even though a large number of publicly available KGs contain additional temporal (validity time intervals) and textual data (entity descriptions). While recent temporal KGC methods utilize time information to enhance link prediction, they do not leverage textual descriptions or support inductive inference (prediction for entities that have not been seen during training).In this work, we propose a novel framework called TEMT that exploits the power of pre-trained language models (PLMs) for temporal KGC. TEMT predicts time intervals of facts by fusing their textual and temporal information. It also supports inductive inference by utilizing PLMs. In order to showcase the power of TEMT, we carry out several experiments including time interval prediction, both in transductive and inductive settings, and triple classification. The experimental results demonstrate that TEMT is competitive with the state-of-the-art, while also supporting inductiveness.",

keywords = "Knowledge Graph Completion, Pre-trained Language Models, Temporal Knowledge Graphs",

author = "Islakoglu, {Duygu Sezen} and Chekol, {Melisachew Wudage} and Yannis Velegrakis",

note = "Publisher Copyright: {\textcopyright} The Author(s), under exclusive license to Springer Nature Switzerland AG 2024.",

year = "2024",

doi = "10.1007/978-3-031-60626-7_4",

language = "English",

isbn = "978-3-031-60625-0",

series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",

publisher = "Springer",

pages = "59–78",

editor = "{Mero{\~n}o Pe{\~n}uela}, Albert and Anastasia Dimou and Rapha{\"e}l Troncy and Pasquale Lisena and Olaf Hartig and Maribel Acosta and Mehwish Alam and Heiko Paulheim",

booktitle = "The Semantic Web - 21st International Conference, ESWC 2024, Proceedings",

address = "Germany",

}

Islakoglu, DS , Chekol, MW & Velegrakis, Y 2024, Leveraging Pre-trained Language Models for Time Interval Prediction in Text-Enhanced Temporal Knowledge Graphs. in A Meroño Peñuela, A Dimou, R Troncy, P Lisena, O Hartig, M Acosta, M Alam & H Paulheim (eds), The Semantic Web - 21st International Conference, ESWC 2024, Proceedings: 21st International Conference, ESWC 2024 Hersonissos, Crete, Greece, May 26–30, 2024 Proceedings, Part I. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 14664 LNCS, Springer, pp. 59–78. https://doi.org/10.1007/978-3-031-60626-7_4

Leveraging Pre-trained Language Models for Time Interval Prediction in Text-Enhanced Temporal Knowledge Graphs. / Islakoglu, Duygu Sezen ; Chekol, Melisachew Wudage ; Velegrakis, Yannis.
The Semantic Web - 21st International Conference, ESWC 2024, Proceedings: 21st International Conference, ESWC 2024 Hersonissos, Crete, Greece, May 26–30, 2024 Proceedings, Part I. ed. / Albert Meroño Peñuela; Anastasia Dimou; Raphaël Troncy; Pasquale Lisena; Olaf Hartig; Maribel Acosta; Mehwish Alam; Heiko Paulheim. Springer, 2024. p. 59–78 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 14664 LNCS).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › Academic › peer-review

TY - GEN

T1 - Leveraging Pre-trained Language Models for Time Interval Prediction in Text-Enhanced Temporal Knowledge Graphs

AU - Islakoglu, Duygu Sezen

AU - Chekol, Melisachew Wudage

AU - Velegrakis, Yannis

N1 - Publisher Copyright: © The Author(s), under exclusive license to Springer Nature Switzerland AG 2024.

PY - 2024

Y1 - 2024

N2 - Most knowledge graph completion (KGC) methods rely solely on structural information, even though a large number of publicly available KGs contain additional temporal (validity time intervals) and textual data (entity descriptions). While recent temporal KGC methods utilize time information to enhance link prediction, they do not leverage textual descriptions or support inductive inference (prediction for entities that have not been seen during training).In this work, we propose a novel framework called TEMT that exploits the power of pre-trained language models (PLMs) for temporal KGC. TEMT predicts time intervals of facts by fusing their textual and temporal information. It also supports inductive inference by utilizing PLMs. In order to showcase the power of TEMT, we carry out several experiments including time interval prediction, both in transductive and inductive settings, and triple classification. The experimental results demonstrate that TEMT is competitive with the state-of-the-art, while also supporting inductiveness.

AB - Most knowledge graph completion (KGC) methods rely solely on structural information, even though a large number of publicly available KGs contain additional temporal (validity time intervals) and textual data (entity descriptions). While recent temporal KGC methods utilize time information to enhance link prediction, they do not leverage textual descriptions or support inductive inference (prediction for entities that have not been seen during training).In this work, we propose a novel framework called TEMT that exploits the power of pre-trained language models (PLMs) for temporal KGC. TEMT predicts time intervals of facts by fusing their textual and temporal information. It also supports inductive inference by utilizing PLMs. In order to showcase the power of TEMT, we carry out several experiments including time interval prediction, both in transductive and inductive settings, and triple classification. The experimental results demonstrate that TEMT is competitive with the state-of-the-art, while also supporting inductiveness.

KW - Knowledge Graph Completion

KW - Pre-trained Language Models

KW - Temporal Knowledge Graphs

UR - http://www.scopus.com/inward/record.url?scp=85194227035&partnerID=8YFLogxK

U2 - 10.1007/978-3-031-60626-7_4

DO - 10.1007/978-3-031-60626-7_4

M3 - Conference contribution

SN - 978-3-031-60625-0

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 59

EP - 78

BT - The Semantic Web - 21st International Conference, ESWC 2024, Proceedings

A2 - Meroño Peñuela, Albert

A2 - Dimou, Anastasia

A2 - Troncy, Raphaël

A2 - Lisena, Pasquale

A2 - Hartig, Olaf

A2 - Acosta, Maribel

A2 - Alam, Mehwish

A2 - Paulheim, Heiko

PB - Springer

ER -

Islakoglu DS , Chekol MW , Velegrakis Y. Leveraging Pre-trained Language Models for Time Interval Prediction in Text-Enhanced Temporal Knowledge Graphs. In Meroño Peñuela A, Dimou A, Troncy R, Lisena P, Hartig O, Acosta M, Alam M, Paulheim H, editors, The Semantic Web - 21st International Conference, ESWC 2024, Proceedings: 21st International Conference, ESWC 2024 Hersonissos, Crete, Greece, May 26–30, 2024 Proceedings, Part I. Springer. 2024. p. 59–78. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). doi: 10.1007/978-3-031-60626-7_4