SceneFND: Multimodal fake news detection by modelling scene context information

Guobiao Zhang, Anastasia Giachanou*, Paolo Rosso

*Corresponding author for this work

Research output: Contribution to journalArticleAcademicpeer-review

Abstract

Fake news is a threat for the society and can create a lot of confusion to people regarding what is true and what not. Fake news usually contain manipulated content, such as text or images that attract the interest of the readers with the aim to convince them on their truthfulness. In this article, we propose SceneFND (Scene Fake News Detection), a system that combines textual, contextual scene and visual representation to address the problem of multimodal fake news detection. The textual representation is based on word embeddings that are passed into a bidirectional long short-term memory network. Both the contextual scene and the visual representations are based on the images contained in the news post. The place, weather and season scenes are extracted from the image. Our statistical analysis on the scenes showed that there are statistically significant differences regarding their frequency in fake and real news. In addition, our experimental results on two real world datasets show that the integration of the contextual scenes is effective for fake news detection. In particular, SceneFND improved the performance of the textual baseline by 3.48% in PolitiFact and by 3.32% in GossipCop datasets. Finally, we show the suitability of the scene information for the task and present some examples to explain its effectiveness in capturing the relevance between images and text.

Original languageEnglish
Pages (from-to)355-367
Number of pages13
JournalJournal of Information Science
Volume50
Issue number2
Early online date23 Apr 2022
DOIs
Publication statusPublished - Apr 2024

Keywords

  • Fake news detection
  • multimodal feature fusion
  • social media
  • visual scene information

Fingerprint

Dive into the research topics of 'SceneFND: Multimodal fake news detection by modelling scene context information'. Together they form a unique fingerprint.

Cite this