DetCat: Detecting Categorical Outliers in Relational Datasets

Arthur Zylinski; Abdulhakim A. Qahtan

doi:10.1145/3627673.3679212

DetCat: Detecting Categorical Outliers in Relational Datasets

Arthur Zylinski
, Abdulhakim A. Qahtan^*

^*Corresponding author for this work

Utrecht University

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › Academic › peer-review

Abstract

Poor data quality significantly affects different data analytics tasks, leading to inaccurate decisions and poor predictions of the machine learning models. Outliers represent one of the most common data glitches that impact data quality. While detecting outliers in numerical data has been extensively studied, few attempts were made to solve the problem of detecting categorical outliers. In this paper, we introduce DetCat for detecting categorical outliers in relational datasets, by utilizing the syntactic structure of the values. For a given attribute, DetCat identifies a set of patterns that represents the majority of the values as dominating patterns. Data values that cannot be generated by the dominating patterns are declared as outliers. The demo will show the effectiveness of our tool in detecting categorical outliers and discovering the syntactical data patterns.

Original language	English
Title of host publication	CIKM 2024 - Proceedings of the 33rd ACM International Conference on Information and Knowledge Management
Publisher	Association for Computing Machinery
Pages	5318-5322
Number of pages	5
ISBN (Electronic)	9798400704369
DOIs	https://doi.org/10.1145/3627673.3679212
Publication status	Published - 21 Oct 2024
Event	33rd ACM International Conference on Information and Knowledge Management, CIKM 2024 - Boise, United States Duration: 21 Oct 2024 → 25 Oct 2024

Publication series

Name	International Conference on Information and Knowledge Management, Proceedings
ISSN (Print)	2155-0751

Conference

Conference	33rd ACM International Conference on Information and Knowledge Management, CIKM 2024
Country/Territory	United States
City	Boise
Period	21/10/24 → 25/10/24

Bibliographical note

Publisher Copyright:
© 2024 ACM.

Keywords

categorical values
outliers
similarity metrics
syntactic structure

Access to Document

10.1145/3627673.3679212

3627673.3679212Final published version, 1.2 MBLicence: Taverne

Cite this

Zylinski, A., & Qahtan, A. A. (2024). DetCat: Detecting Categorical Outliers in Relational Datasets. In CIKM 2024 - Proceedings of the 33rd ACM International Conference on Information and Knowledge Management (pp. 5318-5322). (International Conference on Information and Knowledge Management, Proceedings). Association for Computing Machinery. https://doi.org/10.1145/3627673.3679212

@inproceedings{e4d761c0b21c4bb9bbc235c3ae9aa343,

title = "DetCat: Detecting Categorical Outliers in Relational Datasets",

abstract = "Poor data quality significantly affects different data analytics tasks, leading to inaccurate decisions and poor predictions of the machine learning models. Outliers represent one of the most common data glitches that impact data quality. While detecting outliers in numerical data has been extensively studied, few attempts were made to solve the problem of detecting categorical outliers. In this paper, we introduce DetCat for detecting categorical outliers in relational datasets, by utilizing the syntactic structure of the values. For a given attribute, DetCat identifies a set of patterns that represents the majority of the values as dominating patterns. Data values that cannot be generated by the dominating patterns are declared as outliers. The demo will show the effectiveness of our tool in detecting categorical outliers and discovering the syntactical data patterns.",

keywords = "categorical values, outliers, similarity metrics, syntactic structure",

author = "Arthur Zylinski and Qahtan, \{Abdulhakim A.\}",

note = "Publisher Copyright: {\textcopyright} 2024 ACM.; 33rd ACM International Conference on Information and Knowledge Management, CIKM 2024 ; Conference date: 21-10-2024 Through 25-10-2024",

year = "2024",

month = oct,

day = "21",

doi = "10.1145/3627673.3679212",

language = "English",

series = "International Conference on Information and Knowledge Management, Proceedings",

publisher = "Association for Computing Machinery",

pages = "5318--5322",

booktitle = "CIKM 2024 - Proceedings of the 33rd ACM International Conference on Information and Knowledge Management",

address = "United States",

}

Zylinski, A & Qahtan, AA 2024, DetCat: Detecting Categorical Outliers in Relational Datasets. in CIKM 2024 - Proceedings of the 33rd ACM International Conference on Information and Knowledge Management. International Conference on Information and Knowledge Management, Proceedings, Association for Computing Machinery, pp. 5318-5322, 33rd ACM International Conference on Information and Knowledge Management, CIKM 2024, Boise, United States, 21/10/24. https://doi.org/10.1145/3627673.3679212

DetCat: Detecting Categorical Outliers in Relational Datasets. / Zylinski, Arthur; Qahtan, Abdulhakim A.
CIKM 2024 - Proceedings of the 33rd ACM International Conference on Information and Knowledge Management. Association for Computing Machinery, 2024. p. 5318-5322 (International Conference on Information and Knowledge Management, Proceedings).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › Academic › peer-review

TY - GEN

T1 - DetCat

T2 - 33rd ACM International Conference on Information and Knowledge Management, CIKM 2024

AU - Zylinski, Arthur

AU - Qahtan, Abdulhakim A.

PY - 2024/10/21

Y1 - 2024/10/21

N2 - Poor data quality significantly affects different data analytics tasks, leading to inaccurate decisions and poor predictions of the machine learning models. Outliers represent one of the most common data glitches that impact data quality. While detecting outliers in numerical data has been extensively studied, few attempts were made to solve the problem of detecting categorical outliers. In this paper, we introduce DetCat for detecting categorical outliers in relational datasets, by utilizing the syntactic structure of the values. For a given attribute, DetCat identifies a set of patterns that represents the majority of the values as dominating patterns. Data values that cannot be generated by the dominating patterns are declared as outliers. The demo will show the effectiveness of our tool in detecting categorical outliers and discovering the syntactical data patterns.

AB - Poor data quality significantly affects different data analytics tasks, leading to inaccurate decisions and poor predictions of the machine learning models. Outliers represent one of the most common data glitches that impact data quality. While detecting outliers in numerical data has been extensively studied, few attempts were made to solve the problem of detecting categorical outliers. In this paper, we introduce DetCat for detecting categorical outliers in relational datasets, by utilizing the syntactic structure of the values. For a given attribute, DetCat identifies a set of patterns that represents the majority of the values as dominating patterns. Data values that cannot be generated by the dominating patterns are declared as outliers. The demo will show the effectiveness of our tool in detecting categorical outliers and discovering the syntactical data patterns.

KW - categorical values

KW - outliers

KW - similarity metrics

KW - syntactic structure

UR - https://www.scopus.com/pages/publications/85209989726

U2 - 10.1145/3627673.3679212

DO - 10.1145/3627673.3679212

M3 - Conference contribution

AN - SCOPUS:85209989726

T3 - International Conference on Information and Knowledge Management, Proceedings

SP - 5318

EP - 5322

BT - CIKM 2024 - Proceedings of the 33rd ACM International Conference on Information and Knowledge Management

PB - Association for Computing Machinery

Y2 - 21 October 2024 through 25 October 2024

ER -

DetCat: Detecting Categorical Outliers in Relational Datasets

Abstract

Publication series

Conference

Bibliographical note

Keywords

Access to Document

Other files and links

Fingerprint

Cite this