Improving Self-Supervised Dimensionality Reduction: Exploring Hyperparameters and Pseudo-labeling Strategies

Artur Oliveira; Mateus Espadoto; Roberto Hirata; Nina Hirata; Alex Telea

doi:10.1007/978-3-031-25477-2_7

Improving Self-Supervised Dimensionality Reduction: Exploring Hyperparameters and Pseudo-labeling Strategies

Artur Oliveira
, Mateus Espadoto
, Roberto Hirata
, Nina Hirata
, Alex Telea

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › Academic › peer-review

Abstract

Dimensionality reduction (DR) is an essential tool for the visualization of high-dimensional data. The recently proposed Self-Supervised Network Projection (SSNP) method addresses DR with a number of attractive features, such as high computational scalability, genericity, stability and out-of-sample support, computation of an inverse mapping, and the ability of data clustering. Yet, SSNP has an involved computational pipeline using self-supervision based on labels produced by clustering methods and two separate deep learning networks with multiple hyperparameters. In this paper we explore the SSNP method in detail by studying its hyperparameter space and pseudo-labeling strategies. We show how these affect SSNP’s quality and how to set them to optimal values based on extensive evaluations involving multiple datasets, DR methods, and clustering algorithms.

Original language	English
Title of host publication	Computer Vision, Imaging and Computer Graphics Theory and Applications - 16th International Joint Conference, VISIGRAPP 2021, Revised Selected Papers
Subtitle of host publication	16th International Joint Conference, VISIGRAPP 2021, Virtual Event, February 8–10, 2021, Revised Selected Papers
Editors	A. Augusto de Sousa, Vlastimil Havran, Alexis Paljic, Tabitha Peck, Christophe Hurter, Helen Purchase, Helen Purchase, Giovanni Maria Farinella, Petia Radeva, Kadi Bouatouch
Publisher	Springer
Pages	135-161
Number of pages	27
Edition	1
ISBN (Electronic)	978-3-031-25477-2
ISBN (Print)	978-3-031-25476-5
DOIs	https://doi.org/10.1007/978-3-031-25477-2_7
Publication status	Published - 2 Feb 2023

Publication series

Name	Communications in Computer and Information Science
Volume	1691 CCIS
ISSN (Print)	1865-0929
ISSN (Electronic)	1865-0937

Bibliographical note

Funding Information:
Acknowledgments. This study was financed in part by FAPESP grants 2015/22308-2, 2017/25835-9 and 2020/13275-1, and the Coordenac¸ão de Aperfeic¸oamento de Pessoal de Nível Superior - Brasil (CAPES) - Finance Code 001.

Publisher Copyright:
© 2023, Springer Nature Switzerland AG.

Keywords

Dimensionality reduction
Machine learning
Deep learning
Neural networks
Autoencoders

Access to Document

10.1007/978-3-031-25477-2_7

978-3-031-25477-2_7Final published version, 6.55 MBLicence: Taverne

Cite this

Oliveira, A., Espadoto, M., Hirata, R., Hirata, N., & Telea, A. (2023). Improving Self-Supervised Dimensionality Reduction: Exploring Hyperparameters and Pseudo-labeling Strategies. In A. A. de Sousa, V. Havran, A. Paljic, T. Peck, C. Hurter, H. Purchase, H. Purchase, G. M. Farinella, P. Radeva, & K. Bouatouch (Eds.), Computer Vision, Imaging and Computer Graphics Theory and Applications - 16th International Joint Conference, VISIGRAPP 2021, Revised Selected Papers: 16th International Joint Conference, VISIGRAPP 2021, Virtual Event, February 8–10, 2021, Revised Selected Papers (1 ed., pp. 135-161). (Communications in Computer and Information Science; Vol. 1691 CCIS). Springer. https://doi.org/10.1007/978-3-031-25477-2_7

Oliveira, Artur ; Espadoto, Mateus ; Hirata, Roberto et al. / Improving Self-Supervised Dimensionality Reduction: Exploring Hyperparameters and Pseudo-labeling Strategies. Computer Vision, Imaging and Computer Graphics Theory and Applications - 16th International Joint Conference, VISIGRAPP 2021, Revised Selected Papers: 16th International Joint Conference, VISIGRAPP 2021, Virtual Event, February 8–10, 2021, Revised Selected Papers. editor / A. Augusto de Sousa ; Vlastimil Havran ; Alexis Paljic ; Tabitha Peck ; Christophe Hurter ; Helen Purchase ; Helen Purchase ; Giovanni Maria Farinella ; Petia Radeva ; Kadi Bouatouch. 1. ed. Springer, 2023. pp. 135-161 (Communications in Computer and Information Science).

@inproceedings{20955dca2c6243fcb8d6a9fb9d0779be,

title = "Improving Self-Supervised Dimensionality Reduction: Exploring Hyperparameters and Pseudo-labeling Strategies",

abstract = "Dimensionality reduction (DR) is an essential tool for the visualization of high-dimensional data. The recently proposed Self-Supervised Network Projection (SSNP) method addresses DR with a number of attractive features, such as high computational scalability, genericity, stability and out-of-sample support, computation of an inverse mapping, and the ability of data clustering. Yet, SSNP has an involved computational pipeline using self-supervision based on labels produced by clustering methods and two separate deep learning networks with multiple hyperparameters. In this paper we explore the SSNP method in detail by studying its hyperparameter space and pseudo-labeling strategies. We show how these affect SSNP{\textquoteright}s quality and how to set them to optimal values based on extensive evaluations involving multiple datasets, DR methods, and clustering algorithms.",

keywords = "Dimensionality reduction, Machine learning, Deep learning, Neural networks, Autoencoders",

author = "Artur Oliveira and Mateus Espadoto and Roberto Hirata and Nina Hirata and Alex Telea",

note = "Funding Information: Acknowledgments. This study was financed in part by FAPESP grants 2015/22308-2, 2017/25835-9 and 2020/13275-1, and the Coordenac¸{\~a}o de Aperfeic¸oamento de Pessoal de N{\'i}vel Superior - Brasil (CAPES) - Finance Code 001. Publisher Copyright: {\textcopyright} 2023, Springer Nature Switzerland AG.",

year = "2023",

month = feb,

day = "2",

doi = "10.1007/978-3-031-25477-2\_7",

language = "English",

isbn = "978-3-031-25476-5",

series = "Communications in Computer and Information Science",

publisher = "Springer",

pages = "135--161",

editor = "\{de Sousa\}, \{A. Augusto\} and Vlastimil Havran and Alexis Paljic and Tabitha Peck and Christophe Hurter and Helen Purchase and Helen Purchase and Farinella, \{Giovanni Maria\} and Petia Radeva and Kadi Bouatouch",

booktitle = "Computer Vision, Imaging and Computer Graphics Theory and Applications - 16th International Joint Conference, VISIGRAPP 2021, Revised Selected Papers",

edition = "1",

}

Oliveira, A, Espadoto, M, Hirata, R, Hirata, N & Telea, A 2023, Improving Self-Supervised Dimensionality Reduction: Exploring Hyperparameters and Pseudo-labeling Strategies. in AA de Sousa, V Havran, A Paljic, T Peck, C Hurter, H Purchase, H Purchase, GM Farinella, P Radeva & K Bouatouch (eds), Computer Vision, Imaging and Computer Graphics Theory and Applications - 16th International Joint Conference, VISIGRAPP 2021, Revised Selected Papers: 16th International Joint Conference, VISIGRAPP 2021, Virtual Event, February 8–10, 2021, Revised Selected Papers. 1 edn, Communications in Computer and Information Science, vol. 1691 CCIS, Springer, pp. 135-161. https://doi.org/10.1007/978-3-031-25477-2_7

Improving Self-Supervised Dimensionality Reduction: Exploring Hyperparameters and Pseudo-labeling Strategies. / Oliveira, Artur; Espadoto, Mateus; Hirata, Roberto et al.
Computer Vision, Imaging and Computer Graphics Theory and Applications - 16th International Joint Conference, VISIGRAPP 2021, Revised Selected Papers: 16th International Joint Conference, VISIGRAPP 2021, Virtual Event, February 8–10, 2021, Revised Selected Papers. ed. / A. Augusto de Sousa; Vlastimil Havran; Alexis Paljic; Tabitha Peck; Christophe Hurter; Helen Purchase; Helen Purchase; Giovanni Maria Farinella; Petia Radeva; Kadi Bouatouch. 1. ed. Springer, 2023. p. 135-161 (Communications in Computer and Information Science; Vol. 1691 CCIS).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › Academic › peer-review

TY - GEN

T1 - Improving Self-Supervised Dimensionality Reduction: Exploring Hyperparameters and Pseudo-labeling Strategies

AU - Oliveira, Artur

AU - Espadoto, Mateus

AU - Hirata, Roberto

AU - Hirata, Nina

AU - Telea, Alex

N1 - Funding Information: Acknowledgments. This study was financed in part by FAPESP grants 2015/22308-2, 2017/25835-9 and 2020/13275-1, and the Coordenac¸ão de Aperfeic¸oamento de Pessoal de Nível Superior - Brasil (CAPES) - Finance Code 001. Publisher Copyright: © 2023, Springer Nature Switzerland AG.

PY - 2023/2/2

Y1 - 2023/2/2

N2 - Dimensionality reduction (DR) is an essential tool for the visualization of high-dimensional data. The recently proposed Self-Supervised Network Projection (SSNP) method addresses DR with a number of attractive features, such as high computational scalability, genericity, stability and out-of-sample support, computation of an inverse mapping, and the ability of data clustering. Yet, SSNP has an involved computational pipeline using self-supervision based on labels produced by clustering methods and two separate deep learning networks with multiple hyperparameters. In this paper we explore the SSNP method in detail by studying its hyperparameter space and pseudo-labeling strategies. We show how these affect SSNP’s quality and how to set them to optimal values based on extensive evaluations involving multiple datasets, DR methods, and clustering algorithms.

AB - Dimensionality reduction (DR) is an essential tool for the visualization of high-dimensional data. The recently proposed Self-Supervised Network Projection (SSNP) method addresses DR with a number of attractive features, such as high computational scalability, genericity, stability and out-of-sample support, computation of an inverse mapping, and the ability of data clustering. Yet, SSNP has an involved computational pipeline using self-supervision based on labels produced by clustering methods and two separate deep learning networks with multiple hyperparameters. In this paper we explore the SSNP method in detail by studying its hyperparameter space and pseudo-labeling strategies. We show how these affect SSNP’s quality and how to set them to optimal values based on extensive evaluations involving multiple datasets, DR methods, and clustering algorithms.

KW - Dimensionality reduction

KW - Machine learning

KW - Deep learning

KW - Neural networks

KW - Autoencoders

UR - https://www.scopus.com/pages/publications/85149679531

U2 - 10.1007/978-3-031-25477-2_7

DO - 10.1007/978-3-031-25477-2_7

M3 - Conference contribution

SN - 978-3-031-25476-5

T3 - Communications in Computer and Information Science

SP - 135

EP - 161

BT - Computer Vision, Imaging and Computer Graphics Theory and Applications - 16th International Joint Conference, VISIGRAPP 2021, Revised Selected Papers

A2 - de Sousa, A. Augusto

A2 - Havran, Vlastimil

A2 - Paljic, Alexis

A2 - Peck, Tabitha

A2 - Hurter, Christophe

A2 - Purchase, Helen

A2 - Farinella, Giovanni Maria

A2 - Radeva, Petia

A2 - Bouatouch, Kadi

PB - Springer

ER -

Oliveira A, Espadoto M, Hirata R, Hirata N, Telea A. Improving Self-Supervised Dimensionality Reduction: Exploring Hyperparameters and Pseudo-labeling Strategies. In de Sousa AA, Havran V, Paljic A, Peck T, Hurter C, Purchase H, Purchase H, Farinella GM, Radeva P, Bouatouch K, editors, Computer Vision, Imaging and Computer Graphics Theory and Applications - 16th International Joint Conference, VISIGRAPP 2021, Revised Selected Papers: 16th International Joint Conference, VISIGRAPP 2021, Virtual Event, February 8–10, 2021, Revised Selected Papers. 1 ed. Springer. 2023. p. 135-161. (Communications in Computer and Information Science). doi: 10.1007/978-3-031-25477-2_7

Improving Self-Supervised Dimensionality Reduction: Exploring Hyperparameters and Pseudo-labeling Strategies

Abstract

Publication series

Bibliographical note

Keywords

Access to Document

Other files and links

Fingerprint

Cite this