Evaluating Software Quality Through User Reviews: The ISOftSentiment Tool

Fang Hou; Liang Feng; Siamak Farshidi; Slinger Jansen

doi:10.1007/978-3-031-78386-9_6

Evaluating Software Quality Through User Reviews: The ISOftSentiment Tool

Fang Hou^*, Liang Feng, Siamak Farshidi, Slinger Jansen

^*Corresponding author for this work

Sub Software Production

Utrecht University

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › Academic › peer-review

Abstract

User reviews on software platforms offer direct insights into user attitudes toward software packages, serving as invaluable sources for assessing software quality. However, there is a lack of effective methods or tools to analyze the quality attributes that are of most concern in user reviews, as well as the strengths and weaknesses of software packages in terms of quality. This paper introduces ISOftSentiment, a hybrid tool developed to overcome this challenge and facilitate efficient evaluation of software package quality attributes based on user reviews. ISOftSentiment integrates a sentiment analysis model with the ISO/IEC 25010 product quality model and a fuzzy consensus mechanism. This integration processes user reviews to produce a quality sentiment matrix, which not only summarizes the quality analysis results but also highlights the consensus sentiment for relevant quality attributes. The matrix provides insights into both the strengths and weaknesses of software packages. Based on these insights, software engineers gain a clearer understanding of a package’s quality and trustworthiness, thereby making more informed decisions during the software selection or evaluation process and avoiding the use of unreliable components. Consequently, this tool ensures that the final product maintains high standards of quality.

Original language	English
Title of host publication	Product-Focused Software Process Improvement
Subtitle of host publication	25th International Conference, PROFES 2024, Proceedings
Editors	Dietmar Pfahl, Hina Anwar, Javier Gonzalez Huerta, Jil Klünder
Publisher	Springer
Pages	75-91
Number of pages	17
ISBN (Electronic)	978-3-031-78386-9
ISBN (Print)	978-3-031-78385-2
DOIs	https://doi.org/10.1007/978-3-031-78386-9_6
Publication status	Published - 2025
Event	25th International Conference on Product-Focused Software Process Improvement, PROFES 2024 - Tartu, Estonia Duration: 2 Dec 2024 → 4 Dec 2024

Publication series

Name	Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume	15452 LNCS
ISSN (Print)	0302-9743
ISSN (Electronic)	1611-3349

Conference

Conference	25th International Conference on Product-Focused Software Process Improvement, PROFES 2024
Country/Territory	Estonia
City	Tartu
Period	2/12/24 → 4/12/24

Bibliographical note

Publisher Copyright:
© The Author(s), under exclusive license to Springer Nature Switzerland AG 2025.

Keywords

Package Selection
Sentiment Analysis
Software Engineering
Software Quality

Access to Document

10.1007/978-3-031-78386-9_6

Cite this

Hou, F., Feng, L., Farshidi, S., & Jansen, S. (2025). Evaluating Software Quality Through User Reviews: The ISOftSentiment Tool. In D. Pfahl, H. Anwar, J. Gonzalez Huerta, & J. Klünder (Eds.), Product-Focused Software Process Improvement: 25th International Conference, PROFES 2024, Proceedings (pp. 75-91). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 15452 LNCS). Springer. https://doi.org/10.1007/978-3-031-78386-9_6

Hou, Fang ; Feng, Liang ; Farshidi, Siamak et al. / Evaluating Software Quality Through User Reviews : The ISOftSentiment Tool. Product-Focused Software Process Improvement: 25th International Conference, PROFES 2024, Proceedings. editor / Dietmar Pfahl ; Hina Anwar ; Javier Gonzalez Huerta ; Jil Klünder. Springer, 2025. pp. 75-91 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).

@inproceedings{c44650179fc04dc59babc2c394e05009,

title = "Evaluating Software Quality Through User Reviews: The ISOftSentiment Tool",

abstract = "User reviews on software platforms offer direct insights into user attitudes toward software packages, serving as invaluable sources for assessing software quality. However, there is a lack of effective methods or tools to analyze the quality attributes that are of most concern in user reviews, as well as the strengths and weaknesses of software packages in terms of quality. This paper introduces ISOftSentiment, a hybrid tool developed to overcome this challenge and facilitate efficient evaluation of software package quality attributes based on user reviews. ISOftSentiment integrates a sentiment analysis model with the ISO/IEC 25010 product quality model and a fuzzy consensus mechanism. This integration processes user reviews to produce a quality sentiment matrix, which not only summarizes the quality analysis results but also highlights the consensus sentiment for relevant quality attributes. The matrix provides insights into both the strengths and weaknesses of software packages. Based on these insights, software engineers gain a clearer understanding of a package{\textquoteright}s quality and trustworthiness, thereby making more informed decisions during the software selection or evaluation process and avoiding the use of unreliable components. Consequently, this tool ensures that the final product maintains high standards of quality.",

keywords = "Package Selection, Sentiment Analysis, Software Engineering, Software Quality",

author = "Fang Hou and Liang Feng and Siamak Farshidi and Slinger Jansen",

note = "Publisher Copyright: {\textcopyright} The Author(s), under exclusive license to Springer Nature Switzerland AG 2025.; 25th International Conference on Product-Focused Software Process Improvement, PROFES 2024 ; Conference date: 02-12-2024 Through 04-12-2024",

year = "2025",

doi = "10.1007/978-3-031-78386-9_6",

language = "English",

isbn = "978-3-031-78385-2",

series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",

publisher = "Springer",

pages = "75--91",

editor = "Dietmar Pfahl and Hina Anwar and {Gonzalez Huerta}, Javier and Jil Kl{\"u}nder",

booktitle = "Product-Focused Software Process Improvement",

}

Hou, F, Feng, L, Farshidi, S & Jansen, S 2025, Evaluating Software Quality Through User Reviews: The ISOftSentiment Tool. in D Pfahl, H Anwar, J Gonzalez Huerta & J Klünder (eds), Product-Focused Software Process Improvement: 25th International Conference, PROFES 2024, Proceedings. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 15452 LNCS, Springer, pp. 75-91, 25th International Conference on Product-Focused Software Process Improvement, PROFES 2024, Tartu, Estonia, 2/12/24. https://doi.org/10.1007/978-3-031-78386-9_6

Evaluating Software Quality Through User Reviews: The ISOftSentiment Tool. / Hou, Fang; Feng, Liang; Farshidi, Siamak et al.
Product-Focused Software Process Improvement: 25th International Conference, PROFES 2024, Proceedings. ed. / Dietmar Pfahl; Hina Anwar; Javier Gonzalez Huerta; Jil Klünder. Springer, 2025. p. 75-91 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 15452 LNCS).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › Academic › peer-review

TY - GEN

T1 - Evaluating Software Quality Through User Reviews

T2 - 25th International Conference on Product-Focused Software Process Improvement, PROFES 2024

AU - Hou, Fang

AU - Feng, Liang

AU - Farshidi, Siamak

AU - Jansen, Slinger

N1 - Publisher Copyright: © The Author(s), under exclusive license to Springer Nature Switzerland AG 2025.

PY - 2025

Y1 - 2025

N2 - User reviews on software platforms offer direct insights into user attitudes toward software packages, serving as invaluable sources for assessing software quality. However, there is a lack of effective methods or tools to analyze the quality attributes that are of most concern in user reviews, as well as the strengths and weaknesses of software packages in terms of quality. This paper introduces ISOftSentiment, a hybrid tool developed to overcome this challenge and facilitate efficient evaluation of software package quality attributes based on user reviews. ISOftSentiment integrates a sentiment analysis model with the ISO/IEC 25010 product quality model and a fuzzy consensus mechanism. This integration processes user reviews to produce a quality sentiment matrix, which not only summarizes the quality analysis results but also highlights the consensus sentiment for relevant quality attributes. The matrix provides insights into both the strengths and weaknesses of software packages. Based on these insights, software engineers gain a clearer understanding of a package’s quality and trustworthiness, thereby making more informed decisions during the software selection or evaluation process and avoiding the use of unreliable components. Consequently, this tool ensures that the final product maintains high standards of quality.

AB - User reviews on software platforms offer direct insights into user attitudes toward software packages, serving as invaluable sources for assessing software quality. However, there is a lack of effective methods or tools to analyze the quality attributes that are of most concern in user reviews, as well as the strengths and weaknesses of software packages in terms of quality. This paper introduces ISOftSentiment, a hybrid tool developed to overcome this challenge and facilitate efficient evaluation of software package quality attributes based on user reviews. ISOftSentiment integrates a sentiment analysis model with the ISO/IEC 25010 product quality model and a fuzzy consensus mechanism. This integration processes user reviews to produce a quality sentiment matrix, which not only summarizes the quality analysis results but also highlights the consensus sentiment for relevant quality attributes. The matrix provides insights into both the strengths and weaknesses of software packages. Based on these insights, software engineers gain a clearer understanding of a package’s quality and trustworthiness, thereby making more informed decisions during the software selection or evaluation process and avoiding the use of unreliable components. Consequently, this tool ensures that the final product maintains high standards of quality.

KW - Package Selection

KW - Sentiment Analysis

KW - Software Engineering

KW - Software Quality

UR - http://www.scopus.com/inward/record.url?scp=85211942849&partnerID=8YFLogxK

U2 - 10.1007/978-3-031-78386-9_6

DO - 10.1007/978-3-031-78386-9_6

M3 - Conference contribution

AN - SCOPUS:85211942849

SN - 978-3-031-78385-2

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 75

EP - 91

BT - Product-Focused Software Process Improvement

A2 - Pfahl, Dietmar

A2 - Anwar, Hina

A2 - Gonzalez Huerta, Javier

A2 - Klünder, Jil

PB - Springer

Y2 - 2 December 2024 through 4 December 2024

ER -

Hou F, Feng L, Farshidi S, Jansen S. Evaluating Software Quality Through User Reviews: The ISOftSentiment Tool. In Pfahl D, Anwar H, Gonzalez Huerta J, Klünder J, editors, Product-Focused Software Process Improvement: 25th International Conference, PROFES 2024, Proceedings. Springer. 2025. p. 75-91. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). Epub 2024 Nov 27. doi: 10.1007/978-3-031-78386-9_6

Evaluating Software Quality Through User Reviews: The ISOftSentiment Tool

Abstract

Publication series

Conference

Bibliographical note

Keywords

Access to Document

Other files and links

Embargoed Document

Fingerprint

Cite this