An experiment on comparing textual vs. visual industrial methods for security risk assessment

Katsiaryna Labunets; Federica Paci; Fabio Massacci; Raminder Ruprai

doi:10.1109/EmpiRE.2014.6890113

An experiment on comparing textual vs. visual industrial methods for security risk assessment

Katsiaryna Labunets^*, Federica Paci, Fabio Massacci, Raminder Ruprai

^*Corresponding author for this work

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › Academic › peer-review

Abstract

Many security risk assessment methods have been proposed both from academia and industry. However, little empirical evaluation has been done to investigate how these methods are effective in practice. In this paper we report a controlled experiment that we conducted to compare the effectiveness and participants' perception of visual versus textual methods for security risk assessment used in industry. As instances of the methods we selected CORAS, a method by SINTEF used to provide security risk assessment consulting services, and SecRAM, a method by EUROCONTROL used to conduct security risk assessment within air traffic management. The experiment involved 29 MSc students who applied both methods to an application scenario from Smart Grid domain. The dependent variables were effectiveness of the methods measured as number of specific threats and security controls identified, and perception of the methods measured through post-task questionnaires based on the Technology Acceptance Model. The experiment shows that while there is no difference in the actual effectiveness of the two methods, the visual method is better perceived by the participants.

Original language	English
Title of host publication	2014 IEEE 4th International Workshop on Empirical Requirements Engineering, EmpiRE 2014 - Proceedings
Publisher	IEEE
Pages	28-35
Number of pages	8
ISBN (Electronic)	9781479963379
DOIs	https://doi.org/10.1109/EmpiRE.2014.6890113
Publication status	Published - 3 Sept 2014
Event	2014 IEEE 4th International Workshop on Empirical Requirements Engineering, EmpiRE 2014 - Karlskrona, Sweden Duration: 25 Aug 2014 → …

Publication series

Name	2014 IEEE 4th International Workshop on Empirical Requirements Engineering, EmpiRE 2014 - Proceedings

Conference

Conference	2014 IEEE 4th International Workshop on Empirical Requirements Engineering, EmpiRE 2014
Country/Territory	Sweden
City	Karlskrona
Period	25/08/14 → …

Bibliographical note

Publisher Copyright:
© 2014 IEEE.

Keywords

controlled experiment
security risk assessment methods
technology acceptance model

UN SDGs

This output contributes to the following UN Sustainable Development Goals (SDGs)

Access to Document

10.1109/EmpiRE.2014.6890113

Cite this

Labunets, K., Paci, F., Massacci, F., & Ruprai, R. (2014). An experiment on comparing textual vs. visual industrial methods for security risk assessment. In 2014 IEEE 4th International Workshop on Empirical Requirements Engineering, EmpiRE 2014 - Proceedings (pp. 28-35). Article 6890113 (2014 IEEE 4th International Workshop on Empirical Requirements Engineering, EmpiRE 2014 - Proceedings). IEEE. https://doi.org/10.1109/EmpiRE.2014.6890113

Labunets, Katsiaryna ; Paci, Federica ; Massacci, Fabio et al. / An experiment on comparing textual vs. visual industrial methods for security risk assessment. 2014 IEEE 4th International Workshop on Empirical Requirements Engineering, EmpiRE 2014 - Proceedings. IEEE, 2014. pp. 28-35 (2014 IEEE 4th International Workshop on Empirical Requirements Engineering, EmpiRE 2014 - Proceedings).

@inproceedings{7a2acc0a58df43fa99c3721ef60ffd46,

title = "An experiment on comparing textual vs. visual industrial methods for security risk assessment",

abstract = "Many security risk assessment methods have been proposed both from academia and industry. However, little empirical evaluation has been done to investigate how these methods are effective in practice. In this paper we report a controlled experiment that we conducted to compare the effectiveness and participants' perception of visual versus textual methods for security risk assessment used in industry. As instances of the methods we selected CORAS, a method by SINTEF used to provide security risk assessment consulting services, and SecRAM, a method by EUROCONTROL used to conduct security risk assessment within air traffic management. The experiment involved 29 MSc students who applied both methods to an application scenario from Smart Grid domain. The dependent variables were effectiveness of the methods measured as number of specific threats and security controls identified, and perception of the methods measured through post-task questionnaires based on the Technology Acceptance Model. The experiment shows that while there is no difference in the actual effectiveness of the two methods, the visual method is better perceived by the participants.",

keywords = "controlled experiment, security risk assessment methods, technology acceptance model",

author = "Katsiaryna Labunets and Federica Paci and Fabio Massacci and Raminder Ruprai",

note = "Publisher Copyright: {\textcopyright} 2014 IEEE.; 2014 IEEE 4th International Workshop on Empirical Requirements Engineering, EmpiRE 2014 ; Conference date: 25-08-2014",

year = "2014",

month = sep,

day = "3",

doi = "10.1109/EmpiRE.2014.6890113",

language = "English",

series = "2014 IEEE 4th International Workshop on Empirical Requirements Engineering, EmpiRE 2014 - Proceedings",

publisher = "IEEE",

pages = "28--35",

booktitle = "2014 IEEE 4th International Workshop on Empirical Requirements Engineering, EmpiRE 2014 - Proceedings",

address = "United States",

}

Labunets, K, Paci, F, Massacci, F & Ruprai, R 2014, An experiment on comparing textual vs. visual industrial methods for security risk assessment. in 2014 IEEE 4th International Workshop on Empirical Requirements Engineering, EmpiRE 2014 - Proceedings., 6890113, 2014 IEEE 4th International Workshop on Empirical Requirements Engineering, EmpiRE 2014 - Proceedings, IEEE, pp. 28-35, 2014 IEEE 4th International Workshop on Empirical Requirements Engineering, EmpiRE 2014, Karlskrona, Sweden, 25/08/14. https://doi.org/10.1109/EmpiRE.2014.6890113

An experiment on comparing textual vs. visual industrial methods for security risk assessment. / Labunets, Katsiaryna; Paci, Federica; Massacci, Fabio et al.
2014 IEEE 4th International Workshop on Empirical Requirements Engineering, EmpiRE 2014 - Proceedings. IEEE, 2014. p. 28-35 6890113 (2014 IEEE 4th International Workshop on Empirical Requirements Engineering, EmpiRE 2014 - Proceedings).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › Academic › peer-review

TY - GEN

T1 - An experiment on comparing textual vs. visual industrial methods for security risk assessment

AU - Labunets, Katsiaryna

AU - Paci, Federica

AU - Massacci, Fabio

AU - Ruprai, Raminder

PY - 2014/9/3

Y1 - 2014/9/3

N2 - Many security risk assessment methods have been proposed both from academia and industry. However, little empirical evaluation has been done to investigate how these methods are effective in practice. In this paper we report a controlled experiment that we conducted to compare the effectiveness and participants' perception of visual versus textual methods for security risk assessment used in industry. As instances of the methods we selected CORAS, a method by SINTEF used to provide security risk assessment consulting services, and SecRAM, a method by EUROCONTROL used to conduct security risk assessment within air traffic management. The experiment involved 29 MSc students who applied both methods to an application scenario from Smart Grid domain. The dependent variables were effectiveness of the methods measured as number of specific threats and security controls identified, and perception of the methods measured through post-task questionnaires based on the Technology Acceptance Model. The experiment shows that while there is no difference in the actual effectiveness of the two methods, the visual method is better perceived by the participants.

AB - Many security risk assessment methods have been proposed both from academia and industry. However, little empirical evaluation has been done to investigate how these methods are effective in practice. In this paper we report a controlled experiment that we conducted to compare the effectiveness and participants' perception of visual versus textual methods for security risk assessment used in industry. As instances of the methods we selected CORAS, a method by SINTEF used to provide security risk assessment consulting services, and SecRAM, a method by EUROCONTROL used to conduct security risk assessment within air traffic management. The experiment involved 29 MSc students who applied both methods to an application scenario from Smart Grid domain. The dependent variables were effectiveness of the methods measured as number of specific threats and security controls identified, and perception of the methods measured through post-task questionnaires based on the Technology Acceptance Model. The experiment shows that while there is no difference in the actual effectiveness of the two methods, the visual method is better perceived by the participants.

KW - controlled experiment

KW - security risk assessment methods

KW - technology acceptance model

UR - http://www.scopus.com/inward/record.url?scp=84928137931&partnerID=8YFLogxK

U2 - 10.1109/EmpiRE.2014.6890113

DO - 10.1109/EmpiRE.2014.6890113

M3 - Conference contribution

AN - SCOPUS:84928137931

T3 - 2014 IEEE 4th International Workshop on Empirical Requirements Engineering, EmpiRE 2014 - Proceedings

SP - 28

EP - 35

BT - 2014 IEEE 4th International Workshop on Empirical Requirements Engineering, EmpiRE 2014 - Proceedings

PB - IEEE

T2 - 2014 IEEE 4th International Workshop on Empirical Requirements Engineering, EmpiRE 2014

Y2 - 25 August 2014

ER -

Labunets K, Paci F, Massacci F, Ruprai R. An experiment on comparing textual vs. visual industrial methods for security risk assessment. In 2014 IEEE 4th International Workshop on Empirical Requirements Engineering, EmpiRE 2014 - Proceedings. IEEE. 2014. p. 28-35. 6890113. (2014 IEEE 4th International Workshop on Empirical Requirements Engineering, EmpiRE 2014 - Proceedings). doi: 10.1109/EmpiRE.2014.6890113

An experiment on comparing textual vs. visual industrial methods for security risk assessment

Abstract

Publication series

Conference

Bibliographical note

Keywords

UN SDGs

Access to Document

Other files and links

Fingerprint

Cite this