Abstract
Rationale
Errors occur in different phases of the image interpretation process of learners. Insight in the error types made by learners is crucial for giving effective feedback. Using a step-by-step-questions assessment procedure to identify error types in image interpretation and reveal partial knowledge of or hidden errors in the interpretation process, we investigated which error types can be identified in the image interpretation process of radiology clerks and the reliability of this procedure in terms of inter-rater agreement.
Methods
Hundred-nine radiology clerks took a radiology image interpretation test consisting of ten CT image cases and one to three X-ray cases. The image interpretation questions concerned step-by-step questions: labelling an abnormality (perception), describing the abnormality (analysis) and giving a diagnosis and/or advice (synthesis) (1). Errors were coded as perception, analysis, synthesis or undefined errors by two independent observers. A hidden error was identified if a correct diagnosis was given, based on an incorrect perception or analysis. Partial knowledge was identified if an incorrect diagnosis was given based on a correct perception and/or analysis.
Consensus was reached after discussion, in case of discrepancies. Prevalence of error types and inter-rater reliability of the procedure were calculated.
Results
With our step-by-step questions procedure applied to 1351 cases, 831 errors were identified. 638 errors were found in the process of image interpretation (77%), of which 29.6% were perception errors, 15.7% analysis errors and 31.7% synthesis errors. The step-by-step questions revealed hidden errors in 125 cases (9%) and partial knowledge in 243 cases (18%). We found a mean inter-rater reliability of Cohen’s κ = 0.8.
Conclusions
A step-by-step question approach can reliably distinguish perception, analysis and synthesis errors. Besides, the approach reveals hidden errors and partial knowledge of students.
References
[1] van der Gijp A, van der Schaaf MF, van der Schaaf IC, Huige JC, Ravesloot CJ, van Schaik JP, et al. Interpretation of radiological images: towards a framework of knowledge and skills. Adv Health Sci Educ Theory Pract. 2014 Jan 22.
Errors occur in different phases of the image interpretation process of learners. Insight in the error types made by learners is crucial for giving effective feedback. Using a step-by-step-questions assessment procedure to identify error types in image interpretation and reveal partial knowledge of or hidden errors in the interpretation process, we investigated which error types can be identified in the image interpretation process of radiology clerks and the reliability of this procedure in terms of inter-rater agreement.
Methods
Hundred-nine radiology clerks took a radiology image interpretation test consisting of ten CT image cases and one to three X-ray cases. The image interpretation questions concerned step-by-step questions: labelling an abnormality (perception), describing the abnormality (analysis) and giving a diagnosis and/or advice (synthesis) (1). Errors were coded as perception, analysis, synthesis or undefined errors by two independent observers. A hidden error was identified if a correct diagnosis was given, based on an incorrect perception or analysis. Partial knowledge was identified if an incorrect diagnosis was given based on a correct perception and/or analysis.
Consensus was reached after discussion, in case of discrepancies. Prevalence of error types and inter-rater reliability of the procedure were calculated.
Results
With our step-by-step questions procedure applied to 1351 cases, 831 errors were identified. 638 errors were found in the process of image interpretation (77%), of which 29.6% were perception errors, 15.7% analysis errors and 31.7% synthesis errors. The step-by-step questions revealed hidden errors in 125 cases (9%) and partial knowledge in 243 cases (18%). We found a mean inter-rater reliability of Cohen’s κ = 0.8.
Conclusions
A step-by-step question approach can reliably distinguish perception, analysis and synthesis errors. Besides, the approach reveals hidden errors and partial knowledge of students.
References
[1] van der Gijp A, van der Schaaf MF, van der Schaaf IC, Huige JC, Ravesloot CJ, van Schaik JP, et al. Interpretation of radiological images: towards a framework of knowledge and skills. Adv Health Sci Educ Theory Pract. 2014 Jan 22.
| Original language | English |
|---|---|
| Publication status | Unpublished - 3 Jun 2015 |
| Event | Medical Image Perception Society (MIPS) - Ghent, Belgium Duration: 3 Jun 2015 → 5 Jun 2015 |
Conference
| Conference | Medical Image Perception Society (MIPS) |
|---|---|
| Country/Territory | Belgium |
| City | Ghent |
| Period | 3/06/15 → 5/06/15 |