Abstract
Background:
Combining quantitative and qualitative data in a portfolio for high stake assessment is under debate. Although building a portfolio should support the learning of students, it should improve the reliability and validity of the competence level.
Methods:
At Faculty of Veterinary Medicine at Utrecht University (FVMUU) students follow 3 years of clinical rotations in the Master phase of the study. During this period student fill a portfolio with documentation on their skills development in 7 competency domains (veterinary expertise, communication, collaboration, scholarship, health and welfare, personal development and entrepeneurship). The portfolio of the 3td year is assessed by 2 assessors and both grades were compared for reliability. Master track, background of the assessors and student cohort group were taken into account. A mixed effect logistic regression model was used to evaluate the similarity between both grades by estimating odds ratio’s (OR) and intra class correlation coefficients (ICC) for assessor.
Results:
A total of 412 portfolios were assessed. Both grades are correlated (r= 0.8) and 78% of portfolios were given equal grades. Cohort group and master track show differences in dissimilarity between both grades but the background of the assessor did not show a difference in dissimilarity. The estimated ICC was low meaning grade (difference) was not clustered within assessor.
Conclusions:
Although a unique grading system the implemented program of portfolio assessment at FVMUU is showing a high reliability suggesting an effectively working grading system.
Societal impact:
Students and FVMUU have a better insight in the results of the grading system of Master portfolios.
Combining quantitative and qualitative data in a portfolio for high stake assessment is under debate. Although building a portfolio should support the learning of students, it should improve the reliability and validity of the competence level.
Methods:
At Faculty of Veterinary Medicine at Utrecht University (FVMUU) students follow 3 years of clinical rotations in the Master phase of the study. During this period student fill a portfolio with documentation on their skills development in 7 competency domains (veterinary expertise, communication, collaboration, scholarship, health and welfare, personal development and entrepeneurship). The portfolio of the 3td year is assessed by 2 assessors and both grades were compared for reliability. Master track, background of the assessors and student cohort group were taken into account. A mixed effect logistic regression model was used to evaluate the similarity between both grades by estimating odds ratio’s (OR) and intra class correlation coefficients (ICC) for assessor.
Results:
A total of 412 portfolios were assessed. Both grades are correlated (r= 0.8) and 78% of portfolios were given equal grades. Cohort group and master track show differences in dissimilarity between both grades but the background of the assessor did not show a difference in dissimilarity. The estimated ICC was low meaning grade (difference) was not clustered within assessor.
Conclusions:
Although a unique grading system the implemented program of portfolio assessment at FVMUU is showing a high reliability suggesting an effectively working grading system.
Societal impact:
Students and FVMUU have a better insight in the results of the grading system of Master portfolios.
| Original language | English |
|---|---|
| Publication status | Published - 15 Nov 2018 |
| Event | "Making a Difference" - Veterinary Science Day 2018 - De Landgoederij, Bunnik, Netherlands Duration: 15 Nov 2018 → 15 Nov 2018 https://www.veterinaryscienceday.nl/ |
Conference
| Conference | "Making a Difference" - Veterinary Science Day 2018 |
|---|---|
| Abbreviated title | VSD 2018 |
| Country/Territory | Netherlands |
| City | Bunnik |
| Period | 15/11/18 → 15/11/18 |
| Internet address |
Keywords
- reliability
- grading