Abstract
Introduction
Combining quantitative and qualitative data in a portfolio for high stake assessment is under debate. Although building a portfolio should support the learning of students, it should improve the reliability and validity of the competence level.
Methods
At Faculty of Veterinary Medicine at Utrecht University (FVMUU) students follow 3 years of clinical rotations in the Master phase of the study. During this period student fill a portfolio with documentation on their skills development in 7 competency domains (veterinary expertise, communication, collaboration, scholarship, health and welfare, personal development and entrepeneurship). The portfolio of the 3td year, which is less rich compared to the portfolio of the first 2 years, is assessed by 2 assessors and both grades were compared for reliability. Master track, background of the assessors and student cohort group were taken into account.
A mixed effect logistic regression model was used to evaluate the similarity between both grades by estimating odds ratio’s (OR) and intra class correlation coefficients (ICC) for assessor.
Results
A total of 412 portfolios were assessed. Both grades are correlated (r= 0.8) and 78% of portfolios were given equal grades. Cohort group and master track show differences in dissimilarity between both grades but the background of the assessor did not show a difference in dissimilarity. The estimated ICC was low meaning grade (difference) was not clustered within assessor.
Discussion
Although a unique grading system the implemented program of portfolio assessment at FVMUU is showing a high reliability suggesting an effectively working grading system.
Combining quantitative and qualitative data in a portfolio for high stake assessment is under debate. Although building a portfolio should support the learning of students, it should improve the reliability and validity of the competence level.
Methods
At Faculty of Veterinary Medicine at Utrecht University (FVMUU) students follow 3 years of clinical rotations in the Master phase of the study. During this period student fill a portfolio with documentation on their skills development in 7 competency domains (veterinary expertise, communication, collaboration, scholarship, health and welfare, personal development and entrepeneurship). The portfolio of the 3td year, which is less rich compared to the portfolio of the first 2 years, is assessed by 2 assessors and both grades were compared for reliability. Master track, background of the assessors and student cohort group were taken into account.
A mixed effect logistic regression model was used to evaluate the similarity between both grades by estimating odds ratio’s (OR) and intra class correlation coefficients (ICC) for assessor.
Results
A total of 412 portfolios were assessed. Both grades are correlated (r= 0.8) and 78% of portfolios were given equal grades. Cohort group and master track show differences in dissimilarity between both grades but the background of the assessor did not show a difference in dissimilarity. The estimated ICC was low meaning grade (difference) was not clustered within assessor.
Discussion
Although a unique grading system the implemented program of portfolio assessment at FVMUU is showing a high reliability suggesting an effectively working grading system.
| Original language | English |
|---|---|
| Publication status | Published - 5 Jul 2018 |
| Event | VetEd symposium 2018 - Utrecht University Duration: 4 Jul 2018 → 6 Jul 2018 |
Conference
| Conference | VetEd symposium 2018 |
|---|---|
| Period | 4/07/18 → 6/07/18 |