Abstract
Feedback is the most powerful engine of any learning process. In mathematics education, the possibilities to assess automatedly are thoroughly explored. However, students face difficulties expressing themselves mathematically on a computer and learning systems can often only assess the outcome, not the solving method. Research indicates that automated tests focus too much on procedural fluency at the expense of higherorder thinking questions. It takes much effort to develop digital tests, and teachers are sceptical of using automated assessments, meaning that paperandpencil tests still dominate mathematics classrooms. One of the characteristics of mathematical assessment is that wrong answers tend to exhibit patterns among the student population. Consequently, teachers often repeat their feedback and grades, bringing us to the idea of semiautomated feedback and assessment: by correcting handwritten tasks digitally, feedback can be saved and reused. This could lead to more elaborate feedback, time savings, and enhanced interrater reliability. Specifically, two semiautomatic assessment approaches were developed and studied.
In the first study, teachers write feedback for a student, and the computer saves it so that it can be reused when subsequent students make the same or similar mistakes. The concept of atomic feedback has been introduced to train teachers on how to write reusable feedback. Atomic feedback consists of a set of format requirements for mathematical feedback items, which has been shown to increase the reusability of feedback. A remarkable result was discovered during a crossover experiment with 45 mathematics teachers: the semiautomated approach led teachers to give significantly more feedback instead of saving time. Moreover, the teachers’ feedback with the semiautomatic tool did not always have better properties than classic penandpaper feedback.
The second study was conducted in collaboration with the Flemish Exam Commission. Their traditional grading method of handwritten mathematics exams was transformed into a semiautomated one called ‘checkbox grading.’ Every assessor receives a list of checkboxes, and they must tick those that apply to the student’s solution. Dependencies between these checkboxes can be set to ensure all assessors take the same path down the grading scheme. The system automatically calculates the grade and results in atomic feedback giving a detailed insight into what went wrong and how the grade was obtained. The approach requires more time for assessors and did not enhance interrater reliability compared to the traditional method (did not make it worse either). However, the resulting transparency and students’ feedback were highly valued. Moreover, students could easily understand the resulting feedback, even the lowerperforming ones.
In the first study, teachers write feedback for a student, and the computer saves it so that it can be reused when subsequent students make the same or similar mistakes. The concept of atomic feedback has been introduced to train teachers on how to write reusable feedback. Atomic feedback consists of a set of format requirements for mathematical feedback items, which has been shown to increase the reusability of feedback. A remarkable result was discovered during a crossover experiment with 45 mathematics teachers: the semiautomated approach led teachers to give significantly more feedback instead of saving time. Moreover, the teachers’ feedback with the semiautomatic tool did not always have better properties than classic penandpaper feedback.
The second study was conducted in collaboration with the Flemish Exam Commission. Their traditional grading method of handwritten mathematics exams was transformed into a semiautomated one called ‘checkbox grading.’ Every assessor receives a list of checkboxes, and they must tick those that apply to the student’s solution. Dependencies between these checkboxes can be set to ensure all assessors take the same path down the grading scheme. The system automatically calculates the grade and results in atomic feedback giving a detailed insight into what went wrong and how the grade was obtained. The approach requires more time for assessors and did not enhance interrater reliability compared to the traditional method (did not make it worse either). However, the resulting transparency and students’ feedback were highly valued. Moreover, students could easily understand the resulting feedback, even the lowerperforming ones.
Original language  English 

Qualification  Doctor of Philosophy 
Awarding Institution 

Supervisors/Advisors 

Thesis sponsors  
Award date  6 Sept 2023 
Place of Publication  Antwerp 
Print ISBNs  9789057287978 
Electronic ISBNs  9789057287978 
Publication status  Published  6 Sept 2023 