Designing an instrument to measure the development of techno-mathematical literacies in an innovative mathematics course for future engineers in STEM education

Nathalie van der Wal; Arthur Bakker; Paul Drijvers

doi:10.1007/s11858-023-01507-1

Designing an instrument to measure the development of techno-mathematical literacies in an innovative mathematics course for future engineers in STEM education

Nathalie van der Wal^*, Arthur Bakker, Paul Drijvers

^*Corresponding author for this work

Research output: Contribution to journal › Article › Academic › peer-review

Abstract

Techno-mathematical Literacies (TmL), which are defined as a combination of mathematical, workplace and ICT knowledge, and communicative skills, are acknowledged as important learning goals in STEM education. Still, much remains unknown about ways to address them in teaching and to assess their development. To investigate this, we designed and implemented an innovative course in applied mathematics with a focus on Techno-mathematical Literacies for 1st-year engineering students, and we set out to measure the learning effect of the course. Because measuring TmL is an uncharted terrain, we designed tests that could serve as pre- or posttests. To prevent a test learning effect, we aimed to design two different but equally difficult tests A and B. These were assigned randomly to 68 chemistry students, as a pretest, with the other one serving as posttest after the course. A significant development in TmL was found in the B-pre group, but not in the A-pre group. Therefore, as a follow-up analysis we investigated whether the two tests were equally difficult and searched for possible explanations. We found that test B was indeed perceived as more difficult than test A, but also that students who were assigned B (pre) were previously higher achieving than A (pre), and a sound mastery level of basic skills that ground the higher-order TmL seemed necessary. Furthermore, as TmL are very heterogenous by nature, some of them are easier learned and measured than others. Based on the results, we propose ways of testing TmL, which should be validated in future research.

Original language	English
Pages (from-to)	1243–1254
Number of pages	12
Journal	ZDM - Mathematics Education
Volume	55
Issue number	7
Early online date	29 Jun 2023
DOIs	https://doi.org/10.1007/s11858-023-01507-1
Publication status	Published - Dec 2023

Bibliographical note

Funding Information:
This work is part of the research program Doctoral Grant for Teachers, with project number 023.009.061, which is financed by the Netherlands Organisation for Scientific Research (NWO). We would like to thank Amina Mokhtar, Jan Roelofs, Joost Giesbers, Ehsan Lakzaei, and Vlad Svarnovics, students from Electrical Engineering at Fontys University of Applied Sciences, for their creative work in designing test items. We would also especially like to thank Edwin Melis, colleague at Avans University of Applied Sciences, for his support and help in conducting the tests.

Publisher Copyright:
© 2023, The Author(s).

Funding

This work is part of the research program Doctoral Grant for Teachers, with project number 023.009.061, which is financed by the Netherlands Organisation for Scientific Research (NWO). We would like to thank Amina Mokhtar, Jan Roelofs, Joost Giesbers, Ehsan Lakzaei, and Vlad Svarnovics, students from Electrical Engineering at Fontys University of Applied Sciences, for their creative work in designing test items. We would also especially like to thank Edwin Melis, colleague at Avans University of Applied Sciences, for his support and help in conducting the tests. This work is part of the research program Doctoral Grant for Teachers, with project number 023.009.061, which is financed by the Netherlands Organisation for Scientific Research (NWO). We would like to thank Amina Mokhtar, Jan Roelofs, Joost Giesbers, Ehsan Lakzaei, and Vlad Svarnovics, students from Electrical Engineering at Fontys University of Applied Sciences, for their creative work in designing test items. We would also especially like to thank Edwin Melis, colleague at Avans University of Applied Sciences, for his support and help in conducting the tests.

Funders	Funder number
Edwin Melis
Fontys University of Applied Sciences
Nederlandse Organisatie voor Wetenschappelijk Onderzoek
Laurea University of Applied Sciences

Keywords

Assessment
Design research
Mathematics education
STEM education
Techno-mathematical literacies
Validation

Access to Document

10.1007/s11858-023-01507-1Licence: CC BY

s11858-023-01507-1 Final published version, 984 KBLicence: CC BY

Cite this

@article{4ef1e273acf143b7b1c067fe8aa9da35,

title = "Designing an instrument to measure the development of techno-mathematical literacies in an innovative mathematics course for future engineers in STEM education",

abstract = "Techno-mathematical Literacies (TmL), which are defined as a combination of mathematical, workplace and ICT knowledge, and communicative skills, are acknowledged as important learning goals in STEM education. Still, much remains unknown about ways to address them in teaching and to assess their development. To investigate this, we designed and implemented an innovative course in applied mathematics with a focus on Techno-mathematical Literacies for 1st-year engineering students, and we set out to measure the learning effect of the course. Because measuring TmL is an uncharted terrain, we designed tests that could serve as pre- or posttests. To prevent a test learning effect, we aimed to design two different but equally difficult tests A and B. These were assigned randomly to 68 chemistry students, as a pretest, with the other one serving as posttest after the course. A significant development in TmL was found in the B-pre group, but not in the A-pre group. Therefore, as a follow-up analysis we investigated whether the two tests were equally difficult and searched for possible explanations. We found that test B was indeed perceived as more difficult than test A, but also that students who were assigned B (pre) were previously higher achieving than A (pre), and a sound mastery level of basic skills that ground the higher-order TmL seemed necessary. Furthermore, as TmL are very heterogenous by nature, some of them are easier learned and measured than others. Based on the results, we propose ways of testing TmL, which should be validated in future research.",

keywords = "Assessment, Design research, Mathematics education, STEM education, Techno-mathematical literacies, Validation",

author = "\{van der Wal\}, Nathalie and Arthur Bakker and Paul Drijvers",

note = "Funding Information: This work is part of the research program Doctoral Grant for Teachers, with project number 023.009.061, which is financed by the Netherlands Organisation for Scientific Research (NWO). We would like to thank Amina Mokhtar, Jan Roelofs, Joost Giesbers, Ehsan Lakzaei, and Vlad Svarnovics, students from Electrical Engineering at Fontys University of Applied Sciences, for their creative work in designing test items. We would also especially like to thank Edwin Melis, colleague at Avans University of Applied Sciences, for his support and help in conducting the tests. Publisher Copyright: {\textcopyright} 2023, The Author(s).",

year = "2023",

month = dec,

doi = "10.1007/s11858-023-01507-1",

language = "English",

volume = "55",

pages = "1243–1254",

journal = "ZDM - Mathematics Education",

issn = "1863-9690",

publisher = "Springer Verlag",

number = "7",

}

Designing an instrument to measure the development of techno-mathematical literacies in an innovative mathematics course for future engineers in STEM education. / van der Wal, Nathalie; Bakker, Arthur ; Drijvers, Paul.
In: ZDM - Mathematics Education, Vol. 55, No. 7, 12.2023, p. 1243–1254.

Research output: Contribution to journal › Article › Academic › peer-review

TY - JOUR

T1 - Designing an instrument to measure the development of techno-mathematical literacies in an innovative mathematics course for future engineers in STEM education

AU - van der Wal, Nathalie

AU - Bakker, Arthur

AU - Drijvers, Paul

N1 - Funding Information: This work is part of the research program Doctoral Grant for Teachers, with project number 023.009.061, which is financed by the Netherlands Organisation for Scientific Research (NWO). We would like to thank Amina Mokhtar, Jan Roelofs, Joost Giesbers, Ehsan Lakzaei, and Vlad Svarnovics, students from Electrical Engineering at Fontys University of Applied Sciences, for their creative work in designing test items. We would also especially like to thank Edwin Melis, colleague at Avans University of Applied Sciences, for his support and help in conducting the tests. Publisher Copyright: © 2023, The Author(s).

PY - 2023/12

Y1 - 2023/12

N2 - Techno-mathematical Literacies (TmL), which are defined as a combination of mathematical, workplace and ICT knowledge, and communicative skills, are acknowledged as important learning goals in STEM education. Still, much remains unknown about ways to address them in teaching and to assess their development. To investigate this, we designed and implemented an innovative course in applied mathematics with a focus on Techno-mathematical Literacies for 1st-year engineering students, and we set out to measure the learning effect of the course. Because measuring TmL is an uncharted terrain, we designed tests that could serve as pre- or posttests. To prevent a test learning effect, we aimed to design two different but equally difficult tests A and B. These were assigned randomly to 68 chemistry students, as a pretest, with the other one serving as posttest after the course. A significant development in TmL was found in the B-pre group, but not in the A-pre group. Therefore, as a follow-up analysis we investigated whether the two tests were equally difficult and searched for possible explanations. We found that test B was indeed perceived as more difficult than test A, but also that students who were assigned B (pre) were previously higher achieving than A (pre), and a sound mastery level of basic skills that ground the higher-order TmL seemed necessary. Furthermore, as TmL are very heterogenous by nature, some of them are easier learned and measured than others. Based on the results, we propose ways of testing TmL, which should be validated in future research.

AB - Techno-mathematical Literacies (TmL), which are defined as a combination of mathematical, workplace and ICT knowledge, and communicative skills, are acknowledged as important learning goals in STEM education. Still, much remains unknown about ways to address them in teaching and to assess their development. To investigate this, we designed and implemented an innovative course in applied mathematics with a focus on Techno-mathematical Literacies for 1st-year engineering students, and we set out to measure the learning effect of the course. Because measuring TmL is an uncharted terrain, we designed tests that could serve as pre- or posttests. To prevent a test learning effect, we aimed to design two different but equally difficult tests A and B. These were assigned randomly to 68 chemistry students, as a pretest, with the other one serving as posttest after the course. A significant development in TmL was found in the B-pre group, but not in the A-pre group. Therefore, as a follow-up analysis we investigated whether the two tests were equally difficult and searched for possible explanations. We found that test B was indeed perceived as more difficult than test A, but also that students who were assigned B (pre) were previously higher achieving than A (pre), and a sound mastery level of basic skills that ground the higher-order TmL seemed necessary. Furthermore, as TmL are very heterogenous by nature, some of them are easier learned and measured than others. Based on the results, we propose ways of testing TmL, which should be validated in future research.

KW - Assessment

KW - Design research

KW - Mathematics education

KW - STEM education

KW - Techno-mathematical literacies

KW - Validation

UR - http://www.scopus.com/inward/record.url?scp=85163651956&partnerID=8YFLogxK

U2 - 10.1007/s11858-023-01507-1

DO - 10.1007/s11858-023-01507-1

M3 - Article

SN - 1863-9690

VL - 55

SP - 1243

EP - 1254

JO - ZDM - Mathematics Education

JF - ZDM - Mathematics Education

IS - 7

ER -

Designing an instrument to measure the development of techno-mathematical literacies in an innovative mathematics course for future engineers in STEM education

Abstract

Bibliographical note

Funding

Keywords

Access to Document

Other files and links

Fingerprint

Cite this