Math items about real-world content lower test-scores of students from families with low socioeconomic status

Marjolein Muskens; Willem E. Frankenhuis; Lex Borghans

doi:10.1038/s41539-024-00228-8

Math items about real-world content lower test-scores of students from families with low socioeconomic status

Marjolein Muskens^*
, Willem E. Frankenhuis
, Lex Borghans

^*Corresponding author for this work

Chair Denissen

Research output: Contribution to journal › Article › Academic › peer-review

Abstract

In many countries, standardized math tests are important for achieving academic success. Here, we examine whether content of items, the story that explains a mathematical question, biases performance of low-SES students. In a large-scale cohort study of Trends in International Mathematics and Science Studies (TIMSS)—including data from 58 countries from students in grades 4 and 8 (N = 5501,165)—we examine whether item content that is more likely related to challenges for low-SES students (money, food, social relationships) improves their performance, compared with their average math performance. Results show that low-SES students scored lower on items with this specific content than expected based on an individual’s average performance. The effect sizes are substantial: on average, the chance to answer correctly is 18% lower. From a hidden talents approach, these results are unexpected. However, they align with other theoretical frameworks such as scarcity mindset, providing new insights for fair testing.

Original language	English
Article number	19
Number of pages	12
Journal	npj Science of Learning
Volume	9
Issue number	1
DOIs	https://doi.org/10.1038/s41539-024-00228-8
Publication status	Published - 15 Mar 2024

Bibliographical note

Publisher Copyright:
© The Author(s) 2024.

Funding

We thank Ethan Young for valuable suggestions and comments. WF's contributions have been supported by the Dutch Research Council (016.155.195 and V1.Vidi.195.130), the James S. McDonnell Foundation (https://doi.org/10.37717/220020502), and the Jacobs Foundation (2017 1261 02). The funders played no role in study design, data collection, analysis and interpretation of data, or the writing of this manuscript.

Funders	Funder number
Nederlandse Organisatie voor Wetenschappelijk Onderzoek (Netherlands Organisation for Scientific Research)	016.155.195, V1.Vidi.195.130
James S. McDonnell Foundation
Jacobs Foundation	2017 1261 02

Access to Document

10.1038/s41539-024-00228-8Licence: CC BY

s41539-024-00228-8Final published version, 673 KBLicence: CC BY

Cite this

@article{7a025138ca14477ebda3914f3d861005,

title = "Math items about real-world content lower test-scores of students from families with low socioeconomic status",

abstract = "In many countries, standardized math tests are important for achieving academic success. Here, we examine whether content of items, the story that explains a mathematical question, biases performance of low-SES students. In a large-scale cohort study of Trends in International Mathematics and Science Studies (TIMSS)—including data from 58 countries from students in grades 4 and 8 (N = 5501,165)—we examine whether item content that is more likely related to challenges for low-SES students (money, food, social relationships) improves their performance, compared with their average math performance. Results show that low-SES students scored lower on items with this specific content than expected based on an individual{\textquoteright}s average performance. The effect sizes are substantial: on average, the chance to answer correctly is 18\% lower. From a hidden talents approach, these results are unexpected. However, they align with other theoretical frameworks such as scarcity mindset, providing new insights for fair testing.",

author = "Marjolein Muskens and Frankenhuis, \{Willem E.\} and Lex Borghans",

note = "Publisher Copyright: {\textcopyright} The Author(s) 2024.",

year = "2024",

month = mar,

day = "15",

doi = "10.1038/s41539-024-00228-8",

language = "English",

volume = "9",

journal = "npj Science of Learning",

issn = "2056-7936",

publisher = "Nature Publishing Group",

number = "1",

}

TY - JOUR

T1 - Math items about real-world content lower test-scores of students from families with low socioeconomic status

AU - Muskens, Marjolein

AU - Frankenhuis, Willem E.

AU - Borghans, Lex

PY - 2024/3/15

Y1 - 2024/3/15

N2 - In many countries, standardized math tests are important for achieving academic success. Here, we examine whether content of items, the story that explains a mathematical question, biases performance of low-SES students. In a large-scale cohort study of Trends in International Mathematics and Science Studies (TIMSS)—including data from 58 countries from students in grades 4 and 8 (N = 5501,165)—we examine whether item content that is more likely related to challenges for low-SES students (money, food, social relationships) improves their performance, compared with their average math performance. Results show that low-SES students scored lower on items with this specific content than expected based on an individual’s average performance. The effect sizes are substantial: on average, the chance to answer correctly is 18% lower. From a hidden talents approach, these results are unexpected. However, they align with other theoretical frameworks such as scarcity mindset, providing new insights for fair testing.

AB - In many countries, standardized math tests are important for achieving academic success. Here, we examine whether content of items, the story that explains a mathematical question, biases performance of low-SES students. In a large-scale cohort study of Trends in International Mathematics and Science Studies (TIMSS)—including data from 58 countries from students in grades 4 and 8 (N = 5501,165)—we examine whether item content that is more likely related to challenges for low-SES students (money, food, social relationships) improves their performance, compared with their average math performance. Results show that low-SES students scored lower on items with this specific content than expected based on an individual’s average performance. The effect sizes are substantial: on average, the chance to answer correctly is 18% lower. From a hidden talents approach, these results are unexpected. However, they align with other theoretical frameworks such as scarcity mindset, providing new insights for fair testing.

UR - https://www.scopus.com/pages/publications/85187939912

U2 - 10.1038/s41539-024-00228-8

DO - 10.1038/s41539-024-00228-8

M3 - Article

C2 - 38491021

AN - SCOPUS:85187939912

SN - 2056-7936

VL - 9

JO - npj Science of Learning

JF - npj Science of Learning

IS - 1

M1 - 19

ER -

Math items about real-world content lower test-scores of students from families with low socioeconomic status

Abstract

Bibliographical note

Funding

Access to Document

Other files and links

Fingerprint

Cite this