Quantifying the informational value of classification images

Loek Brinkman*, Stanny Goffin, Rens van de Schoot, Neeltje E.M. van Haren, Ron Dotsch, Henk Aarts

*Corresponding author for this work

Research output: Contribution to journalArticleAcademicpeer-review


Reverse correlation is an influential psychophysical paradigm that uses a participant’s responses to randomly varying images to build a classification image (CI), which is commonly interpreted as a visualization of the participant’s mental representation. It is unclear, however, how to statistically quantify the amount of signal present in CIs, which limits the interpretability of these images. In this article, we propose a novel metric, infoVal, which assesses informational value relative to a resampled random distribution and can be interpreted like a z score. In the first part, we define the infoVal metric and show, through simulations, that it adheres to typical Type I error rates under various task conditions (internal validity). In the second part, we show that the metric correlates with markers of data quality in empirical reverse-correlation data, such as the subjective recognizability, objective discriminability, and test–retest reliability of the CIs (convergent validity). In the final part, we demonstrate how the infoVal metric can be used to compare the informational value of reverse-correlation datasets, by comparing data acquired online with data acquired in a controlled lab environment. We recommend a new standard of good practice in which researchers assess the infoVal scores of reverse-correlation data in order to ensure that they do not read signal in CIs where no signal is present. The infoVal metric is implemented in the open-source rcicr R package, to facilitate its adoption.

Original languageEnglish
Pages (from-to)2059-2073
Number of pages15
JournalBehavior Research Methods
Issue number5
Publication statusPublished - Oct 2019


  • Data quality
  • Online experiments
  • Reliability
  • Reverse correlation
  • Social psychology


Dive into the research topics of 'Quantifying the informational value of classification images'. Together they form a unique fingerprint.

Cite this