Ecological Validity in Testing Language Comprehension. A New Coloring Task

Offline methods for investigating language comprehension can be divided into three main categories: Act out tasks, picture selection tasks and truth value judgement tasks. All these methods try to resolve the tension between ecological validity and controllability. Ecological validity requires that the circumstances and procedures of the test remain as natural as possible and resemble similar activities in daily life, while controllability aims to decrease possible noise and limit participants’ performance to a small and controllable set of responses. Act out tasks, which are high on ecological validity, are known to be too open ended and thus low on controllability, while PST and TVJT are low on ecological validity, as they require participants to explicitly evaluate or compare alternatives.
We present a new Coloring Task, in which participants are required to fill in objects in a coloring page based on instructions given by the experimenter. We claim that this new methods successfully balances the conflicting requirements of ecological validity and controllability as well as decreasing the processing costs.
We present experimental results which show that children perform closer to adult-like level with the new method.