An Electrophysiological Investigation of Embodied Language Processing

Abstract

How we draw meaning from strings of letters is one of the most popular topics under discussion in cognitive science. Traditional theories posited that words represent amodal symbols and meaning is derived through their relationship to other amodal symbols (Galetzka, 2017). The problem with this cognitivist approach is that it was unclear how these symbols came to have meaning, known as the “grounding problem” (Harnad, 1990; Searle, 1980). Theories of embodied cognition were developed in an attempt to resolve this issue (Barsalou, 1999; Glenberg & Robertson, 1999). For example, the Indexical Hypothesis proposed that nouns are indexed to mental representations (such as mental pictures) of the objects they refer to (Glenberg & Robertson, 1999). Subsequently, when a noun is processed, the affordances (behavioural possibilities) of the referent object are made available (e.g., a mug affords being grasped by its handle). According to the Indexical Hypothesis, access to these affordances is crucial for noun comprehension (Glenberg & Gallese, 2012; Glenberg & Robertson, 1999). More recent theories of language comprehension posit that both amodal and embodied representations contribute to understanding (Barsalou, Santos, Simmons & Wilson, 2008; Louwerse, 2007; 2018; Louwerse & Jeuniaux, 2008). The present thesis discusses experiments using event-related potentials (ERPs), measured while participants read object names, in order to explore the timing of access to information related to these embodied representations of the referent objects. The first experiment revealed that the earliest information related to object affordance was available from 175 ms, soon after retrieval of lexical and semantic information begins (around 160 ms; Hauk, Coutout, Holden & Chen, 2012). Our second study revealed that functionally manipulable objects have a richer semantic representation, compared to objects that are graspable based purely on their geometric properties. Semantic processing of functionally manipulable objects incorporates knowledge about the actions associated with object use, perceptual information related to the object, and the specific motor programs necessary for manually manipulating the object. Actions associated with the use of functionally manipulable objects were accessed from as early as 190 ms. Affordances based on the geometric properties of an object were available from 224 ms and reflected pattern-matching between semantic information about the object’s size and shape, accessed from processing the object name, with proprioceptive information provided by the participant’s body, about the size and shape of the object they were holding during the experiment. Pulvermüller (1999) argued that sensorimotor representations of language are developed during our personal experiences in the world, through Hebbian learning. The findings of our final experiment supported this argument, indicating that the conceptual representation of objects, and the actions associated with their use, were developed during the participants’ previous experience of using the object.