Analyzing Multimodal Interaction

Human interactions are multimodal in nature. From simple to complex forms of transferal of information, human beings draw on a multiplicity of communicative modes, such as intonation and gaze, to make sense of everyday experiences. Likewise, the learning process, either within traditional classrooms or Virtual Learning Environments, is shaped by learners’ perceptions of what is being communicated multimodally to them intentionally or not, and by the perceptible pedagogical affordances of the environment.

This paper examines the specific place of action and multimodal interaction within the learning process. It starts by defining learning and multimodal interaction. Next, it expands on an existing methodological framework for analyzing multimodal interaction in order to include affordances for learning and to visually map the central role of action to learning. Finally, it makes use of the reviewed methodological framework to analyze a video ethnographic study of interactions that take place within a graduate Design classroom.

Any view of learning reflects its underlying theories. In (he present study, it is assumed that learning is situated in particular socioculturel contexts, and it is the result of mediated experiences that are afforded (Gibson, 1986) or constrained by interactions with the situation (King et al., 2001). In this way, the possibilities and limitations for action in particular situations affect learning. Furthermore, learning takes place whenever and wherever the individual is receptive. It can have different purposes or intentions, which, according to King, Young, Drivere-Richmond and Schrader (2001), can be classified into: a) objective-driven learning, such as in instruction; b) non-objective driven learning, such as in exploration; and c) unintended learning. With regard to the relation between learning and multimodal interaction, it is possible to affirm that learning is woven with multimodal interaction. Discourse analysis studies in educational settings (Cazden, 2001; Adger, 2001; Mehan, 1979; Gumperz and Herasimrhuk, 1975) have been trying to uncover the way in which talk in school is unique, helping to explicate the actions in which learning is realized. The emphasis on the linguistic aspect of classroom interaction, however, fails to account for the multiple fused semiotic modalities that together, rather than separately, help extend the understanding of the learning that takes place.

Continue to read Abstract on Find Articles