Situation Models and Embodied Language Processes
U. Osnabrueck, Germany
This course will be focussed on mental models or situation models and their role in memory and language understanding processes. Situation models represent the state of affairs of some circumscribed world situation in the human mind. Situations are assumed to be encoded by perceptual symbols (Barsalou, 1999) which constitute indices to the objects and structures of the external world. They may be formed by direct perception as well as by language or text understanding. Situation models form a central concept in theories of situated cognition and may guide studies about the embodiment of mental representations and processes.
After an introduction to the traditional experimental paradigms on memory and language, the pivotal role of situation models in human cognition will be discussed. A particular focus will be on inference processes which contribute to build situation models from texts and how computational theories, such as Kintsch`s (1998) construction-integration theory account for different aspects of experimental findings. It will be argued that the goal of using and modifying a unified theory is very promising. Thereby, computational modelling techniques are applied to combine the insights obtained from behavioural studies and the results about neural correlates of cognitive processes. After an explanation of event-related potentials and functional Magnetic Resonance Imaging, several experiments will be presented and it will be discussed which difficulties arise when a variety of different measures are to be integrated into a model. This course will thus provide insights on how general frameworks of cognition, computational modelling, behavioural and neuroscience experiments are related to each other in the realm of memory and language processes.
Day 1: Introduction to situation models - PowerPoint Presentation
Zwaan, R. A. & Radvansky, G. A. (1998) Situation models in language comprehension and memory. Psychological Bulletin, 123 (2) 162-185.
Day 2: Computational modelling of inference processes - PowerPoint Presentation
Schmalhofer, F., McDaniel, M.A & Keefe, D. (2002) A unified model of predictive and bridging inferences. Discourse Processes. 33 (2), 105-132.
Frank, S.L., Koppen, M., Noordman, L.G., & Vonk, W. (2003) Modeling knowledge-based inferences in story comprehension. Cognitive Science, 27, 875-910.
Graesser, A. C., Singer, M., Trabasso, T. (1994) Constructing inferences during narrative text comprehension. Psychological Review, 101, 371-395.
Schmalhofer, F. (1998) Constructive knowledge acquisition: A computational model and experimental evaluation. Mahwah, N.J. Chapter 3: The levels approach toward cognitive modelling. pp 49-68
Day 3: What memory and language are for - PowerPoint Presentation
Zwaan, R. A. (in press) The Immersed Experiencer: Toward an Embodied Theory of Language Comprehension. To appear in B.H. Ross (Ed.) The Psychology of Learning and Motivation, Vol. 44. New York: Academic Press.
Barsalou, L. W. (1999) Perceptual Symbol Systems. Behavioral and Brain Sciences, 22, 577-660.
Glenberg, A.M. & Kaschak, M. P. (2002) Grounding language in action. Psychonomic Bulletin & Review, 9, 558-565.
Zwaan, R. A., Stanfield, R.A. & Yaxley, R. H. (2002) Language comprehenders mentally represent the shapes of objects. Psychological Science. 13(2), 168-171.
Day 4: Neural correlates of text comprehension - PowerPoint Presentation
Ferstl, E. C. & von Cramon, D. Y. (2001) The role of coherence and cohesion in text comprehension: an event-related fMRI study. Cognitive Brain Research, 11, 325-340.
Perfetti, C. A. (1999) Comprehending written language: A blueprint of the reader. In C. M. Brown & P. Hagoort (Eds) The neurocognition of language processing (pp. 167-208) Oxford University Press.
Gazzaniga, M.S., Ivry, R. B. & Mangun, G. R. (2002) Cognitive Neuroscience: The biology of the mind. New York: W.W. Norton & Company, Chapter 4: The methods of cognitive neuroscience (in particular pp 129-147)
Schmalhofer, F., Raabe, M., Friese, u., Pietruska, K., & Rutschmann, R. (2004) Evidence from an fMRI experiment for the minimal encoding and subsequent substantiation of predictive inferences. 1-page abstract.
Day 5: The integration of behavioural experiments, computational modelling and neural correlates - PowerPoint Presentation
Anderson, J. R., Qin, Y., Sohn, M-H, Stenger, V. A. & Carter, C. S. (2003) An information-processing model of the BOLD response in symbol manipulation tasks. Psychonomic Bulletin & Review. 10(2) 241-261.
Franz Schmalhofer studied psychology, mathematics and computer science at Regensburg University/Germany and the University of Colorado at Boulder. He received a PhD from the University of Colorado in 1982 with a thesis on "The comprehension of a technical text as a function of expertise".From 1982 to 1989 he has held academic positions at the University of Heidelberg and the University of Freiburg and was assistant professor at the Cognitive Science Centre at McGill University/Montreal. From 1989 to 2000 he was a senior scientist at the German Research Center for Artificial Intelligence (DFKI) in Kaiserslautern and a lecturer in psychology at the University of Heidelberg. In 2000, he became professor of cognitive psychology at the University of Osnabrueck. His current interests are in text comprehension, situation models, neural correlates of inferencing, computational modeling and how people develop shared situation models.