Embodied Language Processing

Embodied Language Processing at CSRI

Natural language processing does not take place in a cognitive vacuum, isolated from perception and action. Contrary to the traditional approaches of computational language analysis and generation that operate in a 'language-only' space, we introduce a new theoretical and computational look at language as an active system in multimodal cognition applications. We develop the first suite of embodied language processing tools, and new enactive lexicons that take state of the art research closer to experimental findings on how the human brain works. Our tools aim at bridging the gap between natural language and the sensorimotor space, allowing intelligent systems to go beyond using language as an interface medium, to taking full advantage of its potential for behavior generalization, creativity and intention attribution. In doing so, we bring the notion of referentiality at the core of language analysis, because it holds a key-role for its interaction with perception, the motor system and generalization and learning in semantic memory. We capitalize on fundamental mechanisms of the language system such as the productivity mechanisms of derivation and compounding, and the notion of irregularity; such mechanisms render natural language not just another symbol system, but a highly powerful one.