A core part of CSRI research and development has been formulated and enabled through the POETICON project series. POETICON is an interdisciplinary European funded project in the field of Cognitive Systems and Robotics. It started in January 2008, continued as POETICON++ in January 2012 and is currently running its seventh year of focused research. POETICON started with an ambitious basic research objective: to explore and model the “poetics of everyday life”, i.e. the synthesis of sensorymotor representations and natural language in everyday human interaction. This is related to an old problem in Artificial Intelligence on how meaning emerges, which was approached from an embodied and enactive cognition perspective. Basic tools for language, vision and action parsing, were developed and the modeling of their integration dynamics was explored. Experimental research fed the technology development, while a humanoid platform was used for a proof of concept demonstration of what could be achieved in Cognitive Systems through real integration of individual cognitive modules.
POETICON++ builds on these preliminary results arguing that robots need natural language for controlled generalisation of learned behaviours and for creativity. Its main objective is the development of an innovative computational mechanism for robust generalisation of motor programs and visual experiences for robots that will utilise the hierarchical and generative nature of language for ‘indexing’ (labelling) sensorymotor experiences at different levels of abstraction. The mechanism will integrate natural language and visual action and object recognition tools with advanced manipulation and mobility skills, affordance-based self-exploration abilities and a bio-inspired action-language learning module for: (a) behaviour generation through verbal instruction, and (b) visual scene understanding by a humanoid.
The POETICON project series is coordinated by the CSRI Director and brings together an international team of PIs and their teams, including Prof. Yiannis Aloimonos (University of Maryland, USA), Prof. Giulio Sandini, Prof. Luciano Fadiga and Prof. Giorgio Metta (Italian Institute of Technology, Italy), Prof. Angelo Cangelosi (University of Plymouth, U.K.) and Prof. Jose Santos Victor (Istituto Superior Tecnico, Portugal).