embodied language processing 

Natural language processing does not take place in a cognitive vacuum, isolated from perception and action. Contrary to the traditional approaches of computational language analysis and generation that operate in a ‘language-only’ space, we advocate for a new theoretical and computational look at language as an active system in multimodal cognition applications.

     Our ultimate objective in this research line is the development of such new description of language and of a corresponding suite of embodied language processing tools, and new enactive lexicons that take state of the art research closer to experimental findings on how the human brain works. The aim is to bridge the gap between natural language and the sensorimotor space, allowing intelligent systems to go beyond using language as an interface medium, to taking full advantage of its potential for behavior generalization, creativity and intention attribution.

    We believe that key for the attainment of this objective is to bring the notion of referentiality at the core of language analysis, because it holds a core-role for its interaction with perception, the motor system and generalization and learning in semantic memory. Furthermore, one needs to capitalize on fundamental mechanisms of the language system such as the productivity mechanisms of derivation and compounding, and the notion of irregularity; such mechanisms render natural language not just another symbol system, but a highly powerful one.     

     In the framework of the POETICON projects, we have developed a number of proof of concept prototypes, which actually automate the transition from a traditional syntacticosemantic analysis of language to what is actually meant and/or implied. In these tools we employ the Minimalist Grammar of Action for language analysis and create PRAXICON-like representations, where implied information is solved through prior common sense knowledge reasoning and generalisation. We are currently extending this research and correlate it to Generative AI techniques for two specific applications:  
(1) Robotic Culinary Assistants: human-robot interaction in the home or professional kitchen, focusing on verbal instruction analysis, &
(2) Multimodal Deception Detection in real, high stake criminal cases,  focusing on cases of public/media deception.