Language. Robots. Intelligence
language within the “Robot percieving, Learning, doing” Loop
Our research lab runs within the Institute for Language and Speech Processing (ILSP), Athena Research Center. It is headed by Dr. Katerina Pastra and comprises a team of researchers with diverse backgrounds and seniority. The lab has a highly interdisciplinary specialization involving Natural Language Processing and Knowledge Representation & Reasoning for Cognitive Systems and Robotics. Its core activities comprise theoretical, experimental and computational research and development for exploration and modeling of fundamental mechanisms in human language and multimodal cognition, enabling thus the integration of language, perception and action in Artificial Intelligence systems.
- to develop cognitive systems/robots with human-level intelligence for the betterment of our everyday life
- to foster highly interdisciplinary research integrating methods, findings, skills and competences from Engineering/Computer Science, Humanities/Linguistics, Life Sciences/Neuroscience and Cognitive Psychology;
- to develop human-centric and biology inspired intelligent technology that addresses societal needs and improves the quality of life;
- to establish language as a central component in the “robot/agent perceiving, robot/agent doing” research landscape;
- to bring science to society through transfer of interdisciplinary knowledge and research practices to students of all levels of education and the general public.
- we promote scientific excellence and integrity;
- we encourage initiative, innovation, and creativity;
- we believe in active collaboration, continuous learning, and interdisciplinary exploration;
- we consider the development of systems with human-level intelligence feasible primarily through exploration of human cognition;
- we view intelligent technology as a platform for exploring how the human brain works and a means for addressing grand societal challenges;
- we actively support the open-source movement providing our software and data resources freely to the public;
- we abide to diversity, equity and inclusion principles and strive for their wide adoption
Our Story Highlights
- “The Language of Caricatures” (1999), BA thesis, University of Athens, Philology Department, Linguistics Division / revisited in “Image-Language Dialectics in Greek Caricatures“, Latsis Award (2009)
- British Computer Society Paper Award on “Intelligent Indexing of Crime-Scene Photographs” (Multimodal Forensics Application) (2002), University of Sheffield, Department of Computer Science
- “Vision-Language Integration as a Double Grounding Case“, PhD thesis, University of Sheffield, Department of Computer Science (2005)
- COSMOROE – cross media semantic interplay published (2008)
- Launch of the European research & development project POETICON, Athena R.C. Coordinator (2008)
- Cognitive Systems Research Institute (CSRI) founded (2011)
- “Embodied Language Processing: A new generation of language technology” published (2011)
- “Minimalist Grammar of Action” published (2012)
- Launch of the European research & development project POETICON Plus Plus, Athena R.C & CSRI Coordinator (2012)
- Launch of the COSMOROE search engine (2015)
- Release of video demos of end-to-end iCub integrated cognitive architecture in action: (a) from verbal instruction to task execution, and (b) visual scene intention prediction (2016)
- Nature Paper on “Multimodal object affordances” published & related large scale dataset of spontaneous speech and actions released (2016)
- Integration of CSRI to Athena R.C. (2021)
- Launch of the Course “Language and Cognitive Robotics“, in the “Language Technology” Master Programme, University of Athens & Athena R.C. (2021)
- Launch of the Language and Robots Lab (2022)
Language interacts closely with Perception and the Motor system, and the dynamics of such interaction feed and are fed -among others- by the Semantic Memory; the distributed and associative nature of these dynamics regulate Learning and Reasoning processes. Our research in this direction aims at modeling the autonomous, developmental acquisition of sensorimotor experiences and symbols. We contribute theory, tools and experimental methodologies for exploring and modeling such processes, comprising large-scale semantic memory modules, embodied lexicons, common sense reasoners, and cognitive semantic similarity metrics. The intelligent technology developed along these lines has a wide range of applications, including Visual Scene Understanding, Multimodal Discourse Analysis and Generation, Audiovisual indexing, Retrieval and Summarization for Big Data Processing.
Embodied Language Processing
Natural language processing does not take place in a cognitive vacuum, isolated from perception and action. Contrary to the traditional approaches of computational language analysis and generation that operate in a ‘language-only’ space, we introduce a new theoretical and computational look at language as an active system in multimodal cognition applications. We develop the first suite of embodied language processing tools, and new enactive lexicons that take state of the art research closer to experimental findings on how the human brain works. Our tools aim at bridging the gap between natural language and the sensorimotor space, allowing intelligent systems to go beyond using language as an interface medium, to taking full advantage of its potential for behavior generalization, creativity and intention attribution. In doing so, we bring the notion of reference at the core of language analysis, because it holds a key-role for its interaction with perception, the motor system and generalization and learning in semantic memory. We capitalize on fundamental mechanisms of the language system such as the productivity mechanisms of derivation and compounding, and the notion of irregularity; such mechanisms render natural language not just another symbol system, but a highly powerful one.
Human perception is multisensory and multimodal. We focus our experimental research on the role of language in multisensory object and event perception. This involves research on language modulated perception of object saliency and attention, co-speech exploration of object affordances through active touch, and verbal acquisition of the motoric ‘how to’ in action observation. Our contributions provide ground for establishing an emerging research direction that incorporates language dynamically into the exploration of multisensory perception. We incorporate the corresponding findings in the development of intelligent artificial agents, bridging thus experimental research and technology development.