DECOLLAGE

DECOLLAGE ("Deep Cognition Learning for Language Generation") is a research project funded by the European Research Council (ERC) (ERC Consolidator Grant) and coordinated by André Martins. It runs 2023-2028 within Instituto de Telecomunicações at Instituto Superior Técnico, and Unbabel, in Lisbon, Portugal.

Large-scale language models have led to impressive results in many NLP tasks, exhibiting transfer and few-shot learning capabilities. When interacting with such systems, users commonly find them capable of reasoning, planning, and explaining their decisions, often in convincing ways. However, despite the enormous advances in the last years, current deep learning models for NLP are still very limited in fundamental ways and many important ingredients are still missing to achieve a satisfactory level of "intelligence". Some of these limitations partly stem from their monolithic architectures, which are good for some perceptual tasks, but unsuitable for tasks requiring higher-level cognition.

The overarching goal of DECOLLAGE is to attack these fundamental problems by bringing together tools and ideas from machine learning, sparse modeling, information theory, and cognitive science, in an interdisciplinary approach.

Three research directions are:
1. Designing new components for utility guidance, control, and contextualization. This will endow the model with the ability to predict its own quality (an "inner voice" or a critic) and to handle contextual information (e.g. document-level, conversation-level, meta-information about the surrounding environment), doing so in a modular, selective, and efficient manner.
2. Developing dynamic memory structures that facilitate continual learning, by supporting efficient reading and writing access, fast adaptation, and representation of world and self-knowledge. We will exploit synergies with sparse modeling and information retrieval.
3. Formalizing and implementing new mathematical models for sparse communication, bridging the gap between discrete (symbolic) and continuous representations, and developing techniques to integrate multiple modalities (such as text, speech, and image signals) into a shared representation space. This will draw links between information theory, formal languages, and neuroscience.

We will apply these innovations to highly challenging language generation tasks, including machine translation and open-ended generation.

See here for more information about the SARDINE Lab and some of our recent publications.