This call is now closed.

Natural language is not limited to the written modality. It includes and interacts with many others. On the one hand, a message can be conveyed through other language modalities, including audio (speech), gestures and facial expressions (sign language or completed speech). It may also be accompanied by social attitudes and non-verbal dimensions, including signs of affect, spontaneity, pathology, co-adaptation with dialogue participants, etc. Natural language processing (NLP) is thus a joint processing of multiple information channels. On the other hand, natural language is often used to describe concepts and denote entities that are essentially multimodal (description of an image, an event, etc.). Many problems then require bridges between different modalities.

The objective of this special issue of the journal TAL is to promote NLP in multimodal contexts (several modalities contribute to the resolution of a problem) or inter-modal contexts (passage from one modality to another). Thus, the contributions expected for this special issue are, among others (but not exclusively), in the following fields of application:

• multimodal dialogue, multimodal question-answering ;
• sign language, completed spoken language ;
• speech processing, automatic speech recognition, speech synthesis in multimodal contexts;
• synthesis of animated emotional agents;
• handwriting recognition and analysis of handwritten documents;
• understanding, translation and summarisation of multimodal documents;
• indexing, search and mining of multimedia and/or multimodal documents;
• biological signal processing, computational psychology or sociology, for NLP;
• inter-/multimodal human-computer interface for NLP;
• other multimodal or inter-modal applications (automatic image captioning, image-to-text generation, generation/analysis of songs and lyrics, etc.).

In the face of the predominance of work on written language processing and the historical compartmentalisation of communities specific to each modality (image processing, signal processing, neuroscience, etc.), authors are encouraged to highlight the specificities (benefits, difficulties, perspectives, etc.) linked to inter- or multimodality in their work, for instance concerning:

• understanding the interactions between modalities
• harmonisation or compatibility of representations
• the development of joint models or transfer from one modality to another;
• the constitution (or even annotation) of multimodal resources;
• etc.

IMPORTANT DATES (for information purposes only)

• Full paper submission deadline: 31 August 2022
• First reviews to the authors: mid-December 2022
  
• Rebuttal to the reviewers and revised version: 27 January 2023
  
• Second review and final decision: March 2023
  
• Publication: April-May, 2023
  

THE JOURNAL

TAL (Traitement Automatique des Langues / Natural Language Processing) is an international journal published by ATALA (French Association for Natural Language Processing, http://www.atala.org) since 1959 with the support of CNRS (National Centre for Scientific Research). It has moved to an electronic mode of publication, with printing on demand.