The recent developments of very large language models (LLMs) has suggested that, when trained with a mixed language set of data, these models could learn multilingual representations useful in transfert learning scenarios, and could also perform multilingual natural language processing tasks, such as Machine Translation. In this talk I will discuss and may be challenge the multilingual capacities of these large LMs, based on a serie of experiments performed with openly accessible versions of these models, such as mBart, mT5, mGPT and BLOOM.