Multilingual large language models (LLMs) seem to work better in English. These AIs are designed to respond to queries in multiple languages but they respond better if asked to translate the request into English first.
LLMs have become a key part of the artificial intelligence revolution since the release of ChatGPT by OpenAI in November 2022. But interacting with them is primarily done in English 鈥� an issue some developers have tried to overcome with the release of multilingual models. These models are trained on multiple languages at once, allowing them to make links between languages with a large amount of training data and those with less.
at the University of the Basque Country in Spain and his colleagues compared two methods for putting non-English language inputs into multilingual LLMs and getting a response.
Advertisement
The first method simply puts the query in its original language into the LLM and gets a response in the original language. The second method first asks the same model to translate the query from the original language into English, then poses the translated question to the LLM, before getting a response also in English.
The researchers tested seven LLMs, including the first version of LLaMA, Meta鈥檚 AI language model. For all seven, the second approach produced results that were between 0.3 and 3.5 percentage points more accurate than those generated by the first method.
鈥淲e were surprised that for multilingual models built with a focus on being as smooth as possible in other languages, it was better to translate into English,鈥� says Etxaniz. 鈥淭his tells us there鈥檚 a lot of work to do to make multilingual language models really multilingual,鈥� he says.
AI has a language representation problem, partly because the majority of the language spoken on the internet 鈥� which acts as a major source of training data for AI models 鈥� is English.
However, using LLMs to translate into English first won鈥檛 be practical in every scenario. 鈥淭hese methods are still reliant on machine-translating text which can contain errors and terms that language speakers don鈥檛 use,鈥� says at the Centre for Democracy & Technology in the US. 鈥淭his is of particular concern when these models are conducting extremely language and context-specific tasks like content moderation.鈥�
Reference:
arXiv
