We can struggle to maintain working relationships when our social group grows too large, but it seems that artificial intelligence models may not face the same limitation, suggesting thousands of AIs could work together to solve problems that humans can鈥檛.
The idea that there is a fundamental limit on how many people we can interact with dates back to the 1990s, when anthropologist noticed a link between the size of a primate鈥檚 brain and the typical size of its social group. Extrapolating to humans, he suggested that there is a limit on the number of relationships we can maintain, typically around 150, although it varies from person to person. Now, researchers have applied the idea of Dunbar鈥檚 number, as it is known, to AI models and found that the most powerful 鈥� those with the largest 鈥渂rains鈥� 鈥� are able to coordinate in larger groups of up to 1000.
Since speaking to large language models like ChatGPT can feel like talking to a human, at the University of Konstanz, Germany, and his colleagues wondered if these models also act like humans when 鈥渢alking鈥� to each other in groups. To investigate, they ran many copies of the same AI model at once, assigning each a random opinion on a binary problem with no obvious answer, such as which side of the road a brand new country should drive on. At each step of the experiment, they chose one copy at random and told it what opinion all the other models held and why, then asked if it would like to update its own. The researchers say that this is analogous to humans attempting to reach consensus in loose, disorganised social groups.
Advertisement
In a test with 5o copies of Claude 3 Opus or GPT-4 Turbo, two high-end AI models, the team found that the group reached consensus every time. Meanwhile, copies of smaller and less powerful models like Claude 3 Haiku and GPT-3.5 Turbo never reached consensus. The results show that although the models in each test are identical, there is no inherent mechanism to converge on agreement, at least until they become sufficiently capable.
The researchers then tried to find an upper limit on each model鈥檚 ability to reach consensus 鈥� their own version of Dunbar鈥檚 number. For some models, at a certain size of group, the time taken to reach consensus started to grow exponentially, with Llama 3 70b ending up with a Dunbar鈥檚 number of 50. But for other models, like GPT-4 Turbo, this ability never slowed down even once 1000 copies were cooperating. The researchers鈥� ability to run larger and larger experiments ran out before the AI model stopped reaching agreement.
鈥淚 was very surprised,鈥� says De Marzo. 鈥淏asically, with the computational resources we have and the money we have, we [were able to] simulate up to thousands of agents, and there was no sign at all of a breaking of the ability to form a community.鈥�
He says memory is key. While we may struggle to recall facts, faces and opinions at a certain point, AI is limited only by its hardware. 鈥淚f you鈥檙e in an assembly of 10,000 people, it doesn鈥檛 work, because you cannot really let everybody talk, can鈥檛 remember all the things that people said,鈥� says De Marzo.
Dunbar, who聽is currently working with Google to , believes that as models grow larger and more powerful, they will improve these so-called mentalising abilities, which are key to cooperation in humans.
鈥淪cientific breakthroughs require the ability to engage with other people and come up with new ideas as a result of trying to find consensus between different groups of people with different views,鈥� says Dunbar, and De Marzo鈥檚 work shows that AI models may be able to do this at scale. 鈥淚t certainly looks promising that they could get together a group of different opinions and come to a consensus much faster than we could do, and with a bigger group of opinions,鈥� he says.
at the University of Maryland, Baltimore County, says that AI models with a high Dunbar number may be able to reach consensus on a problem, but that doesn鈥檛 necessarily mean they will find a good solution. He believes that diversity is key to problem-solving, which is difficult in groups made up of the same AI model.
鈥淭he way that living organisms have solved this for as long as there have been living organisms is you have populations that differ in the way that they approach exploration,鈥� says Feldman. 鈥淚f you have this really high Dunbar number, and everybody coordinates really quickly because they鈥檙e basically the same thing, that鈥檚 a tool that you can apply, there鈥檚 nothing wrong with that, but it鈥檚 not a general solution [to wide-ranging problems]. The general solution is diversity. In egalitarian communities, everybody argues a lot, but they tend to get very accurate answers.鈥�
A larger issue is whether it even makes sense to talk about copies of an AI model as a group of distinct individuals, says聽 at the University of Edinburgh, UK. The models don鈥檛 understand what they are, how they are separate from other models or what the purpose of the experiment is, he says.
鈥淚t is, of course, capable of executing instructions 鈥� a very broad range of instructions 鈥� by generating what it thinks might be the best response on a very general level,鈥� says Rovatsos. 鈥淏ut presence of others isn鈥檛 treated any differently from saying 鈥榞ive me a pizza recipe鈥�. I think the most problematic thing one could infer is that their ability to capture social sense is stronger or weaker than that of humans.鈥�
arXiv
