Two AIs built by Meta can beat expert human players at the classic first world war strategy game Diplomacy. The AIs had to learn how to partly cooperate with other players, an element most other game-playing AIs don鈥檛 have to take into consideration.
鈥淭here鈥檚 a quite unique feature here of Diplomacy being collaborative and competitive at the same time, which makes it exciting,鈥� says at the University of Malta, who was not involved in the study.
Diplomacy involves up to seven players fighting for control of European land and water territories. To build an AI proficient in playing it, a team led by researchers at Facebook鈥檚 parent company Meta first tried a typical AI trial-and-error algorithm called reinforcement learning.
Advertisement
This technique has helped AI dominate games such as chess, Go and poker 鈥� but the team found that it is far less effective for Diplomacy. 鈥淸In] games involving cooperation, self-play alone no longer guarantees good performance when playing with humans,鈥� wrote the researchers in their paper.
The team improved the AI by adding an extra algorithm that modifies the self-trained AI鈥檚 gameplay decisions to more closely resemble human strategies while accounting for different opponent playing styles or game situations. This was achieved by training the AI on 46,000 online Diplomacy games played by humans.
The researchers organised an online tournament featuring 62 human players ranging from beginners to experts and entered two versions of the AI, each trained to respond to different types of players. Each AI was evaluated through 50 games involving one AI and six human players, with the humans being aware of an AI opponent鈥檚 presence but not being told which player was the AI.
The AIs finished the tournament with the top two average game scores among those who played more than two games in the tournament. They also ranked first and third among all tournament participants according to a common player rating system.
at New York University says that the results demonstrate how AIs will need to use human-like strategies to best cooperate with people. 鈥淚f you have a really strong bot that plays really weirdly, it鈥檚 not going to do well in the game with humans present,鈥� he says.
The version of Diplomacy in the tournament was simpler than the traditional board game, as it didn鈥檛 include negotiations among players. While removing that social complexity alters the game, players still have to cooperate by taking actions such as sending armies or fleets to reinforce another player鈥檚 assault on a territory. 鈥淚 think we can still learn a lot about AI-human coordination and collaboration even in this very simplified version of Diplomacy,鈥� says Yannakakis.
Despite the AI performing well against some veteran Diplomacy players, the Meta researchers cautioned that their study does not definitively demonstrate the AI鈥檚 capability to outperform the best human players. An even greater challenge would involve learning the full version of Diplomacy that includes open communication and negotiation.
鈥淚n the full Diplomacy game, there鈥檚 a lot of 鈥榞o out into the kitchen, shake hands, look each other in the eye, and phrase your words carefully鈥�,鈥� says Togelius.
Reference:
