When soldiers are teamed with robots, the human need to interfere may negate the benefits of robotic assistance, a new US military project has discovered. But letting military artificial intelligence proceed without human supervision raises troubling ethical questions.
The project foresees a team of around 200 to 300 soldiers augmented with swarms of small drones and robotic ground vehicles. The lightly equipped unit would fight in zones where the enemy controls the airspace and yet be able to defeat enemy forces 鈥渙verwhelmingly superior in size and armament鈥�.
While a parallel US Defense Advanced Research Projects Agency (DARPA) programme, (OFFSET), is developing robot hardware, SESU is concentrating on the command and control of large numbers of robots. Rather than being operated individually, as is the case with most current drones, SESU robots will have AI and be largely autonomous.
Advertisement
Researchers described the results of recent virtual simulations at the US Army Futures Command Conference in Washington DC in October.
鈥淚t鈥檚 very interesting to watch how the AI discovers, on its own, some very tricky and interesting tactics,鈥� said a US Army scientist, speaking on condition of anonymity. 鈥淥ften you say, 鈥極h whoa, that鈥檚 pretty smart. How did it figure out that one?鈥欌��
However, the robots were impeded by human operators who may not have understood their actions.
鈥淲hat we found, as we ran the simulations, was that the humans constantly want to interrupt them,鈥� said the scientist. This interference could have a serious effect on the outcome, they said, leading to the stark conclusion: 鈥淚f we slow the AI to human speed鈥� we鈥檙e going to lose.鈥�
This is the first time researchers have discussed SESU, for which DARPA awarded more than $45 million to contractors Raytheon, Northrop Grumman and Collins Aerospace earlier this year. Current Pentagon policy calls for lethal military AI to be under meaningful human control, but the findings suggest this is at odds with military effectiveness.
Fast, tactical decision-making is a key advantage of AI, says Robert Bunker at US consultancy firm C/O Futures, who earlier this year on effective control of armed robots. Making rapid decisions could yield easy victories against slower opponents, he says, similar to those won by sides able to outmanoeuvre opponents in earlier wars.
鈥淎 prime example of this competitive process is the German blitzkrieg launched against France in 1940,鈥� says Bunker.
Stuart Russell at the University of California, Berkeley, who has long campaigned against autonomous weapons on ethical grounds, says the findings look like an attempt to justify using them.
鈥淚t points to the slippery slope whereby partial autonomy and human-on-the-loop and partial human oversight and so on will evaporate almost immediately under the pressure of war, and militaries will go straight to full autonomy if they can,鈥� he says.
Russell believes the research highlights the need for legal controls on autonomous weapons. This wouldn鈥檛 necessarily mean a total ban, but might confine autonomy to some situations.
鈥淵ou could allow full autonomy only for undersea warfare, or you could allow full autonomy for large-scale weapons 鈥� tanks, fighter aircraft, ships 鈥� while banning small anti-personnel swarm weapons,鈥� says Russell.
Even this may lead to a difficult balance between effective military robots and ethical ones.
鈥淵ou鈥檙e damned if you use them, and damned if you don鈥檛,鈥� says Bunker.
