(Image: Renaud Vigourt)
When robots fight in place of soldiers, will wars have fewer casualties? Or will the world slide into non-stop conflict?
QANDI AGHA used to be a cashier in Afghanistan’s Ministry of Culture. But he claims that in 2012 he was arrested by an elite US Special Forces unit and tortured for six and a half weeks. He was held under water until he felt like he was dying, says Joanne Mariner, an expert in humanitarian law for Amnesty International. “They also tied a cord around his penis so he couldn’t urinate,” she says. “They left it on for four days.”
Advertisement
Agha’s torture is one of many such cases recorded by Mariner for a recent . Demonstrations of inhumanity by trained soldiers are not uncommon, from the infamous abuse of prisoners at Baghdad’s Abu Ghraib to the . “Obviously this doesn’t go anywhere, fellas… I’ve just broken the Geneva Convention,” the marine said after pulling the trigger. “When you start getting into the business of killing people it takes a lot of care and control to make sure that you’re killing only the people that you’re legally allowed to kill,” says Mariner.
Could machines do better? For many, it’s an outrageous idea. But others not only think machines could, but that they must. “Humans are currently slaughtering other humans unjustly on the battlefield,” says Ronald Arkin, a robotics expert at the Georgia Institute of Technology in Atlanta. “I can’t sit idly by and do nothing. I believe technology can help.”
Unlike humans, robots won’t break the rules. For this reason, the argument goes, wars fought by machines would be more humane. And sending robots in place of human soldiers would also save lives, particularly for the nation possessing such advanced tech. So the development of so-called lethal autonomous weapons systems – or “killer robots” – is accelerating, with many of the world’s armies looking for ways to keep their soldiers out of the line of fire.
But there is strident opposition. This week, representatives of dozens of nations will gather at the UN headquarters in Geneva, Switzerland, to discuss whether the UN should consider a ban. The aim is to get killer robots classed alongside chemical weapons, and blinding lasers. With fully autonomous weapons a technical possibility, campaigners are keen to halt their development – and soon. But what’s at stake?
Robots currently play several roles on the battlefield. Some carry equipment, others dismantle bombs and provide surveillance. And remote-controlled drones let their operators control attacks on targets from thousands of kilometres away. The latest machines, though, take drones to the next level. Capable of selecting and engaging targets with little or no human intervention, the authorisation to open fire is sometimes all that remains under human control.
The US navy’s Phalanx anti-missile system on board its Aegis ships can perform its own “kill assessment” – weighing up the likelihood that a target can be successfully attacked. UK firm BAE is developing an uncrewed jet called Taranis. It can take off, fly to a given destination, and identify objects of interest with little intervention from ground-based human operators unless required. The jet is a prototype and carries no weapons, but it demonstrates the technical feasibility of such aircraft.
Meanwhile, Russia’s “mobile robotic complex” – an uncrewed tank-like vehicle that guards ballistic missile installations – and South Korea’s Super Aegis II gun turret are reportedly able to detect and fire on moving targets without any human supervision at all. The Super Aegis II can pinpoint an individual from 2.2 kilometres away.
Arms manufacturers don’t like to talk about the details. The specifics, in general, are classified. What’s clear, though, is that technology is no longer the limiting factor. “Technology is not the likely restriction as to what is feasible in the future,” says a spokesman for UK missile manufacturers . Instead, he says, autonomous weapons will be constrained by policy, not capability.
“The specifics are classified but it’s clear that technology is no longer the limiting factor”
What, then, are the relevant rules of war? , but all weapons must comply with existing conventions. One key principle is that civilians and civilian property must not be intentionally targeted. Weapons must also be capable of discriminating between civilians and soldiers. And the use of force must be proportional – the expected military advantage of an attack cannot be outweighed by collateral damage. “I would argue there is a robust body of law that applies to autonomous weapons systems,” says at Columbia Law School in New York. “Any weapons system that is developed has to be reviewed for its basic compliance.”
Within such a framework, Arkin believes that if a lethal autonomous system can be shown to be better than a human soldier at limiting civilian casualties, then a ban on such technology would be a mistake. “We have to remember the fallibility and frailty of humans in modern warfare,” he says. “If we can do better than them, we are saving lives.”
Others make the case just as strongly. Erik Schechter, a mathematician at Vanderbilt University in Nashville, Tennessee, , for example: “If the goal of international humanitarian law is to reduce non-combatant suffering in wartime, then using sharpshooting robots would be more than appropriate, it would be a moral imperative.”
Robots could spare soldiers’ lives, too. Instead of calling in an airstrike to bomb a suspected enemy base, which may be situated in a densely populated urban area, for example, robots could enter the building ahead of human soldiers, allowing them to take the initial risks. Where aspects of a mission are especially risky, machines could take the lead.
The right to dignity
The idea is deeply divisive, however. For many, the prospect of a computer chip having the power of life or death over someone is discomforting. According to , the UN’s Special Rapporteur on extrajudicial, summary or arbitrary executions, it could contravene humanitarian law and the human right to dignity.
“Humans need to be quite closely involved in the decision for it not to violate your human dignity,” says Heyns. He notes that a robot’s targets don’t have the option of an appeal to humanity the way they might if a person was behind the weapon. It would be like being exterminated, he says. Remote-controlled drones already give little opportunity to make such appeals. But they at least have a human operator, however distant, who can make ethical judgements. “The hope that this is possible is at least not completely absent,” says Heyns. “And hope is part of a dignified life.”
Heyns isn’t totally against the rise of autonomous systems, however. Imagine a team of snipers with their sights trained on members of a militant group such as Boko Haram, he says. Hostages are nearby, so the snipers all need a clean shot at the same time for the hostages to escape unharmed. As long as the decision to kill is taken by a human commander, the actual shots could be triggered by a computer controlling the rifles. That may be the only way to get each gun to fire at the exact moment all the snipers have a clean shot.
Ultimately, however, Heyns is wary of what he calls the “depersonalisation of force”. In a 2013 , he warned that “tireless war machines, ready for deployment at the push of a button” could lead to a future of permanent conflict. If governments don’t have to put boots on the ground, going to war could become too easy. Even in scenarios where machines fight machines, significant collateral damage could destroy a nation’s infrastructure. And as casualty numbers will be lower, wars could go on for longer, preventing post-war reconstruction.
“If governments don’t have to put boots on the ground, going to war could become too easy”
Fears of depersonalising war have convinced some that weaponised robots should be vetoed. “People die in wars,” says US Congressman . “And we ought to never get to the point where we forget how horrible that is.” Concerned that technologically advanced nations are distancing themselves from the reality of war, McGovern wants the US to take the lead in prohibiting autonomous weapons.
“I think people ought to be concerned about the impact of bombing, of drone attacks,” he says. “You take the human out of that, you desensitise people. I want people to know that war is awful and ugly and terrible.”
There’s substance to McGovern’s fear. Research suggests drone strikes against human targets may radicalise members of a population previously unconnected to the conflict, for . “It’s possible that the sense of injustice will be enhanced even more because the enemy you’re fighting is so disconnected from the conflict that it’s not even bothering to send soldiers,” says , an expert in terrorism at London’s Royal United Services Institute.
This summer, McGovern organised a briefing for US politicians by members of the – a boon for , one of the campaign’s leaders, who has been trying to bring the issue to international attention for nearly a decade.
Sharkey is an expert in artificial intelligence and robotics at the University of Sheffield, UK. A key driver of his dogged campaigning is his awareness of the shortcomings of current technology. While Arkin has an eye on next-generation tech, Sharkey is concerned with the present. So is Mariner. “A calculation is only as good as the data that goes into it,” she says. Can a drone tell the difference between a bearded taxi driver and a bearded Taliban member? “There’s no perfect computer that can do this assessment,” she says. Sharkey agrees. “I could make you a killer robot within weeks that detects human body signatures and fires at them,” he says. “The problem is being able to discriminate between a civilian and a combatant.”
Bottling that ability is hard. “It’s a very formidable problem,” says Glynn Wright, founder of , a UK-based firm that provides image-analysing software for security applications, “and it’s not solved by any means.” Aralia’s software can highlight suspicious activity in CCTV footage. A couple of years ago the system flagged up the activity of a group of people who were later found to have been scoping out a public area for a suitable place to plant a bomb. The individuals were apprehended and successfully prosecuted, says Wright. But he readily admits that making snap decisions in busy urban environments is a long way off.
Even if a machine was capable of such discrimination, what of its ability to make moral decisions based on such data? Arkin has argued that we could develop software that acts as an “ethical governor” to guide a robot’s response to various circumstances. The required complexity of such software, though, means that these proposals remain on the drawing board. For now, a human must remain in control.
Calling the shots
But what does that mean? The debate on killer robots turns on this question and the call for a ban will succeed or fail depending on what counts as having a human calling the shots (see “In the loop“). There is no accepted definition. And there is another issue: focusing only on the question of killer robots conceals more fundamental problems with the way humans and machines interact. Just because a human is in the loop doesn’t mean the problems with high-tech killing go away.
In 2003, for example, an operator at a US Patriot missile battery received an automated alert that an incoming Iraqi missile had been detected. She had a split second to make a decision and chose to take defensive action, authorising the battery to fire. But the target turned out to be an RAF Tornado jet whose two pilots were killed when the Patriot missile hit. What went wrong? The system certainly misidentified the jet. But an inquest found that this was because . As a result, the system hadn’t been connected to a wider network that would have told it that the jet was in contact with air traffic controllers and not a threat. In this case, having humans in the loop was arguably the root of the problem.
Whatever the outcome in the UN, the issues raised in the debate over autonomous weapons systems aren’t as simple as they first seem. Whether fully autonomous or not, machines are already part of war and the promise to keep people in the loop is certainly no guarantee that mistakes won’t happen. Machines don’t streamline war, they complicate it. It’s something they have in common with us.
Leader: “Automated killer robots come with enormous risks“
In the loop
The surge in drone warfare has raised the profile of remotely piloted military vehicles. Humans remain ostensibly “in the loop”, but that doesn’t mean there aren’t problems. And “in the loop” can mean anything from controlling almost every facet of a remotely piloted vehicle, to simply saying “yes” or “no” when asked if an automatically selected target should be destroyed.
Nancy Cooke is an expert in the psychological impact of drone piloting at Arizona State University in Tempe. Her contract with the US military is to consult on “human systems integration” – how well people are in control of the machines at their fingertips. For pilots of Predator and Reaper drones, that means the “cockpit” deep inside a military base that directs aircraft flying thousands of kilometres away.
Maintaining control
Cooke’s research helps identify aspects of the drone-piloting interface that could be improved, aspects of training that can be changed to benefit pilots, and situations in which social and psychological support is most important.
The whole aim is to put the human in fullest control, says Cooke – but that isn’t easy. Drone operators themselves don’t make kill decisions, though they carry out all the functions necessary to do so. With endless video footage of their activities rolling in front of their eyes, and orders constantly issued from a nearby commander, the effects of this aren’t trivial.
Cooke says operators describe themselves as “bus drivers” or feel roboticised, while their commanders load up on “Predator crack” – the video feed beamed back from the drones. The removal of people from the front line has created a strange situation, in which pilots are more acutely exposed to the effects of what they do, while feeling less in control overall, says Cooke.
“The video imagery is of body parts and injuries, fatalities and lots of destruction,” she explains. “What’s relayed to me, at least from operators, is that that’s really a horrible thing because it’s only too real, what you’ve just done. Unlike other cases where you’re just in and out and don’t really see what you did.”
Arms control from crossbows to cluster bombs
1139: Crossbow banned by Pope Innocent II for use against Christians
The pope said: “We prohibit under anathema that murderous art of crossbowmen and archers, which is hateful to God, to be employed against Christians and Catholics from now on.” The use of crossbows against non-Christians was of course acceptable.
1675: Strasbourg agreement
This outlawed the use of poison bullets.
1899: Hague convention
This included:
- Declaration concerning the Prohibition of the Discharge of Projectiles and Explosives from Balloons or by Other New Analogous Methods
- Declaration concerning the Prohibition of the Use of Projectiles with the Sole Object to Spread Asphyxiating Poisonous Gases
- Declaration concerning the Prohibition of the Use of Bullets which can Easily Expand or Change their Form inside the Human Body such as Bullets with a Hard Covering which does not Completely Cover the Core, or containing Indentations
All major powers at the time ratified the prohibition of aerial bombing, except the UK and the US. This ban expired in 1905.
1925: Geneva protocol
This is the “Protocol for the Prohibition of the Use in War of Asphyxiating, Poisonous or other Gases, and of Bacteriological Methods of Warfare”.
1928: Kellogg-Briand pact
This was the “General Treaty for Renunciation of War as an Instrument of National Policy” and was signed by the leading powers of the time. Although it evidently failed in its stated purpose, it formed the basis of some important international laws that relate to the use of force.
1949: Geneva conventions
These updated previous treaties of 1864, 1906 and 1929. They established wartime rights for prisoners of war, wounded combatants and civilians.
1967: Outer space treaty
This banned the installation of weapons of mass destruction in orbit around the Earth or on the moon.
1970: Treaty on the Non-Proliferation of Nuclear Weapons
This was intended to prevent the spread of nuclear weapons technology beyond those countries that already had it and to promote the use of nuclear technology for non-violent purposes.
1975: Biological Weapons Convention
This bans the production and stockpiling of biological weapons (the use of which was already illegal under the 1925 Geneva protocol). One hundred and seventy states have ratified it – Egypt and Syria are among those that have signed but not ratified, and Israel is perhaps the most significant state that has not signed.
1978: Environmental Modification Treaty
This bans “weather warfare”, the use of weather modification techniques for destructive purposes.
1995: Protocol on Blinding Laser Weapons
This bans weapons that aim to cause permanent blindness
1997: Chemical Weapons Convention
This bans the production and stockpiling of chemical weapons (as with biological weapons, their use was already illegal under the 1925 Geneva protocol); 190 states have ratified it – Israel and Burma have signed but not ratified, while Egypt and North Korea have not signed.
1997: Ottawa treaty
This bans the use, production and stockpiling of anti-personnel landmines. One hundred and sixty-two states are party to the treaty; 34 UN states, including China, Egypt, Israel, North and South Korea, Russia and the US, are not.
2010: Convention on Cluster Munitions
This bans the use and stockpiling of cluster bombs – explosives that scatter “bomblets” over an area. Eight-eight states have ratified it so far – those who have not include China, Egypt, Israel, North and South Korea, Russia and the US.
2014: Lethal Autonomous Weapons Systems
This year, the question of whether to ban robot warriors was discussed at the UN as part of the ongoing Convention on Conventional Weapons, updates to which included the ban on landmines and blinding lasers.
This article appeared in print under the headline “World War R”
