鈥淎RE we the baddies?鈥� That punchline from a Mitchell and Webb comedy sketch about two Nazi soldiers is something that scientists ought to ask听themselves more frequently. Even if听the knowledge that people create is neutral, its applications often aren鈥檛.
The potential for misuse often goes unnoticed until it is too late. Consider a Facebook app called This Is Your Digital Life, which gathered information about users鈥� personalities. What looked like an innocuous, even fun, research project was later used by Cambridge Analytica to听harvest personal data and manipulate people鈥檚 exposure to political messaging without their knowledge or consent.听
Even though the effectiveness of Cambridge Analytica鈥檚 manipulations has been questioned, this demonstrates how well-intentioned research can be听hijacked by bad actors.
Advertisement
This is why the pre-emptive steps being taken by the scientists developing a technology called multi-agent artificial intelligence (MAAI) are praiseworthy. Simulations that use MAAI allow computer scientists to build detailed digital models of human societies and see how decisions would play out in the听real world (see 鈥淎I can predict your future behaviour with powerful new simulations鈥�).听
Although the technology has immense potential for good 鈥� for example, it is already being used to find ways to peacefully integrate refugees into Western societies 鈥� it has equal potential for bad. Politicians could use听it听to do the exact opposite: stoking anti-immigrant sentiment and violence for political advantage.
The developers of MAAI are aware of听the potential for misuse 鈥� they actually use the word 鈥渆vil鈥� 鈥� and are drawing up a code of conduct to prevent misuse. But self-policing can get us only so far; the temptation to bend the rules will prove irresistible to somebody.听
What we need is robust debate about听the technology and how it should听and shouldn鈥檛 be used. Looking at听UK and US听politics at the moment, however, it isn鈥檛 hard to suspect that it may already be too late to prevent good science from听being misused.
