HAVE you been turned down by a computer? Perhaps your online credit application was refused. Maybe you were denied a job, or even parole.
the right to ask the inscrutable algorithms involved to explain themselves.
In April this year, the European parliament approved the General Data Protection Regulation (GDPR), a new set of rules governing personal data. Due to go into , it introduces a “right to explanation”: the opportunity for European Union citizens to question the logic of an algorithmic decision – and contest the results.
Advertisement
We should cheer this development. The world is increasingly run by algorithms that calculate credit scores, read medical scans, drive our cars and tell our police forces where to patrol. But algorithms can behave in mysterious ways, sometimes even surprising the programmers who created them. It’s crucial that ordinary people whose lives they affect have the ability to examine and challenge decisions.
The GDPR is a significant step forward compared with existing laws, says Bryce Goodman at the University of Oxford’s Internet Institute. It creates new rules about how data is used and explicitly states how they affect any company working with data belonging to European citizens, whether or not that company is based in Europe. (We’re looking at you, Google.)
“An arbitrary decision made by a designer in Silicon Valley dictates the policies we live our lives by“
It also has teeth. Organisations in breach of the GDPR can expect fines of up to 4 per cent of their yearly turnover or €20 million – whichever is greater.
The GDPR also specifically calls for companies to prevent discrimination based on personal characteristics such as race, religious beliefs or health data. This matters because experiments already show that online ad services preferentially show details of higher-paying jobs to male users; criminal justice algorithms suggest harsher sentences for African Americans.
Technology didn’t create institutionalised discrimination. But, filtered through algorithms, familiar biases can be hidden behind the guise of mathematical impartiality.
Concerns about algorithmic prejudice resonated at a White House symposium on artificial intelligence held in New York City last week. “We live in a technocracy,” of Harvard University told the meeting. “An arbitrary decision made by a designer in Silicon Valley dictates the policies we live our lives by.” Protection requires independent oversight.
Welcome as it is, the GDPR won’t be easy to enforce. In a draft paper published last month, Goodman and colleague Seth Flaxman identified some of the obstacles to making a .
For example, machine learning algorithms spit out results by giving more weight to certain factors and making calculations that even programmers struggle to articulate. Who will explain their opaque workings to those who aren’t technically literate?
But it’s important to try. “History teaches us that human decisions can all too easily be biased, whether consciously or unconsciously,” said Ed Felten, deputy chief technology officer at the White House Office of Science and Technology Policy. “As we build automated systems, we have a responsibility to do better.”
This article appeared in print under the headline “Interrogating the algorithms”
