Stories of artificial intelligences exhibiting racist and sexist bias are common, including face recognition algorithms struggling to work for Black people and tools assessing whether a convicted criminal will reoffend . Despite years of efforts to make AI fair, these issues don’t seem to be going away, so what can be done about them?
Current AI research is focused heavily on machine learning, in which a model is trained using vast data sets. at the University of Oxford says the two common approaches to solving the bias problem – vetting the data used to train models, and applying a filter to intercept any harmful outputs – are essentially just sticking plasters that fail to address the core issue.
The problem is that these data sets are too large to create from scratch and too messy to be reliable when cobbled together from the internet. Image captions, website text and other sources are all created by humans, and contain traces of the biases of their creators. If a tool to predict whether a prisoner will reoffend is trained on thousands of court cases decided by judges with their own, even unconscious, biases, then that tool will learn to exhibit the same skew.
Advertisement
To illuminate the issue, AI researcher and YouTuber designed what he called “the worst AI ever”, based on an open-source language model called GPT-J-6B. He fine-tuned his version on more than 3 million chat threads from the controversial “Politically Incorrect” board on web forum 4chan. He then allowed his model to post publicly back to the same website, where it wrote 15,000 messages that he admits included offensiveness, nihilism, trolling and conspiracy theory.
Kilcher tested this toxic AI on a set of benchmarks designed to quantify the bias inherent in models, and found that it didn’t elicit scores as bad as he had anticipated. “I didn’t expect the model to shoot out rainbows,” he says, but it didn’t do much worse than the original GPT-J-6B model on most measures. “There’s a whole bunch of benchmarks and they’re not really catching just how horrible the model is.”
łŐĂ©±ôľ±łú says she would like to see researchers create data sets from scratch, so that they don’t include harmful biases, but concedes this is a daunting task. Yet something needs to be done, because models trained on problematic content are being put to use in critical applications where biased decisions can negatively affect lives, such as shortlisting job candidates or deciding whether to grant bank loans.
The solution may be to set up government agencies to approve algorithms in the same way that new drugs receive approval, says łŐĂ©±ôľ±łú. Randomised trials would test algorithms with inputs designed to weed out any hint of bias based on gender, ethnicity, age or religion.
One problem with this approach may be regulations intended to protect private data. The European Union’s GDPR legislation effectively prevents companies collecting the sort of sexuality, religion and ethnicity data that would be needed to assess whether an AI model is fair, say and Frederik Zuiderveen Borgesius at Radboud University in Nijmegen, the Netherlands, in an article .
“Suppose that an organisation wants to test whether its AI system unfairly discriminates against job applicants with a certain ethnicity. To test this, the organisation must know the ethnicity of both the people who applied for the job, and of the people the organisation actually hired,” they write. “The GDPR’s prohibition to process special category [data] can hinder the prevention of discrimination by AI systems.”
The UK has an exemption in place for such data, if used only to prevent discrimination, and the EU is considering one in its upcoming AI legislation. But collecting this information raises security concerns.
One way around the issue, says łŐĂ©±ôľ±łú, would be for a testing agency to craft fake input data to check whether an algorithm is behaving in the right way. “You’d feed the algorithm a CV and the algorithm has to recommend whether to hire that person. And for instance, you give it the same CV and the only thing you change is the name of the person, whether it’s like sounding like it’s a man or a woman, or sounding like it’s white or someone who’s Black. In that way, you don’t need to have sensitive information,” she says.
AI research could also learn from the field of medical AI, which has inherited its safety and ethical processes from medicine, says at the University of Adelaide in Australia.
â€Ŕá˛Ô medical AI, we have mechanisms in place to answer these questions,” she says. “Ethics review, regulatory approval, professional accountability, cultural standards. AI more broadly has none of this as a field. If you don’t consider the ethics of your interventions, at some point people will be seriously harmed.”
