Making an artificial intelligence less biased makes it less accurate, according to conventional wisdom, but that may not be true. A new way of testing AIs could help us build algorithms that are both fairer and more effective.
The data sets we gather from society are infused with historical prejudice and AIs trained on them absorb this bias. This is worrying, as the technology is creeping into areas聽like job recruitment and the聽criminal justice system. New techniques can make AIs fairer, such as by preprocessing training data to remove bias, but in practice these lead to less precise results.
Or do they? 鈥淭he trade-off that we see is kind of an illusion,鈥� says Sanghamitra Dutta at Carnegie Mellon University, Pennsylvania.
Advertisement
For example, a firm may employ more men as its predominantly male management has hired fewer women due to unconscious bias.聽If聽that company uses its employment data to train an AI to聽assess job applicants and hire staff, the dearth of information on聽women makes it harder for the聽system to judge their aptitude, putting them at a disadvantage.
The company could use existing fairer training techniques to create a new AI, but if it is tested on the original, biased data it will appear to be less accurate than the original AI, says Dutta.
That doesn鈥檛 mean the fairer AI聽is no good though, says Dutta. Biased hiring practices have made the firm鈥檚 data unrepresentative of聽the entire pool of job candidates. Instead, AIs should be tested using an ideal data set, says Dutta. When you do this, the trade-off between accuracy and fairness disappears.
Dutta, who carried out the work with colleagues while at IBM, has developed a way to create this ideal data set. The technique draws on a field of mathematics called information theory to equalise the amount of information on each group, providing a statistical guarantee of fairness. In the case of the hiring company, that might mean using the existing data to invent some fictional women to balance the amount of information on each group, though Dutta says the approach works with multiple categories and more complex data than just numbers of employees.
In work due to be presented at the virtual International Conference on Machine Learning on 16 July, the team shows that by using these techniques, it is possible for an AI to improve in accuracy and fairness simultaneously.
The approach can help evaluate AI, says Dutta. If two AIs perform similarly on biased data, but one performs better on the ideal data set, it has greater potential for both fairness and accuracy. If fair algorithms perform much better when using ideal data sets, this could also alert companies to serious bias in their data.
Fairness tests are important, says Sandra Wachter at the University of Oxford, but she cautions that they only reveal the problem of bias in society. 鈥淭hat鈥檚 the first step, but the actual hard work is how are we going to fix that problem.鈥� To do so, computer scientists can鈥檛 rely on automated fixes and will need to engage more with social scientists, she says.
