Driverless cars may be worse at detecting children and people with darker skin, tests on artificial intelligence systems suggest. The researchers who carried out the work say that tighter government regulation is needed and that car-makers must be transparent about the development and testing of these vehicles.
at King鈥檚 College London and her colleagues assessed eight AI-based pedestrian detectors used in driverless car research. The overall detection accuracy for adults was almost 20 per cent higher than it was for children, and was 7.5 per cent higher for pedestrians with light skin compared with those with dark skin.
Car-makers don鈥檛 release details of their own software, but Zhang says it is likely to have the same problems. 鈥淚t鈥檚 their confidential information and they won鈥檛 allow other people to know what models they use, but what we know is that they鈥檙e usually built upon the existing open-source models,鈥� she says. 鈥淲e can be certain that their models must also have similar issues.鈥�
Advertisement
Zhang says part of the problem is that the main open-source collections of pedestrian images, which are used to train AI models, feature more people with light skin than dark skin.
Manufacturers don鈥檛 reveal the data they use to train their models, but they may use the same open-source collections used in this experiment, proprietary collections or a combination. Zhang says that more transparency is needed on how commercial models are trained and how they perform, so their safety can be objectively measured.
鈥淲e need much better regulations,鈥� says Zhang. 鈥淚 actually checked the current UK regulations. They just briefly mention the importance of fairness, but it鈥檚 very vague.鈥�
Racial bias in AI has been revealed before, including some AIs being less accurate at identifying the faces of Black people than those of white people. But while these problems can block people from accessing vital services or even skew the severity of sentences handed out in court, the repercussions of such bias in driverless car software could be fatal.
快猫短视频 contacted several leading companies developing autonomous cars, but Tesla, Ford, Toyota and Volkswagen did not respond. Mercedes declined an interview and to provide information about what its own software is based on, but said in a statement that it attaches 鈥済reat importance to the greatest possible safety for all traffic participants鈥�.
at the University of Oxford says the problems identified in the study need to be fixed before AI systems are used in cars on real roads. However, engineers must make sure their solutions don鈥檛 have unintended consequences that harm overall safety, she adds.
鈥淗ypothetically, [efforts to improve safety] could make the car more unsafe because it stops every time a small object is in its view in case it鈥檚 a child, and that, of course, can create more accidents in theory,鈥� says V茅liz. 鈥淥ne of the lessons of AI ethics is that there鈥檚 no easy solution. What鈥檚 important is first to identify what the problems are, then to make sure that the company is auditing those algorithms to be as fair as possible, and that the trade-offs are made in a conscious way and in a way that can be defensible and explained.鈥�
arXiv
