A diversity score for artificial intelligence inspired by how ecologists measure biodiversity could help identify bias in data sets and AI systems.
AIs can learn and mimic certain biases if their training data sets are not diverse enough. This has already been seen in AI systems perpetuating racist and sexist prejudice and in governments using racially biased AI for checking passport photos. Such AI-powered bias has also shown up in law enforcement efforts to predict crime.
and Dan Friedman at Princeton University developed their Vendi Score by taking inspiration from how ecologists measure species diversity. The scoring system measures the effective number of unique or dissimilar items within AI systems or data sets.
Advertisement
The Vendi Score system computes the similarity between any two pairs of items in a collection and arranges the similarity scores in an ordered table of numbers. It then computes the randomness of the table鈥檚 similarity scores to provide a final diversity score.
The new scoring system has already shown that it can evaluate diversity in both AI systems and data sets commonly used to train AI. For example, a Vendi Score analysis that defined diversity as showing many kinds of objects found that a huge image database used for visual object recognition training 鈥� filled with more than 14 million images 鈥� had the highest Vendi Score of 43.93. By comparison, image databases that only contain one class of objects, such as cats or bedrooms, had lower scores of 15.12 and 8.99 according to this diversity measure.
But the Vendi Score鈥檚 definition of diversity is flexible. Another analysis of the same data sets that is focused on measuring diversity according to different types of cats could instead lead to the cat image database having the highest Vendi Score, says Dieng.
鈥淚 think we cannot overestimate the importance of identifying and mitigating algorithmic bias in automated decision systems,鈥� says at the University of Oxford, who was not involved in the study. 鈥淲ho gets promoted, who gets a loan or insurance, who gets to go to university and who needs to go to prison 鈥� are all decisions that are made by algorithms.鈥�
Although Wachter praised such efforts to test for algorithmic bias, she cautioned that bias testing effectiveness is dependent on different cultural definitions of fairness and equality. Her found that many bias tests developed in the US did not meet the standards of UK and EU equality laws. 鈥淎 one-size-fits-all solution is not possible and not desirable,鈥� says Wachter.
Such diversity scoring could even improve the performance of AIs focused on predicting new molecules and materials for science, says Dieng. The Vendi Score was able to detect duplicates in molecular structures generated by such an AI, which highlighted an existing issue that could hinder the AI鈥檚 potential to discover unique molecules.
Reference:
