AI image generators can be made more culturally sensitive and accurate by feeding them just a small number of photographs provided by people living in countries around the world.
The images used to train these artificial intelligence systems 鈥渁re mostly about the Western world鈥�, says at Carnegie Mellon University in Pennsylvania. As a result of this kind of limited training, generative AI image creators, such as Stable Diffusion, often misrepresent or stereotype non-Western cultures.
Oh and her colleagues asked people living in these underrepresented countries to provide 20 to 30 captioned images that better represented their society. The researchers received around 1,000 images and captions in total from China, South Korea, India, Mexico and Nigeria.
Advertisement
That data was then shown to a pre-trained version of the Stable Diffusion model and presented as accurate cultural representations. At the same time, images generated by the model that were stereotyped or inaccurate were automatically flagged as incorrect 鈥� a method called self-contrastive fine-tuning.
To judge the effect of the fine-tuning, 51 people from the five countries reviewed outputs from both the retrained and original models. On average, the fine-tuned model was perceived to produce less offensive images between 56 and 63 per cent of the time. Stability AI, the company that created Stable Diffusion, did not respond to a request for comment.
Feeding the model a small number of additional images 鈥� Stable Diffusion was reportedly trained on 2.3 billion images 鈥� seems to successfully remove reliance on stereotypes, suggesting this form of AI bias can be countered comparatively cheaply and simply.
鈥淲e know our training dataset is small, but using self-contrastive fine-tuning, we still leveraged the knowledge of the original model,鈥� says at Carnegie Mellon University. Next, Oh plans to expand the countries beyond the original five used.
鈥淥verall, this is a great approach,鈥� says at Humane Intelligence, a non-profit that reviews AI models. 鈥淚 applaud reaching out to communities to gather their feedback and develop approaches.鈥澛燞owever, Chowdhury points out that the issue isn鈥檛 always that AI models lack diversity, but that they don鈥檛 understand what that is 鈥� or when it is appropriate and important to consider it.
arXiv
