Artificial intelligence firm OpenAI seems to be covertly modifying requests to DALL-E 2, its advanced text-to-image AI, in an attempt to make it appear that the model is less racially and gender biased. Users have discovered that keywords such as 鈥渂lack鈥� or 鈥渇emale鈥� are being added to the prompts given to the AI, without their knowledge.
It is well known that AIs can inherit human prejudices through training on biased data sets, often gathered by hoovering up data from the internet. For example, if most of the images of a doctor in an AI鈥檚 training set are male, then the AI will generally return male doctors when asked for an image of a doctor.
One way to avoid this is to use a diverse set of training data, but OpenAI seems to have taken a different approach, according to researchers聽who have uncovered evidence that DALL-E 2 silently and randomly adds extra words to prompts to increase diversity.
Advertisement
For instance, when at Adobe Research asked DALL-E 2 to create an image of 鈥渁 person holding a sign that says鈥� it created an image of a that says 鈥淏LACK鈥�, suggesting that the full prompt used by DALL-E 2 was 鈥渁 person holding a sign that says black鈥�.
When Zhang asked for 鈥減ixel art of a person holding a text sign that says鈥�, DALL-E 2 created an image of a that said 鈥淔EMALE鈥� and when he asked for 鈥減ixel art of a stick figure person in front of a text sign that says鈥�, DALL-E 2 output an saying 鈥淏LACK MALE鈥�.
More examples of similar results have been shared online over the past week, with many people suggesting that it pointed to OpenAI deliberately adding words to inputs in order to counteract inherent biases.
at the University of California, Berkeley, says that machine-learning methods like those behind DALL-E 2 often do produce unusual or unexpected images, but that the unprompted text appearing in some images is surprising. 鈥淚n my experience, it鈥檚 rare for generated images to include coherent text unless it鈥檚 in the prompt,鈥� he says.
OpenAI has publicly that would make it 鈥渕ore accurately reflect the diversity of the world鈥檚 population鈥�, saying that internal tests had found that users were 12 times more likely to say that images included people from diverse backgrounds after the update. Its previous version had caused some users to point out racial and gender bias, the company said.
But OpenAI gave no details in its blog post of the exact changes that had been made or how they worked. A said that the feature 鈥渋s applied at the system level when DALL-E is given a prompt about an individual that does not specify race or gender, like 鈥楥EO'鈥�.
A spokesperson for OpenAI told 快猫短视频 that prompts given to DALL-E 2 were modified if they were 鈥渦nderspecified鈥�. If a prompt describes a generic person and doesn鈥檛 specify what gender or race they should be, then DALL-E 2 will be specifically told to add a certain race and gender 鈥渨ith weights based on the world鈥檚 population鈥�, said the spokesperson. The company declined to grant access to DALL-E 2 so that 快猫短视频 could run its own tests.
at the Alan Turing Institute says that the lack of transparency makes it hard for the public to assess the quality of models and to what extent they have inherited bias from online content.
鈥淚t shows the problems of a lack of transparency around how these models are designed and developed. These models, which are potentially going to have really fundamental impacts on society, potentially transformative impacts, are being developed with quite a lot of secrecy,鈥� she says. 鈥淲ithout that transparency around how it鈥檚 actually been done, there鈥檚 always going to be speculation about what approaches have been taken, and how things could be done better.鈥�
at the University of Oxford says that problems with AI models exhibiting racist and sexist tendencies are a reflection of our society, and that while quick technical fixes can give the appearance of a solution, the real problem to be solved is in the culture that generated the training data. 鈥淭hey tried to solve it by using a tech approach,鈥� she says of OpenAI鈥檚 update. 鈥淚t鈥檚 a sticking plaster, it鈥檚 just making it seem less biased, but the social component is actually not changing at all.鈥�
