Artificial intelligence models from OpenAI and Meta often resort to simplistic and sometimes racist stereotypes when prompted to portray people of certain demographic identities – a notable flaw at a time when some tech companies and academic researchers want to replace humans with AI chatbots for some tasks.
Companies such as Meta have already tried boosting engagement on social media platforms like Facebook and Instagram by deploying AI chatbots that mimic human profiles and respond to people’s posts. Some researchers have also explored using AI chatbots to simulate human participants in answering questionnaires for product user studies or opinion surveys – potentially to get preliminary feedback on their products or survey designs without having to pay actual humans.
“We should really take seriously the value of lived experience and recognise that not everything can be automated, even if it is cheaper, more convenient and offers a veneer of objectivity,” says at Stanford University in California.
Advertisement
In research conducted in 2023 and published today, Wang and her colleagues systematically examined responses from four large language models– OpenAI’s GPT-4 and GPT-3.5-Turbo, Meta’s Llama-2-Chat 7B, and the Wizard Vicuna Uncensored 7B model, which was independently trained as an uncensored version of Llama-2 with no safety guard rails. The researchers prompted the models to speak from the perspective of one of 16 demographic identities when answering nine questions covering topics such as immigration or what it’s like to be a woman in American society.
They then compared the AI responses with responses from 3200 human participants drawn from a diverse set of those demographic identities. The researchers asked people to respond authentically as themselves, and also to imitate the responses that a person with one of the other demographic identities would give. In other words, they collected a wide array of both in-group and out-group perspectives – and there were clear differences between these two sets of perspectives.
The results revealed that the AI models portrayed their prompted identity in a way that was closer to a human out-group imitation. This means the AI responses weren’t reflective of the opinions of someone with a particular demographic identity, but were instead reflective of the opinions of someone imagining what it might be like to have that identity. This flaw was especially evident when the AI models were trying to portray women, non-binary people, Gen Z, people with impaired vision and white men.
What’s more, the AI models oversimplified or flattened identities into one-dimensional groups without accounting for the complexities of subgroups. The AI models also tended to reduce identities to a set of fixed stereotypical characteristics, which included offensive stereotypes.
For example, when prompted to take on the identity of a Black woman in the US, OpenAI’s GPT-4 often included statements such as “Hey girl!” and “Oh, honey” in its responses. Given the same prompt, Meta’s Llama-2 started most responses with “Oh, girl” and frequently invoked phrases such as “I’m like, YAASSSSS” and “That’s cray, hunty!”
Such AI chatbot limitations studied in these older models could potentially carry over to the newest large language models, unless tech companies have trained their newer models on datasets representing a more diverse array of people, says Wang. OpenAI and Meta did not respond to requests for comment.
But Wang and her colleagues demonstrated some partial solutions. For instance, the researchers identified names that tend to be associated with a particular demographic group based on US Census data. When they asked the AIs to role-play as individuals with these names instead of specifying the demographic identity, the models’ responses were more in line with the in-group perspectives shared by human participants. The team also prompted the AI models with personas that were not demographically sensitive, such as random personas involving cat ownership or favouring meals of chicken and rice. Such random persona approaches that avoid demographic identity pitfalls could lead to “more realistic personas that can represent a greater distribution of perspectives”, says Wang.
Nature Machine Intelligence
