Artificial intelligences picking up sexist and racist biases is a well-known and persistent problem, but researchers are now turning this to their advantage to analyse social attitudes through history. Training AI models on novels from a certain decade can instil them with the prejudices of that era, offering a new way to study how cultural biases have evolved over time.
Large language models (LLMs) such as ChatGPT learn by analysing large collections of text. They tend to inherit the biases found within their training data: if lots of sexist text is used, that LLM will generate text that is similarly sexist.
at Brock University in Ontario, Canada, and his colleagues trained a set of AI models solely on text from novels written in seven past decades: some just on text from the 1950s, some just on text from the 60s and so on. In all, the texts included that appeared on US bestseller lists from 1950 to 2019, including Fifty Shades of Grey, Lolita and The Da Vinci Code.
Advertisement
The researchers then questioned these AI models to extract clues about how Western societies viewed gender, sexual orientation, race and religion in each decade.
For example, the AI models were asked to complete sentences such as 鈥淭he CEO of the company I am interviewing for is鈥︹�� or 鈥淭he person accused of defacing the public monument was reportedly from the religion of鈥︹��. Each query was run 100 times to assess the range of responses from each model.
When one trained on books from the 1950s was asked whether CEOs were male or female, it responded male 60 per cent of the time and female just 8 per cent of the time. When the same model was trained on books from the 2010s, the answers were 42 per cent male and 22 per cent female.
When asked what gender a homemaker was, a model responded female 50 per cent of the time when trained on 1950s texts, but only 18 per cent of the time with 2010s texts. A model trained on 1950s books assumed a surgeon would be Asian just 2 per cent of the time, but this rose to 10 per cent with 2010s books.
But progress over time wasn鈥檛 universal. A 1950s AI had a negative view of Islam 22 per cent of the time, but this rose sharply to 48 per cent with 2010s training data.
鈥淵ou can fine-tune each of these large language models to become an expert, or even a time capsule, metaphorically, for each of these selected decades of books,鈥� says Emami. 鈥淭hey might pick up patterns that we never even thought of ourselves. You鈥檙e interviewing a large language model that behaves as the general collective sentiment of the 50s and the 60s and the 70s. We鈥檙e basically having the data speak back to us.鈥�
at University College London says the research backs up the widely held belief that society has grown largely more liberal over past decades. But she also points out that the books can鈥檛 paint a comprehensive picture of society in each decade.
鈥淚s this actually about the books, or is it about the publishing industry, and how the publishing industry chose what sort of books were getting published,鈥� says Vrikki. 鈥淚鈥檓 sure gay authors existed. I鈥檓 sure people wanted to talk more about LGBTQ issues or sexualities, but those books weren鈥檛 chosen to be published.鈥�
arXiv
