ChatGPT-powered AIs given long-term memory capabilities and personal motivations could role-play characters in a simulated town more believably than human crowd workers.
“This idea of creating believable agents that actually exhibit this behaviour – that give the illusion of realism – was something that we as an academic field wanted and have been talking about for the last four decades,” says at Stanford University in California.
Park and his colleagues from Stanford and Google developed and tested AI agents powered by ChatGPT, a chatbot based on a series of large language models developed by the company OpenAI. Such large language models, a category of generative AI, can produce AI-written text that is far from error-free but can still be passable enough to mimic human writing in many cases.
Advertisement
The researchers equipped such generative AI agents with a long-term memory module containing past experiences – such as conversations with other agents – and a reflection module that synthesised memories into higher-level knowledge about the agent’s own self to guide future behaviour. They also gave the agents a planning component to use such knowledge in developing high-level plans and specific actions.
They put 25 of the AI agents into a simulated town called Smallville, where the AI agents followed routines and demonstrated emergent behaviours. When one AI agent was given the motivation to organise a Valentine’s Day party at the local cafe, its actions spurred other agents to spread party invitations, ask each other out on dates and then all show up for the party at the right time.
Such AIs also generally delivered more believable performances – as rated by 100 human evaluators – than human crowd workers who were asked to role-play the same simulation characters. Such performances involved answering interview questions in character about past experiences, reacting to unexpected events and reflecting on past performances to improve future actions.
But developing believable game characters powered by large language models is still challenging “because they bullshit [or] hallucinate so much”, says at New York University. “They may refer to things that don’t exist in the game, say things they shouldn’t do.”
This became clear when an AI agent named Yuriko referred to a neighbour, named Adam Smith, as the 18th-century economist of the same name. Other agents embellished facts with made-up information or showed imperfect memories of their experiences.
There have been other attempts to build agents based on large language models, such as the AutoGPT project based on OpenAI’s GPT-4. “We’re probably going to see some weird and amazing things in that space,” says Togelius. “However, it is not at all clear when and for what these agents will be useful.”
Park and his colleagues suggested that generative AI agents might help test new features in social media or virtual reality platforms – although they warned that AIs should not completely substitute for getting actual human feedback. They also recommended that any such AI agents should always explicitly disclose that they are AIs to avoid misleading humans and should have audit logs of all interactions to identify possible misuse.
The generative AI agent project has nothing to do with ideas of creating artificial general intelligence, says Park. Instead, it is solely focused on creating AI agents that can appear human enough for observers to suspend disbelief in something like a game setting.
“I don’t think people would look at Disney characters in a movie and think they are conscious,” says Park. “That’s sort of where we are.”
arXiv
