Instructing an AI chatbot to answer questions as if it were in the TV show Star Trek seems to improve its mathematical ability, although no one is exactly sure why.
People who use chatbots like ChatGPT have already recognised that the quality of outputs can be improved by asking the AI to adopt a certain persona, or by bribing or threatening it. Rather than use trial and error to discover which such prompts are most effective, and at software firm VMware in California turned to the large language models (LLMs) that power the chatbots. They used them to fine-tune human-created prompts and then rated their effectiveness at called the GSM8K benchmark.
The researchers gave 60 initial prompts to three LLMs: one developed by French firm Mistral, which has recently partnered with Microsoft, and two versions of Llama2, made by Meta. The AIs were then tasked with improving the wording of the messages to make them more effective.
Advertisement
For example, from an initial prompt of âYou are an expert mathematician. Solve the following math problem. Take a deep breath and think carefully,â the AI-improved prompt might add requirements to define any assumptions, or to flag any loopholes used.
Those resulting prompts were then fed back into the AIs in an effort to tackle the GSM8K questions, which require simple arithmetic to solve, but take between two and eight steps to complete.
In nearly all instances, the AI models produced prompts that generated more correct answers to the GSM8K questions than human-created prompts. âIn my opinion, nobody should ever attempt to hand-write a prompt again,â says Battle. âLet the model do it for you.â
However, entrusting a chatbot to write prompts to help the same chatbot answer maths questions can result in some unusual exhortations. The highest-scoring prompt generated by the Llama2-70B model, for instance, asked the chatbot to adopt the persona of the captain of a Star Trek spaceship, jotting down answers in its âCaptainâs Logâ â something generated entirely spontaneously by the AI and not suggested by the initial prompts.
Why the AI produced such unusual prompts is âthe $64 million questionâ, says Battle. âTo a certain extent, the answer is âI donât care, just give the model what it wants.ââ However, thinking more scientifically, Battle believes it is a product of the data the model was trained on, perhaps with Star Trek content appearing more often with correct information. âWho knows? Thereâs a lot of Star Trek references on the internet.â
âThe key thing to remember from the beginning is that these models are black boxes,â says at Staffordshire University, UK. âWe wonât ever know why they do what they do because ultimately they are a melange of weights and probabilities and at the end a result is spat out.â
However, despite the fact that the Star Trek-themed prompt was the most successful, both Flick and Battle say that you shouldnât be addressing ChatGPT or other chatbots as âcommanderâ any time soon.
âOne thing is for sure: the model is not a Trekkie,â says Flick. âIt doesnât âunderstandâ anything better or worse when preloaded with the prompt, it just accesses a different set of weights and probabilities for acceptability of the outputs than it does with the other prompts.â
Instead, the response to the discovery tells us more about ourselves and our perceptions of AI than the AIâs performance, believes Flick. âIt shows a fun result, but itâs a coincidental one that just so happens to match the desire for LLMs to be science fiction brought to life,â she says.
arxiv
