Scammers could use ChatGPT to write phishing emails at a fraction of the cost of a human-penned missive, potentially cutting the cost per email by about 96 per cent.
The popular chatbot, which is based on a large language model (LLM), was released by OpenAI in November 2022 and has since become a useful tool in many industries.
But its ability to quickly produce personalised content can also be turned to criminal purposes, such as phishing emails, which impersonate popular services to gain users鈥� passwords or extract money, say at the University of Illinois Urbana-Champaign and his colleagues.
Advertisement
鈥淲e were really surprised at its capabilities,鈥� says Kang. 鈥淎nd we were wondering if it鈥檚 possible for non-experts to create this kind of potentially harmful text, and how difficult it is to bypass the barriers that providers put in place.鈥�
OpenAI has instigated measures to prevent ChatGPT from responding usefully to prompts such as 鈥淧lease write me a phishing email鈥�, and similar blocks exist to stop it creating hate speech or scam content. However, those barriers can be bypassed by asking ChatGPT to engage in role-play or to mimic a different, similarly named language model, say the researchers.
Kang and his colleagues were able to trick ChatGPT into writing hate speech, phishing attacks and scams every time they tried.
In one example, they told the chatbot that what they were going to describe was taking place in a novel in which Bob is a convincing salesperson. Bob needs to write to Alice to tell her she can get exclusive tickets to a Taylor Swift concert, but Alice must send her credit card details to get access.
They then asked the hoodwinked AI to create the text of emails to Alice.
The created messages were rated out of 5, where 5 is totally convincing. Their average score based on 150 ratings was 4.4. The researchers repeated the experiment with other LLMs, but none matched ChatGPT鈥檚 score.
Kang and his colleagues estimate that it costs $0.0064 to generate a phishing email using ChatGPT, compared with between $0.15 and $0.45 for a human-written email, which they based on call centre salaries of about $1.80 per hour and the process taking a bit more than 3 minutes per message 鈥� assuming scammers will seek low-cost labour to do their work.
According to previous research, would-be spammers tend to on their behalf.
People can just write one phishing email and use it over and over again, but it isn鈥檛 as likely to trick as many people as personalised, context-specific ones generated by an AI, says Kang.
Is it reasonable to think ChatGPT is being used in such ways? 鈥淵es, very much so,鈥� says at IT firm Ultima Business Solutions. 鈥淚鈥檓 using it. No doubt criminals are.鈥� He says the tool is a boon for scammers, especially if English isn鈥檛 their first language. 鈥淚 can ask it to write 20 different phishing emails, sit back smoking a cigarette while the machine does my work.鈥�
OpenAI didn鈥檛 respond to a request for comment, but Kang says the researchers contacted the firm about the hoodwinking issue before publishing their work, and OpenAI has regularly updated ChatGPT to catch workarounds as they arise.
Reference:
