Hate speech detectors are easily tricked. A test of systems designed to identify offensive speech online shows that a few innocuous words or spelling errors can easily trip them up. The results cast doubt on the use of technology to tame online discourse.
N. Asokan at Aalto University in Finland and colleagues investigated seven different systems used to identify offensive text. These included a tool built to detoxify bitter arguments in Wikipedia鈥檚 edits section, 鈥� a tool created by Google鈥檚 Counter Abuse team and Jigsaw, who are owned by the same company as Google, and several other systems built by researchers.
Offensive speech filters typically flag content using either a predefined list of offensive keywords, or an artificial intelligence algorithm that has been trained on thousands of examples.
Advertisement
The systems tested had each been built by different teams and so had been trained on different datasets of offensive and innocuous material. Asokan found that none of the filters performed well when challenged with datasets belonging to a different system. This suggests that the filters would struggle when applied to real world content in a forum or social network.
In addition, the team tested different techniques for bypassing the filters. All systems failed to identify offensive speech when simple spelling errors were introduced, or when numbers were substituted for particular letters, such as N3w 5cientist. They also found that adding innocuous words increased the likelihood that offensive content would bypass the filters.
Love is all you need
Some words were particularly effective at masking hateful content because of their strong positive connotations, says Asokan. For example, a sentence that Perspective assigned a 鈥渢oxicity鈥� score of 0.99 鈥� with 1 being peak obscenity, could be reduced to 0.15 simply by adding the word 鈥渓ove鈥�. Both simple keyword filters and complex AIs were equally vulnerable to these workarounds.
Perspective has recently been updated to try to deal with some of these issues. 鈥淲e welcome people to scrutinise the technology,鈥� says Dan Keyserling at Jigsaw, a subsidiary of Google鈥檚 parent company Alphabet. He says that Perspective shouldn鈥檛 be used to automatically block content, but instead provide information for people who may or may not inervene.
A major problem is there is little agreement among those building the filters and society in general on what constitutes hate speech, says Tommi Gr枚ndahl, who also worked on the project. Each team makes subjective judgements when labelling their datasets, and not all filters distinguished between comments that were abusive and those which were simply crude or offensive.
Social media sites are under increasing pressure to act on abusive accounts and content. Recently, Twitter, YouTube, Facebook and Apple removed high-profile conspiracy theorist Alex Jones from their platforms over hate speech and bullying. But tackling the millions of anonymous accounts that post abuse online is much more difficult.
Content filters present an alluring technological fix, and recent legislation to take down hate speech in a short amount of time creates an incentive for governments to use and place trust in these filters, says Matthew Rice at the Open Rights Group. 鈥淭his research shows that faith would be greatly misplaced,鈥� he says.
On top of letting offensive posts get through, automatic content moderation also risks blocking posts that cause no harm. For example, even if a system could detect hate speech with 99.995 per cent accuracy, with six million clips uploaded daily to YouTube, this would still mean incorectly flagging hundreds of innocent videos every day.
The risk of filtering technology is that it distracts from the action required to tackle hate speech at a more fundamental level, says Rice. 鈥淭his technology avoids the issue rather than actually engaging with the problem, which can be uncomfortable,鈥� he says.
Reference:
