THE internet is quite popular. The billion people were online 鈥� over half the world鈥檚 population. That is hardly news. What鈥檚 more surprising, perhaps, is that human users are almost outnumbered by non-human ones. Governed by coded instructions, these bots creep around in the background, largely out of sight, browsing websites, clicking links, downloading content and typing text. What are they up to?
We need to know. Although for the most part we still outsmart these free-roaming bots, they are growing more intelligent and destructive, capable of destabilising everything from financial markets to public debate and even our shared sources of knowledge. Only by understanding their secret, complex ecosystem can we hope to tame them.
Advertisement
If you have been online, it is almost impossible for your experience not to have been shaped by bots. These automated chunks of code, preprogrammed to perform a certain task over and over again, account for as much as 39 per cent of activity on the web, says Matthew Prince, CEO of Cloudflare, a firm that helps companies securely manage web traffic. As much as 1 per cent of that comes from Google鈥檚 army of web crawlers, known as spiders, which scuttle around websites plucking out information like text and links that determine where pages appear in search results. Every search engine, from Bing to Baidu, has its own spider armies too.
Other bots help to maintain order on the web. Monitoring bots are digital meerkats on constant alert, periodically checking websites for bugs, poor performance and outages. Moderator bots live inside social media platforms such as Reddit, Twitch and Discord, automatically hiding or flagging inappropriate content. The same spiders that Google relies on can also scour the web for copyright infringement and stolen content.
Bots play a critical role in cybersecurity, too. Defenders and hackers alike use them to scan for flaws in the code of websites, either to patch up holes or exploit them. 鈥淭hey鈥檙e like bloodhounds,鈥� says Prince. 鈥淭he bots are almost acting like a scout, running around looking for some new security vulnerability.鈥�
But the relative anonymity of bots makes them popular agents for digital crime. One common technique is to use a virus to hijack thousands of computers and run bots from them, like a parasitic fungus using a host to spread its spores. Most commonly, these networks of bots are used for 鈥渄istributed denial of service鈥� attacks, in which they flood websites and jam up their servers. Scraping bots, meanwhile, flit about like digital magpies looking for shiny objects to hoard, including email addresses to feed into spam bots and valuable web content that can be reused on copycat sites to pull web traffic and make advertising money.
For some big firms, bots are their eyes and ears. E-commerce giants such as Amazon and Walmart are engaged in a never-ending bot war, gathering information on competitors鈥� products and pricing while fending off nosy rivals. 鈥淪ome of the brightest minds I know are working on trying to figure out how to corral these bots and to manage them,鈥� says Patrick Sullivan at Akamai, another company that manages and secures clients鈥� web traffic.
These automated processes can lead to absurd outcomes. In 2011, bots belonging to two competing Amazon booksellers got locked in an algorithmic loop when setting the price of an obscure academic tome on fruit flies. One was scraping prices to ensure it was always around 27 per cent more expensive than alternatives, while the other set its price just below those of competitors. The final asking price was a dizzying $23 million.
For most of us, social media is by far the most likely place we will knowingly or otherwise encounter bot activity. Millions of automated accounts post, share and follow, often disguised as human users. Swarms of these bots are wielded by spammers or nefarious actors aiming to stoke division and promote disinformation. During the 2016 US presidential election, Alessandro Bessi and Emilio Ferrara at the University of Southern California found that more than 400,000 bots were politically active on Twitter.
Not all social media bots are malicious. In 2019, Islamic prayers from users鈥� accounts is responsible for up to 10 per cent of Arabic language tweets. Alongside the trolls and provocateurs are also benign 鈥渁rt bots鈥� that post cloud pictures or craft satirical tweets, many of which are hugely popular.
鈥淚t鈥檚 so sweet seeing people develop this love for this sort of fictional character,鈥� says Nora Reed, a prolific maker of Twitter bots, including one that parodies op-ed titles. Such bots take a finite database of phrases and simple rules for recombining them, but their output can often be surprising. 鈥淭here鈥檚 so much room for playfulness and satire,鈥� says Reed.
Now, however, bots are getting radically smarter. Thanks to advances in AI, they no longer need content themselves with doing just the same thing over and over. Social bots can adapt to their environment and learn from their interactions with internet users. Even simple bots can exhibit complex behaviour when their networks get large enough, says Iyad Rahwan at the Max Planck Institute for Human Development in Berlin, who studies human-machine interactions. 鈥淚f you look at biology, you see many examples of extremely sophisticated collective behaviour that emerges from very simple rules,鈥� he says 鈥� termite colonies collaborating to build towering cathedral-like structures, for instance. 鈥淵ou can imagine swarms of bots could potentially also synchronise and create collective phenomena online.鈥�
Hints as to where this might lead come from financial trading. In 2010, interactions between high-frequency trading bots wiped a trillion dollars off the stock market in a 鈥渇lash crash鈥�, although losses were regained in just 36 minutes. Another example was spotted by Taha Yasseri at University College Dublin, who in 2014 noticed that bots designed to clean up Wikipedia articles were engaging in attritional edit wars stretching over years. Both cases are cautionary tales of how unregulated interactions between automated systems can get out of hand. 鈥淲e need to understand this jungle,鈥� says Yasseri.
鈥淭wo bots got locked in a bidding war on an obscure academic tome on fruit flies鈥�
Rahwan thinks this growing complexity means we should be using tools and perspectives from ecology and economics to understand the collective behaviour of bots. This means looking beyond their underlying code to consider how their environment shapes them. It also means investigating how a bot鈥檚 adaptations affect its success or failure, and how human intervention shapes the evolution of bot technology.
Lessons gleaned from these approaches should help develop new tools to detect and understand the impact of bots and guide regulation. More transparency over where bots operate will be important, says Rahwan. That could even involve segregating online areas in which you can be confident that you are dealing only with humans. This might mean having to verify your identity on social media or receive warnings if customer service channels are managed by chatbots.
The clock is already counting down for getting to grips with the world of online bots. Rapid advances mean pretty much anything that humans do online could soon be done by bots, says Rahwan. 鈥淭he whole ecosystem is just going to be extremely complex.鈥�
