Article Lead Image

Sorry, humanity: The Web is now 60 percent bot

It's the rise of the machines.


Tim Sampson


Posted on Dec 13, 2013   Updated on May 31, 2021, 11:53 pm CDT

The robot apocalypse is coming. And it begins online.

OK, so “apocalypse” may be a tad hyperbolic. But new data suggest that humans are a minority on the Web, where 61 percent of all traffic is now generated by automated bots.

That’s a 21 percent increase in bot traffic since last year, according to Incapsula, the firm behind the study. Their previous study in 2012 found that 51 percent of Web traffic at the time came from bots.

But humans need not tremble in fear before their all-powerful robot overlords. Incapsula says the highest bot growth sector is among so called “good bots.” These bots, certified agents of legitimate software, such as search engines, have increased their portion of total Web traffic activity from 20 percent to 31 percent – now accounting for roughly half of all bot traffic.


Chart by Incapsula

“Emergence of new online services introduces new bot types into the pool,” the researchers wrote in a blog post. “For instance, we see newly established SEO oriented services that crawl a site at a rate of 30-50 daily visits or more.”

Even with this increase in friendly bot, nearly one third of all Web traffic comes from malicious bots—spammers, hackers and the like. And evidence uncovered by Incapsula indicates a rise in more complex “bad bots.”

The only sector of malicious bots that showed a rise in activity were the so called “Other Impersonators,” which grew by 8 percent this year. This catch-all group for bots that don’t fit into the traditional spammer, hacker or scraper categories represents a collection of more sophisticated automated programs devoted to nefarious activities.

“The common denominator for this group is that all of its members are trying to assume someone else’s identity,” the researchers wrote. “For example, some of these bots use browser user-agents while others try to pass themselves as search engine bots or agents of other legitimate services. The goal is always the same – to infiltrate their way through the website’s security measures.”

The report says these bots are usually custom-made for very specific activities. Generally considered a “higher-tier in the bot hierarchy,” they often serve as automated spies or “human-like DDoS agents.”

A rise in this kind of bot activity is likely behind the decline in other kind of malicious bots. The total number of hacking tools declined by about 10 percent to account for about 4.5 percent of total Web traffic. And spam saw a massive 75 percent decline in activity, now accounting for 0.5 percent of Web traffic.

H/T CNET | Photo by Gary Forrest/Flickr

Share this article
*First Published: Dec 13, 2013, 2:28 pm CST