Advertisers globally are expected to lose $6.3-billion (U.S.) in 2015, paying for ads that no humans actually see. That is because 11 per cent of banner ads online and a full 23 per cent of video ads were viewed by fraudulent sources such as "bots" and not by real consumers, according to a new study.
And the study's authors have said the real numbers across the entire industry are likely even higher than that.
The research released on Tuesday, commissioned by U.S. industry group the Association of National Advertisers (ANA) from online bot detection firm White Ops Inc., represents the largest study of advertising fraud ever publicly released. It quantifies a massive problem – not just for advertisers, but for publishers struggling to get by in a digital economy and for consumers whose home computers are hacked to play a role in these fraudulent schemes.
The study based its forecast for money lost in the coming year by dividing the roughly $40-billion spent on online display ads and $8.3-billion spent on online video ads by the percentage of fraudulent traffic it found.
The study analyzed 181 online campaigns from 36 ANA-member advertisers across nine product categories – with budgets ranging from less than $10-million per year to more than $1-billion – in September and August. It looked at 5.5 billion impressions – the industry term for each time an ad is seen – on 3-million websites.
"I don't think they know [the extent of the problem.] This report will be an eye-opener to advertisers," said Bill Duggan, group executive vice president at the ANA. "... This is going to organized crime, in many cases. It's bad enough to waste money ... it's worse to have your money going to criminals."
Here's how the fraud works: networks of bots are designed to look like real humans, often by installing themselves on people's personal computers. By surfing around the Web the way a real human might, and being associated with that computer's human browsing habits, those bots start to look like the real consumers that advertisers pay to reach.
A common way fraudsters profit from this practice is by directing bots to visit websites they own, selling ads to companies wanting to reach someone who might visit that website.
They may also engage in "ad injection" – profiting from unauthorized placement of ads on premium websites to poach their ad revenue. The study found more than 500,000 injected ads per day at one publisher's site. This activity can hurt a publisher's reputation, making it even more difficult to sell ads online at reasonable prices.
They are also commonly involved in selling traffic. Websites that sell ads – such as newspaper sites, TV networks' sites, you name it – often make a deal with an advertisers to deliver a certain number of views of their ad, called impressions, in a certain amount of time. If that website's traffic is lagging behind what it promised advertisers, some publishers will pay firms that promise to deliver more traffic. Or it may team up with another website it is affiliated with to find the people the advertiser is targeting there, and deliver ads. That website may not always source its traffic organically. The study found that this kind of paid traffic was 52 per cent bots.
Bot traffic was also higher – 17 per cent fraudulent – when advertisers purchased ads "programmatically," a term for real-time digital trading that makes the ad buying process more automated.
A particularly troubling finding was that 23 per cent of video impressions were not actually humans. That's partly because there is more money to be made in video advertising, which costs more.
White Ops determined the level of fraud by using its technology for spotting the fake, often invisible "shell" of a Web browser that fraudsters operate on a computer that has been hacked.
Both organizations believe the numbers in the study are low, partly because it was publicly announced in July. White Ops saw fraudulent activity drop during the announced study period.
"We know our numbers are underestimating, and they're still terrible," said Dan Kaminsky, co-founder and chief scientist at White Ops Inc. "This is a global problem."
How advertisers can fight it
1) Buy ads during the day: while the study found higher overall bot traffic during daylight hours, bots weren't smart enough to go to sleep when humans do – the percentage of bot traffic online spikes dramatically between midnight and 6 a.m.
2) Yes, White Ops stands to make money from the study, which recommends advertisers use some type of fraud detection technology (not necessarily White Ops's) and monitor their traffic closely.
3) Include websites with good reputations in that analysis. Even high-quality sites can have bot traffic.
4) Demand that publishers and ad/media agencies track traffic and can provide numbers on the reach of a campaign to actual humans. Demand the option to rule out "sourced" traffic purchased from third parties as part of the audience for ad campaigns.
5) Target ad campaigns to people using newer browsers. Older browsers are more vulnerable to bot traffic (such as Internet Explorer versions 6 and 7, which demonstrated 58 per cent and 46 per cent bot impressions, respectively)
6) Don't rely only on blacklists. Fraudulent players can adapt quickly, so these need to be updated at least once a day. Trying to track the bad players is often compared to a game of Whack-a-Mole.
7) Set aside 1 to 3 per cent of the advertising budget for security measures. This level of spending in other industries – credit cards are a good example – has been shown to reduce fraud.
The websites with the most bot traffic
Legitimate and even premium publishers' websites still had bot traffic, according to the study. Here are the percentages of bot traffic detected, by content.
Finance: 22 per cent
Family: 18 per cent
Food: 16 per cent
Travel: 11 per cent
Health: 10 per cent
Home: 10 per cent
Economy: 9 per cent
Fashion: 7 per cent
Society: 7 per cent
Education: 6 per cent
Entertainment: 6 per cent
Business: 4 per cent
Politics: 4 per cent
News: 4 per cent
Tech: 4 per cent
Sport: 3 per cent
Science: 3 per cent
The worst of the worst
The sites with the most bot traffic shared some characteristics:
30 per cent: A minority of sites actually published unique content
51 per cent: Sites that published very similar content across multiple web pages
6: average ads per page on sites with high bot traffic, compared to two ads on average, on more legitimate sites
Red flags: certain types of ads corresponded to bot traffic – for example, ads that automatically start playing video or audio when a visitor lands on the page, as well as pop-ups