« More On Click Fraud, The Lawsuit & The Need For Third Party Auditors | Main | Kanoodle - Computer Generated Clicks »
April 07, 2005
Pay Per Click Advertising Fraud - The Inside Story
As my readers know I do my best to always write from an impartial viewpoint but when talking about click fraud it will be extremely difficult for me to maintain that perspective. I have strong personal beliefs on this issue and many of those philosophies have become ingrained into the way BlowSearch is currently doing business. If this article reads as promotional in any way, my apologies.
Keep in mind that this article is not the "official word" of BlowSearch, it
is my own. If you have specific comments and questions relating to this article
submit a comment to me on my weblog. I'll be happy to answer your questions on
my own site. With that disclaimer in place let's move on.
I will be disclosing some common business practices in the pay per click
business model. Some of these comments will make a lot of advertisers extremely
unhappy and it may draw backlash from some of the engines. Frankly, I don't
really care. My only concern is that someone from inside the pay per click
industry finally address the issue of click fraud, head on, without skirting the
truth. I will be confirming some of the pay per click advertiser's biggest
concerns and telling you the real story about the PPC business model from behind
the scenes. This is my insider information. It's what I know from working within
the search engine market and talking to employees of other engines over the last
(nearly) three years. Out of respect for those people I will not name names and
I will not point fingers. click fraud is an industry wide issue and that is how
I am approaching this article.
Now for a little history lesson on the origin of click fraud.
Cost per click advertising (CPC, hence PPC) actually began in large
scale with PPC banner and text link advertising programs in the adult market
around 5 to 7 years ago. The most educated people I know on the topic of click
fraud come from the adult industry. PPC banner ads and text links was a major
revenue stream for adult site affiliates before PPC finally was fazed out of the
adult market due to click fraud. PPC banner and text advertising was a norm on
adult sites for quite some time. Ultimately, it was also responsible for the
closure of many adult affiliate programs.
Adult companies built huge affiliate networks on the PPC model of paying
affiliates for displaying banners and text links. In the end, many of these same
companies also went out of business because of click fraud. The adult industry
could not stop click fraud back then because the technology was no good. The
same issues exist today with pay per click search engine advertising. Nothing
has changed in over 5 years.
Today click fraud in the adult market is related to "traffic trading scripts".
These scripts automatically trade traffic from one adult site to another in an
effort to maximize traffic flow. Some of the network's out there trading traffic
in the adult space would make Google's monthly search volume figures look
minuscule. There are some huge networks in the adult market controlling more
user traffic than you could ever imagine.
With a traffic trading script in place the user clicks a link on the adult site
and is redirected to another adult site automatically. The scripting does a
little math and sends traffic back to other sites based on the amount of traffic
incoming from that site. Traffic trading scripts suffer the same dilemmas as PPC
advertising. Someone in the network can artificially inflate clicks in order to
earn more traffic from their traffic trading partner(s). It's a "you send me
clicks - I send you clicks back" framework. This process can be abused with an
automated "hit bot" or "click bot". These are similar in nature to the bots that
plague the PPC advertising industry.
Detection of fraudulent clicks is increasingly difficult to accomplish and the
tools fraudsters use to perpetrate their CRIME are becoming more advanced. Click
fraud is as much of a problem now as it was back then in the adult market. In
both cases we're dealing with artificial clicks but with different objectives.
In adult, it's for more traffic and there is no revenue involved. In PPC
advertising, it's for your hard-earned dollars and almost $1 billion dollars per
year is involved.
There are two basic types of click fraud in pay per click advertising. There is
"competitor" and "automated" click fraud. Competitor based click fraud is
when your competitors, a disgruntled employee, someone is paid to physically
click, or just an average user maliciously clicks on your PPC ads. I define
competitor click fraud as any physical click made on a paid link that is
malicious and intended to purposely drain the advertiser's budget.
Automated click fraud is a piece of software, a web server script, or other
mechanism that is engineered or designed to artificially click on pay per click
ads. This is the kind of stuff that is the scourge of online advertising. Many
people believe that competitor based click fraud is the larger issue. That is
what the search engines are telling the public because it is what they want you
to believe. It's PR spin folks. It masks the real problem of what is going on
right now.
Competitor based click fraud is minuscule as compared to automated click fraud.
The automated fraud is the larger issue industry wide. If click fraud
is estimated at 20% of all clicks, then automated click fraud is 19% of all
those clicks. There are some pretty rich software programmers out there who do
nothing but "game the engines" on a daily basis. Engines have no way of
automatically finding this kind of fraud right now. It has to be found with
manual checking. Looking at server logs and checking IP's, referrer URL's, and
other info gleaned from server logs. It is a very tedious and time-consuming
process.
I am not happy about the click fraud situation in the PPC industry.
I do not think advertisers are speaking loud enough about the click
fraud issue. All existing technology to detect fraudulent clicks is severely
lacking in functionality. Pay Per Click networks must be forced to find a way to
automate the solution to click fraud. I spoke to Matt Cutts from Google at SES
and he said that they have a team of "engineers" looking at raw data logs all
day long. Well Matt, Google needs to go back to the drawing board because it is
obviously not working. Google is no better than anyone else is when it comes to
detecting click fraud.
The days of PPC services "snowing" their advertisers into believing "we do
everything possible to deter click fraud"are over.
Why aren't engines directly addressing the click fraud issue? Frankly
it is because it will blow up in their face if they do. Google, Yahoo!,
FindWhat, and every other provider is simply helpless to stop click fraud. They
just don't know how to. Nor do they want to until they are forced to do so.
There is too much revenue involved for them to block all that traffic, or
rather, that revenue.
None of the major PPC providers can turn around tomorrow and say that they have
a "new" solution in place to block "all" click fraud. Why? That would raise the
question of whether or not they have had the solution all along and simply chose
to ignore it because their revenue was growing.
All PPC providers use similar technology to detect click fraud. As a matter of
fact
Digital Envoy is one solution used by Google and a number of other PPC
programs. Just about every engine has a team, or at least one employee,
dedicated to detecting click fraud and getting rid of the sites that cheat the
system. However those employees are only to "effectively manage the bad
traffic". It is not necessarily their job to eliminate it. Remember we are
talking about public companies here with lots on stock and revenue projections
involved. They can't eliminate traffic unless they know where they can replace
the lost revenue first. They have financial numbers to hit every quarter.
Snippet from a recent
New York Times Article:
"There are a lot of players in this game," said Patrick Giordani, the head of loss prevention at Overture, part of Yahoo. "It's our job to stay one step ahead of them." Mr. Giordani described a team of data analysts, statisticians, computer scientists and others working hard at Overture to prevent click fraud. Like Google, Overture does not charge advertisers for many clicks that fail at the outset to pass through a host of filters configured to block suspect traffic. The companies then scrutinize the remaining clicks.
"We do a series of very thorough investigations," Mr. Giordani said. "We're looking at hundreds of thousands of rows of data, looking for unqualified traffic. If we see any, we immediately credit the advertiser's account and update the filters."
In addition to monitoring and enforcement efforts under way at Google, a series of processes built into its systems limit the impact of click fraud, said Salar Kamangar, director for product management for Google. On the most basic level, the Internet remains a measurable medium, he said, adding that advertisers can help prevent or detect click fraud by closely monitoring their pay-per-click campaigns.
"There will be some component of this that will remain with us, but its scope is limited and manageable," Mr. Kamangar said.
Er, WHAT? Since when is a potential $1 Billion dollars a year in click fraud
industry wide 'limited in scope and manageable' Mr. Kamangar? Are you kidding
me?
The sad part is that you, the advertiser, is really buying this garbage!
Look, did you notice that Mr. Giordani said "If we see any, we
immediately credit the advertiser's account and update the
filters." Yes, THAT advertiser. What about all the other advertisers that got
hit by the traffic? AYE, there's the rub!
They don't credit the others and I bet they can't provide documentation to prove
it either. All the advertisers on that specific term during the time frame that
the fraud was taking place received refunds for that traffic? Hmmm.... Somehow I
really don't think so especially if it is a heavy revenue category.
Human eyes trained to look for abnormalities and patterns in click behavior can
only do so much. I have sat in front of a screen for days on end looking through
server logs and still missed obvious click fraud. I am an experienced web
developer and marketer with over 7 years of experience. I have seen thousands of
rows of data. Even I will miss fraud looking at a log and investigating it. You
could have a team of 25 people and they will still miss 25% of the click fraud
scrolling past their face in a server log.
click fraud is a cat and mouse game. When a PPC provider figures out a way to
block a fraudster then the fraudster builds better click bots or switches around
their network. Then the engine spots a new problem and the fraudsters build
another solution. It's a vicious cycle.
The bottom line is that the click fraud detection technology that exists today
is pathetic and the fraudsters know it. Google, Yahoo!, and others are no better
than any other engine. Don't assume bigger companies have better technology in
this area. You'll be sadly disappointed.
Good click fraud detection technology DOES NOT exist, YET.
click fraud in pay per click advertising is a hot topic right now.
At search Engine Strategies in New York I attended the session "Click
Fraud: A Legal Look". Tired as all hell from the Yahoo! Party (and a few more
than that) the night before I muddled my way through the session and took away a
couple of interesting tidbits:
What is the most glaring issue?
You might be interested to hear that at the end of the session the representatives from various PPC providers in the audience were invited up to the front of the room to discuss click fraud issues with interested attendees one-on-one. In attendance were representatives from Google, Overture, FindWhat, Kanoodle, and Enhance Interactive, among others. Point of note. Matt Cutts of Google and I were the only two representatives brave enough to stick around and answer questions. Kudos to Matt because it's not even his job to answer those kinds of questions. All of the other engines ducked out of the room as fast as possible. What does that tell you?
Pay per click advertisers seem to feel that there is an "acceptable level" of click fraud. It's looked at as a "cost of doing business”.
In my view, there is no acceptable level of click fraud, period. Some experts say that up to 20% of all clicks are bogus. I believe the number is much higher than that across all PPC programs. Not just middle market players either. I estimate that click fraud is really around 35% on most engines and can be a higher or lower percentage depending on the amount of unique traffic the PPC provider has on its own domain(s).
click fraud is a much larger issue industry wide than the engines are telling the public. If you are advertising in pay per click I would be severely concerned about the quality of traffic you receive from your campaign. If you are not tracking and screening the traffic you are being provided you are ....flat out...bottom line ... ... foolish.
Look, click fraud is not acceptable. As an advertiser you should be policing your listings, tracking your ROI, and making absolutely sure you can trust the providers you do business with. Implement a click fraud detection system from one of the many providers out there who specialize in click fraud detection. Is their technology capable of catching all click fraud? No, but even if they know a little more than you do it does not hurt to use one of their solutions. It might cost you a little extra but at least you'll have a third party looking out for you. Some of these companies include:
...among others who are out there.
Most people feel the "middle market" or "second tier" PPC advertising companies
are rampant with click fraud.
Web searchers do not go to findwhat.com, goclick.com, 7search.com,
xuppa.com, kanoodle.com, enhance.com, to perform their web searches. Those
domains have NO branding to the average search user. Therefore they have very
little unique traffic of their own. Only search engine marketers and advertisers
know about these sites and what they are for.
The traffic advertisers are getting from middle market pay per click programs is
generated by networks of "distribution partners" who display the PPC results of
other programs. These partners are typically called "affiliates”. This means
that because SearchFeed displays pay per click listings from FindWhat in its
search results, that SearchFeed is an affiliate of FindWhat (Yes, I know
FindWhat owns SearchFeed.). On the same token since MSN search displays paid
listings from Overture that means that MSN Search is an affiliate of Overture.
Small and large publishers alike, whenever another site carries PPC listings
from another web property, it makes that site an affiliate.
Is the middle market rife with click fraud? Let me point something out here. The
middle market does not have any more click fraud than the top tier PPC providers
do. Here is my hypothesis and I think it bears some weight.
What we're dealing with is a "percentage game". Sites like MSN, Yahoo!, or
Google have a much higher percentage of visitors stemming directly from their
own domain(s). Therefore the ratio of fraudulent clicks to real clicks on top
tier sites would be a much lower percentage than with middle market PPC
networks. The fraudulent clicks are easier to hide on bigger PPC networks
because the massive volume of users buries them. Yes, there is the aspect of
bigger PPC's have a better network of partners, and hence, better traffic as a
result. There is a lot to be said for that but that's not necessarily true. By
the way of example only, who is to say MSN doesn't run click bots on their
Overture advertising? There is so much traffic that they could feasibly get away
with it and maybe never get caught.
Middle market providers rely heavily on their network of affiliates to create
traffic to PPC ads because they have little user traffic on their own domains.
They must deliver their PPC listings to their affiliates in order to gain
traffic. This can result in higher click fraud percentages simply because there
is much less traffic to go around. The percentages are skewed and out of balance
in proportion to the top tier engines. If the middle market players had more of
their own traffic their situation would not be as dire relating to click fraud,
and their reputation in the industry would not be as bad.
BlowSearch actually does 30 million unique searches per month on its own
domain(s) so we're a little different than most middle market players. We "own"
quite a bit of our own traffic. There's the
BlowSearch Toolbar and the
BlowSearch Secured Instant Messenger thrown into the mix too. The company is
focused on tools to retain users which makes better traffic for the advertiser
in the long run. However, even BlowSearch has affiliates who display our
advertiser's listings and those affiliates need to be monitored for quality.
Should advertisers ignore the middle market search engines? Yes, with
exceptions. Since none of them will come clean as to exactly what they do, then
guess what? You should not be advertising with them. It's just common sense. Why
advertise with a company that is going to rip you off, knowingly or unknowingly,
on their part? This same logic should apply to the top tier PPC's too as far as
I am concerned.
Did the SVP of Marketing for a middle market PPC player just say that middle
market traffic should be avoided by advertisers?
Yes I did, with exceptions. The exceptions can only be
determined by educating yourself on what to look for in a reputable search
engine pay per click advertising program and being proactive in detecting click
fraud. It is your money. If you don't care how it is being spent why do you
expect a pay per click network to care?
If you find click fraud you need to work with the PPC provider so that they can
block it - if they CHOOSE to. Most often a PPC provider will not completely
block a whole traffic source? Nope, blocking on most PPC providers is done on
the account or individual keyword level. In most cases they DO NOT eliminate the
traffic source responsible for the problem. They just warn them and if too many
advertisers catch on that the traffic is bad - then the source gets eliminated.
Keep in mind there is a difference between traffic that is legitimately click
fraud and traffic that "just didn't work for me”. Most advertisers think that
just because their sites did not convert into one sale with a $50.00 PPC ad
spend and 10 keywords in their account, that the traffic must be bad. This is
just not true. It takes a lot more analysis than simply saying, "engine XYZ
sucks because they didn't send me a sale”. How much did you spend? Did you track
it? What real proof do you have to claim click fraud? Could it possibly be your
own web site that is the culprit of your poor conversions? "NO, never, not my
site!" Hmmmmm.
More than 70% of the time advertisers have no idea what their results are with
their pay per click advertising. They are not tracking their ROI even though the
tools are available to do it. That's foolish on the advertiser's part, and in my
opinion, irresponsible sales and marketing by the PPC providers for letting it
happen. Education is the key to success. Educate your advertiser and you have a
happy advertiser.
If I had it my way every advertiser would be forced to use ROI tracking or a
campaign would not be allowed to go live. Would you believe that there is an
incredible amount of resistance in getting the advertiser to implement one line
of HTML code on their web page in order to make ROI tracking work? You have no
idea how hard cutting, pasting, and uploading one line of HTML really is! The
tool is there and the advertiser still does not care to use it. By the way,
these are the same advertisers who come running back screaming about click fraud
even though they can't prove it.
When given refunds advertisers expect a detailed explanation and
breakdown of why an account credit is issued. Including a detailed report of the
abuse, who is responsible, and the methods used to detect it.
Advertisers have every right to expect PPC companies to disclose what
happened to their advertising dollars. It is the advertiser's hard earned money
that PPC providers are spending. However a detailed report of fraudulent
activity is not something the PPC provider will deliver. Why? It would mean that
they would have to refund every advertiser who received traffic
from that bad source. They prefer to refund only their premier advertisers
instead.
If you don't tell the advertiser specifically what happened to cause the refund
then there is no risk that other advertisers will seek refunds related to that
traffic source. This is sneaky, but it's a common practice across all major PPC
providers. They keep the info in house so that they do not have to massively
issue refunds to advertisers. It helps them retain revenue. Oh, I bet Sergey,
Larry, and Eric are fuming mad at me right now! I'm glad I don't work for
Google...
Do no evil, right?
Most advertisers believe that the PPC programs will never fully eliminate click
fraud because it is not in their financial best interest to do so. By not fully
disclosing click fraud PPC advertising programs minimize their revenue losses.
Most PPC services do selectively issue refunds. Yes, there is even
preferential treatment given to the largest advertisers. From largest to
smallest PPC provider they are all hiding this information from you.
Yes, it is true folks. PPC providers don't tell you about their bad traffic
because they simply don't want you to know. As I said before, there is far too
much click fraud out there and none of them truly know how to stop it. So as
long as they keep you in the dark and give you lip service about "we have
dedicated teams of engineers consistently monitoring our traffic" then you'll
leave them alone. Advertisers seem to have bought into that rhetoric. It's a
complete line of bull folks. None of the engines have a clue how to completely
stop click fraud and they won't because there is too much money involved.
Advertisers are certainly not forcing PPC networks into disclosure with their
wallets, so why should they disclose anything? If you are a PPC advertiser then
you're being complacent. It's not the PPC's fault. You haven't stopped
advertising long enough to force them into changing their ways. You haven't made
them accountable. I personally think that you should.
Remember that all of the PPC's are limited by the technology they have to detect
click fraud. Until better technology comes along there will always be fraudulent
clicks that get through the system. When they do catch it should they issue
account credits across the board to all advertisers? ABSOLUTELY! But they won't.
Most engines are too large in scale to implement a working solution to prevent
click fraud even if they had a clue on how to stop it. They would have to rework
their entire infrastructure and place their click fraud detection system on
every machine handling their click through traffic. This causes huge burdens on
the click stream, slows page displays dramatically, and is just not practical to
implement. Your platform must be capable of handling the new scale required to
process the data if you put a solution in place. It must be built into your
system from day one or you need to rebuild your entire infrastructure to make it
work.
People don't realize that there are ways to eliminate most, if not all, click
fraud from pay per click advertising.
The solution to the click fraud problem lies solely on the shoulders of
the PPC's to invent the technologies necessary to stop click fraud from taking
place. The pay per click industry is looking awfully dirty lately. How do we
clean it up? Force the engines to change their ways. Speak with a louder voice
and withhold your ad dollars from the engines that don't comply. Work with
SEMPO, the
SMA-NA, and other marketing organizations to bring these issues to the
forefront of the industry. Nothing speaks louder than money. Choose not to spend
a dime until the engines solve this problem.
Look, I will always be a site owner first and a search engine employee second.
I'm on the advertiser's side on the click fraud issue. I'm going to do
everything in my power to make sure the issue gets addressed.
Am I getting through to you? I don't care if your conversions have never been
better. I assure you that they would be even better if you knew for a fact that
all the traffic was legitimate. Take back control of your PPC advertising. It's
on you, the advertiser, to force the industry to change. If you want to keep
getting "snowed" by PPC providers that's up to you. I told you here how it all
works. The choice is yours to make. As the Verizon guy says in the
commercials...
"Can you hear me now?"
If interested please see
Part II. It
is more promotional but it will tell you what BlowSearch is doing to combat
click fraud.
Posted by Hans A. Koch at April 7, 2005 10:53 AM