CONTACT US
Search Engine Marketing Group
303 Second Street
South Tower, 9th Floor
San Francisco, CA 94107
415-615-0710 x226
Email Us
http://www.jwtsc-seo.com
IN THE MARCH ISSUE...
GOOGLE: CLICK FRAUD IS 0.02% OF CLICKS
Danny Sullivan, Search Engine Land, 3/01/07
Finally, we have a click fraud rate from Google itself: less than 0.02 percent of all clicks slip past its filters and are caught after advertisers request reviews. That low figure is sure to bring out the critics who will disagree. Below, more about how Google comes up with the figure plus some click fraud fighting initiatives it plans to implement later this year.
Why release this figure now, when many have wanted it for literally years?
"We've been working to be more transparent and informative on the issues related to click fraud. Recently, this metric has been something advertisers have specifically asked for and we agree that is useful in describing the scope of the problem. Further, it is something we measure and use to monitor the performance of our click fraud detection systems," said Shuman Ghosemajumder, business product manager for trust & safety at Google.
To understand the figure more, I want to revisit some of the click fraud terms I explained back in December. The first term below is from Google itself. The rest are terms in common use, though people might define them differently. That's why I'm providing my own definitions.
- Invalid Clicks: This is a Google term. It means that of ALL the clicks Google ads generate, a percentage of these are deemed "invalid." These are clicks that Google makes no money from. It either never bills for invalid clicks or eventually issues a refund. Not all invalid clicks are necessarily fraudulent (such as in cases of quick double-clicks that aren't billed).
- Fraudulent Clicks: These are the percentage of clicks that an advertiser is charged for when the person (or bot) clicking is doing so to purposely cost the advertiser money or make money for themselves through an ad program.
- Overall Click Fraud Rate: This is the number of fraudulent clicks as a percentage of ALL clicks that happen, regardless of whether Google issues a refund or doesn't bill.
- Detected Click Fraud Rate: This is the number of fraudulent clicks as a percentage of ALL clicks that happen where Google generates a refund after performing a requested or "reactive" investigation asked for by an advertiser. Google would call this the "reactively identified invalid click" rate.
- Undetected Click Fraud Rate: This is the number of fraudulent clicks as a percentage of ALL clicks that happen where no refund has happened because neither Google or the advertiser spotted the fraud.
Google could have an extremely high overall click fraud rate, yet that's not a problem if most of those clicks are detected and either never charged for or refunds issued promptly. When I talk about the Google "click fraud rate," I specifically mean the detected click fraud rate -- clicks that got past Google's own filtering systems and only were refunded after a manual investigation was requested.
As for the undetected click fraud rate -- that's impossible to know. There are clicks that might get past Google and advertisers both. Potentially, this could be a high percentage. However, if the detected rate is low, I tend to think the undetected rate is also low or near the same level.
Next, let me recap Google's three step process for catching click fraud. You're going to hear lots of people talking about this in relation to today's news, so it's helpful to review:
- Proactive Filters: Google automatically watches for activity and throws out clicks without billing. These are the "invalid clicks" I mentioned earlier. This is a "proactive" filter in that Google does it proactively without being asked by the advertiser.
- Proactive Offline Analysis: Google uses both automated and manual checks on clicks after billing, in particular clicks on websites using running AdSense For Content. This is also a proactive filter, done without being asked. It's this work that causes advertisers to sometimes get credits out of the blue.
- Reactive Investigations: This is where the advertiser comes to Google concerned about charges. Google says all queries from advertisers are investigated and that refunds are relatively rare.
Now let's talk percentages. First, less than 10 percent of all clicks are deemed invalid -- clicks that either are never charged for or where refunds are issued.
"The overall rate can fluctuate. We've said before that this is in the single digits and want to clarify that this means it's from one to nine percent. It's not zero, but these are lower bounds," Ghosemajumder said.
Sound too low? Ghosemajumder anticipates this, pointing out: "Even one percent of Google's revenue is a significantly large amount of money, more than the actual click fraud settlement," referring to last year's class action case.
What about the main mystery figure that I and others have been wanting -- the detected click fraud rate, the percentage of clicks that got past all Google filters and would have cost advertisers money, had they not raised an alarm? The detected click fraud rate is less than 0.02 percent of all clicks. To use Google's term, this is the rate for "reactively identified invalid clicks" -- refunds given after advertisers asked for an investigation.
It's low, and Google says it's been getting lower all the time.
"Our system have gotten better and better at catching this stuff," Ghosemajumder said. "Not only is the percentage a small amount but percentage as revenue overall is smaller."
As I said, no doubt critics are going to be in disbelief, especially when you have recent reports like that from the Click Forensics' Click Fraud Index of an average industry click fraud rate of 14.2 percent for last quarter.
Google's answer -- which is pretty compelling -- is that some reports such the Click Fraud Index don't take clicks that aren't billed or refunded into account. In other words, clicks might be flagged as suspicious or fraudulent in the monitoring system, but there's no reconciling to see if they actually resulted in a final charge.
Think of it as a credit card bill. Perhaps at the end of the month, you've got 500 charges. You're concerned that some of those might be fraudulent, so you hire someone to comb through your bill. They flag all the suspicious items and declare you have a 15 percent fraud rate. However, they don't see the part of the bill showing credits issued to you. If they had, some -- maybe many -- of those "fraudulent" charges resulted in refunds. That would lower the fraud rate.
It's not an entirely accurate metaphor, but hopefully it gets the main point across. To know a click fraud rate, you have to know what clicks were and were not charged. Some auditing companies do this -- looking at actually billed records, then combing through and raising those "reactive investigations" mentioned above. But talking about a "click fraud rate" without first removing clicks that were never charged for or refunded isn't a useful figure.
It also gets more complicated. Ghosemajumder was adamant that it's not just a case that Google says the slice of click fraud is smaller than what auditing firm may find. He says that the chunk of clicks that Google catches as invalid might be significantly different than the chunk of clicks an auditing firm considers suspicious, because of different tracking methods. For more on this, see Why Third-Party Click Fraud Estimates Don't Add Up and Why Third-Party Click Fraud Estimates Don't Add Up - Part 2 that he posted on his personal blog last year.
Google, of course, reports invalid clicks to advertisers through their advertising accounts. That started last July. Those individual reports could be much higher or lower than the average, and Google says there's definitely variations between advertisers and advertisers in particular vertical market segments.
Going forward this year, Google plans four things it hopes will help combat click fraud and reassure advertisers more:
- IP Filtering: Advertisers can block particular IP addresses that they think are fraudulent. When this happens, people coming from these IP addresses will not see ads.
- Invalid Revenue & Enhanced Reporting: Advertisers will get more detailed information that currently provided.
- Invalid Clicks Resource Center: Further guidance about filtering and click fraud questions will be added.
- Standardized
Inquiry Reporting:
Google already has a reporting form for advertisers. But some advertisers
want to provide even more information, logs, and other data. So Google
says it will explore a more standardize way to take material in.
THE STATE OF SEARCH ENGINE MARKETING 2006
Chris Sherman, Search Engine Land, 02/08/07
Search marketing is thriving, according to SEMPO's annual "State of Search Engine Marketing" survey, with North American advertisers spending $9.4 billion on search engine marketing in 2006, a 62% increase over 2005 spending.
SEMPO collected data on spending trends on paid placement, paid inclusion, organic search engine optimization (SEO)and SEM technology platforms. The survey drew 587 respondents from both agency and in-house advertisers, and was conducted in November and December 2006 by Radar Research, LLC and Intellisurvey.
Where is the money being spent? Organic search optimization is still the most popular form of SEM, with almost three-quarters of advertisers using this method, with paid placement a very close second at 71%, even though total spend for paid placement was greater, accounting for 86% of total spending, up slightly from 83% last year, at $8 billion.
By contrast, even though the most popular form of SEM, organic SEO accounted for just $1.1 billion in spending, or 12% of the total, also up slightly from last year's 11%. Paid inclusion continued its decline, down from 4% last year to 1% of total spending, or $94 million this year. SEM technology platforms account for the remainder, at 1.3%, or $122 million in spending.
And no surprise: Google's AdWords is the most popular search advertising program, used by 96% of respondents, followed closely by Yahoo Search Marketing, with 86%. Microsoft turned out to be a real dark horse last year, however, with 68% reporting they are using the company's new adCenter program, up from just 29% in the previous year.
What's search marketing being used for? Last year, the SEMPO study found that the majority of search marketers (62%) said branding was the primary objective of search marketing campaigns. Nearly as many, however (60%), said that selling products was a key objective. This year, direct sales was the top choice, at 58%, followed by brand awareness at 57%.
Despite this, less than 21% track or measure branding impact, but 73% track increased traffic volume, 71% measure conversion rates, and 68% track click-through rates.
Other key takeaways from the report:
- 75% of advertisers said they could afford a mild increase in paid placement; the remaining 25% say they have topped out in cost per lead spending
- SEM continues to poach budget from other marketing channels, especially offline marketing programs
- In-house marketing programs continue to grow, portending further consolidation on the agency side of the business
Another interesting finding was that for the first time, a majority of senior executives (52%) consider SEM a high business priority. Kevin Lee, member of the Board of Directors of SEMPO and chair of its Research Committee, said "This is important for the continued success of the industry, and a great sign."
Lee also said, "One thing that I feel is important to note (and suggests additional study) is that in-house SEM professionals who cared enough to take the survey may have different opinions about SEO and paid search than their superiors.
What about the future? Spending will grow to $18.6 billion by 2011 in North America, according to SEMPO, and will start to plateau about then as the industry matures. Growth will be driven by strong advertiser demand, rising keyword pricing, and perhaps most importantly, a second wave of small-to-midsized businesses discovering the effectiveness of search marketing. Vertical and local search will also show strong growth.
This is the third annual State of the Search Marketing survey released by SEMPO. I covered the first two reports over at Search Engine Watch, for 2004 and 2005.
The report is available for download at the SEMPO web site.
RELEVANCE IS RELATIVE - SOME THOUGHTS ON [GOOGLE] QUALITY SCORE
Mark Simon, MediaPost Search Insider, 02/28/07
As of last week, Google now tells advertisers how their ads’ Quality Scores rate. Advertisers still won’t know their precise Quality Score numbers, but they will get grades of “good,” “mediocre,” or “poor,” as well as a sense of how that score is affecting their click cost. That’s a huge change for Google, which is generally known to search insiders as an enormous black box.
But it’s clear why Google would want to tell advertisers a bit more about their ads’ quality. Quality, as Google has it, means relevance; delivering Quality Score grading is Google’s gentle nudge that advertisers create more relevant ads. It’s all part of Google’s effort to make paid listings more helpful to searchers — which ultimately brings the engine more click revenue.
And if Quality Score is making paid ads more relevant for searchers who are looking to shop, it’s worth asking where the future of organic search really lies. I think the answer to that question is found in the two different types of searchers out there — researchers and shoppers. Down the road, I see organic results as geared exclusively toward researchers. Paid search will be geared towards shoppers.
Organic rankings, after all, are designed in a way that suits researchers a great deal, but might not be the most informative way to guide a purchase. Organic rankings tell searchers which pages have the most to say on a given topic, which is exactly what researchers want to know.
But shoppers aren’t looking for Web pages. Nor are they really looking for information. They’re looking for the businesses that offer the best value and the best customer experience. They want to know about things like price, shipping cost, customer satisfaction, and rewards programs. For that kind of information, it’s far more valuable to match businesses to keywords than it is to match Web pages to keywords.
It’s search’s paid ads, not its organic results, that make those business/keyword matches. And it’s search ads that provide additional shopper-relevant information — the information that’s not directly related to the search term, but that shoppers are looking to find. It’s search ads that routinely add in information about special pricing, great selection, or free shipping.
In other words, while organic rankings are based on content relevance, paid search operates on purchase relevance. And for shoppers, purchase relevance has a whole lot more value.
Meanwhile, that purchase relevance only increases as Quality Score and other quality-ranking systems become smarter and more deeply entrenched. That’s so, because quality ranking systems offer discounts for high click-through rates. An ad’s click-through rate is largely a product of how purchase-relevant its copy really is — so the engines are incentivizing advertisers to increase purchase relevance.
Which is why we’re heading toward a search universe in which paid ads become significantly more relevant for shoppers than the organic listings ever will be. At the same time, organic rankings were never best-designed for product searches to begin with.
Given that, a logical next step would be for the search engines to divide the search results completely. Text-heavy pages that are likely to be research-helpful get pushed to the top of the organic results. Pages that are commerce-based get pushed down. This will make it easier for researchers to do real research, without being bombarded with commerce pages that won’t offer them much information. It will also drive more purchase-focused searchers towards paid listings, as they won’t get distracted by research-focused organic results.
Which would, of course, be a win for both engines and searchers. The searchers would get more relevant results, no matter what kind of search they’re looking to run. The engines would get more clicks on ads, which would bring them more money. Advertisers would also get their ads clicked more often — which means more business for them.
Just one clarification before I finish. I’m not suggesting that the engines will knock businesses out of search listings. I’m suggesting that the engines will ultimately look to knock commerce pages down in the rankings; they’ll keep content pages of all kinds higher up.
Take CreditCards.com, whose home page has the No. 1 organic ranking in Google as I write. While CreditCards.com has truly valuable articles that could be of value to a researcher, those articles don’t live on the home page — they live on a deeper page within the site. The home page is focused on driving visitors to fill out credit card applications, which is CreditCards.com’s business. In the new organic ranking I’m envisioning, CreditCards.com’s articles could still hit the top Google organic listing — but its current home page wouldn’t.
Scott Van Achte, StepForth SEO Blog, 02/28/07
Near the end of 2006 Yahoo officially unveiled the new back end for Yahoo Marketing Solutions, widely referred to as the ‘Panama’ Update. Since then they have been slowly allowing the upgrading of accounts from the old into the new system. While not everyone has had a chance to have their accounts switched over, it is expected that all will be upgraded by the end of this quarter.
After several months of waiting, this new backend is a welcomed change as Yahoo finally moves into the future but as with any new system, it is not without its pros and cons.
New System Pros
The new system is certainly a great improvement over the old. While adjusting to a new interface and ranking algorithm may take a little time for advertisers, the end result will present more relevant ads to the searchers, which will ultimately mean a higher quality of visitor and should provide a better bang for one’s buck.
- Previously you were stuck with only one live ad for any given phrase; now you can create multiple ads for a given ad group. With the ability to have multiple ads for each phrase opens the doors to easier testing of which ad copy is better suited for your ads and prospective clients.
- Ad approval is now instant for most ads and keywords. While there is still a “pending approval” process for some sets of phrases, in many cases your ad changes and keyword additions are now instant. No longer do advertisers have to wait a number of days for their ads to go live.
- With the old system, the level of geographic targeting was very limited. Under the new Yahoo Panama the options are much more sophisticated. For example, there is an option for blocking entire continents for your ad campaigns. Rather than selecting each continent/country you want to appear in, if you are only trying to avoid one specific region, you can easily block it from your campaign.
- The geo targeting options have also expanded to allow for more specific ad targeting. Advertisers now have the ability to target specific states and provinces – a feature previously unavailable. Canadian advertisers were forced to advertise to the US even though they wanted ads only to appear for the Canadian market. This update now provides the option to target Canada only – a big plus for any advertisers who are only able to sell in the Canadian marketplace.
New System Cons
There are a few negative aspects that have been widely noted in many forums and by Yahoo advertisers. As with any new design and back end, it is expected that there will be some wrinkles. Most of the problems are relatively minor, and for advertisers on top of their campaigns, these should not present any major issues.
- One of the downsides most discussed is the transfer of account stats from the old system to the new. Once your account has been transferred, the new system will not log any of the old system historical data. While this information is still accessible by logging into the old interface (which is made read only), it would certainly make more sense if stats were available under the new account login.
- Another issue campaign tracking. After the switch has been made, as long as you do not change any tracking settings everything should continue to work as normal; once you adjust the analytics or tracking options, you will need to replace your tracking code with a new piece of code provided by Yahoo. There have been reports where after the account is upgraded, the conversion stats no longer appear and the code needs to be immediately replaced. While not everyone seems to be having this problem, it is certainly a bug worth noting. If you have not yet upgraded, or upgraded but not checked your tracking codes, it would be best to tackle this issue right away to ensure everything will work correctly.
- There have been some reports that after the account transition some ad titles, descriptions, and even keywords, have been disrupted. Upon upgrading your account be sure to check all your ad copy and phrases to ensure that they are still correct or you could end up spending money on incorrect phrases, or damaged ad copy.
The system is certainly not without its flaws, but generally the interface is much cleaner and certainly more functional. While the ability to edit many more aspects of one’s account are now in place, it can be a little daunting to find the right place to make the change. Some items such as campaign and ad group settings are more difficult to find than they need to be, and require more steps than is really necessary. As an example, below I compare the steps required for the simple task of renaming a campaign for Yahoo versus Google:
Yahoo
1. Log into your
account. You will be presented with your account dashboard.
2. Click on the name of the campaign you wish to change
3. Click on “Edit Campaign” in the top right.
4. Click “Edit” at the top right of the “Campaign General
Information” box.
5. Change the account name.
6. Click “Save Changes”.
7. Then click “Dashboard” to return to where you were.
Google AdWords
1. Log into your
account. You will be presented with the “All campaigns” view.
2. Click the checkbox beside the campaign you wish to rename and click
“Edit Campaign”.
3. Enter a new campaign name.
4. Click “Save Changes” (also brings you back to where you
were).
Why is it that Yahoo takes 7 steps to do something that Google can do
in only 4? Not to mention multiple campaigns edits. If for some reason
you wanted to rename all your Google campaigns at once, it would still
take only 3 main steps plus 1 step for each campaign name changed. For
Yahoo you would need to repeat all 7 steps for each campaign. These same
basic steps are also required for other edits such as campaign budgets,
start and end dates, and to turn a campaign on or off.
This example is not really a big deal in the large scheme of things, but is simply an indication that while Panama is a large improvement, there is still much room for refinement. As Yahoo moves forward into 2007 I am sure we will see more advancements and changes to the main navigation.
Click Through Rate Increased by New Ranking Algorithm
Along with an entirely new backend system loaded with new features, Yahoo has also adjusted the way in which they rank ads. The new ranking algorithm is very reminiscent of Google AdWords, and miles away from the old bid-for-position model previously used. Under the old algorithm an advertiser could dominate the top ranking simply by having deeper pockets. Now under the new Panama algorithm to guarantee top spot for a competitive phrase, not only may you still need deep pockets, but you also need the right phrases, relevant ad text, and a quality website – giving an opportunity for top rankings to those with smaller budgets.
According to comScore there has been s significant increase in the overall click through rate of Yahoo paid ads since the introduction of the new ranking algorithm. Compared to the week ending February 4, the last day of the old system, the week ending February 11 saw a 5% increase in clicks. By February 18th the total increase in clicks was reportedly at the 9% mark; a sign that the new system is of significant value to advertisers, and of course Yahoo. Now these figures may have been skewed slightly with both Valentines Day and Presidents Day falling into that range, however, “Bank of America analyst Brian Pitz said in a research note that he expects click through rates to grow about 15 percent to 25 percent starting in the second half of the year.”
What has been specifically responsibly for the increase in traffic? While it may be too early to know for sure, the most likely reasoning is ad quality. Under the old ranking schematic it didn’t matter what your ad looked like or where you directed traffic, if you had the dollars, you had the rank. Now that there are other factors at work, the most relevant, high quality ads, take the rank, and searchers are obviously noticing the increased relevance and clicking through.
Scott’s personal take on the new system
Generally I have
to say I like the new system. It makes account management easier than
using the archaic system we had all grown used to. The new system is much
more streamlined and clean cut compared to the old. That said, when compared
to the Google AdWords, it still falls short. It’s a great first
try, and within the next year or so I expect to see some more adjustments
to make navigation and functionality improved. For a system with so much
hype and such a long time coming, it seems less desirable than expected.
Editing ad copy and URL’s is still much more cumbersome than its
competitor Google, requiring more steps. The account transition could
have been made smoother and items such as historic stats should have been
made more easily accessible and transferred over to the new account.
Since its early days the Google AdWords system has evolved a great deal,
and the same can be expected from Yahoo. Now that they have implemented
a more scalable and comprehensive system, it will certainly improve as
advertiser feedback is received.
One irrelevant feature, if you can call it that, is that I am very grateful for the removal of the “security code” requirement when signing into the account. While I can appreciate the reasons for having it there in the first place, these security codes that were popping up everywhere really did drive me crazy, and it is nice to see this condition being removed. What does it mean to an advertisers account? Well, really nothing, but it does put a small smile on my face.
While it is not without its short comings, this new system is a large improvement and I for one welcome it with open arms.