A recent change at Google had a lot of people in the SEM industry puzzled. Last Thursday, the tool that measures the PageRank Google assigns a page or document went blank, showing a gray PR0 for every site in its index. The bar remained blank until early this afternoon when PageRank values became visible to searchers on the east coast of North America. It is expected to be fully active everywhere in the next 24-hours.
Even after Google warned consumers that the Google PR Toolbar should be viewed for entertainment purposes only, the handy gray to green bar produced some of the most widely quoted website statistics. When the only stat it showed was a universal gray bar, webmasters worried. Gray bars are generally associated with a site being dropped or penalized by Google. When they found out that everyone was seeing gray and that they had nothing to worry about, most seemed to miss the tiny status meter.
For over three years, the PR Toolbar provided a relative view of the importance of a website in Google's view on a sliding scale of 1 - 10. When a site was assigned a high PageRank score it tended to perform better in Google searches and links from that site were assumed to be stronger than links from a site with a lower PageRank score. As explained at Google's PageRank page (linked above), "PageRank relies on the uniquely democratic nature of the web by using its vast link structure as an indicator of an individual page's value. In essence, Google interprets a link from page A to page B as a vote, by page A, for page B. But, Google looks at more than the sheer volume of votes, or links a page receives; it also analyzes the page that casts the vote. Votes cast by pages that are themselves "important" weigh more heavily and help to make other pages "important.""
In other words, links from sites with high PageRanks could be considered gold for webmasters working in highly competitive sectors. After basic keyword analysis, Google's rankings are based primarily on the value of links directed to a document. Over time, Google has improved on its ability to analyze the relevancy of incoming links and the value of information provided by the Toolbar has decreased significantly.
Nevertheless, many webmasters and SEOs continue to obsess on PageRank values assigned to their sites. PageRank continues to be a favourite topic at many search related forums. Businesses selling links based on Google PageRank emerged about three years ago, forcing Google to add more weight to document and link histories when considering the value of links. Tools to measure PageRank were built into Firefox extensions, site rank checkers and link analysis software, putting a larger load on Google's servers. Massive consumer interest might in fact be the direct cause of the Toolbar's downtime.
According to a post from Webmasterworld's GoogleGuy (a Google employee), "I talked to a few people and there was some new infrastructure that was swapping in. They expect normal toolbar PageRank display to resume in a few hours--no need for concern."
Bill Gates is blowing smoke at Google again. Twice this week Gates has said nasty things about Google. Yesterday he was sounding sort of high-school churlish with his statement, "Google is still perfect; the bubble is floating, and they can do everything. You should buy their stock at any price. We had a 10-year period just like that." Earlier today, he and his buddy, lil' Stevie Ballmer, sounded like they were playing king-of-the-castle at recess when he predicted that Google was going to be dethroned because the other kids will want to play with new-tech toys that Microsoft has but Google doesn't.
Bill Gates is not a man who likes to play catch-up. Being the world's wealthiest person and running the most lucrative corporation that has ever existed, he is not often in a position where he feels behind anyone's eight-ball. One can only imagine the frustration he must feel when he thinks about Google and the threat they pose to his empire.
As everyone knows, Gates' empire, Microsoft is enormous. With the copyright over the operating systems and business software used on the vast majority of personal and business computers around the world, Microsoft has long used that advantage to dictate how things would be done. Over the past 30 years, Gates has seen Microsoft's revenues grow from the $16,005 they made with 3 employees in 1975 to the $36,840,000,000 they made with 57,086 employees in 2004. For the most part Microsoft has concentrated its energies on developing the software that runs personal computers, web servers, mobile phones, vehicle components, and much of the global digital infrastructure.
Riding the strength of IBM's reputation Microsoft was established as the predominant maker of operating systems and operating system accessories by the mid 90's. A short and brutal period in the early to mid-90's marked the point Microsoft understood the importance of the Internet. An upstart company named Netscape Communications had created a web browser that was not only better than Microsoft's Internet Explorer; it was being given away for free. This was in the first few years of the commercial Internet and Gates correctly sensed the importance of controlling how people view information found on the Net. Through a strategy of bundling a free browser into the Windows OS and providing tools that would create non-Netscape friendly code, Gates managed to seize significant parts of the browser market back from Netscape. Netscape was forced to sell itself and its technologies to AOL which later introduced the nail in Netscape's coffin, the terrible AOL-heavy Netscape 6.0.
Even with AOL's unwitting assistance, establishing control of the global digital environment was hard work. Over the thousands of person-years of programming, many a mortal soul was invested in producing that outcome. Whatever anyone says about the darker nature of Microsoft's monopoly and corporate business practices, Gates' obsessive interest and passion for IT drives his thinking. One just has to accept that a big part of the thought process involves further ingraining Microsoft's control over the IT environment, as it has for the past 30-years. What he missed, as he elegantly admitted at the January 2004 World Economic Forum meeting in Davos Switzerland, was how important finding information on the internet was going to be. Another upstart rose to become king of the only castle that made sense of the Internet for average users, search. "Frankly, Google kicked our butts...", said Gates in an interview at Davos. Imagine how he must have felt the day he realized the foundation of his empire had to be supported by the shifting sands of search.
Since at least December 2003, Gates has sensed a shift in the sands of fortune. He was already working on plans to deal with Google's dominance over the search market but while spying around on their website, he noticed that Google was hiring software engineers with qualifications that had little to do with search. Google was (and still is), hiring operating system designers, systems architects, email specialists, and other IT engineers who could easily fit into one of many of Microsoft's divisions. Everything about the Google Job Opportunities page screamed to him "INTRUDER ALERT". The moment is practically legendary in Internet history. Apparently he was more than a little annoyed and after firing off a warning email to key executives, he has spent much of the past year in a public pique over Google's continued growth.
The growth and nature of the Internet has allowed serious rivals to emerge. Before the mid 90's, Microsoft could be reasonably certain it fully controlled the operating environment by controlling the delivery system. By controlling the means of delivery, Microsoft could control the software Internet users used. Software bundled into the Windows operating system became the standard used around the world because it was instantly accessible and guaranteed to work with Windows at least 99% of the time. When upstart competitors popped up, Microsoft was generally able to limit their market penetration by simply being there. Gates has good reason to worry.
Today, Microsoft faces a host of challenges. The open source movement has propelled development of literally thousands of variations of the Linux OS. Internet users can download a variety of applications that work on multiple operating systems as well as the Windows OS. Microsoft no longer controls the means of information delivery in the way it once did and it no longer can. Their major attempt at dictating a standard information infrastructure of the Internet, the massive .Net project has likely been shelved at least according to speculation at Slashdot. The forum conversation was brought to me by my very own personalized Google homepage incidentally.
Microsoft actually has a plan to deal with Google. It is in development and code-named Longhorn. Longhorn is the long-awaited Swiss-army knife operating system Microsoft has been hyping for almost two years. With rumoured features such as desktop search and a host of other Internet related tools, Google, Yahoo, Ask Jeeves and several million open source developers have already made it's greatest offerings seem obsolete. Longhorn, in turn has been delayed time and time again, forcing the cynical to speculate on why. My personal favourite theory is they are swapping out the Solitaire game and replacing it with a stripped down version of Duke Nukem Forever. In reality, it appears Microsoft is actually far behind the tech-curve race for the first time in its corporate existence.
The speed and stability of the Internet combined with vast improvements in storage and file compression allows Google (and other rivals) to offer server-side applications such as GMail, Blogger and Google Earth. These applications are precursors to more robust server-side applications and when presented by Google at least, they are available at no-cost to users. Google has also used the Internet to invade what used to be Microsoft's sacred space, the desktops of individual users. This is especially troubling to Gates as the desktop was the domain that Longhorn was supposed to rule. The desktop has always been Microsoft's greatest asset.
In its history, Microsoft has been known to move heaven, earth and entire corporate law divisions defending its turf and defending against lawsuits stemming from stuff they did to defend their turf. Google is a different sort of foe and after a decade of practice, Internet users are now better prepared to pick and choose which products are useful to them and which are not. The days when a product line could be protected with proprietary practices are long over. While bigger than any other entity on the Net, Microsoft does not control it and therefore no longer controls the means of delivery.
That's where search comes in. While the Internet is the primary delivery vehicle, search engines provide the primary means of finding the stuff that is being delivered, whatever that stuff might be. Controlling search is the key to controlling Ecommerce and the future of advertising. He who controls search gets richer than rich, and that is why Mr. Gates thinks about Google so much of the time.
Google is not Microsoft's richest competitor. Sun, Apple and Yahoo are. The thing that sets Sun, Apple and Yahoo apart from Google is that Google has the most momentum and a hell of a lot of available capital. It also has stock values that rival those of the late 90's. Google closed the day at $266 per share. Microsoft on the other hand is sitting in the $26 per share range. While we all know stock values are hardly a measure of competence they are a measure of available capital and right now, Google is enjoying more cash-flow than they ever dreamed of.
For the vast majority of computer users around the world, Microsoft's enduring legacy is seen in the generations of Windows operating systems whose universal popularity created a standards benchmark in software creation. Programmers had to design software that would work with the Windows OS. Today, the Internet itself has become a secondary universal operating system. That and the fact that Google is synonymous with the Net's most important application; search, puts Gates and the wizards of Redmond behind someone else's eight-ball in one of the highest staked billiard games ever.
Is the Butler finally being dismissed now that Ask has been purchased by InterActiveCorp? Rumour has it Barry Diller, IAC CEO is preparing to rename the Ask Jeeves search engine, paring its name down to a single word, ASK. This wouldn't be the first time the jovial Butler has been dumped in Ask Jeeves' history. Just after the dot-com crash of 2000 Jeeves was laid-off. He was brought back after former Ask CEO Skip Battle found him drinking by the Oakland docks, slumped over his last remaining friend, the Pets.com sock puppet mascot.
They say that everything old becomes new again. This adage is proving true in the search engine world as well with Google adopting a personalization plan that makes it look a lot like Yahoo and other search portals. Designed to allow Google users access to its various search tools, the portal displays Gmail, Google News, and Google Maps (labeled Driving Directions). It also calls US Movie Listings (by zip code), stock tickers, weather information, Wired News headlines, a quote of the day (from The Quotes Page), word of the day (from Dictionary.Com), and headlines from the NYTimes, Slashdot and the BBC. There are currently no selections following subscribed Google Groups or Google News Alerts.
Personalizing your own version of the Google homepage is made easy with small edit links appearing above each enhancement. Users are able to select up to 9 references from each source with the exception of the Driving Directions feature, which requires new searches each time it is used. Users can also alter the layout of their personalized Google homepage with a simple drag and drop interface. Google, of course, tracks information on every user though it is not serving personalized advertising at this time.
While this is Google's first obvious foray into presenting themselves as a user-driven portal, personalization features offered by Yahoo and MSN are older and therefore more developed. Google is not likely going to swipe any users from its rivals with their portal design but they are likely to retain long-term users, especially those who become addicted to one-page access to their various Google accounts.
The Open Directory Project is the largest human edited directory of web sites and documents existing online at this time. While many search engines such as Google, Yahoo, Ask and MSN maintain larger databases of electronically spidered sites, the volunteer editors at the ODP read, sort and classify all submitted content before it is added to their search-database. Started in 1998 in reaction to difficulties webmasters had getting their content into Yahoo's then human edited directory, the Open Directory Project was a simple and effective idea.
Founded in June 1998 by Rich Skrenta and Bob Truel, the ODP drew its early inspiration from the first major open-source cooperative initiative, the GNU Project. It was even originally named after the GNU project, launched as GnuHoo. The name was quickly changed to NewHoo in order to avoid confusion between the two projects. Over time, the NewHoo morphed into the more organized Open Directory Project. The ODP is owned and operated by AOL's Netscape division which has pledged to keep the directory 100% free as part of Netscape's social contract with web users.
Over the years, inclusion in the ODP became increasingly important, especially after Google began using it as the primary database for a Google directory. Getting a site listed at the ODP almost guaranteed a beneficial visit from Googlebot as a listing there was seen as a vote of confidence from a live-human reviewer.
For the past twenty-four months however, webmasters and search marketers have expressed extreme frustration while waiting for their sites to get listed in DMOZ. The Open Directory is a volunteer driven initiative, and like other non-paying projects they often have a hard time finding good help. Submissions to categories are backed up for months and in some cases, even years with many of the over 1500 unique directory categories lacking volunteers assigned to edit them. A backlog in sites awaiting review is one thing but recently, accusations of bribery, favouritism and editors lashing out at critics have caused many to lower their previously favourable estimations of the Open Directory.
The submission backlog, incidentally, grew so rapidly that the ODP editors opened a discussion forum known as the Resource Zone specifically to address questions and concerns from webmasters. After operating for over a year, the collective of ODP editors that ran the Resource Zone elected to close down the most used service available on the forum, the Site Submission Zone. While the forum was established to discuss ODP issues in an open and public space, the Site Submission Zone took far too much energy to maintain and moderate. Editors felt it did not offer users enough relevant information as much of what could or perhaps should be said to site owners would fall into the confidential category.
For several months, there have been accusations that some ODP editors are accepting payments for faster attention. Stemming from the Blog, Corrupt DMOZ Editor which was started in December 2004 by DMOZ editor "Ana Thema", the blog lists several entries detailing systemic corruption throughout the Open Directory editorial structure. In her February 8 posting, Ana states, "Links are a commodity. Links from DMOZ are a hot commodity. Everything in this world is a commodity: everything. If you disbelieve that someone would be so corrupt as to sell submissions into the ODP, then Dorothy, this is your wake up call." In other posts she claims she uses at least a dozen unique editor names and maintains a network with dozens of other ODP editors. Reading Ana Thema's blog is much like watching one's first episode of the corrupt-cop drama, The Shield.
Another issue critics have had with editors at the Open Directory Project is one of favouritism. Editors have almost total control over their sections of the directory. While there is a hierarchy of editors with Meta-Editors having the power to re-edit categories that have received complaints, most meta-editors don't have a lot of spare time. This has led to some "fixing" the listings to favour their friends and associates. There are stories of search engine marketers becoming editors at the ODP and then gently favouring sites that would benefit their clients. A more sophisticated story tells of a search engine marketer manipulating ODP results to generate stronger Google page-rank scores for his clients. Another tells of ODP editors networking with each other to provide reciprocal favours.
In a case of reverse favouritism, Ana Thema posted a story at corruptdmozeditor.com from another DMOZ editor that states, "My arch competitor had a dupe content subdomain that they set up for traffic overflow and I changed their dmoz listing to the subdomain with duplicate content and it slaughtered their rankings for a couple of months. Speaking as someone with 4 years of sabotaging experience, switch their listing from www. to non-www from time-to-time. Switch them from www.example.com to www.example.com/index.html, stuff like that."
After complaining about abuses and neglect, some webmasters might expect an apology or a reasonable explanation from the Open Directory Project. None has been forthcoming though the Resource Zone was intended to be a space for DMOZ editors to communicate with DMOZ users. A growing problem for the ODP is the lack of patience users and editors are showing with each other in various search related discussion forums. A post over at the Search Engine Watch Forums likens the accountability of some ODP editors to Seinfeld's character, the Soup-Nazi. According to the post, criticize these editors and, "NO SOUP FOR YOU! NEXT!"
In its defense, the Open Directory Project is staffed by volunteers, all of whom are humans with real lives, real jobs and other responsibilities. Given the backlog of submissions and the deterioration of the directory, it is rather difficult to see them being able to straighten out the mess quickly or easily. While many DMOZ editors put up with a lot of abuse, almost all of them (with the possible exception of folks like Ana Thema) take great pride in the size and scope of the Open Directory Project.
For search engine marketers however, the question of relevance vs. effort comes into play. At one time, a listing at the Open Directory was mandatory in order to guarantee strong listings at Google, Yahoo and other search engines. Today, while still helpful, the strength of a Open Directory listing has been diluted by the search engines themselves. In an article titled, "...Time for The ODP to Close?", Search Engine Watch editor Danny Sullivan suggests three ways the venerable DMOZ could reorganize and revitalize itself. Whatever it does, it should do it soon as the importance of the largest human edited directory of websites is decreasing as quickly as the backlog of submitted sites is increasing.
The Butler is growing bigger again. This morning, Ask Jeeves announced the acquisition of the Excite European network from Italy's Tiscali SpA., giving Ask open access to the rapidly expanding European search market.
"This deal is a next step in Ask Jeeves' European expansion strategy," said Steve Berkowitz, CEO of Ask Jeeves, Inc. "Access to Excite's pan-European operational infrastructure and market knowledge will accelerate our European growth initiative and provide an instant revenue stream from additional users and advertisers."
Ask Jeeves, which is about to be acquired by Barry Diller's IAC/InterActiveCorp, already owns the US-based portal Exite.Com which it acquired in March of last year.
The purchase gives Ask a substantial leg up in the European search advertising market where Ask Jeeves already has a fairly strong UK foothold. Acquired assets include portals in Germany, Spain, France, Italy, the Netherlands, the UK, and Austria, along with a decade of pan-European marketing knowledge and connections.
While Excite will continue to operate under its own brand name, paid search advertising is expected to be generated by AJInteractive, the US-based partnership between Ask Jeeves and Interactive Search Holdings. Excite Europe will continue to operate out of its Rome headquarters.
Working out a marketing plan for new and evolving websites is a bit more complicated than it used to be. There are a few new things to be considered before embarking on a search marketing campaign than in previous years. Search marketing has become more important and is thus becoming more professional. With growing acceptance of online communication tools, and a number of alterations to the faces of search engines themselves, the marketing arena has been upgraded from a three-ring circus venue to a Super Bowl sized stadium.
Three general factors push the increasing sophistication of the search-marketing sector. The first is simple; consumers are becoming much more Internet savvy, as are the businesses that advertise on the net. The second is far more complicated; the nature of search marketing has been affected by the popularization of new communication techniques such as instant messaging, email/desktop search, blogs, press releases, social networks and special interest forums. The major search engines are both driving and being driven by the development and proliferation of cool new tools. The third factor is the changing faces of the search engines themselves. Over the past six months the major search engines have introduced new features and advertising opportunities, and new forms of search engines have emerged, giving SEOs and SEMs a lot more to think about and plan for.
Smarter stuff makes us all smarter While the domain-specific website remains the central vehicle of an online advertising campaign, websites have become larger and more sophisticated. Over the years, search engine users have become comfortable using a variety of tools to read information, most of which are already bundled into their browsers. Search engines are able to find and spider information from multiple file types, permitting search marketers and advertisers entry into a number of new venues. Increasingly, SEOs and SEMs are being asked to help clients understand how new applications or technologies can affect their marketing campaigns, an increasingly difficult task as the environment is evolving so rapidly.
New Tools Today, many advertisers are interested in establishing blogs, both as a means of client communications and as a search-ranking device. Blogs are very easy to establish and with proper maintenance can be very effective tools. They can also have a significant impact on search marketing and site content, provided the blog offers RSS feeds for the major search engines and personal blog-reading appliances. Blogs allow businesses to easily update their clients with new information and receive important consumer feedback from readers. Blog postings have a habit of being proliferated across other blogs. As each blog posting likely has a link to a feature or product page as well as a link to the index page of its keeper's site, the proliferation of good postings can dramatically increase relevant link popularity at the major search engines.
Advertisers in the information sector are taking advantage of social networking applications such as Google or Yahoo Groups, or one of the thousands of special interest forums around the web. Along with a growing community of other knowledgeable workers to bounce ideas off and receive information from, social-network applications help knowledge-based workers promote their strongest assets, their knowledge. Social networking groups are extremely interesting for the engineers at the major search engines. While they are somewhat interested in what is being said, they are even more engrossed in who is saying what to whom and how that knowledge spreads across a network of related persons. Smart SEOs and SEMs are helping their clients use these types of groups to subtly promote their websites by teaching them how to use forum-signature links and how to responsibly offer good and relevant advice.
The new applications and tools offered and/or honoured by the search engines have changed how search engine marketing is practiced. Both blogs and social networking tools have been around for a few years. After the early halcyon days of spam-exploitation, most search marketers have settled down to use these tools wisely to offer long-term benefits for their clients. In this way, search marketing requires a longer-term commitment between client and practitioner especially in light of the changes in the search engine environment brought by new communications technologies.
The changing face of search The search engine environment has fundamentally changed over the past six months. There are four major general search engines and dozens of smaller ones. That part hasn't changed. For the most part, general search has not changed that much either though a number of algorithm shifts have kept SEOs on their toes lately.
What has changed is the stuff behind the veil at every major search engine though; the more things change the more they seem to be the same. Each search engine has similar features and applications such as local-search, paid-search advertising and desktop tools. The smaller search firms also have similar features and applications, some having more innovative core-functions than anything the Big4 currently offer. There are also a growing number of sector or interest specific search tools called vertical search engines.
Part of the search engine environment is fragmenting into a more specific list of tools from highly specialized search tools to local search engines designed to find shops or products just down the street from you. While played on an enormous field, search marketing is often seen as a game of inches. Knowing how to get sites placed in local and vertical venues is important for your clients.
Local Search and Mapping Local search is becoming more important as cell phone users are now accessing the search engines to plan their general shopping itineraries. By combining geographically based listings and highly detailed maps, local search tools have carved a useful niche that is growing more popular with busy consumers. A local search listing will soon be as important as Yellow Pages listings currently are.
Luckily, it isn't that difficult to get your client listed in a local-search tool. The largest like Ask, Google and Yahoo have deals with the phone companies and publishers of the Yellow Pages to include all their listings, regardless of whether the businesses listed even have websites. The first thing a search marketer should do when considering local search is to use an internet based local yellow pages or telephone directory to see if their client is listed. If they are a new business, the search marketer should call the phone company or phone directory publisher in their client's region to get them into the local-business database for future spidering. Search marketers should also place geographic identifiers such as street address, telephone numbers, zip or area codes and even GPS coordinates in the footer of each page of the site.
Vertical Search This is an area that is much more commercial than general or even local search. A vertical search engine is one that hones in on a specific topic such as travel, shopping, books or cars. Populated by the a number of well known names such as Travelocity, Expedia, ABE Books, the field is rapidly expanding with new entries such as Become.Com for shoppers, CyberGolfSearch.com for golfers and EdComp.com for education and training opportunities. The Big4 are already on board in one way or another with features such as Google's Froogle and Yahoo Shopping.
Vertical search engines are betting that as the Internet grows more complicated, search users will turn to a search tool they know specializes in the product, service or activity they are directly interested in. LookSmart has jumped on the vertical search bandwagon with five unique verticals and another vertical search tool, Answers.Com has developed a deal with Google to provide information culled from its various vertical databases. Earlier this year, the shopping focused vertical search tool Become.Com was launched by industry leaders Michael Yang and Yeogirl Yun who quickly hired industry legend Jon Glick away from Yahoo.
Most high-quality vertical search engines are spider-driven or draw from spider-driven databases so getting a site into them should be as easy as paying attention to relevant link-building however SEOs and SEMs are advised to check into vertical search tools for clients by their specific sector and to ask their clients if they know of any sector-specific search engines.
Desktop Search Desktop search and other personalized search applications have emerged to help specific users find information they have already seen. Desktop search locates documents and files housed on the hard-drive of the user, including references to websites that user previously visited. Optimizing for desktop search is fairly simple in that most of the same basic rules apply. Once visited, clear titles and single-focused page content should help clients' sites re-appear when a specific desktop user types keywords relevant to the client's site, page or documents.
The number of methods used to express and recall information across the Internet are increasing and becoming simpler to use. Webmasters and advertisers now incorporate audio/video files, blogs, and Flash animations into their websites and the major search engines are indexing them. Search engine marketers are finding the environment in which they practice evolving faster than the techniques used in their practices. There are literally dozens of different tools to use when building a web presence and each approach will have an effect on search marketing efforts. Fortunately, much of what is new is based on the foundation of how spider-driven search engines have always worked. New technologies provide better ways to communicate ideas, services and products and savvy search marketers are learning to use them. As long as spiders act like spiders and search engines continue to spider sites, finding the way to the future by following the paths of the past continues to be the best marketing plan.
Click Fraud is the greatest threat to the rapid growth of the paid-search marketing sector. Speaking about click fraud to an investor conference in December, Google CFO George Reyes stated, "I think something has to be done about this really, really quickly, because I think, potentially, it threatens our business model." Accounting for an estimated 5 - 15 percent of all PPC clicks (estimates differ by sector), click fraud is assumed to cost advertisers tens or even hundreds of millions of dollars per year. The problem has become so pervasive the April 7 edition of Wall Street Journal ran a front-page center column story titled, "In Click Fraud, Web Outfits Have A Costly Problem". (subscription req.)
What is click fraud and what makes it so dangerous to the stability of the major search engines' business models?
The pay per click or PPC business model generates a lot of money for search engines and webmasters who allow paid advertising to be displayed on their sites. A search conducted on Google, Yahoo or MSN will show several advertisements running down the right hand side of the organic search results and sometimes across the top of a search engine results page. Every time a search-user clicks on one of these ads, the advertiser pays the search engine for that click. Advertisers bid for placement under associated keywords and phrases in a virtual auction format. Generally the advertiser with the highest bids, or in Google's case, the one that generates the most revenues through a combination of high bids and stronger click-through rates, wins the highest placement in the list.
For over three years paid-search has been the primary revenue generator in the search engine industry. Promising front-page placement and massive contextual distribution across associated network websites, paid-search delivers rapid exposure to an audience that pre-selects itself based on keywords entered in their search query or found on a document. As a means of reaching a market too massive for TV, paid-search and contextual distribution is an obviously winning idea. Unlike organic SEO, paid-search marketers can make guarantees and back them up with quantifiable (and easily understood) data.
For major search firms such as Google, Yahoo and Ask, paid-search is to one degree or another a fundamental cornerstone of their increasingly bountiful bottom lines. Actually in Google's case, paid-search represents about 95% of annual revenues. Built on the power of paid-search, the major search engines are reporting record revenues quarter after quarter. Paid-search marketing, for the most part, is much easier for an agency or advertiser to facilitate. It is also one of the easiest ways to scam money or damage one's competitors. In this often-unscrupulous Internet age, there are a lot of very talented people stealing other people's money in one way or another. The provision of paid-search advertising which experiences double and triple digit growth from quarter to quarter is a tempting place to practice their larcenous skills.
Savvy webmasters and advertisers, along with a growing number of forensic click-analysts are getting better at detecting simple fraudulent click activity. The growing specter of click fraud has given rise to an industry that detects fraud and works to find ways to combat it and help clients seek refunds.
Jesse Stricchiola, the founder of Alchemist Media, has emerged as one of the leading experts on click fraud detection among SEM practitioners. She was one of the first SEMs to approach the major search engines with highly detailed forensics and is now recognized as a credible advocate for advertisers who feel they have fallen victim to click fraud. In a presentation at the recent Toronto Search Engine Strategies Conference session, "Auditing Paid Listings and Click Fraud Issues", she outlined the two basic forms of click fraud.
The first and likely most pervasive is generally known as "competitor clicks". In this scenario, a business works to drive their competition into financial distress by wasting their paid-advertising budget. Every time an ad is clicked, the cash register dings at the search firm providing that ad space. While keyword bids might be as low as $.015 per click, they can reach into tens and in some extreme cases hundreds of dollars per click. Through simple and often stupid means or highly elaborate robot driven campaigns, one clicks away on their competitors' ads. This form of click fraud is increasingly easy to detect and deal with however the onus is on the advertiser or their agent to diligently inspect and analyze their web-logs. Jesse noted two case studies in which one business worked to burn the budget of competitors. One case involved having staff members click on paid-ads from their workstations. The other involved the use of a specialized "hitbot" commissioned to spend as much of a particular competitor's money as possible. After tracking several identifiable signatures such as IP address and repetition, click-times and the succession of clicks on an ad, she was able to help both clients get their money refunded.
The second basic form of click fraud is called "Affiliate Fraud". Stemming from the vast networks of small affiliate partners who display paid ads generated by the major search engines on their websites, this type of click fraud is more difficult to detect and manage. When looking at a typical website, users are increasingly noticing paid advertising discretely appearing somewhere on the page. These ads usually relate to the topic of the page and are delivered by the search engines based on keywords found on the page or through a specific choice by the webmaster. Every time one of these ads is clicked, the search engine bills the advertiser and gives 50% to the webmaster of the site the click came from. The dozens of ways to scam this sort of system are obvious and as click fraud becomes a bigger concern for advertisers and search engines, people with a propensity for illicit gain are jumping on the short-term bandwagon.
There are a lot of highly talented programmers looking for work around the world. While all of us live in a time of legal transition in relation to cyber-crime, some people live in nations with relatively weak legal systems and abysmal cyber-investigative infrastructures. Faced with huge brains and tiny employment prospects, several turn to cyber-shenanigans for fun and profit. By exploiting fake IP addresses, using clever algorithms to determine click behaviours and finding ways to destroy identifying references, fraud artists can stretch their gains over several months or even years without getting detected. There have been stories of click-for-pay positions offered to web users in Europe, South East Asia and Oceana in which surfers are paid to click on paid-ads on one of their employer's thousands of websites. If done properly, it can be next to impossible for the major search engines to keep up.
Lori Wieman from KeywordMax noted this issue in the same session. KeywordMax provides PPC analytic software for SEMs and advertisers. While the search firms must bear responsibility for dealing with legitimate refund requests and work to limit click fraud, it is unrealistic to expect them to take a firm stand on the issue. Lori stressed that the onus for detecting click fraud remains on the advertisers themselves. In her presentation, Lori outlined a number of specific things advertisers should look for in their web-logs and if found, record using a spreadsheet program like Excel which allows easy comparison of several points of data.
Lori stressed that advertisers (or their agents) should capture IP numbers of those who visit their sites. Every visitor has an IP address and your server will record it. If you record the IP of all visits to your site, you can eventually see how often a specific IP visits the site and how the user got there. If the same IP appears day after day after day (and hour after hour after hour) and it comes from one of your paid ads, you are almost certainly the victim of click fraud. While recording visitor IP addresses, Lori also recommends tracking competitor IP addresses to see if they show up in your web logs. Every visit from an affiliate partner site generates a unique reference number. Lori suggests recording those numbers whenever possible. Visitors can also be tracked geographically and advertisers are advised to check where visits originate from to see if any odd or unexplainable patterns emerge.
Ultimately, Lori notes the responsibility for auditing billings and listings falls to the advertiser. The search engines are improving their ability to track fraudulent clicks but the volumes of clicks they work with make specific-campaign analytics a lower priority. The search firms could help advertisers by offering more detailed billing, creating fraud investigation departments and becoming more communicative and responsive to advertisers' complaints.
At the same time, it is up to the consumer to diligently pursue their complaints. If you think you are a victim of click fraud, Lori urges you to file a detailed report within 60-days of the fraud and to provide as much documentation as possible.
Backing Lori's call for consumer diligence was Danielle Leitch from MoreVisibility, a Florida based SEM. Danielle urged the audience to audit their own server logs as well as use analytic software noting that some things might be missed by software but obvious for human eyes. Danielle also pointed out that if you do receive a refund from one of the search firms due to click fraud, you need to manually adjust your stats as the report activity generated by the search engines will not be changed. Keeping track of campaign metrics is the key to detecting and dealing with click-fraud.
Danielle used a number of PowerPoint slides to show a forensic audit she conducted for a client. She found a number of click-fraud signatures including similar IP addresses, tell-tale timing and geographic patterning. By compiling very detailed notes and developing both verbal and written contact with the search engines, Danielle was able to get her case-study client a substantial refund.
As an issue, click fraud can be managed and perhaps even eventually tamed but that will require a high degree of cooperation between the search firms that sell the ads and the advertisers or their agents who purchase them. It is in the best interest of the major search firms to work with advertisers and their SEM agents to protect the integrity of the paid-advertising systems. Facing a number of paid-ad focused lawsuits, one of which is threatening to be granted class action status the search engines are being forced to be more open to the concerns of their consumers. With the advent of click forensic analytics and advocacy as a professional segment of the Search Marketing industry, a strong foundation for such cooperation is being built. While click fraud remains the most frightening issue on the paid-advertising front, increasing sophistication of consumers, advertisers and search media and a willingness to collaborate presents a strong and responsive defense.
Is MSN better than Google? How does Ask Jeeves stack up against Yahoo? Which of the Big4 search firms produces the most relevant results? Those questions are difficult to answer as what is relevant to one searcher might not be particularly relevant to another. Search engines are the tools we use to thread the eye of the needles found in the universal haystack we know as the Net. As our primary provider of information references, search engine relevancy is an important issue. Everyone has a favourite search engine and most SEOs complain about the relevancy of results found on one or more of the Big4. The easiest way to get quantifiable data is to hold a blind-test challenge in which participants can rate results without knowing which search engine those results originate from.
RustyBrick, a Website/webtools construction firm based in Rockland County New York has created a search tool designed to act as a user-driven survey of the relevance of search results delivered by the Big4 search engines. Described as a "white-label" search engine, "RustySearch" offers users a sparse white page with a general text box to enter their search query. It then serves generic results pulled from one of the Big4 with a side link asking users to rate the relevancy of the listing. Results of this survey will be released on the SEORoundtable Blog on or about June 1st.
Search Engine Marketing is a sensible vocation. Driven by many of the same basic tenets that inform the traditional marketing sector, the goal is to be sure one's clients' products are among the first people think of when looking for that certain something those clients create. In the olden days it was all about placement, positioning and repetition. Elaborate campaigns involving radio, television and print would be conceived and executed with the goal of establishing a foothold for new products in the households of the nation or solidifying the stability of a pre-existing brand. Those olden days may be, like so '80's, in relation to the crazed new world that search brings however, humans being humans, the ideas of an older generation often remain the ones that play best on the Internet today.
The overall impression left from Jupiter Media's Toronto Search Engine Strategies Conference is that the business of search has gotten a lot more serious about itself. 2005 is going to be another watershed year for the search marketing industry with the combined revenues of Google and Yahoo projected to outstrip those of the three major TV networks by years end. Search marketing has firmly become one of the streams that make up mainstream advertising. This is not fresh news for most of the folks already populating the search industry but it might come as a surprise to smaller advertising and marketing firms who make their minor fortunes in the traditional ad market. Search is arguably the most important facet of a marketing campaign. A word to the wise; if you like arguments, get in on this one now because in a few years the debate will most certainly be settled on the side of search.
As things stand today, the search marketing industry is on the cusp of tremendous growth with several sub-streams of search marketing evolving and interacting with each other. The sector still seems to divide down two main lines labeled Organic and Paid however that division seems smaller as both streams of SEM have a symbiotic relationship with each other and are often practiced in the same shop. From that division come several sub-streams such as PR Marketing, Blog-casting, Second-Tier PPC and Highly Strategic PPC. The rapid growth of the paid search sector has even prompted the creation of firms dedicated to click-fraud forensics such as Alchemist Media. By the end of the two-day conference, attendees were very clear on these basic points. Search is an advertising and marketing medium built on the foundation of placement, positioning and branding through repetition.
Organic Search
The field of organic search engine optimization has changed subtly. In previous years, a wide division existed between SEO practitioners who identify as "white-hat" and those who identify as "dark-hat". While that divide is still very much present, the numbers on the darker side seem much smaller this year due in part perhaps to Google's recent de-listing of a couple very well known SEO firms. Another reason many SEO shops are moving away from what are considered questionable tactics is that clients have become more educated and sophisticated. With many already understanding the potential volatility of organic placements they are expressing their clear desire to be represented ethically.
Another client-driven shift in the SEO landscape is the increasing tendency for SEO shops to provide other forms of website analysis and business service including, usability studies, competitive market research, visitor tracking and inter-business networking. This movement is reflected by the rebranding efforts underway at a number of SEO / SEM shops and was written about by UK tech-writer, Suzy Bashford in an article published on Monday in Revolution Magazine.
A third shift in the search-marketing sphere is a renewed respect for SEO practitioners in the industry. Though the SEO sector predates the paid-search sector by almost five years, paid-search has generated far more interest in terms of page real estate and subsequently, in terms of dollars invested by advertisers. For the most part, the mainstream media has ignored SEO over the past two years focusing almost exclusively on the easier to understand and highly profitable paid-advertising arm of search marketing. Organic optimization was consistently under-reported on likely because it is harder to explain than paid-advertising and much more difficult to quantify long-term results aside from the simple metric of Top10 placement.
According to a survey done by SEMPO in December 2004, 82% of money invested in search marketing went to paid-search. Organic optimization accounted for only 12% of search advertising dollars. While there are many more dollars invested in paid-search advertising than in organic search engine placement, an October 2004 study of search engine user's click-through habits by Enquiro's CEO Gord Hotchkiss shows that 69% of the users chose organic listings over the 25% that tended towards paid listings. While they do not bring the instant but often expensive instant gratification of same-day first-page placement, organic listings are again being recognized as a solid foundation to build a long-term search marketing campaign off of. A drawback to the organic listings is that many search-users see these as "free", with a greater commercial weight initially placed on links found in the sponsored or paid listings.
Paid Search Advertising
The Paid-Search marketing sector is healthy and growing rapidly. Led in Canada by firms such as PageZero Media, PPC and other forms of paid-search form the backbone of the search-engine economy. Working the paid-search sector is becoming more complex thus prompting the development of in-house specialization at growing SEM shops. Job tasks such as designing customized landing pages, campaign analytics, and ad-copywriting could once be performed by individual employees but are now often sent to a staff/team member specialist. The increasing sophistication of expanding SEM firms is directly related to the seriousness of the sector and the increasing presence of larger traditional ad-firms such as Avenue A/Razorfish in addition to the massive advertiser support offered by the various search firms.
Paid search is where the truly revolutionary action is focused these days. Paid-search is by far the fastest growing section of the Internet economy generating annual revenues measured in billions. Paid search is the reason Google surpasses itself each quarter and Yahoo is going Hollywood . Paid search drives the industry and the potentials for advertisers and providers continue to be astounding. The paid-search sphere is populated by the ENORMOUS, the Big, the small to medium sized, and the start-ups. From the search marketing perspective, every inch counts and there are too many unique factors to each client to make sweeping general statements about the merits of one program or another.
Google is the first in line for congratulations in growing the paid-search marketing sector this year. They are enormous. While GoTo/Overture might have started it all, Google made it sexy and had the IPO to prove it. Awash with fortune and high expectation, Google has plastered major portions of the web with paid-search advertising. Approximately 95% of their revenues stem from paid search advertising and their ingenuity in acquiring or developing greater cyber-real estate assets to place it on is key to their growth. Google, by the way, is growing and that growth pushes the buttons of everyone else in the industry. This growth has a downside though. Google has a number of dark clouds gathering over what was once the rain-shadow of their Mountain View home. They have attracted the lion's share of attention over the past few years and some of that attention has brought unwanted results. Webmasters and search marketers no longer see Google as an innocent friend but more as an invaluable business partner who is capable of turning on them at a moment's notice. From complaining of client poaching in Europe to Network-ad distribution partners grumbling about payout irregularities to the growing concerns about Click-Fraud, Google has a wide array of rainy days on the public relations front ahead of them.
Yahoo (Overture), MSN and ASK make up the next level, collectively known as the Big 3. Of these three, Yahoo is by far the largest with a massive contextual ad-distribution network and the power of the original brand name behind them. MSN is entering the paid-search ad distribution market with MSN Paid Search Solutions a webmaster focused program that will likely resemble AdSense. ASK also provides a high level of contextually driven paid-advertising traffic and is exceptionally popular in Britain .
Rounding out the pack are the small to medium sized paid-search programs, the Tier 2 and 3's. PPC engines such as FindWhat, Go-Click, Snap, LookSmart, and Mamma.com populate this part of the sector. While they may seem like small potatoes when compared to their much larger competitors they can drive a significant amount of traffic through affiliations with regional portal sites, promotional advertising companies (think your local radio station), local ISPs and other online business relationships.
A recent adage states that a ton of money will attract a mass equal to fifteen thousand times its weight. The number of start-ups on the paid-search scene indicates there is validity to this highly unscientific statement. There is simply so much money to go around every pair of geniuses along with five or six friendly investors are starting their own paid-search engines. Many of these start-ups have very limited marketing budgets and don't make it to the trade shows and conferences. There were a few paid-search startups at the conference including Toronto based SearchForIt.com. What made SearchforIt more interesting than others was their focus on solving the greatest challenge to the paid-search industry, click-fraud. Unfortunately they don't talk about it on their website but at the conference trade-show they were extremely optimistic about its ability to detect and identify unwanted clicks.
There were a number of other forms of search engine marketing represented at SES Toronto, the most interesting of which was Press Release marketing as practiced by Greg Jarboe of SEO-PR. Greg made one of the most interesting points last week when he mentioned how the first three Google News results are displayed above organic listings on a typical search results page. Along with the valuable service of getting his client's message out to the media, Greg gets clients' news releases placed on the first page of search results under their target keyword phrases. Coupled with positive link-building potential (news sources can offer highly relevant links), front-page placement on Google drives large amounts of traffic.
Last week I mentioned an incident that happened at a session Greg was speaking at. An unnamed representative from Google happened to be sitting in the front row and I couldn't help but notice his look of surprise when Greg showed a slide that looked as if a press release he wrote had captured 6 of the 10 first page references at Google News under the keyword phrase "home improvement guide". A closer look reveals that these are different press releases with text relating to the phrase as part of a springtime promotion run by Verizon's Yellow Pages. I reported on it, mostly because it was quite humourous and also because Google is intensely interesting. I did not intend to imply that Greg was spamming Google in any way. He is not. I regret the use of the word manipulated though only because of negative connotations associated with the word, not because I think it unfit for the sentence. This is advertising after-all eh? I apologize if the reference caused him or his partners any concerns. (Y'all should have been there though. It was pretty funny.)
The search engines, as we know them today are rapidly changing. Those changes in turn alter the environment in which SEO and SEM practitioners work. The past couple of weeks have seemed like the eye of a hurricane compared to the barrage of changes and initiatives announced in the first quarter. Even though we are involved in the most rapidly evolving advertising medium of all time, the old notions of product placement and branding continue to dictate the direction of our thinking. In that, the future philosophy of search can likely be found in the past.
Toronto is the largest city in Canada and the fifth largest in North America . There are more people in this city than there are in the entire province of British Columbia. It is also my home town so I was more than pleased to be asked to speak at the Toronto Search Engine Strategies Conference again this year.
Having lived away from Toronto for so many years, I keep forgetting how vast the city is. In most towns, a person can budget ten to fifteen minutes to travel between places. Here in Toronto, one must budget that time to move between venues. My only complaint about SES thus far is that the nearest cup of real coffee is a fifteen minute round-trip from the conference floor to the middle of Union Station. There is a large urn of convention centre coffee available on the exhibit floor but no one wants to drink it and chances are you are not interested in reading about it. Suffice it to say, myself and 2000 other search focused folks are suffering through but we are surviving. It is cold in Toronto this week. West coast readers might be interested to know it snowed for a short period on Monday with freezing rain yesterday and temperatures dipping towards freezing in the middle of the night.
For me, real conference started last night with a large group of speakers heading out for supper at a pub more Irish than Ireland . After supper, we returned to the hotel bar where the real fun began. Myself, Jill Whelan, Matt Bailey, Debra Mastaler, Mike Grehan, Christine Churchill, Danny Sullivan and Greg Jarboe and a rotating number of other SEO/SEM practitioners spent much of the night drinking viciously overpriced drinks in the hotel bar while trading stories about o
Blink
Del.icio.us
Digg
Furl
Google
Simpy
Spurl
Y! MyWeb
Is the Butler finally being dismissed now that Ask has been purchased by InterActiveCorp? Rumour has it Barry Diller, IAC CEO is preparing to rename the Ask Jeeves search engine, paring its name down to a single word, ASK. This wouldn't be the first time the jovial Butler has been dumped in Ask Jeeves' history. Just after the dot-com crash of 2000 Jeeves was laid-off. He was brought back after former Ask CEO Skip Battle found him drinking by the Oakland docks, slumped over his last remaining friend, the Pets.com sock puppet mascot. 