The first six months of 2006 have been a period of expansion, growth and significant change in the search marketing universe. There have never been as many online options available to commercial advertisers and their agents as there is right now, a trend that shows no signs of abating. Just to keep things interesting, lot more change is expected in the second half of the year as well.
Search engine marketing, as an online business sector, covers a lot of virtual ground. Search engine optimization started as a cottage industry in the mid 90's. By the mid 2K's, Cottage country had become an industrial district. Today, the search marketing sector extends far beyond the organic search results the business was originally built on.
Search is part of the mainstream marketing consciousness but as the online environment evolves so quickly, what works in the mainstream world often needs to morph through several experimental stages before finding a way to adapt. The scariest and most beautiful thing about Internet is that nothing lasts long enough to be considered a constant. There is still obviously so much room for innovation and outright invention that the playing field is naturally prone to change at any given time. That's been the back-story of the first six months of 2006.
The sudden expansion of advertising options has presented a dilemma for the traditional, independent-operator, search marketing sector. There are far too many methods and means of online marketing for any one person or company to keep track of. The sector appears to be responding by developing a sense of segmentation based on specialization that goes far beyond the separation of organic SERPs and Pay-per-click advertising.
Today, there are at least three major search related marketing venues, and each of which has developed industrial sub-sectors and interacts with the others. The first is the traditional organic search results, the Top10 lists that started it all. The second venue is the paid search advertising market that now ranges from typical PPC to video placement. The third venue is found in the emerging fabric of social networking taking place on MySpace, Flickr, in the blogosphere and other interactive media, an environment that can be safely tagged Web2.know.
Each of these three venues relies on some form of search to guide users towards information and there is a great deal of integration and interaction between the three venues. As search functions are always reliant on key-triggers of some sort or another, there is always a way to include key-triggers in web documents and files.
The fall-out from change has hit the SEO sector the hardest. Consistently overshadowed by the PPC market and suddenly challenged by social networking, organic search engine optimization has fundamentally changed over the past three years as Google and Yahoo have improved and innovated on methods for ranking web documents.
There are now several sub-sections of SEO, each of which can be considered a specialty of its own. As a practice, SEOs focus the myriad of on-site/on-page issues as well as several off-site issues such as hosting, link-acquisition and competitive research. There are also an increasing number of applications on which organic search results are displayed including local, mobile and shopping search results.
There are dozens of on-site or on-page elements SEOs work on which range from correct site structure to various forms of copywriting to accessibility and usability issues. It is an increasingly complicated atmosphere that involves more hours per file than ever before, leading to the necessity to outsource various components of a well rounded campaign.
One of the rules in most established SEO firms is that every SEO has to have a background in website design. Without it, the SEO is lost among the source code with no real knowledge of how a web page is actually written or a website is structured.
Over time, the process of website design itself has become far more complex with a widening array of file types accessible to search engine spiders and an increasing number of CMS tools. SEOs now deal with Flash, Acrobat, variants of HTML, and the learning curves associated with unique content management systems. Similarly, where websites used to tend to follow a relatively simple linear tree, they can now descend into seemingly limitless databases with file-paths that are subject to sudden change.
The array of file types that can be present on a single web document and the complexity of most commercial websites create a market for usability specialization, one of the fastest growing sub-sections of the industry.
As search spiders tend to favour sites featuring fresh and focused content, SEO copywriting has long been a specialization in the industry. There is continued growth in this sub-sector of the industry, especially in light of the rapid growth of the second major venue, paid search advertising.
In highly competitive areas such as tourism, real estate, clothing and web services, SEO has always been a game of inches as competing website optimizers work to better each other on behalf of their clients. In order to compete, we are increasingly depending on website statistics and analysis to tell us how visitors actually use our clients' sites. This allows us to make (or recommend) changes in real time, as compared to the two-three week delay in previous years. Knowing how visitors use the site gives us the advantage of improving traffic patterns and thus improving overall site quality.
The second major search marketing venue is the Paid Search Advertising market. This is the area that has seen the most growth over the past few years, having provided the successful business model that has propelled Google and Yahoo to the front of the pack.
Pay-per-click advertising, on the surface, is fairly straightforward. Advertisers bid a certain amount that is billed every time their ads are clicked on. Google, Yahoo and MSN each have significant paid-search advertising programs but each of those programs significantly differs from the others. Widely recognized as Canada 's leading paid-search marketing firm, Page Zero Media employs experts in both Google AdWords and Yahoo Search Marketing due to the different techniques required to work within the competing programs.
Paid search advertising is undergoing several forms of expansion at this time, allowing a number of new options for advertisers. Earlier this month, Yahoo unveiled a revamped Yahoo Search Marketing interface that gives advertisers more control over the timing and placement of their campaigns. Last week, Google announced it was allowing advertisers to bid for video ad placements alongside typical paid-search ads. Microsoft is also rapidly moving forward with their PPC division, AdCenter.
The most interesting facet of paid search advertising continues to be in the distribution of paid ads to third-party web properties such as the websites of major newspaper or privately operated websites. As the major search engines continue to find ways to distribute paid search advertising, especially video ads, the sector servicing this market will continue to grow and areas of specialization will continue to form.
Another specialization associated with PPC is the business of click-fraud detection and advocacy. Click fraud is a continuing problem facing paid search advertisers and vendors. There are now about a dozen reputable firms that can credibly offer assistance in determining if click fraud is happening and advocate on behalf of a client with the major search engines.
The third and most fluid search marketing venue is found in the mysterious space known as the social networking sphere. This is an emerging area that is growing and changing so rapidly that it appears to be more of a transition than a plateau in web-interactive development.
Social networking is a complicated topic best described through the parlor game, Six Degrees to Kevin Bacon. (Name an actor from the 20 th century and chances are you can put him or her beside someone who worked with someone who worked with Kevin Bacon.) The web of personal connections we have is immense. There is no place to prove this better than MySpace, the fastest growing network of young people on the Internet.
MySpace is a phenomenon in the field of online communication in the same way Google was seven years ago. There are several similar social networks including Facebook, Orkut and Friendster however, none is as widely known or visited than MySpace. Its immense popularity is spread, primarily, through word of mouth. For younger Internet users, its reach and usefulness is incredible. From the sharing of hosted music tracks, images and videos, to the formation of user-interest groups, MySpace and other social networks are quickly becoming the live-time meeting spaces and marketplaces of the Internet.
The basic element that makes something "Web2.0" is the ability of users to communicate, add to, augment, debate, inform, and, in doing all of the above, somehow alter a shared or personal environment.
Knowing how to establish social networking identities and add items or information of interest to users of those networks is a segment of search marketing services that is just now starting to grow rapidly.
Google, Yahoo, Ask and MSN all use elements of Web2.0 ideas in formulating their search rankings and in new features and applications they offer to their users. Yahoo is an especially enthusiastic adopter with services ranging from Yahoo 360, Yahoo Shopping, Yahoo Answers and the Yahoo Publisher Network all touching in some way on Web2.0 elements. Google is also experimenting with Web2.0 thinking in allowing its users to present quality feedback and to affect search listings through personalization and bookmarking.
Being involved in the search marketing industry over the past few years must be similar to the experiences of cartographers during the various ages of nautical exploration. From the days when humans first learned to sail, approximately 4800 years ago, we've been going places, seeing new things, establishing trade, disrupting established economies, altering societies, and documenting our experiences along the way.
According to the American Mariner's Museum in Newport News , Virginia , the ages of nautical exploration spanned nearly 45 centuries, stretching from 3200BC to 1779AD, the year Capitan Cook was killed in Hawaii . It took 4400 years for humans to produce an accurate map of the world and, though we are knocking them off at an alarming rate, we are still discovering undocumented species of insect, plants and even animals. Exploration and documentation takes a long time. Cook took nearly two years to travel from Portsmouth England to the Pacific Coast of North America on his third and final voyage.
The Age of Search, on the other hand, is only about a decade old and exists in a world in which one can electronically travel from the Pacific Northwest to Portsmouth England via Virginia before eating breakfast. Documentation still takes a while but the commute isn't as challenging as it once was.
Links are the primary arteries of the Internet, the underlying connectors between different places. Links are the transporters that take you everywhere on the web. You likely came to this space via a link and are as likely to follow one out again. Links keep you going online, hopefully to places you want or need to get to.
Google created the most successful information retrieval device of all time based on sending spiders to follow each and every link they can find on each and every web document they come across. Yahoo, MSN, Ask, and all the other search databases have acquired the vast amounts of information they contain in similar fashion. Links play important roles in the ranking formulas of all search engines, especially Google, by providing numerous pieces of data for their algorithms to chew through.
How links are valued by search engines and by savvy webmasters who build sites have changed over the years however the way most web users think about links has not changed all that much. When discussing link strategies with clients, a lot of confusion comes from having two different understandings of what links are used for and how they should be used.
For most Internet users, links have always been pathways between points of interest. Link-paths might be intricate and informative but they are not necessarily seen as complex.
Search marketers however, see links a multi-dimensional universe that vaguely resembles the central nervous system. A good visual representation can be found by typing a URL into Touchgraph.com. (Readers are advised to do so before continuing...) Links exist in spatial relation to the document or set of documents associated with them. Some links are internal, or contained within a unique URL. Others are external, pointing into or out from a specific document. Some links are located close to a specific page while others are two or more documents removed. Links also exist in time. Each link was placed on a certain date and will be removed at a later date. The timing of the placement and the longevity of the link have an effect on Google rankings. Lastly, and most importantly, they exist in context. Each link was originally placed for a reason and each document linked together will contain some form of information. If the information found on both pages is topically similar, the link was likely placed in order to benefit site visitors. If, on the other hand, information found on the two pages is not topically similar, the link might have been placed in order to artificially inflate Google's perception of the site.
For search marketers and the engines they work on, links are powerful directional tools that act as indicators of a site visitor's interests. They are used to move spiders from A - Z through a website and steer visitors towards useful information and/or specific conversion points. They are also used to gain the attention and respect of search engines. Having a number of links pointed towards your site from other topically relevant sites is a good thing. Since the earliest days of Google, link strategies have played a key role in winning strong search engine placements.
Links can give search marketers information on how a website operates. By examining the link structure of a website, most good SEOs can find points or techniques that might cause problems for search engine spiders and make suggestions for improving access. Similarly, good search marketers can show a client how live-visitors use their website. By examining data found in site logs, search marketers can often see and explain user patterns and suggest ways to reorganize the site in order to drive visitors towards conversion points.
Though links provide important tools for search engine marketers, they are amazingly difficult to understand and employ without somehow exploiting the way search algorithms are written. Learning how to acquire links without intentionally exploiting Google's algorithm is one of the hardest facets of search engine optimization.
Building links has long been considered an important component of search engine optimization. Search engines use links to find and catalog information from one page to the next. They compare what they find on pages linked together in order to determine the topical relevancy of linked pages. Shared topicality, or context, is extremely important for strong search placements. Search engines expect that links, in all cases, should lead to useful information. As search users travel across linked pages, the search engines track how much time users spends on linked pages and the pathways they take when leaving a document.
The first commercial search engines, Alta Vista, Infoseek and Lycos tended to place weight on keywords found in the text of websites or documents, without considering the effects of links between documents. Back then, webmasters had to submit URLs to each search engine as an invitation for spiders to visit and record site information. This tedious and repetitive task is no longer necessary though hand-submission of sites to certain search directories is sometimes recommended.
In the early days, links between documents were placed by hand. A website designer had to open the source-code in Notepad and write a new line expressing the link in the appropriate place. It was a process that took a few minutes, especially when using a 14.4 or 28.8K dial-up modem.
Understanding the amount of work it took to place links, Google's co-founders figured that each link could be valued as a positive vote. That was the original assumption behind the PageRank algorithm. When combined with traditional on-page factors such as titles, meta tags and content, link-popularity became a fairly accurate gauge of the relevance of information found on pages linked together.
As Google rapidly grew in popularity, many people working in the search engine marketing sector began using links as a means of manipulating or influencing Google's rankings. That is where the practice of building link farms and wild networks of unrelated links came from. Over time, an expanding sub-section of the search marketing industry developed, devoting itself to building, buying or bidding for links in order to boost their clients' chances of scoring high rankings in Google's index.
Though links provide important tools for search engine marketers, they are amazingly difficult to understand and employ without somehow exploiting the way search algorithms are written. There is absolutely no question however, since the earliest days of Google, link strategies have played a key role in winning strong search engine placements. Learning how to build links without egregiously exploiting Google's algorithm is one of the hardest facets of search engine optimization.
Nobody likes exploitation; especially search engineers frustrated by seeing their brilliant sorting algorithms serving spam-filled results. In order to combat link-based exploits, Google has gotten far stricter over the past two years about the types of links it considers relevant or important. While doing so however, Google and the other major search engines seek to place high value on links that are useful to site visitors, leading them to information that is likely to create a better user-experience and more loyal customers. Links that provide a positive outcome, or lead to a series of pages that can guide the user towards what they want, are highly valued in search rankings. In other words, search engineers are required to balance the bad against the good and try to weave rewards to sites with relevant links that are beneficial to site visitors.
Many webmasters and SEOs take the time to search out complimentary websites, writing personal emails requesting a link. Some send out mass-emails to tens or even hundreds of thousands of other webmasters asking for a link to their site. Some hire link-building firms to do the job for them.
Ultimately, the best way to get good incoming links is to work for them by providing extremely good content and finding ways to let others in your business or interest sector know that content exists. Other webmasters will often link to good content; the trick is finding an effective and acceptable way to present it to them.
Be very careful when hiring a company to do link building for you. The "rules" around link building can change and vary depending on the circumstance. There is no one-size-fits-all solution for link acquisition. There are a number of ways to bungle the job though.
In one well known case, a large SEO firm in California claims to have hired a link building firm that created a dense web of reciprocal links between tens of thousands of unrelated sites. The system worked well for over a year but infamously broke-down late last summer, resulting in a system-wide ban for several websites associated in the link-network. Google banned the firm, many on its client list, and many associated with the link-network. The penalty was doled out for the crime of link-farming, a practice in which a lengthy list of sites are linked together (knowingly or unknowingly) in order to artificially boost the link density Google perceives for sites within the network.
Links will continue to be an ongoing point of conversation within the search marketing community. It is a complex subject with enormous implications, regardless of where one's opinion falls in the discussion.
Phoenix over at the SEORoundTable points out a discussion thread at the High Rankings forums that includes a quotable line from HR member Jonathan Hochman, "Looking for links is like trying to get a date. If you are desperate, nobody respectable will be interested, but if you relax and behave nicely, you'll have plenty of success."
The thread also contains links to a number of good articles about link building along with a lot of common sense advice. Anyone interested in thinking about link acquisition is well advised to read through it.
Last week, I had a conversation with Dirk Johnson, one of the owners of Domain Drivers. Domain Drivers uses a reciprocal linking strategy that is different, as Dirk explains it, from other recip-linking programs. Dirk's thoughts, along with others I find digging around the web, will be the subject of another piece, very soon. I think I've thought about links enough for one day.
Does anyone remember the hissy-fit Google threw at the beginning of this month over the default settings of Microsoft's new web browser IE7?
To recap, Google gave Microsoft a week of headaches, going as far as complaining to the US Department of Justice over the fact that IE7 defaults to MSN search unless another search engine had been set as the user-preference before installation.
At the time, Google spokesperson Marissa Mayer was quoted by the New York Times saying, "The market favors open choice for search, and companies should compete for users based on the quality of their search services. We don't think it's right for Microsoft to just set the default to MSN. We believe users should choose."
Flash ahead twenty one days to this week's big Google news. Google has signed a deal with Dell, buying its way into the default settings of millions of new Dell computers. The agreement will have Dell installing the Google Toolbar on new PCs, along with the development of a co-branded homepage, effectively putting Google in the same position they heavily criticized Microsoft for less than one month ago.
While financial terms of the deal were not released to the public, Google CEO Eric Schmidt, speaking at Goldman Sachs investors' conference in Las Vegas, said the companies will be sharing revenues generated through search advertising fees.
CNET News quotes Schmidt saying, "The real reason we do this is for users," Schmidt said. People "turn the Dell machine on, and everything is integrated right there. (This deal) is a turnkey solution for search."
Meanwhile, over at Microsoft, folks were rumoured to be giggling into their coffee mugs this morning, no doubt pleased that their long weekend begins with a grin at Google's expense. In an unrelated (but wonderfully timed) interview with the San Francisco Chronical, Microsoft CEO Steve Ballmer weighs in with his thoughts on Google's famous corporate motto, "Don’t be evil". This part of the exchange between Ballmer and SF Chronical reporters Ken Howe and Dan Frost tells its own story.
"Q: This is an old question, but it always comes up: What do you think of the "Don't be evil" mantra as a corporate culture?
A: Who are we talking about?
Q: Google.
A: Do they follow it? (Laughs.)
Q: What do you think of that?
A: I don't have any comment. I'll ask you. I mean, it's one thing to have a mantra and it's another thing to follow it. Dude, you've got to ask yourself that question.
Q: Are you saying that they don't?
A: I didn't say anything. No comment about it. It is important for all companies to -- we have a mission. We believe in empowering people and businesses around the world to realize their full potential. We have a set of core values that we believe in. I think many companies do have a mission. Many companies do have values. Many companies do have a mantra.
The key is not whether you have one. The key is: What does it mean in practice? Do you drive it? Do you follow it? We do. That's important. But even when we do, there are people who say well, if you do, shouldn't X or Y or Z not happen? I mean, the truth of the matter is, getting a very high market share in a business is not inconsistent with our values or our mission, but sometimes when you get a good position in the market, people need to look at you in a different way.
We have good positions in some markets, and some can argue other guys, including some of the guys that you're talking about, have good positions in other markets. I don't study other companies' mantras."
Mr. Ballmer might not spend a lot of time studying other companies' corporate mantras but if he did, he might not be surprised to hear Google is being advised to change theirs. The current one, "Don't be evil" is getting harder and harder to say without a devious smile. The suggested change, "Live long and prosper in big glass houses but don't throw stones."
Yahoo and eBay have formed a strategic partnership in order to tie up a number of loose ends both firms feared losing to Google. The agreement, announced earlier today, will see the each firm provide functions and services to each other and each other's users in areas ranging from search results, graphical advertising, online payments, the creation of a co-branded toolbar and an opportunity for both firms to explore "pay-per-call" advertising options.
According to a press release issued by eBay, the two firms will begin to introduce services and features based on the agreement, later this year. A trial and error test phase is anticipated with full implementation expected sometime in 2007. Initially, the partnership is limited to the United States.
In the area of search, Yahoo will be the exclusive third party provider of all graphical advertising and sponsored results on some eBay search results.The two firms are also exploring ways to integrate Yahoo web search results with product results drawn from the eBay search engine. In the future, eBay would like to see some of the products offered by its membership appearing in Yahoo search results.
The second part of the agreement covers the online payment system, PayPal, which eBay purchased last year. PayPal will be used by Yahoo as a means of accepting payment for its services. Yahoo will also promote PayPal to its users, subscribers and ecommerce/Yahoo Shopping clients.
The partnership will also cover the development of a co-branded toolbar. Currently, the eBay toolbar is installed on over 4-million computers. Yahoo will provide web search results and sponsored search results for complementary products on some US-based eBay searches.
Perhaps the most interesting part of the agreement addresses exploration of the pay-per-call sponsored search billing model in which advertisers are charged by the number of phone calls generated by an ad, as opposed to the number of clicks generated by an ad. Yahoo recently updated its instant messenging application Yahoo Messenger to allow users to communicate verbally as well as through text messages. Last year, eBay bought the popular VOIP start-up company Skype. Users should look for some form of integration between the two systems, allowing for both to be used to facilitate pay-per-call advertising.
Both eBay and Yahoo come out of this agreement as winners though it could be argued that eBay receives the most value of the two.
For Yahoo, the agreement offers access to a huge user market numbering in the tens of millions along with a convienient and already trusted online payment system in the form of PayPal. The agreement will also grant Yahoo access to the statistics and demographic information provided by eBay users.
For eBay, the deal offers a measure of security against the growth of Google in the online advertising and sales market, most notably through Google Base and the pending introduction of other Google ecommerce focused products such as Google Wallet. It also offers a direct line-in to eBay for users of the world's most popular search portal.
Barry issued the first known marriage proposal via search engine when the folks at Ask Jeeves (now Ask.com) helped him pop the question by placing his proposal page at the top of search results for a keyword phrase he knew Yisha would be searching for (her name).
Yisha, who is about to marry one of the nicest guys in the industry, immediately said yes and hopeless romantics through-out the industry (like us here at StepForth) smiled and wiped stray tears from our eyes when Barry shared the news.
Due to Barry's noteriety, the search industry paparazzi are rumoured to be planning a stakeout of the wedding. Watch for photos to be published on Flickr sometime next week, or perhaps on the site, Yisha & Barry.com.
Congratulations Barry and Yisha. Best wishes from your friends (and readers) at StepForth Placement.
As it turns out, the revolution will be televised, or sort of televised anyway. Actually, it starts as a commercial.
Google introduced a new age of advertising earlier this week by announcing plans to distribute video commercials on websites in its pay per click AdWords network. The move opens the once exclusive marketing channel of televised ads to a far wider array of small and medium businesses.
Video advertising on websites is not, as such, revolutionary. Online readers of major newspapers such as the New York Times or Wired News are likely familiar with the 30-second IBM dinosaur-head commercials or video ads from the History Channel and Microsoft. The spaces these examples run in often cost as much or more than standard TV ads would. The revolution, as always with the web, revolves around an axis of access and contextual distribution.
The sudden appearance of Google's online video ad distribution service is going to have multiple repercussions for the $38 Billion US television advertising market. It is also going to present multiple opportunities for search marketers and their clients.
Google users shouldn't expect to see video ad content on Google branded websites such as Froogle, Google News, or in the general search results, not yet at any rate. The New York Times reported yesterday that the company is, "... considering doing so in the future."
Starting today, Internet users in the US , Canada and Japan , will see an increasing number of click-to-play video boxes appearing amongst standard text-based AdWords ads on websites displaying Google generated advertising. An example video ad is posted on Google's Inside AdWords Blog, appearing in a passive-play mode allowing viewers to click on it before it runs. Others have been spotted at, dictionary and translation site, WordReference.com, and on the front page of Blogthings.com.
According to Inside AdWords, "... video ads will compete for placement on sites in the Google content network with other text, Flash and image ads." Advertisers will bid for placement of their video ads using the same cost-per-click (CPC or PPC) or cost-per-thousand impressions formulas used by competing AdWords advertisers.
The pay per click advertising model is widely credited as the primary factor behind the rapid growth and evolution of the business of search. Google makes over 9/10 of its money from paid search ads showing consistently higher revenues based on the strong growth of its various search advertising services.
A sizable chunk of those revenues have come from the traditional media. The newspaper and magazine sector is trying to diversify online revenue streams, reeling under the weight of eBay, Craigslist, Google and Yahoo. The advent of pay per click video advertising is likely to place broadcast and cable networks in a similar or even less enviable position.
In an Adweek essay published in December 2005, Noreen O'Leary details the multiple challenges faced by traditional advertising distributors when confronting consumers who expect on-demand content delivered immediately. Noting the shift in advertiser spending patterns, she writes, "There was a 25-year lag between when network TV reached mass consumption and when ad dollars caught up. It took 15 years for cable. After 10 years, is the same about to occur for the Internet?"
A lot more money is about to move into the online medium. Even if the next six to twelve months are more a time of trial and testing, there is no way advertisers are going to resist creating their own video ads. The applications for advertisers are endless and suddenly very inexpensive. Search engine marketers, website owners and online advertisers should be preparing to deal with the video advertising market as it expands.
Google appears to be making it as easy as possible for smaller advertisers to upload video ads by allowing for any video format. Video ads are uploaded to Google servers where they are stored and served based on contextually similar content found on the sites displaying them.
"A large percentage of video ads will come from small advertisers," said Gokul Rajaram, a director of product management at Google in an interview with the New York Times. "A small resort owner in Maui probably already has video of their great beachfront property. Now they can put it in an ad and reach a qualified set of users."
Small to medium sized advertisers also have the ability to geographically target ads and to test different versions of their video ads. One of the greatest benefits of online advertising is the ability to track where every click came from and when, giving advertisers and marketing professionals a great deal of instant information about what does and does not work.
In his Screenwerk blog, Greg Stirling notes that the number of targeting options, combined with the ability to track click patters could also benefit larger advertisers testing consumer reaction to commercials before they commit to a much more expensive ad-spend across broadcast or cable television. "One use of this system (and presumably others that will develop shortly) is to "A/B test" creative before launching a campaign on TV. This is sort of like what's being done by some marketers who get users to vote on several different versions of the same campaign online."
This is potentially the most disruptive move Google has made since the day it opened its doors. Paid search advertising will have a driving force in the migration of advertising money away from the traditional print and broadcast media. That migration, in turn, will cause enormous changes in the search marketing sector.
The timing of Google's announcement coincides with the purchase of the broadband video advertising firm Lightningcast by AOL last week. Lighningcast inserts commercials into online video content found on sites such as Space.com and Nascar.com.
Unlike Google's video ads, Lighningcast executive Matt Wasserlauf is quoted by the NYTimes saying, larger marketers would rather their commercials be included in streaming content instead of being an include on a static web page. "No one will click to watch a Pampers ad," he said. For example, he said, Procter & Gamble would rather "put Pampers on relevant or entertaining content."
What consumers prefer remains to be seen but, if Google has placed its bets well, allowing consumers and advertisers more control over choices made in the process of exercising their preferences can only be a good, and incredibly profitable, thing.
Sometime around January or February, a number of webmasters began to notice that Google had somehow "lost" huge portions of their websites. Reference to their sites, generally to the index pages and a seemingly random selection of internal pages existed in Google listings but pages that once drove sizable amounts of traffic appeared to vanish into the ether. As February rolled into March, more reports were posted to blogs and forums by frustrated webmasters who started to notice the number of pages from their sites had declined, significantly, in Google's index.
Many SEO firms, including StepForth, received information requests and research projects from clients who wanted to know what had happened to their sites. In all cases, we did the best we could but, given the obvious complexity of the update and the lack of fresh information from Google, recommendations given during this period have more resembled shotgun style SEO advice than the finer laser focus most of us would normally prefer to offer our clients. As is the case with most major updates, investigation as often as not leads to more questions.
Matt Cutts, Google's Search Quality Officer and #1 communicator, answered many of those questions yesterday in an open and wide ranging post titled, "Indexing Timeline".
The post outlines how Google staff have examined and responded to webmasters' queries and complaints stemming from the Bigdaddy update. It also addresses a number of issues webmasters who have seen sections of their pages disappear from the SERPs including the quality of both in-bound and out-bound links, irrelevant reciprocal linking schemes, and duplicate text found on vertical reference and affiliate sites.
According to his timeline, on March 13, Googleguy asked webmasters to offer example sites for Google's analysis in a post at WebmasterWorld. Commenting on the sites offered up for examination Cutts wrote,
"After looking at the example sites, I could tell the issue in a few minutes. The sites that fit "no pages in Bigdaddy" criteria were sites where our algorithms had very low trust in the inlinks or the outlinks of that site. Examples that might cause that include excessive reciprocal links, linking to spammy neighborhoods on the web, or link buying/selling. The Bigdaddy update is independent of our supplemental results, so when Bigdaddy didn't select pages from a site, that would expose more supplemental results for a site."
That quote covers a lot of ground but it explains a great deal of Google's post-Bigdaddy behaviour.
Google bases its ranking algorithm on trust. That might sound naïve to the uninformed, but we are discussing one of the most informed electronic entities that has ever existed. Google also keeps historic records on every item contained in its index. Though it bases its opinions on a baseline of trust, those opinions are extremely well informed.
In order to remain continually informed, it spiders everything it can and sorts the data later. Google maintains a massive number of indexes including one known as the supplemental index. The supplemental index is a much larger representation of documents found on the web than those included in the main Google index.
"We're able to place fewer restraints on sites that we crawl for this supplemental index than we do on sites that are crawled for our main index. For example, the number of parameters in a URL might exclude a site from being crawled for inclusion in our main index; however, it could still be crawled and added to our supplemental index." (source: Google Help Center)
Many of the results that appeared to have disappeared are assumed to have been drawn from the supplemental results before the update. "A supplemental result is just like a regular web result, except that it's pulled from our supplemental index".
As Cutts is quoted saying above, Bigdaddy results are separate from supplemental results. When a reference to a site is found in the main (Bigdaddy) results, Google does not necessarily dip into supplemental results as often as it might have previously.
Quality On, Quality In and Quality Out
Google has gotten better at judging the quality of content found on a document and within a site. Content includes text, images, titles, tags and both inbound and outbound links. Consistently said that well-built sites offering quality information and a positive user experience should perform well throughout its search indexes, Google provides a wealth of information via the Google Help Center and through its webmaster focused spokespersons, Cutts and Googleguy.
As Google has gotten better at determining the origin and history of content found in its various indexes, it tries to snip away at duplicate forms of on-site content, with the goal of listing the most trust worthy sites under any given user query in the main index.
Having been inundated over the years with multiple replications of what was already considered duplicate content. Google (and other search engines) has gotten very good at knowing if it has already indexed similar or duplicate content. Google is capable of examining text (including individual paragraphs), images and link networks (in and outbound links), looking for telltale signs of duplicate content.
If, for example, it perceives a site displaying product information pulled from the same product database that 25,000 other sites pull duplicate product information from, Google is not likely to rank that site well. Similarly, if it finds duplicate networks of reciprocal links shared among several pages in its index, it is not likely to assign a high trust value to that document.
Reciprocal linking strategies
"As these indexing changes have rolled out, we've improving how we handle reciprocal link exchanges and link buying/selling."
Though Cutts points at reciprocal linking as an indicator to Google that there might be issues with a website's credibility, that doesn't automatically mean that all reciprocal links are going to cause problems for webmasters. Common sense and the value of delivering a quality user experience should dictate decisions around link strategies.
For example, if a professional landscaper provided links to plant nurseries in his or her region, and those nurseries in turn provided links to that landscaper, Google would likely consider those to be quality links. There is a direct relevance between the two sources of information. A network of links between local landscaping businesses, nurseries, horticultural institutes, permaculture initiatives, non-profit volunteer groups and a number of gardening centers, shared amongst a relevant set of websites would also likely be judged beneficial to Google users and not subject to supplemental penalization.
On the other hand, a network of obviously purchased links between anyone who will exchange links with each other, regardless of relevancy or direct user benefit is likely to trip any number of filters present in the Bigdaddy/Jagger upgrades.
Cutts provided an example of a simple error made by a real estate site. Along with a number of internal reference links to exotic properties displayed as a footer-style site-map, Cutts found several out-bound links with anchor text reading; 1-exersize-equiptment.com, Credit Cards, Quit Smoking Forum, Hair Care, and GoSearchFor.com. When he reset, he saw a similar set of links, only this time, the out-bound links were directed towards, mortgages sites, credit card sites, and exercise equipment. Cutts commented, "...if you were getting crawled more before and you're trading a bunch of reciprocal links, don't be surprised if the new crawler has different crawl priorities and doesn't crawl as much. "
Affiliate Text and Content
Cutts devoted a long paragraph covering affiliate text, mentioning a T-shirt site that once had about 100 pages indexed, a number recently reduced to only 5.
"The person said that every page has original content, but every link that I clicked was an affiliate link that went to the site that actually sold the T-shirts. And the snippet of text that I happened to grab was also taken from the site that actually sold the T-shirts. The site has a blog, which I'd normally recommend as a good way to get links, but every link on the blog is just an affiliate link. The first several posts didn't even have any text, and when I found an entry that did, it was copied from somewhere else. So I don't think that the drop in indexed pages for this domain necessarily points to an issue on Google's side. The question I'd be asking is why anyone would choose your "favourites" site instead of going directly to the site that sells T-shirts?"
The Ghosts of minutes past
We live in the present. Our websites live in the past as well as the present. Google keeps tabs on all documents in its index and even if it has, "... spidered content that was posted only moments before,” it has an elephant's memory for previous details and a computer's ability to pull lots of information together to get a bigger picture of how all those details fit together.
Google works by following links. Google ranks by examining the quality of content found on a site and also on the sites that link into, or are linked to from, sites in its indexes. If you have seen a great deal of page content fall away from Google's index, or if you are just generally interested in how Google is working, read Cutts' Bigdaddy "Indexing Timeline".
Yahoo! has given itself a cosmetic but functional facelift that blends most (if not all) of Yahoo!'s subscriber based products and features with a sleeker, easier to use layout.
Yahoo! remains an information dense portal but good use DHTML and Ajax in designing an expanding menu system makes finding your way around the vast array of options much simpler. Yahoo! has also introduced a number of personalized information tools that present emails, messages, and music options, along with localized content such as movie reviews, playtimes, local traffic and local weather.
The first thing users will notice is the cool blue-grey toned color scheme used on most of the page that replaces the compilation of colors used on the old home page. A Page Options link below the Web Search button allows users to change the basic color scheme or return to the old version of the page. Though the page feels less busy than its predecessor a second glance shows that there is access to far more content and subscriber based services.
Citing this as the biggest visible change to the Yahoo home page in its twelve year history, notes Yahoo! search blog editor Havi Hoffman. She goes on to say, "The new home page reflects Yahoo!'s unique position at the intersection of people, media, and knowledge."
That might be taking it a bit far but Yahoo!'s new home page looks and feels far more accessible than the previous one did. A third glance at the page reveals that Yahoo! has far more features and services available than most users could have realized. On the front page, there are almost 50 visible user listed options above or below the search query box.
Yahoo!'s designers were faced with the terrible task of presenting as many usable products as possible, preferably above the fold. A key directive was to accentuate Web Search. Whether the query box is more prominent or the additional information provided by the portal is less cluttered, it appears easier for the eye to see than on the old version of the home page. Directly above the search box are links to the eight specific search types supported by Yahoo!, web, images, video, audio, directory, local, news, and shopping.
Directly below the query box is the new and improved Yahoo! Answers feature. Yahoo! Answers is a live-time question and answer forum where users can pose questions and receive response from other Yahoo! users. Many of the questions seem to come from younger users asking relationship advice.
In response to "JustCurious", go for it dude! Moving in with your girlfriend at age 21 is something everyone should do at least once. It's the 2K's so chances are; you'll do it a couple of times. For "Wonderingwife", yes, given the information you've offered, most reasonable people would agree they guy is a jerk. I think you already know that though or you probably wouldn't be asking. This is fun. Yahoo! Answers could quickly become addictive. Moving right along...
The page splits itself into three sections at this point. To the far left is an eighteen item menu listing general Yahoo! services ranging from Automobile listings to Yellow Pages search. A link to All Yahoo! Services can be found below this menu. Below that, links to small business services such as web hosting and Yahoo! Shopping and feature services such as 360 and Mobile are available.
Moving to the center of the screen, the main features box offers users four infotainment options including from Featured content, Entertainment, Sports and Money news. Today's Featured content includes a humorous video introducing the new home page starring Yahoo! co-founders David Filo and Jerry Yang.
The video opens with a director's assistant noting this is the third and final take before showing Filo and Yang sitting cross legged on a jury-rigged desk. Mixing elements from Bob Dylan's Subterranean Homesick Blues video and any number of Jay and Silent Bob routines, Yang and Filo welcome users to the new Yahoo!, getting a clever Mothers' Day greeting in at the end.
Access to Yahoo news is provided in a space below the Featured content box with tabs offering localized and world news headlines and video sources.
Below the News box is a section labeled Marketplace that appears to display third-party commercial advertising including ads touting 1yr Degree granting schools, Blockbuster Online, Sylvan Learning Centers, and Financial services.
The personalized section mentioned previously appears on the far right hand side of the monitor. The personalized menu is functionally cool using DHTML to provide a series of expanding selection boxes. For instance, I can check emails received by at my Yahoo! address by mousing-over the mail link. A box expands showing the four most recent emails. Similarly, I can check traffic conditions in the nearest large US city (Seattle) by simply drawing my mouse over the link marked "local". The menu item expands to show a map of the Seattle area displaying a number of traffic snarls that have developed over the past few minutes. (It is the start of rush hour on the Pacific coast).
Two new features appear on the revamped Yahoo! home page. The first shows the most popular entertainment searches of the day, with Angelina beating Jessica and Shakira as today's most popular persona. The second and more interesting feature is called Yahoo! Pulse. Pulse details the top searches under a variety of categories, today's being apparel. Shoes, prom dresses and sunglasses are the three most popular apparel items being sought today, in case you were interested.
The redesign of Yahoo!'s home page has been a long time coming. This design is a good leap forward and will likely be welcomed by Yahoo! users. That said, the new look can't compensate for the enormous volume of information that threatens to overwhelm casual searchers. Fortunately for those who like a sparse look, Yahoo! maintains a much sparser interface at Search.Yahoo.Com.
Though Yahoo! editor, Havi Hoffman sees Yahoo! at the intersection of people, media and knowledge, the new interface makes me feel like I am being deluged by data streams. If there actually is an intersection of People, Media and Knowledge, Yahoo! is hoping the traffic signals it has built will help users navigate a vast ocean of information fled by multiple services.
A working analogy is difficult enough to draw. Imagine how intimidating it would be to create a functional application. Yahoo! designers should take credit for making a friendlier and more easily approachable home page.
Yahoo is not going to sell its self in part or whole to Microsoft. Over the past two weeks, there have been rumblings of a potential merger or outright acquisition involving the two tech giants. In an interview with the Financial Times, "Yahoo rebuffs Microsoft offer", Yahoo CEO Terry Semel confirmed rumours regarding discussions between the two firms but stated unequivocally that Yahoo is not for sale.
"The search business has been formed”, said Semel. “My impartial advice to Microsoft is that you have no chance."
Semel was speaking to a group of media executives at a conference organized by Syracuse University's Newhouse School of Journalism on Wednesday. In his session, he shed some light on the topic of talks between Yahoo and Microsoft. He also spoke about his views on the future of search noting social search as the next evolutionary step.
Microsoft and Yahoo had been in discussions for some time as part of a general exploration of plans to cope with Google's dominance of the search business. From the tone of Semel's comments, those talks did not lead to any deepening of relationships between Yahoo and Microsoft.
In fact, relationships between the two firms appear to be strained. Semel spoke specifically about Yahoo's wish to continue serving paid advertising results via Yahoo Search Marketing (YSM) to MSN search. Microsoft decided to allow that deal to expire in order to replace YSM with paid ads generated through its new internal paid advertising program AdCenter.
Microsoft was interested in buying a stake in Yahoo search however Semel says he was not willing to sell a portion of the company. Semel admitted Yahoo and Microsoft had discussed, "... Microsoft co-owning some of our search", but stated, "I will not sell a piece of search - it is like selling your right arm while keeping your left. It does not make any sense."
Semel also mentioned some of his ideas on the future of the search medium. The business of search and provision of search results is changing rapidly though Semel expects those changes to be manageable for larger firms like Yahoo.
"I do not expect search to decline but it may not be the way you do it two or three years later," he said. Semel said he sees a great deal of growth in social networking and online video, noting the growing availability of broadband access in the US and around the world is ushering in an era in which video becomes standard content on the web.
In 2004, Yahoo announced its intention to create video content for its users however Semel said many of those plans have been shelved because they were too similar to content already available on commercial television.
"Television does a very good job," Semel said. "This medium better look like something new. This medium better take advantage of its assets, take advantage of what it does well. ... If what we do looks like television, then we're making a huge mistake."
Yahoo continues to make content provision a high priority however much of that content will likely be syndicated from other sources.
Semel mentioned social networking as a likely future model for search saying, "Knowledge search, as they call it in Korea, or social search, as we call it, has blown through the roof. There may be changing dynamics."
Meanwhile, over in Microsoft's camp, CEO Steve Ballmer appears to be in fighting trim playing point guard against Google's towering offensive while lobbing set-up passes regarding Microsoft's plans toward the press. In a wide-ranging article published in Cnet News.com, Ballmer responds to a number of comments directed at Microsoft in the past weeks.
Ballmer doesn't appear to be concerned about Yahoo but he is obviously burning over some of the comments Google co-founder Sergey Brin levelled at Microsoft during a press conference on Wednesday's Google Press Day.
Referring to the default settings of Microsoft's upgraded browser IE 7 and the pending release of the Microsoft Vista operating system, Brin was quoted saying, "We just see the history of that company behaving anti-competitively and not playing fair. So I think we want to... look at the areas where that power can be abused."
Suggesting Google expects special treatment from Microsoft, Ballmer couldn't resist the temptation to take a few pot shot at Google while comparing Microsoft products to Google's offerings.
"Can you imagine writing a letter to someone," he said, "'Hey, Mom, I am upset with the gun policy.' Then an ad pops up and says, 'Hey, do you want to buy a gun?'"
Ballmer stated that most Microsoft products will remain separate from advertising noting that, "... Even though Microsoft will have ad-supported applications, a lot of people will want a standard desktop Office."
Microsoft is a distant third behind Google AdWords and Yahoo Search Marketing in terms of market share and revenues. It hopes to change that through the introduction and marketing of AdCenter, as well as by expanding into ad distribution in video games.
Earlier this month, Microsoft purchased Massive, a company that inserts advertisements into online video games. Imagine, if you will, a first-person strategy game that takes place in a city. Billboards, posters and other pieces of scenery can be instantly updated throughout MSNs gaming network.
"We want to make sure there is good, healthy competition in the advertising space," Ballmer said. "Everybody deserves good competition. People have been telling me that for years."
Based on notes taken by CNet news writer Michael Kanelloa, Ballmer is also interested in social networking, with a particular emphasis on FaceBook. He quotes Ballmer saying, "I've spent a lot of time studying FaceBook. I think there is a lot we can learn from the FaceBook concept." Remaining bearish on technologies that do not have clear revenue models such as YouTube and BitTorrent Ballmer suggested that Microsoft would continue to look at acquiring start-up companies.
Yahoo CEO Terry Semel and Microsoft CEO Steve Ballmer are both concerned about Google. They appear interested in perusing similar technologies and both perceive virtually limitless futures.
Yahoo's strongest assets are in its network of branded information directories, its emerging Yahoo Publisher Network, and in its search marketing business YSM. Having one of the most well recognized global brand names, combined with the enormous amount of content it generates, Yahoo is capable of retaining its second place position in search advertising.
Putting aside its control of the basic operating system on most personal and corporate computers, Ballmer sees patience as Microsoft's greatest asset. "We have tenacity and a persistence and patience to stay after it and stay after it and stay after it. Patience is what distinguishes us from many technology companies that are important...."
Patience has proven to be a key ingredient in the development of the Vista operating system. That makes sense given patience is a necessary virtue practiced by all Windows users.
Historically each of the major players, Google, Yahoo and MSN has competed by staking their own turfs and making minor, and in many cases major incursions into each other's turf. It often resembles a game of "follow the leader" in which 3 firms, based on their own merits are in a position to act as the leader. How it all works out will be part of our collective history but while it plays through, it is one of the most interesting games going.
Lee Roberts makes a shopping cart, content management solution known as ApplePieCart, or more appropriately, he used to. On April 11, 2006, Lee received a note from Apple Computer's lawyers demanding he stop
Blink
Del.icio.us
Digg
Furl
Google
Simpy
Spurl
Y! MyWeb