Size Less Important Than User Utility

Last week saw the resumption of the search engine size wars in which one major search engine claims to be larger than its rivals, prompting those rivals to rapidly upsize themselves. Yahoo fired the first round at Google, claiming to have over 20billion objects accessible in their database. Google, which can only claim about 13billion objects fired back with questions about measurements, basically stating Yahoo was mistaken or misleading in its claims. Others got in on the act and the blog-o-sphere was full of stories about Yahoo's obsession with size. By the beginning of this week, the search marketing community was fed up with being fed tripe about the importance of size, as reflected August 16 th in Danny Sullivan's post to Search Engine Watch, " Screw Size! I dare Google and Yahoo to Report on Relevancy "

The frustration with the major search engines felt by serious search marketers is real. Our clients don't care about size and neither does their money. They care about being found when searchers are seeking information about products or services they sell. They care about potential clients and their ability to present information to them. They care about being relevant.

Search engine users don't really care about size either. Given the mind-boggling amount of data available via even the smallest of the major search engines, most users have no idea of the depth of search results, as they tend to look only at the Top10 or 20 listings. Even if Yahoo returns thousands more references than Google for any given keyword query, both know that only the first 20 links tend see any measurable traffic. Again, it isn't about being the biggest; it is about being the best. Being biggest does not necessarily mean being best.

There is no real scientific method of proving which search engine is the biggest, and no real way to gauge which one is best. That's not to say folks aren't trying though. The thing to remember is, "best" means something slightly different to every search engine user.

Over the past four months, well known search-blogger RustyBrick has been tracking search engine relevance through RustySearch an ongoing blind user-test of results drawn from a random Big4 search engine that is reminiscent of the Pepsi-Challenge. The RustyBrick site has posted live-time results and an explanation of the methodology. As it turns out, the Big4 search engines all return relevant results most of the time with Yahoo enjoying a slight lead over Google in overall relevancy. The biggest problem with this study is that users who tend to be very well versed in search engines and search marketing provide the bulk of the data. In other words, the study group is not likely to be highly representative of the far greater majority of general search users. Nevertheless, it does provide the only ongoing view of how its unique study group rates search engine result pages generated by the Big4.

The National Center for Supercomputing Applications (NCSA) at the University of Illinois took up a study of results generated by Google and Yahoo in reaction to Tim Mayer's August 8 th post at the Yahoo Search Blog that started this round of the size war. Basing their findings on the average number of results returned from 10,012 random queries, the results clearly show Google tending to return (on average), three times the number of references Yahoo does. The study-document states quite early that there is no real way to test the actual size of the index so its methodology is borrowed from a method pioneered by search legends Krishna Bharat and Andrei Broder that samples the size of search results based on words derived from previous search queries. The NCSA study shows Google gives a greater number of results than Yahoo though it in no way intended to suggest those results are better or more relevant.

In some ways, the results of the two studies seem to cancel each other out with Yahoo nosing ahead of Google in the first and Google squashing Yahoo in the second. Unfortunately, neither study offers a conclusive analysis of which search engine is the best. RustySearch provides data on which search results are considered most relevant, showing Yahoo and Google receiving similar relevancy ratings even though the NCSA study demonstrates Google returns far more results than Yahoo does.

Perhaps the two studies combined show that Yahoo has better filters than Google does though that still does not show conclusively which is the better of the two. As their results appear to be relatively equally relevant, perhaps a measure of which is the best comes in overall usefulness to searchers and, ultimately to advertisers.

The real interesting competition between all the major search engines is being waged on the battlefield of usefulness and user loyalty. This is the user-focused space in which one or more of the Big4 will eventually rise to dominate the various sectors of search. The question is, what is useful to search engine users?

Over the past few years each of the major search engines has introduced a number of new tools that on their own might not seem to have a lot to do with organic search results but collectively have a lot to do with the business of search.

For example, Google, Yahoo and MSN each offer sizable free email accounts to their users, some of whom like myself maintain addresses at all three. Google and Microsoft are sparring over satellite mapping technologies, expanding on the usefulness of maps in relation to local search users. Earlier this week, Google introduced Froogle Local, a revolutionary simple application for mobile device users.

Yahoo has also made improvements to their local search system, this time by allowing registered users to write Wiki-like reviews for local establishments and services. These reviews will be available to other registered users and distributed to members of the reviewer's social network. Earlier today John Battelle wrote an excellent review of improvements to Yahoo Local and their usefulness to Yahoo searchers.

Business 2.0's Om Malik says Google appears to be ready to introduce a national WiFi network while Yahoo's VOIP based upgrades to its Instant Messaging client. Its purchase of internet phone company Dialpad has analysts speculating Yahoo is about to enter the VOIP cyber-phone market, a rumour Yahoo is vigorously denying . Regardless of their denial, the VOIP based improvements to the IM are indicative of much larger plans around digital voice transmission.

The point to all the information above is simple. Google and Yahoo are both working feverishly to provide their loyal users tools that are useful to them. There are dozens of other examples of user-friendly tools created to capture user-loyalty such as blog support, desktop search applications, toolbars of varying shapes and sizes and personalized syndication feeds. These types of applications and labour saving tools are what the search tools are betting on to retain current users and win new ones.

As demonstrated by the most recent Nielsen Net Ratings , the strategy seems to be working with Google, Yahoo, and MSN (which offer the greatest number of user-focused tools) leading the pack by a wide margin. Guess which of the three offers the most useful tools to the greatest number of searchers? Now, guess which of the three can offer the biggest distributed bang for an advertiser's buck. That's what our clients care about.

by Jim Hedger, News Editor