|
Study Shows Search Engine Variety
By Jim Hedger, StepForth News Editor, StepForth Placement Inc.
August 5, 2005
» Click here for  PDF &  Word
Versions
A study of search results displayed at the four major search engines shows
that each engine is increasingly producing results that differ from the
other engines. In other words, searchers can be reasonably confident that
Google, Yahoo, MSN or Ask Jeeves will, more often than not, return results
unique to the engine being used.
The study, “Different
Engines, Different Results”, which was
conducted by the meta-search engine Dogpile and researchers from Penn State
and the University of Pittsburgh, examined search results from over 12,500
random, user-entered search queries on Google, Yahoo, MSN and Ask Jeeves.
Of the 12,500 inquiries, which produced a total of 485,460 results, only
5,301 or 1.1% produced the same first page results across all engines. The
significance of this change over the past year and a half magnified when
compared with a similar study released by Dogpile in May, which showed a
3% overlap in total search results.
In previous years search engines tended to draw results from two common
databases, Google and Inktomi, or were fed by established directories such
as the ODP or LookSmart. Before Yahoo began producing its own results in
February 2004, searchers routinely reported frustration with the repetitive
nature of results everywhere they searched, leading many users to simply
turn to the largest primary source, Google.
While it has only been a year and a half since Yahoo introduced its own
proprietary search engine, 18-months can represent an entire generation
in the IT world. Since then, MSN introduced its own algorithm search engine,
Ask Jeeves maneuvered itself back into the big leagues and a host of newer
search tools have emerged. The effect on search results has been enormous,
leading proponents of meta-search tools such as Dogpile to suggest that
brand-loyal search engine users are limiting their own options by using
only one or two engines.
The study says that people are already meta-searching when using more than
one search engine to find information. “While Web searchers who use
Google, Yahoo!, MSN and Ask Jeeves may not consciously recognize a problem,
the fact is that searchers use, on average 2.8 searches per month. Couple
this with the fact that a significant percentage of searches fail to elicit
a click on the first page of search results, and we can infer that people
are not necessarily finding what they are looking for with one search engine.”
The research team set out to examine the degree to which results on the
Big4 differed and/or overlapped, how placements differed across each engine,
and how meta-search tools such as Dogpile could do a better job of presenting
results amalgamated from all four engines. The study focused on only first
page results noting that 89.8% of all search result click activity originates
from first page results.
In its methodology the study shows the number of references, both organic
and paid, returned by each search engine. By default, each of the Big4 display
up to ten organic placements for any given search term, along with paid
or sponsored listings. Interestingly, MSN and Google both tended to show
slightly fewer results (paid or unpaid) than did Yahoo or Ask Jeeves. At
the time of the study, Ask Jeeves was seen to be displaying more paid advertising
than its competitors with an overall average of 3.3 ads per (first) page
of results. Yahoo came in second with 2.9 ads per page of results, Google
third with 2.4 and MSN a distant fourth with only 1.9.
Perhaps the most interesting part of the study shows that users of the
major search engines might be missing the majority of the web. By taking
the results of the 485,460 unique references generated using 12,500 queries
and comparing them with each other, the researchers concluded that, on average,
users of each of the Big4 search engines “miss” approximately
70% of materials referenced by rival search engines, as illustrated by this
grid diagram from the research document:
| |
Missed 1st Page Web Search Results
|
% of Web's 1st Page Results Missed |
| Google |
343,700 |
70.8% |
| Yahoo! |
337,144 |
69.4% |
| MSN |
349,561 |
72.0% |
| Ask Jeeves |
329,761 |
67.9% |
Clearly, search engines are starting to show their own personalities
as their algorithms increasingly present different search results for
the same keyword queries. How these findings impact search engine usage
remains to be seen however it is a rather safe assumption that details
from this and subsequent studies will be used to market alternatives
to search engine users in the future. For search engine marketers,
this study
shows the importance of learning about each of the search engines.
As Chris Sherman noted in his SEW
coverage, each of the search firms are
speaking with their own unique voices. It’s only a matter of time
before the searchers hear them clearly.
BACK to
the StepForth Search Engine News
|
