June 30th, 2007
Why do we trust established search engines?
The answer is simple: because we trust well-known brands more than others. Researchers from Penn State University (PSU) have integrated Google results pages into pages looking like coming from other search engines, such as Yahoo!, MSN Live Search or an in-house PSU search application. Then they asked a panel of students to judge which search engine offered the most relevant results. The testers found that the results from Yahoo! or Google were the best — even if they were exactly the same than the ones offered by the two other tools. The researchers conclude that branding matters — even when searching.
You can see on the left a typical search engine results page. It looks like it came directly from a “techno music” query to MSN Live Search. In fact, the researchers built this page by aggregating the branding elements of this search engine, such as its logo. Then they plugged the Google results in it. (Credit: Jim Jansen’s team, PSU).
Even the lead researcher, Jim Jansen, an assistant professor at the College of Information Sciences and Technology, was surprised. “Given that there was no difference in the results, all of the search engines should have had the exact same score,” Jansen said. “Some emotional branding is having an effect here.”
The team starts this research project when it realized that there are several thousands of search engines — in English only — offereing similar results. So why Google and Yahoo! are dominating this market?
And here is how the team proceeded. “To determine each engine’s ‘performance,’ participants rated the returned results on a three-point scale: very relevant, somewhat relevant, and not relevant. After averaging the scores, the researcher determined an average — about 36 percent of all results were judged relevant to the query. The researchers then looked at each engine’s ’score’ to determine whether it fell above or below the average. Participants ranked results from Yahoo! more relevant across the four queries.”
By the way, these four queries were ‘camping Mexico,’ ‘laser removal,’ ‘manufactured home’ and ‘techno music.’
This research work has been presented at two conferences this year, WWW 2007 and CHI ‘07. It has been incorporated into the CHI ‘07 extended abstracts on Human factors in computing systems under the name “The effect of brand awareness on the evaluation of search engine results.” Here are two links to the abstract and to the full paper (PDF format, 6 pages).
Sources: Penn State University news release, June 28, 2007; and various websites
You’ll find related stories by following the links below.
