The future of search: Do you ask Google or the gaggle?
To improve results, new search engines rely on users instead of computers.
Lisa Haney
New York
Late last month, media giant Google launched an online featured called SearchWiki, which allows users to rate, annotate, and store results they鈥檝e found particularly useful. The notes have no direct bearing on public rankings, although individual comments are visible to all users.
On the company鈥檚 blog, Google said SearchWiki moved search one more step toward a 鈥渄ynamic鈥� search experience 鈥� one in which a community will be able to shape, refine, and organize the raw matter of the World Wide Web.
The concept has a considerable amount of ballast in Silicon Valley, where developers have long predicted that the future of search lies not in proprietary algorithms, such as Yahoo or Google, but in the power of the hive mind.
Over the past few years, a score of so-called 鈥減eople-powered鈥� search tools have entered the fray, including Stumpedia, Mahalo, Sproose, and Gravee. Most of these sites couple the raw processing power of an algorithmic engine with the functionality of Digg, the community-controlled news aggregator.
鈥淭here are a lot of smart people who have looked at Google and Yahoo and said the fundamental way of searching has not changed in nine or 10 years,鈥� says Bob Pack, a founder and CEO of Sproose, which allows users to influence search results with a simple voting mechanism. 鈥淵ou鈥檝e got algorithmic search results, organized into a set of blue links going down the page. Search needs to become richer and more intuitive.鈥�
This community-based approach to search will likely never replace traditional engines when it comes to simple searches, such as checking sports scores or the state of the stock market. But more complex tasks are still handled more effectively by a human.
鈥淟et鈥檚 say you鈥檙e looking for a hotel room. What you find on Google is endless realms of affiliate marketing sites, selling you the same rooms,鈥� says Jimmy Wales, founder of Wikipedia and Wikia Search, an 鈥渙pen-sourced鈥� search tool launched earlier this year. (Unlike Google鈥檚 SearchWiki, which is a feature within Google鈥檚 traditional search function, Wikia Search is a stand-alone site.)
鈥淚t鈥檚 hard to get the hotel site itself,鈥� he says.
A people-powered engine, on the other hand, would take into account the experiences of other users and steer you directly to the best-rated hotels in the area.
Battle of philosophies: Democratic or efficient?
Proponents of community-assisted search also point to the essentially undemocratic nature of the top three traditional portals: Google, Yahoo, and Microsoft.
Each company zealously guards its internal algorithms, and search results can be subject to internal editorial control. In an industry with huge profit margins 鈥� search is a multibillion-dollar industry 鈥� the lack of transparency can be unsettling, Mr. Wales argues.
鈥淚 like to make the analogy between search and journalism,鈥� he says. 鈥淪earch in a certain sense is reporting on the world. Now, for most high-quality papers, there鈥檚 a certain amount of transparency. You understand that advertising might influence the paper. But you also have a reporter鈥檚 byline, for instance. Because search is so secretive, and so propriety, there are fewer checks and balances.鈥�
Furthermore, Wales continues, most users rarely stray from the top 10 results coughed up by their favorite search engine.
Since Yahoo, Microsoft, and Google respond to each query in similar ways, there remains a vast swath of relatively pristine Internet wilderness waiting to be discovered. A search tool that effectively utilized human input could open up the Web in a startling way, and allow users easy access to information that once languished on the 475th page of Google results.
Major obstacles remain before users will switch away from their current portal, says Brad Bostic, founder and CEO of ChaCha, a community search tool for mobile phones. Google, for instance, has already won the allegiance of a majority of Web users, and the company name is synonymous in mainstream culture with online search.
Bob Pack, of Sproose, likens the situation to the battle between Coca-Cola and Pepsi.
鈥淕oogle is Coke,鈥� he says, 鈥渁nd Yahoo is Pepsi, and then you鈥檝e got all the other flavors. Google鈥檚 so well-branded.鈥�
It鈥檚 a Catch-22, Mr. Pack adds: If people-powered sites could attract millions of users, the quality of rankings could be more dynamic and democratic than anything yielded by a traditional algorithmic engine. Many Google aficionados, however, would be loath to jump to another engine unless the results were already top-notch.
More practically, there is a concern among some developers that handing over control to a community could engender a flood of spam, or devolve into a mess of internecine backbiting among users. (This is the case on the Yahoo Answers site, where queries are answered haphazardly, or with a string of pejoratives, or 鈥� less frequently 鈥� not at all.)
Finally, the 鈥渞ichness of information鈥� on a people-powered site is often much more unwieldy than the data provided by Google, says James Segil, the president of EdgeCast Networks, an Internet infrastructure company that helps power Mahalo. A major challenge for people-powered search companies is how to juggle the flow of user input, and keep results consistently updated.
There are signs that the search paradigm is already shifting. Google鈥檚 SearchWiki has received some good notices 鈥� and some bad 鈥� from tech reviewers, and brought the community-search debate to center stage.
Some progress for smaller sites
ChaCha, which uses an army of more than 55,000 amateur 鈥済uides鈥� to help relay information via SMS text, has enjoyed what Mr. Bostic calls an extraordinary leap in growth. According to information provided by ChaCha, the company has logged more than 56 million queries to date, up 53 percent from January to November of 2008. Thousands of new users experiment with the service every day.
鈥淭here鈥檚 a feeling of information pollution out there,鈥� says Melek Pulatkonak, president and chief operating officer of a search site called Hakia. 鈥淢illions and millions of results [on Google] terrorize the searcher, preventing them from moving forward. People want something different.鈥�
Hakia is not exactly a people-powered engine. Instead, it uses a homegrown algorithm to analyze the wording of a query, thus delivering results more contextually accurate that than of Google. (If a user were looking to 鈥渢reat鈥� a cold, for example, Hakia would understand that the word 鈥渢reat鈥� did not refer to candy.)
The site, now in public beta testing, simultaneously maintains a growing network of 鈥渃redible鈥� sites, vetted by a network of librarians at hospitals and schools around the country.
This combination of algorithmic and human input, Ms. Pulatkonak says, is very attractive to the average user. She does not anticipate Hakia would ever replace Google, but hopes instead that it will be used alongside Google, as a supplementary tool.
鈥淭o me the question is not whether humans should be making decisions,鈥� Wales says, pointing out that Google鈥檚 internal editorial control is people-power of a kind. 鈥淚t鈥檚 how they should be making the decisions. Should search be democratic? Or should it continue to be a top-down system?鈥�
[Editor's note: The original version of this article misstated Ms. Pulatkonak's first name.]