People Search Digest

It’s close race with Yahoo Search and Google neck and neck. The battle for top search engine rages on as Yahoo Search tries to be more like its worthy opponent, Google. This past week, on February 20, Yahoo shifted a crucial section of its search engine to Hadoop - software that handles large-scale distributed computing tasks with great efficiency. Hadoop enables applications to easily scale out to thousands of nodes and petabytes of data. It was inspired by Google’s MapReduce and Google File System papers.

Hadoop is a high level Apache project that has contributions from all over the world, but Yahoo has been its greatest provider thus far. It basically takes all the links on the Web found by the search engine’s crawlers and then makes them into a condensed map of the whole Internet so that ranking algorithms can be run against them. Yahoo is replacing their own software with Hadoop and running it on a Linux server with 10,000 core processors because it is about 34% percent faster than their old program.

So what does this seemingly small change to their search engine mean? Well, it’s the begging of a major change in page ranking for sites. The new “map” that Hadoop creates will differ from the previous map of information on the Web. This will create a new ranking algorithm, so sites that previously enjoyed top ten status in organic search results, might find themselves bumped down. We’re eager to see how these search cookies crumble!