Yahoo Directory - Retired!

Reposted from Search Engine Land. Click here for full story.

For some, it may seem like there has always been Google, and it’s always been the main way people have found things on the web. But before Google, there was the Yahoo Directory and its hand-compiled list of websites, organized into topics. Now, the venerable Yahoo Directory is closing.
Yahoo shared the news today, a short paragraph that’s part of announcing two other closures, under a headline of “Progress Report: Continued Product Focus.” That sure fails to give the Yahoo Directory its proper due. From Yahoo’s post:

Yahoo was started nearly 20 years ago as a directory of websites that helped users explore the Internet. While we are still committed to connecting users with the information they’re passionate about, our business has evolved and at the end of 2014 (December 31), we will retire the Yahoo Directory. Advertisers will be upgraded to a new service; more details to be communicated directly.

It’s about as terrible a send-off as Yahoo gave AltaVista, another pioneering search resource that Yahoo closed last year. AltaVista got a single sentence to cover its coming death.

I get that companies might not want to celebrate the closure of products. But there had to be something less shameful than the way Yahoo announced the coming closure of the directory from which Yahoo literally takes its name.

When Yahoo Ruled The Web

When Yahoo began in 1994, the most common way people found websites was to browse through lists, where sites were often organized into topics.
Founders Jerry Yang and David Filo quickly realized their own list needed a better name than “Jerry and David’s Guide to the World Wide Web.” They settled on Yahoo! (with exclamation mark). That stood for “Yet Another Hierarchical Officious Oracle” — a reference to organizing sites in a hierarchical order.
Yahoo quickly became the most popular search engine on the web. By search engine, I mean the generic term of any service that helps you locate information, rather than the more technical one of a service that locates information by “crawling” pages automatically and allowing you to keyword search to find any pages that match your keywords.
The distinction is important, because it lead to Yahoo’s eventual downturn as a search resource and directly to the coming closure of the Yahoo Directory that was announced today.
A “directory” relies on humans to review websites, summarize them with short descriptions and organize them into a categories. When Yahoo started, this system was effective, because there weren’t that many pages on the web (relatively speaking) and automated search technology to organize websites wasn’t very good.
That contributed in a big way to Yahoo’s popularity over traditional search engines like InfoSeek or Excite that used automation. Yahoo had better relevance. You could find what you were looking for.

Google: The Yahoo Directory Killer

I don’t have time at the moment to do an extended timeline of the decline of the Yahoo Directory (as well as Yahoo as a search resource), but I’ll hit the highpoints of what led to the Yahoo Directory’s low point today.
First, Google. Google came along in 1998 and quickly proved that you could have search results generated by automation, rather than through human effort, and have better relevancy.
With Yahoo, it was as if you were searching for books in a library using an old-fashioned card catalog system, where all you knew about a book was 25 words or so describing it.
With Google, it was as if you could search through every page of every book in the library. You didn’t miss that needle in a haystack. And importantly, unlike previous search engines that used automation, you didn’t find that the “noise” of looking through all those pages drowned out the important relevancy “signal.” Google’s search algorithm was better than others.
Indeed, Google’s search algorithm was so good that in 2000, Yahoo partnered up with Google. Yahoo had long partnered with crawler-based search engines, so that if there were no matches from its own directory, the crawler search engine provided some “backup” answers. Google became Yahoo’s fourth partner: Open Text was the first, AltaVista the second and Inktomi the third.
By 2002, Yahoo realized that Google’s search results were so good that it made a dramatic shift and presented those as the main listings to searchers at Yahoo. That was effectively the start of the Yahoo Directory’s long, slow-death. If you knew where the directory was, you could search it directly. And category links from the directory were also inserted alongside Google listings when relevant. But it was pretty much an afterthought, forgotten, dethroned.
Indeed, in 2003, Yahoo completed the purchase of Inktomi, so it could have its own crawler-based listings. That led to the eventual end of its deal with Google. But by then, for Google, it didn’t matter. Google had long eclipsed Yahoo as the most popular search engine out there. Eventually, Yahoo even abandoned its own search technology and outsourced to Microsoft for it.