Why Google Isn’t Enough
Web 2.0 has annoyed legions of information technology professionals by providing an experience for consumers that, in many ways, is just plain better than what everyone gets at work. At some point after the year 2000, consumer companies grabbed the ball from business technologists and led the way with innovation. One key refrain that expresses this trend is heard in companies around the world: "Why can’t we have a Google inside the four walls of our company?" While at first this seems like a good idea, the problem of using search inside a company is much more complicated than just indexing documents, throwing up a search box and asking people if they feel lucky. This week, JargonSpy explores just what "enterprise search" means and why it is a complicated challenge that is becoming increasingly urgent for most companies to solve. The first realization that enterprise search is an entirely different animal comes in the rush of disappointment that follows the installation of a search appliance from Google (nyse: GOOG - news - people ) or some other vendor. Instead of the satisfying experience of finding information that was previously unknown, the results page often shows a million results in a random order, or perhaps it shows none. While this is an improvement over no search at all, it is not what we are used to getting from the best searches on the Web, where relevant results usually cluster at the top. The gap has many causes. First, and most important, Internet search is built on the link network that constitutes the Web. Every time someone links to content, he is effectively voting for its importance. This is the genius of Google’s Page Rank and related ranking algorithms: They favor pages based on incoming links. In the enterprise, there is no corresponding link structure and the quality of the results suffer. Other problems include the varying formats of documents–everything is not HTML and PDFs like it is on the Web–as well as varying access and file permissions systems. And it isn’t unusual for confidential documents that were left lying around to show up in search results. On the other hand, people inside companies are usually not trying to manipulate search results through inaccurate tags or link bombing. The clues to the relevance of a document usually can be taken at face value. But the biggest difference is that when you are searching in the enterprise, you might not exactly know what you’re looking for. Your desire is frequently not to find a particular piece of information but to explore the collection of available knowledge and find out what’s there. Even in searches that are not intended to be exploratory, it is common to find that results are ambiguous and in need of interpretation and refinement. In library science, there is a whole set of best practices involving specialization, research interviews and consultation between librarians and researchers. The key to getting enterprise search right is to imitate these processes in an automated system. As my guru for enterprise search, Daniel Tunkelang, chief scientist of Endeca, puts it, "We know the system is doing its job when it is telling you something you don’t know." Resources for
