O'Reilly logo

Search-Based Applications by Laura Wilber, Gregory Grefenstette

Stay ahead with the world's most comprehensive technology and business learning platform.

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, tutorials, and more.

Start Free Trial

No credit card required

CHAPTER 6

Data Processing

At A Glance

Characteristic

Search Engine

Databases

Processing

Natural language processing

Data processing

Principal technology

Semantics

Data mapping

6.1    SEARCH ENGINES

Many search engines prepare extracted content for indexing through a two step process: natural language processing, and assignment of relevancy criteria. Natural Language Processing serves three purposes: normalizing away linguistic variations before a document is indexed, recognizing structure in text such as noun phrases that should be indexed as a unit, and typing the structures found, identifying them, for example, as persons, places or things. These typed normalized features are then indexed. The index contains pointers to where the features ...

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, interactive tutorials, and more.

Start Free Trial

No credit card required