Natural Language Processing with Java

Chapter 3. Finding Sentences

Partitioning text into sentences is also called Sentence Boundary Disambiguation (SBD). This process is useful for many downstream NLP tasks that require analysis within sentences; for example, POS and phrase analysis typically work within a sentence.

In this chapter, we will explain in further detail why SBD is difficult. Then we will examine some core Java approaches that may work in some situations, and move on to the use of models by various NLP APIs. We will also examine training and validating approaches for sentence detection models. We can add additional rules to refine the process further, but this will work only up to a certain point. After that, models must be trained to handle both common and specialized ...