Preparing data
An important step in NLP is finding and preparing data for processing. This includes data for training purposes and the data that needs to be processed. There are several factors that need to be considered. Here, we will focus on the support Java provides for working with characters.
We need to consider how characters are represented. Although we will deal primarily with English text, other languages present unique problems. Not only are there differences in how a character can be encoded, the order in which text is read will vary. For example, Japanese orders its text in columns going from right to left.
There are also a number of possible encodings. These include ASCII, Latin, and Unicode to mention a few. A more complete list is ...
Get Natural Language Processing with Java now with the O’Reilly learning platform.
O’Reilly members experience books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.
