Practical Text Mining and Statistical Analysis for Non-structured Text Data Applications

Tutorial T

Potential Problems That Can Arise in Text Mining

Example Using NALL Aviation Data

Jennifer Hunter Thompson

Contents

Introduction

Spelling Errors

Example: Finding Spelling Errors in Text Miner

Combine Words

Misspellings as Synonyms

Unexpected Terms

Example: Finding Unexpected Terms

Different File Types

Summary

Introduction

When working with unstructured text, a variety of issues can arise. Some of these issues are more easily fixed than others. Awareness is always the first step to a resolution. This tutorial will explore some potential issues you may run into while exploring unstructured text. Suggestions for solutions will also be explored.

Spelling Errors

Some sources of text may present an issue of spelling errors in the text. ...