Programming F# 3.0, 2nd Edition

Chapter 11. Data Processing

You have been exposed to just about every core feature of the F# programming language and seen how to program in the functional, imperative, and object-oriented styles. However, in order to develop a deep understanding of a language, you need to actually go out and use it. If you don’t have an F# starter project in mind, well, this chapter should hopefully serve as the next-best thing.

This chapter focuses on the application of the F# programming language. However, there is not a canonical F# application. The language is well suited across multiple domains and industries. Yet in all cases, F# is best suited for the computational core of a program, or the numeric heavy lifting and data processing.

With that in mind, we will write a simple search engine using F#. Along the way, you will learn about MapReduce-style data processing and using Lex and Yacc for generating parsers. Although this F# search engine will not rival Google any time soon, it will serve as a large-enough code base to practice idiomatic F#.

Of course, a detailed look at the source code for a search engine and the technologies powering it could easily fill several books, so this chapter only covers the key concepts. For more detail, see the source code found on GitHub at: https://github.com/aChrisSmith/Programming-FS-Examples

Before we begin, let’s quickly cover the major components of the project, which, despite each one leading down a technically interesting rabbit hole, are not that difficult ...