Functional Programming in C#: Classic Programming Techniques for Modern Projects

ABSTRACTING THE PROBLEM

Having a MapReduce engine available doesn’t solve any problems automatically. An important step is to abstract any given problem in such a way that it fits the pattern. Another consideration is of making the executable code for a given problem available to the engine together with the data — but again, this is beyond the scope of this book, and the implementation details of the engine you use obviously influence this greatly.

Most programmers remember when they were new to the art of computer programming, and abstraction was, at that time, something that came hard to many. You have some information, you want something out of it, but how do you get there? What steps are needed to reach the goal? You learned how to think within the system, whichever system you worked with. An object oriented programmer learns to model things in classes, shaping elements and things that interact with one another. A functional programmer evaluates the order in which certain results, and thereby calculations, depend upon one another.

With MapReduce, the process is the same, but the system works differently. It is probably closer to functional programming, but the firm requirement to implement everything in only two formalized steps still seems restrictive. Experts claim that the family of problems that can be solved with the help of MapReduce is extremely large, and there’s the benefit of hiding away all the details of parallelization — every implementation of a problem solution ...