O'Reilly logo

Data Mining Techniques in Grid Computing Environments by Werner Dubitzky

Stay ahead with the world's most comprehensive technology and business learning platform.

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, tutorials, and more.

Start Free Trial

No credit card required

8

Building and using analyticalworkflows in Discovery Net

Moustafa Ghanem, Vasa Curcin, Patrick Wendel and Yike Guo

ABSTRACT

The Discovery Net platform is built around a workflow model for integrating distributed data sources and analytical tools. The platform was originally designed to support the design and execution of distributed data mining tasks within a grid-based environment. However, over the years it has evolved into a generic data analysis platform with applications in such diverse areas as bioinformatics, cheminformatics, text mining and business intelligence. In this work we present our experiences in designing the platform. We also map the evolution of its workflow server architecture to meet the demands of these different applications.

8.1 Introduction

The recent interest of the scientific and business communities in data mining has been primarily driven by our increasing ability to generate, capture and share more data than could be analysed using traditional methods.

Motivating examples abound in the domain of bioinformatics where routine scientific experiments can generate thousands of gene measurements from a single biological sample. The samples themselves are typically collected from large numbers of subjects under different conditions. With a large number of data points generated, the analysis can proceed only with the use of high-performance computing resources. Furthermore, the interpretation of the experimental data and analysis results typically requires ...

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, interactive tutorials, and more.

Start Free Trial

No credit card required