O'Reilly logo

Applied Architecture Patterns on the Microsoft Platform by Stephen W. Thomas, Rama Ramani, Richard Seroter, Mike Sexton, Ewan Fairweather

Stay ahead with the world's most comprehensive technology and business learning platform.

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, tutorials, and more.

Start Free Trial

No credit card required

Chapter 12. Debatching Bulk Data

Debatching data is the process of turning one huge pile of data into many small piles of data.

Why is it better to shovel one ton of data using two thousand, one pound shovels instead of one big load from a huge power shovel? After all, large commercial databases and the attendant bulk loader or SQL Loader programs are designed to do just that: insert huge loads of data in a single shot.

The bulk load approach works under certain tightly constrained circumstances. They are as follows:

  1. The "bulk" data comes to you already matching the table structure of the destination system. Of course, this may mean that it was debatched before it gets to your system.
  2. The destination system can accept some, potentially significant, ...

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, interactive tutorials, and more.

Start Free Trial

No credit card required