You are previewing The Nimble Elephant: Agile Delivery of Data Models using a Pattern-based Approach.
O'Reilly logo
The Nimble Elephant: Agile Delivery of Data Models using a Pattern-based Approach

Book Description

"Get it done well and get it done fast" are twin, apparently opposing, demands. Data architects are increasingly expected to deliver quality data models in challenging timeframes, and agile developers are increasingly expected to ensure that their solutions can be easily integrated with the data assets of the overall organization. If you need to deliver quality solutions despite exacting schedules, "The Nimble Elephant" will help by describing proven techniques that leverage the libraries of published data model patterns to rapidly assemble extensible and robust designs. The three sections in the book provide guidelines for applying the lessons to your own situation, so that you can apply the techniques and patterns immediately to your current assignments.

The first section, Foundations for Data Agility, addresses some perceived aspects of friction between "data" and "agile" practitioners. As a starting point for resolving the differences, pattern levels of granularity are classified, and their interdependencies exposed. A context of various types of models is established (e.g. conceptual / logical / physical, and industry / enterprise / project), and you will learn how to customize patterns within specific model types.

The second section, Steps Towards Data Agility, shares guidelines on generalizing and specializing, with cautions on the dangers of going too far. Creativity in using patterns beyond their intended purpose is encouraged. The short-term "You Ain't Gonna Need It" (YAGNI) philosophy of agile practitioners, and the longer-term strategic perspectives of architects, are compared and evaluated. Consideration is given to the potential of enterprise views contributing to project-specific models. Other topics include industry models, iterative modeling, creation of patterns when none exist, and patterns for rules-in-data. The section ends with a perspective on the modeler's possible role in agile projects, followed by a case study.

The final section, A Bridge to the Land of Object Orientation, provides a pathway for re-skilling traditional data modelers who want to expand their options by actively engaging with the ranks of object-oriented developers.

Table of Contents

  1. Copyright
  2. Praise for The Nimble Elephant
  3. List of Figures
  4. List of Tables
  5. Acknowledgments
  6. Foreword
  7. I. Foundations for Data Agility
    1. 1. Understanding the Agile versus Architecture Civil War
      1. The speed versus quality war isn't new
        1. THE NEW CIVIL WAR
        2. TRADITIONAL CONFLICTS
      2. Demands for agile data modeling
        1. ARGUMENTS ALONG A CONTINUUM
        2. THE NEED TO AVOID COMPROMISE
      3. Modeling ain't enough
        1. MULTIPLE SKILLS, MULTIPLE MODELS
        2. SOME CHOICES
        3. ITERATIVE MODELING
      4. Can patterns really help?
        1. THE PROMISE OF SPEED
        2. THE PROMISE OF QUALITY
        3. A TOUCH OF REALITY
      5. Notation: Yet another civil war?
      6. Consolidating the chapter for you
        1. SUMMARY:
        2. APPLYING THIS TO YOUR SITUATION:
    2. 2. Understanding Pattern Aggregations
      1. The "patterns" movement
      2. Levels of aggregation in data model patters
      3. Fine-grained "Elementary patterns"
      4. Medium-grained "Assembly patterns"
      5. Coarse-grained "Integration patterns"
      6. Consolidating the chapter for you
        1. SUMMARY:
        2. APPLYING THIS TO YOUR SITUATION:
    3. 3. Understanding Pattern Variations
      1. The detail behind the facade
      2. Variation via custom extension
      3. Variation via inheritance
      4. Variation when linking patterns
        1. PARTY RELATIONSHIPS – IMPLIED AGREEMENTS, IMPLIED ROLES
        2. PARTY RELATIONSHIPS – IMPLIED AGREEMENTS, EXPLICIT ROLES
        3. PARTY RELATIONSHIPS – EXPLICIT AGREEMENTS, EXPLICIT ROLES
        4. PARTY RELATIONSHIPS – CLOSING COMMENTS
      5. Variations in the real world
      6. Consolidating the chapter for you
        1. SUMMARY:
        2. APPLYING THIS TO YOUR SITUATION:
    4. 4. Understanding Model Types
      1. Different types of models
        1. CONCEPTUAL, LOGICAL, OR PHYSICAL?
          1. SCOPE
          2. LEVELS
          3. STYLE
      2. The way forward
        1. CLEM
      3. Consolidating the chapter for you
        1. SUMMARY:
        2. APPLYING THIS TO YOUR SITUATION:
  8. II. Steps Towards Data Agility
    1. 5. Test the Levels of Generalization
      1. A little bit more on generalization
      2. Whetting your appetite
      3. Too much of a good thing?
      4. Trying to get the balance right
      5. Consolidating the chapter for you
        1. SUMMARY:
        2. APPLYING THIS TO YOUR SITUATION:
    2. 6. Use Patterns Beyond Their Intended Purpose
      1. Thinking outside the box
      2. A multi-faceted challenge
      3. Some suggestions
        1. CHECK FOR OVERWEIGHT BAGGAGE
        2. TO A MAN WITH A HAMMER,…
      4. Consolidating the chapter for you
        1. SUMMARY:
        2. APPLYING THIS TO YOUR SITUATION:
    3. 7. Consider the Longer-term Picture
      1. The YAGNI view
        1. DIFFERING OPINIONS
        2. A LIGHT-HEARTED REQUEST FOR OVER-ENGINEERING
        3. ARCHITECTURE ACCEPTED, BUT PERCEIVED OVER-ENGINEERING BLAMED FOR PROBLEMS
        4. ARCHITECTURE REJECTED ON THE GROUNDS OF PERCEIVED OVER-ENGINEERING
      2. The enterprise view
        1. HOW BIG IS THE ENTERPRISE?
        2. CAN A PROJECT-BASED MODEL BENEFIT FROM THE ENTERPRISE MODEL?
      3. Making peace
        1. WHEN YAGNI IS CHALLENGED BY STANDARDS
        2. OVER-ENGINEERING, OR REALISTIC VISION?
      4. Consolidating the chapter for you
        1. SUMMARY:
        2. APPLYING THIS TO YOUR SITUATION:
    4. 8. Start with Enterprise Integration Patterns
      1. Setting the scene
      2. A pattern starts to appear
      3. A variation on the theme
      4. Complexity or simplicity?
      5. An integration pattern emerges
      6. Where to next?
      7. Consolidating the chapter for you
        1. SUMMARY:
        2. APPLYING THIS TO YOUR SITUATION:
    5. 9. Benchmark Against Wider Standards
      1. Standards for integration patterns
      2. Standards for assembly patterns
      3. Join the community
      4. Consolidating the chapter for you
        1. SUMMARY:
        2. APPLYING THIS TO YOUR SITUATION:
    6. 10. Challenge the Concepts of the Enterprise
      1. Physical models are not the only reality
      2. What details might I find in a conceptual model?
      3. Can conceptual models make a difference?
      4. Linking high level models and implementation models
      5. Name the patterns
      6. It's not arrogance to question
      7. Consolidating the chapter for you
        1. SUMMARY:
        2. APPLYING THIS TO YOUR SITUATION:
    7. 11. Iterate, Iterate, Iterate
      1. Iterate along the model's breadth and depth
        1. IDENTIFYING THE BIG PICTURE
        2. STARTING WELL, ENDING POORLY
        3. A DIFFERENT APPROACH – MILE-WIDE, INCH-DEEP
        4. T-MODEL FOR DEPTH
        5. DOESN'T HAVE TO BE FULL-WIDTH, NOR FULL DEPTH
      2. Iterate on concepts via the implementation feedback loop
      3. Iterate to improve
      4. Consolidating the chapter for you
        1. SUMMARY:
        2. APPLYING THIS TO YOUR SITUATION:
    8. 12. When Patterns Don't Exist
      1. It's not an 80/20 silver bullet
      2. Letting "agile" teach us a trick or two
        1. THE AGILE MANIFESTO AS A FRAMEWORK FOR DATA ARCHITECTURE ITERATION
        2. A CASE STUDY IN USING THE AGILE PRINCIPLES FOR DATA ARCHITECTURE
      3. Consolidating the chapter for you
        1. SUMMARY:
        2. APPLYING THIS TO YOUR SITUATION:
    9. 13. Rules in Data with Decision Tables
      1. An honorable way to make life harder for yourself
      2. Fixed-structure decision tables
      3. Adaptive decision tables
      4. Consolidating the chapter for you
        1. SUMMARY:
        2. APPLYING THIS TO YOUR SITUATION:
    10. 14. Going Further with Rules in Data!
      1. Derivation of computed values
      2. Hybrid solutions
        1. COMBINING DECISION TABLES AND CONSTRAINT LOGIC PROGRAMMING
        2. COMBINING DECISION TABLES AND WORKFLOW
      3. A case study in business rule agility
      4. A few comments
      5. Consolidating the chapter for you
        1. SUMMARY:
        2. APPLYING THIS TO YOUR SITUATION:
    11. 15. Save Your Career, Save Our Profession, Save the World?
      1. We're needed (aren't we?)
      2. Contributing to the agile world
        1. THE BIG PICTURE (FOR TODAY)
        2. THE LONG-TERM VIEW (FOR TOMORROW)
        3. BETTER ABSTRACTIONS AND/OR GENERALIZATIONS
        4. GOOD CORPORATE CITIZENSHIP
        5. AVOIDANCE OF THE DEATH-TRAPS
        6. CONSOLIDATION OF FEEDBACK
        7. GOVERNANCE?
        8. BOILERPLATE ARCHITECTURES
      3. Style and timing of engagement
        1. ARCHITECTURE ROLES BEFORE THE PROJECT STARTS
        2. ARCHITECTURE ENGAGEMENT DURING THE PROJECT
      4. If we want to feel loved…
        1. THEY DON'T WANT OLD-SCHOOL ARCHITECTS
        2. DO WE NEED DETAILED TECHNICAL SKILLS?
        3. SO WHAT'S REQUIRED OF US TO CHANGE?
      5. Consolidating the chapter for you
        1. SUMMARY:
        2. APPLYING THIS TO YOUR SITUATION:
    12. 16. Case Study
      1. Getting ready to start
      2. Scoping the project, or "How big is the enterprise?"
      3. Starting a context model with the first assembly pattern
      4. Some choices appear
      5. Another assembly pattern
      6. A wrong assumption by me: Rules instead of Process
      7. An agile iteration begins
      8. Integration Patterns
      9. Back to reality?
      10. Consolidating the chapter for you
        1. SUMMARY:
        2. ***PLEASE READ– APPLYING THIS TO YOUR SITUATION:
  9. III. A Bridge to the Land of Object Orientation
    1. 17. Object-orientation – A Data Modeler's Primer
      1. Encapsulation, or "They've hidden my data"
      2. Object IDs, or "Who stole my keys?"
      3. Attributes and Data types on steroids
        1. MULTIPLICITY
        2. DEFINING NEW DATA TYPES
        3. DATA TYPES WITH OPERATIONS!
      4. A bit more on associations
        1. THINGS THAT CONTAIN THINGS
        2. MANY-TO-MANY ASSOCIATIONS
      5. Inheritance of data and code
        1. INHERITANCE OF DATA (ATTRIBUTES)
        2. INHERITANCE OF CODE (OPERATIONS)
          1. Multiple inheritance
      6. Persistence, or "What do you mean the data isn't stored?"
      7. Summary of OO concepts and terms
  10. References
  11. Glossary of Terms and Abbreviations