You are previewing Getting Started with Oracle Data Integrator 11g: A Hands-On Tutorial.
O'Reilly logo
Getting Started with Oracle Data Integrator 11g: A Hands-On Tutorial

Book Description

Combine high volume data movement, complex transformations and real-time data integration with the robust capabilities of ODI in this practical guide

  • Discover the comprehensive and sophisticated orchestration of data integration tasks made possible with ODI, including monitoring and error-management

  • Get to grips with the product architecture and building data integration processes with technologies including Oracle, Microsoft SQL Server and XML files

  • A comprehensive tutorial packed with tips, images and best practices

In Detail

Oracle Data Integrator (ODI) is Oracle's strategic data integration platform for high-speed data transformation and movement between different systems. From high-volume, SOA-enabled data services, to trickle operations – ODI is a cutting-edge platform that offers heterogeneous connectivity, enterprise-level deployment, and strong administrative, diagnostic, and management capabilities.

"Getting Started with Oracle Data Integrator 11g: A Hands-On Tutorial" provides you with everything you to get up and running with Oracle Data Integrator, and more! Following an example scenario, the book covers essential information about the ODI architecture and using ODI across different databases (Oracle, Microsoft SQL Server and MySQL), and file types such as XML, before covering Orchestrating Data Integration Workflows, Error Management, Operational Management and Monitoring, and beyond.

"Getting Started with Oracle Data Integrator 11g: A Hands-On Tutorial" begins by describing the Oracle Data Integrator architecture and teaching you to install the product following best practices. You’ll then be introduced to some of the key concepts of ODI such as the Knowledge Modules

Later topics include moving and transforming data from sources to targets including the Oracle Database, Microsoft SQL Server, MySQL, Flat files, and XML files, each with illustrated hands-on examples for the different technologies. Your learning experience will be made all the more rich with chapters introducing, explaining and leveraging additional ODI functionality such as variables, reusable procedures, temporary indexes and more.

Finally ODI's workflow and task orchestration capabilities are explained before introducing you to Error Management with ODI's built-in 'error hospital' and 'error recycling' capabilities for non-compliant data, not to mention tackling ODI Studio, ODI Console and Enterprise Manager Fusion Middleware Control.

Table of Contents

  1. Getting Started with Oracle Data Integrator 11g: A Hands-On Tutorial
    1. Table of Contents
    2. Getting Started with Oracle Data Integrator 11g: A Hands-On Tutorial
    3. Credits
    4. Foreword
    5. About the Authors
    6. About the Reviewers
    7. www.PacktPub.com
      1. Support files, eBooks, discount offers and more
        1. Why Subscribe?
        2. Free Access for Packt account holders
        3. Instant Updates on New Packt Books
    8. Preface
      1. Oracle Data Integrator—background and history
      2. Data integration usage scenarios
        1. Data warehouses and BI
        2. Service-oriented architecture (SOA)
        3. Applications
        4. Master Data Management
        5. Big Data
      3. What this book covers
      4. What you need for this book
      5. Who this book is for
      6. Who this book is not for
      7. Conventions
      8. Reader feedback
      9. Customer support
        1. Errata
        2. Piracy
        3. Questions
    9. 1. Product Overview
      1. ODI product architecture
        1. ODI repository
          1. Repository overview
          2. Repository location
          3. Master repository
          4. Work repository
          5. Execution repository
          6. Lifecycle management and repositories
        2. Studio
        3. Agent
        4. Console
        5. Oracle Enterprise Manager
      2. ODI key concepts
        1. Execution Contexts
        2. Knowledge Modules
        3. Models
        4. Interfaces
          1. Interface descriptions
          2. Interface mappings
          3. Interface flow tab
          4. Interface controls
        5. Packages and Scenarios
      3. Summary
    10. 2. Product Installation
      1. Prerequisites
        1. Prerequisites for the repository
        2. Prerequisites for the Oracle Universal Installer
        3. Prerequisites for the Studio
        4. Prerequisites for the Standalone Agent
      2. Installing ODI 11g
        1. Two installation modes
        2. Creating the repository with RCU
        3. Installing the ODI Studio and the ODI Agent
        4. Starting the ODI Studio for the first time
      3. Post installation—parameter files review
      4. Summary
    11. 3. Using Variables
      1. Defining variables
        1. Variable location and scope
        2. Variable definitions
        3. Refreshing variables
        4. Variable history
      2. Using variables for dynamic information
        1. Assigning a value to a variable
          1. Setting a hardcoded value
          2. Refresh Variable
          3. Passed as a parameter (Declare Variable)
        2. Referencing variables
        3. Variables in interfaces
        4. Variables in models
        5. Variables in topology
      3. Using variables to alter workflows
        1. Packages
        2. Load Plans
      4. Summary
    12. 4. ODI Sources, Targets, and Knowledge Modules
      1. Defining Physical Schemas, Logical Schemas, and Contexts
        1. Defining physical data servers
        2. Defining Physical Schemas
          1. Data schemas and work schemas
        3. Defining Logical Schemas and Contexts
        4. Non-database technologies
            1. Flat Files
            2. XML files
      2. Reverse-engineering metadata into ODI models
        1. Standard reverse-engineering
        2. Custom reverse-engineering
        3. File reverse-engineering
        4. XML reverse-engineering
      3. Examining the anatomy of the interface flow
        1. Example 1: Database and file to database
        2. Example 2: File and database to second file
        3. Example 3: File to Enterprise Application
      4. Importing and choosing Knowledge Modules
        1. Choosing Knowledge Modules
        2. Importing a Knowledge Module
        3. KMs—A quick look under the hood
        4. Configuring behavior with KM options
      5. Examining ODI Interfaces
        1. Overview tab
        2. Mapping tab
        3. Flow tab
        4. Quick-Edit tab
      6. Summary
    13. 5. Working with Databases
      1. Sample scenario description
        1. Integration target
        2. Integration source
        3. Integration mappings
        4. Data flow logistics
      2. Exercise 1: Building the Load_Customer interface
        1. Building the topology
            1. Setting up the topology
        2. Reverse-engineering the model metadata
        3. Moving the data using an ODI interface
        4. Checking the execution with the Operator Navigator
      3. Summary
    14. 6. Working with MySQL
      1. What you can and can't do with MySQL
      2. Working with MySQL
          1. Obtaining and installing the software
        1. Overview of the task
        2. Integrating the product data
          1. Product data target, sources, and mappings
          2. Product interface flow logistics
        3. Integrating inventory data
          1. Inventory target, sources, and mappings
          2. Inventory interface flow logistics
      3. Using MySql with ODI
        1. Adding the MySQL JDBC driver
        2. Expanding the topology
        3. Reverse-engineering revisited
        4. Preparing to move the product data
        5. Using simulation and execution
        6. Moving the inventory data
      4. Summary
    15. 7. Working with Microsoft SQL Server
      1. Example: Working with SQL Server
        1. Overview of the task
        2. Integrating the Sales data
          1. Source
          2. Target
          3. Integrations
      2. Sample scenario
        1. Expanding the ODI topology
        2. Setting up the topology
        3. Reverse-engineering the Model metadata
        4. Creating interfaces and mappings
        5. Load Sales Person interface
        6. Load Sales Person mapping
        7. Automatic Temporary Index Management
        8. Load Sales Region interface
        9. Checking the execution with the Operator Navigator
          1. Execute the Load Sales Person interface
          2. Verify and examine the Load Sales Person results
          3. Verify and examine Load Sales Region results
      3. Summary
    16. 8. Integrating File Data
      1. Working with flat files
        1. Scope
        2. Prerequisites for flat files
        3. Integrate the file data into an Oracle table
          1. Partner data target, source, and mappings
          2. Partner interface flow logistics
      2. Step-by-step example
        1. Expanding the topology for file handling
        2. Integrating the Partner data
          1. Creating and preparing the project
          2. Creating the interface to integrate the Partner data
          3. Running the interface
      3. Summary
    17. 9. Working with XML Files
      1. Introduction to XML
      2. Introducing the ODI JDBC driver for XML
        1. ODI and its XML driver—basic concepts
      3. Example: Working with XML files
        1. Requirements and background
        2. Scope
        3. Overview of the task
          1. Integrating a Purchase Order from an XML file
          2. Creating models from XML files
          3. Integrating the data from a single Purchase Order
          4. Single order interface flow logistics
        4. Sample scenario: Integrating a simple Purchase Order file
          1. Expanding the Topology
          2. Reverse-engineering the metadata
          3. Creating the Interface
          4. Creating procedures
      4. Summary
    18. 10. Creating Workflows—Packages and Load Plans
      1. Packages
        1. Creating a package
        2. Adding steps into a package
        3. Adding tools in a package
          1. Changed Data Capture
          2. Event Detection
          3. Files
          4. Internet
          5. Metadata
          6. ODI Objects
          7. Plugins
          8. SAP
          9. Utilities
          10. Adding tools to a package
          11. Using ODI Tools
        4. Retry versus fail
        5. Best practice: No infinite loop
        6. Generating a scenario from a package
      2. Load Plans
        1. Serial and parallel steps
        2. Objects that can be used in a Load Plan
        3. Exception handling
        4. Using Packages and Load Plans
      3. Summary
    19. 11. Error Management
      1. Managing data errors
        1. Detecting and diverting data errors
          1. Data quality with ODI constraints
          2. ODI error table prefix
          3. Contents of an error table
          4. Using flow control and static control
          5. Using error thresholds
        2. Correcting and recycling data errors
          1. Recycling errors and ODI update keys
      2. Managing execution errors
        1. Handling anticipated errors
          1. Causing a deliberate benign error with OdiBeep
        2. Handling unexpected design-time errors
          1. More detailed error investigation in Operator Navigator
        3. Handling unexpected runtime errors
      3. Handling operational errors
      4. Summary
    20. 12. Managing and Monitoring ODI Components
      1. Scheduling with Oracle Data Integrator
        1. Overview
          1. Illustrating the schedule management user interface
            1. Creating a scheduled execution that will execute exactly once
          2. Using third-party schedulers
      2. Fusion Middleware Console Control
        1. Launching and accessing the FMCC
        2. Domain
        3. Agent
          1. Starting and stopping
          2. Performance summary
        4. Log file visibility and aggregation
          1. Visibility
          2. Aggregation
        5. Repository visibility
          1. Session statistics
      3. Oracle Data Integrator Console
        1. Launching and accessing ODI Console
        2. Data Lineage
        3. Flow Map
      4. Summary
    21. 13. Concluding Remarks
    22. Index