Cover image for Data Warehousing with SAP BW7 BI in SAP Netweaver 2004s

Book description

BI in SAP NetWeaver 2004s is the official abbreviation for the successor of the Business Information Warehouse (BW) which has been completely revised by SAP with its latest release. Core elements of this comprehensive suite for decision making applications are functions for extraction, transformation and data management. With this new release, these functions aim more heavily at company-wide data warehousing.

The book focuses on these core tasks of SAP BW and gives well-founded insights into the system architecture. As practical handbook and well-structured reference book, the book is for SAP consultants and IT staff that are responsible for or planning a BW-based data warehouse implementation.

Apart from system architecture, the book focuses on detailed descriptions of data management (data models and Analytical Engine) as well as the Staging Engine which have been completely revised and deal with new data transfer process technology. The design of the controlled operations has been substantially expanded and besides a comprehensive description of automization techniques by using process chains, regular maintenance and administration tasks are also discussed (model trimming, technical validation).

The book emphasizes a comprehensive view on aspects to manageability and system performance which are discussed in individual chapters but also implicitly in all other ranges of topics.

Table of Contents

  1. Cover Page
  2. Title Page
  3. Copyright Page
  4. Overview
  5. Table of Content
  6. 1. Introduction
    1. Operational Administrative Systems
    2. Decision Support Systems
    3. Data Warehouse Systems
    4. Content
  7. I. Architecture
    1. 2. Architecture Components
      1. 2.1 Extraction Layer
      2. 2.2 Data Warehouse
      3. 2.3 Decision Support Systems
    2. 3. BW Basis System
      1. 3.1 Communication Interfaces
        1. 3.1.1 File and BAPI
          1. Transfer Tools
        2. 3.1.2 Internet Communication Framework
        3. 3.1.3 J2EE Connector Architecture
    3. 4. Metadata Repository
      1. 4.1 Database Objects
      2. 4.2 ABAP Dictionary Objects
        1. 4.2.1 Domains
        2. 4.2.2 Data Elements
        3. 4.2.3 Tables
        4. 4.2.4 Development Namespaces
      3. 4.3 BW Objects
        1. 4.3.1 BW Namespaces
  8. II. Data Model
    1. 5. General Data Models
      1. 5.1 Basics of Data Modeling
        1. 5.1.1 Characteristics
        2. 5.1.2 Attributes
        3. 5.1.3 Key Figures
        4. 5.1.4 Status Tracking
      2. 5.2 Transactional Structures
      3. 5.3 Flat Structures
      4. 5.4 Star Schema
      5. 5.5 Snowflake Schema
      6. 5.6 Summary
    2. 6. BW Data Model
      1. 6.1 InfoObjects
        1. 6.1.1 Characteristics
          1. Permission of Lowercase Letters
          2. Conversion Routine and Output Length
          3. SID Tables
        2. 6.1.2 Time Characteristics
        3. 6.1.3 Key Figures
          1. Aggregation Behavior of Key Figures
        4. 6.1.4 Units
      2. 6.2 Master Data
        1. 6.2.1 Texts
        2. 6.2.2 Master Data of a Characteristic
        3. 6.2.3 Explicit Display Attributes
        4. 6.2.4 Referencing Characteristics
        5. 6.2.5 External Hierarchies
        6. 6.2.6 Summary
      3. 6.3 DataStore Objects
        1. 6.3.1 Key Fields
        2. 6.3.2 Data Fields
      4. 6.4 BasisCubes
        1. 6.4.1 Fact Table
        2. 6.4.2 SID Entries in Dimension Tables
        3. 6.4.3 Dimensions
        4. 6.4.4 Line Item Dimensions
        5. 6.4.5 Real-Time InfoCubes
      5. 6.5 Source System Dependent Data
    3. 7. Performance Tuning
      1. 7.1 Aggregates
        1. 7.1.1 Initial Filling of Aggregates
        2. 7.1.2 Hierarchy and Attribute Changes
      2. 7.2 Indexing
        1. 7.2.1 Index Use with BasisCubes
        2. 7.2.2 Use of indexing on DataStore Objects
        3. 7.2.3 Index Use on InfoObjects
        4. 7.2.4 Optimizer Statistics
      3. 7.3 Partitioning and Clustering
        1. 7.3.1 Range Partitioning
          1. Partitioning of the Uncompressed Fact Table of BasisCubes and Aggregates
          2. Partitioning of the Compressed Fact Table of BasisCubes
          3. Partitioning of the Compressed Fact Table of Aggregates
        2. 7.3.2 Clustering
          1. Clustering of the Compressed Fact Table
          2. Clustering of an Uncompressed Fact Table
        3. 7.3.3 Model Partitioning
      4. 7.4 Dedicated OLAP Storage (BIA)
        1. 7.4.1 BIA Connection to SAP BW
        2. 7.4.2 Managing the BIA in BW
        3. 7.4.3 Structure of Cube Content
          1. Fact Tables in the BI Accelerator
          2. Dimension Tables
          3. Master Data
        4. 7.4.4 Organization of Cube Content
        5. 7.4.5 Creation and Filling of BIA Indexes
        6. 7.4.6 Hierarchy and Attribute Changes
    4. 8. Basic Rules of Data Modeling
      1. 8.1 Modeling BasisCubes
      2. 8.2 Modeling InfoObjects
      3. 8.3 Modeling Hierarchies
      4. 8.4 Modeling Partner Roles
      5. 8.5 Modeling Key Figures
        1. 8.5.1 Key Figures of Characteristics Type
        2. 8.5.2 Grouped Key Figures
        3. 8.5.3 Calculated Key Figures
    5. 9. Data Modeling in the DWWB
      1. 9.1 Modeling of InfoObjects
      2. 9.2 Modeling of BasisCubes
        1. 9.2.1 Remodeling of Dimension Tables
          1. Entering new Characteristics
          2. Deletion of Characteristics
          3. Replacement of Characteristics
        2. 9.2.2 Remodeling of the Fact Table
          1. Entry of New Key Figures
          2. Deletion of Key Figures
          3. Replacement of Key Figures
        3. 9.2.3 Subsequent Zero Value Elimination
      3. 9.3 Modeling DataStore Objects
      4. 9.4 Multidimensional Clustering
  9. III. Analytical Engine
    1. 10. Access to Physical InfoProviders
      1. 10.1 Access to BasisCubes
        1. 10.1.1 Status of the Staging Engine
          1. Definition of the Request Status to Be Used
        2. 10.1.2 Partial Queries
      2. 10.2 Access to DataStore Objects
      3. 10.3 Access to InfoObjects
    2. 11. Virtual InfoProvider
      1. 11.1 MultiProvider
        1. 11.1.1 Consolidating Data
        2. 11.1.2 Creation of Subqueries
      2. 11.2 InfoSets
        1. 11.2.1 InfoObjects in InfoSets
          1. Most Recent Reporting of Master Data
          2. Temporary Join
        2. 11.2.2 DataStore Objects in InfoSets
        3. 11.2.3 BasisCubes in InfoSets
          1. Use of Aggregates
          2. Status of the Staging Engine
      3. 11.3 Service InfoCubes
        1. 11.3.1 Transfer of Selection Conditions
        2. 11.3.2 Import/Export Parameter
    3. 12. OLAP Caching and Access Sequences
      1. 12.1 Local Cache
        1. 12.1.1 Read All Data at Once
        2. 12.1.2 Read Data during Navigation
        3. 12.1.3 Read When You Navigate or Expand Hierarchies
      2. 12.2 Global Cache
        1. 12.2.1 Main Memory Cache
        2. 12.2.2 Persistent Cache
        3. 12.2.3 Cache Invalidation
      3. 12.3 BIA Indexes
      4. 12.4 Aggregates
    4. 13. Monitoring the Analytical Engine
      1. 13.1 Query Monitor
      2. 13.2 Runtime Statistics of the Analytical Engine
  10. IV. Extraction & Staging
    1. 14. Extraction Layer
      1. 14.1 Metadata of the Extraction Layer
        1. 14.1.1 Data Sources and Data Structure
        2. 14.1.2 Application Component Hierarchy
        3. 14.1.3 Supported Delta Procedures
          1. Delta Mode of the Data Records
          2. Delta Procedures of DataSources
      2. 14.2 Extraction from SAP ERP
        1. 14.2.1 Defining Generic DataSources
          1. Extraction from Tables and Views
          2. Extraction from Function Modules
          3. Generic Delta
        2. 14.2.2 Enhancing DataSources
      3. 14.3 Extracting from BW Systems
      4. 14.4 Extracting from Database Systems
      5. 14.5 Extracting from Files
        1. 14.5.1 File Format
        2. 14.5.2 Data Structure
          1. Extracting Transactional Data
          2. Extracting Attributes
          3. Extracting Texts
          4. External Hierarchies
      6. 14.6 Extracting via Web Services
      7. 14.7 Extracting from JDBC, XML/A, and ODBO Sources
        1. 14.7.1 Installing a BI JDBC Connector
        2. 14.7.2 Installing a BI ODBO Connector
        3. 14.7.3 Installing a BI XML/A Connector
        4. 14.7.4 Installing a BI SAP Query Connector
      8. 14.8 Extraction with Third-Party ETL Tools
    2. 15. Inflow Layer
      1. 15.1 Metadata of Source Systems
        1. 15.1.1 Creating SAP ERP Source System Connections
        2. 15.1.2 Creating a BW System Connection
        3. 15.1.3 Creating a DB Connect Connection
        4. 15.1.4 Creating Flat File Connections
        5. 15.1.5 Creating Web Service System Connections
        6. 15.1.6 Creating a Universal Data Connect Connection
        7. 15.1.7 Creating Third-Party ETL Tool Connections
        8. 15.1.8 Source System IDs
          1. Source System IDs in Data Mart Architectures
      2. 15.2 Metadata of DataSources
        1. 15.2.1 Extraction Source
          1. Data Sources for SAP Source Systems
          2. Data Sources for File Systems
          3. Data Sources for DB Connect Source Systems
          4. Data Sources for Web Services
          5. Data Sources for UD connect
        2. 15.2.2 Data Structure
          1. Application Structure
        3. 15.2.3 Input Conversion
        4. 15.2.4 Selection Fields
        5. 15.2.5 Delta Procedure
        6. 15.2.6 Inventory Key Figures
      3. 15.3 Persistent Staging Area (PSA)
      4. 15.4 Definition of Extraction Processes
        1. 15.4.1 Data Selection
          1. Selecting the Date
          2. Free Time Delimitation
          3. Selection through ABAP Routine
          4. Selection through OLAP Variable
        2. 15.4.2 Extraction
        3. 15.4.3 Processing
        4. 15.4.4 Update
        5. 15.4.5 Scheduling
    3. 16. Transformation Layer
      1. 16.1 InfoSources
      2. 16.2 Persisting of Raw Data
    4. 17. Integration Layer
      1. 17.1 Transactional Data in the Integration Layer
        1. 17.1.1 Persistence
        2. 17.1.2 Delta Generation
        3. 17.1.3 Data Integration
      2. 17.2 Master Data in the Integration Layer
    5. 18. Data Mart Layer
      1. 18.1 Transactional Data in the Data Mart Layer
        1. 18.1.1 Schema Transformation
        2. 18.1.2 Aggregation
        3. 18.1.3 Filtering
        4. 18.1.4 Process Integration
        5. 18.1.5 Abstraction
        6. 18.1.6 Model Transformation
        7. 18.1.7 Currency Conversion
      2. 18.2 Master Data in the Data Mart Layer
      3. 18.3 Open Hub Destinations in the Data Mart Layer
    6. 19. Definition of Transformations
      1. 19.1 Global Data Declarations
      2. 19.2 Start Routine
      3. 19.3 Rules and Rule Groups
        1. 19.3.1 Assignment of Constants
        2. 19.3.2 Direct Assignment
        3. 19.3.3 Master Data Attribute of an InfoObject
        4. 19.3.4 Routines
        5. 19.3.5 Formulas
        6. 19.3.6 Conversion Exit
        7. 19.3.7 Handling of Units
      4. 19.4 Aggregation Types
        1. 19.4.1 Aggregation Types for BasisCubes
        2. 19.4.2 Aggregation Types for DataStore Objects
        3. 19.4.3 Aggregation Types for InfoObjects
        4. 19.4.4 Aggregation Types for InfoSources
        5. 19.4.5 Aggregation Types for Open Hub Destinations
      5. 19.5 End Routine
      6. 19.6 Expert Routine
    7. 20. Definition of Load Processes
      1. 20.1 Extraction Mode
      2. 20.2 Filter
        1. 20.2.1 OLAP Variable
        2. 20.2.2 Routine
      3. 20.3 Processing Mode
    8. 21. Direct Staging
      1. 21.1 Definition of Metadata
        1. 21.1.1 Creation of a 3.x DataSource
          1. DataSources for SAP and Third-Party Source Systems
          2. DataSources for DB Connect Source Systems
          3. DataSources for Flat File Source Systems
          4. DataSources for Universal Data Connect
        2. 21.1.2 Selection Fields
        3. 21.1.3 Hierarchy Properties
      2. 21.2 Definition of Transfer Rules
        1. 21.2.1 Start Routine
        2. 21.2.2 Direct Assignment
        3. 21.2.3 Assignment of Constants
        4. 21.2.4 ABAP Routines
          1. Local Transfer Routine
          2. Global Transfer Routine
        5. 21.2.5 Formulas
        6. 21.2.6 Input Conversion
        7. 21.2.7 Source-System-Dependent Data
      3. 21.3 Definition of Extraction Procedures
        1. 21.3.1 External Data
          1. Control File
        2. 21.3.2 Hierarchy Selection
          1. Selection of the Hierarchy to Be Loaded
          2. Update Method of the Hierarchy
          3. Storing the Hierarchy
          4. Activation of the Hierarchy
        3. 21.3.3 Processing
    9. 22. Real-Time Staging
      1. 22.1 Real-Time Data Acquisition
        1. 22.1.1 RDA in the Extraction Layer
        2. 22.1.2 RDA in BW Staging
          1. DataSource for RDA
          2. InfoPackages for RDA
          3. Data Transfer Process for RDA
        3. 22.1.3 Controlling the Real-Time Data Acquisition
      2. 22.2 Direct Access
        1. 22.2.1 Virtual Provider with Staging Connect
        2. 22.2.2 Virtual Provider with BAPI
    10. 23. Data Quality
      1. 23.1 Evaluation of the Extraction
      2. 23.2 Consistency Check
      3. 23.3 Master Data Integrity
      4. 23.4 Referential Integrity
      5. 23.5 Error Handling
        1. 23.5.1 System Behavior without Error Handling
        2. 23.5.2 Continuation of the Check
        3. 23.5.3 Continuation of the Posting
    11. 24. Performance Tuning
      1. 24.1 Parallelization of Extraction and Staging
        1. 24.1.1 Parallelization in the Extractor
        2. 24.1.2 Parallelization in the Transformation
        3. 24.1.3 Parallelization through Direct Staging
        4. 24.1.4 Packaging
        5. 24.1.5 Preconditions for the Parallelization
        6. 24.1.6 Parallelization in RDA Scenarios
      2. 24.2 Inbound Processing in the PSA
      3. 24.3 Index Management
      4. 24.4 Compression of BasisCubes
      5. 24.5 Activation of New Data in DataStore Objects
        1. 24.5.1 Simplification of Delta Determination
        2. 24.5.2 Limitation of the Main Memory Requirement
        3. 24.5.3 Avoiding SID Determination
        4. 24.5.4 Abandoning Optimizer Statistics
        5. 24.5.5 Clustering of DataStore Objects
      6. 24.6 Management of Aggregates
        1. 24.6.1 Rollup Hierarchy
        2. 24.6.2 Delta Procedure of the Change Run
        3. 24.6.3 Block Size for the Rebuild
        4. 24.6.4 Pre-analysis of the Fill Process
      7. 24.7 Rollup to BIA Indexes
  11. V. BW Design
    1. 25. Partitioned InfoProviders
    2. 26. Partitioned Staging
    3. 27. Large-Scale Architecture
      1. 27.1 Replicating Architecture
      2. 27.2 Aggregating Architecture
      3. 27.3 Virtual Hub-and-Spoke Architecture
  12. VI. BW in Live Operation
    1. 28. Process Chains
      1. 28.1 Event Controlling of the BW Basis System
      2. 28.2 Controlling Concept of Process Chains
        1. 28.2.1 Concatenation of Process Steps
        2. 28.2.2 Collection of Alternative Execution Strings
        3. 28.2.3 Collection of Parallel Execution Strings
        4. 28.2.4 Integration of Programs
      3. 28.3 Starting Process Chains
        1. 28.3.1 Starting Local Process Chains
          1. Direct Scheduling of Local Process Chains
          2. Start via Metachain/API
        2. 28.3.2 Starting Remote Process Chains
    2. 29. Time Slots
      1. 29.1 Fixed Time Slots
      2. 29.2 Initiated Time Slots
      3. 29.3 Source-System-Specific Time Slots
    3. 30. Organization and Sequence
      1. 30.1 Transfer of Exchange Rates and Global Settings
      2. 30.2 Loading Master Data
      3. 30.3 Change Run
      4. 30.4 Extraction into the PSA
      5. 30.5 Transformation: PSA into DataStore Objects
      6. 30.6 Activation of DataStore Objects
      7. 30.7 Deletion of Cube Indexes
      8. 30.8 Transformation: DataStore Objects in Data Marts
      9. 30.9 Repair of Cube Indexes
      10. 30.10 Update of Database Statistics
      11. 30.11 Rollup of Filled Aggregates/BIA Indexes
      12. 30.12 Compression
    4. 31. Monitoring
      1. 31.1 Application Log
      2. 31.2 Data Target Administration
      3. 31.3 Monitoring Process Chains
        1. 31.3.1 Overview of Selected Process Chains
        2. 31.3.2 Monitoring a Selected Process Chain
        3. 31.3.3 Detailed Information on a Selected Process Step
      4. 31.4 Monitoring Load Processes
        1. 31.4.1 Monitoring Extraction Processes
        2. 31.4.2 Monitoring Data Transfer Processes
        3. 31.4.3 Error Search in Data Transfer Processes
      5. 31.5 Monitoring the Real-Time Data Acquisition
    5. 32. Model Trimming
      1. 32.1 Trimming the Range Partitioning
        1. 32.1.1 Monitoring and Troubleshooting
      2. 32.2 Trimming Dimensions
      3. 32.3 Trimming Delta Indexes in the BIA
      4. 32.4 Trimming Fact Tables in the BIA
      5. 32.5 Trimming Index Distribution in the BIA
    6. 33. Information Lifecycle Management
      1. 33.1 ILM for BasisCubes and DataStore Objects
        1. 33.1.1 Archiving
        2. 33.1.2 Deletion
        3. 33.1.3 Restore
      2. 33.2 ILM for Master Data
      3. 33.3 ILM for PSA and Change Log
      4. 33.4 ILM for Monitor Information
        1. 33.4.1 Archiving
        2. 33.4.2 Deletion
        3. 33.4.3 Restore
      5. 33.5 ILM for Application Logs
      6. 33.6 ILM for BW Statistics Data
  13. VII. Appendix
    1. A. Currency Conversion
      1. A.1 Exchange Rates/Translation Ratios
        1. A.1.1 Definition of Exchange Rates
        2. A.1.2 Maintenance of Translation Ratios
      2. A.2 Conversion Types
        1. A.2.1 Exchange Rate Type
        2. A.2.2 Source Currency
        3. A.2.3 Target Currency
        4. A.2.4 Time Reference
      3. A.3 Global Settings
        1. A.3.1 Currencies
        2. A.3.2 Measurements
        3. A.3.3 Fiscal Year Variants
        4. A.3.4 Calendar
        5. A.3.5 Transfer from SAP ERP
      4. A.4 Exchange Rates
        1. A.4.1 Transfer from SAP ERP Systems
        2. A.4.2 Transfer from Flat Files
    2. B. Logical Files and Paths
    3. C. Transport System
      1. C.1 BW Transport System
        1. C.1.1 Transport Connection
        2. Standard Transport System
        3. BW Transport Connection
        4. C.1.2 Changeability of Source System Assignments in the Staging
        5. C.1.3 Transport of Process Chains
        6. C.1.4 Transports in Large-Scale Architectures
        7. C.1.5 Developments in the Production System
        8. C.1.6 Creation of Source Systems
      2. C.2 Content Transport System
        1. C.2.1 Content Development
        2. C.2.2 Content Delivery
      3. C.3 Metadata in XMI Format
    4. D. Use of Metadata Content
      1. D.1 BI Content of the SAP ERP Source Systems
      2. D.2 BI Content of BW
        1. D.2.1 Demo Content
        2. D.2.2 Technical Content
    5. E. Tables
      1. E.1 Lock Logic for Processes
      2. E.2 Placeholders
      3. E.3 Properties of Adapters
    6. F. Table of Abbreviations
  14. Index
  15. Footnotes