Title: This presentation contains information proprietary to Oracle
1(No Transcript)
2Oracle Data Integrator Technical Deck
- Mark Pare
- Sr. Sales Consultant
- Oracle Higher Education
3Agenda
ltInsert Picture Heregt
- 4 Key Differentiators
- Architecture
- 6 Steps to Production
- ODI or ESB?
- Popular Usage Scenarios
4Oracle Data Integrator 4 Key Differentiators
5Why Data Integration?
NEED
HAVE
6Challenges Emerging SolutionsIn Data
Integration
CHALLENGE
EMERGING SOLUTION
- Increasing data volumes decreasing batch windows
- Non-integrated integration
- Complexity, manual effort of conventional ETL
design - Lack of knowledge capture
Shift from E-T-L to E-LT Convergence of
integration solutions Shift from custom coding
to declarative design Shift to pattern-driven
development
7Oracle Data Integrator
- Data Movement and Transformation from Multiple
Sources to Heterogeneous Targets - Performance Heterogeneous E-LT
- Flexibility Active Integration Platform
- Productivity Declarative Design
- Hot-Pluggable Knowledge Modules
BENEFITS
KEY DIFFERENTIATED FEATURES
8Differentiator E-LT ArchitectureHigh Performance
1
- Transform in Separate ETL Server
- Proprietary Engine
- Poor Performance
- High Costs
- IBM Informaticas approach
- Transform in Existing RDBMS
- Leverage Resources
- Efficient
- High Performance
- Benefits
- Optimal Performance Scalability
- Easier to Manage Lower Cost
9Differentiator Active IntegrationBatch,
Event-based, and Service-oriented Integration
2
- Evolve from Batch to Near Real-time Warehousing
on Common Platform - Unify the Silos of Data Integration
- Data Integrity on the Fly
- Services Plug into Oracle SOA Suite
- Benefits
- Enables real-time data warehousing and
operational data hubs - Services plug into Oracle SOA Suite for
comprehensive integration
10Differentiator Declarative DesignDeveloper
Productivity
3
- Specify ETL Data Flow Graph
- Developer must define every step of Complex ETL
Flow Logic - Traditional approach requires specialized ETL
skills - And significant development and maintenance
efforts - Declarative Set-based Design
- Simplifies the number of steps
- Automatically generates the Data Flow whatever
the sources and target DB - Example SALAH
- Benefits
- Significantly reduce the learning curve
- Shorter implementation times
- Streamline access to non-IT pros
11Differentiator Knowledge ModulesHot-Pluggable
Modular, Flexible, Extensible
4
Pluggable Knowledge Modules Architecture
Journalize Read from CDC Source
Load From Sources to Staging
Check Constraints before Load
Integrate Transform and Move to Targets
Service Expose Data and Transformation Services
Reverse Engineer Metadata
Reverse
WS
WS
WS
Staging Tables
Load
Integrate
Services
CDC
Check
Journalize
Target Tables
Sources
Error Tables
Sample out-of-the-box Knowledge Modules
SAP/R3
Log Miner
SQL Server Triggers
Oracle DBLink
JMS Queues
Check MS Excel
TPump/ Multiload
Oracle Merge
Oracle Web Services
Siebel
DB2 Journals
DB2 Exp/Imp
Check Sybase
Oracle SQLLoader
Type II SCD
Siebel EIM Schema
DB2 Web Services
Benefits
- Tailor to existing best practices
- Ease administration work
- Reduce cost of ownership
12Oracle Data IntegratorArchitecture
13Architecture Conceptual View
- Java design-time environment
- Runs on any platform
- Thin client for browsing Metadata
- Java runtime environment
- Runs on any platform
- Orchestrates the execution of data flows
- Metadata repository
- Pluggable on many RDBMS
- Ready for deployment
- Modular and extensible metadata
14Architecture Component View
15Oracle Data Integrator6 steps to Production
16Overview 6 steps to Production
- Retrieve/Enrich metadata
- Design transformations
- Orchestrate data flows
- Generate/Deploy data flows
- Monitor executions
- Analyze impact / data lineage
Production
Development
Production Servers and Applications
Development Servers and Applications
Data Warehouse
Data Warehouse
CRM
CRM
Legacy
Legacy
ERP
ERP
ODI Design-Time Environment
ODI Runtime Environment
Agent Data Flow Conductor
Agent Data Flow Conductor
Operator
Metadata Navigator
17Retrieve/Enrich Metadata
1
Design-Time Environment
Development Servers and Applications
- Reverse-engineer Metadata
- Automatic
- Customizable
- 40 technologies supported
- Enrich Metadata
- Documentation
- Declarative rules for Data Integrity
- Cross-technologies references
ODI Designer
ERP
Data Warehouse
Design-time Repositories
CRM
Files / XML
Legacy
18Design Transformations
2
1
Define What You Want
Oracle Data Integrator Interface Declarative
Design
Bulk Load Changed Data Capture
Incremental Update Slowly Changing Dimension
19Orchestrate Data Flows
3
- Sequence Transformations
- Leverage OracleDI Tools
- Data Quality Processes
- Files/Archives Management
- Send/Receive Emails
- Web Services Invokation
- Event Detection
- Create your Own Tools
- Use Control Structures
- Loops
- Conditions
- Error Handling
20Generate and Deploy Data Flows
4
- Create Scenarios
- Compile Data Flows for Run-time
- Version the Data Flows
- Advanced Version Management
- Deploy to Production
21Monitor Executions
5
- View sessions running in real-time
- Review generated code
- Detailed run-time statistics
- Restart failed sessions
22Analyze impact / data lineage
6
- Maintain a large number of data flows in a
complex environment - Web-based end-to-end data lineage
- Understand your data flows
- Follow the path of data
- Drill-down to transformations
?
23Oracle Data IntegratorODI or ESB?
24What tool is best suited for task X?
25ESB and ODI in real-life scenarios
Data Latency
Oracle Data Integrator
Batch (over 2 hours)
Oracle Enterprise Service Bus
Asynchronous
Synchronous (immediate)
Real-life Scenarios
Data Volume Processing
Message by Message
Mini Batches
Large Volume (over 1M)
26Oracle Data IntegratorExtended Capabilities
27Extended Capabilities
- Master Data Management enabled
- Common Format Designer
- Automated generation of canonical format and
transformations - Built-in Data Integrity
- Real-time enabled
- Changed Data Capture
- Message Oriented Integration (JMS)
- SOA enabled
- Generation of Data Services
- Generation of Transformation Services
- Extensibility
- Knowledge Modules Framework
- Scripting Languages
- Open Tools
28Oracle Data IntegratorMaster Data Management
Enabled
29MDM Enabled Canonical Format Design
- Use in conjunction with packaged MDM solution
- Design and Populate Canonical Format
- Use existing metadata artifacts to design MDM
application (entities, fields, relationships) - Generate and maintain Master Data structure
- Generate and deploy transformations using
metadata artifacts
Enterprise Service Bus
CRM
SCM
Legacy
ERP
30MDM Enabled Built-in Data Integrity
- Data Integrity Firewall
- Auditing, cleansing and recycling
- Declare constraints at table level
- Design mappings and check flow integrity
- Audit, cleanse or recycle rejected records
31Oracle Data IntegratorReal-time Enabled
32Real-time enabled Changed Data Capture
- Publish and Subscribe CDC Framework
- Database logs
- Triggers
- Third-tier solutions
- Ensures read transaction integrity across
multiple tables - Design or generate Mappings
- Select Journalized Data Only
- Start Journals
CDC
33Real-time enabled Message Oriented Integration
- Connect to Publish and Subscribe JMS Message
Providers - Ensure messages delivery with transaction
integrity - High-volume bulk transformations
- Design complex bulk transformations mixing
Queues, Databases and Applications - Use JMS Queues and topics as sources or targets
CDC
34Oracle Data IntegratorSOA Enabled
35SOA Enabled Data Access Services
SOA Infrastructure
- Generate and share data access services
- Generate and deploy data services
- Test data services
- Leverage data services in your SOA infrastructure
Business Processes
Services
Data Access
Transform
ESB
Business
36SOA Enabled Data Flow Services
- Expose transformations as Web Services
- Orchestrate data flows
- Publish data flows as web services in your SOA
infrastructure
SOA Infrastructure
Business Processes
Services
Data Access
Transform
ESB
Business
37Oracle Data IntegratorExtensible Framework
38Extensibility Knowledge Modules
- 120 KMs out-of-the-box
- Tailor to existing best practices
- Ease administration work
- Reduce cost of ownership
- Customizable and extensible
KM Interpreter
39Extensibility Scripting Framework
- Extend data flows with scripting procedures
- Leverage all database languages
- SQL, PL/SQL, Transact SQL, etc.
- Use Operating Systems shell scripts
- Win32 DOS, sh, ksh, csh, OS400 commands, JCL,
etc. - Choose from compatible Bean Scripting Framework
languages - Java, JavaScript, Jython (Java Python), Perl,
etc.
40Extensibility Open Tools
- Extend ODI tools
- Add your own tools to the Design Palette
- Implement OdiOpenToolAbstract Java Interface
- Register Open Tool in ODI Designer
- Use Open Tool in your design environment
41Popular Usage Scenarios
42E-LT for Data WarehouseCreate Data Warehouse for
Business Intelligence Populate Warehouse with
High Performance ODI
- Heterogeneous sources and targets
- Incremental load
- Slowly changing dimensions
- Data integrity and consistency
- Changed data capture
- Data lineage
43SOA InitiativeEstablish Messaging Architecture
for Integration Incorporate Efficient Bulk Data
Processing with ODI
- Invoke external services for data integration
- Deploy data services
- Deploy transformation services
- Integrate data and transformation services in
your SOA infrastructure
44Master Data ManagementCreate Single View of the
TruthSynchronize Data with ODI
- Use in conjunction with packaged MDM solution
- Use as infrastructure for designing your own hub
- Create declarative data flows
- Capture changes (CDC)
- Reconcile and cleanse the data
- Publish and share master data
- Extend metadata definitions
45MigrationUpgrade Applications or Migrate to New
Schema Move Bulk Data Once and Keep in Sync with
ODI
- Bulk-load historical data to new application
- Transform source format to target
- Synchronize new and old applications during
overlap time - Capture changes in a bi-directional way (CDC)
46ODI Enhances Oracle BIPopulate Warehouse with
High Performance ODI
- Oracle Business Intelligence Suite EE
- Simplified Business Model View
- Advanced Calculation Integration Engine
- Intelligent Request Generation
- Optimized Data Access
- Oracle Data Integrator
- Populate Enterprise Data Warehouse
- Optimized Performance for Load and Transform
- Extensible Pre-packaged E-LT Content
47ODI Enhances Oracle SOA SuiteAdd Bulk Data
Transformation to BPEL Process
- Oracle SOA Suite
- BPEL Process Manager for Business Process
Orchestration - Oracle Data Integrator
- Efficient Bulk Data Processing as Part of
Business Process - Interact via Data Services and Transformation
Services
48ODI Enhances Oracle SOA SuitePopulate BAM Active
Data Cache Efficiently
- Oracle SOA Suite
- Business Activity Monitoring for Real-time
Insight - Oracle Data Integrator
- High Performance Loading of BAMs Active Data
Cache - Pre-built and Integrated
49Roadmap and Direction
50Oracle Data Integrator Roadmap
- Focus Areas for Next Major Release
- Deep Integration with Fusion Middleware
- Runtime, Design time, Security, Administration,
Events - Functional Integration with Oracle Warehouse
Builder - Runtime Integration, Metadata Sharing, Knowledge
Module Sharing - Deployment of ODI for Embedded Data Integration
- OracleBI Enterprise Edition, Data Hubs,
Application Migrations - Enhanced Usability and Debuggability
- Wizards, New Views, User-definable Debugging
- Improved Support for Native Oracle Database
Features - Oracle OLAP
51ODI Statement of Direction
- Statement of Direction
- http//www.oracle.com/technology/products/odi/stat
ement-of-direction.pdf - Key Points of Direction
- Commitment to heterogeneous systems support
- Including DB2, Teradata, Netezza, Hyperion,
etc. - Commitment to Fusion design principles
- Including J2EE compliance, container
portability - Commitment to best-of-class E-LT performance
- Across platforms, batch realtime, high
complexity
52(No Transcript)
53(No Transcript)