Strategies for All Your Data

1
All Your XML
Sandeepan Banerjee Oracle Corporation Sandeepan.Ba
nerjee_at_Oracle.com
2
One Management System for All Your Data
RelationalCharacters, Numbers and Dates

• Complete
• Integrated
• Robust
• Scalable
• Secure
• Available on all platforms

XML DBIntegrated Native XML Database
Oracle Text Ultra SearchText management and
search
Oracle Locator SpatialLocation and Proximity
Searching
Oracle interMediaMultimedia management
Oracle Collaboration SuiteUnified Messaging and
Files
Extensibility FrameworkChemical, Genetic,
Engineering,
3
What is Oracle XML DB?

• Database support for the XML data model
• XMLType, XMLSchema, DOM Fidelity, Xpath,
• Hierarchical organization of the data
• WebDAV compliant with indexing for fast access
• Transparent storage optimizations
• Query Language SQL/XML (SQL2003) and XQuery

4
Oracle XML DB Features

• New XMLType
• XMLSchema Support
• Object-Relational Storage Maintaining DOM
  fidelity, CLOB storage with document fidelity
• XML-specific memory mgmt for better scalability
  and performance
• Lazily loaded virtual DOM, Schema caching
• Built-in XML operators for SQL/XML
  interchangability
• XPath Search in the server, and piecewise update
  of XML via XPath
• XSL Transforms in the server
• Enhanced XML Views for creating your own
  representations of XML
• New XML Repository
• FTP, WebDAV, HTTP protocol servers to move XML
  content in and out
• Foldering and Repository view over XML Content
  including access control
• Hierarchical Index, SQL Versioning
• SQL Repository Search

5
Common XML Architectures
6
XML DB Architecture
Application Development
JDBC/OCI/.NET
OracleNET
Oracle XML DB
7
XML DB Architecture
Content Oriented Access
HTTP Client
FTP Client
WebDAV Client
HTTP
WebDAV
FTP
XMLType Views/Tables
Repository
Oracle XML DB
Text Index
Path Index
8
XML DB Architecture
Data Content Integration
Oracle XDS (JCA-based) Connectivity
J2EE
Query Results Cache
Web Services Data Sources
XML Query
JVM
SQL
XML Query
XMLType Views
Repository
Oracle XML DB
Oracle HO Connectivity
Relational DBs
File Systems
Web Sources
9
XML DB Architecture
Web Services
Web ServicesDirectory
UDDI
WSDL
Discover
Publish
SOAP
9iAS SOAP Server
Web Service
Client Application
Invoke
HTTP/OracleNet
Oracle XML DB
10
XML DB Architecture
XML Messaging
Messaging Application
IDAP/JMS/AQ
OracleNet
Transform
Transform
XML Document
XML Document
Oracle Streams
Oracle XML DB
Oracle XML DB
DeQueue
EnQueue
11
Classes of XML Applications

• Managing Structured Documents
• Well-formed templated business-documents e.g.
  Purchase Orders, Phone Bills,
• Managing Unstructured Documents (Content)
• Documents, Messages, Instructions
• Life-cycle management of these assets under
  multiple channels
• Querying over integrated, normalized data from
  diverse sources

12
1. Managing Structured Documents

• Relational storage remains the right way to
  store highly structured data
• extended-relational or object-relational
  mechanisms have been available for some time to
  deal with ordering, recursion, substitution etc.
• As an XML programmer, you do not want to think
  about tables
• A hierarchical data model is what you want to
  manipulate
• XML DBs XMLType is about preserving the XML
  paradigm while getting the benefits of relational
  performance and scalability

13
D E M O N S T R A T I O N
Structured XML
14
(No Transcript)
15
(No Transcript)
16
(No Transcript)
17
(No Transcript)
18
(No Transcript)
19
(No Transcript)
20
(No Transcript)
21
(No Transcript)
22
(No Transcript)
23
(No Transcript)
24
Object Relational DB Basics

• Object types
• Collection types
• Object References
• LOBs

XML Schema Construct Object Construct
Complex type Object type
Local complex type Embedded object type
Complex type with maxOccurs gt 1 Collection type
Derived complex type Subtype
25
XMLType

• Native data type for XML
• Used to define columns of tables and views,
  arguments to stored procedures, etc.
• XML specific methods and operators for
• Querying and extracting XML using XPath
• Transforming XML using XSLT
• Validating XML using XML Schema
• Multiple Storage Options
• Unstructured Storage in CLOB
• Structured Storage into object-relational rows
  and columns
• Hybrid Storage
• Maintains application transparency to physical
  storage choice

26
XML DB and XML Schema

• XML Schema controls all aspects of processing
• Storage mappings
• In-memory representations
• Language Bindings
• XML Schema Registration Process
• Associates XML Schema with URL
• Generates Object types
• Creates default tables
• XMLType column can be constrained to a global
  element of registered schema

27
XML Schema Example

• ltschema targetNamespacehttp//www.oracle.com/PO.
  xsd
• xmlnspohttp//www.oracle.com/PO.xsd
• elementFormDefaultqualified
• xmlns"http//www.w3.org/2001/XMLSchema"gt
• ltcomplexType name"PurchaseOrderType"gt
• ltsequencegt
• ltelement name"PONum" type"decimal"/gt
• ltelement name"Company"gt
• ltsimpleTypegt
• ltrestriction base"string"gt
• ltmaxLength value"100"/gt
• lt/restrictiongt
• lt/simpleTypegt
• lt/elementgt

28
XML Schema Example (contd)

• ltelement name"Item" maxOccurs"1000"gt
• ltcomplexTypegt ltsequencegt
• ltelement name"Part"gt
• ltsimpleTypegt
• ltrestriction base"string"gt
• ltmaxLength value"1000"/gt
• lt/restrictiongt
• lt/simpleTypegt
• lt/elementgt
• ltelement name"Price" type"float"/gt
• lt/sequencegt lt/complexTypegt
• lt/elementgt
• lt/sequencegt
• lt/complexTypegt
• ltelement name"PurchaseOrder" type"poPurchaseOrd
  erType"/gt
• lt/schemagt

29
Generated Object Types

• TYPE "Item_T"
• (part varchar2(1000), price number)
• TYPE "Item_COLL" AS
• VARRAY(1000) OF "Item_T"
• TYPE "PurchaseOrderType_T"
• (ponum number,
• company varchar2(100),
• item Item_COLL)

30
Generated Tables

• TABLE po_tab
• OF XMLTYPE
• XMLSCHEMA http//www.oracle.com/PO.xsd"
• ELEMENT "PurchaseOrder
• VARRAY(item) STORE AS item_tab

31
XML Document Example

• ltPurchaseOrder xmlns"http//www.oracle.com/PO.
  xsd" xmlnsxsi"http//www.w3.org/2001/XMLSchema-
  instance" xsischemaLocation"http//www.oracle.
  com/PO.xsd
• http//www.oracle.com/PO.xsd"gt
• ltPONumgt1001lt/PONumgt
• ltCompanygtOracle Corplt/Companygt
• ltItemgt
• ltPartgt9i Doc Setlt/Partgt
• ltPricegt2550lt/Pricegt
• lt/Itemgt
• ltItemgt
• ltPartgt8i Doc Setlt/Partgt
• ltPricegt350lt/Pricegt
• lt/Itemgt
• lt/PurchaseOrdergt

32
XML Storage PO_TAB and LINE_TAB
Row ID ponum company
1 1001 Oracle Corp
Parent Row ID Array Index part price
1 1 9i Doc Set 2250
1 2 8i Doc Set 350
33
Structured Storage

• Attributes and single-valued elements
• Stored as columns in single row
• SQL data types correspond to XML Schema types
• SQL constraints correspond to XML Schema
  constraints
• Multi-valued elements (collections) stored in
  separate nested tables
• One row per item in collection
• Nested table row stores parent key
• Array Index column stores the position
  information
• Number column uses full range of floating
  points
• Elements inserted in the middle of the collection
  get (previous_array_index next_array_index)/2
• Supports multiple levels of nesting
• Embedded object types
• Embedded collection types with multiple nested
  tables

34
DOM Fidelity

• Structured storage guarantees DOM fidelity
• No whitespace fidelity
• System binary attribute in object types
• SYS_XDBPD
• PD attribute stores non-relational information
• Ordering of elements
• Comments
• Processing Instructions
• Namespace declarations
• Prefix information
• Mixed content text nodes that are intermixed
  with elements are stored in the system column

35
Do you need it ?

• Can disable on a Type by Type basis using
  annotation xdbmaintainDOMfalse
• Location of Namespace declarations
• No Comments or Processing Instructions
• Do not care about empty Vs missing elements
• Elements in an All and Choice will be ordered as
  per the Schema, not the instance
• Not worried about defaults
• Type does not allow Mixed text

36
Type defn with DOM Fidelity

• desc LINEITEMS_T
• LINEITEMS_T is NOT FINAL
• Name Null? Type
• ------------------------------- --------
  ------------------
• SYS_XDBPD
  XDB.XDBRAW_LIST_T
• LINEITEM
  LINEITEM_V
• desc LINEITEM_V
• LINEITEM_V VARRAY(2147483647) OF LINEITEM_T
• LINEITEM_T is NOT FINAL
• Name Null? Type
• ------------------------------ --------
  ------------------
• SYS_XDBPD
  XDB.XDBRAW_LIST_T
• ITEMNUMBER
  NUMBER(38)
• DESCRIPTION
  VARCHAR2(256 CHAR)
• PART PART_T

37
Type defn without DOM Fidelity

• desc LINEITEMS_T
• LINEITEMS_T is NOT FINAL
• Name Null? Type
• ------------------------------- --------
  ------------------
• LINEITEM
  LINEITEM_V
• desc LINEITEM_V
• LINEITEM_V VARRAY(2147483647) OF LINEITEM_T
• LINEITEM_T is NOT FINAL
• Name Null? Type
• ------------------------------ --------
  ------------------
• ITEMNUMBER
  NUMBER(38)
• DESCRIPTION
  VARCHAR2(256 CHAR)
• PART
  PART_T

38
xdbmaintainDOM

• ltxscomplexType name"LineItemsType"
  xdbSQLType"LINEITEMS_T"
  xdbmaintainDOM"false"gt
  ltxssequencegt
• ltxselement name"LineItem"
  type"LineItemType"
  maxOccurs"unbounded" xdbSQLName"LINEITEM"
  xdbSQLCollType"LINEITEM_V"/gt
  lt/xssequencegt
• lt/xscomplexTypegt ltxscomplexType
  name"LineItemType" xdbSQLType"LINEITEM_T"
  xdbmaintainDOM"false"gt
• ltxssequencegt
• ltxselement name"Description"
  type"DescriptionType
  xdbSQLName"DESCRIPTION"/gt
• ltxselement name"Part" type"PartType"
  xdbSQLName"PART"/gt lt/xssequencegt
• ltxsattribute name"ItemNumber"
  type"xsinteger"
• xdbSQLName"ITEMNUMBER"
  xdbSQLType"NUMBER"/gt
• lt/xscomplexTypegt

39
Dom Fidelity
40
XDB Attributes in XML Schema

• Mapping details captured as new attributes within
  the XML Schema
• Oracle attributes use namespace
  http//xmlns.oracle.com/xdb
• Input XML Schema does not need to be annotated
• Default assumptions for all XDB attributes
• Input XML Schema can explicitly specify XDB
  attributes
• SQLName Name of column or type attribute
• SQLType Name of object type
• SQLCollType Name of collection type
• MaintainOrder Permits turning off order
  maintenance

41
Hybrid Storage of XML

• Two ends of storage spectrum
• Store entire content in a single LOB
• Full shredding of XML data
• Hybrid Storage Options
• Combination of shredding and LOB storage
• xdbSQLTypeCLOB applied to ltcomplexTypegt
• CLOB storage ideal for fragments that are not
  intended for query or partial updates
• Example XHTML content embedded within resource
  descriptors (metadata)

42
Querying XMLType

• XPath based operators
• existsNode
• Boolean operator
• Checks for existence of node identified by XPath
• extract
• Extracts a fragment identified by XPath
• extractValue
• Retrieves the raw value of leaf node identified
  by XPath
• Namespace Aware
• ANSI SQL/XML Standards effort

43
Query Rewrite

• Automatic rewrite of XPaths during query
  compilation
• Rewritten query directly accesses underlying
  relational columns
• Introduces joins with nested tables
• Enables use of indexes

44
Query Rewrite - Example

• Original Query
• SELECT extractValue(value(p),
  '/PurchaseOrder/PONum')
• FROM po_tab p
• WHERE
• existsNode(value(p),
• '/PurchaseOrderCompanyOracle') 1
• Rewritten Query
• SELECT p.ponum
• FROM po_tab p
• WHERE p.company 'Oracle'

45
Query Rewrite Examples
XPath Expression Rewrite Logic
Simple XPath expressions /PurchaseOrder/Company /PurchaseOrder/_at_poDate Traversals of object attributes 1 or more levels
Collection traversals /PurchaseOrder/LineItem/Part Joins with appropriate nested tables
Predicates CompanyOracle Relational predicates
List indexes Lineitem1 Operator to access i-th item of collection Uses index on array_index column
46
Updating XMLType

• Updating entire XML document
• Partial Update
• Uses UpdateXML() operator
• XPath identifies element or attribute to be
  updated
• New value for the updated node is specified
• Rewritten to directly update underlying column(s)
• Similar mechanisms for
• Inserting new nodes
• Deleting node(s)

47
Update Rewrite - Example

• Original Statement
• UPDATE po_tab p
• SET value(p) updatexml(value(p),
• '/PurchaseOrder/PONum/text()',
• 9999)
• WHERE existsNode(value(p), /PurchaseOrderCompany
  Oracle) 1
• Rewritten Statement
• UPDATE po_tab p
• SET p.ponum 9999
• WHERE p.Company Oracle

48
XMLType Views

• Provide an XML view of relational data
• Good evolutionary strategy
• New XML apps on XML abstraction of existing data
• SQL/XML Standard operators used to generate XML
• Views can generate schema based XML
• Insert / Update / Delete operations via Instead
  of Triggers
• Queries over XML views are rewritten to directly
  access underlying relational columns

49
XMLType View - Example

• CREATE VIEW po_view of XMLTYPE
• XMLSCHEMA "po.xsd" ELEMENT "PurchaseOrder" AS
• SELECT
• XMLElement("PurchaseOrder",
• XMLForest(p.ponum "PONum",
• p.company "Company"),
• (SELECT XMLAGG(
• XMLElement("Item",
• XMLForest(i.part "Part",
• i.price "Price"))
• FROM items_rel_tab i WHERE i.po_id p.id))
• FROM po_rel_tab p

50
Complex XML Schemas

• Cyclic Definitions
• Object types created with references
• Table row contains reference to other rows stored
  in same or different table
• Keys to document and parent rows stored in nested
  rows
• Complex type derivation
• Mapped to object type inheritance
• Wildcards
• Mapped to CLOB attributes

51
Indexing XMLType

• Multiple index types
• B-Tree and bitmap indexes
• Function-based indexes
• Create index on specific XPath expressions
• Text indexes
• Inverted lists provide section-based search
• Also support keyword based search within textual
  content

52
Performance Metrics
53
Summary Native XML Data Type

• Abstraction for storing XML
• Native Server data type
• Use as Table, Column, PL/SQL variable
• Supports constraints and referential integrity
• Structured and Unstructured storage options
• XML Specific methods enable
• XPath based Navigation and Searching of XML
  content
• XPath based manipulation and update of XML
  content
• Server based XSLT Transformation
• XMLSchema validation

54
Summary XML Schema

• Validation of instance Documents
• Basis for Structured Storage of XMLType
• XML shredded and stored as SQL Objects
• DOM Fidelity
• Optimized Collection Management
• B-Tree indexes over collections
• Query Re-write of XPath expressions
• Partial Update
• Lazily Loaded Virtual DOM
• Object model automatically derived from XML Schema

55
Summary SQL / XML Interoperability

• XML generation from SQL Queries
• Generate an XML document from a SELECT statement
• Support for generating complex documents
• XMLType views provide XML access to relational
  content
• SQL query and update of XML Content
• XPath based extraction of XML Content (SELECT
  List)
• XPath based query of XML Content (WHERE Clause)
• XPath based update of XML content
• XPath based relational views over XML content

56
Summary API

• Higher Level API for working with XML with full
  support for
• Generation
• Storage and Retrieval
• Indexing, Searching
• Query and Update
• Transformation

57
Summary Generating XML

• SQL XML operators make it easy to generate XML
  from relational data
• Result set of SQL Query is XML document
• XMLType Views allow persistent XML access to
  relational data
• Content of XMLType view can be exposed as a
  virtual document
• Direct access via HTTP / WebDAV or FTP

58
2. Managing Unstructured Data

• More and more content is being produced as XML
  (Microsoft Word, Corel XMetal, Arbortext Epic, )
• Markup improves search, processing, organization,
  
• XML DBs Repository enables XML document content
  to be stored as files in folders without
  losing strong-management, queryability,
  unbreakable security etc.
• XML is doing for unstructured data what
  Relational did for structured create a standard
  way to store, query and manage unstructured data

59
Managing Unstructured Data with Oracle XML DB

• XML data model and APIs familiar to Content
  Developers
• Integrated Repository
• WebDAV compliant
• Xpath index for fast traversal of foldering
  hierarchies
• SQL Queryable
• Integrated Text Processing
• Optimizations such as tag aware search

60
Using XML for Unstructured Content
Measurable results

• Soft results
• Increase information quality
• Support knowledge management initiatives
• Improve information freshness
• Enhance information accuracy
• Greater content Interactivity
• Improve content flexibility
• Increase customer satisfaction
• Expand customer retention

Source
61
D E M O N S T R A T I O N
Unstructured XML
62
Oracle Corel Integration
63
Oracle XML DB Repository

• XML Repository based IETF DAV Specification
• File / Folder metaphor for storing and managing
  content
• ACL Based access control
• Basic versioning support
• Supports WebDAV, HTTP and FTP protocols
• Access and update content using standard tools
• Full SQL access and update
• Programmatic access via multiple APIs
• Hierarchical Index
• Patented, high performance folder-traversal
  operations and queries

64
XML Searching

• The CONTAINS SQL function is an extension to SQL,
  and can be used in any query.
• The oracontains XPath function is an extension
  to XPath, and can be used in any call to
  existsNode, extract or extractValue..

SELECT ID FROM PURCHASE_ORDERS WHERE CONTAINS(
DOC, 'electric INPATH (/purchaseOrder/items/item/c
omment)' )gt0
SELECT ID FROM PURCHASE_ORDERS_xmltype WHERE
existsNode( DOC, '/purchaseOrder/commentoraconta
ins(text(), "(lawns AND wild) OR flamingo")gt0',
'xmlnsora"http//xmlns.oracle.com/xdb"' ) 1
65
CONTAINS vs oracontains()

• The CONTAINS SQL function
• Needs a CONTEXT index to run - if there is no
  index, you get an error
• does an indexed search, and is generally very
  fast
• returns a score (via the score operator)
• can restrict a search using both Full Text and
  XML structure
• restricts a search based on documents (rows in a
  table), not nodes
• cannot be used for XML structure-based projection
  (pulling out parts of an XML document)
• The oracontains XPath function
• does not need an index to run, so it is very
  flexible - separates application logic from
  storing and indexing considerations
• may do an unindexed search, so it may be
  resource-intensive alternatively, may also get
  rewritten to use an index
• does not return a score
• can restrict a search using Full Text in an XPath
  expression
• can be used for XML structure-based projection
  (pulling out parts of an XML document)

66
More Search Examples
SELECT ID FROM PURCHASE_ORDERS WHERE CONTAINS(
DOC, 'electric INPATH (//items/item_at_partNum"872-
AA"/comment)' )gt0
SELECT ID FROM PURCHASE_ORDERS WHERE CONTAINS(
DOC, 'electric INPATH (///item.//comment)'
)gt0
SELECT ID FROM PURCHASE_ORDERS WHERE CONTAINS(
DOC, '(electric INPATH (//comment)) INPATH
(/purchaseOrder/items)' )gt0
SELECT ID FROM PURCHASE_ORDERS_xmltype WHERE
existsNode( DOC,
'/purchaseOrder/commentoracontains(text(),
"(lawns AND wild) OR flamingo")gt0',
'xmlnsora"http//xmlns.oracle.com/xdb"' ) 1
SELECT extract( DOC, '/purchaseOrder/items/item/c
ommentoracontains(text(), "electric")gt0',
'xmlnsora"http//xmlns.oracle.com/xdb"')
Comment FROM PURCHASE_ORDERS_xmltype WHERE
CONTAINS( DOC, 'electric INPATH
(/purchaseOrder/items/item/comment) ' )gt0
67
Query Rewrite

• A query with existsNode, extract or extractValue,
  where the XPath includes oracontains, may be
  considered for Query Rewrite iff
• The XML is Schema-based
• the first argument to oracontains (text_input)
  is either a single text node whose parent node
  maps to a column, or an attribute that maps to a
  column. The column must be a single relational
  column (possibly in a nested table).
• The re-written query will use a CONTEXT index
  iff
• there is a CONTEXT index on the column that
  text_input's parent node (or attribute node) maps
  to
• the oracontains policy exactly matches the index
  choices of the CONTEXT index
• the CONTEXT index was built with the
  TRANSACTIONAL keyword in the PARAMETERS string

68
3. XML Query-based Integration

• Why XQuery ?
• Declarative way to query XML documents
• Why Java?
• Run in mid-tier or database
• Future server implementation in C
• Why XML Database ?
• Native XML storage
• XML data management
• Performance optimizations
• SQL/XML or XQuery depending on data
• Status
• OTN downloads (pending W3C standard finalization
  in 04)

iAS J2EETM Platform
XML DB
Server JVM
69
What is XQuery?

• New W3C language to query XML
• Evolution from XPath path navigation
• Analogous to SQL
• SQL for structured storage (relational)
• XQuery for structured (schema)/unstructured
• Text extensions work in progress

70
XQuery example

• ltPurchaseOrder pono100gt
• ltCustomergtSCOTTlt/Customergt
• ltPODategt12-Sep-2003lt/PODategt
• ltShipAddr street211,Redwood Lane
  citySeattle
• stateCA/gt
• ltLineItem ItemNo232 ItemBox
  Quantity200/gt
• ltLineItem ItemNo344 ItemCard
  Quantity3/gt
• ltLineItem ItemNo333 ItemPen
  Quantity33/gt
• lt/PurchaseOrdergt

71
XQuery example - query

• Search all of scotts po for gt 20 items
• ltScottLargeItemsgt
• for i in doc(scott.xml)/PurchaseOrder/Line
  Item
• where i/_at_Quantity gt 20
• return
• ltLargeItemgti/_at_Itemlt/LargeItemgt
• lt/ScottLargeItemsgt

ltScottLargeItemsgt ltLargeItem ItemBox/gt
ltLargeItem ItemPen/gt lt/ScottLargeItemsgt
72
Differences from SQL

• Navigation-oriented (using XPath expressions)
• Different type system (XMLSchema based simple
  types)
• Identity-based (XML Node identities and document
  order)
• Namespace aware name-resolution (functions,
  variables, element creation)
• Row based versus Item based
• Results are heterogeneous sequences
• Does not have all SQL extensions (e.g, OLAP,
  Full-Text..)

73
Whats really new?

• New Data Model
• Shared with XSLT-2.0 / XPath 2.0
• Input Output - XQuery data model
• Type system based on XMLSchema
• Supports un-typed and typed values
• Supports construction of XML
• Has prolog with static/dynamic context

74
XQuery Data Model

• XML -gt Infoset -gt PSVI -gt Data model
• Everything is a sequence
• Sequence is list of Items
• Items can be
• XML nodes (document, PI, comment, attributes)
• Simple content (number, date)
• Can be typed (XMLSchema) or untyped

75
XQuery Sequence

• Virtual container of data model items
• Can be homogeneous or heterogeneous
• Example Sequence
• ( ABC (atomic value of type
  xsistring),
• ltagt33.2lt/agt (element a of type xsifloat),
• _at_foox 33 (attribute foox of type
  xsiinteger),
• ltShipAddrgt
• ltstreetgt... lt/streetgt
• ...
• lt/ShipAddrgt (element ShipAddr of type
  fooAddress),
• )

76
XQuery Expressions

• Everything is an expression.
• Inputs defined in the language -
• doc() takes in URI returns data model
• collection() URI input returns collection
• bind variables external variable input
• Function recursive, named, typed
• Defined FO lots - shared with XPath
• Modules Similar to PL/SQL packages

77
XML construction

• Element, Attribute and other constructors
• Direct constructors
• ltEmp name/emp/enamegt21lt/Empgt
• Computed Constructor
• element Emp attribute name /emp/ename,
• text( 21)
• Sequence constructor
• ( a, b, 1 to 10 ) sequence of strings,
  integers

78
XQuery Type system

• XMLSchema primitive and complex types
• Query can be statically typed (conformance level)
• Dynamic types can be more specific
• Validation is implicit lax,strict,skip

79
XQuery path language

• XPath 2.0 new path language (subset)
• Type aware unlike XPath 1.0
• New operators
• Node compares is, isnot
• Value compares eq, gt, lt
• Existential compares - lt, , gt
• Range searches, Satisfies, etc.
• Ordering maintained unordered() hint

80
Java standards

• JSR 225 (XQJ) new JSR in the works
• IBM, BEA, Datadirect, SoftwareAG, Ipedo
• Define standard for access in Java
• Early draft Q1 04?
• Final draft sometime in summer 04?
• Taking good aspects of JDBC

81
Example

• Preliminary API (warning! subject to change)
• import javax.xml.xquery
• XQConnection con Datasource.getconnection(jdbcco
  )
• XQExpression expr con.prepareExpr(for i in
  doc..)
• XQItemSet items expr.executeQuery()
• while (items.next())
• if (items.isOfType(XQTypes.StringType))
• String str items.getString()

82
XQJ main goals

• Get/Set Static and Dynamic context
• Get/Set External variables
• Prepare XQuery Expressions
• Provide Java classes for data model
• Provide all types of access (SAX, StaX)
• Metadata and type supportput of one query is
  input of another

83
SQL/XML standards

• Part of ANSI/ISO SQL2003
• XML datatype will support XQuery data model
• XQuery() function to be part of SQL
• XMLTable function to explode XML to relational
  values using XQuery
• Define relational mapping to XQuery
• Provide XQuery functions to map tables
• Provide mapping from SQL to data model

84
XQuery Support in Oracle

• Standalone Java query engine
• 100 Java
• Integrated into mid-tier, can run in server-tier
  (integrated with XML DB)
• Interoperate with XSLT/XPath
• XMLDB integrated database engine
• SQL/XML standard support
• Optimized queries rewrite to SQL

85
XQuery Java implementation

• XQuery or XQueryX input
• Extensible function implementation
• Compiles into rowsource like structures
• Integrate with SQL data (view function)
• Uses XSU for relational data
• Optimization push XQuery to XMLDB
• Can run in mid-tier or server-tier JVM

86
XQuery Java Architecture
X Q J A P I
P A R S E R
C O M P I L E R
E X E C U T O R
XQueryX
Execution Tree
XQuery
XML
XQueryX
Plan in XML
Datasources
87
Oracles XML Data Services

• XDS provides common XQuery services
• Caching cache in Java/XMLDB
• View text storage, functional definitions
• Web services for XQuery
• Common XML Adapter framework
• XQuery the integration language
• Caching in XMLDB query pushdown
• JNDI tree used to discover datasource

88
XDS Architecture
Query Builder Tool
XDS Client APIs
Applications using XDS e.g. Portals, Reports
etc
EJB
JSP Tags
Web Service
XML Data Synthesis
J2EE Security Framework
Cached XML Data Source
Meta-data Repository
XDS Cache Security
XQuery Result
XQuery Subsystem
XDS Caching Service
XQ4J/JXQI
Oracle Enterprise Manager
XQuery Engine
XMLDataSource modules
Stored Query
XML DB
Cached XML Data Source
In-Memory
XML Data source adaptors
CCI-XML
JCache
File system
J2CA EAI
Oracle Apps
RDBMS
Files
HTTP Web Cache
Java Functions
Web Services
SAP
JMS
89
Example XDS usage

• User registers webservice as datasource
• XDS creates an XQuery module automatically
• User Query for querying webservice
• import module namespace wssdatasrc/stockws
• for i in wssgetcompanies()
• return wssget_stock_price(i/name)
• wss namespace prefix for the loaded module
• JNDI lookup to get datasource implementation
• XDS adapters implement datasource

90
Example Query pushdown

• Query against XML DB
• for i in view(PurchaseOrder)/ROW
• where i/City Fresno
• return i/PoNo
• With rewrite - equivalent to
• for i in sql(select extract(p,PoNo)
• from purchaseorder p
• where existsnode(p,CityFr
  esno)1
• return i

91
Server-tier XML Query

• Query SQL tables/views
• Query XMLtype tables
• Query data in repository
• Query URLs (using UriType in DB)
• Result is XMLType
• Rewrite when possible

92
XQuery Database Architecture
To rewrite or not to rewrite
XQuery
XQuery rewrite/ Execution engine
XMLDB Repository
SQL tables/views
93
Example SQL integration

• SQL tables (get all 150K employees)
• select XQuery(
• for i in view(SCOTT.EMP)/ROW
• where i/SALARY gt 150000
• return i/EMPNO)
• from dual
• ltEMPNOgt2100lt/EMPNOgt
• ltEMPNOgt344lt/EMPNOgt

94
Example XMLType integration

• XMLType tables as input
• select p.XQuery(ltPO pono/PoNo/gt)
• from purchaseorder p
• where p.XQuery(/PurchaseOrderShipAddr/CityFre
  sno)
• is not null
• After rewrite
• select XMLElement(PO,
• XMLAttributes(p.xmldata.PONo)
  )
• from purchaseorder p
• where p.xmldata.ShipAddr.City Fresno

95
Example Repository integration

• Query files from repository
• doc() queries files from repository (rewrite)
• collection() maps to directories
• select XQuery( for i in doc(/public/foo.xml)
• return i)
• from dual
• ltFOOgt ....lt/FOOgt

96
Datasources

• Enables arbitrary input sources
• files, cache, JCA datasources
• xmldatasrc Oracle language addition
• Datasource API
• initialize
• describe
• execute
• Fetch
• Bind (an existing DOM)

97
D E M O N S T R A T I O N
XML Query
98
More Information

• Introduction to XML DB
• http//www.oracle.com/ip/index.html?xmldbtp_intro.
  html
• Oracle Technology Network XML Page
• http//otn.oracle.com/tech/xml/index.html
• Oracle Technology Network XML DB Page
• http//otn.oracle.com/tech/xml/xmldb/index.html
• Oracle Technology Network XML Querying Page
• http//otn.oracle.com/tech/xml/xmldb/htdocs/queryi
  ng_xml.html
• Sample XML DB Applications
• http//otn.oracle.com/sample_code/tech/xml/xmldb/i
  ndex.html

99
A
100
(No Transcript)