Standards in Practice: Conformance Testing

1
Standards in Practice Conformance Testing

• Maurits van der Vlugt

2
Overview

• Barriers to SDI Implementation
• Issues with OGC Interoperability
• Conformance Testing
• CITE - Conformance Testing Initiative
• Demonstration

3
Barriers to SDI Implementation

• Institutional
• Concerns about data sharing
• Financial
• Cost recovery
• Need for new business models
• Technical
• Vendor Dependence
• Lack of Interoperability

4
Interoperability?
I am a 220-250V AC Wall Socket!
I am a 220-250V AC Power plug!
Lets Interoperate!
5
Interoperability?
I am an OGC compliant viewing client!
I am an OGC compliant data server!
Lets Interoperate?
6
OGC Interoperability Specifications

• Mature (e.g. WMS) and Emerging (e.g. WFS)
• WMS widely supported by most major vendors
• Many real-life applications becoming available
• Beyond Proof-of-Concept
• Increasingly part of government procurement
  conditions (USA, Europe, Australia)

7
Issues with (OGC-) Interoperability

• Interpretation of specifications
• Mandatory vs. optional elements
• Compliant, non-interoperable profiles
• Undocumented Implementation Features (aka
  Bugs)
• Anyone can claim compliance
• Outcome
• SCOTS (Standards-based Commercial Off The Shelf)
  often need additional customisation to
  interoperate
• Finger pointing between vendors/integrators
• Barrier to SDI development

8
Solution Conformance Testing

• Evaluate products for conformance with
  specifications
• Resolve interpretation issues
• Independent and objective
• Allow developers and integrators to iteratively
  test implementations
• Lead to Certification

9
OGC - CITE Initiative

• Conformance and Interoperability Testing and
  Evaluation
• OGC Collaborative project
• Sponsored by
• National Imagery and Mapping Agency (NIMA)
• European Union Satellite Center (EUSC)
• 7 participants, including Australian team
  SKM/SCO

10
CITE Mission

• Improve quality and quantity of OGC-compliant
  services
• Build testing software
• Define certification parameters
• Support development of open source reference
  implementations
• Validate GML
• Provide independent test platform for public
  specifications

11
Audience

• Vendors
• Testing and Support during product development
• Product Certification
• Systems Integrators
• Resolve Interoperability issues in multi-vendor
  infrastructure
• Implementing Organisations
• Independent product evaluation
• Procurement support (look before you buy)

12
CITE Deliverables

• Conformance Test Suites
• Collection of (semi-) automated tests for an OGC
  specification
• Online Test Facility
• Service Test Engine
• Runs test suite against service to be tested
• GML Validator
• Validates GML datasets
• Free for public use
• Test Portal
• Single access point
• Free for public use
• Test engines, scripts, background, discussion,
  news, FAQs, etc.

13
How does it work?
14
The Portal
15
Whats in the Portal

• Test facilites for 3 specifications
• GML Validator
• WMS 1.1.1 Service Tester
• WFS 1.0 Service Tester
• Reference Implementations
• WMS 1.1.1 / WFS 1.0
• Technical Resources
• Specification documents
• Test data sets
• Test scripts
• Discussion Forum, News, Events,

16
Test-a-service
17
Demonstration

• Testing the LPI-NSW Topoweb Service
• http//topoweb.lpi.nsw.gov.au/servlet/com.esri.wms
  .Esrimap?REQUESTGetCapabilitiesVERSION1.1.1
• Portal (pre-launch sneak preview)
• http//cite-test.socialchange.net.au
• Test Engine (for OGC members only until launch in
  October)
• http//cite.occamlab.com/tsOGC

18
Project Status

• Final testing of Portal and Test Engine completed
  last week
• Deployment in progress
• Launch 15 October 2003
• Sneak Preview http//cite-test.socialchange.net.a
  u
• WATCH THIS SPACE

19
Conclusion

• OGC specifications support Interoperability and
  enable SDIs
• Issues with specifications
• Interpretation
• Claimed, not proven
• Need for independent conformance testing
• Supports
• Vendors and developers in product development
  process
• Integrators evaluating product
• Pre-procurement assessment
• OGC Test Portal publicly available from 15
  October