Mark W. Skall, Division Chief

1
Program Review
Mark W. Skall, Division Chief Candy Leatherman,
Secretary email skall_at_nist.gov Tel. No.
301-975-3262 Div. Web Site http//www.itl.nist.g
ov/div897/
2
Division 897 Funding - FY00
Projected Balance 257,062 ( 637,570
transferred to 2001 OA Reserve)
OA - 1,615K
ITS - 585K
STRS - 5,207K
Employees 35 Full Time 3 Part Time 10
Intermittent 5 Guest Researchers
3
SDCTs GOAL

• To improve quality of software in industry
  through the development of
• Technology
• Measurements
• Standards

• Philosophy
• Concentrate on key areas at forefront of
  technology
• Get involved early and partner with industry
• Fill industry void
• Transfer technology and move on

• Specific Strategy
• Conformance Tests
• Reference Implementations
• Diagnostic Tests
• Research to reduce costs of testing
• Reference Data
• Standards developed jointly with industry

4
Excerpts from Assessment Panel Report
6/26/00Division Review
The planning and documentation methods used by
SDCT could serve as a model for other ITL
divisions. The process has used clearly-defined
criteria to select state-of-the-art programs with
clearly defined priorities and goals,
well-identified roles, and measurable
contributions to national and international
standards organizations. Examples include the
work on conformance tests for XML, Distributed
Interactive Learning Systems, and for DASE . .
.Directions can change quickly within the
software industry. The division carefully
monitors these changes and acts appropriately. .
. .Industry has provided many testimonials
concerning the great value of the divisions
standards work, including The OASIS-NIST XML
Conformance test suite is critical for our
industry, Norbert Mikula, CTO, OASIS NIST made
strong contributions (X3D) and resolved our
knottiest problems, Don Brutzman, Board of
Directors, Web3D consortium, and The high
quality test suite and certification program is
an invaluable resource for ATA, Robert Peel,
Director of Airworthiness and Standards
5
Excerpts from Assessment Panel Report
6/26/00Laboratory Level Review
New projects are started based on the importance
of the work to U.S. industry, and the work viewed
by the panel generally had specific, focused
goals. An example of this approach is the work
on XML done in SDCT. This project is notable
because it addresses a major standard on which
industry was making little progress, and it also
effectively leveraged skills and approaches
developed in a now terminated project on VRML. .
.Another example of the improved planning process
is the existence of clear termination criteria,
which have been useful in sunsetting projects.
For example, the Real-Time Java project and the
work on RBAC satisfied their completion criteria
and therefore ended this year.
6

• Standards and Conformance Testing Group

• Software Quality Group

• Interoperability Group

7

Group Leader Lynne Rosenthal Supy Proj. Leader
Lisa Carnahan Clare Lucey, Secretary
8

Mary Brady Lisa Carnahan Laurent Ciarletta
(GR) Anthony Cincotta (S) Chris Dabrowski Alden
Dima Leonard Gallagher Leonard Gebase Neil Gima
(S) Alan Goldfine
Martha Gray Michael Kass Clare Lucey Thomas
Logue Carmelo Montanez-Rivera Thomas
Rhodes Richard Rivello Jacqeline Schneider John
Tebbutt Marie-Noelle Terasse (GR) Mark Zimmerman
9
Division 897 Standards and Conformance
Testing Group
10
SQG
Software Quality Group

• Develop methods to automate software testing
• Develop software diagnostic tools
• Develop reference data
• Formal methods

Group Leader John Barkley Debbie Blackstone,
Secretary
11
SQG
Software Quality Group
Tamer Ahmed (GR) Paul Ammann (F) Debbie
Blackstone David Brinkley (F) Paul Black Neva
Carlson John Cherniavsky Anthony Cincotta
Keith Gallagher (F) Roger Gima (S) Michael
Koo Mary Laamanen James Lyle William
Majurski Douglas White
12
Division 897 Software Quality Group
13
IG
Interoperability Group

• Ensure Federal agency requirements are input into
  voluntary standards committees - work through
  Federal CIO Council
• Help Federal agencies and industry achieve
  interoperability through application of Division
  products
• Provide technical support to voluntary standards
  committees
• Support NIST paperless office effort utilizing
  digital signatures

Acting Group Leader Lisa Carnahan Vacant,
Secretary
14
IG
Interoperability Group
Daniel Allen Bruce Bargmeyer (RA) Frederick
Boland Gary Fisher Larry Fitzwater (GR)
Elizabeth Fong Kathryn Harvill Roy Morgan Judith
Newton Gertrude Sherwood
15
Division 897 Interoperability Group
16
Strategy
17
Development of Conformance Tests

• Emphasis on newer technologies

• Tests developed in parallel with standards

• Tests developed before implementations

• Need buy-in from industry

• Need high impact

• Need technology transfer

18
Development of Conformance Tests (contd)

• Legacy activities
• CGM
• SQL
• POSIX
• Ada and other programming language
• VRML

• Current activities
• XML and companion standards (DOM, XSL, etc.)
  Digital TV
• X3D
• Learning Technologies (IMS)

19
XML Conformance Testing
Project Team Mary Brady Carmelo Montanez Ricky
Rivello Mark Zimmerman

20
Electronic Commerce
B2C (business-to-consumer) Interactive Televisio
n (DASE, SMPTE)
B2B (business-to-business) Manufacturing (ICM)
Distributed Learning (IMS)
XML Technologies
Internet / World Wide Web
21
Why NIST?

• Electronic Commerce Growth
• Growth is exponential, already outpacing last
  years predictions
• Changing the way businesses do business
• Revolutionizing the way we consume and play
• Advances are necessary in
• Tools for describing / sharing information
• XML technologies
• Applying IT to solve vertical market demands
• NIST provides
• XML Technologies
• Conformance Testing
• NIST developed tests, available on-line, for use
  in testing applications
• NIST technical leadership, neutral third-party
  
• Unbiased Feedback of Specification Errors
• Vertical Markets
• Applied IT research

22
HTML or XML?

• ltPurchasergt
• ltNamegt Mary Brady lt/Namegt
• ltAddress StreetNN Rm 572 CityGaithersburg,
  StateMD Zip20899lt/Addressgt
• lt/Purchasergt
• ltItemgt
• ltPart_numbergt1234lt/Part_numbergt
• ltItemDescriptiongtTreeslt/ItemDescriptiongt
• ltCostgt50.00lt/Costgt
• ltQuantitygt20lt/Quantitygt
• ltTotalCostgt1000.00lt/TotalCostgt
• ltInstructionsgtBe sure to water the root ball day
  of deliverylt/Instructionsgt
• lt/Itemgt
• ltShipTogt
• ltNamegtSusan Carscaddenlt/Namegt
• ltAddress StreetNIST, Bldg 301 CityGaithersburg
  StateMD Zip20899lt/Addressgt
• lt/ShipTogt

Figure 2 XML Purchase Order
23
XML Technologies
Electronic Commerce, Distance Learning, Health
Care Manufacturing, Interactive Televison
Programming Languages
Transformations Formatting Objects
Signatures
Stylesheets
Query Language
DOM
Domain Specific
Registries Repositories
Namespaces
XML.ORG BIZTalk
E-Business Learning Objects XML/EDI
Boards
Medical
Education
Procurement
Auto Parts
...
...
Ratings
Libraries
Control
...
Information Description
XML Syntax, Information Set, Schema, Linking,
Fragment
24
XML Conformance Testing

• OASIS XML Conformance
• Sun, IBM, Fuji Xerox, DataChannel, MicroStar, W3C
  members

25
DOM Conformance Testing

• W3C, OASIS
• Sun, IBM, Microsoft, Netscape, Oracle

26
FY00 Progress -Tests

• Completed XML Test Suite (release 1)
• Completed XML Test Suite (release 2)
• Incorporated new XML tests and fixes
• Completed DOM-Ecmascript Test Suite (release 1)
• Completed DOM-Java Test Suite (release 1)
• Develop XSLT Test Suite
• partnership with Lotus Corp. and OASIS

27
FY00 Progress -Committee Leadership

• Chair OASIS Conformance and XML Testing
  Committees
• Provide leadership for XSLT and Schema Testing
  committees
• Provide leadership for UN/CEFACT ebXML initiative
  (conformance and reg/rep)
• Provide conformance guidance
• Develop conformance clause
• Harmonize OASIS and ebXML efforts

28
FY01 Plans

• Complete XSLT test suite
• Develop test suites for XML Schema and other XML
  Technologies
• Develop XML-based automated test tools to improve
  test development
• Continue to chair OASIS testing
  committees

29
FY00 Resources

• 3.5 FTE
• 700K (STRS)
• FY01 Resources
• 5 FTE
• Travel required to collaborate in standards and
  test development efforts
• Equipment 10K

30
XML Conformance Testing

• Customers
• IT industry e.g., IBM, Sun, Microsoft, Oracle,
  DataChannel, Documentum
• Vertical industries e.g., Education (IMS),
  Financial (e.g., Dun and Bradstreet), Travel
  (e.g., Sabre)
• Impact
• Conformance test suites have been used to improve
  the quality of XML/DOM processors (many of which
  are embedded in other XML software solutions).
  These are used in many vertical markets by
  millions of customers.
• Feedback to standards developers improve the
  specifications

31
Interactive TV
Project Team Alan Goldfine John Barkley Doug
White Len Gebase
32
Interactive TV

• TV and the Internet are each recognized as a
  technology that seems to change everything (BW,
  Oct 4 99)
• Interactive TV is the convergence of TV and the
  Internet
• Profound impact on Electronic Commerce expected
• WebTV 1 million subscribers, 350 hrs/wk of
  programming
• AOLTV projected 2 million subscribers in 2
  years
• ITL partnering to develop the standards and
  tests required for the success of Interactive TV

33
Interactive TV
Traditionally separate media are converging

• Wireless phones do email
• Internet does broadcast radio and TV
• Broadcast TV references links to Web pages

34
Interactive TV
Currently, for the viewer
35
Interactive TV
Demonstration
Current production
36
Interactive TV
Demonstration
37
Interactive TV
Advanced Television Systems Committee (ATSC)

• Members include ABC, CBS, IBM, Intel, Lucent,
  Microsoft, NBC, Warner Brothers
• DASE (DTV Application Programming Environment)
  Standard for a platform independent, high-level
  abstraction for integrating DTV and the Internet
• DASE specifies programming environment for DTV
  receivers
• Downloadable DASE applications portable across
  different TV set tops

38
Interactive TV
NISTs Role in ATSC DASE

• Joint project with the High Performance Systems
  and Services Division (895)
• Chair DASE Conformance Working Group where key
  players include ABC, Gateway, Microsoft, Sun
• Partner with Unisoft to develop conformance
  tests
• Reference implementation for programming
  environment

39
Interactive TV
Society of Motion Picture and Television
Engineers (SMPTE)

• Members include ABC, CBS, IBM, Intel,
  Microsoft, NBC, Warner Brothers
• DDE (Declarative Data Essence) Standard for
  Internet content and bindings to analog and
  digital streams (formerly ATVEF)
• Declarative Content HTML, CSS, ECMAScript, DOM,
  Triggers (for syncing the two mediums)

40
Interactive TV
NISTs Role in SMPTE DDE

• Contribute text to specification
• Partner with Unisoft to develop conformance
  tests
• Develop conformance tests for triggers and
  stream binding to integrate with existing
  Unisoft DDE element tests
• Develop prototype receiver testbed
• Participate in Applications Data Essence (ADE)
  Study Group

41
FY00 Progress

• Chair DASE conformance task group
• Conformance section of the DASE Standard
  completed
• Review of DASE conformance test assertions from
  Unisoft completed
• DDE Standard completed, adoption underway
• DDE Bindings Standard underway
• Test assertions for DDE triggers underway
• Testbed for DDE receivers underway
• ADE Study draft completed

42
FY01 Plans

• Continue to chair DASE conformance group
• DASE Standard adopted
• Continue DASE conformance test development with
  Unisoft
• DDE and DDE bindings Standards adopted
• Develop DDE conformance tests for triggers,
  integrate with Unisoft tests
• Develop test assertions for DDE bindings
• Complete DDE Receiver testbed
• Begin DDE-2 Standard development

43
FY00 Resources

• 2 FTE
• 400K (STRS, ATP-100K)
• FY01 Resources
• 4.5 FTE
• Equipment 20K
• Travel required for standards meetings and
  testing collaborations

44
Interactive TV

• Customers
• IT, e.g., IBM, Microsoft
• broadcast TV, e.g., NBC, DirecTV
• entertainment, e.g., Disney, Universal
• consumer electronics, e.g., RCA, Intel
• Impact
• New Medium TV/Internet convergence
• Interactive applications portable across
  receivers from different manufacturers
• ATSC and SMPTE invited NIST participation
• NIST chairs DASE Conformance Group

45
X3D Conformance Testing
Project Leader Mike Kass
46
Objectives

• Continue the NIST/Web3D partnership begun in 1996
• ITL created the VRML Test Suite (VTS) and Viper
  parser
• Create an X3D Test Suite ( XTS )
• Modify VRML test requirements and test
  descriptions
• Translate existing VRML tests to X3D format
• Chair the Interoperability/Conformance WG
• Interface with developers and specification
  writers to resolve ambiguities in specification
• Provide feedback to developers on browser
  conformance

47
FY00 Progress

• Completed translation of VTS to X3D Test Suite
  (XTS)
• Developed test harness to enable developers to
  contribute tests
• Interest from DraW and Sun for using the test
  suite and making the results public

48
FY01 Plans

• Low level support - assisting community in using
  the VRML and X3D Test Suites
• No new development planned. Maintenance as
  needed

49
FY00 Resources

• 0.75 FTE
• 150K (STRS)
• FY01 Resources
• 0.2 FTE (minimal)
• No Travel is expected

50
X3D Conformance Testing

• Customers
• X3D Consortium including Sun, Sony DraW
  Computing, Shout, Blaxxun
• Impact
• Conformance test suites are being used to improve
  the quality of X3D browsers
• DraW Computing has said that they would make
  their test results public

51
Learning Technologies
Project team Martha Gray Tom Rhodes Jacki
Schneider Lisa Carnahan
52
Learning Technologies

• Learning technology is
• Learning anytime, anyplace
• Interactive, distributive learning systems
• In collaboration with industry, NIST/ITL is
• defining requirements and specifications
• developing prototypes and reference
  implementations
• developing testing techniques and tools

53
Why Now?

• The Gartner Group estimates that companies in
  the United States spent 58 billion on training
  in 1998, and of that, 9.4 billion went to
  IT-related training. They expect this figure to
  grow to 10.5 billion this year. Gilbane
  Report, July/Aug 1999
• Just as electronic commerce exploded over the
  past two years, the stage is now set for
  e-learning to become the next big wave in
  Internet-based applications Cisco Systems CEO
  and President John Chambers
• Worldwide, the education and training market is
  over a trillion (US) dollars. IEEE

54
Projects

• IMS Global Learning Consortium, Inc.
• Chair conformance team
• advise members on writing IMS specifications
• ensure testability of specifications
• advise on how to test and set up testing programs
• Develop Meta-data repository reference
  implementation
• IEEE and ISO
• Provide conformance expertise
• Harmonization of efforts among specifications
• DoDs Advanced Distributive Learning (ADL)
• Executive Order 13111 on Federal Training -
  NIST/ITL co-leaders with ADL and DOL.

55
FY00 Progress

• Continue collaboration with IMS, IEEE LT
  committee and NIST/ATP
• Chair IMS conformance testing activities
• Developed IMS conformance test policies and
  procedures
• Served as Technical Lead of SBIR solicitation,
  assessed and recommend proposals
• Develop repository of learning objects (this work
  coordinates with Reg/rep subproject)

56
FY01 Plans

• Continue to chair IMS conformance committee
  providing expertise to emerging IMS
  specifications
• Develop a conformance test plan for applicable
  IMS specifications

57
FY00 Resources

• 1.5 FTE
• 300K (STRS, ATP)
• FY01 Resources
• 1 FTE
• Minimal travel required to attend standards
  meetings and interoperability demos

58
Learning Technologies

• Customers
• IMS, DoL, DoD, SimonSchuster, Sun, IBM, Apple,
  Microsoft, UNC, Calif State U
• Impact
• Use of standard metadata for global learning will
  improve discovery, evaluation, and use/reuse of
  learning objects
• NISTs conformance expertise contributed to
  clearer, less ambiguous, testable IMS
  specifications
• Conformance test plans and procedures will enable
  IMS to determine compliant products

59
ATA CGM Testing Program
60
FY00 Progress

• Completed final release of the ATA CGM Test Suite
• Provided guidance to ATA Recognized Test
  Laboratory on using the test suite and assessing
  clients
• Chair Control Board - resolve questions of
  interpretation of the test suite and standard.
• Initiated discussions on ATA's eCommerce
  activities - explore a possible XML testing
  program

61
FY01 Plans

• Continue to support ATA's CGM Test Program
• No new CGM development planned
• Maintenance as needed
• Provide guidance on the use of NIST's XML test
  suite and its applicability and use in ATA

62
FY00 Resources

• 0.25 FTE
• 75K (STRS)
• FY01 Resources
• 0.25 FTE
• Travel required to support ATA test program and
  potential for an ATA XML test program

63
CGM

• Customers
• Air Transport Association Airlines and their
  suppliers (e.g., United, Northwest, Airbus, FedX,
  Boeing, Rolls-Royce)
• CGM Developers (e.g., Autotrol, Intergraphics)
• Impact
• NIST Conformance test suite used by ATA CGM Test
  Program to validate CGM interpreters
• Quality, reliable CGM products used for
  representing graphics in airframe maintenance
  manuals

64
(No Transcript)
65
Technology Transfer from NIST to ATA
Developed CGM Interpreter Test Suite Developed
Test Program Framework Developed test procedures
Formed and chair Control Board Developed Test
Lab criteria and evaluation criteria Assessed
Test Lab for ATA recognition Serve as technical
adviser Completed transfer
66
Testing Services
Status of Software Operational Testing Services
67
Testing Services (contd)
Testing Products or Services - Offered by Other
Organizations -
Developed with NIST Assistance
68
Strategy

• Conformance Tests
• Reference Implementations
• Standards developed jointly with industry
• Diagnostic Tests
• Research to reduce costs of testing
• Reference Data

69
(No Transcript)
70
AromaPervasive Computing Project
Project Team Alden Dima Bill Majurski Mary
Laamanen
71
AromaPervasive Computing Project

• Emerging trend toward
• numerous, easily accessible, often inconspicuous
  computing devices and sensors mobile or embedded
  in the environment connected to ubiquitous
  network structure
• Industry is asking for
• Standards for service discovery, APIs, e-commerce
  services
• Testing and diagnostic tools to improve quality,
  interoperability
• Reference implementations
• Aroma multi-division effort to
• Explore standardization and measurement issues
• Attract industry interest and partnerships

72
Aroma Prototype (Aroma adaptor Aroma projector)

• Goals - To discover technical, standards and
  measurement issues related to
• Connecting portable wireless devices to
  traditional networks
• Service discovery, self-configuration and dynamic
  resource sharing
• Mobile code and data
• Software infrastructure needed to create and
  manage pervasive services and applications

73
FY00 Progress

• Present papers at NIST PC 2000, JavaOne, and
  International Pervasive Computing Workshop
• Co-chaired NIST Pervasive Computing 2000
  conference
• Developed Aroma Adapter-Projector with Div. 892
• Measured performance of Aroma adapter
• Developed conceptual model based on the OSI
  Reference Model
• Exploring low-end pervasive computing
• Embedded and 8,16-bit processors
• Collaboration with US Naval Academy

74
FY01 Plans

• Co-chair NIST Pervasive Computing 2001 conference
• Explore use of ADL in pervasive computing model
• Implement a virtual private smart space (VPSS)
  test bed
• Extend Aroma projector to include digital cameras
  and VPSS technology
• Continue performance measurements
• Develop diagnostic and testing tools

75
FY00 Resources

• 1.50 FTE
• 300K (STRS, OA-DISA)
• FY01 Resources
• 3 FTE
• Travel required to conferences to present work
  and collaborate with colleagues
• Equipment 15K

76
Pervasive Computing

• Customers
• IT industry the usual suspects
• Consumers e.g., GE, Sunbeam, Phillips, Northeast
  Utilities, Xerox
• Universities USNA, UMD, UC Berkeley
• Impact
• Solicited industry needs for standards and
  measurements
• PC 2000 conference launched awareness of NIST as
  key player
• Papers presented at conference stimulated
  discussions on pc and standards
• Ref imp demonstrated feasibility of PC
• Enable widespread adoption and integration with
  software tools to create, manage, measure and
  test pervasive services and applications

77
XML Registry/Repository(Reg/Rep)
Project Team Lisa Carnahan Len Gallagher Mike
Kass
78
XML Registry/RepositoryReference Implementation

• Built to OASIS specification
• Being used by both OASIS and UN/CEFACT (ebXML)
• Built using IMS Meta-data specification
• Apply lessons learned to EC repositories

79
FY00 Progress

• Revised the OASIS Registry and Repository
  Specification
• Implement a reference implementation
• Participate in ebXML Reg/Rep team
• Ensure testability of specification
• ensure compatibility between ebXML and OASIS
  specifications

80
FY01 Plans

• Update reference implementation
• Develop conformance test suite/data for OASIS
  specification
• Coalesce the ebXML and OASIS specifications

81
FY00 Resources

• 1.5 FTE
• 325K (STRS)
• FY01 Resources
• 2 FTE
• Travel required to attend standards meetings

82
XML Registry/Repository

• Customers
• EC/Enterprise Integrators Data Channel,
  Documentum, Oracle, Microsoft, Sun, IBM
• Consumers Vertical market partnerships and
  supply-chains (e.g., automotive, insurance,
  travel, manufacturing)
• Impact
• Repositories allow efficient use of XML and
  eliminate duplication
• Enable SMEs to participate in EC marketplaces and
  supply chains
• Reduce cost of doing business through timeliness
  and increased automation

83
Internet Commerce for Manufacturing
(ICM)(ITL,EEEL, MEL, MEP)
Project team Tom Rhodes Liz Fong Kathy Harvill
84
FY00 Progress

• Assumed project leadership (at request of EEEL)
• Defined standards-based framework for ICM testbed
  (reference implementation)
• Conducted workshops and presented papers/demos
• NAMT, SIMA, and ATP reviews
• Industry conferences IPC, NEMI, ITEM2000
• Collaborate with NEMI, IPC, Georgia Tech, UMBC
• Evaluating industry specifications (e.g.,
  RosettaNet, OAG, C-Net, eCo FW) for B2B
  interactions
• Evaluating applicability of agent and web
  technologies

85
FY01 Plans

• Continue industry collaborations
• Demonstrate distributed ICM testbed operations
• (with GaTech, NEMI, IPC partners)
• Test and validate ICM information model in ICM
  testbed
• Continue evaluation of applicable standards and
  technologies and their incorporation in ICM
  testbed

86
FY00 Resources

• 1.5 FTE
• 350K (STRS-EEEL, ATP)
• FY01 Resources
• 2 FTE (other FTE from EEEL, MEL, MEP)
• Travel required to collaborate, participate in
  standards meetings, and present work

87
ICM

• Customers
• Consumers PC assembly supply chain participants
• Consortia NEMI, IPC, Rosetta Net
• Impact
• Reduce the cost of circuit-board manufacturing by
  enabling the rapid exchange of build information
  and change notices for all supply-chain
  participants
• Demonstrates the use of XML in a vertical industry

88
Strategy

• Conformance Tests
• Reference Implementations
• Standards developed jointly
• with industry
• Diagnostic Tests
• Research to reduce costs of testing
• Reference Data

89
Forward-Looking Standards
Role-Based Access Control (RBAC) Virtual Reality
Modeling Language (VRML)/X3D XML and companion
standards (DOM, CSS, etc) Java ADL IMS DASE/SM
PTE OASIS Registry/Repository GIS
90
STANDARDS COMMITTEES AND RELATED ACTIVITIES
FEDERAL GOVERNMENT CIO Council Interoperability
Committee Standards Working Group
91
Real-Time Java
Project team Lisa Carnahan Alden Dima
92
Real-Time Java

• Assistance asked by industry providers and
  government users
• Collaborate with Sun, HP, IBM, New Monics, FTL
• Hosted continuing workshops for real-time Java
  requirements
• Addresses real-time issues in both Java embedded
  and Java server application environments
• NIST coalesced industry to produce one set of
  requirements leading to one spec
• Open consensus forum for requirements
• BOTH camps (Sun and HP, Microsoft, etc.) have
  used NIST-led requirements document to produce
  specs

93
EE Times

• Interestingly, both the splinter group and
  Sun are using as a serious basis for their
  efforts a requirements document for real-time
  (Java) extensions prepared by a separate group
  working under the auspices of the National
  Institute of Standards and Technology (NIST).
  That is raising hopes that both standards might
  ultimately come together.

94
Real-time Java Requirements Group

• Requirements for Real-time Extensions to the
  Java Platform (final release -September, 1999)

• NISTs Requirements Group effort dissolved after
  successful completion of its goal.

The J Consortium
The J Consortium uses real-time requirements
developed by the Requirements Group for Real-Time
Extensions for the Java Platform, an effort
sponsored by the National Institute of Standards
and Technologies (NIST).
Press Release -J Consortium Unveils First
Public Real-Time Specification For The Java
Platform
CUPERTINO, Calif., September 24, 1999
95
FY00 Progress

• Real-Time Java Requirements document completed
• Worked with both spec development groups to help
  coalesce industry
• Member of Sun RT Experts Group - contributed to
  specification
• Worked with J Consortium to help develop
  specification
• Project sunset

96
FY00 Resources

• 1.25 FTE
• 250K (STRS)
• FY01 Resources
• 0 (Project sunset)

97
Real-Time Java

• Customers
• Sun, IBM, HP, NewMonics and users of real-time
  Java
• Impact
• Coalesced industry to produce one set of
  requirements
• 2 spec efforts compatible
• New book on Real-time Java mentions NIST
• Presented to and endorsed by VCAT

98
ADL (Architecture Description Language)
Project Team Chris Dabrowski John Tebbutt
99
Architecture Description Language

• Focus on high-level structure of the overall
  application (i.e., systems architecture)
• Provide features for modeling software systems
  architecture
• E.g., Structure, components, interfaces,
  interconnections
• Support automated analysis and verification of
  design
• Enable testing during design phase
• Saves over testing at code phase

100
Architecture Description Language (ADL)
Software Architecture ..an abstract system
specification consisting primarily of functional
components described in terms of their behaviors
and interfaces and component-component
interconnections Hayes-Roth, 1994
ADL A language that provides features for
modeling a software systems conceptual
architecture Medvidovic 1996
101
NISTs Unique Role
Standards Role Neutral Expert Applied Research
Contribute to ADL body-of-knowledge

• Standards Collaborations
• Propose additional functionality to SAE (first
  industry standards effort)
• Uniquely positioned to bring academic and
  industry communities together.
• Apply developing AADL standard (based on
  Honeywells MetaH) to Unmanned Ground Vehicle
  build additional functionality into AADL standard
• Explore ADL relationships with IEEE and Open
  Group efforts

• Applied Research Contribute to ADL
  body-of-knowledge
• Research application of formal approaches to
  ADLs
• Enhance ADL features for verifying conformance of
  application systems architectures to reference
  architectures (e.g. RCS)
• Publish successes and lessons-learned for
  industry awareness
• Provide feedback to appropriate ADL tool
  developers

102
FY00 Progress

• Participated in development of SAEs Avionics ADL
  (AADL) standard
• Explore ADL standardization efforts in IEEE and
  Open Group
• Established partnerships with DISA, US Army,
  ITL/Security Division
• Identify domain areas for applying ADL to explore
  standardization and testing issues
• E.g., government smartcard, unmanned ground
  vehicle, pervasive computing

103
FY01 Plans

• Provide support for the formalization of AADL to
  ensure its testability
• Develop prototype of US Army robotics
  architecture
• collaboration with MEL
• dependent on DISA funds
• Develop prototype of Smartcard Architecture w.
  Div 893
• Demonstrate and document the benefits and
  functionality of ADLs
• Produce a comparative study of different ADLs to
  support further development and standardization
  of ADLs

104
FY00 Resources

• 1 FTE
• 200K (STRS)
• FY01 Resources
• 2 FTE
• Equipment and Software 10K
• Travel required to standards meetings and to
  participate in conferences and industry meetings

105
Architecture Description Language

• Customers
• Consumers Industries that rely on
  component-based systems (e.g., manufacturing,
  avionics, real-time)
• ADL tool vendors
• Impact
• Moved ADL from University research to industry
  standard
• Provide uniform descriptions of the structure and
  function of software.
• Increases software quality by reducing errors
  through testing in design phase.
• Increase the producibility, quality, and
  reusability of component-based software systems.

106
GIS
Project Leader Chris Dabrowski
107
FY00 Progress

• Convener of ISO/TC211 Framework and Reference
  Model Working Group
• Initiated and lead effort to use UML to formalize
  and integrate GIS family of standards
• Shared lessons learned with OMG
• Presented UML/GIS work at OMG meetings
• Provided recommendations for the evolution of the
  UML language

108
FY01 Plans

• Complete integration of UML in GIS standards
• Sunset project

109
FY01 Resources

• 0.25 FTE
• Limited Travel anticipated to transition ISO GIS
  work and contribute experiences to OMG

110
Geographic Info Systems

• Customers
• Consumers Industries with GIS requirements
  (e.g., GPS industry)
• Developers of GIS information systems
• ISO TC 211, the OpenGIS consortium, Federal
  Geographic Data Committee (FGDC), Object
  Management Group (OMG)
• Impact
• enable industry growth and preclude barriers to
  business activity
• provide a basis for exchange of information in
  heterogeneous environments distributed over space
  and time
• Less ambiguities in the UML specification
• Increased use of UML in standards (normative)

111
CIO Council Standards Working Group
Technology Transfer Act/OMB Circular A-119 direct
federal agencies to use voluntary consensus
standards in lieu of government unique
standards NIST directed to coordinate federal
technical standards activities with private
sector technical standards activities NIST/ITL
initiated effort to create Standards Working
Group within Federal CIO Council Standards
Working Group approved late last year by the CIO
Council Interoperability Committee Standards
Working Group to be focal point of discussion,
definition, and coordination of federal
requirements for voluntary IT standards, and will
assess the results of voluntary IT standards in
meeting requirements Standards Working Group
chaired by a member of this Division, and will be
supported by this Division in its efforts to
provide a unified voice to the voluntary IT
standards community for Federal IT standards
technical requirements
112
FY00 Progress

• Prototype of web based support software for
  collection, evaluation, and status reporting on
  Federal Government technical requirements for
  voluntary standards developed and demonstrated.
• Electronic Documents conference held featuring
  speakers from private industry, Federal
  Government, and voluntary standards
  organizations.
• Meetings of the Standards Working Group held on
  an as needed basis.
• Standards requirements forwarded to appropriate
  voluntary standards committees as the
  requirements are approved.

113
FY01 Plans

• Continue development of web based support
  software for Standards Working Group.
• Make software available for possible use by other
  Working Groups within the EIEIT structure.
• Meetings of the Standards Working Group held on
  an as needed basis.
• Standards requirements forwarded to appropriate
  voluntary standards committees as the
  requirements are approved.
• Consider holding second Electronic Documents
  conference.

114
FY00 Resources

• 1.5 FTE
• 325K (STRS, OA-CIO Council)
• FY01 Resources
• 1.5 FTE Combination of time from Group Leader
  and others in Interoperability Group

115
CIO Council Standards Working Group

• Customers
• Other Federal Agencies
• Impact
• Federal agencies input requirements to
  standards groups and utilize voluntary consensus
  standards
• Eliminates duplicative standards

116
Strategy

• Conformance Tests
• Reference Implementations
• Standards developed jointly with industry
• Diagnostic Tests
• Research to reduce costs of testing
• Reference Data

117
Software Diagnostic Tools

• Program slicer tool - Project Leader Jim Lyle

• Computes lines of code which could affect given
  variables

• Reduces debugging time by directing effort to
  relevant parts of computation

• Currently only for C programs

• Extending slicer to Java and Fortran

• Commercialized by private company as Y2K solution

• VM View Toolkit - Project Leader Alden Dima

• Diagnostic tool embedded at runtime into the JVM
• Exposes internal VM conditions in the form of an
  execution trace
• Downloaded by Sun, HP, Rockville Collins, Apogle,
  Insignia, Wind River, Ada Core, Johns Hopkins

118
Year 2000 Problems - Project Leader Gary Fisher
Developed source code scanner
Participate in Interagency Year 2000 Task Force
119
FY00 Progress

• Y2K web site completed
• Embedded system study completed
• Report sent to Koskinen
• Problems highlighted and impacts reported
• Produced new emphasis on embedded systems
• Project sunset

120
FY00 Resources

• 0.25 FTE
• 50K (STRS)
• FY01 Resources
• 0 (Project sunset)

121
Year 2000

• Customers
• All businesses that use computers
• Consumers who use computers
• Impact
• Y2K rollover went without incident

122
Strategy

• Conformance Tests
• Reference Implementations
• Standards developed jointly with industry
• Diagnostic Tests
• Research to reduce costs of testing
• Reference Data

123
Automatic Generation of Spec Based Tests Using
Formal Methods
Project Team Paul Black Bill Majurski Jim
Lyle Micheal Koo
124
Industry Need

• Software testing is enormous cost to industry
• 30 - 50 of budget
• Development may be 1000s of staff-years
• Testing cost may be millions
• Current methods mostly hand generated tests
• Inefficient
• Not rigorous enough
• Software testing key to reducing security
  incidents
• More than 75 of the incidentsare the direct
  result of bugs. and more real testing - not
  'beta testing' in the marketplace - will yield
  programs that are less vulnerable to attack.
• (Business Week, February 28 "Locking Out the
  Hackers")

125
(No Transcript)
126
Current Test Development Methodology
Requirements
English Language
review inspect
Specification
English Language
Human Programmers
review inspect
Source Code
Software Tests
Programming Language
127
Advanced Test Development Methodology
Requirements
Mathematical Logic
formal proof
Specification
Mathematical Logic
Test Generator
informal proof
Source Code
Software Tests
Programming Language
128
Accomplishments

• Method for automatically generating tests from
  formal specifications
• Associated coverage metric
• Publications and presentations
• ICFEM (98), WSEST(99), HASE (99), DASC (99)
• GMU, UMBC, College Park, Brisbane, NRL
• Feasibility collaborations leading to
  commercialization
• Argus, a secure computer operating system
• Ford Motor Research Lab powertrain control
  systems
• Northrop Grumman fighter radar computers

129
FY00 Progress

• Feasibility tests on models from Ford and Argus
• Initial formalization of BLAST (Basic Local
  Alignment Search Tool identifies similar DNA
  sequences) Algorithm completed in collaboration
  with CSTL
• Hosted Workshop on Testing Software Using Formal
  Methods
• Hosted Tutorial on Formal Methods
• 3 papers published
• 2 papers submitted and accepted
• 4 papers submitted
• 9 talks (6 invited)

130
FY01 Plans

• Commercialize ATG Method with Ford and software
  company (tbd)
• Continue feasibility studies with Argus
• Complete BLAST formalization

131
FY00 Resources

• 2.5 FTE
• 500K (STRS-Competence)
• FY01 Resources
• 2.5 FTE
• Equipment 5K
• Travel required for validation collaborations and
  presentations

132
Automatic Generation of Tests

• Customers
• Software developers
• Software end users, e.g., web surfers
• Security administrators
• Researchers in software testing methodologies
• Impact
• Reduce the cost of software testing (50 of
  development costs)
• Improve software quality
• Reduce security incidents (75 due to bugs)
• Ford and Argus interested in commercialization
  of ATG methodology
• Strongly endorsed by VCAT

133
Software Testing by Statistical
Methods(Competence Project)

• Objective develop new methods for software
  testing based on stochastic processes and
  metrics, in order to
• Improve the quality of software development
• Develop reliability estimates that software
  correctly adheres to its specification
• Joint project
• Software Diagnostics and Conformance Testing
  Division (897)
• Statistical Engineering division (898)
• Collaboration with other NIST Labs (MEL, EEEL,
  CSTL)

Project Team Len Gallagher Lynne Rosenthal
134
(No Transcript)
135
Specification
Conformance Tests
136
FY00 Progress

• Published paper on Coverage Model of Mutation
  Testing (with Paul Black)
• Produced paper on applying statistical methods to
  component based testing
• Produce paper on Test Input Needed to Improve
  Reliability
• Analysis of effort to apply statistical method
  (partitioning) to new test development

137
FY01 Plans

• Continue effort to apply statistical method
  (partitioning) to new test development
• Apply statistical method to pervasive computing
  applications

138
FY00 Resources

• 1 FTE
• 200K (STRS-Competence, plus 100K given
  to ITL/SED)
• FY01 Resources
• 1 FTE
• 200K competence funding (plus 100K given to
  ITL/SED)

139
Software Testing by Statistical Methods

• Customers
• Conformance Test Developers
• Ourselves
• Users of conformance tests
• Impact
• Improve our knowledge and skills in designing and
  developing quality software.
• Quantitative measures associated with conformance
  testing.

140
Quantum Information
Project Team Paul Black Neva Carlson
141
Quantum Information

• Goal to develop information representations and
  computing machines that take advantage of quantum
  effects
• Joint effort of physicists and computer scientists

142
FY00 Progress

• Invited to develop joint proposal for
  Nanotechnology Initiative with Physics
• Invited by NIST Physicists to present Basic
  Concepts in Theory of Computation
• Developed Quantum Information Demo
• Developed Quantum Information Bibliography
• Attended weekly Physics Quantum Information
  seminar

143
FY01 Plans

• Develop products for Nanotechnology Quantum
  Information project
• Organize Workshop on Computer Science in Quantum
  Information
• Research questions in testing Quantum Algorithms

144
FY00 Resources

• .5 FTE
• 100K (STRS Physics Lab?)
• FY01 Resources
• 1.5 FTE (depends on Nanotechnology Initiative or
  from Physics)
• Equipment 0
• Travel anticipated

145
Quantum Information

• Customers
• Researchers in computational theory, and
  algorithm design and analysis
• Developers of large computer models, e.g.,
  global weather
• Cryptanalists
• Impact
• Computing machines of unprecedented
  capability and small size
• Solutions to intractable computational
  problems, e.g., factoring, searching
• ITL staff invited to present at Physics seminars
  and participate in Physics Lab efforts

146
Strategy

• Conformance Tests
• Reference Implementations
• Standards developed jointly with industry
• Diagnostic Tests
• Research to reduce costs of testing
• Reference Data

147
Error, Fault, and Failure (EFF)Data Collection
and Analysis
Project Team Dolores Wallace Michael Koo Tony
Cincotta Jason Rager
148
Error, Fault, and Failure (EFF)Data Collection
and Analysis

• Repositories of reference data on software
  failures
• Tools to create and manage repositories
• Tools to analyze software failure data
• Research in taxonomies to characterize software
  failures

149
FY00 Progress

• Repository systems
• Completed public repository with predefined
  queries
• Completed internal repository with dynamic
  queries
• Data 1000 data points added
• Paper on failures in medical devices finalized
• Papers report on NASA data submitted to NASA
• Project was sunset

150
FY00 Resources

• 1.5 FTE
• 300K (STRS, ATP - 160K)
• FY01 Resources
• 0 (Project sunset)

151
Error, Fault, and Failures

• Customers
• Software Developers
• Researchers in Software Development
  Methodologies
• Impact
• Identifies the stages in the software life cycle
  whose practices can be improved
• Endorsed by the Software Division of the
  American Society for Quality
• Project methodologies and tools used in
  graduate programs at Johns Hopkins and the
  University of Louisville
• EFF Tool used by RST Corporation

152
Other Agency Projects

• NIJ - Software and Tool Verification
• Project Team Gary Fisher, Len Gebase, Jim Lyle,
  Jackie Schneider, Mark Zimmerman
• VA - Health Care Information Systems
• Project Leader Bill Majurski
• GSA - EC Standards Roadmap
• Project Leader Mary Brady
• EPA - Metadata Descriptions and Registries
• Project Leader Judy Newton

153
NIJNational Software Reference Library Beta
Testing
154
NIJNational Software Reference Library Beta
Testing
155
The NSRL BIG Picture
Requirements Funding and Feedback
SW Manufacturers
NIJ
MOU
Contact
Software
Law Enforcement Agencies
NIST
NSRL
Admin.
Research and Production
Use
NSRL DB
File Signatures
156

• National Software Reference Library
• GOAL Reference Data Set of file signatures
  (hash) of COTS file
• Used during examination of digital evidence to
  identify pertinent files and eliminate others
• BOTTOM LINE One computer may have 5,000-20,000
  files. NSRL data set can eliminate 40-95 of
  those files and save hundreds of staff-hours.
• Beta Testing Project
• GOAL Verification of computer forensic tools and
  their use in analyzing digital evidence
• Used to ensure tool usage yields objective,
  repeatable, reproducible, results that hold up in
  court.

157
FY00 Progress

• Developed NSRL data dictionary and database
  structure for hash codes
• Established contacts with Microsoft, Adobe, et al
  to supply software
• CBD announcement, Press release
• Identified hash algorithms
• Creating Reference Data Set (i.e, file
  signatures)
• Developed draft test plan for beta testing
• Develop procedures and tests for imaging tools

158
FY01 Plans

• Establish the reference data set and distribution
  mechanism
• (SQL database with web interface)
• Develop test metrics for efficacy of hash codes
  used in the reference data set
• Test the NSRL web site and make it available to
  approved users
• Continue to develop tests and procedures for
  forensic tools
• Issue test reports and test suites (publish on
  web site)

159
FY00 Resources

• 3.5 FTE
• 700K (OA-NIJ)
• FY01 Resources
• 3.5 FTE
• Travel required for training courses and
  presentations

160
NIJ

• Customers
• Law Enforcement Agencies (e.g., FBI, Treasury,
  DCFL, state and local agencies)
• Software companies (e.g., Adobe, Microsoft)
• Impact
• Improve the efficiency of forensic examination of
  suspect computer files/date (NSRL data set can
  eliminate 40-95 of suspect files and save
  hundreds of staff-hours).
• Provide testing procedures for forensic software
  tools to ensure the analysis process and results
  holds up in court

161
Healthcare Information Systems
Project Team Bill Majurski Tony Cincotta Roger
Gima John Barkley
162
Objectives

• Improve the quality of Healthcare Information
  Systems by
• Designing distributed models and architectures
  for Healthcare Information Systems
• Developing reference implementations
• Technology transfer of RBAC
• Participating in the development of standards,
  e.g., OMGs RAD, G-8 Healthcare Data Card Project
• Support from Veterans Affairs and NIST ATP

163
NIST Role

• Consulting Architect to VA (major healthcare
  provider 170 hospitals)
• Assess new technologies
• Developed design for RPC (Remote Procedure Call)
  Broker
• Designed and implemented Enterprise Single
  Sign-On (ESSO)
• Designing and implementing Inter-Organizational
  Role Based Access Control (IORBAC) authorization
  mechanism

164
FY00 Progress

• Enterprise Single Sign-On (ESSO) beginning
  national trials
• Inter-Organizational RBAC (IORBAC) V1 Reference
  Implementation completed (to be used for
  requirements development)
• OMG Resource Access Decision Facility (RAD)
  proposed for IORBAC
• NIST designs become permanent part of VA HIS
  Architecture
• Now serve as permanent consultant to VA
  Architecture Design Group
• Invited to speak at two VA HIS Conferences

165
FY01 Plans

• Implement IORBAC Reference Implementation V2
  which includes RAD
• Technology transfer of ESSO to VA developers
• Review G8 Healthcare Smartcard Specification and
  provide advice on conformance testing

166
FY00 Resources

• 2.5 FTE
• 500K (OA-VA, ATP-200K)
• FY01 Resources
• Personnel 3 FTE
• Equipment 20K
• Travel required to participate in Architecture
  Design meetings and deployment of implementations

167
Health Care Information Systems

• Customers
• HIS developers and users
• Standards bodies developing standards in HIS
• Developers of authentication and access control
  mechanisms for protecting the privacy of patient
  information
• Impact
• Higher quality HIS and improved patient privacy
• NIST designs permanent part of VA HIS
  Architecture
• Remote Procedure Call (RPC) Broker in use in all
  VA Hospitals
• ESSO selected for national trails
• RAD adopted by OMG, implemented by 2AB, DASCOM

168
GSA - EC Standards Roadmap
169
FY00 Progress

• Developed initial report on scope of effort,
  standards of interest, and design of web-based
  roadmap
• Develop initial prototype of roadmap
• Coordinate activities with appropriate GSA and
  CIO groups

170
FY01 Plans

• Complete interactive standards roadmap
• Pursue with MEP, deployment of roadmap for SMEs

171
FY00 Resources

• 0.5 FTE
• 75K (OA-GSA)
• FY01 Resources
• 1 FTE
• GSA Funding required

172
EPA - Metadata Registries
173
FY00 Progress

• Advance the standards effort in the ISO/NCITS
  arena by continuing to serve as a member of
  ISO/IEC SC 32 WG 2 and Chair of NCITS L8 SIG
• Prepare new proposed standards and technical
  reports for Metadata Registries (MDR).
• Coordinate and encourage implementations of
  metadata registries based on those standards by
  hosting the Metadata Registry Implementers
  Coalition (MDRIC).
• Advance progress on the EDEN project by COTR
  activities.
• Publicize activities through presentations and
  publications in national and international fora.

174
FY01 Plans

• Continue the work shown above under FY00
  Progress.
• Serve as Editor of Part 5 of ISO/IEC 11179 which
  is up for 5-year review.
• Serve as Editor of the Technical Report
  Procedures for Achieving Metadata Registry
  Content Consistency, which is to be balloted for
  CD.

175
FY00 Resources

• 1 FTE
• 200K (OA-EPA)
• FY01 Resources
• 1 FTE
• Travel required to participate in ISO meetings

176
Industry Testimonials

• The OASIS-NIST XML Conformance Test Suite is
  critical for our industry.
• Norbert Mikula, CTO, OASIS
• N