Tomorrows Needs, Yesterdays Technology DoDs Architectural Dilemma

1
Tomorrows Needs, Yesterdays Technology
DoDs Architectural Dilemma Bold Initiatives
2007

• Dr. Raymond A. Paul
• C2 Policy
• OASD NII
• Raymond.Paul_at_osd.mil

2
Outline of Talk

• Key propositions in this talk
• Observations on DoD architectures
• Dynamic Service-oriented architecture (SOA) and
  its applications
• Dynamic architecture for service-oriented
  Applications
• Dynamic service-oriented system engineering
• Conclusion

3
Key Propositions in this Talk

• Network Centricity poses fundamentally new
  architectural requirements on DoD systems
• Existing architectural approaches (methods,
  tools) are not sufficient to the task, leaving
  some gaping holes
• Extension of existing approaches is fundamentally
  incorrect, since
• it adds yet another patch up, and increases
  interoperability complexity, which is one of the
  key problems that needs to be addressed
• makes existing complex systems more so, making it
  even more difficult to understand use them
• Any new architectural approach must
• lay primary emphasis on unique characteristics of
  Network Centricity
• be discriminating in deciding what to include in
  the approach critical if the goal is to achieve
  any measure of analyzability

4
Observations on DoD Architectures
5
Definition of SOA

• A Service-Oriented Architecture (SOA) is a system
  consisting of modular software components with
  standardized component-access and interfaces that
  are independent of any platform or implementation
  technology. An SOA is simply a collection of
  standardized services on a network that
  communicate with one another.
• Web services does not equal SOA. Web services is
  a collection of technologies, including XML,
  SOAP, WSDL, and UDDI that allow service
  programming.
• SOA is an application architecture within which
  all functions are defined as independent services
  with well-defined invokable interfaces. Note
  that
• All functions are defined as services.
• All services are independent.
• In the most general sense, the interfaces are
  invokable

6
Advantages of SOA

• First, interoperability in SOA is priority one
  due to its system organization. If changes are
  needed, just replace one service with another,
  thus SOA-based systems are tolerant on changes
  and system can be easily reconfigured.
• Second, SOA-based system is also agile, system
  development is in terms of system composition
  rather than code development.
• SOA also changed the entire system development
  paradigm including system requirements, design,
  implementation, testing, operation and
  maintenance.
• For example, in system requirements, knowledge of
  existing services is equally important as
  understanding the problem.

7
SOA Dynamic Discovery

• Dynamic discovery is an important part of SOA. An
  SOA is composed of three service providers,
  service consumers, and a directory service. The
  role of providers and consumers are apparent, but
  the role of the directory service needs some
  explanation. The directory service is an
  intermediary between providers and consumers.
  Providers register with the directory service and
  consumers query the directory service to find
  service providers. Most directory services
  typically organize services based on criteria and
  categorize them. Consumers can then use the
  directory services' search capabilities to find
  providers. Embedding a directory service within
  SOA accomplishes the following
• Scalability of services you can add services
  incrementally.
• Decouples consumers from providers.
• Allows for hot updates of services.
• Provides a look-up service for consumers.
• Allows consumers to choose between providers at
  runtime rather than hard-coding a single
  provider.

8
A Typical Architecture for a Service-Oriented
Application

• Like any distributed application,
  service-oriented applications are multi-tier
  applications and have presentation, business
  logic, and persistence layers. The two key tiers
  in SOA are the services layer and the business
  process layer.

9
SOA Architectural Perspective

• The following three SOA architectures need to be
  considered
• The Application Architecture. This is the
  business facing solution which consumes services
  from one or more providers and integrates them
  into the business processes.
• The Service Architecture. This provides a bridge
  between the implementations and the consuming
  applications, creating a logical view of sets of
  services which are available for use, invoked by
  a common interface and management architecture.
• The Component Architecture. This describes the
  various environments supporting the implemented
  applications, the business objects and their
  implementations.

10
DoD Architecture and UML

• DoD has started several architecture initiatives
  such as DODAF (numerous views such as OV) and in
  some degree GIG can be considered as an
  architecture initiative
• DoD also is a heavy user of UML in specifying
  system behaviors (sequence diagrams, class
  diagrams, use cases, collaboration diagrams) and
  architecture
• Behavior diagrams
• A type of diagram that depicts behavioral
  features of a system or business process. 
• This includes activity, state machine, and use
  case diagrams as well as the four interaction
  diagrams
• Interaction diagrams
• A subset of behavior diagrams which emphasize
  object interactions
• This includes communication, interaction
  overview, sequence, and timing diagrams
• Structure diagrams
• A type of diagram that depicts the elements of a
  specification that are irrespective of time
• This includes class, composite structure,
  component, deployment, object, and package
  diagrams

11
These are excellent, but

• These architecture initiatives provide much
  innovation for DoD, help to ensure system quality
  in numerous aspects such as requirement
  specification, design, implementation, and
  testing.
• As an universal and common language, UML provides
  a common vocabulary between different stake
  holders such as program managers (PMs), system
  engineers, QA, and system architect.
• And yet
• They are not sufficient for modern agile
  warfighting, and
• Leave significant unmet needs

12
Observations

• Recent trends in Network-Centric Warfare (NCW)
  have significant effect on our technology and
  management
• GIG-compliant including policy-driven computing
• Service-Oriented Architecture (SOA)
• Network Centric Enterprise Services (NCES),
  GIG-ES, JBMC2, FCS and Composable FORCEnet
• Dynamic publication, runtime selection and
  discovery, dynamic composition
• Distributed services and agents
• These systems must be dynamic systems and keep on
  changing even at runtime
• These architectures are not static but dynamic
  and evolving
• in other words, modern DoD systems need to
  respond to change at runtime and in real time
• Key question
• Are the existing approaches to architecture
  powerful enough to handle these dynamic structure
  and mechanisms, which are key to building systems
  for Network Centric Warfare?

13
Observations (continued)

• Rapid and adaptive acquisition and deployment for
  NCW
• Agile and adaptive acquisition (Income-Tax model
  for adaptive and incremental acquisition)
• Agile warfighting with dynamic changing tactics
• Dynamic system architecture composed at runtime
• Dynamic system reconfiguration or re-composition
  at runtime even during warfighting
• Rapid but reliable system engineering
• High assurance for C2 applications
• Dynamic and real-time system interoperability
  between two systems not knowing each other before
• Key question
• Is existing technology powerful enough to address
  the NCW issues identified above?

14
Network Centric Enterprise Services(NCES)
Support real-time near-real-time warrior needs
and business users
Community-of-Interest (COI) Capabilities
Users
Levels of Services above core level
Comms Backbone
Core Enterprise Services (CES)
Messaging
ESM
Mediation
Security/IA
User Asst
Discovery
Collaboration
App
Storage
15
GIG Enterprise Services
SOADR Supports real-time near-real-time warrior
needs
DoD (Title 10)
IC (Title 50)
16
Dynamic SOA and its Applications
17
Service Computation Model

• Three parties are involved
• Service providers
• Have access to design and implementation, and the
  interface Web Service Definition Language (WSDL)
• May themselves host services
• UDDI/ebXML
• Provide searching and updating capabilities
• May have access to WSDL only
• Clients
• Customer, may not have access to design and
  implementation
• May have access to WSDL only

18
Traditional SOA Architecture Diagram
Directory services
White page
Directory services
White page


UDDI / WSDL / SOAP
UDDI / WSDL / SOAP
Service brokers
Service brokers
Yellow page
Yellow page
ebXML / CPP
ebXML / CPP
Programming
Programming
Ontology
Ontology
Registry
Registry
Registry
Registry
languages
Green page
languages
Green page
C, C
C, C
Found
Found


Java
Java
Internet
Internet


Publishing
Publishing
Find

Find





Service providers
Service providers
Computing service
Computing service
Application builder
Application builder


development
development
SOAP call
SOAP call
Applications
Applications
.Net (Microsoft)
.Net (Microsoft)
Service agents
Service agents


Results
Results
WebSphere (IBM)
WebSphere (IBM)
Application development platforms
Application development platforms
Web and data service development
Web and data service development
Specification languages .Net, WebSphere,
Specification languages .Net, WebSphere,
XML, RDF, OWL, WSDL
XML, RDF, OWL, WSDL
WSFL, BPEL, PSML for composition, code generation
WSFL, BPEL, PSML for composition, code generation
19
Dynamic SOA search invocation

• Instead of static composition (with dynamic
  objects and dynamic binding) in the
  Object-Oriented (OO) approach, SOA allows dynamic
  composition at runtime, requiring knowledge of
  the service interfaces only. This includes,
• dynamic service discovery and matching
• runtime ranking and selection of services
• runtime collaboration establishment
• runtime interoperability verification

20
Dynamic SOA failure resilience

• In case of system failure, the SOA
  reconfiguration strategy is unique
• In OO, it is necessary to develop the
  reconfiguration strategy manually
• In SOA, a faulty service can be easily replaced
  by another standby service by the DRS (Dynamic
  Reconfiguration Service), and the DRS also is a
  service that can itself be monitored and replaced
• The key is that each service is independent of
  other services, and thus replacement is natural
• Only the affected services will be shut down and
  this allows the mission-critical application to
  proceed with minimum interruption
• Thus, SOA improves reliability of systems and
  systems of systems

21
SOA Dynamism Changes the Entire Lifecycle

• From requirements engineering to VV, SOA and WS
  changes the landscape
• In the requirements stage, knowledge of existing
  services is critical as reusability is often the
  key enabler
• In the design stage, the loosely coupled service
  architecture allows dynamic composition, and
  dynamic re-composition
• In the implementation phase, a majority of work
  will be composition (or linking) rather than code
  development as services will be reused
• In the VV phase, CVV should be used rather than
  IVV as the source code of many services may not
  be available

22
SOA Requirement Analysis

• Reusability will be a key consideration during
  the requirements phase
• Searching and discovering services that can be
  reused will be key this means that a broker
  or library will be needed
• Internet search technologies, e.g. Google,
  Yahoo!, etc. provide tools to help in this
• Profile and ranking of these services will be a
  key consideration
• The system will assume an architecture from the
  very beginning, i.e., SOA
• From the beginning, it is understood that the
  system components (services) and architecture can
  be changed at any given time, even during runtime
• The requirements phase is continuous and
  considers the evolution plan as new services may
  arrive after the system is deployed
• Once the requirements are fully understood and
  specified, lots of code will be available
  immediately
• this is similar to Extreme Programming or Agile
  processes rather than the Waterfall model
• the difference here is, many services will be
  ready for reuse immediately after the
  requirements analysis.

23
Evaluating Web services at Runtime

• Traditional Shrink wrapped shipped software
  has its own QA process, and integration with
  other software is either limited or impossible
• A Web service is different
• it interacts with other software frequently and
  extensively
• it is necessary to evaluate its quality at
  runtime in real time
• What are attributes of quality for web services?
• Trust
• Reliability the service will not crash
• Performance the service will return results
  rapidly
• Security Privacy the service will not leak
  sensitive data to 3rd parties and it will not
  return false, malicious information back to the
  client
• Safety the service will not harm its users,
  mission and environment
• Need Service Level Agreement (SLA) to ensure
  cooperation

24
Evaluate Reliability of Services at Runtime

• Can we test across inter-organizational web
  services in real time and at runtime?
• Functional testing Can we generate the test
  cases/scripts for inter-organization services?
• Coverage analysis What kind of coverage can we
  anticipate? What would be good enough?
• Test, evaluation and monitoring how can we
  collect and evaluate test results including
  security and scalability test results?
• Can we develop reliability models for web
  services?

25
Collaborative VV for SOA
26
SOA Application Design

• SOA applications must assume the SOA, i.e.,
  clients, brokers, and suppliers linked by dynamic
  discovery and matching, and each service is an
  independent unit for computation and reuse
• However, this is just the beginning

27
Dynamic Architecture for Service-Oriented
Applications
28
SOA Applications Architecture

• SOA applications have their own architectures
• The Application Architecture. The application
  architecture will be built on top of an SOA
• The Service Architecture. This is the commonly
  known SOA architecture
• The Component Architecture. This is the sub-SOA
  architecture that describes the various elements
  that support the implementation of services

29
Layered SOA Application Architecture
A SOA application has its own architecture, such
as a layered architecture similar to conventional
enterprise architecture below.

• 5 layers
• Presentation
• Business Process Choreography
• Services
• Enterprise Components
• Operational Systems
• With 2 supporting mechanisms
• Integration Architecture
• QoS, Security, Management Monitoring

30
NCES and GIG-ES

• Both diagrams show only major components from the
  functionality point of view. Both are rather
  incomplete from the architecture point of view,
  and they can be improved.

31
Dynamic SOA Application Architecture

• As SOA applications may be composed at runtime,
  thus the SOA application architecture may be
  formed at runtime, i.e., the application
  architecture can be dynamic.
• In case of system failures or overload, the SOA
  application may be dynamically changed at
  runtime, and this will result in dynamic
  re-composition and dynamic re-architecture.
• Another new concept for SOA Lifecycle Management
  Embedded in Operation Infrastructure

32
Dynamic Architecture via Dynamic Composition
In SOA, the building blocks are services.
Services may be connected to a bus-like
communication backbone such as an ESB (Enterprise
Service Bus). The control center specifies and
controls the application composition via a
workflow specification, thus different
applications with different architectures can be
composed using the same set of services.
(a) Configuration 1
Generic SOA application architecture
(b) Configuration 2
(c) Configuration 3
33
Dynamic Re-Composition / Re-Architecture

• Dynamic composition occurs during the system
  modeling and assembling stages. SOA applications
  can use the same set of services to compose
  application architectures with different
  topologies
• Dynamic re-composition occurs after the target
  application has been deployed. SOA applications
  can change its architecture, including replacing
  services at runtime. This does not change the
  topology of the application
• Dynamic re-architecture this occurs after the
  target application has been deployed, SOA
  applications can change their architecture
  topology at runtime

34
Traditional redundancy fault tolerant vs. Dynamic
Re-Composition

• Traditional fault-tolerance mechanisms are
  applicable in a fixed architecture
• they can replace a failed component but it cannot
  dynamically change the architecture of the
  application
• in SOA, both are possible
• Another difference is that the traditional
  approach has limited resources (replacements) due
  to the high cost of replication
• in an SOA environment, potentially an unlimited
  number of replacements can be available because
  new replacement services may arrive after
  deployment

35
Lifecycle Management Embedded in Operation
Infrastructure
The SOA lifecycle management can be embedded in
the operation infrastructure to facilitate
dynamic software composition. In this way, the
SOA application development infrastructure and
operation infrastructure can be merged together
in a single and unified SOA infrastructure.

• A development infrastructure may include
  modeling, analysis, design, architecture, code
  generation, verification and validation.
• An operation infrastructure may include Code
  deployment, code execution, policy enforcement,
  monitoring, communication, and system
  reconfiguration.

IBM SOA Foundation architecture
36
IBM SOA Foundation architecture

• Modeling This stage will gather requirements and
  establish a model to represent the system
• Assembling In this stage, the designer can
  either create required services from scratch or
  find an existing service from a service broker to
  develop the application according to the model
  constructed in the previous stage
• Deployment In this stage, the runtime
  environment is configured to meet the
  applications requirements, and the application
  is deployed into that environment for execution
• Management After the application is deployed,
  the services used in the application are
  provisioned and monitored for information needed
  to prevent, isolate, diagnose, and fix any
  problem that might occur at runtime
• Governance and processes The entire process will
  be controlled and orchestrated by policies.

IBM SOA Foundation architecture
37
Microsoft Whitehorse SOA Designer
Microsoft also takes this approach in their
Whitehorse and Indigo projects, where they have
capabilities for modeling, architecture, code
generation, code deployment and code execution.
Microsoft Whitehorse SOA Designer
38
SOA Architecture Classification (SOAC)
Based on four factors

• The structure of the application, either static
  or dynamic
• The runtime re-composition capability. If the
  architecture can support runtime re-composition,
  services in the applications using this type of
  architectures can be replaced by some other
  services to meet changed requirements or fix
  runtime failures
• The fault-tolerance capability. This can be
  further decomposed into two sub-factors
  fault-tolerant control center and/or
  fault-tolerant communication backbone
• The system engineering support

39
SOA Architecture Classification (SOAC)
Architecture classification factor notations
40
SOA Architecture Classification using a tuple

• Structure an application can have S (for static
  structure) or D (dynamic structure), but not
  both
• Re-composition an application can have R (for
  with dynamic re-composition capability) or N (for
  without dynamic re-composition capability), but
  not both
• Fault-tolerance an application can have FB
  (fault-tolerant communication backbone), FC
  (fault-tolerant control services), or FN (no
  fault-tolerance)

Continue on next page
41
SOA Application Architecture Classification

• System engineering support an application can
  have any
• SPC policy checking
• SC completeness and consistency checking
• SM model checking
• SS simulation
• SR reliability assessment
• SG code generation
• SD data collection
• SO service composition/re-composition.
• If it has all the supports, it is denoted with SY
  
• If it has none, it will de denoted as SN
• If it has some of them, it will be denoted as SP

42
SOAC Roadmap
As shown in the diagram, the root of the tree
represents the most basic SOA architecture style.
If we use the notation lt to represent a
containment relationship between two
architectures (D, R, FN, XX) lt (D, R, FB, XX) lt
(D, R, FC, XX) lt (D, R, FB/FC, XX)
43
(D, R, FB/FC, SN) Architecture
This architecture has a fault-tolerant
communication backbone as well as a
fault-tolerant control center. The communication
backbone can be a computing platform, such as
.NET, or ESB (Enterprise Service Bus). The
control center may dynamically change the
application architecture at runtime based on data
collected.
(D, R, FB/FC, SN) Architecture
44
New Approach for Software Architecture

• Software architecture research has traditionally
  focused on component interconnection,
  architecture design styles, architecture
  specification, and architecture design evaluation
• However they have not emphasized the dynamic
  aspect of architecture, particularly when the
  application architecture can be determined at
  runtime
• The classification approach proposed is the first
  attempt to classify various architectural
  approaches for dynamic SOA architectures

45
Dynamic Service-Oriented System Engineering
46
A New System Engineering Approach is Needed

• As SOA applications will be composed at runtime,
  the traditional system engineering may not be
  sufficient to address all the needs of SOA
  applications. We need dynamic Service-Oriented
  System Engineering (SOSE)
• Dynamic SOA reliability engineering
• Dynamic SOA security privacy analysis
• Dynamic SOA safety analysis
• Dynamic SOA collaborative verification and
  validation
• Dynamic SOA system composition and re-composition

47
Dynamic System Engineering

• When a new application is being considered in a
  service-oriented environment, we need the
  following
• Dynamic requirement specification by model
  composition
• Dynamic specification and model analysis
  (performance analysis, simulation, model
  checking, completeness and consistency checking,
  reliability analysis, security analysis)
• Dynamic design by composition and discovery
• Runtime code generation and composition
• Dynamic verification and validation
• Dynamic deployment, execution monitoring and
  assessment

48
Service-Oriented System Engineering

• Dynamic service-oriented requirement engineering
  (model-based, architecture-based, reuse-oriented,
  framework-oriented analysis, simulation-based
  analysis with formal analysis)
• Dynamic service-oriented architecture and design
  (enterprise computing, dynamic collaboration,
  system composition, dynamic system analysis)
• Service-oriented programming languages
  (model-based development, support automated code
  generation
• Dynamic service-oriented implementation (by
  dynamic discovery, composition, and model-based
  architecture, and automated code generation)
• Dynamic testing, verification, evaluation,
  simulation, reliability analysis of services
• Dynamic policy construction, verification,
  simulation, enforcement of security and other
  policies using formal policy languages
• Dynamic System maintenance and update will be via
  service re-composition and possibly architectural
  reconfiguration

49
From Object-Oriented Paradigm to Service-Oriented
Paradigm
OO Modeling Languages IDE
OO Technology Framework
OO Languages
OO system engineering (OOSE) OO testing OO
maintenance OO application frameworks OO
databases (OODB) OO lifecycle (XP? MDA?)
Object- Oriented Concept Architecture
Simula Smalltalk Objective C C Java
UML CORBA MS .Net JDK GCC
50
Service-Oriented System Engineering
51
SOSE Summary

• As DoD embraces SOA, it is important to recognize
  that SOA represents a new computing paradigm, and
  the new paradigm needs a new set of system
  engineering tools and techniques including
  architecture, specification, analysis, modeling,
  simulation, and evaluation.
• The core concept of SOA is dynamism including
  dynamic composition and deployment, and this
  dynamism requires a new approach of system
  engineering different from the traditional static
  documentation driven system engineering.

52
Overall Summary

• SOA - particularly SOC - has to be more adaptive
• Services and processes grow with their
  application and human environments. SOA criteria
  have to change with the situation, circumstance,
  and the environment. These are dynamic
  characteristics. Therefore, SOA has to be
  dynamic and adaptive.
• The new paradigm environment MUST be
  transdisciplinary, fast changing with rapid
  diversification and globalization.

• Summary

53
Conclusion

• Service-oriented computing is making strides due
  to acceptance by government and major computer
  and software companies
• However there are several experience issues we
  need to address
• SOA is related to a number of traditional areas
  such as business models, programming languages,
  model construction and verification, software
  architecture and design, software reusability,
  databases, ontology, autonomic computing, grid
  computing, and computer networks.
• While most of these topics are covered in
  universities, they are often scattered into
  different colleges, we need a definitive and
  systemic TRANSDISCIPLINARY factory for SOA
  research, development and education.
• Regarding SOA research, there is a great need to
  perform dynamic service-oriented system
  engineering such as service-oriented requirement
  engineering, service-oriented design,
  service-oriented model and verification, dynamic
  service verification and validation, dynamic
  service maintenance and re-composition, dynamic
  service security analysis, dynamic service
  reliability analysis, and dynamic service
  profiling and collaboration
• Regarding to architecture, there is a shortage of
  both skilled people who have transdisciplinary
  knowledge for developing and applying SOA and
  software services in a variety of domains such
  as, process control, computing and communication
  infrastructure

54
Conclusion (continued)

• SOA application architecture must be dynamic
  including dynamic composition, re-composition and
  re-architecture at runtime. The dynamic SOA
  application architecture needs a new approach for
  architecture specification, classification, and
  evaluation.
• While the existing DoD architecture and system
  engineering may be able to handle the initial
  specification and design of SOA applications, it
  is necessary to focus on changes and dynamic
  composition (and re-composition), and these will
  change the existing DoD practice of SOA.
  Specifically, DoDAF should be updated to address
  the dynamic aspects of SOA application
  architecture, and it is necessary to develop
  dynamic SOSE (Service-Oriented System
  Engineering) techniques.

55
Thank you for your Attention!

• Ray (New System Engineering Paradigm Evangelist)
  Paul

56
Appendix
57
DoD Architecture and UML

• DoD started several architecture initiatives such
  as DODAF (numerous views such as OV) and in some
  degree GIG can be considered as an architecture
  initiative
• DoD is a heavy user of UML in specifying system
  behaviors (sequence diagrams, class diagrams, use
  cases, collaboration diagrams) and architecture
• Behavior diagrams
• A type of diagram that depicts behavioral
  features of a system or business process. 
• This includes activity, state machine, and use
  case diagrams as well as the four interaction
  diagrams
• Interaction diagrams
• A subset of behavior diagrams which emphasize
  object interactions
• This includes communication, interaction
  overview, sequence, and timing diagrams
• Structure diagrams
• A type of diagram that depicts the elements of a
  specification that are irrespective of time
• This includes class, composite structure,
  component, deployment, object, and package
  diagrams

58
These are excellent, however

• These architecture initiatives provide some
  innovation for DoD, they help to ensure system
  quality in numerous aspects such as requirement
  specification, design, implementation, and
  testing.
• As an universal and common language, UML provides
  a common vocabulary between different stake
  holders such as program managers (PMs), system
  engineers, QA, and system architect.
• And yet
• They are not sufficient for modern 21st century
  agile warfighting, and
• Leave significant unmet needs