SelfEvaluating Agile LargeScale Systems

1
Self-Evaluating Agile Large-Scale Systems

• Barry M. Horowitz
• University of Virginia
• May, 2007

2
What is Agility?

• Agile System Agility is relative. A more agile
  system is one that can be modified to respond to
  new opportunities and risks by incorporating
  significant new design features 1) in a shorter
  period of time, and 2) in a more assured manner,
  than a less agile system

3
Adaptive Systems Are Not Necessarily Agile
Systems

• Adaptive System A system that improves its
  performance by dynamically adjusting to specific
  situations based on rules, procedures and
  algorithms that are built into the system.
• While Adaptive Systems can be designed to learn,
  the learning mechanisms are part of the existing
  system design

4
The Economic Boundaries That Demand Agility

• Value of a system opportunity or risk is
  considered as too uncertain to immediately
  implement a major system modification, but
• Potential value of just in time implementation,
  and the risk of competitors implementing
  successful solutions is too great to do nothing

5
Some Capabilities and Features of Agile Systems
(1)

• Managing Time for Developing System Modifications
• Predict future system-related opportunities and
  risks in order to provide more lead time for
  accomplishing design changes that are decided
  upon over time
• Technically structured to reduce the required
  integration efforts for adding new system
  capabilities related to predicted opportunities
  and risks
• Human organization design is conceived to more
  readily permit reorganizations that are related
  to predicted future opportunities and risks

6
Some Capabilities and Features of Agile Systems
(2)

• Managing Confidence in Rapid System Modification
• Automation support for system reconfiguration
• Include fault tolerant designs that allow higher
  confidence in making early operational
  transitions of new designs
• Provide system operators with information that
  increases their confidence during transition
  periods for new system designs

7
The Need for Agility

• Globalization
• Reduce Costs from Infrastucture Systems (Health
  Care, Power, Water, Transportation, etc)
• Businesses in Competition
• National Security and Preparedness Systems
• World Systems

All Systems
8
Agility and Assurance

• Internet
• Very agile
• Unsure reliability
• Unsure Security
• No Performance Assurances
• Large Infrastructure Systems (e.g., ATC System)
• Very Assured Designs
• Assured Testing and Evaluation Processes
• Not Agile (Slow to change)

9
Difficulties in Developing Large-Scale Systems
That Can Respond to Emerging Opportunities and
Risks

• Large systems take a long time to build Lots of
  external and internal changes occur even during
  the initial development time, creating difficult
  to manage instabilities in development efforts
• Large systems cost a lot Need to last a long
  time and lots of things will change during that
  life time, so how much and what flexibilities for
  change should be provided in the design of
  systems
• Large systems involve many direct stakeholders
  Their desires change while developing and
  operating a system, and their opportunity and
  risk assessments can be widely different, so how
  can agreements on priorities be stabilized

10
Current System Development Methodologies

• Issue
• Instabilities in Desired Capabilities
• Flexibility to Change
• Differing Stakeholder Opportunity and Risk
  Assessments

• Methodology
• Fix System Requirements, Budgets and Schedules
• As Technology Readily Permits, Within
  Budget/Schedule
• Dont Start Until Resolved, and Only Change After
  Major Issues Arise That Stimulate New
  Budget/Schedule

11
Results From These Methods Are Not Good

• New systems dont do whats desired even at
  initial delivery because things changed
• Cant readily change the systems because they are
  point-designed
• Cant readily integrate them with new systems
  because they are point-designed
• Significant delays in starting new system
  developments or significantly modernizing
  existing systems due to uncertainties that must
  be resolved prior to starting

12
Extreme ExampleComanche Helicopter

• Unprecedented System
• Stealth
• Multi-sensor/Situation Awareness
• Closely Controlled and Directed
• Army
• Integrated Army/Boeing System Development Team
• Started in 1983 Canceled in 2004 (6.2bb)
• Reasons
• Changing Needs (Cold War War on Terrorism)
• Unanticipated Cost Growth per Helicopter
• Emergence of Unmanned Aircraft for Surveillance

Can major redirections be initiated at earlier
points in time by continuously forecasting the
systems future and having methods for acting on
those forecasts?
13
Four Fundamental Parts to a Proposed Solution Path

• Forecasting System Opportunities and Risks
• Self Evaluation Built-in sub-systems for
  measurement and analysis at multiple levels of a
  system, with the purpose of tracking the growth
  of potential opportunities and risks
• Multi-Scale System Opportunity and Risk Analysis
  Significant opportunities and risks can emerge
  from multiple levels of a complex system
• Agility Solutions Technical and human
  organizational designs established with
  objectives for seizing emerging opportunities and
  reducing emerging risks

14
Relationship Between Self-Evaluation, Agility and
Expected ROIs

• The shorter the prediction time for opportunities
  and risks the more confident predictions are
  likely to be, and the more sure the ROI for
  investments in opportunities and risks
• But, short prediction times require more agility,
  since they leave less time for the required
  changes to seize opportunities and avoid risks.
  If agility cost increases, expected ROIs are
  reduced
• Expected ROI for opportunities and risks will be
  driven up by the shortened prediction time and
  down by the increased cost in agility. Evaluation
  of this ROI trade-off provides a basis for
  selecting look ahead times and agility levels

15
Reducing the Pressure on Forecasting

• When many realistic opportunities and risks are
  possible over the same time horizon
• When an agile system solution (Technological and
  Organizational) can be developed to help speed up
  reconfiguration for all or many of the
  possibilities
• THEN
• Single events no longer dominate the likelihoods
  for success

16
SEALS Design Framework
Multi-Scale Analysis
Self- Evaluation
Agility

• Agile Technical System Architecture and Design
• Agile Human Organizational Architecture and
  Design
• Fault Tolerant Architecture and Design

New System Features
Needed New Operational Capabilities

• Measurements
• Internal
• External

Built-in Measurement And Analysis Sub-Systems
For Emerging Opportunities And Risks
Operational System
Multi-Scale Opportunity Risk Analysis
Needed New Measurement Capabilities
17
Some Research Starting Points

• Multi-Scale Forecasting and Modeling
• Hierarchical Holographic Modeling/Phantom Systems
  
• Collective Intelligence
• Systems Biology Inspired Multi-scale Modeling
• Economic Models
• Game Theory and Real Options
• Technology
• Integration Technology
• Fault Tolerant Technology
• Systems Biology Inspired Architectures
• Human Organization Modeling and Analysis
• Network and Agent-Based models
• Sense Making in Complex Systems