Lessons from the Crisis: Enterprise Software Lifecycle Optimization

1
Lessons from the Crisis Enterprise Software
Lifecycle Optimization
Dr. Sergey V. Zykov, Ph.D. State University
Higher School of Economics
The 2nd International Conference on Complexity,
Informatics and Cybernetics (IMCIC'11) March
2730, 2011, Orlando, Florida, USA
2
Lessons from the Crisis Enterprise Software
Lifecycle Optimization

• Heterogeneous ESE a challenge, esp. under the
  crisis
• The methodology combines formal models (incl.
  modified spiral model) and SDK for class-level
  association-based relationships
• Problem domain features
• heavy data burden - in 2005 total data size of
  Intel Corp. exceeded 3.2 petabytes (over 120,000
  employees in 57 countries)
• - high object classes complexity
• - incomplete information on the structure of
  certain instantiations of the classes
• - the set of class attributes and operations can
  be determined rigorously.
• Reasons for methodology application (besides
  crisis)
• - variety of heterogeneous classes,
• - importance of association-based inter-class
  relationships
• - class inference possible even under certain
  of weak-structured class instances

The 2nd International Conference on Complexity,
Informatics and Cybernetics (IMCIC'11) March
2730, 2011, Orlando, Florida, USA
3
Lessons from the Crisis Enterprise Software
Lifecycle Optimization

• Software production crisis and SE era advent
• 1960-s the software production crisis begins
• Problems/Disproportions
• - anarchic SDL vs. growing system complexity
• - no SDL methodologies (except build-and fix
  approach)
• - the crisis is in our minds, not just in
  economics
• - an adequate (RD-based, adaptive) SDL
  methodology required!
• SE the sci--tech discipline to overcome the
  crisis
• SE development stages
• - 1960-s hand-made art unique and precious
  masterpieces
• - 1970-s manufacturing mission-critical
  systems
• - 1990-s machine-tool conveyor CASE team
  development

The 2nd International Conference on Complexity,
Informatics and Cybernetics (IMCIC'11) March
2730, 2011, Orlando, Florida, USA
4
Lessons from the Crisis Enterprise Software
Lifecycle Optimization

• Comparing SDL and material production
• 1967 - The NATO Conference on SE
• Q. Can SW be constructed/produced as a
  physical/material object?
• Analysis
• - step-by-step elaboration
• - maintenance (bridges may cost 0 for years, not
  SW)
• - intellectual degeneration (complex SW
  platforms change fast)
• - prototype reliability
• - brute force / bulletproof approach (a
  bridge twice thicker)
• - residual faults (NASA flight simulation)
• Conclusions
• - SW production is similar to material one in
  certain aspects
• - SW production is entirely different from
  material one in others
• A. NO, Lifecycles are fundamentally different

The 2nd International Conference on Complexity,
Informatics and Cybernetics (IMCIC'11) March
2730, 2011, Orlando, Florida, USA
5
Lessons from the Crisis Enterprise Software
Lifecycle Optimization

• Optimizing lifecycle the answer to the crisis
• Every SSDL stage can be optimized (analysis,
  req.spec., design, implementation, maintenance,
  etc.)
• Documenting should be optimized as well
• Optimization basis models methods tools
  integration
• - discrete metrics (residual faults, KLOC, etc.)
  heuristics
• - practical applicability vs. mathematically
  single-best solution
• SSDL model basic features
• - iterative nature
• - sequential elaboration
• - incremental development
• - risk analysis
• - modified incremental/spiral model
• - best applicable for enterprise-level projects
  (CASE risks)

The 2nd International Conference on Complexity,
Informatics and Cybernetics (IMCIC'11) March
2730, 2011, Orlando, Florida, USA
6
Lessons from the Crisis Enterprise Software
Lifecycle Optimization

• Heterogeneous ESE a challenge, esp. under the
  crisis
• The methodology combines formal models (incl.
  modified spiral model) and SDK for class-level
  association-based relationships
• Problem domain features
• heavy data burden - in 2005 total data size of
  Intel Corp. exceeded 3.2 petabytes (over 120,000
  employees in 57 countries)
• - high object classes complexity
• - incomplete information on the structure of
  certain instantiations of the classes
• - the set of class attributes and operations can
  be determined rigorously.
• Reasons for methodology application (besides
  crisis)
• - variety of heterogeneous classes,
• - importance of association-based inter-class
  relationships
• - class inference possible even under certain
  of weak-structured class instances

The 2nd International Conference on Complexity,
Informatics and Cybernetics (IMCIC'11) March
2730, 2011, Orlando, Florida, USA
7
Lessons from the Crisis Enterprise Software
Lifecycle Optimization

• The methodology vs. ontology-based approaches
  (OBA)
• OBA (e.g. Cyc) efficiency is comparable only
  under a total class-level uncertainty, which is a
  different problem domain than ECM
• Thesaurus needed for the OBA to meet the
  relevance required
• The methodology uses similar foundations and
  tools as OBA (UML and XML-based tools, predicate
  calculus-based CycL, conceptual model etc.) for
  data modelling and integration
• OBA lack a balanced combination of formal models
  and industry-level SDKs (incl. visualization) for
  ECM lifecycle, resulting in low scalability and
  non-suitability for the major enterprise-level
  tasks

The 2nd International Conference on Complexity,
Informatics and Cybernetics (IMCIC'11) March
2730, 2011, Orlando, Florida, USA
8
Lessons from the Crisis Enterprise Software
Lifecycle Optimization

• Objective, tasks, theoretical background
• Objective to make a software development
  methodology, which supports entire lifecycle of
  the enterprise software in the global
  computational environment
• Tasks
• - formalizing stages and levels of the
  methodology
• - mathematical modeling
• - creating CASE- and RAD-tools
• - implementing the methodology
  (prototype, full-scale).
• Background finite sequence, category,
  computation (D.Scott), semantic networks.

The 2nd International Conference on Complexity,
Informatics and Cybernetics (IMCIC'11) March
2730, 2011, Orlando, Florida, USA
9
Lessons from the Crisis Enterprise Software
Lifecycle Optimization

• Innovations the integrate methodology includes
  
• a set of data models for problem domain objects
  and for computational environment (CM, AMCM)
• algorithm of the new component integration into
  the software implemented
• personalization procedure for enterprise content
  access
• SDKs ConceptModeller, Content Management System

The 2nd International Conference on Complexity,
Informatics and Cybernetics (IMCIC'11) March
2730, 2011, Orlando, Florida, USA
10
Lessons from the Crisis Enterprise Software
Lifecycle Optimization

• EDW problems solved by the methodology

The 2nd International Conference on Complexity,
Informatics and Cybernetics (IMCIC'11) March
2730, 2011, Orlando, Florida, USA
11
Lessons from the Crisis Enterprise Software
Lifecycle Optimization

• Enterprise software lifecycle support by the
  methodology

The 2nd International Conference on Complexity,
Informatics and Cybernetics (IMCIC'11) March
2730, 2011, Orlando, Florida, USA
11
12
Lessons from the Crisis Enterprise Software
Lifecycle Optimization

• Problem domain modeling
• Data object modeling
• class ? object ? value
• Class collection of data objects of the
  integrated problem domain
• Object class instantiation by CMS template
• (metadata partial evaluation)
• Value static HTML page generated by CMS (full
  evaluation).
• Benefits
• - evolves from the object-oriented approach
• - develops the existing models
  (V.E.Wolfengagens CM et al.)
• in relation to global computational
  environment

The 2nd International Conference on Complexity,
Informatics and Cybernetics (IMCIC'11) March
2730, 2011, Orlando, Florida, USA
13
Lessons from the Crisis Enterprise Software
Lifecycle Optimization

• Modeling classes of data objects
• Classes C of problem domain data objects are
  modeled by domains
• C IwD ? vD (w(v) ? ?) vD?,
  where
• 1) C and D are in a partial order relation (C ISA
  D)
• 2) ? is a criterion of data object w belonging to
  class C
• from the viewpoint of a problem domain
  expert.
• Class of n-dimensional data objects is modeled
  by an n-arity relation
• Rn Iw V1,..,Vn ?v1V1 ?vnVn (w
  v1,,vn ? ?)
• v1V1 , , vnVn ?,
  where
• ? n-dimensional criterion of data object w
  belonging to class Rn
• Class is a collection of ordered pairs (vi,Vi),
  where
• vi  is its i-th attribute (either of data or of
  metadata) Vi attribute type.

The 2nd International Conference on Complexity,
Informatics and Cybernetics (IMCIC'11) March
2730, 2011, Orlando, Florida, USA
14
Lessons from the Crisis Enterprise Software
Lifecycle Optimization

• From problem domain to computational environment
  (1)
• Under class C instantiation with assignment a1
  and template ?k of CMS HTML page, evaluation of
  the template collection M sets into true value
  its element mi, which index (k) equals the
  template number
• M (m1,, mk,, mN), ?i1,,N mi?0,1
• M?k (m1,, mk,, mN), ??? mi1, ik
  ? mi0, i?k.
• Certain attributes of class metadata v1,,vn are
  evaluated according to ti conditions of ?
  template
• (v1V1,,vnVn)ti (v1?(t1),, vn?(tn))
  (v1V1,,vnVn),
• ?????? V1 ISA V1,, Vn ISA Vn.

The 2nd International Conference on Complexity,
Informatics and Cybernetics (IMCIC'11) March
2730, 2011, Orlando, Florida, USA
15
Lessons from the Crisis Enterprise Software
Lifecycle Optimization

• From problem domain to computational environment
  (2)
• The second assignment a2
• instatiates non-evaluated
• template elements (v1,,vn)
• of CMS HTML-page
• by content values (c1,, cn)
• (v1V1,,vnVn)c (v1/c1,, vn/cn),
• where c1?1,, cn?n,
• and C1 ISA V1,, Cn ISA Vn.
• ?i class template is Ti (i,(t1,,tn)),
• where (t1,,tn) is the vector of the evaluated
  class metadata

The 2nd International Conference on Complexity,
Informatics and Cybernetics (IMCIC'11) March
2730, 2011, Orlando, Florida, USA
16
Lessons from the Crisis Enterprise Software
Lifecycle Optimization

• Formal syntax of the CMS abstract machine
• Let us collect all the CMS abstract machine
  language identifiers into Ide domain, commands
  into Com domain, and expressions into Exp
  domain
• Ide I I identifier
• Com C C command
• Exp E E expression.

The 2nd International Conference on Complexity,
Informatics and Cybernetics (IMCIC'11) March
2730, 2011, Orlando, Florida, USA
17
Lessons from the Crisis Enterprise Software
Lifecycle Optimization

• Formal semantics of the CMS abstract machine (1)
• Order of construction
• standard domains (most often used)
• finite domains (including explicitly enumerable
  elements)
• domain constructors operations of building new
  domains out of existing ones
• composite domain formalization based on standard
  domains and domain constructors.
• Domain constructors
• - functional space D1?D2
• - Cartesian product D1?D2??Dn
• - sequence D
• - disjunctive sumD1  D2 Dn.

The 2nd International Conference on Complexity,
Informatics and Cybernetics (IMCIC'11) March
2730, 2011, Orlando, Florida, USA
18
Lessons from the Crisis Enterprise Software
Lifecycle Optimization

• Formal semantics of the CMS abstract machine (2)
• State  Memory ? Input ? Output
  Memory  Ide ? Value  unbound
• Input  Value
  
• Output  Value
  
• Value  Type1  Type2  
• Constant denotate ltvariable, valuegt
• Identifier denotate
• ltvariable_in_memory, identifier, stategt

The 2nd International Conference on Complexity,
Informatics and Cybernetics (IMCIC'11) March
2730, 2011, Orlando, Florida, USA
19
Lessons from the Crisis Enterprise Software
Lifecycle Optimization

• Formal semantics of the CMS abstract machine (3)
• Semantic function for expression
• E Exp ?  State ? Value ? State error
• Semantic function for command
• ? Com?State?Stateerror.
• Semantic statement for identifier
• E I s (m, I unbound) error, ? (m, I, s).
• Semantic statement for assignment command
• C IE E E ?v (m , i, o) . (m v/I, i,
  o).

The 2nd International Conference on Complexity,
Informatics and Cybernetics (IMCIC'11) March
2730, 2011, Orlando, Florida, USA
20
Lessons from the Crisis Enterprise Software
Lifecycle Optimization

• Bi-directional software development
• in ConceptModeller CASE-toolkit

FORMAL LANGUAGE
FORMAL LANGUAGE
NATURAL
FRAME
FORMAL LANGUAGE
OF FRAME
OF DIAGRAMS
TRANSLATION
OF DIAGRAMS
LANGUAGE
DESCRIPTION
(IBM RATIONAL /
INTO UML DIAGRAMS
(XML / RATIONAL)
(XML)
MS VISIO)
VISUALIZATION
VISUALIZATION
FORMALIZATION
TRANSLATION
Mapping
Business situations
Business situations
Visual frame
function from
UML diagrams
in terms of
in terms of
representation
frames to UML
visualization
UML diagrams
natural language
diagrams
IBM RATIONAL,
C.NET
C.NET
C.NET
ORACLE DEVELOPER,
MS VISUAL STUDIO
The 2nd International Conference on Complexity,
Informatics and Cybernetics (IMCIC'11) March
2730, 2011, Orlando, Florida, USA
21
Lessons from the Crisis Enterprise Software
Lifecycle Optimization

• Software Solution Arcitecture

The 2nd International Conference on Complexity,
Informatics and Cybernetics (IMCIC'11) March
2730, 2011, Orlando, Florida, USA
22
Lessons from the Crisis Enterprise Software
Lifecycle Optimization

• CMS logical structure

The 2nd International Conference on Complexity,
Informatics and Cybernetics (IMCIC'11) March
2730, 2011, Orlando, Florida, USA
23
Lessons from the Crisis Enterprise Software
Lifecycle Optimization

• Structure of the integrated enterprise program
  system

The 2nd International Conference on Complexity,
Informatics and Cybernetics (IMCIC'11) March
2730, 2011, Orlando, Florida, USA
24
Lessons from the Crisis Enterprise Software
Lifecycle Optimization

• Comparing the software development methodology
• to the commercial methodologies available

Mathematical model
Methodology
Integrated methodology
The 2nd International Conference on Complexity,
Informatics and Cybernetics (IMCIC'11) March
2730, 2011, Orlando, Florida, USA
25
Lessons from the Crisis Enterprise Software
Lifecycle Optimization

• Implementation features comparison

Software
Integration with legacy information systems
Integrated ERP information system reports
Multi-language publication
Smart, template-based design
Complex data object embedding
UML diagrams
WYSIWYG-mapping
Java servlets
.NET web services
ITERA CMS
The 2nd International Conference on Complexity,
Informatics and Cybernetics (IMCIC'11) March
2730, 2011, Orlando, Florida, USA
26
Lessons from the Crisis Enterprise Software
Lifecycle Optimization

• TCO comparison results

/person
CMS ConceptModeller
The 2nd International Conference on Complexity,
Informatics and Cybernetics (IMCIC'11) March
2730, 2011, Orlando, Florida, USA
27
Lessons from the Crisis Enterprise Software
Lifecycle Optimization

• ROI comparison results

, yrs
CMS ConceptModeller
The 2nd International Conference on Complexity,
Informatics and Cybernetics (IMCIC'11) March
2730, 2011, Orlando, Florida, USA
28
Lessons from the Crisis Enterprise Software
Lifecycle Optimization

• Implementation terms comparison results

Optimistic scenario Pessimistic scenario
CMS ConceptModeller
The 2nd International Conference on Complexity,
Informatics and Cybernetics (IMCIC'11) March
2730, 2011, Orlando, Florida, USA
29
Lessons from the Crisis Enterprise Software
Lifecycle Optimization

• Theoretical results
• 1) A system of formal models for problem domain
  and computational environment (rigorous
  semantics, entire lifecycle support, content
  management orientation)
• 2) Algorithm of integrating new components to the
  enterprise software system (problem-oriented,
  heterogeneous software architecture support)
• 3) personalization procedure for accessing
  enterprise content
• (flexible, reliable)

The 2nd International Conference on Complexity,
Informatics and Cybernetics (IMCIC'11) March
2730, 2011, Orlando, Florida, USA
30
Lessons from the Crisis Enterprise Software
Lifecycle Optimization

• Engineering results
• 1) CASE- and RAD-toolkits
•   a)  ConceptModeller (rigorous semantics
  compatible to up-to-date CASE-tools, ERP and ?
  legacy systems re-engineering XML/BPR/UML
  standard support)
• b) ITERA CMS (rigorous semantics rapid
  publishing of complex content WYSIWYG interface
  office products integration).
• 2) Architecture (environment unification of
  heterogeneous enterprise applications role
  personalization with situation dynamics)

The 2nd International Conference on Complexity,
Informatics and Cybernetics (IMCIC'11) March
2730, 2011, Orlando, Florida, USA
31
Lessons from the Crisis Enterprise Software
Lifecycle Optimization

• Practical value of the results obtained
• 1) implementation term-and-cost reduction
• (TCO, ROI) as compared to commercially
  available software by 30 (average)
• 2) major enterprise software features
  improvement
• - scalability
• - reliability
• - ergonomics.

The 2nd International Conference on Complexity,
Informatics and Cybernetics (IMCIC'11) March
2730, 2011, Orlando, Florida, USA
32
Lessons from the Crisis Enterprise Software
Lifecycle Optimization

• Research results approbation
• Over 30 presentations on international
  conferences,
• 4 books and over 50 papers
• Research grants MSR (2002-03), RFBR (1996-2006)
• and HSE (2008-11).
• ITERA implementation (150 companies, 10,000
  employees)
• CMS (2002) Internet-portal (2003)
  Intranet-portal (2004)
• Other implementations ICP (RAS), Sterkh
  Foundation,
• Ashihara Karate Association, Russian Orthodox
  Church, etc.
• Curricula (HSE, MEPhI, MSUFI, INTUIT, LANIT,
  SoftLine,
• TEKAMA, CareerLab) over 3000 graduates

The 2nd International Conference on Complexity,
Informatics and Cybernetics (IMCIC'11) March
2730, 2011, Orlando, Florida, USA