Systems Engineering

1
Systems Engineering A Global PerspectiveSystem
s Engineering Seminar Series

• Dinesh Verma, Ph.D.
• Professor and Associate Dean
• Stevens Institute of Technology
• dverma_at_stevens.edu

2
Underlying TONE for this Presentation
(Actually, Cynical)
Idealistic or Realistic
The Contextual Setting for this Presentation
Rear View Looking or Forward Looking
3
Systems Engineering Expectation

• Successful implementation of proven, disciplined
  systems engineering processes results in a total
  system solution that is
• Robust to changing technical, production, and
  operating conditions
• Adaptive to the needs of the users and
• Balanced among the multiple requirements, design
  considerations, design constraints, and program
  budgets.

3
4
Some Inhibitors to Good Systems
Engineering Based on a survey of IT architects
and project managers

• Customer Related Input
• Isolation from real user
• Customer requirements and (even) identity not
  clear
• Customer doesnt know what they want
• Scope creep Undocumented system scope and
  functionality
• User/buyer too distant
• Dont understand the customer value system
• Management Related Input
• Executive management doesnt buy in
• Lack of teamwork
• Program Managers not empowered
• Program manager and capture managers are
  different
• Unstable funding stream Lack of upper management
  support
• Organizational/Cultural Input (Some Perceptions)
• SEA only adds to the Project Cost
• SEA often seen as an outside team or project
  reviewer role

We would like you to build us a lawn mower please!
4
5
Deploying Systems Engineering within a Commercial
Global Leader Some Results
6
Systems Engineering Has Been Applied to Both
Internal and Commercial Accounts
2001
2000
3rd qtr
1st qtr
2nd qtr
2nd qtr
4th qtr
3rd qtr
1st qtr
4th qtr
2003
2002
2nd qtr
3rd qtr
1st qtr
4th qtr
3rd qtr
1st qtr
2nd qtr
4th qtr
SE process integration - GS Method
13 projects using SE
SE process integration - AMS MS
SEI introduces CMMI 1.1 SE Design Class
introduced
SE team grows to 30 12 completed and 13 active
projects using SE
SE deliverables templates provided
17 completed and over 50 active projects using
SE Over 230 trained in SE Fundamentals
SE team grows to 14
SE process updated for CMMI
7
Systems Engineering Process defines deliverables
and a series of Reviews (Part I)
Need / Opportunity Identification
Conceptual System Specification
Detailed Design
Component Architecture
Customer Baseline
System Baseline
Architecture/Component Baseline
Design Baseline
Systems Requirements Review (SRR)
Preliminary Design Review (PDR)
Critical Design Review CDR)
Business Requirements Review (BRR)
Systems Reqment Specs
Component Reqment Specs
Component RTVM
Component Level Architecture
System Level Architect.
Component Design
Component Test Plan
Business Require. Specs.
Test Architecture
RTVM
Customer Provided
Systems Engineering Provided
Component Developer Provided
8
Systems Engineering Process defines deliverables
and a series of Reviews (Part II)
New Production System
Test and Production System Update
Development
Detail Design
Design Baseline
Test Baseline
Production Baseline
Test Readiness Review (TRR)
Production Readiness Review (PRR)
CDR
System Test Plan / Test Cases
System Test Strategy
Deployment Plan
Release Content
System Test Data
Test Traceability Matrix.
Move to Prod. Plan
Data Migration Plan
Customer Provided
Component Developer Provided
Systems Engineering Provided
Service Delivery / Managed Ops Provided
System Test Provided
9
ISM delivered 5 under budget and with higher
quality in production

• The charts here are based on data collected
  from a recent study analyzing project defects by
  type and phase. Here ISM defects by phase is
  compared to 46 similarly sized projects not
  utilizing SE.
• Total defect counts for non-SE projects
  exhibited 53.4 of total project defects during
  the Test Phase of the project. On ISM defects
  were detected earlier in the project life-cycle.
  In fact 56 of ISM detects were detected in Plan
  Phase.

• The chart on the left illustrates the
  cost implications of early defect detection as
  found with ISM 2.0.
• In effect ISM 2.0 expended 2.4 times less
  than what would have normally been required for
  the non-SE oriented projects compared to in the
  study.

10
IGA Metrics show 8 cost avoidance when comparing
SEA projects to non-SEA projects
15 of 10.7MM Baseline
7 of 10.7MM Baseline
8 Cost Avoidance
11
Similar Initiatives Underway at
11
12
RD spend per product launched is much higher
than key competitors

• Cost per product program is much higher than
  major competitors
• high RD spend as of sales
• absolute spend gt2 any other
• not many more products launched than other
  players with much less RD spend
• (benchmarking takes into account broader scope
  and uses comparable definition of what is a
  product)
• Higher sales volumes per product keep RD costs
  per unit shipped in line
• no clear advantage in terms of product
  competitiveness
• higher volumes come from other resources global
  reach brand strong channels logistics
• No economies of scale from RD despite spend more
  than twice any other player

13
Despite much higher spend, products now lag best
competitors on key dimensions
Nokia versusAverage of Competitors
Nokia versusBest Competitors
-
3
-
2
-
1
0
1
2
3
-
3
-
2
-
1
0
1
2
3
-
2
D
e
s
i
g
n
/

s
t
y
l
e
/

a
p
p
e
a
r
a
n
c
e
0
D
e
s
i
g
n
/

s
t
y
l
e
/

a
p
p
e
a
r
a
n
c
e
Competitors now have better designs
-
1
P
r
i
c
e
0
P
r
i
c
e
-
1
S
i
z
e



w
e
i
g
h
t
0
S
i
z
e



w
e
i
g
h
t
0
B
a
t
t
e
r
y

p
e
r
f
o
r
m
a
n
c
e
1
B
a
t
t
e
r
y

p
e
r
f
o
r
m
a
n
c
e
0
B
u
t
t
o
n
s

a
n
d

k
e
y
p
a
d
0
B
u
t
t
o
n
s

a
n
d

k
e
y
p
a
d
0
D
i
s
p
l
a
y
0
D
i
s
p
l
a
y
0
D
u
r
a
b
i
l
i
t
y
/

b
u
i
l
d

q
u
a
l
i
t
y
1
D
u
r
a
b
i
l
i
t
y
/

b
u
i
l
d

q
u
a
l
i
t
y
0
F
e
a
t
u
r
e
s
/

f
u
n
c
t
i
o
n
s
1
F
e
a
t
u
r
e
s
/

f
u
n
c
t
i
o
n
s
0
Q
u
a
l
i
t
y

o
f

r
e
c
e
p
t
i
o
n
2
Q
u
a
l
i
t
y

o
f

r
e
c
e
p
t
i
o
n
0
R
e
s
i
s
t
a
n
c
e

t
o

s
c
r
a
t
c
h
e
s
2
Nokia still leads on ease of phone user
R
e
s
i
s
t
a
n
c
e

t
o

s
c
r
a
t
c
h
e
s
0
T
e
c
h
n
i
c
a
l

r
e
l
i
a
b
i
l
i
t
y
1
T
e
c
h
n
i
c
a
l

r
e
l
i
a
b
i
l
i
t
y
1
E
a
s
e

o
f


m
e
s
s
a
g
i
n
g
2
E
a
s
e

o
f


m
e
s
s
a
g
i
n
g
1
E
a
s
e

o
f

n
a
m
e

s
e
a
r
c
h
2
E
a
s
e

o
f

n
a
m
e

s
e
a
r
c
h
1
M
e
n
u

s
y
s
t
e
m
2
M
e
n
u

s
y
s
t
e
m
Zero about same as competitors, 1 1 standard
deviation better than competitors, and so
on Competitors Motorola, Panasonic, Sagem,
Samsung, Sharp, Siemens and SonyEricsson
14
Key Concerns Identified Symptoms

• Asset management and technology management is a
  weakness
• Architecture portfolio management is extremely
  poor
• Design consistency is missing
• The norm seems to be reactive design versus
  proactive design
• Architecture studies, tradeoffs, and evolution
  could be better planned (architecture
  archeology!)
• Architecture ownership is not clear, and often
  not executed
• Too many product/platform variants, each
  requiring special testing and support
• Reactive redesign during implementation We
  dont know the performance thresholds of our
  architectures
• Architecture is often the result of
  implementation rather than the driver
• Too much discretion at the lowest implementation
  levels
• Example Data and image formats
• Software quality could be better expensive
  rework currently required
• Requirements are not managed (end to end)
• Competing and conflicting positions in external
  forums
• Requirements management (traceability) is often
  poor
• Interface management is poor
• Not enough linkage to business requirements

RD Productivity Impacted Innovation
Impacted Morale Impacted
Sometimes I feel like I am in a different company
than them Business priorities dont seem to be
linked to Technology priorities
I wish I had tools to help my management team
realize the importance of architecting
short-term-ism gets in the way our headaches
today were born 1.5 to 2 years ago
If these are symptoms, then what are the causes?
15
Resulting Situation

• Systems and devices often developed and tested
  through significant individual efforts and human
  networks under significant schedule pressure
• Existing processes are often uniquely (one time
  use) tailored/used
• Reuse is a myth in some cases, and getting in the
  way of innovation in other cases (because of
  serious schedule pressures)
• Processes and methodologies (and even language)
  are personality dependent, with no time for
  documentation (competency development?)
• In the absence of clear organizational authority
  and accountability, personality based authority
  and accountability has evolved in the current
  situation (e.g., Roles and responsibilities
  between BGs and TPs)
• CHALLENGE Scale up NOKIA RD efficiency (Time to
  Market) and innovation (Feature Leadership),
  without scaling up in size and compromising
  quality
• Architectural thinking can contribute

16
Theory versus (Virtual) RealityPrimary Reasons
for Dysfunctional Behavior My Opinion

• Confusion between What you NEED versus What
  you WANT
• Also called Gold-Plating
• It is the moral duty of a systems engineer to
  articulate the resulting cost and schedule delta
• Confusion with regard to the SYSTEM BOUNDARY
• This is more difficult for legacy systems with
  undocumented and implied interfaces and even
  more so for network-centric systems and SoS
• Confusion (?) with regard to fidelity between the
  technical project scope and its allocated budget
  and schedule
• The result is cynicism and complacency, along
  with other negative behavioral patterns
• Lack of Leadership

17
Holistic Thinking versus Local Thinking
Innovation Associates
0943-98
17
18
A few words about competency development
19
Academic Perspective Discipline and Domain
Centric Systems Engineering Programs

• Discipline Centric Systems Engineering Programs
  These are programs where the major is designated
  only as Systems Engineering

• Domain Centric Systems Engineering Programs
  These are programs where the major is designated
  as X and Systems Engineering or Systems and X
  Engineering.
• In this case, the most common instances of X
  Engineering are
• Industrial Engineering
• Manufacturing Engineering
• Electrical Engineering
• Management Engineering
• Computer Engineering

The primary source of this data is Fabrycky,
W.J., Systems Engineering Education in the
United States, Proceedings, Conference on
Systems Integration (CSI), Stevens Institute of
Technology, New Jersey, March 2003.
20
Air Force Institute of Technology California
State UniversityColorado School of MinesCornell
UniversityGeorge Mason UniversityGeorge
Washington UniversityIowa State UniversityJohns
Hopkins UniversityNational Technological
University Naval Postgraduate SchoolOakland
UniversityPolytechnic University -
FarmingdalePortland State UniversityPurdue
UniversityRochester Institute of Technology
Academic Perspective Universities with
Discipline Centric Systems Engineering Programs -
30
Southern Methodist University Stevens Institute
of Technology University of Alabama -
Huntsville University of ArizonaUniversity of
Idaho University of Illinois at
Urbana-ChampaignUniversity of Maryland
University of MassachusettsUniversity of
MinnesotaUniversity of Missouri -
RollaUniversity of PennsylvaniaUniversity of
Rhode IslandUniversity of Southern
California University of VirginiaVPI and State
University
The list in the primary reference contains 35
records. Four of these referred to Universities
with only an Undergraduate Program in Systems
Engineering, and one to a University with only a
Doctoral Program in Systems Engineering.
21
Academic Perspective Universities with Domain
Centric Systems Engineering Programs - 38
Auburn University Boston University California
State University - Fullerton Case Western Reserve
University Georgia Tech Massachusetts Institute
of Technology New Jersey Institute of
Technology North Carolina A and T
University Northeastern University Ohio State
University Ohio University Polytechnic
University Purdue University Rensselear
Polytechnic University Rutgers, The State
University San Jose State University Stanford
University Texas Tech University University of
Alabama - Huntsville
University of Arizona University of Central
Florida University of Connecticut University of
Florida University of Houston University of
Illinois University of Memphis University of
Michigan Ann Arbor University of
Michigan-Dearborn University of
Minnesota University of Pittsburgh University of
Rhode Island University of South
Florida University of Southern California Universi
ty of Southern Colorado University of St.
Thomas Virginia Tech Wichita State
University Youngstown State University
22
Industry/Government Perspective Systems
Engineering Education and Training

• Definition of Relevant
• Relevance of the curriculum orientation to DoD
  projects and programs
• Portability and flexibility of the delivery
  format distributed organizations
• Hybrid credit and continuing education
• Basis
• The Boeing SE Education Program (50 gt 15 gt 6 gt 2)
• A DoD Component SE Education Program (80 gt 25 gt
  11 gt 2)

Available
Relevant
23
Industry/Government Perspective Systems
Engineering Education and Training

• Definition of Critical Mass
• Number of Tenured or Tenure Track Faculty
• Number of Faculty with DoD/Aerospace Relevant
  Project/Program Experience
• Bench Strength

Why?
Available
Relevant
Critical Mass
24
Industry/Government Perspective Systems
Engineering Education and Training

• Definition of Critical Mass
• Number of Tenured or Tenure Track Faculty
• Number of Faculty with DoD/Aerospace Relevant
  Project/Program Experience
• Bench Strength

Available
Relevant
Critical Mass
It is my opinion that we do not have a critical
mass in graduate SE education in the US However,
we do have a very mature base and recognition
within industry and government to facilitate the
building of this critical mass
25
Industry/Government Expectations Systems
Engineering Education, Training, Education

• Systems Engineering Education
• Relevant graduate level courses, anchored with
  academic rigor and yet sensitive to current
  system development and integration challenges.
• Flexible delivery formats online/distance
  modular
• Flexible for part-time course loads
• Full time leave of absence for educational
  purposes is a thing of the past
• Systems Engineering Training
• Relevant and focused training courses on specific
  subjects of immediate relevance

• Systems Engineering Research
• Development of an SE Toolkit (Templates, Metrics,
  etc.) of immediate utility within an organization
• Domain centric
• Specific purpose of enhancing development
  efficiency and effectiveness
• Usage centric
• Architectural assessment frameworks
• Open systems (a much abused phrase)
• Domain centric
• Others

In an environment of high demand and scarce
supply, there are many experts but who sets the
minimum thresholds on relevance and quality?
26
Crossing the boundaries the Open Academic Model

• Blurring the boundary between academia and
  industry/government
• Bringing a fresh perspective to
  industry/government in an executable form a
  specific method, tool, heuristic, template
• Bringing industry/government reality into
  academia in a researchable or usable form a
  problem statement, a specific challenge, guest
  instructors, heuristics, case studies

The Challenge from Ralph Nelson at IBM Global
Services
27
Wrap-up Essential Elements of a Systems
Engineering Program

• Leadership
• Policy with Executive Measurements
• Investment to develop the process, templates,
  education, mentoring
• Process and tools
• Defined Process
• Templates
• Skilled SEs Core group of SEs with 15 years
  experience on major programs within relevant
  domains
• Certification Program
• Education
• Experience
• Examination
• Ongoing Process Improvement