PhD Research Project: An Improved Methodology for IS Planning and Development

1
PhD Research ProjectAn Improved Methodology for
IS Planning and Development

• Based on IS Strategic Planning and Enterprise
  Architectural Practice

David WiltonIIMS, Massey University, Albany, NZ
2
Sequence

• Introduction/motivation
• IS Strategic Planning (ISSP) Enterprise
  Architectural Practice (EAP) brief intro
• Comparison
• Outline of research task
• Results of survey
• Summary/ way ahead

3
Introduction

• Personal background and motivation
• IS strategic planning study and practice in early
  1990s
• Studied enterprise architecture in 2000-01 in
  preparation for DSTO Task
• Paper comparing approaches in JBT, presented to
  DIE seminar, both March 2001
• Initial proposal for doctoral thesis agreed by
  UNSW, 2001.
• Enrolled Massey 2004.
• Full registration May 2005.

4
Introduction (contd)

• IT represents a significant investment for most
  organisations (Australian Defence Organisation
  multi- Bn in sunk investments, 1.3B per annum
  in operating costs)
• Key issue How to manage information and IT
  assets, and plan investment in an effective, yet
  affordable, manner?

5
Introduction (contd)

• ISSP and EAP are two approaches that have emerged
  similarities apparent
• Scope of ISSP and EAP depends on individual
  perception to meaningfully compare them one
  must choose specific instances or approaches.
• Theoretical comparison (2001 paper lit review)
  
• ISSP CCTA - Central Computer Telecomms Agency
  of UK Treasury now called Office of Government
  Commerce (OGC)
• Enterprise Architectural Practice US DoD C4ISR
  AF (cf Zachman etc)

6
IS Strategic Planning - Brief Overview

• CCTA denotes the following key objectives of IS
  strategic planning
• understanding the aims and objectives of the
  business
• establishing the information requirements of the
  business
• outlining the systems to provide the information,
  and determining the role of technology in
  supporting the information systems
• agreeing policies and plans to develop and
  implement the information systems
• determining the role and use of resources to
  achieve the information systems required
• managing, reviewing and evolving the strategy

7
IS Strategic Planning (Contd)

• The CCTA process is a sequence of actions,
  grouped into the common-sense phases of
• Where are we now?
• Where do we want to be?
• How do we get there?
• The methodology includes definition of an IS
  vision, and the presentation of costed options,
  to realise the vision, to senior management

8
Methods
ISSP Research
Focus of ISSP
Methodologies
Methods and ApproachesGalliers (1987)Lederer
Sethi (1988)Flynn Goleniwska (1993)Earl
(1993)Segars Grover (1999)Min et al
(1999)Doherty et al (1999) Levy Powell
(2000)Salmela Spil (2002)
Pro p r i e t a r y
ISSP Theory and AssessmentChan Huff
(1992)Lederer Sethi (1992)Doukidis et al
(1996)Lederer Salmela (1996)Chan et al
(1997b)Dufner et al (2002)Newkirk et al
(2003)Wang Tai (2003)
CCTA1988
Info Engin-eering 1989
ISSP SuccessGalliers (1991)Fitzgerald
(1993)Segars Grover (1998)
CCTA1999
ISSP for SMEs 2000
Automated Support for ISSPWagner (2004)
Boar2001
IS CapabilityPeppard Ward 2004
9
ISSP Methods
Focus of ISSP
ISSP Methodologies
Evolution of Technology
c. 1960
First commercial hierarchical-model DBMS 1966
Pro p r i e t a r y
DP Era1
CCITT X.25 PacketSwitching (WAN) standard 1976
1980
Relational OLTPSdatabases c. 1980
Desktop PCs c. 1982
Micro Era
IEEE/ISO 802.3 (LAN)standard 1985
CCTA1988
CASE tools c. 1988
Info Engin-eering 1989
1990
Internete-Commercec. 1990
OSF DistributedComputingEnvironment standard
1990
CCTA1999
NetworkEra
ISSP for SMEs 2000
m-Commerce c. 2000
IS CapabilityPeppard Ward 2004
Boar2001
(1) Eras defined by Nolan (2000)
2010
10
Architectural Practice - Brief Overview

• Architecture The structure of components, their
  interrelationships, and the principles and
  guidelines governing their design and evolution
  over time (a blueprint of the enterprise its
  IT)
• Architectures are developed to portray the
  evolution of an IT environment over various
  points in time, beginning with the baseline, or
  as-is architecture.
• The architecture envisioned to meet all future
  operational and business requirements is the
  objective or to-be architecture.
• May go through a series of intermediate
  architectures.
• (US DoD, Command, Control, Communications,
  Computers, Intelligence, Surveillance and
  Reconnaissance (C4ISR) Architecture Framework,
  Version 2, 1997)

11
Enterprise Architecture Framework Evolution
12
IS Strategic Planning and Enterprise AP -
Similarities

• Basic Intent/Vision Both are high-level
  approaches, intended to realise a rational,
  affordable IT infrastructure which is consistent
  with business strategy and goals
• Both include a baseline summary of existing IT
  infrastructure (where are we now?), and an
  objective architecture (where do we want to
  be?)
• Both establish the information requirements of
  the business and determine the systems required,
  to provide and manage the information
• Both produce plans/architectures that are
  dynamic, and need to be reviewed regularly
• Both can be used by, or adapted to, any type or
  size of organisation that uses IT.

13
IS Strategic Planning and Enterprise AP -
Differences
14
Theoretical Conclusions

• There are distinct similarities in the objectives
  and scope of both approaches
• The main difference is in the process-orientation
  of IT strategic planning, compared with the
  product-orientation of EA practice
• Both have been demonstrated in practice to
  produce valid results, assisting enterprises to
  optimise their IT infrastructures
• The two approaches can be viewed as
  complementary, rather than mutually exclusive,
  and there could well be significant benefits in
  combining elements of both, to produce a new
  paradigm in IT planning and management.

15
Outline of Research

• Intention is to study the problem space with the
  view to developing an improved approach, or
  preferably a practical, usable methodology, for
  IT planning and development
• Bulluss and Chen (2001) suggest incorporation of
  other related planning and development approaches
  into EAP eg systems analysis, systems
  engineering, software development methodologies
• Other IS strategic planning methods,
  methodologies and architectural frameworks need
  to be evaluated (eg Zachman, TEAF, TOGAF).

16
Research Questions

• RQ 1. From both theoretical and empirical views,
  what is the relationship between ISSP and EAP?
  (e.g. unrelated, partially overlapping,
  synonymous)
• RQ 2. To what extent are different ISSP and EAP
  methods used in NZ, how successful are they, and
  how have the methods used and success levels
  varied over time?
• RQ 3. Are the methods used and success obtained
  related to organisational factors? (e,g.
  organisation type, size, level of IT maturity,
  senior management commitment, allocation of
  adequate resources).
• RQ 4. Can ISSP and EAP methods be combined to
  produce an improved IS planning methodology?

17
Research Methodology

• Literature reviews of candidate methods,
  methodologies and approaches including (but not
  limited to) ISSP and EAP. (Addresses RQ 1, from
  a theoretical viewpoint.)
• A survey of existing ISSP and EAP, to identify
  usage, success rates, best-of-breed methods and
  tools. (Addresses RQ 1 from an empirical
  viewpoint, also RQ 2 and 3.)
• Case studies of ISSP and EA in selected
  organisations, to provide in-depth information on
  ISSP and EAP, and to explore the feasibility and
  desirability of an improved methodology.
  (Addresses research questions 2, 3 and 4.)
• Investigation of the feasibility of an improved
  methodology for planning and ongoing management
  of IT resources in an enterprise. (Addresses
  research question 4.) It is anticipated that
  this will be accomplished by considering
  best-of-breed methods from ISSP, EAP and other
  relevant domains discovered during the study.
  This approach to development of an improved
  planning methodology is supported in the
  literature, eg (Levy et al., 1999, Levy and
  Powell, 2000)
• Any proposed improved methodology would require
  validation initially intended as an action
  research phase within the project, but advised at
  ACIS that that would be too large in scope (extra
  1-2 years) for a doctoral thesis to be
  conducted as subsequent work

18
Progress-to-Date

• Preliminary work in period 2001-2003 (paper
  published March 2001)
• Provisional enrollment PhD Feb 2004
• Initial literature review completed Aug 2004
  (25,000 words)
• IIMS postgrad seminar Sept 2004
• First draft proposal completed Oct 2004
• ACIS Doctoral Consortium Dec 2004
• Full registration May 2005
• Survey planning, administration S1-2 2005
• Paper presented ACIS 2005 introduces notion of
  an IS meta-strategy (a strategy for doing
  strategies) which could be a possible outcome
  from the main project
• Survey results analysed S1 2006

19
Survey Overview

• First draft survey questionnaire completed Mar
  2005 faculty review, pilot administration (some
  changes)
• Stratified sample (defined in collaboration with
  Barry McDonald)
• 50 large organisations randomly chosen from NZ
  MIS Top 100 directory
• 50 small or medium organisations randomly
  chosen from Telecom Yellow Pages
• Sub-stratified into industry groups as per survey
• Established contact with three major NZ
  corporates undertaking ISSP/EAP via NZ Computer
  Society using as real world sanity check
• Survey administered S2 2005 poor response rate
• Follow-up and re-administration Feb-March 2006

20
Research Model ISSP and EAP(Adapted from
Turban and Aronson, 1998)
(Lederer and Sethi,1992, Earl 1993,
Premkumarand King, 1994)
Success? V7
(Galliers, 1993,Gupta, 2004)
ISSP Methods and Methodologies V5
(Fitzgerald, 1993,Segars and Grover,
1998,Doherty et al, 1999,Newkirk et al,
2003Wang and Tai, 2003)
Resources V13
(Levy et al, 1999, Premkumar and King, 1994)
(Chan et al,1997b)
(Newkirk et al, 2003,Kearns and Lederer,
2004,Premkumar and King, 1994)
Management CommitmentV14
Enterprise ArchitectureFramework(s) V6
(Lederer and Sethi,1992, Earl, 1993)
Key
Intermediate or final
Decision variable
Fixed variable
outcome variable
(Nolan and Gibson, 1974,Galliers, 1991,Cerpa
and Verner, 1998)
Uncertainty
Certainty
Random (risk) variable
21
Initial Hypotheses

• H1 V9 is influenced by V1, V2, V3, and V4.
• H2 V10 is influenced by V1, V2, V3, and V4.
• H3 V7 is influenced by V5, V13 and V14.
• H4 V7 is influenced by V6, V13 and V14.
• H5 V15 is influenced by V13, V14 and (V9 and/or
  V10).
• H6 V9?V10 ? 0.
• H7 V5 and V6 are influenced by V1
• - We also need to measure (V7 ? V5) and (V7 ? V6)
  to determine best of breed methods.

22
Survey Results

• 50 valid responses (another 8 in pipeline?)
• Represents a limitation on this study that must
  be taken into account when interpreting the
  results
• However
• lt 2000 large organisations within NZ (MED,
  2005) a sample of 20 represents gt1 of the
  population
• ISSP research literature papers published (eg
  MISQ) with, e.g., samples of 18, 27, 80
  organisations
• Another limitation majority of respondents in
  large enterprises were IT staff (eg CIO) over
  whole sample, majority were business staff

23
Effect of organisational size on existence of
ISSP and EA
24
If no ISSP or EA, reasons why not

• ISSP and EA (identical)
• Didnt consider we needed one.
• Low benefit/cost ratio.
• Insufficient management commitment.
• Other options (rated by respondents as of
  relatively minor importance).

25
ISSP EA Processes
ISSP
EA
26
Assessed success of proceses
(Likert scale 1 Totally successful, 4
neutral, 7 totally unsuccessful)
ISSP
EA

• Both rated successful (on average)
• ISSP slightly higher success rating on all
  measures

27
Perceived resource allocation and management
commitment
(Likert scale 1 Totally satisfactory, 4
neutral, 7 totally unsatisfactory)
ISSP rated higher for resource levels
management commitment (neutral, on average, for
EA)
28
Techniques used
EA
ISSP
29
Analysis of techniques used

• Little useful data re usability success of
  techniques
• Little data pre-Y2000, so no opportunity to study
  evolution of techniques over time
• ISSP most organisations used a combination of
  tools or methods (few used a comprehensive
  methodology)
• EA majority used in-house technique

30
Realised ISSP and/or EA(The ISSP/EA has been
fully implemented

• ISSP
• no IS strategic plan has been fully implemented,
• most organisations (73.3) are in the mid-range
  between mildly agree and mildly disagree.
• two organisations (13.3) strongly disagree,
  indicating little or no progress towards
  implementing their plan.

• EA
• only 18.2 of organisations have shown any level
  of agreement to the statement reflecting
  successful implementation of their EA (and then
  only mild agreement).
• i.e. 81.8 of organisations have made little or
  no progress towards implementing their EA.

• Reasons for incomplete realisation
• An incremental (phased) approach has been adopted
  but we expect to get there eventually.
• Lack of management commitment.
• Lack of funding.
• Lack of stakeholder commitment or acceptance of
  the need.

31
Hypothesis Testing

• The following hypotheses have been demonstrated
• H1b V9 (the existence of an IS strategic plan)
  is influenced by V2 (organisational size).
• H2b V10 (the existence of an EA) is influenced
  by V2 (organisational size).
• H3b V7 (ISSP success) is influenced by V13
  (resource allocation).
• H3c V7 (ISSP success) is influenced by V14
  (management commitment).
• H4b V7 (EA success) is influenced by V13
  (resource allocation).
• H4c V7 (EA success) is influenced by V14
  (management commitment).
• H5a V15 (realisation of enterprise information
  management infrastructure) is influenced by V7
  (successful ISSP).
• H6 V9?V10 ? 0 (there is a significant overlap
  between the scope of ISSP and EA).

32

• The following hypotheses have not been
  demonstrated
• H1a V9 (the existence of an IS strategic plan)
  is influenced by V1 (age of organisation)
• H1c V9 is influenced byV3 (organisation type).
• H1d V9 is influenced by V4 (level of IS
  maturity).
• H2a V10 (the existence of an EA) is influenced
  by V1 (age of organisation).
• H2c V10 is influenced byV3 (organisation type).
• H2d V10 is influenced by V4 (level of IS
  maturity).
• H3a V7 (ISSP success) is influenced by V5
  (methodology used).
• H4a V7 (EA success) is influenced by V6
  (framework used).
• H5b V15 (realised IS infrastructure) is
  influenced by V7 (successful EA).
• Not tested H7V5 and V6 are influenced by V1
  (the choice of ISSP and/or EA techniques are
  influenced by the year the exercise was
  undertaken)

33
Relationship between ISSP and EAP

• H6 V9?V10 ? 0 (there is a significant overlap
  between the scope of ISSP and of EA).
• Objectives

(Identical, except establish technology path
policies promoted in EA)
34
Key findings

• There is a strong overlap between the objectives
  scope of ISSP and EAP in NZ organisations
  (within the limitations of the survey)
• Organisations may be wasting time resources
  developing both (management commitment lower
  towards EA)
• Advantages in combining best-of-breed approaches
  to create a comprehensive methodology?
• NZ SMEs have a very low incidence of ISSP and/or
  EA (20)
• SMEs may not be taking advantage of the
  opportunities that IT affords
• Simplified DIY methodology?

35
Summary

• Low incidence of ISSP /or EA in SMEs (in NZ)
• IS strategic planning could be combined with EA
  Practice to create a new paradigm for planning
  and management of IT
• My proposed research task is to realize that
  paradigm per medium of a methodology which
  combines the best of all relevant approaches and
  methods
• Questions/comments?

36
Additional Terminology

• Method an individual technique used to achieve
  some purpose, eg cost-benefit analysis, NPV
• Methodology an integrated collection of methods
  (to form some sort of process), eg CCTA IS
  strategic planning
• Approach a high-level, abstract description of a
  means of solving a problem, eg Rockarts
  Critical Success Factors (CSF) approach to IS
  strategic planning