Introduction

1
AFCEA TechNet Lisbon 2005
2
Data Inepte

• Its the data, dummy!

3

• Role of the Computer
• Is it
• To assist human beings to take decisions?
• To provide an expensive desk top accessory?
• To provide employment for the IT department?

4
Answer Almost invariably b) and C) Why? Becau
se we are creatures of habit and dislike change!

5
Historical Perspective

• In the beginning was paper!
• Then - computer automation
• BUT memory very expensive
• Hence RDBMS and tricks of saving space
• Computer storage now very cheap
• BUT we use same tricks.
• WHY?

6
Obsessions

• People are obsessed with Relational database
  models
  
• Major companies make their money out of RDBMS.
• We have millions of information silos throughout
  the government and commercial world
  
• Over 30 years, no common data standards have been
  achieved.
  
• Mergers and acquisitions create complex
  information environments
  
• Very public failures IR, CSA, Police, MoD, etc.

7

• 24 decisions are wrong
• Average annual cost/company is 800,000
• Requirement to be agile
• Collaboration preferred
• Reluctance to take risks, determination to seek
  optimum solution or stick by a decision!
  

8
Cost of Computing
Cost of Memory
Cost of Hardware
9
Cost of Computing
Systems Costs
Cost of Memory
Cost of Hardware
10
High Costs

• The problem is Legacy systems and the RDBMS
• Cost of scrubbing and cleaning data
• Normalisation and renormalisation
• New business process-led initiatives vs. IT
  driven projects
  
• Business data is more often searched and
  analysed than inserted. - IBM

11
What can we do?

• Modern IT systems focus on the system rather
  than the data.
  
• Must shift the focus from the IT department to
  the Boardroom.
  
• Operational imperatives demand that we focus on
  the data not the system.
  
• Data inepte
• Its the data, dummy!

12
What about legacy systems?

• Legacy systems range from those that are old and
  unused any more to those which are still in
  active use.
  
• Traditional answer was to migrate systems to
  new standards.
  
• Legacy systems continue to be of utility.
• Need to build on them not destroy them!
• Must have lower costs, faster delivery and
  improved confidence.

13
Importance of Data
Intelligence
DATA
Governance
Compliance
14
Governance, Compliance and Intelligence

• Essential in any organisation, including in
  military operations
  
• Governance How an organisation delivers its
  strategic goals within the environment within
  which it operates.
  
• Compliance How an organisation manages its
  information environment in such a manner as to
  meet all regulatory demands placed upon it.
  
• Intelligence How an organisation develops a
  clear strategic or tactical picture of its
  operational environment.

15
Basing Decisions on the Intelligent Use of Data

• Importance of context as well as content.
• Development of clear intelligence threads
• Depth of intelligence as well as breadth
• Need to get an integrated picture from
  disparate legacy systems
  
• In the police case address cars, guns, who
  lives next door, bank accounts, previous form,
  etc
  

16
Whats the answer?

• Use of XML native database approach.
• This approach has been endorsed by IBM with Viper
  project a native XML extension to DB2.
  
• Fast, reliable, multi data-source with improved
  confidence of assurance of outcome.
  
• Particularly relevant to intelligence processes,
  especially where multitude of sources.
  
• Police proof of concept underway.

17
Legacy Systems
Applications (of any sort)
TyrrellSmith XML Rummage Store
Legacy Systems structured and unstructured data
18
Actionable Data

• Data which gives that intelligence including all
  key information for a decision to be made.
  
• Clear and uniquivocable?
• Adequate provenance (if necessary to satisfy the
  CPS or equivalent)
  
• Inevitably from an increasingly large database
  pool.
  
• Requirement to be cross-organisational.
• Must be fully auditable.

19
The Intelligence of Intelligence

• Must be a clear understanding of the nature of
  information its strengths and limitations.
  
• Recognise database limitations
• The knowns and the unknowns
• Clear analytical skills.
• Ability to conduct what if? scenarios.

20
Seeing is believing or is it?

• Panorama 1963
• Chinese Embassy, Belgrade
• WMD Iraq 2003
• Understand perceptions.
• We often see what we want to see.
• Interaction of humans and technologies.

21
The Humble Carbon Unit
22
The Humble Carbon Unit
Lateral prefrontal cortex
Trade offs Sophisticated decision making Abstrac
t rewards
Limbic system
Primitive Immediate stimuli Immediate decisions
23
The Humble Carbon Unit
Lateral prefrontal cortex
Trade offs Sophisticated decision making Abstrac
t rewards
Limbic system
Primitive Immediate stimuli Immediate decisions
24
The Humble Carbon Unit
Lateral prefrontal cortex
Trade offs Sophisticated decision making Abstrac
t rewards
Limbic system
Primitive Immediate stimuli Immediate decisions
25
Agile Data

• Understand the timelines within which data is
• Valid
• Relevant
• Focus on key issues BUT remember that too much
  focus can miss the essential facts.
  
• Require a sophisticated review process to ensure
  appropriate data available.
  
• Use of the traditional OODA loop.
• Validity of data, irrespective of data source.

26
The OODA Loop
27
Summary

• Move from a system centric to a data centric
  approach.
  
• Linkage of data silos using native XML
  processes.
  
• Develop agility of data management.
• Recognise strengths (and limitations!) of the
  human element.
  
• Improve governance, compliance and intelligence
  capabilities of organisations.
  
• Give the war fighter that joined up data he or
  she requires, where it is required at the time it
  is needed!