Data%20Governance

1
Data Governance
2

• A common thread runs through in a vast number of
  business problems that most problem solvers
  cannot see

3
Data that is not designed to be interoperable
4
Data design is the heart and soul of how IT
enables government to fulfill its mission

5
What is badly designed data?
6
One type of bad design is information that is
locked into data islands

Data on one island cannot talk to data on another
island
7

• Another term to describe locked data is siloed
  data or stovepiped data

8
Example of how data models affect business
processes
9
A programmer is hired to design a system for a
new library
10
For convenience of illustration, let's say that
patrons can be uniquely identified by first name
11
Problems

• Redundant address
• Change of address difficulty
• Redundant book author problems

12
Data Problems

• This example shows exactly how business
  processes are affected by poor data design

13
Bad data design forces business side to work
harder than necessary and without good
information

14
New layers of inefficient business processes form
to adapt to the poor data design

15
For example, a business process could exist
where librarians have to hire an army of data
entry staff to do error checking to make sure
patron names are correctly spelled

16
Bad data design forces creation of layers of bad
business processes
17
Twenty-year problem lock-in
18
Data design plays a large part in controlling
programming and business processes
19
Enter Data Governance
20
Data governance cleanly resolves inefficient
business processes
PATRON FILE
21
Book file only contains info about books
BOOK FILE
22
Finally, we have a concept that handles the
transaction of a patron checking out a book
INTERSECTION FILE
23
computer screen display of what clients see
24

• Advantages
• Only have to type in the data once
• Address changes handled cleanly. Old address is
  updated for all current books checked out
• Automatic error checking on patron or book name

25

• Data governance improves service to the business
  side

26
Minor changes in data design translate into huge
business improvements throughout the organization
27
Client satisfaction is increased
28
Nation-wide interoperability

• A centralized government database from the
  Library of Congress of all books can be connected
  to so that library staff doesnt have to type in
  book information such as author name, title, etc.
  

29
The business side has benefited from even more
savings in costs and work
30
The data islands are now connected

• Imagine the work savings from not having to type
  in new book information such as title and author
  name

31
This is how data governance brings cost savings
32
Additional improvements are in time, quality, and
organizational agility
33
These are not one-time savings, but ongoing
savings that leverage each other
34
Agility

• Converting to the centralized national book
  database was easier because the data was already
  modeled correctly.
• They just replaced their book file with the
  national centralized one.
• The error checking and streamlined book check-out
  process were already in place.

35
Modeling data guarantees business agility
36
Data design is the heart and soul of how IT
enables government to fulfill its missionThe
library system is an example of how tightly data
design is coupled to business processes
37
Where is bad data design reflected in the real
world?

38
Business clients, e.g., the librarians, cant
visualize that data design caused the various
problems
39
Clients see the problem camouflaged as a lack of
functionality in many areas their business
processes
40
Dept. of Veterans Affairs
41
QUIZ

• How did it happen?
• What was missing?

42

• Answer a comprehensive way looking at the
  interconnection of problems and prioritizing them
  in coordination with the enterprise data vision
• Do you see the correlation to the centralized
  remote book file for the library?

43
FBI

• FBI Director Robert S. Mueller, III

44
FBI had competent vendors and project managers
install their system

• On time
• On budget
• On scope.
• Good data modelers
• QUIZ What was missing?

45
Answer a comprehensive way looking at the
interconnection of problems and prioritizing them
in coordination with the enterprise data vision

• Hidden requirements were not identified.
• No organizational structure existed to ensure
  enterprise-wide interoperability.
• A process that continually takes the
  enterprise-wide view of how to implement change.
  They just added systems to the rest like a
  manager of a fleet of cars adds a new car. They
  didnt check for interoperability.
• The problem was repeated over and over again for
  each new FBI project.
• Data governance was missing.

46
New way of looking at the enterprise1.
Connections2. Potential connections not yet
implemented
47
Using this new method to view an organization,
how pervasive is this problem?

• The problem is huge and expensive. E.g., Fical,
  21st Century projects were designed to handle
  lack of comprehensive integration.
• When someone leaves their organization example.
• Were basically talking about everything.

48
A interpreter is needed that identifies business
problems stemming from bad data design and then
designs the data models that fix the problems at
an enterprise-wide level
49
Data governance

• Data governance is the practice of organizing and
  implementing policies, procedures and standards
  that maximizes data access and interoperability
  for the business mission

50
Why is data governance needed?

• Business needs drive data governance

51
This is what interoperability looks like both to
business side and the IT side
52
Each link represents a business value where data
can reach the business people that need it
53
But beyond the requirements that are represented
in the links, many more business requirements are
invisible to all except the enterprise data
modeler
54
Requirements gathers cannot find them
55
What integration opportunities look like to an
enterprise data modeler
56
Currently, much data is stovepiped

• Many connections are missing

57
Each potential link between data that is not
implemented represents a loss of business value
58
QUIZ

• How do stovepiped systems get built?

59
Forces that create data stovepipes

• Contractors and vendors
• Programmers
• Managers
• COTS
• Budget structures
• Security concerns
• Methodologies SDLC, PMBOK, WATERFALL, AGILE,
  SPIRAL
• Change
• In sum, an absence of data governance

60
Organizations mistakenly think installing new
applications are like replacing one of the cars
in their fleetIts broken, so.replace it
61
QUIZ

• What is a better metaphor for representing a new
  software system than replacing a car?

62
A component of aNeural Network
63
It is important to add to the project team, a
person that can check for enterprise
interoperability opportunities
64
This is a key method regarding how to look at
new projects
65
The neural network will ensure that one hand will
know what the other hand is doing
66
A small sampling what types (part of a much
larger list) of improvements the links represent

• Eliminates manual operations, e.g., when
  redundant tables or fields are eliminated
• Business side has the data it needs when it needs
  it
• Organizational agility
• Closer IT alignment to organizations mission.

67
Example of one of the links at the California
Dept. of Consumer Affairs
68
Calif. Secretary of State website
69
Corporation data interoperability was an
invisible business requirement
70
Because data interoperability problems are
invisible, business processes are unnecessarily
limited throughout the enterprise due to
undiscovered data integration
71
What is data governances goal?

• Bring together the business side, with the
  people that can see the invisible business
  requirements in order to build the data
  structures that will give business the
  information it needs when it needs it

72
Why is the need to implement data governance
urgent?
73
The design phase for projects is relatively
short
74
The opportunity to correct it at the right stage
consisted of only a tiny, fleeting time window
75
So why is lack of data governance urgent?

• Project development is currently going on in
  many areas without the benefit of data
  governance, which could result in permanent
  stove-piping of data.
• Result Many stovepiped systems.

76
QUIZ

• How many years can bad data design affect a
  system?

77
Inefficient business process may get locked in
for 20 years or more
78
Twenty-year problem lock-in
79
How does data governance work?

• The data governance process acts as a central
  planning center to coordinate data design across
  organizations

80
It builds the enterprise-wide blueprint that
guides all IT development towards data
interoperability
81
Simple process!!!

• Data Governance Council reviews changes or new
  development against a checklist

82
The governance council reviews all new software
development to see if it could be shared
enterprise-wide
83
New table or field checklist

• Correct data modeling (3rd normal form), correct
  keys. (Data Architect, IT)
• Cross agency data and process sharing
  opportunities, including SOA and Web Services
  (Data Architect, IT and business side) Image
  waiting clocks text Clients dont have to
  wait until they request data sharing.
  Comprehensive data sharing opportunities check is
  done at the outset.
• Business Intelligence data mart/data warehouse
  opportunities. (Business side)
• Metrics generation opportunities - Can this field
  or table create useful metrics or appear on a DCA
  dashboard? Customers could include boards,
  licensees, the public, finance, governor's office
  and the legislature (business side)
• KPI - Key performance indicators opportunities. 
  Are there opportunities to use the field/table to
  measure performance?   (Data Architect, IT and
  business side)
• Is data subject to legislative oversight or
  mandates?  E.g., Health Insurance Portability and
  Accountability Act (HIPAA), California Database
  Breach Act (California SB 1386),
  http//www.itl.nist.gov/fipspubs/geninfo.htm,
  FIPS, HSPD-12. Create table of federal, state
  and departmental regulatory mandates or voluntary
  guidelines that reviewers check data against
  (Data Architect, business side)

84
Checklist continued

• Are there opportunities from making this
  available to a broader audience? Customers that
  are not immediately evident could include boards,
  licensees, the public, enforcement, DOJ, finance,
  governor's office and the legislature (Data
  Architect, IT and business side)
• Can this data be replaced by a better source of
  data elsewhere or replace other data? Can whole
  tables be eliminated by consolidation and
  sharing? (Data Architect, IT and business side)
• Prioritize super connector fields and super
  connector tables.  (Super connector fields are
  those that cross agency boundaries.  Super
  connector tables are the most important tables
  that can be shared.  These must be listed in a
  transparent, centralized database and reviewed
  for use whenever a new system is designed. 
  Vendor created systems must be reviewed for
  table/data sharing and naming conventions.)  If
  this is a new super-connecter, then it should be
  transparently registered to the repository so
  that other strategic planners can see it. (Data
  Architect, IT and business side)
• Can it be used to validate data or does it need
  validation performed on it?  (Data Architect, IT
  and business side)
• Data harmonization problems or opportunities.
  (Data Architect, IT and business side)
• Standards evaluation Are there standards to be
  adhered to or created? Data standards, business
  standards, naming conventions, etc.  For example,
  every state agency could have the same standard
  for this field Corporation_Tax_ID_Number 40
  characters alphanumeric. Does NIEM. gov
  already have a standard name for this field?
  (Data Architect, IT and business side). Example,
  CAS Class field.

85
Checklist continued

• Alignment to organizational mission. Strategic
  planning problems or opportunities.  (Data
  Architect, IT and business side)
• Enterprise Architecture planning.  How does it
  align with to the To Be architecture? 
• Impact on other systems.  Entered into
  Architects dependency database (what systems
  does it impact or is it impacted by) (Data
  Architect, IT and business side)
• Metadata opportunities (business side)
• Risk (Data Architect, IT and business side)
• Security.  Should it be encrypted? What controls
  should be applied (Data Architect, IT, business
  side and ISO)
• Should client be given control of the data? 
  Would a data steward be useful for this data?
  (Data Architect, IT and business side)
• Backup considerations - how often.  How does it
  get refreshed when there is a crash?  When should
  it be purged? (IT and business side)
• Can data quality be improved? Is data cleansing
  applicable? (IT and business side)
• Quality management are clients satisfied? Is
  quality management and continual process
  improvement built into this system? (Data
  Architect, IT and business side)
• Automated duplicate detection (IT)

86
Checklist continued

• Timeliness. Is there value to the organization
  if the data is refreshed sooner or by other ways?
  (IT and business side)
• Is the data coming from the best sources
  (lineage most reliable, timely)? (IT and
  business side)
• Enterprise architects calendar scheduled for
  periodic data design review every two years
• Priority on architects data design inventory.
  (Data Architect)
• Optimization by combining multiple projects
  (past, future or ongoing projects). (Data
  Architect, business side, IT)
• Review for entry into a table of future
  opportunities and linked to a calendar of related
  opportunities or future change events.  For
  example, if a related component was scheduled for
  updating, that would trigger an automatic
  reminder to review opportunities for this
  component. (Data Architect)
• Audit policy should the field or table be have
  its edit or use history recorded (IT and business
  side)
• Error management (IT and business side)

87
Generally, client requests for government
interoperability arrive inconsistently as clients
struggle to understand how to improve their
systems
88
The above checklist ensures that a whole series
of government improvement opportunities are
checked for at the precise movement when its
most important
89
Clients dont have to wait until they request
data sharing. Comprehensive data sharing and all
other opportunities checks are made at the outset.
90
QUIZ

• What it would it look like 10 years from now if
  every government agency used this checklist for
  every new project?

91
QUIZ

• Which one of the check list items was used to
  discover the SOS opportunity?
• Correct data modeling (3rd normal form), correct
  keys. (Data Architect, IT)
• Cross agency data and process sharing
  opportunities, including SOA and Web Services
  (Data Architect, IT and business side)
• Business Intelligence data mart/data warehouse
  opportunities. (Business side)

92
To handle the complexity of data, a simplified
super-connector check list would assist the Data
Governance Council identify data sharing
opportunities

• Tables and fields that have the most
  intradepartmental and statewide connectivity.
  Examples
• Corporation Number
• License type and license number
• SSN
• Address (including apartment number)
• Criminal case number
• Civil case number
• Agency code

93
Data governance concept is simple

• Whenever theres change we ask a question
• Can this be shared in or outside of our
  organization?

94
Data Governance flow chart
95
Sample integration priority list

• Licensing data model changed to make individual
  unique identifier (QUIZ how was this
  identified?)
• Remove status code constraints from programming
  code and move them to table-based system (Where
  did this idea come from? Data modeler)
• Enforcement measurement fields (QUIZ Where did
  this come from?)

96
Without data governance, how do data improvement
opportunities traditionally become known to IT?

97
Answer

• Business client submits a ticket for a problem.
  This involves delay.
• IT manager or OCIO recognizes a pattern. This
  involves delay.
• Business managers recognize a pattern. E.g., LUG
  and EUG (Board user groups) collectively discover
  a problem. This involves delay.
• Vendor products and recommendations. This
  involves delay.

98
All of the current methods involve delays and do
not provide a consistent and continual process
for identifying and addressing problems
enterprise-wide
99
How do new opportunities come to the Data
Governance Councils attention?

• Project conception
• PMO
• SDLC
• FSR
• SPR
• PIER
• RFP
• Change Management Board
• IT Governance
• PIT (Process Improvement Team)
• Strategic plan
• Executive strategic discussions
• BCP
• ITPP - Information Technology Procurement Plan
• PSP - Proposal Solicitation Package
• Table and field creation process (DBA,
  programmer, etc.)  
• List of business-side data related requests
• Informal business projects, such as potentially
  sharable spread sheet data
• Programmers or business clients

100
Data Governance shortens the time that it takes
to determine business clients need a data change

• Otherwise, IT waits until business side submits
  a ticket for a specific problem. They dont know
  they have a data modeling problem or dont
  realize that their business process can change
  through data design.

101
A comprehensive array of discovery points above
speed up identification of improvement
opportunities
102
QUIZ

• At what discovery point was the SOS opportunity
  found? (Hint)
• Project conception
• PMO
• SDLC
• FSR
• SPR
• PIER
• RFP
• Change Management Board
• IT Governance
• PIT (Process Improvement Team)
• Strategic plan
• Executive strategic discussions
• BCP
• ITPP - Information Technology Procurement Plan
• PSP - Proposal Solicitation Package
• Table and field creation process (DBA,
  programmer, etc.)  
• List of business-side data related requests

103
Was this too late?

• When is the best time to discover data
  opportunities?

104
How can data governance be implemented?

• Stage One To handle urgent problems.
• A quick Stage One with a short time line is
  envisioned without lengthy discussions

105
DBAs simply email Data Architect any planned
schema updates

• Simple, cheap and effective.
• Reasoning Project development is currently going
  on in many areas without the benefit of data
  governance, which could result in permanent
  stove-piping of DCA data.
• We want to catch any emergency problems in the
  bud.

106
Stage Two

• Volunteers from business and IT form an initial,
  first version of Data Governance Council. The
  Data Governance Council designs itself and
  processes are to be improved as we collectively
  gain more experience.

107
Together, business and IT build the data
integration vision

• E.g., data warehouse
• Enterprise connectivity
• Etc.

108
What factors make data governance successful?

• Data governance is between 80 and 95 percent
  communication.

109
The most important factor in most successful data
governance programs is communication

• Clearly, data governance is not a typical IT
  project!

110
How effective is data governance?

• Very cost effective
• Vast scope of business processes improved
• Money savings example Fical

111
All areas of the organization are improved
112
Data Governance advances the efficiency, cost
savings and agility for every service

• PMO
• Process Improvement
• Ticket system
• IT Governance.
• Data governance will help every project become
  more successful wherever its included

113
What are the benefits of data governance?

• Each time there is a single integration
  improvement, it removes roadblocks to the
  organization's mission. Data silos become
  accessible, clients' problems are reduced,
  maintenance problems are reduced and connectivity
  opportunities open up across departments. This
  incremental method is also the least expensive
  approach.

114
Benefits of data governance

• Greater department-wide and statewide
  interoperability
• Citizens receive better service from integrated
  government business processes. Data will be more
  accurate, complete, and timely. Working with
  government will be more convenient, for example,
  when citizens only need to go to a single
  government agency to update their address instead
  of multiple government agencies
• Brings the business side into the IT improvement
  process
• Better business side control over data, privacy
  and project development
• Faster identification and implementation of
  business solutions. Data governance methodically
  discovers the gaps in how IT services business
  and shortens the time from problem discovery to
  solution. Data governance shows the business
  side how to find their voice in collaborative
  problem solving.
• Improved business decisions due to accurate data
  from the recognized source of record

115
Benefits of data governance continued

• Increased user business side trust in data stored
  within the organization's databases
• Helps meet the enterprises business goals
  including adaptation to changing regulatory and
  other environments
• Eliminates data duplication as a result of data
  governance process
• More accurate, consistent, complete, accessible
  and up-to-date data
• Fraud detection is facilitated because all data
  field names across the department and state are
  standardized
• Placing all data related requests in one place
  allows patterns to be identified
• Clear documentation of the lack of integration
  may provide business managers with better new
  project proposals
• Ease of business process refinement due to
  standardization of data components
• Opportunities for harmonizing and standardizing
  business terms because stakeholders are brought
  together in a collective review process. For
  example, if identical meaning terms were "cost
  allocation" and "distributed cost", stakeholders
  could agree to standardize on one of the terms
  and remove the other from business documents such
  as contracts and agreements

116
Benefits of data governance continued

• Business Intelligence. Data warehouse creation
  simplified through standardization of business
  data
• Better programming code due to correctly
  organized data
• Agility in responding to new opportunities
• Stops business system decay. Keeps all systems
  tuned to organizations mission and to each other
  so that no new system re-writes are ever
  necessary.

117
How has data governance worked in real life?
118
UMass Boston
119
Transformational to business side

• Projects were completed much faster
• Project quality was much higher
• Greater programmer collaboration
• Greater business side collaboration
• Productivity went through the roof

120
Solutions in all areas of business

• Inventory
• Finance
• Project management
• Etc.

121
Many unexpected business benefits were revealed
122
Unlocked data allowed more opportunities for
innovation and agility
123
Clients were extremely satisfied
124
Clients were extremely satisfied

• New business advantages
• Data was prescient
• Easier to get reports
• Reduced workload
• Reduced errors

125
Success factors (1) Continual contact with
clients (2) Modeling data to translate data
model into client solutions
126
UMass Boston

• Each time a new software system was installed,
  it was completely and totally integrated, not
  just partly, but with every possible data
  connection fully implemented. Every table and
  field was examined for enterprise integration.

127
Continually tuning the organization
128
UMass Boston

• One of the first data-integrated organizations
  in the country

129
UMass Boston data governance

• Built process improvement into the DNA of the
  organization

130
Vision

• UMass Boston model of continual tuning for
  enterprise-wide integration

131
Where do we go from here?
132
QUIZ

1. Can you name any software product that has a
   tuning process (when one part changes or has new
   components added or a different vendor adds
   something to it, all parts are evaluated)?
2. What is the current process for tuning this
   organizations data?

133
Personnel System
134
Personnel System

• Contenders in the project management competition
• Teams from many state departments
• State Personnel Board
• Private Industry
• Single individual

135
Methodology comparisons

• Teams from many state departments - main tool
  waterfall - failed
• State Personnel Board - main tool waterfall
• Private Industry main tool waterfall - failed
• Single individual main tool (1) working daily
  with client to understand requirements (2)
  modifying data design daily to translate
  requirements into best data model

136
Comparison of old school methodology and data
governance performance

• Time
• Scope
• Budget
• Quality
• Risk
• Client satisfaction

137
Time

• (a) One data governance method developer
  completed the personnel system in two years.
• (b) Waterfall - The only other contestant that
  finished the project was the centralized state
  department team that had six programmers working
  on it for twenty years.

138
Scope

• (a) Data governance method developer completed a
  fully automated personnel system.
• (b) The centralized state department team system
  was not fully automated.

139
Budget

• (a) Data governance - 50,000 a year for one
  developer's salary for two years.
• (b) Waterfall - difficult to estimate, but at
  least ten times as much.
• (c) Private industry - 500,000, but project
  failed.

140
Quality

• (a) Data governance - A quality project was
  delivered without a single flaw.
• (b) Waterfall - difficult to use.

141
Risk

• (a) Data governance - no risk because client
  tested and approved every new feature daily.
  Financial expenditure was minimal.
• (b) Waterfall - very risky as all projects except
  one failed, wasting large sums of taxpayer money.

142
Client satisfaction

• (a) Data governance - clients were extremely
  satisfied.
• (b) Waterfall - all projects failed, except the
  centralized state app, which all clients disliked
  so much, they tried to build their own.

143
QUIZ

• What was the most important criteria item in the
  personnel system project?
• 1. Time
• 2. Scope
• 3. Budget
• 4. Quality
• 5. Risk
• 6. Client satisfaction

144
Project results

• No client requirements were left uncompleted
• Identical requirements
• Individual state teams
• Private industry
• A state department

145
QUIZ

• What was the project manager best at?

146
Answer Client requirements management

• Understanding client needs
• Identifying known and unknown client requirements
• Translating business functions into data models
  that fulfill client requirements

147
Confidence
148
What happens to projects after they are completed?
149
Success factors

• Continual contact with clients
• Modeling data to translate data model into client
  solutions.

150
Enterprise system decay

• Change is inevitable. As new components are
  added or change, the IT systems across the
  enterprise begin to slip out of alignment with
  each other.
• Examples are (1) the statewide procurement system
  (2) library book file before it was connected to
  the centralized database.

151
Even systems that have recently been rewritten
from scratch or purchased new begin to
disintegrate immediately if small changes are not
evaluated for integration opportunities
152
What prevents enterprise system decay?

• Whenever there are local changes, enterprise-wide
  data reviews must be made.
• Change management including data governance
  reviews keep it in tune.

153
Continually tuning a whole organization

• To do this, the Data Governance Council must sit
  with or have access to several critical gateways,
  such as the Change Management Board, FSRs, etc.
• Data Governance Council can then review changes
  with an enterprise-wide perspective.
• It would use the data checklist to look for data
  sharability, harmonization and other
  opportunities.

154
QUIZ

• If enterprise data were always kept fully
  normalized and updated for business rule changes,
  would any system re-writes or replacement
  purchases be necessary?

155
Enormous value of data governance

• Add up the cost of small, medium and largest
  system rewrites, unnecessary maintenance,
  unnecessary labor and lost functionality to see
  the true value of keeping data models fine-tuned.
• Newness of IT equipment is not relevant.

156
Suggested PMO role in data governanceEnsure that
data governance is applied to each project

• Make sure there's an enterprise data modeler
  looking at enterprise-wide and statewide
  integration opportunities, not just a data
  modeler.
• Ensure that data considerations are reviewed at
  the earliest stages of projects, e.g., conception
  phase
• Data governance deliverables to PM for each
  project (1) Data architect initial review of
  overall data interoperability opportunities(2)
  Certification that data is correctly modeled(3)
  Change management comments (comments on changes
  submitted to projects change management process)

157
Additional PMO opportunities

• PMO is a good fit as a member of the Data
  Governance Council. Data Governance discussions
  with PMO would benefit both the Data Governance
  Council and PMO, as PMO would become aware of
  business opportunities
• Critical gateway notifier
• Change management partner
• Cultural change ambassador

158
Concluding thoughts

• The natural trend is to stovepipe
• Data governance reverses that trend

159
Data governance summary

• Its important
• Its urgent timeliness is critical
• It significantly affects all facets of the
  organization. UMass Boston and Dept. of
  Insurance successes were transformational to
  those agencies. Data governance revolutionized
  the business side.
• Its a new discipline for improving BP
• Implementation is simple. Just a checklist.
• It continually tunes IT to business needs

160
QuestionsFeedback?