Data Modeling: The Some Big Issues

1
Data ModelingThe Some Big Issues

• Presentation by Graeme Simsion
• Portland Oregon March 2004

2
Some History My 1994/2000 Manifesto

• 1. Data modeling is design
• 2. Data modeling is important
• and still relevant in an OO, ERP, Internet world
• 3. Data modeling is a discipline
• and users cant model
• 4. Data modelers use patterns
• and patterns for the data warehouse are
  different
• 5. Subtypes help
• and the level of generalisation is critical
• 6. Logical database design is the data modelers
  job
• 7. Corporate data modeling is different

3
Agenda as promised!

• Is data modeling analysis or design?
• Do we need professional data modelers?
• How can we convince others of the need for data
  modeling?
• What other professions can we learn from?
• What should our deliverables be?
• What should the role of the user / business
  specialist be?
• What are the respective roles of data modeler and
  database designer?

4
A little tweaking

• How can we convince others of the need for data
  modeling? (Is it still important?)
• Is data modeling analysis or design?
• What other professions can we learn from?
• What should our deliverables be?
• What should the role of the user / business
  specialist be?
• What are the respective roles of data modeler and
  database designer?
• Do we need professional data modelers?
• PLUS
• A word on enterprise data models
• Reinventing data management
• How does Oregon pinot noir compare with
  Australian pinot noir?

5

• 1. How can we convince others of the need for
  data modeling?

6
Writing on the pink slip

• We dont build applications in-house
• Were an OO shop
• Our data doesnt live in tables
• Our analysts know how to model
• Our DBA knows how to model
• Our users know how to model
• Modeling isnt that hard anyway
• Our architecture isnt model-based
• We just dont like data modelers

7
Most-quoted statement

• Data modeling isnt optional no database was
  ever built without at least an implicit model,
  just as no house was ever built without a plan
  The choice is not whether or not to model, but
  whether to do it formally, whom to involve, and
  how much effort to devote

8
Leverage and stability
9
Who sees the data model?
10
So what about

• Packaged software?
• Commoditization of IT?
• Agile approaches?
• Object-oriented approaches?
• XML?

11
Before we go to the next topic

• What is data modeling?

12

• 2. Is data modeling analysis or design?

13
Which better describes data modeling?

• (a) Describing the data requirements of an
  organization or part of an organization
• or
• (b) Designing data structures to meet the
  requirements of an organization or part of an
  organization

14
Does it matter?

• Reference disciplines
• Process and deliverables
• Mechanising or supporting the process
• Patterns
• Arguments - one right answer?
• Quality criteria
• Package selection
• Good modelers, great modelers (and amateur
  modelers)
• Valuing the contribution - and contributor
• The relevance of creativity
• Designing research
• Teaching data modeling

15
What do modelers think?

• 107 Master-class Attendees

43 chose (a) Describing the data requirements of
an organization or part of an organization
50 chose (b) Designing data structures to
meet the requirements of an organization or part
of an organization 14 said both
16
What do the thought leaders think?
17
What do the books say?

• The usual imprecision with analysis and
  design
• A few address the question of choice explicitly -
  and take a position
• Choice is usually at the level of choice of
  construct (entity, attribute, relationship)
  rather than basic abstractions
• Creativity is sometimes given lip service
• All actual examples have one right answer

18
What about other system components?

• Process modeling?
• Old logical, new logical
• BPR
• User interface design?
• Coding?

19
One right answer?
20
Why models of the same scenario might differ

• Different trivial choices naming etc
• Different abstractions / classifications
• Different levels of generalisation
• Rules held in different places
• Data structure
• Code
• Data
• Externally

21
Business Rules - a Human Construction

• and one persons requirement was once another
  persons design

22
A little experiment

• Masterclass in San Antonio, 2002
• A video and transcript - a real case
• 101 people submitted models
• 82 said they understood the requirement very
  well or fairly well
• 76 said they made no guesses or only obvious
  unimportant guesses about reqs.
• 1 person found the problem very difficult, 22
  fairly difficult

23
So, were the models different?

• Average number of entities - 8.5
• Range 2 to 22
• Number of different entity names across the
  models 302
• Most popular entities
• Questionnaire 56
• Question 49
• Person 41

24
Then we paired off and compared models

• The models were identical 1
• except for naming and agreed errors 6
• Structurally different in minor ways 54
• Structurally different in important ways 40

25
Personal Styles?
26
41 master-classattendees

• A database to record your family tree ancestry
  and marriages.
• A database to record details of various kinds of
  bank loans and the parties involved.

27
And you thought a family tree was simple

• Well it is the average number of entities used
  was 3.3
• BUT 50 different entity names used across the
  models.

28
A taste of the results

• 14 modelers included a marriage entity in their
  family tree model. This group produced
• All 5 of the family tree models containing
  child (cf person or party)
• 7 of the 11 banking models containing guarantor
  (cf Person, Party, Customer)
• 3 of the 5 banking models containing payment
  (cf transaction)

29
What other professions can we learn from?
30
And you thought data models were hard to
understand ...
31

• 4. What should our deliverables be?

32
Terminology
33
Generic methodology (e.g. Navathe, Teorey, us!)
Real World
Physical Design
Logical Design
Conceptualmodeling
Business Requirements?
Conceptual Data Model
Logical Data Model
Physical Data Model
34
Conventions its a two horse race

• UML vs E-R
• Richness vs Elegance
• Technical User vs Business User
• ORDB / OODB vs Relational Database
• Agile Baggage vs Life-cycle baggage etc
• Fanatic vs Fanatic

35
5. What should the role of user / business
specialist be?

36
How do the architect and client interact?

• I want a wine cellar.No you dont.
• How do you love this?
• Well, actually, not a lot.
• What diameter water pipe do you want?
• I dont care.
• You will when you see the prices.

37
Working together ...

• Insistence on client involvement
• Prawns on the barbie
• Multiple designs
• A 3-D model
• Go see one
• Time out - for both parties
• A second opinion?

38
What did the Architect bring?

• Tools for explaining
• Domain knowledge - lots!
• Access to specialists - a one stop shop
• Knowledge of materials and costs
• His own specialization - and idiosyncrasies
• Patterns - lots of them

39
6. What are the roles of data modeler and DB
designer?
40
Two sides of the argument
41
(No Transcript)
42
A slight difference of opinion

• Data modelers thought
• Data modelers responsible for 42 of items
• DBAs thought
• Data modelers responsible for 28 of items

43
The old three schema architecture
User View
Physical Storage
44
Daniel Moodys Seventh Habit

• See it for yourself
• Generate alternatives
• Keep it simple
• Test out the model
• Know when to stop
• Keep an eye on the big picture
• Follow the job through to completion

45
The architect as overseer

• Supervision - and reassurance
• Clarification and explanation
• Good solutions to problems as they arise
• The dodgy floor
• Light in the bedroom
• Another voice in implementation decisions
• Ill issue an instruction

46
Remember these guys?
47
Implications for our skill-set

• How much does the architect need to know about
  building materials and practices?
• How much does a modeler need to know about DBMSs
  and physical database design?

48
7. Do we need professional data modelers?
49

• An expert is someone who knows everything about
  their topic except

Cartoon concept G. Simsion Drawing P. Taylor
50
Humble origins
Policy
Person
PersonPolicyRole
51
Seeing the world differently
Kanban Control (Diagram from Leaders from
Manufacturing Research Group 5)
52
8. Enterprise data models
53
9. Reinventing Data Management(in 10 slides)

• Insanity is when you keep doing the same thing
  in the hope that you will get a different result

54
Data Administration

• A central team
• A Corporate Data Model
• A Data Dictionary
• Focus on data shape (cf content)
• Policy, Procedures Policing
• High expectations
• Long time-frames

55
The 80s (and beyond)Trying to make it work
56
Meantime the world was changing

• Shorter cycle times
• Packages and ERP
• Outsourcing
• Commoditization of IT
• New data formats
• More data
• and the Internet

57
Re-inventing Data Management

• What are our objectives?
• How can we best achieve them?

58
The key changes

• Philosophic sell -gt Hard benefits
• Policy -gt Projects
• Global approach -gt Pareto approach
• Telling -gt Listening

59
Where does it hurt?

• and how much?

60
A good project

• Business Benefits
• measurable
• high-profile
• sooner rather than later
• Manageable as a project
• Contribution to infrastructure?

61
A little digression

• I get my kicks out of using my professional
  skills to achieve worthwhile goals

62
Management perspective

• I get my kicks out of achieving my goals by doing
  whatever it takes

63
A choice of directions

• The (data) management perspective
• Where are the big data management hits
  (generically)?
• The professional perspective
• What are we good at?
• Where is that expertise needed?

64
10. So, how does Oregon Pinot taste to
Australians?

• Make no mistake about it, the Cameron Willamette
  Valley Abbey Ridge Pinot Noir 1999 is a brilliant
  Pinot. Highly aromatic with French toast, offal,
  and dried fruit nuances rising from the glass,
  the wine has genuine intensity and concentration
  in the mouth, with a pulpy texture, nuances of
  beetroot, rhubarb, coffee grounds, and lamb's
  fry, and a lingering finish reminiscent of dates,
  raisins, anise and leaf. A staggering Pinot Noir
  - beautifully weighted, pure, intense and
  concentrated - delicious now and will age to 2006
  and beyond. 96.5 - The Australian Review of
  Wines March, 2002