Government Information Preservation Working Group

1
Government Information Preservation Working Group

• Highlights of Digital Preservation Survey
• December 16th 2003
• Oliver Slattery
• Information Access Division
• National Institute of Standards and Technology

2
Need for Digital Preservation

• .crucial.critical.essential....important
• Legally required.
• Principle role of agency/central to agency
  mission.
• 30-100s years
• Archive distribution and central requirements of
  data assets.
• Important for department to provide secure,
  accessible, archival information on QC testing
  and other technical work.
• Continuity of operations.
• The need to stay current.
• Records are permanent.

3
Challenges in the next 5 years

• Obstacles
• Large/increasing volumes of data
• Multiple formats / format compatibility
• Quality/capacity of media
• Storage space
• Getting customers to use latest media
• Upgrading infrastructure/equipment procurement
  (cost and time)
• Ensuring authenticity

• Specific challenges/tasks
• Websites (archiving of)
• Preservation with online/on demand access
• Coordinating/integrating preservation procedures
• Migration of current archive
• Ensuring authenticity

• Other concerns
• Management/record keeping
• Defining digital preservation
• Test capabilities/equipment (procurement cost
  and time)
• Uniformity among suppliers of digital documents
• Same document through every phase of life cycle

4
Current strategy and its limitations

• Control Formats limit must be done at
  creation.
• Use tapes to store and distribute data limit
  tapes are expensive, will soon no longer be made
  and are susceptible to errors 
• DLT, CD/DVD ROM. PDF/TIFF limit size, cost,
  compatibility
• Networked computer disk drives and backup
  magnetic media. Systems include Access databases
  and laboratory test database called Testream
  (SQL) limit Access portion not secure or
  traceable. Backup may be insufficient. No
  assurances of data accessibility if formats
  change. 
• Coordinate the preservation of born digital items
  Limit resources
• Currently migrating from analog to digital. Still
  acquire in analog, but send out to customers in
  digital. Moving towards full digital acquisition.
  Limit storage space and budget. Process is
  slow.
• From archive to CD/DVD for distribution. Deep
  archive facilities for long term storage.
  Limit Large data sets too big for current
  archive media capacities.
• HD media (tapes) such as DLT and SDLT ect.,
  Servers/LAN, some web based access. Limit
  Network throughput is small nearing limitation.
  Automation not available for HD preservation
  work.

5
Research we want to see

• Information Quality and Access
• Authentication
• Accuracy of rendering.
• Universal media.
• One size fits all.
• Safeguards to ensure authenticity and version
  control of archived docs
• PDF for archiving
• Universal access tool.
• Practices and procedures. Digital is easy to
  change but hard to detect changes!
• Standards analysis and development.

• Reliability
• Media durability
• Physical testing and artificial aging of digital
  media to predict durability.
• Preservation media.
• Testing and evaluation of media. Important to
  share results.
• Large capacity, reliable archive media.
• Development of media analysis tools.
• Detect changes of error rates in media.
• Classical issues such as video archiving,
  microfilm preservation issues, environmental
  studies.

• Procedures/Best Practices
• Methods for migration of legacy information.
• Safeguards to ensure authenticity and version
  control of archived docs
• Practices and procedures. Digital is easy to
  change but hard to detect changes!

• New/Alternative Technologies
• Fiber channel hard drives
• Blue-ray discs
• Solid state storage
• Universal media.
• Keeping an eye on future technologyhardware,
  software, formats.
• Large capacity, reliable archive media.

• Formats
• PDF for archiving
• Preservation media.
• Universal access tool.
• Preservation format.
• Format interconversion.

• New/Alternative Technologies
• Fiber channel hard drives
• Blue-ray discs
• Solid state storage
• Universal media.
• Keeping an eye on future technologyhardware,
  software, formats.
• Large capacity, reliable archive media.

6
Types of data

• Data files
• Microfilm
• Multimedia/web
• Imagery (Scanned, digital)
• Documents (mixed/compound, digital)
• Software
• Video

• Laboratory results (from equipment)
• Records
• Graphics/Drawings
• Support data
• Binary
• Binary seismic
• Binary well logs
• Text
• Audio

Bold multiple hits
7
Capture and Collection

• Absolute Maximum 50
• Very important 5 points
• Quite important 4 points
• Somewhat important 3 points
• Not especially important 2 points
• Not at all important 1

8
Capture and Collection

• The maximum number of hits per level of
  importance is 10.
• The minimum number of hits per importance level
  is 0.

9
Capture and Collection

• The maximum number of hits per level of
  importance is 10.
• The minimum number of hits per importance level
  is 0.

10
Capture and Collection

• The maximum number of hits per level of
  importance is 10.
• The minimum number of hits per importance level
  is 0.

11
Capture and Collection

• The maximum number of hits per level of
  importance is 10.
• The minimum number of hits per importance level
  is 0.

12
Capture and Collection

• The maximum number of hits per level of
  importance is 10.
• The minimum number of hits per importance level
  is 0.

13
Storage Media

• Absolute Maximum 50
• Very important 5 points
• Quite important 4 points
• Somewhat important 3 points
• Not especially important 2 points
• Not at all important 1

14
Storage Media

• The maximum number of hits per level of
  importance is 10.
• The minimum number of hits per importance level
  is 0.

15
Storage Media

• The maximum number of hits per level of
  importance is 10.
• The minimum number of hits per importance level
  is 0.

16
Storage Media

• The maximum number of hits per level of
  importance is 10.
• The minimum number of hits per importance level
  is 0.

17
Storage Media

• The maximum number of hits per level of
  importance is 10.
• The minimum number of hits per importance level
  is 0.

18
Storage Media

• The maximum number of hits per level of
  importance is 10.
• The minimum number of hits per importance level
  is 0.

19
Storage Media

• The maximum number of hits per level of
  importance is 10.
• The minimum number of hits per importance level
  is 0.

20
Storage Media

• The maximum number of hits per level of
  importance is 10.
• The minimum number of hits per importance level
  is 0.

21
Data and Storage Management

• Absolute Maximum 50
• Very important 5 points
• Quite important 4 points
• Somewhat important 3 points
• Not especially important 2 points
• Not at all important 1

22
Data and Storage Management

• The maximum number of hits per level of
  importance is 10.
• The minimum number of hits per importance level
  is 0.

23
Data and Storage Management

• The maximum number of hits per level of
  importance is 10.
• The minimum number of hits per importance level
  is 0.

24
Data and Storage Management

• The maximum number of hits per level of
  importance is 10.
• The minimum number of hits per importance level
  is 0.

25
Data and Storage Management

• The maximum number of hits per level of
  importance is 10.
• The minimum number of hits per importance level
  is 0.

26
Data and Storage Management

• The maximum number of hits per level of
  importance is 10.
• The minimum number of hits per importance level
  is 0.

27
Access and Distribution

• Absolute Maximum 50
• Very important 5 points
• Quite important 4 points
• Somewhat important 3 points
• Not especially important 2 points
• Not at all important 1

28
Access and Distribution

• The maximum number of hits per level of
  importance is 10.
• The minimum number of hits per importance level
  is 0.

29
Access and Distribution

• The maximum number of hits per level of
  importance is 10.
• The minimum number of hits per importance level
  is 0.

30
Access and Distribution

• The maximum number of hits per level of
  importance is 10.
• The minimum number of hits per importance level
  is 0.

31
Access and Distribution

• The maximum number of hits per level of
  importance is 10.
• The minimum number of hits per importance level
  is 0.

32
Thanks

• Thanks to all who replied.
• Survey creation
• Jerry McFaul, Ollie Slattery, Victor McCrary,
  Fred Byers , Xiao Tang, Rich Vining.