The PLANETS Testbed

1

• The PLANETS Testbed
• DPE, PLANETS and CASPAR 2nd Annual
  ConferenceLisbon, 56 September, 2007
• Max Kaiser, Austrian National Librarymax.kaiser_at_o
  nb.ac.at, http//www.onb.ac.at

2
ToC

• Why do we need Testbeds in Digital Preservation?
• What will the PLANETS Testbed deliver?
• Role of Testbed within PLANETS
• PLANETS Testbed services for Digital Preservation
  community
• Testbed Methodology, Experiment 6-step Process
• Testbed Software Application
• Testbed Components
• User Interface Mock-Up Demonstration
• Next Steps

3
Why do we need Testbedsin Digital Preservation?

• Concern about digital preservation rapidly
  increased in recent years
• Several approaches to digital preservation
  implemented by various programs and projects
• A number of digital preservation tools and
  services (e.g. for migration and
  characterization) developed
• ? Need for systematization, benchmarking and
  evaluation urgent in many fields
• Systematic analysis of current and potential
  preservation strategies
• Evaluation and benchmarking of characterisation
  and migration tools
• Assess the role of emulation tools

4
Why do we need Testbedsin Digital Preservation?

• If we want to
• perform scientific research in digital
  preservation
• evaluate preservation approaches in diverse real
  life settings
• avoid duplication of work
• maximise use of invested resources
• we need a dedicated research environment,
• that allows the systematic execution of
  experiments by different institutions
• with experiments that
• follow a formalized methodology
• are reproducible
• are fully documented and accessible for further
  analysis and comparison

5
A Testbed for Digital Preservation

• Definition
• A controlled environment for experimentation and
  evaluation, with metrics and benchmark content
  that allow comparison of preservation tools and
  strategies

6
PLANETS Testbed

• Platform for objective evaluation of preservation
  tools and strategies
• Load content
• Perform digital preservation experiments
• Design the experiment, select data and tools,
  specify resources and evaluation criteria, run
  the experiment, evaluate results against
  criteria, compare outcomes
• Find support for validation of preservation
  strategies
• Benchmark tools and services
• Will consist of
• Hardware, data storage, PLANETS software, Testbed
  application
• Benchmark and other content
• Will provide services for
• The PLANETS project partners
• The preservation community
• Tool developers and vendors

7
Role of Testbed in PLANETS 1

• Test and validate the technical solutions and
  approaches developed in PLANETS
• provide a controlled hardware and software
  environment for testing and evaluating
  preservation action (migration, emulation) and
  characterisation tools and services
• record experiments data in registries for further
  analysis and comparison
• assist the validation of the effectiveness of
  different digital preservation plans

8
Role of Testbed in PLANETS 2

• Assess the suitability of the approaches across
  real life scenarios in various organisations
• Analyse applicability of the outcomes of PLANETS
  in existing workflows and organisational contexts
• Evaluate their efficiency in providing
  practicable solutions for organisations engaged
  in digital preservation

9
Role of PLANETS Testbed in theDigital
Preservation Community

• In a second phase, the PLANETS Testbed will
• Offer services to organisations outside PLANETS
• Support institutions to test preservation tools
  and services against benchmark content
• Assist institutions to validate their
  preservation plans against their policies
  content profiles
• Enable developers and third party vendors to
  submit tools for benchmarking and certification
• Validate the suitability of their tools in
  preservation workflows

10
PLANETS Testbed Foundations

• Build on work undertaken by the Dutch Digital
  Preservation Testbed and DELOS Testbed Framework
• Integration in PLANETS Interoperability Framework
• Focus on formalisation of testing process design
• Strong focus on comparability and traceability of
  results

11
Main Participants

• Austrian National Library
• Humanities Advanced Technology and Information
  Institute at the University of Glasgow (HATII)
• Austrian Research Centers (ARC)
• National Archives of the Netherlands
• British Library
• Vienna University of Technology
• University of Cologne

12
PLANETS Testbed System

• Web based application
• Provides a consistent and coherent evidence base
  for the objective evaluation of different tools,
  services, and complete preservation plans
• Testbed system provides
• Clearly structured and formal process for
  preparing, executing, and evaluating experiments
• Repeatability of experiments, comparability and
  traceability of results
• Assistance in planning preservation strategies
• Benchmarking of services
• Results of ongoing experiments will improve
  strength of preservation plans by means of
  evaluation
• Will provide essential feedback for the
  development of PLANETS tools, services and
  preservation planning

13
Testbed Methodology

• Experiment Process
• Refined to a focused 6-step process that each
  formal experiment must follow
• PLANETS Testbed experiment consist of a
  consistent methodology for
• Defining its metadata (exploration area, scope,
  criteria, references etc.)
• Selecting or creating a workflow of
  (orchestrated) preservation services
• Evaluating the results against the experiments
  initial goals

14
Start
Testbed Experiment Process 1
15
Testbed Experiment Process 2
4. Go / No go
16
Testbed System Design

• Initial understanding of the required
  characteristics of PLANETS Testbed derived from
  the Dutch Preservation Testbed and the DELOS
  Digital Preservation Cluster
• Additional requirements gathered following a
  clearly structured software design process
• Interviews
• User Scenarios ? 60 Use Cases
• Software Requirements
• IF Requirements (Authentication Interfaces,
  Logging, Versioning)
• Perform Experiment, Define Basic Properties,
  Select A Tool, View Available Services, Handling
  Stalled Experiments
• Non-functional requirements
• Usability, Reliability, Performance, Security,
  Supportability
• Design and Implementation Plan
• High-level design of the Testbed application
• Initial breakdown of the system into components

17
Testbed Software

• Platform-independent system based on Java
  Enterprise Edition (EE)
• Three-tiered system

Handles presentation of user interfaces logic
for dynamically creating content. Based on Java
Server Faces and Facelets
Provides business logic (core functionality).
Based on JavaBeans and JAX web service endpoints
Provides data persistency. Uses data registries
provided by PLANETS Interoperability Framework
18
(No Transcript)
19
Experiment Designer component 1
Testbed Application
Experiment
Data
Services
General
Service Manager
Designer
Evaluator
Im/Exporter
Data Manager
Browser
Administration
20
Experiment Designer component 2

• Guides the user trough design of an experiment
• User Interface for steps 13 of the experiment
  process
• Define basic properties
• Design experiment
• Specify resources
• Select whether experiment is formal or informal
• Select type of experiment
• Migration
• Characterisation
• Emulation
• Preservation plan evaluation
• Execution of a predefined workflow

21
Experiment Designer component 3

• Interaction with several other Testbed
  components, most notably the service manager and
  data manager
• Interaction with Service Manager
• Browse PLANETS Service Registry
• Selection and configuration of (wrapped) tools
  for Preservation Action (PA) and/or Preservation
  Characterisation (PC)
• Service PA/PC tool exhibiting well-defined
  functionality through a PLANETS IF compliant web
  service wrapper
• Interaction with Data Manager
• Interface for handling data input and output via
  API for accessing the IF Data Registry

22
Experiment Designer component 4

• Specification of Resources
• E.g. number of output files
• Specification of Outcomes of an experiment
• Will appear at the evaluation stage of experiment
• Experiment stored as BPEL file and handed over to
  Approval component
• Additional features
• Loading existing experiments
• Viewing of experiments in passive mode
  (read-only)
• Commenting on experiments
• Re-use and Repeating experiments
• Re-use certain aspects of existing experiments,
  or
• Repeat entire experiments with original data

23
Experiment Designer component 5

• Rendering of BPEL workflows
• All experiments are executed as BPEL workflows
• Simple migration and characterisation
  experiments
• BPEL files are created within Testbed based on
  templates
• Up to three orchestrated services (e.g.
  characterisation followed by migration followed
  by characterisation)
• Complex workflows are created by using the IF
  Workflow Designer (or any other BPEL tool) and
  imported into the Testbed
• Component does not process the actual execution
  of an experiment!
• Handled within the PLANETS Interoperability
  Framework by the Workflow Execution Engine

24
Experiment Approval Scheduling 1
Testbed Application
Experiment
Data
Services
General
Service Manager
Designer
Evaluator
Im/Exporter
Data Manager
Browser
Administration
System Monitoring Client
Scheduling
Approval
Persistence
Testbed Backend Components
TB Service Registry Client
TB Data Registry Client
25
Experiment Approval Scheduling 2

• Approval component handles technical approval of
  experiments by the system and/or the
  administrator
• Makes use of information provided by the
  experimenter and by the system monitoring
  component e.g.
• Number of input files
• Estimated number of output files
• Current CPU / Memory usage
• Threads / Processes
• Available Disk Space
• Scheduling component allows administrator to
• Select a date/time for an experiment to start
• Assign a maximum run time
• Check progress of experiments
• Stop experiments

26
Experiment Evaluation 1
Testbed Application
Experiment
Data
Services
General
Service Manager
Designer
Evaluator
Im/Exporter
Data Manager
Browser
Administration
27
Experiment Evaluation 2

• View files and results that have been produced
  during the experiment
• Capture experiment conclusions in a structured
  way
• Evaluate results based on the criteria defined
  during experiment design process

28
Experiment Importer / Exporter 1
Testbed Application
Experiment
Data
Services
General
Service Manager
Designer
Evaluator
Im/Exporter
Data Manager
Browser
Administration
29
Experiment Importer / Exporter 2

• Create BPEL files from existing experiments and
  make them available for download
• Upload BPEL files to the Testbed
• Import whole experiments
• Import predefined workflows as part of the
  experiments design process
• Check validity of the BPEL file structure
• Check data contained within BPEL files
• Download experiment results
• Information on experiments in PDF format created
  on-the-fly
• Basic properties, tools used, data used,
  information about approval process, information
  about execution of experiment, experiments
  results, validation of experiment

? local demo
? WWW demo
30
ExperimenterPages
Login / Logout
Homepage
Browse Experiments
Browse Services
My Experiments
New Experiments
Browse Data
Import Experiments
View Experiment Details
Experiment Designer
View Metadata
Run Experiment
Annotate Experiment
Export Experiment Setting
Upload Data
Evaluate Experiment Results
Edit Conclusions
31
Conclusions / Next Steps

• PLANETS Testbed controlled environment for
  experimentation that allows benchmarking of
  preservation tools, services and strategies
• First release to be expected in February / March
  2008
• Will be used to experiment with the preservation
  tools and services developed in PLANETS
• In a 2nd Phase (by end 2008) Testbed will offer
  services to organisations outside PLANETS
• Support institutions to test preservation tools
  and services against benchmark content
• Assist institutions to validate their
  preservation plans against their policies
  content profiles
• Main Testbed Instance hosted by University of
  Glasgow (HATII)
• Dedicated hardware with sufficient processing
  power and storage space

32
The PLANETS Testbed
Questions?

• Max Kaiser, Austrian National Librarymax.kaiser_at_o
  nb.ac.at, http//www.onb.ac.at

Thank you!