WEB ARCHIVING IN THE BRITISH LIBRARY

1
WEB ARCHIVING IN THE BRITISH LIBRARY

• John TuckHead of British CollectionsFebruary
  2004

2
BRITISH LIBRARY CONTEXT

• ? Created by British Library Act 1972.
• ? National Library of the United Kingdom.
• ? Origins from 1753.
• ? One of worlds greatest research libraries.
• ? 160 million collection items.

3
BRITISH LIBRARY COLLECTION DEVELOPMENT

• ? Building as completely as possible the UK
  national published archive - current and
  retrospective gap filling print and
  electronic.
• ? Collecting research-level English- language
  material published world-wide in the
  humanities, social sciences, STM.
• ? Buying foreign-language material selectively
  
• ? Material acquired through legal deposit,
  voluntary deposit from publishers, purchase,
  donation, exchange.

4
LEGISLATION

• ? Legal Deposit Libraries Act 2003 enabling
  legislation.
• ? VDEP Voluntary Deposit of Electronic
  Publications.

5
DOMAIN.UK

• ? Six-month experiment to select and capture
  100 UK web-sites, 2001.
• ? audit change, loss, links, etc.
• ? determine next steps.

6
DOMAIN.UK Why?

• ? Short-lived nature/changing content of many
  web-sites.
• ? loss of information.
• ? increasing reference to web-sites in
  research/scholarship.

7
DOMAIN.UK Voluntary/Rights Cleared Approach

• ? Voluntary.
• ? Requiring explicit agreement of website
  publishers to take part in pilot.
• ? No public access.

8
DOMAIN.UK Selection

• ? Websites of historical or cultural
  significance.
• ? Cross-section of Dewey Decimal
  Classification.

9
DOMAIN.UK Process

• ? E-mail selected sites for approval and to
  check whether already archived.
• ? Measure sites for links, size, change, etc.
• ? Frequency of visits every three weeks or
  more in some cases.
• ? Supported by those sites approached.
• ? Report recommended scaling up.

10
BRITISH LIBRARY WEB ARCHIVING PROGRAMME

• ? Building on Domain.uk.
• ? BL to play leading role in collecting UK web
  presence in partnership with other
  institutions nationally and internationally.
• ? Selective approach.

11
BRITISH LIBRARY WEB ARCHIVING PROGRAMME contd.

• ? Co-ordinate a snapshot of entire UK web
  presence at occasional intervals.
• ? Achieve more regular capture of limited and
  well-defined range of sites.
• ? Sites judged to be research-level, whether in
  terms of stated intentions of sites
  themselves or of potential to be primary
  resources for research.

12
WEB ARCHIVING PROGRAMME

• ? Comprises a series of complementary projects
  and activities.
• ? Based entirely on voluntary, rights-cleared
  basis pending secondary legal deposit
  legislation.
• ? Aims to embed web archiving within the BL's
  overall collection development policy.
• ? Aims to provide the infrastructure to collect,
  preserve and make accessible web-site
  material alongside material in other formats.

13
WEB ARCHIVING PROGRAMME STRANDS

• ? Four main strands
• ? Definition of collection development policy.
• ? UK Web Archiving Consortium.
• ? International Internet Preservation
  Consortium.
• ? Internet Archive incunabula of the internet.

14
COLLECTION DEVELOPMENT

• ? Appointment of Curator, Web Archiving.
• ? Extension of policy defined for Domain.uk.
• ? Sites of national, historical and cultural
  significance.
• ? Research level now/in the future.

15
UK WEB ARCHIVING CONSORTIUM

• ? Two-year project.
• ? Six partners BL (lead) National Library of
  Scotland, National Library of Wales,
  National Archives, Joint Information
  Systems Committee, Wellcome Library.
• ? Plan to use PANDAS software developed by
  National Library of Australia.
• ? Rights to use individual sites to be cleared
  with rights-holders.

16
UK WEB ARCHIVING CONSORTIUM contd.

• ? Procurement exercise in process to recruit
  supplier to host service.
• ? Intention to let contract in April 2004 and to
  be operational in summer 2004.
• ? Sites to be made accessible to users.
• ? Each partner to collect up to 500 sites per
  year, i.e. 6,000 during project.

17
INTERNATIONAL INTERNET PRESERVATION CONSORTIUM

• ? Project involving national libraries.
• ? Led by Bibliotheque Nationale de France.
• ? Also includes BL, Library of Congress,
  Library and Archives of Canada, Nordic
  countries, Italy, Australia, Internet Archive.

18
INTERNATIONAL INTERNET PRESERVATION CONSORTIUM
contd.

• ? Aims to develop automated web-crawler
  mechanism.
• ? Open-source tools to search web at regular
  intervals matching agreed collection
  development policies.
• ? Working groups in access tools content
  management, deep web, framework, metrics and
  test-beds, researcher requirements.
• ? Developmental at this stage.

19
INTERNET ARCHIVE

• ? Collecting and saving sites since 1997.
• ? Wayback machine.
• ? Legal, technical and procurement issues.

20
SOME CHALLENGES

• ? Defining UK.
• ? Rapid technology change.
• ? Third party rights (not always subject to UK
  law).
• ? Libel/defamation issues.
• ? Software issues / which platform?
• ? Validity of a snapshot.

21
SOME CHALLENGES contd.

• ? Formats for archiving.
• ? Metadata standards.
• ? Archiving look and feel.
• ? Authenticity.