CIANT Testbed

1
CIANT Testbed

• Michal Masa, Viliam SimkoPavel
  SmetanaCIANTCentre International pour l' Art et
  les Nouvelles Technologies

2
CIANT DataSets

• Visually oriented (Complementary to audio -
  IRCAM/INA)
• The easy one
• video art archive AMANT
• - video, audio, metadata
• The 3D one
• Langweils model of Prague
• - polygons, textures
• The complex one
• V.I.R.U.S. performance
• - mixture of new media

3
CIANT DataSets

• The developed one
• New Media Performance
• GOLEM

4
Testbed goals

• Demonstrate that we can document (digitally) and
  preserve a new media performance in a way that
  people can re-perform it after a long-term
• See Long-term digital preservation of a new media
  performance "Can we re-perform it in 100 years?
  in Mays issue of International Preservation
  News published by IFLA Core Activity on
  Preservation and Conservation (PAC) group
  accessible online here
• http//www.ifla.org/en/publications/international-
  preservation-news
• http//www.ifla.org/files/pac/IPN_47_web.pdf

5
ENTERmultimediale 2, Prague, 10 th MAY 2005
6

• full multimedia stage
• interdisciplinary interactive performance

7

• two dancers moving in a variable space delimited
  by laser rays
• motion-capturing system transfers the real into
  the virtual world

8

• interdisciplinary multimedia content
• two dancers
• four video projections(three screens the
  stage)
• laser and lights
• music, sound
• fog

9
Video example 3,5
10
ESA testbed
Some processing
11
UNESCO testbed
Photo scan
2D images lots of metadata(position, angle,
calibration, etc.)
LEVEL-0
Computer vision algorithms
3D reconstruction point cloud
LEVEL-1
Special tools lots of manual work
3D reconstruction polygons
LEVEL-2
Simple conversion tools
End user product VRML
12
Common idea

• preserve the final product (image)
• preserve the workflow - the processing of the
  raw data
• preserve the raw data coming from satellites and
  radars and other sources
• future software/hardware/machines are expected to
  deliver better, richer, more interesting final
  products than those we are able to produce now
  with the current HW/SW

13
Artistic testbed - simple
Artistic testbed - complex
New Media Performance
14
HW/SW develops rapidly

• Problem New media performance is called new
  media since it is based on the latest technology
  available
• Question How are they going to re-perform the
  preserved performance in the future?
• on the latest hardware available will look and
  feel probably rather different
• on something to provide the same look and feel we
  documented and preserved today
• We do not know. They should be able to do both.

15
HW/SW develops rapidly

• Examples
• future LCD screens will probably deliver brighter
  picture, larger resolution, etc. Who needs a
  projector then?
• future motion capture device will have a smaller
  latency, greater precision and will be probably
  based on a completely different medium than today

16
HW/SW develops rapidly

• The artist is limited by the capability of the
  current HW/SW
• He/she probably does not insist on using a
  particular projector/keyboard/screen/motion
  capture device/audio device
• As long as it shows the image
• As long as it captures the movement of the dancer
• As long as it produces the sound
• As long as it does what it was supposed to do
• There are exceptions concept is important

17
Importance of the raw data

• If we Now
• focus also on preserving the raw data
  abstract concepts and ideas, not just their
  particular expressions given and constrained by
  the current HW/SW
• Then in the Future
• instead of emulating the expressions (pieces of
  SW in obsolete programming languages running on
  obsolete HW, etc.) we will be able to adapt the
  original ideas to a brand new systems

18
The need for a score

• Musical score has proved to work well for musical
  performances
• We need something similar for new media
  performances
• Digital
• Short-term
• Almost no standards comparing to the scientific
  domain

19
Similar efforts

• We turned out to be on the same boat with other
  projects dealing with preservation of new media
  art like
• Archiving the Avant-Garde
• Variable Media Network
• Forging the Future
• DOCAM, etc.
• However, our focus is slightly different
• installation vs. performance
• Installation one computer, one piece of code,
  one tool
• Performance plethora of HW/SW plus human actors
• Long-term vs. Short-term preservation

20

• Example 1 (before CASPAR)

21

• idea
• reflection on the transmission of multiple
  processes between humans themselvesand between
  humans and machines
• interrelations between diverse environments and
  the penetration of man or robotic intelligence
  into locations where the limits of the body and
  the perception control are getting blurred

22

• elements to preserve

23
script / scenario
24

• roles of artists
• direct the performance
• prepare scenario
• prepare choreography
• prepare video assets
• prepare audio assets

• design 3D assets
• design stage
• design stage lighting
• design costumes
• compose music
• performers(dancers, musicians...)

25
stage elements
26
Examples of used software

• Laser FIESTA
• MoCap Polhemus System, MCspoofer (recorder
  player)
• VR engine UT2004 (version) data
• Audio, Video (MAX/MSP, EyesWeb)

27

• assets
• stage properties, requisites (physical, 3D)
• audio tapes (analog, audio)
• MP3, MIDI files (digital, audio)
• DMX files (digital, light)
• AVI, MPG files (digital, video, audio)
• X3D, VRML, UT2004 (digital, 3D)
• JPG, GIF (digital, video)

28

• instructions
• scenario in PDF file (digital, text)
• scenography in PDF file with illustrations
  (digital, text, image)
• choreography in paper documents (physical, text,
  image)
• musical score (physical or digital, audio)
• audio and video software written by the artists
  EyesWeb patches, MAX/MSP patches, specific C
  applications (digital, audio)
• modifications of game engines scripts in
  UnrealScript (digital, video, audio, 3D)

29

• equipment
• speakers, microphones, cables (physical, audio)
• projection screens, projectors, displays, video
  cameras, cables (physical, video)
• audio mixer (physical, audio)
• video mixer (physical, video)
• motion capture device (physical, 3D)
• computer equipment hardware (physical)
• imaging software tools Photoshop (digital,
  image)
• audio software tools - MAX/MSP, EyesWeb (digital,
  audio)
• video software tools - EyesWeb, Resolume, vvvv,
  DirectShow (digital, video)

30
(No Transcript)
31
V.I.R.U.S. UML Deployment Diagram
32
V.I.R.U.S. CIDOC-CRM model

• V.I.R.U.S. model http//research.ciant.cz/caspar/
  virus-model/ (start browsing the model from
  Individuals -gt V.I.R.U.S. Performance)
• Reworked V.I.R.U.S. model http//research.ciant.c
  z/caspar/virus-model2/ (start browsing the model
  from Individuals -gt The V.I.R.U.S.
  Performance)RDFS http//research.ciant.cz/caspar
  /virus-model2.rdfs
• Software Eclipse 3.2.0 plug-in TopBraid
  2.0.6Ontology modelling language OWL 1.0
  DLOntology CIDOC-CRM 4.2 Task4301 Extensions
  CIANT Extensions (Role, 3D_Model, 3D_Animation,
  Animation)Technical description of the model and
  OWL http//research.ciant.cz/caspar/virus-cidoc-e
  xplained.pdf

33

• conclusion
• Will be anybody able to re-perform it after 100
  years?
• Difficult to preserve only after a year after the
  premiere
• Pieces scattered on various media possessed by
  various people
• Reluctance to provide information, materials,
  etc.
• Solution
• New performance Sheer Curation approachCut
  down version of GOLEM performance

34
GOLEM

• Example 2 (during CASPAR)

35
GOLEM

• 30 minutes dance performance telling the story
  of artificial man
• based on the use of advanced audiovisual
  technologies such as granular sound synthesis,
  real-time 3D environments and computer systems
  for capturing movement.
• The dancer becomes both on-the-fly choreographer
  and co-author of the sound composition. He pilots
  the sound part as well as influences the virtual
  projected scenography.
• Produced as a part of IMMEDIATE (Immersive Media
  Dance Integrating in Telematic Environments)
  project supported by EC Culture Programme

36
GOLEM scenario

• First step document the performance in an OAIS
  compliant way RepInfo and PDI toolkit needed
• Requirements
• Easy to use artists and curators
• Open source
• Free
• Multi platform / online
• Secure
• Flexible
• Collaborative
• ?

37
TWiki

• TWiki (www.twiki.org)
• Meets all the requirements
• It is a structured wiki additional means to
  structure the content
• Hierarchical topics
• Database-like forms associated with topics
• Enables users to create their own wiki
  applications
• Access rights

38
GOLEM _at_ TWiki

• OAIS compliant documentation of GOLEM
• https//research.ciant.cz/twiki42/bin/view/Immedia
  te/WebHome
• Second step automatic export toCIDOC-CRM ? SWKM
  component

39
Process modelling
40
GOLEM model
41
RepInfo Validation

• Third step How do we demonstrate that the
  performance has been preserved?
• The proof is the ability to re-perform it
• Difficult to demonstrate

? Performance Viewer tool
42
CIANT testbed
43
obsolescence of SW

• Fourth step
• 3D rendering software is based on commercial VRML
  browser BS Contact, which is no longer available.
• Solution
• Preserve the 3D original software including the
  BS Contact VRML browser
• Re-implement the 3D rendering software to keep up
  with the latest technology and aesthetical
  requirements
• write a new software based on open source game
  engine Ogre3D
• migrate the VRML 3D model to the new engine