CS461561 Virtual Environments: Principles

1
CS461/561Virtual Environments Principles
ApplicationsIntroduction to Virtual Realityand
Virtools

• 1-17-2006
• Prof. Searleman
• SC375
• jets_at_clarkson.edu

2
Overview

• Administrivia
• Introduction to Virtual Reality
• Four Key Elements
• Virtual World/Virtual Environment
• Immersion
• Sensory Feedback
• Interaction
• VR _at_ Clarkson
• Introduction to Virtools

3
Course Objectives

• To understand the technologies, underlying
  principles, potential, and limits of VR
• To examine the computer hardware and software
  technologies that are used to build VR
  environments
• To learn about immersive, 3D human interaction
  techniques and issues
• To learn about VR applications and explore
  current research in VR
• To gain practical experience in designing,
  implementing, and evaluating a virtual environment

4
Administrivia

• course webpage
• http//www.clarkson.edu/jets/cs461/sp07
• contact info
• SC375, x2377, jets_at_clarkson.edu
• office hours listed on webpage will be held
  either in SC375 or VRlab (SC336)
• course directories
• Atlas (S\courses\Computer Science\CS461)
• AFS (/afs/cu/class/cs461)

5
Textbook

• Building Interactive Worlds by Jean-Marc
  Gauthier, Elsevier, 2005,
• ISBN 0-240-80622-0.

6
Recommended Textbooks

• Introduction to Virtual Reality, by John Vince,
  Springer, 2004, ISBN 1-85233-739-7.
• Understanding Virtual Reality Interface,
  Application, and Design, by William R. Sherman
  Alan B. Craig, Morgan Kaufmann, 2003, ISBN
  1-55860-353-0.

7
Other Resources

• Clarkson Library Books24x7
• Have access to many electronic books
• Go to librarys website register
• Examples
• Developing Future Interactive Systems, 2005, ISBN
  159140418
• Encyclopedia of Virtual Communities, 2006, ISBN
  1591405637
• etc.

8
Software

• Virtools Dev 4.0
• Maya 8.0
• available in
• open labs ITL (SC334), CEC (CAMP 163),
• SwE Lab (Rowley 142), ERC Loft, VR lab (SC336)
• research labs
• VR Lab (SC336), ADVIL (Camp 155)

9
Grading Policy

• 15 Class Presentations
• 25 Homework Class Participation
• 30 Quizzes
• 30 Project
• Attendance Participation in class discussions
  are an essential part of this course, so
  attendance is required and you must be prepared
  for class (i.e. have done the reading in advance).

10
What is Virtual Reality?

• VR is the use of computer technology to create
  the effect of a 3-D environment (virtual
  environment) containing 3-D objects which have a
  strong sense of spatial presence with respect to
  the user.
• In virtual reality you have a sense of and
  interact with three-dimensional things as opposed
  to pictures or movies of things.

11
How is VR Different?

• Objects in the environment have a strong sense of
  spatial presence, creating the effect that the
  objects exist independently of the user.
• Control of and interaction with the environment
  is often through direct manipulation of objects
  in the real world.

12
Different (cont.)?

• The computer interface is hidden in the sense
  that the user interacts with objects in the
  environment rather than a computer which controls
  objects in the environment.
• The user is immersed in the environment, i.e.,
  the user experiences the environment from within.

13
4 Key Elements of VR

• Key Element 1 Virtual World
• Key Element 2 Immersion
• Key Element 3 Sensory Feedback
• Key Element 4 Interactivity

14
Key Element 1 Virtual World

• virtual world
• 1. an imaginary space often manifested through a
  medium
• 2. a description of a collection of objects in a
  space and the rules and relationships governing
  those objects
• key points
• a virtual world is the content of a given medium
• it can exist without being displayed in a VR
  system

15
Key Element 2 Immersion

• Mental Immersion
• - sense of presence state of being deeply
  engaged suspension of disbelief involvement
• Physical Immersion
• - bodily entering into a medium synthetic
  stimulus of the bodys senses via the use of
  technology

16
ImmersionSense of Presence

• The perception of being in a particular space or
  place.

• Physically immerse the participant in a
  computer-generated space.
• Provide computer-generated sensation to one or
  more of the human senses.

17
Key Element 3 Sensory Feedback

• visual
• haptic tactile
• aural
• vestibular (balance equilibrium, acceleration,
  and orientation wrt gravity)
• olfactory, taste, magnetoreception

18
Haptic Interface for Surgery Training
Simulation Scene
VEST Virtual Endoscopic Surgery Training
Haptic instrument interface box
19
Key Element 4 Interactivity

• responsiveness to user actions
• - the ability to affect a virtual world
• - the ability to change ones viewpoint within a
  world
• alternative realities games, computer
  simulations of natural and artificial phenomena,
  flight simulators
• collaborative environments multiple users
  interacting within a virtual world and among
  themselves (multipresence, multiparticipant)

20
Avatars

• avatar 1. a virtual object used to represent a
  participant or physical object in a virtual
  world the (typically visual) representation may
  take any form. 2. the object embodies by a
  participant. 3. adapted from Hindi, meaning the
  earthly embodiment of a deity

21
Interactivity

• Think Virtual do Real digital prototyping as
  key factor for innovation
• Caterpillar, Inc.

22
What is VR? (revisited)

• virtual reality a medium composed of
  interactive computer simulations that sense the
  participants position and actions and replace or
  augment the feedback to one or more senses,
  giving the feeling of being mentally immersed or
  present in the simulation (a virtual world)

23
Related Terms

• Artificial Reality (coined by Myron Krueger,
  1991)
• an artifical reality perceives a participants
  action in terms of the bodys relationship to a
  graphic world and generated responses that
  maintain the illusion that his or her actions are
  taking place within that world
• Virtual Environment
• 1. a virtual world. 2. an instance of a virtual
  world presented in an interactive medium such as
  virtual reality

24
Related Terms

• Cyberspace (coined by William Gibson, 1984
• a location that exists only in the minds of the
  participants, often as a result of technology
  that enables geographically distant people to
  interactively communicate
• Augmented Reality
• a type of virtual reality in which synthetic
  stimuli are registered with and superimposed on
  real-world objects often used to make
  information otherwise imperceptible to human
  senses perceptible

25
Related Terms

• Telepresence
• the ability to directly interact (often via
  computer mediation) with a physically real,
  remote environment from the first-person point of
  view there are no restrictions on the location
  of the remote environment, and there are no
  restrictions on the size of the device used to
  carry out the users commands at the remote
  location

26
VR _at_ Clarkson

• mid 1990s
• - Clarkson Student Chapter of the ACM request a
  VR course VR lab established (SC250)
• - projects include Virtual World Builder, VR
  pong, VR playground, Virtual Reality Wheelchair
  Simulator (VRWC)
• 2003
• - Advanced Visualization Lab (ADVIL, Camp 151)
  established, funded by the Coulter Foundation
  grant for Rehabilitation Engineering
• - projects include Virtual Reality Wheelchair
  Simulator (VRWC), stroke rehabilitation, VPD

27
Advanced Visualization Lab

• CAMP 155 Advanced Visualization Lab (ADVIL)
• Consists of state-of-the-art hardware and
  software used to develop VR-based interventions
  and study their effectiveness
• Collaborating with university, industrial, and
  healthcare partners

28
Overview of Facilities

• Presentation Room/Scenario Lab
• Three wall cube configurable 8'x10' panels
• Suitable for up to 20 simultaneous viewers
• Passive 3D stereoscopic immersion
• HMD Research Area
• Integrated head mounted display (HMD), head
  tracking, data gloves, and Virtools software
• Content Creation/Training Area
• High-end PC workstations with a suite of VR
  software, shutter glasses, etc.

29
Overview of Facilities (cont.)
30
Virtual Reality LabScience Center 250
31
VR Wheelchair Project

• A multidisciplinary team of students designed
  and built an immersive virtual reality power W/C
  simulator modeled after flight simulators

• The user wears an HMD that displays a virtual
  world through which the user has to maneuver the
  virtual power W/C
• Feedback from the joystick and virtual
  environment drive the control system for the
  platform

32
Current VR Rehab Projects

• Patient training
• Virtual wheelchair
• Breathing control, e.g., stress control
• Interventions for individuals post stroke
• Recovery of upper extremity use
• Patient documentation
• Need a consistent user interface for therapists
  to use

33
What is Virtools Dev?

• Virtools Dev 4.0 is a development platform for 3D
  visualization
• It consists of the following components
• an Authoring application
• a Behavioral Engine (CK2)
• a Rendering Engine
• a Web Player
• a Software Development Kit (SDK)

34
Virtools Training Notes
Media Design 3DS, Maya, Lightwave
Sound Design .wav, .mp3
Texture, 2D Interface .jpg, .bmp
Specific BB, Interface Visual C
Integration tool Behavioral tool
35
Getting Help

• Virtools documentation is provided in
  c\Program Files
• \Virtools\Virtools Dev 4.0\documentation
• Reference Guide documentation.chm
• Tutorials
• MiniSite
• CMOS
• VirtoolsResources.rsc

36
Getting Help (cont.)

• Contact me jets_at_clarkson.edu
• Internet Based Resources
• 1. www.virtools.com
• 2. Virtools Newsletter
• 3. www.theswapmeet.com
• 4. User Group mailing list

37
Virtools Packs

• In addition to the basic functionality provided
  by Dev 3.0, there are a number of add-ons. These
  include
• VR pack
• Physics pack
• AI pack
• CAD pack

38
1. Virtools Authoring

• The authoring application allows you to create
  compositions (CMOs) where 3D objects are given
  behaviors and attributes.
• Virtools is not a modeling application. Other
  applications, such as 3D Studio Max can be used
  to create models, which then can be imported into
  Virtools.

39
2. Behavioral Engine

• The behavior engine, CK2, processes the behaviors
  of how each object acts in a given environment.
• Behavior Building Blocks (BBs) are used to
  create behaviors. Virtools has an extensive
  library of BBs.
• Managers are used to support the behavior engine.
  Examples are the LevelManager, ResourceManager
  and the SoundManager

40
3. Render Engine

• This performs the graphics rendering, and can be
  customized with the SDK it consists of 2 parts
• - CK2_3D determines what should be drawn (list of
  visible items)
• - Virtools Rasterizers, which support Direct X
  and OpenGL, draw the items on-screen

41
4. Web Player

• The Web Player allows playback of a virtools
  application via a web browser.
• Virtools compositions (.cmo) can be exported to
  the Web Player (.vmo)

42
5. Software Development Kit

• The SDK allows you to customize the Behavioral
  Engine and the Render Engine. That is, you can
  write your own functions in C and convert them
  to building blocks.
• create new behaviors or modify existing ones
• create new parameter types
• create media plugins to read any media types
• replace the Virtools render engine with a render
  engine of your choice
• create a custom executable file (.exe)
• modify extend the render engine full source
  code to the rendering engine is provided

43
Reminders

• Save your composition often (say every 15 mins or
  so, or when you make a change in the objects)
• Set initial conditions whenever you change any
  setting about an object, before you run the
  composition
• A directory has been set up for the class on
  Atlas
• S\Classes\Computer Science\es305\

44
Basic Concepts

• Level Manager
• organizes the composition
• Schematic
• used to create, edit, and manage scripts a
  script describes how an object behaves

45
Basic concepts

• Building Blocks (BBs)
• a library of behaviors, organized by the type of
  behavior and/or by the type of object to which
  the behavior can be applied
• Data Resources (default VirtoolsResources)
• project management tools you can create a new
  data resource for each project
• Media objects (such as a model, sound, or image)
  are added to the composition by dragging it from
  a Data Resource into the 3D Layout or Level
  Manager

46
Reading Assignment

• for Tuesday,
• explore the internet for interesting articles,
  papers, projects, etc. on Virtual Reality
• bring a list of URLs to class (to turn in), and
  be prepared to show and discuss one of your
  findings in class on Monday