Meme Media and a Meme Pool

1
??????????(7)

• ?? ?

2
Multimedia Application Framework
3

• Typical application functions are also generic
  functions. They form an application library.
• Once these functions in the application library
  are implemented as pads, they can be combined
  with other pads to easily develop sophisticated
  applications.
• For a typical application, we can extract generic
  functions as pads and specify their typical
  construction structure.
• For each typical application, a set of generic
  pads and their typical construction structure
  form the development framework, called an
  application framework.
• This chapter shows frameworks for multimedia and
  hypermedia applications. These frameworks include
  pads to represent texts, images, tables, charts,
  and movies. They also include pads that allow a
  user to cover a portion of a multimedia pad in
  order to articulate an object shown in this area.

4
Component Pads for Multimedia Application
Frameworks

• Text processing pads
• Tables and figures
• spreadsheet
• chart tool
• drawing tool
• Multimedia pads
• image
• sound
• MIDI
• video

5
Composite Spreadsheet Pad
6
Composite Chart Pad with Composite Bar Meter Pads
7
Articulation of Objects
8
Articulation of multimedia objects

• In multimedia systems, we have to deal with
  various types of objects.
• They should be first classified into two
  categories, i.e., container objects and content
  objects.
• Container objects are container media that carry
  content information. Books, pages, cards, display
  windows, and communication packets are all
  examples of container objects. They provide the
  respective structures and operations of these
  containers. A media object is defined as a
  container object with its content objects.
• Content objects are further classified into two
  categories. Some of them are clearly articulated,
  i.e., easily machine-identifiable from their
  representations, while others are not.

9
(No Transcript)
10

• Our main concern here is how to articulate
  non-articulated multimedia objects.
• Multimedia systems should allow us to directly
  select and to directly manipulate not only
  articulated objects but also non-articulated
  content objects.
• The most widely used general solution is the use
  of a reference-frame object for each content
  object.
• For a recorded music or a speech, a
  reference-frame object defines the shortest time
  segment that includes one of the music or speech
  portions you want to identify. This
  reference-frame object indirectly specifies the
  corresponding music or speech portion.

11

• For an image, a reference-frame object defines
  the minimum rectangular area that covers one of
  the content objects you want to identify in this
  image. This reference-frame object indirectly
  specifies the corresponding area of the image.
• Reference-frame objects are articulated objects.
• Time segments work as temporal reference frames,
  while rectangular areas work as spatial reference
  frames. A reference-frame object for a cut in a
  movie defines a time segment.
• For an object appearing in a movie, its
  reference-frame object defines a mobile
  variable-size rectangular area that minimally
  covers this object in every video frame showing
  this object.

12

• Pads in IntelligentPad are suitable for the
  representation of the container objects, media
  objects, and reference frames in multimedia
  systems.
• Pads represent multimedia containers as windows
  or window widgets. They can be easily combined to
  graphically represent books, pages, cards, and
  compound document frameworks.
• Since pads can hold any kind of information and
  can provide slots to access the contents in
  various ways, they can easily represent media
  objects with sufficient interface to access their
  content objects.

13

• For the access of non-articulated content objects
  in a media object, we can provide this media
  object with a special slot named referenceFrame
  that receives the location and size of a
  reference frame and returns the corresponding
  portion of its content information. If the
  content is an image, then the return value is the
  portion of this image specified by the reference
  frame. A set message with the location and size
  of a reference frame as its parameter value is
  used to send this parameter to the
  referenceFrame slot. After this operation, a
  gimme message is used to read the corresponding
  portion of the content information.

14
(No Transcript)
15

• Spatial reference frames can be represented as
  transparent pads that cover the target content
  objects. They can be pasted on top of their
  target media object pads to minimally cover their
  target content objects.
• These reference-frame pads provide slots as
  necessary interface to the target contents. These
  slots include name slot to hold the name of the
  content object, location slot to hold its
  relative location on its parent pad, and extent
  slot to hold its size. The oid (object
  identifier) of this reference-frame pad
  semantically works as the oid of the
  corresponding content object. Any pads with the
  above-mentioned functionality can work as
  reference-frame pads.

16

• IntelligentPad, however, provides some standard
  pads for usage as reference-frame pads. These
  include anchor pads and viewer pads.
• An anchor pad allows us to register a reference
  pointer to any composite pad. When mouse-clicked,
  it loads this registered pad and pops it up on
  the screen. Anchor pads without any further
  functionality are not connected to any of the
  slots provided by its parent media object pads.
• A viewer pad, on the other hand, is connected to
  the referenceFrame slot of the media object
  pads. It has the contents slot to hold the copy
  of the corresponding portion of the content
  information. It also works as an anchor pad.

17
Operations and Relations over Multimedia Objects

• Articulated objects are units of operations.
  Among the operations in multimedia systems are
  the following
• (1) File operations (save, load).
• (2) Edit operations (create, delete, edit).
• (3) Quantification, i.e., condition
  specification in database queries.
• (4) Link operations.
• (5) Context specification.
• (6) Composition operations.
• (7) Overlay operations.

18

• Edit operations are applied to each single
  object, while composition operations are applied
  to a set of multiple objects to combine them and
  to obtain a compound object.
• Each object may be referred to in specifying some
  of its properties.
• Each object may work as an anchor. It may link
  itself to another object, or work as a
  destination anchor that is pointed to by another
  object.
• The specification of an object also specifies its
  context that is also an object. For example, we
  can specify those figures showing a square as one
  of its components. Here we only specify a shape
  object. Those figures are specified as the
  contexts of such objects.
• Multimedia documents and components can be
  overlaid in multiple layers using the ?-channel
  technology.

19

• These operations are applicable not only to
  articulated objects, but also to reference frames
  of non-articulated objects.
• Original images, movies and sounds are kept
  unchanged in files.
• File operations save and load only
  reference-frame objects.
• Edit operations are applied only to the copies of
  the portions selected by reference-frame objects.
  These copies become properties of the
  reference-frame objects.

20

• Quantification is also applied to reference-frame
  objects. It specifies only those properties that
  can be easily calculated in real time. Otherwise,
  such quantification cannot be processed within a
  reasonable amount of time. In images, spatial
  relationships between two reference-frame objects
  are such properties. In movies, we can treat
  spatio-temporal relationships among
  reference-frame objects.
• Each reference-frame object can work as an
  anchor. It can link its corresponding
  non-articulated object to another object, or it
  works as a destination anchor that is pointed to
  by another object.
• Each reference-frame object can tell which object
  is its context.
• Reference-frame objects can be combined to a
  compound object, and also they can be overlaid
  with multimedia documents and components.

21
(No Transcript)
22
Application linkage

• Application linkages in object-oriented systems
  are represented by either the object containment
  or the object wiring. The object containment
  embeds a media object in another media object. It
  just defines a geometrical containment
  relationship or further defines functional
  linkage between the two media objects.
• The object containment can be easily defined in
  IntelligentPad by the paste operation. The pasted
  child pad may just sit on the parent pad without
  any functional linkage, or it may be connected to
  one of the slots provided by the parent pad to
  establish a functional connection.
• An extreme case of the object containment in
  IntelligentPad is the pasting of one translucent
  pad, defined as a pad whose background color is
  set translucent, on another pad of the same size.
  IntelligentPad allows us to make any pad
  translucent. This extreme case is used to overlay
  multiple layers of information that are mutually
  geometrically related with each other. Computer
  mapping systems conventionally use such overlay
  representation.

23

• The object wiring uses wiring links to connect
  two spatially separated objects. Navigation links
  (cold, warm and hot links) in hypermedia and
  transclusion in Xanadu are examples of the object
  wiring.
• In general, messages can be exchanged between the
  two objects through the object wiring.
• IntelligentPad uses the shared copy operation for
  the object wiring. It also provides the wiring
  pad. Its shared copy works together with the
  original as two terminals connected by a cable.
  These two terminals transport any value from one
  to the other. The cable may work as either a
  bidirectional channel or a unidirectional
  channel. These two modes can be easily selected
  by the specification of its mode slot.
• Anchor pads implement cold navigation links,
  i.e., links without functional linkage facility.
• Wiring pads implement warm and hot navigation
  links. Hot navigation links provide automatic
  update propagation along them, while warm
  navigation links propagate updates along them
  only when requested.

24
Hypermedia Framework

• A hypermedia system consists of a set of nodes
  and a set of links.
• Each node is a compound document, while each link
  relates a component object in one node either to
  another node or to another component in another
  node.
• The source component of a link is called its
  source anchor, or simply its anchor, while its
  destination component is called its destination
  anchor.
• Either users or application system providers span
  these links to define associative, quotational,
  annotative, or referential relationship between
  nodes.
• Users use these links to navigate from one node
  to another node.
• In hypertext systems, forerunners of hypermedia
  systems, nodes were just simple text documents,
  and their components were words and phrases.
• In conventional hypermedia systems, nodes are
  compound multimedia documents, and their
  components are words and phrases in texts,
  various kinds of embedded multimedia objects, and
  their articulated objects.

25

• These embedded multimedia objects include charts,
  tables, figures, images, movies, and sounds.
• Reactive hypermedia systems are those with
  reactive nodes. Their nodes may include
  components that change their states against user
  operations. Interactive charts and spreadsheets
  are such reactive components.
• The introduction of reactive nodes has extended
  the roles of links.
• A cold link is a navigational link.
• A hot link is used not only to navigate from its
  source component to its destination node, but
  also to automatically propagate the state change
  of its source to its destination.
• A warm link is also used both for the navigation
  and for the update propagation, but it performs
  the update propagation only when it is requested
  by the destination node.

26

• IntelligentPad has completely removed the
  distinction between multimedia documents and
  application tools based on the compound document
  architecture.
• Here we consider reactive hypermedia systems in
  which any composite pad works as a node.
• The introduction of reactive nodes also requires
  that the function of their links be extended.
• These functional links should be objects. In the
  IntelligentPad architecture, all the objects that
  we can directly manipulate should be basically
  provided as pads. Therefore, links should be
  basically provided as pads.

27

• There are, however, articulated objects that
  cannot be represented by pads. These include
  character strings on text pads, composite objects
  drawn on a drawing pad, lines, points, and bars
  on a chart pad.
• In order to make them work as source anchors of
  links, the linking function must be provided by
  their base pads, i.e., by their text pads,
  drawing pads, and chart pads.
• When considered as non-articulated objects, some
  of these articulated objects that cannot be
  represented by pads can be indirectly treated by
  the use of transparent reference-frame pads that
  minimally cover these objects.

28
a
29
(No Transcript)
30
Anchor Pad
31
Hypermovie Framework (1)

• For a non-articulated object in a movie, the
  anchor pad should be able to change its size and
  location to cover this object wherever it
  appears.
• Such an anchor needs to know the frame changes of
  the underlying movie.
• We call such an anchor pad a hypermovie anchor
  pad.
• The state of a movie pad is the frame number. A
  movie pad has a slot to hold the current frame
  number. Whenever a frame change occurs, a movie
  pad issues an update message to its child pads.
  
• Hypermovie anchor pads can use this update
  message to read out the current frame number from
  its parent movie pad.

32
Hypermovie Framework (2)

• In order to change its size and location for
  different frames, a hypermovie anchor pad keeps a
  table that stores its size and location in some
  sampled frames. It linearly interpolates its size
  and location in the other frames.
• This table is constructed by step-by-step manual
  instructions to this hypermovie anchor pad,
  regarding of its size and location in all the
  sampled frames.
• In each sampled frame, the hypermovie anchor pad
  is manually adjusted with respect to its size and
  location to minimally cover the target object in
  the movie.
• Each frame may contain more than one hypermovie
  anchor pad.

33
Hypermovie Framework (3)
34
Hypermovie Framework (4)