Title: Meme Media and a Meme Pool
1??????????(9)
2Meme Pool Architectures
3- In order to make pads work as memes in a society,
we need a publication repository of pads where
people in this society can publish any composite
pads, down-load any published pads into their own
local environment for their own use, recombine
these down-loaded pads with each other or with
their own pads to compose new pads, and replicate
these composed pads to publish them for other
peoples use. - The more useful or the more interesting pads are
likely to be replicated more frequently and
distributed more widely. Here we can see all the
genetic operations necessary for genetic
evolution, namely, replication, recombination,
mutation, and natural selection. In this sense,
this repository works as a meme pool of pads that
work as meme media.
4- In this chapter, we will propose several
different architectures for the publication and
reuse of pads through the Internet. - These architectures provide different levels of
pad publication facilities ranging from a pad
catalog on a Web page to a marketplace of pads. - The former provides a catalog of pads in which a
click of a pad picture leads to the download of
the corresponding pad from a remote pad server. - The latter provides a virtual piazza where people
can upload and download pads to and from the
corresponding server through drag-and-drop
operations. - This chapter also shows how meme media
technologies can be directly applied to Web
contents to make the Web work as a meme pool.
5Pad Publication Repository and WWW
- The evolution rate of a meme pool depends on the
frequency and the variety of memetic
recombination that are ruled by several factors. - These factors include
- the number of people accessing this pool,
- how easy for them to recombine memes, and
- how often for them to encounter interesting
memes.
6- There are three corresponding ways to accelerate
the meme pool evolution. - To increase the number of people accessing the
pool, we need to establish a worldwide repository
of memes, and to make it easy to access this
repository. - For the ease of meme recombination, we need to
develop a user-friendly editing system for
end-users to compose, to decompose, and to
recombine memes. - Finally, to increase the chance for people to
encounter interesting memes, we need to develop a
good browser or a good reference service system
for people to search the repository of memes for
what they are interested in.
7- While the second requirement is fulfilled by the
meme media architecture, the first and the third
require a worldwide repository of memes, and a
good browser and/or a good reference service
system to access this repository. - In other words, we need to organize a marketplace
where people can publish their achievements as
pads, browse through all the pads published by
other people, and reuse any of these pads for
their own use in their own IntelligentPad
environments. - Pads as meme media require another medium that
works as their marketplace.
8- In addition to the above-mentioned three ways of
accelerating the meme-pool evolution, we should
not forget the fourth way that is based on a
hypothesis of a punctuated equilibrium in
population genetics. - For the acceleration of the meme-pool evolution,
it is not a good strategy to provide a single
monolithic large meme pool. - Instead, it is better to provide such a meme pool
that enables people to dynamically develop
smaller sub-pools for sub-communities, to
cultivate a local culture in each of them, and to
dynamically merge some of them. - The WWW fits this strategy due to its
non-monolithic complex web structure.
9- To bring a punctuated equilibrium to the meme
pool of pads, this marketplace needs to have a
non-monolithic complex structure such as the Web.
- Our IntelligentPad project set up the following
four sub goals in 1993 to develop such
marketplace systems.
10- The first system uses WWW and its browser such as
Netscape Navigator or Microsoft Internet
Explorer. While WWW works as a worldwide pad
repository, Netscape Navigator or Internet
Explorer provides a hypermedia catalog of pads to
navigate this repository. - Each Web page describes various pads using texts
and images. - Pad names in textual descriptions and
display-snapshot images of pads work as special
anchors. A mouse click on one of these special
anchors will pop up an IntelligentPad window with
a real working copy of the requested pad. - The original of this pad may be stored anywhere
in the world. It is stored there using the
exchange format representation. - When the anchor is clicked, the browser issues a
file transfer request to the HTTP server at this
remote site. After a while, the local Web browser
receives this file, and then invokes the
IntelligentPad system to reconstruct the pad
using its exchange format representation.
11- The second system provides a special pad called a
URLAnchorPad. - When mouse-clicked, a URLAnchorPad directly
issues a URL (Universal Resource Locator) to the
HTTP server to retrieve another composite pad
from a remote site. - A URLAnchorPad, when pasted on another pad, works
as a link anchor in a hypermedia network spanned
across the Internet.
12- The third system provides a new Web browser that
basically works in a similar way as Netscape
Navigator and Internet Explorer. This browser is
also a pad called an HTMLViewerPad. It allows us
to publish any composite pads by embedding them
in arbitrary Web pages. - You can publish any documents and tools as
composite pads embedded in Web pages. - You may even publish a database client tool on a
Web page. - Since an HTMLViewerPad is also a pad, you may
embed a Web page in another Web page. - Different from Java applets, embedded pads can be
copied and locally reused. - Different from ActiveX controls, embedded pads,
after dragged out into a local environment from a
Web page, can be decomposed and hence locally
reused in recombination with other local pads, or
even with those embedded in other Web pages.
13- The fourth system integrates the accounting,
billing, and payment mechanism with the preceding
three different pad distribution systems, which
will be discussed in the next chapter.
14Pad Publication and Pad Migration
- IntelligentPad aims at the provision of social
infrastructure for the distribution and exchange
of various knowledge resources. - To achieve this goal, we have to solve the
cross-platform reusability of pads. - Pads should be transportable across different
platforms, and reusable in different
environments. - To achieve this cross-platform reusability, we
have to solve two problems, i.e., - how to migrate each pad from one IntelligentPad
system to another, and - how to cope with different platforms.
15- There are three levels of object migration across
different systems. - The shallowest-level object migration assumes
that the two systems share the same class
library. In this case, the source system cannot
send the destination any object whose class
definition is not in the destination. - The middle-level object migration assumes that
the two systems share only the basic common
portion of the class library. It is further
assumed in this case that each object definition
only inherits its property from those classes in
the basic common class library, but not from any
classes outside of the basic portion of the class
library. In this case, the source system can send
any object to the destination. - The deepest-level object migration assumes no
common class library. In this case, we have to
migrate not only this object but also all the
classes used in the definition of this object.
16Web Pages as Pad Catalog
- The existing Web browsers such as Netscape
Navigator and Internet Explorer allow us to
publish multimedia documents throughout the
world. - They allow us to publish pad catalogs that show,
for each pad, its features and hardcopy image. - The worldwide publication repository of pads can
use this catalog publication function for its
users to browse through pads published into this
repository.
17- We assume that every site in the community of our
concern installs an IntelligentPad system with
the same class library. - Each Web page of a pad catalog to publish
describes various pads using texts and images. - Pad names in textual descriptions and visual
images of pads work as special anchors. - A mouse click on one of these special anchors
will pop up a new window with a copy of the
requested pad. - The original of this pad is stored in the
destination file specified by the URL. This file
may exist anywhere in the Internet. - The pad is stored in this file using its exchange
format representation. - When the anchor is clicked, the Web browser
issues a request to the Web server to transfer
the destination file from a remote site. - When the Web browser at the client site receives
this file, it invokes a special file loader in
the local IntelligentPad system to reconstruct
the pad using its exchange format representation.
18(No Transcript)
19- Such invocation requires a special mechanism to
associate different types of anchors with
different application programs to invoke. - The existing Web browsers have this facility. To
identify each type of anchors, they use the file
name extension of the destination file. - When registered to a Web browser, a file name
extension works as an anchor type called a mime
type. Each mime type can be associated with an
application program to invoke. - Each pad catalog uses pad as the
file-name-extension of pad files.
20(No Transcript)
21- The current version of IntelligentPad assumes
that every site shares the same class library. - This assumption requires periodical update of the
class library in each site. This update can be
manually performed by accessing the on-line
IntelligentPad system journal that is managed at
a special site called the class-library-manager
site. - WWW also simplifies this manual update procedure.
You can just open the home page of the
class-library-manager site, and click the
version-update button on this page. - This anchor button specifies the URL of the
system-update difference file. When clicked, this
anchor fetches this file, and invokes the system
update program. This invocation uses another file
name extension ip as its mime type.
22(No Transcript)
23- WWW and its Web browsers also make it easy for
end-users to publish pads into the worldwide
repository. - IntelligentPad provides a special pad called a
PublicationPad. You can drag and drop your pad
onto this pad, which then saves this pad into a
new file in the local Web server. - You have to specify this file name, while its
extension is automatically set to pad. - The PublicationPad then pops up a text input pad,
and asks you to input a description of this pad
in HTML. - It then rewrites the HTML file storing your pad
catalog to add the hardcopy image and the textual
description of the pad you want to publish. - In addition to the anchors you specified in the
description, the hardcopy of the pad also works
as an anchor when this catalog is viewed through
a Web browser.
24(No Transcript)
25URL-Anchor Pads
- Instead of using Web browsers, we can directly
span hypermedia links among pads distributed
across the Internet. - Anchors of links can be represented as pads.
- To specify such a file, we can use a URL.
IntelligentPad provides a new anchor pad called a
URLAnchorPad that specifies its destination pad
using the URL of its file. - When mouse-clicked, a URLAnchorPad directly
issues an URL to an HTTP server to retrieve a
specified composite pad from a remote site. - This resizable pad, when pasted on another pad,
works as a link anchor in a hypermedia network
spanned across the Internet. It can be made
transparent, and can be pasted at arbitrary
location on an arbitrary pad.
26(No Transcript)
27HTMLViewerPad with Embedded Arbitrary Composite
Pads
- An HTMLViewerPad basically works in a similar way
as Netscape Navigator and Internet Explorer. - Furthermore, it allows us to publish any
composite pads by embedding them in arbitrary Web
pages. - Our pad distribution system uses the exchange
format representation to embed arbitrary
composite pads into HTML text files. - Each composite pad to be embedded is first stored
in a file at an arbitrary site in the Internet
using its exchange format representation. - The file name extension of this file must be
specified as pad. Then a URL pointer to this
file is embedded in an HTML text, and this text
is published as a Web page.
28(No Transcript)
29- When an HTMLViewerPad accesses a Web page without
any embedded pads, it works as a conventional
browser such as Netscape Navigator and Internet
Explorer. - When an HTMLViewerPad accesses a Web page with
some embedded pads, it treats each embedded pad
as follows. - When it receives the save format representation
file of an embedded pad from a remote site, it
first treats this pad as a blank figure of the
same size. This figure makes a space of this size
in the Web page. - Then the HTMLViewerPad constructs the
corresponding composite pad, and put this pad in
the saved space in the Web page. - Any page scroll of this Web page also moves this
pad on the HTMLViewerPad so that it always
exactly sits on the space saved for this pad.
30- You can publish any documents and tools as
composite pads embedded in Web pages. - You may even publish HTMLViewerPads accessing
different Web pages by embedding them in another
Web page. - You may also publish a database client tool on a
Web page. - The next figure shows a Web page with an embedded
composite pad that works as a form interface to a
remote database server. The site of this database
server may be different from the Web page site. - You can specify a query on this form interface to
access this database.
31(No Transcript)
32New Publication Media
- The HTMLViewerPad allows us to publish and
acquire any objects as composite pads to and from
Web servers. - These servers form worldwide publication
repositories. - Objects to publish include multimedia documents,
application programs working as interactive
tools, client tools accessing remote servers, and
client tools communicating with remote
application programs. - When applied to scientific publication, this new
publication repository allows us to publish
scientific papers together with reusable related
data, reusable related tools, and remotely
accessible related database services by embedding
all of them in the documents.
33- The nuclear reaction database stores a collection
of data sets. Each data set is defined as a data
table from a series of nuclear reaction
experiments. - Data sets have attributes. Different data sets
may have different sets of attributes. - To store these data sets in a relational
database, we have to represent each data set as a
set of triples consisting of its data set ID, one
of its attributes, and its value of this
attribute. This set includes all the triples for
all of its attributes. - We used UniSQL database management system, and
defined on it a single relation with three
attributes, i.e., data set ID, attribute name,
and attribute value to store all these triples
for all the data sets.
34- Researchers in nuclear reaction physics use this
database to pick up some set of data sets
satisfying some condition on their attributes
such as the reaction type, the incident particle,
the target nucleus, etc. - Then they analyze the picked-up data sets using
some tools, or compare them with others. - The composite pad in the next figure allows you
to access the remote nuclear reaction database
through the Internet. - It gives you the distribution of data sets with
respect to two arbitrarily selected attributes. - Here in this example, the X coordinate represents
a set of different incident particles, while the
Y coordinate represents a set of different target
nuclei. - Each dot in this chart is actually a pad. We call
it a data-set pad. You may click one of these
data-set pads to show its detail information as
shown in the display snapshot image of this tool.
35(No Transcript)
36- Researchers can easily select one of these
data-set pads and retrieve its contents from the
same database using another tool pad. - The HTMLViewerPad in the next figure shows
another page of a research paper that includes
this tool pad. - You may drag and drop the selected data-set pad
into this tool pad. - This tool accesses the same remote database and
retrieves its content as a pad.
37(No Transcript)
38- You may open one more different Web page to the
right of this Web page. This new Web page shows
another researchers document that includes a
chart drawing tool pad. - You may move the data-set content pad to this
chart tool to plot its content data. This chart
shows an angular distribution of reaction
cross-sections.
39(No Transcript)
40- Different from Java applets and ActiveX controls,
embedded pads can be copied and locally reused in
combination with other local pads. - The recombination of media objects in the market
with local pads is fundamental for them to work
as meme media. - Java and ActiveX inherently lack this function.
41Publication and reuse of documents, tools,
services, and agents
- Objects to be published include multimedia
documents, application programs working as
interactive tools, client tools accessing remote
servers, and client tools communicating with
remote application programs. - The third kind of objects can be regarded as
services. Remote servers provide these services.
Their users can appreciate these services through
their client tools published as pads. - An object of the fourth kind works as an agent.
The person who published this agent provides an
application program. The published pad that works
as an agent works as a proxy of this application
program. It communicates with this program
through Web servers by using CGI commands. - Such an agent pad, when its page is opened,
requests its user for a reply, or for some
operation on this agent pad. For example, it may
ask the user to drag out itself and to drop
itself on a specified local pad to acquire local
information through this pad. The agent pad can
send back this acquired information to its remote
application program. It may even be able to
delete itself when its mission is completed.
42- Service publication can even publish
computer-controlled hardware facilities. - Suppose that your institute has a fully
computer-controlled material-synthesizer system
with various measurement tools to analyze
synthesized materials. Its synthesis process is
fully controlled by a computer through a large
number of parameters. Its measurement tools are
also fully controlled by the same computer
through their parameters, and the measured
results are monitored by the same computer. - If you develop proxy pads to communicate with the
control and monitor programs of this system and
its measurement tools, you can compose console
panels for these facilities as composite pads,
and publish them by embedding them in your Web
pages. - This allows people all over the world to utilize
your facilities.
43- This allows material scientists to use various
public facilities distributed all over the world
one after another to conduct their research
projects. - A researcher in Singapore may synthesize a new
material using a facility in US, measure their
characteristics, and then analyze the measured
data on a super computer installed in Japan. - The measurement data are also transported as a
pad from one site to another site, from one Web
page to another Web page or to a local
environment. - Researchers may even pick up some components of
these published facilities, recombine them to
compose new facilities for their use, and publish
these new tools for their reuse by others.
44Annotation on Web Pages
- Digital technology potentially allows us to
overlay a transparent or translucent sheet for
annotation on the original text, which may
protect both the original and its appearance from
any damage. - In the IntelligentPad system architecture, the
development of such a transparent or translucent
pad for annotation is quite an easy task.
45- The next figure shows a Web-annotator composed
with an IEPad and a WebAnnotationPad. - An IEPad is a pad that is obtained by wrapping
Internet Explorer with a pad wrapper. It has
URIaddress slot to hold the URl of the current
page. - A WebAnnotationPad has a double layered structure
with its base layer and its surface layer. It has
three URI slots, originalURIaddress slot (,
which works as the primary slot),
annotationURIaddress slot, and associationURI
slot that respectively hold the original pages
URI address, the URI of the current annotation
file, and the URI address of a URI conversion
service converting each original pages URI to
the URI of the corresponding annotation file. - A WebAnnotationPad has 3 different operation
modes, namely, the transparent mode, the
translucent mode, and the hiding mode.
46- The surface layer works as a transparent or
translucent film covering the base layer and its
child pads if any. - In its transparent mode, a WebAnnotationPad makes
its surface layer inactive, and the background of
this layer transparent, only showing what are
drawn or pasted on this layer every user event,
including pad pasting events, passes through the
surface layer. You may use this mode to paste an
IEPad directly on the base layer of a
WebAnnotationPad with its connection to
originalURIaddress slot this IEPad is inserted
between the base layer and the surface layer. - In its translucent mode, a WebAnnotationPad makes
its surface layer active, and the background of
this layer translucent only those user events
that are not processed by this layer passes
through the surface layer. - In its hiding mode, a WebAnnotationPad makes its
surface layer, together with all the child pads
of this layer, inactive and invisible every user
event passes through the surface layer.
47(No Transcript)
48- In its translucent mode, a WebAnnotaionPad allows
you to paste any pad on its surface layer. - The pasted pad works as a child pad of the
surface layer. - Any pad pasted on a WebAnnotationPad in its
transparent or hiding mode becomes a child of the
topmost pad at this location under the surface
layer. - The surface layer also works as a drawing tool.
This function allows you to draw any figures as
annotations to the Web page shown by the inserted
IEPad. - When an inserted IEPad scrolls its page, the
WebAnnotationPad also scrolls its surface layer
for the same distance so that every child pad and
every figure on the surface layer keep their
original relative locations on the Web page shown
by the inserted IEPad.
49- Each WebAnnotationPad has its associated local or
remote file specified by the URI address stored
in annotationURIaddress slot, and allows you to
save its annotation information there, including
its child pads and annotation figures, together
with the URI of the Web page shown by the
inserted IEPad. - Each WebAnnotationPad has another slot register
for an external event to make it save its current
state to its file. - When we change the URI of the inserted IEPad,
this new URI is informed by this IEPad to the
base layer of the WebAnnotationPad through
originalURIaddress. Then the WebAnnotationPad
accesses the URI conversion service specified by
associationURI slot with the value of
originalURIaddress slot, sets the returned
address, if any, in annotationURIaddress slot,
and accesses this address to obtain the current
annotations on this original page.
50- A reference to any object as an annotation may
use an AnnotationURIAnchorPad with the URI of
this object. - You may paste such an AnnotationURIAnchorPad at
any location on a WebAnnotationPad in its
translucent mode with a connection to either
originalURIaddress slot or associationURIaddress
slot. - For the selection of originalURIaddress slot, a
click of such an AnnotationURIAnchorPad sets its
content URI to originalURIaddress slot of the
WebAnnotationPad, which tells the inserted IEPad
to read the new URI both the inserted IEPad and
the WebAnnotationPad will respectively load the
corresponding Web page and annotation
information.
51- If you connect an AnnotationURIAnchorPad to
associationURIaddress slot, then the
WebAnnotationPad uses the URI sent from the
anchor pad to access a different URI conversion
server for converting the current original pages
URI to a corresponding annotation files URI. - This mechanism is used to jump to a different
users annotation on the same original page. - The previous figure shows an annotative comment
Get more detail information! with an arrow
pointing to a system name in the Web page. It
also has an AnnotationURIAnchorPad with a caption
Ive got some. See this page, which points to
another Web page. When clicked, it sets a new URI
to the WebAnnotationPad and its inserted IEPad.
52Piazza as a Meme Pool
- Pad publication using HTMLViewerPads require the
writing of an HTML definition to embed each
composite pad in a Web page. - Users like to use drag-and-drop operations not
only to retrieve pads from WWW but also to
publish pads to WWW. - They like to use a modeless browser to perform
both the retrieval and publication seamlessly
without changing the system mode.
53- Furthermore, the HTMLViewerPad as well as other
currently available Web browsers does not allow
any user to publish his or her own information in
anothers Web pages, nor to span any links from
anothers Web pages to his or her own Web pages. - Each user has to ask the owner of the Web page by
sending, say, an e-mail to include his or her own
information there or to span a link from this
page to his or her own Web page. - This relation between users who want to publish
their information and the owners of Web pages is
similar to the relation between tenants and
owners. - Each tenant is required to make a contract with
the owner of the building in which he or she
wants to open his or her store.
54- While the owner-tenant system works well for
owners to supervise the clustering of similar
information contents, there is another way of
forming a marketplace, in which a large public
space is provided for people to freely open their
stores. - We also like to provide this kind of worldwide
publishing repository of pads in which people can
freely publish their pads for others to freely
reuse them.
55- Piazza Web is a world-wide web of piazzas, each
of which works as such a marketplace. - We can browse through Piazza Web, using a
PiazzaBrowserPad. - Each piazza has a corresponding file that stores
a set of pads together with their geometrical
arrangement. Such files are stored in special
remote servers called Piazza servers. - Each piazza is also represented as a pad called a
PiazzaPad. - Pads can be drag-and-dropped to and from a
PiazzaPad to upload and download pads to and from
the associated remote server file. - When a PiazzaPad is opened, all the pads
registered to the associated server file are
immediately downloaded onto this pad, arranged in
their registered locations, and become available.
- A PiazzaPad has a slot to specify a piazza server
with its URL address. When given an update
message, a PiazzaPad saves all the pads on itself
with their current states and current locations
into the corresponding server.
56(No Transcript)
57- When given a new URL address, a PiazzaBrowserPad
either generates a new PiazzaPad on itself or
uses the old PiazzaPad, depending on its
specified mode, and sets this URL address to the
file address slot of the PiazzaPad to download
the registered pads. - An entrance link to a piazza is represented by an
AnnotationAnchorPad, and can be put on another
piazza to define a link, with its connection to
the file address slot of the PiazzaPad. - When clicked, the AnnotationAnchorPad sends its
stored URL address to the file address slot of
the underlying PiazzaBrowserPad, which then opens
this piazza.
58- A PiazzaBrowserPad has a save slot, which, when
accessed by a set message from, say, a buttonPad
connected to this slot, sends an update message
to its child PiazzaPad to make it save its child
pads to the server. - Users are welcome to install their Piazza servers
anywhere, anytime, and to publish their client
pads. - Piazza enables end users to open their own
gallery of pads on the Internet, or to exhibit
their pads in some other private or public space.
- Such pad galleries work as flea markets, shops,
shopping centers, community message boards,
community halls, or plazas. - Some piazzas may be for personal use, while some
others may be shared by communities. - Some piazzas may be shared by more than one
community, while some others may be open to the
public.
59- Transportation of pads undefined at their
destination requires their cross-platform
migration their execution on the destination
platform requires that all the libraries
necessary for their execution should be available
there in advance. - These libraries include pad definition libraries,
API libraries, and class libraries. These are
defined as DLLs (Dynamic Link Libraries), and
dynamically called when required. - Migration of a new pad to a different platform
requires migration of all the required DLLs that
the destination lacks. - Pads that someone has uploaded to a PiazzaPad can
be downloaded from the same PiazzaPad and
executed if and only if the destination platform
has all the required DLLs.
60- Each PiazzaPad allows privileged users to upload
a new pad together with its required DLLs. - When another user opens this PiazzaPad, it checks
if the destination platform has all the required
DLLs. - If yes, this user can drag this pad out of the
PiazzaPad. - If not, the PiazzaPad asks the user if he or she
wants to download the missing DLLs. - The automatic DLL migration by Piazza systems
simplifies the distribution of pads among users.
61Re-editing and Redistributing Web Contents as
Meme Media Objects
62- Meme media and meme pool architectures will play
important roles when they are applied to a
reasonably large accumulation of intellectual
resources. - The current situation of the Web satisfies this
condition. - This section focuses on how to convert the
intellectual resources on the Web to meme media
objects, while the preceding sections in this
chapter have focused on how to convert Web
technologies to implement meme pools for the
intellectual resources represented as pads. - In the latter approach, we cannot deal with
legacy Web contents as memes, i.e., as
re-editable and redistributable objects. - In the former approach, on the other hand, we can
extract any components of Web pages including
multimedia, application tool, and/or service
components, recombine them to define a new layout
and a new composite function, and publish the
result as a new Web page.
63The re-editing of Web contents requires
- Easy extraction of an arbitrary Web-document
portion together with its style. - To keep live contents alive after arbitrary
re-editing. - Easy re-editing of Web contents that may be
extracted from different Web pages by combining
them together to define both a new layout and a
new functional composition. - Easy redistribution of re-edited Web contents
across the Internet. - Easy conversion of re-edited Web contents to HTML
format - Easy registration of an HTML document to an HTTP
server.
64Meme media technologies provide ...
- The wrapping of an arbitrary object with a
standard visual wrapper to define a media object
having a 2D representation on a display screen - A wrapped object may be a multimedia document, an
application program, or any combination of them. - The re-editing of meme-media objects
- You can visually combine a meme-media object with
another meme-media object on the display screen
by mouse operations, and define a functional
linkage between them. - You may take out any component meme-media object
from a composite meme-media object. - The re-distribution of meme-media objects
- Meme-media objects are persistent objects that
you can send and receive for their reuse, by
e-mail for example, across the Internet.
65- Here we use IntelligentPad technologies to
achieve the first four of the above mentioned six
capabilities. Now our goal can be paraphrased as
follows - How to extract any portion of a Web document, and
to wrap it with a pad wrapper with some slot
definitions. - How to incorporate periodic server-access
capabilities in the wrapping of a live Web
content. - How to convert a composite pad to an HTML
document, and to register this to an HTTP server.
66XML and pads
- The conversion of Web document components to pads
requires the way to represent HTML or XHTML
documents as pads. - While the HTMLviewerPad can represent any HTML
document as a pad, we still need a way to make
any HTML component of such a pad work as a slot.
The value of such a slot is the value of the
corresponding HTML component.
67- The display object and the model object of a
primitive pad are usually defined as C code,
which makes it difficult for non-programmers to
develop a new pad. - Some pads have very simple internal mechanism
that requires no coding. They include multimedia
documents with some parameters exported through
their pad slots. - For the definition of such a document, we may use
XHTML, or a pair of XML and XSL to define its
content and style, which requires no programming
expertise. - You may specify any of its phrases enclosed by a
begin-tag and an end-tag to work as a slot value.
- An IEPad, a special HTMLviewerPad, when provided
with document contents in XML and a style in XSL,
generates the corresponding XHTML text to view on
itself. - It also generates a slot for each specified
phrase in the original XML or XSL texts.
68- The next left figure shows a parameterized XHTML
that displays any text string in the specified
orientation. - Two parameters are parenthesized with tags to
specify that they work as slots. - The next right figure shows its viewing by an
IEPad, which has two child pads one is used to
input a string, while the other is to specify the
angle. - In addition to these functions, an IEPad allows
us to embed any composite pad in an XHTML text
using a special tag, and generates this pad on
itself when viewing this XHTML text. - The XHTML text in this example embeds a composite
pad working as an analog clock.
69(No Transcript)
70Extraction of an Arbitrary Web Contents as 2D
Meme Media Objects
- Web documents are defined in HTML format.
- An HTML view denotes an arbitrary HTML document
portion represented in the HTML document format. - The pad wrapper to wrap an arbitrary portion of a
Web document needs to be capable of both
specifying an arbitrary HTML view and rendering
any HTML document. - We call this pad wrapper an HTMLviewPad, which is
different from an HTMLviewerPad. - Its rendering function is implemented by wrapping
a legacy Web browser such as Netscape Navigator
or Internet Explorer.
71- The specification of an arbitrary HTML view over
a given HTML document requires the capability of
editing the internal representation of HTML
documents, namely, DOM trees. - The DOM tree representation allows you to
identify any HTML-document portion, which
corresponds to a DOM tree node, with its path
expression. - The next figure shows an HTML document with its
DOM tree representation. - The highlighted portion in the document
corresponds to the highlighted node whose path
expression is /HTML0/BODY0/TABLE0/TR1/TD1
. - A path expression is a concatenation of node
identifiers along a path from the root to the
specified node. - Each node identifier consists of a node name,
i.e., the tag given to this node element, and the
number of its sibling nodes located to the left
of this node.
72Extraction of a Web Page Portionas a View
Document
Client Document (HTML)
HTML view
Extracted information
Delete (Remove)
Cut out (Extract)
Description of the editing process
Description of view definition
View-Definition Code
doc getHTML(http//www.abc.com/,null)
Retrieval Code
view doc .EXTRACT( editing location )
.REMOVE( editing location )
View-Editing Code
73Node Identification
Node identifier to identify any object in a
document
/HTML0/BODY0/TABLE0/TR1/TD1
ltHTMLgt
ltHEADgt
ltBODYgt
/ node-name index / node-name index1-
index2 / node-name / node-name / text()
/ node-name / attr( attr-name )
ltTABLEgt
0
2
1
ltTRgt
ltTRgt
ltTRgt
ltTDgt
ltTDgt
ltTDgt
ltTDgt
ltTDgt
ltTDgt
ltBgt
ltBgt
ltBgt
????
TOPIX
12,969.05
1,300.98
DOM Tree
74- Sometimes you may need to specify, among sibling
nodes, a node with a specific character string as
a substring of its textual contents. You may
specify such a node as tag-nameMatchingPattern
index, where MatchingPattern is the specified
string, and index selects one node among those
siblings satisfying the condition. - You may need to extract some character string in
a text node. - Its path expression locates this node, but does
not locate such a substring. - We will extend the path expression to use a
regular expression for locating such a substring
in a text node. - For the DOM tree in the next figure, the node
/HTML0/BODY0/P/txt(. (\d\d\d\d) .)
specifies the virtual node shown in this figure.
75(No Transcript)
76- The definition of an HTML view consists of the
specification of the source document, and a
sequence of view editing operations. - The specification of a source document uses its
URL. Its retrieval is performed by the function
getHTML in such a way as - doc getHTML(http//www.abc.com/index.html,
null). - The second parameter will be used to specify a
request to the Web server at the retrieval time. - Such requests include POST and GET. The retrieved
document is kept in DOM format.
77Extraction of a Web Page Portion
Wrapper Object(HTMLviewPad)
Extraction
Drag
URL
http//weather.yahoo.co.jp/weather/
Editing code
CLIP(/HTML0/BODY0/TABLE0/TR1)
ltHTMLgt
ltHEADgt
ltBODYgt
ltTABLEgt
ltTRgt
ltTRgt
ltTRgt
ltTDgt
ltTDgt
ltTDgt
ltTDgt
ltTDgt
ltTDgt
HTMLviewPad
ltBgt
ltBgt
ltBgt
78- The editing of an HTML view is a sequence of DOM
tree manipulation operations selected out of the
followings - EXTRACT Delete all the nodes other than the sub
tree with the specified node as its root. - REMOVE Delete the sub tree with the specified
node as its root. - INSERT Insert a given DOM tree at the specified
relative location of the specified node. You may
select the relative location out of CHILD,
PARENT, BEFORE, and AFTER.
79Editing Operators
EXTRACT( node identifier )
EXTRACT(?)
REMOVE( node identifier )
ltHTMLgt
ltBODYgt
INSERT( node identifier, document , relation)
REMOVE(?)
2nd parameterdocument to insert3rd
parameterwhere to insert lt CHILD
BEFORE AFTER PARENT gt
CHILD
BEFORE
AFTER
PARENT
doc
ltBODYgt
ltBODYgt
ltBODYgt
ltBODYgt
INSERT(?, doc , X )
80- An HTML view is specified as follows
- defined-view source-view.DOM-tree-operation(n
ode), - where source-view may be a Web document or
another HTML document, and node is specified by
its extended path expression. - .
- The following is an example view definition with
the nested use of the above syntax - view1 doc
- .EXTRACT(/HTML/BODY/TABLE0/)
- .EXTRACT(/TABLE0/TR0/)
- .REMOVE(/TR0/TD1/)
81- You may also specify two sub trees extracted
either from the same Web document or from the
different Web documents, and combine them to
define a view. - doc getHTML(http//www.abc.com/index.html,
null) - view2 doc
- .EXTRACT(/HTML/BODY/TABLE0/)
- .EXTRACT(/TABLE0/TR0/)
- view1 doc
- .EXTRACT(/HTML/BODY/TABLE0/)
- .INSERT(/TABLE0/TR0/, view2, BEFORE)
82- You may create a new HTML document and insert it
to an HTML document. -
- doc1 getHTML(http//www.abc.com/index.html,
null) - doc2 createHTML(ltTRgtHello Worldlt/TRgt)
- view1 doc1
- .EXTRACT(/HTML/BODY/TABLE0/)
- .INSERT(/TABLE0/TR0/, doc2, BEFORE)
83Code Description and Evaluation
HTML view
Client Document
Web Server
EditingProcessor
Parser
HTTP Access
HTML
EXTRACT?/HTML/BODY//TABLE0REMOVE
?/TABLE0/TR1
http//www.abc.com/
View Evaluator
View-Definition Code
doc getHTML(http//www.abc.com/,null) view
doc .EXTRACT(/HTML/BODY//TABLE0)
.REMOVE(/TABLE0/TR1)
84Direct editing of HTML views
- Instead of specifying a path expression to
identify a DOM tree node, we will make the
HTMLviewPad to dynamically frame different
extractable document portions for different mouse
locations so that its user may move the mouse
cursor around to see every extractable document
portion. - This method, however, cannot distinguish
different HTML objects with the same display
area. To identify such an object, we use an
additional consol panel with two buttons and the
node specification box. - The node specification box changes its value
while you move the mouse to select different
document portions. The first button is used to
move to the parent node in the corresponding DOM
tree, while the second to move to the first child
node.
85Automatic Code Geneation
HTMLviewPad
Step1 document selection
getHTML(http//www.abc.com/,null)
Step2 specification of a editing location
/TABLE0/TR0/TD1
Step3 execution of the editing
- generation of a view editing code
86- When the HTMLviewPad frames what you want to
extract, you can drag the mouse to create another
HTMLviewPad with this extracted document portion.
The new HTMLviewPad renders the extracted DOM
tree on itself. - The next figure shows an example extraction using
such a mouse-drag operation, which internally
generates the following edit code. - doc getHTML(http//www.abc.com/index.html,
null) - view doc
- .EXTRACT(/HTML/BODY/TABLE0/)
87Editing Operation1 (Live-Cut)
Drag Out
HTMLviewPad
A copy of the code
HTMLviewtPad
doc getHTML(http//www.abc.com/,null) view
doc .EXTRACT(/HTML/BODY//
.EXTRACT(/HTML/BODY//TABLE0)
Automatically generated code
88- The HTMLviewPad provides a pop-up menu of
view-edit operations including EXTRACT, REMOVE
and INSERT. - After you select an arbitrary portion, you may
select either EXTRACT or REMOVE. - The next figure shows an example remove
operation, which generates the following code. - doc getHTML(http//www.abc.com/index.html,
null) - view doc
- .EXTRACT((/HTML/BODY/TABLE0/)
- .REMOVE(/TABLE0/TR1/)
89Editing Operation 2
extract delete
HTMLviewPad
EDIT
REMOVE
HTMLviewPad
CLIP
Menu Selection
doc getHTML(http//www.abc.com/,null) view
doc .EXTRACT(/HTML/BODY//TD2//TABLE0/
)
Generated code
.REMOVE(/TABLE0/TR1/)
90- The INSERT operation uses two HTMLviewPads
showing a source HTML document and a target one. - You may first specify INSERT operation from the
menu, and specify the insertion location on the
target document by directly specifying a document
portion and then specifying relative location
from a menu including CHILD, PARENT, BEFORE, and
AFTER. - Then, you may directly select a document portion
on the source document, and drag and drop this
portion on the target document.
91- The next figure shows an example insert
operation, which generates the following code,
where the target HTMLviewPad uses a different
name space to merge the edit code of the
dragged-out HTMLviewPad to its own edit code -
- Aview Adoc
- .EXTRACT(/HTML/BODY//TD1//TABLE0)
- .REMOVE(/TABLE0/TR1/)
- view doc
- .EXTRACT(/HTML/BODY//TD0//TABLE0/)
- .REMOVE(/TABLE0/TR1/)
- .INSERT(/TABLE0, Aview, AFTER)
92Editing Operation 3
insert
Drop In
CHILD
BEFORE
AFTER
PARENT
HTMLviewPad
HTMLviewPad
Menu Selection( Insert Type)
Combination of codes
doc getHTML(http//www.abc.com/,null)Ad
oc getHTML(http//www.abc.com/,null)Aview
Adoc .EXTRACT(/HTML/BODY//TD1//TAB
LE0/) .REMOVE(/TABLE0/TR1/)
view doc .EXTRACT(/HTML/BODY//TD0/
/TABLE0/) .REMOVE(/TABLE0/TR1/)
.INSERT(/TABLE0,Aview,AFTER)
Adoc getHTML(http//www.abc.com/,null)Av
iew Adoc .EXTRACT(/HTML/BODY//TD1/
/TABLE0/) .REMOVE(/TABLE0/TR1/)
Generated code
.INSERT(/TABLE0,Aview,AFTER)
93Automatic generation of default slots
- The HTMLviewPad allows you to map any node values
of its view and any events on its view to its
newly defined slots. - The definition of such a node-slot mapping takes
the following form - MAP(ltnodegt, NameSpace),
- where ltnodegt is specified by its path expression
and NameSpace defines a slot name. - An example of such a mapping is as follows
MAP(/HTML/BODY/P/txt( ), value)
94Data Mapping for Defining Slots
Mapping view information to slots
View-Document
RenderingEngine
ltHTMLgt
View rendering
ltBODYgt
Event
Mapping of node values
MappingEngine
MappingCode
Mapping DATA
Mapping-definition code
MAP( node-identifier , slot-name)
95- Depending on the node type, the HTMLviewPad
changes the node value evaluation to map the most
appropriate value of a selected node to a newly
defined slot. - We call these evaluation rules node-mapping
rules. - Each node-mapping rule has the following syntax
- target-object gt naming-rule(data-type)ltMappingTy
pegt -
- naming-rule naming rule for the new slot
- data-type data type of the slot
- MappingType ltIN?OUT?EventListener?EventFiregt
96Slot Mappingmechanism for reusing Web resources
Wrapper (HTMLviewPad)
Slot
Textual object
Numerical value
Pad
12,969.05
/HTML//content
TOPIX 1,311.49 ???? 12969.05 ?????? 57.94
Table object
Table data
/HTML//table
Tag descriptor format
InputField form ?Inc.
Form object
Button form false
Submit event http//xxx/xxx.cgi?a?
/HTML//form
anchor object
Character data Yahoo Japan
Yahoo Japan
OnClick event http//www.yahoo.co.jp
/HTML//a
97Text nodes
- For a general node such as lt/HTML//txt( )gt,
lt/HTML//attr( )gt, or lt/HTML//P/gt, the
HTMLviewPad automatically defines a default slot,
and sets the text in the selected node to this
slot. - If the text is a numerical string, it converts
this string to a numerical value, and sets this
value to the slot. -
- a text in the selected node (character string)
- gt NameSpaceText(string)ltOUTgt
- a text in the selected node (numerical string)
- gt NameSpaceText(number)ltOUTgt
98Node Mapping Rules(1)
Different mapping for different node types
Node mapping rules
Slot
Mapping Type
MAP( /HTML/BODY//P/ txt() , Value1)
Slot Type
Numerical value type
12,969.05
ltPgt12,969.05lt/Pgt
ltOUTgt
Mapping value(Slot value)
Value1Text
ViewportPad
HTML Source
Slot name
MAP( /HTML/BODY//TABLE0 , Stock)
ltTABLEgtltTRgtltTDgtTOPIXlt/TDgtltTDgt1,311.49lt/TDgtlt/TRgt
ltTRgtltTDgt????lt/TDgtltTDgt12,969.05lt/TDgtlt/TRgt
ltTRgtltTDgt??????lt/TDgtltTDgt57.94lt/TDgtlt/TRgt
lt/TABLEgt
Character string type (CSV)
TOPIX ,1,311.49???? ,12969.05?????? ,57.94
ltOUTgt
StockData
99Table nodes
- For a table node such as lt/HTML//TABLE/gt, the
HTMLviewPad converts the table value to its CSV
(Comma-Separated Value) representation, and
automatically maps it to a newly defined default
slot of text type.
100Meme Media Meme Pool
meme pool
userB
drag drop
userA
drag drop
redistribution
reediting
userC
101Anchor nodes
- For an anchor node such as lt/HTML//A/gt, the
HTMLviewPad automatically performs the following
three mappings to define three default slots - a text in the selected node
- gt NameSpaceText(string, number)ltOUTgt
- href attribute of the selected node
- gt NameSpacerefURL(string)ltOUTgt
- URL of the target object
- gt NameSpacejumpURL(string)ltEventListenergt
102- For example, let us consider a case in which we
extract an anchor defined as follows - ltA href ./next.htmlgt
- Next Page
- lt/Agt
- The first mapping sets the text Next Page to a
string ( or number ) type default slot
NameSpaceText. - The second mapping sets the href ./next.html to
a string type default slot NameSpacerefURL. - The third mapping has the EventListener type.
Whenever the anchor is clicked, the target URL is
set to a string-type default slot
NameSpacejumpURL.
103Form nodes
- For a form node such as lt/HTML//FORM/gt, the
HTMLviewPad automatically performs the following
three mappings to define three default slots - the value attribute of the INPUT node with
the name attribute in the - selected node
- gt NameSpaceInput_type_name(string,
number)ltIN, OUTgt - Submit action
- gt NameSpaceFORM_Submit(boolean)ltEventFiregt
- the value obtained from the server
- gt NameSpaceFORM_Request(string)ltEventListener
gt -
- type lttext?pasword?file?checkbox?radio?hidden?s
ubmit?reset?button?imagegt - name ltnamegt attribute in the INPUT node
104- For example, let us consider a case in which we
extract a form defined as follows - ltFORM action./searchgt
- ltINPUT Typetxt namekeyword gt
- ltINPUT Typesubmit valuesearchgt
- lt/FORMgt
- The first mapping rule for a form sets the input
keyword to a string (or number ) type default
slot NameSpaceInput_text_keyword. - The second mapping rule is an EventFire-type
mapping. Whenever a TRUE is set to a Boolean type
default slot FORM_Submit, the HTMLviewPad
triggers a form-request event. - The third mapping has the EventListener type.
Whenever an event to send a form request occurs,
the HTMLviewPad sets the corresponding query to a
string type default slot NameSpaceFORM_Request.
105Node Mapping Rule (2)
MAP( /HTML/BODY//A0 , Link1)
Character string type
Next Page
ltOUTgt
Link1Text
ltA href ./next.htmlgtNext Pagelt/Agt
Character string type
./next.html
ltOUTgt
Link1refURL
On jumping
Character string type
http//www.abc.com/next.html
ltEventListenergt
Link1jumpURL
MAP( /HTML/BODY//FORM0 , SearchForm)
Character string type
ltIN,OUTgt
??
SearchFormInput_text_keyword
ltFORM action./searchgtltINPUT Typetxt
namekeyword gtltINPUT Typesubmit
value??gtlt/FORMgt
Boolean type
ltEventFiregt
Query issuing event
true
SearchForm submit
Character string type
ltEventListenergt
http//www.abc.com/search?keyword??
On issuing a query
SearchForm request
106Meme Media Meme Pool
meme pool
userB
drag drop
userA
drag drop
redistribution
reediting
userC
107- Each HTMLviewPad has the additional following
four default slots. - The UpdateInterval slot specifies the time
interval for the periodical polling of referenced
HTTP servers. A view defined over a Web document
refresh its contents by periodically retrieving
this Web document in an HTTP server. - The RetrievalCode slot stores the code to
retrieve the source document. - The ViewEditingCode slot stores the view
definition code. - The MappingCode slot stores the mapping
definition code. - The HTMLviewPad updates itself by accessing the
source document, whenever either the
RetrievalCode slot or the ViewEditingCod slot
is accessed with a set message, the interval
timer invokes the polling, a user specifies its
update, or it becomes active after its loading
from a file.
108Live Document Framework
Extraction of live knowledge resources to make
them work as pads
9791.09
\12,841.76
Dow Jones average
Nikkei average
Estimate
- Reuse of live knowledge on the Web
?? \12,841.76
?? 9791.09
total in
currency rate
- Integration of live knowledge resources from
different web pages
total in yen
109Visual definition of slots for extracted Web
contents
- Our HTMLviewPad also allows users to visually
specify any HTML-node to work as a slot. - In its node specification mode, an HTMLviewPad
frames different extractable document portions of
its content document for different mouse
locations so that its user may change the mouse
location to see every selectable document
portion. - Since each extracted Web component uses an
HTMLviewPad to render its contents, it also
allows users to specify any of its portions to
work as its slot. - We call such a slot thus defined an HTML-node
slot. The value of an HTML-node slot is the HTML
view of the selected portion. - The HTMLviewPad converts ill-formed HTML into
well-formed HTML to construct its DOM-tree.
Therefore, you may connect an HTMLviewPad to an
HTML-node slot to view the corresponding HTML
view. - If the HTML-node slot holds an anchor node, the
HTMLviewPad connected to this slot shows the
target Web page.
110- The HTMLviewPad allows you to suspend the
rendering of its contents. In this mode, you may
use an HTMLviewPad with an HTML view as a blank
pad with an arbitrary size. - Such a pad wraps a Web application, providing
slots for the originals input forms and output
text strings. - We call such a pad a wrapped Web application.
- Since a wrapped Web application is such a pad
that allows you to change its primary slot
assignment, you may specify any one of its slots
to work as a primary slot.
111Explicit Slot Definition
meme pool (2D 3D)
userB
userA
drag drop
drag drop
redistribution
reediting
userC
112HTML and Web Applications
HTML
- ltagt
- ??????
- ???????
- ltbgtlt/bgt
- ????
- ltcgtlt/cgt
- lt/agt
?
????
Web server
Internet
113Web Browser and DOM
- HTML
- Text file for a Web page
- DOM
- Document Object Model
- Internal tree