Smart Style on the Semantic Web

1
Smart Style on the Semantic Web

• Jacco van Ossenbruggen
• Lynda Hardman
• CWI Amsterdam

2
Talk overview

• Requirements for 2nd 3rd generation multimedia
• why multimedia is different from text
• Smart style
• a document engineer meets a graphic designer
• Overview of CWIs Cuypers engine
• testbed for multimedia transformations
• Future directions
• towards 3rd generation multimedia

3
The Web in three generations

• Hand-coded (HTML) Web content
• easy access through uniform interface
• huge authoring and maintenance effort
• hard to deal with dynamically changing content
• Automated on-the fly content generation
• based on templates filled with database content
• later extended with XML document transformations
• Automated processing of content
• The Semantic Web
• explicit meta-data instead of screen scraping
• agreed upon semantics (RDFS, DAMLOIL)

4
Multimedia on the Web

• Real multimedia Web content is still rare
• Mostly bells whistles to enhance HTML text
• or mono-media AV-streams
• Virtually all presentations are hand-authored
• proprietary formats that are hard to generate
• limited support for dynamic content and
  multichanneling
• most Web technology is text/page-oriented
• with SMIL as one of the few exceptions

• Conclusion
• Multimedia has hardly caught
• up with the 1st generation Web!

5
2nd generation multimedia

• Adapts to end-users platform capabilities
• multichanneling PC, PDA, mobile, voice-only, ...
• Adapts to the network resources available
• bandwidth and other QoS parameters
• Personalization
• language, abilities, level of expertise, ...

6
But multimedia is not text...

• Different document and presentation abstractions
• hard to separate style from structure
• Formatting is not based on text flow
• no pages or scrollbars, no line-breaking or
  hyphenation
• templates often do not work well either
• Feedback from the formatting back-end required
• need to check whether proposed layout is feasible
• layout of media items is less flexible than
  text-based layout
• Transformations are hard in a functional language
• need to try out designs and backtrack when
  necessary

7
Problem

• Current document transformation and style
  languages are insufficiently powerful
• They rely on flexibility of textre-flow,
  scrollbars, pagination, etc.
• They are "template-based and thus cannot cater
  for wide variations in
• dynamic media-centric content
• device characteristics
• user preferences

8
Solution (our approach)

• The creation of a web-based infrastructure that
  is
• an extension of the current document engineering
  perspective
• taking into account the graphic design
  perspective

9
Document Engineering Perspective

• Content is selected structured
• Mappings are defined to a new presentation
  structure
• Styles (such as color and font) can be applied
• The transformation process is linear
• Assume that
• Content/document structure,
• presentation structure and,
• style
• are independent of each other.

10
Document Engineering Perspective
Transformation Sheet (XSLT)
Stylesheet (CSS)
Rendering (Web-device)
HTML
WML
Post Script
Document structure
Presentation structure
Styled presentation
Final form
11
Graphic Design Perspective

• Basically
• presentation structure, content and style depend
  on one another.
• In multimedia presentations, spatio-temporal
  layout gives meaning to the presentation (in
  contrast to the "linear" nature of text-flow).

12
Design dependencies
Grouping deter-mines layout style
Presentation structure
Style
Layout aestheticsdetermines grouping
Top-level order/groupingaffects local
arrangements
Overall style determineslocal style of media
items
Grouping deter- mines selection of media items
Style determined byselected media items
Grouping depends on semantic relations among
media items
Style determinesselection of media items
Content
13
Example
14
External Forces on Design Process

• Content provider
• Mission (make profit, promote image)
• Limited resources (cost)
• Preferences (company colors)
• End-user
• Goals, needs
• Delivery context limitations (time, environment)
• Preferences (images vs text, audio vs visual)
• Designer
• Design experience
• Resource limitations

15
Requirements for Smart Style

• Integrate Semantic Web technology with current
  Web engineering technology
• Communicating delivery contexts
• Supporting metadata for content and description
  of its function
• Encapsulating explicit design knowledge
• Processing delivery context, metadata and
  design knowledge within the document engineering
  pipeline.

16
Cuypers multimedia transformation engine

• Cuypers testbed is based on
• media independent presentation abstractions
• transformation rules with built-in backtracking
  andconstraint solving

17
Cuypers Web Embedding

• Prolog-based engine embedded in Apache
• XML-based input/output stream
• Java servlets for XML to Prolog translation

18
Cuypers Web Embedding

• Prolog-based engine embedded in Apache
• XML-based input/output stream
• Java servlets for XML to Prolog translation
• but all knowledge is
• implicit and hidden in the rules
• lost in the generated Web presentation
• not reusable for other Web sites

19
Potential Knowledge Sources
20
Conclusions

• Multimedia is mostly first generation
• hand-authored presentations
• common Web-tools are too text-centric
• Cuypers system realises second generation
• on-the-fly multimedia generation
• media-centric transformations
• Third generation is around the corner
• generating annotated multimedia
• reusing knowledge available on the Semantic Web