Walk-Through Illustrations: Frame-Coherent Pen-and-Ink Style in a Game Engine

1
Walk-Through IllustrationsFrame-Coherent
Pen-and-Ink Style in a Game Engine

• Bert Greudenberg
• Maic Masuch
• Thomas Strothotte

2
It Looks All Crappy

• Cartman Aw, screw it. It probably isn't all that
  good anyway.
• Kyle Cartman! What are you talking about?! You
  LOVE Terrance and Phillip!
• Cartman Yeah, but the animation's all crappy
  it probably can't sustain itself over ninety
  minutes.

3
Who wants crappy animation?

• Tired of the gothic space dungeons?
• Tired of realistic animation that fools you for
  five seconds?
• You need a change
• A non-photorealistic change!

4
3D Game Engines

• 3D game engines have made huge advances
• Engines primarily still focus on photorealistic
  images
• Game images are technically non photorealistic

5
NPR Here to Stay

• Research has turned to NPR of still images
• Great advances in Toon, Painterly, and Pen Ink
  (i.e. South Park)
• However, animating NPR or real-time rendering 2
  problems occur.

6
Whats the Problem?

• Maintenance of frame-to-frame coherence
• Hidden Surface Removal
• Objects appear and disappear correctly
• Ability of a 3D engine to render NP images in
  real time

7
Solution!

• Create an engine that facilitates a hybrid
  rendering pipeline
• Allows specification of the rendering style on a
  per object basis
• More to follow

8
Computer Games and NPR

• No games achieves true photorealism
• Rendering style can be called photorealistic
• Shapes, spatial relationships, surface details

9
NPR Video Games

• Very few games incorporate NPR elements
• Examples
• Cartoon Rendering
• Pen Ink
• Broken Arrow

10
Cartoon Rendering

• Shading Model in 2D cartoon
• Games only use 2D data.
• Visually appealing, but not that interactive
• Monkey Island III
• Broken Sword

11
Monkey Island III
12
Pen and Ink Rendering

• NPRQuake by Gleicher
• Intercepts the calls to graphics libraries
• Replaces them with different drawing primitives
• Uses core GLQuake engine and maintains gameplay

13
NPRQuake Sketch
14
NPRQuake Blue-Print
15
NPRQuake Brush-Stroke
16
Problems with NPRQuake

• Achieves non-photorealistic look
• Style elements do not preserve frame coherence
• Outlines of objects vary from frame to frame
• Results in lively but disturbing experience

17
How to Fix It?

• Few systems deal with frame to frame coherence.
• Many systems can create Pen Ink illustrations
• But, probability is used to model the line
  deviations of hand drawings
• Not suitable for real time rendering in animations

18
Non-Photorealistic Visualization In Game Engine

• NP imaging allows the user to simplify an image
  by omitting details to control the viewers
  attention
• Emphasize, Deemphasize certain elements of a
  scene
• Aim for pen-and-ink illustrations
• Silhouettes and creases are clearly drawn
• Surface detail added through hatching

19
Hybrid Rendering Pipeline

• Able to specify rendering style of each object
• Gain additional freedom in expression for the
  visualization
• Photorealistic Images use normal texture
  mapping and shading
• NP Images consist of modified shading, an
  outline, and NP textures (ink maps or hatch maps)
• Hybrid combine photorealistic rendering and NP
  drawing styles

20
NP Elements in 3D Game Engine

• Properties of 3D game engine
• Provide player with highly immersive experience
• Enhace speed, realism, and image quality
• Preprocess as much data as possible
• Let hardware handle most of the per-frame load
• Mip-Mapping, anti-aliasing

21
NP Elements in 3D Game Engine

• Non-Photorealism uses image space elements
• Pixels generated on the screen are not a strict
  projection of the 3D scene objects
• A triangle outline rendered as lines has a
  constant width independent of the actual distance
  of the triangle to the viewer

22
Outlining

• Silhouettes
• Drawn to enhance the visual separation of objects
  from each other and background
• Silhouette edges need to be determined every
  frame
• Non-Convex edges are never drawn (second)
• Sharp Edges are always drawn (third)
• Only smooth convex edges are drawn (fourth)
• Result with hidden lines removed

23
Hatching

• Textures are supposed to become smaller with
  increasing distance
• Hatching
• shade by drawing or etching fine parallel or
  perpendicular lines
• Pen Ink hatching textures cause serious moiree
  patterns
• In motion this produces even worse artifacts than
  still images
• Prevents frame coherence

24
How to Fix This?

• Common antidote
• Mip-mapping
• Unfortunately, normal filtering process applied
  to large white areas and thin black lines of
  hatching texture cause lines to vanish
• Visible structure is lost

25
Hatch Maps

• Normal mip-mapping
• Lines are black and well separated at the top
  level
• Become lighter and closer at lower levels
• Hatch Map
• Equally space black hatch lines for all mip-map
  levels
• Only very lowest levels are gray values used to
  maintain tone

26
Hatch Maps

• Unfortunately, all over tone is not continuous
• Abrupt switch between hatch-map levels
• Use Tri-linear Filtering

27
Ink Maps

• Hatching is a mechanical process
• Hybrid Pipeline
• Photorealistic Mip-Mapping
• Non-Photorealistic Ink Map
• Adding details to surfaces is artistically
  challenging
• Construct ink maps
• Smooth animation achieved with carefully crafted
  ink maps
• Frame Coherence is maintained using tri-linear
  filtering

28
What They Achieved

• Part of a medieval palace created for the UNREAL
  engine

29
What they Achieved
30
Specifications

• 3D engine runs on
• 750 MHz CPU
• GeForce2 ultra graphics card
• Even on unoptimized models, engine runs at
  interactive speeds, i.e. 25 fps

31
And Now.Zelda!!!

• The Legend of Zelda The Wind Wakers
• Right now, this is the pinnacle of NPR in game
  engines.
• First game to max out hardware of Nintendo
  Gamecube
• Combines great use of interactive 3D and NPR

32
Screenshots
33
More
34
Zelda Video!!

• Thank You
• Any Questions?