Building%20Mental%20Models%20with%20Visuals%20for%20e-Learning

1
Building Mental Models with Visuals for
e-Learning

• Shalin Hai-Jew
• Aug. 7 10, 2008
• Minneapolis, MN
• MERLOT Still Blazing the Trail and Meeting New
  Challenges in the Digital Age

2
Mental Models

• Definitions A learners internal
  conceptualization of a system / paradigm,
  situation, personage, phenomena or equipment a
  learners sense making (vs. an instructors
  conceptual model on which the designed learning
  is based) implicit and explicit knowledge,
  internalized and externalized knowledge
  subconscious and conscious beliefs
• Synonyms An underlying substructure an
  analogue of the world an operationalized mental
  template for meaning and form (Riggins
  Slaughter, 2006, p. 4) worldview

3
Mental Models (cont.)

• Ideas People Act On Theories in use vs.
  espoused theories (the implied theories based on
  peoples actions vs. what they say they believe),
  to paraphrase Argyris Schon
• Structured Knowledge A hypothetical knowledge
  structure that integrates the ideas,
  assumptions, relationships, insights, facts, and
  misconceptions that together shape the way an
  individual views and interacts with reality
  (Steiger Steiger, 2007, p. 1) an embodiment of
  domain knowledge in order to abstractly reason
  about the domain (objects, grouping,
  interrelationships, sequences, processes, and
  behaviors)

4
Theoretical Underpinnings

• Constructivism, which asserts that learners make
  meanings in their own minds
• Cognitivism, the study of the human mind,
  awareness and mental functions, especially the
  Dual Channel Model (auditory-verbal and
  visual-pictorial), with strategies to maintain
  cognitive load (Mayer Moreno, 1998)

5
Metaphoric Mapping
6
Some Types of Mental Models

• Representational describes, articulates,
  renders coherent, illustrates, and defines
• Predictive anticipates, proposes trend lines,
  predicts, and projects / forecasts
• Proscriptive defines how something should be
  ideally
• Speculative proposes an un-testable thesis,
  purely theoretical (may be mental models at
  extreme scales, beyond sight, sound, and human
  perception)

7
Some Types of Mental Models (cont.)

• Live Info-Heavy Modeling collates live,
  dynamic, and multi-variate information (from
  remote sensors, cameras, people, and other
  sources) into a semi-coherent larger view /
  visualization may be user-interactive and
  user-manipulatable for analysis and
  decision-making

8
Pedagogical Considerations

• Identification of threshold conceptsdifficult
  core concepts that once understood provide a
  broad base for comprehension of more advanced
  concepts
• The fomenting of cognitive dissonance related to
  beliefs, perception, attitudes, behaviors to help
  learners adjust internal mental models
• Building on naïve mental models which tend to be
  elusive, poorly formed, incomplete, poorly
  structured, difficult to articulate, illogical,
  overly generalized, and invalid (from Nardi
  Zarmer, 1991, p. 487)

9
Pedagogical Considerations (cont.)

• Surfacing learner mental models offering methods
  to test assumptions avoiding negative transfer
  avoiding undesirable dependencies to the learning
  tools, and reinforcing accurate conclusions and
  observations (consistently, with augmenting
  cues)
• Identifying and addressing misconceptions (of
  objects, of relationships, fragmented knowledge,
  comparison, contrast, over-generalization,
  over-simplification, and others)

10
Pedagogical Considerations (cont.)

• Must have a defined learning approach clear
  context, defined terminology, defined and
  reasonable learning objectives, defined objects,
  clear relationships, interactivity, processes,
  and a systems view in the design offer
  opportunities for discovery methods to build
  mental models (Moreno, 2004, p. 99)
• Should have some consideration for self-regulated
  learning (SRL) and self-discovery learning (SDL),
  particularly with automated learning constructs
  and self discovery learning virtual spaces

11
Pedagogical Considerations (cont.)

• Should consider expert approaches (establishing a
  context for the task classifying problems based
  on underlying principles and concepts (Alexander
  Judy, Winter 1988, p. 382)

12
Designing a Mental Model

1. Identify a learning domain. Select a portion (or
   the whole) to model.
2. Define the foundational realities.
3. Define the learning objectives and outcomes.
4. Define the relevant terminology and nomenclature.
   
5. Define the range of possible variables and
   measures.
6. Define relevant processes within the model.
7. Prototype and build the mental model while
   considering and adhering to mental modeling
   standards.
8. Build learning scenarios.
9. Build test scenarios, and test with novices and
   experts.

13
Setups for Mental Model Learning

• Pre-learning
• Human facilitation / automated facilitation
• Decision supports
• Learner tracking and measures
• Debriefing
• Pre- and post-testing
• Takeaways and downloadables

14
Digital Mental Model Viability

• The accuracy and comprehensiveness of the
  depicted information
• The fidelity and logical alignment of the model
  to the real world
• The applied predictability and utility
  particularly with multiple interrelated factors
  in play (with clear lines of reasoning)
• The efficacy of the model used in conjunction
  with other tested models in a complex environment
• The expressibility or communicability (vs.
  implicit representational forms like metaphors or
  symbols)

15
Digital Mental Model Viability (cont.)

• The timeliness (non-obsolescence / real-time
  value) and updatability of the model (with new
  information)
• The soundness of the assumptions and premises
  (without inherent biases or propensities)
• The portability of the mental model between
  technological systems / of information from the
  model
• The real-time, time-varying, information-rich,
  multi-stream, multi-representational and
  real-time feedback

16
Digital Mental Model Viability (cont.)

• The aesthetic presentation
• The originality and uniqueness of the digital
  mental model
• The malleability of the model to incorporate new,
  foundational design elements
• The legality of the materials (accessibility,
  intellectual property rights, avoidance of libel
  and slander, and others)

17
Mental Models Enablement

• with High-Tech Visualizations and Graphics in
  E-Learning

18
Seeing

• Illusion-making and the hard-wiring of the human
  brain and visual perception
• Flow and movement 1D, 2D, 3D and 4D
  transparency and overlap texture luminance,
  brightness, saturation, and reflectance
  contrast shape, size, boundaries and edges
  axes, planes, and others
• Gestalt Laws of Pattern Perception proximity,
  similarity, connectedness, continuity, symmetry,
  closure, relative size, figure and ground
  (Westheimer, Koffka, Kohler, 1912, 1935, as
  cited in Ware, 2004, pp. 189 197)
• Labels and text, sounds, and voice

19
High Tech Affordances

• Structure mapping with computer coding
  (ontologies, taxonomies) and spatial layouts
  (bubble graphs, node-link diagrams) of mental
  maps
• Uses of multimedia multi-channel modes for
  coherence, transparency, and clarity (some with
  real-time elements)
• Simulation for similarity, experiential learning,
  and full-sensory immersion (with experiential
  continuity)
• Situated cognition for field-dependent learning
  situated action for field-dependent analysis,
  decision-making and behaviors

20
High Tech Affordances (cont.)

• Informational visualizations (2D, 3D and 4D) to
  capture ever greater informational complexity
  without clutter or visual confusion (and without
  cognitive overload)
• Environmental visualizations via animation and
  movement sequential experiences branched
  experiences
• Interactivity and hypothesis testing
• Immersive spaces with automation, intelligent
  agents and / or live human interactions (and
  possible mediation)
• Information gathering in the digital enclosure
  swarm behavior, urban probes, stalked
  trashcans, lost postcard with URLs behaviors
  (Paulos Jenkins, 2005, pp. 341 350)

21
Conveying a Mental Model

• Realia and real-world artifacts (digital stills
  and video feeds, live sensor feeds, mobile
  sensors), introduction of serendipity and
  apparent chance
• Simulations, depictions (sensory details sight,
  sound, smell, taste, and touch)
• Behaviors (agent and user) and interactivity
  (with feedback loops)
• User informed choices
• Avoidance of unintentional negative learning

22
Visualization for Presentation

• Representational, descriptive and realistic or
  theoretical, conceptual or imaginary (alternate
  conceptual universes)
• Holistic or partial, decomposition of images,
  pullouts
• Process dynamism or change vs. static
• Stylized or non-stylized, natural
• High, medium or low fidelity selective or
  non-selective fidelity
• Conveyance of emotions through emo bots and
  agents, building trust and relations with robots
  
• Macro or micro perspectives
• Discrete or continuous (Tory Möller, 2002, as
  cited in Tory Möller, Jan. Feb. 2004, p. 72)

23
Visualization for Prototyping

• Modeling production designs and blueprints
• Representing different phases of a build
• Revising plans
• Effective for virtual teaming shared mental
  models (but avoiding groupthink) (Thomas
  Bostrom, 2007, pp. 1 8)

24
Visualization for Culling Data

• May extract data from visual captures (such as
  road information from a satellite image, traffic
  imagery gas dispersion flows from a live site
  quantifying the number of people or objects at a
  scene facial recognition software checking
  architectural compliances in terms of distances
  in a blueprint 3D imaging or cross-sections of a
  tumor identification of correlations between
  forms / images simulated flows and transitions
  weathering and aging forensically analyzing
  satellite images disaster response natural
  resources management agricultural planning, and
  others)

25
Visualization for Culling Data (cont.)

• Digital re-constructions of events
• Digital cartography / map-making
• Simmed projections of potential events (with or
  without human inputs / interactions)
• Deformation and animation of soft objects (from
  video captures) de-noising image captures (for
  clearer info) feature enhancements
• Analyzing hyperspectral imagery

26
Visualization for Organizing Data

• Defining relationships between informational
  objects in a domain-specific database as compared
  to an expert-based domain competency model
  (Ahmad, et al., June 2007, pp. 452 461)
  checking mental models (naïve and expert)

27
Some Mental Models in E-Learning

• Simulations
• Immersive learning spaces
• Role playing in case studies, team or group
  simulations, simmed decision-making (e.g. game
  theory)
• Knowledge systems, ontologies, and taxonomies,
  with user interfaces that map with learning
  realities
• Traditional e-learning

28
2D Visuals

• Sketches, drawings, blueprints, diagrams, charts,
  tables, timelines, icons, symbols and designs
• Slideshows (static and dynamic)
• Screenshots
• Interactive maps, screencasts, and games
• Photo montage, photorealistic images
• Non-photorealistic images and depictions
• Video
• Animated agents, avatars, maquettes / models for
  intended work
• Satellite imagery, live data-fed images, and
  acoustical imaging

29
3D and 4D Visuals

• Fractals
• Videos, field recordings
• Animated agents, avatars, maquettes (models of
  intended works like a sculpture), and scenes
• Satellite imagery, live data-fed images, and
  acoustical imaging
• Immersive spaces, microworlds, and metaworlds
• Augmented reality, ambient space
• Holography
• Haptic visual interface
• 4D
• 3D with time (temporal changes and motions)

30
Combined 2D and 3D Visualizations

• Orientation indicators (icons, separate windows)
• In-place methods (clip and cutting planes)
• Orthographic 2D overlays around a 3D object
• Medical imaging
• Flow visualization
• Oceanographic visualization
• Computer aided design (Tory, Oct. 2003, p. 371)

31
Capturing Images in Digital Form

• Digital cameras, mobile devices, mini-cams
• Scanners
• Microscopes
• Telescopes
• 3D devices / multiple synced cameras
• Sonar image devices, acoustic image devices
• Remote sensors, mobile robot sensors, unmanned
  aerial vehicles (UAVs)
• 3D game engines

• Database-stored information and statistics
• Radar
• Satellite
• Telephone call registries
• Remote labs
• Digital pens and tablets
• CAD / CATIA
• Desktop screen captures
• 2D to 3D with minimal image sets (as in
  ubiquitous video for immersive flythroughs for
  situational awareness)

32
The Role of Digital Visuals

• in Mental Modeling

digital storytellingscience-based digital
wetlabsmedical diagnosisdeep sea
explorationouter space explorationaerial image
analysishuman facial identificationmuseum and
art gallery capturesmanga illustrationsinformati
on extractionmachine arttelemedicineimmersive
simulationsvisual information accessvideo
tooningarchitectural designslandscape
architectureperformance artmobile
visualizationgeographically mapped spaces via
GPSthermal imagingtime-lapse /time monitoring
of flows / stock portfolios
33
Image Maps

• DESCRIPTION Spatial information, interactive,
  integration of text and images, conveyance of
  forms and distances, spatial relationships
• Tends to be informationally pre-determined and
  static, with designed interactive effects

34
Glyphs or Iconic Visualizations

• DESCRIPTION A sculptured figure or relief
  carving a font type as in an element of writing
  a visual object that contains one or more data
  variables (coded in the shape, color,
  transparency, orientation, or other aspects of an
  icon)
• Often used in cartography (map-making), logic,
  semiotics (signs and symbols), and pictoral
  information systems (Ebert, Shaw, Zwa Starr,
  1996, p. 205)

35
Photomosaics

• DESCRIPTION An arrangement of aerial or seabed
  photos that form a composite image a visual
  effect in which an image is created of many
  smaller images
• Used for forensic analysis

36
Screen Captures / Screenshots

• DESCRIPTION Realistic to the computer screen,
  annotatable static (non-motion) and non-dynamic
  dynamic (with motion) may have voice overlays
• Examples of interfaces
• Authentic at the moment of capture, usually not
  refreshed (as in websites)

37
Screencasts

• DESCRIPTION Process-oriented, sequential,
  annotated, realia, voice narrated, multi-sensory
• Used to teach about how to use software programs
  or interfaces via desktop computers
• Captures of live synchronous interactive
  experiences, including voice, video, text, live
  annotation, and other features
• Used for virtual teaming meetings, classes, and
  live interactions

38
Fractals

• DESCRIPTION 3D and 4D, geometric, elegant,
  relational, a kind of machine art based on
  mathematical formulas
• Shows relationships, trends
• Self-similarity in design (at least
  stochastically)
• Tends towards irregularity
• Is meaningful at both macro and micro levels
• Tends towards recursiveness

39
Photo-realistic Images

• DESCRIPTION Digital photo captures and imagery
• May be microscope-enhanced, may be
  telescope-enhanced
• May originate from satellite, acoustical image
  gathering , sonograms, x-rays, CAT scans
• May be editable and enhanced, and digitally
  augmented
• Requires a sense of objective size and measure
  requires a correct white balance
• May be mixed with overlays of annotation, drawing
  or other information, annotatable
• May be informational, illustrative , decorative,
  and others

40
Non-Photorealistic Images

• Image morphing
• Photo-mosaicing
• Cartoon rendering from images
• Computerized drawing and imaging fictional
  avatars
• Photogravure effects / intaglio printmaking
  etching simulation
• Machine art
• Acoustic-created synced imagery

• Digital sculpting
• Theoretical modeling and visualizations
  (particularly in the sciences and arts)
• Synthesized image overlays for information-rich
  experiences (usually with photo-realistic images
  or real spaces)

41
Digital Video

• DESCRIPTION Involves color, movement and sound
  realistic or fantastical sequential or
  non-sequential may be stylized may include
  sound
• May be interactive if interspersed with Flash and
  other objects
• May be segmented for easier deployment

42
Avatars

• DESCRIPTION Human or animal or symbolic shapes
  playable characters
• May communicate in voice / sound and / or text
• May make decisions and actions in digital spaces
• Represent their animating players from the
  real-world

43
(Semi-)Intelligent Agents

• DESCRIPTIONS Non-playable, automated
  characters may be static or dynamic
• Programmed abilities, roles, emotions, beliefs,
  actions, intelligence and decision-making
  tendencies
• May play a direct pedagogical or instructional
  role
• May be a tutor
• May infuse a sense of telepresence into automated
  learning spaces

44
Flocking Group Behaviors

• DESCRIPTION The automation of autonomous
  digital entity behaviors in coordinated motion,
  with or without individual agent guides, with
  agent attraction / repulsion also swarming,
  schooling, herding, autonomous pedestrians and
  crowd behaviors character or object motion
  simulation
• Members of a crowd as a-agents (alpha agents)
  may be inertial and pre-determined
• Basic C. Reynolds boids approach (1986)
  cohesion or flock centering (staying close with
  fellow agents) alignment (matching velocity or
  speed with a-agents), and separation (avoid
  collisions with nearby agents)

45
Live Data-feed Images

• DESCRIPTION Remote sensor-fed, database-fed,
  representations often in spatial layouts,
  satellite feeds, and other types of
  multi-spectral / multi-source / multivariate
  integrated data
• Evolving and changing
• Real-time
• Potential suggested trend lines
• Macro and micro perspectives

46
Digital WetLabs

• DESCRIPTION Process-based actions,
  causes-and-effects, human-mediated or remote labs
  or simulations, 3D computer sims with game-engine
  physics
• Narration of processes
• Building of context with facts
• Explanations of measures
• Clear definition of materials used
• Explanations of the processes and effects
• Explanations of negationwhat the process is not
  showing

47
Machinima

• DESCRIPTION Machine cinema, captures of
  avatar interactions and 3D immersive digital
  environments and game spaces pre-recorded
  includes sound
• In-world digital effects
• May be performance-based or unscripted and
  unpracticed

48
Machine-Generated Art

• DESCRIPTION Based on math formulas,
  evolutionary art, chance and other factors tends
  towards fractals
• Synthetic art with unique vector imprints and
  style
• Chaos tools, morphogenesis, cellular
  machines, neuronal co-evolution, and
  non-photo-realistic techniques
• Visualization algorithms
• Perpetual Art Machine

49
3D Immersive Spaces

• DESCRIPTION Live, unpredictable,
  human-populated, automated and true serendipity
• The capturing of visual complexity (with
  multi-channel sensory information without
  cognitive overload)
• The highlighting of particular isosurfaces for
  analysis
• Scene updates

50
High-Tech Image Editing

• Video tooning (Wang, Xu, Shum Cohen, 2004, pp.
  574 583) or turning video into a
  spatio-temporally coherent cartoon animation
• Photo-realistic image to manga illustration
  personalized image-to-cartoon stills
• Image relighting, event relighting

51
Augmented Reality (AR)

• AR DESCRIPTION Real-space overlay of digital
  images and sound through backpack wearable
  computers, head-sets and goggles interactive
  live data feeds often full-sensory used for
  coordinated multi-participant practices in real
  space may be place-sensitive (location-based) or
  place-agnostic (fully mobile) visual
  enhancements on user interfaces and overlaid into
  real spaces

52
Ambient Intelligence (AI)

• AI DESCRIPTION Built-in integrated-display
  electronic environments responsive to the
  presence of people, context-aware,
  individual-aware, adaptive, and anticipatory of
  unique human needs

53
Some Examples

• Augmented Reality(AR) and Ambient Intelligence
  (AI) EXAMPLES
• mixed reality outdoor gaming arenas ubi
  (ubiquitous) computing, social gaming, physical
  gaming, mobile gaming
• tangible or haptic / tactile interfaces
• movement / kinesthetic -based interfaces
• immersive real-space and digital installations,
  and
• smart rooms in smart buildings / houses

54
Digitized Visual-Based Mental Models

• Some Examples

55
A Few Live Digital Examples

• A fractal to describe the infection paths of HIV
  transmission between people
• A predictive simulation of molecular interactions
  
• A Doppler Radar weather map and visualizations
• A moving 3D projection representing the 4th
  dimension
• A house floor plan a proposed village
  development
• A social interchange in a leadership situation
  with artificial intelligence (AI) avatars

56
A Few Live Digital Examples (cont.)

• Interactive computer models for analytic
  chemistry instruction, a forest simulation,
  animal physiology and signal transduction,
  acid-base titration in a virtual chemistry lab,
  the self-heating and scaling of silicon
  nano-transistors, crystallography, and others
• A computer-generated smoke dispersion /
  progression over a photo-realistic image of a 2D
  screen
• Deep space exploration depictions

57
The Future

• Increasing sophistication of digital image
  captures and editing, realia and digital
  artifacts
• Increasing realism and increasing synthetic
  digital imagery (both ends of the continuum)
• Synaesthesia with the inclusion of digital smells
  
• Easier end-user editing and publishing tools (for
  do-it-yourself faculty and learners) for richer
  digital image capture and deployment
• Increased collaborative digital image creation

58
The Future (cont.)

• Increased divergence between open source /
  Creative Commons and (cc) imagery and secure,
  private imagery
• More effective automated metadata (and data /
  content) capture and organization
• Content-rich image repository ontologies,
  taxonomies and collections
• Less unwieldy augmented reality / ambient
  intelligence spaces
• Increasing pedagogical sophistication in the use
  of imagery in e-learning for mental modeling

59
References

• Ahmad, F., de la Chica, S., Butcher, K., Tumner,
  T. Martin, J.H. (2007, June ). Towards
  automatic conceptual personalization tools. ACM.
  452 461.
• Alexander, P.A. Judy, J.E. (1988, Winter).
  The interaction of domain-specific and strategic
  knowledge in academic performance. Review of
  Educational Research Vol. 58, No. 4, 375-404.
• Ebert, D.S., Shaw, C.D., Zwa, A., Starr, C.
  (1996). Two-handed interactive stereoscopic
  visualization. IEEE. 205.
• Moreno, R. (2004). Decreasing cognitive load for
  novice students Effects of explanatory versus
  corrective feedback in discovery-based
  multimedia. Instructional Science Vol. 32.
  Kluwer Academic Publishers. 99 113.
• Nardi, B.A. Zarmer, C.L. (1991). Beyond models
  and metaphors Visual formalisms in user
  interface design. IEEE. 487.
• Paulos, E. Jenkins, T. (2005, Apr. 2- 7).
  Urban probes Encountering our emerging urban
  atmospheres. CHI 2006 PAPERS Design Thoughts
  Methods. ACM. 341 - 350.

60
References (cont.)

• Riggins, F.J. Slaughter, K.T. (2006). The role
  of collective mental models in IOS adoption
  Opening the black box of rationality in RFID
  deployment. 4.
• Steiger, N.M. Steiger, D.M. (2007). Knowledge
  management in decision making Instance-based
  cognitive mapping. Proceedings of the 40th
  Hawaii International Conference on System
  Sciences. 1- 2.
• Thomas, D.M. Bostrom, R.P. (2007). The role of
  a shared mental model of collaboration technology
  in facilitating knowledge work in virtual teams.
  Proceedings of the 40th Hawaii International
  Conference on System Sciences 2007. IEEE. 1
  8.
• Tory, M. (2003, Oct.) Mental registration of 2D
  and 3D visualizations (an empirical study). IEEE
  Visualization 2003. 371 378.
• Tory, M. and Möller, T. (2004, Jan. Feb.)
  Human factors in visualization research. IEEE
  Transactions on Visualization and Computer
  Graphics 10 (1). 72.

61
References (cont.)

• Wang, J., Xu, Y., Shum, H-Y., Cohen, M.F.
  (2004). Video tooning. ACM. 574 583.
• Ware, C. (2004). Information Visualization. 2nd
  Ed. San Francisco Elsevier, Morgan Kaufmann.
• Wells, J.D. Fuerst, W.L. (2000).
  Domain-oriented interface metaphors Designing
  Web interfaces for effective customer
  interaction. IEEE. 1.

62
Thanks

• to Kansas State University for the many
  e-learning design opportunities.
• to the Society for Applied Learning Technology
  (SALT) for first showcasing a piece of the mental
  modeling research in Feb. 2008, in Orlando, FL.
• to IGI-Global for supporting this work with a
  forthcoming text.
• to R. Max

63
Conclusion and Contact

• Dr. Shalin Hai-Jew
• Office of Mediated Education / Instructional
  Design
• Kansas State University
• shalin_at_k-state.edu
• [phone] (work phone)
• [phone] (fax number)
• Instructional Design Open Studio (IDOS) Blog
• There is an accompanying activities handout with
  this presentation for the brainstorming of
  visuals for mental model building in context.
• See the Notes area below each slide for
  additional URL links.