Bridging Research and Practice in Distance Education

1
Bridging Research and Practice in Distance
Education

• A communication management approach to a
  state-of-the-art Online Learning Environment.
• Erin Shaw
• CARTE/DEN Collaboration Viterbi School of
  Engineering

2
Goals and Progress

• DEN OLE goals
• Feature innovative teaching and learning
  facilities
• Increase learner-centered participation and
  collaboration
• Encourage instructor-initiated experimentation
  and assessment
• Bridge research and practice in distance
  education
• CARTE progress toward goal
• Deploy ISI discussion board
• Automate registration via single sign on from
  Blackboard
• Render development at ISI independent of
  production at DEN
• Integrate discussion and artifacts (course doc,
  webcasts)
• Allow students to attach and share documents
• Focus discussion and assessment
• Expand communication capabilities
• Support IM (mobile), RSS feed, summarization and
  activity watches
• Extend document capture capabilities
• Generate text, html, and pdf file formats and
  metadata

3
Integrated System Overview
DEN Blackboard System
Student Collaboration Tools
ISI Discussion Board
Agent Response
Agent Analysis
DEN File System
Instructor Assessment Tools
ISI Document Database
Completed In Development Planned


4
DEN Blackboard Link to ISI Discussion Board
Native Blackboard discussion board link replaced
with link to ISI board
5
ISI Discussion Board
Blackboard sends user profile and discussion
board automatically registers user
6
Current State

• Current state
• In use since Spring 2005
• Two DEN courses currently online (CSci577,
  CSci402)
• Four archived courses available (AME599,
  3xCSci402)
• Extensive processing of 402 course materials
  completed
• Current focus
• Utilize discussion board for research
• Data benefits learning, mining, ped. agent and
  NLP research
• Spawned AIED paper, HTL demo, and AERA, IUI
  submissions
• Experimental work (UCLA intern and CSci577 class)
• Proposal planning
• Now is a critical time!
• Funding has ended (funds slated to run out
  12/2005)
• Minimal funding would assure continued support
  for
• current course use
• research evaluation

7
Project Rationale

• Addresses instructional challenges
• Instructors want to experiment but have limited
  support
• E.g. Can maintain own web site but not discussion
  board
• registration, security, and maintenance are
  obstacles
• Students expect to use current communication
  technologies
• Evaluation of Learner centered model difficult
  but necessary
• Addresses technical challenges
• Building own OLE, duplicating DEN system
  impractical
• Blackboards Building Blocks solution incomplete
  
• API incomplete, integration too tight
• ISI development within DEN production environment
  problematic
• Our solution
• Encourages adoption (maximizes ease-of-use)
• Enables development at ISI (minimizes
  disruptions)
• Uses open source development model (maximizes
  code maintenance)
• Allows for evolution from Blackboard
• 18 open source CMSs currently available
  (EduTools, August, 2005)

8
Discussion Board Deployment

• Software
• phpBB, open source, latest version, running at
  ISI
• http//www.phpbb.com, http//melodic.isi.edu/phpbb
  2
• ISI modifications adhere to community development
  standard
• modularized and self-documenting
• we maintain changes, phpBB dev community
  maintains core
• Courses online (2005)
• Spring AME599, CSCI402
• Summer CSCI402
• Fall CSCI555, CSCI402
• Archived 2004 discussion (CSCI402 and CSCI555)
• Usage statistics
• 130 past students, 130 current students
• Over 2000 messages

9
Summary of Accomplishments

• Main accomplishments
• Single sign on from Blackboard (fully integrated)
• Document capture (Blackboard-integrated user
  interface)
• Successfully used in five courses
• Discussion board details
• Auto-creation of lecture discussions
• Integration of webcasts and lecture notes to
  follow
• Annotation system
• Instant messaging/mobile learning integration
• Answer mining of archived discussions
• Archiving features
• Document capture details
• Automate parsing of uploaded document
• Creates PDF, HTML, text and metadata files for
  all documents
• Makes documents searchable via discussion board

10
Summary of New Research

• Research thrusts
• SPADE rhetorical analysis (Shaw)
• Classummary evaluation (Liang Zhou, GRA)
• Q/A processing pipeline (Donghui Feng, GRA)
• 800 messages manually annotated for network
  analysis
• Ontology creation (in planning with Kim)
• Data share tools
• API for message export
• XML schema for message formatting
• API for agent message insertion
• Evaluation tools
• Statistics viewer
• Questionnaire creation tool (auto-send to users)
• Instrumentation for Q/A evaluation (planned)

11
Screen Shots
12
Artifact-Based Discussion
Documents and images displayed next to
discussion Supports all file formats, including
PDF
13
Topic-Based Annotation
Simple markup of HTML version of
document directs processor to create links to
auto-generated topics
14
Two way communication via instant messaging
BBS sends message Notification
User queries BBS for details
15
Mobile Communication via IM
AOL Instant Messenger communication Worked on
all phone types tested
16
Statistics Viewer
Accessible via Administrator Panel Allows
instructors to access ISI data
17
Question Answering Research
Text is incoherent, non-factoid Questions
barely identifiable, often include multiple
sentences or paragraphs Answers require
explanations
18
Question Answering
Placeholder for results screen shot
Answer-bot refers questioner to most similar
discussion and document segment.
19
XML Schema for Message Export
20
Classummary Tool (Liang Zhou)
21
Classummary Archive
22
Classummary Notification Scheduling Interface
23
Classummary Notification Email
24
Research Potential
25
Discourse Analysis (Shaw)

• Apply SPADE (Soricut Marcu) rhetorical parser
  to discussion board data (spring 2005)
• Run on over 1000 messages (Open University
  corpus)
• Used to validate differences in tutor and student
  messages
• AIED 2005 paper
• Assessing and Scaffolding Collaborative Learning
  in Online discussions
• Generalize work (to be proposed)
• Adapt for real-time analysis
• Run on DEN data
• Evaluate different types of messages
• Question vs. comment vs. answer
• Tutor-instructor vs. student
• Develop message classifier (done manually now)
• Develop rules for promoting discussions

26
Pedagogical Interventions (Beal)

• Analyze discussion board communication (spring
  2005)
• Analysis of small corpus of messages (Beals
  UMass course)
• More information here
• Generalize work (proposed in AIED poster, Beal,
  Kim, Shaw)
• Amalgamation of agent and ontology work, NLP, and
  discourse analysis
• More information here

27
Learning Ontologies (Jihie Kim)

• Analyze discussion board communication (summer
  2005)
• Annotation of small corpus of messages (CSci555)
• Classify on-topic messages using keywords
• Compare technical vs general discussion messages
• Generalize work (to be proposed)
• Acquire keywords automatically
• Identify keywords in messages
• Assess students contributions