ASSESSMENT

1
ASSESSMENT ANALYSIS

• 5

2
Types of Methodologies

• Introduction
• Needs assessment
• Analysis criteria
• Rapid analysis method
• Analysis tools

3
Introduction

• Assessment
• An evaluation.
• The act or result of judging the worth or value
  of something or someone.
• Analysis
• The separation of an intellectual or material
  whole into its constituent parts for individual
  study.
• The examination and evaluation of the relevant
  information to select the best course of action
  from among various alternatives.

4
Assessment

• Do we need assessment?

5
Needs Assessment

• Determine if an instructional need exist by
  conducting a needs assessment using some
  combination of methods and techniques.
• Provides an outline of the information that needs
  to be found and the means of collecting the
  information.
• Based on interviews, surveys and group meetings
  to get the opinions and feelings of the people
  involved.

6
Needs Assessment

• Involves as many concerned parties as possible or
  necessary to make sure you get the information
  you need and how confident you are with it.
• Conducted in stages based upon what you've
  already learned.
• Tools extant data analysis, job descriptions,
  customer studies, etc.

7
What to be analyzed?

• Front-end analysis
• Audience analysis
• Situational analysis
• Task analysis
• Critical incident analysis
• Objective analysis
• Extant data analysis
• Cost analysis

8
Front-End Analysis

• It is the first step in the multimedia
  development process.
• FEA - An analysis that is performed in order to
  gather crucial information which will be used to
  develop technically sound instruction.
• FEA consists of goal analysis, performance
  analysis, target population, learning environment
  analysis, instructional strategy.

9
Front-End Analysis

• Goal Analysis
• This technique is used to analyze a goal to
  identify the sequence of operations and decisions
  required to achieve it.
• Performance Analysis
• A systematic process for determining goals,
  identifying discrepancies between optimal and
  actual performance, and establishing priorities
  for action.

10
Front-End Analysis

• Target Population
• A description of those whom the instruction is
  intended (the learners).
• Knowing the audience will greatly aid in training
  development.
• Learning Environment Analysis
• A description of the environment where the
  learning will take place. Knowing the environment
  will greatly aid in training development.

11
Front-End Analysis

• Instructional Strategy
• An instructional strategy is a combination of
  technique, media (or delivery method), and
  location which shapes the instruction.

12
Audience Analysis

• The audience is often referred to as the end
  user, and all communications need to be targeted
  towards the defined audience.
• When defining an audience, factors that must be
  considered include
• Age, education, job function, skills, language,
  culture, background knowledge, training, needs
  and interest.

13
Audience Analysis

• A nalysis - Who are they? How many will be there?
• U nderstanding - What is their knowledge of the
  subject?
• D emographics - What is their age, sex,
  educational background?
• I nterest - Why are they there? Who asked them to
  be there?
• E nvironment - Where will I stand? Can they all
  see hear me?
• N eeds - What are their needs? What are your
  needs as the speaker?
• C ustomized - What specific needs do you need to
  address?
• E xpectations - What do they expect to learn or
  hear from you?
• 
  by Laskowski (1996)

14
Technology Analysis

• Analysis on the technology to be applied or
  implemented while authoring the learning
  application, or when user uses the learning tool.
• Example
• Design technology 3D animation, flash, types of
  authoring tools, etc.
• In the learning tool speaker, online conference,
  online tutoring, online discussion, etc.

15
Situational Analysis

• To identify and prioritize problem situations
  effecting the target population or specific
  segments of the population.
• Answer several questions
• Students characteristics
• Background, etc.

16
Task Analysis

• A Learning Task Analysis consists of selecting
  the instructional goal(s), conducting an
  information processing analysis, and writing
  educational objectives.
• Analyze the learning outcomes and performances
  objectives by identifying the domains and levels
  of learning and determining prerequisite skills
  and task/content structure.

17
Example of Task Analysis
18
Critical Incident Analysis

• The Critical Incident Technique (or CIT) is a set
  of procedures used for collecting direct
  observations of human behavior that have critical
  significance and meet methodically defined
  criteria.
• These observations are then kept track of as
  incidents, which are then used to solve practical
  problems and develop broad psychological
  principles.

19
Critical Incident Analysis

• A critical incident can be described as one that
  makes a significant contribution - either
  positively or negatively - to an activity or
  phenomenon.
• Critical incidents can be gathered in various
  ways, but typically respondents are asked to tell
  a story about an experience they have had.

20
Critical Incident Analysis

• CIT is a flexible method that usually relies on
  five major areas.
• The first is determining and reviewing the
  incident, then fact-finding, which involves
  collecting the details of the incident from the
  participants.
• When all of the facts are collected, the next
  step is to identify the issues.

21
Critical Incident Analysis

• Afterwards a decision can be made on how to
  resolve the issues based on various possible
  solutions.
• The final and most important aspect is the
  evaluation, which will determine if the solution
  that was selected will solve the root cause of
  the situation and will cause no further problems.

22
Critical Incident Analysis

• The user is asked to focus on one or more
  critical incidents which they experienced
  personally in the field of activity being
  analyzed.
• CIT analysis uses a method known as Content
  Analysis in order to summarize the experiences of
  many users or many experiences of the same user.

23
How do we do the CIT?

• You can do CIT by either employing an interview
  or by getting the users to fill out a paper form.
  
• The user is requested to follow the three stages
  described below in that order
• focus on an incident which had a strong positive
  influence on the result of the interaction and
  describe the incident
• describe what led up to the incident
• describe how the incident helped the successful
  completion of the interaction.

24
Reporting Critical Incident Analysis

• For a summative evaluation, you should collect
  enough critical incidents which will enable you
  to make statements such as "x percent of the
  users found feature y in context z was helpful/
  unhelpful."
• For a formative evaluation, you should collect
  enough contextual data around each incident so
  that the designers can place the critical
  incidents in scenarios or use cases.

25
Objective Analysis

• A learning objective answers the question What
  is it that your student should be able to do at
  the end of learning period?
• A learning objective makes clear the intended
  learning outcome or product of instruction,
  rather than what form the instruction will take.

26
Objective Analysis

• Focus on student performance.
• Action verbs that are specific list, describe,
  report, compare, demonstrate, and analyze should
  state the behaviors students will be expected to
  perform.

27
Extant Data Analysis

• Front end analysis to determine the effects of
  student performance by looking at the outcomes
  and inferring the actual performance or
  performance problem that is taking place.
• Purposes of EDA
• To seek the truth, gain a better understanding of
  performance and verify information, through the
  outcomes without relying on opinions from those
  involved.

28
Extant Data Analysis

• Use the results as the basis for later stages of
  TNA.
• Make sure that the accomplishments match the
  goals of the learning.
• Look at data that will measure performance
• Figures, Statistics, Records, Files, etc.

29
Extant Data Analysis

• EDA tools
• Grade and exam results
• Entry test
• Performance review
• Personnel records

30
Cost Analysis

• Cost analysis (also called economic evaluation,
  cost allocation, efficiency assessment,
  cost-benefit analysis, or cost-effectiveness
  analysis by different authors) is currently a
  somewhat controversial set of methods in program
  evaluation.
• One reason for the controversy is that these
  terms cover a wide range of methods, but are
  often used interchangeably.

31
Rapid Analysis Method - Intro

• People must be lined up and their views sought. 
• Ideas need to be sorted through. 
• Data needs to be collected, stored, retrieved,
  examined and displayed. 
• Reports needs to be prepared. 
• Policymakers may have to decide without the full
  benefit of the analysis

32
Rapid Analysis Method

• Objectives
• How analysis could be made faster without
  sacrificing its quality.
• An examination of what takes time in the analysis
  process.
• Where one can speed up the work without affecting
  the quality of the report.

33
Rapid Analysis Method

• Preparation
• Data collection
• Analyze data
• Presentation
• Additional note

34
Analysis tools

• Brainstorm
• SWOT
• Discussion
• Survey

35
Brainstorm

• A sudden clever plan or idea.
• To consider or investigate (an issue, for
  example) by brainstorming.
• To think of or produce (a solution to a problem,
  for example) by brainstorming.

36
Brainstorm

• Brainstorming with a group of people is a
  powerful technique.
• Brainstorming creates new ideas, solves problems,
  motivates and develops teams.
• Brainstorming motivates because it involves
  members of a team in bigger management issues,
  and it gets a team working together.

37
Brainstorming Process

• Define and agree the objective.
• Brainstorm ideas and suggestions having agreed a
  time limit.
• Categorize/condense/combine/refine.
• Assess/analyze effects or results.
• Prioritize options/rank list as appropriate.
• Agree action and timescale.
• Control and monitor follow-up.

38
SWOT

• Strength,
• Weaknesses,
• Opportunities,
• Threats

• After the issue has been defined and the
  environmental influences have been identified you
  can develop a summary of internal strengths and
  weaknesses of the organization managing the
  project and the external opportunities and
  threats that could affect the campaign.

39
SWOT

• Internal strengths and weaknesses are things like
  human and financial resources, expertise,
  management support and internal politics.
• External opportunities or threats include
  cultural norms, demographics, economic situation,
  political or legal issues and the activities of
  external organizations.

40
Discuss

• To speak together and exchange ideas and opinions
  about something.
• Discussion within group members on the process of
  developing a project.
• Individual task, dateline, materials, etc.

41
Survey

• To examine or look at comprehensively.
• To inspect carefully
• To inspect and determine the structural condition
  of (a building).
• To conduct a statistical survey on.

42
Survey

• A sample survey studies part of a group (the
  sample) in order  to make inferences about the
  whole group (the population) from which the
  sample is drawn. 

43
Example (survey)