Portfolios and learner autonomy

1
Portfolios and learner autonomy

• Goal-setting and self-evaluation

Matthew T. Apple Himeji Dokkyo University Himeji,
Japan
Etsuko Shimo Miyazaki Municipal
University Miyazaki, Japan
2
Overall structure of the presentation

• Research questions
• Background description of the courses and
  students
• Defining the portfolio
• The data
• Quantitative (Likert scale)
• Qualitative (Student answers and comments)
• Summary and Analysis

3
Research questions

• What are students perceptions of the portfolio
  tasks?
• Do students perceive that their sense of learner
  autonomy has improved as a result of the
  constructing the portfolio?
• To what extent can portfolio usage improve
  learners meta-awareness of the learning process?
• What level of student proficiency and/or
  motivation is most appropriate for portfolios?

4
Background of pilot study

• 61 students from 2 different 4-year universities
• 14 item questionnaire
• 3 bio-data
• 8 Likert scale (5 point)
• 3 open response
• Additionally, each Likert scale item contained an
  open response follow-up question

5
Class Descriptioncollaborative activities (e.g.,
peer review)
6
Course Evaluation (sample)

• Class Participation (10)
• Journal a minimum of seven entries (30)
• Final Assignment Portfolio (60)
• Cover sheet
• Table of contents
• Two choices in the journal and the reasons why
  youve chosen the two.
• The revision of the two choices in your journal
• Classmates comments on two entries in your
  journal.
• The other five entries in your journal.
• Studying at home vs. studying at school essay
  or asking a famous person one question
  essay (all the drafts from the first to the
  final).
• Final product of the group project (e.g.
  university newsletter, city pamphlet)
• Feedback to the classmates presentation in
  writing
• Reflective writing

7
A Definition of Portfolio

• A purposeful collection of students work
• Showcase portfolio Best work
• Process portfolio Includes drafts of work
• A record of a students development over time
• Focus on process not product
• Normally a physical folder or box
• Contains the 3 elements of Collection, Selection,
  and Reflection (Hamp-Lyons Condon, 2000)
• Representative sample of class work, not
  everything
• Selections chosen by students (in principle)
• Self-evaluation by the students crucial

8
Strengths and Weaknessesof portfolio assessment

• STRENGTHS
• Provides learners opportunity to show what they
  can do
• Encourages learners to become reflective learners
• Helps students take responsibility for their own
  progress toward goals
• Gives learners and instructors opportunities to
  collaborate and reflect on progress
• Gives concrete examples of development over a
  period of time

• WEAKNESSES
• Tend to be time-consuming to assess
• Needs constructive feedback from instructors on
  portfolio as a whole as well as individual pieces
• Learners need guidance on how to construct
• Perception that portfolios are easy
• Tendency toward lack of objective grading
  standards

(based on Linn Gronlund, 2000 Apple, 2004)
9
Methods of assessing the portfolio

• Can be assessed holistically or analytically
• Holistically a single grade (A,B, etc. 1, 2,
  3, etc.) for an overall impression of processes
  (structure, voice, grammar, etc.)
• Analytically multiple grades for individual
  processes
• Combination of holistic and analytic (e.g.,
  analytic for individual pieces within the
  portfolio holistic for portfolio overall)
• Grades the process rather than simply the product
• This grade includes not only an assessment of
  what the result was, but the how and the why how
  a student reached the result and why the result
  came about. (Apple, 2004)
• Allows both instructor and learner to evaluate
  individual progress and effort over time
• Evaluation and learning become collaborative
  rather than confrontational

10
Likert Scale data (Q4-11)
11
Open Response Questions

• Q12 What was beneficial/what have you learned
  through portfolio creation?
• Q13 What would you improve next time? Why?
• Q14 Please write anything about portfolio as a
  means of assessment (for example, how does it
  compare to tests?)

12
What are students perceptions of the portfolio
tasks?

• Positive comments
• Reflection on learning
• Assessment throughout the semester
• Feedback to incomplete products over long period
  of time
• Assessment of process
• Sense of achievement
• Enjoyable learning and assessment tool
• Negative comments
• Time-consuming
• More difficult
• Complicated

13
Has learner autonomy improved as a result of
constructing the portfolio?
14
Q10 Has creating a portfolio led to active
involvement in learning?

• Continuous Learning Opportunities Provided
  Out-of-class Learning Opportunities Increased
• I wouldnt have felt like doing writing
  activities if I had not had this chance of
  making a portfolio.
• I was conscious that I have to turn in the
  portfolio at the end of the semester.
• I have come to like English and watch NHK
  English programs, study independently at home,
  and so on.
• I have come to use my dictionary more often.
• Joy of CreationOriginality ResponsibilityJoy
  of Learning
• I could write my own thoughts
• It was fun to be able to do things on my own.
• Everything was putting my own thoughts into
  English. There was no correct answer, so it
  the portfolio reflects my personal desires, I
  think.
• Nobody else can do it for me.
• I came to enjoy writing in English.

15
Q11(A) Has creating a portfolio led you to
become a more autonomous learner?

• Positive comments
• Responsibility
• We decided what to do ourselves.
• There were many cases where we ourselves
  thought, made decisions, and worked on our
  activity.
• Monitoring Reflecting on Ones Own Learning
• When I compare recent works to old ones, I can
  see the progress, which makes me feel like
  studying harder.
• Confidence
• I have gained confidence in writing.
• ? Positive Affect

16
Q10 Has creating a portfolio led to active
involvement in learning?Q11 Has creating a
portfolio led you to become a more autonomous
learner?

• Negative comments
• Portfolio is what you are forced to make after
  all
• No different from other type of homework
  collection of pieces of homework
• You dont call compulsory work autonomous.
• I felt I was forced to do it.
• Lack of opportunities for Ss to learn what a
  portfolio is?
• Not enough chances for Ss to make decisions and
  choices, taking control over what they do?
• Student view of autonomy may be tempered by the
  perceived compulsory nature of the assignments.

17
Q11(B) What do you think autonomy means?

• Plan yourself and achieve the plan.
• Take responsibility for your own role.
• Regulate oneself by ones own acts.
• Solve problems on your own.
• Without the instructions of someone else, take an
  interest in something and positively and actively
  work at it by yourself.
• Set goals for yourself, head toward them and do
  what you have to do.
• Progress and work hard to achieve something by
  yourself.
• Making a decision about something on your own.

• Cf. autonomy in cooperative learning activities
• Fulfilling your own responsibility in your
  group so others wont get disadvantaged
  because of you failing to do your part
• Expressing and contributing your own ideas to
  the group work

18
Q8 Did you work without others supervision?

• I tried my best not to rely on others.
• I especially tried not to do it the portfolio
  creation with another person, generally doing it
  alone.
• I got lots of advice from others.
• I always got my friends to take a look and check
  my work.
• First I tried to write by myself, no matter
  what. I did ask my friends for a little help,
  though
• I asked my friends when there were things I
  didnt understand.
• I (and my partner) asked each other about
  trouble spots.

19
Portfolio creation and meta-awareness of
learning In theory

• Steers L2 learners away from erroneous belief
  that technically correct language is more
  important
• In the portfolio assessment context, ESL
  learners can be convinced that concentrating on
  ideas, on content, support, text structure, and
  so on, are worthwhile. (Hamp-Lyons Condon,
  2000)
• Self-evaluation aspect of portfolio creation
  leads to
• Improvement though self-corrective feedback
• Monitoring of own progress and methods of
  learning
• Increased self-confidence
• Increased motivation

20
Portfolio creation and meta-awarenessof
learning In practice

• Self-dissatisfaction concerns
• I still have to work harder by myself.
• I thought, Its not good enough yet.
• Structure and expression
• By revising little by little, I was able to
  write more organized essays.
• I felt that I gradually improved my expressive
  ability.
• Constant peer/Instructor feedback
• I looked at my teachers and peers comments,
  which made me think about how to make my English
  more easily understandable.
• Confusion
• I didnt know how to evaluate myself.

21
Benefits of portfolio creationLearners own
words (positive)

• Monitoring
• I was able to discover many of my weaknesses.
• Self-confidence
• At first I was worried that my sentences were
  too short, and I was very uneasy about writing,
  but now Ive been able to do it. Ive learned the
  meaning of the phrase ?????? If you try, you
  will succeed.
• Motivation
• When making the portfolio, I thought I should
  read more foreign books and newspapers. There are
  many phrases I can steal
• Awareness of structure/Planning
• I need to be more persistent in my revisions.
• Deciding the themeof the essay beforehand
  would make it easier to write later.

22
Benefits of portfolio creationLearners own
words (negative)

• Not enough time
• If only I had started earlier and hadnt so
  hastily assembled the portfolio
• I wanted more time
• Nothing
• A little less than one-third of students left
  Question 13 blank
• Didnt know what to do
• I didnt know how to write about the themes I
  wanted more concrete examples.
• Differences in cultural/educational background
  may have led to misperceptions of task
  requirements or reluctance to appear critical
• More than one composition instructor has told
  me, Just write what you thinkas a guideline of
  composition, (this) was not very clear of useful
  to me when I first heard it. (Shen, 1989)

23
Appropriate English proficiency level for
portfolio usage In theory

• Portfolios can be used with any level of
  proficiency
• For lower-level learners, reflection can be done
  in learners L1
• Reflection and self-evaluation are more important
  than being able to write in perfect English
• Raises meta-cognitive awareness of learning
  process
• One of the critical elements of any system of
  assessment is feedbackthe continuous loop of
  response to and reflection on student thinking
  and work in progressthe greatest power in using
  portfolios resides in the personalization of
  assessment through the descriptive processes that
  capture thinking and performance at many stages.
  (Stefanakis, 2002)

24
How do lower-level proficiency students view
portfolio assessment?

• Raises meta-cognitive awareness of learning
• More than a test, I felt I could more accurately
  evaluate myself.
• The portfolio is better because I can evaluate
  all the work Ive done in class up to now.
• In writing class, learn how to write composition
  is important, so getting this kind of evaluation
  is important, I think.
• Leave a permanent record of learner ideas and
  work
• When Im tested, the class comes to an end.
  But with the portfolioit remains afterward, so I
  think it raises my English ability.
• The portfolio shows a record of my thoughts and
  lets me examine the work Ive done.
• Continuous feedback leads to improved ability and
  constant evaluation
• Because I learn many things every week, having
  evaluation every time is good for me. Studying
  for a test doesnt lead to much learning, I
  think.
• I was able to work harder and see my actual
  ability. Unlike studying just before a testI was
  constantly evaluated.

25
Summary of portfolio benefitsfor language
learners

• Reflection on learning
• Leads to higher meta-cognitive awareness of
  learning
• Assessment over a long period of time
• Ample feedback (from teacher and classmates) on
  pieces of work in progress
• Increased chances for goal-setting
  andself-evaluation
• Assessment of a process
• Cooperative learning opportunities
• Sense of achievement
• Enjoyable learning and assessment tool

26
To make portfolio assessment even more beneficial

• Learners should have opportunities to
• discover what a portfolio is and the purposes of
  portfolio creation
• Students examine sample portfolios from previous
  classes
• Instructor explains the purposes explicitly (in
  L1)
• Students share each others work in progress
• learn the importance of cooperation as it affects
  autonomy
• Experience it in the process of making a
  portfolio
• Instructor explains methodology explicitly (in
  L1)
• Students share their reactions to collaborative
  activities with others in the classroom

27
Goal-Setting Self-Evaluation

• Reflection and ample feedback assessment of a
  process
• ? increased chances for goal-setting
  and self-evaluation
• Being aware of the purposes of making a portfolio
• ? purpose-oriented learning
  goal-setting ? self reflection
• Cooperative learning opportunities
• ? reflection on ones own role and
  contribution to the group work
  self-evaluation ? goal-setting
• Goal-Setting
  Self-Evaluation

28
Portfolios and learner autonomy
Thank you for your attention!
Matthew T. Apple Himeji Dokkyo University matthewt
apple_at_yahoo.co.jp http//homepage.mac.com/matthewt
apple
Etsuko Shimo Miyazaki Municipal
University shimo_at_miyazaki-mu.ac.jp