Antonio Mocci

1

• Antonio Mocci
• External evaluation
• Ede - the Netherlands, 11 December 2009

2
Evaluation criteria
3
Evaluation criteria
4
Objectives for evaluating INTENT
Consider the process of result production, of
output realisation Analyse products in
comparison to planned results Consider the
impact of project outcomes on partner
organisations and on VET systems.
5
Process

• Working process was characterised by
• Balanced responsibilities
• Respect for individual expertise and interests
• Use of ICT (Web and virtual workplace).

6
Process

• A two level organisation to achieve two different
  objectives

Results and outputs production
Confrontation with stakeholders
7
Products
Three main products The unit profile on
International Entrepreneurship Training
programme in five sub modules Assessment
criteria and portfolio of International
Entrepreneurship
A robust dissemination programme Stakeholder
involvement Network interrelation
8
Products
Some characteristics of products
The Unit profile is articulated according to the
structure and the requirements of ECVET this
means the declination of a set of knowledge,
skills and competence required to perform in an
international context effectively and
appropriately when interacting with
others. Applicable to different geographical
contexts and to different sectors.
9
Products
Some characteristics of products
The Training programme is articulated in five
submodules with specifications for the KSC
structure of each sub module and for its
articulation in didactic units. Pre requisite are
indicated as well, in terms of personal
motivation, abilities and previous intercultural
experiences. Materials for participants and
teachers are listed according to three levels
beginners, intermediate and advanced.
10
Products
Some characteristics of products
The Assessment criteria set standards for the
assessment of trainees in the programme they
also set guidelines for certification of the
training programme. The Portfolio International
Entrepreneurship Competence provides employers
and employees, trainers and trainees a record of
progress in key attributes of International
Entrepreneurship Competence. It has three
parts A Passport of International
Entrepreneurship Competence which records all
formally assessed competences, A Biography of
International Entrepreneurship Competence in
which assessees may record personal experiences
and encounters that may have contributed to their
international and intercultural development, A
Dossier of Evidence of International
Entrepreneurship Competence in which assessees
may keep any documentary or recorded evidence of
their progress and actual competence to date.
11
Project Impact
Internal impact
External impact
On VET systems
On partner organisations
12
Project Internal Impact

• Internal impact seems to be stronger and more
  visible in relation to four areas first, the
  integration of the results (or part of them) into
  qualification systems. Second, the integration of
  part of results into teachers and trainers
  toolbox. Third, the extension from the trade
  sector to other sectors and to different
  professional roles (workers, managers,
  entrepreneurs). Fourth, the extension to both
  IVET and CVET, although continuous training seems
  to be a more promising field of development.

13
Project Internal Impact

• Points of attention
• Existing results (UP, TP, AC) need to be
  translated in national languages.
• Adaptation needs to consider national legal
  frameworks and standards.
• The assessment criteria have to be adapted to the
  target group (students in the formal education
  system or non-formal system).
• Strong points
• Flexibility of the solution adopted to design and
  deliver the training programme.
• Easy transferability of the unit profile and of
  training programme to other sectors and to
  different professional levels.
• The process of producing results involved several
  target groups (institutions, companies, VET
  providers), and this added value to results.
• Relevance of intercultural issues and competences
  in the process of building the European Union
  (Active citizenship).
• The outputs structure make possible the
  implementation of EQF principles (levels 3, 4,
  5).
• Teachers, trainers and professionals have the
  possibility to use and customize training
  materials and assessment criteria.
• Training materials can be addressed to a variety
  of target groups, including entrepreneurs.

14
Project External Impact

• Intent products - the unit profile, the training
  programme and the assessment criteria - provide
  for concrete and tangible tools to be utilised
  for using and applying ECVET.
• The concrete application of the ECVET tools and
  principles, the establishment of a credible
  network to be considered as a reference for
  system development, the capacity to transfer the
  INTENT approach into EQF application through NQF
  upgrading, seem to be the most outstanding impact
  effects this partnership produced on VET systems
  and represent the concrete added value INTENT
  gave to European lifelong learning.

15
Conclusions

• Relevance To what extent are the project
  objectives justified in relation to needs? Can
  their raison dêtre still be proved? Do they
  correspond to local, national and European
  priorities ?
• In relation objectives formulated in the
  application form and articulated by the
  partnership in the inception phase, it is
  reasonable to say that INTENT was in line with
  the European policies finalised at improving
  mobility of students and workers. Consistency is
  particularly relevant with respect to the ECVET
  and EQF implementation policies. National
  priorities and frameworks have been considered.

16
Conclusions

• Effectiveness To what extent have the
  objectives been achieved? Have the interventions
  and instruments used produced the expected
  effects? Could more effects be obtained by using
  different instruments?
• Objectives have been achieved and planned results
  produced. Though time schedules had to be
  modified sometimes (for example, the research
  activity took longer than expected), outputs have
  been produced and planned effects were obtained.
  Results exploitation process was intense in the
  final part of the project, but it did not finish,
  according to what project partners declared.
  Beside internal effectiveness (the capacity to
  achieve objectives) it seems that the project had
  also external effectiveness (the capacity to
  produce results able to satisfy beneficiaries
  expectations).

17
Conclusions

• Utility Are the expected or unexpected effects
  globally satisfactory from the point of view of
  direct or indirect beneficiaries ?
• Though external evaluation does not deal
  explicitly with beneficiaries satisfaction, the
  analyses carried out by partners on training
  participation testify a visible interest and
  approval from participants. As for indirect
  beneficiaries (partner organisations and key
  actors) collected opinions indicate remarkable
  satisfaction for work methodology and results
  usability. Some recurrent indications (such as
  the need for considering national frameworks and
  the need for translation in national language)
  may improve results utility.

18
Conclusions

• Sustainability Are the results and impacts
  including institutional changes durable over
  time? Will the impacts continue when public
  funding is over ?
• The impact effects realised by the partnership
  seem to have activated a process of change and
  further development of outcomes which is going on
  in many partner countries. Durability over time
  seem guaranteed by partners commitment and by
  their roles as VET policies planners and
  implementers.

19
Thank you for your attention !!!