Evaluation of IP

1
A proposal evaluators view
EVALUATION OF IP AND NoE Sylvia Ilieva, Faculty
of Mathematics and Informatics, University of
Sofia St. Kl. Ohridski
Topics

• Evaluation of IP
• Evaluation of NoE
• Hearings for IPNoE

2
Individual Assessment Report for an Integrated
Project

• 1. Relevance (Threshold 3/5)
• 2. Potential impact (Threshold 3/5)
• 3. ST excellence (Threshold 4/5)
• 4. Quality of the consortium (Threshold 3/5)
• 5. Quality of the management (Threshold 3/5)
• 6. Mobilisation of resources (Threshold 3/5)
• Overall remarks (Threshold 24/30 for an IP)

3
Integrated project - Relevance
1. Relevance (Threshold 3/5 )

• The extent to which
• the proposed project addresses the objectives of
  the work programme

4
Integrated Project - Relevance
Some weaknesses

• IST priority and strategic objectives are not
  mentioned
• The objective is not clearly stated
• Just copy/paste from the workprogram

• IST priority and strategic objectives are not
  mentioned
• The objective is not clearly stated
• Just copy/paste from the workprogram

Recommendations
Recommendations

• Specify the objective,
• state your objective relevance to IST priority
  and
• explain why it is important in your own words

5
Integrated project
2. Potential impact (Threshold 3/5 )

• The extent to which
• the proposed project is suitably ambitious in
  terms of its strategic impact on reinforcing
  competitiveness (including that of SMEs) or on
  solving societal problems
• the innovation-related activities and
  exploitation and/or dissemination plans are
  adequate to ensure optimal use of the project
  results
• the proposal demonstrates a clear added value in
  carrying out the work at European level and takes
  account of research activities at national level
  and under European initiatives (e.g. Eureka)

6
Integrated project Potential impact
Some weaknesses

• Described impact is not in fact related to IST
  priorities
• There is no clear industrial impact
• Unreasoned statements how good the results will
  be

Recommendation

• Clearly identify impact in detailed terms,
  showing deep knowledge of the area and original
  thinking

7
Integrated project
3. ST excellence

• The extent to which
• the project has clearly defined objectives
• the objectives represent clear progress beyond
  the current state-of-the-art
• the proposed ST approach is likely to enable the
  project to achieve its objectives in research and
  innovation

8
Integrated project ST excellence
Some weaknesses

• The objective is not ambitious enough or not
  original
• Lack of fundamental research
• Focus on state-of-the-art
• Artificial integration of activities

Recommendation

• Clear explanation of quality of activities
  advancing the state-of-the-art in radical or
  original way

9
Integrated project
4. Quality of the consortium

• The extent to which
• the participants collectively constitute a
  consortium of high quality
• the participants are well-suited and committed to
  the tasks assigned to them
• there is good complementarity between
  participants
• there is adequate industrial involvement to
  ensure exploitation of results
• the real involvement of SMEs has been adequately
  addressed

10
Integrated project - Quality of the consortium
Some weaknesses

• Including many and long CVs
• General statements about partners activities and
  competences
• Examples of previous work

Recommendations

• Describe clearly partners and their activities,
  focused on the objectives addresses, and facts to
  prove those partners are best in the business

11
Integrated project
5. Quality of the management

• The extent to which
• the organisational structure is well matched to
  the complexity of the project and to the degree
  of integration required
• the project management is demonstrably of high
  quality
• there is a satisfactory plan for the management
  of knowledge, of intellectual property and of
  other innovation-related activities

12
Integrated project - Quality of the management
Some weaknesses

• Copy/paste the management plan from University
  books
• No clear responsibilities for partners
• Potential risks are not identified and there are
  no adequate mechanisms foreseen

Recommendations

• Clear and flexible plan including contingency
  planning and intellectual property rights handling

13
Integrated project
6. Mobilisation of resources

• The extent to which
• the project mobilises the minimum critical mass
  of resources (personnel, equipment, finance )
  necessary for success
• the resources are convincingly integrated to form
  a coherent project
• the overall financial plan for the project is
  adequate

14
Integrated project - Mobilisation of resources
Some weaknesses

• Lack of industry-academia collaboration
• Existence of token partners
• The recourse plan is not specific and detailed
• Resources are over estimated

Recommendations

• Find out the right amount of resources,
  convincingly integrated

15
Integrated projects

• ?
• Scale of ambition is limited, reflected in
  limited activities, duration, size of consortium
  more typical of a STREP
• ?
• Goals are ambitious. The proposers include key
  industry players. A full range of activities
  covering a substantial part of the development
  chain is envisaged. Broad industry sector impact
  is foreseen. SME participation has been
  sought/achieved. Effective project management
  plans are in place

16
A proposal evaluators view
EVALUATION OF IP AND NoE Sylvia Ilieva, Faculty
of Mathematics and Informatics, University of
Sofia St. Kl. Ohridski
Topics

• Evaluation of IP
• Evaluation of NoE
• Hearings for IPNoE

17
Individual Assessment Report for Network of
excellence

• 1. Relevance (Threshold 3/5)
• 2. Potential impact (Threshold 3/5)
• 3. Degree of integration and the JPA (Threshold
  4/5)
• 4. Excellence of the participants (Threshold 3/5)
  
• 5. Organisation and management (Threshold 3/5)
• Overall remarks (Threshold 20/25 for an NoE)

18
Network of excellence
2. Potential impact - 1

• The extent to which
• Europe has a strategic need to strengthen ST
  excellence on the topic by means of restructuring
  of the existing research capacities and the way
  research is carried out
• the goals of the network are in that connection
  suitably ambitious, particularly in terms of
  achieving European leadership and acting as a
  world force on this topic

19
Network of excellence
2. Potential impact - 2

• the proposal demonstrates a clear added value in
  carrying out the work at European level, does it
  take account of research activities at national
  level and under European initiatives (e.g.
  Eureka).
• an effective plan for spreading excellence,
  exploiting results and disseminating knowledge,
  including to SMEs and to those outside the
  network.
• the proposed approach is likely to have a durable
  structuring impact on European research

20
Network of excellence
3. Degree of integration and the JPA

• The extent to which
• the degree of integration justifies supporting
  the proposal as a network of excellence
• the joint programme of activities is sufficiently
  well designed to achieve the expected degree of
  integration
• the participating organisations have made a
  convincing commitment towards a deep and durable
  integration beyond the period of Community support

21
Network of excellence - Degree of integration and
the JPA
Some weaknesses

• Many research activities, no clear integration
• There are no foundations laid for cooperation
  beyond project
• There is no clear restructure of the fragmented
  research

Recommendations

• Permanent integration of the key players in an
  important but fragmented research segment

22
Network of excellence
4. Excellence of the participants

• The extent to which
• the participants are currently conducting
  excellent research relevant to the topic of the
  network or are capable of important contributions
  to the joint programme of activities
• the participants are well suited to the tasks
  assigned to them
• they collectively have the necessary critical
  mass of expertise and resources to carry out the
  Joint Programme of Activities successfully

23
Network of excellence - Excellence of the
participants
Some weaknesses

• Direct involvement of industry
• Long partners descriptions with copy/paste text
  from brochures or other advertise materials
• Unclear commitment to the specified activities

Recommendations

• Involve leading research organizations for the
  specified objectives with clear description what
  they are doing

24
Network of excellence
5. Organisation and management

• The extent to which
• the organisational structure of the network
  provides a secure framework for any necessary
  structural decisions to be taken
• the management of the network is demonstrably of
  high quality
• there is a well-considered plan for promoting
  gender equality in the network

25
Network of excellence Organization and
management
Some weaknesses

• The same partners involved at all management
  levels
• Lack of problem solving mechanism

Recommendations

• Detailed plan reaching beyond the end of the
  project

26
Network of excellence

• ?
• Proposal for ad hoc co-operation between
  organisations for a specific purpose - no
  evidence of durable integration. The participants
  are going to continue to work as they have always
  done with no significant changes of structures,
  portfolios and organisations
• ?
• The research area is fragmented and would gain in
  excellence by re-structuring and all the
  participants intend to undertake a major effort
  to re-structure and re-organise the way research
  in the area is carried out in Europe and
  establish durable integration of their research
  capacities

27
A proposal evaluators view
EVALUATION OF IP AND NoE Sylvia Ilieva, Faculty
of Mathematics and Informatics, University of
Sofia St. Kl. Ohridski
Topics

• Evaluation of IP
• Evaluation of NoE
• Hearings for IPNoE

28
Hearings - recommendations

• It is not expected to improve or add something
  new to the original proposal
• Selection of most appropriate partners who are
  competent to answer the given questions
• Be sure you are answering exactly the given
  questions
• Prepare your presentation professionally

29
Hearings
Examples of questions from hearings

• Which IPR issues are expected to play a role in
  the IP/NoE and how are these managed?
• What is precisely the integration work of each of
  the Work packages, as opposed to the research
  work?
• How will the industrial partners be involved and
  how do they see their role?
• How will the IP project manage the
  complementarity and synergy with NoE project on
  the same subject?

30
A proposal evaluators view
EVALUATION OF IP AND NoE Sylvia Ilieva,
University of Sofia St. Kl. Ohridski
sylvia_at_acad.bg
Thank you for your attention!