Proposers

1
Evaluation of research proposals. Experience of
Moldovan Advisory Expertise Council Science
evaluation as a prerequisite for promoting
excellence in research. Regional workshop
Chi?inau, November 6-th, 2012
Dr. Veaceslav Kulci?ki Advisory Expertise
Council, Moldova
2
Science evaluation as a prerequisite for
promoting excellence in research Regional
workshop Chi?inau, November 6-th, 2012

• Outline
• Competition based selection of research projects
  in RM. Background
• Founding of Advisory Expertise Council (AEC).
  Launching of national competition based research.
• Project management cycle in Moldovan Research
  Institutions
• Evaluation of project proposals performed by
  (AEC)
• Current trends in evaluation process

3

• Competition based selection of research projects
  in Moldova. Background
• Activity of international funds
• Soros foundation (1992), INTAS (1993), NATO,
  MRDA-CRDF (1999)
• 2. Projects supported
• Individual research projects, collaborative
  projects (research teams), conference grants,
  travel grants
• Evaluation activity
• International peer review most projects.
  Limited involvement of local experts

4

• Launching of Moldovan National competition based
  research
• Founding of Advisory Expertise Council 2004.
• Major goal promotion of research on competitive
  basis.
• Project calls launched by Supreme Council For
  Science and Technological Development (SCSTD).
• Priorities promoted state research programs,
  projects for young scientists, scientific
  equipment , international collaborations
• Institutional projects on competitive basis
• 2. Evaluation activity
• Local peer review most projects. Limited
  involvement of international experts.

5

• Moldovan RI area. Evaluation management
  principles
• 1. Separation of evaluation from policy-making
  and executive activities.
• Actions Establishing of separate entities for
  executive activities and separation of Advisory
  Expertise Council (AEC) as an individual Public
  Institution.
• 2. Clear project management procedures and
  selection principles.
• Actions Procedures defined in the official
  agreement between Moldovan Government and Academy
  of Sciences
• 3. Assignment of independent evaluators and peer
  reviewing promotion.
• Actions Setting-up a database (gt 500) of local
  experts grouped by research directions

6

• Moldovan RI area. Project management cycle

Step
Responsible organization
Definition of strategic priorities/policies
SCSTD
Center for BARF
Launching project calls
Center for IntProjects
Agency for I TT
Proposals evaluation
AEC
Research units
Running the projects
Launching units
Projects monitoring
SCSTD
Macro monitoring
7

• EVALUATION PROCEDURE PERFORMED BY MOLDOVAN
  ADVISORY EXPERTISE COUNCIL. GENERAL OBJECTIVES
• Promotion of research activities under
  competition based on excellence
• A transparent and unambiguous definition of the
  entire evaluation process
• Establishing clear responsibilities for all
  process participants
• Reaching high process quality by promotion of
  objective approach
• Monitoring of evaluation process and setting up
  of performance criteria
• Planning of corrective actions for removal of
  discrepancies in evaluation
• Measures for continuous improvement of evaluation
  process

8

• PROJECT PROPOSALS SUBMISSION
• Project proposal collection in printed and soft
  format via www portal. Deadlines are set for each
  call.
• Eligibility check by the project launching
  bodies. Eligibility criteria defined in call
  announcement.
• Valid projects are submitted to Advisory
  Expertise Council for evaluation. A formal letter
  with the list of projects accompanies the
  package.

9

• INDEPENDENT PEER REVIEWERS BASIC EVALUATION
  TOOL
• AEC maintains a database of experts for a set of
  relevant research fields
• The expert database is updated on continuous
  basis. This process is based on recommendations
  from scientific community and as well as minimal
  formal requirements
• Members of scientific Diaspora are actively
  promoted as evaluators
• Highly qualified professionals from industry,
  business sector or NGOs are promoted as
  evaluators on recommendation of Governamental
  institutions (Ministers and other agencies).

10

• SELECTION OF INDEPENDENT PROJECT EVALUATORS
• Selection of evaluators represent a joint task of
  AEC staff and AEC Advisory Board
• On the recommendation of the relevant Advisory
  Board members (research field related), the AEC
  staff selects two evaluators for each national
  project (one for bilateral ones). AEC expert
  database represents an input for this step. The
  decision is taken basing on the narrow
  specialization data of evaluators.
• The expert is contacted by AEC staff and project
  summary is submitted. The selected expert decides
  either to evaluate or not the submitted project.
  On his approval, the integral project is
  submitted.
• AEC concludes a written contract with each
  evaluator. He is paid for his services.
• Evaluators are being evaluated by AEC according
  to a special procedure.

Expert database
AEC Advisory Board
AEC staff
7
20
700
11

• EVALUATORS ACTIVITY
• Evaluation form the basic tool for experts
  activity.Elaboration of these forms a very
  responsible task. Should be done before call
  launch on agreement with entities responsible for
  call launching.
• The first requirement a total match between
  project proposal layout and evaluation criteria
  integrated in the evaluation form. The
  evaluation elements should be made known to
  applicants as priority items of project
  definition.
• Additional tools thresholds and weights
• Mandatory fields strengths and weaknesses of
  proposal
• All this must be made known to public on call
  launch !!!

Project proposal
Evaluation form
Project definition field
Evaluation criteria
Specific items
Evaluation elements
12

• Evaluation form sample

13

• ADVISORY BOARD MEETINGS
• Are launched after evaluation is complete. The
  members of the advisory board examine and discuss
  the evaluations performed by independent experts.
• Projects are recommended for financing basing on
  the total score accumulated and considering
  available resources.
• Winning projects are distributed to all strategic
  directions of research, in proportional number of
  total applications received within the call.
• All the decisions are taken by consensus.
• The competition results are delivered to the
  entities that have launched the call.

14

• CURRENT TRENDS IN PROJECTS EVALUATION
• Internationalization of evaluation. Regional
  cooperation
• Overcoming language barriers switch to
  submission in English
• Deeper involvement of scientific Diaspora in the
  evaluation
• Elaboration of criteria and procedures for a
  better selection of experts and their evaluation.
  Making use of scientometric data.

15

• Thank you for your attention !