EVALUATION & NEGOTIATION

FP7 applications delivered on time in Brussels first undergo a formal control. This formal control is carried out by the staff of DG Research and/or the staff of the DG in charge for the call under the particular theme. A list of formal criteria is normally enclosed in the application package; therefore the applicants can do a check themselves on whether they pass all formal criteria.

The evaluation process starts with a briefing for the evaluators, where the administrative and professional aspects of the evaluation are addressed in detail. Commission priorities will be highlighted here. The issues of Conflict of Interest and a general code of conduct are also described in detail.

Evaluations normally have two phases: Individual Evaluation and the Consensus Evaluation. An optional Phase 3 (Hearing) may be applied to some proposals. Each proposal is evaluated by 3-5 independent evaluators.

Individual evaluation

During the individual evaluation the evaluators are given a copy of the project proposal with all annexes and application forms. The evaluators are expected to be familiar with the professional issues as well as knowing the details of the relevant Work Programme and Guide for Applicants.

The evaluation forms are comprised of a set of questions addressing each evaluation criteria. The general criteria can be downloaded from Cordis and are useful for the proposer during proposal development. Three criteria (Scientific and technological quality, implementations, and impact) are typical for most FP7 proposals but could be different for two stage proposals or calls under the People and Idea programmes.

Remote Evaluation

Remote Evaluation was introduced in FP6 and is now used more intensively in FP7. It allows the evaluators to access the proposals remotely, using RIVET - Remote Individual Evaluation Tool, and carry out the work at home or normal working place. The use of RIVET allows for a more careful assessment of the proposals. Experts must report any conflict of interest immediately after their first login and access to the documents. Individual Assessment Reports are produced electronically and the system also produces a Draft Consensus Report, which is comprised of all comments by the individual experts' comments and marks for each block of criteria and corresponding average marks.

The Consensus Meeting

In order to synchronise the views of the individual evaluations a Consensus Meeting is organised between the independent experts evaluating the same proposal. The objective of this is to reach an agreement on the final score of an application. Since evaluators come from various countries and have different professional backgrounds (e.g. industry, research, university, consultant, etc.) it might take some professional debates to come to an agreement (full consensus).

Panel Meeting

Once all proposals are evaluated applications are ranked on the basis of scores developed by the consensus-evaluation. Usually on the last day of the evaluation period a so-called Panel Meeting takes place with the objective of reviewing all proposals and establishing a final ranking. This list will have the proposals from the best ones to the lowest-scoring one and any financial support will strictly be awarded according to this order. For each programme there are limited financial resources available and these resources will determine the final threshold of this list. Starting from the very best proposal the Commission allocates finances to the proposals in strict order, until all available resources are committed. Proposals that are above the financial threshold will be invited to contract negotiations. The eventual provision of financial support is, however, always subject to the successful completion of contract negotiations.

Proposals of sensitive nature may also be subject of an Ethical Review. Failing to address the ethical aspects of your proposal may result in being rejected by the Ethical Panel. The Ethical Review Report is comprised of two pages with the following sections:

• Agreed consensus commentary
• Additional Comments
• Requirements: (Requirements become contractual obligations)

The Coordinator will receive the feedback from the evaluation in form of an Evaluation Summary Report (ESR) a few months after the submission.  The coordinators for the selected proposals will receive an Invitation letter containing a mandate to start the negotiation.

There are two types of individual evaluation based on the location of the evaluation. In a normal case the evaluation is held at the premises of the Commission in Brussels. The other option is the remote evaluation when the evaluators access the proposals remotely and carry out the evaluation at home or at their normal working place.

First the evaluators are given a copy of the project proposal with all annexes and application forms. It is not allowed for the experts to discuss any project proposals at this stage and it is also quite common that none of the evaluators know who else is reading the same proposal. There is a maximum period of two-three hours for the evaluation of one proposal; meanwhile they have to fill-in the evaluation form including comments. The evaluators are expected to be familiar not only with the professional issues, but also knowing the details of the relevant Work Programme, Guide for Applicants and with the rules for submission of proposals.

In case of remote evaluation, the evaluators are using RIVET - Remote Individual Evaluation Tool that allows for a more careful assessment of the proposals at home or at their normal working place. Experts must report any conflict of interest immediately after their first login and access to the documents. Individual Assessment Reports are produced electronically and the system also produces a Draft Consensus Report, which is comprised of all comments by the individual experts' comments and marks for each block of criteria and corresponding average marks.

The evaluation forms are comprised of a set of questions addressing each criteria. During evaluation each of these criteria will be marked and justified by written comments. Three criteria (Scientific and technological quality, implementation, and impact) are typical for most FP7 proposals but could be different for two stage proposals or calls under the People and Idea programmes. The general criteria can be downloaded from Cordis and useful during proposal preparation.
 

The three general evaluation criteria
1, Scientific and/or technological (S&T) excellence (relevant to the topics addressed by the call):

• Soundness of concept, and quality of objectives
• Progress beyond the state-of-the-art
• Quality and effectiveness of the S/T methodology and associated work plan

 2, Quality and efficiency of the implementation and the management:

• Appropriateness of the management structure and procedures     
• Quality and relevant experience of the Individual participants       
• Quality of the consortium as a whole (including complementarities, balance)
• Appropriateness of the allocation and justification of the resources to be committed (budget, staff, equipment)

3, Potential impact through the development, dissemination and use of project results:

• Contribution, at the European and/or international level, to the expected impacts listed in the relevant work programme(s)
• Appropriateness of measures for spreading excellence, exploiting results and disseminating knowledge through engagement with stakeholders and the public at large

Scoring
Scores are in the range 0-5. Half (sometimes decimal) marks may be given. The interpretation of the scores is as follows:

0 - The proposal fails to address the criterion under examination or cannot be judged due to missing information
1 - Very poor: the criterion is addressed in a unsatisfactory manner
2 - Poor: there are serious inherent weaknesses in relation to the criterion in question
3 - Fair: while the proposal broadly addresses the criterion, there are significant weaknesses that would need correction
4 - Good: the proposal addresses the criterion well, although certain improvements are possible
5 - Excellent: the proposal successfully addresses all relevant aspects of the criterion in question

For each criteria (S&T excellence, implementation, impact) the minimum threshold is mark three from the maximum five and there is also a general threshold of mark 10 from the maximum 15 for the proposal. Special rules may apply for specific rules, however, it is always indicated in the call documents. Passing the minimum threshold does not necessary means to get invited for negotiation. It depends on the final ranking of all proposals.

Individual evaluation typically lasts for 2-3 hours. During this time the evaluators must read and understand the proposals and fill in the evaluation sheets. The time factor is therefore crucial, all chapters of the proposal must be clear, concise and to the point. At the end of the individual evaluation all copies of the proposals are returned to the Commission Secretariat for administrative processing before the Consesus Meeting.

A few months after the submission, the Coordinator will receive the feedback from the evaluation in form of an Evaluation Summary Report (ESR).  Based on the evaluations the European Commission (EC) will decide which projects can receive funding. The coordinators for the selected proposals will receive an Invitation letter containing a mandate to start the negotiation. 

The Participant Portal is a useful information sources that complements the CORDIS during the negotiation stage. At the Participant portal it is possible to check which organisations already have a Partner Identification Code (PIC), appoint  the Legal Entity Appointed Representative (LEAR), apply for a PIC code and many other activities.

Letter of Invitation

Following the positive evaluation of a proposal, and the definition of an appropriate maximum Community financial contribution for the work, the consortia is invited to commence negotiations with the Commission for a Grant Agreement (ECGA - EC Grant Agreement).

The Invitation letter contains information on the next steps to finalise the Grant Agreement, references to Guidance documents, the maximum financial contribution. In addition to the Evaluation Summary Report, attached once again, the letter also contains the name and availability of the Legal and Scientific EC Officers who are responsible for the project during the negotiation.

Grant Agreement - Description of Work
From a scientific point of view, the proposal will be transformed into a Description of Work (DoW). The DoW is sometimes also called the Technical Annex (TA). The outline is the same as the proposal with a few additional tables. The Guide for Negotiations contains the template; it can be downloaded from Cordis.

The proposal will be transformed according to the evaluators’ comments, the input from the Project Officer and the consortium members. In some cases, there may also be general EC priorities and policies that have changed between the proposal preparation and negotiation, which also must be taken into account. This will all be discussed with the Scientific Officer of the project.

Negotiation Electronic Facility (NEF)
Often, the coordinator is invited to Brussels for an initial meeting, but the main part of the negotiation takes place in an electronic system called Electronic Negotiation Facility (NEF).  In the beginning of the negotiation, the coordinator will receive an email with an access code for the coordinator and one for the partners. Some of the fields are pre-filled according to the information already available for the Commission.
 
The partner related information to be filled by each organization:

• Legal data - pre-filled for validated organizations (can only be changed by the LEAR)
• Authorised representatives - persons who will sign the Grant Agreement with legal capacity to enter into a legally binding agreement.
• Contact persons - administrative and scientific responsible persons
• Eligible costs - according to the budget negotiated
• Bank account - to be filled only by the coordinator
• The project related information is filled in by the Coordinator:
• General information - abstract
• Reporting periods - including the amount of funding requested per each period.

When all fields are correctly filled out, the coordinator will generate a PDF version of the information, and then press "submit" and the system closes. Once the Legal Officer accepts the information, the coordinator sends the PDF versions to the partners who will sign the A2:5 form "our commitment". The coordinator collects all the forms, electronically and in paper form, and submits it to the EC.

Each partner must be validated to take part in an FP7 project. Once validated, the organisation will have a Participant Identification Code (PIC) code that is valid for the whole duration of the Seventh Framework programme. You will register the organizations data only once in the Unique Registration Facility (URF) to get a PIC code, which lowers the administrative burden.

The Central Validation Team (CVT) is dealing with the validation process and all documentation will be sent there, by email and by registered post. The validation process may take some time. You can initiate the process already when submitting the proposal. The following is a step-by-step guide to get validated.

1. Register to get an European Commission Authentication Service (EASC) account
2. Via the EASC account, fill in the information in the Unique Registration Facility to get a PIC code
3. Send documents to URF Validation Team

• Legal Entity Form
• Registration documentation

• Private companies: an extract of registration or similar and if applicable, a copy of a document proving VAT registration
• Public bodies: a copy of the resolution, law, decree, decision or any other official document establishing the organisation and if applicable, a copy of a document proving VAT registration

1. Appoint a Legal Entity Appointed Representative (LEAR)

Legal documents can be submitted in any official EU language while documents in other languages should be accompanied by an official translation.


LEAR - Legal Entity Appointed Representative

Each organization participating in FP7 shall appoint a Legal Entity Appointed Representative - LEAR. The LEAR is the legal contact point and will be responsible for providing and updating relevant legal & financial data via the Participant portal.

The organization will appoint a LEAR and send a signed and stamped LEAR Appointment Form and the LEAR role and tasks document that can be downloaded from http://cordis.europa.eu/fp7/pp-lear_en.html

Finalising the Grant Agreement

When both the administrative and the scientific parts have been prepared and completed, the European Commission's services will send a Grant Agreement to the Coordinator. The Agreement is concluded between the Coordinator and the Commission and the other beneficiaries will accede to the same. The core Grant Agreement is only a few pages but contains several Annexes depending on the funding scheme and specific programme. After a minimum number of participants have signed the Grant Agreement, the project can begin.