Portuguese research units are funded by the Fundação para a Ciência
e Tecnologia (FCT), the only national public funding agency for science. FCT
has been carrying out periodic evaluations for more than 20 years, conducted by
scientists from foreign institutions with its outcome determining the levels of
funding.
The latest (and still ongoing) evaluation was outsourced to the
European Science Foundation (ESF). The
practical outcome of this evaluation, if carried out to full completion, will be a huge
cut in the number of viable research units and the concentration of most of the
financial resources in a very small number of institutions. At first sight this
might seem a legitimate option, but it is not.
First of all, this “shrink and concentrate” policy was not announced
by the Portuguese government, and as a consequence was not submitted to a
public discussion with the relevant institutions such as universities, who host
the research units and pay the salaries for the majority of the senior
researchers in the system. Instead, it
was disguised as an evaluation based on merit only, passing the full responsibility
of its outcome to ESF. However, ESF
conducted what was, as shown below, a severely flawed evaluation process, in
which the absolute classifications of the research units had to fit into
predefined quotas.
Contrary to all previous experience and even the law, the evaluation
was conducted in two stages, the first based on documents only (with no visits to laboratories)
and the second based on visits and interviews with the unit director and its researchers. Only units
that made it to the second stage will have access to the major part of the
funding to be awarded. The first stage suffered, for instance, from the lack of
proper coverage of the different areas by the panels who made the decisions,
the change of rules when the process was already underway, and the usage of
incorrect bibliometric data. These serious flaws alone, denounced by a large number of
scientists and associations in Portugal ( http://goo.gl/KHYDwZ ) and many foreign
scientists, of which Amayo Moro-Martin is but one example (article in Nature: http://goo.gl/U8OU9a), would be enough reason
to suspend this evaluation. ESF announced a legal action against
Moro-Martin, in view of her
opinion piece in Nature, but after this became public ESF backed down.
But, even more significantly, the evaluation had a hidden quota from the very beginning. After strong legal pressure, FCT finally made public the contract signed with ESF from which we learnt that ESF had been instructed to a priori exclude 50% of the units from the second stage. Indeed, as may be read in the work plan for the panels: "Stage 1 evaluation will result in a short list of half of the research units that will be selected to proceed to stage 2." The evaluation had thus to abide by this hidden quota while it was being publicized as being driven by quality alone.
But, even more significantly, the evaluation had a hidden quota from the very beginning. After strong legal pressure, FCT finally made public the contract signed with ESF from which we learnt that ESF had been instructed to a priori exclude 50% of the units from the second stage. Indeed, as may be read in the work plan for the panels: "Stage 1 evaluation will result in a short list of half of the research units that will be selected to proceed to stage 2." The evaluation had thus to abide by this hidden quota while it was being publicized as being driven by quality alone.
It should be stressed that this was not a relative evaluation to find
the 50% best units, in some kind of “excellence” program, but the financing of the basal science system and
there is virtually no alternative funding from other Portuguese sources. Units
which did not make it to the second stage will most likely not be able to
continue. This alone would already be cause for concern, but it does become
totally unacceptable when based on a flawed evaluation. Several complaints have
been presented and, given the number and seriousness of formal and scientific irregularities, it may be expected that new independent panels will respond positively to the appeals.
These were the quotas for the first stage. In a recently disclosed
document ( http://www.fct.pt/apoios/unidades/avaliacoes/2013/docs/FCT_Final_Meeting_guidelines_S2.pdf ,
FCT defined the quota for the classifications of the second stage. Only 10% of
the units assessed in the second stage are allowed to have the highest
classification (“exceptional”). And only up to 35% of the units assessed in the
second stage are allowed to have the second higher classification (“excellent”).
As in the first phase, very few evaluators visited the units and some of them
were not specialists in the areas being examined.
We have to bear in mind that of a total of 322 units, only 178 have
made it to the second stage and that those classified as “exceptional” will
have 10 times more funding as the units classified as “good”. This means that a
large part of the Portuguese public funds for research will be awarded to only
17-18 units, several of which will probably be private (private foundations
will be subsidized with public money).
This evaluation has been highly contested by many scientists,
organizations and national scientific societies (Physics, Chemistry,
Philosophy, Mathematics, etc.). More significantly, the Council of Rectors of
Portuguese Universities (CRUP) has completely demolished the process. On the
24th of October the 15 rectors of CRUP released a consensus position which was
sent to the Minister of Education and Science, Nuno Crato To this day there has
been no reaction from the Minister, denoting that the open letter was a serious
political embarrassment for the government. A translation of the letter may be
found here:
The minister has not replied
to any of these issues so far.
In short, this evaluation is politically driven, meant do downsize
science in Portugal and concentrate all the resources in a lucky few, selected
by what many scientists deem to be a ridiculous evaluation process.
Furthermore, it is illegal in various aspects, and indeed a legal action has
already been filed by a researchers union (probably others will follow soon).
Despite all this, the Portuguese Secretary of State for Science and
Technology, has stated recently that the evaluation "is running smoothly
and according to plan.” Preliminary results of the second phase are about to be announced, but a bad plan
and a flawed methodology can only produce bad results.
Carlos Fiolhais, Professor of Physics, University of Coimbra (Portugal)
Sem comentários:
Enviar um comentário