Texto publicado no Euroscientist, uma publicação da associação de investigadores EuroScience (na foto a direcção da FCT agora reconduzida por Nuno Crato, excepto Miguel Seabra):
Portugal not yet cleared of the ESF evaluation fallout, despite FCT president’s resignation
Until a few weeks ago, Miguel Seabra was president of the Portuguese Foundation for Science and Technology (FCT). He resigned on 7th April 2015, invoking personal reasons, as rumours about him having kept his position at Imperial College while being at FCT were beginning to circulate. Maria Arménia Carrondo, a researcher from the same laboratory as Seabra has just been appointed by the government as his successor. Portuguese scientists are waiting to see whether she will maintain the status quo or attempt to find constructive solutions. Above all, the Portuguese science community is now calling for increase in transparency and integrity.
In any event, Seabra’s position at the helm of the Portuguese funding agency was becoming untenable. He had been under tremendous pressure following the discredit shed on the FCT by a controversial evaluation of Portugal’s research units, outsourced to the European Science Foundation (ESF). The move was designed to select units that would need to be closed as part of a national austerity drive and due a reduction of European funds that may be used for research. A big question mark has been hanging over the quality of the evaluation since the Portuguese science community started to realise that, in many ways, those bound to survive the cut were not to do so on scientific merit.
Before Seabra resigned, an unexpected 40% cut on the funding of top ranked research units was announced. This follows the planned elimination of 50% of all research units in the country, made public in June 2014. In addition, FCT also announced further restrictions on doctoral and post-doctoral grants. This move will push many researchers into either unemployment or towards emigration, due to lack of alternatives.
Flawed evaluation
As the questionable conditions of the FCT evaluation emerged, the disastrous situation for the Portuguese research community became increasingly obvious to international observers. As it turns out, Spanish astrophysicist Amaya Moro Martin was right. In October 2014, she wrote an opinion piece in Nature, regarding science policy in Europe. She included a reference to Portugal saying: “Already reeling from budget cuts of 50% for universities and research centres, Portugal may now have to close half of its research units because of a flawed evaluation process supported by the ESF.”
Shortly after that, the head of science support of ESF, Jean-Claude Worms, wrote an email to Amaya, threatening her with a lawsuit. ESF backed off on this ridiculous intention a week later, perhaps realising that victimising a young researcher was not a good move. The ESF wikipedia entry sums up the impact the move has had on ESF’s reputation in the following terms: “ESF was very much discredited after a controversial evaluation of the Portuguese research centres.”
To say that the evaluation process is flawed is an understatement. It was a scam setup to justify the closure of half of the research units in the country and concentrate the major financial resources on a small number of units; incidentally, the units related to the current science policy makers in Portugal were awarded generous funding.
Some of the flaws have been identified by members of the Portuguese science community—with details described here and here. They include changing the rules when the process was well underway. And they also involve the use of incorrect bibliometric data. The Council of Rectors of Portuguese Universities (CRUP), in November 2014, protested with strong words, qualifying the evaluation as ‘a total failure.’ Three of the more outrageous issues identified, include the use of hidden quotas, an inadequate evaluation panel and the arbitrary allocation of funding.
Hidden quotas
But let’s get further into the details of what went wrong. The evaluation was carried out in two stages. Only the units proceeding to stage two were eligible for the large part of the funding available. The others will have no funding at all or funding at such a low level that it will not allow them to pursue any activity. In short, units that are not proceeding to stage two are condemned to disappear.
What nobody knew at first is that half of the research units were condemned even before the evaluation had started. After legal pressure FCT made public the contract with ESF. It then became obvious that ESF had been explicitly instructed to a priori exclude 50% of the units from stage two. The work plan for the panels spells it out in the following terms: “Stage 1 evaluation will result in a short list of half of the research units that will be selected to proceed to stage 2.” ESF complied. To this day FCT stills denies the existence of quotas, making their own very peculiar interpretation of what is written in the contract (please check for yourself). Stage one did not include site visits, as is mandatory by law. Sitting at their desks, ESF evaluators simply killed half of the Portuguese research units at a stroke of the keyboard
Light evaluation panel
However, this was not, by far, the only issue. When it comes to the adequacy of the review panels, the operation details of the evaluation speak for themselves. ESF set up a very small number of panels. For example, in the 2007 international evaluation 15 mathematicians, six physicists and seven chemists were gathered in three panels to analyse the units of the corresponding disciplines.
In 2014, the ESF formed a single panel for the three disciplines, with only 11 members (one engineer, three physicists, four chemists and three mathematicians). This panel, which decided to shut down several condensed matter physics labs in the country, had a single condensed matter physicist. In stage two, a team of evaluators visiting an architecture unit did not include a single architect and there were several other examples of clear mismatches between panels and the units they were supposed to evaluate.
Arbitrary funding
Furthermore, the most recent FCT funding allocation does not display a clear correlation with the ESF assessment, considering comparable unit sizes and laboratory intensity. There are 11 units classified as ‘exceptional,’ which is the highest possible grade. Of the five units with the highest funding awarded, only one was classified as exceptional. The unit with the highest funding per capita is in the humanities and is headed by a former chair of an ESF scientific committee.
So FCT asked ESF to make an assessment and, in the end, “modulated”—this is the actual FCT word —the output to award funds in an almost arbitrary way. This means that FCT called ESF as external institution to enforce predetermined funding intentions, regardless of scientific merit. ESF, who was paid a consultancy fee, was only too happy to oblige.
Out of the 322 research units being evaluated, 178 made it to stage two. Meanwhile, the majority of those, which did not make the grade, have appealed the decision, first to FCT, and then by demanding a new evaluation process. The latter is to be carried out by a different panel according to FCT’s own regulations. Four lawsuits are already underway. Out of the 178 “lucky ones,” 123 filed complaints regarding the second stage results, failing to see the logic of the process. As a final despicable move, FCT gave a three-day deadline for units to accept whatever funding befell them.
Portuguese scientists clearly have reasons to be concerned, but maybe they should not be the only ones. Miguel Seabra still remains president of Brussels-based research advocacy group Science Europe, whose aim is to influence European research policies. With current European commission winds blowing against fundamental research, having someone with such a track record in science management cannot be a good omen.
Professor of Physics, University of Coimbra , Portugal.
Sem comentários:
Enviar um comentário