fbpx Here's how Cariplo assesses merit | Science in the net
  1. Here's how Cariplo assesses merit
  2. Revisions
  3. Here's how Cariplo assesses merit

Here's how Cariplo assesses merit

di Marina Castellano
Pubblicato il 29/09/2011Read time: 4 mins
Research evaluation

Another step towards a fair assessment of research projects comes again from the private sector: we are talking about the Cariplo Foundation. On the wake of Telethon that opened the road by importing the peer-review system inspired by the U.S. National Institutes of Health, which is considered the best in the world, the merit-based and impartial model is taking place among private "funding agencies".

The Cariplo Foundation, officially founded in 1991, represents the historical continuation of the Cassa di Risparmio delle Provincie Lombarde established in Milan in 1823 with philanthropic aims and active in four areas: Environment, Arts and Culture, Scientific research and Services to Citizens. Its activity is based on income arising from the management of its assets, valued at approximately 6.36 billion euro, out of which the Foundation allocates about 190 million euro per year to non-profit entities.

The selection of projects to be financed has an extraordinary strategic importance for all charities and the Cariplo Foundation, at least in the field of Scientific Research, in recent years has set up an ex-ante evaluation system aimed at ensuring that the money invested produce value and scientific results. The recent publication "Project selection and relevance of scientific results" analyzes the results obtained in order to verify, ex-post, that the investments have been scientifically productive.

The selection process in Cariplo Foundation is carried out according to merit-based methods which have been refined over the years also taking inspiration from other national and international situations and includes a call for proposals, an initial assessment by the Scientific Office which assesses eligibility and compliance with the call specifications and a second step carried out by external reviewers, the actual peer-review phase.

In the 2010 call, for example, about 200 foreign and Italian reviewers were involved, holding permanent working positions for over 10 years abroad in order to prevent any possible conflicts of interest. Each project was assigned to three independent reviewers who worked through remote connection and who provided their written evaluations on the basis of specific criteria having different weights in the composition of the final score. Indeed the Cariplo Foundation has set up a special system to develop a final score which is the weighted average of the votes of the three reviewers based on their self-declared specific knowledge of the area of ​​the project. Each project is ranked on the basis of the final score and the Board of Directors of the Cariplo Foundation disburses the funds allocated for the call until fully used up. The system to arrive at the final score is complex, and is probably meant to compensate for the absence of the plenary meetings or "study sections". These meetings are normally the crucial phase of "peer review" selections, because they allow a limited number of international, competent, impartial and independent auditors to receive the external reviewers' evaluations, to have an overview of all the competing projects and to avoid a fragmented evaluation and the application of different voting scales. The Cariplo Foundation does not use them for various reasons, including cost.

The ex-post analysis of funded projects is complex and detailed, and the part relating to scientific publications refers to 120 projects approved from 2001 to 2007, amounting to approximately € 19.5 million in grants. Out of these 120 projects, 545 works were published and indexed in Web of Science (self-declared by the project managers, therefore potentially subject to an opportunistic increase), and, according to Cariplo Foundation, these documents have had an average of 23 citations per article in the analyzed period, and especially the ratio of actual citations to expected citations has always been greater than 1, whether data are analyzed by magazine or by sector, which proves the good quality of the scientific publications.The most interesting data, however, come from the national and international scientific community. The publications resulting from the funded projects in 10 of the most important areas of biomedical research have received a number of citations more than double the Italian publications, exactly twice the publications of the OECD countries and equal to 1.8 times the publications of the 8 most industrialized countries.

The figures of both the projects and the analyzed publications are small but the trend is positive. The peer review merit-based system designed to exclude conflicts of interest, biases, and self-referencing, is slowly catching on here too, at least in the private sector.

To download the Cariplo Foundation Notebook 6 dedicated to scientific research click here.


Scienza in rete è un giornale senza pubblicità e aperto a tutti per garantire l’indipendenza dell’informazione e il diritto universale alla cittadinanza scientifica. Contribuisci a dar voce alla ricerca sostenendo Scienza in rete. In questo modo, potrai entrare a far parte della nostra comunità e condividere il nostro percorso. Clicca sul pulsante e scegli liberamente quanto donare! Anche una piccola somma è importante. Se vuoi fare una donazione ricorrente, ci consenti di programmare meglio il nostro lavoro e resti comunque libero di interromperla quando credi.


prossimo articolo

Why have neural networks won the Nobel Prizes in Physics and Chemistry?

di Chiara Sabelli
Pubblicato il 15/10/2024

This year, Artificial Intelligence played a leading role in the Nobel Prizes for Physics and Chemistry. More specifically, it would be better to say machine learning and neural networks, thanks to whose development we now have systems ranging from image recognition to generative AI like Chat-GPT. In this article, Chiara Sabelli tells the story of the research that led physicist and biologist John J. Hopfield and computer scientist and neuroscientist Geoffrey Hinton to lay the foundations of current machine learning.

Image modified from the article "Biohybrid and Bioinspired Magnetic Microswimmers" https://onlinelibrary.wiley.com/doi/epdf/10.1002/smll.201704374

Physics

The 2024 Nobel Prize in Physics was awarded to John J. Hopfield, an American physicist and biologist from Princeton University, and to Geoffrey Hinton, a British computer scientist and neuroscientist from the University of Toronto, for utilizing tools from statistical physics in the development of methods underlying today's powerful machine learning technologies.