Telethon Scientific and Medical Committee assigned few days ago the new grants for 2013. Projects selection is extremely rigorous and articulate: Telethon, in fact, follows the same process of the US National Institute of Health, which only honours merit and is considered the best in the world. Scientific publications from Telethon Research, on average, obtain 60% more citations than the US average (based on Thomson Reuters citation-index calculations). In 22 years of activity, Telethon Foundation sponsored about 2400 projects and allocated 371 million of euro. This year the Foundation received 260 applications and only 160 reached the final stage of the evaluation process. At the end, only 14.6% obtained the grants.
Table 1. Sponsored projects per Italian Regions
The Scientific Office plays a fundamental role in the whole evaluation process: from the public announcement design (the “call for proposal”) to grants’ allocation. Projects must be firstly adequate from an administrative point of view, than the Research Program Managers of the Scientific Office can examine the proposals. “These ex-researchers have a huge responsibility, they manage the whole evaluation process” explains Lucia Monaco, Scientific Director of Telethon, “protecting both the evaluator and the evaluated with their work”.
Telethon Foundation sponsors research on rare genetic diseases. Priority is given to neuromuscular and monogenetic (caused by the alteration of a single gene) diseases. Research on multifactorial diseases – i.e. where environmental factors add up to genetic defects, as for diabetes and autoimmune diseases – receive grants only when the hereditary nature of the disease is certain. However, distinguishing is not easy since boundaries are often not very clear, as for some typologies of Parkinson disease.
The program managers, then, examine the scientific literature in order to understand if the project is in line with the Foundation mission or not.
Since 1990, when the Foundation was established, Telethon funded research on 445 genetic diseases and this year 9 new diseases (like Cednik and Kabuki syndrome) will receive their first grants.
Eligible projects must undergo a first evaluation, called triage, by three members of the Scientific and Medical Committee - which is composed of 30 internationally known scientists coming from 9 countries (only one is Italian).
Projects are then examined by two external referees: Italian or foreign scientists who work abroad, have no conflict of interests and are able to provide a competent evaluation of the project’s specific theme. The Research Program Managers select the referees from a database of more than 7,000 experts and for the year 2013, Telethon has engaged 324 referees with 24 different nationalities. “Referees only express a recommendation at the end of their work, not a judgement” explains Monaco. At this point, the three commissioners write a report and grade the project. The day before the plenary session, projects are ranked and only the high-ranked will be individually discussed. During the meeting, the primary referee highlights weaknesses and strengths of the project, besides illustrating and explaining the evaluation work carried out by the other referees. All the other members of the Committee will then express their opinions and open the discussion about the project.
Beside scientific quality, which is the predominant evaluation parameter, the Committee also values the project’s capability of providing useful information for elaborating or improving a therapy.
“The plenary session is an investment, not a cost” asserts Lucia Monaco. “The discussion is a fundamental step of the process: other colleagues’ opinions allow reducing mistakes, oversights or possible bias from the primary referees. Rankings before and after the plenary session are often different, which means that the discussion identifies new aspects and cancels conflicts of interest”.
Once the final ranking has been defined, the board of governors allocates the grants for the most deserving ones. “Our model does not consider interviews to researchers. A well-written project does not need further explanations and a skilled scientist should be able to present a competitive project. Besides, interviews would extend even more revision times and costs” explains the Scientific Director of Telethon.
Feedbacks are another relevant aspect of the evaluation process “made in Telethon”. The Scientific Office draws a report of the plenary session, particularly important for the researchers who did not get a grant, since the Committee’s constructive critics usually allow them to improve their projects. In fact, analysing the percentage of successes for Telethon projects between 2005-2010, two categories clearly emerge: renewals and already presented – and reformulated – proposals (25%).
Figure 1. Percentage of successes (2005- 2010)
