Funding for Research ? Innovative Approaches to Assessment and Selection: Workshop Report

"Lottery, artificial intelligence, or contextual mechanism? Innovative approaches for the selection of grant applications by science funding foundations were the main theme of the international Philea Research Forum workshop held at the headquarters of the Foundation for Polish Science (FNP) on March 9, 2023. Over the last 25 years, the role of foundations as supporters of research and innovation in Europe has grown significantly both in terms of scope and scale. A report by the European Commission states that the R&I expenditure of the 991 foundations who participated in the study was of EUR 5.01 billion a year, about half the average annual budget that the EU gave to researchers and innovators throughout the whole duration of the Horizon 2020 programme (EUFORI Study, 2015). In workshops and conferences, the largest European foundations supporting scientific research gathered in Philea shared their experiences in improving methods of selecting the right projects. This time the meeting happened in Warsaw at the FNP headquarters, which is a member of the Philea Research Forum. Dr. Elizabeth Gadd, Vice Chair of the Steering Committee of the Coalition for Advancing Research Assessment (CoARA), drew attention to the discrimination resulting from the current evaluation system, which is based primarily on quantitative indicators while also applying to research conducted in less developed regions of the world and published in languages other than English. She stressed that responsible assessment of research should be largely qualitative and include values crucial to the development of science rather than replicating existing schemes. Tanita Casci, Director of the Research Strategy & Policy Unit at the University of Oxford, presented a contextualized approach to assessing research achievements and narrative résumés. Written in a narrative format, résumés help include the diverse contributions of scientists to their field, other researchers, and science itself, including their impact on the socioeconomic environment. Without taking this diversity into account, the priorities of individual researchers will not effectively serve the development of science. Ulrike Bischler of the Volkswagen Foundation talked about randomization experiments in the selection of funding applications. After selecting the undoubtedly best applications and rejecting the weakest ones that should not receive funding, most research funding institutions face the challenge of selecting the winners from a pool of comparable good-quality applications. The Volkswagen Foundation allows some programs to make a random decision which applications in this group will receive funding. According to the Foundation?s analysis, compared to the traditional method of selecting applications, the results of ?randomization? have performed quite similarly. However, the approach simultaneously increased the number of studies conducted by women, early career scientists or those with less research experience, including members of fields that are typically underrepresented in the pool of submitted applications. The Spanish La Caixa Foundation receives more than 3000 applications a year, from which it selects some 140 projects for a total of EUR 95 million. As an experiment, the Foundation has decided to entrust part of the selection of applications to artificial intelligence. Rich data from previous years of running competitions was used to teach the AI models, one of which assigns applications to appropriate reviewers, while the other one pre-selects applications for further evaluation. The results of the AI work were compared with the results of parallel human work, and they turned out to coincide almost completely, which strongly encourages further development of the method.

• Learn more about the Philea Research Forum