“Chapter 2. The Transition to Open Science” in “The University in Transition”
The Transition to Open Science
Introduction
The prologue and the previous chapter mentioned that if the university wants to add more value to society, both regionally and internationally, then it will have to reflect on itself and its role in the public sphere and fundamentally change how it organises its work. To that end, the university must enter into an open relationship and interaction with society and with its various ‘publics’; the ‘stakeholders’ of various current social problems and issues. The university is still too ‘introverted’, allowing too much of academic life to be determined by the classical ideas about science that are still very dominant, and that largely determine the internal criteria for quality and choice of subjects. This is a problem when it comes to optimally focusing our research and education agenda on the big problems society faces.
This chapter discusses the current transition to Open Science[1], which is gaining international support because working according to the Open Science principles is expected to foster these changes in the academic community. This makes the transition to Open Science a key element in thinking about what the university should want to become by 2030 and beyond, and will require us to think about the comprehensive interpretation and implementation of Open Science.[2],[3] It will involve open co-creative interaction with society, Public Engagement, but also sharing publications, research data and other products of the academic community paid for with public funding, as much and as quickly as possible in a responsible manner.[4] That way, results produced anywhere in the world can be used and applied quickly, anywhere and by anyone. The guiding principle is that the university feels responsible and works with and for society.
By working more openly and transparently within the university, we are responding to society’s signals, often heard in the public debate, that science is a ‘black box’ that is completely obscure as to how it arrives at its claims - and often quite firm statements - on current affairs.
‘Isn’t that also just one researcher’s opinion? Isn’t it coloured by their own perspective and interests?’
By making the process of university knowledge production open wherever possible, people outside the academic community can get a better view of the process by which claims are created. That it is not simply the result of one brilliant individual’s work, but rather of robust discussions between international experts about experiments, studies, data, and their interpretations.
Since 2010, the academic community has become more aware that our way of working does not correspond at all to this image of open science. Many consider it as a ‘classical’, almost mythical ideal, but we rarely share our products, our publications are not free to read, and our data are inaccessible behind payment walls. This lack of access is directly related to potential readers’ financial resources, which is a major problem everywhere, not just the Global South. This shows there are serious obstacles in the organisation of the academic community to shaping an open attitude and open relationship with society.
These barriers will be discussed below in the context of the Open Science issues that focus on actions needed to address the barriers. Open Science and Public Engagement are about a genuine relationship between science and society that is mutually beneficial and essential for both parties. In this movement, which will be placed in a brief historical perspective below, an open interaction between society and the university must be maintained through its staff, who are active in research and teaching. This goes beyond science communication; instead, it aims for co-creation to address society’s problems, setting the research agenda and the vision on education, the production of data and results, and the joint testing of new insights in the relevant social context. Finally, but according to insiders most critically for the success of the transition to Open Science, crucial is modernising our way of recognising and rewarding university staff. This involves the assessment of research and education, but also of academics and the many other members of staff who work at the university. We deliberately chose the word ‘modernise’, because it is an adaptation to the demands of our modern age, and that applies to the entire transition to Open Science and the effects it has on the university. The new ways of working, and the diversity of their results, require different actions and a fundamental cultural shift that the current evaluation system neither facilitates or encourages. As such, adapting our mode of evaluation to this new way in which university staff actually works, is another prerequisite for the transition to Open Science.[5]
The Open Science way of working (‘the practices’) will improve the quality and impact of research and education, and have an essential and decisive effect in shaping modern society. In the process, the complexity of global society and the urgency of the problems we face every day imposes itself on us. These problems call for solutions involving new technology and innovations, but also for new insights regarding the possible and proper interventions to help shape society. This all will require an understanding of historically determined cultural and socio-economic differences and geopolitical situations. That is precisely where the multidisciplinary approach of team science is essential, with researchers from the more exact disciplines and the humanities and social sciences truly working in teams together. In the spirit of Open Science, these teams will also increasingly include stakeholders from society, to generate optimal results and policy options. In a nutshell, this is ‘The Promise of Open Science’ as expressed by the EU, but recently also beautifully formulated by UNESCO.[6]
Open Science, as a major international movement to modernise science and academia, is still very young. It was only in 2015 that the European Union brought a number of projects under the umbrella of Open Science with the title ‘Open Innovation, Open Science, Open to the World’, first in a speech by Carlos Moedas and later in a book in May 2016.[7] As we will discuss below, this helped promote the themes of Open Access, FAIR/Open data and Citizen Science/Public Engagement. These initiatives were already operational, with the goal of sharing knowledge and research results with society. The issue of Recognition and Rewards was added with a high priority in the spring of 2016, in part as a response to the ‘Amsterdam Call for Action on Open Science’.[8]
Separate initiatives for Open Access, FAIR/Open data and Citizen Science/Public Engagement were already underway in the EU and elsewhere in 2016, but Recognition and Rewards were not yet a major international concern. Back then, work was being conducted in a number of countries, mostly at a local level, to modernise evaluation systems for research, universities and researchers. In the EU, a number of working groups were energetically formed as early as 2016, to advise on both the indicator to be used and on how the evaluation of university staff should be adjusted.
The EU Open Science Policy Platform (EUOSPP) combined and drove these various activities. In 2017, a project was launched to explore the implementation of Open Science in the different member states. It immediately became clear to the participants that this should involve very different trajectories for the member states, in line with their local, socio-cultural, legal, and political situations.[9]
The themes of Open Science
This chapter offers a brief summary of Open Science, based on its four major themes. Although Open Science in education has only recently received explicit attention[10], education is an integral element in this context and, as we will discuss elsewhere in this book, education is obviously a major issue for the university of 2030 and beyond. Open Education is already a theme in UU’s Open Science Programme, see Figure 2.1, as well as in the Dutch National Open Science Programme.[11]
Open Access and FAIR Open Data
Open Access and FAIR Open Data and Software are international movements that arose from, and were facilitated by, the digital revolution that has transformed society since 2000, as well as the university’s practice of education and research. The higher goal of these two
Utrecht University Open Science Programme 2023
movements is to make the knowledge produced (articles and all other forms of data or products of research and education) available to a very broad group of stakeholders. Available, not just to our colleagues, but also to other disciplines, institutions, and businesses, anywhere in the world. The idea is that knowledge produced with public funds is a ‘common good’, which should be made available to and for society as soon as possible.[12] This leads to complex issues that affect the role of the commercial publishers who publish academic journals and books. Those articles are only accessible to readers with paid subscriptions, (often through university libraries).
The transition to Open Access has gained momentum over the past few years. In the early days, starting from 2002, it was mainly limited to ‘position papers’ and ‘declarations’ and local efforts spread across Europe, North- and South America, Australia, and Africa. But since 2015 there have been a number of broad institutional initiatives. The most prominent example is PlanS by CoalitionS, an international coalition of public and non-profit organisations that subsidise research and require results to be published immediately via Open Access.[13] CoalitionS was founded in Europe in 2018 by Science Europe, the EU and the Wellcome Trust, but has since gained members from other continents, such as the Bill and Melinda Gates Foundation and the Howard Hughes Medical Institute. Very recently, the US government has also introduced the requirement that articles resulting from government-funded research must immediately become available via Open Access.[14]
These actions are motivated by the fact that knowledge, in the form of articles and books, is not freely available through commercial publishers, as well as by the exorbitant price increases of subscriptions to electronic journals, and the associated commercial interests. As a result, articles and research data have become even less accessible to researchers and audience(s) with fewer financial resources. They simply cannot afford the subscriptions.
PlanS initially focused on publication in Open Access journals by paying an ‘Article Processing Charge’ (APC). That means authors and their institutes pay for the publisher to publish their articles Open Access. This makes articles accessible to readers, free of charge. Publication via so-called ‘repositories’, has also been encouraged. This has had a strong impact in advancing the debate on Open Access and Open Data. But it is not a long-term solution. APC payments, like subscription fees for readers, are, after all, just as much of a barrier, only now for authors and their institutions instead. The fees for APC have risen significantly over the past few years, and even exorbitantly for the most ‘high impact’ journals. This, for instance, concerns journals with a good reputation, because they have a higher ‘Journal Impact Factor’ (JIF)[15]. Journals like Nature, Science, Lancet and Cell. These are examples of journals that have high subscription costs, but allow authors to publish their articles Open Access - for a fee. Authors are charged the APC, which is a form of ‘hybrid Open Access’, also known as ‘double dipping’. For publishers, it has the benefit of duplicate revenue streams: subscription fees and APC. It is lucrative for publishers speculating on the ‘addiction’ to high JIFs, especially among researchers and members of review committees from the exact and biomedical disciplines. We will deal with this in more detail below in the context of Recognition and Rewards.
Because we face a gap between rich countries in the Northwest and poor countries in ‘the Global South’ and Eastern Europe, the latter face a disadvantage in science and the transition to Open Science. The wealthier researchers who feel the need to publish in their ‘favourite journal’ can afford even the most expensive APCs. Researchers from countries in South America or Africa who want to publish in those same journals can rarely afford to do so. It is also good to realise that, even in the prosperous West, financial opportunities between academic disciplines are historically very unequal. Simply compare funding for biomedical and natural sciences to that of the humanities.
So, in 2022, CoalitionS called to move beyond the model of APCs in the transition to Open Access, and to finally switch to non-commercial public Open Access publishing platforms.[16],[17] Those publication channels should then be funded from centralised public resources, with Open Access being the standard and with quality control via peer review.[18] Our colleagues from South America constantly remind us that this has long been the dominant model with them, until it was threatened by PlanS’ promotion of APCs.[19]
It should be noted that the team of researchers bear most of the responsibility for the quality of their work, but some responsibility also lies with the institute. An institute’s reputation should not be based on the JIF, which does not correlate with actual impact and quality, but rather on the intrinsic quality and actual impact of research results, data, claims and other research products published from within the institute, on a ‘Diamond’ platform or a ‘prepublication repository’[20], and in books and journals. Merely passing peer review, it has been widely shown, is not an entirely reliable seal of quality.
Inset 2.1. The forms of Open Access publishing.
There are different forms of Open Access: Gold Open Access: publication in a fully Open Access journal that does not charge a subscription fee. Usually, gold Open Access journals charge publication fees, also known as Article Processing Charges (APCs).Hybrid Open Access: publication in a ‘traditional’ subscription journal that offers Open Access publication for individual articles on payment of an Article Processing Charge (APC).Green Open Access: publication in a closed journal and subsequent archiving of a version of the publication in a trusted Open Access repository. Publications can be archived upon publication or afterwards, depending on the journal’s policy. Often, self-archiving of the final author’s version (post-print) is permitted after an embargo period.Diamond Open Access: publication in a fully Open Access journal or platform that does not charge publication fees (APCs). Publication and hosting costs are paid by one or more organisations, associations, or networks.
FAIR Open Data and Software is the publishing, and thus providing usable and reliable availability (FAIR), of research data and software and code. Considerable attention is already (and rightly) being paid to this, as it is an important part of what a university should do in promoting Open Science. It encourages interdisciplinary collaboration between colleagues in research and education, and interactions with different stakeholders in global society. Aside from the enormous financial, technical and facility issues and conditions that need to be addressed to make it possible for staff and potential users, there are also ethical and political considerations. Ethically opening up datasets may be in conflict with legitimate legal restrictions and privacy objections, or possible misuse for military purposes. The motto is therefore: ‘As open as possible, as closed as necessary’.
As with Open Access, there are legitimate concerns about the unequal balance of power at national and global levels.[21] We saw during the COVID-19 pandemic what a massive impact data sharing can have at the international level, when immediate sharing of genetic information about the virus and publishing the raw data on which publications were based were briefly the standard practice. There are also many great examples in education, the computer sciences, cybersecurity and energy and climate research. But there are also some concerns. Sharing data and code will favour researchers and the public from rich countries over residents of poorer countries, because they have more opportunity to exploit that information, as there is more funding available in rich countries for reuse or commercial and other profitable applications. These are not new problems suddenly rearing their heads in the university; they are problems that the academic community has struggled with before. Consider, for example, the years 1939 to 1945, around the development of the atomic bomb, and after that in the relationship with the Soviet Union during the Cold War. For most of us, especially since 1989, that is a distant past. But studying the specific history of the interactions between science and society is very illuminating today, and provides insights for current geopolitical discussions. During the Cold War, for example, the results of certain scientific research were deliberately not disseminated to other parties. This structural political and socio-economic inequality is of such a fundamental nature that we need to anticipate and mitigate it as much as possible in the context of Open Data and Code in Open Science.
Another risk involves naively sharing data with parties who could misuse it. This could include using data or code for politically and ethically undesirable practices. This cannot always be prevented, but the researcher who produced the data or code should always be very alert to the possibility. They should also be aware of the debate about sharing data and code with parties in countries with authoritarian governments that could use the data and code for military or oppressive purposes. Geopolitics also plays a role here. And here, too, the motto is: ‘As open as possible, as closed as necessary’. We should note that in the world of today, there is not just one science, with one set of norms and values. Open Science depends on an open society, with an open democracy, and it does not function, or at least not so well, in countries without an open society and open democracy.
Open Innovation
When sharing data in an ‘international community of inquirers’, agreements must be made on how the parties involved can benefit. There are examples in the field of drug development, where universities, having developed knowledge and patents, and pharmaceutical companies wanting to bring medicines to market, timely made agreements regarding how, where, and at what price those medicines would become available. In the process, the researchers were able to negotiate lower prices for low and middle-income countries. This differs from the neoliberal free-market economy we find ourselves beset with in the Western world, so it is a difficult, but important new path to take.[22] There are also many initiatives around the world that encourage Open Innovation between companies and academic partners. Novo Nordisk Foundation is one example.[23]
The underlying, structural, social inequalities are linked to geopolitical, national, and social structures that make many of these things difficult or impossible and are not in the academic community’s power. But the university must always take them into consideration and make them a priority on the international agenda.
Public Engagement: Science with and for Society
Public Engagement takes many forms and has a wide variety of outcomes. The classic example is so-called ‘Citizen Science’, and involves citizens and non-academic researchers participating in scientific research, for example by making observations and collecting data. Public engagement, however, goes much further. It involves research based on a question formulated by researchers and citizens working together, and translates into a research project that is carried out by both. This usually involves testing the new knowledge and/or the newly developed product in the context in which the citizens’ question originated. Public participation can also lead to the development of new teaching methods and resources that address social problems. Wonderful examples of this participatory science have been described in every academic discipline, from medicine, psychiatry, pedagogy, international law, local political problems related to human-caused damage to environment, and habitat and welfare, to equal treatment (inclusiveness) in all kinds of social situations, such as education.
The main point is that the researchers involved are convinced that this co-creation can lead to useful results that will actually reach potential users. This is supported by a wealth of empirical research. If a researcher not only aims to apply potential results in society, but also to co-create from the very early stages of articulating the question, it can reduce the distance between the researcher and the potential user, maximising the chance of making an impact. ‘Distance’ here can mean literal, physical distance, as in physically working together, but also mental distance, in the sense of understanding one another.
Inset 2.2. CHARM-EU: an example of Public Engagement.[24]
One example of public engagement in which social stakeholders formulate a research question that researchers, social actors and students address together takes place within the CHARM-EU Master’s programme ‘Global Challenges for Sustainability‘. In this first-ever European joint degree Master’s programme, students simultaneously pursue hybrid education linked to challenge-oriented, transdisciplinary research at five universities. During the Capstone, the final phase of the Master’s programme coordinated by Utrecht University, social stakeholders (companies, NGOs, government agencies, etc.) formulate a sustainability challenge for which they seek a solution. Teams of students from the five CHARM-EU partner universities work under the guidance of researchers and civil society actors to analyse the problem and come up with solutions, recommendations and/or prototypes. The sustainability challenges vary from year to year and involve different disciplines and geographical areas within and outside Europe. Some challenges address global issues, such as how the United Nations can encourage companies to implement and monitor the Sustainable Development Goals. Other challenges focus on local issues. One of the Capstone teams designed a smartphone app as a playful and educational way to stimulate Utrecht residents to explore the region’s urban farms and gardens. In South Africa, Capstone teams are analysing ways to manage conflicts between people, livestock, and wildlife around Kruger National Park. The team starts by formulating a problem statement based on discussions with local community leaders and other stakeholders in the park. This requires flexibility, interdisciplinarity and cultural sensitivity to analyse and address problems from multiple perspectives in ways that locals can accept. They then integrate scientific and indigenous knowledge and share it through a local transdisciplinary research centre; a satellite campus of the University of Pretoria. The centre conducts research and training in the fields of livestock disease control, conservation, and sustainable living in the park. Through the project, the students conduct internationally recognised research, as well as building local capacity and designing innovative solutions for local problems that also affect neighbouring countries. The teaching methods in the Capstone were developed under the leadership of Utrecht University by a transdisciplinary team of researchers and teachers from the five partner universities, in collaboration with civil society actors. it creates a direct link between research and education pertaining to societal issues, and results in joint learning processes by all stakeholders. CHARM-EU is a European university alliance that serves as a testbed for innovations and institutional changes to support Open Science and Public Engagement among the nine CHARM-EU partner institutions.
There is currently a host of Open Science Declarations, recommendations, implementation plans, and strategies issued by the EU, the UN and UNESCO, governments and institutions around the world. In the spirit of Open Science, most researchers, and most universities nowadays, believe that a big part of our research should contribute to helping solve society’s problems.[25] A lot of research is being conducted to help people make the right decisions, for example in a political debate. That means universities and their research and teaching staff are making conscious choices for topics and problems that demand research and new forms of education. As part of this transition, universities will increasingly move towards thematic profiling of their research through a substantive and organisational process.
University Profile and Strategy
If they want to achieve their goals, universities will have to decide on the major themes and topics that can add maximum value and have optimal impact in the academic community and in society at large, based on their individual strengths. By applying this strategy, the quality and excellence of university research will be judged not by the number of publications in Science and Nature, but rather by the actual value and impact on society and the academic community. That will bring universities and researchers very close to the public, private and political arenas, and will pose urgent questions about universities’ relationship with society. These questions touch on classical academic issues: values-free science, neutrality, but also on our responsibility towards society. Which universal, and possibly also more local, values and norms should the university propagate?
The researcher will encounter people with different perspectives, fears, experiences, desires, political beliefs and underlying patterns of norms and values that strongly influences the citizens’ interpretations of scientific work. Consider the approach to the COVID-19 pandemic, or the nitrogen crisis in the Netherlands. Some citizens will not feel supported by the information academics bring to the table. Many will point to those researchers presenting different claims and facts than the consensus, and to the different contexts in which the research was conducted. The differences between the research context and the actual environment where the results are thought to apply are extremely important. They can be used to call into question the evidence that supports scientific claims. This form of argument is similar to how lawyers break down evidence in court cases into preconceived ideas, context-dependent assumptions, interpretations and methods. That makes these discussions substantially different from those the researcher has with peers about their work, where cognitive (‘scientific’) arguments prevail and are often kept ‘neatly’ separate from judgements about the work based on social, socio-economic, political and cultural considerations.[26],[27] Neatly separated in the sense that in academic discussions, both sides try to separate the cognitive from non-cognitive norms and values, where the latter category can often be left out of the picture.
In the era of ‘positivism’, the dominant classical image of science as practiced from 1930 to 1970, the boundaries between the worlds of citizens and science were often kept strictly separate. External influence on science, of any kind, was taboo. But science and society were both constantly developing in parallel and at a rapid pace, and that has continued into the present day. These developments involved a constant interaction and mutual influence between science and society, with both reacting to and anticipating each other’s ‘behaviour’, ideas, communication, and actions in the public sphere. This often affects the progress of scientific research, either directly or indirectly. Recent insights in philosophy of science show that researchers always operate from the norms and values of their socio-cultural context in their research and academic debates. In fact, these can be very important in research, and should always be made explicit and brought up for discussion.[28]
Reflexivity
Truly understanding and empathising with the problems faced by a citizen, patient, or child from a migrant background, teacher, CEO, or public servant, and what the problems mean for the researcher’s position, attitude and engagement, is the most important factor in the quality of participatory research and Public Engagement. To a large extent, this involves the university and researcher reflecting upon their own position in the relevant social context of research, and on their own experiences, expectations, norms and values.
Researchers are faced with dissent as ‘society speaks back’, and there is a high degree of uncertainty about the status of our scientific claims. Classical authority has disappeared. As has been extensively discussed elsewhere, that authority was based on an idealised image of science (‘The Legend’) that has now been replaced by a realistic narrative of how we arrive at knowledge claims in science and how they are always being tested, rejected or improved through constant discussion and research.[29] Interactions should not be about appealing to absolute knowledge or facts, but rather to demonstrably ‘robust and reliable’ knowledge that has proven its worth in continuous debate with colleagues and through actions and interventions in the lab, a testing environment, a clinical trial, or even the real world. Speaking of the latter, it is very important for the researcher to be aware that knowledge and knowledge claims have always been produced and validated in a particular environment, or ‘setting’, and social and cultural context. The field of medicine, for example, has a long and disappointing history of this with regard to medication efficacy. New medicines have often been tested and validated in trials that generally involved patients strictly selected for age, gender, ethnicity, and certain disease characteristics. As a result, they were not always representative for women or people from other continents.[30] There are many examples of this in the development of antidepressants, but also medicines for oncology, cardiology, and patients with immune disorders. Another great example in medicine is the completely different way in which HIV spread in Sub-Saharan Africa compared to Amsterdam and San Francisco. In Sub-Saharan Africa, the virus mainly spread through heterosexual contacts, rather than homosexual contacts. People here initially approached the problem from our Western frame of reference, with its dominance of sequential heterosexual relationships and the probability of spread positively correlated to the frequency and nature of heterosexual contacts. But that didn’t appear to be a plausible explanation in African countries with massive numbers of HIV infections among heterosexuals. Contacts with anthropologists and sociologists in Africa eventually facilitated the relatively late ‘discovery’ of the cause for this difference. It turned out that these countries in Africa had - and still have - completely different culturally determined heterosexual relationships. These networks of multiple, simultaneous sexual relationships between men and women could explain the rapid spread of infection among both men and women through heterosexual contacts. It also explained the much greater spread of HIV among young and new-born children.[31]
Recognition and Rewards
With the institutional transition to Open Science described above, it has quickly become clear in the EU and elsewhere that a successful and lasting transition and implementation will require a new way of recognising and rewarding researchers and their research. The system that Robert Merton, as early as the middle of the last century, described as a ‘reward system’, desperately needs to be updated.[32] If we visualise ‘the credibility cycle’ as shown in Figure 2.2, we can see at a glance how it works out in day-to-day practice. At several points in the cycle, assessment criteria and the indicators derived from them largely determine our evaluation of quality and are therefore decisive for awarding grants and awards.[33] These indicators play a major role in recruiting academics, deciding on academic appointments, permanent contracts and promotion to the positions of assistant, associate or full professor.[34] As Figure 2.2 shows, the system of Recognition and Rewards can be considered as the ‘business model’ of modern science. Concepts such as quality and excellence are defined using the indicators and criteria used explicitly (and sometimes implicitly) around the world, and there is a hierarchy in the academic disciplines that is all too familiar in the academic community. Here lies a real challenge for the university over the coming years, especially with regard to the enormous and growing hyper-competition for academic positions, for the allocation of grants and the associated application pressure, and of course the workload and burden placed on applicants.
Many authors have shown how the system of Recognition and Rewards has developed rapidly since the 1990s.[35] Other works have dealt with the classical perspectives on science upon which that evaluation system was built, which Ziman and Kitcher have called ‘The Legend’. Figure 2.3 illustrates why the dominant evaluation system will not facilitate, let alone promote, the transition to Open Science.
The ‘Credibility Cycle’ Source: ‘Figure 3’ from Hessels, 2009.
Despite this, since 1995 it has become common practice to use the Journal Impact Factor to determine the quality of individual articles and the quality of their authors’ work. Editors and associate editors steer towards acquiring articles that are sure to attract attention, for example because the research is very fundamental, very current, new, and ground-breaking. These articles simply attract the most publicity, both within and outside the scientific community. And this is where the classical views and ideologies of science play a decisive role. Through publicity and citations, editors hope to obtain an even higher JIF for their journal. This is connected to the journal’s reputation, but also the ‘subscription and APC fee’. This fee for Nature, for example, has already exceeded 10,000 euros. Voilà: the revenue model of commercial publishers.
The JIF and the journal’s reputation play a defining role, despite its obviously faulty basis, especially in the exact disciplines and the vast field of biomedical sciences. But this phenomenon has since been replicated in economics, and even the social sciences.
The ‘Legend’ is perhaps best characterised as follows:
‘“There is a unique ‘scientific method’ that guarantees the objective truth of general, universal, and timeless theories and claims. These claims allow for understanding, prediction, and control of our world (nature/men). The method is logical-empirical and has a firm foundation. Facts and values, science and non-science, are neatly separated, which makes science objective and neutral. This method explains the success of the ‘hard’ sciences; the ‘soft’ social sciences and humanities are methodologically problematic.”[36]
Metrics shape Science
What does it imply that ‘metrics’ are the determining factor? To understand this, we should consider everything that happens within the ‘credit cycle’: getting articles accepted in a prestigious journal via peer review, being rated as ‘excellent’ based on that achievement, acquiring research grants and academic positions based on that rating, and then easier access to years of nice and hefty (and prestigious) grants via the ‘Matthew Effect’. There has been widespread international criticism of the use of these metrics for at least a decade. Experts, by the way, had been expressing criticism for much longer. The Declaration on Research Assessment (DORA) is an international movement, started in 2012 and signed by many institutes and individuals, calling for a ban on the use of the JIF in this way. Its implementation by institutes has only started taking off in earnest since 2015 and is now part of the Recognition and Rewards project in many institutes.[37]
DORA focuses on this manifestly faulty use of ‘poor proxies for quality and excellence’, and on broadening the criteria for assessment and evaluation. But it is not only a problem for the academic community. We have to realise that a lot of research of very high quality and high scientific and/or societal impact is not of interest to journals with that high JIF, and that the JIF is decisive even in committees that have to assess very different research proposals for grant providers. This means the choice of research topics in some major disciplines and domains is not determined by actual quality and impact.[38],[39]
Problems of the current dominant reward system in science. (Miedema, 2022).
In the field of biomedical and health research, in 2016, the Health Council of the Netherlands issued an advisory report under the ominous title ‘Onderzoek waar je beter van wordt’ (Research That Makes You Better). This report, requested by the Minister of Health, Welfare and Sport, was prompted by the aforementioned publications claiming that the research agenda was too determined by metrics, rather than by social and clinical impact. The advisory report therefore called for a change in the assessment of research and researchers based on more inclusive criteria, such as also the social relevance of research.[40] This is why Recognition and Rewards was made an integral part of the Open Science programme in the EU and in many institutions in 2016.
Despite the fact that the biases of ‘The Legend’ have been demonstrated by philosophers and sociologists for decades, they are unfortunately still very prevalent in discussions about science in the academic community, government, grant makers, academic institutions (including the Royal Academies) and in the media. The classical perspective on science holds the societal impact and ‘societal need’ of applied and technological sciences in lower esteem than ‘pure’ fundamental science, ‘blue skies science’ or ‘curiosity-driven science’. The qualification ‘curiosity-driven’ is suggestive and incorrect, because all research involves curiosity, to solve problems of any kind. In this classical dichotomy, formal quantitative ‘hard’ science is dominant over ‘soft’ qualitative science, as in the humanities and social sciences. This is despite the proven value offered by both in addressing major social problems, such as the climate crisis, growing inequality, and the COVID-19 pandemic.
In the Netherlands, much progress has been made in recent years by the UNL (formerly VSNU), NWO, NFU and KNAW, as expressed in a quote from the position paper, ‘Ruimte voor ieders talent’ (Room For Everyone’s Talent).[41]
“This requires a system of recognising and rewarding scientists and research that:1. enables diversification of more dynamic career paths to promote excellence in each of the core domains;2. does justice both to academics’ independence and individual qualities and ambitions, as well as team performance; 3. emphasises the quality of the work and places less emphasis on quantitative results (such as number of publications); 4. Promotes all aspects of Open Science; and 5. fosters high-quality academic leadership.Open ScienceWe must pay specific attention to creating more space for Open Science. This new approach to science gives both the individual academic and other stakeholders the opportunity to collaborate and contribute to, and to utilise, the scientific process. This means that academics should share the results of scientific research more widely with society, make research results accessible, and involve society in the research (for example through citizen science). Open Science is inextricably linked with the modernisation of the system of recognition and rewards. It requires time and effort from researchers that cannot automatically be devoted to traditional scientific outputs such as publications, but which can have a major impact on society and science (for example by sharing research data).”
At Utrecht University, we have been working on a comprehensive, inclusive assessment of our academic staff within the framework of our Open Science programme, entirely in line with DORA and the aforementioned projects and their underlying visions. We’ve drawn up an outline for the implementation of such an assessment protocol, which will have to be elaborated by the various faculties and departments and Strategic Themes. This is where the real work begins: what do impact and quality mean in these very different contexts? How do you come up with good narratives and criteria?
Discussion diagram ‘Recognition and Rewards’, Utrecht University 2021
In addition to research in the narrower sense, we now have a strong focus on education as well, but also on academic activities related to ‘academic duties’; efforts that promote the quality of research and education practice. See Figure 2.4 for a discussion diagram developed in 2021 to inform discussions about the topic at Utrecht University. This includes peer review, committee work with grant makers, but also time spent promoting and doing FAIR/Open Data, Open Access and Public Engagement and the corresponding academic leadership. These are activities that should have an important place in a modern evaluation of research, education, and our staff, precisely because of their versatility and plurality. In other countries, these movements, are already visible to a greater or lesser degree in national protocols for evaluating research and academics, and many transitions are already underway.
Inset 2.2. The Research Excellence Framework.
In the UK, the Research Excellence Framework (REF) (2014) was a response to the Research Assessment Exercise (1986-2008), which steered very much towards the classic academic output of ‘science for science’s sake’ and paid little attention to the applied and ‘soft’ sciences. The REF gave explicit attention and weight to societal impact in the assessment of university research. Since UKRI distributes research funds to institutions based on of REF scores, this has immediate financial implications as well. The classic ‘OxBridge’ universities and their epigones no longer received the highest scores and largest amounts more or less automatically as before. The newer, more regionally focused universities gained a more favourable position in the REF.[42]
The Strategy Evaluation Protocol (SEP)
In the Netherlands, the UNL, KNAW and NWO adopted the SEP 2021-2027 in the year 2020, and it has been in effect since 2021. The SEP is a new protocol for the national assessment of research in universities every six years. It focuses heavily on narrative, on substantive and strategic evaluation by experts and peers, with less emphasis on quantitative indicators. The use of JIF is not allowed, and information about the H-index, numbers of publications, or grants acquired is used only to support the narratives. It places much greater emphasis on leadership, academic culture, and talent management.[43]
The SEP 2021-2027
The research unit’s self evaluation:• Vision, strategy and aims of the research are outlined• Results: Narratives (supported by data)*• Free choice of indicators for impact Evaluation is in relation to the unit’s strategy Three criteria: Research Quality, Societal Impact and Viability Four Aspects:1. Open Science practices and efforts2. PhD policy and Training3. Academic Culture (Openess, Safety, Inclusiveness, Research Integrity)4. Human Resources Policy (Diversity, Talent Management*Compatible with DORA
In the EU and in several countries outside the EU, universities are now paying more attention to modernising the assessment of research, education, and university staff. Under the leadership of the European Association of Universities (EUA), Science Europe[44] and the European Commission’s Directorate-General for Research and Innovation, a ‘Coalition for the Advancement of Research Assessment’ (COARA) was formed in January 2022, which drafted an agreement for implementation of research evaluation over the next few years.[45] The coalition entered its next phase in autumn 2022, agreeing on an organisational structure involving more than 350 universities, university umbrella organisations (UNL, EUA, LERU, COIMBRA), public funding bodies (Science Europe) and young academics from more than 40 countries. All universities in the Netherlands and the umbrella organisations UNL and NWO have signed the Strategic Evaluation Protocol.[46],[47]
Fundamental research in the social context
The transition to a system of Recognition and Rewards that facilitates and rewards the necessary changes in science outlined above is essential for society, governments, the public and universities to ensure the universities’ continued impact in the twenty-first century. This new Recognition and Rewards system will have to reflect the desired plurality of research and researchers in the academic community. Quality and excellence are very much bound by context and determined by the strategy and goals of the institute, department, and team. The new system will also have to reflect the plurality and diversity of society in the university staff, especially with regard to equality, diversity and inclusion.
For many, this will mean a logical and proper adaptation to the demands of the times. Others feel that it goes against their idea of excellence in science and academia.[48] That means the transition touches on very deeply held, and sometimes very different, beliefs of professionals that determine their reputation, prestige, position, and power in academia and beyond. This can feel like a loss, especially for researchers of all ages in the ‘hard’ sciences, because they belong to the disciplines high up in the old hierarchy. Some fear that fundamental science will lose out in this transition, in which the scientific agenda is guided more by questions from society compared to the situation after 1945. The concern is that those questions will mainly deal with short-term problems and applied or technological research, so no truly new knowledge will be produced that could have long-term impact. However, our experience in the patient lobby for research into cardiological diseases, AIDS, cancer, lung diseases, depression, and autism, for example, shows that there is actually widespread support for fundamental research, if the researchers also work on problems that could provide short-term solutions.
Aside from the fact that there should and will always be plenty of room for curiosity-driven science, the history of science and innovation has taught us time and again that research in the context of very concrete societal problems can - and should - take the form of very basic research. So referring to fundamental research as ‘curiosity-driven research’ is misleading, because applied research is also very much driven by the researchers’ curiosity. It is precisely the mix of more fundamental and applied research that can have a great deal of impact when carried out in close proximity to real-world practice. The classic example is Pasteur, who did very fundamental work in response to pressing questions from his time. That’s why this kind of use-inspired fundamental research has been referred to as ‘Pasteur’s Quadrant’.[49]
From a very different perspective, objections to abolishing bibliometrics also come from institutions and countries where metrics are used to counter nepotism and political appointments, for example, and where metrics are actually used as an objective indicator. This is understandable, but given the discussion above, using ‘poor proxies for quality’ is not the right way to counter the practice of problematic appointments.
Over the next few years, The Coalition of The Willing (COARA) and other efforts should pay close attention to national and cultural differences in the design of systems for Recognition and Rewards.[50] Pluralism and differences in pace are essential. It seems like a logical development, and many have argued that such a Coalition of the Willing should be set up for education assessment as well.
Other instruments
Clearly, Open Science forces us to use different tools and different forms of research and education. First, students should be trained not only in the conduct of science, but also in reflecting on the daily practice of science. We foresee that the future university may offer a continuous learning track for ‘reflection and scientific literacy’, aimed at teaching students how to independently reflect on the position of science in society, as part of a core curriculum in every Bachelor’s, Master’s and PhD programme. This curriculum will focus on what science, research and the university should be in the present time and our world. In research and education, we will have to innovate together with many different societal partners to bring society, its questions and experiences inside the university lecture halls and laboratories. Open Science must also be visible and guiding in the nature of our education, goals and formats. For more on this topic, see the chapter on education.
We must also develop tools for knowledge dissemination other than the classical scientific output. This output should be an essential complement to the classical articles, including open access and open data, which are mainly aimed at fellow academics. Assessing educational impact should look beyond student satisfaction; we should include quality, but also the role and position of education and how we can reach different target groups in and through education. We have a wide range of communication forms at our disposal today, via an extensive range of media. Many of these are already being used, such as podcasts, media appearances, written or spoken blogs, video recordings of social interactions and science shops. Theatre and music in various forms can also be used to reach a diversity of ‘audiences’. These - often interactive - forms of communication are perfectly natural for young people, but for colleagues who grew up in the even more classical university from 1980 onwards, these may not seem like ‘academic’ products at first sight. Staff will have to be trained and encouraged to develop these tools and to use them appropriately. As we described above, the university is modernising the evaluation (assessment) of its employees and criteria for recruitment and selection procedures, so that its employees are recognised and rewarded for all these forms of academic output and impact.
Looking outwards
Over the next few years, the university will have to direct its view outwards even more than is already the case. In societies that are highly knowledge-intensive and are becoming more so increasingly, the university and its knowledge production is an essential factor. Such knowledge may be technical or technocratic, but the last few decades have also shown that the social sciences’ and the humanities’ contribution to insights and developing solutions is indispensable in today’s complex, ultra-modern society. Indeed, researchers from very diverse disciplines will increasingly need to work together in research projects in collaboration with societal stakeholders.
Until around 2010, most universities focused on the international playing field, on competing in the global market of higher education, research, and rankings. That has changed for several reasons since then, with universities becoming more involved in regional and national issues. At the national level, in the Netherlands and abroad, we are increasingly seeing regional alliances emerge between universities, university medical centres, and universities of applied sciences. The goal is to seek substantive synergy and complementarity, which is necessary to address complex societal problems in an adequate and socially responsible manner.
The EU and beyond
For universities and other higher education and research institutions in the EU, it is crystal clear the European Union will rapidly and substantially increase in importance for the strategy and positioning of universities. This will arise from a well-considered short- and long-term EU strategy, and will certainly involve the allocation of much of its financial resources to institutions and member states. The EU has for some time been considering its position in the global arena, and paying more attention to developments in the US, China, and Russia - the big players - but also those in countries in ‘the Global South’. The contribution of R&D and education to the EU’s economic, social, cultural, and democratic development are of eminent and decisive importance in that EU strategy.
The EU has a long tradition of directing research, innovation, and education towards excellence, but in doing so has also explicitly towards connections with problems in society. For decades, this involved deploying financial resources via the European Research Council (ERC) and more strategically thematically formulated programmes. HORIZON 2020 and HORIZON EUROPE make this very clear, with mission-driven R&D setting the agenda. The EU continues to take an active and directive stance regarding Open Science as well.
In 2017, they decided to take an important and critical step forward with the European University Initiative.[51] Here they actively seek to establish long-term alliances between at least seven universities and universities of applied sciences from different member states. In the process, a European Campus is being built across geographical, historical, cultural, and socio-economic differences. The ambition is bold, and ranges from European Degrees, increased mobility and attracting top talent, to new European entities of higher education, research, and innovation. Research will focus on innovation in education, and on research around major issues and missions at European level. These alliances, which should grow to include 500 universities and universities of applied sciences over the next few years, are committed to European challenge-based research and education, with a focus on complex issues and challenges in society. The EU intends to allocate funding for research and education along the path of the alliances.[52] Clearly, these steps are a response to geopolitical developments since 2016.
The outward perspective should extend beyond Europe for several good reasons. The UNESCO ‘Recommendations on Open Science’ made that very clear once again.[53] Growing inequality around the world deserves our constant attention. In addition to the necessary political actions to promote universal human rights, open society, and democracy through international relations and cooperation, there are also expectations for the role of science. Many of the problems we face today are not confined to national borders but play out on global or continental levels. And as we discussed above, they are usually problems of such a complex nature that they require teams of experts from various academic disciplines, recruited from the various countries and regions involved. This is the only way for research and education to contribute to finding politically workable and broadly acceptable solutions to problems like the climate crisis or combating extreme inequality.
It is the university’s task to impress upon their staff the necessity of this approach in the ‘EU and Beyond’. This must be embedded in the curriculum, and be applied to the inclusiveness and diversity of staff and the research agenda. So here too, it is important for us to reflect on the university’s identity and position in society; in this case the global society. Advocating and guarding EU norms and values, such as open society and liberal democracy, should guide our research and education choices. This is normative, but based on undisputed human values.
Conclusion
We have discussed a number of current important developments that affect the relationship between the academic community, university education and research, and society at large. The transition to Open Science, which emphasises an open perspective and attitude towards society, is not a non-committal study room exercise. It is clear that the university community must respond to the many strong signals from society, due to the urgency of the major issues we now face. Until fairly recently, we believed that we had things under control, whether it was the economy, infectious diseases or wars. We believed the climate crisis would be solved by technically savvy academics. But that has largely turned out to be an illusion. At the same time, we also know that we desperately need scientific insights, and must combine them to start tackling the big challenges we face. Constant, intense attention from academics and the entire international university community is needed to address the United Nations’ Sustainable Development Goals through the exact sciences, biomedical ingenuity, and engineering. But we also desperately need the insights from research in the humanities and social sciences. Society and its problems are certainly not merely technical in nature; they are perhaps to an even greater extent socio-cultural, and so are the solutions.
Actions that will contribute to this, have been mentioned above. They will require universities to reflect on their role, their identity, how they function, and their relationships with society. This is where the diversity of the context plays a major role; regional and national, but also cultural and historical. Based on this reflection, universities will need to take actions, many of which have been discussed here in general terms. These actions will be expressed in choices regarding content, education and research, and the organisation of the university that facilitates these actions and choices. That will place high demands on leadership: not just academic leadership, but also administrative leadership, which is essential in shaping the university. These are some big questions that touch on our pattern of values, responsibility, strategy, and mission, directing, resource allocation, cooperation at various levels, and modern HRM policy. We will discuss all of these aspects in more detail elsewhere in this book.
We use the term ‘Open Science’ because it has been the term of choice internationally for several years. That means it includes all of the sciences, including SS&H. Open Science is understood to include research and education, so ‘Open Science and Education’ would perhaps be a more accurate description. ↑
Miedema, F. (2022). Open Science: The Very Idea. Springer. ↑
Fecher, B. & Friesike, S. (2014). Open Science: One Term, Five Schools of Thought. In F. S. Bartling (Ed.), Opening science. Springer. ↑
Originally it was Open Data, but FAIR was added, which stands for Findable, Accessible, Interoperable and Reusable; see: go-fair.org ↑
Miedema, F. (2022). Open Science: The Very Idea (pp. 67-108). Springer. ↑
United Nations Educational, Scientific and Cultural Organization (UNESCO). (n.d.). Open Science: Making science more accessible, inclusive and equitable for the benefit of all. unesco.org/en/open-science ↑
European Commission. (2015). Open Innovation, Open Science, Open to the World. digital-strategy.ec.europa.eu/nl/node/10395 ↑
See: government.nl ↑
Miedema, F. (2022). Open Science: The Very Idea (pp. 179-210). Springer. ↑
De Knecht, S., Van der Meer, M., Brinkman, L., Kluijtmans, M. & Miedema, F. (2021). Reshaping the Academic Self: Connecting Education & Open Science. Zenodo. doi.org/10.5281/zenodo.5345573 ↑
Utrecht University. Open-education. Utrecht University Available via: uu.nl.
This is one of the normative principles, ideals of the classical sociology of science Merton described in the middle of the last century. See: Miedema, F. (2012). Science 3.0, Real Science, Real Knowledge (Chapter 4). Amsterdam University Press. ↑
See the website of CoalitionS: coalition-s.org ↑
Brainard, J. & Kaiser, J. (2022, August 26). White House requires immediate public access to all U.S.-funded research papers by 2025. ScienceInsider. science.org/content/article/white-house-requires-immediate-public-access-all-u-s--funded-research-papers-2025 ↑
JIF is a measure of the frequency with which articles in a particular journal are cited. That frequency then determines the journal’s status compared to other journals. ↑
Plan S. (2021, September 3). Diamond unearthed: shining light on community-driven Open Access Publishing. Plan S. coalition-s.org/diamond-unearthed-shining-light-on-community-driven-open-access-publishing ↑
Becerril, A., Bosman, J., Bjørnshauge, L., Frantsvåg, J. E., Kramer, B., Langlais, P., Mournier, P., Proudman, V., Redhead, C., Didier, T. (2021). OA Diamond Journals Study. Part 2: Recommendations. Zenodo. zenodo.org/records/4562790 ↑
Preferably ‘open peer review’, in which the names of the authors and reviewers are made public. ↑
Beigel, F. (2021, June 28). Latin America could become a world leader in non-commercial open science. The Conversation. theconversation.com/latin-america-could-become-a-world-leader-in-non-commercial-open-science-161019 ↑
Such an archive or repository contains data or articles that are not yet peer reviewed or accepted by journals. ↑
UNESCO. (n.d.). Open Science: Making science more accessible, inclusive and equitable for the benefit of all. unesco.org/en/open-science ↑
Nederlandse Federatie van Universitair Medische Centra. (n.d.). Valorisatie: Maatschappelijk verantwoord licentiëren. nfu.nl/themas/randvoorwaarden-wetenschappelijk-onderzoek/valorisatie ↑
See the website of the Novo Nordisk Foundation: novonordisk.com ↑
Authors Inset: Marjanneke Vijge and Anissa Triyanti. ↑
In that context, see the initiatives around RRI (‘Responsible Research and Innovation’) in the context of the European Framework Programs. ↑
Jasanoff, S. (1992). What Judges Should Know about the Sociology of Science. Jurimetrics, 32(3), 345-59. doi.org/10.1177/030631298028005007 ↑
Jasanoff, S. (2012). Science and Public Reason. Routledge. ↑
See discussion of the work of Shapin and Longino in: Miedema, F. (2022). Open Science: The Very Idea (pp. 15-66). Springer. ↑
Miedema, F. (2022). Open Science: The Very Idea (Chapter 4). Springer. ↑
Epstein, E. (2007). Inclusion: The Politics of Difference in Medical Research. University of Chicago Press. ↑
Miedema, F. (2010). Wetenschap 3.0. Van Academisch naar Post Academisch Onderzoek (Chapter 9). Amsterdam University Press. ↑
Merton, R. M. (1973). The Sociology of Science: Theoretical and Empirical Investigations. University of Chicago Press. ↑
Hessels, L. K., Van Lente, H. & Smits, R. (2009). In Search of Relevance: The Changing Contract between Science and Society. Science and Public Policy, 36(5), 387-401. doi.org/10.3152/030234209x442034. Below is an adaptation of Latour and Woolgar’s (1986) ‘credibility cycle’ published by Hessels in 2009: doi.org/10.3152/030234209x442034 ↑
Moher, D., Naudet, F., Cristea, I. A., Miedema, F., Ioannidis, J. P. A. & Goodman, S. N. (2018). Assessing Scientists for Hiring, Promotion, and Tenure. PLoS Biology, 16(3). doi.org/10.1371/journal.pbio.2004089 ↑
Wouters, P. (2014). The Citation: From Culture to Infrastructure. In B. Cronin & C. R. Sugimoto (Eds.), Beyond Bibliometrics, Harnessing Multidimensional Indicators of Scholarly Impact (pp. 47-66). MIT Press. ↑
Miedema, F. (2022). Open Science: The Very Idea. Springer. ↑
See for The Declaration on Research Assessment (DORA): sfdora.org ↑
For the position paper, see: scienceintransition.nl/over-science-in-transition/position-paper ↑
Responsible Metrics. (2022). The Metric Tide Revisited. responsiblemetrics.org/2022/08/11/the-metric-tide-revisited/ ↑
gezondheidsraad.nl/documenten/adviezen/2016/10/12/onderzoek-waarvan-je-beter-wordt ↑
Nederlandse Organisatie voor Wetenschappelijk Onderzoek (NWO). (2019). Position paper ‘Ruimte voor ieders talent’. nwo.nl/position-paper-ruimte-voor-ieders-talent ↑
Barker, K. (2007). The UK Research Assessment Exercise: The Evolution of a National Research Evaluation System. Research Evaluation, 16(1), 3-12. doi.org/10.3152/095820207X190674; see the REF website: ref.ac.uk ↑
Vereniging Samenwerkende Nederlandse Universiteiten (NSVU), KNAW & Nederlandse organisatie voor Wetenschappelijk Onderzoek (NWO). (2020). Strategy Evaluation Protocol 2021-2027. Available via: universiteitenvannederland.nl ↑
An association of public organisations that fund or conduct research, see: scienceeurope.org ↑
Directorate-General for Research and Innovation. (2022). Reforming research assessment: The Agreement is now final. European Commission. research-and-innovation.ec.europe.eu ↑
KNAW. (2022, October 11). Nederlandse kennisinstellingen tekenen Europees akkoord Evaluatie van wetenschappelijk onderzoek. KNAW Nieuws. knaw.nl/nieuws/nederlandse-kennisinstellingen-tekenen-europees-akkoord ↑
Poot et al. (2021, July 19). Nieuwe Erkennen en waarderen schaadt Nederlandse wetenschap. Science Guide. scienceguide.nl/2021/07/nieuwe-erkennen-en-waarderen-schaadt-nederlandse-wetenschap/ ↑
Stokes, D. E. (1997). Pasteur’s Quadrant: Basic Science and Technological Innovation. Brookings Institution Press. ↑
European Commission. (2017). MLE on Open Science: Almetrics and Rewards. projects.research-and-innovation.ec.europa.eu/en/statistics/policy-support-facility/mle-open-science-altmetrics-and-rewards ↑
European Commission. (2022, January 18). Read-out of the weekly meeting of the von der Leyen Commission by Margaritas Schinas, Vice-President of the European Commission, and Mariya Gabriel, European Commissioner, on higher education [Video]. EC Audio-Visual Service. ↑
In this context, see the website: charm-eu.eu ↑
UNESCO. (n.d.). Open Science: Making science more accessible, inclusive and equitable for the benefit of all. unesco.org/en/open-science ↑