Setting priorities for conducting and updating systematic reviews

Abstract

Abstract Systematic reviews - appraisal and synthesis of all primary research - are increasingly being used to inform policy and practice in health care. Therefore, it is important to understand how the key questions in systematic reviews are identified and prioritised and whether they are relevant to policy makers, practitioners and members of the public. Research priority setting (RPS) is usually defined as any interpersonal activity that leads to the selection of topics and/or choices of key questions to investigate . Diverse approaches to setting research priorities are used in different countries, regions and organisations. There is no consensus in the literature on the most effective processes with which to set these priorities. However, these decisions define the quality and implications of the evidence, and syntheses of it, available to patients, public and policy makers to help them make informed decisions. My initial scoping work, was to design and conduct a survey across an influential international systematic review organisation (Cochrane Collaboration ) on how they set priorities for their reviews. We identified 13 structured approaches to setting priorities. As part of the project, we developed an evaluation framework that demonstrated whether the priority setting processes meet the values and principles of the Cochrane Collaboration. Subsequently, we developed an equity lens for research priority setting exercises to inform the design of research priority setting processes to ensure that they consider the priorities of disadvantaged groups along with advantaged groups. We used the equity lens to do a second evaluation on the priority setting processes in the Cochrane Collaboration. Both evaluation frameworks demonstrated that the Cochrane Collaboration requires better designed priority setting approaches and must be more transparent in reporting those processes. The evaluation of research priority setting exercises in the Cochrane Collaboration, along with the wider literature, demonstrates that research priority setting exercises cannot be evaluated in isolation from organisational cultures, values and context. Therefore, the next step of the project focused on a specific stakeholder group (major research funders) with significant influence on research, including support for systematic reviews. We selected 11 national research agencies in the UK, Netherlands, France, Norway, Denmark, Germany, Australia, Canada, and the USA. We devised and used a checklist based on Chalmers and Glasziou’s “avoidable research waste” framework (and evaluated the processes and policies of these agencies using this checklist). As previous evaluations had demonstrated, this second evaluation found a lack of transparency in the process of setting priorities for research and other related organisational and policy issues. Increased funding is needed for methodological research to evaluate research practices and to monitor how funding research projects is done and reported. My evaluation of funding agencies and the Cochrane Collaboration found a similar lack of transparency and accountability in the context of conflicting values among stakeholders that decreases accountability and scrutiny of researchers and their institutions. However, the projects have led to organisational and policy changes in the two key stakeholder groups (the Cochrane Collaboration and selected funding agencies). Officials of national health research funding agencies have approached me to collaborate with them to address the issues raised by my work on reducing research waste. This led to the establishment of Funders Forum - the Ensuring Value in Research (EViR) Funders’ Collaboration and Development Forum - to enable agencies in various countries to exchange their experience in addressing issues and creating work groups to address them. The Forum is chaired by individuals from three major research funders: NIHR (UK), ZonMW (Netherlands) and Patient-Centered Outcomes Research Institute (PCORI; USA). The Forum organises several meetings to establish common principles, standards and work plans to achieve the common objective around reducing research waste and adding value for research for a national research funder.