Skip to main content
Download PDF

Protocol for the development of guidance for collaborator and partner engagement in health care evidence syntheses

Systematic Reviews volume 12, Article number: 134 (2023) Cite this article

Abstract

Background

Involving collaborators and partners in research may increase relevance and uptake, while reducing health and social inequities. Collaborators and partners include people and groups interested in health research: health care providers, patients and caregivers, payers of health research, payers of health services, publishers, policymakers, researchers, product makers, program managers, and the public. Evidence syntheses inform decisions about health care services, treatments, and practice, which ultimately affect health outcomes.

Our objectives are to:

A. Identify, map, and synthesize qualitative and quantitative findings related to engagement in evidence syntheses

B. Explore how engagement in evidence synthesis promotes health equity

C. Develop equity-oriented guidance on methods for conducting, evaluating, and reporting engagement in evidence syntheses

Methods

Our diverse, international team will develop guidance for engagement with collaborators and partners throughout multiple sequential steps using an integrated knowledge translation approach:

1. Reviews. We will co-produce 1 scoping review, 3 systematic reviews and 1 evidence map focusing on (a) methods, (b) barriers and facilitators, (c) conflict of interest considerations, (d) impacts, and (e) equity considerations of engagement in evidence synthesis.

2. Methods study, interviews, and survey. We will contextualise the findings of step 1 by assessing a sample of evidence syntheses reporting on engagement with collaborators and partners and through conducting interviews with collaborators and partners who have been involved in producing evidence syntheses. We will use these findings to develop draft guidance checklists and will assess agreement with each item through an international survey.

3. Consensus. The guidance checklists will be co-produced and finalised at a consensus meeting with collaborators and partners.

4. Dissemination. We will develop a dissemination plan with our collaborators and partners and work collaboratively to improve adoption of our guidance by key organizations.

Conclusion

Our international team will develop guidance for collaborator and partner engagement in health care evidence syntheses. Incorporating partnership values and expectations may result in better uptake, potentially reducing health inequities.

Summary

An evidence synthesis, sometimes called a systematic review, is a method which identifies, brings together, and analyses all the research studies which address a specific question about healthcare. They are often used to provide evidence for healthcare decisions, such as the therapies that will be recommended by a physician, or for public health. Including people and groups who have an interest in these decisions, such as, patients, healthcare providers, those who pay for health services, those who manage health programs, and others, can help make sure that the right questions are asked, and the right information is assessed. The best way to engage with these all these groups has not been identified. The goal of this project is to develop guidance for engaging with multiple groups or ‘collaborators and partners’ in each step of the evidence synthesis process. To prepare this guidance, we will identify and synthesize the available information on collaborator and partner engagement in evidence synthesis, conduct interviews and a survey and hold international meetings to develop and finalise draft checklists for collaborator and partner engagement in evidence synthesis.

Background

Evidence syntheses are used to inform the development of clinical practice, health systems, and public health guidelines [1]. While the rate of evidence synthesis production is steadily increasing there are serious questions about whether they are useful, meaningful, or accessible [2,3,4]. Given these challenges, as well as the associated costs of conducting evidence synthesis, maximizing their impact is important [5]. Engaging collaborators and partners in synthesising evidence that is meaningful to them improves equity and accessibility and the overall quality of the synthesis [6].

There is international recognition that the engagement of collaborators and partners in health care research is important for improving its impact. Meaningful engagement benefits the usefulness, relevance, quality, buy-in, uptake, and impact of research [7,8,9]. For example, engaging patients and the public can increase researchers’ understanding of the issues, appropriateness of the research, and interpretation of findings [8, 10]. High-quality research, co-produced with all key collaborators and partners, is fundamental to supporting the reduction of research waste and promoting equity [11]. Engagement can identify evidence gaps and refine scope, address barriers to the uptake of evidence, increase dissemination and application of findings, and thus help formulate recommendations for research [11,12,13,14].

There has been an increase in research teams undertaking collaborator and partner engagement in evidence synthesis as well as greater expectations from funders, but reporting of engagement is poor and there are evaluations or guidance about how to effectively engage different groups [15,16,17,18]. In parallel, there is a growing body of evidence providing guidance for collaborator and partner engagement in research and health care guidelines [16, 19,20,21]. Despite this, existing evidence predominantly focuses on patient and public involvement in primary research with a notable lack of tailored guidance for engaging other groups in evidence synthesis development [7, 8, 22, 23].

However, the most effective methods for engaging different collaborators and partners in evidence synthesis, identifying the barriers or facilitators for engagement, or reporting how collaborators and partners were engaged have not been identified. The Multi-Stakeholder Engagement (MuSE) Consortium has recently addressed these issues in relation to health guidelines [16]. Lessons learned from the guidelines project will inform this work as applicable. The current project aims to apply the lessons learned from that project to the conduct of evidence syntheses. We will develop guidance for collaborator and partner engagement in health care evidence syntheses to facilitate the production of relevant and useful evidence syntheses.

Study aim

Our overarching goals are to synthesize evidence relating to collaborator and partner engagement in health care evidence syntheses and to explore perspectives on how engagement in evidence syntheses can promote health equity. Our specific objectives are to:

  1. A)

    Identify, map, and synthesize qualitative and quantitative findings related to collaborator and partner engagement in evidence syntheses

  2. B)

    Explore perspectives on how collaborator and partner engagement in evidence syntheses promotes health equity

  3. C)

    Develop equity-oriented guidance on methods for collaborator and patient engagement in evidence synthesis

  4. D)

    Develop guidance on methods for evaluating collaborator and partner engagement in evidence syntheses

  5. E)

    Develop a guideline for reporting collaborator and partner engagement in evidence syntheses (Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) extension).

Definitions of key terms

Definitions of terms in this field often lack consistency and differ internationally [19]. For this project, we use the following definitions:

Health care evidence syntheses

Synthesize research evidence to address a health care question using rigorous, explicit, and transparent methods. These include scoping reviews, rapid reviews, and quantitative or qualitative systematic reviews [24].

Collaborator and partner

Any interested person or group who is responsible for or affected by health- and healthcare-related decisions [25]. We previously used the term 'stakeholder' to describe these groups, however the historical use of the word and its relation to colonialism is problematic. Through discussions with the MuSE Consortium members, we have decided to replace the term with 'collaborators and partners'. For this project we have grouped collaborators and partners into the following 11 categories, which we call the 11 ‘Ps’. These categories are designed to be comprehensive of all interested people or groups; however, we are cognisant that roles and terminology vary internationally, and some collaborators and partners may fit under multiple categories:

  • Patients/consumers, caregivers, and patient groups

  • Payers/funders of research

  • Payers and purchasers of health services (e.g. those who pay for or reimburse health-related interventions, including insurers, individuals with deductibles, others, and those entities responsible for underwriting the cost of care, such as employers and governments)

  • Publishers (those involved in the knowledge translation of evidence syntheses, e.g. peer-review editors, scientific publishers, science writers)

  • Policy-makers (e.g. governments and professional associations, those involved in the regulatory processes of drugs and health devices)

  • Principal investigators (e.g. researchers conducting studies that may or may not be relevant to the review)

  • Product makers (e.g. drug, natural products and/or device manufacturers)

  • Producers and commissioners of guidelines (e.g. institutions and organizations that commission, develop, or implement guideline development procedures) [26].

  • Program managers (e.g. managers/directors/administrators and individuals who plan, lead, oversee, or deliver any program that provides public health, community services, or clinical care (e.g. budgeting, hiring, staffing, organizing, coordinating, reporting). These individuals may be health care providers but are not on the point of care delivering health care related to the program of interest (e.g. overseeing an immunisation program but not delivering vaccinations)

  • Providers (individuals and/or organizations providing care, such as nurses, physicians, pharmacists, community-based workers)

  • Public (e.g. communities or general members of the population or community, excluding patients, caregivers, and health professionals, living or working with the condition of interest)

Conflict of interest

“A conflict of interest exists when a past, current or expected interest creates a significant risk of inappropriately influencing an individual’s judgment, decision, or action when carrying out a specific duty” [26] and may be related to financial, intellectual or other interests. Interest refers to a benefit (e.g. money received from industry) or to an attribute of the individual (e.g. having specific beliefs about religion, evidence-based medicine).

Engagement

Refers to the approach to gather input or contribution from collaborators and partners and is multi-directional, resulting in “informed decision-making about the selection, conduct, and use of the research” [27]. Language varies internationally, with other terms including “partnerships”, “involvement”, “consultation”, “co-production”, “co-creation”, and “Patient and Public Involvement (PPI)” [19].

Health equity

“The absence of unfair, avoidable or remediable differences among groups of people, whether those groups are defined socially, economically, demographically, or geographically or by other dimensions of inequality” [28]. We use the acronym PROGRESS-Plus to identify characteristics which may contribute to health inequities (place of residence (e.g. country), race/ethnicity/culture, occupation, gender or sex and other identities, religion, education, socioeconomic status, social capital, and other characteristics including age or career stage) and to identify groups who may be underrepresented in evidence synthesis [29].

The MuSE consortium

The MuSE Consortium is a group of over 160 individuals from 20 countries representing our 11 'Ps' who each have an interest in collaborator and partner engagement in health research, evidence syntheses, and guidelines. This project is complementary to another MuSE project which aims to develop guidance for engagement in health guidelines [16].

Methods

Methodological approach

Our methodological approach is adapted from the guidance for developing research reporting guidelines [30] and our previous project to develop guidance for collaborator and partner engagement in health guideline development [16]: identifying the need for guidance; reviewing the literature; generating a list of candidate items for consideration; conducting key informant interviews to refine the candidate items; conducting a survey to assess agreement with candidate items; holding an in-person consensus meeting; and developing, piloting of draft guidance items an publishing and disseminating the guidance [31].

This project uses a multiple mixed methods design. We will triangulate the findings of the evidence syntheses, methodological study, and interviews to develop a set of draft guidance items related to collaborator and partner engagement in evidence synthesis [32]. The series of evidence syntheses will be planned a priori with separate published protocols. However, step 2 may include emergent components to enable us to explore issues identified in the literature [33].

Positionality statement

The research team includes contributors to the Campbell and Cochrane Collaborations. Our team is comprised of individuals representing our identified collaborator and partner categories (11 Ps) who are all committed to the value of collaborator and partner engagement in evidence synthesis as well as improving the reporting and evaluation of engagement.

Integrated knowledge translation

We have assembled an international core management group which has developed this protocol with our collaborator and partner categories (11 Ps). The core management group will manage the day-to-day aspects of the projects. We are also establishing an international advisory group which will include additional representatives of our 11 Ps. They will be invited to engage with the co-leads of their collaborator and partner category to develop the draft guidance items for their group. We will provide further opportunities for a wide range of relevant collaborators and partners, including other members of the MuSE Consortium and Cochrane Consumers. Please see Fig. 1 and Additional file 1. We will engage with all members of the MuSE Consortium through quarterly newsletters and will invite them to provide feedback throughout the project.

Fig. 1
figure 1

Project team

Full size image

Project design

This project has four sequential steps that will result in the development of consensus-driven guidance (Fig. 2) [34]. We will first conduct a series of evidence syntheses: one scoping review, three systematic reviews and one evidence map. We will conduct a descriptive assessment of published evidence syntheses which report how collaborators and partners were engaged in the process. We will solicit additional details and contextual information about collaborator and partner engagement through key informant interviews so that we can draft items to be included in the guidance documents. We will seek broad input into these guidance items through an international survey. The results of the survey will be taken to a consensus meeting which will aim to finalise the guidance which will then be widely disseminated. The following sections describe each step of the process.

Fig. 2
figure 2

Project plan

Full size image

Step 1: evidence syntheses

Description of the series of evidence syntheses

We will complete four reviews and an evidence map to identify and synthesize qualitative and quantitative findings related to collaborator and partner engagement. The topics of these reviews are:

  1. a)

    Collaborator and partner engagement methods (scoping review and evidence map)

  2. b)

    Barriers and facilitators to collaborator and partner engagement in conduct of evidence synthesis (systematic review)

  3. c)

    Impact of collaborator and partner engagement in evidence synthesis (systematic review)

  4. d)

    Conflicts of interest in collaborator and partner engagement in evidence synthesis (systematic review)

  5. e)

    Equity considerations in collaborator and partner engagement in evidence synthesis (evidence map)

We will follow established methods for high quality evidence syntheses and evidence mapping and publish separate, detailed protocols review will using the Open Science Framework [31, 35,36,37]. Table 1 describes the outcomes of interest and provides an example of an included study for each of the reviews and the evidence map. Briefly, we will coordinate this series of reviews by conducting one broad comprehensive search for published literature combining concepts of “collaborators" and "partners”, “engagement”, and “evidence synthesis”. We will develop and test comprehensive search strategies in consultation with an experienced information specialist which will include electronic databases as well as an extensive grey literature search. The strategies will be peer reviewed using the PRESS checklist [38]. We will adapt the search of the previous scoping review and search databases including health databases (e.g. MEDLINE and CINAHL) as well as sociological, psychological, and international development databases. We will use the citation chaser tool to identify additional potentially relevant studies [39]. This tool conducts backward and forward citation tracking. We will also search the websites of institutions of agencies that produce evidence syntheses and their methods, such as JBI, and the EPPI-Centre. We will not include papers describing engagement in health technology assessments (HTA). While the methods used for evidence synthesis within HTAs are similar, the process is quite distinct and is often more responsive to policy needs [40].

Table 1 Description of each review
Full size table

Records will be imported into Covidence (https://www.covidence.org/) and de-duplicated. Pairs of independent reviewers will conduct title and abstract screening, tagging potentially relevant studies for inclusion into any of our reviews. Full text “tagged” papers will be retrieved for each of the individual reviews and will be considered alongside specific eligibility criteria. Data from studies judged as relevant will be extracted by two independent reviewers and the methodological quality of the evidence assessed, as appropriate, for each review.

We will invite input from MuSE members and our advisory group on our planned search strategies. To include additional perspectives, we will invite recommendations for literature to be included in these reviews by reaching out to non-MuSE members to ensure we capture published and unpublished literature using social media and through our mailing lists.

Step 2: methods study, interviews, and survey

Methods study

To expand on the findings of the evidence syntheses conducted in step 1, we will conduct a descriptive assessment of a random sample of published evidence syntheses that have engaged with collaborators and partners. Using the search strategy developed for step 1, we will identify examples of published evidence syntheses that report on engagement with collaborators and partners in their abstract. Two reviewers will independently screen that titles and abstracts of potential evidence syntheses to assess their eligibility. We will include any evidence synthesis that reports on engagement with any of our identified collaborator and partner categories at any stage of the evidence synthesis process.

We will export the records to Microsoft Excel and sort them randomly using the built-in random number generator. We will review the full texts of the evidence syntheses until we have reached our target sample size of 100 evidence syntheses that report on collaborator and partner engagement.

We will develop and pilot test a data extraction form in Excel. We will extract information related to the type of collaborators and partners involved, the type of evidence synthesis, the training provided to collaborators and partners, the mode and frequency of their engagement, and the stage at which they were involved.

Key informant interviews

We aim for broad relevance of this guidance for different health care areas. Therefore, we will seek input from external international individuals representing our 11 Ps on the proposed guidance via key informant interviews. We will utilize the results of the above systematic reviews, scoping review, evidence map, and methods study to develop draft candidate guidance items and seek perspectives on these as well as suggestions for additional items. We will develop a semi-structured interview guide and invite key informants using purposeful sampling for a maximum variation of our 11 P categories. This will promote participation from low- and middle-income countries (LMICs) and across gender, sex, and other intersecting identity factors. We will utilize the factors for identification and invitation of individual representatives previously published by members of our team [46]. Our advisory group and the MuSE Consortium will suggest participants and we will use snowball sampling for additional interview participants by asking each key informant to identify others. Sample size for the interviews will depend on the available evidence from the systematic reviews, scoping review, and evidence map. Our collaborator and partner co-leads will determine when we have completed enough interviews to clarify the evidence from the reviews. We anticipate conducting between 3 and 10 interviews per collaborator and partner category, based on our previous experience with the MuSE Guidelines project [17].

Two researchers will analyze transcribed interviews using a six-phase process of thematic analysis [47]. The process of qualitative data analysis will use the steps of (1) familiarisation with data; (2) generation of initial codes within each transcript; (3) search for themes; (4) review of themes; (5) define and name themes; and (6) reporting of themes [47]. We will also assess whether identified themes vary by gender or sex and other equity-relevant PROGRESS-Plus characteristics, where possible. We will submit this project to the Bruyère Research Ethics Board for approval. We will use transcription software for the audio recordings and we will use NVIVO software for the qualitative analysis (https://www.qsrinternational.com/).

International survey

We will use the results of the above systematic reviews and key informant interviews to draft a list of candidate items for guidance on collaborator and partner engagement. We will survey international collaborators and partners using this preliminary list of guidance items to gather opinions about each draft item. We will disseminate the survey broadly though the members of the MuSE Consortium and their networks as well as through relevant organizations, such as Campbell, Cochrane, and JBI (formerly the Joanna Briggs Institute). We will aim for diversity in the identification of survey recipients by inviting responses from LMICs and by targeted outreach, using methods our group has used previously, such as sharing the survey invitation through listservs, inviting key individuals or organizations as recommended by the MuSE members, and social media [48].

We will ask respondents to indicate their agreement with each draft guidance item and we will invite open-ended responses, as well as suggestions for additional items. We will tabulate the frequency of agreement for each proposed item. We will code the open-ended responses into themes using the methods described above. This survey will be submitted to the Bruyère Research Ethics Board.

Step 3: consensus activities

We will use the results of the systematic reviews, key informant interviews, and survey to develop and refine the draft guidance and finalise the guidance based on a consensus approach at an in-person (if possible) or virtual meeting. We will use nominal group technique to gain agreement [49], which allows participants to indicate their ideas or opinions privately and then present these in a ‘round robin’ format with other consensus participants [50]. This method allows all collaborators and partners to have an equal voice [51].

Step 4: dissemination

The guidance finalised at the consensus meeting will be included in a series of guidance documents addressing equity-oriented guidance on methods for collaborator and partner engagement in evidence synthesis, evaluating engagement in evidence synthesis, and reporting engagement in evidence synthesis. The core management team, collaborator and partner co-leads, and advisory group will co-produce a dissemination plan and work collaboratively to improve adoption of our guidance by key organizations. We will utilize the international MuSE Consortium network to disseminate guidance documents and share these via social media, including Twitter and blogs, as well as through the multiple listservs maintained by the members of the MuSE Consortium.

Discussion

This work is intended to produce guidance for engaging collaborators and partners in conducting evidence syntheses, transparently reporting this engagement, and evaluating engagement. The guidance for collaborator and partner engagement in evidence syntheses will be tested through an iterative, consensus building process, via in-person meetings, teleconferences, and email correspondence with collaborators and partners of our 11 categories. We will ensure synergy with our related guidance for engagement in guideline development. Developing guidance for collaborator and partner engagement, guidance for evaluating, and reporting engagement may assist with the uptake of these resources by relevant organizations.

Availability of data and materials

Not applicable.

Abbreviations

LMICs:

Low- and middle-income countries

MuSE:

Multi-Stakeholder Engagement

PROGRESS:

Place of residence, Race/ethnicity/culture/language, Occupation, Gender and sex, Religion, Education, Socioeconomic Status, Social capital

References

  1. Fox DM. Evidence and health policy: using and regulating systematic reviews. Am J Public Health. 2017;107(1):88–92.

    Article  PubMed  PubMed Central  Google Scholar 

  2. Gopalakrishnan S, Ganeshkumar P. Systematic reviews and meta-analysis: understanding the best evidence in primary healthcare. J Family Med Prim Care. 2013;2(1):9–14.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  3. Rada G, Pérez D, Araya-Quintanilla F, Ávila C, Bravo-Soto G, Bravo-Jeria R, et al. Epistemonikos: a comprehensive database of systematic reviews for health decision-making. BMC Med Res Methodol. 2020;20(1):286.

    Article  PubMed  PubMed Central  Google Scholar 

  4. Ioannidis JP. The Mass Production of Redundant, Misleading, and Conflicted Systematic Reviews and Meta-analyses. Milbank Q. 2016;94(3):485–514.

    Article  PubMed  PubMed Central  Google Scholar 

  5. Karunananthan S, Maxwell L, Welch V, Petkovic J, Pardo J, Rader T, et al. PROTOCOL: When and how to replicate systematic reviews. Campbell Systematic Reviews. 2020;16(2):e1087.

  6. Cochrane. Strategy to 2020: Cochrane Collaboration; [Available from: https://community.cochrane.org/organizational-info/resources/strategy-2020.

  7. Kreis J, Puhan MA, Schünemann HJ, Dickersin K. Consumer involvement in systematic reviews of comparative effectiveness research. Health Expect. 2013;16(4):323–37.

    Article  PubMed  Google Scholar 

  8. Brett J, Staniszewska S, Mockford C, Herron-Marx S, Hughes J, Tysall C, et al. A systematic review of the impact of patient and public involvement on service users, researchers and communities. Patient. 2014;7(4):387–95.

    Article  PubMed  Google Scholar 

  9. INVOLVE. Factors affecting public engagement by researchers: Reflections on the changing landscape of public engagement by researchers in the UK 2015 [Available from: https://wellcome.ac.uk/sites/default/files/wtp060034.pdf.

  10. INVOLVE. Exploring the impact of public involvement on the quality of research: examples. National Institute of Health Research. Easleigh: INVOLVE; 2013.

  11. Chalmers I, Bracken MB, Djulbegovic B, Garattini S, Grant J, Gülmezoglu AM, et al. How to increase value and reduce waste when research priorities are set. Lancet. 2014;383(9912):156–65.

    Article  PubMed  Google Scholar 

  12. Cochrane Collaboration. Cochrane 2018 Annual Review: Cochrane; 2018 [Available from: https://www.cochrane.org/sites/default/files/public/uploads/2018_annual_review_-_july_2019-final_1.pdf.

  13. Cottrell E, Whitlock E, Kato E, Uhl S, Belinson S, Chang C, et al. Defining the benefits of stakeholder engagement in systematic reviews. Research White Paper. Rockville, MD: Agency for Healthcare Research and Quality. 290–2012–00004-C SRCuCN; 2014.

  14. Keown K, Van Erd D, Irvin E. Stakeholder engagement opportunities in systematic reviews: Knowledge transfer for policy and practice. J Contin Educ Heal Prof. 2008;28(2):67–72.

    Article  Google Scholar 

  15. Pollock A, Campbell P, Struthers C, Synnot A, Nunn J, Hill S, et al. Development of the ACTIVE framework to describe stakeholder involvement in systematic reviews. J Health Serv Res Policy. 2019;24(4):245–55.

    Article  PubMed  Google Scholar 

  16. Petkovic J, Riddle A, Akl E, Khabsa J, Lytvyn L, Atwere P, et al. Protocol for the development of guidance for stakeholder engagement in health and healthcare guideline development and implementation. Syst Rev. 2020;9(1):21.

    Article  PubMed  PubMed Central  Google Scholar 

  17. Concannon TW, Grant S, Welch V, Petkovic J, Selby J, Crowe S, et al. Practical Guidance for Involving Stakeholders in Health Research. J Gen Intern Med. 2019;34(3):458–63.

    Article  PubMed  Google Scholar 

  18. Boaz A, Hanney S, Borst R, O’Shea A, Kok M. How to engage stakeholders in research: design principles to support improvement. Health Res Policy Syst. 2018;16(1):60.

    Article  PubMed  PubMed Central  Google Scholar 

  19. Hoddinott P, Pollock A, O’Cathain A, Boyer I, Taylor J, MacDonald C, et al. How to incorporate patient and public perspectives into the design and conduct of research. F1000Res. 2018;7:752.

    Article  PubMed  PubMed Central  Google Scholar 

  20. Brett J, Staniszewska S, Mockford C, Herron-Marx S, Hughes J, Tysall C, et al. Mapping the impact of patient and public involvement on health and social care research: a systematic review. Health Expect. 2014;17(5):637–50.

    Article  PubMed  Google Scholar 

  21. INVOLVE. Guidance on co-producing a research project: National Institute of Health Research; [Available from: https://www.invo.org.uk/wp-content/uploads/2019/04/Copro_Guidance_Feb19.pdf.

  22. Boote J, Wong R, Booth A. “Talking the talk or walking the walk?” A bibliometric review of the literature on public involvement in health research published between 1995 and 2009. Health Expect. 2015;18(1):44–57.

    Article  PubMed  Google Scholar 

  23. Feldmann J, Puhan MA, Mütsch M. Characteristics of stakeholder involvement in systematic and rapid reviews: a methodological review in the area of health services research. BMJ Open. 2019;9(8):e024587.

    Article  PubMed  PubMed Central  Google Scholar 

  24. Munn Z, Pollock D, Price C, Aromataris E, Stern C, Stone J, et al. Investigating different typologies for the synthesis of evidence: a scoping review protocol. JBI Evid Synth. 2023;21(3):592–600. https://0-doi-org.brum.beds.ac.uk/10.11124/JBIES-22-00122.

  25. Concannon TW, Meissner P, Grunbaum JA, McElwee N, Guise JM, Santa J, et al. A new taxonomy for stakeholder engagement in patient-centered outcomes research. J Gen Intern Med. 2012;27(8):985–91.

    Article  PubMed  PubMed Central  Google Scholar 

  26. Akl EA, Hakoum M, Khamis A, Khabsa J, Vassar M, Guyatt G. A framework is proposed for defining, categorizing, and assessing conflicts of interest in health research. J Clin Epidemiol. 2022;149:236–43.

  27. WHO. Guidelines Review Committee 2022 [Available from: https://www.who.int/groups/guidelines-review-committee.

  28. WHO. Health Equity Geneva: World Health Organization; no date [Available from: https://www.who.int/health-topics/health-equity#tab=tab_1.

  29. O’Neill J, Tabish H, Welch V, Petticrew M, Pottie K, Clarke M, et al. Applying an equity lens to interventions: using PROGRESS ensures consideration of socially stratifying factors to illuminate inequities in health. J Clin Epidemiol. 2014;67(1):56–64.

    Article  PubMed  Google Scholar 

  30. Moher D, Schulz KF, Simera I, Altman DG. Guidance for developers of health research reporting guidelines. PLoS Med. 2010;7(2):e1000217.

    Article  PubMed  PubMed Central  Google Scholar 

  31. Moher D, Liberati A, Tetzlaff J, Altman DG, Group P. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. Int J Surg. 2010;8(5):336–41.

    Article  PubMed  Google Scholar 

  32. Martha D, Sousa VD, Mendes IA. An overview of research designs relevant to nursing: part 3: mixed and multiple methods. Rev Lat Am Enfermagem. 2007;15(5):1046–9.

    Article  PubMed  Google Scholar 

  33. Schoonenboom J, Johnson RB. How to Construct a Mixed Methods Research Design. Kolner Z Soz Sozpsychol. 2017;69(Suppl 2):107–31.

    Article  PubMed  PubMed Central  Google Scholar 

  34. Mertler C. Chapter 8, Mixed-Methods Research. Introduction to Educational Research. Corwin: A SAGE Company; 2016.

    Google Scholar 

  35. Higgins JPT, Lasserson T, Chandler J, Tovey D, Thomas J, Flemyng E, Churchill R, et al. [Eds]. Cochrane Handbook for Systematic Reviews of Interventions version 6.0 (updated July 2019). Cochrane; 2019.

  36. Moher D, Shamseer L, Clarke M, Ghersi D, Liberati A, Petticrew M, et al. Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015 statement. Syst Rev. 2015;4:1.

    Article  PubMed  PubMed Central  Google Scholar 

  37. Saran A, White H. Evidence and gap maps: a comparison of different approaches. Campbell Syst Rev. 2018;14(1):1–38.

    Article  PubMed  PubMed Central  Google Scholar 

  38. McGowan J, Sampson M, Salzwedel DM, Cogo E, Foerster V, Lefebvre C. PRESS Peer Review of Electronic Search Strategies: 2015 Guideline Statement. J Clin Epidemiol. 2016;75:40–6.

    Article  PubMed  Google Scholar 

  39. Haddaway NR, Grainger MJ, Gray CT. Citationchaser: a tool for transparent and efficient forward and backward citation chasing in systematic searching. Res Synth Methods. 2022;13(4):533–45.

    Article  PubMed  Google Scholar 

  40. Rotstein D, Laupacis A. Differences between systematic reviews and health technology assessments: a trade-off between the ideals of scientific rigor and the realities of policy making. Int J Technol Assess Health Care. 2004;20(2):177–83.

    Article  PubMed  Google Scholar 

  41. Boote J, Baird W, Sutton A. Involving the public in systematic reviews: a narrative review of organizational approaches and eight case examples. J Comp Eff Res. 2012;1(5):409–20.

    Article  PubMed  Google Scholar 

  42. Bayliss K, Starling B, Raza K, Johansson EC, Zabalan C, Moore S, et al. Patient involvement in a qualitative meta-synthesis: lessons learnt. Res Involv Engagem. 2016;2:18.

    Article  PubMed  PubMed Central  Google Scholar 

  43. Coon JT, Gwernan-Jones R, Moore D, Richardson M, Shotton C, Pritchard W, et al. End-user involvement in a systematic review of quantitative and qualitative research of non-pharmacological interventions for attention deficit hyperactivity disorder delivered in school settings: reflections on the impacts and challenges. Health Expect. 2016;19(5):1084–97.

    Article  PubMed  Google Scholar 

  44. Bou-Karram L, Hakoun MB, Hammoud MZ, Khamis AM, Al-Gibbawi M, Badour S, Hasbani DJ, et al. Reporting of financial and non-financial conflicts of interest in systematic reviews on health policy and systems research: a cross-sectional study. Int J Health Policy Manag. 2018;7(8):711–17.

  45. Walker SC, White J, Rodriguez V, Turk E, Gubner N, Ngo S, et al. Cocreating evidence-informed health equity policy with community. Health Serv Res. 2022;57(Suppl 1):137–48.

    Article  PubMed  PubMed Central  Google Scholar 

  46. Parker R, Tomlinson E, Concannon TW, Akl E, Petkovic J, Welch VA, et al. Factors to Consider During Identification and Invitation of Individuals in a Multi-stakeholder Research Partnership. J Gen Intern Med. 2022;37(16):4047–53.

  47. Braun V, Clarke V. Using thematic analysis in psychology. Qual Res Psychol. 2008;3(2):77–101.

    Article  Google Scholar 

  48. Welch VA, Norheim OF, Jull J, Cookson R, Sommerfelt H, Tugwell P, et al. CONSORT-Equity 2017 extension and elaboration for better reporting of health equity in randomised trials. BMJ. 2017;359:j5085.

    Article  PubMed  Google Scholar 

  49. Group OW. Chapter 7: Reaching Consensus in OMERACT. OMERACT Handbook: 2018; [Available from: https://omeracthandbook.org/handbook.

  50. Foth T, Efstathiou N, Vanderspank-Wright B, Ufholz LA, Dütthorn N, Zimansky M, et al. The use of delphi and nominal group technique in nursing education: a review. Int J Nurs Stud. 2016;60:112–20.

    Article  PubMed  Google Scholar 

  51. Haire C, Nakamoto E, et al. Engaging stakeholders to identify and prioritize future research needs. Agency for Healthcare Research and Quality: Rockville; 2011.

    Google Scholar 

Download references

Acknowledgements

The authors are grateful to Maureen Smith, Elise Berliner, and Neal Haddaway for their contributions to the research grant proposal.

Funding

This project is funded by a Canadian Institutes of Health Research Project Grant.

Author information

Author notes

  1. Peter Tugwell, Vivian Welch are co-first authors.

Authors and Affiliations

  1. Department of Medicine, Faculty of Medicine, University of Ottawa, Ottawa, Canada

    Peter Tugwell, Jordi Pardo Pardo, Elizabeth Tanjong Ghogomu & Jennifer Petkovic

  2. Ottawa Hospital Research Institute, Clinical Epidemiology Program, Ottawa, Canada

    Peter Tugwell

  3. School of Epidemiology and Public Health, Faculty of Medicine, University of Ottawa, Ottawa, Canada

    Peter Tugwell, Vivian Welch, Ian D. Graham & Tanya Horsley

  4. WHO Collaborating Centre for Knowledge Translation and Health Technology Assessment in Health Equity, Bruyère Research Institute, Ottawa, Canada

    Peter Tugwell

  5. Bruyere Research Institute, Ottawa, Canada

    Vivian Welch, Olivia Magwood, Omar Dewidar, Alison Riddle, Elizabeth Tanjong Ghogomu & Jennifer Petkovic

  6. Interdisciplinary School of Health Sciences, University of Ottawa, Ottawa, Canada

    Olivia Magwood

  7. Nursing Midwifery and Allied Health Professions (NMAHP) Research Unit, Glasgow Caledonian University, Glasgow, UK

    Alex Todhunter-Brown & Pauline Campbell

  8. Department of Internal Medicine, American University of Beirut, Beirut, Lebanon

    Elie A. Akl

  9. Department of Health Research Methods, Evidence, and Impact (HEI), McMaster University, Hamilton, ON, Canada

    Elie A. Akl & Arnav Agarwal

  10. The RAND Corporation and Tufts University School of Medicine, Boston, MA, USA

    Thomas W. Concannon

  11. Clinical Research Institute, American University of Beirut Medical Center, Beirut, Lebanon

    Joanne Khabsa

  12. Cochrane, London, UK

    Richard Morley

  13. Department of Health Research Methods, Evidence, and Impact, Michael G DeGroote Cochrane Canada Centre, Cochrane Canada and McMaster GRADE Centre, McMaster University, Hamilton, ON, Canada

    Holger Schunemann

  14. Department of Medicine, McMaster University, Hamilton, ON, Canada

    Holger Schunemann

  15. Department of Biomedical Sciences, Humanitas University, Milan, Italy

    Holger Schunemann

  16. Cochrane Canada, Hamilton, ON, Canada

    Holger Schunemann

  17. McMaster University, Hamilton, Canada

    Lyubov Lytvyn

  18. Division of General Internal Medicine, Department of Medicine, McMaster University, Hamilton, ON, Canada

    Arnav Agarwal

  19. International Health Department, ISGlobal, Hospital Clínic-Universitat de Barcelona, Barcelona, Spain

    Alba Antequera

  20. Canadian Council On Animal Care, Ottawa, Canada

    Marc T. Avey

  21. Agency for Healthcare Research and Quality, Rockville, MD, USA

    Christine Chang

  22. Annals of Internal Medicine, Philadelphia, PA, USA

    Stephanie Chang

  23. Department of Clinical Epidemiology, University of the Philippines-Manila, Manila, Philippines

    Leonila Dans

  24. Research Translation, National Health and Medical Research Council, Canberra, Australia

    Davina Ghersi

  25. Sydney Medical School, The University of Sydney, Sydney, Australia

    Davina Ghersi

  26. Centre for Implementation Research, Ottawa Hospital Research Institute, Ottawa, Canada

    Ian D. Graham & Chris McCutcheon

  27. Cumming School of Medicine, University of Calgary, Calgary, Canada

    Glen Hazlewood

  28. Cochrane Central Executive Team, London, UK

    Jennifer Hilgart

  29. Royal College of Physicians and Surgeons of Canada, Ottawa, Canada

    Tanya Horsley

  30. PharmaQuant, Kolkata, India

    Denny John

  31. Center for Public Health Research (CPHR), Kolkata, India

    Denny John

  32. School of Rehabilitation Therapy, Queen’s University, Kingston, ON, Canada

    Janet Jull

  33. Faculty of Medicine, University of Ottawa, Ottawa, Canada

    Lara J. Maxwell & Alison Riddle

  34. Faculty of Health and Medical Sciences, JBI, University of Adelaide, Adelaide, South Australia, Australia

    Zachary Munn

  35. Unit of Epidemiology and Statistics, IRCCS Istituto delle Scienze Neurologiche di Bologna, Bologna, Italy

    Francesco Nonino

  36. Ottawa Hospital Research Institute, Ottawa Methods Centre, Ottawa, Canada

    Jordi Pardo Pardo

  37. Cochrane Pain Palliative and Supportive Care, Oxford University Hospitals Trust, Oxford, England

    Roses Parker

  38. Departments of Family Medicine and Epidemiology and Biostatistics, Western University, London, ON, Canada

    Kevin Pottie

  39. Department of Family Medicine, University of Ottawa, Ottawa, Canada

    Kevin Pottie

  40. Epistemonikos Foundation, Santiago, Chile

    Gabriel Rada

  41. UC Evidence Centre and Department of Internal Medicine, Pontificia Universidad Católica de Chile, Santiago, Chile

    Gabriel Rada

  42. School of Public Health and Preventive Medicine, Monash University, Level 4, 553 St Kilda Road, Melbourne Victoria, 3004, Australia

    Anneliese Synnot

  43. Centre for Health Communication and Participation, School of Public Health and Psychological Sciences, La Trobe University, Plenty Rd, Bundoora, VIC, 3086, Australia

    Anneliese Synnot

  44. Bristol Medical School, Population Health Sciences, University of Bristol, Bristol, UK

    Eve Tomlinson

  45. School of Rehabilitation Sciences, Faculty of Health Sciences, University of Ottawa, Ottawa, Canada

    Karine Toupin-April

  46. Department of Pediatrics, Faculty of Medicine, University of Ottawa, Ottawa, Canada

    Karine Toupin-April

  47. Children’s Hospital of Eastern Ontario Research Institute, Ottawa, Canada

    Karine Toupin-April

  48. Institut du Savoir Montfort, Ottawa, Canada

    Karine Toupin-April

Authors
  1. Peter Tugwell
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Vivian Welch
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Olivia Magwood
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Alex Todhunter-Brown
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Elie A. Akl
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Thomas W. Concannon
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Joanne Khabsa
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Richard Morley
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Holger Schunemann
    View author publications

    You can also search for this author in PubMed Google Scholar

  10. Lyubov Lytvyn
    View author publications

    You can also search for this author in PubMed Google Scholar

  11. Arnav Agarwal
    View author publications

    You can also search for this author in PubMed Google Scholar

  12. Alba Antequera
    View author publications

    You can also search for this author in PubMed Google Scholar

  13. Marc T. Avey
    View author publications

    You can also search for this author in PubMed Google Scholar

  14. Pauline Campbell
    View author publications

    You can also search for this author in PubMed Google Scholar

  15. Christine Chang
    View author publications

    You can also search for this author in PubMed Google Scholar

  16. Stephanie Chang
    View author publications

    You can also search for this author in PubMed Google Scholar

  17. Leonila Dans
    View author publications

    You can also search for this author in PubMed Google Scholar

  18. Omar Dewidar
    View author publications

    You can also search for this author in PubMed Google Scholar

  19. Davina Ghersi
    View author publications

    You can also search for this author in PubMed Google Scholar

  20. Ian D. Graham
    View author publications

    You can also search for this author in PubMed Google Scholar

  21. Glen Hazlewood
    View author publications

    You can also search for this author in PubMed Google Scholar

  22. Jennifer Hilgart
    View author publications

    You can also search for this author in PubMed Google Scholar

  23. Tanya Horsley
    View author publications

    You can also search for this author in PubMed Google Scholar

  24. Denny John
    View author publications

    You can also search for this author in PubMed Google Scholar

  25. Janet Jull
    View author publications

    You can also search for this author in PubMed Google Scholar

  26. Lara J. Maxwell
    View author publications

    You can also search for this author in PubMed Google Scholar

  27. Chris McCutcheon
    View author publications

    You can also search for this author in PubMed Google Scholar

  28. Zachary Munn
    View author publications

    You can also search for this author in PubMed Google Scholar

  29. Francesco Nonino
    View author publications

    You can also search for this author in PubMed Google Scholar

  30. Jordi Pardo Pardo
    View author publications

    You can also search for this author in PubMed Google Scholar

  31. Roses Parker
    View author publications

    You can also search for this author in PubMed Google Scholar

  32. Kevin Pottie
    View author publications

    You can also search for this author in PubMed Google Scholar

  33. Gabriel Rada
    View author publications

    You can also search for this author in PubMed Google Scholar

  34. Alison Riddle
    View author publications

    You can also search for this author in PubMed Google Scholar

  35. Anneliese Synnot
    View author publications

    You can also search for this author in PubMed Google Scholar

  36. Elizabeth Tanjong Ghogomu
    View author publications

    You can also search for this author in PubMed Google Scholar

  37. Eve Tomlinson
    View author publications

    You can also search for this author in PubMed Google Scholar

  38. Karine Toupin-April
    View author publications

    You can also search for this author in PubMed Google Scholar

  39. Jennifer Petkovic
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

PT and TC led the establishment of the MuSE Consortium. PT and VW are co-principal applicants on the CIHR project grant funding this project. PT, JP, TWC, VW, EAA, HS, OM, LL, JK, ATB, and RM were involved in the conceptualisation of this paper. JP wrote the first draft and all authors reviewed and approved the final manuscript.

Corresponding author

Correspondence to Jennifer Petkovic.

Ethics declarations

Ethics approval and consent to participate

We will seek ethical approval from the Bruyère Research Ethics Board.

Consent for publication

Not applicable.

Competing interests

LJM is Managing Editor of Cochrane Musculoskeletal. CC is employed by the US Agency for Healthcare Research and Quality. ZM is an employee of JBI. All other authors declare that they have no competing interests.

Additional information

Publisher’ s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Tugwell, P., Welch, V., Magwood, O. et al. Protocol for the development of guidance for collaborator and partner engagement in health care evidence syntheses. Syst Rev 12, 134 (2023). https://0-doi-org.brum.beds.ac.uk/10.1186/s13643-023-02279-1

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://0-doi-org.brum.beds.ac.uk/10.1186/s13643-023-02279-1

Keywords

Systematic Reviews

ISSN: 2046-4053