Skip to main content

Protocol for a systematic review of N-of-1 trial protocol guidelines and protocol reporting guidelines

Abstract

Background

N-of-1 trials are multiple cross-over trials done in individual participants, generating individual treatment effect information. While reporting guidelines for the CONSORT Extension for N-of-1 trials (CENT) and the Standard Protocol Items: Recommendations for Interventional Trials (SPIRIT) already exist, there is no standardized recommendation for the reporting of N-of-1 trial protocols.

Objective

The objective of this study is to evaluate current literature on N-of-1 design and reporting to identify key elements of rigorous N-of-1 protocol design.

Methods

We will conduct a systematic search for all N-of-1 trial guidelines and protocol-reporting guidelines published in peer-reviewed literature. We will search Medline, Embase, PsycINFO, CINAHL, the Cochrane Methodology Register, CENTRAL, and the NHS Economic Evaluation Database. Eligible articles will contain explicit guidance on N-of-1 protocol construction or reporting. Two reviewers will independently screen all titles and abstracts and then undertake full-text reviews of potential articles to determine eligibility. One reviewer will perform data extraction of selected articles, checked by the second reviewer. Data analysis will ascertain common features of N-of-1 trial protocols and compare them to the SPIRIT and CENT items.

Discussion

This systematic review assesses recommendations on the design and reporting of N-of-1 trial protocols. These findings will inform an international Delphi development process for an N-of-1 trial protocol reporting guideline. The development of this guideline is critical for improving the quality of N-of-1 protocols, leading to improvements in the quality of published N-of-1 trial research.

Peer Review reports

Background

N-of-1 trials are a multiple cross-over trial design within a single participant, wherein a participant receives all treatments an equal number of times with at least two tests of each treatment (e.g., ABAB) [1,2,3,4]. They have been recommended for personalizing treatment since the late 1960s [5, 6]. Though useful for situations where substantial heterogeneity of treatment effects may be an issue, N-of-1 trials are especially important when large parallel group randomized controlled trials (RCTs) are not feasible and evidence is limited, such as for evaluation of rare diseases, many pediatric conditions, and patients with co-morbid conditions or multiple concurrent treatments. When done in series, these trials can also facilitate the development of rigorous evidence in those populations [7,8,9,10,11].

Due to the design’s multiple cross-overs within a single participant, some aspects of an N-of-1 trial require particular consideration for feasibility, including: (i) the types of conditions (e.g., best if chronic and stable; if episodic, then frequently occurring) or treatments (e.g., reversible, preferably with quick onset and quick offset) that may be evaluated through the N-of-1 design; (ii) some design constituents (e.g., shorter treatment period lengths preferable, presence or absence of washout); and (iii) opportunities for participant contribution during the design process (e.g., treatment and outcome choice preferences). Methods of analysis of a series of N-of-1 trials, including meta-analysis of N-of-1 data, continue to be refined, increasing their research value [12, 13].

Careful development of a clinical trial protocol is important for researchers, ethics review boards, funders, and journals [14, 15]. Protocol reporting guidelines ensure that the specifics of the planned research trial are reported in a transparent, accurate, standardized manner [14, 16]. For reporting trial protocols, the Standard Protocol Items: Recommendations for Interventional Trials (SPIRIT) guideline [15] provides recommendations for essential elements to address in a clinical trial protocol. A SPIRIT Extension for N-of-1 Trials (SPENT) will help trialists report the important details unique to this trial design. This synthesis of published guidelines on N-of-1 trial protocol design will be the first step in the development process of SPENT [16].

This systematic review will search for both N-of-1 protocol reporting guidelines as well as clinical trial design recommendations. A preliminary exploratory search suggested no N-of-1 protocol reporting guidelines. A number of articles detail methodological recommendations for N-of-1 trials [1, 17,18,19,20,21,22,23], including the Consolidated Standards of Reporting Trials (CONSORT) Extension for N-of-1 Trials (CENT) [8]. Furthermore, this review includes psychological as well as medical literature databases to increase identification of relevant N-of-1 design guidelines.

Objectives

The purpose of this review is to systematically identify published guidelines and reporting guidelines relevant to N-of-1 trial protocols. The main goal is to identify a list of relevant N-of-1 trial protocol items that might be included in an N-of-1 protocol reporting guideline, and summarize similarities and gaps to items in the SPIRIT and CENT reporting guidelines.

Methods and design

This systematic review will be done using recommendations from Systematic Reviews by the Centre for Reviews and Dissemination [24] and is based on the SPIRIT and CENT systematic review protocols [8, 14]. It is being reported using the 2015 Preferred Reporting Items for Systematic Reviews and Meta-Analyses: Protocols (PRISMA-P) statement items [25]. This protocol is not registered with the International Prospective Register of systematic reviews (PROSPERO) because a review of protocol guidelines does not meet the inclusion criteria [26].

Criteria for document inclusion

Types of documents

Published, peer-reviewed articles written in English will be included if they describe a guideline for developing and conducting an N-of-1 trial or for reporting an N-of-1 trial protocol.

Eligibility criteria for guidelines of interest

Potential articles will be eligible for inclusion if they contain an explicit, itemized guide detailing the content or headings for designing or reporting a complete N-of-1 trial protocol [14] or aspects of a such a protocol. Due to lack of specificity, articles that simply describe a generic ABAB trial design for the purpose of reviewing or contrasting different trial designs, or that pertain to other single subject research designs, will be excluded.

Information sources and their search strategies

Three strategies will be used for the search: (1) a systematic review of the literature; (2) searching references cited in the N-of-1 articles found by the search strategy; and (3) requests for guidelines specific to N-of-1 trials will be sent to 31 large private and public research and funding organizations in North America, Europe, and Australia.

Systematic review of the literature

The following databases will be searched: Medline (Ovid interface, 1946 to Feb 2015), Embase (Ovid interface, 1974 to Feb 2015), PsycINFO (Ovid interface, 1806 to Feb 2015), CINAHL (EBSCOHost interface, 1982 to present), Cochrane Methodology register (Wiley interface, through to 2015), CENTRAL (Wiley interface, through to 2015), and the NHS Economic Evaluation Database (Wiley interface, coverage dates unstated). The search strategy for Medline can be found in Appendix 1. It will be modified as appropriate for the individual database search parameters.

This search protocol was developed in collaboration with a health research librarian, based on a previous N-of-1 trial systematic review protocol (used for the CENT guidelines [8]), the protocol used for the SPIRIT Guidelines [14], and a review of the keywords from several N-of-1 guidance documents [1, 20, 27]. The search protocol was validated for effectiveness [28] using a small set of well-established N-of-1 methodology documents [1, 18, 19, 22, 27, 29].

Data management

Screening

All search result references will be directly imported into DistillerSR®, an on-line program for systematic review data management and analysis. For Stage 1 screening, two reviewers (AP and SP) will independently scan titles and abstracts of all references to identify potentially relevant articles according to the inclusion/exclusion criteria. Articles meeting the inclusion criteria, and those where there is any uncertainty, will move to Stage 2 where full articles will be screened. The reviewers developed the screening questions for Stage 1 and 2 based on the inclusion/exclusion criteria, and tested the forms, resolving any discrepancies. For both stages, any differences of screening status between the two reviewers will be discussed, and a third referee (AO) will resolve remaining disagreements.

Data extraction

All articles selected from Stage 2 will be reviewed using a data extraction form to identify potential standard recommended N-of-1 protocol items. The form, reviewed by the team, includes all SPIRIT guideline items, CENT guideline items that are not addressed by SPIRIT, and additional items from our reference articles [1, 18, 19, 22, 29] that address key N-of-1 trial design issues not otherwise addressed in SPIRIT and CENT. Additionally, to learn more about the development of the included guidelines, we have included items from SPIRIT’s initial systematic review protocol that are specific to the development process of the guidelines. The screening form includes text boxes for additional description for N-of-1-specific items and a final open-ended text box for any additional notes on topics of relevance not in the screening list. Piloting of the data extraction form using the N-of-1 guidance documents will occur before undertaking the data extraction. The screening form is in Additional file 1; the key identifying each item’s reporting guideline source is available on request.

Because data extraction is focused on verifying if each item is present in the article, one reviewer will extract (AP), and a second (SP) will check a 15% sample from each paper for extraction accuracy. Any differences of data extraction between the two reviewers will be discussed, and a third referee (AO) will resolve remaining disagreements; if greater than 20% disagreement, full review will be considered.

Data analysis and synthesis

Extracted data will be used for descriptive statistical analysis (n, %, per item and per reporting topic headings in the data extraction form), and the median number of items per guideline. For items represented in at least 50% of the articles, frequencies of representation for each item in the SPIRIT and CENT statement items will be calculated to identify similarities, gaps, and topics not identified in SPIRIT and CENT. To identify other issues or items not addressed by the data extraction form, synthesis of any notes or comments made in the screening form will be assessed using basic qualitative descriptive analysis, in which core topics or ideas are tabulated and clustered, staying within the language and meaning of the note text [30, 31]. This analysis process will be directed by one reviewer (AP) and reviewed by a second. All protocol-specific results and extracted qualitatively-identified topics will form the starting list for the development of SPENT. This list will be assessed and refined using a sample of published and unpublished N-of-1 trial protocols, followed by an international Delphi process.

Discussion

This systematic review will find and synthesize the recommendations on the development and publishing of N-of-1 trial protocols. The results will be reported using the Preferred Reporting Items for Systematic reviews and Meta-Analyses (PRISMA) Statement [32].

Within single subject research, N-of-1 is one of many different established trial designs. The associated terminology regarding these designs is becoming standardized [8, 23]. However, in older articles, “N-of-1” may have referred to any form of research in a single person. Conversely, an actual N-of-1 trial may not have used the explicit N-of-1 designation. Therefore other older and more generic terms are included in the search strategy.

The search strategies are not optimized to capture non-English N-of-1 terminology. However, many journals indexed in PubMed provide English abstracts and MeSH or key terms. A search for “n-of-1” and “n of 1” in the non-English languages included in PubMed found only eight articles, all N-of-1 trials, from 1989 to 2014. Several of these non-English language articles only cited English N-of-1 design articles referenced in this protocol; non-English language N-of-1 guidelines were not identified. It is therefore assumed that excluding non-English language articles will not likely exclude any key relevant knowledge from this systematic review.

Some medical epistemologists argue that N-of-1 RCTs offer the most rigorous form of evidence-based medicine by combining individual patient values, randomization, and clinician expertise into a single experiment or clinical decision-making event [2, 3, 33]. They may be a useful option for addressing the personalized care recommendations of the US Patient-Centered Research Institute (PCORI) and the Canadian Strategy for Patient-Oriented Research (SPOR) [3]. Despite these advantages and calls for the routine use of N-of-1 trials in clinical practice and decision-making, N-of-1 trials remain an underused method and strategy in research and practice. The broad scope of this systematic review will help provide an appropriate basis for developing rigorous N-of-1 trial protocols, and an N-of-1 trial protocol reporting guideline (SPENT). Specifically, the results will form the basis for the Delphi process needed to assess and develop possible items in SPENT. In turn, these should strongly support improvements in the implementation of patient-centred research and the quality of published N-of-1 trial research.

Abbreviations

CENT:

CONSORT Extension for N-of-1 trials

RCT:

Randomized controlled trial

SPENT:

SPIRIT Extension for N-of-1 trials

SPIRIT:

Standard Protocol Items: Recommendations for Interventional Trials

References

  1. Kravitz RL, Duan N, DEcIDE Methods Centre N-of-1 Guidance Panel. Design and Implementation of N-of-1 Trials: A User’s Guide [Internet]. Kravitz RL, Duan N, editors. Rockville, MD, USA: Agency for Healthcare Research and Quality; 2014 Feb p. 94. Report No.: 13(14)-EHC122-EF. http://www.effectivehealthcare.ahrq.gov/N-1-Trials.cfm. Accessed 20 Sept 2015.

  2. Sackett DL. Clinician-trialist rounds: 4. why not do an N-of-1 RCT? Clin Trials. 2011;8:350–2.

    Article  PubMed  Google Scholar 

  3. Schork NJ. Personalized medicine: time for one-person trials. Nature. 2015;520:609–11.

    Article  CAS  PubMed  Google Scholar 

  4. Smith LD, Kingsmore SF. N-of-1 genomic medicine for the rare pediatric genetic diseases. Expert Opin Orphan Drugs. 2014;2:1279–90.

    Article  CAS  Google Scholar 

  5. Guyatt G, Sackett D, Taylor DW, Chong J, Roberts R, Pugsley S. Determining optimal therapy—randomized trials in individual patients. N Engl J Med. 1986;314:889–92.

    Article  CAS  PubMed  Google Scholar 

  6. Keller JL, Guyatt GH, Roberts RS, Adachi JD, Rosenbloom D. An N of 1 service: applying the scientific method in clinical practice. Scand J Gastroenterol. 2009;23:22–9.

    Article  Google Scholar 

  7. Cornu C, Kassai B, Fisch R, Chiron C, Alberti C, Guerrini R, et al. Experimental designs for small randomised clinical trials: an algorithm for choice. Orphanet J of Rare Dis. 2013;8:1–1.

    Article  Google Scholar 

  8. Vohra S, Shamseer L, Sampson M, Bukutu C, Schmid CH, Tate R, et al. CONSORT extension for reporting N-of-1 trials (CENT) 2015 Statement. BMJ. 2015;350:h1738.

    Article  PubMed  Google Scholar 

  9. Vohra S, Punja S. N-of-1 trials: individualized medication effectiveness tests. AMAJ Ethics. 2013;15:42–5.

    Google Scholar 

  10. Gagne JJ, Thompson L, O'Keefe K, Kesselheim AS. Innovative research methods for studying treatments for rare diseases: methodological review. BMJ. 2014;349:g6802–2.

    Article  PubMed  PubMed Central  Google Scholar 

  11. Shamseer L, Sampson M, Bukutu C, Schmid CH, Nikles J, Tate R, et al. CONSORT extension for reporting N-of-1 trials (CENT) 2015: explanation and elaboration. BMJ. 2015;350:h1793–3.

    Article  PubMed  Google Scholar 

  12. Nikles J, Mitchell GK, Schluter P, Good P, Hardy J, Rowett D, et al. Aggregating single patient (n-of-1) trials in populations where recruitment and retention was difficult: the case of palliative care. J Clin Epidemiol. 2011;64:471–80.

    Article  PubMed  Google Scholar 

  13. Cushing CC, Walters RW, Hoffman L. Aggregated N-of-1 randomized controlled trials: modern data analytics applied to a clinically valid method of intervention effectiveness. J Pediatr Psychol. 2014;39:138–50.

    Article  PubMed  Google Scholar 

  14. Tetzlaff JM, Chan A-W, Kitchen J, Sampson M, Tricco AC, Moher D. Guidelines for randomized clinical trial protocol content: a systematic review. Syst Rev. 2012;1:1.

    Article  Google Scholar 

  15. Chan A-W, Tetzlaff JM, Altman DG, Laupacis A, Gøtzsche PC, Krleža-Jerić K, et al. SPIRIT 2013 statement: defining standard protocol items for clinical trials. Ann Intern Med. 2013;158:200–7.

    Article  PubMed  PubMed Central  Google Scholar 

  16. Moher D, Schulz KF, Simera I, Altman DG. Guidance for developers of health research reporting guidelines. PLoS Med. 2010;7:e1000217–9.

    Article  PubMed  PubMed Central  Google Scholar 

  17. Backman CL, Harris SR. Case studies, single-subject research, and N of 1 randomized trials: comparisons and contrasts. Arch Phys Med Rehabil. 1999;78:1145–53.

    Article  Google Scholar 

  18. Duan N, Kravitz RL, Schmid CH. Single-patient (n-of-1) trials: a pragmatic clinical decision methodology for patient-centered comparative effectiveness research. J Clin Epidemiol. 2013;66:S21–8.

    Article  PubMed  PubMed Central  Google Scholar 

  19. Guyatt G, Sackett D, Adachi J, Roberts R, Chong J, Rosenbloom D, et al. A clinician's guide for conducting randomized trials in individual patients. CMAJ. 1988;139:497–503.

    CAS  PubMed  PubMed Central  Google Scholar 

  20. Guyatt HG, Heyting A, Jaeschke R, et al. N of 1 randomized trials for investigating new drugs. Control Clin Trials. 1990;11:88–100.

    Article  CAS  PubMed  Google Scholar 

  21. Kravitz RL, Duan N, Niedzinski EJ, Hay MC. What ever happened to N-of-1 trials? Insiders’ perspectives and a look to the future. Milbank Q. 2008;86:533–55.

    Article  PubMed  PubMed Central  Google Scholar 

  22. Smith JD. Single-case experimental designs: a systematic review of published research and current standards. Psychol Methods. 2012;17:510–50.

    Article  PubMed  Google Scholar 

  23. Tate RL, Perdices M, Rosenkoetter U, Wakim D, Godbee K, Togher L, et al. Revision of a method quality rating scale for single-case experimental designs and n-of-1 trials: the 15-item Risk of Bias in N-of-1 Trials (RoBiNT) Scale. Neuropsychol Rehabil. 2013;23:619–38.

    Article  PubMed  Google Scholar 

  24. Centre for Reviews and Dissemination. Systematic Reviews [Internet]. 3rd ed. York, UK: University of York; 2009. http://www.york.ac.uk/media/crd/Systematic_Reviews.pdf. Accessed 3 Oct 2015.

  25. Moher D, Shamseer L, Clarke M, Ghersi D, Liberati A, Petticrew M, et al. Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015 statement. Syst Rev. 2015;4:1–1.

    Article  PubMed  PubMed Central  Google Scholar 

  26. University of York Centre for Reviews. PROSPERO Inclusion Criteria [Internet]. http://www.crd.york.ac.uk/prospero/. [cited 2015 Jul 2]. https://www.crd.york.ac.uk/prospero/about.php?about=inclusioncriteria. Accessed 15 Mar 2016.

  27. Cook JD. Randomized trials in single subjects: the N of 1 study. Psychopharmacol Bull. 1996;32:363–67.

    CAS  PubMed  Google Scholar 

  28. Hausner E, Waffenschmidt S, Kaiser T, Simon M. Routine development of objectively derived search strategies. Syst Rev. 2012;1:19.

    Article  PubMed  PubMed Central  Google Scholar 

  29. Larson E. N-of-1 clinical trials. A technique for improving medical therapeutics. West J Med. 1990;152:52–6.

    CAS  PubMed  PubMed Central  Google Scholar 

  30. Neergaard MA, Olesen F, Andersen RS, Sondergaard J. Qualitative description—the poor cousin of health research? BMC Med Res Methodol. 2009;9:1–5.

    Article  Google Scholar 

  31. Sandelowski M. Whatever happened to qualitative description? Res Nurs Health. 2000;23:334–40.

    Article  CAS  PubMed  Google Scholar 

  32. Moher D, Liberati A, Tetzlaff J, Altman DG, Prisma Group. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. Int J Surg. 2010;8:336–41.

    Article  PubMed  Google Scholar 

  33. Berlin JA. N-of-1 clinical trials should be incorporated into clinical practice. J Clin Epidemiol. 2010;63:1283–4.

    Article  PubMed  Google Scholar 

Download references

Acknowledgements

We would like to thank Larissa Shamseer from the CENT guideline (CONSORT Extension for N-of-1 Trials) and the CARE Program Librarian Suzanne King-Jones (University of Alberta) for their assistance with development of the search strategy and review of the protocol.

Funding

Funding was provided by SV in the form of post-doctoral support for SP and AP, and for infrastructure including DistillerSR®. There is no independent sponsor or funder of this review.

Availability of data and materials

Not applicable.

Authors’ contributions

AP lead the project, lead the development of this protocol’s search strategies and drafted this manuscript. SP contributed to the development of the protocol, was second on the search implementation and results review, and helped draft the manuscript. AWC contributed to the development of the protocol and helped draft the manuscript. RC contributed to the development of the protocol and helped draft the manuscript. AO contributed to the development of the protocol and helped draft the manuscript. PR contributed to the development of the protocol and helped draft the manuscript. CS contributed to the development of the protocol and helped draft the manuscript. SV conceived of the project and helped guide the development of the protocol and drafting of the manuscript. All authors read and approved the final manuscript.

Authors’ information

SV (guarantor) has published substantially on her work in N-of-1 trials. Her academic program, CARE at the University of Alberta, Canada, has provided an N-of-1 trial service locally, and was the centre of international efforts to create the CENT guideline, on which SP also worked. AP is a Post-doctoral Fellow working for CARE on SPENT, the SPIRIT Extension for N-of-1 trials. AWC was lead on the development of SPIRIT. RK, AO, PR, and CHS have extensive experience in N-of-1 trials. Together, the authors comprise the steering group for the development of SPENT.

Competing interests

The authors declare that they have no competing interests.

Consent for publication

Not applicable.

Ethics approval and consent to participate

Not applicable.

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Sunita Vohra.

Additional file

Additional file 1:

Level 3 Data Extraction Questions. (DOC 31 kb)

Appendix 1 – Electronic search strategies

Appendix 1 – Electronic search strategies

[mp = title, abstract, original title, name of substance word, subject heading word, keyword heading word, protocol supplementary concept word, rare disease supplementary concept word, unique identifier]

MEDLINE

Medline 10 Feb 2015 skj

  1. 1.

    n-of-1.mp.

  2. 2.

    n of 1.tw.

  3. 3.

    (individual$ adj3 trial$).mp.

  4. 4.

    (individual$ adj3 test$).tw.

  5. 5.

    ((single or individual) adj (subject or patient or case) adj3 (trial$ or design)).tw.

  6. 6.

    individuali#ed medication effectiveness test$.tw.

  7. 7.

    patient$ as their own control$.tw.

  8. 8.

    abab.ti,ab.

  9. 9.

    or/1-8

  10. 10.

    Double-Blind Method/

  11. 11.

    Research Design/

  12. 12.

    Randomized Controlled Trials as Topic/

  13. 13.

    cross-over studies/

  14. 14.

    Placebos/

  15. 15.

    Guidelines as Topic/

  16. 16.

    bayes theorem/

  17. 17.

    frequentist.mp.

  18. 18.

    or/10-17

  19. 19.

    9 and 18

  20. 20.

    limit 19 to humans

  21. 21.

    limit 20 to English language

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Porcino, A.J., Punja, S., Chan, AW. et al. Protocol for a systematic review of N-of-1 trial protocol guidelines and protocol reporting guidelines. Syst Rev 6, 132 (2017). https://0-doi-org.brum.beds.ac.uk/10.1186/s13643-017-0525-4

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://0-doi-org.brum.beds.ac.uk/10.1186/s13643-017-0525-4

Keywords