Skip to main content
Download PDF

Seeking adverse effects in systematic reviews of orthodontic interventions: a cross-sectional study (part 1)

Systematic Reviews volume 12, Article number: 112 (2023) Cite this article

Abstract

Background

Systematic reviews that assess the benefits of interventions often do not completely capture all dimensions of the adverse effects. This cross-sectional study (part 1 of 2 studies) assessed whether adverse effects were sought, whether the findings on these effects were reported, and what types of adverse effects were identified in systematic reviews of orthodontic interventions.

Methods

Systematic reviews of orthodontic interventions on human patients of any health status, sex, age, and demographics, and socio-economic status, in any type of setting assessing any type of adverse effect scored at any endpoint or timing were eligible. The Cochrane Database of Systematic Reviews and 5 leading orthodontic journals were manually searched for eligible reviews between August 1 2009 and July 31 2021. Study selection and data extraction was conducted by two researchers independently. Prevalence proportions were calculated for four outcomes on seeking and reporting of adverse effects of orthodontic interventions. Univariable logistic regression models were used to determine the association between each one of these outcomes and the journal in which the systematic review was published using the eligible Cochrane reviews as reference.

Results

Ninety-eight eligible systematic reviews were identified. 35.7% (35/98) of reviews defined seeking of adverse effects as a research objective, 85.7% (84/98) sought adverse effects, 84.7% (83/98) reported findings related to adverse effects, and 90.8% (89/98) considered or discussed potential adverse effects in the review. Reviews in the journal Orthodontics and Craniofacial Research compared with Cochrane reviews had approximately 7 times the odds (OR 7.20, 95% CI 1.08 to 47.96) to define seeking of adverse effects in the research objectives. Five of the 12 categories of adverse effects accounted for 83.1% (162/195) of all adverse effects sought and reported.

Conclusions

Although the majority of included reviews sought and reported adverse effects of orthodontic interventions, end-users of these reviews should beware that these findings do not give the complete spectrum on these effects and that they could be jeopardized by the risk of non-systematically assessing and reporting of adverse effects in these reviews and in the primary studies that feed them. Much research is ahead such as developing core outcome sets on adverse effects of interventions for both primary studies and systematic reviews.

Peer Review reports

Background

To get a balanced perspective of an intervention, systematic reviewers need to report both its beneficial and adverse effects [1]. In this cross-sectional study we assessed whether adverse effects were sought, whether the findings on these effects were reported, and what types of adverse effects were identified in systematic reviews published in the Cochrane Database of Systematic reviews [2] and in 5 leading orthodontic journals.

‘Cochrane defines an adverse effect as ‘an adverse event for which the causal relation between the intervention and the event is at least a reasonable possibility’ [3, 4]. This definition and other key terms used in this manuscript are listed in Table 1 [5, 6]. A wide body of epidemiological studies has shown that adverse effects of interventions in primary research studies are often under-assessed, and/or under-reported, and/or distorted [7,8,9,10,11,12,13]. These issues can misinform anyone trying to make valid decisions on a healthcare intervention. An extension of the Consolidated Standards of Reporting Trials (CONSORT) Statement was developed to tackle poor reporting of harms in randomized trials [14]. Since the publication of this statement, the reporting of adverse events in clinical trials has improved, but is still suboptimal [10, 12, 15, 16].

Table 1 Glossary of terms
Full size table

Systematic reviews could provide even more information on adverse effects, because they assess large amounts of data from a wide spectrum of sources (possibly including both published and unpublished data). By assessing the data of multiple single studies, systematic reviewers can make a more balanced assessment of an intervention. This is an important issue, because serious adverse effects may occur rarely and might be missed in single studies. However, epidemiological research showed that the seeking and reporting of adverse effects of interventions and the methods used to identify and synthesize them [17,18,19,20,21] were also poor in systematic reviews. The Preferred Reporting Items for Systematic reviews and Meta-Analyses (PRISMA) harms checklist was published in 2016 [22] to improve harms reporting in systematic reviews, but its consequences are still largely unknown.

We performed 2 cross-sectional studies on assessing and reporting of adverse effects in systematic reviews of orthodontic interventions. In this study (part 1), we assessed whether adverse effects were sought and reported and what findings on these adverse effects were reported in systematic reviews of orthodontic interventions published in the Cochrane Database of Systematic reviews [2] and in 5 leading orthodontic journals. In a second study (part 2) we assessed the reporting on adverse effects and the presence of spin on adverse effects in the abstracts of these reviews [23]. Adverse effects of orthodontic interventions refer to for example, pain associated with orthodontic tooth movement, root resorption, decalcifications, periodontal problems, relapse, and undesired health experiences [24]. Recent (November 22 2021) scoping searches confirmed that our research objectives have not been addressed previously.

Objectives

The objectives of this research study are formulated in the following four research questions:

  1. 1)

    Was seeking of adverse effects of interventions defined as a research objective of the review?

  2. 2)

    Did the review seek any findings related to adverse effects of interventions in the included studies?

  3. 3)

    Did the review report findings related to adverse effects of interventions sought in the included studies?

  4. 4)

    Were potential adverse effects of the intervention considered, discussed (weighed) anywhere in the review?

We also assessed what adverse effects of interventions were defined as research objectives and what adverse effects of interventions were sought and reported in the review.

Methods

The Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) statement [25] and the PRISMA 2020 statement [26, 27] were consulted for reporting this cross-sectional study. The STROBE checklist of items for reporting cross-sectional studies was presented in Additional file 1. The methods for this cross-sectional study were explained in our published protocol [6] and can be consulted through the following link https://0-systematicreviewsjournal-biomedcentral-com.brum.beds.ac.uk/articles/10.1186/s13643-019-1000-1. We adopted the framework of this protocol to report the methods section of this study and its additional files. Raw data are recorded in Open Science Framework (https://osf.io/ka7mp/). Differences between methods originally planned in the protocol and those implemented in the final research study were given with the rationales for these differences in Additional file 2. No patients were involved in the development of the protocol or in the conduct of this study.

Eligibility criteria

The eligibility criteria have been published previously [6, 28] and are presented again in Table 2 [29].

Table 2 Eligibility criteria
Full size table

Information sources and search strategy

The information sources for this study were the Cochrane Database of Systematic Reviews [2] and the websites of 5 leading orthodontic journals. The selection of these 5 orthodontic journals was based on having been published at least 10 years and the highest impact factor [30]. The impact factor in 2018, i.e., the year when the protocol was developed, was used to select these journals. The 5 selected orthodontic journals are: European Journal of Orthodontics [EJO], American Journal of Orthodontics and Dentofacial Orthopedics [AJODO], Angle Orthodontist (AO), The Korean Journal of Orthodontics (KJO), and Orthodontics and Craniofacial Research (O&CR). The impact factors of these journals are listed in Additional file 2. August 1 2009 was chosen as the inception date for searching the information sources, because it coincides with the publication of the PRISMA statement and guidance document on 21 July 2009 [31, 32]. Eligible systematic reviews were manually searched in these information sources from the inception date until July 31 2021.

Study records

Data management

All study selection and data extraction procedures were conducted independently by 2 authors (PS and RMR). Pilot tests were done a priori to train and calibrate these operators [33]. Disagreements between these reviewers during these study selection and data collection were resolved in the following order: Firstly, through discussions; secondly, through rereading the article in question; and thirdly, through contacting of the authors of the pertinent manuscript by email to obtain additional information that could help with decision-making [27]. Persistent disagreements were resolved through discussions with a methodologist (SB). All eligible systematic reviews with their supplementary files were downloaded as PDFs and merged in binder files [34, 35]. Data were collected in an Excel spreadsheet [36].

Study selection and data collection procedures

Titles and abstracts were screened for eligible reviews in the websites of the 5 selected orthodontic journals. Eligible Cochrane reviews were searched in the ‘Dentistry and Oral health’ section of the Cochrane Database of Systematic Reviews [2]. When Cochrane reviews were updated, we only considered the latest published version. A PRISMA flow diagram was presented to illustrate the selection process of the eligible reviews [26, 27]. All included studies and excluded studies were reported and the rationales for exclusion were given in Additional file 3. Contacting of authors was not necessary to clarify eligibility or data extraction issues. We used our pilot tested data collection forms for the extraction of all pertinent data items. These forms are presented in Additional file 2. The entire eligible review except the abstract and protocol were searched, i.e., the main text, tables, figures, and supplemental files. This strategy was implemented for all eligible reviews. In Cochrane systematic reviews, we also did not search data items in the plain language summary.

Assigning adverse effects of orthodontic interventions

Cochrane defines an adverse effect as ‘an adverse event for which the causal relation between the intervention and the event is at least a reasonable possibility’ [3, 4]. These events can have a permanent or temporary adverse effect on the health condition of the patient. Root resorption, decalcifications of enamel or caries around orthodontic appliances are well known permanent adverse effects of orthodontic interventions, while pain and discomfort during tooth movement are generally temporary adverse effects. Events associated with orthodontic interventions that could have an adverse effect on the health condition were also labeled as adverse events, e.g., breaking of appliances, failure to complete treatment, and tolerability of orthodontic appliances.

According to our protocol we adopted the framework of known orthodontic adverse effects as reported previously by Preoteasa et al. [24] (Additional file 2) and made some changes in labeling the headings of the various categories of adverse effects (Additional file 2). A total of 12 categories of adverse effects were defined. Additional adverse effects identified during our data extraction procedures were also included in this framework and when ambiguous the rationale for including these adverse effects was given. The following types of adverse events were not labeled as adverse effects: (1) effects that do not refer to health conditions and could be ambiguous, e.g., costs, duration of treatment, number of appointments etc. (2) effects that refer to pre-existing health problems that can actually improve as a result of the intervention, e.g., respiratory problems as a result of maxillary expansion or self-esteem as a result of the retraction of protruding maxillary incisors.

Power calculation

Epitools epidemiological software was used to calculate the required sample size of eligible systematic reviews of orthodontic interventions [37]. We calculated the required sample size of 73 reviews based on the following input: estimated proportion 0.25, desired precision 0.1, and confidence level 0.95. The estimated proportion was based on the findings in our pilot tests as reported in our protocol [6]. These pilot test showed that findings related to adverse effects were sought in 3 of 12 systematic reviews on orthodontic interventions representing the estimated proportion of 0.25 (3/12).

Outcomes and statistical analyses

We reported the number of retrieved systematic reviews and eligible reviews and calculated the prevalence proportions that addressed our research questions. All outcomes were calculated as originally planned in our published protocol [6]. Prevalence proportions were calculated for: (1) all journals together (2) each journal separately and (3) the group of 5 leading orthodontic journals together and the Cochrane reviews separately. Univariable logistic regression models were built to determine the association between each one of four outcomes and the journal in which the systematic review was published, using the Cochrane Database of Systematic reviews as reference. The strength of association was quantified using odds ratios (OR), and 95% confidence intervals (95% CI). Analyses were performed with the use of commercial software (IBM SPSS 22.0, SPSS Inc, Chicago, IL). A two-sided P value of 0.05 was considered to be statistically significant.

Results

Results of the search

Through our searches in the databases of the Cochrane Database of Systematic reviews and the 5 leading orthodontic journals we identified 324 reports. One Cochrane review was excluded, because it was later updated leaving 323 reports for screening. A total of 180 papers was excluded during the title and abstract screening and 45 during full text screening. A total of 98 systematic reviews fulfilled the eligibility criteria of this study. The results of the individual selection steps are presented in a PRISMA flow diagram (Fig. 1) [26, 27]. All included studies are listed in Additional file 3 and excluded studies with the rationale for their exclusion are given in Additional file 4.

Fig. 1
figure 1

Flow diagram for the selection of systematic reviews of orthodontic interventions

Full size image

Included studies

Figure 2 presents the number of eligible systematic reviews of orthodontic interventions published during the eligible time span. Table 3 presents the number of eligible reviews for each selected journal and shows that 72.4% (71/98) of the included reviews came from the EJO, AJODO, and AO. Table 3 also gives the types of orthodontic interventions for each of these journals, which are divided in three categories. Category 1 refers to orthodontic interventions to move teeth modify jaws such as fixed orthodontic appliances or palatal expansion appliances. Category 2 refers to orthodontic interventions that also include additional surgical, pharmacological or vibrational interventions such as mini-implants, prostaglandins, piezo surgery, or vibratory stimulation. Category 3 refers to orthodontic interventions with appliances to maintain or stabilize orthodontic treatment results such as retainers. The majority of included reviews, 70.4% (69/98), assessed orthodontic interventions to move teeth or modify jaws and 28.6% (28/98) assessed orthodontic interventions with additional surgical, pharmacological or vibratory interventions.

Fig. 2
figure 2

Line chart of the number of eligible systematic reviews published between August 1 2009 and July 31 2021

Full size image
Table 3 Characteristics of included reviews
Full size table

Outcomes to the research questions

Figure 3 presents the answers to each individual research question and Table 4 gives the proportions. We reported the proportions in answering the four research questions over time in Table 5. The prevalence of reviews that defined seeking of adverse effects of interventions as a research objective was low, i.e., 35.7% in the 98 eligible reviews. Instead, the proportions that addressed the other 3 research question were 85% and higher indicating that seeking and reporting of findings related to adverse effects of interventions in the included studies and considering or discussing potential adverse effects anywhere in the review were implemented in most of the eligible reviews. As compared to the Cochrane Database of Systematic Reviews, the journal of Orthodontics and Craniofacial research had approximately 7 times the odds (OR 7.20, 95%CI 1.08 to 47.96) to report that adverse effects were sought in the research objectives. The other journals were not significantly more likely to report that adverse effects were sought in the research objectives (Table 6). For the other 3 outcomes, no statistical analysis was performed considering the low variability in the response scored (prevalence of ‘no’ ranging from 9.2 to 15.3%) and the overall small sample sizes (Table 4).

Fig. 3
figure 3

Seeking adverse effects in systematic reviews of orthodontic interventions

Full size image
Table 4 Outcomes on seeking adverse effects in systematic reviews of orthodontic interventions
Full size table
Table 5 Outcomes on seeking adverse effects in systematic reviews of orthodontic interventions by year of publication
Full size table
Table 6 Results of univariable logistic regression including journal as a predictor variable of seeking adverse effects as an objective of the systematic review
Full size table

Labeling adverse effects of orthodontic interventions

The type of adverse effects most frequently defined as research objectives were adverse effects related to (1) tooth structures, (2) periodontal tissues, (3) undesired treatment results, (4) relapse and stability, and (5) negative qualitative experiences by the patient or carer(s) (Table 7). These were also the most prevalent types of adverse effects sought in the included studies and reported in the review and accounted for 83.1% (162/195) of all adverse effects sought and reported (Table 8). We were able to categorize all 195 adverse effects except one and labeled it ‘Additional adverse effects’ (Table 8).

Table 7 Type of adverse effects defined as research objectives of the review
Full size table
Table 8 Type of adverse effects sought and reported in the review
Full size table

Discussion

Principal findings of the study

This cross-sectional study showed that in 35.7% (35/98) of reviews of orthodontic interventions seeking of adverse effects was defined as an objective. In 85.7% (84/98) of these reviews, findings related to adverse effects of interventions were sought and in 84.7% (83/98) the reviewers reported on these findings. In more than 90% (89/98) of included systematic reviews, the reviewers discussed (weighed) potential adverse effects of interventions somewhere in the review. Five types of adverse effects accounted for 83.1% (162/195) of adverse effects that were sought and reported in the eligible reviews.

Comparisons with other studies

The proportion of included reviews that defined seeking of adverse effects as a research objective was low, i.e., 35.7% (35/98) in both Cochrane and non-Cochrane systematic reviews (Table 4). Assessing potential adverse effects of interventions is considered a mandatory item when setting the research question for Cochrane intervention reviews [1]. Not defining seeking of adverse effects as a research objective can mislead end-users of systematic reviews. Authors therefore need to include this item in their research objectives and editors and peer reviewers should verify its implementation.

The proportions of reviews that reported findings related to adverse effects of interventions were higher in this sample of orthodontic reviews (84.7% (83/98) compared with gastroenterology reviews (66.7% (52/78) [18], Cochrane reviews of interventions (75.6% (59/78), and Database of Abstracts of Reviews of Effects (DAREs) reviews (48.1% (38/79) [38]. Explanations for these higher proportions could be: (1) the time period of inclusion of reviews (2) the research design and type of interventions of the studies included in the reviews (3) the field of research. Orthodontic research could be more focused on assessing adverse effects of interventions than other fields, because this assessment is an integral part of routine clinical practice. For example, assessing adverse effects such as undesired treatment results and relapse and stability are part of everyday problems in orthodontic practice and accounted for 40.5% (79/195) of adverse effects sought and reported in this sample of systematic reviews of orthodontic interventions (Table 8).

Strengths and limitations

This cross-sectional study has the following strengths: (1) scoping searches were conducted to identify knowledge gaps, (2) pilot studies were conducted to calibrate researchers and fine-tune research questions and methodology, (3) a protocol was developed and published a priori [6], and (4) all raw data were included with this manuscript or recorded in Open Science Framework (https://osf.io/ka7mp/). This study also has limitations. First, the findings of this cross-sectional study are expected to be better than those reported in the entire body of orthodontic literature, because we assessed reviews published in the five leading orthodontic journals and those listed in the Cochrane Database of Systematic Reviews. Second, the risk of selective (non) reporting bias regarding adverse effects in the eligible reviews. Third, only reviews published in a pre-established period (August 1 2009 until July 31 2021) were eligible, instead of having considered a larger sample, e.g., by having included reviews prior to the inception date. However, we chose this inception date, because it coincides with the launch of the PRISMA statement [31, 32], which provides reviewers better guidance on reporting.

Implications and future research

Several of our findings seem promising at a first glance. For example, the proportion of reviews that sought and reported adverse effects was relatively high, i.e., (84.7% (83/98), but a variety of issues has to be considered when interpreting this finding. First, this proportion only refers to whether or not reviewers implemented this item, but not how. For example, the reviewers could have reported on just one or a selection of all adverse effects assessed and reported in the eligible studies for their reviews. Second, this proportion also does not give any information on the magnitude, and duration of adverse effects nor on the time points for assessing them. Third, we do not know whether all adverse effects were indeed sought and reported as originally planned in the registered protocols of the included reviews. For example, Parsons et al. [39] showed that this was not the case in their sample of systematic reviews of health care interventions. In 35% (51/146) of these reviews they found discrepancies between what was planned in the protocol as registered in PROSPERO and what was reported on adverse effects in the final published reviews. Fourth, a wide body of evidence has shown that adverse events were often assessed inconsistently and reported inadequately in clinical trials and that most results on these events were not available in public sources [8, 40,41,42]. If these limitations also apply to the clinical trials that fed the reviews of this study one should further question the validity of the findings on adverse effect of systematic reviews of orthodontic interventions.

Strategies to improve the validity of what is reported on adverse effects of orthodontic interventions in systematic reviews include developing tailored core outcome sets on these effects [43] as well as guidelines for assessing and reporting them in both primary research and systematic reviews. Additional strategies on synthesizing adverse effects in systematic reviews at multiple levels were published in a recent paper by Qureshi et al. [19]. By implementing such strategies progress on the assessing and reporting of adverse effects of orthodontic interventions in both primary studies and systematic reviews can be made.

In conclusion the promising findings of this study should be interpreted with caution by its end users, because they could be jeopardized by numerous uncertainties. Much research is ahead to create valid and usable knowledge on adverse effects of orthodontic interventions involving a wide body of stakeholders.

Availability of data and materials

Not applicable.

Abbreviations

AJODO:

American Journal of Orthodontics and Dentofacial Orthopedics

AO:

Angle Orthodontist

CONSORT:

Consolidated Standards of Reporting Trials

EJO:

European Journal of Orthodontics

KJO:

Korean Journal of Orthodontics

O&CR:

Orthodontics and Craniofacial Research

PRISMA:

Preferred reporting items for systematic reviews and meta-analyses

PROSPERO:

Prospective Register of Systematic Reviews

STROBE:

Strengthening the Reporting of Observational Studies in Epidemiology

References

  1. Higgins JPT, Lasserson T, Chandler J, Tovey D, Thomas, J, Flemyng E, et al. Methodological expectations of cochrane intervention reviews. Cochrane: London. Version February 2022. Available from: https://community.cochrane.org/mecir-manual. Accessed 27 June 2023.

  2. The Cochrane Database of Systematic Reviews (CDSR). Available from: https://www.cochranelibrary.com/cdsr/about-cdsr. Accessed 4 Dec 2021.

  3. Glossary of terms in the Cochrane Collaboration. Version 4.2.5. Updated May 2005.  Available from: http://aaz.hr/resources/pages/57/7.%20Cochrane%20glossary.pdf. Accessed 4 Dec 2021.

  4. Peryer G, Golder S, Junqueira D, Vohra S, Loke YK. Chapter 19: Adverse effects. In: Higgins JPT, Thomas J, Chandler J, Cumpston M, Li T, Page MJ, Welch VA, editors. Cochrane handbook for systematic reviews of Interventions version 6.2 (updated February 2021). Cochrane; 2021. Available from www.training.cochrane.org/handbook.

  5. Cochrane library. Available from: https://www.cochranelibrary.com/about/about-cochrane-reviews. Accessed 27 June 2023.

  6. Steegmans PAJ, Bipat S, Meursinge Reynders RA. Seeking adverse effects in systematic reviews of orthodontic interventions: protocol for a cross-sectional study. Syst Rev. 2019;8(1):89. https://0-doi-org.brum.beds.ac.uk/10.1186/s13643-019-1000-1. PMID: 30953538; PMCID: PMC6449933.

    Article  PubMed  PubMed Central  Google Scholar 

  7. Bagul NB, Kirkham JJ. The reporting of harms in randomized controlled trials of hypertension using the CONSORT criteria for harm reporting. Clin Exp Hypertens. 2012;34(8):548–54. https://0-doi-org.brum.beds.ac.uk/10.3109/10641963.2012.681724. Epub 2012 May 9.

    Article  PubMed  Google Scholar 

  8. Golder S, Loke YK, Wright K, Norman G. Reporting of adverse events in published and unpublished studies of health care interventions: a systematic review. PLoS Med. 2016;13(9):e1002127. https://0-doi-org.brum.beds.ac.uk/10.1371/journal.pmed.1002127. eCollection 2016 Sep.

    Article  PubMed  PubMed Central  Google Scholar 

  9. Hodkinson A, Kirkham JJ, Tudur-Smith C, Gamble C. Reporting of harms data in RCTs: a systematic review of empirical assessments against the CONSORT harms extension. BMJ Open. 2013;3(9):e003436. https://0-doi-org.brum.beds.ac.uk/10.1136/bmjopen-2013-003436.

    Article  PubMed  PubMed Central  Google Scholar 

  10. Péron J, Maillet D, Gan HK, Chen EX, You B. Adherence to CONSORT adverse event reporting guidelines in randomized clinical trials evaluating systemic cancer therapy: a systematic review. J Clin Oncol. 2013;31(31):3957–63. https://0-doi-org.brum.beds.ac.uk/10.1200/JCO.2013.49.3981. Epub 2013 Sep 23.

    Article  PubMed  Google Scholar 

  11. Pradhan R, Singh S. Comparison of data on serious adverse events and mortality in ClinicalTrials.gov, corresponding journal articles, and FDA medical reviews: cross-sectional analysis. Drug Saf. 2018;41(9):849–57. https://0-doi-org.brum.beds.ac.uk/10.1007/s40264-018-0666-y. PMID: 29644579.

    Article  CAS  PubMed  Google Scholar 

  12. Smith SM, Chang RD, Pereira A, Shah N, Gilron I, Katz NP, Lin AH, McDermott MP, Rappaport BA, Rowbotham MC, Sampaio C, Turk DC, Dworkin RH. Adherence to CONSORT harms-reporting recommendations in publications of recent analgesic clinical trials: an ACTTION systematic review. Pain. 2012;153(12):2415–21. https://0-doi-org.brum.beds.ac.uk/10.1016/j.pain.2012.08.009. Epub 2012 Sep 15.

    Article  PubMed  Google Scholar 

  13. Tang E, Ravaud P, Riveros C, Perrodeau E, Dechartres A. Comparison of serious adverse events posted at ClinicalTrials.gov and published in corresponding journal articles. BMC Med. 2015;14(13):189. https://0-doi-org.brum.beds.ac.uk/10.1186/s12916-015-0430-4. PMID: 26269118; PMCID: PMC4535304.

    Article  Google Scholar 

  14. Ioannidis JP, Evans SJ, Gøtzsche PC, O’Neill RT, Altman DG, Schulz K, Moher D, CONSORT Group. Better reporting of harms in randomized trials: an extension of the CONSORT statement. Ann Intern Med. 2004;141(10):781–8.

    Article  PubMed  Google Scholar 

  15. Haidich AB, Birtsou C, Dardavessis T, Tirodimos I, Arvanitidou M. The quality of safety reporting in trials is still suboptimal: survey of major general medical journals. J Clin Epidemiol. 2011;64(2):124–35. https://0-doi-org.brum.beds.ac.uk/10.1016/j.jclinepi.2010.03.005. Epub 2010 Jun 17.

    Article  PubMed  Google Scholar 

  16. Moskowitz A, Andersen LW, Holmberg MJ, Grossestreuer AV, Berg KM, Granfeldt A. Identification, collection, and reporting of harms among non-industry-sponsored randomized clinical trials of pharmacologic interventions in the critically ill population: a systematic review. Crit Care. 2020;24(1):398. https://0-doi-org.brum.beds.ac.uk/10.1186/s13054-020-03113-z. PMID:32641148;PMCID:PMC7346341.

    Article  PubMed  PubMed Central  Google Scholar 

  17. Li L, Xu C, Deng K, Zhou X, Liu Z, Busse JW, Ren Y, Zou K, Sun X. The reporting of safety among drug systematic reviews was poor before the implementation of the PRISMA harms checklist. J Clin Epidemiol. 2019;105:125–35. https://0-doi-org.brum.beds.ac.uk/10.1016/j.jclinepi.2018.09.014. Epub 2018 Sep 29 PMID: 30278212.

    Article  PubMed  Google Scholar 

  18. Mahady SE, Schlub T, Bero L, Moher D, Tovey D, George J, Craig JC. Side effects are incompletely reported among systematic reviews in gastroenterology. J Clin Epidemiol. 2015;68(2):144–53.

    Article  PubMed  Google Scholar 

  19. Qureshi R, Mayo-Wilson E, Rittiphairoj T, McAdams-DeMarco M, Guallar E, Li T. Summaries of harms in systematic reviews are unreliable Paper 2: Methods used to assess harms are neglected in systematic reviews of gabapentin. J Clin Epidemiol. 2021. https://0-doi-org.brum.beds.ac.uk/10.1016/j.jclinepi.2021.10.024.

    Article  PubMed  PubMed Central  Google Scholar 

  20. Qureshi R, Mayo-Wilson E, Rittiphairoj T, McAdams-DeMarco M, Guallar E, Li T. Summaries of harms in systematic reviews are unreliable Paper 3: Given the same data sources, systematic reviews of gabapentin have different results for harms. J Clin Epidemiol. 2021. https://0-doi-org.brum.beds.ac.uk/10.1016/j.jclinepi.2021.10.025.

    Article  PubMed  PubMed Central  Google Scholar 

  21. Zorzela L, Golder S, Liu Y, Pilkington K, Hartling L, Joffe A, Loke Y, Vohra S. Quality of reporting in systematic reviews of adverse events: systematic review. BMJ. 2014;8(348):f7668. https://0-doi-org.brum.beds.ac.uk/10.1136/bmj.f7668. PMID: 24401468; PMCID: PMC3898583.

    Article  Google Scholar 

  22. Zorzela L, Loke YK, Ioannidis JP, Golder S, Santaguida P, Altman DG, Moher D, Vohra S, PRISMA Harms Group. PRISMA harms checklist: improving harms reporting in systematic reviews. BMJ. 2016;1(352):i157. https://0-doi-org.brum.beds.ac.uk/10.1136/bmj.i157.

    Article  Google Scholar 

  23. Steegmans PAJ, Di Girolamo N, Meursinge Reynders RA. Spin on adverse effects in abstracts of systematic reviews of orthodontic interventions: a cross-sectional study (part 2). Syst Rev. 2023;12(1):99. https://0-doi-org.brum.beds.ac.uk/10.1186/s13643-023-02269-3.

  24. Preoteasa CT, Ionescu E, Preoteasa E. Chapter 18: Risks and complications associated with orthodontic treatment. In: Bourzgui F, editor. Orthodontics-basic aspects and clinical considerations. March 9, 2012 under CC BY 3.0 license. Available from: www.intechopen.com.  https://cdn.intechopen.com/pdfs/31388/InTech-Risks_and_complicationsassociated_with_orthodontic_treatment.pdf. Accessed 4 Dec 2021.

  25. von Elm E, Altman DG, Egger M, Pocock SJ, Gøtzsche PC, Vandenbroucke JP, STROBE Initiative. The Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) statement: guidelines for reporting observational studies. Ann Intern Med. 2007;147(8):573–7.

    Article  Google Scholar 

  26. Page MJ, McKenzie JE, Bossuyt PM, Boutron I, Hoffmann TC, Mulrow CD, Shamseer L, Tetzlaff JM, Akl EA, Brennan SE, Chou R, Glanville J, Grimshaw JM, Hróbjartsson A, Lalu MM, Li T, Loder EW, Mayo-Wilson E, McDonald S, McGuinness LA, Stewart LA, Thomas J, Tricco AC, Welch VA, Whiting P, Moher D. The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. BMJ. 2021;29(372):n71. https://0-doi-org.brum.beds.ac.uk/10.1136/bmj.n71. PMID: 33782057; PMCID: PMC8005924.

    Article  Google Scholar 

  27. Page MJ, Moher D, Bossuyt PM, Boutron I, Hoffmann TC, Mulrow CD, Shamseer L, Tetzlaff JM, Akl EA, Brennan SE, Chou R, Glanville J, Grimshaw JM, Hróbjartsson A, Lalu MM, Li T, Loder EW, Mayo-Wilson E, McDonald S, McGuinness LA, Stewart LA, Thomas J, Tricco AC, Welch VA, Whiting P, McKenzie JE. PRISMA 2020 explanation and elaboration: updated guidance and exemplars for reporting systematic reviews. BMJ. 2021;29(372):n160. https://0-doi-org.brum.beds.ac.uk/10.1136/bmj.n160. PMID: 33781993.

    Article  Google Scholar 

  28. Steegmans PAJ, Di Girolamo N, Meursinge Reynders RA. Spin in the reporting, interpretation, and extrapolation of adverse effects of orthodontic interventions: protocol for a cross-sectional study of systematic reviews. Res Integr Peer Rev. 2019;4:27. https://0-doi-org.brum.beds.ac.uk/10.1186/s41073-019-0084-4.

    Article  PubMed  PubMed Central  Google Scholar 

  29. Lefebvre C, Glanville J, Briscoe S, Littlewood A, Marshall C, Metzendorf M-I, Noel-Storr A, Rader T, Shokraneh F, Thomas J, Wieland LS. Chapter 4: searching for and selecting studies. In: Higgins JPT, Thomas J, Chandler J, Cumpston M, Li T, Page MJ, Welch VA, editors. Cochrane handbook for systematic reviews of Interventions version 6.2 (updated February 2021). Cochrane; 2021. Available from www.training.cochrane.org/handbook.

  30. Clarivate Analytics. Available from: https://clarivate.com/. Accessed 4 Dec 2021.

  31. Liberati A, Altman DG, Tetzlaff J, et al. The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate health care interventions: explanation and elaboration. PLoS Med. 2009;6:e1000100 Epub 2009 Jul 21.

    Article  PubMed  PubMed Central  Google Scholar 

  32. Moher D, Liberati A, Tetzlaff J, Altman DG, PRISMA Group. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. PLoS Med. 2009;6:e1000097.

    Article  PubMed  PubMed Central  Google Scholar 

  33. Shamseer L, Moher D, Clarke M, Ghersi D, Liberati A, Petticrew M, Shekelle P, Stewart LA. the PRISMA-P Group: Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015: elaboration and explanation. BMJ. 2015;349:g7647.

    Article  Google Scholar 

  34. Adobe Merge PDFs, combine files into one PDF. Available from: http://www.wikihow.com/Merge-PDF-Files. Accessed 4 Dec 2021.

  35. Acrobat X action: find & highlight words & phrases. Available from: https://acrobatusers.com/assets/uploads/actions/Find_and_Highlight_Words_and_Phrases.pdf . Accessed 4 Dec 2021.

  36. Microsoft Excel. Available from: https://products.office.com/it-IT/excel. Accessed 4 Dec 2021.

  37. Sergeant ESG. Sergeant, ESG, 2018. Epitools epidemiological calculators. Ausvet. Available from: http://epitools.ausvet.com.au. Accessed 4 Dec 2021.

  38. Hopewell S, Wolfenden L, Clarke M. Reporting of adverse events in systematic reviews can be improved: survey results. J Clin Epidemiol. 2008;61(6):597–602. https://0-doi-org.brum.beds.ac.uk/10.1016/j.jclinepi.2007.10.005. Epub 2008 Apr 14 PMID: 18411039.

    Article  PubMed  Google Scholar 

  39. Parsons R, Golder S, Watt I. More than one-third of systematic reviews did not fully report the adverse events outcome. J Clin Epidemiol. 2019;108:95–101. https://0-doi-org.brum.beds.ac.uk/10.1016/j.jclinepi.2018.12.007. Epub 2018 Dec 13 PMID: 30553831.

    Article  PubMed  Google Scholar 

  40. Golder S, Loke YK, Wright K, Sterrantino C. Most systematic reviews of adverse effects did not include unpublished data. J Clin Epidemiol. 2016;77:125–33. https://0-doi-org.brum.beds.ac.uk/10.1016/j.jclinepi.2016.05.003. Epub 2016 Jun 1 PMID: 27259470.

    Article  PubMed  Google Scholar 

  41. Mayo-Wilson E, Fusco N, Li T, Hong H, Canner JK, Dickersin K, MUDS investigators. Harms are assessed inconsistently and reported inadequately part 1: systematic adverse events. J Clin Epidemiol. 2019;113:20–7. https://0-doi-org.brum.beds.ac.uk/10.1016/j.jclinepi.2019.04.022. Epub 2019 May 2 PMID: 31055175.

    Article  PubMed  Google Scholar 

  42. Mayo-Wilson E, Fusco N, Li T, Hong H, Canner JK, Dickersin K, MUDS investigators. Harms are assessed inconsistently and reported inadequately Part 2: nonsystematic adverse events. J Clin Epidemiol. 2019;113:11–9. https://0-doi-org.brum.beds.ac.uk/10.1016/j.jclinepi.2019.04.020. Epub 2019 May 2 PMID: 31055176.

    Article  PubMed  Google Scholar 

  43. Tsichlaki A, O’Brien K, Benson PE, Marshman Z, Johal A, Colonio-Salazar FB, Harman NL, Fleming PS. Development of a core outcome set for use in routine orthodontic clinical trials. Am J Orthod Dentofacial Orthop. 2020;158(5):650–60. https://0-doi-org.brum.beds.ac.uk/10.1016/j.ajodo.2020.05.010. Epub 2020 Sep 16 PMID: 32950336.

    Article  PubMed  Google Scholar 

Download references

Acknowledgements

Not applicable.

Funding

All expenses for preparing and conducting of the submitted research study were paid evenly by each author.

Author information

Authors and Affiliations

  1. Department of Orthodontics, Academisch Centrum Tandheelkunde Amsterdam (ACTA), University of Amsterdam, Gustav Mahlerlaan 3004, Amsterdam, 1081 LA, The Netherlands

    Pauline A. J. Steegmans

  2. Department of Clinical Sciences, College of Veterinary Medicine, Cornell University, Ithaca, NY, USA

    Nicola Di Girolamo

  3. EBMVet, Via Sigismondo Trecchi 20, Cremona, CR, 26100, Italy

    Nicola Di Girolamo

  4. Department of Radiology and Nuclear Medicine, Amsterdam University Medical Center (Amsterdam UMC), Location AMC, Meibergdreef 9, Amsterdam, 1105 AZ, The Netherlands

    Shandra Bipat

  5. Department of Oral and Maxillofacial Surgery, Amsterdam University Medical Center (Amsterdam UMC), Location AMC, Meibergdreef 9, Amsterdam, 1105 AZ, The Netherlands

    Reint A. Meursinge Reynders

  6. Studio Di Ortodonzia, Via Matteo Bandello 15, Milan, 20123, Italy

    Reint A. Meursinge Reynders

Authors
  1. Pauline A. J. Steegmans
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Nicola Di Girolamo
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Shandra Bipat
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Reint A. Meursinge Reynders
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

PS and RMR conceived and designed this cross-sectional study and RMR is the guarantor. PS and RMR conducted the study selection procedures and data extraction. PS, RMR, and NDG conducted the data analyses and statistical analyses. SB provided support on methodological issues and assisted in the overall fine-tuning of this cross-sectional study. All four authors revised the various draft versions of this paper and read and approved the final submitted manuscript.

Corresponding author

Correspondence to Reint A. Meursinge Reynders.

Ethics declarations

Ethics approval and consent to participate

Not applicable.

Consent for publication

Not applicable.

Competing interests

Reint Meursinge Reynders and Shandra Bipat are both Associate Editors for Systematic Reviews. All four authors declare that they have no further competing interests.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Additional file 1.

STROBE Statement: Checklist of items that should be included in reports of cross-sectional studies [25].

Additional file 2.

 A. Differences between the protocol and the completed cross-sectional study. B. Selected journals and their 2018 impact factor (Clarivate Analytics 2021). C. Data collection forms*. D. Adverse effects hypothetically linked to orthodontic interventions according to Preoteasa et al. [24]*. E. Adverse effects hypothetically linked to orthodontic interventions*.

Additional file 3.

Included reviews.

Additional file 4.

Excluded studies.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Steegmans, P.A.J., Di Girolamo, N., Bipat, S. et al. Seeking adverse effects in systematic reviews of orthodontic interventions: a cross-sectional study (part 1). Syst Rev 12, 112 (2023). https://0-doi-org.brum.beds.ac.uk/10.1186/s13643-023-02273-7

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://0-doi-org.brum.beds.ac.uk/10.1186/s13643-023-02273-7

Keywords

Systematic Reviews

ISSN: 2046-4053