Skip to main content
Download PDF

Citation of updated and co-published Cochrane Methodology Reviews

Systematic Reviews volume 12, Article number: 120 (2023) Cite this article

Abstract

Background

To evaluate the number of citations for Cochrane Methodology Reviews after they have been updated or co-published in another journal, and the effect of co-publishing the review on the co-publishing journal’s impact factor (IF).

Methods

We identified all Cochrane Methodology Reviews published in the Cochrane Database of Systematic Reviews (CDSR) before 2018 and searched for co-published versions in the Web of Science Core Collection database up to 16 August 2022. The included reviews were in two cohorts: those that had been published and updated in CDSR and those that had been published in CDSR and co-published in another journal. The primary outcome measured the citation number to updated and original reviews in the first five years after publication of the updated review, and assessed the citation number of co-published and non-co-published reviews in the first five years after publication of the co-published version.

The secondary outcome was the ratio of an adjusted IF and the actual IF of the co-publishing journal.

Results

Eight updated and six original reviews were identified for the updated cohort of reviews, and four co-published reviews were included in the co-published cohort. The original reviews continued to be cited after the update was published but the median for the total number of citations was non-significantly higher for the updated reviews than for their original version[161 (Interquartile range (IQR) 85, 198) versus 113 (IQR 15, 433)]. The median number of total citations [362 (IQR 179, 840) versus 145 (IQR 75, 445)] and the median number of citations to the review in the first five years after co-publication combined and in each of those years was higher in the co-published group than in the non-co-published group. One of the three journals that co-published Reviews in the first year and two journals in the second year had a lower IF after co-publication.

Conclusions

Earlier versions of Cochrane Methodology Reviews continue to be cited after an update is published, which raises doubts about whether those citing are using the most recent evidence or are aware of the update. Co-publication facilitates broader application and dissemination of Cochrane methodology evidence.

Peer Review reports

Background

Systematic reviews can facilitate decision-making by relevant professionals, policy makers and the public and they need to be kept up to date if new studies would change their conclusions. Reviews that are out of date might provide misleading information [1] due to the absence of the latest evidence, and cause cascading damage to the quality of research [2]. Since its inception, the Cochrane Collaboration (now, Cochrane) has been dedicated to the periodic updating of Cochrane Reviews [1] with an update defined as “a new edition of a published Cochrane review with changes that can include new data, new methods, or new analyses to the previous edition” [3]. These updates include an updated search for eligible studies [4] and are marked to indicate whether or not the updating led to a change in the review’s conclusions [4]. An update of a Cochrane Review can provide users with the latest outcomes or conclusions from the evidence and, even without any changes in the results or conclusions, the update can reassure users that no recent evidence is missing [1]. However, the original Cochrane Review might continue to be cited and used after the update is published. Bodil et al. found that twenty-five percent of Cochrane reviews were still cited 10 years after last update and were on average cited 4.3 times in the 10th year [5].

Cochrane allows co-publication of Cochrane Reviews in other journals in certain circumstances [6]. Co-publication is not a duplicate publication or academic misconduct, but rather the co-published version is a secondary publication of the Cochrane Review [7], and should be peer reviewed and edited according to the co-publishing journal’s editorial process. Co-publication of Cochrane Reviews should be carried out with the agreement of Cochrane and the co-publishing journals [8], and might take the form of be an abridged version of the full review, a “Cochrane corner” of a journal (where the summary of the review is accompanied by commentary), or a short version translated into a language other than English [6].

In an earlier study, the annual co-publication rate for Cochrane Reviews fluctuated from 0.96% to 3.94% [9] between 2005 and 2015. It has also been shown to vary across different Cochrane groups. For example, 19.6% of reviews from the Cochrane Eyes and Vision Group were co-published in 2007, and 16.2% in 2014 [10, 11]. Co-publication of Cochrane Reviews in other journals might promote access to the evidence and increase citations [10, 12] and journals that co-publish Cochrane Reviews might also increase their impact factor (IF) [12]. Our previous study [9] found that the total number of citations for co-published Cochrane Reviews (combining the citations for the original Cochrane Review and the co-published version) was significantly higher than that for non-co-published reviews. This showed that co-publication not only improved the dissemination and accessibility of Cochrane evidence but also facilitated its uptake.

In the study reported here, we investigated the effect of updating and of co-publication on the number of citations to Cochrane Methodology Reviews. Most Cochrane Reviews relate to the effects of health and social care, but Cochrane Methodology Reviews are a subset focused on research into the methods used in research into health and social care. We have compared citations to the updated and the original Cochrane Methodology Review in the years after the update was published, the citations for co-published and non-co-published Cochrane Methodology Reviews and the effect of co-publishing the review on the co-publishing journal’s IF.

Methods

Study design

This was a retrospective study of Cochrane Methodology Reviews. In August 2022, we identified all Cochrane Methodology Reviews in the Cochrane Database of Systematic Reviews (CDSR) in the Cochrane Library (https://www.cochranelibrary.com/) which had been published before 31 December 2018. We searched for co-published versions of these reviews in another journal in the Web of Science Core Collection database up to 16 August 2022. The number of citations for each version of the Cochrane Methodology Review and any co-published versions was also obtained from the Web of Science Core Collection database on 16 August 2022.

The included reviews were divided into two cohorts: (1) those that had been updated before 2018 and (2) those that had been co-published in another journal. For the updated cohort, the most recent versions or previous version of the updated Cochrane Methodology Reviews were the updated group and the original review were original group. If a review is updated multiple times, each updated version is included in the updated group. For the co-published cohort, reviews which were co-published in another journal at the same time or later than the original review were the co-publication group, and the Cochrane Methodology Reviews had not been co-published were non-co-publication group.

Inclusion and exclusion criteria

Inclusion criteria were (1) Cochrane Methodology Reviews that had been updated before 2018; or (2) Cochrane Methodology Reviews that had been published before 2018 and co-published in another journal at the same time or after the publication of the Cochrane Methodology Review in the CDSR. Exclusion criteria were (1) Cochrane Methodology Reviews that had not been updated or co-published before 2018; and (2) Cochrane Reviews or updated versions or co-published versions for which the number of citations could not obtained. Although there are examples of co-publication of Cochrane Methodology Reviews after 2018 [13, 14], we restricted this study to reviews published before 2018 in order to be able to investigate the trajectory of citations in the five years after updating or co-publication.

Review selection and data extraction

Two authors (LLZ and ZYY) reviewed and extracted data independently. Disagreements were resolved by consensus, with arbitration by a third author (HYD) if necessary. A standardized extraction form was used to collect the following data for eligible reviews: title, number of included studies, results and conclusions, authorship of CDSR and co-published versions, journal of co-publication, time interval between original and updated or co-published review. The number of citations to the Cochrane Methodology Reviews and to the co-published versions, and the IF of the co-publishing journal were taken from the Web of Science database (Journal Citation Report).

Outcomes

The primary outcomes were the average (mean and median) number of citations to the updated and the original Cochrane Methodology Reviews in the five years after the update had been published and the average number of citations to the co-published and non-co-published Cochrane Methodology Reviews in the first five years after co-publication. The secondary outcome was the ratio of an adjusted IF (excluding the data for the co-published Cochrane Methodology Review) and the actual IF of the co-publishing journal.

We calculated the number of citations for each co-published review as the sum of the citations to the Cochrane Methodology Review and to its co-published version. The journal’s actual IF is calculated by:

$$\mathrm{IF}=\frac{\mathrm{Citations\ in\ year\ {\rm X}}\mathrm{\ to\ items\ published\ in\ year\ }\left(\mathrm{\rm X}-2\right)\ +\ \mathrm{Citations\ in\ year\ {\rm X}}\mathrm{\ to\ items\ published\ in\ year\ }\left(\mathrm{\rm X}-1\right)}{\mathrm{Number\ of\ citable\ items\ in\ year\ }\left(\mathrm{\rm X}-2\right)\ +\ \mathrm{Number\ of\ citable\ items\ in\ year}\ \left(\mathrm{\rm X}-1\right)}$$

Our adjusted IF was calculated by:

$$\mathrm{Adjusted\ IF}=\frac{\mathrm{Citations\ in\ year\ {\rm X}}\mathrm{\ to\ items\ published\ in\ year }\left(\mathrm{\rm X}-2\right)\ +\ \mathrm{A }}{\mathrm{Number\ of\ citable\ items\ in\ year\ }\left(\mathrm{\rm X}-2\right)\ +\ \mathrm{B}}$$
$$\mathrm{\rm A}=\left[\mathrm{Citations\ in\ year\ {\rm X}}\mathrm{\ to\ items\ published\ in\ year\ }\left(\mathrm{\rm X}-1\right)\right]-[\mathrm{Citations\ in\ year\ {\rm X}}\mathrm{\ to\ Cochrane\ review\ copublications\ in\ year\ }\left(\mathrm{\rm X}-1\right)]$$
$$\mathrm{\rm B}=\left[\mathrm{Number\ of\ citable\ items\ in\ year}\left(\mathrm{\rm X}-1\right)\right]-\mathrm{\ Number\ of\ Cochrane\ review\ copublications}$$

We then calculated a ratio by dividing the adjusted IF by the journal’s actual IF. If this ratio was greater than or equal to 1, the co-published reviews did not increase the journal IF; while values less than 1 indicated that the co-published reviews increased the journal IF. When more than one co-publication of a Cochrane Methodology Review was identified in a single journal, we calculated the mean ratio of IF and its 95% confidence interval (CI).

Statistical snalysis

We used the Fisher’s exact test to compare categorical variables, the Student t-test for continuous and normally distributed variables and the Mann–Whitney U-test to compare medians if the data were not normally distributed. We conducted a normality test on the data using a histogram and a normal P-P plot. A p-value of < 0.05 was considered statistically significant. We used SPSS 26.0 software for the statistical analyses.

Results

Baseline characteristics of Cochrane reviews

There were 38 full Cochrane Methodology Reviews in CDSR on 16 August 2022, which had been updated between zero and five times by that date. Six of these reviews [15,16,17,18,19,20] had been updated [21,22,23,24,25,26] before 2018. We also retrieved four versions of Cochrane Methodology Reviews (two original reviews [17, 27] and two updated reviews [23, 28]) that had been co-published in another journal [29,30,31,32] before 2018 from the Web of Science Core Collection database on 16 August 2022. In total, our sample comprises 14 published versions of Cochrane Methodology Reviews and four co-published articles. The flow diagram is shown in Fig. 1.

Fig. 1
figure 1

The flow diagram of included Cochrane Methodology Reviews

Full size image

We categorized the included records into two cohorts as described above. In the updated cohort, the updated group included 8 updated reviews, while the original group included 6 original Cochrane Methodology Reviews. The co-published cohort consisted of 4 co-published and 11 non-co-published Cochrane Methodology Reviews. The included reviews had been published between 2007 and 2018. The main characteristics of the two cohorts are shown in Tables 1 and 2.

Table 1 Characteristics of the updated and original Cochrane Methodology Reviews
Full size table
Table 2 Characteristics of the co-published and non-co-published Cochrane Methodology Reviews
Full size table

Characteristics of updated and co-published Cochrane review

Topics addressed in the eight updated Cochrane Methodology Reviews included issues relating to randomization and recruitment to randomized trials (4 reviews) [25, 26, 28, 33], conflict of interest (1 review) [23], technical editing (1 review) [21], publication (1 review) [22] and questionnaire response (1 review) [24]. Five (62.5%) had been updated only once before 2018 [21,22,23,24,25], while the other had been updated three times [26, 28, 33]. The interval time between versions of the reviews varied from 0.5 year to 11 years. Characteristics are summarized in Table 3.

Table 3 Characteristics of eight updated Cochrane Methodology Reviews
Full size table

The four co-published Cochrane Methodology Reviews were co-published in three journals. Three (75%) were co-published more than two years after the publication of the Cochrane Methodology Review. The types of co-publication were mostly a short version of the full review (3 reviews), with one being a specially prepared summary of the review. One Cochrane Methodology Review had a co-publication for both its original and updated version. The other two co-publications were for the original version of the Cochrane Methodology Review (1 review) and for the updated review (1 review). For all four cases, the results and conclusions were the same between the Cochrane Methodology Review and the co-published version. Characteristics are summarized in Table 4.

Table 4 Characteristics of four co-publications (journal version)
Full size table

Number of citations for updated and original Cochrane Methodology Reviews

As shown in Table 5, the original Cochrane Methodology Review was still being cited after the updated Cochrane review had been published in all six cases and, in one, there were 485 citations to the original version of the review and only 105 to the update in the five years after publication of the update. Combining all the reviews and considering the total number of citations between the publication date of the updated version and 16 August 2022, this was non-significantly higher for the updated Cochrane Reviews than for their original version [161 (IQR 85, 198) versus 113 (IQR 15, 433), p = 0.518]. There were also non-significantly more citations to the updated review than the original review if this analysis is limited to the first five years after publication of the update [78 (IQR 61, 149) versus 91 (IQR 14, 247), p = 0.897]. Table 6 shows that in the first three years after the publication of the update, the median number of citations to the updated review was lower than to the original review but it was higher in each of the next two years.

Table 5 Total number of 5-year citations for each of the updated reviews
Full size table
Table 6 Number of citations for the updated and original Cochrane reviews after the updated year
Full size table

Number of citations for co-published and non-co-published Cochrane Methodology Reviews

The median for the total number of citations (combining citations to the original Cochrane Methodology Review and to its co-publication) up to 16 August 2022 was non-significantly higher in the group of co-published reviews than in the non-co-published group [362 (IQR 179, 840) versus 145 (IQR 75, 445), p = 0.090]. Similarly, the median number of citations in each of the five years (Table 7) and sum of the first five years after publication in the co-published group was higher than in the non-co-published group [177 (IQR 99, 338) versus 75 (IQR 37, 126); p = 0.037].

Table 7 Number of citations for the co-published and non-co-published reviews
Full size table

Number of citations for the Cochrane Methodology Review and its co-publication

As shown in Table 8, in the five years after co-publication, the original Cochrane Methodology Review received more citations than the co-published version for two reviews while the co-published version received more citations for the other two reviews. The median for the number of citations was non-significantly higher for the original Cochrane Methodology Review than for its co-published version for the total number of citations up to 16 August 2022 [157 (IQR 102, 720) vs 85 (IQR 54, 265), p = 0.248], for the number of citations in the first five years after co-publication [92 (IQR 45, 290) versus 49 (IQR 37, 102)] and for each of those five years (Table 9).

Table 8 Total number of 5-year citations for each of the co-published reviews
Full size table
Table 9 Number of citations for the Cochrane Methodology Review and its co-publication
Full size table

Effect of co-publication on journal impact factor

The four co-published Cochrane Methodology Reviews were co-published in three journals (Table 10). Among these three journals, the ratio of the adjusted IF to the actual IF was less than 1 in one journal for the first year after publication, and less than 1 in two of the three journals in the second year after publication. This means that the co-published review increased the IF for these journals in those years.

Table 10 Ratio of the adjusted IF to the actual IF
Full size table

Discussion

This study is the first that we are aware of that investigates the citing of updated and original Cochrane Methodology Reviews after an update has been published, and the impact of co-publication for this type of Cochrane Review. Our results show that the original Cochrane Methodology Review was still being cited after its updated version had been published and, in some cases, there were more citations to the original review than to the update in the five years after the update was published.

We speculate that there are several reasons for the original version of a Cochrane Methodology Review to continue to be cited after an update is published. For instance, the citing authors may have a preference based on the conclusions of the two versions of the review [35], they may be more familiar with the original version or might not have been aware of the updated version at the time they prepared their article, they might be referring to something in the original version that was not included in the update or they might be re-using text that they have written previously. It is also possible that the cited Cochrane Methodology Review was updated between the submission and acceptance of the citing article [36]. Another possible explanation is “second-hand” citation of the Cochrane Methodology Review based on it being cited elsewhere before the update [36], with the citing author using the citation from another paper without checking the original [37]. Actually, this does not comply with the International Committee of Medical Journal standards [38].

It is also worth noting that the number of citations for the updated Cochrane Methodology Reviews were much higher than those for the original version in the fourth and fifth year after the update was published, which might suggest that there is a lag of a few years in the uptake of evidence from Cochrane Methodology Reviews into the wider literature. This is in accordance with academic productivity trends, which have found that there were more citations from 2.5 to 5 years after publication compared with the first 2.5 years [39].

Turning to co-publication, the number of citations for co-published Cochrane Methodology Reviews is higher than that for non-co-published reviews at least for the first five years after co-publication, with the number of citations to the co-published version being higher than the number of citations to the Cochrane Methodology Review in two cases and lower in two cases. This is consistent with a previous study [9, 10] and suggests benefits for dissemination of the findings of Cochrane Reviews. First, if the co-publishing journal has a specialist audience, the Cochrane evidence would be brought to that audience [40,41,42] and some previous studies have shown that meta-analyses have higher rates of citation in specialty journals [39, 43,44,45]. Second, the higher number of citations suggest that the evidence in the Cochrane Review will have received more attention because of the co-publication. Third, the co-published version might make the Cochrane evidence more accessible to potential users than the full Cochrane Review and might be more useful to them as a reference source [46]. Among the three journals that co-published a Cochrane Methodology Review, the IF was raised by this in the first year in one journal and in the second year in two journals. This is also consistent with a previous study [9, 10]. The longer term effects on journals are less clear because of how the IF is calculated but citation rate peaks have been shown to vary across different journals, ranging from 2.5 to 7.2 years after publication [47, 48], which suggests that journals may benefit from co-publication beyond the two years used to calculate their IF [49].

Our study has some limitations. First, our sample may be too small to allow sufficiently powered analyses and conclusions, the results might be influenced by chance [50], and the generalisability to other types of Cochrane Review needs to be investigated in further research. Second, we only included data on citations and co-publications from the Web of Science Core Collection database, which may underestimate the true number of citations and may have missed some co-publications. Third, although some studies have found that meta-analyses have higher rates of citation, which may make it more likely that co-publishing a version of a Cochrane Review may have more impact on a journal’s IF than co-publishing of any other type of article, other studies have argued that study design does not significantly influence citation rate [51, 52]. For example, there may also be important associations with number of authors [53,54,55] or their geographic location [56]. These are issues that should be investigated in further research.

In conclusion, we have shown that the citation of updated Cochrane Methodology Reviews lags a few years after their publication and confirmed that co-publication increases the number of citations for this type of Cochrane Review and may increase the IF of the co-publishing journal. However, this raises issues about whether people wishing to use the evidence from Cochrane Methodology Reviews are always using the most up to date and complete information. This highlights the importance that authors who cite any previous research should confirm the validity of the reference and the need for ways to allow people to more clearly see that a review has been updated.

Availability of data and materials

The datasets used and/or analysed during the current study are available from the corresponding author on reasonable request.

Abbreviations

IQR:

Interquartile Range

SD:

Standard Deviation

IF:

Impact Factor

CDSR:

Cochrane Database of Systematic Reviews

References

  1. Cumpston M, Chandler J. Chapter IV: Updating a review. In: Higgins JPT, Thomas J, Chandler J, Cumpston M, Li T, Page MJ, Welch VA (editors).Cochrane Handbook for Systematic Reviews of Interventions version 6.3 (updated February 2022). Available from www.training.cochrane.org/handbook.

  2. Wang Z, Shi Q, Zhou Q, Zhao S, Hou R, Lu S, et al. Retracted systematic reviews were continued to be frequently cited: a citation analysis. J Clin Epidemiol. 2022;149:137–45. https://0-doi-org.brum.beds.ac.uk/10.1016/j.jclinepi.2022.05.013.

    Article  PubMed  Google Scholar 

  3. Garner P, Hopewell S, Chandler J, MacLehose H, Schünemann HJ, Akl EA, et al. When and how to update systematic reviews: consensus and checklist. BMJ. 2016;354:i3507. https://0-doi-org.brum.beds.ac.uk/10.1136/bmj.i3507.

    Article  PubMed  PubMed Central  Google Scholar 

  4. Higgins JP, Green S, Scholten RJ. Chapter 3: Maintaining reviews: updates, amendments and feedback. In: Higgins JPT, Green S (editors). Cochrane Handbook for Systematic Reviews of Interventions. Version 5.1.0 [updated March 2011]. The Cochrane Collaboration. 2011. Available from www.cochrane-handbook.org.

  5. Hoffmeyer B, Fonnes S, Andresen K, Rosenberg J. Use of inactive Cochrane reviews in academia: A citation analysis. Scientometrics. 2023;128:2923–34.

    Article  Google Scholar 

  6. MacLehose H, Mann J. Policy and overview of co-publication. 2021. Accessed 16 Februrary. https://documentation.cochrane.org/display/EPPR/Co-publication.

  7. Mann J. Arranging co-publication of a Cochrane Review. 2021. Accessed 18 June. https://documentation.cochrane.org/display/EPPR/Arranging+co-publication+of+a+Cochrane+Review#ArrangingcopublicationofaCochraneReview-Requirementsforco-publication.

  8. MacLehose H, Mann J. Co-publication agreements with other journals. 2021. Accessed 20 July. https://documentation.cochrane.org/display/EPPR/Co-publication+agreements+with+other+journals.

  9. Zhu L, Zhang Y, Yang R, Wu Y, Lei Y, Zhang Y, et al. Copublication improved the dissemination of Cochrane reviews and benefited copublishing journals: a retrospective cohort study. J Clin Epidemiol. 2022;149:110–7. https://0-doi-org.brum.beds.ac.uk/10.1016/j.jclinepi.2022.05.016.

    Article  PubMed  Google Scholar 

  10. Wang X, Hawkins BS, Dickersin K. Cochrane systematic reviews and co-publication: dissemination of evidence on interventions for ophthalmic conditions. Syst Rev. 2015;4:118. https://0-doi-org.brum.beds.ac.uk/10.1186/s13643-015-0104-5.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  11. Tianjing L, W SR, Claire T, Bruehl A, Kay D. Identification and characterization of systematic reviews in eyes and vision. Presented at the XV Cochrane Colloquium, 2007 Oct 23–27, Sao Paulo, Brazil. The Cochrane Methodology Register (CMR); 2012. http://pesquisa.bvsalud.org/evidences/resource/es/CMR-11343.

  12. Greene RE, Freeman SR, Lundahl K, Dellavalle RP. Co-publishing Cochrane Library systematic reviews: journal editor and Cochrane Skin Group author experiences. Br J Dermatol. 2008;158:1380–2. https://0-doi-org.brum.beds.ac.uk/10.1111/j.1365-2133.2008.08552.x.

    Article  CAS  PubMed  Google Scholar 

  13. Li L, Smith HE, Atun R, Tudor Car L. Search strategies to identify observational studies in MEDLINE and Embase. Cochrane Database Syst Rev. 2019;3:Mr000041. https://0-doi-org.brum.beds.ac.uk/10.1002/14651858.MR000041.pub2.

    Article  PubMed  Google Scholar 

  14. Tudor Car L, Li L, Smith H, Atun R. Cochrane review: Search strategies to identify observational studies in MEDLINE and EMBASE. J Evid Based Med. 2019;12:225–6. https://0-doi-org.brum.beds.ac.uk/10.1111/jebm.12358.

    Article  PubMed  Google Scholar 

  15. Wager E, Middleton P. Technical editing of research reports in biomedical journals. Cochrane Database Syst Rev. 2007:Mr000002. https://0-doi-org.brum.beds.ac.uk/10.1002/14651858.MR000002.pub2.

  16. Scherer RW, Langenberg P, von Elm E. Full publication of results initially presented in abstracts. Cochrane Database Syst Rev. 2007:Mr000005. https://0-doi-org.brum.beds.ac.uk/10.1002/14651858.MR000005.pub3.

  17. Lundh A, Sismondo S, Lexchin J, Busuioc OA, Bero L. Industry sponsorship and research outcome. Cochrane Database Syst Rev. 2012;12:Mr000033. https://0-doi-org.brum.beds.ac.uk/10.1002/14651858.MR000033.pub2.

    Article  PubMed  Google Scholar 

  18. Edwards P, Roberts I, Clarke M, DiGuiseppi C, Pratap S, Wentz R, et al. Methods to increase response rates to postal questionnaires. Cochrane Database Syst Rev. 2007:Mr000008. https://0-doi-org.brum.beds.ac.uk/10.1002/14651858.MR000008.pub3.

  19. Kunz R, Vist G, Oxman AD. Randomisation to protect against selection bias in healthcare trials. Cochrane Database Syst Rev. 2007:Mr000012. https://0-doi-org.brum.beds.ac.uk/10.1002/14651858.MR000012.pub2.

  20. Mapstone J, Elbourne D, Roberts I. Strategies to improve recruitment to research studies. Cochrane Database Syst Rev. 2007:Mr000013. https://0-doi-org.brum.beds.ac.uk/10.1002/14651858.MR000013.pub3.

  21. Wager E, Middleton P. Technical editing of research reports in biomedical journals. Cochrane Database Syst Rev. 2008;2008:Mr000002. https://0-doi-org.brum.beds.ac.uk/10.1002/14651858.MR000002.pub3.

    Article  PubMed  PubMed Central  Google Scholar 

  22. Scherer RW, Meerpohl JJ, Pfeifer N, Schmucker C, Schwarzer G, von Elm E. Full publication of results initially presented in abstracts. Cochrane Database Syst Rev. 2018;11:Mr000005. https://0-doi-org.brum.beds.ac.uk/10.1002/14651858.MR000005.pub4.

    Article  PubMed  Google Scholar 

  23. Lundh A, Lexchin J, Mintzes B, Schroll JB, Bero L. Industry sponsorship and research outcome. Cochrane Database Syst Rev. 2017;2:Mr000033. https://0-doi-org.brum.beds.ac.uk/10.1002/14651858.MR000033.pub3.

    Article  PubMed  Google Scholar 

  24. Edwards PJ, Roberts I, Clarke MJ, Diguiseppi C, Wentz R, Kwan I, et al. Methods to increase response to postal and electronic questionnaires. Cochrane Database Syst Rev. 2009;2009:Mr00008. https://0-doi-org.brum.beds.ac.uk/10.1002/14651858.MR000008.pub4.

    Article  Google Scholar 

  25. Odgaard-Jensen J, Vist GE, Timmer A, Kunz R, Akl EA, Schünemann H, et al. Randomisation to protect against selection bias in healthcare trials. Cochrane Database Syst Rev. 2011;2011:Mr00012. https://0-doi-org.brum.beds.ac.uk/10.1002/14651858.MR000012.pub3.

    Article  Google Scholar 

  26. Treweek S, Pitkethly M, Cook J, Fraser C, Mitchell E, Sullivan F, et al. Strategies to improve recruitment to randomised trials. Cochrane Database Syst Rev. 2018;2:Mr000013. https://0-doi-org.brum.beds.ac.uk/10.1002/14651858.MR000013.pub6.

    Article  PubMed  Google Scholar 

  27. Brueton VC, Tierney J, Stenning S, Harding S, Meredith S, Nazareth I, et al. Strategies to improve retention in randomised trials. Cochrane Database Syst Rev. 2013:Mr000032. https://0-doi-org.brum.beds.ac.uk/10.1002/14651858.MR000032.pub2.

  28. Treweek S, Pitkethly M, Cook J, Kjeldstrøm M, Taskila T, Johansen M, et al. Strategies to improve recruitment to randomised controlled trials. Cochrane Database Syst Rev. 2010:Mr000013. https://0-doi-org.brum.beds.ac.uk/10.1002/14651858.MR000013.pub5.

  29. Brueton VC, Tierney JF, Stenning S, Meredith S, Harding S, Nazareth I, et al. Strategies to improve retention in randomised trials: a Cochrane systematic review and meta-analysis. BMJ Open. 2014;4:e003821. https://0-doi-org.brum.beds.ac.uk/10.1136/bmjopen-2013-003821.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  30. Treweek S, Lockhart P, Pitkethly M, Cook JA, Kjeldstrøm M, Johansen M, et al. Methods to improve recruitment to randomised controlled trials: Cochrane systematic review and meta-analysis. BMJ Open. 2013;3. https://0-doi-org.brum.beds.ac.uk/10.1136/bmjopen-2012-002360.

  31. Bero L. Industry sponsorship and research outcome: a Cochrane review. JAMA Intern Med. 2013;173:580–1. https://0-doi-org.brum.beds.ac.uk/10.1001/jamainternmed.2013.4190.

    Article  PubMed  Google Scholar 

  32. Lundh A, Lexchin J, Mintzes B, Schroll JB, Bero L. Industry sponsorship and research outcome: systematic review with meta-analysis. Intensive Care Med. 2018;44:1603–12. https://0-doi-org.brum.beds.ac.uk/10.1007/s00134-018-5293-7.

    Article  CAS  PubMed  Google Scholar 

  33. Treweek S, Mitchell E, Pitkethly M, Cook J, Kjeldstrøm M, Taskila T, et al. Strategies to improve recruitment to randomised controlled trials. Cochrane Database Syst Rev. 2010:Mr000013. https://0-doi-org.brum.beds.ac.uk/10.1002/14651858.MR000013.pub4.

  34. Summaries of Cochrane Reviews in Journals. 2021. Accessed March. https://training.cochrane.org/online-learning/knowledge-translation/how-share-cochrane-evidence/choose-right-dissemination-produ-0.

  35. Urlings MJE, Duyx B, Swaen GMH, Bouter LM, Zeegers MP. Citation bias and other determinants of citation in biomedical research: findings from six citation networks. J Clin Epidemiol. 2021;132:71–8. https://0-doi-org.brum.beds.ac.uk/10.1016/j.jclinepi.2020.11.019.

    Article  PubMed  Google Scholar 

  36. Wang Z, Shi Q, Zhou Q, Zhao S, Hou R, Lu S, et al. Retracted systematic reviews were continued to be frequently cited: a citation analysis. J Clin Epidemiol. 2022:137–45. https://0-doi-org.brum.beds.ac.uk/10.1016/j.jclinepi.2022.05.013.

  37. Candal-Pedreira C, Ruano-Ravina A, Fernández E, Ramos J, Campos-Varela I, Pérez-Ríos M. Does retraction after misconduct have an impact on citations? A pre-post study BMJ Glob Health. 2020;5:e003719. https://0-doi-org.brum.beds.ac.uk/10.1136/bmjgh-2020-003719.

    Article  PubMed  Google Scholar 

  38. International Committee of Medical Journal Editors. Manuscript preparation: preparing for submission. Available at http://www.icmje.org/recommendations/browse/manuscript-preparation/preparing-forsubmission.html. Accessed 22 Dec 2021.

  39. Lee KE, McMullen N, Kota H, Peterson K, Oravec C, Frey C, et al. Predictors of Citations in Neurosurgical Research: A 5-Year Follow-Up. World Neurosurg. 2021;153:e66–75. https://0-doi-org.brum.beds.ac.uk/10.1016/j.wneu.2021.06.029.

    Article  PubMed  Google Scholar 

  40. P C. Escape from the impact factor. Ethics Sci Environ Polit. 2008;8:5–7.

    Article  Google Scholar 

  41. Neylon C, Wu S. Article-level metrics and the evolution of scientific impact. PLoS Biol. 2009;7:e1000242. https://0-doi-org.brum.beds.ac.uk/10.1371/journal.pbio.1000242.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  42. Haven TL, Bouter LM, Smulders YM, Tijdink JK. Perceived publication pressure in Amsterdam: Survey of all disciplinary fields and academic ranks. PLoS One. 2019;14:e0217931. https://0-doi-org.brum.beds.ac.uk/10.1371/journal.pone.0217931.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  43. Bhandari M, Busse J, Devereaux PJ, Montori VM, Swiontkowski M, Tornetta Iii P, et al. Factors associated with citation rates in the orthopedic literature. Can J Surg. 2007;50:119–23.

    PubMed  PubMed Central  Google Scholar 

  44. Willis DL, Bahler CD, Neuberger MM, Dahm P. Predictors of citations in the urological literature. BJU Int. 2011;107:1876–80. https://0-doi-org.brum.beds.ac.uk/10.1111/j.1464-410X.2010.10028.x.

    Article  PubMed  Google Scholar 

  45. Winnik S, Raptis DA, Walker JH, Hasun M, Speer T, Clavien PA, et al. From abstract to impact in cardiovascular research: factors predicting publication and citation. Eur Heart J. 2012;33:3034–45. https://0-doi-org.brum.beds.ac.uk/10.1093/eurheartj/ehs113.

    Article  PubMed  PubMed Central  Google Scholar 

  46. Pearlman AS. Co-publication-what you always wanted to know but have been afraid to ask! J Am Soc Echocardiogr. 2011;24:346–7. https://0-doi-org.brum.beds.ac.uk/10.1016/j.echo.2011.02.001.

    Article  PubMed  Google Scholar 

  47. Gd W. The citation life cycle of articles published in 13 American Psychological Associa- tion journals: a 25-year longitudinal analysis. J Am Soc Inf Sci Technol. 2011;62:1629–36.

    Article  Google Scholar 

  48. Madhugiri VS, Sasidharan GM, Subeikshanan V, Dutt A, Ambekar S, Strom SF. An analysis of the citation climate in neurosurgical literature and description of an interfield citation metric. Neurosurgery. 2015;76:505–12. https://0-doi-org.brum.beds.ac.uk/10.1227/neu.0000000000000656.

    Article  PubMed  Google Scholar 

  49. Roldan-Valadez E, Salazar-Ruiz SY, Ibarra-Contreras R, Rios C. Current concepts on bibliometrics: a brief review about impact factor, Eigenfactor score, CiteScore, SCImago Journal Rank, Source-Normalised Impact per Paper, H-index, and alternative metrics. Ir J Med Sci. 2019;188:939–51. https://0-doi-org.brum.beds.ac.uk/10.1007/s11845-018-1936-5.

    Article  PubMed  Google Scholar 

  50. Clarke M. The true meaning of DICE: don’t ignore chance effects. J R Soc Med. 2021;114:575–7. https://0-doi-org.brum.beds.ac.uk/10.1177/01410768211064102.

    Article  PubMed  Google Scholar 

  51. Sochacki KR, Jack RA 2nd, Nauert R, Harris JD. Correlation Between Quality of Evidence and Number of Citations in Top 50 Cited Articles in Rotator Cuff Repair Surgery. Orthop J Sports Med. 2018;6:2325967118776635. https://0-doi-org.brum.beds.ac.uk/10.1177/2325967118776635.

    Article  PubMed  PubMed Central  Google Scholar 

  52. Callaham M, Wears RL, Weber E. Journal prestige, publication bias, and other characteristics associated with citation of published studies in peer-reviewed journals. JAMA. 2002;287:2847–50. https://0-doi-org.brum.beds.ac.uk/10.1001/jama.287.21.2847.

    Article  PubMed  Google Scholar 

  53. Oravec CS, Frey CD, Berwick BW, Vilella L, Aschenbrenner CA, Wolfe SQ, et al. Predictors of Citations in Neurosurgical Research. World Neurosurg. 2019;130:e82–9. https://0-doi-org.brum.beds.ac.uk/10.1016/j.wneu.2019.05.226.

    Article  PubMed  Google Scholar 

  54. Harsh D, Adnan HS, Raees AP, Manjul T, Anil N. How Many Neurosurgeons Does It Take to Author an Article and What Are the Other Factors That Impact Citations? World Neurosurg. 2021;146:e993–1002. https://0-doi-org.brum.beds.ac.uk/10.1016/j.wneu.2020.11.058.

    Article  PubMed  Google Scholar 

  55. Ponce FA, Lozano AM. Highly cited works in neurosurgery. Part I: the 100 top-cited papers in neurosurgical journals. J Neurosurg. 2010;112:223–32. https://0-doi-org.brum.beds.ac.uk/10.3171/2009.12.Jns091599.

    Article  PubMed  Google Scholar 

  56. Yom KH, Jenkins NW, Parrish JM, Brundage TS, Hrynewycz NM, Narain AS, et al. Predictors of Citation Rate in the Spine Literature. Clin Spine Surg. 2020;33:76–81. https://0-doi-org.brum.beds.ac.uk/10.1097/bsd.0000000000000921.

    Article  PubMed  Google Scholar 

Download references

Acknowledgements

National key clinical specialty.

Funding

None.

Author information

Authors and Affiliations

  1. General Practice Ward/International Medical Center Ward, General Practice Medical Center and National Clinical Research Center for Geriatrics, West China Hospital, Sichuan University, Chengdu, 610041, China

    Linlin Zhu, Ziyu Yang, Hongyu Deng & Xiaoyang Liao

  2. Department of Periodical Press and National Clinical Research Center for Geriatrics, West China Hospital, Sichuan University, Chengdu, 610041, China

    Yonggang Zhang

  3. Northern Ireland Methodology Hub, Centre for Public Health, Queen’s University Belfast, Belfast, BT12 6BJ, Northern Ireland

    Mike Clarke

Authors
  1. Linlin Zhu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ziyu Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Hongyu Deng
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Yonggang Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Xiaoyang Liao
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Mike Clarke
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

Linlin Zhu: Formal analysis, Investigation, Visualization and Writing-Original Draft. Ziyu Yang: Visualization, Data curation. Hongyu Deng: Investigation, Software. Yonggang Zhang: Conceptualization, Project administration, Supervision, Writing-reviewing & Editing. Xiaoyang Liao: Project administration, Supervision. Mike Clarke: Conceptualization, Project administration, Supervision.

Corresponding authors

Correspondence to Yonggang Zhang, Xiaoyang Liao or Mike Clarke.

Ethics declarations

Ethics approval and consent to participate

N/A.

Consent for publication

N/A.

Competing of interest

Mike Clarke is Co-ordinating Editor of the Cochrane Methodology Review Group and co-editor-in-chief of the Journal of Evidence-Based Medicine, which has a co-publication agreement with Cochrane. Yonggang Zhang is the managing editor of the Journal of Evidence-Based Medicine and an Editor with the Cochrane Methodology Review Group. All authors declared there was no other potential commercial, professional or other conflicts of interest relevant to this article.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhu, L., Yang, Z., Deng, H. et al. Citation of updated and co-published Cochrane Methodology Reviews. Syst Rev 12, 120 (2023). https://0-doi-org.brum.beds.ac.uk/10.1186/s13643-023-02270-w

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://0-doi-org.brum.beds.ac.uk/10.1186/s13643-023-02270-w

Keywords

Systematic Reviews

ISSN: 2046-4053