Skip to main content

Online dissemination of Cochrane reviews on digital health technologies: a cross-sectional study

Abstract

Background

This cross-sectional study investigated the online dissemination of Cochrane reviews on digital health technologies.

Methods

We searched the Cochrane Database of Systematic Reviews from inception up to May 2023. Cochrane reviews with any population (P), intervention or concept supported by any digital technology (I), any or no comparison (C), and any health outcome (O) were included. Data on review characteristics (bibliographic information, PICO, and evidence quality) and dissemination strategies were extracted and processed. Dissemination was assessed using review information on the Cochrane website and Altmetric data that trace the mentions of academic publications in nonacademic online channels. Data were analysed using descriptive statistics and binary logistic regression analysis.

Results

Out of 170 records identified in the search, 100 Cochrane reviews, published between 2005 and 2023, were included. The reviews focused on consumers (e.g. patients, n = 86), people of any age (n = 44), and clinical populations (n = 68). All reviews addressed interventions or concepts supported by digital technologies with any devices (n = 73), mobile devices (n = 17), or computers (n = 10). The outcomes focused on disease treatment (n = 56), health promotion and disease prevention (n = 27), or management of care delivery (n = 17). All reviews included 1–132 studies, and half included 1–10 studies. Meta-analysis was performed in 69 reviews, and certainty of evidence was rated as high or moderate for at least one outcome in 46 reviews.

In agreement with the Cochrane guidelines, all reviews had a plain language summary (PLS) that was available in 3–14 languages. The reviews were disseminated (i.e. mentioned online) predominantly via X/Twitter (n = 99) and Facebook (n = 69). Overall, 51 reviews were mentioned in up to 25% and 49 reviews in 5% of all research outputs traced by Altmetric data. Dissemination (i.e. higher Altmetric scores) was associated with bibliographic review characteristics (i.e. earlier publication year and PLS available in more languages), but not with evidence quality (i.e. certainty of evidence rating, number of studies, or meta-analysis performed in review).

Conclusions

Online attention towards Cochrane reviews on digital health technologies is high. Dissemination is higher for older reviews and reviews with more PLS translations. Measures are required to improve dissemination of Cochrane reviews based on evidence quality.

Systematic review registration

The study was prospectively registered at the Open Science Framework (https://osf.io/mpw8u/).

Peer Review reports

Background

Digital health technologies

Digital technologies have revolutionised healthcare and public health by offering unprecedented opportunities for improved medical interventions and diagnostics as well as monitoring of health [1, 2]. Digital technologies, such as smartphones and wearables, can assist healthcare decisions, including more accurate and timely diagnoses [3], and contribute to health promotion and disease prevention by empowering their users to take an active role in managing their health via access to personalised health information and self-monitoring [4]. Despite the growing accessibility of digital technologies, a persistent digital divide exists that limits the equitable access to innovations due to socioeconomic factors, geographical disparities, and varying levels of technological literacy [5]. Furthermore, digital technologies require systematic evaluation to investigate if they contribute to any health benefits in their users [6].

Scientific evidence required to evaluate health outcomes of digital technologies can be synthesised using systematic reviews. Systematic reviews play a crucial role as a valuable source of evidence for stakeholders, including policy makers, clinicians, researchers, and the general population. Among systematic reviews, Cochrane reviews employ high methodological standards [7] and are therefore especially valuable for stakeholders. Considering the rapid development of digital health, it is important to assess how Cochrane reviews addressed this topic so far, in terms of digital technology types, target populations, and health outcomes. Furthermore, it is also of interest to investigate how these reviews are disseminated online to improve their impact by potentially reaching any relevant stakeholders [8, 9].

Dissemination and analysis of its impact

Dissemination is defined as a proactive method of distributing scientific evidence to a specific audience through selected channels and planned strategies [10]. While peer-reviewed publications and conference presentations are effective strategies for disseminating research to academic audiences, other strategies, such as plain language summaries (PLS), policy briefs, reports, blogs, and communication via social media, may be better suited to reach policy makers or the interested public [11,12,13].

As there are a multitude of channels which can be used to disseminate research, different measures can be used to assess impact of academic publications. For example, the number of citations of an academic publication is one method for estimating its academic impact. Different measures are needed to objectively assess the nonacademic impact of academic publications due to multiple and heterogeneous channels that could be used for dissemination. As such, the nonacademic impact of academic publications could be assessed by examining citations in policy documents, reports, guidelines, and mentions on social media. Altmetric data provide one method of tracing mentions of academic publications in nonacademic online channels [14] and thus could be used to objectively quantify the nonacademic impact of such publications.

Study aims

The aim of this study was to investigate the online dissemination of Cochrane reviews on digital health technologies. This study addressed the following research questions:

  1. (1)

    What are the characteristics of Cochrane reviews on digital technology use in healthcare and public health, including bibliographic, PICO (Population, Intervention, Comparison, Outcome), and evidence quality characteristics?

  2. (2)

    How are such Cochrane reviews disseminated online?

Methods

Study design

This study is a meta-research (i.e. research on research) study. We used Cochrane reviews as the unit of analysis and a cross-sectional design to collect and analyse data from such reviews (i.e. data on review content and review dissemination). A protocol for this study was prospectively registered at the Open Science Framework [15]. The study adheres to the Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) guideline [16]. The STROBE checklist is reported in Additional file 1. There were no changes to the original protocol.

Data source

The data source for this study was the Cochrane Database of Systematic Reviews [17]. This database listed 9033 records on 10 May 2023, including reviews with systematic methodology (e.g. systematic, scoping, or rapid reviews) and review protocols on any topics within healthcare and public health.

Eligibility criteria

This study included Cochrane reviews with any methodology (e.g. systematic reviews) published up to 10 May 2023. The eligibility criteria for this study were defined based on the PICO (Population, Intervention, Comparison, Outcome) framework. The inclusion criteria were as follows: (1) any human population (P), (2) intervention or concept supported by any digital technology (I), (3) any or no comparison (C), and (4) any health outcome (O). The exclusion criteria were as follows: (1) nonhuman populations, (2) no focus on digital technologies, (3) no health outcome, and (4) other Cochrane publications (e.g. review protocols due to lack of data or original reviews if review update was published to reduce bias due to duplication).

Data collection process

Data collection process involved the electronic search for Cochrane reviews, selection of eligible reviews, and data extraction from the included reviews.

Electronic search

The search strategy was created in consultation with a professional librarian and is reported in Additional file 2. The search for Cochrane reviews was performed by one researcher (M. K.) on 10 May 2023. All search results were imported to EndNote 20 (Clarivate) for study management, downloaded as full-text documents, and stored for further processing.

Review selection

Three researchers (M. K., T. J., K. K. D. S.) selected the reviews. One researcher (M. K.) performed the title/abstract and full-text screening. Two researchers (T. J. and K. K. D. S.) checked all excluded reviews to reduce the selection bias. The final consensus was reached by discussion between two researchers (M. K. and K. K. D. S.).

Data extraction

A data extraction sheet was developed in Excel 10 (Microsoft Inc.), pilot-tested using two reviews, and calibrated within the team. One researcher (M. K.) extracted all data from all reviews. To reduce bias in data extraction, team assistants helped with data entry and checked the extracted data in 10% of reviews. As no errors were detected, no further checks were performed.

Variables (data items) and data processing

A list of variables (data items) for data extraction was developed by the team (Table 1).

Table 1 Data items

Data items 1 to 5 (Table 1) were extracted as quantitative information (e.g. publication year) or qualitative, verbatim statements (e.g. description of digital technologies according to review authors) from the full text of the individual reviews. The qualitative data were subsequently processed into quantitative categories based on meaningful themes that inductively emerged from the data. For example, we assigned digital technologies into categories ‘mobile technologies’ or ‘nonmobile technologies’ based on the digital devices described in reviews. During data processing, we also considered the heterogeneous terminology used by review authors. For example, ‘mobile phone’ was used as a category to describe any portable telephone, including mobile phones, mobile telephones, videophones, smartphones, or cell phones. To reduce any biases in data processing, two researchers (M. K. and K. K. D. S.) discussed and agreed on all categories, one researcher (M. K.) processed all data, and another researcher (K. K. D. S.) checked all processed data. Any discrepancies were discussed, and the final agreement was reached by consensus between both researchers.

Data items 6 to 8 (Table 1) were extracted as quantitative information from the Cochrane website and the Altmetric data available open-access online. The Cochrane website lists various details of each Cochrane review, including a review abstract and a link to a full-text document, a plain language summary (PLS) in English that is a mandatory part of any Cochrane review, any translations of the PLS, and other review information (e.g. the number of clinical guidelines that cited the review and Altmetric data for each review). The Altmetric data are used to compute the Altmetric (attention) score that measures the attention towards academic publications by tracing their mentions in nonacademic online channels [14]. Therefore, it provides an objective method to quantify and explore online dissemination channels of academic publications. The Altmetric score is a weighted count of online mentions. It is computed as the sum of the quantity of online mentions (i.e. the number of mentions) weighted by the quality of online mentions (i.e. the source of mentions, such as Wikipedia) [18]. As of June 2023, the Altmetric score was computed based on the following 17 online channels with different weights [19] that were considered in this study:

  • News (weight of 8)

  • Blogs (weight of 5)

  • Policy documents, patents, and Wikipedia (weight of 3)

  • Post-publication peer review (on Publons or PubPeer), Weibo (traced until 2015), Google + (traced until 2019), F1000, and Syllabi (weight of 1)

  • LinkedIn (traced until 2014, weight of 0.5)

  • X/Twitter (posts and reposts), Facebook, Reddit, Pinterest (data until 2013), Q&A (Stack Exchange), and YouTube videos (weight of 0.25)

To prevent score inflation, the computation algorithm for the Altmetric score considers channel types and their weights as well as other factors, such as duplicate posts in the same channels [19]. In general, only the first mention in a specific online channel counts towards the Altmetric score (e.g. if a Cochrane review is mentioned in multiple articles in the same news channel, then only the first mention contributes to the Altmetric score for that Cochrane review) [19, 20].

The academic online channels, including Mendeley readers and Dimension and Web of Science citations, were also traced by Altmetric data in June 2023, but not included in the computation of the Altmetric score [19] and, thus, not considered in this study.

To reduce any bias due to updates in Altmetric data that occur daily at midnight West European time, Altmetric data for all reviews were extracted on a single day (29 June 2023) during business hours in Germany. One researcher (M. K.) made a screenshot of Altmetric data for each Cochrane review and collated all screenshots into one document. A team assistant manually entered all Altmetric data into the data extraction sheet in Excel. To reduce bias in data entry, one researcher (M. K.) checked the Altmetric data in 10% of reviews. Since no errors were detected, no further checks were performed.

Data synthesis

The data were synthesised using descriptive statistics (e.g. absolute and relative frequencies or measures of central tendency, including mean and standard deviation) in Excel and IBM-SPSS24. Depending on data availability and type, we planned to investigate if dissemination (i.e. Altmetric scores) is associated with any review characteristics (e.g. publication year) or evidence quality characteristics (e.g. certainty of evidence ratings) using a univariate linear or binary logistic regression analysis in IBM-SPSS24.

Results

Sample size

This study includes data from 100 Cochrane reviews that met the eligibility criteria out of 170 records identified in the search of Cochrane Database of Systematic Reviews (Fig. 1). A list of 100 included reviews and excluded reviews with individual reasons for exclusion is reported in Additional file 3. All data extracted from the 100 reviews are reported in Additional files 4 and 5, and data synthesis is reported in Additional file 6.

Fig. 1
figure 1

Study selection (PRISMA 2020 flowchart)

Review characteristics

Bibliographic characteristics

The included 100 Cochrane reviews were published between 2005 and 2023. The reviews included systematic reviews (n = 97), rapid reviews (n = 2), and an overview of reviews (n = 1). The reviews originated from Europe (n = 61), Australia (n = 21), North America (n = 13), Asia (n = 3), or Africa (n = 2).

PICO characteristics

The PICO characteristics of the included 100 Cochrane reviews are summarised in Fig. 2.

Fig. 2
figure 2

PICO characteristics of the included Cochrane reviews

The reviews focused predominantly on consumers (i.e. patients, clients, or carers; n = 86), people of any age (i.e. adults or children; n = 44), and clinical populations (n = 68). Disease focus in most reviews (n = 63) was on brain, respiratory, or any diseases, while other specific diseases (e.g. cardiovascular) were addressed in less than 10 reviews each.

All reviews addressed interventions or concepts supported by digital health technologies. Depending on modality, most reviews (n = 62) included only digital interventions or concepts (e.g. interventions delivered via mobile phones in all primary studies). Depending on digital technology type, most reviews (n = 70) included any mobile or nonmobile interventions or concepts (i.e. those delivered using mobile or nonmobile devices). The digital devices used to deliver the interventions or concepts were any devices (e.g. mobile phones or computers) in most reviews (n = 73). The interaction between users and digital technologies occurred via emails, text messages, websites, apps, social media, interactive voice response systems, video calls, virtual reality, or electronic patient records. Most reviews (n = 57) compared the outcomes of digital interventions or concepts with non-digital comparison conditions.

All reviews addressed any outcomes in the healthcare context. Most reviews focused on disease treatment (i.e. therapy, adherence, disease management and monitoring, or rehabilitation; n = 56) and management of care delivery (i.e. support systems for care delivery, medical training, or patient management via electronic patient records; n = 17), while some (n = 27) also focused on public health (i.e. health promotion and disease prevention). Specifically, the 27 public health reviews addressed health education (e.g. in the context of reproductive and sexual health or vaccination uptake) and lifestyle monitoring (e.g. secondary and tertiary prevention for people with chronic diseases, management of weight or stress, prevention of social isolation or cognitive decline, or substance use reduction).

Evidence quality

All reviews included 1–132 studies (50 reviews included 1–10 studies, and further 50 reviews included 11–132 studies), and meta-analysis was performed in 69 reviews. The certainty of evidence was rated for a total of 767 quantitative outcomes assessed in 87 reviews. Among the 87 reviews, the certainty of evidence for at least one outcome was high in 10 (11%) reviews, moderate in 45 (52%) reviews, low in 71 (82%) reviews, or very low in 58 (67%) reviews. Overall, 46 (53%) reviews rated the certainty of evidence for at least one outcome as high or moderate, and 41 (47%) reviews did not rate any outcomes as high or moderate. Among the 767 quantitative outcomes, the certainty of evidence was high for 22 (3%) outcomes, moderate for 170 (22%) outcomes, low for 298 (39%) outcomes, and very low for 277 (36%) outcomes.

Dissemination strategies

In agreement with the Cochrane guidelines, all reviews had a text-based plain language summary (PLS) that was available in 3–14 languages, including English. Most common translation languages were Spanish (n = 100), French (n = 88), and Arabic (n = 72). According to review information on the Cochrane website, 60 reviews were cited in 1–18 clinical guidelines.

According to the Altmetric data, the included Cochrane reviews were disseminated (i.e. mentioned online at least once) in 9/17 online channels that are used to compute the Altmetric score (Fig. 3). These included predominantly social media (X/Twitter and Facebook), while less than 50% of reviews were disseminated via blogs, policy sources, news, Wikipedia, Google + , YouTube, or patents. The Cochrane reviews were not mentioned in 8/17 online channels, including (post-publication) peer reviews, Weibo, Syllabi, F1000, LinkedIn, Reddit, Q&A, and Pinterest.

Fig. 3
figure 3

Dissemination (i.e. at least one mention) of Cochrane reviews by online channel

The included Cochrane reviews were mentioned 4661 times in 9/17 online channels (Fig. 4). Most traced mentions were found in the social media (X/Twitter and Facebook) and news outlets. Each Cochrane review was mentioned 1–271 times (50 reviews were mentioned 1–33 times, and further 50 were mentioned 34–271 times).

Fig. 4
figure 4

Number of mentions of Cochrane reviews by online channel

The Altmetric score for all reviews ranged between 1 and 553 (mean = 48, standard deviation = 67; 1–28 in 50 reviews and 29–553 in further 50 reviews). Most reviews (n = 97) had Altmetric scores between 1 and 150, and n = 3 reviews had Altmetric scores of more than 200 (i.e. 207 [21], 237 [22], and 553 [23]). These n = 3 reviews addressed topics relevant for the COVID-19 pandemic, including interventions to improve vaccination rates [23], digital technologies for contact tracing during epidemics [22], and interventions to reduce social isolation and loneliness among the elderly [21].

The online attention towards most reviews was high. Altmetric scores in 51 reviews were rated in up to the top 25% of all research outputs traced by Altmetric data (i.e. Altmetric scores of 1–6 in 10 reviews were rated below the top 25%, and Altmetric scores of 7–29 in 41 reviews were rated in the top 25%). Altmetric scores of 30–553 (in 49 reviews) were rated in the top 5% of all research outputs traced by Altmetric data.

Association between dissemination and review characteristics

To investigate if dissemination (i.e. Altmetric scores) is associated with any review characteristics, we performed a binary logistic regression analysis with one dependent and five independent variables (Table 2).

Table 2 Association between dissemination (Altmetric scores) and review characteristics

Dissemination (i.e. higher Altmetric scores) was associated with bibliographic review characteristics (i.e. older review publication year and PLS available in more languages). Dissemination was not associated with evidence quality (i.e. certainty of evidence, studies in review, or meta-analysis performed in review; Table 2). The same pattern of results was obtained in a sensitivity analysis when the three reviews with the highest Altmetric scores were removed from the binary logistic regression analysis (Additional file 6).

Dissemination of this study

In addition to this academic publication, we planned to disseminate the results of this study via a conference poster at a scientific meeting [24] and PLS in English and in German (Additional file 7).

Discussion

Summary of main findings

This cross-sectional study based on 100 Cochrane reviews on digital health technologies shows that such reviews (1) addressed different types of digital technologies focusing mainly on disease treatment or management in adult clinical populations and (2) were widely disseminated via nonacademic online channels. The online attention towards 90 reviews was high based on Altmetric scores in the top 25% or 5% of all research outputs traced by the Altmetric data. Dissemination was associated with bibliographic review characteristics (earlier publication year and more PLS translations), but not with evidence quality.

Dissemination

Our study shows that Cochrane reviews on digital health technologies were predominantly disseminated via the social media (X/Twitter or Facebook). Thus, Cochrane reviews might be used as important source of health information for users of such media. Interestingly, dissemination was not associated with evidence quality but rather with relevance of review topic (i.e. the highest Altmetric scores were recorded for review related to COVID-19 topics) and accessibility of review (i.e. more years since publication and with PLS in more languages). Similar trends are also observed in citations of scientific articles that reflect the scientific impact and relevance of article topics but less so research quality [25]. Previous studies using Altmetric data suggest that higher visibility in social media is associated with other characteristics of scientific articles not considered in this study, including high journal impact factors, published open access, and having informative titles [26,27,28]. Various measures could be used to improve dissemination of Cochrane reviews based on their evidence quality rather than their mere presence online in various languages. For example, improvements in science communication may help academic authors to clearly communicate their scientific findings [9]. This may enhance the lay understanding of PLS for nonacademic stakeholders and thus facilitate their decision whether or not to disseminate a specific content. In particular, the evidence limitations, such as very low and low certainty of evidence, need to be adequately explained in the PLS. Furthermore, although not investigated in this study, more focus on enhancing health literacy and digital health literacy is needed to improve the understanding of scientific content for any stakeholders or population groups [29].

Poor understanding of scientific content and restrictions in access to scientific evidence may contribute to the delay in dissemination of findings from newer reviews and consequently translating research evidence into clinical practice (the so-called research to practice gap) [30]. Such time lags in the translation process are prevalent in diverse fields of healthcare, and it can take up to several years between publication and being implemented or mentioned in clinical guidelines [31, 32]. Cochrane attempted to reduce the research to practice gap for COVID-19 research by publishing relevant reviews in a timely fashion (e.g. as rapid reviews) and by establishing a register of COVID-19 publications so that they could be located online faster [33]. These measures together with the global interest in COVID-19 may have contributed to the finding that COVID-19 reviews had the highest Altmetric scores in this study indicating that they were disseminated online via various channels despite their young age (recent publication date) and potentially reaching the relevant stakeholders. Future research may examine in more detail the dissemination approaches used by Cochrane for their COVID-19 reviews to find out if similar approaches could also improve the online attention towards other Cochrane or non-Cochrane reviews.

According to Altmetric data, we show that there is high online attention to Cochrane reviews on digital health technologies. This result is in line with a high academic impact of Cochrane reviews. Specifically, Cochrane reviews in public health were cited on average 240% more than other papers in this field [8]. In contrast to academic impact, Altmetric data can be used as a proxy of online interest in academic publications. In the current study, X/Twitter was the platform where the included Cochrane reviews were mentioned most by far. However, such data have various limitations because it is unclear who does the dissemination (i.e. review authors or anyone with the internet access), what are the motivations to disseminate the scientific content, and how the receivers of online mentions interact with and use the information. Furthermore, the Altmetric score only counts the interactions on different online media platforms, but it cannot distinguish between positive and critical attention [34,35,36]. Altmetric data may also be more prone to manipulation than traditional bibliometrics [35]. For example, Facebook mentions can be purchased [37], and it is unclear if mentions on social media platforms only passively exist or are actually read [35]. However, the strength of the Altmetric data is that the attention metrics for an academic publication are immediately traced and available within hours of publication [35, 36]. In contrast, scientific citation metrics, such as citation counts or journal impact factors, are typically available several years after publication [35, 36].

Altmetric score relies on a selection of online channels, some of which (e.g. Google +) are no longer traced. Potentially, there is a need to capture other online channels for dissemination, such as TikTok videos or Instagram clips. Online attention could also be increased when authors of academic publications use more knowledge translation strategies that is in turn associated with higher impact of such publications on end users via health policy and practice [38]. Especially, social media dissemination channels could be used to reach out to knowledge users [11, 12]. More visibility and potentially higher Altmetric scores could result from using hashtags for social media posts that enhance searching and finding of content and including a unique identifier of the study (i.e. a link to the Cochrane review and not only its abstract) in social media posts or blogs in the main text [39].

Future priorities for research on digital health technologies

Based on the content of the Cochrane reviews included in this study, we provide some recommendations for future research on digital health technologies.

  1. (1)

    Focus on specific digital health technology types: Digital technologies encompass a wide range of tools and systems, making it challenging to create a comprehensive and universally applicable definition [6]. Most included Cochrane reviews adopted a broad perspective by focusing on any digital health technologies and thus consequently compared ‘apples and oranges’. The evolving landscape of digital technologies shows that more targeted evaluations of specific technology types, such as smartphone apps or websites accessed via computers for specific health purposes, such as physical activity promotion, are needed [6]. Thus, a future research priority is to evaluate specific digital health technology types to establish if their use contributes to any health benefits and if such benefits have any clinical relevance.

  2. (2)

    Focus on the needs of diverse populations: Most included reviews included broad target populations, such as healthcare consumers (i.e. patients or carers) and people of any age. Healthcare consumers may have different needs and acceptance of digital health offers than healthcare professionals [40,41,42], who were less often studied. Furthermore, different age cohorts may have distinct needs, preferences, and responses to digital health technologies due to their digital experience. While younger age cohorts are likely to be digitised (and thus considered as ‘digital natives’), older age cohorts have varying levels of digital experience that could affect their uptake and use of digital health technologies [43, 44]. Thus, a future research priority is to assess the needs of diverse populations regarding digital health technologies.

  3. (3)

    Focus expanding beyond healthcare to public health: Most included Cochrane reviews focused on healthcare for brain or respiratory diseases, while only about a quarter addressed issues central to public health, including health promotion and disease prevention. Digital technologies have already been extensively used in the public-health response to the COVID-19 pandemic [45]. Some features of digital health technologies could be used for preventing or managing of other common diseases, such as cardiovascular diseases, which despite being the leading causes of death worldwide [46] were surprisingly addressed in very few reviews. For example, wearable technologies with feedback and nudging functions could be used to encourage physical activity and reduce sedentary behaviour [47], and smartphone apps could be used to provide recommendations on nutrition [48]. Thus, a future research priority is to focus on the role of digital technologies in health promotion and disease prevention.

Limitations

There were several limitations in this study. First, we did not investigate who disseminated the Cochrane reviews and why. Although review authors can disseminate their own Cochrane reviews, any planned dissemination is rarely described in the text of such reviews [9]. Dissemination via channels traced by Altmetric data can be done by anyone with Internet access. Future studies could investigate the motivations for disseminating the Cochrane reviews. Second, it is unclear in what specific (online) news outlets the Cochrane reviews were disseminated. Unlike listing the names of various online media channels (e.g. Facebook or Wikipedia), Altmetric data do not specify which (online) news outlets are traced despite that news mentions contribute the highest weight to the Altmetric score. Third, the associations between dissemination (i.e. Altmetric scores) and review characteristics were only weak, possibly because most Cochrane reviews had relatively high Altmetric scores (in the top 25% or 5% of all research outputs traced by Altmetric data). Future studies could investigate such associations in non-Cochrane reviews with more variable Altmetric scores and compare Altmetric data between Cochrane and non-Cochrane reviews. Fourth, the lack of association between dissemination and certainty of evidence rating could be due to a high heterogeneity of outcomes and digital technologies in the included Cochrane reviews. Certainty of evidence ratings in reviews (Cochrane and non-Cochrane) with the same outcome and the same digital technology could be used in future analyses. Fifth, despite a large sample size (100 reviews), the results of this study might not be generalisable to non-Cochrane reviews, Cochrane reviews in other fields than digital health technologies, and other methods of assessing dissemination than Altmetric scores.

Conclusions

Online attention towards Cochrane reviews on digital health technologies is high. In particular, social media act as nonacademic dissemination channels for such Cochrane reviews. Dissemination is higher for older reviews and reviews with more PLS translations. Measures are required to improve dissemination of Cochrane reviews based on evidence quality. Future research is needed (1) to evaluate specific digital health technology types, (2) to assess the needs of diverse populations regarding digital health technologies, and (3) to focus on the role of digital technologies in health promotion and disease prevention.

Availability of data and materials

The dataset supporting the conclusions of this article is included within the article and its additional file.

Abbreviations

PICO:

Population, Intervention, Comparison, Outcome

PLS:

Plain language summary

PRISMA:

Preferred Reporting Items for Systematic Reviews and Meta-Analyses

STROBE:

Strengthening the Reporting of Observational Studies in Epidemiology

References

  1. Mbunge E, Muchemwa B, Jiyane SE, Batani J. Sensors and healthcare 5.0: transformative shift in virtual care through emerging digital health technologies. Global Health J. 2021;5(4):169–77. https://0-doi-org.brum.beds.ac.uk/10.1016/j.glohj.2021.11.008.

    Article  Google Scholar 

  2. Roberts N, Carrigan A, Clay-Williams R, Hibbert PD, Mahmoud Z, Pomare C, Fajardo Pulido D, Meulenbroeks I, Knaggs GT, Austin EE, et al. Innovative models of healthcare delivery: an umbrella review of reviews. BMJ Open. 2023;13(2):e066270. https://0-doi-org.brum.beds.ac.uk/10.1136/bmjopen-2022-066270.

    Article  PubMed  Google Scholar 

  3. Hernández-Neuta I, Neumann F, Brightmeyer J, Ba Tis T, Madaboosi N, Wei Q, Ozcan A, Nilsson M. Smartphone-based clinical diagnostics: towards democratization of evidence-based health care. J Intern Med. 2019;285(1):19–39. https://0-doi-org.brum.beds.ac.uk/10.1111/joim.12820.

    Article  PubMed  Google Scholar 

  4. Jones GB, Bryant A, Wright J. Democratizing Global Health Care through scalable emergent (beyond the mobile) wireless technologies. JMIR Biomed Eng. 2022;7(1):e31079. https://0-doi-org.brum.beds.ac.uk/10.2196/31079.

    Article  PubMed Central  Google Scholar 

  5. Yao R, Zhang W, Evans R, Cao G, Rui T, Shen L. Inequities in health care services caused by the adoption of digital health technologies: scoping review. J Med Internet Res. 2022;24(3):e34144. https://0-doi-org.brum.beds.ac.uk/10.2196/34144.

    Article  PubMed  PubMed Central  Google Scholar 

  6. De Santis KK, Jahnel T, Matthias K, Mergenthal L, Al Khayyal H, Zeeb H. Evaluation of digital interventions for physical activity promotion: scoping review. JMIR Public Health Surveill. 2022;8(5):e37820. https://0-doi-org.brum.beds.ac.uk/10.2196/37820.

    Article  PubMed  PubMed Central  Google Scholar 

  7. Cipriani A, Furukawa TA, Barbui C. What is a Cochrane review? Epidemiol Psychiatr Sci. 2011;20(3):231–3. https://0-doi-org.brum.beds.ac.uk/10.1017/s2045796011000436.

    Article  CAS  PubMed  Google Scholar 

  8. Wolfenden L, Close S, Finch M, Lecathelinais C, Ramanathan S, De Santis KK, Car LT, Doyle J, McCrabb S. Improving academic and public health impact of Cochrane Public Health reviews: what can we learn from bibliographic metrics and author dissemination strategies? A cross-sectional study J Public Health. 2023;45(3):e577–86. https://0-doi-org.brum.beds.ac.uk/10.1093/pubmed/fdad039.

    Article  Google Scholar 

  9. Helmer SM, Matthias K, Mergenthal L, Reimer M, De Santis KK. Dissemination of knowledge from Cochrane Public Health reviews: a bibliographic study. Syst Rev. 2023;12(1):113. https://0-doi-org.brum.beds.ac.uk/10.1186/s13643-023-02272-8.

    Article  PubMed  PubMed Central  Google Scholar 

  10. Brownson RC, Eyler AA, Harris JK, Moore JB, Tabak RG. Getting the word out: new approaches for disseminating public health science. J Public Health Manag Pract. 2018;24(2):102–11. https://0-doi-org.brum.beds.ac.uk/10.1097/PHH.0000000000000673.

    Article  PubMed  PubMed Central  Google Scholar 

  11. Ross-Hellauer T, Tennant JP, Banelytė V, Gorogh E, Luzi D, Kraker P, Pisacane L, Ruggieri R, Sifacaki E, Vignoli M. Ten simple rules for innovative dissemination of research. PLoS Comput Biol. 2020;16(4):e1007704. https://0-doi-org.brum.beds.ac.uk/10.1371/journal.pcbi.1007704.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  12. Tripathy JP, Bhatnagar A, Shewade HD, Kumar AMV, Zachariah R, Harries AD. Ten tips to improve the visibility and dissemination of research for policy makers and practitioners. Public Health Action. 2017;7(1):10–4. https://0-doi-org.brum.beds.ac.uk/10.5588/pha.16.0090.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  13. McCrabb S, Hall A, Milat A, Bauman A, Hodder R, Mooney K, Webb E, Barnes C, Yoong S, Sutherland R, et al. Disseminating health research to public health policy-makers and practitioners: a survey of source, message content and delivery modality preferences. Health Res Policy Syst. 2023;21(1):121. https://0-doi-org.brum.beds.ac.uk/10.1186/s12961-023-01066-7.

    Article  PubMed  PubMed Central  Google Scholar 

  14. Altmetric Score. The donut and Altmetric attention score. https://www.altmetric.com/about-us/our-data/donut-and-altmetric-attention-score/. Accessed 13 May 2024.

  15. De Santis K, Kirstein M, Kien C, Griebler U, Jahnel T. Dissemination of Cochrane reviews on the application of digital technologies in health care and public health: a cross-sectional study (study protocol). Open Science Framework. https://osf.io/mpw8u/ Accessed 26 May 2023.

  16. von Elm E, Altman DG, Egger M, Pocock SJ, Gøtzsche PC, Vandenbroucke JP. The Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) statement: guidelines for reporting observational studies. J Clin Epidemiol. 2008;61(4):344–9. https://0-doi-org.brum.beds.ac.uk/10.1016/j.jclinepi.2007.11.008.

    Article  Google Scholar 

  17. Cochrane Database of Systematic Reviews. https://www.cochranelibrary.com/cdsr/reviews. Accessed 13 May 2024.

  18. Altmetric Score. Altmetric score explanation. https://help.altmetric.com/support/solutions/articles/6000232839-how-are-outputs-scored. Accessed 13 May 2024.

  19. Altmetric Score. Altmetric score calculation. https://help.altmetric.com/support/solutions/articles/6000233311-how-is-the-altmetric-attention-score-calculated. Accessed 13 May 2024.

  20. Altmetric Score. Altmetric Attention Score modifiers. https://help.altmetric.com/support/solutions/articles/6000234288-altmetric-attention-score-modifiers. Accessed 13 May 2024.

  21. Noone C, McSharry J, Smalle M, Burns A, Dwan K, Devane D, Morrissey EC. Video calls for reducing social isolation and loneliness in older people: a rapid review. Cochrane Database of Systematic Reviews. 2020;(5). https://0-doi-org.brum.beds.ac.uk/10.1002/14651858.CD013632.

  22. Anglemyer A, Moore THM, Parker L, Chambers T, Grady A, Chiu K, Parry M, Wilczynska M, Flemyng E, Bero L. Digital contact tracing technologies in epidemics: a rapid review. Cochrane Database of Systematic Reviews. 2020;(8). https://0-doi-org.brum.beds.ac.uk/10.1002/14651858.CD013699.

  23. Jacobson Vann JC, Jacobson RM, Coyne‐Beasley T, Asafu‐Adjei JK, Szilagyi PG. Patient reminder and recall interventions to improve immunization rates. Cochrane Database of Systematic Reviews. 2018;(1). https://0-doi-org.brum.beds.ac.uk/10.1002/14651858.CD003941.pub3.

  24. De Santis K, Kirstein M, Kien C, Griebler U, Jahnel T. How are Cochrane reviews on the application of digital technologies in health care or public health disseminated online? A cross-sectional study. Conference poster. 25th Congress of the Network Evidence-Based Medicine (EbM). 13-15 March 2024, Berlin, Germany. 2024. https://0-doi-org.brum.beds.ac.uk/10.3205/24ebm119. Accessed 13 May 2024.

  25. Aksnes DW, Langfeldt L, Wouters P. Citations, citation indicators, and research quality: an overview of basic concepts and theories. SAGE Open. 2019;9(1):2158244019829575. https://0-doi-org.brum.beds.ac.uk/10.1177/2158244019829575.

    Article  Google Scholar 

  26. Araujo AC, Gonzalez GZ, Nascimento DP, Costa LOP. The impact of low back pain systematic reviews and clinical practice guidelines measured by the Altmetric score: cross-sectional study. Braz J Phys Ther. 2021;25(1):48–55. https://0-doi-org.brum.beds.ac.uk/10.1016/j.bjpt.2020.01.002.

    Article  PubMed  Google Scholar 

  27. Araujo AC, Nascimento DP, Gonzalez GZ, Maher CG, Costa LOP. Impact of low back pain clinical trials measured by the Altmetric score: cross-sectional study. J Med Internet Res. 2018;20(4):e86. https://0-doi-org.brum.beds.ac.uk/10.2196/jmir.9368.

    Article  PubMed  PubMed Central  Google Scholar 

  28. Araujo AC, Vanin AA, Nascimento DP, Gonzalez GZ, Costa LOP. What are the variables associated with Altmetric scores? Syst Rev. 2021;10(1):193. https://0-doi-org.brum.beds.ac.uk/10.1186/s13643-021-01735-0.

    Article  PubMed  PubMed Central  Google Scholar 

  29. Qi S, Hua F, Xu S, Zhou Z, Liu F. Trends of global health literacy research (1995–2020): analysis of mapping knowledge domains based on citation data mining. PLoS ONE. 2021;16(8):e0254988. https://0-doi-org.brum.beds.ac.uk/10.1371/journal.pone.0254988.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  30. Abu-Odah H, Said NB, Nair SC, Allsop MJ, Currow DC, Salah MS, Hamad BA, Elessi K, Alkhatib A, ElMokhallalati Y, et al. Identifying barriers and facilitators of translating research evidence into clinical practice: a systematic review of reviews. Health Soc Care Community. 2022;30(6):e3265–76. https://0-doi-org.brum.beds.ac.uk/10.1111/hsc.13898.

    Article  PubMed  Google Scholar 

  31. Morris ZS, Wooding S, Grant J. The answer is 17 years, what is the question: understanding time lags in translational research. J R Soc Med. 2011;104(12):510–20. https://0-doi-org.brum.beds.ac.uk/10.1258/jrsm.2011.110180.

    Article  PubMed  PubMed Central  Google Scholar 

  32. Khan S, Chambers D, Neta G. Revisiting time to translation: implementation of evidence-based practices (EBPs) in cancer control. Cancer Causes Control. 2021;32(3):221–30. https://0-doi-org.brum.beds.ac.uk/10.1007/s10552-020-01376-z.

    Article  PubMed  Google Scholar 

  33. Cochrane. Cochrane COVID-19 publications. https://www.cochranelibrary.com/covid-19. Accessed 13 May 2024.

  34. Bajwa SJS, Mehdiratta L. From traditional bibliometrics to altmetrics: socialising the research metrics. Indian J Anaesth. 2021;65(12):849–52. https://0-doi-org.brum.beds.ac.uk/10.4103/ija.ija_1058_21.

    Article  PubMed  PubMed Central  Google Scholar 

  35. Bornmann L. Do altmetrics point to the broader impact of research? An overview of benefits and disadvantages of altmetrics. J Informet. 2014;8(4):895–903. https://0-doi-org.brum.beds.ac.uk/10.1016/j.joi.2014.09.005.

    Article  Google Scholar 

  36. Sud P, Thelwall M. Evaluating altmetrics. Scientometrics. 2014;98(2):1131–43. https://0-doi-org.brum.beds.ac.uk/10.1007/s11192-013-1117-2.

    Article  Google Scholar 

  37. Brigham TJ. An introduction to altmetrics. Med Ref Serv Q. 2014;33(4):438–47. https://0-doi-org.brum.beds.ac.uk/10.1080/02763869.2014.957093.

    Article  PubMed  Google Scholar 

  38. Wolfenden L, Mooney K, Gonzalez S, Hall A, Hodder R, Nathan N, Yoong S, Ditton E, Sutherland R, Lecathelinais C, et al. Increased use of knowledge translation strategies is associated with greater research impact on public health policy and practice: an analysis of trials of nutrition, physical activity, sexual health, tobacco, alcohol and substance use interventions. Health Res Policy Syst. 2022;20(1):15. https://0-doi-org.brum.beds.ac.uk/10.1186/s12961-022-00817-2.

    Article  PubMed  PubMed Central  Google Scholar 

  39. Cochrane Community. Cochrane Review’s Altmetric score: understanding the value. https://community.cochrane.org/news/cochrane-review%E2%80%99s-altmetric-score-understanding-value. Accessed 13 May 2024.

  40. De Santis K, Muellmann S, Zeeb H. Use and acceptance of digital health offers in Germany: survey of adults insured by a nationwide health insurance provider. Gesundheitswesen. 2023;85:759. https://0-doi-org.brum.beds.ac.uk/10.1055/s-0043-1770422.

    Article  Google Scholar 

  41. Diel S, Doctor E, Reith R, Buck C, Eymann T. Examining supporting and constraining factors of physicians’ acceptance of telemedical online consultations: a survey study. BMC Health Serv Res. 2023;23(1):1128. https://0-doi-org.brum.beds.ac.uk/10.1186/s12913-023-10032-6.

    Article  PubMed  PubMed Central  Google Scholar 

  42. Posselt J, Baumann E, Dierks M-L. A qualitative interview study of patients’ attitudes towards and intention to use digital interventions for depressive disorders on prescription. Frontiers in Digital Health. 2024;6:1275569. https://0-doi-org.brum.beds.ac.uk/10.3389/fdgth.2024.1275569.

    Article  PubMed  PubMed Central  Google Scholar 

  43. De Santis KK, Mergenthal L, Christianson L, Busskamp A, Vonstein C, Zeeb H. Digital technologies for health promotion and disease prevention in older people: scoping review. J Med Internet Res. 2023;25:e43542. https://0-doi-org.brum.beds.ac.uk/10.2196/43542.

    Article  PubMed  PubMed Central  Google Scholar 

  44. Wilson J, Heinsch M, Betts D, Booth D, Kay-Lambkin F. Barriers and facilitators to the use of e-health by older adults: a scoping review. BMC Public Health. 2021;21(1):1556. https://0-doi-org.brum.beds.ac.uk/10.1186/s12889-021-11623-w.

    Article  PubMed  PubMed Central  Google Scholar 

  45. Budd J, Miller BS, Manning EM, Lampos V, Zhuang M, Edelstein M, Rees G, Emery VC, Stevens MM, Keegan N, et al. Digital technologies in the public-health response to COVID-19. Nat Med. 2020;26(8):1183–92. https://0-doi-org.brum.beds.ac.uk/10.1038/s41591-020-1011-4.

    Article  CAS  PubMed  Google Scholar 

  46. World Health Organization. Cardiovascular diseases (CVDs). 2021. https://www.who.int/news-room/fact-sheets/detail/cardiovascular-diseases-(cvds). Accessed 13 May 2024.

  47. Forberger S, Wichmann F, Comito CN. Nudges used to promote physical activity and to reduce sedentary behaviour in the workplace: results of a scoping review. Prev Med. 2022;155:106922. https://0-doi-org.brum.beds.ac.uk/10.1016/j.ypmed.2021.106922.

    Article  PubMed  Google Scholar 

  48. Bondevik JN, Bennin KE, Babur Ö, Ersch C. A systematic review on food recommender systems. Expert Syst Appl. 2024;238:122166. https://0-doi-org.brum.beds.ac.uk/10.1016/j.eswa.2023.122166.

    Article  Google Scholar 

Download references

Acknowledgements

We thank Irma Klerings (librarian) from the Cochrane Public Health Europe network for feedback on the search strategy and team assistants (Lisa Stiens and Indira Tramm at BIPS) for their help with data entry and data checking.

Funding

Open Access funding enabled and organized by Projekt DEAL. This study did not receive any funding.

Author information

Authors and Affiliations

Authors

Contributions

K.K.D.S., conceptualisation, methodology, data curation (study selection, data processing), data analysis, data visualisation, writing—first draft, and writing—review and editing. M.K., conceptualisation, methodology, data curation (study selection, data extraction, data processing), and writing—review and editing. C.K., conceptualisation, methodology, writing—first draft, and writing—review and editing. U.G., conceptualisation, methodology, writing—first draft, and writing—review and editing. S.M., writing—first draft and writing—review and editing. T.J., conceptualization, methodology, data curation (study selection), writing—first draft, and writing—review and editing. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Karina Karolina De Santis.

Ethics declarations

Ethics approval and consent to participate

Not applicable.

Consent for publication

Not applicable.

Competing interests

The authors declare that they have no competing interests.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

De Santis, K.K., Kirstein, M., Kien, C. et al. Online dissemination of Cochrane reviews on digital health technologies: a cross-sectional study. Syst Rev 13, 133 (2024). https://0-doi-org.brum.beds.ac.uk/10.1186/s13643-024-02557-6

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://0-doi-org.brum.beds.ac.uk/10.1186/s13643-024-02557-6

Keywords