Skip to main content

Response to ‘Increasing value and reducing waste in data extraction for systematic reviews: tracking data in data extraction forms’

The Original Article was published on 04 August 2017

Abstract

This is a response to a Letter. Data abstraction is a time-consuming and error-prone systematic review task. Shokraneh and Adams categorize available techniques for tracking data during data abstraction into three methods: simple annotation, descriptive addressing, and Cartesian coordinate system. While we agree with the categorization of the techniques, we disagree with the authors’ statement that descriptive addressing is a PDF-independent method, i.e., any sort of descriptive addressing must reference a specific version of PDF file and not just any PDF of said report. Different versions of PDFs of the same report might place text and tables on different locations of the same page and/or on different pages. Consequently, it is our opinion that any kind of source location information should be accompanied by the source or linked by an intermediary service such as the Data Abstraction Assistant (DAA).

Peer Review reports

We read with great interest Shokraneh and Adams’ letter pertaining to data abstraction (or “data extraction”) during systematic reviews [1]. We agree with the authors that data abstraction is perhaps the most time-consuming task during systematic reviews and one that is error-prone [2,3,4,5]. Manual data abstraction, which is largely the current norm in the systematic review enterprise, is likely not sustainable in the long run. Software tools that help tracking of data to published reports of studies (i.e., PDFs) have the potential to greatly reduce the time spent and errors inherent to the data abstraction process [6]. We developed Data Abstraction Assistant (DAA), a software tool with this potential. By recording the exact location and mapping this location to the data entered into extraction forms, DAA could reduce errors and time spent reviewing extracted data [6].

Shokraneh and Adams categorize available techniques for tracking data into three methods: simple annotation, descriptive addressing, and Cartesian coordinate system [1]. The authors describe the second method, i.e., descriptive addressing, as one where the data abstractor abstracts data and notes each data point’s source of information (“address”) in the PDF, using the page, paragraph, line, table, figure, box, and/or headline numbers.

While we agree with the categorization of the techniques, we disagree with the authors’ statement that descriptive addressing is “PDF-independent.” In our experience, descriptive addressing indeed is dependent on the version of the PDF used; different versions of PDFs of the same report might place text and tables on different locations of the same page and/or on different pages. Consequently, it is our opinion that any kind of source location information should be accompanied by the source or linked by an intermediary service such as DAA. We developed DAA to facilitate data tracking between data abstraction forms and PDFs, thereby possibly reducing errors and saving time [6]. DAA allows users to mark and record the exact location of information found on a PDF. The locations are linked to data elements on a data extraction form. DAA enables users to create a link between information extracted and its source location. We are currently analyzing the results of a randomized controlled trial that formally evaluates the effectiveness of DAA (compared with standard data abstraction approaches) in improving these outcomes (i.e., error rates and time).

The use of DAA would not solve the challenge that copyright poses in sharing PDFs. However, by serving as an intermediary, linking the abstracted data and the exact location in the PDF source, DAA facilitates the efficient tracking of abstracted data.

References

  1. Shokraneh F, Adams CE. Increasing value and reducing waste in data extraction for systematic reviews: tracking data in data extraction forms. Syst Rev. 2017;6:133.

    Article  Google Scholar 

  2. Buscemi N, Hartling L, Vandermeer B, Tjosvold L, Klassen TP. Single data extraction generated more errors than double data extraction in systematic reviews. J Clin Epidemiol. 2006;59(7):697–703.

    Article  PubMed  Google Scholar 

  3. Gøtzsche PC, Hróbjartsson A, Maric K, Tendal B. Data extraction errors in meta-analyses that use standardized mean differences. JAMA. 2007;298(4):430–7.

    Article  PubMed  Google Scholar 

  4. Jones AP, Remmington T, Williamson PR, Ashby D, Smyth RL. High prevalence but low impact of data extraction and reporting errors were found in Cochrane systematic reviews. J Clin Epidemiol. 2005;58(7):741–2.

    Article  PubMed  Google Scholar 

  5. Horton J, Vandermeer B, Hartling L, Tjosvold L, Klassen TP, Buscemi N. Systematic review data extraction: cross-sectional study showed that experience did not increase accuracy. J Clin Epidemiol. 2010;63(3):289–98.

    Article  PubMed  Google Scholar 

  6. Saldanha IJ, Schmid CH, Lau J, Dickersin K, Berlin JA, Jap J, Smith BT, Carini S, Chan W, De Bruijn B, Wallace BC, Hutfless SM, Sim I, Murad MH, Walsh SA, Whamond EJ, Li T. Evaluating data abstraction assistant, a novel software application for data abstraction during systematic reviews: protocol for a randomized controlled trial. Syst Rev. 2016;5(1):196.

    Article  PubMed  PubMed Central  Google Scholar 

Download references

Acknowledgements

Not applicable.

Funding

This work is funded by a PCORI contract (Contract Number: ME-1310-07009).

Availability of data and materials

Not applicable.

Author information

Authors and Affiliations

Authors

Contributions

JJ conceived this manuscript. JJ and IJS drafted this manuscript, and all authors provided input and approved the final version of this manuscript.

Corresponding author

Correspondence to Jens Jap.

Ethics declarations

Ethics approval and consent to participate

Not applicable.

Consent for publication

All authors provide consent for publication of this manuscript.

Competing interests

All authors are investigators of a Patient-Centered Outcomes Research Institute (PCORI)-funded project (Contract Number: ME-1310-07009) that consists of developing DAA and conducting a randomized controlled trial to evaluate its effectiveness.

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Jap, J., Saldanha, I.J., Smith, B.T. et al. Response to ‘Increasing value and reducing waste in data extraction for systematic reviews: tracking data in data extraction forms’. Syst Rev 7, 18 (2018). https://0-doi-org.brum.beds.ac.uk/10.1186/s13643-018-0677-x

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://0-doi-org.brum.beds.ac.uk/10.1186/s13643-018-0677-x