Skip to main content

Table 2 Adapted GRRAS tool for quality assessment of measurement comparison studies

From: Availability and performance of image/video-based vital signs monitoring methods: a systematic review protocol

Section

Original guidelines from GRRAS

Scoring

Title and abstract

Identify in title or abstract that interrater/intrarater reliability or agreement was investigated

N/Aa

 

Introduction

Name and describe the diagnostic or measurement device of interest explicitly

The image-based vital signs monitor is described

+

No description

–

Specify the subject population of interest

Specifies the subject population of interest

+

No population specified

–

Specify the rater population of interest (if applicable)

N/A

 

Describe what is already known about reliability/agreement and provide a rationale for the study

Describes what is already known about the reliability of image-based monitoring method and provide a rationale for the study

+

No statement on the current knowledge of the method and no rationale stated

–

Methods

Explain how the sample size was chosen. State the determined number of raters/subjects/objects/replicate observations

Explains how the sample size was chosen and/or state the determined number of subjects/replicate observationsb

+

No explanation of sample size. Number of subjects/replicate observations not stated

–

Describe sampling method

Describes sampling method

+

No description of sampling method

–

Describe the measurement/rating process (e.g. time interval between repeated measurements, availability of clinical information, blinding)

Describes the measurement process

+

No description of measurement process

–

State whether measurements/ratings were conducted independently

Two (or more) methods of measurements conducted independently

+

Measurements not conducted independently

–

Describe the statistical analysis

Describes the statistical analysis planned

+

No description of statistical analysis planned

–

Results

State the actual number of raters and subjects/objects which were included and the number of replicate observations which were conducted

States the actual number of subjects who were included and number of observations (e.g. duration of recording)

+

No statement of the number of subjects/observations

–

Describe the sample characteristics of raters/subjects

N/A

 

Report estimates of reliability and agreement including measure of statistical uncertainty

Reports estimates of reliability and agreement including measure of statistical uncertainty using gold standard measures

+

Reports estimates of reliability and agreement including measure of statistical uncertainty using acceptable standard measures

?

Reports estimates of reliability and agreement including measure of statistical uncertainty but using inappropriate measures/no estimates of reliability/agreement given

–

Discussion

Discuss the practical relevance of results

Discusses the practical relevance of results

+

No discussion of practical relevance of results

–

Auxillary material

Provide detailed results if possible (e.g. online)

N/A

 
  1. N/A not applicable
  2. aStudies will not be penalised for not stating the gold standard/reference method of monitoring in the title/abstract as we aim to include studies where this comparison was performed with an alternative primary aim (e.g. improve image analysis protocol)
  3. bPilot studies will not be penalised for not stating how the sample size was chosen