Skip to main content

Table 1 ROBINS-E user experience themes and concerns

From: The risk of bias in observational studies of exposures (ROBINS-E) tool: concerns arising from application to observational studies of exposures

1. Comparison to an ‘ideal’ randomized controlled trial (RCT)

RCTs are not available for exposure studies and, therefore, not relevant to decision makers who must rely on observational studies of exposures.

Assessing observational studies based on RCTs results in a default rating of high risk of bias

Some of the questions derived from evaluating RCTs of interventions are inappropriate or impossible to apply for observational studies.

Sources of bias specific to observational studies may not be captured by comparison to an RCT.

2. Inadequate assessment of bias related to confounding

Does not capture bias related to over-adjustment for confounders or inappropriate modelling of confounders.

Does not capture advantages of newer statistical methods used for control for confounding.

Clearer guidance is needed on method for identifying confounders.

Does not differentiate between confounders, co-exposures and complex exposures.

3. Inadequate assessment of bias related to measurement of exposure

Assessment is limited to validity and reliability of the measurement, and these concepts are not clearly defined.

4. Use of an overall risk of bias rating

Does not distinguish between studies that have a ‘serious’ risk of bias in one domain and those that have multiple ‘serious’ risks of bias.

Assumes all risk of bias domains are weighted equally.

5. Additional risks of bias relevant to observational studies are not assessed (e.g. funding source)

6. Signalling questions

Do not consistently help the raters come to a consensus on how to rate a bias domain.

Specific questions unclear or confusing.

7. Practical considerations

Time-consuming to use.

There are limitations of using a single tool to rate different study designs.