Tool to assess Risk of Bias in Randomized Controlled Trials

This tool has been contributed by the CLARITY Group at McMaster University

It can be downloaded from Evidence Partners: Link to tools

Tool to Assess Risk of Bias in Randomized Controlled Trials

1. Was the allocation sequence adequately generated?

Definitely yes                    Probably yes                       Probably no                         Definitely no
 (low risk of bias)                                                                                                  (high risk of bias)

Examples of low risk of bias: Referring to a random number table; Using a computer random number generator; Coin tossing; Shuffling cards or envelopes; Throwing dice; Drawing of lots; Minimization with or without a random element. 

Examples of high risk of bias: Sequence generated by odd or even date of birth; Sequence generated by some rule based on date (or day) of admission; Sequence generated by some rule based on hospital or clinic record number; Allocation by judgement of the clinician; Allocation by preference of the participant; Allocation based on the results of a laboratory test or a series of tests; Allocation by availability of the intervention.

2. Was allocation adequately concealed?

Definitely yes                    Probably yes                       Probably no                         Definitely no
      (low risk of bias)                                                                                                   (high risk of bias)

Examples of low risk of bias allocation concealment techniques: Central allocation (including telephone, web-based, and pharmacy-controlled, randomization); 

Examples of possible low risk of bias: Sequentially numbered drug containers of identical appearance; Sequentially numbered, opaque, sealed envelopes. 

Examples of high risk of bias allocation generation techniques: Using an open random allocation schedule (e.g. a list of random numbers); Assignment envelopes were used without appropriate safeguards (e.g. if envelopes were unsealed or non-opaque or not sequentially numbered); Alternation or rotation; Date of birth; Case record number; Any other explicitly unconcealed procedure.

3. Blinding: Was knowledge of the allocated interventions adequately prevented?

(This global rating is challenging.  May want to omit and use only the ratings below)

Definitely yes                    Probably yes                       Probably no                         Definitely no
   (low risk of bias)                                                                                                      (high risk of bias)

Examples of low risk of bias: No blinding, but the review authors judge that the outcome and the outcome measurement are not likely to be influenced by lack of blinding; Blinding of participants and key study personnel ensured, and unlikely that the blinding could have been broken; Either participants or some key study personnel were not blinded, but outcome assessment was blinded and the nonblinding of others unlikely to introduce bias.

Examples of high risk of bias: No blinding or incomplete blinding, and the outcome or outcome measurement is likely to be influenced by lack of blinding; Blinding of key study participants and personnel attempted, but likely that the blinding could have been broken; Either participants or some key study personnel were not blinded, and the non-blinding of others likely to introduce bias.

3.a)  Were patients blinded?

Definitely yes                    Probably yes                       Probably no                         Definitely no 

3.b). Were healthcare providers blinded?

Definitely yes                    Probably yes                       Probably no                         Definitely no

3.c). Were data collectors blinded?

Definitely yes                    Probably yes                       Probably no                         Definitely no

3.d). Were outcome assessors blinded?

Definitely yes                    Probably yes                       Probably no                         Definitely no

3.e). Were data analysts blinded?

Definitely yes                    Probably yes                       Probably no                         Definitely no

4. Was loss to follow-up (missing outcome data) infrequent?

Definitely yes                    Probably yes                       Probably no                         Definitely no
         (low risk of bias)                                                                                                     (high risk of bias)

Examples of low risk of bias: No missing outcome data; Reasons for missing outcome data unlikely to be related to true outcome (for survival data, censoring unlikely to be introducing bias); Missing outcome data balanced in numbers across intervention groups, with similar reasons for missing data across groups; For dichotomous outcome data, the proportion of missing outcomes compared with observed event risk not enough to have a important impact on the intervention effect estimate; For continuous outcome data, plausible effect size (difference in means or standardized difference in means) among missing outcomes not enough to have an important impact on observed effect size; Missing data have been imputed using appropriate methods.

Examples of high risk of bias: Reason for missing outcome data likely to be related to true outcome, with either imbalance in numbers or reasons for missing data across intervention groups; For dichotomous outcome data, the proportion of missing outcomes compared with observed event risk enough to induce important bias in intervention effect estimate; For continuous outcome data, plausible effect size (difference in means or standardized difference in means) among missing outcomes enough to induce clinically relevant bias in observed effect size; ‘As-treated’ analysis done with substantial departure of the intervention received from that assigned at randomization; Potentially inappropriate application of simple imputation.

5. Are reports of the study free of suggestion of selective outcome reporting?.

(This item sufficiently difficult to judge that may be omitted)

Definitely yes                    Probably yes                       Probably no                         Definitely no
          (low risk of bias)                                                                                                     (high risk of bias)

Examples of high risk of bias: Not all of the study’s pre-specified primary outcomes have been reported; One or more primary outcomes is reported using measurements, analysis methods or subsets of the data (e.g. subscales) that were not pre-specified; One or more reported primary outcomes were not pre-specified (unless clear justification for their reporting is provided, such as an unexpected adverse effect); One or more outcomes of interest in the review are reported incompletely so that they cannot be entered in a meta-analysis; The study report fails to include results for a key outcome that would be expected to have been reported for such a study

6. Was the study apparently free of other problems that could put it at a risk of bias?

(May omit this item)

Definitely yes                    Probably yes                       Probably no                         Definitely no
          (low risk of bias)                                                                                                    (high risk of bias)

Examples of low risk of bias: The study appears to be free of other sources of bias.

Examples of high risk of bias: Had a potential source of bias related to the specific study design used; Stopped early due to some data-dependent process (including a formal-stopping rule); Had extreme baseline imbalance; Has been claimed to have been fraudulent; Had some other problem.