See this topic in the GRADE handbook: Indirectness of evidence
The text below is taken from the GRADE workinggroup official JCE series, article number 8:
GRADE guidelines: 8. Rating the quality of evidence—indirectness
Four types of indirectness
We are more confident in the results when we have direct evidence. By direct evidence, we mean research that directly compares the interventions in which we are interested delivered to the populations in which we are interested and measures the outcomes important to patients. Thus, we can have concerns about indirectness when the population, intervention, or outcomes differ from those in which we are interested. A fourth, different type of indirectness, occurs when there are no head-to-head comparisons between the alternative management strategies under comparison. Indirectness of outcomes and indirect comparisons are equally relevant to systematic reviews and practice guidelines; indirectness related to populations and interventions (sometimes referred to as applicability) is more relevant to guidelines.
Evidence is lower quality if comparisons are indirect
Question of interest: Oseltamivir for prophylaxis of avian flu caused by influenza A virus
Source of indirectness: Differences in population: Randomized trials of oseltamivir are available for seasonal influenza, but not for avian flu
Question of interest: Colonoscopic screening for prevention of colon cancer mortality
Source of indirectness: Differences in intervention: Randomized trials of fecal occult blood screening provide indirect evidence bearing on the potential effectiveness of colonoscopy
Question of interest: Sevelamer- vs. calcium-based phosphate binders in chronic renal failure
Source of indirectness: Differences in outcome: Reducing the calcium-phosphate load is hypothesized to reduce vascular calcification, which is hypothesized to reduce vascular events
Question of interest: Choice of antidepressant
Source of indirectness: Indirect comparison: Some antidepressants have been compared directly with others, but many have not
Indirectness: differences in outcome measures (surrogate outcomes)
GRADE specifies that both those conducting systematic reviews and those developing practice guidelines should begin by specifying every important outcome of interest. The available studies may have measured the impact of the intervention of interest on outcomes related to, but different from, those of primary importance to patients.
The difference between desired and measured outcomes may relate to time frame. For example, a systematic review of behavioral and cognitive-behavioral interventions for outwardly directed aggressive behavior in people with learning disabilities showed that a program of 3-week relaxation training significantly reduced disruptive behaviors at 3 months [3]. Unfortunately, no eligible trial assessed the review authors' predefined outcome of interest, the long-term impact defined as effect at 9 months or greater. The argument for rating down becomes even stronger when one considers that other types of behavioral interventions have shown an early beneficial effect that was not sustained at 6 months follow-up [3]. When there is a discrepancy between the time frame of measurement and that of interest, whether to rate down by one or two levels will depend on the magnitude of the discrepancy. In this case, one could argue for either option.
Another source of indirectness related to measurement of outcomes is the use of substitute or surrogate endpoints in place of the patient-important outcome of interest. Table 2 lists a number of such surrogate measures that are common in current clinical investigation"
Table 2. Examples of surrogate outcomes
Condition | Patient-important outcome(s) | Surrogate outcome(s) |
|---|
Diabetes mellitus | Diabetic symptoms, hospital admission, complications (cardiovascular, eye, renal, neuropathic) | Blood glucose, A1C |
Hypertension | Cardiovascular death, myocardial infarction, stroke | Blood pressure |
Dementia | Patient function, behavior, caregiver burden | Cognitive function |
Osteoporosis | Fractures | Bone density |
Adult Respiratory Distress Syndrome | Mortality | Oxygenation |
End-stage renal disease | Quality of life, morbidity (such as shunt thrombosis or heart failure), mortality | Hemoglobin |
Venous thrombosis | Symptomatic venous thrombosis | Asymptomatic venous thrombosis |
Chronic respiratory disease | Quality of life, exacerbations, mortality | Pulmonary function, exercise capacity |
Cardiovascular disease/risk | Vascular events, mortality |
Go to the orginal article for full text, or go to a specific chapter in the article: