What to do with a paper which only reports P-values, without any other data

We would distinguish two kinds of situations:

    a) an effect estimates is provided (e.g. RR, RD, OR) along with just a p-value, but no confidence interval

    b) there are no effect estimates, just p-values.

Both situation are problematic, and we should ideally look for better kind of evidence in both cases. However, we would suggest you should always ignore (b) as evidence, as it says nothing of the effect, and just depends on the sample size.

Situation (a) is also problematic, because the p-value alone, do not allow us to easily appraise the (im)precision of the effect (i.e. one of the key dimensions in GRADE). This is why MAGIC cannot help you automatically calculate the confidence intervals, etc. This is a methods issue, not a technical /logistic issue of the software.

However, if no better evidence is found, and you still wish to proceed with summarizing and appraising this evidence, it is sometimes possible to compute or at least estimate the sample sizes. One needs to know:

- the statistical test that were performed

- the SD or SE in both study arms and/or for the difference.

- the number of patients in each arm and/or in both arms combined

- the exact p-value (not just signifiant or not significant)

With this information, a statistician can back-calculate the CI for the difference. The skills needed depend on the complexity (e.g. It is quite hard, and often impossible in case of adjustments / multivariable models, for which lots of info are often missing.)

Assuming you succeed with this calculation, you would still need to provide a baseline risk in your evidence summary, in order to assess the absolute effect estimates, and move from evidence to recommendation.

For all this reasons, it is better if you could obtain better evidence than one reporting just p-values, and we would urge you to look elsewhere or contact the authors of the paper to get more details.

We are very sorry this isn't simpler, and we look forward to the day complete results from studies are reported as a standard.