Editor—We would extend Olsen et al's observations on Cochrane reviews.^1-1 Last year we undertook a study of the utility of the Cochrane Database of Systematic Reviews in informing health policy and practice.^1-2 We produced summary documents listing the conditions or diseases reviewed; the statements of evidence and effect; and, where available, conclusions for policy and practice for the reviews from collaborative review groups that covered cancer (including tobacco addiction), vascular disease, and fractures. In assessing the Cochrane reviews we scrutinised high profile sections (review title, abstract, objectives, conclusions, synopsis), just as a busy healthcare professional would do.

When necessary we inspected other sections of the review. Although we did not critically appraise review methodology, we recorded any errors, discrepancies (including discordance between the conclusions of effect and the available evidence), and other items needing clarification. We reported such information direct to the coordinators of the collaborative review groups.

We sent specific comments on 62 of the 159 reviews processed in Issue 2, 2000, of the Cochrane Library. Although most comments were minor, the inappropriate interpretation of results leading to spurious conclusions was considered likely in two reviews, the disregard of problems with the unit of analysis was thought likely in four, and sections were missing in two. Failure to collect outcome data on adverse effects of treatment and quality of life and function was also commented on in several reviews.

Our experience confirms that most Cochrane reviews are of a good standard. This is a considerable achievement, especially given the unpaid and voluntary nature of the work. The regularly updated electronic publication and the comments and criticisms facility offer great advantages. For instance, in cases where reviews with serious defects cannot be remedied speedily their temporary removal is important. Like Olsen et al, we emphasise the importance of feedback from users of the Cochrane Library.

Finally, Olsen et al conclude that readers should themselves assess the reliability of individual Cochrane reviews, and they emphasise the need to learn the skills of critical appraisal. We support their recommendation, but we are concerned that this may seem like advice to let the buyer beware. Given the broad readership (including lay people) of Cochrane reviews, the main emphasis must be on good quality and reliable reviews that people can trust.