David J Woods
MPharm MRPharmS
Drug Information Consultant and Senior Teaching Fellow
School of Pharmacy
University of Otago
Dunedin
New Zealand
Critical appraisal is the process of questioning the validity and applicability of information and scientific investigations. Critical appraisal is usually recognised in the context of clinical studies for which there is a conventional hierarchy of evidence (see Table 1). Checklists have been published to guide practitioners through the critical appraisal process (see examples in Table 2), where the emphasis is on identifying sources of potential bias in the conduct and design of the study (internal validity) and the relevance of the results to actual practice in a “real” population (external validity).
[[HPE05_table1_18]]
[[HPE05_table2_19]]
Many journals are now available full-text on the internet, allowing practitioners rapid access to the latest trials. Media summaries of these trials are also available online, and some of these may cause concern to patients. The drug in question is likely to be promoted in journal advertisements, websites and direct to consumer media in some countries. A practitioner must be able to efficiently critically appraise all these sources of information in order to give a balanced and informative viewpoint. Guidelines for the evaluation of web pages have been published (see Table 2). The following comments are made in the context of drug information on the internet.
For more information and links see http://www.pharminfotech.co.nz/hpe.htm.
Currency
Websites pose a significant challenge when it comes to an assessment of currency, but good sites always have the date of latest update on the homepage. There are thousands of websites that started off as good ideas but have not been maintained and remain idle on a server to be picked up by search engines.
A developing publishing strategy is to set up a companion website with the publication (textbook or CD) so that updates and addenda can be posted on the website or emailed to registrants. For example, see the “Red Book” updates on Hale’s “Drugs in Lactation” website.(1) Other developments are complete web-based resources available on subscription, and CD products with hyperlinks to web-based updates.
Accuracy
Even the best textbooks contain errors or inaccurate statements, and the same applies to the most reputable websites. However, the unregulated nature of the internet compounds this problem, and inaccuracies range from genuine mistakes to misinformation and potentially dangerous errors. Information on a site’s purpose and the author’s credentials (authority) is a useful, but ultimately it is necessary to maintain an inquiring mind and to digest the information in conjunction with your own knowledge.
It is often relatively easy to crosscheck the information on a website with a current textbook and a Medline search. This can quickly indicate the currency and accuracy of the information. For example, visit www.perinatology.com(2) and compare the pregnancy and lactation information in the drug monographs with the latest copy of Briggs(3) and a Medline search. In your opinion, how useful is this website as a quick and accessible reference source?
Relevance and clinical utility
Is the information provided applicable or relevant to the clinical scenario that you are dealing with?
The ability to assess the relevance and clinical utility of information is vital, and this reflects the knowledge of the practitioner. For example, a website could list a drug as having a milk:plasma (M:P) ratio of 2.5 and imply that the drug is unsafe to administer to a breastfeeding mother. The information about the M:P ratio may be factually correct but it is of minor relevance without an estimate of the weight- adjusted infant dose. Thus the clinical utility of this M:P ratio is very limited without more detail.
Coverage
- How much detail does the website provide, and does it cover all the important issues?
- Does a promotional site leave out important information about adverse reactions?
There are numerous drug databases available on the internet, but these are often quite superficial and, for example, may not provide comprehensive information about drug interactions and adverse reactions.
Many websites are compilations and summaries of primary literature, and if the authors failed to perform a comprehensive search and evaluation this will compromise the quality of the final publication.
Accessibility and design.
A website should be readily accessible and easy to use with good navigation design. Websites that are slow to load or often offline are very frustrating to use.
Cost
Cost is an obvious consideration, but be prepared to pay for quality. Several sites have started as free resources but now require paid subscription.
Nevertheless, there are some excellent free resources available now on the internet, and it is possible to use essentially free web-based resources for the majority of commonly encountered situations. To do this requires efficient searching and evaluation skills, and organisational ability.
References
- Hale T. Breastfeeding pharmacology [online]. Available from URL: http://neonatal.ama.ttuhsc.edu/lact/index.html.
- Perinatology.com [online]. Available from URL: http://www.perinatology.com/exposures/druglist.htm.
- Briggs GG, Freeman RK, Yaffe SJ. Drugs in pregnancy and lactation. 6th ed. Baltimore: Williams & Wilkins; 2001.
- Greenhalgh T. How to read a paper. Getting your bearings. BMJ 1997;315:243-6.
- Grimes DA, Schulz KF. An overview of clinical research: the lay of the land. Lancet 2002;359:57-61.
Resources
Pharminfotech
Medicines
information, technology and eLearning software
W:www.pharminfotech.co.nz
Medline (PubMed)
W:www.ncbi.nlm.nih.gov/entrez/query.fcgi
