Last month a Letter was published in the peer-reviewed journal Medical Teacher titled, “Is Wikipedia unsuitable as a clinical information resource for medical students? “ [1].  That paper came on the heels of a Letter published in The Annals of Pharmacotherapy on a related topic, “Evaluation of pharmacist use and perception of Wikipedia as a drug information resource“ [2].  The Annals paper had some serious shortcomings (e.g., survey response rate) which likely contributed to its abbreviated publication form.  Its most eye-opening point was that only one-third of the respondents who used Wikipedia were aware that anyone could edit the entries.  This is perhaps the real value of the Letter and why it was published – it helps illustrate the need for education about appropriate online resources in that group.

The Pender, et al. paper also has some methodological aspects that probably limited it to a Letter.  For those without access to Medical Teacher, the results were initially presented as a conference case study.  Because the work of Pender, et al. was accepted for publication, and reminiscent of the Annals Letter, it went on to generate quite a bit of interest among academics, clinicians, librarians, and social media enthusiasts.  The unfortunate thing about this interesting topic is, like all Letters, the available level of detail was below what the authors envisioned and the readers sought.  However, in this case, the dialogue it has helped stimulate may be as valuable as the research itself.  Because I am in the midst of working on a follow-up study to the Wikipedia study we did a couple years ago [3], I searched for more information about the paper – some of which is discussed below.  (As an aside, the new wiki study is the first I’ve worked on that was partially driven by ‘unsolicited, crowdsourced post-publication peer review’.  More anon.)

In the Pender, et al. study, three content experts each evaluated one medical topic according to five criteria (e.g., accuracy, suitability) in Wikipedia, eMedicine, AccessMedicine, and UpToDate.  All criteria used a three point rating scale.  For example, the accuracy scale was: 1=numerous important errors, 2 = some minor factual errors, and 3 = no factual errors.  Accuracy scores for Wikipedia on this scale were 3, 2, and 2 for the topics.  Interestingly, eMedicine scored the single lowest accuracy rating for a topic (rating of 1) of any of the resources.  It did perform well for the other two topics.  Wikipedia fared even worse for suitability with all three topics rated as ‘1’ (“unsuitable”).  An example of the full scores for the otitis media topic is complements of @LKruesi and detailed below. 

 

Data resources

 

Wikipedia

UpToDate

eMedicine

AccessMedicine

Otitis Media

Accuracy

2

3

2

3

Coverage

3

3

2

3

Concision

2

1

3

1

Currency

3

3

2

3

Suitability

1

2

1

1

 

Two librarians led the project, blinded the resource entries for the content experts, and assessed each resource for accessibility and usability.  They used criteria like cost, ease of finding information, and presentation quality to support their decisions.  Wikipedia did very well here, earning the distinction as the most accessible and easiest to use. 

Ultimately, I think this study adds to the literature and has already contributed to the wider community by sparking debate and discussion.  I hope this supplemental information helps address some of the questions I have seen about this study and thanks again to Lisa Kruesi for the spirit of openness and transparency in electing to make the data available. 
 
UPDATE (7FEB10): The authors of the Med Teach letter have archived a full version with results and tables here.

 
   

 @kevinclauson

[1] Pender MP, Lasserre KE, Del Mar C, Kruesi L, Anuradha S. Is Wikipedia unsuitable as a clinical information resource for medical students? Med Teach. 2009;31(12):1095-6.

[2] Brokowski L, Sheehan AH. Evaluation of pharmacist use and perception of Wikipedia as a drug information resource. Ann Pharmacother. 2009;43(11):1912-3. 

 [3] Clauson KA, Polen HH, Boulos MN, Dzenowagis JH. Scope, completeness, and accuracy of drug information in Wikipedia. Ann Pharmacother. 2008;42(12):1814-21. 

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s