Wikipedia isn’t good enough for anybody except nurses?

The verdict is in.  The quality of health information in Wikipedia is inadequate as a sole source for pharmacists [1], medical students [2], dentists [3], and patients [4].  However, it is good enough for use by nursing students [5]…well, sort of.

Determinations about adequacy are based on studies which evaluated the freely editable, online encyclopedia based on characteristics such as reliability, scope, and accuracy.  A clear consensus has emerged from that body of literature collectively rendering a decision that Wikipedia is not a suitable resource for high level consultation or citation.  The use (and citation in particular) of Wikipedia by healthcare students and professionals seems to irk practitioners and educators moreso when there are high quality alternatives, suggesting the perception that citing Wikipedia in those cases simply reflects a lack of awareness and laziness.  To be fair, Wikipedia founder Jimmy Wales has been unwavering in his stance that no encyclopedia should be used as a reference source for college level work or above. 

All of this leads to the most recent paper on this topic in Nurse Education Today [5].  It, too, is an assessment of health-related entries in Wikipedia.  However, it is notable for two reasons.  First, it differs from almost all the other articles in that it uses a methodology driven by compiling and analyzing the citations from Wikipedia entries rather than the content itself.  Second, the difference in language between the abstract and conclusion is fascinating.  The abstract closes with:

The quality of the evidence taken obtained from the 2500 plus references from over 50 Wikipedia pages was of sufficiently sound quality to suggest that, for health related entries, Wikipedia is appropriate for use by nursing students

Whereas the conclusion of the article is:

Whilst it is acknowledged that Wikipedia citations should be treated with some caution, the results of this modest study suggest that Wikipedia does have a role to play as a source of health related evidence for use by nursing students.

While this type of journal article ‘abstract-text dissonance’ is not completely rare [6,7] it is exacerbated here due to this article’s findings conflicting with every other study on the topic – at least based the abstract.  It also magnifies the problems that can occur when people draw conclusions based on only reading an abstract.  This has long been an issue for busy clinicians desperately trying to stay current.  However, today biomedical journal abstracts are easily accessible by anyone via PubMed (but the full-text usually remains shrouded by subscription access).  Dissemination of these abstracts is even more rapid, sometimes occurring real-time through tools like Twitter, Facebook, and email.  As healthcare professionals, we need to be careful not to fall into the trap of taking a shortcut and assuming skimming an abstract will allow us to critically evaluate a study.  The onus is also on us to help educate aspiring e-patients avoid these same missteps.

Overall, it’s quite possible that the contribution made by the Haigh article may be more significant as a teaching tool than as a piece of the research puzzle regarding the quality of Wikipedia.

@kevinclauson

P.S.  If you have gotten this far, it means you have not fallen prey to a similar phenomena with blog post titles – kudos!


[1] Clauson KA, Polen HH, Boulos MN, Dzenowagis JH. Scope, completeness, and accuracy of drug information in Wikipedia. Ann Pharmacother. 2008 Dec;42(12):1814-21.
[2] Pender MP, Lasserre KE, Del Mar C, Kruesi L, Anuradha S. Is Wikipedia unsuitable as a clinical information resource for medical students? Med Teach. 2009 Dec;31(12):1095-6.
[3] Stillman-Lowe C. Wikipedia comes second. Br Dent J. 2008 Nov 22;205(10):525.
[4] Leithner A, Maurer-Ertl W, Glehr M, Friesenbichler J, Leithner K, Windhager R. Wikipedia and osteosarcoma: a trustworthy patients’ information? J Am Med Inform Assoc. 2010 Jul-Aug;17(4):373-4.
[5] Haigh CA. Wikipedia as an evidence source for nursing and healthcare students. Nurse Educ Today. 2010 Jun 19. [Epub ahead of print]
[6] Ward LG, Kendrach MG, Price SO. Accuracy of abstracts for original research articles in pharmacy journals. Ann Pharmacother. 2004 Jul-Aug;38(7-8):1173-7.
[7] Pitkin RM, Branagan MA, Burmeister LF. Accuracy of data in abstracts of published research articles. JAMA. 1999 Mar 24-31;281(12):1110-1.

Pharmacists’ duty to warn in the age of social media

Healthcare in general and pharmacy in particular, is still finding its way with social media.  One of the least developed elements of Health 2.0 remains the legal aspect.  A few years ago several of us starting discussing scenarios in which a legally valid pharmacist-patient relationship might be created based exclusively on Web 2.0 mediated interactions.  This discussion has been aided, of course, by social media.  Also, as part of an interactive panel at Medicine 2.0 a couple years ago we posed this question (attendees’ responses here).

This discussion has recently been formalized as a Commentary published along with Matthew Seamon PharmD, JD and Brent Fox, PharmD, PhD (@Brent_Fox) in the American Journal of Health System Pharmacy.  An accompanying podcast has also been produced for it by AJHP.  Ideally the article and podcast help promote dialogue and encourage the profession to think proactively on the subject.

@kevinclauson

Federal Pharmacy Forum, PAHO, OAS, APhA and Other Acronyms

 
 

I love living in South Florida, but after this week DC may be my new mistress.  I find the magnitude of what is going on here bordering on unbelievable.  I’ve been here previously, but have never really had an opportunity to take a look around.  I was also struck by the sheer volume of living history in DC.  There are some pretty good restaurants as well from Matchbox (introduced to me by @lostonroute66) to Zaytinya.

Having just returned from a health promotion conference in Puerto Rico, I was able to visit with several folks at the Pan American Health Organization (PAHO) to explore opportunities during this visit.  Some I had met here previously and some were new to me.  There was a common thread throughout the PAHO folks in that they are driven, but warm people…and it is always nice to see the face behind @eqpaho.  I also learned a lot about the Organization of American States (OAS).  I met with a couple people there as well to discuss some upcoming projects.  Finally, I had the opportunity to open the second day of presenters for the Federal Pharmacy Forum.  The Forum directly precedes the American Pharmacists Association (APhA) Annual Meeting (thashtag #apha2010).  The Forum was very much an eye-opening experience for me as the attendees were primarily military pharmacists and technicians with other representatives from the Federal sector as well.  I presented “Debunking Myths About Generational Use of Social Media and Healthcare“.

The keynote on the first day of the Forum (Ginny Beeson) did a great job of laying the groundwork for a dialogue about generational considerations in military pharmacy.  There were also several other topics particularly of interest to me including telepharmacy in the Navy (apparently each branch has a different certification body making widespread rollout across branches almost impossible), pharmacoeconomic studies, medication therapy management (MTM), deployment of pharmacists and the current state of practice of US military pharmacy in Afghanistan.  It definitely made me more thankful for the setting I operate out of and appreciate the challenges of this segment of the profession.

Overall it was a great visit, I learned a lot (including how little I thought I knew), and am looking forward to my next visit.

@kevinclauson

Posted via email from kevinclauson’s posterous

Superiority, Equivalence, and Non-inferiority Trial Design Lecture – Web 2.0 Style

A couple of weeks ago I read a very thoughtful post on one of my favorite blogs (authored by @laikas).  One reason I enjoy Jacqueline’s blog is that it contains evaluation (or at minimum consideration) of the information and literature that its posts are written about…rather than simply repackaging it without any context.  I do believe there is also clear value for blog posts that primarily serve current awareness, etc.  However, when a blog is augmented with the occasional reflective or methodical examination, it is elevated to a different level IMHO.  It’s somewhat akin to the difference in nominations for the medGadget Awards versus the Research Blogging Awards (chapeau tip to @DrVal).  The information found in both award contenders is valuable, but serves different purposes.     

In any event, the post about randomized, controlled trials (RCTs) and evidence based medicine (EBM) on Laika’s MedLibLog made me reconsider a lecture I had just finished putting together on trial design for a Drug Literature Evaluation course.  In that course, there are several lectures on different types of trial design as well as separate ones about EBM and clinical practice guidelines (CPGs) –  but they are all probably a little too insular.  Each lecture may be a little too focused on the technical aspects, and guilty of ignoring how each fits into the grand scheme of things.

The Point Of All This

When I first prepared my lecture on trial design, I think I was too preoccupied with conveying the importance of fundamentals to the students.  Reading the aforementioned post made me re-think my approach a bit and prompted me to revise the lecture/slides to incorporate a little more integration and application.  To that end, I posted them on Slideshare.  For me, this sequence of events is a perfect example of the underlying concepts of Web 2.0 in action…and that’s pretty cool.

@kevinclauson

P.S. I used the Guided Notes approach (which is why it looks like there is a preponderance of underlined text) and an audience response system in this lecture.

Medical Information Resource Deathmatch – A Closer Look

Last month a Letter was published in the peer-reviewed journal Medical Teacher titled, “Is Wikipedia unsuitable as a clinical information resource for medical students? “ [1].  That paper came on the heels of a Letter published in The Annals of Pharmacotherapy on a related topic, “Evaluation of pharmacist use and perception of Wikipedia as a drug information resource“ [2].  The Annals paper had some serious shortcomings (e.g., survey response rate) which likely contributed to its abbreviated publication form.  Its most eye-opening point was that only one-third of the respondents who used Wikipedia were aware that anyone could edit the entries.  This is perhaps the real value of the Letter and why it was published – it helps illustrate the need for education about appropriate online resources in that group.

The Pender, et al. paper also has some methodological aspects that probably limited it to a Letter.  For those without access to Medical Teacher, the results were initially presented as a conference case study.  Because the work of Pender, et al. was accepted for publication, and reminiscent of the Annals Letter, it went on to generate quite a bit of interest among academics, clinicians, librarians, and social media enthusiasts.  The unfortunate thing about this interesting topic is, like all Letters, the available level of detail was below what the authors envisioned and the readers sought.  However, in this case, the dialogue it has helped stimulate may be as valuable as the research itself.  Because I am in the midst of working on a follow-up study to the Wikipedia study we did a couple years ago [3], I searched for more information about the paper – some of which is discussed below.  (As an aside, the new wiki study is the first I’ve worked on that was partially driven by ‘unsolicited, crowdsourced post-publication peer review’.  More anon.)

In the Pender, et al. study, three content experts each evaluated one medical topic according to five criteria (e.g., accuracy, suitability) in Wikipedia, eMedicine, AccessMedicine, and UpToDate.  All criteria used a three point rating scale.  For example, the accuracy scale was: 1=numerous important errors, 2 = some minor factual errors, and 3 = no factual errors.  Accuracy scores for Wikipedia on this scale were 3, 2, and 2 for the topics.  Interestingly, eMedicine scored the single lowest accuracy rating for a topic (rating of 1) of any of the resources.  It did perform well for the other two topics.  Wikipedia fared even worse for suitability with all three topics rated as ‘1’ (“unsuitable”).  An example of the full scores for the otitis media topic is complements of @LKruesi and detailed below. 

 

Data resources

 

Wikipedia

UpToDate

eMedicine

AccessMedicine

Otitis Media

Accuracy

2

3

2

3

Coverage

3

3

2

3

Concision

2

1

3

1

Currency

3

3

2

3

Suitability

1

2

1

1

 

Two librarians led the project, blinded the resource entries for the content experts, and assessed each resource for accessibility and usability.  They used criteria like cost, ease of finding information, and presentation quality to support their decisions.  Wikipedia did very well here, earning the distinction as the most accessible and easiest to use. 

Ultimately, I think this study adds to the literature and has already contributed to the wider community by sparking debate and discussion.  I hope this supplemental information helps address some of the questions I have seen about this study and thanks again to Lisa Kruesi for the spirit of openness and transparency in electing to make the data available. 
 
UPDATE (7FEB10): The authors of the Med Teach letter have archived a full version with results and tables here.

 
   

 @kevinclauson

[1] Pender MP, Lasserre KE, Del Mar C, Kruesi L, Anuradha S. Is Wikipedia unsuitable as a clinical information resource for medical students? Med Teach. 2009;31(12):1095-6.

[2] Brokowski L, Sheehan AH. Evaluation of pharmacist use and perception of Wikipedia as a drug information resource. Ann Pharmacother. 2009;43(11):1912-3. 

 [3] Clauson KA, Polen HH, Boulos MN, Dzenowagis JH. Scope, completeness, and accuracy of drug information in Wikipedia. Ann Pharmacother. 2008;42(12):1814-21. 

Posted via web from kevinclauson’s posterous

Soapbox 2.0: Blog and microblog use by pharmacists for ranting and discourse

This poster project was driven by a couple of very motivated pharmacy students and recently presented in poster form at the ASHP Midyear Clinical Meeting.  I was prompted to post it because of a recent tweet by @SusannahFox about the Health On Net Code.  As with all posters, the complete results are not contained here.  Ironically, the fact that while about a third of pharmacist blogs contained disclaimers, only 7% contained evidence of HON certification is not included on this poster – despite that piece being the impetus for this post.  Of note, the Healthcare Blogger Code of Ethics seal was not similarly tracked at the time of the study.

The full manuscript will hopefully be published in a peer-reviewed journal in the not-too-distant future [Update: Article will be published in AJHP later in 2010].

@kevinclauson

Source:

Elkins J, Goncz CE, Somnarain R, Clauson KA. Soapbox 2.0: Blog and microblog use by pharmacists for ranting and discourse. Presented at the American Society of Health-System Pharmacists Midyear Clinical Meeting, Las Vegas, NV, December 2009.