Photo: Flickr user lifeontheedge

Saturday, December 17, 2005

English words with uncommon properties

Nature to scientsts: edit Wikipedia!

So can Wikipedia move up a gear and match the quality of rival reference works? Imagine the result if it did: a comprehensive, accurate and up-to-date reference work that can be accessed free from Manhattan to rural Mongolia. To achieve this, Wikipedia's administrators will have to tackle everything from future funding problems — the site is maintained by public donations [note from Wikipedia blog: Wikimedia's running a fund drive until January 8th] — to doubts about whether enough new contributors can be found to increase the quality of the mushrooming number of entries. That latter point is critical, and here scientists can make a difference.

Judging by a survey of Nature authors, conducted in parallel with the accuracy investigation, only a small percentage of scientists currently contribute to Wikipedia. Yet when they do, they can make a significant difference. Wikipedia's non-expert contributors are, by and large, dedicated to getting things right on the site. But scientists can bring a critical eye to entries on subjects they study, often highlighting errors and misunderstandings that others have unintentionally introduced. They can also start entries on topics that other users may not want to tackle. It is no surprise, for example, that the entry on 'spin density wave' was originated by a physicist."


The article also tackles wikipedia protocol -- how to get your edits to stick.

Editing pages is not always straightforward, as some users may disagree with changes. In politically sensitive areas such as climate change, researchers have had to do battle with sceptics pushing an editorial line that is out of kilter with mainstream scientific thinking. But this usually requires no more than a little patience. Wikipedia's users are generally interested in the reasoning behind proposed changes to articles. Backing up a claim with a peer-reviewed reference, for example, makes a world of difference.

Nature would like to encourage its readers to help. The idea is not to seek a replacement for established sources such as the Encyclopaedia Britannica, but to push forward the grand experiment that is Wikipedia, and to see how much it can improve. Select a topic close to your work and look it up on Wikipedia. If the entry contains errors or important omissions, dive in and help fix them. It need not take too long. And imagine the pay-off: you could be one of the people who helped turn an apparently stupid idea into a free, high-quality global resource.


(Thanks to the readers who told me about this article.)

Thursday, December 15, 2005

Nature magazine: Wikipedia's scientific accuracy comes close to Britannica's, 1 in 10 Nature authors are wikipedians.

Wales: More expert wikipedians still needed; some articles will eventually have both static and dynamic versions.

The exercise revealed numerous errors in both encyclopaedias, but among 42 entries tested, the difference in accuracy was not particularly great: the average science entry in Wikipedia contained around four inaccuracies; Britannica, about three.

Considering how Wikipedia articles are written, that result might seem surprising. A solar physicist could, for example, work on the entry on the Sun, but would have the same status as a contributor without an academic background. Disputes about content are usually resolved by discussion among users.

...

Only eight serious errors, such as misinterpretations of important concepts, were detected in the pairs of articles reviewed, four from each encyclopaedia. But reviewers also found many factual errors, omissions or misleading statements: 162 and 123 in Wikipedia and Britannica, respectively.

...

Several Nature reviewers agreed with Panelas' point on readability, commenting that the Wikipedia article they reviewed was poorly structured and confusing. This criticism is common among information scientists, who also point to other problems with article quality, such as undue prominence given to controversial scientific theories. But Michael Twidale, an information scientist at the University of Illinois at Urbana-Champaign, says that Wikipedia's strongest suit is the speed at which it can updated, a factor not considered by Nature's reviewers.

"People will find it shocking to see how many errors there are in Britannica," Twidale adds. "Print encyclopaedias are often set up as the gold standards of information quality against which the failings of faster or cheaper resources can be compared. These findings remind us that we have an 18-carat standard, not a 24-carat one."

The most error-strewn article, that on Dmitry Mendeleev, co-creator of the periodic table, illustrates this. Michael Gordin, a science historian at Princeton University who wrote a 2004 book on Mendeleev, identified 19 errors in Wikipedia and 8 in Britannica. These range from minor mistakes, such as describing Mendeleev as the 14th child in his family when he was the 13th, to more significant inaccuracies. Wikipedia, for example, incorrectly describes how Mendeleev's work relates to that of British chemist John Dalton. "Who wrote this stuff?" asked another reviewer. "Do they bother to check with experts?"

But to improve Wikipedia, Wales is not so much interested in checking articles with experts as getting them to write the articles in the first place.

As well as comparing the two encyclopaedias, Nature surveyed more than 1,000 Nature authors and found that although more than 70% had heard of Wikipedia and 17% of those consulted it on a weekly basis, less than 10% help to update it. The steady trickle of scientists who have contributed to articles describe the experience as rewarding, if occasionally frustrating (see 'Challenges of being a Wikipedian').

Greater involvement by scientists would lead to a "multiplier effect", says Wales. Most entries are edited by enthusiasts, and the addition of a researcher can boost article quality hugely. "Experts can help write specifics in a nuanced way," he says.

Wales also plans to introduce a 'stable' version of each entry. Once an article reaches a specific quality threshold it will be tagged as stable. Further edits will be made to a separate 'live' version that would replace the stable version when deemed to be a significant improvement. One method for determining that threshold, where users rate article quality, will be trialled early next year.

Tuesday, December 13, 2005

I just talked on the phone to WikipediaClassAction.org

Previous: Class action lawsuit against wikipedia?

I asked the man who answered (Smith himself?) whether his organization had any connection with QuakeAID (a qustionable organization that claims to be a charity -- again, see previous post). He said they didn't. I pointed out that WikipediaClassAction.org had the same listed P.O. Box as QuakeAID. He said that that was because they had the same ISP, and the ISP provided the P.O. Box (do ISPs provide P.O. boxes? I suspect not.)

I asked him what wikipedia did to him to prompt this lawsuit. He refused to answer.

I talked to him for awhile (I don't know if he's normally this sarcastic and confrontational or was just in a bad mood). He suggested wikipedia requrie credit cards for use -- I said that wikimedia is an international organization and the overwhelming majority of people worldwide don't have credit cards. He said "the ones who don't have computers don't have credit cards" -- not even remotely true.

At this point "I don't have a credit card!" would have been the best thing to say; instead, I pointed out that even the majority of people in Europe don't have credit cards; they're mostly a North American phenomenon. "You don't know what you're talking about." he said. "I've lived in Europe for 20 years. Have you even been to Europe?"

"Of course I've been to Europe."

"I'm in Europe right now! I'm talking to you from London!"

"But this is an american area code."

"It's called a telephone!"

"Uh, but this is an american area code."

"Yeah! It's called a telephone! You can talk to people with it! Amazing, huh?"

At that point the conversation wound down.

For the record, I called the number posted on the organization's website: (866) 871-7368. 866 is, of course, a north american area code. It's also worth pointing out that he didn't have a british accent, though I think he started making some effort to put one on.

Previous: Class action lawsuit against wikipedia?

Class action lawsuit against wikipedia?

Update: not at the moment, no.

A group calling itself "WikipediaClassAction.org" is trying to start a class action suit against wikipedia.

The site has the same listed address as QuakeAID, which appears to be a fake charitable organization that seems to have run a scam following the south asian tsunami (and had a subsequnet run-in with wikipedia, when an article questioned the organization's credibility). (I'm not linking to WikipediaClassAction.org because it's running google ads -- in lieu of that, a metafilter thread on the topic with a huge range of well-considered opinions.)

As you've probably guessed, I think the proposed lawsuit (if it's not just a bid for google ad revenue, or an attempt to muddy the waters surrounding QuakeAID's qustionable practices and bad wikipedia PR) is so misguided as to verge on the insane. Individual editors should be held responsible if they libel someone, not wikimedia as a whole.

It would be tragic (perhaps even in the literal Greek fashion) if wikipedia's open model couldn't coexist with the U.S.'s legal system. Wikipedia's case is strong at the moment -- but we'll see what happens in the 2006 and 2008 election seasons. If wikipedia ceases to be publically editable, that would end the grand experiment and make it an ordinary (and frustratingly static) encyclopedia.

As an open-source project, of course, it couldn't be wiped out completely -- there are too many copies and backups floating around (public and private). But if the organization is fragmented or driven underground, editing it will impose a greater cost (in effort and perhaps risk) and the balance of power will shift towards the anonymous bots and vandals.

It's more likely that wikimedia will take steps to protect itself. Here are a couple things it could do:

  • Prevent anonymous edits -- not just edits by unregistered users, but edits by all users with a persona not tied to a real-life person.
    Downsides:
    • Honest people sometimes rely on anonymity for protection (especially in third world countries).
    • Online ID systems are immature right now. Among other problems, they usually impose burdens so great that they'd destroy wikipedia's biggest advantage (its huge potential userbase). For example, requiring a credit card to sign up -- only as identity verification, not to institute a charge -- would exclude not only the overwhelming majority of non-americans but also a sizable minority of americans (including this broke blogger).

  • Stop calling Wikipedia an encyclopedia. Stop calling users "editors". Make sure no member of the wikimedia board ever edits an article.
    This sounds a bit strange, but there's a quirk in the U.S. legal system regarding online forums: if a forum is moderated, even slightly, the administrators can be held liable for what's posted; if not, they can't.
    Downsides:
    • Thinking of the project as encyclopedic encourages good edits.
    • How much of wikipedia's administrative structure would have to be dismantled? Admins have useful powers, like quick edit reversion -- would those powers have to be removed or made universally available?


Legal persecution seems unlikely right now, but if it ever comes down to it, I'll be marching down the national mall with a mesh LAN, handing out wikipedia DVD freezes.

Sunday, December 11, 2005

The guy who wrote the false wikipedia entry on John Seigenthaler has confessed. He wasn't a conspiracy nut, just someone who (he says) thought of wikipedia as a joke site.

This whole affair will have been good for wikipedia, I think. Readers will take articles with a grain of salt, and potential vandals will think twice before posting false information (the linked story of the noose slowly closing around the vandal's neck was satasfying to read and will probably provide good disincentives).