Photo: Flickr user lifeontheedge

Thursday, August 04, 2005

The Future of Open Source

5 years ago Linux was cool (coolness propogated down from the highest levels of geekery to the merely internet-addicted, who absorb it eventually through long exposure, like they do web design); now some developers are deserting -- proprietary OS X gives them a command line, and that's all they really want.

At least in the first world, the thrill is gone.* Most people are more concerned with operating systems as tools than as ends in themselves; tinkering has lost its allure; programming should focus on rewarding, important problems, not getting your sound card to play two sounds at the same time.

(Tangent. In the 3rd world, unnoticed on Slashdot, Linux could be really, really fucking important. Commodity hardware, newly inexpensive; free software. People using computers to look up crop prices. Small towns using local builds customized in the local language. (No sound, no video-in, but who cares?) A homegrown software industry. Unglamorous stuff, but far more important than how cool your G4 Tower looks.)

Sever-side linux is still a powerful force, but what happened to the desktop utopia that was supposed to unseat Windows? And will the same developed-world disenchantment hit Wikipedia as it grows?

While in principle anyone can contribute to an open source project, Linux's barriers to entry are higher than Wikipedia's. Even correcting minor Linux bugs is well beyond (for example) my expertise, but my grandmother could edit Wikipedia. All you need is an internet connection and literacy (mid-level literacy, at that; other people will fix your grammar and spelling).

Wikipedia can draw on half a billion potential contributors; only about 100,000 people can code Linux.

It's hard to overstate this difference.

This is about where people start throwing around terms like "emergent behavior" and "paradigm shift". Over the next few years, Wikipedia (and some of its Wikimedia sister sites) will become comparable to Linux in economic and social significance. (Maybe Linux will catch up again a few decades later, if schools start teaching as many kids to program as they teach to write.)

(Note: I'm not saying wikis in general are going to be terribly revolutionary -- making a good, scalable wiki is about as hard as setting up a democracy in the middle east. And momentum is vital: like Ebay and Craigslist, wikis are only visited if they're contributed to and only contributed to if they're visited. Wikitravel might yet find its legs, but right now it's the Betamax to Wikipedia's VHS -- Wikipedia has much more info on most locations.)

Of course, the picture is very different if you look at the number of actual contributors:

56 thousand people have edited wikipedia and about a hundred thousand (though this number is difficult to estimate) have worked on the Linux kernel and its best-known desktop environments. (Other Linux applications aren't counted, and neither are wikipedia's sister projects.)

But wikipiedia is very young: less than a third the age of Linux. Compare the most influential contributors (so that linux data is available) over time and the picture changes again:

Those are only the most die-hard fans -- Wikipedians with more than 100 edits a month and Linux developers in the Credits file. But the huge upward sweep is exactly what graph #1 would lead one would expect.


Software mavens are tinkerers by nature. How many people who use Wikipedia will actually edit it?

There's a large gap between the skill necessary to run linux and (for example) to compile it, and a similar gap between compilation and real, gritty coding. Wikipedia's learning curve is shallower: you can reorganize and reformat pages, yes; you can contribute esoteric trivia to articles on history and philosophy and find diplomatic dispute resolutions. But you can also correct a typo in an article about your city.

Some fun articles to write are not particularly useful, and vice versa. Linux has been saddled with this problem: volenteers ignore boring but necessary work.

Scale solves this one -- yes, there will be mountains of fun-to-write content on wikiepdia (it's possibly the world's best Harry Potter resource). But there are so many contributors that some of them will want to write detailed histories of the Democratic Republic of Congo and help compile comprehensive lists of the etymologies of the names of Ohio counties.

Won't wikipedia deteriorate as it gets bigger?

It's gotten a lot bigger in the past few years, and much better as well -- if wikipedia can scale from 50 to 50 thousand contributors (and scale superbly), it can probably scale from 50 thousand to 50 million. Wikipedia is not anarchy; it's a complex system with many subtle but well-honed rules and balances.

Wikipedia contributors don't work together -- pages are hacked out through interminable debate; huge amounts of effort are wasted on minutia.

I think this is the exception rather than the rule. But more importantly: huge amounts of effort may be expended, but only in sum; the effort is distributed with incredible efficiency -- anyone can put in five minutes, or less. 50,000 people, times five minutes a day of labor, times 30 days, is 125 thousand man-hours: 14 years of continuous labor, or about 60 years of 40-hour weeks. Even if 90 percent of that is reverted (an absurdly pessimistic estimate), it's still more than 6 years' work completed in a month. Wikipedia is one of the few places where such small individual contributions can be easily combined. (This brings up another point: people work much more efficiently in shorter increments.)

The important thing is that people aren't made to feel their work has been wasted -- nothing kills the contributory spirit like having your painstaking paragraphs reverted. This is a problem that needs a social solution, and hopefully one will evolve/spring up.

Wikipedia and Linux are completely different! They certainly aren't competitors; wikipedia runs on linux.

They're not competitors, but they're both open-source projects. Open source is an important organizational framework that until wikipedia had been used mostly for programming, not writing human-language text. The most general point is taht the visibility and importance of open source writing will eclipse the visibility and importance of open source programming.

Linux isn't just the kernal and desktop environments. What about all the software?

Of course, there are hundreds of other programs for linux, but there are hundreds of potential structures, programmatic, community-based, and otherwise, that can run off the mediawiki base; most just haven't appeared yet because wikipedia is only three years old.

Linux bugfixes aren't that hard to code.

You've obviously never tried to teach basic HTML to nursing students. It's easy to overestimate the technical savvy of most people, especially when you hang out with programmers.

Your 3rd graph sucks.

You're right, it does. But I think the reasons it sucks counterbalance each other.

  • More people have contributed the linux kernal than are listed in the Credits file

  • Linux actually began in 1991; 1994 is just the first year the Credits file was published -- effectively, the graph gives linux a 3-year head start.

  • The credits file is cumulative: if you're in it once, you'll be there forever. The number of very active wikipedians is monthly: edit like mad in August, and you're counted for August -- drop below the threshold when you go back to school, and you're off this list.

*Desktop Linux isn't fading away! (section added June 2006)

I agree, and Ubuntu seems to be gaining steam. But I remember when people were seriously talking about windows getting unseated. Hell, Firefox hasn't even unseated IE for 80% of the market -- but Wikipedia's unseated Britannica and World Book as the most widely read encyclopedia.

Because I'm addicted to pageviews, you should share this article:, Digg, Furl, ma.gnolia, reddit, Spurl, Yahoo.

Wales: 10 things will be free.

Wikipedia founder Jimbo Wales is guest blogging for Lawrence Lessig. Theme: Ten things that will be free. ("...the point of naming the list "will be free" rather than "should be free" or "must be free" is that I am making concrete predictions rather than listing a pie in the sky list of things I wish to see.")

1. The encyclopedia. (Of course.) Wales charts a plan to get wikipedia into the developing world, where the vast majority of people are.

Third, while it is important to provide our work in important global or "colonial" languages, we also think it is extremely important to provide our work in languages that people speak natively, at home. (Swahili, Hindi, etc.)

I will define a reasonable degree of success as follows, while recognizing that it does leave out a handful of people around the world who only speak rare languages: this problem will be solved when Wikipedia versions with at least 250,000 articles exists in every language which has at least 1,000,000 speakers and significant efforts exist for even very small languages. There are many local languages which are spoken by people who also speak a more common international language -- both facts are relevant.

I predict this will be completed in 15 years. With a 250,000-article cutoff, English and German are both past the threshold. Japanese and French will be there in a year. Several other languages will be there in two years.

2. School curricula, from kindergarten to college. "...this is a much bigger job than the encyclopedia, and it will take much longer ... An open project with dozens of professors adapting and refining a textbook on a particular subject will be a very difficult thing for a proprietary publisher to compete with.

(Update: Here's the full list.)

Wikisnip of the day

Quantum suicide is a thought experiment that attempts to distinguish between the Copenhagen interpretation of quantum mechanics and the many-worlds interpretation (though there are other interpretations, too). The experiment essentially involves participating in the Schrödinger's cat experiment as the cat. (See also: Quantum Immortality.)