Eric S. Raymond writes:
This is a public draft of a new section for “The Cathedral and
the Bazaar”. As always, comments and criticism are invited.
An issue related to whether one can start projects from zero in
the bazaar style is whether the bazaar style is capable of
supporting truly innovative work. Some claim that, lacking strong
leadership, the bazaar can only handle the cloning and improvement
of ideas already present at the engineering state of the art but is
unable to push the state of the art. This argument was perhaps most
infamously made by the Halloween Documents two
embarrassing internal Microsoft memoranda written about the
open-source phenomenon. The author compared Linux’s development of
a Unix-like OS to “chasing taillights”, and opined “(once a project
has achieved “parity” with the state-of-the-art), the level of
management necessary to push towards new frontiers becomes
massive.”
There are serious errors of fact implied in this argument.
Historically, the open-source community did not invent Emacs or the
World Wide Web or the Internet itself by chasing taillights or
being managed — and in the present, there is so much innovative
work going on that one is spoiled for choice. The GNOME project (to
pick one of many) is pushing the state of the art in GUIs and
object technology hard enough to have attracted considerable notice
in the computer trade press well outside the Linux community. Other
examples are legion, as a visit to Freshmeat on any given day will quickly
prove.
But there is a more fundamental error in the implicit assumption
that the cathedral model (or the bazaar model, or any
other kind of management structure) can somehow make innovation
happen reliably. This is nonsense. Gangs don’t have breakthrough
insights — even volunteer groups of bazaar anarchists are usually
incapable of genuine originality, let alone corporate committees of
people with a survival stake in some status quo ante. Insight
comes from individuals. The most their surrounding social
machinery can ever hope to do is to be responsive to
breakthrough insights — to nourish and reward and rigorously test
them instead of squashing them.
Some will characterize this as a romantic view, a reversion to
outmoded lone-inventor stereotypes. Not so; I am not asserting that
groups are incapable of developing breakthrough insights
once they have been hatched; indeed, we learn from the peer-review
process that groups are essential. Rather I am pointing out that
the every such group development starts from — is necessarily
sparked by — one good idea in one person’s head. Cathedrals and
bazaars and other social structures can catch that lightning and
refine it, but they cannot make it on demand.
Therefore the root problem of innovation (in software, or
anywhere else) is indeed how not to squash it — but, even more
fundamentally, it is how to grow lots of people who can have
insights in the first place.
To suppose that cathedral-style development could manage this
trick when the low entry barriers and process fluidity of the
bazaar cannot would be absurd. If what it takes is one person with
one good idea, then a social milieu in which one person can rapidly
attract the cooperation of hundreds or thousands of others with
that good idea is going inevitably to out-innovate any in which the
person has to do a sales job to a hierarchy before he can work on
it without risk of getting fired.
That, however, is a negative point. The reader would be better
served by a positive one. I suggest, as an experiment, the
following;
- Pick any cloded-source operating system competing with Linux,
and a best source for accounts of current development work on
it. - Watch that source and Freshmeat for one month. Every day, count
the number of release announcements on Freshmeat that you consider
`original’ work. Apply the same definition of `original’ to
announcements for that other OS and count them. If your definition
is “I know it when I see it”, that’s not a problem for purposes
of this test. - Thirty days later, total up both figures.
The day I wrote this, Freshmeat carried twenty-two release
announcements, of which three appear they might push state of the
art in some respect, This was a slow day for Freshmeat, but I will
be astonished if any reader reports as many as three likely
innovations a month ion any closed-source channel.