Linux Today: Linux News On Internet Time.

Fedora Project: Why No Foundation?

Apr 05, 2006, 17:00 (5 Talkback[s])
(Other stories by Max Spevack)

To my fellow Fedora community members:

As many of you are aware, FUDCon Boston is this Friday. One of the most important topics that we will be discussing there is the future of the Fedora Project, specifically with regard to the Fedora Foundation.

I'd like to ask you all to read the document that follows this note. It reviews Red Hat's intentions in initially announcing the Fedora Foundation, and outlines the problems that have led us to the decision to move in a different direction. It also discusses the plan that we are implementing instead, and the steps that we are taking to ensure that the Fedora Project continues to thrive and grow.

It is as complete, honest, and transparent as we can make it. If you feel that there are places in which it lacks those qualities, call us on it, and we will respond.

This document represents the work of many people both inside of Red Hat and within the Fedora community. It is a long read, but a very worthwhile one.

So take a look, read, digest, and share your thoughts. I look forward to discussing this in great detail on email, and also with as many of you as possible in person at LinuxWorld and at FUDCon over the next few days. Many of Red Hat's most active Fedora folks will be at those two shows, so please come and talk with us.

Max Spevack

Last June, Red Hat announced its intention to launch the Fedora Foundation. We've had a lot of smart people working hard to make this Foundation happen, but in the end, it just didn't help to accomplish our goals for Fedora. Instead, we are restructuring Fedora Project, with dramatically increased leadership from within the Fedora community.

The next obvious question--"Why no Foundation?"--deserves a detailed explanation.

When we announced the Foundation, it was with a very specific purpose, and in a very specific context. The announcement was made by Mark Webbink, who has been the intellectual property guru at Red Hat for a long time now. His stated goal for the Foundation: to act as a repository for patents that would protect the interests of the open source community.

Once we announced the intention to form a Foundation, people inside and outside of Red Hat were interested in working beyond the stated purpose--an intellectual property repository--and instead saw this new Foundation as a potential tool to solve all sorts of Fedora-related issues. Every Fedora issue became a nail for the Foundation hammer, and the scope of the Foundation quickly became too large for efficient progress.

A team moved forward to create the Foundation itself. We created the legal entity, came up with some very basic and flexible bylaws, and appointed a board to run it temporarily. This all happened pretty quickly, because this was the easy part. We had articles of incorporation in September 2005.

Then came the hard part: articulating the precise responsibilities of the Foundation. This conversation took months, but ultimately it came back around, again and again, to a single question: "What could a Fedora Foundation accomplish that the Fedora Project, with strong community leadership, could not accomplish?"

So here, in order, were the possible answers to that question--and why we found, in every single case, that the Fedora Foundation was not the right answer.

ONE: The Fedora Foundation could be an entity for the development of an open source patent commons.

This was the obvious starting place, and what we actually announced. One of the lurking concerns of the open source community is the threat of software patents. The Fedora Foundation could have been an ideal repository for defensive patents. We envisioned soliciting patentable ideas from businesses and/or individuals, paying for the prosecution of these patents, and then guaranteeing open source developers the unrestricted right to code against these patents using a similar mechanism to the Red Hat patent promise. (http://www.redhat.com/legal/patent_policy.html).

What we weren't counting on was the rapid progress of the Open Invention Network (http://www.openinventionnetwork.com/press.html), which serves a similar purpose for businesses in a much more compelling way. Without going into too much detail, it became clear to us that OIN is going to be the 800-pound gorilla in the patent commons space, and we were eager to join forces.

OK, so much for soliciting patents from businesses. What about individuals? If we were to focus the Fedora Foundation's efforts on soliciting patentable ideas from individuals, how many could we get? Our gut decision: not many. Most developers who actually work for a living have agreements with their employers that prevent them from pursuing patents independently. Many university students who pursue patents are required to grant them to the university.

After putting a lot of work into the idea of a Fedora Foundation patent commons, in the end it just didn't seem compelling. So we shelved the idea.

TWO: The Fedora Foundation could act as a single point of standing for legal issues.

The Free Software Foundation serves this purpose for the GNU projects. We thought that the Fedora Foundation might successfully serve the same purpose for Fedora projects. Have you ever noticed that the GNU projects all require contributors to assign copyright to the FSF? That's because there's this legal idea called "standing" that matters deeply to lawyers and judges. Here's a little skit that helps to explain why standing is important:

BAILIFF: Come to order for case Z-38-BB-92. Plaintiff is Small Software Project. Defendant is Great Big Computer Corporation.

JUDGE: OK, have a seat, folks. The docket is busy today, and I've got a doctor's appointment in two hours. Plaintiff, what's this all about?

PLAINTIFF'S COUNSEL: Well, your honor, there's this license called the GPL that the defendant is *totally* violating. Basically, they stole the plaintiff's code and put it into their software program.

DEFENDANT'S COUNSEL: Hold it right there. Your Honor, plaintiff doesn't have standing in this case. There's 100 different developers that wrote this code, and the plaintiff only represents six of them. Plaintiff clearly doesn't even have the legal right to sue us, Your Honor.

JUDGE: Looks like this case could be Pretty Hard, and this whole "standing" thing gives me a perfect excuse not to think about it. Counsel, get back to me when you've got the other 94 plaintiffs.

So, standing is a big concern. In the world of lawyers, it's one of the big potential unknowns around defending open source projects, especially projects that have lots of contributors.

The obvious problem with establishing standing in this way, though, is that a single entity *must* own *everything* in your project. That's why the FSF *requires* copyright assignment.

What Fedora projects currently exist where copyright assignment makes sense?

Well... none, as it turns out. Let's look at some of the current Fedora projects as examples.

At present, the two most successful Fedora projects are Core and Extras--which, together, basically constitute a big Linux distribution. And what is a distribution? Ideally, it's a high-quality repackaging and integration of content owned by others. That's the whole point. In such cases, copyright assignment makes no sense at all.

Then there's the Fedora Documentation project, which produces documentation and makes it available under the Open Publication License (http://opencontent.org/openpub/) without options. Given the liberal nature of this license, it just doesn't seem all that useful to ask contributors to assign copyright for defense of these works.

Then there's the Fedora Directory Server, which Red Hat purchased and open sourced. No question who holds standing there; it's Red Hat. The time may come when the Fedora Directory Server project is ready to incorporate lots of changes from the community, but until that time comes, the question of copyright assignment is pretty much a theoretical question.

Which is what a lot of this comes down to--the question of legal standing is either an open or theoretical question at best, and probably better left to an organization such as the FSF that focuses a great deal more attention on these types of questions.

Put another way: we have a finite amount of resources to make Fedora better. How much of that cash should be going to expensive lawyers--especially if Red Hat already has lawyers who have a strong incentive to defend Fedora, should such a defense prove to be necessary?

So the Fedora Foundation didn't seem compelling as a mechanism for copyright assignment, either.

THREE: The Fedora Foundation could act as an entity for funding Fedora-related activities that Red Hat didn't have great interest in funding.

Funny thing, that. We asked some of our closest friends this question: "Would you donate to an independent Fedora Foundation?" The answers were very interesting, and ran the gamut. Some people were incredibly enthusiastic: "We'd love to give money!" Some were neutral: "Thanks, but we'd rather contribute code." And some were less enthusiastic: "Red Hat is a successful, profitable company. Why are you asking *me* for money?"

Here's another funny thing: if you choose to incorporate as a non-profit entity in the United States, then you subject yourself to a number of rigorous IRS tax tests. One of these tests is the "public support test." If you say you're a public charity, well by golly, you have to prove it. If, within four years, you aren't collecting fully one third of your money from public sources, then you're not actually a public charity.

People are always shocked when we tell them how many resources Red Hat puts into Fedora. If we were to make the Fedora Foundation a truly independent entity, then we'd have to track every dime of that expense as "in-kind contributions". That means we'd have to track:

  • The cost of bandwidth for distributing Fedora to the world;
  • Every hour that Red Hat engineers spend working on Fedora, whether that is the actual writing of code, release engineering, testing, etc.;
  • Legal expenses of running a Foundation;
  • Administrative expenses of running a Foundation.

As an intellectual exercise, let's ignore all of those numbers for now except for bandwidth. Back in the day, when Red Hat would release a distro, we would regularly get angry calls from network admins at big datacenters, complaining that we were eating all of their bandwidth. If you ever meet any of our IT guys over a beer, be sure to ask them about the time we melted a switch at UUNet.

The demand for Fedora is every bit as high, and the March 20 release of Fedora Core 5 was no exception. So let's take a conservative guess and say that the bandwidth cost for distributing Fedora comes to $1.5 million a year. Yes, even though we have BitTorrent trackers and Fedora mirror sites worldwide.

That means that a public Fedora Foundation would have to raise $750k in public funds--remember the one-third public support test--every single year, just to pay for *bandwidth*, assuming no growth and no other expenses.

So what would happen, under such a scenario, if Red Hat were to decide to spend more money on Fedora? Because that's exactly what Red Hat wants to do.

There were alternatives to the public charity angle. We could have set up a private operating foundation, and we explored this avenue--but then it wouldn't really be an independent entity. It would be a shell. The fact that Red Hat would still likely bear the legal risk of Foundation decisions, and the complication of raising public funds, made any 501(c) less attractive.

In short: the fund raising burden for a truly independent Fedora Foundation would be terrifying. So the Fedora Foundation clearly wasn't compelling as a fund raising entity--if anything, it represented an impediment to building a better Fedora Project.

FOUR: The Fedora Foundation could provide mechanisms for more community participation in key decision-making processes.

From the day the Fedora Project was started over two years ago, it's been our goal to build these mechanisms, Foundation or no Foundation. How successful have we been?

Initially, we had some problems. In the last year, though, we've had some pretty clear successes. The Fedora Extras project is a good example here. When we officially launched it in February 2005 at FUDCon Boston, we put together a steering committee that consisted of a pretty even mix of Red Hat and community packagers. At FUDCon Germany last summer, we strengthened the group with more European members. Earlier this year, we successfully handed off leadership of the committee to a community member. Red Hat continues to provide logistical and legal support, but Fedora Extras policy is determined by the community.

So what happens when the Fedora Extras Steering Committee (also known as FESCO) runs into difficulty? Well, they escalate the issue to "the Board." And who is "the Board?" It's been the people running the Fedora Foundation--but it's also been the people running the Fedora Project. Whenever "the Board" had been asked to make a decision, there's been no practical distinction between "Project" and "Foundation."

What *is* vital, whether we're talking about "The Foundation" or "The Project," is the actual presence of community members on the board--but more on that later.

FIVE: The Fedora Foundation could serve as a truly independent entity, providing the ability for Fedora to grow separately from Red Hat's interests.

This is the real heart of the matter. This is what some people want to see: a more independent Fedora. This is The Question That Must Be Answered.

The simple and honest answer: Red Hat *must* maintain a certain amount of control over Fedora decisions, because Red Hat's business model *depends* upon Fedora. Red Hat contributes millions of dollars in staff and resources to the success of Fedora, and Red Hat also accepts all of the legal risk for Fedora. Therefore, Red Hat will sometimes need to make tough decisions about Fedora. We won't do it often, and when we do, we will discuss the rationale behind such decisions as openly as we can--as we did with the recent Mono decision.

But just because Red Hat has veto power over decisions, it does not follow that Red Hat wants to use that power. Nor does it follow that Red Hat must make all of the important decisions about Fedora. In fact, effective community decision making is one of the most direct measures of Fedora's success.

The most important promise about Fedora--once free, always free--still stands. We aim to set the standard for open source innovation. A truly open Fedora Project is what makes that possible.

The New Fedora Project Leadership Model

Since Fedora's inception two years ago, a diverse global community has developed around Fedora--and, as in any open source project, natural leaders have emerged. The time has come to reward some of these leaders with the opportunity to define the direction of the Fedora Project at the highest level.

Therefore, we've reconstituted the Fedora Project Board to include these community leaders directly.

Initially, there are nine board members: five Red Hat members and four Fedora community members. This Board is responsible for making all of the operational decisions of the greater Fedora project, including decisions about budget and strategic direction.

In addition to the nine board members, there is also be a chairman appointed by Red Hat, who has veto power over any decision. It's our expectation that this veto power will be used infrequently, since we're all aware of the negative consequences that could arise from the use of such power in a community project.

The chairman of the Fedora Project is Max Spevack. Max has been with Red Hat since 2004, previously as a QA engineer and QA team lead for Red Hat Network. He is a member of the Fedora Ambassadors steering committee, and has been a Linux user since 1999.

The Fedora Project board members from Red Hat are Jeremy Katz, Bill Nottingham, Elliot Lee, Chris Blizzard, and Rahul Sundaram.

Jeremy Katz is a Red Hat engineer. He is the longtime maintainer for Anaconda, and a founding member of the Fedora Extras steering committee.

Bill Nottingham joined Red Hat in May of 1998, working on projects ranging from the initial port of Red Hat Linux to ia64, booting and hardware detection, multilib content definition and fixing, and is currently doing work related to stateless Linux. He's also been involved in various technical lead details, such as package CVS infrastructure and distribution content definition.

Elliot Lee has been a software engineer at Red Hat since 1996. His open source contributions include release engineering for Fedora Core, co-founding the GNOME project, and maintaining assorted open source libraries and utilities. He is a founding member of the Fedora Extras steering committee. Elliot current leads the Fedora infrastructure team, making it easier and enjoyable for contributors to get more done.

Chris Blizzard is an engineering manager for Red Hat. He has served on the board of the Mozilla Foundation, and is currently leading the One Laptop Per Child project for Red Hat.

Rahul Sundaram is a Red Hat associate based in Pune, India. He is a longstanding contributor to multiple Fedora projects, a Fedora Ambassador for India, and a member of the Fedora Ambassadors steering committee.

The Fedora Project board members from the community are Seth Vidal, Paul W. Frields, Rex Dieter, and a fourth board member to be named as soon as possible.

Seth Vidal is the project lead for yum, which is one of the key building blocks for software management in Fedora. He also maintains mock, the basis for the Fedora Extras build system. He is a founding member of the Fedora Extras steering committee, and he was one of the people chiefly responsible for the first ever release of Fedora Extras packages in 2005. Seth is also the lead administrator of the infrastructure at fedoraproject.org, which includes the Fedora project wiki, RSS feed aggregator, and bittorrent server.

Paul W. Frields has been a Linux user and enthusiast since 1997, and joined the Fedora Documentation Project in 2003, shortly after the launch of Fedora. As contributing writer, editor, and a founding member of the Documentation Project steering committee, Paul has worked on a variety of tasks, including the Documentation Guide, the Installation Guide, the document building infrastructure, and the soon-to-emerge RPM packaging toolchain. Paul is also a Fedora Extras package maintainer.

Rex Dieter works as Computer System Administrator in the Mathematics Department at the University of Nebraska Lincoln. Rex is a KDE advocate and founded the KDE Red Hat project. He is also an active contributor to Fedora Extras. Rex lives in Omaha, Nebraka, with his wife, 2 children, and 4 cats.

It's true that a lot of the key governance details--term length, board composition, election or appointment process--have yet to be resolved. One of the first responsibilities of the new board will be to work with the Fedora community to answer these questions.

Red Hat has been supporting a free Linux distribution for over ten years, and Red Hat will *always* support a free Linux distribution. We want to work together with the Fedora community to make Fedora better. We want a Fedora that is a true partnership between Red Hat and the community. We want to build effective models to make that partnership real. We want to see the folks at MySQL managing MySQL in Fedora. We want to see the folks from kde.org managing KDE in Fedora. We want to see the folks at Planet CCRMA managing audio production applications in Fedora. We want Fedora to be a launching pad not just for open source software, but for open content of all kinds. We want the Fedora Project to be a way to fill the community with high quality software and content, and we want to empower the Fedora community to innovate in ways we'd never even considered.

The new Fedora Project Board has a strong mandate to make these things happen, and has the full support of Red Hat. We ask that you, the members of the Fedora community, give them your full support as well, and we thank you for all the support you've given us so far. We would not have made it nearly this far without your patience, your friendship, and your tireless help.

Related Stories: