The Debian Project http://www.debian.org/
Joint Statement about GNU/Linux Security press@debian.org
April 6th, 2004 http://www.debian.org/News/2004/20040406
Joint Statement about GNU/Linux Security
Executive Summary:
GNU/Linux vendors Debian, Mandrake, Red Hat, and SUSE have
joined together to give a common statement about the Forrester
report entitled “Is Linux more Secure than Windows?”. Despite the
report’s claim to incorporate a qualitative assessment of vendor
reactions to serious vulnerabilities, it treats all vulnerabilities
as equal, regardless of their risk to users. As a result, the
conclusions drawn by Forrester have extremely limited real-world
value for customers assessing the practical issue of how quickly
serious vulnerabilities get fixed.
Full Statement:
The security response teams of GNU/Linux distributors Debian,
Mandrakesoft, Red Hat and SUSE have assisted Forrester in gathering
and correcting data about vulnerabilities in their products. The
gathered data was used at Forrester for a report that became titled
“Is Linux more secure than Windows?”. While the vulnerability data
regarding GNU/Linux which is the basis for the report is considered
to be sufficiently accurate and useful, Debian, Mandrakesoft, Red
Hat and SUSE, from now on referred to as “We”, are concerned about
the correctness of the conclusions made in the report.
We believe that it is in the interest of our usership and the
Free Software community to respond to the Forrester report in the
form of a common statement:
We were approached by Forrester in February 2004 to help them
refine their raw data. Forrester collected data about the
vulnerabilities that affected GNU/Linux during a one year period
(June 2002 – May 2003) and looked at how many days it took us to
provide corrections to our users. Significant efforts have been put
in not only making sure that the underlying dataset for the
vulnerabilities was correct, but also to articulate the special
technical and organizational care taken in the response processes
in the professional Free Software security field. This expertise is
greatly appreciated by our usership since it adds a high value to
our products, but we see that most of this value has been ignored
in the methods used for the analysis of the vulnerability data,
leading to erroneous conclusions.
Our Security Response Teams and security specialized
organizations of respectable reputation (such as the CERT/DHS, BSI,
NIST, NISCC) exchange information about vulnerabilities and
cooperate on the measures and procedures to react to them. Each
vulnerability gets individually investigated and evaluated; the
severity of the vulnerability is then determined by each of the
individual teams based on the risk and impact as well as other,
mostly technical, properties of the weakness and the software
affected. This severity is then used to determine the priority at
which a fix for a vulnerability is being worked on weighed against
other vulnerabilities in our current queues. Our users will know
that for critical flaws we can respond within hours. This
prioritization means that lower severity issues will often be
delayed to let the more important issues get resolved first.
Even though the Forrester report claims so, it does not make
that distinction when it measures the time elapsed between the
public knowledge of a security flaw and the availiability of a
vendor’s fix. For each vendor the report gives just a simple
average, the “All/Distribution days of risk”, which gives an
inconclusive picture of the reality that users experience. The
average erroneously treats all vulnerabilities as equal, regardless
of the risk they pose. Not all vulnerabilities have an equal impact
on all users. An attempt has been made to allocate a severity to
vulnerabilities using data from a third party, however the
classification of “high-severity” vulnerabilities is not
sufficient: The mere announcement of a vulnerability by a
particular security organization does not necessarily make the
vulnerability severe – similarly, the ability to exploit a weakness
over the network (remote) is often irrelevant to the
vulnerability’s severity.
We believe the report does not treat vendors of Free Software
and the single closed source vendor in the same way. Free Software
is known for its variety and its freedom of choice amongst the
standards it defines. Multiple implementations of these standards
are typically offered for both desktop and server use, which gives
users the freedom to select software based on their own criteria
rather than those of the vendor. The openness, transparency and
traceability of the source code is added value in addition to the
larger variety of software packages available. Finally, the claim
that one software vendor had fixed 100% of their flaws during the
period of the report should be incentive for a closer investigation
of the conclusions the report presents.
signed,
Noah Meyerhans, Debian
Vincent Danen, Mandrakesoft
Mark J Cox, Red Hat
Roman Drahtmüller, SUSE
Additional Information:
Javier Fernández-Sanguino Peña composed a survey
in 2001[*] and discovered that it has taken the Debian security
team an average of 35 days to fix vulnerbilities posted to the
Bugtraq list. However, over 50% of the vulnerabilities where fixed
in a 10-days time frame, and over 15% of them where fixed the same
day the advisory was released! For this analysis, all
vulnerabilities were treated the same, though.
He has rerun the survey based on vulnerabilities discovered
between June 1st 2002 and May 31st 2003 and found out that the
median value of delays between the disclosure and releasing an
advisory including a correction was 10 days (average is 13.5 days).
Again, for this analysis advisories were not classified with
different priorities.