By Michael Hall,
LinuxToday
In a message to the Debian developers list, Debian Project
Leader Wichert Akkerman said early results from the Linux Standards
Board’s Filesystem Hierarchy Standard tests were misleading due to
factors including incomplete installations on the test systems and
bugs in the tests themselves.
According to Akkerman, though the Debian Woody installation
tested failed 17 of the 243 items in the test for FHS compliance,
only nine of the problems identified in the test were the result of
actual non-compliance on the part of the distribution.
“Not all of the test results are fair in my opinion: some are
real bugs in Debian, others are bugs in the testsuite or the result
of using an incomplete install,” said Akkerman in his message. He
went on to comment on the prematurity of making claims about the
quality of a distribution based on these results:
“Please also note that the testsuite isn’t finished yet, and
neither is the LSB standard, so it is too early to draw any
conclusions as to how compliant any distribution will be to the
final product. So do not be fooled by things like the SuSE press
release, who seem to be ignoring that little fact.“
Akkerman’s comment was in reaction to yesterday’s press release
by SuSE, in which representatives of the company used the test
results to claim their distribution is “the most
standards-compliant Linux distribution tested.”
In a followup letter to the list, Andrew Josey, LSB Test leader,
and author of the LSB-FHS test suite, allowed that the test and the
specification it was designed for still have some “issues:
“We should not be expecting any distributions to pass the
current version of the test suite. Although we believe it to be a
fair and accurate test of the LSB FHS 2.1 specification, there are
issues with the specification and tests that need to be resolved.
The policy with test development is that we test the specification
“as is”, and it’s the specification owners that get to judge
whether the spec is right or otherwise and not the test suite
developers. Going forward if we were to roll out a formal
certification process, and issues are agreed with the spec owners
then we will modify the test accordingly or issue waivers.”
Results of the distributions tested so far are listed below, and
taken from The LSB’s test
results page.
Distribution Version # Tests Passed # Tests Failed -------------------------------------------------------------- Caldera 2.4 (Intel) 204 39 Debian Woody (Intel) 226 17 Kondara Server 2000 205 38 Mandrake 7.1 (Intel) 200 43 OpenLinux eServer 2.3 (Intel) 199 44 Red Hat Linux 7.0 (Intel) 223 20 Red Hat Linux 6.2 (Intel) 209 34 SuSE Linux 6.4 (Intel) 219 24 SuSE Linux 7.0 (Intel) 238 5 Turbo Linux 6.0.5 (Intel) 219 24