Linux Today: Linux News On Internet Time.

Red Hat Joins Debian Leader in Questioning Test Results, LSB Chair: Tests Are a "Work in Progress"

Oct 24, 2000, 14:01 (13 Talkback[s])
(Other stories by Michael Hall)

By Michael Hall, LinuxToday

Last week Debian Project Leader Wichert Akkerman fired off a message to the Debian developer's list taking issue with a press release from SuSE claiming the German distribution is the "most standards-compliant Linux distribution tested" based on the strength of results from the LSB FHS test suite.

Red Hat's Vice President of Engineering Erik Troan echoed Akkerman's concerns in a statement, saying that Red Hat will comply with the LSB when it's complete:

"As evidenced by Debian's response to the test, this is a standard that's still under development. While we haven't reviewed the tests which are used in any detail, we are not surpised that we have a similiar number of "failures" as other distributions. This test suite looked for some items which are wrong (such as the location of termcap and the name of the mtools configuration file), some items which need further debate, (such as requiring the tclX program /usr/bin/tcl), and some things which are quite new (such as/etc/opt). When the LSB is a complete, implementable standard Red Hat will follow it enthusastically. Comparing levels of conformance to standards which are still undergoing active work is of questionable value, however."

According to Linux Standard Base Steering Committee Chairman Dan Quinlan, Akkerman's message "definitely raised some valid points, all of which are being investigated for FHS 2.2." In a separate message to the Debian developer's list, Quinlan addressed many of the points in Akkerman's mail.

Quinlan also acknowledged that the test suite is still incomplete:

"The LSB-FHS test-suite is a work in progress, the bugs are mostly due to problems in FHS primarily due to the evolution of Linux. The first version of FHS (called FSSTND back then) was released in 1994, but until recently, we didn't have a test-suite. Developers reading the FHS generally filter out minor issues so they can go unreported for a while. Test suites developed using rigorous methodology are not so forgiving."

"Linux distributions have put a lot of effort into FHS compliance and SuSE should be happy about their low number of failures. Their results were very good. Unfortunately, the announcement was premature because (1) it's not really possible to know how any distribution will fare when the LSB-FHS test suite is finished, (2) it's possible that there are other distributions that will do just as well or better, (3) the test suite is still in development, and (4) the test results posted are known to be inaccurate and there are some bugs with FHS and the LSB-FHS test suite (the LSB web pages said as much last week)."

Quinlan pointed out that there's a new warning at the top of the test results page reading "This is a work in progress, no claims of LSB compliance or passing these test suites should be made. These are unapproved test suites and issues remain both with the test suites and the specifications under test."

Related Stories: