Linux Today: Linux News On Internet Time.

Community: Where Have All the HOWTOs Gone?

Nov 06, 2004, 01:00 (8 Talkback[s])
(Other stories by Jim Sansing)

By Jim Sansing

A lot has been written about the direction of Linux and Free/Open Source Software. Most of it is unsolicited advice on how the Linux and FOSS community can make Linux and FOSS applications more business or consumer friendly (translation: what you people should be doing for me).

Many of the conclusions are improbable because they are reached by starting from wrong assumptions, such as that the community consists of a bunch of egotistical code addicts who can be taunted into programming anything. Actually, the community is made up of more than coders, and all members of the community have diverse interests outside of computers. And we especially like it when we can use FOSS applications to support those interests, which is a hint on where we find our inspiration.

We also give feedback to developers and provide help to each other because that is how communities behave. In this article, I would like to address the whole community on the direction of Linux and Free/Open Source Software (translation: what we should be doing for ourselves).

I recently built a new desktop computer from scratch for the first time. It is built with server quality parts and should last me for well over 5 years. My new machine is built around an Opteron CPU, so I have gotten to see the 2.6 kernel in 64-bit mode.

The short version of this story is that I have installed Gentoo, Fedora Core 2, and SuSE 9.1 Professional several times on my personal ultimate box, and it has been more difficult than my first installation of Slackware 96 on a 386-33. Some of the problems were because of the motherboard for the Opteron (finally fixed with 2 BIOS updates). Some were because there are not 64-bit versions of applications (especially browser plugins).

I have ongoing issues with my SBLive 5.1 card because the ALSA fixes have not been included in the kernel updates that I have applied. But what has made this experience particularly upleasant has been wading through the hundreds of postings of false hope that turn up in my searches for what I need to do to solve my problems.

When I installed Slackware, I had to examine the supported hardware lists before even trying to install the kernel. To install XFree86, I ran the xf86config script, but because the database of chipsets was limited, I had to enter video timing information by hand. To customize FVWM desktop menus, I editted M4 macro files. I had to configure PPP manually and then wrote a shell script to choose between setting my modem to connect to my ISP, my employer, and my wife's employer. I configured applications in their dot files. Once, I fought a Sendmail configuration. I even learned how to custom configure x3270 in .Xresources.

This may sound overwhelming to Linux and FOSS converts who have only installed a Linux distribution using a GUI, but it really wasn't that bad. Of course, I was an IBM mainframer, working in the world of Job Control Language (JCL) and assembly language, so mere shell scripts and configuration files were not obstacles. But even more importantly the information was easy to find. There were current HOWTOs or man pages on every one of the above tasks to get me started.

Today, the Linux kernel supports 32 bit and 64 bit CPUs, SMP, IDE, EIDE, SCSI, SATA, RAID, USB, 3D video, soundcards, serial/cable/ISDN/DSL modems, NICs, printers, and many other hardware components, each with multiple chipset and firmware possiblities, that a self-administering user must manage when automatic detection is not successful or errors occur. But at least the amount of hardware to configure is limited.

There are multiple levels of Free/Open Source Software choices, including file systems, X and its window managers, the desktop environment, web browsers, email and newsgroup clients and servers, multimedia applications, office software, editors, programming languages and their libraries, development tools for desktop and web applications, databases and other server applications, and of course, games.

There are interdependent effects between the hardware and software. And then there are the dependencies, which may have dependencies that also have dependencies. We have the hard-working FOSS developers to thank for this abundance of choice. But it makes putting together a system more complicated every year.

Finding answers to all of the questions this situation poses is becoming more and more difficult. It is not because the answers are not there, we have plenty of people who are willing and able to help. Instead, the growth and commercialization of Linux and FOSS applications has had a subtle effect on the ability of the community to support its members. It is not differences between distributions raising the specter of forking. It is not the GUIfication of interfaces. It is not even proprietary code. The problem is that as the complexity of the system has increased, the information on how to manage that complexity has not kept pace. To put it bluntly, the documentation stinks.

This is not to say that documentation from proprietary sources is any better. Those manuals mostly explain the obvious and their troubleshooting sections blame the user for nearly all problems. But I am not the only one who is frustrated by the situation.

I believe that this is final barrier to widespread adoption of Linux. I really don't care about Aunt Tilly or Joe Sixpack. The newbies I would like to see are the somewhat technical users with zero Linux/UNIX background who are still intimidated by the learning curve. If this issue is eliminated, then more of them will be interested in trying out FOSS applications like Firefox and OpenOffice.org on Linux.

And sooner rather than later, the Linux and FOSS "market share" will be in double digits, and vendors will be motivated to support Linux on an equal footing with MS Windows and Mac OS. This will (1) make it easier to develop and support FOSS applications and (2) add to the number of people developing the cute little applications that are the only reason Aunt Tilly and Joe Sixpack own a computer.

Information is the key to control. The reason that the GPL has been successful is because it guarantees everyone access to information. But it is also the reason Google has been successful. When there is too much data to sort through, information is lost in the noise. Almost all of the information needed to install and maintain Linux and FOSS applications is available, it is just poorly organized. If the community is to maintain control of Linux and FOSS applications, we must ensure that the necessary information is openly available in Living HOWTOs. I am not arrogant enough to expect developers to do it for us or naive enough to expect somebody else to do it. The truth is, we in the Linux and FOSS user community have talked the talk, but we have not walked the walk when it comes to contributing back to documentation. We owe the FOSS developers the best documentation on the planet, and we can create it by working together more intelligently.

Living HOWTOs

The Linux Kernel Mailing List works because everyone on it has a similar background, so assumptions are legitimate. When someone says, "See so-and-so's patches to fix that," nothing more needs to be said. On the other hand, forums and newsgroups, when used for general support, should not make those assumptions but do so anyway. When seeking specific installation or configuration help, searching forums or googling newsgroups for the component plus an error message or other keywords might actually return similar questions, but the answers are more often than not incomplete. Most forum threads follow one of a few basic scripts:

  1. If the original post is particularly sarcastic or downright hostile, the responses are RTFM flames. These tend to be the longest threads.
  2. Other threads start with, "It's broke, tell me how to fix it," followed by an agonizing exchange, in which more experienced users desperately try to help a newbie ask the right question.
  3. Some requests get responses along the lines of, "I had a problem a little bit like yours and here is what I did... YMMV." And there it stops. You never know if the solution worked or not.
  4. Other queries get answers like, "Here are steps 3 and 4 in a 6 step process. Glad to be of help."
  5. But far too many have only the original post, either because the question has already been answered elsewhere but the poster was not creative enough with keywords to find it in his searching, or because no member of that particular forum knows the answer.

This is no more than throwing stuff against a wall to see what sticks. We need to admit that forums are our enemy.

Unfortunately, the best help from the past is no longer up to the job, because it is simply too overwhelming. There are several problems with HOWTOs as they exist today. Using the 8 RAID HOWTOs from the Linux Documentation Project, as an example, here are some major ones:

  1. There is usually one author, so the subject matter is often focused on a single issue which fairly quickly becomes outdated, such as the Antares-RAID-sparcLinux-HOWTO which is for hardware RAID built around the 5070 SBUS controller by Antares Microsystems.
  2. There is overlapping information. In the RAID group, there are 3 explanations of the RAID levels and 4 descriptions of building a kernel in varying degrees of detail.
  3. There are inconsistencies. Some have troubleshooting information in the chapters, others put it in appendices. One uses a question and answer format which makes it feel disjointed. Of the 8, there are 3 html navigation formats.
  4. I did not analyze the RAID HOWTOs for contradictions, but considering that they were written over a period between 1998 and 2004, there are undoubtedly differences in status displays and possibly command arguments. So to get a basic understanding of running RAID on Linux, at least 2 of the HOWTOs, and maybe more should be read.
  5. HOWTO maintenance depends on the author. Most ask for contributions to be emailed to them. This means that contributions might not be added, even if the email address is current. And to get an idea of how successful this is proving to be, here is the distribution of the latest Updates to all HOWTOs from the LDP:
    • 1995: 1
    • 1996: 5
    • 1997: 22
    • 1998: 36
    • 1999: 26
    • 2000: 63
    • 2001: 75
    • 2002: 100
    • 2003: 63
    • 2004: 78
    • Total: 469 (NOTE: The majority of these were originally written before 2000. However, there are 13 new ones since I started working on this article in July.)

I'm not putting down the folks at the LDP. They are doing their job which is not to write HOWTOs, but to maintain them. I just wish all the ones I need were there and up to date (and no, I am not going to write a dozen HOWTOs about subjects I know only a little about). The community needs living HOWTOs. The way to do accomplish this in a truly open way is with Wiki. Have you seen wikipedia lately? Starting from essentially nothing, it has grown to rival the best of commercial encyclopedias. This is not a new idea within the Linux and FOSS community, either. There are some Linux Wiki sites with basic information and a few projects are starting to use it. The information is fairly thin, which emphasizes the need for broad support from the community. To see what has been done, check out:

  1. LinuxQuestions.org Wiki
  2. The Waikato Linux Users Group Wiki
  3. GNU/Linux Users Group del Litoral Wiki
  4. The Fedora Core HOWTO (This is not a RedHat site, it is maintained by the fedora.us site and is hosted by the University of Hawaii.)
  5. The Arch Linux Wiki (Primarily information specific to Arch Linux.)
  6. The Knoppix Linux Wiki (Primarily information specific to Knoppix Linux.)
  7. The Alsa Open Source Wiki
  8. The CE Linux Public Wiki (This is used by an industry group that is focused on Linux for consumer electronics devices to coordinate their activities.)
  9. Linux Wiki (Apparently, I am not the only one interested in this type of project.)

Notice that the Fedora Core community (a subset of the Linux community) has set up their own Wiki to collect information about FC in one place. As a side note, I am blown away by what these folks have done. In less than a year, they have created multiple FC repositories, news sites, and help sites and have influenced the look and feel of a RedHat distribution with the addition of yum and non-FOSS licensed RPMs. There may be those who dislike RedHat, but it is obvious that there are also many who are completely dedicated to the distribution. RedHat should be thinking hard about the first birthday present (or party) for this community.

To produce the kind of documentation worthy of the Linux and FOSS community, we must take it upon ourselves to create and maintain a Linux HOWTO Wiki. We must focus on a central Wiki with links to FOSS project Wikis where they exist. If there are dozens of Linux Wikis competing with each other, then none will cover topics completely, and the situation will not be much better than it is now. If the distributions want to maintain their own Wikis to complement the main one with information unique to their distribution, that would be fine. But the Linux and FOSS community must be agnostic in documenting how Linux and FOSS works. We need detailed instructions for every hardware component, software driver, utility, and application, and examples, examples, and more examples. With dozens of people refining the most minute details, we may eventually discover vendor personnel reading it to learn how their own products work under Linux.

One section of the documentation should cover the infrastructure. This includes the Linux kernel, drivers, networking, system utilities, and X. Another section should document window managers, desktop environments, and FOSS desktop and server applications. While there are many introductory tutorials for major applications, we should aim for complete coverage. (I have seen too many postings sneeringly correcting someone who has claimed that some function cannot be performed. But if it is not documented, it may as well not exist.) For each subject, there should be:

  1. Overview: History, definitions, and the target audience as appropriate
  2. Installation and configuration: Step-by-step instructions with links to related procedures (ie. the configuration of /etc files would link to text editors), hardware, system, and library requirements, and useful explanations of all parameters
  3. Administration: Customized system-wide defaults, software plugins and hardware add-ons, upgrade procedures, security issues, how to collect and use performance information
  4. User guide: In some cases complete documentation, in others supplements to project documentation, links to tutorials
  5. Troubleshooting: Where to find error information, error message definitions, common gotchas
  6. References: Relevant man pages, where to get source code, links to helpful articles, related applications
  7. Acknowledgements: Project leaders, major contributors

Another value this Wiki could provide is the means for the Linux and FOSS community to set its own direction and express itself to the rest of the industry. Topics that could be covered in other sections include:

  • Explanations of Linux and Free/Open Source philosophy and licenses, with links to real-world examples of the advantages they provide to individual users, businesses, non-profits, and governments
  • Anti-FUD literature, with links to articles and sites that have supporting statistics
  • Honest and thorough evaluations of hardware in terms of Linux driver support
  • Programming topics, such as how to build useful scripts, how to write plug-ins for various FOSS applications, how to create X, Gnome, and KDE applications, how to write a driver module, and lots of sample code
  • Discussions of design standards for desktop and GUI interfaces, help pages, etc.
  • Discussions of new projects to test the need for applications and get a clear understanding of the feature set interested users really want, which has the potential of establishing the Linux and FOSS community as the leader in innovation, since the best source of innovation is from free and open discussion of ideas
  • Accumulations of prior art to aid in fighting any patent infringement cases brought against Linux or FOSS applications

To get the Linux HOWTO Wiki started, HOWTOs, info documents, man pages, and READMEs could be converted, depending on copyright limitations. There will need to be ongoing maintenance to make sure that requests for information are answered and ensure coherency. And in the true spirit of the Linux and FOSS community, it will need to be translated into many languages. Of course, this sounds like a lot of work, and it is. Freedom is not cheap. But we have a resource that is worth its weight in gold: Linux User Groups.

I believe that the main reason Linux and FOSS have had such incredible success is because of all the installfests held by LUGs. That created a critical mass of users and made it feasible for the for-profit distributions to invest in creating more user friendly installers and applications which led to more users. So lately, installfests have been fewer and farther between. If the LUGs were now to take ownership of sections of the Linux HOWTO Wiki, a strong foundation could be built quickly and maintained effectively. Each page could have a text or graphic logo of the LUG sponsoring it, which would help newbies understand what is really meant by the term "Linux and FOSS community."

To endure over the long term, the Linux HOWTO Wiki will have to be hosted by a neutral party with the resources to help maintain it. I thought of several, but narrowed it to 3 candidates:

  • The Linux Documenation Project: This site is hosted by ibiblio.org. Unfortunately, there is a thread on the general discussion list from 2002 about converting HOWTOs from docbook to Wiki in which most comments were negative and the end result was that it did not happen. Now that Wiki is more mature, and if there is a groundswell of support, they might change their position.
  • Existing help sites: The sites Linux Questions, JustLinux (LinuxNewbie.org), Linux Forums, and several others are already providing a variety of tools for helping users. Together, they could create a group of sites, each specializing in complementary areas, such as:
    • Kernel, Drivers, and GNU Utilities
    • X, Window Managers, and Desktop Environments
    • Office Applications
    • Server Applications
    • Development tools
  • Google: Our favorite search engine folks built one hell of a business from Linux and FOSS. Helping to document it would be an appropriate way to contribute back to the community that gave them that opportunity.

I will be contacting as many LUGs as I can to get a sense of how much support there is for this project. If there is a reasonable number in favor, then I will try to contact potential hosting sites to find a home for the Linux HOWTO Wiki. I will report back here on the status of this effort in a couple of months.

If you are interested in learning more about Wiki, here are two packages:

Jim Sansing was a network admin for 10 years and has been a network software developer for the last 10 years. He has used Linux as his only personal OS since 1996 and is looking forward to world domination.

Related Stories: