Joe Pranevich: Wonderful World of Linux 2.4 (Final Draft)

[ Joe has reluctantly concluded that “final” is too elusive
a goal — the
“Turkey Day” update is now available – LT ed. ]

The Many Flavors of Linux

In terms of sheer lines of code, the Linux kernel is predominantly
drivers. In fact, the size of the Linux core has not increased much
over the last several revisions. Some of these drivers are
architecture independent such as the IDE driver. That is, these
drivers have been written to work on multiple platforms. Other
drivers are dependent on a particular architecture. For example,
the ADB (Apple Desktop Bus) mouse driver isn’t really applicable on
the i386 port and so isn’t supported. Linux kernel developers
strive to make drivers as general as possible, so as to allow a
driver to be reused with relatively little effort on a different
platform if a device becomes available. It is easiest to imagine
the Linux kernel being a single entity with subtle variations
depending on the platform it is bring used on.

Having said this, this document will mostly stick to Intel
hardware as that is the hardware that I use most often at home.
While I won’t go into the specifics of each individual port because
I lack the time and the knowledge, it should be mentioned that
Linux 2.4 adds support for three new architectures: ia64 (Itanium,
the successor to i386), S/390 (an IBM mainframe), and SuperH
(Windows CE hardware). Linux 2.4 also includes support for the
newer 64-bit MIPS processors. I have no experience with these
platforms so am unsure as to their level of hardware support, etc.
It is likely that they will mature over the years just as the other
Linux ports have. Exact feature support is different from port to
port and certain limitations on hardware, memory, etc. will differ
depending on the underlying architecture.

In terms of Intel-like hardware, Linux 2.4 is very similar in
support to Linux 2.2. All Intel chips since the 386 are still
supported, up to the Pentium III. MMX and MMX2 are also supported.
Optimizations have been added to speed up Linux on all processors,
but especially on the Pentium III. Compatible chips such as those
produced by AMD and Cyrix are also supported. Additionally, Linux
2.4 will include support for other hardware often present on newer
chips including non-Intel varieties of the MTRRs (Memory Type Range
Registers) (aka MCRs) which will improve performance on some kinds
of high bandwidth devices. While Linux 2.2 included support for the
IO-APIC (Advanced Programmable Interrupt Controller) on
multi-processor systems, Linux 2.4 will support these on
uni-processor systems and also support machines with multiple
IO-APICs. The support for multiple IO-APICs will allow Linux 2.4 to
scale much better than previous incarnations of Linux on high-end
hardware.

Linux 2.4 and ia64 (formerly Merced)

While not yet delivered to the starving masses, Intel’s 64-bit
replacement to the x86 line is coming down the pipeline. While no
real hardware is available, patches that include support for this
chip and its successors have been included in the mainstream kernel
release. This porting process was no doubt simplified by Linux’s
existing support for 64-bit processors (including Compaq’s Alpha
chips and the Sparc64) which were already merged into the main
Linux tree.

Linux 2.4 and Pre-386 Intel Chips

Surprisingly, I do get a number of questions about pre-386 Linux.
The answer, at least right now, is that there is no such animal. A
sister project, ELKS (Embedable Linux Kernel Subset) is working to
make a Linux-like operating system run on these machines, including
protected mode support for chips that support it. This project is
separate from Linux-proper however and is outside the scope of this
document.

A separate port of the Linux kernel, called uLinux, also exists
and is working to provide Linux on embedded and older processors,
including processors without MMUs. The work presently is based
around the Linux 2.0 kernel and has largely not been integrated
into the master tree.

Linux Internals

Linux 2.2 was a major improvement over Linux 2.0 and the Linux 1.x
series. It supported many new filesystems, a new system of file
caching, and it was much more scalable. (If you want a list of
features new to Linux 2.2, you can read my article about it.) Linux
2.4 builds on these things and more to be the best darned Linux
kernel yet in a variety of situations.

The Linux kernel is an assortment of modular components and
subsystems including device drivers, protocols, and other component
types. These are glued to the core of the Linux kernel by APIs,
programming interfaces, that provide a standard method by which the
Linux kernel can be expanded. Most of this document will focus on
these components of the Linux OS as these are the components that
seem to do the most work: drive your disks, read your files, and do
all of the obvious and physical things. Linux 2.4 is however much
more than just these components. These assorted drivers and APIs
all revolve around a common center of the Linux kernel. This center
includes such fundamental features as the scheduler, the memory
manager, the virtual filesystem, and the resource allocator.

Linux 2.4 is the first release of the Linux kernel which will
include a full-featured resource management subsystem. Previous
incarnations of Linux included some vestiges of support, but it was
considered kludgy and did not provide the functionality needed for
the “Plug and Play” world that we live in today. Unlike many of the
other internal changes, many users will be able to directly
experience this change as it impacts the way resources are
allocated and reported in the kernel. As part of this change, the
PCI card database deprecated in Linux 2.2 has been un-deprecated so
that all resources can have an associated device name, rather than
just an associated driver.

The new release of the Linux kernel also fixes some problems
with the way the VFS (virtual filesystem) layer and the file caches
were handled. In older versions of Linux, file caching was
dependent on a dual-buffer system which simplified some issues, but
caused many headaches for kernel developers who had to make sure
that it was not possible for these buffers to be out of synch.
Additionally, the presence of the redundant buffer increased memory
use and slowed down the system as the kernel would have to do extra
work to keep things in synch. Linux 2.4 solves these problems by
moving to a simpler single-buffer system.

A number of changes in Linux 2.4 can be described as “enterprise
level.” That is, they may not be immediately useful to many desktop
users by work to strengthen Linux as a whole. For the most part,
the addition of these features does not degrade Linux in more
“normal” environments. First, Linux 2.4 can handle many more
simultaneous processes by being more scalable on multiprocessor
systems and also by providing a configurable process limit. Second,
the scheduler has been revised somewhat to be more efficient on
systems with a larger number of concurrent processes. Third, the
revised Linux kernel can now handle an amazing number of users and
groups– about 4.2 billion. (And that’s a lot of users!) In
addition, support for more powerful hardware is provided in the new
kernel which now supports 64 gigabytes of RAM on Intel hardware, up
to 16 ethernet cards, 10 IDE controllers, multiple IO-APICs, and
other pointless abuses of good hardware. The 2 gigabyte file size
restriction has also been lifted. With these changes and others,
the Linux kernel development team is proving that Linux can be an
option in many new environments.

The way Linux handles shared memory has also been changed in
Linux 2.4 to be more standards compliant. One side effect of this
set of changes is that Linux 2.4 will require a special “shared
memory” filesystem to be mounted in order for shared memory
segments to work. This should be handled by the distributions when
they become ready for Linux 2.4.

Linux 2.4 also includes a much larger assortment of device
drivers and supported hardware than any other Linux revision and
any particular device you care to name has a decent shot at working
under Linux 2.4. (Of course, you should consult the documentation
before you go out and buy any new hardware, just in case. New
hardware especially may not be supported yet.)

One frequently asked question about Linux 2.4 is how much memory
it will require. Many operating systems seem to require more and
more memory and resources as they mature, but Linux 2.4 will
largely buck that trend by actually requiring less memory in
certain situations. Of course, Linux 2.4 includes much more
functionality than does Linux 2.2 and many of these features do
take up space so your mileage may vary. (Remember that most kernel
components can be disabled at compile-time, unlike many other
operating systems.)

Hardware Support – /dev/*

Before we can talk about Linux 2.4’s hardware support, I have to
bring attention to one of Linux 2.4’s latest and most controversial
features: DevFS: the device filesystem. DevFS is a (currently
optional) feature that fundamentally rewrites the way Linux users
handle interactions with devices. This will be primarily presented
to the user in two very obvious ways. First, nearly all device
names have been changed. For example, “/dev/hda” may have been your
harddisk, but it would now be located at “/dev/ide0/…” (I’m not
sure exactly what the new naming convention will be.) This modified
scheme increases the available namespace for devices and allows USB
and other “modern” device systems to be more easily integrated into
the UNIX/Linux device model. Secondly, device names will now be
added to the /dev/ directory as drivers are loaded into the kernel
rather than having all possible device names pre-existing in the
directory. Old names will still be available for compatibility
using a userspace program “devfsd.”

While this may impact distribution maintainers who want to make
sure that all applications are modified to use the new names, end
users should not be overly affected by these changes. This change
will, if nothing else, come as a surprise to many users. While some
users may be turned off by the more verbose naming of devices, it
is easy to see how limiting the older names could be. (What, for
instance, would happen if you had more than 26 harddisks, such as a
large fileserver doing software RAID?) There are some commonly
accepted drawbacks to the new system (persistent permissions on
device nodes, as an example) but nearly all of these can be solved
in userspace.

Buses – ISA, PCI, USB, MCA, etc.

Processors are just one small part of the nifty world that exists
inside your computer. Equally important is the computer’s bus
architecture, the component(s) of the system that is often
responsible for internal and external devices. Some bus
architectures, such as the original ISA, are more irresponsible
towards their hardware than anything else– they don’t provide any
resource management functionality, just a place to put in cards.
Others, such as PCI, support much more advanced levels of
configuration and allow for devices to be relocated and other
things. As all primary internal Intel-hardware busses were
supported by Linux 2.2 ((E)ISA, VLB, PCI, MCA), there are no really
amazing announcements to be made in this area. Linux 2.4 does
however improve on each of these busses by stringing them into
Linux’s new resource subsystem.

There are two major improvement that were made in this area,
however. Linux 2.4 includes, for the first time in the kernel,
support for ISA Plug-and-Play devices. ISA Plug-and-Play is an
extension to the ISA architecture which allowed for PCI-like
intelligence on cheaper hardware. Previously, Linux could support
these devices through a user-mode utility and some elbow grease on
the part of the user or distribution. Linux 2.4 however will allow
these devices to be used during the boot process (for example,
booting from an ISAPnP IDE controller) and the configuration of
these devices happen auto-magically. Linux 2.4 also includes I2O
support in the kernel. I2O (Intelligent Input/Output) is a superset
of PCI that attempts to allow OS independent drivers to be written
for many devices. Between these two changes, many users will find
that Linux 2.4 supports many more pieces of PC-class hardware.

Linux 2.4 also has increased support for external devices. PC
Card (aka PCMCIA) support has been added into Linux 2.4. This
support was available in many distributions from an outside source
and most distributions included it by default. Linux 2.4 supports
better integration with the PCMCIA driver set and should ease the
installation and configuration process for these devices for many
users. Like previous versions of this driver however, Linux 2.4
will still require an external daemon and components to get the
most out of these devices.

Perhaps the most exciting news on this front is the Universal
Serial Bus (USB), an external bus that is coming into prominence
for devices such as keyboards, mice, sound systems, scanners, and
printers. USB is a popular option on many new pieces of hardware,
including non-Intel hardware. Linux’s support for these devices is
still in early stages but a large percentage of common USB hardware
(including keyboards, mice, speakers, etc.) is already supported in
the kernel.

More recently, Firewire (IEE1394) support has been added into
the Linux kernel. Firewire is a popular option for many
high-bandwidth devices. Not many drivers (or devices) exist for
this hardware architecture yet, but this support is likely to
improve over time, as the architecture matures.

Block Devices – LVM, Disk Drives, etc.

In its simplest form, a block device is a device which can be
expressed as an array of bytes that can be accessed
non-sequentially. This would include devices such as disks (where
you can read any sector you want) but not serial ports (because you
can only read what is at the end of the wire.) Extensions to this
concept (such as ejecting a disk, etc.) are handled in Linux
through ioctls (I/O Controls). The concept of block devices hasn’t
changed in quite a while and support for things such as IDE and
SCSI disk drives has been present since the first revisions of the
Linux kernel.

In Linux 2.4, all the block device drivers have been rewritten
somewhat as the block device API has been changed to remove legacy
garbage from the interface and to completely separate the block API
from the file API at the kernel level. The changes required for
this API rewrite have not been major. However, module maintainers
who have modules outside the main tree may need to update their
code. (One should never assume full API compatibility for kernel
modules between major revisions.)

On the desktop at least, disk devices that use the IDE bus are
most prevalent. Linux has supported IDE since the earliest kernels,
but Linux 2.4 has improved support for these devices in a number of
ways. First off, high-end systems that have multiple IDE
controllers may benefit as the number of supported IDE controllers
has been increased from 4 to 10. As most motherboards are shipped
with a maximum of two, this is not likely to impact many desktop
users. Secondly, there have been changes in the IDE driver which
will improve Linux 2.4’s support for PCI and PnP IDE controllers,
IDE floppies and tapes, DVDs and CD-ROM changers. And finally,
Linux 2.4 includes driver updates which should work around bugs
present in some IDE chipsets and better support the advanced
features of others, such as ATA66.

While it would seem that the SCSI subsystem has not changed as
much as the IDE subsystem, the SCSI subsystem has been largely
rewritten “under the hood.” Additionally, a number of new SCSI
controllers are supported in this release. A further SCSI cleanup
is expected sometime during the 2.5 development cycle.

One completely new feature in the Linux 2.4 kernel is the
implementation of a “raw” I/O device. A raw device is one whose
accesses are not handled through the caching layer, instead going
right to the low-level device itself. A raw device could be used in
cases where a sophisticated application wants complete control over
how it does data caching and the expense of the usual cache is not
wanted. Alternatively, a raw device could be used in data critical
situations where we want to ensure that the data gets written to
the disk immediately so that, in the event of a system failure, no
data will be lost. Previous incarnations of this support were not
fit for inclusion as they required literally doubling the number of
device nodes so that every block device would also have a raw
device node. (This is the implementation that many commercial
UNIXes use.) The current implementation uses a pool of device nodes
which can be associated with arbitrary block devices.

One huge area of improvement for Linux 2.4 has been the
inclusion of the LVM (Logical Volume Manager) subsystem into the
mainstream kernel. This is a system, standard in Enterprise-class
UNIXes such as HP-UX and Tru64 UNIX (formerly Digital UNIX), that
completely rethinks the way filesystems and volumes are managed.
Without going into detail, the LVM allows filesystems to span
disks, be resized, and be managed in a more flexible way. Some of
the features of the LVM subsystem can be replicated with the md
(multiple device) driver or some userspace tools. However, the LVM
subsystem offers this support in a (de facto) standards-compliant
manner that will also be at least somewhat familiar to users of
commercial UNIXes.

Another huge area of improvement for Linux 2.4 will be in its
support for RAID devices, multiple disks working together to
provide redundant storage or speedier read accesses. In the new
kernel, nearly the entire RAID subsystem has been rewritten.
Performance, probably the most important facet of a complete RAID
implementation, has been improved on both SMP and uniprocessor
systems. (On SMP, it’s now better threaded.) Additionally, the code
has been made much more robust with the ability to make RAID arrays
recursive and be able to mount sets for a root disk without the use
of a ramdisk image. As more enterprise-level users approach Linux,
features like a roubst RAID subsystem become gating factors between
acceptance and non-acceptance. Linux 2.4 again raises the bar.

Filesystems and Partition Tables

Block devices can be used in a number of ways. The most common way
to use a block device is to make a filesystem out of it.
(Internally, the filesystem code is like an overlay on the block
device driver.) Other ways that a block device can be used include
partitioning (which is a lot like a filesystem, just handled in a
completely different way), and using it raw.

Linux 2.4 includes all of the filesystems present in Linux 2.2.
Those filesystems include FAT (for the assorted DOSes), NTFS (for
Windows NT– Windows 2000 support is spotty), VFAT and FAT32 (for
Windows 9x), HFS (for MacOS), HPFS (for OS/2), and a variety of
others. New filesystems have been added, most notably the UDF
filesystem used on DVD disks and the EFS filesystem used by older
versions of IRIX. All filesystems have been rewritten to some
extent to support the new page caching system and will be more
efficient because of this change. The only exception to this
appears to be the NTFS filesystem which is currently lacking a
solid maintainer and is not currently stable coming into the home
stretch before Linux 2.4.

There are a number of improvements which will improve
compatibility with other systems. OS/2 users will finally be able
to write to their filesystems from within Linux. NT users do not
have this luxury yet as their driver is still in the experimental
stage. NextStep users will be able to mount their CD-ROMs under
Linux as Linux supports an extension to the UFS filesystem that
NextStep uses. It should be noted that HFS+, the new Macintosh
filesystem, is not yet supported by Linux.

Linux 2.4 does not yet include a journaling filesystem, although
several projects are close to providing this functionality in a
stable fashion. While it is almost certain at this point that a
journaling filesystem will not be supported for Linux 2.4, it is
expected that at least one journaling filesystem will be added
during the 2.4 cycle (before the developers break off again to
begin work which will eventually lead to Linux 2.6 or
whatever.)

Additionally, the partition table handling code has been
rewritten and now allows for a much larger selection of non-native
partition table types to be used. This would be useful if you have,
for example, an external SCSI drive from a Macintosh and you want
to use it on your Linux PC. A number of new partition table types
have been added, including the format for IRIX machines.

Not all filesystems are mounted over block devices. Some, like
the proc, shared memory, and devfs filesystems, are completely
virtual. Others are “mounted” over the network. There are a number
of ways to accomplish this end and many OSes provide their own
methods of doing this.

The Windows world uses the Server Message Block (SMB) protocol
for their network filesystems. The new Linux kernel removes the
compile-time workaround that required you to choose whether you
would be mounting drives from Windows 9x or NT/2000. This updated
Linux kernel will be able to auto-detect the remote system type and
enable bug fixes on an as-needed basis. This will vastly improve
Linux’s ability to operate in networks with multiple versions of
Windows.

In the UNIX world, the Network Filesystem (NFS) protocol is the
method of choice for sharing files. Linux 2.4 includes for the
first the the ability to access filesystem shares over the most
recent version of the NFS protocol, NFSv3. NFSv3 includes many
advantages over previous versions and is one of Linux’s most
requested features. (Support for NFSv4 has been announced to be
under development.)

Character Devices – Keyboards, Mice, Consoles, and Ports

The class of devices which can only be accessed sequentially is the
character device. These are devices, such as serial devices, which
allow you to read from a stream or push data onto it, but not to
“skip” ahead or behind. This includes serial and parallel ports,
keyboards, mice, and terminal devices. There have been several
major improvements in this area for the latest incarnation of the
Linux kernel.

One of the largest improvements in this area is in regards to
Linux 2.4’s support for keyboards and mice. Previous incarnations
of Linux included support for serial and PS/2 mice and keyboards
(and ADB, for instance, on the Macintosh.) Linux 2.4 also supports
using keyboards and mice attached to the USB ports. Additionally,
Linux 2.4 also supports keyboards on some systems where the
keyboard is not initialized by the BIOS and systems that have
trouble determining whether a keyboard is attached or not. And
finally, Linux 2.4 includes expanded support for digitizer pads and
features an emulation option to allow them to be used as normal
mice, even when this is not directly supported in hardware.

Linux’s support for serial ports has not changed much since the
days of Linux 2.2. Linux 2.4 (and some later versions of Linux 2.2)
supports sharing IRQs on PCI serial boards; previously, this
feature was limited to ISA and on-board serial ports. Additionally,
Linux 2.4 has added a number of new drivers for multi-port serial
cards. It is hoped that these changes and others will make using
your serial ports under Linux 2.4 easier than before.

In a separate department, there has been some work since 2.2 on
supporting so-called “WinModems” (or “soft modems”). These are
modems which exist largely in software and whose drivers are often
only provided by the manufacturer for Windows. (Often the DSP or
other parts of the hardware must be implemented in software rather
than on the board.) While no code has been submitted to Linus for
the support of these beasts, several independent driver projects
have been working to get some support for these beasts in and the
first fruits of these labors are becoming usable outside the main
tree. While it will be a long time before we see most of these
devices supported under Linux, for the first time it actually
appears that the Open Source snowball is beginning to roll in this
direction.

Linux 2.4 also includes a largely rewritten parallel port
subsystem. One of the major changes in this area is support for
so-called “generic” parallel devices. This functionality can be
used by programs which access the parallel ports in unusual ways
or, more likely, just want to probe the port for PnP information.
Additionally, this rewrite allows Linux 2.4 users to access all the
enhanced modes of their parallel ports, including using UDMA (for
faster I/O) if supported by the hardware. Under the new Linux
kernel, it is also possible to direct all console messages to a
parallel port device such as a printer. This allows Linux to match
the functionality of many commercial UNIXes by being able to put
kernel and debug messages on a line printer.

Multimedia: Sound, TV, Radio, etc.

On the complicated side of the character device list, we have some
of the less essential devices to be supported by Linux. Linux, in
its emerging role as a desktop platform, tries very hard to support
sound cards, TV and radio tuners, and other sound and video output
devices. To be honest, Linux 2.4 does not include as many
ground-breaking changes as Linux 2.2 did in this respect. Linux 2.4
does, however, include updates and new drivers for a variety of
sound and video cards, including adding full duplex support. Linux
2.4 and some later versions of Linux 2.2 also include code which
will allow some sound devices to more easily allocate memory in
required ranges; this should make the configuration and use of some
cards much easier.

Work is in progress on a completely rewritten sound subsystem
which will support many of the more advanced features of today’s
sound cards. This support will not be present in Linux 2.4, but may
make it into the kernel for Linux 2.6.

Video Cards

Another more complicated variety of device is the frame-buffer, a
way of looking at many video cards. A frame-buffer is simply a
section of memory that represents (or is) video memory to such an
extent that writing to this memory affects the colors of the pixels
on a screen. This is more complicated than most other devices
because it requires ioctls to change the palette and to perform
other video-related functions.

Linux 2.4 includes a number of new drivers and improvements to
old drivers. Especially important here is Linux’s support for many
more “standard” VGA cards and configurations, at least in some
modes. (Even if the mode is only 16 colors– at least it works.)
Please remember that this feature can be bypassed and (on i386) is
only necessary for people with certain systems which cannot be
supported in any other way. At this time, the XFree project
provides many more drivers to many more video cards than the kernel
can support so it is not necessary to use this feature to get
support for the X Window System. (SVGAlib and other libraries also
allow you to do direct video manipulation on supported hardware,
however the use of these libraries must be done carefully as there
are some security concerns and race conditions.)

One of the biggest changes in this respect is the addition of
the Direct Rendering Manager to the Linux kernel. The DRM cleans up
access to the graphics hardware and eliminates many ways in which
multiple processes which write to your video cards at once could
cause a crash. This should improve stability in many situations.
The DRM also works as an entry point for DMA accesses for video
cards. In total, these changes will allow Linux 2.4 (in conjunction
with Xfree4.x and other compatible programs) to be more stable and
more secure when doing some types of graphics-intensive work. These
changes should also make some kinds of television tuner cards more
workable under Linux.

Accessibility

When thinking of Linux, the words “user friendly” do not generally
come immediately to mind. Therefore, one might be surprised to
learn that Linux 2.4 (and some later editions of the Linux 2.2
kernel) supports speech synthesizer cards. This driver and the
appropriate hardware will allow vision-impaired Linux users to hear
all Linux output, including kernel messages very early in the boot
process. Very few operating systems can boast such low level
support for these devices. (There will be other patches and
utilities that will be required for full use of these devices, this
kernel driver is only a component of the system.)

Networking and Protocols

Networking and network hardware is one of the major areas where
Linux has always excelled. These devices are neither “character”
nor “block” but inhabit a special space free of the need for device
nodes. Linux 2.4 will include many improvements to this layer
including new drivers, bug fixes, and new functionality added on to
existing drivers.

The Linux model of network sockets is one which is standard
across most UNIX variants. Unfortunately however, the standard does
have some correctable deficiencies. Under Linux 2.2 and previous
versions, if you have a number of processes all waiting on an event
from a network socket (a web server, for instance), they will all
be woken up when activity is detected. So, for every web page
request received, Linux would wake up a number of processes which
would each try and get at the request. As it does not make sense
for multiple processes to serve the same request, only one will get
to the data; the remainder will discover nothing to process and
fall back asleep. Linux is quite efficient at making this all
happen as quickly as possible, however it could still be made more
efficient by removing the redundant wakeups. Linux 2.4 includes
changes which implement “wake one” under Linux which will allow us
to completely remove this “stampede effect” of multiple processes.
In short, “wake one” does exactly as its name indicates: wakes up
only one process in the case of activity. This will allow
applications such as Apache to be even more efficient and make
Linux an even better choice as a web server.

Linux 2.4 also includes a completely rewritten networking layer.
In fact, it has been made as unserialized as possible so that it
will scale far better than any previous version of Linux.
Additionally, the entire subsystem has been redesigned to be as
stable as possible on multiprocessor systems and many possible
crashes have been eliminated (this is part of the so called “soft
net” changes that have been recently integrated.). In addition, it
contains optimizations to allow it to work with the particular
quirks of the networking stacks in use in many common operating
systems, including Windows. It should also be mentioned at this
point that Linux is still the only operating system completely
compatible with the letter of the IPv4 specification (Yes, IPv4;
the one we’ve been using all this time) and Linux 2.4 boasts an
IPv4 implementation that is much more scalable than its
predecessor.

As part of this major rewrite, the firewall and IP masquerading
functionality of the kernel has been completely rewritten again.
(Older users may remember that these same components were largely
rewritten for Linux 2.2 also.) The new subsystem has been split
into two parts: a packet filtering layer and a network address
translation (NAT) layer. These new subsystems are considerably more
generic than their predecessors, and it is now possible to do most
types of sophisticated (level 3) routing through any Linux box.
Previously, this kind of functionality was largely only available
with dedicated and proprietary routing hardware. Unfortunately,
this major rewrite also includes yet another new userspace tool to
manage the available functionality. For compatibility, modules
exist which will allow you to use either the Linux 2.0 (ipfwadm) or
Linux 2.2 (ipchains) tools without a major loss of functionality.
This will make the upgrade from either of these kernel versions
relatively seamless.

For Enterprise-level users, there are a number of features that
will better enable Linux to integrate into older and newer
components of existing network infrastructures. One important
addition in this respect is Linux 2.4’s new (partial) support for
the DECNet and ARCNet protocols and hardware. This allows for
better interoperation with specialized systems, including older
Digital/Compaq ones. Also of special interest to this class of
users, Linux 2.4 will include support for ATM network adapters for
high-speed networking.

For the low-end desktop users, PPP is an important part of day
to day life. Linux 2.4 includes some major rewrites and
modularization of much of the code, including a long awaited
combination of the PPP layers from the ISDN layer and the serial
device PPP layer, such as for dial-up connections with modems. In
addition to the modularity, ISDN has been updated to support many
new cards. The PLIP (PPP over parallel ports) layer has also been
improved and uses the new parallel port abstraction layer. And
finally, PPP over Ethernet (PPPoE, used by some DSL providers)
support has been added to the kernel.

Although not present in Linux 2.4, there is work now on
supporting the NetBEUI protocol used by MS operating systems. While
Microsoft will be moving away from this protocol in its products
and towards TCP/IP, this protocol is still important for a number
of Windows-based network environments. (Previously, kernel
developers had commented that the protocol is too convoluted and
buggy to be supported in the kernel. Now that an implementation has
surfaced, it remains to be seen whether it will be stable enough to
ever be in an official kernel.)

The Program Loader

One often overlooked portion of the Linux kernel is the program
loader; the bit that takes your programs, loads them properly, and
runs them. Many people are not aware however that Linux 2.2 added
support for a “miscellaneous” binary loader, a flexible module
designed to allow you to associate binary types (based on extension
or file header information) with “helper” applications in much the
same way as Windows or a comparable operating system would. This
would, for example, allow you to associate all Windows applications
on your machine with WINE (Windows Emulator) so that when you typed
“./notepad.exe” the right thing would happen. (However it is
generally not a good idea to take this concept to the extreme at
the kernel level and many of the “associations” provided by Windows
would be best left handled by your file manager or desktop
environment. It would be a bad idea, for example, to be able to run
“/etc/passwd” and have it come up in a text editor. My personal
recommendation is to use this functionality only when the file type
is best imagined as “executable.”) This was considered a big win by
many because it allowed many different groups, such as the WINE
(Windows non-Emulator) and Dosemu (DOS Emulator) groups, to publish
instructions for making their programs run “native” by the kernel.

Linux 2.2 and Linux 2.0 included built-in support for starting a
Java interpreter (if present) whenever a Java application was
executed. (It was one of the first OSes to do this at the kernel
level.) Linux 2.4 still includes support for loading Java
interpreters as necessary, but the specific Java driver has been
removed and users will need to upgrade their configurations to use
the “Misc.” driver.

The Kernel Web Daemon

One of the most striking features in the Linux 2.4 kernel is the
kernel web daemon, or khttpd. It’s true, Linux actually supports a
kernel module which can process HTTP requests without having to
communicate with any user space servers (such as Apache.) This
feature is often misunderstood– it is not designed to replace
Apache or any other web server and it can be used only when serving
raw files (no CGI.) If it receives a request for something that it
cannot handle, it will “pass through” the request to user space
where a web server can snatch it up and process it without ever
being able to know the difference. This feature will make Linux an
even better choice for rapid-fire web serving of static content,
such as dedicated image servers.

Kernel-level Encryption

Although export laws in the US have been changed since Linux 2.2,
Linux 2.4 is not likely to include support for cryptography in the
main distribution. Import and export regulations for cryptography
are different around the world and many Linux developers are loth
to make it harder for anyone in the world to develop for the Linux
kernel. Already however, patches are available to add encryption to
several applicable kernel subsystems. It remains to be seen how
official these patches will be made in the future.

Documentation

In addition to many, many feature changes, Linux 2.4 also includes
a much more expansive set of documentation included with the
kernel. Included in the set is, for the first time, documentation
in the DocBook documentation format, a format similar to HTML that
has been embraced by GNOME and other GNU projects.

—

Linux 2.4 is almost ready and already has shown itself to be the
best Linux kernel yet in a number of respects. Although I do not
expect many new features before release, I will attempt to keep
this up to date. If you have a suggestion, please email me and I’ll
think about adding it. I tend to lose suggestions over time (I do
get paid for something else, and I don’t always have time to add
things to this document in a timely fashion) so if I don’t respond
or miss out on you in the next release, you should probably try
again. I’m not even going to tell you how much mail I get
everyday…

This is the probably not the final version, but it’s close. If
you haven’t commented on it by now and I missed something that you
feel is important, please email me at jpranevich@linuxtoday.com or
jpranevich@lycos-inc.com and
let me know.

About this Document

This document has been written by me (Joe Pranevich) based on lots
of personal research and suggestions by many Linux users.
Reproduction and translation are permitted (and encouraged)
provided that the entire text remains intact. (Formatting changes
and any changes required for translation are fine. Additional
“Translated by” or similar credits may be added.) For reproduction
on a website, I request that you send me a note and let me know the
URL. For reproduction in a print medium, email me first. (In those
cases, I generally ask for a copy or two for my shelf.) All other
rights are reserved, of course.