Reprinted by permission from Windows NT Magazine UPDATE –
Special Edition
by Mark Russinovich
Hello all,
Linux has received considerable attention from the trade press
and major computer companies over the past several months. More and
more, you see the word “enterprise” in a headline with Linux. For
example, “The Enterprise Comes To Linux,” “Enterprise Vendors Line
Up Behind New Red Hat Linux Distribution,” and “Major Players Bring
Linux to Enterprise” are all recent headlines from popular trade
rags. Major players in the computer world, such as IBM, HP, and
Compaq, have announced formal support for Linux, and have begun
selling hardware preinstalled with Linux. The ties between Linux
and enterprise computing extend to software vendors who have
announced Linux ports for applications common in the enterprise.
Oracle has ported its database offering to Linux, IBM has ported
DB2, and enterprise resource planning (ERP) solution providers are
porting, or considering porting, their applications to Linux.
The promotional momentum behind Linux as an enterprise-ready OS
gives the IS community a sense that Linux is a suitable enterprise
OS that has passed the scrutiny of enterprise-savvy hardware and
software companies. However, no evidence exists that these
companies or the trade press has looked at Linux critically with
respect to its enterprise-class capabilities. Enterprise-class
environments don’t widely use Linux–corporate environments have
only just put Linux to use as a uniprocessor server OS in the past
year or two. To date, not one Linux benchmark submission to the
Standard Performance Evaluation Corporation (SPEC) or the
Transaction Processing Council (TPC) has shown that Linux offers
performance comparable with what commercial versions of UNIX and
Windows NT deliver on large workloads. In fact, there are no SPEC
or TPC benchmark submissions with Linux as the OS.
The phrase “enterprise computing” means different things to
different people. Some see enterprise computing as a client/server
environment where many clients, or users, rely on few servers to
provide computing services. Thus, an enterprise environment is one
that demands hardware resources to handle large workloads that
basic PCs just can’t support. My interpretation of the phrase
defines an enterprise-class OS as one that must deliver high
performance in high-load environments, scale well on
multiprocessors, provide reliable service, and have high-
availability options built-in or that exist as add-ons. Example
enterprise environments include running an email server that
supports several thousands of users, hosting a Web site that
processes hundreds of thousands or millions of requests per day,
and maintaining a database that serves as a corporate data
repository with thousands or tens of thousands of employees.
A common theme to all these interpretations is that enterprise
computing involves a network server application with high compute
and I/O demands, and that server reliability and availability are
of utmost importance to a company’s bottom line. Another common
theme is that the necessary hardware typically consists of a
multiprocessor system with two to eight processors, 1GB or more of
RAM, and dozens of GB of disk space: Uniprocessor departmental
servers, small Web sites, and casually used company servers are not
the enterprise. What lies beyond these requirements is the high-end
of the enterprise (requiring more than eight processors). That
space probably accounts for less than a fraction of 1 percent of
the computing in the world and is reserved for mainframe OSs such
as OS/390 or highly-clustered UNIX solutions.
An enterprise-level OS must meet several requirements. First,
the OS must be reliable–it must not crash, even under extreme
loads. Second, availability services must be in place to let
clients continue to have their processing requests satisfied, even
when the OS or hardware fails. This functionality implies
redundancy at the hardware level, with OS or application support to
redirect client requests from a failed system to one that is
functional. Finally, the OS kernel’s architecture and
implementation must support high performance and scalability. In
other words, the kernel must be sufficiently parallel in its
operation to take advantage of multiple processors. The kernel must
also provide applications with the APIs and infrastructure
necessary to build high-performance, scalable applications.
How does Linux measure up to these requirements compared to the
current players in the enterprise space (i.e., UNIX and NT)? Linux
has a reputation as a reliable OS, and plenty of Web sites rely on
it. However, its high-availability support is still virtually
non-existent. Its Beowulf clustering, which has been in the trade
press recently, is not suitable for providing high availability to
network server applications. The only other measure to use to
compare Linux to the rest of the field is performance, including
multiprocessor scalability.
As I previously mentioned, no industry-accepted benchmark
results exist for Linux running network server applications.
Preparing for and executing such a benchmark is very costly, both
in terms of hardware dollars and technical expertise required to
execute a benchmark test that will yield competitive numbers. Thus,
the lack of any Linux benchmark results means one of three things.
First, no company with an interest in promoting Linux has run
benchmarks. Second, it’s not in the best interest of any company
capable of submitting a benchmark with good Linux numbers to do so.
Or third, companies are trying to get good Linux numbers and are
failing.
It’s not clear which companies capable of generating Linux
benchmark results have it in their best interest to do so. Compaq,
HP, Sun Microsystems, and IBM have all made pronounced gestures in
support of Linux, and so they are all obvious candidates. However,
they all have proprietary OSs that will compete with Linux as soon
as someone demonstrates that Linux is ready for the enterprise. In
fact, each company has its own version of UNIX. So it’s not obvious
which reason explains the lack of Linux numbers. The best we can do
is analyze the Linux kernel to determine whether its design and
implementation is capable of delivering good numbers.
Over the past few months, several writers in the technical press
have thrown out claims regarding the capabilities of the Linux
kernel. However, none of the authors that I’ve come across back
their claims with any technical facts. As someone that specializes
in OS internals and network server application design, I decided to
examine the Linux kernel to determine whether it incorporates
several key features required to implement high-performance network
server applications. I discovered that Linux is lacking in several
critical areas and that these limitations will hurt Linux’s
performance when compared to various versions of UNIX and NT. I
published my findings in “Linux and the Enterprise,” Windows NT
Magazine, April 1999. (Windows NT Magazine has decided to post the
electronic form of this article to its Web site at this time to
help dispel some misplaced Linux hype and to open discourse on this
subject. Go to http://www.winntmag.com/Magazine/Article.cfm?ArticleID=5048.)
Although my current findings are negative, the intent of revealing
them is not. I hope to promote Linux as a competitor in the
enterprise area, as competition is good for the industry and good
for consumers.
A report by the analyst group D. H. Brown entitled, “Linux: How
Good Is It?” includes a study of the Linux kernel that comes to the
same conclusion that I did. In fact, much of this group’s
conclusion related to Linux SMP support is based on my research–my
preliminary findings during my study of the Linux kernel were
posted, without my knowledge or permission, to the Linux kernel
USENET newsgroup in January by a member of the Linux community.
Pages 44 and 45, the heart of the SMP analysis, mirror my email
almost exactly. The fact is, Linux has a kernel that internally
looks very much like the prevalent UNIX kernel of the mid-1980s.
All commercial versions of UNIX and NT have evolved 10 to 15 years
past where Linux is as far as SMP scalability and network-server
application support. And the gains that have been made for these
other OSs have been difficult to achieve and expensive.
Despite a prevalent misconception in certain circles to the
contrary, NT does classify as an enterprise-class OS. NT’s
performance on network server applications rivals or beats the
performance the various versions of UNIX can provide. Don’t take my
word for it, go to http://www.specbench.org or http://www.tpc.org and see which OSs own
the top three positions on systems varying in size from
uniprocessor to 8-way systems. Across the board, NT ranks at the
top or near the top when competing against the major UNIX variants,
including Tru64 UNIX (formerly Digital UNIX), Solaris, and HP/UX.
As far as the belief widely espoused in advocacy USENET newsgroups
and Web sites to the contrary, NT is a reliable OS with
high-availability options. If it wasn’t reliable, no company would
use it as an enterprise OS, and companies with no particular
allegiance to Microsoft wouldn’t use it at all. The number of
companies visibly using NT in an enterprise-serving role are
plenty: Walt Disney, Dell, Intel, eBay, Barnes and Nobles, United
Airlines, Buy.com, and Car Point are just a sampling of some of the
world’s most demanding Internet sites running NT.
Will Linux gain the capabilities of its cousins and NT? Yes. Can
it gain them in the next few years based solely on work done by a
band of loose-knit developers scattered around the world? That
remains to be seen. The lessons offered by the evolution of the
other UNIX variants suggests that Linux requires a sponsor with
experience in this area to help both financially and technically.
Perhaps we’ll see the formation of an industry consortium that will
work together to help evolve Linux. However, it will be at least 2
years before the Linux kernel can mature to a competitive
level.
As a closing note, I want to provide a warning concerning
independent party OS benchmarks. Several publications have recently
reported benchmark results comparing Linux to NT. Some have shown
NT to outperform Linux, and some have shown the opposite. My advice
is to disregard independent tests because they suffer from several
flaws. First, by definition, an independent tester has no vested
interest in the outcome of a benchmark. Thus, they also have no
interest in understanding the hardware, OS, application, or
benchmark used in ways required for achieving the best results
possible. Only someone with a vested interest in good numbers can
do the best job possible to achieve those numbers.
Second, independent tests are almost never performed using
industry- accepted benchmarks such as SpecWeb or TPC-C/TPC-D. They
are typically run using workload programs developed in-house or by
third parties. Only tests that competing members of an industry
deem representative of real-world environments can be readily
accepted as meaningful.
Finally, no one audits independent tests. Representatives of
each member of the benchmark consortium review benchmark results
submitted to TPC or SPEC. This review ensures that these
organizations will publish only numbers obtained in adherence with
benchmark guidelines, and that means the submitter’s competitors
have critically viewed the numbers.
Thanks,
Mark Russinovich,
Contributing Editor, Windows NT Magazine
Mark@sysinternals.com
Copyright 1999, Windows NT Magazine