Spec.org, the web site for the Standard Performance Evaluation Corporation, has
published their latest SPECweb99 Benchmark for about 20 computer systems. Their mission
is: "To establish, maintain, and endorse a standardized set of relevant benchmarks and
metrics for performance evaluation of modern computer systems."
The SPECweb99 Benchmark is:
"the next-generation SPEC benchmark for evaluating the performance of World Wide Web
Servers. As the successor to SPECweb96, SPECweb99 continues the SPEC tradition of giving Web users the
most objective and representative benchmark for measuring a system's ability to act as a web server. In response
to rapidly advancing Web technology, the SPECweb99 benchmark includes many sophisticated and
state-of-the-art enhancements to meet the modern demands of Web users of today and tomorrow."
Spec.org has just published the
"Second Quarter 2000 SPECweb99 Results".
Twenty one machines from various vendors including IBM, Dell, Compaq, and Hewlett Packard were tested and
scored based on their ability to handle "Conforming Simultaneous Connections".
The interesting part is that two of the machines are from Dell, with almost exactly the
same hardware and yet very different benchmark results.
Both machines are Dell PowerEdge 6400/700, one running Red Hat Tux 1.0 and the other
running Windows 2000 Advanced Server. Both were running with 4 CPU's (700MHz Pentium III Xeon),
4 Network Controllers and 8GB of RAM.
There were a few differences in the hardware: the W2K machine had 7 9GB 10kRPM hard drives
with a Dell PERC2 Disk Controllers and the Tux 1.0 machine had 5 9GB 10kRPM hard drives with
an Adaptec AIC-7899 SCSI Disk Controller. In addition, the Client machines, while being
very similar in configuration, had the same brand of Network Controller (Alteon) but a
different model number for the W2K test (AceNIC PCI) than the one for the Tux 1.0 test (AceNIC 1000SX).
Here is the interesting part: The W2K machine received a score of SPECweb99 = 1598,
which means that it was able to handle a median of 1598 Conforming Simultaneous
Connections. The Red Hat Tux 1.0 machine received a score of SPECweb99 = 4200, which means that
is was able to handle a median of 4200 Conforming Simultaneous Connections.
The full disclosure page for the Windows 2000 machine may be found
here.
The full disclosure page for the Red Hat Tux 1.0 machine may be found
here.
What does this mean? In the real world, probably not as much as it would seem. Benchmarks
in general are typically set up in an ideal environment. Real world environments tend
to be quite different. However, this does indicate that Linux is moving in the right direction.
I freely admit that I am not a hardware expert. So my question to those who know more about
hardware than I do is: Could the seemingly minor hardware differences result in this huge difference
in the results of the tests? Please feel free to comment below on these results and in benchmarks in general.
