Linux Today - SPECweb99 Benchmark clarification

search.internet.com

Become a Marketplace Partner

internet.commerce

Be a Commerce Partner

Linux Today
Enterprise Linux Today
Apache Today
JustLinux.com
Linux Planet
PHPBuilder
All Linux Devices

Technology Jobs

LinuxToday Newsletters

Server Daily
IT Management Daily
Subscribe News
Subscribe PR
Subscribe Security

internet.com

IT
Developer
Internet News
Small Business
Personal Technology

Search internet.com
Advertise
Corporate Info
Newsletters
Tech Jobs
E-mail Offers

Current Newswire:

Oracle Squelches Open Source Spirit

Samsung unveils Android tablet with 3G telephony

Linux machines with no rebooting�? Is this what we want?

SECURITY: Best Practices with sudo on Linux

ArtistX 0.9 Screenshots

Three locks for your SSH door

KDE 4.5: KDE 3.5's Stability + KDE 4.0's Promised Features

Qt and Layouts

Who Contributed The Most During X Server 1.9?

Microsoft Argues That Android Isn't 'Free'

Implementation Engineer Sr – UNIX (TX)
Next Step Systems
US-TX-Houston

Post A Job | Post A Resume

:SPECweb99 Benchmark clarification

SPECweb99 Benchmark clarification
Jul 5, 2000, 23 :23 UTC (38 Talkback[s]) (29346 reads)
(Other stories by Marty Pitts)

By Marty Pitts, Linux Today

There has been a lot of interest in the results of the SPECweb99 benchmark tests that were preformed on two similar Dell PowerEdge 6400/700 servers.

At the end of the article I asked if the seemingly minor differences in the hardware could account for the large difference in the results. In addition to the talkbacks posted, I received several emails with new information about the tests as well as questions about the hardware involved.

One individual asked about the difference in the hard drive controllers. This person noted:

The difference is that the Linux machine had a controller capable of 160MB/sec, while the Win2k machine had a controller capable of 80MB/sec. I do think that could have impacted the results."

As it turns out, I was able to speak with an informed source who has knowledge of the machines involved and tests performed on them. When I relayed the concerns about the hard drive controllers, this person was able to clarify it this way:

Anyway, the disk configuration is nearly irrelevant to the W2K results. Look at the result page - the fileset size is about 5GB, and (the machines) had 8GB of RAM. Everything was cached - the only disk activity was logging, which is pretty minimal. All the PERC2 did was get the data to memory quicker on the first access."

The individual who raise the hard drive controller question also asked about the Network Controllers:

The difference here is obvious, and you did not state at what speed the Win2k boxes network connection was running at. That card is capable of running either 1Gbit/s, 100Mbit/s, or 10Mbit/s, autosensing. If plugged into the wrong port, or into a slower hub, or with substandard cable, it could have been running at as much as 1/100 the speed of the card in the linux machine."

The person close the the tests responded:

"Actually his assumption is wrong. The client and network switch setup was identical for both results. (The testers) were just trying to clarify that the fiber NIC was being used. ACEnic PCI (PCI bus) could be either fiber or copper now - in the past there was only fiber, so ACEnic PCI was enough." One person wrote in with concerns that the Linux machine had an advantage since it was using an in-kernel HTTP cache. This would give the Linux box a definite advantage when serving static pages. While Windows does have a similar technology, doubts were raised about it being used for the tests.

The person close to the test responded:

SNCA, FRCA, SWC and NWSA were used on SPECweb96, which was 100% static content, but none have been used on SPECweb99 with 70% static and 30% dynamic content. Certainly if they could be used, they would be - competition for top SPECweb numbers is intense.

TUX includes an in-kernel HTTP cache but is also a full- featured http server itself. The others above are only caches. Ingo Molnar at Red Hat did the vast majority of the work on TUX and can answer specific TUX questions."

When asked about Tux 1.0 and the performance difference Ingo Monar of Red Hat responded:

Indeed, and I believe the reason is fundamental. I'd like to refer to the following comment as a background on TUX's design:

http://slashdot.org/comments.pl?sid=00/07/05/0211257&cid=218

This comment should explains some fundamental properties of TUX.

SNCA, FRCA, SWC and NWSA are standalone static-only web-reply caches.

TUX is a very different thing, it's a:

event-based HTTP protocol stack providing encapsulated HTTP services to user-space code
object cache, where objects can be combined with dynamic content freely. You can witness this in the SPECweb99 submission, the SPECweb99 TUX dynamic module (every vendor has to write a SPECweb99 module to serve dynamic content) 'embedds' TUX objects in dynamic replies. The TUX object cache is not a dumb in-memory HTTP-reply cache, it's a complex object cache providing async disk IO capabilities to user-space as well. User-space application code can request objects from the cache and can use them in dynamic (or static) replies.
full fledged webserver providing HTTP 1.1 keepalive, CGI, logging, transparent redirection, and many other features.

So in our opinion TUX is a new and unique class of webserver, there is no prior art implementing such kind of 'HTTP stack' and 'abstract object cache' approach. It's i believe a completely new approach to webserving. Please read this comment too, which (i hope) further explains the in-kernel issue:

http://slashdot.org/comments.pl?sid=00/07/05/0211257&cid=434

Also, you might want to take a look at the TUX SPECweb99 module source code to see how the TUX HTTP protocol stack works in practice:

http://www.spec.org/osg/web99/results/api-src/Dell-20000626.tar.gz

I'd like to point out that I have no affiliation with SPEC, and we (Red Hat) picked the SPECweb99 suite because of it's sophisticated and realistic workload and the independence SPEC guarantees. The SPECweb99 suite is designed to create a complex workload, and such workload defeats in-kernel static webcaches such as SWC. The workload is 'mixed up', ie. the same web objects are requested in static and dynamic requests, and it's forbidden to cache dynamic replies. CGI requests are part of the workload as well.

I have specifically designed TUX to be integrated into Apache. While the GPL (TUX is under the GPL) and the Apache Software License are incompatible, I do plan to allow licensing the TUX 'user-space glue code' under the ASL as well - further easing the integration of TUX capabilities into Apache 2.0 or 3.0."

Please feel free to post further comments and questions on these tests below.