Microsoft issued a statement on May 12 requesting the Linux Community
to respond to Mindcraft's Open Benchmark challenge. In an apparent attempt
at "baiting" the Community, Microsoft digs deep into the FUD barrel -- a
surefire way to win the confidence and cooperation of the Community.
Concentrating on the Mindcraft study rather than the FUD, there is
a great deal to be learned from the PC
Week benchmark cited in the Microsoft statement. It appears that the
Linux Community's concerns about the initial Mindcraft test have largely
been validated, namely, that the results of the initial Mindcraft test did
not accurately represent the performance capabilities of Linux.
First, lets look at the WebBench numbers. PC Week attained up to 2,100
requests/sec with Apache on Linux. An amazing 110% improvement over
Mindcraft's 1,000 requests/sec result.
Secondly, the NetBench numbers tell a similar story. Linux with Samba
achieved up to 197Mbps, a significant 73% improvement over Mindcraft's
figure of 114Mbps. What's more, drilling deeper into the PC Week article,
we find that the NT server achieved a meager 150Mbps of throughput when
tested against WinNT clients. In others words, the Samba/Linux combo
outperformed Windows NT server by a very healthy 31% when tested against
Microsoft's self-proclaimed business class desktop product. PC Week goes
on to state, "More importantly, Samba had minimal performance degradation
at higher client loads. In tests with 60 clients, Windows NT managed only
110M-bps throughput compared with 183M-bps for Samba." Strangely, the
Microsoft statement makes no mention of these results.
So what exactly is Microsoft complaining about? The PC Week results
make a strong case that Mindcraft did not properly tune their Linux
configuration. The Open Benchmark would likely yield similar results.
Microsoft's demand that the Linux Community "withdraw their criticisms of
the initial Mindcraft report", if they do participate in the Open
Benchmark, rings hollow. Moreover, Mindcraft states that the purpose of
the Open Benchmark is "to see if Mindcraft's second benchmark results
are biased and not representative of Linux's true performance". But what
of the initial Mindcraft report? Apparently, not even Mindcraft has
confidence in the numbers from the initial report.
