[ The opinions expressed by authors on Linux Today are their
own. They speak only for themselves and not for Linux Today.
]

By Eric Lee
Green

Welcome to Benchmarketing 101, boys and girls! Today we learn a
new word: Benchmarketing. Can you say benchmarketing? Good! Now we
learn how to do it!

What is benchmarketing? Benchmarketing is a simple application
of the principles of FUD, specifically, FUD principle #1,
“exaggeration” (see the FUD 101 HOWTO
for a detailed explanation). What you have to do is find some area,
no matter how trivial, where your product is better than the
opponent’s. It doesn’t matter that other products are better than
yours in that area. It doesn’t even matter whether the benchmark is
related to how the product is used. All that matters is bashing the
opponent.

Here’s an example of benchmarketing:

Herring represents a better value proposition than Windows NT.
It has a lower initial cost, a lower total cost of operation, and
has a 500% speed advantage over Windows NT. In addition, it has all
those nifty features that consumers want, like taste, nutrition,
and a pleasing texture, while Windows NT has none of those
features. In conclusion, please buy herring for all of your
computing needs.

Part Deaux: HUH?!

By now, boys and girls, you have probably figured out that you were
duped. Let’s detail exactly how you were duped, and how you can
tell you’re being benchmarketed tomorrow.

1. Statistics unrelated to use: The statistics
   gathered above are totally unrelated to the use of either Windows
   NT or herring as a computing platform.
2. Ignoring superior competitors: A shark is a
   faster swimmer than a herring, but we conveniently left it out of
   our chart.
3. Irrelevant features: The features gathered are
   totally irrelevant to the use of herring or Windows NT as a
   computing platform.
4. Leaving out statistics: We deliberately left
   out any statistics that might make herring look bad as a computing
   platform. For example, the herring’s brain is less powerful at
   number-crunching than any known microprocessor, thus making herring
   a poor choice for mathematical programming. We don’t mention that,
   just as Microsoft never mentions that NT 4.0 is ill-suited for use
   as a corporate firewall when compared to FreeBSD or Linux.
5. Conclusions not following from the facts: The
   conclusion reached (that herring was preferable to Windows NT as a
   computing platform) was not supported by any of the facts
   mentioned. What we are trying to do here is snow you with facts,
   and then slip in a conclusion and hope (FUD distraction method #1)
   that the fact that we’ve sandwiched the false statement between
   true facts will distract you enough so that you don’t spot that it
   is false.

Part Tres: Conclusions

Whenever you see benchmarks, you should be careful to examine
whether the benchmarks involved are applicable to the problem set
that you’re trying to solve. If they are not, then the benchmark is
irrelevant.

Secondly, you should examine how meaningful the benchmark is. If
the benchmark says that solution (a) does ‘x’ operations per
second, solution (b) does ‘x+1000’ operations per second, but only
need to do ‘x/10’ operations per second, then the fact that one
solution can do more operations per second than the other is
meaningless. Both will fill your needs, and you should choose which
one is most cost-effective for your situation. Why buy a Mercedes
when all you need is a Honda Civic?

For example, if you’re trying to set up a low-cost web server
for your small business to feed a T1 line, a $499 eMachine 333
running Linux will saturate that T1 just as assuredly as a $35,000
quad-processor Dell with four network cards and Windows NT. This is
a case where a benchmark (the Dell is MUCH faster than the
eMachine) is irrelevant.

Finally: Be aware. Benchmarketing is nothing new. IBM was a past
master of benchmarketing back in the 70’s, for example (they
invented ‘TPS’, Transactions Per Second, to quantify how their
mainframes were superior to the competition). Whenever you see
benchmarks in the press, there is a 90% certainty that you are
seeing benchmarketing in action. Be careful out there.

Eric Lee Green
is the networking and systems guru for Enhanced Software
Technologies Inc., “The Bru Guys”. He deeply regrets the use of a
hotmail.com EMAIL address but hastily explains that he’s had it
since before Hotmail was sold to Microsoft, and further excuses it
by saying that he’d prefer SPAM to burden Microsoft rather than his
employer.