HPC Benchmarking: A Quick Primer

[ Thanks to Douglas Eadline for this
link. ]

“Mention HPC benchmarking and the first thing that
comes to mind is the Top500 list. The actual benchmark is called
HPL, which stands for High Performance Linpack. The Linpack
benchmark was designed to measure the floating point performance
for solving a system of linear equations. If you don’t know what
that means, not to worry, it is simply one way to measure how much
floating point performance a computer can achieve. There are other
worthwhile benchmarks, but by virtue of a bi-annual Top500 list,
HPL has become the most famous. As benchmarks go, HPL has many
tunable options and thus can take a considerable amount of time to
optimize. Also, consider that HPL can take hours or days to run, so
getting a good “HPL number” can take a long time.

“For the purposes of this article, we are going to assume that a
benchmark is a reproducible rate of performance. For instance, in
HPC the FLOPS metric is often used where FLOPS stands for Floating
Point Operations per Second. Clusters are very good at delivering
FLOPS, so good that the prefix of Tera (or T) or Giga (or G) are
used. One TFLOP is 1x10E12 floating point operations per second,
while one GFLOP is 1,000 times less or 1x10E9. The world’s fastest
machines are now measured in PFLOPS, where P is Peta or
1x10E15.

“FLOPS is not the only measure of performance for a cluster. In
some cases, one may want to measure integer, or I/O, performance.
The Standard Performance Evaluation Corporation (SPEC) is perhaps
the best known independent set of benchmarks. SPEC has many
benchmarks, including a specific set for MPI and OpenMP. The SPEC
benchmarks are actually a suite of benchmarks from which a
composite rating is computed. The SPEC benchmarks must be purchased
and are mainly used by vendors to report/rank performance of new
computer systems.”

Complete Story