[ Thanks to Amy Newman for this link.
]
“Weekly and sometimes daily benchmarks claiming that a given
vendor has surpassed other vendors on XYZ I/O benchmark provides
little useful information. Most benchmarks tell us:“1. How much hardware the vendor can fit (and I really mean
stuff) into the box it is trying to benchmark“2. How good the benchmark team is at optimizing the software
(OS, file system, network stack, firmware for all hardware) and the
hardware“3. How much time and money the company will want to spend
running case after case to achieve the results it wants“You might ask how I know this, and the answer is that 1996, I
was part of a team that benchmarked some hardware, OS and file
systems to achieve 1 GiB/sec. At the time that was huge. Back then,
we got 60 percent of the performance needed with 50 percent of the
hardware, so we were not scaling very well.“Fast forward to today, and nothing has changed.”