Harping on Metadata Performance: New Benchmarks

“If you have been reading the articles in this column for any
amount of time you will see a number of them focus on metadata
performance. The primary reason for focusing on metadata
performance is that it is one of the most critical aspects of
overall performance and is one of the most neglected aspects. When
people talk about drive or storage performance it is invariably
given in terms of MB per second or something similar. In other
words – throughput. However, people do not realize how important
metadata performance is to overall performance.

“Metadata performance refers to the how quickly files and
directories can be created, removed, their status checked (stat),
as well as other data functions. This aspect of storage performance
is becoming more important because of the increasing number of
files and directories on systems. Creating files, deleting them,
and performing a status check of them is important for more
applications than ever before. There are applications that can
produce millions of files in a single directory and applications
that create very deep and wide directory structures. As the number
of cores increases the number of files and number of directories
increases putting more and more pressure on the metadata
performance of storage solutions.

“In previous articles a simple metadata benchmark, fdtree, was
used to measure benchmark performance. We saw that many times there
was not a lot of change in performance when the file system
metadata performance was changed. This led me to ask the question,
is fdtree a good enough benchmark to show metadata performance as
one tweaks a system for performance? If you like analogies,
it’s like wondering if you doctor has the right diagnosis.
Since I don’t always trust single opinions, I always like to
get second and even third opinions. So in this article I want to
examine a new metadata performance benchmark, run it on the same
hardware as past articles that used fdtree, and see if they both
show the same trends.”

Complete
Story