E-mail servers, data base servers, file servers, web servers struggle to cope with LATENCY on accessing millions of small files under heavy concurrent load.
Low latency is key factor to a good user experience.
With the Debian GNU / Linux 6.x kernel tuning, combined with the previous article hints about XenServer I/O latency, tuning filesystems configuring multipath, and data storage, we reduced eighty-four times our virtual block device latency at our environment.
Thousands concurrent synchronous writes of random small files on filesystems with millions of them while honoring write barriers for POSIX compliance, stress the limits of the data storage, file systems, kernel I/O, virtualization I/O, hardware data paths.
These are our findings and Linux kernel tunings in Cyrus IMAP e-mail servers grid under heavy load production since November 2011.
http://www.techforce.com.br/news/linux_blog/debian_linux_kernel_tuning_low_latency_small_files