PostgreSQL performance on ReiserFS


2015/09/28 by Tomas Vondra

In the last two post, I've briefly discussed results of a benchmark measuring PostgreSQL performance with different filesystems. The first post compared EXT4 and XFS, and the second one was mostly dealing with the problems I've ran into when benchmarking BTRFS.

In the discussion, I've been asked to benchmark ReiserFS. I haven't really planned to benchmark this file system because ... Let's say ReiserFS does not have the best reputation when it comes to reliability due to historical reasons. According to some sources the motto of the development was essentially "performance over correctness" which is approach database folks don't really like. I don't know how much that's true for the current state of ReiserFS, or how that applies to the new ReiserFS 4 (which is not merged and I don't think it'll ever happen).

But database people are rather conservative when it comes to storage reliability, so even if all the issues got fixed and the current version is reliable, it's unlikely to become very popular in this group soon. Factor in the fact that the development of ReiserFS pretty much stopped (in favor of the Reiser4, which did not get merged for years), so the code pretty much in maintenance mode only. Which is not necessarily bad, but it also means no significant reworks necessary for new features and performance improvements (which was the purpose of Reiser4).

So let's see if ReiserFS 3 actually performs so much better than the common file systems (EXT4 and XFS), as the users often believe ...


The short answer is "no". At least not on the workload measured by pgbench (a lot of random I/O) and on this particular hardware (good SSD), ReiserFS 3 actually performs worse than both XFS and EXT4.

read-only

On read-only benchmarks, all the filesystems perform pretty much exactly the same, irrespectedly of the amount of data. For example on the large data set, you'll get something like this:

reiser3-compare-large-rw.png

The chart shows throughput (transactions per second) with different number of clients, for those three filesystems. And there's pretty much no difference at all.

The same is true for all read-only benchmarks and pretty much all traditional file systems, actually - if your workload is read-mostly, the choice of file system is going to do very little difference. The limit is going to be either at the storage or CPU level, and all the file systems perform pretty much exactly the same. (CoW-based filesystems are an obvious exception, of course.)

read-write

Read-write benchmarks are a different story - that's where the filesystems start to differ. Let's see read-write bechmarks on the large data set:

reiser3-compare-large-rw.png

EXT4 seems to be slightly faster than XFS (the difference is rather small until 8 clients, so it seems like a scalability issue). But ReiserFS is losing very visibly even on low client counts, losing ~25% on average. That's a lot, especially for a file system that was designed with performance as a primary goal.

How does it look on a smaller data set, that fits into RAM?

reiser3-compare-medium-rw.png

The difference between EXT4 and XFS pretty much disappeared and ReiserFS is catching up, but the difference is still pretty obvious. I'm not goint to show results for the small data set - in that case all the file systems perform exactly the same.

In the previous posts I've repeatedly nagged about jitter - getting high throughput is nice, but it's much less useful if the performance fluctuates a lot. This is the usual "throughput vs. latency" tradeoff where you exchange a bit of throughput for lower latency, but how much througput you need to sacrifice depends on how well the system is designed, implemented and optimized. EXT4 and XFS handle this quite well - EXT4 has slightly better throughput, XFS has a bit lower jitter. BTRFS is not mature yet and both throughput and jitter are rather awful. So what about ReiserFS?

reiser3-large-rw-16-clients.png

Not bad, I guess. Compare this to XFS (lower jitter than EXT4):

xfs-16.png

This is the one thing where ReiserFS 3 performs better - the jitter is slighly lower, and also the dips (that happen due to full page writes after CHECKPOINT) seem to be gone. That's nice, but you could also ask whether this is worth the 25% throughput drop.

Configuration

The benchmarks were done on a system with Intel i5-2500 CPU, 8GB of RAM and Intel S3700 100GB SSD, connected using SATA-III (I've previously incorrectly stated it's a S3500 model, i.e. a "lower" model, sorry for that - I do have S3500 in another system and got a bit confused).

The system is running a 4.0.4 kernel and the file systems were created/configured like this:

mkfs.ext4 -E stride=128,stripe-width=128 /dev/sda4
mkfs.xfs -f -d agcount=32,su=512k,sw=1 -l su=256k /dev/sda4
mkfs.reiserfs -f /dev/sda4

with these mount options:

/dev/sda4 on /mnt/data type ext4     (rw,noatime,nobarrier,discard)
/dev/sda4 on /mnt/data type xfs      (rw,noatime,nobarrier,discard)
/dev/sda4 on /mnt/data type reiserfs (rw,noatime,barrier=none)

It was recommended to also use "notail" mount option for ReiserFS, I don't think that makes any difference as that only applies to small files that can fit into the tree directly. But the files used for PostgreSQL are rather large and always multiples of 8kB, so not using this option should make no difference.

Summary

I'm a bit disappointed by the ReiserFS results - it's not terrible, but I've expected a bit more from a file system so much praised by the users for it's performance. I'm wondering where does this belief that it beats EXT4 or XFS comes from.

I'd guess it's at least partially due to obsolete experience - it's quite possible ReiserFS really was much faster than the other filesystems back in 2001 when it was introduced, but while EXT4 and XFS made a solid progress since then, the development of ReiserFS v3 mostly ceased (due to Namesys vanishing in 2007 and concentrating on Reiser4 which is not ready 8 years later).

Of course, ReiserFS 3 was designed for the rotational storage common in 2001, so the design might have done compromises that were irrelevant on spinning rust but are hurtful on flash storage (which is what the benchmark is using). But that only confirms the previous argument of stagnating development.

For me, the results only confirm that I'm not interested in using ReiserFS with PostgreSQL. Even if the data corruption issues were just a FUD or fixed long time ago, I don't see what I'd gain from using it instead of EXT4 or XFS.





comments powered by Disqus