I'm playing with btrfs to get a feel for what's coming up in linux filesystems. To be daring, i've configured a test machine using only btrfs for its on-disk filesystems. I really like some of the ideas put forward in the btrfs design. (i'm aware that btrfs is considered experimental-only at this point).
I'm happy to report that despite several weeks of regular upgrade/churn from unstable and experimental, i have yet to see any data loss or other serious forms of failure.
Unfortunately, i'm not impressed with the performance. The machine feels sluggish in this configuratiyon, compared to how i remember it running with previous non-btrfs installations. So i ran some benchmarks. The results don't look good for btrfs in its present incarnation.
UPDATE: see the comments section for revised statistics from a quieter system, with the filesystems over the same partition (btrfs is still much slower).
The simplified test system i'm running has Linux kernel
2.6.39-rc6-686-pae (from experimental), 1GiB of RAM (no swap), and a
single 2GHz P4 CPU. It has one parallel ATA hard disk
(WDC WD400EB-00CPF0), with two primary partitions (one btrfs and one
ext3). The root filesystem is btrfs. The ext3 filesystem is mounted at
/mnt
I used bonnie++ to benchmark the ext3 filesystem against the btrfs filesystem as a non-privileged user.
Here are the results on the test ext3 filesystem:
consoleuser@loki:~$ cat bonnie-stats.ext3 Reading a byte at a time...doneReading intelligently...donestart 'em...done...done...done...done...done...Create files in sequential order...done.Stat files in sequential order...done.Delete files in sequential order...done.Create files in random order...done.Stat files in random order...done.Delete files in random order...done.Version 1.96 ------Sequential Output------ --Sequential Input- --Random-Concurrency 1 -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec %CPloki 2264M 331 98 23464 11 10988 4 1174 85 39629 6 130.4 5Latency 92041us 1128ms 1835ms 166ms 308ms 6549msVersion 1.96 ------Sequential Create------ --------Random Create--------loki -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete-- files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP 16 9964 26 +++++ +++ 13035 26 11089 27 +++++ +++ 11888 24Latency 17882us 1418us 1929us 489us 51us 650us1.96,1.96,loki,1,1305039600,2264M,,331,98,23464,11,10988,4,1174,85,39629,6,130.4,5,16,,,,,9964,26,+++++,+++,13035,26,11089,27,+++++,+++,11888,24,92041us,1128ms,1835ms,166ms,308ms,6549ms,17882us,1418us,1929us,489us,51us,650usconsoleuser@loki:~$
And here are the results for btrfs (on the main filesystem):
consoleuser@loki:~$ cat bonnie-stats.btrfs Reading a byte at a time...doneReading intelligently...donestart 'em...done...done...done...done...done...Create files in sequential order...done.Stat files in sequential order...done.Delete files in sequential order...done.Create files in random order...done.Stat files in random order...done.Delete files in random order...done.Version 1.96 ------Sequential Output------ --Sequential Input- --Random-Concurrency 1 -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec %CPloki 2264M 43 99 22682 17 10356 6 1038 79 28796 6 86.8 99Latency 293ms 727ms 1222ms 46541us 504ms 13094msVersion 1.96 ------Sequential Create------ --------Random Create--------loki -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete-- files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP 16 1623 33 +++++ +++ 2182 57 1974 27 +++++ +++ 1907 44Latency 78474us 6839us 8791us 1746us 66us 64034us1.96,1.96,loki,1,1305040411,2264M,,43,99,22682,17,10356,6,1038,79,28796,6,86.8,99,16,,,,,1623,33,+++++,+++,2182,57,1974,27,+++++,+++,1907,44,293ms,727ms,1222ms,46541us,504ms,13094ms,78474us,6839us,8791us,1746us,66us,64034usconsoleuser@loki:~$
As you can see, btrfs is significantly slower in several categories:
- writing character-at-a-time is *much* slower: 43K/sec vs. 331K/sec
- reading block-at-a-time is slower: 28796K/sec vs. 39629K/sec
- all forms of file creation and deletion are nearly an order of magnitude slower
- Random seeks are almost as fast, but they swamp the CPU
I'm hoping that i just configured the test wrong somehow, or that i've
done something grossly unfair in the system setup and configuration. (or
maybe i'm mis-reading the bonnie++ output?) Maybe someone can point
out my mistake, or give me pointers for what to do to try to speed up
btrfs.
I like the sound of the features we will eventually get from btrfs, but these performance figures seem like a pretty rough tradeoff.
Tags: benchmarks, bonnie, btrfs, ext3