Btrfs Linux 4.10 RAID

Btrfs multi-disk tests on the Linux 4.10 kernel with various RAID options. Tests by Michael Larabel for a future article on Phoronix.com.

HTML result view exported from: https://openbenchmarking.org/result/1701298-PTS-BTRFSLIN55&sor.

Btrfs Linux 4.10 RAIDProcessorMotherboardChipsetMemoryDiskGraphicsAudioMonitorNetworkOSKernelDesktopDisplay ServerDisplay DriverOpenGLVulkanCompilerFile-SystemScreen ResolutionSingle Disk2-Disk RAID02-Disk RAID14-Disk RAID04-Disk RAID14-Disk RAID104-Disk RAID54-Disk RAID6Intel Core i7-6800K @ 3.80GHz (12 Cores)MSI X99A WORKSTATION (MS-7A54) v1.0Intel Xeon E7 v4/Xeon16384MB120GB Samsung SSD 850 + 4 x 120GB TOSHIBA-TR150NVIDIA GeForce GTX TITAN X 12288MBRealtek ALC1150ASUS PB278Intel ConnectionUbuntu 16.044.10.0-999-generic (x86_64) 20170126Unity 7.4.0X Server 1.18.4nouveau 1.0.123.3 Mesa 11.2.0 Gallium 0.41.0.8GCC 5.4.0 20160609 + CUDA 8.0btrfs2560x1440OpenBenchmarking.orgCompiler Details- --build=x86_64-linux-gnu --disable-browser-plugin --disable-vtable-verify --disable-werror --enable-checking=release --enable-clocale=gnu --enable-gnu-unique-object --enable-gtk-cairo --enable-java-awt=gtk --enable-java-home --enable-languages=c,ada,c++,java,go,d,fortran,objc,obj-c++ --enable-libmpx --enable-libstdcxx-debug --enable-libstdcxx-time=yes --enable-multiarch --enable-multilib --enable-nls --enable-objc-gc --enable-plugin --enable-shared --enable-threads=posix --host=x86_64-linux-gnu --target=x86_64-linux-gnu --with-abi=m64 --with-arch-32=i686 --with-arch-directory=amd64 --with-default-libstdcxx-abi=new --with-multilib-list=m32,m64,mx32 --with-tune=generic -v Disk Details- Single Disk: CFQ / relatime,rw,space_cache,ssd,subvol=/,subvolid=5- 2-Disk RAID0: CFQ / relatime,rw,space_cache,ssd,subvol=/,subvolid=5 / RAID0- 2-Disk RAID1: CFQ / relatime,rw,space_cache,ssd,subvol=/,subvolid=5 / RAID1- 4-Disk RAID0: CFQ / relatime,rw,space_cache,ssd,subvol=/,subvolid=5 / RAID0- 4-Disk RAID1: CFQ / relatime,rw,space_cache,ssd,subvol=/,subvolid=5 / RAID1- 4-Disk RAID10: CFQ / relatime,rw,space_cache,ssd,subvol=/,subvolid=5 / RAID10- 4-Disk RAID5: CFQ / relatime,rw,space_cache,ssd,subvol=/,subvolid=5 / RAID5- 4-Disk RAID6: CFQ / relatime,rw,space_cache,ssd,subvol=/,subvolid=5 / RAID6Processor Details- Scaling Governor: intel_pstate powersaveSystem Details- Python 2.7.12.

Btrfs Linux 4.10 RAIDsqlite: Default Test Directoryfio: Rand Read - Libaio - No - Yes - 4KB - Default Test Directory - MB/sfio: Rand Write - Libaio - No - Yes - 4KB - Default Test Directory - MB/sfio: Seq Read - Libaio - No - Yes - 4KB - Default Test Directory - MB/sfio: Seq Write - Libaio - No - Yes - 4KB - Default Test Directory - MB/sfs-mark: 1000 Files, 1MB Sizefs-mark: 5000 Files, 1MB Size, 4 Threadsfs-mark: 4000 Files, 32 Sub Dirs, 1MB Sizeblogbench: Writecompilebench: Compilecompilebench: Initial Createpostmark: Disk Transaction Performancepgbench: On-Disk - Normal Load - Read Writemcperf: Getmcperf: SetSingle Disk2-Disk RAID02-Disk RAID14-Disk RAID04-Disk RAID14-Disk RAID104-Disk RAID54-Disk RAID685.56210.3791.71243.9190.3661.0350.7346.634942816.97126.4763591196.72123117.2072553.7787.56439.91126.24525.49125.8878.1780.6074.556617986.31271.6864101626.47128206.7371691.73105.90213.4649.58331.3249.5059.3542.9542.054939784.53118.8263031147.83127284.5575791.3785.22720.89160.19719.56166.98121.72132.25135.7775371413.04296.6163572887.00126550.6375311.0091.33413.4695.64267.38105.2978.5761.3761.425542915.54226.876359972.52131257.1372707.6097.09424.44103.63558.34116.2763.1765.6765.636420903.57246.8963741427.30126501.2074309.63107.49585.34156.98678.77113.3382.83100.8384.2870791001.93263.9464102012.79126768.9074259.57122.07422.97119.44591.20101.5883.7066.6057.655808878.04237.0664131303.60130668.8377837.00OpenBenchmarking.org

SQLite

Test Target: Default Test Directory

OpenBenchmarking.orgSeconds, Fewer Is BetterSQLite 3.8.10.2Test Target: Default Test Directory4-Disk RAID0Single Disk2-Disk RAID04-Disk RAID14-Disk RAID102-Disk RAID14-Disk RAID54-Disk RAID6306090120150SE +/- 1.37, N = 6SE +/- 1.23, N = 3SE +/- 1.66, N = 3SE +/- 2.29, N = 6SE +/- 2.82, N = 6SE +/- 0.24, N = 3SE +/- 1.92, N = 6SE +/- 2.42, N = 685.2285.5687.5691.3397.09105.90107.49122.071. (CC) gcc options: -O2 -lcurses -ldl -lpthread

Flexible IO Tester

Type: Random Read - IO Engine: Libaio - Buffered: No - Direct: Yes - Block Size: 4KB - Disk Target: Default Test Directory - Result: MB/s

OpenBenchmarking.orgMB/s, More Is BetterFlexible IO Tester 2.1.13Type: Random Read - IO Engine: Libaio - Buffered: No - Direct: Yes - Block Size: 4KB - Disk Target: Default Test Directory - Result: MB/s4-Disk RAID04-Disk RAID52-Disk RAID04-Disk RAID104-Disk RAID64-Disk RAID12-Disk RAID1Single Disk160320480640800SE +/- 1.34, N = 3SE +/- 10.32, N = 6SE +/- 1.57, N = 3SE +/- 2.93, N = 3SE +/- 0.15, N = 3SE +/- 1.02, N = 3SE +/- 4.00, N = 3SE +/- 3.38, N = 3720.89585.34439.91424.44422.97413.46213.46210.371. (CC) gcc options: -rdynamic -std=gnu99 -O3 -ffast-math -include -ll -lnuma -libverbs -lrt -laio -lz -lm -lpthread -ldl

Flexible IO Tester

Type: Random Write - IO Engine: Libaio - Buffered: No - Direct: Yes - Block Size: 4KB - Disk Target: Default Test Directory - Result: MB/s

OpenBenchmarking.orgMB/s, More Is BetterFlexible IO Tester 2.1.13Type: Random Write - IO Engine: Libaio - Buffered: No - Direct: Yes - Block Size: 4KB - Disk Target: Default Test Directory - Result: MB/s4-Disk RAID04-Disk RAID52-Disk RAID04-Disk RAID64-Disk RAID104-Disk RAID1Single Disk2-Disk RAID14080120160200SE +/- 9.64, N = 6SE +/- 2.32, N = 3SE +/- 17.35, N = 6SE +/- 12.68, N = 6SE +/- 18.82, N = 6SE +/- 11.89, N = 6SE +/- 1.37, N = 3SE +/- 12.34, N = 6160.19156.98126.24119.44103.6395.6491.7149.581. (CC) gcc options: -rdynamic -std=gnu99 -O3 -ffast-math -include -ll -lnuma -libverbs -lrt -laio -lz -lm -lpthread -ldl

Flexible IO Tester

Type: Sequential Read - IO Engine: Libaio - Buffered: No - Direct: Yes - Block Size: 4KB - Disk Target: Default Test Directory - Result: MB/s

OpenBenchmarking.orgMB/s, More Is BetterFlexible IO Tester 2.1.13Type: Sequential Read - IO Engine: Libaio - Buffered: No - Direct: Yes - Block Size: 4KB - Disk Target: Default Test Directory - Result: MB/s4-Disk RAID04-Disk RAID54-Disk RAID64-Disk RAID102-Disk RAID02-Disk RAID14-Disk RAID1Single Disk160320480640800SE +/- 12.16, N = 3SE +/- 3.19, N = 3SE +/- 28.72, N = 6SE +/- 57.02, N = 6SE +/- 2.48, N = 3SE +/- 5.25, N = 3SE +/- 27.92, N = 6SE +/- 4.26, N = 4719.56678.77591.20558.34525.49331.32267.38243.911. (CC) gcc options: -rdynamic -std=gnu99 -O3 -ffast-math -include -ll -lnuma -libverbs -lrt -laio -lz -lm -lpthread -ldl

Flexible IO Tester

Type: Sequential Write - IO Engine: Libaio - Buffered: No - Direct: Yes - Block Size: 4KB - Disk Target: Default Test Directory - Result: MB/s

OpenBenchmarking.orgMB/s, More Is BetterFlexible IO Tester 2.1.13Type: Sequential Write - IO Engine: Libaio - Buffered: No - Direct: Yes - Block Size: 4KB - Disk Target: Default Test Directory - Result: MB/s4-Disk RAID02-Disk RAID04-Disk RAID104-Disk RAID54-Disk RAID14-Disk RAID6Single Disk2-Disk RAID14080120160200SE +/- 2.96, N = 6SE +/- 9.93, N = 6SE +/- 15.32, N = 6SE +/- 14.60, N = 6SE +/- 17.02, N = 6SE +/- 10.38, N = 6SE +/- 1.49, N = 3SE +/- 10.95, N = 6166.98125.88116.27113.33105.29101.5890.3649.501. (CC) gcc options: -rdynamic -std=gnu99 -O3 -ffast-math -include -ll -lnuma -libverbs -lrt -laio -lz -lm -lpthread -ldl

FS-Mark

Test: 1000 Files, 1MB Size

OpenBenchmarking.orgFiles/s, More Is BetterFS-Mark 3.3Test: 1000 Files, 1MB Size4-Disk RAID04-Disk RAID64-Disk RAID54-Disk RAID12-Disk RAID04-Disk RAID10Single Disk2-Disk RAID1306090120150SE +/- 21.94, N = 6SE +/- 15.67, N = 6SE +/- 10.74, N = 6SE +/- 16.14, N = 6SE +/- 3.66, N = 6SE +/- 9.26, N = 6SE +/- 11.55, N = 6SE +/- 16.45, N = 6121.7283.7082.8378.5778.1763.1761.0359.351. (CC) gcc options: -static

FS-Mark

Test: 5000 Files, 1MB Size, 4 Threads

OpenBenchmarking.orgFiles/s, More Is BetterFS-Mark 3.3Test: 5000 Files, 1MB Size, 4 Threads4-Disk RAID04-Disk RAID52-Disk RAID04-Disk RAID64-Disk RAID104-Disk RAID1Single Disk2-Disk RAID1306090120150SE +/- 7.24, N = 6SE +/- 2.44, N = 6SE +/- 1.78, N = 6SE +/- 1.30, N = 3SE +/- 1.23, N = 6SE +/- 1.36, N = 6SE +/- 0.32, N = 3SE +/- 0.83, N = 6132.25100.8380.6066.6065.6761.3750.7342.951. (CC) gcc options: -static

FS-Mark

Test: 4000 Files, 32 Sub Dirs, 1MB Size

OpenBenchmarking.orgFiles/s, More Is BetterFS-Mark 3.3Test: 4000 Files, 32 Sub Dirs, 1MB Size4-Disk RAID04-Disk RAID52-Disk RAID04-Disk RAID104-Disk RAID14-Disk RAID6Single Disk2-Disk RAID1306090120150SE +/- 19.35, N = 6SE +/- 3.17, N = 6SE +/- 7.08, N = 6SE +/- 5.47, N = 6SE +/- 5.71, N = 6SE +/- 4.24, N = 6SE +/- 0.66, N = 6SE +/- 2.07, N = 6135.7784.2874.5565.6361.4257.6546.6342.051. (CC) gcc options: -static

BlogBench

Test: Write

OpenBenchmarking.orgFinal Score, More Is BetterBlogBench 1.0Test: Write4-Disk RAID04-Disk RAID52-Disk RAID04-Disk RAID104-Disk RAID64-Disk RAID1Single Disk2-Disk RAID116003200480064008000SE +/- 92.04, N = 3SE +/- 88.83, N = 3SE +/- 39.06, N = 3SE +/- 106.36, N = 3SE +/- 89.05, N = 6SE +/- 50.28, N = 3SE +/- 38.37, N = 3SE +/- 20.76, N = 3753770796617642058085542494249391. (CC) gcc options: -O2 -pthread

Compile Bench

Test: Compile

OpenBenchmarking.orgMB/s, More Is BetterCompile Bench 0.6Test: Compile4-Disk RAID04-Disk RAID52-Disk RAID04-Disk RAID14-Disk RAID104-Disk RAID6Single Disk2-Disk RAID130060090012001500SE +/- 41.11, N = 6SE +/- 33.99, N = 6SE +/- 24.76, N = 6SE +/- 13.10, N = 6SE +/- 21.92, N = 6SE +/- 4.10, N = 3SE +/- 1.60, N = 3SE +/- 13.08, N = 41413.041001.93986.31915.54903.57878.04816.97784.53

Compile Bench

Test: Initial Create

OpenBenchmarking.orgMB/s, More Is BetterCompile Bench 0.6Test: Initial Create4-Disk RAID02-Disk RAID04-Disk RAID54-Disk RAID104-Disk RAID64-Disk RAID1Single Disk2-Disk RAID160120180240300SE +/- 4.29, N = 3SE +/- 7.91, N = 3SE +/- 9.00, N = 3SE +/- 18.71, N = 3SE +/- 18.38, N = 3SE +/- 22.14, N = 3SE +/- 30.50, N = 3SE +/- 8.51, N = 3296.61271.68263.94246.89237.06226.87126.47118.82

PostMark

Disk Transaction Performance

OpenBenchmarking.orgTPS, More Is BetterPostMark 1.51Disk Transaction Performance4-Disk RAID64-Disk RAID52-Disk RAID04-Disk RAID104-Disk RAID1Single Disk4-Disk RAID02-Disk RAID114002800420056007000SE +/- 94.69, N = 3SE +/- 0.00, N = 3SE +/- 0.00, N = 3SE +/- 100.39, N = 4SE +/- 109.33, N = 3SE +/- 109.33, N = 3SE +/- 53.33, N = 3SE +/- 53.33, N = 3641364106410637463596359635763031. (CC) gcc options: -O3

PostgreSQL pgbench

Scaling: On-Disk - Test: Normal Load - Mode: Read Write

OpenBenchmarking.orgTPS, More Is BetterPostgreSQL pgbench 9.4.3Scaling: On-Disk - Test: Normal Load - Mode: Read Write4-Disk RAID04-Disk RAID52-Disk RAID04-Disk RAID104-Disk RAID6Single Disk2-Disk RAID14-Disk RAID16001200180024003000SE +/- 94.97, N = 6SE +/- 52.22, N = 62887.002012.791626.471427.301303.601196.721147.83972.521. (CC) gcc options: -fno-strict-aliasing -fwrapv -O2 -pthread -lpgcommon -lpgport -lpq -lpthread -lrt -lcrypt -ldl -lm

Memcached 1.3.34 mcperf

Method: Get

OpenBenchmarking.orgOperations Per Second, More Is BetterMemcached 1.3.34 mcperf 0.1.1Method: Get4-Disk RAID14-Disk RAID62-Disk RAID02-Disk RAID14-Disk RAID54-Disk RAID04-Disk RAID10Single Disk30K60K90K120K150KSE +/- 303.17, N = 3SE +/- 1578.76, N = 3SE +/- 620.72, N = 3SE +/- 2179.14, N = 6SE +/- 310.35, N = 3SE +/- 441.12, N = 3SE +/- 376.26, N = 3SE +/- 983.17, N = 3131257.13130668.83128206.73127284.55126768.90126550.63126501.20123117.201. (CC) gcc options: -O2 -lm -rdynamic

Memcached 1.3.34 mcperf

Method: Set

OpenBenchmarking.orgOperations Per Second, More Is BetterMemcached 1.3.34 mcperf 0.1.1Method: Set4-Disk RAID62-Disk RAID14-Disk RAID04-Disk RAID104-Disk RAID54-Disk RAID1Single Disk2-Disk RAID020K40K60K80K100KSE +/- 915.80, N = 3SE +/- 369.29, N = 3SE +/- 79.41, N = 3SE +/- 392.11, N = 3SE +/- 602.01, N = 3SE +/- 773.93, N = 3SE +/- 276.18, N = 3SE +/- 426.86, N = 377837.0075791.3775311.0074309.6374259.5772707.6072553.7771691.731. (CC) gcc options: -O2 -lm -rdynamic

Phoronix Test Suite v10.8.4