Ubuntu Linux RAID 0/1/5/6/10 Disk Testing

RAID mdadm benchmarks with four Intel SSDs using RAID 0, 1, 5, 6, and 10 levels on Linux 3.18 kernel with various file-systems. Benchmarks by Michael Larabel for a future article on Phoronix.com.

HTML result view exported from: https://openbenchmarking.org/result/1411054-LI-BTRFSRAID55&grr&rdt.

ProcessorMotherboardChipsetMemoryDiskGraphicsAudioMonitorNetworkOSKernelDesktopDisplay ServerDisplay DriverOpenGLCompilerFile-SystemScreen ResolutionNative Btrfsmdadm EXT4mdadm Btrfsmdadm XFSmdadm F2FS RAID 0 RAID 1 RAID 10 RAID 5 RAID 6 RAID 0 RAID 1 RAID 10 RAID 5 RAID 6 RAID 0 RAID 1 RAID 10 RAID 5 RAID 6 RAID 0 RAID 1 RAID 10 RAID 5 RAID 6 RAID 0 RAID 1 RAID 10 RAID 5 RAID 6Intel Core i7-5960X @ 4.00GHz (16 Cores)Gigabyte X99-UD4-CFIntel Xeon E5 v3/Core16384MB4 x 120GB INTEL SSDSC2BW12 + 128GB Crucial_CT128MX1XFX AMD Radeon HD 7950/8950 OEM / R9 280 3072MBIntel C610/X99SyncMasterIntel ConnectionUbuntu 14.103.18.0-031800rc1-generic (x86_64)Unity 7.3.1X Server 1.16.0radeon 7.4.03.3 Mesa 10.3.0 Gallium 0.4GCC 4.9.1btrfs2560x1600ext4btrfsxfsf2fsOpenBenchmarking.orgCompiler Details- --build=x86_64-linux-gnu --disable-browser-plugin --disable-vtable-verify --disable-werror --enable-checking=release --enable-clocale=gnu --enable-gnu-unique-object --enable-gtk-cairo --enable-java-awt=gtk --enable-java-home --enable-languages=c,c++,java,go,d,fortran,objc,obj-c++ --enable-libstdcxx-debug --enable-libstdcxx-time=yes --enable-multiarch --enable-multilib --enable-nls --enable-objc-gc --enable-plugin --enable-shared --enable-threads=posix --host=x86_64-linux-gnu --target=x86_64-linux-gnu --with-abi=m64 --with-arch-32=i686 --with-arch-directory=amd64 --with-multilib-list=m32,m64,mx32 --with-tune=generic -v Disk Details- Native Btrfs: RAID 0: DEADLINE / relatime,rw,space_cache,ssd- Native Btrfs: RAID 1: DEADLINE / relatime,rw,space_cache,ssd- Native Btrfs: RAID 10: DEADLINE / relatime,rw,space_cache,ssd- Native Btrfs: RAID 5: DEADLINE / relatime,rw,space_cache,ssd- Native Btrfs: RAID 6: DEADLINE / relatime,rw,space_cache,ssd- mdadm EXT4: RAID 0: DEADLINE / data=ordered,relatime,rw,stripe=512- mdadm EXT4: RAID 1: DEADLINE / data=ordered,relatime,rw- mdadm EXT4: RAID 10: DEADLINE / data=ordered,relatime,rw,stripe=256- mdadm EXT4: RAID 5: DEADLINE / data=ordered,relatime,rw,stripe=384- mdadm EXT4: RAID 6: DEADLINE / data=ordered,relatime,rw,stripe=256- mdadm Btrfs: RAID 0: DEADLINE / relatime,rw,space_cache- mdadm Btrfs: RAID 1: DEADLINE / relatime,rw,space_cache- mdadm Btrfs: RAID 10: DEADLINE / relatime,rw,space_cache- mdadm Btrfs: RAID 5: DEADLINE / relatime,rw,space_cache- mdadm Btrfs: RAID 6: DEADLINE / relatime,rw,space_cache- mdadm XFS: RAID 0: DEADLINE / attr2,inode64,noquota,relatime,rw,sunit=1024,swidth=4096- mdadm XFS: RAID 1: DEADLINE / attr2,inode64,noquota,relatime,rw- mdadm XFS: RAID 10: DEADLINE / attr2,inode64,noquota,relatime,rw,sunit=1024,swidth=2048- mdadm XFS: RAID 5: DEADLINE / attr2,inode64,noquota,relatime,rw,sunit=1024,swidth=3072- mdadm XFS: RAID 6: DEADLINE / attr2,inode64,noquota,relatime,rw,sunit=1024,swidth=2048- mdadm F2FS: RAID 0: DEADLINE / acl,active_logs=6,background_gc=on,relatime,rw,user_xattr- mdadm F2FS: RAID 1: DEADLINE / acl,active_logs=6,background_gc=on,relatime,rw,user_xattr- mdadm F2FS: RAID 10: DEADLINE / acl,active_logs=6,background_gc=on,relatime,rw,user_xattr- mdadm F2FS: RAID 5: DEADLINE / acl,active_logs=6,background_gc=on,relatime,rw,user_xattr- mdadm F2FS: RAID 6: DEADLINE / acl,active_logs=6,background_gc=on,relatime,rw,user_xattrProcessor Details- Scaling Governor: intel_pstate powersaveSystem Details- Disk Scheduler: DEADLINE. Python 2.7.8.

compilebench: Initial Createiozone: 4Kb - 8GB - Write Performancefs-mark: 4000 Files, 32 Sub Dirs, 1MB Sizefs-mark: 5000 Files, 1MB Size, 4 Threadsfs-mark: 1000 Files, 1MB Sizefio: Seq Write - Libaio - No - 16KB - Default Test Directory - MB/sfio: Seq Read - Libaio - No - 16KB - Default Test Directory - MB/sfio: Rand Write - Libaio - No - 16KB - Default Test Directory - MB/sfio: Rand Read - Libaio - No - 16KB - Default Test Directory - MB/sNative Btrfsmdadm EXT4mdadm Btrfsmdadm XFSmdadm F2FS RAID 0 RAID 1 RAID 10 RAID 5 RAID 6 RAID 0 RAID 1 RAID 10 RAID 5 RAID 6 RAID 0 RAID 1 RAID 10 RAID 5 RAID 6 RAID 0 RAID 1 RAID 10 RAID 5 RAID 6 RAID 0 RAID 1 RAID 10 RAID 5 RAID 6206.741142.4340.7383.7341.43319.41824.97497.23758.26187.72584.8535.6369.0335.10130.32323.80128.00233.00177.54599.3238.5077.0738.33272.19450.78267.48469.10189.59876.7739.1779.8738.87242.55869.68248.93742.06195.02598.4337.7375.4738.20208.96960.89214.56653.54325.181111.2638.7390.4038.97508.73421.37818.33303.04302.7534.8099.5733.90115.34303.30110.32214.45306.37585.3036.8078.1337.03199.95163.86684.95316.17441.273184.0731.10192.80957.59146.07689.39320.82385.3932.0070.4332.27155.78134.18854.92178.311141.1839.4392.0039.60385.34381.45816.02160.58280.3933.4364.0034.03109.97854.98109.22224.69156.97478.6536.9375.6037.80161.90843.68159.53695.44156.21438.5532.1064.7333.37209.60879.09214.31792.61151.64315.8530.6356.8030.43172.72925.05179.73909.98264.381111.5640.7388.7342.43398.74393.49841.97228.19303.5032.7768.9034.07115.82301.16115.51236.43228.20588.4235.8381.7039.57214.94166.75384.53255.99580.9530.3358.1032.10197.19151.05751.77271.62473.1729.2353.8330.93154.10129.62928.27331.331100.4266.4086.5765.87419.29387.04872.84328.79303.3253.5780.6353.50113.35299.58109.58239.73352.47566.1461.3077.9762.87200.36177.32741.30377.77421.8652.4775.5056.37196.78149.41736.18348.76350.8451.4067.8350.73152.53997.53131.26916.30OpenBenchmarking.org

Compile Bench

Test: Initial Create

MinAvgMaxNative Btrfs177.5191.3206.7mdadm EXT4303.0314.3325.2mdadm Btrfs151.6160.7178.3mdadm XFS228.2249.7271.6mdadm F2FS328.8347.8377.8OpenBenchmarking.orgMB/s, More Is BetterCompile Bench 0.6Test: Initial CreateRAID 0RAID 1RAID 10RAID 5RAID 6100200300400500

IOzone

Record Size: 4Kb - File Size: 8GB - Disk Test: Write Performance

MinAvgMaxNative Btrfs5857601142mdadm EXT43035651111mdadm Btrfs2805311141mdadm XFS3046121112mdadm F2FS3035491100OpenBenchmarking.orgMB/s, More Is BetterIOzone 3.405Record Size: 4Kb - File Size: 8GB - Disk Test: Write PerformanceRAID 0RAID 1RAID 10RAID 5RAID 62004006008001000

FS-Mark

Test: 4000 Files, 32 Sub Dirs, 1MB Size

MinAvgMaxNative Btrfs35.638.440.7mdadm EXT431.034.738.7mdadm Btrfs30.634.539.4mdadm XFS29.233.840.7mdadm F2FS51.457.066.4OpenBenchmarking.orgFiles/s, More Is BetterFS-Mark 3.3Test: 4000 Files, 32 Sub Dirs, 1MB SizeRAID 0RAID 1RAID 10RAID 5RAID 620406080100

FS-Mark

Test: 5000 Files, 1MB Size, 4 Threads

MinAvgMaxNative Btrfs69.077.083.7mdadm EXT470.484.599.6mdadm Btrfs56.870.692.0mdadm XFS53.870.388.7mdadm F2FS67.877.786.6OpenBenchmarking.orgFiles/s, More Is BetterFS-Mark 3.3Test: 5000 Files, 1MB Size, 4 ThreadsRAID 0RAID 1RAID 10RAID 5RAID 620406080100

FS-Mark

Test: 1000 Files, 1MB Size

MinAvgMaxNative Btrfs35.138.441.4mdadm EXT431.134.739.0mdadm Btrfs30.435.039.6mdadm XFS30.935.842.4mdadm F2FS50.757.965.9OpenBenchmarking.orgFiles/s, More Is BetterFS-Mark 3.3Test: 1000 Files, 1MB SizeRAID 0RAID 1RAID 10RAID 5RAID 620406080100

Flexible IO Tester

Type: Sequential Write - IO Engine: Libaio - Buffered: No - Block Size: 16KB - Disk Target: Default Test Directory - Result: MB/s

MinAvgMaxNative Btrfs130.3234.7319.4mdadm EXT4115.3234.5508.7mdadm Btrfs110.0207.9385.3mdadm XFS115.8216.2398.7mdadm F2FS113.4216.5419.3OpenBenchmarking.orgMB/s, More Is BetterFlexible IO Tester 2.1.11Type: Sequential Write - IO Engine: Libaio - Buffered: No - Block Size: 16KB - Disk Target: Default Test Directory - Result: MB/sRAID 0RAID 1RAID 10RAID 5RAID 6130260390520650

Flexible IO Tester

Type: Sequential Read - IO Engine: Libaio - Buffered: No - Block Size: 16KB - Disk Target: Default Test Directory - Result: MB/s

MinAvgMaxNative Btrfs324686961mdadm EXT4303630958mdadm Btrfs844876925mdadm XFS301301301mdadm F2FS300649998OpenBenchmarking.orgMB/s, More Is BetterFlexible IO Tester 2.1.11Type: Sequential Read - IO Engine: Libaio - Buffered: No - Block Size: 16KB - Disk Target: Default Test Directory - Result: MB/sRAID 0RAID 1RAID 10RAID 5RAID 62004006008001000

Flexible IO Tester

Type: Random Write - IO Engine: Libaio - Buffered: No - Block Size: 16KB - Disk Target: Default Test Directory - Result: MB/s

MinAvgMaxNative Btrfs128.0271.2497.2mdadm EXT4110.3195.2421.4mdadm Btrfs109.2208.8381.5mdadm XFS115.5191.3393.5mdadm F2FS109.6190.9387.0OpenBenchmarking.orgMB/s, More Is BetterFlexible IO Tester 2.1.11Type: Random Write - IO Engine: Libaio - Buffered: No - Block Size: 16KB - Disk Target: Default Test Directory - Result: MB/sRAID 0RAID 1RAID 10RAID 5RAID 6130260390520650

Flexible IO Tester

Type: Random Read - IO Engine: Libaio - Buffered: No - Block Size: 16KB - Disk Target: Default Test Directory - Result: MB/s

MinAvgMaxNative Btrfs233571758mdadm EXT4214652855mdadm Btrfs225688910mdadm XFS236629928mdadm F2FS240701916OpenBenchmarking.orgMB/s, More Is BetterFlexible IO Tester 2.1.11Type: Random Read - IO Engine: Libaio - Buffered: No - Block Size: 16KB - Disk Target: Default Test Directory - Result: MB/sRAID 0RAID 1RAID 10RAID 5RAID 62004006008001000

Phoronix Test Suite v10.8.4