Ubuntu Linux RAID 0/1/5/6/10 Disk Testing

RAID mdadm benchmarks with four Intel SSDs using RAID 0, 1, 5, 6, and 10 levels on Linux 3.18 kernel with various file-systems. Benchmarks by Michael Larabel for a future article on Phoronix.com.

HTML result view exported from: https://openbenchmarking.org/result/1411054-LI-BTRFSRAID55&grs&rdt.

ProcessorMotherboardChipsetMemoryDiskGraphicsAudioMonitorNetworkOSKernelDesktopDisplay ServerDisplay DriverOpenGLCompilerFile-SystemScreen ResolutionNative Btrfsmdadm EXT4mdadm Btrfsmdadm XFSmdadm F2FS RAID 0 RAID 1 RAID 10 RAID 5 RAID 6 RAID 0 RAID 1 RAID 10 RAID 5 RAID 6 RAID 0 RAID 1 RAID 10 RAID 5 RAID 6 RAID 0 RAID 1 RAID 10 RAID 5 RAID 6 RAID 0 RAID 1 RAID 10 RAID 5 RAID 6Intel Core i7-5960X @ 4.00GHz (16 Cores)Gigabyte X99-UD4-CFIntel Xeon E5 v3/Core16384MB4 x 120GB INTEL SSDSC2BW12 + 128GB Crucial_CT128MX1XFX AMD Radeon HD 7950/8950 OEM / R9 280 3072MBIntel C610/X99SyncMasterIntel ConnectionUbuntu 14.103.18.0-031800rc1-generic (x86_64)Unity 7.3.1X Server 1.16.0radeon 7.4.03.3 Mesa 10.3.0 Gallium 0.4GCC 4.9.1btrfs2560x1600ext4btrfsxfsf2fsOpenBenchmarking.orgCompiler Details- --build=x86_64-linux-gnu --disable-browser-plugin --disable-vtable-verify --disable-werror --enable-checking=release --enable-clocale=gnu --enable-gnu-unique-object --enable-gtk-cairo --enable-java-awt=gtk --enable-java-home --enable-languages=c,c++,java,go,d,fortran,objc,obj-c++ --enable-libstdcxx-debug --enable-libstdcxx-time=yes --enable-multiarch --enable-multilib --enable-nls --enable-objc-gc --enable-plugin --enable-shared --enable-threads=posix --host=x86_64-linux-gnu --target=x86_64-linux-gnu --with-abi=m64 --with-arch-32=i686 --with-arch-directory=amd64 --with-multilib-list=m32,m64,mx32 --with-tune=generic -v Disk Details- Native Btrfs: RAID 0: DEADLINE / relatime,rw,space_cache,ssd- Native Btrfs: RAID 1: DEADLINE / relatime,rw,space_cache,ssd- Native Btrfs: RAID 10: DEADLINE / relatime,rw,space_cache,ssd- Native Btrfs: RAID 5: DEADLINE / relatime,rw,space_cache,ssd- Native Btrfs: RAID 6: DEADLINE / relatime,rw,space_cache,ssd- mdadm EXT4: RAID 0: DEADLINE / data=ordered,relatime,rw,stripe=512- mdadm EXT4: RAID 1: DEADLINE / data=ordered,relatime,rw- mdadm EXT4: RAID 10: DEADLINE / data=ordered,relatime,rw,stripe=256- mdadm EXT4: RAID 5: DEADLINE / data=ordered,relatime,rw,stripe=384- mdadm EXT4: RAID 6: DEADLINE / data=ordered,relatime,rw,stripe=256- mdadm Btrfs: RAID 0: DEADLINE / relatime,rw,space_cache- mdadm Btrfs: RAID 1: DEADLINE / relatime,rw,space_cache- mdadm Btrfs: RAID 10: DEADLINE / relatime,rw,space_cache- mdadm Btrfs: RAID 5: DEADLINE / relatime,rw,space_cache- mdadm Btrfs: RAID 6: DEADLINE / relatime,rw,space_cache- mdadm XFS: RAID 0: DEADLINE / attr2,inode64,noquota,relatime,rw,sunit=1024,swidth=4096- mdadm XFS: RAID 1: DEADLINE / attr2,inode64,noquota,relatime,rw- mdadm XFS: RAID 10: DEADLINE / attr2,inode64,noquota,relatime,rw,sunit=1024,swidth=2048- mdadm XFS: RAID 5: DEADLINE / attr2,inode64,noquota,relatime,rw,sunit=1024,swidth=3072- mdadm XFS: RAID 6: DEADLINE / attr2,inode64,noquota,relatime,rw,sunit=1024,swidth=2048- mdadm F2FS: RAID 0: DEADLINE / acl,active_logs=6,background_gc=on,relatime,rw,user_xattr- mdadm F2FS: RAID 1: DEADLINE / acl,active_logs=6,background_gc=on,relatime,rw,user_xattr- mdadm F2FS: RAID 10: DEADLINE / acl,active_logs=6,background_gc=on,relatime,rw,user_xattr- mdadm F2FS: RAID 5: DEADLINE / acl,active_logs=6,background_gc=on,relatime,rw,user_xattr- mdadm F2FS: RAID 6: DEADLINE / acl,active_logs=6,background_gc=on,relatime,rw,user_xattrProcessor Details- Scaling Governor: intel_pstate powersaveSystem Details- Disk Scheduler: DEADLINE. Python 2.7.8.

fio: Seq Write - Libaio - No - 16KB - Default Test Directory - MB/sfio: Rand Write - Libaio - No - 16KB - Default Test Directory - MB/sfio: Rand Read - Libaio - No - 16KB - Default Test Directory - MB/siozone: 4Kb - 8GB - Write Performancefio: Seq Read - Libaio - No - 16KB - Default Test Directory - MB/sfs-mark: 4000 Files, 32 Sub Dirs, 1MB Sizefs-mark: 1000 Files, 1MB Sizefs-mark: 5000 Files, 1MB Size, 4 Threadscompilebench: Initial CreateNative Btrfsmdadm EXT4mdadm Btrfsmdadm XFSmdadm F2FS RAID 0 RAID 1 RAID 10 RAID 5 RAID 6 RAID 0 RAID 1 RAID 10 RAID 5 RAID 6 RAID 0 RAID 1 RAID 10 RAID 5 RAID 6 RAID 0 RAID 1 RAID 10 RAID 5 RAID 6 RAID 0 RAID 1 RAID 10 RAID 5 RAID 6319.41497.23758.261142.43824.9740.7341.4383.73206.74130.32128.00233.00584.85323.8035.6335.1069.03187.72272.19267.48469.10599.32450.7838.5038.3377.07177.54242.55248.93742.06876.77869.6839.1738.8779.87189.59208.96214.56653.54598.43960.8937.7338.2075.47195.02508.73421.37818.331111.2638.7338.9790.40325.18115.34110.32214.45302.75303.3034.8033.9099.57303.04199.95163.86684.95585.3036.8037.0378.13306.37192.80146.07689.39441.27957.593131.1084.07316.17155.78134.18854.92385.3932.0032.2770.43320.82385.34381.45816.021141.1839.4339.6092.00178.31109.97109.22224.69280.39854.9833.4334.0364.00160.58161.90159.53695.44478.65843.6836.9337.8075.60156.97209.60214.31792.61438.55879.0932.1033.3764.73156.21172.72179.73909.98315.85925.0530.6330.4356.80151.64398.74393.49841.971111.5640.7342.4388.73264.38115.82115.51236.43303.50301.1632.7734.0768.90228.19214.94166.75384.53588.4235.8339.5781.70228.20197.19151.05751.77580.9530.3332.1058.10255.99154.10129.62928.27473.1729.2330.9353.83271.62419.29387.04872.841100.4266.4065.8786.57331.33113.35109.58239.73303.32299.5853.5753.5080.63328.79200.36177.32741.30566.1461.3062.8777.97352.47196.78149.41736.18421.8652.4756.3775.50377.77152.53131.26916.30350.84997.5351.4050.7367.83348.76OpenBenchmarking.org

Flexible IO Tester

Type: Sequential Write - IO Engine: Libaio - Buffered: No - Block Size: 16KB - Disk Target: Default Test Directory - Result: MB/s

MinAvgMaxNative Btrfs130.3234.7319.4mdadm EXT4115.3234.5508.7mdadm Btrfs110.0207.9385.3mdadm XFS115.8216.2398.7mdadm F2FS113.4216.5419.3OpenBenchmarking.orgMB/s, More Is BetterFlexible IO Tester 2.1.11Type: Sequential Write - IO Engine: Libaio - Buffered: No - Block Size: 16KB - Disk Target: Default Test Directory - Result: MB/sRAID 0RAID 1RAID 10RAID 5RAID 6130260390520650

Flexible IO Tester

Type: Random Write - IO Engine: Libaio - Buffered: No - Block Size: 16KB - Disk Target: Default Test Directory - Result: MB/s

MinAvgMaxNative Btrfs128.0271.2497.2mdadm EXT4110.3195.2421.4mdadm Btrfs109.2208.8381.5mdadm XFS115.5191.3393.5mdadm F2FS109.6190.9387.0OpenBenchmarking.orgMB/s, More Is BetterFlexible IO Tester 2.1.11Type: Random Write - IO Engine: Libaio - Buffered: No - Block Size: 16KB - Disk Target: Default Test Directory - Result: MB/sRAID 0RAID 1RAID 10RAID 5RAID 6130260390520650

Flexible IO Tester

Type: Random Read - IO Engine: Libaio - Buffered: No - Block Size: 16KB - Disk Target: Default Test Directory - Result: MB/s

MinAvgMaxNative Btrfs233571758mdadm EXT4214652855mdadm Btrfs225688910mdadm XFS236629928mdadm F2FS240701916OpenBenchmarking.orgMB/s, More Is BetterFlexible IO Tester 2.1.11Type: Random Read - IO Engine: Libaio - Buffered: No - Block Size: 16KB - Disk Target: Default Test Directory - Result: MB/sRAID 0RAID 1RAID 10RAID 5RAID 62004006008001000

IOzone

Record Size: 4Kb - File Size: 8GB - Disk Test: Write Performance

MinAvgMaxNative Btrfs5857601142mdadm EXT43035651111mdadm Btrfs2805311141mdadm XFS3046121112mdadm F2FS3035491100OpenBenchmarking.orgMB/s, More Is BetterIOzone 3.405Record Size: 4Kb - File Size: 8GB - Disk Test: Write PerformanceRAID 0RAID 1RAID 10RAID 5RAID 62004006008001000

Flexible IO Tester

Type: Sequential Read - IO Engine: Libaio - Buffered: No - Block Size: 16KB - Disk Target: Default Test Directory - Result: MB/s

MinAvgMaxNative Btrfs324686961mdadm EXT4303630958mdadm Btrfs844876925mdadm XFS301301301mdadm F2FS300649998OpenBenchmarking.orgMB/s, More Is BetterFlexible IO Tester 2.1.11Type: Sequential Read - IO Engine: Libaio - Buffered: No - Block Size: 16KB - Disk Target: Default Test Directory - Result: MB/sRAID 0RAID 1RAID 10RAID 5RAID 62004006008001000

FS-Mark

Test: 4000 Files, 32 Sub Dirs, 1MB Size

MinAvgMaxNative Btrfs35.638.440.7mdadm EXT431.034.738.7mdadm Btrfs30.634.539.4mdadm XFS29.233.840.7mdadm F2FS51.457.066.4OpenBenchmarking.orgFiles/s, More Is BetterFS-Mark 3.3Test: 4000 Files, 32 Sub Dirs, 1MB SizeRAID 0RAID 1RAID 10RAID 5RAID 620406080100

FS-Mark

Test: 1000 Files, 1MB Size

MinAvgMaxNative Btrfs35.138.441.4mdadm EXT431.134.739.0mdadm Btrfs30.435.039.6mdadm XFS30.935.842.4mdadm F2FS50.757.965.9OpenBenchmarking.orgFiles/s, More Is BetterFS-Mark 3.3Test: 1000 Files, 1MB SizeRAID 0RAID 1RAID 10RAID 5RAID 620406080100

FS-Mark

Test: 5000 Files, 1MB Size, 4 Threads

MinAvgMaxNative Btrfs69.077.083.7mdadm EXT470.484.599.6mdadm Btrfs56.870.692.0mdadm XFS53.870.388.7mdadm F2FS67.877.786.6OpenBenchmarking.orgFiles/s, More Is BetterFS-Mark 3.3Test: 5000 Files, 1MB Size, 4 ThreadsRAID 0RAID 1RAID 10RAID 5RAID 620406080100

Compile Bench

Test: Initial Create

MinAvgMaxNative Btrfs177.5191.3206.7mdadm EXT4303.0314.3325.2mdadm Btrfs151.6160.7178.3mdadm XFS228.2249.7271.6mdadm F2FS328.8347.8377.8OpenBenchmarking.orgMB/s, More Is BetterCompile Bench 0.6Test: Initial CreateRAID 0RAID 1RAID 10RAID 5RAID 6100200300400500

Phoronix Test Suite v10.8.5