SSD RAID Linux 5.5 Performance

Linux 5.5 solid-state drive RAID benchmarks by Michael Larabel.

HTML result view exported from: https://openbenchmarking.org/result/2001248-PTS-SSDRAID643&sro&grs.

ProcessorMotherboardChipsetMemoryDiskGraphicsNetworkOSKernelDesktopDisplay ServerCompilerFile-SystemSingleRAID0RAID1RAID5RAID6RAID10 Btrfs EXT4 F2FS XFS Btrfs EXT4 F2FS XFS Btrfs EXT4 F2FS XFS Btrfs EXT4 F2FS XFS Btrfs EXT4 F2FS XFS Btrfs EXT4 F2FS XFS2 x AMD EPYC 7601 32-Core (64 Cores / 128 Threads)Dell 02MJ3T (1.2.5 BIOS)AMD 17h16 x 32 GB DDR4-2666MT/s 36ASF4G72PZ-2G6D2280GB INTEL SSDPED1D280GA + 13 x 500GB Samsung SSD 860 + 120GB SSDSCKJB120G7RMatrox G200eW32 x Broadcom BCM57416 NetXtreme-E Dual-Media 10G RDMA + 2 x Broadcom NetXtreme BCM5720 2-port PCIeUbuntu 19.105.5.0-999-generic (x86_64) 20191231GNOME Shell 3.34.1X ServerGCC 9.2.1 20191008btrfsext4f2fsxfsbtrfsext4f2fsxfsbtrfsext4f2fsxfsbtrfsext4f2fsxfsbtrfsext4f2fsxfsbtrfsext4f2fsxfsOpenBenchmarking.orgCompiler Details- --build=x86_64-linux-gnu --disable-vtable-verify --disable-werror --enable-bootstrap --enable-checking=release --enable-clocale=gnu --enable-default-pie --enable-gnu-unique-object --enable-languages=c,ada,c++,go,brig,d,fortran,objc,obj-c++,gm2 --enable-libstdcxx-debug --enable-libstdcxx-time=yes --enable-multiarch --enable-multilib --enable-nls --enable-offload-targets=nvptx-none,hsa --enable-plugin --enable-shared --enable-threads=posix --host=x86_64-linux-gnu --program-prefix=x86_64-linux-gnu- --target=x86_64-linux-gnu --with-abi=m64 --with-arch-32=i686 --with-default-libstdcxx-abi=new --with-gcc-major-version-only --with-multilib-list=m32,m64,mx32 --with-target-system-zlib=auto --with-tune=generic --without-cuda-driver -v Disk Details- Single: Btrfs: MQ-DEADLINE / relatime,rw,space_cache,ssd,subvol=/,subvolid=5- Single: EXT4: MQ-DEADLINE / relatime,rw- Single: F2FS: MQ-DEADLINE / acl,active_logs=6,alloc_mode=default,background_gc=on,discard,extent_cache,flush_merge,fsync_mode=posix,inline_data,inline_dentry,inline_xattr,lazytime,mode=adaptive,no_heap,relatime,rw,user_xattr - Single: XFS: MQ-DEADLINE / attr2,inode64,logbsize=32k,logbufs=8,noquota,relatime,rw- RAID0: Btrfs: MQ-DEADLINE / relatime,rw,space_cache,ssd,subvol=/,subvolid=5 / RAID0- RAID0: EXT4: NONE / relatime,rw,stripe=512 / raid0 sde[3] sdd[2] sdc[1] sdb[0]- RAID0: F2FS: NONE / acl,active_logs=6,alloc_mode=default,background_gc=on,discard,extent_cache,flush_merge,fsync_mode=posix,inline_data,inline_dentry,inline_xattr,lazytime,mode=adaptive,no_heap,relatime,rw,user_xattr / raid0 sde[3] sdd[2] sdc[1] sdb[0] - RAID0: XFS: NONE / attr2,inode64,logbsize=32k,logbufs=8,noquota,relatime,rw,sunit=1024,swidth=4096 / raid0 sde[3] sdb[0] sdc[1] sdd[2]- RAID1: Btrfs: MQ-DEADLINE / relatime,rw,space_cache,ssd,subvol=/,subvolid=5 / RAID1- RAID1: EXT4: NONE / relatime,rw / raid1 sde[3] sdd[2] sdc[1] sdb[0]- RAID1: F2FS: NONE / acl,active_logs=6,alloc_mode=default,background_gc=on,discard,extent_cache,flush_merge,fsync_mode=posix,inline_data,inline_dentry,inline_xattr,lazytime,mode=adaptive,no_heap,relatime,rw,user_xattr / raid1 sdb[0] sdc[2] sdd[3] sdn[1] - RAID1: XFS: NONE / attr2,inode64,logbsize=32k,logbufs=8,noquota,relatime,rw / raid1 sdb[0] sdc[2] sdd[3] sdn[1]- RAID5: Btrfs: MQ-DEADLINE / relatime,rw,space_cache,ssd,subvol=/,subvolid=5 / RAID5- RAID5: EXT4: NONE / relatime,rw,stripe=384 / raid5 sde[4] sdd[2] sdc[1] sdb[0]- RAID5: F2FS: NONE / acl,active_logs=6,alloc_mode=default,background_gc=on,discard,extent_cache,flush_merge,fsync_mode=posix,inline_data,inline_dentry,inline_xattr,lazytime,mode=adaptive,no_heap,relatime,rw,user_xattr / raid5 sde[4] sdd[2] sdc[1] sdb[0] - RAID5: XFS: NONE / attr2,inode64,logbsize=32k,logbufs=8,noquota,relatime,rw,sunit=1024,swidth=3072 / raid5 sde[4] sdd[2] sdb[0] sdc[1]- RAID6: Btrfs: MQ-DEADLINE / relatime,rw,space_cache,ssd,subvol=/,subvolid=5 / RAID6- RAID6: EXT4: NONE / relatime,rw,stripe=256 / raid6 sde[3] sdd[2] sdc[1] sdb[0]- RAID6: F2FS: NONE / acl,active_logs=6,alloc_mode=default,background_gc=on,discard,extent_cache,flush_merge,fsync_mode=posix,inline_data,inline_dentry,inline_xattr,lazytime,mode=adaptive,no_heap,relatime,rw,user_xattr / raid6 sdd[2] sde[3] sdc[1] sdb[0] - RAID6: XFS: NONE / attr2,inode64,logbsize=32k,logbufs=8,noquota,relatime,rw,sunit=1024,swidth=2048 / raid6 sde[3] sdc[1] sdd[2] sdb[0]- RAID10: Btrfs: MQ-DEADLINE / relatime,rw,space_cache,ssd,subvol=/,subvolid=5 / RAID10- RAID10: EXT4: NONE / relatime,rw,stripe=256 / raid10 sde[3] sdd[2] sdc[1] sdb[0]- RAID10: F2FS: NONE / acl,active_logs=6,alloc_mode=default,background_gc=on,discard,extent_cache,flush_merge,fsync_mode=posix,inline_data,inline_dentry,inline_xattr,lazytime,mode=adaptive,no_heap,relatime,rw,user_xattr / raid10 sdd[1] sde[2] sdc[0] sdb[3] - RAID10: XFS: NONE / attr2,inode64,logbsize=32k,logbufs=8,noquota,relatime,rw,sunit=1024,swidth=2048 / raid10 sdb[0] sdd[2] sdc[1] sdh[3]Processor Details- CPU Microcode: 0x8001227Python Details- Python 2.7.17rc1 + Python 3.7.5rc1Security Details- itlb_multihit: Not affected + l1tf: Not affected + mds: Not affected + meltdown: Not affected + spec_store_bypass: Mitigation of SSB disabled via prctl and seccomp + spectre_v1: Mitigation of usercopy/swapgs barriers and __user pointer sanitization + spectre_v2: Mitigation of Full AMD retpoline IBPB: conditional STIBP: disabled RSB filling + tsx_async_abort: Not affected

fio: Seq Read - Linux AIO - Yes - No - 2MB - Default Test Directoryfio: Seq Write - Linux AIO - Yes - No - 2MB - Default Test Directoryfio: Rand Write - Linux AIO - Yes - No - 4KB - Default Test Directoryior: Write Testsqlite: 1ior: Read Testcompilebench: Initial Createfio: Rand Read - Linux AIO - Yes - No - 4KB - Default Test Directorypgbench: Buffer Test - Normal Load - Read Writefs-mark: 4000 Files, 32 Sub Dirs, 1MB Sizefs-mark: 5000 Files, 1MB Size, 4 Threadsfs-mark: 1000 Files, 1MB Sizesqlite: 8startup-time: Seq Reads + Writes - LibreOffice Writer - Default Test Directorystartup-time: Seq Reads + Writes - GNOME Terminal - Default Test Directorystartup-time: Only Seq Reads - LibreOffice Writer - Default Test Directorystartup-time: Only Seq Reads - GNOME Terminal - Default Test Directorystartup-time: Seq Reads + Writes - xterm - Default Test Directorystartup-time: Only Seq Reads - xterm - Default Test DirectorySingleRAID0RAID1RAID5RAID6RAID10 Btrfs EXT4 F2FS XFS Btrfs EXT4 F2FS XFS Btrfs EXT4 F2FS XFS Btrfs EXT4 F2FS XFS Btrfs EXT4 F2FS XFS Btrfs EXT4 F2FS XFS45740569300176.7169.708343.45214.673281.905051161.9178.8165.3147.86533.546053.012761.646095.732017.967434.77555253357436790.0032.356377.69369.62105334877.47482886.5179.0171.9118.0404.854006.512336.014337.811221.420671.7633352226153200134.7146.928386.43257.994961.477653128.6180.0131.7217.6662.926004.183113.611785.348891.404331.7064452635275533189.2832.824396.46304.76105004691.47166784.4160.3185.484.4734.376225.632896.038677.648441.267001.7940016601284132200239.1671.887657.36292.014509.814990124.3282.5223.8251.8856.258118.7575012.787819.84583.044647.115781733864160667184.1736.032388.52362.18104676514.514369209.0583.2210.5121.71516.661022.163034.607446.46374.9927810.70601893831135533167.7453.900458.05329.9683575840.845624176.1304.2188.0218.3066.109259.4082212.570817.76363.217586.507951907957165667196.6235.303417.31319.5788456003.792328230.4442.9231.983.03517.052922.846335.440645.82605.0565010.0078452237564000157.62116.394297.90281.2874512873.470726103.1195.299.0278.1505.687176.630198.1217814.93302.881534.000222331673276780.5140.998373.21358.08103332433.65325286.3133.5136.9133.8511.453561.1536401.820441.464250.3212780.38508350816736550164.7748.580386.14204.3281443489.692139108.9151.6105.1256.8210.6498890.8182501.1571111.585560.3018520.516639504116972533168.7938.916386.08296.03103333691.18238589.2125.5140.8107.1941.0188150.7483892.140421.682780.3175930.4632961545101911566792.7983.301490.12289.6475283232.321890117.8204.4173.3227.6806.9306710.6795414.340122.70713.682818.6906418214249358068.9940.358369.20367.2096852681.12484474.2209.2129.2308.0226.057897.1859722.943730.39052.647780.20440717674278583386.0346.165452.46252.3183153609.47339475.8127.0103.0412.0522.915504.7766710.284515.26381.623395.53866185260512266766.4946.384369.78305.02103672625.42203373.0125.599.4221.8036.136508.6306728.068139.19071.859038.762441517728102667104.67139.651513.75291.5975513188.28236693.5188.8110.2293.4675.417568.6526711.670017.28863.369208.2269717683377532762.1642.278362.44367.3780522174.39279163.5148.773.7398.3383.267335.2401418.78924.74011.500534.9396317833205881772.9847.713336.81218.4881663235.97797858.992.664.7562.3192.164583.392577.2776711.21011.150613.7503717945259960055.0943.136355.75305.4996772079.27189842.690.069.6332.6804.693785.5524420.133226.70001.375176.03853987741104000197.1186.543500.32281.3474333068.922340110.9221.7189.8290.6935.647898.7558313.020220.77152.896146.841567023016825391.3437.344349.95369.2691032639.03309798.2210.799.5317.2381.867033.020564.035565.951220.9643331.8434564430062033174.5643.916432.59261.5182254328.782317177.6244.9180.7240.9580.5271390.6413061.464861.849310.2583330.672917686104286125185.8436.399392.51304.57103334654.438466106.7184.0220.788.4871.502252.381173.850315.029450.6847221.35563OpenBenchmarking.org

Flexible IO Tester

Type: Sequential Read - IO Engine: Linux AIO - Buffered: Yes - Direct: No - Block Size: 2MB - Disk Target: Default Test Directory

RAID0RAID10RAID1RAID5RAID6SingleOpenBenchmarking.orgMB/s, More Is BetterFlexible IO Tester 3.16Type: Sequential Read - IO Engine: Linux AIO - Buffered: Yes - Direct: No - Block Size: 2MB - Disk Target: Default Test DirectoryBtrfsEXT4F2FSXFS400800120016002000SE +/- 10.50, N = 3SE +/- 0.67, N = 3SE +/- 17.83, N = 12SE +/- 2.85, N = 3SE +/- 0.88, N = 3SE +/- 2.31, N = 3SE +/- 0.88, N = 3SE +/- 2.56, N = 6SE +/- 3.06, N = 3SE +/- 16.75, N = 3SE +/- 1.20, N = 3SE +/- 28.53, N = 15SE +/- 1.20, N = 3SE +/- 22.81, N = 3SE +/- 2.19, N = 3SE +/- 3.21, N = 3SE +/- 0.67, N = 3SE +/- 0.58, N = 31660173318931907987702644686522233508504154518211767185215171768178317944575255225261. (CC) gcc options: -rdynamic -std=gnu99 -ffast-math -include -O3 -U_FORTIFY_SOURCE -march=native -ll -lcurl -lssl -lcrypto -lnuma -libverbs -lrt -laio -lz -lm -lpthread -ldl

Flexible IO Tester

Type: Sequential Write - IO Engine: Linux AIO - Buffered: Yes - Direct: No - Block Size: 2MB - Disk Target: Default Test Directory

RAID0RAID10RAID1RAID5RAID6SingleOpenBenchmarking.orgMB/s, More Is BetterFlexible IO Tester 3.16Type: Sequential Write - IO Engine: Linux AIO - Buffered: Yes - Direct: No - Block Size: 2MB - Disk Target: Default Test DirectoryBtrfsEXT4F2FSXFS30060090012001500SE +/- 12.56, N = 9SE +/- 11.50, N = 3SE +/- 6.03, N = 3SE +/- 8.09, N = 3SE +/- 1.86, N = 3SE +/- 1.33, N = 3SE +/- 16.77, N = 3SE +/- 1.76, N = 3SE +/- 5.45, N = 15SE +/- 5.10, N = 15SE +/- 8.01, N = 3SE +/- 14.33, N = 3SE +/- 3.28, N = 3SE +/- 6.51, N = 15SE +/- 6.64, N = 3SE +/- 11.26, N = 3SE +/- 7.48, N = 15SE +/- 5.24, N = 3SE +/- 5.96, N = 15SE +/- 5.02, N = 5SE +/- 0.67, N = 3SE +/- 6.63, N = 15SE +/- 2.67, N = 312848648319577413013001042375167167116910194244276057283373205254053352613521. (CC) gcc options: -rdynamic -std=gnu99 -ffast-math -include -O3 -U_FORTIFY_SOURCE -march=native -ll -lcurl -lssl -lcrypto -lnuma -libverbs -lrt -laio -lz -lm -lpthread -ldl

Flexible IO Tester

Type: Random Write - IO Engine: Linux AIO - Buffered: Yes - Direct: No - Block Size: 4KB - Disk Target: Default Test Directory

RAID0RAID10RAID1RAID5RAID6SingleOpenBenchmarking.orgIOPS, More Is BetterFlexible IO Tester 3.16Type: Random Write - IO Engine: Linux AIO - Buffered: Yes - Direct: No - Block Size: 4KB - Disk Target: Default Test DirectoryBtrfsEXT4F2FSXFS40K80K120K160K200KSE +/- 1087.59, N = 15SE +/- 1452.97, N = 3SE +/- 2396.16, N = 15SE +/- 881.92, N = 3SE +/- 1036.88, N = 15SE +/- 560.75, N = 3SE +/- 1132.38, N = 4SE +/- 305.51, N = 3SE +/- 887.57, N = 3SE +/- 512.35, N = 4SE +/- 3679.91, N = 12SE +/- 1201.85, N = 3SE +/- 1335.89, N = 15SE +/- 765.03, N = 9SE +/- 1655.20, N = 15SE +/- 666.67, N = 3SE +/- 1432.20, N = 15SE +/- 1935.51, N = 12SE +/- 1249.00, N = 3SE +/- 1274.10, N = 3SE +/- 352.77, N = 3SE +/- 173.21, N = 3SE +/- 536.45, N = 3132200160667135533165667104000682536203386125640003276736550725331156679358085833122667102667753275881799600693007436753200755331. (CC) gcc options: -rdynamic -std=gnu99 -ffast-math -include -O3 -U_FORTIFY_SOURCE -march=native -ll -lcurl -lssl -lcrypto -lnuma -libverbs -lrt -laio -lz -lm -lpthread -ldl

IOR

Write Test

RAID0RAID10RAID1RAID5RAID6SingleOpenBenchmarking.orgMB/s, More Is BetterIOR 3.2.1Write TestBtrfsEXT4F2FSXFS50100150200250SE +/- 1.56, N = 3SE +/- 0.95, N = 3SE +/- 1.13, N = 3SE +/- 0.44, N = 3SE +/- 1.59, N = 3SE +/- 0.53, N = 3SE +/- 0.70, N = 3SE +/- 0.63, N = 3SE +/- 1.50, N = 3SE +/- 0.37, N = 3SE +/- 8.84, N = 15SE +/- 0.62, N = 3SE +/- 0.65, N = 3SE +/- 0.33, N = 3SE +/- 0.78, N = 11SE +/- 1.31, N = 15SE +/- 1.37, N = 3SE +/- 0.62, N = 8SE +/- 1.29, N = 15SE +/- 0.92, N = 15SE +/- 0.91, N = 3SE +/- 0.28, N = 3SE +/- 1.97, N = 4SE +/- 0.78, N = 3239.16184.17167.74196.62197.1191.34174.56185.84157.6280.51164.77168.7992.7968.9986.0366.49104.6762.1672.9855.09176.7190.00134.71189.281. (CC) gcc options: -O2 -lm -pthread -lmpi

SQLite

Threads / Copies: 1

RAID0RAID10RAID1RAID5RAID6SingleOpenBenchmarking.orgSeconds, Fewer Is BetterSQLite 3.30.1Threads / Copies: 1BtrfsEXT4F2FSXFS306090120150SE +/- 0.83, N = 15SE +/- 0.06, N = 3SE +/- 0.52, N = 3SE +/- 0.02, N = 3SE +/- 3.42, N = 12SE +/- 0.17, N = 3SE +/- 0.29, N = 3SE +/- 0.01, N = 3SE +/- 6.10, N = 12SE +/- 0.24, N = 3SE +/- 0.71, N = 3SE +/- 0.13, N = 3SE +/- 0.68, N = 13SE +/- 0.12, N = 3SE +/- 0.13, N = 3SE +/- 0.08, N = 3SE +/- 5.38, N = 10SE +/- 0.12, N = 3SE +/- 0.16, N = 3SE +/- 0.14, N = 3SE +/- 0.69, N = 15SE +/- 0.02, N = 3SE +/- 2.68, N = 15SE +/- 0.02, N = 371.8936.0353.9035.3086.5437.3443.9236.40116.3941.0048.5838.9283.3040.3646.1746.38139.6542.2847.7143.1469.7132.3646.9332.821. (CC) gcc options: -O2 -lz -lm -ldl -lpthread

IOR

Read Test

RAID0RAID10RAID1RAID5RAID6SingleOpenBenchmarking.orgMB/s, More Is BetterIOR 3.2.1Read TestBtrfsEXT4F2FSXFS140280420560700SE +/- 4.52, N = 3SE +/- 1.62, N = 3SE +/- 1.69, N = 3SE +/- 0.39, N = 3SE +/- 5.60, N = 3SE +/- 8.71, N = 3SE +/- 1.64, N = 3SE +/- 1.27, N = 3SE +/- 14.92, N = 3SE +/- 1.12, N = 3SE +/- 4.95, N = 15SE +/- 0.35, N = 3SE +/- 22.70, N = 3SE +/- 0.84, N = 3SE +/- 1.54, N = 11SE +/- 9.42, N = 15SE +/- 9.91, N = 3SE +/- 2.01, N = 8SE +/- 17.95, N = 15SE +/- 8.69, N = 15SE +/- 0.93, N = 3SE +/- 1.12, N = 3SE +/- 6.45, N = 4SE +/- 0.43, N = 3657.36388.52458.05417.31500.32349.95432.59392.51297.90373.21386.14386.08490.12369.20452.46369.78513.75362.44336.81355.75343.45377.69386.43396.461. (CC) gcc options: -O2 -lm -pthread -lmpi

Compile Bench

Test: Initial Create

RAID0RAID10RAID1RAID5RAID6SingleOpenBenchmarking.orgMB/s, More Is BetterCompile Bench 0.6Test: Initial CreateBtrfsEXT4F2FSXFS80160240320400SE +/- 3.53, N = 6SE +/- 6.21, N = 3SE +/- 5.10, N = 3SE +/- 2.39, N = 3SE +/- 4.58, N = 3SE +/- 0.20, N = 3SE +/- 2.69, N = 15SE +/- 2.83, N = 3SE +/- 2.62, N = 10SE +/- 1.64, N = 3SE +/- 3.52, N = 12SE +/- 1.60, N = 3SE +/- 1.41, N = 3SE +/- 1.06, N = 3SE +/- 3.75, N = 3SE +/- 4.01, N = 3SE +/- 3.24, N = 3SE +/- 3.93, N = 3SE +/- 2.98, N = 15SE +/- 2.63, N = 3SE +/- 2.04, N = 3SE +/- 1.14, N = 3SE +/- 3.87, N = 15SE +/- 1.46, N = 3292.01362.18329.96319.57281.34369.26261.51304.57281.28358.08204.32296.03289.64367.20252.31305.02291.59367.37218.48305.49214.67369.62257.99304.76

Flexible IO Tester

Type: Random Read - IO Engine: Linux AIO - Buffered: Yes - Direct: No - Block Size: 4KB - Disk Target: Default Test Directory

RAID0RAID10RAID1RAID5RAID6SingleOpenBenchmarking.orgIOPS, More Is BetterFlexible IO Tester 3.16Type: Random Read - IO Engine: Linux AIO - Buffered: Yes - Direct: No - Block Size: 4KB - Disk Target: Default Test DirectoryEXT4F2FSXFSBtrfs2K4K6K8K10KSE +/- 33.33, N = 3SE +/- 6.81, N = 3SE +/- 11.20, N = 3SE +/- 86.52, N = 3SE +/- 5.51, N = 3SE +/- 141.18, N = 15SE +/- 33.33, N = 3SE +/- 93.63, N = 3SE +/- 66.67, N = 3SE +/- 25.79, N = 3SE +/- 33.33, N = 3SE +/- 47.77, N = 10SE +/- 25.08, N = 3SE +/- 7.84, N = 3SE +/- 33.33, N = 3SE +/- 19.60, N = 3SE +/- 17.95, N = 3SE +/- 11.89, N = 3SE +/- 14.81, N = 3SE +/- 33.33, N = 310467835788457433910382251033374511033381441033375289685831510367755180528166967710533105001. (CC) gcc options: -rdynamic -std=gnu99 -ffast-math -include -O3 -U_FORTIFY_SOURCE -march=native -ll -lcurl -lssl -lcrypto -lnuma -libverbs -lrt -laio -lz -lm -lpthread -ldl

PostgreSQL pgbench

Scaling: Buffer Test - Test: Normal Load - Mode: Read Write

RAID0RAID10RAID1RAID5RAID6SingleOpenBenchmarking.orgTPS, More Is BetterPostgreSQL pgbench 12.0Scaling: Buffer Test - Test: Normal Load - Mode: Read WriteBtrfsEXT4F2FSXFS14002800420056007000SE +/- 70.65, N = 9SE +/- 84.80, N = 5SE +/- 409.58, N = 6SE +/- 812.21, N = 9SE +/- 46.64, N = 9SE +/- 73.63, N = 9SE +/- 186.03, N = 9SE +/- 193.53, N = 6SE +/- 86.05, N = 9SE +/- 26.43, N = 3SE +/- 50.59, N = 3SE +/- 26.90, N = 3SE +/- 52.03, N = 3SE +/- 150.35, N = 9SE +/- 91.08, N = 9SE +/- 114.01, N = 9SE +/- 60.68, N = 9SE +/- 24.93, N = 3SE +/- 6.06, N = 3SE +/- 28.50, N = 9SE +/- 251.33, N = 6SE +/- 61.03, N = 3SE +/- 41.82, N = 3SE +/- 356.17, N = 94509.816514.515840.856003.793068.922639.034328.784654.442873.472433.653489.693691.183232.322681.123609.472625.423188.282174.393235.982079.273281.914877.474961.484691.471. (CC) gcc options: -fno-strict-aliasing -fwrapv -O2 -lpgcommon -lpgport -lpq -lpthread -lrt -lcrypt -ldl -lm

FS-Mark

Test: 4000 Files, 32 Sub Dirs, 1MB Size

RAID0RAID10RAID1RAID5RAID6SingleOpenBenchmarking.orgFiles/s, More Is BetterFS-Mark 3.3Test: 4000 Files, 32 Sub Dirs, 1MB SizeBtrfsEXT4F2FSXFS50100150200250SE +/- 5.80, N = 15SE +/- 0.60, N = 3SE +/- 3.02, N = 15SE +/- 1.14, N = 3SE +/- 4.73, N = 15SE +/- 0.93, N = 15SE +/- 1.03, N = 3SE +/- 1.39, N = 14SE +/- 3.45, N = 14SE +/- 5.68, N = 15SE +/- 1.33, N = 15SE +/- 7.05, N = 12SE +/- 5.90, N = 15SE +/- 0.23, N = 3SE +/- 0.75, N = 8SE +/- 5.77, N = 15SE +/- 3.30, N = 15SE +/- 0.72, N = 7SE +/- 0.09, N = 3SE +/- 3.39, N = 15SE +/- 1.22, N = 3SE +/- 0.09, N = 3SE +/- 0.20, N = 3SE +/- 0.44, N = 3124.3209.0176.1230.4110.998.2177.6106.7103.186.3108.989.2117.874.275.873.093.563.558.942.6161.986.5128.684.41. (CC) gcc options: -static

FS-Mark

Test: 5000 Files, 1MB Size, 4 Threads

RAID0RAID10RAID1RAID5RAID6SingleOpenBenchmarking.orgFiles/s, More Is BetterFS-Mark 3.3Test: 5000 Files, 1MB Size, 4 ThreadsBtrfsEXT4F2FSXFS130260390520650SE +/- 7.86, N = 15SE +/- 0.61, N = 3SE +/- 4.03, N = 5SE +/- 1.54, N = 3SE +/- 13.29, N = 15SE +/- 1.41, N = 3SE +/- 0.33, N = 3SE +/- 1.99, N = 7SE +/- 1.62, N = 3SE +/- 2.00, N = 12SE +/- 1.98, N = 3SE +/- 4.24, N = 12SE +/- 3.12, N = 3SE +/- 3.44, N = 15SE +/- 5.33, N = 9SE +/- 5.60, N = 12SE +/- 4.74, N = 15SE +/- 0.84, N = 3SE +/- 1.59, N = 12SE +/- 2.28, N = 12SE +/- 11.17, N = 15SE +/- 3.76, N = 14SE +/- 1.07, N = 3SE +/- 11.10, N = 12282.5583.2304.2442.9221.7210.7244.9184.0195.2133.5151.6125.5204.4209.2127.0125.5188.8148.792.690.0178.8179.0180.0160.31. (CC) gcc options: -static

FS-Mark

Test: 1000 Files, 1MB Size

RAID0RAID10RAID1RAID5RAID6SingleOpenBenchmarking.orgFiles/s, More Is BetterFS-Mark 3.3Test: 1000 Files, 1MB SizeBtrfsEXT4F2FSXFS50100150200250SE +/- 2.32, N = 3SE +/- 0.65, N = 3SE +/- 1.72, N = 3SE +/- 0.81, N = 3SE +/- 2.02, N = 3SE +/- 0.23, N = 3SE +/- 0.33, N = 3SE +/- 0.38, N = 3SE +/- 6.08, N = 15SE +/- 0.44, N = 3SE +/- 1.67, N = 3SE +/- 1.95, N = 4SE +/- 2.03, N = 3SE +/- 1.12, N = 3SE +/- 1.64, N = 15SE +/- 4.39, N = 12SE +/- 5.50, N = 15SE +/- 0.91, N = 15SE +/- 1.74, N = 15SE +/- 6.27, N = 12SE +/- 1.47, N = 3SE +/- 0.43, N = 3SE +/- 6.76, N = 15SE +/- 1.20, N = 3223.8210.5188.0231.9189.899.5180.7220.799.0136.9105.1140.8173.3129.2103.099.4110.273.764.769.6165.3171.9131.7185.41. (CC) gcc options: -static

SQLite

Threads / Copies: 8

RAID0RAID10RAID1RAID5RAID6SingleOpenBenchmarking.orgSeconds, Fewer Is BetterSQLite 3.30.1Threads / Copies: 8BtrfsEXT4F2FSXFS120240360480600SE +/- 14.70, N = 9SE +/- 0.53, N = 3SE +/- 2.43, N = 6SE +/- 0.49, N = 3SE +/- 14.01, N = 9SE +/- 12.80, N = 9SE +/- 0.72, N = 3SE +/- 0.52, N = 3SE +/- 15.06, N = 6SE +/- 0.04, N = 3SE +/- 0.60, N = 3SE +/- 0.94, N = 3SE +/- 0.89, N = 3SE +/- 15.91, N = 9SE +/- 6.79, N = 3SE +/- 13.52, N = 10SE +/- 1.92, N = 3SE +/- 11.35, N = 9SE +/- 10.63, N = 6SE +/- 27.89, N = 9SE +/- 1.40, N = 3SE +/- 0.38, N = 3SE +/- 0.53, N = 3SE +/- 0.82, N = 3251.89121.72218.3183.04290.69317.24240.9688.49278.15133.85256.82107.19227.68308.02412.05221.80293.47398.34562.32332.68147.87118.04217.6784.471. (CC) gcc options: -O2 -lz -lm -ldl -lpthread

Application Start-up Time

Background I/O Mix: Sequential Reads + Writes - Application To Start: LibreOffice Writer - Disk Target: Default Test Directory

RAID0RAID10RAID1RAID5RAID6SingleOpenBenchmarking.orgsec, Fewer Is BetterApplication Start-up Time 3.4.0Background I/O Mix: Sequential Reads + Writes - Application To Start: LibreOffice Writer - Disk Target: Default Test DirectoryBtrfsEXT4F2FSXFS816243240SE +/- 0.014474, N = 3SE +/- 0.096656, N = 3SE +/- 0.063263, N = 8SE +/- 0.050982, N = 3SE +/- 0.063041, N = 3SE +/- 0.050685, N = 12SE +/- 0.016241, N = 12SE +/- 0.020590, N = 4SE +/- 0.169130, N = 12SE +/- 0.035673, N = 12SE +/- 0.030886, N = 12SE +/- 0.098608, N = 9SE +/- 0.100570, N = 4SE +/- 0.027941, N = 3SE +/- 0.039545, N = 4SE +/- 0.083129, N = 4SE +/- 0.082398, N = 12SE +/- 0.126927, N = 12SE +/- 0.023827, N = 12SE +/- 0.508145, N = 9SE +/- 0.251992, N = 3SE +/- 0.028225, N = 3SE +/- 0.022316, N = 3SE +/- 0.009721, N = 36.25811016.6610006.10925017.0529005.6478901.8670300.5271391.5022505.6871701.4535600.6498891.0188156.9306706.0578902.9155006.1365005.4175603.2673302.1645804.69378033.5460004.8540002.9260004.376220

Application Start-up Time

Background I/O Mix: Sequential Reads + Writes - Application To Start: GNOME Terminal - Disk Target: Default Test Directory

RAID0RAID10RAID1RAID5RAID6SingleOpenBenchmarking.orgsec, Fewer Is BetterApplication Start-up Time 3.4.0Background I/O Mix: Sequential Reads + Writes - Application To Start: GNOME Terminal - Disk Target: Default Test DirectoryBtrfsEXT4F2FSXFS1224364860SE +/- 0.120651, N = 4SE +/- 0.078585, N = 3SE +/- 0.153551, N = 3SE +/- 0.192940, N = 3SE +/- 0.137693, N = 12SE +/- 0.048914, N = 3SE +/- 0.015592, N = 12SE +/- 0.158264, N = 12SE +/- 0.277801, N = 12SE +/- 0.028505, N = 12SE +/- 0.055703, N = 12SE +/- 0.026823, N = 12SE +/- 0.149649, N = 12SE +/- 0.423513, N = 12SE +/- 0.062336, N = 5SE +/- 0.072080, N = 3SE +/- 0.046415, N = 3SE +/- 0.130933, N = 12SE +/- 0.031917, N = 10SE +/- 0.187340, N = 12SE +/- 0.711449, N = 3SE +/- 0.009393, N = 3SE +/- 0.019191, N = 3SE +/- 0.013567, N = 38.75750022.1630009.40822022.8463008.7558303.0205600.6413062.3811706.6301901.1536400.8182500.74838910.6795407.1859704.7766708.6306708.6526705.2401403.3925705.55244053.0127006.5123304.1831105.632890

Application Start-up Time

Background I/O Mix: Only Sequential Reads - Application To Start: LibreOffice Writer - Disk Target: Default Test Directory

RAID0RAID10RAID1RAID5RAID6SingleOpenBenchmarking.orgsec, Fewer Is BetterApplication Start-up Time 3.4.0Background I/O Mix: Only Sequential Reads - Application To Start: LibreOffice Writer - Disk Target: Default Test DirectoryBtrfsEXT4F2FSXFS1428425670SE +/- 0.195276, N = 12SE +/- 0.449117, N = 5SE +/- 0.201190, N = 3SE +/- 0.349625, N = 8SE +/- 0.181354, N = 12SE +/- 0.078180, N = 12SE +/- 0.062840, N = 12SE +/- 0.093885, N = 12SE +/- 0.570641, N = 9SE +/- 0.102438, N = 12SE +/- 0.053631, N = 12SE +/- 0.075823, N = 12SE +/- 0.220382, N = 12SE +/- 1.950546, N = 9SE +/- 0.088326, N = 3SE +/- 0.365878, N = 5SE +/- 0.191342, N = 3SE +/- 0.220749, N = 5SE +/- 0.112266, N = 3SE +/- 0.012418, N = 3SE +/- 0.827731, N = 9SE +/- 0.011398, N = 3SE +/- 0.041717, N = 3SE +/- 0.005102, N = 312.78780034.60740012.57080035.44060013.0202004.0355601.4648603.8503108.1217801.8204401.1571112.14042014.34010022.94370010.28450028.06810011.67000018.7890007.27767020.13320061.6460006.0143303.6117806.038670

Application Start-up Time

Background I/O Mix: Only Sequential Reads - Application To Start: GNOME Terminal - Disk Target: Default Test Directory

RAID0RAID10RAID1RAID5RAID6SingleOpenBenchmarking.orgsec, Fewer Is BetterApplication Start-up Time 3.4.0Background I/O Mix: Only Sequential Reads - Application To Start: GNOME Terminal - Disk Target: Default Test DirectoryBtrfsEXT4F2FSXFS20406080100SE +/- 0.31761, N = 12SE +/- 0.47888, N = 8SE +/- 0.10985, N = 3SE +/- 0.47926, N = 3SE +/- 0.23225, N = 6SE +/- 0.17274, N = 12SE +/- 0.03872, N = 12SE +/- 0.06866, N = 3SE +/- 0.78610, N = 12SE +/- 0.04657, N = 12SE +/- 0.04695, N = 12SE +/- 0.07719, N = 12SE +/- 0.34990, N = 3SE +/- 2.55892, N = 9SE +/- 0.16662, N = 3SE +/- 0.50722, N = 3SE +/- 0.24142, N = 3SE +/- 0.36026, N = 3SE +/- 0.19153, N = 3SE +/- 0.22180, N = 3SE +/- 0.71070, N = 3SE +/- 0.01634, N = 3SE +/- 0.04835, N = 3SE +/- 0.01631, N = 319.8458046.4637017.7636045.8260020.771505.951221.849315.0294514.933001.464251.585561.6827822.7071030.3905015.2638039.1907017.2886024.7401011.2101026.7000095.732007.811225.348897.64844

Application Start-up Time

Background I/O Mix: Sequential Reads + Writes - Application To Start: xterm - Disk Target: Default Test Directory

RAID0RAID10RAID1RAID5RAID6SingleOpenBenchmarking.orgsec, Fewer Is BetterApplication Start-up Time 3.4.0Background I/O Mix: Sequential Reads + Writes - Application To Start: xterm - Disk Target: Default Test DirectoryBtrfsEXT4F2FSXFS48121620SE +/- 0.037447, N = 12SE +/- 0.024902, N = 3SE +/- 0.053719, N = 12SE +/- 0.070902, N = 4SE +/- 0.047374, N = 12SE +/- 0.027558, N = 12SE +/- 0.008816, N = 12SE +/- 0.024292, N = 12SE +/- 0.121435, N = 12SE +/- 0.014444, N = 12SE +/- 0.018173, N = 9SE +/- 0.015736, N = 9SE +/- 0.086821, N = 9SE +/- 0.419100, N = 12SE +/- 0.017753, N = 12SE +/- 0.131339, N = 12SE +/- 0.055449, N = 12SE +/- 0.029180, N = 12SE +/- 0.015238, N = 12SE +/- 0.061645, N = 12SE +/- 0.199033, N = 6SE +/- 0.014640, N = 3SE +/- 0.023516, N = 3SE +/- 0.009528, N = 33.0446404.9927803.2175805.0565002.8961400.9643330.2583330.6847222.8815300.3212780.3018520.3175933.6828102.6477801.6233901.8590303.3692001.5005301.1506101.37517017.9674001.4206701.4043301.267000

Application Start-up Time

Background I/O Mix: Only Sequential Reads - Application To Start: xterm - Disk Target: Default Test Directory

RAID0RAID10RAID1RAID5RAID6SingleOpenBenchmarking.orgsec, Fewer Is BetterApplication Start-up Time 3.4.0Background I/O Mix: Only Sequential Reads - Application To Start: xterm - Disk Target: Default Test DirectoryBtrfsEXT4F2FSXFS816243240SE +/- 0.034642, N = 3SE +/- 0.134867, N = 12SE +/- 0.152081, N = 12SE +/- 0.085574, N = 12SE +/- 0.106575, N = 12SE +/- 0.087434, N = 9SE +/- 0.022046, N = 12SE +/- 0.062133, N = 9SE +/- 0.264581, N = 9SE +/- 0.025577, N = 12SE +/- 0.037649, N = 12SE +/- 0.037580, N = 9SE +/- 0.155843, N = 12SE +/- 0.026604, N = 9SE +/- 0.035620, N = 3SE +/- 0.123068, N = 3SE +/- 0.151773, N = 12SE +/- 0.197645, N = 9SE +/- 0.035936, N = 9SE +/- 0.054491, N = 12SE +/- 0.121450, N = 3SE +/- 0.004017, N = 3SE +/- 0.007893, N = 3SE +/- 0.007412, N = 37.11578010.7060006.50795010.0078406.8415601.8434500.6729171.3556304.0002200.3850830.5166390.4632968.6906400.2044075.5386608.7624408.2269704.9396303.7503706.03853034.7755001.7633301.7064401.794000


Phoronix Test Suite v10.8.5