SSD RAID Linux 5.5 Performance

Linux 5.5 solid-state drive RAID benchmarks by Michael Larabel.

HTML result view exported from: https://openbenchmarking.org/result/2001248-PTS-SSDRAID643&grr&sor.

ProcessorMotherboardChipsetMemoryDiskGraphicsNetworkOSKernelDesktopDisplay ServerCompilerFile-SystemSingleRAID0RAID1RAID5RAID6RAID10 Btrfs EXT4 F2FS XFS Btrfs EXT4 F2FS XFS Btrfs EXT4 F2FS XFS Btrfs EXT4 F2FS XFS Btrfs EXT4 F2FS XFS Btrfs EXT4 F2FS XFS2 x AMD EPYC 7601 32-Core (64 Cores / 128 Threads)Dell 02MJ3T (1.2.5 BIOS)AMD 17h16 x 32 GB DDR4-2666MT/s 36ASF4G72PZ-2G6D2280GB INTEL SSDPED1D280GA + 13 x 500GB Samsung SSD 860 + 120GB SSDSCKJB120G7RMatrox G200eW32 x Broadcom BCM57416 NetXtreme-E Dual-Media 10G RDMA + 2 x Broadcom NetXtreme BCM5720 2-port PCIeUbuntu 19.105.5.0-999-generic (x86_64) 20191231GNOME Shell 3.34.1X ServerGCC 9.2.1 20191008btrfsext4f2fsxfsbtrfsext4f2fsxfsbtrfsext4f2fsxfsbtrfsext4f2fsxfsbtrfsext4f2fsxfsbtrfsext4f2fsxfsOpenBenchmarking.orgCompiler Details- --build=x86_64-linux-gnu --disable-vtable-verify --disable-werror --enable-bootstrap --enable-checking=release --enable-clocale=gnu --enable-default-pie --enable-gnu-unique-object --enable-languages=c,ada,c++,go,brig,d,fortran,objc,obj-c++,gm2 --enable-libstdcxx-debug --enable-libstdcxx-time=yes --enable-multiarch --enable-multilib --enable-nls --enable-offload-targets=nvptx-none,hsa --enable-plugin --enable-shared --enable-threads=posix --host=x86_64-linux-gnu --program-prefix=x86_64-linux-gnu- --target=x86_64-linux-gnu --with-abi=m64 --with-arch-32=i686 --with-default-libstdcxx-abi=new --with-gcc-major-version-only --with-multilib-list=m32,m64,mx32 --with-target-system-zlib=auto --with-tune=generic --without-cuda-driver -v Disk Details- Single: Btrfs: MQ-DEADLINE / relatime,rw,space_cache,ssd,subvol=/,subvolid=5- Single: EXT4: MQ-DEADLINE / relatime,rw- Single: F2FS: MQ-DEADLINE / acl,active_logs=6,alloc_mode=default,background_gc=on,discard,extent_cache,flush_merge,fsync_mode=posix,inline_data,inline_dentry,inline_xattr,lazytime,mode=adaptive,no_heap,relatime,rw,user_xattr - Single: XFS: MQ-DEADLINE / attr2,inode64,logbsize=32k,logbufs=8,noquota,relatime,rw- RAID0: Btrfs: MQ-DEADLINE / relatime,rw,space_cache,ssd,subvol=/,subvolid=5 / RAID0- RAID0: EXT4: NONE / relatime,rw,stripe=512 / raid0 sde[3] sdd[2] sdc[1] sdb[0]- RAID0: F2FS: NONE / acl,active_logs=6,alloc_mode=default,background_gc=on,discard,extent_cache,flush_merge,fsync_mode=posix,inline_data,inline_dentry,inline_xattr,lazytime,mode=adaptive,no_heap,relatime,rw,user_xattr / raid0 sde[3] sdd[2] sdc[1] sdb[0] - RAID0: XFS: NONE / attr2,inode64,logbsize=32k,logbufs=8,noquota,relatime,rw,sunit=1024,swidth=4096 / raid0 sde[3] sdb[0] sdc[1] sdd[2]- RAID1: Btrfs: MQ-DEADLINE / relatime,rw,space_cache,ssd,subvol=/,subvolid=5 / RAID1- RAID1: EXT4: NONE / relatime,rw / raid1 sde[3] sdd[2] sdc[1] sdb[0]- RAID1: F2FS: NONE / acl,active_logs=6,alloc_mode=default,background_gc=on,discard,extent_cache,flush_merge,fsync_mode=posix,inline_data,inline_dentry,inline_xattr,lazytime,mode=adaptive,no_heap,relatime,rw,user_xattr / raid1 sdb[0] sdc[2] sdd[3] sdn[1] - RAID1: XFS: NONE / attr2,inode64,logbsize=32k,logbufs=8,noquota,relatime,rw / raid1 sdb[0] sdc[2] sdd[3] sdn[1]- RAID5: Btrfs: MQ-DEADLINE / relatime,rw,space_cache,ssd,subvol=/,subvolid=5 / RAID5- RAID5: EXT4: NONE / relatime,rw,stripe=384 / raid5 sde[4] sdd[2] sdc[1] sdb[0]- RAID5: F2FS: NONE / acl,active_logs=6,alloc_mode=default,background_gc=on,discard,extent_cache,flush_merge,fsync_mode=posix,inline_data,inline_dentry,inline_xattr,lazytime,mode=adaptive,no_heap,relatime,rw,user_xattr / raid5 sde[4] sdd[2] sdc[1] sdb[0] - RAID5: XFS: NONE / attr2,inode64,logbsize=32k,logbufs=8,noquota,relatime,rw,sunit=1024,swidth=3072 / raid5 sde[4] sdd[2] sdb[0] sdc[1]- RAID6: Btrfs: MQ-DEADLINE / relatime,rw,space_cache,ssd,subvol=/,subvolid=5 / RAID6- RAID6: EXT4: NONE / relatime,rw,stripe=256 / raid6 sde[3] sdd[2] sdc[1] sdb[0]- RAID6: F2FS: NONE / acl,active_logs=6,alloc_mode=default,background_gc=on,discard,extent_cache,flush_merge,fsync_mode=posix,inline_data,inline_dentry,inline_xattr,lazytime,mode=adaptive,no_heap,relatime,rw,user_xattr / raid6 sdd[2] sde[3] sdc[1] sdb[0] - RAID6: XFS: NONE / attr2,inode64,logbsize=32k,logbufs=8,noquota,relatime,rw,sunit=1024,swidth=2048 / raid6 sde[3] sdc[1] sdd[2] sdb[0]- RAID10: Btrfs: MQ-DEADLINE / relatime,rw,space_cache,ssd,subvol=/,subvolid=5 / RAID10- RAID10: EXT4: NONE / relatime,rw,stripe=256 / raid10 sde[3] sdd[2] sdc[1] sdb[0]- RAID10: F2FS: NONE / acl,active_logs=6,alloc_mode=default,background_gc=on,discard,extent_cache,flush_merge,fsync_mode=posix,inline_data,inline_dentry,inline_xattr,lazytime,mode=adaptive,no_heap,relatime,rw,user_xattr / raid10 sdd[1] sde[2] sdc[0] sdb[3] - RAID10: XFS: NONE / attr2,inode64,logbsize=32k,logbufs=8,noquota,relatime,rw,sunit=1024,swidth=2048 / raid10 sdb[0] sdd[2] sdc[1] sdh[3]Processor Details- CPU Microcode: 0x8001227Python Details- Python 2.7.17rc1 + Python 3.7.5rc1Security Details- itlb_multihit: Not affected + l1tf: Not affected + mds: Not affected + meltdown: Not affected + spec_store_bypass: Mitigation of SSB disabled via prctl and seccomp + spectre_v1: Mitigation of usercopy/swapgs barriers and __user pointer sanitization + spectre_v2: Mitigation of Full AMD retpoline IBPB: conditional STIBP: disabled RSB filling + tsx_async_abort: Not affected

pgbench: Buffer Test - Normal Load - Read Writestartup-time: Seq Reads + Writes - xterm - Default Test Directorysqlite: 8startup-time: Only Seq Reads - xterm - Default Test Directorystartup-time: Only Seq Reads - LibreOffice Writer - Default Test Directorystartup-time: Seq Reads + Writes - GNOME Terminal - Default Test Directoryfs-mark: 5000 Files, 1MB Size, 4 Threadsstartup-time: Only Seq Reads - GNOME Terminal - Default Test Directorystartup-time: Seq Reads + Writes - LibreOffice Writer - Default Test Directoryfs-mark: 4000 Files, 32 Sub Dirs, 1MB Sizesqlite: 1compilebench: Initial Createfio: Rand Write - Linux AIO - Yes - No - 4KB - Default Test Directoryfio: Seq Write - Linux AIO - Yes - No - 2MB - Default Test Directoryfio: Seq Read - Linux AIO - Yes - No - 2MB - Default Test Directoryfio: Rand Read - Linux AIO - Yes - No - 4KB - Default Test Directoryfs-mark: 1000 Files, 1MB Sizeior: Read Testior: Write TestSingleRAID0RAID1RAID5RAID6RAID10 Btrfs EXT4 F2FS XFS Btrfs EXT4 F2FS XFS Btrfs EXT4 F2FS XFS Btrfs EXT4 F2FS XFS Btrfs EXT4 F2FS XFS Btrfs EXT4 F2FS XFS3281.90505117.9674147.86534.775561.646053.0127178.895.732033.5460161.969.708214.6769300405457165.3343.45176.714877.4748281.42067118.0401.763336.014336.51233179.07.811224.8540086.532.356369.627436733552510533171.9377.6990.004961.4776531.40433217.6661.706443.611784.18311180.05.348892.92600128.646.928257.9953200261522131.7386.43134.714691.4716671.2670084.4731.794006.038675.63289160.37.648444.3762284.432.824304.767553335252610500185.4396.46189.284509.8149903.04464251.8857.1157812.78788.75750282.519.84586.25811124.371.887292.0113220012841660223.8657.36239.166514.5143694.99278121.71510.706034.607422.1630583.246.463716.6610209.036.032362.18160667864173310467210.5388.52184.175840.8456243.21758218.3066.5079512.57089.40822304.217.76366.10925176.153.900329.9613553383118938357188.0458.05167.746003.7923285.0565083.03510.0078435.440622.8463442.945.826017.0529230.435.303319.5716566795719078845231.9417.31196.622873.4707262.88153278.1504.000228.121786.63019195.214.93305.68717103.1116.394281.2864000375522745199.0297.90157.622433.6532520.321278133.8510.3850831.820441.153640133.51.464251.4535686.340.998358.083276716723310333136.9373.2180.513489.6921390.301852256.8210.5166391.1571110.818250151.61.585560.649889108.948.580204.32365501675088144105.1386.14164.773691.1823850.317593107.1940.4632962.140420.748389125.51.682781.01881589.238.916296.0372533116950410333140.8386.08168.793232.3218903.68281227.6808.6906414.340110.67954204.422.70716.93067117.883.301289.64115667101915457528173.3490.1292.792681.1248442.64778308.0220.20440722.94377.18597209.230.39056.0578974.240.358367.209358042418219685129.2369.2068.993609.4733941.62339412.0525.5386610.28454.77667127.015.26382.9155075.846.165252.318583342717678315103.0452.4686.032625.4220331.85903221.8038.7624428.06818.63067125.539.19076.1365073.046.384305.0212266760518521036799.4369.7866.493188.2823663.36920293.4678.2269711.67008.65267188.817.28865.4175693.5139.651291.5910266772815177551110.2513.75104.672174.3927911.50053398.3384.9396318.7895.24014148.724.74013.2673363.542.278367.37753273371768805273.7362.4462.163235.9779781.15061562.3193.750377.277673.3925792.611.21012.1645858.947.713218.48588173201783816664.7336.8172.982079.2718981.37517332.6806.0385320.13325.5524490.026.70004.6937842.643.136305.49996005251794967769.6355.7555.093068.9223402.89614290.6936.8415613.02028.75583221.720.77155.64789110.986.543281.341040007419877433189.8500.32197.112639.0330970.964333317.2381.843454.035563.02056210.75.951221.8670398.237.344369.2668253301702910399.5349.9591.344328.7823170.258333240.9580.6729171.464860.641306244.91.849310.527139177.643.916261.51620333006448225180.7432.59174.564654.4384660.68472288.4871.355633.850312.38117184.05.029451.50225106.736.399304.5786125104268610333220.7392.51185.84OpenBenchmarking.org

PostgreSQL pgbench

Scaling: Buffer Test - Test: Normal Load - Mode: Read Write

RAID0SingleRAID10RAID1RAID5RAID6OpenBenchmarking.orgTPS, More Is BetterPostgreSQL pgbench 12.0Scaling: Buffer Test - Test: Normal Load - Mode: Read WriteEXT4XFSF2FSBtrfs14002800420056007000SE +/- 84.80, N = 5SE +/- 812.21, N = 9SE +/- 409.58, N = 6SE +/- 70.65, N = 9SE +/- 41.82, N = 3SE +/- 61.03, N = 3SE +/- 356.17, N = 9SE +/- 251.33, N = 6SE +/- 193.53, N = 6SE +/- 186.03, N = 9SE +/- 46.64, N = 9SE +/- 73.63, N = 9SE +/- 26.90, N = 3SE +/- 50.59, N = 3SE +/- 86.05, N = 9SE +/- 26.43, N = 3SE +/- 91.08, N = 9SE +/- 52.03, N = 3SE +/- 150.35, N = 9SE +/- 114.01, N = 9SE +/- 6.06, N = 3SE +/- 60.68, N = 9SE +/- 24.93, N = 3SE +/- 28.50, N = 96514.516003.795840.854509.814961.484877.474691.473281.914654.444328.783068.922639.033691.183489.692873.472433.653609.473232.322681.122625.423235.983188.282174.392079.271. (CC) gcc options: -fno-strict-aliasing -fwrapv -O2 -lpgcommon -lpgport -lpq -lpthread -lrt -lcrypt -ldl -lm

Application Start-up Time

Background I/O Mix: Sequential Reads + Writes - Application To Start: xterm - Disk Target: Default Test Directory

RAID10RAID1RAID6SingleRAID5RAID0OpenBenchmarking.orgsec, Fewer Is BetterApplication Start-up Time 3.4.0Background I/O Mix: Sequential Reads + Writes - Application To Start: xterm - Disk Target: Default Test DirectoryF2FSXFSEXT4Btrfs48121620SE +/- 0.008816, N = 12SE +/- 0.024292, N = 12SE +/- 0.027558, N = 12SE +/- 0.047374, N = 12SE +/- 0.018173, N = 9SE +/- 0.015736, N = 9SE +/- 0.014444, N = 12SE +/- 0.121435, N = 12SE +/- 0.015238, N = 12SE +/- 0.061645, N = 12SE +/- 0.029180, N = 12SE +/- 0.055449, N = 12SE +/- 0.009528, N = 3SE +/- 0.023516, N = 3SE +/- 0.014640, N = 3SE +/- 0.199033, N = 6SE +/- 0.017753, N = 12SE +/- 0.131339, N = 12SE +/- 0.419100, N = 12SE +/- 0.086821, N = 9SE +/- 0.037447, N = 12SE +/- 0.053719, N = 12SE +/- 0.024902, N = 3SE +/- 0.070902, N = 40.2583330.6847220.9643332.8961400.3018520.3175930.3212782.8815301.1506101.3751701.5005303.3692001.2670001.4043301.42067017.9674001.6233901.8590302.6477803.6828103.0446403.2175804.9927805.056500

SQLite

Threads / Copies: 8

RAID0SingleRAID10RAID1RAID5RAID6OpenBenchmarking.orgSeconds, Fewer Is BetterSQLite 3.30.1Threads / Copies: 8XFSEXT4BtrfsF2FS120240360480600SE +/- 0.49, N = 3SE +/- 0.53, N = 3SE +/- 2.43, N = 6SE +/- 14.70, N = 9SE +/- 0.82, N = 3SE +/- 0.38, N = 3SE +/- 1.40, N = 3SE +/- 0.53, N = 3SE +/- 0.52, N = 3SE +/- 0.72, N = 3SE +/- 14.01, N = 9SE +/- 12.80, N = 9SE +/- 0.94, N = 3SE +/- 0.04, N = 3SE +/- 0.60, N = 3SE +/- 15.06, N = 6SE +/- 13.52, N = 10SE +/- 0.89, N = 3SE +/- 15.91, N = 9SE +/- 6.79, N = 3SE +/- 1.92, N = 3SE +/- 27.89, N = 9SE +/- 11.35, N = 9SE +/- 10.63, N = 683.04121.72218.31251.8984.47118.04147.87217.6788.49240.96290.69317.24107.19133.85256.82278.15221.80227.68308.02412.05293.47332.68398.34562.321. (CC) gcc options: -O2 -lz -lm -ldl -lpthread

Application Start-up Time

Background I/O Mix: Only Sequential Reads - Application To Start: xterm - Disk Target: Default Test Directory

RAID5RAID1RAID10SingleRAID6RAID0OpenBenchmarking.orgsec, Fewer Is BetterApplication Start-up Time 3.4.0Background I/O Mix: Only Sequential Reads - Application To Start: xterm - Disk Target: Default Test DirectoryEXT4XFSF2FSBtrfs816243240SE +/- 0.026604, N = 9SE +/- 0.035620, N = 3SE +/- 0.155843, N = 12SE +/- 0.123068, N = 3SE +/- 0.025577, N = 12SE +/- 0.037580, N = 9SE +/- 0.037649, N = 12SE +/- 0.264581, N = 9SE +/- 0.022046, N = 12SE +/- 0.062133, N = 9SE +/- 0.087434, N = 9SE +/- 0.106575, N = 12SE +/- 0.007893, N = 3SE +/- 0.004017, N = 3SE +/- 0.007412, N = 3SE +/- 0.121450, N = 3SE +/- 0.035936, N = 9SE +/- 0.197645, N = 9SE +/- 0.054491, N = 12SE +/- 0.151773, N = 12SE +/- 0.152081, N = 12SE +/- 0.034642, N = 3SE +/- 0.085574, N = 12SE +/- 0.134867, N = 120.2044075.5386608.6906408.7624400.3850830.4632960.5166394.0002200.6729171.3556301.8434506.8415601.7064401.7633301.79400034.7755003.7503704.9396306.0385308.2269706.5079507.11578010.00784010.706000

Application Start-up Time

Background I/O Mix: Only Sequential Reads - Application To Start: LibreOffice Writer - Disk Target: Default Test Directory

RAID1RAID10SingleRAID6RAID5RAID0OpenBenchmarking.orgsec, Fewer Is BetterApplication Start-up Time 3.4.0Background I/O Mix: Only Sequential Reads - Application To Start: LibreOffice Writer - Disk Target: Default Test DirectoryF2FSEXT4XFSBtrfs1428425670SE +/- 0.053631, N = 12SE +/- 0.102438, N = 12SE +/- 0.075823, N = 12SE +/- 0.570641, N = 9SE +/- 0.062840, N = 12SE +/- 0.093885, N = 12SE +/- 0.078180, N = 12SE +/- 0.181354, N = 12SE +/- 0.041717, N = 3SE +/- 0.011398, N = 3SE +/- 0.005102, N = 3SE +/- 0.827731, N = 9SE +/- 0.112266, N = 3SE +/- 0.191342, N = 3SE +/- 0.220749, N = 5SE +/- 0.012418, N = 3SE +/- 0.088326, N = 3SE +/- 0.220382, N = 12SE +/- 1.950546, N = 9SE +/- 0.365878, N = 5SE +/- 0.201190, N = 3SE +/- 0.195276, N = 12SE +/- 0.449117, N = 5SE +/- 0.349625, N = 81.1571111.8204402.1404208.1217801.4648603.8503104.03556013.0202003.6117806.0143306.03867061.6460007.27767011.67000018.78900020.13320010.28450014.34010022.94370028.06810012.57080012.78780034.60740035.440600

Application Start-up Time

Background I/O Mix: Sequential Reads + Writes - Application To Start: GNOME Terminal - Disk Target: Default Test Directory

RAID10RAID1RAID6SingleRAID5RAID0OpenBenchmarking.orgsec, Fewer Is BetterApplication Start-up Time 3.4.0Background I/O Mix: Sequential Reads + Writes - Application To Start: GNOME Terminal - Disk Target: Default Test DirectoryF2FSXFSEXT4Btrfs1224364860SE +/- 0.015592, N = 12SE +/- 0.158264, N = 12SE +/- 0.048914, N = 3SE +/- 0.137693, N = 12SE +/- 0.026823, N = 12SE +/- 0.055703, N = 12SE +/- 0.028505, N = 12SE +/- 0.277801, N = 12SE +/- 0.031917, N = 10SE +/- 0.130933, N = 12SE +/- 0.187340, N = 12SE +/- 0.046415, N = 3SE +/- 0.019191, N = 3SE +/- 0.013567, N = 3SE +/- 0.009393, N = 3SE +/- 0.711449, N = 3SE +/- 0.062336, N = 5SE +/- 0.423513, N = 12SE +/- 0.072080, N = 3SE +/- 0.149649, N = 12SE +/- 0.120651, N = 4SE +/- 0.153551, N = 3SE +/- 0.078585, N = 3SE +/- 0.192940, N = 30.6413062.3811703.0205608.7558300.7483890.8182501.1536406.6301903.3925705.2401405.5524408.6526704.1831105.6328906.51233053.0127004.7766707.1859708.63067010.6795408.7575009.40822022.16300022.846300

FS-Mark

Test: 5000 Files, 1MB Size, 4 Threads

RAID0RAID10RAID5RAID1RAID6SingleOpenBenchmarking.orgFiles/s, More Is BetterFS-Mark 3.3Test: 5000 Files, 1MB Size, 4 ThreadsEXT4XFSF2FSBtrfs130260390520650SE +/- 0.61, N = 3SE +/- 1.54, N = 3SE +/- 4.03, N = 5SE +/- 7.86, N = 15SE +/- 0.33, N = 3SE +/- 13.29, N = 15SE +/- 1.41, N = 3SE +/- 1.99, N = 7SE +/- 3.44, N = 15SE +/- 3.12, N = 3SE +/- 5.33, N = 9SE +/- 5.60, N = 12SE +/- 1.62, N = 3SE +/- 1.98, N = 3SE +/- 2.00, N = 12SE +/- 4.24, N = 12SE +/- 4.74, N = 15SE +/- 0.84, N = 3SE +/- 1.59, N = 12SE +/- 2.28, N = 12SE +/- 1.07, N = 3SE +/- 3.76, N = 14SE +/- 11.17, N = 15SE +/- 11.10, N = 12583.2442.9304.2282.5244.9221.7210.7184.0209.2204.4127.0125.5195.2151.6133.5125.5188.8148.792.690.0180.0179.0178.8160.31. (CC) gcc options: -static

Application Start-up Time

Background I/O Mix: Only Sequential Reads - Application To Start: GNOME Terminal - Disk Target: Default Test Directory

RAID1RAID10SingleRAID6RAID5RAID0OpenBenchmarking.orgsec, Fewer Is BetterApplication Start-up Time 3.4.0Background I/O Mix: Only Sequential Reads - Application To Start: GNOME Terminal - Disk Target: Default Test DirectoryEXT4F2FSXFSBtrfs20406080100SE +/- 0.04657, N = 12SE +/- 0.04695, N = 12SE +/- 0.07719, N = 12SE +/- 0.78610, N = 12SE +/- 0.03872, N = 12SE +/- 0.06866, N = 3SE +/- 0.17274, N = 12SE +/- 0.23225, N = 6SE +/- 0.04835, N = 3SE +/- 0.01631, N = 3SE +/- 0.01634, N = 3SE +/- 0.71070, N = 3SE +/- 0.19153, N = 3SE +/- 0.24142, N = 3SE +/- 0.36026, N = 3SE +/- 0.22180, N = 3SE +/- 0.16662, N = 3SE +/- 0.34990, N = 3SE +/- 2.55892, N = 9SE +/- 0.50722, N = 3SE +/- 0.10985, N = 3SE +/- 0.31761, N = 12SE +/- 0.47926, N = 3SE +/- 0.47888, N = 81.464251.585561.6827814.933001.849315.029455.9512220.771505.348897.648447.8112295.7320011.2101017.2886024.7401026.7000015.2638022.7071030.3905039.1907017.7636019.8458045.8260046.46370

Application Start-up Time

Background I/O Mix: Sequential Reads + Writes - Application To Start: LibreOffice Writer - Disk Target: Default Test Directory

RAID10RAID1RAID6RAID5SingleRAID0OpenBenchmarking.orgsec, Fewer Is BetterApplication Start-up Time 3.4.0Background I/O Mix: Sequential Reads + Writes - Application To Start: LibreOffice Writer - Disk Target: Default Test DirectoryF2FSXFSEXT4Btrfs816243240SE +/- 0.016241, N = 12SE +/- 0.020590, N = 4SE +/- 0.050685, N = 12SE +/- 0.063041, N = 3SE +/- 0.030886, N = 12SE +/- 0.098608, N = 9SE +/- 0.035673, N = 12SE +/- 0.169130, N = 12SE +/- 0.023827, N = 12SE +/- 0.126927, N = 12SE +/- 0.508145, N = 9SE +/- 0.082398, N = 12SE +/- 0.039545, N = 4SE +/- 0.027941, N = 3SE +/- 0.083129, N = 4SE +/- 0.100570, N = 4SE +/- 0.022316, N = 3SE +/- 0.009721, N = 3SE +/- 0.028225, N = 3SE +/- 0.251992, N = 3SE +/- 0.063263, N = 8SE +/- 0.014474, N = 3SE +/- 0.096656, N = 3SE +/- 0.050982, N = 30.5271391.5022501.8670305.6478900.6498891.0188151.4535605.6871702.1645803.2673304.6937805.4175602.9155006.0578906.1365006.9306702.9260004.3762204.85400033.5460006.1092506.25811016.66100017.052900

FS-Mark

Test: 4000 Files, 32 Sub Dirs, 1MB Size

RAID0RAID10SingleRAID5RAID1RAID6OpenBenchmarking.orgFiles/s, More Is BetterFS-Mark 3.3Test: 4000 Files, 32 Sub Dirs, 1MB SizeXFSEXT4F2FSBtrfs50100150200250SE +/- 1.14, N = 3SE +/- 0.60, N = 3SE +/- 3.02, N = 15SE +/- 5.80, N = 15SE +/- 1.03, N = 3SE +/- 4.73, N = 15SE +/- 1.39, N = 14SE +/- 0.93, N = 15SE +/- 1.22, N = 3SE +/- 0.20, N = 3SE +/- 0.09, N = 3SE +/- 0.44, N = 3SE +/- 5.90, N = 15SE +/- 0.75, N = 8SE +/- 0.23, N = 3SE +/- 5.77, N = 15SE +/- 1.33, N = 15SE +/- 3.45, N = 14SE +/- 7.05, N = 12SE +/- 5.68, N = 15SE +/- 3.30, N = 15SE +/- 0.72, N = 7SE +/- 0.09, N = 3SE +/- 3.39, N = 15230.4209.0176.1124.3177.6110.9106.798.2161.9128.686.584.4117.875.874.273.0108.9103.189.286.393.563.558.942.61. (CC) gcc options: -static

SQLite

Threads / Copies: 1

SingleRAID0RAID10RAID1RAID5RAID6OpenBenchmarking.orgSeconds, Fewer Is BetterSQLite 3.30.1Threads / Copies: 1EXT4XFSF2FSBtrfs306090120150SE +/- 0.02, N = 3SE +/- 0.02, N = 3SE +/- 2.68, N = 15SE +/- 0.69, N = 15SE +/- 0.02, N = 3SE +/- 0.06, N = 3SE +/- 0.52, N = 3SE +/- 0.83, N = 15SE +/- 0.01, N = 3SE +/- 0.17, N = 3SE +/- 0.29, N = 3SE +/- 3.42, N = 12SE +/- 0.13, N = 3SE +/- 0.24, N = 3SE +/- 0.71, N = 3SE +/- 6.10, N = 12SE +/- 0.12, N = 3SE +/- 0.13, N = 3SE +/- 0.08, N = 3SE +/- 0.68, N = 13SE +/- 0.12, N = 3SE +/- 0.14, N = 3SE +/- 0.16, N = 3SE +/- 5.38, N = 1032.3632.8246.9369.7135.3036.0353.9071.8936.4037.3443.9286.5438.9241.0048.58116.3940.3646.1746.3883.3042.2843.1447.71139.651. (CC) gcc options: -O2 -lz -lm -ldl -lpthread

Compile Bench

Test: Initial Create

SingleRAID10RAID6RAID5RAID0RAID1OpenBenchmarking.orgMB/s, More Is BetterCompile Bench 0.6Test: Initial CreateEXT4F2FSXFSBtrfs80160240320400SE +/- 1.14, N = 3SE +/- 1.46, N = 3SE +/- 3.87, N = 15SE +/- 2.04, N = 3SE +/- 0.20, N = 3SE +/- 2.83, N = 3SE +/- 4.58, N = 3SE +/- 2.69, N = 15SE +/- 3.93, N = 3SE +/- 2.63, N = 3SE +/- 3.24, N = 3SE +/- 2.98, N = 15SE +/- 1.06, N = 3SE +/- 4.01, N = 3SE +/- 1.41, N = 3SE +/- 3.75, N = 3SE +/- 6.21, N = 3SE +/- 5.10, N = 3SE +/- 2.39, N = 3SE +/- 3.53, N = 6SE +/- 1.64, N = 3SE +/- 1.60, N = 3SE +/- 2.62, N = 10SE +/- 3.52, N = 12369.62304.76257.99214.67369.26304.57281.34261.51367.37305.49291.59218.48367.20305.02289.64252.31362.18329.96319.57292.01358.08296.03281.28204.32

Flexible IO Tester

Type: Random Write - IO Engine: Linux AIO - Buffered: Yes - Direct: No - Block Size: 4KB - Disk Target: Default Test Directory

RAID0RAID5RAID10RAID6SingleRAID1OpenBenchmarking.orgIOPS, More Is BetterFlexible IO Tester 3.16Type: Random Write - IO Engine: Linux AIO - Buffered: Yes - Direct: No - Block Size: 4KB - Disk Target: Default Test DirectoryXFSEXT4F2FSBtrfs40K80K120K160K200KSE +/- 881.92, N = 3SE +/- 1452.97, N = 3SE +/- 2396.16, N = 15SE +/- 1087.59, N = 15SE +/- 1655.20, N = 15SE +/- 1201.85, N = 3SE +/- 1335.89, N = 15SE +/- 765.03, N = 9SE +/- 1132.38, N = 4SE +/- 1036.88, N = 15SE +/- 560.75, N = 3SE +/- 666.67, N = 3SE +/- 1249.00, N = 3SE +/- 1432.20, N = 15SE +/- 1935.51, N = 12SE +/- 536.45, N = 3SE +/- 352.77, N = 3SE +/- 1274.10, N = 3SE +/- 173.21, N = 3SE +/- 3679.91, N = 12SE +/- 305.51, N = 3SE +/- 512.35, N = 4SE +/- 887.57, N = 3165667160667135533132200122667115667935808583310400086125682536203310266799600753275881775533743676930053200725336400036550327671. (CC) gcc options: -rdynamic -std=gnu99 -ffast-math -include -O3 -U_FORTIFY_SOURCE -march=native -ll -lcurl -lssl -lcrypto -lnuma -libverbs -lrt -laio -lz -lm -lpthread -ldl

Flexible IO Tester

Type: Sequential Write - IO Engine: Linux AIO - Buffered: Yes - Direct: No - Block Size: 2MB - Disk Target: Default Test Directory

RAID0RAID1RAID10RAID5RAID6SingleOpenBenchmarking.orgMB/s, More Is BetterFlexible IO Tester 3.16Type: Sequential Write - IO Engine: Linux AIO - Buffered: Yes - Direct: No - Block Size: 2MB - Disk Target: Default Test DirectoryBtrfsXFSEXT4F2FS30060090012001500SE +/- 12.56, N = 9SE +/- 8.09, N = 3SE +/- 11.50, N = 3SE +/- 6.03, N = 3SE +/- 8.01, N = 3SE +/- 1.76, N = 3SE +/- 5.10, N = 15SE +/- 5.45, N = 15SE +/- 16.77, N = 3SE +/- 1.86, N = 3SE +/- 1.33, N = 3SE +/- 14.33, N = 3SE +/- 6.64, N = 3SE +/- 6.51, N = 15SE +/- 3.28, N = 3SE +/- 11.26, N = 3SE +/- 5.96, N = 15SE +/- 7.48, N = 15SE +/- 5.24, N = 3SE +/- 5.02, N = 5SE +/- 2.67, N = 3SE +/- 0.67, N = 3SE +/- 6.63, N = 1512849578648311169375167167104274130130010196054274247285253373204053523352611. (CC) gcc options: -rdynamic -std=gnu99 -ffast-math -include -O3 -U_FORTIFY_SOURCE -march=native -ll -lcurl -lssl -lcrypto -lnuma -libverbs -lrt -laio -lz -lm -lpthread -ldl

Flexible IO Tester

Type: Sequential Read - IO Engine: Linux AIO - Buffered: Yes - Direct: No - Block Size: 2MB - Disk Target: Default Test Directory

RAID0RAID5RAID6RAID10SingleRAID1OpenBenchmarking.orgMB/s, More Is BetterFlexible IO Tester 3.16Type: Sequential Read - IO Engine: Linux AIO - Buffered: Yes - Direct: No - Block Size: 2MB - Disk Target: Default Test DirectoryXFSF2FSEXT4Btrfs400800120016002000SE +/- 2.85, N = 3SE +/- 17.83, N = 12SE +/- 0.67, N = 3SE +/- 10.50, N = 3SE +/- 1.20, N = 3SE +/- 1.20, N = 3SE +/- 28.53, N = 15SE +/- 16.75, N = 3SE +/- 3.21, N = 3SE +/- 2.19, N = 3SE +/- 22.81, N = 3SE +/- 0.88, N = 3SE +/- 0.88, N = 3SE +/- 2.31, N = 3SE +/- 0.58, N = 3SE +/- 0.67, N = 3SE +/- 3.06, N = 3SE +/- 2.56, N = 61907189317331660185218211767154517941783176815179877026866445265255224575225085042331. (CC) gcc options: -rdynamic -std=gnu99 -ffast-math -include -O3 -U_FORTIFY_SOURCE -march=native -ll -lcurl -lssl -lcrypto -lnuma -libverbs -lrt -laio -lz -lm -lpthread -ldl

Flexible IO Tester

Type: Random Read - IO Engine: Linux AIO - Buffered: Yes - Direct: No - Block Size: 4KB - Disk Target: Default Test Directory

SingleRAID0RAID5RAID10RAID1RAID6OpenBenchmarking.orgIOPS, More Is BetterFlexible IO Tester 3.16Type: Random Read - IO Engine: Linux AIO - Buffered: Yes - Direct: No - Block Size: 4KB - Disk Target: Default Test DirectoryEXT4XFSF2FSBtrfs2K4K6K8K10KSE +/- 33.33, N = 3SE +/- 33.33, N = 3SE +/- 11.20, N = 3SE +/- 6.81, N = 3SE +/- 33.33, N = 3SE +/- 25.08, N = 3SE +/- 7.84, N = 3SE +/- 47.77, N = 10SE +/- 33.33, N = 3SE +/- 5.51, N = 3SE +/- 141.18, N = 15SE +/- 86.52, N = 3SE +/- 33.33, N = 3SE +/- 66.67, N = 3SE +/- 25.79, N = 3SE +/- 93.63, N = 3SE +/- 14.81, N = 3SE +/- 11.89, N = 3SE +/- 17.95, N = 3SE +/- 19.60, N = 310533105001046788458357103679685831575281033391038225743310333103338144745196778166805275511. (CC) gcc options: -rdynamic -std=gnu99 -ffast-math -include -O3 -U_FORTIFY_SOURCE -march=native -ll -lcurl -lssl -lcrypto -lnuma -libverbs -lrt -laio -lz -lm -lpthread -ldl

FS-Mark

Test: 1000 Files, 1MB Size

RAID0RAID10SingleRAID5RAID1RAID6OpenBenchmarking.orgFiles/s, More Is BetterFS-Mark 3.3Test: 1000 Files, 1MB SizeXFSBtrfsEXT4F2FS50100150200250SE +/- 0.81, N = 3SE +/- 2.32, N = 3SE +/- 0.65, N = 3SE +/- 1.72, N = 3SE +/- 0.38, N = 3SE +/- 2.02, N = 3SE +/- 0.33, N = 3SE +/- 0.23, N = 3SE +/- 1.20, N = 3SE +/- 0.43, N = 3SE +/- 1.47, N = 3SE +/- 6.76, N = 15SE +/- 2.03, N = 3SE +/- 1.12, N = 3SE +/- 1.64, N = 15SE +/- 4.39, N = 12SE +/- 1.95, N = 4SE +/- 0.44, N = 3SE +/- 1.67, N = 3SE +/- 6.08, N = 15SE +/- 5.50, N = 15SE +/- 0.91, N = 15SE +/- 6.27, N = 12SE +/- 1.74, N = 15231.9223.8210.5188.0220.7189.8180.799.5185.4171.9165.3131.7173.3129.2103.099.4140.8136.9105.199.0110.273.769.664.71. (CC) gcc options: -static

IOR

Read Test

RAID0RAID6RAID10RAID5SingleRAID1OpenBenchmarking.orgMB/s, More Is BetterIOR 3.2.1Read TestBtrfsF2FSXFSEXT4140280420560700SE +/- 4.52, N = 3SE +/- 1.69, N = 3SE +/- 0.39, N = 3SE +/- 1.62, N = 3SE +/- 9.91, N = 3SE +/- 2.01, N = 8SE +/- 8.69, N = 15SE +/- 17.95, N = 15SE +/- 5.60, N = 3SE +/- 1.64, N = 3SE +/- 1.27, N = 3SE +/- 8.71, N = 3SE +/- 22.70, N = 3SE +/- 1.54, N = 11SE +/- 9.42, N = 15SE +/- 0.84, N = 3SE +/- 0.43, N = 3SE +/- 6.45, N = 4SE +/- 1.12, N = 3SE +/- 0.93, N = 3SE +/- 4.95, N = 15SE +/- 0.35, N = 3SE +/- 1.12, N = 3SE +/- 14.92, N = 3657.36458.05417.31388.52513.75362.44355.75336.81500.32432.59392.51349.95490.12452.46369.78369.20396.46386.43377.69343.45386.14386.08373.21297.901. (CC) gcc options: -O2 -lm -pthread -lmpi

IOR

Write Test

RAID0RAID10SingleRAID1RAID6RAID5OpenBenchmarking.orgMB/s, More Is BetterIOR 3.2.1Write TestBtrfsXFSEXT4F2FS50100150200250SE +/- 1.56, N = 3SE +/- 0.44, N = 3SE +/- 0.95, N = 3SE +/- 1.13, N = 3SE +/- 1.59, N = 3SE +/- 0.63, N = 3SE +/- 0.70, N = 3SE +/- 0.53, N = 3SE +/- 0.78, N = 3SE +/- 0.91, N = 3SE +/- 1.97, N = 4SE +/- 0.28, N = 3SE +/- 0.62, N = 3SE +/- 8.84, N = 15SE +/- 1.50, N = 3SE +/- 0.37, N = 3SE +/- 1.37, N = 3SE +/- 1.29, N = 15SE +/- 0.62, N = 8SE +/- 0.92, N = 15SE +/- 0.65, N = 3SE +/- 0.78, N = 11SE +/- 0.33, N = 3SE +/- 1.31, N = 15239.16196.62184.17167.74197.11185.84174.5691.34189.28176.71134.7190.00168.79164.77157.6280.51104.6772.9862.1655.0992.7986.0368.9966.491. (CC) gcc options: -O2 -lm -pthread -lmpi


Phoronix Test Suite v10.8.5