Intel Optane 900p RAID Linux File-System Benchmarks

Intel Optane 900p benchmarks with various file-systems on Ubuntu Linux and RAID arrays. Benchmarks by Michael Larabel for a future article..

HTML result view exported from: https://openbenchmarking.org/result/1906195-HV-STORAGETE92.

Intel Optane 900p RAID Linux File-System BenchmarksProcessorMotherboardChipsetMemoryDiskGraphicsAudioMonitorNetworkOSKernelDesktopDisplay ServerOpenGLCompilerFile-SystemScreen ResolutionEXT4 Samsung 970 EVO 500GBEXT4 900pXFS 900pBtrfs 900pF2FS 900pEXT4 RAID0 Dual 900pXFS RAID0 Dual 900pF2FS RAID0 Dual 900pEXT4 RAID1 Dual 900pXFS RAID1 Dual 900pF2FS RAID1 Dual 900pBtrfs RAID0 Dual 900pBtrfs RAID1 Dual 900pZFS 900pZFS RAIDZ Dual 900pAMD Ryzen Threadripper 2990WX 32-Core @ 3.00GHz (32 Cores / 64 Threads)ASUS ROG ZENITH EXTREME (1701 BIOS)AMD 17h4 x 8192 MB DDR4-3200MT/s F4-3200C14-8GFXSamsung SSD 970 EVO 500GBAMD Radeon RX 64 8GB (1590/800MHz)Realtek ALC1220ASUS VP28UIntel I211 + Qualcomm Atheros QCA6174 802.11ac + Wilocity Wil6200 802.11adUbuntu 19.045.2.0-999-generic (x86_64) 20190605GNOME Shell 3.32.0X Server4.5 Mesa 19.0.2 (LLVM 8.0.0)GCC 8.3.0ext43840x2160280GB INTEL SSDPED1D280GA + Samsung SSD 970 EVO 500GBxfsbtrfsf2fs2 x 280GB INTEL SSDPED1D280GA + Samsung SSD 970 EVO 500GBext4Samsung SSD 970 EVO 500GB + 2 x 280GB INTEL SSDPED1D280GAxfs2 x 280GB INTEL SSDPED1D280GA + Samsung SSD 970 EVO 500GBf2fsext4xfsf2fsbtrfsSamsung SSD 970 EVO 500GB + 2 x 280GB INTEL SSDPED1D280GAzfs2 x 280GB INTEL SSDPED1D280GA + Samsung SSD 970 EVO 500GBOpenBenchmarking.orgCompiler Details- --build=x86_64-linux-gnu --disable-vtable-verify --disable-werror --enable-bootstrap --enable-checking=release --enable-clocale=gnu --enable-default-pie --enable-gnu-unique-object --enable-languages=c,ada,c++,go,brig,d,fortran,objc,obj-c++ --enable-libmpx --enable-libstdcxx-debug --enable-libstdcxx-time=yes --enable-multiarch --enable-multilib --enable-nls --enable-objc-gc=auto --enable-offload-targets=nvptx-none --enable-plugin --enable-shared --enable-threads=posix --host=x86_64-linux-gnu --program-prefix=x86_64-linux-gnu- --target=x86_64-linux-gnu --with-abi=m64 --with-arch-32=i686 --with-default-libstdcxx-abi=new --with-gcc-major-version-only --with-multilib-list=m32,m64,mx32 --with-target-system-zlib --with-tune=generic --without-cuda-driver -vDisk Details- EXT4 Samsung 970 EVO 500GB: NONE / errors=remount-ro,relatime,rw- EXT4 900p: NONE / relatime,rw- XFS 900p: NONE / attr2,inode64,noquota,relatime,rw- Btrfs 900p: NONE / relatime,rw,space_cache,ssd,subvol=/,subvolid=5- F2FS 900p: NONE / acl,active_logs=6,alloc_mode=default,background_gc=on,discard,extent_cache,flush_merge,fsync_mode=posix,inline_data,inline_dentry,inline_xattr,lazytime,mode=adaptive,no_heap,relatime,rw,user_xattr- EXT4 RAID0 Dual 900p: NONE / relatime,rw,stripe=256 / raid0 nvme1n1p1[1] nvme0n1p1[0]- XFS RAID0 Dual 900p: NONE / attr2,inode64,noquota,relatime,rw,sunit=1024,swidth=2048 / raid0 nvme1n1p1[0] nvme2n1p1[1]- F2FS RAID0 Dual 900p: NONE / acl,active_logs=6,alloc_mode=default,background_gc=on,discard,extent_cache,flush_merge,fsync_mode=posix,inline_data,inline_dentry,inline_xattr,lazytime,mode=adaptive,no_heap,relatime,rw,user_xattr / raid0 nvme1n1p1[1] nvme0n1p1[0]- EXT4 RAID1 Dual 900p: NONE / relatime,rw / raid1 nvme2n1p1[1] nvme0n1p1[0]- XFS RAID1 Dual 900p: NONE / attr2,inode64,noquota,relatime,rw / raid1 nvme1n1p1[0] nvme0n1p1[1]- F2FS RAID1 Dual 900p: NONE / acl,active_logs=6,alloc_mode=default,background_gc=on,discard,extent_cache,flush_merge,fsync_mode=posix,inline_data,inline_dentry,inline_xattr,lazytime,mode=adaptive,no_heap,relatime,rw,user_xattr / raid1 nvme2n1p1[1] nvme0n1p1[0]- Btrfs RAID0 Dual 900p: NONE / relatime,rw,space_cache,ssd,subvol=/,subvolid=5 / RAID0- Btrfs RAID1 Dual 900p: NONE / relatime,rw,space_cache,ssd,subvol=/,subvolid=5 / RAID1Processor Details- Scaling Governor: acpi-cpufreq ondemandPython Details- Python 2.7.16 + Python 3.7.3Security Details- l1tf: Not affected + mds: Not affected + meltdown: Not affected + spec_store_bypass: Mitigation of SSB disabled via prctl and seccomp + spectre_v1: Mitigation of __user pointer sanitization + spectre_v2: Mitigation of Full AMD retpoline IBPB: conditional STIBP: disabled RSB fillingDisk Scheduler Details- ZFS 900p, ZFS RAIDZ Dual 900p: NONE

Intel Optane 900p RAID Linux File-System Benchmarksstartup-time: Only Seq Reads - xterm - Default Test Directorystartup-time: Seq Reads + Writes - xterm - Default Test Directorystartup-time: Only Seq Reads - GNOME Terminal - Default Test Directorystartup-time: Only Seq Reads - LibreOffice Writer - Default Test Directorystartup-time: Seq Reads + Writes - GNOME Terminal - Default Test Directorystartup-time: Seq Reads + Writes - LibreOffice Writer - Default Test Directorysqlite: Timed SQLite Insertionsfio: Rand Read - Linux AIO - No - Yes - 2MB - Default Test Directoryfio: Rand Read - Linux AIO - No - Yes - 4KB - Default Test Directoryfio: Rand Read - Linux AIO - No - Yes - 4KB - Default Test Directoryfio: Rand Write - Linux AIO - No - Yes - 2MB - Default Test Directoryfio: Seq Read - Linux AIO - No - Yes - 2MB - Default Test Directoryfio: Seq Read - Linux AIO - No - Yes - 4KB - Default Test Directoryfio: Seq Read - Linux AIO - No - Yes - 4KB - Default Test Directoryfio: Seq Write - Linux AIO - No - Yes - 2MB - Default Test Directoryfs-mark: 1000 Files, 1MB Sizefs-mark: 5000 Files, 1MB Size, 4 Threadsfs-mark: 4000 Files, 32 Sub Dirs, 1MB Sizedbench: 12compilebench: Compilecompilebench: Read Compiled Treepgbench: Mostly RAM - Normal Load - Read Onlypgbench: Mostly RAM - Normal Load - Read Writepgbench: Buffer Test - Normal Load - Read WriteEXT4 Samsung 970 EVO 500GBEXT4 900pXFS 900pBtrfs 900pF2FS 900pEXT4 RAID0 Dual 900pXFS RAID0 Dual 900pF2FS RAID0 Dual 900pEXT4 RAID1 Dual 900pXFS RAID1 Dual 900pF2FS RAID1 Dual 900pBtrfs RAID0 Dual 900pBtrfs RAID1 Dual 900pZFS 900pZFS RAIDZ Dual 900p0.531.002.141.784.323.1235.67336575119233323691030955244667239723626513631416891385114035511452620.520.641.991.353.101.682.0626131106283000215926151135290333215969869844091972158025442828983449230.530.722.001.342.971.71261310502690002161261411552958672162881151588659062683150425748031481432930.670.381.440.661.070.666.69261055714253321452610572146333217174414496611870253414042153899686200880.360.401.030.721.040.682.02261310632720002160261411923049332161790139082268182466169224025320938416570.460.481.941.332.131.422.25520810522694004319521910722740004318819162886043572009154033826332193467000.460.652.231.212.251.42521010282630004317522410552700004318948240395860202757152334528133546453750.280.291.060.600.720.552.1551761007257667432251721033264667432410341789100566912573171334082521284424850.210.590.710.602.061.492.545218953243867216329759922543332163628119867339721978158033611525976417690.210.430.710.601.591.182.2852189442416672164522110082580002164849140885555622741147933848627146400630.210.260.530.410.480.472.795220924236333216352269832516152162742100173365702226167531583318367370590.310.370.960.641.030.726.73517654213863642595206558143000431093222418392240243414852357229816206210.300.300.820.530.800.547.092610540138333212026105601433332156809.371541.43745.301380.252427.321451.73235903.919760.4119777.760.150.200.320.260.340.335.644404234597333963442011843033333802630.27865.30595.93600.701420.74678.869529.103491.0919394.730.140.170.260.220.290.225.924593205524671649443511682993331347498.51736.53482.13607.661322.00669.5915472.394171.4317949.98OpenBenchmarking.org

Application Start-up Time

Background I/O Mix: Only Sequential Reads - Application To Start: xterm - Disk Target: Default Test Directory

OpenBenchmarking.orgsec, Fewer Is BetterApplication Start-up Time 3.4.0Background I/O Mix: Only Sequential Reads - Application To Start: xterm - Disk Target: Default Test DirectoryEXT4 Samsung 970 EVO 500GBEXT4 900pXFS 900pBtrfs 900pF2FS 900pEXT4 RAID0 Dual 900pXFS RAID0 Dual 900pF2FS RAID0 Dual 900pEXT4 RAID1 Dual 900pXFS RAID1 Dual 900pF2FS RAID1 Dual 900pBtrfs RAID0 Dual 900pBtrfs RAID1 Dual 900pZFS 900pZFS RAIDZ Dual 900p0.15080.30160.45240.60320.754SE +/- 0.01, N = 7SE +/- 0.00, N = 3SE +/- 0.01, N = 3SE +/- 0.11, N = 9SE +/- 0.00, N = 12SE +/- 0.01, N = 9SE +/- 0.02, N = 9SE +/- 0.01, N = 12SE +/- 0.00, N = 3SE +/- 0.00, N = 3SE +/- 0.00, N = 12SE +/- 0.00, N = 3SE +/- 0.00, N = 3SE +/- 0.00, N = 3SE +/- 0.00, N = 120.530.520.530.670.360.460.460.280.210.210.210.310.300.150.14

Application Start-up Time

Background I/O Mix: Sequential Reads + Writes - Application To Start: xterm - Disk Target: Default Test Directory

OpenBenchmarking.orgsec, Fewer Is BetterApplication Start-up Time 3.4.0Background I/O Mix: Sequential Reads + Writes - Application To Start: xterm - Disk Target: Default Test DirectoryEXT4 Samsung 970 EVO 500GBEXT4 900pXFS 900pBtrfs 900pF2FS 900pEXT4 RAID0 Dual 900pXFS RAID0 Dual 900pF2FS RAID0 Dual 900pEXT4 RAID1 Dual 900pXFS RAID1 Dual 900pF2FS RAID1 Dual 900pBtrfs RAID0 Dual 900pBtrfs RAID1 Dual 900pZFS 900pZFS RAIDZ Dual 900p0.2250.450.6750.91.125SE +/- 0.04, N = 9SE +/- 0.00, N = 3SE +/- 0.08, N = 9SE +/- 0.04, N = 12SE +/- 0.03, N = 12SE +/- 0.01, N = 12SE +/- 0.00, N = 3SE +/- 0.01, N = 12SE +/- 0.02, N = 12SE +/- 0.02, N = 12SE +/- 0.02, N = 12SE +/- 0.02, N = 12SE +/- 0.00, N = 12SE +/- 0.03, N = 12SE +/- 0.02, N = 121.000.640.720.380.400.480.650.290.590.430.260.370.300.200.17

Application Start-up Time

Background I/O Mix: Only Sequential Reads - Application To Start: GNOME Terminal - Disk Target: Default Test Directory

OpenBenchmarking.orgsec, Fewer Is BetterApplication Start-up Time 3.4.0Background I/O Mix: Only Sequential Reads - Application To Start: GNOME Terminal - Disk Target: Default Test DirectoryEXT4 Samsung 970 EVO 500GBEXT4 900pXFS 900pBtrfs 900pF2FS 900pEXT4 RAID0 Dual 900pXFS RAID0 Dual 900pF2FS RAID0 Dual 900pEXT4 RAID1 Dual 900pXFS RAID1 Dual 900pF2FS RAID1 Dual 900pBtrfs RAID0 Dual 900pBtrfs RAID1 Dual 900pZFS 900pZFS RAIDZ Dual 900p0.50181.00361.50542.00722.509SE +/- 0.03, N = 4SE +/- 0.00, N = 3SE +/- 0.02, N = 3SE +/- 0.17, N = 12SE +/- 0.01, N = 3SE +/- 0.07, N = 9SE +/- 0.17, N = 9SE +/- 0.08, N = 9SE +/- 0.00, N = 3SE +/- 0.01, N = 3SE +/- 0.01, N = 3SE +/- 0.04, N = 12SE +/- 0.05, N = 9SE +/- 0.00, N = 3SE +/- 0.00, N = 122.141.992.001.441.031.942.231.060.710.710.530.960.820.320.26

Application Start-up Time

Background I/O Mix: Only Sequential Reads - Application To Start: LibreOffice Writer - Disk Target: Default Test Directory

OpenBenchmarking.orgsec, Fewer Is BetterApplication Start-up Time 3.4.0Background I/O Mix: Only Sequential Reads - Application To Start: LibreOffice Writer - Disk Target: Default Test DirectoryEXT4 Samsung 970 EVO 500GBEXT4 900pXFS 900pBtrfs 900pF2FS 900pEXT4 RAID0 Dual 900pXFS RAID0 Dual 900pF2FS RAID0 Dual 900pEXT4 RAID1 Dual 900pXFS RAID1 Dual 900pF2FS RAID1 Dual 900pBtrfs RAID0 Dual 900pBtrfs RAID1 Dual 900pZFS 900pZFS RAIDZ Dual 900p0.40050.8011.20151.6022.0025SE +/- 0.01, N = 3SE +/- 0.01, N = 3SE +/- 0.01, N = 8SE +/- 0.02, N = 12SE +/- 0.01, N = 4SE +/- 0.11, N = 12SE +/- 0.07, N = 12SE +/- 0.05, N = 12SE +/- 0.00, N = 3SE +/- 0.01, N = 3SE +/- 0.00, N = 9SE +/- 0.06, N = 12SE +/- 0.00, N = 12SE +/- 0.01, N = 12SE +/- 0.00, N = 121.781.351.340.660.721.331.210.600.600.600.410.640.530.260.22

Application Start-up Time

Background I/O Mix: Sequential Reads + Writes - Application To Start: GNOME Terminal - Disk Target: Default Test Directory

OpenBenchmarking.orgsec, Fewer Is BetterApplication Start-up Time 3.4.0Background I/O Mix: Sequential Reads + Writes - Application To Start: GNOME Terminal - Disk Target: Default Test DirectoryEXT4 Samsung 970 EVO 500GBEXT4 900pXFS 900pBtrfs 900pF2FS 900pEXT4 RAID0 Dual 900pXFS RAID0 Dual 900pF2FS RAID0 Dual 900pEXT4 RAID1 Dual 900pXFS RAID1 Dual 900pF2FS RAID1 Dual 900pBtrfs RAID0 Dual 900pBtrfs RAID1 Dual 900pZFS 900pZFS RAIDZ Dual 900p0.9721.9442.9163.8884.86SE +/- 0.11, N = 12SE +/- 0.03, N = 3SE +/- 0.22, N = 9SE +/- 0.08, N = 12SE +/- 0.05, N = 12SE +/- 0.15, N = 12SE +/- 0.13, N = 9SE +/- 0.02, N = 12SE +/- 0.11, N = 9SE +/- 0.02, N = 3SE +/- 0.01, N = 3SE +/- 0.09, N = 12SE +/- 0.03, N = 11SE +/- 0.03, N = 9SE +/- 0.03, N = 94.323.102.971.071.042.132.250.722.061.590.481.030.800.340.29

Application Start-up Time

Background I/O Mix: Sequential Reads + Writes - Application To Start: LibreOffice Writer - Disk Target: Default Test Directory

OpenBenchmarking.orgsec, Fewer Is BetterApplication Start-up Time 3.4.0Background I/O Mix: Sequential Reads + Writes - Application To Start: LibreOffice Writer - Disk Target: Default Test DirectoryEXT4 Samsung 970 EVO 500GBEXT4 900pXFS 900pBtrfs 900pF2FS 900pEXT4 RAID0 Dual 900pXFS RAID0 Dual 900pF2FS RAID0 Dual 900pEXT4 RAID1 Dual 900pXFS RAID1 Dual 900pF2FS RAID1 Dual 900pBtrfs RAID0 Dual 900pBtrfs RAID1 Dual 900pZFS 900pZFS RAIDZ Dual 900p0.7021.4042.1062.8083.51SE +/- 0.09, N = 9SE +/- 0.03, N = 12SE +/- 0.05, N = 12SE +/- 0.03, N = 9SE +/- 0.03, N = 10SE +/- 0.06, N = 12SE +/- 0.04, N = 12SE +/- 0.03, N = 12SE +/- 0.07, N = 12SE +/- 0.02, N = 12SE +/- 0.02, N = 12SE +/- 0.06, N = 12SE +/- 0.00, N = 3SE +/- 0.03, N = 12SE +/- 0.01, N = 123.121.681.710.660.681.421.420.551.491.180.470.720.540.330.22

SQLite

Timed SQLite Insertions

OpenBenchmarking.orgSeconds, Fewer Is BetterSQLite 3.22Timed SQLite InsertionsEXT4 Samsung 970 EVO 500GBEXT4 900pBtrfs 900pF2FS 900pEXT4 RAID0 Dual 900pF2FS RAID0 Dual 900pEXT4 RAID1 Dual 900pXFS RAID1 Dual 900pF2FS RAID1 Dual 900pBtrfs RAID0 Dual 900pBtrfs RAID1 Dual 900pZFS 900pZFS RAIDZ Dual 900p816243240SE +/- 0.03, N = 3SE +/- 0.01, N = 3SE +/- 0.11, N = 3SE +/- 0.00, N = 2SE +/- 0.01, N = 3SE +/- 0.01, N = 3SE +/- 0.02, N = 3SE +/- 0.00, N = 3SE +/- 0.01, N = 3SE +/- 0.09, N = 15SE +/- 0.07, N = 8SE +/- 0.02, N = 3SE +/- 0.10, N = 335.672.066.692.022.252.152.542.282.796.737.095.645.921. (CC) gcc options: -O2 -lz -ldl -lpthread

Flexible IO Tester

Type: Random Read - IO Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 2MB - Disk Target: Default Test Directory

OpenBenchmarking.orgMB/s, More Is BetterFlexible IO Tester 3.1Type: Random Read - IO Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 2MB - Disk Target: Default Test DirectoryEXT4 Samsung 970 EVO 500GBEXT4 900pXFS 900pBtrfs 900pF2FS 900pEXT4 RAID0 Dual 900pXFS RAID0 Dual 900pF2FS RAID0 Dual 900pEXT4 RAID1 Dual 900pXFS RAID1 Dual 900pF2FS RAID1 Dual 900pBtrfs RAID0 Dual 900pBtrfs RAID1 Dual 900pZFS 900pZFS RAIDZ Dual 900p11002200330044005500SE +/- 17.82, N = 3SE +/- 0.33, N = 3SE +/- 1.00, N = 3SE +/- 2.73, N = 3SE +/- 1.33, N = 3SE +/- 0.88, N = 3SE +/- 14.24, N = 3SE +/- 0.88, N = 3SE +/- 39.01, N = 3SE +/- 4.37, N = 3336526132613261026135208521051765218521852205176261044044593-lnuma -libverbs-lnuma -libverbs-lnuma -libverbs-lnuma -libverbs-lnuma -libverbs-lnuma -libverbs-lnuma -libverbs-lnuma -libverbs-lnuma -libverbs-lnuma -libverbs1. (CC) gcc options: -rdynamic -std=gnu99 -ffast-math -include -O3 -U_FORTIFY_SOURCE -ll -lrt -laio -lz -lm -lpthread -ldl

Flexible IO Tester

Type: Random Read - IO Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 4KB - Disk Target: Default Test Directory

OpenBenchmarking.orgMB/s, More Is BetterFlexible IO Tester 3.1Type: Random Read - IO Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 4KB - Disk Target: Default Test DirectoryEXT4 Samsung 970 EVO 500GBEXT4 900pXFS 900pBtrfs 900pF2FS 900pEXT4 RAID0 Dual 900pXFS RAID0 Dual 900pF2FS RAID0 Dual 900pEXT4 RAID1 Dual 900pXFS RAID1 Dual 900pF2FS RAID1 Dual 900pBtrfs RAID0 Dual 900pBtrfs RAID1 Dual 900pZFS 900pZFS RAIDZ Dual 900p2004006008001000SE +/- 6.17, N = 3SE +/- 7.31, N = 3SE +/- 2.31, N = 3SE +/- 6.07, N = 15SE +/- 15.82, N = 3SE +/- 15.11, N = 15SE +/- 2.40, N = 3SE +/- 10.97, N = 3SE +/- 11.36, N = 15SE +/- 10.74, N = 3SE +/- 3.18, N = 3SE +/- 4.89, N = 11SE +/- 3.76, N = 3SE +/- 0.67, N = 3SE +/- 2.47, N = 15751110610505571063105210281007953944924542540234205-lnuma -libverbs-lnuma -libverbs-lnuma -libverbs-lnuma -libverbs-lnuma -libverbs-lnuma -libverbs-lnuma -libverbs-lnuma -libverbs-lnuma -libverbs-lnuma -libverbs1. (CC) gcc options: -rdynamic -std=gnu99 -ffast-math -include -O3 -U_FORTIFY_SOURCE -ll -lrt -laio -lz -lm -lpthread -ldl

Flexible IO Tester

Type: Random Read - IO Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 4KB - Disk Target: Default Test Directory

OpenBenchmarking.orgIOPS, More Is BetterFlexible IO Tester 3.1Type: Random Read - IO Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 4KB - Disk Target: Default Test DirectoryEXT4 Samsung 970 EVO 500GBEXT4 900pXFS 900pBtrfs 900pF2FS 900pEXT4 RAID0 Dual 900pXFS RAID0 Dual 900pF2FS RAID0 Dual 900pEXT4 RAID1 Dual 900pXFS RAID1 Dual 900pF2FS RAID1 Dual 900pBtrfs RAID0 Dual 900pBtrfs RAID1 Dual 900pZFS 900pZFS RAIDZ Dual 900p60K120K180K240K300KSE +/- 1452.97, N = 3SE +/- 2081.67, N = 3SE +/- 577.35, N = 3SE +/- 1545.71, N = 15SE +/- 4041.45, N = 3SE +/- 3876.42, N = 15SE +/- 577.35, N = 3SE +/- 2962.73, N = 3SE +/- 2893.12, N = 15SE +/- 2603.42, N = 3SE +/- 881.92, N = 3SE +/- 1252.44, N = 11SE +/- 881.92, N = 3SE +/- 185.59, N = 3SE +/- 593.67, N = 151923332830002690001425332720002694002630002576672438672416672363331386361383335973352467-lnuma -libverbs-lnuma -libverbs-lnuma -libverbs-lnuma -libverbs-lnuma -libverbs-lnuma -libverbs-lnuma -libverbs-lnuma -libverbs-lnuma -libverbs-lnuma -libverbs1. (CC) gcc options: -rdynamic -std=gnu99 -ffast-math -include -O3 -U_FORTIFY_SOURCE -ll -lrt -laio -lz -lm -lpthread -ldl

Flexible IO Tester

Type: Random Write - IO Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 2MB - Disk Target: Default Test Directory

OpenBenchmarking.orgMB/s, More Is BetterFlexible IO Tester 3.1Type: Random Write - IO Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 2MB - Disk Target: Default Test DirectoryEXT4 Samsung 970 EVO 500GBEXT4 900pXFS 900pBtrfs 900pF2FS 900pEXT4 RAID0 Dual 900pXFS RAID0 Dual 900pF2FS RAID0 Dual 900pEXT4 RAID1 Dual 900pXFS RAID1 Dual 900pF2FS RAID1 Dual 900pBtrfs RAID0 Dual 900pBtrfs RAID1 Dual 900pZFS 900pZFS RAIDZ Dual 900p9001800270036004500SE +/- 0.88, N = 3SE +/- 2.03, N = 3SE +/- 1.76, N = 3SE +/- 8.69, N = 3SE +/- 2.08, N = 3SE +/- 0.88, N = 3SE +/- 0.58, N = 3SE +/- 1.45, N = 3SE +/- 0.88, N = 3SE +/- 2.03, N = 3SE +/- 2.19, N = 3SE +/- 43.76, N = 8SE +/- 21.79, N = 8SE +/- 61.26, N = 3SE +/- 13.20, N = 3236921592161214521604319431743222163216421634259212039631649-lnuma -libverbs-lnuma -libverbs-lnuma -libverbs-lnuma -libverbs-lnuma -libverbs-lnuma -libverbs-lnuma -libverbs-lnuma -libverbs-lnuma -libverbs-lnuma -libverbs1. (CC) gcc options: -rdynamic -std=gnu99 -ffast-math -include -O3 -U_FORTIFY_SOURCE -ll -lrt -laio -lz -lm -lpthread -ldl

Flexible IO Tester

Type: Sequential Read - IO Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 2MB - Disk Target: Default Test Directory

OpenBenchmarking.orgMB/s, More Is BetterFlexible IO Tester 3.1Type: Sequential Read - IO Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 2MB - Disk Target: Default Test DirectoryEXT4 Samsung 970 EVO 500GBEXT4 900pXFS 900pBtrfs 900pF2FS 900pEXT4 RAID0 Dual 900pXFS RAID0 Dual 900pF2FS RAID0 Dual 900pEXT4 RAID1 Dual 900pXFS RAID1 Dual 900pF2FS RAID1 Dual 900pBtrfs RAID0 Dual 900pBtrfs RAID1 Dual 900pZFS 900pZFS RAIDZ Dual 900p11002200330044005500SE +/- 0.58, N = 3SE +/- 1.15, N = 3SE +/- 0.33, N = 3SE +/- 0.58, N = 3SE +/- 1.00, N = 3SE +/- 1.76, N = 3SE +/- 0.58, N = 3SE +/- 7.75, N = 3SE +/- 10.40, N = 3103026152614261026145219522451722975522152265206261044204435-lnuma -libverbs-lnuma -libverbs-lnuma -libverbs-lnuma -libverbs-lnuma -libverbs-lnuma -libverbs-lnuma -libverbs-lnuma -libverbs-lnuma -libverbs-lnuma -libverbs1. (CC) gcc options: -rdynamic -std=gnu99 -ffast-math -include -O3 -U_FORTIFY_SOURCE -ll -lrt -laio -lz -lm -lpthread -ldl

Flexible IO Tester

Type: Sequential Read - IO Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 4KB - Disk Target: Default Test Directory

OpenBenchmarking.orgMB/s, More Is BetterFlexible IO Tester 3.1Type: Sequential Read - IO Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 4KB - Disk Target: Default Test DirectoryEXT4 Samsung 970 EVO 500GBEXT4 900pXFS 900pBtrfs 900pF2FS 900pEXT4 RAID0 Dual 900pXFS RAID0 Dual 900pF2FS RAID0 Dual 900pEXT4 RAID1 Dual 900pXFS RAID1 Dual 900pF2FS RAID1 Dual 900pBtrfs RAID0 Dual 900pBtrfs RAID1 Dual 900pZFS 900pZFS RAIDZ Dual 900p30060090012001500SE +/- 7.69, N = 3SE +/- 5.21, N = 3SE +/- 19.96, N = 15SE +/- 6.77, N = 6SE +/- 38.25, N = 15SE +/- 4.93, N = 3SE +/- 12.01, N = 3SE +/- 6.69, N = 3SE +/- 14.15, N = 15SE +/- 9.44, N = 13SE +/- 5.00, N = 3SE +/- 7.51, N = 3SE +/- 5.29, N = 3SE +/- 5.36, N = 3955113511555721192107210551033992100898355856011841168-lnuma -libverbs-lnuma -libverbs-lnuma -libverbs-lnuma -libverbs-lnuma -libverbs-lnuma -libverbs-lnuma -libverbs-lnuma -libverbs-lnuma -libverbs-lnuma -libverbs1. (CC) gcc options: -rdynamic -std=gnu99 -ffast-math -include -O3 -U_FORTIFY_SOURCE -ll -lrt -laio -lz -lm -lpthread -ldl

Flexible IO Tester

Type: Sequential Read - IO Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 4KB - Disk Target: Default Test Directory

OpenBenchmarking.orgIOPS, More Is BetterFlexible IO Tester 3.1Type: Sequential Read - IO Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 4KB - Disk Target: Default Test DirectoryEXT4 Samsung 970 EVO 500GBEXT4 900pXFS 900pBtrfs 900pF2FS 900pEXT4 RAID0 Dual 900pXFS RAID0 Dual 900pF2FS RAID0 Dual 900pEXT4 RAID1 Dual 900pXFS RAID1 Dual 900pF2FS RAID1 Dual 900pBtrfs RAID0 Dual 900pBtrfs RAID1 Dual 900pZFS 900pZFS RAIDZ Dual 900p70K140K210K280K350KSE +/- 1763.83, N = 3SE +/- 1452.97, N = 3SE +/- 5123.12, N = 15SE +/- 1837.87, N = 6SE +/- 9791.38, N = 15SE +/- 1154.70, N = 3SE +/- 3179.80, N = 3SE +/- 1763.83, N = 3SE +/- 3620.05, N = 15SE +/- 2474.33, N = 13SE +/- 1000.00, N = 3SE +/- 2027.59, N = 3SE +/- 1452.97, N = 3SE +/- 1333.33, N = 3244667290333295867146333304933274000270000264667254333258000251615143000143333303333299333-lnuma -libverbs-lnuma -libverbs-lnuma -libverbs-lnuma -libverbs-lnuma -libverbs-lnuma -libverbs-lnuma -libverbs-lnuma -libverbs-lnuma -libverbs-lnuma -libverbs1. (CC) gcc options: -rdynamic -std=gnu99 -ffast-math -include -O3 -U_FORTIFY_SOURCE -ll -lrt -laio -lz -lm -lpthread -ldl

Flexible IO Tester

Type: Sequential Write - IO Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 2MB - Disk Target: Default Test Directory

OpenBenchmarking.orgMB/s, More Is BetterFlexible IO Tester 3.1Type: Sequential Write - IO Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 2MB - Disk Target: Default Test DirectoryEXT4 Samsung 970 EVO 500GBEXT4 900pXFS 900pBtrfs 900pF2FS 900pEXT4 RAID0 Dual 900pXFS RAID0 Dual 900pF2FS RAID0 Dual 900pEXT4 RAID1 Dual 900pXFS RAID1 Dual 900pF2FS RAID1 Dual 900pBtrfs RAID0 Dual 900pBtrfs RAID1 Dual 900pZFS 900pZFS RAIDZ Dual 900p9001800270036004500SE +/- 2.67, N = 3SE +/- 1.20, N = 3SE +/- 0.33, N = 3SE +/- 0.58, N = 3SE +/- 2.33, N = 3SE +/- 0.58, N = 3SE +/- 1.53, N = 3SE +/- 2.03, N = 3SE +/- 3.71, N = 3SE +/- 2.00, N = 3SE +/- 55.23, N = 4SE +/- 16.46, N = 3239721592162217121614318431843242163216421624310215638021347-lnuma -libverbs-lnuma -libverbs-lnuma -libverbs-lnuma -libverbs-lnuma -libverbs-lnuma -libverbs-lnuma -libverbs-lnuma -libverbs-lnuma -libverbs-lnuma -libverbs1. (CC) gcc options: -rdynamic -std=gnu99 -ffast-math -include -O3 -U_FORTIFY_SOURCE -ll -lrt -laio -lz -lm -lpthread -ldl

FS-Mark

Test: 1000 Files, 1MB Size

OpenBenchmarking.orgFiles/s, More Is BetterFS-Mark 3.3Test: 1000 Files, 1MB SizeEXT4 Samsung 970 EVO 500GBEXT4 900pXFS 900pBtrfs 900pF2FS 900pEXT4 RAID0 Dual 900pXFS RAID0 Dual 900pF2FS RAID0 Dual 900pEXT4 RAID1 Dual 900pXFS RAID1 Dual 900pF2FS RAID1 Dual 900pBtrfs RAID0 Dual 900pBtrfs RAID1 Dual 900pZFS 900pZFS RAIDZ Dual 900p2004006008001000SE +/- 1.76, N = 3SE +/- 11.91, N = 3SE +/- 3.72, N = 3SE +/- 6.56, N = 3SE +/- 3.49, N = 3SE +/- 8.15, N = 15SE +/- 8.70, N = 3SE +/- 8.24, N = 15SE +/- 5.67, N = 3SE +/- 5.25, N = 3SE +/- 1.55, N = 3SE +/- 6.74, N = 3SE +/- 4.55, N = 3SE +/- 2.84, N = 3SE +/- 6.93, N = 15236.00698.00881.00744.00790.00819.00948.001034.00628.00849.00742.00932.00809.37630.27498.511. (CC) gcc options: -static

FS-Mark

Test: 5000 Files, 1MB Size, 4 Threads

OpenBenchmarking.orgFiles/s, More Is BetterFS-Mark 3.3Test: 5000 Files, 1MB Size, 4 ThreadsEXT4 Samsung 970 EVO 500GBXFS 900pBtrfs 900pF2FS 900pEXT4 RAID0 Dual 900pXFS RAID0 Dual 900pF2FS RAID0 Dual 900pEXT4 RAID1 Dual 900pXFS RAID1 Dual 900pF2FS RAID1 Dual 900pBtrfs RAID0 Dual 900pBtrfs RAID1 Dual 900pZFS 900pZFS RAIDZ Dual 900p5001000150020002500SE +/- 29.37, N = 15SE +/- 12.55, N = 13SE +/- 4.48, N = 3SE +/- 4.24, N = 3SE +/- 13.00, N = 2SE +/- 10.50, N = 3SE +/- 18.85, N = 3SE +/- 19.86, N = 3SE +/- 12.57, N = 15SE +/- 7.25, N = 3SE +/- 28.99, N = 15SE +/- 24.99, N = 3SE +/- 5.71, N = 3SE +/- 3.27, N = 3265.001515.001449.001390.001628.002403.001789.001198.001408.001001.002241.001541.43865.30736.531. (CC) gcc options: -static

FS-Mark

Test: 4000 Files, 32 Sub Dirs, 1MB Size

OpenBenchmarking.orgFiles/s, More Is BetterFS-Mark 3.3Test: 4000 Files, 32 Sub Dirs, 1MB SizeEXT4 Samsung 970 EVO 500GBEXT4 900pXFS 900pBtrfs 900pF2FS 900pEXT4 RAID0 Dual 900pXFS RAID0 Dual 900pF2FS RAID0 Dual 900pEXT4 RAID1 Dual 900pXFS RAID1 Dual 900pF2FS RAID1 Dual 900pBtrfs RAID0 Dual 900pBtrfs RAID1 Dual 900pZFS 900pZFS RAIDZ Dual 900p2004006008001000SE +/- 8.53, N = 15SE +/- 13.00, N = 12SE +/- 4.10, N = 3SE +/- 6.99, N = 15SE +/- 9.81, N = 15SE +/- 12.37, N = 4SE +/- 13.47, N = 3SE +/- 11.73, N = 15SE +/- 5.19, N = 15SE +/- 4.79, N = 3SE +/- 11.75, N = 3SE +/- 10.53, N = 5SE +/- 10.90, N = 4SE +/- 2.05, N = 3SE +/- 3.66, N = 3136.00698.00886.00661.00822.00860.00958.001005.00673.00855.00733.00839.00745.30595.93482.131. (CC) gcc options: -static

Dbench

Client Count: 12

OpenBenchmarking.orgMB/s, More Is BetterDbench 4.0Client Count: 12EXT4 Samsung 970 EVO 500GBEXT4 900pXFS 900pBtrfs 900pF2FS 900pEXT4 RAID0 Dual 900pXFS RAID0 Dual 900pF2FS RAID0 Dual 900pEXT4 RAID1 Dual 900pXFS RAID1 Dual 900pF2FS RAID1 Dual 900pBtrfs RAID0 Dual 900pBtrfs RAID1 Dual 900pZFS 900pZFS RAIDZ Dual 900p15003000450060007500SE +/- 4.00, N = 5SE +/- 3.54, N = 3SE +/- 9.21, N = 3SE +/- 0.79, N = 3SE +/- 3.59, N = 3SE +/- 8.58, N = 3SE +/- 3.35, N = 3SE +/- 14.82, N = 3SE +/- 1.70, N = 3SE +/- 8.97, N = 3SE +/- 6.23, N = 3SE +/- 15.22, N = 3SE +/- 1.39, N = 3SE +/- 1.49, N = 3SE +/- 1.68, N = 3314.004409.005906.001870.006818.004357.006020.006691.003972.005562.006570.002240.001380.25600.70607.661. (CC) gcc options: -lpopt -O2

Compile Bench

Test: Compile

OpenBenchmarking.orgMB/s, More Is BetterCompile Bench 0.6Test: CompileEXT4 Samsung 970 EVO 500GBEXT4 900pXFS 900pBtrfs 900pF2FS 900pEXT4 RAID0 Dual 900pXFS RAID0 Dual 900pF2FS RAID0 Dual 900pEXT4 RAID1 Dual 900pXFS RAID1 Dual 900pF2FS RAID1 Dual 900pBtrfs RAID0 Dual 900pBtrfs RAID1 Dual 900pZFS 900pZFS RAIDZ Dual 900p6001200180024003000SE +/- 29.88, N = 15SE +/- 14.01, N = 3SE +/- 12.65, N = 3SE +/- 20.47, N = 13SE +/- 7.03, N = 3SE +/- 11.99, N = 3SE +/- 19.67, N = 3SE +/- 4.35, N = 3SE +/- 4.77, N = 3SE +/- 38.28, N = 4SE +/- 23.50, N = 3SE +/- 39.37, N = 3SE +/- 27.91, N = 3SE +/- 21.86, N = 3SE +/- 10.65, N = 151689.001972.002683.002534.002466.002009.002757.002573.001978.002741.002226.002434.002427.321420.741322.00

Compile Bench

Test: Read Compiled Tree

OpenBenchmarking.orgMB/s, More Is BetterCompile Bench 0.6Test: Read Compiled TreeEXT4 Samsung 970 EVO 500GBEXT4 900pXFS 900pBtrfs 900pF2FS 900pEXT4 RAID0 Dual 900pXFS RAID0 Dual 900pF2FS RAID0 Dual 900pEXT4 RAID1 Dual 900pXFS RAID1 Dual 900pF2FS RAID1 Dual 900pBtrfs RAID0 Dual 900pBtrfs RAID1 Dual 900pZFS 900pZFS RAIDZ Dual 900p400800120016002000SE +/- 24.73, N = 3SE +/- 22.84, N = 3SE +/- 25.60, N = 3SE +/- 14.69, N = 3SE +/- 4.92, N = 3SE +/- 21.18, N = 3SE +/- 0.92, N = 3SE +/- 23.14, N = 3SE +/- 29.52, N = 3SE +/- 19.40, N = 3SE +/- 6.44, N = 3SE +/- 29.15, N = 3SE +/- 14.48, N = 3SE +/- 38.85, N = 3SE +/- 43.74, N = 31385.001580.001504.001404.001692.001540.001523.001713.001580.001479.001675.001485.001451.73678.86669.59

PostgreSQL pgbench

Scaling: Mostly RAM - Test: Normal Load - Mode: Read Only

OpenBenchmarking.orgTPS, More Is BetterPostgreSQL pgbench 10.3Scaling: Mostly RAM - Test: Normal Load - Mode: Read OnlyEXT4 Samsung 970 EVO 500GBEXT4 900pXFS 900pBtrfs 900pF2FS 900pEXT4 RAID0 Dual 900pXFS RAID0 Dual 900pF2FS RAID0 Dual 900pEXT4 RAID1 Dual 900pXFS RAID1 Dual 900pF2FS RAID1 Dual 900pBtrfs RAID0 Dual 900pBtrfs RAID1 Dual 900pZFS 900pZFS RAIDZ Dual 900p70K140K210K280K350KSE +/- 1522.34, N = 3SE +/- 1834.32, N = 3SE +/- 490.41, N = 3SE +/- 2461.40, N = 9SE +/- 4049.06, N = 3SE +/- 3217.49, N = 3SE +/- 840.52, N = 3SE +/- 1022.47, N = 3SE +/- 2140.96, N = 3SE +/- 1025.24, N = 3SE +/- 5044.01, N = 3SE +/- 1599.92, N = 3SE +/- 625.92, N = 3SE +/- 74.87, N = 3SE +/- 185.08, N = 5114035.00254428.00257480.00215389.00240253.00338263.00345281.00340825.00336115.00338486.00315833.00235722.00235903.919529.1015472.391. (CC) gcc options: -fno-strict-aliasing -fwrapv -O2 -lpgcommon -lpgport -lpq -lpthread -lrt -lcrypt -ldl -lm

PostgreSQL pgbench

Scaling: Mostly RAM - Test: Normal Load - Mode: Read Write

OpenBenchmarking.orgTPS, More Is BetterPostgreSQL pgbench 10.3Scaling: Mostly RAM - Test: Normal Load - Mode: Read WriteEXT4 Samsung 970 EVO 500GBEXT4 900pXFS 900pBtrfs 900pF2FS 900pEXT4 RAID0 Dual 900pXFS RAID0 Dual 900pF2FS RAID0 Dual 900pEXT4 RAID1 Dual 900pXFS RAID1 Dual 900pF2FS RAID1 Dual 900pBtrfs RAID0 Dual 900pBtrfs RAID1 Dual 900pZFS 900pZFS RAIDZ Dual 900p7K14K21K28K35KSE +/- 62.64, N = 9SE +/- 199.16, N = 3SE +/- 277.25, N = 3SE +/- 143.98, N = 3SE +/- 161.12, N = 3SE +/- 352.94, N = 3SE +/- 115.30, N = 3SE +/- 191.47, N = 3SE +/- 35.14, N = 3SE +/- 75.64, N = 3SE +/- 79.41, N = 3SE +/- 107.35, N = 7SE +/- 131.95, N = 3SE +/- 8.49, N = 3SE +/- 19.09, N = 35114.0028983.0031481.009686.0020938.0032193.0033546.0021284.0025976.0027146.0018367.009816.009760.413491.094171.431. (CC) gcc options: -fno-strict-aliasing -fwrapv -O2 -lpgcommon -lpgport -lpq -lpthread -lrt -lcrypt -ldl -lm

PostgreSQL pgbench

Scaling: Buffer Test - Test: Normal Load - Mode: Read Write

OpenBenchmarking.orgTPS, More Is BetterPostgreSQL pgbench 10.3Scaling: Buffer Test - Test: Normal Load - Mode: Read WriteEXT4 Samsung 970 EVO 500GBEXT4 900pXFS 900pBtrfs 900pF2FS 900pEXT4 RAID0 Dual 900pXFS RAID0 Dual 900pF2FS RAID0 Dual 900pEXT4 RAID1 Dual 900pXFS RAID1 Dual 900pF2FS RAID1 Dual 900pBtrfs RAID0 Dual 900pBtrfs RAID1 Dual 900pZFS 900pZFS RAIDZ Dual 900p10K20K30K40K50KSE +/- 51.93, N = 9SE +/- 282.98, N = 3SE +/- 197.57, N = 3SE +/- 7.74, N = 3SE +/- 267.77, N = 3SE +/- 210.27, N = 3SE +/- 391.59, N = 3SE +/- 394.21, N = 3SE +/- 254.89, N = 3SE +/- 180.17, N = 3SE +/- 36.01, N = 3SE +/- 267.89, N = 3SE +/- 156.22, N = 3SE +/- 41.57, N = 3SE +/- 59.86, N = 35262.0044923.0043293.0020088.0041657.0046700.0045375.0042485.0041769.0040063.0037059.0020621.0019777.7619394.7317949.981. (CC) gcc options: -fno-strict-aliasing -fwrapv -O2 -lpgcommon -lpgport -lpq -lpthread -lrt -lcrypt -ldl -lm

Phoronix Test Suite v10.8.4