pts-disk-different-nvmes

AMD Ryzen Threadripper 1900X 8-Core testing with a Gigabyte X399 DESIGNARE EX-CF (F13a BIOS) and NVIDIA Quadro P400 on Debian 11 via the Phoronix Test Suite.

Compare your own system(s) to this result file with the Phoronix Test Suite by running the command: phoronix-test-suite benchmark 2211142-NE-PTSDISKDI73
Jump To Table - Results

View

Do Not Show Noisy Results
Do Not Show Results With Incomplete Data
Do Not Show Results With Little Change/Spread
List Notable Results
Show Result Confidence Charts
Allow Limiting Results To Certain Suite(s)

Statistics

Show Overall Harmonic Mean(s)
Show Overall Geometric Mean
Show Wins / Losses Counts (Pie Chart)
Normalize Results
Remove Outliers Before Calculating Averages

Graph Settings

Force Line Graphs Where Applicable
Convert To Scalar Where Applicable
Prefer Vertical Bar Graphs

Multi-Way Comparison

Condense Multi-Option Tests Into Single Result Graphs

Table

Show Detailed System Result Table

Run Management

Highlight
Result
Toggle/Hide
Result
Result
Identifier
Performance Per
Dollar
Date
Run
  Test
  Duration
ZFS raidz1 4xNVME
November 12 2022
  2 Hours, 25 Minutes
ext4 soft raid5 4xNVME
November 12 2022
  3 Hours, 9 Minutes
ext4 Crucial P5 Plus 1TB NVME
November 13 2022
  3 Hours, 15 Minutes
ZFS raidz1 8xNVME
November 13 2022
  2 Hours, 19 Minutes
ZFS raidz1 8xNVME no Compression
November 13 2022
  2 Hours, 19 Minutes
ZFS mirror 8xNVME
November 13 2022
  2 Hours, 14 Minutes
ext4 soft raid5 8xNVME
November 14 2022
  2 Hours, 44 Minutes
ext4 WD_Black SN770 2TB NVMe
November 14 2022
  2 Hours, 28 Minutes
Invert Behavior (Only Show Selected Data)
  2 Hours, 37 Minutes

Only show results where is faster than
Only show results matching title/arguments (delimit multiple options with a comma):
Do not show results matching title/arguments (delimit multiple options with a comma):


pts-disk-different-nvmesOpenBenchmarking.orgPhoronix Test SuiteAMD Ryzen Threadripper 1900X 8-Core @ 3.80GHz (8 Cores / 16 Threads)Gigabyte X399 DESIGNARE EX-CF (F13a BIOS)AMD 17h64GBSamsung SSD 960 EVO 500GB + 8 x 2000GB Western Digital WD_BLACK SN770 2TB + 1000GB CT1000P5PSSD8NVIDIA Quadro P400NVIDIA GP107GL HD AudioDELL S2340T4 x Intel I350 + Intel 8265 / 8275Debian 115.10.0-19-amd64 (x86_64)GCC 10.2.1 20210110zfsext41920x1080ProcessorMotherboardChipsetMemoryDiskGraphicsAudioMonitorNetworkOSKernelCompilerFile-SystemsScreen ResolutionPts-disk-different-nvmes PerformanceSystem Logs- Transparent Huge Pages: always- --build=x86_64-linux-gnu --disable-vtable-verify --disable-werror --enable-bootstrap --enable-checking=release --enable-clocale=gnu --enable-default-pie --enable-gnu-unique-object --enable-languages=c,ada,c++,go,brig,d,fortran,objc,obj-c++,m2 --enable-libphobos-checking=release --enable-libstdcxx-debug --enable-libstdcxx-time=yes --enable-link-mutex --enable-multiarch --enable-multilib --enable-nls --enable-objc-gc=auto --enable-offload-targets=nvptx-none=/build/gcc-10-Km9U7s/gcc-10-10.2.1/debian/tmp-nvptx/usr,amdgcn-amdhsa=/build/gcc-10-Km9U7s/gcc-10-10.2.1/debian/tmp-gcn/usr,hsa --enable-plugin --enable-shared --enable-threads=posix --host=x86_64-linux-gnu --program-prefix=x86_64-linux-gnu- --target=x86_64-linux-gnu --with-abi=m64 --with-arch-32=i686 --with-build-config=bootstrap-lto-lean --with-default-libstdcxx-abi=new --with-gcc-major-version-only --with-multilib-list=m32,m64,mx32 --with-target-system-zlib=auto --with-tune=generic --without-cuda-driver -v - Scaling Governor: acpi-cpufreq schedutil (Boost: Enabled) - CPU Microcode: 0x8001137- ZFS raidz1 4xNVME, ZFS raidz1 8xNVME, ZFS raidz1 8xNVME no Compression, ZFS mirror 8xNVME: NONE- Python 3.9.2- itlb_multihit: Not affected + l1tf: Not affected + mds: Not affected + meltdown: Not affected + mmio_stale_data: Not affected + retbleed: Mitigation of untrained return thunk; SMT vulnerable + spec_store_bypass: Mitigation of SSB disabled via prctl and seccomp + spectre_v1: Mitigation of usercopy/swapgs barriers and __user pointer sanitization + spectre_v2: Mitigation of Retpolines IBPB: conditional STIBP: disabled RSB filling PBRSB-eIBRS: Not affected + srbds: Not affected + tsx_async_abort: Not affected - ext4 soft raid5 4xNVME: NONE / relatime,rw,stripe=384 / raid5 nvme4n1p1[4] nvme3n1p1[2] nvme2n1p1[1] nvme1n1p1[0] Block Size: 4096- ext4 Crucial P5 Plus 1TB NVME: NONE / relatime,rw / Block Size: 4096- ext4 soft raid5 8xNVME: NONE / relatime,rw,stripe=896 / raid5 nvme9n1[8] nvme8n1[6] nvme7n1[5] nvme6n1[4] nvme4n1[3] nvme3n1[2] nvme2n1[1] nvme1n1[0] Block Size: 4096 - ext4 WD_Black SN770 2TB NVMe: NONE / relatime,rw / Block Size: 4096

ZFS raidz1 4xNVMEext4 soft raid5 4xNVMEext4 Crucial P5 Plus 1TB NVMEZFS raidz1 8xNVMEZFS raidz1 8xNVME no CompressionZFS mirror 8xNVMEext4 soft raid5 8xNVMEext4 WD_Black SN770 2TB NVMeLogarithmic Result OverviewPhoronix Test SuiteDbenchFS-MarkFlexible IO TesterCompile BenchSQLitePostMark

pts-disk-different-nvmesdbench: 12 Clientsdbench: 1 Clientsfs-mark: 4000 Files, 32 Sub Dirs, 1MB Sizefs-mark: 5000 Files, 1MB Size, 4 Threadsfs-mark: 1000 Files, 1MB Size, No Sync/FSyncfs-mark: 1000 Files, 1MB Sizesqlite: 128postmark: Disk Transaction Performancefio: Seq Write - Linux AIO - No - Yes - 2MB - Default Test Directoryfio: Seq Write - Linux AIO - No - Yes - 2MB - Default Test Directorysqlite: 64fio: Rand Write - Linux AIO - No - Yes - 4KB - Default Test Directoryfio: Rand Write - Linux AIO - No - Yes - 4KB - Default Test Directoryfio: Rand Read - Linux AIO - No - Yes - 2MB - Default Test Directoryfio: Seq Read - Linux AIO - No - Yes - 4KB - Default Test Directoryfio: Seq Read - Linux AIO - No - Yes - 4KB - Default Test Directoryfio: Rand Read - Linux AIO - No - Yes - 4KB - Default Test Directoryfio: Rand Read - Linux AIO - No - Yes - 4KB - Default Test Directoryfio: Seq Read - Linux AIO - No - Yes - 2MB - Default Test Directoryfio: Rand Read - Linux AIO - No - Yes - 2MB - Default Test Directoryfio: Seq Read - Linux AIO - No - Yes - 2MB - Default Test Directoryfio: Rand Write - Linux AIO - No - Yes - 2MB - Default Test Directoryfio: Rand Write - Linux AIO - No - Yes - 2MB - Default Test Directoryfio: Seq Write - Linux AIO - No - Yes - 4KB - Default Test Directoryfio: Seq Write - Linux AIO - No - Yes - 4KB - Default Test Directorysqlite: 32compilebench: Compilesqlite: 8sqlite: 1compilebench: Read Compiled Treecompilebench: Initial CreateZFS raidz1 4xNVMEext4 soft raid5 4xNVMEext4 Crucial P5 Plus 1TB NVMEZFS raidz1 8xNVMEZFS raidz1 8xNVME no CompressionZFS mirror 8xNVMEext4 soft raid5 8xNVMEext4 WD_Black SN770 2TB NVMe2626.03410.171684.21632.71359.3646.576.08832891029206537.4484150016221193293331287495671942034424540751225245817133367023.1911398.5711.2677.8521212.73222.042484.85453.170284.7537.91733.3254.080.1455068650130845.7208296732466662213338653066671199670647695811500044931.7931478.0020.0058.8102642.03411.07658.794102.985435.0522.11791.6584.159.89051371614323640.9792246678781765335667131132966712861765353835381623325222800089127.5781483.0715.7678.3672747.01423.672472.42380.354644.71655.61340.7646.778.15332751186238140.3014536717720153263331275574332241999403740051327266117066766526.9991390.7112.8578.3801217.80221.832783.64458.913592.11240.01160.9589.777.04732751220244640.0034820018820813310001294561672191995417039981337268117466768226.8921247.3612.8278.3791694.75206.772730.75435.704658.21711.41338.4581.177.88033181774355536.59868333267330533200012962243338772073661741541783357319733377117.7441408.429.0437.2521454.31216.212330.16423.832286.6570.01782.9264.483.8595137814163650.309750672931322519966777927266710651320051210329906738735.9931474.8022.82910.4262683.76413.463196.68579.240558.21631.51806.6693.890.49052111669334567.6263814001491177191333357414000161617753550355816593325387333151445.1631513.9721.5626.4872738.07420.47OpenBenchmarking.org

Dbench

Dbench is a benchmark designed by the Samba project as a free alternative to netbench, but dbench contains only file-system calls for testing the disk performance. Learn more via the OpenBenchmarking.org test page.

OpenBenchmarking.orgMB/s, More Is BetterDbench 4.012 Clientsext4 WD_Black SN770 2TB NVMeZFS raidz1 8xNVME no CompressionZFS mirror 8xNVMEZFS raidz1 4xNVMEext4 soft raid5 4xNVMEZFS raidz1 8xNVMEext4 soft raid5 8xNVMEext4 Crucial P5 Plus 1TB NVME7001400210028003500SE +/- 2.70, N = 3SE +/- 16.46, N = 3SE +/- 3.38, N = 3SE +/- 5.07, N = 3SE +/- 21.52, N = 3SE +/- 2.03, N = 3SE +/- 16.88, N = 3SE +/- 1.20, N = 33196.682783.642730.752626.032484.852472.422330.16658.791. (CC) gcc options: -lpopt -O2

OpenBenchmarking.orgMB/s, More Is BetterDbench 4.01 Clientsext4 WD_Black SN770 2TB NVMeZFS raidz1 8xNVME no Compressionext4 soft raid5 4xNVMEZFS mirror 8xNVMEext4 soft raid5 8xNVMEZFS raidz1 4xNVMEZFS raidz1 8xNVMEext4 Crucial P5 Plus 1TB NVME130260390520650SE +/- 0.89, N = 3SE +/- 1.41, N = 3SE +/- 0.61, N = 3SE +/- 0.70, N = 3SE +/- 1.02, N = 3SE +/- 0.85, N = 3SE +/- 0.25, N = 3SE +/- 0.03, N = 3579.24458.91453.17435.70423.83410.17380.35102.991. (CC) gcc options: -lpopt -O2

FS-Mark

FS_Mark is designed to test a system's file-system performance. Learn more via the OpenBenchmarking.org test page.

OpenBenchmarking.orgFiles/s, More Is BetterFS-Mark 3.3Test: 4000 Files, 32 Sub Dirs, 1MB SizeZFS raidz1 4xNVMEZFS mirror 8xNVMEZFS raidz1 8xNVMEZFS raidz1 8xNVME no Compressionext4 WD_Black SN770 2TB NVMeext4 Crucial P5 Plus 1TB NVMEext4 soft raid5 8xNVMEext4 soft raid5 4xNVME150300450600750SE +/- 7.90, N = 3SE +/- 5.50, N = 15SE +/- 3.56, N = 3SE +/- 3.57, N = 3SE +/- 63.54, N = 11SE +/- 16.63, N = 12SE +/- 6.52, N = 12SE +/- 2.79, N = 3684.2658.2644.7592.1558.2435.0286.6284.7

OpenBenchmarking.orgFiles/s, More Is BetterFS-Mark 3.3Test: 5000 Files, 1MB Size, 4 ThreadsZFS mirror 8xNVMEZFS raidz1 8xNVMEZFS raidz1 4xNVMEext4 WD_Black SN770 2TB NVMeZFS raidz1 8xNVME no Compressionext4 soft raid5 8xNVMEext4 soft raid5 4xNVMEext4 Crucial P5 Plus 1TB NVME400800120016002000SE +/- 2.79, N = 3SE +/- 6.18, N = 3SE +/- 4.95, N = 3SE +/- 8.47, N = 3SE +/- 2.97, N = 3SE +/- 4.70, N = 3SE +/- 4.55, N = 12SE +/- 125.73, N = 91711.41655.61632.71631.51240.0570.0537.9522.1

OpenBenchmarking.orgFiles/s, More Is BetterFS-Mark 3.3Test: 1000 Files, 1MB Size, No Sync/FSyncext4 WD_Black SN770 2TB NVMeext4 Crucial P5 Plus 1TB NVMEext4 soft raid5 8xNVMEext4 soft raid5 4xNVMEZFS raidz1 4xNVMEZFS raidz1 8xNVMEZFS mirror 8xNVMEZFS raidz1 8xNVME no Compression400800120016002000SE +/- 2.57, N = 3SE +/- 16.52, N = 3SE +/- 11.02, N = 3SE +/- 19.84, N = 4SE +/- 12.01, N = 15SE +/- 10.88, N = 15SE +/- 7.95, N = 3SE +/- 13.85, N = 41806.61791.61782.91733.31359.31340.71338.41160.9

OpenBenchmarking.orgFiles/s, More Is BetterFS-Mark 3.3Test: 1000 Files, 1MB Sizeext4 WD_Black SN770 2TB NVMeZFS raidz1 8xNVMEZFS raidz1 4xNVMEZFS raidz1 8xNVME no Compressionext4 Crucial P5 Plus 1TB NVMEZFS mirror 8xNVMEext4 soft raid5 8xNVMEext4 soft raid5 4xNVME150300450600750SE +/- 9.94, N = 3SE +/- 1.87, N = 3SE +/- 5.49, N = 15SE +/- 6.13, N = 3SE +/- 13.64, N = 12SE +/- 4.66, N = 3SE +/- 0.85, N = 3SE +/- 9.50, N = 15693.8646.7646.5589.7584.1581.1264.4254.0

SQLite

This is a simple benchmark of SQLite. At present this test profile just measures the time to perform a pre-defined number of insertions on an indexed database. Learn more via the OpenBenchmarking.org test page.

OpenBenchmarking.orgSeconds, Fewer Is BetterSQLite 3.30.1Threads / Copies: 128ext4 Crucial P5 Plus 1TB NVMEZFS raidz1 4xNVMEZFS raidz1 8xNVME no CompressionZFS mirror 8xNVMEZFS raidz1 8xNVMEext4 soft raid5 4xNVMEext4 soft raid5 8xNVMEext4 WD_Black SN770 2TB NVMe20406080100SE +/- 0.03, N = 3SE +/- 0.10, N = 3SE +/- 0.27, N = 3SE +/- 0.16, N = 3SE +/- 0.10, N = 3SE +/- 0.28, N = 3SE +/- 0.16, N = 3SE +/- 0.01, N = 359.8976.0977.0577.8878.1580.1583.8690.491. (CC) gcc options: -O2 -lz -lm -ldl -lpthread

PostMark

This is a test of NetApp's PostMark benchmark designed to simulate small-file testing similar to the tasks endured by web and mail servers. This test profile will set PostMark to perform 25,000 transactions with 500 files simultaneously with the file sizes ranging between 5 and 512 kilobytes. Learn more via the OpenBenchmarking.org test page.

OpenBenchmarking.orgTPS, More Is BetterPostMark 1.51Disk Transaction Performanceext4 WD_Black SN770 2TB NVMeext4 soft raid5 8xNVMEext4 Crucial P5 Plus 1TB NVMEext4 soft raid5 4xNVMEZFS mirror 8xNVMEZFS raidz1 4xNVMEZFS raidz1 8xNVME no CompressionZFS raidz1 8xNVME11002200330044005500SE +/- 58.25, N = 5SE +/- 35.33, N = 3SE +/- 35.33, N = 3SE +/- 34.00, N = 3SE +/- 14.67, N = 3SE +/- 14.33, N = 3SE +/- 29.00, N = 3521151375137506833183289327532751. (CC) gcc options: -O3

Flexible IO Tester

FIO, the Flexible I/O Tester, is an advanced Linux disk benchmark supporting multiple I/O engines and a wealth of options. FIO was written by Jens Axboe for testing of the Linux I/O subsystem and schedulers. Learn more via the OpenBenchmarking.org test page.

OpenBenchmarking.orgIOPS, More Is BetterFlexible IO Tester 3.29Type: Sequential Write - IO Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 2MB - Disk Target: Default Test DirectoryZFS mirror 8xNVMEext4 WD_Black SN770 2TB NVMeext4 Crucial P5 Plus 1TB NVMEZFS raidz1 8xNVME no CompressionZFS raidz1 8xNVMEZFS raidz1 4xNVMEext4 soft raid5 8xNVMEext4 soft raid5 4xNVME400800120016002000SE +/- 6.01, N = 3SE +/- 0.67, N = 3SE +/- 2.85, N = 3SE +/- 11.98, N = 15SE +/- 2.33, N = 3SE +/- 2.91, N = 3SE +/- 0.88, N = 31774166916141220118610298146501. (CC) gcc options: -rdynamic -lrt -lz -lpthread -lm -ldl -laio -std=gnu99 -ffast-math -include -O3 -fcommon -U_FORTIFY_SOURCE -march=native

OpenBenchmarking.orgMB/s, More Is BetterFlexible IO Tester 3.29Type: Sequential Write - IO Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 2MB - Disk Target: Default Test DirectoryZFS mirror 8xNVMEext4 WD_Black SN770 2TB NVMeext4 Crucial P5 Plus 1TB NVMEZFS raidz1 8xNVME no CompressionZFS raidz1 8xNVMEZFS raidz1 4xNVMEext4 soft raid5 8xNVMEext4 soft raid5 4xNVME8001600240032004000SE +/- 11.93, N = 3SE +/- 0.67, N = 3SE +/- 5.70, N = 3SE +/- 23.91, N = 15SE +/- 4.67, N = 3SE +/- 6.36, N = 3SE +/- 0.67, N = 3SE +/- 1.73, N = 3355533453236244623812065163613081. (CC) gcc options: -rdynamic -lrt -lz -lpthread -lm -ldl -laio -std=gnu99 -ffast-math -include -O3 -fcommon -U_FORTIFY_SOURCE -march=native

SQLite

This is a simple benchmark of SQLite. At present this test profile just measures the time to perform a pre-defined number of insertions on an indexed database. Learn more via the OpenBenchmarking.org test page.

OpenBenchmarking.orgSeconds, Fewer Is BetterSQLite 3.30.1Threads / Copies: 64ZFS mirror 8xNVMEZFS raidz1 4xNVMEZFS raidz1 8xNVME no CompressionZFS raidz1 8xNVMEext4 Crucial P5 Plus 1TB NVMEext4 soft raid5 4xNVMEext4 soft raid5 8xNVMEext4 WD_Black SN770 2TB NVMe1530456075SE +/- 0.08, N = 3SE +/- 0.01, N = 3SE +/- 0.01, N = 3SE +/- 0.05, N = 3SE +/- 0.01, N = 3SE +/- 0.13, N = 3SE +/- 0.21, N = 3SE +/- 0.07, N = 336.6037.4540.0040.3040.9845.7250.3167.631. (CC) gcc options: -O2 -lz -lm -ldl -lpthread

Flexible IO Tester

FIO, the Flexible I/O Tester, is an advanced Linux disk benchmark supporting multiple I/O engines and a wealth of options. FIO was written by Jens Axboe for testing of the Linux I/O subsystem and schedulers. Learn more via the OpenBenchmarking.org test page.

OpenBenchmarking.orgIOPS, More Is BetterFlexible IO Tester 3.29Type: Random Write - IO Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 4KB - Disk Target: Default Test Directoryext4 WD_Black SN770 2TB NVMeext4 Crucial P5 Plus 1TB NVMEext4 soft raid5 4xNVMEext4 soft raid5 8xNVMEZFS mirror 8xNVMEZFS raidz1 8xNVME no CompressionZFS raidz1 8xNVMEZFS raidz1 4xNVME80K160K240K320K400KSE +/- 3841.87, N = 5SE +/- 666.67, N = 3SE +/- 569.60, N = 3SE +/- 88.19, N = 3SE +/- 88.19, N = 3SE +/- 57.74, N = 3SE +/- 33.33, N = 3SE +/- 57.74, N = 33814002246678296775067683334820045367415001. (CC) gcc options: -rdynamic -lrt -lz -lpthread -lm -ldl -laio -std=gnu99 -ffast-math -include -O3 -fcommon -U_FORTIFY_SOURCE -march=native

OpenBenchmarking.orgMB/s, More Is BetterFlexible IO Tester 3.29Type: Random Write - IO Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 4KB - Disk Target: Default Test Directoryext4 WD_Black SN770 2TB NVMeext4 Crucial P5 Plus 1TB NVMEext4 soft raid5 4xNVMEext4 soft raid5 8xNVMEZFS mirror 8xNVMEZFS raidz1 8xNVME no CompressionZFS raidz1 8xNVMEZFS raidz1 4xNVME30060090012001500SE +/- 14.63, N = 5SE +/- 2.67, N = 3SE +/- 2.00, N = 3SE +/- 0.33, N = 314918783242932671881771621. (CC) gcc options: -rdynamic -lrt -lz -lpthread -lm -ldl -laio -std=gnu99 -ffast-math -include -O3 -fcommon -U_FORTIFY_SOURCE -march=native

OpenBenchmarking.orgIOPS, More Is BetterFlexible IO Tester 3.29Type: Random Read - IO Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 2MB - Disk Target: Default Test Directoryext4 soft raid5 8xNVMEext4 soft raid5 4xNVMEZFS mirror 8xNVMEZFS raidz1 4xNVMEZFS raidz1 8xNVME no CompressionZFS raidz1 8xNVMEext4 WD_Black SN770 2TB NVMeext4 Crucial P5 Plus 1TB NVME3K6K9K12K15KSE +/- 143.61, N = 4SE +/- 14.99, N = 3SE +/- 12.06, N = 3SE +/- 29.49, N = 3SE +/- 20.66, N = 3SE +/- 6.57, N = 3SE +/- 3.67, N = 31322566663305211920812015177117651. (CC) gcc options: -rdynamic -lrt -lz -lpthread -lm -ldl -laio -std=gnu99 -ffast-math -include -O3 -fcommon -U_FORTIFY_SOURCE -march=native

OpenBenchmarking.orgIOPS, More Is BetterFlexible IO Tester 3.29Type: Sequential Read - IO Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 4KB - Disk Target: Default Test Directoryext4 Crucial P5 Plus 1TB NVMEZFS mirror 8xNVMEZFS raidz1 8xNVME no CompressionZFS raidz1 4xNVMEZFS raidz1 8xNVMEext4 soft raid5 4xNVMEext4 soft raid5 8xNVMEext4 WD_Black SN770 2TB NVMe70K140K210K280K350KSE +/- 4666.67, N = 3SE +/- 2516.61, N = 3SE +/- 1000.00, N = 3SE +/- 1855.92, N = 3SE +/- 3382.96, N = 3SE +/- 2728.45, N = 3SE +/- 333.33, N = 3SE +/- 202.76, N = 3335667332000331000329333326333221333199667913331. (CC) gcc options: -rdynamic -lrt -lz -lpthread -lm -ldl -laio -std=gnu99 -ffast-math -include -O3 -fcommon -U_FORTIFY_SOURCE -march=native

OpenBenchmarking.orgMB/s, More Is BetterFlexible IO Tester 3.29Type: Sequential Read - IO Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 4KB - Disk Target: Default Test Directoryext4 Crucial P5 Plus 1TB NVMEZFS mirror 8xNVMEZFS raidz1 8xNVME no CompressionZFS raidz1 4xNVMEZFS raidz1 8xNVMEext4 soft raid5 4xNVMEext4 soft raid5 8xNVMEext4 WD_Black SN770 2TB NVMe30060090012001500SE +/- 17.89, N = 3SE +/- 9.94, N = 3SE +/- 4.84, N = 3SE +/- 7.09, N = 3SE +/- 13.25, N = 3SE +/- 11.46, N = 3SE +/- 0.67, N = 3131112961294128712758657793571. (CC) gcc options: -rdynamic -lrt -lz -lpthread -lm -ldl -laio -std=gnu99 -ffast-math -include -O3 -fcommon -U_FORTIFY_SOURCE -march=native

OpenBenchmarking.orgIOPS, More Is BetterFlexible IO Tester 3.29Type: Random Read - IO Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 4KB - Disk Target: Default Test Directoryext4 WD_Black SN770 2TB NVMeext4 Crucial P5 Plus 1TB NVMEext4 soft raid5 4xNVMEext4 soft raid5 8xNVMEZFS mirror 8xNVMEZFS raidz1 8xNVMEZFS raidz1 8xNVME no CompressionZFS raidz1 4xNVME90K180K270K360K450KSE +/- 2886.75, N = 3SE +/- 881.92, N = 3SE +/- 1333.33, N = 3SE +/- 333.33, N = 3SE +/- 666.67, N = 3SE +/- 176.38, N = 3SE +/- 633.33, N = 3SE +/- 185.59, N = 34140003296673066672726672243335743356167495671. (CC) gcc options: -rdynamic -lrt -lz -lpthread -lm -ldl -laio -std=gnu99 -ffast-math -include -O3 -fcommon -U_FORTIFY_SOURCE -march=native

OpenBenchmarking.orgMB/s, More Is BetterFlexible IO Tester 3.29Type: Random Read - IO Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 4KB - Disk Target: Default Test Directoryext4 WD_Black SN770 2TB NVMeext4 Crucial P5 Plus 1TB NVMEext4 soft raid5 4xNVMEext4 soft raid5 8xNVMEZFS mirror 8xNVMEZFS raidz1 8xNVMEZFS raidz1 8xNVME no CompressionZFS raidz1 4xNVME30060090012001500SE +/- 10.97, N = 3SE +/- 3.48, N = 3SE +/- 5.17, N = 3SE +/- 2.03, N = 3SE +/- 1.86, N = 3SE +/- 0.67, N = 3SE +/- 2.67, N = 3SE +/- 0.88, N = 316161286119910658772242191941. (CC) gcc options: -rdynamic -lrt -lz -lpthread -lm -ldl -laio -std=gnu99 -ffast-math -include -O3 -fcommon -U_FORTIFY_SOURCE -march=native

OpenBenchmarking.orgIOPS, More Is BetterFlexible IO Tester 3.29Type: Sequential Read - IO Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 2MB - Disk Target: Default Test Directoryext4 soft raid5 8xNVMEext4 soft raid5 4xNVMEZFS mirror 8xNVMEZFS raidz1 4xNVMEZFS raidz1 8xNVMEZFS raidz1 8xNVME no Compressionext4 WD_Black SN770 2TB NVMeext4 Crucial P5 Plus 1TB NVME3K6K9K12K15KSE +/- 57.74, N = 3SE +/- 3.61, N = 3SE +/- 18.84, N = 3SE +/- 18.75, N = 3SE +/- 25.32, N = 3SE +/- 23.68, N = 31320067062073203419991995177517651. (CC) gcc options: -rdynamic -lrt -lz -lpthread -lm -ldl -laio -std=gnu99 -ffast-math -include -O3 -fcommon -U_FORTIFY_SOURCE -march=native

OpenBenchmarking.orgMB/s, More Is BetterFlexible IO Tester 3.29Type: Random Read - IO Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 2MB - Disk Target: Default Test DirectoryZFS mirror 8xNVMEZFS raidz1 4xNVMEZFS raidz1 8xNVME no CompressionZFS raidz1 8xNVMEext4 WD_Black SN770 2TB NVMeext4 Crucial P5 Plus 1TB NVME14002800420056007000SE +/- 23.92, N = 3SE +/- 58.64, N = 3SE +/- 41.66, N = 3SE +/- 13.37, N = 3SE +/- 7.67, N = 36617424541704037355035381. (CC) gcc options: -rdynamic -lrt -lz -lpthread -lm -ldl -laio -std=gnu99 -ffast-math -include -O3 -fcommon -U_FORTIFY_SOURCE -march=native

OpenBenchmarking.orgMB/s, More Is BetterFlexible IO Tester 3.29Type: Sequential Read - IO Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 2MB - Disk Target: Default Test DirectoryZFS mirror 8xNVMEZFS raidz1 4xNVMEZFS raidz1 8xNVMEZFS raidz1 8xNVME no Compressionext4 WD_Black SN770 2TB NVMeext4 Crucial P5 Plus 1TB NVME9001800270036004500SE +/- 37.36, N = 3SE +/- 37.36, N = 3SE +/- 50.44, N = 3SE +/- 47.58, N = 34154407540053998355835381. (CC) gcc options: -rdynamic -lrt -lz -lpthread -lm -ldl -laio -std=gnu99 -ffast-math -include -O3 -fcommon -U_FORTIFY_SOURCE -march=native

OpenBenchmarking.orgIOPS, More Is BetterFlexible IO Tester 3.29Type: Random Write - IO Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 2MB - Disk Target: Default Test DirectoryZFS mirror 8xNVMEext4 WD_Black SN770 2TB NVMeext4 Crucial P5 Plus 1TB NVMEZFS raidz1 8xNVME no CompressionZFS raidz1 8xNVMEZFS raidz1 4xNVMEext4 soft raid5 8xNVMEext4 soft raid5 4xNVME400800120016002000SE +/- 4.16, N = 3SE +/- 1.20, N = 3SE +/- 4.91, N = 3SE +/- 8.99, N = 3SE +/- 1.45, N = 3SE +/- 1.76, N = 31783165916231337132712255124761. (CC) gcc options: -rdynamic -lrt -lz -lpthread -lm -ldl -laio -std=gnu99 -ffast-math -include -O3 -fcommon -U_FORTIFY_SOURCE -march=native

OpenBenchmarking.orgMB/s, More Is BetterFlexible IO Tester 3.29Type: Random Write - IO Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 2MB - Disk Target: Default Test DirectoryZFS mirror 8xNVMEext4 WD_Black SN770 2TB NVMeext4 Crucial P5 Plus 1TB NVMEZFS raidz1 8xNVME no CompressionZFS raidz1 8xNVMEZFS raidz1 4xNVMEext4 soft raid5 8xNVMEext4 soft raid5 4xNVME8001600240032004000SE +/- 8.25, N = 3SE +/- 2.65, N = 3SE +/- 10.27, N = 3SE +/- 6.39, N = 3SE +/- 17.98, N = 3SE +/- 2.60, N = 3SE +/- 3.53, N = 335733325325226812661245810329581. (CC) gcc options: -rdynamic -lrt -lz -lpthread -lm -ldl -laio -std=gnu99 -ffast-math -include -O3 -fcommon -U_FORTIFY_SOURCE -march=native

OpenBenchmarking.orgIOPS, More Is BetterFlexible IO Tester 3.29Type: Sequential Write - IO Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 4KB - Disk Target: Default Test Directoryext4 WD_Black SN770 2TB NVMeext4 Crucial P5 Plus 1TB NVMEZFS mirror 8xNVMEZFS raidz1 8xNVME no CompressionZFS raidz1 4xNVMEZFS raidz1 8xNVMEext4 soft raid5 4xNVMEext4 soft raid5 8xNVME80K160K240K320K400KSE +/- 2603.42, N = 3SE +/- 1000.00, N = 3SE +/- 333.33, N = 3SE +/- 333.33, N = 3SE +/- 666.67, N = 3SE +/- 333.33, N = 3SE +/- 577.35, N = 3SE +/- 617.34, N = 3387333228000197333174667171333170667115000990671. (CC) gcc options: -rdynamic -lrt -lz -lpthread -lm -ldl -laio -std=gnu99 -ffast-math -include -O3 -fcommon -U_FORTIFY_SOURCE -march=native

OpenBenchmarking.orgMB/s, More Is BetterFlexible IO Tester 3.29Type: Sequential Write - IO Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 4KB - Disk Target: Default Test Directoryext4 WD_Black SN770 2TB NVMeext4 Crucial P5 Plus 1TB NVMEZFS mirror 8xNVMEZFS raidz1 8xNVME no CompressionZFS raidz1 4xNVMEZFS raidz1 8xNVMEext4 soft raid5 4xNVMEext4 soft raid5 8xNVME30060090012001500SE +/- 9.82, N = 3SE +/- 3.93, N = 3SE +/- 0.67, N = 3SE +/- 2.08, N = 3SE +/- 1.86, N = 3SE +/- 1.67, N = 3SE +/- 1.15, N = 3SE +/- 2.33, N = 315148917716826706654493871. (CC) gcc options: -rdynamic -lrt -lz -lpthread -lm -ldl -laio -std=gnu99 -ffast-math -include -O3 -fcommon -U_FORTIFY_SOURCE -march=native

SQLite

This is a simple benchmark of SQLite. At present this test profile just measures the time to perform a pre-defined number of insertions on an indexed database. Learn more via the OpenBenchmarking.org test page.

OpenBenchmarking.orgSeconds, Fewer Is BetterSQLite 3.30.1Threads / Copies: 32ZFS mirror 8xNVMEZFS raidz1 4xNVMEZFS raidz1 8xNVME no CompressionZFS raidz1 8xNVMEext4 Crucial P5 Plus 1TB NVMEext4 soft raid5 4xNVMEext4 soft raid5 8xNVMEext4 WD_Black SN770 2TB NVMe1020304050SE +/- 0.05, N = 3SE +/- 0.08, N = 3SE +/- 0.07, N = 3SE +/- 0.04, N = 3SE +/- 0.00, N = 3SE +/- 0.01, N = 3SE +/- 0.02, N = 3SE +/- 0.09, N = 317.7423.1926.8927.0027.5831.7935.9945.161. (CC) gcc options: -O2 -lz -lm -ldl -lpthread

Compile Bench

Compilebench tries to age a filesystem by simulating some of the disk IO common in creating, compiling, patching, stating and reading kernel trees. It indirectly measures how well filesystems can maintain directory locality as the disk fills up and directories age. This current test is setup to use the makej mode with 10 initial directories Learn more via the OpenBenchmarking.org test page.

OpenBenchmarking.orgMB/s, More Is BetterCompile Bench 0.6Test: Compileext4 WD_Black SN770 2TB NVMeext4 Crucial P5 Plus 1TB NVMEext4 soft raid5 4xNVMEext4 soft raid5 8xNVMEZFS mirror 8xNVMEZFS raidz1 4xNVMEZFS raidz1 8xNVMEZFS raidz1 8xNVME no Compression30060090012001500SE +/- 3.97, N = 3SE +/- 0.00, N = 3SE +/- 11.73, N = 3SE +/- 12.18, N = 3SE +/- 9.96, N = 3SE +/- 6.06, N = 3SE +/- 7.47, N = 3SE +/- 3.89, N = 31513.971483.071478.001474.801408.421398.571390.711247.36

SQLite

This is a simple benchmark of SQLite. At present this test profile just measures the time to perform a pre-defined number of insertions on an indexed database. Learn more via the OpenBenchmarking.org test page.

OpenBenchmarking.orgSeconds, Fewer Is BetterSQLite 3.30.1Threads / Copies: 8ZFS mirror 8xNVMEZFS raidz1 4xNVMEZFS raidz1 8xNVME no CompressionZFS raidz1 8xNVMEext4 Crucial P5 Plus 1TB NVMEext4 soft raid5 4xNVMEext4 WD_Black SN770 2TB NVMeext4 soft raid5 8xNVME510152025SE +/- 0.057, N = 3SE +/- 0.079, N = 3SE +/- 0.170, N = 3SE +/- 0.174, N = 3SE +/- 0.009, N = 3SE +/- 0.047, N = 3SE +/- 0.045, N = 3SE +/- 0.168, N = 159.04311.26712.82712.85715.76720.00521.56222.8291. (CC) gcc options: -O2 -lz -lm -ldl -lpthread

OpenBenchmarking.orgSeconds, Fewer Is BetterSQLite 3.30.1Threads / Copies: 1ext4 WD_Black SN770 2TB NVMeZFS mirror 8xNVMEZFS raidz1 4xNVMEext4 Crucial P5 Plus 1TB NVMEZFS raidz1 8xNVME no CompressionZFS raidz1 8xNVMEext4 soft raid5 4xNVMEext4 soft raid5 8xNVME3691215SE +/- 0.057, N = 3SE +/- 0.055, N = 3SE +/- 0.071, N = 7SE +/- 0.067, N = 3SE +/- 0.104, N = 3SE +/- 0.096, N = 3SE +/- 0.069, N = 10SE +/- 0.093, N = 36.4877.2527.8528.3678.3798.3808.81010.4261. (CC) gcc options: -O2 -lz -lm -ldl -lpthread

Compile Bench

Compilebench tries to age a filesystem by simulating some of the disk IO common in creating, compiling, patching, stating and reading kernel trees. It indirectly measures how well filesystems can maintain directory locality as the disk fills up and directories age. This current test is setup to use the makej mode with 10 initial directories Learn more via the OpenBenchmarking.org test page.

OpenBenchmarking.orgMB/s, More Is BetterCompile Bench 0.6Test: Read Compiled Treeext4 Crucial P5 Plus 1TB NVMEext4 WD_Black SN770 2TB NVMeext4 soft raid5 8xNVMEext4 soft raid5 4xNVMEZFS raidz1 8xNVME no CompressionZFS mirror 8xNVMEZFS raidz1 8xNVMEZFS raidz1 4xNVME6001200180024003000SE +/- 8.94, N = 3SE +/- 16.46, N = 3SE +/- 15.05, N = 3SE +/- 42.24, N = 3SE +/- 19.98, N = 3SE +/- 12.22, N = 3SE +/- 8.89, N = 3SE +/- 8.00, N = 32747.012738.072683.762642.031694.751454.311217.801212.73

OpenBenchmarking.orgMB/s, More Is BetterCompile Bench 0.6Test: Initial Createext4 Crucial P5 Plus 1TB NVMEext4 WD_Black SN770 2TB NVMeext4 soft raid5 8xNVMEext4 soft raid5 4xNVMEZFS raidz1 4xNVMEZFS raidz1 8xNVMEZFS mirror 8xNVMEZFS raidz1 8xNVME no Compression90180270360450SE +/- 1.23, N = 3SE +/- 2.28, N = 3SE +/- 3.20, N = 3SE +/- 0.77, N = 3SE +/- 1.57, N = 3SE +/- 0.50, N = 3SE +/- 1.18, N = 3SE +/- 1.46, N = 3423.67420.47413.46411.07222.04221.83216.21206.77

31 Results Shown

Dbench:
  12 Clients
  1 Clients
FS-Mark:
  4000 Files, 32 Sub Dirs, 1MB Size
  5000 Files, 1MB Size, 4 Threads
  1000 Files, 1MB Size, No Sync/FSync
  1000 Files, 1MB Size
SQLite
PostMark
Flexible IO Tester:
  Seq Write - Linux AIO - No - Yes - 2MB - Default Test Directory:
    IOPS
    MB/s
SQLite
Flexible IO Tester:
  Rand Write - Linux AIO - No - Yes - 4KB - Default Test Directory:
    IOPS
    MB/s
  Rand Read - Linux AIO - No - Yes - 2MB - Default Test Directory:
    IOPS
  Seq Read - Linux AIO - No - Yes - 4KB - Default Test Directory:
    IOPS
    MB/s
  Rand Read - Linux AIO - No - Yes - 4KB - Default Test Directory:
    IOPS
    MB/s
  Seq Read - Linux AIO - No - Yes - 2MB - Default Test Directory:
    IOPS
  Rand Read - Linux AIO - No - Yes - 2MB - Default Test Directory:
    MB/s
  Seq Read - Linux AIO - No - Yes - 2MB - Default Test Directory:
    MB/s
  Rand Write - Linux AIO - No - Yes - 2MB - Default Test Directory:
    IOPS
    MB/s
  Seq Write - Linux AIO - No - Yes - 4KB - Default Test Directory:
    IOPS
    MB/s
SQLite
Compile Bench
SQLite:
  8
  1
Compile Bench:
  Read Compiled Tree
  Initial Create