05112023-TCST

PVE-NVME-R10-4DISK-ZFS2.2-1

Compare your own system(s) to this result file with the Phoronix Test Suite by running the command: phoronix-test-suite benchmark 2311132-TCST-231105569
Jump To Table - Results

View

Do Not Show Noisy Results
Do Not Show Results With Incomplete Data
Do Not Show Results With Little Change/Spread
List Notable Results
Show Result Confidence Charts
Allow Limiting Results To Certain Suite(s)

Statistics

Show Overall Harmonic Mean(s)
Show Overall Geometric Mean
Show Wins / Losses Counts (Pie Chart)
Normalize Results
Remove Outliers Before Calculating Averages

Graph Settings

Force Line Graphs Where Applicable
Convert To Scalar Where Applicable
Prefer Vertical Bar Graphs

Multi-Way Comparison

Condense Multi-Option Tests Into Single Result Graphs

Table

Show Detailed System Result Table

Run Management

Highlight
Result
Toggle/Hide
Result
Result
Identifier
Performance Per
Dollar
Date
Run
  Test
  Duration
PVE-NVME-R10-4DISK-ZFS2.1
November 05 2023
  2 Hours, 18 Minutes
PVE-NVME-R10-4DISK-ZFS2.2
November 05 2023
  2 Hours, 18 Minutes
PVE-NVME-R10-4DISK-ZFS2.2-0
November 13 2023
 
PVE-NVME-R10-4DISK-ZFS2.2-1
November 13 2023
  2 Hours, 18 Minutes
Invert Behavior (Only Show Selected Data)
  1 Hour, 44 Minutes

Only show results where is faster than
Only show results matching title/arguments (delimit multiple options with a comma):
Do not show results matching title/arguments (delimit multiple options with a comma):

PVE-NVME-R10-4DISK-ZFS2.1

Processor: AMD EPYC 7543 32-Core (12 Cores), Motherboard: QEMU Standard PC (Q35 + ICH9 2009) (3.20230228-4 BIOS), Chipset: Intel 82G33/G31/P35/P31 + ICH9, Memory: 16GB, Disk: 34GB QEMU HDD, Graphics: bochs-drmdrmfb, Audio: Intel 82801I, Monitor: QEMU Monitor, Network: Red Hat Virtio device

OS: Ubuntu 22.04, Kernel: 5.15.0-86-generic (x86_64), Vulkan: 1.3.238, Compiler: GCC 11.4.0, File-System: ext4, Screen Resolution: 1280x800, System Layer: KVM

Kernel Notes: Transparent Huge Pages: madvise
Compiler Notes: --build=x86_64-linux-gnu --disable-vtable-verify --disable-werror --enable-bootstrap --enable-cet --enable-checking=release --enable-clocale=gnu --enable-default-pie --enable-gnu-unique-object --enable-languages=c,ada,c++,go,brig,d,fortran,objc,obj-c++,m2 --enable-libphobos-checking=release --enable-libstdcxx-debug --enable-libstdcxx-time=yes --enable-link-serialization=2 --enable-multiarch --enable-multilib --enable-nls --enable-objc-gc=auto --enable-offload-targets=nvptx-none=/build/gcc-11-XeT9lY/gcc-11-11.4.0/debian/tmp-nvptx/usr,amdgcn-amdhsa=/build/gcc-11-XeT9lY/gcc-11-11.4.0/debian/tmp-gcn/usr --enable-plugin --enable-shared --enable-threads=posix --host=x86_64-linux-gnu --program-prefix=x86_64-linux-gnu- --target=x86_64-linux-gnu --with-abi=m64 --with-arch-32=i686 --with-build-config=bootstrap-lto-lean --with-default-libstdcxx-abi=new --with-gcc-major-version-only --with-multilib-list=m32,m64,mx32 --with-target-system-zlib=auto --with-tune=generic --without-cuda-driver -v
Disk Notes: NONE / relatime,rw / Block Size: 4096
Processor Notes: CPU Microcode: 0xa00110d
Security Notes: gather_data_sampling: Not affected + itlb_multihit: Not affected + l1tf: Not affected + mds: Not affected + meltdown: Not affected + mmio_stale_data: Not affected + retbleed: Not affected + spec_rstack_overflow: Mitigation of safe RET no microcode + spec_store_bypass: Mitigation of SSB disabled via prctl and seccomp + spectre_v1: Mitigation of usercopy/swapgs barriers and __user pointer sanitization + spectre_v2: Mitigation of Retpolines IBPB: conditional IBRS_FW STIBP: disabled RSB filling PBRSB-eIBRS: Not affected + srbds: Not affected + tsx_async_abort: Not affected

PVE-NVME-R10-4DISK-ZFS2.2

PVE-NVME-R10-4DISK-ZFS2.2-0

PVE-NVME-R10-4DISK-ZFS2.2-1

OS: Ubuntu 22.04, Kernel: 5.15.0-88-generic (x86_64), Vulkan: 1.3.238, Compiler: GCC 11.4.0, File-System: ext4, Screen Resolution: 1280x800, System Layer: KVM

05112023-TCSTOpenBenchmarking.orgPhoronix Test SuiteAMD EPYC 7543 32-Core (12 Cores)QEMU Standard PC (Q35 + ICH9 2009) (3.20230228-4 BIOS)Intel 82G33/G31/P35/P31 + ICH916GB34GB QEMU HDDbochs-drmdrmfbIntel 82801IQEMU MonitorRed Hat Virtio deviceUbuntu 22.045.15.0-86-generic (x86_64)5.15.0-88-generic (x86_64)1.3.238GCC 11.4.0ext41280x800KVMProcessorMotherboardChipsetMemoryDiskGraphicsAudioMonitorNetworkOSKernelsVulkanCompilerFile-SystemScreen ResolutionSystem Layer05112023-TCST BenchmarksSystem Logs- Transparent Huge Pages: madvise- --build=x86_64-linux-gnu --disable-vtable-verify --disable-werror --enable-bootstrap --enable-cet --enable-checking=release --enable-clocale=gnu --enable-default-pie --enable-gnu-unique-object --enable-languages=c,ada,c++,go,brig,d,fortran,objc,obj-c++,m2 --enable-libphobos-checking=release --enable-libstdcxx-debug --enable-libstdcxx-time=yes --enable-link-serialization=2 --enable-multiarch --enable-multilib --enable-nls --enable-objc-gc=auto --enable-offload-targets=nvptx-none=/build/gcc-11-XeT9lY/gcc-11-11.4.0/debian/tmp-nvptx/usr,amdgcn-amdhsa=/build/gcc-11-XeT9lY/gcc-11-11.4.0/debian/tmp-gcn/usr --enable-plugin --enable-shared --enable-threads=posix --host=x86_64-linux-gnu --program-prefix=x86_64-linux-gnu- --target=x86_64-linux-gnu --with-abi=m64 --with-arch-32=i686 --with-build-config=bootstrap-lto-lean --with-default-libstdcxx-abi=new --with-gcc-major-version-only --with-multilib-list=m32,m64,mx32 --with-target-system-zlib=auto --with-tune=generic --without-cuda-driver -v - NONE / relatime,rw / Block Size: 4096- CPU Microcode: 0xa00110d- gather_data_sampling: Not affected + itlb_multihit: Not affected + l1tf: Not affected + mds: Not affected + meltdown: Not affected + mmio_stale_data: Not affected + retbleed: Not affected + spec_rstack_overflow: Mitigation of safe RET no microcode + spec_store_bypass: Mitigation of SSB disabled via prctl and seccomp + spectre_v1: Mitigation of usercopy/swapgs barriers and __user pointer sanitization + spectre_v2: Mitigation of Retpolines IBPB: conditional IBRS_FW STIBP: disabled RSB filling PBRSB-eIBRS: Not affected + srbds: Not affected + tsx_async_abort: Not affected

05112023-TCSTfio: Seq Read - IO_uring - No - No - 64KB - 1 - Default Test Directoryfio: Seq Read - IO_uring - No - No - 64KB - 1 - Default Test Directoryfio: Rand Read - IO_uring - No - No - 4KB - 1 - Default Test Directoryfio: Rand Read - IO_uring - No - No - 4KB - 1 - Default Test Directoryfio: Seq Read - IO_uring - No - No - 4KB - 1 - Default Test Directoryfio: Seq Read - IO_uring - No - No - 4KB - 1 - Default Test Directoryfio: Rand Read - IO_uring - No - No - 1MB - 1 - Default Test Directoryfio: Rand Read - IO_uring - No - No - 1MB - 1 - Default Test Directoryfio: Rand Read - IO_uring - No - No - 64KB - 1 - Default Test Directoryfio: Rand Read - IO_uring - No - No - 64KB - 1 - Default Test Directoryfio: Seq Read - IO_uring - No - No - 1MB - 1 - Default Test Directoryfio: Seq Read - IO_uring - No - No - 1MB - 1 - Default Test Directoryfio: Seq Write - IO_uring - No - No - 64KB - 1 - Default Test Directoryfio: Seq Write - IO_uring - No - No - 64KB - 1 - Default Test Directoryfio: Seq Write - IO_uring - No - No - 1MB - 1 - Default Test Directoryfio: Seq Write - IO_uring - No - No - 1MB - 1 - Default Test Directoryfio: Rand Write - IO_uring - No - No - 1MB - 1 - Default Test Directoryfio: Rand Write - IO_uring - No - No - 1MB - 1 - Default Test Directoryfio: Rand Write - IO_uring - No - No - 64KB - 1 - Default Test Directoryfio: Rand Write - IO_uring - No - No - 64KB - 1 - Default Test Directoryfio: Seq Write - IO_uring - No - No - 4KB - 1 - Default Test Directoryfio: Seq Write - IO_uring - No - No - 4KB - 1 - Default Test Directoryfio: Rand Write - IO_uring - No - No - 4KB - 1 - Default Test Directoryfio: Rand Write - IO_uring - No - No - 4KB - 1 - Default Test DirectoryPVE-NVME-R10-4DISK-ZFS2.1PVE-NVME-R10-4DISK-ZFS2.2PVE-NVME-R10-4DISK-ZFS2.2-0PVE-NVME-R10-4DISK-ZFS2.2-1521193257.47127708510833.62131558526231.7340334032.60605263782.8538043803.88271421696.3924202419.8325542554.28314371964.7993001372002.8388678354712.28481573009.81118691474764.96122330489319.7338143814.21558023487.6436063606.5810264656928.1213781378.7126432643.0810978702583.6891556366223.5288991355963.941162617266.30146131584524.11149574598294.793860738607.541142547140.863275632755.76686704291.9080738072.7477327732.61705124407.00121819487276.45116590466359.46OpenBenchmarking.org

Flexible IO Tester

OpenBenchmarking.orgIOPS, More Is BetterFlexible IO Tester 3.35Type: Sequential Read - Engine: IO_uring - Buffered: No - Direct: No - Block Size: 64KB - Job Count: 1 - Disk Target: Default Test DirectoryPVE-NVME-R10-4DISK-ZFS2.2-1PVE-NVME-R10-4DISK-ZFS2.2PVE-NVME-R10-4DISK-ZFS2.120K40K60K80K100KSE +/- 2746.65, N = 4SE +/- 62.81, N = 4SE +/- 98.17, N = 41162614815752119MIN: 111674.96 / MAX: 123071.89MIN: 47997.17 / MAX: 48303.78MIN: 51928.62 / MAX: 52312.581. (CC) gcc options: -rdynamic -lm -laio -lpthread -ldl -std=gnu99 -ffast-math -include -O3 -fcommon -march=native

OpenBenchmarking.orgMiB/s, More Is BetterFlexible IO Tester 3.35Type: Sequential Read - Engine: IO_uring - Buffered: No - Direct: No - Block Size: 64KB - Job Count: 1 - Disk Target: Default Test DirectoryPVE-NVME-R10-4DISK-ZFS2.2-1PVE-NVME-R10-4DISK-ZFS2.2PVE-NVME-R10-4DISK-ZFS2.116003200480064008000SE +/- 171.66, N = 4SE +/- 3.93, N = 4SE +/- 6.14, N = 47266.303009.813257.47MIN: 6979.69 / MAX: 7691.99MIN: 2999.83 / MAX: 3018.99MIN: 3245.54 / MAX: 3269.541. (CC) gcc options: -rdynamic -lm -laio -lpthread -ldl -std=gnu99 -ffast-math -include -O3 -fcommon -march=native

OpenBenchmarking.orgIOPS, More Is BetterFlexible IO Tester 3.35Type: Random Read - Engine: IO_uring - Buffered: No - Direct: No - Block Size: 4KB - Job Count: 1 - Disk Target: Default Test DirectoryPVE-NVME-R10-4DISK-ZFS2.2-1PVE-NVME-R10-4DISK-ZFS2.2PVE-NVME-R10-4DISK-ZFS2.130K60K90K120K150KSE +/- 595.19, N = 4SE +/- 1404.16, N = 4SE +/- 1742.30, N = 4146131118691127708MIN: 144726.44 / MAX: 147318.97MIN: 114916.45 / MAX: 121357.02MIN: 122932.47 / MAX: 131080.91. (CC) gcc options: -rdynamic -lm -laio -lpthread -ldl -std=gnu99 -ffast-math -include -O3 -fcommon -march=native

OpenBenchmarking.orgMiB/s, More Is BetterFlexible IO Tester 3.35Type: Random Read - Engine: IO_uring - Buffered: No - Direct: No - Block Size: 4KB - Job Count: 1 - Disk Target: Default Test DirectoryPVE-NVME-R10-4DISK-ZFS2.2-1PVE-NVME-R10-4DISK-ZFS2.2PVE-NVME-R10-4DISK-ZFS2.1130K260K390K520K650KSE +/- 2380.78, N = 4SE +/- 5616.59, N = 4SE +/- 6969.17, N = 4584524.11474764.96510833.62MIN: 578905.65 / MAX: 589275.96MIN: 459665.87 / MAX: 485427.97MIN: 491730.18 / MAX: 524323.851. (CC) gcc options: -rdynamic -lm -laio -lpthread -ldl -std=gnu99 -ffast-math -include -O3 -fcommon -march=native

OpenBenchmarking.orgIOPS, More Is BetterFlexible IO Tester 3.35Type: Sequential Read - Engine: IO_uring - Buffered: No - Direct: No - Block Size: 4KB - Job Count: 1 - Disk Target: Default Test DirectoryPVE-NVME-R10-4DISK-ZFS2.2-1PVE-NVME-R10-4DISK-ZFS2.2PVE-NVME-R10-4DISK-ZFS2.130K60K90K120K150KSE +/- 1164.52, N = 4SE +/- 941.64, N = 4SE +/- 368.35, N = 4149574122330131558MIN: 146846.33 / MAX: 152080.29MIN: 119532.43 / MAX: 123585.36MIN: 130702.69 / MAX: 132302.271. (CC) gcc options: -rdynamic -lm -laio -lpthread -ldl -std=gnu99 -ffast-math -include -O3 -fcommon -march=native

OpenBenchmarking.orgMiB/s, More Is BetterFlexible IO Tester 3.35Type: Sequential Read - Engine: IO_uring - Buffered: No - Direct: No - Block Size: 4KB - Job Count: 1 - Disk Target: Default Test DirectoryPVE-NVME-R10-4DISK-ZFS2.2-1PVE-NVME-R10-4DISK-ZFS2.2PVE-NVME-R10-4DISK-ZFS2.1130K260K390K520K650KSE +/- 4658.11, N = 4SE +/- 3766.56, N = 4SE +/- 1473.41, N = 4598294.79489319.73526231.73MIN: 587385.32 / MAX: 608321.28MIN: 478129.83 / MAX: 494341.44MIN: 522810.86 / MAX: 529209.121. (CC) gcc options: -rdynamic -lm -laio -lpthread -ldl -std=gnu99 -ffast-math -include -O3 -fcommon -march=native

OpenBenchmarking.orgIOPS, More Is BetterFlexible IO Tester 3.35Type: Random Read - Engine: IO_uring - Buffered: No - Direct: No - Block Size: 1MB - Job Count: 1 - Disk Target: Default Test DirectoryPVE-NVME-R10-4DISK-ZFS2.2-1PVE-NVME-R10-4DISK-ZFS2.2PVE-NVME-R10-4DISK-ZFS2.18K16K24K32K40KSE +/- 380.09, N = 4SE +/- 2.10, N = 4SE +/- 2.23, N = 43860738144033MIN: 37591.27 / MAX: 39279.26MIN: 4028.6 / MAX: 4038.781. (CC) gcc options: -rdynamic -lm -laio -lpthread -ldl -std=gnu99 -ffast-math -include -O3 -fcommon -march=native

OpenBenchmarking.orgMiB/s, More Is BetterFlexible IO Tester 3.35Type: Random Read - Engine: IO_uring - Buffered: No - Direct: No - Block Size: 1MB - Job Count: 1 - Disk Target: Default Test DirectoryPVE-NVME-R10-4DISK-ZFS2.2-1PVE-NVME-R10-4DISK-ZFS2.2PVE-NVME-R10-4DISK-ZFS2.18K16K24K32K40KSE +/- 380.08, N = 4SE +/- 2.11, N = 4SE +/- 2.23, N = 438607.543814.214032.60MIN: 37591.38 / MAX: 39279.351. (CC) gcc options: -rdynamic -lm -laio -lpthread -ldl -std=gnu99 -ffast-math -include -O3 -fcommon -march=native

OpenBenchmarking.orgIOPS, More Is BetterFlexible IO Tester 3.35Type: Random Read - Engine: IO_uring - Buffered: No - Direct: No - Block Size: 64KB - Job Count: 1 - Disk Target: Default Test DirectoryPVE-NVME-R10-4DISK-ZFS2.2-1PVE-NVME-R10-4DISK-ZFS2.2PVE-NVME-R10-4DISK-ZFS2.120K40K60K80K100KSE +/- 865.89, N = 4SE +/- 82.11, N = 4SE +/- 251.91, N = 41142545580260526MIN: 112099.1 / MAX: 115877.31MIN: 55571.32 / MAX: 55958.14MIN: 60127.11 / MAX: 61228.361. (CC) gcc options: -rdynamic -lm -laio -lpthread -ldl -std=gnu99 -ffast-math -include -O3 -fcommon -march=native

OpenBenchmarking.orgMiB/s, More Is BetterFlexible IO Tester 3.35Type: Random Read - Engine: IO_uring - Buffered: No - Direct: No - Block Size: 64KB - Job Count: 1 - Disk Target: Default Test DirectoryPVE-NVME-R10-4DISK-ZFS2.2-1PVE-NVME-R10-4DISK-ZFS2.2PVE-NVME-R10-4DISK-ZFS2.115003000450060007500SE +/- 54.12, N = 4SE +/- 5.13, N = 4SE +/- 15.75, N = 47140.863487.643782.85MIN: 7006.19 / MAX: 7242.33MIN: 3473.22 / MAX: 3497.39MIN: 3757.95 / MAX: 3826.781. (CC) gcc options: -rdynamic -lm -laio -lpthread -ldl -std=gnu99 -ffast-math -include -O3 -fcommon -march=native

OpenBenchmarking.orgIOPS, More Is BetterFlexible IO Tester 3.35Type: Sequential Read - Engine: IO_uring - Buffered: No - Direct: No - Block Size: 1MB - Job Count: 1 - Disk Target: Default Test DirectoryPVE-NVME-R10-4DISK-ZFS2.2-1PVE-NVME-R10-4DISK-ZFS2.2PVE-NVME-R10-4DISK-ZFS2.17K14K21K28K35KSE +/- 106.01, N = 4SE +/- 0.68, N = 4SE +/- 2.29, N = 43275636063804MIN: 32463.58 / MAX: 32968.66MIN: 3604.97 / MAX: 3607.71MIN: 3797.73 / MAX: 3808.691. (CC) gcc options: -rdynamic -lm -laio -lpthread -ldl -std=gnu99 -ffast-math -include -O3 -fcommon -march=native

OpenBenchmarking.orgMiB/s, More Is BetterFlexible IO Tester 3.35Type: Sequential Read - Engine: IO_uring - Buffered: No - Direct: No - Block Size: 1MB - Job Count: 1 - Disk Target: Default Test DirectoryPVE-NVME-R10-4DISK-ZFS2.2-1PVE-NVME-R10-4DISK-ZFS2.2PVE-NVME-R10-4DISK-ZFS2.17K14K21K28K35KSE +/- 106.02, N = 4SE +/- 0.67, N = 4SE +/- 2.30, N = 432755.763606.583803.88MIN: 32463.63 / MAX: 32968.821. (CC) gcc options: -rdynamic -lm -laio -lpthread -ldl -std=gnu99 -ffast-math -include -O3 -fcommon -march=native

OpenBenchmarking.orgIOPS, More Is BetterFlexible IO Tester 3.35Type: Sequential Write - Engine: IO_uring - Buffered: No - Direct: No - Block Size: 64KB - Job Count: 1 - Disk Target: Default Test DirectoryPVE-NVME-R10-4DISK-ZFS2.2-1PVE-NVME-R10-4DISK-ZFS2.2PVE-NVME-R10-4DISK-ZFS2.115K30K45K60K75KSE +/- 835.37, N = 4SE +/- 288.54, N = 4SE +/- 419.10, N = 4686701026427142MIN: 67153.2 / MAX: 71038.27MIN: 9432.11 / MAX: 10758.13MIN: 26088.08 / MAX: 27844.441. (CC) gcc options: -rdynamic -lm -laio -lpthread -ldl -std=gnu99 -ffast-math -include -O3 -fcommon -march=native

OpenBenchmarking.orgMiB/s, More Is BetterFlexible IO Tester 3.35Type: Sequential Write - Engine: IO_uring - Buffered: No - Direct: No - Block Size: 64KB - Job Count: 1 - Disk Target: Default Test DirectoryPVE-NVME-R10-4DISK-ZFS2.2-1PVE-NVME-R10-4DISK-ZFS2.2PVE-NVME-R10-4DISK-ZFS2.1140K280K420K560K700KSE +/- 52.21, N = 4SE +/- 18466.98, N = 4SE +/- 26.19, N = 44291.90656928.121696.39MIN: 603671.44 / MAX: 688537.191. (CC) gcc options: -rdynamic -lm -laio -lpthread -ldl -std=gnu99 -ffast-math -include -O3 -fcommon -march=native

OpenBenchmarking.orgIOPS, More Is BetterFlexible IO Tester 3.35Type: Sequential Write - Engine: IO_uring - Buffered: No - Direct: No - Block Size: 1MB - Job Count: 1 - Disk Target: Default Test DirectoryPVE-NVME-R10-4DISK-ZFS2.2-1PVE-NVME-R10-4DISK-ZFS2.2PVE-NVME-R10-4DISK-ZFS2.12K4K6K8K10KSE +/- 106.06, N = 4SE +/- 16.25, N = 4SE +/- 101.00, N = 4807313782420MIN: 7763.24 / MAX: 8217.84MIN: 1352.95 / MAX: 1424.59MIN: 2216.08 / MAX: 2646.041. (CC) gcc options: -rdynamic -lm -laio -lpthread -ldl -std=gnu99 -ffast-math -include -O3 -fcommon -march=native

OpenBenchmarking.orgMiB/s, More Is BetterFlexible IO Tester 3.35Type: Sequential Write - Engine: IO_uring - Buffered: No - Direct: No - Block Size: 1MB - Job Count: 1 - Disk Target: Default Test DirectoryPVE-NVME-R10-4DISK-ZFS2.2-1PVE-NVME-R10-4DISK-ZFS2.2PVE-NVME-R10-4DISK-ZFS2.12K4K6K8K10KSE +/- 106.08, N = 4SE +/- 16.25, N = 4SE +/- 100.97, N = 48072.741378.712419.83MIN: 7763.33 / MAX: 8217.98MIN: 2216.36 / MAX: 2646.21. (CC) gcc options: -rdynamic -lm -laio -lpthread -ldl -std=gnu99 -ffast-math -include -O3 -fcommon -march=native

OpenBenchmarking.orgIOPS, More Is BetterFlexible IO Tester 3.35Type: Random Write - Engine: IO_uring - Buffered: No - Direct: No - Block Size: 1MB - Job Count: 1 - Disk Target: Default Test DirectoryPVE-NVME-R10-4DISK-ZFS2.2-1PVE-NVME-R10-4DISK-ZFS2.2PVE-NVME-R10-4DISK-ZFS2.117003400510068008500SE +/- 97.60, N = 4SE +/- 10.38, N = 4SE +/- 50.98, N = 4773226432554MIN: 7548.35 / MAX: 8001.2MIN: 2613.24 / MAX: 2660.24MIN: 2405.24 / MAX: 26211. (CC) gcc options: -rdynamic -lm -laio -lpthread -ldl -std=gnu99 -ffast-math -include -O3 -fcommon -march=native

OpenBenchmarking.orgMiB/s, More Is BetterFlexible IO Tester 3.35Type: Random Write - Engine: IO_uring - Buffered: No - Direct: No - Block Size: 1MB - Job Count: 1 - Disk Target: Default Test DirectoryPVE-NVME-R10-4DISK-ZFS2.2-1PVE-NVME-R10-4DISK-ZFS2.2PVE-NVME-R10-4DISK-ZFS2.117003400510068008500SE +/- 97.60, N = 4SE +/- 10.38, N = 4SE +/- 50.98, N = 47732.612643.082554.28MIN: 7548.52 / MAX: 8001.4MIN: 2613.46 / MAX: 2660.47MIN: 2405.5 / MAX: 2621.261. (CC) gcc options: -rdynamic -lm -laio -lpthread -ldl -std=gnu99 -ffast-math -include -O3 -fcommon -march=native

OpenBenchmarking.orgIOPS, More Is BetterFlexible IO Tester 3.35Type: Random Write - Engine: IO_uring - Buffered: No - Direct: No - Block Size: 64KB - Job Count: 1 - Disk Target: Default Test DirectoryPVE-NVME-R10-4DISK-ZFS2.2-1PVE-NVME-R10-4DISK-ZFS2.2PVE-NVME-R10-4DISK-ZFS2.115K30K45K60K75KSE +/- 1241.12, N = 4SE +/- 18.77, N = 4SE +/- 160.70, N = 4705121097831437MIN: 68257.81 / MAX: 72741.99MIN: 10937.74 / MAX: 11026.18MIN: 31030.13 / MAX: 31758.51. (CC) gcc options: -rdynamic -lm -laio -lpthread -ldl -std=gnu99 -ffast-math -include -O3 -fcommon -march=native

OpenBenchmarking.orgMiB/s, More Is BetterFlexible IO Tester 3.35Type: Random Write - Engine: IO_uring - Buffered: No - Direct: No - Block Size: 64KB - Job Count: 1 - Disk Target: Default Test DirectoryPVE-NVME-R10-4DISK-ZFS2.2-1PVE-NVME-R10-4DISK-ZFS2.2PVE-NVME-R10-4DISK-ZFS2.1150K300K450K600K750KSE +/- 77.57, N = 4SE +/- 1201.47, N = 4SE +/- 10.04, N = 44407.00702583.681964.79MIN: 700032.56 / MAX: 705692.671. (CC) gcc options: -rdynamic -lm -laio -lpthread -ldl -std=gnu99 -ffast-math -include -O3 -fcommon -march=native

OpenBenchmarking.orgIOPS, More Is BetterFlexible IO Tester 3.35Type: Sequential Write - Engine: IO_uring - Buffered: No - Direct: No - Block Size: 4KB - Job Count: 1 - Disk Target: Default Test DirectoryPVE-NVME-R10-4DISK-ZFS2.2-1PVE-NVME-R10-4DISK-ZFS2.2PVE-NVME-R10-4DISK-ZFS2.130K60K90K120K150KSE +/- 681.78, N = 4SE +/- 471.34, N = 4SE +/- 612.87, N = 41218199155693001MIN: 119976.59 / MAX: 122880MIN: 90268.39 / MAX: 92441.34MIN: 91598.8 / MAX: 94217.311. (CC) gcc options: -rdynamic -lm -laio -lpthread -ldl -std=gnu99 -ffast-math -include -O3 -fcommon -march=native

OpenBenchmarking.orgMiB/s, More Is BetterFlexible IO Tester 3.35Type: Sequential Write - Engine: IO_uring - Buffered: No - Direct: No - Block Size: 4KB - Job Count: 1 - Disk Target: Default Test DirectoryPVE-NVME-R10-4DISK-ZFS2.2-1PVE-NVME-R10-4DISK-ZFS2.2PVE-NVME-R10-4DISK-ZFS2.1100K200K300K400K500KSE +/- 2727.11, N = 4SE +/- 1885.36, N = 4SE +/- 2451.46, N = 4487276.45366223.52372002.83MIN: 479906.43 / MAX: 491520.09MIN: 361073.65 / MAX: 369765.43MIN: 366395.47 / MAX: 376869.381. (CC) gcc options: -rdynamic -lm -laio -lpthread -ldl -std=gnu99 -ffast-math -include -O3 -fcommon -march=native

OpenBenchmarking.orgIOPS, More Is BetterFlexible IO Tester 3.35Type: Random Write - Engine: IO_uring - Buffered: No - Direct: No - Block Size: 4KB - Job Count: 1 - Disk Target: Default Test DirectoryPVE-NVME-R10-4DISK-ZFS2.2-1PVE-NVME-R10-4DISK-ZFS2.2PVE-NVME-R10-4DISK-ZFS2.120K40K60K80K100KSE +/- 722.58, N = 4SE +/- 766.78, N = 4SE +/- 519.90, N = 41165908899188678MIN: 115087.08 / MAX: 118057.92MIN: 87994.01 / MAX: 91232.19MIN: 87337.43 / MAX: 89676.771. (CC) gcc options: -rdynamic -lm -laio -lpthread -ldl -std=gnu99 -ffast-math -include -O3 -fcommon -march=native

OpenBenchmarking.orgMiB/s, More Is BetterFlexible IO Tester 3.35Type: Random Write - Engine: IO_uring - Buffered: No - Direct: No - Block Size: 4KB - Job Count: 1 - Disk Target: Default Test DirectoryPVE-NVME-R10-4DISK-ZFS2.2-1PVE-NVME-R10-4DISK-ZFS2.2PVE-NVME-R10-4DISK-ZFS2.1100K200K300K400K500KSE +/- 2890.32, N = 4SE +/- 3067.11, N = 4SE +/- 2079.65, N = 4466359.46355963.94354712.28MIN: 460348.44 / MAX: 472231.83MIN: 351976.19 / MAX: 364928.78MIN: 349349.75 / MAX: 358707.211. (CC) gcc options: -rdynamic -lm -laio -lpthread -ldl -std=gnu99 -ffast-math -include -O3 -fcommon -march=native

24 Results Shown

Flexible IO Tester:
  Seq Read - IO_uring - No - No - 64KB - 1 - Default Test Directory:
    IOPS
    MiB/s
  Rand Read - IO_uring - No - No - 4KB - 1 - Default Test Directory:
    IOPS
    MiB/s
  Seq Read - IO_uring - No - No - 4KB - 1 - Default Test Directory:
    IOPS
    MiB/s
  Rand Read - IO_uring - No - No - 1MB - 1 - Default Test Directory:
    IOPS
    MiB/s
  Rand Read - IO_uring - No - No - 64KB - 1 - Default Test Directory:
    IOPS
    MiB/s
  Seq Read - IO_uring - No - No - 1MB - 1 - Default Test Directory:
    IOPS
    MiB/s
  Seq Write - IO_uring - No - No - 64KB - 1 - Default Test Directory:
    IOPS
    MiB/s
  Seq Write - IO_uring - No - No - 1MB - 1 - Default Test Directory:
    IOPS
    MiB/s
  Rand Write - IO_uring - No - No - 1MB - 1 - Default Test Directory:
    IOPS
    MiB/s
  Rand Write - IO_uring - No - No - 64KB - 1 - Default Test Directory:
    IOPS
    MiB/s
  Seq Write - IO_uring - No - No - 4KB - 1 - Default Test Directory:
    IOPS
    MiB/s
  Rand Write - IO_uring - No - No - 4KB - 1 - Default Test Directory:
    IOPS
    MiB/s

PVE-NVME-R10-4DISK-ZFS2.1

Processor: AMD EPYC 7543 32-Core (12 Cores), Motherboard: QEMU Standard PC (Q35 + ICH9 2009) (3.20230228-4 BIOS), Chipset: Intel 82G33/G31/P35/P31 + ICH9, Memory: 16GB, Disk: 34GB QEMU HDD, Graphics: bochs-drmdrmfb, Audio: Intel 82801I, Monitor: QEMU Monitor, Network: Red Hat Virtio device

OS: Ubuntu 22.04, Kernel: 5.15.0-86-generic (x86_64), Vulkan: 1.3.238, Compiler: GCC 11.4.0, File-System: ext4, Screen Resolution: 1280x800, System Layer: KVM

Kernel Notes: Transparent Huge Pages: madvise
Compiler Notes: --build=x86_64-linux-gnu --disable-vtable-verify --disable-werror --enable-bootstrap --enable-cet --enable-checking=release --enable-clocale=gnu --enable-default-pie --enable-gnu-unique-object --enable-languages=c,ada,c++,go,brig,d,fortran,objc,obj-c++,m2 --enable-libphobos-checking=release --enable-libstdcxx-debug --enable-libstdcxx-time=yes --enable-link-serialization=2 --enable-multiarch --enable-multilib --enable-nls --enable-objc-gc=auto --enable-offload-targets=nvptx-none=/build/gcc-11-XeT9lY/gcc-11-11.4.0/debian/tmp-nvptx/usr,amdgcn-amdhsa=/build/gcc-11-XeT9lY/gcc-11-11.4.0/debian/tmp-gcn/usr --enable-plugin --enable-shared --enable-threads=posix --host=x86_64-linux-gnu --program-prefix=x86_64-linux-gnu- --target=x86_64-linux-gnu --with-abi=m64 --with-arch-32=i686 --with-build-config=bootstrap-lto-lean --with-default-libstdcxx-abi=new --with-gcc-major-version-only --with-multilib-list=m32,m64,mx32 --with-target-system-zlib=auto --with-tune=generic --without-cuda-driver -v
Disk Notes: NONE / relatime,rw / Block Size: 4096
Processor Notes: CPU Microcode: 0xa00110d
Security Notes: gather_data_sampling: Not affected + itlb_multihit: Not affected + l1tf: Not affected + mds: Not affected + meltdown: Not affected + mmio_stale_data: Not affected + retbleed: Not affected + spec_rstack_overflow: Mitigation of safe RET no microcode + spec_store_bypass: Mitigation of SSB disabled via prctl and seccomp + spectre_v1: Mitigation of usercopy/swapgs barriers and __user pointer sanitization + spectre_v2: Mitigation of Retpolines IBPB: conditional IBRS_FW STIBP: disabled RSB filling PBRSB-eIBRS: Not affected + srbds: Not affected + tsx_async_abort: Not affected

Testing initiated at 5 November 2023 00:50 by user tcstudios.

PVE-NVME-R10-4DISK-ZFS2.2

Kernel Notes: Transparent Huge Pages: madvise
Compiler Notes: --build=x86_64-linux-gnu --disable-vtable-verify --disable-werror --enable-bootstrap --enable-cet --enable-checking=release --enable-clocale=gnu --enable-default-pie --enable-gnu-unique-object --enable-languages=c,ada,c++,go,brig,d,fortran,objc,obj-c++,m2 --enable-libphobos-checking=release --enable-libstdcxx-debug --enable-libstdcxx-time=yes --enable-link-serialization=2 --enable-multiarch --enable-multilib --enable-nls --enable-objc-gc=auto --enable-offload-targets=nvptx-none=/build/gcc-11-XeT9lY/gcc-11-11.4.0/debian/tmp-nvptx/usr,amdgcn-amdhsa=/build/gcc-11-XeT9lY/gcc-11-11.4.0/debian/tmp-gcn/usr --enable-plugin --enable-shared --enable-threads=posix --host=x86_64-linux-gnu --program-prefix=x86_64-linux-gnu- --target=x86_64-linux-gnu --with-abi=m64 --with-arch-32=i686 --with-build-config=bootstrap-lto-lean --with-default-libstdcxx-abi=new --with-gcc-major-version-only --with-multilib-list=m32,m64,mx32 --with-target-system-zlib=auto --with-tune=generic --without-cuda-driver -v
Disk Notes: NONE / relatime,rw / Block Size: 4096
Processor Notes: CPU Microcode: 0xa00110d
Security Notes: gather_data_sampling: Not affected + itlb_multihit: Not affected + l1tf: Not affected + mds: Not affected + meltdown: Not affected + mmio_stale_data: Not affected + retbleed: Not affected + spec_rstack_overflow: Mitigation of safe RET no microcode + spec_store_bypass: Mitigation of SSB disabled via prctl and seccomp + spectre_v1: Mitigation of usercopy/swapgs barriers and __user pointer sanitization + spectre_v2: Mitigation of Retpolines IBPB: conditional IBRS_FW STIBP: disabled RSB filling PBRSB-eIBRS: Not affected + srbds: Not affected + tsx_async_abort: Not affected

Testing initiated at 5 November 2023 02:59 by user tcstudios.

PVE-NVME-R10-4DISK-ZFS2.2-0

Kernel Notes: Transparent Huge Pages: madvise
Compiler Notes: --build=x86_64-linux-gnu --disable-vtable-verify --disable-werror --enable-bootstrap --enable-cet --enable-checking=release --enable-clocale=gnu --enable-default-pie --enable-gnu-unique-object --enable-languages=c,ada,c++,go,brig,d,fortran,objc,obj-c++,m2 --enable-libphobos-checking=release --enable-libstdcxx-debug --enable-libstdcxx-time=yes --enable-link-serialization=2 --enable-multiarch --enable-multilib --enable-nls --enable-objc-gc=auto --enable-offload-targets=nvptx-none=/build/gcc-11-XeT9lY/gcc-11-11.4.0/debian/tmp-nvptx/usr,amdgcn-amdhsa=/build/gcc-11-XeT9lY/gcc-11-11.4.0/debian/tmp-gcn/usr --enable-plugin --enable-shared --enable-threads=posix --host=x86_64-linux-gnu --program-prefix=x86_64-linux-gnu- --target=x86_64-linux-gnu --with-abi=m64 --with-arch-32=i686 --with-build-config=bootstrap-lto-lean --with-default-libstdcxx-abi=new --with-gcc-major-version-only --with-multilib-list=m32,m64,mx32 --with-target-system-zlib=auto --with-tune=generic --without-cuda-driver -v
Disk Notes: NONE / relatime,rw / Block Size: 4096
Processor Notes: CPU Microcode: 0xa00110d
Security Notes: gather_data_sampling: Not affected + itlb_multihit: Not affected + l1tf: Not affected + mds: Not affected + meltdown: Not affected + mmio_stale_data: Not affected + retbleed: Not affected + spec_rstack_overflow: Mitigation of safe RET no microcode + spec_store_bypass: Mitigation of SSB disabled via prctl and seccomp + spectre_v1: Mitigation of usercopy/swapgs barriers and __user pointer sanitization + spectre_v2: Mitigation of Retpolines IBPB: conditional IBRS_FW STIBP: disabled RSB filling PBRSB-eIBRS: Not affected + srbds: Not affected + tsx_async_abort: Not affected

Testing initiated at 13 November 2023 08:16 by user tcstudios.

PVE-NVME-R10-4DISK-ZFS2.2-1

Processor: AMD EPYC 7543 32-Core (12 Cores), Motherboard: QEMU Standard PC (Q35 + ICH9 2009) (3.20230228-4 BIOS), Chipset: Intel 82G33/G31/P35/P31 + ICH9, Memory: 16GB, Disk: 34GB QEMU HDD, Graphics: bochs-drmdrmfb, Audio: Intel 82801I, Monitor: QEMU Monitor, Network: Red Hat Virtio device

OS: Ubuntu 22.04, Kernel: 5.15.0-88-generic (x86_64), Vulkan: 1.3.238, Compiler: GCC 11.4.0, File-System: ext4, Screen Resolution: 1280x800, System Layer: KVM

Kernel Notes: Transparent Huge Pages: madvise
Compiler Notes: --build=x86_64-linux-gnu --disable-vtable-verify --disable-werror --enable-bootstrap --enable-cet --enable-checking=release --enable-clocale=gnu --enable-default-pie --enable-gnu-unique-object --enable-languages=c,ada,c++,go,brig,d,fortran,objc,obj-c++,m2 --enable-libphobos-checking=release --enable-libstdcxx-debug --enable-libstdcxx-time=yes --enable-link-serialization=2 --enable-multiarch --enable-multilib --enable-nls --enable-objc-gc=auto --enable-offload-targets=nvptx-none=/build/gcc-11-XeT9lY/gcc-11-11.4.0/debian/tmp-nvptx/usr,amdgcn-amdhsa=/build/gcc-11-XeT9lY/gcc-11-11.4.0/debian/tmp-gcn/usr --enable-plugin --enable-shared --enable-threads=posix --host=x86_64-linux-gnu --program-prefix=x86_64-linux-gnu- --target=x86_64-linux-gnu --with-abi=m64 --with-arch-32=i686 --with-build-config=bootstrap-lto-lean --with-default-libstdcxx-abi=new --with-gcc-major-version-only --with-multilib-list=m32,m64,mx32 --with-target-system-zlib=auto --with-tune=generic --without-cuda-driver -v
Disk Notes: NONE / relatime,rw / Block Size: 4096
Processor Notes: CPU Microcode: 0xa00110d
Security Notes: gather_data_sampling: Not affected + itlb_multihit: Not affected + l1tf: Not affected + mds: Not affected + meltdown: Not affected + mmio_stale_data: Not affected + retbleed: Not affected + spec_rstack_overflow: Mitigation of safe RET no microcode + spec_store_bypass: Mitigation of SSB disabled via prctl and seccomp + spectre_v1: Mitigation of usercopy/swapgs barriers and __user pointer sanitization + spectre_v2: Mitigation of Retpolines IBPB: conditional IBRS_FW STIBP: disabled RSB filling PBRSB-eIBRS: Not affected + srbds: Not affected + tsx_async_abort: Not affected

Testing initiated at 13 November 2023 08:16 by user tcstudios.