05112023-TCST

PVE-NVME-R10-8DISK-ZFS2.2 Genoa

Compare your own system(s) to this result file with the Phoronix Test Suite by running the command: phoronix-test-suite benchmark 2312184-TCST-231125580
Jump To Table - Results

View

Do Not Show Noisy Results
Do Not Show Results With Incomplete Data
Do Not Show Results With Little Change/Spread
List Notable Results
Show Result Confidence Charts
Allow Limiting Results To Certain Suite(s)

Statistics

Show Overall Harmonic Mean(s)
Show Overall Geometric Mean
Show Wins / Losses Counts (Pie Chart)
Normalize Results
Remove Outliers Before Calculating Averages

Graph Settings

Force Line Graphs Where Applicable
Convert To Scalar Where Applicable
Prefer Vertical Bar Graphs

Additional Graphs

Show Perf Per Core/Thread Calculation Graphs Where Applicable

Multi-Way Comparison

Condense Multi-Option Tests Into Single Result Graphs

Table

Show Detailed System Result Table

Run Management

Highlight
Result
Toggle/Hide
Result
Result
Identifier
Performance Per
Dollar
Date
Run
  Test
  Duration
PVE-NVME-R10-4DISK-ZFS2.1
November 05 2023
  2 Hours, 18 Minutes
PVE-NVME-R10-4DISK-ZFS2.2
November 05 2023
  2 Hours, 18 Minutes
PVE-NVME-R10-4DISK-LVM
November 15 2023
  1 Hour, 43 Minutes
PVE-NVME-R10-4DISK-LVM-THIN
November 25 2023
  1 Hour, 43 Minutes
XEN-NVME-R10-4DISK-EXT4
November 25 2023
  1 Hour, 44 Minutes
PVE-NVME-R10-8DISK-ZFS2.2
December 18 2023
  1 Hour, 9 Minutes
Invert Behavior (Only Show Selected Data)
  1 Hour, 49 Minutes

Only show results where is faster than
Only show results matching title/arguments (delimit multiple options with a comma):
Do not show results matching title/arguments (delimit multiple options with a comma):

PVE-NVME-R10-4DISK-ZFS2.1

Processor: AMD EPYC 7543 32-Core (12 Cores), Motherboard: QEMU Standard PC (Q35 + ICH9 2009) (3.20230228-4 BIOS), Chipset: Intel 82G33/G31/P35/P31 + ICH9, Memory: 16GB, Disk: 34GB QEMU HDD, Graphics: bochs-drmdrmfb, Audio: Intel 82801I, Monitor: QEMU Monitor, Network: Red Hat Virtio device

OS: Ubuntu 22.04, Kernel: 5.15.0-86-generic (x86_64), Vulkan: 1.3.238, Compiler: GCC 11.4.0, File-System: ext4, Screen Resolution: 1280x800, System Layer: KVM

Kernel Notes: Transparent Huge Pages: madvise
Compiler Notes: --build=x86_64-linux-gnu --disable-vtable-verify --disable-werror --enable-bootstrap --enable-cet --enable-checking=release --enable-clocale=gnu --enable-default-pie --enable-gnu-unique-object --enable-languages=c,ada,c++,go,brig,d,fortran,objc,obj-c++,m2 --enable-libphobos-checking=release --enable-libstdcxx-debug --enable-libstdcxx-time=yes --enable-link-serialization=2 --enable-multiarch --enable-multilib --enable-nls --enable-objc-gc=auto --enable-offload-targets=nvptx-none=/build/gcc-11-XeT9lY/gcc-11-11.4.0/debian/tmp-nvptx/usr,amdgcn-amdhsa=/build/gcc-11-XeT9lY/gcc-11-11.4.0/debian/tmp-gcn/usr --enable-plugin --enable-shared --enable-threads=posix --host=x86_64-linux-gnu --program-prefix=x86_64-linux-gnu- --target=x86_64-linux-gnu --with-abi=m64 --with-arch-32=i686 --with-build-config=bootstrap-lto-lean --with-default-libstdcxx-abi=new --with-gcc-major-version-only --with-multilib-list=m32,m64,mx32 --with-target-system-zlib=auto --with-tune=generic --without-cuda-driver -v
Disk Notes: NONE / relatime,rw / Block Size: 4096
Processor Notes: CPU Microcode: 0xa00110d
Security Notes: gather_data_sampling: Not affected + itlb_multihit: Not affected + l1tf: Not affected + mds: Not affected + meltdown: Not affected + mmio_stale_data: Not affected + retbleed: Not affected + spec_rstack_overflow: Mitigation of safe RET no microcode + spec_store_bypass: Mitigation of SSB disabled via prctl and seccomp + spectre_v1: Mitigation of usercopy/swapgs barriers and __user pointer sanitization + spectre_v2: Mitigation of Retpolines IBPB: conditional IBRS_FW STIBP: disabled RSB filling PBRSB-eIBRS: Not affected + srbds: Not affected + tsx_async_abort: Not affected

PVE-NVME-R10-4DISK-ZFS2.2

PVE-NVME-R10-4DISK-LVM

OS: Ubuntu 22.04, Kernel: 5.15.0-88-generic (x86_64), Vulkan: 1.3.238, Compiler: GCC 11.4.0, File-System: ext4, Screen Resolution: 1280x800, System Layer: KVM

PVE-NVME-R10-4DISK-LVM-THIN

Processor: AMD EPYC 9354 32-Core (12 Cores), Motherboard: QEMU Standard PC (Q35 + ICH9 2009) (4.2023.08-2 BIOS), Chipset: Intel 82G33/G31/P35/P31 + ICH9, Memory: 16GB, Disk: 34GB QEMU HDD, Graphics: bochs-drmdrmfb, Audio: Intel 82801I, Monitor: QEMU Monitor, Network: Red Hat Virtio device

OS: Ubuntu 22.04, Kernel: 5.15.0-84-generic (x86_64), Vulkan: 1.3.238, Compiler: GCC 11.4.0, File-System: ext4, Screen Resolution: 1280x800, System Layer: KVM

Kernel Notes: Transparent Huge Pages: madvise
Compiler Notes: --build=x86_64-linux-gnu --disable-vtable-verify --disable-werror --enable-bootstrap --enable-cet --enable-checking=release --enable-clocale=gnu --enable-default-pie --enable-gnu-unique-object --enable-languages=c,ada,c++,go,brig,d,fortran,objc,obj-c++,m2 --enable-libphobos-checking=release --enable-libstdcxx-debug --enable-libstdcxx-time=yes --enable-link-serialization=2 --enable-multiarch --enable-multilib --enable-nls --enable-objc-gc=auto --enable-offload-targets=nvptx-none=/build/gcc-11-XeT9lY/gcc-11-11.4.0/debian/tmp-nvptx/usr,amdgcn-amdhsa=/build/gcc-11-XeT9lY/gcc-11-11.4.0/debian/tmp-gcn/usr --enable-plugin --enable-shared --enable-threads=posix --host=x86_64-linux-gnu --program-prefix=x86_64-linux-gnu- --target=x86_64-linux-gnu --with-abi=m64 --with-arch-32=i686 --with-build-config=bootstrap-lto-lean --with-default-libstdcxx-abi=new --with-gcc-major-version-only --with-multilib-list=m32,m64,mx32 --with-target-system-zlib=auto --with-tune=generic --without-cuda-driver -v
Disk Notes: NONE / relatime,rw / Block Size: 4096
Processor Notes: CPU Microcode: 0xa101121
Security Notes: gather_data_sampling: Not affected + itlb_multihit: Not affected + l1tf: Not affected + mds: Not affected + meltdown: Not affected + mmio_stale_data: Not affected + retbleed: Not affected + spec_store_bypass: Mitigation of SSB disabled via prctl and seccomp + spectre_v1: Mitigation of usercopy/swapgs barriers and __user pointer sanitization + spectre_v2: Mitigation of Retpolines IBPB: conditional IBRS_FW STIBP: disabled RSB filling PBRSB-eIBRS: Not affected + srbds: Not affected + tsx_async_abort: Not affected

XEN-NVME-R10-4DISK-EXT4

Processor: AMD EPYC 9354 32-Core (12 Cores), Motherboard: Xen HVM domU v4.13, Chipset: Intel 440FX 82441FX PMC, Memory: 16GB, Disk: 30GB, Graphics: bochs-drmdrmfb

OS: Ubuntu 22.04, Kernel: 5.15.0-89-generic (x86_64), Vulkan: 1.3.238, Compiler: GCC 11.4.0, File-System: ext4, Screen Resolution: 1024x768, System Layer: Xen HVM domU 4.13

Kernel Notes: Transparent Huge Pages: madvise
Compiler Notes: --build=x86_64-linux-gnu --disable-vtable-verify --disable-werror --enable-bootstrap --enable-cet --enable-checking=release --enable-clocale=gnu --enable-default-pie --enable-gnu-unique-object --enable-languages=c,ada,c++,go,brig,d,fortran,objc,obj-c++,m2 --enable-libphobos-checking=release --enable-libstdcxx-debug --enable-libstdcxx-time=yes --enable-link-serialization=2 --enable-multiarch --enable-multilib --enable-nls --enable-objc-gc=auto --enable-offload-targets=nvptx-none=/build/gcc-11-XeT9lY/gcc-11-11.4.0/debian/tmp-nvptx/usr,amdgcn-amdhsa=/build/gcc-11-XeT9lY/gcc-11-11.4.0/debian/tmp-gcn/usr --enable-plugin --enable-shared --enable-threads=posix --host=x86_64-linux-gnu --program-prefix=x86_64-linux-gnu- --target=x86_64-linux-gnu --with-abi=m64 --with-arch-32=i686 --with-build-config=bootstrap-lto-lean --with-default-libstdcxx-abi=new --with-gcc-major-version-only --with-multilib-list=m32,m64,mx32 --with-target-system-zlib=auto --with-tune=generic --without-cuda-driver -v
Processor Notes: CPU Microcode: 0xa101144
Disk Mount Options Notes: relatime,rw
Disk Details Notes: Block Size: 4096
Security Notes: gather_data_sampling: Not affected + itlb_multihit: Not affected + l1tf: Not affected + mds: Not affected + meltdown: Not affected + mmio_stale_data: Not affected + retbleed: Not affected + spec_rstack_overflow: Mitigation of safe RET + spec_store_bypass: Mitigation of SSB disabled via prctl and seccomp + spectre_v1: Mitigation of usercopy/swapgs barriers and __user pointer sanitization + spectre_v2: Mitigation of Retpolines IBPB: conditional IBRS_FW STIBP: disabled RSB filling PBRSB-eIBRS: Not affected + srbds: Not affected + tsx_async_abort: Not affected

PVE-NVME-R10-8DISK-ZFS2.2

Processor: AMD EPYC 9354 32-Core (16 Cores), Motherboard: QEMU Standard PC (Q35 + ICH9 2009) (4.2023.08-2 BIOS), Chipset: Intel 82G33/G31/P35/P31 + ICH9, Memory: 16GB, Disk: 34GB QEMU HDD, Graphics: bochs-drmdrmfb, Audio: Intel 82801I, Monitor: QEMU Monitor, Network: Red Hat Virtio device

OS: Ubuntu 22.04, Kernel: 5.15.0-91-generic (x86_64), Vulkan: 1.3.238, Compiler: GCC 11.4.0, File-System: ext4, Screen Resolution: 1280x800, System Layer: KVM

Kernel Notes: Transparent Huge Pages: madvise
Compiler Notes: --build=x86_64-linux-gnu --disable-vtable-verify --disable-werror --enable-bootstrap --enable-cet --enable-checking=release --enable-clocale=gnu --enable-default-pie --enable-gnu-unique-object --enable-languages=c,ada,c++,go,brig,d,fortran,objc,obj-c++,m2 --enable-libphobos-checking=release --enable-libstdcxx-debug --enable-libstdcxx-time=yes --enable-link-serialization=2 --enable-multiarch --enable-multilib --enable-nls --enable-objc-gc=auto --enable-offload-targets=nvptx-none=/build/gcc-11-XeT9lY/gcc-11-11.4.0/debian/tmp-nvptx/usr,amdgcn-amdhsa=/build/gcc-11-XeT9lY/gcc-11-11.4.0/debian/tmp-gcn/usr --enable-plugin --enable-shared --enable-threads=posix --host=x86_64-linux-gnu --program-prefix=x86_64-linux-gnu- --target=x86_64-linux-gnu --with-abi=m64 --with-arch-32=i686 --with-build-config=bootstrap-lto-lean --with-default-libstdcxx-abi=new --with-gcc-major-version-only --with-multilib-list=m32,m64,mx32 --with-target-system-zlib=auto --with-tune=generic --without-cuda-driver -v
Disk Notes: NONE / relatime,rw / Block Size: 4096
Processor Notes: CPU Microcode: 0xa101121
Security Notes: gather_data_sampling: Not affected + itlb_multihit: Not affected + l1tf: Not affected + mds: Not affected + meltdown: Not affected + mmio_stale_data: Not affected + retbleed: Not affected + spec_rstack_overflow: Mitigation of safe RET no microcode + spec_store_bypass: Mitigation of SSB disabled via prctl and seccomp + spectre_v1: Mitigation of usercopy/swapgs barriers and __user pointer sanitization + spectre_v2: Mitigation of Retpolines IBPB: conditional IBRS_FW STIBP: disabled RSB filling PBRSB-eIBRS: Not affected + srbds: Not affected + tsx_async_abort: Not affected

05112023-TCSTProcessorMotherboardChipsetMemoryDiskGraphicsAudioMonitorNetworkOSKernelVulkanCompilerFile-SystemScreen ResolutionSystem LayerPVE-NVME-R10-4DISK-ZFS2.1PVE-NVME-R10-4DISK-ZFS2.2PVE-NVME-R10-4DISK-LVMPVE-NVME-R10-4DISK-LVM-THINXEN-NVME-R10-4DISK-EXT4PVE-NVME-R10-8DISK-ZFS2.2AMD EPYC 7543 32-Core (12 Cores)QEMU Standard PC (Q35 + ICH9 2009) (3.20230228-4 BIOS)Intel 82G33/G31/P35/P31 + ICH916GB34GB QEMU HDDbochs-drmdrmfbIntel 82801IQEMU MonitorRed Hat Virtio deviceUbuntu 22.045.15.0-86-generic (x86_64)1.3.238GCC 11.4.0ext41280x800KVM5.15.0-88-generic (x86_64)AMD EPYC 9354 32-Core (12 Cores)QEMU Standard PC (Q35 + ICH9 2009) (4.2023.08-2 BIOS)5.15.0-84-generic (x86_64)Xen HVM domU v4.13Intel 440FX 82441FX PMC30GB5.15.0-89-generic (x86_64)1024x768Xen HVM domU 4.13AMD EPYC 9354 32-Core (16 Cores)QEMU Standard PC (Q35 + ICH9 2009) (4.2023.08-2 BIOS)Intel 82G33/G31/P35/P31 + ICH934GB QEMU HDDIntel 82801IQEMU MonitorRed Hat Virtio device5.15.0-91-generic (x86_64)1280x800KVMOpenBenchmarking.orgKernel Details- Transparent Huge Pages: madviseCompiler Details- --build=x86_64-linux-gnu --disable-vtable-verify --disable-werror --enable-bootstrap --enable-cet --enable-checking=release --enable-clocale=gnu --enable-default-pie --enable-gnu-unique-object --enable-languages=c,ada,c++,go,brig,d,fortran,objc,obj-c++,m2 --enable-libphobos-checking=release --enable-libstdcxx-debug --enable-libstdcxx-time=yes --enable-link-serialization=2 --enable-multiarch --enable-multilib --enable-nls --enable-objc-gc=auto --enable-offload-targets=nvptx-none=/build/gcc-11-XeT9lY/gcc-11-11.4.0/debian/tmp-nvptx/usr,amdgcn-amdhsa=/build/gcc-11-XeT9lY/gcc-11-11.4.0/debian/tmp-gcn/usr --enable-plugin --enable-shared --enable-threads=posix --host=x86_64-linux-gnu --program-prefix=x86_64-linux-gnu- --target=x86_64-linux-gnu --with-abi=m64 --with-arch-32=i686 --with-build-config=bootstrap-lto-lean --with-default-libstdcxx-abi=new --with-gcc-major-version-only --with-multilib-list=m32,m64,mx32 --with-target-system-zlib=auto --with-tune=generic --without-cuda-driver -v Disk Details- PVE-NVME-R10-4DISK-ZFS2.1, PVE-NVME-R10-4DISK-ZFS2.2, PVE-NVME-R10-4DISK-LVM, PVE-NVME-R10-4DISK-LVM-THIN, PVE-NVME-R10-8DISK-ZFS2.2: NONE / relatime,rw / Block Size: 4096Processor Details- PVE-NVME-R10-4DISK-ZFS2.1: CPU Microcode: 0xa00110d- PVE-NVME-R10-4DISK-ZFS2.2: CPU Microcode: 0xa00110d- PVE-NVME-R10-4DISK-LVM: CPU Microcode: 0xa00110d- PVE-NVME-R10-4DISK-LVM-THIN: CPU Microcode: 0xa101121- XEN-NVME-R10-4DISK-EXT4: CPU Microcode: 0xa101144- PVE-NVME-R10-8DISK-ZFS2.2: CPU Microcode: 0xa101121Security Details- PVE-NVME-R10-4DISK-ZFS2.1: gather_data_sampling: Not affected + itlb_multihit: Not affected + l1tf: Not affected + mds: Not affected + meltdown: Not affected + mmio_stale_data: Not affected + retbleed: Not affected + spec_rstack_overflow: Mitigation of safe RET no microcode + spec_store_bypass: Mitigation of SSB disabled via prctl and seccomp + spectre_v1: Mitigation of usercopy/swapgs barriers and __user pointer sanitization + spectre_v2: Mitigation of Retpolines IBPB: conditional IBRS_FW STIBP: disabled RSB filling PBRSB-eIBRS: Not affected + srbds: Not affected + tsx_async_abort: Not affected - PVE-NVME-R10-4DISK-ZFS2.2: gather_data_sampling: Not affected + itlb_multihit: Not affected + l1tf: Not affected + mds: Not affected + meltdown: Not affected + mmio_stale_data: Not affected + retbleed: Not affected + spec_rstack_overflow: Mitigation of safe RET no microcode + spec_store_bypass: Mitigation of SSB disabled via prctl and seccomp + spectre_v1: Mitigation of usercopy/swapgs barriers and __user pointer sanitization + spectre_v2: Mitigation of Retpolines IBPB: conditional IBRS_FW STIBP: disabled RSB filling PBRSB-eIBRS: Not affected + srbds: Not affected + tsx_async_abort: Not affected - PVE-NVME-R10-4DISK-LVM: gather_data_sampling: Not affected + itlb_multihit: Not affected + l1tf: Not affected + mds: Not affected + meltdown: Not affected + mmio_stale_data: Not affected + retbleed: Not affected + spec_rstack_overflow: Mitigation of safe RET no microcode + spec_store_bypass: Mitigation of SSB disabled via prctl and seccomp + spectre_v1: Mitigation of usercopy/swapgs barriers and __user pointer sanitization + spectre_v2: Mitigation of Retpolines IBPB: conditional IBRS_FW STIBP: disabled RSB filling PBRSB-eIBRS: Not affected + srbds: Not affected + tsx_async_abort: Not affected - PVE-NVME-R10-4DISK-LVM-THIN: gather_data_sampling: Not affected + itlb_multihit: Not affected + l1tf: Not affected + mds: Not affected + meltdown: Not affected + mmio_stale_data: Not affected + retbleed: Not affected + spec_store_bypass: Mitigation of SSB disabled via prctl and seccomp + spectre_v1: Mitigation of usercopy/swapgs barriers and __user pointer sanitization + spectre_v2: Mitigation of Retpolines IBPB: conditional IBRS_FW STIBP: disabled RSB filling PBRSB-eIBRS: Not affected + srbds: Not affected + tsx_async_abort: Not affected - XEN-NVME-R10-4DISK-EXT4: gather_data_sampling: Not affected + itlb_multihit: Not affected + l1tf: Not affected + mds: Not affected + meltdown: Not affected + mmio_stale_data: Not affected + retbleed: Not affected + spec_rstack_overflow: Mitigation of safe RET + spec_store_bypass: Mitigation of SSB disabled via prctl and seccomp + spectre_v1: Mitigation of usercopy/swapgs barriers and __user pointer sanitization + spectre_v2: Mitigation of Retpolines IBPB: conditional IBRS_FW STIBP: disabled RSB filling PBRSB-eIBRS: Not affected + srbds: Not affected + tsx_async_abort: Not affected - PVE-NVME-R10-8DISK-ZFS2.2: gather_data_sampling: Not affected + itlb_multihit: Not affected + l1tf: Not affected + mds: Not affected + meltdown: Not affected + mmio_stale_data: Not affected + retbleed: Not affected + spec_rstack_overflow: Mitigation of safe RET no microcode + spec_store_bypass: Mitigation of SSB disabled via prctl and seccomp + spectre_v1: Mitigation of usercopy/swapgs barriers and __user pointer sanitization + spectre_v2: Mitigation of Retpolines IBPB: conditional IBRS_FW STIBP: disabled RSB filling PBRSB-eIBRS: Not affected + srbds: Not affected + tsx_async_abort: Not affected Disk Mount Options Details- XEN-NVME-R10-4DISK-EXT4: relatime,rwDisk Details Details- XEN-NVME-R10-4DISK-EXT4: Block Size: 4096

PVE-NVME-R10-4DISK-ZFS2.1PVE-NVME-R10-4DISK-ZFS2.2PVE-NVME-R10-4DISK-LVMPVE-NVME-R10-4DISK-LVM-THINXEN-NVME-R10-4DISK-EXT4PVE-NVME-R10-8DISK-ZFS2.2Logarithmic Result OverviewPhoronix Test SuiteFlexible IO TesterFlexible IO TesterFlexible IO TesterFlexible IO TesterFlexible IO TesterFlexible IO TesterFlexible IO TesterFlexible IO TesterFlexible IO TesterFlexible IO TesterFlexible IO TesterFlexible IO TesterFlexible IO TesterFlexible IO TesterFlexible IO TesterFlexible IO TesterFlexible IO TesterFlexible IO TesterFlexible IO TesterFlexible IO TesterFlexible IO TesterFlexible IO TesterFlexible IO TesterFlexible IO TesterSeq Write - IO_uring - No - No - 64KB - 1Rand Write - IO_uring - No - No - 64KB - 1Seq Write - IO_uring - No - No - 1MB - 1Seq Write - IO_uring - No - No - 1MB - 1Rand Write - IO_uring - No - No - 64KB - 1Seq Write - IO_uring - No - No - 64KB - 1Seq Read - IO_uring - No - No - 64KB - 1Seq Read - IO_uring - No - No - 64KB - 1Rand Write - IO_uring - No - No - 1MB - 1Rand Write - IO_uring - No - No - 1MB - 1Rand Read - IO_uring - No - No - 64KB - 1Rand Read - IO_uring - No - No - 64KB - 1Seq Read - IO_uring - No - No - 1MB - 1Seq Read - IO_uring - No - No - 1MB - 1Rand Read - IO_uring - No - No - 1MB - 1Rand Read - IO_uring - No - No - 1MB - 1Seq Write - IO_uring - No - No - 4KB - 1Seq Write - IO_uring - No - No - 4KB - 1Rand Write - IO_uring - No - No - 4KB - 1Rand Write - IO_uring - No - No - 4KB - 1Rand Read - IO_uring - No - No - 4KB - 1Rand Read - IO_uring - No - No - 4KB - 1Seq Read - IO_uring - No - No - 4KB - 1Seq Read - IO_uring - No - No - 4KB - 1

05112023-TCSTfio: Rand Read - IO_uring - No - No - 4KB - 1 - Default Test Directoryfio: Rand Read - IO_uring - No - No - 4KB - 1 - Default Test Directoryfio: Seq Read - IO_uring - No - No - 64KB - 1 - Default Test Directoryfio: Seq Read - IO_uring - No - No - 64KB - 1 - Default Test Directoryfio: Seq Read - IO_uring - No - No - 4KB - 1 - Default Test Directoryfio: Seq Read - IO_uring - No - No - 4KB - 1 - Default Test Directoryfio: Rand Read - IO_uring - No - No - 1MB - 1 - Default Test Directoryfio: Rand Read - IO_uring - No - No - 1MB - 1 - Default Test Directoryfio: Rand Read - IO_uring - No - No - 64KB - 1 - Default Test Directoryfio: Rand Read - IO_uring - No - No - 64KB - 1 - Default Test Directoryfio: Seq Read - IO_uring - No - No - 1MB - 1 - Default Test Directoryfio: Seq Read - IO_uring - No - No - 1MB - 1 - Default Test Directoryfio: Seq Write - IO_uring - No - No - 1MB - 1 - Default Test Directoryfio: Seq Write - IO_uring - No - No - 1MB - 1 - Default Test Directoryfio: Seq Write - IO_uring - No - No - 64KB - 1 - Default Test Directoryfio: Seq Write - IO_uring - No - No - 64KB - 1 - Default Test Directoryfio: Rand Write - IO_uring - No - No - 1MB - 1 - Default Test Directoryfio: Rand Write - IO_uring - No - No - 1MB - 1 - Default Test Directoryfio: Rand Write - IO_uring - No - No - 64KB - 1 - Default Test Directoryfio: Rand Write - IO_uring - No - No - 64KB - 1 - Default Test Directoryfio: Seq Write - IO_uring - No - No - 4KB - 1 - Default Test Directoryfio: Seq Write - IO_uring - No - No - 4KB - 1 - Default Test Directoryfio: Rand Write - IO_uring - No - No - 4KB - 1 - Default Test Directoryfio: Rand Write - IO_uring - No - No - 4KB - 1 - Default Test DirectoryPVE-NVME-R10-4DISK-ZFS2.1PVE-NVME-R10-4DISK-ZFS2.2PVE-NVME-R10-4DISK-LVMPVE-NVME-R10-4DISK-LVM-THINXEN-NVME-R10-4DISK-EXT4PVE-NVME-R10-8DISK-ZFS2.2127708510833.62521193257.47131558526231.7340334032.60605263782.8538043803.8824202419.83271421696.3925542554.28314371964.7993001372002.8388678354712.28118691474764.96481573009.81122330489319.7338143814.21558023487.6436063606.5813781378.7110264656928.1226432643.0810978702583.6891556366223.5288991355963.94174062696247.301279967999.78164873659491.141880818807.881324188276.152003820037.9468076807.311002596266.2269226922.49979906124.38124815499262.03117717470866.56200093800373.641367458546.58178629714517.971900419003.621400158750.941969219691.6861096108.63892465577.9163926392.34775424846.36175268701071.06152386609544.81114841459364.69242101513.12118749474996.8116381637.85264681654.2314871486.9614951494.97208521303.2515041504.33217661360.37126259505037.27110091440366.12175996703983.63857405358.76173927695706.6287088708.001111296945.5566956695.3784308429.73897955612.1782258225.43881495509.34161774647097.87156992627966.71OpenBenchmarking.org

Flexible IO Tester

OpenBenchmarking.orgIOPS, More Is BetterFlexible IO Tester 3.35Type: Random Read - Engine: IO_uring - Buffered: No - Direct: No - Block Size: 4KB - Job Count: 1 - Disk Target: Default Test DirectoryPVE-NVME-R10-4DISK-ZFS2.1PVE-NVME-R10-4DISK-ZFS2.2PVE-NVME-R10-4DISK-LVMPVE-NVME-R10-4DISK-LVM-THINXEN-NVME-R10-4DISK-EXT4PVE-NVME-R10-8DISK-ZFS2.240K80K120K160K200KSE +/- 1742.30, N = 4SE +/- 1404.16, N = 4SE +/- 771.29, N = 3SE +/- 1592.60, N = 3SE +/- 920.94, N = 3SE +/- 364.67, N = 2127708118691174062200093114841175996MIN: 122932.47 / MAX: 131080.9MIN: 114916.45 / MAX: 121357.02MIN: 172527.28 / MAX: 174965.41MIN: 196961.07 / MAX: 202160.02MIN: 113187.22 / MAX: 116370.14MIN: 175631.32 / MAX: 176360.661. (CC) gcc options: -rdynamic -lm -laio -lpthread -ldl -std=gnu99 -ffast-math -include -O3 -fcommon -march=native

OpenBenchmarking.orgMiB/s, More Is BetterFlexible IO Tester 3.35Type: Random Read - Engine: IO_uring - Buffered: No - Direct: No - Block Size: 4KB - Job Count: 1 - Disk Target: Default Test DirectoryPVE-NVME-R10-4DISK-ZFS2.1PVE-NVME-R10-4DISK-ZFS2.2PVE-NVME-R10-4DISK-LVMPVE-NVME-R10-4DISK-LVM-THINXEN-NVME-R10-4DISK-EXT4PVE-NVME-R10-8DISK-ZFS2.2200K400K600K800K1000KSE +/- 6969.17, N = 4SE +/- 5616.59, N = 4SE +/- 3085.12, N = 3SE +/- 6370.47, N = 3SE +/- 3683.77, N = 3SE +/- 1458.64, N = 2510833.62474764.96696247.30800373.64459364.69703983.63MIN: 491730.18 / MAX: 524323.85MIN: 459665.87 / MAX: 485427.97MIN: 690109.28 / MAX: 699861.68MIN: 787844.15 / MAX: 808640.32MIN: 452749.3 / MAX: 465480.92MIN: 702524.99 / MAX: 705442.261. (CC) gcc options: -rdynamic -lm -laio -lpthread -ldl -std=gnu99 -ffast-math -include -O3 -fcommon -march=native

OpenBenchmarking.orgIOPS, More Is BetterFlexible IO Tester 3.35Type: Sequential Read - Engine: IO_uring - Buffered: No - Direct: No - Block Size: 64KB - Job Count: 1 - Disk Target: Default Test DirectoryPVE-NVME-R10-4DISK-ZFS2.1PVE-NVME-R10-4DISK-ZFS2.2PVE-NVME-R10-4DISK-LVMPVE-NVME-R10-4DISK-LVM-THINXEN-NVME-R10-4DISK-EXT4PVE-NVME-R10-8DISK-ZFS2.230K60K90K120K150KSE +/- 98.17, N = 4SE +/- 62.81, N = 4SE +/- 1105.99, N = 3SE +/- 1493.37, N = 3SE +/- 385.98, N = 3SE +/- 213.50, N = 252119481571279961367452421085740MIN: 51928.62 / MAX: 52312.58MIN: 47997.17 / MAX: 48303.78MIN: 125799.47 / MAX: 129317.81MIN: 133827.61 / MAX: 138757.15MIN: 23570.81 / MAX: 24904.42MIN: 85526.58 / MAX: 85953.591. (CC) gcc options: -rdynamic -lm -laio -lpthread -ldl -std=gnu99 -ffast-math -include -O3 -fcommon -march=native

OpenBenchmarking.orgMiB/s, More Is BetterFlexible IO Tester 3.35Type: Sequential Read - Engine: IO_uring - Buffered: No - Direct: No - Block Size: 64KB - Job Count: 1 - Disk Target: Default Test DirectoryPVE-NVME-R10-4DISK-ZFS2.1PVE-NVME-R10-4DISK-ZFS2.2PVE-NVME-R10-4DISK-LVMPVE-NVME-R10-4DISK-LVM-THINXEN-NVME-R10-4DISK-EXT4PVE-NVME-R10-8DISK-ZFS2.22K4K6K8K10KSE +/- 6.14, N = 4SE +/- 3.93, N = 4SE +/- 69.13, N = 3SE +/- 93.33, N = 3SE +/- 24.12, N = 3SE +/- 13.35, N = 23257.473009.817999.788546.581513.125358.76MIN: 3245.54 / MAX: 3269.54MIN: 2999.83 / MAX: 3018.99MIN: 7862.47 / MAX: 8082.37MIN: 8364.23 / MAX: 8672.32MIN: 5345.41 / MAX: 5372.11. (CC) gcc options: -rdynamic -lm -laio -lpthread -ldl -std=gnu99 -ffast-math -include -O3 -fcommon -march=native

OpenBenchmarking.orgIOPS, More Is BetterFlexible IO Tester 3.35Type: Sequential Read - Engine: IO_uring - Buffered: No - Direct: No - Block Size: 4KB - Job Count: 1 - Disk Target: Default Test DirectoryPVE-NVME-R10-4DISK-ZFS2.1PVE-NVME-R10-4DISK-ZFS2.2PVE-NVME-R10-4DISK-LVMPVE-NVME-R10-4DISK-LVM-THINXEN-NVME-R10-4DISK-EXT4PVE-NVME-R10-8DISK-ZFS2.240K80K120K160K200KSE +/- 368.35, N = 4SE +/- 941.64, N = 4SE +/- 221.27, N = 3SE +/- 850.37, N = 3SE +/- 10833.35, N = 3SE +/- 115.40, N = 2131558122330164873178629118749173927MIN: 130702.69 / MAX: 132302.27MIN: 119532.43 / MAX: 123585.36MIN: 164496.33 / MAX: 165262.48MIN: 176968.27 / MAX: 179775.83MIN: 106387.39 / MAX: 140340.22MIN: 173811.28 / MAX: 174042.091. (CC) gcc options: -rdynamic -lm -laio -lpthread -ldl -std=gnu99 -ffast-math -include -O3 -fcommon -march=native

OpenBenchmarking.orgMiB/s, More Is BetterFlexible IO Tester 3.35Type: Sequential Read - Engine: IO_uring - Buffered: No - Direct: No - Block Size: 4KB - Job Count: 1 - Disk Target: Default Test DirectoryPVE-NVME-R10-4DISK-ZFS2.1PVE-NVME-R10-4DISK-ZFS2.2PVE-NVME-R10-4DISK-LVMPVE-NVME-R10-4DISK-LVM-THINXEN-NVME-R10-4DISK-EXT4PVE-NVME-R10-8DISK-ZFS2.2150K300K450K600K750KSE +/- 1473.41, N = 4SE +/- 3766.56, N = 4SE +/- 885.09, N = 3SE +/- 3401.42, N = 3SE +/- 43333.44, N = 3SE +/- 461.66, N = 2526231.73489319.73659491.14714517.97474996.81695706.62MIN: 522810.86 / MAX: 529209.12MIN: 478129.83 / MAX: 494341.44MIN: 657985.42 / MAX: 661050.08MIN: 707873.32 / MAX: 719103.4MIN: 425549.84 / MAX: 561361.27MIN: 695244.96 / MAX: 696168.281. (CC) gcc options: -rdynamic -lm -laio -lpthread -ldl -std=gnu99 -ffast-math -include -O3 -fcommon -march=native

OpenBenchmarking.orgIOPS, More Is BetterFlexible IO Tester 3.35Type: Random Read - Engine: IO_uring - Buffered: No - Direct: No - Block Size: 1MB - Job Count: 1 - Disk Target: Default Test DirectoryPVE-NVME-R10-4DISK-ZFS2.1PVE-NVME-R10-4DISK-ZFS2.2PVE-NVME-R10-4DISK-LVMPVE-NVME-R10-4DISK-LVM-THINXEN-NVME-R10-4DISK-EXT4PVE-NVME-R10-8DISK-ZFS2.24K8K12K16K20KSE +/- 2.23, N = 4SE +/- 2.10, N = 4SE +/- 69.50, N = 3SE +/- 203.09, N = 3SE +/- 6.83, N = 3SE +/- 111.63, N = 240333814188081900416388708MIN: 4028.6 / MAX: 4038.78MIN: 3810.7 / MAX: 3819.55MIN: 18689.13 / MAX: 18929.83MIN: 18621.31 / MAX: 19313.65MIN: 8596.19 / MAX: 8819.451. (CC) gcc options: -rdynamic -lm -laio -lpthread -ldl -std=gnu99 -ffast-math -include -O3 -fcommon -march=native

OpenBenchmarking.orgMiB/s, More Is BetterFlexible IO Tester 3.35Type: Random Read - Engine: IO_uring - Buffered: No - Direct: No - Block Size: 1MB - Job Count: 1 - Disk Target: Default Test DirectoryPVE-NVME-R10-4DISK-ZFS2.1PVE-NVME-R10-4DISK-ZFS2.2PVE-NVME-R10-4DISK-LVMPVE-NVME-R10-4DISK-LVM-THINXEN-NVME-R10-4DISK-EXT4PVE-NVME-R10-8DISK-ZFS2.24K8K12K16K20KSE +/- 2.23, N = 4SE +/- 2.11, N = 4SE +/- 69.50, N = 3SE +/- 203.10, N = 3SE +/- 6.83, N = 3SE +/- 111.65, N = 24032.603814.2118807.8819003.621637.858708.00MIN: 4028.61 / MAX: 4038.79MIN: 3810.97 / MAX: 3819.87MIN: 18689.17 / MAX: 18929.86MIN: 18621.35 / MAX: 19313.72MIN: 8596.35 / MAX: 8819.641. (CC) gcc options: -rdynamic -lm -laio -lpthread -ldl -std=gnu99 -ffast-math -include -O3 -fcommon -march=native

OpenBenchmarking.orgIOPS, More Is BetterFlexible IO Tester 3.35Type: Random Read - Engine: IO_uring - Buffered: No - Direct: No - Block Size: 64KB - Job Count: 1 - Disk Target: Default Test DirectoryPVE-NVME-R10-4DISK-ZFS2.1PVE-NVME-R10-4DISK-ZFS2.2PVE-NVME-R10-4DISK-LVMPVE-NVME-R10-4DISK-LVM-THINXEN-NVME-R10-4DISK-EXT4PVE-NVME-R10-8DISK-ZFS2.230K60K90K120K150KSE +/- 251.91, N = 4SE +/- 82.11, N = 4SE +/- 1734.03, N = 3SE +/- 1973.06, N = 3SE +/- 28.67, N = 3SE +/- 112.60, N = 2605265580213241814001526468111129MIN: 60127.11 / MAX: 61228.36MIN: 55571.32 / MAX: 55958.14MIN: 128981.55 / MAX: 134538.74MIN: 137810.86 / MAX: 143951.73MIN: 26426.26 / MAX: 26522.63MIN: 111016.18 / MAX: 111241.371. (CC) gcc options: -rdynamic -lm -laio -lpthread -ldl -std=gnu99 -ffast-math -include -O3 -fcommon -march=native

OpenBenchmarking.orgMiB/s, More Is BetterFlexible IO Tester 3.35Type: Random Read - Engine: IO_uring - Buffered: No - Direct: No - Block Size: 64KB - Job Count: 1 - Disk Target: Default Test DirectoryPVE-NVME-R10-4DISK-ZFS2.1PVE-NVME-R10-4DISK-ZFS2.2PVE-NVME-R10-4DISK-LVMPVE-NVME-R10-4DISK-LVM-THINXEN-NVME-R10-4DISK-EXT4PVE-NVME-R10-8DISK-ZFS2.22K4K6K8K10KSE +/- 15.75, N = 4SE +/- 5.13, N = 4SE +/- 108.38, N = 3SE +/- 123.32, N = 3SE +/- 1.79, N = 3SE +/- 7.03, N = 23782.853487.648276.158750.941654.236945.55MIN: 3757.95 / MAX: 3826.78MIN: 3473.22 / MAX: 3497.39MIN: 8061.35 / MAX: 8408.67MIN: 8613.18 / MAX: 8996.99MIN: 1651.65 / MAX: 1657.67MIN: 6938.51 / MAX: 6952.581. (CC) gcc options: -rdynamic -lm -laio -lpthread -ldl -std=gnu99 -ffast-math -include -O3 -fcommon -march=native

OpenBenchmarking.orgIOPS, More Is BetterFlexible IO Tester 3.35Type: Sequential Read - Engine: IO_uring - Buffered: No - Direct: No - Block Size: 1MB - Job Count: 1 - Disk Target: Default Test DirectoryPVE-NVME-R10-4DISK-ZFS2.1PVE-NVME-R10-4DISK-ZFS2.2PVE-NVME-R10-4DISK-LVMPVE-NVME-R10-4DISK-LVM-THINXEN-NVME-R10-4DISK-EXT4PVE-NVME-R10-8DISK-ZFS2.24K8K12K16K20KSE +/- 2.29, N = 4SE +/- 0.68, N = 4SE +/- 89.43, N = 3SE +/- 246.63, N = 3SE +/- 5.92, N = 3SE +/- 11.67, N = 238043606200381969214876695MIN: 3797.73 / MAX: 3808.69MIN: 3604.97 / MAX: 3607.71MIN: 19859.03 / MAX: 20129.09MIN: 19239.28 / MAX: 20088.16MIN: 6683.45 / MAX: 6706.781. (CC) gcc options: -rdynamic -lm -laio -lpthread -ldl -std=gnu99 -ffast-math -include -O3 -fcommon -march=native

OpenBenchmarking.orgMiB/s, More Is BetterFlexible IO Tester 3.35Type: Sequential Read - Engine: IO_uring - Buffered: No - Direct: No - Block Size: 1MB - Job Count: 1 - Disk Target: Default Test DirectoryPVE-NVME-R10-4DISK-ZFS2.1PVE-NVME-R10-4DISK-ZFS2.2PVE-NVME-R10-4DISK-LVMPVE-NVME-R10-4DISK-LVM-THINXEN-NVME-R10-4DISK-EXT4PVE-NVME-R10-8DISK-ZFS2.24K8K12K16K20KSE +/- 2.30, N = 4SE +/- 0.67, N = 4SE +/- 89.44, N = 3SE +/- 246.64, N = 3SE +/- 5.87, N = 3SE +/- 11.65, N = 23803.883606.5820037.9419691.681486.966695.37MIN: 3797.8 / MAX: 3808.79MIN: 3605.37 / MAX: 3608.03MIN: 19859.08 / MAX: 20129.14MIN: 19239.35 / MAX: 20088.27MIN: 6683.72 / MAX: 6707.021. (CC) gcc options: -rdynamic -lm -laio -lpthread -ldl -std=gnu99 -ffast-math -include -O3 -fcommon -march=native

OpenBenchmarking.orgIOPS, More Is BetterFlexible IO Tester 3.35Type: Sequential Write - Engine: IO_uring - Buffered: No - Direct: No - Block Size: 1MB - Job Count: 1 - Disk Target: Default Test DirectoryPVE-NVME-R10-4DISK-ZFS2.1PVE-NVME-R10-4DISK-ZFS2.2PVE-NVME-R10-4DISK-LVMPVE-NVME-R10-4DISK-LVM-THINXEN-NVME-R10-4DISK-EXT4PVE-NVME-R10-8DISK-ZFS2.22K4K6K8K10KSE +/- 101.00, N = 4SE +/- 16.25, N = 4SE +/- 99.06, N = 3SE +/- 63.77, N = 3SE +/- 48.17, N = 3SE +/- 13.27, N = 2242013786807610914958430MIN: 2216.08 / MAX: 2646.04MIN: 1352.95 / MAX: 1424.59MIN: 6655.38 / MAX: 6993.37MIN: 6018.52 / MAX: 6231.83MIN: 1398.93 / MAX: 1551.64MIN: 8416.33 / MAX: 8442.871. (CC) gcc options: -rdynamic -lm -laio -lpthread -ldl -std=gnu99 -ffast-math -include -O3 -fcommon -march=native

OpenBenchmarking.orgMiB/s, More Is BetterFlexible IO Tester 3.35Type: Sequential Write - Engine: IO_uring - Buffered: No - Direct: No - Block Size: 1MB - Job Count: 1 - Disk Target: Default Test DirectoryPVE-NVME-R10-4DISK-ZFS2.1PVE-NVME-R10-4DISK-ZFS2.2PVE-NVME-R10-4DISK-LVMPVE-NVME-R10-4DISK-LVM-THINXEN-NVME-R10-4DISK-EXT4PVE-NVME-R10-8DISK-ZFS2.22K4K6K8K10KSE +/- 100.97, N = 4SE +/- 16.25, N = 4SE +/- 99.06, N = 3SE +/- 63.77, N = 3SE +/- 48.17, N = 3SE +/- 13.29, N = 22419.831378.716807.316108.631494.978429.73MIN: 2216.36 / MAX: 2646.2MIN: 6655.42 / MAX: 6993.41MIN: 6018.55 / MAX: 6231.87MIN: 8416.44 / MAX: 8443.021. (CC) gcc options: -rdynamic -lm -laio -lpthread -ldl -std=gnu99 -ffast-math -include -O3 -fcommon -march=native

OpenBenchmarking.orgIOPS, More Is BetterFlexible IO Tester 3.35Type: Sequential Write - Engine: IO_uring - Buffered: No - Direct: No - Block Size: 64KB - Job Count: 1 - Disk Target: Default Test DirectoryPVE-NVME-R10-4DISK-ZFS2.1PVE-NVME-R10-4DISK-ZFS2.2PVE-NVME-R10-4DISK-LVMPVE-NVME-R10-4DISK-LVM-THINXEN-NVME-R10-4DISK-EXT4PVE-NVME-R10-8DISK-ZFS2.220K40K60K80K100KSE +/- 419.10, N = 4SE +/- 288.54, N = 4SE +/- 1523.58, N = 3SE +/- 3443.78, N = 3SE +/- 235.86, N = 3SE +/- 608.82, N = 22714210264100259892462085289795MIN: 26088.08 / MAX: 27844.44MIN: 97220.11 / MAX: 101967.93MIN: 84204.33 / MAX: 95830.97MIN: 20567.79 / MAX: 21320.09MIN: 89185.9 / MAX: 90403.531. (CC) gcc options: -rdynamic -lm -laio -lpthread -ldl -std=gnu99 -ffast-math -include -O3 -fcommon -march=native

OpenBenchmarking.orgMiB/s, More Is BetterFlexible IO Tester 3.35Type: Sequential Write - Engine: IO_uring - Buffered: No - Direct: No - Block Size: 64KB - Job Count: 1 - Disk Target: Default Test DirectoryPVE-NVME-R10-4DISK-ZFS2.1PVE-NVME-R10-4DISK-ZFS2.2PVE-NVME-R10-4DISK-LVMPVE-NVME-R10-4DISK-LVM-THINXEN-NVME-R10-4DISK-EXT4PVE-NVME-R10-8DISK-ZFS2.2140K280K420K560K700KSE +/- 26.19, N = 4SE +/- 18466.98, N = 4SE +/- 95.22, N = 3SE +/- 215.24, N = 3SE +/- 14.74, N = 3SE +/- 38.05, N = 21696.39656928.126266.225577.911303.255612.17MIN: 603671.44 / MAX: 688537.191. (CC) gcc options: -rdynamic -lm -laio -lpthread -ldl -std=gnu99 -ffast-math -include -O3 -fcommon -march=native

OpenBenchmarking.orgIOPS, More Is BetterFlexible IO Tester 3.35Type: Random Write - Engine: IO_uring - Buffered: No - Direct: No - Block Size: 1MB - Job Count: 1 - Disk Target: Default Test DirectoryPVE-NVME-R10-4DISK-ZFS2.1PVE-NVME-R10-4DISK-ZFS2.2PVE-NVME-R10-4DISK-LVMPVE-NVME-R10-4DISK-LVM-THINXEN-NVME-R10-4DISK-EXT4PVE-NVME-R10-8DISK-ZFS2.22K4K6K8K10KSE +/- 50.98, N = 4SE +/- 10.38, N = 4SE +/- 31.06, N = 3SE +/- 53.32, N = 3SE +/- 123.07, N = 3SE +/- 20.71, N = 2255426436922639215048225MIN: 2405.24 / MAX: 2621MIN: 2613.24 / MAX: 2660.24MIN: 6861.28 / MAX: 6962.45MIN: 6335.18 / MAX: 6498.81MIN: 1372.1 / MAX: 1749.99MIN: 8204.63 / MAX: 8246.051. (CC) gcc options: -rdynamic -lm -laio -lpthread -ldl -std=gnu99 -ffast-math -include -O3 -fcommon -march=native

OpenBenchmarking.orgMiB/s, More Is BetterFlexible IO Tester 3.35Type: Random Write - Engine: IO_uring - Buffered: No - Direct: No - Block Size: 1MB - Job Count: 1 - Disk Target: Default Test DirectoryPVE-NVME-R10-4DISK-ZFS2.1PVE-NVME-R10-4DISK-ZFS2.2PVE-NVME-R10-4DISK-LVMPVE-NVME-R10-4DISK-LVM-THINXEN-NVME-R10-4DISK-EXT4PVE-NVME-R10-8DISK-ZFS2.22K4K6K8K10KSE +/- 50.98, N = 4SE +/- 10.38, N = 4SE +/- 31.04, N = 3SE +/- 53.32, N = 3SE +/- 123.03, N = 3SE +/- 20.70, N = 22554.282643.086922.496392.341504.338225.43MIN: 2405.5 / MAX: 2621.26MIN: 2613.46 / MAX: 2660.47MIN: 6861.37 / MAX: 6962.5MIN: 6335.25 / MAX: 6498.89MIN: 8204.72 / MAX: 8246.131. (CC) gcc options: -rdynamic -lm -laio -lpthread -ldl -std=gnu99 -ffast-math -include -O3 -fcommon -march=native

OpenBenchmarking.orgIOPS, More Is BetterFlexible IO Tester 3.35Type: Random Write - Engine: IO_uring - Buffered: No - Direct: No - Block Size: 64KB - Job Count: 1 - Disk Target: Default Test DirectoryPVE-NVME-R10-4DISK-ZFS2.1PVE-NVME-R10-4DISK-ZFS2.2PVE-NVME-R10-4DISK-LVMPVE-NVME-R10-4DISK-LVM-THINXEN-NVME-R10-4DISK-EXT4PVE-NVME-R10-8DISK-ZFS2.220K40K60K80K100KSE +/- 160.70, N = 4SE +/- 18.77, N = 4SE +/- 194.67, N = 3SE +/- 142.13, N = 3SE +/- 779.05, N = 3SE +/- 360.52, N = 2314371097897990775422176688149MIN: 31030.13 / MAX: 31758.5MIN: 97740.13 / MAX: 98373.57MIN: 77272.31 / MAX: 77754.9MIN: 20720.73 / MAX: 23289.14MIN: 87788.79 / MAX: 88509.831. (CC) gcc options: -rdynamic -lm -laio -lpthread -ldl -std=gnu99 -ffast-math -include -O3 -fcommon -march=native

OpenBenchmarking.orgMiB/s, More Is BetterFlexible IO Tester 3.35Type: Random Write - Engine: IO_uring - Buffered: No - Direct: No - Block Size: 64KB - Job Count: 1 - Disk Target: Default Test DirectoryPVE-NVME-R10-4DISK-ZFS2.1PVE-NVME-R10-4DISK-ZFS2.2PVE-NVME-R10-4DISK-LVMPVE-NVME-R10-4DISK-LVM-THINXEN-NVME-R10-4DISK-EXT4PVE-NVME-R10-8DISK-ZFS2.2150K300K450K600K750KSE +/- 10.04, N = 4SE +/- 1201.47, N = 4SE +/- 12.17, N = 3SE +/- 8.88, N = 3SE +/- 48.69, N = 3SE +/- 22.53, N = 21964.79702583.686124.384846.361360.375509.34MIN: 700032.56 / MAX: 705692.671. (CC) gcc options: -rdynamic -lm -laio -lpthread -ldl -std=gnu99 -ffast-math -include -O3 -fcommon -march=native

OpenBenchmarking.orgIOPS, More Is BetterFlexible IO Tester 3.35Type: Sequential Write - Engine: IO_uring - Buffered: No - Direct: No - Block Size: 4KB - Job Count: 1 - Disk Target: Default Test DirectoryPVE-NVME-R10-4DISK-ZFS2.1PVE-NVME-R10-4DISK-ZFS2.2PVE-NVME-R10-4DISK-LVMPVE-NVME-R10-4DISK-LVM-THINXEN-NVME-R10-4DISK-EXT4PVE-NVME-R10-8DISK-ZFS2.240K80K120K160K200KSE +/- 612.87, N = 4SE +/- 471.34, N = 4SE +/- 780.03, N = 3SE +/- 2142.72, N = 3SE +/- 11854.44, N = 3SE +/- 635.79, N = 29300191556124815175268126259161774MIN: 91598.8 / MAX: 94217.31MIN: 90268.39 / MAX: 92441.34MIN: 123408.48 / MAX: 126102.61MIN: 171044.61 / MAX: 178009.88MIN: 109325.07 / MAX: 149096.74MIN: 161138.67 / MAX: 162410.251. (CC) gcc options: -rdynamic -lm -laio -lpthread -ldl -std=gnu99 -ffast-math -include -O3 -fcommon -march=native

OpenBenchmarking.orgMiB/s, More Is BetterFlexible IO Tester 3.35Type: Sequential Write - Engine: IO_uring - Buffered: No - Direct: No - Block Size: 4KB - Job Count: 1 - Disk Target: Default Test DirectoryPVE-NVME-R10-4DISK-ZFS2.1PVE-NVME-R10-4DISK-ZFS2.2PVE-NVME-R10-4DISK-LVMPVE-NVME-R10-4DISK-LVM-THINXEN-NVME-R10-4DISK-EXT4PVE-NVME-R10-8DISK-ZFS2.2150K300K450K600K750KSE +/- 2451.46, N = 4SE +/- 1885.36, N = 4SE +/- 3120.10, N = 3SE +/- 8570.97, N = 3SE +/- 47417.67, N = 3SE +/- 2543.12, N = 2372002.83366223.52499262.03701071.06505037.27647097.87MIN: 366395.47 / MAX: 376869.38MIN: 361073.65 / MAX: 369765.43MIN: 493634.13 / MAX: 504410.54MIN: 684178.38 / MAX: 712039.82MIN: 437300.68 / MAX: 596387.05MIN: 644554.75 / MAX: 649640.991. (CC) gcc options: -rdynamic -lm -laio -lpthread -ldl -std=gnu99 -ffast-math -include -O3 -fcommon -march=native

OpenBenchmarking.orgIOPS, More Is BetterFlexible IO Tester 3.35Type: Random Write - Engine: IO_uring - Buffered: No - Direct: No - Block Size: 4KB - Job Count: 1 - Disk Target: Default Test DirectoryPVE-NVME-R10-4DISK-ZFS2.1PVE-NVME-R10-4DISK-ZFS2.2PVE-NVME-R10-4DISK-LVMPVE-NVME-R10-4DISK-LVM-THINXEN-NVME-R10-4DISK-EXT4PVE-NVME-R10-8DISK-ZFS2.230K60K90K120K150KSE +/- 519.90, N = 4SE +/- 766.78, N = 4SE +/- 340.68, N = 3SE +/- 845.39, N = 3SE +/- 2883.09, N = 3SE +/- 1995.35, N = 28867888991117717152386110091156992MIN: 87337.43 / MAX: 89676.77MIN: 87994.01 / MAX: 91232.19MIN: 117255.4 / MAX: 118381.57MIN: 150996.74 / MAX: 153915.28MIN: 106581.88 / MAX: 115808.42MIN: 154996.32 / MAX: 158987.021. (CC) gcc options: -rdynamic -lm -laio -lpthread -ldl -std=gnu99 -ffast-math -include -O3 -fcommon -march=native

OpenBenchmarking.orgMiB/s, More Is BetterFlexible IO Tester 3.35Type: Random Write - Engine: IO_uring - Buffered: No - Direct: No - Block Size: 4KB - Job Count: 1 - Disk Target: Default Test DirectoryPVE-NVME-R10-4DISK-ZFS2.1PVE-NVME-R10-4DISK-ZFS2.2PVE-NVME-R10-4DISK-LVMPVE-NVME-R10-4DISK-LVM-THINXEN-NVME-R10-4DISK-EXT4PVE-NVME-R10-8DISK-ZFS2.2130K260K390K520K650KSE +/- 2079.65, N = 4SE +/- 3067.11, N = 4SE +/- 1362.71, N = 3SE +/- 3381.35, N = 3SE +/- 11532.29, N = 3SE +/- 7981.55, N = 2354712.28355963.94470866.56609544.81440366.12627966.71MIN: 349349.75 / MAX: 358707.21MIN: 351976.19 / MAX: 364928.78MIN: 469021.73 / MAX: 473526.32MIN: 603987.04 / MAX: 615660.46MIN: 426327.83 / MAX: 463233.79MIN: 619985.16 / MAX: 635948.261. (CC) gcc options: -rdynamic -lm -laio -lpthread -ldl -std=gnu99 -ffast-math -include -O3 -fcommon -march=native

24 Results Shown

Flexible IO Tester:
  Rand Read - IO_uring - No - No - 4KB - 1 - Default Test Directory:
    IOPS
    MiB/s
  Seq Read - IO_uring - No - No - 64KB - 1 - Default Test Directory:
    IOPS
    MiB/s
  Seq Read - IO_uring - No - No - 4KB - 1 - Default Test Directory:
    IOPS
    MiB/s
  Rand Read - IO_uring - No - No - 1MB - 1 - Default Test Directory:
    IOPS
    MiB/s
  Rand Read - IO_uring - No - No - 64KB - 1 - Default Test Directory:
    IOPS
    MiB/s
  Seq Read - IO_uring - No - No - 1MB - 1 - Default Test Directory:
    IOPS
    MiB/s
  Seq Write - IO_uring - No - No - 1MB - 1 - Default Test Directory:
    IOPS
    MiB/s
  Seq Write - IO_uring - No - No - 64KB - 1 - Default Test Directory:
    IOPS
    MiB/s
  Rand Write - IO_uring - No - No - 1MB - 1 - Default Test Directory:
    IOPS
    MiB/s
  Rand Write - IO_uring - No - No - 64KB - 1 - Default Test Directory:
    IOPS
    MiB/s
  Seq Write - IO_uring - No - No - 4KB - 1 - Default Test Directory:
    IOPS
    MiB/s
  Rand Write - IO_uring - No - No - 4KB - 1 - Default Test Directory:
    IOPS
    MiB/s

PVE-NVME-R10-4DISK-ZFS2.1

Processor: AMD EPYC 7543 32-Core (12 Cores), Motherboard: QEMU Standard PC (Q35 + ICH9 2009) (3.20230228-4 BIOS), Chipset: Intel 82G33/G31/P35/P31 + ICH9, Memory: 16GB, Disk: 34GB QEMU HDD, Graphics: bochs-drmdrmfb, Audio: Intel 82801I, Monitor: QEMU Monitor, Network: Red Hat Virtio device

OS: Ubuntu 22.04, Kernel: 5.15.0-86-generic (x86_64), Vulkan: 1.3.238, Compiler: GCC 11.4.0, File-System: ext4, Screen Resolution: 1280x800, System Layer: KVM

Kernel Notes: Transparent Huge Pages: madvise
Compiler Notes: --build=x86_64-linux-gnu --disable-vtable-verify --disable-werror --enable-bootstrap --enable-cet --enable-checking=release --enable-clocale=gnu --enable-default-pie --enable-gnu-unique-object --enable-languages=c,ada,c++,go,brig,d,fortran,objc,obj-c++,m2 --enable-libphobos-checking=release --enable-libstdcxx-debug --enable-libstdcxx-time=yes --enable-link-serialization=2 --enable-multiarch --enable-multilib --enable-nls --enable-objc-gc=auto --enable-offload-targets=nvptx-none=/build/gcc-11-XeT9lY/gcc-11-11.4.0/debian/tmp-nvptx/usr,amdgcn-amdhsa=/build/gcc-11-XeT9lY/gcc-11-11.4.0/debian/tmp-gcn/usr --enable-plugin --enable-shared --enable-threads=posix --host=x86_64-linux-gnu --program-prefix=x86_64-linux-gnu- --target=x86_64-linux-gnu --with-abi=m64 --with-arch-32=i686 --with-build-config=bootstrap-lto-lean --with-default-libstdcxx-abi=new --with-gcc-major-version-only --with-multilib-list=m32,m64,mx32 --with-target-system-zlib=auto --with-tune=generic --without-cuda-driver -v
Disk Notes: NONE / relatime,rw / Block Size: 4096
Processor Notes: CPU Microcode: 0xa00110d
Security Notes: gather_data_sampling: Not affected + itlb_multihit: Not affected + l1tf: Not affected + mds: Not affected + meltdown: Not affected + mmio_stale_data: Not affected + retbleed: Not affected + spec_rstack_overflow: Mitigation of safe RET no microcode + spec_store_bypass: Mitigation of SSB disabled via prctl and seccomp + spectre_v1: Mitigation of usercopy/swapgs barriers and __user pointer sanitization + spectre_v2: Mitigation of Retpolines IBPB: conditional IBRS_FW STIBP: disabled RSB filling PBRSB-eIBRS: Not affected + srbds: Not affected + tsx_async_abort: Not affected

Testing initiated at 5 November 2023 00:50 by user tcstudios.

PVE-NVME-R10-4DISK-ZFS2.2

Kernel Notes: Transparent Huge Pages: madvise
Compiler Notes: --build=x86_64-linux-gnu --disable-vtable-verify --disable-werror --enable-bootstrap --enable-cet --enable-checking=release --enable-clocale=gnu --enable-default-pie --enable-gnu-unique-object --enable-languages=c,ada,c++,go,brig,d,fortran,objc,obj-c++,m2 --enable-libphobos-checking=release --enable-libstdcxx-debug --enable-libstdcxx-time=yes --enable-link-serialization=2 --enable-multiarch --enable-multilib --enable-nls --enable-objc-gc=auto --enable-offload-targets=nvptx-none=/build/gcc-11-XeT9lY/gcc-11-11.4.0/debian/tmp-nvptx/usr,amdgcn-amdhsa=/build/gcc-11-XeT9lY/gcc-11-11.4.0/debian/tmp-gcn/usr --enable-plugin --enable-shared --enable-threads=posix --host=x86_64-linux-gnu --program-prefix=x86_64-linux-gnu- --target=x86_64-linux-gnu --with-abi=m64 --with-arch-32=i686 --with-build-config=bootstrap-lto-lean --with-default-libstdcxx-abi=new --with-gcc-major-version-only --with-multilib-list=m32,m64,mx32 --with-target-system-zlib=auto --with-tune=generic --without-cuda-driver -v
Disk Notes: NONE / relatime,rw / Block Size: 4096
Processor Notes: CPU Microcode: 0xa00110d
Security Notes: gather_data_sampling: Not affected + itlb_multihit: Not affected + l1tf: Not affected + mds: Not affected + meltdown: Not affected + mmio_stale_data: Not affected + retbleed: Not affected + spec_rstack_overflow: Mitigation of safe RET no microcode + spec_store_bypass: Mitigation of SSB disabled via prctl and seccomp + spectre_v1: Mitigation of usercopy/swapgs barriers and __user pointer sanitization + spectre_v2: Mitigation of Retpolines IBPB: conditional IBRS_FW STIBP: disabled RSB filling PBRSB-eIBRS: Not affected + srbds: Not affected + tsx_async_abort: Not affected

Testing initiated at 5 November 2023 02:59 by user tcstudios.

PVE-NVME-R10-4DISK-LVM

Processor: AMD EPYC 7543 32-Core (12 Cores), Motherboard: QEMU Standard PC (Q35 + ICH9 2009) (3.20230228-4 BIOS), Chipset: Intel 82G33/G31/P35/P31 + ICH9, Memory: 16GB, Disk: 34GB QEMU HDD, Graphics: bochs-drmdrmfb, Audio: Intel 82801I, Monitor: QEMU Monitor, Network: Red Hat Virtio device

OS: Ubuntu 22.04, Kernel: 5.15.0-88-generic (x86_64), Vulkan: 1.3.238, Compiler: GCC 11.4.0, File-System: ext4, Screen Resolution: 1280x800, System Layer: KVM

Kernel Notes: Transparent Huge Pages: madvise
Compiler Notes: --build=x86_64-linux-gnu --disable-vtable-verify --disable-werror --enable-bootstrap --enable-cet --enable-checking=release --enable-clocale=gnu --enable-default-pie --enable-gnu-unique-object --enable-languages=c,ada,c++,go,brig,d,fortran,objc,obj-c++,m2 --enable-libphobos-checking=release --enable-libstdcxx-debug --enable-libstdcxx-time=yes --enable-link-serialization=2 --enable-multiarch --enable-multilib --enable-nls --enable-objc-gc=auto --enable-offload-targets=nvptx-none=/build/gcc-11-XeT9lY/gcc-11-11.4.0/debian/tmp-nvptx/usr,amdgcn-amdhsa=/build/gcc-11-XeT9lY/gcc-11-11.4.0/debian/tmp-gcn/usr --enable-plugin --enable-shared --enable-threads=posix --host=x86_64-linux-gnu --program-prefix=x86_64-linux-gnu- --target=x86_64-linux-gnu --with-abi=m64 --with-arch-32=i686 --with-build-config=bootstrap-lto-lean --with-default-libstdcxx-abi=new --with-gcc-major-version-only --with-multilib-list=m32,m64,mx32 --with-target-system-zlib=auto --with-tune=generic --without-cuda-driver -v
Disk Notes: NONE / relatime,rw / Block Size: 4096
Processor Notes: CPU Microcode: 0xa00110d
Security Notes: gather_data_sampling: Not affected + itlb_multihit: Not affected + l1tf: Not affected + mds: Not affected + meltdown: Not affected + mmio_stale_data: Not affected + retbleed: Not affected + spec_rstack_overflow: Mitigation of safe RET no microcode + spec_store_bypass: Mitigation of SSB disabled via prctl and seccomp + spectre_v1: Mitigation of usercopy/swapgs barriers and __user pointer sanitization + spectre_v2: Mitigation of Retpolines IBPB: conditional IBRS_FW STIBP: disabled RSB filling PBRSB-eIBRS: Not affected + srbds: Not affected + tsx_async_abort: Not affected

Testing initiated at 15 November 2023 06:29 by user tcstudios.

PVE-NVME-R10-4DISK-LVM-THIN

Processor: AMD EPYC 9354 32-Core (12 Cores), Motherboard: QEMU Standard PC (Q35 + ICH9 2009) (4.2023.08-2 BIOS), Chipset: Intel 82G33/G31/P35/P31 + ICH9, Memory: 16GB, Disk: 34GB QEMU HDD, Graphics: bochs-drmdrmfb, Audio: Intel 82801I, Monitor: QEMU Monitor, Network: Red Hat Virtio device

OS: Ubuntu 22.04, Kernel: 5.15.0-84-generic (x86_64), Vulkan: 1.3.238, Compiler: GCC 11.4.0, File-System: ext4, Screen Resolution: 1280x800, System Layer: KVM

Kernel Notes: Transparent Huge Pages: madvise
Compiler Notes: --build=x86_64-linux-gnu --disable-vtable-verify --disable-werror --enable-bootstrap --enable-cet --enable-checking=release --enable-clocale=gnu --enable-default-pie --enable-gnu-unique-object --enable-languages=c,ada,c++,go,brig,d,fortran,objc,obj-c++,m2 --enable-libphobos-checking=release --enable-libstdcxx-debug --enable-libstdcxx-time=yes --enable-link-serialization=2 --enable-multiarch --enable-multilib --enable-nls --enable-objc-gc=auto --enable-offload-targets=nvptx-none=/build/gcc-11-XeT9lY/gcc-11-11.4.0/debian/tmp-nvptx/usr,amdgcn-amdhsa=/build/gcc-11-XeT9lY/gcc-11-11.4.0/debian/tmp-gcn/usr --enable-plugin --enable-shared --enable-threads=posix --host=x86_64-linux-gnu --program-prefix=x86_64-linux-gnu- --target=x86_64-linux-gnu --with-abi=m64 --with-arch-32=i686 --with-build-config=bootstrap-lto-lean --with-default-libstdcxx-abi=new --with-gcc-major-version-only --with-multilib-list=m32,m64,mx32 --with-target-system-zlib=auto --with-tune=generic --without-cuda-driver -v
Disk Notes: NONE / relatime,rw / Block Size: 4096
Processor Notes: CPU Microcode: 0xa101121
Security Notes: gather_data_sampling: Not affected + itlb_multihit: Not affected + l1tf: Not affected + mds: Not affected + meltdown: Not affected + mmio_stale_data: Not affected + retbleed: Not affected + spec_store_bypass: Mitigation of SSB disabled via prctl and seccomp + spectre_v1: Mitigation of usercopy/swapgs barriers and __user pointer sanitization + spectre_v2: Mitigation of Retpolines IBPB: conditional IBRS_FW STIBP: disabled RSB filling PBRSB-eIBRS: Not affected + srbds: Not affected + tsx_async_abort: Not affected

Testing initiated at 25 November 2023 01:54 by user tcstudios.

XEN-NVME-R10-4DISK-EXT4

Processor: AMD EPYC 9354 32-Core (12 Cores), Motherboard: Xen HVM domU v4.13, Chipset: Intel 440FX 82441FX PMC, Memory: 16GB, Disk: 30GB, Graphics: bochs-drmdrmfb

OS: Ubuntu 22.04, Kernel: 5.15.0-89-generic (x86_64), Vulkan: 1.3.238, Compiler: GCC 11.4.0, File-System: ext4, Screen Resolution: 1024x768, System Layer: Xen HVM domU 4.13

Kernel Notes: Transparent Huge Pages: madvise
Compiler Notes: --build=x86_64-linux-gnu --disable-vtable-verify --disable-werror --enable-bootstrap --enable-cet --enable-checking=release --enable-clocale=gnu --enable-default-pie --enable-gnu-unique-object --enable-languages=c,ada,c++,go,brig,d,fortran,objc,obj-c++,m2 --enable-libphobos-checking=release --enable-libstdcxx-debug --enable-libstdcxx-time=yes --enable-link-serialization=2 --enable-multiarch --enable-multilib --enable-nls --enable-objc-gc=auto --enable-offload-targets=nvptx-none=/build/gcc-11-XeT9lY/gcc-11-11.4.0/debian/tmp-nvptx/usr,amdgcn-amdhsa=/build/gcc-11-XeT9lY/gcc-11-11.4.0/debian/tmp-gcn/usr --enable-plugin --enable-shared --enable-threads=posix --host=x86_64-linux-gnu --program-prefix=x86_64-linux-gnu- --target=x86_64-linux-gnu --with-abi=m64 --with-arch-32=i686 --with-build-config=bootstrap-lto-lean --with-default-libstdcxx-abi=new --with-gcc-major-version-only --with-multilib-list=m32,m64,mx32 --with-target-system-zlib=auto --with-tune=generic --without-cuda-driver -v
Processor Notes: CPU Microcode: 0xa101144
Disk Mount Options Notes: relatime,rw
Disk Details Notes: Block Size: 4096
Security Notes: gather_data_sampling: Not affected + itlb_multihit: Not affected + l1tf: Not affected + mds: Not affected + meltdown: Not affected + mmio_stale_data: Not affected + retbleed: Not affected + spec_rstack_overflow: Mitigation of safe RET + spec_store_bypass: Mitigation of SSB disabled via prctl and seccomp + spectre_v1: Mitigation of usercopy/swapgs barriers and __user pointer sanitization + spectre_v2: Mitigation of Retpolines IBPB: conditional IBRS_FW STIBP: disabled RSB filling PBRSB-eIBRS: Not affected + srbds: Not affected + tsx_async_abort: Not affected

Testing initiated at 25 November 2023 23:01 by user tcstudios.

PVE-NVME-R10-8DISK-ZFS2.2

Processor: AMD EPYC 9354 32-Core (16 Cores), Motherboard: QEMU Standard PC (Q35 + ICH9 2009) (4.2023.08-2 BIOS), Chipset: Intel 82G33/G31/P35/P31 + ICH9, Memory: 16GB, Disk: 34GB QEMU HDD, Graphics: bochs-drmdrmfb, Audio: Intel 82801I, Monitor: QEMU Monitor, Network: Red Hat Virtio device

OS: Ubuntu 22.04, Kernel: 5.15.0-91-generic (x86_64), Vulkan: 1.3.238, Compiler: GCC 11.4.0, File-System: ext4, Screen Resolution: 1280x800, System Layer: KVM

Kernel Notes: Transparent Huge Pages: madvise
Compiler Notes: --build=x86_64-linux-gnu --disable-vtable-verify --disable-werror --enable-bootstrap --enable-cet --enable-checking=release --enable-clocale=gnu --enable-default-pie --enable-gnu-unique-object --enable-languages=c,ada,c++,go,brig,d,fortran,objc,obj-c++,m2 --enable-libphobos-checking=release --enable-libstdcxx-debug --enable-libstdcxx-time=yes --enable-link-serialization=2 --enable-multiarch --enable-multilib --enable-nls --enable-objc-gc=auto --enable-offload-targets=nvptx-none=/build/gcc-11-XeT9lY/gcc-11-11.4.0/debian/tmp-nvptx/usr,amdgcn-amdhsa=/build/gcc-11-XeT9lY/gcc-11-11.4.0/debian/tmp-gcn/usr --enable-plugin --enable-shared --enable-threads=posix --host=x86_64-linux-gnu --program-prefix=x86_64-linux-gnu- --target=x86_64-linux-gnu --with-abi=m64 --with-arch-32=i686 --with-build-config=bootstrap-lto-lean --with-default-libstdcxx-abi=new --with-gcc-major-version-only --with-multilib-list=m32,m64,mx32 --with-target-system-zlib=auto --with-tune=generic --without-cuda-driver -v
Disk Notes: NONE / relatime,rw / Block Size: 4096
Processor Notes: CPU Microcode: 0xa101121
Security Notes: gather_data_sampling: Not affected + itlb_multihit: Not affected + l1tf: Not affected + mds: Not affected + meltdown: Not affected + mmio_stale_data: Not affected + retbleed: Not affected + spec_rstack_overflow: Mitigation of safe RET no microcode + spec_store_bypass: Mitigation of SSB disabled via prctl and seccomp + spectre_v1: Mitigation of usercopy/swapgs barriers and __user pointer sanitization + spectre_v2: Mitigation of Retpolines IBPB: conditional IBRS_FW STIBP: disabled RSB filling PBRSB-eIBRS: Not affected + srbds: Not affected + tsx_async_abort: Not affected

Testing initiated at 18 December 2023 10:32 by user tcstudios.