PVE-SATA-ZFS-RAID10-6Disks

PVE-SATA-ZFS-RAID10-8Disks-EPYC7543

Compare your own system(s) to this result file with the Phoronix Test Suite by running the command: phoronix-test-suite benchmark 2310134-TCST-231008420
Jump To Table - Results

View

Do Not Show Noisy Results
Do Not Show Results With Incomplete Data
Do Not Show Results With Little Change/Spread
List Notable Results

Statistics

Show Overall Harmonic Mean(s)
Show Overall Geometric Mean
Show Wins / Losses Counts (Pie Chart)
Normalize Results
Remove Outliers Before Calculating Averages

Graph Settings

Force Line Graphs Where Applicable
Convert To Scalar Where Applicable
Disable Color Branding
Prefer Vertical Bar Graphs

Multi-Way Comparison

Condense Multi-Option Tests Into Single Result Graphs

Table

Show Detailed System Result Table

Run Management

Highlight
Result
Hide
Result
Result
Identifier
Performance Per
Dollar
Date
Run
  Test
  Duration
PVE-SATA-ZFS-RAID10-6Disks
October 02 2023
  1 Hour, 9 Minutes
PVE-SATA-ZFS-RAID10-8Disks
October 02 2023
  35 Minutes
PVE-NVME-ZFS-RAID1-2Disks
October 06 2023
  1 Hour, 10 Minutes
PVE-SATA-ZFS-RAID10-8Disks 64K rec
October 08 2023
  58 Minutes
PVE-SATA-ZFS-RAID10-8Disks-EPYC7543
October 13 2023
  59 Minutes
Invert Hiding All Results Option
  58 Minutes

Only show results where is faster than
Only show results matching title/arguments (delimit multiple options with a comma):
Do not show results matching title/arguments (delimit multiple options with a comma):

PVE-SATA-ZFS-RAID10-6Disks

Kernel Notes: Transparent Huge Pages: madvise
Compiler Notes: --build=x86_64-linux-gnu --disable-vtable-verify --disable-werror --enable-bootstrap --enable-cet --enable-checking=release --enable-clocale=gnu --enable-default-pie --enable-gnu-unique-object --enable-languages=c,ada,c++,go,brig,d,fortran,objc,obj-c++,m2 --enable-libphobos-checking=release --enable-libstdcxx-debug --enable-libstdcxx-time=yes --enable-link-serialization=2 --enable-multiarch --enable-multilib --enable-nls --enable-objc-gc=auto --enable-offload-targets=nvptx-none=/build/gcc-11-XeT9lY/gcc-11-11.4.0/debian/tmp-nvptx/usr,amdgcn-amdhsa=/build/gcc-11-XeT9lY/gcc-11-11.4.0/debian/tmp-gcn/usr --enable-plugin --enable-shared --enable-threads=posix --host=x86_64-linux-gnu --program-prefix=x86_64-linux-gnu- --target=x86_64-linux-gnu --with-abi=m64 --with-arch-32=i686 --with-build-config=bootstrap-lto-lean --with-default-libstdcxx-abi=new --with-gcc-major-version-only --with-multilib-list=m32,m64,mx32 --with-target-system-zlib=auto --with-tune=generic --without-cuda-driver -v
Disk Notes: NONE / relatime,rw / Block Size: 4096
Processor Notes: CPU Microcode: 0x830104d
Security Notes: gather_data_sampling: Not affected + itlb_multihit: Not affected + l1tf: Not affected + mds: Not affected + meltdown: Not affected + mmio_stale_data: Not affected + retbleed: Mitigation of untrained return thunk; SMT disabled + spec_store_bypass: Mitigation of SSB disabled via prctl and seccomp + spectre_v1: Mitigation of usercopy/swapgs barriers and __user pointer sanitization + spectre_v2: Mitigation of Retpolines IBPB: conditional STIBP: disabled RSB filling PBRSB-eIBRS: Not affected + srbds: Not affected + tsx_async_abort: Not affected

PVE-SATA-ZFS-RAID10-8Disks

PVE-NVME-ZFS-RAID1-2Disks

Processor: AMD EPYC 7402 24-Core (12 Cores), Motherboard: QEMU Standard PC (Q35 + ICH9 2009) (3.20230228-4 BIOS), Chipset: Intel 82G33/G31/P35/P31 + ICH9, Memory: 16GB, Disk: 34GB QEMU HDD, Graphics: bochs-drmdrmfb, Audio: Intel 82801I, Monitor: QEMU Monitor, Network: Red Hat Virtio device

OS: Ubuntu 22.04, Kernel: 5.15.0-84-generic (x86_64), Vulkan: 1.3.238, Compiler: GCC 11.4.0, File-System: ext4, Screen Resolution: 1280x800, System Layer: KVM

PVE-SATA-ZFS-RAID10-8Disks 64K rec

OS: Ubuntu 22.04, Kernel: 5.15.0-86-generic (x86_64), Vulkan: 1.3.238, Compiler: GCC 11.4.0, File-System: ext4, Screen Resolution: 1280x800, System Layer: KVM

Kernel Notes: Transparent Huge Pages: madvise
Compiler Notes: --build=x86_64-linux-gnu --disable-vtable-verify --disable-werror --enable-bootstrap --enable-cet --enable-checking=release --enable-clocale=gnu --enable-default-pie --enable-gnu-unique-object --enable-languages=c,ada,c++,go,brig,d,fortran,objc,obj-c++,m2 --enable-libphobos-checking=release --enable-libstdcxx-debug --enable-libstdcxx-time=yes --enable-link-serialization=2 --enable-multiarch --enable-multilib --enable-nls --enable-objc-gc=auto --enable-offload-targets=nvptx-none=/build/gcc-11-XeT9lY/gcc-11-11.4.0/debian/tmp-nvptx/usr,amdgcn-amdhsa=/build/gcc-11-XeT9lY/gcc-11-11.4.0/debian/tmp-gcn/usr --enable-plugin --enable-shared --enable-threads=posix --host=x86_64-linux-gnu --program-prefix=x86_64-linux-gnu- --target=x86_64-linux-gnu --with-abi=m64 --with-arch-32=i686 --with-build-config=bootstrap-lto-lean --with-default-libstdcxx-abi=new --with-gcc-major-version-only --with-multilib-list=m32,m64,mx32 --with-target-system-zlib=auto --with-tune=generic --without-cuda-driver -v
Disk Notes: NONE / relatime,rw / Block Size: 4096
Processor Notes: CPU Microcode: 0x830104d
Security Notes: gather_data_sampling: Not affected + itlb_multihit: Not affected + l1tf: Not affected + mds: Not affected + meltdown: Not affected + mmio_stale_data: Not affected + retbleed: Mitigation of untrained return thunk; SMT disabled + spec_rstack_overflow: Mitigation of SMT disabled + spec_store_bypass: Mitigation of SSB disabled via prctl and seccomp + spectre_v1: Mitigation of usercopy/swapgs barriers and __user pointer sanitization + spectre_v2: Mitigation of Retpolines IBPB: conditional STIBP: disabled RSB filling PBRSB-eIBRS: Not affected + srbds: Not affected + tsx_async_abort: Not affected

PVE-SATA-ZFS-RAID10-8Disks-EPYC7543

Changed Processor to AMD EPYC 7543 32-Core (12 Cores).

Processor Change: CPU Microcode: 0xa00110d
Security Change: gather_data_sampling: Not affected + itlb_multihit: Not affected + l1tf: Not affected + mds: Not affected + meltdown: Not affected + mmio_stale_data: Not affected + retbleed: Not affected + spec_rstack_overflow: Mitigation of safe RET no microcode + spec_store_bypass: Mitigation of SSB disabled via prctl and seccomp + spectre_v1: Mitigation of usercopy/swapgs barriers and __user pointer sanitization + spectre_v2: Mitigation of Retpolines IBPB: conditional IBRS_FW STIBP: disabled RSB filling PBRSB-eIBRS: Not affected + srbds: Not affected + tsx_async_abort: Not affected

PVE-SATA-ZFS-RAID10-6DisksProcessorMotherboardChipsetMemoryDiskGraphicsAudioMonitorNetworkOSKernelVulkanCompilerFile-SystemScreen ResolutionSystem LayerPVE-SATA-ZFS-RAID10-6DisksPVE-SATA-ZFS-RAID10-8DisksPVE-NVME-ZFS-RAID1-2DisksPVE-SATA-ZFS-RAID10-8Disks 64K recPVE-SATA-ZFS-RAID10-8Disks-EPYC7543AMD EPYC 7402 24-Core (12 Cores)QEMU Standard PC (Q35 + ICH9 2009) (3.20230228-4 BIOS)Intel 82G33/G31/P35/P31 + ICH916GB34GB QEMU HDDbochs-drmdrmfbIntel 82801IQEMU MonitorRed Hat Virtio deviceUbuntu 22.045.15.0-84-generic (x86_64)1.3.238GCC 11.4.0ext41280x800KVM5.15.0-86-generic (x86_64)AMD EPYC 7543 32-Core (12 Cores)OpenBenchmarking.orgKernel Details- Transparent Huge Pages: madviseCompiler Details- --build=x86_64-linux-gnu --disable-vtable-verify --disable-werror --enable-bootstrap --enable-cet --enable-checking=release --enable-clocale=gnu --enable-default-pie --enable-gnu-unique-object --enable-languages=c,ada,c++,go,brig,d,fortran,objc,obj-c++,m2 --enable-libphobos-checking=release --enable-libstdcxx-debug --enable-libstdcxx-time=yes --enable-link-serialization=2 --enable-multiarch --enable-multilib --enable-nls --enable-objc-gc=auto --enable-offload-targets=nvptx-none=/build/gcc-11-XeT9lY/gcc-11-11.4.0/debian/tmp-nvptx/usr,amdgcn-amdhsa=/build/gcc-11-XeT9lY/gcc-11-11.4.0/debian/tmp-gcn/usr --enable-plugin --enable-shared --enable-threads=posix --host=x86_64-linux-gnu --program-prefix=x86_64-linux-gnu- --target=x86_64-linux-gnu --with-abi=m64 --with-arch-32=i686 --with-build-config=bootstrap-lto-lean --with-default-libstdcxx-abi=new --with-gcc-major-version-only --with-multilib-list=m32,m64,mx32 --with-target-system-zlib=auto --with-tune=generic --without-cuda-driver -v Disk Details- NONE / relatime,rw / Block Size: 4096Processor Details- PVE-SATA-ZFS-RAID10-6Disks: CPU Microcode: 0x830104d- PVE-SATA-ZFS-RAID10-8Disks: CPU Microcode: 0x830104d- PVE-NVME-ZFS-RAID1-2Disks: CPU Microcode: 0x830104d- PVE-SATA-ZFS-RAID10-8Disks 64K rec: CPU Microcode: 0x830104d- PVE-SATA-ZFS-RAID10-8Disks-EPYC7543: CPU Microcode: 0xa00110dSecurity Details- PVE-SATA-ZFS-RAID10-6Disks: gather_data_sampling: Not affected + itlb_multihit: Not affected + l1tf: Not affected + mds: Not affected + meltdown: Not affected + mmio_stale_data: Not affected + retbleed: Mitigation of untrained return thunk; SMT disabled + spec_store_bypass: Mitigation of SSB disabled via prctl and seccomp + spectre_v1: Mitigation of usercopy/swapgs barriers and __user pointer sanitization + spectre_v2: Mitigation of Retpolines IBPB: conditional STIBP: disabled RSB filling PBRSB-eIBRS: Not affected + srbds: Not affected + tsx_async_abort: Not affected - PVE-SATA-ZFS-RAID10-8Disks: gather_data_sampling: Not affected + itlb_multihit: Not affected + l1tf: Not affected + mds: Not affected + meltdown: Not affected + mmio_stale_data: Not affected + retbleed: Mitigation of untrained return thunk; SMT disabled + spec_store_bypass: Mitigation of SSB disabled via prctl and seccomp + spectre_v1: Mitigation of usercopy/swapgs barriers and __user pointer sanitization + spectre_v2: Mitigation of Retpolines IBPB: conditional STIBP: disabled RSB filling PBRSB-eIBRS: Not affected + srbds: Not affected + tsx_async_abort: Not affected - PVE-NVME-ZFS-RAID1-2Disks: gather_data_sampling: Not affected + itlb_multihit: Not affected + l1tf: Not affected + mds: Not affected + meltdown: Not affected + mmio_stale_data: Not affected + retbleed: Mitigation of untrained return thunk; SMT disabled + spec_store_bypass: Mitigation of SSB disabled via prctl and seccomp + spectre_v1: Mitigation of usercopy/swapgs barriers and __user pointer sanitization + spectre_v2: Mitigation of Retpolines IBPB: conditional STIBP: disabled RSB filling PBRSB-eIBRS: Not affected + srbds: Not affected + tsx_async_abort: Not affected - PVE-SATA-ZFS-RAID10-8Disks 64K rec: gather_data_sampling: Not affected + itlb_multihit: Not affected + l1tf: Not affected + mds: Not affected + meltdown: Not affected + mmio_stale_data: Not affected + retbleed: Mitigation of untrained return thunk; SMT disabled + spec_rstack_overflow: Mitigation of SMT disabled + spec_store_bypass: Mitigation of SSB disabled via prctl and seccomp + spectre_v1: Mitigation of usercopy/swapgs barriers and __user pointer sanitization + spectre_v2: Mitigation of Retpolines IBPB: conditional STIBP: disabled RSB filling PBRSB-eIBRS: Not affected + srbds: Not affected + tsx_async_abort: Not affected - PVE-SATA-ZFS-RAID10-8Disks-EPYC7543: gather_data_sampling: Not affected + itlb_multihit: Not affected + l1tf: Not affected + mds: Not affected + meltdown: Not affected + mmio_stale_data: Not affected + retbleed: Not affected + spec_rstack_overflow: Mitigation of safe RET no microcode + spec_store_bypass: Mitigation of SSB disabled via prctl and seccomp + spectre_v1: Mitigation of usercopy/swapgs barriers and __user pointer sanitization + spectre_v2: Mitigation of Retpolines IBPB: conditional IBRS_FW STIBP: disabled RSB filling PBRSB-eIBRS: Not affected + srbds: Not affected + tsx_async_abort: Not affected

PVE-SATA-ZFS-RAID10-6DisksPVE-SATA-ZFS-RAID10-8DisksPVE-NVME-ZFS-RAID1-2DisksPVE-SATA-ZFS-RAID10-8Disks 64K recPVE-SATA-ZFS-RAID10-8Disks-EPYC7543Logarithmic Result OverviewPhoronix Test SuiteFlexible IO TesterFlexible IO TesterFlexible IO TesterFlexible IO TesterFlexible IO TesterFlexible IO TesterFlexible IO TesterFlexible IO TesterRand Write - Linux AIO - No - No - 4KB - 1Rand Write - Linux AIO - No - No - 4KB - 1Rand Write - Linux AIO - No - No - 64KB - 1Rand Read - Linux AIO - No - No - 64KB - 1Rand Read - Linux AIO - No - No - 64KB - 1Rand Write - Linux AIO - No - No - 64KB - 1Rand Read - Linux AIO - No - No - 4KB - 1Rand Read - Linux AIO - No - No - 4KB - 1

PVE-SATA-ZFS-RAID10-6Disksfio: Rand Read - Linux AIO - No - No - 4KB - 1 - Default Test Directoryfio: Rand Read - Linux AIO - No - No - 64KB - 1 - Default Test Directoryfio: Rand Write - Linux AIO - No - No - 4KB - 1 - Default Test Directoryfio: Rand Write - Linux AIO - No - No - 64KB - 1 - Default Test Directoryfio: Rand Read - Linux AIO - No - No - 4KB - 1 - Default Test Directoryfio: Rand Read - Linux AIO - No - No - 64KB - 1 - Default Test Directoryfio: Rand Write - Linux AIO - No - No - 4KB - 1 - Default Test Directoryfio: Rand Write - Linux AIO - No - No - 64KB - 1 - Default Test DirectoryPVE-SATA-ZFS-RAID10-6DisksPVE-SATA-ZFS-RAID10-8DisksPVE-NVME-ZFS-RAID1-2DisksPVE-SATA-ZFS-RAID10-8Disks 64K recPVE-SATA-ZFS-RAID10-8Disks-EPYC75431385014760310388617226554004.192975.23415546.151076.611421807522611497422278568722.144701.61459895.501392.38131797999189295030889527188.416244.88371799.241930.5715647712537247782064625906.397835.7419110.37132112.152042528673050391397817008.535420.6120155.9289388.65OpenBenchmarking.org

Flexible IO Tester

OpenBenchmarking.orgIOPS, More Is BetterFlexible IO Tester 3.35Type: Random Read - Engine: Linux AIO - Buffered: No - Direct: No - Block Size: 4KB - Job Count: 1 - Disk Target: Default Test DirectoryPVE-NVME-ZFS-RAID1-2DisksPVE-SATA-ZFS-RAID10-6DisksPVE-SATA-ZFS-RAID10-8DisksPVE-SATA-ZFS-RAID10-8Disks 64K recPVE-SATA-ZFS-RAID10-8Disks-EPYC754340K80K120K160K200KSE +/- 361.19, N = 3SE +/- 226.74, N = 3SE +/- 740.76, N = 3SE +/- 779.58, N = 5SE +/- 3063.09, N = 5131797138501142180156477204252MIN: 131251.14 / MAX: 132479.72MIN: 138226.02 / MAX: 138950.81MIN: 141056.82 / MAX: 143578.51MIN: 153965.84 / MAX: 158510.43MIN: 192042.12 / MAX: 207954.571. (CC) gcc options: -rdynamic -lm -laio -lpthread -ldl -std=gnu99 -ffast-math -include -O3 -fcommon -march=native
OpenBenchmarking.orgIOPS, More Is BetterFlexible IO Tester 3.35Type: Random Read - Engine: Linux AIO - Buffered: No - Direct: No - Block Size: 4KB - Job Count: 1 - Disk Target: Default Test DirectoryPVE-NVME-ZFS-RAID1-2DisksPVE-SATA-ZFS-RAID10-6DisksPVE-SATA-ZFS-RAID10-8DisksPVE-SATA-ZFS-RAID10-8Disks 64K recPVE-SATA-ZFS-RAID10-8Disks-EPYC754340K80K120K160K200KMin: 131251.14 / Avg: 131797.06 / Max: 132479.72Min: 138226.02 / Avg: 138501.05 / Max: 138950.81Min: 141056.82 / Avg: 142180.47 / Max: 143578.51Min: 153965.84 / Avg: 156476.56 / Max: 158510.43Min: 192042.12 / Avg: 204252.03 / Max: 207954.571. (CC) gcc options: -rdynamic -lm -laio -lpthread -ldl -std=gnu99 -ffast-math -include -O3 -fcommon -march=native

OpenBenchmarking.orgIOPS, More Is BetterFlexible IO Tester 3.35Type: Random Read - Engine: Linux AIO - Buffered: No - Direct: No - Block Size: 64KB - Job Count: 1 - Disk Target: Default Test DirectoryPVE-NVME-ZFS-RAID1-2DisksPVE-SATA-ZFS-RAID10-6DisksPVE-SATA-ZFS-RAID10-8DisksPVE-SATA-ZFS-RAID10-8Disks 64K recPVE-SATA-ZFS-RAID10-8Disks-EPYC754330K60K90K120K150KSE +/- 1454.83, N = 15SE +/- 188.54, N = 3SE +/- 819.79, N = 3SE +/- 3342.65, N = 5SE +/- 4102.59, N = 599918476037522612537286730MIN: 88116.28 / MAX: 109282.62MIN: 47226.44 / MAX: 47796.94MIN: 73996.52 / MAX: 76779.97MIN: 120840.15 / MAX: 138556.61MIN: 76356.28 / MAX: 96716.821. (CC) gcc options: -rdynamic -lm -laio -lpthread -ldl -std=gnu99 -ffast-math -include -O3 -fcommon -march=native
OpenBenchmarking.orgIOPS, More Is BetterFlexible IO Tester 3.35Type: Random Read - Engine: Linux AIO - Buffered: No - Direct: No - Block Size: 64KB - Job Count: 1 - Disk Target: Default Test DirectoryPVE-NVME-ZFS-RAID1-2DisksPVE-SATA-ZFS-RAID10-6DisksPVE-SATA-ZFS-RAID10-8DisksPVE-SATA-ZFS-RAID10-8Disks 64K recPVE-SATA-ZFS-RAID10-8Disks-EPYC754320K40K60K80K100KMin: 88116.28 / Avg: 99918.01 / Max: 109282.62Min: 47226.44 / Avg: 47603.47 / Max: 47796.94Min: 73996.52 / Avg: 75225.67 / Max: 76779.97Min: 120840.15 / Avg: 125371.76 / Max: 138556.61Min: 76356.28 / Avg: 86729.71 / Max: 96716.821. (CC) gcc options: -rdynamic -lm -laio -lpthread -ldl -std=gnu99 -ffast-math -include -O3 -fcommon -march=native

OpenBenchmarking.orgIOPS, More Is BetterFlexible IO Tester 3.35Type: Random Write - Engine: Linux AIO - Buffered: No - Direct: No - Block Size: 4KB - Job Count: 1 - Disk Target: Default Test DirectoryPVE-NVME-ZFS-RAID1-2DisksPVE-SATA-ZFS-RAID10-6DisksPVE-SATA-ZFS-RAID10-8DisksPVE-SATA-ZFS-RAID10-8Disks 64K recPVE-SATA-ZFS-RAID10-8Disks-EPYC754320K40K60K80K100KSE +/- 509.01, N = 3SE +/- 549.53, N = 3SE +/- 668.16, N = 3SE +/- 20.06, N = 5SE +/- 110.94, N = 59295010388611497447785039MIN: 92104.66 / MAX: 93863.87MIN: 103087.2 / MAX: 104939.44MIN: 113657.43 / MAX: 115831.021. (CC) gcc options: -rdynamic -lm -laio -lpthread -ldl -std=gnu99 -ffast-math -include -O3 -fcommon -march=native
OpenBenchmarking.orgIOPS, More Is BetterFlexible IO Tester 3.35Type: Random Write - Engine: Linux AIO - Buffered: No - Direct: No - Block Size: 4KB - Job Count: 1 - Disk Target: Default Test DirectoryPVE-NVME-ZFS-RAID1-2DisksPVE-SATA-ZFS-RAID10-6DisksPVE-SATA-ZFS-RAID10-8DisksPVE-SATA-ZFS-RAID10-8Disks 64K recPVE-SATA-ZFS-RAID10-8Disks-EPYC754320K40K60K80K100KMin: 92104.66 / Avg: 92949.75 / Max: 93863.87Min: 103087.2 / Avg: 103886.48 / Max: 104939.44Min: 113657.43 / Avg: 114973.84 / Max: 115831.02Min: 4732 / Avg: 4777.56 / Max: 4830.42Min: 4869.74 / Avg: 5038.94 / Max: 5464.461. (CC) gcc options: -rdynamic -lm -laio -lpthread -ldl -std=gnu99 -ffast-math -include -O3 -fcommon -march=native

OpenBenchmarking.orgIOPS, More Is BetterFlexible IO Tester 3.35Type: Random Write - Engine: Linux AIO - Buffered: No - Direct: No - Block Size: 64KB - Job Count: 1 - Disk Target: Default Test DirectoryPVE-NVME-ZFS-RAID1-2DisksPVE-SATA-ZFS-RAID10-6DisksPVE-SATA-ZFS-RAID10-8DisksPVE-SATA-ZFS-RAID10-8Disks 64K recPVE-SATA-ZFS-RAID10-8Disks-EPYC75437K14K21K28K35KSE +/- 237.15, N = 3SE +/- 152.69, N = 15SE +/- 44.49, N = 3SE +/- 2.95, N = 5SE +/- 26.45, N = 530889172262227820641397MIN: 30476.82 / MAX: 31298.32MIN: 16559.33 / MAX: 18616.54MIN: 22202 / MAX: 22356.081. (CC) gcc options: -rdynamic -lm -laio -lpthread -ldl -std=gnu99 -ffast-math -include -O3 -fcommon -march=native
OpenBenchmarking.orgIOPS, More Is BetterFlexible IO Tester 3.35Type: Random Write - Engine: Linux AIO - Buffered: No - Direct: No - Block Size: 64KB - Job Count: 1 - Disk Target: Default Test DirectoryPVE-NVME-ZFS-RAID1-2DisksPVE-SATA-ZFS-RAID10-6DisksPVE-SATA-ZFS-RAID10-8DisksPVE-SATA-ZFS-RAID10-8Disks 64K recPVE-SATA-ZFS-RAID10-8Disks-EPYC75435K10K15K20K25KMin: 30476.82 / Avg: 30888.88 / Max: 31298.32Min: 16559.33 / Avg: 17225.6 / Max: 18616.54Min: 22202 / Avg: 22278.07 / Max: 22356.08Min: 2059.15 / Avg: 2064.21 / Max: 2073.53Min: 1355.04 / Avg: 1396.57 / Max: 1499.811. (CC) gcc options: -rdynamic -lm -laio -lpthread -ldl -std=gnu99 -ffast-math -include -O3 -fcommon -march=native

OpenBenchmarking.orgMiB/s, More Is BetterFlexible IO Tester 3.35Type: Random Read - Engine: Linux AIO - Buffered: No - Direct: No - Block Size: 4KB - Job Count: 1 - Disk Target: Default Test DirectoryPVE-NVME-ZFS-RAID1-2DisksPVE-SATA-ZFS-RAID10-6DisksPVE-SATA-ZFS-RAID10-8DisksPVE-SATA-ZFS-RAID10-8Disks 64K recPVE-SATA-ZFS-RAID10-8Disks-EPYC7543200K400K600K800K1000KSE +/- 1444.82, N = 3SE +/- 907.07, N = 3SE +/- 2963.15, N = 3SE +/- 3118.29, N = 5SE +/- 12252.38, N = 5527188.41554004.19568722.14625906.39817008.53MIN: 525004.58 / MAX: 529919.12MIN: 552904.03 / MAX: 555803.49MIN: 564227.4 / MAX: 574314.47MIN: 615863.53 / MAX: 634041.79MIN: 768168.87 / MAX: 831818.771. (CC) gcc options: -rdynamic -lm -laio -lpthread -ldl -std=gnu99 -ffast-math -include -O3 -fcommon -march=native
OpenBenchmarking.orgMiB/s, More Is BetterFlexible IO Tester 3.35Type: Random Read - Engine: Linux AIO - Buffered: No - Direct: No - Block Size: 4KB - Job Count: 1 - Disk Target: Default Test DirectoryPVE-NVME-ZFS-RAID1-2DisksPVE-SATA-ZFS-RAID10-6DisksPVE-SATA-ZFS-RAID10-8DisksPVE-SATA-ZFS-RAID10-8Disks 64K recPVE-SATA-ZFS-RAID10-8Disks-EPYC7543140K280K420K560K700KMin: 525004.58 / Avg: 527188.41 / Max: 529919.12Min: 552904.03 / Avg: 554004.19 / Max: 555803.49Min: 564227.4 / Avg: 568722.14 / Max: 574314.47Min: 615863.53 / Avg: 625906.39 / Max: 634041.79Min: 768168.87 / Avg: 817008.53 / Max: 831818.771. (CC) gcc options: -rdynamic -lm -laio -lpthread -ldl -std=gnu99 -ffast-math -include -O3 -fcommon -march=native

OpenBenchmarking.orgMiB/s, More Is BetterFlexible IO Tester 3.35Type: Random Read - Engine: Linux AIO - Buffered: No - Direct: No - Block Size: 64KB - Job Count: 1 - Disk Target: Default Test DirectoryPVE-NVME-ZFS-RAID1-2DisksPVE-SATA-ZFS-RAID10-6DisksPVE-SATA-ZFS-RAID10-8DisksPVE-SATA-ZFS-RAID10-8Disks 64K recPVE-SATA-ZFS-RAID10-8Disks-EPYC75432K4K6K8K10KSE +/- 90.93, N = 15SE +/- 11.78, N = 3SE +/- 51.24, N = 3SE +/- 208.92, N = 5SE +/- 256.41, N = 56244.882975.234701.617835.745420.61MIN: 5507.27 / MAX: 6830.17MIN: 2951.66 / MAX: 2987.32MIN: 4624.79 / MAX: 4798.76MIN: 7552.51 / MAX: 8659.79MIN: 4772.27 / MAX: 6044.81. (CC) gcc options: -rdynamic -lm -laio -lpthread -ldl -std=gnu99 -ffast-math -include -O3 -fcommon -march=native
OpenBenchmarking.orgMiB/s, More Is BetterFlexible IO Tester 3.35Type: Random Read - Engine: Linux AIO - Buffered: No - Direct: No - Block Size: 64KB - Job Count: 1 - Disk Target: Default Test DirectoryPVE-NVME-ZFS-RAID1-2DisksPVE-SATA-ZFS-RAID10-6DisksPVE-SATA-ZFS-RAID10-8DisksPVE-SATA-ZFS-RAID10-8Disks 64K recPVE-SATA-ZFS-RAID10-8Disks-EPYC754314002800420056007000Min: 5507.27 / Avg: 6244.88 / Max: 6830.17Min: 2951.66 / Avg: 2975.23 / Max: 2987.32Min: 4624.79 / Avg: 4701.61 / Max: 4798.76Min: 7552.51 / Avg: 7835.74 / Max: 8659.79Min: 4772.27 / Avg: 5420.61 / Max: 6044.81. (CC) gcc options: -rdynamic -lm -laio -lpthread -ldl -std=gnu99 -ffast-math -include -O3 -fcommon -march=native

OpenBenchmarking.orgMiB/s, More Is BetterFlexible IO Tester 3.35Type: Random Write - Engine: Linux AIO - Buffered: No - Direct: No - Block Size: 4KB - Job Count: 1 - Disk Target: Default Test DirectoryPVE-NVME-ZFS-RAID1-2DisksPVE-SATA-ZFS-RAID10-6DisksPVE-SATA-ZFS-RAID10-8DisksPVE-SATA-ZFS-RAID10-8Disks 64K recPVE-SATA-ZFS-RAID10-8Disks-EPYC7543100K200K300K400K500KSE +/- 2036.04, N = 3SE +/- 2198.09, N = 3SE +/- 2672.63, N = 3SE +/- 80.22, N = 5SE +/- 443.77, N = 5371799.24415546.15459895.5019110.3720155.92MIN: 368418.88 / MAX: 375455.69MIN: 412349 / MAX: 419757.86MIN: 454629.83 / MAX: 463324.231. (CC) gcc options: -rdynamic -lm -laio -lpthread -ldl -std=gnu99 -ffast-math -include -O3 -fcommon -march=native
OpenBenchmarking.orgMiB/s, More Is BetterFlexible IO Tester 3.35Type: Random Write - Engine: Linux AIO - Buffered: No - Direct: No - Block Size: 4KB - Job Count: 1 - Disk Target: Default Test DirectoryPVE-NVME-ZFS-RAID1-2DisksPVE-SATA-ZFS-RAID10-6DisksPVE-SATA-ZFS-RAID10-8DisksPVE-SATA-ZFS-RAID10-8Disks 64K recPVE-SATA-ZFS-RAID10-8Disks-EPYC754380K160K240K320K400KMin: 368418.88 / Avg: 371799.24 / Max: 375455.69Min: 412349 / Avg: 415546.15 / Max: 419757.86Min: 454629.83 / Avg: 459895.5 / Max: 463324.23Min: 18928.14 / Avg: 19110.37 / Max: 19321.74Min: 19479.17 / Avg: 20155.92 / Max: 21858.041. (CC) gcc options: -rdynamic -lm -laio -lpthread -ldl -std=gnu99 -ffast-math -include -O3 -fcommon -march=native

OpenBenchmarking.orgMiB/s, More Is BetterFlexible IO Tester 3.35Type: Random Write - Engine: Linux AIO - Buffered: No - Direct: No - Block Size: 64KB - Job Count: 1 - Disk Target: Default Test DirectoryPVE-NVME-ZFS-RAID1-2DisksPVE-SATA-ZFS-RAID10-6DisksPVE-SATA-ZFS-RAID10-8DisksPVE-SATA-ZFS-RAID10-8Disks 64K recPVE-SATA-ZFS-RAID10-8Disks-EPYC754330K60K90K120K150KSE +/- 14.82, N = 3SE +/- 9.54, N = 15SE +/- 2.77, N = 3SE +/- 188.68, N = 5SE +/- 1692.33, N = 51930.571076.611392.38132112.1589388.65MIN: 131788.07 / MAX: 132708.91MIN: 86732.47 / MAX: 95994.761. (CC) gcc options: -rdynamic -lm -laio -lpthread -ldl -std=gnu99 -ffast-math -include -O3 -fcommon -march=native
OpenBenchmarking.orgMiB/s, More Is BetterFlexible IO Tester 3.35Type: Random Write - Engine: Linux AIO - Buffered: No - Direct: No - Block Size: 64KB - Job Count: 1 - Disk Target: Default Test DirectoryPVE-NVME-ZFS-RAID1-2DisksPVE-SATA-ZFS-RAID10-6DisksPVE-SATA-ZFS-RAID10-8DisksPVE-SATA-ZFS-RAID10-8Disks 64K recPVE-SATA-ZFS-RAID10-8Disks-EPYC754320K40K60K80K100KMin: 1904.82 / Avg: 1930.57 / Max: 1956.16Min: 1034.96 / Avg: 1076.61 / Max: 1163.54Min: 1387.63 / Avg: 1392.38 / Max: 1397.24Min: 131788.07 / Avg: 132112.15 / Max: 132708.91Min: 86732.47 / Avg: 89388.65 / Max: 95994.761. (CC) gcc options: -rdynamic -lm -laio -lpthread -ldl -std=gnu99 -ffast-math -include -O3 -fcommon -march=native

PVE-SATA-ZFS-RAID10-6Disks

Kernel Notes: Transparent Huge Pages: madvise
Compiler Notes: --build=x86_64-linux-gnu --disable-vtable-verify --disable-werror --enable-bootstrap --enable-cet --enable-checking=release --enable-clocale=gnu --enable-default-pie --enable-gnu-unique-object --enable-languages=c,ada,c++,go,brig,d,fortran,objc,obj-c++,m2 --enable-libphobos-checking=release --enable-libstdcxx-debug --enable-libstdcxx-time=yes --enable-link-serialization=2 --enable-multiarch --enable-multilib --enable-nls --enable-objc-gc=auto --enable-offload-targets=nvptx-none=/build/gcc-11-XeT9lY/gcc-11-11.4.0/debian/tmp-nvptx/usr,amdgcn-amdhsa=/build/gcc-11-XeT9lY/gcc-11-11.4.0/debian/tmp-gcn/usr --enable-plugin --enable-shared --enable-threads=posix --host=x86_64-linux-gnu --program-prefix=x86_64-linux-gnu- --target=x86_64-linux-gnu --with-abi=m64 --with-arch-32=i686 --with-build-config=bootstrap-lto-lean --with-default-libstdcxx-abi=new --with-gcc-major-version-only --with-multilib-list=m32,m64,mx32 --with-target-system-zlib=auto --with-tune=generic --without-cuda-driver -v
Disk Notes: NONE / relatime,rw / Block Size: 4096
Processor Notes: CPU Microcode: 0x830104d
Security Notes: gather_data_sampling: Not affected + itlb_multihit: Not affected + l1tf: Not affected + mds: Not affected + meltdown: Not affected + mmio_stale_data: Not affected + retbleed: Mitigation of untrained return thunk; SMT disabled + spec_store_bypass: Mitigation of SSB disabled via prctl and seccomp + spectre_v1: Mitigation of usercopy/swapgs barriers and __user pointer sanitization + spectre_v2: Mitigation of Retpolines IBPB: conditional STIBP: disabled RSB filling PBRSB-eIBRS: Not affected + srbds: Not affected + tsx_async_abort: Not affected

Testing initiated at 2 October 2023 10:05 by user tcstudios.

PVE-SATA-ZFS-RAID10-8Disks

Kernel Notes: Transparent Huge Pages: madvise
Compiler Notes: --build=x86_64-linux-gnu --disable-vtable-verify --disable-werror --enable-bootstrap --enable-cet --enable-checking=release --enable-clocale=gnu --enable-default-pie --enable-gnu-unique-object --enable-languages=c,ada,c++,go,brig,d,fortran,objc,obj-c++,m2 --enable-libphobos-checking=release --enable-libstdcxx-debug --enable-libstdcxx-time=yes --enable-link-serialization=2 --enable-multiarch --enable-multilib --enable-nls --enable-objc-gc=auto --enable-offload-targets=nvptx-none=/build/gcc-11-XeT9lY/gcc-11-11.4.0/debian/tmp-nvptx/usr,amdgcn-amdhsa=/build/gcc-11-XeT9lY/gcc-11-11.4.0/debian/tmp-gcn/usr --enable-plugin --enable-shared --enable-threads=posix --host=x86_64-linux-gnu --program-prefix=x86_64-linux-gnu- --target=x86_64-linux-gnu --with-abi=m64 --with-arch-32=i686 --with-build-config=bootstrap-lto-lean --with-default-libstdcxx-abi=new --with-gcc-major-version-only --with-multilib-list=m32,m64,mx32 --with-target-system-zlib=auto --with-tune=generic --without-cuda-driver -v
Disk Notes: NONE / relatime,rw / Block Size: 4096
Processor Notes: CPU Microcode: 0x830104d
Security Notes: gather_data_sampling: Not affected + itlb_multihit: Not affected + l1tf: Not affected + mds: Not affected + meltdown: Not affected + mmio_stale_data: Not affected + retbleed: Mitigation of untrained return thunk; SMT disabled + spec_store_bypass: Mitigation of SSB disabled via prctl and seccomp + spectre_v1: Mitigation of usercopy/swapgs barriers and __user pointer sanitization + spectre_v2: Mitigation of Retpolines IBPB: conditional STIBP: disabled RSB filling PBRSB-eIBRS: Not affected + srbds: Not affected + tsx_async_abort: Not affected

Testing initiated at 2 October 2023 11:32 by user tcstudios.

PVE-NVME-ZFS-RAID1-2Disks

Processor: AMD EPYC 7402 24-Core (12 Cores), Motherboard: QEMU Standard PC (Q35 + ICH9 2009) (3.20230228-4 BIOS), Chipset: Intel 82G33/G31/P35/P31 + ICH9, Memory: 16GB, Disk: 34GB QEMU HDD, Graphics: bochs-drmdrmfb, Audio: Intel 82801I, Monitor: QEMU Monitor, Network: Red Hat Virtio device

OS: Ubuntu 22.04, Kernel: 5.15.0-84-generic (x86_64), Vulkan: 1.3.238, Compiler: GCC 11.4.0, File-System: ext4, Screen Resolution: 1280x800, System Layer: KVM

Kernel Notes: Transparent Huge Pages: madvise
Compiler Notes: --build=x86_64-linux-gnu --disable-vtable-verify --disable-werror --enable-bootstrap --enable-cet --enable-checking=release --enable-clocale=gnu --enable-default-pie --enable-gnu-unique-object --enable-languages=c,ada,c++,go,brig,d,fortran,objc,obj-c++,m2 --enable-libphobos-checking=release --enable-libstdcxx-debug --enable-libstdcxx-time=yes --enable-link-serialization=2 --enable-multiarch --enable-multilib --enable-nls --enable-objc-gc=auto --enable-offload-targets=nvptx-none=/build/gcc-11-XeT9lY/gcc-11-11.4.0/debian/tmp-nvptx/usr,amdgcn-amdhsa=/build/gcc-11-XeT9lY/gcc-11-11.4.0/debian/tmp-gcn/usr --enable-plugin --enable-shared --enable-threads=posix --host=x86_64-linux-gnu --program-prefix=x86_64-linux-gnu- --target=x86_64-linux-gnu --with-abi=m64 --with-arch-32=i686 --with-build-config=bootstrap-lto-lean --with-default-libstdcxx-abi=new --with-gcc-major-version-only --with-multilib-list=m32,m64,mx32 --with-target-system-zlib=auto --with-tune=generic --without-cuda-driver -v
Disk Notes: NONE / relatime,rw / Block Size: 4096
Processor Notes: CPU Microcode: 0x830104d
Security Notes: gather_data_sampling: Not affected + itlb_multihit: Not affected + l1tf: Not affected + mds: Not affected + meltdown: Not affected + mmio_stale_data: Not affected + retbleed: Mitigation of untrained return thunk; SMT disabled + spec_store_bypass: Mitigation of SSB disabled via prctl and seccomp + spectre_v1: Mitigation of usercopy/swapgs barriers and __user pointer sanitization + spectre_v2: Mitigation of Retpolines IBPB: conditional STIBP: disabled RSB filling PBRSB-eIBRS: Not affected + srbds: Not affected + tsx_async_abort: Not affected

Testing initiated at 6 October 2023 04:38 by user tcstudios.

PVE-SATA-ZFS-RAID10-8Disks 64K rec

Processor: AMD EPYC 7402 24-Core (12 Cores), Motherboard: QEMU Standard PC (Q35 + ICH9 2009) (3.20230228-4 BIOS), Chipset: Intel 82G33/G31/P35/P31 + ICH9, Memory: 16GB, Disk: 34GB QEMU HDD, Graphics: bochs-drmdrmfb, Audio: Intel 82801I, Monitor: QEMU Monitor, Network: Red Hat Virtio device

OS: Ubuntu 22.04, Kernel: 5.15.0-86-generic (x86_64), Vulkan: 1.3.238, Compiler: GCC 11.4.0, File-System: ext4, Screen Resolution: 1280x800, System Layer: KVM

Kernel Notes: Transparent Huge Pages: madvise
Compiler Notes: --build=x86_64-linux-gnu --disable-vtable-verify --disable-werror --enable-bootstrap --enable-cet --enable-checking=release --enable-clocale=gnu --enable-default-pie --enable-gnu-unique-object --enable-languages=c,ada,c++,go,brig,d,fortran,objc,obj-c++,m2 --enable-libphobos-checking=release --enable-libstdcxx-debug --enable-libstdcxx-time=yes --enable-link-serialization=2 --enable-multiarch --enable-multilib --enable-nls --enable-objc-gc=auto --enable-offload-targets=nvptx-none=/build/gcc-11-XeT9lY/gcc-11-11.4.0/debian/tmp-nvptx/usr,amdgcn-amdhsa=/build/gcc-11-XeT9lY/gcc-11-11.4.0/debian/tmp-gcn/usr --enable-plugin --enable-shared --enable-threads=posix --host=x86_64-linux-gnu --program-prefix=x86_64-linux-gnu- --target=x86_64-linux-gnu --with-abi=m64 --with-arch-32=i686 --with-build-config=bootstrap-lto-lean --with-default-libstdcxx-abi=new --with-gcc-major-version-only --with-multilib-list=m32,m64,mx32 --with-target-system-zlib=auto --with-tune=generic --without-cuda-driver -v
Disk Notes: NONE / relatime,rw / Block Size: 4096
Processor Notes: CPU Microcode: 0x830104d
Security Notes: gather_data_sampling: Not affected + itlb_multihit: Not affected + l1tf: Not affected + mds: Not affected + meltdown: Not affected + mmio_stale_data: Not affected + retbleed: Mitigation of untrained return thunk; SMT disabled + spec_rstack_overflow: Mitigation of SMT disabled + spec_store_bypass: Mitigation of SSB disabled via prctl and seccomp + spectre_v1: Mitigation of usercopy/swapgs barriers and __user pointer sanitization + spectre_v2: Mitigation of Retpolines IBPB: conditional STIBP: disabled RSB filling PBRSB-eIBRS: Not affected + srbds: Not affected + tsx_async_abort: Not affected

Testing initiated at 8 October 2023 04:56 by user tcstudios.

PVE-SATA-ZFS-RAID10-8Disks-EPYC7543

Processor: AMD EPYC 7543 32-Core (12 Cores), Motherboard: QEMU Standard PC (Q35 + ICH9 2009) (3.20230228-4 BIOS), Chipset: Intel 82G33/G31/P35/P31 + ICH9, Memory: 16GB, Disk: 34GB QEMU HDD, Graphics: bochs-drmdrmfb, Audio: Intel 82801I, Monitor: QEMU Monitor, Network: Red Hat Virtio device

OS: Ubuntu 22.04, Kernel: 5.15.0-86-generic (x86_64), Vulkan: 1.3.238, Compiler: GCC 11.4.0, File-System: ext4, Screen Resolution: 1280x800, System Layer: KVM

Kernel Notes: Transparent Huge Pages: madvise
Compiler Notes: --build=x86_64-linux-gnu --disable-vtable-verify --disable-werror --enable-bootstrap --enable-cet --enable-checking=release --enable-clocale=gnu --enable-default-pie --enable-gnu-unique-object --enable-languages=c,ada,c++,go,brig,d,fortran,objc,obj-c++,m2 --enable-libphobos-checking=release --enable-libstdcxx-debug --enable-libstdcxx-time=yes --enable-link-serialization=2 --enable-multiarch --enable-multilib --enable-nls --enable-objc-gc=auto --enable-offload-targets=nvptx-none=/build/gcc-11-XeT9lY/gcc-11-11.4.0/debian/tmp-nvptx/usr,amdgcn-amdhsa=/build/gcc-11-XeT9lY/gcc-11-11.4.0/debian/tmp-gcn/usr --enable-plugin --enable-shared --enable-threads=posix --host=x86_64-linux-gnu --program-prefix=x86_64-linux-gnu- --target=x86_64-linux-gnu --with-abi=m64 --with-arch-32=i686 --with-build-config=bootstrap-lto-lean --with-default-libstdcxx-abi=new --with-gcc-major-version-only --with-multilib-list=m32,m64,mx32 --with-target-system-zlib=auto --with-tune=generic --without-cuda-driver -v
Disk Notes: NONE / relatime,rw / Block Size: 4096
Processor Notes: CPU Microcode: 0xa00110d
Security Notes: gather_data_sampling: Not affected + itlb_multihit: Not affected + l1tf: Not affected + mds: Not affected + meltdown: Not affected + mmio_stale_data: Not affected + retbleed: Not affected + spec_rstack_overflow: Mitigation of safe RET no microcode + spec_store_bypass: Mitigation of SSB disabled via prctl and seccomp + spectre_v1: Mitigation of usercopy/swapgs barriers and __user pointer sanitization + spectre_v2: Mitigation of Retpolines IBPB: conditional IBRS_FW STIBP: disabled RSB filling PBRSB-eIBRS: Not affected + srbds: Not affected + tsx_async_abort: Not affected

Testing initiated at 13 October 2023 06:48 by user tcstudios.