PVE-SATA-ZFS-RAID10-6Disks

PVE-SATA-ZFS-RAID10-8Disks (64K rec)

Compare your own system(s) to this result file with the Phoronix Test Suite by running the command: phoronix-test-suite benchmark 2310084-TCST-231006491
Jump To Table - Results

View

Do Not Show Noisy Results
Do Not Show Results With Incomplete Data
Do Not Show Results With Little Change/Spread
List Notable Results

Statistics

Show Overall Harmonic Mean(s)
Show Overall Geometric Mean
Show Wins / Losses Counts (Pie Chart)
Normalize Results
Remove Outliers Before Calculating Averages

Graph Settings

Force Line Graphs Where Applicable
Convert To Scalar Where Applicable
Prefer Vertical Bar Graphs

Multi-Way Comparison

Condense Multi-Option Tests Into Single Result Graphs

Table

Show Detailed System Result Table

Run Management

Highlight
Result
Hide
Result
Result
Identifier
Performance Per
Dollar
Date
Run
  Test
  Duration
PVE-SATA-ZFS-RAID10-6Disks
October 02 2023
  1 Hour, 9 Minutes
PVE-SATA-ZFS-RAID10-8Disks
October 02 2023
  35 Minutes
PVE-NVME-ZFS-RAID1-2Disks
October 06 2023
  1 Hour, 10 Minutes
PVE-SATA-ZFS-RAID10-8Disks 64K rec
October 08 2023
  58 Minutes
Invert Hiding All Results Option
  58 Minutes

Only show results where is faster than
Only show results matching title/arguments (delimit multiple options with a comma):
Do not show results matching title/arguments (delimit multiple options with a comma):

PVE-SATA-ZFS-RAID10-6Disks

Kernel Notes: Transparent Huge Pages: madvise
Compiler Notes: --build=x86_64-linux-gnu --disable-vtable-verify --disable-werror --enable-bootstrap --enable-cet --enable-checking=release --enable-clocale=gnu --enable-default-pie --enable-gnu-unique-object --enable-languages=c,ada,c++,go,brig,d,fortran,objc,obj-c++,m2 --enable-libphobos-checking=release --enable-libstdcxx-debug --enable-libstdcxx-time=yes --enable-link-serialization=2 --enable-multiarch --enable-multilib --enable-nls --enable-objc-gc=auto --enable-offload-targets=nvptx-none=/build/gcc-11-XeT9lY/gcc-11-11.4.0/debian/tmp-nvptx/usr,amdgcn-amdhsa=/build/gcc-11-XeT9lY/gcc-11-11.4.0/debian/tmp-gcn/usr --enable-plugin --enable-shared --enable-threads=posix --host=x86_64-linux-gnu --program-prefix=x86_64-linux-gnu- --target=x86_64-linux-gnu --with-abi=m64 --with-arch-32=i686 --with-build-config=bootstrap-lto-lean --with-default-libstdcxx-abi=new --with-gcc-major-version-only --with-multilib-list=m32,m64,mx32 --with-target-system-zlib=auto --with-tune=generic --without-cuda-driver -v
Disk Notes: NONE / relatime,rw / Block Size: 4096
Processor Notes: CPU Microcode: 0x830104d
Security Notes: gather_data_sampling: Not affected + itlb_multihit: Not affected + l1tf: Not affected + mds: Not affected + meltdown: Not affected + mmio_stale_data: Not affected + retbleed: Mitigation of untrained return thunk; SMT disabled + spec_store_bypass: Mitigation of SSB disabled via prctl and seccomp + spectre_v1: Mitigation of usercopy/swapgs barriers and __user pointer sanitization + spectre_v2: Mitigation of Retpolines IBPB: conditional STIBP: disabled RSB filling PBRSB-eIBRS: Not affected + srbds: Not affected + tsx_async_abort: Not affected

PVE-SATA-ZFS-RAID10-8Disks

PVE-NVME-ZFS-RAID1-2Disks

Processor: AMD EPYC 7402 24-Core (12 Cores), Motherboard: QEMU Standard PC (Q35 + ICH9 2009) (3.20230228-4 BIOS), Chipset: Intel 82G33/G31/P35/P31 + ICH9, Memory: 16GB, Disk: 34GB QEMU HDD, Graphics: bochs-drmdrmfb, Audio: Intel 82801I, Monitor: QEMU Monitor, Network: Red Hat Virtio device

OS: Ubuntu 22.04, Kernel: 5.15.0-84-generic (x86_64), Vulkan: 1.3.238, Compiler: GCC 11.4.0, File-System: ext4, Screen Resolution: 1280x800, System Layer: KVM

PVE-SATA-ZFS-RAID10-8Disks 64K rec

OS: Ubuntu 22.04, Kernel: 5.15.0-86-generic (x86_64), Vulkan: 1.3.238, Compiler: GCC 11.4.0, File-System: ext4, Screen Resolution: 1280x800, System Layer: KVM

Kernel Notes: Transparent Huge Pages: madvise
Compiler Notes: --build=x86_64-linux-gnu --disable-vtable-verify --disable-werror --enable-bootstrap --enable-cet --enable-checking=release --enable-clocale=gnu --enable-default-pie --enable-gnu-unique-object --enable-languages=c,ada,c++,go,brig,d,fortran,objc,obj-c++,m2 --enable-libphobos-checking=release --enable-libstdcxx-debug --enable-libstdcxx-time=yes --enable-link-serialization=2 --enable-multiarch --enable-multilib --enable-nls --enable-objc-gc=auto --enable-offload-targets=nvptx-none=/build/gcc-11-XeT9lY/gcc-11-11.4.0/debian/tmp-nvptx/usr,amdgcn-amdhsa=/build/gcc-11-XeT9lY/gcc-11-11.4.0/debian/tmp-gcn/usr --enable-plugin --enable-shared --enable-threads=posix --host=x86_64-linux-gnu --program-prefix=x86_64-linux-gnu- --target=x86_64-linux-gnu --with-abi=m64 --with-arch-32=i686 --with-build-config=bootstrap-lto-lean --with-default-libstdcxx-abi=new --with-gcc-major-version-only --with-multilib-list=m32,m64,mx32 --with-target-system-zlib=auto --with-tune=generic --without-cuda-driver -v
Disk Notes: NONE / relatime,rw / Block Size: 4096
Processor Notes: CPU Microcode: 0x830104d
Security Notes: gather_data_sampling: Not affected + itlb_multihit: Not affected + l1tf: Not affected + mds: Not affected + meltdown: Not affected + mmio_stale_data: Not affected + retbleed: Mitigation of untrained return thunk; SMT disabled + spec_rstack_overflow: Mitigation of SMT disabled + spec_store_bypass: Mitigation of SSB disabled via prctl and seccomp + spectre_v1: Mitigation of usercopy/swapgs barriers and __user pointer sanitization + spectre_v2: Mitigation of Retpolines IBPB: conditional STIBP: disabled RSB filling PBRSB-eIBRS: Not affected + srbds: Not affected + tsx_async_abort: Not affected

PVE-SATA-ZFS-RAID10-6DisksOpenBenchmarking.orgPhoronix Test SuiteAMD EPYC 7402 24-Core (12 Cores)QEMU Standard PC (Q35 + ICH9 2009) (3.20230228-4 BIOS)Intel 82G33/G31/P35/P31 + ICH916GB34GB QEMU HDDbochs-drmdrmfbIntel 82801IQEMU MonitorRed Hat Virtio deviceUbuntu 22.045.15.0-84-generic (x86_64)5.15.0-86-generic (x86_64)1.3.238GCC 11.4.0ext41280x800KVMProcessorMotherboardChipsetMemoryDiskGraphicsAudioMonitorNetworkOSKernelsVulkanCompilerFile-SystemScreen ResolutionSystem LayerPVE-SATA-ZFS-RAID10-6Disks PerformanceSystem Logs- Transparent Huge Pages: madvise- --build=x86_64-linux-gnu --disable-vtable-verify --disable-werror --enable-bootstrap --enable-cet --enable-checking=release --enable-clocale=gnu --enable-default-pie --enable-gnu-unique-object --enable-languages=c,ada,c++,go,brig,d,fortran,objc,obj-c++,m2 --enable-libphobos-checking=release --enable-libstdcxx-debug --enable-libstdcxx-time=yes --enable-link-serialization=2 --enable-multiarch --enable-multilib --enable-nls --enable-objc-gc=auto --enable-offload-targets=nvptx-none=/build/gcc-11-XeT9lY/gcc-11-11.4.0/debian/tmp-nvptx/usr,amdgcn-amdhsa=/build/gcc-11-XeT9lY/gcc-11-11.4.0/debian/tmp-gcn/usr --enable-plugin --enable-shared --enable-threads=posix --host=x86_64-linux-gnu --program-prefix=x86_64-linux-gnu- --target=x86_64-linux-gnu --with-abi=m64 --with-arch-32=i686 --with-build-config=bootstrap-lto-lean --with-default-libstdcxx-abi=new --with-gcc-major-version-only --with-multilib-list=m32,m64,mx32 --with-target-system-zlib=auto --with-tune=generic --without-cuda-driver -v - NONE / relatime,rw / Block Size: 4096- CPU Microcode: 0x830104d- PVE-SATA-ZFS-RAID10-6Disks: gather_data_sampling: Not affected + itlb_multihit: Not affected + l1tf: Not affected + mds: Not affected + meltdown: Not affected + mmio_stale_data: Not affected + retbleed: Mitigation of untrained return thunk; SMT disabled + spec_store_bypass: Mitigation of SSB disabled via prctl and seccomp + spectre_v1: Mitigation of usercopy/swapgs barriers and __user pointer sanitization + spectre_v2: Mitigation of Retpolines IBPB: conditional STIBP: disabled RSB filling PBRSB-eIBRS: Not affected + srbds: Not affected + tsx_async_abort: Not affected - PVE-SATA-ZFS-RAID10-8Disks: gather_data_sampling: Not affected + itlb_multihit: Not affected + l1tf: Not affected + mds: Not affected + meltdown: Not affected + mmio_stale_data: Not affected + retbleed: Mitigation of untrained return thunk; SMT disabled + spec_store_bypass: Mitigation of SSB disabled via prctl and seccomp + spectre_v1: Mitigation of usercopy/swapgs barriers and __user pointer sanitization + spectre_v2: Mitigation of Retpolines IBPB: conditional STIBP: disabled RSB filling PBRSB-eIBRS: Not affected + srbds: Not affected + tsx_async_abort: Not affected - PVE-NVME-ZFS-RAID1-2Disks: gather_data_sampling: Not affected + itlb_multihit: Not affected + l1tf: Not affected + mds: Not affected + meltdown: Not affected + mmio_stale_data: Not affected + retbleed: Mitigation of untrained return thunk; SMT disabled + spec_store_bypass: Mitigation of SSB disabled via prctl and seccomp + spectre_v1: Mitigation of usercopy/swapgs barriers and __user pointer sanitization + spectre_v2: Mitigation of Retpolines IBPB: conditional STIBP: disabled RSB filling PBRSB-eIBRS: Not affected + srbds: Not affected + tsx_async_abort: Not affected - PVE-SATA-ZFS-RAID10-8Disks 64K rec: gather_data_sampling: Not affected + itlb_multihit: Not affected + l1tf: Not affected + mds: Not affected + meltdown: Not affected + mmio_stale_data: Not affected + retbleed: Mitigation of untrained return thunk; SMT disabled + spec_rstack_overflow: Mitigation of SMT disabled + spec_store_bypass: Mitigation of SSB disabled via prctl and seccomp + spectre_v1: Mitigation of usercopy/swapgs barriers and __user pointer sanitization + spectre_v2: Mitigation of Retpolines IBPB: conditional STIBP: disabled RSB filling PBRSB-eIBRS: Not affected + srbds: Not affected + tsx_async_abort: Not affected

PVE-SATA-ZFS-RAID10-6DisksPVE-SATA-ZFS-RAID10-8DisksPVE-NVME-ZFS-RAID1-2DisksPVE-SATA-ZFS-RAID10-8Disks 64K recLogarithmic Result OverviewPhoronix Test SuiteFlexible IO TesterFlexible IO TesterFlexible IO TesterFlexible IO TesterFlexible IO TesterFlexible IO TesterFlexible IO TesterFlexible IO TesterRand Write - Linux AIO - No - No - 4KB - 1Rand Write - Linux AIO - No - No - 4KB - 1Rand Read - Linux AIO - No - No - 64KB - 1Rand Read - Linux AIO - No - No - 64KB - 1Rand Write - Linux AIO - No - No - 64KB - 1Rand Write - Linux AIO - No - No - 64KB - 1Rand Read - Linux AIO - No - No - 4KB - 1Rand Read - Linux AIO - No - No - 4KB - 1

PVE-SATA-ZFS-RAID10-6Disksfio: Rand Read - Linux AIO - No - No - 4KB - 1 - Default Test Directoryfio: Rand Read - Linux AIO - No - No - 4KB - 1 - Default Test Directoryfio: Rand Read - Linux AIO - No - No - 64KB - 1 - Default Test Directoryfio: Rand Read - Linux AIO - No - No - 64KB - 1 - Default Test Directoryfio: Rand Write - Linux AIO - No - No - 4KB - 1 - Default Test Directoryfio: Rand Write - Linux AIO - No - No - 4KB - 1 - Default Test Directoryfio: Rand Write - Linux AIO - No - No - 64KB - 1 - Default Test Directoryfio: Rand Write - Linux AIO - No - No - 64KB - 1 - Default Test DirectoryPVE-SATA-ZFS-RAID10-6DisksPVE-SATA-ZFS-RAID10-8DisksPVE-NVME-ZFS-RAID1-2DisksPVE-SATA-ZFS-RAID10-8Disks 64K rec554004.191385012975.2347603415546.151038861076.6117226568722.141421804701.6175226459895.501149741392.3822278527188.411317976244.8899918371799.24929501930.5730889625906.391564777835.7412537219110.374778132112.152064OpenBenchmarking.org

Flexible IO Tester

OpenBenchmarking.orgMiB/s, More Is BetterFlexible IO Tester 3.35Type: Random Read - Engine: Linux AIO - Buffered: No - Direct: No - Block Size: 4KB - Job Count: 1 - Disk Target: Default Test DirectoryPVE-SATA-ZFS-RAID10-8Disks 64K recPVE-NVME-ZFS-RAID1-2DisksPVE-SATA-ZFS-RAID10-8DisksPVE-SATA-ZFS-RAID10-6Disks130K260K390K520K650KSE +/- 3118.29, N = 5SE +/- 1444.82, N = 3SE +/- 2963.15, N = 3SE +/- 907.07, N = 3625906.39527188.41568722.14554004.19MIN: 615863.53 / MAX: 634041.79MIN: 525004.58 / MAX: 529919.12MIN: 564227.4 / MAX: 574314.47MIN: 552904.03 / MAX: 555803.491. (CC) gcc options: -rdynamic -lm -laio -lpthread -ldl -std=gnu99 -ffast-math -include -O3 -fcommon -march=native
OpenBenchmarking.orgMiB/s, More Is BetterFlexible IO Tester 3.35Type: Random Read - Engine: Linux AIO - Buffered: No - Direct: No - Block Size: 4KB - Job Count: 1 - Disk Target: Default Test DirectoryPVE-SATA-ZFS-RAID10-8Disks 64K recPVE-NVME-ZFS-RAID1-2DisksPVE-SATA-ZFS-RAID10-8DisksPVE-SATA-ZFS-RAID10-6Disks110K220K330K440K550KMin: 615863.53 / Avg: 625906.39 / Max: 634041.79Min: 525004.58 / Avg: 527188.41 / Max: 529919.12Min: 564227.4 / Avg: 568722.14 / Max: 574314.47Min: 552904.03 / Avg: 554004.19 / Max: 555803.491. (CC) gcc options: -rdynamic -lm -laio -lpthread -ldl -std=gnu99 -ffast-math -include -O3 -fcommon -march=native

OpenBenchmarking.orgIOPS, More Is BetterFlexible IO Tester 3.35Type: Random Read - Engine: Linux AIO - Buffered: No - Direct: No - Block Size: 4KB - Job Count: 1 - Disk Target: Default Test DirectoryPVE-SATA-ZFS-RAID10-8Disks 64K recPVE-NVME-ZFS-RAID1-2DisksPVE-SATA-ZFS-RAID10-8DisksPVE-SATA-ZFS-RAID10-6Disks30K60K90K120K150KSE +/- 779.58, N = 5SE +/- 361.19, N = 3SE +/- 740.76, N = 3SE +/- 226.74, N = 3156477131797142180138501MIN: 153965.84 / MAX: 158510.43MIN: 131251.14 / MAX: 132479.72MIN: 141056.82 / MAX: 143578.51MIN: 138226.02 / MAX: 138950.811. (CC) gcc options: -rdynamic -lm -laio -lpthread -ldl -std=gnu99 -ffast-math -include -O3 -fcommon -march=native
OpenBenchmarking.orgIOPS, More Is BetterFlexible IO Tester 3.35Type: Random Read - Engine: Linux AIO - Buffered: No - Direct: No - Block Size: 4KB - Job Count: 1 - Disk Target: Default Test DirectoryPVE-SATA-ZFS-RAID10-8Disks 64K recPVE-NVME-ZFS-RAID1-2DisksPVE-SATA-ZFS-RAID10-8DisksPVE-SATA-ZFS-RAID10-6Disks30K60K90K120K150KMin: 153965.84 / Avg: 156476.56 / Max: 158510.43Min: 131251.14 / Avg: 131797.06 / Max: 132479.72Min: 141056.82 / Avg: 142180.47 / Max: 143578.51Min: 138226.02 / Avg: 138501.05 / Max: 138950.811. (CC) gcc options: -rdynamic -lm -laio -lpthread -ldl -std=gnu99 -ffast-math -include -O3 -fcommon -march=native

OpenBenchmarking.orgMiB/s, More Is BetterFlexible IO Tester 3.35Type: Random Read - Engine: Linux AIO - Buffered: No - Direct: No - Block Size: 64KB - Job Count: 1 - Disk Target: Default Test DirectoryPVE-SATA-ZFS-RAID10-8Disks 64K recPVE-NVME-ZFS-RAID1-2DisksPVE-SATA-ZFS-RAID10-8DisksPVE-SATA-ZFS-RAID10-6Disks2K4K6K8K10KSE +/- 208.92, N = 5SE +/- 90.93, N = 15SE +/- 51.24, N = 3SE +/- 11.78, N = 37835.746244.884701.612975.23MIN: 7552.51 / MAX: 8659.79MIN: 5507.27 / MAX: 6830.17MIN: 4624.79 / MAX: 4798.76MIN: 2951.66 / MAX: 2987.321. (CC) gcc options: -rdynamic -lm -laio -lpthread -ldl -std=gnu99 -ffast-math -include -O3 -fcommon -march=native
OpenBenchmarking.orgMiB/s, More Is BetterFlexible IO Tester 3.35Type: Random Read - Engine: Linux AIO - Buffered: No - Direct: No - Block Size: 64KB - Job Count: 1 - Disk Target: Default Test DirectoryPVE-SATA-ZFS-RAID10-8Disks 64K recPVE-NVME-ZFS-RAID1-2DisksPVE-SATA-ZFS-RAID10-8DisksPVE-SATA-ZFS-RAID10-6Disks14002800420056007000Min: 7552.51 / Avg: 7835.74 / Max: 8659.79Min: 5507.27 / Avg: 6244.88 / Max: 6830.17Min: 4624.79 / Avg: 4701.61 / Max: 4798.76Min: 2951.66 / Avg: 2975.23 / Max: 2987.321. (CC) gcc options: -rdynamic -lm -laio -lpthread -ldl -std=gnu99 -ffast-math -include -O3 -fcommon -march=native

OpenBenchmarking.orgIOPS, More Is BetterFlexible IO Tester 3.35Type: Random Read - Engine: Linux AIO - Buffered: No - Direct: No - Block Size: 64KB - Job Count: 1 - Disk Target: Default Test DirectoryPVE-SATA-ZFS-RAID10-8Disks 64K recPVE-NVME-ZFS-RAID1-2DisksPVE-SATA-ZFS-RAID10-8DisksPVE-SATA-ZFS-RAID10-6Disks30K60K90K120K150KSE +/- 3342.65, N = 5SE +/- 1454.83, N = 15SE +/- 819.79, N = 3SE +/- 188.54, N = 3125372999187522647603MIN: 120840.15 / MAX: 138556.61MIN: 88116.28 / MAX: 109282.62MIN: 73996.52 / MAX: 76779.97MIN: 47226.44 / MAX: 47796.941. (CC) gcc options: -rdynamic -lm -laio -lpthread -ldl -std=gnu99 -ffast-math -include -O3 -fcommon -march=native
OpenBenchmarking.orgIOPS, More Is BetterFlexible IO Tester 3.35Type: Random Read - Engine: Linux AIO - Buffered: No - Direct: No - Block Size: 64KB - Job Count: 1 - Disk Target: Default Test DirectoryPVE-SATA-ZFS-RAID10-8Disks 64K recPVE-NVME-ZFS-RAID1-2DisksPVE-SATA-ZFS-RAID10-8DisksPVE-SATA-ZFS-RAID10-6Disks20K40K60K80K100KMin: 120840.15 / Avg: 125371.76 / Max: 138556.61Min: 88116.28 / Avg: 99918.01 / Max: 109282.62Min: 73996.52 / Avg: 75225.67 / Max: 76779.97Min: 47226.44 / Avg: 47603.47 / Max: 47796.941. (CC) gcc options: -rdynamic -lm -laio -lpthread -ldl -std=gnu99 -ffast-math -include -O3 -fcommon -march=native

OpenBenchmarking.orgMiB/s, More Is BetterFlexible IO Tester 3.35Type: Random Write - Engine: Linux AIO - Buffered: No - Direct: No - Block Size: 4KB - Job Count: 1 - Disk Target: Default Test DirectoryPVE-SATA-ZFS-RAID10-8Disks 64K recPVE-NVME-ZFS-RAID1-2DisksPVE-SATA-ZFS-RAID10-8DisksPVE-SATA-ZFS-RAID10-6Disks100K200K300K400K500KSE +/- 80.22, N = 5SE +/- 2036.04, N = 3SE +/- 2672.63, N = 3SE +/- 2198.09, N = 319110.37371799.24459895.50415546.15MIN: 368418.88 / MAX: 375455.69MIN: 454629.83 / MAX: 463324.23MIN: 412349 / MAX: 419757.861. (CC) gcc options: -rdynamic -lm -laio -lpthread -ldl -std=gnu99 -ffast-math -include -O3 -fcommon -march=native
OpenBenchmarking.orgMiB/s, More Is BetterFlexible IO Tester 3.35Type: Random Write - Engine: Linux AIO - Buffered: No - Direct: No - Block Size: 4KB - Job Count: 1 - Disk Target: Default Test DirectoryPVE-SATA-ZFS-RAID10-8Disks 64K recPVE-NVME-ZFS-RAID1-2DisksPVE-SATA-ZFS-RAID10-8DisksPVE-SATA-ZFS-RAID10-6Disks80K160K240K320K400KMin: 18928.14 / Avg: 19110.37 / Max: 19321.74Min: 368418.88 / Avg: 371799.24 / Max: 375455.69Min: 454629.83 / Avg: 459895.5 / Max: 463324.23Min: 412349 / Avg: 415546.15 / Max: 419757.861. (CC) gcc options: -rdynamic -lm -laio -lpthread -ldl -std=gnu99 -ffast-math -include -O3 -fcommon -march=native

OpenBenchmarking.orgIOPS, More Is BetterFlexible IO Tester 3.35Type: Random Write - Engine: Linux AIO - Buffered: No - Direct: No - Block Size: 4KB - Job Count: 1 - Disk Target: Default Test DirectoryPVE-SATA-ZFS-RAID10-8Disks 64K recPVE-NVME-ZFS-RAID1-2DisksPVE-SATA-ZFS-RAID10-8DisksPVE-SATA-ZFS-RAID10-6Disks20K40K60K80K100KSE +/- 20.06, N = 5SE +/- 509.01, N = 3SE +/- 668.16, N = 3SE +/- 549.53, N = 3477892950114974103886MIN: 92104.66 / MAX: 93863.87MIN: 113657.43 / MAX: 115831.02MIN: 103087.2 / MAX: 104939.441. (CC) gcc options: -rdynamic -lm -laio -lpthread -ldl -std=gnu99 -ffast-math -include -O3 -fcommon -march=native
OpenBenchmarking.orgIOPS, More Is BetterFlexible IO Tester 3.35Type: Random Write - Engine: Linux AIO - Buffered: No - Direct: No - Block Size: 4KB - Job Count: 1 - Disk Target: Default Test DirectoryPVE-SATA-ZFS-RAID10-8Disks 64K recPVE-NVME-ZFS-RAID1-2DisksPVE-SATA-ZFS-RAID10-8DisksPVE-SATA-ZFS-RAID10-6Disks20K40K60K80K100KMin: 4732 / Avg: 4777.56 / Max: 4830.42Min: 92104.66 / Avg: 92949.75 / Max: 93863.87Min: 113657.43 / Avg: 114973.84 / Max: 115831.02Min: 103087.2 / Avg: 103886.48 / Max: 104939.441. (CC) gcc options: -rdynamic -lm -laio -lpthread -ldl -std=gnu99 -ffast-math -include -O3 -fcommon -march=native

OpenBenchmarking.orgMiB/s, More Is BetterFlexible IO Tester 3.35Type: Random Write - Engine: Linux AIO - Buffered: No - Direct: No - Block Size: 64KB - Job Count: 1 - Disk Target: Default Test DirectoryPVE-SATA-ZFS-RAID10-8Disks 64K recPVE-NVME-ZFS-RAID1-2DisksPVE-SATA-ZFS-RAID10-8DisksPVE-SATA-ZFS-RAID10-6Disks30K60K90K120K150KSE +/- 188.68, N = 5SE +/- 14.82, N = 3SE +/- 2.77, N = 3SE +/- 9.54, N = 15132112.151930.571392.381076.61MIN: 131788.07 / MAX: 132708.911. (CC) gcc options: -rdynamic -lm -laio -lpthread -ldl -std=gnu99 -ffast-math -include -O3 -fcommon -march=native
OpenBenchmarking.orgMiB/s, More Is BetterFlexible IO Tester 3.35Type: Random Write - Engine: Linux AIO - Buffered: No - Direct: No - Block Size: 64KB - Job Count: 1 - Disk Target: Default Test DirectoryPVE-SATA-ZFS-RAID10-8Disks 64K recPVE-NVME-ZFS-RAID1-2DisksPVE-SATA-ZFS-RAID10-8DisksPVE-SATA-ZFS-RAID10-6Disks20K40K60K80K100KMin: 131788.07 / Avg: 132112.15 / Max: 132708.91Min: 1904.82 / Avg: 1930.57 / Max: 1956.16Min: 1387.63 / Avg: 1392.38 / Max: 1397.24Min: 1034.96 / Avg: 1076.61 / Max: 1163.541. (CC) gcc options: -rdynamic -lm -laio -lpthread -ldl -std=gnu99 -ffast-math -include -O3 -fcommon -march=native

OpenBenchmarking.orgIOPS, More Is BetterFlexible IO Tester 3.35Type: Random Write - Engine: Linux AIO - Buffered: No - Direct: No - Block Size: 64KB - Job Count: 1 - Disk Target: Default Test DirectoryPVE-SATA-ZFS-RAID10-8Disks 64K recPVE-NVME-ZFS-RAID1-2DisksPVE-SATA-ZFS-RAID10-8DisksPVE-SATA-ZFS-RAID10-6Disks7K14K21K28K35KSE +/- 2.95, N = 5SE +/- 237.15, N = 3SE +/- 44.49, N = 3SE +/- 152.69, N = 152064308892227817226MIN: 30476.82 / MAX: 31298.32MIN: 22202 / MAX: 22356.08MIN: 16559.33 / MAX: 18616.541. (CC) gcc options: -rdynamic -lm -laio -lpthread -ldl -std=gnu99 -ffast-math -include -O3 -fcommon -march=native
OpenBenchmarking.orgIOPS, More Is BetterFlexible IO Tester 3.35Type: Random Write - Engine: Linux AIO - Buffered: No - Direct: No - Block Size: 64KB - Job Count: 1 - Disk Target: Default Test DirectoryPVE-SATA-ZFS-RAID10-8Disks 64K recPVE-NVME-ZFS-RAID1-2DisksPVE-SATA-ZFS-RAID10-8DisksPVE-SATA-ZFS-RAID10-6Disks5K10K15K20K25KMin: 2059.15 / Avg: 2064.21 / Max: 2073.53Min: 30476.82 / Avg: 30888.88 / Max: 31298.32Min: 22202 / Avg: 22278.07 / Max: 22356.08Min: 16559.33 / Avg: 17225.6 / Max: 18616.541. (CC) gcc options: -rdynamic -lm -laio -lpthread -ldl -std=gnu99 -ffast-math -include -O3 -fcommon -march=native

8 Results Shown

Flexible IO Tester:
  Rand Read - Linux AIO - No - No - 4KB - 1 - Default Test Directory:
    MiB/s
    IOPS
  Rand Read - Linux AIO - No - No - 64KB - 1 - Default Test Directory:
    MiB/s
    IOPS
  Rand Write - Linux AIO - No - No - 4KB - 1 - Default Test Directory:
    MiB/s
    IOPS
  Rand Write - Linux AIO - No - No - 64KB - 1 - Default Test Directory:
    MiB/s
    IOPS

PVE-SATA-ZFS-RAID10-6Disks

Kernel Notes: Transparent Huge Pages: madvise
Compiler Notes: --build=x86_64-linux-gnu --disable-vtable-verify --disable-werror --enable-bootstrap --enable-cet --enable-checking=release --enable-clocale=gnu --enable-default-pie --enable-gnu-unique-object --enable-languages=c,ada,c++,go,brig,d,fortran,objc,obj-c++,m2 --enable-libphobos-checking=release --enable-libstdcxx-debug --enable-libstdcxx-time=yes --enable-link-serialization=2 --enable-multiarch --enable-multilib --enable-nls --enable-objc-gc=auto --enable-offload-targets=nvptx-none=/build/gcc-11-XeT9lY/gcc-11-11.4.0/debian/tmp-nvptx/usr,amdgcn-amdhsa=/build/gcc-11-XeT9lY/gcc-11-11.4.0/debian/tmp-gcn/usr --enable-plugin --enable-shared --enable-threads=posix --host=x86_64-linux-gnu --program-prefix=x86_64-linux-gnu- --target=x86_64-linux-gnu --with-abi=m64 --with-arch-32=i686 --with-build-config=bootstrap-lto-lean --with-default-libstdcxx-abi=new --with-gcc-major-version-only --with-multilib-list=m32,m64,mx32 --with-target-system-zlib=auto --with-tune=generic --without-cuda-driver -v
Disk Notes: NONE / relatime,rw / Block Size: 4096
Processor Notes: CPU Microcode: 0x830104d
Security Notes: gather_data_sampling: Not affected + itlb_multihit: Not affected + l1tf: Not affected + mds: Not affected + meltdown: Not affected + mmio_stale_data: Not affected + retbleed: Mitigation of untrained return thunk; SMT disabled + spec_store_bypass: Mitigation of SSB disabled via prctl and seccomp + spectre_v1: Mitigation of usercopy/swapgs barriers and __user pointer sanitization + spectre_v2: Mitigation of Retpolines IBPB: conditional STIBP: disabled RSB filling PBRSB-eIBRS: Not affected + srbds: Not affected + tsx_async_abort: Not affected

Testing initiated at 2 October 2023 10:05 by user tcstudios.

PVE-SATA-ZFS-RAID10-8Disks

Kernel Notes: Transparent Huge Pages: madvise
Compiler Notes: --build=x86_64-linux-gnu --disable-vtable-verify --disable-werror --enable-bootstrap --enable-cet --enable-checking=release --enable-clocale=gnu --enable-default-pie --enable-gnu-unique-object --enable-languages=c,ada,c++,go,brig,d,fortran,objc,obj-c++,m2 --enable-libphobos-checking=release --enable-libstdcxx-debug --enable-libstdcxx-time=yes --enable-link-serialization=2 --enable-multiarch --enable-multilib --enable-nls --enable-objc-gc=auto --enable-offload-targets=nvptx-none=/build/gcc-11-XeT9lY/gcc-11-11.4.0/debian/tmp-nvptx/usr,amdgcn-amdhsa=/build/gcc-11-XeT9lY/gcc-11-11.4.0/debian/tmp-gcn/usr --enable-plugin --enable-shared --enable-threads=posix --host=x86_64-linux-gnu --program-prefix=x86_64-linux-gnu- --target=x86_64-linux-gnu --with-abi=m64 --with-arch-32=i686 --with-build-config=bootstrap-lto-lean --with-default-libstdcxx-abi=new --with-gcc-major-version-only --with-multilib-list=m32,m64,mx32 --with-target-system-zlib=auto --with-tune=generic --without-cuda-driver -v
Disk Notes: NONE / relatime,rw / Block Size: 4096
Processor Notes: CPU Microcode: 0x830104d
Security Notes: gather_data_sampling: Not affected + itlb_multihit: Not affected + l1tf: Not affected + mds: Not affected + meltdown: Not affected + mmio_stale_data: Not affected + retbleed: Mitigation of untrained return thunk; SMT disabled + spec_store_bypass: Mitigation of SSB disabled via prctl and seccomp + spectre_v1: Mitigation of usercopy/swapgs barriers and __user pointer sanitization + spectre_v2: Mitigation of Retpolines IBPB: conditional STIBP: disabled RSB filling PBRSB-eIBRS: Not affected + srbds: Not affected + tsx_async_abort: Not affected

Testing initiated at 2 October 2023 11:32 by user tcstudios.

PVE-NVME-ZFS-RAID1-2Disks

Processor: AMD EPYC 7402 24-Core (12 Cores), Motherboard: QEMU Standard PC (Q35 + ICH9 2009) (3.20230228-4 BIOS), Chipset: Intel 82G33/G31/P35/P31 + ICH9, Memory: 16GB, Disk: 34GB QEMU HDD, Graphics: bochs-drmdrmfb, Audio: Intel 82801I, Monitor: QEMU Monitor, Network: Red Hat Virtio device

OS: Ubuntu 22.04, Kernel: 5.15.0-84-generic (x86_64), Vulkan: 1.3.238, Compiler: GCC 11.4.0, File-System: ext4, Screen Resolution: 1280x800, System Layer: KVM

Kernel Notes: Transparent Huge Pages: madvise
Compiler Notes: --build=x86_64-linux-gnu --disable-vtable-verify --disable-werror --enable-bootstrap --enable-cet --enable-checking=release --enable-clocale=gnu --enable-default-pie --enable-gnu-unique-object --enable-languages=c,ada,c++,go,brig,d,fortran,objc,obj-c++,m2 --enable-libphobos-checking=release --enable-libstdcxx-debug --enable-libstdcxx-time=yes --enable-link-serialization=2 --enable-multiarch --enable-multilib --enable-nls --enable-objc-gc=auto --enable-offload-targets=nvptx-none=/build/gcc-11-XeT9lY/gcc-11-11.4.0/debian/tmp-nvptx/usr,amdgcn-amdhsa=/build/gcc-11-XeT9lY/gcc-11-11.4.0/debian/tmp-gcn/usr --enable-plugin --enable-shared --enable-threads=posix --host=x86_64-linux-gnu --program-prefix=x86_64-linux-gnu- --target=x86_64-linux-gnu --with-abi=m64 --with-arch-32=i686 --with-build-config=bootstrap-lto-lean --with-default-libstdcxx-abi=new --with-gcc-major-version-only --with-multilib-list=m32,m64,mx32 --with-target-system-zlib=auto --with-tune=generic --without-cuda-driver -v
Disk Notes: NONE / relatime,rw / Block Size: 4096
Processor Notes: CPU Microcode: 0x830104d
Security Notes: gather_data_sampling: Not affected + itlb_multihit: Not affected + l1tf: Not affected + mds: Not affected + meltdown: Not affected + mmio_stale_data: Not affected + retbleed: Mitigation of untrained return thunk; SMT disabled + spec_store_bypass: Mitigation of SSB disabled via prctl and seccomp + spectre_v1: Mitigation of usercopy/swapgs barriers and __user pointer sanitization + spectre_v2: Mitigation of Retpolines IBPB: conditional STIBP: disabled RSB filling PBRSB-eIBRS: Not affected + srbds: Not affected + tsx_async_abort: Not affected

Testing initiated at 6 October 2023 04:38 by user tcstudios.

PVE-SATA-ZFS-RAID10-8Disks 64K rec

Processor: AMD EPYC 7402 24-Core (12 Cores), Motherboard: QEMU Standard PC (Q35 + ICH9 2009) (3.20230228-4 BIOS), Chipset: Intel 82G33/G31/P35/P31 + ICH9, Memory: 16GB, Disk: 34GB QEMU HDD, Graphics: bochs-drmdrmfb, Audio: Intel 82801I, Monitor: QEMU Monitor, Network: Red Hat Virtio device

OS: Ubuntu 22.04, Kernel: 5.15.0-86-generic (x86_64), Vulkan: 1.3.238, Compiler: GCC 11.4.0, File-System: ext4, Screen Resolution: 1280x800, System Layer: KVM

Kernel Notes: Transparent Huge Pages: madvise
Compiler Notes: --build=x86_64-linux-gnu --disable-vtable-verify --disable-werror --enable-bootstrap --enable-cet --enable-checking=release --enable-clocale=gnu --enable-default-pie --enable-gnu-unique-object --enable-languages=c,ada,c++,go,brig,d,fortran,objc,obj-c++,m2 --enable-libphobos-checking=release --enable-libstdcxx-debug --enable-libstdcxx-time=yes --enable-link-serialization=2 --enable-multiarch --enable-multilib --enable-nls --enable-objc-gc=auto --enable-offload-targets=nvptx-none=/build/gcc-11-XeT9lY/gcc-11-11.4.0/debian/tmp-nvptx/usr,amdgcn-amdhsa=/build/gcc-11-XeT9lY/gcc-11-11.4.0/debian/tmp-gcn/usr --enable-plugin --enable-shared --enable-threads=posix --host=x86_64-linux-gnu --program-prefix=x86_64-linux-gnu- --target=x86_64-linux-gnu --with-abi=m64 --with-arch-32=i686 --with-build-config=bootstrap-lto-lean --with-default-libstdcxx-abi=new --with-gcc-major-version-only --with-multilib-list=m32,m64,mx32 --with-target-system-zlib=auto --with-tune=generic --without-cuda-driver -v
Disk Notes: NONE / relatime,rw / Block Size: 4096
Processor Notes: CPU Microcode: 0x830104d
Security Notes: gather_data_sampling: Not affected + itlb_multihit: Not affected + l1tf: Not affected + mds: Not affected + meltdown: Not affected + mmio_stale_data: Not affected + retbleed: Mitigation of untrained return thunk; SMT disabled + spec_rstack_overflow: Mitigation of SMT disabled + spec_store_bypass: Mitigation of SSB disabled via prctl and seccomp + spectre_v1: Mitigation of usercopy/swapgs barriers and __user pointer sanitization + spectre_v2: Mitigation of Retpolines IBPB: conditional STIBP: disabled RSB filling PBRSB-eIBRS: Not affected + srbds: Not affected + tsx_async_abort: Not affected

Testing initiated at 8 October 2023 04:56 by user tcstudios.