sa3400d

PVE-NVME-R10-4DISK-fio

Compare your own system(s) to this result file with the Phoronix Test Suite by running the command: phoronix-test-suite benchmark 2311037-TCST-231001402
Jump To Table - Results

View

Do Not Show Noisy Results
Do Not Show Results With Incomplete Data
Do Not Show Results With Little Change/Spread
List Notable Results

Statistics

Show Overall Harmonic Mean(s)
Show Overall Geometric Mean
Show Wins / Losses Counts (Pie Chart)
Normalize Results
Remove Outliers Before Calculating Averages

Graph Settings

Force Line Graphs Where Applicable
Convert To Scalar Where Applicable
Prefer Vertical Bar Graphs

Additional Graphs

Show Perf Per Core/Thread Calculation Graphs Where Applicable

Multi-Way Comparison

Condense Multi-Option Tests Into Single Result Graphs

Table

Show Detailed System Result Table

Run Management

Highlight
Result
Hide
Result
Result
Identifier
Performance Per
Dollar
Date
Run
  Test
  Duration
iSCSI XCP-NG
August 31 2023
  2 Hours, 29 Minutes
NFS XCP-NG
August 31 2023
  3 Hours, 53 Minutes
PVE-SAS-SSD-fio
September 30 2023
  4 Hours, 45 Minutes
PVE-SATA-SSD-fio
October 01 2023
  2 Hours, 45 Minutes
XCP-NVME-R1-fio
October 01 2023
  3 Hours, 53 Minutes
PVE-NVME-R10-4DISK-fio
November 03 2023
  2 Hours, 19 Minutes
Invert Hiding All Results Option
  3 Hours, 21 Minutes

Only show results where is faster than
Only show results matching title/arguments (delimit multiple options with a comma):
Do not show results matching title/arguments (delimit multiple options with a comma):

iSCSI XCP-NG

Kernel Notes: Transparent Huge Pages: madvise
Compiler Notes: --build=x86_64-linux-gnu --disable-vtable-verify --disable-werror --enable-bootstrap --enable-cet --enable-checking=release --enable-clocale=gnu --enable-default-pie --enable-gnu-unique-object --enable-languages=c,ada,c++,go,brig,d,fortran,objc,obj-c++,m2 --enable-libphobos-checking=release --enable-libstdcxx-debug --enable-libstdcxx-time=yes --enable-link-serialization=2 --enable-multiarch --enable-multilib --enable-nls --enable-objc-gc=auto --enable-offload-targets=nvptx-none=/build/gcc-11-XeT9lY/gcc-11-11.4.0/debian/tmp-nvptx/usr,amdgcn-amdhsa=/build/gcc-11-XeT9lY/gcc-11-11.4.0/debian/tmp-gcn/usr --enable-plugin --enable-shared --enable-threads=posix --host=x86_64-linux-gnu --program-prefix=x86_64-linux-gnu- --target=x86_64-linux-gnu --with-abi=m64 --with-arch-32=i686 --with-build-config=bootstrap-lto-lean --with-default-libstdcxx-abi=new --with-gcc-major-version-only --with-multilib-list=m32,m64,mx32 --with-target-system-zlib=auto --with-tune=generic --without-cuda-driver -v
Processor Notes: CPU Microcode: 0xa20120a
Disk Mount Options Notes: relatime,rw
Disk Details Notes: Block Size: 4096
Security Notes: gather_data_sampling: Not affected + itlb_multihit: Not affected + l1tf: Not affected + mds: Not affected + meltdown: Not affected + mmio_stale_data: Not affected + retbleed: Not affected + spec_store_bypass: Mitigation of SSB disabled via prctl and seccomp + spectre_v1: Mitigation of usercopy/swapgs barriers and __user pointer sanitization + spectre_v2: Mitigation of Retpolines IBPB: conditional IBRS_FW STIBP: disabled RSB filling PBRSB-eIBRS: Not affected + srbds: Not affected + tsx_async_abort: Not affected

NFS XCP-NG

Processor: 2 x AMD Ryzen 9 5900X 12-Core (32 Cores), Motherboard: Xen HVM domU v4.13, Chipset: Intel 440FX 82441FX PMC, Memory: 16GB, Disk: 29GB, Graphics: bochs-drmdrmfb

OS: Ubuntu 22.04, Kernel: 5.15.0-82-generic (x86_64), Vulkan: 1.3.238, Compiler: GCC 11.4.0, File-System: ext4, Screen Resolution: 1024x768, System Layer: Xen HVM domU 4.13

PVE-SAS-SSD-fio

Kernel Notes: Transparent Huge Pages: madvise
Compiler Notes: --build=x86_64-linux-gnu --disable-vtable-verify --disable-werror --enable-bootstrap --enable-cet --enable-checking=release --enable-clocale=gnu --enable-default-pie --enable-gnu-unique-object --enable-languages=c,ada,c++,go,brig,d,fortran,objc,obj-c++,m2 --enable-libphobos-checking=release --enable-libstdcxx-debug --enable-libstdcxx-time=yes --enable-link-serialization=2 --enable-multiarch --enable-multilib --enable-nls --enable-objc-gc=auto --enable-offload-targets=nvptx-none=/build/gcc-11-XeT9lY/gcc-11-11.4.0/debian/tmp-nvptx/usr,amdgcn-amdhsa=/build/gcc-11-XeT9lY/gcc-11-11.4.0/debian/tmp-gcn/usr --enable-plugin --enable-shared --enable-threads=posix --host=x86_64-linux-gnu --program-prefix=x86_64-linux-gnu- --target=x86_64-linux-gnu --with-abi=m64 --with-arch-32=i686 --with-build-config=bootstrap-lto-lean --with-default-libstdcxx-abi=new --with-gcc-major-version-only --with-multilib-list=m32,m64,mx32 --with-target-system-zlib=auto --with-tune=generic --without-cuda-driver -v
Disk Notes: NONE / relatime,rw / Block Size: 4096
Processor Notes: CPU Microcode: 0x2007006
Security Notes: gather_data_sampling: Unknown: Dependent on hypervisor status + itlb_multihit: Not affected + l1tf: Mitigation of PTE Inversion; VMX: flush not necessary SMT disabled + mds: Mitigation of Clear buffers; SMT Host state unknown + meltdown: Mitigation of PTI + mmio_stale_data: Vulnerable: Clear buffers attempted no microcode; SMT Host state unknown + retbleed: Mitigation of IBRS + spec_store_bypass: Mitigation of SSB disabled via prctl and seccomp + spectre_v1: Mitigation of usercopy/swapgs barriers and __user pointer sanitization + spectre_v2: Mitigation of IBRS IBPB: conditional STIBP: disabled RSB filling PBRSB-eIBRS: Not affected + srbds: Not affected + tsx_async_abort: Mitigation of Clear buffers; SMT Host state unknown

PVE-SATA-SSD-fio

Processor: 2 x Intel Xeon Gold 6152 (12 Cores), Motherboard: QEMU Standard PC (Q35 + ICH9 2009) (3.20230228-4 BIOS), Chipset: Intel 82G33/G31/P35/P31 + ICH9, Memory: 16GB, Disk: 34GB QEMU HDD, Graphics: bochs-drmdrmfb, Audio: Intel 82801I, Monitor: QEMU Monitor, Network: Red Hat Virtio device

OS: Ubuntu 22.04, Kernel: 5.15.0-84-generic (x86_64), Vulkan: 1.3.238, Compiler: GCC 11.4.0, File-System: ext4, Screen Resolution: 1280x800, System Layer: KVM

XCP-NVME-R1-fio

Processor: 3 x AMD EPYC 7402 24-Core (12 Cores), Motherboard: Xen HVM domU v4.13, Chipset: Intel 440FX 82441FX PMC, Memory: 16GB, Disk: 30GB, Graphics: bochs-drmdrmfb

OS: Ubuntu 22.04, Kernel: 5.15.0-84-generic (x86_64), Vulkan: 1.3.238, Compiler: GCC 11.4.0, File-System: ext4, Screen Resolution: 1024x768, System Layer: Xen HVM domU 4.13

Kernel Notes: Transparent Huge Pages: madvise
Compiler Notes: --build=x86_64-linux-gnu --disable-vtable-verify --disable-werror --enable-bootstrap --enable-cet --enable-checking=release --enable-clocale=gnu --enable-default-pie --enable-gnu-unique-object --enable-languages=c,ada,c++,go,brig,d,fortran,objc,obj-c++,m2 --enable-libphobos-checking=release --enable-libstdcxx-debug --enable-libstdcxx-time=yes --enable-link-serialization=2 --enable-multiarch --enable-multilib --enable-nls --enable-objc-gc=auto --enable-offload-targets=nvptx-none=/build/gcc-11-XeT9lY/gcc-11-11.4.0/debian/tmp-nvptx/usr,amdgcn-amdhsa=/build/gcc-11-XeT9lY/gcc-11-11.4.0/debian/tmp-gcn/usr --enable-plugin --enable-shared --enable-threads=posix --host=x86_64-linux-gnu --program-prefix=x86_64-linux-gnu- --target=x86_64-linux-gnu --with-abi=m64 --with-arch-32=i686 --with-build-config=bootstrap-lto-lean --with-default-libstdcxx-abi=new --with-gcc-major-version-only --with-multilib-list=m32,m64,mx32 --with-target-system-zlib=auto --with-tune=generic --without-cuda-driver -v
Processor Notes: CPU Microcode: 0x830107a
Disk Mount Options Notes: relatime,rw
Disk Details Notes: Block Size: 4096
Security Notes: gather_data_sampling: Not affected + itlb_multihit: Not affected + l1tf: Not affected + mds: Not affected + meltdown: Not affected + mmio_stale_data: Not affected + retbleed: Mitigation of untrained return thunk; SMT disabled + spec_store_bypass: Mitigation of SSB disabled via prctl and seccomp + spectre_v1: Mitigation of usercopy/swapgs barriers and __user pointer sanitization + spectre_v2: Mitigation of Retpolines IBPB: conditional STIBP: disabled RSB filling PBRSB-eIBRS: Not affected + srbds: Not affected + tsx_async_abort: Not affected

PVE-NVME-R10-4DISK-fio

Processor: AMD EPYC 7543 32-Core (12 Cores), Motherboard: QEMU Standard PC (Q35 + ICH9 2009) (3.20230228-4 BIOS), Chipset: Intel 82G33/G31/P35/P31 + ICH9, Memory: 16GB, Disk: 34GB QEMU HDD, Graphics: bochs-drmdrmfb, Audio: Intel 82801I, Monitor: QEMU Monitor, Network: Red Hat Virtio device

OS: Ubuntu 22.04, Kernel: 5.15.0-86-generic (x86_64), Vulkan: 1.3.238, Compiler: GCC 11.4.0, File-System: ext4, Screen Resolution: 1280x800, System Layer: KVM

Kernel Notes: Transparent Huge Pages: madvise
Compiler Notes: --build=x86_64-linux-gnu --disable-vtable-verify --disable-werror --enable-bootstrap --enable-cet --enable-checking=release --enable-clocale=gnu --enable-default-pie --enable-gnu-unique-object --enable-languages=c,ada,c++,go,brig,d,fortran,objc,obj-c++,m2 --enable-libphobos-checking=release --enable-libstdcxx-debug --enable-libstdcxx-time=yes --enable-link-serialization=2 --enable-multiarch --enable-multilib --enable-nls --enable-objc-gc=auto --enable-offload-targets=nvptx-none=/build/gcc-11-XeT9lY/gcc-11-11.4.0/debian/tmp-nvptx/usr,amdgcn-amdhsa=/build/gcc-11-XeT9lY/gcc-11-11.4.0/debian/tmp-gcn/usr --enable-plugin --enable-shared --enable-threads=posix --host=x86_64-linux-gnu --program-prefix=x86_64-linux-gnu- --target=x86_64-linux-gnu --with-abi=m64 --with-arch-32=i686 --with-build-config=bootstrap-lto-lean --with-default-libstdcxx-abi=new --with-gcc-major-version-only --with-multilib-list=m32,m64,mx32 --with-target-system-zlib=auto --with-tune=generic --without-cuda-driver -v
Disk Notes: NONE / relatime,rw / Block Size: 4096
Processor Notes: CPU Microcode: 0xa00110d
Security Notes: gather_data_sampling: Not affected + itlb_multihit: Not affected + l1tf: Not affected + mds: Not affected + meltdown: Not affected + mmio_stale_data: Not affected + retbleed: Not affected + spec_rstack_overflow: Mitigation of safe RET no microcode + spec_store_bypass: Mitigation of SSB disabled via prctl and seccomp + spectre_v1: Mitigation of usercopy/swapgs barriers and __user pointer sanitization + spectre_v2: Mitigation of Retpolines IBPB: conditional IBRS_FW STIBP: disabled RSB filling PBRSB-eIBRS: Not affected + srbds: Not affected + tsx_async_abort: Not affected

sa3400dProcessorMotherboardChipsetMemoryDiskGraphicsAudioMonitorNetworkOSKernelVulkanCompilerFile-SystemScreen ResolutionSystem LayeriSCSI XCP-NGNFS XCP-NGPVE-SAS-SSD-fioPVE-SATA-SSD-fioXCP-NVME-R1-fioPVE-NVME-R10-4DISK-fio2 x AMD Ryzen 9 5900X 12-Core (32 Cores)Xen HVM domU v4.13Intel 440FX 82441FX PMC16GB29GBbochs-drmdrmfbUbuntu 22.045.15.0-82-generic (x86_64)1.3.238GCC 11.4.0ext41024x768Xen HVM domU 4.132 x Intel Xeon Gold 6152 (12 Cores)QEMU Standard PC (Q35 + ICH9 2009) (3.20230228-4 BIOS)Intel 82G33/G31/P35/P31 + ICH934GB QEMU HDDIntel 82801IQEMU MonitorRed Hat Virtio device5.15.0-84-generic (x86_64)1280x800KVM3 x AMD EPYC 7402 24-Core (12 Cores)Xen HVM domU v4.13Intel 440FX 82441FX PMC30GB1024x768Xen HVM domU 4.13AMD EPYC 7543 32-Core (12 Cores)QEMU Standard PC (Q35 + ICH9 2009) (3.20230228-4 BIOS)Intel 82G33/G31/P35/P31 + ICH934GB QEMU HDDIntel 82801IQEMU MonitorRed Hat Virtio device5.15.0-86-generic (x86_64)1280x800KVMOpenBenchmarking.orgKernel Details- Transparent Huge Pages: madviseCompiler Details- --build=x86_64-linux-gnu --disable-vtable-verify --disable-werror --enable-bootstrap --enable-cet --enable-checking=release --enable-clocale=gnu --enable-default-pie --enable-gnu-unique-object --enable-languages=c,ada,c++,go,brig,d,fortran,objc,obj-c++,m2 --enable-libphobos-checking=release --enable-libstdcxx-debug --enable-libstdcxx-time=yes --enable-link-serialization=2 --enable-multiarch --enable-multilib --enable-nls --enable-objc-gc=auto --enable-offload-targets=nvptx-none=/build/gcc-11-XeT9lY/gcc-11-11.4.0/debian/tmp-nvptx/usr,amdgcn-amdhsa=/build/gcc-11-XeT9lY/gcc-11-11.4.0/debian/tmp-gcn/usr --enable-plugin --enable-shared --enable-threads=posix --host=x86_64-linux-gnu --program-prefix=x86_64-linux-gnu- --target=x86_64-linux-gnu --with-abi=m64 --with-arch-32=i686 --with-build-config=bootstrap-lto-lean --with-default-libstdcxx-abi=new --with-gcc-major-version-only --with-multilib-list=m32,m64,mx32 --with-target-system-zlib=auto --with-tune=generic --without-cuda-driver -v Processor Details- iSCSI XCP-NG: CPU Microcode: 0xa20120a- NFS XCP-NG: CPU Microcode: 0xa20120a- PVE-SAS-SSD-fio: CPU Microcode: 0x2007006- PVE-SATA-SSD-fio: CPU Microcode: 0x2007006- XCP-NVME-R1-fio: CPU Microcode: 0x830107a- PVE-NVME-R10-4DISK-fio: CPU Microcode: 0xa00110dDisk Mount Options Details- iSCSI XCP-NG, NFS XCP-NG, XCP-NVME-R1-fio: relatime,rwDisk Details Details- iSCSI XCP-NG, NFS XCP-NG, XCP-NVME-R1-fio: Block Size: 4096Security Details- iSCSI XCP-NG: gather_data_sampling: Not affected + itlb_multihit: Not affected + l1tf: Not affected + mds: Not affected + meltdown: Not affected + mmio_stale_data: Not affected + retbleed: Not affected + spec_store_bypass: Mitigation of SSB disabled via prctl and seccomp + spectre_v1: Mitigation of usercopy/swapgs barriers and __user pointer sanitization + spectre_v2: Mitigation of Retpolines IBPB: conditional IBRS_FW STIBP: disabled RSB filling PBRSB-eIBRS: Not affected + srbds: Not affected + tsx_async_abort: Not affected - NFS XCP-NG: gather_data_sampling: Not affected + itlb_multihit: Not affected + l1tf: Not affected + mds: Not affected + meltdown: Not affected + mmio_stale_data: Not affected + retbleed: Not affected + spec_store_bypass: Mitigation of SSB disabled via prctl and seccomp + spectre_v1: Mitigation of usercopy/swapgs barriers and __user pointer sanitization + spectre_v2: Mitigation of Retpolines IBPB: conditional IBRS_FW STIBP: disabled RSB filling PBRSB-eIBRS: Not affected + srbds: Not affected + tsx_async_abort: Not affected - PVE-SAS-SSD-fio: gather_data_sampling: Unknown: Dependent on hypervisor status + itlb_multihit: Not affected + l1tf: Mitigation of PTE Inversion; VMX: flush not necessary SMT disabled + mds: Mitigation of Clear buffers; SMT Host state unknown + meltdown: Mitigation of PTI + mmio_stale_data: Vulnerable: Clear buffers attempted no microcode; SMT Host state unknown + retbleed: Mitigation of IBRS + spec_store_bypass: Mitigation of SSB disabled via prctl and seccomp + spectre_v1: Mitigation of usercopy/swapgs barriers and __user pointer sanitization + spectre_v2: Mitigation of IBRS IBPB: conditional STIBP: disabled RSB filling PBRSB-eIBRS: Not affected + srbds: Not affected + tsx_async_abort: Mitigation of Clear buffers; SMT Host state unknown - PVE-SATA-SSD-fio: gather_data_sampling: Unknown: Dependent on hypervisor status + itlb_multihit: Not affected + l1tf: Mitigation of PTE Inversion; VMX: flush not necessary SMT disabled + mds: Mitigation of Clear buffers; SMT Host state unknown + meltdown: Mitigation of PTI + mmio_stale_data: Vulnerable: Clear buffers attempted no microcode; SMT Host state unknown + retbleed: Mitigation of IBRS + spec_store_bypass: Mitigation of SSB disabled via prctl and seccomp + spectre_v1: Mitigation of usercopy/swapgs barriers and __user pointer sanitization + spectre_v2: Mitigation of IBRS IBPB: conditional STIBP: disabled RSB filling PBRSB-eIBRS: Not affected + srbds: Not affected + tsx_async_abort: Mitigation of Clear buffers; SMT Host state unknown - XCP-NVME-R1-fio: gather_data_sampling: Not affected + itlb_multihit: Not affected + l1tf: Not affected + mds: Not affected + meltdown: Not affected + mmio_stale_data: Not affected + retbleed: Mitigation of untrained return thunk; SMT disabled + spec_store_bypass: Mitigation of SSB disabled via prctl and seccomp + spectre_v1: Mitigation of usercopy/swapgs barriers and __user pointer sanitization + spectre_v2: Mitigation of Retpolines IBPB: conditional STIBP: disabled RSB filling PBRSB-eIBRS: Not affected + srbds: Not affected + tsx_async_abort: Not affected - PVE-NVME-R10-4DISK-fio: gather_data_sampling: Not affected + itlb_multihit: Not affected + l1tf: Not affected + mds: Not affected + meltdown: Not affected + mmio_stale_data: Not affected + retbleed: Not affected + spec_rstack_overflow: Mitigation of safe RET no microcode + spec_store_bypass: Mitigation of SSB disabled via prctl and seccomp + spectre_v1: Mitigation of usercopy/swapgs barriers and __user pointer sanitization + spectre_v2: Mitigation of Retpolines IBPB: conditional IBRS_FW STIBP: disabled RSB filling PBRSB-eIBRS: Not affected + srbds: Not affected + tsx_async_abort: Not affected Disk Details- PVE-SAS-SSD-fio, PVE-SATA-SSD-fio, PVE-NVME-R10-4DISK-fio: NONE / relatime,rw / Block Size: 4096

iSCSI XCP-NGNFS XCP-NGPVE-SAS-SSD-fioPVE-SATA-SSD-fioXCP-NVME-R1-fioPVE-NVME-R10-4DISK-fioLogarithmic Result OverviewPhoronix Test SuiteFlexible IO TesterFlexible IO TesterFlexible IO TesterFlexible IO TesterFlexible IO TesterFlexible IO TesterFlexible IO TesterFlexible IO TesterFlexible IO TesterFlexible IO TesterFlexible IO TesterFlexible IO TesterFlexible IO TesterFlexible IO TesterFlexible IO TesterFlexible IO TesterFlexible IO TesterFlexible IO TesterFlexible IO TesterFlexible IO TesterFlexible IO TesterFlexible IO TesterFlexible IO TesterFlexible IO TesterRand Write - Linux AIO - No - No - 1MB - 1Seq Write - Linux AIO - No - No - 1MB - 1Seq Write - Linux AIO - No - No - 64KB - 1Rand Write - Linux AIO - No - No - 64KB - 1Seq Write - Linux AIO - No - No - 64KB - 1Rand Read - Linux AIO - No - No - 64KB - 1Rand Read - Linux AIO - No - No - 64KB - 1Seq Read - Linux AIO - No - No - 64KB - 1Seq Read - Linux AIO - No - No - 64KB - 1Seq Write - Linux AIO - No - No - 1MB - 1Rand Write - Linux AIO - No - No - 1MB - 1Rand Read - Linux AIO - No - No - 4KB - 1Rand Read - Linux AIO - No - No - 4KB - 1Seq Read - Linux AIO - No - No - 4KB - 1Seq Read - Linux AIO - No - No - 4KB - 1Seq Write - Linux AIO - No - No - 4KB - 1Seq Write - Linux AIO - No - No - 4KB - 1Seq Read - Linux AIO - No - No - 1MB - 1Seq Read - Linux AIO - No - No - 1MB - 1Rand Read - Linux AIO - No - No - 1MB - 1Rand Read - Linux AIO - No - No - 1MB - 1Rand Write - Linux AIO - No - No - 64KB - 1Rand Write - Linux AIO - No - No - 4KB - 1Rand Write - Linux AIO - No - No - 4KB - 1

sa3400dfio: Rand Read - Linux AIO - No - No - 1MB - 1 - Default Test Directoryfio: Rand Read - Linux AIO - No - No - 4KB - 1 - Default Test Directoryfio: Rand Read - Linux AIO - No - No - 64KB - 1 - Default Test Directoryfio: Rand Write - Linux AIO - No - No - 1MB - 1 - Default Test Directoryfio: Rand Write - Linux AIO - No - No - 4KB - 1 - Default Test Directoryfio: Rand Write - Linux AIO - No - No - 64KB - 1 - Default Test Directoryfio: Seq Read - Linux AIO - No - No - 1MB - 1 - Default Test Directoryfio: Seq Read - Linux AIO - No - No - 4KB - 1 - Default Test Directoryfio: Seq Read - Linux AIO - No - No - 64KB - 1 - Default Test Directoryfio: Seq Write - Linux AIO - No - No - 1MB - 1 - Default Test Directoryfio: Seq Write - Linux AIO - No - No - 4KB - 1 - Default Test Directoryfio: Seq Write - Linux AIO - No - No - 64KB - 1 - Default Test Directoryfio: Rand Read - Linux AIO - No - No - 1MB - 1 - Default Test Directoryfio: Rand Read - Linux AIO - No - No - 4KB - 1 - Default Test Directoryfio: Rand Read - Linux AIO - No - No - 64KB - 1 - Default Test Directoryfio: Rand Write - Linux AIO - No - No - 1MB - 1 - Default Test Directoryfio: Rand Write - Linux AIO - No - No - 4KB - 1 - Default Test Directoryfio: Rand Write - Linux AIO - No - No - 64KB - 1 - Default Test Directoryfio: Seq Read - Linux AIO - No - No - 1MB - 1 - Default Test Directoryfio: Seq Read - Linux AIO - No - No - 4KB - 1 - Default Test Directoryfio: Seq Read - Linux AIO - No - No - 64KB - 1 - Default Test Directoryfio: Seq Write - Linux AIO - No - No - 1MB - 1 - Default Test Directoryfio: Seq Write - Linux AIO - No - No - 4KB - 1 - Default Test Directoryfio: Seq Write - Linux AIO - No - No - 64KB - 1 - Default Test DirectoryiSCSI XCP-NGNFS XCP-NGPVE-SAS-SSD-fioPVE-SATA-SSD-fioXCP-NVME-R1-fioPVE-NVME-R10-4DISK-fio1421114883225631413104425184961408172976230521426152111216541421.57459533.601410.171412.98417699.011155.991408.40691904.821440.741426.38608445.481353.3914287836021320129338995182771433128753218291233119967192871428.42313440.581332.501292.83155981.561142.291433.42515012.431364.321232.61479869.151205.4436381131476341625559284746184434713329870995257998926389643637.90452587.873963.502554.68371388.682886.524303.10533191.804437.182579.66395704.272435.241760776892621394272796148481751828782631893975861150311760.23310754.791638.35964833.15291184.11950301.231751.68331512.341644.91961583.68303445.16961989.1012309873220140105174646146491241117149191061057115737168701229.70394930.201258.781050.91298582.89937533.041241.23468597.961194.141057.44462947.31270225.16146702063951553494511035040811720520631914114643810696396814670.08825579.899709.30461886.6541400.86261184.3217205.02825277.618821.62449049.2742783.45253974.01OpenBenchmarking.org

Flexible IO Tester

OpenBenchmarking.orgIOPS, More Is BetterFlexible IO Tester 3.35Type: Random Read - Engine: Linux AIO - Buffered: No - Direct: No - Block Size: 1MB - Job Count: 1 - Disk Target: Default Test DirectoryPVE-NVME-R10-4DISK-fioPVE-SAS-SSD-fioPVE-SATA-SSD-fioNFS XCP-NGiSCSI XCP-NGXCP-NVME-R1-fio3K6K9K12K15KSE +/- 124.48, N = 4SE +/- 43.79, N = 3SE +/- 6.70, N = 3SE +/- 7.08, N = 3SE +/- 10.95, N = 3SE +/- 13.38, N = 31467036381760142814211230MIN: 14466.32 / MAX: 14979.93MIN: 3555.58 / MAX: 3705.08MIN: 1748.42 / MAX: 1771.62-ll -lz - MIN: 1414.53 / MAX: 1438.12-ll -lz - MIN: 1409.27 / MAX: 1443.321. (CC) gcc options: -rdynamic -lm -laio -lpthread -ldl -std=gnu99 -ffast-math -include -O3 -fcommon -march=native
OpenBenchmarking.orgIOPS, More Is BetterFlexible IO Tester 3.35Type: Random Read - Engine: Linux AIO - Buffered: No - Direct: No - Block Size: 1MB - Job Count: 1 - Disk Target: Default Test DirectoryPVE-NVME-R10-4DISK-fioPVE-SAS-SSD-fioPVE-SATA-SSD-fioNFS XCP-NGiSCSI XCP-NGXCP-NVME-R1-fio3K6K9K12K15KMin: 14466.32 / Avg: 14670.05 / Max: 14979.93Min: 3555.58 / Avg: 3637.77 / Max: 3705.08Min: 1748.42 / Avg: 1760.04 / Max: 1771.62Min: 1414.53 / Avg: 1428.26 / Max: 1438.12Min: 1409.27 / Avg: 1421.48 / Max: 1443.32Min: 1207.03 / Avg: 1229.61 / Max: 1253.331. (CC) gcc options: -rdynamic -lm -laio -lpthread -ldl -std=gnu99 -ffast-math -include -O3 -fcommon -march=native

OpenBenchmarking.orgIOPS, More Is BetterFlexible IO Tester 3.35Type: Random Read - Engine: Linux AIO - Buffered: No - Direct: No - Block Size: 4KB - Job Count: 1 - Disk Target: Default Test DirectoryPVE-NVME-R10-4DISK-fioiSCSI XCP-NGPVE-SAS-SSD-fioXCP-NVME-R1-fioNFS XCP-NGPVE-SATA-SSD-fio40K80K120K160K200KSE +/- 864.83, N = 4SE +/- 1073.46, N = 7SE +/- 1046.16, N = 3SE +/- 1384.83, N = 3SE +/- 315.48, N = 3SE +/- 118.26, N = 3206395114883113147987327836077689MIN: 204839.38 / MAX: 208145.78-ll -lz - MIN: 108466.13 / MAX: 116317.99MIN: 111945.09 / MAX: 115231.11MIN: 97011.92 / MAX: 101472.4-ll -lz - MIN: 77739.45 / MAX: 78768.72MIN: 77452.21 / MAX: 77812.661. (CC) gcc options: -rdynamic -lm -laio -lpthread -ldl -std=gnu99 -ffast-math -include -O3 -fcommon -march=native
OpenBenchmarking.orgIOPS, More Is BetterFlexible IO Tester 3.35Type: Random Read - Engine: Linux AIO - Buffered: No - Direct: No - Block Size: 4KB - Job Count: 1 - Disk Target: Default Test DirectoryPVE-NVME-R10-4DISK-fioiSCSI XCP-NGPVE-SAS-SSD-fioXCP-NVME-R1-fioNFS XCP-NGPVE-SATA-SSD-fio40K80K120K160K200KMin: 204839.38 / Avg: 206394.93 / Max: 208145.78Min: 108466.13 / Avg: 114883.39 / Max: 116317.99Min: 111945.09 / Avg: 113146.94 / Max: 115231.11Min: 97011.92 / Avg: 98732.5 / Max: 101472.4Min: 77739.45 / Avg: 78360.13 / Max: 78768.72Min: 77452.21 / Avg: 77688.63 / Max: 77812.661. (CC) gcc options: -rdynamic -lm -laio -lpthread -ldl -std=gnu99 -ffast-math -include -O3 -fcommon -march=native

OpenBenchmarking.orgIOPS, More Is BetterFlexible IO Tester 3.35Type: Random Read - Engine: Linux AIO - Buffered: No - Direct: No - Block Size: 64KB - Job Count: 1 - Disk Target: Default Test DirectoryPVE-NVME-R10-4DISK-fioPVE-SAS-SSD-fioPVE-SATA-SSD-fioiSCSI XCP-NGNFS XCP-NGXCP-NVME-R1-fio30K60K90K120K150KSE +/- 9937.57, N = 4SE +/- 2288.28, N = 12SE +/- 54.14, N = 3SE +/- 25.97, N = 3SE +/- 85.97, N = 3SE +/- 111.10, N = 31553496341626213225632132020140MIN: 127172.01 / MAX: 172218.08MIN: 54563.72 / MAX: 84020.6MIN: 26153.77 / MAX: 26321.49-ll -lz - MIN: 22518.24 / MAX: 22608.17-ll -lz - MIN: 21154.67 / MAX: 21443.71MIN: 19977.28 / MAX: 20352.561. (CC) gcc options: -rdynamic -lm -laio -lpthread -ldl -std=gnu99 -ffast-math -include -O3 -fcommon -march=native
OpenBenchmarking.orgIOPS, More Is BetterFlexible IO Tester 3.35Type: Random Read - Engine: Linux AIO - Buffered: No - Direct: No - Block Size: 64KB - Job Count: 1 - Disk Target: Default Test DirectoryPVE-NVME-R10-4DISK-fioPVE-SAS-SSD-fioPVE-SATA-SSD-fioiSCSI XCP-NGNFS XCP-NGXCP-NVME-R1-fio30K60K90K120K150KMin: 127172.01 / Avg: 155348.72 / Max: 172218.08Min: 54563.72 / Avg: 63415.86 / Max: 84020.6Min: 26153.77 / Avg: 26213.41 / Max: 26321.49Min: 22518.24 / Avg: 22562.63 / Max: 22608.17Min: 21154.67 / Avg: 21319.88 / Max: 21443.71Min: 19977.28 / Avg: 20140.29 / Max: 20352.561. (CC) gcc options: -rdynamic -lm -laio -lpthread -ldl -std=gnu99 -ffast-math -include -O3 -fcommon -march=native

OpenBenchmarking.orgIOPS, More Is BetterFlexible IO Tester 3.35Type: Random Write - Engine: Linux AIO - Buffered: No - Direct: No - Block Size: 1MB - Job Count: 1 - Disk Target: Default Test DirectoryPVE-SAS-SSD-fioiSCSI XCP-NGNFS XCP-NGXCP-NVME-R1-fioPVE-SATA-SSD-fioPVE-NVME-R10-4DISK-fio5001000150020002500SE +/- 49.92, N = 12SE +/- 7.01, N = 3SE +/- 11.36, N = 3SE +/- 2.50, N = 3SE +/- 7.01, N = 3SE +/- 2.28, N = 42555141312931051942451MIN: 2272.82 / MAX: 2796.93-ll -lz - MIN: 1398.97 / MAX: 1421.33-ll -lz - MIN: 1275.93 / MAX: 1314.38MIN: 1047.1 / MAX: 1055.62MIN: 933.89 / MAX: 955.84MIN: 446.96 / MAX: 455.331. (CC) gcc options: -rdynamic -lm -laio -lpthread -ldl -std=gnu99 -ffast-math -include -O3 -fcommon -march=native
OpenBenchmarking.orgIOPS, More Is BetterFlexible IO Tester 3.35Type: Random Write - Engine: Linux AIO - Buffered: No - Direct: No - Block Size: 1MB - Job Count: 1 - Disk Target: Default Test DirectoryPVE-SAS-SSD-fioiSCSI XCP-NGNFS XCP-NGXCP-NVME-R1-fioPVE-SATA-SSD-fioPVE-NVME-R10-4DISK-fio400800120016002000Min: 2272.82 / Avg: 2554.51 / Max: 2796.93Min: 1398.97 / Avg: 1412.87 / Max: 1421.33Min: 1275.93 / Avg: 1292.71 / Max: 1314.38Min: 1047.1 / Avg: 1050.89 / Max: 1055.62Min: 933.89 / Avg: 941.86 / Max: 955.84Min: 446.96 / Avg: 451.02 / Max: 455.331. (CC) gcc options: -rdynamic -lm -laio -lpthread -ldl -std=gnu99 -ffast-math -include -O3 -fcommon -march=native

OpenBenchmarking.orgIOPS, More Is BetterFlexible IO Tester 3.35Type: Random Write - Engine: Linux AIO - Buffered: No - Direct: No - Block Size: 4KB - Job Count: 1 - Disk Target: Default Test DirectoryiSCSI XCP-NGPVE-SAS-SSD-fioXCP-NVME-R1-fioPVE-SATA-SSD-fioNFS XCP-NGPVE-NVME-R10-4DISK-fio20K40K60K80K100KSE +/- 624.88, N = 3SE +/- 803.63, N = 3SE +/- 1696.14, N = 13SE +/- 913.82, N = 12SE +/- 742.00, N = 12SE +/- 57.87, N = 41044259284774646727963899510350-ll -lz - MIN: 103204.38 / MAX: 105268.29MIN: 91490.96 / MAX: 94272.26MIN: 67194.39 / MAX: 86221.09MIN: 68229.61 / MAX: 78245.37-ll -lz - MIN: 34495.97 / MAX: 43391.881. (CC) gcc options: -rdynamic -lm -laio -lpthread -ldl -std=gnu99 -ffast-math -include -O3 -fcommon -march=native
OpenBenchmarking.orgIOPS, More Is BetterFlexible IO Tester 3.35Type: Random Write - Engine: Linux AIO - Buffered: No - Direct: No - Block Size: 4KB - Job Count: 1 - Disk Target: Default Test DirectoryiSCSI XCP-NGPVE-SAS-SSD-fioXCP-NVME-R1-fioPVE-SATA-SSD-fioNFS XCP-NGPVE-NVME-R10-4DISK-fio20K40K60K80K100KMin: 103204.38 / Avg: 104424.74 / Max: 105268.29Min: 91490.96 / Avg: 92847.1 / Max: 94272.26Min: 67194.39 / Avg: 74645.65 / Max: 86221.09Min: 68229.61 / Avg: 72795.95 / Max: 78245.37Min: 34495.97 / Avg: 38995.34 / Max: 43391.88Min: 10207.45 / Avg: 10350.19 / Max: 10457.791. (CC) gcc options: -rdynamic -lm -laio -lpthread -ldl -std=gnu99 -ffast-math -include -O3 -fcommon -march=native

OpenBenchmarking.orgIOPS, More Is BetterFlexible IO Tester 3.35Type: Random Write - Engine: Linux AIO - Buffered: No - Direct: No - Block Size: 64KB - Job Count: 1 - Disk Target: Default Test DirectoryPVE-SAS-SSD-fioiSCSI XCP-NGNFS XCP-NGPVE-SATA-SSD-fioXCP-NVME-R1-fioPVE-NVME-R10-4DISK-fio10K20K30K40K50KSE +/- 202.40, N = 3SE +/- 105.36, N = 3SE +/- 131.07, N = 15SE +/- 46.66, N = 3SE +/- 27.87, N = 3SE +/- 6.27, N = 446184184961827714848146494081MIN: 45856.09 / MAX: 46553.58-ll -lz - MIN: 18361.65 / MAX: 18703.55-ll -lz - MIN: 17165.66 / MAX: 18956.57MIN: 14766.95 / MAX: 14928.59MIN: 14612.21 / MAX: 14703.571. (CC) gcc options: -rdynamic -lm -laio -lpthread -ldl -std=gnu99 -ffast-math -include -O3 -fcommon -march=native
OpenBenchmarking.orgIOPS, More Is BetterFlexible IO Tester 3.35Type: Random Write - Engine: Linux AIO - Buffered: No - Direct: No - Block Size: 64KB - Job Count: 1 - Disk Target: Default Test DirectoryPVE-SAS-SSD-fioiSCSI XCP-NGNFS XCP-NGPVE-SATA-SSD-fioXCP-NVME-R1-fioPVE-NVME-R10-4DISK-fio8K16K24K32K40KMin: 45856.09 / Avg: 46184.18 / Max: 46553.58Min: 18361.65 / Avg: 18495.74 / Max: 18703.55Min: 17165.66 / Avg: 18276.6 / Max: 18956.57Min: 14766.95 / Avg: 14848.26 / Max: 14928.59Min: 14612.21 / Avg: 14648.88 / Max: 14703.57Min: 4062.78 / Avg: 4080.96 / Max: 4091.381. (CC) gcc options: -rdynamic -lm -laio -lpthread -ldl -std=gnu99 -ffast-math -include -O3 -fcommon -march=native

OpenBenchmarking.orgIOPS, More Is BetterFlexible IO Tester 3.35Type: Sequential Read - Engine: Linux AIO - Buffered: No - Direct: No - Block Size: 1MB - Job Count: 1 - Disk Target: Default Test DirectoryPVE-NVME-R10-4DISK-fioPVE-SAS-SSD-fioPVE-SATA-SSD-fioNFS XCP-NGiSCSI XCP-NGXCP-NVME-R1-fio4K8K12K16K20KSE +/- 2352.28, N = 4SE +/- 202.37, N = 11SE +/- 5.16, N = 3SE +/- 0.97, N = 3SE +/- 2.79, N = 3SE +/- 14.37, N = 31720543471751143314081241MIN: 14668.4 / MAX: 24257.14MIN: 3664.53 / MAX: 5656.23MIN: 1742.87 / MAX: 1760.721. (CC) gcc options: -rdynamic -lm -laio -lpthread -ldl -std=gnu99 -ffast-math -include -O3 -fcommon -march=native
OpenBenchmarking.orgIOPS, More Is BetterFlexible IO Tester 3.35Type: Sequential Read - Engine: Linux AIO - Buffered: No - Direct: No - Block Size: 1MB - Job Count: 1 - Disk Target: Default Test DirectoryPVE-NVME-R10-4DISK-fioPVE-SAS-SSD-fioPVE-SATA-SSD-fioNFS XCP-NGiSCSI XCP-NGXCP-NVME-R1-fio3K6K9K12K15KMin: 14668.4 / Avg: 17204.98 / Max: 24257.14Min: 3664.53 / Avg: 4346.96 / Max: 5656.23Min: 1742.87 / Avg: 1751.47 / Max: 1760.72Min: 1431.58 / Avg: 1433.24 / Max: 1434.95Min: 1404.38 / Avg: 1408.32 / Max: 1413.71Min: 1218.67 / Avg: 1241.13 / Max: 1267.881. (CC) gcc options: -rdynamic -lm -laio -lpthread -ldl -std=gnu99 -ffast-math -include -O3 -fcommon -march=native

OpenBenchmarking.orgIOPS, More Is BetterFlexible IO Tester 3.35Type: Sequential Read - Engine: Linux AIO - Buffered: No - Direct: No - Block Size: 4KB - Job Count: 1 - Disk Target: Default Test DirectoryPVE-NVME-R10-4DISK-fioiSCSI XCP-NGPVE-SAS-SSD-fioNFS XCP-NGXCP-NVME-R1-fioPVE-SATA-SSD-fio40K80K120K160K200KSE +/- 622.34, N = 4SE +/- 1799.07, N = 3SE +/- 1409.20, N = 3SE +/- 1758.86, N = 15SE +/- 1777.46, N = 15SE +/- 271.46, N = 320631917297613329812875311714982878MIN: 204697.03 / MAX: 207717.76-ll -lz - MIN: 169871.42 / MAX: 176103.45MIN: 130529.06 / MAX: 135137.88-ll -lz - MIN: 111689.26 / MAX: 138445.07MIN: 108826.53 / MAX: 130870.88MIN: 82403.55 / MAX: 83343.81. (CC) gcc options: -rdynamic -lm -laio -lpthread -ldl -std=gnu99 -ffast-math -include -O3 -fcommon -march=native
OpenBenchmarking.orgIOPS, More Is BetterFlexible IO Tester 3.35Type: Sequential Read - Engine: Linux AIO - Buffered: No - Direct: No - Block Size: 4KB - Job Count: 1 - Disk Target: Default Test DirectoryPVE-NVME-R10-4DISK-fioiSCSI XCP-NGPVE-SAS-SSD-fioNFS XCP-NGXCP-NVME-R1-fioPVE-SATA-SSD-fio40K80K120K160K200KMin: 204697.03 / Avg: 206319.34 / Max: 207717.76Min: 169871.42 / Avg: 172976.2 / Max: 176103.45Min: 130529.06 / Avg: 133297.95 / Max: 135137.88Min: 111689.26 / Avg: 128753.11 / Max: 138445.07Min: 108826.53 / Avg: 117149.43 / Max: 130870.88Min: 82403.55 / Avg: 82878.01 / Max: 83343.81. (CC) gcc options: -rdynamic -lm -laio -lpthread -ldl -std=gnu99 -ffast-math -include -O3 -fcommon -march=native

OpenBenchmarking.orgIOPS, More Is BetterFlexible IO Tester 3.35Type: Sequential Read - Engine: Linux AIO - Buffered: No - Direct: No - Block Size: 64KB - Job Count: 1 - Disk Target: Default Test DirectoryPVE-NVME-R10-4DISK-fioPVE-SAS-SSD-fioPVE-SATA-SSD-fioiSCSI XCP-NGNFS XCP-NGXCP-NVME-R1-fio30K60K90K120K150KSE +/- 1882.32, N = 4SE +/- 703.79, N = 15SE +/- 81.39, N = 3SE +/- 162.60, N = 3SE +/- 20.33, N = 3SE +/- 202.92, N = 51411467099526318230522182919106MIN: 137012.33 / MAX: 145928.53MIN: 66518.12 / MAX: 75378.1MIN: 26215.38 / MAX: 26479.1-ll -lz - MIN: 22831.78 / MAX: 23369.16-ll -lz - MIN: 21790.82 / MAX: 21860.21MIN: 18613.54 / MAX: 19827.621. (CC) gcc options: -rdynamic -lm -laio -lpthread -ldl -std=gnu99 -ffast-math -include -O3 -fcommon -march=native
OpenBenchmarking.orgIOPS, More Is BetterFlexible IO Tester 3.35Type: Sequential Read - Engine: Linux AIO - Buffered: No - Direct: No - Block Size: 64KB - Job Count: 1 - Disk Target: Default Test DirectoryPVE-NVME-R10-4DISK-fioPVE-SAS-SSD-fioPVE-SATA-SSD-fioiSCSI XCP-NGNFS XCP-NGXCP-NVME-R1-fio20K40K60K80K100KMin: 137012.33 / Avg: 141145.92 / Max: 145928.53Min: 66518.12 / Avg: 70994.73 / Max: 75378.1Min: 26215.38 / Avg: 26318.45 / Max: 26479.1Min: 22831.78 / Avg: 23051.76 / Max: 23369.16Min: 21790.82 / Avg: 21828.99 / Max: 21860.21Min: 18613.54 / Avg: 19106.22 / Max: 19827.621. (CC) gcc options: -rdynamic -lm -laio -lpthread -ldl -std=gnu99 -ffast-math -include -O3 -fcommon -march=native

OpenBenchmarking.orgIOPS, More Is BetterFlexible IO Tester 3.35Type: Sequential Write - Engine: Linux AIO - Buffered: No - Direct: No - Block Size: 1MB - Job Count: 1 - Disk Target: Default Test DirectoryPVE-SAS-SSD-fioiSCSI XCP-NGNFS XCP-NGXCP-NVME-R1-fioPVE-SATA-SSD-fioPVE-NVME-R10-4DISK-fio6001200180024003000SE +/- 55.39, N = 15SE +/- 4.29, N = 3SE +/- 3.52, N = 3SE +/- 8.67, N = 3SE +/- 2.34, N = 3SE +/- 0.90, N = 42579142612331057939438MIN: 2145.02 / MAX: 2879.4-ll -lz - MIN: 1418.87 / MAX: 1433.73-ll -lz - MIN: 1225.95 / MAX: 1237.98MIN: 1044.53 / MAX: 1073.92MIN: 935.67 / MAX: 943.3MIN: 436.22 / MAX: 440.541. (CC) gcc options: -rdynamic -lm -laio -lpthread -ldl -std=gnu99 -ffast-math -include -O3 -fcommon -march=native
OpenBenchmarking.orgIOPS, More Is BetterFlexible IO Tester 3.35Type: Sequential Write - Engine: Linux AIO - Buffered: No - Direct: No - Block Size: 1MB - Job Count: 1 - Disk Target: Default Test DirectoryPVE-SAS-SSD-fioiSCSI XCP-NGNFS XCP-NGXCP-NVME-R1-fioPVE-SATA-SSD-fioPVE-NVME-R10-4DISK-fio400800120016002000Min: 2145.02 / Avg: 2579.46 / Max: 2879.4Min: 1418.87 / Avg: 1426.25 / Max: 1433.73Min: 1225.95 / Avg: 1232.54 / Max: 1237.98Min: 1044.53 / Avg: 1057.42 / Max: 1073.92Min: 935.67 / Avg: 938.7 / Max: 943.3Min: 436.22 / Avg: 438.48 / Max: 440.541. (CC) gcc options: -rdynamic -lm -laio -lpthread -ldl -std=gnu99 -ffast-math -include -O3 -fcommon -march=native

OpenBenchmarking.orgIOPS, More Is BetterFlexible IO Tester 3.35Type: Sequential Write - Engine: Linux AIO - Buffered: No - Direct: No - Block Size: 4KB - Job Count: 1 - Disk Target: Default Test DirectoryiSCSI XCP-NGNFS XCP-NGXCP-NVME-R1-fioPVE-SAS-SSD-fioPVE-SATA-SSD-fioPVE-NVME-R10-4DISK-fio30K60K90K120K150KSE +/- 5903.26, N = 12SE +/- 3728.16, N = 15SE +/- 6276.07, N = 12SE +/- 709.81, N = 3SE +/- 804.62, N = 15SE +/- 26.83, N = 4152111119967115737989267586110696-ll -lz - MIN: 103077.64 / MAX: 172524.33-ll -lz - MIN: 85674.41 / MAX: 134860MIN: 89048.78 / MAX: 150974.03MIN: 97970.63 / MAX: 100313.05MIN: 70844.74 / MAX: 80423.511. (CC) gcc options: -rdynamic -lm -laio -lpthread -ldl -std=gnu99 -ffast-math -include -O3 -fcommon -march=native
OpenBenchmarking.orgIOPS, More Is BetterFlexible IO Tester 3.35Type: Sequential Write - Engine: Linux AIO - Buffered: No - Direct: No - Block Size: 4KB - Job Count: 1 - Disk Target: Default Test DirectoryiSCSI XCP-NGNFS XCP-NGXCP-NVME-R1-fioPVE-SAS-SSD-fioPVE-SATA-SSD-fioPVE-NVME-R10-4DISK-fio30K60K90K120K150KMin: 103077.64 / Avg: 152111.36 / Max: 172524.33Min: 85674.41 / Avg: 119967.28 / Max: 134860Min: 89048.78 / Avg: 115736.76 / Max: 150974.03Min: 97970.63 / Avg: 98925.99 / Max: 100313.05Min: 70844.74 / Avg: 75861.21 / Max: 80423.51Min: 10640.62 / Avg: 10695.82 / Max: 10753.021. (CC) gcc options: -rdynamic -lm -laio -lpthread -ldl -std=gnu99 -ffast-math -include -O3 -fcommon -march=native

OpenBenchmarking.orgIOPS, More Is BetterFlexible IO Tester 3.35Type: Sequential Write - Engine: Linux AIO - Buffered: No - Direct: No - Block Size: 64KB - Job Count: 1 - Disk Target: Default Test DirectoryPVE-SAS-SSD-fioiSCSI XCP-NGNFS XCP-NGXCP-NVME-R1-fioPVE-SATA-SSD-fioPVE-NVME-R10-4DISK-fio8K16K24K32K40KSE +/- 720.63, N = 15SE +/- 214.28, N = 6SE +/- 154.31, N = 3SE +/- 254.53, N = 15SE +/- 15.36, N = 3SE +/- 3.80, N = 438964216541928716870150313968MIN: 34966.58 / MAX: 44257.78-ll -lz - MIN: 20588.04 / MAX: 21930.05-ll -lz - MIN: 18980.62 / MAX: 19472.58MIN: 15150.57 / MAX: 18484.55MIN: 15005.17 / MAX: 15058.29MIN: 3958.54 / MAX: 3976.921. (CC) gcc options: -rdynamic -lm -laio -lpthread -ldl -std=gnu99 -ffast-math -include -O3 -fcommon -march=native
OpenBenchmarking.orgIOPS, More Is BetterFlexible IO Tester 3.35Type: Sequential Write - Engine: Linux AIO - Buffered: No - Direct: No - Block Size: 64KB - Job Count: 1 - Disk Target: Default Test DirectoryPVE-SAS-SSD-fioiSCSI XCP-NGNFS XCP-NGXCP-NVME-R1-fioPVE-SATA-SSD-fioPVE-NVME-R10-4DISK-fio7K14K21K28K35KMin: 34966.58 / Avg: 38963.78 / Max: 44257.78Min: 20588.04 / Avg: 21654.19 / Max: 21930.05Min: 18980.62 / Avg: 19286.96 / Max: 19472.58Min: 15150.57 / Avg: 16869.97 / Max: 18484.55Min: 15005.17 / Avg: 15030.92 / Max: 15058.29Min: 3958.54 / Avg: 3968.27 / Max: 3976.921. (CC) gcc options: -rdynamic -lm -laio -lpthread -ldl -std=gnu99 -ffast-math -include -O3 -fcommon -march=native

OpenBenchmarking.orgMiB/s, More Is BetterFlexible IO Tester 3.35Type: Random Read - Engine: Linux AIO - Buffered: No - Direct: No - Block Size: 1MB - Job Count: 1 - Disk Target: Default Test DirectoryPVE-NVME-R10-4DISK-fioPVE-SAS-SSD-fioPVE-SATA-SSD-fioNFS XCP-NGiSCSI XCP-NGXCP-NVME-R1-fio3K6K9K12K15KSE +/- 124.47, N = 4SE +/- 43.81, N = 3SE +/- 6.71, N = 3SE +/- 7.07, N = 3SE +/- 10.95, N = 3SE +/- 13.38, N = 314670.083637.901760.231428.421421.571229.70MIN: 14466.35 / MAX: 14979.95MIN: 3555.69 / MAX: 3705.251. (CC) gcc options: -rdynamic -lm -laio -lpthread -ldl -std=gnu99 -ffast-math -include -O3 -fcommon -march=native
OpenBenchmarking.orgMiB/s, More Is BetterFlexible IO Tester 3.35Type: Random Read - Engine: Linux AIO - Buffered: No - Direct: No - Block Size: 1MB - Job Count: 1 - Disk Target: Default Test DirectoryPVE-NVME-R10-4DISK-fioPVE-SAS-SSD-fioPVE-SATA-SSD-fioNFS XCP-NGiSCSI XCP-NGXCP-NVME-R1-fio3K6K9K12K15KMin: 14466.35 / Avg: 14670.08 / Max: 14979.95Min: 3555.69 / Avg: 3637.9 / Max: 3705.25Min: 1748.58 / Avg: 1760.23 / Max: 1771.83Min: 1414.7 / Avg: 1428.42 / Max: 1438.26Min: 1409.37 / Avg: 1421.57 / Max: 1443.41Min: 1207.13 / Avg: 1229.7 / Max: 1253.441. (CC) gcc options: -rdynamic -lm -laio -lpthread -ldl -std=gnu99 -ffast-math -include -O3 -fcommon -march=native

OpenBenchmarking.orgMiB/s, More Is BetterFlexible IO Tester 3.35Type: Random Read - Engine: Linux AIO - Buffered: No - Direct: No - Block Size: 4KB - Job Count: 1 - Disk Target: Default Test DirectoryPVE-NVME-R10-4DISK-fioiSCSI XCP-NGPVE-SAS-SSD-fioXCP-NVME-R1-fioNFS XCP-NGPVE-SATA-SSD-fio200K400K600K800K1000KSE +/- 3459.36, N = 4SE +/- 4293.82, N = 7SE +/- 4184.69, N = 3SE +/- 5539.28, N = 3SE +/- 1261.94, N = 3SE +/- 472.95, N = 3825579.89459533.60452587.87394930.20313440.58310754.79MIN: 819357.59 / MAX: 832583.42-ll -lz - MIN: 433864.7 / MAX: 465271.88MIN: 447780.52 / MAX: 460924.67MIN: 388047.96 / MAX: 405889.76-ll -lz - MIN: 310957.83 / MAX: 315075MIN: 309809.26 / MAX: 311250.691. (CC) gcc options: -rdynamic -lm -laio -lpthread -ldl -std=gnu99 -ffast-math -include -O3 -fcommon -march=native
OpenBenchmarking.orgMiB/s, More Is BetterFlexible IO Tester 3.35Type: Random Read - Engine: Linux AIO - Buffered: No - Direct: No - Block Size: 4KB - Job Count: 1 - Disk Target: Default Test DirectoryPVE-NVME-R10-4DISK-fioiSCSI XCP-NGPVE-SAS-SSD-fioXCP-NVME-R1-fioNFS XCP-NGPVE-SATA-SSD-fio140K280K420K560K700KMin: 819357.59 / Avg: 825579.89 / Max: 832583.42Min: 433864.7 / Avg: 459533.6 / Max: 465271.88Min: 447780.52 / Avg: 452587.87 / Max: 460924.67Min: 388047.96 / Avg: 394930.2 / Max: 405889.76Min: 310957.83 / Avg: 313440.58 / Max: 315075Min: 309809.26 / Avg: 310754.79 / Max: 311250.691. (CC) gcc options: -rdynamic -lm -laio -lpthread -ldl -std=gnu99 -ffast-math -include -O3 -fcommon -march=native

OpenBenchmarking.orgMiB/s, More Is BetterFlexible IO Tester 3.35Type: Random Read - Engine: Linux AIO - Buffered: No - Direct: No - Block Size: 64KB - Job Count: 1 - Disk Target: Default Test DirectoryPVE-NVME-R10-4DISK-fioPVE-SAS-SSD-fioPVE-SATA-SSD-fioiSCSI XCP-NGNFS XCP-NGXCP-NVME-R1-fio2K4K6K8K10KSE +/- 621.10, N = 4SE +/- 143.02, N = 12SE +/- 3.39, N = 3SE +/- 1.62, N = 3SE +/- 5.37, N = 3SE +/- 6.94, N = 39709.303963.501638.351410.171332.501258.78MIN: 7948.25 / MAX: 10763.63MIN: 3410.24 / MAX: 5251.29MIN: 1634.62 / MAX: 1645.111. (CC) gcc options: -rdynamic -lm -laio -lpthread -ldl -std=gnu99 -ffast-math -include -O3 -fcommon -march=native
OpenBenchmarking.orgMiB/s, More Is BetterFlexible IO Tester 3.35Type: Random Read - Engine: Linux AIO - Buffered: No - Direct: No - Block Size: 64KB - Job Count: 1 - Disk Target: Default Test DirectoryPVE-NVME-R10-4DISK-fioPVE-SAS-SSD-fioPVE-SATA-SSD-fioiSCSI XCP-NGNFS XCP-NGXCP-NVME-R1-fio2K4K6K8K10KMin: 7948.25 / Avg: 9709.3 / Max: 10763.63Min: 3410.24 / Avg: 3963.5 / Max: 5251.29Min: 1634.62 / Avg: 1638.35 / Max: 1645.11Min: 1407.39 / Avg: 1410.17 / Max: 1413.01Min: 1322.17 / Avg: 1332.5 / Max: 1340.24Min: 1248.59 / Avg: 1258.78 / Max: 1272.041. (CC) gcc options: -rdynamic -lm -laio -lpthread -ldl -std=gnu99 -ffast-math -include -O3 -fcommon -march=native

OpenBenchmarking.orgMiB/s, More Is BetterFlexible IO Tester 3.35Type: Random Write - Engine: Linux AIO - Buffered: No - Direct: No - Block Size: 1MB - Job Count: 1 - Disk Target: Default Test DirectoryPVE-SATA-SSD-fioPVE-NVME-R10-4DISK-fioPVE-SAS-SSD-fioiSCSI XCP-NGNFS XCP-NGXCP-NVME-R1-fio200K400K600K800K1000KSE +/- 7165.39, N = 3SE +/- 2341.54, N = 4SE +/- 49.92, N = 12SE +/- 7.01, N = 3SE +/- 11.37, N = 3SE +/- 2.50, N = 3964833.15461886.652554.681412.981292.831050.91MIN: 956687.49 / MAX: 979116.99MIN: 457742.04 / MAX: 466318.331. (CC) gcc options: -rdynamic -lm -laio -lpthread -ldl -std=gnu99 -ffast-math -include -O3 -fcommon -march=native
OpenBenchmarking.orgMiB/s, More Is BetterFlexible IO Tester 3.35Type: Random Write - Engine: Linux AIO - Buffered: No - Direct: No - Block Size: 1MB - Job Count: 1 - Disk Target: Default Test DirectoryPVE-SATA-SSD-fioPVE-NVME-R10-4DISK-fioPVE-SAS-SSD-fioiSCSI XCP-NGNFS XCP-NGXCP-NVME-R1-fio200K400K600K800K1000KMin: 956687.49 / Avg: 964833.15 / Max: 979116.99Min: 457742.04 / Avg: 461886.65 / Max: 466318.33Min: 2273 / Avg: 2554.68 / Max: 2797.11Min: 1399.07 / Avg: 1412.98 / Max: 1421.43Min: 1276.04 / Avg: 1292.83 / Max: 1314.5Min: 1047.12 / Avg: 1050.91 / Max: 1055.641. (CC) gcc options: -rdynamic -lm -laio -lpthread -ldl -std=gnu99 -ffast-math -include -O3 -fcommon -march=native

OpenBenchmarking.orgMiB/s, More Is BetterFlexible IO Tester 3.35Type: Random Write - Engine: Linux AIO - Buffered: No - Direct: No - Block Size: 4KB - Job Count: 1 - Disk Target: Default Test DirectoryiSCSI XCP-NGPVE-SAS-SSD-fioXCP-NVME-R1-fioPVE-SATA-SSD-fioNFS XCP-NGPVE-NVME-R10-4DISK-fio90K180K270K360K450KSE +/- 2499.52, N = 3SE +/- 3214.61, N = 3SE +/- 6784.56, N = 13SE +/- 3655.27, N = 12SE +/- 2968.01, N = 12SE +/- 231.48, N = 4417699.01371388.68298582.89291184.11155981.5641400.86-ll -lz - MIN: 412817.57 / MAX: 421073.22MIN: 365963.97 / MAX: 377089.44MIN: 268777.81 / MAX: 344884.63MIN: 272918.71 / MAX: 312981.77-ll -lz - MIN: 137984.09 / MAX: 173567.741. (CC) gcc options: -rdynamic -lm -laio -lpthread -ldl -std=gnu99 -ffast-math -include -O3 -fcommon -march=native
OpenBenchmarking.orgMiB/s, More Is BetterFlexible IO Tester 3.35Type: Random Write - Engine: Linux AIO - Buffered: No - Direct: No - Block Size: 4KB - Job Count: 1 - Disk Target: Default Test DirectoryiSCSI XCP-NGPVE-SAS-SSD-fioXCP-NVME-R1-fioPVE-SATA-SSD-fioNFS XCP-NGPVE-NVME-R10-4DISK-fio70K140K210K280K350KMin: 412817.57 / Avg: 417699.01 / Max: 421073.22Min: 365963.97 / Avg: 371388.68 / Max: 377089.44Min: 268777.81 / Avg: 298582.89 / Max: 344884.63Min: 272918.71 / Avg: 291184.11 / Max: 312981.77Min: 137984.09 / Avg: 155981.56 / Max: 173567.74Min: 40829.94 / Avg: 41400.86 / Max: 41831.261. (CC) gcc options: -rdynamic -lm -laio -lpthread -ldl -std=gnu99 -ffast-math -include -O3 -fcommon -march=native

OpenBenchmarking.orgMiB/s, More Is BetterFlexible IO Tester 3.35Type: Random Write - Engine: Linux AIO - Buffered: No - Direct: No - Block Size: 64KB - Job Count: 1 - Disk Target: Default Test DirectoryPVE-SATA-SSD-fioXCP-NVME-R1-fioPVE-NVME-R10-4DISK-fioPVE-SAS-SSD-fioiSCSI XCP-NGNFS XCP-NG200K400K600K800K1000KSE +/- 2986.07, N = 3SE +/- 1783.38, N = 3SE +/- 401.63, N = 4SE +/- 12.65, N = 3SE +/- 6.59, N = 3SE +/- 8.19, N = 15950301.23937533.04261184.322886.521155.991142.29MIN: 945098.68 / MAX: 955442.17MIN: 935186.02 / MAX: 941032.48MIN: 260020.42 / MAX: 261851.331. (CC) gcc options: -rdynamic -lm -laio -lpthread -ldl -std=gnu99 -ffast-math -include -O3 -fcommon -march=native
OpenBenchmarking.orgMiB/s, More Is BetterFlexible IO Tester 3.35Type: Random Write - Engine: Linux AIO - Buffered: No - Direct: No - Block Size: 64KB - Job Count: 1 - Disk Target: Default Test DirectoryPVE-SATA-SSD-fioXCP-NVME-R1-fioPVE-NVME-R10-4DISK-fioPVE-SAS-SSD-fioiSCSI XCP-NGNFS XCP-NG160K320K480K640K800KMin: 945098.68 / Avg: 950301.23 / Max: 955442.17Min: 935186.02 / Avg: 937533.04 / Max: 941032.48Min: 260020.42 / Avg: 261184.32 / Max: 261851.33Min: 2866.01 / Avg: 2886.52 / Max: 2909.6Min: 1147.61 / Avg: 1155.99 / Max: 1168.98Min: 1072.86 / Avg: 1142.29 / Max: 1184.791. (CC) gcc options: -rdynamic -lm -laio -lpthread -ldl -std=gnu99 -ffast-math -include -O3 -fcommon -march=native

OpenBenchmarking.orgMiB/s, More Is BetterFlexible IO Tester 3.35Type: Sequential Read - Engine: Linux AIO - Buffered: No - Direct: No - Block Size: 1MB - Job Count: 1 - Disk Target: Default Test DirectoryPVE-NVME-R10-4DISK-fioPVE-SAS-SSD-fioPVE-SATA-SSD-fioNFS XCP-NGiSCSI XCP-NGXCP-NVME-R1-fio4K8K12K16K20KSE +/- 2352.28, N = 4SE +/- 189.90, N = 12SE +/- 5.15, N = 3SE +/- 0.97, N = 3SE +/- 2.79, N = 3SE +/- 14.38, N = 317205.024303.101751.681433.421408.401241.23MIN: 14668.43 / MAX: 24257.2MIN: 3664.73 / MAX: 5656.331. (CC) gcc options: -rdynamic -lm -laio -lpthread -ldl -std=gnu99 -ffast-math -include -O3 -fcommon -march=native
OpenBenchmarking.orgMiB/s, More Is BetterFlexible IO Tester 3.35Type: Sequential Read - Engine: Linux AIO - Buffered: No - Direct: No - Block Size: 1MB - Job Count: 1 - Disk Target: Default Test DirectoryPVE-NVME-R10-4DISK-fioPVE-SAS-SSD-fioPVE-SATA-SSD-fioNFS XCP-NGiSCSI XCP-NGXCP-NVME-R1-fio3K6K9K12K15KMin: 14668.43 / Avg: 17205.02 / Max: 24257.2Min: 3664.73 / Avg: 4303.1 / Max: 5656.33Min: 1743.09 / Avg: 1751.68 / Max: 1760.89Min: 1431.76 / Avg: 1433.42 / Max: 1435.12Min: 1404.46 / Avg: 1408.4 / Max: 1413.79Min: 1218.75 / Avg: 1241.23 / Max: 12681. (CC) gcc options: -rdynamic -lm -laio -lpthread -ldl -std=gnu99 -ffast-math -include -O3 -fcommon -march=native

OpenBenchmarking.orgMiB/s, More Is BetterFlexible IO Tester 3.35Type: Sequential Read - Engine: Linux AIO - Buffered: No - Direct: No - Block Size: 4KB - Job Count: 1 - Disk Target: Default Test DirectoryPVE-NVME-R10-4DISK-fioiSCSI XCP-NGPVE-SAS-SSD-fioNFS XCP-NGXCP-NVME-R1-fioPVE-SATA-SSD-fio200K400K600K800K1000KSE +/- 2489.36, N = 4SE +/- 7196.24, N = 3SE +/- 5636.73, N = 3SE +/- 7035.45, N = 15SE +/- 7109.83, N = 15SE +/- 1085.66, N = 3825277.61691904.82533191.80515012.43468597.96331512.34MIN: 818788.42 / MAX: 830871.32-ll -lz - MIN: 679485.78 / MAX: 704413.8MIN: 522116.32 / MAX: 540551.18-ll -lz - MIN: 446757.07 / MAX: 553780.17MIN: 435306.38 / MAX: 523483.82MIN: 329614.76 / MAX: 333375.11. (CC) gcc options: -rdynamic -lm -laio -lpthread -ldl -std=gnu99 -ffast-math -include -O3 -fcommon -march=native
OpenBenchmarking.orgMiB/s, More Is BetterFlexible IO Tester 3.35Type: Sequential Read - Engine: Linux AIO - Buffered: No - Direct: No - Block Size: 4KB - Job Count: 1 - Disk Target: Default Test DirectoryPVE-NVME-R10-4DISK-fioiSCSI XCP-NGPVE-SAS-SSD-fioNFS XCP-NGXCP-NVME-R1-fioPVE-SATA-SSD-fio140K280K420K560K700KMin: 818788.42 / Avg: 825277.61 / Max: 830871.32Min: 679485.78 / Avg: 691904.82 / Max: 704413.8Min: 522116.32 / Avg: 533191.8 / Max: 540551.18Min: 446757.07 / Avg: 515012.43 / Max: 553780.17Min: 435306.38 / Avg: 468597.96 / Max: 523483.82Min: 329614.76 / Avg: 331512.34 / Max: 333375.11. (CC) gcc options: -rdynamic -lm -laio -lpthread -ldl -std=gnu99 -ffast-math -include -O3 -fcommon -march=native

OpenBenchmarking.orgMiB/s, More Is BetterFlexible IO Tester 3.35Type: Sequential Read - Engine: Linux AIO - Buffered: No - Direct: No - Block Size: 64KB - Job Count: 1 - Disk Target: Default Test DirectoryPVE-NVME-R10-4DISK-fioPVE-SAS-SSD-fioPVE-SATA-SSD-fioiSCSI XCP-NGNFS XCP-NGXCP-NVME-R1-fio2K4K6K8K10KSE +/- 117.65, N = 4SE +/- 43.99, N = 15SE +/- 5.09, N = 3SE +/- 10.16, N = 3SE +/- 1.27, N = 3SE +/- 12.68, N = 58821.624437.181644.911440.741364.321194.14MIN: 8563.27 / MAX: 9120.54MIN: 4157.39 / MAX: 4711.14MIN: 1638.47 / MAX: 1654.951. (CC) gcc options: -rdynamic -lm -laio -lpthread -ldl -std=gnu99 -ffast-math -include -O3 -fcommon -march=native
OpenBenchmarking.orgMiB/s, More Is BetterFlexible IO Tester 3.35Type: Sequential Read - Engine: Linux AIO - Buffered: No - Direct: No - Block Size: 64KB - Job Count: 1 - Disk Target: Default Test DirectoryPVE-NVME-R10-4DISK-fioPVE-SAS-SSD-fioPVE-SATA-SSD-fioiSCSI XCP-NGNFS XCP-NGXCP-NVME-R1-fio15003000450060007500Min: 8563.27 / Avg: 8821.62 / Max: 9120.54Min: 4157.39 / Avg: 4437.18 / Max: 4711.14Min: 1638.47 / Avg: 1644.91 / Max: 1654.95Min: 1426.99 / Avg: 1440.74 / Max: 1460.58Min: 1361.93 / Avg: 1364.32 / Max: 1366.27Min: 1163.35 / Avg: 1194.14 / Max: 1239.231. (CC) gcc options: -rdynamic -lm -laio -lpthread -ldl -std=gnu99 -ffast-math -include -O3 -fcommon -march=native

OpenBenchmarking.orgMiB/s, More Is BetterFlexible IO Tester 3.35Type: Sequential Write - Engine: Linux AIO - Buffered: No - Direct: No - Block Size: 1MB - Job Count: 1 - Disk Target: Default Test DirectoryPVE-SATA-SSD-fioPVE-NVME-R10-4DISK-fioPVE-SAS-SSD-fioiSCSI XCP-NGNFS XCP-NGXCP-NVME-R1-fio200K400K600K800K1000KSE +/- 2399.85, N = 3SE +/- 928.79, N = 4SE +/- 55.39, N = 15SE +/- 4.29, N = 3SE +/- 3.52, N = 3SE +/- 8.67, N = 3961583.68449049.272579.661426.381232.611057.44MIN: 958465.17 / MAX: 966302.68MIN: 446726.53 / MAX: 451164.931. (CC) gcc options: -rdynamic -lm -laio -lpthread -ldl -std=gnu99 -ffast-math -include -O3 -fcommon -march=native
OpenBenchmarking.orgMiB/s, More Is BetterFlexible IO Tester 3.35Type: Sequential Write - Engine: Linux AIO - Buffered: No - Direct: No - Block Size: 1MB - Job Count: 1 - Disk Target: Default Test DirectoryPVE-SATA-SSD-fioPVE-NVME-R10-4DISK-fioPVE-SAS-SSD-fioiSCSI XCP-NGNFS XCP-NGXCP-NVME-R1-fio200K400K600K800K1000KMin: 958465.17 / Avg: 961583.68 / Max: 966302.68Min: 446726.53 / Avg: 449049.27 / Max: 451164.93Min: 2145.2 / Avg: 2579.66 / Max: 2879.63Min: 1419.01 / Avg: 1426.38 / Max: 1433.87Min: 1226.03 / Avg: 1232.61 / Max: 1238.07Min: 1044.55 / Avg: 1057.44 / Max: 1073.941. (CC) gcc options: -rdynamic -lm -laio -lpthread -ldl -std=gnu99 -ffast-math -include -O3 -fcommon -march=native

OpenBenchmarking.orgMiB/s, More Is BetterFlexible IO Tester 3.35Type: Sequential Write - Engine: Linux AIO - Buffered: No - Direct: No - Block Size: 4KB - Job Count: 1 - Disk Target: Default Test DirectoryiSCSI XCP-NGNFS XCP-NGXCP-NVME-R1-fioPVE-SAS-SSD-fioPVE-SATA-SSD-fioPVE-NVME-R10-4DISK-fio130K260K390K520K650KSE +/- 23613.03, N = 12SE +/- 14912.65, N = 15SE +/- 25104.31, N = 12SE +/- 2839.24, N = 3SE +/- 3218.47, N = 15SE +/- 107.29, N = 4608445.48479869.15462947.31395704.27303445.1642783.45-ll -lz - MIN: 412310.6 / MAX: 690097.59-ll -lz - MIN: 342697.58 / MAX: 539439.96MIN: 356195.36 / MAX: 603896.43MIN: 391883.02 / MAX: 401252.52MIN: 283379 / MAX: 321694.321. (CC) gcc options: -rdynamic -lm -laio -lpthread -ldl -std=gnu99 -ffast-math -include -O3 -fcommon -march=native
OpenBenchmarking.orgMiB/s, More Is BetterFlexible IO Tester 3.35Type: Sequential Write - Engine: Linux AIO - Buffered: No - Direct: No - Block Size: 4KB - Job Count: 1 - Disk Target: Default Test DirectoryiSCSI XCP-NGNFS XCP-NGXCP-NVME-R1-fioPVE-SAS-SSD-fioPVE-SATA-SSD-fioPVE-NVME-R10-4DISK-fio110K220K330K440K550KMin: 412310.6 / Avg: 608445.48 / Max: 690097.59Min: 342697.58 / Avg: 479869.15 / Max: 539439.96Min: 356195.36 / Avg: 462947.31 / Max: 603896.43Min: 391883.02 / Avg: 395704.27 / Max: 401252.52Min: 283379 / Avg: 303445.16 / Max: 321694.32Min: 42562.76 / Avg: 42783.45 / Max: 43012.161. (CC) gcc options: -rdynamic -lm -laio -lpthread -ldl -std=gnu99 -ffast-math -include -O3 -fcommon -march=native

OpenBenchmarking.orgMiB/s, More Is BetterFlexible IO Tester 3.35Type: Sequential Write - Engine: Linux AIO - Buffered: No - Direct: No - Block Size: 64KB - Job Count: 1 - Disk Target: Default Test DirectoryPVE-SATA-SSD-fioXCP-NVME-R1-fioPVE-NVME-R10-4DISK-fioPVE-SAS-SSD-fioiSCSI XCP-NGNFS XCP-NG200K400K600K800K1000KSE +/- 984.67, N = 3SE +/- 119375.03, N = 15SE +/- 243.95, N = 4SE +/- 45.04, N = 15SE +/- 13.39, N = 6SE +/- 9.65, N = 3961989.10270225.16253974.012435.241353.391205.44MIN: 960336.93 / MAX: 963743.34MIN: 1025.26 / MAX: 1047960.13MIN: 253349.81 / MAX: 254529.671. (CC) gcc options: -rdynamic -lm -laio -lpthread -ldl -std=gnu99 -ffast-math -include -O3 -fcommon -march=native
OpenBenchmarking.orgMiB/s, More Is BetterFlexible IO Tester 3.35Type: Sequential Write - Engine: Linux AIO - Buffered: No - Direct: No - Block Size: 64KB - Job Count: 1 - Disk Target: Default Test DirectoryPVE-SATA-SSD-fioXCP-NVME-R1-fioPVE-NVME-R10-4DISK-fioPVE-SAS-SSD-fioiSCSI XCP-NGNFS XCP-NG200K400K600K800K1000KMin: 960336.93 / Avg: 961989.1 / Max: 963743.34Min: 1025.26 / Avg: 270225.16 / Max: 1047960.13Min: 253349.81 / Avg: 253974.01 / Max: 254529.67Min: 2185.42 / Avg: 2435.24 / Max: 2766.12Min: 1286.76 / Avg: 1353.39 / Max: 1370.63Min: 1186.29 / Avg: 1205.44 / Max: 1217.041. (CC) gcc options: -rdynamic -lm -laio -lpthread -ldl -std=gnu99 -ffast-math -include -O3 -fcommon -march=native

24 Results Shown

Flexible IO Tester:
  Rand Read - Linux AIO - No - No - 1MB - 1 - Default Test Directory
  Rand Read - Linux AIO - No - No - 4KB - 1 - Default Test Directory
  Rand Read - Linux AIO - No - No - 64KB - 1 - Default Test Directory
  Rand Write - Linux AIO - No - No - 1MB - 1 - Default Test Directory
  Rand Write - Linux AIO - No - No - 4KB - 1 - Default Test Directory
  Rand Write - Linux AIO - No - No - 64KB - 1 - Default Test Directory
  Seq Read - Linux AIO - No - No - 1MB - 1 - Default Test Directory
  Seq Read - Linux AIO - No - No - 4KB - 1 - Default Test Directory
  Seq Read - Linux AIO - No - No - 64KB - 1 - Default Test Directory
  Seq Write - Linux AIO - No - No - 1MB - 1 - Default Test Directory
  Seq Write - Linux AIO - No - No - 4KB - 1 - Default Test Directory
  Seq Write - Linux AIO - No - No - 64KB - 1 - Default Test Directory
  Rand Read - Linux AIO - No - No - 1MB - 1 - Default Test Directory
  Rand Read - Linux AIO - No - No - 4KB - 1 - Default Test Directory
  Rand Read - Linux AIO - No - No - 64KB - 1 - Default Test Directory
  Rand Write - Linux AIO - No - No - 1MB - 1 - Default Test Directory
  Rand Write - Linux AIO - No - No - 4KB - 1 - Default Test Directory
  Rand Write - Linux AIO - No - No - 64KB - 1 - Default Test Directory
  Seq Read - Linux AIO - No - No - 1MB - 1 - Default Test Directory
  Seq Read - Linux AIO - No - No - 4KB - 1 - Default Test Directory
  Seq Read - Linux AIO - No - No - 64KB - 1 - Default Test Directory
  Seq Write - Linux AIO - No - No - 1MB - 1 - Default Test Directory
  Seq Write - Linux AIO - No - No - 4KB - 1 - Default Test Directory
  Seq Write - Linux AIO - No - No - 64KB - 1 - Default Test Directory

iSCSI XCP-NG

Kernel Notes: Transparent Huge Pages: madvise
Compiler Notes: --build=x86_64-linux-gnu --disable-vtable-verify --disable-werror --enable-bootstrap --enable-cet --enable-checking=release --enable-clocale=gnu --enable-default-pie --enable-gnu-unique-object --enable-languages=c,ada,c++,go,brig,d,fortran,objc,obj-c++,m2 --enable-libphobos-checking=release --enable-libstdcxx-debug --enable-libstdcxx-time=yes --enable-link-serialization=2 --enable-multiarch --enable-multilib --enable-nls --enable-objc-gc=auto --enable-offload-targets=nvptx-none=/build/gcc-11-XeT9lY/gcc-11-11.4.0/debian/tmp-nvptx/usr,amdgcn-amdhsa=/build/gcc-11-XeT9lY/gcc-11-11.4.0/debian/tmp-gcn/usr --enable-plugin --enable-shared --enable-threads=posix --host=x86_64-linux-gnu --program-prefix=x86_64-linux-gnu- --target=x86_64-linux-gnu --with-abi=m64 --with-arch-32=i686 --with-build-config=bootstrap-lto-lean --with-default-libstdcxx-abi=new --with-gcc-major-version-only --with-multilib-list=m32,m64,mx32 --with-target-system-zlib=auto --with-tune=generic --without-cuda-driver -v
Processor Notes: CPU Microcode: 0xa20120a
Disk Mount Options Notes: relatime,rw
Disk Details Notes: Block Size: 4096
Security Notes: gather_data_sampling: Not affected + itlb_multihit: Not affected + l1tf: Not affected + mds: Not affected + meltdown: Not affected + mmio_stale_data: Not affected + retbleed: Not affected + spec_store_bypass: Mitigation of SSB disabled via prctl and seccomp + spectre_v1: Mitigation of usercopy/swapgs barriers and __user pointer sanitization + spectre_v2: Mitigation of Retpolines IBPB: conditional IBRS_FW STIBP: disabled RSB filling PBRSB-eIBRS: Not affected + srbds: Not affected + tsx_async_abort: Not affected

Testing initiated at 31 August 2023 13:01 by user lts.

NFS XCP-NG

Processor: 2 x AMD Ryzen 9 5900X 12-Core (32 Cores), Motherboard: Xen HVM domU v4.13, Chipset: Intel 440FX 82441FX PMC, Memory: 16GB, Disk: 29GB, Graphics: bochs-drmdrmfb

OS: Ubuntu 22.04, Kernel: 5.15.0-82-generic (x86_64), Vulkan: 1.3.238, Compiler: GCC 11.4.0, File-System: ext4, Screen Resolution: 1024x768, System Layer: Xen HVM domU 4.13

Kernel Notes: Transparent Huge Pages: madvise
Compiler Notes: --build=x86_64-linux-gnu --disable-vtable-verify --disable-werror --enable-bootstrap --enable-cet --enable-checking=release --enable-clocale=gnu --enable-default-pie --enable-gnu-unique-object --enable-languages=c,ada,c++,go,brig,d,fortran,objc,obj-c++,m2 --enable-libphobos-checking=release --enable-libstdcxx-debug --enable-libstdcxx-time=yes --enable-link-serialization=2 --enable-multiarch --enable-multilib --enable-nls --enable-objc-gc=auto --enable-offload-targets=nvptx-none=/build/gcc-11-XeT9lY/gcc-11-11.4.0/debian/tmp-nvptx/usr,amdgcn-amdhsa=/build/gcc-11-XeT9lY/gcc-11-11.4.0/debian/tmp-gcn/usr --enable-plugin --enable-shared --enable-threads=posix --host=x86_64-linux-gnu --program-prefix=x86_64-linux-gnu- --target=x86_64-linux-gnu --with-abi=m64 --with-arch-32=i686 --with-build-config=bootstrap-lto-lean --with-default-libstdcxx-abi=new --with-gcc-major-version-only --with-multilib-list=m32,m64,mx32 --with-target-system-zlib=auto --with-tune=generic --without-cuda-driver -v
Processor Notes: CPU Microcode: 0xa20120a
Disk Mount Options Notes: relatime,rw
Disk Details Notes: Block Size: 4096
Security Notes: gather_data_sampling: Not affected + itlb_multihit: Not affected + l1tf: Not affected + mds: Not affected + meltdown: Not affected + mmio_stale_data: Not affected + retbleed: Not affected + spec_store_bypass: Mitigation of SSB disabled via prctl and seccomp + spectre_v1: Mitigation of usercopy/swapgs barriers and __user pointer sanitization + spectre_v2: Mitigation of Retpolines IBPB: conditional IBRS_FW STIBP: disabled RSB filling PBRSB-eIBRS: Not affected + srbds: Not affected + tsx_async_abort: Not affected

Testing initiated at 31 August 2023 14:29 by user lts.

PVE-SAS-SSD-fio

Kernel Notes: Transparent Huge Pages: madvise
Compiler Notes: --build=x86_64-linux-gnu --disable-vtable-verify --disable-werror --enable-bootstrap --enable-cet --enable-checking=release --enable-clocale=gnu --enable-default-pie --enable-gnu-unique-object --enable-languages=c,ada,c++,go,brig,d,fortran,objc,obj-c++,m2 --enable-libphobos-checking=release --enable-libstdcxx-debug --enable-libstdcxx-time=yes --enable-link-serialization=2 --enable-multiarch --enable-multilib --enable-nls --enable-objc-gc=auto --enable-offload-targets=nvptx-none=/build/gcc-11-XeT9lY/gcc-11-11.4.0/debian/tmp-nvptx/usr,amdgcn-amdhsa=/build/gcc-11-XeT9lY/gcc-11-11.4.0/debian/tmp-gcn/usr --enable-plugin --enable-shared --enable-threads=posix --host=x86_64-linux-gnu --program-prefix=x86_64-linux-gnu- --target=x86_64-linux-gnu --with-abi=m64 --with-arch-32=i686 --with-build-config=bootstrap-lto-lean --with-default-libstdcxx-abi=new --with-gcc-major-version-only --with-multilib-list=m32,m64,mx32 --with-target-system-zlib=auto --with-tune=generic --without-cuda-driver -v
Disk Notes: NONE / relatime,rw / Block Size: 4096
Processor Notes: CPU Microcode: 0x2007006
Security Notes: gather_data_sampling: Unknown: Dependent on hypervisor status + itlb_multihit: Not affected + l1tf: Mitigation of PTE Inversion; VMX: flush not necessary SMT disabled + mds: Mitigation of Clear buffers; SMT Host state unknown + meltdown: Mitigation of PTI + mmio_stale_data: Vulnerable: Clear buffers attempted no microcode; SMT Host state unknown + retbleed: Mitigation of IBRS + spec_store_bypass: Mitigation of SSB disabled via prctl and seccomp + spectre_v1: Mitigation of usercopy/swapgs barriers and __user pointer sanitization + spectre_v2: Mitigation of IBRS IBPB: conditional STIBP: disabled RSB filling PBRSB-eIBRS: Not affected + srbds: Not affected + tsx_async_abort: Mitigation of Clear buffers; SMT Host state unknown

Testing initiated at 30 September 2023 04:44 by user tcstudios.

PVE-SATA-SSD-fio

Processor: 2 x Intel Xeon Gold 6152 (12 Cores), Motherboard: QEMU Standard PC (Q35 + ICH9 2009) (3.20230228-4 BIOS), Chipset: Intel 82G33/G31/P35/P31 + ICH9, Memory: 16GB, Disk: 34GB QEMU HDD, Graphics: bochs-drmdrmfb, Audio: Intel 82801I, Monitor: QEMU Monitor, Network: Red Hat Virtio device

OS: Ubuntu 22.04, Kernel: 5.15.0-84-generic (x86_64), Vulkan: 1.3.238, Compiler: GCC 11.4.0, File-System: ext4, Screen Resolution: 1280x800, System Layer: KVM

Kernel Notes: Transparent Huge Pages: madvise
Compiler Notes: --build=x86_64-linux-gnu --disable-vtable-verify --disable-werror --enable-bootstrap --enable-cet --enable-checking=release --enable-clocale=gnu --enable-default-pie --enable-gnu-unique-object --enable-languages=c,ada,c++,go,brig,d,fortran,objc,obj-c++,m2 --enable-libphobos-checking=release --enable-libstdcxx-debug --enable-libstdcxx-time=yes --enable-link-serialization=2 --enable-multiarch --enable-multilib --enable-nls --enable-objc-gc=auto --enable-offload-targets=nvptx-none=/build/gcc-11-XeT9lY/gcc-11-11.4.0/debian/tmp-nvptx/usr,amdgcn-amdhsa=/build/gcc-11-XeT9lY/gcc-11-11.4.0/debian/tmp-gcn/usr --enable-plugin --enable-shared --enable-threads=posix --host=x86_64-linux-gnu --program-prefix=x86_64-linux-gnu- --target=x86_64-linux-gnu --with-abi=m64 --with-arch-32=i686 --with-build-config=bootstrap-lto-lean --with-default-libstdcxx-abi=new --with-gcc-major-version-only --with-multilib-list=m32,m64,mx32 --with-target-system-zlib=auto --with-tune=generic --without-cuda-driver -v
Disk Notes: NONE / relatime,rw / Block Size: 4096
Processor Notes: CPU Microcode: 0x2007006
Security Notes: gather_data_sampling: Unknown: Dependent on hypervisor status + itlb_multihit: Not affected + l1tf: Mitigation of PTE Inversion; VMX: flush not necessary SMT disabled + mds: Mitigation of Clear buffers; SMT Host state unknown + meltdown: Mitigation of PTI + mmio_stale_data: Vulnerable: Clear buffers attempted no microcode; SMT Host state unknown + retbleed: Mitigation of IBRS + spec_store_bypass: Mitigation of SSB disabled via prctl and seccomp + spectre_v1: Mitigation of usercopy/swapgs barriers and __user pointer sanitization + spectre_v2: Mitigation of IBRS IBPB: conditional STIBP: disabled RSB filling PBRSB-eIBRS: Not affected + srbds: Not affected + tsx_async_abort: Mitigation of Clear buffers; SMT Host state unknown

Testing initiated at 1 October 2023 03:41 by user tcstudios.

XCP-NVME-R1-fio

Processor: 3 x AMD EPYC 7402 24-Core (12 Cores), Motherboard: Xen HVM domU v4.13, Chipset: Intel 440FX 82441FX PMC, Memory: 16GB, Disk: 30GB, Graphics: bochs-drmdrmfb

OS: Ubuntu 22.04, Kernel: 5.15.0-84-generic (x86_64), Vulkan: 1.3.238, Compiler: GCC 11.4.0, File-System: ext4, Screen Resolution: 1024x768, System Layer: Xen HVM domU 4.13

Kernel Notes: Transparent Huge Pages: madvise
Compiler Notes: --build=x86_64-linux-gnu --disable-vtable-verify --disable-werror --enable-bootstrap --enable-cet --enable-checking=release --enable-clocale=gnu --enable-default-pie --enable-gnu-unique-object --enable-languages=c,ada,c++,go,brig,d,fortran,objc,obj-c++,m2 --enable-libphobos-checking=release --enable-libstdcxx-debug --enable-libstdcxx-time=yes --enable-link-serialization=2 --enable-multiarch --enable-multilib --enable-nls --enable-objc-gc=auto --enable-offload-targets=nvptx-none=/build/gcc-11-XeT9lY/gcc-11-11.4.0/debian/tmp-nvptx/usr,amdgcn-amdhsa=/build/gcc-11-XeT9lY/gcc-11-11.4.0/debian/tmp-gcn/usr --enable-plugin --enable-shared --enable-threads=posix --host=x86_64-linux-gnu --program-prefix=x86_64-linux-gnu- --target=x86_64-linux-gnu --with-abi=m64 --with-arch-32=i686 --with-build-config=bootstrap-lto-lean --with-default-libstdcxx-abi=new --with-gcc-major-version-only --with-multilib-list=m32,m64,mx32 --with-target-system-zlib=auto --with-tune=generic --without-cuda-driver -v
Processor Notes: CPU Microcode: 0x830107a
Disk Mount Options Notes: relatime,rw
Disk Details Notes: Block Size: 4096
Security Notes: gather_data_sampling: Not affected + itlb_multihit: Not affected + l1tf: Not affected + mds: Not affected + meltdown: Not affected + mmio_stale_data: Not affected + retbleed: Mitigation of untrained return thunk; SMT disabled + spec_store_bypass: Mitigation of SSB disabled via prctl and seccomp + spectre_v1: Mitigation of usercopy/swapgs barriers and __user pointer sanitization + spectre_v2: Mitigation of Retpolines IBPB: conditional STIBP: disabled RSB filling PBRSB-eIBRS: Not affected + srbds: Not affected + tsx_async_abort: Not affected

Testing initiated at 1 October 2023 22:24 by user tcstudios.

PVE-NVME-R10-4DISK-fio

Processor: AMD EPYC 7543 32-Core (12 Cores), Motherboard: QEMU Standard PC (Q35 + ICH9 2009) (3.20230228-4 BIOS), Chipset: Intel 82G33/G31/P35/P31 + ICH9, Memory: 16GB, Disk: 34GB QEMU HDD, Graphics: bochs-drmdrmfb, Audio: Intel 82801I, Monitor: QEMU Monitor, Network: Red Hat Virtio device

OS: Ubuntu 22.04, Kernel: 5.15.0-86-generic (x86_64), Vulkan: 1.3.238, Compiler: GCC 11.4.0, File-System: ext4, Screen Resolution: 1280x800, System Layer: KVM

Kernel Notes: Transparent Huge Pages: madvise
Compiler Notes: --build=x86_64-linux-gnu --disable-vtable-verify --disable-werror --enable-bootstrap --enable-cet --enable-checking=release --enable-clocale=gnu --enable-default-pie --enable-gnu-unique-object --enable-languages=c,ada,c++,go,brig,d,fortran,objc,obj-c++,m2 --enable-libphobos-checking=release --enable-libstdcxx-debug --enable-libstdcxx-time=yes --enable-link-serialization=2 --enable-multiarch --enable-multilib --enable-nls --enable-objc-gc=auto --enable-offload-targets=nvptx-none=/build/gcc-11-XeT9lY/gcc-11-11.4.0/debian/tmp-nvptx/usr,amdgcn-amdhsa=/build/gcc-11-XeT9lY/gcc-11-11.4.0/debian/tmp-gcn/usr --enable-plugin --enable-shared --enable-threads=posix --host=x86_64-linux-gnu --program-prefix=x86_64-linux-gnu- --target=x86_64-linux-gnu --with-abi=m64 --with-arch-32=i686 --with-build-config=bootstrap-lto-lean --with-default-libstdcxx-abi=new --with-gcc-major-version-only --with-multilib-list=m32,m64,mx32 --with-target-system-zlib=auto --with-tune=generic --without-cuda-driver -v
Disk Notes: NONE / relatime,rw / Block Size: 4096
Processor Notes: CPU Microcode: 0xa00110d
Security Notes: gather_data_sampling: Not affected + itlb_multihit: Not affected + l1tf: Not affected + mds: Not affected + meltdown: Not affected + mmio_stale_data: Not affected + retbleed: Not affected + spec_rstack_overflow: Mitigation of safe RET no microcode + spec_store_bypass: Mitigation of SSB disabled via prctl and seccomp + spectre_v1: Mitigation of usercopy/swapgs barriers and __user pointer sanitization + spectre_v2: Mitigation of Retpolines IBPB: conditional IBRS_FW STIBP: disabled RSB filling PBRSB-eIBRS: Not affected + srbds: Not affected + tsx_async_abort: Not affected

Testing initiated at 3 November 2023 05:45 by user tcstudios.