dev01-fs VMware I/O performance testing xfs vNVME vSAN: Processor: 112 x Intel Xeon Gold 6258R (223 Cores), Motherboard: Intel 440BX (VMW71.00V.16460286.B64.2006250725 BIOS), Chipset: Intel 440BX/ZX/DX, Memory: 494GB, Disk: 537GB VMware Virtual NVMe Disk + 137GB VMware Virtual NVMe Disk + 1100GB VMware Virtual NVMe Disk + 4948GB VMware Virtual NVMe Disk, Graphics: VMware SVGA II, Network: VMware VMXNET3 OS: Ubuntu 20.04, Kernel: 5.8.0-59-generic (x86_64), Compiler: GCC 9.3.0, File-System: xfs, Screen Resolution: 1176x885, System Layer: VMware xfs passthru raid0 2x Hynix: Processor: 2 x Intel Xeon Gold 6258R (112 Cores), Motherboard: Intel 440BX (6.00 BIOS), Chipset: Intel 440BX/ZX/DX, Memory: 474GB, Disk: 2 x SK hynix PC401 NVMe 512GB + 4 x 2000GB Samsung SSD 980 PRO 2TB + 172GB Virtual disk, Graphics: VMware SVGA II, Network: VMware VMXNET3 OS: Ubuntu 20.04, Kernel: 5.11.0-34-generic (x86_64), Vulkan: 1.0.2, Compiler: GCC 9.3.0 + Clang 10.0.0-4ubuntu1, File-System: xfs, Screen Resolution: 1176x885, System Layer: VMware xfs passthru 1x 980: Processor: 2 x Intel Xeon Gold 6258R (112 Cores), Motherboard: Intel 440BX (6.00 BIOS), Chipset: Intel 440BX/ZX/DX, Memory: 474GB, Disk: 2 x SK hynix PC401 NVMe 512GB + 4 x 2000GB Samsung SSD 980 PRO 2TB + 172GB Virtual disk, Graphics: VMware SVGA II, Network: VMware VMXNET3 OS: Ubuntu 20.04, Kernel: 5.11.0-34-generic (x86_64), Vulkan: 1.0.2, Compiler: GCC 9.3.0 + Clang 10.0.0-4ubuntu1, File-System: xfs, Screen Resolution: 1176x885, System Layer: VMware xfs passthru raid0 4x 980: Processor: 2 x Intel Xeon Gold 6258R (112 Cores), Motherboard: Intel 440BX (6.00 BIOS), Chipset: Intel 440BX/ZX/DX, Memory: 474GB, Disk: 2 x SK hynix PC401 NVMe 512GB + 4 x 2000GB Samsung SSD 980 PRO 2TB + 172GB Virtual disk, Graphics: VMware SVGA II, Network: VMware VMXNET3 OS: Ubuntu 20.04, Kernel: 5.11.0-34-generic (x86_64), Vulkan: 1.0.2, Compiler: GCC 9.3.0 + Clang 10.0.0-4ubuntu1, File-System: xfs, Screen Resolution: 1176x885, System Layer: VMware xfs passthru raid5 4x 980: Processor: 2 x Intel Xeon Gold 6258R (112 Cores), Motherboard: Intel 440BX (6.00 BIOS), Chipset: Intel 440BX/ZX/DX, Memory: 474GB, Disk: 2 x SK hynix PC401 NVMe 512GB + 4 x 2000GB Samsung SSD 980 PRO 2TB + 172GB Virtual disk, Graphics: VMware SVGA II, Network: VMware VMXNET3 OS: Ubuntu 20.04, Kernel: 5.4.0-84-generic (x86_64), Vulkan: 1.0.2, Compiler: GCC 9.3.0 + Clang 10.0.0-4ubuntu1, File-System: xfs, Screen Resolution: 1176x885, System Layer: VMware zfs passthru raidz1 4x 980: Processor: 2 x Intel Xeon Gold 6258R (112 Cores), Motherboard: Intel 440BX (6.00 BIOS), Chipset: Intel 440BX/ZX/DX, Memory: 474GB, Disk: 2 x SK hynix PC401 NVMe 512GB + 4 x 2000GB Samsung SSD 980 PRO 2TB + 172GB Virtual disk, Graphics: VMware SVGA II, Network: VMware VMXNET3 OS: Ubuntu 20.04, Kernel: 5.11.0-34-generic (x86_64), Vulkan: 1.0.2, Compiler: GCC 9.3.0 + Clang 10.0.0-4ubuntu1, File-System: zfs, Screen Resolution: 1176x885, System Layer: VMware zfs passthru raidz1 4x 980 dedup+lz4: Processor: 2 x Intel Xeon Gold 6258R (112 Cores), Motherboard: Intel 440BX (6.00 BIOS), Chipset: Intel 440BX/ZX/DX, Memory: 474GB, Disk: 2 x SK hynix PC401 NVMe 512GB + 4 x 2000GB Samsung SSD 980 PRO 2TB + 172GB Virtual disk, Graphics: VMware SVGA II, Network: VMware VMXNET3 OS: Ubuntu 20.04, Kernel: 5.11.0-34-generic (x86_64), Vulkan: 1.0.2, Compiler: GCC 9.3.0 + Clang 10.0.0-4ubuntu1, File-System: zfs, Screen Resolution: 1176x885, System Layer: VMware ext4 passthru raid5 4x 980: Processor: 2 x Intel Xeon Gold 6258R (112 Cores), Motherboard: Intel 440BX (6.00 BIOS), Chipset: Intel 440BX/ZX/DX, Memory: 474GB, Disk: 2 x SK hynix PC401 NVMe 512GB + 4 x 2000GB Samsung SSD 980 PRO 2TB + 172GB Virtual disk, Graphics: VMware SVGA II, Network: VMware VMXNET3 OS: Ubuntu 20.04, Kernel: 5.11.0-34-generic (x86_64), Vulkan: 1.0.2, Compiler: GCC 9.3.0 + Clang 10.0.0-4ubuntu1, File-System: ext4, Screen Resolution: 1176x885, System Layer: VMware Flexible IO Tester 3.25 Type: Random Read - Engine: IO_uring - Buffered: Yes - Direct: No - Block Size: 4KB - Disk Target: Default Test Directory MB/s > Higher Is Better xfs vNVME vSAN ....................... 367.0 |========= xfs passthru raid0 2x Hynix .......... 949.0 |======================== xfs passthru 1x 980 .................. 983.0 |======================== xfs passthru raid0 4x 980 ............ 1200.0 |============================== xfs passthru raid5 4x 980 ............ 21.9 |= zfs passthru raidz1 4x 980 ........... 1290.0 |================================ zfs passthru raidz1 4x 980 dedup+lz4 . 1271.0 |================================ ext4 passthru raid5 4x 980 ........... 1176.0 |============================= Flexible IO Tester 3.25 Type: Random Read - Engine: IO_uring - Buffered: Yes - Direct: No - Block Size: 4KB - Disk Target: Default Test Directory IOPS > Higher Is Better xfs vNVME vSAN ....................... 93880 |========= xfs passthru raid0 2x Hynix .......... 242667 |======================== xfs passthru 1x 980 .................. 251667 |======================== xfs passthru raid0 4x 980 ............ 307200 |============================== xfs passthru raid5 4x 980 ............ 7457 |= zfs passthru raidz1 4x 980 ........... 330133 |================================ zfs passthru raidz1 4x 980 dedup+lz4 . 325200 |================================ ext4 passthru raid5 4x 980 ........... 301267 |============================= Flexible IO Tester 3.25 Type: Random Read - Engine: IO_uring - Buffered: Yes - Direct: No - Block Size: 8MB - Disk Target: Default Test Directory MB/s > Higher Is Better xfs vNVME vSAN .............. 2705 |========================================= xfs passthru raid0 2x Hynix . 1895 |============================= xfs passthru 1x 980 ......... 2674 |======================================== xfs passthru raid0 4x 980 ... 2856 |=========================================== xfs passthru raid5 4x 980 ... 2711 |========================================= ext4 passthru raid5 4x 980 .. 2731 |========================================= Flexible IO Tester 3.25 Type: Random Read - Engine: IO_uring - Buffered: Yes - Direct: No - Block Size: 8MB - Disk Target: Default Test Directory IOPS > Higher Is Better xfs vNVME vSAN ....................... 335 |===== xfs passthru raid0 2x Hynix .......... 233 |==== xfs passthru 1x 980 .................. 331 |===== xfs passthru raid0 4x 980 ............ 353 |===== xfs passthru raid5 4x 980 ............ 451 |======= zfs passthru raidz1 4x 980 ........... 2174 |================================= zfs passthru raidz1 4x 980 ........... 2081 |================================ zfs passthru raidz1 4x 980 dedup+lz4 . 2234 |================================== ext4 passthru raid5 4x 980 ........... 338 |===== Flexible IO Tester 3.25 Type: Random Write - Engine: IO_uring - Buffered: Yes - Direct: No - Block Size: 4KB - Disk Target: Default Test Directory MB/s > Higher Is Better xfs vNVME vSAN ....................... 387 |=========== xfs passthru raid0 2x Hynix .......... 734 |===================== xfs passthru 1x 980 .................. 860 |======================== xfs passthru raid0 4x 980 ............ 942 |========================== xfs passthru raid5 4x 980 ............ 188 |===== zfs passthru raidz1 4x 980 ........... 1216 |================================== zfs passthru raidz1 4x 980 dedup+lz4 . 1051 |============================= ext4 passthru raid5 4x 980 ........... 459 |============= Flexible IO Tester 3.25 Type: Random Write - Engine: IO_uring - Buffered: Yes - Direct: No - Block Size: 4KB - Disk Target: Default Test Directory IOPS > Higher Is Better xfs vNVME vSAN ....................... 98700 |========== xfs passthru raid0 2x Hynix .......... 188000 |=================== xfs passthru 1x 980 .................. 220000 |======================= xfs passthru raid0 4x 980 ............ 241000 |========================= xfs passthru raid5 4x 980 ............ 64000 |======= zfs passthru raidz1 4x 980 ........... 311667 |================================ zfs passthru raidz1 4x 980 dedup+lz4 . 268667 |============================ ext4 passthru raid5 4x 980 ........... 117667 |============ Flexible IO Tester 3.25 Type: Random Write - Engine: IO_uring - Buffered: Yes - Direct: No - Block Size: 8MB - Disk Target: Default Test Directory MB/s > Higher Is Better xfs vNVME vSAN ....................... 586 |== xfs passthru raid0 2x Hynix .......... 1350 |===== xfs passthru 1x 980 .................. 1835 |====== xfs passthru raid0 4x 980 ............ 1788 |====== xfs passthru raid5 4x 980 ............ 2274 |======== zfs passthru raidz1 4x 980 ........... 9329 |================================ zfs passthru raidz1 4x 980 dedup+lz4 . 9772 |================================== ext4 passthru raid5 4x 980 ........... 695 |== Flexible IO Tester 3.25 Type: Random Write - Engine: IO_uring - Buffered: Yes - Direct: No - Block Size: 8MB - Disk Target: Default Test Directory IOPS > Higher Is Better xfs vNVME vSAN ....................... 70 |== xfs passthru raid0 2x Hynix .......... 165 |==== xfs passthru 1x 980 .................. 226 |====== xfs passthru raid0 4x 980 ............ 220 |====== xfs passthru raid5 4x 980 ............ 378 |========== zfs passthru raidz1 4x 980 ........... 1252 |================================== zfs passthru raidz1 4x 980 dedup+lz4 . 1218 |================================= ext4 passthru raid5 4x 980 ........... 83 |== Flexible IO Tester 3.25 Type: Random Read - Engine: IO_uring - Buffered: Yes - Direct: No - Block Size: 512KB - Disk Target: Default Test Directory MB/s > Higher Is Better xfs vNVME vSAN .............. 2234 |===================================== xfs passthru raid0 2x Hynix . 1943 |================================ xfs passthru 1x 980 ......... 2616 |=========================================== xfs passthru raid0 4x 980 ... 2365 |======================================= xfs passthru raid5 4x 980 ... 1714 |============================ ext4 passthru raid5 4x 980 .. 2316 |====================================== Flexible IO Tester 3.25 Type: Random Read - Engine: IO_uring - Buffered: Yes - Direct: No - Block Size: 512KB - Disk Target: Default Test Directory IOPS > Higher Is Better xfs vNVME vSAN ....................... 4464 |=== xfs passthru raid0 2x Hynix .......... 3882 |=== xfs passthru 1x 980 .................. 5228 |==== xfs passthru raid0 4x 980 ............ 4726 |=== xfs passthru raid5 4x 980 ............ 4571 |=== zfs passthru raidz1 4x 980 ........... 38392 |=========================== zfs passthru raidz1 4x 980 ........... 38950 |============================ zfs passthru raidz1 4x 980 dedup+lz4 . 46487 |================================= ext4 passthru raid5 4x 980 ........... 4629 |=== Flexible IO Tester 3.25 Type: Random Write - Engine: IO_uring - Buffered: Yes - Direct: No - Block Size: 512KB - Disk Target: Default Test Directory MB/s > Higher Is Better xfs vNVME vSAN .............. 478 |=============== xfs passthru raid0 2x Hynix . 983 |=============================== xfs passthru 1x 980 ......... 1182 |===================================== xfs passthru raid0 4x 980 ... 1359 |=========================================== xfs passthru raid5 4x 980 ... 1193 |====================================== ext4 passthru raid5 4x 980 .. 533 |================= Flexible IO Tester 3.25 Type: Random Write - Engine: IO_uring - Buffered: Yes - Direct: No - Block Size: 512KB - Disk Target: Default Test Directory IOPS > Higher Is Better xfs vNVME vSAN ....................... 953 |= xfs passthru raid0 2x Hynix .......... 1962 |== xfs passthru 1x 980 .................. 2361 |=== xfs passthru raid0 4x 980 ............ 2715 |=== xfs passthru raid5 4x 980 ............ 3182 |==== zfs passthru raidz1 4x 980 ........... 27300 |================================ zfs passthru raidz1 4x 980 ........... 28308 |================================= zfs passthru raidz1 4x 980 dedup+lz4 . 26907 |=============================== ext4 passthru raid5 4x 980 ........... 1061 |= Flexible IO Tester 3.25 Type: Sequential Read - Engine: IO_uring - Buffered: Yes - Direct: No - Block Size: 4KB - Disk Target: Default Test Directory MB/s > Higher Is Better xfs vNVME vSAN ....................... 593.0 |============= xfs passthru raid0 2x Hynix .......... 774.0 |================= xfs passthru 1x 980 .................. 776.0 |================== xfs passthru raid0 4x 980 ............ 748.0 |================= xfs passthru raid5 4x 980 ............ 15.6 | zfs passthru raidz1 4x 980 ........... 1418.0 |================================ zfs passthru raidz1 4x 980 dedup+lz4 . 1357.0 |=============================== ext4 passthru raid5 4x 980 ........... 1034.0 |======================= Flexible IO Tester 3.25 Type: Sequential Read - Engine: IO_uring - Buffered: Yes - Direct: No - Block Size: 4KB - Disk Target: Default Test Directory IOPS > Higher Is Better xfs vNVME vSAN ....................... 151667 |============= xfs passthru raid0 2x Hynix .......... 198333 |================= xfs passthru 1x 980 .................. 199000 |================== xfs passthru raid0 4x 980 ............ 191667 |================= xfs passthru raid5 4x 980 ............ 5333 | zfs passthru raidz1 4x 980 ........... 363000 |================================ zfs passthru raidz1 4x 980 dedup+lz4 . 347333 |=============================== ext4 passthru raid5 4x 980 ........... 264800 |======================= Flexible IO Tester 3.25 Type: Sequential Read - Engine: IO_uring - Buffered: Yes - Direct: No - Block Size: 8MB - Disk Target: Default Test Directory MB/s > Higher Is Better xfs vNVME vSAN .............. 2646 |========================================= xfs passthru raid0 2x Hynix . 1968 |============================== xfs passthru 1x 980 ......... 2650 |========================================= xfs passthru raid0 4x 980 ... 2777 |=========================================== xfs passthru raid5 4x 980 ... 1512 |======================= ext4 passthru raid5 4x 980 .. 2702 |========================================== Flexible IO Tester 3.25 Type: Sequential Read - Engine: IO_uring - Buffered: Yes - Direct: No - Block Size: 8MB - Disk Target: Default Test Directory IOPS > Higher Is Better xfs vNVME vSAN ....................... 327 |===== xfs passthru raid0 2x Hynix .......... 242 |==== xfs passthru 1x 980 .................. 328 |===== xfs passthru raid0 4x 980 ............ 344 |====== xfs passthru raid5 4x 980 ............ 252 |==== zfs passthru raidz1 4x 980 ........... 1945 |================================ zfs passthru raidz1 4x 980 ........... 2018 |================================= zfs passthru raidz1 4x 980 dedup+lz4 . 2079 |================================== ext4 passthru raid5 4x 980 ........... 334 |===== Flexible IO Tester 3.25 Type: Sequential Write - Engine: IO_uring - Buffered: Yes - Direct: No - Block Size: 4KB - Disk Target: Default Test Directory MB/s > Higher Is Better xfs vNVME vSAN ....................... 385 |========== xfs passthru raid0 2x Hynix .......... 757 |=================== xfs passthru 1x 980 .................. 868 |====================== xfs passthru raid0 4x 980 ............ 952 |======================== xfs passthru raid5 4x 980 ............ 188 |===== zfs passthru raidz1 4x 980 ........... 1361 |================================== zfs passthru raidz1 4x 980 dedup+lz4 . 1211 |============================== ext4 passthru raid5 4x 980 ........... 446 |=========== Flexible IO Tester 3.25 Type: Sequential Write - Engine: IO_uring - Buffered: Yes - Direct: No - Block Size: 4KB - Disk Target: Default Test Directory IOPS > Higher Is Better xfs vNVME vSAN ....................... 98475 |========= xfs passthru raid0 2x Hynix .......... 193667 |================== xfs passthru 1x 980 .................. 222000 |==================== xfs passthru raid0 4x 980 ............ 243667 |====================== xfs passthru raid5 4x 980 ............ 64000 |====== zfs passthru raidz1 4x 980 ........... 348333 |================================ zfs passthru raidz1 4x 980 dedup+lz4 . 310000 |============================ ext4 passthru raid5 4x 980 ........... 114267 |========== Flexible IO Tester 3.25 Type: Sequential Write - Engine: IO_uring - Buffered: Yes - Direct: No - Block Size: 8MB - Disk Target: Default Test Directory MB/s > Higher Is Better xfs vNVME vSAN ....................... 608 |== xfs passthru raid0 2x Hynix .......... 1195 |==== xfs passthru 1x 980 .................. 1699 |====== xfs passthru raid0 4x 980 ............ 1791 |====== xfs passthru raid5 4x 980 ............ 2222 |======== zfs passthru raidz1 4x 980 ........... 9402 |================================= zfs passthru raidz1 4x 980 dedup+lz4 . 9669 |================================== ext4 passthru raid5 4x 980 ........... 646 |== Flexible IO Tester 3.25 Type: Sequential Write - Engine: IO_uring - Buffered: Yes - Direct: No - Block Size: 8MB - Disk Target: Default Test Directory IOPS > Higher Is Better xfs vNVME vSAN ....................... 72 |== xfs passthru raid0 2x Hynix .......... 146 |==== xfs passthru 1x 980 .................. 209 |====== xfs passthru raid0 4x 980 ............ 220 |====== xfs passthru raid5 4x 980 ............ 370 |========== zfs passthru raidz1 4x 980 ........... 1212 |================================== zfs passthru raidz1 4x 980 dedup+lz4 . 1205 |================================== ext4 passthru raid5 4x 980 ........... 77 |== Flexible IO Tester 3.25 Type: Sequential Read - Engine: IO_uring - Buffered: Yes - Direct: No - Block Size: 512KB - Disk Target: Default Test Directory MB/s > Higher Is Better xfs vNVME vSAN .............. 1725 |============================ xfs passthru raid0 2x Hynix . 2000 |================================= xfs passthru 1x 980 ......... 2606 |=========================================== xfs passthru raid0 4x 980 ... 2367 |======================================= xfs passthru raid5 4x 980 ... 545 |========= ext4 passthru raid5 4x 980 .. 2300 |====================================== Flexible IO Tester 3.25 Type: Sequential Read - Engine: IO_uring - Buffered: Yes - Direct: No - Block Size: 512KB - Disk Target: Default Test Directory IOPS > Higher Is Better xfs vNVME vSAN ....................... 3448 |=== xfs passthru raid0 2x Hynix .......... 3997 |=== xfs passthru 1x 980 .................. 5208 |==== xfs passthru raid0 4x 980 ............ 4729 |==== xfs passthru raid5 4x 980 ............ 1454 |= zfs passthru raidz1 4x 980 ........... 35192 |============================= zfs passthru raidz1 4x 980 dedup+lz4 . 39540 |================================= ext4 passthru raid5 4x 980 ........... 4595 |==== Flexible IO Tester 3.25 Type: Sequential Write - Engine: IO_uring - Buffered: Yes - Direct: No - Block Size: 512KB - Disk Target: Default Test Directory MB/s > Higher Is Better xfs vNVME vSAN .............. 442 |============= xfs passthru raid0 2x Hynix . 959 |============================= xfs passthru 1x 980 ......... 1159 |=================================== xfs passthru raid0 4x 980 ... 1409 |=========================================== xfs passthru raid5 4x 980 ... 1186 |==================================== ext4 passthru raid5 4x 980 .. 532 |================ Flexible IO Tester 3.25 Type: Sequential Write - Engine: IO_uring - Buffered: Yes - Direct: No - Block Size: 512KB - Disk Target: Default Test Directory IOPS > Higher Is Better xfs vNVME vSAN ....................... 881 |= xfs passthru raid0 2x Hynix .......... 1915 |== xfs passthru 1x 980 .................. 2315 |=== xfs passthru raid0 4x 980 ............ 2815 |=== xfs passthru raid5 4x 980 ............ 3162 |==== zfs passthru raidz1 4x 980 ........... 26653 |================================= zfs passthru raidz1 4x 980 dedup+lz4 . 25300 |=============================== ext4 passthru raid5 4x 980 ........... 1060 |= Compile Bench 0.6 Test: Compile MB/s > Higher Is Better xfs vNVME vSAN ....................... 1872.66 |========================= xfs passthru raid0 2x Hynix .......... 2309.72 |=============================== xfs passthru 1x 980 .................. 2278.30 |=============================== xfs passthru raid0 4x 980 ............ 2298.12 |=============================== xfs passthru raid5 4x 980 ............ 2152.95 |============================= zfs passthru raidz1 4x 980 ........... 1032.06 |============== zfs passthru raidz1 4x 980 dedup+lz4 . 970.40 |============= ext4 passthru raid5 4x 980 ........... 1613.80 |====================== Compile Bench 0.6 Test: Initial Create MB/s > Higher Is Better xfs vNVME vSAN ....................... 407.78 |========================== xfs passthru raid0 2x Hynix .......... 487.30 |=============================== xfs passthru 1x 980 .................. 462.56 |============================== xfs passthru raid0 4x 980 ............ 483.31 |=============================== xfs passthru raid5 4x 980 ............ 462.01 |============================== zfs passthru raidz1 4x 980 ........... 201.85 |============= zfs passthru raidz1 4x 980 dedup+lz4 . 227.15 |=============== ext4 passthru raid5 4x 980 ........... 500.93 |================================ Compile Bench 0.6 Test: Read Compiled Tree MB/s > Higher Is Better xfs vNVME vSAN ....................... 2465.65 |============================= xfs passthru raid0 2x Hynix .......... 2626.19 |=============================== xfs passthru 1x 980 .................. 2610.23 |=============================== xfs passthru raid0 4x 980 ............ 2513.94 |============================== xfs passthru raid5 4x 980 ............ 2419.80 |============================= zfs passthru raidz1 4x 980 ........... 1138.71 |============= zfs passthru raidz1 4x 980 dedup+lz4 . 1075.94 |============= ext4 passthru raid5 4x 980 ........... 2500.45 |==============================