pts-disk-different-nvmes AMD Ryzen Threadripper 1900X 8-Core testing with a Gigabyte X399 DESIGNARE EX-CF (F13a BIOS) and NVIDIA Quadro P400 on Debian 11 via the Phoronix Test Suite. ZFS zraid1 4xNVME Pool: Processor: AMD Ryzen Threadripper 1900X 8-Core @ 3.80GHz (8 Cores / 16 Threads), Motherboard: Gigabyte X399 DESIGNARE EX-CF (F13a BIOS), Chipset: AMD 17h, Memory: 64GB, Disk: Samsung SSD 960 EVO 500GB + 8 x 2000GB Western Digital WD_BLACK SN770 2TB + 1000GB CT1000P5PSSD8, Graphics: NVIDIA Quadro P400, Audio: NVIDIA GP107GL HD Audio, Monitor: DELL S2340T, Network: 4 x Intel I350 + Intel 8265 / 8275 OS: Debian 11, Kernel: 5.10.0-19-amd64 (x86_64), Compiler: GCC 10.2.1 20210110, File-System: zfs, Screen Resolution: 1920x1080 SQLite 3.30.1 Threads / Copies: 1 Seconds < Lower Is Better ZFS zraid1 4xNVME Pool . 7.852 |=============================================== SQLite 3.30.1 Threads / Copies: 8 Seconds < Lower Is Better ZFS zraid1 4xNVME Pool . 11.27 |=============================================== SQLite 3.30.1 Threads / Copies: 32 Seconds < Lower Is Better ZFS zraid1 4xNVME Pool . 23.19 |=============================================== SQLite 3.30.1 Threads / Copies: 64 Seconds < Lower Is Better ZFS zraid1 4xNVME Pool . 37.45 |=============================================== SQLite 3.30.1 Threads / Copies: 128 Seconds < Lower Is Better ZFS zraid1 4xNVME Pool . 76.09 |=============================================== Flexible IO Tester 3.29 Type: Random Read - IO Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 2MB - Disk Target: Default Test Directory MB/s > Higher Is Better ZFS zraid1 4xNVME Pool . 4245 |================================================ Flexible IO Tester 3.29 Type: Random Read - IO Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 2MB - Disk Target: Default Test Directory IOPS > Higher Is Better ZFS zraid1 4xNVME Pool . 2119 |================================================ Flexible IO Tester 3.29 Type: Random Read - IO Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 4KB - Disk Target: Default Test Directory MB/s > Higher Is Better ZFS zraid1 4xNVME Pool . 194 |================================================= Flexible IO Tester 3.29 Type: Random Read - IO Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 4KB - Disk Target: Default Test Directory IOPS > Higher Is Better ZFS zraid1 4xNVME Pool . 49567 |=============================================== Flexible IO Tester 3.29 Type: Random Write - IO Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 2MB - Disk Target: Default Test Directory MB/s > Higher Is Better ZFS zraid1 4xNVME Pool . 2458 |================================================ Flexible IO Tester 3.29 Type: Random Write - IO Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 2MB - Disk Target: Default Test Directory IOPS > Higher Is Better ZFS zraid1 4xNVME Pool . 1225 |================================================ Flexible IO Tester 3.29 Type: Random Write - IO Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 4KB - Disk Target: Default Test Directory MB/s > Higher Is Better ZFS zraid1 4xNVME Pool . 162 |================================================= Flexible IO Tester 3.29 Type: Random Write - IO Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 4KB - Disk Target: Default Test Directory IOPS > Higher Is Better ZFS zraid1 4xNVME Pool . 41500 |=============================================== Flexible IO Tester 3.29 Type: Sequential Read - IO Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 2MB - Disk Target: Default Test Directory MB/s > Higher Is Better ZFS zraid1 4xNVME Pool . 4075 |================================================ Flexible IO Tester 3.29 Type: Sequential Read - IO Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 2MB - Disk Target: Default Test Directory IOPS > Higher Is Better ZFS zraid1 4xNVME Pool . 2034 |================================================ Flexible IO Tester 3.29 Type: Sequential Read - IO Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 4KB - Disk Target: Default Test Directory MB/s > Higher Is Better ZFS zraid1 4xNVME Pool . 1287 |================================================ Flexible IO Tester 3.29 Type: Sequential Read - IO Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 4KB - Disk Target: Default Test Directory IOPS > Higher Is Better ZFS zraid1 4xNVME Pool . 329333 |============================================== Flexible IO Tester 3.29 Type: Sequential Write - IO Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 2MB - Disk Target: Default Test Directory MB/s > Higher Is Better ZFS zraid1 4xNVME Pool . 2065 |================================================ Flexible IO Tester 3.29 Type: Sequential Write - IO Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 2MB - Disk Target: Default Test Directory IOPS > Higher Is Better ZFS zraid1 4xNVME Pool . 1029 |================================================ Flexible IO Tester 3.29 Type: Sequential Write - IO Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 4KB - Disk Target: Default Test Directory MB/s > Higher Is Better ZFS zraid1 4xNVME Pool . 670 |================================================= Flexible IO Tester 3.29 Type: Sequential Write - IO Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 4KB - Disk Target: Default Test Directory IOPS > Higher Is Better ZFS zraid1 4xNVME Pool . 171333 |============================================== FS-Mark 3.3 Test: 1000 Files, 1MB Size Files/s > Higher Is Better ZFS zraid1 4xNVME Pool . 646.5 |=============================================== FS-Mark 3.3 Test: 5000 Files, 1MB Size, 4 Threads Files/s > Higher Is Better ZFS zraid1 4xNVME Pool . 1632.7 |============================================== FS-Mark 3.3 Test: 4000 Files, 32 Sub Dirs, 1MB Size Files/s > Higher Is Better ZFS zraid1 4xNVME Pool . 684.2 |=============================================== FS-Mark 3.3 Test: 1000 Files, 1MB Size, No Sync/FSync Files/s > Higher Is Better ZFS zraid1 4xNVME Pool . 1359.3 |============================================== Dbench 4.0 12 Clients MB/s > Higher Is Better ZFS zraid1 4xNVME Pool . 2626.03 |============================================= Dbench 4.0 1 Clients MB/s > Higher Is Better ZFS zraid1 4xNVME Pool . 410.17 |============================================== IOR 3.3.0 Block Size: 2MB - Disk Target: /nvme_pool MB/s > Higher Is Better IOR 3.3.0 Block Size: 4MB - Disk Target: /nvme_pool MB/s > Higher Is Better IOR 3.3.0 Block Size: 8MB - Disk Target: /nvme_pool MB/s > Higher Is Better IOR 3.3.0 Block Size: 16MB - Disk Target: /nvme_pool MB/s > Higher Is Better IOR 3.3.0 Block Size: 32MB - Disk Target: /nvme_pool MB/s > Higher Is Better IOR 3.3.0 Block Size: 64MB - Disk Target: /nvme_pool MB/s > Higher Is Better IOR 3.3.0 Block Size: 256MB - Disk Target: /nvme_pool MB/s > Higher Is Better IOR 3.3.0 Block Size: 512MB - Disk Target: /nvme_pool MB/s > Higher Is Better IOR 3.3.0 Block Size: 1024MB - Disk Target: /nvme_pool MB/s > Higher Is Better IOR 3.3.0 Block Size: 2MB - Disk Target: /mnt/single_ssd MB/s > Higher Is Better IOR 3.3.0 Block Size: 4MB - Disk Target: /mnt/single_ssd MB/s > Higher Is Better IOR 3.3.0 Block Size: 8MB - Disk Target: /mnt/single_ssd MB/s > Higher Is Better IOR 3.3.0 Block Size: 16MB - Disk Target: /mnt/single_ssd MB/s > Higher Is Better IOR 3.3.0 Block Size: 32MB - Disk Target: /mnt/single_ssd MB/s > Higher Is Better IOR 3.3.0 Block Size: 64MB - Disk Target: /mnt/single_ssd MB/s > Higher Is Better IOR 3.3.0 Block Size: 256MB - Disk Target: /mnt/single_ssd MB/s > Higher Is Better IOR 3.3.0 Block Size: 512MB - Disk Target: /mnt/single_ssd MB/s > Higher Is Better IOR 3.3.0 Block Size: 1024MB - Disk Target: /mnt/single_ssd MB/s > Higher Is Better IOR 3.3.0 Block Size: 2MB - Disk Target: Default Test Directory MB/s > Higher Is Better IOR 3.3.0 Block Size: 4MB - Disk Target: Default Test Directory MB/s > Higher Is Better IOR 3.3.0 Block Size: 8MB - Disk Target: Default Test Directory MB/s > Higher Is Better IOR 3.3.0 Block Size: 16MB - Disk Target: Default Test Directory MB/s > Higher Is Better IOR 3.3.0 Block Size: 2MB - Disk Target: /mnt/mdadm_raid5_4disks MB/s > Higher Is Better IOR 3.3.0 Block Size: 32MB - Disk Target: Default Test Directory MB/s > Higher Is Better IOR 3.3.0 Block Size: 4MB - Disk Target: /mnt/mdadm_raid5_4disks MB/s > Higher Is Better IOR 3.3.0 Block Size: 64MB - Disk Target: Default Test Directory MB/s > Higher Is Better IOR 3.3.0 Block Size: 8MB - Disk Target: /mnt/mdadm_raid5_4disks MB/s > Higher Is Better IOR 3.3.0 Block Size: 16MB - Disk Target: /mnt/mdadm_raid5_4disks MB/s > Higher Is Better IOR 3.3.0 Block Size: 256MB - Disk Target: Default Test Directory MB/s > Higher Is Better IOR 3.3.0 Block Size: 32MB - Disk Target: /mnt/mdadm_raid5_4disks MB/s > Higher Is Better IOR 3.3.0 Block Size: 512MB - Disk Target: Default Test Directory MB/s > Higher Is Better IOR 3.3.0 Block Size: 64MB - Disk Target: /mnt/mdadm_raid5_4disks MB/s > Higher Is Better IOR 3.3.0 Block Size: 1024MB - Disk Target: Default Test Directory MB/s > Higher Is Better IOR 3.3.0 Block Size: 256MB - Disk Target: /mnt/mdadm_raid5_4disks MB/s > Higher Is Better IOR 3.3.0 Block Size: 512MB - Disk Target: /mnt/mdadm_raid5_4disks MB/s > Higher Is Better IOR 3.3.0 Block Size: 1024MB - Disk Target: /mnt/mdadm_raid5_4disks MB/s > Higher Is Better IOzone 3.465 8GB Write Performance MB/s > Higher Is Better Compile Bench 0.6 Test: Compile MB/s > Higher Is Better ZFS zraid1 4xNVME Pool . 1398.57 |============================================= Compile Bench 0.6 Test: Initial Create MB/s > Higher Is Better ZFS zraid1 4xNVME Pool . 222.04 |============================================== Compile Bench 0.6 Test: Read Compiled Tree MB/s > Higher Is Better ZFS zraid1 4xNVME Pool . 1212.73 |============================================= PostMark 1.51 Disk Transaction Performance TPS > Higher Is Better ZFS zraid1 4xNVME Pool . 3289 |================================================