2020-06-24-2159 no tuning, compression, or dedup. ashift=12 (last was 9) zfs_6x_ssd_in_2x_raidz1: Processor: 2 x Intel Xeon E5-2640 v3 @ 3.40GHz (16 Cores / 32 Threads), Motherboard: Dell 0CNCJW (2.11.0 BIOS), Chipset: Intel Xeon E7 v3/Xeon, Memory: 16 x GB DDR4-2133MT/s HMA42GR7MFR4N-TF, Disk: 6 x 481GB HSCP0480S5xnNMRI + 99GB PERC H730 Mini + 16GB Cruzer Blade + 16GB Cruzer Fit, Graphics: Matrox G200eR2, Monitor: DELL E196FP, Network: 2 x Intel 10-Gigabit X540-AT2 + 2 x Intel I350 + Mellanox MT27500 OS: CentOS Linux 8, Kernel: 4.18.0-193.6.3.el8_2.x86_64 (x86_64), Compiler: GCC 8.3.1 20191121, File-System: xfs, Screen Resolution: 1280x1024 zfs_6x_ssd_in_2x_raidz1_ashift=12: Processor: 2 x Intel Xeon E5-2640 v3 @ 3.40GHz (16 Cores / 32 Threads), Motherboard: Dell 0CNCJW (2.11.0 BIOS), Chipset: Intel Xeon E7 v3/Xeon, Memory: 16 x GB DDR4-2133MT/s HMA42GR7MFR4N-TF, Disk: 6 x 481GB HSCP0480S5xnNMRI + 99GB PERC H730 Mini + 16GB Cruzer Blade + 16GB Cruzer Fit, Graphics: Matrox G200eR2, Monitor: DELL E196FP, Network: 2 x Intel 10-Gigabit X540-AT2 + 2 x Intel I350 + Mellanox MT27500 OS: CentOS Linux 8, Kernel: 4.18.0-193.6.3.el8_2.x86_64 (x86_64), Compiler: GCC 8.3.1 20191121, File-System: xfs, Screen Resolution: 1280x1024 6 x 481GB HSCP0480S5xnNMRI: Processor: 2 x Intel Xeon E5-2640 v3 @ 3.40GHz (16 Cores / 32 Threads), Motherboard: Dell 0CNCJW (2.11.0 BIOS), Chipset: Intel Xeon E7 v3/Xeon, Memory: 16 x GB DDR4-2133MT/s HMA42GR7MFR4N-TF, Disk: 6 x 481GB HSCP0480S5xnNMRI + 99GB PERC H730 Mini + 16GB Cruzer Blade + 16GB Cruzer Fit, Graphics: Matrox G200eR2, Monitor: DELL E196FP, Network: 2 x Intel 10-Gigabit X540-AT2 + 2 x Intel I350 + Mellanox MT27500 OS: CentOS Linux 8, Kernel: 4.18.0-193.6.3.el8_2.x86_64 (x86_64), Compiler: GCC 8.3.1 20191121, File-System: xfs, Screen Resolution: 1280x1024 Flexible IO Tester 3.18 Type: Random Read - Engine: Sync - Buffered: No - Direct: Yes - Block Size: 4KB - Disk Target: /mnt/ztest1 MB/s > Higher Is Better zfs_6x_ssd_in_2x_raidz1 .... 946 |============================================= 6 x 481GB HSCP0480S5xnNMRI . 928 |============================================ Flexible IO Tester 3.18 Type: Random Read - Engine: Sync - Buffered: No - Direct: Yes - Block Size: 4KB - Disk Target: /mnt/ztest1 IOPS > Higher Is Better zfs_6x_ssd_in_2x_raidz1 .... 242133 |========================================== 6 x 481GB HSCP0480S5xnNMRI . 237467 |========================================= Flexible IO Tester 3.18 Type: Random Read - Engine: Sync - Buffered: No - Direct: Yes - Block Size: 16KB - Disk Target: /mnt/ztest1 MB/s > Higher Is Better zfs_6x_ssd_in_2x_raidz1 .... 2535 |============================================ 6 x 481GB HSCP0480S5xnNMRI . 2407 |========================================== Flexible IO Tester 3.18 Type: Random Read - Engine: Sync - Buffered: No - Direct: Yes - Block Size: 16KB - Disk Target: /mnt/ztest1 IOPS > Higher Is Better zfs_6x_ssd_in_2x_raidz1 .... 162267 |========================================== 6 x 481GB HSCP0480S5xnNMRI . 154333 |======================================== Flexible IO Tester 3.18 Type: Random Read - Engine: Sync - Buffered: No - Direct: Yes - Block Size: 64KB - Disk Target: /mnt/ztest1 MB/s > Higher Is Better zfs_6x_ssd_in_2x_raidz1 .... 4553 |============================================ 6 x 481GB HSCP0480S5xnNMRI . 4260 |========================================= Flexible IO Tester 3.18 Type: Random Read - Engine: Sync - Buffered: No - Direct: Yes - Block Size: 64KB - Disk Target: /mnt/ztest1 IOPS > Higher Is Better zfs_6x_ssd_in_2x_raidz1 .... 72773 |=========================================== 6 x 481GB HSCP0480S5xnNMRI . 68167 |======================================== Flexible IO Tester 3.18 Type: Random Write - Engine: Sync - Buffered: No - Direct: Yes - Block Size: 4KB - Disk Target: /mnt/ztest1 MB/s > Higher Is Better zfs_6x_ssd_in_2x_raidz1 .... 211 |========================================= 6 x 481GB HSCP0480S5xnNMRI . 231 |============================================= Flexible IO Tester 3.18 Type: Random Write - Engine: Sync - Buffered: No - Direct: Yes - Block Size: 4KB - Disk Target: /mnt/ztest1 IOPS > Higher Is Better zfs_6x_ssd_in_2x_raidz1 .... 54027 |======================================= 6 x 481GB HSCP0480S5xnNMRI . 58887 |=========================================== Flexible IO Tester 3.18 Type: Random Write - Engine: Sync - Buffered: No - Direct: Yes - Block Size: 16KB - Disk Target: /mnt/ztest1 MB/s > Higher Is Better zfs_6x_ssd_in_2x_raidz1 .... 718 |========================================== 6 x 481GB HSCP0480S5xnNMRI . 776 |============================================= Flexible IO Tester 3.18 Type: Random Write - Engine: Sync - Buffered: No - Direct: Yes - Block Size: 16KB - Disk Target: /mnt/ztest1 IOPS > Higher Is Better zfs_6x_ssd_in_2x_raidz1 .... 45667 |======================================== 6 x 481GB HSCP0480S5xnNMRI . 49520 |=========================================== Flexible IO Tester 3.18 Type: Random Write - Engine: Sync - Buffered: No - Direct: Yes - Block Size: 64KB - Disk Target: /mnt/ztest1 MB/s > Higher Is Better zfs_6x_ssd_in_2x_raidz1 .... 1879 |============================================ 6 x 481GB HSCP0480S5xnNMRI . 1865 |============================================ Flexible IO Tester 3.18 Type: Random Write - Engine: Sync - Buffered: No - Direct: Yes - Block Size: 64KB - Disk Target: /mnt/ztest1 IOPS > Higher Is Better zfs_6x_ssd_in_2x_raidz1 .... 30067 |=========================================== 6 x 481GB HSCP0480S5xnNMRI . 29867 |=========================================== Flexible IO Tester 3.18 Type: Sequential Read - Engine: Sync - Buffered: No - Direct: Yes - Block Size: 4KB - Disk Target: /mnt/ztest1 MB/s > Higher Is Better zfs_6x_ssd_in_2x_raidz1 .... 1396 |=========================================== 6 x 481GB HSCP0480S5xnNMRI . 1416 |============================================ Flexible IO Tester 3.18 Type: Sequential Read - Engine: Sync - Buffered: No - Direct: Yes - Block Size: 4KB - Disk Target: /mnt/ztest1 IOPS > Higher Is Better zfs_6x_ssd_in_2x_raidz1 .... 357667 |========================================= 6 x 481GB HSCP0480S5xnNMRI . 362667 |========================================== Flexible IO Tester 3.18 Type: Random Read - Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 4KB - Disk Target: /mnt/ztest1 MB/s > Higher Is Better zfs_6x_ssd_in_2x_raidz1 .... 872 |============================================= 6 x 481GB HSCP0480S5xnNMRI . 860 |============================================ Flexible IO Tester 3.18 Type: Random Read - Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 4KB - Disk Target: /mnt/ztest1 IOPS > Higher Is Better zfs_6x_ssd_in_2x_raidz1 .... 223333 |========================================== 6 x 481GB HSCP0480S5xnNMRI . 220000 |========================================= Flexible IO Tester 3.18 Type: Sequential Read - Engine: Sync - Buffered: No - Direct: Yes - Block Size: 16KB - Disk Target: /mnt/ztest1 MB/s > Higher Is Better zfs_6x_ssd_in_2x_raidz1 .... 3254 |============================================ 6 x 481GB HSCP0480S5xnNMRI . 3244 |============================================ Flexible IO Tester 3.18 Type: Sequential Read - Engine: Sync - Buffered: No - Direct: Yes - Block Size: 16KB - Disk Target: /mnt/ztest1 IOPS > Higher Is Better zfs_6x_ssd_in_2x_raidz1 .... 208333 |========================================== 6 x 481GB HSCP0480S5xnNMRI . 207667 |========================================== Flexible IO Tester 3.18 Type: Sequential Read - Engine: Sync - Buffered: No - Direct: Yes - Block Size: 64KB - Disk Target: /mnt/ztest1 MB/s > Higher Is Better zfs_6x_ssd_in_2x_raidz1 .... 4822 |=========================================== 6 x 481GB HSCP0480S5xnNMRI . 4897 |============================================ Flexible IO Tester 3.18 Type: Sequential Read - Engine: Sync - Buffered: No - Direct: Yes - Block Size: 64KB - Disk Target: /mnt/ztest1 IOPS > Higher Is Better zfs_6x_ssd_in_2x_raidz1 .... 77167 |========================================== 6 x 481GB HSCP0480S5xnNMRI . 78333 |=========================================== Flexible IO Tester 3.18 Type: Sequential Write - Engine: Sync - Buffered: No - Direct: Yes - Block Size: 4KB - Disk Target: /mnt/ztest1 MB/s > Higher Is Better zfs_6x_ssd_in_2x_raidz1 .... 700 |============================================= 6 x 481GB HSCP0480S5xnNMRI . 703 |============================================= Flexible IO Tester 3.18 Type: Sequential Write - Engine: Sync - Buffered: No - Direct: Yes - Block Size: 4KB - Disk Target: /mnt/ztest1 IOPS > Higher Is Better zfs_6x_ssd_in_2x_raidz1 .... 179333 |========================================== 6 x 481GB HSCP0480S5xnNMRI . 180333 |========================================== Flexible IO Tester 3.18 Type: Random Read - Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 16KB - Disk Target: /mnt/ztest1 MB/s > Higher Is Better zfs_6x_ssd_in_2x_raidz1 .... 2288 |============================================ 6 x 481GB HSCP0480S5xnNMRI . 2283 |============================================ Flexible IO Tester 3.18 Type: Random Read - Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 16KB - Disk Target: /mnt/ztest1 IOPS > Higher Is Better zfs_6x_ssd_in_2x_raidz1 .... 146667 |========================================== 6 x 481GB HSCP0480S5xnNMRI . 146000 |========================================== Flexible IO Tester 3.18 Type: Random Read - Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 64KB - Disk Target: /mnt/ztest1 MB/s > Higher Is Better zfs_6x_ssd_in_2x_raidz1 .... 4092 |=========================================== 6 x 481GB HSCP0480S5xnNMRI . 4153 |============================================ Flexible IO Tester 3.18 Type: Random Read - Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 64KB - Disk Target: /mnt/ztest1 IOPS > Higher Is Better zfs_6x_ssd_in_2x_raidz1 .... 65450 |========================================== 6 x 481GB HSCP0480S5xnNMRI . 66256 |=========================================== Flexible IO Tester 3.18 Type: Random Write - Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 4KB - Disk Target: /mnt/ztest1 MB/s > Higher Is Better zfs_6x_ssd_in_2x_raidz1 .... 212 |============================================ 6 x 481GB HSCP0480S5xnNMRI . 218 |============================================= Flexible IO Tester 3.18 Type: Random Write - Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 4KB - Disk Target: /mnt/ztest1 IOPS > Higher Is Better zfs_6x_ssd_in_2x_raidz1 .... 54133 |========================================== 6 x 481GB HSCP0480S5xnNMRI . 55667 |=========================================== Flexible IO Tester 3.18 Type: Sequential Write - Engine: Sync - Buffered: No - Direct: Yes - Block Size: 16KB - Disk Target: /mnt/ztest1 MB/s > Higher Is Better zfs_6x_ssd_in_2x_raidz1 .... 1625 |============================================ 6 x 481GB HSCP0480S5xnNMRI . 1581 |=========================================== Flexible IO Tester 3.18 Type: Sequential Write - Engine: Sync - Buffered: No - Direct: Yes - Block Size: 16KB - Disk Target: /mnt/ztest1 IOPS > Higher Is Better zfs_6x_ssd_in_2x_raidz1 .... 104000 |========================================== 6 x 481GB HSCP0480S5xnNMRI . 101150 |========================================= Flexible IO Tester 3.18 Type: Sequential Write - Engine: Sync - Buffered: No - Direct: Yes - Block Size: 64KB - Disk Target: /mnt/ztest1 MB/s > Higher Is Better zfs_6x_ssd_in_2x_raidz1 .... 2106 |============================================ 6 x 481GB HSCP0480S5xnNMRI . 2097 |============================================ Flexible IO Tester 3.18 Type: Sequential Write - Engine: Sync - Buffered: No - Direct: Yes - Block Size: 64KB - Disk Target: /mnt/ztest1 IOPS > Higher Is Better zfs_6x_ssd_in_2x_raidz1 .... 33475 |=========================================== 6 x 481GB HSCP0480S5xnNMRI . 33567 |=========================================== Flexible IO Tester 3.18 Type: Random Write - Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 16KB - Disk Target: /mnt/ztest1 MB/s > Higher Is Better zfs_6x_ssd_in_2x_raidz1 .... 662 |========================================= 6 x 481GB HSCP0480S5xnNMRI . 726 |============================================= Flexible IO Tester 3.18 Type: Random Write - Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 16KB - Disk Target: /mnt/ztest1 IOPS > Higher Is Better zfs_6x_ssd_in_2x_raidz1 .... 42225 |======================================= 6 x 481GB HSCP0480S5xnNMRI . 46364 |=========================================== Flexible IO Tester 3.18 Type: Random Write - Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 64KB - Disk Target: /mnt/ztest1 MB/s > Higher Is Better zfs_6x_ssd_in_2x_raidz1 .... 1611 |============================================ 6 x 481GB HSCP0480S5xnNMRI . 1613 |============================================ Flexible IO Tester 3.18 Type: Random Write - Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 64KB - Disk Target: /mnt/ztest1 IOPS > Higher Is Better zfs_6x_ssd_in_2x_raidz1 .... 25707 |=========================================== 6 x 481GB HSCP0480S5xnNMRI . 25800 |=========================================== Flexible IO Tester 3.18 Type: Sequential Read - Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 4KB - Disk Target: /mnt/ztest1 MB/s > Higher Is Better zfs_6x_ssd_in_2x_raidz1 .... 1086 |=========================================== 6 x 481GB HSCP0480S5xnNMRI . 1117 |============================================ Flexible IO Tester 3.18 Type: Sequential Read - Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 4KB - Disk Target: /mnt/ztest1 IOPS > Higher Is Better zfs_6x_ssd_in_2x_raidz1 .... 278333 |========================================= 6 x 481GB HSCP0480S5xnNMRI . 286333 |========================================== Flexible IO Tester 3.18 Type: Sequential Read - Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 16KB - Disk Target: /mnt/ztest1 MB/s > Higher Is Better zfs_6x_ssd_in_2x_raidz1 .... 2753 |=========================================== 6 x 481GB HSCP0480S5xnNMRI . 2820 |============================================ Flexible IO Tester 3.18 Type: Sequential Read - Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 16KB - Disk Target: /mnt/ztest1 IOPS > Higher Is Better zfs_6x_ssd_in_2x_raidz1 .... 176000 |========================================= 6 x 481GB HSCP0480S5xnNMRI . 180667 |========================================== Flexible IO Tester 3.18 Type: Sequential Read - Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 64KB - Disk Target: /mnt/ztest1 MB/s > Higher Is Better zfs_6x_ssd_in_2x_raidz1 .... 4385 |=========================================== 6 x 481GB HSCP0480S5xnNMRI . 4495 |============================================ Flexible IO Tester 3.18 Type: Sequential Read - Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 64KB - Disk Target: /mnt/ztest1 IOPS > Higher Is Better zfs_6x_ssd_in_2x_raidz1 .... 70167 |========================================== 6 x 481GB HSCP0480S5xnNMRI . 71900 |=========================================== Flexible IO Tester 3.18 Type: Sequential Write - Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 4KB - Disk Target: /mnt/ztest1 MB/s > Higher Is Better zfs_6x_ssd_in_2x_raidz1 .... 615 |============================================ 6 x 481GB HSCP0480S5xnNMRI . 628 |============================================= Flexible IO Tester 3.18 Type: Sequential Write - Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 4KB - Disk Target: /mnt/ztest1 IOPS > Higher Is Better zfs_6x_ssd_in_2x_raidz1 .... 157333 |========================================= 6 x 481GB HSCP0480S5xnNMRI . 161000 |========================================== Flexible IO Tester 3.18 Type: Sequential Write - Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 16KB - Disk Target: /mnt/ztest1 MB/s > Higher Is Better zfs_6x_ssd_in_2x_raidz1 .... 1423 |=========================================== 6 x 481GB HSCP0480S5xnNMRI . 1441 |============================================ Flexible IO Tester 3.18 Type: Sequential Write - Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 16KB - Disk Target: /mnt/ztest1 IOPS > Higher Is Better zfs_6x_ssd_in_2x_raidz1 .... 91067 |========================================== 6 x 481GB HSCP0480S5xnNMRI . 92233 |=========================================== Flexible IO Tester 3.18 Type: Sequential Write - Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 64KB - Disk Target: /mnt/ztest1 MB/s > Higher Is Better zfs_6x_ssd_in_2x_raidz1 .... 1811 |====================================== 6 x 481GB HSCP0480S5xnNMRI . 2072 |============================================ Flexible IO Tester 3.18 Type: Sequential Write - Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 64KB - Disk Target: /mnt/ztest1 IOPS > Higher Is Better zfs_6x_ssd_in_2x_raidz1 .... 28967 |====================================== 6 x 481GB HSCP0480S5xnNMRI . 33167 |===========================================