dbox 2 x Intel Xeon L5640 testing with a Supermicro X8DTH and Matrox s MGA G200eW WPCM450 on Ubuntu 16.04 via the Phoronix Test Suite. 3sas_zfs_opts: Processor: 2 x Intel Xeon L5640 @ 2.27GHz (24 Cores), Motherboard: Supermicro X8DTH, Chipset: Intel 5520 I/O + ICH10R, Memory: 80896MB, Disk: 500GB Western Digital WD5000ABYS-0 + 500GB Seagate ST9500420ASG + 2 x 1000GB Seagate ST1000LM014-1EJ1 + 4 x 4001GB TOSHIBA HDWQ140 + 1000GB Western Digital WD10EZEX-00B + 2 x 4001GB HGST HDN726040AL + 64GB Type-C + 8GB Cruzer Blade + 3 x 1000GB Seagate ST31000528AS + Samsung SSD 950 PRO 512GB, Graphics: Matrox s MGA G200eW WPCM450, Network: Intel 82576 Gigabit Connection + Mellanox MT27500 Family OS: Ubuntu 16.04, Kernel: 4.10.0-35-lowlatency (x86_64), Compiler: GCC 5.4.0 20160609, File-System: zfs, Screen Resolution: 1024x768 3sas_zfs_opts_all_tests: Processor: 2 x Intel Xeon L5640 @ 2.27GHz (24 Cores), Motherboard: Supermicro X8DTH, Chipset: Intel 5520 I/O + ICH10R, Memory: 80896MB, Disk: 500GB Western Digital WD5000ABYS-0 + 500GB Seagate ST9500420ASG + 2 x 1000GB Seagate ST1000LM014-1EJ1 + 4 x 4001GB TOSHIBA HDWQ140 + 1000GB Western Digital WD10EZEX-00B + 2 x 4001GB HGST HDN726040AL + 64GB Type-C + 8GB Cruzer Blade + 3 x 1000GB Seagate ST31000528AS + Samsung SSD 950 PRO 512GB, Graphics: Matrox s MGA G200eW WPCM450, Network: Intel 82576 Gigabit Connection + Mellanox MT27500 Family OS: Ubuntu 16.04, Kernel: 4.10.0-35-lowlatency (x86_64), Compiler: GCC 5.4.0 20160609, File-System: zfs, Screen Resolution: 1024x768 3sas_zfs_z2_txg_inflight_32: Processor: 2 x Intel Xeon L5640 @ 2.27GHz (24 Cores), Motherboard: Supermicro X8DTH, Chipset: Intel 5520 I/O + ICH10R, Memory: 80896MB, Disk: 500GB Western Digital WD5000ABYS-0 + 500GB Seagate ST9500420ASG + 2 x 1000GB Seagate ST1000LM014-1EJ1 + 4 x 4001GB TOSHIBA HDWQ140 + 1000GB Western Digital WD10EZEX-00B + 2 x 4001GB HGST HDN726040AL + 64GB Type-C + 8GB Cruzer Blade + 3 x 1000GB Seagate ST31000528AS + Samsung SSD 950 PRO 512GB, Graphics: Matrox s MGA G200eW WPCM450, Network: Intel 82576 Gigabit Connection + Mellanox MT27500 Family OS: Ubuntu 16.04, Kernel: 4.10.0-35-lowlatency (x86_64), Compiler: GCC 5.4.0 20160609, File-System: zfs, Screen Resolution: 1024x768 3sas_zfs_z2_txg_128_volthrds_64: Processor: 2 x Intel Xeon L5640 @ 2.27GHz (24 Cores), Motherboard: Supermicro X8DTH, Chipset: Intel 5520 I/O + ICH10R, Memory: 80896MB, Disk: 500GB Western Digital WD5000ABYS-0 + 500GB Seagate ST9500420ASG + 2 x 1000GB Seagate ST1000LM014-1EJ1 + 4 x 4001GB TOSHIBA HDWQ140 + 1000GB Western Digital WD10EZEX-00B + 2 x 4001GB HGST HDN726040AL + 64GB Type-C + 8GB Cruzer Blade + 3 x 1000GB Seagate ST31000528AS + Samsung SSD 950 PRO 512GB, Graphics: Matrox s MGA G200eW WPCM450, Network: Intel 82576 Gigabit Connection + Mellanox MT27500 Family OS: Ubuntu 16.04, Kernel: 4.10.0-35-lowlatency (x86_64), Compiler: GCC 5.4.0 20160609, File-System: zfs, Screen Resolution: 1024x768 1sas_exp_zfs_z2_txg_128_volthrds_64: Processor: 2 x Intel Xeon L5640 @ 2.27GHz (24 Cores), Motherboard: Supermicro X8DTH, Chipset: Intel 5520 I/O + ICH10R, Memory: 80896MB, Disk: 8GB Cruzer Blade + 64GB Type-C + 3 x 1000GB Seagate ST31000528AS + 2 x 1000GB Seagate ST1000LM014-1EJ1 + 4 x 4001GB TOSHIBA HDWQ140 + 500GB Western Digital WD5000ABYS-0 + 500GB Seagate ST9500420ASG + 1000GB Western Digital WD10EZEX-00B + 2 x 4001GB HGST HDN726040AL + Samsung SSD 950 PRO 512GB, Graphics: Matrox s MGA G200eW WPCM450, Network: Intel 82576 Gigabit Connection + Mellanox MT27500 Family OS: Ubuntu 16.04, Kernel: 4.10.0-35-lowlatency (x86_64), Compiler: GCC 5.4.0 20160609, File-System: zfs, Screen Resolution: 1024x768 1sas_exp_zfs_z2_prefetch_rw_2_other_opts: Processor: 2 x Intel Xeon L5640 @ 2.27GHz (24 Cores), Motherboard: Supermicro X8DTH, Chipset: Intel 5520 I/O + ICH10R, Memory: 80896MB, Disk: 8GB Cruzer Blade + 64GB Type-C + 3 x 1000GB Seagate ST31000528AS + 2 x 1000GB Seagate ST1000LM014-1EJ1 + 4 x 4001GB TOSHIBA HDWQ140 + 500GB Western Digital WD5000ABYS-0 + 500GB Seagate ST9500420ASG + 1000GB Western Digital WD10EZEX-00B + 2 x 4001GB HGST HDN726040AL + Samsung SSD 950 PRO 512GB, Graphics: Matrox s MGA G200eW WPCM450, Network: Intel 82576 Gigabit Connection + Mellanox MT27500 Family OS: Ubuntu 16.04, Kernel: 4.10.0-35-lowlatency (x86_64), Compiler: GCC 5.4.0 20160609, File-System: zfs, Screen Resolution: 1024x768 sync_off_1sas_zfs_z2_prefetch_rw_2_other: Processor: 2 x Intel Xeon L5640 @ 2.27GHz (24 Cores), Motherboard: Supermicro X8DTH, Chipset: Intel 5520 I/O + ICH10R, Memory: 80896MB, Disk: 8GB Cruzer Blade + 64GB Type-C + 3 x 1000GB Seagate ST31000528AS + 2 x 1000GB Seagate ST1000LM014-1EJ1 + 4 x 4001GB TOSHIBA HDWQ140 + 500GB Western Digital WD5000ABYS-0 + 500GB Seagate ST9500420ASG + 1000GB Western Digital WD10EZEX-00B + 2 x 4001GB HGST HDN726040AL + Samsung SSD 950 PRO 512GB, Graphics: Matrox s MGA G200eW WPCM450, Network: Intel 82576 Gigabit Connection + Mellanox MT27500 Family OS: Ubuntu 16.04, Kernel: 4.10.0-35-lowlatency (x86_64), Compiler: GCC 5.4.0 20160609, File-System: zfs, Screen Resolution: 1024x768 sync_dva_1sas_z2_128_inflight_64_thrds: Processor: 2 x Intel Xeon L5640 @ 2.27GHz (24 Cores), Motherboard: Supermicro X8DTH, Chipset: Intel 5520 I/O + ICH10R, Memory: 80896MB, Disk: 8GB Cruzer Blade + 64GB Type-C + 3 x 1000GB Seagate ST31000528AS + 2 x 1000GB Seagate ST1000LM014-1EJ1 + 4 x 4001GB TOSHIBA HDWQ140 + 500GB Western Digital WD5000ABYS-0 + 500GB Seagate ST9500420ASG + 1000GB Western Digital WD10EZEX-00B + 2 x 4001GB HGST HDN726040AL + Samsung SSD 950 PRO 512GB, Graphics: Matrox s MGA G200eW WPCM450, Network: Intel 82576 Gigabit Connection + Mellanox MT27500 Family OS: Ubuntu 16.04, Kernel: 4.10.0-35-lowlatency (x86_64), Compiler: GCC 5.4.0 20160609, File-System: zfs, Screen Resolution: 1024x768 1sas_all_opts: Processor: 2 x Intel Xeon L5640 @ 2.27GHz (24 Cores), Motherboard: Supermicro X8DTH, Chipset: Intel 5520 I/O + ICH10R, Memory: 80896MB, Disk: 8GB Cruzer Blade + 64GB Type-C + 3 x 1000GB Seagate ST31000528AS + 2 x 1000GB Seagate ST1000LM014-1EJ1 + 4 x 4001GB TOSHIBA HDWQ140 + 500GB Western Digital WD5000ABYS-0 + 500GB Seagate ST9500420ASG + 1000GB Western Digital WD10EZEX-00B + 2 x 4001GB HGST HDN726040AL + Samsung SSD 950 PRO 512GB, Graphics: Matrox s MGA G200eW WPCM450, Network: Intel 82576 Gigabit Connection + Mellanox MT27500 Family OS: Ubuntu 16.04, Kernel: 4.10.0-35-lowlatency (x86_64), Compiler: GCC 5.4.0 20160609, File-System: zfs, Screen Resolution: 1024x768 1sas_all_opts_run2: Processor: 2 x Intel Xeon L5640 @ 2.27GHz (24 Cores), Motherboard: Supermicro X8DTH, Chipset: Intel 5520 I/O + ICH10R, Memory: 80896MB, Disk: 8GB Cruzer Blade + 64GB Type-C + 3 x 1000GB Seagate ST31000528AS + 2 x 1000GB Seagate ST1000LM014-1EJ1 + 4 x 4001GB TOSHIBA HDWQ140 + 500GB Western Digital WD5000ABYS-0 + 500GB Seagate ST9500420ASG + 1000GB Western Digital WD10EZEX-00B + 2 x 4001GB HGST HDN726040AL + Samsung SSD 950 PRO 512GB, Graphics: Matrox s MGA G200eW WPCM450, Network: Intel 82576 Gigabit Connection + Mellanox MT27500 Family OS: Ubuntu 16.04, Kernel: 4.10.0-35-lowlatency (x86_64), Compiler: GCC 5.4.0 20160609, File-System: zfs, Screen Resolution: 1024x768 1sas_all_deadline: Processor: 2 x Intel Xeon L5640 @ 2.27GHz (24 Cores), Motherboard: Supermicro X8DTH, Chipset: Intel 5520 I/O + ICH10R, Memory: 80896MB, Disk: 8GB Cruzer Blade + 64GB Type-C + 3 x 1000GB Seagate ST31000528AS + 2 x 1000GB Seagate ST1000LM014-1EJ1 + 4 x 4001GB TOSHIBA HDWQ140 + 500GB Western Digital WD5000ABYS-0 + 500GB Seagate ST9500420ASG + 1000GB Western Digital WD10EZEX-00B + 2 x 4001GB HGST HDN726040AL + Samsung SSD 950 PRO 512GB, Graphics: Matrox s MGA G200eW WPCM450, Network: Intel 82576 Gigabit Connection + Mellanox MT27500 Family OS: Ubuntu 16.04, Kernel: 4.10.0-35-lowlatency (x86_64), Compiler: GCC 5.4.0 20160609, File-System: zfs, Screen Resolution: 1024x768 1sas_all_opts_deadline: Processor: 2 x Intel Xeon L5640 @ 2.27GHz (24 Cores), Motherboard: Supermicro X8DTH, Chipset: Intel 5520 I/O + ICH10R, Memory: 80896MB, Disk: 8GB Cruzer Blade + 64GB Type-C + 3 x 1000GB Seagate ST31000528AS + 2 x 1000GB Seagate ST1000LM014-1EJ1 + 4 x 4001GB TOSHIBA HDWQ140 + 500GB Western Digital WD5000ABYS-0 + 500GB Seagate ST9500420ASG + 1000GB Western Digital WD10EZEX-00B + 2 x 4001GB HGST HDN726040AL + Samsung SSD 950 PRO 512GB, Graphics: Matrox s MGA G200eW WPCM450, Network: Intel 82576 Gigabit Connection + Mellanox MT27500 Family OS: Ubuntu 16.04, Kernel: 4.10.0-35-lowlatency (x86_64), Compiler: GCC 5.4.0 20160609, File-System: zfs, Screen Resolution: 1024x768 1sas_all_noop_nr_requests_1024: Processor: 2 x Intel Xeon L5640 @ 2.27GHz (24 Cores), Motherboard: Supermicro X8DTH, Chipset: Intel 5520 I/O + ICH10R, Memory: 80896MB, Disk: 8GB Cruzer Blade + 64GB Type-C + 3 x 1000GB Seagate ST31000528AS + 2 x 1000GB Seagate ST1000LM014-1EJ1 + 4 x 4001GB TOSHIBA HDWQ140 + 500GB Western Digital WD5000ABYS-0 + 500GB Seagate ST9500420ASG + 1000GB Western Digital WD10EZEX-00B + 2 x 4001GB HGST HDN726040AL + Samsung SSD 950 PRO 512GB, Graphics: Matrox s MGA G200eW WPCM450, Network: Intel 82576 Gigabit Connection + Mellanox MT27500 Family OS: Ubuntu 16.04, Kernel: 4.10.0-35-lowlatency (x86_64), Compiler: GCC 5.4.0 20160609, File-System: zfs, Screen Resolution: 1024x768 1sas_all_noop_nr_requests_1024_48_thrds_tx_48: Processor: 2 x Intel Xeon L5640 @ 2.27GHz (24 Cores), Motherboard: Supermicro X8DTH, Chipset: Intel 5520 I/O + ICH10R, Memory: 80896MB, Disk: 8GB Cruzer Blade + 64GB Type-C + 3 x 1000GB Seagate ST31000528AS + 2 x 1000GB Seagate ST1000LM014-1EJ1 + 4 x 4001GB TOSHIBA HDWQ140 + 500GB Western Digital WD5000ABYS-0 + 500GB Seagate ST9500420ASG + 1000GB Western Digital WD10EZEX-00B + 2 x 4001GB HGST HDN726040AL + Samsung SSD 950 PRO 512GB, Graphics: Matrox s MGA G200eW WPCM450, Network: Intel 82576 Gigabit Connection + Mellanox MT27500 Family OS: Ubuntu 16.04, Kernel: 4.10.0-35-lowlatency (x86_64), Compiler: GCC 5.4.0 20160609, File-System: zfs, Screen Resolution: 1024x768 1sas_all_noop_nr_requests_1024_deadline_32_threads: Processor: 2 x Intel Xeon L5640 @ 2.27GHz (24 Cores), Motherboard: Supermicro X8DTH, Chipset: Intel 5520 I/O + ICH10R, Memory: 80896MB, Disk: 8GB Cruzer Blade + 64GB Type-C + 3 x 1000GB Seagate ST31000528AS + 2 x 1000GB Seagate ST1000LM014-1EJ1 + 4 x 4001GB TOSHIBA HDWQ140 + 500GB Western Digital WD5000ABYS-0 + 500GB Seagate ST9500420ASG + 1000GB Western Digital WD10EZEX-00B + 2 x 4001GB HGST HDN726040AL + Samsung SSD 950 PRO 512GB, Graphics: Matrox s MGA G200eW WPCM450, Network: Intel 82576 Gigabit Connection + Mellanox MT27500 Family OS: Ubuntu 16.04, Kernel: 4.10.0-35-lowlatency (x86_64), Compiler: GCC 5.4.0 20160609, File-System: zfs, Screen Resolution: 1024x768 1sas_nr_1024_txg_128_64_thrds: Processor: 2 x Intel Xeon L5640 @ 2.27GHz (24 Cores), Motherboard: Supermicro X8DTH, Chipset: Intel 5520 I/O + ICH10R, Memory: 80896MB, Disk: 8GB Cruzer Blade + 64GB Type-C + 3 x 1000GB Seagate ST31000528AS + 2 x 1000GB Seagate ST1000LM014-1EJ1 + 4 x 4001GB TOSHIBA HDWQ140 + 500GB Western Digital WD5000ABYS-0 + 500GB Seagate ST9500420ASG + 1000GB Western Digital WD10EZEX-00B + 2 x 4001GB HGST HDN726040AL + Samsung SSD 950 PRO 512GB, Graphics: Matrox s MGA G200eW WPCM450, Network: Intel 82576 Gigabit Connection + Mellanox MT27500 Family OS: Ubuntu 16.04, Kernel: 4.10.0-35-lowlatency (x86_64), Compiler: GCC 5.4.0 20160609, File-System: zfs, Screen Resolution: 1024x768 1sas_nr_1024_txg_48_48_thrds: Processor: 2 x Intel Xeon L5640 @ 2.27GHz (24 Cores), Motherboard: Supermicro X8DTH, Chipset: Intel 5520 I/O + ICH10R, Memory: 80896MB, Disk: 8GB Cruzer Blade + 64GB Type-C + 3 x 1000GB Seagate ST31000528AS + 2 x 1000GB Seagate ST1000LM014-1EJ1 + 4 x 4001GB TOSHIBA HDWQ140 + 500GB Western Digital WD5000ABYS-0 + 500GB Seagate ST9500420ASG + 1000GB Western Digital WD10EZEX-00B + 2 x 4001GB HGST HDN726040AL + Samsung SSD 950 PRO 512GB, Graphics: Matrox s MGA G200eW WPCM450, Network: Intel 82576 Gigabit Connection + Mellanox MT27500 Family OS: Ubuntu 16.04, Kernel: 4.10.0-35-lowlatency (x86_64), Compiler: GCC 5.4.0 20160609, File-System: zfs, Screen Resolution: 1024x768 1sas_nr_1024_txg_32_32_tx_thrds: Processor: 2 x Intel Xeon L5640 @ 2.27GHz (24 Cores), Motherboard: Supermicro X8DTH, Chipset: Intel 5520 I/O + ICH10R, Memory: 80896MB, Disk: 8GB Cruzer Blade + 64GB Type-C + 3 x 1000GB Seagate ST31000528AS + 2 x 1000GB Seagate ST1000LM014-1EJ1 + 4 x 4001GB TOSHIBA HDWQ140 + 500GB Western Digital WD5000ABYS-0 + 500GB Seagate ST9500420ASG + 1000GB Western Digital WD10EZEX-00B + 2 x 4001GB HGST HDN726040AL + Samsung SSD 950 PRO 512GB, Graphics: Matrox s MGA G200eW WPCM450, Network: Intel 82576 Gigabit Connection + Mellanox MT27500 Family OS: Ubuntu 16.04, Kernel: 4.10.0-35-lowlatency (x86_64), Compiler: GCC 5.4.0 20160609, File-System: zfs, Screen Resolution: 1024x768 1sas_nr_1024_txg_64_64_noop: Processor: 2 x Intel Xeon L5640 @ 2.27GHz (24 Cores), Motherboard: Supermicro X8DTH, Chipset: Intel 5520 I/O + ICH10R, Memory: 80896MB, Disk: 8GB Cruzer Blade + 64GB Type-C + 3 x 1000GB Seagate ST31000528AS + 2 x 1000GB Seagate ST1000LM014-1EJ1 + 4 x 4001GB TOSHIBA HDWQ140 + 500GB Western Digital WD5000ABYS-0 + 500GB Seagate ST9500420ASG + 1000GB Western Digital WD10EZEX-00B + 2 x 4001GB HGST HDN726040AL + Samsung SSD 950 PRO 512GB, Graphics: Matrox s MGA G200eW WPCM450, Network: Intel 82576 Gigabit Connection + Mellanox MT27500 Family OS: Ubuntu 16.04, Kernel: 4.10.0-35-lowlatency (x86_64), Compiler: GCC 5.4.0 20160609, File-System: zfs, Screen Resolution: 1024x768 raidz3: Processor: 2 x Intel Xeon L5640 @ 2.27GHz (24 Cores), Motherboard: Supermicro X8DTH, Chipset: Intel 5520 I/O + ICH10R, Memory: 80896MB, Disk: 8GB Cruzer Blade + 64GB Type-C + 3 x 1000GB Seagate ST31000528AS + 2 x 1000GB Seagate ST1000LM014-1EJ1 + 4 x 4001GB TOSHIBA HDWQ140 + 500GB Western Digital WD5000ABYS-0 + 500GB Seagate ST9500420ASG + 1000GB Western Digital WD10EZEX-00B + 2 x 4001GB HGST HDN726040AL + Samsung SSD 950 PRO 512GB, Graphics: Matrox s MGA G200eW WPCM450, Network: Intel 82576 Gigabit Connection + Mellanox MT27500 Family OS: Ubuntu 16.04, Kernel: 4.10.0-35-lowlatency (x86_64), Compiler: GCC 5.4.0 20160609, File-System: zfs, Screen Resolution: 1024x768 1sas_nr_1024_32_32_noop: Processor: 2 x Intel Xeon L5640 @ 2.27GHz (24 Cores), Motherboard: Supermicro X8DTH, Chipset: Intel 5520 I/O + ICH10R, Memory: 80896MB, Disk: 8GB Cruzer Blade + 64GB Type-C + 3 x 1000GB Seagate ST31000528AS + 2 x 1000GB Seagate ST1000LM014-1EJ1 + 4 x 4001GB TOSHIBA HDWQ140 + 500GB Western Digital WD5000ABYS-0 + 500GB Seagate ST9500420ASG + 1000GB Western Digital WD10EZEX-00B + 2 x 4001GB HGST HDN726040AL + Samsung SSD 950 PRO 512GB, Graphics: Matrox s MGA G200eW WPCM450, Network: Intel 82576 Gigabit Connection + Mellanox MT27500 Family OS: Ubuntu 16.04, Kernel: 4.10.0-35-lowlatency (x86_64), Compiler: GCC 5.4.0 20160609, File-System: zfs, Screen Resolution: 1024x768 1sas_1024_128_32_noop: Processor: 2 x Intel Xeon L5640 @ 2.27GHz (24 Cores), Motherboard: Supermicro X8DTH, Chipset: Intel 5520 I/O + ICH10R, Memory: 80896MB, Disk: 8GB Cruzer Blade + 64GB Type-C + 3 x 1000GB Seagate ST31000528AS + 2 x 1000GB Seagate ST1000LM014-1EJ1 + 4 x 4001GB TOSHIBA HDWQ140 + 500GB Western Digital WD5000ABYS-0 + 500GB Seagate ST9500420ASG + 1000GB Western Digital WD10EZEX-00B + 2 x 4001GB HGST HDN726040AL + Samsung SSD 950 PRO 512GB, Graphics: Matrox s MGA G200eW WPCM450, Network: Intel 82576 Gigabit Connection + Mellanox MT27500 Family OS: Ubuntu 16.04, Kernel: 4.10.0-35-lowlatency (x86_64), Compiler: GCC 5.4.0 20160609, File-System: zfs, Screen Resolution: 1024x768 1sas_1024_256_noop: Processor: 2 x Intel Xeon L5640 @ 2.27GHz (24 Cores), Motherboard: Supermicro X8DTH, Chipset: Intel 5520 I/O + ICH10R, Memory: 80896MB, Disk: 8GB Cruzer Blade + 64GB Type-C + 3 x 1000GB Seagate ST31000528AS + 2 x 1000GB Seagate ST1000LM014-1EJ1 + 4 x 4001GB TOSHIBA HDWQ140 + 500GB Western Digital WD5000ABYS-0 + 500GB Seagate ST9500420ASG + 1000GB Western Digital WD10EZEX-00B + 2 x 4001GB HGST HDN726040AL + Samsung SSD 950 PRO 512GB, Graphics: Matrox s MGA G200eW WPCM450, Network: Intel 82576 Gigabit Connection + Mellanox MT27500 Family OS: Ubuntu 16.04, Kernel: 4.10.0-35-lowlatency (x86_64), Compiler: GCC 5.4.0 20160609, File-System: zfs, Screen Resolution: 1024x768 1sas_1024_deadline_64_64: Processor: 2 x Intel Xeon L5640 @ 2.27GHz (24 Cores), Motherboard: Supermicro X8DTH, Chipset: Intel 5520 I/O + ICH10R, Memory: 80896MB, Disk: 8GB Cruzer Blade + 64GB Type-C + 3 x 1000GB Seagate ST31000528AS + 2 x 1000GB Seagate ST1000LM014-1EJ1 + 4 x 4001GB TOSHIBA HDWQ140 + 500GB Western Digital WD5000ABYS-0 + 500GB Seagate ST9500420ASG + 1000GB Western Digital WD10EZEX-00B + 2 x 4001GB HGST HDN726040AL + Samsung SSD 950 PRO 512GB, Graphics: Matrox s MGA G200eW WPCM450, Network: Intel 82576 Gigabit Connection + Mellanox MT27500 Family OS: Ubuntu 16.04, Kernel: 4.10.0-35-lowlatency (x86_64), Compiler: GCC 5.4.0 20160609, File-System: zfs, Screen Resolution: 1024x768 1sas_128_noop_64_64: Processor: 2 x Intel Xeon L5640 @ 2.27GHz (24 Cores), Motherboard: Supermicro X8DTH, Chipset: Intel 5520 I/O + ICH10R, Memory: 80896MB, Disk: 8GB Cruzer Blade + 64GB Type-C + 3 x 1000GB Seagate ST31000528AS + 2 x 1000GB Seagate ST1000LM014-1EJ1 + 4 x 4001GB TOSHIBA HDWQ140 + 500GB Western Digital WD5000ABYS-0 + 500GB Seagate ST9500420ASG + 1000GB Western Digital WD10EZEX-00B + 2 x 4001GB HGST HDN726040AL + Samsung SSD 950 PRO 512GB, Graphics: Matrox s MGA G200eW WPCM450, Network: Intel 82576 Gigabit Connection + Mellanox MT27500 Family OS: Ubuntu 16.04, Kernel: 4.10.0-35-lowlatency (x86_64), Compiler: GCC 5.4.0 20160609, File-System: zfs, Screen Resolution: 1024x768 1sas_128_noop_64_64_run2: Processor: 2 x Intel Xeon L5640 @ 2.27GHz (24 Cores), Motherboard: Supermicro X8DTH, Chipset: Intel 5520 I/O + ICH10R, Memory: 80896MB, Disk: 8GB Cruzer Blade + 64GB Type-C + 3 x 1000GB Seagate ST31000528AS + 2 x 1000GB Seagate ST1000LM014-1EJ1 + 4 x 4001GB TOSHIBA HDWQ140 + 500GB Western Digital WD5000ABYS-0 + 500GB Seagate ST9500420ASG + 1000GB Western Digital WD10EZEX-00B + 2 x 4001GB HGST HDN726040AL + Samsung SSD 950 PRO 512GB, Graphics: Matrox s MGA G200eW WPCM450, Network: Intel 82576 Gigabit Connection + Mellanox MT27500 Family OS: Ubuntu 16.04, Kernel: 4.10.0-35-lowlatency (x86_64), Compiler: GCC 5.4.0 20160609, File-System: zfs, Screen Resolution: 1024x768 zfs_raidz2_x3stripe_18_disks: Processor: 2 x Intel Xeon L5640 @ 2.27GHz (24 Cores), Motherboard: Supermicro X8DTH, Chipset: Intel 5520 I/O + ICH10R, Memory: 80896MB, Disk: 64GB Type-C + 8GB Cruzer Blade + 4 x 1000GB Western Digital WD1004FBYZ-0 + 2 x 1000GB Western Digital WD1005FBYZ-0 + 500GB Western Digital WD5000ABYS-0 + 500GB Seagate ST9500420ASG + 2 x 1000GB Seagate ST1000LM014-1EJ1 + 4 x 4001GB TOSHIBA HDWQ140 + 2 x 4001GB HGST HDN726040AL + 1000GB Western Digital WD10EZEX-00B + 3 x 1000GB Seagate ST31000528AS + Samsung SSD 950 PRO 512GB, Graphics: Matrox s MGA G200eW WPCM450, Network: Intel 82576 Gigabit Connection + Mellanox MT27500 Family OS: Ubuntu 16.04, Kernel: 4.10.0-35-lowlatency (x86_64), Compiler: GCC 5.4.0 20160609, File-System: zfs, Screen Resolution: 1024x768 IOzone 3.465 Record Size: 1MB - File Size: 2GB - Disk Test: Read Performance MB/s > Higher Is Better 3sas_zfs_opts ...................................... 2801.53 |================ 3sas_zfs_opts_all_tests ............................ 2796.52 |================ 3sas_zfs_z2_txg_inflight_32 ........................ 2797.91 |================ 3sas_zfs_z2_txg_128_volthrds_64 .................... 2750.93 |================ 1sas_exp_zfs_z2_txg_128_volthrds_64 ................ 2751.86 |================ 1sas_exp_zfs_z2_prefetch_rw_2_other_opts ........... 2647.08 |=============== sync_off_1sas_zfs_z2_prefetch_rw_2_other ........... 2443.74 |============== sync_dva_1sas_z2_128_inflight_64_thrds ............. 2451.62 |============== 1sas_all_opts ...................................... 2515.09 |============== 1sas_all_opts_run2 ................................. 2391.45 |============== 1sas_all_deadline .................................. 2694.19 |=============== 1sas_all_opts_deadline ............................. 2449.32 |============== 1sas_all_noop_nr_requests_1024 ..................... 2470.46 |============== 1sas_all_noop_nr_requests_1024_48_thrds_tx_48 ...... 2426.31 |============== 1sas_all_noop_nr_requests_1024_deadline_32_threads . 3004.06 |================= 1sas_nr_1024_txg_128_64_thrds ...................... 2446.04 |============== 1sas_nr_1024_txg_48_48_thrds ....................... 2471.08 |============== 1sas_nr_1024_txg_32_32_tx_thrds .................... 2410.05 |============== 1sas_nr_1024_txg_64_64_noop ........................ 2832.34 |================ raidz3 ............................................. 2241.48 |============= 1sas_nr_1024_32_32_noop ............................ 2533.27 |============== 1sas_1024_128_32_noop .............................. 2398.11 |============== 1sas_1024_256_noop ................................. 2482.68 |============== 1sas_1024_deadline_64_64 ........................... 2275.86 |============= 1sas_128_noop_64_64 ................................ 2834.34 |================ 1sas_128_noop_64_64_run2 ........................... 2342.52 |============= zfs_raidz2_x3stripe_18_disks ....................... 2443.60 |============== IOzone 3.465 Record Size: 4Kb - File Size: 2GB - Disk Test: Read Performance MB/s > Higher Is Better 3sas_zfs_opts ...................................... 1163.72 |================ 3sas_zfs_opts_all_tests ............................ 1179.34 |================= 3sas_zfs_z2_txg_inflight_32 ........................ 1168.99 |================ 3sas_zfs_z2_txg_128_volthrds_64 .................... 1159.48 |================ 1sas_exp_zfs_z2_txg_128_volthrds_64 ................ 1148.54 |================ 1sas_exp_zfs_z2_prefetch_rw_2_other_opts ........... 1161.19 |================ sync_off_1sas_zfs_z2_prefetch_rw_2_other ........... 1146.45 |================ sync_dva_1sas_z2_128_inflight_64_thrds ............. 1140.44 |================ 1sas_all_opts ...................................... 1152.08 |================ 1sas_all_opts_run2 ................................. 1152.68 |================ 1sas_all_deadline .................................. 1165.44 |================ 1sas_all_opts_deadline ............................. 1130.65 |================ 1sas_all_noop_nr_requests_1024 ..................... 1155.58 |================ 1sas_all_noop_nr_requests_1024_deadline_32_threads . 1134.66 |================ 1sas_nr_1024_txg_128_64_thrds ...................... 1154.68 |================ 1sas_nr_1024_txg_32_32_tx_thrds .................... 1144.92 |================ 1sas_nr_1024_txg_64_64_noop ........................ 1161.94 |================ raidz3 ............................................. 1145.70 |================ 1sas_1024_256_noop ................................. 1131.29 |================ 1sas_1024_deadline_64_64 ........................... 1166.10 |================ 1sas_128_noop_64_64 ................................ 1137.31 |================ 1sas_128_noop_64_64_run2 ........................... 1161.69 |================ zfs_raidz2_x3stripe_18_disks ....................... 1205.33 |================= IOzone 3.465 Record Size: 1MB - File Size: 2GB - Disk Test: Write Performance MB/s > Higher Is Better 3sas_zfs_opts ...................................... 1288.97 |========== 3sas_zfs_opts_all_tests ............................ 1137.12 |========= 3sas_zfs_z2_txg_inflight_32 ........................ 1337.84 |========== 3sas_zfs_z2_txg_128_volthrds_64 .................... 1350.50 |========== 1sas_exp_zfs_z2_txg_128_volthrds_64 ................ 1381.80 |=========== 1sas_exp_zfs_z2_prefetch_rw_2_other_opts ........... 1291.30 |========== sync_off_1sas_zfs_z2_prefetch_rw_2_other ........... 2072.87 |================ sync_dva_1sas_z2_128_inflight_64_thrds ............. 2206.61 |================= 1sas_all_opts ...................................... 2126.22 |================ 1sas_all_opts_run2 ................................. 2218.22 |================= 1sas_all_deadline .................................. 1411.25 |=========== 1sas_all_opts_deadline ............................. 2193.61 |================= 1sas_all_noop_nr_requests_1024 ..................... 2082.08 |================ 1sas_all_noop_nr_requests_1024_deadline_32_threads . 2152.70 |================ 1sas_nr_1024_txg_32_32_tx_thrds .................... 2128.97 |================ 1sas_nr_1024_txg_64_64_noop ........................ 2096.24 |================ raidz3 ............................................. 2084.16 |================ 1sas_128_noop_64_64 ................................ 1968.61 |=============== 1sas_128_noop_64_64_run2 ........................... 2020.16 |=============== zfs_raidz2_x3stripe_18_disks ....................... 2082.47 |================ IOzone 3.465 Record Size: 4Kb - File Size: 2GB - Disk Test: Write Performance MB/s > Higher Is Better 3sas_zfs_opts ...................................... 411.13 |================ 3sas_zfs_opts_all_tests ............................ 406.69 |================ 3sas_zfs_z2_txg_inflight_32 ........................ 403.05 |================ 3sas_zfs_z2_txg_128_volthrds_64 .................... 408.30 |================ 1sas_exp_zfs_z2_txg_128_volthrds_64 ................ 409.14 |================ 1sas_exp_zfs_z2_prefetch_rw_2_other_opts ........... 409.75 |================ sync_off_1sas_zfs_z2_prefetch_rw_2_other ........... 456.63 |================== sync_dva_1sas_z2_128_inflight_64_thrds ............. 461.92 |================== 1sas_all_opts ...................................... 455.01 |================== 1sas_all_opts_run2 ................................. 465.76 |================== 1sas_all_deadline .................................. 413.36 |================ 1sas_all_opts_deadline ............................. 456.38 |================== 1sas_all_noop_nr_requests_1024 ..................... 461.23 |================== 1sas_all_noop_nr_requests_1024_deadline_32_threads . 454.63 |================== 1sas_nr_1024_txg_32_32_tx_thrds .................... 458.67 |================== 1sas_nr_1024_txg_64_64_noop ........................ 459.81 |================== raidz3 ............................................. 448.79 |================= 1sas_128_noop_64_64 ................................ 460.68 |================== 1sas_128_noop_64_64_run2 ........................... 461.15 |================== zfs_raidz2_x3stripe_18_disks ....................... 465.74 |================== IOzone 3.465 Record Size: 64Kb - File Size: 2GB - Disk Test: Read Performance MB/s > Higher Is Better 3sas_zfs_opts ...................................... 2693.47 |============= 3sas_zfs_opts_all_tests ............................ 2568.81 |============ 3sas_zfs_z2_txg_inflight_32 ........................ 2906.12 |============== 3sas_zfs_z2_txg_128_volthrds_64 .................... 2746.75 |============= 1sas_exp_zfs_z2_txg_128_volthrds_64 ................ 2738.77 |============= 1sas_exp_zfs_z2_prefetch_rw_2_other_opts ........... 2870.51 |============== sync_off_1sas_zfs_z2_prefetch_rw_2_other ........... 2548.72 |============ sync_dva_1sas_z2_128_inflight_64_thrds ............. 2559.49 |============ 1sas_all_opts ...................................... 2783.86 |============= 1sas_all_opts_run2 ................................. 2665.86 |============= 1sas_all_opts_deadline ............................. 2488.72 |============ 1sas_all_noop_nr_requests_1024 ..................... 3599.63 |================= 1sas_all_noop_nr_requests_1024_deadline_32_threads . 2596.61 |============ 1sas_nr_1024_txg_32_32_tx_thrds .................... 2570.67 |============ 1sas_nr_1024_txg_64_64_noop ........................ 2610.35 |============ raidz3 ............................................. 2510.40 |============ 1sas_128_noop_64_64 ................................ 2529.35 |============ 1sas_128_noop_64_64_run2 ........................... 2644.90 |============ zfs_raidz2_x3stripe_18_disks ....................... 2759.56 |============= IOzone 3.465 Record Size: 64Kb - File Size: 2GB - Disk Test: Write Performance MB/s > Higher Is Better 3sas_zfs_opts ...................................... 1036.24 |============= 3sas_zfs_opts_all_tests ............................ 932.72 |=========== 3sas_zfs_z2_txg_inflight_32 ........................ 1075.62 |============= 3sas_zfs_z2_txg_128_volthrds_64 .................... 1040.77 |============= 1sas_exp_zfs_z2_txg_128_volthrds_64 ................ 1086.85 |============= 1sas_exp_zfs_z2_prefetch_rw_2_other_opts ........... 1093.30 |============= sync_off_1sas_zfs_z2_prefetch_rw_2_other ........... 1325.67 |================ sync_dva_1sas_z2_128_inflight_64_thrds ............. 1349.36 |================ 1sas_all_opts ...................................... 1350.65 |================ 1sas_all_opts_run2 ................................. 1325.56 |================ 1sas_all_opts_deadline ............................. 1371.30 |================= 1sas_all_noop_nr_requests_1024 ..................... 1331.19 |================ 1sas_all_noop_nr_requests_1024_deadline_32_threads . 1384.97 |================= 1sas_nr_1024_txg_32_32_tx_thrds .................... 1368.78 |================= 1sas_nr_1024_txg_64_64_noop ........................ 1402.50 |================= raidz3 ............................................. 1348.90 |================ 1sas_128_noop_64_64 ................................ 1370.12 |================= 1sas_128_noop_64_64_run2 ........................... 1359.45 |================ zfs_raidz2_x3stripe_18_disks ....................... 1322.62 |================ IOzone 3.465 Record Size: 1MB - File Size: 4GB - Disk Test: Read Performance MB/s > Higher Is Better 3sas_zfs_opts_all_tests . 2738.57 |============================================ 1sas_all_opts_run2 ...... 2598.80 |========================================== IOzone 3.465 Record Size: 1MB - File Size: 8GB - Disk Test: Read Performance MB/s > Higher Is Better 3sas_zfs_opts_all_tests . 2721.32 |============================================ 1sas_all_opts_run2 ...... 2722.78 |============================================ IOzone 3.465 Record Size: 4Kb - File Size: 4GB - Disk Test: Read Performance MB/s > Higher Is Better 3sas_zfs_opts_all_tests . 1178.72 |============================================ 1sas_all_opts_run2 ...... 1141.95 |=========================================== IOzone 3.465 Record Size: 4Kb - File Size: 8GB - Disk Test: Read Performance MB/s > Higher Is Better 3sas_zfs_opts_all_tests . 1168.91 |============================================ 1sas_all_opts_run2 ...... 1172.22 |============================================ IOzone 3.465 Record Size: 1MB - File Size: 4GB - Disk Test: Write Performance MB/s > Higher Is Better 3sas_zfs_opts_all_tests . 1588.41 |================================== 1sas_all_opts_run2 ...... 2076.47 |============================================ IOzone 3.465 Record Size: 1MB - File Size: 8GB - Disk Test: Write Performance MB/s > Higher Is Better 3sas_zfs_opts_all_tests . 1739.93 |===================================== 1sas_all_opts_run2 ...... 2052.31 |============================================ IOzone 3.465 Record Size: 4Kb - File Size: 4GB - Disk Test: Write Performance MB/s > Higher Is Better 3sas_zfs_opts_all_tests . 417.36 |======================================== 1sas_all_opts_run2 ...... 465.28 |============================================= IOzone 3.465 Record Size: 4Kb - File Size: 8GB - Disk Test: Write Performance MB/s > Higher Is Better 3sas_zfs_opts_all_tests . 417.61 |========================================= 1sas_all_opts_run2 ...... 463.85 |============================================= IOzone 3.465 Record Size: 64Kb - File Size: 4GB - Disk Test: Read Performance MB/s > Higher Is Better 3sas_zfs_opts_all_tests . 2751.03 |============================================ 1sas_all_opts_run2 ...... 2508.90 |======================================== IOzone 3.465 Record Size: 64Kb - File Size: 8GB - Disk Test: Read Performance MB/s > Higher Is Better 3sas_zfs_opts_all_tests . 2845.78 |============================================ 1sas_all_opts_run2 ...... 2677.06 |========================================= IOzone 3.465 Record Size: 1MB - File Size: 512MB - Disk Test: Read Performance MB/s > Higher Is Better 3sas_zfs_opts_all_tests . 2613.48 |============================================ 1sas_all_opts_run2 ...... 2487.79 |========================================== IOzone 3.465 Record Size: 4Kb - File Size: 512MB - Disk Test: Read Performance MB/s > Higher Is Better 3sas_zfs_opts_all_tests . 1189.00 |============================================ 1sas_all_opts_run2 ...... 1141.72 |========================================== IOzone 3.465 Record Size: 64Kb - File Size: 4GB - Disk Test: Write Performance MB/s > Higher Is Better 3sas_zfs_opts_all_tests . 1172.90 |======================================= 1sas_all_opts_run2 ...... 1330.81 |============================================ IOzone 3.465 Record Size: 64Kb - File Size: 8GB - Disk Test: Write Performance MB/s > Higher Is Better 3sas_zfs_opts_all_tests . 1227.27 |======================================== 1sas_all_opts_run2 ...... 1364.61 |============================================ IOzone 3.465 Record Size: 1MB - File Size: 512MB - Disk Test: Write Performance MB/s > Higher Is Better 3sas_zfs_opts_all_tests . 756.05 |================= 1sas_all_opts_run2 ...... 2008.43 |============================================ IOzone 3.465 Record Size: 4Kb - File Size: 512MB - Disk Test: Write Performance MB/s > Higher Is Better 3sas_zfs_opts_all_tests . 372.47 |===================================== 1sas_all_opts_run2 ...... 459.08 |============================================= IOzone 3.465 Record Size: 64Kb - File Size: 512MB - Disk Test: Read Performance MB/s > Higher Is Better 3sas_zfs_opts_all_tests . 2877.05 |============================================ 1sas_all_opts_run2 ...... 2528.66 |======================================= IOzone 3.465 Record Size: 64Kb - File Size: 512MB - Disk Test: Write Performance MB/s > Higher Is Better 3sas_zfs_opts_all_tests . 717.72 |====================== 1sas_all_opts_run2 ...... 1412.99 |============================================