201907041004 2 x Intel Xeon E5-2620 v4 testing with a Intel S2600CWR (SE5C610.86B.01.01.0024.021320181901 BIOS) and Matrox MGA G200e [Pilot] on CentOS Linux 7 via the Phoronix Test Suite. ZFS RaidZ3 12TB 7EA recordsize 4K: Processor: 2 x Intel Xeon E5-2620 v4 @ 3.00GHz (16 Cores / 32 Threads), Motherboard: Intel S2600CWR (SE5C610.86B.01.01.0024.021320181901 BIOS), Chipset: Intel Xeon E7 v4/Xeon, Memory: 4 x 32 GB DDR4-2134MT/s Samsung M393A4K40CB2-CTD, Disk: 7 x 12000GB Seagate ST12000NM0027 + 239GB Logical Volume, Graphics: Matrox MGA G200e [Pilot], Network: 2 x Intel I350 + 2 x Intel 82599ES 10-Gigabit SFI/SFP+ OS: CentOS Linux 7, Kernel: 3.10.0-957.21.2.el7.x86_64 (x86_64), Compiler: GCC 4.8.5 20150623, File-System: zfs, Screen Resolution: 1024x768 FS-Mark 3.3 Test: 1000 Files, 4K Size Files/s > Higher Is Better ZFS RaidZ3 12TB 7EA recordsize 4K . 105.40 |=================================== FS-Mark 3.3 Test: 1000 Files, 1MB Size Files/s > Higher Is Better ZFS RaidZ3 12TB 7EA recordsize 4K . 18.80 |==================================== FS-Mark 3.3 Test: 1000 Files, 64K Size Files/s > Higher Is Better ZFS RaidZ3 12TB 7EA recordsize 4K . 72.75 |==================================== FS-Mark 3.3 Test: 1000 Files, 128K Size Files/s > Higher Is Better ZFS RaidZ3 12TB 7EA recordsize 4K . 43.57 |==================================== FS-Mark 3.3 Test: 1000 Files, 4K Size, 4 Threads Files/s > Higher Is Better ZFS RaidZ3 12TB 7EA recordsize 4K . 180.48 |=================================== FS-Mark 3.3 Test: 1000 Files, 1MB Size, 4 Threads Files/s > Higher Is Better ZFS RaidZ3 12TB 7EA recordsize 4K . 29.33 |==================================== FS-Mark 3.3 Test: 1000 Files, 4K Size, 16 Threads Files/s > Higher Is Better ZFS RaidZ3 12TB 7EA recordsize 4K . 451.25 |=================================== FS-Mark 3.3 Test: 1000 Files, 64K Size, 4 Threads Files/s > Higher Is Better ZFS RaidZ3 12TB 7EA recordsize 4K . 93.30 |==================================== FS-Mark 3.3 Test: 1000 Files, 128K Size, 4 Threads Files/s > Higher Is Better ZFS RaidZ3 12TB 7EA recordsize 4K . 73.97 |==================================== FS-Mark 3.3 Test: 1000 Files, 1MB Size, 16 Threads Files/s > Higher Is Better ZFS RaidZ3 12TB 7EA recordsize 4K . 24 |======================================= FS-Mark 3.3 Test: 1000 Files, 64K Size, 16 Threads Files/s > Higher Is Better ZFS RaidZ3 12TB 7EA recordsize 4K . 144.17 |=================================== FS-Mark 3.3 Test: 1000 Files, 128K Size, 16 Threads Files/s > Higher Is Better ZFS RaidZ3 12TB 7EA recordsize 4K . 109.83 |=================================== FS-Mark 3.3 Test: No FSYNC/SYNC 1000 Files, 4K Size Files/s > Higher Is Better ZFS RaidZ3 12TB 7EA recordsize 4K . 3576.94 |================================== FS-Mark 3.3 Test: No FSYNC/SYNC 1000 Files, 1MB Size Files/s > Higher Is Better ZFS RaidZ3 12TB 7EA recordsize 4K . 370.20 |=================================== FS-Mark 3.3 Test: No FSYNC/SYNC 1000 Files, 64K Size Files/s > Higher Is Better ZFS RaidZ3 12TB 7EA recordsize 4K . 1407.66 |================================== FS-Mark 3.3 Test: No FSYNC/SYNC 1000 Files, 128K Size Files/s > Higher Is Better ZFS RaidZ3 12TB 7EA recordsize 4K . 881.30 |=================================== FS-Mark 3.3 Test: No FSYNC/SYNC 1000 Files, 4K Size, 4 Threads Files/s > Higher Is Better ZFS RaidZ3 12TB 7EA recordsize 4K . 2815.17 |================================== FS-Mark 3.3 Test: No FSYNC/SYNC 1000 Files, 1MB Size, 4 Threads Files/s > Higher Is Better ZFS RaidZ3 12TB 7EA recordsize 4K . 56.43 |==================================== FS-Mark 3.3 Test: No FSYNC/SYNC 1000 Files, 4K Size, 16 Threads Files/s > Higher Is Better ZFS RaidZ3 12TB 7EA recordsize 4K . 2943.67 |================================== FS-Mark 3.3 Test: No FSYNC/SYNC 1000 Files, 64K Size, 4 Threads Files/s > Higher Is Better ZFS RaidZ3 12TB 7EA recordsize 4K . 2947.66 |================================== FS-Mark 3.3 Test: No FSYNC/SYNC 1000 Files, 128K Size, 4 Threads Files/s > Higher Is Better ZFS RaidZ3 12TB 7EA recordsize 4K . 3822.43 |================================== FS-Mark 3.3 Test: No FSYNC/SYNC 1000 Files, 1MB Size, 16 Threads Files/s > Higher Is Better ZFS RaidZ3 12TB 7EA recordsize 4K . 24 |======================================= FS-Mark 3.3 Test: No FSYNC/SYNC 1000 Files, 64K Size, 16 Threads Files/s > Higher Is Better ZFS RaidZ3 12TB 7EA recordsize 4K . 2649.82 |================================== FS-Mark 3.3 Test: No FSYNC/SYNC 1000 Files, 128K Size, 16 Threads Files/s > Higher Is Better ZFS RaidZ3 12TB 7EA recordsize 4K . 705.93 |===================================