201907011142-zfs-z3 2 x Intel Xeon E5-2620 v4 testing with a Intel S2600CWR (SE5C610.86B.01.01.0024.021320181901 BIOS) and Matrox MGA G200e [Pilot] on CentOS Linux 7 via the Phoronix Test Suite. ZFS RaidZ3 12TB 5EA recordsize 128K: Processor: 2 x Intel Xeon E5-2620 v4 @ 3.00GHz (16 Cores / 32 Threads), Motherboard: Intel S2600CWR (SE5C610.86B.01.01.0024.021320181901 BIOS), Chipset: Intel Xeon E7 v4/Xeon, Memory: 4 x 32 GB DDR4-2134MT/s Samsung M393A4K40CB2-CTD, Disk: 11 x 12000GB Seagate ST12000NM0027 + 239GB Logical Volume, Graphics: Matrox MGA G200e [Pilot], Network: 2 x Intel I350 + 2 x Intel 82599ES 10-Gigabit SFI/SFP+ OS: CentOS Linux 7, Kernel: 3.10.0-957.21.2.el7.x86_64 (x86_64), Compiler: GCC 4.8.5 20150623, File-System: zfs, Screen Resolution: 1024x768 ZFS RaidZ3 12TB 5EA recordsize 4K: Processor: 2 x Intel Xeon E5-2620 v4 @ 3.00GHz (16 Cores / 32 Threads), Motherboard: Intel S2600CWR (SE5C610.86B.01.01.0024.021320181901 BIOS), Chipset: Intel Xeon E7 v4/Xeon, Memory: 4 x 32 GB DDR4-2134MT/s Samsung M393A4K40CB2-CTD, Disk: 11 x 12000GB Seagate ST12000NM0027 + 239GB Logical Volume, Graphics: Matrox MGA G200e [Pilot], Network: 2 x Intel I350 + 2 x Intel 82599ES 10-Gigabit SFI/SFP+ OS: CentOS Linux 7, Kernel: 3.10.0-957.21.2.el7.x86_64 (x86_64), Compiler: GCC 4.8.5 20150623, File-System: zfs, Screen Resolution: 1024x768 ZFS RaidZ3 12TB 5EA recordsize 64K: Processor: 2 x Intel Xeon E5-2620 v4 @ 3.00GHz (16 Cores / 32 Threads), Motherboard: Intel S2600CWR (SE5C610.86B.01.01.0024.021320181901 BIOS), Chipset: Intel Xeon E7 v4/Xeon, Memory: 4 x 32 GB DDR4-2134MT/s Samsung M393A4K40CB2-CTD, Disk: 11 x 12000GB Seagate ST12000NM0027 + 239GB Logical Volume, Graphics: Matrox MGA G200e [Pilot], Network: 2 x Intel I350 + 2 x Intel 82599ES 10-Gigabit SFI/SFP+ OS: CentOS Linux 7, Kernel: 3.10.0-957.21.2.el7.x86_64 (x86_64), Compiler: GCC 4.8.5 20150623, File-System: zfs, Screen Resolution: 1024x768 ZFS RaidZ3 12TB 5EA recordsize 1M: Processor: 2 x Intel Xeon E5-2620 v4 @ 3.00GHz (16 Cores / 32 Threads), Motherboard: Intel S2600CWR (SE5C610.86B.01.01.0024.021320181901 BIOS), Chipset: Intel Xeon E7 v4/Xeon, Memory: 4 x 32 GB DDR4-2134MT/s Samsung M393A4K40CB2-CTD, Disk: 11 x 12000GB Seagate ST12000NM0027 + 239GB Logical Volume, Graphics: Matrox MGA G200e [Pilot], Network: 2 x Intel I350 + 2 x Intel 82599ES 10-Gigabit SFI/SFP+ OS: CentOS Linux 7, Kernel: 3.10.0-957.21.2.el7.x86_64 (x86_64), Compiler: GCC 4.8.5 20150623, File-System: zfs, Screen Resolution: 1024x768 ZFS RaidZ3 12TB 7EA 128K: Processor: 2 x Intel Xeon E5-2620 v4 @ 3.00GHz (16 Cores / 32 Threads), Motherboard: Intel S2600CWR (SE5C610.86B.01.01.0024.021320181901 BIOS), Chipset: Intel Xeon E7 v4/Xeon, Memory: 4 x 32 GB DDR4-2134MT/s Samsung M393A4K40CB2-CTD, Disk: 11 x 12000GB Seagate ST12000NM0027 + 239GB Logical Volume, Graphics: Matrox MGA G200e [Pilot], Network: 2 x Intel I350 + 2 x Intel 82599ES 10-Gigabit SFI/SFP+ OS: CentOS Linux 7, Kernel: 3.10.0-957.21.2.el7.x86_64 (x86_64), Compiler: GCC 4.8.5 20150623, File-System: zfs, Screen Resolution: 1024x768 ZFS RaidZ3 12TB 7EA recordsize 4K: Processor: 2 x Intel Xeon E5-2620 v4 @ 3.00GHz (16 Cores / 32 Threads), Motherboard: Intel S2600CWR (SE5C610.86B.01.01.0024.021320181901 BIOS), Chipset: Intel Xeon E7 v4/Xeon, Memory: 4 x 32 GB DDR4-2134MT/s Samsung M393A4K40CB2-CTD, Disk: 11 x 12000GB Seagate ST12000NM0027 + 239GB Logical Volume, Graphics: Matrox MGA G200e [Pilot], Network: 2 x Intel I350 + 2 x Intel 82599ES 10-Gigabit SFI/SFP+ OS: CentOS Linux 7, Kernel: 3.10.0-957.21.2.el7.x86_64 (x86_64), Compiler: GCC 4.8.5 20150623, File-System: zfs, Screen Resolution: 1024x768 ZFS RaidZ3 12TB 7EA recordsize 64K: Processor: 2 x Intel Xeon E5-2620 v4 @ 3.00GHz (16 Cores / 32 Threads), Motherboard: Intel S2600CWR (SE5C610.86B.01.01.0024.021320181901 BIOS), Chipset: Intel Xeon E7 v4/Xeon, Memory: 4 x 32 GB DDR4-2134MT/s Samsung M393A4K40CB2-CTD, Disk: 11 x 12000GB Seagate ST12000NM0027 + 239GB Logical Volume, Graphics: Matrox MGA G200e [Pilot], Network: 2 x Intel I350 + 2 x Intel 82599ES 10-Gigabit SFI/SFP+ OS: CentOS Linux 7, Kernel: 3.10.0-957.21.2.el7.x86_64 (x86_64), Compiler: GCC 4.8.5 20150623, File-System: zfs, Screen Resolution: 1024x768 ZFS RaidZ3 12TB 7EA recordsize 1M: Processor: 2 x Intel Xeon E5-2620 v4 @ 3.00GHz (16 Cores / 32 Threads), Motherboard: Intel S2600CWR (SE5C610.86B.01.01.0024.021320181901 BIOS), Chipset: Intel Xeon E7 v4/Xeon, Memory: 4 x 32 GB DDR4-2134MT/s Samsung M393A4K40CB2-CTD, Disk: 11 x 12000GB Seagate ST12000NM0027 + 239GB Logical Volume, Graphics: Matrox MGA G200e [Pilot], Network: 2 x Intel I350 + 2 x Intel 82599ES 10-Gigabit SFI/SFP+ OS: CentOS Linux 7, Kernel: 3.10.0-957.21.2.el7.x86_64 (x86_64), Compiler: GCC 4.8.5 20150623, File-System: zfs, Screen Resolution: 1024x768 ZFS RaidZ3 12TB 11EA recordsize 128K: Processor: 2 x Intel Xeon E5-2620 v4 @ 3.00GHz (16 Cores / 32 Threads), Motherboard: Intel S2600CWR (SE5C610.86B.01.01.0024.021320181901 BIOS), Chipset: Intel Xeon E7 v4/Xeon, Memory: 4 x 32 GB DDR4-2134MT/s Samsung M393A4K40CB2-CTD, Disk: 11 x 12000GB Seagate ST12000NM0027 + 239GB Logical Volume, Graphics: Matrox MGA G200e [Pilot], Network: 2 x Intel I350 + 2 x Intel 82599ES 10-Gigabit SFI/SFP+ OS: CentOS Linux 7, Kernel: 3.10.0-957.21.2.el7.x86_64 (x86_64), Compiler: GCC 4.8.5 20150623, File-System: zfs, Screen Resolution: 1024x768 ZFS RaidZ3 12TB 11EA recordsize 4K: Processor: 2 x Intel Xeon E5-2620 v4 @ 3.00GHz (16 Cores / 32 Threads), Motherboard: Intel S2600CWR (SE5C610.86B.01.01.0024.021320181901 BIOS), Chipset: Intel Xeon E7 v4/Xeon, Memory: 4 x 32 GB DDR4-2134MT/s Samsung M393A4K40CB2-CTD, Disk: 11 x 12000GB Seagate ST12000NM0027 + 239GB Logical Volume, Graphics: Matrox MGA G200e [Pilot], Network: 2 x Intel I350 + 2 x Intel 82599ES 10-Gigabit SFI/SFP+ OS: CentOS Linux 7, Kernel: 3.10.0-957.21.2.el7.x86_64 (x86_64), Compiler: GCC 4.8.5 20150623, File-System: zfs, Screen Resolution: 1024x768 ZFS RaidZ3 12TB 11EA recordsize 64K: Processor: 2 x Intel Xeon E5-2620 v4 @ 3.00GHz (16 Cores / 32 Threads), Motherboard: Intel S2600CWR (SE5C610.86B.01.01.0024.021320181901 BIOS), Chipset: Intel Xeon E7 v4/Xeon, Memory: 4 x 32 GB DDR4-2134MT/s Samsung M393A4K40CB2-CTD, Disk: 11 x 12000GB Seagate ST12000NM0027 + 239GB Logical Volume, Graphics: Matrox MGA G200e [Pilot], Network: 2 x Intel I350 + 2 x Intel 82599ES 10-Gigabit SFI/SFP+ OS: CentOS Linux 7, Kernel: 3.10.0-957.21.2.el7.x86_64 (x86_64), Compiler: GCC 4.8.5 20150623, File-System: zfs, Screen Resolution: 1024x768 ZFS RaidZ3 12TB 11EA recordsize 1M: Processor: 2 x Intel Xeon E5-2620 v4 @ 3.00GHz (16 Cores / 32 Threads), Motherboard: Intel S2600CWR (SE5C610.86B.01.01.0024.021320181901 BIOS), Chipset: Intel Xeon E7 v4/Xeon, Memory: 4 x 32 GB DDR4-2134MT/s Samsung M393A4K40CB2-CTD, Disk: 11 x 12000GB Seagate ST12000NM0027 + 239GB Logical Volume, Graphics: Matrox MGA G200e [Pilot], Network: 2 x Intel I350 + 2 x Intel 82599ES 10-Gigabit SFI/SFP+ OS: CentOS Linux 7, Kernel: 3.10.0-957.21.2.el7.x86_64 (x86_64), Compiler: GCC 4.8.5 20150623, File-System: zfs, Screen Resolution: 1024x768 IOzone 3.465 Record Size: 128Kb - File Size: 2GB - Disk Test: Write Performance MB/s > Higher Is Better ZFS RaidZ3 12TB 5EA recordsize 128K .. 185 |=========== ZFS RaidZ3 12TB 7EA 128K ............. 322 |==================== ZFS RaidZ3 12TB 11EA recordsize 128K . 571 |=================================== IOzone 3.465 Record Size: 128Kb - File Size: 4GB - Disk Test: Write Performance MB/s > Higher Is Better ZFS RaidZ3 12TB 5EA recordsize 128K .. 170 |=========== ZFS RaidZ3 12TB 7EA 128K ............. 321 |====================== ZFS RaidZ3 12TB 11EA recordsize 128K . 518 |=================================== IOzone 3.465 Record Size: 128Kb - File Size: 8GB - Disk Test: Write Performance MB/s > Higher Is Better ZFS RaidZ3 12TB 5EA recordsize 128K .. 220 |=============== ZFS RaidZ3 12TB 7EA 128K ............. 360 |========================= ZFS RaidZ3 12TB 11EA recordsize 128K . 511 |=================================== IOzone 3.465 Record Size: 128Kb - File Size: 512MB - Disk Test: Write Performance MB/s > Higher Is Better ZFS RaidZ3 12TB 5EA recordsize 128K .. 193 |============ ZFS RaidZ3 12TB 7EA 128K ............. 366 |======================== ZFS RaidZ3 12TB 11EA recordsize 128K . 544 |=================================== IOzone 3.465 Record Size: 4Kb - File Size: 2GB - Disk Test: Write Performance MB/s > Higher Is Better ZFS RaidZ3 12TB 5EA recordsize 4K .. 176 |=============================== ZFS RaidZ3 12TB 7EA recordsize 4K .. 179 |=============================== ZFS RaidZ3 12TB 11EA recordsize 4K . 212 |===================================== IOzone 3.465 Record Size: 4Kb - File Size: 4GB - Disk Test: Write Performance MB/s > Higher Is Better ZFS RaidZ3 12TB 5EA recordsize 4K .. 40.32 |======================== ZFS RaidZ3 12TB 7EA recordsize 4K .. 47.69 |============================ ZFS RaidZ3 12TB 11EA recordsize 4K . 58.80 |=================================== IOzone 3.465 Record Size: 4Kb - File Size: 8GB - Disk Test: Write Performance MB/s > Higher Is Better ZFS RaidZ3 12TB 5EA recordsize 4K .. 22.36 |====================== ZFS RaidZ3 12TB 7EA recordsize 4K .. 27.79 |=========================== ZFS RaidZ3 12TB 11EA recordsize 4K . 35.48 |=================================== IOzone 3.465 Record Size: 4Kb - File Size: 512MB - Disk Test: Write Performance MB/s > Higher Is Better ZFS RaidZ3 12TB 5EA recordsize 4K .. 139 |======================= ZFS RaidZ3 12TB 7EA recordsize 4K .. 198 |================================= ZFS RaidZ3 12TB 11EA recordsize 4K . 222 |===================================== IOzone 3.465 Record Size: 64Kb - File Size: 2GB - Disk Test: Write Performance MB/s > Higher Is Better ZFS RaidZ3 12TB 5EA recordsize 64K .. 175 |============ ZFS RaidZ3 12TB 7EA recordsize 64K .. 374 |========================== ZFS RaidZ3 12TB 11EA recordsize 64K . 513 |==================================== IOzone 3.465 Record Size: 64Kb - File Size: 4GB - Disk Test: Write Performance MB/s > Higher Is Better ZFS RaidZ3 12TB 5EA recordsize 64K .. 164 |============ ZFS RaidZ3 12TB 7EA recordsize 64K .. 284 |====================== ZFS RaidZ3 12TB 11EA recordsize 64K . 474 |==================================== IOzone 3.465 Record Size: 64Kb - File Size: 8GB - Disk Test: Write Performance MB/s > Higher Is Better ZFS RaidZ3 12TB 5EA recordsize 64K .. 201 |============= ZFS RaidZ3 12TB 7EA recordsize 64K .. 271 |================== ZFS RaidZ3 12TB 11EA recordsize 64K . 551 |==================================== IOzone 3.465 Record Size: 64Kb - File Size: 512MB - Disk Test: Write Performance MB/s > Higher Is Better ZFS RaidZ3 12TB 5EA recordsize 64K .. 209 |============== ZFS RaidZ3 12TB 7EA recordsize 64K .. 406 |=========================== ZFS RaidZ3 12TB 11EA recordsize 64K . 547 |==================================== IOzone 3.465 Record Size: 1MB - File Size: 2GB - Disk Test: Write Performance MB/s > Higher Is Better ZFS RaidZ3 12TB 5EA recordsize 1M .. 204 |============= ZFS RaidZ3 12TB 7EA recordsize 1M .. 345 |===================== ZFS RaidZ3 12TB 11EA recordsize 1M . 602 |===================================== IOzone 3.465 Record Size: 1MB - File Size: 4GB - Disk Test: Write Performance MB/s > Higher Is Better ZFS RaidZ3 12TB 5EA recordsize 1M .. 235 |============== ZFS RaidZ3 12TB 7EA recordsize 1M .. 446 |=========================== ZFS RaidZ3 12TB 11EA recordsize 1M . 620 |===================================== IOzone 3.465 Record Size: 1MB - File Size: 8GB - Disk Test: Write Performance MB/s > Higher Is Better ZFS RaidZ3 12TB 5EA recordsize 1M .. 245 |============ ZFS RaidZ3 12TB 7EA recordsize 1M .. 432 |===================== ZFS RaidZ3 12TB 11EA recordsize 1M . 750 |===================================== IOzone 3.465 Record Size: 1MB - File Size: 512MB - Disk Test: Write Performance MB/s > Higher Is Better ZFS RaidZ3 12TB 5EA recordsize 1M .. 227 |============== ZFS RaidZ3 12TB 7EA recordsize 1M .. 344 |===================== ZFS RaidZ3 12TB 11EA recordsize 1M . 598 |=====================================