201906040855 2 x Intel Xeon E5-2620 v4 testing with a Intel S2600CWR (SE5C610.86B.01.01.0024.021320181901 BIOS) and Matrox MGA G200e [Pilot] on CentOS Linux 7 via the Phoronix Test Suite. ZFS RAIDZ2 12TB 4EA: Processor: 2 x Intel Xeon E5-2620 v4 @ 3.00GHz (16 Cores / 32 Threads), Motherboard: Intel S2600CWR (SE5C610.86B.01.01.0024.021320181901 BIOS), Memory: 4 x 32 GB DDR4-2134MT/s Samsung M393A4K40CB2-CTD, Disk: 7 x 12000GB Seagate ST12000NM0027 + 239GB Logical Volume, Graphics: mgadrmfb OS: CentOS Linux 7, Kernel: 3.10.0-957.el7.x86_64 (x86_64), Compiler: GCC 4.8.5 20150623, File-System: zfs, Screen Resolution: 1024x768 ZFS RAIDZ2 12TB 6EA: Processor: 2 x Intel Xeon E5-2620 v4 @ 3.00GHz (16 Cores / 32 Threads), Motherboard: Intel S2600CWR (SE5C610.86B.01.01.0024.021320181901 BIOS), Memory: 4 x 32 GB DDR4-2134MT/s Samsung M393A4K40CB2-CTD, Disk: 7 x 12000GB Seagate ST12000NM0027 + 239GB Logical Volume, Graphics: mgadrmfb OS: CentOS Linux 7, Kernel: 3.10.0-957.el7.x86_64 (x86_64), Compiler: GCC 4.8.5 20150623, File-System: zfs, Screen Resolution: 1024x768 ZFS RAIDZ2 12TB 10EA: Processor: 2 x Intel Xeon E5-2620 v4 @ 3.00GHz (16 Cores / 32 Threads), Motherboard: Intel S2600CWR (SE5C610.86B.01.01.0024.021320181901 BIOS), Chipset: Intel Xeon E7 v4/Xeon, Memory: 4 x 32 GB DDR4-2134MT/s Samsung M393A4K40CB2-CTD, Disk: 11 x 12000GB Seagate ST12000NM0027 + 239GB Logical Volume, Graphics: Matrox MGA G200e [Pilot], Network: 2 x Intel I350 + 2 x Intel 82599ES 10-Gigabit SFI/SFP+ OS: CentOS Linux 7, Kernel: 3.10.0-957.el7.x86_64 (x86_64), Compiler: GCC 4.8.5 20150623, File-System: zfs, Screen Resolution: 1024x768 ZFS RAIDZ3 12TB 5EA: Processor: 2 x Intel Xeon E5-2620 v4 @ 3.00GHz (16 Cores / 32 Threads), Motherboard: Intel S2600CWR (SE5C610.86B.01.01.0024.021320181901 BIOS), Chipset: Intel Xeon E7 v4/Xeon, Memory: 4 x 32 GB DDR4-2134MT/s Samsung M393A4K40CB2-CTD, Disk: 11 x 12000GB Seagate ST12000NM0027 + 239GB Logical Volume, Graphics: Matrox MGA G200e [Pilot], Network: 2 x Intel I350 + 2 x Intel 82599ES 10-Gigabit SFI/SFP+ OS: CentOS Linux 7, Kernel: 3.10.0-957.el7.x86_64 (x86_64), Compiler: GCC 4.8.5 20150623, File-System: zfs, Screen Resolution: 1024x768 ZFS RAIDZ3 12TB 7EA: Processor: 2 x Intel Xeon E5-2620 v4 @ 3.00GHz (16 Cores / 32 Threads), Motherboard: Intel S2600CWR (SE5C610.86B.01.01.0024.021320181901 BIOS), Chipset: Intel Xeon E7 v4/Xeon, Memory: 4 x 32 GB DDR4-2134MT/s Samsung M393A4K40CB2-CTD, Disk: 11 x 12000GB Seagate ST12000NM0027 + 239GB Logical Volume, Graphics: Matrox MGA G200e [Pilot], Network: 2 x Intel I350 + 2 x Intel 82599ES 10-Gigabit SFI/SFP+ OS: CentOS Linux 7, Kernel: 3.10.0-957.el7.x86_64 (x86_64), Compiler: GCC 4.8.5 20150623, File-System: zfs, Screen Resolution: 1024x768 IOzone 3.465 Record Size: 1MB - File Size: 2GB - Disk Test: Read Performance MB/s > Higher Is Better ZFS RAIDZ2 12TB 4EA .. 2149 |========================================== ZFS RAIDZ2 12TB 6EA .. 2299 |============================================= ZFS RAIDZ2 12TB 10EA . 2557 |================================================== ZFS RAIDZ3 12TB 5EA .. 2199 |=========================================== ZFS RAIDZ3 12TB 7EA .. 2375 |============================================== IOzone 3.465 Record Size: 1MB - File Size: 4GB - Disk Test: Read Performance MB/s > Higher Is Better ZFS RAIDZ2 12TB 4EA .. 2225 |================================================== ZFS RAIDZ2 12TB 6EA .. 2172 |================================================ ZFS RAIDZ2 12TB 10EA . 2243 |================================================== ZFS RAIDZ3 12TB 5EA .. 2224 |================================================== ZFS RAIDZ3 12TB 7EA .. 2209 |================================================= IOzone 3.465 Record Size: 1MB - File Size: 8GB - Disk Test: Read Performance MB/s > Higher Is Better ZFS RAIDZ2 12TB 4EA .. 2279 |================================================ ZFS RAIDZ2 12TB 6EA .. 2374 |================================================== ZFS RAIDZ2 12TB 10EA . 2270 |================================================ ZFS RAIDZ3 12TB 5EA .. 2263 |================================================ ZFS RAIDZ3 12TB 7EA .. 2354 |================================================== IOzone 3.465 Record Size: 4Kb - File Size: 2GB - Disk Test: Read Performance MB/s > Higher Is Better ZFS RAIDZ2 12TB 4EA .. 992 |================================================ ZFS RAIDZ2 12TB 6EA .. 1003 |================================================= ZFS RAIDZ2 12TB 10EA . 1019 |================================================= ZFS RAIDZ3 12TB 5EA .. 1030 |================================================== ZFS RAIDZ3 12TB 7EA .. 1024 |================================================== IOzone 3.465 Record Size: 4Kb - File Size: 4GB - Disk Test: Read Performance MB/s > Higher Is Better ZFS RAIDZ2 12TB 4EA .. 1050 |================================================ ZFS RAIDZ2 12TB 6EA .. 1057 |================================================ ZFS RAIDZ2 12TB 10EA . 1091 |================================================== ZFS RAIDZ3 12TB 5EA .. 1048 |================================================ ZFS RAIDZ3 12TB 7EA .. 1046 |================================================ IOzone 3.465 Record Size: 4Kb - File Size: 8GB - Disk Test: Read Performance MB/s > Higher Is Better ZFS RAIDZ2 12TB 4EA .. 1106 |================================================== ZFS RAIDZ2 12TB 6EA .. 1098 |================================================== ZFS RAIDZ2 12TB 10EA . 1097 |================================================== ZFS RAIDZ3 12TB 5EA .. 1093 |================================================= ZFS RAIDZ3 12TB 7EA .. 1103 |================================================== IOzone 3.465 Record Size: 1MB - File Size: 2GB - Disk Test: Write Performance MB/s > Higher Is Better ZFS RAIDZ2 12TB 4EA .. 181 |================ ZFS RAIDZ2 12TB 6EA .. 337 |============================== ZFS RAIDZ2 12TB 10EA . 565 |=================================================== ZFS RAIDZ3 12TB 5EA .. 177 |================ ZFS RAIDZ3 12TB 7EA .. 360 |================================ IOzone 3.465 Record Size: 1MB - File Size: 4GB - Disk Test: Write Performance MB/s > Higher Is Better ZFS RAIDZ2 12TB 4EA .. 165 |================= ZFS RAIDZ2 12TB 6EA .. 335 |================================== ZFS RAIDZ2 12TB 10EA . 503 |=================================================== ZFS RAIDZ3 12TB 5EA .. 170 |================= ZFS RAIDZ3 12TB 7EA .. 323 |================================= IOzone 3.465 Record Size: 1MB - File Size: 8GB - Disk Test: Write Performance MB/s > Higher Is Better ZFS RAIDZ2 12TB 4EA .. 206 |==================== ZFS RAIDZ2 12TB 6EA .. 377 |===================================== ZFS RAIDZ2 12TB 10EA . 516 |=================================================== ZFS RAIDZ3 12TB 5EA .. 216 |===================== ZFS RAIDZ3 12TB 7EA .. 366 |==================================== IOzone 3.465 Record Size: 4Kb - File Size: 2GB - Disk Test: Write Performance MB/s > Higher Is Better ZFS RAIDZ2 12TB 4EA .. 205 |============================== ZFS RAIDZ2 12TB 6EA .. 315 |============================================== ZFS RAIDZ2 12TB 10EA . 346 |=================================================== ZFS RAIDZ3 12TB 5EA .. 209 |=============================== ZFS RAIDZ3 12TB 7EA .. 306 |============================================= IOzone 3.465 Record Size: 4Kb - File Size: 4GB - Disk Test: Write Performance MB/s > Higher Is Better ZFS RAIDZ2 12TB 4EA .. 227 |================================== ZFS RAIDZ2 12TB 6EA .. 344 |=================================================== ZFS RAIDZ2 12TB 10EA . 343 |=================================================== ZFS RAIDZ3 12TB 5EA .. 241 |==================================== ZFS RAIDZ3 12TB 7EA .. 325 |================================================ IOzone 3.465 Record Size: 4Kb - File Size: 8GB - Disk Test: Write Performance MB/s > Higher Is Better ZFS RAIDZ2 12TB 4EA .. 240 |================================= ZFS RAIDZ2 12TB 6EA .. 363 |================================================= ZFS RAIDZ2 12TB 10EA . 376 |=================================================== ZFS RAIDZ3 12TB 5EA .. 266 |==================================== ZFS RAIDZ3 12TB 7EA .. 370 |================================================== IOzone 3.465 Record Size: 64Kb - File Size: 2GB - Disk Test: Read Performance MB/s > Higher Is Better ZFS RAIDZ2 12TB 4EA .. 1947 |========================================== ZFS RAIDZ2 12TB 6EA .. 2147 |============================================== ZFS RAIDZ2 12TB 10EA . 2309 |================================================== ZFS RAIDZ3 12TB 5EA .. 1993 |=========================================== ZFS RAIDZ3 12TB 7EA .. 1949 |========================================== IOzone 3.465 Record Size: 64Kb - File Size: 4GB - Disk Test: Read Performance MB/s > Higher Is Better ZFS RAIDZ2 12TB 4EA .. 1963 |============================================== ZFS RAIDZ2 12TB 6EA .. 2109 |================================================== ZFS RAIDZ2 12TB 10EA . 2124 |================================================== ZFS RAIDZ3 12TB 5EA .. 1953 |============================================== ZFS RAIDZ3 12TB 7EA .. 2115 |================================================== IOzone 3.465 Record Size: 64Kb - File Size: 8GB - Disk Test: Read Performance MB/s > Higher Is Better ZFS RAIDZ2 12TB 4EA .. 2124 |================================================= ZFS RAIDZ2 12TB 6EA .. 2104 |================================================ ZFS RAIDZ2 12TB 10EA . 2017 |============================================== ZFS RAIDZ3 12TB 5EA .. 2173 |================================================== ZFS RAIDZ3 12TB 7EA .. 2105 |================================================ IOzone 3.465 Record Size: 1MB - File Size: 512MB - Disk Test: Read Performance MB/s > Higher Is Better ZFS RAIDZ2 12TB 4EA .. 2842 |=========================================== ZFS RAIDZ2 12TB 6EA .. 3192 |================================================ ZFS RAIDZ2 12TB 10EA . 2932 |============================================ ZFS RAIDZ3 12TB 5EA .. 3321 |================================================== ZFS RAIDZ3 12TB 7EA .. 3169 |================================================ IOzone 3.465 Record Size: 4Kb - File Size: 512MB - Disk Test: Read Performance MB/s > Higher Is Better ZFS RAIDZ2 12TB 4EA .. 979 |=================================================== ZFS RAIDZ2 12TB 6EA .. 945 |================================================= ZFS RAIDZ2 12TB 10EA . 920 |================================================ ZFS RAIDZ3 12TB 5EA .. 961 |================================================== ZFS RAIDZ3 12TB 7EA .. 943 |================================================= IOzone 3.465 Record Size: 64Kb - File Size: 2GB - Disk Test: Write Performance MB/s > Higher Is Better ZFS RAIDZ2 12TB 4EA .. 151 |================ ZFS RAIDZ2 12TB 6EA .. 330 |================================== ZFS RAIDZ2 12TB 10EA . 495 |=================================================== ZFS RAIDZ3 12TB 5EA .. 138 |============== ZFS RAIDZ3 12TB 7EA .. 291 |============================== IOzone 3.465 Record Size: 64Kb - File Size: 4GB - Disk Test: Write Performance MB/s > Higher Is Better ZFS RAIDZ2 12TB 4EA .. 108 |============== ZFS RAIDZ2 12TB 6EA .. 260 |=================================== ZFS RAIDZ2 12TB 10EA . 382 |=================================================== ZFS RAIDZ3 12TB 5EA .. 112 |=============== ZFS RAIDZ3 12TB 7EA .. 244 |================================= IOzone 3.465 Record Size: 64Kb - File Size: 8GB - Disk Test: Write Performance MB/s > Higher Is Better ZFS RAIDZ2 12TB 4EA .. 160 |=================== ZFS RAIDZ2 12TB 6EA .. 323 |======================================= ZFS RAIDZ2 12TB 10EA . 426 |=================================================== ZFS RAIDZ3 12TB 5EA .. 158 |=================== ZFS RAIDZ3 12TB 7EA .. 275 |================================= IOzone 3.465 Record Size: 1MB - File Size: 512MB - Disk Test: Write Performance MB/s > Higher Is Better ZFS RAIDZ2 12TB 4EA .. 183 |================ ZFS RAIDZ2 12TB 6EA .. 330 |============================= ZFS RAIDZ2 12TB 10EA . 590 |=================================================== ZFS RAIDZ3 12TB 5EA .. 177 |=============== ZFS RAIDZ3 12TB 7EA .. 336 |============================= IOzone 3.465 Record Size: 4Kb - File Size: 512MB - Disk Test: Write Performance MB/s > Higher Is Better ZFS RAIDZ2 12TB 4EA .. 180 |================================== ZFS RAIDZ2 12TB 6EA .. 244 |============================================== ZFS RAIDZ2 12TB 10EA . 272 |=================================================== ZFS RAIDZ3 12TB 5EA .. 173 |================================ ZFS RAIDZ3 12TB 7EA .. 220 |========================================= IOzone 3.465 Record Size: 64Kb - File Size: 512MB - Disk Test: Read Performance MB/s > Higher Is Better ZFS RAIDZ2 12TB 4EA .. 2082 |==================================== ZFS RAIDZ2 12TB 6EA .. 2332 |========================================= ZFS RAIDZ2 12TB 10EA . 2857 |================================================== ZFS RAIDZ3 12TB 5EA .. 2633 |============================================== ZFS RAIDZ3 12TB 7EA .. 2579 |============================================= IOzone 3.465 Record Size: 64Kb - File Size: 512MB - Disk Test: Write Performance MB/s > Higher Is Better ZFS RAIDZ2 12TB 4EA .. 167 |================== ZFS RAIDZ2 12TB 6EA .. 344 |==================================== ZFS RAIDZ2 12TB 10EA . 486 |=================================================== ZFS RAIDZ3 12TB 5EA .. 157 |================ ZFS RAIDZ3 12TB 7EA .. 275 |=============================