201906040855 2 x Intel Xeon E5-2620 v4 testing with a Intel S2600CWR (SE5C610.86B.01.01.0024.021320181901 BIOS) and Matrox MGA G200e [Pilot] on CentOS Linux 7 via the Phoronix Test Suite.
HTML result view exported from: https://openbenchmarking.org/result/1906059-SDS-2019060402 .
201906040855 Processor Motherboard Memory Disk Graphics Chipset Network OS Kernel Compiler File-System Screen Resolution ZFS RAIDZ2 12TB 4EA ZFS RAIDZ2 12TB 6EA ZFS RAIDZ2 12TB 10EA ZFS RAIDZ3 12TB 5EA ZFS RAIDZ3 12TB 7EA 2 x Intel Xeon E5-2620 v4 @ 3.00GHz (16 Cores / 32 Threads) Intel S2600CWR (SE5C610.86B.01.01.0024.021320181901 BIOS) 4 x 32 GB DDR4-2134MT/s Samsung M393A4K40CB2-CTD 7 x 12000GB Seagate ST12000NM0027 + 239GB Logical Volume mgadrmfb CentOS Linux 7 3.10.0-957.el7.x86_64 (x86_64) GCC 4.8.5 20150623 zfs 1024x768 Intel Xeon E7 v4/Xeon 11 x 12000GB Seagate ST12000NM0027 + 239GB Logical Volume Matrox MGA G200e [Pilot] 2 x Intel I350 + 2 x Intel 82599ES 10-Gigabit SFI/SFP+ OpenBenchmarking.org Compiler Details - --build=x86_64-redhat-linux --disable-libgcj --disable-libunwind-exceptions --enable-__cxa_atexit --enable-bootstrap --enable-checking=release --enable-gnu-indirect-function --enable-gnu-unique-object --enable-initfini-array --enable-languages=c,c++,objc,obj-c++,java,fortran,ada,go,lto --enable-plugin --enable-shared --enable-threads=posix --mandir=/usr/share/man --with-arch_32=x86-64 --with-linker-hash-style=gnu --with-tune=generic Disk Details - DEADLINE / attr2,inode64,noquota,relatime,rw,seclabel Processor Details - Scaling Governor: intel_pstate powersave Security Details - SELinux + l1tf: Mitigation of PTE Inversion + meltdown: Mitigation of PTI + spec_store_bypass: Vulnerable + spectre_v1: Mitigation of Load fences __user pointer sanitization + spectre_v2: Mitigation of Full retpoline
201906040855 iozone: 1MB - 2GB - Read Performance iozone: 1MB - 4GB - Read Performance iozone: 1MB - 8GB - Read Performance iozone: 4Kb - 2GB - Read Performance iozone: 4Kb - 4GB - Read Performance iozone: 4Kb - 8GB - Read Performance iozone: 1MB - 2GB - Write Performance iozone: 1MB - 4GB - Write Performance iozone: 1MB - 8GB - Write Performance iozone: 4Kb - 2GB - Write Performance iozone: 4Kb - 4GB - Write Performance iozone: 4Kb - 8GB - Write Performance iozone: 64Kb - 2GB - Read Performance iozone: 64Kb - 4GB - Read Performance iozone: 64Kb - 8GB - Read Performance iozone: 1MB - 512MB - Read Performance iozone: 4Kb - 512MB - Read Performance iozone: 64Kb - 2GB - Write Performance iozone: 64Kb - 4GB - Write Performance iozone: 64Kb - 8GB - Write Performance iozone: 1MB - 512MB - Write Performance iozone: 4Kb - 512MB - Write Performance iozone: 64Kb - 512MB - Read Performance iozone: 64Kb - 512MB - Write Performance ZFS RAIDZ2 12TB 4EA ZFS RAIDZ2 12TB 6EA ZFS RAIDZ2 12TB 10EA ZFS RAIDZ3 12TB 5EA ZFS RAIDZ3 12TB 7EA 2149 2225 2279 992 1050 1106 181 165 206 205 227 240 1947 1963 2124 2842 979 151 108 160 183 180 2082 167 2299 2172 2374 1003 1057 1098 337 335 377 315 344 363 2147 2109 2104 3192 945 330 260 323 330 244 2332 344 2557 2243 2270 1019 1091 1097 565 503 516 346 343 376 2309 2124 2017 2932 920 495 382 426 590 272 2857 486 2199 2224 2263 1030 1048 1093 177 170 216 209 241 266 1993 1953 2173 3321 961 138 112 158 177 173 2633 157 2375 2209 2354 1024 1046 1103 360 323 366 306 325 370 1949 2115 2105 3169 943 291 244 275 336 220 2579 275 OpenBenchmarking.org
IOzone Record Size: 1MB - File Size: 2GB - Disk Test: Read Performance OpenBenchmarking.org MB/s, More Is Better IOzone 3.465 Record Size: 1MB - File Size: 2GB - Disk Test: Read Performance ZFS RAIDZ2 12TB 4EA ZFS RAIDZ2 12TB 6EA ZFS RAIDZ2 12TB 10EA ZFS RAIDZ3 12TB 5EA ZFS RAIDZ3 12TB 7EA 500 1000 1500 2000 2500 SE +/- 26.57, N = 15 SE +/- 91.27, N = 15 SE +/- 114.77, N = 15 SE +/- 21.51, N = 3 SE +/- 144.36, N = 12 2149 2299 2557 2199 2375 1. (CC) gcc options: -O3
IOzone Record Size: 1MB - File Size: 4GB - Disk Test: Read Performance OpenBenchmarking.org MB/s, More Is Better IOzone 3.465 Record Size: 1MB - File Size: 4GB - Disk Test: Read Performance ZFS RAIDZ2 12TB 4EA ZFS RAIDZ2 12TB 6EA ZFS RAIDZ2 12TB 10EA ZFS RAIDZ3 12TB 5EA ZFS RAIDZ3 12TB 7EA 500 1000 1500 2000 2500 SE +/- 25.59, N = 15 SE +/- 27.63, N = 15 SE +/- 26.71, N = 15 SE +/- 26.29, N = 3 SE +/- 29.41, N = 15 2225 2172 2243 2224 2209 1. (CC) gcc options: -O3
IOzone Record Size: 1MB - File Size: 8GB - Disk Test: Read Performance OpenBenchmarking.org MB/s, More Is Better IOzone 3.465 Record Size: 1MB - File Size: 8GB - Disk Test: Read Performance ZFS RAIDZ2 12TB 4EA ZFS RAIDZ2 12TB 6EA ZFS RAIDZ2 12TB 10EA ZFS RAIDZ3 12TB 5EA ZFS RAIDZ3 12TB 7EA 500 1000 1500 2000 2500 SE +/- 33.97, N = 15 SE +/- 17.55, N = 3 SE +/- 50.20, N = 12 SE +/- 36.92, N = 12 SE +/- 17.80, N = 3 2279 2374 2270 2263 2354 1. (CC) gcc options: -O3
IOzone Record Size: 4Kb - File Size: 2GB - Disk Test: Read Performance OpenBenchmarking.org MB/s, More Is Better IOzone 3.465 Record Size: 4Kb - File Size: 2GB - Disk Test: Read Performance ZFS RAIDZ2 12TB 4EA ZFS RAIDZ2 12TB 6EA ZFS RAIDZ2 12TB 10EA ZFS RAIDZ3 12TB 5EA ZFS RAIDZ3 12TB 7EA 200 400 600 800 1000 SE +/- 11.55, N = 6 SE +/- 15.98, N = 15 SE +/- 14.55, N = 15 SE +/- 19.75, N = 15 SE +/- 9.51, N = 3 992 1003 1019 1030 1024 1. (CC) gcc options: -O3
IOzone Record Size: 4Kb - File Size: 4GB - Disk Test: Read Performance OpenBenchmarking.org MB/s, More Is Better IOzone 3.465 Record Size: 4Kb - File Size: 4GB - Disk Test: Read Performance ZFS RAIDZ2 12TB 4EA ZFS RAIDZ2 12TB 6EA ZFS RAIDZ2 12TB 10EA ZFS RAIDZ3 12TB 5EA ZFS RAIDZ3 12TB 7EA 200 400 600 800 1000 SE +/- 16.52, N = 15 SE +/- 8.02, N = 3 SE +/- 14.10, N = 4 SE +/- 18.44, N = 15 SE +/- 11.77, N = 6 1050 1057 1091 1048 1046 1. (CC) gcc options: -O3
IOzone Record Size: 4Kb - File Size: 8GB - Disk Test: Read Performance OpenBenchmarking.org MB/s, More Is Better IOzone 3.465 Record Size: 4Kb - File Size: 8GB - Disk Test: Read Performance ZFS RAIDZ2 12TB 4EA ZFS RAIDZ2 12TB 6EA ZFS RAIDZ2 12TB 10EA ZFS RAIDZ3 12TB 5EA ZFS RAIDZ3 12TB 7EA 200 400 600 800 1000 SE +/- 12.41, N = 15 SE +/- 10.83, N = 15 SE +/- 11.39, N = 15 SE +/- 13.04, N = 15 SE +/- 11.49, N = 15 1106 1098 1097 1093 1103 1. (CC) gcc options: -O3
IOzone Record Size: 1MB - File Size: 2GB - Disk Test: Write Performance OpenBenchmarking.org MB/s, More Is Better IOzone 3.465 Record Size: 1MB - File Size: 2GB - Disk Test: Write Performance ZFS RAIDZ2 12TB 4EA ZFS RAIDZ2 12TB 6EA ZFS RAIDZ2 12TB 10EA ZFS RAIDZ3 12TB 5EA ZFS RAIDZ3 12TB 7EA 120 240 360 480 600 SE +/- 2.17, N = 15 SE +/- 5.45, N = 3 SE +/- 12.76, N = 15 SE +/- 3.58, N = 15 SE +/- 11.37, N = 15 181 337 565 177 360 1. (CC) gcc options: -O3
IOzone Record Size: 1MB - File Size: 4GB - Disk Test: Write Performance OpenBenchmarking.org MB/s, More Is Better IOzone 3.465 Record Size: 1MB - File Size: 4GB - Disk Test: Write Performance ZFS RAIDZ2 12TB 4EA ZFS RAIDZ2 12TB 6EA ZFS RAIDZ2 12TB 10EA ZFS RAIDZ3 12TB 5EA ZFS RAIDZ3 12TB 7EA 110 220 330 440 550 SE +/- 7.65, N = 15 SE +/- 4.11, N = 3 SE +/- 4.39, N = 3 SE +/- 1.97, N = 6 SE +/- 12.48, N = 15 165 335 503 170 323 1. (CC) gcc options: -O3
IOzone Record Size: 1MB - File Size: 8GB - Disk Test: Write Performance OpenBenchmarking.org MB/s, More Is Better IOzone 3.465 Record Size: 1MB - File Size: 8GB - Disk Test: Write Performance ZFS RAIDZ2 12TB 4EA ZFS RAIDZ2 12TB 6EA ZFS RAIDZ2 12TB 10EA ZFS RAIDZ3 12TB 5EA ZFS RAIDZ3 12TB 7EA 110 220 330 440 550 SE +/- 2.06, N = 3 SE +/- 9.54, N = 15 SE +/- 3.46, N = 3 SE +/- 3.29, N = 3 SE +/- 10.62, N = 12 206 377 516 216 366 1. (CC) gcc options: -O3
IOzone Record Size: 4Kb - File Size: 2GB - Disk Test: Write Performance OpenBenchmarking.org MB/s, More Is Better IOzone 3.465 Record Size: 4Kb - File Size: 2GB - Disk Test: Write Performance ZFS RAIDZ2 12TB 4EA ZFS RAIDZ2 12TB 6EA ZFS RAIDZ2 12TB 10EA ZFS RAIDZ3 12TB 5EA ZFS RAIDZ3 12TB 7EA 80 160 240 320 400 SE +/- 11.08, N = 12 SE +/- 6.50, N = 15 SE +/- 2.85, N = 3 SE +/- 9.35, N = 15 SE +/- 9.46, N = 15 205 315 346 209 306 1. (CC) gcc options: -O3
IOzone Record Size: 4Kb - File Size: 4GB - Disk Test: Write Performance OpenBenchmarking.org MB/s, More Is Better IOzone 3.465 Record Size: 4Kb - File Size: 4GB - Disk Test: Write Performance ZFS RAIDZ2 12TB 4EA ZFS RAIDZ2 12TB 6EA ZFS RAIDZ2 12TB 10EA ZFS RAIDZ3 12TB 5EA ZFS RAIDZ3 12TB 7EA 70 140 210 280 350 SE +/- 10.61, N = 15 SE +/- 11.21, N = 12 SE +/- 7.25, N = 15 SE +/- 7.45, N = 15 SE +/- 10.94, N = 15 227 344 343 241 325 1. (CC) gcc options: -O3
IOzone Record Size: 4Kb - File Size: 8GB - Disk Test: Write Performance OpenBenchmarking.org MB/s, More Is Better IOzone 3.465 Record Size: 4Kb - File Size: 8GB - Disk Test: Write Performance ZFS RAIDZ2 12TB 4EA ZFS RAIDZ2 12TB 6EA ZFS RAIDZ2 12TB 10EA ZFS RAIDZ3 12TB 5EA ZFS RAIDZ3 12TB 7EA 80 160 240 320 400 SE +/- 1.36, N = 3 SE +/- 8.28, N = 12 SE +/- 3.92, N = 15 SE +/- 11.34, N = 15 SE +/- 8.54, N = 15 240 363 376 266 370 1. (CC) gcc options: -O3
IOzone Record Size: 64Kb - File Size: 2GB - Disk Test: Read Performance OpenBenchmarking.org MB/s, More Is Better IOzone 3.465 Record Size: 64Kb - File Size: 2GB - Disk Test: Read Performance ZFS RAIDZ2 12TB 4EA ZFS RAIDZ2 12TB 6EA ZFS RAIDZ2 12TB 10EA ZFS RAIDZ3 12TB 5EA ZFS RAIDZ3 12TB 7EA 500 1000 1500 2000 2500 SE +/- 36.06, N = 15 SE +/- 132.09, N = 15 SE +/- 76.01, N = 15 SE +/- 33.83, N = 15 SE +/- 48.81, N = 15 1947 2147 2309 1993 1949 1. (CC) gcc options: -O3
IOzone Record Size: 64Kb - File Size: 4GB - Disk Test: Read Performance OpenBenchmarking.org MB/s, More Is Better IOzone 3.465 Record Size: 64Kb - File Size: 4GB - Disk Test: Read Performance ZFS RAIDZ2 12TB 4EA ZFS RAIDZ2 12TB 6EA ZFS RAIDZ2 12TB 10EA ZFS RAIDZ3 12TB 5EA ZFS RAIDZ3 12TB 7EA 500 1000 1500 2000 2500 SE +/- 31.87, N = 3 SE +/- 30.77, N = 3 SE +/- 12.73, N = 3 SE +/- 17.96, N = 3 SE +/- 32.00, N = 3 1963 2109 2124 1953 2115 1. (CC) gcc options: -O3
IOzone Record Size: 64Kb - File Size: 8GB - Disk Test: Read Performance OpenBenchmarking.org MB/s, More Is Better IOzone 3.465 Record Size: 64Kb - File Size: 8GB - Disk Test: Read Performance ZFS RAIDZ2 12TB 4EA ZFS RAIDZ2 12TB 6EA ZFS RAIDZ2 12TB 10EA ZFS RAIDZ3 12TB 5EA ZFS RAIDZ3 12TB 7EA 500 1000 1500 2000 2500 SE +/- 31.23, N = 12 SE +/- 22.93, N = 15 SE +/- 28.44, N = 3 SE +/- 27.70, N = 15 SE +/- 20.37, N = 3 2124 2104 2017 2173 2105 1. (CC) gcc options: -O3
IOzone Record Size: 1MB - File Size: 512MB - Disk Test: Read Performance OpenBenchmarking.org MB/s, More Is Better IOzone 3.465 Record Size: 1MB - File Size: 512MB - Disk Test: Read Performance ZFS RAIDZ2 12TB 4EA ZFS RAIDZ2 12TB 6EA ZFS RAIDZ2 12TB 10EA ZFS RAIDZ3 12TB 5EA ZFS RAIDZ3 12TB 7EA 700 1400 2100 2800 3500 SE +/- 102.87, N = 15 SE +/- 222.42, N = 15 SE +/- 166.08, N = 15 SE +/- 29.96, N = 3 SE +/- 175.56, N = 15 2842 3192 2932 3321 3169 1. (CC) gcc options: -O3
IOzone Record Size: 4Kb - File Size: 512MB - Disk Test: Read Performance OpenBenchmarking.org MB/s, More Is Better IOzone 3.465 Record Size: 4Kb - File Size: 512MB - Disk Test: Read Performance ZFS RAIDZ2 12TB 4EA ZFS RAIDZ2 12TB 6EA ZFS RAIDZ2 12TB 10EA ZFS RAIDZ3 12TB 5EA ZFS RAIDZ3 12TB 7EA 200 400 600 800 1000 SE +/- 28.43, N = 15 SE +/- 44.29, N = 15 SE +/- 11.02, N = 15 SE +/- 28.65, N = 15 SE +/- 20.10, N = 12 979 945 920 961 943 1. (CC) gcc options: -O3
IOzone Record Size: 64Kb - File Size: 2GB - Disk Test: Write Performance OpenBenchmarking.org MB/s, More Is Better IOzone 3.465 Record Size: 64Kb - File Size: 2GB - Disk Test: Write Performance ZFS RAIDZ2 12TB 4EA ZFS RAIDZ2 12TB 6EA ZFS RAIDZ2 12TB 10EA ZFS RAIDZ3 12TB 5EA ZFS RAIDZ3 12TB 7EA 110 220 330 440 550 SE +/- 2.09, N = 3 SE +/- 24.53, N = 12 SE +/- 34.64, N = 15 SE +/- 3.84, N = 12 SE +/- 23.11, N = 12 151 330 495 138 291 1. (CC) gcc options: -O3
IOzone Record Size: 64Kb - File Size: 4GB - Disk Test: Write Performance OpenBenchmarking.org MB/s, More Is Better IOzone 3.465 Record Size: 64Kb - File Size: 4GB - Disk Test: Write Performance ZFS RAIDZ2 12TB 4EA ZFS RAIDZ2 12TB 6EA ZFS RAIDZ2 12TB 10EA ZFS RAIDZ3 12TB 5EA ZFS RAIDZ3 12TB 7EA 80 160 240 320 400 SE +/- 2.89, N = 15 SE +/- 21.18, N = 12 SE +/- 15.36, N = 15 SE +/- 2.77, N = 12 SE +/- 7.10, N = 14 108 260 382 112 244 1. (CC) gcc options: -O3
IOzone Record Size: 64Kb - File Size: 8GB - Disk Test: Write Performance OpenBenchmarking.org MB/s, More Is Better IOzone 3.465 Record Size: 64Kb - File Size: 8GB - Disk Test: Write Performance ZFS RAIDZ2 12TB 4EA ZFS RAIDZ2 12TB 6EA ZFS RAIDZ2 12TB 10EA ZFS RAIDZ3 12TB 5EA ZFS RAIDZ3 12TB 7EA 90 180 270 360 450 SE +/- 1.40, N = 15 SE +/- 20.72, N = 15 SE +/- 22.32, N = 12 SE +/- 1.11, N = 3 SE +/- 4.00, N = 15 160 323 426 158 275 1. (CC) gcc options: -O3
IOzone Record Size: 1MB - File Size: 512MB - Disk Test: Write Performance OpenBenchmarking.org MB/s, More Is Better IOzone 3.465 Record Size: 1MB - File Size: 512MB - Disk Test: Write Performance ZFS RAIDZ2 12TB 4EA ZFS RAIDZ2 12TB 6EA ZFS RAIDZ2 12TB 10EA ZFS RAIDZ3 12TB 5EA ZFS RAIDZ3 12TB 7EA 130 260 390 520 650 SE +/- 4.55, N = 15 SE +/- 12.49, N = 15 SE +/- 21.44, N = 12 SE +/- 8.53, N = 12 SE +/- 15.73, N = 15 183 330 590 177 336 1. (CC) gcc options: -O3
IOzone Record Size: 4Kb - File Size: 512MB - Disk Test: Write Performance OpenBenchmarking.org MB/s, More Is Better IOzone 3.465 Record Size: 4Kb - File Size: 512MB - Disk Test: Write Performance ZFS RAIDZ2 12TB 4EA ZFS RAIDZ2 12TB 6EA ZFS RAIDZ2 12TB 10EA ZFS RAIDZ3 12TB 5EA ZFS RAIDZ3 12TB 7EA 60 120 180 240 300 SE +/- 9.97, N = 15 SE +/- 6.50, N = 15 SE +/- 5.14, N = 15 SE +/- 10.57, N = 15 SE +/- 7.96, N = 15 180 244 272 173 220 1. (CC) gcc options: -O3
IOzone Record Size: 64Kb - File Size: 512MB - Disk Test: Read Performance OpenBenchmarking.org MB/s, More Is Better IOzone 3.465 Record Size: 64Kb - File Size: 512MB - Disk Test: Read Performance ZFS RAIDZ2 12TB 4EA ZFS RAIDZ2 12TB 6EA ZFS RAIDZ2 12TB 10EA ZFS RAIDZ3 12TB 5EA ZFS RAIDZ3 12TB 7EA 600 1200 1800 2400 3000 SE +/- 125.45, N = 12 SE +/- 181.31, N = 12 SE +/- 237.99, N = 15 SE +/- 62.09, N = 15 SE +/- 241.60, N = 15 2082 2332 2857 2633 2579 1. (CC) gcc options: -O3
IOzone Record Size: 64Kb - File Size: 512MB - Disk Test: Write Performance OpenBenchmarking.org MB/s, More Is Better IOzone 3.465 Record Size: 64Kb - File Size: 512MB - Disk Test: Write Performance ZFS RAIDZ2 12TB 4EA ZFS RAIDZ2 12TB 6EA ZFS RAIDZ2 12TB 10EA ZFS RAIDZ3 12TB 5EA ZFS RAIDZ3 12TB 7EA 110 220 330 440 550 SE +/- 2.31, N = 3 SE +/- 23.20, N = 15 SE +/- 35.15, N = 12 SE +/- 1.72, N = 3 SE +/- 20.72, N = 12 167 344 486 157 275 1. (CC) gcc options: -O3
Phoronix Test Suite v10.8.4