AMD Opteron vs. AMD EPYC Server - Ubuntu Linux AMD EPYC 7601 32-Core testing compared to older AMD Opteron CPUs on Linux. Tests by Michael Larabel for a future article. AMD Opteron 2356: Processor: AMD Opteron 2356 @ 2.30GHz (4 Cores), Motherboard: TYAN S2932/S2932-E, Chipset: NVIDIA MCP55, Memory: 16384MB, Disk: 128GB ADATA SU800, Graphics: LLVMpipe, Audio: NVIDIA Device 0fb9 OS: Ubuntu 17.04, Kernel: 4.10.0-19-generic (x86_64), Desktop: Unity 7.5.0, Display Server: X Server 1.19.3, Display Driver: modesetting 1.19.3, OpenGL: 3.3 Mesa 17.0.3 Gallium 0.4 (LLVM 4.0 128 bits), Compiler: GCC 6.3.0 20170406, File-System: ext4, Screen Resolution: 640x480 AMD Opteron 2384: Processor: AMD Opteron 2384 @ 2.70GHz (4 Cores), Motherboard: TYAN S2932/S2932-E, Chipset: NVIDIA MCP55, Memory: 16384MB, Disk: 128GB ADATA SU800, Graphics: LLVMpipe, Audio: NVIDIA Device 0fb9 OS: Ubuntu 17.04, Kernel: 4.10.0-19-generic (x86_64), Desktop: Unity 7.5.0, Display Server: X Server 1.19.3, Display Driver: modesetting 1.19.3, OpenGL: 3.3 Mesa 17.0.3 Gallium 0.4 (LLVM 4.0 128 bits), Compiler: GCC 6.3.0 20170406, File-System: ext4, Screen Resolution: 640x480 2 x AMD Opteron 2384: Processor: 2 x AMD Opteron 2384 @ 2.70GHz (8 Cores), Motherboard: TYAN S2932/S2932-E, Chipset: NVIDIA MCP55, Memory: 16384MB, Disk: 128GB ADATA SU800, Graphics: LLVMpipe, Audio: NVIDIA Device 0fb9 OS: Ubuntu 17.04, Kernel: 4.10.0-19-generic (x86_64), Desktop: Unity 7.5.0, Display Server: X Server 1.19.3, Display Driver: modesetting 1.19.3, OpenGL: 3.3 Mesa 17.0.3 Gallium 0.4 (LLVM 4.0 128 bits), Compiler: GCC 6.3.0 20170406, File-System: ext4, Screen Resolution: 640x480 AMD EPYC 7601: Processor: AMD EPYC 7601 32-Core @ 2.20GHz (64 Cores), Motherboard: TYAN B8026T70AE24HR, Chipset: AMD Device 1450, Memory: 129024MB, Disk: 256GB Samsung SSD 850, Graphics: ASPEED ASPEED Family, Monitor: Acer P243W, Network: Broadcom Limited NetXtreme BCM5720 Gigabit PCIe OS: Ubuntu 17.04, Kernel: 4.10.0-19-generic (x86_64), Desktop: Unity 7.5.0, Display Driver: modesetting 1.19.3, Compiler: GCC 6.3.0 20170406, File-System: ext4, Screen Resolution: 1920x1200 AMD EPYC 7601 - NUMA Interleave All: Processor: AMD EPYC 7601 32-Core @ 2.20GHz (64 Cores), Motherboard: TYAN B8026T70AE24HR, Chipset: AMD Device 1450, Memory: 129024MB, Disk: 256GB Samsung SSD 850, Graphics: ASPEED ASPEED Family, Monitor: Acer P243W, Network: Broadcom Limited NetXtreme BCM5720 Gigabit PCIe OS: Ubuntu 17.04, Kernel: 4.10.0-19-generic (x86_64), Desktop: Unity 7.5.0, Display Driver: modesetting 1.19.3, Compiler: GCC 6.3.0 20170406, File-System: ext4, Screen Resolution: 1920x1200 Opteron6366HEvs1709158-TY-OPTYEPYC154: Processor: 4 x AMD Opteron 6366 HE @ 1.00GHz (32 Cores / 64 Threads), Motherboard: Supermicro H8QG6 (3.5b BIOS), Chipset: AMD RD890 + SB7x0/SB8x0/SB9x0, Memory: 258048MB, Disk: 2 x 250GB Western Digital WD2502ABYS-1 + 20GB SATA SSD, Graphics: Matrox MGA G200eW WPCM450, Monitor: ASUS VN247, Network: 2 x Intel 82576 OS: Ubuntu 18.04, Kernel: 4.15.0-54-generic (x86_64), Compiler: GCC 7.4.0, File-System: ext4, Screen Resolution: 1280x1024 NAS Parallel Benchmarks 3.3 Test / Class: LU.C Total Mop/s > Higher Is Better AMD Opteron 2356 ...................... 1030 |= AMD Opteron 2384 ...................... 512 | 2 x AMD Opteron 2384 .................. 1878 |= AMD EPYC 7601 ......................... 47673 |============================== AMD EPYC 7601 - NUMA Interleave All ... 50095 |================================ Opteron6366HEvs1709158-TY-OPTYEPYC154 . 19516 |============ OpenSSL 1.0.1g RSA 4096-bit Performance Signs Per Second Per Watt > Higher Is Better AMD Opteron 2356 .................... 1.68 |==== AMD Opteron 2384 .................... 2.24 |===== 2 x AMD Opteron 2384 ................ 2.67 |====== AMD EPYC 7601 ....................... 14.85 |================================== AMD EPYC 7601 - NUMA Interleave All . 13.50 |=============================== NAS Parallel Benchmarks 3.3 Test / Class: FT.B Total Mop/s > Higher Is Better AMD Opteron 2356 ...................... 694 |======== AMD Opteron 2384 ...................... 608 |======= 2 x AMD Opteron 2384 .................. 592 |======= AMD EPYC 7601 ......................... 1608 |=================== AMD EPYC 7601 - NUMA Interleave All ... 2866 |================================= Opteron6366HEvs1709158-TY-OPTYEPYC154 . 325 |==== RAMspeed SMP 3.5.0 Type: Add - Benchmark: Integer MB/s Per Watt > Higher Is Better AMD Opteron 2356 .................... 27.43 |==== AMD Opteron 2384 .................... 29.24 |==== 2 x AMD Opteron 2384 ................ 42.96 |====== AMD EPYC 7601 ....................... 215.92 |================================ AMD EPYC 7601 - NUMA Interleave All . 222.31 |================================= RAMspeed SMP 3.5.0 Type: Add - Benchmark: Integer MB/s > Higher Is Better AMD Opteron 2356 ...................... 4391 |==== AMD Opteron 2384 ...................... 4279 |==== 2 x AMD Opteron 2384 .................. 7987 |======= AMD EPYC 7601 ......................... 32306 |============================== AMD EPYC 7601 - NUMA Interleave All ... 34651 |================================ Opteron6366HEvs1709158-TY-OPTYEPYC154 . 6770 |====== RAMspeed SMP 3.5.0 Type: Triad - Benchmark: Integer MB/s Per Watt > Higher Is Better AMD Opteron 2356 .................... 26.58 |==== AMD Opteron 2384 .................... 28.66 |===== 2 x AMD Opteron 2384 ................ 41.62 |======= AMD EPYC 7601 ....................... 176.44 |============================ AMD EPYC 7601 - NUMA Interleave All . 207.60 |================================= RAMspeed SMP 3.5.0 Type: Scale - Benchmark: Integer MB/s Per Watt > Higher Is Better AMD Opteron 2356 .................... 24.87 |==== AMD Opteron 2384 .................... 26.85 |===== 2 x AMD Opteron 2384 ................ 38.88 |======= AMD EPYC 7601 ....................... 112.58 |==================== AMD EPYC 7601 - NUMA Interleave All . 188.42 |================================= RAMspeed SMP 3.5.0 Type: Copy - Benchmark: Integer MB/s Per Watt > Higher Is Better AMD Opteron 2356 .................... 26.28 |==== AMD Opteron 2384 .................... 27.69 |===== 2 x AMD Opteron 2384 ................ 42.91 |======= AMD EPYC 7601 ....................... 197.79 |================================= AMD EPYC 7601 - NUMA Interleave All . 197.74 |================================= RAMspeed SMP 3.5.0 Type: Triad - Benchmark: Integer MB/s > Higher Is Better AMD Opteron 2356 ...................... 4255 |==== AMD Opteron 2384 ...................... 4204 |==== 2 x AMD Opteron 2384 .................. 7766 |======== AMD EPYC 7601 ......................... 26213 |=========================== AMD EPYC 7601 - NUMA Interleave All ... 31130 |================================ Opteron6366HEvs1709158-TY-OPTYEPYC154 . 6751 |======= RAMspeed SMP 3.5.0 Type: Copy - Benchmark: Integer MB/s > Higher Is Better AMD Opteron 2356 ...................... 4200 |===== AMD Opteron 2384 ...................... 4023 |==== 2 x AMD Opteron 2384 .................. 7960 |========= AMD EPYC 7601 ......................... 28492 |=============================== AMD EPYC 7601 - NUMA Interleave All ... 29157 |================================ Opteron6366HEvs1709158-TY-OPTYEPYC154 . 6384 |======= RAMspeed SMP 3.5.0 Type: Scale - Benchmark: Integer MB/s > Higher Is Better AMD Opteron 2356 ...................... 3955 |===== AMD Opteron 2384 ...................... 3903 |==== 2 x AMD Opteron 2384 .................. 7144 |======== AMD EPYC 7601 ......................... 15602 |================== AMD EPYC 7601 - NUMA Interleave All ... 28053 |================================ Opteron6366HEvs1709158-TY-OPTYEPYC154 . 6140 |======= x264 2017-09-08 H.264 Video Encoding Frames Per Second > Higher Is Better AMD Opteron 2356 ...................... 41.84 |==== AMD Opteron 2384 ...................... 51.55 |====== 2 x AMD Opteron 2384 .................. 93.98 |========== AMD EPYC 7601 ......................... 289.05 |=============================== AMD EPYC 7601 - NUMA Interleave All ... 289.06 |=============================== Opteron6366HEvs1709158-TY-OPTYEPYC154 . 103.72 |=========== NAS Parallel Benchmarks 3.3 Test / Class: LU.C Total Mop/s Per Watt > Higher Is Better AMD Opteron 2356 .................... 5.72 |= AMD Opteron 2384 .................... 3.10 |= 2 x AMD Opteron 2384 ................ 7.33 |= AMD EPYC 7601 ....................... 158.56 |============================= AMD EPYC 7601 - NUMA Interleave All . 179.62 |================================= NAS Parallel Benchmarks 3.3 Test / Class: EP.C Total Mop/s > Higher Is Better AMD Opteron 2356 ...................... 40.35 |= AMD Opteron 2384 ...................... 41.75 |= 2 x AMD Opteron 2384 .................. 55.68 |= AMD EPYC 7601 ......................... 1647.70 |============================== AMD EPYC 7601 - NUMA Interleave All ... 1622.09 |============================== Opteron6366HEvs1709158-TY-OPTYEPYC154 . 414.93 |======== LAME MP3 Encoding 3.99.5 WAV To MP3 Seconds < Lower Is Better AMD Opteron 2356 ...................... 26.29 |=============== AMD Opteron 2384 ...................... 22.08 |============ 2 x AMD Opteron 2384 .................. 22.05 |============ AMD EPYC 7601 ......................... 11.47 |====== AMD EPYC 7601 - NUMA Interleave All ... 11.50 |====== Opteron6366HEvs1709158-TY-OPTYEPYC154 . 57.12 |================================ NAS Parallel Benchmarks 3.3 Test / Class: FT.B Total Mop/s Per Watt > Higher Is Better AMD Opteron 2356 .................... 3.88 |=========== AMD Opteron 2384 .................... 3.70 |=========== 2 x AMD Opteron 2384 ................ 2.50 |======= AMD EPYC 7601 ....................... 5.87 |================= AMD EPYC 7601 - NUMA Interleave All . 11.62 |================================== NAS Parallel Benchmarks 3.3 System Power Consumption Monitor Watts < Lower Is Better AMD Opteron 2356 .................... MIN: 174 AVG: 180 MAX: 181 AMD Opteron 2384 .................... MIN: 88 AVG: 165 MAX: 166 2 x AMD Opteron 2384 ................ MIN: 114 AVG: 256 MAX: 258 AMD EPYC 7601 ....................... MIN: 278 AVG: 301 MAX: 349 AMD EPYC 7601 - NUMA Interleave All . MIN: 146 AVG: 279 MAX: 291 NAS Parallel Benchmarks 3.3 System Power Consumption Monitor Watts < Lower Is Better AMD Opteron 2356 .................... MIN: 142 AVG: 179 MAX: 189 AMD Opteron 2384 .................... MIN: 89 AVG: 165 MAX: 176 2 x AMD Opteron 2384 ................ MIN: 115 AVG: 237 MAX: 255 AMD EPYC 7601 ....................... MIN: 135 AVG: 274 MAX: 324 AMD EPYC 7601 - NUMA Interleave All . MIN: 96 AVG: 247 MAX: 267 FFmpeg 3.3.3 H.264 HD To NTSC DV Seconds < Lower Is Better AMD Opteron 2356 ...................... 22.70 |===================== AMD Opteron 2384 ...................... 20.79 |=================== 2 x AMD Opteron 2384 .................. 15.75 |=============== AMD EPYC 7601 ......................... 10.98 |========== AMD EPYC 7601 - NUMA Interleave All ... 10.81 |========== Opteron6366HEvs1709158-TY-OPTYEPYC154 . 34.49 |================================ Xsbench 2017-07-06 System Power Consumption Monitor Watts < Lower Is Better AMD Opteron 2356 .................... MIN: 99 AVG: 158 MAX: 170 AMD Opteron 2384 .................... MIN: 107 AVG: 146 MAX: 159 2 x AMD Opteron 2384 ................ MIN: 114 AVG: 213 MAX: 225 AMD EPYC 7601 ....................... MIN: 107 AVG: 234 MAX: 278 AMD EPYC 7601 - NUMA Interleave All . MIN: 98 AVG: 224 MAX: 278 Timed Linux Kernel Compilation 4.9 System Power Consumption Monitor Watts < Lower Is Better AMD Opteron 2356 .................... MIN: 111 AVG: 169 MAX: 175 AMD Opteron 2384 .................... MIN: 99 AVG: 158 MAX: 163 2 x AMD Opteron 2384 ................ MIN: 114 AVG: 226 MAX: 249 AMD EPYC 7601 ....................... MIN: 121 AVG: 201 MAX: 280 AMD EPYC 7601 - NUMA Interleave All . MIN: 119 AVG: 205 MAX: 281 Stream 2013-01-17 System Power Consumption Monitor Watts < Lower Is Better AMD Opteron 2356 .................... MIN: 100 AVG: 174 MAX: 177 AMD Opteron 2384 .................... MIN: 89 AVG: 161 MAX: 162 2 x AMD Opteron 2384 ................ MIN: 113 AVG: 234 MAX: 246 AMD EPYC 7601 ....................... MIN: 95 AVG: 207 MAX: 238 AMD EPYC 7601 - NUMA Interleave All . MIN: 142 AVG: 206 MAX: 239 OpenSSL 1.0.1g System Power Consumption Monitor Watts < Lower Is Better AMD Opteron 2356 .................... MIN: 110 AVG: 157 MAX: 163 AMD Opteron 2384 .................... MIN: 98 AVG: 138 MAX: 149 2 x AMD Opteron 2384 ................ MIN: 194 AVG: 233 MAX: 238 AMD EPYC 7601 ....................... MIN: 97 AVG: 221 MAX: 242 AMD EPYC 7601 - NUMA Interleave All . MIN: 242 AVG: 242 MAX: 243 John The Ripper 1.8.0 System Power Consumption Monitor Watts < Lower Is Better AMD Opteron 2356 .................... MIN: 118 AVG: 151 MAX: 168 AMD Opteron 2384 .................... MIN: 110 AVG: 140 MAX: 153 2 x AMD Opteron 2384 ................ MIN: 136 AVG: 207 MAX: 244 AMD EPYC 7601 ....................... MIN: 121 AVG: 167 MAX: 243 AMD EPYC 7601 - NUMA Interleave All . MIN: 109 AVG: 183 MAX: 253 RAMspeed SMP 3.5.0 System Power Consumption Monitor Watts < Lower Is Better AMD Opteron 2356 .................... MIN: 100 AVG: 159 MAX: 161 AMD Opteron 2384 .................... MIN: 89 AVG: 145 MAX: 147 2 x AMD Opteron 2384 ................ MIN: 120 AVG: 184 MAX: 187 AMD EPYC 7601 ....................... MIN: 96 AVG: 139 MAX: 151 AMD EPYC 7601 - NUMA Interleave All . MIN: 96 AVG: 149 MAX: 160 Stream 2013-01-17 Type: Add MB/s > Higher Is Better AMD Opteron 2356 ...................... 4517 |== AMD Opteron 2384 ...................... 4624 |== 2 x AMD Opteron 2384 .................. 12005 |==== AMD EPYC 7601 ......................... 86806 |================================ AMD EPYC 7601 - NUMA Interleave All ... 75109 |============================ Opteron6366HEvs1709158-TY-OPTYEPYC154 . 59527 |====================== Stream 2013-01-17 Type: Triad MB/s > Higher Is Better AMD Opteron 2356 ...................... 4492 |== AMD Opteron 2384 ...................... 4701 |== 2 x AMD Opteron 2384 .................. 12074 |==== AMD EPYC 7601 ......................... 86296 |================================ AMD EPYC 7601 - NUMA Interleave All ... 75017 |============================ Opteron6366HEvs1709158-TY-OPTYEPYC154 . 59673 |====================== Stream 2013-01-17 Type: Scale MB/s > Higher Is Better AMD Opteron 2356 ...................... 4218 |== AMD Opteron 2384 ...................... 4101 |== 2 x AMD Opteron 2384 .................. 10958 |==== AMD EPYC 7601 ......................... 78452 |================================ AMD EPYC 7601 - NUMA Interleave All ... 68761 |============================ Opteron6366HEvs1709158-TY-OPTYEPYC154 . 56055 |======================= Xsbench 2017-07-06 Lookups/s > Higher Is Better AMD Opteron 2356 ...................... 396873 |== AMD Opteron 2384 ...................... 337774 |== 2 x AMD Opteron 2384 .................. 573891 |=== AMD EPYC 7601 ......................... 3871555 |==================== AMD EPYC 7601 - NUMA Interleave All ... 5711596 |============================== Opteron6366HEvs1709158-TY-OPTYEPYC154 . 1962525 |========== Stream 2013-01-17 Type: Copy MB/s > Higher Is Better AMD Opteron 2356 ...................... 7373 |== AMD Opteron 2384 ...................... 7558 |== 2 x AMD Opteron 2384 .................. 18828 |===== AMD EPYC 7601 ......................... 113651 |=============================== AMD EPYC 7601 - NUMA Interleave All ... 101900 |============================ Opteron6366HEvs1709158-TY-OPTYEPYC154 . 68704 |=================== Stream 2013-01-17 Type: Copy MB/s Per Watt > Higher Is Better AMD Opteron 2356 .................... 42.35 |=== AMD Opteron 2384 .................... 46.93 |=== 2 x AMD Opteron 2384 ................ 80.63 |===== AMD EPYC 7601 ....................... 550.37 |================================= AMD EPYC 7601 - NUMA Interleave All . 493.70 |============================== OpenSSL 1.0.1g RSA 4096-bit Performance Signs Per Second > Higher Is Better AMD Opteron 2356 ...................... 263 |=== AMD Opteron 2384 ...................... 310 |=== 2 x AMD Opteron 2384 .................. 622 |====== AMD EPYC 7601 ......................... 3280 |================================= AMD EPYC 7601 - NUMA Interleave All ... 3271 |================================= Opteron6366HEvs1709158-TY-OPTYEPYC154 . 1395 |============== John The Ripper 1.8.0 Test: Blowfish Real C/S > Higher Is Better AMD Opteron 2356 ...................... 2540 |=== AMD Opteron 2384 ...................... 3011 |=== 2 x AMD Opteron 2384 .................. 5963 |====== AMD EPYC 7601 ......................... 26338 |=========================== AMD EPYC 7601 - NUMA Interleave All ... 31260 |================================ Opteron6366HEvs1709158-TY-OPTYEPYC154 . 13200 |============== Timed Linux Kernel Compilation 4.9 Time To Compile Seconds < Lower Is Better AMD Opteron 2356 .................... 424.20 |================================= AMD Opteron 2384 .................... 331.32 |========================== 2 x AMD Opteron 2384 ................ 174.69 |============== AMD EPYC 7601 ....................... 38.43 |=== AMD EPYC 7601 - NUMA Interleave All . 40.28 |=== Xsbench 2017-07-06 Lookups/s Per Watt > Higher Is Better AMD Opteron 2356 .................... 2508.76 |=== AMD Opteron 2384 .................... 2311.47 |=== 2 x AMD Opteron 2384 ................ 2696.03 |=== AMD EPYC 7601 ....................... 16574.62 |==================== AMD EPYC 7601 - NUMA Interleave All . 25469.77 |=============================== John The Ripper 1.8.0 Test: Blowfish Real C/S Per Watt > Higher Is Better AMD Opteron 2356 .................... 16.79 |=== AMD Opteron 2384 .................... 21.53 |==== 2 x AMD Opteron 2384 ................ 28.85 |====== AMD EPYC 7601 ....................... 157.42 |============================== AMD EPYC 7601 - NUMA Interleave All . 170.58 |================================= FFmpeg 3.3.3 System Power Consumption Monitor Watts < Lower Is Better AMD Opteron 2356 .................... MIN: 155 AVG: 162 MAX: 165 AMD Opteron 2384 .................... MIN: 95 AVG: 138 MAX: 149 2 x AMD Opteron 2384 ................ MIN: 162 AVG: 182 MAX: 189 AMD EPYC 7601 ....................... MIN: 96 AVG: 128 MAX: 148 AMD EPYC 7601 - NUMA Interleave All . MIN: 95 AVG: 132 MAX: 151 RAMspeed SMP 3.5.0 System Power Consumption Monitor Watts < Lower Is Better AMD Opteron 2356 .................... MIN: 143 AVG: 160 MAX: 161 AMD Opteron 2384 .................... MIN: 144 AVG: 145 MAX: 147 2 x AMD Opteron 2384 ................ MIN: 182 AVG: 186 MAX: 187 AMD EPYC 7601 ....................... MIN: 112 AVG: 144 MAX: 151 AMD EPYC 7601 - NUMA Interleave All . MIN: 96 AVG: 147 MAX: 158 RAMspeed SMP 3.5.0 System Power Consumption Monitor Watts < Lower Is Better AMD Opteron 2356 .................... MIN: 159 AVG: 160 MAX: 161 AMD Opteron 2384 .................... MIN: 134 AVG: 147 MAX: 150 2 x AMD Opteron 2384 ................ MIN: 185 AVG: 187 MAX: 189 AMD EPYC 7601 ....................... MIN: 142 AVG: 149 MAX: 151 AMD EPYC 7601 - NUMA Interleave All . MIN: 112 AVG: 150 MAX: 159 LAME MP3 Encoding 3.99.5 System Power Consumption Monitor Watts < Lower Is Better AMD Opteron 2356 .................... MIN: 100 AVG: 119 MAX: 123 AMD Opteron 2384 .................... MIN: 87 AVG: 106 MAX: 109 2 x AMD Opteron 2384 ................ MIN: 125 AVG: 136 MAX: 143 AMD EPYC 7601 ....................... MIN: 117 AVG: 129 MAX: 133 AMD EPYC 7601 - NUMA Interleave All . MIN: 99 AVG: 124 MAX: 134 RAMspeed SMP 3.5.0 System Power Consumption Monitor Watts < Lower Is Better AMD Opteron 2356 .................... MIN: 152 AVG: 160 MAX: 161 AMD Opteron 2384 .................... MIN: 146 AVG: 146 MAX: 147 2 x AMD Opteron 2384 ................ MIN: 185 AVG: 186 MAX: 187 AMD EPYC 7601 ....................... MIN: 143 AVG: 150 MAX: 152 AMD EPYC 7601 - NUMA Interleave All . MIN: 139 AVG: 156 MAX: 160 Stream 2013-01-17 System Power Consumption Monitor Watts < Lower Is Better 2 x AMD Opteron 2384 . 122.8 |================================================= System Power Consumption Monitor Phoronix Test Suite System Monitoring Watts AMD Opteron 2356 .................... MIN: 98 AVG: 167 MAX: 203 AMD Opteron 2384 .................... MIN: 86 AVG: 152 MAX: 185 2 x AMD Opteron 2384 ................ MIN: 113 AVG: 209 MAX: 288 AMD EPYC 7601 ....................... MIN: 94 AVG: 180 MAX: 349 AMD EPYC 7601 - NUMA Interleave All . MIN: 95 AVG: 178 MAX: 358