AMD Opteron vs. AMD EPYC Server - Ubuntu Linux AMD EPYC 7601 32-Core testing compared to older AMD Opteron CPUs on Linux. Tests by Michael Larabel for a future article. AMD Opteron 2356: Processor: AMD Opteron 2356 @ 2.30GHz (4 Cores), Motherboard: TYAN S2932/S2932-E, Chipset: NVIDIA MCP55, Memory: 16384MB, Disk: 128GB ADATA SU800, Graphics: LLVMpipe, Audio: NVIDIA Device 0fb9 OS: Ubuntu 17.04, Kernel: 4.10.0-19-generic (x86_64), Desktop: Unity 7.5.0, Display Server: X Server 1.19.3, Display Driver: modesetting 1.19.3, OpenGL: 3.3 Mesa 17.0.3 Gallium 0.4 (LLVM 4.0 128 bits), Compiler: GCC 6.3.0 20170406, File-System: ext4, Screen Resolution: 640x480 AMD Opteron 2384: Processor: AMD Opteron 2384 @ 2.70GHz (4 Cores), Motherboard: TYAN S2932/S2932-E, Chipset: NVIDIA MCP55, Memory: 16384MB, Disk: 128GB ADATA SU800, Graphics: LLVMpipe, Audio: NVIDIA Device 0fb9 OS: Ubuntu 17.04, Kernel: 4.10.0-19-generic (x86_64), Desktop: Unity 7.5.0, Display Server: X Server 1.19.3, Display Driver: modesetting 1.19.3, OpenGL: 3.3 Mesa 17.0.3 Gallium 0.4 (LLVM 4.0 128 bits), Compiler: GCC 6.3.0 20170406, File-System: ext4, Screen Resolution: 640x480 2 x AMD Opteron 2384: Processor: 2 x AMD Opteron 2384 @ 2.70GHz (8 Cores), Motherboard: TYAN S2932/S2932-E, Chipset: NVIDIA MCP55, Memory: 16384MB, Disk: 128GB ADATA SU800, Graphics: LLVMpipe, Audio: NVIDIA Device 0fb9 OS: Ubuntu 17.04, Kernel: 4.10.0-19-generic (x86_64), Desktop: Unity 7.5.0, Display Server: X Server 1.19.3, Display Driver: modesetting 1.19.3, OpenGL: 3.3 Mesa 17.0.3 Gallium 0.4 (LLVM 4.0 128 bits), Compiler: GCC 6.3.0 20170406, File-System: ext4, Screen Resolution: 640x480 AMD EPYC 7601: Processor: AMD EPYC 7601 32-Core @ 2.20GHz (64 Cores), Motherboard: TYAN B8026T70AE24HR, Chipset: AMD Device 1450, Memory: 129024MB, Disk: 256GB Samsung SSD 850, Graphics: ASPEED ASPEED Family, Monitor: Acer P243W, Network: Broadcom Limited NetXtreme BCM5720 Gigabit PCIe OS: Ubuntu 17.04, Kernel: 4.10.0-19-generic (x86_64), Desktop: Unity 7.5.0, Display Driver: modesetting 1.19.3, Compiler: GCC 6.3.0 20170406, File-System: ext4, Screen Resolution: 1920x1200 AMD EPYC 7601 - NUMA Interleave All: Processor: AMD EPYC 7601 32-Core @ 2.20GHz (64 Cores), Motherboard: TYAN B8026T70AE24HR, Chipset: AMD Device 1450, Memory: 129024MB, Disk: 256GB Samsung SSD 850, Graphics: ASPEED ASPEED Family, Monitor: Acer P243W, Network: Broadcom Limited NetXtreme BCM5720 Gigabit PCIe OS: Ubuntu 17.04, Kernel: 4.10.0-19-generic (x86_64), Desktop: Unity 7.5.0, Display Driver: modesetting 1.19.3, Compiler: GCC 6.3.0 20170406, File-System: ext4, Screen Resolution: 1920x1200 AMD Opteron 6276 4P HT Assist On: Processor: 4 x AMD Opteron 6276 @ 2.30GHz (32 Cores / 64 Threads), Motherboard: Dell 0272WF (3.2.2 BIOS), Chipset: AMD RD890S/SR5650 + SB7x0/SB8x0/SB9x0, Memory: 129024MB, Disk: 479GB PERC H700 + 62GB Ultra, Graphics: Matrox MGA G200eW WPCM450, Network: Broadcom Limited NetXtreme II BCM5709 Gigabit OS: Ubuntu 18.04, Kernel: 4.15.0-34-generic (x86_64), Compiler: GCC 7.3.0, File-System: ext4, Screen Resolution: 1024x768 NAS Parallel Benchmarks 3.3 Test / Class: EP.C Total Mop/s > Higher Is Better AMD Opteron 2356 .................... 40.35 |= AMD Opteron 2384 .................... 41.75 |= 2 x AMD Opteron 2384 ................ 55.68 |= AMD EPYC 7601 ....................... 1647.70 |================================ AMD EPYC 7601 - NUMA Interleave All . 1622.09 |================================ AMD Opteron 6276 4P HT Assist On .... 816.67 |================ NAS Parallel Benchmarks 3.3 Test / Class: FT.B Total Mop/s > Higher Is Better AMD Opteron 2356 .................... 694 |======== AMD Opteron 2384 .................... 608 |======= 2 x AMD Opteron 2384 ................ 592 |======= AMD EPYC 7601 ....................... 1608 |==================== AMD EPYC 7601 - NUMA Interleave All . 2866 |=================================== AMD Opteron 6276 4P HT Assist On .... 790 |========== NAS Parallel Benchmarks 3.3 Test / Class: FT.B Total Mop/s Per Watt > Higher Is Better AMD Opteron 2356 .................... 3.88 |=========== AMD Opteron 2384 .................... 3.70 |=========== 2 x AMD Opteron 2384 ................ 2.50 |======= AMD EPYC 7601 ....................... 5.87 |================= AMD EPYC 7601 - NUMA Interleave All . 11.62 |================================== NAS Parallel Benchmarks 3.3 System Power Consumption Monitor Watts < Lower Is Better AMD Opteron 2356 .................... MIN: 142 AVG: 179 MAX: 189 AMD Opteron 2384 .................... MIN: 89 AVG: 165 MAX: 176 2 x AMD Opteron 2384 ................ MIN: 115 AVG: 237 MAX: 255 AMD EPYC 7601 ....................... MIN: 135 AVG: 274 MAX: 324 AMD EPYC 7601 - NUMA Interleave All . MIN: 96 AVG: 247 MAX: 267 NAS Parallel Benchmarks 3.3 Test / Class: LU.C Total Mop/s > Higher Is Better AMD Opteron 2356 .................... 1030 |= AMD Opteron 2384 .................... 512 | 2 x AMD Opteron 2384 ................ 1878 |= AMD EPYC 7601 ....................... 47673 |================================ AMD EPYC 7601 - NUMA Interleave All . 50095 |================================== AMD Opteron 6276 4P HT Assist On .... 30700 |===================== NAS Parallel Benchmarks 3.3 Test / Class: LU.C Total Mop/s Per Watt > Higher Is Better AMD Opteron 2356 .................... 5.72 |= AMD Opteron 2384 .................... 3.10 |= 2 x AMD Opteron 2384 ................ 7.33 |= AMD EPYC 7601 ....................... 158.56 |============================= AMD EPYC 7601 - NUMA Interleave All . 179.62 |================================= NAS Parallel Benchmarks 3.3 System Power Consumption Monitor Watts < Lower Is Better AMD Opteron 2356 .................... MIN: 174 AVG: 180 MAX: 181 AMD Opteron 2384 .................... MIN: 88 AVG: 165 MAX: 166 2 x AMD Opteron 2384 ................ MIN: 114 AVG: 256 MAX: 258 AMD EPYC 7601 ....................... MIN: 278 AVG: 301 MAX: 349 AMD EPYC 7601 - NUMA Interleave All . MIN: 146 AVG: 279 MAX: 291 John The Ripper 1.8.0 Test: Blowfish Real C/S > Higher Is Better AMD Opteron 2356 .................... 2540 |=== AMD Opteron 2384 .................... 3011 |=== 2 x AMD Opteron 2384 ................ 5963 |====== AMD EPYC 7601 ....................... 26338 |============================= AMD EPYC 7601 - NUMA Interleave All . 31260 |================================== AMD Opteron 6276 4P HT Assist On .... 31118 |================================== John The Ripper 1.8.0 Test: Blowfish Real C/S Per Watt > Higher Is Better AMD Opteron 2356 .................... 16.79 |=== AMD Opteron 2384 .................... 21.53 |==== 2 x AMD Opteron 2384 ................ 28.85 |====== AMD EPYC 7601 ....................... 157.42 |============================== AMD EPYC 7601 - NUMA Interleave All . 170.58 |================================= John The Ripper 1.8.0 System Power Consumption Monitor Watts < Lower Is Better AMD Opteron 2356 .................... MIN: 118 AVG: 151 MAX: 168 AMD Opteron 2384 .................... MIN: 110 AVG: 140 MAX: 153 2 x AMD Opteron 2384 ................ MIN: 136 AVG: 207 MAX: 244 AMD EPYC 7601 ....................... MIN: 121 AVG: 167 MAX: 243 AMD EPYC 7601 - NUMA Interleave All . MIN: 109 AVG: 183 MAX: 253 OpenSSL 1.0.1g RSA 4096-bit Performance Signs Per Second > Higher Is Better AMD Opteron 2356 .................... 263 |=== AMD Opteron 2384 .................... 310 |=== 2 x AMD Opteron 2384 ................ 622 |======= AMD EPYC 7601 ....................... 3280 |=================================== AMD EPYC 7601 - NUMA Interleave All . 3271 |=================================== AMD Opteron 6276 4P HT Assist On .... 3113 |================================= OpenSSL 1.0.1g RSA 4096-bit Performance Signs Per Second Per Watt > Higher Is Better AMD Opteron 2356 .................... 1.68 |==== AMD Opteron 2384 .................... 2.24 |===== 2 x AMD Opteron 2384 ................ 2.67 |====== AMD EPYC 7601 ....................... 14.85 |================================== AMD EPYC 7601 - NUMA Interleave All . 13.50 |=============================== OpenSSL 1.0.1g System Power Consumption Monitor Watts < Lower Is Better AMD Opteron 2356 .................... MIN: 110 AVG: 157 MAX: 163 AMD Opteron 2384 .................... MIN: 98 AVG: 138 MAX: 149 2 x AMD Opteron 2384 ................ MIN: 194 AVG: 233 MAX: 238 AMD EPYC 7601 ....................... MIN: 97 AVG: 221 MAX: 242 AMD EPYC 7601 - NUMA Interleave All . MIN: 242 AVG: 242 MAX: 243 x264 2017-09-08 H.264 Video Encoding Frames Per Second > Higher Is Better AMD Opteron 2356 .................... 41.84 |===== AMD Opteron 2384 .................... 51.55 |====== 2 x AMD Opteron 2384 ................ 93.98 |=========== AMD EPYC 7601 ....................... 289.05 |================================= AMD EPYC 7601 - NUMA Interleave All . 289.06 |================================= AMD Opteron 6276 4P HT Assist On .... 209.74 |======================== LAME MP3 Encoding 3.99.5 WAV To MP3 Seconds < Lower Is Better AMD Opteron 2356 .................... 26.29 |================================== AMD Opteron 2384 .................... 22.08 |============================= 2 x AMD Opteron 2384 ................ 22.05 |============================= AMD EPYC 7601 ....................... 11.47 |=============== AMD EPYC 7601 - NUMA Interleave All . 11.50 |=============== AMD Opteron 6276 4P HT Assist On .... 18.87 |======================== LAME MP3 Encoding 3.99.5 System Power Consumption Monitor Watts < Lower Is Better AMD Opteron 2356 .................... MIN: 100 AVG: 119 MAX: 123 AMD Opteron 2384 .................... MIN: 87 AVG: 106 MAX: 109 2 x AMD Opteron 2384 ................ MIN: 125 AVG: 136 MAX: 143 AMD EPYC 7601 ....................... MIN: 117 AVG: 129 MAX: 133 AMD EPYC 7601 - NUMA Interleave All . MIN: 99 AVG: 124 MAX: 134 Xsbench 2017-07-06 Lookups/s > Higher Is Better AMD Opteron 2356 .................... 396873 |== AMD Opteron 2384 .................... 337774 |== 2 x AMD Opteron 2384 ................ 573891 |=== AMD EPYC 7601 ....................... 3871555 |====================== AMD EPYC 7601 - NUMA Interleave All . 5711596 |================================ AMD Opteron 6276 4P HT Assist On .... 2424038 |============== Xsbench 2017-07-06 Lookups/s Per Watt > Higher Is Better AMD Opteron 2356 .................... 2508.76 |=== AMD Opteron 2384 .................... 2311.47 |=== 2 x AMD Opteron 2384 ................ 2696.03 |=== AMD EPYC 7601 ....................... 16574.62 |==================== AMD EPYC 7601 - NUMA Interleave All . 25469.77 |=============================== Xsbench 2017-07-06 System Power Consumption Monitor Watts < Lower Is Better AMD Opteron 2356 .................... MIN: 99 AVG: 158 MAX: 170 AMD Opteron 2384 .................... MIN: 107 AVG: 146 MAX: 159 2 x AMD Opteron 2384 ................ MIN: 114 AVG: 213 MAX: 225 AMD EPYC 7601 ....................... MIN: 107 AVG: 234 MAX: 278 AMD EPYC 7601 - NUMA Interleave All . MIN: 98 AVG: 224 MAX: 278 Timed Linux Kernel Compilation 4.9 Time To Compile Seconds < Lower Is Better AMD Opteron 2356 .................... 424.20 |================================= AMD Opteron 2384 .................... 331.32 |========================== 2 x AMD Opteron 2384 ................ 174.69 |============== AMD EPYC 7601 ....................... 38.43 |=== AMD EPYC 7601 - NUMA Interleave All . 40.28 |=== Timed Linux Kernel Compilation 4.9 System Power Consumption Monitor Watts < Lower Is Better AMD Opteron 2356 .................... MIN: 111 AVG: 169 MAX: 175 AMD Opteron 2384 .................... MIN: 99 AVG: 158 MAX: 163 2 x AMD Opteron 2384 ................ MIN: 114 AVG: 226 MAX: 249 AMD EPYC 7601 ....................... MIN: 121 AVG: 201 MAX: 280 AMD EPYC 7601 - NUMA Interleave All . MIN: 119 AVG: 205 MAX: 281 FFmpeg 3.3.3 H.264 HD To NTSC DV Seconds < Lower Is Better AMD Opteron 2356 .................... 22.70 |================================== AMD Opteron 2384 .................... 20.79 |=============================== 2 x AMD Opteron 2384 ................ 15.75 |======================== AMD EPYC 7601 ....................... 10.98 |================ AMD EPYC 7601 - NUMA Interleave All . 10.81 |================ AMD Opteron 6276 4P HT Assist On .... 16.67 |========================= FFmpeg 3.3.3 System Power Consumption Monitor Watts < Lower Is Better AMD Opteron 2356 .................... MIN: 155 AVG: 162 MAX: 165 AMD Opteron 2384 .................... MIN: 95 AVG: 138 MAX: 149 2 x AMD Opteron 2384 ................ MIN: 162 AVG: 182 MAX: 189 AMD EPYC 7601 ....................... MIN: 96 AVG: 128 MAX: 148 AMD EPYC 7601 - NUMA Interleave All . MIN: 95 AVG: 132 MAX: 151 Stream 2013-01-17 Type: Copy MB/s > Higher Is Better AMD Opteron 2356 .................... 7373 |== AMD Opteron 2384 .................... 7558 |== 2 x AMD Opteron 2384 ................ 18828 |===== AMD EPYC 7601 ....................... 113651 |================================= AMD EPYC 7601 - NUMA Interleave All . 101900 |============================== AMD Opteron 6276 4P HT Assist On .... 104138 |============================== Stream 2013-01-17 Type: Copy MB/s Per Watt > Higher Is Better AMD Opteron 2356 .................... 42.35 |=== AMD Opteron 2384 .................... 46.93 |=== 2 x AMD Opteron 2384 ................ 80.63 |===== AMD EPYC 7601 ....................... 550.37 |================================= AMD EPYC 7601 - NUMA Interleave All . 493.70 |============================== Stream 2013-01-17 System Power Consumption Monitor Watts < Lower Is Better AMD Opteron 2356 .................... MIN: 100 AVG: 174 MAX: 177 AMD Opteron 2384 .................... MIN: 89 AVG: 161 MAX: 162 2 x AMD Opteron 2384 ................ MIN: 113 AVG: 234 MAX: 246 AMD EPYC 7601 ....................... MIN: 95 AVG: 207 MAX: 238 AMD EPYC 7601 - NUMA Interleave All . MIN: 142 AVG: 206 MAX: 239 Stream 2013-01-17 Type: Scale MB/s > Higher Is Better AMD Opteron 2356 .................... 4218 |== AMD Opteron 2384 .................... 4101 |== 2 x AMD Opteron 2384 ................ 10958 |===== AMD EPYC 7601 ....................... 78452 |================================== AMD EPYC 7601 - NUMA Interleave All . 68761 |============================== AMD Opteron 6276 4P HT Assist On .... 61820 |=========================== Stream 2013-01-17 Type: Add MB/s > Higher Is Better AMD Opteron 2356 .................... 4517 |== AMD Opteron 2384 .................... 4624 |== 2 x AMD Opteron 2384 ................ 12005 |===== AMD EPYC 7601 ....................... 86806 |================================== AMD EPYC 7601 - NUMA Interleave All . 75109 |============================= AMD Opteron 6276 4P HT Assist On .... 67851 |=========================== Stream 2013-01-17 Type: Triad MB/s > Higher Is Better AMD Opteron 2356 .................... 4492 |== AMD Opteron 2384 .................... 4701 |== 2 x AMD Opteron 2384 ................ 12074 |===== AMD EPYC 7601 ....................... 86296 |================================== AMD EPYC 7601 - NUMA Interleave All . 75017 |============================== AMD Opteron 6276 4P HT Assist On .... 67908 |=========================== Stream 2013-01-17 System Power Consumption Monitor Watts < Lower Is Better 2 x AMD Opteron 2384 . 122.8 |================================================= RAMspeed SMP 3.5.0 Type: Copy - Benchmark: Integer MB/s > Higher Is Better AMD Opteron 2356 .................... 4200.48 |==== AMD Opteron 2384 .................... 4023.25 |==== 2 x AMD Opteron 2384 ................ 7960.18 |======== AMD EPYC 7601 ....................... 28491.99 |============================== AMD EPYC 7601 - NUMA Interleave All . 29156.82 |=============================== RAMspeed SMP 3.5.0 Type: Copy - Benchmark: Integer MB/s Per Watt > Higher Is Better AMD Opteron 2356 .................... 26.28 |==== AMD Opteron 2384 .................... 27.69 |===== 2 x AMD Opteron 2384 ................ 42.91 |======= AMD EPYC 7601 ....................... 197.79 |================================= AMD EPYC 7601 - NUMA Interleave All . 197.74 |================================= RAMspeed SMP 3.5.0 System Power Consumption Monitor Watts < Lower Is Better AMD Opteron 2356 .................... MIN: 143 AVG: 160 MAX: 161 AMD Opteron 2384 .................... MIN: 144 AVG: 145 MAX: 147 2 x AMD Opteron 2384 ................ MIN: 182 AVG: 186 MAX: 187 AMD EPYC 7601 ....................... MIN: 112 AVG: 144 MAX: 151 AMD EPYC 7601 - NUMA Interleave All . MIN: 96 AVG: 147 MAX: 158 RAMspeed SMP 3.5.0 Type: Scale - Benchmark: Integer MB/s > Higher Is Better AMD Opteron 2356 .................... 3954.60 |==== AMD Opteron 2384 .................... 3903.34 |==== 2 x AMD Opteron 2384 ................ 7144.05 |======== AMD EPYC 7601 ....................... 15602.17 |================= AMD EPYC 7601 - NUMA Interleave All . 28052.87 |=============================== RAMspeed SMP 3.5.0 Type: Scale - Benchmark: Integer MB/s Per Watt > Higher Is Better AMD Opteron 2356 .................... 24.87 |==== AMD Opteron 2384 .................... 26.85 |===== 2 x AMD Opteron 2384 ................ 38.88 |======= AMD EPYC 7601 ....................... 112.58 |==================== AMD EPYC 7601 - NUMA Interleave All . 188.42 |================================= RAMspeed SMP 3.5.0 System Power Consumption Monitor Watts < Lower Is Better AMD Opteron 2356 .................... MIN: 100 AVG: 159 MAX: 161 AMD Opteron 2384 .................... MIN: 89 AVG: 145 MAX: 147 2 x AMD Opteron 2384 ................ MIN: 120 AVG: 184 MAX: 187 AMD EPYC 7601 ....................... MIN: 96 AVG: 139 MAX: 151 AMD EPYC 7601 - NUMA Interleave All . MIN: 96 AVG: 149 MAX: 160 RAMspeed SMP 3.5.0 Type: Add - Benchmark: Integer MB/s > Higher Is Better AMD Opteron 2356 .................... 4390.54 |==== AMD Opteron 2384 .................... 4279.40 |==== 2 x AMD Opteron 2384 ................ 7987.36 |======= AMD EPYC 7601 ....................... 32305.69 |============================= AMD EPYC 7601 - NUMA Interleave All . 34651.41 |=============================== RAMspeed SMP 3.5.0 Type: Add - Benchmark: Integer MB/s Per Watt > Higher Is Better AMD Opteron 2356 .................... 27.43 |==== AMD Opteron 2384 .................... 29.24 |==== 2 x AMD Opteron 2384 ................ 42.96 |====== AMD EPYC 7601 ....................... 215.92 |================================ AMD EPYC 7601 - NUMA Interleave All . 222.31 |================================= RAMspeed SMP 3.5.0 System Power Consumption Monitor Watts < Lower Is Better AMD Opteron 2356 .................... MIN: 152 AVG: 160 MAX: 161 AMD Opteron 2384 .................... MIN: 146 AVG: 146 MAX: 147 2 x AMD Opteron 2384 ................ MIN: 185 AVG: 186 MAX: 187 AMD EPYC 7601 ....................... MIN: 143 AVG: 150 MAX: 152 AMD EPYC 7601 - NUMA Interleave All . MIN: 139 AVG: 156 MAX: 160 RAMspeed SMP 3.5.0 Type: Triad - Benchmark: Integer MB/s > Higher Is Better AMD Opteron 2356 .................... 4254.78 |==== AMD Opteron 2384 .................... 4204.06 |==== 2 x AMD Opteron 2384 ................ 7765.50 |======== AMD EPYC 7601 ....................... 26213.39 |========================== AMD EPYC 7601 - NUMA Interleave All . 31129.52 |=============================== RAMspeed SMP 3.5.0 Type: Triad - Benchmark: Integer MB/s Per Watt > Higher Is Better AMD Opteron 2356 .................... 26.58 |==== AMD Opteron 2384 .................... 28.66 |===== 2 x AMD Opteron 2384 ................ 41.62 |======= AMD EPYC 7601 ....................... 176.44 |============================ AMD EPYC 7601 - NUMA Interleave All . 207.60 |================================= RAMspeed SMP 3.5.0 System Power Consumption Monitor Watts < Lower Is Better AMD Opteron 2356 .................... MIN: 159 AVG: 160 MAX: 161 AMD Opteron 2384 .................... MIN: 134 AVG: 147 MAX: 150 2 x AMD Opteron 2384 ................ MIN: 185 AVG: 187 MAX: 189 AMD EPYC 7601 ....................... MIN: 142 AVG: 149 MAX: 151 AMD EPYC 7601 - NUMA Interleave All . MIN: 112 AVG: 150 MAX: 159 System Power Consumption Monitor Phoronix Test Suite System Monitoring Watts AMD Opteron 2356 .................... MIN: 98 AVG: 167 MAX: 203 AMD Opteron 2384 .................... MIN: 86 AVG: 152 MAX: 185 2 x AMD Opteron 2384 ................ MIN: 113 AVG: 209 MAX: 288 AMD EPYC 7601 ....................... MIN: 94 AVG: 180 MAX: 349 AMD EPYC 7601 - NUMA Interleave All . MIN: 95 AVG: 178 MAX: 358