Ubuntu 18.04 + Linux 4.19-rc1 - Xeon vs. EPYC AMD EPYC 7601 32-Core testing for a future article. Xeon Gold 6138: Default: Processor: 2 x Intel Xeon Gold 6138 @ 3.70GHz (40 Cores / 80 Threads), Motherboard: TYAN S7106 (V1.01 BIOS), Chipset: Intel Sky Lake-E DMI3 Registers, Memory: 96256MB, Disk: Samsung SSD 970 EVO 250GB, Graphics: ASPEED ASPEED Family, Monitor: VE228, Network: Intel I210 Gigabit Connection OS: Ubuntu 18.04, Kernel: 4.19.0-041900rc1-generic (x86_64) 20180826, Desktop: GNOME Shell 3.28.3, Display Server: X Server 1.19.6, Display Driver: modesetting 1.19.6, Compiler: GCC 7.3.0, File-System: ext4, Screen Resolution: 1920x1080 Xeon Gold: Mitigations Disabled: Processor: 2 x Intel Xeon Gold 6138 @ 3.70GHz (40 Cores / 80 Threads), Motherboard: TYAN S7106 (V1.01 BIOS), Chipset: Intel Sky Lake-E DMI3 Registers, Memory: 96256MB, Disk: Samsung SSD 970 EVO 250GB, Graphics: ASPEED ASPEED Family, Monitor: VE228, Network: Intel I210 Gigabit Connection OS: Ubuntu 18.04, Kernel: 4.19.0-041900rc1-generic (x86_64) 20180826, Desktop: GNOME Shell 3.28.3, Display Server: X Server 1.19.6, Display Driver: modesetting 1.19.6, Compiler: GCC 7.3.0, File-System: ext4, Screen Resolution: 1920x1080 Xeon Gold VM: Default: Processor: 64 x Intel Xeon (Skylake IBRS) @ 2.00GHz (64 Cores), Motherboard: QEMU Standard PC (i440FX + PIIX 1996) (1.10.2-1ubuntu1 BIOS), Chipset: Intel 440FX- 82441FX PMC, Memory: 49152MB, Disk: 113GB, Graphics: Red Hat QXL paravirtual graphic card, Audio: QEMU Generic, Network: Red Hat Virtio device OS: Ubuntu 18.04, Kernel: 4.19.0-041900rc1-generic (x86_64) 20180826, Desktop: GNOME Shell 3.28.3, Display Server: X Server 1.19.6, Display Driver: modesetting 1.19.6, Compiler: GCC 7.3.0, File-System: ext4, Screen Resolution: 1024x768, System Layer: KVM Xeon Gold VM: Mitigations Disabled: Processor: 64 x Intel Xeon (Skylake IBRS) @ 2.00GHz (64 Cores), Motherboard: QEMU Standard PC (i440FX + PIIX 1996) (1.10.2-1ubuntu1 BIOS), Chipset: Intel 440FX- 82441FX PMC, Memory: 49152MB, Disk: 113GB, Graphics: Red Hat QXL paravirtual graphic card, Audio: QEMU Generic, Network: Red Hat Virtio device OS: Ubuntu 18.04, Kernel: 4.19.0-041900rc1-generic (x86_64) 20180826, Desktop: GNOME Shell 3.28.3, Display Server: X Server 1.19.6, Display Driver: modesetting 1.19.6, Compiler: GCC 7.3.0, File-System: ext4, Screen Resolution: 1024x768, System Layer: KVM EPYC 7601: Default: Processor: AMD EPYC 7601 32-Core @ 2.20GHz (32 Cores / 64 Threads), Motherboard: TYAN B8026T70AE24HR (V1.02.B10 BIOS), Chipset: AMD Family 17h, Memory: 129024MB, Disk: 280GB INTEL SSDPE21D280GA, Graphics: ASPEED ASPEED Family, Monitor: VE228, Network: Broadcom Limited NetXtreme BCM5720 Gigabit PCIe OS: Ubuntu 18.04, Kernel: 4.19.0-041900rc1-generic (x86_64) 20180826, Desktop: GNOME Shell 3.28.2, Display Server: X Server 1.19.6, Display Driver: modesetting 1.19.6, Compiler: GCC 7.3.0, File-System: ext4, Screen Resolution: 1920x1080 EPYC 7601: Mitigations Disabled: Processor: AMD EPYC 7601 32-Core @ 2.20GHz (32 Cores / 64 Threads), Motherboard: TYAN B8026T70AE24HR (V1.02.B10 BIOS), Chipset: AMD Family 17h, Memory: 129024MB, Disk: 280GB INTEL SSDPE21D280GA, Graphics: ASPEED ASPEED Family, Monitor: VE228, Network: Broadcom Limited NetXtreme BCM5720 Gigabit PCIe OS: Ubuntu 18.04, Kernel: 4.19.0-041900rc1-generic (x86_64) 20180826, Desktop: GNOME Shell 3.28.2, Display Server: X Server 1.19.6, Display Driver: modesetting 1.19.6, Compiler: GCC 7.3.0, File-System: ext4, Screen Resolution: 1920x1080 FS-Mark 3.3 Test: 1000 Files, 1MB Size Files/s > Higher Is Better Xeon Gold 6138: Default ............ 181.79 |========== Xeon Gold: Mitigations Disabled .... 141.41 |======== Xeon Gold VM: Default .............. 89.20 |===== Xeon Gold VM: Mitigations Disabled . 101.92 |====== EPYC 7601: Default ................. 557.27 |================================ EPYC 7601: Mitigations Disabled .... 594.83 |================================== FS-Mark 3.3 Test: 4000 Files, 32 Sub Dirs, 1MB Size Files/s > Higher Is Better Xeon Gold 6138: Default ............ 33.67 |== Xeon Gold: Mitigations Disabled .... 48.33 |=== Xeon Gold VM: Default .............. 42.33 |== Xeon Gold VM: Mitigations Disabled . 46.70 |=== EPYC 7601: Default ................. 570.50 |================================ EPYC 7601: Mitigations Disabled .... 605.47 |================================== Compile Bench 0.6 Test: Compile MB/s > Higher Is Better Xeon Gold 6138: Default ............ 1719.39 |=============================== Xeon Gold: Mitigations Disabled .... 1827.66 |================================= Xeon Gold VM: Default .............. 1339.61 |======================== Xeon Gold VM: Mitigations Disabled . 1513.36 |=========================== EPYC 7601: Default ................. 1633.97 |============================== EPYC 7601: Mitigations Disabled .... 1707.90 |=============================== Compile Bench 0.6 Test: Initial Create MB/s > Higher Is Better Xeon Gold 6138: Default ............ 535.06 |=============================== Xeon Gold: Mitigations Disabled .... 593.49 |================================== Xeon Gold VM: Default .............. 450.47 |========================== Xeon Gold VM: Mitigations Disabled . 480.25 |============================ EPYC 7601: Default ................. 392.71 |====================== EPYC 7601: Mitigations Disabled .... 402.68 |======================= Compile Bench 0.6 Test: Read Compiled Tree MB/s > Higher Is Better Xeon Gold 6138: Default ............ 2497.40 |============================= Xeon Gold: Mitigations Disabled .... 2800.95 |================================= Xeon Gold VM: Default .............. 2196.27 |========================== Xeon Gold VM: Mitigations Disabled . 2556.98 |============================== EPYC 7601: Default ................. 2404.97 |============================ EPYC 7601: Mitigations Disabled .... 2501.18 |============================= Tinymembench 2018-05-28 Standard Memset MB/s > Higher Is Better Xeon Gold 6138: Default ......... 9361.23 |==================================== Xeon Gold: Mitigations Disabled . 9160.33 |=================================== Go Benchmarks Test: http Nanoseconds/Operation < Lower Is Better Xeon Gold 6138: Default ............ 8513 |============== Xeon Gold: Mitigations Disabled .... 8474 |============== Xeon Gold VM: Default .............. 13787 |======================= Xeon Gold VM: Mitigations Disabled . 13574 |====================== EPYC 7601: Default ................. 21196 |=================================== EPYC 7601: Mitigations Disabled .... 21193 |=================================== Go Benchmarks Test: json Nanoseconds/Operation < Lower Is Better Xeon Gold 6138: Default ............ 2613153 |========================== Xeon Gold: Mitigations Disabled .... 2551080 |========================== Xeon Gold VM: Default .............. 2417470 |======================== Xeon Gold VM: Mitigations Disabled . 2382902 |======================== EPYC 7601: Default ................. 3275376 |================================= EPYC 7601: Mitigations Disabled .... 3140715 |================================ Go Benchmarks Test: build Nanoseconds/Operation < Lower Is Better Xeon Gold 6138: Default ............ 20937768065 |=========================== Xeon Gold: Mitigations Disabled .... 20882740113 |=========================== Xeon Gold VM: Default .............. 21561391496 |=========================== Xeon Gold VM: Mitigations Disabled . 21491356147 |=========================== EPYC 7601: Default ................. 22755638036 |============================= EPYC 7601: Mitigations Disabled .... 22658988247 |============================= Go Benchmarks Test: garbage Nanoseconds/Operation < Lower Is Better Xeon Gold 6138: Default ............ 858908 |================================ Xeon Gold: Mitigations Disabled .... 844869 |=============================== Xeon Gold VM: Default .............. 923605 |================================== Xeon Gold VM: Mitigations Disabled . 922905 |================================== EPYC 7601: Default ................. 871931 |================================ EPYC 7601: Mitigations Disabled .... 869260 |================================ LuaJIT 2.0.5 Test: Composite Mflops > Higher Is Better Xeon Gold 6138: Default ............ 1353.27 |================================= Xeon Gold: Mitigations Disabled .... 1353.74 |================================= Xeon Gold VM: Default .............. 1323.36 |================================ Xeon Gold VM: Mitigations Disabled . 1314.01 |================================ EPYC 7601: Default ................. 1130.28 |============================ EPYC 7601: Mitigations Disabled .... 1132.40 |============================ Timed Linux Kernel Compilation 4.18 Time To Compile Seconds < Lower Is Better Xeon Gold 6138: Default ............ 32.61 |============================ Xeon Gold: Mitigations Disabled .... 32.19 |=========================== Xeon Gold VM: Default .............. 37.05 |================================ Xeon Gold VM: Mitigations Disabled . 36.50 |=============================== EPYC 7601: Default ................. 41.02 |=================================== EPYC 7601: Mitigations Disabled .... 41.09 |=================================== Timed LLVM Compilation 6.0.1 Time To Compile Seconds < Lower Is Better Xeon Gold 6138: Default ............ 170.07 |========================= Xeon Gold: Mitigations Disabled .... 167.34 |========================= Xeon Gold VM: Default .............. 210.52 |=============================== Xeon Gold VM: Mitigations Disabled . 201.36 |============================== EPYC 7601: Default ................. 225.81 |================================= EPYC 7601: Mitigations Disabled .... 230.54 |================================== Hackbench Count: 32 - Type: Process Seconds < Lower Is Better Xeon Gold 6138: Default ............ 35.51 |================= Xeon Gold: Mitigations Disabled .... 38.28 |================== Xeon Gold VM: Default .............. 73.10 |=================================== Xeon Gold VM: Mitigations Disabled . 60.37 |============================= EPYC 7601: Default ................. 28.24 |============== EPYC 7601: Mitigations Disabled .... 29.20 |============== PostgreSQL pgbench 10.3 Scaling: Buffer Test - Test: Normal Load - Mode: Read Only TPS > Higher Is Better Xeon Gold 6138: Default ............ 580318.53 |============================== Xeon Gold: Mitigations Disabled .... 609717.46 |=============================== Xeon Gold VM: Default .............. 381149.45 |=================== Xeon Gold VM: Mitigations Disabled . 410005.21 |===================== EPYC 7601: Default ................. 396825.40 |==================== EPYC 7601: Mitigations Disabled .... 398195.96 |==================== GIMP 2.8.22 Test: resize Seconds < Lower Is Better Xeon Gold 6138: Default ............ 15.00 |=================================== Xeon Gold: Mitigations Disabled .... 12.88 |============================== Xeon Gold VM: Default .............. 10.98 |========================== Xeon Gold VM: Mitigations Disabled . 10.59 |========================= EPYC 7601: Default ................. 10.66 |========================= EPYC 7601: Mitigations Disabled .... 10.40 |======================== GIMP 2.8.22 Test: rotate Seconds < Lower Is Better Xeon Gold 6138: Default ............ 26.55 |=================================== Xeon Gold: Mitigations Disabled .... 23.96 |================================ Xeon Gold VM: Default .............. 24.72 |================================= Xeon Gold VM: Mitigations Disabled . 21.99 |============================= EPYC 7601: Default ................. 22.22 |============================= EPYC 7601: Mitigations Disabled .... 21.97 |============================= GIMP 2.8.22 Test: auto-levels Seconds < Lower Is Better Xeon Gold 6138: Default ............ 27.14 |=================================== Xeon Gold: Mitigations Disabled .... 24.33 |=============================== Xeon Gold VM: Default .............. 25.75 |================================= Xeon Gold VM: Mitigations Disabled . 23.32 |============================== EPYC 7601: Default ................. 21.96 |============================ EPYC 7601: Mitigations Disabled .... 21.54 |============================ GIMP 2.8.22 Test: unsharp-mask Seconds < Lower Is Better Xeon Gold 6138: Default ............ 55.04 |=================================== Xeon Gold: Mitigations Disabled .... 47.99 |============================== Xeon Gold VM: Default .............. 55.24 |=================================== Xeon Gold VM: Mitigations Disabled . 49.71 |=============================== EPYC 7601: Default ................. 49.52 |=============================== EPYC 7601: Mitigations Disabled .... 49.07 |=============================== Redis 4.0.8 Test: SET Requests Per Second > Higher Is Better Xeon Gold 6138: Default ............ 1674690.92 |============================= Xeon Gold: Mitigations Disabled .... 1758574.02 |============================== Xeon Gold VM: Default .............. 1440300.91 |========================= Xeon Gold VM: Mitigations Disabled . 1534292.49 |========================== EPYC 7601: Default ................. 1335771.71 |======================= EPYC 7601: Mitigations Disabled .... 1343290.71 |======================= Stress-NG 0.07.26 Test: Semaphores Bogo Ops/s > Higher Is Better Xeon Gold 6138: Default ............ 1844314.50 |============================= Xeon Gold: Mitigations Disabled .... 1929446.14 |============================== Xeon Gold VM: Default .............. 575288.16 |========= Xeon Gold VM: Mitigations Disabled . 581783.33 |========= EPYC 7601: Default ................. 1500494.71 |======================= EPYC 7601: Mitigations Disabled .... 1508383.30 |======================= Stress-NG 0.07.26 Test: Socket Activity Bogo Ops/s > Higher Is Better Xeon Gold 6138: Default ............ 19412.60 |=========================== Xeon Gold: Mitigations Disabled .... 23350.32 |================================ Xeon Gold VM: Default .............. 13806.50 |=================== Xeon Gold VM: Mitigations Disabled . 14187.34 |=================== EPYC 7601: Default ................. 18231.14 |========================= EPYC 7601: Mitigations Disabled .... 18808.58 |========================== Stress-NG 0.07.26 Test: Context Switching Bogo Ops/s > Higher Is Better Xeon Gold 6138: Default ............ 21384939.00 |============================= Xeon Gold: Mitigations Disabled .... 14117739.60 |=================== Xeon Gold VM: Default .............. 12038250.91 |================ Xeon Gold VM: Mitigations Disabled . 15103597.63 |==================== EPYC 7601: Default ................. 12582879.48 |================= EPYC 7601: Mitigations Disabled .... 14596757.68 |==================== Stress-NG 0.07.26 Test: System V Message Passing Bogo Ops/s > Higher Is Better Xeon Gold 6138: Default ............ 6356626.36 |=============== Xeon Gold: Mitigations Disabled .... 6294347.72 |=============== Xeon Gold VM: Default .............. 11184789.67 |=========================== Xeon Gold VM: Mitigations Disabled . 12206790.21 |============================= EPYC 7601: Default ................. 10719745.17 |========================= EPYC 7601: Mitigations Disabled .... 10965490.72 |========================== Blender 2.79a Blend File: Barbershop - Compute: CPU-Only Seconds < Lower Is Better Xeon Gold 6138: Default ............ 375.50 |========================= Xeon Gold: Mitigations Disabled .... 375.02 |========================= Xeon Gold VM: Default .............. 424.76 |============================= Xeon Gold VM: Mitigations Disabled . 419.07 |============================ EPYC 7601: Default ................. 504.73 |================================== EPYC 7601: Mitigations Disabled .... 503.21 |================================== NGINX Benchmark 1.9.9 Static Web Page Serving Requests Per Second > Higher Is Better Xeon Gold 6138: Default ............ 29934.52 |========================= Xeon Gold: Mitigations Disabled .... 37582.46 |================================ Xeon Gold VM: Default .............. 28083.61 |======================== Xeon Gold VM: Mitigations Disabled . 30464.94 |========================== EPYC 7601: Default ................. 23712.72 |==================== EPYC 7601: Mitigations Disabled .... 23880.50 |==================== Apache Benchmark 2.4.29 Static Web Page Serving Requests Per Second > Higher Is Better Xeon Gold 6138: Default ............ 23475.91 |=========================== Xeon Gold: Mitigations Disabled .... 27681.13 |================================ Xeon Gold VM: Default .............. 23902.44 |============================ Xeon Gold VM: Mitigations Disabled . 27221.00 |=============================== EPYC 7601: Default ................. 16650.66 |=================== EPYC 7601: Mitigations Disabled .... 16592.90 |=================== Apache Siege 2.4.29 Concurrent Users: 250 Transactions Per Second > Higher Is Better Xeon Gold 6138: Default ............ 29419.71 |============================== Xeon Gold: Mitigations Disabled .... 30905.77 |=============================== Xeon Gold VM: Mitigations Disabled . 31650.35 |================================ EPYC 7601: Default ................. 25958.02 |========================== EPYC 7601: Mitigations Disabled .... 26967.24 |=========================== Git Time To Complete Common Git Commands Seconds < Lower Is Better Xeon Gold 6138: Default ............ 6.20 |============================== Xeon Gold: Mitigations Disabled .... 6.18 |============================== Xeon Gold VM: Default .............. 7.13 |=================================== Xeon Gold VM: Mitigations Disabled . 6.87 |================================== EPYC 7601: Default ................. 7.35 |==================================== EPYC 7601: Mitigations Disabled .... 7.19 |=================================== Systemd Total Boot Time Test: Total ms < Lower Is Better Xeon Gold 6138: Default ............ 28059 |=================================== Xeon Gold: Mitigations Disabled .... 26056 |================================= Xeon Gold VM: Default .............. 8062 |========== Xeon Gold VM: Mitigations Disabled . 9073 |=========== EPYC 7601: Default ................. 27047 |================================== EPYC 7601: Mitigations Disabled .... 27055 |================================== Systemd Total Boot Time Test: Kernel ms < Lower Is Better Xeon Gold 6138: Default ............ 4442 |================================= Xeon Gold: Mitigations Disabled .... 4345 |================================ Xeon Gold VM: Default .............. 2742 |==================== Xeon Gold VM: Mitigations Disabled . 2893 |===================== EPYC 7601: Default ................. 4649 |================================== EPYC 7601: Mitigations Disabled .... 4859 |==================================== Systemd Total Boot Time Test: Loader ms < Lower Is Better Xeon Gold 6138: Default ......... 10489 |====================================== EPYC 7601: Default .............. 10534 |====================================== EPYC 7601: Mitigations Disabled . 10367 |===================================== Systemd Total Boot Time Test: Firmware ms < Lower Is Better Xeon Gold 6138: Default ......... 17649 |============= EPYC 7601: Default .............. 2867 |== EPYC 7601: Mitigations Disabled . 50528 |====================================== Systemd Total Boot Time Test: Userspace ms < Lower Is Better Xeon Gold 6138: Default ............ 23617 |=================================== Xeon Gold: Mitigations Disabled .... 21711 |================================ Xeon Gold VM: Default .............. 5320 |======== Xeon Gold VM: Mitigations Disabled . 6180 |========= EPYC 7601: Default ................. 22398 |================================= EPYC 7601: Mitigations Disabled .... 22196 |================================= OSBench Test: Create Files us Per Event < Lower Is Better Xeon Gold 6138: Default ............ 15.68 |============================== Xeon Gold: Mitigations Disabled .... 13.46 |========================= Xeon Gold VM: Default .............. 16.14 |============================== Xeon Gold VM: Mitigations Disabled . 14.16 |=========================== EPYC 7601: Default ................. 18.55 |=================================== EPYC 7601: Mitigations Disabled .... 17.31 |================================= OSBench Test: Create Threads us Per Event < Lower Is Better Xeon Gold 6138: Default ............ 21.81 |============================= Xeon Gold: Mitigations Disabled .... 18.82 |========================= Xeon Gold VM: Default .............. 26.61 |=================================== Xeon Gold VM: Mitigations Disabled . 22.96 |============================== EPYC 7601: Default ................. 24.87 |================================= EPYC 7601: Mitigations Disabled .... 23.53 |=============================== OSBench Test: Launch Programs us Per Event < Lower Is Better Xeon Gold 6138: Default ............ 44.18 |====================== Xeon Gold: Mitigations Disabled .... 40.85 |==================== Xeon Gold VM: Default .............. 70.67 |=================================== Xeon Gold VM: Mitigations Disabled . 69.12 |================================== EPYC 7601: Default ................. 57.25 |============================ EPYC 7601: Mitigations Disabled .... 55.72 |============================ OSBench Test: Create Processes us Per Event < Lower Is Better Xeon Gold 6138: Default ............ 43.67 |============================ Xeon Gold: Mitigations Disabled .... 39.88 |========================= Xeon Gold VM: Default .............. 55.57 |=================================== Xeon Gold VM: Mitigations Disabled . 54.37 |================================== EPYC 7601: Default ................. 42.58 |=========================== EPYC 7601: Mitigations Disabled .... 42.17 |=========================== OSBench Test: Memory Allocations Ns Per Event < Lower Is Better Xeon Gold 6138: Default ............ 94.58 |============================= Xeon Gold: Mitigations Disabled .... 93.38 |============================= Xeon Gold VM: Default .............. 110.07 |================================== Xeon Gold VM: Mitigations Disabled . 100.61 |=============================== EPYC 7601: Default ................. 95.30 |============================= EPYC 7601: Mitigations Disabled .... 94.53 |============================= Flexible IO Tester 3.1 Type: Random Read - IO Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 2MB - Disk Target: Default Test Directory IOPS > Higher Is Better Xeon Gold VM: Default .............. 11167 |============================== Xeon Gold VM: Mitigations Disabled . 13233 |=================================== Flexible IO Tester 3.1 Type: Sequential Read - IO Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 2MB - Disk Target: Default Test Directory IOPS > Higher Is Better Xeon Gold VM: Default .............. 10433 |============================= Xeon Gold VM: Mitigations Disabled . 12467 |=================================== Flexible IO Tester 3.1 Type: Random Read - IO Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 2MB - Disk Target: Default Test Directory MB/s > Higher Is Better EPYC 7601: Default .............. 2614 |======================================= EPYC 7601: Mitigations Disabled . 2612 |======================================= Flexible IO Tester 3.1 Type: Sequential Read - IO Engine: Linux AIO - Buffered: No - Direct: Yes - Block Size: 2MB - Disk Target: Default Test Directory MB/s > Higher Is Better EPYC 7601: Default .............. 2618 |======================================= EPYC 7601: Mitigations Disabled . 2618 |=======================================