AMD EPYC vs. Intel Xeon vs. Amazon EC2 Instances

Tests for a future article. Some initial AMD EPYC 7601 tests on Ubuntu 17.04 with Linux 4.13. Tests for a future article on Phoronix.com. Benchmarks by Michael Larabel.

HTML result view exported from: https://openbenchmarking.org/result/1709185-TY-EC2COMPUT15.

AMD EPYC vs. Intel Xeon vs. Amazon EC2 InstancesProcessorMotherboardChipsetMemoryDiskGraphicsMonitorNetworkOSKernelDisplay DriverCompilerFile-SystemScreen ResolutionDesktopSystem LayerAMD EPYC 7601AMD EPYC 7601 (NUMA Interleave All)2 x Intel Xeon Gold 6138m4.10xlargem4.16xlargec4.4xlargec4.8xlargeAMD EPYC 7601 32-Core @ 2.20GHz (64 Cores)TYAN B8026T70AE24HRAMD Device 1450129024MB234GBASPEED ASPEED FamilyAcer P243WBroadcom Limited NetXtreme BCM5720 Gigabit PCIeUbuntu 17.044.13.0-041300-generic (x86_64)modesetting 1.19.3GCC 6.3.0 20170406ext41920x1200Unity 7.5.02 x Intel Xeon Gold 6138 @ 3.70GHz (80 Cores)TYAN S7106Intel Device 202096256MB256GB Samsung SSD 850 + 2000GB Seagate ST2000DM006-2DM1 + 2 x 120GB TOSHIBA-TR150Intel I210 Gigabit Connection2 x Intel Xeon E5-2676 v3 @ 3.00GHz (40 Cores)Xen HVM domUIntel 440FX- 82441FX PMC161792MB8GBCirrus Logic GD 5446Intel 82599 Virtual FunctionUbuntu 16.044.4.0-1022-aws (x86_64)GCC 5.4.0 20160609Xen HVM domU 4.2.amazon2 x Intel Xeon E5-2686 v4 @ 3.00GHz (64 Cores)258048MBDevice 1d0f:ec20Intel Xeon E5-2666 v3 @ 2.90GHz (16 Cores)30720MBIntel 82599 Virtual Function2 x Intel Xeon E5-2666 v3 @ 3.50GHz (36 Cores)60416MBOpenBenchmarking.orgCompiler Details- AMD EPYC 7601: --build=x86_64-linux-gnu --disable-browser-plugin --disable-vtable-verify --disable-werror --enable-checking=release --enable-clocale=gnu --enable-default-pie --enable-gnu-unique-object --enable-gtk-cairo --enable-java-awt=gtk --enable-java-home --enable-languages=c,ada,c++,java,go,d,fortran,objc,obj-c++ --enable-libmpx --enable-libstdcxx-debug --enable-libstdcxx-time=yes --enable-multiarch --enable-multilib --enable-nls --enable-objc-gc=auto --enable-plugin --enable-shared --enable-threads=posix --host=x86_64-linux-gnu --program-prefix=x86_64-linux-gnu- --target=x86_64-linux-gnu --with-abi=m64 --with-arch-32=i686 --with-arch-directory=amd64 --with-default-libstdcxx-abi=new --with-multilib-list=m32,m64,mx32 --with-target-system-zlib --with-tune=generic -v - AMD EPYC 7601 (NUMA Interleave All): --build=x86_64-linux-gnu --disable-browser-plugin --disable-vtable-verify --disable-werror --enable-checking=release --enable-clocale=gnu --enable-default-pie --enable-gnu-unique-object --enable-gtk-cairo --enable-java-awt=gtk --enable-java-home --enable-languages=c,ada,c++,java,go,d,fortran,objc,obj-c++ --enable-libmpx --enable-libstdcxx-debug --enable-libstdcxx-time=yes --enable-multiarch --enable-multilib --enable-nls --enable-objc-gc=auto --enable-plugin --enable-shared --enable-threads=posix --host=x86_64-linux-gnu --program-prefix=x86_64-linux-gnu- --target=x86_64-linux-gnu --with-abi=m64 --with-arch-32=i686 --with-arch-directory=amd64 --with-default-libstdcxx-abi=new --with-multilib-list=m32,m64,mx32 --with-target-system-zlib --with-tune=generic -v - 2 x Intel Xeon Gold 6138: --build=x86_64-linux-gnu --disable-browser-plugin --disable-vtable-verify --disable-werror --enable-checking=release --enable-clocale=gnu --enable-default-pie --enable-gnu-unique-object --enable-gtk-cairo --enable-java-awt=gtk --enable-java-home --enable-languages=c,ada,c++,java,go,d,fortran,objc,obj-c++ --enable-libmpx --enable-libstdcxx-debug --enable-libstdcxx-time=yes --enable-multiarch --enable-multilib --enable-nls --enable-objc-gc=auto --enable-plugin --enable-shared --enable-threads=posix --host=x86_64-linux-gnu --program-prefix=x86_64-linux-gnu- --target=x86_64-linux-gnu --with-abi=m64 --with-arch-32=i686 --with-arch-directory=amd64 --with-default-libstdcxx-abi=new --with-multilib-list=m32,m64,mx32 --with-target-system-zlib --with-tune=generic -v - m4.10xlarge: --build=x86_64-linux-gnu --disable-browser-plugin --disable-vtable-verify --disable-werror --enable-checking=release --enable-clocale=gnu --enable-gnu-unique-object --enable-gtk-cairo --enable-java-awt=gtk --enable-java-home --enable-languages=c,ada,c++,java,go,d,fortran,objc,obj-c++ --enable-libmpx --enable-libstdcxx-debug --enable-libstdcxx-time=yes --enable-multiarch --enable-multilib --enable-nls --enable-objc-gc --enable-plugin --enable-shared --enable-threads=posix --host=x86_64-linux-gnu --target=x86_64-linux-gnu --with-abi=m64 --with-arch-32=i686 --with-arch-directory=amd64 --with-default-libstdcxx-abi=new --with-multilib-list=m32,m64,mx32 --with-tune=generic -v - m4.16xlarge: --build=x86_64-linux-gnu --disable-browser-plugin --disable-vtable-verify --disable-werror --enable-checking=release --enable-clocale=gnu --enable-gnu-unique-object --enable-gtk-cairo --enable-java-awt=gtk --enable-java-home --enable-languages=c,ada,c++,java,go,d,fortran,objc,obj-c++ --enable-libmpx --enable-libstdcxx-debug --enable-libstdcxx-time=yes --enable-multiarch --enable-multilib --enable-nls --enable-objc-gc --enable-plugin --enable-shared --enable-threads=posix --host=x86_64-linux-gnu --target=x86_64-linux-gnu --with-abi=m64 --with-arch-32=i686 --with-arch-directory=amd64 --with-default-libstdcxx-abi=new --with-multilib-list=m32,m64,mx32 --with-tune=generic -v - c4.4xlarge: --build=x86_64-linux-gnu --disable-browser-plugin --disable-vtable-verify --disable-werror --enable-checking=release --enable-clocale=gnu --enable-gnu-unique-object --enable-gtk-cairo --enable-java-awt=gtk --enable-java-home --enable-languages=c,ada,c++,java,go,d,fortran,objc,obj-c++ --enable-libmpx --enable-libstdcxx-debug --enable-libstdcxx-time=yes --enable-multiarch --enable-multilib --enable-nls --enable-objc-gc --enable-plugin --enable-shared --enable-threads=posix --host=x86_64-linux-gnu --target=x86_64-linux-gnu --with-abi=m64 --with-arch-32=i686 --with-arch-directory=amd64 --with-default-libstdcxx-abi=new --with-multilib-list=m32,m64,mx32 --with-tune=generic -v - c4.8xlarge: --build=x86_64-linux-gnu --disable-browser-plugin --disable-vtable-verify --disable-werror --enable-checking=release --enable-clocale=gnu --enable-gnu-unique-object --enable-gtk-cairo --enable-java-awt=gtk --enable-java-home --enable-languages=c,ada,c++,java,go,d,fortran,objc,obj-c++ --enable-libmpx --enable-libstdcxx-debug --enable-libstdcxx-time=yes --enable-multiarch --enable-multilib --enable-nls --enable-objc-gc --enable-plugin --enable-shared --enable-threads=posix --host=x86_64-linux-gnu --target=x86_64-linux-gnu --with-abi=m64 --with-arch-32=i686 --with-arch-directory=amd64 --with-default-libstdcxx-abi=new --with-multilib-list=m32,m64,mx32 --with-tune=generic -v Processor Details- AMD EPYC 7601: Scaling Governor: acpi-cpufreq ondemand- AMD EPYC 7601 (NUMA Interleave All): Scaling Governor: acpi-cpufreq ondemand- 2 x Intel Xeon Gold 6138: Scaling Governor: intel_pstate powersave- m4.10xlarge: Scaling Governor: intel_pstate powersave- m4.16xlarge: Scaling Governor: intel_pstate powersave- c4.8xlarge: Scaling Governor: intel_pstate powersave

AMD EPYC vs. Intel Xeon vs. Amazon EC2 Instancesnpb: EP.Cnpb: LU.Anpb: LU.Cparboil: OpenMP LBMparboil: OpenMP Stencilrodinia: OpenMP LavaMDrodinia: OpenMP Streamclusterjohn-the-ripper: Blowfishx264: H.264 Video Encodingbuild-linux-kernel: Time To Compilebuild-llvm: Time To Compilec-ray: Total Timeprimesieve: 1e12 Prime Number Generationopenssl: RSA 4096-bit PerformanceAMD EPYC 7601AMD EPYC 7601 (NUMA Interleave All)2 x Intel Xeon Gold 6138m4.10xlargem4.16xlargec4.4xlargec4.8xlarge1610.1763042.8046983.3450.9513.8230.9623.1529553292.6337.40175.112.8414.083294.531607.6862479.0850046.9038.377.7530.1414.6634335288.2839.28192.262.8514.063306.371815.8953974.4950072.2550.047.7531.4422.5230373310.7930.54134.642.8411.814826.70629.3837949.8536509.8674.4311.4756.4723.6623520264.8441.84213.946.4522.182229.101036.5066710.1643175.9047.457.6334.1920.2139039304.3630.62147.244.2114.393835.70302.1720546.7918060.63114.3112.86115.9125.6711382257.8373.54435.3613.2245.961070.07672.8865.3811.8652.7723.4525152281.3339.89209.766.0820.752382.30OpenBenchmarking.org

NAS Parallel Benchmarks

Test / Class: EP.C

OpenBenchmarking.orgTotal Mop/s, More Is BetterNAS Parallel Benchmarks 3.3Test / Class: EP.CAMD EPYC 7601AMD EPYC 7601 (NUMA Interleave All)2 x Intel Xeon Gold 6138m4.10xlargem4.16xlargec4.4xlargec4.8xlarge400800120016002000SE +/- 0.32, N = 3SE +/- 0.90, N = 3SE +/- 34.76, N = 6SE +/- 1.36, N = 3SE +/- 4.41, N = 3SE +/- 0.85, N = 3SE +/- 1.49, N = 31610.171607.681815.89629.381036.50302.17672.881. (F9X) gfortran options: -O3 -march=native -pthread -lmpi_usempif08 -lmpi_mpifh -lmpi2. AMD EPYC 7601: Open MPI 2.0.23. AMD EPYC 7601 (NUMA Interleave All): Open MPI 2.0.24. 2 x Intel Xeon Gold 6138: Open MPI 2.0.25. m4.10xlarge: Open MPI 1.10.26. m4.16xlarge: Open MPI 1.10.27. c4.4xlarge: Open MPI 1.10.28. c4.8xlarge: Open MPI 1.10.2

NAS Parallel Benchmarks

Test / Class: LU.A

OpenBenchmarking.orgTotal Mop/s, More Is BetterNAS Parallel Benchmarks 3.3Test / Class: LU.AAMD EPYC 7601AMD EPYC 7601 (NUMA Interleave All)2 x Intel Xeon Gold 6138m4.10xlargem4.16xlargec4.4xlarge14K28K42K56K70KSE +/- 1051.01, N = 4SE +/- 437.89, N = 3SE +/- 3783.72, N = 6SE +/- 902.21, N = 6SE +/- 637.44, N = 3SE +/- 22.31, N = 363042.8062479.0853974.4937949.8566710.1620546.791. (F9X) gfortran options: -O3 -march=native -pthread -lmpi_usempif08 -lmpi_mpifh -lmpi2. AMD EPYC 7601: Open MPI 2.0.23. AMD EPYC 7601 (NUMA Interleave All): Open MPI 2.0.24. 2 x Intel Xeon Gold 6138: Open MPI 2.0.25. m4.10xlarge: Open MPI 1.10.26. m4.16xlarge: Open MPI 1.10.27. c4.4xlarge: Open MPI 1.10.2

NAS Parallel Benchmarks

Test / Class: LU.C

OpenBenchmarking.orgTotal Mop/s, More Is BetterNAS Parallel Benchmarks 3.3Test / Class: LU.CAMD EPYC 7601AMD EPYC 7601 (NUMA Interleave All)2 x Intel Xeon Gold 6138m4.10xlargem4.16xlargec4.4xlarge11K22K33K44K55KSE +/- 747.98, N = 3SE +/- 25.06, N = 3SE +/- 637.78, N = 3SE +/- 143.51, N = 3SE +/- 341.99, N = 3SE +/- 20.94, N = 346983.3450046.9050072.2536509.8643175.9018060.631. (F9X) gfortran options: -O3 -march=native -pthread -lmpi_usempif08 -lmpi_mpifh -lmpi2. AMD EPYC 7601: Open MPI 2.0.23. AMD EPYC 7601 (NUMA Interleave All): Open MPI 2.0.24. 2 x Intel Xeon Gold 6138: Open MPI 2.0.25. m4.10xlarge: Open MPI 1.10.26. m4.16xlarge: Open MPI 1.10.27. c4.4xlarge: Open MPI 1.10.2

Parboil

Test: OpenMP LBM

OpenBenchmarking.orgSeconds, Fewer Is BetterParboil 2.5Test: OpenMP LBMAMD EPYC 7601AMD EPYC 7601 (NUMA Interleave All)2 x Intel Xeon Gold 6138m4.10xlargem4.16xlargec4.4xlargec4.8xlarge306090120150SE +/- 0.66, N = 3SE +/- 0.26, N = 3SE +/- 1.14, N = 6SE +/- 3.88, N = 6SE +/- 3.52, N = 6SE +/- 0.37, N = 3SE +/- 1.81, N = 650.9538.3750.0474.4347.45114.3165.381. (CXX) g++ options: -lm -lpthread -lgomp -ffast-math -fopenmp

Parboil

Test: OpenMP Stencil

OpenBenchmarking.orgSeconds, Fewer Is BetterParboil 2.5Test: OpenMP StencilAMD EPYC 7601AMD EPYC 7601 (NUMA Interleave All)2 x Intel Xeon Gold 6138m4.10xlargem4.16xlargec4.4xlargec4.8xlarge48121620SE +/- 0.68, N = 6SE +/- 0.00, N = 3SE +/- 0.17, N = 6SE +/- 0.08, N = 3SE +/- 0.15, N = 3SE +/- 0.11, N = 3SE +/- 0.31, N = 613.827.757.7511.477.6312.8611.861. (CXX) g++ options: -lm -lpthread -lgomp -ffast-math -fopenmp

Rodinia

Test: OpenMP LavaMD

OpenBenchmarking.orgSeconds, Fewer Is BetterRodinia 2.4Test: OpenMP LavaMDAMD EPYC 7601AMD EPYC 7601 (NUMA Interleave All)2 x Intel Xeon Gold 6138m4.10xlargem4.16xlargec4.4xlargec4.8xlarge306090120150SE +/- 0.01, N = 3SE +/- 0.15, N = 3SE +/- 0.10, N = 3SE +/- 0.17, N = 3SE +/- 0.15, N = 3SE +/- 0.12, N = 3SE +/- 0.15, N = 330.9630.1431.4456.4734.19115.9152.771. (CXX) g++ options: -O2 -lOpenCL

Rodinia

Test: OpenMP Streamcluster

OpenBenchmarking.orgSeconds, Fewer Is BetterRodinia 2.4Test: OpenMP StreamclusterAMD EPYC 7601AMD EPYC 7601 (NUMA Interleave All)2 x Intel Xeon Gold 6138m4.10xlargem4.16xlargec4.4xlargec4.8xlarge612182430SE +/- 1.43, N = 6SE +/- 0.27, N = 6SE +/- 0.42, N = 3SE +/- 0.36, N = 3SE +/- 1.20, N = 6SE +/- 0.53, N = 6SE +/- 1.06, N = 623.1514.6622.5223.6620.2125.6723.451. (CXX) g++ options: -O2 -lOpenCL

John The Ripper

Test: Blowfish

OpenBenchmarking.orgReal C/S, More Is BetterJohn The Ripper 1.8.0Test: BlowfishAMD EPYC 7601AMD EPYC 7601 (NUMA Interleave All)2 x Intel Xeon Gold 6138m4.10xlargem4.16xlargec4.4xlargec4.8xlarge8K16K24K32K40KSE +/- 2253.07, N = 6SE +/- 764.17, N = 6SE +/- 2076.33, N = 6SE +/- 73.10, N = 3SE +/- 25.17, N = 3SE +/- 181.43, N = 3295533433530373235203903911382251521. (CC) gcc options: -fopenmp -lcrypt

x264

H.264 Video Encoding

OpenBenchmarking.orgFrames Per Second, More Is Betterx264 2017-09-08H.264 Video EncodingAMD EPYC 7601AMD EPYC 7601 (NUMA Interleave All)2 x Intel Xeon Gold 6138m4.10xlargem4.16xlargec4.4xlargec4.8xlarge70140210280350SE +/- 1.27, N = 3SE +/- 0.45, N = 3SE +/- 3.63, N = 3SE +/- 5.12, N = 3SE +/- 3.42, N = 3SE +/- 0.88, N = 3SE +/- 5.81, N = 6292.63288.28310.79264.84304.36257.83281.33-lavformat -lavcodec -lavutil -lswscale-lavformat -lavcodec -lavutil -lswscale1. (CC) gcc options: -ldl -m64 -lm -lpthread -O3 -ffast-math -std=gnu99 -fPIC -fomit-frame-pointer -fno-tree-vectorize

Timed Linux Kernel Compilation

Time To Compile

OpenBenchmarking.orgSeconds, Fewer Is BetterTimed Linux Kernel Compilation 4.9Time To CompileAMD EPYC 7601AMD EPYC 7601 (NUMA Interleave All)2 x Intel Xeon Gold 6138m4.10xlargem4.16xlargec4.4xlargec4.8xlarge1632486480SE +/- 0.51, N = 6SE +/- 0.58, N = 5SE +/- 0.89, N = 6SE +/- 0.63, N = 5SE +/- 0.52, N = 6SE +/- 0.82, N = 3SE +/- 0.67, N = 437.4039.2830.5441.8430.6273.5439.89

Timed LLVM Compilation

Time To Compile

OpenBenchmarking.orgSeconds, Fewer Is BetterTimed LLVM Compilation 4.0.1Time To CompileAMD EPYC 7601AMD EPYC 7601 (NUMA Interleave All)2 x Intel Xeon Gold 6138m4.10xlargem4.16xlargec4.4xlargec4.8xlarge90180270360450SE +/- 3.02, N = 4SE +/- 2.34, N = 3SE +/- 0.81, N = 3SE +/- 1.94, N = 3SE +/- 1.14, N = 3SE +/- 1.06, N = 3SE +/- 1.88, N = 3175.11192.26134.64213.94147.24435.36209.76

C-Ray

Total Time

OpenBenchmarking.orgSeconds, Fewer Is BetterC-Ray 1.1Total TimeAMD EPYC 7601AMD EPYC 7601 (NUMA Interleave All)2 x Intel Xeon Gold 6138m4.10xlargem4.16xlargec4.4xlargec4.8xlarge3691215SE +/- 0.02, N = 3SE +/- 0.03, N = 3SE +/- 0.01, N = 3SE +/- 0.01, N = 3SE +/- 0.04, N = 3SE +/- 0.01, N = 3SE +/- 0.01, N = 32.842.852.846.454.2113.226.081. (CC) gcc options: -lm -lpthread -O3

Primesieve

1e12 Prime Number Generation

OpenBenchmarking.orgSeconds, Fewer Is BetterPrimesieve 5.4.21e12 Prime Number GenerationAMD EPYC 7601AMD EPYC 7601 (NUMA Interleave All)2 x Intel Xeon Gold 6138m4.10xlargem4.16xlargec4.4xlargec4.8xlarge1020304050SE +/- 0.05, N = 3SE +/- 0.04, N = 3SE +/- 0.03, N = 3SE +/- 0.05, N = 3SE +/- 0.01, N = 3SE +/- 0.05, N = 3SE +/- 0.03, N = 314.0814.0611.8122.1814.3945.9620.751. (CXX) g++ options: -O2 -fopenmp

OpenSSL

RSA 4096-bit Performance

OpenBenchmarking.orgSigns Per Second, More Is BetterOpenSSL 1.0.1gRSA 4096-bit PerformanceAMD EPYC 7601AMD EPYC 7601 (NUMA Interleave All)2 x Intel Xeon Gold 6138m4.10xlargem4.16xlargec4.4xlargec4.8xlarge10002000300040005000SE +/- 14.45, N = 3SE +/- 13.22, N = 3SE +/- 23.22, N = 3SE +/- 1.32, N = 3SE +/- 3.53, N = 3SE +/- 0.20, N = 3SE +/- 4.10, N = 33294.533306.374826.702229.103835.701070.072382.301. (CC) gcc options: -m64 -O3 -lssl -lcrypto -ldl

Phoronix Test Suite v10.8.4