Intel Xeon Max AMX HBM2e Performance Benchmark

Benchmarks for a future article on Phoronix by Michael Larabel.

Compare your own system(s) to this result file with the Phoronix Test Suite by running the command: phoronix-test-suite benchmark 2307017-NE-XEONMAXAM71
Jump To Table - Results

View

Do Not Show Noisy Results
Do Not Show Results With Incomplete Data
Do Not Show Results With Little Change/Spread
List Notable Results
Show Result Confidence Charts
Allow Limiting Results To Certain Suite(s)

Statistics

Show Overall Harmonic Mean(s)
Show Overall Geometric Mean
Show Wins / Losses Counts (Pie Chart)
Normalize Results
Remove Outliers Before Calculating Averages

Graph Settings

Force Line Graphs Where Applicable
Convert To Scalar Where Applicable
Disable Color Branding
Prefer Vertical Bar Graphs
No Box Plots
On Line Graphs With Missing Data, Connect The Line Gaps

Additional Graphs

Show Perf Per Core/Thread Calculation Graphs Where Applicable
Show Perf Per Clock Calculation Graphs Where Applicable

Multi-Way Comparison

Condense Multi-Option Tests Into Single Result Graphs
Condense Test Profiles With Multiple Version Results Into Single Result Graphs

Table

Show Detailed System Result Table

Run Management

Highlight
Result
Toggle/Hide
Result
Result
Identifier
Performance Per
Dollar
Date
Run
  Test
  Duration
Xeon Max 9480 2P, No HBM, Max AVX512 FP16
July 01 2023
  1 Hour, 35 Minutes
Xeon Max 9480 2P, No HBM
June 30 2023
  2 Hours, 13 Minutes
Xeon Max 9480 2P, HBM Caching
June 29 2023
  2 Hours, 13 Minutes
Xeon Max 9480 2P, HBM Only
June 30 2023
  1 Hour, 35 Minutes
EPYC 9554 2P
June 28 2023
  1 Hour, 36 Minutes
EPYC 9654 2P
June 28 2023
  1 Hour, 34 Minutes
Invert Behavior (Only Show Selected Data)
  1 Hour, 48 Minutes

Only show results where is faster than
Only show results matching title/arguments (delimit multiple options with a comma):
Do not show results matching title/arguments (delimit multiple options with a comma):

Xeon Max 9480 2P, No HBM, Max AVX512 FP16

Kernel Notes: Transparent Huge Pages: madvise
Compiler Notes: --build=x86_64-linux-gnu --disable-vtable-verify --disable-werror --enable-cet --enable-checking=release --enable-clocale=gnu --enable-default-pie --enable-gnu-unique-object --enable-languages=c,ada,c++,go,d,fortran,objc,obj-c++,m2 --enable-libphobos-checking=release --enable-libstdcxx-debug --enable-libstdcxx-time=yes --enable-multiarch --enable-multilib --enable-nls --enable-objc-gc=auto --enable-offload-defaulted --enable-offload-targets=nvptx-none=/build/gcc-12-Pa930Z/gcc-12-12.2.0/debian/tmp-nvptx/usr,amdgcn-amdhsa=/build/gcc-12-Pa930Z/gcc-12-12.2.0/debian/tmp-gcn/usr --enable-plugin --enable-shared --enable-threads=posix --host=x86_64-linux-gnu --program-prefix=x86_64-linux-gnu- --target=x86_64-linux-gnu --with-abi=m64 --with-arch-32=i686 --with-default-libstdcxx-abi=new --with-gcc-major-version-only --with-multilib-list=m32,m64,mx32 --with-target-system-zlib=auto --with-tune=generic --without-cuda-driver -v
Processor Notes: Scaling Governor: intel_cpufreq performance - CPU Microcode: 0x2c0001d1
Python Notes: Python 3.11.2
Security Notes: itlb_multihit: Not affected + l1tf: Not affected + mds: Not affected + meltdown: Not affected + mmio_stale_data: Not affected + retbleed: Not affected + spec_store_bypass: Mitigation of SSB disabled via prctl + spectre_v1: Mitigation of usercopy/swapgs barriers and __user pointer sanitization + spectre_v2: Mitigation of Enhanced IBRS IBPB: conditional RSB filling PBRSB-eIBRS: SW sequence + srbds: Not affected + tsx_async_abort: Not affected

Xeon Max 9480 2P, No HBM

Xeon Max 9480 2P, HBM Caching

Processor: 2 x Intel Xeon Max 9480 @ 3.50GHz (112 Cores / 224 Threads), Motherboard: Supermicro X13DEM v1.10 (1.3 BIOS), Chipset: Intel Device 1bce, Memory: 512GB, Disk: 7682GB INTEL SSDPF2KX076TZ, Graphics: ASPEED, Monitor: VE228, Network: 2 x Broadcom BCM57508 NetXtreme-E 10Gb/25Gb/40Gb/50Gb/100Gb/200Gb

OS: Ubuntu 23.04, Kernel: 6.2.0-20-generic (x86_64), Desktop: GNOME Shell 44.0, Display Server: X Server 1.21.1.7, Compiler: GCC 12.2.0, File-System: ext4, Screen Resolution: 1920x1080

Xeon Max 9480 2P, HBM Only

Changed Memory to 128GB.

EPYC 9554 2P

Processor: 2 x AMD EPYC 9554 64-Core @ 3.10GHz (128 Cores / 256 Threads), Motherboard: AMD Titanite_4G (RTI1007B BIOS), Chipset: AMD Device 14a4, Memory: 1520GB, Disk: 7682GB INTEL SSDPF2KX076TZ, Graphics: ASPEED, Monitor: VGA HDMI, Network: Broadcom NetXtreme BCM5720 PCIe

Kernel Notes: Transparent Huge Pages: madvise
Compiler Notes: --build=x86_64-linux-gnu --disable-vtable-verify --disable-werror --enable-cet --enable-checking=release --enable-clocale=gnu --enable-default-pie --enable-gnu-unique-object --enable-languages=c,ada,c++,go,d,fortran,objc,obj-c++,m2 --enable-libphobos-checking=release --enable-libstdcxx-debug --enable-libstdcxx-time=yes --enable-multiarch --enable-multilib --enable-nls --enable-objc-gc=auto --enable-offload-defaulted --enable-offload-targets=nvptx-none=/build/gcc-12-Pa930Z/gcc-12-12.2.0/debian/tmp-nvptx/usr,amdgcn-amdhsa=/build/gcc-12-Pa930Z/gcc-12-12.2.0/debian/tmp-gcn/usr --enable-plugin --enable-shared --enable-threads=posix --host=x86_64-linux-gnu --program-prefix=x86_64-linux-gnu- --target=x86_64-linux-gnu --with-abi=m64 --with-arch-32=i686 --with-default-libstdcxx-abi=new --with-gcc-major-version-only --with-multilib-list=m32,m64,mx32 --with-target-system-zlib=auto --with-tune=generic --without-cuda-driver -v
Processor Notes: Scaling Governor: acpi-cpufreq performance (Boost: Enabled) - CPU Microcode: 0xa101121
Python Notes: Python 3.11.2
Security Notes: itlb_multihit: Not affected + l1tf: Not affected + mds: Not affected + meltdown: Not affected + mmio_stale_data: Not affected + retbleed: Not affected + spec_store_bypass: Mitigation of SSB disabled via prctl + spectre_v1: Mitigation of usercopy/swapgs barriers and __user pointer sanitization + spectre_v2: Mitigation of Retpolines IBPB: conditional IBRS_FW STIBP: always-on RSB filling PBRSB-eIBRS: Not affected + srbds: Not affected + tsx_async_abort: Not affected

EPYC 9654 2P

Changed Processor to 2 x AMD EPYC 9654 96-Core @ 2.40GHz (192 Cores / 384 Threads).

Intel Xeon Max AMX HBM2e Performance BenchmarkProcessorMotherboardChipsetMemoryDiskGraphicsMonitorNetworkOSKernelDesktopDisplay ServerCompilerFile-SystemScreen ResolutionXeon Max 9480 2P, No HBM, Max AVX512 FP16Xeon Max 9480 2P, No HBMXeon Max 9480 2P, HBM CachingXeon Max 9480 2P, HBM OnlyEPYC 9554 2PEPYC 9654 2P2 x Intel Xeon Max 9480 @ 3.50GHz (112 Cores / 224 Threads)Supermicro X13DEM v1.10 (1.3 BIOS)Intel Device 1bce512GB7682GB INTEL SSDPF2KX076TZASPEEDVE2282 x Broadcom BCM57508 NetXtreme-E 10Gb/25Gb/40Gb/50Gb/100Gb/200GbUbuntu 23.046.2.0-20-generic (x86_64)GNOME Shell 44.0X Server 1.21.1.7GCC 12.2.0ext41920x1080128GB2 x AMD EPYC 9554 64-Core @ 3.10GHz (128 Cores / 256 Threads)AMD Titanite_4G (RTI1007B BIOS)AMD Device 14a41520GBVGA HDMIBroadcom NetXtreme BCM5720 PCIe2 x AMD EPYC 9654 96-Core @ 2.40GHz (192 Cores / 384 Threads)OpenBenchmarking.orgKernel Details- Transparent Huge Pages: madviseCompiler Details- --build=x86_64-linux-gnu --disable-vtable-verify --disable-werror --enable-cet --enable-checking=release --enable-clocale=gnu --enable-default-pie --enable-gnu-unique-object --enable-languages=c,ada,c++,go,d,fortran,objc,obj-c++,m2 --enable-libphobos-checking=release --enable-libstdcxx-debug --enable-libstdcxx-time=yes --enable-multiarch --enable-multilib --enable-nls --enable-objc-gc=auto --enable-offload-defaulted --enable-offload-targets=nvptx-none=/build/gcc-12-Pa930Z/gcc-12-12.2.0/debian/tmp-nvptx/usr,amdgcn-amdhsa=/build/gcc-12-Pa930Z/gcc-12-12.2.0/debian/tmp-gcn/usr --enable-plugin --enable-shared --enable-threads=posix --host=x86_64-linux-gnu --program-prefix=x86_64-linux-gnu- --target=x86_64-linux-gnu --with-abi=m64 --with-arch-32=i686 --with-default-libstdcxx-abi=new --with-gcc-major-version-only --with-multilib-list=m32,m64,mx32 --with-target-system-zlib=auto --with-tune=generic --without-cuda-driver -v Processor Details- Xeon Max 9480 2P, No HBM, Max AVX512 FP16: Scaling Governor: intel_cpufreq performance - CPU Microcode: 0x2c0001d1- Xeon Max 9480 2P, No HBM: Scaling Governor: intel_cpufreq performance - CPU Microcode: 0x2c0001d1- Xeon Max 9480 2P, HBM Caching: Scaling Governor: intel_cpufreq performance - CPU Microcode: 0x2c0001d1- Xeon Max 9480 2P, HBM Only: Scaling Governor: intel_cpufreq performance - CPU Microcode: 0x2c0001d1- EPYC 9554 2P: Scaling Governor: acpi-cpufreq performance (Boost: Enabled) - CPU Microcode: 0xa101121- EPYC 9654 2P: Scaling Governor: acpi-cpufreq performance (Boost: Enabled) - CPU Microcode: 0xa101121Python Details- Python 3.11.2Security Details- Xeon Max 9480 2P, No HBM, Max AVX512 FP16: itlb_multihit: Not affected + l1tf: Not affected + mds: Not affected + meltdown: Not affected + mmio_stale_data: Not affected + retbleed: Not affected + spec_store_bypass: Mitigation of SSB disabled via prctl + spectre_v1: Mitigation of usercopy/swapgs barriers and __user pointer sanitization + spectre_v2: Mitigation of Enhanced IBRS IBPB: conditional RSB filling PBRSB-eIBRS: SW sequence + srbds: Not affected + tsx_async_abort: Not affected - Xeon Max 9480 2P, No HBM: itlb_multihit: Not affected + l1tf: Not affected + mds: Not affected + meltdown: Not affected + mmio_stale_data: Not affected + retbleed: Not affected + spec_store_bypass: Mitigation of SSB disabled via prctl + spectre_v1: Mitigation of usercopy/swapgs barriers and __user pointer sanitization + spectre_v2: Mitigation of Enhanced IBRS IBPB: conditional RSB filling PBRSB-eIBRS: SW sequence + srbds: Not affected + tsx_async_abort: Not affected - Xeon Max 9480 2P, HBM Caching: itlb_multihit: Not affected + l1tf: Not affected + mds: Not affected + meltdown: Not affected + mmio_stale_data: Not affected + retbleed: Not affected + spec_store_bypass: Mitigation of SSB disabled via prctl + spectre_v1: Mitigation of usercopy/swapgs barriers and __user pointer sanitization + spectre_v2: Mitigation of Enhanced IBRS IBPB: conditional RSB filling PBRSB-eIBRS: SW sequence + srbds: Not affected + tsx_async_abort: Not affected - Xeon Max 9480 2P, HBM Only: itlb_multihit: Not affected + l1tf: Not affected + mds: Not affected + meltdown: Not affected + mmio_stale_data: Not affected + retbleed: Not affected + spec_store_bypass: Mitigation of SSB disabled via prctl + spectre_v1: Mitigation of usercopy/swapgs barriers and __user pointer sanitization + spectre_v2: Mitigation of Enhanced IBRS IBPB: conditional RSB filling PBRSB-eIBRS: SW sequence + srbds: Not affected + tsx_async_abort: Not affected - EPYC 9554 2P: itlb_multihit: Not affected + l1tf: Not affected + mds: Not affected + meltdown: Not affected + mmio_stale_data: Not affected + retbleed: Not affected + spec_store_bypass: Mitigation of SSB disabled via prctl + spectre_v1: Mitigation of usercopy/swapgs barriers and __user pointer sanitization + spectre_v2: Mitigation of Retpolines IBPB: conditional IBRS_FW STIBP: always-on RSB filling PBRSB-eIBRS: Not affected + srbds: Not affected + tsx_async_abort: Not affected - EPYC 9654 2P: itlb_multihit: Not affected + l1tf: Not affected + mds: Not affected + meltdown: Not affected + mmio_stale_data: Not affected + retbleed: Not affected + spec_store_bypass: Mitigation of SSB disabled via prctl + spectre_v1: Mitigation of usercopy/swapgs barriers and __user pointer sanitization + spectre_v2: Mitigation of Retpolines IBPB: conditional IBRS_FW STIBP: always-on RSB filling PBRSB-eIBRS: Not affected + srbds: Not affected + tsx_async_abort: Not affected

Xeon Max 9480 2P, No HBM, Max AVX512 FP16Xeon Max 9480 2P, No HBMXeon Max 9480 2P, HBM CachingXeon Max 9480 2P, HBM OnlyEPYC 9554 2PEPYC 9654 2PLogarithmic Result OverviewPhoronix Test SuiteOpenVINOOpenVINOOpenVINOOpenVINOOpenVINOOpenVINOOpenVINOOpenVINOOpenVINOOpenVINOONNX RuntimeOpenVINOOpenVINOONNX RuntimeONNX RuntimeONNX RuntimeW.P.D.F - CPUF.D.F - CPUF.D.F - CPUP.D.F - CPUP.D.F - CPUF.D.F.I - CPUP.D.F - CPUW.P.D.F.I - CPUP.D.F - CPUW.P.D.F.I - CPUGPT-2 - CPU - StandardW.P.D.F - CPUF.D.F.I - CPUCaffeNet 12-int8 - CPU - StandardGPT-2 - CPU - StandardCaffeNet 12-int8 - CPU - Standard

Intel Xeon Max AMX HBM2e Performance Benchmarkopenvino: Face Detection FP16 - CPUopenvino: Person Detection FP16 - CPUopenvino: Person Detection FP32 - CPUopenvino: Face Detection FP16-INT8 - CPUopenvino: Weld Porosity Detection FP16 - CPUopenvino: Weld Porosity Detection FP16-INT8 - CPUonnx: GPT-2 - CPU - Standardonnx: CaffeNet 12-int8 - CPU - Standardonnx: GPT-2 - CPU - Standardonnx: CaffeNet 12-int8 - CPU - Standardopenvino: Face Detection FP16 - CPUopenvino: Person Detection FP16 - CPUopenvino: Person Detection FP32 - CPUopenvino: Face Detection FP16-INT8 - CPUopenvino: Weld Porosity Detection FP16 - CPUopenvino: Weld Porosity Detection FP16-INT8 - CPUXeon Max 9480 2P, No HBM, Max AVX512 FP16Xeon Max 9480 2P, No HBMXeon Max 9480 2P, HBM CachingXeon Max 9480 2P, HBM OnlyEPYC 9554 2PEPYC 9654 2P19.8311.3111.26106.271977.0810680.95157.758543.8116.416921.837971398.322445.492449.29262.8456.6010.47115.2032.5233.79293.4916532.4031482.78155.155537.8216.508891.86415242.64857.32824.93380.606.653.45120.4033.4033.97334.8916941.9930938.50218.692562.2184.708491.78296232.01834.97819.76333.516.443.47175.7439.4738.77369.6418452.0235643.79246.700596.9084.052371.68307159.13706.83719.49302.455.923.0782.0438.2438.53155.288039.5715659.32132.721495.2407.647352.01949389.53832.02825.52205.713.978.09101.5643.6443.40191.419843.0319170.61103.184435.1949.689042.30611471.821092.971098.96250.394.869.95OpenBenchmarking.org

CPU Peak Freq (Highest CPU Core Frequency) Monitor

OpenBenchmarking.orgMegahertzCPU Peak Freq (Highest CPU Core Frequency) MonitorPhoronix Test Suite System MonitoringXeon Max 9480 2P, No HBM, Max AVX512 FP16Xeon Max 9480 2P, No HBMXeon Max 9480 2P, HBM OnlyXeon Max 9480 2P, HBM Caching6001200180024003000Min: 1800 / Avg: 3336.55 / Max: 3518Min: 1000 / Avg: 3056.85 / Max: 3576Min: 1566 / Avg: 3308.69 / Max: 3551Min: 1100 / Avg: 3207.85 / Max: 3577

CPU Power Consumption Monitor

OpenBenchmarking.orgWattsCPU Power Consumption MonitorPhoronix Test Suite System MonitoringXeon Max 9480 2P, No HBM, Max AVX512 FP16Xeon Max 9480 2P, No HBMXeon Max 9480 2P, HBM OnlyXeon Max 9480 2P, HBM CachingEPYC 9654 2PEPYC 9554 2P2004006008001000Min: 166.66 / Avg: 528.06 / Max: 636.62Min: 93.37 / Avg: 522.99 / Max: 664.03Min: 129.15 / Avg: 563.95 / Max: 1026.74Min: 87.33 / Avg: 548.37 / Max: 899.87Min: 31.88 / Avg: 448.81 / Max: 725.51Min: 34.39 / Avg: 413.82 / Max: 710.63

System Power Consumption Monitor

OpenBenchmarking.orgWattsSystem Power Consumption MonitorPhoronix Test Suite System MonitoringXeon Max 9480 2P, No HBM, Max AVX512 FP16Xeon Max 9480 2P, No HBMXeon Max 9480 2P, HBM OnlyXeon Max 9480 2P, HBM Caching160320480640800Min: 276 / Avg: 733.34 / Max: 937Min: 251 / Avg: 735.78 / Max: 904Min: 250 / Avg: 752.26 / Max: 929Min: 260 / Avg: 751.74 / Max: 907

OpenVINO

This is a test of the Intel OpenVINO, a toolkit around neural networks, using its built-in benchmarking support and analyzing the throughput and latency for various models. Learn more via the OpenBenchmarking.org test page.

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2022.3Model: Face Detection FP16 - Device: CPUXeon Max 9480 2P, No HBM, Max AVX512 FP16Xeon Max 9480 2P, No HBMXeon Max 9480 2P, HBM OnlyXeon Max 9480 2P, HBM CachingEPYC 9654 2PEPYC 9554 2P4080120160200SE +/- 0.11, N = 3SE +/- 0.29, N = 3SE +/- 0.12, N = 3SE +/- 0.41, N = 3SE +/- 0.25, N = 3SE +/- 0.04, N = 319.83115.20175.74120.40101.5682.041. (CXX) g++ options: -isystem -fsigned-char -ffunction-sections -fdata-sections -msse4.1 -msse4.2 -O3 -fno-strict-overflow -fwrapv -fPIC -fvisibility=hidden -Os -std=c++11 -MD -MT -MF

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2022.3Model: Person Detection FP16 - Device: CPUXeon Max 9480 2P, No HBM, Max AVX512 FP16Xeon Max 9480 2P, No HBMXeon Max 9480 2P, HBM OnlyXeon Max 9480 2P, HBM CachingEPYC 9654 2PEPYC 9554 2P1020304050SE +/- 0.04, N = 3SE +/- 0.42, N = 3SE +/- 0.08, N = 3SE +/- 0.24, N = 3SE +/- 0.21, N = 3SE +/- 0.23, N = 311.3132.5239.4733.4043.6438.241. (CXX) g++ options: -isystem -fsigned-char -ffunction-sections -fdata-sections -msse4.1 -msse4.2 -O3 -fno-strict-overflow -fwrapv -fPIC -fvisibility=hidden -Os -std=c++11 -MD -MT -MF

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2022.3Model: Person Detection FP32 - Device: CPUXeon Max 9480 2P, No HBM, Max AVX512 FP16Xeon Max 9480 2P, No HBMXeon Max 9480 2P, HBM OnlyXeon Max 9480 2P, HBM CachingEPYC 9654 2PEPYC 9554 2P1020304050SE +/- 0.07, N = 3SE +/- 0.25, N = 3SE +/- 0.42, N = 3SE +/- 0.31, N = 3SE +/- 0.20, N = 3SE +/- 0.04, N = 311.2633.7938.7733.9743.4038.531. (CXX) g++ options: -isystem -fsigned-char -ffunction-sections -fdata-sections -msse4.1 -msse4.2 -O3 -fno-strict-overflow -fwrapv -fPIC -fvisibility=hidden -Os -std=c++11 -MD -MT -MF

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2022.3Model: Face Detection FP16-INT8 - Device: CPUXeon Max 9480 2P, No HBM, Max AVX512 FP16Xeon Max 9480 2P, No HBMXeon Max 9480 2P, HBM OnlyXeon Max 9480 2P, HBM CachingEPYC 9654 2PEPYC 9554 2P80160240320400SE +/- 0.10, N = 3SE +/- 0.25, N = 3SE +/- 0.06, N = 3SE +/- 0.45, N = 3SE +/- 0.06, N = 3SE +/- 0.14, N = 3106.27293.49369.64334.89191.41155.281. (CXX) g++ options: -isystem -fsigned-char -ffunction-sections -fdata-sections -msse4.1 -msse4.2 -O3 -fno-strict-overflow -fwrapv -fPIC -fvisibility=hidden -Os -std=c++11 -MD -MT -MF

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2022.3Model: Weld Porosity Detection FP16 - Device: CPUXeon Max 9480 2P, No HBM, Max AVX512 FP16Xeon Max 9480 2P, No HBMXeon Max 9480 2P, HBM OnlyXeon Max 9480 2P, HBM CachingEPYC 9654 2PEPYC 9554 2P4K8K12K16K20KSE +/- 2.85, N = 3SE +/- 65.25, N = 3SE +/- 33.75, N = 3SE +/- 13.63, N = 3SE +/- 2.32, N = 3SE +/- 1.22, N = 31977.0816532.4018452.0216941.999843.038039.571. (CXX) g++ options: -isystem -fsigned-char -ffunction-sections -fdata-sections -msse4.1 -msse4.2 -O3 -fno-strict-overflow -fwrapv -fPIC -fvisibility=hidden -Os -std=c++11 -MD -MT -MF

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2022.3Model: Weld Porosity Detection FP16-INT8 - Device: CPUXeon Max 9480 2P, No HBM, Max AVX512 FP16Xeon Max 9480 2P, No HBMXeon Max 9480 2P, HBM OnlyXeon Max 9480 2P, HBM CachingEPYC 9654 2PEPYC 9554 2P8K16K24K32K40KSE +/- 12.56, N = 3SE +/- 37.83, N = 3SE +/- 37.65, N = 3SE +/- 38.32, N = 3SE +/- 4.81, N = 3SE +/- 3.60, N = 310680.9531482.7835643.7930938.5019170.6115659.321. (CXX) g++ options: -isystem -fsigned-char -ffunction-sections -fdata-sections -msse4.1 -msse4.2 -O3 -fno-strict-overflow -fwrapv -fPIC -fvisibility=hidden -Os -std=c++11 -MD -MT -MF

ONNX Runtime

ONNX Runtime is developed by Microsoft and partners as a open-source, cross-platform, high performance machine learning inferencing and training accelerator. This test profile runs the ONNX Runtime with various models available from the ONNX Model Zoo. Learn more via the OpenBenchmarking.org test page.

OpenBenchmarking.orgInferences Per Second, More Is BetterONNX Runtime 1.14Model: GPT-2 - Device: CPU - Executor: StandardXeon Max 9480 2P, No HBM, Max AVX512 FP16Xeon Max 9480 2P, No HBMXeon Max 9480 2P, HBM OnlyXeon Max 9480 2P, HBM CachingEPYC 9654 2PEPYC 9554 2P50100150200250SE +/- 4.54, N = 15SE +/- 4.09, N = 15SE +/- 3.11, N = 3SE +/- 9.23, N = 15SE +/- 0.03, N = 3SE +/- 4.25, N = 15157.76155.16246.70218.69103.18132.721. (CXX) g++ options: -ffunction-sections -fdata-sections -march=native -mtune=native -O3 -flto=auto -fno-fat-lto-objects -ldl -lrt
OpenBenchmarking.orgInference Time Cost (ms), Fewer Is BetterONNX Runtime 1.14Model: GPT-2 - Device: CPU - Executor: StandardXeon Max 9480 2P, No HBM, Max AVX512 FP16Xeon Max 9480 2P, No HBMXeon Max 9480 2P, HBM OnlyXeon Max 9480 2P, HBM CachingEPYC 9654 2PEPYC 9554 2P3691215SE +/- 0.19970, N = 15SE +/- 0.18098, N = 15SE +/- 0.05143, N = 3SE +/- 0.23765, N = 15SE +/- 0.00311, N = 3SE +/- 0.25707, N = 156.416926.508894.052374.708499.689047.647351. (CXX) g++ options: -ffunction-sections -fdata-sections -march=native -mtune=native -O3 -flto=auto -fno-fat-lto-objects -ldl -lrt
OpenBenchmarking.orgMegahertz, More Is BetterONNX Runtime 1.14CPU Peak Freq (Highest CPU Core Frequency) MonitorXeon Max 9480 2P, No HBM, Max AVX512 FP166001200180024003000Min: 3500 / Avg: 3501.18 / Max: 3518
OpenBenchmarking.orgWatts, Fewer Is BetterONNX Runtime 1.14System Power Consumption MonitorXeon Max 9480 2P, No HBM, Max AVX512 FP16130260390520650Min: 386 / Avg: 718.32 / Max: 749

OpenBenchmarking.orgInferences Per Second, More Is BetterONNX Runtime 1.14Model: CaffeNet 12-int8 - Device: CPU - Executor: StandardXeon Max 9480 2P, No HBM, Max AVX512 FP16Xeon Max 9480 2P, No HBMXeon Max 9480 2P, HBM OnlyXeon Max 9480 2P, HBM CachingEPYC 9654 2PEPYC 9554 2P130260390520650SE +/- 3.29, N = 3SE +/- 7.97, N = 15SE +/- 10.95, N = 15SE +/- 8.12, N = 15SE +/- 7.41, N = 15SE +/- 5.94, N = 4543.81537.82596.91562.22435.19495.241. (CXX) g++ options: -ffunction-sections -fdata-sections -march=native -mtune=native -O3 -flto=auto -fno-fat-lto-objects -ldl -lrt
OpenBenchmarking.orgInference Time Cost (ms), Fewer Is BetterONNX Runtime 1.14Model: CaffeNet 12-int8 - Device: CPU - Executor: StandardXeon Max 9480 2P, No HBM, Max AVX512 FP16Xeon Max 9480 2P, No HBMXeon Max 9480 2P, HBM OnlyXeon Max 9480 2P, HBM CachingEPYC 9654 2PEPYC 9554 2P0.51891.03781.55672.07562.5945SE +/- 0.01109, N = 3SE +/- 0.02958, N = 15SE +/- 0.03485, N = 15SE +/- 0.02850, N = 15SE +/- 0.03734, N = 15SE +/- 0.02472, N = 41.837971.864151.683071.782962.306112.019491. (CXX) g++ options: -ffunction-sections -fdata-sections -march=native -mtune=native -O3 -flto=auto -fno-fat-lto-objects -ldl -lrt
OpenBenchmarking.orgMegahertz, More Is BetterONNX Runtime 1.14CPU Peak Freq (Highest CPU Core Frequency) MonitorXeon Max 9480 2P, No HBM, Max AVX512 FP1680016002400320040003500
MinAvgMaxXeon Max 9480 2P, No HBM, Max AVX512 FP16416765857OpenBenchmarking.orgWatts, Fewer Is BetterONNX Runtime 1.14System Power Consumption Monitor2004006008001000

OpenVINO

MinAvgMaxXeon Max 9480 2P, No HBM, Max AVX512 FP16350035003512Xeon Max 9480 2P, No HBM110016543503Xeon Max 9480 2P, HBM Only200034933508Xeon Max 9480 2P, HBM Caching120017403505OpenBenchmarking.orgMegahertz, More Is BetterOpenVINO 2022.3CPU Peak Freq (Highest CPU Core Frequency) Monitor10002000300040005000

MinAvgMaxXeon Max 9480 2P, No HBM, Max AVX512 FP16220034813515Xeon Max 9480 2P, No HBM180033133518Xeon Max 9480 2P, HBM Only350035003517Xeon Max 9480 2P, HBM Caching190030233512OpenBenchmarking.orgMegahertz, More Is BetterOpenVINO 2022.3CPU Peak Freq (Highest CPU Core Frequency) Monitor10002000300040005000

MinAvgMaxXeon Max 9480 2P, No HBM, Max AVX512 FP16350035003511Xeon Max 9480 2P, No HBM208733963518Xeon Max 9480 2P, HBM Only264634963508Xeon Max 9480 2P, HBM Caching190030743506OpenBenchmarking.orgMegahertz, More Is BetterOpenVINO 2022.3CPU Peak Freq (Highest CPU Core Frequency) Monitor10002000300040005000

MinAvgMaxXeon Max 9480 2P, No HBM, Max AVX512 FP16180024923510Xeon Max 9480 2P, No HBM150021953515Xeon Max 9480 2P, HBM Only190033593505Xeon Max 9480 2P, HBM Caching170023703505OpenBenchmarking.orgMegahertz, More Is BetterOpenVINO 2022.3CPU Peak Freq (Highest CPU Core Frequency) Monitor10002000300040005000

MinAvgMaxXeon Max 9480 2P, No HBM, Max AVX512 FP16350035003509Xeon Max 9480 2P, No HBM160018463500Xeon Max 9480 2P, HBM Only190026183506Xeon Max 9480 2P, HBM Caching170019003500OpenBenchmarking.orgMegahertz, More Is BetterOpenVINO 2022.3CPU Peak Freq (Highest CPU Core Frequency) Monitor10002000300040005000

MinAvgMaxXeon Max 9480 2P, No HBM, Max AVX512 FP16180020233500Xeon Max 9480 2P, No HBM160019343502Xeon Max 9480 2P, HBM Only180029323508Xeon Max 9480 2P, HBM Caching150019023504OpenBenchmarking.orgMegahertz, More Is BetterOpenVINO 2022.3CPU Peak Freq (Highest CPU Core Frequency) Monitor10002000300040005000

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2022.3Model: Face Detection FP16 - Device: CPUXeon Max 9480 2P, No HBM, Max AVX512 FP16Xeon Max 9480 2P, No HBMXeon Max 9480 2P, HBM OnlyXeon Max 9480 2P, HBM CachingEPYC 9654 2PEPYC 9554 2P30060090012001500SE +/- 8.00, N = 3SE +/- 0.61, N = 3SE +/- 0.10, N = 3SE +/- 0.80, N = 3SE +/- 0.93, N = 3SE +/- 0.19, N = 31398.32242.64159.13232.01471.82389.53MIN: 943.82 / MAX: 1957.07MIN: 431.51 / MAX: 569.3MIN: 380.82 / MAX: 432.571. (CXX) g++ options: -isystem -fsigned-char -ffunction-sections -fdata-sections -msse4.1 -msse4.2 -O3 -fno-strict-overflow -fwrapv -fPIC -fvisibility=hidden -Os -std=c++11 -MD -MT -MF
MinAvgMaxXeon Max 9480 2P, No HBM, Max AVX512 FP16259561637Xeon Max 9480 2P, No HBM163519586Xeon Max 9480 2P, HBM Only256574649Xeon Max 9480 2P, HBM Caching136533628EPYC 9654 2P58646726EPYC 9554 2P49638711OpenBenchmarking.orgWatts, Fewer Is BetterOpenVINO 2022.3CPU Power Consumption Monitor2004006008001000
MinAvgMaxXeon Max 9480 2P, No HBM, Max AVX512 FP16350035003512Xeon Max 9480 2P, No HBM110016543503Xeon Max 9480 2P, HBM Only200034933508Xeon Max 9480 2P, HBM Caching120017403505OpenBenchmarking.orgMegahertz, More Is BetterOpenVINO 2022.3CPU Peak Freq (Highest CPU Core Frequency) Monitor10002000300040005000
MinAvgMaxXeon Max 9480 2P, No HBM, Max AVX512 FP16399769937Xeon Max 9480 2P, No HBM274734840Xeon Max 9480 2P, HBM Only362757833Xeon Max 9480 2P, HBM Caching281724839OpenBenchmarking.orgWatts, Fewer Is BetterOpenVINO 2022.3System Power Consumption Monitor2004006008001000

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2022.3Model: Person Detection FP16 - Device: CPUXeon Max 9480 2P, No HBM, Max AVX512 FP16Xeon Max 9480 2P, No HBMXeon Max 9480 2P, HBM OnlyXeon Max 9480 2P, HBM CachingEPYC 9654 2PEPYC 9554 2P5001000150020002500SE +/- 9.42, N = 3SE +/- 10.44, N = 3SE +/- 1.42, N = 3SE +/- 6.17, N = 3SE +/- 5.11, N = 3SE +/- 5.30, N = 32445.49857.32706.83834.971092.97832.02MIN: 1611.66 / MAX: 3114.74MIN: 481.73 / MAX: 1422.59MIN: 455.92 / MAX: 1885.53MIN: 558.93 / MAX: 2258.39MIN: 825.28 / MAX: 1814.82MIN: 743.95 / MAX: 1258.171. (CXX) g++ options: -isystem -fsigned-char -ffunction-sections -fdata-sections -msse4.1 -msse4.2 -O3 -fno-strict-overflow -fwrapv -fPIC -fvisibility=hidden -Os -std=c++11 -MD -MT -MF
MinAvgMaxXeon Max 9480 2P, No HBM, Max AVX512 FP16267546628Xeon Max 9480 2P, No HBM205515616Xeon Max 9480 2P, HBM Only252563639Xeon Max 9480 2P, HBM Caching204538618EPYC 9654 2P60608699EPYC 9554 2P49596683OpenBenchmarking.orgWatts, Fewer Is BetterOpenVINO 2022.3CPU Power Consumption Monitor2004006008001000
MinAvgMaxXeon Max 9480 2P, No HBM, Max AVX512 FP16220034813515Xeon Max 9480 2P, No HBM180033133518Xeon Max 9480 2P, HBM Only350035003517Xeon Max 9480 2P, HBM Caching190030233512OpenBenchmarking.orgMegahertz, More Is BetterOpenVINO 2022.3CPU Peak Freq (Highest CPU Core Frequency) Monitor10002000300040005000
MinAvgMaxXeon Max 9480 2P, No HBM, Max AVX512 FP16408757822Xeon Max 9480 2P, No HBM295737874Xeon Max 9480 2P, HBM Only360745835Xeon Max 9480 2P, HBM Caching298736817OpenBenchmarking.orgWatts, Fewer Is BetterOpenVINO 2022.3System Power Consumption Monitor2004006008001000

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2022.3Model: Person Detection FP32 - Device: CPUXeon Max 9480 2P, No HBM, Max AVX512 FP16Xeon Max 9480 2P, No HBMXeon Max 9480 2P, HBM OnlyXeon Max 9480 2P, HBM CachingEPYC 9654 2PEPYC 9554 2P5001000150020002500SE +/- 11.98, N = 3SE +/- 6.05, N = 3SE +/- 7.70, N = 3SE +/- 7.20, N = 3SE +/- 5.33, N = 3SE +/- 1.07, N = 32449.29824.93719.49819.761098.96825.52MIN: 1558.62 / MAX: 3306.73MIN: 445.3 / MAX: 1462.45MIN: 473.54 / MAX: 2264.09MIN: 516.88 / MAX: 2598.39MIN: 825.62 / MAX: 1835.76MIN: 747.05 / MAX: 1238.391. (CXX) g++ options: -isystem -fsigned-char -ffunction-sections -fdata-sections -msse4.1 -msse4.2 -O3 -fno-strict-overflow -fwrapv -fPIC -fvisibility=hidden -Os -std=c++11 -MD -MT -MF
MinAvgMaxXeon Max 9480 2P, No HBM, Max AVX512 FP16241553629Xeon Max 9480 2P, No HBM245531618Xeon Max 9480 2P, HBM Only223567643Xeon Max 9480 2P, HBM Caching228547624EPYC 9654 2P60606703EPYC 9554 2P49600684OpenBenchmarking.orgWatts, Fewer Is BetterOpenVINO 2022.3CPU Power Consumption Monitor2004006008001000
MinAvgMaxXeon Max 9480 2P, No HBM, Max AVX512 FP16350035003511Xeon Max 9480 2P, No HBM208733963518Xeon Max 9480 2P, HBM Only264634963508Xeon Max 9480 2P, HBM Caching190030743506OpenBenchmarking.orgMegahertz, More Is BetterOpenVINO 2022.3CPU Peak Freq (Highest CPU Core Frequency) Monitor10002000300040005000
MinAvgMaxXeon Max 9480 2P, No HBM, Max AVX512 FP16413768860Xeon Max 9480 2P, No HBM378758872Xeon Max 9480 2P, HBM Only373752832Xeon Max 9480 2P, HBM Caching369748857OpenBenchmarking.orgWatts, Fewer Is BetterOpenVINO 2022.3System Power Consumption Monitor2004006008001000

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2022.3Model: Face Detection FP16-INT8 - Device: CPUXeon Max 9480 2P, No HBM, Max AVX512 FP16Xeon Max 9480 2P, No HBMXeon Max 9480 2P, HBM OnlyXeon Max 9480 2P, HBM CachingEPYC 9654 2PEPYC 9554 2P80160240320400SE +/- 0.33, N = 3SE +/- 0.34, N = 3SE +/- 0.04, N = 3SE +/- 0.44, N = 3SE +/- 0.04, N = 3SE +/- 0.22, N = 3262.84380.60302.45333.51250.39205.71MIN: 192.97 / MAX: 452.59MIN: 286.27 / MAX: 602.06MIN: 217.26 / MAX: 450.48MIN: 285.32 / MAX: 516.28MIN: 222.36 / MAX: 303.72MIN: 202.38 / MAX: 252.141. (CXX) g++ options: -isystem -fsigned-char -ffunction-sections -fdata-sections -msse4.1 -msse4.2 -O3 -fno-strict-overflow -fwrapv -fPIC -fvisibility=hidden -Os -std=c++11 -MD -MT -MF
MinAvgMaxXeon Max 9480 2P, No HBM, Max AVX512 FP16184553616Xeon Max 9480 2P, No HBM157500632Xeon Max 9480 2P, HBM Only249561682Xeon Max 9480 2P, HBM Caching168531616EPYC 9654 2P59613687EPYC 9554 2P50606675OpenBenchmarking.orgWatts, Fewer Is BetterOpenVINO 2022.3CPU Power Consumption Monitor2004006008001000
MinAvgMaxXeon Max 9480 2P, No HBM, Max AVX512 FP16180024923510Xeon Max 9480 2P, No HBM150021953515Xeon Max 9480 2P, HBM Only190033593505Xeon Max 9480 2P, HBM Caching170023703505OpenBenchmarking.orgMegahertz, More Is BetterOpenVINO 2022.3CPU Peak Freq (Highest CPU Core Frequency) Monitor10002000300040005000
MinAvgMaxXeon Max 9480 2P, No HBM, Max AVX512 FP16307762847Xeon Max 9480 2P, No HBM256717841Xeon Max 9480 2P, HBM Only365742888Xeon Max 9480 2P, HBM Caching270722818OpenBenchmarking.orgWatts, Fewer Is BetterOpenVINO 2022.3System Power Consumption Monitor2004006008001000

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2022.3Model: Weld Porosity Detection FP16 - Device: CPUXeon Max 9480 2P, No HBM, Max AVX512 FP16Xeon Max 9480 2P, No HBMXeon Max 9480 2P, HBM OnlyXeon Max 9480 2P, HBM CachingEPYC 9654 2PEPYC 9554 2P1326395265SE +/- 0.08, N = 3SE +/- 0.03, N = 3SE +/- 0.01, N = 3SE +/- 0.01, N = 3SE +/- 0.00, N = 3SE +/- 0.00, N = 356.606.655.926.444.863.97MIN: 44.94 / MAX: 112.121. (CXX) g++ options: -isystem -fsigned-char -ffunction-sections -fdata-sections -msse4.1 -msse4.2 -O3 -fno-strict-overflow -fwrapv -fPIC -fvisibility=hidden -Os -std=c++11 -MD -MT -MF
MinAvgMaxXeon Max 9480 2P, No HBM, Max AVX512 FP16268564605Xeon Max 9480 2P, No HBM165538595Xeon Max 9480 2P, HBM Only235582661Xeon Max 9480 2P, HBM Caching163557629EPYC 9654 2P57655705EPYC 9554 2P48648698OpenBenchmarking.orgWatts, Fewer Is BetterOpenVINO 2022.3CPU Power Consumption Monitor2004006008001000
MinAvgMaxXeon Max 9480 2P, No HBM, Max AVX512 FP16350035003509Xeon Max 9480 2P, No HBM160018463500Xeon Max 9480 2P, HBM Only190026183506Xeon Max 9480 2P, HBM Caching170019003500OpenBenchmarking.orgMegahertz, More Is BetterOpenVINO 2022.3CPU Peak Freq (Highest CPU Core Frequency) Monitor10002000300040005000
MinAvgMaxXeon Max 9480 2P, No HBM, Max AVX512 FP16420774811Xeon Max 9480 2P, No HBM268759840Xeon Max 9480 2P, HBM Only342768875Xeon Max 9480 2P, HBM Caching279756840OpenBenchmarking.orgWatts, Fewer Is BetterOpenVINO 2022.3System Power Consumption Monitor2004006008001000

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2022.3Model: Weld Porosity Detection FP16-INT8 - Device: CPUXeon Max 9480 2P, No HBM, Max AVX512 FP16Xeon Max 9480 2P, No HBMXeon Max 9480 2P, HBM OnlyXeon Max 9480 2P, HBM CachingEPYC 9654 2PEPYC 9554 2P3691215SE +/- 0.01, N = 3SE +/- 0.00, N = 3SE +/- 0.01, N = 3SE +/- 0.00, N = 3SE +/- 0.00, N = 3SE +/- 0.00, N = 310.473.453.073.479.958.09MIN: 7.24 / MAX: 56.73MIN: 2.47 / MAX: 46.27MIN: 2.48 / MAX: 68.92MIN: 2.48 / MAX: 56.57MIN: 8.36 / MAX: 40.39MIN: 7.89 / MAX: 45.391. (CXX) g++ options: -isystem -fsigned-char -ffunction-sections -fdata-sections -msse4.1 -msse4.2 -O3 -fno-strict-overflow -fwrapv -fPIC -fvisibility=hidden -Os -std=c++11 -MD -MT -MF
MinAvgMaxXeon Max 9480 2P, No HBM, Max AVX512 FP16168556603Xeon Max 9480 2P, No HBM161541594Xeon Max 9480 2P, HBM Only256583675Xeon Max 9480 2P, HBM Caching137552614EPYC 9654 2P59632685EPYC 9554 2P49623675OpenBenchmarking.orgWatts, Fewer Is BetterOpenVINO 2022.3CPU Power Consumption Monitor2004006008001000
MinAvgMaxXeon Max 9480 2P, No HBM, Max AVX512 FP16180020233500Xeon Max 9480 2P, No HBM160019343502Xeon Max 9480 2P, HBM Only180029323508Xeon Max 9480 2P, HBM Caching150019023504OpenBenchmarking.orgMegahertz, More Is BetterOpenVINO 2022.3CPU Peak Freq (Highest CPU Core Frequency) Monitor10002000300040005000
MinAvgMaxXeon Max 9480 2P, No HBM, Max AVX512 FP16276765811Xeon Max 9480 2P, No HBM284753808Xeon Max 9480 2P, HBM Only370768854Xeon Max 9480 2P, HBM Caching275752830OpenBenchmarking.orgWatts, Fewer Is BetterOpenVINO 2022.3System Power Consumption Monitor2004006008001000

Xeon Max 9480 2P, No HBM, Max AVX512 FP16

Kernel Notes: Transparent Huge Pages: madvise
Compiler Notes: --build=x86_64-linux-gnu --disable-vtable-verify --disable-werror --enable-cet --enable-checking=release --enable-clocale=gnu --enable-default-pie --enable-gnu-unique-object --enable-languages=c,ada,c++,go,d,fortran,objc,obj-c++,m2 --enable-libphobos-checking=release --enable-libstdcxx-debug --enable-libstdcxx-time=yes --enable-multiarch --enable-multilib --enable-nls --enable-objc-gc=auto --enable-offload-defaulted --enable-offload-targets=nvptx-none=/build/gcc-12-Pa930Z/gcc-12-12.2.0/debian/tmp-nvptx/usr,amdgcn-amdhsa=/build/gcc-12-Pa930Z/gcc-12-12.2.0/debian/tmp-gcn/usr --enable-plugin --enable-shared --enable-threads=posix --host=x86_64-linux-gnu --program-prefix=x86_64-linux-gnu- --target=x86_64-linux-gnu --with-abi=m64 --with-arch-32=i686 --with-default-libstdcxx-abi=new --with-gcc-major-version-only --with-multilib-list=m32,m64,mx32 --with-target-system-zlib=auto --with-tune=generic --without-cuda-driver -v
Processor Notes: Scaling Governor: intel_cpufreq performance - CPU Microcode: 0x2c0001d1
Python Notes: Python 3.11.2
Security Notes: itlb_multihit: Not affected + l1tf: Not affected + mds: Not affected + meltdown: Not affected + mmio_stale_data: Not affected + retbleed: Not affected + spec_store_bypass: Mitigation of SSB disabled via prctl + spectre_v1: Mitigation of usercopy/swapgs barriers and __user pointer sanitization + spectre_v2: Mitigation of Enhanced IBRS IBPB: conditional RSB filling PBRSB-eIBRS: SW sequence + srbds: Not affected + tsx_async_abort: Not affected

Testing initiated at 1 July 2023 13:22 by user phoronix.

Xeon Max 9480 2P, No HBM

Kernel Notes: Transparent Huge Pages: madvise
Compiler Notes: --build=x86_64-linux-gnu --disable-vtable-verify --disable-werror --enable-cet --enable-checking=release --enable-clocale=gnu --enable-default-pie --enable-gnu-unique-object --enable-languages=c,ada,c++,go,d,fortran,objc,obj-c++,m2 --enable-libphobos-checking=release --enable-libstdcxx-debug --enable-libstdcxx-time=yes --enable-multiarch --enable-multilib --enable-nls --enable-objc-gc=auto --enable-offload-defaulted --enable-offload-targets=nvptx-none=/build/gcc-12-Pa930Z/gcc-12-12.2.0/debian/tmp-nvptx/usr,amdgcn-amdhsa=/build/gcc-12-Pa930Z/gcc-12-12.2.0/debian/tmp-gcn/usr --enable-plugin --enable-shared --enable-threads=posix --host=x86_64-linux-gnu --program-prefix=x86_64-linux-gnu- --target=x86_64-linux-gnu --with-abi=m64 --with-arch-32=i686 --with-default-libstdcxx-abi=new --with-gcc-major-version-only --with-multilib-list=m32,m64,mx32 --with-target-system-zlib=auto --with-tune=generic --without-cuda-driver -v
Processor Notes: Scaling Governor: intel_cpufreq performance - CPU Microcode: 0x2c0001d1
Python Notes: Python 3.11.2
Security Notes: itlb_multihit: Not affected + l1tf: Not affected + mds: Not affected + meltdown: Not affected + mmio_stale_data: Not affected + retbleed: Not affected + spec_store_bypass: Mitigation of SSB disabled via prctl + spectre_v1: Mitigation of usercopy/swapgs barriers and __user pointer sanitization + spectre_v2: Mitigation of Enhanced IBRS IBPB: conditional RSB filling PBRSB-eIBRS: SW sequence + srbds: Not affected + tsx_async_abort: Not affected

Testing initiated at 30 June 2023 05:43 by user phoronix.

Xeon Max 9480 2P, HBM Caching

Processor: 2 x Intel Xeon Max 9480 @ 3.50GHz (112 Cores / 224 Threads), Motherboard: Supermicro X13DEM v1.10 (1.3 BIOS), Chipset: Intel Device 1bce, Memory: 512GB, Disk: 7682GB INTEL SSDPF2KX076TZ, Graphics: ASPEED, Monitor: VE228, Network: 2 x Broadcom BCM57508 NetXtreme-E 10Gb/25Gb/40Gb/50Gb/100Gb/200Gb

OS: Ubuntu 23.04, Kernel: 6.2.0-20-generic (x86_64), Desktop: GNOME Shell 44.0, Display Server: X Server 1.21.1.7, Compiler: GCC 12.2.0, File-System: ext4, Screen Resolution: 1920x1080

Kernel Notes: Transparent Huge Pages: madvise
Compiler Notes: --build=x86_64-linux-gnu --disable-vtable-verify --disable-werror --enable-cet --enable-checking=release --enable-clocale=gnu --enable-default-pie --enable-gnu-unique-object --enable-languages=c,ada,c++,go,d,fortran,objc,obj-c++,m2 --enable-libphobos-checking=release --enable-libstdcxx-debug --enable-libstdcxx-time=yes --enable-multiarch --enable-multilib --enable-nls --enable-objc-gc=auto --enable-offload-defaulted --enable-offload-targets=nvptx-none=/build/gcc-12-Pa930Z/gcc-12-12.2.0/debian/tmp-nvptx/usr,amdgcn-amdhsa=/build/gcc-12-Pa930Z/gcc-12-12.2.0/debian/tmp-gcn/usr --enable-plugin --enable-shared --enable-threads=posix --host=x86_64-linux-gnu --program-prefix=x86_64-linux-gnu- --target=x86_64-linux-gnu --with-abi=m64 --with-arch-32=i686 --with-default-libstdcxx-abi=new --with-gcc-major-version-only --with-multilib-list=m32,m64,mx32 --with-target-system-zlib=auto --with-tune=generic --without-cuda-driver -v
Processor Notes: Scaling Governor: intel_cpufreq performance - CPU Microcode: 0x2c0001d1
Python Notes: Python 3.11.2
Security Notes: itlb_multihit: Not affected + l1tf: Not affected + mds: Not affected + meltdown: Not affected + mmio_stale_data: Not affected + retbleed: Not affected + spec_store_bypass: Mitigation of SSB disabled via prctl + spectre_v1: Mitigation of usercopy/swapgs barriers and __user pointer sanitization + spectre_v2: Mitigation of Enhanced IBRS IBPB: conditional RSB filling PBRSB-eIBRS: SW sequence + srbds: Not affected + tsx_async_abort: Not affected

Testing initiated at 29 June 2023 09:57 by user phoronix.

Xeon Max 9480 2P, HBM Only

Processor: 2 x Intel Xeon Max 9480 @ 3.50GHz (112 Cores / 224 Threads), Motherboard: Supermicro X13DEM v1.10 (1.3 BIOS), Chipset: Intel Device 1bce, Memory: 128GB, Disk: 7682GB INTEL SSDPF2KX076TZ, Graphics: ASPEED, Monitor: VE228, Network: 2 x Broadcom BCM57508 NetXtreme-E 10Gb/25Gb/40Gb/50Gb/100Gb/200Gb

OS: Ubuntu 23.04, Kernel: 6.2.0-20-generic (x86_64), Desktop: GNOME Shell 44.0, Display Server: X Server 1.21.1.7, Compiler: GCC 12.2.0, File-System: ext4, Screen Resolution: 1920x1080

Kernel Notes: Transparent Huge Pages: madvise
Compiler Notes: --build=x86_64-linux-gnu --disable-vtable-verify --disable-werror --enable-cet --enable-checking=release --enable-clocale=gnu --enable-default-pie --enable-gnu-unique-object --enable-languages=c,ada,c++,go,d,fortran,objc,obj-c++,m2 --enable-libphobos-checking=release --enable-libstdcxx-debug --enable-libstdcxx-time=yes --enable-multiarch --enable-multilib --enable-nls --enable-objc-gc=auto --enable-offload-defaulted --enable-offload-targets=nvptx-none=/build/gcc-12-Pa930Z/gcc-12-12.2.0/debian/tmp-nvptx/usr,amdgcn-amdhsa=/build/gcc-12-Pa930Z/gcc-12-12.2.0/debian/tmp-gcn/usr --enable-plugin --enable-shared --enable-threads=posix --host=x86_64-linux-gnu --program-prefix=x86_64-linux-gnu- --target=x86_64-linux-gnu --with-abi=m64 --with-arch-32=i686 --with-default-libstdcxx-abi=new --with-gcc-major-version-only --with-multilib-list=m32,m64,mx32 --with-target-system-zlib=auto --with-tune=generic --without-cuda-driver -v
Processor Notes: Scaling Governor: intel_cpufreq performance - CPU Microcode: 0x2c0001d1
Python Notes: Python 3.11.2
Security Notes: itlb_multihit: Not affected + l1tf: Not affected + mds: Not affected + meltdown: Not affected + mmio_stale_data: Not affected + retbleed: Not affected + spec_store_bypass: Mitigation of SSB disabled via prctl + spectre_v1: Mitigation of usercopy/swapgs barriers and __user pointer sanitization + spectre_v2: Mitigation of Enhanced IBRS IBPB: conditional RSB filling PBRSB-eIBRS: SW sequence + srbds: Not affected + tsx_async_abort: Not affected

Testing initiated at 30 June 2023 11:10 by user phoronix.

EPYC 9554 2P

Processor: 2 x AMD EPYC 9554 64-Core @ 3.10GHz (128 Cores / 256 Threads), Motherboard: AMD Titanite_4G (RTI1007B BIOS), Chipset: AMD Device 14a4, Memory: 1520GB, Disk: 7682GB INTEL SSDPF2KX076TZ, Graphics: ASPEED, Monitor: VGA HDMI, Network: Broadcom NetXtreme BCM5720 PCIe

OS: Ubuntu 23.04, Kernel: 6.2.0-20-generic (x86_64), Desktop: GNOME Shell 44.0, Display Server: X Server 1.21.1.7, Compiler: GCC 12.2.0, File-System: ext4, Screen Resolution: 1920x1080

Kernel Notes: Transparent Huge Pages: madvise
Compiler Notes: --build=x86_64-linux-gnu --disable-vtable-verify --disable-werror --enable-cet --enable-checking=release --enable-clocale=gnu --enable-default-pie --enable-gnu-unique-object --enable-languages=c,ada,c++,go,d,fortran,objc,obj-c++,m2 --enable-libphobos-checking=release --enable-libstdcxx-debug --enable-libstdcxx-time=yes --enable-multiarch --enable-multilib --enable-nls --enable-objc-gc=auto --enable-offload-defaulted --enable-offload-targets=nvptx-none=/build/gcc-12-Pa930Z/gcc-12-12.2.0/debian/tmp-nvptx/usr,amdgcn-amdhsa=/build/gcc-12-Pa930Z/gcc-12-12.2.0/debian/tmp-gcn/usr --enable-plugin --enable-shared --enable-threads=posix --host=x86_64-linux-gnu --program-prefix=x86_64-linux-gnu- --target=x86_64-linux-gnu --with-abi=m64 --with-arch-32=i686 --with-default-libstdcxx-abi=new --with-gcc-major-version-only --with-multilib-list=m32,m64,mx32 --with-target-system-zlib=auto --with-tune=generic --without-cuda-driver -v
Processor Notes: Scaling Governor: acpi-cpufreq performance (Boost: Enabled) - CPU Microcode: 0xa101121
Python Notes: Python 3.11.2
Security Notes: itlb_multihit: Not affected + l1tf: Not affected + mds: Not affected + meltdown: Not affected + mmio_stale_data: Not affected + retbleed: Not affected + spec_store_bypass: Mitigation of SSB disabled via prctl + spectre_v1: Mitigation of usercopy/swapgs barriers and __user pointer sanitization + spectre_v2: Mitigation of Retpolines IBPB: conditional IBRS_FW STIBP: always-on RSB filling PBRSB-eIBRS: Not affected + srbds: Not affected + tsx_async_abort: Not affected

Testing initiated at 28 June 2023 20:58 by user phoronix.

EPYC 9654 2P

Processor: 2 x AMD EPYC 9654 96-Core @ 2.40GHz (192 Cores / 384 Threads), Motherboard: AMD Titanite_4G (RTI1007B BIOS), Chipset: AMD Device 14a4, Memory: 1520GB, Disk: 7682GB INTEL SSDPF2KX076TZ, Graphics: ASPEED, Monitor: VGA HDMI, Network: Broadcom NetXtreme BCM5720 PCIe

OS: Ubuntu 23.04, Kernel: 6.2.0-20-generic (x86_64), Desktop: GNOME Shell 44.0, Display Server: X Server 1.21.1.7, Compiler: GCC 12.2.0, File-System: ext4, Screen Resolution: 1920x1080

Kernel Notes: Transparent Huge Pages: madvise
Compiler Notes: --build=x86_64-linux-gnu --disable-vtable-verify --disable-werror --enable-cet --enable-checking=release --enable-clocale=gnu --enable-default-pie --enable-gnu-unique-object --enable-languages=c,ada,c++,go,d,fortran,objc,obj-c++,m2 --enable-libphobos-checking=release --enable-libstdcxx-debug --enable-libstdcxx-time=yes --enable-multiarch --enable-multilib --enable-nls --enable-objc-gc=auto --enable-offload-defaulted --enable-offload-targets=nvptx-none=/build/gcc-12-Pa930Z/gcc-12-12.2.0/debian/tmp-nvptx/usr,amdgcn-amdhsa=/build/gcc-12-Pa930Z/gcc-12-12.2.0/debian/tmp-gcn/usr --enable-plugin --enable-shared --enable-threads=posix --host=x86_64-linux-gnu --program-prefix=x86_64-linux-gnu- --target=x86_64-linux-gnu --with-abi=m64 --with-arch-32=i686 --with-default-libstdcxx-abi=new --with-gcc-major-version-only --with-multilib-list=m32,m64,mx32 --with-target-system-zlib=auto --with-tune=generic --without-cuda-driver -v
Processor Notes: Scaling Governor: acpi-cpufreq performance (Boost: Enabled) - CPU Microcode: 0xa101121
Python Notes: Python 3.11.2
Security Notes: itlb_multihit: Not affected + l1tf: Not affected + mds: Not affected + meltdown: Not affected + mmio_stale_data: Not affected + retbleed: Not affected + spec_store_bypass: Mitigation of SSB disabled via prctl + spectre_v1: Mitigation of usercopy/swapgs barriers and __user pointer sanitization + spectre_v2: Mitigation of Retpolines IBPB: conditional IBRS_FW STIBP: always-on RSB filling PBRSB-eIBRS: Not affected + srbds: Not affected + tsx_async_abort: Not affected

Testing initiated at 28 June 2023 15:03 by user phoronix.