Xeon Max AMX HBM2e

Benchmarks for a future article on Phoronix.

Compare your own system(s) to this result file with the Phoronix Test Suite by running the command: phoronix-test-suite benchmark 2306302-NE-XEONMAXAM96
Jump To Table - Results

View

Do Not Show Noisy Results
Do Not Show Results With Incomplete Data
Do Not Show Results With Little Change/Spread
List Notable Results
Show Result Confidence Charts

Limit displaying results to tests within:

CPU Massive 2 Tests
Creator Workloads 2 Tests
HPC - High Performance Computing 4 Tests
Machine Learning 4 Tests
Multi-Core 2 Tests
Intel oneAPI 2 Tests
Python Tests 3 Tests

Statistics

Show Overall Harmonic Mean(s)
Show Overall Geometric Mean
Show Geometric Means Per-Suite/Category
Show Wins / Losses Counts (Pie Chart)
Normalize Results
Remove Outliers Before Calculating Averages

Graph Settings

Force Line Graphs Where Applicable
Convert To Scalar Where Applicable
Disable Color Branding
Prefer Vertical Bar Graphs
No Box Plots
On Line Graphs With Missing Data, Connect The Line Gaps

Additional Graphs

Show Perf Per Core/Thread Calculation Graphs Where Applicable
Show Perf Per Clock Calculation Graphs Where Applicable

Multi-Way Comparison

Condense Multi-Option Tests Into Single Result Graphs
Condense Test Profiles With Multiple Version Results Into Single Result Graphs

Table

Show Detailed System Result Table

Run Management

Highlight
Result
Hide
Result
Result
Identifier
Performance Per
Dollar
Date
Run
  Test
  Duration
EPYC 9654 2P
June 28 2023
  6 Hours, 59 Minutes
EPYC 9554 2P
June 28 2023
  5 Hours, 42 Minutes
Xeon Max 9480 2P, HBM Caching
June 29 2023
  13 Hours, 47 Minutes
Invert Hiding All Results Option
  8 Hours, 49 Minutes

Only show results where is faster than
Only show results matching title/arguments (delimit multiple options with a comma):
Do not show results matching title/arguments (delimit multiple options with a comma):

EPYC 9654 2P

Processor: 2 x AMD EPYC 9654 96-Core @ 2.40GHz (192 Cores / 384 Threads), Motherboard: AMD Titanite_4G (RTI1007B BIOS), Chipset: AMD Device 14a4, Memory: 1520GB, Disk: 7682GB INTEL SSDPF2KX076TZ, Graphics: ASPEED, Monitor: VGA HDMI, Network: Broadcom NetXtreme BCM5720 PCIe

OS: Ubuntu 23.04, Kernel: 6.2.0-20-generic (x86_64), Desktop: GNOME Shell 44.0, Display Server: X Server 1.21.1.7, Compiler: GCC 12.2.0, File-System: ext4, Screen Resolution: 1920x1080

Kernel Notes: Transparent Huge Pages: madvise
Compiler Notes: --build=x86_64-linux-gnu --disable-vtable-verify --disable-werror --enable-cet --enable-checking=release --enable-clocale=gnu --enable-default-pie --enable-gnu-unique-object --enable-languages=c,ada,c++,go,d,fortran,objc,obj-c++,m2 --enable-libphobos-checking=release --enable-libstdcxx-debug --enable-libstdcxx-time=yes --enable-multiarch --enable-multilib --enable-nls --enable-objc-gc=auto --enable-offload-defaulted --enable-offload-targets=nvptx-none=/build/gcc-12-Pa930Z/gcc-12-12.2.0/debian/tmp-nvptx/usr,amdgcn-amdhsa=/build/gcc-12-Pa930Z/gcc-12-12.2.0/debian/tmp-gcn/usr --enable-plugin --enable-shared --enable-threads=posix --host=x86_64-linux-gnu --program-prefix=x86_64-linux-gnu- --target=x86_64-linux-gnu --with-abi=m64 --with-arch-32=i686 --with-default-libstdcxx-abi=new --with-gcc-major-version-only --with-multilib-list=m32,m64,mx32 --with-target-system-zlib=auto --with-tune=generic --without-cuda-driver -v
Processor Notes: Scaling Governor: acpi-cpufreq performance (Boost: Enabled) - CPU Microcode: 0xa101121
Python Notes: Python 3.11.2
Security Notes: itlb_multihit: Not affected + l1tf: Not affected + mds: Not affected + meltdown: Not affected + mmio_stale_data: Not affected + retbleed: Not affected + spec_store_bypass: Mitigation of SSB disabled via prctl + spectre_v1: Mitigation of usercopy/swapgs barriers and __user pointer sanitization + spectre_v2: Mitigation of Retpolines IBPB: conditional IBRS_FW STIBP: always-on RSB filling PBRSB-eIBRS: Not affected + srbds: Not affected + tsx_async_abort: Not affected

EPYC 9554 2P

Changed Processor to 2 x AMD EPYC 9554 64-Core @ 3.10GHz (128 Cores / 256 Threads).

Xeon Max 9480 2P, HBM Caching

Processor: 2 x Intel Xeon Max 9480 @ 3.50GHz (112 Cores / 224 Threads), Motherboard: Supermicro X13DEM v1.10 (1.3 BIOS), Chipset: Intel Device 1bce, Memory: 512GB, Disk: 7682GB INTEL SSDPF2KX076TZ, Graphics: ASPEED, Monitor: VE228, Network: 2 x Broadcom BCM57508 NetXtreme-E 10Gb/25Gb/40Gb/50Gb/100Gb/200Gb

Kernel Notes: Transparent Huge Pages: madvise
Compiler Notes: --build=x86_64-linux-gnu --disable-vtable-verify --disable-werror --enable-cet --enable-checking=release --enable-clocale=gnu --enable-default-pie --enable-gnu-unique-object --enable-languages=c,ada,c++,go,d,fortran,objc,obj-c++,m2 --enable-libphobos-checking=release --enable-libstdcxx-debug --enable-libstdcxx-time=yes --enable-multiarch --enable-multilib --enable-nls --enable-objc-gc=auto --enable-offload-defaulted --enable-offload-targets=nvptx-none=/build/gcc-12-Pa930Z/gcc-12-12.2.0/debian/tmp-nvptx/usr,amdgcn-amdhsa=/build/gcc-12-Pa930Z/gcc-12-12.2.0/debian/tmp-gcn/usr --enable-plugin --enable-shared --enable-threads=posix --host=x86_64-linux-gnu --program-prefix=x86_64-linux-gnu- --target=x86_64-linux-gnu --with-abi=m64 --with-arch-32=i686 --with-default-libstdcxx-abi=new --with-gcc-major-version-only --with-multilib-list=m32,m64,mx32 --with-target-system-zlib=auto --with-tune=generic --without-cuda-driver -v
Processor Notes: Scaling Governor: intel_cpufreq performance - CPU Microcode: 0x2c0001d1
Python Notes: Python 3.11.2
Security Notes: itlb_multihit: Not affected + l1tf: Not affected + mds: Not affected + meltdown: Not affected + mmio_stale_data: Not affected + retbleed: Not affected + spec_store_bypass: Mitigation of SSB disabled via prctl + spectre_v1: Mitigation of usercopy/swapgs barriers and __user pointer sanitization + spectre_v2: Mitigation of Enhanced IBRS IBPB: conditional RSB filling PBRSB-eIBRS: SW sequence + srbds: Not affected + tsx_async_abort: Not affected

Xeon Max AMX HBM2eProcessorMotherboardChipsetMemoryDiskGraphicsMonitorNetworkOSKernelDesktopDisplay ServerCompilerFile-SystemScreen ResolutionEPYC 9654 2PEPYC 9554 2PXeon Max 9480 2P, HBM Caching2 x AMD EPYC 9654 96-Core @ 2.40GHz (192 Cores / 384 Threads)AMD Titanite_4G (RTI1007B BIOS)AMD Device 14a41520GB7682GB INTEL SSDPF2KX076TZASPEEDVGA HDMIBroadcom NetXtreme BCM5720 PCIeUbuntu 23.046.2.0-20-generic (x86_64)GNOME Shell 44.0X Server 1.21.1.7GCC 12.2.0ext41920x10802 x AMD EPYC 9554 64-Core @ 3.10GHz (128 Cores / 256 Threads)2 x Intel Xeon Max 9480 @ 3.50GHz (112 Cores / 224 Threads)Supermicro X13DEM v1.10 (1.3 BIOS)Intel Device 1bce512GBVE2282 x Broadcom BCM57508 NetXtreme-E 10Gb/25Gb/40Gb/50Gb/100Gb/200GbOpenBenchmarking.orgKernel Details- Transparent Huge Pages: madviseCompiler Details- --build=x86_64-linux-gnu --disable-vtable-verify --disable-werror --enable-cet --enable-checking=release --enable-clocale=gnu --enable-default-pie --enable-gnu-unique-object --enable-languages=c,ada,c++,go,d,fortran,objc,obj-c++,m2 --enable-libphobos-checking=release --enable-libstdcxx-debug --enable-libstdcxx-time=yes --enable-multiarch --enable-multilib --enable-nls --enable-objc-gc=auto --enable-offload-defaulted --enable-offload-targets=nvptx-none=/build/gcc-12-Pa930Z/gcc-12-12.2.0/debian/tmp-nvptx/usr,amdgcn-amdhsa=/build/gcc-12-Pa930Z/gcc-12-12.2.0/debian/tmp-gcn/usr --enable-plugin --enable-shared --enable-threads=posix --host=x86_64-linux-gnu --program-prefix=x86_64-linux-gnu- --target=x86_64-linux-gnu --with-abi=m64 --with-arch-32=i686 --with-default-libstdcxx-abi=new --with-gcc-major-version-only --with-multilib-list=m32,m64,mx32 --with-target-system-zlib=auto --with-tune=generic --without-cuda-driver -v Processor Details- EPYC 9654 2P: Scaling Governor: acpi-cpufreq performance (Boost: Enabled) - CPU Microcode: 0xa101121- EPYC 9554 2P: Scaling Governor: acpi-cpufreq performance (Boost: Enabled) - CPU Microcode: 0xa101121- Xeon Max 9480 2P, HBM Caching: Scaling Governor: intel_cpufreq performance - CPU Microcode: 0x2c0001d1Python Details- Python 3.11.2Security Details- EPYC 9654 2P: itlb_multihit: Not affected + l1tf: Not affected + mds: Not affected + meltdown: Not affected + mmio_stale_data: Not affected + retbleed: Not affected + spec_store_bypass: Mitigation of SSB disabled via prctl + spectre_v1: Mitigation of usercopy/swapgs barriers and __user pointer sanitization + spectre_v2: Mitigation of Retpolines IBPB: conditional IBRS_FW STIBP: always-on RSB filling PBRSB-eIBRS: Not affected + srbds: Not affected + tsx_async_abort: Not affected - EPYC 9554 2P: itlb_multihit: Not affected + l1tf: Not affected + mds: Not affected + meltdown: Not affected + mmio_stale_data: Not affected + retbleed: Not affected + spec_store_bypass: Mitigation of SSB disabled via prctl + spectre_v1: Mitigation of usercopy/swapgs barriers and __user pointer sanitization + spectre_v2: Mitigation of Retpolines IBPB: conditional IBRS_FW STIBP: always-on RSB filling PBRSB-eIBRS: Not affected + srbds: Not affected + tsx_async_abort: Not affected - Xeon Max 9480 2P, HBM Caching: itlb_multihit: Not affected + l1tf: Not affected + mds: Not affected + meltdown: Not affected + mmio_stale_data: Not affected + retbleed: Not affected + spec_store_bypass: Mitigation of SSB disabled via prctl + spectre_v1: Mitigation of usercopy/swapgs barriers and __user pointer sanitization + spectre_v2: Mitigation of Enhanced IBRS IBPB: conditional RSB filling PBRSB-eIBRS: SW sequence + srbds: Not affected + tsx_async_abort: Not affected

EPYC 9654 2PEPYC 9554 2PXeon Max 9480 2P, HBM CachingResult OverviewPhoronix Test Suite100%171%242%313%384%oneDNNlibxsmmTensorFlowONNX RuntimeOpenVINO

EPYC 9654 2PEPYC 9554 2PXeon Max 9480 2P, HBM CachingPer Watt Result OverviewPhoronix Test Suite100%168%236%303%libxsmmTensorFlowTensorFlowTensorFlowTensorFlowTensorFlowTensorFlowTensorFlowTensorFlow128CPU - 16 - VGG-16CPU - 16 - AlexNetCPU - 512 - VGG-16CPU - 512 - AlexNetCPU - 16 - GoogLeNetCPU - 16 - ResNet-50CPU - 512 - GoogLeNetCPU - 512 - ResNet-50

Xeon Max AMX HBM2eopenvino: Face Detection FP16 - CPUopenvino: Person Detection FP16 - CPUopenvino: Person Detection FP32 - CPUopenvino: Vehicle Detection FP16 - CPUopenvino: Face Detection FP16-INT8 - CPUopenvino: Vehicle Detection FP16-INT8 - CPUopenvino: Weld Porosity Detection FP16 - CPUopenvino: Machine Translation EN To DE FP16 - CPUopenvino: Weld Porosity Detection FP16-INT8 - CPUopenvino: Person Vehicle Bike Detection FP16 - CPUopenvino: Age Gender Recognition Retail 0013 FP16 - CPUopenvino: Age Gender Recognition Retail 0013 FP16-INT8 - CPUlibxsmm: 128libxsmm: 256libxsmm: 32libxsmm: 64tensorflow: CPU - 16 - VGG-16tensorflow: CPU - 16 - AlexNettensorflow: CPU - 512 - VGG-16tensorflow: CPU - 512 - AlexNettensorflow: CPU - 16 - GoogLeNettensorflow: CPU - 16 - ResNet-50tensorflow: CPU - 512 - GoogLeNettensorflow: CPU - 512 - ResNet-50onnx: GPT-2 - CPU - Standardonnx: yolov4 - CPU - Standardonnx: bertsquad-12 - CPU - Standardonnx: CaffeNet 12-int8 - CPU - Standardonnx: fcn-resnet101-11 - CPU - Standardonnx: ArcFace ResNet-100 - CPU - Standardonnx: ResNet50 v1-12-int8 - CPU - Standardonnx: GPT-2 - CPU - Standardonnx: yolov4 - CPU - Standardonnx: bertsquad-12 - CPU - Standardonnx: CaffeNet 12-int8 - CPU - Standardonnx: fcn-resnet101-11 - CPU - Standardonnx: ArcFace ResNet-100 - CPU - Standardonnx: ResNet50 v1-12-int8 - CPU - Standardonednn: IP Shapes 1D - bf16bf16bf16 - CPUonednn: IP Shapes 3D - bf16bf16bf16 - CPUonednn: Convolution Batch Shapes Auto - bf16bf16bf16 - CPUonednn: Deconvolution Batch shapes_1d - bf16bf16bf16 - CPUonednn: Deconvolution Batch shapes_3d - bf16bf16bf16 - CPUonednn: Recurrent Neural Network Training - bf16bf16bf16 - CPUonednn: Recurrent Neural Network Inference - bf16bf16bf16 - CPUopenvino: Face Detection FP16 - CPUopenvino: Person Detection FP16 - CPUopenvino: Person Detection FP32 - CPUopenvino: Vehicle Detection FP16 - CPUopenvino: Face Detection FP16-INT8 - CPUopenvino: Vehicle Detection FP16-INT8 - CPUopenvino: Weld Porosity Detection FP16 - CPUopenvino: Machine Translation EN To DE FP16 - CPUopenvino: Weld Porosity Detection FP16-INT8 - CPUopenvino: Person Vehicle Bike Detection FP16 - CPUopenvino: Age Gender Recognition Retail 0013 FP16 - CPUopenvino: Age Gender Recognition Retail 0013 FP16-INT8 - CPUEPYC 9654 2PEPYC 9554 2PXeon Max 9480 2P, HBM Caching101.5643.6443.407390.97191.4110985.179843.03950.9419170.619133.22146867.48124618.323426.14379.61298.02270.247.27166.20120.621757.1664.4925.25535.52171.56103.1844.9003310.17944435.1944.6979719.3084146.5559.68904204.09198.99572.30611212.86351.78986.8705316.98233.802090.4610532.533260.7321002492.692790.90471.821092.971098.966.49250.394.364.8650.429.955.250.550.9982.0438.2438.535981.57155.289010.458039.57786.0115659.327447.60162207.97128593.244112.94603.31402.22692.760.31257.94109.891813.6894.4133.40593.20188.17132.7215.1050511.6871495.2404.6598224.0122152.4397.64735195.89985.90092.01949214.60641.64456.5598911.196692.205960.3113781.761620.4572721305.66925.767389.53832.02825.525.34205.713.543.9740.678.094.290.520.88120.4033.4033.972315.12334.895528.0316941.99590.9230938.506265.77108515.1259457.141814.321.09230.1130.19718.5498.4340.15236.4476.93218.6927.7004611.4576562.2185.3351423.7729125.2844.70849130.37087.76381.78296191.62342.18018.021667.8801691.98143.673560.4830753.5352515294.61325.45232.01834.97819.7612.07333.5120.236.4447.293.4717.830.441.40OpenBenchmarking.org

CPU Peak Freq (Highest CPU Core Frequency) Monitor

OpenBenchmarking.orgMegahertzCPU Peak Freq (Highest CPU Core Frequency) MonitorPhoronix Test Suite System MonitoringXeon Max 9480 2P, HBM Caching6001200180024003000Min: 1100 / Avg: 3207.85 / Max: 3577

CPU Power Consumption Monitor

OpenBenchmarking.orgWattsCPU Power Consumption MonitorPhoronix Test Suite System MonitoringEPYC 9554 2PEPYC 9654 2PXeon Max 9480 2P, HBM Caching160320480640800Min: 34.39 / Avg: 413.82 / Max: 710.63Min: 31.88 / Avg: 448.81 / Max: 725.51Min: 87.33 / Avg: 548.37 / Max: 899.87

System Power Consumption Monitor

OpenBenchmarking.orgWattsSystem Power Consumption MonitorPhoronix Test Suite System MonitoringXeon Max 9480 2P, HBM Caching160320480640800Min: 260 / Avg: 751.74 / Max: 907

OpenVINO

This is a test of the Intel OpenVINO, a toolkit around neural networks, using its built-in benchmarking support and analyzing the throughput and latency for various models. Learn more via the OpenBenchmarking.org test page.

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2022.3Model: Face Detection FP16 - Device: CPUXeon Max 9480 2P, HBM CachingEPYC 9654 2PEPYC 9554 2P306090120150SE +/- 0.41, N = 3SE +/- 0.25, N = 3SE +/- 0.04, N = 3120.40101.5682.041. (CXX) g++ options: -isystem -fsigned-char -ffunction-sections -fdata-sections -msse4.1 -msse4.2 -O3 -fno-strict-overflow -fwrapv -fPIC -fvisibility=hidden -Os -std=c++11 -MD -MT -MF

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2022.3Model: Person Detection FP16 - Device: CPUEPYC 9654 2PEPYC 9554 2PXeon Max 9480 2P, HBM Caching1020304050SE +/- 0.21, N = 3SE +/- 0.23, N = 3SE +/- 0.24, N = 343.6438.2433.401. (CXX) g++ options: -isystem -fsigned-char -ffunction-sections -fdata-sections -msse4.1 -msse4.2 -O3 -fno-strict-overflow -fwrapv -fPIC -fvisibility=hidden -Os -std=c++11 -MD -MT -MF

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2022.3Model: Person Detection FP32 - Device: CPUEPYC 9654 2PEPYC 9554 2PXeon Max 9480 2P, HBM Caching1020304050SE +/- 0.20, N = 3SE +/- 0.04, N = 3SE +/- 0.31, N = 343.4038.5333.971. (CXX) g++ options: -isystem -fsigned-char -ffunction-sections -fdata-sections -msse4.1 -msse4.2 -O3 -fno-strict-overflow -fwrapv -fPIC -fvisibility=hidden -Os -std=c++11 -MD -MT -MF

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2022.3Model: Vehicle Detection FP16 - Device: CPUEPYC 9654 2PEPYC 9554 2PXeon Max 9480 2P, HBM Caching16003200480064008000SE +/- 2.12, N = 3SE +/- 3.49, N = 3SE +/- 17.29, N = 137390.975981.572315.121. (CXX) g++ options: -isystem -fsigned-char -ffunction-sections -fdata-sections -msse4.1 -msse4.2 -O3 -fno-strict-overflow -fwrapv -fPIC -fvisibility=hidden -Os -std=c++11 -MD -MT -MF

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2022.3Model: Face Detection FP16-INT8 - Device: CPUXeon Max 9480 2P, HBM CachingEPYC 9654 2PEPYC 9554 2P70140210280350SE +/- 0.45, N = 3SE +/- 0.06, N = 3SE +/- 0.14, N = 3334.89191.41155.281. (CXX) g++ options: -isystem -fsigned-char -ffunction-sections -fdata-sections -msse4.1 -msse4.2 -O3 -fno-strict-overflow -fwrapv -fPIC -fvisibility=hidden -Os -std=c++11 -MD -MT -MF

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2022.3Model: Vehicle Detection FP16-INT8 - Device: CPUEPYC 9654 2PEPYC 9554 2PXeon Max 9480 2P, HBM Caching2K4K6K8K10KSE +/- 0.53, N = 3SE +/- 2.20, N = 3SE +/- 25.18, N = 310985.179010.455528.031. (CXX) g++ options: -isystem -fsigned-char -ffunction-sections -fdata-sections -msse4.1 -msse4.2 -O3 -fno-strict-overflow -fwrapv -fPIC -fvisibility=hidden -Os -std=c++11 -MD -MT -MF

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2022.3Model: Weld Porosity Detection FP16 - Device: CPUXeon Max 9480 2P, HBM CachingEPYC 9654 2PEPYC 9554 2P4K8K12K16K20KSE +/- 13.63, N = 3SE +/- 2.32, N = 3SE +/- 1.22, N = 316941.999843.038039.571. (CXX) g++ options: -isystem -fsigned-char -ffunction-sections -fdata-sections -msse4.1 -msse4.2 -O3 -fno-strict-overflow -fwrapv -fPIC -fvisibility=hidden -Os -std=c++11 -MD -MT -MF

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2022.3Model: Machine Translation EN To DE FP16 - Device: CPUEPYC 9654 2PEPYC 9554 2PXeon Max 9480 2P, HBM Caching2004006008001000SE +/- 0.77, N = 3SE +/- 1.56, N = 3SE +/- 0.19, N = 3950.94786.01590.921. (CXX) g++ options: -isystem -fsigned-char -ffunction-sections -fdata-sections -msse4.1 -msse4.2 -O3 -fno-strict-overflow -fwrapv -fPIC -fvisibility=hidden -Os -std=c++11 -MD -MT -MF

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2022.3Model: Weld Porosity Detection FP16-INT8 - Device: CPUXeon Max 9480 2P, HBM CachingEPYC 9654 2PEPYC 9554 2P7K14K21K28K35KSE +/- 38.32, N = 3SE +/- 4.81, N = 3SE +/- 3.60, N = 330938.5019170.6115659.321. (CXX) g++ options: -isystem -fsigned-char -ffunction-sections -fdata-sections -msse4.1 -msse4.2 -O3 -fno-strict-overflow -fwrapv -fPIC -fvisibility=hidden -Os -std=c++11 -MD -MT -MF

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2022.3Model: Person Vehicle Bike Detection FP16 - Device: CPUEPYC 9654 2PEPYC 9554 2PXeon Max 9480 2P, HBM Caching2K4K6K8K10KSE +/- 3.38, N = 3SE +/- 8.54, N = 3SE +/- 8.17, N = 39133.227447.606265.771. (CXX) g++ options: -isystem -fsigned-char -ffunction-sections -fdata-sections -msse4.1 -msse4.2 -O3 -fno-strict-overflow -fwrapv -fPIC -fvisibility=hidden -Os -std=c++11 -MD -MT -MF

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2022.3Model: Age Gender Recognition Retail 0013 FP16 - Device: CPUEPYC 9554 2PEPYC 9654 2PXeon Max 9480 2P, HBM Caching30K60K90K120K150KSE +/- 685.52, N = 3SE +/- 1825.86, N = 3SE +/- 967.90, N = 3162207.97146867.48108515.121. (CXX) g++ options: -isystem -fsigned-char -ffunction-sections -fdata-sections -msse4.1 -msse4.2 -O3 -fno-strict-overflow -fwrapv -fPIC -fvisibility=hidden -Os -std=c++11 -MD -MT -MF

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2022.3Model: Age Gender Recognition Retail 0013 FP16-INT8 - Device: CPUEPYC 9554 2PEPYC 9654 2PXeon Max 9480 2P, HBM Caching30K60K90K120K150KSE +/- 574.97, N = 3SE +/- 611.95, N = 3SE +/- 1694.26, N = 12128593.24124618.3259457.141. (CXX) g++ options: -isystem -fsigned-char -ffunction-sections -fdata-sections -msse4.1 -msse4.2 -O3 -fno-strict-overflow -fwrapv -fPIC -fvisibility=hidden -Os -std=c++11 -MD -MT -MF

libxsmm

OpenBenchmarking.orgGFLOPS/s Per Watt, More Is Betterlibxsmm 2-1.17-3645M N K: 256EPYC 9554 2PEPYC 9654 2P369121511.6710.07

OpenBenchmarking.orgGFLOPS/s Per Watt, More Is Betterlibxsmm 2-1.17-3645M N K: 32EPYC 9554 2PEPYC 9654 2P1.0762.1523.2284.3045.384.7823.609

OpenBenchmarking.orgGFLOPS/s Per Watt, More Is Betterlibxsmm 2-1.17-3645M N K: 64EPYC 9554 2PEPYC 9654 2P2468107.8395.975

OpenBenchmarking.orgGFLOPS/s, More Is Betterlibxsmm 2-1.17-3645M N K: 128EPYC 9554 2PEPYC 9654 2PXeon Max 9480 2P, HBM Caching9001800270036004500SE +/- 21.03, N = 3SE +/- 29.11, N = 3SE +/- 22.78, N = 94112.93426.11814.31. (CXX) g++ options: -dynamic -Bstatic -static-libgcc -lgomp -lm -lrt -ldl -lquadmath -lstdc++ -pthread -fPIC -std=c++14 -O2 -fopenmp-simd -funroll-loops -ftree-vectorize -fdata-sections -ffunction-sections -fvisibility=hidden -msse4.2
OpenBenchmarking.orgGFLOPS/s Per Watt, More Is Betterlibxsmm 2-1.17-3645M N K: 128EPYC 9554 2PEPYC 9654 2PXeon Max 9480 2P, HBM Caching369121510.2857.7663.209
OpenBenchmarking.orgWatts, Fewer Is Betterlibxsmm 2-1.17-3645CPU Power Consumption MonitorEPYC 9554 2PEPYC 9654 2PXeon Max 9480 2P, HBM Caching120240360480600Min: 47.03 / Avg: 399.88 / Max: 668.24Min: 99.14 / Avg: 441.18 / Max: 632.27Min: 256.12 / Avg: 565.34 / Max: 625.181. EPYC 9554 2P: Approximate power consumption of 286182 Joules per run.2. EPYC 9654 2P: Approximate power consumption of 365441 Joules per run.3. Xeon Max 9480 2P, HBM Caching: Approximate power consumption of 387511 Joules per run.
OpenBenchmarking.orgMegahertz, More Is Betterlibxsmm 2-1.17-3645CPU Peak Freq (Highest CPU Core Frequency) MonitorXeon Max 9480 2P, HBM Caching6001200180024003000Min: 1122 / Avg: 3487.45 / Max: 3507
OpenBenchmarking.orgWatts, Fewer Is Betterlibxsmm 2-1.17-3645System Power Consumption MonitorXeon Max 9480 2P, HBM Caching150300450600750Min: 395 / Avg: 776.46 / Max: 8321. Xeon Max 9480 2P, HBM Caching: Approximate power consumption of 533258 Joules per run.

OpenBenchmarking.orgGFLOPS/s, More Is Betterlibxsmm 2-1.17-3645M N K: 256EPYC 9554 2PEPYC 9654 2P10002000300040005000SE +/- 34.70, N = 3SE +/- 47.44, N = 54603.34379.61. (CXX) g++ options: -dynamic -Bstatic -static-libgcc -lgomp -lm -lrt -ldl -lquadmath -lstdc++ -pthread -fPIC -std=c++14 -O2 -fopenmp-simd -funroll-loops -ftree-vectorize -fdata-sections -ffunction-sections -fvisibility=hidden -msse4.2
OpenBenchmarking.orgGFLOPS/s Per Watt, More Is Betterlibxsmm 2-1.17-3645M N K: 256EPYC 9554 2PEPYC 9654 2P369121511.6710.07
MinAvgMaxEPYC 9554 2P48395695EPYC 9654 2P90435665OpenBenchmarking.orgWatts, Fewer Is Betterlibxsmm 2-1.17-3645CPU Power Consumption Monitor20040060080010001. EPYC 9554 2P: Approximate power consumption of 97312 Joules per run.2. EPYC 9654 2P: Approximate power consumption of 128179 Joules per run.

OpenBenchmarking.orgGFLOPS/s, More Is Betterlibxsmm 2-1.17-3645M N K: 32EPYC 9554 2PEPYC 9654 2P30060090012001500SE +/- 4.06, N = 8SE +/- 9.98, N = 151402.21298.01. (CXX) g++ options: -dynamic -Bstatic -static-libgcc -lgomp -lm -lrt -ldl -lquadmath -lstdc++ -pthread -fPIC -std=c++14 -O2 -fopenmp-simd -funroll-loops -ftree-vectorize -fdata-sections -ffunction-sections -fvisibility=hidden -msse4.2
OpenBenchmarking.orgGFLOPS/s Per Watt, More Is Betterlibxsmm 2-1.17-3645M N K: 32EPYC 9554 2PEPYC 9654 2P1.0762.1523.2284.3045.384.7823.609
MinAvgMaxEPYC 9554 2P46.9293.2543.9EPYC 9654 2P94.5359.7590.3OpenBenchmarking.orgWatts, Fewer Is Betterlibxsmm 2-1.17-3645CPU Power Consumption Monitor1603204806408001. EPYC 9554 2P: Approximate power consumption of 2566 Joules per run.2. EPYC 9654 2P: Approximate power consumption of 3309 Joules per run.

OpenBenchmarking.orgGFLOPS/s, More Is Betterlibxsmm 2-1.17-3645M N K: 64EPYC 9554 2PEPYC 9654 2P6001200180024003000SE +/- 5.63, N = 7SE +/- 33.30, N = 152692.72270.21. (CXX) g++ options: -dynamic -Bstatic -static-libgcc -lgomp -lm -lrt -ldl -lquadmath -lstdc++ -pthread -fPIC -std=c++14 -O2 -fopenmp-simd -funroll-loops -ftree-vectorize -fdata-sections -ffunction-sections -fvisibility=hidden -msse4.2
OpenBenchmarking.orgGFLOPS/s Per Watt, More Is Betterlibxsmm 2-1.17-3645M N K: 64EPYC 9554 2PEPYC 9654 2P2468107.8395.975
MinAvgMaxEPYC 9554 2P47344635EPYC 9654 2P98380613OpenBenchmarking.orgWatts, Fewer Is Betterlibxsmm 2-1.17-3645CPU Power Consumption Monitor20040060080010001. EPYC 9554 2P: Approximate power consumption of 3141 Joules per run.2. EPYC 9654 2P: Approximate power consumption of 3622 Joules per run.

TensorFlow

OpenBenchmarking.orgimages/sec Per Watt, More Is BetterTensorFlow 2.12Device: CPU - Batch Size: 16 - Model: VGG-16EPYC 9554 2PEPYC 9654 2PXeon Max 9480 2P, HBM Caching0.03170.06340.09510.12680.15850.1410.1110.038

OpenBenchmarking.orgimages/sec, More Is BetterTensorFlow 2.12Device: CPU - Batch Size: 16 - Model: VGG-16EPYC 9554 2PEPYC 9654 2PXeon Max 9480 2P, HBM Caching1326395265SE +/- 0.59, N = 3SE +/- 0.39, N = 3SE +/- 0.61, N = 1560.3147.2721.09
OpenBenchmarking.orgimages/sec Per Watt, More Is BetterTensorFlow 2.12Device: CPU - Batch Size: 16 - Model: VGG-16EPYC 9554 2PEPYC 9654 2PXeon Max 9480 2P, HBM Caching0.03170.06340.09510.12680.15850.1410.1110.038
MinAvgMaxEPYC 9554 2P47426517EPYC 9654 2P104427498Xeon Max 9480 2P, HBM Caching265554662OpenBenchmarking.orgWatts, Fewer Is BetterTensorFlow 2.12CPU Power Consumption Monitor20040060080010001. EPYC 9554 2P: Approximate power consumption of 15349 Joules per run.2. EPYC 9654 2P: Approximate power consumption of 19060 Joules per run.3. Xeon Max 9480 2P, HBM Caching: Approximate power consumption of 50817 Joules per run.
OpenBenchmarking.orgMegahertz, More Is BetterTensorFlow 2.12CPU Peak Freq (Highest CPU Core Frequency) MonitorXeon Max 9480 2P, HBM Caching6001200180024003000Min: 2600 / Avg: 3476.25 / Max: 3555
OpenBenchmarking.orgWatts, Fewer Is BetterTensorFlow 2.12System Power Consumption MonitorXeon Max 9480 2P, HBM Caching140280420560700Min: 410 / Avg: 755.75 / Max: 8221. Xeon Max 9480 2P, HBM Caching: Approximate power consumption of 69377 Joules per run.

OpenBenchmarking.orgimages/sec, More Is BetterTensorFlow 2.12Device: CPU - Batch Size: 16 - Model: AlexNetEPYC 9554 2PXeon Max 9480 2P, HBM CachingEPYC 9654 2P60120180240300SE +/- 3.19, N = 15SE +/- 1.10, N = 5SE +/- 2.70, N = 15257.94230.11166.20
OpenBenchmarking.orgimages/sec Per Watt, More Is BetterTensorFlow 2.12Device: CPU - Batch Size: 16 - Model: AlexNetEPYC 9554 2PEPYC 9654 2PXeon Max 9480 2P, HBM Caching0.24210.48420.72630.96841.21051.0760.5180.493
MinAvgMaxEPYC 9554 2P47.1239.6360.5EPYC 9654 2P91.1320.9424.6Xeon Max 9480 2P, HBM Caching260.2466.3587.5OpenBenchmarking.orgWatts, Fewer Is BetterTensorFlow 2.12CPU Power Consumption Monitor1603204806408001. EPYC 9554 2P: Approximate power consumption of 3019 Joules per run.2. EPYC 9654 2P: Approximate power consumption of 5264 Joules per run.3. Xeon Max 9480 2P, HBM Caching: Approximate power consumption of 6529 Joules per run.
MinAvgMaxXeon Max 9480 2P, HBM Caching260033063511OpenBenchmarking.orgMegahertz, More Is BetterTensorFlow 2.12CPU Peak Freq (Highest CPU Core Frequency) Monitor10002000300040005000
MinAvgMaxXeon Max 9480 2P, HBM Caching408636785OpenBenchmarking.orgWatts, Fewer Is BetterTensorFlow 2.12System Power Consumption Monitor20040060080010001. Xeon Max 9480 2P, HBM Caching: Approximate power consumption of 8910 Joules per run.

OpenBenchmarking.orgimages/sec Per Watt, More Is BetterTensorFlow 2.12Device: CPU - Batch Size: 512 - Model: VGG-16EPYC 9654 2PEPYC 9554 2PXeon Max 9480 2P, HBM Caching0.04410.08820.13230.17640.22050.1960.1840.056

OpenBenchmarking.orgimages/sec Per Watt, More Is BetterTensorFlow 2.12Device: CPU - Batch Size: 512 - Model: AlexNetEPYC 9554 2PEPYC 9654 2PXeon Max 9480 2P, HBM Caching0.88271.76542.64813.53084.41353.9233.8501.305

OpenBenchmarking.orgimages/sec Per Watt, More Is BetterTensorFlow 2.12Device: CPU - Batch Size: 16 - Model: GoogLeNetEPYC 9554 2PEPYC 9654 2PXeon Max 9480 2P, HBM Caching0.07540.15080.22620.30160.3770.3350.1960.193

OpenBenchmarking.orgimages/sec Per Watt, More Is BetterTensorFlow 2.12Device: CPU - Batch Size: 16 - Model: ResNet-50EPYC 9554 2PEPYC 9654 2PXeon Max 9480 2P, HBM Caching0.02590.05180.07770.10360.12950.1150.0770.074

OpenBenchmarking.orgimages/sec, More Is BetterTensorFlow 2.12Device: CPU - Batch Size: 512 - Model: VGG-16EPYC 9654 2PEPYC 9554 2PXeon Max 9480 2P, HBM Caching306090120150SE +/- 0.08, N = 3SE +/- 1.46, N = 3SE +/- 0.33, N = 5120.62109.8930.19
OpenBenchmarking.orgimages/sec Per Watt, More Is BetterTensorFlow 2.12Device: CPU - Batch Size: 512 - Model: VGG-16EPYC 9654 2PEPYC 9554 2PXeon Max 9480 2P, HBM Caching0.04410.08820.13230.17640.22050.1960.1840.056
OpenBenchmarking.orgWatts, Fewer Is BetterTensorFlow 2.12CPU Power Consumption MonitorXeon Max 9480 2P, HBM CachingEPYC 9554 2PEPYC 9654 2P160320480640800Min: 163.29 / Avg: 539.75 / Max: 899.87Min: 47.29 / Avg: 597.07 / Max: 647.99Min: 95.79 / Avg: 616.66 / Max: 655.381. Xeon Max 9480 2P, HBM Caching: Approximate power consumption of 1002533 Joules per run.2. EPYC 9554 2P: Approximate power consumption of 309285 Joules per run.3. EPYC 9654 2P: Approximate power consumption of 291885 Joules per run.
OpenBenchmarking.orgMegahertz, More Is BetterTensorFlow 2.12CPU Peak Freq (Highest CPU Core Frequency) MonitorXeon Max 9480 2P, HBM Caching6001200180024003000Min: 1671 / Avg: 3334.65 / Max: 3571
OpenBenchmarking.orgWatts, Fewer Is BetterTensorFlow 2.12System Power Consumption MonitorXeon Max 9480 2P, HBM Caching160320480640800Min: 275 / Avg: 740.43 / Max: 9071. Xeon Max 9480 2P, HBM Caching: Approximate power consumption of 1378229 Joules per run.

OpenBenchmarking.orgimages/sec, More Is BetterTensorFlow 2.12Device: CPU - Batch Size: 512 - Model: AlexNetEPYC 9554 2PEPYC 9654 2PXeon Max 9480 2P, HBM Caching400800120016002000SE +/- 12.94, N = 3SE +/- 16.13, N = 7SE +/- 9.04, N = 31813.681757.16718.54
OpenBenchmarking.orgimages/sec Per Watt, More Is BetterTensorFlow 2.12Device: CPU - Batch Size: 512 - Model: AlexNetEPYC 9554 2PEPYC 9654 2PXeon Max 9480 2P, HBM Caching0.88271.76542.64813.53084.41353.9233.8501.305
MinAvgMaxEPYC 9654 2P58.3456.4541.6EPYC 9554 2P48.8462.3567.3Xeon Max 9480 2P, HBM Caching209.1550.7614.5OpenBenchmarking.orgWatts, Fewer Is BetterTensorFlow 2.12CPU Power Consumption Monitor1603204806408001. EPYC 9654 2P: Approximate power consumption of 17409 Joules per run.2. EPYC 9554 2P: Approximate power consumption of 17107 Joules per run.3. Xeon Max 9480 2P, HBM Caching: Approximate power consumption of 46994 Joules per run.
MinAvgMaxXeon Max 9480 2P, HBM Caching260034743512OpenBenchmarking.orgMegahertz, More Is BetterTensorFlow 2.12CPU Peak Freq (Highest CPU Core Frequency) Monitor10002000300040005000
MinAvgMaxXeon Max 9480 2P, HBM Caching307752824OpenBenchmarking.orgWatts, Fewer Is BetterTensorFlow 2.12System Power Consumption Monitor20040060080010001. Xeon Max 9480 2P, HBM Caching: Approximate power consumption of 64146 Joules per run.

OpenBenchmarking.orgimages/sec, More Is BetterTensorFlow 2.12Device: CPU - Batch Size: 16 - Model: GoogLeNetXeon Max 9480 2P, HBM CachingEPYC 9554 2PEPYC 9654 2P20406080100SE +/- 0.98, N = 3SE +/- 0.91, N = 15SE +/- 1.21, N = 1598.4394.4164.49
OpenBenchmarking.orgimages/sec Per Watt, More Is BetterTensorFlow 2.12Device: CPU - Batch Size: 16 - Model: GoogLeNetEPYC 9554 2PEPYC 9654 2PXeon Max 9480 2P, HBM Caching0.07540.15080.22620.30160.3770.3350.1960.193
MinAvgMaxEPYC 9554 2P47.7281.8351.3EPYC 9654 2P57.3328.9406.8Xeon Max 9480 2P, HBM Caching265.6510.5597.2OpenBenchmarking.orgWatts, Fewer Is BetterTensorFlow 2.12CPU Power Consumption Monitor1603204806408001. EPYC 9554 2P: Approximate power consumption of 7213 Joules per run.2. EPYC 9654 2P: Approximate power consumption of 11271 Joules per run.3. Xeon Max 9480 2P, HBM Caching: Approximate power consumption of 12763 Joules per run.
MinAvgMaxXeon Max 9480 2P, HBM Caching266634893505OpenBenchmarking.orgMegahertz, More Is BetterTensorFlow 2.12CPU Peak Freq (Highest CPU Core Frequency) Monitor10002000300040005000
MinAvgMaxXeon Max 9480 2P, HBM Caching405693801OpenBenchmarking.orgWatts, Fewer Is BetterTensorFlow 2.12System Power Consumption Monitor20040060080010001. Xeon Max 9480 2P, HBM Caching: Approximate power consumption of 17564 Joules per run.

OpenBenchmarking.orgimages/sec, More Is BetterTensorFlow 2.12Device: CPU - Batch Size: 16 - Model: ResNet-50Xeon Max 9480 2P, HBM CachingEPYC 9554 2PEPYC 9654 2P918273645SE +/- 0.32, N = 3SE +/- 0.23, N = 3SE +/- 0.31, N = 340.1533.4025.25
OpenBenchmarking.orgimages/sec Per Watt, More Is BetterTensorFlow 2.12Device: CPU - Batch Size: 16 - Model: ResNet-50EPYC 9554 2PEPYC 9654 2PXeon Max 9480 2P, HBM Caching0.02590.05180.07770.10360.12950.1150.0770.074
MinAvgMaxEPYC 9554 2P46.8291.1331.1EPYC 9654 2P57.2327.2393.8Xeon Max 9480 2P, HBM Caching276.7539.1593.1OpenBenchmarking.orgWatts, Fewer Is BetterTensorFlow 2.12CPU Power Consumption Monitor1603204806408001. EPYC 9554 2P: Approximate power consumption of 17662 Joules per run.2. EPYC 9654 2P: Approximate power consumption of 25519 Joules per run.3. Xeon Max 9480 2P, HBM Caching: Approximate power consumption of 28395 Joules per run.
MinAvgMaxXeon Max 9480 2P, HBM Caching350035003510OpenBenchmarking.orgMegahertz, More Is BetterTensorFlow 2.12CPU Peak Freq (Highest CPU Core Frequency) Monitor10002000300040005000
MinAvgMaxXeon Max 9480 2P, HBM Caching415733808OpenBenchmarking.orgWatts, Fewer Is BetterTensorFlow 2.12System Power Consumption Monitor20040060080010001. Xeon Max 9480 2P, HBM Caching: Approximate power consumption of 38625 Joules per run.

OpenBenchmarking.orgimages/sec, More Is BetterTensorFlow 2.12Device: CPU - Batch Size: 512 - Model: GoogLeNetEPYC 9554 2PEPYC 9654 2PXeon Max 9480 2P, HBM Caching130260390520650SE +/- 0.96, N = 3SE +/- 0.69, N = 3SE +/- 1.79, N = 3593.20535.52236.44
OpenBenchmarking.orgimages/sec Per Watt, More Is BetterTensorFlow 2.12Device: CPU - Batch Size: 512 - Model: GoogLeNetEPYC 9554 2PEPYC 9654 2PXeon Max 9480 2P, HBM Caching0.27250.5450.81751.091.36251.2111.0200.421
MinAvgMaxEPYC 9554 2P47490537EPYC 9654 2P56525568Xeon Max 9480 2P, HBM Caching275562692OpenBenchmarking.orgWatts, Fewer Is BetterTensorFlow 2.12CPU Power Consumption Monitor20040060080010001. EPYC 9554 2P: Approximate power consumption of 50120 Joules per run.2. EPYC 9654 2P: Approximate power consumption of 59497 Joules per run.3. Xeon Max 9480 2P, HBM Caching: Approximate power consumption of 138872 Joules per run.
MinAvgMaxXeon Max 9480 2P, HBM Caching195833363515OpenBenchmarking.orgMegahertz, More Is BetterTensorFlow 2.12CPU Peak Freq (Highest CPU Core Frequency) Monitor10002000300040005000
MinAvgMaxXeon Max 9480 2P, HBM Caching425771850OpenBenchmarking.orgWatts, Fewer Is BetterTensorFlow 2.12System Power Consumption Monitor20040060080010001. Xeon Max 9480 2P, HBM Caching: Approximate power consumption of 190783 Joules per run.

OpenBenchmarking.orgimages/sec, More Is BetterTensorFlow 2.12Device: CPU - Batch Size: 512 - Model: ResNet-50EPYC 9554 2PEPYC 9654 2PXeon Max 9480 2P, HBM Caching4080120160200SE +/- 0.70, N = 3SE +/- 0.26, N = 3SE +/- 0.66, N = 3188.17171.5676.93
OpenBenchmarking.orgimages/sec Per Watt, More Is BetterTensorFlow 2.12Device: CPU - Batch Size: 512 - Model: ResNet-50EPYC 9554 2PEPYC 9654 2PXeon Max 9480 2P, HBM Caching0.08710.17420.26130.34840.43550.3870.3410.136
MinAvgMaxEPYC 9554 2P49486557EPYC 9654 2P58503562Xeon Max 9480 2P, HBM Caching231565721OpenBenchmarking.orgWatts, Fewer Is BetterTensorFlow 2.12CPU Power Consumption Monitor20040060080010001. EPYC 9554 2P: Approximate power consumption of 151225 Joules per run.2. EPYC 9654 2P: Approximate power consumption of 171080 Joules per run.3. Xeon Max 9480 2P, HBM Caching: Approximate power consumption of 428896 Joules per run.
OpenBenchmarking.orgMegahertz, More Is BetterTensorFlow 2.12CPU Peak Freq (Highest CPU Core Frequency) MonitorXeon Max 9480 2P, HBM Caching6001200180024003000Min: 1900 / Avg: 3481.66 / Max: 3577
OpenBenchmarking.orgWatts, Fewer Is BetterTensorFlow 2.12System Power Consumption MonitorXeon Max 9480 2P, HBM Caching150300450600750Min: 401 / Avg: 775.8 / Max: 8501. Xeon Max 9480 2P, HBM Caching: Approximate power consumption of 590643 Joules per run.

ONNX Runtime

ONNX Runtime is developed by Microsoft and partners as a open-source, cross-platform, high performance machine learning inferencing and training accelerator. This test profile runs the ONNX Runtime with various models available from the ONNX Model Zoo. Learn more via the OpenBenchmarking.org test page.

OpenBenchmarking.orgInferences Per Second, More Is BetterONNX Runtime 1.14Model: GPT-2 - Device: CPU - Executor: StandardXeon Max 9480 2P, HBM CachingEPYC 9554 2PEPYC 9654 2P50100150200250SE +/- 9.23, N = 15SE +/- 4.25, N = 15SE +/- 0.03, N = 3218.69132.72103.181. (CXX) g++ options: -ffunction-sections -fdata-sections -march=native -mtune=native -O3 -flto=auto -fno-fat-lto-objects -ldl -lrt
OpenBenchmarking.orgInference Time Cost (ms), Fewer Is BetterONNX Runtime 1.14Model: GPT-2 - Device: CPU - Executor: StandardXeon Max 9480 2P, HBM CachingEPYC 9554 2PEPYC 9654 2P3691215SE +/- 0.23765, N = 15SE +/- 0.25707, N = 15SE +/- 0.00311, N = 34.708497.647359.689041. (CXX) g++ options: -ffunction-sections -fdata-sections -march=native -mtune=native -O3 -flto=auto -fno-fat-lto-objects -ldl -lrt
OpenBenchmarking.orgWatts, Fewer Is BetterONNX Runtime 1.14CPU Power Consumption MonitorEPYC 9554 2PEPYC 9654 2PXeon Max 9480 2P, HBM Caching100200300400500Min: 46.8 / Avg: 221.99 / Max: 233.39Min: 58.72 / Avg: 257.53 / Max: 270.63Min: 169.41 / Avg: 521.38 / Max: 555.811. EPYC 9554 2P: Approximate power consumption of 21000 Joules per run.2. EPYC 9654 2P: Approximate power consumption of 24379 Joules per run.3. Xeon Max 9480 2P, HBM Caching: Approximate power consumption of 49462 Joules per run.
OpenBenchmarking.orgMegahertz, More Is BetterONNX Runtime 1.14CPU Peak Freq (Highest CPU Core Frequency) MonitorXeon Max 9480 2P, HBM Caching6001200180024003000Min: 3500 / Avg: 3500.97 / Max: 3517
OpenBenchmarking.orgWatts, Fewer Is BetterONNX Runtime 1.14System Power Consumption MonitorXeon Max 9480 2P, HBM Caching140280420560700Min: 281 / Avg: 720.02 / Max: 7701. Xeon Max 9480 2P, HBM Caching: Approximate power consumption of 68450 Joules per run.

OpenBenchmarking.orgInferences Per Second, More Is BetterONNX Runtime 1.14Model: yolov4 - Device: CPU - Executor: StandardXeon Max 9480 2P, HBM CachingEPYC 9554 2PEPYC 9654 2P246810SE +/- 0.13594, N = 12SE +/- 0.03523, N = 3SE +/- 0.04001, N = 37.700465.105054.900331. (CXX) g++ options: -ffunction-sections -fdata-sections -march=native -mtune=native -O3 -flto=auto -fno-fat-lto-objects -ldl -lrt
OpenBenchmarking.orgInference Time Cost (ms), Fewer Is BetterONNX Runtime 1.14Model: yolov4 - Device: CPU - Executor: StandardXeon Max 9480 2P, HBM CachingEPYC 9554 2PEPYC 9654 2P4080120160200SE +/- 2.64, N = 12SE +/- 1.35, N = 3SE +/- 1.68, N = 3130.37195.90204.091. (CXX) g++ options: -ffunction-sections -fdata-sections -march=native -mtune=native -O3 -flto=auto -fno-fat-lto-objects -ldl -lrt
MinAvgMaxEPYC 9554 2P47360380EPYC 9654 2P57456482Xeon Max 9480 2P, HBM Caching262573626OpenBenchmarking.orgWatts, Fewer Is BetterONNX Runtime 1.14CPU Power Consumption Monitor20040060080010001. EPYC 9554 2P: Approximate power consumption of 34123 Joules per run.2. EPYC 9654 2P: Approximate power consumption of 43132 Joules per run.3. Xeon Max 9480 2P, HBM Caching: Approximate power consumption of 53984 Joules per run.
MinAvgMaxXeon Max 9480 2P, HBM Caching350035003507OpenBenchmarking.orgMegahertz, More Is BetterONNX Runtime 1.14CPU Peak Freq (Highest CPU Core Frequency) Monitor10002000300040005000
MinAvgMaxXeon Max 9480 2P, HBM Caching390783827OpenBenchmarking.orgWatts, Fewer Is BetterONNX Runtime 1.14System Power Consumption Monitor20040060080010001. Xeon Max 9480 2P, HBM Caching: Approximate power consumption of 73942 Joules per run.

OpenBenchmarking.orgInferences Per Second, More Is BetterONNX Runtime 1.14Model: bertsquad-12 - Device: CPU - Executor: StandardEPYC 9554 2PXeon Max 9480 2P, HBM CachingEPYC 9654 2P3691215SE +/- 0.19, N = 15SE +/- 0.23, N = 15SE +/- 0.24, N = 1511.6911.4610.181. (CXX) g++ options: -ffunction-sections -fdata-sections -march=native -mtune=native -O3 -flto=auto -fno-fat-lto-objects -ldl -lrt
OpenBenchmarking.orgInference Time Cost (ms), Fewer Is BetterONNX Runtime 1.14Model: bertsquad-12 - Device: CPU - Executor: StandardEPYC 9554 2PXeon Max 9480 2P, HBM CachingEPYC 9654 2P20406080100SE +/- 1.46, N = 15SE +/- 1.73, N = 15SE +/- 2.35, N = 1585.9087.7699.001. (CXX) g++ options: -ffunction-sections -fdata-sections -march=native -mtune=native -O3 -flto=auto -fno-fat-lto-objects -ldl -lrt
OpenBenchmarking.orgWatts, Fewer Is BetterONNX Runtime 1.14CPU Power Consumption MonitorEPYC 9554 2PEPYC 9654 2PXeon Max 9480 2P, HBM Caching110220330440550Min: 47.62 / Avg: 368.65 / Max: 387.58Min: 57.25 / Avg: 394.67 / Max: 418.06Min: 264.43 / Avg: 572.92 / Max: 616.321. EPYC 9554 2P: Approximate power consumption of 34727 Joules per run.2. EPYC 9654 2P: Approximate power consumption of 37152 Joules per run.3. Xeon Max 9480 2P, HBM Caching: Approximate power consumption of 54160 Joules per run.
OpenBenchmarking.orgMegahertz, More Is BetterONNX Runtime 1.14CPU Peak Freq (Highest CPU Core Frequency) MonitorXeon Max 9480 2P, HBM Caching6001200180024003000Min: 3111 / Avg: 3499.75 / Max: 3508
OpenBenchmarking.orgWatts, Fewer Is BetterONNX Runtime 1.14System Power Consumption MonitorXeon Max 9480 2P, HBM Caching150300450600750Min: 399 / Avg: 783.38 / Max: 8281. Xeon Max 9480 2P, HBM Caching: Approximate power consumption of 74264 Joules per run.

OpenBenchmarking.orgInferences Per Second, More Is BetterONNX Runtime 1.14Model: CaffeNet 12-int8 - Device: CPU - Executor: StandardXeon Max 9480 2P, HBM CachingEPYC 9554 2PEPYC 9654 2P120240360480600SE +/- 8.12, N = 15SE +/- 5.94, N = 4SE +/- 7.41, N = 15562.22495.24435.191. (CXX) g++ options: -ffunction-sections -fdata-sections -march=native -mtune=native -O3 -flto=auto -fno-fat-lto-objects -ldl -lrt
OpenBenchmarking.orgInference Time Cost (ms), Fewer Is BetterONNX Runtime 1.14Model: CaffeNet 12-int8 - Device: CPU - Executor: StandardXeon Max 9480 2P, HBM CachingEPYC 9554 2PEPYC 9654 2P0.51891.03781.55672.07562.5945SE +/- 0.02850, N = 15SE +/- 0.02472, N = 4SE +/- 0.03734, N = 151.782962.019492.306111. (CXX) g++ options: -ffunction-sections -fdata-sections -march=native -mtune=native -O3 -flto=auto -fno-fat-lto-objects -ldl -lrt
MinAvgMaxEPYC 9554 2P47.5357.7374.8EPYC 9654 2P56.7454.2477.7Xeon Max 9480 2P, HBM Caching261.6573.7607.1OpenBenchmarking.orgWatts, Fewer Is BetterONNX Runtime 1.14CPU Power Consumption Monitor1603204806408001. EPYC 9554 2P: Approximate power consumption of 33537 Joules per run.2. EPYC 9654 2P: Approximate power consumption of 42513 Joules per run.3. Xeon Max 9480 2P, HBM Caching: Approximate power consumption of 53817 Joules per run.
OpenBenchmarking.orgMegahertz, More Is BetterONNX Runtime 1.14CPU Peak Freq (Highest CPU Core Frequency) MonitorXeon Max 9480 2P, HBM Caching6001200180024003000Min: 3500 / Avg: 3500.01 / Max: 3507
OpenBenchmarking.orgWatts, Fewer Is BetterONNX Runtime 1.14System Power Consumption MonitorXeon Max 9480 2P, HBM Caching150300450600750Min: 410 / Avg: 784.54 / Max: 8441. Xeon Max 9480 2P, HBM Caching: Approximate power consumption of 73747 Joules per run.

OpenBenchmarking.orgInferences Per Second, More Is BetterONNX Runtime 1.14Model: fcn-resnet101-11 - Device: CPU - Executor: StandardXeon Max 9480 2P, HBM CachingEPYC 9654 2PEPYC 9554 2P1.20042.40083.60124.80166.002SE +/- 0.18914, N = 15SE +/- 0.02211, N = 3SE +/- 0.02211, N = 35.335144.697974.659821. (CXX) g++ options: -ffunction-sections -fdata-sections -march=native -mtune=native -O3 -flto=auto -fno-fat-lto-objects -ldl -lrt
OpenBenchmarking.orgInference Time Cost (ms), Fewer Is BetterONNX Runtime 1.14Model: fcn-resnet101-11 - Device: CPU - Executor: StandardXeon Max 9480 2P, HBM CachingEPYC 9654 2PEPYC 9554 2P50100150200250SE +/- 8.54, N = 15SE +/- 1.01, N = 3SE +/- 1.01, N = 3191.62212.86214.611. (CXX) g++ options: -ffunction-sections -fdata-sections -march=native -mtune=native -O3 -flto=auto -fno-fat-lto-objects -ldl -lrt
MinAvgMaxEPYC 9554 2P47420447EPYC 9654 2P57514546Xeon Max 9480 2P, HBM Caching244569665OpenBenchmarking.orgWatts, Fewer Is BetterONNX Runtime 1.14CPU Power Consumption Monitor20040060080010001. EPYC 9554 2P: Approximate power consumption of 39636 Joules per run.2. EPYC 9654 2P: Approximate power consumption of 48346 Joules per run.3. Xeon Max 9480 2P, HBM Caching: Approximate power consumption of 53650 Joules per run.
OpenBenchmarking.orgMegahertz, More Is BetterONNX Runtime 1.14CPU Peak Freq (Highest CPU Core Frequency) MonitorXeon Max 9480 2P, HBM Caching6001200180024003000Min: 1900 / Avg: 3482.84 / Max: 3511
OpenBenchmarking.orgWatts, Fewer Is BetterONNX Runtime 1.14System Power Consumption MonitorXeon Max 9480 2P, HBM Caching150300450600750Min: 400 / Avg: 778.48 / Max: 8531. Xeon Max 9480 2P, HBM Caching: Approximate power consumption of 73540 Joules per run.

OpenBenchmarking.orgInferences Per Second, More Is BetterONNX Runtime 1.14Model: ArcFace ResNet-100 - Device: CPU - Executor: StandardEPYC 9554 2PXeon Max 9480 2P, HBM CachingEPYC 9654 2P612182430SE +/- 0.09, N = 3SE +/- 0.34, N = 15SE +/- 0.09, N = 324.0123.7719.311. (CXX) g++ options: -ffunction-sections -fdata-sections -march=native -mtune=native -O3 -flto=auto -fno-fat-lto-objects -ldl -lrt
OpenBenchmarking.orgInference Time Cost (ms), Fewer Is BetterONNX Runtime 1.14Model: ArcFace ResNet-100 - Device: CPU - Executor: StandardEPYC 9554 2PXeon Max 9480 2P, HBM CachingEPYC 9654 2P1224364860SE +/- 0.16, N = 3SE +/- 0.60, N = 15SE +/- 0.23, N = 341.6442.1851.791. (CXX) g++ options: -ffunction-sections -fdata-sections -march=native -mtune=native -O3 -flto=auto -fno-fat-lto-objects -ldl -lrt
MinAvgMaxEPYC 9554 2P47.3373.5394.1EPYC 9654 2P58.4464.9493.7Xeon Max 9480 2P, HBM Caching261.0569.9613.0OpenBenchmarking.orgWatts, Fewer Is BetterONNX Runtime 1.14CPU Power Consumption Monitor1603204806408001. EPYC 9554 2P: Approximate power consumption of 35109 Joules per run.2. EPYC 9654 2P: Approximate power consumption of 43705 Joules per run.3. Xeon Max 9480 2P, HBM Caching: Approximate power consumption of 53765 Joules per run.
OpenBenchmarking.orgMegahertz, More Is BetterONNX Runtime 1.14CPU Peak Freq (Highest CPU Core Frequency) MonitorXeon Max 9480 2P, HBM Caching6001200180024003000Min: 3500 / Avg: 3500.03 / Max: 3508
OpenBenchmarking.orgWatts, Fewer Is BetterONNX Runtime 1.14System Power Consumption MonitorXeon Max 9480 2P, HBM Caching140280420560700Min: 405 / Avg: 783.58 / Max: 8211. Xeon Max 9480 2P, HBM Caching: Approximate power consumption of 74126 Joules per run.

OpenBenchmarking.orgInferences Per Second, More Is BetterONNX Runtime 1.14Model: ResNet50 v1-12-int8 - Device: CPU - Executor: StandardEPYC 9554 2PEPYC 9654 2PXeon Max 9480 2P, HBM Caching306090120150SE +/- 1.11, N = 3SE +/- 3.31, N = 15SE +/- 2.29, N = 15152.44146.56125.281. (CXX) g++ options: -ffunction-sections -fdata-sections -march=native -mtune=native -O3 -flto=auto -fno-fat-lto-objects -ldl -lrt
OpenBenchmarking.orgInference Time Cost (ms), Fewer Is BetterONNX Runtime 1.14Model: ResNet50 v1-12-int8 - Device: CPU - Executor: StandardEPYC 9554 2PEPYC 9654 2PXeon Max 9480 2P, HBM Caching246810SE +/- 0.04787, N = 3SE +/- 0.15229, N = 15SE +/- 0.15815, N = 156.559896.870538.021661. (CXX) g++ options: -ffunction-sections -fdata-sections -march=native -mtune=native -O3 -flto=auto -fno-fat-lto-objects -ldl -lrt
MinAvgMaxEPYC 9554 2P47.6368.6386.5EPYC 9654 2P31.9465.2489.6Xeon Max 9480 2P, HBM Caching270.5569.3613.3OpenBenchmarking.orgWatts, Fewer Is BetterONNX Runtime 1.14CPU Power Consumption Monitor1603204806408001. EPYC 9554 2P: Approximate power consumption of 34769 Joules per run.2. EPYC 9654 2P: Approximate power consumption of 43547 Joules per run.3. Xeon Max 9480 2P, HBM Caching: Approximate power consumption of 53284 Joules per run.
OpenBenchmarking.orgMegahertz, More Is BetterONNX Runtime 1.14CPU Peak Freq (Highest CPU Core Frequency) MonitorXeon Max 9480 2P, HBM Caching6001200180024003000Min: 2006 / Avg: 3497.11 / Max: 3509
OpenBenchmarking.orgWatts, Fewer Is BetterONNX Runtime 1.14System Power Consumption MonitorXeon Max 9480 2P, HBM Caching150300450600750Min: 410 / Avg: 784.18 / Max: 8301. Xeon Max 9480 2P, HBM Caching: Approximate power consumption of 73608 Joules per run.

oneDNN

MinAvgMaxXeon Max 9480 2P, HBM Caching260027973507OpenBenchmarking.orgMegahertz, More Is BetteroneDNN 3.1CPU Peak Freq (Highest CPU Core Frequency) Monitor10002000300040005000

MinAvgMaxXeon Max 9480 2P, HBM Caching260030873508OpenBenchmarking.orgMegahertz, More Is BetteroneDNN 3.1CPU Peak Freq (Highest CPU Core Frequency) Monitor10002000300040005000

MinAvgMaxXeon Max 9480 2P, HBM Caching209629513506OpenBenchmarking.orgMegahertz, More Is BetteroneDNN 3.1CPU Peak Freq (Highest CPU Core Frequency) Monitor10002000300040005000

MinAvgMaxXeon Max 9480 2P, HBM Caching216626513508OpenBenchmarking.orgMegahertz, More Is BetteroneDNN 3.1CPU Peak Freq (Highest CPU Core Frequency) Monitor10002000300040005000

MinAvgMaxXeon Max 9480 2P, HBM Caching190031073507OpenBenchmarking.orgMegahertz, More Is BetteroneDNN 3.1CPU Peak Freq (Highest CPU Core Frequency) Monitor10002000300040005000

OpenBenchmarking.orgMegahertz, More Is BetteroneDNN 3.1CPU Peak Freq (Highest CPU Core Frequency) MonitorXeon Max 9480 2P, HBM Caching6001200180024003000Min: 1951 / Avg: 2982.06 / Max: 3514

MinAvgMaxXeon Max 9480 2P, HBM Caching230027023507OpenBenchmarking.orgMegahertz, More Is BetteroneDNN 3.1CPU Peak Freq (Highest CPU Core Frequency) Monitor10002000300040005000

OpenVINO

MinAvgMaxXeon Max 9480 2P, HBM Caching120017403505OpenBenchmarking.orgMegahertz, More Is BetterOpenVINO 2022.3CPU Peak Freq (Highest CPU Core Frequency) Monitor10002000300040005000

MinAvgMaxXeon Max 9480 2P, HBM Caching190030233512OpenBenchmarking.orgMegahertz, More Is BetterOpenVINO 2022.3CPU Peak Freq (Highest CPU Core Frequency) Monitor10002000300040005000

MinAvgMaxXeon Max 9480 2P, HBM Caching190030743506OpenBenchmarking.orgMegahertz, More Is BetterOpenVINO 2022.3CPU Peak Freq (Highest CPU Core Frequency) Monitor10002000300040005000

MinAvgMaxXeon Max 9480 2P, HBM Caching130017383501OpenBenchmarking.orgMegahertz, More Is BetterOpenVINO 2022.3CPU Peak Freq (Highest CPU Core Frequency) Monitor10002000300040005000

MinAvgMaxXeon Max 9480 2P, HBM Caching170023703505OpenBenchmarking.orgMegahertz, More Is BetterOpenVINO 2022.3CPU Peak Freq (Highest CPU Core Frequency) Monitor10002000300040005000

MinAvgMaxXeon Max 9480 2P, HBM Caching130016853500OpenBenchmarking.orgMegahertz, More Is BetterOpenVINO 2022.3CPU Peak Freq (Highest CPU Core Frequency) Monitor10002000300040005000

MinAvgMaxXeon Max 9480 2P, HBM Caching170019003500OpenBenchmarking.orgMegahertz, More Is BetterOpenVINO 2022.3CPU Peak Freq (Highest CPU Core Frequency) Monitor10002000300040005000

MinAvgMaxXeon Max 9480 2P, HBM Caching170020893501OpenBenchmarking.orgMegahertz, More Is BetterOpenVINO 2022.3CPU Peak Freq (Highest CPU Core Frequency) Monitor10002000300040005000

MinAvgMaxXeon Max 9480 2P, HBM Caching150019023504OpenBenchmarking.orgMegahertz, More Is BetterOpenVINO 2022.3CPU Peak Freq (Highest CPU Core Frequency) Monitor10002000300040005000

MinAvgMaxXeon Max 9480 2P, HBM Caching160021253502OpenBenchmarking.orgMegahertz, More Is BetterOpenVINO 2022.3CPU Peak Freq (Highest CPU Core Frequency) Monitor10002000300040005000

MinAvgMaxXeon Max 9480 2P, HBM Caching190034263501OpenBenchmarking.orgMegahertz, More Is BetterOpenVINO 2022.3CPU Peak Freq (Highest CPU Core Frequency) Monitor10002000300040005000

MinAvgMaxXeon Max 9480 2P, HBM Caching110020993500OpenBenchmarking.orgMegahertz, More Is BetterOpenVINO 2022.3CPU Peak Freq (Highest CPU Core Frequency) Monitor10002000300040005000

oneDNN

This is a test of the Intel oneDNN as an Intel-optimized library for Deep Neural Networks and making use of its built-in benchdnn functionality. The result is the total perf time reported. Intel oneDNN was formerly known as DNNL (Deep Neural Network Library) and MKL-DNN before being rebranded as part of the Intel oneAPI toolkit. Learn more via the OpenBenchmarking.org test page.

OpenBenchmarking.orgms, Fewer Is BetteroneDNN 3.1Harness: IP Shapes 1D - Data Type: bf16bf16bf16 - Engine: CPUXeon Max 9480 2P, HBM CachingEPYC 9554 2PEPYC 9654 2P48121620SE +/- 0.22497, N = 15SE +/- 0.95058, N = 12SE +/- 0.81790, N = 157.8801611.1966916.98230MIN: 4.45MIN: 5.64MIN: 8.091. (CXX) g++ options: -O3 -march=native -fopenmp -msse4.1 -fPIC -pie -ldl -lpthread
MinAvgMaxEPYC 9554 2P47267375EPYC 9654 2P90348491Xeon Max 9480 2P, HBM Caching260530626OpenBenchmarking.orgWatts, Fewer Is BetteroneDNN 3.1CPU Power Consumption Monitor20040060080010001. EPYC 9554 2P: Approximate power consumption of 5123 Joules per run.2. EPYC 9654 2P: Approximate power consumption of 6653 Joules per run.3. Xeon Max 9480 2P, HBM Caching: Approximate power consumption of 10134 Joules per run.
MinAvgMaxXeon Max 9480 2P, HBM Caching260027973507OpenBenchmarking.orgMegahertz, More Is BetteroneDNN 3.1CPU Peak Freq (Highest CPU Core Frequency) Monitor10002000300040005000
MinAvgMaxXeon Max 9480 2P, HBM Caching385721838OpenBenchmarking.orgWatts, Fewer Is BetteroneDNN 3.1System Power Consumption Monitor20040060080010001. Xeon Max 9480 2P, HBM Caching: Approximate power consumption of 13796 Joules per run.

OpenBenchmarking.orgms, Fewer Is BetteroneDNN 3.1Harness: IP Shapes 3D - Data Type: bf16bf16bf16 - Engine: CPUEPYC 9554 2PEPYC 9654 2PXeon Max 9480 2P, HBM Caching20406080100SE +/- 0.02374, N = 15SE +/- 0.03459, N = 15SE +/- 0.61972, N = 52.205963.8020991.98140MIN: 79.161. (CXX) g++ options: -O3 -march=native -fopenmp -msse4.1 -fPIC -pie -ldl -lpthread
MinAvgMaxEPYC 9554 2P47256402EPYC 9654 2P89333497Xeon Max 9480 2P, HBM Caching260498639OpenBenchmarking.orgWatts, Fewer Is BetteroneDNN 3.1CPU Power Consumption Monitor20040060080010001. EPYC 9554 2P: Approximate power consumption of 3367 Joules per run.2. EPYC 9654 2P: Approximate power consumption of 4392 Joules per run.3. Xeon Max 9480 2P, HBM Caching: Approximate power consumption of 6670 Joules per run.
MinAvgMaxXeon Max 9480 2P, HBM Caching260030873508OpenBenchmarking.orgMegahertz, More Is BetteroneDNN 3.1CPU Peak Freq (Highest CPU Core Frequency) Monitor10002000300040005000
MinAvgMaxXeon Max 9480 2P, HBM Caching400678830OpenBenchmarking.orgWatts, Fewer Is BetteroneDNN 3.1System Power Consumption Monitor20040060080010001. Xeon Max 9480 2P, HBM Caching: Approximate power consumption of 9085 Joules per run.

OpenBenchmarking.orgms, Fewer Is BetteroneDNN 3.1Harness: Convolution Batch Shapes Auto - Data Type: bf16bf16bf16 - Engine: CPUEPYC 9554 2PEPYC 9654 2PXeon Max 9480 2P, HBM Caching0.82661.65322.47983.30644.133SE +/- 0.002531, N = 7SE +/- 0.003345, N = 15SE +/- 0.025287, N = 70.3113780.4610533.673560MIN: 3.031. (CXX) g++ options: -O3 -march=native -fopenmp -msse4.1 -fPIC -pie -ldl -lpthread
MinAvgMaxEPYC 9554 2P47325610EPYC 9654 2P87337534Xeon Max 9480 2P, HBM Caching267471667OpenBenchmarking.orgWatts, Fewer Is BetteroneDNN 3.1CPU Power Consumption Monitor20040060080010001. EPYC 9554 2P: Approximate power consumption of 3295 Joules per run.2. EPYC 9654 2P: Approximate power consumption of 3441 Joules per run.3. Xeon Max 9480 2P, HBM Caching: Approximate power consumption of 4840 Joules per run.
MinAvgMaxXeon Max 9480 2P, HBM Caching209629513506OpenBenchmarking.orgMegahertz, More Is BetteroneDNN 3.1CPU Peak Freq (Highest CPU Core Frequency) Monitor10002000300040005000
MinAvgMaxXeon Max 9480 2P, HBM Caching400642881OpenBenchmarking.orgWatts, Fewer Is BetteroneDNN 3.1System Power Consumption Monitor20040060080010001. Xeon Max 9480 2P, HBM Caching: Approximate power consumption of 6608 Joules per run.

OpenBenchmarking.orgms, Fewer Is BetteroneDNN 3.1Harness: Deconvolution Batch shapes_1d - Data Type: bf16bf16bf16 - Engine: CPUXeon Max 9480 2P, HBM CachingEPYC 9554 2PEPYC 9654 2P0.571.141.712.282.85SE +/- 0.009112, N = 15SE +/- 0.003878, N = 3SE +/- 0.004515, N = 30.4830751.7616202.533260MIN: 1.59MIN: 1.71. (CXX) g++ options: -O3 -march=native -fopenmp -msse4.1 -fPIC -pie -ldl -lpthread
MinAvgMaxEPYC 9554 2P47388564EPYC 9654 2P106424558Xeon Max 9480 2P, HBM Caching138534633OpenBenchmarking.orgWatts, Fewer Is BetteroneDNN 3.1CPU Power Consumption Monitor20040060080010001. EPYC 9554 2P: Approximate power consumption of 9960 Joules per run.2. EPYC 9654 2P: Approximate power consumption of 10745 Joules per run.3. Xeon Max 9480 2P, HBM Caching: Approximate power consumption of 13281 Joules per run.
MinAvgMaxXeon Max 9480 2P, HBM Caching216626513508OpenBenchmarking.orgMegahertz, More Is BetteroneDNN 3.1CPU Peak Freq (Highest CPU Core Frequency) Monitor10002000300040005000
MinAvgMaxXeon Max 9480 2P, HBM Caching404727850OpenBenchmarking.orgWatts, Fewer Is BetteroneDNN 3.1System Power Consumption Monitor20040060080010001. Xeon Max 9480 2P, HBM Caching: Approximate power consumption of 18187 Joules per run.

OpenBenchmarking.orgms, Fewer Is BetteroneDNN 3.1Harness: Deconvolution Batch shapes_3d - Data Type: bf16bf16bf16 - Engine: CPUEPYC 9554 2PEPYC 9654 2PXeon Max 9480 2P, HBM Caching0.79541.59082.38623.18163.977SE +/- 0.002054, N = 9SE +/- 0.004782, N = 9SE +/- 0.029518, N = 150.4572720.7321003.535250MIN: 2.461. (CXX) g++ options: -O3 -march=native -fopenmp -msse4.1 -fPIC -pie -ldl -lpthread
MinAvgMaxEPYC 9554 2P47270572EPYC 9654 2P93286525Xeon Max 9480 2P, HBM Caching268426673OpenBenchmarking.orgWatts, Fewer Is BetteroneDNN 3.1CPU Power Consumption Monitor20040060080010001. EPYC 9554 2P: Approximate power consumption of 1921 Joules per run.2. EPYC 9654 2P: Approximate power consumption of 2037 Joules per run.3. Xeon Max 9480 2P, HBM Caching: Approximate power consumption of 3042 Joules per run.
MinAvgMaxXeon Max 9480 2P, HBM Caching190031073507OpenBenchmarking.orgMegahertz, More Is BetteroneDNN 3.1CPU Peak Freq (Highest CPU Core Frequency) Monitor10002000300040005000
MinAvgMaxXeon Max 9480 2P, HBM Caching404592897OpenBenchmarking.orgWatts, Fewer Is BetteroneDNN 3.1System Power Consumption Monitor20040060080010001. Xeon Max 9480 2P, HBM Caching: Approximate power consumption of 4220 Joules per run.

OpenBenchmarking.orgms, Fewer Is BetteroneDNN 3.1Harness: Recurrent Neural Network Training - Data Type: bf16bf16bf16 - Engine: CPUEPYC 9554 2PEPYC 9654 2PXeon Max 9480 2P, HBM Caching3K6K9K12K15KSE +/- 13.50, N = 5SE +/- 18.00, N = 3SE +/- 507.18, N = 101305.662492.6915294.60MIN: 7750.751. (CXX) g++ options: -O3 -march=native -fopenmp -msse4.1 -fPIC -pie -ldl -lpthread
MinAvgMaxEPYC 9554 2P47365577EPYC 9654 2P104414522Xeon Max 9480 2P, HBM Caching266573647OpenBenchmarking.orgWatts, Fewer Is BetteroneDNN 3.1CPU Power Consumption Monitor20040060080010001. EPYC 9554 2P: Approximate power consumption of 31196 Joules per run.2. EPYC 9654 2P: Approximate power consumption of 37245 Joules per run.3. Xeon Max 9480 2P, HBM Caching: Approximate power consumption of 91132 Joules per run.
OpenBenchmarking.orgMegahertz, More Is BetteroneDNN 3.1CPU Peak Freq (Highest CPU Core Frequency) MonitorXeon Max 9480 2P, HBM Caching6001200180024003000Min: 1951 / Avg: 2982.06 / Max: 3514
OpenBenchmarking.orgWatts, Fewer Is BetteroneDNN 3.1System Power Consumption MonitorXeon Max 9480 2P, HBM Caching160320480640800Min: 410 / Avg: 782.09 / Max: 8891. Xeon Max 9480 2P, HBM Caching: Approximate power consumption of 124744 Joules per run.

OpenBenchmarking.orgms, Fewer Is BetteroneDNN 3.1Harness: Recurrent Neural Network Inference - Data Type: bf16bf16bf16 - Engine: CPUEPYC 9554 2PXeon Max 9480 2P, HBM CachingEPYC 9654 2P6001200180024003000SE +/- 2.71, N = 3SE +/- 20.64, N = 12SE +/- 29.80, N = 15925.771325.452790.90MIN: 909.67MIN: 1025.18MIN: 2391.981. (CXX) g++ options: -O3 -march=native -fopenmp -msse4.1 -fPIC -pie -ldl -lpthread
MinAvgMaxEPYC 9554 2P47356602EPYC 9654 2P96412555Xeon Max 9480 2P, HBM Caching259561644OpenBenchmarking.orgWatts, Fewer Is BetteroneDNN 3.1CPU Power Consumption Monitor20040060080010001. EPYC 9554 2P: Approximate power consumption of 29795 Joules per run.2. EPYC 9654 2P: Approximate power consumption of 37006 Joules per run.3. Xeon Max 9480 2P, HBM Caching: Approximate power consumption of 47136 Joules per run.
MinAvgMaxXeon Max 9480 2P, HBM Caching230027023507OpenBenchmarking.orgMegahertz, More Is BetteroneDNN 3.1CPU Peak Freq (Highest CPU Core Frequency) Monitor10002000300040005000
MinAvgMaxXeon Max 9480 2P, HBM Caching402767901OpenBenchmarking.orgWatts, Fewer Is BetteroneDNN 3.1System Power Consumption Monitor20040060080010001. Xeon Max 9480 2P, HBM Caching: Approximate power consumption of 64567 Joules per run.

OpenVINO

This is a test of the Intel OpenVINO, a toolkit around neural networks, using its built-in benchmarking support and analyzing the throughput and latency for various models. Learn more via the OpenBenchmarking.org test page.

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2022.3Model: Face Detection FP16 - Device: CPUXeon Max 9480 2P, HBM CachingEPYC 9554 2PEPYC 9654 2P100200300400500SE +/- 0.80, N = 3SE +/- 0.19, N = 3SE +/- 0.93, N = 3232.01389.53471.82MIN: 138.61 / MAX: 770.45MIN: 380.82 / MAX: 432.57MIN: 431.51 / MAX: 569.31. (CXX) g++ options: -isystem -fsigned-char -ffunction-sections -fdata-sections -msse4.1 -msse4.2 -O3 -fno-strict-overflow -fwrapv -fPIC -fvisibility=hidden -Os -std=c++11 -MD -MT -MF
MinAvgMaxXeon Max 9480 2P, HBM Caching136533628EPYC 9554 2P49638711EPYC 9654 2P58646726OpenBenchmarking.orgWatts, Fewer Is BetterOpenVINO 2022.3CPU Power Consumption Monitor20040060080010001. Xeon Max 9480 2P, HBM Caching: Approximate power consumption of 39944 Joules per run.2. EPYC 9554 2P: Approximate power consumption of 43188 Joules per run.3. EPYC 9654 2P: Approximate power consumption of 44817 Joules per run.
MinAvgMaxXeon Max 9480 2P, HBM Caching120017403505OpenBenchmarking.orgMegahertz, More Is BetterOpenVINO 2022.3CPU Peak Freq (Highest CPU Core Frequency) Monitor10002000300040005000
MinAvgMaxXeon Max 9480 2P, HBM Caching281724839OpenBenchmarking.orgWatts, Fewer Is BetterOpenVINO 2022.3System Power Consumption Monitor20040060080010001. Xeon Max 9480 2P, HBM Caching: Approximate power consumption of 54275 Joules per run.

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2022.3Model: Person Detection FP16 - Device: CPUEPYC 9554 2PXeon Max 9480 2P, HBM CachingEPYC 9654 2P2004006008001000SE +/- 5.30, N = 3SE +/- 6.17, N = 3SE +/- 5.11, N = 3832.02834.971092.97MIN: 743.95 / MAX: 1258.17MIN: 558.93 / MAX: 2258.39MIN: 825.28 / MAX: 1814.821. (CXX) g++ options: -isystem -fsigned-char -ffunction-sections -fdata-sections -msse4.1 -msse4.2 -O3 -fno-strict-overflow -fwrapv -fPIC -fvisibility=hidden -Os -std=c++11 -MD -MT -MF
MinAvgMaxXeon Max 9480 2P, HBM Caching204538618EPYC 9554 2P49596683EPYC 9654 2P60608699OpenBenchmarking.orgWatts, Fewer Is BetterOpenVINO 2022.3CPU Power Consumption Monitor20040060080010001. Xeon Max 9480 2P, HBM Caching: Approximate power consumption of 38946 Joules per run.2. EPYC 9554 2P: Approximate power consumption of 41100 Joules per run.3. EPYC 9654 2P: Approximate power consumption of 42760 Joules per run.
MinAvgMaxXeon Max 9480 2P, HBM Caching190030233512OpenBenchmarking.orgMegahertz, More Is BetterOpenVINO 2022.3CPU Peak Freq (Highest CPU Core Frequency) Monitor10002000300040005000
MinAvgMaxXeon Max 9480 2P, HBM Caching298736817OpenBenchmarking.orgWatts, Fewer Is BetterOpenVINO 2022.3System Power Consumption Monitor20040060080010001. Xeon Max 9480 2P, HBM Caching: Approximate power consumption of 53264 Joules per run.

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2022.3Model: Person Detection FP32 - Device: CPUXeon Max 9480 2P, HBM CachingEPYC 9554 2PEPYC 9654 2P2004006008001000SE +/- 7.20, N = 3SE +/- 1.07, N = 3SE +/- 5.33, N = 3819.76825.521098.96MIN: 516.88 / MAX: 2598.39MIN: 747.05 / MAX: 1238.39MIN: 825.62 / MAX: 1835.761. (CXX) g++ options: -isystem -fsigned-char -ffunction-sections -fdata-sections -msse4.1 -msse4.2 -O3 -fno-strict-overflow -fwrapv -fPIC -fvisibility=hidden -Os -std=c++11 -MD -MT -MF
MinAvgMaxXeon Max 9480 2P, HBM Caching228547624EPYC 9554 2P49600684EPYC 9654 2P60606703OpenBenchmarking.orgWatts, Fewer Is BetterOpenVINO 2022.3CPU Power Consumption Monitor20040060080010001. Xeon Max 9480 2P, HBM Caching: Approximate power consumption of 38131 Joules per run.2. EPYC 9554 2P: Approximate power consumption of 41612 Joules per run.3. EPYC 9654 2P: Approximate power consumption of 42612 Joules per run.
MinAvgMaxXeon Max 9480 2P, HBM Caching190030743506OpenBenchmarking.orgMegahertz, More Is BetterOpenVINO 2022.3CPU Peak Freq (Highest CPU Core Frequency) Monitor10002000300040005000
MinAvgMaxXeon Max 9480 2P, HBM Caching369748857OpenBenchmarking.orgWatts, Fewer Is BetterOpenVINO 2022.3System Power Consumption Monitor20040060080010001. Xeon Max 9480 2P, HBM Caching: Approximate power consumption of 52353 Joules per run.

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2022.3Model: Vehicle Detection FP16 - Device: CPUEPYC 9554 2PEPYC 9654 2PXeon Max 9480 2P, HBM Caching3691215SE +/- 0.00, N = 3SE +/- 0.00, N = 3SE +/- 0.08, N = 135.346.4912.07MIN: 4.89 / MAX: 34.7MIN: 5.03 / MAX: 55.65MIN: 7.38 / MAX: 250.921. (CXX) g++ options: -isystem -fsigned-char -ffunction-sections -fdata-sections -msse4.1 -msse4.2 -O3 -fno-strict-overflow -fwrapv -fPIC -fvisibility=hidden -Os -std=c++11 -MD -MT -MF
MinAvgMaxXeon Max 9480 2P, HBM Caching87548657EPYC 9554 2P50629675EPYC 9654 2P58636689OpenBenchmarking.orgWatts, Fewer Is BetterOpenVINO 2022.3CPU Power Consumption Monitor20040060080010001. Xeon Max 9480 2P, HBM Caching: Approximate power consumption of 36629 Joules per run.2. EPYC 9554 2P: Approximate power consumption of 40490 Joules per run.3. EPYC 9654 2P: Approximate power consumption of 41344 Joules per run.
MinAvgMaxXeon Max 9480 2P, HBM Caching130017383501OpenBenchmarking.orgMegahertz, More Is BetterOpenVINO 2022.3CPU Peak Freq (Highest CPU Core Frequency) Monitor10002000300040005000
MinAvgMaxXeon Max 9480 2P, HBM Caching272742868OpenBenchmarking.orgWatts, Fewer Is BetterOpenVINO 2022.3System Power Consumption Monitor20040060080010001. Xeon Max 9480 2P, HBM Caching: Approximate power consumption of 49737 Joules per run.

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2022.3Model: Face Detection FP16-INT8 - Device: CPUEPYC 9554 2PEPYC 9654 2PXeon Max 9480 2P, HBM Caching70140210280350SE +/- 0.22, N = 3SE +/- 0.04, N = 3SE +/- 0.44, N = 3205.71250.39333.51MIN: 202.38 / MAX: 252.14MIN: 222.36 / MAX: 303.72MIN: 285.32 / MAX: 516.281. (CXX) g++ options: -isystem -fsigned-char -ffunction-sections -fdata-sections -msse4.1 -msse4.2 -O3 -fno-strict-overflow -fwrapv -fPIC -fvisibility=hidden -Os -std=c++11 -MD -MT -MF
MinAvgMaxXeon Max 9480 2P, HBM Caching168531616EPYC 9554 2P50606675EPYC 9654 2P59613687OpenBenchmarking.orgWatts, Fewer Is BetterOpenVINO 2022.3CPU Power Consumption Monitor20040060080010001. Xeon Max 9480 2P, HBM Caching: Approximate power consumption of 46887 Joules per run.2. EPYC 9554 2P: Approximate power consumption of 41215 Joules per run.3. EPYC 9654 2P: Approximate power consumption of 42688 Joules per run.
MinAvgMaxXeon Max 9480 2P, HBM Caching170023703505OpenBenchmarking.orgMegahertz, More Is BetterOpenVINO 2022.3CPU Peak Freq (Highest CPU Core Frequency) Monitor10002000300040005000
MinAvgMaxXeon Max 9480 2P, HBM Caching270722818OpenBenchmarking.orgWatts, Fewer Is BetterOpenVINO 2022.3System Power Consumption Monitor20040060080010001. Xeon Max 9480 2P, HBM Caching: Approximate power consumption of 64059 Joules per run.

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2022.3Model: Vehicle Detection FP16-INT8 - Device: CPUEPYC 9554 2PEPYC 9654 2PXeon Max 9480 2P, HBM Caching510152025SE +/- 0.00, N = 3SE +/- 0.00, N = 3SE +/- 0.09, N = 33.544.3620.23MIN: 3.45 / MAX: 21.7MIN: 3.52 / MAX: 38.49MIN: 12.69 / MAX: 166.761. (CXX) g++ options: -isystem -fsigned-char -ffunction-sections -fdata-sections -msse4.1 -msse4.2 -O3 -fno-strict-overflow -fwrapv -fPIC -fvisibility=hidden -Os -std=c++11 -MD -MT -MF
MinAvgMaxXeon Max 9480 2P, HBM Caching170552614EPYC 9554 2P49590636EPYC 9654 2P58600646OpenBenchmarking.orgWatts, Fewer Is BetterOpenVINO 2022.3CPU Power Consumption Monitor20040060080010001. Xeon Max 9480 2P, HBM Caching: Approximate power consumption of 37922 Joules per run.2. EPYC 9554 2P: Approximate power consumption of 37983 Joules per run.3. EPYC 9654 2P: Approximate power consumption of 38779 Joules per run.
MinAvgMaxXeon Max 9480 2P, HBM Caching130016853500OpenBenchmarking.orgMegahertz, More Is BetterOpenVINO 2022.3CPU Peak Freq (Highest CPU Core Frequency) Monitor10002000300040005000
MinAvgMaxXeon Max 9480 2P, HBM Caching262755817OpenBenchmarking.orgWatts, Fewer Is BetterOpenVINO 2022.3System Power Consumption Monitor20040060080010001. Xeon Max 9480 2P, HBM Caching: Approximate power consumption of 51849 Joules per run.

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2022.3Model: Weld Porosity Detection FP16 - Device: CPUEPYC 9554 2PEPYC 9654 2PXeon Max 9480 2P, HBM Caching246810SE +/- 0.00, N = 3SE +/- 0.00, N = 3SE +/- 0.01, N = 33.974.866.44MIN: 3.87 / MAX: 17.35MIN: 4.02 / MAX: 36.01MIN: 4.24 / MAX: 49.431. (CXX) g++ options: -isystem -fsigned-char -ffunction-sections -fdata-sections -msse4.1 -msse4.2 -O3 -fno-strict-overflow -fwrapv -fPIC -fvisibility=hidden -Os -std=c++11 -MD -MT -MF
MinAvgMaxXeon Max 9480 2P, HBM Caching163557629EPYC 9554 2P48648698EPYC 9654 2P57655705OpenBenchmarking.orgWatts, Fewer Is BetterOpenVINO 2022.3CPU Power Consumption Monitor20040060080010001. Xeon Max 9480 2P, HBM Caching: Approximate power consumption of 37337 Joules per run.2. EPYC 9554 2P: Approximate power consumption of 41467 Joules per run.3. EPYC 9654 2P: Approximate power consumption of 42138 Joules per run.
MinAvgMaxXeon Max 9480 2P, HBM Caching170019003500OpenBenchmarking.orgMegahertz, More Is BetterOpenVINO 2022.3CPU Peak Freq (Highest CPU Core Frequency) Monitor10002000300040005000
MinAvgMaxXeon Max 9480 2P, HBM Caching279756840OpenBenchmarking.orgWatts, Fewer Is BetterOpenVINO 2022.3System Power Consumption Monitor20040060080010001. Xeon Max 9480 2P, HBM Caching: Approximate power consumption of 50630 Joules per run.

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2022.3Model: Machine Translation EN To DE FP16 - Device: CPUEPYC 9554 2PXeon Max 9480 2P, HBM CachingEPYC 9654 2P1122334455SE +/- 0.08, N = 3SE +/- 0.01, N = 3SE +/- 0.04, N = 340.6747.2950.42MIN: 34.96 / MAX: 124.31MIN: 32.14 / MAX: 577.78MIN: 39.07 / MAX: 280.941. (CXX) g++ options: -isystem -fsigned-char -ffunction-sections -fdata-sections -msse4.1 -msse4.2 -O3 -fno-strict-overflow -fwrapv -fPIC -fvisibility=hidden -Os -std=c++11 -MD -MT -MF
MinAvgMaxXeon Max 9480 2P, HBM Caching165554623EPYC 9554 2P51622683EPYC 9654 2P59642706OpenBenchmarking.orgWatts, Fewer Is BetterOpenVINO 2022.3CPU Power Consumption Monitor20040060080010001. Xeon Max 9480 2P, HBM Caching: Approximate power consumption of 38229 Joules per run.2. EPYC 9554 2P: Approximate power consumption of 40668 Joules per run.3. EPYC 9654 2P: Approximate power consumption of 42185 Joules per run.
MinAvgMaxXeon Max 9480 2P, HBM Caching170020893501OpenBenchmarking.orgMegahertz, More Is BetterOpenVINO 2022.3CPU Peak Freq (Highest CPU Core Frequency) Monitor10002000300040005000
MinAvgMaxXeon Max 9480 2P, HBM Caching274752852OpenBenchmarking.orgWatts, Fewer Is BetterOpenVINO 2022.3System Power Consumption Monitor20040060080010001. Xeon Max 9480 2P, HBM Caching: Approximate power consumption of 51910 Joules per run.

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2022.3Model: Weld Porosity Detection FP16-INT8 - Device: CPUXeon Max 9480 2P, HBM CachingEPYC 9554 2PEPYC 9654 2P3691215SE +/- 0.00, N = 3SE +/- 0.00, N = 3SE +/- 0.00, N = 33.478.099.95MIN: 2.48 / MAX: 56.57MIN: 7.89 / MAX: 45.39MIN: 8.36 / MAX: 40.391. (CXX) g++ options: -isystem -fsigned-char -ffunction-sections -fdata-sections -msse4.1 -msse4.2 -O3 -fno-strict-overflow -fwrapv -fPIC -fvisibility=hidden -Os -std=c++11 -MD -MT -MF
MinAvgMaxXeon Max 9480 2P, HBM Caching137552614EPYC 9554 2P49623675EPYC 9654 2P59632685OpenBenchmarking.orgWatts, Fewer Is BetterOpenVINO 2022.3CPU Power Consumption Monitor20040060080010001. Xeon Max 9480 2P, HBM Caching: Approximate power consumption of 37176 Joules per run.2. EPYC 9554 2P: Approximate power consumption of 40518 Joules per run.3. EPYC 9654 2P: Approximate power consumption of 41474 Joules per run.
MinAvgMaxXeon Max 9480 2P, HBM Caching150019023504OpenBenchmarking.orgMegahertz, More Is BetterOpenVINO 2022.3CPU Peak Freq (Highest CPU Core Frequency) Monitor10002000300040005000
MinAvgMaxXeon Max 9480 2P, HBM Caching275752830OpenBenchmarking.orgWatts, Fewer Is BetterOpenVINO 2022.3System Power Consumption Monitor20040060080010001. Xeon Max 9480 2P, HBM Caching: Approximate power consumption of 51165 Joules per run.

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2022.3Model: Person Vehicle Bike Detection FP16 - Device: CPUEPYC 9554 2PEPYC 9654 2PXeon Max 9480 2P, HBM Caching48121620SE +/- 0.01, N = 3SE +/- 0.00, N = 3SE +/- 0.02, N = 34.295.2517.83MIN: 4.11 / MAX: 21.83MIN: 4.36 / MAX: 23.21MIN: 13.06 / MAX: 147.361. (CXX) g++ options: -isystem -fsigned-char -ffunction-sections -fdata-sections -msse4.1 -msse4.2 -O3 -fno-strict-overflow -fwrapv -fPIC -fvisibility=hidden -Os -std=c++11 -MD -MT -MF
MinAvgMaxXeon Max 9480 2P, HBM Caching176531642EPYC 9554 2P49586647EPYC 9654 2P59599664OpenBenchmarking.orgWatts, Fewer Is BetterOpenVINO 2022.3CPU Power Consumption Monitor20040060080010001. Xeon Max 9480 2P, HBM Caching: Approximate power consumption of 41580 Joules per run.2. EPYC 9554 2P: Approximate power consumption of 38464 Joules per run.3. EPYC 9654 2P: Approximate power consumption of 39943 Joules per run.
MinAvgMaxXeon Max 9480 2P, HBM Caching160021253502OpenBenchmarking.orgMegahertz, More Is BetterOpenVINO 2022.3CPU Peak Freq (Highest CPU Core Frequency) Monitor10002000300040005000
MinAvgMaxXeon Max 9480 2P, HBM Caching284724867OpenBenchmarking.orgWatts, Fewer Is BetterOpenVINO 2022.3System Power Consumption Monitor20040060080010001. Xeon Max 9480 2P, HBM Caching: Approximate power consumption of 56743 Joules per run.

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2022.3Model: Age Gender Recognition Retail 0013 FP16 - Device: CPUXeon Max 9480 2P, HBM CachingEPYC 9554 2PEPYC 9654 2P0.12380.24760.37140.49520.619SE +/- 0.00, N = 3SE +/- 0.00, N = 3SE +/- 0.00, N = 30.440.520.55MIN: 0.33 / MAX: 42.85MIN: 0.49 / MAX: 50.12MIN: 0.5 / MAX: 45.731. (CXX) g++ options: -isystem -fsigned-char -ffunction-sections -fdata-sections -msse4.1 -msse4.2 -O3 -fno-strict-overflow -fwrapv -fPIC -fvisibility=hidden -Os -std=c++11 -MD -MT -MF
MinAvgMaxXeon Max 9480 2P, HBM Caching265570619EPYC 9554 2P49580637EPYC 9654 2P62581643OpenBenchmarking.orgWatts, Fewer Is BetterOpenVINO 2022.3CPU Power Consumption Monitor20040060080010001. Xeon Max 9480 2P, HBM Caching: Approximate power consumption of 37226 Joules per run.2. EPYC 9554 2P: Approximate power consumption of 37918 Joules per run.3. EPYC 9654 2P: Approximate power consumption of 38142 Joules per run.
MinAvgMaxXeon Max 9480 2P, HBM Caching190034263501OpenBenchmarking.orgMegahertz, More Is BetterOpenVINO 2022.3CPU Peak Freq (Highest CPU Core Frequency) Monitor10002000300040005000
MinAvgMaxXeon Max 9480 2P, HBM Caching392772837OpenBenchmarking.orgWatts, Fewer Is BetterOpenVINO 2022.3System Power Consumption Monitor20040060080010001. Xeon Max 9480 2P, HBM Caching: Approximate power consumption of 50450 Joules per run.

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2022.3Model: Age Gender Recognition Retail 0013 FP16-INT8 - Device: CPUEPYC 9554 2PEPYC 9654 2PXeon Max 9480 2P, HBM Caching0.3150.630.9451.261.575SE +/- 0.00, N = 3SE +/- 0.00, N = 3SE +/- 0.04, N = 120.880.991.40MIN: 0.83 / MAX: 24.85MIN: 0.85 / MAX: 37.79MIN: 0.7 / MAX: 123.261. (CXX) g++ options: -isystem -fsigned-char -ffunction-sections -fdata-sections -msse4.1 -msse4.2 -O3 -fno-strict-overflow -fwrapv -fPIC -fvisibility=hidden -Os -std=c++11 -MD -MT -MF
MinAvgMaxXeon Max 9480 2P, HBM Caching164545651EPYC 9654 2P59595651EPYC 9554 2P48610665OpenBenchmarking.orgWatts, Fewer Is BetterOpenVINO 2022.3CPU Power Consumption Monitor20040060080010001. Xeon Max 9480 2P, HBM Caching: Approximate power consumption of 36125 Joules per run.2. EPYC 9654 2P: Approximate power consumption of 38859 Joules per run.3. EPYC 9554 2P: Approximate power consumption of 39444 Joules per run.
MinAvgMaxXeon Max 9480 2P, HBM Caching110020993500OpenBenchmarking.orgMegahertz, More Is BetterOpenVINO 2022.3CPU Peak Freq (Highest CPU Core Frequency) Monitor10002000300040005000
MinAvgMaxXeon Max 9480 2P, HBM Caching268742856OpenBenchmarking.orgWatts, Fewer Is BetterOpenVINO 2022.3System Power Consumption Monitor20040060080010001. Xeon Max 9480 2P, HBM Caching: Approximate power consumption of 49351 Joules per run.

80 Results Shown

CPU Peak Freq (Highest CPU Core Frequency) Monitor:
  Phoronix Test Suite System Monitoring:
    Megahertz
    Watts
    Watts
OpenVINO:
  Face Detection FP16 - CPU
  Person Detection FP16 - CPU
  Person Detection FP32 - CPU
  Vehicle Detection FP16 - CPU
  Face Detection FP16-INT8 - CPU
  Vehicle Detection FP16-INT8 - CPU
  Weld Porosity Detection FP16 - CPU
  Machine Translation EN To DE FP16 - CPU
  Weld Porosity Detection FP16-INT8 - CPU
  Person Vehicle Bike Detection FP16 - CPU
  Age Gender Recognition Retail 0013 FP16 - CPU
  Age Gender Recognition Retail 0013 FP16-INT8 - CPU
libxsmm:
  256
  32
  64
libxsmm:
  128
  256
  32
  64
TensorFlow
TensorFlow:
  CPU - 16 - VGG-16
  CPU - 16 - AlexNet
TensorFlow:
  CPU - 512 - VGG-16
  CPU - 512 - AlexNet
  CPU - 16 - GoogLeNet
  CPU - 16 - ResNet-50
TensorFlow:
  CPU - 512 - VGG-16
  CPU - 512 - AlexNet
  CPU - 16 - GoogLeNet
  CPU - 16 - ResNet-50
  CPU - 512 - GoogLeNet
  CPU - 512 - ResNet-50
ONNX Runtime:
  GPT-2 - CPU - Standard
  yolov4 - CPU - Standard
  bertsquad-12 - CPU - Standard
  CaffeNet 12-int8 - CPU - Standard
  fcn-resnet101-11 - CPU - Standard
  ArcFace ResNet-100 - CPU - Standard
  ResNet50 v1-12-int8 - CPU - Standard
oneDNN:
  CPU Peak Freq (Highest CPU Core Frequency) Monitor:
    Megahertz
    Megahertz
    Megahertz
    Megahertz
    Megahertz
    Megahertz
    Megahertz
    Megahertz
    Megahertz
    Megahertz
    Megahertz
    Megahertz
    Megahertz
    Megahertz
    Megahertz
    Megahertz
    Megahertz
    Megahertz
    Megahertz
oneDNN:
  IP Shapes 1D - bf16bf16bf16 - CPU
  IP Shapes 3D - bf16bf16bf16 - CPU
  Convolution Batch Shapes Auto - bf16bf16bf16 - CPU
  Deconvolution Batch shapes_1d - bf16bf16bf16 - CPU
  Deconvolution Batch shapes_3d - bf16bf16bf16 - CPU
  Recurrent Neural Network Training - bf16bf16bf16 - CPU
  Recurrent Neural Network Inference - bf16bf16bf16 - CPU
OpenVINO:
  Face Detection FP16 - CPU
  Person Detection FP16 - CPU
  Person Detection FP32 - CPU
  Vehicle Detection FP16 - CPU
  Face Detection FP16-INT8 - CPU
  Vehicle Detection FP16-INT8 - CPU
  Weld Porosity Detection FP16 - CPU
  Machine Translation EN To DE FP16 - CPU
  Weld Porosity Detection FP16-INT8 - CPU
  Person Vehicle Bike Detection FP16 - CPU
  Age Gender Recognition Retail 0013 FP16 - CPU
  Age Gender Recognition Retail 0013 FP16-INT8 - CPU

EPYC 9654 2P

Processor: 2 x AMD EPYC 9654 96-Core @ 2.40GHz (192 Cores / 384 Threads), Motherboard: AMD Titanite_4G (RTI1007B BIOS), Chipset: AMD Device 14a4, Memory: 1520GB, Disk: 7682GB INTEL SSDPF2KX076TZ, Graphics: ASPEED, Monitor: VGA HDMI, Network: Broadcom NetXtreme BCM5720 PCIe

OS: Ubuntu 23.04, Kernel: 6.2.0-20-generic (x86_64), Desktop: GNOME Shell 44.0, Display Server: X Server 1.21.1.7, Compiler: GCC 12.2.0, File-System: ext4, Screen Resolution: 1920x1080

Kernel Notes: Transparent Huge Pages: madvise
Compiler Notes: --build=x86_64-linux-gnu --disable-vtable-verify --disable-werror --enable-cet --enable-checking=release --enable-clocale=gnu --enable-default-pie --enable-gnu-unique-object --enable-languages=c,ada,c++,go,d,fortran,objc,obj-c++,m2 --enable-libphobos-checking=release --enable-libstdcxx-debug --enable-libstdcxx-time=yes --enable-multiarch --enable-multilib --enable-nls --enable-objc-gc=auto --enable-offload-defaulted --enable-offload-targets=nvptx-none=/build/gcc-12-Pa930Z/gcc-12-12.2.0/debian/tmp-nvptx/usr,amdgcn-amdhsa=/build/gcc-12-Pa930Z/gcc-12-12.2.0/debian/tmp-gcn/usr --enable-plugin --enable-shared --enable-threads=posix --host=x86_64-linux-gnu --program-prefix=x86_64-linux-gnu- --target=x86_64-linux-gnu --with-abi=m64 --with-arch-32=i686 --with-default-libstdcxx-abi=new --with-gcc-major-version-only --with-multilib-list=m32,m64,mx32 --with-target-system-zlib=auto --with-tune=generic --without-cuda-driver -v
Processor Notes: Scaling Governor: acpi-cpufreq performance (Boost: Enabled) - CPU Microcode: 0xa101121
Python Notes: Python 3.11.2
Security Notes: itlb_multihit: Not affected + l1tf: Not affected + mds: Not affected + meltdown: Not affected + mmio_stale_data: Not affected + retbleed: Not affected + spec_store_bypass: Mitigation of SSB disabled via prctl + spectre_v1: Mitigation of usercopy/swapgs barriers and __user pointer sanitization + spectre_v2: Mitigation of Retpolines IBPB: conditional IBRS_FW STIBP: always-on RSB filling PBRSB-eIBRS: Not affected + srbds: Not affected + tsx_async_abort: Not affected

Testing initiated at 28 June 2023 15:03 by user phoronix.

EPYC 9554 2P

Processor: 2 x AMD EPYC 9554 64-Core @ 3.10GHz (128 Cores / 256 Threads), Motherboard: AMD Titanite_4G (RTI1007B BIOS), Chipset: AMD Device 14a4, Memory: 1520GB, Disk: 7682GB INTEL SSDPF2KX076TZ, Graphics: ASPEED, Monitor: VGA HDMI, Network: Broadcom NetXtreme BCM5720 PCIe

OS: Ubuntu 23.04, Kernel: 6.2.0-20-generic (x86_64), Desktop: GNOME Shell 44.0, Display Server: X Server 1.21.1.7, Compiler: GCC 12.2.0, File-System: ext4, Screen Resolution: 1920x1080

Kernel Notes: Transparent Huge Pages: madvise
Compiler Notes: --build=x86_64-linux-gnu --disable-vtable-verify --disable-werror --enable-cet --enable-checking=release --enable-clocale=gnu --enable-default-pie --enable-gnu-unique-object --enable-languages=c,ada,c++,go,d,fortran,objc,obj-c++,m2 --enable-libphobos-checking=release --enable-libstdcxx-debug --enable-libstdcxx-time=yes --enable-multiarch --enable-multilib --enable-nls --enable-objc-gc=auto --enable-offload-defaulted --enable-offload-targets=nvptx-none=/build/gcc-12-Pa930Z/gcc-12-12.2.0/debian/tmp-nvptx/usr,amdgcn-amdhsa=/build/gcc-12-Pa930Z/gcc-12-12.2.0/debian/tmp-gcn/usr --enable-plugin --enable-shared --enable-threads=posix --host=x86_64-linux-gnu --program-prefix=x86_64-linux-gnu- --target=x86_64-linux-gnu --with-abi=m64 --with-arch-32=i686 --with-default-libstdcxx-abi=new --with-gcc-major-version-only --with-multilib-list=m32,m64,mx32 --with-target-system-zlib=auto --with-tune=generic --without-cuda-driver -v
Processor Notes: Scaling Governor: acpi-cpufreq performance (Boost: Enabled) - CPU Microcode: 0xa101121
Python Notes: Python 3.11.2
Security Notes: itlb_multihit: Not affected + l1tf: Not affected + mds: Not affected + meltdown: Not affected + mmio_stale_data: Not affected + retbleed: Not affected + spec_store_bypass: Mitigation of SSB disabled via prctl + spectre_v1: Mitigation of usercopy/swapgs barriers and __user pointer sanitization + spectre_v2: Mitigation of Retpolines IBPB: conditional IBRS_FW STIBP: always-on RSB filling PBRSB-eIBRS: Not affected + srbds: Not affected + tsx_async_abort: Not affected

Testing initiated at 28 June 2023 20:58 by user phoronix.

Xeon Max 9480 2P, HBM Caching

Processor: 2 x Intel Xeon Max 9480 @ 3.50GHz (112 Cores / 224 Threads), Motherboard: Supermicro X13DEM v1.10 (1.3 BIOS), Chipset: Intel Device 1bce, Memory: 512GB, Disk: 7682GB INTEL SSDPF2KX076TZ, Graphics: ASPEED, Monitor: VE228, Network: 2 x Broadcom BCM57508 NetXtreme-E 10Gb/25Gb/40Gb/50Gb/100Gb/200Gb

OS: Ubuntu 23.04, Kernel: 6.2.0-20-generic (x86_64), Desktop: GNOME Shell 44.0, Display Server: X Server 1.21.1.7, Compiler: GCC 12.2.0, File-System: ext4, Screen Resolution: 1920x1080

Kernel Notes: Transparent Huge Pages: madvise
Compiler Notes: --build=x86_64-linux-gnu --disable-vtable-verify --disable-werror --enable-cet --enable-checking=release --enable-clocale=gnu --enable-default-pie --enable-gnu-unique-object --enable-languages=c,ada,c++,go,d,fortran,objc,obj-c++,m2 --enable-libphobos-checking=release --enable-libstdcxx-debug --enable-libstdcxx-time=yes --enable-multiarch --enable-multilib --enable-nls --enable-objc-gc=auto --enable-offload-defaulted --enable-offload-targets=nvptx-none=/build/gcc-12-Pa930Z/gcc-12-12.2.0/debian/tmp-nvptx/usr,amdgcn-amdhsa=/build/gcc-12-Pa930Z/gcc-12-12.2.0/debian/tmp-gcn/usr --enable-plugin --enable-shared --enable-threads=posix --host=x86_64-linux-gnu --program-prefix=x86_64-linux-gnu- --target=x86_64-linux-gnu --with-abi=m64 --with-arch-32=i686 --with-default-libstdcxx-abi=new --with-gcc-major-version-only --with-multilib-list=m32,m64,mx32 --with-target-system-zlib=auto --with-tune=generic --without-cuda-driver -v
Processor Notes: Scaling Governor: intel_cpufreq performance - CPU Microcode: 0x2c0001d1
Python Notes: Python 3.11.2
Security Notes: itlb_multihit: Not affected + l1tf: Not affected + mds: Not affected + meltdown: Not affected + mmio_stale_data: Not affected + retbleed: Not affected + spec_store_bypass: Mitigation of SSB disabled via prctl + spectre_v1: Mitigation of usercopy/swapgs barriers and __user pointer sanitization + spectre_v2: Mitigation of Enhanced IBRS IBPB: conditional RSB filling PBRSB-eIBRS: SW sequence + srbds: Not affected + tsx_async_abort: Not affected

Testing initiated at 29 June 2023 09:57 by user phoronix.