Xeon Max AMX HBM2e

Benchmarks for a future article on Phoronix.

HTML result view exported from: https://openbenchmarking.org/result/2306302-NE-XEONMAXAM96&grs.

Xeon Max AMX HBM2eProcessorMotherboardChipsetMemoryDiskGraphicsMonitorNetworkOSKernelDesktopDisplay ServerCompilerFile-SystemScreen ResolutionEPYC 9654 2PEPYC 9554 2PXeon Max 9480 2P, HBM Caching2 x AMD EPYC 9654 96-Core @ 2.40GHz (192 Cores / 384 Threads)AMD Titanite_4G (RTI1007B BIOS)AMD Device 14a41520GB7682GB INTEL SSDPF2KX076TZASPEEDVGA HDMIBroadcom NetXtreme BCM5720 PCIeUbuntu 23.046.2.0-20-generic (x86_64)GNOME Shell 44.0X Server 1.21.1.7GCC 12.2.0ext41920x10802 x AMD EPYC 9554 64-Core @ 3.10GHz (128 Cores / 256 Threads)2 x Intel Xeon Max 9480 @ 3.50GHz (112 Cores / 224 Threads)Supermicro X13DEM v1.10 (1.3 BIOS)Intel Device 1bce512GBVE2282 x Broadcom BCM57508 NetXtreme-E 10Gb/25Gb/40Gb/50Gb/100Gb/200GbOpenBenchmarking.orgKernel Details- Transparent Huge Pages: madviseCompiler Details- --build=x86_64-linux-gnu --disable-vtable-verify --disable-werror --enable-cet --enable-checking=release --enable-clocale=gnu --enable-default-pie --enable-gnu-unique-object --enable-languages=c,ada,c++,go,d,fortran,objc,obj-c++,m2 --enable-libphobos-checking=release --enable-libstdcxx-debug --enable-libstdcxx-time=yes --enable-multiarch --enable-multilib --enable-nls --enable-objc-gc=auto --enable-offload-defaulted --enable-offload-targets=nvptx-none=/build/gcc-12-Pa930Z/gcc-12-12.2.0/debian/tmp-nvptx/usr,amdgcn-amdhsa=/build/gcc-12-Pa930Z/gcc-12-12.2.0/debian/tmp-gcn/usr --enable-plugin --enable-shared --enable-threads=posix --host=x86_64-linux-gnu --program-prefix=x86_64-linux-gnu- --target=x86_64-linux-gnu --with-abi=m64 --with-arch-32=i686 --with-default-libstdcxx-abi=new --with-gcc-major-version-only --with-multilib-list=m32,m64,mx32 --with-target-system-zlib=auto --with-tune=generic --without-cuda-driver -v Processor Details- EPYC 9654 2P: Scaling Governor: acpi-cpufreq performance (Boost: Enabled) - CPU Microcode: 0xa101121- EPYC 9554 2P: Scaling Governor: acpi-cpufreq performance (Boost: Enabled) - CPU Microcode: 0xa101121- Xeon Max 9480 2P, HBM Caching: Scaling Governor: intel_cpufreq performance - CPU Microcode: 0x2c0001d1Python Details- Python 3.11.2Security Details- EPYC 9654 2P: itlb_multihit: Not affected + l1tf: Not affected + mds: Not affected + meltdown: Not affected + mmio_stale_data: Not affected + retbleed: Not affected + spec_store_bypass: Mitigation of SSB disabled via prctl + spectre_v1: Mitigation of usercopy/swapgs barriers and __user pointer sanitization + spectre_v2: Mitigation of Retpolines IBPB: conditional IBRS_FW STIBP: always-on RSB filling PBRSB-eIBRS: Not affected + srbds: Not affected + tsx_async_abort: Not affected - EPYC 9554 2P: itlb_multihit: Not affected + l1tf: Not affected + mds: Not affected + meltdown: Not affected + mmio_stale_data: Not affected + retbleed: Not affected + spec_store_bypass: Mitigation of SSB disabled via prctl + spectre_v1: Mitigation of usercopy/swapgs barriers and __user pointer sanitization + spectre_v2: Mitigation of Retpolines IBPB: conditional IBRS_FW STIBP: always-on RSB filling PBRSB-eIBRS: Not affected + srbds: Not affected + tsx_async_abort: Not affected - Xeon Max 9480 2P, HBM Caching: itlb_multihit: Not affected + l1tf: Not affected + mds: Not affected + meltdown: Not affected + mmio_stale_data: Not affected + retbleed: Not affected + spec_store_bypass: Mitigation of SSB disabled via prctl + spectre_v1: Mitigation of usercopy/swapgs barriers and __user pointer sanitization + spectre_v2: Mitigation of Enhanced IBRS IBPB: conditional RSB filling PBRSB-eIBRS: SW sequence + srbds: Not affected + tsx_async_abort: Not affected

Xeon Max AMX HBM2eonednn: Deconvolution Batch shapes_3d - bf16bf16bf16 - CPUopenvino: Vehicle Detection FP16-INT8 - CPUonednn: IP Shapes 3D - bf16bf16bf16 - CPUopenvino: Person Vehicle Bike Detection FP16 - CPUtensorflow: CPU - 512 - VGG-16openvino: Vehicle Detection FP16 - CPUonednn: Recurrent Neural Network Inference - bf16bf16bf16 - CPUopenvino: Weld Porosity Detection FP16-INT8 - CPUtensorflow: CPU - 512 - AlexNettensorflow: CPU - 512 - GoogLeNettensorflow: CPU - 512 - ResNet-50libxsmm: 128openvino: Vehicle Detection FP16 - CPUopenvino: Face Detection FP16-INT8 - CPUopenvino: Weld Porosity Detection FP16 - CPUopenvino: Face Detection FP16 - CPUonednn: Convolution Batch Shapes Auto - bf16bf16bf16 - CPUopenvino: Vehicle Detection FP16-INT8 - CPUopenvino: Weld Porosity Detection FP16-INT8 - CPUopenvino: Weld Porosity Detection FP16 - CPUopenvino: Face Detection FP16-INT8 - CPUopenvino: Machine Translation EN To DE FP16 - CPUtensorflow: CPU - 16 - ResNet-50openvino: Age Gender Recognition Retail 0013 FP16 - CPUopenvino: Face Detection FP16 - CPUopenvino: Person Vehicle Bike Detection FP16 - CPUopenvino: Person Detection FP32 - CPUopenvino: Person Detection FP16 - CPUopenvino: Person Detection FP16 - CPUopenvino: Person Detection FP32 - CPUopenvino: Age Gender Recognition Retail 0013 FP16 - CPUonnx: ArcFace ResNet-100 - CPU - Standardopenvino: Machine Translation EN To DE FP16 - CPUlibxsmm: 64libxsmm: 32libxsmm: 256onnx: ResNet50 v1-12-int8 - CPU - Standardonnx: ResNet50 v1-12-int8 - CPU - Standardonnx: ArcFace ResNet-100 - CPU - Standardonnx: fcn-resnet101-11 - CPU - Standardonnx: fcn-resnet101-11 - CPU - Standardonnx: CaffeNet 12-int8 - CPU - Standardonnx: CaffeNet 12-int8 - CPU - Standardonnx: bertsquad-12 - CPU - Standardonnx: bertsquad-12 - CPU - Standardonnx: yolov4 - CPU - Standardonnx: yolov4 - CPU - Standardonnx: GPT-2 - CPU - Standardonnx: GPT-2 - CPU - Standardopenvino: Age Gender Recognition Retail 0013 FP16-INT8 - CPUopenvino: Age Gender Recognition Retail 0013 FP16-INT8 - CPUtensorflow: CPU - 16 - GoogLeNettensorflow: CPU - 16 - AlexNettensorflow: CPU - 16 - VGG-16onednn: Recurrent Neural Network Training - bf16bf16bf16 - CPUonednn: Deconvolution Batch shapes_1d - bf16bf16bf16 - CPUonednn: IP Shapes 1D - bf16bf16bf16 - CPUEPYC 9654 2PEPYC 9554 2PXeon Max 9480 2P, HBM Caching0.7321004.363.802095.25120.627390.972790.909.951757.16535.52171.563426.16.49191.419843.03471.820.46105310985.1719170.614.86250.39950.9425.25146867.48101.569133.221098.961092.9743.6443.400.5519.308450.422270.21298.04379.66.87053146.55551.7898212.8634.697972.30611435.19498.995710.17944204.0914.900339.68904103.1840.99124618.3264.49166.2047.272492.692.5332616.98230.4572723.542.205964.29109.895981.57925.7678.091813.68593.20188.174112.95.34155.288039.57389.530.3113789010.4515659.323.97205.71786.0133.40162207.9782.047447.60825.52832.0238.2438.530.5224.012240.672692.71402.24603.36.55989152.43941.6445214.6064.659822.01949495.24085.900911.6871195.8995.105057.64735132.7210.88128593.2494.41257.9460.311305.661.7616211.196693.5352520.2391.981417.8330.192315.121325.453.47718.54236.4476.931814.312.07334.8916941.99232.013.673565528.0330938.506.44333.51590.9240.15108515.12120.406265.77819.76834.9733.4033.970.4423.772947.298.02166125.28442.1801191.6235.335141.78296562.21887.763811.4576130.3707.700464.70849218.6921.4059457.1498.43230.1121.0915294.60.4830757.88016OpenBenchmarking.org

oneDNN

Harness: Deconvolution Batch shapes_3d - Data Type: bf16bf16bf16 - Engine: CPU

OpenBenchmarking.orgms, Fewer Is BetteroneDNN 3.1Harness: Deconvolution Batch shapes_3d - Data Type: bf16bf16bf16 - Engine: CPUEPYC 9654 2PEPYC 9554 2PXeon Max 9480 2P, HBM Caching0.79541.59082.38623.18163.977SE +/- 0.004782, N = 9SE +/- 0.002054, N = 9SE +/- 0.029518, N = 150.7321000.4572723.535250MIN: 2.461. (CXX) g++ options: -O3 -march=native -fopenmp -msse4.1 -fPIC -pie -ldl -lpthread

OpenVINO

Model: Vehicle Detection FP16-INT8 - Device: CPU

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2022.3Model: Vehicle Detection FP16-INT8 - Device: CPUEPYC 9654 2PEPYC 9554 2PXeon Max 9480 2P, HBM Caching510152025SE +/- 0.00, N = 3SE +/- 0.00, N = 3SE +/- 0.09, N = 34.363.5420.23MIN: 3.52 / MAX: 38.49MIN: 3.45 / MAX: 21.7MIN: 12.69 / MAX: 166.761. (CXX) g++ options: -isystem -fsigned-char -ffunction-sections -fdata-sections -msse4.1 -msse4.2 -O3 -fno-strict-overflow -fwrapv -fPIC -fvisibility=hidden -Os -std=c++11 -MD -MT -MF

oneDNN

Harness: IP Shapes 3D - Data Type: bf16bf16bf16 - Engine: CPU

OpenBenchmarking.orgms, Fewer Is BetteroneDNN 3.1Harness: IP Shapes 3D - Data Type: bf16bf16bf16 - Engine: CPUEPYC 9654 2PEPYC 9554 2PXeon Max 9480 2P, HBM Caching20406080100SE +/- 0.03459, N = 15SE +/- 0.02374, N = 15SE +/- 0.61972, N = 53.802092.2059691.98140MIN: 79.161. (CXX) g++ options: -O3 -march=native -fopenmp -msse4.1 -fPIC -pie -ldl -lpthread

OpenVINO

Model: Person Vehicle Bike Detection FP16 - Device: CPU

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2022.3Model: Person Vehicle Bike Detection FP16 - Device: CPUEPYC 9654 2PEPYC 9554 2PXeon Max 9480 2P, HBM Caching48121620SE +/- 0.00, N = 3SE +/- 0.01, N = 3SE +/- 0.02, N = 35.254.2917.83MIN: 4.36 / MAX: 23.21MIN: 4.11 / MAX: 21.83MIN: 13.06 / MAX: 147.361. (CXX) g++ options: -isystem -fsigned-char -ffunction-sections -fdata-sections -msse4.1 -msse4.2 -O3 -fno-strict-overflow -fwrapv -fPIC -fvisibility=hidden -Os -std=c++11 -MD -MT -MF

TensorFlow

Device: CPU - Batch Size: 512 - Model: VGG-16

OpenBenchmarking.orgimages/sec, More Is BetterTensorFlow 2.12Device: CPU - Batch Size: 512 - Model: VGG-16EPYC 9654 2PEPYC 9554 2PXeon Max 9480 2P, HBM Caching306090120150SE +/- 0.08, N = 3SE +/- 1.46, N = 3SE +/- 0.33, N = 5120.62109.8930.19

OpenVINO

Model: Vehicle Detection FP16 - Device: CPU

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2022.3Model: Vehicle Detection FP16 - Device: CPUEPYC 9654 2PEPYC 9554 2PXeon Max 9480 2P, HBM Caching16003200480064008000SE +/- 2.12, N = 3SE +/- 3.49, N = 3SE +/- 17.29, N = 137390.975981.572315.121. (CXX) g++ options: -isystem -fsigned-char -ffunction-sections -fdata-sections -msse4.1 -msse4.2 -O3 -fno-strict-overflow -fwrapv -fPIC -fvisibility=hidden -Os -std=c++11 -MD -MT -MF

oneDNN

Harness: Recurrent Neural Network Inference - Data Type: bf16bf16bf16 - Engine: CPU

OpenBenchmarking.orgms, Fewer Is BetteroneDNN 3.1Harness: Recurrent Neural Network Inference - Data Type: bf16bf16bf16 - Engine: CPUEPYC 9654 2PEPYC 9554 2PXeon Max 9480 2P, HBM Caching6001200180024003000SE +/- 29.80, N = 15SE +/- 2.71, N = 3SE +/- 20.64, N = 122790.90925.771325.45MIN: 2391.98MIN: 909.67MIN: 1025.181. (CXX) g++ options: -O3 -march=native -fopenmp -msse4.1 -fPIC -pie -ldl -lpthread

OpenVINO

Model: Weld Porosity Detection FP16-INT8 - Device: CPU

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2022.3Model: Weld Porosity Detection FP16-INT8 - Device: CPUEPYC 9654 2PEPYC 9554 2PXeon Max 9480 2P, HBM Caching3691215SE +/- 0.00, N = 3SE +/- 0.00, N = 3SE +/- 0.00, N = 39.958.093.47MIN: 8.36 / MAX: 40.39MIN: 7.89 / MAX: 45.39MIN: 2.48 / MAX: 56.571. (CXX) g++ options: -isystem -fsigned-char -ffunction-sections -fdata-sections -msse4.1 -msse4.2 -O3 -fno-strict-overflow -fwrapv -fPIC -fvisibility=hidden -Os -std=c++11 -MD -MT -MF

TensorFlow

Device: CPU - Batch Size: 512 - Model: AlexNet

OpenBenchmarking.orgimages/sec, More Is BetterTensorFlow 2.12Device: CPU - Batch Size: 512 - Model: AlexNetEPYC 9654 2PEPYC 9554 2PXeon Max 9480 2P, HBM Caching400800120016002000SE +/- 16.13, N = 7SE +/- 12.94, N = 3SE +/- 9.04, N = 31757.161813.68718.54

TensorFlow

Device: CPU - Batch Size: 512 - Model: GoogLeNet

OpenBenchmarking.orgimages/sec, More Is BetterTensorFlow 2.12Device: CPU - Batch Size: 512 - Model: GoogLeNetEPYC 9654 2PEPYC 9554 2PXeon Max 9480 2P, HBM Caching130260390520650SE +/- 0.69, N = 3SE +/- 0.96, N = 3SE +/- 1.79, N = 3535.52593.20236.44

TensorFlow

Device: CPU - Batch Size: 512 - Model: ResNet-50

OpenBenchmarking.orgimages/sec, More Is BetterTensorFlow 2.12Device: CPU - Batch Size: 512 - Model: ResNet-50EPYC 9654 2PEPYC 9554 2PXeon Max 9480 2P, HBM Caching4080120160200SE +/- 0.26, N = 3SE +/- 0.70, N = 3SE +/- 0.66, N = 3171.56188.1776.93

libxsmm

M N K: 128

OpenBenchmarking.orgGFLOPS/s, More Is Betterlibxsmm 2-1.17-3645M N K: 128EPYC 9654 2PEPYC 9554 2PXeon Max 9480 2P, HBM Caching9001800270036004500SE +/- 29.11, N = 3SE +/- 21.03, N = 3SE +/- 22.78, N = 93426.14112.91814.31. (CXX) g++ options: -dynamic -Bstatic -static-libgcc -lgomp -lm -lrt -ldl -lquadmath -lstdc++ -pthread -fPIC -std=c++14 -O2 -fopenmp-simd -funroll-loops -ftree-vectorize -fdata-sections -ffunction-sections -fvisibility=hidden -msse4.2

OpenVINO

Model: Vehicle Detection FP16 - Device: CPU

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2022.3Model: Vehicle Detection FP16 - Device: CPUEPYC 9654 2PEPYC 9554 2PXeon Max 9480 2P, HBM Caching3691215SE +/- 0.00, N = 3SE +/- 0.00, N = 3SE +/- 0.08, N = 136.495.3412.07MIN: 5.03 / MAX: 55.65MIN: 4.89 / MAX: 34.7MIN: 7.38 / MAX: 250.921. (CXX) g++ options: -isystem -fsigned-char -ffunction-sections -fdata-sections -msse4.1 -msse4.2 -O3 -fno-strict-overflow -fwrapv -fPIC -fvisibility=hidden -Os -std=c++11 -MD -MT -MF

OpenVINO

Model: Face Detection FP16-INT8 - Device: CPU

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2022.3Model: Face Detection FP16-INT8 - Device: CPUEPYC 9654 2PEPYC 9554 2PXeon Max 9480 2P, HBM Caching70140210280350SE +/- 0.06, N = 3SE +/- 0.14, N = 3SE +/- 0.45, N = 3191.41155.28334.891. (CXX) g++ options: -isystem -fsigned-char -ffunction-sections -fdata-sections -msse4.1 -msse4.2 -O3 -fno-strict-overflow -fwrapv -fPIC -fvisibility=hidden -Os -std=c++11 -MD -MT -MF

OpenVINO

Model: Weld Porosity Detection FP16 - Device: CPU

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2022.3Model: Weld Porosity Detection FP16 - Device: CPUEPYC 9654 2PEPYC 9554 2PXeon Max 9480 2P, HBM Caching4K8K12K16K20KSE +/- 2.32, N = 3SE +/- 1.22, N = 3SE +/- 13.63, N = 39843.038039.5716941.991. (CXX) g++ options: -isystem -fsigned-char -ffunction-sections -fdata-sections -msse4.1 -msse4.2 -O3 -fno-strict-overflow -fwrapv -fPIC -fvisibility=hidden -Os -std=c++11 -MD -MT -MF

OpenVINO

Model: Face Detection FP16 - Device: CPU

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2022.3Model: Face Detection FP16 - Device: CPUEPYC 9654 2PEPYC 9554 2PXeon Max 9480 2P, HBM Caching100200300400500SE +/- 0.93, N = 3SE +/- 0.19, N = 3SE +/- 0.80, N = 3471.82389.53232.01MIN: 431.51 / MAX: 569.3MIN: 380.82 / MAX: 432.57MIN: 138.61 / MAX: 770.451. (CXX) g++ options: -isystem -fsigned-char -ffunction-sections -fdata-sections -msse4.1 -msse4.2 -O3 -fno-strict-overflow -fwrapv -fPIC -fvisibility=hidden -Os -std=c++11 -MD -MT -MF

oneDNN

Harness: Convolution Batch Shapes Auto - Data Type: bf16bf16bf16 - Engine: CPU

OpenBenchmarking.orgms, Fewer Is BetteroneDNN 3.1Harness: Convolution Batch Shapes Auto - Data Type: bf16bf16bf16 - Engine: CPUEPYC 9654 2PEPYC 9554 2PXeon Max 9480 2P, HBM Caching0.82661.65322.47983.30644.133SE +/- 0.003345, N = 15SE +/- 0.002531, N = 7SE +/- 0.025287, N = 70.4610530.3113783.673560MIN: 3.031. (CXX) g++ options: -O3 -march=native -fopenmp -msse4.1 -fPIC -pie -ldl -lpthread

OpenVINO

Model: Vehicle Detection FP16-INT8 - Device: CPU

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2022.3Model: Vehicle Detection FP16-INT8 - Device: CPUEPYC 9654 2PEPYC 9554 2PXeon Max 9480 2P, HBM Caching2K4K6K8K10KSE +/- 0.53, N = 3SE +/- 2.20, N = 3SE +/- 25.18, N = 310985.179010.455528.031. (CXX) g++ options: -isystem -fsigned-char -ffunction-sections -fdata-sections -msse4.1 -msse4.2 -O3 -fno-strict-overflow -fwrapv -fPIC -fvisibility=hidden -Os -std=c++11 -MD -MT -MF

OpenVINO

Model: Weld Porosity Detection FP16-INT8 - Device: CPU

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2022.3Model: Weld Porosity Detection FP16-INT8 - Device: CPUEPYC 9654 2PEPYC 9554 2PXeon Max 9480 2P, HBM Caching7K14K21K28K35KSE +/- 4.81, N = 3SE +/- 3.60, N = 3SE +/- 38.32, N = 319170.6115659.3230938.501. (CXX) g++ options: -isystem -fsigned-char -ffunction-sections -fdata-sections -msse4.1 -msse4.2 -O3 -fno-strict-overflow -fwrapv -fPIC -fvisibility=hidden -Os -std=c++11 -MD -MT -MF

OpenVINO

Model: Weld Porosity Detection FP16 - Device: CPU

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2022.3Model: Weld Porosity Detection FP16 - Device: CPUEPYC 9654 2PEPYC 9554 2PXeon Max 9480 2P, HBM Caching246810SE +/- 0.00, N = 3SE +/- 0.00, N = 3SE +/- 0.01, N = 34.863.976.44MIN: 4.02 / MAX: 36.01MIN: 3.87 / MAX: 17.35MIN: 4.24 / MAX: 49.431. (CXX) g++ options: -isystem -fsigned-char -ffunction-sections -fdata-sections -msse4.1 -msse4.2 -O3 -fno-strict-overflow -fwrapv -fPIC -fvisibility=hidden -Os -std=c++11 -MD -MT -MF

OpenVINO

Model: Face Detection FP16-INT8 - Device: CPU

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2022.3Model: Face Detection FP16-INT8 - Device: CPUEPYC 9654 2PEPYC 9554 2PXeon Max 9480 2P, HBM Caching70140210280350SE +/- 0.04, N = 3SE +/- 0.22, N = 3SE +/- 0.44, N = 3250.39205.71333.51MIN: 222.36 / MAX: 303.72MIN: 202.38 / MAX: 252.14MIN: 285.32 / MAX: 516.281. (CXX) g++ options: -isystem -fsigned-char -ffunction-sections -fdata-sections -msse4.1 -msse4.2 -O3 -fno-strict-overflow -fwrapv -fPIC -fvisibility=hidden -Os -std=c++11 -MD -MT -MF

OpenVINO

Model: Machine Translation EN To DE FP16 - Device: CPU

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2022.3Model: Machine Translation EN To DE FP16 - Device: CPUEPYC 9654 2PEPYC 9554 2PXeon Max 9480 2P, HBM Caching2004006008001000SE +/- 0.77, N = 3SE +/- 1.56, N = 3SE +/- 0.19, N = 3950.94786.01590.921. (CXX) g++ options: -isystem -fsigned-char -ffunction-sections -fdata-sections -msse4.1 -msse4.2 -O3 -fno-strict-overflow -fwrapv -fPIC -fvisibility=hidden -Os -std=c++11 -MD -MT -MF

TensorFlow

Device: CPU - Batch Size: 16 - Model: ResNet-50

OpenBenchmarking.orgimages/sec, More Is BetterTensorFlow 2.12Device: CPU - Batch Size: 16 - Model: ResNet-50EPYC 9654 2PEPYC 9554 2PXeon Max 9480 2P, HBM Caching918273645SE +/- 0.31, N = 3SE +/- 0.23, N = 3SE +/- 0.32, N = 325.2533.4040.15

OpenVINO

Model: Age Gender Recognition Retail 0013 FP16 - Device: CPU

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2022.3Model: Age Gender Recognition Retail 0013 FP16 - Device: CPUEPYC 9654 2PEPYC 9554 2PXeon Max 9480 2P, HBM Caching30K60K90K120K150KSE +/- 1825.86, N = 3SE +/- 685.52, N = 3SE +/- 967.90, N = 3146867.48162207.97108515.121. (CXX) g++ options: -isystem -fsigned-char -ffunction-sections -fdata-sections -msse4.1 -msse4.2 -O3 -fno-strict-overflow -fwrapv -fPIC -fvisibility=hidden -Os -std=c++11 -MD -MT -MF

OpenVINO

Model: Face Detection FP16 - Device: CPU

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2022.3Model: Face Detection FP16 - Device: CPUEPYC 9654 2PEPYC 9554 2PXeon Max 9480 2P, HBM Caching306090120150SE +/- 0.25, N = 3SE +/- 0.04, N = 3SE +/- 0.41, N = 3101.5682.04120.401. (CXX) g++ options: -isystem -fsigned-char -ffunction-sections -fdata-sections -msse4.1 -msse4.2 -O3 -fno-strict-overflow -fwrapv -fPIC -fvisibility=hidden -Os -std=c++11 -MD -MT -MF

OpenVINO

Model: Person Vehicle Bike Detection FP16 - Device: CPU

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2022.3Model: Person Vehicle Bike Detection FP16 - Device: CPUEPYC 9654 2PEPYC 9554 2PXeon Max 9480 2P, HBM Caching2K4K6K8K10KSE +/- 3.38, N = 3SE +/- 8.54, N = 3SE +/- 8.17, N = 39133.227447.606265.771. (CXX) g++ options: -isystem -fsigned-char -ffunction-sections -fdata-sections -msse4.1 -msse4.2 -O3 -fno-strict-overflow -fwrapv -fPIC -fvisibility=hidden -Os -std=c++11 -MD -MT -MF

OpenVINO

Model: Person Detection FP32 - Device: CPU

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2022.3Model: Person Detection FP32 - Device: CPUEPYC 9654 2PEPYC 9554 2PXeon Max 9480 2P, HBM Caching2004006008001000SE +/- 5.33, N = 3SE +/- 1.07, N = 3SE +/- 7.20, N = 31098.96825.52819.76MIN: 825.62 / MAX: 1835.76MIN: 747.05 / MAX: 1238.39MIN: 516.88 / MAX: 2598.391. (CXX) g++ options: -isystem -fsigned-char -ffunction-sections -fdata-sections -msse4.1 -msse4.2 -O3 -fno-strict-overflow -fwrapv -fPIC -fvisibility=hidden -Os -std=c++11 -MD -MT -MF

OpenVINO

Model: Person Detection FP16 - Device: CPU

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2022.3Model: Person Detection FP16 - Device: CPUEPYC 9654 2PEPYC 9554 2PXeon Max 9480 2P, HBM Caching2004006008001000SE +/- 5.11, N = 3SE +/- 5.30, N = 3SE +/- 6.17, N = 31092.97832.02834.97MIN: 825.28 / MAX: 1814.82MIN: 743.95 / MAX: 1258.17MIN: 558.93 / MAX: 2258.391. (CXX) g++ options: -isystem -fsigned-char -ffunction-sections -fdata-sections -msse4.1 -msse4.2 -O3 -fno-strict-overflow -fwrapv -fPIC -fvisibility=hidden -Os -std=c++11 -MD -MT -MF

OpenVINO

Model: Person Detection FP16 - Device: CPU

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2022.3Model: Person Detection FP16 - Device: CPUEPYC 9654 2PEPYC 9554 2PXeon Max 9480 2P, HBM Caching1020304050SE +/- 0.21, N = 3SE +/- 0.23, N = 3SE +/- 0.24, N = 343.6438.2433.401. (CXX) g++ options: -isystem -fsigned-char -ffunction-sections -fdata-sections -msse4.1 -msse4.2 -O3 -fno-strict-overflow -fwrapv -fPIC -fvisibility=hidden -Os -std=c++11 -MD -MT -MF

OpenVINO

Model: Person Detection FP32 - Device: CPU

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2022.3Model: Person Detection FP32 - Device: CPUEPYC 9654 2PEPYC 9554 2PXeon Max 9480 2P, HBM Caching1020304050SE +/- 0.20, N = 3SE +/- 0.04, N = 3SE +/- 0.31, N = 343.4038.5333.971. (CXX) g++ options: -isystem -fsigned-char -ffunction-sections -fdata-sections -msse4.1 -msse4.2 -O3 -fno-strict-overflow -fwrapv -fPIC -fvisibility=hidden -Os -std=c++11 -MD -MT -MF

OpenVINO

Model: Age Gender Recognition Retail 0013 FP16 - Device: CPU

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2022.3Model: Age Gender Recognition Retail 0013 FP16 - Device: CPUEPYC 9654 2PEPYC 9554 2PXeon Max 9480 2P, HBM Caching0.12380.24760.37140.49520.619SE +/- 0.00, N = 3SE +/- 0.00, N = 3SE +/- 0.00, N = 30.550.520.44MIN: 0.5 / MAX: 45.73MIN: 0.49 / MAX: 50.12MIN: 0.33 / MAX: 42.851. (CXX) g++ options: -isystem -fsigned-char -ffunction-sections -fdata-sections -msse4.1 -msse4.2 -O3 -fno-strict-overflow -fwrapv -fPIC -fvisibility=hidden -Os -std=c++11 -MD -MT -MF

ONNX Runtime

Model: ArcFace ResNet-100 - Device: CPU - Executor: Standard

OpenBenchmarking.orgInferences Per Second, More Is BetterONNX Runtime 1.14Model: ArcFace ResNet-100 - Device: CPU - Executor: StandardEPYC 9654 2PEPYC 9554 2PXeon Max 9480 2P, HBM Caching612182430SE +/- 0.09, N = 3SE +/- 0.09, N = 3SE +/- 0.34, N = 1519.3124.0123.771. (CXX) g++ options: -ffunction-sections -fdata-sections -march=native -mtune=native -O3 -flto=auto -fno-fat-lto-objects -ldl -lrt

OpenVINO

Model: Machine Translation EN To DE FP16 - Device: CPU

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2022.3Model: Machine Translation EN To DE FP16 - Device: CPUEPYC 9654 2PEPYC 9554 2PXeon Max 9480 2P, HBM Caching1122334455SE +/- 0.04, N = 3SE +/- 0.08, N = 3SE +/- 0.01, N = 350.4240.6747.29MIN: 39.07 / MAX: 280.94MIN: 34.96 / MAX: 124.31MIN: 32.14 / MAX: 577.781. (CXX) g++ options: -isystem -fsigned-char -ffunction-sections -fdata-sections -msse4.1 -msse4.2 -O3 -fno-strict-overflow -fwrapv -fPIC -fvisibility=hidden -Os -std=c++11 -MD -MT -MF

libxsmm

M N K: 64

OpenBenchmarking.orgGFLOPS/s, More Is Betterlibxsmm 2-1.17-3645M N K: 64EPYC 9654 2PEPYC 9554 2P6001200180024003000SE +/- 33.30, N = 15SE +/- 5.63, N = 72270.22692.71. (CXX) g++ options: -dynamic -Bstatic -static-libgcc -lgomp -lm -lrt -ldl -lquadmath -lstdc++ -pthread -fPIC -std=c++14 -O2 -fopenmp-simd -funroll-loops -ftree-vectorize -fdata-sections -ffunction-sections -fvisibility=hidden -msse4.2

libxsmm

M N K: 32

OpenBenchmarking.orgGFLOPS/s, More Is Betterlibxsmm 2-1.17-3645M N K: 32EPYC 9654 2PEPYC 9554 2P30060090012001500SE +/- 9.98, N = 15SE +/- 4.06, N = 81298.01402.21. (CXX) g++ options: -dynamic -Bstatic -static-libgcc -lgomp -lm -lrt -ldl -lquadmath -lstdc++ -pthread -fPIC -std=c++14 -O2 -fopenmp-simd -funroll-loops -ftree-vectorize -fdata-sections -ffunction-sections -fvisibility=hidden -msse4.2

libxsmm

M N K: 256

OpenBenchmarking.orgGFLOPS/s, More Is Betterlibxsmm 2-1.17-3645M N K: 256EPYC 9654 2PEPYC 9554 2P10002000300040005000SE +/- 47.44, N = 5SE +/- 34.70, N = 34379.64603.31. (CXX) g++ options: -dynamic -Bstatic -static-libgcc -lgomp -lm -lrt -ldl -lquadmath -lstdc++ -pthread -fPIC -std=c++14 -O2 -fopenmp-simd -funroll-loops -ftree-vectorize -fdata-sections -ffunction-sections -fvisibility=hidden -msse4.2

System Power Consumption Monitor

Phoronix Test Suite System Monitoring

OpenBenchmarking.orgWattsSystem Power Consumption MonitorPhoronix Test Suite System MonitoringXeon Max 9480 2P, HBM Caching160320480640800Min: 260 / Avg: 751.74 / Max: 907

CPU Peak Freq (Highest CPU Core Frequency) Monitor

Phoronix Test Suite System Monitoring

OpenBenchmarking.orgMegahertzCPU Peak Freq (Highest CPU Core Frequency) MonitorPhoronix Test Suite System MonitoringXeon Max 9480 2P, HBM Caching6001200180024003000Min: 1100 / Avg: 3207.85 / Max: 3577

ONNX Runtime

System Power Consumption Monitor

OpenBenchmarking.orgWatts, Fewer Is BetterONNX Runtime 1.14System Power Consumption MonitorXeon Max 9480 2P, HBM Caching150300450600750Min: 410 / Avg: 784.18 / Max: 830

ONNX Runtime

CPU Peak Freq (Highest CPU Core Frequency) Monitor

OpenBenchmarking.orgMegahertz, More Is BetterONNX Runtime 1.14CPU Peak Freq (Highest CPU Core Frequency) MonitorXeon Max 9480 2P, HBM Caching6001200180024003000Min: 2006 / Avg: 3497.11 / Max: 3509

ONNX Runtime

System Power Consumption Monitor

OpenBenchmarking.orgWatts, Fewer Is BetterONNX Runtime 1.14System Power Consumption MonitorXeon Max 9480 2P, HBM Caching140280420560700Min: 405 / Avg: 783.58 / Max: 821

ONNX Runtime

CPU Peak Freq (Highest CPU Core Frequency) Monitor

OpenBenchmarking.orgMegahertz, More Is BetterONNX Runtime 1.14CPU Peak Freq (Highest CPU Core Frequency) MonitorXeon Max 9480 2P, HBM Caching6001200180024003000Min: 3500 / Avg: 3500.03 / Max: 3508

ONNX Runtime

System Power Consumption Monitor

OpenBenchmarking.orgWatts, Fewer Is BetterONNX Runtime 1.14System Power Consumption MonitorXeon Max 9480 2P, HBM Caching150300450600750Min: 400 / Avg: 778.48 / Max: 853

ONNX Runtime

CPU Peak Freq (Highest CPU Core Frequency) Monitor

OpenBenchmarking.orgMegahertz, More Is BetterONNX Runtime 1.14CPU Peak Freq (Highest CPU Core Frequency) MonitorXeon Max 9480 2P, HBM Caching6001200180024003000Min: 1900 / Avg: 3482.84 / Max: 3511

ONNX Runtime

System Power Consumption Monitor

OpenBenchmarking.orgWatts, Fewer Is BetterONNX Runtime 1.14System Power Consumption MonitorXeon Max 9480 2P, HBM Caching150300450600750Min: 410 / Avg: 784.54 / Max: 844

ONNX Runtime

CPU Peak Freq (Highest CPU Core Frequency) Monitor

OpenBenchmarking.orgMegahertz, More Is BetterONNX Runtime 1.14CPU Peak Freq (Highest CPU Core Frequency) MonitorXeon Max 9480 2P, HBM Caching6001200180024003000Min: 3500 / Avg: 3500.01 / Max: 3507

ONNX Runtime

System Power Consumption Monitor

OpenBenchmarking.orgWatts, Fewer Is BetterONNX Runtime 1.14System Power Consumption MonitorXeon Max 9480 2P, HBM Caching150300450600750Min: 399 / Avg: 783.38 / Max: 828

ONNX Runtime

CPU Peak Freq (Highest CPU Core Frequency) Monitor

OpenBenchmarking.orgMegahertz, More Is BetterONNX Runtime 1.14CPU Peak Freq (Highest CPU Core Frequency) MonitorXeon Max 9480 2P, HBM Caching6001200180024003000Min: 3111 / Avg: 3499.75 / Max: 3508

ONNX Runtime

System Power Consumption Monitor

MinAvgMaxXeon Max 9480 2P, HBM Caching390783827OpenBenchmarking.orgWatts, Fewer Is BetterONNX Runtime 1.14System Power Consumption Monitor2004006008001000

ONNX Runtime

CPU Peak Freq (Highest CPU Core Frequency) Monitor

MinAvgMaxXeon Max 9480 2P, HBM Caching350035003507OpenBenchmarking.orgMegahertz, More Is BetterONNX Runtime 1.14CPU Peak Freq (Highest CPU Core Frequency) Monitor10002000300040005000

ONNX Runtime

System Power Consumption Monitor

OpenBenchmarking.orgWatts, Fewer Is BetterONNX Runtime 1.14System Power Consumption MonitorXeon Max 9480 2P, HBM Caching140280420560700Min: 281 / Avg: 720.02 / Max: 770

ONNX Runtime

CPU Peak Freq (Highest CPU Core Frequency) Monitor

OpenBenchmarking.orgMegahertz, More Is BetterONNX Runtime 1.14CPU Peak Freq (Highest CPU Core Frequency) MonitorXeon Max 9480 2P, HBM Caching6001200180024003000Min: 3500 / Avg: 3500.97 / Max: 3517

OpenVINO

System Power Consumption Monitor

MinAvgMaxXeon Max 9480 2P, HBM Caching268742856OpenBenchmarking.orgWatts, Fewer Is BetterOpenVINO 2022.3System Power Consumption Monitor2004006008001000

OpenVINO

CPU Peak Freq (Highest CPU Core Frequency) Monitor

MinAvgMaxXeon Max 9480 2P, HBM Caching110020993500OpenBenchmarking.orgMegahertz, More Is BetterOpenVINO 2022.3CPU Peak Freq (Highest CPU Core Frequency) Monitor10002000300040005000

OpenVINO

System Power Consumption Monitor

MinAvgMaxXeon Max 9480 2P, HBM Caching392772837OpenBenchmarking.orgWatts, Fewer Is BetterOpenVINO 2022.3System Power Consumption Monitor2004006008001000

OpenVINO

CPU Peak Freq (Highest CPU Core Frequency) Monitor

MinAvgMaxXeon Max 9480 2P, HBM Caching190034263501OpenBenchmarking.orgMegahertz, More Is BetterOpenVINO 2022.3CPU Peak Freq (Highest CPU Core Frequency) Monitor10002000300040005000

OpenVINO

System Power Consumption Monitor

MinAvgMaxXeon Max 9480 2P, HBM Caching284724867OpenBenchmarking.orgWatts, Fewer Is BetterOpenVINO 2022.3System Power Consumption Monitor2004006008001000

OpenVINO

CPU Peak Freq (Highest CPU Core Frequency) Monitor

MinAvgMaxXeon Max 9480 2P, HBM Caching160021253502OpenBenchmarking.orgMegahertz, More Is BetterOpenVINO 2022.3CPU Peak Freq (Highest CPU Core Frequency) Monitor10002000300040005000

OpenVINO

System Power Consumption Monitor

MinAvgMaxXeon Max 9480 2P, HBM Caching275752830OpenBenchmarking.orgWatts, Fewer Is BetterOpenVINO 2022.3System Power Consumption Monitor2004006008001000

OpenVINO

CPU Peak Freq (Highest CPU Core Frequency) Monitor

MinAvgMaxXeon Max 9480 2P, HBM Caching150019023504OpenBenchmarking.orgMegahertz, More Is BetterOpenVINO 2022.3CPU Peak Freq (Highest CPU Core Frequency) Monitor10002000300040005000

OpenVINO

System Power Consumption Monitor

MinAvgMaxXeon Max 9480 2P, HBM Caching274752852OpenBenchmarking.orgWatts, Fewer Is BetterOpenVINO 2022.3System Power Consumption Monitor2004006008001000

OpenVINO

CPU Peak Freq (Highest CPU Core Frequency) Monitor

MinAvgMaxXeon Max 9480 2P, HBM Caching170020893501OpenBenchmarking.orgMegahertz, More Is BetterOpenVINO 2022.3CPU Peak Freq (Highest CPU Core Frequency) Monitor10002000300040005000

OpenVINO

System Power Consumption Monitor

MinAvgMaxXeon Max 9480 2P, HBM Caching279756840OpenBenchmarking.orgWatts, Fewer Is BetterOpenVINO 2022.3System Power Consumption Monitor2004006008001000

OpenVINO

CPU Peak Freq (Highest CPU Core Frequency) Monitor

MinAvgMaxXeon Max 9480 2P, HBM Caching170019003500OpenBenchmarking.orgMegahertz, More Is BetterOpenVINO 2022.3CPU Peak Freq (Highest CPU Core Frequency) Monitor10002000300040005000

OpenVINO

System Power Consumption Monitor

MinAvgMaxXeon Max 9480 2P, HBM Caching262755817OpenBenchmarking.orgWatts, Fewer Is BetterOpenVINO 2022.3System Power Consumption Monitor2004006008001000

OpenVINO

CPU Peak Freq (Highest CPU Core Frequency) Monitor

MinAvgMaxXeon Max 9480 2P, HBM Caching130016853500OpenBenchmarking.orgMegahertz, More Is BetterOpenVINO 2022.3CPU Peak Freq (Highest CPU Core Frequency) Monitor10002000300040005000

OpenVINO

System Power Consumption Monitor

MinAvgMaxXeon Max 9480 2P, HBM Caching270722818OpenBenchmarking.orgWatts, Fewer Is BetterOpenVINO 2022.3System Power Consumption Monitor2004006008001000

OpenVINO

CPU Peak Freq (Highest CPU Core Frequency) Monitor

MinAvgMaxXeon Max 9480 2P, HBM Caching170023703505OpenBenchmarking.orgMegahertz, More Is BetterOpenVINO 2022.3CPU Peak Freq (Highest CPU Core Frequency) Monitor10002000300040005000

OpenVINO

System Power Consumption Monitor

MinAvgMaxXeon Max 9480 2P, HBM Caching272742868OpenBenchmarking.orgWatts, Fewer Is BetterOpenVINO 2022.3System Power Consumption Monitor2004006008001000

OpenVINO

CPU Peak Freq (Highest CPU Core Frequency) Monitor

MinAvgMaxXeon Max 9480 2P, HBM Caching130017383501OpenBenchmarking.orgMegahertz, More Is BetterOpenVINO 2022.3CPU Peak Freq (Highest CPU Core Frequency) Monitor10002000300040005000

OpenVINO

System Power Consumption Monitor

MinAvgMaxXeon Max 9480 2P, HBM Caching369748857OpenBenchmarking.orgWatts, Fewer Is BetterOpenVINO 2022.3System Power Consumption Monitor2004006008001000

OpenVINO

CPU Peak Freq (Highest CPU Core Frequency) Monitor

MinAvgMaxXeon Max 9480 2P, HBM Caching190030743506OpenBenchmarking.orgMegahertz, More Is BetterOpenVINO 2022.3CPU Peak Freq (Highest CPU Core Frequency) Monitor10002000300040005000

OpenVINO

System Power Consumption Monitor

MinAvgMaxXeon Max 9480 2P, HBM Caching298736817OpenBenchmarking.orgWatts, Fewer Is BetterOpenVINO 2022.3System Power Consumption Monitor2004006008001000

OpenVINO

CPU Peak Freq (Highest CPU Core Frequency) Monitor

MinAvgMaxXeon Max 9480 2P, HBM Caching190030233512OpenBenchmarking.orgMegahertz, More Is BetterOpenVINO 2022.3CPU Peak Freq (Highest CPU Core Frequency) Monitor10002000300040005000

OpenVINO

System Power Consumption Monitor

MinAvgMaxXeon Max 9480 2P, HBM Caching281724839OpenBenchmarking.orgWatts, Fewer Is BetterOpenVINO 2022.3System Power Consumption Monitor2004006008001000

OpenVINO

CPU Peak Freq (Highest CPU Core Frequency) Monitor

MinAvgMaxXeon Max 9480 2P, HBM Caching120017403505OpenBenchmarking.orgMegahertz, More Is BetterOpenVINO 2022.3CPU Peak Freq (Highest CPU Core Frequency) Monitor10002000300040005000

TensorFlow

System Power Consumption Monitor

OpenBenchmarking.orgWatts, Fewer Is BetterTensorFlow 2.12System Power Consumption MonitorXeon Max 9480 2P, HBM Caching150300450600750Min: 401 / Avg: 775.8 / Max: 850

TensorFlow

CPU Peak Freq (Highest CPU Core Frequency) Monitor

OpenBenchmarking.orgMegahertz, More Is BetterTensorFlow 2.12CPU Peak Freq (Highest CPU Core Frequency) MonitorXeon Max 9480 2P, HBM Caching6001200180024003000Min: 1900 / Avg: 3481.66 / Max: 3577

TensorFlow

System Power Consumption Monitor

MinAvgMaxXeon Max 9480 2P, HBM Caching425771850OpenBenchmarking.orgWatts, Fewer Is BetterTensorFlow 2.12System Power Consumption Monitor2004006008001000

TensorFlow

CPU Peak Freq (Highest CPU Core Frequency) Monitor

MinAvgMaxXeon Max 9480 2P, HBM Caching195833363515OpenBenchmarking.orgMegahertz, More Is BetterTensorFlow 2.12CPU Peak Freq (Highest CPU Core Frequency) Monitor10002000300040005000

TensorFlow

System Power Consumption Monitor

MinAvgMaxXeon Max 9480 2P, HBM Caching415733808OpenBenchmarking.orgWatts, Fewer Is BetterTensorFlow 2.12System Power Consumption Monitor2004006008001000

TensorFlow

CPU Peak Freq (Highest CPU Core Frequency) Monitor

MinAvgMaxXeon Max 9480 2P, HBM Caching350035003510OpenBenchmarking.orgMegahertz, More Is BetterTensorFlow 2.12CPU Peak Freq (Highest CPU Core Frequency) Monitor10002000300040005000

TensorFlow

System Power Consumption Monitor

MinAvgMaxXeon Max 9480 2P, HBM Caching405693801OpenBenchmarking.orgWatts, Fewer Is BetterTensorFlow 2.12System Power Consumption Monitor2004006008001000

TensorFlow

CPU Peak Freq (Highest CPU Core Frequency) Monitor

MinAvgMaxXeon Max 9480 2P, HBM Caching266634893505OpenBenchmarking.orgMegahertz, More Is BetterTensorFlow 2.12CPU Peak Freq (Highest CPU Core Frequency) Monitor10002000300040005000

TensorFlow

System Power Consumption Monitor

MinAvgMaxXeon Max 9480 2P, HBM Caching307752824OpenBenchmarking.orgWatts, Fewer Is BetterTensorFlow 2.12System Power Consumption Monitor2004006008001000

TensorFlow

CPU Peak Freq (Highest CPU Core Frequency) Monitor

MinAvgMaxXeon Max 9480 2P, HBM Caching260034743512OpenBenchmarking.orgMegahertz, More Is BetterTensorFlow 2.12CPU Peak Freq (Highest CPU Core Frequency) Monitor10002000300040005000

TensorFlow

System Power Consumption Monitor

OpenBenchmarking.orgWatts, Fewer Is BetterTensorFlow 2.12System Power Consumption MonitorXeon Max 9480 2P, HBM Caching160320480640800Min: 275 / Avg: 740.43 / Max: 907

TensorFlow

CPU Peak Freq (Highest CPU Core Frequency) Monitor

OpenBenchmarking.orgMegahertz, More Is BetterTensorFlow 2.12CPU Peak Freq (Highest CPU Core Frequency) MonitorXeon Max 9480 2P, HBM Caching6001200180024003000Min: 1671 / Avg: 3334.65 / Max: 3571

TensorFlow

System Power Consumption Monitor

MinAvgMaxXeon Max 9480 2P, HBM Caching408636785OpenBenchmarking.orgWatts, Fewer Is BetterTensorFlow 2.12System Power Consumption Monitor2004006008001000

TensorFlow

CPU Peak Freq (Highest CPU Core Frequency) Monitor

MinAvgMaxXeon Max 9480 2P, HBM Caching260033063511OpenBenchmarking.orgMegahertz, More Is BetterTensorFlow 2.12CPU Peak Freq (Highest CPU Core Frequency) Monitor10002000300040005000

TensorFlow

System Power Consumption Monitor

OpenBenchmarking.orgWatts, Fewer Is BetterTensorFlow 2.12System Power Consumption MonitorXeon Max 9480 2P, HBM Caching140280420560700Min: 410 / Avg: 755.75 / Max: 822

TensorFlow

CPU Peak Freq (Highest CPU Core Frequency) Monitor

OpenBenchmarking.orgMegahertz, More Is BetterTensorFlow 2.12CPU Peak Freq (Highest CPU Core Frequency) MonitorXeon Max 9480 2P, HBM Caching6001200180024003000Min: 2600 / Avg: 3476.25 / Max: 3555

oneDNN

System Power Consumption Monitor

MinAvgMaxXeon Max 9480 2P, HBM Caching402767901OpenBenchmarking.orgWatts, Fewer Is BetteroneDNN 3.1System Power Consumption Monitor2004006008001000

oneDNN

CPU Peak Freq (Highest CPU Core Frequency) Monitor

MinAvgMaxXeon Max 9480 2P, HBM Caching230027023507OpenBenchmarking.orgMegahertz, More Is BetteroneDNN 3.1CPU Peak Freq (Highest CPU Core Frequency) Monitor10002000300040005000

oneDNN

System Power Consumption Monitor

OpenBenchmarking.orgWatts, Fewer Is BetteroneDNN 3.1System Power Consumption MonitorXeon Max 9480 2P, HBM Caching160320480640800Min: 410 / Avg: 782.09 / Max: 889

oneDNN

CPU Peak Freq (Highest CPU Core Frequency) Monitor

OpenBenchmarking.orgMegahertz, More Is BetteroneDNN 3.1CPU Peak Freq (Highest CPU Core Frequency) MonitorXeon Max 9480 2P, HBM Caching6001200180024003000Min: 1951 / Avg: 2982.06 / Max: 3514

oneDNN

System Power Consumption Monitor

MinAvgMaxXeon Max 9480 2P, HBM Caching404592897OpenBenchmarking.orgWatts, Fewer Is BetteroneDNN 3.1System Power Consumption Monitor2004006008001000

oneDNN

CPU Peak Freq (Highest CPU Core Frequency) Monitor

MinAvgMaxXeon Max 9480 2P, HBM Caching190031073507OpenBenchmarking.orgMegahertz, More Is BetteroneDNN 3.1CPU Peak Freq (Highest CPU Core Frequency) Monitor10002000300040005000

oneDNN

System Power Consumption Monitor

MinAvgMaxXeon Max 9480 2P, HBM Caching404727850OpenBenchmarking.orgWatts, Fewer Is BetteroneDNN 3.1System Power Consumption Monitor2004006008001000

oneDNN

CPU Peak Freq (Highest CPU Core Frequency) Monitor

MinAvgMaxXeon Max 9480 2P, HBM Caching216626513508OpenBenchmarking.orgMegahertz, More Is BetteroneDNN 3.1CPU Peak Freq (Highest CPU Core Frequency) Monitor10002000300040005000

oneDNN

System Power Consumption Monitor

MinAvgMaxXeon Max 9480 2P, HBM Caching400642881OpenBenchmarking.orgWatts, Fewer Is BetteroneDNN 3.1System Power Consumption Monitor2004006008001000

oneDNN

CPU Peak Freq (Highest CPU Core Frequency) Monitor

MinAvgMaxXeon Max 9480 2P, HBM Caching209629513506OpenBenchmarking.orgMegahertz, More Is BetteroneDNN 3.1CPU Peak Freq (Highest CPU Core Frequency) Monitor10002000300040005000

oneDNN

System Power Consumption Monitor

MinAvgMaxXeon Max 9480 2P, HBM Caching400678830OpenBenchmarking.orgWatts, Fewer Is BetteroneDNN 3.1System Power Consumption Monitor2004006008001000

oneDNN

CPU Peak Freq (Highest CPU Core Frequency) Monitor

MinAvgMaxXeon Max 9480 2P, HBM Caching260030873508OpenBenchmarking.orgMegahertz, More Is BetteroneDNN 3.1CPU Peak Freq (Highest CPU Core Frequency) Monitor10002000300040005000

oneDNN

System Power Consumption Monitor

MinAvgMaxXeon Max 9480 2P, HBM Caching385721838OpenBenchmarking.orgWatts, Fewer Is BetteroneDNN 3.1System Power Consumption Monitor2004006008001000

oneDNN

CPU Peak Freq (Highest CPU Core Frequency) Monitor

MinAvgMaxXeon Max 9480 2P, HBM Caching260027973507OpenBenchmarking.orgMegahertz, More Is BetteroneDNN 3.1CPU Peak Freq (Highest CPU Core Frequency) Monitor10002000300040005000

libxsmm

System Power Consumption Monitor

OpenBenchmarking.orgWatts, Fewer Is Betterlibxsmm 2-1.17-3645System Power Consumption MonitorXeon Max 9480 2P, HBM Caching150300450600750Min: 395 / Avg: 776.46 / Max: 832

libxsmm

CPU Peak Freq (Highest CPU Core Frequency) Monitor

OpenBenchmarking.orgMegahertz, More Is Betterlibxsmm 2-1.17-3645CPU Peak Freq (Highest CPU Core Frequency) MonitorXeon Max 9480 2P, HBM Caching6001200180024003000Min: 1122 / Avg: 3487.45 / Max: 3507

CPU Power Consumption Monitor

Phoronix Test Suite System Monitoring

OpenBenchmarking.orgWattsCPU Power Consumption MonitorPhoronix Test Suite System MonitoringEPYC 9654 2PEPYC 9554 2PXeon Max 9480 2P, HBM Caching160320480640800Min: 31.88 / Avg: 448.81 / Max: 725.51Min: 34.39 / Avg: 413.82 / Max: 710.63Min: 87.33 / Avg: 548.37 / Max: 899.87

ONNX Runtime

CPU Power Consumption Monitor

OpenBenchmarking.orgWatts, Fewer Is BetterONNX Runtime 1.14CPU Power Consumption MonitorEPYC 9654 2PEPYC 9554 2PXeon Max 9480 2P, HBM Caching110220330440550Min: 31.88 / Avg: 465.24 / Max: 489.61Min: 47.64 / Avg: 368.58 / Max: 386.5Min: 270.52 / Avg: 569.27 / Max: 613.29

ONNX Runtime

Model: ResNet50 v1-12-int8 - Device: CPU - Executor: Standard

OpenBenchmarking.orgInference Time Cost (ms), Fewer Is BetterONNX Runtime 1.14Model: ResNet50 v1-12-int8 - Device: CPU - Executor: StandardEPYC 9654 2PEPYC 9554 2PXeon Max 9480 2P, HBM Caching246810SE +/- 0.15229, N = 15SE +/- 0.04787, N = 3SE +/- 0.15815, N = 156.870536.559898.021661. (CXX) g++ options: -ffunction-sections -fdata-sections -march=native -mtune=native -O3 -flto=auto -fno-fat-lto-objects -ldl -lrt

ONNX Runtime

Model: ResNet50 v1-12-int8 - Device: CPU - Executor: Standard

OpenBenchmarking.orgInferences Per Second, More Is BetterONNX Runtime 1.14Model: ResNet50 v1-12-int8 - Device: CPU - Executor: StandardEPYC 9654 2PEPYC 9554 2PXeon Max 9480 2P, HBM Caching306090120150SE +/- 3.31, N = 15SE +/- 1.11, N = 3SE +/- 2.29, N = 15146.56152.44125.281. (CXX) g++ options: -ffunction-sections -fdata-sections -march=native -mtune=native -O3 -flto=auto -fno-fat-lto-objects -ldl -lrt

ONNX Runtime

CPU Power Consumption Monitor

MinAvgMaxEPYC 9654 2P58.4464.9493.7EPYC 9554 2P47.3373.5394.1Xeon Max 9480 2P, HBM Caching261.0569.9613.0OpenBenchmarking.orgWatts, Fewer Is BetterONNX Runtime 1.14CPU Power Consumption Monitor160320480640800

ONNX Runtime

Model: ArcFace ResNet-100 - Device: CPU - Executor: Standard

OpenBenchmarking.orgInference Time Cost (ms), Fewer Is BetterONNX Runtime 1.14Model: ArcFace ResNet-100 - Device: CPU - Executor: StandardEPYC 9654 2PEPYC 9554 2PXeon Max 9480 2P, HBM Caching1224364860SE +/- 0.23, N = 3SE +/- 0.16, N = 3SE +/- 0.60, N = 1551.7941.6442.181. (CXX) g++ options: -ffunction-sections -fdata-sections -march=native -mtune=native -O3 -flto=auto -fno-fat-lto-objects -ldl -lrt

ONNX Runtime

CPU Power Consumption Monitor

MinAvgMaxEPYC 9654 2P57514546EPYC 9554 2P47420447Xeon Max 9480 2P, HBM Caching244569665OpenBenchmarking.orgWatts, Fewer Is BetterONNX Runtime 1.14CPU Power Consumption Monitor2004006008001000

ONNX Runtime

Model: fcn-resnet101-11 - Device: CPU - Executor: Standard

OpenBenchmarking.orgInference Time Cost (ms), Fewer Is BetterONNX Runtime 1.14Model: fcn-resnet101-11 - Device: CPU - Executor: StandardEPYC 9654 2PEPYC 9554 2PXeon Max 9480 2P, HBM Caching50100150200250SE +/- 1.01, N = 3SE +/- 1.01, N = 3SE +/- 8.54, N = 15212.86214.61191.621. (CXX) g++ options: -ffunction-sections -fdata-sections -march=native -mtune=native -O3 -flto=auto -fno-fat-lto-objects -ldl -lrt

ONNX Runtime

Model: fcn-resnet101-11 - Device: CPU - Executor: Standard

OpenBenchmarking.orgInferences Per Second, More Is BetterONNX Runtime 1.14Model: fcn-resnet101-11 - Device: CPU - Executor: StandardEPYC 9654 2PEPYC 9554 2PXeon Max 9480 2P, HBM Caching1.20042.40083.60124.80166.002SE +/- 0.02211, N = 3SE +/- 0.02211, N = 3SE +/- 0.18914, N = 154.697974.659825.335141. (CXX) g++ options: -ffunction-sections -fdata-sections -march=native -mtune=native -O3 -flto=auto -fno-fat-lto-objects -ldl -lrt

ONNX Runtime

CPU Power Consumption Monitor

OpenBenchmarking.orgWatts, Fewer Is BetterONNX Runtime 1.14CPU Power Consumption MonitorEPYC 9654 2PEPYC 9554 2PXeon Max 9480 2P, HBM Caching110220330440550Min: 56.74 / Avg: 454.2 / Max: 477.67Min: 47.45 / Avg: 357.73 / Max: 374.81Min: 261.61 / Avg: 573.75 / Max: 607.12

ONNX Runtime

Model: CaffeNet 12-int8 - Device: CPU - Executor: Standard

OpenBenchmarking.orgInference Time Cost (ms), Fewer Is BetterONNX Runtime 1.14Model: CaffeNet 12-int8 - Device: CPU - Executor: StandardEPYC 9654 2PEPYC 9554 2PXeon Max 9480 2P, HBM Caching0.51891.03781.55672.07562.5945SE +/- 0.03734, N = 15SE +/- 0.02472, N = 4SE +/- 0.02850, N = 152.306112.019491.782961. (CXX) g++ options: -ffunction-sections -fdata-sections -march=native -mtune=native -O3 -flto=auto -fno-fat-lto-objects -ldl -lrt

ONNX Runtime

Model: CaffeNet 12-int8 - Device: CPU - Executor: Standard

OpenBenchmarking.orgInferences Per Second, More Is BetterONNX Runtime 1.14Model: CaffeNet 12-int8 - Device: CPU - Executor: StandardEPYC 9654 2PEPYC 9554 2PXeon Max 9480 2P, HBM Caching120240360480600SE +/- 7.41, N = 15SE +/- 5.94, N = 4SE +/- 8.12, N = 15435.19495.24562.221. (CXX) g++ options: -ffunction-sections -fdata-sections -march=native -mtune=native -O3 -flto=auto -fno-fat-lto-objects -ldl -lrt

ONNX Runtime

CPU Power Consumption Monitor

OpenBenchmarking.orgWatts, Fewer Is BetterONNX Runtime 1.14CPU Power Consumption MonitorEPYC 9654 2PEPYC 9554 2PXeon Max 9480 2P, HBM Caching110220330440550Min: 57.25 / Avg: 394.67 / Max: 418.06Min: 47.62 / Avg: 368.65 / Max: 387.58Min: 264.43 / Avg: 572.92 / Max: 616.32

ONNX Runtime

Model: bertsquad-12 - Device: CPU - Executor: Standard

OpenBenchmarking.orgInference Time Cost (ms), Fewer Is BetterONNX Runtime 1.14Model: bertsquad-12 - Device: CPU - Executor: StandardEPYC 9654 2PEPYC 9554 2PXeon Max 9480 2P, HBM Caching20406080100SE +/- 2.35, N = 15SE +/- 1.46, N = 15SE +/- 1.73, N = 1599.0085.9087.761. (CXX) g++ options: -ffunction-sections -fdata-sections -march=native -mtune=native -O3 -flto=auto -fno-fat-lto-objects -ldl -lrt

ONNX Runtime

Model: bertsquad-12 - Device: CPU - Executor: Standard

OpenBenchmarking.orgInferences Per Second, More Is BetterONNX Runtime 1.14Model: bertsquad-12 - Device: CPU - Executor: StandardEPYC 9654 2PEPYC 9554 2PXeon Max 9480 2P, HBM Caching3691215SE +/- 0.24, N = 15SE +/- 0.19, N = 15SE +/- 0.23, N = 1510.1811.6911.461. (CXX) g++ options: -ffunction-sections -fdata-sections -march=native -mtune=native -O3 -flto=auto -fno-fat-lto-objects -ldl -lrt

ONNX Runtime

CPU Power Consumption Monitor

MinAvgMaxEPYC 9654 2P57456482EPYC 9554 2P47360380Xeon Max 9480 2P, HBM Caching262573626OpenBenchmarking.orgWatts, Fewer Is BetterONNX Runtime 1.14CPU Power Consumption Monitor2004006008001000

ONNX Runtime

Model: yolov4 - Device: CPU - Executor: Standard

OpenBenchmarking.orgInference Time Cost (ms), Fewer Is BetterONNX Runtime 1.14Model: yolov4 - Device: CPU - Executor: StandardEPYC 9654 2PEPYC 9554 2PXeon Max 9480 2P, HBM Caching4080120160200SE +/- 1.68, N = 3SE +/- 1.35, N = 3SE +/- 2.64, N = 12204.09195.90130.371. (CXX) g++ options: -ffunction-sections -fdata-sections -march=native -mtune=native -O3 -flto=auto -fno-fat-lto-objects -ldl -lrt

ONNX Runtime

Model: yolov4 - Device: CPU - Executor: Standard

OpenBenchmarking.orgInferences Per Second, More Is BetterONNX Runtime 1.14Model: yolov4 - Device: CPU - Executor: StandardEPYC 9654 2PEPYC 9554 2PXeon Max 9480 2P, HBM Caching246810SE +/- 0.04001, N = 3SE +/- 0.03523, N = 3SE +/- 0.13594, N = 124.900335.105057.700461. (CXX) g++ options: -ffunction-sections -fdata-sections -march=native -mtune=native -O3 -flto=auto -fno-fat-lto-objects -ldl -lrt

ONNX Runtime

CPU Power Consumption Monitor

MinAvgMaxEPYC 9654 2P58.7257.5270.6EPYC 9554 2P46.8222.0233.4Xeon Max 9480 2P, HBM Caching169.4521.4555.8OpenBenchmarking.orgWatts, Fewer Is BetterONNX Runtime 1.14CPU Power Consumption Monitor140280420560700

ONNX Runtime

Model: GPT-2 - Device: CPU - Executor: Standard

OpenBenchmarking.orgInference Time Cost (ms), Fewer Is BetterONNX Runtime 1.14Model: GPT-2 - Device: CPU - Executor: StandardEPYC 9654 2PEPYC 9554 2PXeon Max 9480 2P, HBM Caching3691215SE +/- 0.00311, N = 3SE +/- 0.25707, N = 15SE +/- 0.23765, N = 159.689047.647354.708491. (CXX) g++ options: -ffunction-sections -fdata-sections -march=native -mtune=native -O3 -flto=auto -fno-fat-lto-objects -ldl -lrt

ONNX Runtime

Model: GPT-2 - Device: CPU - Executor: Standard

OpenBenchmarking.orgInferences Per Second, More Is BetterONNX Runtime 1.14Model: GPT-2 - Device: CPU - Executor: StandardEPYC 9654 2PEPYC 9554 2PXeon Max 9480 2P, HBM Caching50100150200250SE +/- 0.03, N = 3SE +/- 4.25, N = 15SE +/- 9.23, N = 15103.18132.72218.691. (CXX) g++ options: -ffunction-sections -fdata-sections -march=native -mtune=native -O3 -flto=auto -fno-fat-lto-objects -ldl -lrt

OpenVINO

CPU Power Consumption Monitor

MinAvgMaxEPYC 9654 2P59595651EPYC 9554 2P48610665Xeon Max 9480 2P, HBM Caching164545651OpenBenchmarking.orgWatts, Fewer Is BetterOpenVINO 2022.3CPU Power Consumption Monitor2004006008001000

OpenVINO

Model: Age Gender Recognition Retail 0013 FP16-INT8 - Device: CPU

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2022.3Model: Age Gender Recognition Retail 0013 FP16-INT8 - Device: CPUEPYC 9654 2PEPYC 9554 2PXeon Max 9480 2P, HBM Caching0.3150.630.9451.261.575SE +/- 0.00, N = 3SE +/- 0.00, N = 3SE +/- 0.04, N = 120.990.881.40MIN: 0.85 / MAX: 37.79MIN: 0.83 / MAX: 24.85MIN: 0.7 / MAX: 123.261. (CXX) g++ options: -isystem -fsigned-char -ffunction-sections -fdata-sections -msse4.1 -msse4.2 -O3 -fno-strict-overflow -fwrapv -fPIC -fvisibility=hidden -Os -std=c++11 -MD -MT -MF

OpenVINO

Model: Age Gender Recognition Retail 0013 FP16-INT8 - Device: CPU

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2022.3Model: Age Gender Recognition Retail 0013 FP16-INT8 - Device: CPUEPYC 9654 2PEPYC 9554 2PXeon Max 9480 2P, HBM Caching30K60K90K120K150KSE +/- 611.95, N = 3SE +/- 574.97, N = 3SE +/- 1694.26, N = 12124618.32128593.2459457.141. (CXX) g++ options: -isystem -fsigned-char -ffunction-sections -fdata-sections -msse4.1 -msse4.2 -O3 -fno-strict-overflow -fwrapv -fPIC -fvisibility=hidden -Os -std=c++11 -MD -MT -MF

OpenVINO

CPU Power Consumption Monitor

MinAvgMaxEPYC 9654 2P62581643EPYC 9554 2P49580637Xeon Max 9480 2P, HBM Caching265570619OpenBenchmarking.orgWatts, Fewer Is BetterOpenVINO 2022.3CPU Power Consumption Monitor2004006008001000

OpenVINO

CPU Power Consumption Monitor

MinAvgMaxEPYC 9654 2P59599664EPYC 9554 2P49586647Xeon Max 9480 2P, HBM Caching176531642OpenBenchmarking.orgWatts, Fewer Is BetterOpenVINO 2022.3CPU Power Consumption Monitor2004006008001000

OpenVINO

CPU Power Consumption Monitor

MinAvgMaxEPYC 9654 2P59632685EPYC 9554 2P49623675Xeon Max 9480 2P, HBM Caching137552614OpenBenchmarking.orgWatts, Fewer Is BetterOpenVINO 2022.3CPU Power Consumption Monitor2004006008001000

OpenVINO

CPU Power Consumption Monitor

MinAvgMaxEPYC 9654 2P59642706EPYC 9554 2P51622683Xeon Max 9480 2P, HBM Caching165554623OpenBenchmarking.orgWatts, Fewer Is BetterOpenVINO 2022.3CPU Power Consumption Monitor2004006008001000

OpenVINO

CPU Power Consumption Monitor

MinAvgMaxEPYC 9654 2P57655705EPYC 9554 2P48648698Xeon Max 9480 2P, HBM Caching163557629OpenBenchmarking.orgWatts, Fewer Is BetterOpenVINO 2022.3CPU Power Consumption Monitor2004006008001000

OpenVINO

CPU Power Consumption Monitor

MinAvgMaxEPYC 9654 2P58600646EPYC 9554 2P49590636Xeon Max 9480 2P, HBM Caching170552614OpenBenchmarking.orgWatts, Fewer Is BetterOpenVINO 2022.3CPU Power Consumption Monitor2004006008001000

OpenVINO

CPU Power Consumption Monitor

MinAvgMaxEPYC 9654 2P59613687EPYC 9554 2P50606675Xeon Max 9480 2P, HBM Caching168531616OpenBenchmarking.orgWatts, Fewer Is BetterOpenVINO 2022.3CPU Power Consumption Monitor2004006008001000

OpenVINO

CPU Power Consumption Monitor

MinAvgMaxEPYC 9654 2P58636689EPYC 9554 2P50629675Xeon Max 9480 2P, HBM Caching87548657OpenBenchmarking.orgWatts, Fewer Is BetterOpenVINO 2022.3CPU Power Consumption Monitor2004006008001000

OpenVINO

CPU Power Consumption Monitor

MinAvgMaxEPYC 9654 2P60606703EPYC 9554 2P49600684Xeon Max 9480 2P, HBM Caching228547624OpenBenchmarking.orgWatts, Fewer Is BetterOpenVINO 2022.3CPU Power Consumption Monitor2004006008001000

OpenVINO

CPU Power Consumption Monitor

MinAvgMaxEPYC 9654 2P60608699EPYC 9554 2P49596683Xeon Max 9480 2P, HBM Caching204538618OpenBenchmarking.orgWatts, Fewer Is BetterOpenVINO 2022.3CPU Power Consumption Monitor2004006008001000

OpenVINO

CPU Power Consumption Monitor

MinAvgMaxEPYC 9654 2P58646726EPYC 9554 2P49638711Xeon Max 9480 2P, HBM Caching136533628OpenBenchmarking.orgWatts, Fewer Is BetterOpenVINO 2022.3CPU Power Consumption Monitor2004006008001000

TensorFlow

CPU Power Consumption Monitor

MinAvgMaxEPYC 9654 2P58503562EPYC 9554 2P49486557Xeon Max 9480 2P, HBM Caching231565721OpenBenchmarking.orgWatts, Fewer Is BetterTensorFlow 2.12CPU Power Consumption Monitor2004006008001000

TensorFlow

Device: CPU - Batch Size: 512 - Model: ResNet-50

OpenBenchmarking.orgimages/sec Per Watt, More Is BetterTensorFlow 2.12Device: CPU - Batch Size: 512 - Model: ResNet-50EPYC 9654 2PEPYC 9554 2PXeon Max 9480 2P, HBM Caching0.08710.17420.26130.34840.43550.3410.3870.136

TensorFlow

CPU Power Consumption Monitor

MinAvgMaxEPYC 9654 2P56525568EPYC 9554 2P47490537Xeon Max 9480 2P, HBM Caching275562692OpenBenchmarking.orgWatts, Fewer Is BetterTensorFlow 2.12CPU Power Consumption Monitor2004006008001000

TensorFlow

Device: CPU - Batch Size: 512 - Model: GoogLeNet

OpenBenchmarking.orgimages/sec Per Watt, More Is BetterTensorFlow 2.12Device: CPU - Batch Size: 512 - Model: GoogLeNetEPYC 9654 2PEPYC 9554 2PXeon Max 9480 2P, HBM Caching0.27250.5450.81751.091.36251.0201.2110.421

TensorFlow

CPU Power Consumption Monitor

MinAvgMaxEPYC 9654 2P57.2327.2393.8EPYC 9554 2P46.8291.1331.1Xeon Max 9480 2P, HBM Caching276.7539.1593.1OpenBenchmarking.orgWatts, Fewer Is BetterTensorFlow 2.12CPU Power Consumption Monitor160320480640800

TensorFlow

Device: CPU - Batch Size: 16 - Model: ResNet-50

OpenBenchmarking.orgimages/sec Per Watt, More Is BetterTensorFlow 2.12Device: CPU - Batch Size: 16 - Model: ResNet-50EPYC 9654 2PEPYC 9554 2PXeon Max 9480 2P, HBM Caching0.02590.05180.07770.10360.12950.0770.1150.074

TensorFlow

CPU Power Consumption Monitor

MinAvgMaxEPYC 9654 2P57.3328.9406.8EPYC 9554 2P47.7281.8351.3Xeon Max 9480 2P, HBM Caching265.6510.5597.2OpenBenchmarking.orgWatts, Fewer Is BetterTensorFlow 2.12CPU Power Consumption Monitor160320480640800

TensorFlow

Device: CPU - Batch Size: 16 - Model: GoogLeNet

OpenBenchmarking.orgimages/sec Per Watt, More Is BetterTensorFlow 2.12Device: CPU - Batch Size: 16 - Model: GoogLeNetEPYC 9654 2PEPYC 9554 2PXeon Max 9480 2P, HBM Caching0.07540.15080.22620.30160.3770.1960.3350.193

TensorFlow

Device: CPU - Batch Size: 16 - Model: GoogLeNet

OpenBenchmarking.orgimages/sec, More Is BetterTensorFlow 2.12Device: CPU - Batch Size: 16 - Model: GoogLeNetEPYC 9654 2PEPYC 9554 2PXeon Max 9480 2P, HBM Caching20406080100SE +/- 1.21, N = 15SE +/- 0.91, N = 15SE +/- 0.98, N = 364.4994.4198.43

TensorFlow

CPU Power Consumption Monitor

MinAvgMaxEPYC 9654 2P58.3456.4541.6EPYC 9554 2P48.8462.3567.3Xeon Max 9480 2P, HBM Caching209.1550.7614.5OpenBenchmarking.orgWatts, Fewer Is BetterTensorFlow 2.12CPU Power Consumption Monitor160320480640800

TensorFlow

Device: CPU - Batch Size: 512 - Model: AlexNet

OpenBenchmarking.orgimages/sec Per Watt, More Is BetterTensorFlow 2.12Device: CPU - Batch Size: 512 - Model: AlexNetEPYC 9654 2PEPYC 9554 2PXeon Max 9480 2P, HBM Caching0.88271.76542.64813.53084.41353.8503.9231.305

TensorFlow

CPU Power Consumption Monitor

OpenBenchmarking.orgWatts, Fewer Is BetterTensorFlow 2.12CPU Power Consumption MonitorEPYC 9654 2PEPYC 9554 2PXeon Max 9480 2P, HBM Caching160320480640800Min: 95.79 / Avg: 616.66 / Max: 655.38Min: 47.29 / Avg: 597.07 / Max: 647.99Min: 163.29 / Avg: 539.75 / Max: 899.87

TensorFlow

Device: CPU - Batch Size: 512 - Model: VGG-16

OpenBenchmarking.orgimages/sec Per Watt, More Is BetterTensorFlow 2.12Device: CPU - Batch Size: 512 - Model: VGG-16EPYC 9654 2PEPYC 9554 2PXeon Max 9480 2P, HBM Caching0.04410.08820.13230.17640.22050.1960.1840.056

TensorFlow

CPU Power Consumption Monitor

MinAvgMaxEPYC 9654 2P91.1320.9424.6EPYC 9554 2P47.1239.6360.5Xeon Max 9480 2P, HBM Caching260.2466.3587.5OpenBenchmarking.orgWatts, Fewer Is BetterTensorFlow 2.12CPU Power Consumption Monitor160320480640800

TensorFlow

Device: CPU - Batch Size: 16 - Model: AlexNet

OpenBenchmarking.orgimages/sec Per Watt, More Is BetterTensorFlow 2.12Device: CPU - Batch Size: 16 - Model: AlexNetEPYC 9654 2PEPYC 9554 2PXeon Max 9480 2P, HBM Caching0.24210.48420.72630.96841.21050.5181.0760.493

TensorFlow

Device: CPU - Batch Size: 16 - Model: AlexNet

OpenBenchmarking.orgimages/sec, More Is BetterTensorFlow 2.12Device: CPU - Batch Size: 16 - Model: AlexNetEPYC 9654 2PEPYC 9554 2PXeon Max 9480 2P, HBM Caching60120180240300SE +/- 2.70, N = 15SE +/- 3.19, N = 15SE +/- 1.10, N = 5166.20257.94230.11

TensorFlow

CPU Power Consumption Monitor

MinAvgMaxEPYC 9654 2P104427498EPYC 9554 2P47426517Xeon Max 9480 2P, HBM Caching265554662OpenBenchmarking.orgWatts, Fewer Is BetterTensorFlow 2.12CPU Power Consumption Monitor2004006008001000

TensorFlow

Device: CPU - Batch Size: 16 - Model: VGG-16

OpenBenchmarking.orgimages/sec Per Watt, More Is BetterTensorFlow 2.12Device: CPU - Batch Size: 16 - Model: VGG-16EPYC 9654 2PEPYC 9554 2PXeon Max 9480 2P, HBM Caching0.03170.06340.09510.12680.15850.1110.1410.038

TensorFlow

Device: CPU - Batch Size: 16 - Model: VGG-16

OpenBenchmarking.orgimages/sec, More Is BetterTensorFlow 2.12Device: CPU - Batch Size: 16 - Model: VGG-16EPYC 9654 2PEPYC 9554 2PXeon Max 9480 2P, HBM Caching1326395265SE +/- 0.39, N = 3SE +/- 0.59, N = 3SE +/- 0.61, N = 1547.2760.3121.09

oneDNN

CPU Power Consumption Monitor

OpenBenchmarking.orgWatts, Fewer Is BetteroneDNN 3.1CPU Power Consumption MonitorEPYC 9654 2PEPYC 9554 2PXeon Max 9480 2P, HBM Caching110220330440550Min: 95.79 / Avg: 412.4 / Max: 555.18Min: 47.41 / Avg: 356.12 / Max: 602.46Min: 259.4 / Avg: 561.14 / Max: 643.82

oneDNN

CPU Power Consumption Monitor

MinAvgMaxEPYC 9654 2P104414522EPYC 9554 2P47365577Xeon Max 9480 2P, HBM Caching266573647OpenBenchmarking.orgWatts, Fewer Is BetteroneDNN 3.1CPU Power Consumption Monitor2004006008001000

oneDNN

Harness: Recurrent Neural Network Training - Data Type: bf16bf16bf16 - Engine: CPU

OpenBenchmarking.orgms, Fewer Is BetteroneDNN 3.1Harness: Recurrent Neural Network Training - Data Type: bf16bf16bf16 - Engine: CPUEPYC 9654 2PEPYC 9554 2PXeon Max 9480 2P, HBM Caching3K6K9K12K15KSE +/- 18.00, N = 3SE +/- 13.50, N = 5SE +/- 507.18, N = 102492.691305.6615294.60MIN: 7750.751. (CXX) g++ options: -O3 -march=native -fopenmp -msse4.1 -fPIC -pie -ldl -lpthread

oneDNN

CPU Power Consumption Monitor

MinAvgMaxEPYC 9654 2P93286525EPYC 9554 2P47270572Xeon Max 9480 2P, HBM Caching268426673OpenBenchmarking.orgWatts, Fewer Is BetteroneDNN 3.1CPU Power Consumption Monitor2004006008001000

oneDNN

CPU Power Consumption Monitor

MinAvgMaxEPYC 9654 2P106424558EPYC 9554 2P47388564Xeon Max 9480 2P, HBM Caching138534633OpenBenchmarking.orgWatts, Fewer Is BetteroneDNN 3.1CPU Power Consumption Monitor2004006008001000

oneDNN

Harness: Deconvolution Batch shapes_1d - Data Type: bf16bf16bf16 - Engine: CPU

OpenBenchmarking.orgms, Fewer Is BetteroneDNN 3.1Harness: Deconvolution Batch shapes_1d - Data Type: bf16bf16bf16 - Engine: CPUEPYC 9654 2PEPYC 9554 2PXeon Max 9480 2P, HBM Caching0.571.141.712.282.85SE +/- 0.004515, N = 3SE +/- 0.003878, N = 3SE +/- 0.009112, N = 152.5332601.7616200.483075MIN: 1.7MIN: 1.591. (CXX) g++ options: -O3 -march=native -fopenmp -msse4.1 -fPIC -pie -ldl -lpthread

oneDNN

CPU Power Consumption Monitor

MinAvgMaxEPYC 9654 2P87337534EPYC 9554 2P47325610Xeon Max 9480 2P, HBM Caching267471667OpenBenchmarking.orgWatts, Fewer Is BetteroneDNN 3.1CPU Power Consumption Monitor2004006008001000

oneDNN

CPU Power Consumption Monitor

MinAvgMaxEPYC 9654 2P89333497EPYC 9554 2P47256402Xeon Max 9480 2P, HBM Caching260498639OpenBenchmarking.orgWatts, Fewer Is BetteroneDNN 3.1CPU Power Consumption Monitor2004006008001000

oneDNN

CPU Power Consumption Monitor

MinAvgMaxEPYC 9654 2P90348491EPYC 9554 2P47267375Xeon Max 9480 2P, HBM Caching260530626OpenBenchmarking.orgWatts, Fewer Is BetteroneDNN 3.1CPU Power Consumption Monitor2004006008001000

oneDNN

Harness: IP Shapes 1D - Data Type: bf16bf16bf16 - Engine: CPU

OpenBenchmarking.orgms, Fewer Is BetteroneDNN 3.1Harness: IP Shapes 1D - Data Type: bf16bf16bf16 - Engine: CPUEPYC 9654 2PEPYC 9554 2PXeon Max 9480 2P, HBM Caching48121620SE +/- 0.81790, N = 15SE +/- 0.95058, N = 12SE +/- 0.22497, N = 1516.9823011.196697.88016MIN: 8.09MIN: 5.64MIN: 4.451. (CXX) g++ options: -O3 -march=native -fopenmp -msse4.1 -fPIC -pie -ldl -lpthread

libxsmm

CPU Power Consumption Monitor

MinAvgMaxEPYC 9654 2P98380613EPYC 9554 2P47344635OpenBenchmarking.orgWatts, Fewer Is Betterlibxsmm 2-1.17-3645CPU Power Consumption Monitor2004006008001000

libxsmm

M N K: 64

OpenBenchmarking.orgGFLOPS/s Per Watt, More Is Betterlibxsmm 2-1.17-3645M N K: 64EPYC 9654 2PEPYC 9554 2P2468105.9757.839

libxsmm

CPU Power Consumption Monitor

MinAvgMaxEPYC 9654 2P94.5359.7590.3EPYC 9554 2P46.9293.2543.9OpenBenchmarking.orgWatts, Fewer Is Betterlibxsmm 2-1.17-3645CPU Power Consumption Monitor160320480640800

libxsmm

M N K: 32

OpenBenchmarking.orgGFLOPS/s Per Watt, More Is Betterlibxsmm 2-1.17-3645M N K: 32EPYC 9654 2PEPYC 9554 2P1.0762.1523.2284.3045.383.6094.782

libxsmm

CPU Power Consumption Monitor

OpenBenchmarking.orgWatts, Fewer Is Betterlibxsmm 2-1.17-3645CPU Power Consumption MonitorEPYC 9654 2PEPYC 9554 2P120240360480600Min: 89.89 / Avg: 434.8 / Max: 664.56Min: 47.99 / Avg: 394.51 / Max: 694.95

libxsmm

M N K: 256

OpenBenchmarking.orgGFLOPS/s Per Watt, More Is Betterlibxsmm 2-1.17-3645M N K: 256EPYC 9654 2PEPYC 9554 2P369121510.0711.67

libxsmm

CPU Power Consumption Monitor

OpenBenchmarking.orgWatts, Fewer Is Betterlibxsmm 2-1.17-3645CPU Power Consumption MonitorEPYC 9654 2PEPYC 9554 2PXeon Max 9480 2P, HBM Caching120240360480600Min: 99.14 / Avg: 441.18 / Max: 632.27Min: 47.03 / Avg: 399.88 / Max: 668.24Min: 256.12 / Avg: 565.34 / Max: 625.18

libxsmm

M N K: 128

OpenBenchmarking.orgGFLOPS/s Per Watt, More Is Betterlibxsmm 2-1.17-3645M N K: 128EPYC 9654 2PEPYC 9554 2PXeon Max 9480 2P, HBM Caching36912157.76610.2853.209

Phoronix Test Suite v10.8.5