Benchmarks by Michael Larabel for a future article.
Compare your own system(s) to this result file with the
Phoronix Test Suite by running the command:
phoronix-test-suite benchmark 2301166-NE-OPENVINOA64 OpenVINO AMX Benchmarks AMD EPYC Genoa - Phoronix Test Suite OpenVINO AMX Benchmarks AMD EPYC Genoa Benchmarks by Michael Larabel for a future article.
HTML result view exported from: https://openbenchmarking.org/result/2301166-NE-OPENVINOA64&sor&gru&export=pdf .
OpenVINO AMX Benchmarks AMD EPYC Genoa Processor Motherboard Chipset Memory Disk Graphics Monitor Network OS Kernel Desktop Display Server Vulkan Compiler File-System Screen Resolution Xeon Platinum 8490H 2P Xeon Platinum 8490H 2P - No AMX EPYC 9654 2P 2 x Intel Xeon Platinum 8490H @ 3.50GHz (120 Cores / 240 Threads) Quanta Cloud S6Q-MB-MPS (3A10.uh BIOS) Intel Device 1bce 16 x 64 GB 4800MT/s Samsung M321R8GA0BB0-CQKEG 2 x 1920GB SAMSUNG MZWLJ1T9HBJR-00007 + 960GB INTEL SSDSC2KG96 ASPEED VGA HDMI 4 x Intel E810-C for QSFP + 2 x Intel X710 for 10GBASE-T Ubuntu 22.04 6.1.4-060104-generic (x86_64) GNOME Shell 42.2 X Server 1.21.1.3 1.2.204 GCC 11.3.0 + Clang 14.0.0-1ubuntu1 ext4 1920x1080 2 x AMD EPYC 9654 96-Core @ 2.40GHz (192 Cores / 384 Threads) AMD Titanite_4G (RTI1002E BIOS) AMD Device 14a4 24 x 64 GB 4800MT/s Samsung M321R8GA0BB0-CQKEG 2 x 1920GB SAMSUNG MZWLJ1T9HBJR-00007 Broadcom NetXtreme BCM5720 PCIe OpenBenchmarking.org Kernel Details - Transparent Huge Pages: madvise Compiler Details - --build=x86_64-linux-gnu --disable-vtable-verify --disable-werror --enable-bootstrap --enable-cet --enable-checking=release --enable-clocale=gnu --enable-default-pie --enable-gnu-unique-object --enable-languages=c,ada,c++,go,brig,d,fortran,objc,obj-c++,m2 --enable-libphobos-checking=release --enable-libstdcxx-debug --enable-libstdcxx-time=yes --enable-link-serialization=2 --enable-multiarch --enable-multilib --enable-nls --enable-objc-gc=auto --enable-offload-targets=nvptx-none=/build/gcc-11-xKiWfi/gcc-11-11.3.0/debian/tmp-nvptx/usr,amdgcn-amdhsa=/build/gcc-11-xKiWfi/gcc-11-11.3.0/debian/tmp-gcn/usr --enable-plugin --enable-shared --enable-threads=posix --host=x86_64-linux-gnu --program-prefix=x86_64-linux-gnu- --target=x86_64-linux-gnu --with-abi=m64 --with-arch-32=i686 --with-build-config=bootstrap-lto-lean --with-default-libstdcxx-abi=new --with-gcc-major-version-only --with-multilib-list=m32,m64,mx32 --with-target-system-zlib=auto --with-tune=generic --without-cuda-driver -v Processor Details - Xeon Platinum 8490H 2P: Scaling Governor: intel_pstate performance (EPP: performance) - CPU Microcode: 0x2b0000c0 - Xeon Platinum 8490H 2P - No AMX: Scaling Governor: intel_pstate performance (EPP: performance) - CPU Microcode: 0x2b0000c0 - EPYC 9654 2P: Scaling Governor: acpi-cpufreq performance (Boost: Enabled) - CPU Microcode: 0xa10110d Python Details - Python 3.10.6 Security Details - Xeon Platinum 8490H 2P: itlb_multihit: Not affected + l1tf: Not affected + mds: Not affected + meltdown: Not affected + mmio_stale_data: Not affected + retbleed: Not affected + spec_store_bypass: Mitigation of SSB disabled via prctl + spectre_v1: Mitigation of usercopy/swapgs barriers and __user pointer sanitization + spectre_v2: Mitigation of Enhanced IBRS IBPB: conditional RSB filling PBRSB-eIBRS: SW sequence + srbds: Not affected + tsx_async_abort: Not affected - Xeon Platinum 8490H 2P - No AMX: itlb_multihit: Not affected + l1tf: Not affected + mds: Not affected + meltdown: Not affected + mmio_stale_data: Not affected + retbleed: Not affected + spec_store_bypass: Mitigation of SSB disabled via prctl + spectre_v1: Mitigation of usercopy/swapgs barriers and __user pointer sanitization + spectre_v2: Mitigation of Enhanced IBRS IBPB: conditional RSB filling PBRSB-eIBRS: SW sequence + srbds: Not affected + tsx_async_abort: Not affected - EPYC 9654 2P: itlb_multihit: Not affected + l1tf: Not affected + mds: Not affected + meltdown: Not affected + mmio_stale_data: Not affected + retbleed: Not affected + spec_store_bypass: Mitigation of SSB disabled via prctl + spectre_v1: Mitigation of usercopy/swapgs barriers and __user pointer sanitization + spectre_v2: Mitigation of Retpolines IBPB: conditional IBRS_FW STIBP: always-on RSB filling PBRSB-eIBRS: Not affected + srbds: Not affected + tsx_async_abort: Not affected
OpenVINO AMX Benchmarks AMD EPYC Genoa openvino: Face Detection FP16-INT8 - CPU openvino: Weld Porosity Detection FP16-INT8 - CPU openvino: Person Vehicle Bike Detection FP16 - CPU openvino: Machine Translation EN To DE FP16 - CPU openvino: Face Detection FP16-INT8 - CPU openvino: Weld Porosity Detection FP16-INT8 - CPU openvino: Person Vehicle Bike Detection FP16 - CPU openvino: Machine Translation EN To DE FP16 - CPU Xeon Platinum 8490H 2P Xeon Platinum 8490H 2P - No AMX EPYC 9654 2P 282.25 23066.63 4778.34 724.10 424.16 5.18 25.09 82.79 146.08 8897.10 2730.07 321.28 409.81 13.46 21.95 186.53 188.64 18678.16 5320.83 989.16 507.25 10.26 18.03 97.00 OpenBenchmarking.org
CPU Peak Freq (Highest CPU Core Frequency) Monitor Phoronix Test Suite System Monitoring OpenBenchmarking.org Megahertz CPU Peak Freq (Highest CPU Core Frequency) Monitor Phoronix Test Suite System Monitoring Xeon Platinum 8490H 2P - No AMX Xeon Platinum 8490H 2P EPYC 9654 2P 700 1400 2100 2800 3500 Min: 1000 / Avg: 2603.67 / Max: 3512 Min: 1900 / Avg: 2744.33 / Max: 3516 Min: 2400 / Avg: 3129.46 / Max: 3763
CPU Power Consumption Monitor Phoronix Test Suite System Monitoring OpenBenchmarking.org Watts CPU Power Consumption Monitor Phoronix Test Suite System Monitoring EPYC 9654 2P Xeon Platinum 8490H 2P Xeon Platinum 8490H 2P - No AMX 130 260 390 520 650 Min: 53.42 / Avg: 596.25 / Max: 739.81 Min: 208.07 / Avg: 624.35 / Max: 760.79 Min: 122.95 / Avg: 646.09 / Max: 763.9
OpenVINO Model: Face Detection FP16-INT8 - Device: CPU OpenBenchmarking.org FPS, More Is Better OpenVINO 2022.3 Model: Face Detection FP16-INT8 - Device: CPU Xeon Platinum 8490H 2P EPYC 9654 2P Xeon Platinum 8490H 2P - No AMX 60 120 180 240 300 SE +/- 2.56, N = 3 SE +/- 0.17, N = 3 SE +/- 0.29, N = 3 282.25 188.64 146.08 1. (CXX) g++ options: -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -pie -ldl
OpenVINO Model: Weld Porosity Detection FP16-INT8 - Device: CPU OpenBenchmarking.org FPS, More Is Better OpenVINO 2022.3 Model: Weld Porosity Detection FP16-INT8 - Device: CPU Xeon Platinum 8490H 2P EPYC 9654 2P Xeon Platinum 8490H 2P - No AMX 5K 10K 15K 20K 25K SE +/- 92.31, N = 3 SE +/- 10.84, N = 3 SE +/- 24.48, N = 3 23066.63 18678.16 8897.10 1. (CXX) g++ options: -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -pie -ldl
OpenVINO Model: Person Vehicle Bike Detection FP16 - Device: CPU OpenBenchmarking.org FPS, More Is Better OpenVINO 2022.3 Model: Person Vehicle Bike Detection FP16 - Device: CPU EPYC 9654 2P Xeon Platinum 8490H 2P Xeon Platinum 8490H 2P - No AMX 1100 2200 3300 4400 5500 SE +/- 44.61, N = 8 SE +/- 14.65, N = 3 SE +/- 11.17, N = 3 5320.83 4778.34 2730.07 1. (CXX) g++ options: -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -pie -ldl
OpenVINO Model: Machine Translation EN To DE FP16 - Device: CPU OpenBenchmarking.org FPS, More Is Better OpenVINO 2022.3 Model: Machine Translation EN To DE FP16 - Device: CPU EPYC 9654 2P Xeon Platinum 8490H 2P Xeon Platinum 8490H 2P - No AMX 200 400 600 800 1000 SE +/- 8.13, N = 8 SE +/- 5.21, N = 3 SE +/- 0.77, N = 3 989.16 724.10 321.28 1. (CXX) g++ options: -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -pie -ldl
OpenVINO CPU Peak Freq (Highest CPU Core Frequency) Monitor Min Avg Max EPYC 9654 2P 2400 2990 3702 Xeon Platinum 8490H 2P 1900 2561 3507 Xeon Platinum 8490H 2P - No AMX 1231 2374 3508 OpenBenchmarking.org Megahertz, More Is Better OpenVINO 2022.3 CPU Peak Freq (Highest CPU Core Frequency) Monitor 1000 2000 3000 4000 5000
OpenVINO CPU Peak Freq (Highest CPU Core Frequency) Monitor Min Avg Max Xeon Platinum 8490H 2P - No AMX 1900 2928 3512 Xeon Platinum 8490H 2P 1900 2860 3501 EPYC 9654 2P 2400 2809 3702 OpenBenchmarking.org Megahertz, More Is Better OpenVINO 2022.3 CPU Peak Freq (Highest CPU Core Frequency) Monitor 1000 2000 3000 4000 5000
OpenVINO CPU Peak Freq (Highest CPU Core Frequency) Monitor Min Avg Max EPYC 9654 2P 2400 3493 3763 Xeon Platinum 8490H 2P 1900 2965 3501 Xeon Platinum 8490H 2P - No AMX 1900 2739 3511 OpenBenchmarking.org Megahertz, More Is Better OpenVINO 2022.3 CPU Peak Freq (Highest CPU Core Frequency) Monitor 1000 2000 3000 4000 5000
OpenVINO CPU Peak Freq (Highest CPU Core Frequency) Monitor Min Avg Max EPYC 9654 2P 2400 2965 3700 Xeon Platinum 8490H 2P 1900 2792 3516 Xeon Platinum 8490H 2P - No AMX 1900 2656 3508 OpenBenchmarking.org Megahertz, More Is Better OpenVINO 2022.3 CPU Peak Freq (Highest CPU Core Frequency) Monitor 1000 2000 3000 4000 5000
OpenVINO Model: Face Detection FP16-INT8 - Device: CPU OpenBenchmarking.org ms, Fewer Is Better OpenVINO 2022.3 Model: Face Detection FP16-INT8 - Device: CPU Xeon Platinum 8490H 2P - No AMX Xeon Platinum 8490H 2P EPYC 9654 2P 110 220 330 440 550 SE +/- 0.81, N = 3 SE +/- 3.75, N = 3 SE +/- 0.31, N = 3 409.81 424.16 507.25 MIN: 203.37 / MAX: 909.92 MIN: 153.43 / MAX: 1952.95 MIN: 231.44 / MAX: 623.69 1. (CXX) g++ options: -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -pie -ldl
OpenVINO Model: Weld Porosity Detection FP16-INT8 - Device: CPU OpenBenchmarking.org ms, Fewer Is Better OpenVINO 2022.3 Model: Weld Porosity Detection FP16-INT8 - Device: CPU Xeon Platinum 8490H 2P EPYC 9654 2P Xeon Platinum 8490H 2P - No AMX 3 6 9 12 15 SE +/- 0.02, N = 3 SE +/- 0.01, N = 3 SE +/- 0.04, N = 3 5.18 10.26 13.46 MIN: 1.43 / MAX: 102.62 MIN: 4.21 / MAX: 95.33 MIN: 3.82 / MAX: 88.11 1. (CXX) g++ options: -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -pie -ldl
OpenVINO Model: Person Vehicle Bike Detection FP16 - Device: CPU OpenBenchmarking.org ms, Fewer Is Better OpenVINO 2022.3 Model: Person Vehicle Bike Detection FP16 - Device: CPU EPYC 9654 2P Xeon Platinum 8490H 2P - No AMX Xeon Platinum 8490H 2P 6 12 18 24 30 SE +/- 0.15, N = 8 SE +/- 0.09, N = 3 SE +/- 0.08, N = 3 18.03 21.95 25.09 MIN: 5.34 / MAX: 112.51 MIN: 11.62 / MAX: 120.04 MIN: 11.66 / MAX: 132.09 1. (CXX) g++ options: -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -pie -ldl
OpenVINO Model: Machine Translation EN To DE FP16 - Device: CPU OpenBenchmarking.org ms, Fewer Is Better OpenVINO 2022.3 Model: Machine Translation EN To DE FP16 - Device: CPU Xeon Platinum 8490H 2P EPYC 9654 2P Xeon Platinum 8490H 2P - No AMX 40 80 120 160 200 SE +/- 0.60, N = 3 SE +/- 0.78, N = 8 SE +/- 0.45, N = 3 82.79 97.00 186.53 MIN: 31.64 / MAX: 1588.97 MIN: 41.26 / MAX: 1899.38 MIN: 92.82 / MAX: 808.7 1. (CXX) g++ options: -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -pie -ldl
OpenVINO CPU Power Consumption Monitor Min Avg Max EPYC 9654 2P 54 602 717 Xeon Platinum 8490H 2P 208 616 761 Xeon Platinum 8490H 2P - No AMX 210 641 748 OpenBenchmarking.org Watts, Fewer Is Better OpenVINO 2022.3 CPU Power Consumption Monitor 200 400 600 800 1000
OpenVINO CPU Power Consumption Monitor Min Avg Max EPYC 9654 2P 59 652 717 Xeon Platinum 8490H 2P 217 663 757 Xeon Platinum 8490H 2P - No AMX 210 665 756 OpenBenchmarking.org Watts, Fewer Is Better OpenVINO 2022.3 CPU Power Consumption Monitor 200 400 600 800 1000
OpenVINO CPU Power Consumption Monitor Min Avg Max EPYC 9654 2P 55 567 643 Xeon Platinum 8490H 2P 213 640 755 Xeon Platinum 8490H 2P - No AMX 132 653 756 OpenBenchmarking.org Watts, Fewer Is Better OpenVINO 2022.3 CPU Power Consumption Monitor 200 400 600 800 1000
OpenVINO CPU Power Consumption Monitor Min Avg Max Xeon Platinum 8490H 2P 212 644 742 EPYC 9654 2P 55 650 740 Xeon Platinum 8490H 2P - No AMX 211 652 756 OpenBenchmarking.org Watts, Fewer Is Better OpenVINO 2022.3 CPU Power Consumption Monitor 200 400 600 800 1000
Phoronix Test Suite v10.8.4