Benchmarks by Michael Larabel for a future article.
Compare your own system(s) to this result file with the
Phoronix Test Suite by running the command:
phoronix-test-suite benchmark 2301166-NE-OPENVINOA64
OpenVINO AMX Benchmarks AMD EPYC Genoa
Benchmarks by Michael Larabel for a future article.
Xeon Platinum 8490H 2P:
Processor: 2 x Intel Xeon Platinum 8490H @ 3.50GHz (120 Cores / 240 Threads), Motherboard: Quanta Cloud S6Q-MB-MPS (3A10.uh BIOS), Chipset: Intel Device 1bce, Memory: 16 x 64 GB 4800MT/s Samsung M321R8GA0BB0-CQKEG, Disk: 2 x 1920GB SAMSUNG MZWLJ1T9HBJR-00007 + 960GB INTEL SSDSC2KG96, Graphics: ASPEED, Monitor: VGA HDMI, Network: 4 x Intel E810-C for QSFP + 2 x Intel X710 for 10GBASE-T
OS: Ubuntu 22.04, Kernel: 6.1.4-060104-generic (x86_64), Desktop: GNOME Shell 42.2, Display Server: X Server 1.21.1.3, Vulkan: 1.2.204, Compiler: GCC 11.3.0 + Clang 14.0.0-1ubuntu1, File-System: ext4, Screen Resolution: 1920x1080
Xeon Platinum 8490H 2P - No AMX:
Processor: 2 x Intel Xeon Platinum 8490H @ 3.50GHz (120 Cores / 240 Threads), Motherboard: Quanta Cloud S6Q-MB-MPS (3A10.uh BIOS), Chipset: Intel Device 1bce, Memory: 16 x 64 GB 4800MT/s Samsung M321R8GA0BB0-CQKEG, Disk: 2 x 1920GB SAMSUNG MZWLJ1T9HBJR-00007 + 960GB INTEL SSDSC2KG96, Graphics: ASPEED, Monitor: VGA HDMI, Network: 4 x Intel E810-C for QSFP + 2 x Intel X710 for 10GBASE-T
OS: Ubuntu 22.04, Kernel: 6.1.4-060104-generic (x86_64), Desktop: GNOME Shell 42.2, Display Server: X Server 1.21.1.3, Vulkan: 1.2.204, Compiler: GCC 11.3.0 + Clang 14.0.0-1ubuntu1, File-System: ext4, Screen Resolution: 1920x1080
EPYC 9654 2P:
Processor: 2 x AMD EPYC 9654 96-Core @ 2.40GHz (192 Cores / 384 Threads), Motherboard: AMD Titanite_4G (RTI1002E BIOS), Chipset: AMD Device 14a4, Memory: 24 x 64 GB 4800MT/s Samsung M321R8GA0BB0-CQKEG, Disk: 2 x 1920GB SAMSUNG MZWLJ1T9HBJR-00007, Graphics: ASPEED, Monitor: VGA HDMI, Network: Broadcom NetXtreme BCM5720 PCIe
OS: Ubuntu 22.04, Kernel: 6.1.4-060104-generic (x86_64), Desktop: GNOME Shell 42.2, Display Server: X Server 1.21.1.3, Vulkan: 1.2.204, Compiler: GCC 11.3.0 + Clang 14.0.0-1ubuntu1, File-System: ext4, Screen Resolution: 1920x1080
OpenVINO 2022.3
Model: Face Detection FP16-INT8 - Device: CPU
FPS > Higher Is Better
Xeon Platinum 8490H 2P .......... 282.25 |=====================================
Xeon Platinum 8490H 2P - No AMX . 146.08 |===================
EPYC 9654 2P .................... 188.64 |=========================
OpenVINO 2022.3
Model: Face Detection FP16-INT8 - Device: CPU
ms < Lower Is Better
Xeon Platinum 8490H 2P .......... 424.16 |===============================
Xeon Platinum 8490H 2P - No AMX . 409.81 |==============================
EPYC 9654 2P .................... 507.25 |=====================================
OpenVINO 2022.3
Model: Weld Porosity Detection FP16-INT8 - Device: CPU
FPS > Higher Is Better
Xeon Platinum 8490H 2P .......... 23066.63 |===================================
Xeon Platinum 8490H 2P - No AMX . 8897.10 |=============
EPYC 9654 2P .................... 18678.16 |============================
OpenVINO 2022.3
Model: Weld Porosity Detection FP16-INT8 - Device: CPU
ms < Lower Is Better
Xeon Platinum 8490H 2P .......... 5.18 |===============
Xeon Platinum 8490H 2P - No AMX . 13.46 |======================================
EPYC 9654 2P .................... 10.26 |=============================
OpenVINO 2022.3
Model: Person Vehicle Bike Detection FP16 - Device: CPU
FPS > Higher Is Better
Xeon Platinum 8490H 2P .......... 4778.34 |================================
Xeon Platinum 8490H 2P - No AMX . 2730.07 |==================
EPYC 9654 2P .................... 5320.83 |====================================
OpenVINO 2022.3
Model: Person Vehicle Bike Detection FP16 - Device: CPU
ms < Lower Is Better
Xeon Platinum 8490H 2P .......... 25.09 |======================================
Xeon Platinum 8490H 2P - No AMX . 21.95 |=================================
EPYC 9654 2P .................... 18.03 |===========================
OpenVINO 2022.3
Model: Machine Translation EN To DE FP16 - Device: CPU
FPS > Higher Is Better
Xeon Platinum 8490H 2P .......... 724.10 |===========================
Xeon Platinum 8490H 2P - No AMX . 321.28 |============
EPYC 9654 2P .................... 989.16 |=====================================
OpenVINO 2022.3
Model: Machine Translation EN To DE FP16 - Device: CPU
ms < Lower Is Better
Xeon Platinum 8490H 2P .......... 82.79 |================
Xeon Platinum 8490H 2P - No AMX . 186.53 |=====================================
EPYC 9654 2P .................... 97.00 |===================
OpenVINO 2022.3
CPU Peak Freq (Highest CPU Core Frequency) Monitor
Megahertz > Higher Is Better
Xeon Platinum 8490H 2P .......... MIN: 1900 AVG: 2561 MAX: 3507
Xeon Platinum 8490H 2P - No AMX . MIN: 1231 AVG: 2374 MAX: 3508
EPYC 9654 2P .................... MIN: 2400 AVG: 2990 MAX: 3702
OpenVINO 2022.3
CPU Power Consumption Monitor
Watts < Lower Is Better
Xeon Platinum 8490H 2P .......... MIN: 208 AVG: 616 MAX: 761
Xeon Platinum 8490H 2P - No AMX . MIN: 210 AVG: 641 MAX: 748
EPYC 9654 2P .................... MIN: 54 AVG: 602 MAX: 717
OpenVINO 2022.3
CPU Peak Freq (Highest CPU Core Frequency) Monitor
Megahertz > Higher Is Better
Xeon Platinum 8490H 2P .......... MIN: 1900 AVG: 2860 MAX: 3501
Xeon Platinum 8490H 2P - No AMX . MIN: 1900 AVG: 2928 MAX: 3512
EPYC 9654 2P .................... MIN: 2400 AVG: 2809 MAX: 3702
OpenVINO 2022.3
CPU Power Consumption Monitor
Watts < Lower Is Better
Xeon Platinum 8490H 2P .......... MIN: 217 AVG: 663 MAX: 757
Xeon Platinum 8490H 2P - No AMX . MIN: 210 AVG: 665 MAX: 756
EPYC 9654 2P .................... MIN: 59 AVG: 652 MAX: 717
OpenVINO 2022.3
CPU Peak Freq (Highest CPU Core Frequency) Monitor
Megahertz > Higher Is Better
Xeon Platinum 8490H 2P .......... MIN: 1900 AVG: 2965 MAX: 3501
Xeon Platinum 8490H 2P - No AMX . MIN: 1900 AVG: 2739 MAX: 3511
EPYC 9654 2P .................... MIN: 2400 AVG: 3493 MAX: 3763
OpenVINO 2022.3
CPU Power Consumption Monitor
Watts < Lower Is Better
Xeon Platinum 8490H 2P .......... MIN: 213 AVG: 640 MAX: 755
Xeon Platinum 8490H 2P - No AMX . MIN: 132 AVG: 653 MAX: 756
EPYC 9654 2P .................... MIN: 55 AVG: 567 MAX: 643
OpenVINO 2022.3
CPU Peak Freq (Highest CPU Core Frequency) Monitor
Megahertz > Higher Is Better
Xeon Platinum 8490H 2P .......... MIN: 1900 AVG: 2792 MAX: 3516
Xeon Platinum 8490H 2P - No AMX . MIN: 1900 AVG: 2656 MAX: 3508
EPYC 9654 2P .................... MIN: 2400 AVG: 2965 MAX: 3700
OpenVINO 2022.3
CPU Power Consumption Monitor
Watts < Lower Is Better
Xeon Platinum 8490H 2P .......... MIN: 212 AVG: 644 MAX: 742
Xeon Platinum 8490H 2P - No AMX . MIN: 211 AVG: 652 MAX: 756
EPYC 9654 2P .................... MIN: 55 AVG: 650 MAX: 740
CPU Peak Freq (Highest CPU Core Frequency) Monitor
Phoronix Test Suite System Monitoring
Megahertz
Xeon Platinum 8490H 2P .......... MIN: 1900 AVG: 2744 MAX: 3516
Xeon Platinum 8490H 2P - No AMX . MIN: 1000 AVG: 2604 MAX: 3512
EPYC 9654 2P .................... MIN: 2400 AVG: 3129 MAX: 3763
CPU Power Consumption Monitor
Phoronix Test Suite System Monitoring
Watts
Xeon Platinum 8490H 2P .......... MIN: 208 AVG: 624 MAX: 761
Xeon Platinum 8490H 2P - No AMX . MIN: 123 AVG: 646 MAX: 764
EPYC 9654 2P .................... MIN: 53 AVG: 596 MAX: 740