openvino-1.4.0-amx Intel Xeon w9-3495X testing with a ASUS Pro WS W790E-SAGE SE (0705 BIOS) and llvmpipe on Ubuntu 22.04 via the Phoronix Test Suite. AVX512_CORE_AMX: Processor: Intel Xeon w9-3495X @ 4.80GHz (56 Cores / 112 Threads), Motherboard: ASUS Pro WS W790E-SAGE SE (0705 BIOS), Chipset: Intel Device 7aa7, Memory: 8 x 32 GB DDR5-4812MT/s Hynix HMCG88AEBRA115N, Disk: 6401GB Micron_9300_MTFDHAL6T4TDR + 0GB Virtual HDisk0, Graphics: llvmpipe (210/405MHz), Audio: Realtek ALC1220, Network: 2 x Intel X710 for 10GBASE-T OS: Ubuntu 22.04, Kernel: 6.3.0-060300-generic (x86_64), Desktop: GNOME Shell 42.9, Display Server: X Server 1.21.1.4, Display Driver: NVIDIA 535.129.03, OpenGL: 4.5 Mesa 23.0.4-0ubuntu1~22.04.1 (LLVM 15.0.7 256 bits), OpenCL: OpenCL 3.0 CUDA 12.2.147, Vulkan: 1.3.242, Compiler: GCC 11.4.0 + CUDA 12.1, File-System: ext4, Screen Resolution: 1920x1200 OpenVINO 2023.2.dev Model: Face Detection FP16 - Device: CPU FPS > Higher Is Better AVX512_CORE_AMX . 77.14 |====================================================== OpenVINO 2023.2.dev Model: Face Detection FP16 - Device: CPU ms < Lower Is Better AVX512_CORE_AMX . 362.17 |===================================================== OpenVINO 2023.2.dev CPU Peak Freq (Highest CPU Core Frequency) Monitor Megahertz > Higher Is Better AVX512_CORE_AMX . MIN: 1900 AVG: 2679 MAX: 4330 OpenVINO 2023.2.dev CPU Power Consumption Monitor Watts < Lower Is Better AVX512_CORE_AMX . MIN: 96 AVG: 408 MAX: 446 OpenVINO 2023.2.dev CPU Temperature Monitor Celsius < Lower Is Better AVX512_CORE_AMX . MIN: 36.0 AVG: 65.9 MAX: 71.0 OpenVINO 2023.2.dev Model: Face Detection FP16-INT8 - Device: CPU FPS > Higher Is Better AVX512_CORE_AMX . 165.65 |===================================================== OpenVINO 2023.2.dev Model: Face Detection FP16-INT8 - Device: CPU ms < Lower Is Better AVX512_CORE_AMX . 337.49 |===================================================== OpenVINO 2023.2.dev CPU Peak Freq (Highest CPU Core Frequency) Monitor Megahertz > Higher Is Better AVX512_CORE_AMX . MIN: 1900 AVG: 2668 MAX: 4305 OpenVINO 2023.2.dev CPU Power Consumption Monitor Watts < Lower Is Better AVX512_CORE_AMX . MIN: 113 AVG: 419 MAX: 460 OpenVINO 2023.2.dev CPU Temperature Monitor Celsius < Lower Is Better AVX512_CORE_AMX . MIN: 42.0 AVG: 68.8 MAX: 74.0 OpenVINO 2023.2.dev Model: Age Gender Recognition Retail 0013 FP16 - Device: CPU FPS > Higher Is Better AVX512_CORE_AMX . 91906.18 |=================================================== OpenVINO 2023.2.dev Model: Age Gender Recognition Retail 0013 FP16 - Device: CPU ms < Lower Is Better AVX512_CORE_AMX . 0.54 |======================================================= OpenVINO 2023.2.dev CPU Peak Freq (Highest CPU Core Frequency) Monitor Megahertz > Higher Is Better AVX512_CORE_AMX . MIN: 1900 AVG: 2391 MAX: 4326 OpenVINO 2023.2.dev CPU Power Consumption Monitor Watts < Lower Is Better AVX512_CORE_AMX . MIN: 105 AVG: 409 MAX: 437 OpenVINO 2023.2.dev CPU Temperature Monitor Celsius < Lower Is Better AVX512_CORE_AMX . MIN: 44.0 AVG: 67.5 MAX: 70.0 OpenVINO 2023.2.dev Model: Age Gender Recognition Retail 0013 FP16-INT8 - Device: CPU FPS > Higher Is Better AVX512_CORE_AMX . 108977.14 |================================================== OpenVINO 2023.2.dev Model: Age Gender Recognition Retail 0013 FP16-INT8 - Device: CPU ms < Lower Is Better AVX512_CORE_AMX . 0.44 |======================================================= OpenVINO 2023.2.dev CPU Peak Freq (Highest CPU Core Frequency) Monitor Megahertz > Higher Is Better AVX512_CORE_AMX . MIN: 1900 AVG: 2375 MAX: 4300 OpenVINO 2023.2.dev CPU Power Consumption Monitor Watts < Lower Is Better AVX512_CORE_AMX . MIN: 100 AVG: 391 MAX: 418 OpenVINO 2023.2.dev CPU Temperature Monitor Celsius < Lower Is Better AVX512_CORE_AMX . MIN: 41.0 AVG: 65.4 MAX: 70.0 OpenVINO 2023.2.dev Model: Person Detection FP16 - Device: CPU FPS > Higher Is Better AVX512_CORE_AMX . 289.57 |===================================================== OpenVINO 2023.2.dev Model: Person Detection FP16 - Device: CPU ms < Lower Is Better AVX512_CORE_AMX . 96.62 |====================================================== OpenVINO 2023.2.dev CPU Peak Freq (Highest CPU Core Frequency) Monitor Megahertz > Higher Is Better AVX512_CORE_AMX . MIN: 1900 AVG: 2691 MAX: 4355 OpenVINO 2023.2.dev CPU Power Consumption Monitor Watts < Lower Is Better AVX512_CORE_AMX . MIN: 100 AVG: 432 MAX: 469 OpenVINO 2023.2.dev CPU Temperature Monitor Celsius < Lower Is Better AVX512_CORE_AMX . MIN: 43.0 AVG: 69.0 MAX: 73.0 OpenVINO 2023.2.dev Model: Person Detection FP32 - Device: CPU FPS > Higher Is Better AVX512_CORE_AMX . 289.89 |===================================================== OpenVINO 2023.2.dev Model: Person Detection FP32 - Device: CPU ms < Lower Is Better AVX512_CORE_AMX . 96.52 |====================================================== OpenVINO 2023.2.dev CPU Peak Freq (Highest CPU Core Frequency) Monitor Megahertz > Higher Is Better AVX512_CORE_AMX . MIN: 1900 AVG: 2682 MAX: 4200 OpenVINO 2023.2.dev CPU Power Consumption Monitor Watts < Lower Is Better AVX512_CORE_AMX . MIN: 59 AVG: 435 MAX: 475 OpenVINO 2023.2.dev CPU Temperature Monitor Celsius < Lower Is Better AVX512_CORE_AMX . MIN: 43.0 AVG: 69.9 MAX: 75.0 OpenVINO 2023.2.dev Model: Weld Porosity Detection FP16-INT8 - Device: CPU FPS > Higher Is Better AVX512_CORE_AMX . 17523.90 |=================================================== OpenVINO 2023.2.dev Model: Weld Porosity Detection FP16-INT8 - Device: CPU ms < Lower Is Better AVX512_CORE_AMX . 3.18 |======================================================= OpenVINO 2023.2.dev CPU Peak Freq (Highest CPU Core Frequency) Monitor Megahertz > Higher Is Better AVX512_CORE_AMX . MIN: 1900 AVG: 2301 MAX: 4198 OpenVINO 2023.2.dev CPU Power Consumption Monitor Watts < Lower Is Better AVX512_CORE_AMX . MIN: 100 AVG: 410 MAX: 442 OpenVINO 2023.2.dev CPU Temperature Monitor Celsius < Lower Is Better AVX512_CORE_AMX . MIN: 44.0 AVG: 68.0 MAX: 71.0 OpenVINO 2023.2.dev Model: Weld Porosity Detection FP16 - Device: CPU FPS > Higher Is Better AVX512_CORE_AMX . 9417.38 |==================================================== OpenVINO 2023.2.dev Model: Weld Porosity Detection FP16 - Device: CPU ms < Lower Is Better AVX512_CORE_AMX . 5.93 |======================================================= OpenVINO 2023.2.dev CPU Peak Freq (Highest CPU Core Frequency) Monitor Megahertz > Higher Is Better AVX512_CORE_AMX . MIN: 1900 AVG: 2405 MAX: 4119 OpenVINO 2023.2.dev CPU Power Consumption Monitor Watts < Lower Is Better AVX512_CORE_AMX . MIN: 94 AVG: 405 MAX: 437 OpenVINO 2023.2.dev CPU Temperature Monitor Celsius < Lower Is Better AVX512_CORE_AMX . MIN: 40.0 AVG: 67.6 MAX: 73.0 OpenVINO 2023.2.dev Model: Vehicle Detection FP16-INT8 - Device: CPU FPS > Higher Is Better AVX512_CORE_AMX . 3234.97 |==================================================== OpenVINO 2023.2.dev Model: Vehicle Detection FP16-INT8 - Device: CPU ms < Lower Is Better AVX512_CORE_AMX . 17.28 |====================================================== OpenVINO 2023.2.dev CPU Peak Freq (Highest CPU Core Frequency) Monitor Megahertz > Higher Is Better AVX512_CORE_AMX . MIN: 1900 AVG: 2501 MAX: 4200 OpenVINO 2023.2.dev CPU Power Consumption Monitor Watts < Lower Is Better AVX512_CORE_AMX . MIN: 94 AVG: 457 MAX: 500 OpenVINO 2023.2.dev CPU Temperature Monitor Celsius < Lower Is Better AVX512_CORE_AMX . MIN: 42.0 AVG: 71.7 MAX: 76.0 OpenVINO 2023.2.dev Model: Vehicle Detection FP16 - Device: CPU FPS > Higher Is Better AVX512_CORE_AMX . 1921.78 |==================================================== OpenVINO 2023.2.dev Model: Vehicle Detection FP16 - Device: CPU ms < Lower Is Better AVX512_CORE_AMX . 14.54 |====================================================== OpenVINO 2023.2.dev CPU Peak Freq (Highest CPU Core Frequency) Monitor Megahertz > Higher Is Better AVX512_CORE_AMX . MIN: 1900 AVG: 2543 MAX: 3789 OpenVINO 2023.2.dev CPU Power Consumption Monitor Watts < Lower Is Better AVX512_CORE_AMX . MIN: 98 AVG: 455 MAX: 490 OpenVINO 2023.2.dev CPU Temperature Monitor Celsius < Lower Is Better AVX512_CORE_AMX . MIN: 41 AVG: 72 MAX: 78 OpenVINO 2023.2.dev Model: Person Vehicle Bike Detection FP16 - Device: CPU FPS > Higher Is Better AVX512_CORE_AMX . 3661.12 |==================================================== OpenVINO 2023.2.dev Model: Person Vehicle Bike Detection FP16 - Device: CPU ms < Lower Is Better AVX512_CORE_AMX . 15.27 |====================================================== OpenVINO 2023.2.dev CPU Peak Freq (Highest CPU Core Frequency) Monitor Megahertz > Higher Is Better AVX512_CORE_AMX . MIN: 1900 AVG: 2460 MAX: 4300 OpenVINO 2023.2.dev CPU Power Consumption Monitor Watts < Lower Is Better AVX512_CORE_AMX . MIN: 98 AVG: 436 MAX: 476 OpenVINO 2023.2.dev CPU Temperature Monitor Celsius < Lower Is Better AVX512_CORE_AMX . MIN: 44.0 AVG: 70.7 MAX: 75.0 OpenVINO 2023.2.dev Model: Machine Translation EN To DE FP16 - Device: CPU FPS > Higher Is Better OpenVINO 2023.2.dev Model: Face Detection Retail FP16 - Device: CPU FPS > Higher Is Better AVX512_CORE_AMX . 6610.96 |==================================================== OpenVINO 2023.2.dev Model: Face Detection Retail FP16 - Device: CPU ms < Lower Is Better AVX512_CORE_AMX . 8.45 |======================================================= OpenVINO 2023.2.dev CPU Peak Freq (Highest CPU Core Frequency) Monitor Megahertz > Higher Is Better AVX512_CORE_AMX . MIN: 1900 AVG: 2397 MAX: 3397 OpenVINO 2023.2.dev CPU Power Consumption Monitor Watts < Lower Is Better AVX512_CORE_AMX . MIN: 94 AVG: 452 MAX: 490 OpenVINO 2023.2.dev CPU Temperature Monitor Celsius < Lower Is Better AVX512_CORE_AMX . MIN: 39 AVG: 71 MAX: 78 OpenVINO 2023.2.dev Model: Face Detection Retail FP16-INT8 - Device: CPU FPS > Higher Is Better AVX512_CORE_AMX . 10083.57 |=================================================== OpenVINO 2023.2.dev Model: Face Detection Retail FP16-INT8 - Device: CPU ms < Lower Is Better AVX512_CORE_AMX . 5.54 |======================================================= OpenVINO 2023.2.dev CPU Peak Freq (Highest CPU Core Frequency) Monitor Megahertz > Higher Is Better AVX512_CORE_AMX . MIN: 1900 AVG: 2376 MAX: 3703 OpenVINO 2023.2.dev CPU Power Consumption Monitor Watts < Lower Is Better AVX512_CORE_AMX . MIN: 95 AVG: 440 MAX: 485 OpenVINO 2023.2.dev CPU Temperature Monitor Celsius < Lower Is Better AVX512_CORE_AMX . MIN: 44.0 AVG: 71.3 MAX: 75.0 OpenVINO 2023.2.dev Model: Handwritten English Recognition FP16 - Device: CPU FPS > Higher Is Better AVX512_CORE_AMX . 1597.47 |==================================================== OpenVINO 2023.2.dev Model: Handwritten English Recognition FP16 - Device: CPU ms < Lower Is Better AVX512_CORE_AMX . 35.03 |====================================================== OpenVINO 2023.2.dev CPU Peak Freq (Highest CPU Core Frequency) Monitor Megahertz > Higher Is Better AVX512_CORE_AMX . MIN: 1900 AVG: 2610 MAX: 4200 OpenVINO 2023.2.dev CPU Power Consumption Monitor Watts < Lower Is Better AVX512_CORE_AMX . MIN: 95 AVG: 427 MAX: 461 OpenVINO 2023.2.dev CPU Temperature Monitor Celsius < Lower Is Better AVX512_CORE_AMX . MIN: 41.0 AVG: 69.6 MAX: 75.0 OpenVINO 2023.2.dev Model: Handwritten English Recognition FP16-INT8 - Device: CPU FPS > Higher Is Better AVX512_CORE_AMX . 1244.36 |==================================================== OpenVINO 2023.2.dev Model: Handwritten English Recognition FP16-INT8 - Device: CPU ms < Lower Is Better AVX512_CORE_AMX . 44.97 |====================================================== OpenVINO 2023.2.dev CPU Peak Freq (Highest CPU Core Frequency) Monitor Megahertz > Higher Is Better AVX512_CORE_AMX . MIN: 1900 AVG: 2743 MAX: 3382 OpenVINO 2023.2.dev CPU Power Consumption Monitor Watts < Lower Is Better AVX512_CORE_AMX . MIN: 96 AVG: 419 MAX: 451 OpenVINO 2023.2.dev CPU Temperature Monitor Celsius < Lower Is Better AVX512_CORE_AMX . MIN: 44.0 AVG: 69.9 MAX: 74.0 OpenVINO 2023.2.dev Model: Road Segmentation ADAS FP16 - Device: CPU FPS > Higher Is Better AVX512_CORE_AMX . 807.46 |===================================================== OpenVINO 2023.2.dev Model: Road Segmentation ADAS FP16 - Device: CPU ms < Lower Is Better AVX512_CORE_AMX . 34.62 |====================================================== OpenVINO 2023.2.dev CPU Peak Freq (Highest CPU Core Frequency) Monitor Megahertz > Higher Is Better AVX512_CORE_AMX . MIN: 1900 AVG: 2694 MAX: 4232 OpenVINO 2023.2.dev CPU Power Consumption Monitor Watts < Lower Is Better AVX512_CORE_AMX . MIN: 94 AVG: 466 MAX: 507 OpenVINO 2023.2.dev CPU Temperature Monitor Celsius < Lower Is Better AVX512_CORE_AMX . MIN: 41 AVG: 72 MAX: 78 OpenVINO 2023.2.dev Model: Road Segmentation ADAS FP16-INT8 - Device: CPU FPS > Higher Is Better AVX512_CORE_AMX . 1129.41 |==================================================== OpenVINO 2023.2.dev Model: Road Segmentation ADAS FP16-INT8 - Device: CPU ms < Lower Is Better AVX512_CORE_AMX . 49.52 |====================================================== OpenVINO 2023.2.dev CPU Peak Freq (Highest CPU Core Frequency) Monitor Megahertz > Higher Is Better AVX512_CORE_AMX . MIN: 1900 AVG: 2670 MAX: 4300 OpenVINO 2023.2.dev CPU Power Consumption Monitor Watts < Lower Is Better AVX512_CORE_AMX . MIN: 98 AVG: 443 MAX: 481 OpenVINO 2023.2.dev CPU Temperature Monitor Celsius < Lower Is Better AVX512_CORE_AMX . MIN: 45.0 AVG: 71.1 MAX: 75.0 CPU Peak Freq (Highest CPU Core Frequency) Monitor Phoronix Test Suite System Monitoring Megahertz AVX512_CORE_AMX . MIN: 1900 AVG: 2523 MAX: 4355 CPU Power Consumption Monitor Phoronix Test Suite System Monitoring Watts AVX512_CORE_AMX . MIN: 54 AVG: 408 MAX: 507 CPU Temperature Monitor Phoronix Test Suite System Monitoring Celsius AVX512_CORE_AMX . MIN: 36 AVG: 68 MAX: 78