Intel Xeon w9-3495X testing with a ASUS Pro WS W790E-SAGE SE (0705 BIOS) and llvmpipe on Ubuntu 22.04 via the Phoronix Test Suite.
Compare your own system(s) to this result file with the
Phoronix Test Suite by running the command:
phoronix-test-suite benchmark 2311114-NE-2311110NE81 openvino-1.4.0-amx - Phoronix Test Suite openvino-1.4.0-amx Intel Xeon w9-3495X testing with a ASUS Pro WS W790E-SAGE SE (0705 BIOS) and llvmpipe on Ubuntu 22.04 via the Phoronix Test Suite.
HTML result view exported from: https://openbenchmarking.org/result/2311114-NE-2311110NE81&rdt&grt .
openvino-1.4.0-amx Processor Motherboard Chipset Memory Disk Graphics Audio Network OS Kernel Desktop Display Server Display Driver OpenGL OpenCL Vulkan Compiler File-System Screen Resolution AVX512_CORE_AMX AVX512_CORE Intel Xeon w9-3495X @ 4.80GHz (56 Cores / 112 Threads) ASUS Pro WS W790E-SAGE SE (0705 BIOS) Intel Device 7aa7 8 x 32 GB DDR5-4812MT/s Hynix HMCG88AEBRA115N 6401GB Micron_9300_MTFDHAL6T4TDR + 0GB Virtual HDisk0 llvmpipe (210/405MHz) Realtek ALC1220 2 x Intel X710 for 10GBASE-T Ubuntu 22.04 6.3.0-060300-generic (x86_64) GNOME Shell 42.9 X Server 1.21.1.4 NVIDIA 535.129.03 4.5 Mesa 23.0.4-0ubuntu1~22.04.1 (LLVM 15.0.7 256 bits) OpenCL 3.0 CUDA 12.2.147 1.3.242 GCC 11.4.0 + CUDA 12.1 ext4 1920x1200 6401GB Micron_9300_MTFDHAL6T4TDR + 4097GB Lexar SSD ARES 4TB + 0GB Virtual HDisk0 OpenBenchmarking.org Kernel Details - Transparent Huge Pages: madvise Compiler Details - --build=x86_64-linux-gnu --disable-vtable-verify --disable-werror --enable-bootstrap --enable-cet --enable-checking=release --enable-clocale=gnu --enable-default-pie --enable-gnu-unique-object --enable-languages=c,ada,c++,go,brig,d,fortran,objc,obj-c++,m2 --enable-libphobos-checking=release --enable-libstdcxx-debug --enable-libstdcxx-time=yes --enable-link-serialization=2 --enable-multiarch --enable-multilib --enable-nls --enable-objc-gc=auto --enable-offload-targets=nvptx-none=/build/gcc-11-XeT9lY/gcc-11-11.4.0/debian/tmp-nvptx/usr,amdgcn-amdhsa=/build/gcc-11-XeT9lY/gcc-11-11.4.0/debian/tmp-gcn/usr --enable-plugin --enable-shared --enable-threads=posix --host=x86_64-linux-gnu --program-prefix=x86_64-linux-gnu- --target=x86_64-linux-gnu --with-abi=m64 --with-arch-32=i686 --with-build-config=bootstrap-lto-lean --with-default-libstdcxx-abi=new --with-gcc-major-version-only --with-multilib-list=m32,m64,mx32 --with-target-system-zlib=auto --with-tune=generic --without-cuda-driver -v Processor Details - Scaling Governor: intel_pstate powersave (EPP: performance) - CPU Microcode: 0x2b0004b1 Python Details - Python 3.10.12 Security Details - itlb_multihit: Not affected + l1tf: Not affected + mds: Not affected + meltdown: Not affected + mmio_stale_data: Not affected + retbleed: Not affected + spec_store_bypass: Mitigation of SSB disabled via prctl + spectre_v1: Mitigation of usercopy/swapgs barriers and __user pointer sanitization + spectre_v2: Mitigation of Enhanced / Automatic IBRS IBPB: conditional RSB filling PBRSB-eIBRS: SW sequence + srbds: Not affected + tsx_async_abort: Not affected
openvino-1.4.0-amx openvino: Face Detection FP16 - CPU openvino: Face Detection FP16 - CPU openvino: Face Detection FP16-INT8 - CPU openvino: Face Detection FP16-INT8 - CPU openvino: Age Gender Recognition Retail 0013 FP16 - CPU openvino: Age Gender Recognition Retail 0013 FP16 - CPU openvino: Age Gender Recognition Retail 0013 FP16-INT8 - CPU openvino: Age Gender Recognition Retail 0013 FP16-INT8 - CPU openvino: Person Detection FP16 - CPU openvino: Person Detection FP16 - CPU openvino: Person Detection FP32 - CPU openvino: Person Detection FP32 - CPU openvino: Weld Porosity Detection FP16-INT8 - CPU openvino: Weld Porosity Detection FP16-INT8 - CPU openvino: Weld Porosity Detection FP16 - CPU openvino: Weld Porosity Detection FP16 - CPU openvino: Vehicle Detection FP16-INT8 - CPU openvino: Vehicle Detection FP16-INT8 - CPU openvino: Vehicle Detection FP16 - CPU openvino: Vehicle Detection FP16 - CPU openvino: Person Vehicle Bike Detection FP16 - CPU openvino: Person Vehicle Bike Detection FP16 - CPU openvino: Face Detection Retail FP16 - CPU openvino: Face Detection Retail FP16 - CPU openvino: Face Detection Retail FP16-INT8 - CPU openvino: Face Detection Retail FP16-INT8 - CPU openvino: Handwritten English Recognition FP16 - CPU openvino: Handwritten English Recognition FP16 - CPU openvino: Handwritten English Recognition FP16-INT8 - CPU openvino: Handwritten English Recognition FP16-INT8 - CPU openvino: Road Segmentation ADAS FP16 - CPU openvino: Road Segmentation ADAS FP16 - CPU openvino: Road Segmentation ADAS FP16-INT8 - CPU openvino: Road Segmentation ADAS FP16-INT8 - CPU AVX512_CORE_AMX AVX512_CORE 77.14 362.17 165.65 337.49 91906.18 0.54 108977.14 0.44 289.57 96.62 289.89 96.52 17523.90 3.18 9417.38 5.93 3234.97 17.28 1921.78 14.54 3661.12 15.27 6610.96 8.45 10083.57 5.54 1597.47 35.03 1244.36 44.97 807.46 34.62 1129.41 49.52 12.26 2269.70 86.16 324.21 36370.46 1.4 83670.38 0.58 131.41 212.82 133.14 210.10 8530.81 6.55 1273.78 43.93 4193.97 6.66 1077.28 25.97 1591.03 17.58 3153.69 8.87 11238.54 4.96 844.59 66.26 886.14 63.15 540.69 51.73 1187.75 23.47 OpenBenchmarking.org
CPU Peak Freq (Highest CPU Core Frequency) Monitor Phoronix Test Suite System Monitoring OpenBenchmarking.org Megahertz CPU Peak Freq (Highest CPU Core Frequency) Monitor Phoronix Test Suite System Monitoring AVX512_CORE_AMX AVX512_CORE 800 1600 2400 3200 4000 Min: 1900 / Avg: 2523.31 / Max: 4355 Min: 1900 / Avg: 2564.64 / Max: 4586
CPU Power Consumption Monitor Phoronix Test Suite System Monitoring OpenBenchmarking.org Watts CPU Power Consumption Monitor Phoronix Test Suite System Monitoring AVX512_CORE_AMX AVX512_CORE 110 220 330 440 550 Min: 54.4 / Avg: 407.76 / Max: 507.26 Min: 48.23 / Avg: 428.13 / Max: 642.82
CPU Temperature Monitor Phoronix Test Suite System Monitoring OpenBenchmarking.org Celsius CPU Temperature Monitor Phoronix Test Suite System Monitoring AVX512_CORE_AMX AVX512_CORE 16 32 48 64 80 Min: 36 / Avg: 68.03 / Max: 78 Min: 36 / Avg: 69.48 / Max: 85
OpenVINO Model: Face Detection FP16 - Device: CPU OpenBenchmarking.org FPS, More Is Better OpenVINO 2023.2.dev Model: Face Detection FP16 - Device: CPU AVX512_CORE_AMX AVX512_CORE 20 40 60 80 100 SE +/- 0.08, N = 3 SE +/- 0.01, N = 3 77.14 12.26 1. (CXX) g++ options: -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -pie
OpenVINO Model: Face Detection FP16 - Device: CPU OpenBenchmarking.org ms, Fewer Is Better OpenVINO 2023.2.dev Model: Face Detection FP16 - Device: CPU AVX512_CORE_AMX AVX512_CORE 500 1000 1500 2000 2500 SE +/- 0.35, N = 3 SE +/- 0.78, N = 3 362.17 2269.70 MIN: 125.7 / MAX: 443.52 MIN: 1653.74 / MAX: 2733.95 1. (CXX) g++ options: -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -pie
OpenVINO CPU Peak Freq (Highest CPU Core Frequency) Monitor Min Avg Max AVX512_CORE_AMX 1900 2679 4330 AVX512_CORE 1900 2661 4300 OpenBenchmarking.org Megahertz, More Is Better OpenVINO 2023.2.dev CPU Peak Freq (Highest CPU Core Frequency) Monitor 1100 2200 3300 4400 5500
OpenVINO CPU Power Consumption Monitor Min Avg Max AVX512_CORE_AMX 95.5 407.5 446.0 AVX512_CORE 115.9 407.9 451.2 OpenBenchmarking.org Watts, Fewer Is Better OpenVINO 2023.2.dev CPU Power Consumption Monitor 120 240 360 480 600
OpenVINO CPU Temperature Monitor Min Avg Max AVX512_CORE_AMX 36.0 65.9 71.0 AVX512_CORE 37.0 65.8 73.0 OpenBenchmarking.org Celsius, Fewer Is Better OpenVINO 2023.2.dev CPU Temperature Monitor 20 40 60 80 100
OpenVINO Model: Face Detection FP16-INT8 - Device: CPU OpenBenchmarking.org FPS, More Is Better OpenVINO 2023.2.dev Model: Face Detection FP16-INT8 - Device: CPU AVX512_CORE_AMX AVX512_CORE 40 80 120 160 200 SE +/- 0.07, N = 3 SE +/- 0.43, N = 3 165.65 86.16 1. (CXX) g++ options: -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -pie
OpenVINO Model: Face Detection FP16-INT8 - Device: CPU OpenBenchmarking.org ms, Fewer Is Better OpenVINO 2023.2.dev Model: Face Detection FP16-INT8 - Device: CPU AVX512_CORE_AMX AVX512_CORE 70 140 210 280 350 SE +/- 0.16, N = 3 SE +/- 1.55, N = 3 337.49 324.21 MIN: 123.15 / MAX: 428.33 MIN: 173.46 / MAX: 419.38 1. (CXX) g++ options: -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -pie
OpenVINO CPU Peak Freq (Highest CPU Core Frequency) Monitor Min Avg Max AVX512_CORE_AMX 1900 2668 4305 AVX512_CORE 1900 2573 4300 OpenBenchmarking.org Megahertz, More Is Better OpenVINO 2023.2.dev CPU Peak Freq (Highest CPU Core Frequency) Monitor 1100 2200 3300 4400 5500
OpenVINO CPU Power Consumption Monitor Min Avg Max AVX512_CORE_AMX 112.9 419.1 459.6 AVX512_CORE 107.3 506.9 613.1 OpenBenchmarking.org Watts, Fewer Is Better OpenVINO 2023.2.dev CPU Power Consumption Monitor 160 320 480 640 800
OpenVINO CPU Temperature Monitor Min Avg Max AVX512_CORE_AMX 42.0 68.8 74.0 AVX512_CORE 44.0 79.2 84.0 OpenBenchmarking.org Celsius, Fewer Is Better OpenVINO 2023.2.dev CPU Temperature Monitor 20 40 60 80 100
OpenVINO Model: Age Gender Recognition Retail 0013 FP16 - Device: CPU OpenBenchmarking.org FPS, More Is Better OpenVINO 2023.2.dev Model: Age Gender Recognition Retail 0013 FP16 - Device: CPU AVX512_CORE_AMX AVX512_CORE 20K 40K 60K 80K 100K SE +/- 56.21, N = 3 SE +/- 31.52, N = 3 91906.18 36370.46 1. (CXX) g++ options: -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -pie
OpenVINO Model: Age Gender Recognition Retail 0013 FP16 - Device: CPU OpenBenchmarking.org ms, Fewer Is Better OpenVINO 2023.2.dev Model: Age Gender Recognition Retail 0013 FP16 - Device: CPU AVX512_CORE_AMX AVX512_CORE 0.315 0.63 0.945 1.26 1.575 SE +/- 0.00, N = 3 SE +/- 0.00, N = 3 0.54 1.40 MIN: 0.24 / MAX: 55.37 MIN: 0.74 / MAX: 52.71 1. (CXX) g++ options: -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -pie
OpenVINO CPU Peak Freq (Highest CPU Core Frequency) Monitor Min Avg Max AVX512_CORE_AMX 1900 2391 4326 AVX512_CORE 1900 2537 4258 OpenBenchmarking.org Megahertz, More Is Better OpenVINO 2023.2.dev CPU Peak Freq (Highest CPU Core Frequency) Monitor 1100 2200 3300 4400 5500
OpenVINO CPU Power Consumption Monitor Min Avg Max AVX512_CORE_AMX 104.6 408.8 436.9 AVX512_CORE 102.7 432.4 464.3 OpenBenchmarking.org Watts, Fewer Is Better OpenVINO 2023.2.dev CPU Power Consumption Monitor 120 240 360 480 600
OpenVINO CPU Temperature Monitor Min Avg Max AVX512_CORE_AMX 44.0 67.5 70.0 AVX512_CORE 45.0 69.4 74.0 OpenBenchmarking.org Celsius, Fewer Is Better OpenVINO 2023.2.dev CPU Temperature Monitor 20 40 60 80 100
OpenVINO Model: Age Gender Recognition Retail 0013 FP16-INT8 - Device: CPU OpenBenchmarking.org FPS, More Is Better OpenVINO 2023.2.dev Model: Age Gender Recognition Retail 0013 FP16-INT8 - Device: CPU AVX512_CORE_AMX AVX512_CORE 20K 40K 60K 80K 100K SE +/- 222.39, N = 3 SE +/- 99.28, N = 3 108977.14 83670.38 1. (CXX) g++ options: -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -pie
OpenVINO Model: Age Gender Recognition Retail 0013 FP16-INT8 - Device: CPU OpenBenchmarking.org ms, Fewer Is Better OpenVINO 2023.2.dev Model: Age Gender Recognition Retail 0013 FP16-INT8 - Device: CPU AVX512_CORE_AMX AVX512_CORE 0.1305 0.261 0.3915 0.522 0.6525 SE +/- 0.00, N = 3 SE +/- 0.00, N = 3 0.44 0.58 MIN: 0.22 / MAX: 54.33 MIN: 0.31 / MAX: 50.67 1. (CXX) g++ options: -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -pie
OpenVINO CPU Peak Freq (Highest CPU Core Frequency) Monitor Min Avg Max AVX512_CORE_AMX 1900 2375 4300 AVX512_CORE 1900 2569 4426 OpenBenchmarking.org Megahertz, More Is Better OpenVINO 2023.2.dev CPU Peak Freq (Highest CPU Core Frequency) Monitor 1100 2200 3300 4400 5500
OpenVINO CPU Power Consumption Monitor Min Avg Max AVX512_CORE_AMX 100.3 390.6 418.2 AVX512_CORE 57.3 435.5 465.7 OpenBenchmarking.org Watts, Fewer Is Better OpenVINO 2023.2.dev CPU Power Consumption Monitor 120 240 360 480 600
OpenVINO CPU Temperature Monitor Min Avg Max AVX512_CORE_AMX 41.0 65.4 70.0 AVX512_CORE 44.0 69.2 73.0 OpenBenchmarking.org Celsius, Fewer Is Better OpenVINO 2023.2.dev CPU Temperature Monitor 20 40 60 80 100
OpenVINO Model: Person Detection FP16 - Device: CPU OpenBenchmarking.org FPS, More Is Better OpenVINO 2023.2.dev Model: Person Detection FP16 - Device: CPU AVX512_CORE_AMX AVX512_CORE 60 120 180 240 300 SE +/- 0.19, N = 3 SE +/- 0.03, N = 3 289.57 131.41 1. (CXX) g++ options: -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -pie
OpenVINO Model: Person Detection FP16 - Device: CPU OpenBenchmarking.org ms, Fewer Is Better OpenVINO 2023.2.dev Model: Person Detection FP16 - Device: CPU AVX512_CORE_AMX AVX512_CORE 50 100 150 200 250 SE +/- 0.07, N = 3 SE +/- 0.06, N = 3 96.62 212.82 MIN: 31.84 / MAX: 190 MIN: 114.27 / MAX: 308.21 1. (CXX) g++ options: -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -pie
OpenVINO CPU Peak Freq (Highest CPU Core Frequency) Monitor Min Avg Max AVX512_CORE_AMX 1900 2691 4355 AVX512_CORE 1900 2664 4385 OpenBenchmarking.org Megahertz, More Is Better OpenVINO 2023.2.dev CPU Peak Freq (Highest CPU Core Frequency) Monitor 1100 2200 3300 4400 5500
OpenVINO CPU Power Consumption Monitor Min Avg Max AVX512_CORE_AMX 100.4 431.9 468.9 AVX512_CORE 99.1 424.2 462.2 OpenBenchmarking.org Watts, Fewer Is Better OpenVINO 2023.2.dev CPU Power Consumption Monitor 120 240 360 480 600
OpenVINO CPU Temperature Monitor Min Avg Max AVX512_CORE_AMX 43.0 69.0 73.0 AVX512_CORE 42.0 67.7 73.0 OpenBenchmarking.org Celsius, Fewer Is Better OpenVINO 2023.2.dev CPU Temperature Monitor 20 40 60 80 100
OpenVINO Model: Person Detection FP32 - Device: CPU OpenBenchmarking.org FPS, More Is Better OpenVINO 2023.2.dev Model: Person Detection FP32 - Device: CPU AVX512_CORE_AMX AVX512_CORE 60 120 180 240 300 SE +/- 0.35, N = 3 SE +/- 1.70, N = 3 289.89 133.14 1. (CXX) g++ options: -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -pie
OpenVINO Model: Person Detection FP32 - Device: CPU OpenBenchmarking.org ms, Fewer Is Better OpenVINO 2023.2.dev Model: Person Detection FP32 - Device: CPU AVX512_CORE_AMX AVX512_CORE 50 100 150 200 250 SE +/- 0.11, N = 3 SE +/- 2.67, N = 3 96.52 210.10 MIN: 34.11 / MAX: 190.76 MIN: 115.17 / MAX: 637.54 1. (CXX) g++ options: -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -pie
OpenVINO CPU Peak Freq (Highest CPU Core Frequency) Monitor Min Avg Max AVX512_CORE_AMX 1900 2682 4200 AVX512_CORE 1900 2650 4586 OpenBenchmarking.org Megahertz, More Is Better OpenVINO 2023.2.dev CPU Peak Freq (Highest CPU Core Frequency) Monitor 1200 2400 3600 4800 6000
OpenVINO CPU Power Consumption Monitor Min Avg Max AVX512_CORE_AMX 59.2 434.7 475.1 AVX512_CORE 103.5 429.1 466.3 OpenBenchmarking.org Watts, Fewer Is Better OpenVINO 2023.2.dev CPU Power Consumption Monitor 120 240 360 480 600
OpenVINO CPU Temperature Monitor Min Avg Max AVX512_CORE_AMX 43.0 69.9 75.0 AVX512_CORE 46.0 68.8 72.0 OpenBenchmarking.org Celsius, Fewer Is Better OpenVINO 2023.2.dev CPU Temperature Monitor 20 40 60 80 100
OpenVINO Model: Weld Porosity Detection FP16-INT8 - Device: CPU OpenBenchmarking.org FPS, More Is Better OpenVINO 2023.2.dev Model: Weld Porosity Detection FP16-INT8 - Device: CPU AVX512_CORE_AMX AVX512_CORE 4K 8K 12K 16K 20K SE +/- 19.41, N = 3 SE +/- 120.35, N = 3 17523.90 8530.81 1. (CXX) g++ options: -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -pie
OpenVINO Model: Weld Porosity Detection FP16-INT8 - Device: CPU OpenBenchmarking.org ms, Fewer Is Better OpenVINO 2023.2.dev Model: Weld Porosity Detection FP16-INT8 - Device: CPU AVX512_CORE_AMX AVX512_CORE 2 4 6 8 10 SE +/- 0.00, N = 3 SE +/- 0.09, N = 3 3.18 6.55 MIN: 1.4 / MAX: 68.79 MIN: 3.46 / MAX: 460.84 1. (CXX) g++ options: -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -pie
OpenVINO CPU Peak Freq (Highest CPU Core Frequency) Monitor Min Avg Max AVX512_CORE_AMX 1900 2301 4198 AVX512_CORE 1900 2490 3475 OpenBenchmarking.org Megahertz, More Is Better OpenVINO 2023.2.dev CPU Peak Freq (Highest CPU Core Frequency) Monitor 1100 2200 3300 4400 5500
OpenVINO CPU Power Consumption Monitor Min Avg Max AVX512_CORE_AMX 100 410 442 AVX512_CORE 121 520 643 OpenBenchmarking.org Watts, Fewer Is Better OpenVINO 2023.2.dev CPU Power Consumption Monitor 200 400 600 800 1000
OpenVINO CPU Temperature Monitor Min Avg Max AVX512_CORE_AMX 44.0 68.0 71.0 AVX512_CORE 44.0 79.8 85.0 OpenBenchmarking.org Celsius, Fewer Is Better OpenVINO 2023.2.dev CPU Temperature Monitor 20 40 60 80 100
OpenVINO Model: Weld Porosity Detection FP16 - Device: CPU OpenBenchmarking.org FPS, More Is Better OpenVINO 2023.2.dev Model: Weld Porosity Detection FP16 - Device: CPU AVX512_CORE_AMX AVX512_CORE 2K 4K 6K 8K 10K SE +/- 1.68, N = 3 SE +/- 0.25, N = 3 9417.38 1273.78 1. (CXX) g++ options: -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -pie
OpenVINO Model: Weld Porosity Detection FP16 - Device: CPU OpenBenchmarking.org ms, Fewer Is Better OpenVINO 2023.2.dev Model: Weld Porosity Detection FP16 - Device: CPU AVX512_CORE_AMX AVX512_CORE 10 20 30 40 50 SE +/- 0.00, N = 3 SE +/- 0.01, N = 3 5.93 43.93 MIN: 2.16 / MAX: 72.84 MIN: 22.22 / MAX: 112.86 1. (CXX) g++ options: -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -pie
OpenVINO CPU Peak Freq (Highest CPU Core Frequency) Monitor Min Avg Max AVX512_CORE_AMX 1900 2405 4119 AVX512_CORE 1900 2520 4200 OpenBenchmarking.org Megahertz, More Is Better OpenVINO 2023.2.dev CPU Peak Freq (Highest CPU Core Frequency) Monitor 1100 2200 3300 4400 5500
OpenVINO CPU Power Consumption Monitor Min Avg Max AVX512_CORE_AMX 93.9 405.3 437.1 AVX512_CORE 111.4 436.9 465.5 OpenBenchmarking.org Watts, Fewer Is Better OpenVINO 2023.2.dev CPU Power Consumption Monitor 120 240 360 480 600
OpenVINO CPU Temperature Monitor Min Avg Max AVX512_CORE_AMX 40.0 67.6 73.0 AVX512_CORE 47.0 71.0 75.0 OpenBenchmarking.org Celsius, Fewer Is Better OpenVINO 2023.2.dev CPU Temperature Monitor 20 40 60 80 100
OpenVINO Model: Vehicle Detection FP16-INT8 - Device: CPU OpenBenchmarking.org FPS, More Is Better OpenVINO 2023.2.dev Model: Vehicle Detection FP16-INT8 - Device: CPU AVX512_CORE_AMX AVX512_CORE 900 1800 2700 3600 4500 SE +/- 4.47, N = 3 SE +/- 56.25, N = 3 3234.97 4193.97 1. (CXX) g++ options: -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -pie
OpenVINO Model: Vehicle Detection FP16-INT8 - Device: CPU OpenBenchmarking.org ms, Fewer Is Better OpenVINO 2023.2.dev Model: Vehicle Detection FP16-INT8 - Device: CPU AVX512_CORE_AMX AVX512_CORE 4 8 12 16 20 SE +/- 0.02, N = 3 SE +/- 0.09, N = 3 17.28 6.66 MIN: 8.29 / MAX: 106.97 MIN: 3.76 / MAX: 73.82 1. (CXX) g++ options: -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -pie
OpenVINO CPU Peak Freq (Highest CPU Core Frequency) Monitor Min Avg Max AVX512_CORE_AMX 1900 2501 4200 AVX512_CORE 1900 2555 2909 OpenBenchmarking.org Megahertz, More Is Better OpenVINO 2023.2.dev CPU Peak Freq (Highest CPU Core Frequency) Monitor 1100 2200 3300 4400 5500
OpenVINO CPU Power Consumption Monitor Min Avg Max AVX512_CORE_AMX 94.4 456.6 499.8 AVX512_CORE 113.8 522.4 604.9 OpenBenchmarking.org Watts, Fewer Is Better OpenVINO 2023.2.dev CPU Power Consumption Monitor 160 320 480 640 800
OpenVINO CPU Temperature Monitor Min Avg Max AVX512_CORE_AMX 42.0 71.7 76.0 AVX512_CORE 42.0 79.2 84.0 OpenBenchmarking.org Celsius, Fewer Is Better OpenVINO 2023.2.dev CPU Temperature Monitor 20 40 60 80 100
OpenVINO Model: Vehicle Detection FP16 - Device: CPU OpenBenchmarking.org FPS, More Is Better OpenVINO 2023.2.dev Model: Vehicle Detection FP16 - Device: CPU AVX512_CORE_AMX AVX512_CORE 400 800 1200 1600 2000 SE +/- 0.57, N = 3 SE +/- 0.86, N = 3 1921.78 1077.28 1. (CXX) g++ options: -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -pie
OpenVINO Model: Vehicle Detection FP16 - Device: CPU OpenBenchmarking.org ms, Fewer Is Better OpenVINO 2023.2.dev Model: Vehicle Detection FP16 - Device: CPU AVX512_CORE_AMX AVX512_CORE 6 12 18 24 30 SE +/- 0.01, N = 3 SE +/- 0.02, N = 3 14.54 25.97 MIN: 6.89 / MAX: 93.43 MIN: 14.87 / MAX: 84.67 1. (CXX) g++ options: -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -pie
OpenVINO CPU Peak Freq (Highest CPU Core Frequency) Monitor Min Avg Max AVX512_CORE_AMX 1900 2543 3789 AVX512_CORE 1900 2562 4171 OpenBenchmarking.org Megahertz, More Is Better OpenVINO 2023.2.dev CPU Peak Freq (Highest CPU Core Frequency) Monitor 1100 2200 3300 4400 5500
OpenVINO CPU Power Consumption Monitor Min Avg Max AVX512_CORE_AMX 97.6 454.6 490.5 AVX512_CORE 104.5 460.3 489.8 OpenBenchmarking.org Watts, Fewer Is Better OpenVINO 2023.2.dev CPU Power Consumption Monitor 130 260 390 520 650
OpenVINO CPU Temperature Monitor Min Avg Max AVX512_CORE_AMX 41.0 71.7 78.0 AVX512_CORE 48.0 73.2 77.0 OpenBenchmarking.org Celsius, Fewer Is Better OpenVINO 2023.2.dev CPU Temperature Monitor 20 40 60 80 100
OpenVINO Model: Person Vehicle Bike Detection FP16 - Device: CPU OpenBenchmarking.org FPS, More Is Better OpenVINO 2023.2.dev Model: Person Vehicle Bike Detection FP16 - Device: CPU AVX512_CORE_AMX AVX512_CORE 800 1600 2400 3200 4000 SE +/- 7.35, N = 3 SE +/- 0.66, N = 3 3661.12 1591.03 1. (CXX) g++ options: -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -pie
OpenVINO Model: Person Vehicle Bike Detection FP16 - Device: CPU OpenBenchmarking.org ms, Fewer Is Better OpenVINO 2023.2.dev Model: Person Vehicle Bike Detection FP16 - Device: CPU AVX512_CORE_AMX AVX512_CORE 4 8 12 16 20 SE +/- 0.03, N = 3 SE +/- 0.01, N = 3 15.27 17.58 MIN: 8.47 / MAX: 83.39 MIN: 11.53 / MAX: 73.39 1. (CXX) g++ options: -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -pie
OpenVINO CPU Peak Freq (Highest CPU Core Frequency) Monitor Min Avg Max AVX512_CORE_AMX 1900 2460 4300 AVX512_CORE 1900 2600 4363 OpenBenchmarking.org Megahertz, More Is Better OpenVINO 2023.2.dev CPU Peak Freq (Highest CPU Core Frequency) Monitor 1100 2200 3300 4400 5500
OpenVINO CPU Power Consumption Monitor Min Avg Max AVX512_CORE_AMX 98.3 435.9 476.2 AVX512_CORE 102.8 409.7 442.2 OpenBenchmarking.org Watts, Fewer Is Better OpenVINO 2023.2.dev CPU Power Consumption Monitor 120 240 360 480 600
OpenVINO CPU Temperature Monitor Min Avg Max AVX512_CORE_AMX 44.0 70.7 75.0 AVX512_CORE 44.0 66.5 71.0 OpenBenchmarking.org Celsius, Fewer Is Better OpenVINO 2023.2.dev CPU Temperature Monitor 20 40 60 80 100
OpenVINO Model: Face Detection Retail FP16 - Device: CPU OpenBenchmarking.org FPS, More Is Better OpenVINO 2023.2.dev Model: Face Detection Retail FP16 - Device: CPU AVX512_CORE_AMX AVX512_CORE 1400 2800 4200 5600 7000 SE +/- 6.91, N = 3 SE +/- 3.93, N = 3 6610.96 3153.69 1. (CXX) g++ options: -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -pie
OpenVINO Model: Face Detection Retail FP16 - Device: CPU OpenBenchmarking.org ms, Fewer Is Better OpenVINO 2023.2.dev Model: Face Detection Retail FP16 - Device: CPU AVX512_CORE_AMX AVX512_CORE 2 4 6 8 10 SE +/- 0.01, N = 3 SE +/- 0.01, N = 3 8.45 8.87 MIN: 4.69 / MAX: 315.08 MIN: 4.92 / MAX: 63.24 1. (CXX) g++ options: -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -pie
OpenVINO CPU Peak Freq (Highest CPU Core Frequency) Monitor Min Avg Max AVX512_CORE_AMX 1900 2397 3397 AVX512_CORE 1900 2488 2901 OpenBenchmarking.org Megahertz, More Is Better OpenVINO 2023.2.dev CPU Peak Freq (Highest CPU Core Frequency) Monitor 800 1600 2400 3200 4000
OpenVINO CPU Power Consumption Monitor Min Avg Max AVX512_CORE_AMX 93.8 451.7 490.4 AVX512_CORE 113.1 433.1 463.4 OpenBenchmarking.org Watts, Fewer Is Better OpenVINO 2023.2.dev CPU Power Consumption Monitor 130 260 390 520 650
OpenVINO CPU Temperature Monitor Min Avg Max AVX512_CORE_AMX 39.0 71.4 78.0 AVX512_CORE 42.0 69.3 73.0 OpenBenchmarking.org Celsius, Fewer Is Better OpenVINO 2023.2.dev CPU Temperature Monitor 20 40 60 80 100
OpenVINO Model: Face Detection Retail FP16-INT8 - Device: CPU OpenBenchmarking.org FPS, More Is Better OpenVINO 2023.2.dev Model: Face Detection Retail FP16-INT8 - Device: CPU AVX512_CORE_AMX AVX512_CORE 2K 4K 6K 8K 10K SE +/- 4.26, N = 3 SE +/- 15.03, N = 3 10083.57 11238.54 1. (CXX) g++ options: -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -pie
OpenVINO Model: Face Detection Retail FP16-INT8 - Device: CPU OpenBenchmarking.org ms, Fewer Is Better OpenVINO 2023.2.dev Model: Face Detection Retail FP16-INT8 - Device: CPU AVX512_CORE_AMX AVX512_CORE 1.2465 2.493 3.7395 4.986 6.2325 SE +/- 0.00, N = 3 SE +/- 0.01, N = 3 5.54 4.96 MIN: 3.48 / MAX: 64.43 MIN: 2.8 / MAX: 86.65 1. (CXX) g++ options: -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -pie
OpenVINO CPU Peak Freq (Highest CPU Core Frequency) Monitor Min Avg Max AVX512_CORE_AMX 1900 2376 3703 AVX512_CORE 1900 2754 4298 OpenBenchmarking.org Megahertz, More Is Better OpenVINO 2023.2.dev CPU Peak Freq (Highest CPU Core Frequency) Monitor 1100 2200 3300 4400 5500
OpenVINO CPU Power Consumption Monitor Min Avg Max AVX512_CORE_AMX 95.3 440.4 485.2 AVX512_CORE 101.4 494.4 542.9 OpenBenchmarking.org Watts, Fewer Is Better OpenVINO 2023.2.dev CPU Power Consumption Monitor 140 280 420 560 700
OpenVINO CPU Temperature Monitor Min Avg Max AVX512_CORE_AMX 44.0 71.3 75.0 AVX512_CORE 42.0 75.6 81.0 OpenBenchmarking.org Celsius, Fewer Is Better OpenVINO 2023.2.dev CPU Temperature Monitor 20 40 60 80 100
OpenVINO Model: Handwritten English Recognition FP16 - Device: CPU OpenBenchmarking.org FPS, More Is Better OpenVINO 2023.2.dev Model: Handwritten English Recognition FP16 - Device: CPU AVX512_CORE_AMX AVX512_CORE 300 600 900 1200 1500 SE +/- 2.41, N = 3 SE +/- 0.11, N = 3 1597.47 844.59 1. (CXX) g++ options: -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -pie
OpenVINO Model: Handwritten English Recognition FP16 - Device: CPU OpenBenchmarking.org ms, Fewer Is Better OpenVINO 2023.2.dev Model: Handwritten English Recognition FP16 - Device: CPU AVX512_CORE_AMX AVX512_CORE 15 30 45 60 75 SE +/- 0.05, N = 3 SE +/- 0.01, N = 3 35.03 66.26 MIN: 20.47 / MAX: 91.06 MIN: 40.67 / MAX: 120.74 1. (CXX) g++ options: -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -pie
OpenVINO CPU Peak Freq (Highest CPU Core Frequency) Monitor Min Avg Max AVX512_CORE_AMX 1900 2610 4200 AVX512_CORE 1900 2522 3189 OpenBenchmarking.org Megahertz, More Is Better OpenVINO 2023.2.dev CPU Peak Freq (Highest CPU Core Frequency) Monitor 1100 2200 3300 4400 5500
OpenVINO CPU Power Consumption Monitor Min Avg Max AVX512_CORE_AMX 94.8 426.6 461.1 AVX512_CORE 48.2 428.0 460.3 OpenBenchmarking.org Watts, Fewer Is Better OpenVINO 2023.2.dev CPU Power Consumption Monitor 120 240 360 480 600
OpenVINO CPU Temperature Monitor Min Avg Max AVX512_CORE_AMX 41.0 69.6 75.0 AVX512_CORE 44.0 68.1 72.0 OpenBenchmarking.org Celsius, Fewer Is Better OpenVINO 2023.2.dev CPU Temperature Monitor 20 40 60 80 100
OpenVINO Model: Handwritten English Recognition FP16-INT8 - Device: CPU OpenBenchmarking.org FPS, More Is Better OpenVINO 2023.2.dev Model: Handwritten English Recognition FP16-INT8 - Device: CPU AVX512_CORE_AMX AVX512_CORE 300 600 900 1200 1500 SE +/- 1.40, N = 3 SE +/- 3.11, N = 3 1244.36 886.14 1. (CXX) g++ options: -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -pie
OpenVINO Model: Handwritten English Recognition FP16-INT8 - Device: CPU OpenBenchmarking.org ms, Fewer Is Better OpenVINO 2023.2.dev Model: Handwritten English Recognition FP16-INT8 - Device: CPU AVX512_CORE_AMX AVX512_CORE 14 28 42 56 70 SE +/- 0.05, N = 3 SE +/- 0.22, N = 3 44.97 63.15 MIN: 27.58 / MAX: 116.58 MIN: 48.78 / MAX: 123.42 1. (CXX) g++ options: -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -pie
OpenVINO CPU Peak Freq (Highest CPU Core Frequency) Monitor Min Avg Max AVX512_CORE_AMX 1900 2743 3382 AVX512_CORE 1900 2745 2900 OpenBenchmarking.org Megahertz, More Is Better OpenVINO 2023.2.dev CPU Peak Freq (Highest CPU Core Frequency) Monitor 800 1600 2400 3200 4000
OpenVINO CPU Power Consumption Monitor Min Avg Max AVX512_CORE_AMX 96.4 418.7 451.0 AVX512_CORE 96.2 429.6 465.9 OpenBenchmarking.org Watts, Fewer Is Better OpenVINO 2023.2.dev CPU Power Consumption Monitor 120 240 360 480 600
OpenVINO CPU Temperature Monitor Min Avg Max AVX512_CORE_AMX 44.0 69.9 74.0 AVX512_CORE 40.0 67.6 73.0 OpenBenchmarking.org Celsius, Fewer Is Better OpenVINO 2023.2.dev CPU Temperature Monitor 20 40 60 80 100
OpenVINO Model: Road Segmentation ADAS FP16 - Device: CPU OpenBenchmarking.org FPS, More Is Better OpenVINO 2023.2.dev Model: Road Segmentation ADAS FP16 - Device: CPU AVX512_CORE_AMX AVX512_CORE 200 400 600 800 1000 SE +/- 0.30, N = 3 SE +/- 0.17, N = 3 807.46 540.69 1. (CXX) g++ options: -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -pie
OpenVINO Model: Road Segmentation ADAS FP16 - Device: CPU OpenBenchmarking.org ms, Fewer Is Better OpenVINO 2023.2.dev Model: Road Segmentation ADAS FP16 - Device: CPU AVX512_CORE_AMX AVX512_CORE 12 24 36 48 60 SE +/- 0.01, N = 3 SE +/- 0.02, N = 3 34.62 51.73 MIN: 18.6 / MAX: 100.86 MIN: 27.75 / MAX: 106.93 1. (CXX) g++ options: -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -pie
OpenVINO CPU Peak Freq (Highest CPU Core Frequency) Monitor Min Avg Max AVX512_CORE_AMX 1900 2694 4232 AVX512_CORE 1900 2569 4200 OpenBenchmarking.org Megahertz, More Is Better OpenVINO 2023.2.dev CPU Peak Freq (Highest CPU Core Frequency) Monitor 1100 2200 3300 4400 5500
OpenVINO CPU Power Consumption Monitor Min Avg Max AVX512_CORE_AMX 94.0 465.6 507.3 AVX512_CORE 96.2 444.3 478.4 OpenBenchmarking.org Watts, Fewer Is Better OpenVINO 2023.2.dev CPU Power Consumption Monitor 130 260 390 520 650
OpenVINO CPU Temperature Monitor Min Avg Max AVX512_CORE_AMX 41.0 72.1 78.0 AVX512_CORE 42.0 68.9 73.0 OpenBenchmarking.org Celsius, Fewer Is Better OpenVINO 2023.2.dev CPU Temperature Monitor 20 40 60 80 100
OpenVINO Model: Road Segmentation ADAS FP16-INT8 - Device: CPU OpenBenchmarking.org FPS, More Is Better OpenVINO 2023.2.dev Model: Road Segmentation ADAS FP16-INT8 - Device: CPU AVX512_CORE_AMX AVX512_CORE 300 600 900 1200 1500 SE +/- 1.67, N = 3 SE +/- 0.84, N = 3 1129.41 1187.75 1. (CXX) g++ options: -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -pie
OpenVINO Model: Road Segmentation ADAS FP16-INT8 - Device: CPU OpenBenchmarking.org ms, Fewer Is Better OpenVINO 2023.2.dev Model: Road Segmentation ADAS FP16-INT8 - Device: CPU AVX512_CORE_AMX AVX512_CORE 11 22 33 44 55 SE +/- 0.07, N = 3 SE +/- 0.01, N = 3 49.52 23.47 MIN: 25.16 / MAX: 117.59 MIN: 13.03 / MAX: 80.04 1. (CXX) g++ options: -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -pie
OpenVINO CPU Peak Freq (Highest CPU Core Frequency) Monitor Min Avg Max AVX512_CORE_AMX 1900 2670 4300 AVX512_CORE 1900 2638 4222 OpenBenchmarking.org Megahertz, More Is Better OpenVINO 2023.2.dev CPU Peak Freq (Highest CPU Core Frequency) Monitor 1100 2200 3300 4400 5500
OpenVINO CPU Power Consumption Monitor Min Avg Max AVX512_CORE_AMX 98.4 443.4 480.6 AVX512_CORE 98.1 443.0 479.7 OpenBenchmarking.org Watts, Fewer Is Better OpenVINO 2023.2.dev CPU Power Consumption Monitor 120 240 360 480 600
OpenVINO CPU Temperature Monitor Min Avg Max AVX512_CORE_AMX 45.0 71.1 75.0 AVX512_CORE 39.0 67.6 71.0 OpenBenchmarking.org Celsius, Fewer Is Better OpenVINO 2023.2.dev CPU Temperature Monitor 20 40 60 80 100
Phoronix Test Suite v10.8.4