openvino-1.4.0-amx

Intel Xeon w9-3495X testing with a ASUS Pro WS W790E-SAGE SE (0705 BIOS) and llvmpipe on Ubuntu 22.04 via the Phoronix Test Suite.

Compare your own system(s) to this result file with the Phoronix Test Suite by running the command: phoronix-test-suite benchmark 2311114-NE-2311110NE81
Jump To Table - Results

View

Do Not Show Noisy Results
Do Not Show Results With Incomplete Data
Do Not Show Results With Little Change/Spread
List Notable Results

Statistics

Show Overall Harmonic Mean(s)
Show Overall Geometric Mean
Show Wins / Losses Counts (Pie Chart)
Normalize Results
Remove Outliers Before Calculating Averages

Graph Settings

Force Line Graphs Where Applicable
Convert To Scalar Where Applicable
Prefer Vertical Bar Graphs
No Box Plots
On Line Graphs With Missing Data, Connect The Line Gaps

Multi-Way Comparison

Condense Multi-Option Tests Into Single Result Graphs
Condense Test Profiles With Multiple Version Results Into Single Result Graphs

Table

Show Detailed System Result Table

Run Management

Highlight
Result
Hide
Result
Result
Identifier
View Logs
Performance Per
Dollar
Date
Run
  Test
  Duration
AVX512_CORE_AMX
November 11 2023
  1 Hour, 45 Minutes
AVX512_CORE
November 12 2023
  1 Hour, 45 Minutes
Invert Hiding All Results Option
  1 Hour, 45 Minutes
Only show results matching title/arguments (delimit multiple options with a comma):
Do not show results matching title/arguments (delimit multiple options with a comma):

AVX512_CORE_AMX

Processor: Intel Xeon w9-3495X @ 4.80GHz (56 Cores / 112 Threads), Motherboard: ASUS Pro WS W790E-SAGE SE (0705 BIOS), Chipset: Intel Device 7aa7, Memory: 8 x 32 GB DDR5-4812MT/s Hynix HMCG88AEBRA115N, Disk: 6401GB Micron_9300_MTFDHAL6T4TDR + 0GB Virtual HDisk0, Graphics: llvmpipe (210/405MHz), Audio: Realtek ALC1220, Network: 2 x Intel X710 for 10GBASE-T

OS: Ubuntu 22.04, Kernel: 6.3.0-060300-generic (x86_64), Desktop: GNOME Shell 42.9, Display Server: X Server 1.21.1.4, Display Driver: NVIDIA 535.129.03, OpenGL: 4.5 Mesa 23.0.4-0ubuntu1~22.04.1 (LLVM 15.0.7 256 bits), OpenCL: OpenCL 3.0 CUDA 12.2.147, Vulkan: 1.3.242, Compiler: GCC 11.4.0 + CUDA 12.1, File-System: ext4, Screen Resolution: 1920x1200

Kernel Notes: Transparent Huge Pages: madvise
Compiler Notes: --build=x86_64-linux-gnu --disable-vtable-verify --disable-werror --enable-bootstrap --enable-cet --enable-checking=release --enable-clocale=gnu --enable-default-pie --enable-gnu-unique-object --enable-languages=c,ada,c++,go,brig,d,fortran,objc,obj-c++,m2 --enable-libphobos-checking=release --enable-libstdcxx-debug --enable-libstdcxx-time=yes --enable-link-serialization=2 --enable-multiarch --enable-multilib --enable-nls --enable-objc-gc=auto --enable-offload-targets=nvptx-none=/build/gcc-11-XeT9lY/gcc-11-11.4.0/debian/tmp-nvptx/usr,amdgcn-amdhsa=/build/gcc-11-XeT9lY/gcc-11-11.4.0/debian/tmp-gcn/usr --enable-plugin --enable-shared --enable-threads=posix --host=x86_64-linux-gnu --program-prefix=x86_64-linux-gnu- --target=x86_64-linux-gnu --with-abi=m64 --with-arch-32=i686 --with-build-config=bootstrap-lto-lean --with-default-libstdcxx-abi=new --with-gcc-major-version-only --with-multilib-list=m32,m64,mx32 --with-target-system-zlib=auto --with-tune=generic --without-cuda-driver -v
Processor Notes: Scaling Governor: intel_pstate powersave (EPP: performance) - CPU Microcode: 0x2b0004b1
Python Notes: Python 3.10.12
Security Notes: itlb_multihit: Not affected + l1tf: Not affected + mds: Not affected + meltdown: Not affected + mmio_stale_data: Not affected + retbleed: Not affected + spec_store_bypass: Mitigation of SSB disabled via prctl + spectre_v1: Mitigation of usercopy/swapgs barriers and __user pointer sanitization + spectre_v2: Mitigation of Enhanced / Automatic IBRS IBPB: conditional RSB filling PBRSB-eIBRS: SW sequence + srbds: Not affected + tsx_async_abort: Not affected

AVX512_CORE

Changed Disk to 6401GB Micron_9300_MTFDHAL6T4TDR + 4097GB Lexar SSD ARES 4TB + 0GB Virtual HDisk0.

openvino-1.4.0-amxOpenBenchmarking.orgPhoronix Test SuiteIntel Xeon w9-3495X @ 4.80GHz (56 Cores / 112 Threads)ASUS Pro WS W790E-SAGE SE (0705 BIOS)Intel Device 7aa78 x 32 GB DDR5-4812MT/s Hynix HMCG88AEBRA115N6401GB Micron_9300_MTFDHAL6T4TDR + 0GB Virtual HDisk06401GB Micron_9300_MTFDHAL6T4TDR + 4097GB Lexar SSD ARES 4TB + 0GB Virtual HDisk0llvmpipe (210/405MHz)Realtek ALC12202 x Intel X710 for 10GBASE-TUbuntu 22.046.3.0-060300-generic (x86_64)GNOME Shell 42.9X Server 1.21.1.4NVIDIA 535.129.034.5 Mesa 23.0.4-0ubuntu1~22.04.1 (LLVM 15.0.7 256 bits)OpenCL 3.0 CUDA 12.2.1471.3.242GCC 11.4.0 + CUDA 12.1ext41920x1200ProcessorMotherboardChipsetMemoryDisksGraphicsAudioNetworkOSKernelDesktopDisplay ServerDisplay DriverOpenGLOpenCLVulkanCompilerFile-SystemScreen ResolutionOpenvino-1.4.0-amx BenchmarksSystem Logs- Transparent Huge Pages: madvise- --build=x86_64-linux-gnu --disable-vtable-verify --disable-werror --enable-bootstrap --enable-cet --enable-checking=release --enable-clocale=gnu --enable-default-pie --enable-gnu-unique-object --enable-languages=c,ada,c++,go,brig,d,fortran,objc,obj-c++,m2 --enable-libphobos-checking=release --enable-libstdcxx-debug --enable-libstdcxx-time=yes --enable-link-serialization=2 --enable-multiarch --enable-multilib --enable-nls --enable-objc-gc=auto --enable-offload-targets=nvptx-none=/build/gcc-11-XeT9lY/gcc-11-11.4.0/debian/tmp-nvptx/usr,amdgcn-amdhsa=/build/gcc-11-XeT9lY/gcc-11-11.4.0/debian/tmp-gcn/usr --enable-plugin --enable-shared --enable-threads=posix --host=x86_64-linux-gnu --program-prefix=x86_64-linux-gnu- --target=x86_64-linux-gnu --with-abi=m64 --with-arch-32=i686 --with-build-config=bootstrap-lto-lean --with-default-libstdcxx-abi=new --with-gcc-major-version-only --with-multilib-list=m32,m64,mx32 --with-target-system-zlib=auto --with-tune=generic --without-cuda-driver -v - Scaling Governor: intel_pstate powersave (EPP: performance) - CPU Microcode: 0x2b0004b1 - Python 3.10.12- itlb_multihit: Not affected + l1tf: Not affected + mds: Not affected + meltdown: Not affected + mmio_stale_data: Not affected + retbleed: Not affected + spec_store_bypass: Mitigation of SSB disabled via prctl + spectre_v1: Mitigation of usercopy/swapgs barriers and __user pointer sanitization + spectre_v2: Mitigation of Enhanced / Automatic IBRS IBPB: conditional RSB filling PBRSB-eIBRS: SW sequence + srbds: Not affected + tsx_async_abort: Not affected

AVX512_CORE_AMX vs. AVX512_CORE ComparisonPhoronix Test SuiteBaseline+160.2%+160.2%+320.4%+320.4%+480.6%+480.6%159.5%111%29.6%11.7%11.5%5.2%4.1%W.P.D.F - CPU640.8%W.P.D.F - CPU639.3%F.D.F - CPU529.2%F.D.F - CPU526.7%V.D.F.I - CPUA.G.R.R.0.F - CPU159.3%A.G.R.R.0.F - CPU152.7%P.V.B.D.F - CPU130.1%P.D.F - CPU120.4%P.D.F - CPU120.3%P.D.F - CPU117.7%P.D.F - CPU117.7%R.S.A.F.I - CPUF.D.R.F - CPU109.6%W.P.D.F.I - CPU106%W.P.D.F.I - CPU105.4%F.D.F.I - CPU92.3%H.E.R.F - CPU89.2%H.E.R.F - CPU89.1%V.D.F - CPU78.6%V.D.F - CPU78.4%R.S.A.F - CPU49.4%R.S.A.F - CPU49.3%H.E.R.F.I - CPU40.4%H.E.R.F.I - CPU40.4%A.G.R.R.0.F.I - CPU31.8%A.G.R.R.0.F.I - CPU30.2%V.D.F.I - CPUP.V.B.D.F - CPU15.1%F.D.R.F.I - CPUF.D.R.F.I - CPUR.S.A.F.I - CPUF.D.R.F - CPU5%F.D.F.I - CPUOpenVINOOpenVINOOpenVINOOpenVINOOpenVINOOpenVINOOpenVINOOpenVINOOpenVINOOpenVINOOpenVINOOpenVINOOpenVINOOpenVINOOpenVINOOpenVINOOpenVINOOpenVINOOpenVINOOpenVINOOpenVINOOpenVINOOpenVINOOpenVINOOpenVINOOpenVINOOpenVINOOpenVINOOpenVINOOpenVINOOpenVINOOpenVINOOpenVINOOpenVINOAVX512_CORE_AMXAVX512_CORE

openvino-1.4.0-amxopenvino: Face Detection FP16 - CPUopenvino: Face Detection FP16 - CPUopenvino: Face Detection FP16-INT8 - CPUopenvino: Face Detection FP16-INT8 - CPUopenvino: Age Gender Recognition Retail 0013 FP16 - CPUopenvino: Age Gender Recognition Retail 0013 FP16 - CPUopenvino: Age Gender Recognition Retail 0013 FP16-INT8 - CPUopenvino: Age Gender Recognition Retail 0013 FP16-INT8 - CPUopenvino: Person Detection FP16 - CPUopenvino: Person Detection FP16 - CPUopenvino: Person Detection FP32 - CPUopenvino: Person Detection FP32 - CPUopenvino: Weld Porosity Detection FP16-INT8 - CPUopenvino: Weld Porosity Detection FP16-INT8 - CPUopenvino: Weld Porosity Detection FP16 - CPUopenvino: Weld Porosity Detection FP16 - CPUopenvino: Vehicle Detection FP16-INT8 - CPUopenvino: Vehicle Detection FP16-INT8 - CPUopenvino: Vehicle Detection FP16 - CPUopenvino: Vehicle Detection FP16 - CPUopenvino: Person Vehicle Bike Detection FP16 - CPUopenvino: Person Vehicle Bike Detection FP16 - CPUopenvino: Face Detection Retail FP16 - CPUopenvino: Face Detection Retail FP16 - CPUopenvino: Face Detection Retail FP16-INT8 - CPUopenvino: Face Detection Retail FP16-INT8 - CPUopenvino: Handwritten English Recognition FP16 - CPUopenvino: Handwritten English Recognition FP16 - CPUopenvino: Handwritten English Recognition FP16-INT8 - CPUopenvino: Handwritten English Recognition FP16-INT8 - CPUopenvino: Road Segmentation ADAS FP16 - CPUopenvino: Road Segmentation ADAS FP16 - CPUopenvino: Road Segmentation ADAS FP16-INT8 - CPUopenvino: Road Segmentation ADAS FP16-INT8 - CPUAVX512_CORE_AMXAVX512_CORE77.14362.17165.65337.4991906.180.54108977.140.44289.5796.62289.8996.5217523.903.189417.385.933234.9717.281921.7814.543661.1215.276610.968.4510083.575.541597.4735.031244.3644.97807.4634.621129.4149.5212.262269.7086.16324.2136370.461.483670.380.58131.41212.82133.14210.108530.816.551273.7843.934193.976.661077.2825.971591.0317.583153.698.8711238.544.96844.5966.26886.1463.15540.6951.731187.7523.47OpenBenchmarking.org

CPU Peak Freq (Highest CPU Core Frequency) Monitor

OpenBenchmarking.orgMegahertzCPU Peak Freq (Highest CPU Core Frequency) MonitorPhoronix Test Suite System MonitoringAVX512_COREAVX512_CORE_AMX8001600240032004000Min: 1900 / Avg: 2564.64 / Max: 4586Min: 1900 / Avg: 2523.31 / Max: 4355

CPU Power Consumption Monitor

OpenBenchmarking.orgWattsCPU Power Consumption MonitorPhoronix Test Suite System MonitoringAVX512_COREAVX512_CORE_AMX110220330440550Min: 48.23 / Avg: 428.13 / Max: 642.82Min: 54.4 / Avg: 407.76 / Max: 507.26

CPU Temperature Monitor

OpenBenchmarking.orgCelsiusCPU Temperature MonitorPhoronix Test Suite System MonitoringAVX512_COREAVX512_CORE_AMX1632486480Min: 36 / Avg: 69.48 / Max: 85Min: 36 / Avg: 68.03 / Max: 78

OpenVINO

This is a test of the Intel OpenVINO, a toolkit around neural networks, using its built-in benchmarking support and analyzing the throughput and latency for various models. Learn more via the OpenBenchmarking.org test page.

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2023.2.devModel: Face Detection FP16 - Device: CPUAVX512_COREAVX512_CORE_AMX20406080100SE +/- 0.01, N = 3SE +/- 0.08, N = 312.2677.141. (CXX) g++ options: -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -pie
OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2023.2.devModel: Face Detection FP16 - Device: CPUAVX512_COREAVX512_CORE_AMX1530456075Min: 12.25 / Avg: 12.26 / Max: 12.28Min: 76.99 / Avg: 77.14 / Max: 77.271. (CXX) g++ options: -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -pie

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2023.2.devModel: Face Detection FP16 - Device: CPUAVX512_COREAVX512_CORE_AMX5001000150020002500SE +/- 0.78, N = 3SE +/- 0.35, N = 32269.70362.17MIN: 1653.74 / MAX: 2733.95MIN: 125.7 / MAX: 443.521. (CXX) g++ options: -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -pie
MinAvgMaxAVX512_CORE190026614300AVX512_CORE_AMX190026794330OpenBenchmarking.orgMegahertz, More Is BetterOpenVINO 2023.2.devCPU Peak Freq (Highest CPU Core Frequency) Monitor11002200330044005500
MinAvgMaxAVX512_CORE115.9407.9451.2AVX512_CORE_AMX95.5407.5446.0OpenBenchmarking.orgWatts, Fewer Is BetterOpenVINO 2023.2.devCPU Power Consumption Monitor1202403604806001. AVX512_CORE: Approximate power consumption of 28686 Joules per run.2. AVX512_CORE_AMX: Approximate power consumption of 27575 Joules per run.
MinAvgMaxAVX512_CORE37.065.873.0AVX512_CORE_AMX36.065.971.0OpenBenchmarking.orgCelsius, Fewer Is BetterOpenVINO 2023.2.devCPU Temperature Monitor20406080100
OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2023.2.devModel: Face Detection FP16 - Device: CPUAVX512_COREAVX512_CORE_AMX400800120016002000Min: 2268.56 / Avg: 2269.7 / Max: 2271.19Min: 361.6 / Avg: 362.17 / Max: 362.811. (CXX) g++ options: -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -pie

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2023.2.devModel: Face Detection FP16-INT8 - Device: CPUAVX512_COREAVX512_CORE_AMX4080120160200SE +/- 0.43, N = 3SE +/- 0.07, N = 386.16165.651. (CXX) g++ options: -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -pie
OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2023.2.devModel: Face Detection FP16-INT8 - Device: CPUAVX512_COREAVX512_CORE_AMX306090120150Min: 85.72 / Avg: 86.16 / Max: 87.01Min: 165.54 / Avg: 165.65 / Max: 165.781. (CXX) g++ options: -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -pie

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2023.2.devModel: Face Detection FP16-INT8 - Device: CPUAVX512_COREAVX512_CORE_AMX70140210280350SE +/- 1.55, N = 3SE +/- 0.16, N = 3324.21337.49MIN: 173.46 / MAX: 419.38MIN: 123.15 / MAX: 428.331. (CXX) g++ options: -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -pie
MinAvgMaxAVX512_CORE190025734300AVX512_CORE_AMX190026684305OpenBenchmarking.orgMegahertz, More Is BetterOpenVINO 2023.2.devCPU Peak Freq (Highest CPU Core Frequency) Monitor11002200330044005500
MinAvgMaxAVX512_CORE107.3506.9613.1AVX512_CORE_AMX112.9419.1459.6OpenBenchmarking.orgWatts, Fewer Is BetterOpenVINO 2023.2.devCPU Power Consumption Monitor1603204806408001. AVX512_CORE: Approximate power consumption of 34128 Joules per run.2. AVX512_CORE_AMX: Approximate power consumption of 28638 Joules per run.
MinAvgMaxAVX512_CORE44.079.284.0AVX512_CORE_AMX42.068.874.0OpenBenchmarking.orgCelsius, Fewer Is BetterOpenVINO 2023.2.devCPU Temperature Monitor20406080100
OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2023.2.devModel: Face Detection FP16-INT8 - Device: CPUAVX512_COREAVX512_CORE_AMX60120180240300Min: 321.12 / Avg: 324.21 / Max: 325.81Min: 337.19 / Avg: 337.49 / Max: 337.731. (CXX) g++ options: -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -pie

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2023.2.devModel: Age Gender Recognition Retail 0013 FP16 - Device: CPUAVX512_COREAVX512_CORE_AMX20K40K60K80K100KSE +/- 31.52, N = 3SE +/- 56.21, N = 336370.4691906.181. (CXX) g++ options: -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -pie
OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2023.2.devModel: Age Gender Recognition Retail 0013 FP16 - Device: CPUAVX512_COREAVX512_CORE_AMX16K32K48K64K80KMin: 36315.23 / Avg: 36370.46 / Max: 36424.38Min: 91824.27 / Avg: 91906.18 / Max: 92013.811. (CXX) g++ options: -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -pie

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2023.2.devModel: Age Gender Recognition Retail 0013 FP16 - Device: CPUAVX512_COREAVX512_CORE_AMX0.3150.630.9451.261.575SE +/- 0.00, N = 3SE +/- 0.00, N = 31.400.54MIN: 0.74 / MAX: 52.71MIN: 0.24 / MAX: 55.371. (CXX) g++ options: -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -pie
MinAvgMaxAVX512_CORE190025374258AVX512_CORE_AMX190023914326OpenBenchmarking.orgMegahertz, More Is BetterOpenVINO 2023.2.devCPU Peak Freq (Highest CPU Core Frequency) Monitor11002200330044005500
MinAvgMaxAVX512_CORE102.7432.4464.3AVX512_CORE_AMX104.6408.8436.9OpenBenchmarking.orgWatts, Fewer Is BetterOpenVINO 2023.2.devCPU Power Consumption Monitor1202403604806001. AVX512_CORE: Approximate power consumption of 27819 Joules per run.2. AVX512_CORE_AMX: Approximate power consumption of 26573 Joules per run.
MinAvgMaxAVX512_CORE45.069.474.0AVX512_CORE_AMX44.067.570.0OpenBenchmarking.orgCelsius, Fewer Is BetterOpenVINO 2023.2.devCPU Temperature Monitor20406080100
OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2023.2.devModel: Age Gender Recognition Retail 0013 FP16 - Device: CPUAVX512_COREAVX512_CORE_AMX246810Min: 1.4 / Avg: 1.4 / Max: 1.4Min: 0.54 / Avg: 0.54 / Max: 0.541. (CXX) g++ options: -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -pie

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2023.2.devModel: Age Gender Recognition Retail 0013 FP16-INT8 - Device: CPUAVX512_COREAVX512_CORE_AMX20K40K60K80K100KSE +/- 99.28, N = 3SE +/- 222.39, N = 383670.38108977.141. (CXX) g++ options: -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -pie
OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2023.2.devModel: Age Gender Recognition Retail 0013 FP16-INT8 - Device: CPUAVX512_COREAVX512_CORE_AMX20K40K60K80K100KMin: 83473.09 / Avg: 83670.38 / Max: 83788.47Min: 108645.2 / Avg: 108977.14 / Max: 109399.51. (CXX) g++ options: -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -pie

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2023.2.devModel: Age Gender Recognition Retail 0013 FP16-INT8 - Device: CPUAVX512_COREAVX512_CORE_AMX0.13050.2610.39150.5220.6525SE +/- 0.00, N = 3SE +/- 0.00, N = 30.580.44MIN: 0.31 / MAX: 50.67MIN: 0.22 / MAX: 54.331. (CXX) g++ options: -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -pie
MinAvgMaxAVX512_CORE190025694426AVX512_CORE_AMX190023754300OpenBenchmarking.orgMegahertz, More Is BetterOpenVINO 2023.2.devCPU Peak Freq (Highest CPU Core Frequency) Monitor11002200330044005500
MinAvgMaxAVX512_CORE57.3435.5465.7AVX512_CORE_AMX100.3390.6418.2OpenBenchmarking.orgWatts, Fewer Is BetterOpenVINO 2023.2.devCPU Power Consumption Monitor1202403604806001. AVX512_CORE: Approximate power consumption of 28307 Joules per run.2. AVX512_CORE_AMX: Approximate power consumption of 25386 Joules per run.
MinAvgMaxAVX512_CORE44.069.273.0AVX512_CORE_AMX41.065.470.0OpenBenchmarking.orgCelsius, Fewer Is BetterOpenVINO 2023.2.devCPU Temperature Monitor20406080100
OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2023.2.devModel: Age Gender Recognition Retail 0013 FP16-INT8 - Device: CPUAVX512_COREAVX512_CORE_AMX246810Min: 0.58 / Avg: 0.58 / Max: 0.58Min: 0.43 / Avg: 0.44 / Max: 0.441. (CXX) g++ options: -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -pie

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2023.2.devModel: Person Detection FP16 - Device: CPUAVX512_COREAVX512_CORE_AMX60120180240300SE +/- 0.03, N = 3SE +/- 0.19, N = 3131.41289.571. (CXX) g++ options: -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -pie
OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2023.2.devModel: Person Detection FP16 - Device: CPUAVX512_COREAVX512_CORE_AMX50100150200250Min: 131.36 / Avg: 131.41 / Max: 131.46Min: 289.19 / Avg: 289.57 / Max: 289.831. (CXX) g++ options: -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -pie

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2023.2.devModel: Person Detection FP16 - Device: CPUAVX512_COREAVX512_CORE_AMX50100150200250SE +/- 0.06, N = 3SE +/- 0.07, N = 3212.8296.62MIN: 114.27 / MAX: 308.21MIN: 31.84 / MAX: 1901. (CXX) g++ options: -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -pie
MinAvgMaxAVX512_CORE190026644385AVX512_CORE_AMX190026914355OpenBenchmarking.orgMegahertz, More Is BetterOpenVINO 2023.2.devCPU Peak Freq (Highest CPU Core Frequency) Monitor11002200330044005500
MinAvgMaxAVX512_CORE99.1424.2462.2AVX512_CORE_AMX100.4431.9468.9OpenBenchmarking.orgWatts, Fewer Is BetterOpenVINO 2023.2.devCPU Power Consumption Monitor1202403604806001. AVX512_CORE: Approximate power consumption of 28139 Joules per run.2. AVX512_CORE_AMX: Approximate power consumption of 28363 Joules per run.
MinAvgMaxAVX512_CORE42.067.773.0AVX512_CORE_AMX43.069.073.0OpenBenchmarking.orgCelsius, Fewer Is BetterOpenVINO 2023.2.devCPU Temperature Monitor20406080100
OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2023.2.devModel: Person Detection FP16 - Device: CPUAVX512_COREAVX512_CORE_AMX4080120160200Min: 212.74 / Avg: 212.82 / Max: 212.93Min: 96.53 / Avg: 96.62 / Max: 96.751. (CXX) g++ options: -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -pie

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2023.2.devModel: Person Detection FP32 - Device: CPUAVX512_COREAVX512_CORE_AMX60120180240300SE +/- 1.70, N = 3SE +/- 0.35, N = 3133.14289.891. (CXX) g++ options: -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -pie
OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2023.2.devModel: Person Detection FP32 - Device: CPUAVX512_COREAVX512_CORE_AMX50100150200250Min: 131.42 / Avg: 133.14 / Max: 136.55Min: 289.39 / Avg: 289.89 / Max: 290.551. (CXX) g++ options: -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -pie

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2023.2.devModel: Person Detection FP32 - Device: CPUAVX512_COREAVX512_CORE_AMX50100150200250SE +/- 2.67, N = 3SE +/- 0.11, N = 3210.1096.52MIN: 115.17 / MAX: 637.54MIN: 34.11 / MAX: 190.761. (CXX) g++ options: -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -pie
MinAvgMaxAVX512_CORE190026504586AVX512_CORE_AMX190026824200OpenBenchmarking.orgMegahertz, More Is BetterOpenVINO 2023.2.devCPU Peak Freq (Highest CPU Core Frequency) Monitor12002400360048006000
MinAvgMaxAVX512_CORE103.5429.1466.3AVX512_CORE_AMX59.2434.7475.1OpenBenchmarking.orgWatts, Fewer Is BetterOpenVINO 2023.2.devCPU Power Consumption Monitor1202403604806001. AVX512_CORE: Approximate power consumption of 28176 Joules per run.2. AVX512_CORE_AMX: Approximate power consumption of 28689 Joules per run.
MinAvgMaxAVX512_CORE46.068.872.0AVX512_CORE_AMX43.069.975.0OpenBenchmarking.orgCelsius, Fewer Is BetterOpenVINO 2023.2.devCPU Temperature Monitor20406080100
OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2023.2.devModel: Person Detection FP32 - Device: CPUAVX512_COREAVX512_CORE_AMX4080120160200Min: 204.77 / Avg: 210.1 / Max: 212.79Min: 96.3 / Avg: 96.52 / Max: 96.681. (CXX) g++ options: -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -pie

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2023.2.devModel: Weld Porosity Detection FP16-INT8 - Device: CPUAVX512_COREAVX512_CORE_AMX4K8K12K16K20KSE +/- 120.35, N = 3SE +/- 19.41, N = 38530.8117523.901. (CXX) g++ options: -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -pie
OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2023.2.devModel: Weld Porosity Detection FP16-INT8 - Device: CPUAVX512_COREAVX512_CORE_AMX3K6K9K12K15KMin: 8341.06 / Avg: 8530.81 / Max: 8753.95Min: 17485.88 / Avg: 17523.9 / Max: 17549.691. (CXX) g++ options: -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -pie

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2023.2.devModel: Weld Porosity Detection FP16-INT8 - Device: CPUAVX512_COREAVX512_CORE_AMX246810SE +/- 0.09, N = 3SE +/- 0.00, N = 36.553.18MIN: 3.46 / MAX: 460.84MIN: 1.4 / MAX: 68.791. (CXX) g++ options: -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -pie
MinAvgMaxAVX512_CORE190024903475AVX512_CORE_AMX190023014198OpenBenchmarking.orgMegahertz, More Is BetterOpenVINO 2023.2.devCPU Peak Freq (Highest CPU Core Frequency) Monitor11002200330044005500
MinAvgMaxAVX512_CORE121520643AVX512_CORE_AMX100410442OpenBenchmarking.orgWatts, Fewer Is BetterOpenVINO 2023.2.devCPU Power Consumption Monitor20040060080010001. AVX512_CORE: Approximate power consumption of 33596 Joules per run.2. AVX512_CORE_AMX: Approximate power consumption of 26622 Joules per run.
MinAvgMaxAVX512_CORE44.079.885.0AVX512_CORE_AMX44.068.071.0OpenBenchmarking.orgCelsius, Fewer Is BetterOpenVINO 2023.2.devCPU Temperature Monitor20406080100
OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2023.2.devModel: Weld Porosity Detection FP16-INT8 - Device: CPUAVX512_COREAVX512_CORE_AMX3691215Min: 6.38 / Avg: 6.55 / Max: 6.69Min: 3.17 / Avg: 3.18 / Max: 3.181. (CXX) g++ options: -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -pie

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2023.2.devModel: Weld Porosity Detection FP16 - Device: CPUAVX512_COREAVX512_CORE_AMX2K4K6K8K10KSE +/- 0.25, N = 3SE +/- 1.68, N = 31273.789417.381. (CXX) g++ options: -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -pie
OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2023.2.devModel: Weld Porosity Detection FP16 - Device: CPUAVX512_COREAVX512_CORE_AMX16003200480064008000Min: 1273.34 / Avg: 1273.78 / Max: 1274.2Min: 9414.69 / Avg: 9417.38 / Max: 9420.481. (CXX) g++ options: -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -pie

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2023.2.devModel: Weld Porosity Detection FP16 - Device: CPUAVX512_COREAVX512_CORE_AMX1020304050SE +/- 0.01, N = 3SE +/- 0.00, N = 343.935.93MIN: 22.22 / MAX: 112.86MIN: 2.16 / MAX: 72.841. (CXX) g++ options: -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -pie
MinAvgMaxAVX512_CORE190025204200AVX512_CORE_AMX190024054119OpenBenchmarking.orgMegahertz, More Is BetterOpenVINO 2023.2.devCPU Peak Freq (Highest CPU Core Frequency) Monitor11002200330044005500
MinAvgMaxAVX512_CORE111.4436.9465.5AVX512_CORE_AMX93.9405.3437.1OpenBenchmarking.orgWatts, Fewer Is BetterOpenVINO 2023.2.devCPU Power Consumption Monitor1202403604806001. AVX512_CORE: Approximate power consumption of 28397 Joules per run.2. AVX512_CORE_AMX: Approximate power consumption of 26077 Joules per run.
MinAvgMaxAVX512_CORE47.071.075.0AVX512_CORE_AMX40.067.673.0OpenBenchmarking.orgCelsius, Fewer Is BetterOpenVINO 2023.2.devCPU Temperature Monitor20406080100
OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2023.2.devModel: Weld Porosity Detection FP16 - Device: CPUAVX512_COREAVX512_CORE_AMX918273645Min: 43.92 / Avg: 43.93 / Max: 43.95Min: 5.93 / Avg: 5.93 / Max: 5.931. (CXX) g++ options: -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -pie

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2023.2.devModel: Vehicle Detection FP16-INT8 - Device: CPUAVX512_COREAVX512_CORE_AMX9001800270036004500SE +/- 56.25, N = 3SE +/- 4.47, N = 34193.973234.971. (CXX) g++ options: -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -pie
OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2023.2.devModel: Vehicle Detection FP16-INT8 - Device: CPUAVX512_COREAVX512_CORE_AMX7001400210028003500Min: 4095.33 / Avg: 4193.97 / Max: 4290.15Min: 3226.15 / Avg: 3234.97 / Max: 3240.681. (CXX) g++ options: -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -pie

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2023.2.devModel: Vehicle Detection FP16-INT8 - Device: CPUAVX512_COREAVX512_CORE_AMX48121620SE +/- 0.09, N = 3SE +/- 0.02, N = 36.6617.28MIN: 3.76 / MAX: 73.82MIN: 8.29 / MAX: 106.971. (CXX) g++ options: -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -pie
MinAvgMaxAVX512_CORE190025552909AVX512_CORE_AMX190025014200OpenBenchmarking.orgMegahertz, More Is BetterOpenVINO 2023.2.devCPU Peak Freq (Highest CPU Core Frequency) Monitor11002200330044005500
MinAvgMaxAVX512_CORE113.8522.4604.9AVX512_CORE_AMX94.4456.6499.8OpenBenchmarking.orgWatts, Fewer Is BetterOpenVINO 2023.2.devCPU Power Consumption Monitor1603204806408001. AVX512_CORE: Approximate power consumption of 33606 Joules per run.2. AVX512_CORE_AMX: Approximate power consumption of 29828 Joules per run.
MinAvgMaxAVX512_CORE42.079.284.0AVX512_CORE_AMX42.071.776.0OpenBenchmarking.orgCelsius, Fewer Is BetterOpenVINO 2023.2.devCPU Temperature Monitor20406080100
OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2023.2.devModel: Vehicle Detection FP16-INT8 - Device: CPUAVX512_COREAVX512_CORE_AMX48121620Min: 6.51 / Avg: 6.66 / Max: 6.82Min: 17.25 / Avg: 17.28 / Max: 17.331. (CXX) g++ options: -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -pie

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2023.2.devModel: Vehicle Detection FP16 - Device: CPUAVX512_COREAVX512_CORE_AMX400800120016002000SE +/- 0.86, N = 3SE +/- 0.57, N = 31077.281921.781. (CXX) g++ options: -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -pie
OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2023.2.devModel: Vehicle Detection FP16 - Device: CPUAVX512_COREAVX512_CORE_AMX30060090012001500Min: 1075.65 / Avg: 1077.28 / Max: 1078.56Min: 1920.66 / Avg: 1921.78 / Max: 1922.551. (CXX) g++ options: -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -pie

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2023.2.devModel: Vehicle Detection FP16 - Device: CPUAVX512_COREAVX512_CORE_AMX612182430SE +/- 0.02, N = 3SE +/- 0.01, N = 325.9714.54MIN: 14.87 / MAX: 84.67MIN: 6.89 / MAX: 93.431. (CXX) g++ options: -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -pie
MinAvgMaxAVX512_CORE190025624171AVX512_CORE_AMX190025433789OpenBenchmarking.orgMegahertz, More Is BetterOpenVINO 2023.2.devCPU Peak Freq (Highest CPU Core Frequency) Monitor11002200330044005500
MinAvgMaxAVX512_CORE104.5460.3489.8AVX512_CORE_AMX97.6454.6490.5OpenBenchmarking.orgWatts, Fewer Is BetterOpenVINO 2023.2.devCPU Power Consumption Monitor1302603905206501. AVX512_CORE: Approximate power consumption of 29766 Joules per run.2. AVX512_CORE_AMX: Approximate power consumption of 29397 Joules per run.
MinAvgMaxAVX512_CORE48.073.277.0AVX512_CORE_AMX41.071.778.0OpenBenchmarking.orgCelsius, Fewer Is BetterOpenVINO 2023.2.devCPU Temperature Monitor20406080100
OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2023.2.devModel: Vehicle Detection FP16 - Device: CPUAVX512_COREAVX512_CORE_AMX612182430Min: 25.94 / Avg: 25.97 / Max: 26.01Min: 14.53 / Avg: 14.54 / Max: 14.551. (CXX) g++ options: -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -pie

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2023.2.devModel: Person Vehicle Bike Detection FP16 - Device: CPUAVX512_COREAVX512_CORE_AMX8001600240032004000SE +/- 0.66, N = 3SE +/- 7.35, N = 31591.033661.121. (CXX) g++ options: -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -pie
OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2023.2.devModel: Person Vehicle Bike Detection FP16 - Device: CPUAVX512_COREAVX512_CORE_AMX6001200180024003000Min: 1589.78 / Avg: 1591.03 / Max: 1592.05Min: 3646.68 / Avg: 3661.12 / Max: 3670.731. (CXX) g++ options: -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -pie

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2023.2.devModel: Person Vehicle Bike Detection FP16 - Device: CPUAVX512_COREAVX512_CORE_AMX48121620SE +/- 0.01, N = 3SE +/- 0.03, N = 317.5815.27MIN: 11.53 / MAX: 73.39MIN: 8.47 / MAX: 83.391. (CXX) g++ options: -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -pie
MinAvgMaxAVX512_CORE190026004363AVX512_CORE_AMX190024604300OpenBenchmarking.orgMegahertz, More Is BetterOpenVINO 2023.2.devCPU Peak Freq (Highest CPU Core Frequency) Monitor11002200330044005500
MinAvgMaxAVX512_CORE102.8409.7442.2AVX512_CORE_AMX98.3435.9476.2OpenBenchmarking.orgWatts, Fewer Is BetterOpenVINO 2023.2.devCPU Power Consumption Monitor1202403604806001. AVX512_CORE: Approximate power consumption of 26906 Joules per run.2. AVX512_CORE_AMX: Approximate power consumption of 28769 Joules per run.
MinAvgMaxAVX512_CORE44.066.571.0AVX512_CORE_AMX44.070.775.0OpenBenchmarking.orgCelsius, Fewer Is BetterOpenVINO 2023.2.devCPU Temperature Monitor20406080100
OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2023.2.devModel: Person Vehicle Bike Detection FP16 - Device: CPUAVX512_COREAVX512_CORE_AMX48121620Min: 17.57 / Avg: 17.58 / Max: 17.6Min: 15.23 / Avg: 15.27 / Max: 15.331. (CXX) g++ options: -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -pie

Model: Machine Translation EN To DE FP16 - Device: CPU

AVX512_CORE_AMX: The test quit with a non-zero exit status. The test quit with a non-zero exit status. The test quit with a non-zero exit status.

AVX512_CORE: The test quit with a non-zero exit status. The test quit with a non-zero exit status. The test quit with a non-zero exit status.

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2023.2.devModel: Face Detection Retail FP16 - Device: CPUAVX512_COREAVX512_CORE_AMX14002800420056007000SE +/- 3.93, N = 3SE +/- 6.91, N = 33153.696610.961. (CXX) g++ options: -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -pie
OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2023.2.devModel: Face Detection Retail FP16 - Device: CPUAVX512_COREAVX512_CORE_AMX11002200330044005500Min: 3149.08 / Avg: 3153.69 / Max: 3161.52Min: 6602.02 / Avg: 6610.96 / Max: 6624.561. (CXX) g++ options: -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -pie

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2023.2.devModel: Face Detection Retail FP16 - Device: CPUAVX512_COREAVX512_CORE_AMX246810SE +/- 0.01, N = 3SE +/- 0.01, N = 38.878.45MIN: 4.92 / MAX: 63.24MIN: 4.69 / MAX: 315.081. (CXX) g++ options: -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -pie
MinAvgMaxAVX512_CORE190024882901AVX512_CORE_AMX190023973397OpenBenchmarking.orgMegahertz, More Is BetterOpenVINO 2023.2.devCPU Peak Freq (Highest CPU Core Frequency) Monitor8001600240032004000
MinAvgMaxAVX512_CORE113.1433.1463.4AVX512_CORE_AMX93.8451.7490.4OpenBenchmarking.orgWatts, Fewer Is BetterOpenVINO 2023.2.devCPU Power Consumption Monitor1302603905206501. AVX512_CORE: Approximate power consumption of 28004 Joules per run.2. AVX512_CORE_AMX: Approximate power consumption of 29360 Joules per run.
MinAvgMaxAVX512_CORE42.069.373.0AVX512_CORE_AMX39.071.478.0OpenBenchmarking.orgCelsius, Fewer Is BetterOpenVINO 2023.2.devCPU Temperature Monitor20406080100
OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2023.2.devModel: Face Detection Retail FP16 - Device: CPUAVX512_COREAVX512_CORE_AMX3691215Min: 8.85 / Avg: 8.87 / Max: 8.88Min: 8.43 / Avg: 8.45 / Max: 8.461. (CXX) g++ options: -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -pie

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2023.2.devModel: Face Detection Retail FP16-INT8 - Device: CPUAVX512_COREAVX512_CORE_AMX2K4K6K8K10KSE +/- 15.03, N = 3SE +/- 4.26, N = 311238.5410083.571. (CXX) g++ options: -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -pie
OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2023.2.devModel: Face Detection Retail FP16-INT8 - Device: CPUAVX512_COREAVX512_CORE_AMX2K4K6K8K10KMin: 11215.29 / Avg: 11238.54 / Max: 11266.67Min: 10079.21 / Avg: 10083.57 / Max: 10092.091. (CXX) g++ options: -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -pie

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2023.2.devModel: Face Detection Retail FP16-INT8 - Device: CPUAVX512_COREAVX512_CORE_AMX1.24652.4933.73954.9866.2325SE +/- 0.01, N = 3SE +/- 0.00, N = 34.965.54MIN: 2.8 / MAX: 86.65MIN: 3.48 / MAX: 64.431. (CXX) g++ options: -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -pie
MinAvgMaxAVX512_CORE190027544298AVX512_CORE_AMX190023763703OpenBenchmarking.orgMegahertz, More Is BetterOpenVINO 2023.2.devCPU Peak Freq (Highest CPU Core Frequency) Monitor11002200330044005500
MinAvgMaxAVX512_CORE101.4494.4542.9AVX512_CORE_AMX95.3440.4485.2OpenBenchmarking.orgWatts, Fewer Is BetterOpenVINO 2023.2.devCPU Power Consumption Monitor1402804205607001. AVX512_CORE: Approximate power consumption of 32136 Joules per run.2. AVX512_CORE_AMX: Approximate power consumption of 28917 Joules per run.
MinAvgMaxAVX512_CORE42.075.681.0AVX512_CORE_AMX44.071.375.0OpenBenchmarking.orgCelsius, Fewer Is BetterOpenVINO 2023.2.devCPU Temperature Monitor20406080100
OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2023.2.devModel: Face Detection Retail FP16-INT8 - Device: CPUAVX512_COREAVX512_CORE_AMX246810Min: 4.95 / Avg: 4.96 / Max: 4.97Min: 5.54 / Avg: 5.54 / Max: 5.541. (CXX) g++ options: -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -pie

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2023.2.devModel: Handwritten English Recognition FP16 - Device: CPUAVX512_COREAVX512_CORE_AMX30060090012001500SE +/- 0.11, N = 3SE +/- 2.41, N = 3844.591597.471. (CXX) g++ options: -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -pie
OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2023.2.devModel: Handwritten English Recognition FP16 - Device: CPUAVX512_COREAVX512_CORE_AMX30060090012001500Min: 844.46 / Avg: 844.59 / Max: 844.81Min: 1593.42 / Avg: 1597.47 / Max: 1601.751. (CXX) g++ options: -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -pie

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2023.2.devModel: Handwritten English Recognition FP16 - Device: CPUAVX512_COREAVX512_CORE_AMX1530456075SE +/- 0.01, N = 3SE +/- 0.05, N = 366.2635.03MIN: 40.67 / MAX: 120.74MIN: 20.47 / MAX: 91.061. (CXX) g++ options: -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -pie
MinAvgMaxAVX512_CORE190025223189AVX512_CORE_AMX190026104200OpenBenchmarking.orgMegahertz, More Is BetterOpenVINO 2023.2.devCPU Peak Freq (Highest CPU Core Frequency) Monitor11002200330044005500
MinAvgMaxAVX512_CORE48.2428.0460.3AVX512_CORE_AMX94.8426.6461.1OpenBenchmarking.orgWatts, Fewer Is BetterOpenVINO 2023.2.devCPU Power Consumption Monitor1202403604806001. AVX512_CORE: Approximate power consumption of 27823 Joules per run.2. AVX512_CORE_AMX: Approximate power consumption of 27589 Joules per run.
MinAvgMaxAVX512_CORE44.068.172.0AVX512_CORE_AMX41.069.675.0OpenBenchmarking.orgCelsius, Fewer Is BetterOpenVINO 2023.2.devCPU Temperature Monitor20406080100
OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2023.2.devModel: Handwritten English Recognition FP16 - Device: CPUAVX512_COREAVX512_CORE_AMX1326395265Min: 66.24 / Avg: 66.26 / Max: 66.27Min: 34.94 / Avg: 35.03 / Max: 35.121. (CXX) g++ options: -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -pie

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2023.2.devModel: Handwritten English Recognition FP16-INT8 - Device: CPUAVX512_COREAVX512_CORE_AMX30060090012001500SE +/- 3.11, N = 3SE +/- 1.40, N = 3886.141244.361. (CXX) g++ options: -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -pie
OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2023.2.devModel: Handwritten English Recognition FP16-INT8 - Device: CPUAVX512_COREAVX512_CORE_AMX2004006008001000Min: 880 / Avg: 886.14 / Max: 890.01Min: 1242.17 / Avg: 1244.36 / Max: 1246.981. (CXX) g++ options: -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -pie

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2023.2.devModel: Handwritten English Recognition FP16-INT8 - Device: CPUAVX512_COREAVX512_CORE_AMX1428425670SE +/- 0.22, N = 3SE +/- 0.05, N = 363.1544.97MIN: 48.78 / MAX: 123.42MIN: 27.58 / MAX: 116.581. (CXX) g++ options: -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -pie
MinAvgMaxAVX512_CORE190027452900AVX512_CORE_AMX190027433382OpenBenchmarking.orgMegahertz, More Is BetterOpenVINO 2023.2.devCPU Peak Freq (Highest CPU Core Frequency) Monitor8001600240032004000
MinAvgMaxAVX512_CORE96.2429.6465.9AVX512_CORE_AMX96.4418.7451.0OpenBenchmarking.orgWatts, Fewer Is BetterOpenVINO 2023.2.devCPU Power Consumption Monitor1202403604806001. AVX512_CORE: Approximate power consumption of 27783 Joules per run.2. AVX512_CORE_AMX: Approximate power consumption of 27217 Joules per run.
MinAvgMaxAVX512_CORE40.067.673.0AVX512_CORE_AMX44.069.974.0OpenBenchmarking.orgCelsius, Fewer Is BetterOpenVINO 2023.2.devCPU Temperature Monitor20406080100
OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2023.2.devModel: Handwritten English Recognition FP16-INT8 - Device: CPUAVX512_COREAVX512_CORE_AMX1224364860Min: 62.88 / Avg: 63.15 / Max: 63.59Min: 44.87 / Avg: 44.97 / Max: 45.051. (CXX) g++ options: -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -pie

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2023.2.devModel: Road Segmentation ADAS FP16 - Device: CPUAVX512_COREAVX512_CORE_AMX2004006008001000SE +/- 0.17, N = 3SE +/- 0.30, N = 3540.69807.461. (CXX) g++ options: -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -pie
OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2023.2.devModel: Road Segmentation ADAS FP16 - Device: CPUAVX512_COREAVX512_CORE_AMX140280420560700Min: 540.36 / Avg: 540.69 / Max: 540.87Min: 807.05 / Avg: 807.46 / Max: 808.051. (CXX) g++ options: -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -pie

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2023.2.devModel: Road Segmentation ADAS FP16 - Device: CPUAVX512_COREAVX512_CORE_AMX1224364860SE +/- 0.02, N = 3SE +/- 0.01, N = 351.7334.62MIN: 27.75 / MAX: 106.93MIN: 18.6 / MAX: 100.861. (CXX) g++ options: -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -pie
MinAvgMaxAVX512_CORE190025694200AVX512_CORE_AMX190026944232OpenBenchmarking.orgMegahertz, More Is BetterOpenVINO 2023.2.devCPU Peak Freq (Highest CPU Core Frequency) Monitor11002200330044005500
MinAvgMaxAVX512_CORE96.2444.3478.4AVX512_CORE_AMX94.0465.6507.3OpenBenchmarking.orgWatts, Fewer Is BetterOpenVINO 2023.2.devCPU Power Consumption Monitor1302603905206501. AVX512_CORE: Approximate power consumption of 29030 Joules per run.2. AVX512_CORE_AMX: Approximate power consumption of 30421 Joules per run.
MinAvgMaxAVX512_CORE42.068.973.0AVX512_CORE_AMX41.072.178.0OpenBenchmarking.orgCelsius, Fewer Is BetterOpenVINO 2023.2.devCPU Temperature Monitor20406080100
OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2023.2.devModel: Road Segmentation ADAS FP16 - Device: CPUAVX512_COREAVX512_CORE_AMX1020304050Min: 51.71 / Avg: 51.73 / Max: 51.76Min: 34.59 / Avg: 34.62 / Max: 34.631. (CXX) g++ options: -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -pie

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2023.2.devModel: Road Segmentation ADAS FP16-INT8 - Device: CPUAVX512_COREAVX512_CORE_AMX30060090012001500SE +/- 0.84, N = 3SE +/- 1.67, N = 31187.751129.411. (CXX) g++ options: -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -pie
OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2023.2.devModel: Road Segmentation ADAS FP16-INT8 - Device: CPUAVX512_COREAVX512_CORE_AMX2004006008001000Min: 1186.08 / Avg: 1187.75 / Max: 1188.74Min: 1126.84 / Avg: 1129.41 / Max: 1132.531. (CXX) g++ options: -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -pie

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2023.2.devModel: Road Segmentation ADAS FP16-INT8 - Device: CPUAVX512_COREAVX512_CORE_AMX1122334455SE +/- 0.01, N = 3SE +/- 0.07, N = 323.4749.52MIN: 13.03 / MAX: 80.04MIN: 25.16 / MAX: 117.591. (CXX) g++ options: -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -pie
MinAvgMaxAVX512_CORE190026384222AVX512_CORE_AMX190026704300OpenBenchmarking.orgMegahertz, More Is BetterOpenVINO 2023.2.devCPU Peak Freq (Highest CPU Core Frequency) Monitor11002200330044005500
MinAvgMaxAVX512_CORE98.1443.0479.7AVX512_CORE_AMX98.4443.4480.6OpenBenchmarking.orgWatts, Fewer Is BetterOpenVINO 2023.2.devCPU Power Consumption Monitor1202403604806001. AVX512_CORE: Approximate power consumption of 28940 Joules per run.2. AVX512_CORE_AMX: Approximate power consumption of 29118 Joules per run.
MinAvgMaxAVX512_CORE39.067.671.0AVX512_CORE_AMX45.071.175.0OpenBenchmarking.orgCelsius, Fewer Is BetterOpenVINO 2023.2.devCPU Temperature Monitor20406080100
OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2023.2.devModel: Road Segmentation ADAS FP16-INT8 - Device: CPUAVX512_COREAVX512_CORE_AMX1020304050Min: 23.46 / Avg: 23.47 / Max: 23.5Min: 49.39 / Avg: 49.52 / Max: 49.631. (CXX) g++ options: -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -pie

37 Results Shown

CPU Peak Freq (Highest CPU Core Frequency) Monitor:
  Phoronix Test Suite System Monitoring:
    Megahertz
    Watts
    Celsius
OpenVINO:
  Face Detection FP16 - CPU:
    FPS
    ms
  Face Detection FP16-INT8 - CPU:
    FPS
    ms
  Age Gender Recognition Retail 0013 FP16 - CPU:
    FPS
    ms
  Age Gender Recognition Retail 0013 FP16-INT8 - CPU:
    FPS
    ms
  Person Detection FP16 - CPU:
    FPS
    ms
  Person Detection FP32 - CPU:
    FPS
    ms
  Weld Porosity Detection FP16-INT8 - CPU:
    FPS
    ms
  Weld Porosity Detection FP16 - CPU:
    FPS
    ms
  Vehicle Detection FP16-INT8 - CPU:
    FPS
    ms
  Vehicle Detection FP16 - CPU:
    FPS
    ms
  Person Vehicle Bike Detection FP16 - CPU:
    FPS
    ms
  Face Detection Retail FP16 - CPU:
    FPS
    ms
  Face Detection Retail FP16-INT8 - CPU:
    FPS
    ms
  Handwritten English Recognition FP16 - CPU:
    FPS
    ms
  Handwritten English Recognition FP16-INT8 - CPU:
    FPS
    ms
  Road Segmentation ADAS FP16 - CPU:
    FPS
    ms
  Road Segmentation ADAS FP16-INT8 - CPU:
    FPS
    ms

AVX512_CORE_AMX

Processor: Intel Xeon w9-3495X @ 4.80GHz (56 Cores / 112 Threads), Motherboard: ASUS Pro WS W790E-SAGE SE (0705 BIOS), Chipset: Intel Device 7aa7, Memory: 8 x 32 GB DDR5-4812MT/s Hynix HMCG88AEBRA115N, Disk: 6401GB Micron_9300_MTFDHAL6T4TDR + 0GB Virtual HDisk0, Graphics: llvmpipe (210/405MHz), Audio: Realtek ALC1220, Network: 2 x Intel X710 for 10GBASE-T

OS: Ubuntu 22.04, Kernel: 6.3.0-060300-generic (x86_64), Desktop: GNOME Shell 42.9, Display Server: X Server 1.21.1.4, Display Driver: NVIDIA 535.129.03, OpenGL: 4.5 Mesa 23.0.4-0ubuntu1~22.04.1 (LLVM 15.0.7 256 bits), OpenCL: OpenCL 3.0 CUDA 12.2.147, Vulkan: 1.3.242, Compiler: GCC 11.4.0 + CUDA 12.1, File-System: ext4, Screen Resolution: 1920x1200

Kernel Notes: Transparent Huge Pages: madvise
Compiler Notes: --build=x86_64-linux-gnu --disable-vtable-verify --disable-werror --enable-bootstrap --enable-cet --enable-checking=release --enable-clocale=gnu --enable-default-pie --enable-gnu-unique-object --enable-languages=c,ada,c++,go,brig,d,fortran,objc,obj-c++,m2 --enable-libphobos-checking=release --enable-libstdcxx-debug --enable-libstdcxx-time=yes --enable-link-serialization=2 --enable-multiarch --enable-multilib --enable-nls --enable-objc-gc=auto --enable-offload-targets=nvptx-none=/build/gcc-11-XeT9lY/gcc-11-11.4.0/debian/tmp-nvptx/usr,amdgcn-amdhsa=/build/gcc-11-XeT9lY/gcc-11-11.4.0/debian/tmp-gcn/usr --enable-plugin --enable-shared --enable-threads=posix --host=x86_64-linux-gnu --program-prefix=x86_64-linux-gnu- --target=x86_64-linux-gnu --with-abi=m64 --with-arch-32=i686 --with-build-config=bootstrap-lto-lean --with-default-libstdcxx-abi=new --with-gcc-major-version-only --with-multilib-list=m32,m64,mx32 --with-target-system-zlib=auto --with-tune=generic --without-cuda-driver -v
Processor Notes: Scaling Governor: intel_pstate powersave (EPP: performance) - CPU Microcode: 0x2b0004b1
Python Notes: Python 3.10.12
Security Notes: itlb_multihit: Not affected + l1tf: Not affected + mds: Not affected + meltdown: Not affected + mmio_stale_data: Not affected + retbleed: Not affected + spec_store_bypass: Mitigation of SSB disabled via prctl + spectre_v1: Mitigation of usercopy/swapgs barriers and __user pointer sanitization + spectre_v2: Mitigation of Enhanced / Automatic IBRS IBPB: conditional RSB filling PBRSB-eIBRS: SW sequence + srbds: Not affected + tsx_async_abort: Not affected

Testing initiated at 11 November 2023 23:57 by user root.

AVX512_CORE

Processor: Intel Xeon w9-3495X @ 4.80GHz (56 Cores / 112 Threads), Motherboard: ASUS Pro WS W790E-SAGE SE (0705 BIOS), Chipset: Intel Device 7aa7, Memory: 8 x 32 GB DDR5-4812MT/s Hynix HMCG88AEBRA115N, Disk: 6401GB Micron_9300_MTFDHAL6T4TDR + 4097GB Lexar SSD ARES 4TB + 0GB Virtual HDisk0, Graphics: llvmpipe (210/405MHz), Audio: Realtek ALC1220, Network: 2 x Intel X710 for 10GBASE-T

OS: Ubuntu 22.04, Kernel: 6.3.0-060300-generic (x86_64), Desktop: GNOME Shell 42.9, Display Server: X Server 1.21.1.4, Display Driver: NVIDIA 535.129.03, OpenGL: 4.5 Mesa 23.0.4-0ubuntu1~22.04.1 (LLVM 15.0.7 256 bits), OpenCL: OpenCL 3.0 CUDA 12.2.147, Vulkan: 1.3.242, Compiler: GCC 11.4.0 + CUDA 12.1, File-System: ext4, Screen Resolution: 1920x1200

Kernel Notes: Transparent Huge Pages: madvise
Compiler Notes: --build=x86_64-linux-gnu --disable-vtable-verify --disable-werror --enable-bootstrap --enable-cet --enable-checking=release --enable-clocale=gnu --enable-default-pie --enable-gnu-unique-object --enable-languages=c,ada,c++,go,brig,d,fortran,objc,obj-c++,m2 --enable-libphobos-checking=release --enable-libstdcxx-debug --enable-libstdcxx-time=yes --enable-link-serialization=2 --enable-multiarch --enable-multilib --enable-nls --enable-objc-gc=auto --enable-offload-targets=nvptx-none=/build/gcc-11-XeT9lY/gcc-11-11.4.0/debian/tmp-nvptx/usr,amdgcn-amdhsa=/build/gcc-11-XeT9lY/gcc-11-11.4.0/debian/tmp-gcn/usr --enable-plugin --enable-shared --enable-threads=posix --host=x86_64-linux-gnu --program-prefix=x86_64-linux-gnu- --target=x86_64-linux-gnu --with-abi=m64 --with-arch-32=i686 --with-build-config=bootstrap-lto-lean --with-default-libstdcxx-abi=new --with-gcc-major-version-only --with-multilib-list=m32,m64,mx32 --with-target-system-zlib=auto --with-tune=generic --without-cuda-driver -v
Processor Notes: Scaling Governor: intel_pstate powersave (EPP: performance) - CPU Microcode: 0x2b0004b1
Python Notes: Python 3.10.12
Security Notes: itlb_multihit: Not affected + l1tf: Not affected + mds: Not affected + meltdown: Not affected + mmio_stale_data: Not affected + retbleed: Not affected + spec_store_bypass: Mitigation of SSB disabled via prctl + spectre_v1: Mitigation of usercopy/swapgs barriers and __user pointer sanitization + spectre_v2: Mitigation of Enhanced / Automatic IBRS IBPB: conditional RSB filling PBRSB-eIBRS: SW sequence + srbds: Not affected + tsx_async_abort: Not affected

Testing initiated at 12 November 2023 01:04 by user root.