Benchmarks by Michael Larabel for a future article.
Compare your own system(s) to this result file with the
Phoronix Test Suite by running the command:
phoronix-test-suite benchmark 2301160-NE-OPENVINOA07
OpenVINO AMX Benchmarks,
"OpenVINO 2022.3 - Model: Face Detection FP16-INT8 - Device: CPU",
Higher Results Are Better
"AVX512_CORE_AMX",279.92,279.46,287.36
"AVX512_CORE_FP16",145.5,146.35,146.38
"AVX512_CORE_BF16",146.32,145.62,146.25
"AVX512_CORE_VNNI",146.6,145.89,145.58
"AVX512_CORE",63.26,63.16,63.2
"OpenVINO 2022.3 - Model: Face Detection FP16-INT8 - Device: CPU",
Lower Results Are Better
"AVX512_CORE_AMX",427.47,428.34,416.68
"AVX512_CORE_FP16",411.42,408.97,409.03
"AVX512_CORE_BF16",409.12,411.13,409.27
"AVX512_CORE_VNNI",408.4,410.32,411.13
"AVX512_CORE",943.09,943.09,942.2
"OpenVINO 2022.3 - CPU Peak Freq (Highest CPU Core Frequency) Monitor",
Higher Results Are Better
"AVX512_CORE_AMX",
"AVX512_CORE_FP16",
"AVX512_CORE_BF16",
"AVX512_CORE_VNNI",
"AVX512_CORE",
"OpenVINO 2022.3 - CPU Power Consumption Monitor",
Lower Results Are Better
"AVX512_CORE_AMX",
"AVX512_CORE_FP16",
"AVX512_CORE_BF16",
"AVX512_CORE_VNNI",
"AVX512_CORE",
"OpenVINO 2022.3 - CPU Temperature Monitor",
Lower Results Are Better
"AVX512_CORE_AMX",
"AVX512_CORE_FP16",
"AVX512_CORE_BF16",
"AVX512_CORE_VNNI",
"AVX512_CORE",
"OpenVINO 2022.3 - Model: Weld Porosity Detection FP16-INT8 - Device: CPU",
Higher Results Are Better
"AVX512_CORE_AMX",23199.14,23111.69,22889.05
"AVX512_CORE_FP16",8945.56,8866.78,8878.97
"AVX512_CORE_BF16",9089.19,8823.84,8842.84
"AVX512_CORE_VNNI",9010.78,8878.64,8837.74
"AVX512_CORE",5953.59,5687.62,5864.97
"OpenVINO 2022.3 - Model: Weld Porosity Detection FP16-INT8 - Device: CPU",
Lower Results Are Better
"AVX512_CORE_AMX",5.15,5.17,5.22
"AVX512_CORE_FP16",13.39,13.51,13.49
"AVX512_CORE_BF16",13.18,13.57,13.54
"AVX512_CORE_VNNI",13.29,13.49,13.55
"AVX512_CORE",20.13,21.07,20.43
"OpenVINO 2022.3 - CPU Peak Freq (Highest CPU Core Frequency) Monitor",
Higher Results Are Better
"AVX512_CORE_AMX",
"AVX512_CORE_FP16",
"AVX512_CORE_BF16",
"AVX512_CORE_VNNI",
"AVX512_CORE",
"OpenVINO 2022.3 - CPU Power Consumption Monitor",
Lower Results Are Better
"AVX512_CORE_AMX",
"AVX512_CORE_FP16",
"AVX512_CORE_BF16",
"AVX512_CORE_VNNI",
"AVX512_CORE",
"OpenVINO 2022.3 - CPU Temperature Monitor",
Lower Results Are Better
"AVX512_CORE_AMX",
"AVX512_CORE_FP16",
"AVX512_CORE_BF16",
"AVX512_CORE_VNNI",
"AVX512_CORE",
"OpenVINO 2022.3 - Model: Person Vehicle Bike Detection FP16 - Device: CPU",
Higher Results Are Better
"AVX512_CORE_AMX",4805.85,4773.35,4755.83
"AVX512_CORE_FP16",2750.42,2727.87,2711.92
"AVX512_CORE_BF16",2734.02,2781.41,2786.05
"AVX512_CORE_VNNI",1830.81,1806.7,1825.76
"AVX512_CORE",1836.65,1817,1807.15
"OpenVINO 2022.3 - Model: Person Vehicle Bike Detection FP16 - Device: CPU",
Lower Results Are Better
"AVX512_CORE_AMX",24.95,25.12,25.21
"AVX512_CORE_FP16",21.79,21.97,22.1
"AVX512_CORE_BF16",21.92,21.55,21.51
"AVX512_CORE_VNNI",32.75,33.19,32.84
"AVX512_CORE",32.65,33,33.18
"OpenVINO 2022.3 - CPU Peak Freq (Highest CPU Core Frequency) Monitor",
Higher Results Are Better
"AVX512_CORE_AMX",
"AVX512_CORE_FP16",
"AVX512_CORE_BF16",
"AVX512_CORE_VNNI",
"AVX512_CORE",
"OpenVINO 2022.3 - CPU Power Consumption Monitor",
Lower Results Are Better
"AVX512_CORE_AMX",
"AVX512_CORE_FP16",
"AVX512_CORE_BF16",
"AVX512_CORE_VNNI",
"AVX512_CORE",
"OpenVINO 2022.3 - CPU Temperature Monitor",
Lower Results Are Better
"AVX512_CORE_AMX",
"AVX512_CORE_FP16",
"AVX512_CORE_BF16",
"AVX512_CORE_VNNI",
"AVX512_CORE",
"OpenVINO 2022.3 - Model: Machine Translation EN To DE FP16 - Device: CPU",
Higher Results Are Better
"AVX512_CORE_AMX",726.7,731.53,714.06
"AVX512_CORE_FP16",321.21,319.99,322.65
"AVX512_CORE_BF16",318.03,308.57,317.22
"AVX512_CORE_VNNI",391.9,387.7,383.26
"AVX512_CORE",387.31,369.12,379.73
"OpenVINO 2022.3 - Model: Machine Translation EN To DE FP16 - Device: CPU",
Lower Results Are Better
"AVX512_CORE_AMX",82.52,81.91,83.94
"AVX512_CORE_FP16",186.56,187.3,185.74
"AVX512_CORE_BF16",188.39,194.17,188.91
"AVX512_CORE_VNNI",152.75,154.54,156.36
"AVX512_CORE",154.68,162.33,157.78
"OpenVINO 2022.3 - CPU Peak Freq (Highest CPU Core Frequency) Monitor",
Higher Results Are Better
"AVX512_CORE_AMX",
"AVX512_CORE_FP16",
"AVX512_CORE_BF16",
"AVX512_CORE_VNNI",
"AVX512_CORE",
"OpenVINO 2022.3 - CPU Power Consumption Monitor",
Lower Results Are Better
"AVX512_CORE_AMX",
"AVX512_CORE_FP16",
"AVX512_CORE_BF16",
"AVX512_CORE_VNNI",
"AVX512_CORE",
"OpenVINO 2022.3 - CPU Temperature Monitor",
Lower Results Are Better
"AVX512_CORE_AMX",
"AVX512_CORE_FP16",
"AVX512_CORE_BF16",
"AVX512_CORE_VNNI",
"AVX512_CORE",