openvino-2024

Intel Core Ultra 7 155H testing with a MTL Swift SFG14-72T Coral_MTH (V1.01 BIOS) and Intel Arc MTL 8GB on Ubuntu 23.10 via the Phoronix Test Suite.

HTML result view exported from: https://openbenchmarking.org/result/2403078-NE-OPENVINO276&sor&gru.

openvino-2024 ProcessorMotherboardChipsetMemoryDiskGraphicsAudioNetworkOSKernelDesktopDisplay ServerOpenGLCompilerFile-SystemScreen ResolutionabcdeIntel Core Ultra 7 155H @ 4.80GHz (16 Cores / 22 Threads)MTL Swift SFG14-72T Coral_MTH (V1.01 BIOS)Intel Device 7e7f8 x 2GB DRAM-6400MT/s Micron MT62F1G32D2DS-0261024GB Micron_2550_MTFDKBA1T0TGEIntel Arc MTL 8GB (2250MHz)Intel Meteor Lake-P HD AudioIntel Device 7e40Ubuntu 23.106.8.0-060800rc1daily20240126-generic (x86_64)GNOME Shell 45.2X Server 1.21.1.7 + Wayland4.6 Mesa 24.1~git2401200600.ebcab1~oibaf~m (git-ebcab14 2024-01-20 mantic-oibaf-ppa)GCC 13.2.0ext41920x1200OpenBenchmarking.orgKernel Details- Transparent Huge Pages: madviseCompiler Details- --build=x86_64-linux-gnu --disable-vtable-verify --disable-werror --enable-bootstrap --enable-cet --enable-checking=release --enable-clocale=gnu --enable-default-pie --enable-gnu-unique-object --enable-languages=c,ada,c++,go,d,fortran,objc,obj-c++,m2 --enable-libphobos-checking=release --enable-libstdcxx-debug --enable-libstdcxx-time=yes --enable-link-serialization=2 --enable-multiarch --enable-multilib --enable-nls --enable-objc-gc=auto --enable-offload-defaulted --enable-offload-targets=nvptx-none=/build/gcc-13-XYspKM/gcc-13-13.2.0/debian/tmp-nvptx/usr,amdgcn-amdhsa=/build/gcc-13-XYspKM/gcc-13-13.2.0/debian/tmp-gcn/usr --enable-plugin --enable-shared --enable-threads=posix --host=x86_64-linux-gnu --program-prefix=x86_64-linux-gnu- --target=x86_64-linux-gnu --with-abi=m64 --with-arch-32=i686 --with-build-config=bootstrap-lto-lean --with-default-libstdcxx-abi=new --with-gcc-major-version-only --with-multilib-list=m32,m64,mx32 --with-target-system-zlib=auto --with-tune=generic --without-cuda-driver -v Processor Details- Scaling Governor: intel_pstate powersave (EPP: balance_performance) - CPU Microcode: 0x13 - Thermald 2.5.4Python Details- Python 3.11.6Security Details- gather_data_sampling: Not affected + itlb_multihit: Not affected + l1tf: Not affected + mds: Not affected + meltdown: Not affected + mmio_stale_data: Not affected + retbleed: Not affected + spec_rstack_overflow: Not affected + spec_store_bypass: Mitigation of SSB disabled via prctl + spectre_v1: Mitigation of usercopy/swapgs barriers and __user pointer sanitization + spectre_v2: Mitigation of Enhanced / Automatic IBRS IBPB: conditional RSB filling PBRSB-eIBRS: Not affected + srbds: Not affected + tsx_async_abort: Not affected

openvino-2024 openvino: Face Detection FP16 - CPUopenvino: Person Detection FP16 - CPUopenvino: Person Detection FP32 - CPUopenvino: Vehicle Detection FP16 - CPUopenvino: Face Detection FP16-INT8 - CPUopenvino: Face Detection Retail FP16 - CPUopenvino: Road Segmentation ADAS FP16 - CPUopenvino: Vehicle Detection FP16-INT8 - CPUopenvino: Weld Porosity Detection FP16 - CPUopenvino: Face Detection Retail FP16-INT8 - CPUopenvino: Road Segmentation ADAS FP16-INT8 - CPUopenvino: Machine Translation EN To DE FP16 - CPUopenvino: Weld Porosity Detection FP16-INT8 - CPUopenvino: Person Vehicle Bike Detection FP16 - CPUopenvino: Noise Suppression Poconet-Like FP16 - CPUopenvino: Handwritten English Recognition FP16 - CPUopenvino: Person Re-Identification Retail FP16 - CPUopenvino: Age Gender Recognition Retail 0013 FP16 - CPUopenvino: Handwritten English Recognition FP16-INT8 - CPUopenvino: Age Gender Recognition Retail 0013 FP16-INT8 - CPUopenvino: Face Detection FP16 - CPUopenvino: Person Detection FP16 - CPUopenvino: Person Detection FP32 - CPUopenvino: Vehicle Detection FP16 - CPUopenvino: Face Detection FP16-INT8 - CPUopenvino: Face Detection Retail FP16 - CPUopenvino: Road Segmentation ADAS FP16 - CPUopenvino: Vehicle Detection FP16-INT8 - CPUopenvino: Weld Porosity Detection FP16 - CPUopenvino: Face Detection Retail FP16-INT8 - CPUopenvino: Road Segmentation ADAS FP16-INT8 - CPUopenvino: Machine Translation EN To DE FP16 - CPUopenvino: Weld Porosity Detection FP16-INT8 - CPUopenvino: Person Vehicle Bike Detection FP16 - CPUopenvino: Noise Suppression Poconet-Like FP16 - CPUopenvino: Handwritten English Recognition FP16 - CPUopenvino: Person Re-Identification Retail FP16 - CPUopenvino: Age Gender Recognition Retail 0013 FP16 - CPUopenvino: Handwritten English Recognition FP16-INT8 - CPUopenvino: Age Gender Recognition Retail 0013 FP16-INT8 - CPUabcde1.8220.4719.97146.156.33454.1966.70280.81173.861002.41118.8119.67643.51230.73323.3190.88256.364823.26108.7112466.993797.30341.26349.7847.861099.3315.36104.8624.8897.4316.5558.84355.6126.0130.2952.32186.7127.243.31156.101.271.9122.0121.43163.566.82501.5371.39304.16189.191059.77127.221.39693.74246.61345.9695.15265.614958.11111.612915.873557.43317.45325.8242.721021.7613.997.922.9589.3615.6354.94326.624.128.3148.8178.2426.283.23152.021.221.9521.9721.41163.336.76506.7671.56304.5188.181058.96127.721.45693.66246.59348.6995.34267.314962.19112.2112894.543543.9318.02326.3842.781023.4413.7597.6722.9389.9415.6554.73325.624.0928.3248.42177.8126.123.22151.191.221.9121.6821.07159.696.74493.1470.24299.19185.521045.93125.6420.94682.02243.84337.2992.52257.644908.52111.2912467.63562.54321.69331.4843.751030.0814.1399.5223.3391.1915.8255.6333.5124.4928.6450.08183.2827.093.26152.481.261.7619.4619.10145.056.30455.7467.33282.22176.09996.96119.7419.86653.79231.80315.3888.60252.964709.82105.7012467.503912.27359.37366.1048.221103.1215.31103.8824.7596.1616.6458.38351.7325.5930.1353.70191.3627.613.39160.661.27OpenBenchmarking.org

OpenVINO

Model: Face Detection FP16 - Device: CPU

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2024.0Model: Face Detection FP16 - Device: CPUcdbae0.43880.87761.31641.75522.194SE +/- 0.02, N = 8SE +/- 0.01, N = 151.951.911.911.821.761. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl

OpenVINO

Model: Person Detection FP16 - Device: CPU

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2024.0Model: Person Detection FP16 - Device: CPUbcdae510152025SE +/- 0.23, N = 3SE +/- 0.20, N = 1522.0121.9721.6820.4719.461. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl

OpenVINO

Model: Person Detection FP32 - Device: CPU

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2024.0Model: Person Detection FP32 - Device: CPUbcdae510152025SE +/- 0.27, N = 3SE +/- 0.21, N = 1521.4321.4121.0719.9719.101. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl

OpenVINO

Model: Vehicle Detection FP16 - Device: CPU

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2024.0Model: Vehicle Detection FP16 - Device: CPUbcdae4080120160200SE +/- 1.22, N = 15SE +/- 1.21, N = 15163.56163.33159.69146.15145.051. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl

OpenVINO

Model: Face Detection FP16-INT8 - Device: CPU

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2024.0Model: Face Detection FP16-INT8 - Device: CPUbcdae246810SE +/- 0.05, N = 15SE +/- 0.04, N = 136.826.766.746.336.301. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl

OpenVINO

Model: Face Detection Retail FP16 - Device: CPU

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2024.0Model: Face Detection Retail FP16 - Device: CPUcbdea110220330440550SE +/- 3.85, N = 15SE +/- 4.97, N = 5506.76501.53493.14455.74454.191. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl

OpenVINO

Model: Road Segmentation ADAS FP16 - Device: CPU

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2024.0Model: Road Segmentation ADAS FP16 - Device: CPUcbdea1632486480SE +/- 0.68, N = 6SE +/- 0.75, N = 471.5671.3970.2467.3366.701. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl

OpenVINO

Model: Vehicle Detection FP16-INT8 - Device: CPU

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2024.0Model: Vehicle Detection FP16-INT8 - Device: CPUcbdea70140210280350SE +/- 2.03, N = 15SE +/- 2.21, N = 15304.50304.16299.19282.22280.811. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl

OpenVINO

Model: Weld Porosity Detection FP16 - Device: CPU

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2024.0Model: Weld Porosity Detection FP16 - Device: CPUbcdea4080120160200SE +/- 1.26, N = 12SE +/- 1.41, N = 15189.19188.18185.52176.09173.861. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl

OpenVINO

Model: Face Detection Retail FP16-INT8 - Device: CPU

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2024.0Model: Face Detection Retail FP16-INT8 - Device: CPUbcdae2004006008001000SE +/- 6.75, N = 15SE +/- 7.15, N = 151059.771058.961045.931002.41996.961. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl

OpenVINO

Model: Road Segmentation ADAS FP16-INT8 - Device: CPU

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2024.0Model: Road Segmentation ADAS FP16-INT8 - Device: CPUcbdea306090120150SE +/- 1.39, N = 4SE +/- 1.16, N = 6127.70127.20125.64119.74118.811. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl

OpenVINO

Model: Machine Translation EN To DE FP16 - Device: CPU

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2024.0Model: Machine Translation EN To DE FP16 - Device: CPUcbdea510152025SE +/- 0.15, N = 15SE +/- 0.20, N = 1521.4521.3920.9419.8619.671. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl

OpenVINO

Model: Weld Porosity Detection FP16-INT8 - Device: CPU

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2024.0Model: Weld Porosity Detection FP16-INT8 - Device: CPUbcdea150300450600750SE +/- 5.14, N = 10SE +/- 5.15, N = 15693.74693.66682.02653.79643.511. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl

OpenVINO

Model: Person Vehicle Bike Detection FP16 - Device: CPU

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2024.0Model: Person Vehicle Bike Detection FP16 - Device: CPUbcdea50100150200250SE +/- 1.91, N = 3SE +/- 1.67, N = 15246.61246.59243.84231.80230.731. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl

OpenVINO

Model: Noise Suppression Poconet-Like FP16 - Device: CPU

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2024.0Model: Noise Suppression Poconet-Like FP16 - Device: CPUcbdae80160240320400SE +/- 2.66, N = 15SE +/- 3.89, N = 15348.69345.96337.29323.31315.381. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl

OpenVINO

Model: Handwritten English Recognition FP16 - Device: CPU

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2024.0Model: Handwritten English Recognition FP16 - Device: CPUcbdae20406080100SE +/- 0.70, N = 10SE +/- 0.61, N = 395.3495.1592.5290.8888.601. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl

OpenVINO

Model: Person Re-Identification Retail FP16 - Device: CPU

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2024.0Model: Person Re-Identification Retail FP16 - Device: CPUcbdae60120180240300SE +/- 2.88, N = 4SE +/- 2.45, N = 6267.31265.61257.64256.36252.961. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl

OpenVINO

Model: Age Gender Recognition Retail 0013 FP16 - Device: CPU

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2024.0Model: Age Gender Recognition Retail 0013 FP16 - Device: CPUcbdae11002200330044005500SE +/- 39.34, N = 3SE +/- 46.64, N = 64962.194958.114908.524823.264709.821. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl

OpenVINO

Model: Handwritten English Recognition FP16-INT8 - Device: CPU

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2024.0Model: Handwritten English Recognition FP16-INT8 - Device: CPUcbdae306090120150SE +/- 1.48, N = 3SE +/- 0.83, N = 15112.21111.60111.29108.71105.701. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl

OpenVINO

Model: Age Gender Recognition Retail 0013 FP16-INT8 - Device: CPU

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2024.0Model: Age Gender Recognition Retail 0013 FP16-INT8 - Device: CPUbcdea3K6K9K12K15KSE +/- 146.93, N = 3SE +/- 151.02, N = 412915.8712894.5412467.6012467.5012466.991. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl

OpenVINO

Model: Face Detection FP16 - Device: CPU

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2024.0Model: Face Detection FP16 - Device: CPUcbdae8001600240032004000SE +/- 32.88, N = 8SE +/- 31.91, N = 153543.903557.433562.543797.303912.27MIN: 2743.85 / MAX: 5185.69MIN: 2757.58 / MAX: 5221.58MIN: 1816.87 / MAX: 5174.72MIN: 1742.11 / MAX: 5264.59MIN: 2798.1 / MAX: 5581.531. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl

OpenVINO

Model: Person Detection FP16 - Device: CPU

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2024.0Model: Person Detection FP16 - Device: CPUbcdae80160240320400SE +/- 3.98, N = 3SE +/- 3.72, N = 15317.45318.02321.69341.26359.37MIN: 224.7 / MAX: 464.25MIN: 175.18 / MAX: 464.55MIN: 228.05 / MAX: 462.91MIN: 177.09 / MAX: 488.59MIN: 181.92 / MAX: 496.21. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl

OpenVINO

Model: Person Detection FP32 - Device: CPU

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2024.0Model: Person Detection FP32 - Device: CPUbcdae80160240320400SE +/- 4.98, N = 3SE +/- 3.98, N = 15325.82326.38331.48349.78366.10MIN: 220.85 / MAX: 464.29MIN: 225.82 / MAX: 460.08MIN: 221.77 / MAX: 461.89MIN: 176.64 / MAX: 492.35MIN: 173.32 / MAX: 500.31. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl

OpenVINO

Model: Vehicle Detection FP16 - Device: CPU

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2024.0Model: Vehicle Detection FP16 - Device: CPUbcdae1122334455SE +/- 0.40, N = 15SE +/- 0.40, N = 1542.7242.7843.7547.8648.22MIN: 23.22 / MAX: 67.87MIN: 19.12 / MAX: 66.42MIN: 23.92 / MAX: 118.28MIN: 18.18 / MAX: 104.89MIN: 21.22 / MAX: 118.951. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl

OpenVINO

Model: Face Detection FP16-INT8 - Device: CPU

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2024.0Model: Face Detection FP16-INT8 - Device: CPUbcdae2004006008001000SE +/- 9.38, N = 15SE +/- 7.33, N = 131021.761023.441030.081099.331103.12MIN: 721.29 / MAX: 1599.18MIN: 727.38 / MAX: 1604.26MIN: 733.36 / MAX: 1612.47MIN: 459.81 / MAX: 1740.17MIN: 406.78 / MAX: 1728.531. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl

OpenVINO

Model: Face Detection Retail FP16 - Device: CPU

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2024.0Model: Face Detection Retail FP16 - Device: CPUcbdea48121620SE +/- 0.13, N = 15SE +/- 0.17, N = 513.7513.9014.1315.3115.36MIN: 7.71 / MAX: 30.86MIN: 7.85 / MAX: 35.11MIN: 8.44 / MAX: 37.62MIN: 6.83 / MAX: 39.1MIN: 6.94 / MAX: 39.471. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl

OpenVINO

Model: Road Segmentation ADAS FP16 - Device: CPU

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2024.0Model: Road Segmentation ADAS FP16 - Device: CPUcbdea20406080100SE +/- 1.04, N = 6SE +/- 1.16, N = 497.6797.9099.52103.88104.86MIN: 51.53 / MAX: 347.08MIN: 37.33 / MAX: 376.3MIN: 39.25 / MAX: 383.84MIN: 44.02 / MAX: 396.62MIN: 43.38 / MAX: 358.761. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl

OpenVINO

Model: Vehicle Detection FP16-INT8 - Device: CPU

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2024.0Model: Vehicle Detection FP16-INT8 - Device: CPUcbdea612182430SE +/- 0.18, N = 15SE +/- 0.20, N = 1522.9322.9523.3324.7524.88MIN: 11.91 / MAX: 43.96MIN: 11.9 / MAX: 43.97MIN: 13.63 / MAX: 40.97MIN: 11.42 / MAX: 47.13MIN: 12.66 / MAX: 47.231. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl

OpenVINO

Model: Weld Porosity Detection FP16 - Device: CPU

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2024.0Model: Weld Porosity Detection FP16 - Device: CPUbcdea20406080100SE +/- 0.70, N = 12SE +/- 0.81, N = 1589.3689.9491.1996.1697.43MIN: 46.01 / MAX: 136.05MIN: 46.46 / MAX: 140.34MIN: 46.11 / MAX: 142.48MIN: 45.85 / MAX: 157.97MIN: 46.27 / MAX: 163.71. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl

OpenVINO

Model: Face Detection Retail FP16-INT8 - Device: CPU

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2024.0Model: Face Detection Retail FP16-INT8 - Device: CPUbcdae48121620SE +/- 0.11, N = 15SE +/- 0.12, N = 1515.6315.6515.8216.5516.64MIN: 7.37 / MAX: 26.39MIN: 7.08 / MAX: 26.33MIN: 6.52 / MAX: 29.45MIN: 6.65 / MAX: 34.71MIN: 7.02 / MAX: 37.411. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl

OpenVINO

Model: Road Segmentation ADAS FP16-INT8 - Device: CPU

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2024.0Model: Road Segmentation ADAS FP16-INT8 - Device: CPUcbdea1326395265SE +/- 0.67, N = 4SE +/- 0.58, N = 654.7354.9455.6058.3858.84MIN: 32.28 / MAX: 86.23MIN: 32.3 / MAX: 87.41MIN: 32.5 / MAX: 89.87MIN: 25.03 / MAX: 96.63MIN: 28.65 / MAX: 92.741. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl

OpenVINO

Model: Machine Translation EN To DE FP16 - Device: CPU

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2024.0Model: Machine Translation EN To DE FP16 - Device: CPUcbdea80160240320400SE +/- 2.64, N = 15SE +/- 3.64, N = 15325.60326.60333.51351.73355.61MIN: 217.98 / MAX: 455.75MIN: 127.63 / MAX: 466.63MIN: 227.46 / MAX: 479.39MIN: 137.63 / MAX: 624.33MIN: 163.07 / MAX: 515.111. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl

OpenVINO

Model: Weld Porosity Detection FP16-INT8 - Device: CPU

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2024.0Model: Weld Porosity Detection FP16-INT8 - Device: CPUcbdea612182430SE +/- 0.20, N = 10SE +/- 0.21, N = 1524.0924.1024.4925.5926.01MIN: 11.69 / MAX: 42.69MIN: 12.07 / MAX: 40.91MIN: 12.24 / MAX: 41.08MIN: 12.13 / MAX: 55.42MIN: 10.92 / MAX: 57.131. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl

OpenVINO

Model: Person Vehicle Bike Detection FP16 - Device: CPU

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2024.0Model: Person Vehicle Bike Detection FP16 - Device: CPUbcdea714212835SE +/- 0.25, N = 3SE +/- 0.22, N = 1528.3128.3228.6430.1330.29MIN: 17.01 / MAX: 55.63MIN: 17.27 / MAX: 57.04MIN: 16.72 / MAX: 56.71MIN: 14.67 / MAX: 57.69MIN: 13.26 / MAX: 59.591. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl

OpenVINO

Model: Noise Suppression Poconet-Like FP16 - Device: CPU

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2024.0Model: Noise Suppression Poconet-Like FP16 - Device: CPUcbdae1224364860SE +/- 0.44, N = 15SE +/- 0.67, N = 1548.4248.8050.0852.3253.70MIN: 24.4 / MAX: 71.85MIN: 24.49 / MAX: 74.38MIN: 24.33 / MAX: 73.52MIN: 24.2 / MAX: 79.7MIN: 24.32 / MAX: 89.771. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl

OpenVINO

Model: Handwritten English Recognition FP16 - Device: CPU

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2024.0Model: Handwritten English Recognition FP16 - Device: CPUcbdae4080120160200SE +/- 1.41, N = 10SE +/- 1.32, N = 3177.81178.24183.28186.71191.36MIN: 102.85 / MAX: 239.14MIN: 101.95 / MAX: 247.18MIN: 101.8 / MAX: 248.36MIN: 101.83 / MAX: 274.05MIN: 101.79 / MAX: 268.91. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl

OpenVINO

Model: Person Re-Identification Retail FP16 - Device: CPU

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2024.0Model: Person Re-Identification Retail FP16 - Device: CPUcbdae612182430SE +/- 0.31, N = 4SE +/- 0.27, N = 626.1226.2827.0927.2427.61MIN: 15.46 / MAX: 41.86MIN: 16.16 / MAX: 43.57MIN: 15.78 / MAX: 43.25MIN: 15.08 / MAX: 47.91MIN: 11.88 / MAX: 54.331. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl

OpenVINO

Model: Age Gender Recognition Retail 0013 FP16 - Device: CPU

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2024.0Model: Age Gender Recognition Retail 0013 FP16 - Device: CPUcbdae0.76281.52562.28843.05123.814SE +/- 0.03, N = 3SE +/- 0.03, N = 63.223.233.263.313.39MIN: 1.04 / MAX: 15.36MIN: 1.1 / MAX: 8.44MIN: 1.04 / MAX: 8MIN: 1.1 / MAX: 12.43MIN: 1.06 / MAX: 14.991. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl

OpenVINO

Model: Handwritten English Recognition FP16-INT8 - Device: CPU

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2024.0Model: Handwritten English Recognition FP16-INT8 - Device: CPUcbdae4080120160200SE +/- 2.14, N = 3SE +/- 1.27, N = 15151.19152.02152.48156.10160.66MIN: 89.51 / MAX: 211.52MIN: 88.01 / MAX: 226.83MIN: 89.77 / MAX: 219.3MIN: 89.74 / MAX: 233.97MIN: 88.95 / MAX: 239.441. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl

OpenVINO

Model: Age Gender Recognition Retail 0013 FP16-INT8 - Device: CPU

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2024.0Model: Age Gender Recognition Retail 0013 FP16-INT8 - Device: CPUbcdae0.28580.57160.85741.14321.429SE +/- 0.02, N = 4SE +/- 0.01, N = 31.221.221.261.271.27MIN: 0.39 / MAX: 6.57MIN: 0.4 / MAX: 5.49MIN: 0.4 / MAX: 6.52MIN: 0.38 / MAX: 11.64MIN: 0.38 / MAX: 12.951. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl

Phoronix Test Suite v10.8.5