openvino 2024

Intel Core i9-14900K testing with a ASUS PRIME Z790-P WIFI (1402 BIOS) and ASUS Intel RPL-S 31GB on Ubuntu 23.10 via the Phoronix Test Suite.

Compare your own system(s) to this result file with the Phoronix Test Suite by running the command: phoronix-test-suite benchmark 2403078-PTS-OPENVINO47
Jump To Table - Results

View

Do Not Show Noisy Results
Do Not Show Results With Incomplete Data
Do Not Show Results With Little Change/Spread
List Notable Results

Statistics

Show Overall Harmonic Mean(s)
Show Overall Geometric Mean
Show Wins / Losses Counts (Pie Chart)
Normalize Results
Remove Outliers Before Calculating Averages

Graph Settings

Force Line Graphs Where Applicable
Convert To Scalar Where Applicable
Prefer Vertical Bar Graphs

Multi-Way Comparison

Condense Multi-Option Tests Into Single Result Graphs

Table

Show Detailed System Result Table

Run Management

Highlight
Result
Hide
Result
Result
Identifier
Performance Per
Dollar
Date
Run
  Test
  Duration
a
March 07
  2 Hours, 4 Minutes
b
March 07
  2 Hours, 2 Minutes
c
March 07
  2 Hours, 2 Minutes
d
March 07
  41 Minutes
Invert Hiding All Results Option
  1 Hour, 42 Minutes

Only show results where is faster than
Only show results matching title/arguments (delimit multiple options with a comma):
Do not show results matching title/arguments (delimit multiple options with a comma):

a

Kernel Notes: Transparent Huge Pages: madvise
Compiler Notes: --build=x86_64-linux-gnu --disable-vtable-verify --disable-werror --enable-bootstrap --enable-cet --enable-checking=release --enable-clocale=gnu --enable-default-pie --enable-gnu-unique-object --enable-languages=c,ada,c++,go,d,fortran,objc,obj-c++,m2 --enable-libphobos-checking=release --enable-libstdcxx-debug --enable-libstdcxx-time=yes --enable-link-serialization=2 --enable-multiarch --enable-multilib --enable-nls --enable-objc-gc=auto --enable-offload-defaulted --enable-offload-targets=nvptx-none=/build/gcc-13-XYspKM/gcc-13-13.2.0/debian/tmp-nvptx/usr,amdgcn-amdhsa=/build/gcc-13-XYspKM/gcc-13-13.2.0/debian/tmp-gcn/usr --enable-plugin --enable-shared --enable-threads=posix --host=x86_64-linux-gnu --program-prefix=x86_64-linux-gnu- --target=x86_64-linux-gnu --with-abi=m64 --with-arch-32=i686 --with-build-config=bootstrap-lto-lean --with-default-libstdcxx-abi=new --with-gcc-major-version-only --with-multilib-list=m32,m64,mx32 --with-target-system-zlib=auto --with-tune=generic --without-cuda-driver -v
Processor Notes: Scaling Governor: intel_pstate powersave (EPP: performance) - CPU Microcode: 0x11d - Thermald 2.5.4
Python Notes: Python 3.11.6
Security Notes: gather_data_sampling: Not affected + itlb_multihit: Not affected + l1tf: Not affected + mds: Not affected + meltdown: Not affected + mmio_stale_data: Not affected + retbleed: Not affected + spec_rstack_overflow: Not affected + spec_store_bypass: Mitigation of SSB disabled via prctl + spectre_v1: Mitigation of usercopy/swapgs barriers and __user pointer sanitization + spectre_v2: Mitigation of Enhanced / Automatic IBRS IBPB: conditional RSB filling PBRSB-eIBRS: SW sequence + srbds: Not affected + tsx_async_abort: Not affected

b

c

d

Processor: Intel Core i9-14900K @ 5.70GHz (24 Cores / 32 Threads), Motherboard: ASUS PRIME Z790-P WIFI (1402 BIOS), Chipset: Intel Device 7a27, Memory: 2 x 16GB DRAM-6000MT/s Corsair CMK32GX5M2B6000C36, Disk: Western Digital WD_BLACK SN850X 1000GB, Graphics: ASUS Intel RPL-S 31GB (1650MHz), Audio: Realtek ALC897, Monitor: ASUS VP28U

OS: Ubuntu 23.10, Kernel: 6.7.0-060700rc7daily20231224-generic (x86_64), Desktop: GNOME Shell 45.1, Display Server: X Server 1.21.1.7, OpenGL: 4.6 Mesa 24.0~git2312240600.c05261~oibaf~m (git-c05261a 2023-12-24 mantic-oibaf-ppa), OpenCL: OpenCL 3.0, Compiler: GCC 13.2.0, File-System: ext4, Screen Resolution: 1920x1080

openvino 2024OpenBenchmarking.orgPhoronix Test SuiteIntel Core i9-14900K @ 5.70GHz (24 Cores / 32 Threads)ASUS PRIME Z790-P WIFI (1402 BIOS)Intel Device 7a272 x 16GB DRAM-6000MT/s Corsair CMK32GX5M2B6000C36Western Digital WD_BLACK SN850X 1000GBASUS Intel RPL-S 31GB (1650MHz)Realtek ALC897ASUS VP28UUbuntu 23.106.7.0-060700rc7daily20231224-generic (x86_64)GNOME Shell 45.1X Server 1.21.1.74.6 Mesa 24.0~git2312240600.c05261~oibaf~m (git-c05261a 2023-12-24 mantic-oibaf-ppa)OpenCL 3.0GCC 13.2.0ext41920x1080ProcessorMotherboardChipsetMemoryDiskGraphicsAudioMonitorOSKernelDesktopDisplay ServerOpenGLOpenCLCompilerFile-SystemScreen ResolutionOpenvino 2024 BenchmarksSystem Logs- Transparent Huge Pages: madvise- --build=x86_64-linux-gnu --disable-vtable-verify --disable-werror --enable-bootstrap --enable-cet --enable-checking=release --enable-clocale=gnu --enable-default-pie --enable-gnu-unique-object --enable-languages=c,ada,c++,go,d,fortran,objc,obj-c++,m2 --enable-libphobos-checking=release --enable-libstdcxx-debug --enable-libstdcxx-time=yes --enable-link-serialization=2 --enable-multiarch --enable-multilib --enable-nls --enable-objc-gc=auto --enable-offload-defaulted --enable-offload-targets=nvptx-none=/build/gcc-13-XYspKM/gcc-13-13.2.0/debian/tmp-nvptx/usr,amdgcn-amdhsa=/build/gcc-13-XYspKM/gcc-13-13.2.0/debian/tmp-gcn/usr --enable-plugin --enable-shared --enable-threads=posix --host=x86_64-linux-gnu --program-prefix=x86_64-linux-gnu- --target=x86_64-linux-gnu --with-abi=m64 --with-arch-32=i686 --with-build-config=bootstrap-lto-lean --with-default-libstdcxx-abi=new --with-gcc-major-version-only --with-multilib-list=m32,m64,mx32 --with-target-system-zlib=auto --with-tune=generic --without-cuda-driver -v - Scaling Governor: intel_pstate powersave (EPP: performance) - CPU Microcode: 0x11d - Thermald 2.5.4- Python 3.11.6- gather_data_sampling: Not affected + itlb_multihit: Not affected + l1tf: Not affected + mds: Not affected + meltdown: Not affected + mmio_stale_data: Not affected + retbleed: Not affected + spec_rstack_overflow: Not affected + spec_store_bypass: Mitigation of SSB disabled via prctl + spectre_v1: Mitigation of usercopy/swapgs barriers and __user pointer sanitization + spectre_v2: Mitigation of Enhanced / Automatic IBRS IBPB: conditional RSB filling PBRSB-eIBRS: SW sequence + srbds: Not affected + tsx_async_abort: Not affected

abcdResult OverviewPhoronix Test Suite100%100%101%101%OpenVINOOpenVINOOpenVINOOpenVINOOpenVINOOpenVINOOpenVINOOpenVINOOpenVINOOpenVINOOpenVINOOpenVINOOpenVINOOpenVINOOpenVINOOpenVINOOpenVINOOpenVINOOpenVINOOpenVINOOpenVINOOpenVINOOpenVINOOpenVINOOpenVINOOpenVINOOpenVINOOpenVINOOpenVINOOpenVINOOpenVINOOpenVINOOpenVINOOpenVINOOpenVINOOpenVINOOpenVINOOpenVINOOpenVINOOpenVINOP.D.F - CPUP.D.F - CPUF.D.F - CPUF.D.F - CPUP.V.B.D.F - CPUP.V.B.D.F - CPUH.E.R.F.I - CPUH.E.R.F.I - CPUW.P.D.F.I - CPUP.D.F - CPUP.D.F - CPUR.S.A.F.I - CPUR.S.A.F.I - CPUV.D.F.I - CPUN.S.P.L.F - CPUV.D.F.I - CPUN.S.P.L.F - CPUM.T.E.T.D.F - CPUM.T.E.T.D.F - CPUA.G.R.R.0.F - CPUF.D.R.F - CPUR.S.A.F - CPUR.S.A.F - CPUF.D.R.F - CPUA.G.R.R.0.F - CPUA.G.R.R.0.F.I - CPUV.D.F - CPUV.D.F - CPUF.D.F.I - CPUW.P.D.F - CPUW.P.D.F - CPUH.E.R.F - CPUH.E.R.F - CPUP.R.I.R.F - CPUF.D.F.I - CPUP.R.I.R.F - CPUW.P.D.F.I - CPUF.D.R.F.I - CPUF.D.R.F.I - CPUA.G.R.R.0.F.I - CPU

openvino 2024openvino: Face Detection FP16 - CPUopenvino: Face Detection FP16 - CPUopenvino: Person Detection FP16 - CPUopenvino: Person Detection FP16 - CPUopenvino: Person Detection FP32 - CPUopenvino: Person Detection FP32 - CPUopenvino: Vehicle Detection FP16 - CPUopenvino: Vehicle Detection FP16 - CPUopenvino: Face Detection FP16-INT8 - CPUopenvino: Face Detection FP16-INT8 - CPUopenvino: Face Detection Retail FP16 - CPUopenvino: Face Detection Retail FP16 - CPUopenvino: Road Segmentation ADAS FP16 - CPUopenvino: Road Segmentation ADAS FP16 - CPUopenvino: Vehicle Detection FP16-INT8 - CPUopenvino: Vehicle Detection FP16-INT8 - CPUopenvino: Weld Porosity Detection FP16 - CPUopenvino: Weld Porosity Detection FP16 - CPUopenvino: Face Detection Retail FP16-INT8 - CPUopenvino: Face Detection Retail FP16-INT8 - CPUopenvino: Road Segmentation ADAS FP16-INT8 - CPUopenvino: Road Segmentation ADAS FP16-INT8 - CPUopenvino: Machine Translation EN To DE FP16 - CPUopenvino: Machine Translation EN To DE FP16 - CPUopenvino: Weld Porosity Detection FP16-INT8 - CPUopenvino: Weld Porosity Detection FP16-INT8 - CPUopenvino: Person Vehicle Bike Detection FP16 - CPUopenvino: Person Vehicle Bike Detection FP16 - CPUopenvino: Noise Suppression Poconet-Like FP16 - CPUopenvino: Noise Suppression Poconet-Like FP16 - CPUopenvino: Handwritten English Recognition FP16 - CPUopenvino: Handwritten English Recognition FP16 - CPUopenvino: Person Re-Identification Retail FP16 - CPUopenvino: Person Re-Identification Retail FP16 - CPUopenvino: Age Gender Recognition Retail 0013 FP16 - CPUopenvino: Age Gender Recognition Retail 0013 FP16 - CPUopenvino: Handwritten English Recognition FP16-INT8 - CPUopenvino: Handwritten English Recognition FP16-INT8 - CPUopenvino: Age Gender Recognition Retail 0013 FP16-INT8 - CPUopenvino: Age Gender Recognition Retail 0013 FP16-INT8 - CPUabcd5.541431.1355.47143.8954.93145.33359.1222.1918.55430.291562.695.08124.9663.91878.058.95545.8443.552769.958.33368.4721.6264.52123.881794.4313.02679.7011.691036.4222.92300.1879.80828.449.5214614.481.51352.1168.0537990.690.575.551431.8355.03145.0654.85145.47359.9622.1418.57429.541569.945.06124.6664.07880.348.94548.7443.342771.598.32370.4721.5264.95123.021807.9113.00679.9211.691036.0422.92301.7079.41828.919.4814529.641.52353.9167.7038013.050.575.611418.3755.42144.0455.22144.52360.6722.1018.53430.401560.245.09125.4263.69880.308.93548.0343.382770.848.32368.5821.6364.61123.661791.9613.03675.9011.761037.1622.89301.6279.41829.299.5014538.011.51354.0767.6637977.980.575.641409.9554.98145.1354.23147.2358.6422.2218.63428.971569.185.06124.7564.04884.458.88548.4443.322772.328.32371.5221.4564.71123.441791.0313.02672.6511.811043.422.75301.1179.558319.514593.461.51355.5267.4138193.920.57OpenBenchmarking.org

OpenVINO

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2024.0Model: Face Detection FP16 - Device: CPUdcba1.2692.5383.8075.0766.345SE +/- 0.06, N = 3SE +/- 0.07, N = 3SE +/- 0.06, N = 45.645.615.555.541. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl
OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2024.0Model: Face Detection FP16 - Device: CPUdcba246810Min: 5.55 / Avg: 5.61 / Max: 5.72Min: 5.45 / Avg: 5.55 / Max: 5.68Min: 5.45 / Avg: 5.54 / Max: 5.721. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2024.0Model: Face Detection FP16 - Device: CPUdcba30060090012001500SE +/- 15.78, N = 3SE +/- 13.01, N = 3SE +/- 12.71, N = 41409.951418.371431.831431.13MIN: 1082.33 / MAX: 2001.47MIN: 1085.81 / MAX: 2027.91MIN: 1082.62 / MAX: 2031.97MIN: 605.23 / MAX: 2285.531. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl
OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2024.0Model: Face Detection FP16 - Device: CPUdcba30060090012001500Min: 1386.83 / Avg: 1418.37 / Max: 1434.84Min: 1405.95 / Avg: 1431.83 / Max: 1447.12Min: 1393.22 / Avg: 1431.13 / Max: 1446.571. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2024.0Model: Person Detection FP16 - Device: CPUdcba1224364860SE +/- 0.41, N = 3SE +/- 0.34, N = 3SE +/- 0.59, N = 354.9855.4255.0355.471. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl
OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2024.0Model: Person Detection FP16 - Device: CPUdcba1122334455Min: 54.81 / Avg: 55.42 / Max: 56.21Min: 54.4 / Avg: 55.03 / Max: 55.57Min: 54.32 / Avg: 55.47 / Max: 56.271. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2024.0Model: Person Detection FP16 - Device: CPUdcba306090120150SE +/- 1.08, N = 3SE +/- 0.90, N = 3SE +/- 1.52, N = 3145.13144.04145.06143.89MIN: 61.17 / MAX: 331.83MIN: 103.76 / MAX: 357.67MIN: 104.99 / MAX: 332.47MIN: 102.76 / MAX: 364.231. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl
OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2024.0Model: Person Detection FP16 - Device: CPUdcba306090120150Min: 141.98 / Avg: 144.04 / Max: 145.6Min: 143.64 / Avg: 145.06 / Max: 146.72Min: 141.83 / Avg: 143.89 / Max: 146.861. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2024.0Model: Person Detection FP32 - Device: CPUdcba1224364860SE +/- 0.10, N = 3SE +/- 0.29, N = 3SE +/- 0.78, N = 354.2355.2254.8554.931. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl
OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2024.0Model: Person Detection FP32 - Device: CPUdcba1122334455Min: 55.02 / Avg: 55.22 / Max: 55.33Min: 54.28 / Avg: 54.85 / Max: 55.16Min: 53.43 / Avg: 54.93 / Max: 56.051. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2024.0Model: Person Detection FP32 - Device: CPUdcba306090120150SE +/- 0.23, N = 3SE +/- 0.79, N = 3SE +/- 2.13, N = 3147.20144.52145.47145.33MIN: 107.25 / MAX: 337.03MIN: 64.95 / MAX: 367.87MIN: 68.34 / MAX: 419.01MIN: 71.2 / MAX: 360.271. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl
OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2024.0Model: Person Detection FP32 - Device: CPUdcba306090120150Min: 144.24 / Avg: 144.52 / Max: 144.98Min: 144.63 / Avg: 145.47 / Max: 147.05Min: 142.32 / Avg: 145.33 / Max: 149.441. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2024.0Model: Vehicle Detection FP16 - Device: CPUdcba80160240320400SE +/- 0.80, N = 3SE +/- 0.80, N = 3SE +/- 0.77, N = 3358.64360.67359.96359.121. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl
OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2024.0Model: Vehicle Detection FP16 - Device: CPUdcba60120180240300Min: 359.64 / Avg: 360.67 / Max: 362.25Min: 358.41 / Avg: 359.96 / Max: 361.08Min: 358.18 / Avg: 359.12 / Max: 360.641. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2024.0Model: Vehicle Detection FP16 - Device: CPUdcba510152025SE +/- 0.05, N = 3SE +/- 0.05, N = 3SE +/- 0.05, N = 322.2222.1022.1422.19MIN: 8.85 / MAX: 89.64MIN: 8.57 / MAX: 94.59MIN: 10.45 / MAX: 95.33MIN: 8.4 / MAX: 93.51. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl
OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2024.0Model: Vehicle Detection FP16 - Device: CPUdcba510152025Min: 22 / Avg: 22.1 / Max: 22.16Min: 22.07 / Avg: 22.14 / Max: 22.23Min: 22.1 / Avg: 22.19 / Max: 22.251. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2024.0Model: Face Detection FP16-INT8 - Device: CPUdcba510152025SE +/- 0.04, N = 3SE +/- 0.01, N = 3SE +/- 0.01, N = 318.6318.5318.5718.551. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl
OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2024.0Model: Face Detection FP16-INT8 - Device: CPUdcba510152025Min: 18.45 / Avg: 18.53 / Max: 18.59Min: 18.56 / Avg: 18.57 / Max: 18.59Min: 18.54 / Avg: 18.55 / Max: 18.571. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2024.0Model: Face Detection FP16-INT8 - Device: CPUdcba90180270360450SE +/- 0.49, N = 3SE +/- 0.40, N = 3SE +/- 0.18, N = 3428.97430.40429.54430.29MIN: 257.33 / MAX: 1070.48MIN: 257.56 / MAX: 1108.13MIN: 257.8 / MAX: 1074.48MIN: 257.52 / MAX: 1279.771. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl
OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2024.0Model: Face Detection FP16-INT8 - Device: CPUdcba80160240320400Min: 429.57 / Avg: 430.4 / Max: 431.25Min: 429.07 / Avg: 429.54 / Max: 430.33Min: 429.99 / Avg: 430.29 / Max: 430.61. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2024.0Model: Face Detection Retail FP16 - Device: CPUdcba30060090012001500SE +/- 9.44, N = 3SE +/- 7.83, N = 3SE +/- 12.14, N = 31569.181560.241569.941562.691. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl
OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2024.0Model: Face Detection Retail FP16 - Device: CPUdcba30060090012001500Min: 1541.38 / Avg: 1560.24 / Max: 1570.27Min: 1560.24 / Avg: 1569.94 / Max: 1585.44Min: 1543.37 / Avg: 1562.69 / Max: 1585.081. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2024.0Model: Face Detection Retail FP16 - Device: CPUdcba1.14532.29063.43594.58125.7265SE +/- 0.03, N = 3SE +/- 0.03, N = 3SE +/- 0.04, N = 35.065.095.065.08MIN: 2.02 / MAX: 31.5MIN: 1.97 / MAX: 30.6MIN: 1.96 / MAX: 30.61MIN: 2 / MAX: 30.71. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl
OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2024.0Model: Face Detection Retail FP16 - Device: CPUdcba246810Min: 5.06 / Avg: 5.09 / Max: 5.15Min: 5.01 / Avg: 5.06 / Max: 5.09Min: 5.01 / Avg: 5.08 / Max: 5.141. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2024.0Model: Road Segmentation ADAS FP16 - Device: CPUdcba306090120150SE +/- 0.08, N = 3SE +/- 0.50, N = 3SE +/- 0.10, N = 3124.75125.42124.66124.961. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl
OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2024.0Model: Road Segmentation ADAS FP16 - Device: CPUdcba20406080100Min: 125.28 / Avg: 125.42 / Max: 125.56Min: 123.67 / Avg: 124.66 / Max: 125.17Min: 124.78 / Avg: 124.96 / Max: 125.121. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2024.0Model: Road Segmentation ADAS FP16 - Device: CPUdcba1428425670SE +/- 0.05, N = 3SE +/- 0.26, N = 3SE +/- 0.04, N = 364.0463.6964.0763.91MIN: 20.67 / MAX: 176.81MIN: 16.35 / MAX: 179.48MIN: 16.45 / MAX: 185.4MIN: 16.56 / MAX: 179.661. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl
OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2024.0Model: Road Segmentation ADAS FP16 - Device: CPUdcba1326395265Min: 63.62 / Avg: 63.69 / Max: 63.78Min: 63.8 / Avg: 64.07 / Max: 64.58Min: 63.84 / Avg: 63.91 / Max: 63.991. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2024.0Model: Vehicle Detection FP16-INT8 - Device: CPUdcba2004006008001000SE +/- 2.34, N = 3SE +/- 3.18, N = 3SE +/- 1.46, N = 3884.45880.30880.34878.051. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl
OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2024.0Model: Vehicle Detection FP16-INT8 - Device: CPUdcba160320480640800Min: 877.26 / Avg: 880.3 / Max: 884.9Min: 873.99 / Avg: 880.34 / Max: 883.73Min: 875.18 / Avg: 878.05 / Max: 879.961. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2024.0Model: Vehicle Detection FP16-INT8 - Device: CPUdcba3691215SE +/- 0.02, N = 3SE +/- 0.03, N = 3SE +/- 0.01, N = 38.888.938.948.95MIN: 3.79 / MAX: 62.98MIN: 3.76 / MAX: 63.07MIN: 3.61 / MAX: 64.78MIN: 3.77 / MAX: 63.031. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl
OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2024.0Model: Vehicle Detection FP16-INT8 - Device: CPUdcba3691215Min: 8.88 / Avg: 8.93 / Max: 8.96Min: 8.91 / Avg: 8.94 / Max: 8.99Min: 8.93 / Avg: 8.95 / Max: 8.971. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2024.0Model: Weld Porosity Detection FP16 - Device: CPUdcba120240360480600SE +/- 1.63, N = 3SE +/- 1.71, N = 3SE +/- 1.65, N = 3548.44548.03548.74545.841. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl
OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2024.0Model: Weld Porosity Detection FP16 - Device: CPUdcba100200300400500Min: 545.8 / Avg: 548.03 / Max: 551.2Min: 545.35 / Avg: 548.74 / Max: 550.88Min: 542.78 / Avg: 545.84 / Max: 548.441. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2024.0Model: Weld Porosity Detection FP16 - Device: CPUdcba1020304050SE +/- 0.13, N = 3SE +/- 0.14, N = 3SE +/- 0.13, N = 343.3243.3843.3443.55MIN: 23.33 / MAX: 168.29MIN: 30.58 / MAX: 167.2MIN: 22.94 / MAX: 153.89MIN: 27.71 / MAX: 168.541. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl
OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2024.0Model: Weld Porosity Detection FP16 - Device: CPUdcba918273645Min: 43.12 / Avg: 43.38 / Max: 43.52Min: 43.2 / Avg: 43.34 / Max: 43.62Min: 43.32 / Avg: 43.55 / Max: 43.781. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2024.0Model: Face Detection Retail FP16-INT8 - Device: CPUdcba6001200180024003000SE +/- 1.46, N = 3SE +/- 3.98, N = 3SE +/- 2.31, N = 32772.322770.842771.592769.951. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl
OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2024.0Model: Face Detection Retail FP16-INT8 - Device: CPUdcba5001000150020002500Min: 2767.92 / Avg: 2770.84 / Max: 2772.38Min: 2766.99 / Avg: 2771.59 / Max: 2779.52Min: 2766.02 / Avg: 2769.95 / Max: 2774.011. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2024.0Model: Face Detection Retail FP16-INT8 - Device: CPUdcba246810SE +/- 0.00, N = 3SE +/- 0.01, N = 3SE +/- 0.00, N = 38.328.328.328.33MIN: 4.24 / MAX: 45.21MIN: 4.19 / MAX: 44.33MIN: 4.14 / MAX: 42.15MIN: 4.31 / MAX: 43.341. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl
OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2024.0Model: Face Detection Retail FP16-INT8 - Device: CPUdcba3691215Min: 8.31 / Avg: 8.32 / Max: 8.32Min: 8.3 / Avg: 8.32 / Max: 8.33Min: 8.32 / Avg: 8.33 / Max: 8.331. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2024.0Model: Road Segmentation ADAS FP16-INT8 - Device: CPUdcba80160240320400SE +/- 1.20, N = 3SE +/- 0.92, N = 3SE +/- 0.66, N = 3371.52368.58370.47368.471. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl
OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2024.0Model: Road Segmentation ADAS FP16-INT8 - Device: CPUdcba70140210280350Min: 366.17 / Avg: 368.58 / Max: 369.88Min: 369.43 / Avg: 370.47 / Max: 372.3Min: 367.35 / Avg: 368.47 / Max: 369.631. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2024.0Model: Road Segmentation ADAS FP16-INT8 - Device: CPUdcba510152025SE +/- 0.07, N = 3SE +/- 0.05, N = 3SE +/- 0.04, N = 321.4521.6321.5221.62MIN: 11.1 / MAX: 130.92MIN: 10.36 / MAX: 139.06MIN: 9.3 / MAX: 140.98MIN: 10.64 / MAX: 138.551. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl
OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2024.0Model: Road Segmentation ADAS FP16-INT8 - Device: CPUdcba510152025Min: 21.55 / Avg: 21.63 / Max: 21.77Min: 21.41 / Avg: 21.52 / Max: 21.58Min: 21.55 / Avg: 21.62 / Max: 21.691. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2024.0Model: Machine Translation EN To DE FP16 - Device: CPUdcba1428425670SE +/- 0.15, N = 3SE +/- 0.29, N = 3SE +/- 0.08, N = 364.7164.6164.9564.521. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl
OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2024.0Model: Machine Translation EN To DE FP16 - Device: CPUdcba1326395265Min: 64.32 / Avg: 64.61 / Max: 64.83Min: 64.37 / Avg: 64.95 / Max: 65.24Min: 64.37 / Avg: 64.52 / Max: 64.661. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2024.0Model: Machine Translation EN To DE FP16 - Device: CPUdcba306090120150SE +/- 0.29, N = 3SE +/- 0.60, N = 3SE +/- 0.15, N = 3123.44123.66123.02123.88MIN: 82.38 / MAX: 360.11MIN: 83.99 / MAX: 339.58MIN: 71.26 / MAX: 344.75MIN: 80.92 / MAX: 345.321. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl
OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2024.0Model: Machine Translation EN To DE FP16 - Device: CPUdcba20406080100Min: 123.25 / Avg: 123.66 / Max: 124.23Min: 122.38 / Avg: 123.02 / Max: 124.21Min: 123.64 / Avg: 123.88 / Max: 124.171. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2024.0Model: Weld Porosity Detection FP16-INT8 - Device: CPUdcba400800120016002000SE +/- 2.14, N = 3SE +/- 9.52, N = 3SE +/- 1.23, N = 31791.031791.961807.911794.431. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl
OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2024.0Model: Weld Porosity Detection FP16-INT8 - Device: CPUdcba30060090012001500Min: 1787.67 / Avg: 1791.96 / Max: 1794.24Min: 1795.43 / Avg: 1807.91 / Max: 1826.61Min: 1792.57 / Avg: 1794.43 / Max: 1796.761. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2024.0Model: Weld Porosity Detection FP16-INT8 - Device: CPUdcba3691215SE +/- 0.01, N = 3SE +/- 0.02, N = 3SE +/- 0.01, N = 313.0213.0313.0013.02MIN: 6.66 / MAX: 73.73MIN: 6.57 / MAX: 80.02MIN: 6.29 / MAX: 87.94MIN: 6.59 / MAX: 76.061. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl
OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2024.0Model: Weld Porosity Detection FP16-INT8 - Device: CPUdcba48121620Min: 13.02 / Avg: 13.03 / Max: 13.04Min: 12.97 / Avg: 13 / Max: 13.02Min: 13 / Avg: 13.02 / Max: 13.031. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2024.0Model: Person Vehicle Bike Detection FP16 - Device: CPUdcba150300450600750SE +/- 3.61, N = 3SE +/- 3.96, N = 3SE +/- 3.82, N = 3672.65675.90679.92679.701. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl
OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2024.0Model: Person Vehicle Bike Detection FP16 - Device: CPUdcba120240360480600Min: 672.06 / Avg: 675.9 / Max: 683.12Min: 674.94 / Avg: 679.92 / Max: 687.74Min: 673.61 / Avg: 679.7 / Max: 686.731. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2024.0Model: Person Vehicle Bike Detection FP16 - Device: CPUdcba3691215SE +/- 0.07, N = 3SE +/- 0.06, N = 3SE +/- 0.07, N = 311.8111.7611.6911.69MIN: 5.07 / MAX: 58.59MIN: 5.18 / MAX: 59.34MIN: 5.25 / MAX: 61.44MIN: 4.9 / MAX: 58.961. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl
OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2024.0Model: Person Vehicle Bike Detection FP16 - Device: CPUdcba3691215Min: 11.63 / Avg: 11.76 / Max: 11.83Min: 11.56 / Avg: 11.69 / Max: 11.77Min: 11.57 / Avg: 11.69 / Max: 11.81. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2024.0Model: Noise Suppression Poconet-Like FP16 - Device: CPUdcba2004006008001000SE +/- 0.95, N = 3SE +/- 0.29, N = 3SE +/- 3.66, N = 31043.401037.161036.041036.421. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl
OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2024.0Model: Noise Suppression Poconet-Like FP16 - Device: CPUdcba2004006008001000Min: 1035.25 / Avg: 1037.16 / Max: 1038.18Min: 1035.47 / Avg: 1036.04 / Max: 1036.37Min: 1031.46 / Avg: 1036.42 / Max: 1043.551. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2024.0Model: Noise Suppression Poconet-Like FP16 - Device: CPUdcba510152025SE +/- 0.02, N = 3SE +/- 0.01, N = 3SE +/- 0.08, N = 322.7522.8922.9222.92MIN: 13.27 / MAX: 84.75MIN: 16.2 / MAX: 85.68MIN: 12.52 / MAX: 85.7MIN: 12.82 / MAX: 88.31. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl
OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2024.0Model: Noise Suppression Poconet-Like FP16 - Device: CPUdcba510152025Min: 22.86 / Avg: 22.89 / Max: 22.93Min: 22.9 / Avg: 22.92 / Max: 22.93Min: 22.77 / Avg: 22.92 / Max: 23.021. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2024.0Model: Handwritten English Recognition FP16 - Device: CPUdcba70140210280350SE +/- 1.02, N = 3SE +/- 1.69, N = 3SE +/- 1.26, N = 3301.11301.62301.70300.181. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl
OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2024.0Model: Handwritten English Recognition FP16 - Device: CPUdcba50100150200250Min: 299.94 / Avg: 301.62 / Max: 303.47Min: 298.72 / Avg: 301.7 / Max: 304.57Min: 297.97 / Avg: 300.18 / Max: 302.351. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2024.0Model: Handwritten English Recognition FP16 - Device: CPUdcba20406080100SE +/- 0.27, N = 3SE +/- 0.44, N = 3SE +/- 0.33, N = 379.5579.4179.4179.80MIN: 53.81 / MAX: 254.14MIN: 50.66 / MAX: 284.03MIN: 51.94 / MAX: 280.13MIN: 47.2 / MAX: 301.271. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl
OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2024.0Model: Handwritten English Recognition FP16 - Device: CPUdcba1530456075Min: 78.93 / Avg: 79.41 / Max: 79.85Min: 78.66 / Avg: 79.41 / Max: 80.19Min: 79.23 / Avg: 79.8 / Max: 80.381. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2024.0Model: Person Re-Identification Retail FP16 - Device: CPUdcba2004006008001000SE +/- 1.98, N = 3SE +/- 0.77, N = 3SE +/- 1.67, N = 3831.00829.29828.91828.441. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl
OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2024.0Model: Person Re-Identification Retail FP16 - Device: CPUdcba150300450600750Min: 827.02 / Avg: 829.29 / Max: 833.23Min: 827.4 / Avg: 828.91 / Max: 829.9Min: 825.8 / Avg: 828.44 / Max: 831.531. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2024.0Model: Person Re-Identification Retail FP16 - Device: CPUdcba3691215SE +/- 0.02, N = 3SE +/- 0.02, N = 3SE +/- 0.02, N = 39.509.509.489.52MIN: 4.01 / MAX: 56.18MIN: 3.91 / MAX: 57.65MIN: 3.92 / MAX: 63.93MIN: 3.74 / MAX: 55.081. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl
OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2024.0Model: Person Re-Identification Retail FP16 - Device: CPUdcba3691215Min: 9.46 / Avg: 9.5 / Max: 9.53Min: 9.45 / Avg: 9.48 / Max: 9.5Min: 9.49 / Avg: 9.52 / Max: 9.561. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2024.0Model: Age Gender Recognition Retail 0013 FP16 - Device: CPUdcba3K6K9K12K15KSE +/- 8.93, N = 3SE +/- 42.40, N = 3SE +/- 11.80, N = 314593.4614538.0114529.6414614.481. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl
OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2024.0Model: Age Gender Recognition Retail 0013 FP16 - Device: CPUdcba3K6K9K12K15KMin: 14526.26 / Avg: 14538.01 / Max: 14555.53Min: 14480.37 / Avg: 14529.64 / Max: 14614.05Min: 14591.91 / Avg: 14614.48 / Max: 14631.761. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2024.0Model: Age Gender Recognition Retail 0013 FP16 - Device: CPUdcba0.3420.6841.0261.3681.71SE +/- 0.00, N = 3SE +/- 0.00, N = 3SE +/- 0.00, N = 31.511.511.521.51MIN: 0.69 / MAX: 8.28MIN: 0.68 / MAX: 10.22MIN: 0.68 / MAX: 8.23MIN: 0.67 / MAX: 8.281. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl
OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2024.0Model: Age Gender Recognition Retail 0013 FP16 - Device: CPUdcba246810Min: 1.51 / Avg: 1.51 / Max: 1.51Min: 1.51 / Avg: 1.52 / Max: 1.52Min: 1.51 / Avg: 1.51 / Max: 1.511. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2024.0Model: Handwritten English Recognition FP16-INT8 - Device: CPUdcba80160240320400SE +/- 1.54, N = 3SE +/- 1.22, N = 3SE +/- 1.31, N = 3355.52354.07353.91352.111. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl
OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2024.0Model: Handwritten English Recognition FP16-INT8 - Device: CPUdcba60120180240300Min: 350.99 / Avg: 354.07 / Max: 355.73Min: 352.25 / Avg: 353.91 / Max: 356.3Min: 349.62 / Avg: 352.11 / Max: 354.031. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2024.0Model: Handwritten English Recognition FP16-INT8 - Device: CPUdcba1530456075SE +/- 0.30, N = 3SE +/- 0.24, N = 3SE +/- 0.26, N = 367.4167.6667.7068.05MIN: 46.04 / MAX: 238.59MIN: 41.62 / MAX: 242.27MIN: 41.88 / MAX: 256.01MIN: 43.11 / MAX: 237.321. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl
OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2024.0Model: Handwritten English Recognition FP16-INT8 - Device: CPUdcba1326395265Min: 67.35 / Avg: 67.66 / Max: 68.26Min: 67.24 / Avg: 67.7 / Max: 68.04Min: 67.67 / Avg: 68.05 / Max: 68.541. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2024.0Model: Age Gender Recognition Retail 0013 FP16-INT8 - Device: CPUdcba8K16K24K32K40KSE +/- 41.99, N = 3SE +/- 66.76, N = 3SE +/- 92.87, N = 338193.9237977.9838013.0537990.691. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl
OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2024.0Model: Age Gender Recognition Retail 0013 FP16-INT8 - Device: CPUdcba7K14K21K28K35KMin: 37903.95 / Avg: 37977.98 / Max: 38049.35Min: 37880.24 / Avg: 38013.05 / Max: 38091.38Min: 37817.71 / Avg: 37990.69 / Max: 38135.791. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2024.0Model: Age Gender Recognition Retail 0013 FP16-INT8 - Device: CPUdcba0.12830.25660.38490.51320.6415SE +/- 0.00, N = 3SE +/- 0.00, N = 3SE +/- 0.00, N = 30.570.570.570.57MIN: 0.26 / MAX: 5.16MIN: 0.26 / MAX: 5.44MIN: 0.26 / MAX: 6.35MIN: 0.26 / MAX: 7.311. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl
OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2024.0Model: Age Gender Recognition Retail 0013 FP16-INT8 - Device: CPUdcba246810Min: 0.57 / Avg: 0.57 / Max: 0.57Min: 0.57 / Avg: 0.57 / Max: 0.57Min: 0.57 / Avg: 0.57 / Max: 0.571. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl

40 Results Shown

OpenVINO:
  Face Detection FP16 - CPU:
    FPS
    ms
  Person Detection FP16 - CPU:
    FPS
    ms
  Person Detection FP32 - CPU:
    FPS
    ms
  Vehicle Detection FP16 - CPU:
    FPS
    ms
  Face Detection FP16-INT8 - CPU:
    FPS
    ms
  Face Detection Retail FP16 - CPU:
    FPS
    ms
  Road Segmentation ADAS FP16 - CPU:
    FPS
    ms
  Vehicle Detection FP16-INT8 - CPU:
    FPS
    ms
  Weld Porosity Detection FP16 - CPU:
    FPS
    ms
  Face Detection Retail FP16-INT8 - CPU:
    FPS
    ms
  Road Segmentation ADAS FP16-INT8 - CPU:
    FPS
    ms
  Machine Translation EN To DE FP16 - CPU:
    FPS
    ms
  Weld Porosity Detection FP16-INT8 - CPU:
    FPS
    ms
  Person Vehicle Bike Detection FP16 - CPU:
    FPS
    ms
  Noise Suppression Poconet-Like FP16 - CPU:
    FPS
    ms
  Handwritten English Recognition FP16 - CPU:
    FPS
    ms
  Person Re-Identification Retail FP16 - CPU:
    FPS
    ms
  Age Gender Recognition Retail 0013 FP16 - CPU:
    FPS
    ms
  Handwritten English Recognition FP16-INT8 - CPU:
    FPS
    ms
  Age Gender Recognition Retail 0013 FP16-INT8 - CPU:
    FPS
    ms

a

Kernel Notes: Transparent Huge Pages: madvise
Compiler Notes: --build=x86_64-linux-gnu --disable-vtable-verify --disable-werror --enable-bootstrap --enable-cet --enable-checking=release --enable-clocale=gnu --enable-default-pie --enable-gnu-unique-object --enable-languages=c,ada,c++,go,d,fortran,objc,obj-c++,m2 --enable-libphobos-checking=release --enable-libstdcxx-debug --enable-libstdcxx-time=yes --enable-link-serialization=2 --enable-multiarch --enable-multilib --enable-nls --enable-objc-gc=auto --enable-offload-defaulted --enable-offload-targets=nvptx-none=/build/gcc-13-XYspKM/gcc-13-13.2.0/debian/tmp-nvptx/usr,amdgcn-amdhsa=/build/gcc-13-XYspKM/gcc-13-13.2.0/debian/tmp-gcn/usr --enable-plugin --enable-shared --enable-threads=posix --host=x86_64-linux-gnu --program-prefix=x86_64-linux-gnu- --target=x86_64-linux-gnu --with-abi=m64 --with-arch-32=i686 --with-build-config=bootstrap-lto-lean --with-default-libstdcxx-abi=new --with-gcc-major-version-only --with-multilib-list=m32,m64,mx32 --with-target-system-zlib=auto --with-tune=generic --without-cuda-driver -v
Processor Notes: Scaling Governor: intel_pstate powersave (EPP: performance) - CPU Microcode: 0x11d - Thermald 2.5.4
Python Notes: Python 3.11.6
Security Notes: gather_data_sampling: Not affected + itlb_multihit: Not affected + l1tf: Not affected + mds: Not affected + meltdown: Not affected + mmio_stale_data: Not affected + retbleed: Not affected + spec_rstack_overflow: Not affected + spec_store_bypass: Mitigation of SSB disabled via prctl + spectre_v1: Mitigation of usercopy/swapgs barriers and __user pointer sanitization + spectre_v2: Mitigation of Enhanced / Automatic IBRS IBPB: conditional RSB filling PBRSB-eIBRS: SW sequence + srbds: Not affected + tsx_async_abort: Not affected

Testing initiated at 7 March 2024 13:29 by user pts.

b

Kernel Notes: Transparent Huge Pages: madvise
Compiler Notes: --build=x86_64-linux-gnu --disable-vtable-verify --disable-werror --enable-bootstrap --enable-cet --enable-checking=release --enable-clocale=gnu --enable-default-pie --enable-gnu-unique-object --enable-languages=c,ada,c++,go,d,fortran,objc,obj-c++,m2 --enable-libphobos-checking=release --enable-libstdcxx-debug --enable-libstdcxx-time=yes --enable-link-serialization=2 --enable-multiarch --enable-multilib --enable-nls --enable-objc-gc=auto --enable-offload-defaulted --enable-offload-targets=nvptx-none=/build/gcc-13-XYspKM/gcc-13-13.2.0/debian/tmp-nvptx/usr,amdgcn-amdhsa=/build/gcc-13-XYspKM/gcc-13-13.2.0/debian/tmp-gcn/usr --enable-plugin --enable-shared --enable-threads=posix --host=x86_64-linux-gnu --program-prefix=x86_64-linux-gnu- --target=x86_64-linux-gnu --with-abi=m64 --with-arch-32=i686 --with-build-config=bootstrap-lto-lean --with-default-libstdcxx-abi=new --with-gcc-major-version-only --with-multilib-list=m32,m64,mx32 --with-target-system-zlib=auto --with-tune=generic --without-cuda-driver -v
Processor Notes: Scaling Governor: intel_pstate powersave (EPP: performance) - CPU Microcode: 0x11d - Thermald 2.5.4
Python Notes: Python 3.11.6
Security Notes: gather_data_sampling: Not affected + itlb_multihit: Not affected + l1tf: Not affected + mds: Not affected + meltdown: Not affected + mmio_stale_data: Not affected + retbleed: Not affected + spec_rstack_overflow: Not affected + spec_store_bypass: Mitigation of SSB disabled via prctl + spectre_v1: Mitigation of usercopy/swapgs barriers and __user pointer sanitization + spectre_v2: Mitigation of Enhanced / Automatic IBRS IBPB: conditional RSB filling PBRSB-eIBRS: SW sequence + srbds: Not affected + tsx_async_abort: Not affected

Testing initiated at 7 March 2024 14:42 by user pts.

c

Kernel Notes: Transparent Huge Pages: madvise
Compiler Notes: --build=x86_64-linux-gnu --disable-vtable-verify --disable-werror --enable-bootstrap --enable-cet --enable-checking=release --enable-clocale=gnu --enable-default-pie --enable-gnu-unique-object --enable-languages=c,ada,c++,go,d,fortran,objc,obj-c++,m2 --enable-libphobos-checking=release --enable-libstdcxx-debug --enable-libstdcxx-time=yes --enable-link-serialization=2 --enable-multiarch --enable-multilib --enable-nls --enable-objc-gc=auto --enable-offload-defaulted --enable-offload-targets=nvptx-none=/build/gcc-13-XYspKM/gcc-13-13.2.0/debian/tmp-nvptx/usr,amdgcn-amdhsa=/build/gcc-13-XYspKM/gcc-13-13.2.0/debian/tmp-gcn/usr --enable-plugin --enable-shared --enable-threads=posix --host=x86_64-linux-gnu --program-prefix=x86_64-linux-gnu- --target=x86_64-linux-gnu --with-abi=m64 --with-arch-32=i686 --with-build-config=bootstrap-lto-lean --with-default-libstdcxx-abi=new --with-gcc-major-version-only --with-multilib-list=m32,m64,mx32 --with-target-system-zlib=auto --with-tune=generic --without-cuda-driver -v
Processor Notes: Scaling Governor: intel_pstate powersave (EPP: performance) - CPU Microcode: 0x11d - Thermald 2.5.4
Python Notes: Python 3.11.6
Security Notes: gather_data_sampling: Not affected + itlb_multihit: Not affected + l1tf: Not affected + mds: Not affected + meltdown: Not affected + mmio_stale_data: Not affected + retbleed: Not affected + spec_rstack_overflow: Not affected + spec_store_bypass: Mitigation of SSB disabled via prctl + spectre_v1: Mitigation of usercopy/swapgs barriers and __user pointer sanitization + spectre_v2: Mitigation of Enhanced / Automatic IBRS IBPB: conditional RSB filling PBRSB-eIBRS: SW sequence + srbds: Not affected + tsx_async_abort: Not affected

Testing initiated at 7 March 2024 16:11 by user pts.

d

Processor: Intel Core i9-14900K @ 5.70GHz (24 Cores / 32 Threads), Motherboard: ASUS PRIME Z790-P WIFI (1402 BIOS), Chipset: Intel Device 7a27, Memory: 2 x 16GB DRAM-6000MT/s Corsair CMK32GX5M2B6000C36, Disk: Western Digital WD_BLACK SN850X 1000GB, Graphics: ASUS Intel RPL-S 31GB (1650MHz), Audio: Realtek ALC897, Monitor: ASUS VP28U

OS: Ubuntu 23.10, Kernel: 6.7.0-060700rc7daily20231224-generic (x86_64), Desktop: GNOME Shell 45.1, Display Server: X Server 1.21.1.7, OpenGL: 4.6 Mesa 24.0~git2312240600.c05261~oibaf~m (git-c05261a 2023-12-24 mantic-oibaf-ppa), OpenCL: OpenCL 3.0, Compiler: GCC 13.2.0, File-System: ext4, Screen Resolution: 1920x1080

Kernel Notes: Transparent Huge Pages: madvise
Compiler Notes: --build=x86_64-linux-gnu --disable-vtable-verify --disable-werror --enable-bootstrap --enable-cet --enable-checking=release --enable-clocale=gnu --enable-default-pie --enable-gnu-unique-object --enable-languages=c,ada,c++,go,d,fortran,objc,obj-c++,m2 --enable-libphobos-checking=release --enable-libstdcxx-debug --enable-libstdcxx-time=yes --enable-link-serialization=2 --enable-multiarch --enable-multilib --enable-nls --enable-objc-gc=auto --enable-offload-defaulted --enable-offload-targets=nvptx-none=/build/gcc-13-XYspKM/gcc-13-13.2.0/debian/tmp-nvptx/usr,amdgcn-amdhsa=/build/gcc-13-XYspKM/gcc-13-13.2.0/debian/tmp-gcn/usr --enable-plugin --enable-shared --enable-threads=posix --host=x86_64-linux-gnu --program-prefix=x86_64-linux-gnu- --target=x86_64-linux-gnu --with-abi=m64 --with-arch-32=i686 --with-build-config=bootstrap-lto-lean --with-default-libstdcxx-abi=new --with-gcc-major-version-only --with-multilib-list=m32,m64,mx32 --with-target-system-zlib=auto --with-tune=generic --without-cuda-driver -v
Processor Notes: Scaling Governor: intel_pstate powersave (EPP: performance) - CPU Microcode: 0x11d - Thermald 2.5.4
Python Notes: Python 3.11.6
Security Notes: gather_data_sampling: Not affected + itlb_multihit: Not affected + l1tf: Not affected + mds: Not affected + meltdown: Not affected + mmio_stale_data: Not affected + retbleed: Not affected + spec_rstack_overflow: Not affected + spec_store_bypass: Mitigation of SSB disabled via prctl + spectre_v1: Mitigation of usercopy/swapgs barriers and __user pointer sanitization + spectre_v2: Mitigation of Enhanced / Automatic IBRS IBPB: conditional RSB filling PBRSB-eIBRS: SW sequence + srbds: Not affected + tsx_async_abort: Not affected

Testing initiated at 7 March 2024 17:19 by user pts.