openvino raptor lake Intel Core i9-13900K testing with a ASUS PRIME Z790-P WIFI (0602 BIOS) and eVGA NVIDIA GeForce RTX 3060 12GB on Ubuntu 22.10 via the Phoronix Test Suite.
HTML result view exported from: https://openbenchmarking.org/result/2212216-PTS-OPENVINO35 .
openvino raptor lake Processor Motherboard Chipset Memory Disk Graphics Audio Monitor Network OS Kernel Desktop Display Server Display Driver OpenGL OpenCL Vulkan Compiler File-System Screen Resolution a b c d Intel Core i9-13900K @ 4.00GHz (24 Cores / 32 Threads) ASUS PRIME Z790-P WIFI (0602 BIOS) Intel Device 7a27 32GB 1000GB Western Digital WDS100T1X0E-00AFY0 eVGA NVIDIA GeForce RTX 3060 12GB Realtek ALC897 ASUS VP28U Realtek RTL8125 2.5GbE + Intel Device 7a70 Ubuntu 22.10 5.19.0-26-generic (x86_64) GNOME Shell 43.1 X Server 1.21.1.4 NVIDIA 525.60.11 4.6.0 OpenCL 3.0 CUDA 12.0.89 1.3.224 GCC 12.2.0 ext4 2560x1600 OpenBenchmarking.org Kernel Details - Transparent Huge Pages: madvise Compiler Details - --build=x86_64-linux-gnu --disable-vtable-verify --disable-werror --enable-cet --enable-checking=release --enable-clocale=gnu --enable-default-pie --enable-gnu-unique-object --enable-languages=c,ada,c++,go,d,fortran,objc,obj-c++,m2 --enable-libphobos-checking=release --enable-libstdcxx-debug --enable-libstdcxx-time=yes --enable-multiarch --enable-multilib --enable-nls --enable-objc-gc=auto --enable-offload-defaulted --enable-offload-targets=nvptx-none=/build/gcc-12-U8K4Qv/gcc-12-12.2.0/debian/tmp-nvptx/usr,amdgcn-amdhsa=/build/gcc-12-U8K4Qv/gcc-12-12.2.0/debian/tmp-gcn/usr --enable-plugin --enable-shared --enable-threads=posix --host=x86_64-linux-gnu --program-prefix=x86_64-linux-gnu- --target=x86_64-linux-gnu --with-abi=m64 --with-arch-32=i686 --with-default-libstdcxx-abi=new --with-gcc-major-version-only --with-multilib-list=m32,m64,mx32 --with-target-system-zlib=auto --with-tune=generic --without-cuda-driver -v Processor Details - Scaling Governor: intel_pstate powersave (EPP: balance_performance) - CPU Microcode: 0x10e - Thermald 2.5.1 Python Details - Python 3.10.7 Security Details - itlb_multihit: Not affected + l1tf: Not affected + mds: Not affected + meltdown: Not affected + mmio_stale_data: Not affected + retbleed: Not affected + spec_store_bypass: Mitigation of SSB disabled via prctl + spectre_v1: Mitigation of usercopy/swapgs barriers and __user pointer sanitization + spectre_v2: Mitigation of Enhanced IBRS IBPB: conditional RSB filling PBRSB-eIBRS: SW sequence + srbds: Not affected + tsx_async_abort: Not affected
openvino raptor lake openvino: Face Detection FP16 - CPU openvino: Face Detection FP16 - CPU openvino: Person Detection FP16 - CPU openvino: Person Detection FP16 - CPU openvino: Person Detection FP32 - CPU openvino: Person Detection FP32 - CPU openvino: Vehicle Detection FP16 - CPU openvino: Vehicle Detection FP16 - CPU openvino: Face Detection FP16-INT8 - CPU openvino: Face Detection FP16-INT8 - CPU openvino: Vehicle Detection FP16-INT8 - CPU openvino: Vehicle Detection FP16-INT8 - CPU openvino: Weld Porosity Detection FP16 - CPU openvino: Weld Porosity Detection FP16 - CPU openvino: Machine Translation EN To DE FP16 - CPU openvino: Machine Translation EN To DE FP16 - CPU openvino: Weld Porosity Detection FP16-INT8 - CPU openvino: Weld Porosity Detection FP16-INT8 - CPU openvino: Person Vehicle Bike Detection FP16 - CPU openvino: Person Vehicle Bike Detection FP16 - CPU openvino: Age Gender Recognition Retail 0013 FP16 - CPU openvino: Age Gender Recognition Retail 0013 FP16 - CPU openvino: Age Gender Recognition Retail 0013 FP16-INT8 - CPU openvino: Age Gender Recognition Retail 0013 FP16-INT8 - CPU a b c d 5.57 1419.72 3.6 2207.86 3.58 2223.11 376.60 21.22 18.36 434.55 956.53 8.36 472.65 50.68 64.07 124.69 1655.57 14.49 744.16 10.74 15483.60 1.55 16831.03 1.42 5.64 1401.8 3.62 2200.14 3.61 2198.94 376.67 21.22 18.3 434.72 957.06 8.35 476.33 50.29 64.36 124.1 1662.48 14.43 742.03 10.77 15426.72 1.55 16830.79 1.42 5.64 1401.09 3.6 2201.38 3.59 2218.77 377.28 21.18 18.35 433.89 957.65 8.35 475.9 50.35 64.29 124.24 1664.63 14.41 745.59 10.72 15412.63 1.55 16839.44 1.42 5.64 1402.77 3.6 2202.6 3.58 2223.42 377.45 21.17 18.35 433.79 957.1 8.35 475.81 50.34 64.19 124.45 1598.2 15 744.39 10.73 15433.26 1.55 16861.9 1.42 OpenBenchmarking.org
OpenVINO Model: Face Detection FP16 - Device: CPU OpenBenchmarking.org FPS, More Is Better OpenVINO 2022.3 Model: Face Detection FP16 - Device: CPU a b c d 1.269 2.538 3.807 5.076 6.345 SE +/- 0.03, N = 3 5.57 5.64 5.64 5.64 1. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared
OpenVINO Model: Face Detection FP16 - Device: CPU OpenBenchmarking.org ms, Fewer Is Better OpenVINO 2022.3 Model: Face Detection FP16 - Device: CPU a b c d 300 600 900 1200 1500 SE +/- 8.72, N = 3 1419.72 1401.80 1401.09 1402.77 MIN: 1134.36 / MAX: 1882.77 MIN: 1129.3 / MAX: 1844.12 MIN: 1136.99 / MAX: 1832.13 MIN: 1135.93 / MAX: 1867.52 1. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared
OpenVINO Model: Person Detection FP16 - Device: CPU OpenBenchmarking.org FPS, More Is Better OpenVINO 2022.3 Model: Person Detection FP16 - Device: CPU a b c d 0.8145 1.629 2.4435 3.258 4.0725 SE +/- 0.00, N = 3 3.60 3.62 3.60 3.60 1. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared
OpenVINO Model: Person Detection FP16 - Device: CPU OpenBenchmarking.org ms, Fewer Is Better OpenVINO 2022.3 Model: Person Detection FP16 - Device: CPU a b c d 500 1000 1500 2000 2500 SE +/- 2.27, N = 3 2207.86 2200.14 2201.38 2202.60 MIN: 1673.83 / MAX: 2908.83 MIN: 1684.28 / MAX: 2958.95 MIN: 1686.01 / MAX: 2941.32 MIN: 1675.81 / MAX: 2906.09 1. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared
OpenVINO Model: Person Detection FP32 - Device: CPU OpenBenchmarking.org FPS, More Is Better OpenVINO 2022.3 Model: Person Detection FP32 - Device: CPU a b c d 0.8123 1.6246 2.4369 3.2492 4.0615 SE +/- 0.01, N = 3 3.58 3.61 3.59 3.58 1. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared
OpenVINO Model: Person Detection FP32 - Device: CPU OpenBenchmarking.org ms, Fewer Is Better OpenVINO 2022.3 Model: Person Detection FP32 - Device: CPU a b c d 500 1000 1500 2000 2500 SE +/- 4.49, N = 3 2223.11 2198.94 2218.77 2223.42 MIN: 1689.29 / MAX: 2969.01 MIN: 1688 / MAX: 2883.02 MIN: 1727.72 / MAX: 2908.79 MIN: 1746.38 / MAX: 2958.09 1. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared
OpenVINO Model: Vehicle Detection FP16 - Device: CPU OpenBenchmarking.org FPS, More Is Better OpenVINO 2022.3 Model: Vehicle Detection FP16 - Device: CPU a b c d 80 160 240 320 400 SE +/- 0.49, N = 3 376.60 376.67 377.28 377.45 1. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared
OpenVINO Model: Vehicle Detection FP16 - Device: CPU OpenBenchmarking.org ms, Fewer Is Better OpenVINO 2022.3 Model: Vehicle Detection FP16 - Device: CPU a b c d 5 10 15 20 25 SE +/- 0.03, N = 3 21.22 21.22 21.18 21.17 MIN: 12.04 / MAX: 80.03 MIN: 12.08 / MAX: 79.96 MIN: 11.26 / MAX: 82.65 MIN: 12 / MAX: 77.59 1. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared
OpenVINO Model: Face Detection FP16-INT8 - Device: CPU OpenBenchmarking.org FPS, More Is Better OpenVINO 2022.3 Model: Face Detection FP16-INT8 - Device: CPU a b c d 5 10 15 20 25 SE +/- 0.02, N = 3 18.36 18.30 18.35 18.35 1. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared
OpenVINO Model: Face Detection FP16-INT8 - Device: CPU OpenBenchmarking.org ms, Fewer Is Better OpenVINO 2022.3 Model: Face Detection FP16-INT8 - Device: CPU a b c d 90 180 270 360 450 SE +/- 0.20, N = 3 434.55 434.72 433.89 433.79 MIN: 272.08 / MAX: 1054.76 MIN: 272.19 / MAX: 969.01 MIN: 272.23 / MAX: 1004.38 MIN: 272.11 / MAX: 1053.1 1. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared
OpenVINO Model: Vehicle Detection FP16-INT8 - Device: CPU OpenBenchmarking.org FPS, More Is Better OpenVINO 2022.3 Model: Vehicle Detection FP16-INT8 - Device: CPU a b c d 200 400 600 800 1000 SE +/- 1.17, N = 3 956.53 957.06 957.65 957.10 1. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared
OpenVINO Model: Vehicle Detection FP16-INT8 - Device: CPU OpenBenchmarking.org ms, Fewer Is Better OpenVINO 2022.3 Model: Vehicle Detection FP16-INT8 - Device: CPU a b c d 2 4 6 8 10 SE +/- 0.01, N = 3 8.36 8.35 8.35 8.35 MIN: 5.61 / MAX: 60.05 MIN: 5.62 / MAX: 54.66 MIN: 5.62 / MAX: 58.92 MIN: 5.62 / MAX: 59.89 1. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared
OpenVINO Model: Weld Porosity Detection FP16 - Device: CPU OpenBenchmarking.org FPS, More Is Better OpenVINO 2022.3 Model: Weld Porosity Detection FP16 - Device: CPU a b c d 100 200 300 400 500 SE +/- 0.51, N = 3 472.65 476.33 475.90 475.81 1. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared
OpenVINO Model: Weld Porosity Detection FP16 - Device: CPU OpenBenchmarking.org ms, Fewer Is Better OpenVINO 2022.3 Model: Weld Porosity Detection FP16 - Device: CPU a b c d 11 22 33 44 55 SE +/- 0.05, N = 3 50.68 50.29 50.35 50.34 MIN: 22.67 / MAX: 238.75 MIN: 27.83 / MAX: 180.56 MIN: 22.11 / MAX: 161.3 MIN: 32.35 / MAX: 187.9 1. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared
OpenVINO Model: Machine Translation EN To DE FP16 - Device: CPU OpenBenchmarking.org FPS, More Is Better OpenVINO 2022.3 Model: Machine Translation EN To DE FP16 - Device: CPU a b c d 14 28 42 56 70 SE +/- 0.23, N = 3 64.07 64.36 64.29 64.19 1. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared
OpenVINO Model: Machine Translation EN To DE FP16 - Device: CPU OpenBenchmarking.org ms, Fewer Is Better OpenVINO 2022.3 Model: Machine Translation EN To DE FP16 - Device: CPU a b c d 30 60 90 120 150 SE +/- 0.44, N = 3 124.69 124.10 124.24 124.45 MIN: 85.84 / MAX: 383.5 MIN: 87.91 / MAX: 317.47 MIN: 89.89 / MAX: 315.41 MIN: 91.81 / MAX: 309.85 1. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared
OpenVINO Model: Weld Porosity Detection FP16-INT8 - Device: CPU OpenBenchmarking.org FPS, More Is Better OpenVINO 2022.3 Model: Weld Porosity Detection FP16-INT8 - Device: CPU a b c d 400 800 1200 1600 2000 SE +/- 2.42, N = 3 1655.57 1662.48 1664.63 1598.20 1. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared
OpenVINO Model: Weld Porosity Detection FP16-INT8 - Device: CPU OpenBenchmarking.org ms, Fewer Is Better OpenVINO 2022.3 Model: Weld Porosity Detection FP16-INT8 - Device: CPU a b c d 4 8 12 16 20 SE +/- 0.02, N = 3 14.49 14.43 14.41 15.00 MIN: 7.88 / MAX: 126.12 MIN: 8.87 / MAX: 115.85 MIN: 8.75 / MAX: 113.39 MIN: 6.56 / MAX: 146.68 1. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared
OpenVINO Model: Person Vehicle Bike Detection FP16 - Device: CPU OpenBenchmarking.org FPS, More Is Better OpenVINO 2022.3 Model: Person Vehicle Bike Detection FP16 - Device: CPU a b c d 160 320 480 640 800 SE +/- 1.44, N = 3 744.16 742.03 745.59 744.39 1. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared
OpenVINO Model: Person Vehicle Bike Detection FP16 - Device: CPU OpenBenchmarking.org ms, Fewer Is Better OpenVINO 2022.3 Model: Person Vehicle Bike Detection FP16 - Device: CPU a b c d 3 6 9 12 15 SE +/- 0.02, N = 3 10.74 10.77 10.72 10.73 MIN: 7.6 / MAX: 53.62 MIN: 7.66 / MAX: 49.95 MIN: 7.61 / MAX: 53.19 MIN: 7.61 / MAX: 50.38 1. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared
OpenVINO Model: Age Gender Recognition Retail 0013 FP16 - Device: CPU OpenBenchmarking.org FPS, More Is Better OpenVINO 2022.3 Model: Age Gender Recognition Retail 0013 FP16 - Device: CPU a b c d 3K 6K 9K 12K 15K SE +/- 13.32, N = 3 15483.60 15426.72 15412.63 15433.26 1. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared
OpenVINO Model: Age Gender Recognition Retail 0013 FP16 - Device: CPU OpenBenchmarking.org ms, Fewer Is Better OpenVINO 2022.3 Model: Age Gender Recognition Retail 0013 FP16 - Device: CPU a b c d 0.3488 0.6976 1.0464 1.3952 1.744 SE +/- 0.00, N = 3 1.55 1.55 1.55 1.55 MIN: 0.93 / MAX: 9.11 MIN: 0.92 / MAX: 9.16 MIN: 0.97 / MAX: 9.21 MIN: 0.97 / MAX: 9.07 1. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared
OpenVINO Model: Age Gender Recognition Retail 0013 FP16-INT8 - Device: CPU OpenBenchmarking.org FPS, More Is Better OpenVINO 2022.3 Model: Age Gender Recognition Retail 0013 FP16-INT8 - Device: CPU a b c d 4K 8K 12K 16K 20K SE +/- 7.70, N = 3 16831.03 16830.79 16839.44 16861.90 1. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared
OpenVINO Model: Age Gender Recognition Retail 0013 FP16-INT8 - Device: CPU OpenBenchmarking.org ms, Fewer Is Better OpenVINO 2022.3 Model: Age Gender Recognition Retail 0013 FP16-INT8 - Device: CPU a b c d 0.3195 0.639 0.9585 1.278 1.5975 SE +/- 0.00, N = 3 1.42 1.42 1.42 1.42 MIN: 0.66 / MAX: 8.42 MIN: 0.67 / MAX: 8.47 MIN: 0.84 / MAX: 8.42 MIN: 0.89 / MAX: 8.42 1. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared
Phoronix Test Suite v10.8.4