openvino-all-1-5-5600RAM 2 x INTEL XEON PLATINUM 8568Y+ testing with a Supermicro X13DAI-T v1.01 (2.4 BIOS) and NVIDIA A100-SXM4-40GB on Ubuntu 24.04 via the Phoronix Test Suite.
HTML result view exported from: https://openbenchmarking.org/result/2412095-NE-OPENVINOA94&grw .
openvino-all-1-5-5600RAM Processor Motherboard Chipset Memory Disk Graphics Audio Monitor Network OS Kernel Desktop Display Server Display Driver OpenGL OpenCL Compiler File-System Screen Resolution 2 x INTEL XEON PLATINUM 8568Y+ - NVIDIA 2 x INTEL XEON PLATINUM 8568Y+ @ 4.00GHz (96 Cores / 192 Threads) Supermicro X13DAI-T v1.01 (2.4 BIOS) Intel Device 1bce 1008GB 2 x 15363GB SAMSUNG MZQL215THBLA-00A07 + 30725GB 9620-16i NVIDIA A100-SXM4-40GB Realtek ALC888-VD VGA HDMI 2 x Broadcom BCM57416 NetXtreme-E Dual-Media 10G RDMA Ubuntu 24.04 6.8.0-49-generic (x86_64) GNOME Shell 46.0 X Server 1.21.1.11 NVIDIA 565.57.01 4.6.0 OpenCL 3.0 CUDA 12.7.33 GCC 13.2.0 + CUDA 12.0 ext4 1920x1080 OpenBenchmarking.org - Transparent Huge Pages: madvise - __GLX_VENDOR_LIBRARY_NAME=nvidia - --build=x86_64-linux-gnu --disable-vtable-verify --disable-werror --enable-cet --enable-checking=release --enable-clocale=gnu --enable-default-pie --enable-gnu-unique-object --enable-languages=c,ada,c++,go,d,fortran,objc,obj-c++,m2 --enable-libphobos-checking=release --enable-libstdcxx-backtrace --enable-libstdcxx-debug --enable-libstdcxx-time=yes --enable-multiarch --enable-multilib --enable-nls --enable-objc-gc=auto --enable-offload-defaulted --enable-offload-targets=nvptx-none=/build/gcc-13-uJ7kn6/gcc-13-13.2.0/debian/tmp-nvptx/usr,amdgcn-amdhsa=/build/gcc-13-uJ7kn6/gcc-13-13.2.0/debian/tmp-gcn/usr --enable-plugin --enable-shared --enable-threads=posix --host=x86_64-linux-gnu --program-prefix=x86_64-linux-gnu- --target=x86_64-linux-gnu --with-abi=m64 --with-arch-32=i686 --with-default-libstdcxx-abi=new --with-gcc-major-version-only --with-multilib-list=m32,m64,mx32 --with-target-system-zlib=auto --with-tune=generic --without-cuda-driver -v - Scaling Governor: intel_cpufreq performance - CPU Microcode: 0x21000283 - Python 3.12.3 - gather_data_sampling: Not affected + itlb_multihit: Not affected + l1tf: Not affected + mds: Not affected + meltdown: Not affected + mmio_stale_data: Not affected + reg_file_data_sampling: Not affected + retbleed: Not affected + spec_rstack_overflow: Not affected + spec_store_bypass: Mitigation of SSB disabled via prctl + spectre_v1: Mitigation of usercopy/swapgs barriers and __user pointer sanitization + spectre_v2: Mitigation of Enhanced / Automatic IBRS; IBPB: conditional; RSB filling; PBRSB-eIBRS: SW sequence; BHI: BHI_DIS_S + srbds: Not affected + tsx_async_abort: Not affected
openvino-all-1-5-5600RAM openvino: Face Detection FP16 - CPU openvino: Face Detection FP16 - CPU openvino: Person Detection FP16 - CPU openvino: Person Detection FP16 - CPU openvino: Person Detection FP32 - CPU openvino: Person Detection FP32 - CPU openvino: Vehicle Detection FP16 - CPU openvino: Vehicle Detection FP16 - CPU openvino: Face Detection FP16-INT8 - CPU openvino: Face Detection FP16-INT8 - CPU openvino: Face Detection Retail FP16 - CPU openvino: Face Detection Retail FP16 - CPU openvino: Road Segmentation ADAS FP16 - CPU openvino: Road Segmentation ADAS FP16 - CPU openvino: Vehicle Detection FP16-INT8 - CPU openvino: Vehicle Detection FP16-INT8 - CPU openvino: Weld Porosity Detection FP16 - CPU openvino: Weld Porosity Detection FP16 - CPU openvino: Face Detection Retail FP16-INT8 - CPU openvino: Face Detection Retail FP16-INT8 - CPU openvino: Road Segmentation ADAS FP16-INT8 - CPU openvino: Road Segmentation ADAS FP16-INT8 - CPU openvino: Machine Translation EN To DE FP16 - CPU openvino: Machine Translation EN To DE FP16 - CPU openvino: Weld Porosity Detection FP16-INT8 - CPU openvino: Weld Porosity Detection FP16-INT8 - CPU openvino: Person Vehicle Bike Detection FP16 - CPU openvino: Person Vehicle Bike Detection FP16 - CPU openvino: Noise Suppression Poconet-Like FP16 - CPU openvino: Noise Suppression Poconet-Like FP16 - CPU openvino: Handwritten English Recognition FP16 - CPU openvino: Handwritten English Recognition FP16 - CPU openvino: Person Re-Identification Retail FP16 - CPU openvino: Person Re-Identification Retail FP16 - CPU openvino: Age Gender Recognition Retail 0013 FP16 - CPU openvino: Age Gender Recognition Retail 0013 FP16 - CPU openvino: Handwritten English Recognition FP16-INT8 - CPU openvino: Handwritten English Recognition FP16-INT8 - CPU openvino: Age Gender Recognition Retail 0013 FP16-INT8 - CPU openvino: Age Gender Recognition Retail 0013 FP16-INT8 - CPU 2 x INTEL XEON PLATINUM 8568Y+ - NVIDIA 217.65 110.15 640.18 37.44 639.27 37.50 4718.51 5.08 485.08 197.61 15822.91 6.06 1533.13 15.62 7453.60 12.86 27449.74 3.48 21237.30 4.51 1911.37 50.12 979.61 24.40 41978.13 2.19 9369.40 10.23 7708.27 12.33 4811.57 19.94 15649.17 6.12 112806.06 0.45 4866.77 19.71 124878.26 0.39 OpenBenchmarking.org
OpenVINO Model: Face Detection FP16 - Device: CPU OpenBenchmarking.org FPS, More Is Better OpenVINO 2024.0 Model: Face Detection FP16 - Device: CPU 2 x INTEL XEON PLATINUM 8568Y+ - NVIDIA 50 100 150 200 250 SE +/- 0.23, N = 3 217.65 1. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl
OpenVINO Model: Face Detection FP16 - Device: CPU OpenBenchmarking.org ms, Fewer Is Better OpenVINO 2024.0 Model: Face Detection FP16 - Device: CPU 2 x INTEL XEON PLATINUM 8568Y+ - NVIDIA 20 40 60 80 100 SE +/- 0.11, N = 3 110.15 MIN: 89.68 / MAX: 203.46 1. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl
OpenVINO Model: Person Detection FP16 - Device: CPU OpenBenchmarking.org FPS, More Is Better OpenVINO 2024.0 Model: Person Detection FP16 - Device: CPU 2 x INTEL XEON PLATINUM 8568Y+ - NVIDIA 140 280 420 560 700 SE +/- 1.19, N = 3 640.18 1. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl
OpenVINO Model: Person Detection FP16 - Device: CPU OpenBenchmarking.org ms, Fewer Is Better OpenVINO 2024.0 Model: Person Detection FP16 - Device: CPU 2 x INTEL XEON PLATINUM 8568Y+ - NVIDIA 9 18 27 36 45 SE +/- 0.07, N = 3 37.44 MIN: 28.8 / MAX: 66.52 1. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl
OpenVINO Model: Person Detection FP32 - Device: CPU OpenBenchmarking.org FPS, More Is Better OpenVINO 2024.0 Model: Person Detection FP32 - Device: CPU 2 x INTEL XEON PLATINUM 8568Y+ - NVIDIA 140 280 420 560 700 SE +/- 0.77, N = 3 639.27 1. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl
OpenVINO Model: Person Detection FP32 - Device: CPU OpenBenchmarking.org ms, Fewer Is Better OpenVINO 2024.0 Model: Person Detection FP32 - Device: CPU 2 x INTEL XEON PLATINUM 8568Y+ - NVIDIA 9 18 27 36 45 SE +/- 0.05, N = 3 37.50 MIN: 29.54 / MAX: 74.5 1. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl
OpenVINO Model: Vehicle Detection FP16 - Device: CPU OpenBenchmarking.org FPS, More Is Better OpenVINO 2024.0 Model: Vehicle Detection FP16 - Device: CPU 2 x INTEL XEON PLATINUM 8568Y+ - NVIDIA 1000 2000 3000 4000 5000 SE +/- 11.89, N = 3 4718.51 1. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl
OpenVINO Model: Vehicle Detection FP16 - Device: CPU OpenBenchmarking.org ms, Fewer Is Better OpenVINO 2024.0 Model: Vehicle Detection FP16 - Device: CPU 2 x INTEL XEON PLATINUM 8568Y+ - NVIDIA 1.143 2.286 3.429 4.572 5.715 SE +/- 0.01, N = 3 5.08 MIN: 4.47 / MAX: 23.86 1. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl
OpenVINO Model: Face Detection FP16-INT8 - Device: CPU OpenBenchmarking.org FPS, More Is Better OpenVINO 2024.0 Model: Face Detection FP16-INT8 - Device: CPU 2 x INTEL XEON PLATINUM 8568Y+ - NVIDIA 100 200 300 400 500 SE +/- 1.00, N = 3 485.08 1. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl
OpenVINO Model: Face Detection FP16-INT8 - Device: CPU OpenBenchmarking.org ms, Fewer Is Better OpenVINO 2024.0 Model: Face Detection FP16-INT8 - Device: CPU 2 x INTEL XEON PLATINUM 8568Y+ - NVIDIA 40 80 120 160 200 SE +/- 0.40, N = 3 197.61 MIN: 136.33 / MAX: 342.48 1. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl
OpenVINO Model: Face Detection Retail FP16 - Device: CPU OpenBenchmarking.org FPS, More Is Better OpenVINO 2024.0 Model: Face Detection Retail FP16 - Device: CPU 2 x INTEL XEON PLATINUM 8568Y+ - NVIDIA 3K 6K 9K 12K 15K SE +/- 6.11, N = 3 15822.91 1. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl
OpenVINO Model: Face Detection Retail FP16 - Device: CPU OpenBenchmarking.org ms, Fewer Is Better OpenVINO 2024.0 Model: Face Detection Retail FP16 - Device: CPU 2 x INTEL XEON PLATINUM 8568Y+ - NVIDIA 2 4 6 8 10 SE +/- 0.00, N = 3 6.06 MIN: 5.27 / MAX: 22.22 1. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl
OpenVINO Model: Road Segmentation ADAS FP16 - Device: CPU OpenBenchmarking.org FPS, More Is Better OpenVINO 2024.0 Model: Road Segmentation ADAS FP16 - Device: CPU 2 x INTEL XEON PLATINUM 8568Y+ - NVIDIA 300 600 900 1200 1500 SE +/- 1.20, N = 3 1533.13 1. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl
OpenVINO Model: Road Segmentation ADAS FP16 - Device: CPU OpenBenchmarking.org ms, Fewer Is Better OpenVINO 2024.0 Model: Road Segmentation ADAS FP16 - Device: CPU 2 x INTEL XEON PLATINUM 8568Y+ - NVIDIA 4 8 12 16 20 SE +/- 0.01, N = 3 15.62 MIN: 13.53 / MAX: 43.51 1. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl
OpenVINO Model: Vehicle Detection FP16-INT8 - Device: CPU OpenBenchmarking.org FPS, More Is Better OpenVINO 2024.0 Model: Vehicle Detection FP16-INT8 - Device: CPU 2 x INTEL XEON PLATINUM 8568Y+ - NVIDIA 1600 3200 4800 6400 8000 SE +/- 3.67, N = 3 7453.60 1. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl
OpenVINO Model: Vehicle Detection FP16-INT8 - Device: CPU OpenBenchmarking.org ms, Fewer Is Better OpenVINO 2024.0 Model: Vehicle Detection FP16-INT8 - Device: CPU 2 x INTEL XEON PLATINUM 8568Y+ - NVIDIA 3 6 9 12 15 SE +/- 0.01, N = 3 12.86 MIN: 10.87 / MAX: 27.61 1. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl
OpenVINO Model: Weld Porosity Detection FP16 - Device: CPU OpenBenchmarking.org FPS, More Is Better OpenVINO 2024.0 Model: Weld Porosity Detection FP16 - Device: CPU 2 x INTEL XEON PLATINUM 8568Y+ - NVIDIA 6K 12K 18K 24K 30K SE +/- 100.49, N = 3 27449.74 1. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl
OpenVINO Model: Weld Porosity Detection FP16 - Device: CPU OpenBenchmarking.org ms, Fewer Is Better OpenVINO 2024.0 Model: Weld Porosity Detection FP16 - Device: CPU 2 x INTEL XEON PLATINUM 8568Y+ - NVIDIA 0.783 1.566 2.349 3.132 3.915 SE +/- 0.01, N = 3 3.48 MIN: 2.95 / MAX: 23.87 1. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl
OpenVINO Model: Face Detection Retail FP16-INT8 - Device: CPU OpenBenchmarking.org FPS, More Is Better OpenVINO 2024.0 Model: Face Detection Retail FP16-INT8 - Device: CPU 2 x INTEL XEON PLATINUM 8568Y+ - NVIDIA 5K 10K 15K 20K 25K SE +/- 9.45, N = 3 21237.30 1. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl
OpenVINO Model: Face Detection Retail FP16-INT8 - Device: CPU OpenBenchmarking.org ms, Fewer Is Better OpenVINO 2024.0 Model: Face Detection Retail FP16-INT8 - Device: CPU 2 x INTEL XEON PLATINUM 8568Y+ - NVIDIA 1.0148 2.0296 3.0444 4.0592 5.074 SE +/- 0.00, N = 3 4.51 MIN: 3.92 / MAX: 22.33 1. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl
OpenVINO Model: Road Segmentation ADAS FP16-INT8 - Device: CPU OpenBenchmarking.org FPS, More Is Better OpenVINO 2024.0 Model: Road Segmentation ADAS FP16-INT8 - Device: CPU 2 x INTEL XEON PLATINUM 8568Y+ - NVIDIA 400 800 1200 1600 2000 SE +/- 4.42, N = 3 1911.37 1. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl
OpenVINO Model: Road Segmentation ADAS FP16-INT8 - Device: CPU OpenBenchmarking.org ms, Fewer Is Better OpenVINO 2024.0 Model: Road Segmentation ADAS FP16-INT8 - Device: CPU 2 x INTEL XEON PLATINUM 8568Y+ - NVIDIA 11 22 33 44 55 SE +/- 0.11, N = 3 50.12 MIN: 41.23 / MAX: 78.02 1. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl
OpenVINO Model: Machine Translation EN To DE FP16 - Device: CPU OpenBenchmarking.org FPS, More Is Better OpenVINO 2024.0 Model: Machine Translation EN To DE FP16 - Device: CPU 2 x INTEL XEON PLATINUM 8568Y+ - NVIDIA 200 400 600 800 1000 SE +/- 1.88, N = 3 979.61 1. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl
OpenVINO Model: Machine Translation EN To DE FP16 - Device: CPU OpenBenchmarking.org ms, Fewer Is Better OpenVINO 2024.0 Model: Machine Translation EN To DE FP16 - Device: CPU 2 x INTEL XEON PLATINUM 8568Y+ - NVIDIA 6 12 18 24 30 SE +/- 0.03, N = 3 24.40 MIN: 16.77 / MAX: 198.92 1. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl
OpenVINO Model: Weld Porosity Detection FP16-INT8 - Device: CPU OpenBenchmarking.org FPS, More Is Better OpenVINO 2024.0 Model: Weld Porosity Detection FP16-INT8 - Device: CPU 2 x INTEL XEON PLATINUM 8568Y+ - NVIDIA 9K 18K 27K 36K 45K SE +/- 167.08, N = 3 41978.13 1. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl
OpenVINO Model: Weld Porosity Detection FP16-INT8 - Device: CPU OpenBenchmarking.org ms, Fewer Is Better OpenVINO 2024.0 Model: Weld Porosity Detection FP16-INT8 - Device: CPU 2 x INTEL XEON PLATINUM 8568Y+ - NVIDIA 0.4928 0.9856 1.4784 1.9712 2.464 SE +/- 0.01, N = 3 2.19 MIN: 1.83 / MAX: 29.97 1. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl
OpenVINO Model: Person Vehicle Bike Detection FP16 - Device: CPU OpenBenchmarking.org FPS, More Is Better OpenVINO 2024.0 Model: Person Vehicle Bike Detection FP16 - Device: CPU 2 x INTEL XEON PLATINUM 8568Y+ - NVIDIA 2K 4K 6K 8K 10K SE +/- 3.18, N = 3 9369.40 1. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl
OpenVINO Model: Person Vehicle Bike Detection FP16 - Device: CPU OpenBenchmarking.org ms, Fewer Is Better OpenVINO 2024.0 Model: Person Vehicle Bike Detection FP16 - Device: CPU 2 x INTEL XEON PLATINUM 8568Y+ - NVIDIA 3 6 9 12 15 SE +/- 0.00, N = 3 10.23 MIN: 9.05 / MAX: 33.92 1. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl
OpenVINO Model: Noise Suppression Poconet-Like FP16 - Device: CPU OpenBenchmarking.org FPS, More Is Better OpenVINO 2024.0 Model: Noise Suppression Poconet-Like FP16 - Device: CPU 2 x INTEL XEON PLATINUM 8568Y+ - NVIDIA 1700 3400 5100 6800 8500 SE +/- 1.82, N = 3 7708.27 1. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl
OpenVINO Model: Noise Suppression Poconet-Like FP16 - Device: CPU OpenBenchmarking.org ms, Fewer Is Better OpenVINO 2024.0 Model: Noise Suppression Poconet-Like FP16 - Device: CPU 2 x INTEL XEON PLATINUM 8568Y+ - NVIDIA 3 6 9 12 15 SE +/- 0.00, N = 3 12.33 MIN: 7.74 / MAX: 51.39 1. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl
OpenVINO Model: Handwritten English Recognition FP16 - Device: CPU OpenBenchmarking.org FPS, More Is Better OpenVINO 2024.0 Model: Handwritten English Recognition FP16 - Device: CPU 2 x INTEL XEON PLATINUM 8568Y+ - NVIDIA 1000 2000 3000 4000 5000 SE +/- 7.38, N = 3 4811.57 1. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl
OpenVINO Model: Handwritten English Recognition FP16 - Device: CPU OpenBenchmarking.org ms, Fewer Is Better OpenVINO 2024.0 Model: Handwritten English Recognition FP16 - Device: CPU 2 x INTEL XEON PLATINUM 8568Y+ - NVIDIA 5 10 15 20 25 SE +/- 0.03, N = 3 19.94 MIN: 18.35 / MAX: 41.46 1. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl
OpenVINO Model: Person Re-Identification Retail FP16 - Device: CPU OpenBenchmarking.org FPS, More Is Better OpenVINO 2024.0 Model: Person Re-Identification Retail FP16 - Device: CPU 2 x INTEL XEON PLATINUM 8568Y+ - NVIDIA 3K 6K 9K 12K 15K SE +/- 18.41, N = 3 15649.17 1. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl
OpenVINO Model: Person Re-Identification Retail FP16 - Device: CPU OpenBenchmarking.org ms, Fewer Is Better OpenVINO 2024.0 Model: Person Re-Identification Retail FP16 - Device: CPU 2 x INTEL XEON PLATINUM 8568Y+ - NVIDIA 2 4 6 8 10 SE +/- 0.01, N = 3 6.12 MIN: 5.44 / MAX: 18.13 1. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl
OpenVINO Model: Age Gender Recognition Retail 0013 FP16 - Device: CPU OpenBenchmarking.org FPS, More Is Better OpenVINO 2024.0 Model: Age Gender Recognition Retail 0013 FP16 - Device: CPU 2 x INTEL XEON PLATINUM 8568Y+ - NVIDIA 20K 40K 60K 80K 100K SE +/- 656.05, N = 3 112806.06 1. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl
OpenVINO Model: Age Gender Recognition Retail 0013 FP16 - Device: CPU OpenBenchmarking.org ms, Fewer Is Better OpenVINO 2024.0 Model: Age Gender Recognition Retail 0013 FP16 - Device: CPU 2 x INTEL XEON PLATINUM 8568Y+ - NVIDIA 0.1013 0.2026 0.3039 0.4052 0.5065 SE +/- 0.00, N = 3 0.45 MIN: 0.23 / MAX: 16.23 1. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl
OpenVINO Model: Handwritten English Recognition FP16-INT8 - Device: CPU OpenBenchmarking.org FPS, More Is Better OpenVINO 2024.0 Model: Handwritten English Recognition FP16-INT8 - Device: CPU 2 x INTEL XEON PLATINUM 8568Y+ - NVIDIA 1000 2000 3000 4000 5000 SE +/- 3.49, N = 3 4866.77 1. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl
OpenVINO Model: Handwritten English Recognition FP16-INT8 - Device: CPU OpenBenchmarking.org ms, Fewer Is Better OpenVINO 2024.0 Model: Handwritten English Recognition FP16-INT8 - Device: CPU 2 x INTEL XEON PLATINUM 8568Y+ - NVIDIA 5 10 15 20 25 SE +/- 0.01, N = 3 19.71 MIN: 18.05 / MAX: 41.25 1. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl
OpenVINO Model: Age Gender Recognition Retail 0013 FP16-INT8 - Device: CPU OpenBenchmarking.org FPS, More Is Better OpenVINO 2024.0 Model: Age Gender Recognition Retail 0013 FP16-INT8 - Device: CPU 2 x INTEL XEON PLATINUM 8568Y+ - NVIDIA 30K 60K 90K 120K 150K SE +/- 445.26, N = 3 124878.26 1. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl
OpenVINO Model: Age Gender Recognition Retail 0013 FP16-INT8 - Device: CPU OpenBenchmarking.org ms, Fewer Is Better OpenVINO 2024.0 Model: Age Gender Recognition Retail 0013 FP16-INT8 - Device: CPU 2 x INTEL XEON PLATINUM 8568Y+ - NVIDIA 0.0878 0.1756 0.2634 0.3512 0.439 SE +/- 0.00, N = 3 0.39 MIN: 0.18 / MAX: 26.55 1. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl
Phoronix Test Suite v10.8.5