altra openvino ARMv8 Neoverse-N1 testing with a System76 Thelio Astra (3.02 BIOS) and NVIDIA RTX A400/PCIe 4GB on Ubuntu 24.04 via the Phoronix Test Suite.
Compare your own system(s) to this result file with the
Phoronix Test Suite by running the command:
phoronix-test-suite benchmark 2411240-PTS-ALTRAOPE71 ARMv8 Neoverse-N1 - NVIDIA RTX A400/PCIe 4GB - Processor: ARMv8 Neoverse-N1 @ 3.00GHz (128 Cores), Motherboard: System76 Thelio Astra (3.02 BIOS), Chipset: Ampere Computing LLC Altra PCI Root Complex A, Memory: 8 x 32GB DDR4-3200MT/s Micron 18ASF4G72PDZ-3G2F1, Disk: 1024GB KINGSTON SKC3000S1024G, Graphics: NVIDIA RTX A400/PCIe 4GB, Audio: NVIDIA Device 2291, Monitor: DELL P2415Q, Network: 2 x Intel X550 + Intel I210
OS: Ubuntu 24.04, Kernel: 6.8.0-48-generic-64k (aarch64), Desktop: GNOME Shell 46.0, Display Server: X Server, Display Driver: NVIDIA 550.120, OpenGL: 4.6.0, Compiler: GCC 13.2.0, File-System: ext4, Screen Resolution: 3840x2160
Kernel Notes: Transparent Huge Pages: madviseCompiler Notes: --build=aarch64-linux-gnu --disable-libquadmath --disable-libquadmath-support --disable-werror --enable-checking=release --enable-clocale=gnu --enable-default-pie --enable-fix-cortex-a53-843419 --enable-gnu-unique-object --enable-languages=c,ada,c++,go,d,fortran,objc,obj-c++,m2 --enable-libphobos-checking=release --enable-libstdcxx-backtrace --enable-libstdcxx-debug --enable-libstdcxx-time=yes --enable-multiarch --enable-nls --enable-objc-gc=auto --enable-offload-defaulted --enable-offload-targets=nvptx-none=/build/gcc-13-dIwDw0/gcc-13-13.2.0/debian/tmp-nvptx/usr --enable-plugin --enable-shared --enable-threads=posix --host=aarch64-linux-gnu --program-prefix=aarch64-linux-gnu- --target=aarch64-linux-gnu --with-default-libstdcxx-abi=new --with-gcc-major-version-only --with-target-system-zlib=auto --without-cuda-driver -vProcessor Notes: Scaling Governor: cppc_cpufreq performance (Boost: Disabled)Python Notes: Python 3.12.3Security Notes: gather_data_sampling: Not affected + itlb_multihit: Not affected + l1tf: Not affected + mds: Not affected + meltdown: Not affected + mmio_stale_data: Not affected + reg_file_data_sampling: Not affected + retbleed: Not affected + spec_rstack_overflow: Not affected + spec_store_bypass: Mitigation of SSB disabled via prctl + spectre_v1: Mitigation of __user pointer sanitization + spectre_v2: Mitigation of CSV2 BHB + srbds: Not affected + tsx_async_abort: Not affected
OpenVINO GenAI OpenBenchmarking.org ms, Fewer Is Better OpenVINO GenAI 2024.5 Model: Gemma-7b-int4-ov - Device: CPU - Time Per Output Token ARMv8 Neoverse-N1 - NVIDIA RTX A400/PCIe 4GB - 40 80 120 160 200 SE +/- 1.21, N = 3 166.58
OpenBenchmarking.org ms, Fewer Is Better OpenVINO GenAI 2024.5 Model: TinyLlama-1.1B-Chat-v1.0 - Device: CPU - Time Per Output Token ARMv8 Neoverse-N1 - NVIDIA RTX A400/PCIe 4GB - 11 22 33 44 55 SE +/- 0.67, N = 15 50.72
OpenBenchmarking.org ms, Fewer Is Better OpenVINO GenAI 2024.5 Model: TinyLlama-1.1B-Chat-v1.0 - Device: CPU - Time To First Token ARMv8 Neoverse-N1 - NVIDIA RTX A400/PCIe 4GB - 16 32 48 64 80 SE +/- 1.17, N = 15 72.65
OpenVINO OpenBenchmarking.org ms, Fewer Is Better OpenVINO 2024.5 Model: Face Detection Retail FP16 - Device: CPU ARMv8 Neoverse-N1 - NVIDIA RTX A400/PCIe 4GB - 7 14 21 28 35 SE +/- 0.68, N = 15 27.94 MIN: 6.03 / MAX: 100.21 1. (CXX) g++ options: -isystem -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -std=c++14 -fPIC -fvisibility=hidden -MD -MT -MF
OpenBenchmarking.org FPS, More Is Better OpenVINO 2024.5 Model: Face Detection Retail FP16 - Device: CPU ARMv8 Neoverse-N1 - NVIDIA RTX A400/PCIe 4GB - 200 400 600 800 1000 SE +/- 25.65, N = 15 1153.54 1. (CXX) g++ options: -isystem -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -std=c++14 -fPIC -fvisibility=hidden -MD -MT -MF
OpenVINO GenAI OpenBenchmarking.org ms, Fewer Is Better OpenVINO GenAI 2024.5 Model: Falcon-7b-instruct-int4-ov - Device: CPU - Time Per Output Token ARMv8 Neoverse-N1 - NVIDIA RTX A400/PCIe 4GB - 20 40 60 80 100 SE +/- 0.29, N = 3 101.57
OpenBenchmarking.org ms, Fewer Is Better OpenVINO GenAI 2024.5 Model: Falcon-7b-instruct-int4-ov - Device: CPU - Time To First Token ARMv8 Neoverse-N1 - NVIDIA RTX A400/PCIe 4GB - 30 60 90 120 150 SE +/- 0.60, N = 3 121.94
OpenBenchmarking.org ms, Fewer Is Better OpenVINO GenAI 2024.5 Model: Phi-3-mini-128k-instruct-int4-ov - Device: CPU - Time Per Output Token ARMv8 Neoverse-N1 - NVIDIA RTX A400/PCIe 4GB - 20 40 60 80 100 SE +/- 0.39, N = 3 99.0
OpenBenchmarking.org ms, Fewer Is Better OpenVINO GenAI 2024.5 Model: Phi-3-mini-128k-instruct-int4-ov - Device: CPU - Time To First Token ARMv8 Neoverse-N1 - NVIDIA RTX A400/PCIe 4GB - 30 60 90 120 150 SE +/- 0.24, N = 3 125.29
OpenVINO OpenBenchmarking.org ms, Fewer Is Better OpenVINO 2024.5 Model: Vehicle Detection FP16 - Device: CPU ARMv8 Neoverse-N1 - NVIDIA RTX A400/PCIe 4GB - 13 26 39 52 65 SE +/- 0.60, N = 5 60.15 MIN: 16.01 / MAX: 147.7 1. (CXX) g++ options: -isystem -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -std=c++14 -fPIC -fvisibility=hidden -MD -MT -MF
OpenBenchmarking.org FPS, More Is Better OpenVINO 2024.5 Model: Vehicle Detection FP16 - Device: CPU ARMv8 Neoverse-N1 - NVIDIA RTX A400/PCIe 4GB - 110 220 330 440 550 SE +/- 5.23, N = 5 531.69 1. (CXX) g++ options: -isystem -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -std=c++14 -fPIC -fvisibility=hidden -MD -MT -MF
OpenBenchmarking.org ms, Fewer Is Better OpenVINO 2024.5 Model: Face Detection FP16-INT8 - Device: CPU ARMv8 Neoverse-N1 - NVIDIA RTX A400/PCIe 4GB - 1000 2000 3000 4000 5000 SE +/- 2.68, N = 3 4789.33 MIN: 3410.69 / MAX: 13276.72 1. (CXX) g++ options: -isystem -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -std=c++14 -fPIC -fvisibility=hidden -MD -MT -MF
OpenBenchmarking.org FPS, More Is Better OpenVINO 2024.5 Model: Face Detection FP16-INT8 - Device: CPU ARMv8 Neoverse-N1 - NVIDIA RTX A400/PCIe 4GB - 2 4 6 8 10 SE +/- 0.00, N = 3 6.53 1. (CXX) g++ options: -isystem -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -std=c++14 -fPIC -fvisibility=hidden -MD -MT -MF
OpenBenchmarking.org ms, Fewer Is Better OpenVINO 2024.5 Model: Face Detection FP16 - Device: CPU ARMv8 Neoverse-N1 - NVIDIA RTX A400/PCIe 4GB - 1000 2000 3000 4000 5000 SE +/- 2.59, N = 3 4566.75 MIN: 2624.73 / MAX: 12337.22 1. (CXX) g++ options: -isystem -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -std=c++14 -fPIC -fvisibility=hidden -MD -MT -MF
OpenBenchmarking.org FPS, More Is Better OpenVINO 2024.5 Model: Face Detection FP16 - Device: CPU ARMv8 Neoverse-N1 - NVIDIA RTX A400/PCIe 4GB - 2 4 6 8 10 SE +/- 0.01, N = 3 6.90 1. (CXX) g++ options: -isystem -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -std=c++14 -fPIC -fvisibility=hidden -MD -MT -MF
OpenBenchmarking.org ms, Fewer Is Better OpenVINO 2024.5 Model: Road Segmentation ADAS FP16-INT8 - Device: CPU ARMv8 Neoverse-N1 - NVIDIA RTX A400/PCIe 4GB - 200 400 600 800 1000 SE +/- 0.56, N = 3 1024.28 MIN: 720.37 / MAX: 1124.87 1. (CXX) g++ options: -isystem -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -std=c++14 -fPIC -fvisibility=hidden -MD -MT -MF
OpenBenchmarking.org FPS, More Is Better OpenVINO 2024.5 Model: Road Segmentation ADAS FP16-INT8 - Device: CPU ARMv8 Neoverse-N1 - NVIDIA RTX A400/PCIe 4GB - 7 14 21 28 35 SE +/- 0.01, N = 3 31.04 1. (CXX) g++ options: -isystem -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -std=c++14 -fPIC -fvisibility=hidden -MD -MT -MF
OpenBenchmarking.org ms, Fewer Is Better OpenVINO 2024.5 Model: Person Detection FP16 - Device: CPU ARMv8 Neoverse-N1 - NVIDIA RTX A400/PCIe 4GB - 200 400 600 800 1000 SE +/- 0.52, N = 3 937.64 MIN: 472.78 / MAX: 1164.55 1. (CXX) g++ options: -isystem -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -std=c++14 -fPIC -fvisibility=hidden -MD -MT -MF
OpenBenchmarking.org FPS, More Is Better OpenVINO 2024.5 Model: Person Detection FP16 - Device: CPU ARMv8 Neoverse-N1 - NVIDIA RTX A400/PCIe 4GB - 8 16 24 32 40 SE +/- 0.02, N = 3 34.02 1. (CXX) g++ options: -isystem -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -std=c++14 -fPIC -fvisibility=hidden -MD -MT -MF
OpenBenchmarking.org ms, Fewer Is Better OpenVINO 2024.5 Model: Person Detection FP32 - Device: CPU ARMv8 Neoverse-N1 - NVIDIA RTX A400/PCIe 4GB - 200 400 600 800 1000 SE +/- 0.26, N = 3 937.25 MIN: 366.98 / MAX: 1169.47 1. (CXX) g++ options: -isystem -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -std=c++14 -fPIC -fvisibility=hidden -MD -MT -MF
OpenBenchmarking.org FPS, More Is Better OpenVINO 2024.5 Model: Person Detection FP32 - Device: CPU ARMv8 Neoverse-N1 - NVIDIA RTX A400/PCIe 4GB - 8 16 24 32 40 SE +/- 0.01, N = 3 34.03 1. (CXX) g++ options: -isystem -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -std=c++14 -fPIC -fvisibility=hidden -MD -MT -MF
OpenBenchmarking.org ms, Fewer Is Better OpenVINO 2024.5 Model: Machine Translation EN To DE FP16 - Device: CPU ARMv8 Neoverse-N1 - NVIDIA RTX A400/PCIe 4GB - 20 40 60 80 100 SE +/- 0.37, N = 3 106.44 MIN: 96.69 / MAX: 176.18 1. (CXX) g++ options: -isystem -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -std=c++14 -fPIC -fvisibility=hidden -MD -MT -MF
OpenBenchmarking.org FPS, More Is Better OpenVINO 2024.5 Model: Machine Translation EN To DE FP16 - Device: CPU ARMv8 Neoverse-N1 - NVIDIA RTX A400/PCIe 4GB - 70 140 210 280 350 SE +/- 1.00, N = 3 300.36 1. (CXX) g++ options: -isystem -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -std=c++14 -fPIC -fvisibility=hidden -MD -MT -MF
OpenBenchmarking.org ms, Fewer Is Better OpenVINO 2024.5 Model: Noise Suppression Poconet-Like FP16 - Device: CPU ARMv8 Neoverse-N1 - NVIDIA RTX A400/PCIe 4GB - 80 160 240 320 400 SE +/- 0.91, N = 3 352.06 MIN: 97.37 / MAX: 723.01 1. (CXX) g++ options: -isystem -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -std=c++14 -fPIC -fvisibility=hidden -MD -MT -MF
OpenBenchmarking.org FPS, More Is Better OpenVINO 2024.5 Model: Noise Suppression Poconet-Like FP16 - Device: CPU ARMv8 Neoverse-N1 - NVIDIA RTX A400/PCIe 4GB - 20 40 60 80 100 SE +/- 0.23, N = 3 90.76 1. (CXX) g++ options: -isystem -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -std=c++14 -fPIC -fvisibility=hidden -MD -MT -MF
OpenBenchmarking.org ms, Fewer Is Better OpenVINO 2024.5 Model: Person Vehicle Bike Detection FP16 - Device: CPU ARMv8 Neoverse-N1 - NVIDIA RTX A400/PCIe 4GB - 20 40 60 80 100 SE +/- 0.38, N = 3 108.51 MIN: 23.78 / MAX: 188.92 1. (CXX) g++ options: -isystem -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -std=c++14 -fPIC -fvisibility=hidden -MD -MT -MF
OpenBenchmarking.org FPS, More Is Better OpenVINO 2024.5 Model: Person Vehicle Bike Detection FP16 - Device: CPU ARMv8 Neoverse-N1 - NVIDIA RTX A400/PCIe 4GB - 60 120 180 240 300 SE +/- 1.02, N = 3 294.75 1. (CXX) g++ options: -isystem -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -std=c++14 -fPIC -fvisibility=hidden -MD -MT -MF
OpenBenchmarking.org ms, Fewer Is Better OpenVINO 2024.5 Model: Vehicle Detection FP16-INT8 - Device: CPU ARMv8 Neoverse-N1 - NVIDIA RTX A400/PCIe 4GB - 80 160 240 320 400 SE +/- 1.26, N = 3 370.01 MIN: 247.37 / MAX: 435.92 1. (CXX) g++ options: -isystem -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -std=c++14 -fPIC -fvisibility=hidden -MD -MT -MF
OpenBenchmarking.org FPS, More Is Better OpenVINO 2024.5 Model: Vehicle Detection FP16-INT8 - Device: CPU ARMv8 Neoverse-N1 - NVIDIA RTX A400/PCIe 4GB - 20 40 60 80 100 SE +/- 0.30, N = 3 86.31 1. (CXX) g++ options: -isystem -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -std=c++14 -fPIC -fvisibility=hidden -MD -MT -MF
OpenBenchmarking.org ms, Fewer Is Better OpenVINO 2024.5 Model: Face Detection Retail FP16-INT8 - Device: CPU ARMv8 Neoverse-N1 - NVIDIA RTX A400/PCIe 4GB - 20 40 60 80 100 SE +/- 0.12, N = 3 81.47 MIN: 63.53 / MAX: 114.03 1. (CXX) g++ options: -isystem -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -std=c++14 -fPIC -fvisibility=hidden -MD -MT -MF
OpenBenchmarking.org FPS, More Is Better OpenVINO 2024.5 Model: Face Detection Retail FP16-INT8 - Device: CPU ARMv8 Neoverse-N1 - NVIDIA RTX A400/PCIe 4GB - 90 180 270 360 450 SE +/- 0.57, N = 3 392.26 1. (CXX) g++ options: -isystem -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -std=c++14 -fPIC -fvisibility=hidden -MD -MT -MF
OpenBenchmarking.org ms, Fewer Is Better OpenVINO 2024.5 Model: Road Segmentation ADAS FP16 - Device: CPU ARMv8 Neoverse-N1 - NVIDIA RTX A400/PCIe 4GB - 30 60 90 120 150 SE +/- 0.07, N = 3 137.21 MIN: 64.84 / MAX: 275.79 1. (CXX) g++ options: -isystem -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -std=c++14 -fPIC -fvisibility=hidden -MD -MT -MF
OpenBenchmarking.org FPS, More Is Better OpenVINO 2024.5 Model: Road Segmentation ADAS FP16 - Device: CPU ARMv8 Neoverse-N1 - NVIDIA RTX A400/PCIe 4GB - 50 100 150 200 250 SE +/- 0.12, N = 3 232.83 1. (CXX) g++ options: -isystem -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -std=c++14 -fPIC -fvisibility=hidden -MD -MT -MF
OpenBenchmarking.org ms, Fewer Is Better OpenVINO 2024.5 Model: Weld Porosity Detection FP16-INT8 - Device: CPU ARMv8 Neoverse-N1 - NVIDIA RTX A400/PCIe 4GB - 14 28 42 56 70 SE +/- 0.02, N = 3 61.36 MIN: 26.53 / MAX: 1259.53 1. (CXX) g++ options: -isystem -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -std=c++14 -fPIC -fvisibility=hidden -MD -MT -MF
OpenBenchmarking.org FPS, More Is Better OpenVINO 2024.5 Model: Weld Porosity Detection FP16-INT8 - Device: CPU ARMv8 Neoverse-N1 - NVIDIA RTX A400/PCIe 4GB - 110 220 330 440 550 SE +/- 0.18, N = 3 521.12 1. (CXX) g++ options: -isystem -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -std=c++14 -fPIC -fvisibility=hidden -MD -MT -MF
OpenBenchmarking.org ms, Fewer Is Better OpenVINO 2024.5 Model: Person Re-Identification Retail FP16 - Device: CPU ARMv8 Neoverse-N1 - NVIDIA RTX A400/PCIe 4GB - 40 80 120 160 200 SE +/- 0.61, N = 3 178.76 MIN: 16.36 / MAX: 249.37 1. (CXX) g++ options: -isystem -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -std=c++14 -fPIC -fvisibility=hidden -MD -MT -MF
OpenBenchmarking.org FPS, More Is Better OpenVINO 2024.5 Model: Person Re-Identification Retail FP16 - Device: CPU ARMv8 Neoverse-N1 - NVIDIA RTX A400/PCIe 4GB - 40 80 120 160 200 SE +/- 0.62, N = 3 178.91 1. (CXX) g++ options: -isystem -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -std=c++14 -fPIC -fvisibility=hidden -MD -MT -MF
OpenBenchmarking.org ms, Fewer Is Better OpenVINO 2024.5 Model: Handwritten English Recognition FP16-INT8 - Device: CPU ARMv8 Neoverse-N1 - NVIDIA RTX A400/PCIe 4GB - 50 100 150 200 250 SE +/- 2.74, N = 3 214.64 MIN: 206.81 / MAX: 304.42 1. (CXX) g++ options: -isystem -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -std=c++14 -fPIC -fvisibility=hidden -MD -MT -MF
OpenBenchmarking.org FPS, More Is Better OpenVINO 2024.5 Model: Handwritten English Recognition FP16-INT8 - Device: CPU ARMv8 Neoverse-N1 - NVIDIA RTX A400/PCIe 4GB - 30 60 90 120 150 SE +/- 1.86, N = 3 148.84 1. (CXX) g++ options: -isystem -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -std=c++14 -fPIC -fvisibility=hidden -MD -MT -MF
OpenBenchmarking.org ms, Fewer Is Better OpenVINO 2024.5 Model: Handwritten English Recognition FP16 - Device: CPU ARMv8 Neoverse-N1 - NVIDIA RTX A400/PCIe 4GB - 40 80 120 160 200 SE +/- 0.12, N = 3 184.02 MIN: 179.6 / MAX: 276.19 1. (CXX) g++ options: -isystem -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -std=c++14 -fPIC -fvisibility=hidden -MD -MT -MF
OpenBenchmarking.org FPS, More Is Better OpenVINO 2024.5 Model: Handwritten English Recognition FP16 - Device: CPU ARMv8 Neoverse-N1 - NVIDIA RTX A400/PCIe 4GB - 40 80 120 160 200 SE +/- 0.09, N = 3 173.59 1. (CXX) g++ options: -isystem -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -std=c++14 -fPIC -fvisibility=hidden -MD -MT -MF
OpenBenchmarking.org ms, Fewer Is Better OpenVINO 2024.5 Model: Weld Porosity Detection FP16 - Device: CPU ARMv8 Neoverse-N1 - NVIDIA RTX A400/PCIe 4GB - 14 28 42 56 70 SE +/- 0.43, N = 3 64.03 MIN: 8.55 / MAX: 1282.19 1. (CXX) g++ options: -isystem -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -std=c++14 -fPIC -fvisibility=hidden -MD -MT -MF
OpenBenchmarking.org FPS, More Is Better OpenVINO 2024.5 Model: Weld Porosity Detection FP16 - Device: CPU ARMv8 Neoverse-N1 - NVIDIA RTX A400/PCIe 4GB - 110 220 330 440 550 SE +/- 3.35, N = 3 499.46 1. (CXX) g++ options: -isystem -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -std=c++14 -fPIC -fvisibility=hidden -MD -MT -MF
OpenBenchmarking.org ms, Fewer Is Better OpenVINO 2024.5 Model: Age Gender Recognition Retail 0013 FP16-INT8 - Device: CPU ARMv8 Neoverse-N1 - NVIDIA RTX A400/PCIe 4GB - 4 8 12 16 20 SE +/- 0.23, N = 3 17.01 MIN: 1.5 / MAX: 294.6 1. (CXX) g++ options: -isystem -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -std=c++14 -fPIC -fvisibility=hidden -MD -MT -MF
OpenBenchmarking.org FPS, More Is Better OpenVINO 2024.5 Model: Age Gender Recognition Retail 0013 FP16-INT8 - Device: CPU ARMv8 Neoverse-N1 - NVIDIA RTX A400/PCIe 4GB - 400 800 1200 1600 2000 SE +/- 25.61, N = 3 1880.88 1. (CXX) g++ options: -isystem -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -std=c++14 -fPIC -fvisibility=hidden -MD -MT -MF
OpenBenchmarking.org ms, Fewer Is Better OpenVINO 2024.5 Model: Age Gender Recognition Retail 0013 FP16 - Device: CPU ARMv8 Neoverse-N1 - NVIDIA RTX A400/PCIe 4GB - 4 8 12 16 20 SE +/- 0.23, N = 3 17.55 MIN: 0.91 / MAX: 295.1 1. (CXX) g++ options: -isystem -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -std=c++14 -fPIC -fvisibility=hidden -MD -MT -MF
OpenBenchmarking.org FPS, More Is Better OpenVINO 2024.5 Model: Age Gender Recognition Retail 0013 FP16 - Device: CPU ARMv8 Neoverse-N1 - NVIDIA RTX A400/PCIe 4GB - 400 800 1200 1600 2000 SE +/- 24.18, N = 3 1822.54 1. (CXX) g++ options: -isystem -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -std=c++14 -fPIC -fvisibility=hidden -MD -MT -MF
ARMv8 Neoverse-N1 - NVIDIA RTX A400/PCIe 4GB - Processor: ARMv8 Neoverse-N1 @ 3.00GHz (128 Cores), Motherboard: System76 Thelio Astra (3.02 BIOS), Chipset: Ampere Computing LLC Altra PCI Root Complex A, Memory: 8 x 32GB DDR4-3200MT/s Micron 18ASF4G72PDZ-3G2F1, Disk: 1024GB KINGSTON SKC3000S1024G, Graphics: NVIDIA RTX A400/PCIe 4GB, Audio: NVIDIA Device 2291, Monitor: DELL P2415Q, Network: 2 x Intel X550 + Intel I210
OS: Ubuntu 24.04, Kernel: 6.8.0-48-generic-64k (aarch64), Desktop: GNOME Shell 46.0, Display Server: X Server, Display Driver: NVIDIA 550.120, OpenGL: 4.6.0, Compiler: GCC 13.2.0, File-System: ext4, Screen Resolution: 3840x2160
Kernel Notes: Transparent Huge Pages: madviseCompiler Notes: --build=aarch64-linux-gnu --disable-libquadmath --disable-libquadmath-support --disable-werror --enable-checking=release --enable-clocale=gnu --enable-default-pie --enable-fix-cortex-a53-843419 --enable-gnu-unique-object --enable-languages=c,ada,c++,go,d,fortran,objc,obj-c++,m2 --enable-libphobos-checking=release --enable-libstdcxx-backtrace --enable-libstdcxx-debug --enable-libstdcxx-time=yes --enable-multiarch --enable-nls --enable-objc-gc=auto --enable-offload-defaulted --enable-offload-targets=nvptx-none=/build/gcc-13-dIwDw0/gcc-13-13.2.0/debian/tmp-nvptx/usr --enable-plugin --enable-shared --enable-threads=posix --host=aarch64-linux-gnu --program-prefix=aarch64-linux-gnu- --target=aarch64-linux-gnu --with-default-libstdcxx-abi=new --with-gcc-major-version-only --with-target-system-zlib=auto --without-cuda-driver -vProcessor Notes: Scaling Governor: cppc_cpufreq performance (Boost: Disabled)Python Notes: Python 3.12.3Security Notes: gather_data_sampling: Not affected + itlb_multihit: Not affected + l1tf: Not affected + mds: Not affected + meltdown: Not affected + mmio_stale_data: Not affected + reg_file_data_sampling: Not affected + retbleed: Not affected + spec_rstack_overflow: Not affected + spec_store_bypass: Mitigation of SSB disabled via prctl + spectre_v1: Mitigation of __user pointer sanitization + spectre_v2: Mitigation of CSV2 BHB + srbds: Not affected + tsx_async_abort: Not affected
Testing initiated at 24 November 2024 22:24 by user pts.