2 x Intel Xeon Max 9480 testing with a Supermicro X13DEM v1.10 (1.3 BIOS) and ASPEED on CentOS Stream 9 via the Phoronix Test Suite.
2 x Intel Xeon Max 9480 Kernel Notes: Transparent Huge Pages: alwaysCompiler Notes: --build=x86_64-redhat-linux --disable-libunwind-exceptions --enable-__cxa_atexit --enable-bootstrap --enable-cet --enable-checking=release --enable-gnu-indirect-function --enable-gnu-unique-object --enable-host-bind-now --enable-host-pie --enable-initfini-array --enable-languages=c,c++,fortran,lto --enable-link-serialization=1 --enable-multilib --enable-offload-targets=nvptx-none --enable-plugin --enable-shared --enable-threads=posix --mandir=/usr/share/man --with-arch_32=x86-64 --with-arch_64=x86-64-v2 --with-build-config=bootstrap-lto --with-gcc-major-version-only --with-linker-hash-style=gnu --with-tune=generic --without-cuda-driver --without-islProcessor Notes: Scaling Governor: intel_cpufreq performance - CPU Microcode: 0x2c000271Python Notes: Python 3.9.17Security Notes: SELinux + gather_data_sampling: Not affected + itlb_multihit: Not affected + l1tf: Not affected + mds: Not affected + meltdown: Not affected + mmio_stale_data: Not affected + retbleed: Not affected + spec_store_bypass: Mitigation of SSB disabled via prctl + spectre_v1: Mitigation of usercopy/swapgs barriers and __user pointer sanitization + spectre_v2: Mitigation of Enhanced / Automatic IBRS IBPB: conditional RSB filling PBRSB-eIBRS: SW sequence + srbds: Not affected + tsx_async_abort: Not affected
b c Processor: 2 x Intel Xeon Max 9480 @ 3.50GHz (112 Cores / 224 Threads), Motherboard: Supermicro X13DEM v1.10 (1.3 BIOS), Chipset: Intel Device 1bce, Memory: 512GB, Disk: 2 x 7682GB INTEL SSDPF2KX076TZ, Graphics: ASPEED, Monitor: VE228, Network: 2 x Broadcom BCM57508 NetXtreme-E 10Gb/25Gb/40Gb/50Gb/100Gb/200Gb
OS: CentOS Stream 9, Kernel: 5.14.0-373.el9.x86_64 (x86_64), Desktop: GNOME Shell 40.10, Display Server: X Server, Compiler: GCC 11.4.1 20230605, File-System: xfs, Screen Resolution: 1920x1080
Embree Binary: Pathtracer - Model: Crown
2 x Intel Xeon Max 9480: The test quit with a non-zero exit status. E: ./bin/embree_pathtracer: error while loading shared libraries: libOpenGL.so.0: cannot open shared object file: No such file or directory
b: The test quit with a non-zero exit status. E: ./bin/embree_pathtracer: error while loading shared libraries: libOpenGL.so.0: cannot open shared object file: No such file or directory
c: The test quit with a non-zero exit status. E: ./bin/embree_pathtracer: error while loading shared libraries: libOpenGL.so.0: cannot open shared object file: No such file or directory
Binary: Pathtracer ISPC - Model: Crown
2 x Intel Xeon Max 9480: The test quit with a non-zero exit status. E: ./bin/embree_pathtracer_ispc: error while loading shared libraries: libOpenGL.so.0: cannot open shared object file: No such file or directory
b: The test quit with a non-zero exit status. E: ./bin/embree_pathtracer_ispc: error while loading shared libraries: libOpenGL.so.0: cannot open shared object file: No such file or directory
c: The test quit with a non-zero exit status. E: ./bin/embree_pathtracer_ispc: error while loading shared libraries: libOpenGL.so.0: cannot open shared object file: No such file or directory
Binary: Pathtracer - Model: Asian Dragon
2 x Intel Xeon Max 9480: The test quit with a non-zero exit status. E: ./bin/embree_pathtracer: error while loading shared libraries: libOpenGL.so.0: cannot open shared object file: No such file or directory
b: The test quit with a non-zero exit status. E: ./bin/embree_pathtracer: error while loading shared libraries: libOpenGL.so.0: cannot open shared object file: No such file or directory
c: The test quit with a non-zero exit status. E: ./bin/embree_pathtracer: error while loading shared libraries: libOpenGL.so.0: cannot open shared object file: No such file or directory
Binary: Pathtracer - Model: Asian Dragon Obj
2 x Intel Xeon Max 9480: The test quit with a non-zero exit status. E: ./bin/embree_pathtracer: error while loading shared libraries: libOpenGL.so.0: cannot open shared object file: No such file or directory
b: The test quit with a non-zero exit status. E: ./bin/embree_pathtracer: error while loading shared libraries: libOpenGL.so.0: cannot open shared object file: No such file or directory
c: The test quit with a non-zero exit status. E: ./bin/embree_pathtracer: error while loading shared libraries: libOpenGL.so.0: cannot open shared object file: No such file or directory
Binary: Pathtracer ISPC - Model: Asian Dragon
2 x Intel Xeon Max 9480: The test quit with a non-zero exit status. E: ./bin/embree_pathtracer_ispc: error while loading shared libraries: libOpenGL.so.0: cannot open shared object file: No such file or directory
b: The test quit with a non-zero exit status. E: ./bin/embree_pathtracer_ispc: error while loading shared libraries: libOpenGL.so.0: cannot open shared object file: No such file or directory
c: The test quit with a non-zero exit status. E: ./bin/embree_pathtracer_ispc: error while loading shared libraries: libOpenGL.so.0: cannot open shared object file: No such file or directory
Binary: Pathtracer ISPC - Model: Asian Dragon Obj
2 x Intel Xeon Max 9480: The test quit with a non-zero exit status. E: ./bin/embree_pathtracer_ispc: error while loading shared libraries: libOpenGL.so.0: cannot open shared object file: No such file or directory
b: The test quit with a non-zero exit status. E: ./bin/embree_pathtracer_ispc: error while loading shared libraries: libOpenGL.so.0: cannot open shared object file: No such file or directory
c: The test quit with a non-zero exit status. E: ./bin/embree_pathtracer_ispc: error while loading shared libraries: libOpenGL.so.0: cannot open shared object file: No such file or directory
OpenBenchmarking.org Items / Sec, More Is Better OpenVKL 2.0.0 Benchmark: vklBenchmarkCPU Scalar 2 x Intel Xeon Max 9480 b c 110 220 330 440 550 484 516 488 MIN: 44 / MAX: 7212 MIN: 48 / MAX: 7431 MIN: 49 / MAX: 7159
oneDNN This is a test of the Intel oneDNN as an Intel-optimized library for Deep Neural Networks and making use of its built-in benchdnn functionality. The result is the total perf time reported. Intel oneDNN was formerly known as DNNL (Deep Neural Network Library) and MKL-DNN before being rebranded as part of the Intel oneAPI toolkit. Learn more via the OpenBenchmarking.org test page.
OpenBenchmarking.org ms, Fewer Is Better oneDNN 3.3 Harness: IP Shapes 1D - Data Type: f32 - Engine: CPU 2 x Intel Xeon Max 9480 b c 2 4 6 8 10 6.39867 3.44345 5.42410 MIN: 5.47 MIN: 2.76 MIN: 4.54 1. (CXX) g++ options: -O3 -march=native -fopenmp -msse4.1 -fPIC -pie -ldl -lpthread
OpenBenchmarking.org ms, Fewer Is Better oneDNN 3.3 Harness: IP Shapes 3D - Data Type: f32 - Engine: CPU 2 x Intel Xeon Max 9480 b c 2 4 6 8 10 8.18813 7.48884 7.91051 MIN: 6.83 MIN: 6.39 MIN: 6.61 1. (CXX) g++ options: -O3 -march=native -fopenmp -msse4.1 -fPIC -pie -ldl -lpthread
OpenBenchmarking.org ms, Fewer Is Better oneDNN 3.3 Harness: IP Shapes 1D - Data Type: u8s8f32 - Engine: CPU 2 x Intel Xeon Max 9480 b c 2 4 6 8 10 6.90661 6.50478 3.76885 MIN: 5.54 MIN: 5.31 MIN: 2.93 1. (CXX) g++ options: -O3 -march=native -fopenmp -msse4.1 -fPIC -pie -ldl -lpthread
OpenBenchmarking.org ms, Fewer Is Better oneDNN 3.3 Harness: IP Shapes 3D - Data Type: u8s8f32 - Engine: CPU 2 x Intel Xeon Max 9480 b c 1.1941 2.3882 3.5823 4.7764 5.9705 5.24744 5.30718 4.22247 MIN: 3.67 MIN: 3.81 MIN: 2.99 1. (CXX) g++ options: -O3 -march=native -fopenmp -msse4.1 -fPIC -pie -ldl -lpthread
OpenBenchmarking.org ms, Fewer Is Better oneDNN 3.3 Harness: IP Shapes 1D - Data Type: bf16bf16bf16 - Engine: CPU 2 x Intel Xeon Max 9480 b c 2 4 6 8 10 8.02865 8.37883 8.47846 MIN: 6.31 MIN: 6.6 MIN: 6.67 1. (CXX) g++ options: -O3 -march=native -fopenmp -msse4.1 -fPIC -pie -ldl -lpthread
OpenBenchmarking.org ms, Fewer Is Better oneDNN 3.3 Harness: IP Shapes 3D - Data Type: bf16bf16bf16 - Engine: CPU 2 x Intel Xeon Max 9480 b c 20 40 60 80 100 80.95 84.30 81.85 MIN: 70.18 MIN: 73.12 MIN: 70.51 1. (CXX) g++ options: -O3 -march=native -fopenmp -msse4.1 -fPIC -pie -ldl -lpthread
OpenBenchmarking.org ms, Fewer Is Better oneDNN 3.3 Harness: Convolution Batch Shapes Auto - Data Type: f32 - Engine: CPU 2 x Intel Xeon Max 9480 b c 3 6 9 12 15 9.07183 8.85383 9.18192 MIN: 7.95 MIN: 7.98 MIN: 7.96 1. (CXX) g++ options: -O3 -march=native -fopenmp -msse4.1 -fPIC -pie -ldl -lpthread
OpenBenchmarking.org ms, Fewer Is Better oneDNN 3.3 Harness: Deconvolution Batch shapes_1d - Data Type: f32 - Engine: CPU 2 x Intel Xeon Max 9480 b c 4 8 12 16 20 14.81 14.75 14.21 MIN: 12.76 MIN: 12.76 MIN: 12.52 1. (CXX) g++ options: -O3 -march=native -fopenmp -msse4.1 -fPIC -pie -ldl -lpthread
OpenBenchmarking.org ms, Fewer Is Better oneDNN 3.3 Harness: Deconvolution Batch shapes_3d - Data Type: f32 - Engine: CPU 2 x Intel Xeon Max 9480 b c 0.9617 1.9234 2.8851 3.8468 4.8085 4.27423 4.06129 3.93714 MIN: 2.85 MIN: 2.94 MIN: 2.83 1. (CXX) g++ options: -O3 -march=native -fopenmp -msse4.1 -fPIC -pie -ldl -lpthread
OpenBenchmarking.org ms, Fewer Is Better oneDNN 3.3 Harness: Convolution Batch Shapes Auto - Data Type: u8s8f32 - Engine: CPU 2 x Intel Xeon Max 9480 b c 2 4 6 8 10 7.11116 7.01769 6.90576 MIN: 6.28 MIN: 6.23 MIN: 6.2 1. (CXX) g++ options: -O3 -march=native -fopenmp -msse4.1 -fPIC -pie -ldl -lpthread
OpenBenchmarking.org ms, Fewer Is Better oneDNN 3.3 Harness: Deconvolution Batch shapes_1d - Data Type: u8s8f32 - Engine: CPU 2 x Intel Xeon Max 9480 b c 0.1371 0.2742 0.4113 0.5484 0.6855 0.609168 0.570467 0.595536 MIN: 0.46 MIN: 0.45 MIN: 0.46 1. (CXX) g++ options: -O3 -march=native -fopenmp -msse4.1 -fPIC -pie -ldl -lpthread
OpenBenchmarking.org ms, Fewer Is Better oneDNN 3.3 Harness: Deconvolution Batch shapes_3d - Data Type: u8s8f32 - Engine: CPU 2 x Intel Xeon Max 9480 b c 0.2016 0.4032 0.6048 0.8064 1.008 0.816029 0.895853 0.814304 MIN: 0.63 MIN: 0.65 MIN: 0.63 1. (CXX) g++ options: -O3 -march=native -fopenmp -msse4.1 -fPIC -pie -ldl -lpthread
OpenBenchmarking.org ms, Fewer Is Better oneDNN 3.3 Harness: Recurrent Neural Network Training - Data Type: f32 - Engine: CPU 2 x Intel Xeon Max 9480 b c 3K 6K 9K 12K 15K 13799.3 8774.0 14073.0 MIN: 12882.7 MIN: 8085.71 MIN: 12966.8 1. (CXX) g++ options: -O3 -march=native -fopenmp -msse4.1 -fPIC -pie -ldl -lpthread
OpenBenchmarking.org ms, Fewer Is Better oneDNN 3.3 Harness: Recurrent Neural Network Inference - Data Type: f32 - Engine: CPU 2 x Intel Xeon Max 9480 b c 300 600 900 1200 1500 1389.60 1415.42 1303.30 MIN: 1235.38 MIN: 1317.1 MIN: 1163.26 1. (CXX) g++ options: -O3 -march=native -fopenmp -msse4.1 -fPIC -pie -ldl -lpthread
OpenBenchmarking.org ms, Fewer Is Better oneDNN 3.3 Harness: Recurrent Neural Network Training - Data Type: u8s8f32 - Engine: CPU 2 x Intel Xeon Max 9480 b c 4K 8K 12K 16K 20K 16878.5 14404.0 15467.0 MIN: 15171.7 MIN: 13476.1 MIN: 13719.2 1. (CXX) g++ options: -O3 -march=native -fopenmp -msse4.1 -fPIC -pie -ldl -lpthread
OpenBenchmarking.org ms, Fewer Is Better oneDNN 3.3 Harness: Convolution Batch Shapes Auto - Data Type: bf16bf16bf16 - Engine: CPU 2 x Intel Xeon Max 9480 b c 0.8863 1.7726 2.6589 3.5452 4.4315 3.88111 3.85134 3.93910 MIN: 3.25 MIN: 3.36 MIN: 3.25 1. (CXX) g++ options: -O3 -march=native -fopenmp -msse4.1 -fPIC -pie -ldl -lpthread
OpenBenchmarking.org ms, Fewer Is Better oneDNN 3.3 Harness: Deconvolution Batch shapes_1d - Data Type: bf16bf16bf16 - Engine: CPU 2 x Intel Xeon Max 9480 b c 0.1501 0.3002 0.4503 0.6004 0.7505 0.647092 0.667091 0.605543 MIN: 0.51 MIN: 0.51 MIN: 0.49 1. (CXX) g++ options: -O3 -march=native -fopenmp -msse4.1 -fPIC -pie -ldl -lpthread
OpenBenchmarking.org ms, Fewer Is Better oneDNN 3.3 Harness: Deconvolution Batch shapes_3d - Data Type: bf16bf16bf16 - Engine: CPU 2 x Intel Xeon Max 9480 b c 0.6471 1.2942 1.9413 2.5884 3.2355 2.87581 2.68958 2.66555 MIN: 2.35 MIN: 2.13 MIN: 2.08 1. (CXX) g++ options: -O3 -march=native -fopenmp -msse4.1 -fPIC -pie -ldl -lpthread
OpenBenchmarking.org ms, Fewer Is Better oneDNN 3.3 Harness: Recurrent Neural Network Inference - Data Type: u8s8f32 - Engine: CPU 2 x Intel Xeon Max 9480 b c 300 600 900 1200 1500 1358.56 1421.14 1432.04 MIN: 1274.34 MIN: 1315.46 MIN: 1315.4 1. (CXX) g++ options: -O3 -march=native -fopenmp -msse4.1 -fPIC -pie -ldl -lpthread
OpenBenchmarking.org ms, Fewer Is Better oneDNN 3.3 Harness: Recurrent Neural Network Training - Data Type: bf16bf16bf16 - Engine: CPU 2 x Intel Xeon Max 9480 b c 4K 8K 12K 16K 20K 15485.1 17354.5 13214.9 MIN: 13354.1 MIN: 16239.7 MIN: 8210.66 1. (CXX) g++ options: -O3 -march=native -fopenmp -msse4.1 -fPIC -pie -ldl -lpthread
OpenBenchmarking.org ms, Fewer Is Better oneDNN 3.3 Harness: Recurrent Neural Network Inference - Data Type: bf16bf16bf16 - Engine: CPU 2 x Intel Xeon Max 9480 b c 300 600 900 1200 1500 1233.21 1395.16 1339.74 MIN: 1125.02 MIN: 1306.92 MIN: 1229.23 1. (CXX) g++ options: -O3 -march=native -fopenmp -msse4.1 -fPIC -pie -ldl -lpthread
OpenVINO OpenBenchmarking.org FPS, More Is Better OpenVINO 2023.1 Model: Face Detection FP16 - Device: CPU 2 x Intel Xeon Max 9480 b c 20 40 60 80 100 99.50 102.16 100.58 1. (CXX) g++ options: -isystem -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -fPIC -fvisibility=hidden -std=c++11 -MD -MT -MF
OpenBenchmarking.org ms, Fewer Is Better OpenVINO 2023.1 Model: Face Detection FP16 - Device: CPU 2 x Intel Xeon Max 9480 b c 80 160 240 320 400 370.41 360.67 366.12 MIN: 188.28 / MAX: 1143.06 MIN: 191.19 / MAX: 1244.17 MIN: 184.46 / MAX: 1083.67 1. (CXX) g++ options: -isystem -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -fPIC -fvisibility=hidden -std=c++11 -MD -MT -MF
OpenBenchmarking.org FPS, More Is Better OpenVINO 2023.1 Model: Person Detection FP16 - Device: CPU 2 x Intel Xeon Max 9480 b c 90 180 270 360 450 402.19 396.12 387.61 1. (CXX) g++ options: -isystem -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -fPIC -fvisibility=hidden -std=c++11 -MD -MT -MF
OpenBenchmarking.org ms, Fewer Is Better OpenVINO 2023.1 Model: Person Detection FP16 - Device: CPU 2 x Intel Xeon Max 9480 b c 20 40 60 80 100 91.89 93.25 95.29 MIN: 50.89 / MAX: 322.98 MIN: 51.93 / MAX: 743.64 MIN: 53.05 / MAX: 745.33 1. (CXX) g++ options: -isystem -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -fPIC -fvisibility=hidden -std=c++11 -MD -MT -MF
OpenBenchmarking.org FPS, More Is Better OpenVINO 2023.1 Model: Person Detection FP32 - Device: CPU 2 x Intel Xeon Max 9480 b c 90 180 270 360 450 397.31 389.31 391.02 1. (CXX) g++ options: -isystem -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -fPIC -fvisibility=hidden -std=c++11 -MD -MT -MF
OpenBenchmarking.org ms, Fewer Is Better OpenVINO 2023.1 Model: Person Detection FP32 - Device: CPU 2 x Intel Xeon Max 9480 b c 20 40 60 80 100 93.01 94.90 94.46 MIN: 49.97 / MAX: 447.8 MIN: 53.38 / MAX: 377.91 MIN: 50.39 / MAX: 395.2 1. (CXX) g++ options: -isystem -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -fPIC -fvisibility=hidden -std=c++11 -MD -MT -MF
OpenBenchmarking.org FPS, More Is Better OpenVINO 2023.1 Model: Vehicle Detection FP16 - Device: CPU 2 x Intel Xeon Max 9480 b c 400 800 1200 1600 2000 2083.79 2054.36 1914.54 1. (CXX) g++ options: -isystem -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -fPIC -fvisibility=hidden -std=c++11 -MD -MT -MF
OpenBenchmarking.org ms, Fewer Is Better OpenVINO 2023.1 Model: Vehicle Detection FP16 - Device: CPU 2 x Intel Xeon Max 9480 b c 5 10 15 20 25 17.71 17.96 19.27 MIN: 8.79 / MAX: 229.39 MIN: 9.08 / MAX: 231.3 MIN: 9.17 / MAX: 219.84 1. (CXX) g++ options: -isystem -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -fPIC -fvisibility=hidden -std=c++11 -MD -MT -MF
OpenBenchmarking.org FPS, More Is Better OpenVINO 2023.1 Model: Face Detection FP16-INT8 - Device: CPU 2 x Intel Xeon Max 9480 b c 70 140 210 280 350 324.92 324.12 325.04 1. (CXX) g++ options: -isystem -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -fPIC -fvisibility=hidden -std=c++11 -MD -MT -MF
OpenBenchmarking.org ms, Fewer Is Better OpenVINO 2023.1 Model: Face Detection FP16-INT8 - Device: CPU 2 x Intel Xeon Max 9480 b c 70 140 210 280 350 343.57 344.50 343.60 MIN: 279.79 / MAX: 642.34 MIN: 263.98 / MAX: 1205.69 MIN: 254.33 / MAX: 730.72 1. (CXX) g++ options: -isystem -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -fPIC -fvisibility=hidden -std=c++11 -MD -MT -MF
OpenBenchmarking.org FPS, More Is Better OpenVINO 2023.1 Model: Face Detection Retail FP16 - Device: CPU 2 x Intel Xeon Max 9480 b c 2K 4K 6K 8K 10K 10933.63 10917.91 11057.32 1. (CXX) g++ options: -isystem -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -fPIC -fvisibility=hidden -std=c++11 -MD -MT -MF
OpenBenchmarking.org ms, Fewer Is Better OpenVINO 2023.1 Model: Face Detection Retail FP16 - Device: CPU 2 x Intel Xeon Max 9480 b c 3 6 9 12 15 10.22 10.23 10.11 MIN: 7.99 / MAX: 47.71 MIN: 7.34 / MAX: 51.03 MIN: 7.6 / MAX: 47.72 1. (CXX) g++ options: -isystem -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -fPIC -fvisibility=hidden -std=c++11 -MD -MT -MF
OpenBenchmarking.org FPS, More Is Better OpenVINO 2023.1 Model: Road Segmentation ADAS FP16 - Device: CPU 2 x Intel Xeon Max 9480 b c 200 400 600 800 1000 909.89 966.99 947.32 1. (CXX) g++ options: -isystem -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -fPIC -fvisibility=hidden -std=c++11 -MD -MT -MF
OpenBenchmarking.org ms, Fewer Is Better OpenVINO 2023.1 Model: Road Segmentation ADAS FP16 - Device: CPU 2 x Intel Xeon Max 9480 b c 9 18 27 36 45 40.58 38.20 38.99 MIN: 26.74 / MAX: 152.15 MIN: 26.14 / MAX: 101.74 MIN: 25.7 / MAX: 110.21 1. (CXX) g++ options: -isystem -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -fPIC -fvisibility=hidden -std=c++11 -MD -MT -MF
OpenBenchmarking.org FPS, More Is Better OpenVINO 2023.1 Model: Vehicle Detection FP16-INT8 - Device: CPU 2 x Intel Xeon Max 9480 b c 1100 2200 3300 4400 5500 5108.28 5114.40 5121.05 1. (CXX) g++ options: -isystem -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -fPIC -fvisibility=hidden -std=c++11 -MD -MT -MF
OpenBenchmarking.org ms, Fewer Is Better OpenVINO 2023.1 Model: Vehicle Detection FP16-INT8 - Device: CPU 2 x Intel Xeon Max 9480 b c 5 10 15 20 25 21.90 21.86 21.83 MIN: 16.31 / MAX: 96.24 MIN: 16.73 / MAX: 100.58 MIN: 15.02 / MAX: 101.76 1. (CXX) g++ options: -isystem -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -fPIC -fvisibility=hidden -std=c++11 -MD -MT -MF
OpenBenchmarking.org FPS, More Is Better OpenVINO 2023.1 Model: Weld Porosity Detection FP16 - Device: CPU 2 x Intel Xeon Max 9480 b c 4K 8K 12K 16K 20K 15569.86 16043.36 16509.91 1. (CXX) g++ options: -isystem -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -fPIC -fvisibility=hidden -std=c++11 -MD -MT -MF
OpenBenchmarking.org ms, Fewer Is Better OpenVINO 2023.1 Model: Weld Porosity Detection FP16 - Device: CPU 2 x Intel Xeon Max 9480 b c 2 4 6 8 10 7.11 6.90 6.70 MIN: 4.45 / MAX: 76.17 MIN: 4.38 / MAX: 93.9 MIN: 4.41 / MAX: 66.66 1. (CXX) g++ options: -isystem -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -fPIC -fvisibility=hidden -std=c++11 -MD -MT -MF
OpenBenchmarking.org FPS, More Is Better OpenVINO 2023.1 Model: Face Detection Retail FP16-INT8 - Device: CPU 2 x Intel Xeon Max 9480 b c 3K 6K 9K 12K 15K 15732.26 15673.54 15741.68 1. (CXX) g++ options: -isystem -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -fPIC -fvisibility=hidden -std=c++11 -MD -MT -MF
OpenBenchmarking.org ms, Fewer Is Better OpenVINO 2023.1 Model: Face Detection Retail FP16-INT8 - Device: CPU 2 x Intel Xeon Max 9480 b c 2 4 6 8 10 7.10 7.12 7.07 MIN: 5.48 / MAX: 56.37 MIN: 5.46 / MAX: 57.88 MIN: 5.06 / MAX: 53.17 1. (CXX) g++ options: -isystem -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -fPIC -fvisibility=hidden -std=c++11 -MD -MT -MF
OpenBenchmarking.org FPS, More Is Better OpenVINO 2023.1 Model: Road Segmentation ADAS FP16-INT8 - Device: CPU 2 x Intel Xeon Max 9480 b c 300 600 900 1200 1500 1438.58 1447.80 1465.99 1. (CXX) g++ options: -isystem -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -fPIC -fvisibility=hidden -std=c++11 -MD -MT -MF
OpenBenchmarking.org ms, Fewer Is Better OpenVINO 2023.1 Model: Road Segmentation ADAS FP16-INT8 - Device: CPU 2 x Intel Xeon Max 9480 b c 20 40 60 80 100 77.76 77.27 76.33 MIN: 56.41 / MAX: 122.24 MIN: 66.37 / MAX: 117.25 MIN: 55.8 / MAX: 123.47 1. (CXX) g++ options: -isystem -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -fPIC -fvisibility=hidden -std=c++11 -MD -MT -MF
OpenBenchmarking.org FPS, More Is Better OpenVINO 2023.1 Model: Machine Translation EN To DE FP16 - Device: CPU 2 x Intel Xeon Max 9480 b c 70 140 210 280 350 279.73 317.24 303.33 1. (CXX) g++ options: -isystem -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -fPIC -fvisibility=hidden -std=c++11 -MD -MT -MF
OpenBenchmarking.org ms, Fewer Is Better OpenVINO 2023.1 Model: Machine Translation EN To DE FP16 - Device: CPU 2 x Intel Xeon Max 9480 b c 30 60 90 120 150 132.15 116.51 121.86 MIN: 35.31 / MAX: 443.72 MIN: 36.69 / MAX: 460.49 MIN: 40.59 / MAX: 437.84 1. (CXX) g++ options: -isystem -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -fPIC -fvisibility=hidden -std=c++11 -MD -MT -MF
OpenBenchmarking.org FPS, More Is Better OpenVINO 2023.1 Model: Weld Porosity Detection FP16-INT8 - Device: CPU 2 x Intel Xeon Max 9480 b c 6K 12K 18K 24K 30K 25154.51 24505.40 25725.95 1. (CXX) g++ options: -isystem -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -fPIC -fvisibility=hidden -std=c++11 -MD -MT -MF
OpenBenchmarking.org ms, Fewer Is Better OpenVINO 2023.1 Model: Weld Porosity Detection FP16-INT8 - Device: CPU 2 x Intel Xeon Max 9480 b c 0.9923 1.9846 2.9769 3.9692 4.9615 4.30 4.41 4.19 MIN: 2.77 / MAX: 70.91 MIN: 2.68 / MAX: 54.34 MIN: 2.7 / MAX: 72.58 1. (CXX) g++ options: -isystem -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -fPIC -fvisibility=hidden -std=c++11 -MD -MT -MF
OpenBenchmarking.org FPS, More Is Better OpenVINO 2023.1 Model: Person Vehicle Bike Detection FP16 - Device: CPU 2 x Intel Xeon Max 9480 b c 1300 2600 3900 5200 6500 6126.49 6150.01 6200.74 1. (CXX) g++ options: -isystem -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -fPIC -fvisibility=hidden -std=c++11 -MD -MT -MF
OpenBenchmarking.org ms, Fewer Is Better OpenVINO 2023.1 Model: Person Vehicle Bike Detection FP16 - Device: CPU 2 x Intel Xeon Max 9480 b c 4 8 12 16 20 18.25 18.18 18.04 MIN: 12.34 / MAX: 134.57 MIN: 12.71 / MAX: 50.87 MIN: 12.84 / MAX: 58.35 1. (CXX) g++ options: -isystem -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -fPIC -fvisibility=hidden -std=c++11 -MD -MT -MF
OpenBenchmarking.org FPS, More Is Better OpenVINO 2023.1 Model: Handwritten English Recognition FP16 - Device: CPU 2 x Intel Xeon Max 9480 b c 800 1600 2400 3200 4000 3506.92 3522.96 3558.58 1. (CXX) g++ options: -isystem -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -fPIC -fvisibility=hidden -std=c++11 -MD -MT -MF
OpenBenchmarking.org ms, Fewer Is Better OpenVINO 2023.1 Model: Handwritten English Recognition FP16 - Device: CPU 2 x Intel Xeon Max 9480 b c 7 14 21 28 35 31.90 31.77 31.44 MIN: 25.84 / MAX: 72.49 MIN: 25.35 / MAX: 81.87 MIN: 26.5 / MAX: 78.01 1. (CXX) g++ options: -isystem -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -fPIC -fvisibility=hidden -std=c++11 -MD -MT -MF
OpenBenchmarking.org FPS, More Is Better OpenVINO 2023.1 Model: Age Gender Recognition Retail 0013 FP16 - Device: CPU 2 x Intel Xeon Max 9480 b c 16K 32K 48K 64K 80K 68584.85 73923.61 73632.17 1. (CXX) g++ options: -isystem -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -fPIC -fvisibility=hidden -std=c++11 -MD -MT -MF
OpenBenchmarking.org ms, Fewer Is Better OpenVINO 2023.1 Model: Age Gender Recognition Retail 0013 FP16 - Device: CPU 2 x Intel Xeon Max 9480 b c 0.144 0.288 0.432 0.576 0.72 0.64 0.61 0.55 MIN: 0.3 / MAX: 64.49 MIN: 0.3 / MAX: 54.1 MIN: 0.29 / MAX: 35.36 1. (CXX) g++ options: -isystem -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -fPIC -fvisibility=hidden -std=c++11 -MD -MT -MF
OpenBenchmarking.org FPS, More Is Better OpenVINO 2023.1 Model: Handwritten English Recognition FP16-INT8 - Device: CPU 2 x Intel Xeon Max 9480 b c 500 1000 1500 2000 2500 2330.76 2332.98 2329.65 1. (CXX) g++ options: -isystem -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -fPIC -fvisibility=hidden -std=c++11 -MD -MT -MF
OpenBenchmarking.org ms, Fewer Is Better OpenVINO 2023.1 Model: Handwritten English Recognition FP16-INT8 - Device: CPU 2 x Intel Xeon Max 9480 b c 11 22 33 44 55 48.02 47.97 48.04 MIN: 40.77 / MAX: 97.23 MIN: 40.56 / MAX: 126.63 MIN: 38.44 / MAX: 119.56 1. (CXX) g++ options: -isystem -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -fPIC -fvisibility=hidden -std=c++11 -MD -MT -MF
OpenBenchmarking.org FPS, More Is Better OpenVINO 2023.1 Model: Age Gender Recognition Retail 0013 FP16-INT8 - Device: CPU 2 x Intel Xeon Max 9480 b c 20K 40K 60K 80K 100K 114066.70 115792.65 110854.06 1. (CXX) g++ options: -isystem -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -fPIC -fvisibility=hidden -std=c++11 -MD -MT -MF
OpenBenchmarking.org ms, Fewer Is Better OpenVINO 2023.1 Model: Age Gender Recognition Retail 0013 FP16-INT8 - Device: CPU 2 x Intel Xeon Max 9480 b c 0.0765 0.153 0.2295 0.306 0.3825 0.34 0.34 0.33 MIN: 0.28 / MAX: 31.55 MIN: 0.28 / MAX: 18.6 MIN: 0.27 / MAX: 11.81 1. (CXX) g++ options: -isystem -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -fPIC -fvisibility=hidden -std=c++11 -MD -MT -MF
2 x Intel Xeon Max 9480 Kernel Notes: Transparent Huge Pages: alwaysCompiler Notes: --build=x86_64-redhat-linux --disable-libunwind-exceptions --enable-__cxa_atexit --enable-bootstrap --enable-cet --enable-checking=release --enable-gnu-indirect-function --enable-gnu-unique-object --enable-host-bind-now --enable-host-pie --enable-initfini-array --enable-languages=c,c++,fortran,lto --enable-link-serialization=1 --enable-multilib --enable-offload-targets=nvptx-none --enable-plugin --enable-shared --enable-threads=posix --mandir=/usr/share/man --with-arch_32=x86-64 --with-arch_64=x86-64-v2 --with-build-config=bootstrap-lto --with-gcc-major-version-only --with-linker-hash-style=gnu --with-tune=generic --without-cuda-driver --without-islProcessor Notes: Scaling Governor: intel_cpufreq performance - CPU Microcode: 0x2c000271Python Notes: Python 3.9.17Security Notes: SELinux + gather_data_sampling: Not affected + itlb_multihit: Not affected + l1tf: Not affected + mds: Not affected + meltdown: Not affected + mmio_stale_data: Not affected + retbleed: Not affected + spec_store_bypass: Mitigation of SSB disabled via prctl + spectre_v1: Mitigation of usercopy/swapgs barriers and __user pointer sanitization + spectre_v2: Mitigation of Enhanced / Automatic IBRS IBPB: conditional RSB filling PBRSB-eIBRS: SW sequence + srbds: Not affected + tsx_async_abort: Not affected
Testing initiated at 17 October 2023 06:25 by user .
b Kernel Notes: Transparent Huge Pages: alwaysCompiler Notes: --build=x86_64-redhat-linux --disable-libunwind-exceptions --enable-__cxa_atexit --enable-bootstrap --enable-cet --enable-checking=release --enable-gnu-indirect-function --enable-gnu-unique-object --enable-host-bind-now --enable-host-pie --enable-initfini-array --enable-languages=c,c++,fortran,lto --enable-link-serialization=1 --enable-multilib --enable-offload-targets=nvptx-none --enable-plugin --enable-shared --enable-threads=posix --mandir=/usr/share/man --with-arch_32=x86-64 --with-arch_64=x86-64-v2 --with-build-config=bootstrap-lto --with-gcc-major-version-only --with-linker-hash-style=gnu --with-tune=generic --without-cuda-driver --without-islProcessor Notes: Scaling Governor: intel_cpufreq performance - CPU Microcode: 0x2c000271Python Notes: Python 3.9.17Security Notes: SELinux + gather_data_sampling: Not affected + itlb_multihit: Not affected + l1tf: Not affected + mds: Not affected + meltdown: Not affected + mmio_stale_data: Not affected + retbleed: Not affected + spec_store_bypass: Mitigation of SSB disabled via prctl + spectre_v1: Mitigation of usercopy/swapgs barriers and __user pointer sanitization + spectre_v2: Mitigation of Enhanced / Automatic IBRS IBPB: conditional RSB filling PBRSB-eIBRS: SW sequence + srbds: Not affected + tsx_async_abort: Not affected
Testing initiated at 17 October 2023 07:57 by user .
c Processor: 2 x Intel Xeon Max 9480 @ 3.50GHz (112 Cores / 224 Threads), Motherboard: Supermicro X13DEM v1.10 (1.3 BIOS), Chipset: Intel Device 1bce, Memory: 512GB, Disk: 2 x 7682GB INTEL SSDPF2KX076TZ, Graphics: ASPEED, Monitor: VE228, Network: 2 x Broadcom BCM57508 NetXtreme-E 10Gb/25Gb/40Gb/50Gb/100Gb/200Gb
OS: CentOS Stream 9, Kernel: 5.14.0-373.el9.x86_64 (x86_64), Desktop: GNOME Shell 40.10, Display Server: X Server, Compiler: GCC 11.4.1 20230605, File-System: xfs, Screen Resolution: 1920x1080
Kernel Notes: Transparent Huge Pages: alwaysCompiler Notes: --build=x86_64-redhat-linux --disable-libunwind-exceptions --enable-__cxa_atexit --enable-bootstrap --enable-cet --enable-checking=release --enable-gnu-indirect-function --enable-gnu-unique-object --enable-host-bind-now --enable-host-pie --enable-initfini-array --enable-languages=c,c++,fortran,lto --enable-link-serialization=1 --enable-multilib --enable-offload-targets=nvptx-none --enable-plugin --enable-shared --enable-threads=posix --mandir=/usr/share/man --with-arch_32=x86-64 --with-arch_64=x86-64-v2 --with-build-config=bootstrap-lto --with-gcc-major-version-only --with-linker-hash-style=gnu --with-tune=generic --without-cuda-driver --without-islProcessor Notes: Scaling Governor: intel_cpufreq performance - CPU Microcode: 0x2c000271Python Notes: Python 3.9.17Security Notes: SELinux + gather_data_sampling: Not affected + itlb_multihit: Not affected + l1tf: Not affected + mds: Not affected + meltdown: Not affected + mmio_stale_data: Not affected + retbleed: Not affected + spec_store_bypass: Mitigation of SSB disabled via prctl + spectre_v1: Mitigation of usercopy/swapgs barriers and __user pointer sanitization + spectre_v2: Mitigation of Enhanced / Automatic IBRS IBPB: conditional RSB filling PBRSB-eIBRS: SW sequence + srbds: Not affected + tsx_async_abort: Not affected
Testing initiated at 17 October 2023 09:14 by user .