2 x Intel Xeon Max 9480 testing with a Supermicro X13DEM v1.10 (1.3 BIOS) and ASPEED on CentOS Stream 9 via the Phoronix Test Suite.
2 x Intel Xeon Max 9480 Kernel Notes: Transparent Huge Pages: alwaysCompiler Notes: --build=x86_64-redhat-linux --disable-libunwind-exceptions --enable-__cxa_atexit --enable-bootstrap --enable-cet --enable-checking=release --enable-gnu-indirect-function --enable-gnu-unique-object --enable-host-bind-now --enable-host-pie --enable-initfini-array --enable-languages=c,c++,fortran,lto --enable-link-serialization=1 --enable-multilib --enable-offload-targets=nvptx-none --enable-plugin --enable-shared --enable-threads=posix --mandir=/usr/share/man --with-arch_32=x86-64 --with-arch_64=x86-64-v2 --with-build-config=bootstrap-lto --with-gcc-major-version-only --with-linker-hash-style=gnu --with-tune=generic --without-cuda-driver --without-islProcessor Notes: Scaling Governor: intel_cpufreq performance - CPU Microcode: 0x2c000271Python Notes: Python 3.9.17Security Notes: SELinux + gather_data_sampling: Not affected + itlb_multihit: Not affected + l1tf: Not affected + mds: Not affected + meltdown: Not affected + mmio_stale_data: Not affected + retbleed: Not affected + spec_store_bypass: Mitigation of SSB disabled via prctl + spectre_v1: Mitigation of usercopy/swapgs barriers and __user pointer sanitization + spectre_v2: Mitigation of Enhanced / Automatic IBRS IBPB: conditional RSB filling PBRSB-eIBRS: SW sequence + srbds: Not affected + tsx_async_abort: Not affected
b c Processor: 2 x Intel Xeon Max 9480 @ 3.50GHz (112 Cores / 224 Threads), Motherboard: Supermicro X13DEM v1.10 (1.3 BIOS), Chipset: Intel Device 1bce, Memory: 512GB, Disk: 2 x 7682GB INTEL SSDPF2KX076TZ, Graphics: ASPEED, Monitor: VE228, Network: 2 x Broadcom BCM57508 NetXtreme-E 10Gb/25Gb/40Gb/50Gb/100Gb/200Gb
OS: CentOS Stream 9, Kernel: 5.14.0-373.el9.x86_64 (x86_64), Desktop: GNOME Shell 40.10, Display Server: X Server, Compiler: GCC 11.4.1 20230605, File-System: xfs, Screen Resolution: 1920x1080
Embree Binary: Pathtracer - Model: Crown
2 x Intel Xeon Max 9480: The test quit with a non-zero exit status. E: ./bin/embree_pathtracer: error while loading shared libraries: libOpenGL.so.0: cannot open shared object file: No such file or directory
b: The test quit with a non-zero exit status. E: ./bin/embree_pathtracer: error while loading shared libraries: libOpenGL.so.0: cannot open shared object file: No such file or directory
c: The test quit with a non-zero exit status. E: ./bin/embree_pathtracer: error while loading shared libraries: libOpenGL.so.0: cannot open shared object file: No such file or directory
Binary: Pathtracer ISPC - Model: Crown
2 x Intel Xeon Max 9480: The test quit with a non-zero exit status. E: ./bin/embree_pathtracer_ispc: error while loading shared libraries: libOpenGL.so.0: cannot open shared object file: No such file or directory
b: The test quit with a non-zero exit status. E: ./bin/embree_pathtracer_ispc: error while loading shared libraries: libOpenGL.so.0: cannot open shared object file: No such file or directory
c: The test quit with a non-zero exit status. E: ./bin/embree_pathtracer_ispc: error while loading shared libraries: libOpenGL.so.0: cannot open shared object file: No such file or directory
Binary: Pathtracer - Model: Asian Dragon
2 x Intel Xeon Max 9480: The test quit with a non-zero exit status. E: ./bin/embree_pathtracer: error while loading shared libraries: libOpenGL.so.0: cannot open shared object file: No such file or directory
b: The test quit with a non-zero exit status. E: ./bin/embree_pathtracer: error while loading shared libraries: libOpenGL.so.0: cannot open shared object file: No such file or directory
c: The test quit with a non-zero exit status. E: ./bin/embree_pathtracer: error while loading shared libraries: libOpenGL.so.0: cannot open shared object file: No such file or directory
Binary: Pathtracer - Model: Asian Dragon Obj
2 x Intel Xeon Max 9480: The test quit with a non-zero exit status. E: ./bin/embree_pathtracer: error while loading shared libraries: libOpenGL.so.0: cannot open shared object file: No such file or directory
b: The test quit with a non-zero exit status. E: ./bin/embree_pathtracer: error while loading shared libraries: libOpenGL.so.0: cannot open shared object file: No such file or directory
c: The test quit with a non-zero exit status. E: ./bin/embree_pathtracer: error while loading shared libraries: libOpenGL.so.0: cannot open shared object file: No such file or directory
Binary: Pathtracer ISPC - Model: Asian Dragon
2 x Intel Xeon Max 9480: The test quit with a non-zero exit status. E: ./bin/embree_pathtracer_ispc: error while loading shared libraries: libOpenGL.so.0: cannot open shared object file: No such file or directory
b: The test quit with a non-zero exit status. E: ./bin/embree_pathtracer_ispc: error while loading shared libraries: libOpenGL.so.0: cannot open shared object file: No such file or directory
c: The test quit with a non-zero exit status. E: ./bin/embree_pathtracer_ispc: error while loading shared libraries: libOpenGL.so.0: cannot open shared object file: No such file or directory
Binary: Pathtracer ISPC - Model: Asian Dragon Obj
2 x Intel Xeon Max 9480: The test quit with a non-zero exit status. E: ./bin/embree_pathtracer_ispc: error while loading shared libraries: libOpenGL.so.0: cannot open shared object file: No such file or directory
b: The test quit with a non-zero exit status. E: ./bin/embree_pathtracer_ispc: error while loading shared libraries: libOpenGL.so.0: cannot open shared object file: No such file or directory
c: The test quit with a non-zero exit status. E: ./bin/embree_pathtracer_ispc: error while loading shared libraries: libOpenGL.so.0: cannot open shared object file: No such file or directory
OpenBenchmarking.org Items / Sec, More Is Better OpenVKL 2.0.0 Benchmark: vklBenchmarkCPU Scalar 2 x Intel Xeon Max 9480 c b 110 220 330 440 550 484 488 516 MIN: 44 / MAX: 7212 MIN: 49 / MAX: 7159 MIN: 48 / MAX: 7431
oneDNN This is a test of the Intel oneDNN as an Intel-optimized library for Deep Neural Networks and making use of its built-in benchdnn functionality. The result is the total perf time reported. Intel oneDNN was formerly known as DNNL (Deep Neural Network Library) and MKL-DNN before being rebranded as part of the Intel oneAPI toolkit. Learn more via the OpenBenchmarking.org test page.
OpenBenchmarking.org ms, Fewer Is Better oneDNN 3.3 Harness: IP Shapes 1D - Data Type: f32 - Engine: CPU 2 x Intel Xeon Max 9480 c b 2 4 6 8 10 6.39867 5.42410 3.44345 MIN: 5.47 MIN: 4.54 MIN: 2.76 1. (CXX) g++ options: -O3 -march=native -fopenmp -msse4.1 -fPIC -pie -ldl -lpthread
OpenBenchmarking.org ms, Fewer Is Better oneDNN 3.3 Harness: IP Shapes 3D - Data Type: f32 - Engine: CPU 2 x Intel Xeon Max 9480 c b 2 4 6 8 10 8.18813 7.91051 7.48884 MIN: 6.83 MIN: 6.61 MIN: 6.39 1. (CXX) g++ options: -O3 -march=native -fopenmp -msse4.1 -fPIC -pie -ldl -lpthread
OpenBenchmarking.org ms, Fewer Is Better oneDNN 3.3 Harness: IP Shapes 1D - Data Type: u8s8f32 - Engine: CPU 2 x Intel Xeon Max 9480 b c 2 4 6 8 10 6.90661 6.50478 3.76885 MIN: 5.54 MIN: 5.31 MIN: 2.93 1. (CXX) g++ options: -O3 -march=native -fopenmp -msse4.1 -fPIC -pie -ldl -lpthread
OpenBenchmarking.org ms, Fewer Is Better oneDNN 3.3 Harness: IP Shapes 3D - Data Type: u8s8f32 - Engine: CPU b 2 x Intel Xeon Max 9480 c 1.1941 2.3882 3.5823 4.7764 5.9705 5.30718 5.24744 4.22247 MIN: 3.81 MIN: 3.67 MIN: 2.99 1. (CXX) g++ options: -O3 -march=native -fopenmp -msse4.1 -fPIC -pie -ldl -lpthread
OpenBenchmarking.org ms, Fewer Is Better oneDNN 3.3 Harness: IP Shapes 1D - Data Type: bf16bf16bf16 - Engine: CPU c b 2 x Intel Xeon Max 9480 2 4 6 8 10 8.47846 8.37883 8.02865 MIN: 6.67 MIN: 6.6 MIN: 6.31 1. (CXX) g++ options: -O3 -march=native -fopenmp -msse4.1 -fPIC -pie -ldl -lpthread
OpenBenchmarking.org ms, Fewer Is Better oneDNN 3.3 Harness: IP Shapes 3D - Data Type: bf16bf16bf16 - Engine: CPU b c 2 x Intel Xeon Max 9480 20 40 60 80 100 84.30 81.85 80.95 MIN: 73.12 MIN: 70.51 MIN: 70.18 1. (CXX) g++ options: -O3 -march=native -fopenmp -msse4.1 -fPIC -pie -ldl -lpthread
OpenBenchmarking.org ms, Fewer Is Better oneDNN 3.3 Harness: Convolution Batch Shapes Auto - Data Type: f32 - Engine: CPU c 2 x Intel Xeon Max 9480 b 3 6 9 12 15 9.18192 9.07183 8.85383 MIN: 7.96 MIN: 7.95 MIN: 7.98 1. (CXX) g++ options: -O3 -march=native -fopenmp -msse4.1 -fPIC -pie -ldl -lpthread
OpenBenchmarking.org ms, Fewer Is Better oneDNN 3.3 Harness: Deconvolution Batch shapes_1d - Data Type: f32 - Engine: CPU 2 x Intel Xeon Max 9480 b c 4 8 12 16 20 14.81 14.75 14.21 MIN: 12.76 MIN: 12.76 MIN: 12.52 1. (CXX) g++ options: -O3 -march=native -fopenmp -msse4.1 -fPIC -pie -ldl -lpthread
OpenBenchmarking.org ms, Fewer Is Better oneDNN 3.3 Harness: Deconvolution Batch shapes_3d - Data Type: f32 - Engine: CPU 2 x Intel Xeon Max 9480 b c 0.9617 1.9234 2.8851 3.8468 4.8085 4.27423 4.06129 3.93714 MIN: 2.85 MIN: 2.94 MIN: 2.83 1. (CXX) g++ options: -O3 -march=native -fopenmp -msse4.1 -fPIC -pie -ldl -lpthread
OpenBenchmarking.org ms, Fewer Is Better oneDNN 3.3 Harness: Convolution Batch Shapes Auto - Data Type: u8s8f32 - Engine: CPU 2 x Intel Xeon Max 9480 b c 2 4 6 8 10 7.11116 7.01769 6.90576 MIN: 6.28 MIN: 6.23 MIN: 6.2 1. (CXX) g++ options: -O3 -march=native -fopenmp -msse4.1 -fPIC -pie -ldl -lpthread
OpenBenchmarking.org ms, Fewer Is Better oneDNN 3.3 Harness: Deconvolution Batch shapes_1d - Data Type: u8s8f32 - Engine: CPU 2 x Intel Xeon Max 9480 c b 0.1371 0.2742 0.4113 0.5484 0.6855 0.609168 0.595536 0.570467 MIN: 0.46 MIN: 0.46 MIN: 0.45 1. (CXX) g++ options: -O3 -march=native -fopenmp -msse4.1 -fPIC -pie -ldl -lpthread
OpenBenchmarking.org ms, Fewer Is Better oneDNN 3.3 Harness: Deconvolution Batch shapes_3d - Data Type: u8s8f32 - Engine: CPU b 2 x Intel Xeon Max 9480 c 0.2016 0.4032 0.6048 0.8064 1.008 0.895853 0.816029 0.814304 MIN: 0.65 MIN: 0.63 MIN: 0.63 1. (CXX) g++ options: -O3 -march=native -fopenmp -msse4.1 -fPIC -pie -ldl -lpthread
OpenBenchmarking.org ms, Fewer Is Better oneDNN 3.3 Harness: Recurrent Neural Network Training - Data Type: f32 - Engine: CPU c 2 x Intel Xeon Max 9480 b 3K 6K 9K 12K 15K 14073.0 13799.3 8774.0 MIN: 12966.8 MIN: 12882.7 MIN: 8085.71 1. (CXX) g++ options: -O3 -march=native -fopenmp -msse4.1 -fPIC -pie -ldl -lpthread
OpenBenchmarking.org ms, Fewer Is Better oneDNN 3.3 Harness: Recurrent Neural Network Inference - Data Type: f32 - Engine: CPU b 2 x Intel Xeon Max 9480 c 300 600 900 1200 1500 1415.42 1389.60 1303.30 MIN: 1317.1 MIN: 1235.38 MIN: 1163.26 1. (CXX) g++ options: -O3 -march=native -fopenmp -msse4.1 -fPIC -pie -ldl -lpthread
OpenBenchmarking.org ms, Fewer Is Better oneDNN 3.3 Harness: Recurrent Neural Network Training - Data Type: u8s8f32 - Engine: CPU 2 x Intel Xeon Max 9480 c b 4K 8K 12K 16K 20K 16878.5 15467.0 14404.0 MIN: 15171.7 MIN: 13719.2 MIN: 13476.1 1. (CXX) g++ options: -O3 -march=native -fopenmp -msse4.1 -fPIC -pie -ldl -lpthread
OpenBenchmarking.org ms, Fewer Is Better oneDNN 3.3 Harness: Convolution Batch Shapes Auto - Data Type: bf16bf16bf16 - Engine: CPU c 2 x Intel Xeon Max 9480 b 0.8863 1.7726 2.6589 3.5452 4.4315 3.93910 3.88111 3.85134 MIN: 3.25 MIN: 3.25 MIN: 3.36 1. (CXX) g++ options: -O3 -march=native -fopenmp -msse4.1 -fPIC -pie -ldl -lpthread
OpenBenchmarking.org ms, Fewer Is Better oneDNN 3.3 Harness: Deconvolution Batch shapes_1d - Data Type: bf16bf16bf16 - Engine: CPU b 2 x Intel Xeon Max 9480 c 0.1501 0.3002 0.4503 0.6004 0.7505 0.667091 0.647092 0.605543 MIN: 0.51 MIN: 0.51 MIN: 0.49 1. (CXX) g++ options: -O3 -march=native -fopenmp -msse4.1 -fPIC -pie -ldl -lpthread
OpenBenchmarking.org ms, Fewer Is Better oneDNN 3.3 Harness: Deconvolution Batch shapes_3d - Data Type: bf16bf16bf16 - Engine: CPU 2 x Intel Xeon Max 9480 b c 0.6471 1.2942 1.9413 2.5884 3.2355 2.87581 2.68958 2.66555 MIN: 2.35 MIN: 2.13 MIN: 2.08 1. (CXX) g++ options: -O3 -march=native -fopenmp -msse4.1 -fPIC -pie -ldl -lpthread
OpenBenchmarking.org ms, Fewer Is Better oneDNN 3.3 Harness: Recurrent Neural Network Inference - Data Type: u8s8f32 - Engine: CPU c b 2 x Intel Xeon Max 9480 300 600 900 1200 1500 1432.04 1421.14 1358.56 MIN: 1315.4 MIN: 1315.46 MIN: 1274.34 1. (CXX) g++ options: -O3 -march=native -fopenmp -msse4.1 -fPIC -pie -ldl -lpthread
OpenBenchmarking.org ms, Fewer Is Better oneDNN 3.3 Harness: Recurrent Neural Network Training - Data Type: bf16bf16bf16 - Engine: CPU b 2 x Intel Xeon Max 9480 c 4K 8K 12K 16K 20K 17354.5 15485.1 13214.9 MIN: 16239.7 MIN: 13354.1 MIN: 8210.66 1. (CXX) g++ options: -O3 -march=native -fopenmp -msse4.1 -fPIC -pie -ldl -lpthread
OpenBenchmarking.org ms, Fewer Is Better oneDNN 3.3 Harness: Recurrent Neural Network Inference - Data Type: bf16bf16bf16 - Engine: CPU b c 2 x Intel Xeon Max 9480 300 600 900 1200 1500 1395.16 1339.74 1233.21 MIN: 1306.92 MIN: 1229.23 MIN: 1125.02 1. (CXX) g++ options: -O3 -march=native -fopenmp -msse4.1 -fPIC -pie -ldl -lpthread
OpenVINO OpenBenchmarking.org FPS, More Is Better OpenVINO 2023.1 Model: Face Detection FP16 - Device: CPU 2 x Intel Xeon Max 9480 c b 20 40 60 80 100 99.50 100.58 102.16 1. (CXX) g++ options: -isystem -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -fPIC -fvisibility=hidden -std=c++11 -MD -MT -MF
OpenBenchmarking.org ms, Fewer Is Better OpenVINO 2023.1 Model: Face Detection FP16 - Device: CPU 2 x Intel Xeon Max 9480 c b 80 160 240 320 400 370.41 366.12 360.67 MIN: 188.28 / MAX: 1143.06 MIN: 184.46 / MAX: 1083.67 MIN: 191.19 / MAX: 1244.17 1. (CXX) g++ options: -isystem -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -fPIC -fvisibility=hidden -std=c++11 -MD -MT -MF
OpenBenchmarking.org FPS, More Is Better OpenVINO 2023.1 Model: Person Detection FP16 - Device: CPU c b 2 x Intel Xeon Max 9480 90 180 270 360 450 387.61 396.12 402.19 1. (CXX) g++ options: -isystem -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -fPIC -fvisibility=hidden -std=c++11 -MD -MT -MF
OpenBenchmarking.org ms, Fewer Is Better OpenVINO 2023.1 Model: Person Detection FP16 - Device: CPU c b 2 x Intel Xeon Max 9480 20 40 60 80 100 95.29 93.25 91.89 MIN: 53.05 / MAX: 745.33 MIN: 51.93 / MAX: 743.64 MIN: 50.89 / MAX: 322.98 1. (CXX) g++ options: -isystem -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -fPIC -fvisibility=hidden -std=c++11 -MD -MT -MF
OpenBenchmarking.org FPS, More Is Better OpenVINO 2023.1 Model: Person Detection FP32 - Device: CPU b c 2 x Intel Xeon Max 9480 90 180 270 360 450 389.31 391.02 397.31 1. (CXX) g++ options: -isystem -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -fPIC -fvisibility=hidden -std=c++11 -MD -MT -MF
OpenBenchmarking.org ms, Fewer Is Better OpenVINO 2023.1 Model: Person Detection FP32 - Device: CPU b c 2 x Intel Xeon Max 9480 20 40 60 80 100 94.90 94.46 93.01 MIN: 53.38 / MAX: 377.91 MIN: 50.39 / MAX: 395.2 MIN: 49.97 / MAX: 447.8 1. (CXX) g++ options: -isystem -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -fPIC -fvisibility=hidden -std=c++11 -MD -MT -MF
OpenBenchmarking.org FPS, More Is Better OpenVINO 2023.1 Model: Vehicle Detection FP16 - Device: CPU c b 2 x Intel Xeon Max 9480 400 800 1200 1600 2000 1914.54 2054.36 2083.79 1. (CXX) g++ options: -isystem -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -fPIC -fvisibility=hidden -std=c++11 -MD -MT -MF
OpenBenchmarking.org ms, Fewer Is Better OpenVINO 2023.1 Model: Vehicle Detection FP16 - Device: CPU c b 2 x Intel Xeon Max 9480 5 10 15 20 25 19.27 17.96 17.71 MIN: 9.17 / MAX: 219.84 MIN: 9.08 / MAX: 231.3 MIN: 8.79 / MAX: 229.39 1. (CXX) g++ options: -isystem -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -fPIC -fvisibility=hidden -std=c++11 -MD -MT -MF
OpenBenchmarking.org FPS, More Is Better OpenVINO 2023.1 Model: Face Detection FP16-INT8 - Device: CPU b 2 x Intel Xeon Max 9480 c 70 140 210 280 350 324.12 324.92 325.04 1. (CXX) g++ options: -isystem -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -fPIC -fvisibility=hidden -std=c++11 -MD -MT -MF
OpenBenchmarking.org ms, Fewer Is Better OpenVINO 2023.1 Model: Face Detection FP16-INT8 - Device: CPU b c 2 x Intel Xeon Max 9480 70 140 210 280 350 344.50 343.60 343.57 MIN: 263.98 / MAX: 1205.69 MIN: 254.33 / MAX: 730.72 MIN: 279.79 / MAX: 642.34 1. (CXX) g++ options: -isystem -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -fPIC -fvisibility=hidden -std=c++11 -MD -MT -MF
OpenBenchmarking.org FPS, More Is Better OpenVINO 2023.1 Model: Face Detection Retail FP16 - Device: CPU b 2 x Intel Xeon Max 9480 c 2K 4K 6K 8K 10K 10917.91 10933.63 11057.32 1. (CXX) g++ options: -isystem -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -fPIC -fvisibility=hidden -std=c++11 -MD -MT -MF
OpenBenchmarking.org ms, Fewer Is Better OpenVINO 2023.1 Model: Face Detection Retail FP16 - Device: CPU b 2 x Intel Xeon Max 9480 c 3 6 9 12 15 10.23 10.22 10.11 MIN: 7.34 / MAX: 51.03 MIN: 7.99 / MAX: 47.71 MIN: 7.6 / MAX: 47.72 1. (CXX) g++ options: -isystem -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -fPIC -fvisibility=hidden -std=c++11 -MD -MT -MF
OpenBenchmarking.org FPS, More Is Better OpenVINO 2023.1 Model: Road Segmentation ADAS FP16 - Device: CPU 2 x Intel Xeon Max 9480 c b 200 400 600 800 1000 909.89 947.32 966.99 1. (CXX) g++ options: -isystem -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -fPIC -fvisibility=hidden -std=c++11 -MD -MT -MF
OpenBenchmarking.org ms, Fewer Is Better OpenVINO 2023.1 Model: Road Segmentation ADAS FP16 - Device: CPU 2 x Intel Xeon Max 9480 c b 9 18 27 36 45 40.58 38.99 38.20 MIN: 26.74 / MAX: 152.15 MIN: 25.7 / MAX: 110.21 MIN: 26.14 / MAX: 101.74 1. (CXX) g++ options: -isystem -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -fPIC -fvisibility=hidden -std=c++11 -MD -MT -MF
OpenBenchmarking.org FPS, More Is Better OpenVINO 2023.1 Model: Vehicle Detection FP16-INT8 - Device: CPU 2 x Intel Xeon Max 9480 b c 1100 2200 3300 4400 5500 5108.28 5114.40 5121.05 1. (CXX) g++ options: -isystem -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -fPIC -fvisibility=hidden -std=c++11 -MD -MT -MF
OpenBenchmarking.org ms, Fewer Is Better OpenVINO 2023.1 Model: Vehicle Detection FP16-INT8 - Device: CPU 2 x Intel Xeon Max 9480 b c 5 10 15 20 25 21.90 21.86 21.83 MIN: 16.31 / MAX: 96.24 MIN: 16.73 / MAX: 100.58 MIN: 15.02 / MAX: 101.76 1. (CXX) g++ options: -isystem -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -fPIC -fvisibility=hidden -std=c++11 -MD -MT -MF
OpenBenchmarking.org FPS, More Is Better OpenVINO 2023.1 Model: Weld Porosity Detection FP16 - Device: CPU 2 x Intel Xeon Max 9480 b c 4K 8K 12K 16K 20K 15569.86 16043.36 16509.91 1. (CXX) g++ options: -isystem -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -fPIC -fvisibility=hidden -std=c++11 -MD -MT -MF
OpenBenchmarking.org ms, Fewer Is Better OpenVINO 2023.1 Model: Weld Porosity Detection FP16 - Device: CPU 2 x Intel Xeon Max 9480 b c 2 4 6 8 10 7.11 6.90 6.70 MIN: 4.45 / MAX: 76.17 MIN: 4.38 / MAX: 93.9 MIN: 4.41 / MAX: 66.66 1. (CXX) g++ options: -isystem -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -fPIC -fvisibility=hidden -std=c++11 -MD -MT -MF
OpenBenchmarking.org FPS, More Is Better OpenVINO 2023.1 Model: Face Detection Retail FP16-INT8 - Device: CPU b 2 x Intel Xeon Max 9480 c 3K 6K 9K 12K 15K 15673.54 15732.26 15741.68 1. (CXX) g++ options: -isystem -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -fPIC -fvisibility=hidden -std=c++11 -MD -MT -MF
OpenBenchmarking.org ms, Fewer Is Better OpenVINO 2023.1 Model: Face Detection Retail FP16-INT8 - Device: CPU b 2 x Intel Xeon Max 9480 c 2 4 6 8 10 7.12 7.10 7.07 MIN: 5.46 / MAX: 57.88 MIN: 5.48 / MAX: 56.37 MIN: 5.06 / MAX: 53.17 1. (CXX) g++ options: -isystem -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -fPIC -fvisibility=hidden -std=c++11 -MD -MT -MF
OpenBenchmarking.org FPS, More Is Better OpenVINO 2023.1 Model: Road Segmentation ADAS FP16-INT8 - Device: CPU 2 x Intel Xeon Max 9480 b c 300 600 900 1200 1500 1438.58 1447.80 1465.99 1. (CXX) g++ options: -isystem -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -fPIC -fvisibility=hidden -std=c++11 -MD -MT -MF
OpenBenchmarking.org ms, Fewer Is Better OpenVINO 2023.1 Model: Road Segmentation ADAS FP16-INT8 - Device: CPU 2 x Intel Xeon Max 9480 b c 20 40 60 80 100 77.76 77.27 76.33 MIN: 56.41 / MAX: 122.24 MIN: 66.37 / MAX: 117.25 MIN: 55.8 / MAX: 123.47 1. (CXX) g++ options: -isystem -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -fPIC -fvisibility=hidden -std=c++11 -MD -MT -MF
OpenBenchmarking.org FPS, More Is Better OpenVINO 2023.1 Model: Machine Translation EN To DE FP16 - Device: CPU 2 x Intel Xeon Max 9480 c b 70 140 210 280 350 279.73 303.33 317.24 1. (CXX) g++ options: -isystem -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -fPIC -fvisibility=hidden -std=c++11 -MD -MT -MF
OpenBenchmarking.org ms, Fewer Is Better OpenVINO 2023.1 Model: Machine Translation EN To DE FP16 - Device: CPU 2 x Intel Xeon Max 9480 c b 30 60 90 120 150 132.15 121.86 116.51 MIN: 35.31 / MAX: 443.72 MIN: 40.59 / MAX: 437.84 MIN: 36.69 / MAX: 460.49 1. (CXX) g++ options: -isystem -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -fPIC -fvisibility=hidden -std=c++11 -MD -MT -MF
OpenBenchmarking.org FPS, More Is Better OpenVINO 2023.1 Model: Weld Porosity Detection FP16-INT8 - Device: CPU b 2 x Intel Xeon Max 9480 c 6K 12K 18K 24K 30K 24505.40 25154.51 25725.95 1. (CXX) g++ options: -isystem -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -fPIC -fvisibility=hidden -std=c++11 -MD -MT -MF
OpenBenchmarking.org ms, Fewer Is Better OpenVINO 2023.1 Model: Weld Porosity Detection FP16-INT8 - Device: CPU b 2 x Intel Xeon Max 9480 c 0.9923 1.9846 2.9769 3.9692 4.9615 4.41 4.30 4.19 MIN: 2.68 / MAX: 54.34 MIN: 2.77 / MAX: 70.91 MIN: 2.7 / MAX: 72.58 1. (CXX) g++ options: -isystem -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -fPIC -fvisibility=hidden -std=c++11 -MD -MT -MF
OpenBenchmarking.org FPS, More Is Better OpenVINO 2023.1 Model: Person Vehicle Bike Detection FP16 - Device: CPU 2 x Intel Xeon Max 9480 b c 1300 2600 3900 5200 6500 6126.49 6150.01 6200.74 1. (CXX) g++ options: -isystem -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -fPIC -fvisibility=hidden -std=c++11 -MD -MT -MF
OpenBenchmarking.org ms, Fewer Is Better OpenVINO 2023.1 Model: Person Vehicle Bike Detection FP16 - Device: CPU 2 x Intel Xeon Max 9480 b c 4 8 12 16 20 18.25 18.18 18.04 MIN: 12.34 / MAX: 134.57 MIN: 12.71 / MAX: 50.87 MIN: 12.84 / MAX: 58.35 1. (CXX) g++ options: -isystem -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -fPIC -fvisibility=hidden -std=c++11 -MD -MT -MF
OpenBenchmarking.org FPS, More Is Better OpenVINO 2023.1 Model: Handwritten English Recognition FP16 - Device: CPU 2 x Intel Xeon Max 9480 b c 800 1600 2400 3200 4000 3506.92 3522.96 3558.58 1. (CXX) g++ options: -isystem -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -fPIC -fvisibility=hidden -std=c++11 -MD -MT -MF
OpenBenchmarking.org ms, Fewer Is Better OpenVINO 2023.1 Model: Handwritten English Recognition FP16 - Device: CPU 2 x Intel Xeon Max 9480 b c 7 14 21 28 35 31.90 31.77 31.44 MIN: 25.84 / MAX: 72.49 MIN: 25.35 / MAX: 81.87 MIN: 26.5 / MAX: 78.01 1. (CXX) g++ options: -isystem -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -fPIC -fvisibility=hidden -std=c++11 -MD -MT -MF
OpenBenchmarking.org FPS, More Is Better OpenVINO 2023.1 Model: Age Gender Recognition Retail 0013 FP16 - Device: CPU 2 x Intel Xeon Max 9480 c b 16K 32K 48K 64K 80K 68584.85 73632.17 73923.61 1. (CXX) g++ options: -isystem -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -fPIC -fvisibility=hidden -std=c++11 -MD -MT -MF
OpenBenchmarking.org ms, Fewer Is Better OpenVINO 2023.1 Model: Age Gender Recognition Retail 0013 FP16 - Device: CPU 2 x Intel Xeon Max 9480 b c 0.144 0.288 0.432 0.576 0.72 0.64 0.61 0.55 MIN: 0.3 / MAX: 64.49 MIN: 0.3 / MAX: 54.1 MIN: 0.29 / MAX: 35.36 1. (CXX) g++ options: -isystem -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -fPIC -fvisibility=hidden -std=c++11 -MD -MT -MF
OpenBenchmarking.org FPS, More Is Better OpenVINO 2023.1 Model: Handwritten English Recognition FP16-INT8 - Device: CPU c 2 x Intel Xeon Max 9480 b 500 1000 1500 2000 2500 2329.65 2330.76 2332.98 1. (CXX) g++ options: -isystem -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -fPIC -fvisibility=hidden -std=c++11 -MD -MT -MF
OpenBenchmarking.org ms, Fewer Is Better OpenVINO 2023.1 Model: Handwritten English Recognition FP16-INT8 - Device: CPU c 2 x Intel Xeon Max 9480 b 11 22 33 44 55 48.04 48.02 47.97 MIN: 38.44 / MAX: 119.56 MIN: 40.77 / MAX: 97.23 MIN: 40.56 / MAX: 126.63 1. (CXX) g++ options: -isystem -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -fPIC -fvisibility=hidden -std=c++11 -MD -MT -MF
OpenBenchmarking.org FPS, More Is Better OpenVINO 2023.1 Model: Age Gender Recognition Retail 0013 FP16-INT8 - Device: CPU c 2 x Intel Xeon Max 9480 b 20K 40K 60K 80K 100K 110854.06 114066.70 115792.65 1. (CXX) g++ options: -isystem -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -fPIC -fvisibility=hidden -std=c++11 -MD -MT -MF
OpenBenchmarking.org ms, Fewer Is Better OpenVINO 2023.1 Model: Age Gender Recognition Retail 0013 FP16-INT8 - Device: CPU b 2 x Intel Xeon Max 9480 c 0.0765 0.153 0.2295 0.306 0.3825 0.34 0.34 0.33 MIN: 0.28 / MAX: 18.6 MIN: 0.28 / MAX: 31.55 MIN: 0.27 / MAX: 11.81 1. (CXX) g++ options: -isystem -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -fPIC -fvisibility=hidden -std=c++11 -MD -MT -MF
2 x Intel Xeon Max 9480 Kernel Notes: Transparent Huge Pages: alwaysCompiler Notes: --build=x86_64-redhat-linux --disable-libunwind-exceptions --enable-__cxa_atexit --enable-bootstrap --enable-cet --enable-checking=release --enable-gnu-indirect-function --enable-gnu-unique-object --enable-host-bind-now --enable-host-pie --enable-initfini-array --enable-languages=c,c++,fortran,lto --enable-link-serialization=1 --enable-multilib --enable-offload-targets=nvptx-none --enable-plugin --enable-shared --enable-threads=posix --mandir=/usr/share/man --with-arch_32=x86-64 --with-arch_64=x86-64-v2 --with-build-config=bootstrap-lto --with-gcc-major-version-only --with-linker-hash-style=gnu --with-tune=generic --without-cuda-driver --without-islProcessor Notes: Scaling Governor: intel_cpufreq performance - CPU Microcode: 0x2c000271Python Notes: Python 3.9.17Security Notes: SELinux + gather_data_sampling: Not affected + itlb_multihit: Not affected + l1tf: Not affected + mds: Not affected + meltdown: Not affected + mmio_stale_data: Not affected + retbleed: Not affected + spec_store_bypass: Mitigation of SSB disabled via prctl + spectre_v1: Mitigation of usercopy/swapgs barriers and __user pointer sanitization + spectre_v2: Mitigation of Enhanced / Automatic IBRS IBPB: conditional RSB filling PBRSB-eIBRS: SW sequence + srbds: Not affected + tsx_async_abort: Not affected
Testing initiated at 17 October 2023 06:25 by user .
b Kernel Notes: Transparent Huge Pages: alwaysCompiler Notes: --build=x86_64-redhat-linux --disable-libunwind-exceptions --enable-__cxa_atexit --enable-bootstrap --enable-cet --enable-checking=release --enable-gnu-indirect-function --enable-gnu-unique-object --enable-host-bind-now --enable-host-pie --enable-initfini-array --enable-languages=c,c++,fortran,lto --enable-link-serialization=1 --enable-multilib --enable-offload-targets=nvptx-none --enable-plugin --enable-shared --enable-threads=posix --mandir=/usr/share/man --with-arch_32=x86-64 --with-arch_64=x86-64-v2 --with-build-config=bootstrap-lto --with-gcc-major-version-only --with-linker-hash-style=gnu --with-tune=generic --without-cuda-driver --without-islProcessor Notes: Scaling Governor: intel_cpufreq performance - CPU Microcode: 0x2c000271Python Notes: Python 3.9.17Security Notes: SELinux + gather_data_sampling: Not affected + itlb_multihit: Not affected + l1tf: Not affected + mds: Not affected + meltdown: Not affected + mmio_stale_data: Not affected + retbleed: Not affected + spec_store_bypass: Mitigation of SSB disabled via prctl + spectre_v1: Mitigation of usercopy/swapgs barriers and __user pointer sanitization + spectre_v2: Mitigation of Enhanced / Automatic IBRS IBPB: conditional RSB filling PBRSB-eIBRS: SW sequence + srbds: Not affected + tsx_async_abort: Not affected
Testing initiated at 17 October 2023 07:57 by user .
c Processor: 2 x Intel Xeon Max 9480 @ 3.50GHz (112 Cores / 224 Threads), Motherboard: Supermicro X13DEM v1.10 (1.3 BIOS), Chipset: Intel Device 1bce, Memory: 512GB, Disk: 2 x 7682GB INTEL SSDPF2KX076TZ, Graphics: ASPEED, Monitor: VE228, Network: 2 x Broadcom BCM57508 NetXtreme-E 10Gb/25Gb/40Gb/50Gb/100Gb/200Gb
OS: CentOS Stream 9, Kernel: 5.14.0-373.el9.x86_64 (x86_64), Desktop: GNOME Shell 40.10, Display Server: X Server, Compiler: GCC 11.4.1 20230605, File-System: xfs, Screen Resolution: 1920x1080
Kernel Notes: Transparent Huge Pages: alwaysCompiler Notes: --build=x86_64-redhat-linux --disable-libunwind-exceptions --enable-__cxa_atexit --enable-bootstrap --enable-cet --enable-checking=release --enable-gnu-indirect-function --enable-gnu-unique-object --enable-host-bind-now --enable-host-pie --enable-initfini-array --enable-languages=c,c++,fortran,lto --enable-link-serialization=1 --enable-multilib --enable-offload-targets=nvptx-none --enable-plugin --enable-shared --enable-threads=posix --mandir=/usr/share/man --with-arch_32=x86-64 --with-arch_64=x86-64-v2 --with-build-config=bootstrap-lto --with-gcc-major-version-only --with-linker-hash-style=gnu --with-tune=generic --without-cuda-driver --without-islProcessor Notes: Scaling Governor: intel_cpufreq performance - CPU Microcode: 0x2c000271Python Notes: Python 3.9.17Security Notes: SELinux + gather_data_sampling: Not affected + itlb_multihit: Not affected + l1tf: Not affected + mds: Not affected + meltdown: Not affected + mmio_stale_data: Not affected + retbleed: Not affected + spec_store_bypass: Mitigation of SSB disabled via prctl + spectre_v1: Mitigation of usercopy/swapgs barriers and __user pointer sanitization + spectre_v2: Mitigation of Enhanced / Automatic IBRS IBPB: conditional RSB filling PBRSB-eIBRS: SW sequence + srbds: Not affected + tsx_async_abort: Not affected
Testing initiated at 17 October 2023 09:14 by user .