centos stream extra

2 x Intel Xeon Max 9480 testing with a Supermicro X13DEM v1.10 (1.3 BIOS) and ASPEED on CentOS Stream 9 via the Phoronix Test Suite.

HTML result view exported from: https://openbenchmarking.org/result/2310174-NE-CENTOSSTR52&sro&grr.

centos stream extraProcessorMotherboardChipsetMemoryDiskGraphicsMonitorNetworkOSKernelDesktopDisplay ServerCompilerFile-SystemScreen Resolution2 x Intel Xeon Max 9480bc2 x Intel Xeon Max 9480 @ 3.50GHz (112 Cores / 224 Threads)Supermicro X13DEM v1.10 (1.3 BIOS)Intel Device 1bce512GB2 x 7682GB INTEL SSDPF2KX076TZASPEEDVE2282 x Broadcom BCM57508 NetXtreme-E 10Gb/25Gb/40Gb/50Gb/100Gb/200GbCentOS Stream 95.14.0-373.el9.x86_64 (x86_64)GNOME Shell 40.10X ServerGCC 11.4.1 20230605xfs1920x1080OpenBenchmarking.orgKernel Details- Transparent Huge Pages: alwaysCompiler Details- --build=x86_64-redhat-linux --disable-libunwind-exceptions --enable-__cxa_atexit --enable-bootstrap --enable-cet --enable-checking=release --enable-gnu-indirect-function --enable-gnu-unique-object --enable-host-bind-now --enable-host-pie --enable-initfini-array --enable-languages=c,c++,fortran,lto --enable-link-serialization=1 --enable-multilib --enable-offload-targets=nvptx-none --enable-plugin --enable-shared --enable-threads=posix --mandir=/usr/share/man --with-arch_32=x86-64 --with-arch_64=x86-64-v2 --with-build-config=bootstrap-lto --with-gcc-major-version-only --with-linker-hash-style=gnu --with-tune=generic --without-cuda-driver --without-isl Processor Details- Scaling Governor: intel_cpufreq performance - CPU Microcode: 0x2c000271Python Details- Python 3.9.17Security Details- SELinux + gather_data_sampling: Not affected + itlb_multihit: Not affected + l1tf: Not affected + mds: Not affected + meltdown: Not affected + mmio_stale_data: Not affected + retbleed: Not affected + spec_store_bypass: Mitigation of SSB disabled via prctl + spectre_v1: Mitigation of usercopy/swapgs barriers and __user pointer sanitization + spectre_v2: Mitigation of Enhanced / Automatic IBRS IBPB: conditional RSB filling PBRSB-eIBRS: SW sequence + srbds: Not affected + tsx_async_abort: Not affected

centos stream extraopenvkl: vklBenchmarkCPU ISPCopenvkl: vklBenchmarkCPU Scalaronednn: Recurrent Neural Network Training - u8s8f32 - CPUonednn: Recurrent Neural Network Training - bf16bf16bf16 - CPUonednn: Recurrent Neural Network Training - f32 - CPUopenvino: Face Detection FP16-INT8 - CPUopenvino: Face Detection FP16-INT8 - CPUonednn: Recurrent Neural Network Inference - u8s8f32 - CPUonednn: Recurrent Neural Network Inference - f32 - CPUonednn: Recurrent Neural Network Inference - bf16bf16bf16 - CPUopenvino: Person Vehicle Bike Detection FP16 - CPUopenvino: Person Vehicle Bike Detection FP16 - CPUopenvino: Person Detection FP32 - CPUopenvino: Person Detection FP32 - CPUopenvino: Person Detection FP16 - CPUopenvino: Person Detection FP16 - CPUopenvino: Face Detection FP16 - CPUopenvino: Face Detection FP16 - CPUopenvino: Vehicle Detection FP16-INT8 - CPUopenvino: Vehicle Detection FP16-INT8 - CPUopenvino: Road Segmentation ADAS FP16-INT8 - CPUopenvino: Road Segmentation ADAS FP16-INT8 - CPUopenvino: Road Segmentation ADAS FP16 - CPUopenvino: Road Segmentation ADAS FP16 - CPUopenvino: Face Detection Retail FP16-INT8 - CPUopenvino: Face Detection Retail FP16-INT8 - CPUopenvino: Vehicle Detection FP16 - CPUopenvino: Vehicle Detection FP16 - CPUopenvino: Face Detection Retail FP16 - CPUopenvino: Face Detection Retail FP16 - CPUopenvino: Machine Translation EN To DE FP16 - CPUopenvino: Machine Translation EN To DE FP16 - CPUopenvino: Handwritten English Recognition FP16-INT8 - CPUopenvino: Handwritten English Recognition FP16-INT8 - CPUopenvino: Weld Porosity Detection FP16 - CPUopenvino: Weld Porosity Detection FP16 - CPUopenvino: Handwritten English Recognition FP16 - CPUopenvino: Handwritten English Recognition FP16 - CPUopenvino: Age Gender Recognition Retail 0013 FP16 - CPUopenvino: Age Gender Recognition Retail 0013 FP16 - CPUopenvino: Weld Porosity Detection FP16-INT8 - CPUopenvino: Weld Porosity Detection FP16-INT8 - CPUopenvino: Age Gender Recognition Retail 0013 FP16-INT8 - CPUopenvino: Age Gender Recognition Retail 0013 FP16-INT8 - CPUoidn: RTLightmap.hdr.4096x4096 - CPU-Onlyoidn: RT.ldr_alb_nrm.3840x2160 - CPU-Onlyoidn: RT.hdr_alb_nrm.3840x2160 - CPU-Onlyonednn: Deconvolution Batch shapes_1d - f32 - CPUonednn: Deconvolution Batch shapes_1d - bf16bf16bf16 - CPUonednn: Deconvolution Batch shapes_1d - u8s8f32 - CPUonednn: IP Shapes 1D - bf16bf16bf16 - CPUonednn: IP Shapes 1D - f32 - CPUonednn: IP Shapes 1D - u8s8f32 - CPUonednn: IP Shapes 3D - bf16bf16bf16 - CPUonednn: IP Shapes 3D - f32 - CPUonednn: IP Shapes 3D - u8s8f32 - CPUonednn: Convolution Batch Shapes Auto - u8s8f32 - CPUonednn: Convolution Batch Shapes Auto - f32 - CPUonednn: Convolution Batch Shapes Auto - bf16bf16bf16 - CPUonednn: Deconvolution Batch shapes_3d - f32 - CPUonednn: Deconvolution Batch shapes_3d - u8s8f32 - CPUonednn: Deconvolution Batch shapes_3d - bf16bf16bf16 - CPUembree: Pathtracer ISPC - Crown2 x Intel Xeon Max 9480bc98248416878.515485.113799.3343.57324.921358.561389.61233.2118.256126.4993.01397.3191.89402.19370.4199.521.95108.2877.761438.5840.58909.897.115732.2617.712083.7910.2210933.63132.15279.7348.022330.767.1115569.8631.93506.920.6468584.854.325154.510.34114066.70.941.201.3814.80810.6470920.6091688.028656.398676.9066180.9468.188135.247447.111169.071833.881114.274230.8160292.8758110585161440417354.58774344.5324.121421.141415.421395.1618.186150.0194.9389.3193.25396.12360.67102.1621.865114.477.271447.838.2966.997.1215673.5417.962054.3610.2310917.91116.51317.2447.972332.986.916043.3631.773522.960.6173923.614.4124505.40.34115792.650.501.581.3214.75250.6670910.5704678.378833.443456.5047884.29667.488845.307187.017698.853833.851344.061290.8958532.6895810464881546713214.914073343.6325.041432.041303.31339.7418.046200.7494.46391.0295.29387.61366.12100.5821.835121.0576.331465.9938.99947.327.0715741.6819.271914.5410.1111057.32121.86303.3348.042329.656.716509.9131.443558.580.5573632.174.1925725.950.33110854.060.701.411.4414.2070.6055430.5955368.478465.42413.7688581.84737.910514.222476.905769.181923.93913.937140.8143042.66555OpenBenchmarking.org

OpenVKL

Benchmark: vklBenchmarkCPU ISPC

OpenBenchmarking.orgItems / Sec, More Is BetterOpenVKL 2.0.0Benchmark: vklBenchmarkCPU ISPC2 x Intel Xeon Max 9480bc200400600800100098210581046MIN: 77 / MAX: 12444MIN: 79 / MAX: 12504MIN: 82 / MAX: 12371

OpenVKL

Benchmark: vklBenchmarkCPU Scalar

OpenBenchmarking.orgItems / Sec, More Is BetterOpenVKL 2.0.0Benchmark: vklBenchmarkCPU Scalar2 x Intel Xeon Max 9480bc110220330440550484516488MIN: 44 / MAX: 7212MIN: 48 / MAX: 7431MIN: 49 / MAX: 7159

oneDNN

Harness: Recurrent Neural Network Training - Data Type: u8s8f32 - Engine: CPU

OpenBenchmarking.orgms, Fewer Is BetteroneDNN 3.3Harness: Recurrent Neural Network Training - Data Type: u8s8f32 - Engine: CPU2 x Intel Xeon Max 9480bc4K8K12K16K20K16878.514404.015467.0MIN: 15171.7MIN: 13476.1MIN: 13719.21. (CXX) g++ options: -O3 -march=native -fopenmp -msse4.1 -fPIC -pie -ldl -lpthread

oneDNN

Harness: Recurrent Neural Network Training - Data Type: bf16bf16bf16 - Engine: CPU

OpenBenchmarking.orgms, Fewer Is BetteroneDNN 3.3Harness: Recurrent Neural Network Training - Data Type: bf16bf16bf16 - Engine: CPU2 x Intel Xeon Max 9480bc4K8K12K16K20K15485.117354.513214.9MIN: 13354.1MIN: 16239.7MIN: 8210.661. (CXX) g++ options: -O3 -march=native -fopenmp -msse4.1 -fPIC -pie -ldl -lpthread

oneDNN

Harness: Recurrent Neural Network Training - Data Type: f32 - Engine: CPU

OpenBenchmarking.orgms, Fewer Is BetteroneDNN 3.3Harness: Recurrent Neural Network Training - Data Type: f32 - Engine: CPU2 x Intel Xeon Max 9480bc3K6K9K12K15K13799.38774.014073.0MIN: 12882.7MIN: 8085.71MIN: 12966.81. (CXX) g++ options: -O3 -march=native -fopenmp -msse4.1 -fPIC -pie -ldl -lpthread

OpenVINO

Model: Face Detection FP16-INT8 - Device: CPU

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2023.1Model: Face Detection FP16-INT8 - Device: CPU2 x Intel Xeon Max 9480bc70140210280350343.57344.50343.60MIN: 279.79 / MAX: 642.34MIN: 263.98 / MAX: 1205.69MIN: 254.33 / MAX: 730.721. (CXX) g++ options: -isystem -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -fPIC -fvisibility=hidden -std=c++11 -MD -MT -MF

OpenVINO

Model: Face Detection FP16-INT8 - Device: CPU

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2023.1Model: Face Detection FP16-INT8 - Device: CPU2 x Intel Xeon Max 9480bc70140210280350324.92324.12325.041. (CXX) g++ options: -isystem -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -fPIC -fvisibility=hidden -std=c++11 -MD -MT -MF

oneDNN

Harness: Recurrent Neural Network Inference - Data Type: u8s8f32 - Engine: CPU

OpenBenchmarking.orgms, Fewer Is BetteroneDNN 3.3Harness: Recurrent Neural Network Inference - Data Type: u8s8f32 - Engine: CPU2 x Intel Xeon Max 9480bc300600900120015001358.561421.141432.04MIN: 1274.34MIN: 1315.46MIN: 1315.41. (CXX) g++ options: -O3 -march=native -fopenmp -msse4.1 -fPIC -pie -ldl -lpthread

oneDNN

Harness: Recurrent Neural Network Inference - Data Type: f32 - Engine: CPU

OpenBenchmarking.orgms, Fewer Is BetteroneDNN 3.3Harness: Recurrent Neural Network Inference - Data Type: f32 - Engine: CPU2 x Intel Xeon Max 9480bc300600900120015001389.601415.421303.30MIN: 1235.38MIN: 1317.1MIN: 1163.261. (CXX) g++ options: -O3 -march=native -fopenmp -msse4.1 -fPIC -pie -ldl -lpthread

oneDNN

Harness: Recurrent Neural Network Inference - Data Type: bf16bf16bf16 - Engine: CPU

OpenBenchmarking.orgms, Fewer Is BetteroneDNN 3.3Harness: Recurrent Neural Network Inference - Data Type: bf16bf16bf16 - Engine: CPU2 x Intel Xeon Max 9480bc300600900120015001233.211395.161339.74MIN: 1125.02MIN: 1306.92MIN: 1229.231. (CXX) g++ options: -O3 -march=native -fopenmp -msse4.1 -fPIC -pie -ldl -lpthread

OpenVINO

Model: Person Vehicle Bike Detection FP16 - Device: CPU

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2023.1Model: Person Vehicle Bike Detection FP16 - Device: CPU2 x Intel Xeon Max 9480bc4812162018.2518.1818.04MIN: 12.34 / MAX: 134.57MIN: 12.71 / MAX: 50.87MIN: 12.84 / MAX: 58.351. (CXX) g++ options: -isystem -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -fPIC -fvisibility=hidden -std=c++11 -MD -MT -MF

OpenVINO

Model: Person Vehicle Bike Detection FP16 - Device: CPU

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2023.1Model: Person Vehicle Bike Detection FP16 - Device: CPU2 x Intel Xeon Max 9480bc130026003900520065006126.496150.016200.741. (CXX) g++ options: -isystem -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -fPIC -fvisibility=hidden -std=c++11 -MD -MT -MF

OpenVINO

Model: Person Detection FP32 - Device: CPU

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2023.1Model: Person Detection FP32 - Device: CPU2 x Intel Xeon Max 9480bc2040608010093.0194.9094.46MIN: 49.97 / MAX: 447.8MIN: 53.38 / MAX: 377.91MIN: 50.39 / MAX: 395.21. (CXX) g++ options: -isystem -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -fPIC -fvisibility=hidden -std=c++11 -MD -MT -MF

OpenVINO

Model: Person Detection FP32 - Device: CPU

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2023.1Model: Person Detection FP32 - Device: CPU2 x Intel Xeon Max 9480bc90180270360450397.31389.31391.021. (CXX) g++ options: -isystem -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -fPIC -fvisibility=hidden -std=c++11 -MD -MT -MF

OpenVINO

Model: Person Detection FP16 - Device: CPU

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2023.1Model: Person Detection FP16 - Device: CPU2 x Intel Xeon Max 9480bc2040608010091.8993.2595.29MIN: 50.89 / MAX: 322.98MIN: 51.93 / MAX: 743.64MIN: 53.05 / MAX: 745.331. (CXX) g++ options: -isystem -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -fPIC -fvisibility=hidden -std=c++11 -MD -MT -MF

OpenVINO

Model: Person Detection FP16 - Device: CPU

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2023.1Model: Person Detection FP16 - Device: CPU2 x Intel Xeon Max 9480bc90180270360450402.19396.12387.611. (CXX) g++ options: -isystem -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -fPIC -fvisibility=hidden -std=c++11 -MD -MT -MF

OpenVINO

Model: Face Detection FP16 - Device: CPU

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2023.1Model: Face Detection FP16 - Device: CPU2 x Intel Xeon Max 9480bc80160240320400370.41360.67366.12MIN: 188.28 / MAX: 1143.06MIN: 191.19 / MAX: 1244.17MIN: 184.46 / MAX: 1083.671. (CXX) g++ options: -isystem -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -fPIC -fvisibility=hidden -std=c++11 -MD -MT -MF

OpenVINO

Model: Face Detection FP16 - Device: CPU

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2023.1Model: Face Detection FP16 - Device: CPU2 x Intel Xeon Max 9480bc2040608010099.50102.16100.581. (CXX) g++ options: -isystem -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -fPIC -fvisibility=hidden -std=c++11 -MD -MT -MF

OpenVINO

Model: Vehicle Detection FP16-INT8 - Device: CPU

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2023.1Model: Vehicle Detection FP16-INT8 - Device: CPU2 x Intel Xeon Max 9480bc51015202521.9021.8621.83MIN: 16.31 / MAX: 96.24MIN: 16.73 / MAX: 100.58MIN: 15.02 / MAX: 101.761. (CXX) g++ options: -isystem -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -fPIC -fvisibility=hidden -std=c++11 -MD -MT -MF

OpenVINO

Model: Vehicle Detection FP16-INT8 - Device: CPU

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2023.1Model: Vehicle Detection FP16-INT8 - Device: CPU2 x Intel Xeon Max 9480bc110022003300440055005108.285114.405121.051. (CXX) g++ options: -isystem -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -fPIC -fvisibility=hidden -std=c++11 -MD -MT -MF

OpenVINO

Model: Road Segmentation ADAS FP16-INT8 - Device: CPU

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2023.1Model: Road Segmentation ADAS FP16-INT8 - Device: CPU2 x Intel Xeon Max 9480bc2040608010077.7677.2776.33MIN: 56.41 / MAX: 122.24MIN: 66.37 / MAX: 117.25MIN: 55.8 / MAX: 123.471. (CXX) g++ options: -isystem -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -fPIC -fvisibility=hidden -std=c++11 -MD -MT -MF

OpenVINO

Model: Road Segmentation ADAS FP16-INT8 - Device: CPU

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2023.1Model: Road Segmentation ADAS FP16-INT8 - Device: CPU2 x Intel Xeon Max 9480bc300600900120015001438.581447.801465.991. (CXX) g++ options: -isystem -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -fPIC -fvisibility=hidden -std=c++11 -MD -MT -MF

OpenVINO

Model: Road Segmentation ADAS FP16 - Device: CPU

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2023.1Model: Road Segmentation ADAS FP16 - Device: CPU2 x Intel Xeon Max 9480bc91827364540.5838.2038.99MIN: 26.74 / MAX: 152.15MIN: 26.14 / MAX: 101.74MIN: 25.7 / MAX: 110.211. (CXX) g++ options: -isystem -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -fPIC -fvisibility=hidden -std=c++11 -MD -MT -MF

OpenVINO

Model: Road Segmentation ADAS FP16 - Device: CPU

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2023.1Model: Road Segmentation ADAS FP16 - Device: CPU2 x Intel Xeon Max 9480bc2004006008001000909.89966.99947.321. (CXX) g++ options: -isystem -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -fPIC -fvisibility=hidden -std=c++11 -MD -MT -MF

OpenVINO

Model: Face Detection Retail FP16-INT8 - Device: CPU

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2023.1Model: Face Detection Retail FP16-INT8 - Device: CPU2 x Intel Xeon Max 9480bc2468107.107.127.07MIN: 5.48 / MAX: 56.37MIN: 5.46 / MAX: 57.88MIN: 5.06 / MAX: 53.171. (CXX) g++ options: -isystem -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -fPIC -fvisibility=hidden -std=c++11 -MD -MT -MF

OpenVINO

Model: Face Detection Retail FP16-INT8 - Device: CPU

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2023.1Model: Face Detection Retail FP16-INT8 - Device: CPU2 x Intel Xeon Max 9480bc3K6K9K12K15K15732.2615673.5415741.681. (CXX) g++ options: -isystem -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -fPIC -fvisibility=hidden -std=c++11 -MD -MT -MF

OpenVINO

Model: Vehicle Detection FP16 - Device: CPU

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2023.1Model: Vehicle Detection FP16 - Device: CPU2 x Intel Xeon Max 9480bc51015202517.7117.9619.27MIN: 8.79 / MAX: 229.39MIN: 9.08 / MAX: 231.3MIN: 9.17 / MAX: 219.841. (CXX) g++ options: -isystem -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -fPIC -fvisibility=hidden -std=c++11 -MD -MT -MF

OpenVINO

Model: Vehicle Detection FP16 - Device: CPU

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2023.1Model: Vehicle Detection FP16 - Device: CPU2 x Intel Xeon Max 9480bc4008001200160020002083.792054.361914.541. (CXX) g++ options: -isystem -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -fPIC -fvisibility=hidden -std=c++11 -MD -MT -MF

OpenVINO

Model: Face Detection Retail FP16 - Device: CPU

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2023.1Model: Face Detection Retail FP16 - Device: CPU2 x Intel Xeon Max 9480bc369121510.2210.2310.11MIN: 7.99 / MAX: 47.71MIN: 7.34 / MAX: 51.03MIN: 7.6 / MAX: 47.721. (CXX) g++ options: -isystem -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -fPIC -fvisibility=hidden -std=c++11 -MD -MT -MF

OpenVINO

Model: Face Detection Retail FP16 - Device: CPU

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2023.1Model: Face Detection Retail FP16 - Device: CPU2 x Intel Xeon Max 9480bc2K4K6K8K10K10933.6310917.9111057.321. (CXX) g++ options: -isystem -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -fPIC -fvisibility=hidden -std=c++11 -MD -MT -MF

OpenVINO

Model: Machine Translation EN To DE FP16 - Device: CPU

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2023.1Model: Machine Translation EN To DE FP16 - Device: CPU2 x Intel Xeon Max 9480bc306090120150132.15116.51121.86MIN: 35.31 / MAX: 443.72MIN: 36.69 / MAX: 460.49MIN: 40.59 / MAX: 437.841. (CXX) g++ options: -isystem -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -fPIC -fvisibility=hidden -std=c++11 -MD -MT -MF

OpenVINO

Model: Machine Translation EN To DE FP16 - Device: CPU

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2023.1Model: Machine Translation EN To DE FP16 - Device: CPU2 x Intel Xeon Max 9480bc70140210280350279.73317.24303.331. (CXX) g++ options: -isystem -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -fPIC -fvisibility=hidden -std=c++11 -MD -MT -MF

OpenVINO

Model: Handwritten English Recognition FP16-INT8 - Device: CPU

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2023.1Model: Handwritten English Recognition FP16-INT8 - Device: CPU2 x Intel Xeon Max 9480bc112233445548.0247.9748.04MIN: 40.77 / MAX: 97.23MIN: 40.56 / MAX: 126.63MIN: 38.44 / MAX: 119.561. (CXX) g++ options: -isystem -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -fPIC -fvisibility=hidden -std=c++11 -MD -MT -MF

OpenVINO

Model: Handwritten English Recognition FP16-INT8 - Device: CPU

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2023.1Model: Handwritten English Recognition FP16-INT8 - Device: CPU2 x Intel Xeon Max 9480bc50010001500200025002330.762332.982329.651. (CXX) g++ options: -isystem -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -fPIC -fvisibility=hidden -std=c++11 -MD -MT -MF

OpenVINO

Model: Weld Porosity Detection FP16 - Device: CPU

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2023.1Model: Weld Porosity Detection FP16 - Device: CPU2 x Intel Xeon Max 9480bc2468107.116.906.70MIN: 4.45 / MAX: 76.17MIN: 4.38 / MAX: 93.9MIN: 4.41 / MAX: 66.661. (CXX) g++ options: -isystem -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -fPIC -fvisibility=hidden -std=c++11 -MD -MT -MF

OpenVINO

Model: Weld Porosity Detection FP16 - Device: CPU

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2023.1Model: Weld Porosity Detection FP16 - Device: CPU2 x Intel Xeon Max 9480bc4K8K12K16K20K15569.8616043.3616509.911. (CXX) g++ options: -isystem -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -fPIC -fvisibility=hidden -std=c++11 -MD -MT -MF

OpenVINO

Model: Handwritten English Recognition FP16 - Device: CPU

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2023.1Model: Handwritten English Recognition FP16 - Device: CPU2 x Intel Xeon Max 9480bc71421283531.9031.7731.44MIN: 25.84 / MAX: 72.49MIN: 25.35 / MAX: 81.87MIN: 26.5 / MAX: 78.011. (CXX) g++ options: -isystem -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -fPIC -fvisibility=hidden -std=c++11 -MD -MT -MF

OpenVINO

Model: Handwritten English Recognition FP16 - Device: CPU

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2023.1Model: Handwritten English Recognition FP16 - Device: CPU2 x Intel Xeon Max 9480bc80016002400320040003506.923522.963558.581. (CXX) g++ options: -isystem -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -fPIC -fvisibility=hidden -std=c++11 -MD -MT -MF

OpenVINO

Model: Age Gender Recognition Retail 0013 FP16 - Device: CPU

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2023.1Model: Age Gender Recognition Retail 0013 FP16 - Device: CPU2 x Intel Xeon Max 9480bc0.1440.2880.4320.5760.720.640.610.55MIN: 0.3 / MAX: 64.49MIN: 0.3 / MAX: 54.1MIN: 0.29 / MAX: 35.361. (CXX) g++ options: -isystem -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -fPIC -fvisibility=hidden -std=c++11 -MD -MT -MF

OpenVINO

Model: Age Gender Recognition Retail 0013 FP16 - Device: CPU

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2023.1Model: Age Gender Recognition Retail 0013 FP16 - Device: CPU2 x Intel Xeon Max 9480bc16K32K48K64K80K68584.8573923.6173632.171. (CXX) g++ options: -isystem -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -fPIC -fvisibility=hidden -std=c++11 -MD -MT -MF

OpenVINO

Model: Weld Porosity Detection FP16-INT8 - Device: CPU

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2023.1Model: Weld Porosity Detection FP16-INT8 - Device: CPU2 x Intel Xeon Max 9480bc0.99231.98462.97693.96924.96154.304.414.19MIN: 2.77 / MAX: 70.91MIN: 2.68 / MAX: 54.34MIN: 2.7 / MAX: 72.581. (CXX) g++ options: -isystem -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -fPIC -fvisibility=hidden -std=c++11 -MD -MT -MF

OpenVINO

Model: Weld Porosity Detection FP16-INT8 - Device: CPU

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2023.1Model: Weld Porosity Detection FP16-INT8 - Device: CPU2 x Intel Xeon Max 9480bc6K12K18K24K30K25154.5124505.4025725.951. (CXX) g++ options: -isystem -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -fPIC -fvisibility=hidden -std=c++11 -MD -MT -MF

OpenVINO

Model: Age Gender Recognition Retail 0013 FP16-INT8 - Device: CPU

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2023.1Model: Age Gender Recognition Retail 0013 FP16-INT8 - Device: CPU2 x Intel Xeon Max 9480bc0.07650.1530.22950.3060.38250.340.340.33MIN: 0.28 / MAX: 31.55MIN: 0.28 / MAX: 18.6MIN: 0.27 / MAX: 11.811. (CXX) g++ options: -isystem -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -fPIC -fvisibility=hidden -std=c++11 -MD -MT -MF

OpenVINO

Model: Age Gender Recognition Retail 0013 FP16-INT8 - Device: CPU

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2023.1Model: Age Gender Recognition Retail 0013 FP16-INT8 - Device: CPU2 x Intel Xeon Max 9480bc20K40K60K80K100K114066.70115792.65110854.061. (CXX) g++ options: -isystem -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -fPIC -fvisibility=hidden -std=c++11 -MD -MT -MF

Intel Open Image Denoise

Run: RTLightmap.hdr.4096x4096 - Device: CPU-Only

OpenBenchmarking.orgImages / Sec, More Is BetterIntel Open Image Denoise 2.1Run: RTLightmap.hdr.4096x4096 - Device: CPU-Only2 x Intel Xeon Max 9480bc0.21150.4230.63450.8461.05750.940.500.70

Intel Open Image Denoise

Run: RT.ldr_alb_nrm.3840x2160 - Device: CPU-Only

OpenBenchmarking.orgImages / Sec, More Is BetterIntel Open Image Denoise 2.1Run: RT.ldr_alb_nrm.3840x2160 - Device: CPU-Only2 x Intel Xeon Max 9480bc0.35550.7111.06651.4221.77751.201.581.41

Intel Open Image Denoise

Run: RT.hdr_alb_nrm.3840x2160 - Device: CPU-Only

OpenBenchmarking.orgImages / Sec, More Is BetterIntel Open Image Denoise 2.1Run: RT.hdr_alb_nrm.3840x2160 - Device: CPU-Only2 x Intel Xeon Max 9480bc0.3240.6480.9721.2961.621.381.321.44

oneDNN

Harness: Deconvolution Batch shapes_1d - Data Type: f32 - Engine: CPU

OpenBenchmarking.orgms, Fewer Is BetteroneDNN 3.3Harness: Deconvolution Batch shapes_1d - Data Type: f32 - Engine: CPU2 x Intel Xeon Max 9480bc4812162014.8114.7514.21MIN: 12.76MIN: 12.76MIN: 12.521. (CXX) g++ options: -O3 -march=native -fopenmp -msse4.1 -fPIC -pie -ldl -lpthread

oneDNN

Harness: Deconvolution Batch shapes_1d - Data Type: bf16bf16bf16 - Engine: CPU

OpenBenchmarking.orgms, Fewer Is BetteroneDNN 3.3Harness: Deconvolution Batch shapes_1d - Data Type: bf16bf16bf16 - Engine: CPU2 x Intel Xeon Max 9480bc0.15010.30020.45030.60040.75050.6470920.6670910.605543MIN: 0.51MIN: 0.51MIN: 0.491. (CXX) g++ options: -O3 -march=native -fopenmp -msse4.1 -fPIC -pie -ldl -lpthread

oneDNN

Harness: Deconvolution Batch shapes_1d - Data Type: u8s8f32 - Engine: CPU

OpenBenchmarking.orgms, Fewer Is BetteroneDNN 3.3Harness: Deconvolution Batch shapes_1d - Data Type: u8s8f32 - Engine: CPU2 x Intel Xeon Max 9480bc0.13710.27420.41130.54840.68550.6091680.5704670.595536MIN: 0.46MIN: 0.45MIN: 0.461. (CXX) g++ options: -O3 -march=native -fopenmp -msse4.1 -fPIC -pie -ldl -lpthread

oneDNN

Harness: IP Shapes 1D - Data Type: bf16bf16bf16 - Engine: CPU

OpenBenchmarking.orgms, Fewer Is BetteroneDNN 3.3Harness: IP Shapes 1D - Data Type: bf16bf16bf16 - Engine: CPU2 x Intel Xeon Max 9480bc2468108.028658.378838.47846MIN: 6.31MIN: 6.6MIN: 6.671. (CXX) g++ options: -O3 -march=native -fopenmp -msse4.1 -fPIC -pie -ldl -lpthread

oneDNN

Harness: IP Shapes 1D - Data Type: f32 - Engine: CPU

OpenBenchmarking.orgms, Fewer Is BetteroneDNN 3.3Harness: IP Shapes 1D - Data Type: f32 - Engine: CPU2 x Intel Xeon Max 9480bc2468106.398673.443455.42410MIN: 5.47MIN: 2.76MIN: 4.541. (CXX) g++ options: -O3 -march=native -fopenmp -msse4.1 -fPIC -pie -ldl -lpthread

oneDNN

Harness: IP Shapes 1D - Data Type: u8s8f32 - Engine: CPU

OpenBenchmarking.orgms, Fewer Is BetteroneDNN 3.3Harness: IP Shapes 1D - Data Type: u8s8f32 - Engine: CPU2 x Intel Xeon Max 9480bc2468106.906616.504783.76885MIN: 5.54MIN: 5.31MIN: 2.931. (CXX) g++ options: -O3 -march=native -fopenmp -msse4.1 -fPIC -pie -ldl -lpthread

oneDNN

Harness: IP Shapes 3D - Data Type: bf16bf16bf16 - Engine: CPU

OpenBenchmarking.orgms, Fewer Is BetteroneDNN 3.3Harness: IP Shapes 3D - Data Type: bf16bf16bf16 - Engine: CPU2 x Intel Xeon Max 9480bc2040608010080.9584.3081.85MIN: 70.18MIN: 73.12MIN: 70.511. (CXX) g++ options: -O3 -march=native -fopenmp -msse4.1 -fPIC -pie -ldl -lpthread

oneDNN

Harness: IP Shapes 3D - Data Type: f32 - Engine: CPU

OpenBenchmarking.orgms, Fewer Is BetteroneDNN 3.3Harness: IP Shapes 3D - Data Type: f32 - Engine: CPU2 x Intel Xeon Max 9480bc2468108.188137.488847.91051MIN: 6.83MIN: 6.39MIN: 6.611. (CXX) g++ options: -O3 -march=native -fopenmp -msse4.1 -fPIC -pie -ldl -lpthread

oneDNN

Harness: IP Shapes 3D - Data Type: u8s8f32 - Engine: CPU

OpenBenchmarking.orgms, Fewer Is BetteroneDNN 3.3Harness: IP Shapes 3D - Data Type: u8s8f32 - Engine: CPU2 x Intel Xeon Max 9480bc1.19412.38823.58234.77645.97055.247445.307184.22247MIN: 3.67MIN: 3.81MIN: 2.991. (CXX) g++ options: -O3 -march=native -fopenmp -msse4.1 -fPIC -pie -ldl -lpthread

oneDNN

Harness: Convolution Batch Shapes Auto - Data Type: u8s8f32 - Engine: CPU

OpenBenchmarking.orgms, Fewer Is BetteroneDNN 3.3Harness: Convolution Batch Shapes Auto - Data Type: u8s8f32 - Engine: CPU2 x Intel Xeon Max 9480bc2468107.111167.017696.90576MIN: 6.28MIN: 6.23MIN: 6.21. (CXX) g++ options: -O3 -march=native -fopenmp -msse4.1 -fPIC -pie -ldl -lpthread

oneDNN

Harness: Convolution Batch Shapes Auto - Data Type: f32 - Engine: CPU

OpenBenchmarking.orgms, Fewer Is BetteroneDNN 3.3Harness: Convolution Batch Shapes Auto - Data Type: f32 - Engine: CPU2 x Intel Xeon Max 9480bc36912159.071838.853839.18192MIN: 7.95MIN: 7.98MIN: 7.961. (CXX) g++ options: -O3 -march=native -fopenmp -msse4.1 -fPIC -pie -ldl -lpthread

oneDNN

Harness: Convolution Batch Shapes Auto - Data Type: bf16bf16bf16 - Engine: CPU

OpenBenchmarking.orgms, Fewer Is BetteroneDNN 3.3Harness: Convolution Batch Shapes Auto - Data Type: bf16bf16bf16 - Engine: CPU2 x Intel Xeon Max 9480bc0.88631.77262.65893.54524.43153.881113.851343.93910MIN: 3.25MIN: 3.36MIN: 3.251. (CXX) g++ options: -O3 -march=native -fopenmp -msse4.1 -fPIC -pie -ldl -lpthread

oneDNN

Harness: Deconvolution Batch shapes_3d - Data Type: f32 - Engine: CPU

OpenBenchmarking.orgms, Fewer Is BetteroneDNN 3.3Harness: Deconvolution Batch shapes_3d - Data Type: f32 - Engine: CPU2 x Intel Xeon Max 9480bc0.96171.92342.88513.84684.80854.274234.061293.93714MIN: 2.85MIN: 2.94MIN: 2.831. (CXX) g++ options: -O3 -march=native -fopenmp -msse4.1 -fPIC -pie -ldl -lpthread

oneDNN

Harness: Deconvolution Batch shapes_3d - Data Type: u8s8f32 - Engine: CPU

OpenBenchmarking.orgms, Fewer Is BetteroneDNN 3.3Harness: Deconvolution Batch shapes_3d - Data Type: u8s8f32 - Engine: CPU2 x Intel Xeon Max 9480bc0.20160.40320.60480.80641.0080.8160290.8958530.814304MIN: 0.63MIN: 0.65MIN: 0.631. (CXX) g++ options: -O3 -march=native -fopenmp -msse4.1 -fPIC -pie -ldl -lpthread

oneDNN

Harness: Deconvolution Batch shapes_3d - Data Type: bf16bf16bf16 - Engine: CPU

OpenBenchmarking.orgms, Fewer Is BetteroneDNN 3.3Harness: Deconvolution Batch shapes_3d - Data Type: bf16bf16bf16 - Engine: CPU2 x Intel Xeon Max 9480bc0.64711.29421.94132.58843.23552.875812.689582.66555MIN: 2.35MIN: 2.13MIN: 2.081. (CXX) g++ options: -O3 -march=native -fopenmp -msse4.1 -fPIC -pie -ldl -lpthread

Phoronix Test Suite v10.8.5