Intel Xeon Platinum 8490H Linux Benchmarks Tests for a future article on Phoronix of Sapphire Rapids launch day review using the Intel Xeon Platinum 8490H processors. More benchmarks to come by Michael Larabel.
HTML result view exported from: https://openbenchmarking.org/result/2301109-PTS-SPRREVIE33&sgm=1&rmm=EPYC+7713%2CEPYC+7713+2P%2CEPYC+7763%2CEPYC+7763+2P%2CEPYC+7773X%2CEPYC+7773X+2P%2CEPYC+9374F%2CEPYC+9554%2CEPYC+9554+2P%2CEPYC+9654%2CEPYC+9654+2P%2CXeon+Platinum+8362%2CXeon+Platinum+8362+2P%2CXeon+Platinum+8380%2CXeon+Platinum+8380+2P%2CXeon+Platinum+8490H+2P&ppt=D&sgm=1&rmm=EPYC+7713%2CEPYC+7713+2P%2CEPYC+7763%2CEPYC+7763+2P%2CEPYC+7773X%2CEPYC+7773X+2P%2CEPYC+9374F+2P%2CEPYC+9554%2CEPYC+9554+2P%2CEPYC+9654%2CEPYC+9654+2P%2CXeon+Platinum+8362%2CXeon+Platinum+8362+2P%2CXeon+Platinum+8380%2CXeon+Platinum+8380+2P%2CXeon+Platinum+8490H+2P&ppt=D&sgm=1&rmm=EPYC+7713%2CEPYC+7713+2P%2CEPYC+7763%2CEPYC+7763+2P%2CEPYC+7773X%2CEPYC+7773X+2P%2CEPYC+9374F%2CEPYC+9554%2CEPYC+9554+2P%2CEPYC+9654%2CEPYC+9654+2P%2CXeon+Platinum+8362%2CXeon+Platinum+8362+2P%2CXeon+Platinum+8380%2CXeon+Platinum+8380+2P%2CXeon+Platinum+8490H+2P&ppt=D&sgm=1&rmm=EPYC+7713%2CEPYC+7713+2P%2CEPYC+7763%2CEPYC+7763+2P%2CEPYC+7773X%2CEPYC+7773X+2P%2CEPYC+9374F%2CEPYC+9374F+2P%2CEPYC+9554%2CEPYC+9554+2P%2CEPYC+9654%2CEPYC+9654+2P%2CXeon+Platinum+8362%2CXeon+Platinum+8362+2P%2CXeon+Platinum+8380%2CXeon+Platinum+8380+2P&ppt=D&stis=bWFjaGluZS1sZWFybmluZw&sgm=1&rmm=EPYC+7713%2CEPYC+7713+2P%2CEPYC+7763%2CEPYC+7763+2P%2CEPYC+7773X%2CEPYC+7773X+2P%2CEPYC+9374F%2CEPYC+9374F+2P%2CEPYC+9554%2CEPYC+9554+2P%2CEPYC+9654%2CEPYC+9654+2P%2CXeon+Platinum+8362%2CXeon+Platinum+8362+2P%2CXeon+Platinum+8380%2CXeon+Platinum+8380+2P&ppt=D .
Intel Xeon Platinum 8490H Linux Benchmarks Processor Motherboard Chipset Memory Disk Graphics Monitor Network OS Kernel Desktop Display Server Vulkan Compiler File-System Screen Resolution Xeon Platinum 8490H Xeon Platinum 8490H 2P Intel Xeon Platinum 8490H @ 3.50GHz (60 Cores / 120 Threads) Quanta Cloud S6Q-MB-MPS (3A10.uh BIOS) Intel Device 1bce 512GB 3841GB Micron_9300_MTFDHAL3T8TDP ASPEED VGA HDMI 4 x Intel E810-C for QSFP Ubuntu 22.10 6.0.0-060000rc3daily20220904-generic (x86_64) GNOME Shell X Server 1.21.1.3 1.3.211 GCC 12.2.0 ext4 1920x1080 2 x Intel Xeon Platinum 8490H @ 3.50GHz (120 Cores / 240 Threads) 1008GB 4 x Intel E810-C for QSFP + 2 x Intel X710 for 10GBASE-T OpenBenchmarking.org Kernel Details - Transparent Huge Pages: madvise Compiler Details - --build=x86_64-linux-gnu --disable-vtable-verify --disable-werror --enable-cet --enable-checking=release --enable-clocale=gnu --enable-default-pie --enable-gnu-unique-object --enable-languages=c,ada,c++,go,d,fortran,objc,obj-c++,m2 --enable-libphobos-checking=release --enable-libstdcxx-debug --enable-libstdcxx-time=yes --enable-multiarch --enable-multilib --enable-nls --enable-objc-gc=auto --enable-offload-defaulted --enable-offload-targets=nvptx-none=/build/gcc-12-Wbc0TK/gcc-12-12.2.0/debian/tmp-nvptx/usr,amdgcn-amdhsa=/build/gcc-12-Wbc0TK/gcc-12-12.2.0/debian/tmp-gcn/usr --enable-plugin --enable-shared --enable-threads=posix --host=x86_64-linux-gnu --program-prefix=x86_64-linux-gnu- --target=x86_64-linux-gnu --with-abi=m64 --with-arch-32=i686 --with-default-libstdcxx-abi=new --with-gcc-major-version-only --with-multilib-list=m32,m64,mx32 --with-target-system-zlib=auto --with-tune=generic --without-cuda-driver -v Processor Details - Scaling Governor: intel_pstate performance (EPP: performance) - CPU Microcode: 0x2b0000c0 Java Details - OpenJDK Runtime Environment (build 11.0.16+8-post-Ubuntu-0ubuntu1) Python Details - Python 3.10.6 Security Details - itlb_multihit: Not affected + l1tf: Not affected + mds: Not affected + meltdown: Not affected + mmio_stale_data: Not affected + retbleed: Not affected + spec_store_bypass: Mitigation of SSB disabled via prctl + spectre_v1: Mitigation of usercopy/swapgs barriers and __user pointer sanitization + spectre_v2: Mitigation of Enhanced IBRS IBPB: conditional RSB filling PBRSB-eIBRS: SW sequence + srbds: Not affected + tsx_async_abort: Not affected
Intel Xeon Platinum 8490H Linux Benchmarks openvino: Face Detection FP16 - CPU openvino: Face Detection FP16 - CPU openvino: Face Detection FP16-INT8 - CPU openvino: Age Gender Recognition Retail 0013 FP16 - CPU openvino: Age Gender Recognition Retail 0013 FP16 - CPU openvino: Person Detection FP16 - CPU openvino: Person Detection FP16 - CPU openvino: Person Detection FP32 - CPU openvino: Person Detection FP32 - CPU openvino: Weld Porosity Detection FP16-INT8 - CPU openvino: Weld Porosity Detection FP16 - CPU openvino: Weld Porosity Detection FP16 - CPU openvino: Person Vehicle Bike Detection FP16 - CPU openvino: Machine Translation EN To DE FP16 - CPU openvino: Machine Translation EN To DE FP16 - CPU lczero: Eigen onednn: Convolution Batch Shapes Auto - f32 - CPU onednn: Convolution Batch Shapes Auto - u8s8f32 - CPU onednn: Deconvolution Batch shapes_3d - f32 - CPU onednn: Convolution Batch Shapes Auto - f32 - CPU onednn: Convolution Batch Shapes Auto - u8s8f32 - CPU onednn: Convolution Batch Shapes Auto - bf16bf16bf16 - CPU deepsparse: CV Classification, ResNet-50 ImageNet - Synchronous Single-Stream deepsparse: CV Classification, ResNet-50 ImageNet - Synchronous Single-Stream deepsparse: CV Classification, ResNet-50 ImageNet - Asynchronous Multi-Stream deepsparse: NLP Token Classification, BERT base uncased conll2003 - Asynchronous Multi-Stream deepsparse: NLP Question Answering, BERT base uncased SQuaD 12layer Pruned90 - Asynchronous Multi-Stream deepsparse: CV Detection,YOLOv5s COCO - Asynchronous Multi-Stream deepsparse: NLP Document Classification, oBERT base uncased on IMDB - Asynchronous Multi-Stream Xeon Platinum 8490H Xeon Platinum 8490H 2P 70.58 424.67 150.87 82929.06 0.62 25.55 1167.83 25.53 1169.15 16703.64 7839.51 7.52 3656.94 448.59 66.66 10513 0.636394 0.492516 1.10812 0.645779 0.519451 0.281812 279.4595 3.5734 769.7793 47.4454 192.9925 318.9325 47.2877 197.81 151.49 376.67 152169.33 0.45 40.14 744.65 40.08 745.86 45149.75 22725.25 5.16 7533.49 719.22 41.51 11976 0.435654 0.267939 0.744021 0.431788 0.262114 0.264160 281.8155 3.5434 1499.5004 92.1557 364.7952 594.0229 92.1671 OpenBenchmarking.org
OpenVINO Model: Face Detection FP16 - Device: CPU OpenBenchmarking.org FPS, More Is Better OpenVINO 2022.2.dev Model: Face Detection FP16 - Device: CPU Xeon Platinum 8490H Xeon Platinum 8490H 2P 40 80 120 160 200 SE +/- 0.05, N = 3 SE +/- 0.12, N = 3 70.58 197.81 1. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -flto -shared
OpenVINO Model: Face Detection FP16 - Device: CPU OpenBenchmarking.org ms, Fewer Is Better OpenVINO 2022.2.dev Model: Face Detection FP16 - Device: CPU Xeon Platinum 8490H Xeon Platinum 8490H 2P 90 180 270 360 450 SE +/- 0.29, N = 3 SE +/- 0.09, N = 3 424.67 151.49 MIN: 324.89 / MAX: 491.29 MIN: 123.14 / MAX: 273.7 1. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -flto -shared
OpenVINO CPU Power Consumption Monitor Min Avg Max Xeon Platinum 8490H 80 324 377 Xeon Platinum 8490H 2P 206 645 752 OpenBenchmarking.org Watts, Fewer Is Better OpenVINO 2022.2.dev CPU Power Consumption Monitor 200 400 600 800 1000
OpenVINO Model: Face Detection FP16-INT8 - Device: CPU OpenBenchmarking.org FPS, More Is Better OpenVINO 2022.2.dev Model: Face Detection FP16-INT8 - Device: CPU Xeon Platinum 8490H Xeon Platinum 8490H 2P 80 160 240 320 400 SE +/- 0.45, N = 3 SE +/- 0.66, N = 3 150.87 376.67 1. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -flto -shared
OpenVINO Model: Age Gender Recognition Retail 0013 FP16 - Device: CPU OpenBenchmarking.org FPS, More Is Better OpenVINO 2022.2.dev Model: Age Gender Recognition Retail 0013 FP16 - Device: CPU Xeon Platinum 8490H Xeon Platinum 8490H 2P 30K 60K 90K 120K 150K SE +/- 111.96, N = 3 SE +/- 667.44, N = 3 82929.06 152169.33 1. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -flto -shared
OpenVINO Model: Age Gender Recognition Retail 0013 FP16 - Device: CPU OpenBenchmarking.org ms, Fewer Is Better OpenVINO 2022.2.dev Model: Age Gender Recognition Retail 0013 FP16 - Device: CPU Xeon Platinum 8490H Xeon Platinum 8490H 2P 0.1395 0.279 0.4185 0.558 0.6975 SE +/- 0.00, N = 3 SE +/- 0.00, N = 3 0.62 0.45 MIN: 0.31 / MAX: 23.57 MIN: 0.33 / MAX: 78.34 1. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -flto -shared
OpenVINO CPU Power Consumption Monitor Min Avg Max Xeon Platinum 8490H 83 331 376 Xeon Platinum 8490H 2P 205 646 698 OpenBenchmarking.org Watts, Fewer Is Better OpenVINO 2022.2.dev CPU Power Consumption Monitor 200 400 600 800 1000
OpenVINO Model: Person Detection FP16 - Device: CPU OpenBenchmarking.org FPS, More Is Better OpenVINO 2022.2.dev Model: Person Detection FP16 - Device: CPU Xeon Platinum 8490H Xeon Platinum 8490H 2P 9 18 27 36 45 SE +/- 0.05, N = 3 SE +/- 0.10, N = 3 25.55 40.14 1. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -flto -shared
OpenVINO Model: Person Detection FP16 - Device: CPU OpenBenchmarking.org ms, Fewer Is Better OpenVINO 2022.2.dev Model: Person Detection FP16 - Device: CPU Xeon Platinum 8490H Xeon Platinum 8490H 2P 300 600 900 1200 1500 SE +/- 2.53, N = 3 SE +/- 2.09, N = 3 1167.83 744.65 MIN: 600.36 / MAX: 1656.84 MIN: 515.46 / MAX: 1463.52 1. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -flto -shared
OpenVINO CPU Power Consumption Monitor Min Avg Max Xeon Platinum 8490H 82 323 369 Xeon Platinum 8490H 2P 208 627 703 OpenBenchmarking.org Watts, Fewer Is Better OpenVINO 2022.2.dev CPU Power Consumption Monitor 200 400 600 800 1000
OpenVINO Model: Person Detection FP32 - Device: CPU OpenBenchmarking.org FPS, More Is Better OpenVINO 2022.2.dev Model: Person Detection FP32 - Device: CPU Xeon Platinum 8490H Xeon Platinum 8490H 2P 9 18 27 36 45 SE +/- 0.01, N = 3 SE +/- 0.45, N = 3 25.53 40.08 1. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -flto -shared
OpenVINO Model: Person Detection FP32 - Device: CPU OpenBenchmarking.org ms, Fewer Is Better OpenVINO 2022.2.dev Model: Person Detection FP32 - Device: CPU Xeon Platinum 8490H Xeon Platinum 8490H 2P 300 600 900 1200 1500 SE +/- 0.26, N = 3 SE +/- 8.26, N = 3 1169.15 745.86 MIN: 870.58 / MAX: 1651.76 MIN: 475.57 / MAX: 1408.18 1. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -flto -shared
OpenVINO CPU Power Consumption Monitor Min Avg Max Xeon Platinum 8490H 81 326 366 Xeon Platinum 8490H 2P 201 633 710 OpenBenchmarking.org Watts, Fewer Is Better OpenVINO 2022.2.dev CPU Power Consumption Monitor 200 400 600 800 1000
OpenVINO Model: Weld Porosity Detection FP16-INT8 - Device: CPU OpenBenchmarking.org FPS, More Is Better OpenVINO 2022.2.dev Model: Weld Porosity Detection FP16-INT8 - Device: CPU Xeon Platinum 8490H Xeon Platinum 8490H 2P 10K 20K 30K 40K 50K SE +/- 9.01, N = 3 SE +/- 54.08, N = 3 16703.64 45149.75 1. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -flto -shared
OpenVINO Model: Weld Porosity Detection FP16 - Device: CPU OpenBenchmarking.org FPS, More Is Better OpenVINO 2022.2.dev Model: Weld Porosity Detection FP16 - Device: CPU Xeon Platinum 8490H Xeon Platinum 8490H 2P 5K 10K 15K 20K 25K SE +/- 60.72, N = 10 SE +/- 252.27, N = 4 7839.51 22725.25 1. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -flto -shared
OpenVINO Model: Weld Porosity Detection FP16 - Device: CPU OpenBenchmarking.org ms, Fewer Is Better OpenVINO 2022.2.dev Model: Weld Porosity Detection FP16 - Device: CPU Xeon Platinum 8490H Xeon Platinum 8490H 2P 2 4 6 8 10 SE +/- 0.05, N = 10 SE +/- 0.05, N = 4 7.52 5.16 MIN: 2.66 / MAX: 31.82 MIN: 4.27 / MAX: 91.31 1. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -flto -shared
OpenVINO CPU Power Consumption Monitor Min Avg Max Xeon Platinum 8490H 79 332 377 Xeon Platinum 8490H 2P 200 661 754 OpenBenchmarking.org Watts, Fewer Is Better OpenVINO 2022.2.dev CPU Power Consumption Monitor 200 400 600 800 1000
OpenVINO Model: Person Vehicle Bike Detection FP16 - Device: CPU OpenBenchmarking.org FPS, More Is Better OpenVINO 2022.2.dev Model: Person Vehicle Bike Detection FP16 - Device: CPU Xeon Platinum 8490H Xeon Platinum 8490H 2P 1600 3200 4800 6400 8000 SE +/- 5.03, N = 3 SE +/- 17.07, N = 3 3656.94 7533.49 1. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -flto -shared
OpenVINO Model: Machine Translation EN To DE FP16 - Device: CPU OpenBenchmarking.org FPS, More Is Better OpenVINO 2022.2.dev Model: Machine Translation EN To DE FP16 - Device: CPU Xeon Platinum 8490H Xeon Platinum 8490H 2P 160 320 480 640 800 SE +/- 0.03, N = 3 SE +/- 2.72, N = 3 448.59 719.22 1. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -flto -shared
OpenVINO Model: Machine Translation EN To DE FP16 - Device: CPU OpenBenchmarking.org ms, Fewer Is Better OpenVINO 2022.2.dev Model: Machine Translation EN To DE FP16 - Device: CPU Xeon Platinum 8490H Xeon Platinum 8490H 2P 15 30 45 60 75 SE +/- 0.01, N = 3 SE +/- 0.15, N = 3 66.66 41.51 MIN: 23.93 / MAX: 106.71 MIN: 27.82 / MAX: 338.88 1. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -flto -shared
OpenVINO CPU Power Consumption Monitor Min Avg Max Xeon Platinum 8490H 82 328 376 Xeon Platinum 8490H 2P 202 655 749 OpenBenchmarking.org Watts, Fewer Is Better OpenVINO 2022.2.dev CPU Power Consumption Monitor 200 400 600 800 1000
LeelaChessZero Backend: Eigen OpenBenchmarking.org Nodes Per Second, More Is Better LeelaChessZero 0.28 Backend: Eigen Xeon Platinum 8490H Xeon Platinum 8490H 2P 3K 6K 9K 12K 15K SE +/- 64.57, N = 3 SE +/- 76.29, N = 3 10513 11976 1. (CXX) g++ options: -flto -pthread
LeelaChessZero Backend: Eigen OpenBenchmarking.org Nodes Per Second Per Watt, More Is Better LeelaChessZero 0.28 Backend: Eigen Xeon Platinum 8490H Xeon Platinum 8490H 2P 7 14 21 28 35 31.74 22.12
LeelaChessZero CPU Power Consumption Monitor Min Avg Max Xeon Platinum 8490H 85 331 379 Xeon Platinum 8490H 2P 203 541 740 OpenBenchmarking.org Watts, Fewer Is Better LeelaChessZero 0.28 CPU Power Consumption Monitor 200 400 600 800 1000
oneDNN Harness: Convolution Batch Shapes Auto - Data Type: f32 - Engine: CPU OpenBenchmarking.org ms, Fewer Is Better oneDNN 2.6 Harness: Convolution Batch Shapes Auto - Data Type: f32 - Engine: CPU Xeon Platinum 8490H Xeon Platinum 8490H 2P 0.1432 0.2864 0.4296 0.5728 0.716 SE +/- 0.000967, N = 7 SE +/- 0.002329, N = 7 0.636394 0.435654 MIN: 0.54 MIN: 0.38 1. (CXX) g++ options: -O3 -march=native -fopenmp -msse4.1 -fPIC -std=c++11 -pie -ldl -lpthread
oneDNN CPU Power Consumption Monitor Min Avg Max Xeon Platinum 8490H 80 238 377 Xeon Platinum 8490H 2P 201 477 753 OpenBenchmarking.org Watts, Fewer Is Better oneDNN 2.6 CPU Power Consumption Monitor 200 400 600 800 1000
oneDNN Harness: Convolution Batch Shapes Auto - Data Type: u8s8f32 - Engine: CPU OpenBenchmarking.org ms, Fewer Is Better oneDNN 2.6 Harness: Convolution Batch Shapes Auto - Data Type: u8s8f32 - Engine: CPU Xeon Platinum 8490H Xeon Platinum 8490H 2P 0.1108 0.2216 0.3324 0.4432 0.554 SE +/- 0.000571, N = 7 SE +/- 0.001562, N = 7 0.492516 0.267939 MIN: 0.42 MIN: 0.2 1. (CXX) g++ options: -O3 -march=native -fopenmp -msse4.1 -fPIC -std=c++11 -pie -ldl -lpthread
oneDNN CPU Power Consumption Monitor Min Avg Max Xeon Platinum 8490H 83 249 377 Xeon Platinum 8490H 2P 196 497 735 OpenBenchmarking.org Watts, Fewer Is Better oneDNN 2.6 CPU Power Consumption Monitor 200 400 600 800 1000
oneDNN Harness: Deconvolution Batch shapes_3d - Data Type: f32 - Engine: CPU OpenBenchmarking.org ms, Fewer Is Better oneDNN 2.6 Harness: Deconvolution Batch shapes_3d - Data Type: f32 - Engine: CPU Xeon Platinum 8490H Xeon Platinum 8490H 2P 0.2493 0.4986 0.7479 0.9972 1.2465 SE +/- 0.000893, N = 9 SE +/- 0.001670, N = 9 1.108120 0.744021 MIN: 1.08 MIN: 0.67 1. (CXX) g++ options: -O3 -march=native -fopenmp -msse4.1 -fPIC -std=c++11 -pie -ldl -lpthread
oneDNN CPU Power Consumption Monitor Min Avg Max Xeon Platinum 8490H 83 208 368 Xeon Platinum 8490H 2P 202 432 729 OpenBenchmarking.org Watts, Fewer Is Better oneDNN 2.6 CPU Power Consumption Monitor 200 400 600 800 1000
oneDNN Harness: Convolution Batch Shapes Auto - Data Type: f32 - Engine: CPU OpenBenchmarking.org ms, Fewer Is Better oneDNN 2.7 Harness: Convolution Batch Shapes Auto - Data Type: f32 - Engine: CPU Xeon Platinum 8490H Xeon Platinum 8490H 2P 0.1453 0.2906 0.4359 0.5812 0.7265 SE +/- 0.000493, N = 7 SE +/- 0.002510, N = 7 0.645779 0.431788 MIN: 0.55 MIN: 0.38 1. (CXX) g++ options: -O3 -march=native -fopenmp -msse4.1 -fPIC -pie -ldl -lpthread
oneDNN CPU Power Consumption Monitor Min Avg Max Xeon Platinum 8490H 82 238 378 Xeon Platinum 8490H 2P 201 479 752 OpenBenchmarking.org Watts, Fewer Is Better oneDNN 2.7 CPU Power Consumption Monitor 200 400 600 800 1000
oneDNN Harness: Convolution Batch Shapes Auto - Data Type: u8s8f32 - Engine: CPU OpenBenchmarking.org ms, Fewer Is Better oneDNN 2.7 Harness: Convolution Batch Shapes Auto - Data Type: u8s8f32 - Engine: CPU Xeon Platinum 8490H Xeon Platinum 8490H 2P 0.1169 0.2338 0.3507 0.4676 0.5845 SE +/- 0.001949, N = 7 SE +/- 0.002009, N = 10 0.519451 0.262114 MIN: 0.46 MIN: 0.2 1. (CXX) g++ options: -O3 -march=native -fopenmp -msse4.1 -fPIC -pie -ldl -lpthread
oneDNN CPU Power Consumption Monitor Min Avg Max Xeon Platinum 8490H 84 250 377 Xeon Platinum 8490H 2P 194 502 738 OpenBenchmarking.org Watts, Fewer Is Better oneDNN 2.7 CPU Power Consumption Monitor 200 400 600 800 1000
oneDNN Harness: Convolution Batch Shapes Auto - Data Type: bf16bf16bf16 - Engine: CPU OpenBenchmarking.org ms, Fewer Is Better oneDNN 2.7 Harness: Convolution Batch Shapes Auto - Data Type: bf16bf16bf16 - Engine: CPU Xeon Platinum 8490H Xeon Platinum 8490H 2P 0.0634 0.1268 0.1902 0.2536 0.317 SE +/- 0.002592, N = 15 SE +/- 0.001768, N = 7 0.281812 0.264160 MIN: 0.25 MIN: 0.22 1. (CXX) g++ options: -O3 -march=native -fopenmp -msse4.1 -fPIC -pie -ldl -lpthread
oneDNN CPU Power Consumption Monitor Min Avg Max Xeon Platinum 8490H 82 241 372 Xeon Platinum 8490H 2P 199 473 694 OpenBenchmarking.org Watts, Fewer Is Better oneDNN 2.7 CPU Power Consumption Monitor 200 400 600 800 1000
Neural Magic DeepSparse Model: CV Classification, ResNet-50 ImageNet - Scenario: Synchronous Single-Stream OpenBenchmarking.org items/sec, More Is Better Neural Magic DeepSparse 1.1 Model: CV Classification, ResNet-50 ImageNet - Scenario: Synchronous Single-Stream Xeon Platinum 8490H Xeon Platinum 8490H 2P 60 120 180 240 300 SE +/- 0.50, N = 3 SE +/- 0.80, N = 3 279.46 281.82
Neural Magic DeepSparse Model: CV Classification, ResNet-50 ImageNet - Scenario: Synchronous Single-Stream OpenBenchmarking.org ms/batch, Fewer Is Better Neural Magic DeepSparse 1.1 Model: CV Classification, ResNet-50 ImageNet - Scenario: Synchronous Single-Stream Xeon Platinum 8490H Xeon Platinum 8490H 2P 0.804 1.608 2.412 3.216 4.02 SE +/- 0.0063, N = 3 SE +/- 0.0099, N = 3 3.5734 3.5434
Neural Magic DeepSparse CPU Power Consumption Monitor Min Avg Max Xeon Platinum 8490H 84.1 284.0 325.8 Xeon Platinum 8490H 2P 203.4 470.1 517.5 OpenBenchmarking.org Watts, Fewer Is Better Neural Magic DeepSparse 1.1 CPU Power Consumption Monitor 130 260 390 520 650
Neural Magic DeepSparse Model: CV Classification, ResNet-50 ImageNet - Scenario: Asynchronous Multi-Stream OpenBenchmarking.org items/sec, More Is Better Neural Magic DeepSparse 1.1 Model: CV Classification, ResNet-50 ImageNet - Scenario: Asynchronous Multi-Stream Xeon Platinum 8490H Xeon Platinum 8490H 2P 300 600 900 1200 1500 SE +/- 0.49, N = 3 SE +/- 2.06, N = 3 769.78 1499.50
Neural Magic DeepSparse Model: NLP Token Classification, BERT base uncased conll2003 - Scenario: Asynchronous Multi-Stream OpenBenchmarking.org items/sec, More Is Better Neural Magic DeepSparse 1.1 Model: NLP Token Classification, BERT base uncased conll2003 - Scenario: Asynchronous Multi-Stream Xeon Platinum 8490H Xeon Platinum 8490H 2P 20 40 60 80 100 SE +/- 0.06, N = 3 SE +/- 0.09, N = 3 47.45 92.16
Neural Magic DeepSparse Model: NLP Question Answering, BERT base uncased SQuaD 12layer Pruned90 - Scenario: Asynchronous Multi-Stream OpenBenchmarking.org items/sec, More Is Better Neural Magic DeepSparse 1.1 Model: NLP Question Answering, BERT base uncased SQuaD 12layer Pruned90 - Scenario: Asynchronous Multi-Stream Xeon Platinum 8490H Xeon Platinum 8490H 2P 80 160 240 320 400 SE +/- 0.12, N = 3 SE +/- 2.04, N = 3 192.99 364.80
Neural Magic DeepSparse Model: CV Detection,YOLOv5s COCO - Scenario: Asynchronous Multi-Stream OpenBenchmarking.org items/sec, More Is Better Neural Magic DeepSparse 1.1 Model: CV Detection,YOLOv5s COCO - Scenario: Asynchronous Multi-Stream Xeon Platinum 8490H Xeon Platinum 8490H 2P 130 260 390 520 650 SE +/- 0.14, N = 3 SE +/- 0.43, N = 3 318.93 594.02
Neural Magic DeepSparse Model: NLP Document Classification, oBERT base uncased on IMDB - Scenario: Asynchronous Multi-Stream OpenBenchmarking.org items/sec, More Is Better Neural Magic DeepSparse 1.1 Model: NLP Document Classification, oBERT base uncased on IMDB - Scenario: Asynchronous Multi-Stream Xeon Platinum 8490H Xeon Platinum 8490H 2P 20 40 60 80 100 SE +/- 0.15, N = 3 SE +/- 0.24, N = 3 47.29 92.17
Geometric Mean Of All Test Results Result Composite - Intel Xeon Platinum 8490H Linux Benchmarks OpenBenchmarking.org Geometric Mean, More Is Better Geometric Mean Of All Test Results Result Composite - Intel Xeon Platinum 8490H Linux Benchmarks Xeon Platinum 8490H Xeon Platinum 8490H 2P 40 80 120 160 200 110.54 190.28
Phoronix Test Suite v10.8.4