2 x Intel Xeon Platinum 8490H testing with a Quanta Cloud S6Q-MB-MPS (3A10.uh BIOS) and ASPEED on Ubuntu 23.04 via the Phoronix Test Suite.
Compare your own system(s) to this result file with the
Phoronix Test Suite by running the command:
phoronix-test-suite benchmark 2302115-NE-OONXSAPPH22 OONX Sapphire Rapids - Phoronix Test Suite OONX Sapphire Rapids 2 x Intel Xeon Platinum 8490H testing with a Quanta Cloud S6Q-MB-MPS (3A10.uh BIOS) and ASPEED on Ubuntu 23.04 via the Phoronix Test Suite.
HTML result view exported from: https://openbenchmarking.org/result/2302115-NE-OONXSAPPH22&rdt&gru .
OONX Sapphire Rapids Processor Motherboard Chipset Memory Disk Graphics Monitor Network OS Kernel Desktop Display Server Compiler File-System Screen Resolution ONNX Runtime 1.14 ONNX Runtime 1.14 - No AMX ONNX Runtime 1.14 use_dnnl 2 x Intel Xeon Platinum 8490H @ 3.50GHz (120 Cores / 240 Threads) Quanta Cloud S6Q-MB-MPS (3A10.uh BIOS) Intel Device 1bce 1008GB 2 x 1920GB SAMSUNG MZWLJ1T9HBJR-00007 + 960GB INTEL SSDSC2KG96 ASPEED VGA HDMI 4 x Intel E810-C for QSFP + 2 x Intel X710 for 10GBASE-T Ubuntu 23.04 5.19.0-21-generic (x86_64) GNOME Shell 43.2 X Server 1.21.1.6 GCC 12.2.0 ext4 1920x1080 OpenBenchmarking.org Kernel Details - Transparent Huge Pages: madvise Compiler Details - --build=x86_64-linux-gnu --disable-vtable-verify --disable-werror --enable-cet --enable-checking=release --enable-clocale=gnu --enable-default-pie --enable-gnu-unique-object --enable-languages=c,ada,c++,go,d,fortran,objc,obj-c++,m2 --enable-libphobos-checking=release --enable-libstdcxx-debug --enable-libstdcxx-time=yes --enable-multiarch --enable-multilib --enable-nls --enable-objc-gc=auto --enable-offload-defaulted --enable-offload-targets=nvptx-none=/build/gcc-12-AKimc9/gcc-12-12.2.0/debian/tmp-nvptx/usr,amdgcn-amdhsa=/build/gcc-12-AKimc9/gcc-12-12.2.0/debian/tmp-gcn/usr --enable-plugin --enable-shared --enable-threads=posix --host=x86_64-linux-gnu --program-prefix=x86_64-linux-gnu- --target=x86_64-linux-gnu --with-abi=m64 --with-arch-32=i686 --with-default-libstdcxx-abi=new --with-gcc-major-version-only --with-multilib-list=m32,m64,mx32 --with-target-system-zlib=auto --with-tune=generic --without-cuda-driver -v Processor Details - Scaling Governor: intel_pstate performance (EPP: performance) - CPU Microcode: 0x2b0000c0 Python Details - Python 3.11.1 Security Details - itlb_multihit: Not affected + l1tf: Not affected + mds: Not affected + meltdown: Not affected + mmio_stale_data: Not affected + retbleed: Not affected + spec_store_bypass: Mitigation of SSB disabled via prctl + spectre_v1: Mitigation of usercopy/swapgs barriers and __user pointer sanitization + spectre_v2: Mitigation of Enhanced IBRS IBPB: conditional RSB filling PBRSB-eIBRS: SW sequence + srbds: Not affected + tsx_async_abort: Not affected
OONX Sapphire Rapids onnx: yolov4 - CPU - Standard onnx: yolov4 - CPU - Parallel onnx: fcn-resnet101-11 - CPU - Standard onnx: fcn-resnet101-11 - CPU - Parallel onnx: super-resolution-10 - CPU - Standard onnx: super-resolution-10 - CPU - Parallel onnx: bertsquad-12 - CPU - Standard onnx: bertsquad-12 - CPU - Parallel onnx: GPT-2 - CPU - Standard onnx: GPT-2 - CPU - Parallel onnx: ArcFace ResNet-100 - CPU - Standard onnx: ArcFace ResNet-100 - CPU - Parallel onnx: ResNet50 v1-12-int8 - CPU - Standard onnx: ResNet50 v1-12-int8 - CPU - Parallel onnx: CaffeNet 12-int8 - CPU - Standard onnx: CaffeNet 12-int8 - CPU - Parallel onnx: Faster R-CNN R-50-FPN-int8 - CPU - Standard onnx: Faster R-CNN R-50-FPN-int8 - CPU - Parallel onnx: yolov4 - CPU - Standard onnx: yolov4 - CPU - Parallel onnx: fcn-resnet101-11 - CPU - Standard onnx: fcn-resnet101-11 - CPU - Parallel onnx: super-resolution-10 - CPU - Standard onnx: super-resolution-10 - CPU - Parallel onnx: bertsquad-12 - CPU - Standard onnx: bertsquad-12 - CPU - Parallel onnx: GPT-2 - CPU - Standard onnx: GPT-2 - CPU - Parallel onnx: ArcFace ResNet-100 - CPU - Standard onnx: ArcFace ResNet-100 - CPU - Parallel onnx: ResNet50 v1-12-int8 - CPU - Standard onnx: ResNet50 v1-12-int8 - CPU - Parallel onnx: CaffeNet 12-int8 - CPU - Standard onnx: CaffeNet 12-int8 - CPU - Parallel onnx: Faster R-CNN R-50-FPN-int8 - CPU - Standard onnx: Faster R-CNN R-50-FPN-int8 - CPU - Parallel ONNX Runtime 1.14 ONNX Runtime 1.14 - No AMX ONNX Runtime 1.14 use_dnnl 11.5378 9.36451 10.00997 3.03253 209.662 179.895 14.9354 15.0267 190.019 153.596 33.9663 27.2147 177.925 162.477 684.146 589.197 39.7690 32.8994 86.6683 106.799 100.2871 329.874 4.76912 5.55742 66.9705 66.5490 5.26053 6.50527 29.4406 36.7433 5.61969 6.15564 1.46149 1.69517 25.1440 30.4046 11.26627 9.47631 10.4618 2.95924 209.703 180.391 15.1688 14.9954 195.149 158.305 88.9286 105.531 95.6002 337.921 4.76804 5.54240 65.9243 66.6888 5.13582 6.31109 11.2020 9.56208 9.93450 3.00654 211.423 178.529 15.1855 14.9979 194.522 150.860 32.6092 27.4259 175.674 164.899 695.777 590.795 40.0070 31.7784 89.3915 104.582 101.0160 333.098 4.72937 5.60021 65.8822 66.6855 5.15414 6.62309 30.6762 36.4666 5.69175 6.06317 1.43703 1.69067 24.9956 31.4694 OpenBenchmarking.org
CPU Temperature Monitor Phoronix Test Suite System Monitoring OpenBenchmarking.org Celsius CPU Temperature Monitor Phoronix Test Suite System Monitoring ONNX Runtime 1.14 ONNX Runtime 1.14 use_dnnl 11 22 33 44 55 Min: 28 / Avg: 47.63 / Max: 55 Min: 32 / Avg: 48.36 / Max: 55
CPU Peak Freq (Highest CPU Core Frequency) Monitor Phoronix Test Suite System Monitoring OpenBenchmarking.org Megahertz CPU Peak Freq (Highest CPU Core Frequency) Monitor Phoronix Test Suite System Monitoring ONNX Runtime 1.14 ONNX Runtime 1.14 use_dnnl 600 1200 1800 2400 3000 Min: 1900 / Avg: 3026.38 / Max: 3666 Min: 1900 / Avg: 3073.43 / Max: 3677
CPU Power Consumption Monitor Phoronix Test Suite System Monitoring OpenBenchmarking.org Watts CPU Power Consumption Monitor Phoronix Test Suite System Monitoring ONNX Runtime 1.14 ONNX Runtime 1.14 use_dnnl 120 240 360 480 600 Min: 122.34 / Avg: 577.11 / Max: 709.46 Min: 104.64 / Avg: 571.11 / Max: 709.06
ONNX Runtime Model: yolov4 - Device: CPU - Executor: Standard OpenBenchmarking.org Inferences Per Second, More Is Better ONNX Runtime 1.14 Model: yolov4 - Device: CPU - Executor: Standard ONNX Runtime 1.14 ONNX Runtime 1.14 - No AMX ONNX Runtime 1.14 use_dnnl 3 6 9 12 15 SE +/- 0.02, N = 3 SE +/- 0.13, N = 15 SE +/- 0.11, N = 15 11.54 11.27 11.20 1. (CXX) g++ options: -ffunction-sections -fdata-sections -march=native -mtune=native -O3 -flto=auto -fno-fat-lto-objects -ldl -lrt
ONNX Runtime Model: yolov4 - Device: CPU - Executor: Parallel OpenBenchmarking.org Inferences Per Second, More Is Better ONNX Runtime 1.14 Model: yolov4 - Device: CPU - Executor: Parallel ONNX Runtime 1.14 ONNX Runtime 1.14 - No AMX ONNX Runtime 1.14 use_dnnl 3 6 9 12 15 SE +/- 0.08393, N = 3 SE +/- 0.06211, N = 3 SE +/- 0.05377, N = 3 9.36451 9.47631 9.56208 1. (CXX) g++ options: -ffunction-sections -fdata-sections -march=native -mtune=native -O3 -flto=auto -fno-fat-lto-objects -ldl -lrt
ONNX Runtime Model: fcn-resnet101-11 - Device: CPU - Executor: Standard OpenBenchmarking.org Inferences Per Second, More Is Better ONNX Runtime 1.14 Model: fcn-resnet101-11 - Device: CPU - Executor: Standard ONNX Runtime 1.14 ONNX Runtime 1.14 - No AMX ONNX Runtime 1.14 use_dnnl 3 6 9 12 15 SE +/- 0.15984, N = 15 SE +/- 0.10214, N = 3 SE +/- 0.15346, N = 15 10.00997 10.46180 9.93450 1. (CXX) g++ options: -ffunction-sections -fdata-sections -march=native -mtune=native -O3 -flto=auto -fno-fat-lto-objects -ldl -lrt
ONNX Runtime Model: fcn-resnet101-11 - Device: CPU - Executor: Parallel OpenBenchmarking.org Inferences Per Second, More Is Better ONNX Runtime 1.14 Model: fcn-resnet101-11 - Device: CPU - Executor: Parallel ONNX Runtime 1.14 ONNX Runtime 1.14 - No AMX ONNX Runtime 1.14 use_dnnl 0.6823 1.3646 2.0469 2.7292 3.4115 SE +/- 0.04052, N = 3 SE +/- 0.00278, N = 3 SE +/- 0.03052, N = 15 3.03253 2.95924 3.00654 1. (CXX) g++ options: -ffunction-sections -fdata-sections -march=native -mtune=native -O3 -flto=auto -fno-fat-lto-objects -ldl -lrt
ONNX Runtime Model: super-resolution-10 - Device: CPU - Executor: Standard OpenBenchmarking.org Inferences Per Second, More Is Better ONNX Runtime 1.14 Model: super-resolution-10 - Device: CPU - Executor: Standard ONNX Runtime 1.14 ONNX Runtime 1.14 - No AMX ONNX Runtime 1.14 use_dnnl 50 100 150 200 250 SE +/- 0.73, N = 3 SE +/- 0.42, N = 3 SE +/- 0.60, N = 3 209.66 209.70 211.42 1. (CXX) g++ options: -ffunction-sections -fdata-sections -march=native -mtune=native -O3 -flto=auto -fno-fat-lto-objects -ldl -lrt
ONNX Runtime Model: super-resolution-10 - Device: CPU - Executor: Parallel OpenBenchmarking.org Inferences Per Second, More Is Better ONNX Runtime 1.14 Model: super-resolution-10 - Device: CPU - Executor: Parallel ONNX Runtime 1.14 ONNX Runtime 1.14 - No AMX ONNX Runtime 1.14 use_dnnl 40 80 120 160 200 SE +/- 0.28, N = 3 SE +/- 1.01, N = 3 SE +/- 0.94, N = 3 179.90 180.39 178.53 1. (CXX) g++ options: -ffunction-sections -fdata-sections -march=native -mtune=native -O3 -flto=auto -fno-fat-lto-objects -ldl -lrt
ONNX Runtime Model: bertsquad-12 - Device: CPU - Executor: Standard OpenBenchmarking.org Inferences Per Second, More Is Better ONNX Runtime 1.14 Model: bertsquad-12 - Device: CPU - Executor: Standard ONNX Runtime 1.14 ONNX Runtime 1.14 - No AMX ONNX Runtime 1.14 use_dnnl 4 8 12 16 20 SE +/- 0.17, N = 3 SE +/- 0.04, N = 3 SE +/- 0.12, N = 9 14.94 15.17 15.19 1. (CXX) g++ options: -ffunction-sections -fdata-sections -march=native -mtune=native -O3 -flto=auto -fno-fat-lto-objects -ldl -lrt
ONNX Runtime Model: bertsquad-12 - Device: CPU - Executor: Parallel OpenBenchmarking.org Inferences Per Second, More Is Better ONNX Runtime 1.14 Model: bertsquad-12 - Device: CPU - Executor: Parallel ONNX Runtime 1.14 ONNX Runtime 1.14 - No AMX ONNX Runtime 1.14 use_dnnl 4 8 12 16 20 SE +/- 0.08, N = 3 SE +/- 0.09, N = 3 SE +/- 0.14, N = 3 15.03 15.00 15.00 1. (CXX) g++ options: -ffunction-sections -fdata-sections -march=native -mtune=native -O3 -flto=auto -fno-fat-lto-objects -ldl -lrt
ONNX Runtime Model: GPT-2 - Device: CPU - Executor: Standard OpenBenchmarking.org Inferences Per Second, More Is Better ONNX Runtime 1.14 Model: GPT-2 - Device: CPU - Executor: Standard ONNX Runtime 1.14 ONNX Runtime 1.14 - No AMX ONNX Runtime 1.14 use_dnnl 40 80 120 160 200 SE +/- 0.21, N = 3 SE +/- 3.06, N = 14 SE +/- 3.35, N = 13 190.02 195.15 194.52 1. (CXX) g++ options: -ffunction-sections -fdata-sections -march=native -mtune=native -O3 -flto=auto -fno-fat-lto-objects -ldl -lrt
ONNX Runtime Model: GPT-2 - Device: CPU - Executor: Parallel OpenBenchmarking.org Inferences Per Second, More Is Better ONNX Runtime 1.14 Model: GPT-2 - Device: CPU - Executor: Parallel ONNX Runtime 1.14 ONNX Runtime 1.14 - No AMX ONNX Runtime 1.14 use_dnnl 30 60 90 120 150 SE +/- 1.30, N = 3 SE +/- 1.03, N = 3 SE +/- 1.24, N = 3 153.60 158.31 150.86 1. (CXX) g++ options: -ffunction-sections -fdata-sections -march=native -mtune=native -O3 -flto=auto -fno-fat-lto-objects -ldl -lrt
ONNX Runtime Model: ArcFace ResNet-100 - Device: CPU - Executor: Standard OpenBenchmarking.org Inferences Per Second, More Is Better ONNX Runtime 1.14 Model: ArcFace ResNet-100 - Device: CPU - Executor: Standard ONNX Runtime 1.14 ONNX Runtime 1.14 use_dnnl 8 16 24 32 40 SE +/- 0.22, N = 3 SE +/- 0.46, N = 3 33.97 32.61 1. (CXX) g++ options: -ffunction-sections -fdata-sections -march=native -mtune=native -O3 -flto=auto -fno-fat-lto-objects -ldl -lrt
ONNX Runtime Model: ArcFace ResNet-100 - Device: CPU - Executor: Parallel OpenBenchmarking.org Inferences Per Second, More Is Better ONNX Runtime 1.14 Model: ArcFace ResNet-100 - Device: CPU - Executor: Parallel ONNX Runtime 1.14 ONNX Runtime 1.14 use_dnnl 6 12 18 24 30 SE +/- 0.06, N = 3 SE +/- 0.26, N = 3 27.21 27.43 1. (CXX) g++ options: -ffunction-sections -fdata-sections -march=native -mtune=native -O3 -flto=auto -fno-fat-lto-objects -ldl -lrt
ONNX Runtime Model: ResNet50 v1-12-int8 - Device: CPU - Executor: Standard OpenBenchmarking.org Inferences Per Second, More Is Better ONNX Runtime 1.14 Model: ResNet50 v1-12-int8 - Device: CPU - Executor: Standard ONNX Runtime 1.14 ONNX Runtime 1.14 use_dnnl 40 80 120 160 200 SE +/- 0.57, N = 3 SE +/- 0.56, N = 3 177.93 175.67 1. (CXX) g++ options: -ffunction-sections -fdata-sections -march=native -mtune=native -O3 -flto=auto -fno-fat-lto-objects -ldl -lrt
ONNX Runtime Model: ResNet50 v1-12-int8 - Device: CPU - Executor: Parallel OpenBenchmarking.org Inferences Per Second, More Is Better ONNX Runtime 1.14 Model: ResNet50 v1-12-int8 - Device: CPU - Executor: Parallel ONNX Runtime 1.14 ONNX Runtime 1.14 use_dnnl 40 80 120 160 200 SE +/- 1.61, N = 5 SE +/- 1.24, N = 3 162.48 164.90 1. (CXX) g++ options: -ffunction-sections -fdata-sections -march=native -mtune=native -O3 -flto=auto -fno-fat-lto-objects -ldl -lrt
ONNX Runtime Model: CaffeNet 12-int8 - Device: CPU - Executor: Standard OpenBenchmarking.org Inferences Per Second, More Is Better ONNX Runtime 1.14 Model: CaffeNet 12-int8 - Device: CPU - Executor: Standard ONNX Runtime 1.14 ONNX Runtime 1.14 use_dnnl 150 300 450 600 750 SE +/- 9.35, N = 3 SE +/- 9.80, N = 3 684.15 695.78 1. (CXX) g++ options: -ffunction-sections -fdata-sections -march=native -mtune=native -O3 -flto=auto -fno-fat-lto-objects -ldl -lrt
ONNX Runtime Model: CaffeNet 12-int8 - Device: CPU - Executor: Parallel OpenBenchmarking.org Inferences Per Second, More Is Better ONNX Runtime 1.14 Model: CaffeNet 12-int8 - Device: CPU - Executor: Parallel ONNX Runtime 1.14 ONNX Runtime 1.14 use_dnnl 130 260 390 520 650 SE +/- 3.23, N = 3 SE +/- 4.15, N = 3 589.20 590.80 1. (CXX) g++ options: -ffunction-sections -fdata-sections -march=native -mtune=native -O3 -flto=auto -fno-fat-lto-objects -ldl -lrt
ONNX Runtime Model: Faster R-CNN R-50-FPN-int8 - Device: CPU - Executor: Standard OpenBenchmarking.org Inferences Per Second, More Is Better ONNX Runtime 1.14 Model: Faster R-CNN R-50-FPN-int8 - Device: CPU - Executor: Standard ONNX Runtime 1.14 ONNX Runtime 1.14 use_dnnl 9 18 27 36 45 SE +/- 0.12, N = 3 SE +/- 0.24, N = 3 39.77 40.01 1. (CXX) g++ options: -ffunction-sections -fdata-sections -march=native -mtune=native -O3 -flto=auto -fno-fat-lto-objects -ldl -lrt
ONNX Runtime Model: Faster R-CNN R-50-FPN-int8 - Device: CPU - Executor: Parallel OpenBenchmarking.org Inferences Per Second, More Is Better ONNX Runtime 1.14 Model: Faster R-CNN R-50-FPN-int8 - Device: CPU - Executor: Parallel ONNX Runtime 1.14 ONNX Runtime 1.14 use_dnnl 8 16 24 32 40 SE +/- 0.46, N = 3 SE +/- 0.27, N = 3 32.90 31.78 1. (CXX) g++ options: -ffunction-sections -fdata-sections -march=native -mtune=native -O3 -flto=auto -fno-fat-lto-objects -ldl -lrt
ONNX Runtime CPU Peak Freq (Highest CPU Core Frequency) Monitor Min Avg Max ONNX Runtime 1.14 1900 2919 3512 ONNX Runtime 1.14 - No AMX 1900 2905 3511 ONNX Runtime 1.14 use_dnnl 1900 2911 3514 OpenBenchmarking.org Megahertz, More Is Better ONNX Runtime 1.14 CPU Peak Freq (Highest CPU Core Frequency) Monitor 1000 2000 3000 4000 5000
ONNX Runtime CPU Peak Freq (Highest CPU Core Frequency) Monitor Min Avg Max ONNX Runtime 1.14 1900 2904 3509 ONNX Runtime 1.14 - No AMX 1900 2917 3506 ONNX Runtime 1.14 use_dnnl 1900 2921 3515 OpenBenchmarking.org Megahertz, More Is Better ONNX Runtime 1.14 CPU Peak Freq (Highest CPU Core Frequency) Monitor 1000 2000 3000 4000 5000
ONNX Runtime CPU Peak Freq (Highest CPU Core Frequency) Monitor OpenBenchmarking.org Megahertz, More Is Better ONNX Runtime 1.14 CPU Peak Freq (Highest CPU Core Frequency) Monitor ONNX Runtime 1.14 ONNX Runtime 1.14 - No AMX ONNX Runtime 1.14 use_dnnl 600 1200 1800 2400 3000 Min: 1900 / Avg: 2752.96 / Max: 3514 Min: 1900 / Avg: 2751.37 / Max: 3515 Min: 1900 / Avg: 2770.59 / Max: 3511
ONNX Runtime CPU Peak Freq (Highest CPU Core Frequency) Monitor Min Avg Max ONNX Runtime 1.14 1900 2922 3551 ONNX Runtime 1.14 - No AMX 1900 2905 3508 ONNX Runtime 1.14 use_dnnl 1900 2916 3677 OpenBenchmarking.org Megahertz, More Is Better ONNX Runtime 1.14 CPU Peak Freq (Highest CPU Core Frequency) Monitor 1000 2000 3000 4000 5000
ONNX Runtime CPU Peak Freq (Highest CPU Core Frequency) Monitor Min Avg Max ONNX Runtime 1.14 1900 3457 3530 ONNX Runtime 1.14 - No AMX 1900 3450 3516 ONNX Runtime 1.14 use_dnnl 1900 3457 3517 OpenBenchmarking.org Megahertz, More Is Better ONNX Runtime 1.14 CPU Peak Freq (Highest CPU Core Frequency) Monitor 1000 2000 3000 4000 5000
ONNX Runtime CPU Peak Freq (Highest CPU Core Frequency) Monitor Min Avg Max ONNX Runtime 1.14 1900 3495 3515 ONNX Runtime 1.14 - No AMX 1900 3478 3517 ONNX Runtime 1.14 use_dnnl 1900 3489 3514 OpenBenchmarking.org Megahertz, More Is Better ONNX Runtime 1.14 CPU Peak Freq (Highest CPU Core Frequency) Monitor 1000 2000 3000 4000 5000
ONNX Runtime CPU Peak Freq (Highest CPU Core Frequency) Monitor Min Avg Max ONNX Runtime 1.14 1900 3460 3512 ONNX Runtime 1.14 - No AMX 1900 3470 3512 ONNX Runtime 1.14 use_dnnl 1900 3466 3516 OpenBenchmarking.org Megahertz, More Is Better ONNX Runtime 1.14 CPU Peak Freq (Highest CPU Core Frequency) Monitor 1000 2000 3000 4000 5000
ONNX Runtime CPU Peak Freq (Highest CPU Core Frequency) Monitor Min Avg Max ONNX Runtime 1.14 1900 3179 3512 ONNX Runtime 1.14 - No AMX 1900 3181 3510 ONNX Runtime 1.14 use_dnnl 1900 3179 3515 OpenBenchmarking.org Megahertz, More Is Better ONNX Runtime 1.14 CPU Peak Freq (Highest CPU Core Frequency) Monitor 1000 2000 3000 4000 5000
ONNX Runtime CPU Peak Freq (Highest CPU Core Frequency) Monitor Min Avg Max ONNX Runtime 1.14 1900 3482 3515 ONNX Runtime 1.14 - No AMX 1900 3486 3524 ONNX Runtime 1.14 use_dnnl 1900 3494 3520 OpenBenchmarking.org Megahertz, More Is Better ONNX Runtime 1.14 CPU Peak Freq (Highest CPU Core Frequency) Monitor 1000 2000 3000 4000 5000
ONNX Runtime CPU Peak Freq (Highest CPU Core Frequency) Monitor Min Avg Max ONNX Runtime 1.14 1900 3494 3516 ONNX Runtime 1.14 - No AMX 1900 3475 3510 ONNX Runtime 1.14 use_dnnl 1900 3478 3511 OpenBenchmarking.org Megahertz, More Is Better ONNX Runtime 1.14 CPU Peak Freq (Highest CPU Core Frequency) Monitor 1000 2000 3000 4000 5000
ONNX Runtime CPU Peak Freq (Highest CPU Core Frequency) Monitor Min Avg Max ONNX Runtime 1.14 2900 2935 3514 ONNX Runtime 1.14 use_dnnl 1900 2932 3512 OpenBenchmarking.org Megahertz, More Is Better ONNX Runtime 1.14 CPU Peak Freq (Highest CPU Core Frequency) Monitor 1000 2000 3000 4000 5000
ONNX Runtime CPU Peak Freq (Highest CPU Core Frequency) Monitor Min Avg Max ONNX Runtime 1.14 1900 2911 3514 ONNX Runtime 1.14 use_dnnl 1900 2929 3510 OpenBenchmarking.org Megahertz, More Is Better ONNX Runtime 1.14 CPU Peak Freq (Highest CPU Core Frequency) Monitor 1000 2000 3000 4000 5000
ONNX Runtime CPU Peak Freq (Highest CPU Core Frequency) Monitor Min Avg Max ONNX Runtime 1.14 1900 2904 3506 ONNX Runtime 1.14 use_dnnl 1900 2905 3503 OpenBenchmarking.org Megahertz, More Is Better ONNX Runtime 1.14 CPU Peak Freq (Highest CPU Core Frequency) Monitor 1000 2000 3000 4000 5000
ONNX Runtime CPU Peak Freq (Highest CPU Core Frequency) Monitor Min Avg Max ONNX Runtime 1.14 1900 2890 3666 ONNX Runtime 1.14 use_dnnl 1900 2889 3507 OpenBenchmarking.org Megahertz, More Is Better ONNX Runtime 1.14 CPU Peak Freq (Highest CPU Core Frequency) Monitor 1000 2000 3000 4000 5000
ONNX Runtime CPU Peak Freq (Highest CPU Core Frequency) Monitor Min Avg Max ONNX Runtime 1.14 1900 2908 3400 ONNX Runtime 1.14 use_dnnl 1900 2914 3501 OpenBenchmarking.org Megahertz, More Is Better ONNX Runtime 1.14 CPU Peak Freq (Highest CPU Core Frequency) Monitor 1000 2000 3000 4000 5000
ONNX Runtime CPU Peak Freq (Highest CPU Core Frequency) Monitor Min Avg Max ONNX Runtime 1.14 1900 2908 3500 ONNX Runtime 1.14 use_dnnl 1900 2903 3501 OpenBenchmarking.org Megahertz, More Is Better ONNX Runtime 1.14 CPU Peak Freq (Highest CPU Core Frequency) Monitor 1000 2000 3000 4000 5000
ONNX Runtime CPU Peak Freq (Highest CPU Core Frequency) Monitor Min Avg Max ONNX Runtime 1.14 1900 2982 3510 ONNX Runtime 1.14 use_dnnl 1900 2997 3509 OpenBenchmarking.org Megahertz, More Is Better ONNX Runtime 1.14 CPU Peak Freq (Highest CPU Core Frequency) Monitor 1000 2000 3000 4000 5000
ONNX Runtime CPU Peak Freq (Highest CPU Core Frequency) Monitor Min Avg Max ONNX Runtime 1.14 1900 3070 3525 ONNX Runtime 1.14 use_dnnl 1900 3034 3508 OpenBenchmarking.org Megahertz, More Is Better ONNX Runtime 1.14 CPU Peak Freq (Highest CPU Core Frequency) Monitor 1000 2000 3000 4000 5000
ONNX Runtime CPU Temperature Monitor Min Avg Max ONNX Runtime 1.14 28.0 46.2 50.0 ONNX Runtime 1.14 - No AMX 29.0 48.2 51.0 ONNX Runtime 1.14 use_dnnl 33.0 48.4 51.0 OpenBenchmarking.org Celsius, Fewer Is Better ONNX Runtime 1.14 CPU Temperature Monitor 15 30 45 60 75
ONNX Runtime CPU Temperature Monitor Min Avg Max ONNX Runtime 1.14 36.0 47.2 50.0 ONNX Runtime 1.14 - No AMX 37.0 47.7 50.0 ONNX Runtime 1.14 use_dnnl 36.0 47.5 50.0 OpenBenchmarking.org Celsius, Fewer Is Better ONNX Runtime 1.14 CPU Temperature Monitor 14 28 42 56 70
ONNX Runtime CPU Temperature Monitor OpenBenchmarking.org Celsius, Fewer Is Better ONNX Runtime 1.14 CPU Temperature Monitor ONNX Runtime 1.14 ONNX Runtime 1.14 - No AMX ONNX Runtime 1.14 use_dnnl 11 22 33 44 55 Min: 36 / Avg: 51.5 / Max: 54 Min: 36 / Avg: 50.98 / Max: 54 Min: 36 / Avg: 51.5 / Max: 54
ONNX Runtime CPU Temperature Monitor Min Avg Max ONNX Runtime 1.14 39.0 48.2 50.0 ONNX Runtime 1.14 - No AMX 39.0 48.5 50.0 ONNX Runtime 1.14 use_dnnl 38.0 48.5 51.0 OpenBenchmarking.org Celsius, Fewer Is Better ONNX Runtime 1.14 CPU Temperature Monitor 15 30 45 60 75
ONNX Runtime CPU Temperature Monitor Min Avg Max ONNX Runtime 1.14 37.0 53.0 55.0 ONNX Runtime 1.14 - No AMX 37.0 53.2 55.0 ONNX Runtime 1.14 use_dnnl 36.0 53.2 55.0 OpenBenchmarking.org Celsius, Fewer Is Better ONNX Runtime 1.14 CPU Temperature Monitor 15 30 45 60 75
ONNX Runtime CPU Temperature Monitor Min Avg Max ONNX Runtime 1.14 34.0 37.5 40.0 ONNX Runtime 1.14 - No AMX 35.0 40.9 45.0 ONNX Runtime 1.14 use_dnnl 35.0 39.1 45.0 OpenBenchmarking.org Celsius, Fewer Is Better ONNX Runtime 1.14 CPU Temperature Monitor 12 24 36 48 60
ONNX Runtime CPU Temperature Monitor Min Avg Max ONNX Runtime 1.14 32.0 47.6 50.0 ONNX Runtime 1.14 - No AMX 33.0 47.9 51.0 ONNX Runtime 1.14 use_dnnl 32.0 49.2 51.0 OpenBenchmarking.org Celsius, Fewer Is Better ONNX Runtime 1.14 CPU Temperature Monitor 15 30 45 60 75
ONNX Runtime CPU Temperature Monitor Min Avg Max ONNX Runtime 1.14 36.0 45.3 47.0 ONNX Runtime 1.14 - No AMX 36.0 45.1 48.0 ONNX Runtime 1.14 use_dnnl 37.0 45.8 49.0 OpenBenchmarking.org Celsius, Fewer Is Better ONNX Runtime 1.14 CPU Temperature Monitor 14 28 42 56 70
ONNX Runtime CPU Temperature Monitor Min Avg Max ONNX Runtime 1.14 35.0 48.1 50.0 ONNX Runtime 1.14 - No AMX 36.0 48.9 51.0 ONNX Runtime 1.14 use_dnnl 35.0 49.3 52.0 OpenBenchmarking.org Celsius, Fewer Is Better ONNX Runtime 1.14 CPU Temperature Monitor 15 30 45 60 75
ONNX Runtime CPU Temperature Monitor Min Avg Max ONNX Runtime 1.14 34.0 37.4 42.0 ONNX Runtime 1.14 - No AMX 34.0 37.5 40.0 ONNX Runtime 1.14 use_dnnl 34.0 38.1 42.0 OpenBenchmarking.org Celsius, Fewer Is Better ONNX Runtime 1.14 CPU Temperature Monitor 12 24 36 48 60
ONNX Runtime CPU Temperature Monitor Min Avg Max ONNX Runtime 1.14 33.0 48.4 52.0 ONNX Runtime 1.14 use_dnnl 32.0 48.9 52.0 OpenBenchmarking.org Celsius, Fewer Is Better ONNX Runtime 1.14 CPU Temperature Monitor 15 30 45 60 75
ONNX Runtime CPU Temperature Monitor Min Avg Max ONNX Runtime 1.14 37.0 47.9 50.0 ONNX Runtime 1.14 use_dnnl 37.0 48.4 51.0 OpenBenchmarking.org Celsius, Fewer Is Better ONNX Runtime 1.14 CPU Temperature Monitor 15 30 45 60 75
ONNX Runtime CPU Temperature Monitor Min Avg Max ONNX Runtime 1.14 36.0 50.1 52.0 ONNX Runtime 1.14 use_dnnl 36.0 50.7 53.0 OpenBenchmarking.org Celsius, Fewer Is Better ONNX Runtime 1.14 CPU Temperature Monitor 15 30 45 60 75
ONNX Runtime CPU Temperature Monitor Min Avg Max ONNX Runtime 1.14 38.0 50.7 53.0 ONNX Runtime 1.14 use_dnnl 40.0 48.9 51.0 OpenBenchmarking.org Celsius, Fewer Is Better ONNX Runtime 1.14 CPU Temperature Monitor 15 30 45 60 75
ONNX Runtime CPU Temperature Monitor Min Avg Max ONNX Runtime 1.14 40.0 48.8 51.0 ONNX Runtime 1.14 use_dnnl 40.0 49.7 51.0 OpenBenchmarking.org Celsius, Fewer Is Better ONNX Runtime 1.14 CPU Temperature Monitor 15 30 45 60 75
ONNX Runtime CPU Temperature Monitor Min Avg Max ONNX Runtime 1.14 40.0 46.9 49.0 ONNX Runtime 1.14 use_dnnl 40.0 48.1 51.0 OpenBenchmarking.org Celsius, Fewer Is Better ONNX Runtime 1.14 CPU Temperature Monitor 15 30 45 60 75
ONNX Runtime CPU Temperature Monitor Min Avg Max ONNX Runtime 1.14 37.0 46.3 48.0 ONNX Runtime 1.14 use_dnnl 40.0 47.9 50.0 OpenBenchmarking.org Celsius, Fewer Is Better ONNX Runtime 1.14 CPU Temperature Monitor 14 28 42 56 70
ONNX Runtime CPU Temperature Monitor Min Avg Max ONNX Runtime 1.14 37.0 45.1 48.0 ONNX Runtime 1.14 use_dnnl 40.0 46.0 51.0 OpenBenchmarking.org Celsius, Fewer Is Better ONNX Runtime 1.14 CPU Temperature Monitor 15 30 45 60 75
ONNX Runtime Model: yolov4 - Device: CPU - Executor: Standard OpenBenchmarking.org Inference Time Cost (ms), Fewer Is Better ONNX Runtime 1.14 Model: yolov4 - Device: CPU - Executor: Standard ONNX Runtime 1.14 ONNX Runtime 1.14 - No AMX ONNX Runtime 1.14 use_dnnl 20 40 60 80 100 SE +/- 0.12, N = 3 SE +/- 1.09, N = 15 SE +/- 0.92, N = 15 86.67 88.93 89.39 1. (CXX) g++ options: -ffunction-sections -fdata-sections -march=native -mtune=native -O3 -flto=auto -fno-fat-lto-objects -ldl -lrt
ONNX Runtime Model: yolov4 - Device: CPU - Executor: Parallel OpenBenchmarking.org Inference Time Cost (ms), Fewer Is Better ONNX Runtime 1.14 Model: yolov4 - Device: CPU - Executor: Parallel ONNX Runtime 1.14 ONNX Runtime 1.14 - No AMX ONNX Runtime 1.14 use_dnnl 20 40 60 80 100 SE +/- 0.95, N = 3 SE +/- 0.70, N = 3 SE +/- 0.59, N = 3 106.80 105.53 104.58 1. (CXX) g++ options: -ffunction-sections -fdata-sections -march=native -mtune=native -O3 -flto=auto -fno-fat-lto-objects -ldl -lrt
ONNX Runtime Model: fcn-resnet101-11 - Device: CPU - Executor: Standard OpenBenchmarking.org Inference Time Cost (ms), Fewer Is Better ONNX Runtime 1.14 Model: fcn-resnet101-11 - Device: CPU - Executor: Standard ONNX Runtime 1.14 ONNX Runtime 1.14 - No AMX ONNX Runtime 1.14 use_dnnl 20 40 60 80 100 SE +/- 1.75, N = 15 SE +/- 0.93, N = 3 SE +/- 1.67, N = 15 100.29 95.60 101.02 1. (CXX) g++ options: -ffunction-sections -fdata-sections -march=native -mtune=native -O3 -flto=auto -fno-fat-lto-objects -ldl -lrt
ONNX Runtime Model: fcn-resnet101-11 - Device: CPU - Executor: Parallel OpenBenchmarking.org Inference Time Cost (ms), Fewer Is Better ONNX Runtime 1.14 Model: fcn-resnet101-11 - Device: CPU - Executor: Parallel ONNX Runtime 1.14 ONNX Runtime 1.14 - No AMX ONNX Runtime 1.14 use_dnnl 70 140 210 280 350 SE +/- 4.47, N = 3 SE +/- 0.32, N = 3 SE +/- 3.47, N = 15 329.87 337.92 333.10 1. (CXX) g++ options: -ffunction-sections -fdata-sections -march=native -mtune=native -O3 -flto=auto -fno-fat-lto-objects -ldl -lrt
ONNX Runtime Model: super-resolution-10 - Device: CPU - Executor: Standard OpenBenchmarking.org Inference Time Cost (ms), Fewer Is Better ONNX Runtime 1.14 Model: super-resolution-10 - Device: CPU - Executor: Standard ONNX Runtime 1.14 ONNX Runtime 1.14 - No AMX ONNX Runtime 1.14 use_dnnl 1.0731 2.1462 3.2193 4.2924 5.3655 SE +/- 0.01656, N = 3 SE +/- 0.00962, N = 3 SE +/- 0.01348, N = 3 4.76912 4.76804 4.72937 1. (CXX) g++ options: -ffunction-sections -fdata-sections -march=native -mtune=native -O3 -flto=auto -fno-fat-lto-objects -ldl -lrt
ONNX Runtime Model: super-resolution-10 - Device: CPU - Executor: Parallel OpenBenchmarking.org Inference Time Cost (ms), Fewer Is Better ONNX Runtime 1.14 Model: super-resolution-10 - Device: CPU - Executor: Parallel ONNX Runtime 1.14 ONNX Runtime 1.14 - No AMX ONNX Runtime 1.14 use_dnnl 1.26 2.52 3.78 5.04 6.3 SE +/- 0.00850, N = 3 SE +/- 0.03108, N = 3 SE +/- 0.02952, N = 3 5.55742 5.54240 5.60021 1. (CXX) g++ options: -ffunction-sections -fdata-sections -march=native -mtune=native -O3 -flto=auto -fno-fat-lto-objects -ldl -lrt
ONNX Runtime Model: bertsquad-12 - Device: CPU - Executor: Standard OpenBenchmarking.org Inference Time Cost (ms), Fewer Is Better ONNX Runtime 1.14 Model: bertsquad-12 - Device: CPU - Executor: Standard ONNX Runtime 1.14 ONNX Runtime 1.14 - No AMX ONNX Runtime 1.14 use_dnnl 15 30 45 60 75 SE +/- 0.75, N = 3 SE +/- 0.16, N = 3 SE +/- 0.49, N = 9 66.97 65.92 65.88 1. (CXX) g++ options: -ffunction-sections -fdata-sections -march=native -mtune=native -O3 -flto=auto -fno-fat-lto-objects -ldl -lrt
ONNX Runtime Model: bertsquad-12 - Device: CPU - Executor: Parallel OpenBenchmarking.org Inference Time Cost (ms), Fewer Is Better ONNX Runtime 1.14 Model: bertsquad-12 - Device: CPU - Executor: Parallel ONNX Runtime 1.14 ONNX Runtime 1.14 - No AMX ONNX Runtime 1.14 use_dnnl 15 30 45 60 75 SE +/- 0.34, N = 3 SE +/- 0.38, N = 3 SE +/- 0.65, N = 3 66.55 66.69 66.69 1. (CXX) g++ options: -ffunction-sections -fdata-sections -march=native -mtune=native -O3 -flto=auto -fno-fat-lto-objects -ldl -lrt
ONNX Runtime Model: GPT-2 - Device: CPU - Executor: Standard OpenBenchmarking.org Inference Time Cost (ms), Fewer Is Better ONNX Runtime 1.14 Model: GPT-2 - Device: CPU - Executor: Standard ONNX Runtime 1.14 ONNX Runtime 1.14 - No AMX ONNX Runtime 1.14 use_dnnl 1.1836 2.3672 3.5508 4.7344 5.918 SE +/- 0.00587, N = 3 SE +/- 0.06843, N = 14 SE +/- 0.07549, N = 13 5.26053 5.13582 5.15414 1. (CXX) g++ options: -ffunction-sections -fdata-sections -march=native -mtune=native -O3 -flto=auto -fno-fat-lto-objects -ldl -lrt
ONNX Runtime Model: GPT-2 - Device: CPU - Executor: Parallel OpenBenchmarking.org Inference Time Cost (ms), Fewer Is Better ONNX Runtime 1.14 Model: GPT-2 - Device: CPU - Executor: Parallel ONNX Runtime 1.14 ONNX Runtime 1.14 - No AMX ONNX Runtime 1.14 use_dnnl 2 4 6 8 10 SE +/- 0.05508, N = 3 SE +/- 0.04118, N = 3 SE +/- 0.05441, N = 3 6.50527 6.31109 6.62309 1. (CXX) g++ options: -ffunction-sections -fdata-sections -march=native -mtune=native -O3 -flto=auto -fno-fat-lto-objects -ldl -lrt
ONNX Runtime Model: ArcFace ResNet-100 - Device: CPU - Executor: Standard OpenBenchmarking.org Inference Time Cost (ms), Fewer Is Better ONNX Runtime 1.14 Model: ArcFace ResNet-100 - Device: CPU - Executor: Standard ONNX Runtime 1.14 ONNX Runtime 1.14 use_dnnl 7 14 21 28 35 SE +/- 0.19, N = 3 SE +/- 0.44, N = 3 29.44 30.68 1. (CXX) g++ options: -ffunction-sections -fdata-sections -march=native -mtune=native -O3 -flto=auto -fno-fat-lto-objects -ldl -lrt
ONNX Runtime Model: ArcFace ResNet-100 - Device: CPU - Executor: Parallel OpenBenchmarking.org Inference Time Cost (ms), Fewer Is Better ONNX Runtime 1.14 Model: ArcFace ResNet-100 - Device: CPU - Executor: Parallel ONNX Runtime 1.14 ONNX Runtime 1.14 use_dnnl 8 16 24 32 40 SE +/- 0.08, N = 3 SE +/- 0.35, N = 3 36.74 36.47 1. (CXX) g++ options: -ffunction-sections -fdata-sections -march=native -mtune=native -O3 -flto=auto -fno-fat-lto-objects -ldl -lrt
ONNX Runtime Model: ResNet50 v1-12-int8 - Device: CPU - Executor: Standard OpenBenchmarking.org Inference Time Cost (ms), Fewer Is Better ONNX Runtime 1.14 Model: ResNet50 v1-12-int8 - Device: CPU - Executor: Standard ONNX Runtime 1.14 ONNX Runtime 1.14 use_dnnl 1.2806 2.5612 3.8418 5.1224 6.403 SE +/- 0.01809, N = 3 SE +/- 0.01821, N = 3 5.61969 5.69175 1. (CXX) g++ options: -ffunction-sections -fdata-sections -march=native -mtune=native -O3 -flto=auto -fno-fat-lto-objects -ldl -lrt
ONNX Runtime Model: ResNet50 v1-12-int8 - Device: CPU - Executor: Parallel OpenBenchmarking.org Inference Time Cost (ms), Fewer Is Better ONNX Runtime 1.14 Model: ResNet50 v1-12-int8 - Device: CPU - Executor: Parallel ONNX Runtime 1.14 ONNX Runtime 1.14 use_dnnl 2 4 6 8 10 SE +/- 0.06169, N = 5 SE +/- 0.04559, N = 3 6.15564 6.06317 1. (CXX) g++ options: -ffunction-sections -fdata-sections -march=native -mtune=native -O3 -flto=auto -fno-fat-lto-objects -ldl -lrt
ONNX Runtime Model: CaffeNet 12-int8 - Device: CPU - Executor: Standard OpenBenchmarking.org Inference Time Cost (ms), Fewer Is Better ONNX Runtime 1.14 Model: CaffeNet 12-int8 - Device: CPU - Executor: Standard ONNX Runtime 1.14 ONNX Runtime 1.14 use_dnnl 0.3288 0.6576 0.9864 1.3152 1.644 SE +/- 0.02000, N = 3 SE +/- 0.02005, N = 3 1.46149 1.43703 1. (CXX) g++ options: -ffunction-sections -fdata-sections -march=native -mtune=native -O3 -flto=auto -fno-fat-lto-objects -ldl -lrt
ONNX Runtime Model: CaffeNet 12-int8 - Device: CPU - Executor: Parallel OpenBenchmarking.org Inference Time Cost (ms), Fewer Is Better ONNX Runtime 1.14 Model: CaffeNet 12-int8 - Device: CPU - Executor: Parallel ONNX Runtime 1.14 ONNX Runtime 1.14 use_dnnl 0.3814 0.7628 1.1442 1.5256 1.907 SE +/- 0.00918, N = 3 SE +/- 0.01190, N = 3 1.69517 1.69067 1. (CXX) g++ options: -ffunction-sections -fdata-sections -march=native -mtune=native -O3 -flto=auto -fno-fat-lto-objects -ldl -lrt
ONNX Runtime Model: Faster R-CNN R-50-FPN-int8 - Device: CPU - Executor: Standard OpenBenchmarking.org Inference Time Cost (ms), Fewer Is Better ONNX Runtime 1.14 Model: Faster R-CNN R-50-FPN-int8 - Device: CPU - Executor: Standard ONNX Runtime 1.14 ONNX Runtime 1.14 use_dnnl 6 12 18 24 30 SE +/- 0.08, N = 3 SE +/- 0.15, N = 3 25.14 25.00 1. (CXX) g++ options: -ffunction-sections -fdata-sections -march=native -mtune=native -O3 -flto=auto -fno-fat-lto-objects -ldl -lrt
ONNX Runtime Model: Faster R-CNN R-50-FPN-int8 - Device: CPU - Executor: Parallel OpenBenchmarking.org Inference Time Cost (ms), Fewer Is Better ONNX Runtime 1.14 Model: Faster R-CNN R-50-FPN-int8 - Device: CPU - Executor: Parallel ONNX Runtime 1.14 ONNX Runtime 1.14 use_dnnl 7 14 21 28 35 SE +/- 0.42, N = 3 SE +/- 0.26, N = 3 30.40 31.47 1. (CXX) g++ options: -ffunction-sections -fdata-sections -march=native -mtune=native -O3 -flto=auto -fno-fat-lto-objects -ldl -lrt
ONNX Runtime CPU Power Consumption Monitor Min Avg Max ONNX Runtime 1.14 203 611 638 ONNX Runtime 1.14 - No AMX 201 614 639 ONNX Runtime 1.14 use_dnnl 197 613 639 OpenBenchmarking.org Watts, Fewer Is Better ONNX Runtime 1.14 CPU Power Consumption Monitor 200 400 600 800 1000
ONNX Runtime CPU Power Consumption Monitor Min Avg Max ONNX Runtime 1.14 198 611 637 ONNX Runtime 1.14 - No AMX 210 612 639 ONNX Runtime 1.14 use_dnnl 203 612 638 OpenBenchmarking.org Watts, Fewer Is Better ONNX Runtime 1.14 CPU Power Consumption Monitor 200 400 600 800 1000
ONNX Runtime CPU Power Consumption Monitor OpenBenchmarking.org Watts, Fewer Is Better ONNX Runtime 1.14 CPU Power Consumption Monitor ONNX Runtime 1.14 ONNX Runtime 1.14 - No AMX ONNX Runtime 1.14 use_dnnl 120 240 360 480 600 Min: 122.34 / Avg: 668.72 / Max: 709.46 Min: 202.65 / Avg: 673.64 / Max: 706.82 Min: 195.07 / Avg: 668.74 / Max: 709.06
ONNX Runtime CPU Power Consumption Monitor Min Avg Max ONNX Runtime 1.14 195 611 638 ONNX Runtime 1.14 - No AMX 210 611 638 ONNX Runtime 1.14 use_dnnl 199 613 641 OpenBenchmarking.org Watts, Fewer Is Better ONNX Runtime 1.14 CPU Power Consumption Monitor 200 400 600 800 1000
ONNX Runtime CPU Power Consumption Monitor Min Avg Max ONNX Runtime 1.14 209.4 437.4 451.4 ONNX Runtime 1.14 - No AMX 198.6 438.8 453.5 ONNX Runtime 1.14 use_dnnl 198.9 440.5 454.7 OpenBenchmarking.org Watts, Fewer Is Better ONNX Runtime 1.14 CPU Power Consumption Monitor 120 240 360 480 600
ONNX Runtime CPU Power Consumption Monitor Min Avg Max ONNX Runtime 1.14 191.9 446.6 462.1 ONNX Runtime 1.14 - No AMX 197.3 445.5 460.4 ONNX Runtime 1.14 use_dnnl 204.6 446.4 461.0 OpenBenchmarking.org Watts, Fewer Is Better ONNX Runtime 1.14 CPU Power Consumption Monitor 120 240 360 480 600
ONNX Runtime CPU Power Consumption Monitor Min Avg Max ONNX Runtime 1.14 201.2 561.6 584.0 ONNX Runtime 1.14 - No AMX 195.6 564.4 586.3 ONNX Runtime 1.14 use_dnnl 199.2 565.5 588.4 OpenBenchmarking.org Watts, Fewer Is Better ONNX Runtime 1.14 CPU Power Consumption Monitor 160 320 480 640 800
ONNX Runtime CPU Power Consumption Monitor Min Avg Max ONNX Runtime 1.14 202.7 562.2 591.1 ONNX Runtime 1.14 - No AMX 194.9 560.2 590.7 ONNX Runtime 1.14 use_dnnl 212.9 563.2 593.5 OpenBenchmarking.org Watts, Fewer Is Better ONNX Runtime 1.14 CPU Power Consumption Monitor 160 320 480 640 800
ONNX Runtime CPU Power Consumption Monitor Min Avg Max ONNX Runtime 1.14 194.3 434.5 450.1 ONNX Runtime 1.14 - No AMX 193.3 435.5 452.2 ONNX Runtime 1.14 use_dnnl 104.6 436.2 453.4 OpenBenchmarking.org Watts, Fewer Is Better ONNX Runtime 1.14 CPU Power Consumption Monitor 120 240 360 480 600
ONNX Runtime CPU Power Consumption Monitor Min Avg Max ONNX Runtime 1.14 202.4 438.8 455.7 ONNX Runtime 1.14 - No AMX 201.2 439.1 454.6 ONNX Runtime 1.14 use_dnnl 203.6 438.6 458.8 OpenBenchmarking.org Watts, Fewer Is Better ONNX Runtime 1.14 CPU Power Consumption Monitor 120 240 360 480 600
ONNX Runtime CPU Power Consumption Monitor Min Avg Max ONNX Runtime 1.14 202 622 649 ONNX Runtime 1.14 use_dnnl 199 621 652 OpenBenchmarking.org Watts, Fewer Is Better ONNX Runtime 1.14 CPU Power Consumption Monitor 200 400 600 800 1000
ONNX Runtime CPU Power Consumption Monitor Min Avg Max ONNX Runtime 1.14 207 623 650 ONNX Runtime 1.14 use_dnnl 201 625 651 OpenBenchmarking.org Watts, Fewer Is Better ONNX Runtime 1.14 CPU Power Consumption Monitor 200 400 600 800 1000
ONNX Runtime CPU Power Consumption Monitor Min Avg Max ONNX Runtime 1.14 202 616 637 ONNX Runtime 1.14 use_dnnl 208 617 639 OpenBenchmarking.org Watts, Fewer Is Better ONNX Runtime 1.14 CPU Power Consumption Monitor 200 400 600 800 1000
ONNX Runtime CPU Power Consumption Monitor Min Avg Max ONNX Runtime 1.14 290 633 659 ONNX Runtime 1.14 use_dnnl 289 638 660 OpenBenchmarking.org Watts, Fewer Is Better ONNX Runtime 1.14 CPU Power Consumption Monitor 200 400 600 800 1000
ONNX Runtime CPU Power Consumption Monitor Min Avg Max ONNX Runtime 1.14 289 602 623 ONNX Runtime 1.14 use_dnnl 283 607 628 OpenBenchmarking.org Watts, Fewer Is Better ONNX Runtime 1.14 CPU Power Consumption Monitor 200 400 600 800 1000
ONNX Runtime CPU Power Consumption Monitor Min Avg Max ONNX Runtime 1.14 278 615 633 ONNX Runtime 1.14 use_dnnl 272 617 637 OpenBenchmarking.org Watts, Fewer Is Better ONNX Runtime 1.14 CPU Power Consumption Monitor 200 400 600 800 1000
ONNX Runtime CPU Power Consumption Monitor Min Avg Max ONNX Runtime 1.14 239.7 582.2 599.4 ONNX Runtime 1.14 use_dnnl 257.9 583.0 603.0 OpenBenchmarking.org Watts, Fewer Is Better ONNX Runtime 1.14 CPU Power Consumption Monitor 160 320 480 640 800
ONNX Runtime CPU Power Consumption Monitor Min Avg Max ONNX Runtime 1.14 236.5 561.9 589.2 ONNX Runtime 1.14 use_dnnl 267.7 561.7 588.0 OpenBenchmarking.org Watts, Fewer Is Better ONNX Runtime 1.14 CPU Power Consumption Monitor 160 320 480 640 800
Phoronix Test Suite v10.8.5