2 x Intel Xeon Platinum 8490H testing with a Quanta Cloud S6Q-MB-MPS (3A10.uh BIOS) and ASPEED on Ubuntu 23.04 via the Phoronix Test Suite.
Compare your own system(s) to this result file with the
Phoronix Test Suite by running the command:
phoronix-test-suite benchmark 2302115-NE-OONXSAPPH22
OONX Sapphire Rapids
2 x Intel Xeon Platinum 8490H testing with a Quanta Cloud S6Q-MB-MPS (3A10.uh BIOS) and ASPEED on Ubuntu 23.04 via the Phoronix Test Suite.
ONNX Runtime 1.14:
Processor: 2 x Intel Xeon Platinum 8490H @ 3.50GHz (120 Cores / 240 Threads), Motherboard: Quanta Cloud S6Q-MB-MPS (3A10.uh BIOS), Chipset: Intel Device 1bce, Memory: 1008GB, Disk: 2 x 1920GB SAMSUNG MZWLJ1T9HBJR-00007 + 960GB INTEL SSDSC2KG96, Graphics: ASPEED, Monitor: VGA HDMI, Network: 4 x Intel E810-C for QSFP + 2 x Intel X710 for 10GBASE-T
OS: Ubuntu 23.04, Kernel: 5.19.0-21-generic (x86_64), Desktop: GNOME Shell 43.2, Display Server: X Server 1.21.1.6, Compiler: GCC 12.2.0, File-System: ext4, Screen Resolution: 1920x1080
ONNX Runtime 1.14 - No AMX:
Processor: 2 x Intel Xeon Platinum 8490H @ 3.50GHz (120 Cores / 240 Threads), Motherboard: Quanta Cloud S6Q-MB-MPS (3A10.uh BIOS), Chipset: Intel Device 1bce, Memory: 1008GB, Disk: 2 x 1920GB SAMSUNG MZWLJ1T9HBJR-00007 + 960GB INTEL SSDSC2KG96, Graphics: ASPEED, Monitor: VGA HDMI, Network: 4 x Intel E810-C for QSFP + 2 x Intel X710 for 10GBASE-T
OS: Ubuntu 23.04, Kernel: 5.19.0-21-generic (x86_64), Desktop: GNOME Shell 43.2, Display Server: X Server 1.21.1.6, Compiler: GCC 12.2.0, File-System: ext4, Screen Resolution: 1920x1080
ONNX Runtime 1.14 use_dnnl:
Processor: 2 x Intel Xeon Platinum 8490H @ 3.50GHz (120 Cores / 240 Threads), Motherboard: Quanta Cloud S6Q-MB-MPS (3A10.uh BIOS), Chipset: Intel Device 1bce, Memory: 1008GB, Disk: 2 x 1920GB SAMSUNG MZWLJ1T9HBJR-00007 + 960GB INTEL SSDSC2KG96, Graphics: ASPEED, Monitor: VGA HDMI, Network: 4 x Intel E810-C for QSFP + 2 x Intel X710 for 10GBASE-T
OS: Ubuntu 23.04, Kernel: 5.19.0-21-generic (x86_64), Desktop: GNOME Shell 43.2, Display Server: X Server 1.21.1.6, Compiler: GCC 12.2.0, File-System: ext4, Screen Resolution: 1920x1080
ONNX Runtime 1.14
Model: yolov4 - Device: CPU - Executor: Standard
Inference Time Cost (ms) < Lower Is Better
ONNX Runtime 1.14 .......... 86.67 |==========================================
ONNX Runtime 1.14 - No AMX . 88.93 |===========================================
ONNX Runtime 1.14 use_dnnl . 89.39 |===========================================
ONNX Runtime 1.14
Model: yolov4 - Device: CPU - Executor: Standard
Inferences Per Second > Higher Is Better
ONNX Runtime 1.14 .......... 11.54 |===========================================
ONNX Runtime 1.14 - No AMX . 11.27 |==========================================
ONNX Runtime 1.14 use_dnnl . 11.20 |==========================================
ONNX Runtime 1.14
Model: fcn-resnet101-11 - Device: CPU - Executor: Standard
Inference Time Cost (ms) < Lower Is Better
ONNX Runtime 1.14 .......... 100.29 |==========================================
ONNX Runtime 1.14 - No AMX . 95.60 |========================================
ONNX Runtime 1.14 use_dnnl . 101.02 |==========================================
ONNX Runtime 1.14
Model: fcn-resnet101-11 - Device: CPU - Executor: Standard
Inferences Per Second > Higher Is Better
ONNX Runtime 1.14 .......... 10.00997 |======================================
ONNX Runtime 1.14 - No AMX . 10.46180 |========================================
ONNX Runtime 1.14 use_dnnl . 9.93450 |======================================
ONNX Runtime 1.14
Model: GPT-2 - Device: CPU - Executor: Standard
Inference Time Cost (ms) < Lower Is Better
ONNX Runtime 1.14 .......... 5.26053 |=========================================
ONNX Runtime 1.14 - No AMX . 5.13582 |========================================
ONNX Runtime 1.14 use_dnnl . 5.15414 |========================================
ONNX Runtime 1.14
Model: GPT-2 - Device: CPU - Executor: Standard
Inferences Per Second > Higher Is Better
ONNX Runtime 1.14 .......... 190.02 |=========================================
ONNX Runtime 1.14 - No AMX . 195.15 |==========================================
ONNX Runtime 1.14 use_dnnl . 194.52 |==========================================
ONNX Runtime 1.14
Model: fcn-resnet101-11 - Device: CPU - Executor: Parallel
Inference Time Cost (ms) < Lower Is Better
ONNX Runtime 1.14 .......... 329.87 |=========================================
ONNX Runtime 1.14 - No AMX . 337.92 |==========================================
ONNX Runtime 1.14 use_dnnl . 333.10 |=========================================
ONNX Runtime 1.14
Model: fcn-resnet101-11 - Device: CPU - Executor: Parallel
Inferences Per Second > Higher Is Better
ONNX Runtime 1.14 .......... 3.03253 |=========================================
ONNX Runtime 1.14 - No AMX . 2.95924 |========================================
ONNX Runtime 1.14 use_dnnl . 3.00654 |=========================================
ONNX Runtime 1.14
Model: bertsquad-12 - Device: CPU - Executor: Standard
Inference Time Cost (ms) < Lower Is Better
ONNX Runtime 1.14 .......... 66.97 |===========================================
ONNX Runtime 1.14 - No AMX . 65.92 |==========================================
ONNX Runtime 1.14 use_dnnl . 65.88 |==========================================
ONNX Runtime 1.14
Model: bertsquad-12 - Device: CPU - Executor: Standard
Inferences Per Second > Higher Is Better
ONNX Runtime 1.14 .......... 14.94 |==========================================
ONNX Runtime 1.14 - No AMX . 15.17 |===========================================
ONNX Runtime 1.14 use_dnnl . 15.19 |===========================================
ONNX Runtime 1.14
Model: ResNet50 v1-12-int8 - Device: CPU - Executor: Parallel
Inference Time Cost (ms) < Lower Is Better
ONNX Runtime 1.14 .......... 6.15564 |=========================================
ONNX Runtime 1.14 use_dnnl . 6.06317 |========================================
ONNX Runtime 1.14
Model: ResNet50 v1-12-int8 - Device: CPU - Executor: Parallel
Inferences Per Second > Higher Is Better
ONNX Runtime 1.14 .......... 162.48 |=========================================
ONNX Runtime 1.14 use_dnnl . 164.90 |==========================================
ONNX Runtime 1.14
Model: GPT-2 - Device: CPU - Executor: Parallel
Inference Time Cost (ms) < Lower Is Better
ONNX Runtime 1.14 .......... 6.50527 |========================================
ONNX Runtime 1.14 - No AMX . 6.31109 |=======================================
ONNX Runtime 1.14 use_dnnl . 6.62309 |=========================================
ONNX Runtime 1.14
Model: GPT-2 - Device: CPU - Executor: Parallel
Inferences Per Second > Higher Is Better
ONNX Runtime 1.14 .......... 153.60 |=========================================
ONNX Runtime 1.14 - No AMX . 158.31 |==========================================
ONNX Runtime 1.14 use_dnnl . 150.86 |========================================
ONNX Runtime 1.14
Model: bertsquad-12 - Device: CPU - Executor: Parallel
Inference Time Cost (ms) < Lower Is Better
ONNX Runtime 1.14 .......... 66.55 |===========================================
ONNX Runtime 1.14 - No AMX . 66.69 |===========================================
ONNX Runtime 1.14 use_dnnl . 66.69 |===========================================
ONNX Runtime 1.14
Model: bertsquad-12 - Device: CPU - Executor: Parallel
Inferences Per Second > Higher Is Better
ONNX Runtime 1.14 .......... 15.03 |===========================================
ONNX Runtime 1.14 - No AMX . 15.00 |===========================================
ONNX Runtime 1.14 use_dnnl . 15.00 |===========================================
ONNX Runtime 1.14
Model: ArcFace ResNet-100 - Device: CPU - Executor: Standard
Inference Time Cost (ms) < Lower Is Better
ONNX Runtime 1.14 .......... 29.44 |=========================================
ONNX Runtime 1.14 use_dnnl . 30.68 |===========================================
ONNX Runtime 1.14
Model: ArcFace ResNet-100 - Device: CPU - Executor: Standard
Inferences Per Second > Higher Is Better
ONNX Runtime 1.14 .......... 33.97 |===========================================
ONNX Runtime 1.14 use_dnnl . 32.61 |=========================================
ONNX Runtime 1.14
Model: ArcFace ResNet-100 - Device: CPU - Executor: Parallel
Inference Time Cost (ms) < Lower Is Better
ONNX Runtime 1.14 .......... 36.74 |===========================================
ONNX Runtime 1.14 use_dnnl . 36.47 |===========================================
ONNX Runtime 1.14
Model: ArcFace ResNet-100 - Device: CPU - Executor: Parallel
Inferences Per Second > Higher Is Better
ONNX Runtime 1.14 .......... 27.21 |===========================================
ONNX Runtime 1.14 use_dnnl . 27.43 |===========================================
ONNX Runtime 1.14
Model: yolov4 - Device: CPU - Executor: Parallel
Inference Time Cost (ms) < Lower Is Better
ONNX Runtime 1.14 .......... 106.80 |==========================================
ONNX Runtime 1.14 - No AMX . 105.53 |==========================================
ONNX Runtime 1.14 use_dnnl . 104.58 |=========================================
ONNX Runtime 1.14
Model: yolov4 - Device: CPU - Executor: Parallel
Inferences Per Second > Higher Is Better
ONNX Runtime 1.14 .......... 9.36451 |========================================
ONNX Runtime 1.14 - No AMX . 9.47631 |=========================================
ONNX Runtime 1.14 use_dnnl . 9.56208 |=========================================
ONNX Runtime 1.14
Model: Faster R-CNN R-50-FPN-int8 - Device: CPU - Executor: Standard
Inference Time Cost (ms) < Lower Is Better
ONNX Runtime 1.14 .......... 25.14 |===========================================
ONNX Runtime 1.14 use_dnnl . 25.00 |===========================================
ONNX Runtime 1.14
Model: Faster R-CNN R-50-FPN-int8 - Device: CPU - Executor: Standard
Inferences Per Second > Higher Is Better
ONNX Runtime 1.14 .......... 39.77 |===========================================
ONNX Runtime 1.14 use_dnnl . 40.01 |===========================================
ONNX Runtime 1.14
Model: Faster R-CNN R-50-FPN-int8 - Device: CPU - Executor: Parallel
Inference Time Cost (ms) < Lower Is Better
ONNX Runtime 1.14 .......... 30.40 |==========================================
ONNX Runtime 1.14 use_dnnl . 31.47 |===========================================
ONNX Runtime 1.14
Model: Faster R-CNN R-50-FPN-int8 - Device: CPU - Executor: Parallel
Inferences Per Second > Higher Is Better
ONNX Runtime 1.14 .......... 32.90 |===========================================
ONNX Runtime 1.14 use_dnnl . 31.78 |==========================================
ONNX Runtime 1.14
Model: CaffeNet 12-int8 - Device: CPU - Executor: Parallel
Inference Time Cost (ms) < Lower Is Better
ONNX Runtime 1.14 .......... 1.69517 |=========================================
ONNX Runtime 1.14 use_dnnl . 1.69067 |=========================================
ONNX Runtime 1.14
Model: CaffeNet 12-int8 - Device: CPU - Executor: Parallel
Inferences Per Second > Higher Is Better
ONNX Runtime 1.14 .......... 589.20 |==========================================
ONNX Runtime 1.14 use_dnnl . 590.80 |==========================================
ONNX Runtime 1.14
Model: CaffeNet 12-int8 - Device: CPU - Executor: Standard
Inference Time Cost (ms) < Lower Is Better
ONNX Runtime 1.14 .......... 1.46149 |=========================================
ONNX Runtime 1.14 use_dnnl . 1.43703 |========================================
ONNX Runtime 1.14
Model: CaffeNet 12-int8 - Device: CPU - Executor: Standard
Inferences Per Second > Higher Is Better
ONNX Runtime 1.14 .......... 684.15 |=========================================
ONNX Runtime 1.14 use_dnnl . 695.78 |==========================================
ONNX Runtime 1.14
Model: ResNet50 v1-12-int8 - Device: CPU - Executor: Standard
Inference Time Cost (ms) < Lower Is Better
ONNX Runtime 1.14 .......... 5.61969 |========================================
ONNX Runtime 1.14 use_dnnl . 5.69175 |=========================================
ONNX Runtime 1.14
Model: ResNet50 v1-12-int8 - Device: CPU - Executor: Standard
Inferences Per Second > Higher Is Better
ONNX Runtime 1.14 .......... 177.93 |==========================================
ONNX Runtime 1.14 use_dnnl . 175.67 |=========================================
ONNX Runtime 1.14
Model: super-resolution-10 - Device: CPU - Executor: Parallel
Inference Time Cost (ms) < Lower Is Better
ONNX Runtime 1.14 .......... 5.55742 |=========================================
ONNX Runtime 1.14 - No AMX . 5.54240 |=========================================
ONNX Runtime 1.14 use_dnnl . 5.60021 |=========================================
ONNX Runtime 1.14
Model: super-resolution-10 - Device: CPU - Executor: Parallel
Inferences Per Second > Higher Is Better
ONNX Runtime 1.14 .......... 179.90 |==========================================
ONNX Runtime 1.14 - No AMX . 180.39 |==========================================
ONNX Runtime 1.14 use_dnnl . 178.53 |==========================================
ONNX Runtime 1.14
Model: super-resolution-10 - Device: CPU - Executor: Standard
Inference Time Cost (ms) < Lower Is Better
ONNX Runtime 1.14 .......... 4.76912 |=========================================
ONNX Runtime 1.14 - No AMX . 4.76804 |=========================================
ONNX Runtime 1.14 use_dnnl . 4.72937 |=========================================
ONNX Runtime 1.14
Model: super-resolution-10 - Device: CPU - Executor: Standard
Inferences Per Second > Higher Is Better
ONNX Runtime 1.14 .......... 209.66 |==========================================
ONNX Runtime 1.14 - No AMX . 209.70 |==========================================
ONNX Runtime 1.14 use_dnnl . 211.42 |==========================================
CPU Temperature Monitor
Phoronix Test Suite System Monitoring
Celsius
ONNX Runtime 1.14 .......... MIN: 28.0 AVG: 47.6 MAX: 55.0
ONNX Runtime 1.14 use_dnnl . MIN: 32.0 AVG: 48.4 MAX: 55.0
CPU Power Consumption Monitor
Phoronix Test Suite System Monitoring
Watts
ONNX Runtime 1.14 .......... MIN: 122 AVG: 577 MAX: 709
ONNX Runtime 1.14 use_dnnl . MIN: 105 AVG: 571 MAX: 709
CPU Peak Freq (Highest CPU Core Frequency) Monitor
Phoronix Test Suite System Monitoring
Megahertz
ONNX Runtime 1.14 .......... MIN: 1900 AVG: 3026 MAX: 3666
ONNX Runtime 1.14 use_dnnl . MIN: 1900 AVG: 3073 MAX: 3677
ONNX Runtime 1.14
CPU Temperature Monitor
Celsius < Lower Is Better
ONNX Runtime 1.14 .......... MIN: 37.0 AVG: 45.1 MAX: 48.0
ONNX Runtime 1.14 use_dnnl . MIN: 40.0 AVG: 46.0 MAX: 51.0
ONNX Runtime 1.14
CPU Power Consumption Monitor
Watts < Lower Is Better
ONNX Runtime 1.14 .......... MIN: 237 AVG: 562 MAX: 589
ONNX Runtime 1.14 use_dnnl . MIN: 268 AVG: 562 MAX: 588
ONNX Runtime 1.14
CPU Peak Freq (Highest CPU Core Frequency) Monitor
Megahertz > Higher Is Better
ONNX Runtime 1.14 .......... MIN: 1900 AVG: 3070 MAX: 3525
ONNX Runtime 1.14 use_dnnl . MIN: 1900 AVG: 3034 MAX: 3508
ONNX Runtime 1.14
CPU Temperature Monitor
Celsius < Lower Is Better
ONNX Runtime 1.14 .......... MIN: 37.0 AVG: 46.3 MAX: 48.0
ONNX Runtime 1.14 use_dnnl . MIN: 40.0 AVG: 47.9 MAX: 50.0
ONNX Runtime 1.14
CPU Power Consumption Monitor
Watts < Lower Is Better
ONNX Runtime 1.14 .......... MIN: 240 AVG: 582 MAX: 599
ONNX Runtime 1.14 use_dnnl . MIN: 258 AVG: 583 MAX: 603
ONNX Runtime 1.14
CPU Peak Freq (Highest CPU Core Frequency) Monitor
Megahertz > Higher Is Better
ONNX Runtime 1.14 .......... MIN: 1900 AVG: 2982 MAX: 3510
ONNX Runtime 1.14 use_dnnl . MIN: 1900 AVG: 2997 MAX: 3509
ONNX Runtime 1.14
CPU Temperature Monitor
Celsius < Lower Is Better
ONNX Runtime 1.14 .......... MIN: 40.0 AVG: 46.9 MAX: 49.0
ONNX Runtime 1.14 use_dnnl . MIN: 40.0 AVG: 48.1 MAX: 51.0
ONNX Runtime 1.14
CPU Power Consumption Monitor
Watts < Lower Is Better
ONNX Runtime 1.14 .......... MIN: 278 AVG: 615 MAX: 633
ONNX Runtime 1.14 use_dnnl . MIN: 272 AVG: 617 MAX: 637
ONNX Runtime 1.14
CPU Peak Freq (Highest CPU Core Frequency) Monitor
Megahertz > Higher Is Better
ONNX Runtime 1.14 .......... MIN: 1900 AVG: 2908 MAX: 3500
ONNX Runtime 1.14 use_dnnl . MIN: 1900 AVG: 2903 MAX: 3501
ONNX Runtime 1.14
CPU Temperature Monitor
Celsius < Lower Is Better
ONNX Runtime 1.14 .......... MIN: 40.0 AVG: 48.8 MAX: 51.0
ONNX Runtime 1.14 use_dnnl . MIN: 40.0 AVG: 49.7 MAX: 51.0
ONNX Runtime 1.14
CPU Power Consumption Monitor
Watts < Lower Is Better
ONNX Runtime 1.14 .......... MIN: 289 AVG: 602 MAX: 623
ONNX Runtime 1.14 use_dnnl . MIN: 283 AVG: 607 MAX: 628
ONNX Runtime 1.14
CPU Peak Freq (Highest CPU Core Frequency) Monitor
Megahertz > Higher Is Better
ONNX Runtime 1.14 .......... MIN: 1900 AVG: 2908 MAX: 3400
ONNX Runtime 1.14 use_dnnl . MIN: 1900 AVG: 2914 MAX: 3501
ONNX Runtime 1.14
CPU Temperature Monitor
Celsius < Lower Is Better
ONNX Runtime 1.14 .......... MIN: 38.0 AVG: 50.7 MAX: 53.0
ONNX Runtime 1.14 use_dnnl . MIN: 40.0 AVG: 48.9 MAX: 51.0
ONNX Runtime 1.14
CPU Power Consumption Monitor
Watts < Lower Is Better
ONNX Runtime 1.14 .......... MIN: 290 AVG: 633 MAX: 659
ONNX Runtime 1.14 use_dnnl . MIN: 289 AVG: 638 MAX: 660
ONNX Runtime 1.14
CPU Peak Freq (Highest CPU Core Frequency) Monitor
Megahertz > Higher Is Better
ONNX Runtime 1.14 .......... MIN: 1900 AVG: 2890 MAX: 3666
ONNX Runtime 1.14 use_dnnl . MIN: 1900 AVG: 2889 MAX: 3507
ONNX Runtime 1.14
CPU Temperature Monitor
Celsius < Lower Is Better
ONNX Runtime 1.14 .......... MIN: 36.0 AVG: 50.1 MAX: 52.0
ONNX Runtime 1.14 use_dnnl . MIN: 36.0 AVG: 50.7 MAX: 53.0
ONNX Runtime 1.14
CPU Power Consumption Monitor
Watts < Lower Is Better
ONNX Runtime 1.14 .......... MIN: 202 AVG: 616 MAX: 637
ONNX Runtime 1.14 use_dnnl . MIN: 208 AVG: 617 MAX: 639
ONNX Runtime 1.14
CPU Peak Freq (Highest CPU Core Frequency) Monitor
Megahertz > Higher Is Better
ONNX Runtime 1.14 .......... MIN: 1900 AVG: 2904 MAX: 3506
ONNX Runtime 1.14 use_dnnl . MIN: 1900 AVG: 2905 MAX: 3503
ONNX Runtime 1.14
CPU Temperature Monitor
Celsius < Lower Is Better
ONNX Runtime 1.14 .......... MIN: 37.0 AVG: 47.9 MAX: 50.0
ONNX Runtime 1.14 use_dnnl . MIN: 37.0 AVG: 48.4 MAX: 51.0
ONNX Runtime 1.14
CPU Power Consumption Monitor
Watts < Lower Is Better
ONNX Runtime 1.14 .......... MIN: 207 AVG: 623 MAX: 650
ONNX Runtime 1.14 use_dnnl . MIN: 201 AVG: 625 MAX: 651
ONNX Runtime 1.14
CPU Peak Freq (Highest CPU Core Frequency) Monitor
Megahertz > Higher Is Better
ONNX Runtime 1.14 .......... MIN: 1900 AVG: 2911 MAX: 3514
ONNX Runtime 1.14 use_dnnl . MIN: 1900 AVG: 2929 MAX: 3510
ONNX Runtime 1.14
CPU Temperature Monitor
Celsius < Lower Is Better
ONNX Runtime 1.14 .......... MIN: 33.0 AVG: 48.4 MAX: 52.0
ONNX Runtime 1.14 use_dnnl . MIN: 32.0 AVG: 48.9 MAX: 52.0
ONNX Runtime 1.14
CPU Power Consumption Monitor
Watts < Lower Is Better
ONNX Runtime 1.14 .......... MIN: 202 AVG: 622 MAX: 649
ONNX Runtime 1.14 use_dnnl . MIN: 199 AVG: 621 MAX: 652
ONNX Runtime 1.14
CPU Peak Freq (Highest CPU Core Frequency) Monitor
Megahertz > Higher Is Better
ONNX Runtime 1.14 .......... MIN: 2900 AVG: 2935 MAX: 3514
ONNX Runtime 1.14 use_dnnl . MIN: 1900 AVG: 2932 MAX: 3512
ONNX Runtime 1.14
CPU Temperature Monitor
Celsius < Lower Is Better
ONNX Runtime 1.14 .......... MIN: 34.0 AVG: 37.4 MAX: 42.0
ONNX Runtime 1.14 - No AMX . MIN: 34.0 AVG: 37.5 MAX: 40.0
ONNX Runtime 1.14 use_dnnl . MIN: 34.0 AVG: 38.1 MAX: 42.0
ONNX Runtime 1.14
CPU Power Consumption Monitor
Watts < Lower Is Better
ONNX Runtime 1.14 .......... MIN: 202 AVG: 439 MAX: 456
ONNX Runtime 1.14 - No AMX . MIN: 201 AVG: 439 MAX: 455
ONNX Runtime 1.14 use_dnnl . MIN: 204 AVG: 439 MAX: 459
ONNX Runtime 1.14
CPU Peak Freq (Highest CPU Core Frequency) Monitor
Megahertz > Higher Is Better
ONNX Runtime 1.14 .......... MIN: 1900 AVG: 3494 MAX: 3516
ONNX Runtime 1.14 - No AMX . MIN: 1900 AVG: 3475 MAX: 3510
ONNX Runtime 1.14 use_dnnl . MIN: 1900 AVG: 3478 MAX: 3511
ONNX Runtime 1.14
CPU Temperature Monitor
Celsius < Lower Is Better
ONNX Runtime 1.14 .......... MIN: 35.0 AVG: 48.1 MAX: 50.0
ONNX Runtime 1.14 - No AMX . MIN: 36.0 AVG: 48.9 MAX: 51.0
ONNX Runtime 1.14 use_dnnl . MIN: 35.0 AVG: 49.3 MAX: 52.0
ONNX Runtime 1.14
CPU Power Consumption Monitor
Watts < Lower Is Better
ONNX Runtime 1.14 .......... MIN: 194 AVG: 435 MAX: 450
ONNX Runtime 1.14 - No AMX . MIN: 193 AVG: 436 MAX: 452
ONNX Runtime 1.14 use_dnnl . MIN: 105 AVG: 436 MAX: 453
ONNX Runtime 1.14
CPU Peak Freq (Highest CPU Core Frequency) Monitor
Megahertz > Higher Is Better
ONNX Runtime 1.14 .......... MIN: 1900 AVG: 3482 MAX: 3515
ONNX Runtime 1.14 - No AMX . MIN: 1900 AVG: 3486 MAX: 3524
ONNX Runtime 1.14 use_dnnl . MIN: 1900 AVG: 3494 MAX: 3520
ONNX Runtime 1.14
CPU Temperature Monitor
Celsius < Lower Is Better
ONNX Runtime 1.14 .......... MIN: 36.0 AVG: 45.3 MAX: 47.0
ONNX Runtime 1.14 - No AMX . MIN: 36.0 AVG: 45.1 MAX: 48.0
ONNX Runtime 1.14 use_dnnl . MIN: 37.0 AVG: 45.8 MAX: 49.0
ONNX Runtime 1.14
CPU Power Consumption Monitor
Watts < Lower Is Better
ONNX Runtime 1.14 .......... MIN: 203 AVG: 562 MAX: 591
ONNX Runtime 1.14 - No AMX . MIN: 195 AVG: 560 MAX: 591
ONNX Runtime 1.14 use_dnnl . MIN: 213 AVG: 563 MAX: 593
ONNX Runtime 1.14
CPU Peak Freq (Highest CPU Core Frequency) Monitor
Megahertz > Higher Is Better
ONNX Runtime 1.14 .......... MIN: 1900 AVG: 3179 MAX: 3512
ONNX Runtime 1.14 - No AMX . MIN: 1900 AVG: 3181 MAX: 3510
ONNX Runtime 1.14 use_dnnl . MIN: 1900 AVG: 3179 MAX: 3515
ONNX Runtime 1.14
CPU Temperature Monitor
Celsius < Lower Is Better
ONNX Runtime 1.14 .......... MIN: 32.0 AVG: 47.6 MAX: 50.0
ONNX Runtime 1.14 - No AMX . MIN: 33.0 AVG: 47.9 MAX: 51.0
ONNX Runtime 1.14 use_dnnl . MIN: 32.0 AVG: 49.2 MAX: 51.0
ONNX Runtime 1.14
CPU Power Consumption Monitor
Watts < Lower Is Better
ONNX Runtime 1.14 .......... MIN: 201 AVG: 562 MAX: 584
ONNX Runtime 1.14 - No AMX . MIN: 196 AVG: 564 MAX: 586
ONNX Runtime 1.14 use_dnnl . MIN: 199 AVG: 566 MAX: 588
ONNX Runtime 1.14
CPU Peak Freq (Highest CPU Core Frequency) Monitor
Megahertz > Higher Is Better
ONNX Runtime 1.14 .......... MIN: 1900 AVG: 3460 MAX: 3512
ONNX Runtime 1.14 - No AMX . MIN: 1900 AVG: 3470 MAX: 3512
ONNX Runtime 1.14 use_dnnl . MIN: 1900 AVG: 3466 MAX: 3516
ONNX Runtime 1.14
CPU Temperature Monitor
Celsius < Lower Is Better
ONNX Runtime 1.14 .......... MIN: 34.0 AVG: 37.5 MAX: 40.0
ONNX Runtime 1.14 - No AMX . MIN: 35.0 AVG: 40.9 MAX: 45.0
ONNX Runtime 1.14 use_dnnl . MIN: 35.0 AVG: 39.1 MAX: 45.0
ONNX Runtime 1.14
CPU Power Consumption Monitor
Watts < Lower Is Better
ONNX Runtime 1.14 .......... MIN: 192 AVG: 447 MAX: 462
ONNX Runtime 1.14 - No AMX . MIN: 197 AVG: 446 MAX: 460
ONNX Runtime 1.14 use_dnnl . MIN: 205 AVG: 446 MAX: 461
ONNX Runtime 1.14
CPU Peak Freq (Highest CPU Core Frequency) Monitor
Megahertz > Higher Is Better
ONNX Runtime 1.14 .......... MIN: 1900 AVG: 3495 MAX: 3515
ONNX Runtime 1.14 - No AMX . MIN: 1900 AVG: 3478 MAX: 3517
ONNX Runtime 1.14 use_dnnl . MIN: 1900 AVG: 3489 MAX: 3514
ONNX Runtime 1.14
CPU Temperature Monitor
Celsius < Lower Is Better
ONNX Runtime 1.14 .......... MIN: 37.0 AVG: 53.0 MAX: 55.0
ONNX Runtime 1.14 - No AMX . MIN: 37.0 AVG: 53.2 MAX: 55.0
ONNX Runtime 1.14 use_dnnl . MIN: 36.0 AVG: 53.2 MAX: 55.0
ONNX Runtime 1.14
CPU Power Consumption Monitor
Watts < Lower Is Better
ONNX Runtime 1.14 .......... MIN: 209 AVG: 437 MAX: 451
ONNX Runtime 1.14 - No AMX . MIN: 199 AVG: 439 MAX: 454
ONNX Runtime 1.14 use_dnnl . MIN: 199 AVG: 440 MAX: 455
ONNX Runtime 1.14
CPU Peak Freq (Highest CPU Core Frequency) Monitor
Megahertz > Higher Is Better
ONNX Runtime 1.14 .......... MIN: 1900 AVG: 3457 MAX: 3530
ONNX Runtime 1.14 - No AMX . MIN: 1900 AVG: 3450 MAX: 3516
ONNX Runtime 1.14 use_dnnl . MIN: 1900 AVG: 3457 MAX: 3517
ONNX Runtime 1.14
CPU Temperature Monitor
Celsius < Lower Is Better
ONNX Runtime 1.14 .......... MIN: 39.0 AVG: 48.2 MAX: 50.0
ONNX Runtime 1.14 - No AMX . MIN: 39.0 AVG: 48.5 MAX: 50.0
ONNX Runtime 1.14 use_dnnl . MIN: 38.0 AVG: 48.5 MAX: 51.0
ONNX Runtime 1.14
CPU Power Consumption Monitor
Watts < Lower Is Better
ONNX Runtime 1.14 .......... MIN: 195 AVG: 611 MAX: 638
ONNX Runtime 1.14 - No AMX . MIN: 210 AVG: 611 MAX: 638
ONNX Runtime 1.14 use_dnnl . MIN: 199 AVG: 613 MAX: 641
ONNX Runtime 1.14
CPU Peak Freq (Highest CPU Core Frequency) Monitor
Megahertz > Higher Is Better
ONNX Runtime 1.14 .......... MIN: 1900 AVG: 2922 MAX: 3551
ONNX Runtime 1.14 - No AMX . MIN: 1900 AVG: 2905 MAX: 3508
ONNX Runtime 1.14 use_dnnl . MIN: 1900 AVG: 2916 MAX: 3677
ONNX Runtime 1.14
CPU Temperature Monitor
Celsius < Lower Is Better
ONNX Runtime 1.14 .......... MIN: 36.0 AVG: 51.5 MAX: 54.0
ONNX Runtime 1.14 - No AMX . MIN: 36.0 AVG: 51.0 MAX: 54.0
ONNX Runtime 1.14 use_dnnl . MIN: 36.0 AVG: 51.5 MAX: 54.0
ONNX Runtime 1.14
CPU Power Consumption Monitor
Watts < Lower Is Better
ONNX Runtime 1.14 .......... MIN: 122 AVG: 669 MAX: 709
ONNX Runtime 1.14 - No AMX . MIN: 203 AVG: 674 MAX: 707
ONNX Runtime 1.14 use_dnnl . MIN: 195 AVG: 669 MAX: 709
ONNX Runtime 1.14
CPU Peak Freq (Highest CPU Core Frequency) Monitor
Megahertz > Higher Is Better
ONNX Runtime 1.14 .......... MIN: 1900 AVG: 2753 MAX: 3514
ONNX Runtime 1.14 - No AMX . MIN: 1900 AVG: 2751 MAX: 3515
ONNX Runtime 1.14 use_dnnl . MIN: 1900 AVG: 2771 MAX: 3511
ONNX Runtime 1.14
CPU Temperature Monitor
Celsius < Lower Is Better
ONNX Runtime 1.14 .......... MIN: 36.0 AVG: 47.2 MAX: 50.0
ONNX Runtime 1.14 - No AMX . MIN: 37.0 AVG: 47.7 MAX: 50.0
ONNX Runtime 1.14 use_dnnl . MIN: 36.0 AVG: 47.5 MAX: 50.0
ONNX Runtime 1.14
CPU Power Consumption Monitor
Watts < Lower Is Better
ONNX Runtime 1.14 .......... MIN: 198 AVG: 611 MAX: 637
ONNX Runtime 1.14 - No AMX . MIN: 210 AVG: 612 MAX: 639
ONNX Runtime 1.14 use_dnnl . MIN: 203 AVG: 612 MAX: 638
ONNX Runtime 1.14
CPU Peak Freq (Highest CPU Core Frequency) Monitor
Megahertz > Higher Is Better
ONNX Runtime 1.14 .......... MIN: 1900 AVG: 2904 MAX: 3509
ONNX Runtime 1.14 - No AMX . MIN: 1900 AVG: 2917 MAX: 3506
ONNX Runtime 1.14 use_dnnl . MIN: 1900 AVG: 2921 MAX: 3515
ONNX Runtime 1.14
CPU Temperature Monitor
Celsius < Lower Is Better
ONNX Runtime 1.14 .......... MIN: 28.0 AVG: 46.2 MAX: 50.0
ONNX Runtime 1.14 - No AMX . MIN: 29.0 AVG: 48.2 MAX: 51.0
ONNX Runtime 1.14 use_dnnl . MIN: 33.0 AVG: 48.4 MAX: 51.0
ONNX Runtime 1.14
CPU Power Consumption Monitor
Watts < Lower Is Better
ONNX Runtime 1.14 .......... MIN: 203 AVG: 611 MAX: 638
ONNX Runtime 1.14 - No AMX . MIN: 201 AVG: 614 MAX: 639
ONNX Runtime 1.14 use_dnnl . MIN: 197 AVG: 613 MAX: 639
ONNX Runtime 1.14
CPU Peak Freq (Highest CPU Core Frequency) Monitor
Megahertz > Higher Is Better
ONNX Runtime 1.14 .......... MIN: 1900 AVG: 2919 MAX: 3512
ONNX Runtime 1.14 - No AMX . MIN: 1900 AVG: 2905 MAX: 3511
ONNX Runtime 1.14 use_dnnl . MIN: 1900 AVG: 2911 MAX: 3514