Tests for a future article. AMD Ryzen Threadripper PRO 5965WX 24-Cores testing with a ASUS Pro WS WRX80E-SAGE SE WIFI (1201 BIOS) and ASUS NVIDIA NV106 2GB on Ubuntu 23.10 via the Phoronix Test Suite.
Compare your own system(s) to this result file with the
Phoronix Test Suite by running the command:
phoronix-test-suite benchmark 2402042-NE-ONNXTR51458
onnx tr
Tests for a future article. AMD Ryzen Threadripper PRO 5965WX 24-Cores testing with a ASUS Pro WS WRX80E-SAGE SE WIFI (1201 BIOS) and ASUS NVIDIA NV106 2GB on Ubuntu 23.10 via the Phoronix Test Suite.
a:
Processor: AMD Ryzen Threadripper PRO 5965WX 24-Cores @ 3.80GHz (24 Cores / 48 Threads), Motherboard: ASUS Pro WS WRX80E-SAGE SE WIFI (1201 BIOS), Chipset: AMD Starship/Matisse, Memory: 8 x 16GB DDR4-2133MT/s Corsair CMK32GX4M2E3200C16, Disk: 2048GB SOLIDIGM SSDPFKKW020X7, Graphics: ASUS NVIDIA NV106 2GB, Audio: AMD Starship/Matisse, Monitor: VA2431, Network: 2 x Intel X550 + Intel Wi-Fi 6 AX200
OS: Ubuntu 23.10, Kernel: 6.5.0-13-generic (x86_64), Desktop: GNOME Shell 45.0, Display Server: X Server + Wayland, Display Driver: nouveau, OpenGL: 4.3 Mesa 23.2.1-1ubuntu3, Compiler: GCC 13.2.0, File-System: ext4, Screen Resolution: 1920x1080
AMD Ryzen Threadripper PRO 5965WX 24-Cores - ASUS:
Processor: AMD Ryzen Threadripper PRO 5965WX 24-Cores @ 3.80GHz (24 Cores / 48 Threads), Motherboard: ASUS Pro WS WRX80E-SAGE SE WIFI (1201 BIOS), Chipset: AMD Starship/Matisse, Memory: 8 x 16GB DDR4-2133MT/s Corsair CMK32GX4M2E3200C16, Disk: 2048GB SOLIDIGM SSDPFKKW020X7, Graphics: ASUS NVIDIA NV106 2GB, Audio: AMD Starship/Matisse, Monitor: VA2431, Network: 2 x Intel X550 + Intel Wi-Fi 6 AX200
OS: Ubuntu 23.10, Kernel: 6.5.0-13-generic (x86_64), Desktop: GNOME Shell 45.0, Display Server: X Server + Wayland, Display Driver: nouveau, OpenGL: 4.3 Mesa 23.2.1-1ubuntu3, Compiler: GCC 13.2.0, File-System: ext4, Screen Resolution: 1920x1080
c:
Processor: AMD Ryzen Threadripper PRO 5965WX 24-Cores @ 3.80GHz (24 Cores / 48 Threads), Motherboard: ASUS Pro WS WRX80E-SAGE SE WIFI (1201 BIOS), Chipset: AMD Starship/Matisse, Memory: 8 x 16GB DDR4-2133MT/s Corsair CMK32GX4M2E3200C16, Disk: 2048GB SOLIDIGM SSDPFKKW020X7, Graphics: ASUS NVIDIA NV106 2GB, Audio: AMD Starship/Matisse, Monitor: VA2431, Network: 2 x Intel X550 + Intel Wi-Fi 6 AX200
OS: Ubuntu 23.10, Kernel: 6.5.0-13-generic (x86_64), Desktop: GNOME Shell 45.0, Display Server: X Server + Wayland, Display Driver: nouveau, OpenGL: 4.3 Mesa 23.2.1-1ubuntu3, Compiler: GCC 13.2.0, File-System: ext4, Screen Resolution: 1920x1080
ONNX Runtime 1.17
Model: GPT-2 - Device: CPU - Executor: Parallel
Inferences Per Second > Higher Is Better
a ................................................. 131.03 |==================
AMD Ryzen Threadripper PRO 5965WX 24-Cores - ASUS . 137.54 |===================
c ................................................. 131.26 |==================
ONNX Runtime 1.17
Model: GPT-2 - Device: CPU - Executor: Parallel
Inference Time Cost (ms) < Lower Is Better
a ................................................. 7.62456 |==================
AMD Ryzen Threadripper PRO 5965WX 24-Cores - ASUS . 7.26342 |=================
c ................................................. 7.61116 |==================
ONNX Runtime 1.17
Model: GPT-2 - Device: CPU - Executor: Standard
Inferences Per Second > Higher Is Better
a ................................................. 137.64 |==================
AMD Ryzen Threadripper PRO 5965WX 24-Cores - ASUS . 145.11 |===================
c ................................................. 146.78 |===================
ONNX Runtime 1.17
Model: GPT-2 - Device: CPU - Executor: Standard
Inference Time Cost (ms) < Lower Is Better
a ................................................. 7.26082 |==================
AMD Ryzen Threadripper PRO 5965WX 24-Cores - ASUS . 6.88710 |=================
c ................................................. 6.86398 |=================
ONNX Runtime 1.17
Model: yolov4 - Device: CPU - Executor: Parallel
Inferences Per Second > Higher Is Better
a ................................................. 9.54753 |==================
AMD Ryzen Threadripper PRO 5965WX 24-Cores - ASUS . 9.15942 |=================
c ................................................. 9.49134 |==================
ONNX Runtime 1.17
Model: yolov4 - Device: CPU - Executor: Parallel
Inference Time Cost (ms) < Lower Is Better
a ................................................. 104.74 |==================
AMD Ryzen Threadripper PRO 5965WX 24-Cores - ASUS . 109.17 |===================
c ................................................. 105.36 |==================
ONNX Runtime 1.17
Model: yolov4 - Device: CPU - Executor: Standard
Inferences Per Second > Higher Is Better
a ................................................. 10.34020 |=================
AMD Ryzen Threadripper PRO 5965WX 24-Cores - ASUS . 9.18180 |===============
c ................................................. 9.70516 |================
ONNX Runtime 1.17
Model: yolov4 - Device: CPU - Executor: Standard
Inference Time Cost (ms) < Lower Is Better
a ................................................. 96.71 |=================
AMD Ryzen Threadripper PRO 5965WX 24-Cores - ASUS . 108.91 |===================
c ................................................. 103.65 |==================
ONNX Runtime 1.17
Model: T5 Encoder - Device: CPU - Executor: Parallel
Inferences Per Second > Higher Is Better
a ................................................. 181.53 |===================
AMD Ryzen Threadripper PRO 5965WX 24-Cores - ASUS . 173.72 |==================
c ................................................. 176.90 |===================
ONNX Runtime 1.17
Model: T5 Encoder - Device: CPU - Executor: Parallel
Inference Time Cost (ms) < Lower Is Better
a ................................................. 5.50730 |=================
AMD Ryzen Threadripper PRO 5965WX 24-Cores - ASUS . 5.75502 |==================
c ................................................. 5.65170 |==================
ONNX Runtime 1.17
Model: T5 Encoder - Device: CPU - Executor: Standard
Inferences Per Second > Higher Is Better
a ................................................. 201.08 |===================
AMD Ryzen Threadripper PRO 5965WX 24-Cores - ASUS . 197.04 |===================
c ................................................. 193.94 |==================
ONNX Runtime 1.17
Model: T5 Encoder - Device: CPU - Executor: Standard
Inference Time Cost (ms) < Lower Is Better
a ................................................. 4.97262 |=================
AMD Ryzen Threadripper PRO 5965WX 24-Cores - ASUS . 5.07413 |==================
c ................................................. 5.15764 |==================
ONNX Runtime 1.17
Model: bertsquad-12 - Device: CPU - Executor: Parallel
Inferences Per Second > Higher Is Better
a ................................................. 12.53 |====================
AMD Ryzen Threadripper PRO 5965WX 24-Cores - ASUS . 12.73 |====================
c ................................................. 12.47 |====================
ONNX Runtime 1.17
Model: bertsquad-12 - Device: CPU - Executor: Parallel
Inference Time Cost (ms) < Lower Is Better
a ................................................. 79.80 |====================
AMD Ryzen Threadripper PRO 5965WX 24-Cores - ASUS . 78.56 |====================
c ................................................. 80.21 |====================
ONNX Runtime 1.17
Model: bertsquad-12 - Device: CPU - Executor: Standard
Inferences Per Second > Higher Is Better
a ................................................. 13.09 |====================
AMD Ryzen Threadripper PRO 5965WX 24-Cores - ASUS . 11.91 |==================
c ................................................. 13.35 |====================
ONNX Runtime 1.17
Model: bertsquad-12 - Device: CPU - Executor: Standard
Inference Time Cost (ms) < Lower Is Better
a ................................................. 76.38 |==================
AMD Ryzen Threadripper PRO 5965WX 24-Cores - ASUS . 83.94 |====================
c ................................................. 75.73 |==================
ONNX Runtime 1.17
Model: CaffeNet 12-int8 - Device: CPU - Executor: Parallel
Inferences Per Second > Higher Is Better
a ................................................. 687.28 |===================
AMD Ryzen Threadripper PRO 5965WX 24-Cores - ASUS . 686.14 |===================
c ................................................. 692.18 |===================
ONNX Runtime 1.17
Model: CaffeNet 12-int8 - Device: CPU - Executor: Parallel
Inference Time Cost (ms) < Lower Is Better
a ................................................. 1.45330 |==================
AMD Ryzen Threadripper PRO 5965WX 24-Cores - ASUS . 1.45579 |==================
c ................................................. 1.44343 |==================
ONNX Runtime 1.17
Model: CaffeNet 12-int8 - Device: CPU - Executor: Standard
Inferences Per Second > Higher Is Better
a ................................................. 794.13 |===================
AMD Ryzen Threadripper PRO 5965WX 24-Cores - ASUS . 662.19 |================
c ................................................. 763.04 |==================
ONNX Runtime 1.17
Model: CaffeNet 12-int8 - Device: CPU - Executor: Standard
Inference Time Cost (ms) < Lower Is Better
a ................................................. 1.25873 |===============
AMD Ryzen Threadripper PRO 5965WX 24-Cores - ASUS . 1.50952 |==================
c ................................................. 1.31876 |================
ONNX Runtime 1.17
Model: fcn-resnet101-11 - Device: CPU - Executor: Parallel
Inferences Per Second > Higher Is Better
a ................................................. 1.49191 |==================
AMD Ryzen Threadripper PRO 5965WX 24-Cores - ASUS . 1.44815 |=================
c ................................................. 1.48414 |==================
ONNX Runtime 1.17
Model: fcn-resnet101-11 - Device: CPU - Executor: Parallel
Inference Time Cost (ms) < Lower Is Better
a ................................................. 670.28 |==================
AMD Ryzen Threadripper PRO 5965WX 24-Cores - ASUS . 690.53 |===================
c ................................................. 673.79 |===================
ONNX Runtime 1.17
Model: fcn-resnet101-11 - Device: CPU - Executor: Standard
Inferences Per Second > Higher Is Better
a ................................................. 2.33114 |==============
AMD Ryzen Threadripper PRO 5965WX 24-Cores - ASUS . 2.32498 |==============
c ................................................. 2.95286 |==================
ONNX Runtime 1.17
Model: fcn-resnet101-11 - Device: CPU - Executor: Standard
Inference Time Cost (ms) < Lower Is Better
a ................................................. 428.97 |===================
AMD Ryzen Threadripper PRO 5965WX 24-Cores - ASUS . 430.11 |===================
c ................................................. 347.23 |===============
ONNX Runtime 1.17
Model: ArcFace ResNet-100 - Device: CPU - Executor: Parallel
Inferences Per Second > Higher Is Better
a ................................................. 29.95 |====================
AMD Ryzen Threadripper PRO 5965WX 24-Cores - ASUS . 29.28 |===================
c ................................................. 30.41 |====================
ONNX Runtime 1.17
Model: ArcFace ResNet-100 - Device: CPU - Executor: Parallel
Inference Time Cost (ms) < Lower Is Better
a ................................................. 33.39 |====================
AMD Ryzen Threadripper PRO 5965WX 24-Cores - ASUS . 34.15 |====================
c ................................................. 32.89 |===================
ONNX Runtime 1.17
Model: ArcFace ResNet-100 - Device: CPU - Executor: Standard
Inferences Per Second > Higher Is Better
a ................................................. 29.04 |==================
AMD Ryzen Threadripper PRO 5965WX 24-Cores - ASUS . 32.67 |====================
c ................................................. 32.28 |====================
ONNX Runtime 1.17
Model: ArcFace ResNet-100 - Device: CPU - Executor: Standard
Inference Time Cost (ms) < Lower Is Better
a ................................................. 34.43 |====================
AMD Ryzen Threadripper PRO 5965WX 24-Cores - ASUS . 30.60 |==================
c ................................................. 31.14 |==================
ONNX Runtime 1.17
Model: ResNet50 v1-12-int8 - Device: CPU - Executor: Parallel
Inferences Per Second > Higher Is Better
a ................................................. 233.83 |===================
AMD Ryzen Threadripper PRO 5965WX 24-Cores - ASUS . 233.76 |===================
c ................................................. 236.74 |===================
ONNX Runtime 1.17
Model: ResNet50 v1-12-int8 - Device: CPU - Executor: Parallel
Inference Time Cost (ms) < Lower Is Better
a ................................................. 4.27552 |==================
AMD Ryzen Threadripper PRO 5965WX 24-Cores - ASUS . 4.27661 |==================
c ................................................. 4.22286 |==================
ONNX Runtime 1.17
Model: ResNet50 v1-12-int8 - Device: CPU - Executor: Standard
Inferences Per Second > Higher Is Better
a ................................................. 263.28 |==================
AMD Ryzen Threadripper PRO 5965WX 24-Cores - ASUS . 254.40 |==================
c ................................................. 275.56 |===================
ONNX Runtime 1.17
Model: ResNet50 v1-12-int8 - Device: CPU - Executor: Standard
Inference Time Cost (ms) < Lower Is Better
a ................................................. 3.79772 |=================
AMD Ryzen Threadripper PRO 5965WX 24-Cores - ASUS . 3.93035 |==================
c ................................................. 3.64067 |=================
ONNX Runtime 1.17
Model: super-resolution-10 - Device: CPU - Executor: Parallel
Inferences Per Second > Higher Is Better
a ................................................. 121.57 |===================
AMD Ryzen Threadripper PRO 5965WX 24-Cores - ASUS . 119.33 |==================
c ................................................. 123.18 |===================
ONNX Runtime 1.17
Model: super-resolution-10 - Device: CPU - Executor: Parallel
Inference Time Cost (ms) < Lower Is Better
a ................................................. 8.22445 |==================
AMD Ryzen Threadripper PRO 5965WX 24-Cores - ASUS . 8.37853 |==================
c ................................................. 8.11762 |=================
ONNX Runtime 1.17
Model: super-resolution-10 - Device: CPU - Executor: Standard
Inferences Per Second > Higher Is Better
a ................................................. 99.84 |====================
AMD Ryzen Threadripper PRO 5965WX 24-Cores - ASUS . 95.02 |===================
ONNX Runtime 1.17
Model: super-resolution-10 - Device: CPU - Executor: Standard
Inference Time Cost (ms) < Lower Is Better
a ................................................. 10.02 |===================
AMD Ryzen Threadripper PRO 5965WX 24-Cores - ASUS . 10.52 |====================
ONNX Runtime 1.17
Model: Faster R-CNN R-50-FPN-int8 - Device: CPU - Executor: Parallel
Inferences Per Second > Higher Is Better
a ................................................. 30.71 |====================
AMD Ryzen Threadripper PRO 5965WX 24-Cores - ASUS . 31.14 |====================
ONNX Runtime 1.17
Model: Faster R-CNN R-50-FPN-int8 - Device: CPU - Executor: Parallel
Inference Time Cost (ms) < Lower Is Better
a ................................................. 32.57 |====================
AMD Ryzen Threadripper PRO 5965WX 24-Cores - ASUS . 32.11 |====================
ONNX Runtime 1.17
Model: Faster R-CNN R-50-FPN-int8 - Device: CPU - Executor: Standard
Inferences Per Second > Higher Is Better
a ................................................. 35.09 |===================
AMD Ryzen Threadripper PRO 5965WX 24-Cores - ASUS . 36.41 |====================
ONNX Runtime 1.17
Model: Faster R-CNN R-50-FPN-int8 - Device: CPU - Executor: Standard
Inference Time Cost (ms) < Lower Is Better
a ................................................. 28.49 |====================
AMD Ryzen Threadripper PRO 5965WX 24-Cores - ASUS . 27.46 |===================