AMD Ryzen Threadripper 3990X 64-Core testing with a Gigabyte TRX40 AORUS PRO WIFI (F4p BIOS) and AMD Radeon RX 5700 8GB on Ubuntu 22.10 via the Phoronix Test Suite.
Compare your own system(s) to this result file with the
Phoronix Test Suite by running the command:
phoronix-test-suite benchmark 2301232-NE-DEEPSPARS10
deepsparse tr
AMD Ryzen Threadripper 3990X 64-Core testing with a Gigabyte TRX40 AORUS PRO WIFI (F4p BIOS) and AMD Radeon RX 5700 8GB on Ubuntu 22.10 via the Phoronix Test Suite.
a:
Processor: AMD Ryzen Threadripper 3990X 64-Core @ 2.90GHz (64 Cores / 128 Threads), Motherboard: Gigabyte TRX40 AORUS PRO WIFI (F4p BIOS), Chipset: AMD Starship/Matisse, Memory: 128GB, Disk: Samsung SSD 970 EVO Plus 500GB, Graphics: AMD Radeon RX 5700 8GB (1750/875MHz), Audio: AMD Navi 10 HDMI Audio, Monitor: DELL P2415Q, Network: Intel I211 + Intel Wi-Fi 6 AX200
OS: Ubuntu 22.10, Kernel: 6.1.0-rc8-phx-mglru (x86_64), Desktop: GNOME Shell 43.0, Display Server: X Server 1.21.1.4 + Wayland, OpenGL: 4.6 Mesa 22.2.1 (LLVM 15.0.2 DRM 3.49), Vulkan: 1.3.224, Compiler: GCC 12.2.0 + LLVM 15.0.2, File-System: ext4, Screen Resolution: 3840x2160
b:
Processor: AMD Ryzen Threadripper 3990X 64-Core @ 2.90GHz (64 Cores / 128 Threads), Motherboard: Gigabyte TRX40 AORUS PRO WIFI (F4p BIOS), Chipset: AMD Starship/Matisse, Memory: 128GB, Disk: Samsung SSD 970 EVO Plus 500GB, Graphics: AMD Radeon RX 5700 8GB (1750/875MHz), Audio: AMD Navi 10 HDMI Audio, Monitor: DELL P2415Q, Network: Intel I211 + Intel Wi-Fi 6 AX200
OS: Ubuntu 22.10, Kernel: 6.1.0-rc8-phx-mglru (x86_64), Desktop: GNOME Shell 43.0, Display Server: X Server 1.21.1.4 + Wayland, OpenGL: 4.6 Mesa 22.2.1 (LLVM 15.0.2 DRM 3.49), Vulkan: 1.3.224, Compiler: GCC 12.2.0 + LLVM 15.0.2, File-System: ext4, Screen Resolution: 3840x2160
c:
Processor: AMD Ryzen Threadripper 3990X 64-Core @ 2.90GHz (64 Cores / 128 Threads), Motherboard: Gigabyte TRX40 AORUS PRO WIFI (F4p BIOS), Chipset: AMD Starship/Matisse, Memory: 128GB, Disk: Samsung SSD 970 EVO Plus 500GB, Graphics: AMD Radeon RX 5700 8GB (1750/875MHz), Audio: AMD Navi 10 HDMI Audio, Monitor: DELL P2415Q, Network: Intel I211 + Intel Wi-Fi 6 AX200
OS: Ubuntu 22.10, Kernel: 6.1.0-rc8-phx-mglru (x86_64), Desktop: GNOME Shell 43.0, Display Server: X Server 1.21.1.4 + Wayland, OpenGL: 4.6 Mesa 22.2.1 (LLVM 15.0.2 DRM 3.49), Vulkan: 1.3.224, Compiler: GCC 12.2.0 + LLVM 15.0.2, File-System: ext4, Screen Resolution: 3840x2160
Neural Magic DeepSparse 1.3.2
Model: NLP Sentiment Analysis, 80% Pruned Quantized BERT Base Uncased - Scenario: Asynchronous Multi-Stream
ms/batch < Lower Is Better
a . 121.60 |===================================================================
b . 117.87 |=================================================================
c . 115.78 |================================================================
Neural Magic DeepSparse 1.3.2
Model: NLP Sentiment Analysis, 80% Pruned Quantized BERT Base Uncased - Scenario: Asynchronous Multi-Stream
items/sec > Higher Is Better
a . 263.27 |================================================================
b . 271.23 |==================================================================
c . 276.20 |===================================================================
Neural Magic DeepSparse 1.3.2
Model: NLP Question Answering, BERT base uncased SQuaD 12layer Pruned90 - Scenario: Asynchronous Multi-Stream
ms/batch < Lower Is Better
a . 360.55 |===================================================================
b . 359.24 |===================================================================
c . 358.03 |===================================================================
Neural Magic DeepSparse 1.3.2
Model: NLP Question Answering, BERT base uncased SQuaD 12layer Pruned90 - Scenario: Asynchronous Multi-Stream
items/sec > Higher Is Better
a . 88.55 |====================================================================
b . 88.77 |====================================================================
c . 89.07 |====================================================================
Neural Magic DeepSparse 1.3.2
Model: CV Segmentation, 90% Pruned YOLACT Pruned - Scenario: Asynchronous Multi-Stream
ms/batch < Lower Is Better
a . 728.34 |===================================================================
b . 725.26 |===================================================================
c . 724.52 |===================================================================
Neural Magic DeepSparse 1.3.2
Model: CV Segmentation, 90% Pruned YOLACT Pruned - Scenario: Asynchronous Multi-Stream
items/sec > Higher Is Better
a . 43.54 |===================================================================
b . 43.87 |====================================================================
c . 43.73 |====================================================================
Neural Magic DeepSparse 1.3.2
Model: NLP Document Classification, oBERT base uncased on IMDB - Scenario: Asynchronous Multi-Stream
ms/batch < Lower Is Better
a . 911.39 |==================================================================
b . 916.60 |==================================================================
c . 929.83 |===================================================================
Neural Magic DeepSparse 1.3.2
Model: NLP Document Classification, oBERT base uncased on IMDB - Scenario: Asynchronous Multi-Stream
items/sec > Higher Is Better
a . 34.89 |====================================================================
b . 34.50 |===================================================================
c . 34.21 |===================================================================
Neural Magic DeepSparse 1.3.2
Model: NLP Token Classification, BERT base uncased conll2003 - Scenario: Asynchronous Multi-Stream
ms/batch < Lower Is Better
a . 943.82 |===================================================================
b . 941.51 |===================================================================
c . 937.82 |===================================================================
Neural Magic DeepSparse 1.3.2
Model: NLP Token Classification, BERT base uncased conll2003 - Scenario: Asynchronous Multi-Stream
items/sec > Higher Is Better
a . 33.49 |====================================================================
b . 33.63 |====================================================================
c . 33.68 |====================================================================
Neural Magic DeepSparse 1.3.2
Model: NLP Text Classification, BERT base uncased SST2 - Scenario: Asynchronous Multi-Stream
ms/batch < Lower Is Better
a . 224.91 |===================================================================
b . 222.26 |==================================================================
c . 222.60 |==================================================================
Neural Magic DeepSparse 1.3.2
Model: NLP Text Classification, BERT base uncased SST2 - Scenario: Asynchronous Multi-Stream
items/sec > Higher Is Better
a . 142.21 |==================================================================
b . 143.93 |===================================================================
c . 143.71 |===================================================================
Neural Magic DeepSparse 1.3.2
Model: NLP Question Answering, BERT base uncased SQuaD 12layer Pruned90 - Scenario: Synchronous Single-Stream
ms/batch < Lower Is Better
a . 48.01 |====================================================================
b . 47.35 |===================================================================
c . 47.08 |===================================================================
Neural Magic DeepSparse 1.3.2
Model: NLP Question Answering, BERT base uncased SQuaD 12layer Pruned90 - Scenario: Synchronous Single-Stream
items/sec > Higher Is Better
a . 20.83 |===================================================================
b . 21.11 |====================================================================
c . 21.23 |====================================================================
Neural Magic DeepSparse 1.3.2
Model: NLP Sentiment Analysis, 80% Pruned Quantized BERT Base Uncased - Scenario: Synchronous Single-Stream
ms/batch < Lower Is Better
a . 21.08 |====================================================================
b . 20.11 |=================================================================
c . 20.13 |=================================================================
Neural Magic DeepSparse 1.3.2
Model: NLP Sentiment Analysis, 80% Pruned Quantized BERT Base Uncased - Scenario: Synchronous Single-Stream
items/sec > Higher Is Better
a . 47.42 |=================================================================
b . 49.70 |====================================================================
c . 49.66 |====================================================================
Neural Magic DeepSparse 1.3.2
Model: NLP Text Classification, BERT base uncased SST2 - Scenario: Synchronous Single-Stream
ms/batch < Lower Is Better
a . 32.73 |====================================================================
b . 32.87 |====================================================================
c . 32.25 |===================================================================
Neural Magic DeepSparse 1.3.2
Model: NLP Text Classification, BERT base uncased SST2 - Scenario: Synchronous Single-Stream
items/sec > Higher Is Better
a . 30.55 |===================================================================
b . 30.42 |===================================================================
c . 31.00 |====================================================================
Neural Magic DeepSparse 1.3.2
Model: NLP Document Classification, oBERT base uncased on IMDB - Scenario: Synchronous Single-Stream
ms/batch < Lower Is Better
a . 73.16 |====================================================================
b . 73.43 |====================================================================
c . 72.61 |===================================================================
Neural Magic DeepSparse 1.3.2
Model: NLP Document Classification, oBERT base uncased on IMDB - Scenario: Synchronous Single-Stream
items/sec > Higher Is Better
a . 13.67 |====================================================================
b . 13.62 |===================================================================
c . 13.77 |====================================================================
Neural Magic DeepSparse 1.3.2
Model: NLP Token Classification, BERT base uncased conll2003 - Scenario: Synchronous Single-Stream
ms/batch < Lower Is Better
a . 74.59 |====================================================================
b . 72.15 |==================================================================
c . 72.85 |==================================================================
Neural Magic DeepSparse 1.3.2
Model: NLP Token Classification, BERT base uncased conll2003 - Scenario: Synchronous Single-Stream
items/sec > Higher Is Better
a . 13.40 |==================================================================
b . 13.86 |====================================================================
c . 13.73 |===================================================================
Neural Magic DeepSparse 1.3.2
Model: CV Segmentation, 90% Pruned YOLACT Pruned - Scenario: Synchronous Single-Stream
ms/batch < Lower Is Better
a . 54.62 |====================================================================
b . 54.55 |====================================================================
c . 54.45 |====================================================================
Neural Magic DeepSparse 1.3.2
Model: CV Segmentation, 90% Pruned YOLACT Pruned - Scenario: Synchronous Single-Stream
items/sec > Higher Is Better
a . 18.30 |====================================================================
b . 18.33 |====================================================================
c . 18.36 |====================================================================
Neural Magic DeepSparse 1.3.2
Model: NLP Text Classification, DistilBERT mnli - Scenario: Asynchronous Multi-Stream
ms/batch < Lower Is Better
a . 114.46 |===================================================================
b . 112.57 |==================================================================
c . 112.89 |==================================================================
Neural Magic DeepSparse 1.3.2
Model: NLP Text Classification, DistilBERT mnli - Scenario: Asynchronous Multi-Stream
items/sec > Higher Is Better
a . 279.35 |==================================================================
b . 284.14 |===================================================================
c . 283.32 |===================================================================
Neural Magic DeepSparse 1.3.2
Model: NLP Text Classification, DistilBERT mnli - Scenario: Synchronous Single-Stream
ms/batch < Lower Is Better
a . 17.28 |====================================================================
b . 16.63 |=================================================================
c . 16.80 |==================================================================
Neural Magic DeepSparse 1.3.2
Model: NLP Text Classification, DistilBERT mnli - Scenario: Synchronous Single-Stream
items/sec > Higher Is Better
a . 57.86 |=================================================================
b . 60.11 |====================================================================
c . 59.50 |===================================================================
Neural Magic DeepSparse 1.3.2
Model: CV Detection, YOLOv5s COCO - Scenario: Synchronous Single-Stream
ms/batch < Lower Is Better
a . 15.69 |=================================================================
b . 16.44 |====================================================================
c . 16.05 |==================================================================
Neural Magic DeepSparse 1.3.2
Model: CV Detection, YOLOv5s COCO - Scenario: Synchronous Single-Stream
items/sec > Higher Is Better
a . 63.67 |====================================================================
b . 60.78 |=================================================================
c . 62.26 |==================================================================
Neural Magic DeepSparse 1.3.2
Model: CV Detection, YOLOv5s COCO - Scenario: Asynchronous Multi-Stream
ms/batch < Lower Is Better
a . 185.76 |===================================================================
b . 185.93 |===================================================================
c . 184.79 |===================================================================
Neural Magic DeepSparse 1.3.2
Model: CV Detection, YOLOv5s COCO - Scenario: Asynchronous Multi-Stream
items/sec > Higher Is Better
a . 171.82 |===================================================================
b . 171.55 |===================================================================
c . 172.62 |===================================================================
Neural Magic DeepSparse 1.3.2
Model: CV Classification, ResNet-50 ImageNet - Scenario: Synchronous Single-Stream
ms/batch < Lower Is Better
a . 10.56 |===================================================================
b . 10.51 |===================================================================
c . 10.67 |====================================================================
Neural Magic DeepSparse 1.3.2
Model: CV Classification, ResNet-50 ImageNet - Scenario: Synchronous Single-Stream
items/sec > Higher Is Better
a . 94.58 |====================================================================
b . 95.08 |====================================================================
c . 93.62 |===================================================================
Neural Magic DeepSparse 1.3.2
Model: CV Classification, ResNet-50 ImageNet - Scenario: Asynchronous Multi-Stream
ms/batch < Lower Is Better
a . 77.17 |====================================================================
b . 76.91 |====================================================================
c . 77.15 |====================================================================
Neural Magic DeepSparse 1.3.2
Model: CV Classification, ResNet-50 ImageNet - Scenario: Asynchronous Multi-Stream
items/sec > Higher Is Better
a . 414.43 |===================================================================
b . 415.63 |===================================================================
c . 414.55 |===================================================================