AMD Ryzen 9 7950X 16-Core testing with a ASUS ROG STRIX X670E-E GAMING WIFI (1905 BIOS) and NVIDIA GeForce RTX 3080 10GB on Ubuntu 23.10 via the Phoronix Test Suite.
Compare your own system(s) to this result file with the
Phoronix Test Suite by running the command:
phoronix-test-suite benchmark 2403151-PTS-DEEPSPAA58
deepspaarse 17
AMD Ryzen 9 7950X 16-Core testing with a ASUS ROG STRIX X670E-E GAMING WIFI (1905 BIOS) and NVIDIA GeForce RTX 3080 10GB on Ubuntu 23.10 via the Phoronix Test Suite.
a:
Processor: AMD Ryzen 9 7950X 16-Core @ 5.88GHz (16 Cores / 32 Threads), Motherboard: ASUS ROG STRIX X670E-E GAMING WIFI (1905 BIOS), Chipset: AMD Device 14d8, Memory: 2 x 16GB DRAM-6000MT/s G Skill F5-6000J3038F16G, Disk: 2000GB Samsung SSD 980 PRO 2TB + Western Digital WD_BLACK SN850X 2000GB, Graphics: NVIDIA GeForce RTX 3080 10GB, Audio: NVIDIA GA102 HD Audio, Monitor: DELL U2723QE, Network: Intel I225-V + Intel Wi-Fi 6 AX210/AX211/AX411
OS: Ubuntu 23.10, Kernel: 6.7.0-060700-generic (x86_64), Desktop: GNOME Shell 45.2, Display Server: X Server 1.21.1.7, Display Driver: NVIDIA 550.54.14, OpenGL: 4.6.0, OpenCL: OpenCL 3.0 CUDA 12.4.89, Compiler: GCC 13.2.0, File-System: ext4, Screen Resolution: 3840x2160
b:
Processor: AMD Ryzen 9 7950X 16-Core @ 5.88GHz (16 Cores / 32 Threads), Motherboard: ASUS ROG STRIX X670E-E GAMING WIFI (1905 BIOS), Chipset: AMD Device 14d8, Memory: 2 x 16GB DRAM-6000MT/s G Skill F5-6000J3038F16G, Disk: 2000GB Samsung SSD 980 PRO 2TB + Western Digital WD_BLACK SN850X 2000GB, Graphics: NVIDIA GeForce RTX 3080 10GB, Audio: NVIDIA GA102 HD Audio, Monitor: DELL U2723QE, Network: Intel I225-V + Intel Wi-Fi 6 AX210/AX211/AX411
OS: Ubuntu 23.10, Kernel: 6.7.0-060700-generic (x86_64), Desktop: GNOME Shell 45.2, Display Server: X Server 1.21.1.7, Display Driver: NVIDIA 550.54.14, OpenGL: 4.6.0, OpenCL: OpenCL 3.0 CUDA 12.4.89, Compiler: GCC 13.2.0, File-System: ext4, Screen Resolution: 3840x2160
c:
Processor: AMD Ryzen 9 7950X 16-Core @ 5.88GHz (16 Cores / 32 Threads), Motherboard: ASUS ROG STRIX X670E-E GAMING WIFI (1905 BIOS), Chipset: AMD Device 14d8, Memory: 2 x 16GB DRAM-6000MT/s G Skill F5-6000J3038F16G, Disk: 2000GB Samsung SSD 980 PRO 2TB + Western Digital WD_BLACK SN850X 2000GB, Graphics: NVIDIA GeForce RTX 3080 10GB, Audio: NVIDIA GA102 HD Audio, Monitor: DELL U2723QE, Network: Intel I225-V + Intel Wi-Fi 6 AX210/AX211/AX411
OS: Ubuntu 23.10, Kernel: 6.7.0-060700-generic (x86_64), Desktop: GNOME Shell 45.2, Display Server: X Server 1.21.1.7, Display Driver: NVIDIA 550.54.14, OpenGL: 4.6.0, OpenCL: OpenCL 3.0 CUDA 12.4.89, Compiler: GCC 13.2.0, File-System: ext4, Screen Resolution: 3840x2160
d:
Processor: AMD Ryzen 9 7950X 16-Core @ 5.88GHz (16 Cores / 32 Threads), Motherboard: ASUS ROG STRIX X670E-E GAMING WIFI (1905 BIOS), Chipset: AMD Device 14d8, Memory: 2 x 16GB DRAM-6000MT/s G Skill F5-6000J3038F16G, Disk: 2000GB Samsung SSD 980 PRO 2TB + Western Digital WD_BLACK SN850X 2000GB, Graphics: NVIDIA GeForce RTX 3080 10GB, Audio: NVIDIA GA102 HD Audio, Monitor: DELL U2723QE, Network: Intel I225-V + Intel Wi-Fi 6 AX210/AX211/AX411
OS: Ubuntu 23.10, Kernel: 6.7.0-060700-generic (x86_64), Desktop: GNOME Shell 45.2, Display Server: X Server 1.21.1.7, Display Driver: NVIDIA 550.54.14, OpenGL: 4.6.0, OpenCL: OpenCL 3.0 CUDA 12.4.89, Compiler: GCC 13.2.0, File-System: ext4, Screen Resolution: 3840x2160
e:
Processor: AMD Ryzen 9 7950X 16-Core @ 5.88GHz (16 Cores / 32 Threads), Motherboard: ASUS ROG STRIX X670E-E GAMING WIFI (1905 BIOS), Chipset: AMD Device 14d8, Memory: 2 x 16GB DRAM-6000MT/s G Skill F5-6000J3038F16G, Disk: 2000GB Samsung SSD 980 PRO 2TB + Western Digital WD_BLACK SN850X 2000GB, Graphics: NVIDIA GeForce RTX 3080 10GB, Audio: NVIDIA GA102 HD Audio, Monitor: DELL U2723QE, Network: Intel I225-V + Intel Wi-Fi 6 AX210/AX211/AX411
OS: Ubuntu 23.10, Kernel: 6.7.0-060700-generic (x86_64), Desktop: GNOME Shell 45.2, Display Server: X Server 1.21.1.7, Display Driver: NVIDIA 550.54.14, OpenGL: 4.6.0, OpenCL: OpenCL 3.0 CUDA 12.4.89, Compiler: GCC 13.2.0, File-System: ext4, Screen Resolution: 3840x2160
Neural Magic DeepSparse 1.7
Model: CV Classification, ResNet-50 ImageNet - Scenario: Asynchronous Multi-Stream
ms/batch < Lower Is Better
a . 28.62 |====================================================================
b . 28.51 |====================================================================
c . 28.46 |====================================================================
d . 28.51 |====================================================================
e . 28.31 |===================================================================
Neural Magic DeepSparse 1.7
Model: CV Classification, ResNet-50 ImageNet - Scenario: Asynchronous Multi-Stream
items/sec > Higher Is Better
a . 279.44 |==================================================================
b . 280.45 |===================================================================
c . 280.98 |===================================================================
d . 280.51 |===================================================================
e . 282.48 |===================================================================
Neural Magic DeepSparse 1.7
Model: Llama2 Chat 7b Quantized - Scenario: Asynchronous Multi-Stream
ms/batch < Lower Is Better
a . 1900.48 |==================================================================
b . 1894.22 |==================================================================
c . 1905.42 |==================================================================
d . 1908.61 |==================================================================
e . 1900.56 |==================================================================
Neural Magic DeepSparse 1.7
Model: Llama2 Chat 7b Quantized - Scenario: Asynchronous Multi-Stream
items/sec > Higher Is Better
a . 4.0959 |===================================================================
b . 4.1098 |===================================================================
c . 4.0853 |===================================================================
d . 4.0791 |==================================================================
e . 4.0962 |===================================================================
Neural Magic DeepSparse 1.7
Model: Llama2 Chat 7b Quantized - Scenario: Synchronous Single-Stream
ms/batch < Lower Is Better
a . 131.38 |===================================================================
b . 131.41 |===================================================================
c . 131.46 |===================================================================
d . 131.60 |===================================================================
e . 131.55 |===================================================================
Neural Magic DeepSparse 1.7
Model: Llama2 Chat 7b Quantized - Scenario: Synchronous Single-Stream
items/sec > Higher Is Better
a . 7.6108 |===================================================================
b . 7.6089 |===================================================================
c . 7.6060 |===================================================================
d . 7.5982 |===================================================================
e . 7.6006 |===================================================================
Neural Magic DeepSparse 1.7
Model: NLP Text Classification, BERT base uncased SST2, Sparse INT8 - Scenario: Synchronous Single-Stream
ms/batch < Lower Is Better
a . 3.3206 |===================================================================
b . 3.3168 |===================================================================
c . 3.3287 |===================================================================
d . 3.3113 |===================================================================
e . 3.3176 |===================================================================
Neural Magic DeepSparse 1.7
Model: NLP Text Classification, BERT base uncased SST2, Sparse INT8 - Scenario: Synchronous Single-Stream
items/sec > Higher Is Better
a . 300.87 |===================================================================
b . 301.24 |===================================================================
c . 300.11 |===================================================================
d . 301.70 |===================================================================
e . 301.17 |===================================================================
Neural Magic DeepSparse 1.7
Model: NLP Text Classification, BERT base uncased SST2, Sparse INT8 - Scenario: Asynchronous Multi-Stream
ms/batch < Lower Is Better
a . 8.5925 |==================================================================
b . 8.6079 |==================================================================
c . 8.6128 |==================================================================
d . 8.6817 |===================================================================
e . 8.6244 |===================================================================
Neural Magic DeepSparse 1.7
Model: NLP Text Classification, BERT base uncased SST2, Sparse INT8 - Scenario: Asynchronous Multi-Stream
items/sec > Higher Is Better
a . 929.85 |===================================================================
b . 928.17 |===================================================================
c . 927.61 |===================================================================
d . 920.28 |==================================================================
e . 926.39 |===================================================================
Neural Magic DeepSparse 1.7
Model: BERT-Large, NLP Question Answering, Sparse INT8 - Scenario: Asynchronous Multi-Stream
ms/batch < Lower Is Better
a . 18.57 |====================================================================
b . 18.51 |====================================================================
c . 18.55 |====================================================================
d . 18.56 |====================================================================
e . 18.63 |====================================================================
Neural Magic DeepSparse 1.7
Model: BERT-Large, NLP Question Answering, Sparse INT8 - Scenario: Asynchronous Multi-Stream
items/sec > Higher Is Better
a . 430.43 |===================================================================
b . 431.81 |===================================================================
c . 430.89 |===================================================================
d . 430.66 |===================================================================
e . 429.15 |===================================================================
Neural Magic DeepSparse 1.7
Model: BERT-Large, NLP Question Answering, Sparse INT8 - Scenario: Synchronous Single-Stream
ms/batch < Lower Is Better
a . 8.8521 |===================================================================
b . 8.8696 |===================================================================
c . 8.9004 |===================================================================
d . 8.8748 |===================================================================
e . 8.8780 |===================================================================
Neural Magic DeepSparse 1.7
Model: BERT-Large, NLP Question Answering, Sparse INT8 - Scenario: Synchronous Single-Stream
items/sec > Higher Is Better
a . 112.87 |===================================================================
b . 112.65 |===================================================================
c . 112.25 |===================================================================
d . 112.57 |===================================================================
e . 112.53 |===================================================================
Neural Magic DeepSparse 1.7
Model: NLP Token Classification, BERT base uncased conll2003 - Scenario: Synchronous Single-Stream
ms/batch < Lower Is Better
a . 53.76 |====================================================================
b . 53.93 |====================================================================
c . 53.71 |====================================================================
d . 53.60 |====================================================================
e . 53.71 |====================================================================
Neural Magic DeepSparse 1.7
Model: NLP Token Classification, BERT base uncased conll2003 - Scenario: Synchronous Single-Stream
items/sec > Higher Is Better
a . 18.60 |====================================================================
b . 18.54 |====================================================================
c . 18.61 |====================================================================
d . 18.65 |====================================================================
e . 18.62 |====================================================================
Neural Magic DeepSparse 1.7
Model: NLP Document Classification, oBERT base uncased on IMDB - Scenario: Asynchronous Multi-Stream
ms/batch < Lower Is Better
a . 369.73 |==================================================================
b . 376.36 |===================================================================
c . 375.87 |===================================================================
d . 376.52 |===================================================================
e . 374.97 |===================================================================
Neural Magic DeepSparse 1.7
Model: NLP Document Classification, oBERT base uncased on IMDB - Scenario: Asynchronous Multi-Stream
items/sec > Higher Is Better
a . 21.60 |====================================================================
b . 21.25 |===================================================================
c . 21.28 |===================================================================
d . 21.24 |===================================================================
e . 21.31 |===================================================================
Neural Magic DeepSparse 1.7
Model: NLP Token Classification, BERT base uncased conll2003 - Scenario: Asynchronous Multi-Stream
ms/batch < Lower Is Better
a . 371.57 |===================================================================
b . 370.83 |==================================================================
c . 374.08 |===================================================================
d . 371.95 |===================================================================
e . 372.49 |===================================================================
Neural Magic DeepSparse 1.7
Model: NLP Token Classification, BERT base uncased conll2003 - Scenario: Asynchronous Multi-Stream
items/sec > Higher Is Better
a . 21.51 |====================================================================
b . 21.47 |====================================================================
c . 21.32 |===================================================================
d . 21.45 |====================================================================
e . 21.42 |====================================================================
Neural Magic DeepSparse 1.7
Model: NLP Document Classification, oBERT base uncased on IMDB - Scenario: Synchronous Single-Stream
ms/batch < Lower Is Better
a . 53.15 |===================================================================
b . 53.59 |====================================================================
c . 53.49 |====================================================================
d . 53.42 |====================================================================
e . 53.57 |====================================================================
Neural Magic DeepSparse 1.7
Model: NLP Document Classification, oBERT base uncased on IMDB - Scenario: Synchronous Single-Stream
items/sec > Higher Is Better
a . 18.81 |====================================================================
b . 18.66 |===================================================================
c . 18.69 |====================================================================
d . 18.72 |====================================================================
e . 18.66 |===================================================================
Neural Magic DeepSparse 1.7
Model: CV Segmentation, 90% Pruned YOLACT Pruned - Scenario: Asynchronous Multi-Stream
ms/batch < Lower Is Better
a . 212.45 |===================================================================
b . 211.75 |===================================================================
c . 211.82 |===================================================================
d . 212.09 |===================================================================
e . 212.71 |===================================================================
Neural Magic DeepSparse 1.7
Model: CV Segmentation, 90% Pruned YOLACT Pruned - Scenario: Asynchronous Multi-Stream
items/sec > Higher Is Better
a . 37.65 |====================================================================
b . 37.78 |====================================================================
c . 37.76 |====================================================================
d . 37.72 |====================================================================
e . 37.61 |====================================================================
Neural Magic DeepSparse 1.7
Model: CV Segmentation, 90% Pruned YOLACT Pruned - Scenario: Synchronous Single-Stream
ms/batch < Lower Is Better
a . 31.66 |====================================================================
b . 31.57 |====================================================================
c . 31.57 |====================================================================
d . 31.53 |====================================================================
e . 31.67 |====================================================================
Neural Magic DeepSparse 1.7
Model: CV Segmentation, 90% Pruned YOLACT Pruned - Scenario: Synchronous Single-Stream
items/sec > Higher Is Better
a . 31.57 |====================================================================
b . 31.66 |====================================================================
c . 31.66 |====================================================================
d . 31.70 |====================================================================
e . 31.56 |====================================================================
Neural Magic DeepSparse 1.7
Model: NLP Text Classification, DistilBERT mnli - Scenario: Synchronous Single-Stream
ms/batch < Lower Is Better
a . 8.4942 |==================================================================
b . 8.5902 |==================================================================
c . 8.5743 |==================================================================
d . 8.6715 |===================================================================
e . 8.5269 |==================================================================
Neural Magic DeepSparse 1.7
Model: NLP Text Classification, DistilBERT mnli - Scenario: Synchronous Single-Stream
items/sec > Higher Is Better
a . 117.66 |===================================================================
b . 116.35 |==================================================================
c . 116.56 |==================================================================
d . 115.26 |==================================================================
e . 117.21 |===================================================================
Neural Magic DeepSparse 1.7
Model: NLP Text Classification, DistilBERT mnli - Scenario: Asynchronous Multi-Stream
ms/batch < Lower Is Better
a . 41.71 |====================================================================
b . 41.53 |===================================================================
c . 41.52 |===================================================================
d . 41.58 |===================================================================
e . 41.92 |====================================================================
Neural Magic DeepSparse 1.7
Model: NLP Text Classification, DistilBERT mnli - Scenario: Asynchronous Multi-Stream
items/sec > Higher Is Better
a . 191.70 |===================================================================
b . 192.51 |===================================================================
c . 192.61 |===================================================================
d . 192.31 |===================================================================
e . 190.79 |==================================================================
Neural Magic DeepSparse 1.7
Model: ResNet-50, Baseline - Scenario: Synchronous Single-Stream
ms/batch < Lower Is Better
a . 5.1149 |===================================================================
b . 5.0976 |===================================================================
c . 5.1095 |===================================================================
d . 5.0963 |===================================================================
e . 5.1030 |===================================================================
Neural Magic DeepSparse 1.7
Model: ResNet-50, Baseline - Scenario: Synchronous Single-Stream
items/sec > Higher Is Better
a . 195.29 |===================================================================
b . 195.96 |===================================================================
c . 195.50 |===================================================================
d . 196.02 |===================================================================
e . 195.75 |===================================================================
Neural Magic DeepSparse 1.7
Model: ResNet-50, Sparse INT8 - Scenario: Synchronous Single-Stream
ms/batch < Lower Is Better
a . 0.6937 |==================================================================
b . 0.6932 |==================================================================
c . 0.6899 |==================================================================
d . 0.7019 |===================================================================
e . 0.6957 |==================================================================
Neural Magic DeepSparse 1.7
Model: ResNet-50, Sparse INT8 - Scenario: Synchronous Single-Stream
items/sec > Higher Is Better
a . 1437.55 |==================================================================
b . 1438.27 |==================================================================
c . 1444.83 |==================================================================
d . 1420.74 |=================================================================
e . 1433.24 |=================================================================
Neural Magic DeepSparse 1.7
Model: ResNet-50, Sparse INT8 - Scenario: Asynchronous Multi-Stream
ms/batch < Lower Is Better
a . 3.3518 |==================================================================
b . 3.3559 |===================================================================
c . 3.3425 |==================================================================
d . 3.3488 |==================================================================
e . 3.3800 |===================================================================
Neural Magic DeepSparse 1.7
Model: ResNet-50, Sparse INT8 - Scenario: Asynchronous Multi-Stream
items/sec > Higher Is Better
a . 2379.95 |==================================================================
b . 2376.57 |==================================================================
c . 2385.77 |==================================================================
d . 2381.27 |==================================================================
e . 2359.66 |=================================================================
Neural Magic DeepSparse 1.7
Model: CV Detection, YOLOv5s COCO, Sparse INT8 - Scenario: Asynchronous Multi-Stream
ms/batch < Lower Is Better
a . 63.27 |====================================================================
b . 63.12 |====================================================================
c . 62.95 |====================================================================
d . 63.26 |====================================================================
e . 63.10 |====================================================================
Neural Magic DeepSparse 1.7
Model: CV Detection, YOLOv5s COCO, Sparse INT8 - Scenario: Asynchronous Multi-Stream
items/sec > Higher Is Better
a . 126.34 |===================================================================
b . 126.68 |===================================================================
c . 127.03 |===================================================================
d . 126.41 |===================================================================
e . 126.73 |===================================================================
Neural Magic DeepSparse 1.7
Model: CV Detection, YOLOv5s COCO, Sparse INT8 - Scenario: Synchronous Single-Stream
ms/batch < Lower Is Better
a . 10.10 |====================================================================
b . 10.05 |====================================================================
c . 10.05 |====================================================================
d . 10.05 |====================================================================
e . 10.09 |====================================================================
Neural Magic DeepSparse 1.7
Model: CV Detection, YOLOv5s COCO, Sparse INT8 - Scenario: Synchronous Single-Stream
items/sec > Higher Is Better
a . 99.00 |====================================================================
b . 99.50 |====================================================================
c . 99.41 |====================================================================
d . 99.50 |====================================================================
e . 99.10 |====================================================================
Neural Magic DeepSparse 1.7
Model: ResNet-50, Baseline - Scenario: Asynchronous Multi-Stream
ms/batch < Lower Is Better
a . 28.49 |====================================================================
b . 28.51 |====================================================================
c . 28.46 |====================================================================
d . 28.51 |====================================================================
e . 28.51 |====================================================================
Neural Magic DeepSparse 1.7
Model: ResNet-50, Baseline - Scenario: Asynchronous Multi-Stream
items/sec > Higher Is Better
a . 280.62 |===================================================================
b . 280.43 |===================================================================
c . 281.03 |===================================================================
d . 280.45 |===================================================================
e . 280.54 |===================================================================
Neural Magic DeepSparse 1.7
Model: CV Classification, ResNet-50 ImageNet - Scenario: Synchronous Single-Stream
ms/batch < Lower Is Better
a . 5.1378 |===================================================================
b . 5.1120 |===================================================================
c . 5.0880 |==================================================================
d . 5.0871 |==================================================================
e . 5.0981 |==================================================================
Neural Magic DeepSparse 1.7
Model: CV Classification, ResNet-50 ImageNet - Scenario: Synchronous Single-Stream
items/sec > Higher Is Better
a . 194.43 |==================================================================
b . 195.40 |===================================================================
c . 196.34 |===================================================================
d . 196.37 |===================================================================
e . 195.94 |===================================================================