deepsparse tr AMD Ryzen Threadripper 3990X 64-Core testing with a Gigabyte TRX40 AORUS PRO WIFI (F4p BIOS) and AMD Radeon RX 5700 8GB on Ubuntu 22.10 via the Phoronix Test Suite. ,,"a","b","c" Processor,,AMD Ryzen Threadripper 3990X 64-Core @ 2.90GHz (64 Cores / 128 Threads),AMD Ryzen Threadripper 3990X 64-Core @ 2.90GHz (64 Cores / 128 Threads),AMD Ryzen Threadripper 3990X 64-Core @ 2.90GHz (64 Cores / 128 Threads) Motherboard,,Gigabyte TRX40 AORUS PRO WIFI (F4p BIOS),Gigabyte TRX40 AORUS PRO WIFI (F4p BIOS),Gigabyte TRX40 AORUS PRO WIFI (F4p BIOS) Chipset,,AMD Starship/Matisse,AMD Starship/Matisse,AMD Starship/Matisse Memory,,128GB,128GB,128GB Disk,,Samsung SSD 970 EVO Plus 500GB,Samsung SSD 970 EVO Plus 500GB,Samsung SSD 970 EVO Plus 500GB Graphics,,AMD Radeon RX 5700 8GB (1750/875MHz),AMD Radeon RX 5700 8GB (1750/875MHz),AMD Radeon RX 5700 8GB (1750/875MHz) Audio,,AMD Navi 10 HDMI Audio,AMD Navi 10 HDMI Audio,AMD Navi 10 HDMI Audio Monitor,,DELL P2415Q,DELL P2415Q,DELL P2415Q Network,,Intel I211 + Intel Wi-Fi 6 AX200,Intel I211 + Intel Wi-Fi 6 AX200,Intel I211 + Intel Wi-Fi 6 AX200 OS,,Ubuntu 22.10,Ubuntu 22.10,Ubuntu 22.10 Kernel,,6.1.0-rc8-phx-mglru (x86_64),6.1.0-rc8-phx-mglru (x86_64),6.1.0-rc8-phx-mglru (x86_64) Desktop,,GNOME Shell 43.0,GNOME Shell 43.0,GNOME Shell 43.0 Display Server,,X Server 1.21.1.4 + Wayland,X Server 1.21.1.4 + Wayland,X Server 1.21.1.4 + Wayland OpenGL,,4.6 Mesa 22.2.1 (LLVM 15.0.2 DRM 3.49),4.6 Mesa 22.2.1 (LLVM 15.0.2 DRM 3.49),4.6 Mesa 22.2.1 (LLVM 15.0.2 DRM 3.49) Vulkan,,1.3.224,1.3.224,1.3.224 Compiler,,GCC 12.2.0 + LLVM 15.0.2,GCC 12.2.0 + LLVM 15.0.2,GCC 12.2.0 + LLVM 15.0.2 File-System,,ext4,ext4,ext4 Screen Resolution,,3840x2160,3840x2160,3840x2160 ,,"a","b","c" "Neural Magic DeepSparse - Model: NLP Sentiment Analysis, 80% Pruned Quantized BERT Base Uncased - Scenario: Asynchronous Multi-Stream (ms/batch)",LIB,121.5987,117.8674,115.7776 "Neural Magic DeepSparse - Model: NLP Sentiment Analysis, 80% Pruned Quantized BERT Base Uncased - Scenario: Asynchronous Multi-Stream (items/sec)",HIB,263.2698,271.2283,276.2048 "Neural Magic DeepSparse - Model: NLP Sentiment Analysis, 80% Pruned Quantized BERT Base Uncased - Scenario: Synchronous Single-Stream (ms/batch)",LIB,21.0843,20.1109,20.1262 "Neural Magic DeepSparse - Model: NLP Sentiment Analysis, 80% Pruned Quantized BERT Base Uncased - Scenario: Synchronous Single-Stream (items/sec)",HIB,47.4189,49.7013,49.6628 "Neural Magic DeepSparse - Model: CV Detection, YOLOv5s COCO - Scenario: Synchronous Single-Stream (ms/batch)",LIB,15.6921,16.4399,16.0481 "Neural Magic DeepSparse - Model: CV Detection, YOLOv5s COCO - Scenario: Synchronous Single-Stream (items/sec)",HIB,63.6711,60.7788,62.2604 "Neural Magic DeepSparse - Model: NLP Text Classification, DistilBERT mnli - Scenario: Synchronous Single-Stream (ms/batch)",LIB,17.2750,16.627,16.7985 "Neural Magic DeepSparse - Model: NLP Text Classification, DistilBERT mnli - Scenario: Synchronous Single-Stream (items/sec)",HIB,57.8622,60.1136,59.4981 "Neural Magic DeepSparse - Model: NLP Token Classification, BERT base uncased conll2003 - Scenario: Synchronous Single-Stream (ms/batch)",LIB,74.5910,72.1548,72.8486 "Neural Magic DeepSparse - Model: NLP Token Classification, BERT base uncased conll2003 - Scenario: Synchronous Single-Stream (items/sec)",HIB,13.4049,13.8574,13.7254 "Neural Magic DeepSparse - Model: NLP Document Classification, oBERT base uncased on IMDB - Scenario: Asynchronous Multi-Stream (ms/batch)",LIB,911.3941,916.5997,929.8296 "Neural Magic DeepSparse - Model: NLP Question Answering, BERT base uncased SQuaD 12layer Pruned90 - Scenario: Synchronous Single-Stream (ms/batch)",LIB,48.0131,47.3506,47.0812 "Neural Magic DeepSparse - Model: NLP Document Classification, oBERT base uncased on IMDB - Scenario: Asynchronous Multi-Stream (items/sec)",HIB,34.8860,34.4957,34.2097 "Neural Magic DeepSparse - Model: NLP Question Answering, BERT base uncased SQuaD 12layer Pruned90 - Scenario: Synchronous Single-Stream (items/sec)",HIB,20.8260,21.1142,21.2348 "Neural Magic DeepSparse - Model: NLP Text Classification, BERT base uncased SST2 - Scenario: Synchronous Single-Stream (items/sec)",HIB,30.5467,30.4182,31.0025 "Neural Magic DeepSparse - Model: NLP Text Classification, BERT base uncased SST2 - Scenario: Synchronous Single-Stream (ms/batch)",LIB,32.7294,32.8655,32.2466 "Neural Magic DeepSparse - Model: NLP Text Classification, DistilBERT mnli - Scenario: Asynchronous Multi-Stream (items/sec)",HIB,279.3454,284.1417,283.3205 "Neural Magic DeepSparse - Model: NLP Text Classification, DistilBERT mnli - Scenario: Asynchronous Multi-Stream (ms/batch)",LIB,114.4637,112.5672,112.8945 "Neural Magic DeepSparse - Model: CV Classification, ResNet-50 ImageNet - Scenario: Synchronous Single-Stream (ms/batch)",LIB,10.5637,10.5084,10.6724 "Neural Magic DeepSparse - Model: CV Classification, ResNet-50 ImageNet - Scenario: Synchronous Single-Stream (items/sec)",HIB,94.5820,95.0838,93.6235 "Neural Magic DeepSparse - Model: NLP Text Classification, BERT base uncased SST2 - Scenario: Asynchronous Multi-Stream (items/sec)",HIB,142.2132,143.9332,143.714 "Neural Magic DeepSparse - Model: NLP Text Classification, BERT base uncased SST2 - Scenario: Asynchronous Multi-Stream (ms/batch)",LIB,224.9088,222.2642,222.6035 "Neural Magic DeepSparse - Model: NLP Document Classification, oBERT base uncased on IMDB - Scenario: Synchronous Single-Stream (items/sec)",HIB,13.6680,13.6164,13.7707 "Neural Magic DeepSparse - Model: NLP Document Classification, oBERT base uncased on IMDB - Scenario: Synchronous Single-Stream (ms/batch)",LIB,73.1554,73.4315,72.6091 "Neural Magic DeepSparse - Model: CV Segmentation, 90% Pruned YOLACT Pruned - Scenario: Asynchronous Multi-Stream (items/sec)",HIB,43.5414,43.8726,43.7343 "Neural Magic DeepSparse - Model: NLP Question Answering, BERT base uncased SQuaD 12layer Pruned90 - Scenario: Asynchronous Multi-Stream (ms/batch)",LIB,360.5545,359.2449,358.0307 "Neural Magic DeepSparse - Model: NLP Token Classification, BERT base uncased conll2003 - Scenario: Asynchronous Multi-Stream (ms/batch)",LIB,943.8212,941.5145,937.8174 "Neural Magic DeepSparse - Model: CV Detection, YOLOv5s COCO - Scenario: Asynchronous Multi-Stream (items/sec)",HIB,171.8227,171.5536,172.622 "Neural Magic DeepSparse - Model: CV Detection, YOLOv5s COCO - Scenario: Asynchronous Multi-Stream (ms/batch)",LIB,185.7590,185.9302,184.7932 "Neural Magic DeepSparse - Model: NLP Token Classification, BERT base uncased conll2003 - Scenario: Asynchronous Multi-Stream (items/sec)",HIB,33.4855,33.6307,33.6845 "Neural Magic DeepSparse - Model: NLP Question Answering, BERT base uncased SQuaD 12layer Pruned90 - Scenario: Asynchronous Multi-Stream (items/sec)",HIB,88.5450,88.7699,89.0653 "Neural Magic DeepSparse - Model: CV Segmentation, 90% Pruned YOLACT Pruned - Scenario: Asynchronous Multi-Stream (ms/batch)",LIB,728.3385,725.2629,724.5184 "Neural Magic DeepSparse - Model: CV Classification, ResNet-50 ImageNet - Scenario: Asynchronous Multi-Stream (ms/batch)",LIB,77.1694,76.9125,77.1478 "Neural Magic DeepSparse - Model: CV Segmentation, 90% Pruned YOLACT Pruned - Scenario: Synchronous Single-Stream (ms/batch)",LIB,54.6204,54.5493,54.4505 "Neural Magic DeepSparse - Model: CV Segmentation, 90% Pruned YOLACT Pruned - Scenario: Synchronous Single-Stream (items/sec)",HIB,18.3022,18.3262,18.3593 "Neural Magic DeepSparse - Model: CV Classification, ResNet-50 ImageNet - Scenario: Asynchronous Multi-Stream (items/sec)",HIB,414.4296,415.6349,414.552