deepsparse-54 AMD Ryzen 9 3900X 12-Core testing with a ASUS ROG STRIX X570-E GAMING (4021 BIOS) and MSI NVIDIA GeForce GTX 1050 Ti 4GB on Ubuntu 20.04 via the Phoronix Test Suite.
Compare your own system(s) to this result file with the
Phoronix Test Suite by running the command:
phoronix-test-suite benchmark 2211300-NE-DEEPSPARS99 AMD Ryzen 9 3900X 12-Core - MSI NVIDIA GeForce GTX Processor: AMD Ryzen 9 3900X 12-Core @ 3.80GHz (12 Cores / 24 Threads), Motherboard: ASUS ROG STRIX X570-E GAMING (4021 BIOS), Chipset: AMD Starship/Matisse, Memory: 32GB, Disk: 500GB Western Digital WDS500G3X0C-00SJG0 + Samsung SSD 970 EVO Plus 500GB, Graphics: MSI NVIDIA GeForce GTX 1050 Ti 4GB, Audio: NVIDIA GP107GL HD Audio, Network: Realtek RTL8125 2.5GbE + Intel I211 + Intel Wi-Fi 6 AX200
OS: Ubuntu 20.04, Kernel: 5.4.0-132-generic (x86_64), Display Driver: NVIDIA, Vulkan: 1.1.182, Compiler: GCC 9.4.0, File-System: ext4, Screen Resolution: 1024x768
Kernel Notes: Transparent Huge Pages: madviseProcessor Notes: Scaling Governor: acpi-cpufreq ondemand (Boost: Enabled) - CPU Microcode: 0x8701021Python Notes: Python 3.8.10Security Notes: itlb_multihit: Not affected + l1tf: Not affected + mds: Not affected + meltdown: Not affected + mmio_stale_data: Not affected + retbleed: Vulnerable + spec_store_bypass: Mitigation of SSB disabled via prctl and seccomp + spectre_v1: Mitigation of usercopy/swapgs barriers and __user pointer sanitization + spectre_v2: Mitigation of Retpolines IBPB: conditional STIBP: conditional RSB filling PBRSB-eIBRS: Not affected + srbds: Not affected + tsx_async_abort: Not affected
deepsparse-54 OpenBenchmarking.org Phoronix Test Suite AMD Ryzen 9 3900X 12-Core @ 3.80GHz (12 Cores / 24 Threads) ASUS ROG STRIX X570-E GAMING (4021 BIOS) AMD Starship/Matisse 32GB 500GB Western Digital WDS500G3X0C-00SJG0 + Samsung SSD 970 EVO Plus 500GB MSI NVIDIA GeForce GTX 1050 Ti 4GB NVIDIA GP107GL HD Audio Realtek RTL8125 2.5GbE + Intel I211 + Intel Wi-Fi 6 AX200 Ubuntu 20.04 5.4.0-132-generic (x86_64) NVIDIA 1.1.182 GCC 9.4.0 ext4 1024x768 Processor Motherboard Chipset Memory Disk Graphics Audio Network OS Kernel Display Driver Vulkan Compiler File-System Screen Resolution Deepsparse-54 Benchmarks System Logs - Transparent Huge Pages: madvise - Scaling Governor: acpi-cpufreq ondemand (Boost: Enabled) - CPU Microcode: 0x8701021 - Python 3.8.10 - itlb_multihit: Not affected + l1tf: Not affected + mds: Not affected + meltdown: Not affected + mmio_stale_data: Not affected + retbleed: Vulnerable + spec_store_bypass: Mitigation of SSB disabled via prctl and seccomp + spectre_v1: Mitigation of usercopy/swapgs barriers and __user pointer sanitization + spectre_v2: Mitigation of Retpolines IBPB: conditional STIBP: conditional RSB filling PBRSB-eIBRS: Not affected + srbds: Not affected + tsx_async_abort: Not affected
deepsparse-54 deepsparse: NLP Document Classification, oBERT base uncased on IMDB - Asynchronous Multi-Stream deepsparse: NLP Document Classification, oBERT base uncased on IMDB - Asynchronous Multi-Stream deepsparse: NLP Document Classification, oBERT base uncased on IMDB - Synchronous Single-Stream deepsparse: NLP Document Classification, oBERT base uncased on IMDB - Synchronous Single-Stream deepsparse: NLP Question Answering, BERT base uncased SQuaD 12layer Pruned90 - Asynchronous Multi-Stream deepsparse: NLP Question Answering, BERT base uncased SQuaD 12layer Pruned90 - Asynchronous Multi-Stream deepsparse: NLP Question Answering, BERT base uncased SQuaD 12layer Pruned90 - Synchronous Single-Stream deepsparse: NLP Question Answering, BERT base uncased SQuaD 12layer Pruned90 - Synchronous Single-Stream deepsparse: CV Detection,YOLOv5s COCO - Asynchronous Multi-Stream deepsparse: CV Detection,YOLOv5s COCO - Asynchronous Multi-Stream deepsparse: CV Detection,YOLOv5s COCO - Synchronous Single-Stream deepsparse: CV Detection,YOLOv5s COCO - Synchronous Single-Stream deepsparse: CV Classification, ResNet-50 ImageNet - Asynchronous Multi-Stream deepsparse: CV Classification, ResNet-50 ImageNet - Asynchronous Multi-Stream deepsparse: CV Classification, ResNet-50 ImageNet - Synchronous Single-Stream deepsparse: CV Classification, ResNet-50 ImageNet - Synchronous Single-Stream deepsparse: NLP Text Classification, DistilBERT mnli - Asynchronous Multi-Stream deepsparse: NLP Text Classification, DistilBERT mnli - Asynchronous Multi-Stream deepsparse: NLP Text Classification, DistilBERT mnli - Synchronous Single-Stream deepsparse: NLP Text Classification, DistilBERT mnli - Synchronous Single-Stream deepsparse: NLP Text Classification, BERT base uncased SST2 - Asynchronous Multi-Stream deepsparse: NLP Text Classification, BERT base uncased SST2 - Asynchronous Multi-Stream deepsparse: NLP Text Classification, BERT base uncased SST2 - Synchronous Single-Stream deepsparse: NLP Text Classification, BERT base uncased SST2 - Synchronous Single-Stream deepsparse: NLP Token Classification, BERT base uncased conll2003 - Asynchronous Multi-Stream deepsparse: NLP Token Classification, BERT base uncased conll2003 - Asynchronous Multi-Stream deepsparse: NLP Token Classification, BERT base uncased conll2003 - Synchronous Single-Stream deepsparse: NLP Token Classification, BERT base uncased conll2003 - Synchronous Single-Stream AMD Ryzen 9 3900X 12-Core - MSI NVIDIA GeForce GTX 10.1252 589.4118 7.8989 126.5932 36.0780 166.2700 19.4622 51.3745 56.3606 106.3857 42.6638 23.4277 113.8984 52.6262 76.9237 12.9916 82.2290 72.9129 58.0689 17.2140 41.4159 144.7796 29.2849 34.1403 10.1663 588.7073 7.9030 126.5269 OpenBenchmarking.org
Neural Magic DeepSparse OpenBenchmarking.org items/sec, More Is Better Neural Magic DeepSparse 1.1 Model: NLP Document Classification, oBERT base uncased on IMDB - Scenario: Asynchronous Multi-Stream AMD Ryzen 9 3900X 12-Core - MSI NVIDIA GeForce GTX 3 6 9 12 15 SE +/- 0.05, N = 3 10.13
OpenBenchmarking.org ms/batch, Fewer Is Better Neural Magic DeepSparse 1.1 Model: NLP Document Classification, oBERT base uncased on IMDB - Scenario: Asynchronous Multi-Stream AMD Ryzen 9 3900X 12-Core - MSI NVIDIA GeForce GTX 130 260 390 520 650 SE +/- 2.34, N = 3 589.41
OpenBenchmarking.org items/sec, More Is Better Neural Magic DeepSparse 1.1 Model: NLP Document Classification, oBERT base uncased on IMDB - Scenario: Synchronous Single-Stream AMD Ryzen 9 3900X 12-Core - MSI NVIDIA GeForce GTX 2 4 6 8 10 SE +/- 0.0091, N = 3 7.8989
OpenBenchmarking.org ms/batch, Fewer Is Better Neural Magic DeepSparse 1.1 Model: NLP Document Classification, oBERT base uncased on IMDB - Scenario: Synchronous Single-Stream AMD Ryzen 9 3900X 12-Core - MSI NVIDIA GeForce GTX 30 60 90 120 150 SE +/- 0.15, N = 3 126.59
OpenBenchmarking.org items/sec, More Is Better Neural Magic DeepSparse 1.1 Model: NLP Question Answering, BERT base uncased SQuaD 12layer Pruned90 - Scenario: Asynchronous Multi-Stream AMD Ryzen 9 3900X 12-Core - MSI NVIDIA GeForce GTX 8 16 24 32 40 SE +/- 0.01, N = 3 36.08
OpenBenchmarking.org ms/batch, Fewer Is Better Neural Magic DeepSparse 1.1 Model: NLP Question Answering, BERT base uncased SQuaD 12layer Pruned90 - Scenario: Asynchronous Multi-Stream AMD Ryzen 9 3900X 12-Core - MSI NVIDIA GeForce GTX 40 80 120 160 200 SE +/- 0.03, N = 3 166.27
OpenBenchmarking.org items/sec, More Is Better Neural Magic DeepSparse 1.1 Model: NLP Question Answering, BERT base uncased SQuaD 12layer Pruned90 - Scenario: Synchronous Single-Stream AMD Ryzen 9 3900X 12-Core - MSI NVIDIA GeForce GTX 5 10 15 20 25 SE +/- 0.07, N = 3 19.46
OpenBenchmarking.org ms/batch, Fewer Is Better Neural Magic DeepSparse 1.1 Model: NLP Question Answering, BERT base uncased SQuaD 12layer Pruned90 - Scenario: Synchronous Single-Stream AMD Ryzen 9 3900X 12-Core - MSI NVIDIA GeForce GTX 12 24 36 48 60 SE +/- 0.18, N = 3 51.37
OpenBenchmarking.org items/sec, More Is Better Neural Magic DeepSparse 1.1 Model: CV Classification, ResNet-50 ImageNet - Scenario: Asynchronous Multi-Stream AMD Ryzen 9 3900X 12-Core - MSI NVIDIA GeForce GTX 30 60 90 120 150 SE +/- 0.15, N = 3 113.90
OpenBenchmarking.org ms/batch, Fewer Is Better Neural Magic DeepSparse 1.1 Model: CV Classification, ResNet-50 ImageNet - Scenario: Asynchronous Multi-Stream AMD Ryzen 9 3900X 12-Core - MSI NVIDIA GeForce GTX 12 24 36 48 60 SE +/- 0.07, N = 3 52.63
OpenBenchmarking.org items/sec, More Is Better Neural Magic DeepSparse 1.1 Model: CV Classification, ResNet-50 ImageNet - Scenario: Synchronous Single-Stream AMD Ryzen 9 3900X 12-Core - MSI NVIDIA GeForce GTX 20 40 60 80 100 SE +/- 0.15, N = 3 76.92
OpenBenchmarking.org items/sec, More Is Better Neural Magic DeepSparse 1.1 Model: NLP Text Classification, DistilBERT mnli - Scenario: Asynchronous Multi-Stream AMD Ryzen 9 3900X 12-Core - MSI NVIDIA GeForce GTX 20 40 60 80 100 SE +/- 0.20, N = 3 82.23
OpenBenchmarking.org ms/batch, Fewer Is Better Neural Magic DeepSparse 1.1 Model: NLP Text Classification, DistilBERT mnli - Scenario: Asynchronous Multi-Stream AMD Ryzen 9 3900X 12-Core - MSI NVIDIA GeForce GTX 16 32 48 64 80 SE +/- 0.16, N = 3 72.91
OpenBenchmarking.org items/sec, More Is Better Neural Magic DeepSparse 1.1 Model: NLP Text Classification, DistilBERT mnli - Scenario: Synchronous Single-Stream AMD Ryzen 9 3900X 12-Core - MSI NVIDIA GeForce GTX 13 26 39 52 65 SE +/- 0.04, N = 3 58.07
OpenBenchmarking.org ms/batch, Fewer Is Better Neural Magic DeepSparse 1.1 Model: NLP Text Classification, DistilBERT mnli - Scenario: Synchronous Single-Stream AMD Ryzen 9 3900X 12-Core - MSI NVIDIA GeForce GTX 4 8 12 16 20 SE +/- 0.01, N = 3 17.21
OpenBenchmarking.org items/sec, More Is Better Neural Magic DeepSparse 1.1 Model: NLP Text Classification, BERT base uncased SST2 - Scenario: Asynchronous Multi-Stream AMD Ryzen 9 3900X 12-Core - MSI NVIDIA GeForce GTX 9 18 27 36 45 SE +/- 0.08, N = 3 41.42
OpenBenchmarking.org ms/batch, Fewer Is Better Neural Magic DeepSparse 1.1 Model: NLP Text Classification, BERT base uncased SST2 - Scenario: Asynchronous Multi-Stream AMD Ryzen 9 3900X 12-Core - MSI NVIDIA GeForce GTX 30 60 90 120 150 SE +/- 0.30, N = 3 144.78
OpenBenchmarking.org items/sec, More Is Better Neural Magic DeepSparse 1.1 Model: NLP Text Classification, BERT base uncased SST2 - Scenario: Synchronous Single-Stream AMD Ryzen 9 3900X 12-Core - MSI NVIDIA GeForce GTX 7 14 21 28 35 SE +/- 0.01, N = 3 29.28
OpenBenchmarking.org ms/batch, Fewer Is Better Neural Magic DeepSparse 1.1 Model: NLP Text Classification, BERT base uncased SST2 - Scenario: Synchronous Single-Stream AMD Ryzen 9 3900X 12-Core - MSI NVIDIA GeForce GTX 8 16 24 32 40 SE +/- 0.01, N = 3 34.14
OpenBenchmarking.org items/sec, More Is Better Neural Magic DeepSparse 1.1 Model: NLP Token Classification, BERT base uncased conll2003 - Scenario: Asynchronous Multi-Stream AMD Ryzen 9 3900X 12-Core - MSI NVIDIA GeForce GTX 3 6 9 12 15 SE +/- 0.02, N = 3 10.17
OpenBenchmarking.org ms/batch, Fewer Is Better Neural Magic DeepSparse 1.1 Model: NLP Token Classification, BERT base uncased conll2003 - Scenario: Asynchronous Multi-Stream AMD Ryzen 9 3900X 12-Core - MSI NVIDIA GeForce GTX 130 260 390 520 650 SE +/- 0.55, N = 3 588.71
OpenBenchmarking.org items/sec, More Is Better Neural Magic DeepSparse 1.1 Model: NLP Token Classification, BERT base uncased conll2003 - Scenario: Synchronous Single-Stream AMD Ryzen 9 3900X 12-Core - MSI NVIDIA GeForce GTX 2 4 6 8 10 SE +/- 0.0080, N = 3 7.9030
OpenBenchmarking.org ms/batch, Fewer Is Better Neural Magic DeepSparse 1.1 Model: NLP Token Classification, BERT base uncased conll2003 - Scenario: Synchronous Single-Stream AMD Ryzen 9 3900X 12-Core - MSI NVIDIA GeForce GTX 30 60 90 120 150 SE +/- 0.13, N = 3 126.53
AMD Ryzen 9 3900X 12-Core - MSI NVIDIA GeForce GTX Processor: AMD Ryzen 9 3900X 12-Core @ 3.80GHz (12 Cores / 24 Threads), Motherboard: ASUS ROG STRIX X570-E GAMING (4021 BIOS), Chipset: AMD Starship/Matisse, Memory: 32GB, Disk: 500GB Western Digital WDS500G3X0C-00SJG0 + Samsung SSD 970 EVO Plus 500GB, Graphics: MSI NVIDIA GeForce GTX 1050 Ti 4GB, Audio: NVIDIA GP107GL HD Audio, Network: Realtek RTL8125 2.5GbE + Intel I211 + Intel Wi-Fi 6 AX200
OS: Ubuntu 20.04, Kernel: 5.4.0-132-generic (x86_64), Display Driver: NVIDIA, Vulkan: 1.1.182, Compiler: GCC 9.4.0, File-System: ext4, Screen Resolution: 1024x768
Kernel Notes: Transparent Huge Pages: madviseProcessor Notes: Scaling Governor: acpi-cpufreq ondemand (Boost: Enabled) - CPU Microcode: 0x8701021Python Notes: Python 3.8.10Security Notes: itlb_multihit: Not affected + l1tf: Not affected + mds: Not affected + meltdown: Not affected + mmio_stale_data: Not affected + retbleed: Vulnerable + spec_store_bypass: Mitigation of SSB disabled via prctl and seccomp + spectre_v1: Mitigation of usercopy/swapgs barriers and __user pointer sanitization + spectre_v2: Mitigation of Retpolines IBPB: conditional STIBP: conditional RSB filling PBRSB-eIBRS: Not affected + srbds: Not affected + tsx_async_abort: Not affected
Testing initiated at 30 November 2022 05:30 by user dechache.