deepsparse new AMD Ryzen 9 5900HX testing with a ASUS G513QY v1.0 (G513QY.318 BIOS) and ASUS AMD Cezanne 512MB on Ubuntu 22.10 via the Phoronix Test Suite.
Compare your own system(s) to this result file with the
Phoronix Test Suite by running the command:
phoronix-test-suite benchmark 2301230-PTS-DEEPSPAR46 a Kernel Notes: Transparent Huge Pages: madviseProcessor Notes: Scaling Governor: amd-pstate schedutil (Boost: Enabled) - Platform Profile: balanced - CPU Microcode: 0xa50000c - ACPI Profile: balancedPython Notes: Python 3.10.7Security Notes: itlb_multihit: Not affected + l1tf: Not affected + mds: Not affected + meltdown: Not affected + mmio_stale_data: Not affected + retbleed: Not affected + spec_store_bypass: Mitigation of SSB disabled via prctl + spectre_v1: Mitigation of usercopy/swapgs barriers and __user pointer sanitization + spectre_v2: Mitigation of Retpolines IBPB: conditional IBRS_FW STIBP: always-on RSB filling PBRSB-eIBRS: Not affected + srbds: Not affected + tsx_async_abort: Not affected
b c Processor: AMD Ryzen 9 5900HX @ 4.89GHz (8 Cores / 16 Threads), Motherboard: ASUS G513QY v1.0 (G513QY.318 BIOS), Chipset: AMD Renoir/Cezanne, Memory: 16GB, Disk: 512GB SAMSUNG MZVLQ512HBLU-00B00, Graphics: ASUS AMD Cezanne 512MB (2500/1000MHz), Audio: AMD Navi 21/23, Monitor: LQ156M1JW25, Network: Realtek RTL8111/8168/8411 + MEDIATEK MT7921 802.11ax PCI
OS: Ubuntu 22.10, Kernel: 5.19.0-28-generic (x86_64), Desktop: GNOME Shell 43.0, Display Server: X Server 1.21.1.4 + Wayland, OpenGL: 4.6 Mesa 22.2.1 (LLVM 15.0.2 DRM 3.47), Vulkan: 1.3.224, Compiler: GCC 12.2.0, File-System: ext4, Screen Resolution: 1920x1080
deepsparse new OpenBenchmarking.org Phoronix Test Suite AMD Ryzen 9 5900HX @ 4.89GHz (8 Cores / 16 Threads) ASUS G513QY v1.0 (G513QY.318 BIOS) AMD Renoir/Cezanne 16GB 512GB SAMSUNG MZVLQ512HBLU-00B00 ASUS AMD Cezanne 512MB (2500/1000MHz) AMD Navi 21/23 LQ156M1JW25 Realtek RTL8111/8168/8411 + MEDIATEK MT7921 802.11ax PCI Ubuntu 22.10 5.19.0-28-generic (x86_64) GNOME Shell 43.0 X Server 1.21.1.4 + Wayland 4.6 Mesa 22.2.1 (LLVM 15.0.2 DRM 3.47) 1.3.224 GCC 12.2.0 ext4 1920x1080 Processor Motherboard Chipset Memory Disk Graphics Audio Monitor Network OS Kernel Desktop Display Server OpenGL Vulkan Compiler File-System Screen Resolution Deepsparse New Benchmarks System Logs - Transparent Huge Pages: madvise - Scaling Governor: amd-pstate schedutil (Boost: Enabled) - Platform Profile: balanced - CPU Microcode: 0xa50000c - ACPI Profile: balanced - Python 3.10.7 - itlb_multihit: Not affected + l1tf: Not affected + mds: Not affected + meltdown: Not affected + mmio_stale_data: Not affected + retbleed: Not affected + spec_store_bypass: Mitigation of SSB disabled via prctl + spectre_v1: Mitigation of usercopy/swapgs barriers and __user pointer sanitization + spectre_v2: Mitigation of Retpolines IBPB: conditional IBRS_FW STIBP: always-on RSB filling PBRSB-eIBRS: Not affected + srbds: Not affected + tsx_async_abort: Not affected
a b c Result Overview Phoronix Test Suite 100% 102% 104% 106% 108% Neural Magic DeepSparse Neural Magic DeepSparse Neural Magic DeepSparse Neural Magic DeepSparse Neural Magic DeepSparse Neural Magic DeepSparse Neural Magic DeepSparse Neural Magic DeepSparse Neural Magic DeepSparse Neural Magic DeepSparse Neural Magic DeepSparse Neural Magic DeepSparse Neural Magic DeepSparse Neural Magic DeepSparse Neural Magic DeepSparse Neural Magic DeepSparse Neural Magic DeepSparse Neural Magic DeepSparse Neural Magic DeepSparse Neural Magic DeepSparse Neural Magic DeepSparse Neural Magic DeepSparse Neural Magic DeepSparse Neural Magic DeepSparse Neural Magic DeepSparse Neural Magic DeepSparse Neural Magic DeepSparse Neural Magic DeepSparse Neural Magic DeepSparse Neural Magic DeepSparse Neural Magic DeepSparse Neural Magic DeepSparse Neural Magic DeepSparse Neural Magic DeepSparse Neural Magic DeepSparse Neural Magic DeepSparse C.S.9.P.Y.P - A.M.S C.S.9.P.Y.P - A.M.S N.T.C.B.b.u.S - A.M.S N.T.C.B.b.u.S - A.M.S C.C.R.5.I - S.S.S C.C.R.5.I - S.S.S N.T.C.B.b.u.c - S.S.S N.T.C.B.b.u.c - S.S.S C.S.9.P.Y.P - S.S.S C.S.9.P.Y.P - S.S.S N.Q.A.B.b.u.S.1.P - A.M.S N.Q.A.B.b.u.S.1.P - A.M.S N.S.A.8.P.Q.B.B.U - S.S.S C.D.Y.C - A.M.S N.S.A.8.P.Q.B.B.U - S.S.S N.T.C.D.m - A.M.S C.D.Y.C - A.M.S N.T.C.D.m - A.M.S N.T.C.B.b.u.c - A.M.S N.T.C.B.b.u.c - A.M.S N.T.C.D.m - S.S.S N.T.C.D.m - S.S.S N.T.C.B.b.u.S - S.S.S N.T.C.B.b.u.S - S.S.S C.C.R.5.I - A.M.S C.C.R.5.I - A.M.S N.D.C.o.b.u.o.I - S.S.S N.D.C.o.b.u.o.I - S.S.S C.D.Y.C - S.S.S C.D.Y.C - S.S.S N.D.C.o.b.u.o.I - A.M.S N.D.C.o.b.u.o.I - A.M.S N.S.A.8.P.Q.B.B.U - A.M.S N.S.A.8.P.Q.B.B.U - A.M.S N.Q.A.B.b.u.S.1.P - S.S.S N.Q.A.B.b.u.S.1.P - S.S.S
deepsparse new deepsparse: NLP Document Classification, oBERT base uncased on IMDB - Asynchronous Multi-Stream deepsparse: NLP Document Classification, oBERT base uncased on IMDB - Synchronous Single-Stream deepsparse: NLP Sentiment Analysis, 80% Pruned Quantized BERT Base Uncased - Asynchronous Multi-Stream deepsparse: NLP Sentiment Analysis, 80% Pruned Quantized BERT Base Uncased - Synchronous Single-Stream deepsparse: NLP Question Answering, BERT base uncased SQuaD 12layer Pruned90 - Asynchronous Multi-Stream deepsparse: NLP Question Answering, BERT base uncased SQuaD 12layer Pruned90 - Synchronous Single-Stream deepsparse: CV Detection, YOLOv5s COCO - Asynchronous Multi-Stream deepsparse: CV Detection, YOLOv5s COCO - Synchronous Single-Stream deepsparse: CV Classification, ResNet-50 ImageNet - Asynchronous Multi-Stream deepsparse: CV Classification, ResNet-50 ImageNet - Synchronous Single-Stream deepsparse: NLP Text Classification, DistilBERT mnli - Asynchronous Multi-Stream deepsparse: NLP Text Classification, DistilBERT mnli - Synchronous Single-Stream deepsparse: CV Segmentation, 90% Pruned YOLACT Pruned - Asynchronous Multi-Stream deepsparse: CV Segmentation, 90% Pruned YOLACT Pruned - Synchronous Single-Stream deepsparse: NLP Text Classification, BERT base uncased SST2 - Asynchronous Multi-Stream deepsparse: NLP Text Classification, BERT base uncased SST2 - Synchronous Single-Stream deepsparse: NLP Token Classification, BERT base uncased conll2003 - Asynchronous Multi-Stream deepsparse: NLP Token Classification, BERT base uncased conll2003 - Synchronous Single-Stream deepsparse: NLP Document Classification, oBERT base uncased on IMDB - Asynchronous Multi-Stream deepsparse: NLP Document Classification, oBERT base uncased on IMDB - Synchronous Single-Stream deepsparse: NLP Sentiment Analysis, 80% Pruned Quantized BERT Base Uncased - Asynchronous Multi-Stream deepsparse: NLP Sentiment Analysis, 80% Pruned Quantized BERT Base Uncased - Synchronous Single-Stream deepsparse: NLP Question Answering, BERT base uncased SQuaD 12layer Pruned90 - Asynchronous Multi-Stream deepsparse: NLP Question Answering, BERT base uncased SQuaD 12layer Pruned90 - Synchronous Single-Stream deepsparse: CV Detection, YOLOv5s COCO - Asynchronous Multi-Stream deepsparse: CV Detection, YOLOv5s COCO - Synchronous Single-Stream deepsparse: CV Classification, ResNet-50 ImageNet - Asynchronous Multi-Stream deepsparse: CV Classification, ResNet-50 ImageNet - Synchronous Single-Stream deepsparse: NLP Text Classification, DistilBERT mnli - Asynchronous Multi-Stream deepsparse: NLP Text Classification, DistilBERT mnli - Synchronous Single-Stream deepsparse: CV Segmentation, 90% Pruned YOLACT Pruned - Asynchronous Multi-Stream deepsparse: CV Segmentation, 90% Pruned YOLACT Pruned - Synchronous Single-Stream deepsparse: NLP Text Classification, BERT base uncased SST2 - Asynchronous Multi-Stream deepsparse: NLP Text Classification, BERT base uncased SST2 - Synchronous Single-Stream deepsparse: NLP Token Classification, BERT base uncased conll2003 - Asynchronous Multi-Stream deepsparse: NLP Token Classification, BERT base uncased conll2003 - Synchronous Single-Stream a b c 6.1849 5.8779 71.1662 57.3632 19.2686 18.5832 31.8864 28.8554 76.0952 71.668 55.5122 47.0176 8.5515 8.1972 26.8409 23.916 6.1594 5.8462 646.7167 170.1219 56.1912 17.4245 207.5649 53.8018 125.384 34.6433 52.5513 13.9424 72.0407 21.2615 467.7261 121.9787 148.9317 41.8069 649.3885 171.042 6.1688 5.9081 71.0519 57.659 19.5795 18.5719 31.5504 28.9365 76.0289 69.4477 54.9277 47.4231 7.9379 8.3643 28.2256 24.0462 6.142 5.975 648.4005 169.251 56.2815 17.3351 204.2325 53.8347 126.7324 34.5469 52.5794 14.389 72.8102 21.0804 502.3104 119.5417 141.6979 41.579 651.2184 167.3562 6.1581 5.8876 71.3201 57.9803 19.3064 18.5157 31.6469 29.0018 75.5245 70.6570 55.0102 47.3174 8.4650 8.2039 28.1413 23.8628 6.1952 5.8801 649.4201 169.9081 56.0700 17.2396 207.0196 53.9980 126.3038 34.4683 52.9213 14.1479 72.6940 21.1276 472.5227 121.8985 142.0899 41.8992 645.6068 170.1116 OpenBenchmarking.org
OpenBenchmarking.org items/sec, More Is Better Neural Magic DeepSparse 1.3.2 Model: NLP Document Classification, oBERT base uncased on IMDB - Scenario: Synchronous Single-Stream c b a 1.3293 2.6586 3.9879 5.3172 6.6465 SE +/- 0.0840, N = 3 5.8876 5.9081 5.8779
a Kernel Notes: Transparent Huge Pages: madviseProcessor Notes: Scaling Governor: amd-pstate schedutil (Boost: Enabled) - Platform Profile: balanced - CPU Microcode: 0xa50000c - ACPI Profile: balancedPython Notes: Python 3.10.7Security Notes: itlb_multihit: Not affected + l1tf: Not affected + mds: Not affected + meltdown: Not affected + mmio_stale_data: Not affected + retbleed: Not affected + spec_store_bypass: Mitigation of SSB disabled via prctl + spectre_v1: Mitigation of usercopy/swapgs barriers and __user pointer sanitization + spectre_v2: Mitigation of Retpolines IBPB: conditional IBRS_FW STIBP: always-on RSB filling PBRSB-eIBRS: Not affected + srbds: Not affected + tsx_async_abort: Not affected
Testing initiated at 22 January 2023 19:43 by user pts.
b Kernel Notes: Transparent Huge Pages: madviseProcessor Notes: Scaling Governor: amd-pstate schedutil (Boost: Enabled) - Platform Profile: balanced - CPU Microcode: 0xa50000c - ACPI Profile: balancedPython Notes: Python 3.10.7Security Notes: itlb_multihit: Not affected + l1tf: Not affected + mds: Not affected + meltdown: Not affected + mmio_stale_data: Not affected + retbleed: Not affected + spec_store_bypass: Mitigation of SSB disabled via prctl + spectre_v1: Mitigation of usercopy/swapgs barriers and __user pointer sanitization + spectre_v2: Mitigation of Retpolines IBPB: conditional IBRS_FW STIBP: always-on RSB filling PBRSB-eIBRS: Not affected + srbds: Not affected + tsx_async_abort: Not affected
Testing initiated at 22 January 2023 20:01 by user pts.
c Processor: AMD Ryzen 9 5900HX @ 4.89GHz (8 Cores / 16 Threads), Motherboard: ASUS G513QY v1.0 (G513QY.318 BIOS), Chipset: AMD Renoir/Cezanne, Memory: 16GB, Disk: 512GB SAMSUNG MZVLQ512HBLU-00B00, Graphics: ASUS AMD Cezanne 512MB (2500/1000MHz), Audio: AMD Navi 21/23, Monitor: LQ156M1JW25, Network: Realtek RTL8111/8168/8411 + MEDIATEK MT7921 802.11ax PCI
OS: Ubuntu 22.10, Kernel: 5.19.0-28-generic (x86_64), Desktop: GNOME Shell 43.0, Display Server: X Server 1.21.1.4 + Wayland, OpenGL: 4.6 Mesa 22.2.1 (LLVM 15.0.2 DRM 3.47), Vulkan: 1.3.224, Compiler: GCC 12.2.0, File-System: ext4, Screen Resolution: 1920x1080
Kernel Notes: Transparent Huge Pages: madviseProcessor Notes: Scaling Governor: amd-pstate schedutil (Boost: Enabled) - Platform Profile: balanced - CPU Microcode: 0xa50000c - ACPI Profile: balancedPython Notes: Python 3.10.7Security Notes: itlb_multihit: Not affected + l1tf: Not affected + mds: Not affected + meltdown: Not affected + mmio_stale_data: Not affected + retbleed: Not affected + spec_store_bypass: Mitigation of SSB disabled via prctl + spectre_v1: Mitigation of usercopy/swapgs barriers and __user pointer sanitization + spectre_v2: Mitigation of Retpolines IBPB: conditional IBRS_FW STIBP: always-on RSB filling PBRSB-eIBRS: Not affected + srbds: Not affected + tsx_async_abort: Not affected
Testing initiated at 22 January 2023 20:27 by user pts.