ds lkey AMD Ryzen 7 PRO 6850U testing with a LENOVO 21CM0001US (R22ET51W 1.21 BIOS) and AMD Radeon 680M 1GB on Ubuntu 22.10 via the Phoronix Test Suite. a: Processor: AMD Ryzen 7 PRO 6850U @ 4.77GHz (8 Cores / 16 Threads), Motherboard: LENOVO 21CM0001US (R22ET51W 1.21 BIOS), Chipset: AMD Device 14b5, Memory: 16GB, Disk: 512GB Micron MTFDKBA512TFK, Graphics: AMD Radeon 680M 1GB (2200/400MHz), Audio: AMD Rembrandt Radeon HD Audio, Network: Qualcomm QCNFA765 OS: Ubuntu 22.10, Kernel: 6.1.0-060100rc2daily20221028-generic (x86_64), Desktop: GNOME Shell 43.0, Display Server: X Server + Wayland, OpenGL: 4.6 Mesa 22.2.1 (LLVM 15.0.2 DRM 3.49), Vulkan: 1.3.224, Compiler: GCC 12.2.0, File-System: ext4, Screen Resolution: 1920x1200 b: Processor: AMD Ryzen 7 PRO 6850U @ 4.77GHz (8 Cores / 16 Threads), Motherboard: LENOVO 21CM0001US (R22ET51W 1.21 BIOS), Chipset: AMD Device 14b5, Memory: 16GB, Disk: 512GB Micron MTFDKBA512TFK, Graphics: AMD Radeon 680M 1GB (2200/400MHz), Audio: AMD Rembrandt Radeon HD Audio, Network: Qualcomm QCNFA765 OS: Ubuntu 22.10, Kernel: 6.1.0-060100rc2daily20221028-generic (x86_64), Desktop: GNOME Shell 43.0, Display Server: X Server + Wayland, OpenGL: 4.6 Mesa 22.2.1 (LLVM 15.0.2 DRM 3.49), Vulkan: 1.3.224, Compiler: GCC 12.2.0, File-System: ext4, Screen Resolution: 1920x1200 c: Processor: AMD Ryzen 7 PRO 6850U @ 4.77GHz (8 Cores / 16 Threads), Motherboard: LENOVO 21CM0001US (R22ET51W 1.21 BIOS), Chipset: AMD Device 14b5, Memory: 16GB, Disk: 512GB Micron MTFDKBA512TFK, Graphics: AMD Radeon 680M 1GB (2200/400MHz), Audio: AMD Rembrandt Radeon HD Audio, Network: Qualcomm QCNFA765 OS: Ubuntu 22.10, Kernel: 6.1.0-060100rc2daily20221028-generic (x86_64), Desktop: GNOME Shell 43.0, Display Server: X Server + Wayland, OpenGL: 4.6 Mesa 22.2.1 (LLVM 15.0.2 DRM 3.49), Vulkan: 1.3.224, Compiler: GCC 12.2.0, File-System: ext4, Screen Resolution: 1920x1200 Neural Magic DeepSparse 1.3.2 Model: NLP Document Classification, oBERT base uncased on IMDB - Scenario: Asynchronous Multi-Stream items/sec > Higher Is Better a . 5.1659 |=================================================================== b . 5.1708 |=================================================================== c . 5.1593 |=================================================================== Neural Magic DeepSparse 1.3.2 Model: NLP Document Classification, oBERT base uncased on IMDB - Scenario: Synchronous Single-Stream items/sec > Higher Is Better a . 5.0356 |=================================================================== b . 5.0409 |=================================================================== c . 5.0291 |=================================================================== Neural Magic DeepSparse 1.3.2 Model: NLP Sentiment Analysis, 80% Pruned Quantized BERT Base Uncased - Scenario: Asynchronous Multi-Stream items/sec > Higher Is Better a . 64.94 |==================================================================== b . 64.89 |==================================================================== c . 64.57 |==================================================================== Neural Magic DeepSparse 1.3.2 Model: NLP Sentiment Analysis, 80% Pruned Quantized BERT Base Uncased - Scenario: Synchronous Single-Stream items/sec > Higher Is Better a . 51.44 |=================================================================== b . 51.88 |==================================================================== c . 51.85 |==================================================================== Neural Magic DeepSparse 1.3.2 Model: NLP Question Answering, BERT base uncased SQuaD 12layer Pruned90 - Scenario: Asynchronous Multi-Stream items/sec > Higher Is Better a . 20.04 |=================================================================== b . 20.20 |==================================================================== c . 20.22 |==================================================================== Neural Magic DeepSparse 1.3.2 Model: NLP Question Answering, BERT base uncased SQuaD 12layer Pruned90 - Scenario: Synchronous Single-Stream items/sec > Higher Is Better a . 18.10 |==================================================================== b . 18.13 |==================================================================== c . 18.19 |==================================================================== Neural Magic DeepSparse 1.3.2 Model: CV Detection, YOLOv5s COCO - Scenario: Asynchronous Multi-Stream items/sec > Higher Is Better a . 28.50 |==================================================================== b . 28.48 |==================================================================== c . 28.41 |==================================================================== Neural Magic DeepSparse 1.3.2 Model: CV Detection, YOLOv5s COCO - Scenario: Synchronous Single-Stream items/sec > Higher Is Better a . 26.14 |==================================================================== b . 26.16 |==================================================================== c . 26.14 |==================================================================== Neural Magic DeepSparse 1.3.2 Model: CV Classification, ResNet-50 ImageNet - Scenario: Asynchronous Multi-Stream items/sec > Higher Is Better a . 62.17 |==================================================================== b . 62.03 |==================================================================== c . 62.33 |==================================================================== Neural Magic DeepSparse 1.3.2 Model: CV Classification, ResNet-50 ImageNet - Scenario: Synchronous Single-Stream items/sec > Higher Is Better a . 56.32 |==================================================================== b . 56.35 |==================================================================== c . 56.29 |==================================================================== Neural Magic DeepSparse 1.3.2 Model: NLP Text Classification, DistilBERT mnli - Scenario: Asynchronous Multi-Stream items/sec > Higher Is Better a . 45.60 |==================================================================== b . 45.66 |==================================================================== c . 45.73 |==================================================================== Neural Magic DeepSparse 1.3.2 Model: NLP Text Classification, DistilBERT mnli - Scenario: Synchronous Single-Stream items/sec > Higher Is Better a . 41.25 |==================================================================== b . 41.28 |==================================================================== c . 41.12 |==================================================================== Neural Magic DeepSparse 1.3.2 Model: CV Segmentation, 90% Pruned YOLACT Pruned - Scenario: Asynchronous Multi-Stream items/sec > Higher Is Better a . 7.0942 |=================================================================== b . 7.1392 |=================================================================== c . 7.1222 |=================================================================== Neural Magic DeepSparse 1.3.2 Model: CV Segmentation, 90% Pruned YOLACT Pruned - Scenario: Synchronous Single-Stream items/sec > Higher Is Better a . 7.0091 |=================================================================== b . 7.0162 |=================================================================== c . 7.0170 |=================================================================== Neural Magic DeepSparse 1.3.2 Model: NLP Text Classification, BERT base uncased SST2 - Scenario: Asynchronous Multi-Stream items/sec > Higher Is Better a . 22.19 |==================================================================== b . 22.27 |==================================================================== c . 22.19 |==================================================================== Neural Magic DeepSparse 1.3.2 Model: NLP Text Classification, BERT base uncased SST2 - Scenario: Synchronous Single-Stream items/sec > Higher Is Better a . 20.28 |==================================================================== b . 20.31 |==================================================================== c . 20.21 |==================================================================== Neural Magic DeepSparse 1.3.2 Model: NLP Token Classification, BERT base uncased conll2003 - Scenario: Asynchronous Multi-Stream items/sec > Higher Is Better a . 5.1498 |=================================================================== b . 5.1574 |=================================================================== c . 5.1690 |=================================================================== Neural Magic DeepSparse 1.3.2 Model: NLP Token Classification, BERT base uncased conll2003 - Scenario: Synchronous Single-Stream items/sec > Higher Is Better a . 5.0161 |=================================================================== b . 5.0184 |=================================================================== c . 5.0327 |=================================================================== KeyDB 6.3.2 Test: GET - Parallel Connections: 50 Requests Per Second > Higher Is Better a . 400374.75 |================================================================ b . 397769.31 |================================================================ c . 396253.03 |=============================================================== KeyDB 6.3.2 Test: SET - Parallel Connections: 50 Requests Per Second > Higher Is Better a . 391239.34 |================================================================ b . 389462.69 |================================================================ c . 389289.88 |================================================================ KeyDB 6.3.2 Test: GET - Parallel Connections: 100 Requests Per Second > Higher Is Better a . 317397.84 |================================================================ b . 318068.16 |================================================================ c . 315931.81 |================================================================ KeyDB 6.3.2 Test: GET - Parallel Connections: 500 Requests Per Second > Higher Is Better a . 312160.53 |================================================================ b . 310491.50 |================================================================ c . 312734.53 |================================================================ KeyDB 6.3.2 Test: GET - Parallel Connections: 900 Requests Per Second > Higher Is Better a . 308276.59 |================================================================ b . 305945.12 |================================================================ c . 306020.03 |================================================================ KeyDB 6.3.2 Test: LPOP - Parallel Connections: 50 Requests Per Second > Higher Is Better a . 399629.16 |================================================================ b . 370255.16 |=========================================================== c . 368172.25 |=========================================================== KeyDB 6.3.2 Test: SADD - Parallel Connections: 50 Requests Per Second > Higher Is Better a . 395707.34 |================================================================ b . 395848.34 |================================================================ c . 396227.91 |================================================================ KeyDB 6.3.2 Test: SET - Parallel Connections: 100 Requests Per Second > Higher Is Better a . 316035.62 |=============================================================== b . 323218.75 |================================================================ c . 306421.38 |============================================================= KeyDB 6.3.2 Test: SET - Parallel Connections: 500 Requests Per Second > Higher Is Better a . 302518.75 |================================================================ b . 297642.09 |=============================================================== c . 300516.88 |================================================================ KeyDB 6.3.2 Test: SET - Parallel Connections: 900 Requests Per Second > Higher Is Better a . 297450.84 |================================================================ b . 294012.12 |=============================================================== c . 294821.75 |=============================================================== KeyDB 6.3.2 Test: HMSET - Parallel Connections: 50 Requests Per Second > Higher Is Better a . 301397.28 |================================================================ b . 301688.25 |================================================================ c . 299625.47 |================================================================ KeyDB 6.3.2 Test: LPOP - Parallel Connections: 100 Requests Per Second > Higher Is Better a . 326756.81 |================================================================ b . 288963.91 |========================================================= c . 307917.81 |============================================================ KeyDB 6.3.2 Test: LPOP - Parallel Connections: 500 Requests Per Second > Higher Is Better a . 319443.91 |================================================================ b . 279348.31 |======================================================== c . 281265.47 |======================================================== KeyDB 6.3.2 Test: LPOP - Parallel Connections: 900 Requests Per Second > Higher Is Better a . 313709.75 |================================================================ b . 279925.44 |========================================================= c . 289251.41 |=========================================================== KeyDB 6.3.2 Test: LPUSH - Parallel Connections: 50 Requests Per Second > Higher Is Better a . 377803.31 |================================================================ b . 378526.91 |================================================================ c . 378091.84 |================================================================ KeyDB 6.3.2 Test: SADD - Parallel Connections: 100 Requests Per Second > Higher Is Better a . 314192.72 |=============================================================== b . 319358.22 |================================================================ c . 314827.75 |=============================================================== KeyDB 6.3.2 Test: SADD - Parallel Connections: 500 Requests Per Second > Higher Is Better a . 309906.47 |================================================================ b . 311701.25 |================================================================ c . 310927.88 |================================================================ KeyDB 6.3.2 Test: SADD - Parallel Connections: 900 Requests Per Second > Higher Is Better a . 307442.56 |================================================================ b . 304371.41 |=============================================================== c . 305295.03 |================================================================ KeyDB 6.3.2 Test: HMSET - Parallel Connections: 100 Requests Per Second > Higher Is Better a . 220859.77 |================================================================ b . 214634.66 |============================================================== c . 219591.98 |================================================================ KeyDB 6.3.2 Test: HMSET - Parallel Connections: 500 Requests Per Second > Higher Is Better a . 205129.89 |================================================================ b . 204631.22 |================================================================ c . 203269.38 |=============================================================== KeyDB 6.3.2 Test: HMSET - Parallel Connections: 900 Requests Per Second > Higher Is Better a . 201398.50 |=============================================================== b . 203707.48 |================================================================ c . 202206.47 |================================================================ KeyDB 6.3.2 Test: LPUSH - Parallel Connections: 100 Requests Per Second > Higher Is Better a . 318623.53 |================================================================ b . 290675.72 |========================================================== c . 299638.03 |============================================================ KeyDB 6.3.2 Test: LPUSH - Parallel Connections: 500 Requests Per Second > Higher Is Better a . 277528.00 |============================================================== b . 285523.38 |================================================================ c . 270480.81 |============================================================= KeyDB 6.3.2 Test: LPUSH - Parallel Connections: 900 Requests Per Second > Higher Is Better a . 270349.19 |================================================================ b . 269025.47 |================================================================ c . 267054.06 |=============================================================== Neural Magic DeepSparse 1.3.2 Model: NLP Document Classification, oBERT base uncased on IMDB - Scenario: Asynchronous Multi-Stream ms/batch < Lower Is Better a . 774.29 |=================================================================== b . 773.55 |=================================================================== c . 775.28 |=================================================================== Neural Magic DeepSparse 1.3.2 Model: NLP Document Classification, oBERT base uncased on IMDB - Scenario: Synchronous Single-Stream ms/batch < Lower Is Better a . 198.57 |=================================================================== b . 198.36 |=================================================================== c . 198.82 |=================================================================== Neural Magic DeepSparse 1.3.2 Model: NLP Sentiment Analysis, 80% Pruned Quantized BERT Base Uncased - Scenario: Asynchronous Multi-Stream ms/batch < Lower Is Better a . 61.58 |==================================================================== b . 61.62 |==================================================================== c . 61.92 |==================================================================== Neural Magic DeepSparse 1.3.2 Model: NLP Sentiment Analysis, 80% Pruned Quantized BERT Base Uncased - Scenario: Synchronous Single-Stream ms/batch < Lower Is Better a . 19.42 |==================================================================== b . 19.26 |=================================================================== c . 19.27 |=================================================================== Neural Magic DeepSparse 1.3.2 Model: NLP Question Answering, BERT base uncased SQuaD 12layer Pruned90 - Scenario: Asynchronous Multi-Stream ms/batch < Lower Is Better a . 199.16 |=================================================================== b . 197.82 |=================================================================== c . 197.79 |=================================================================== Neural Magic DeepSparse 1.3.2 Model: NLP Question Answering, BERT base uncased SQuaD 12layer Pruned90 - Scenario: Synchronous Single-Stream ms/batch < Lower Is Better a . 55.22 |==================================================================== b . 55.12 |==================================================================== c . 54.95 |==================================================================== Neural Magic DeepSparse 1.3.2 Model: CV Detection, YOLOv5s COCO - Scenario: Asynchronous Multi-Stream ms/batch < Lower Is Better a . 140.27 |=================================================================== b . 140.36 |=================================================================== c . 140.73 |=================================================================== Neural Magic DeepSparse 1.3.2 Model: CV Detection, YOLOv5s COCO - Scenario: Synchronous Single-Stream ms/batch < Lower Is Better a . 38.23 |==================================================================== b . 38.20 |==================================================================== c . 38.23 |==================================================================== Neural Magic DeepSparse 1.3.2 Model: CV Classification, ResNet-50 ImageNet - Scenario: Asynchronous Multi-Stream ms/batch < Lower Is Better a . 64.32 |==================================================================== b . 64.46 |==================================================================== c . 64.16 |==================================================================== Neural Magic DeepSparse 1.3.2 Model: CV Classification, ResNet-50 ImageNet - Scenario: Synchronous Single-Stream ms/batch < Lower Is Better a . 17.74 |==================================================================== b . 17.73 |==================================================================== c . 17.75 |==================================================================== Neural Magic DeepSparse 1.3.2 Model: NLP Text Classification, DistilBERT mnli - Scenario: Asynchronous Multi-Stream ms/batch < Lower Is Better a . 87.70 |==================================================================== b . 87.58 |==================================================================== c . 87.45 |==================================================================== Neural Magic DeepSparse 1.3.2 Model: NLP Text Classification, DistilBERT mnli - Scenario: Synchronous Single-Stream ms/batch < Lower Is Better a . 24.23 |==================================================================== b . 24.21 |==================================================================== c . 24.31 |==================================================================== Neural Magic DeepSparse 1.3.2 Model: CV Segmentation, 90% Pruned YOLACT Pruned - Scenario: Asynchronous Multi-Stream ms/batch < Lower Is Better a . 563.81 |=================================================================== b . 560.26 |=================================================================== c . 561.59 |=================================================================== Neural Magic DeepSparse 1.3.2 Model: CV Segmentation, 90% Pruned YOLACT Pruned - Scenario: Synchronous Single-Stream ms/batch < Lower Is Better a . 142.65 |=================================================================== b . 142.50 |=================================================================== c . 142.48 |=================================================================== Neural Magic DeepSparse 1.3.2 Model: NLP Text Classification, BERT base uncased SST2 - Scenario: Asynchronous Multi-Stream ms/batch < Lower Is Better a . 180.24 |=================================================================== b . 179.60 |=================================================================== c . 180.23 |=================================================================== Neural Magic DeepSparse 1.3.2 Model: NLP Text Classification, BERT base uncased SST2 - Scenario: Synchronous Single-Stream ms/batch < Lower Is Better a . 49.28 |==================================================================== b . 49.22 |==================================================================== c . 49.46 |==================================================================== Neural Magic DeepSparse 1.3.2 Model: NLP Token Classification, BERT base uncased conll2003 - Scenario: Asynchronous Multi-Stream ms/batch < Lower Is Better a . 776.67 |=================================================================== b . 773.86 |=================================================================== c . 773.81 |=================================================================== Neural Magic DeepSparse 1.3.2 Model: NLP Token Classification, BERT base uncased conll2003 - Scenario: Synchronous Single-Stream ms/batch < Lower Is Better a . 199.34 |=================================================================== b . 199.25 |=================================================================== c . 198.68 |===================================================================