new tests Intel Core i9-13900K testing with a ASUS PRIME Z790-P WIFI (0602 BIOS) and AMD Radeon RX 6700 XT 12GB on Ubuntu 22.04 via the Phoronix Test Suite. a: Processor: Intel Core i9-13900K @ 4.00GHz (24 Cores / 32 Threads), Motherboard: ASUS PRIME Z790-P WIFI (0602 BIOS), Chipset: Intel Device 7a27, Memory: 32GB, Disk: 1000GB Western Digital WDS100T1X0E-00AFY0, Graphics: AMD Radeon RX 6700 XT 12GB (2855/1000MHz), Audio: Realtek ALC897, Monitor: ASUS VP28U, Network: Intel Device 7a70 OS: Ubuntu 22.04, Kernel: 6.0.0-060000-generic (x86_64), Desktop: GNOME Shell 42.4, Display Server: X Server 1.21.1.3 + Wayland, OpenGL: 4.6 Mesa 22.3.0-devel (git-8945375 2022-10-04 jammy-oibaf-ppa) (LLVM 14.0.6 DRM 3.48), Vulkan: 1.3.230, Compiler: GCC 11.2.0, File-System: ext4, Screen Resolution: 3840x2160 b: Processor: Intel Core i9-13900K @ 4.00GHz (24 Cores / 32 Threads), Motherboard: ASUS PRIME Z790-P WIFI (0602 BIOS), Chipset: Intel Device 7a27, Memory: 32GB, Disk: 1000GB Western Digital WDS100T1X0E-00AFY0, Graphics: AMD Radeon RX 6700 XT 12GB (2855/1000MHz), Audio: Realtek ALC897, Monitor: ASUS VP28U, Network: Intel Device 7a70 OS: Ubuntu 22.04, Kernel: 6.0.0-060000-generic (x86_64), Desktop: GNOME Shell 42.4, Display Server: X Server 1.21.1.3 + Wayland, OpenGL: 4.6 Mesa 22.3.0-devel (git-8945375 2022-10-04 jammy-oibaf-ppa) (LLVM 14.0.6 DRM 3.48), Vulkan: 1.3.230, Compiler: GCC 11.2.0, File-System: ext4, Screen Resolution: 3840x2160 c: Processor: Intel Core i9-13900K @ 4.00GHz (24 Cores / 32 Threads), Motherboard: ASUS PRIME Z790-P WIFI (0602 BIOS), Chipset: Intel Device 7a27, Memory: 32GB, Disk: 1000GB Western Digital WDS100T1X0E-00AFY0, Graphics: AMD Radeon RX 6700 XT 12GB (2855/1000MHz), Audio: Realtek ALC897, Monitor: ASUS VP28U, Network: Intel Device 7a70 OS: Ubuntu 22.04, Kernel: 6.0.0-060000-generic (x86_64), Desktop: GNOME Shell 42.4, Display Server: X Server 1.21.1.3 + Wayland, OpenGL: 4.6 Mesa 22.3.0-devel (git-8945375 2022-10-04 jammy-oibaf-ppa) (LLVM 14.0.6 DRM 3.48), Vulkan: 1.3.230, Compiler: GCC 11.2.0, File-System: ext4, Screen Resolution: 3840x2160 d: Processor: Intel Core i9-13900K @ 4.00GHz (24 Cores / 32 Threads), Motherboard: ASUS PRIME Z790-P WIFI (0602 BIOS), Chipset: Intel Device 7a27, Memory: 32GB, Disk: 1000GB Western Digital WDS100T1X0E-00AFY0, Graphics: AMD Radeon RX 6700 XT 12GB (2855/1000MHz), Audio: Realtek ALC897, Monitor: ASUS VP28U, Network: Intel Device 7a70 OS: Ubuntu 22.04, Kernel: 6.0.0-060000-generic (x86_64), Desktop: GNOME Shell 42.4, Display Server: X Server 1.21.1.3 + Wayland, OpenGL: 4.6 Mesa 22.3.0-devel (git-8945375 2022-10-04 jammy-oibaf-ppa) (LLVM 14.0.6 DRM 3.48), Vulkan: 1.3.230, Compiler: GCC 11.2.0, File-System: ext4, Screen Resolution: 3840x2160 Neural Magic DeepSparse 1.3.2 Model: NLP Document Classification, oBERT base uncased on IMDB - Scenario: Asynchronous Multi-Stream items/sec > Higher Is Better a . 17.10 |=================================================================== b . 17.03 |=================================================================== c . 17.13 |=================================================================== d . 17.26 |==================================================================== Neural Magic DeepSparse 1.3.2 Model: NLP Document Classification, oBERT base uncased on IMDB - Scenario: Synchronous Single-Stream items/sec > Higher Is Better a . 15.15 |==================================================================== b . 15.15 |==================================================================== c . 15.14 |==================================================================== d . 15.16 |==================================================================== Neural Magic DeepSparse 1.3.2 Model: NLP Sentiment Analysis, 80% Pruned Quantized BERT Base Uncased - Scenario: Asynchronous Multi-Stream items/sec > Higher Is Better a . 201.89 |================================================================== b . 202.87 |=================================================================== c . 202.74 |=================================================================== d . 203.72 |=================================================================== Neural Magic DeepSparse 1.3.2 Model: NLP Sentiment Analysis, 80% Pruned Quantized BERT Base Uncased - Scenario: Synchronous Single-Stream items/sec > Higher Is Better a . 97.03 |==================================================================== b . 96.60 |=================================================================== c . 97.70 |==================================================================== d . 97.05 |==================================================================== Neural Magic DeepSparse 1.3.2 Model: NLP Question Answering, BERT base uncased SQuaD 12layer Pruned90 - Scenario: Asynchronous Multi-Stream items/sec > Higher Is Better a . 62.47 |=================================================================== b . 62.03 |=================================================================== c . 62.43 |=================================================================== d . 62.94 |==================================================================== Neural Magic DeepSparse 1.3.2 Model: NLP Question Answering, BERT base uncased SQuaD 12layer Pruned90 - Scenario: Synchronous Single-Stream items/sec > Higher Is Better a . 44.41 |==================================================================== b . 44.35 |==================================================================== c . 44.20 |==================================================================== d . 44.38 |==================================================================== Neural Magic DeepSparse 1.3.2 Model: CV Detection, YOLOv5s COCO - Scenario: Asynchronous Multi-Stream items/sec > Higher Is Better a . 100.60 |================================================================== b . 102.19 |=================================================================== c . 100.92 |================================================================== d . 101.58 |=================================================================== Neural Magic DeepSparse 1.3.2 Model: CV Detection, YOLOv5s COCO - Scenario: Synchronous Single-Stream items/sec > Higher Is Better a . 67.85 |=================================================================== b . 68.41 |=================================================================== c . 68.23 |=================================================================== d . 69.23 |==================================================================== Neural Magic DeepSparse 1.3.2 Model: CV Classification, ResNet-50 ImageNet - Scenario: Asynchronous Multi-Stream items/sec > Higher Is Better a . 220.22 |================================================================== b . 223.11 |=================================================================== c . 222.58 |=================================================================== d . 220.63 |================================================================== Neural Magic DeepSparse 1.3.2 Model: CV Classification, ResNet-50 ImageNet - Scenario: Synchronous Single-Stream items/sec > Higher Is Better a . 125.70 |================================================================== b . 126.99 |=================================================================== c . 126.71 |=================================================================== d . 126.85 |=================================================================== Neural Magic DeepSparse 1.3.2 Model: NLP Text Classification, DistilBERT mnli - Scenario: Asynchronous Multi-Stream items/sec > Higher Is Better a . 146.84 |================================================================== b . 146.76 |================================================================== c . 148.98 |=================================================================== d . 146.71 |================================================================== Neural Magic DeepSparse 1.3.2 Model: NLP Text Classification, DistilBERT mnli - Scenario: Synchronous Single-Stream items/sec > Higher Is Better a . 84.21 |==================================================================== b . 84.34 |==================================================================== c . 84.62 |==================================================================== d . 84.70 |==================================================================== Neural Magic DeepSparse 1.3.2 Model: CV Segmentation, 90% Pruned YOLACT Pruned - Scenario: Asynchronous Multi-Stream items/sec > Higher Is Better a . 22.46 |=================================================================== b . 22.57 |=================================================================== c . 22.48 |=================================================================== d . 22.88 |==================================================================== Neural Magic DeepSparse 1.3.2 Model: CV Segmentation, 90% Pruned YOLACT Pruned - Scenario: Synchronous Single-Stream items/sec > Higher Is Better a . 17.92 |==================================================================== b . 17.97 |==================================================================== c . 17.83 |=================================================================== d . 17.90 |==================================================================== Neural Magic DeepSparse 1.3.2 Model: NLP Text Classification, BERT base uncased SST2 - Scenario: Asynchronous Multi-Stream items/sec > Higher Is Better a . 74.68 |==================================================================== b . 73.96 |=================================================================== c . 75.21 |==================================================================== d . 74.44 |=================================================================== Neural Magic DeepSparse 1.3.2 Model: NLP Text Classification, BERT base uncased SST2 - Scenario: Synchronous Single-Stream items/sec > Higher Is Better a . 42.41 |==================================================================== b . 42.41 |==================================================================== c . 42.59 |==================================================================== d . 42.42 |==================================================================== Neural Magic DeepSparse 1.3.2 Model: NLP Token Classification, BERT base uncased conll2003 - Scenario: Asynchronous Multi-Stream items/sec > Higher Is Better a . 17.06 |=================================================================== b . 17.26 |==================================================================== c . 17.00 |=================================================================== d . 17.24 |==================================================================== Neural Magic DeepSparse 1.3.2 Model: NLP Token Classification, BERT base uncased conll2003 - Scenario: Synchronous Single-Stream items/sec > Higher Is Better a . 15.19 |==================================================================== b . 15.20 |==================================================================== c . 15.12 |==================================================================== d . 15.18 |==================================================================== KeyDB 6.3.2 Test: GET - Parallel Connections: 50 Requests Per Second > Higher Is Better a . 2301457.08 |=============================================================== b . 2203905.25 |============================================================ c . 2258764.00 |============================================================== d . 2273450.75 |============================================================== KeyDB 6.3.2 Test: SET - Parallel Connections: 50 Requests Per Second > Higher Is Better a . 2051262.03 |============================================================== b . 2074172.38 |=============================================================== c . 2007790.25 |============================================================= d . 2013693.12 |============================================================= KeyDB 6.3.2 Test: GET - Parallel Connections: 100 Requests Per Second > Higher Is Better a . 1526251.21 |=============================================================== b . 1482228.00 |============================================================= c . 1488316.62 |============================================================= d . 1484384.38 |============================================================= KeyDB 6.3.2 Test: GET - Parallel Connections: 500 Requests Per Second > Higher Is Better a . 1493932.89 |============================================================ b . 1558846.38 |=============================================================== c . 1476886.75 |============================================================ d . 1553518.62 |=============================================================== KeyDB 6.3.2 Test: GET - Parallel Connections: 900 Requests Per Second > Higher Is Better a . 1563299.29 |============================================================== b . 1582378.62 |=============================================================== c . 1527463.75 |============================================================= d . 1510802.12 |============================================================ KeyDB 6.3.2 Test: LPOP - Parallel Connections: 50 Requests Per Second > Higher Is Better a . 2355768.58 |=============================================================== b . 2032602.88 |====================================================== c . 2014748.00 |====================================================== d . 2016942.25 |====================================================== KeyDB 6.3.2 Test: SADD - Parallel Connections: 50 Requests Per Second > Higher Is Better a . 2240005.25 |=============================================================== b . 2210042.50 |============================================================== c . 2215133.75 |============================================================== d . 2186557.00 |============================================================= KeyDB 6.3.2 Test: SET - Parallel Connections: 100 Requests Per Second > Higher Is Better a . 1277181.12 |============================================================== b . 1180776.88 |========================================================= c . 1105998.88 |====================================================== d . 1296008.25 |=============================================================== KeyDB 6.3.2 Test: SET - Parallel Connections: 500 Requests Per Second > Higher Is Better a . 1242079.00 |============================================================ b . 1314786.12 |=============================================================== c . 1188354.12 |========================================================= d . 1307531.38 |=============================================================== KeyDB 6.3.2 Test: SET - Parallel Connections: 900 Requests Per Second > Higher Is Better a . 1227964.69 |=========================================================== b . 1200999.25 |========================================================== c . 1209599.38 |========================================================== d . 1305926.38 |=============================================================== KeyDB 6.3.2 Test: HMSET - Parallel Connections: 50 Requests Per Second > Higher Is Better a . 770419.02 |============================================================ b . 827088.81 |================================================================ c . 822828.56 |================================================================ d . 781066.94 |============================================================ KeyDB 6.3.2 Test: LPOP - Parallel Connections: 100 Requests Per Second > Higher Is Better a . 1548101.63 |=============================================================== b . 1082813.62 |============================================ c . 1133555.50 |============================================== d . 1110666.75 |============================================= KeyDB 6.3.2 Test: LPOP - Parallel Connections: 500 Requests Per Second > Higher Is Better a . 1541362.82 |=============================================================== b . 1142413.25 |=============================================== c . 1166752.25 |================================================ d . 1101928.38 |============================================= KeyDB 6.3.2 Test: LPOP - Parallel Connections: 900 Requests Per Second > Higher Is Better a . 1572282.98 |=============================================================== b . 1132092.50 |============================================= c . 1171454.00 |=============================================== d . 1168661.12 |=============================================== KeyDB 6.3.2 Test: LPUSH - Parallel Connections: 50 Requests Per Second > Higher Is Better a . 1885806.56 |============================================================ b . 1849522.75 |=========================================================== c . 1964096.25 |=============================================================== d . 1816596.38 |========================================================== KeyDB 6.3.2 Test: SADD - Parallel Connections: 100 Requests Per Second > Higher Is Better a . 1301147.20 |========================================================= b . 1326330.25 |========================================================== c . 1443334.75 |=============================================================== d . 1235605.25 |====================================================== KeyDB 6.3.2 Test: SADD - Parallel Connections: 500 Requests Per Second > Higher Is Better a . 1369793.36 |=============================================================== b . 1365262.25 |=============================================================== c . 1294096.25 |============================================================ d . 1296243.38 |============================================================ KeyDB 6.3.2 Test: SADD - Parallel Connections: 900 Requests Per Second > Higher Is Better a . 1318361.98 |=========================================================== b . 1402721.25 |=============================================================== c . 1331770.75 |============================================================ d . 1264958.12 |========================================================= KeyDB 6.3.2 Test: HMSET - Parallel Connections: 100 Requests Per Second > Higher Is Better a . 811193.33 |============================================================= b . 838926.19 |=============================================================== c . 858045.06 |================================================================ d . 762636.94 |========================================================= KeyDB 6.3.2 Test: HMSET - Parallel Connections: 500 Requests Per Second > Higher Is Better a . 540612.07 |============================================================ b . 575162.19 |================================================================ c . 544525.88 |============================================================= d . 514398.03 |========================================================= KeyDB 6.3.2 Test: HMSET - Parallel Connections: 900 Requests Per Second > Higher Is Better a . 405828.74 |================================================================ b . 385409.94 |============================================================= c . 373870.94 |=========================================================== d . 405919.94 |================================================================ KeyDB 6.3.2 Test: LPUSH - Parallel Connections: 100 Requests Per Second > Higher Is Better a . 1079455.03 |=========================================================== b . 1009102.12 |======================================================= c . 1053252.50 |========================================================= d . 1161008.62 |=============================================================== KeyDB 6.3.2 Test: LPUSH - Parallel Connections: 500 Requests Per Second > Higher Is Better a . 972928.70 |============================================================ b . 932940.25 |========================================================== c . 1020158.31 |=============================================================== d . 925805.94 |========================================================= KeyDB 6.3.2 Test: LPUSH - Parallel Connections: 900 Requests Per Second > Higher Is Better a . 913352.58 |=========================================================== b . 988728.50 |================================================================ c . 967361.25 |=============================================================== d . 934195.31 |============================================================ Neural Magic DeepSparse 1.3.2 Model: NLP Document Classification, oBERT base uncased on IMDB - Scenario: Asynchronous Multi-Stream ms/batch < Lower Is Better a . 693.75 |=================================================================== b . 697.46 |=================================================================== c . 696.46 |=================================================================== d . 693.88 |=================================================================== Neural Magic DeepSparse 1.3.2 Model: NLP Document Classification, oBERT base uncased on IMDB - Scenario: Synchronous Single-Stream ms/batch < Lower Is Better a . 66.00 |==================================================================== b . 66.01 |==================================================================== c . 66.03 |==================================================================== d . 65.97 |==================================================================== Neural Magic DeepSparse 1.3.2 Model: NLP Sentiment Analysis, 80% Pruned Quantized BERT Base Uncased - Scenario: Asynchronous Multi-Stream ms/batch < Lower Is Better a . 59.37 |==================================================================== b . 59.04 |==================================================================== c . 59.11 |==================================================================== d . 58.80 |=================================================================== Neural Magic DeepSparse 1.3.2 Model: NLP Sentiment Analysis, 80% Pruned Quantized BERT Base Uncased - Scenario: Synchronous Single-Stream ms/batch < Lower Is Better a . 10.30 |==================================================================== b . 10.35 |==================================================================== c . 10.23 |=================================================================== d . 10.30 |==================================================================== Neural Magic DeepSparse 1.3.2 Model: NLP Question Answering, BERT base uncased SQuaD 12layer Pruned90 - Scenario: Asynchronous Multi-Stream ms/batch < Lower Is Better a . 191.41 |=================================================================== b . 192.56 |=================================================================== c . 191.39 |=================================================================== d . 189.93 |================================================================== Neural Magic DeepSparse 1.3.2 Model: NLP Question Answering, BERT base uncased SQuaD 12layer Pruned90 - Scenario: Synchronous Single-Stream ms/batch < Lower Is Better a . 22.51 |==================================================================== b . 22.54 |==================================================================== c . 22.62 |==================================================================== d . 22.53 |==================================================================== Neural Magic DeepSparse 1.3.2 Model: CV Detection, YOLOv5s COCO - Scenario: Asynchronous Multi-Stream ms/batch < Lower Is Better a . 118.97 |=================================================================== b . 117.18 |================================================================== c . 118.67 |=================================================================== d . 117.95 |================================================================== Neural Magic DeepSparse 1.3.2 Model: CV Detection, YOLOv5s COCO - Scenario: Synchronous Single-Stream ms/batch < Lower Is Better a . 14.73 |==================================================================== b . 14.61 |=================================================================== c . 14.65 |==================================================================== d . 14.43 |=================================================================== Neural Magic DeepSparse 1.3.2 Model: CV Classification, ResNet-50 ImageNet - Scenario: Asynchronous Multi-Stream ms/batch < Lower Is Better a . 54.45 |==================================================================== b . 53.75 |=================================================================== c . 53.85 |=================================================================== d . 54.38 |==================================================================== Neural Magic DeepSparse 1.3.2 Model: CV Classification, ResNet-50 ImageNet - Scenario: Synchronous Single-Stream ms/batch < Lower Is Better a . 7.9509 |=================================================================== b . 7.8702 |================================================================== c . 7.8877 |================================================================== d . 7.8787 |================================================================== Neural Magic DeepSparse 1.3.2 Model: NLP Text Classification, DistilBERT mnli - Scenario: Asynchronous Multi-Stream ms/batch < Lower Is Better a . 81.64 |==================================================================== b . 81.69 |==================================================================== c . 80.42 |=================================================================== d . 81.76 |==================================================================== Neural Magic DeepSparse 1.3.2 Model: NLP Text Classification, DistilBERT mnli - Scenario: Synchronous Single-Stream ms/batch < Lower Is Better a . 11.87 |==================================================================== b . 11.85 |==================================================================== c . 11.81 |==================================================================== d . 11.80 |==================================================================== Neural Magic DeepSparse 1.3.2 Model: CV Segmentation, 90% Pruned YOLACT Pruned - Scenario: Asynchronous Multi-Stream ms/batch < Lower Is Better a . 531.49 |=================================================================== b . 530.18 |=================================================================== c . 528.70 |=================================================================== d . 523.94 |================================================================== Neural Magic DeepSparse 1.3.2 Model: CV Segmentation, 90% Pruned YOLACT Pruned - Scenario: Synchronous Single-Stream ms/batch < Lower Is Better a . 55.80 |==================================================================== b . 55.63 |=================================================================== c . 56.09 |==================================================================== d . 55.85 |==================================================================== Neural Magic DeepSparse 1.3.2 Model: NLP Text Classification, BERT base uncased SST2 - Scenario: Asynchronous Multi-Stream ms/batch < Lower Is Better a . 160.48 |================================================================== b . 161.86 |=================================================================== c . 159.44 |================================================================== d . 161.02 |=================================================================== Neural Magic DeepSparse 1.3.2 Model: NLP Text Classification, BERT base uncased SST2 - Scenario: Synchronous Single-Stream ms/batch < Lower Is Better a . 23.57 |==================================================================== b . 23.58 |==================================================================== c . 23.48 |==================================================================== d . 23.57 |==================================================================== Neural Magic DeepSparse 1.3.2 Model: NLP Token Classification, BERT base uncased conll2003 - Scenario: Asynchronous Multi-Stream ms/batch < Lower Is Better a . 694.20 |=================================================================== b . 693.70 |=================================================================== c . 695.23 |=================================================================== d . 693.83 |=================================================================== Neural Magic DeepSparse 1.3.2 Model: NLP Token Classification, BERT base uncased conll2003 - Scenario: Synchronous Single-Stream ms/batch < Lower Is Better a . 65.85 |==================================================================== b . 65.78 |==================================================================== c . 66.11 |==================================================================== d . 65.89 |====================================================================