Intel Xeon Platinum 8490H testing with a Quanta Cloud S6Q-MB-MPS (3A10.uh BIOS) and ASPEED on Ubuntu 22.04 via the Phoronix Test Suite.
Compare your own system(s) to this result file with the
Phoronix Test Suite by running the command:
phoronix-test-suite benchmark 2307296-NE-8490H1S1663
8490h 1s
Intel Xeon Platinum 8490H testing with a Quanta Cloud S6Q-MB-MPS (3A10.uh BIOS) and ASPEED on Ubuntu 22.04 via the Phoronix Test Suite.
a:
Processor: Intel Xeon Platinum 8490H @ 3.50GHz (60 Cores / 120 Threads), Motherboard: Quanta Cloud S6Q-MB-MPS (3A10.uh BIOS), Chipset: Intel Device 1bce, Memory: 512GB, Disk: 3 x 3841GB Micron_9300_MTFDHAL3T8TDP, Graphics: ASPEED, Network: 4 x Intel E810-C for QSFP
OS: Ubuntu 22.04, Kernel: 5.15.0-47-generic (x86_64), Desktop: GNOME Shell 42.4, Display Server: X Server 1.21.1.3, Vulkan: 1.2.204, Compiler: GCC 11.2.0, File-System: ext4, Screen Resolution: 1024x768
b:
Processor: Intel Xeon Platinum 8490H @ 3.50GHz (60 Cores / 120 Threads), Motherboard: Quanta Cloud S6Q-MB-MPS (3A10.uh BIOS), Chipset: Intel Device 1bce, Memory: 512GB, Disk: 3 x 3841GB Micron_9300_MTFDHAL3T8TDP, Graphics: ASPEED, Network: 4 x Intel E810-C for QSFP
OS: Ubuntu 22.04, Kernel: 5.15.0-47-generic (x86_64), Desktop: GNOME Shell 42.4, Display Server: X Server 1.21.1.3, Vulkan: 1.2.204, Compiler: GCC 11.2.0, File-System: ext4, Screen Resolution: 1024x768
c:
Processor: Intel Xeon Platinum 8490H @ 3.50GHz (60 Cores / 120 Threads), Motherboard: Quanta Cloud S6Q-MB-MPS (3A10.uh BIOS), Chipset: Intel Device 1bce, Memory: 512GB, Disk: 3 x 3841GB Micron_9300_MTFDHAL3T8TDP, Graphics: ASPEED, Network: 4 x Intel E810-C for QSFP
OS: Ubuntu 22.04, Kernel: 5.15.0-47-generic (x86_64), Desktop: GNOME Shell 42.4, Display Server: X Server 1.21.1.3, Vulkan: 1.2.204, Compiler: GCC 11.2.0, File-System: ext4, Screen Resolution: 1024x768
d:
Processor: Intel Xeon Platinum 8490H @ 3.50GHz (60 Cores / 120 Threads), Motherboard: Quanta Cloud S6Q-MB-MPS (3A10.uh BIOS), Chipset: Intel Device 1bce, Memory: 512GB, Disk: 3 x 3841GB Micron_9300_MTFDHAL3T8TDP, Graphics: ASPEED, Network: 4 x Intel E810-C for QSFP
OS: Ubuntu 22.04, Kernel: 5.15.0-47-generic (x86_64), Desktop: GNOME Shell 42.4, Display Server: X Server 1.21.1.3, Vulkan: 1.2.204, Compiler: GCC 11.2.0, File-System: ext4, Screen Resolution: 1024x768
e:
Processor: Intel Xeon Platinum 8490H @ 3.50GHz (60 Cores / 120 Threads), Motherboard: Quanta Cloud S6Q-MB-MPS (3A10.uh BIOS), Chipset: Intel Device 1bce, Memory: 512GB, Disk: 3 x 3841GB Micron_9300_MTFDHAL3T8TDP, Graphics: ASPEED, Network: 4 x Intel E810-C for QSFP
OS: Ubuntu 22.04, Kernel: 5.15.0-47-generic (x86_64), Desktop: GNOME Shell 42.4, Display Server: X Server 1.21.1.3, Vulkan: 1.2.204, Compiler: GCC 11.2.0, File-System: ext4, Screen Resolution: 1024x768
Apache Cassandra 4.1.3
Test: Writes
Op/s > Higher Is Better
a . 134694 |================================================================
b . 134932 |================================================================
c . 121708 |==========================================================
d . 140934 |===================================================================
e . 137849 |==================================================================
Redis 7.0.12 + memtier_benchmark 2.0
Protocol: Redis - Clients: 100 - Set To Get Ratio: 1:10
Ops/sec > Higher Is Better
a . 2929613.17 |===============================================================
b . 2583878.29 |========================================================
c . 2611551.59 |========================================================
d . 2593991.10 |========================================================
e . 2755166.14 |===========================================================
Neural Magic DeepSparse 1.5
Model: ResNet-50, Sparse INT8 - Scenario: Synchronous Single-Stream
items/sec > Higher Is Better
a . 759.48 |===================================================================
b . 746.71 |==================================================================
c . 758.34 |===================================================================
d . 759.43 |===================================================================
e . 709.15 |===============================================================
Neural Magic DeepSparse 1.5
Model: ResNet-50, Sparse INT8 - Scenario: Synchronous Single-Stream
ms/batch < Lower Is Better
a . 1.3144 |===============================================================
b . 1.3371 |================================================================
c . 1.3167 |===============================================================
d . 1.3146 |===============================================================
e . 1.4076 |===================================================================
Redis 7.0.12 + memtier_benchmark 2.0
Protocol: Redis - Clients: 100 - Set To Get Ratio: 1:1
Ops/sec > Higher Is Better
a . 2503378.33 |============================================================
b . 2630752.03 |===============================================================
c . 2460387.75 |===========================================================
d . 2493158.97 |============================================================
e . 2474272.95 |===========================================================
Redis 7.0.12 + memtier_benchmark 2.0
Protocol: Redis - Clients: 50 - Set To Get Ratio: 1:5
Ops/sec > Higher Is Better
a . 2613416.65 |===============================================================
b . 2546591.86 |=============================================================
c . 2516320.69 |=============================================================
d . 2452685.99 |===========================================================
e . 2444450.11 |===========================================================
Redis 7.0.12 + memtier_benchmark 2.0
Protocol: Redis - Clients: 100 - Set To Get Ratio: 1:5
Ops/sec > Higher Is Better
a . 2627400.98 |=============================================================
b . 2710324.97 |===============================================================
c . 2554516.65 |===========================================================
d . 2549066.22 |===========================================================
e . 2541163.68 |===========================================================
Redis 7.0.12 + memtier_benchmark 2.0
Protocol: Redis - Clients: 50 - Set To Get Ratio: 1:10
Ops/sec > Higher Is Better
a . 2646929.55 |===============================================================
b . 2544897.58 |=============================================================
c . 2508415.59 |============================================================
d . 2487096.51 |===========================================================
e . 2540848.35 |============================================================
Neural Magic DeepSparse 1.5
Model: NLP Text Classification, BERT base uncased SST2 - Scenario: Asynchronous Multi-Stream
ms/batch < Lower Is Better
a . 137.51 |===================================================================
b . 133.78 |=================================================================
c . 136.00 |==================================================================
d . 130.52 |===============================================================
e . 137.76 |===================================================================
Neural Magic DeepSparse 1.5
Model: NLP Text Classification, BERT base uncased SST2 - Scenario: Asynchronous Multi-Stream
items/sec > Higher Is Better
a . 218.11 |================================================================
b . 224.16 |=================================================================
c . 220.49 |================================================================
d . 229.45 |===================================================================
e . 217.72 |================================================================
Neural Magic DeepSparse 1.5
Model: NLP Document Classification, oBERT base uncased on IMDB - Scenario: Asynchronous Multi-Stream
ms/batch < Lower Is Better
a . 534.50 |===================================================================
b . 517.16 |=================================================================
c . 517.84 |=================================================================
d . 511.42 |================================================================
e . 520.22 |=================================================================
Redis 7.0.12 + memtier_benchmark 2.0
Protocol: Redis - Clients: 50 - Set To Get Ratio: 1:1
Ops/sec > Higher Is Better
a . 2462536.66 |===============================================================
b . 2470315.91 |===============================================================
c . 2377430.14 |=============================================================
d . 2383147.07 |=============================================================
e . 2414601.07 |==============================================================
Neural Magic DeepSparse 1.5
Model: NLP Document Classification, oBERT base uncased on IMDB - Scenario: Asynchronous Multi-Stream
items/sec > Higher Is Better
a . 56.12 |==================================================================
b . 57.79 |===================================================================
c . 57.93 |====================================================================
d . 58.25 |====================================================================
e . 57.21 |===================================================================
Dragonflydb 1.6.2
Clients Per Thread: 10 - Set To Get Ratio: 1:5
Ops/sec > Higher Is Better
a . 14247982.45 |============================================================
b . 14476834.10 |=============================================================
c . 14235868.61 |============================================================
d . 14750102.52 |==============================================================
e . 14392511.79 |============================================================
Neural Magic DeepSparse 1.5
Model: BERT-Large, NLP Question Answering, Sparse INT8 - Scenario: Synchronous Single-Stream
items/sec > Higher Is Better
a . 117.23 |===================================================================
b . 114.61 |==================================================================
c . 116.35 |==================================================================
d . 117.22 |===================================================================
e . 113.24 |=================================================================
Neural Magic DeepSparse 1.5
Model: BERT-Large, NLP Question Answering, Sparse INT8 - Scenario: Synchronous Single-Stream
ms/batch < Lower Is Better
a . 8.5248 |=================================================================
b . 8.7193 |==================================================================
c . 8.5903 |=================================================================
d . 8.5255 |=================================================================
e . 8.8247 |===================================================================
Neural Magic DeepSparse 1.5
Model: NLP Text Classification, DistilBERT mnli - Scenario: Synchronous Single-Stream
items/sec > Higher Is Better
a . 121.89 |==================================================================
b . 123.68 |==================================================================
c . 124.62 |===================================================================
d . 123.45 |==================================================================
e . 120.44 |=================================================================
Neural Magic DeepSparse 1.5
Model: NLP Text Classification, DistilBERT mnli - Scenario: Synchronous Single-Stream
ms/batch < Lower Is Better
a . 8.1989 |==================================================================
b . 8.0804 |=================================================================
c . 8.0190 |=================================================================
d . 8.0945 |=================================================================
e . 8.2973 |===================================================================
Dragonflydb 1.6.2
Clients Per Thread: 10 - Set To Get Ratio: 1:10
Ops/sec > Higher Is Better
a . 14662941.42 |==============================================================
b . 14292262.52 |============================================================
c . 14204640.25 |============================================================
d . 14205235.71 |============================================================
e . 14390358.99 |=============================================================
Neural Magic DeepSparse 1.5
Model: CV Classification, ResNet-50 ImageNet - Scenario: Synchronous Single-Stream
ms/batch < Lower Is Better
a . 3.7271 |==================================================================
b . 3.6871 |=================================================================
c . 3.6770 |=================================================================
d . 3.6743 |=================================================================
e . 3.7788 |===================================================================
Neural Magic DeepSparse 1.5
Model: CV Classification, ResNet-50 ImageNet - Scenario: Synchronous Single-Stream
items/sec > Higher Is Better
a . 267.99 |==================================================================
b . 270.85 |===================================================================
c . 271.62 |===================================================================
d . 271.78 |===================================================================
e . 264.31 |=================================================================
Neural Magic DeepSparse 1.5
Model: NLP Text Classification, DistilBERT mnli - Scenario: Asynchronous Multi-Stream
items/sec > Higher Is Better
a . 475.72 |=================================================================
b . 487.46 |===================================================================
c . 487.18 |===================================================================
d . 485.72 |===================================================================
e . 488.25 |===================================================================
Neural Magic DeepSparse 1.5
Model: NLP Text Classification, DistilBERT mnli - Scenario: Asynchronous Multi-Stream
ms/batch < Lower Is Better
a . 63.01 |====================================================================
b . 61.51 |==================================================================
c . 61.55 |==================================================================
d . 61.69 |===================================================================
e . 61.42 |==================================================================
Neural Magic DeepSparse 1.5
Model: NLP Question Answering, BERT base uncased SQuaD 12layer Pruned90 - Scenario: Synchronous Single-Stream
items/sec > Higher Is Better
a . 86.93 |====================================================================
b . 87.16 |====================================================================
c . 86.79 |====================================================================
d . 87.34 |====================================================================
e . 85.16 |==================================================================
Neural Magic DeepSparse 1.5
Model: NLP Question Answering, BERT base uncased SQuaD 12layer Pruned90 - Scenario: Synchronous Single-Stream
ms/batch < Lower Is Better
a . 11.49 |===================================================================
b . 11.47 |==================================================================
c . 11.51 |===================================================================
d . 11.44 |==================================================================
e . 11.73 |====================================================================
Neural Magic DeepSparse 1.5
Model: CV Detection, YOLOv5s COCO - Scenario: Synchronous Single-Stream
items/sec > Higher Is Better
a . 184.39 |===================================================================
b . 185.75 |===================================================================
c . 184.20 |==================================================================
d . 184.93 |===================================================================
e . 181.20 |=================================================================
Neural Magic DeepSparse 1.5
Model: CV Detection, YOLOv5s COCO - Scenario: Synchronous Single-Stream
ms/batch < Lower Is Better
a . 5.4182 |==================================================================
b . 5.3786 |=================================================================
c . 5.4239 |==================================================================
d . 5.4020 |==================================================================
e . 5.5134 |===================================================================
Neural Magic DeepSparse 1.5
Model: NLP Sentiment Analysis, 80% Pruned Quantized BERT Base Uncased - Scenario: Asynchronous Multi-Stream
items/sec > Higher Is Better
a . 648.32 |===================================================================
b . 644.05 |==================================================================
c . 649.30 |===================================================================
d . 648.39 |===================================================================
e . 633.45 |=================================================================
Neural Magic DeepSparse 1.5
Model: NLP Sentiment Analysis, 80% Pruned Quantized BERT Base Uncased - Scenario: Asynchronous Multi-Stream
ms/batch < Lower Is Better
a . 46.25 |==================================================================
b . 46.54 |===================================================================
c . 46.18 |==================================================================
d . 46.24 |==================================================================
e . 47.32 |====================================================================
Neural Magic DeepSparse 1.5
Model: BERT-Large, NLP Question Answering - Scenario: Asynchronous Multi-Stream
items/sec > Higher Is Better
a . 62.58 |====================================================================
b . 61.60 |===================================================================
c . 61.41 |===================================================================
d . 62.17 |====================================================================
e . 61.07 |==================================================================
Neural Magic DeepSparse 1.5
Model: ResNet-50, Baseline - Scenario: Synchronous Single-Stream
ms/batch < Lower Is Better
a . 3.6608 |=================================================================
b . 3.6961 |==================================================================
c . 3.7248 |===================================================================
d . 3.6938 |==================================================================
e . 3.7499 |===================================================================
Neural Magic DeepSparse 1.5
Model: ResNet-50, Baseline - Scenario: Synchronous Single-Stream
items/sec > Higher Is Better
a . 272.80 |===================================================================
b . 270.23 |==================================================================
c . 268.15 |==================================================================
d . 270.35 |==================================================================
e . 266.34 |=================================================================
Neural Magic DeepSparse 1.5
Model: NLP Text Classification, BERT base uncased SST2, Sparse INT8 - Scenario: Synchronous Single-Stream
items/sec > Higher Is Better
a . 208.96 |===================================================================
b . 209.10 |===================================================================
c . 208.43 |===================================================================
d . 209.66 |===================================================================
e . 205.24 |==================================================================
Neural Magic DeepSparse 1.5
Model: NLP Text Classification, BERT base uncased SST2, Sparse INT8 - Scenario: Synchronous Single-Stream
ms/batch < Lower Is Better
a . 4.7834 |==================================================================
b . 4.7802 |==================================================================
c . 4.7956 |==================================================================
d . 4.7675 |==================================================================
e . 4.8699 |===================================================================
Neural Magic DeepSparse 1.5
Model: BERT-Large, NLP Question Answering - Scenario: Synchronous Single-Stream
ms/batch < Lower Is Better
a . 47.24 |===================================================================
b . 47.25 |===================================================================
c . 47.42 |===================================================================
d . 47.47 |===================================================================
e . 48.22 |====================================================================
Neural Magic DeepSparse 1.5
Model: BERT-Large, NLP Question Answering - Scenario: Synchronous Single-Stream
items/sec > Higher Is Better
a . 21.16 |====================================================================
b . 21.16 |====================================================================
c . 21.09 |====================================================================
d . 21.06 |====================================================================
e . 20.74 |===================================================================
Blender 3.6
Blend File: Fishy Cat - Compute: CPU-Only
Seconds < Lower Is Better
a . 35.75 |====================================================================
b . 35.26 |===================================================================
c . 35.03 |===================================================================
d . 35.16 |===================================================================
e . 35.50 |====================================================================
Neural Magic DeepSparse 1.5
Model: NLP Token Classification, BERT base uncased conll2003 - Scenario: Asynchronous Multi-Stream
items/sec > Higher Is Better
a . 57.08 |===================================================================
b . 57.57 |===================================================================
c . 57.99 |====================================================================
d . 58.21 |====================================================================
e . 57.52 |===================================================================
Neural Magic DeepSparse 1.5
Model: NLP Token Classification, BERT base uncased conll2003 - Scenario: Asynchronous Multi-Stream
ms/batch < Lower Is Better
a . 525.14 |===================================================================
b . 520.87 |==================================================================
c . 516.04 |==================================================================
d . 514.90 |==================================================================
e . 519.27 |==================================================================
Neural Magic DeepSparse 1.5
Model: NLP Sentiment Analysis, 80% Pruned Quantized BERT Base Uncased - Scenario: Synchronous Single-Stream
items/sec > Higher Is Better
a . 180.14 |===================================================================
b . 179.68 |===================================================================
c . 178.72 |==================================================================
d . 180.44 |===================================================================
e . 177.01 |==================================================================
Neural Magic DeepSparse 1.5
Model: NLP Sentiment Analysis, 80% Pruned Quantized BERT Base Uncased - Scenario: Synchronous Single-Stream
ms/batch < Lower Is Better
a . 5.5454 |==================================================================
b . 5.5592 |==================================================================
c . 5.5886 |==================================================================
d . 5.5358 |==================================================================
e . 5.6420 |===================================================================
Neural Magic DeepSparse 1.5
Model: BERT-Large, NLP Question Answering - Scenario: Asynchronous Multi-Stream
ms/batch < Lower Is Better
a . 479.13 |==================================================================
b . 486.19 |===================================================================
c . 488.25 |===================================================================
d . 482.44 |==================================================================
e . 488.23 |===================================================================
Dragonflydb 1.6.2
Clients Per Thread: 10 - Set To Get Ratio: 1:100
Ops/sec > Higher Is Better
a . 14338166.15 |=============================================================
b . 14432034.21 |=============================================================
c . 14307949.03 |=============================================================
d . 14571297.77 |==============================================================
e . 14492478.36 |==============================================================
BRL-CAD 7.36
VGR Performance Metric
VGR Performance Metric > Higher Is Better
a . 825917 |===================================================================
b . 823602 |===================================================================
c . 820410 |===================================================================
d . 812806 |==================================================================
e . 822977 |===================================================================
Neural Magic DeepSparse 1.5
Model: CV Detection, YOLOv5s COCO, Sparse INT8 - Scenario: Synchronous Single-Stream
ms/batch < Lower Is Better
a . 5.4431 |==================================================================
b . 5.4340 |==================================================================
c . 5.4372 |==================================================================
d . 5.4307 |==================================================================
e . 5.5183 |===================================================================
Neural Magic DeepSparse 1.5
Model: CV Detection, YOLOv5s COCO, Sparse INT8 - Scenario: Synchronous Single-Stream
items/sec > Higher Is Better
a . 183.63 |===================================================================
b . 183.93 |===================================================================
c . 183.82 |===================================================================
d . 184.05 |===================================================================
e . 181.13 |==================================================================
Blender 3.6
Blend File: Classroom - Compute: CPU-Only
Seconds < Lower Is Better
a . 68.93 |===================================================================
b . 69.16 |===================================================================
c . 70.03 |====================================================================
d . 69.16 |===================================================================
e . 69.25 |===================================================================
Neural Magic DeepSparse 1.5
Model: NLP Document Classification, oBERT base uncased on IMDB - Scenario: Synchronous Single-Stream
items/sec > Higher Is Better
a . 35.49 |====================================================================
b . 35.36 |====================================================================
c . 35.62 |====================================================================
d . 35.59 |====================================================================
e . 35.09 |===================================================================
Neural Magic DeepSparse 1.5
Model: NLP Document Classification, oBERT base uncased on IMDB - Scenario: Synchronous Single-Stream
ms/batch < Lower Is Better
a . 28.17 |===================================================================
b . 28.27 |===================================================================
c . 28.07 |===================================================================
d . 28.09 |===================================================================
e . 28.49 |====================================================================
Neural Magic DeepSparse 1.5
Model: NLP Question Answering, BERT base uncased SQuaD 12layer Pruned90 - Scenario: Asynchronous Multi-Stream
items/sec > Higher Is Better
a . 192.50 |===================================================================
b . 191.99 |===================================================================
c . 191.61 |===================================================================
d . 190.82 |==================================================================
e . 191.50 |===================================================================
Neural Magic DeepSparse 1.5
Model: NLP Question Answering, BERT base uncased SQuaD 12layer Pruned90 - Scenario: Asynchronous Multi-Stream
ms/batch < Lower Is Better
a . 155.80 |==================================================================
b . 156.21 |===================================================================
c . 156.46 |===================================================================
d . 157.18 |===================================================================
e . 156.56 |===================================================================
Neural Magic DeepSparse 1.5
Model: NLP Text Classification, BERT base uncased SST2, Sparse INT8 - Scenario: Asynchronous Multi-Stream
ms/batch < Lower Is Better
a . 15.70 |===================================================================
b . 15.83 |====================================================================
c . 15.83 |====================================================================
d . 15.84 |====================================================================
e . 15.72 |===================================================================
Blender 3.6
Blend File: Pabellon Barcelona - Compute: CPU-Only
Seconds < Lower Is Better
a . 88.73 |====================================================================
b . 88.50 |====================================================================
c . 88.83 |====================================================================
d . 88.18 |====================================================================
e . 88.09 |===================================================================
Neural Magic DeepSparse 1.5
Model: NLP Text Classification, BERT base uncased SST2, Sparse INT8 - Scenario: Asynchronous Multi-Stream
items/sec > Higher Is Better
a . 1905.98 |==================================================================
b . 1891.06 |=================================================================
c . 1891.33 |=================================================================
d . 1890.57 |=================================================================
e . 1905.15 |==================================================================
Neural Magic DeepSparse 1.5
Model: CV Segmentation, 90% Pruned YOLACT Pruned - Scenario: Synchronous Single-Stream
ms/batch < Lower Is Better
a . 25.00 |====================================================================
b . 24.99 |====================================================================
c . 25.01 |====================================================================
d . 25.05 |====================================================================
e . 25.13 |====================================================================
Neural Magic DeepSparse 1.5
Model: CV Segmentation, 90% Pruned YOLACT Pruned - Scenario: Synchronous Single-Stream
items/sec > Higher Is Better
a . 39.94 |====================================================================
b . 39.96 |====================================================================
c . 39.93 |====================================================================
d . 39.87 |====================================================================
e . 39.74 |====================================================================
Neural Magic DeepSparse 1.5
Model: CV Classification, ResNet-50 ImageNet - Scenario: Asynchronous Multi-Stream
items/sec > Higher Is Better
a . 780.40 |===================================================================
b . 781.30 |===================================================================
c . 780.33 |===================================================================
d . 778.76 |===================================================================
e . 782.54 |===================================================================
Blender 3.6
Blend File: BMW27 - Compute: CPU-Only
Seconds < Lower Is Better
a . 25.74 |====================================================================
b . 25.63 |====================================================================
c . 25.69 |====================================================================
d . 25.75 |====================================================================
e . 25.71 |====================================================================
Neural Magic DeepSparse 1.5
Model: CV Classification, ResNet-50 ImageNet - Scenario: Asynchronous Multi-Stream
ms/batch < Lower Is Better
a . 38.39 |====================================================================
b . 38.37 |====================================================================
c . 38.41 |====================================================================
d . 38.49 |====================================================================
e . 38.31 |====================================================================
Neural Magic DeepSparse 1.5
Model: NLP Text Classification, BERT base uncased SST2 - Scenario: Synchronous Single-Stream
items/sec > Higher Is Better
a . 61.65 |====================================================================
b . 61.89 |====================================================================
c . 61.61 |====================================================================
d . 61.83 |====================================================================
e . 61.85 |====================================================================
Neural Magic DeepSparse 1.5
Model: NLP Text Classification, BERT base uncased SST2 - Scenario: Synchronous Single-Stream
ms/batch < Lower Is Better
a . 16.21 |====================================================================
b . 16.15 |====================================================================
c . 16.22 |====================================================================
d . 16.17 |====================================================================
e . 16.16 |====================================================================
Neural Magic DeepSparse 1.5
Model: NLP Token Classification, BERT base uncased conll2003 - Scenario: Synchronous Single-Stream
items/sec > Higher Is Better
a . 35.76 |====================================================================
b . 35.72 |====================================================================
c . 35.86 |====================================================================
d . 35.76 |====================================================================
e . 35.88 |====================================================================
Neural Magic DeepSparse 1.5
Model: NLP Token Classification, BERT base uncased conll2003 - Scenario: Synchronous Single-Stream
ms/batch < Lower Is Better
a . 27.96 |====================================================================
b . 27.99 |====================================================================
c . 27.88 |====================================================================
d . 27.96 |====================================================================
e . 27.86 |====================================================================
Neural Magic DeepSparse 1.5
Model: CV Segmentation, 90% Pruned YOLACT Pruned - Scenario: Asynchronous Multi-Stream
ms/batch < Lower Is Better
a . 396.73 |===================================================================
b . 395.21 |===================================================================
c . 396.68 |===================================================================
d . 396.99 |===================================================================
e . 396.17 |===================================================================
Neural Magic DeepSparse 1.5
Model: CV Segmentation, 90% Pruned YOLACT Pruned - Scenario: Asynchronous Multi-Stream
items/sec > Higher Is Better
a . 75.60 |====================================================================
b . 75.88 |====================================================================
c . 75.61 |====================================================================
d . 75.55 |====================================================================
e . 75.71 |====================================================================
Neural Magic DeepSparse 1.5
Model: CV Detection, YOLOv5s COCO, Sparse INT8 - Scenario: Asynchronous Multi-Stream
ms/batch < Lower Is Better
a . 86.04 |====================================================================
b . 85.93 |====================================================================
c . 86.28 |====================================================================
d . 86.29 |====================================================================
e . 86.13 |====================================================================
Neural Magic DeepSparse 1.5
Model: CV Detection, YOLOv5s COCO - Scenario: Asynchronous Multi-Stream
ms/batch < Lower Is Better
a . 86.43 |====================================================================
b . 86.25 |====================================================================
c . 86.16 |====================================================================
d . 86.53 |====================================================================
e . 86.17 |====================================================================
Neural Magic DeepSparse 1.5
Model: CV Detection, YOLOv5s COCO - Scenario: Asynchronous Multi-Stream
items/sec > Higher Is Better
a . 346.94 |===================================================================
b . 347.66 |===================================================================
c . 348.01 |===================================================================
d . 346.56 |===================================================================
e . 347.76 |===================================================================
Neural Magic DeepSparse 1.5
Model: ResNet-50, Baseline - Scenario: Asynchronous Multi-Stream
ms/batch < Lower Is Better
a . 38.41 |====================================================================
b . 38.34 |====================================================================
c . 38.41 |====================================================================
d . 38.42 |====================================================================
e . 38.28 |====================================================================
Neural Magic DeepSparse 1.5
Model: ResNet-50, Baseline - Scenario: Asynchronous Multi-Stream
items/sec > Higher Is Better
a . 780.50 |===================================================================
b . 782.04 |===================================================================
c . 780.53 |===================================================================
d . 780.31 |===================================================================
e . 783.20 |===================================================================
Neural Magic DeepSparse 1.5
Model: CV Detection, YOLOv5s COCO, Sparse INT8 - Scenario: Asynchronous Multi-Stream
items/sec > Higher Is Better
a . 348.53 |===================================================================
b . 348.72 |===================================================================
c . 347.59 |===================================================================
d . 347.51 |===================================================================
e . 348.14 |===================================================================
Neural Magic DeepSparse 1.5
Model: BERT-Large, NLP Question Answering, Sparse INT8 - Scenario: Asynchronous Multi-Stream
items/sec > Higher Is Better
a . 868.50 |===================================================================
b . 869.47 |===================================================================
c . 870.76 |===================================================================
d . 871.21 |===================================================================
e . 870.67 |===================================================================
Blender 3.6
Blend File: Barbershop - Compute: CPU-Only
Seconds < Lower Is Better
a . 272.91 |===================================================================
b . 273.14 |===================================================================
c . 272.64 |===================================================================
d . 272.41 |===================================================================
e . 272.44 |===================================================================
Neural Magic DeepSparse 1.5
Model: BERT-Large, NLP Question Answering, Sparse INT8 - Scenario: Asynchronous Multi-Stream
ms/batch < Lower Is Better
a . 34.49 |====================================================================
b . 34.46 |====================================================================
c . 34.42 |====================================================================
d . 34.40 |====================================================================
e . 34.42 |====================================================================
Neural Magic DeepSparse 1.5
Model: ResNet-50, Sparse INT8 - Scenario: Asynchronous Multi-Stream
items/sec > Higher Is Better
a . 5672.80 |==================================================================
b . 5678.52 |==================================================================
c . 5677.04 |==================================================================
d . 5665.47 |==================================================================
e . 5679.78 |==================================================================
Neural Magic DeepSparse 1.5
Model: ResNet-50, Sparse INT8 - Scenario: Asynchronous Multi-Stream
ms/batch < Lower Is Better
a . 5.2680 |===================================================================
b . 5.2627 |===================================================================
c . 5.2632 |===================================================================
d . 5.2743 |===================================================================
e . 5.2615 |===================================================================
Crypto++ 8.8
Test: Unkeyed Algorithms
MiB/second > Higher Is Better
a . 452.34 |===================================================================
Crypto++ 8.8
Test: Keyed Algorithms
MiB/second > Higher Is Better
a . 595.37 |===================================================================
Crypto++ 8.8
Test: All Algorithms
MiB/second > Higher Is Better
a . 1663.92 |==================================================================
Redis 7.0.12 + memtier_benchmark 2.0
Protocol: Redis - Clients: 500 - Set To Get Ratio: 1:10
Redis 7.0.12 + memtier_benchmark 2.0
Protocol: Redis - Clients: 500 - Set To Get Ratio: 1:5
Redis 7.0.12 + memtier_benchmark 2.0
Protocol: Redis - Clients: 500 - Set To Get Ratio: 1:1
Dragonflydb 1.6.2
Clients Per Thread: 20 - Set To Get Ratio: 1:100
Dragonflydb 1.6.2
Clients Per Thread: 20 - Set To Get Ratio: 1:10
Dragonflydb 1.6.2
Clients Per Thread: 20 - Set To Get Ratio: 1:5