Benchmarks for a future article.
Compare your own system(s) to this result file with the
Phoronix Test Suite by running the command:
phoronix-test-suite benchmark 2312143-NE-A8154652071
a
Benchmarks for a future article.
a:
Processor: 2 x INTEL XEON PLATINUM 8592+ @ 3.90GHz (128 Cores / 256 Threads), Motherboard: Quanta Cloud S6Q-MB-MPS (3B05.TEL4P1 BIOS), Chipset: Intel Device 1bce, Memory: 1008GB, Disk: 3201GB Micron_7450_MTFDKCB3T2TFS, Graphics: ASPEED, Network: 2 x Intel X710 for 10GBASE-T
OS: Ubuntu 23.10, Kernel: 6.5.0-13-generic (x86_64), Compiler: GCC 13.2.0, File-System: ext4, Screen Resolution: 1920x1080
b:
Processor: 2 x INTEL XEON PLATINUM 8592+ @ 3.90GHz (128 Cores / 256 Threads), Motherboard: Quanta Cloud S6Q-MB-MPS (3B05.TEL4P1 BIOS), Chipset: Intel Device 1bce, Memory: 1008GB, Disk: 3201GB Micron_7450_MTFDKCB3T2TFS, Graphics: ASPEED, Network: 2 x Intel X710 for 10GBASE-T
OS: Ubuntu 23.10, Kernel: 6.5.0-13-generic (x86_64), Compiler: GCC 13.2.0, File-System: ext4, Screen Resolution: 1920x1080
c:
Processor: 2 x INTEL XEON PLATINUM 8592+ @ 3.90GHz (128 Cores / 256 Threads), Motherboard: Quanta Cloud S6Q-MB-MPS (3B05.TEL4P1 BIOS), Chipset: Intel Device 1bce, Memory: 1008GB, Disk: 3201GB Micron_7450_MTFDKCB3T2TFS, Graphics: ASPEED, Network: 2 x Intel X710 for 10GBASE-T
OS: Ubuntu 23.10, Kernel: 6.5.0-13-generic (x86_64), Compiler: GCC 13.2.0, File-System: ext4, Screen Resolution: 1920x1080
d:
Processor: 2 x INTEL XEON PLATINUM 8592+ @ 3.90GHz (128 Cores / 256 Threads), Motherboard: Quanta Cloud S6Q-MB-MPS (3B05.TEL4P1 BIOS), Chipset: Intel Device 1bce, Memory: 1008GB, Disk: 3201GB Micron_7450_MTFDKCB3T2TFS, Graphics: ASPEED, Network: 2 x Intel X710 for 10GBASE-T
OS: Ubuntu 23.10, Kernel: 6.5.0-13-generic (x86_64), Compiler: GCC 13.2.0, File-System: ext4, Screen Resolution: 1920x1080
WRF 4.2.2
Input: conus 2.5km
Seconds < Lower Is Better
a . 5566.73 |=================================================================
b . 5600.98 |==================================================================
c . 5583.11 |==================================================================
d . 5617.20 |==================================================================
NWChem 7.0.2
Input: C240 Buckyball
Seconds < Lower Is Better
a . 1744.0 |==================================================================
b . 1730.7 |==================================================================
c . 1757.3 |===================================================================
d . 1748.0 |===================================================================
Neural Magic DeepSparse 1.6
Model: NLP Token Classification, BERT base uncased conll2003 - Scenario: Synchronous Single-Stream
ms/batch < Lower Is Better
a . 31.58 |====================================================================
b . 31.73 |====================================================================
c . 31.56 |====================================================================
d . 30.29 |=================================================================
Neural Magic DeepSparse 1.6
Model: NLP Token Classification, BERT base uncased conll2003 - Scenario: Synchronous Single-Stream
items/sec > Higher Is Better
a . 31.70 |=================================================================
b . 31.53 |=================================================================
c . 31.72 |=================================================================
d . 33.05 |====================================================================
Neural Magic DeepSparse 1.6
Model: NLP Text Classification, BERT base uncased SST2, Sparse INT8 - Scenario: Synchronous Single-Stream
ms/batch < Lower Is Better
a . 4.2083 |=================================================================
b . 4.2295 |=================================================================
c . 4.3436 |===================================================================
d . 4.1604 |================================================================
Neural Magic DeepSparse 1.6
Model: NLP Text Classification, BERT base uncased SST2, Sparse INT8 - Scenario: Synchronous Single-Stream
items/sec > Higher Is Better
a . 238.65 |===================================================================
b . 237.05 |==================================================================
c . 230.76 |================================================================
d . 240.23 |===================================================================
Neural Magic DeepSparse 1.6
Model: NLP Document Classification, oBERT base uncased on IMDB - Scenario: Synchronous Single-Stream
ms/batch < Lower Is Better
a . 32.06 |====================================================================
b . 32.04 |====================================================================
c . 31.65 |===================================================================
d . 30.42 |=================================================================
Neural Magic DeepSparse 1.6
Model: NLP Document Classification, oBERT base uncased on IMDB - Scenario: Synchronous Single-Stream
items/sec > Higher Is Better
a . 31.19 |================================================================
b . 31.23 |=================================================================
c . 31.63 |=================================================================
d . 32.89 |====================================================================
Neural Magic DeepSparse 1.6
Model: CV Detection, YOLOv5s COCO, Sparse INT8 - Scenario: Synchronous Single-Stream
ms/batch < Lower Is Better
a . 5.1485 |===================================================================
b . 5.1659 |===================================================================
c . 5.1802 |===================================================================
d . 5.1614 |===================================================================
Neural Magic DeepSparse 1.6
Model: CV Detection, YOLOv5s COCO, Sparse INT8 - Scenario: Synchronous Single-Stream
items/sec > Higher Is Better
a . 194.25 |===================================================================
b . 193.60 |===================================================================
c . 193.09 |===================================================================
d . 193.82 |===================================================================
Neural Magic DeepSparse 1.6
Model: NLP Text Classification, DistilBERT mnli - Scenario: Synchronous Single-Stream
ms/batch < Lower Is Better
a . 5.0073 |==================================================================
b . 5.0196 |==================================================================
c . 5.0179 |==================================================================
d . 5.0808 |===================================================================
Neural Magic DeepSparse 1.6
Model: NLP Text Classification, DistilBERT mnli - Scenario: Synchronous Single-Stream
items/sec > Higher Is Better
a . 199.69 |===================================================================
b . 199.34 |===================================================================
c . 199.33 |===================================================================
d . 196.79 |==================================================================
Neural Magic DeepSparse 1.6
Model: CV Detection, YOLOv5s COCO - Scenario: Synchronous Single-Stream
ms/batch < Lower Is Better
a . 5.2382 |===================================================================
b . 5.2617 |===================================================================
c . 5.2651 |===================================================================
d . 5.2589 |===================================================================
Neural Magic DeepSparse 1.6
Model: CV Detection, YOLOv5s COCO - Scenario: Synchronous Single-Stream
items/sec > Higher Is Better
a . 190.84 |===================================================================
b . 190.04 |===================================================================
c . 189.82 |===================================================================
d . 190.14 |===================================================================
Neural Magic DeepSparse 1.6
Model: ResNet-50, Sparse INT8 - Scenario: Asynchronous Multi-Stream
ms/batch < Lower Is Better
a . 5.6852 |===================================================================
b . 5.7140 |===================================================================
c . 5.7165 |===================================================================
d . 5.6908 |===================================================================
Neural Magic DeepSparse 1.6
Model: ResNet-50, Sparse INT8 - Scenario: Asynchronous Multi-Stream
items/sec > Higher Is Better
a . 11228.66 |=================================================================
b . 11169.85 |=================================================================
c . 11163.66 |=================================================================
d . 11221.14 |=================================================================
Neural Magic DeepSparse 1.6
Model: ResNet-50, Baseline - Scenario: Synchronous Single-Stream
ms/batch < Lower Is Better
a . 2.9492 |===================================================================
b . 2.9602 |===================================================================
c . 2.9530 |===================================================================
d . 2.9573 |===================================================================
Neural Magic DeepSparse 1.6
Model: ResNet-50, Baseline - Scenario: Synchronous Single-Stream
items/sec > Higher Is Better
a . 338.96 |===================================================================
b . 337.68 |===================================================================
c . 338.55 |===================================================================
d . 338.15 |===================================================================
Neural Magic DeepSparse 1.6
Model: CV Classification, ResNet-50 ImageNet - Scenario: Synchronous Single-Stream
ms/batch < Lower Is Better
a . 2.9484 |===================================================================
b . 2.9500 |===================================================================
c . 2.9555 |===================================================================
d . 2.9455 |===================================================================
Neural Magic DeepSparse 1.6
Model: CV Classification, ResNet-50 ImageNet - Scenario: Synchronous Single-Stream
items/sec > Higher Is Better
a . 339.04 |===================================================================
b . 338.86 |===================================================================
c . 338.22 |===================================================================
d . 339.37 |===================================================================
Neural Magic DeepSparse 1.6
Model: CV Segmentation, 90% Pruned YOLACT Pruned - Scenario: Synchronous Single-Stream
ms/batch < Lower Is Better
a . 28.59 |====================================================================
b . 28.47 |====================================================================
c . 28.55 |====================================================================
d . 28.56 |====================================================================
Neural Magic DeepSparse 1.6
Model: CV Segmentation, 90% Pruned YOLACT Pruned - Scenario: Synchronous Single-Stream
items/sec > Higher Is Better
a . 34.96 |====================================================================
b . 35.11 |====================================================================
c . 35.01 |====================================================================
d . 35.00 |====================================================================
Neural Magic DeepSparse 1.6
Model: BERT-Large, NLP Question Answering - Scenario: Synchronous Single-Stream
ms/batch < Lower Is Better
a . 33.40 |===================================================================
b . 32.90 |==================================================================
c . 33.00 |==================================================================
d . 33.86 |====================================================================
Neural Magic DeepSparse 1.6
Model: BERT-Large, NLP Question Answering - Scenario: Synchronous Single-Stream
items/sec > Higher Is Better
a . 29.94 |===================================================================
b . 30.39 |====================================================================
c . 30.29 |====================================================================
d . 29.52 |==================================================================
Neural Magic DeepSparse 1.6
Model: NLP Text Classification, BERT base uncased SST2, Sparse INT8 - Scenario: Asynchronous Multi-Stream
ms/batch < Lower Is Better
a . 14.04 |====================================================================
b . 14.01 |====================================================================
c . 14.01 |====================================================================
d . 14.03 |====================================================================
Neural Magic DeepSparse 1.6
Model: NLP Text Classification, BERT base uncased SST2, Sparse INT8 - Scenario: Asynchronous Multi-Stream
items/sec > Higher Is Better
a . 4554.98 |==================================================================
b . 4563.51 |==================================================================
c . 4563.43 |==================================================================
d . 4556.02 |==================================================================
Neural Magic DeepSparse 1.6
Model: BERT-Large, NLP Question Answering - Scenario: Asynchronous Multi-Stream
ms/batch < Lower Is Better
a . 407.28 |===================================================================
b . 408.63 |===================================================================
c . 408.97 |===================================================================
d . 408.16 |===================================================================
Neural Magic DeepSparse 1.6
Model: BERT-Large, NLP Question Answering - Scenario: Asynchronous Multi-Stream
items/sec > Higher Is Better
a . 156.06 |===================================================================
b . 155.72 |===================================================================
c . 155.77 |===================================================================
d . 155.60 |===================================================================
Neural Magic DeepSparse 1.6
Model: CV Segmentation, 90% Pruned YOLACT Pruned - Scenario: Asynchronous Multi-Stream
ms/batch < Lower Is Better
a . 347.66 |=================================================================
b . 353.69 |==================================================================
c . 359.89 |===================================================================
d . 352.22 |==================================================================
Neural Magic DeepSparse 1.6
Model: CV Segmentation, 90% Pruned YOLACT Pruned - Scenario: Asynchronous Multi-Stream
items/sec > Higher Is Better
a . 183.50 |===================================================================
b . 180.32 |==================================================================
c . 177.15 |=================================================================
d . 181.20 |==================================================================
Neural Magic DeepSparse 1.6
Model: BERT-Large, NLP Question Answering, Sparse INT8 - Scenario: Synchronous Single-Stream
ms/batch < Lower Is Better
a . 10.36 |===================================================================
b . 10.37 |===================================================================
c . 10.35 |===================================================================
d . 10.45 |====================================================================
Neural Magic DeepSparse 1.6
Model: BERT-Large, NLP Question Answering, Sparse INT8 - Scenario: Synchronous Single-Stream
items/sec > Higher Is Better
a . 96.43 |====================================================================
b . 96.38 |====================================================================
c . 96.55 |====================================================================
d . 95.70 |===================================================================
Neural Magic DeepSparse 1.6
Model: ResNet-50, Sparse INT8 - Scenario: Synchronous Single-Stream
ms/batch < Lower Is Better
a . 0.9681 |===================================================================
b . 0.9706 |===================================================================
c . 0.9556 |==================================================================
d . 0.9667 |===================================================================
Neural Magic DeepSparse 1.6
Model: ResNet-50, Sparse INT8 - Scenario: Synchronous Single-Stream
items/sec > Higher Is Better
a . 1030.32 |=================================================================
b . 1027.51 |=================================================================
c . 1043.28 |==================================================================
d . 1032.24 |=================================================================
Neural Magic DeepSparse 1.6
Model: NLP Document Classification, oBERT base uncased on IMDB - Scenario: Asynchronous Multi-Stream
ms/batch < Lower Is Better
a . 475.70 |===================================================================
b . 474.19 |===================================================================
c . 475.68 |===================================================================
d . 477.61 |===================================================================
Neural Magic DeepSparse 1.6
Model: NLP Document Classification, oBERT base uncased on IMDB - Scenario: Asynchronous Multi-Stream
items/sec > Higher Is Better
a . 133.52 |===================================================================
b . 133.72 |===================================================================
c . 133.48 |===================================================================
d . 133.01 |===================================================================
Neural Magic DeepSparse 1.6
Model: NLP Token Classification, BERT base uncased conll2003 - Scenario: Asynchronous Multi-Stream
ms/batch < Lower Is Better
a . 475.13 |===================================================================
b . 474.37 |===================================================================
c . 475.35 |===================================================================
d . 475.45 |===================================================================
Neural Magic DeepSparse 1.6
Model: NLP Token Classification, BERT base uncased conll2003 - Scenario: Asynchronous Multi-Stream
items/sec > Higher Is Better
a . 133.60 |===================================================================
b . 133.82 |===================================================================
c . 133.60 |===================================================================
d . 133.60 |===================================================================
Neural Magic DeepSparse 1.6
Model: BERT-Large, NLP Question Answering, Sparse INT8 - Scenario: Asynchronous Multi-Stream
ms/batch < Lower Is Better
a . 33.98 |===================================================================
b . 34.08 |====================================================================
c . 33.99 |===================================================================
d . 34.25 |====================================================================
Neural Magic DeepSparse 1.6
Model: BERT-Large, NLP Question Answering, Sparse INT8 - Scenario: Asynchronous Multi-Stream
items/sec > Higher Is Better
a . 1880.77 |==================================================================
b . 1875.90 |==================================================================
c . 1879.84 |==================================================================
d . 1865.02 |=================================================================
Neural Magic DeepSparse 1.6
Model: NLP Text Classification, DistilBERT mnli - Scenario: Asynchronous Multi-Stream
ms/batch < Lower Is Better
a . 51.93 |====================================================================
b . 51.93 |====================================================================
c . 51.92 |====================================================================
d . 52.07 |====================================================================
Neural Magic DeepSparse 1.6
Model: NLP Text Classification, DistilBERT mnli - Scenario: Asynchronous Multi-Stream
items/sec > Higher Is Better
a . 1231.20 |==================================================================
b . 1231.29 |==================================================================
c . 1231.27 |==================================================================
d . 1226.80 |==================================================================
Neural Magic DeepSparse 1.6
Model: ResNet-50, Baseline - Scenario: Asynchronous Multi-Stream
ms/batch < Lower Is Better
a . 34.92 |====================================================================
b . 35.01 |====================================================================
c . 34.97 |====================================================================
d . 34.76 |====================================================================
Neural Magic DeepSparse 1.6
Model: ResNet-50, Baseline - Scenario: Asynchronous Multi-Stream
items/sec > Higher Is Better
a . 1829.96 |==================================================================
b . 1824.38 |==================================================================
c . 1827.98 |==================================================================
d . 1837.50 |==================================================================
Neural Magic DeepSparse 1.6
Model: CV Detection, YOLOv5s COCO - Scenario: Asynchronous Multi-Stream
ms/batch < Lower Is Better
a . 77.18 |====================================================================
b . 77.09 |====================================================================
c . 77.12 |====================================================================
d . 77.46 |====================================================================
Neural Magic DeepSparse 1.6
Model: CV Detection, YOLOv5s COCO - Scenario: Asynchronous Multi-Stream
items/sec > Higher Is Better
a . 828.04 |===================================================================
b . 828.65 |===================================================================
c . 828.45 |===================================================================
d . 825.08 |===================================================================
Neural Magic DeepSparse 1.6
Model: CV Classification, ResNet-50 ImageNet - Scenario: Asynchronous Multi-Stream
ms/batch < Lower Is Better
a . 34.87 |===================================================================
b . 35.14 |====================================================================
c . 34.82 |===================================================================
d . 35.10 |====================================================================
Neural Magic DeepSparse 1.6
Model: CV Classification, ResNet-50 ImageNet - Scenario: Asynchronous Multi-Stream
items/sec > Higher Is Better
a . 1832.79 |==================================================================
b . 1817.99 |=================================================================
c . 1834.48 |==================================================================
d . 1819.69 |=================================================================
Neural Magic DeepSparse 1.6
Model: CV Detection, YOLOv5s COCO, Sparse INT8 - Scenario: Asynchronous Multi-Stream
ms/batch < Lower Is Better
a . 74.82 |====================================================================
b . 74.94 |====================================================================
c . 74.79 |====================================================================
d . 75.08 |====================================================================
Neural Magic DeepSparse 1.6
Model: CV Detection, YOLOv5s COCO, Sparse INT8 - Scenario: Asynchronous Multi-Stream
items/sec > Higher Is Better
a . 854.05 |===================================================================
b . 852.95 |===================================================================
c . 854.25 |===================================================================
d . 850.89 |===================================================================