2 x AMD EPYC 9684X 96-Core testing with a AMD Titanite_4G (RTI1007B BIOS) and ASPEED on Ubuntu 23.10 via the Phoronix Test Suite.
Compare your own system(s) to this result file with the
Phoronix Test Suite by running the command:
phoronix-test-suite benchmark 2403164-NE-DDFG2505160
ddfg
2 x AMD EPYC 9684X 96-Core testing with a AMD Titanite_4G (RTI1007B BIOS) and ASPEED on Ubuntu 23.10 via the Phoronix Test Suite.
a:
Processor: 2 x AMD EPYC 9684X 96-Core @ 2.55GHz (192 Cores / 384 Threads), Motherboard: AMD Titanite_4G (RTI1007B BIOS), Chipset: AMD Device 14a4, Memory: 1520GB, Disk: 3201GB Micron_7450_MTFDKCB3T2TFS + 257GB Flash Drive, Graphics: ASPEED, Network: Broadcom NetXtreme BCM5720 PCIe
OS: Ubuntu 23.10, Kernel: 6.5.0-25-generic (x86_64), Compiler: GCC 13.2.0, File-System: ext4, Screen Resolution: 640x480
b:
Processor: 2 x AMD EPYC 9684X 96-Core @ 2.55GHz (192 Cores / 384 Threads), Motherboard: AMD Titanite_4G (RTI1007B BIOS), Chipset: AMD Device 14a4, Memory: 1520GB, Disk: 3201GB Micron_7450_MTFDKCB3T2TFS + 257GB Flash Drive, Graphics: ASPEED, Network: Broadcom NetXtreme BCM5720 PCIe
OS: Ubuntu 23.10, Kernel: 6.5.0-25-generic (x86_64), Compiler: GCC 13.2.0, File-System: ext4, Screen Resolution: 640x480
c:
Processor: 2 x AMD EPYC 9684X 96-Core @ 2.55GHz (192 Cores / 384 Threads), Motherboard: AMD Titanite_4G (RTI1007B BIOS), Chipset: AMD Device 14a4, Memory: 1520GB, Disk: 3201GB Micron_7450_MTFDKCB3T2TFS + 257GB Flash Drive, Graphics: ASPEED, Network: Broadcom NetXtreme BCM5720 PCIe
OS: Ubuntu 23.10, Kernel: 6.5.0-25-generic (x86_64), Compiler: GCC 13.2.0, File-System: ext4, Screen Resolution: 640x480
d:
Processor: 2 x AMD EPYC 9684X 96-Core @ 2.55GHz (192 Cores / 384 Threads), Motherboard: AMD Titanite_4G (RTI1007B BIOS), Chipset: AMD Device 14a4, Memory: 1520GB, Disk: 3201GB Micron_7450_MTFDKCB3T2TFS + 257GB Flash Drive, Graphics: ASPEED, Network: Broadcom NetXtreme BCM5720 PCIe
OS: Ubuntu 23.10, Kernel: 6.5.0-25-generic (x86_64), Compiler: GCC 13.2.0, File-System: ext4, Screen Resolution: 640x480
SVT-AV1 2.0
Encoder Mode: Preset 12 - Input: Bosphorus 4K
Frames Per Second > Higher Is Better
a . 149.07 |=============================================================
b . 163.18 |===================================================================
c . 160.56 |==================================================================
d . 163.50 |===================================================================
SVT-AV1 2.0
Encoder Mode: Preset 13 - Input: Bosphorus 1080p
Frames Per Second > Higher Is Better
a . 564.41 |==============================================================
b . 606.38 |===================================================================
c . 588.45 |=================================================================
d . 598.04 |==================================================================
SVT-AV1 2.0
Encoder Mode: Preset 8 - Input: Bosphorus 4K
Frames Per Second > Higher Is Better
a . 89.28 |===================================================================
b . 91.29 |====================================================================
c . 90.44 |===================================================================
d . 86.59 |================================================================
SVT-AV1 2.0
Encoder Mode: Preset 8 - Input: Bosphorus 1080p
Frames Per Second > Higher Is Better
a . 176.67 |================================================================
b . 182.53 |==================================================================
c . 184.13 |===================================================================
d . 183.29 |===================================================================
SVT-AV1 2.0
Encoder Mode: Preset 4 - Input: Bosphorus 1080p
Frames Per Second > Higher Is Better
a . 22.96 |===================================================================
b . 22.90 |==================================================================
c . 23.47 |====================================================================
d . 22.71 |==================================================================
SVT-AV1 2.0
Encoder Mode: Preset 13 - Input: Bosphorus 4K
Frames Per Second > Higher Is Better
a . 159.22 |==================================================================
b . 161.65 |===================================================================
c . 162.02 |===================================================================
d . 162.38 |===================================================================
Primesieve 12.1
Length: 1e12
Seconds < Lower Is Better
a . 1.164 |====================================================================
b . 1.164 |====================================================================
c . 1.156 |====================================================================
d . 1.142 |===================================================================
SVT-AV1 2.0
Encoder Mode: Preset 12 - Input: Bosphorus 1080p
Frames Per Second > Higher Is Better
a . 572.35 |==================================================================
b . 566.99 |==================================================================
c . 568.21 |==================================================================
d . 577.44 |===================================================================
Neural Magic DeepSparse 1.7
Model: Llama2 Chat 7b Quantized - Scenario: Synchronous Single-Stream
items/sec > Higher Is Better
a . 20.83 |====================================================================
b . 20.81 |====================================================================
c . 20.73 |====================================================================
d . 20.46 |===================================================================
Neural Magic DeepSparse 1.7
Model: Llama2 Chat 7b Quantized - Scenario: Synchronous Single-Stream
ms/batch < Lower Is Better
a . 47.97 |===================================================================
b . 48.03 |===================================================================
c . 48.22 |===================================================================
d . 48.84 |====================================================================
SVT-AV1 2.0
Encoder Mode: Preset 4 - Input: Bosphorus 4K
Frames Per Second > Higher Is Better
a . 8.557 |===================================================================
b . 8.575 |====================================================================
c . 8.484 |===================================================================
d . 8.634 |====================================================================
Neural Magic DeepSparse 1.7
Model: NLP Text Classification, BERT base uncased SST2, Sparse INT8 - Scenario: Synchronous Single-Stream
ms/batch < Lower Is Better
a . 5.2180 |===================================================================
b . 5.2109 |===================================================================
c . 5.1985 |===================================================================
d . 5.1383 |==================================================================
Neural Magic DeepSparse 1.7
Model: NLP Text Classification, BERT base uncased SST2, Sparse INT8 - Scenario: Synchronous Single-Stream
items/sec > Higher Is Better
a . 191.51 |==================================================================
b . 191.77 |==================================================================
c . 192.23 |==================================================================
d . 194.47 |===================================================================
Neural Magic DeepSparse 1.7
Model: ResNet-50, Baseline - Scenario: Synchronous Single-Stream
items/sec > Higher Is Better
a . 206.82 |==================================================================
b . 209.77 |===================================================================
c . 208.74 |===================================================================
d . 208.98 |===================================================================
Neural Magic DeepSparse 1.7
Model: ResNet-50, Baseline - Scenario: Synchronous Single-Stream
ms/batch < Lower Is Better
a . 4.8321 |===================================================================
b . 4.7642 |==================================================================
c . 4.7875 |==================================================================
d . 4.7823 |==================================================================
Neural Magic DeepSparse 1.7
Model: NLP Token Classification, BERT base uncased conll2003 - Scenario: Asynchronous Multi-Stream
ms/batch < Lower Is Better
a . 715.46 |===================================================================
b . 717.26 |===================================================================
c . 709.26 |==================================================================
d . 715.43 |===================================================================
Neural Magic DeepSparse 1.7
Model: NLP Token Classification, BERT base uncased conll2003 - Scenario: Asynchronous Multi-Stream
items/sec > Higher Is Better
a . 132.95 |===================================================================
b . 132.80 |==================================================================
c . 133.81 |===================================================================
d . 132.91 |===================================================================
Neural Magic DeepSparse 1.7
Model: CV Classification, ResNet-50 ImageNet - Scenario: Synchronous Single-Stream
ms/batch < Lower Is Better
a . 4.8123 |===================================================================
b . 4.7882 |===================================================================
c . 4.7905 |===================================================================
d . 4.7773 |===================================================================
Neural Magic DeepSparse 1.7
Model: CV Classification, ResNet-50 ImageNet - Scenario: Synchronous Single-Stream
items/sec > Higher Is Better
a . 207.67 |===================================================================
b . 208.72 |===================================================================
c . 208.63 |===================================================================
d . 209.19 |===================================================================
Primesieve 12.1
Length: 1e13
Seconds < Lower Is Better
a . 11.79 |====================================================================
b . 11.85 |====================================================================
c . 11.87 |====================================================================
d . 11.84 |====================================================================
Neural Magic DeepSparse 1.7
Model: CV Detection, YOLOv5s COCO, Sparse INT8 - Scenario: Synchronous Single-Stream
items/sec > Higher Is Better
a . 212.18 |===================================================================
b . 212.49 |===================================================================
c . 212.64 |===================================================================
d . 213.56 |===================================================================
Neural Magic DeepSparse 1.7
Model: CV Detection, YOLOv5s COCO, Sparse INT8 - Scenario: Synchronous Single-Stream
ms/batch < Lower Is Better
a . 4.7104 |===================================================================
b . 4.7035 |===================================================================
c . 4.7002 |===================================================================
d . 4.6801 |===================================================================
Neural Magic DeepSparse 1.7
Model: NLP Text Classification, BERT base uncased SST2, Sparse INT8 - Scenario: Asynchronous Multi-Stream
items/sec > Higher Is Better
a . 5566.39 |==================================================================
b . 5538.98 |==================================================================
c . 5534.26 |==================================================================
d . 5554.03 |==================================================================
Neural Magic DeepSparse 1.7
Model: NLP Text Classification, BERT base uncased SST2, Sparse INT8 - Scenario: Asynchronous Multi-Stream
ms/batch < Lower Is Better
a . 17.22 |====================================================================
b . 17.30 |====================================================================
c . 17.32 |====================================================================
d . 17.26 |====================================================================
Neural Magic DeepSparse 1.7
Model: NLP Document Classification, oBERT base uncased on IMDB - Scenario: Asynchronous Multi-Stream
ms/batch < Lower Is Better
a . 713.54 |===================================================================
b . 717.49 |===================================================================
c . 715.88 |===================================================================
d . 714.46 |===================================================================
Neural Magic DeepSparse 1.7
Model: BERT-Large, NLP Question Answering, Sparse INT8 - Scenario: Synchronous Single-Stream
ms/batch < Lower Is Better
a . 14.57 |====================================================================
b . 14.61 |====================================================================
c . 14.58 |====================================================================
d . 14.53 |====================================================================
Neural Magic DeepSparse 1.7
Model: BERT-Large, NLP Question Answering, Sparse INT8 - Scenario: Synchronous Single-Stream
items/sec > Higher Is Better
a . 68.58 |====================================================================
b . 68.40 |====================================================================
c . 68.56 |====================================================================
d . 68.77 |====================================================================
Neural Magic DeepSparse 1.7
Model: ResNet-50, Sparse INT8 - Scenario: Asynchronous Multi-Stream
items/sec > Higher Is Better
a . 17421.47 |=================================================================
b . 17350.46 |=================================================================
c . 17333.35 |=================================================================
d . 17400.58 |=================================================================
Neural Magic DeepSparse 1.7
Model: ResNet-50, Sparse INT8 - Scenario: Asynchronous Multi-Stream
ms/batch < Lower Is Better
a . 5.4946 |===================================================================
b . 5.5170 |===================================================================
c . 5.5224 |===================================================================
d . 5.5020 |===================================================================
Neural Magic DeepSparse 1.7
Model: CV Detection, YOLOv5s COCO, Sparse INT8 - Scenario: Asynchronous Multi-Stream
items/sec > Higher Is Better
a . 798.62 |===================================================================
b . 796.60 |===================================================================
c . 795.03 |===================================================================
d . 797.71 |===================================================================
Neural Magic DeepSparse 1.7
Model: CV Detection, YOLOv5s COCO, Sparse INT8 - Scenario: Asynchronous Multi-Stream
ms/batch < Lower Is Better
a . 119.91 |===================================================================
b . 120.18 |===================================================================
c . 120.40 |===================================================================
d . 120.00 |===================================================================
Neural Magic DeepSparse 1.7
Model: CV Classification, ResNet-50 ImageNet - Scenario: Asynchronous Multi-Stream
items/sec > Higher Is Better
a . 1763.19 |==================================================================
b . 1756.17 |==================================================================
c . 1756.56 |==================================================================
d . 1758.31 |==================================================================
Neural Magic DeepSparse 1.7
Model: CV Classification, ResNet-50 ImageNet - Scenario: Asynchronous Multi-Stream
ms/batch < Lower Is Better
a . 54.35 |====================================================================
b . 54.57 |====================================================================
c . 54.56 |====================================================================
d . 54.51 |====================================================================
Neural Magic DeepSparse 1.7
Model: ResNet-50, Sparse INT8 - Scenario: Synchronous Single-Stream
ms/batch < Lower Is Better
a . 1.2364 |===================================================================
b . 1.2409 |===================================================================
c . 1.2381 |===================================================================
d . 1.2375 |===================================================================
Neural Magic DeepSparse 1.7
Model: ResNet-50, Sparse INT8 - Scenario: Synchronous Single-Stream
items/sec > Higher Is Better
a . 807.17 |===================================================================
b . 804.40 |===================================================================
c . 806.23 |===================================================================
d . 806.32 |===================================================================
Neural Magic DeepSparse 1.7
Model: NLP Document Classification, oBERT base uncased on IMDB - Scenario: Synchronous Single-Stream
items/sec > Higher Is Better
a . 48.52 |====================================================================
b . 48.36 |====================================================================
c . 48.52 |====================================================================
d . 48.51 |====================================================================
Neural Magic DeepSparse 1.7
Model: NLP Document Classification, oBERT base uncased on IMDB - Scenario: Synchronous Single-Stream
ms/batch < Lower Is Better
a . 20.60 |====================================================================
b . 20.67 |====================================================================
c . 20.60 |====================================================================
d . 20.61 |====================================================================
Neural Magic DeepSparse 1.7
Model: NLP Text Classification, DistilBERT mnli - Scenario: Asynchronous Multi-Stream
ms/batch < Lower Is Better
a . 83.66 |====================================================================
b . 83.85 |====================================================================
c . 83.93 |====================================================================
d . 83.77 |====================================================================
Neural Magic DeepSparse 1.7
Model: NLP Token Classification, BERT base uncased conll2003 - Scenario: Synchronous Single-Stream
ms/batch < Lower Is Better
a . 20.60 |====================================================================
b . 20.64 |====================================================================
c . 20.66 |====================================================================
d . 20.66 |====================================================================
Neural Magic DeepSparse 1.7
Model: NLP Token Classification, BERT base uncased conll2003 - Scenario: Synchronous Single-Stream
items/sec > Higher Is Better
a . 48.53 |====================================================================
b . 48.42 |====================================================================
c . 48.38 |====================================================================
d . 48.38 |====================================================================
Neural Magic DeepSparse 1.7
Model: CV Segmentation, 90% Pruned YOLACT Pruned - Scenario: Asynchronous Multi-Stream
ms/batch < Lower Is Better
a . 381.04 |===================================================================
b . 382.12 |===================================================================
c . 382.17 |===================================================================
d . 381.12 |===================================================================
Neural Magic DeepSparse 1.7
Model: NLP Text Classification, DistilBERT mnli - Scenario: Asynchronous Multi-Stream
items/sec > Higher Is Better
a . 1144.50 |==================================================================
b . 1142.99 |==================================================================
c . 1141.24 |==================================================================
d . 1144.01 |==================================================================
Neural Magic DeepSparse 1.7
Model: CV Segmentation, 90% Pruned YOLACT Pruned - Scenario: Asynchronous Multi-Stream
items/sec > Higher Is Better
a . 249.72 |===================================================================
b . 249.40 |===================================================================
c . 249.09 |===================================================================
d . 249.78 |===================================================================
Neural Magic DeepSparse 1.7
Model: ResNet-50, Baseline - Scenario: Asynchronous Multi-Stream
items/sec > Higher Is Better
a . 1762.53 |==================================================================
b . 1758.81 |==================================================================
c . 1759.96 |==================================================================
d . 1763.42 |==================================================================
Neural Magic DeepSparse 1.7
Model: ResNet-50, Baseline - Scenario: Asynchronous Multi-Stream
ms/batch < Lower Is Better
a . 54.37 |====================================================================
b . 54.49 |====================================================================
c . 54.46 |====================================================================
d . 54.36 |====================================================================
Neural Magic DeepSparse 1.7
Model: CV Segmentation, 90% Pruned YOLACT Pruned - Scenario: Synchronous Single-Stream
items/sec > Higher Is Better
a . 64.89 |====================================================================
b . 64.86 |====================================================================
c . 64.92 |====================================================================
d . 65.00 |====================================================================
Neural Magic DeepSparse 1.7
Model: CV Segmentation, 90% Pruned YOLACT Pruned - Scenario: Synchronous Single-Stream
ms/batch < Lower Is Better
a . 15.39 |====================================================================
b . 15.40 |====================================================================
c . 15.38 |====================================================================
d . 15.37 |====================================================================
Neural Magic DeepSparse 1.7
Model: BERT-Large, NLP Question Answering, Sparse INT8 - Scenario: Asynchronous Multi-Stream
items/sec > Higher Is Better
a . 2598.21 |==================================================================
b . 2595.18 |==================================================================
c . 2595.48 |==================================================================
d . 2592.85 |==================================================================
Neural Magic DeepSparse 1.7
Model: BERT-Large, NLP Question Answering, Sparse INT8 - Scenario: Asynchronous Multi-Stream
ms/batch < Lower Is Better
a . 36.89 |====================================================================
b . 36.93 |====================================================================
c . 36.92 |====================================================================
d . 36.96 |====================================================================
Neural Magic DeepSparse 1.7
Model: NLP Document Classification, oBERT base uncased on IMDB - Scenario: Asynchronous Multi-Stream
items/sec > Higher Is Better
a . 132.95 |===================================================================
b . 132.98 |===================================================================
c . 132.88 |===================================================================
d . 133.01 |===================================================================
Neural Magic DeepSparse 1.7
Model: NLP Text Classification, DistilBERT mnli - Scenario: Synchronous Single-Stream
items/sec > Higher Is Better
a . 225.47 |===================================================================
b . 225.45 |===================================================================
c . 225.44 |===================================================================
d . 225.56 |===================================================================
Neural Magic DeepSparse 1.7
Model: NLP Text Classification, DistilBERT mnli - Scenario: Synchronous Single-Stream
ms/batch < Lower Is Better
a . 4.4325 |===================================================================
b . 4.4329 |===================================================================
c . 4.4329 |===================================================================
d . 4.4307 |===================================================================
Neural Magic DeepSparse 1.7
Model: Llama2 Chat 7b Quantized - Scenario: Asynchronous Multi-Stream
ms/batch < Lower Is Better
a . 24242.38 |======================================================
b . 24888.16 |========================================================
c . 28968.95 |=================================================================
d . 24331.06 |=======================================================
Neural Magic DeepSparse 1.7
Model: Llama2 Chat 7b Quantized - Scenario: Asynchronous Multi-Stream
items/sec > Higher Is Better
a . 2.9358 |===================================================================
b . 2.8607 |=================================================================
c . 2.5008 |=========================================================
d . 2.9271 |===================================================================