nnn wed AMD EPYC 7551 32-Core testing with a GIGABYTE MZ31-AR0-00 v01010101 (F10 BIOS) and ASPEED on Debian 11 via the Phoronix Test Suite. AMD EPYC 7551 32-Core: Processor: AMD EPYC 7551 32-Core @ 2.00GHz (32 Cores / 64 Threads), Motherboard: GIGABYTE MZ31-AR0-00 v01010101 (F10 BIOS), Chipset: AMD 17h, Memory: 8 x 4 GB DDR4-2133MT/s 9ASF51272PZ-2G6E1, Disk: Samsung SSD 960 EVO 500GB, Graphics: ASPEED, Network: Realtek RTL8111/8168/8411 + 2 x Broadcom NetXtreme II BCM57810 10 OS: Debian 11, Kernel: 5.10.0-9-amd64 (x86_64), Compiler: GCC 10.2.1 20210110, File-System: ext4, Screen Resolution: 1024x768 s: Processor: AMD EPYC 7551 32-Core @ 2.00GHz (32 Cores / 64 Threads), Motherboard: GIGABYTE MZ31-AR0-00 v01010101 (F10 BIOS), Chipset: AMD 17h, Memory: 8 x 4 GB DDR4-2133MT/s 9ASF51272PZ-2G6E1, Disk: Samsung SSD 960 EVO 500GB, Graphics: ASPEED, Network: Realtek RTL8111/8168/8411 + 2 x Broadcom NetXtreme II BCM57810 10 OS: Debian 11, Kernel: 5.10.0-9-amd64 (x86_64), Compiler: GCC 10.2.1 20210110, File-System: ext4, Screen Resolution: 1024x768 c: Processor: AMD EPYC 7551 32-Core @ 2.00GHz (32 Cores / 64 Threads), Motherboard: GIGABYTE MZ31-AR0-00 v01010101 (F10 BIOS), Chipset: AMD 17h, Memory: 8 x 4 GB DDR4-2133MT/s 9ASF51272PZ-2G6E1, Disk: Samsung SSD 960 EVO 500GB, Graphics: ASPEED, Network: Realtek RTL8111/8168/8411 + 2 x Broadcom NetXtreme II BCM57810 10 OS: Debian 11, Kernel: 5.10.0-9-amd64 (x86_64), Compiler: GCC 10.2.1 20210110, File-System: ext4, Screen Resolution: 1024x768 dav1d 1.2.1 Video Input: Chimera 1080p FPS > Higher Is Better AMD EPYC 7551 32-Core . 182.56 |=============================================== s ..................... 183.23 |=============================================== c ..................... 182.49 |=============================================== dav1d 1.2.1 Video Input: Summer Nature 4K FPS > Higher Is Better AMD EPYC 7551 32-Core . 102.99 |=============================================== s ..................... 103.58 |=============================================== c ..................... 101.89 |============================================== dav1d 1.2.1 Video Input: Summer Nature 1080p FPS > Higher Is Better AMD EPYC 7551 32-Core . 269.31 |=============================================== s ..................... 256.58 |============================================= c ..................... 264.87 |============================================== dav1d 1.2.1 Video Input: Chimera 1080p 10-bit FPS > Higher Is Better AMD EPYC 7551 32-Core . 150.66 |=============================================== s ..................... 148.99 |============================================== c ..................... 147.83 |============================================== Neural Magic DeepSparse 1.5 Model: NLP Document Classification, oBERT base uncased on IMDB - Scenario: Asynchronous Multi-Stream items/sec > Higher Is Better AMD EPYC 7551 32-Core . 8.9595 |=========================================== s ..................... 9.5624 |============================================= c ..................... 9.8974 |=============================================== Neural Magic DeepSparse 1.5 Model: NLP Document Classification, oBERT base uncased on IMDB - Scenario: Asynchronous Multi-Stream ms/batch < Lower Is Better AMD EPYC 7551 32-Core . 1748.19 |============================================== s ..................... 1658.04 |============================================ c ..................... 1577.40 |========================================== Neural Magic DeepSparse 1.5 Model: NLP Document Classification, oBERT base uncased on IMDB - Scenario: Synchronous Single-Stream items/sec > Higher Is Better AMD EPYC 7551 32-Core . 2.4912 |============================================ s ..................... 2.6621 |=============================================== c ..................... 2.6317 |============================================== Neural Magic DeepSparse 1.5 Model: NLP Document Classification, oBERT base uncased on IMDB - Scenario: Synchronous Single-Stream ms/batch < Lower Is Better AMD EPYC 7551 32-Core . 401.39 |=============================================== s ..................... 375.63 |============================================ c ..................... 379.97 |============================================ Neural Magic DeepSparse 1.5 Model: NLP Sentiment Analysis, 80% Pruned Quantized BERT Base Uncased - Scenario: Asynchronous Multi-Stream items/sec > Higher Is Better AMD EPYC 7551 32-Core . 45.33 |========================= s ..................... 84.01 |=============================================== c ..................... 85.69 |================================================ Neural Magic DeepSparse 1.5 Model: NLP Sentiment Analysis, 80% Pruned Quantized BERT Base Uncased - Scenario: Asynchronous Multi-Stream ms/batch < Lower Is Better AMD EPYC 7551 32-Core . 352.11 |=============================================== s ..................... 190.04 |========================= c ..................... 186.52 |========================= Neural Magic DeepSparse 1.5 Model: NLP Sentiment Analysis, 80% Pruned Quantized BERT Base Uncased - Scenario: Synchronous Single-Stream items/sec > Higher Is Better AMD EPYC 7551 32-Core . 23.23 |================================================ s ..................... 22.82 |=============================================== c ..................... 22.74 |=============================================== Neural Magic DeepSparse 1.5 Model: NLP Sentiment Analysis, 80% Pruned Quantized BERT Base Uncased - Scenario: Synchronous Single-Stream ms/batch < Lower Is Better AMD EPYC 7551 32-Core . 43.02 |=============================================== s ..................... 43.81 |================================================ c ..................... 43.96 |================================================ Neural Magic DeepSparse 1.5 Model: NLP Question Answering, BERT base uncased SQuaD 12layer Pruned90 - Scenario: Asynchronous Multi-Stream items/sec > Higher Is Better AMD EPYC 7551 32-Core . 21.07 |==================================== s ..................... 26.36 |============================================= c ..................... 28.12 |================================================ Neural Magic DeepSparse 1.5 Model: NLP Question Answering, BERT base uncased SQuaD 12layer Pruned90 - Scenario: Asynchronous Multi-Stream ms/batch < Lower Is Better AMD EPYC 7551 32-Core . 755.83 |=============================================== s ..................... 603.44 |====================================== c ..................... 565.62 |=================================== Neural Magic DeepSparse 1.5 Model: NLP Question Answering, BERT base uncased SQuaD 12layer Pruned90 - Scenario: Synchronous Single-Stream items/sec > Higher Is Better AMD EPYC 7551 32-Core . 8.9719 |=============================================== s ..................... 8.7323 |============================================== c ..................... 6.9893 |===================================== Neural Magic DeepSparse 1.5 Model: NLP Question Answering, BERT base uncased SQuaD 12layer Pruned90 - Scenario: Synchronous Single-Stream ms/batch < Lower Is Better AMD EPYC 7551 32-Core . 111.44 |===================================== s ..................... 114.50 |====================================== c ..................... 143.06 |=============================================== Neural Magic DeepSparse 1.5 Model: CV Detection, YOLOv5s COCO - Scenario: Asynchronous Multi-Stream items/sec > Higher Is Better AMD EPYC 7551 32-Core . 60.93 |============================================ s ..................... 67.11 |================================================ c ..................... 67.14 |================================================ Neural Magic DeepSparse 1.5 Model: CV Detection, YOLOv5s COCO - Scenario: Asynchronous Multi-Stream ms/batch < Lower Is Better AMD EPYC 7551 32-Core . 261.88 |=============================================== s ..................... 237.98 |=========================================== c ..................... 237.55 |=========================================== Neural Magic DeepSparse 1.5 Model: CV Detection, YOLOv5s COCO - Scenario: Synchronous Single-Stream items/sec > Higher Is Better AMD EPYC 7551 32-Core . 15.24 |============================================== s ..................... 15.69 |================================================ c ..................... 15.75 |================================================ Neural Magic DeepSparse 1.5 Model: CV Detection, YOLOv5s COCO - Scenario: Synchronous Single-Stream ms/batch < Lower Is Better AMD EPYC 7551 32-Core . 65.60 |================================================ s ..................... 63.71 |=============================================== c ..................... 63.45 |============================================== Neural Magic DeepSparse 1.5 Model: CV Classification, ResNet-50 ImageNet - Scenario: Asynchronous Multi-Stream items/sec > Higher Is Better AMD EPYC 7551 32-Core . 124.83 |========================================== s ..................... 140.93 |=============================================== c ..................... 139.34 |============================================== Neural Magic DeepSparse 1.5 Model: CV Classification, ResNet-50 ImageNet - Scenario: Asynchronous Multi-Stream ms/batch < Lower Is Better AMD EPYC 7551 32-Core . 127.87 |=============================================== s ..................... 113.36 |========================================== c ..................... 114.56 |========================================== Neural Magic DeepSparse 1.5 Model: CV Classification, ResNet-50 ImageNet - Scenario: Synchronous Single-Stream items/sec > Higher Is Better AMD EPYC 7551 32-Core . 31.33 |============================================= s ..................... 33.06 |================================================ c ..................... 33.07 |================================================ Neural Magic DeepSparse 1.5 Model: CV Classification, ResNet-50 ImageNet - Scenario: Synchronous Single-Stream ms/batch < Lower Is Better AMD EPYC 7551 32-Core . 31.90 |================================================ s ..................... 30.23 |============================================= c ..................... 30.22 |============================================= Neural Magic DeepSparse 1.5 Model: NLP Text Classification, DistilBERT mnli - Scenario: Asynchronous Multi-Stream items/sec > Higher Is Better AMD EPYC 7551 32-Core . 87.76 |================================================ s ..................... 88.15 |================================================ c ..................... 88.01 |================================================ Neural Magic DeepSparse 1.5 Model: NLP Text Classification, DistilBERT mnli - Scenario: Asynchronous Multi-Stream ms/batch < Lower Is Better AMD EPYC 7551 32-Core . 181.85 |=============================================== s ..................... 181.08 |=============================================== c ..................... 181.47 |=============================================== Neural Magic DeepSparse 1.5 Model: NLP Text Classification, DistilBERT mnli - Scenario: Synchronous Single-Stream items/sec > Higher Is Better AMD EPYC 7551 32-Core . 20.80 |================================================ s ..................... 18.46 |=========================================== c ..................... 20.23 |=============================================== Neural Magic DeepSparse 1.5 Model: NLP Text Classification, DistilBERT mnli - Scenario: Synchronous Single-Stream ms/batch < Lower Is Better AMD EPYC 7551 32-Core . 48.06 |=========================================== s ..................... 54.17 |================================================ c ..................... 49.42 |============================================ Neural Magic DeepSparse 1.5 Model: CV Segmentation, 90% Pruned YOLACT Pruned - Scenario: Asynchronous Multi-Stream items/sec > Higher Is Better AMD EPYC 7551 32-Core . 7.4092 |===================================== s ..................... 9.4079 |=============================================== c ..................... 9.4389 |=============================================== Neural Magic DeepSparse 1.5 Model: CV Segmentation, 90% Pruned YOLACT Pruned - Scenario: Asynchronous Multi-Stream ms/batch < Lower Is Better AMD EPYC 7551 32-Core . 2114.18 |============================================== s ..................... 1680.27 |===================================== c ..................... 1666.33 |==================================== Neural Magic DeepSparse 1.5 Model: CV Segmentation, 90% Pruned YOLACT Pruned - Scenario: Synchronous Single-Stream items/sec > Higher Is Better AMD EPYC 7551 32-Core . 1.9302 |============================================= s ..................... 1.9670 |============================================== c ..................... 2.0185 |=============================================== Neural Magic DeepSparse 1.5 Model: CV Segmentation, 90% Pruned YOLACT Pruned - Scenario: Synchronous Single-Stream ms/batch < Lower Is Better AMD EPYC 7551 32-Core . 518.06 |=============================================== s ..................... 508.36 |============================================== c ..................... 495.38 |============================================= Neural Magic DeepSparse 1.5 Model: NLP Text Classification, BERT base uncased SST2 - Scenario: Asynchronous Multi-Stream items/sec > Higher Is Better AMD EPYC 7551 32-Core . 44.56 |================================================ s ..................... 44.33 |================================================ c ..................... 42.75 |============================================== Neural Magic DeepSparse 1.5 Model: NLP Text Classification, BERT base uncased SST2 - Scenario: Asynchronous Multi-Stream ms/batch < Lower Is Better AMD EPYC 7551 32-Core . 357.63 |============================================= s ..................... 359.73 |============================================= c ..................... 372.49 |=============================================== Neural Magic DeepSparse 1.5 Model: NLP Text Classification, BERT base uncased SST2 - Scenario: Synchronous Single-Stream items/sec > Higher Is Better AMD EPYC 7551 32-Core . 10.1448 |============================================= s ..................... 9.8690 |============================================ c ..................... 10.4036 |============================================== Neural Magic DeepSparse 1.5 Model: NLP Text Classification, BERT base uncased SST2 - Scenario: Synchronous Single-Stream ms/batch < Lower Is Better AMD EPYC 7551 32-Core . 98.56 |============================================== s ..................... 101.31 |=============================================== c ..................... 96.11 |============================================= Neural Magic DeepSparse 1.5 Model: NLP Token Classification, BERT base uncased conll2003 - Scenario: Asynchronous Multi-Stream items/sec > Higher Is Better AMD EPYC 7551 32-Core . 9.5030 |========================================== s ..................... 9.3221 |========================================== c ..................... 10.3080 |============================================== Neural Magic DeepSparse 1.5 Model: NLP Token Classification, BERT base uncased conll2003 - Scenario: Asynchronous Multi-Stream ms/batch < Lower Is Better AMD EPYC 7551 32-Core . 1671.59 |============================================= s ..................... 1691.45 |============================================== c ..................... 1526.05 |========================================== Neural Magic DeepSparse 1.5 Model: NLP Token Classification, BERT base uncased conll2003 - Scenario: Synchronous Single-Stream items/sec > Higher Is Better AMD EPYC 7551 32-Core . 2.4982 |============================================= s ..................... 2.6347 |=============================================== c ..................... 2.6157 |=============================================== Neural Magic DeepSparse 1.5 Model: NLP Token Classification, BERT base uncased conll2003 - Scenario: Synchronous Single-Stream ms/batch < Lower Is Better AMD EPYC 7551 32-Core . 400.27 |=============================================== s ..................... 379.53 |============================================= c ..................... 382.29 |=============================================