dfgg AMD Ryzen Threadripper 3970X 32-Core testing with a ASUS ROG ZENITH II EXTREME (1802 BIOS) and AMD Radeon RX 5700 8GB on Ubuntu 22.04 via the Phoronix Test Suite. a: Processor: AMD Ryzen Threadripper 3970X 32-Core @ 3.70GHz (32 Cores / 64 Threads), Motherboard: ASUS ROG ZENITH II EXTREME (1802 BIOS), Chipset: AMD Starship/Matisse, Memory: 4 x 16GB DDR4-3600MT/s Corsair CMT64GX4M4Z3600C16, Disk: Samsung SSD 980 PRO 500GB, Graphics: AMD Radeon RX 5700 8GB (1750/875MHz), Audio: AMD Navi 10 HDMI Audio, Monitor: ASUS VP28U, Network: Aquantia AQC107 NBase-T/IEEE + Intel I211 + Intel Wi-Fi 6 AX200 OS: Ubuntu 22.04, Kernel: 6.5.0-21-generic (x86_64), Desktop: GNOME Shell 42.2, Display Server: X Server + Wayland, OpenGL: 4.6 Mesa 22.0.1 (LLVM 13.0.1 DRM 3.54), Vulkan: 1.2.204, Compiler: GCC 11.4.0, File-System: ext4, Screen Resolution: 3840x2160 b: Processor: AMD Ryzen Threadripper 3970X 32-Core @ 3.70GHz (32 Cores / 64 Threads), Motherboard: ASUS ROG ZENITH II EXTREME (1802 BIOS), Chipset: AMD Starship/Matisse, Memory: 4 x 16GB DDR4-3600MT/s Corsair CMT64GX4M4Z3600C16, Disk: Samsung SSD 980 PRO 500GB, Graphics: AMD Radeon RX 5700 8GB (1750/875MHz), Audio: AMD Navi 10 HDMI Audio, Monitor: ASUS VP28U, Network: Aquantia AQC107 NBase-T/IEEE + Intel I211 + Intel Wi-Fi 6 AX200 OS: Ubuntu 22.04, Kernel: 6.5.0-21-generic (x86_64), Desktop: GNOME Shell 42.2, Display Server: X Server + Wayland, OpenGL: 4.6 Mesa 22.0.1 (LLVM 13.0.1 DRM 3.54), Vulkan: 1.2.204, Compiler: GCC 11.4.0, File-System: ext4, Screen Resolution: 3840x2160 c: Processor: AMD Ryzen Threadripper 3970X 32-Core @ 3.70GHz (32 Cores / 64 Threads), Motherboard: ASUS ROG ZENITH II EXTREME (1802 BIOS), Chipset: AMD Starship/Matisse, Memory: 4 x 16GB DDR4-3600MT/s Corsair CMT64GX4M4Z3600C16, Disk: Samsung SSD 980 PRO 500GB, Graphics: AMD Radeon RX 5700 8GB (1750/875MHz), Audio: AMD Navi 10 HDMI Audio, Monitor: ASUS VP28U, Network: Aquantia AQC107 NBase-T/IEEE + Intel I211 + Intel Wi-Fi 6 AX200 OS: Ubuntu 22.04, Kernel: 6.5.0-21-generic (x86_64), Desktop: GNOME Shell 42.2, Display Server: X Server + Wayland, OpenGL: 4.6 Mesa 22.0.1 (LLVM 13.0.1 DRM 3.54), Vulkan: 1.2.204, Compiler: GCC 11.4.0, File-System: ext4, Screen Resolution: 3840x2160 srsRAN Project 23.10.1-20240219 Test: PDSCH Processor Benchmark, Throughput Total Mbps > Higher Is Better a . 11658.0 |================================================================== b . 11575.9 |================================================================== c . 11512.1 |================================================================= srsRAN Project 23.10.1-20240219 Test: PDSCH Processor Benchmark, Throughput Thread Mbps > Higher Is Better a . 484.5 |=================================================================== b . 490.6 |==================================================================== c . 484.4 |=================================================================== SVT-AV1 2.0 Encoder Mode: Preset 4 - Input: Bosphorus 4K Frames Per Second > Higher Is Better a . 4.698 |=================================================================== b . 4.746 |==================================================================== c . 4.762 |==================================================================== SVT-AV1 2.0 Encoder Mode: Preset 8 - Input: Bosphorus 4K Frames Per Second > Higher Is Better a . 45.49 |=================================================================== b . 46.03 |==================================================================== c . 45.72 |==================================================================== SVT-AV1 2.0 Encoder Mode: Preset 12 - Input: Bosphorus 4K Frames Per Second > Higher Is Better a . 107.18 |================================================================== b . 107.34 |================================================================== c . 108.61 |=================================================================== SVT-AV1 2.0 Encoder Mode: Preset 13 - Input: Bosphorus 4K Frames Per Second > Higher Is Better a . 106.59 |================================================================== b . 107.78 |=================================================================== c . 108.23 |=================================================================== SVT-AV1 2.0 Encoder Mode: Preset 4 - Input: Bosphorus 1080p Frames Per Second > Higher Is Better a . 12.56 |==================================================================== b . 12.27 |================================================================== c . 12.59 |==================================================================== SVT-AV1 2.0 Encoder Mode: Preset 8 - Input: Bosphorus 1080p Frames Per Second > Higher Is Better a . 83.47 |=================================================================== b . 84.22 |==================================================================== c . 83.69 |==================================================================== SVT-AV1 2.0 Encoder Mode: Preset 12 - Input: Bosphorus 1080p Frames Per Second > Higher Is Better a . 297.09 |=================================================================== b . 298.40 |=================================================================== c . 298.84 |=================================================================== SVT-AV1 2.0 Encoder Mode: Preset 13 - Input: Bosphorus 1080p Frames Per Second > Higher Is Better a . 369.31 |=================================================================== b . 368.72 |=================================================================== c . 365.61 |================================================================== Timed Linux Kernel Compilation 6.8 Build: defconfig Seconds < Lower Is Better a . 46.70 |================================================================ b . 49.30 |==================================================================== c . 49.24 |==================================================================== Timed Linux Kernel Compilation 6.8 Build: allmodconfig Seconds < Lower Is Better a . 488.90 |=================================================================== b . 490.32 |=================================================================== c . 490.08 |=================================================================== Parallel BZIP2 Compression 1.1.13 FreeBSD-13.0-RELEASE-amd64-memstick.img Compression Seconds < Lower Is Better a . 2.845905 |================================================================ b . 2.811454 |=============================================================== c . 2.894649 |================================================================= Primesieve 12.1 Length: 1e12 Seconds < Lower Is Better a . 5.503 |==================================================================== b . 5.521 |==================================================================== c . 5.513 |==================================================================== Primesieve 12.1 Length: 1e13 Seconds < Lower Is Better a . 68.35 |==================================================================== b . 68.22 |==================================================================== c . 68.13 |==================================================================== Neural Magic DeepSparse 1.7 Model: NLP Document Classification, oBERT base uncased on IMDB - Scenario: Asynchronous Multi-Stream items/sec > Higher Is Better a . 30.30 |=================================================================== b . 30.54 |==================================================================== c . 30.44 |==================================================================== Neural Magic DeepSparse 1.7 Model: NLP Document Classification, oBERT base uncased on IMDB - Scenario: Asynchronous Multi-Stream ms/batch < Lower Is Better a . 525.40 |=================================================================== b . 522.35 |=================================================================== c . 524.87 |=================================================================== Neural Magic DeepSparse 1.7 Model: NLP Document Classification, oBERT base uncased on IMDB - Scenario: Synchronous Single-Stream items/sec > Higher Is Better a . 17.64 |==================================================================== b . 17.71 |==================================================================== c . 17.70 |==================================================================== Neural Magic DeepSparse 1.7 Model: NLP Document Classification, oBERT base uncased on IMDB - Scenario: Synchronous Single-Stream ms/batch < Lower Is Better a . 56.67 |==================================================================== b . 56.44 |==================================================================== c . 56.49 |==================================================================== Neural Magic DeepSparse 1.7 Model: NLP Text Classification, BERT base uncased SST2, Sparse INT8 - Scenario: Asynchronous Multi-Stream items/sec > Higher Is Better a . 672.34 |=================================================================== b . 669.96 |=================================================================== c . 672.72 |=================================================================== Neural Magic DeepSparse 1.7 Model: NLP Text Classification, BERT base uncased SST2, Sparse INT8 - Scenario: Asynchronous Multi-Stream ms/batch < Lower Is Better a . 23.77 |==================================================================== b . 23.86 |==================================================================== c . 23.76 |==================================================================== Neural Magic DeepSparse 1.7 Model: NLP Text Classification, BERT base uncased SST2, Sparse INT8 - Scenario: Synchronous Single-Stream items/sec > Higher Is Better a . 168.53 |================================================================== b . 170.46 |=================================================================== c . 171.45 |=================================================================== Neural Magic DeepSparse 1.7 Model: NLP Text Classification, BERT base uncased SST2, Sparse INT8 - Scenario: Synchronous Single-Stream ms/batch < Lower Is Better a . 5.9275 |=================================================================== b . 5.8612 |================================================================== c . 5.8272 |================================================================== Neural Magic DeepSparse 1.7 Model: ResNet-50, Baseline - Scenario: Asynchronous Multi-Stream items/sec > Higher Is Better a . 320.60 |================================================================== b . 322.23 |=================================================================== c . 324.37 |=================================================================== Neural Magic DeepSparse 1.7 Model: ResNet-50, Baseline - Scenario: Asynchronous Multi-Stream ms/batch < Lower Is Better a . 49.88 |==================================================================== b . 49.63 |==================================================================== c . 49.30 |=================================================================== Neural Magic DeepSparse 1.7 Model: ResNet-50, Baseline - Scenario: Synchronous Single-Stream items/sec > Higher Is Better a . 143.45 |================================================================= b . 147.89 |=================================================================== c . 148.16 |=================================================================== Neural Magic DeepSparse 1.7 Model: ResNet-50, Baseline - Scenario: Synchronous Single-Stream ms/batch < Lower Is Better a . 6.9635 |=================================================================== b . 6.7539 |================================================================= c . 6.7418 |================================================================= Neural Magic DeepSparse 1.7 Model: ResNet-50, Sparse INT8 - Scenario: Asynchronous Multi-Stream items/sec > Higher Is Better a . 1782.40 |================================================================ b . 1813.59 |================================================================= c . 1833.36 |================================================================== Neural Magic DeepSparse 1.7 Model: ResNet-50, Sparse INT8 - Scenario: Asynchronous Multi-Stream ms/batch < Lower Is Better a . 8.9514 |=================================================================== b . 8.7970 |================================================================== c . 8.7013 |================================================================= Neural Magic DeepSparse 1.7 Model: ResNet-50, Sparse INT8 - Scenario: Synchronous Single-Stream items/sec > Higher Is Better a . 754.33 |================================================================= b . 748.35 |================================================================= c . 774.06 |=================================================================== Neural Magic DeepSparse 1.7 Model: ResNet-50, Sparse INT8 - Scenario: Synchronous Single-Stream ms/batch < Lower Is Better a . 1.3212 |================================================================== b . 1.3317 |=================================================================== c . 1.2875 |================================================================= Neural Magic DeepSparse 1.7 Model: Llama2 Chat 7b Quantized - Scenario: Asynchronous Multi-Stream items/sec > Higher Is Better a . 2.4922 |=================================================================== b . 2.4955 |=================================================================== c . 2.4912 |=================================================================== Neural Magic DeepSparse 1.7 Model: Llama2 Chat 7b Quantized - Scenario: Asynchronous Multi-Stream ms/batch < Lower Is Better a . 5886.95 |================================================================== b . 5875.85 |================================================================== c . 5888.22 |================================================================== Neural Magic DeepSparse 1.7 Model: Llama2 Chat 7b Quantized - Scenario: Synchronous Single-Stream items/sec > Higher Is Better a . 5.4552 |=================================================================== b . 5.4596 |=================================================================== c . 5.4574 |=================================================================== Neural Magic DeepSparse 1.7 Model: Llama2 Chat 7b Quantized - Scenario: Synchronous Single-Stream ms/batch < Lower Is Better a . 183.29 |=================================================================== b . 183.14 |=================================================================== c . 183.21 |=================================================================== Neural Magic DeepSparse 1.7 Model: CV Classification, ResNet-50 ImageNet - Scenario: Asynchronous Multi-Stream items/sec > Higher Is Better a . 321.29 |=================================================================== b . 322.04 |=================================================================== c . 323.61 |=================================================================== Neural Magic DeepSparse 1.7 Model: CV Classification, ResNet-50 ImageNet - Scenario: Asynchronous Multi-Stream ms/batch < Lower Is Better a . 49.76 |==================================================================== b . 49.66 |==================================================================== c . 49.42 |==================================================================== Neural Magic DeepSparse 1.7 Model: CV Classification, ResNet-50 ImageNet - Scenario: Synchronous Single-Stream items/sec > Higher Is Better a . 144.24 |================================================================= b . 149.20 |=================================================================== c . 146.23 |================================================================== Neural Magic DeepSparse 1.7 Model: CV Classification, ResNet-50 ImageNet - Scenario: Synchronous Single-Stream ms/batch < Lower Is Better a . 6.9254 |=================================================================== b . 6.6951 |================================================================= c . 6.8307 |================================================================== Neural Magic DeepSparse 1.7 Model: CV Detection, YOLOv5s COCO, Sparse INT8 - Scenario: Asynchronous Multi-Stream items/sec > Higher Is Better a . 158.70 |=================================================================== b . 158.50 |=================================================================== c . 158.04 |=================================================================== Neural Magic DeepSparse 1.7 Model: CV Detection, YOLOv5s COCO, Sparse INT8 - Scenario: Asynchronous Multi-Stream ms/batch < Lower Is Better a . 100.69 |=================================================================== b . 100.91 |=================================================================== c . 101.10 |=================================================================== Neural Magic DeepSparse 1.7 Model: CV Detection, YOLOv5s COCO, Sparse INT8 - Scenario: Synchronous Single-Stream items/sec > Higher Is Better a . 93.01 |=================================================================== b . 93.79 |==================================================================== c . 93.87 |==================================================================== Neural Magic DeepSparse 1.7 Model: CV Detection, YOLOv5s COCO, Sparse INT8 - Scenario: Synchronous Single-Stream ms/batch < Lower Is Better a . 10.74 |==================================================================== b . 10.65 |=================================================================== c . 10.65 |=================================================================== Neural Magic DeepSparse 1.7 Model: NLP Text Classification, DistilBERT mnli - Scenario: Asynchronous Multi-Stream items/sec > Higher Is Better a . 244.46 |=================================================================== b . 245.35 |=================================================================== c . 245.97 |=================================================================== Neural Magic DeepSparse 1.7 Model: NLP Text Classification, DistilBERT mnli - Scenario: Asynchronous Multi-Stream ms/batch < Lower Is Better a . 65.41 |==================================================================== b . 65.18 |==================================================================== c . 65.02 |==================================================================== Neural Magic DeepSparse 1.7 Model: NLP Text Classification, DistilBERT mnli - Scenario: Synchronous Single-Stream items/sec > Higher Is Better a . 81.15 |================================================================ b . 84.45 |=================================================================== c . 85.70 |==================================================================== Neural Magic DeepSparse 1.7 Model: NLP Text Classification, DistilBERT mnli - Scenario: Synchronous Single-Stream ms/batch < Lower Is Better a . 12.32 |==================================================================== b . 11.83 |================================================================= c . 11.66 |================================================================ Neural Magic DeepSparse 1.7 Model: CV Segmentation, 90% Pruned YOLACT Pruned - Scenario: Asynchronous Multi-Stream items/sec > Higher Is Better a . 33.16 |=================================================================== b . 33.45 |==================================================================== c . 33.33 |==================================================================== Neural Magic DeepSparse 1.7 Model: CV Segmentation, 90% Pruned YOLACT Pruned - Scenario: Asynchronous Multi-Stream ms/batch < Lower Is Better a . 481.31 |=================================================================== b . 477.03 |================================================================== c . 479.29 |=================================================================== Neural Magic DeepSparse 1.7 Model: CV Segmentation, 90% Pruned YOLACT Pruned - Scenario: Synchronous Single-Stream items/sec > Higher Is Better a . 21.13 |=================================================================== b . 21.43 |==================================================================== c . 21.42 |==================================================================== Neural Magic DeepSparse 1.7 Model: CV Segmentation, 90% Pruned YOLACT Pruned - Scenario: Synchronous Single-Stream ms/batch < Lower Is Better a . 47.32 |==================================================================== b . 46.64 |=================================================================== c . 46.67 |=================================================================== Neural Magic DeepSparse 1.7 Model: BERT-Large, NLP Question Answering, Sparse INT8 - Scenario: Asynchronous Multi-Stream items/sec > Higher Is Better a . 344.29 |=================================================================== b . 344.92 |=================================================================== c . 344.66 |=================================================================== Neural Magic DeepSparse 1.7 Model: BERT-Large, NLP Question Answering, Sparse INT8 - Scenario: Asynchronous Multi-Stream ms/batch < Lower Is Better a . 46.42 |==================================================================== b . 46.35 |==================================================================== c . 46.36 |==================================================================== Neural Magic DeepSparse 1.7 Model: BERT-Large, NLP Question Answering, Sparse INT8 - Scenario: Synchronous Single-Stream items/sec > Higher Is Better a . 86.39 |==================================================================== b . 86.50 |==================================================================== c . 86.00 |==================================================================== Neural Magic DeepSparse 1.7 Model: BERT-Large, NLP Question Answering, Sparse INT8 - Scenario: Synchronous Single-Stream ms/batch < Lower Is Better a . 11.57 |==================================================================== b . 11.55 |==================================================================== c . 11.62 |==================================================================== Neural Magic DeepSparse 1.7 Model: NLP Token Classification, BERT base uncased conll2003 - Scenario: Asynchronous Multi-Stream items/sec > Higher Is Better a . 30.07 |==================================================================== b . 30.22 |==================================================================== c . 30.07 |==================================================================== Neural Magic DeepSparse 1.7 Model: NLP Token Classification, BERT base uncased conll2003 - Scenario: Asynchronous Multi-Stream ms/batch < Lower Is Better a . 529.09 |=================================================================== b . 525.92 |================================================================== c . 530.65 |=================================================================== Neural Magic DeepSparse 1.7 Model: NLP Token Classification, BERT base uncased conll2003 - Scenario: Synchronous Single-Stream items/sec > Higher Is Better a . 17.63 |==================================================================== b . 17.63 |==================================================================== c . 17.54 |==================================================================== Neural Magic DeepSparse 1.7 Model: NLP Token Classification, BERT base uncased conll2003 - Scenario: Synchronous Single-Stream ms/batch < Lower Is Better a . 56.72 |==================================================================== b . 56.72 |==================================================================== c . 57.00 |==================================================================== Google Draco 1.5.6 Model: Lion ms < Lower Is Better a . 5258 |===================================================================== b . 5275 |===================================================================== c . 5199 |==================================================================== Google Draco 1.5.6 Model: Church Facade ms < Lower Is Better a . 7989 |=================================================================== b . 8191 |===================================================================== c . 8165 |=====================================================================