dggg Intel Core i9-10980XE testing with a ASRock X299 Steel Legend (P1.50 BIOS) and llvmpipe on Ubuntu 22.04 via the Phoronix Test Suite. a: Processor: Intel Core i9-10980XE @ 4.80GHz (18 Cores / 36 Threads), Motherboard: ASRock X299 Steel Legend (P1.50 BIOS), Chipset: Intel Sky Lake-E DMI3 Registers, Memory: 4 x 8GB DDR4-3600MT/s, Disk: Samsung SSD 970 PRO 512GB, Graphics: llvmpipe, Audio: Realtek ALC1220, Network: Intel I219-V + Intel I211 OS: Ubuntu 22.04, Kernel: 6.5.0-18-generic (x86_64), Desktop: GNOME Shell 42.2, Display Server: X Server 1.21.1.4, OpenGL: 4.5 Mesa 22.0.1 (LLVM 13.0.1 256 bits), Vulkan: 1.2.204, Compiler: GCC 11.4.0, File-System: ext4, Screen Resolution: 1024x768 b: Processor: Intel Core i9-10980XE @ 4.80GHz (18 Cores / 36 Threads), Motherboard: ASRock X299 Steel Legend (P1.50 BIOS), Chipset: Intel Sky Lake-E DMI3 Registers, Memory: 4 x 8GB DDR4-3600MT/s, Disk: Samsung SSD 970 PRO 512GB, Graphics: llvmpipe, Audio: Realtek ALC1220, Network: Intel I219-V + Intel I211 OS: Ubuntu 22.04, Kernel: 6.5.0-18-generic (x86_64), Desktop: GNOME Shell 42.2, Display Server: X Server 1.21.1.4, OpenGL: 4.5 Mesa 22.0.1 (LLVM 13.0.1 256 bits), Vulkan: 1.2.204, Compiler: GCC 11.4.0, File-System: ext4, Screen Resolution: 1024x768 SVT-AV1 2.0 Encoder Mode: Preset 4 - Input: Bosphorus 4K Frames Per Second > Higher Is Better a . 3.220 |==================================================================== b . 3.205 |==================================================================== SVT-AV1 2.0 Encoder Mode: Preset 8 - Input: Bosphorus 4K Frames Per Second > Higher Is Better a . 31.61 |==================================================================== b . 31.69 |==================================================================== SVT-AV1 2.0 Encoder Mode: Preset 12 - Input: Bosphorus 4K Frames Per Second > Higher Is Better a . 82.01 |=================================================================== b . 82.88 |==================================================================== SVT-AV1 2.0 Encoder Mode: Preset 13 - Input: Bosphorus 4K Frames Per Second > Higher Is Better a . 82.84 |==================================================================== b . 82.49 |==================================================================== SVT-AV1 2.0 Encoder Mode: Preset 4 - Input: Bosphorus 1080p Frames Per Second > Higher Is Better a . 8.553 |==================================================================== b . 8.471 |=================================================================== SVT-AV1 2.0 Encoder Mode: Preset 8 - Input: Bosphorus 1080p Frames Per Second > Higher Is Better a . 51.09 |=================================================================== b . 51.87 |==================================================================== SVT-AV1 2.0 Encoder Mode: Preset 12 - Input: Bosphorus 1080p Frames Per Second > Higher Is Better a . 170.02 |=================================================================== b . 168.48 |================================================================== SVT-AV1 2.0 Encoder Mode: Preset 13 - Input: Bosphorus 1080p Frames Per Second > Higher Is Better a . 200.02 |=================================================================== b . 195.77 |================================================================== Neural Magic DeepSparse 1.7 Model: NLP Document Classification, oBERT base uncased on IMDB - Scenario: Asynchronous Multi-Stream items/sec > Higher Is Better a . 18.31 |==================================================================== b . 18.07 |=================================================================== Neural Magic DeepSparse 1.7 Model: NLP Document Classification, oBERT base uncased on IMDB - Scenario: Synchronous Single-Stream items/sec > Higher Is Better a . 15.88 |==================================================================== b . 15.95 |==================================================================== Neural Magic DeepSparse 1.7 Model: NLP Text Classification, BERT base uncased SST2, Sparse INT8 - Scenario: Asynchronous Multi-Stream items/sec > Higher Is Better a . 455.36 |=================================================================== b . 455.46 |=================================================================== Neural Magic DeepSparse 1.7 Model: NLP Text Classification, BERT base uncased SST2, Sparse INT8 - Scenario: Synchronous Single-Stream items/sec > Higher Is Better a . 143.99 |=================================================================== b . 140.50 |================================================================= Neural Magic DeepSparse 1.7 Model: ResNet-50, Baseline - Scenario: Asynchronous Multi-Stream items/sec > Higher Is Better a . 258.60 |=================================================================== b . 257.51 |=================================================================== Neural Magic DeepSparse 1.7 Model: ResNet-50, Baseline - Scenario: Synchronous Single-Stream items/sec > Higher Is Better a . 131.22 |================================================================== b . 133.69 |=================================================================== Neural Magic DeepSparse 1.7 Model: ResNet-50, Sparse INT8 - Scenario: Asynchronous Multi-Stream items/sec > Higher Is Better a . 1226.61 |================================================================== b . 1223.65 |================================================================== Neural Magic DeepSparse 1.7 Model: ResNet-50, Sparse INT8 - Scenario: Synchronous Single-Stream items/sec > Higher Is Better a . 561.72 |================================================================== b . 570.09 |=================================================================== Neural Magic DeepSparse 1.7 Model: Llama2 Chat 7b Quantized - Scenario: Asynchronous Multi-Stream items/sec > Higher Is Better a . 2.6650 |=================================================================== b . 2.6717 |=================================================================== Neural Magic DeepSparse 1.7 Model: Llama2 Chat 7b Quantized - Scenario: Synchronous Single-Stream items/sec > Higher Is Better a . 8.1016 |=================================================================== b . 8.1515 |=================================================================== Neural Magic DeepSparse 1.7 Model: CV Classification, ResNet-50 ImageNet - Scenario: Asynchronous Multi-Stream items/sec > Higher Is Better a . 253.66 |=================================================================== b . 252.50 |=================================================================== Neural Magic DeepSparse 1.7 Model: CV Classification, ResNet-50 ImageNet - Scenario: Synchronous Single-Stream items/sec > Higher Is Better a . 132.94 |=================================================================== b . 128.23 |================================================================= Neural Magic DeepSparse 1.7 Model: CV Detection, YOLOv5s COCO, Sparse INT8 - Scenario: Asynchronous Multi-Stream items/sec > Higher Is Better a . 106.84 |=================================================================== b . 106.73 |=================================================================== Neural Magic DeepSparse 1.7 Model: CV Detection, YOLOv5s COCO, Sparse INT8 - Scenario: Synchronous Single-Stream items/sec > Higher Is Better a . 83.55 |==================================================================== b . 83.23 |==================================================================== Neural Magic DeepSparse 1.7 Model: NLP Text Classification, DistilBERT mnli - Scenario: Asynchronous Multi-Stream items/sec > Higher Is Better a . 149.20 |=================================================================== b . 149.40 |=================================================================== Neural Magic DeepSparse 1.7 Model: NLP Text Classification, DistilBERT mnli - Scenario: Synchronous Single-Stream items/sec > Higher Is Better a . 85.64 |=================================================================== b . 86.34 |==================================================================== Neural Magic DeepSparse 1.7 Model: CV Segmentation, 90% Pruned YOLACT Pruned - Scenario: Asynchronous Multi-Stream items/sec > Higher Is Better a . 21.91 |================================================================== b . 22.70 |==================================================================== Neural Magic DeepSparse 1.7 Model: CV Segmentation, 90% Pruned YOLACT Pruned - Scenario: Synchronous Single-Stream items/sec > Higher Is Better a . 21.60 |==================================================================== b . 21.11 |================================================================== Neural Magic DeepSparse 1.7 Model: BERT-Large, NLP Question Answering, Sparse INT8 - Scenario: Asynchronous Multi-Stream items/sec > Higher Is Better a . 208.28 |================================================================== b . 209.86 |=================================================================== Neural Magic DeepSparse 1.7 Model: BERT-Large, NLP Question Answering, Sparse INT8 - Scenario: Synchronous Single-Stream items/sec > Higher Is Better a . 55.20 |==================================================================== b . 54.77 |=================================================================== Neural Magic DeepSparse 1.7 Model: NLP Token Classification, BERT base uncased conll2003 - Scenario: Asynchronous Multi-Stream items/sec > Higher Is Better a . 18.33 |==================================================================== b . 17.88 |================================================================== Neural Magic DeepSparse 1.7 Model: NLP Token Classification, BERT base uncased conll2003 - Scenario: Synchronous Single-Stream items/sec > Higher Is Better a . 15.83 |==================================================================== b . 15.93 |==================================================================== srsRAN Project 23.10.1-20240219 Test: PDSCH Processor Benchmark, Throughput Total Mbps > Higher Is Better a . 5546.0 |=================================================================== b . 5504.1 |================================================================== srsRAN Project 23.10.1-20240219 Test: PUSCH Processor Benchmark, Throughput Total Mbps > Higher Is Better a . 1105.4 |=================================================================== srsRAN Project 23.10.1-20240219 Test: PDSCH Processor Benchmark, Throughput Thread Mbps > Higher Is Better a . 400.2 |==================================================================== srsRAN Project 23.10.1-20240219 Test: PUSCH Processor Benchmark, Throughput Thread Mbps > Higher Is Better a . 134.5 |==================================================================== Google Draco 1.5.6 Model: Lion ms < Lower Is Better a . 6059 |===================================================================== b . 6065 |===================================================================== Google Draco 1.5.6 Model: Church Facade ms < Lower Is Better a . 8567 |===================================================================== b . 8594 |===================================================================== Neural Magic DeepSparse 1.7 Model: NLP Document Classification, oBERT base uncased on IMDB - Scenario: Asynchronous Multi-Stream ms/batch < Lower Is Better a . 489.37 |================================================================== b . 494.04 |=================================================================== Neural Magic DeepSparse 1.7 Model: NLP Document Classification, oBERT base uncased on IMDB - Scenario: Synchronous Single-Stream ms/batch < Lower Is Better a . 62.96 |==================================================================== b . 62.68 |==================================================================== Neural Magic DeepSparse 1.7 Model: NLP Text Classification, BERT base uncased SST2, Sparse INT8 - Scenario: Asynchronous Multi-Stream ms/batch < Lower Is Better a . 19.74 |==================================================================== b . 19.74 |==================================================================== Neural Magic DeepSparse 1.7 Model: NLP Text Classification, BERT base uncased SST2, Sparse INT8 - Scenario: Synchronous Single-Stream ms/batch < Lower Is Better a . 6.9366 |================================================================= b . 7.1081 |=================================================================== Neural Magic DeepSparse 1.7 Model: ResNet-50, Baseline - Scenario: Asynchronous Multi-Stream ms/batch < Lower Is Better a . 34.78 |==================================================================== b . 34.92 |==================================================================== Neural Magic DeepSparse 1.7 Model: ResNet-50, Baseline - Scenario: Synchronous Single-Stream ms/batch < Lower Is Better a . 7.6112 |=================================================================== b . 7.4680 |================================================================== Neural Magic DeepSparse 1.7 Model: ResNet-50, Sparse INT8 - Scenario: Asynchronous Multi-Stream ms/batch < Lower Is Better a . 7.3180 |=================================================================== b . 7.3353 |=================================================================== Neural Magic DeepSparse 1.7 Model: ResNet-50, Sparse INT8 - Scenario: Synchronous Single-Stream ms/batch < Lower Is Better a . 1.7714 |=================================================================== b . 1.7454 |================================================================== Neural Magic DeepSparse 1.7 Model: Llama2 Chat 7b Quantized - Scenario: Asynchronous Multi-Stream ms/batch < Lower Is Better a . 3224.12 |================================================================== b . 3217.01 |================================================================== Neural Magic DeepSparse 1.7 Model: Llama2 Chat 7b Quantized - Scenario: Synchronous Single-Stream ms/batch < Lower Is Better a . 123.41 |=================================================================== b . 122.65 |=================================================================== Neural Magic DeepSparse 1.7 Model: CV Classification, ResNet-50 ImageNet - Scenario: Asynchronous Multi-Stream ms/batch < Lower Is Better a . 35.46 |==================================================================== b . 35.62 |==================================================================== Neural Magic DeepSparse 1.7 Model: CV Classification, ResNet-50 ImageNet - Scenario: Synchronous Single-Stream ms/batch < Lower Is Better a . 7.5105 |================================================================= b . 7.7867 |=================================================================== Neural Magic DeepSparse 1.7 Model: CV Detection, YOLOv5s COCO, Sparse INT8 - Scenario: Asynchronous Multi-Stream ms/batch < Lower Is Better a . 84.21 |==================================================================== b . 84.30 |==================================================================== Neural Magic DeepSparse 1.7 Model: CV Detection, YOLOv5s COCO, Sparse INT8 - Scenario: Synchronous Single-Stream ms/batch < Lower Is Better a . 11.96 |==================================================================== b . 12.00 |==================================================================== Neural Magic DeepSparse 1.7 Model: NLP Text Classification, DistilBERT mnli - Scenario: Asynchronous Multi-Stream ms/batch < Lower Is Better a . 60.30 |==================================================================== b . 60.22 |==================================================================== Neural Magic DeepSparse 1.7 Model: NLP Text Classification, DistilBERT mnli - Scenario: Synchronous Single-Stream ms/batch < Lower Is Better a . 11.69 |==================================================================== b . 11.57 |=================================================================== Neural Magic DeepSparse 1.7 Model: CV Segmentation, 90% Pruned YOLACT Pruned - Scenario: Asynchronous Multi-Stream ms/batch < Lower Is Better a . 410.68 |=================================================================== b . 396.31 |================================================================= Neural Magic DeepSparse 1.7 Model: CV Segmentation, 90% Pruned YOLACT Pruned - Scenario: Synchronous Single-Stream ms/batch < Lower Is Better a . 46.28 |================================================================== b . 47.36 |==================================================================== Neural Magic DeepSparse 1.7 Model: BERT-Large, NLP Question Answering, Sparse INT8 - Scenario: Asynchronous Multi-Stream ms/batch < Lower Is Better a . 43.18 |==================================================================== b . 42.86 |=================================================================== Neural Magic DeepSparse 1.7 Model: BERT-Large, NLP Question Answering, Sparse INT8 - Scenario: Synchronous Single-Stream ms/batch < Lower Is Better a . 18.14 |==================================================================== b . 18.24 |==================================================================== Neural Magic DeepSparse 1.7 Model: NLP Token Classification, BERT base uncased conll2003 - Scenario: Asynchronous Multi-Stream ms/batch < Lower Is Better a . 489.52 |================================================================== b . 499.66 |=================================================================== Neural Magic DeepSparse 1.7 Model: NLP Token Classification, BERT base uncased conll2003 - Scenario: Synchronous Single-Stream ms/batch < Lower Is Better a . 63.17 |==================================================================== b . 62.77 |==================================================================== Timed Linux Kernel Compilation 6.8 Build: defconfig Seconds < Lower Is Better a . 74.36 |=================================================================== b . 75.74 |==================================================================== Timed Linux Kernel Compilation 6.8 Build: allmodconfig Seconds < Lower Is Better a . 943.62 |=================================================================== b . 944.98 |=================================================================== Parallel BZIP2 Compression 1.1.13 FreeBSD-13.0-RELEASE-amd64-memstick.img Compression Seconds < Lower Is Better a . 6.076621 |================================================================= b . 6.114071 |================================================================= Primesieve 12.1 Length: 1e12 Seconds < Lower Is Better a . 11.10 |==================================================================== b . 11.09 |==================================================================== Primesieve 12.1 Length: 1e13 Seconds < Lower Is Better a . 128.88 |=================================================================== b . 128.88 |===================================================================