new AMD Ryzen 7 7840HS testing with a NB05 TUXEDO Pulse 14 Gen3 R14FA1 (8.06 BIOS) and AMD Phoenix1 4GB on Tuxedo 22.04 via the Phoronix Test Suite. a: Processor: AMD Ryzen 7 7840HS @ 5.29GHz (8 Cores / 16 Threads), Motherboard: NB05 TUXEDO Pulse 14 Gen3 R14FA1 (8.06 BIOS), Chipset: AMD Device 14e8, Memory: 4 x 8GB DRAM-6400MT/s Micron MT62F2G32D4DS-026 WT, Disk: 2000GB Samsung SSD 980 PRO 2TB + 31GB SanDisk 3.2Gen1, Graphics: AMD Phoenix1 4GB (2700/800MHz), Audio: AMD Rembrandt Radeon HD Audio, Network: MEDIATEK MT7921K OS: Tuxedo 22.04, Kernel: 6.5.0-10022-tuxedo (x86_64), Desktop: KDE Plasma 5.27.10, Display Server: X Server 1.21.1.4, OpenGL: 4.6 Mesa 24.0.3-0tux2 (LLVM 15.0.7 DRM 3.54), Vulkan: 1.3.274, Compiler: GCC 11.4.0, File-System: ext4, Screen Resolution: 2880x1800 b: Processor: AMD Ryzen 7 7840HS @ 5.29GHz (8 Cores / 16 Threads), Motherboard: NB05 TUXEDO Pulse 14 Gen3 R14FA1 (8.06 BIOS), Chipset: AMD Device 14e8, Memory: 4 x 8GB DRAM-6400MT/s Micron MT62F2G32D4DS-026 WT, Disk: 2000GB Samsung SSD 980 PRO 2TB + 31GB SanDisk 3.2Gen1, Graphics: AMD Phoenix1 4GB (2700/800MHz), Audio: AMD Rembrandt Radeon HD Audio, Network: MEDIATEK MT7921K OS: Tuxedo 22.04, Kernel: 6.5.0-10022-tuxedo (x86_64), Desktop: KDE Plasma 5.27.10, Display Server: X Server 1.21.1.4, OpenGL: 4.6 Mesa 24.0.3-0tux2 (LLVM 15.0.7 DRM 3.54), Vulkan: 1.3.274, Compiler: GCC 11.4.0, File-System: ext4, Screen Resolution: 2880x1800 c: Processor: AMD Ryzen 7 7840HS @ 5.29GHz (8 Cores / 16 Threads), Motherboard: NB05 TUXEDO Pulse 14 Gen3 R14FA1 (8.06 BIOS), Chipset: AMD Device 14e8, Memory: 4 x 8GB DRAM-6400MT/s Micron MT62F2G32D4DS-026 WT, Disk: 2000GB Samsung SSD 980 PRO 2TB + 31GB SanDisk 3.2Gen1, Graphics: AMD Phoenix1 4GB (2700/800MHz), Audio: AMD Rembrandt Radeon HD Audio, Network: MEDIATEK MT7921K OS: Tuxedo 22.04, Kernel: 6.5.0-10022-tuxedo (x86_64), Desktop: KDE Plasma 5.27.10, Display Server: X Server 1.21.1.4, OpenGL: 4.6 Mesa 24.0.3-0tux2 (LLVM 15.0.7 DRM 3.54), Vulkan: 1.3.274, Compiler: GCC 11.4.0, File-System: ext4, Screen Resolution: 2880x1800 OpenVINO 2024.0 Model: Face Detection FP16 - Device: CPU FPS > Higher Is Better a . 5.94 |===================================================================== b . 5.92 |===================================================================== c . 5.96 |===================================================================== OpenVINO 2024.0 Model: Person Detection FP16 - Device: CPU FPS > Higher Is Better a . 43.60 |==================================================================== b . 43.53 |==================================================================== c . 43.51 |==================================================================== OpenVINO 2024.0 Model: Person Detection FP32 - Device: CPU FPS > Higher Is Better a . 43.37 |==================================================================== b . 43.48 |==================================================================== c . 43.36 |==================================================================== OpenVINO 2024.0 Model: Vehicle Detection FP16 - Device: CPU FPS > Higher Is Better a . 297.96 |=================================================================== b . 296.95 |=================================================================== c . 296.58 |=================================================================== OpenVINO 2024.0 Model: Face Detection FP16-INT8 - Device: CPU FPS > Higher Is Better a . 11.73 |==================================================================== b . 11.76 |==================================================================== c . 11.71 |==================================================================== OpenVINO 2024.0 Model: Face Detection Retail FP16 - Device: CPU FPS > Higher Is Better a . 1368.22 |================================================================== b . 1362.71 |================================================================= c . 1373.25 |================================================================== OpenVINO 2024.0 Model: Road Segmentation ADAS FP16 - Device: CPU FPS > Higher Is Better a . 136.82 |=================================================================== b . 136.66 |=================================================================== c . 135.08 |================================================================== OpenVINO 2024.0 Model: Vehicle Detection FP16-INT8 - Device: CPU FPS > Higher Is Better a . 646.05 |=================================================================== b . 645.11 |=================================================================== c . 645.86 |=================================================================== OpenVINO 2024.0 Model: Weld Porosity Detection FP16 - Device: CPU FPS > Higher Is Better a . 589.54 |=================================================================== b . 590.35 |=================================================================== c . 590.91 |=================================================================== OpenVINO 2024.0 Model: Face Detection Retail FP16-INT8 - Device: CPU FPS > Higher Is Better a . 2038.66 |================================================================== b . 1988.45 |================================================================ c . 1990.07 |================================================================ OpenVINO 2024.0 Model: Road Segmentation ADAS FP16-INT8 - Device: CPU FPS > Higher Is Better a . 206.45 |================================================================== b . 206.40 |================================================================== c . 208.50 |=================================================================== OpenVINO 2024.0 Model: Machine Translation EN To DE FP16 - Device: CPU FPS > Higher Is Better a . 67.02 |==================================================================== b . 66.79 |==================================================================== c . 64.95 |================================================================== OpenVINO 2024.0 Model: Weld Porosity Detection FP16-INT8 - Device: CPU FPS > Higher Is Better a . 1163.76 |================================================================== b . 1163.20 |================================================================== c . 1162.84 |================================================================== OpenVINO 2024.0 Model: Person Vehicle Bike Detection FP16 - Device: CPU FPS > Higher Is Better a . 566.41 |================================================================= b . 562.38 |================================================================ c . 584.85 |=================================================================== OpenVINO 2024.0 Model: Noise Suppression Poconet-Like FP16 - Device: CPU FPS > Higher Is Better a . 747.57 |================================================================== b . 758.67 |=================================================================== c . 738.75 |================================================================= OpenVINO 2024.0 Model: Handwritten English Recognition FP16 - Device: CPU FPS > Higher Is Better a . 304.85 |================================================================== b . 308.63 |=================================================================== c . 306.00 |================================================================== OpenVINO 2024.0 Model: Person Re-Identification Retail FP16 - Device: CPU FPS > Higher Is Better a . 769.47 |=================================================================== b . 770.40 |=================================================================== c . 767.98 |=================================================================== OpenVINO 2024.0 Model: Age Gender Recognition Retail 0013 FP16 - Device: CPU FPS > Higher Is Better a . 17845.54 |================================================================= b . 17817.76 |================================================================= c . 17864.73 |================================================================= OpenVINO 2024.0 Model: Handwritten English Recognition FP16-INT8 - Device: CPU FPS > Higher Is Better a . 328.64 |=================================================================== b . 326.11 |================================================================== c . 327.20 |=================================================================== OpenVINO 2024.0 Model: Age Gender Recognition Retail 0013 FP16-INT8 - Device: CPU FPS > Higher Is Better a . 25643.60 |================================================================= b . 25543.04 |================================================================= c . 25328.05 |================================================================ SVT-AV1 2.0 Encoder Mode: Preset 4 - Input: Bosphorus 4K Frames Per Second > Higher Is Better a . 4.421 |================================================================== b . 4.530 |==================================================================== c . 4.556 |==================================================================== SVT-AV1 2.0 Encoder Mode: Preset 8 - Input: Bosphorus 4K Frames Per Second > Higher Is Better a . 36.21 |==================================================================== b . 36.15 |==================================================================== c . 36.33 |==================================================================== SVT-AV1 2.0 Encoder Mode: Preset 12 - Input: Bosphorus 4K Frames Per Second > Higher Is Better a . 96.41 |==================================================================== b . 91.27 |================================================================ c . 96.68 |==================================================================== SVT-AV1 2.0 Encoder Mode: Preset 13 - Input: Bosphorus 4K Frames Per Second > Higher Is Better a . 92.05 |================================================================= b . 91.14 |================================================================ c . 97.04 |==================================================================== SVT-AV1 2.0 Encoder Mode: Preset 4 - Input: Bosphorus 1080p Frames Per Second > Higher Is Better a . 15.39 |================================================================= b . 15.94 |=================================================================== c . 16.11 |==================================================================== SVT-AV1 2.0 Encoder Mode: Preset 8 - Input: Bosphorus 1080p Frames Per Second > Higher Is Better a . 110.51 |================================================================= b . 112.67 |=================================================================== c . 113.28 |=================================================================== SVT-AV1 2.0 Encoder Mode: Preset 12 - Input: Bosphorus 1080p Frames Per Second > Higher Is Better a . 410.02 |================================================================ b . 425.30 |=================================================================== c . 426.94 |=================================================================== SVT-AV1 2.0 Encoder Mode: Preset 13 - Input: Bosphorus 1080p Frames Per Second > Higher Is Better a . 487.09 |================================================================ b . 509.40 |=================================================================== c . 508.57 |=================================================================== Neural Magic DeepSparse 1.7 Model: NLP Document Classification, oBERT base uncased on IMDB - Scenario: Asynchronous Multi-Stream items/sec > Higher Is Better a . 8.4524 |=================================================================== b . 8.4593 |=================================================================== c . 8.4341 |=================================================================== Neural Magic DeepSparse 1.7 Model: NLP Document Classification, oBERT base uncased on IMDB - Scenario: Synchronous Single-Stream items/sec > Higher Is Better a . 8.5716 |=================================================================== b . 8.5678 |=================================================================== c . 8.5595 |=================================================================== Neural Magic DeepSparse 1.7 Model: NLP Text Classification, BERT base uncased SST2, Sparse INT8 - Scenario: Asynchronous Multi-Stream items/sec > Higher Is Better a . 328.71 |=================================================================== b . 326.33 |=================================================================== c . 327.41 |=================================================================== Neural Magic DeepSparse 1.7 Model: NLP Text Classification, BERT base uncased SST2, Sparse INT8 - Scenario: Synchronous Single-Stream items/sec > Higher Is Better a . 215.41 |=================================================================== b . 214.44 |=================================================================== c . 214.54 |=================================================================== Neural Magic DeepSparse 1.7 Model: ResNet-50, Baseline - Scenario: Asynchronous Multi-Stream items/sec > Higher Is Better a . 105.27 |=================================================================== b . 104.96 |=================================================================== c . 104.96 |=================================================================== Neural Magic DeepSparse 1.7 Model: ResNet-50, Baseline - Scenario: Synchronous Single-Stream items/sec > Higher Is Better a . 76.89 |==================================================================== b . 76.83 |==================================================================== c . 77.17 |==================================================================== Neural Magic DeepSparse 1.7 Model: ResNet-50, Sparse INT8 - Scenario: Asynchronous Multi-Stream items/sec > Higher Is Better a . 772.99 |================================================================== b . 783.88 |=================================================================== c . 759.94 |================================================================= Neural Magic DeepSparse 1.7 Model: ResNet-50, Sparse INT8 - Scenario: Synchronous Single-Stream items/sec > Higher Is Better a . 438.80 |=================================================================== b . 438.91 |=================================================================== c . 437.68 |=================================================================== Neural Magic DeepSparse 1.7 Model: Llama2 Chat 7b Quantized - Scenario: Asynchronous Multi-Stream items/sec > Higher Is Better a . 3.0846 |=================================================================== b . 2.9841 |================================================================= c . 3.0247 |================================================================== Neural Magic DeepSparse 1.7 Model: Llama2 Chat 7b Quantized - Scenario: Synchronous Single-Stream items/sec > Higher Is Better a . 5.4459 |=================================================================== b . 5.4420 |=================================================================== c . 5.4420 |=================================================================== Neural Magic DeepSparse 1.7 Model: CV Classification, ResNet-50 ImageNet - Scenario: Asynchronous Multi-Stream items/sec > Higher Is Better a . 104.62 |=================================================================== b . 104.77 |=================================================================== c . 104.74 |=================================================================== Neural Magic DeepSparse 1.7 Model: CV Classification, ResNet-50 ImageNet - Scenario: Synchronous Single-Stream items/sec > Higher Is Better a . 76.79 |==================================================================== b . 76.79 |==================================================================== c . 76.95 |==================================================================== Neural Magic DeepSparse 1.7 Model: CV Detection, YOLOv5s COCO, Sparse INT8 - Scenario: Asynchronous Multi-Stream items/sec > Higher Is Better a . 50.57 |=================================================================== b . 51.04 |==================================================================== c . 51.21 |==================================================================== Neural Magic DeepSparse 1.7 Model: CV Detection, YOLOv5s COCO, Sparse INT8 - Scenario: Synchronous Single-Stream items/sec > Higher Is Better a . 46.20 |=================================================================== b . 46.67 |==================================================================== c . 46.54 |==================================================================== Neural Magic DeepSparse 1.7 Model: NLP Text Classification, DistilBERT mnli - Scenario: Asynchronous Multi-Stream items/sec > Higher Is Better a . 74.10 |==================================================================== b . 74.23 |==================================================================== c . 73.94 |==================================================================== Neural Magic DeepSparse 1.7 Model: NLP Text Classification, DistilBERT mnli - Scenario: Synchronous Single-Stream items/sec > Higher Is Better a . 67.27 |==================================================================== b . 67.13 |==================================================================== c . 67.00 |==================================================================== Neural Magic DeepSparse 1.7 Model: CV Segmentation, 90% Pruned YOLACT Pruned - Scenario: Asynchronous Multi-Stream items/sec > Higher Is Better a . 14.18 |=================================================================== b . 14.24 |=================================================================== c . 14.43 |==================================================================== Neural Magic DeepSparse 1.7 Model: CV Segmentation, 90% Pruned YOLACT Pruned - Scenario: Synchronous Single-Stream items/sec > Higher Is Better a . 15.01 |==================================================================== b . 14.96 |==================================================================== c . 15.03 |==================================================================== Neural Magic DeepSparse 1.7 Model: BERT-Large, NLP Question Answering, Sparse INT8 - Scenario: Asynchronous Multi-Stream items/sec > Higher Is Better a . 148.53 |=================================================================== b . 148.22 |=================================================================== c . 148.15 |=================================================================== Neural Magic DeepSparse 1.7 Model: BERT-Large, NLP Question Answering, Sparse INT8 - Scenario: Synchronous Single-Stream items/sec > Higher Is Better a . 102.48 |=================================================================== b . 102.45 |=================================================================== c . 102.67 |=================================================================== Neural Magic DeepSparse 1.7 Model: NLP Token Classification, BERT base uncased conll2003 - Scenario: Asynchronous Multi-Stream items/sec > Higher Is Better a . 8.4654 |=================================================================== b . 8.4789 |=================================================================== c . 8.4730 |=================================================================== Neural Magic DeepSparse 1.7 Model: NLP Token Classification, BERT base uncased conll2003 - Scenario: Synchronous Single-Stream items/sec > Higher Is Better a . 8.5704 |=================================================================== b . 8.5637 |=================================================================== c . 8.5850 |=================================================================== JPEG-XL libjxl 0.10.1 Input: PNG - Quality: 80 MP/s > Higher Is Better a . 30.98 |================================================================= b . 32.63 |==================================================================== c . 31.85 |================================================================== JPEG-XL libjxl 0.10.1 Input: PNG - Quality: 90 MP/s > Higher Is Better a . 28.60 |================================================================ b . 30.24 |==================================================================== c . 29.20 |================================================================== JPEG-XL libjxl 0.10.1 Input: JPEG - Quality: 80 MP/s > Higher Is Better a . 32.75 |==================================================================== b . 31.88 |================================================================== c . 32.50 |=================================================================== JPEG-XL libjxl 0.10.1 Input: JPEG - Quality: 90 MP/s > Higher Is Better a . 30.30 |=================================================================== b . 30.41 |=================================================================== c . 30.77 |==================================================================== JPEG-XL libjxl 0.10.1 Input: PNG - Quality: 100 MP/s > Higher Is Better a . 13.44 |=================================================================== b . 13.55 |==================================================================== c . 13.56 |==================================================================== JPEG-XL libjxl 0.10.1 Input: JPEG - Quality: 100 MP/s > Higher Is Better a . 13.36 |=================================================================== b . 13.50 |==================================================================== c . 13.46 |==================================================================== JPEG-XL Decoding libjxl 0.10.1 CPU Threads: 1 MP/s > Higher Is Better a . 72.69 |============================================================== b . 78.40 |=================================================================== c . 79.51 |==================================================================== JPEG-XL Decoding libjxl 0.10.1 CPU Threads: All MP/s > Higher Is Better a . 347.07 |=============================================================== b . 368.57 |=================================================================== c . 369.33 |=================================================================== RocksDB 9.0 Test: Overwrite Op/s > Higher Is Better a . 1112463 |================================================================== b . 1099994 |================================================================= c . 1092989 |================================================================= RocksDB 9.0 Test: Random Fill Op/s > Higher Is Better a . 1099674 |================================================================== b . 1098801 |================================================================== c . 1093000 |================================================================== RocksDB 9.0 Test: Random Read Op/s > Higher Is Better a . 60396550 |================================================================= b . 60416662 |================================================================= c . 60447022 |================================================================= RocksDB 9.0 Test: Update Random Op/s > Higher Is Better a . 667251 |=================================================================== b . 664442 |=================================================================== c . 662151 |================================================================== RocksDB 9.0 Test: Sequential Fill Op/s > Higher Is Better a . 1671410 |================================================================= b . 1592104 |============================================================== c . 1685641 |================================================================== RocksDB 9.0 Test: Random Fill Sync Op/s > Higher Is Better a . 1629193 |================================================================== b . 1600211 |================================================================= c . 1634751 |================================================================== RocksDB 9.0 Test: Read While Writing Op/s > Higher Is Better a . 2097339 |================================================================= b . 2145876 |================================================================== c . 2116553 |================================================================= RocksDB 9.0 Test: Read Random Write Random Op/s > Higher Is Better a . 1927575 |================================================================== b . 1929085 |================================================================== c . 1921626 |================================================================== BRL-CAD 7.38.2 VGR Performance Metric VGR Performance Metric > Higher Is Better a . 197629 |=================================================================== b . 197598 |=================================================================== c . 181031 |============================================================= Chaos Group V-RAY 6.0 Mode: CPU vsamples > Higher Is Better a . 17872 |==================================================================== b . 17691 |=================================================================== Google Draco 1.5.6 Model: Lion ms < Lower Is Better a . 5013 |===================================================================== b . 4958 |==================================================================== c . 4954 |==================================================================== Google Draco 1.5.6 Model: Church Facade ms < Lower Is Better a . 6854 |===================================================================== b . 6752 |==================================================================== c . 6748 |==================================================================== OpenVINO 2024.0 Model: Face Detection FP16 - Device: CPU ms < Lower Is Better a . 672.53 |=================================================================== b . 673.78 |=================================================================== c . 671.26 |=================================================================== OpenVINO 2024.0 Model: Person Detection FP16 - Device: CPU ms < Lower Is Better a . 91.72 |==================================================================== b . 91.88 |==================================================================== c . 91.92 |==================================================================== OpenVINO 2024.0 Model: Person Detection FP32 - Device: CPU ms < Lower Is Better a . 92.20 |==================================================================== b . 91.98 |==================================================================== c . 92.23 |==================================================================== OpenVINO 2024.0 Model: Vehicle Detection FP16 - Device: CPU ms < Lower Is Better a . 13.41 |==================================================================== b . 13.46 |==================================================================== c . 13.47 |==================================================================== OpenVINO 2024.0 Model: Face Detection FP16-INT8 - Device: CPU ms < Lower Is Better a . 340.70 |=================================================================== b . 339.35 |=================================================================== c . 340.84 |=================================================================== OpenVINO 2024.0 Model: Face Detection Retail FP16 - Device: CPU ms < Lower Is Better a . 2.91 |===================================================================== b . 2.92 |===================================================================== c . 2.90 |===================================================================== OpenVINO 2024.0 Model: Road Segmentation ADAS FP16 - Device: CPU ms < Lower Is Better a . 29.21 |=================================================================== b . 29.24 |=================================================================== c . 29.58 |==================================================================== OpenVINO 2024.0 Model: Vehicle Detection FP16-INT8 - Device: CPU ms < Lower Is Better a . 6.18 |===================================================================== b . 6.18 |===================================================================== c . 6.18 |===================================================================== OpenVINO 2024.0 Model: Weld Porosity Detection FP16 - Device: CPU ms < Lower Is Better a . 13.55 |==================================================================== b . 13.53 |==================================================================== c . 13.52 |==================================================================== OpenVINO 2024.0 Model: Face Detection Retail FP16-INT8 - Device: CPU ms < Lower Is Better a . 3.91 |=================================================================== b . 4.01 |===================================================================== c . 4.01 |===================================================================== OpenVINO 2024.0 Model: Road Segmentation ADAS FP16-INT8 - Device: CPU ms < Lower Is Better a . 19.36 |==================================================================== b . 19.36 |==================================================================== c . 19.16 |=================================================================== OpenVINO 2024.0 Model: Machine Translation EN To DE FP16 - Device: CPU ms < Lower Is Better a . 59.64 |================================================================== b . 59.87 |================================================================== c . 61.54 |==================================================================== OpenVINO 2024.0 Model: Weld Porosity Detection FP16-INT8 - Device: CPU ms < Lower Is Better a . 6.86 |===================================================================== b . 6.86 |===================================================================== c . 6.87 |===================================================================== OpenVINO 2024.0 Model: Person Vehicle Bike Detection FP16 - Device: CPU ms < Lower Is Better a . 7.04 |==================================================================== b . 7.10 |===================================================================== c . 6.82 |================================================================== OpenVINO 2024.0 Model: Noise Suppression Poconet-Like FP16 - Device: CPU ms < Lower Is Better a . 10.57 |=================================================================== b . 10.43 |================================================================== c . 10.69 |==================================================================== OpenVINO 2024.0 Model: Handwritten English Recognition FP16 - Device: CPU ms < Lower Is Better a . 26.23 |==================================================================== b . 25.90 |=================================================================== c . 26.12 |==================================================================== OpenVINO 2024.0 Model: Person Re-Identification Retail FP16 - Device: CPU ms < Lower Is Better a . 5.19 |===================================================================== b . 5.18 |===================================================================== c . 5.20 |===================================================================== OpenVINO 2024.0 Model: Age Gender Recognition Retail 0013 FP16 - Device: CPU ms < Lower Is Better a . 0.44 |===================================================================== b . 0.44 |===================================================================== c . 0.44 |===================================================================== OpenVINO 2024.0 Model: Handwritten English Recognition FP16-INT8 - Device: CPU ms < Lower Is Better a . 24.33 |==================================================================== b . 24.51 |==================================================================== c . 24.43 |==================================================================== OpenVINO 2024.0 Model: Age Gender Recognition Retail 0013 FP16-INT8 - Device: CPU ms < Lower Is Better a . 0.31 |===================================================================== b . 0.31 |===================================================================== c . 0.31 |===================================================================== Neural Magic DeepSparse 1.7 Model: NLP Document Classification, oBERT base uncased on IMDB - Scenario: Asynchronous Multi-Stream ms/batch < Lower Is Better a . 353.67 |=================================================================== b . 353.55 |=================================================================== c . 354.51 |=================================================================== Neural Magic DeepSparse 1.7 Model: NLP Document Classification, oBERT base uncased on IMDB - Scenario: Synchronous Single-Stream ms/batch < Lower Is Better a . 116.65 |=================================================================== b . 116.70 |=================================================================== c . 116.81 |=================================================================== Neural Magic DeepSparse 1.7 Model: NLP Text Classification, BERT base uncased SST2, Sparse INT8 - Scenario: Asynchronous Multi-Stream ms/batch < Lower Is Better a . 9.1031 |=================================================================== b . 9.1660 |=================================================================== c . 9.1356 |=================================================================== Neural Magic DeepSparse 1.7 Model: NLP Text Classification, BERT base uncased SST2, Sparse INT8 - Scenario: Synchronous Single-Stream ms/batch < Lower Is Better a . 4.6315 |=================================================================== b . 4.6521 |=================================================================== c . 4.6507 |=================================================================== Neural Magic DeepSparse 1.7 Model: ResNet-50, Baseline - Scenario: Asynchronous Multi-Stream ms/batch < Lower Is Better a . 28.47 |==================================================================== b . 28.55 |==================================================================== c . 28.55 |==================================================================== Neural Magic DeepSparse 1.7 Model: ResNet-50, Baseline - Scenario: Synchronous Single-Stream ms/batch < Lower Is Better a . 12.99 |==================================================================== b . 13.01 |==================================================================== c . 12.95 |==================================================================== Neural Magic DeepSparse 1.7 Model: ResNet-50, Sparse INT8 - Scenario: Asynchronous Multi-Stream ms/batch < Lower Is Better a . 3.8589 |================================================================== b . 3.8037 |================================================================= c . 3.9258 |=================================================================== Neural Magic DeepSparse 1.7 Model: ResNet-50, Sparse INT8 - Scenario: Synchronous Single-Stream ms/batch < Lower Is Better a . 2.2639 |=================================================================== b . 2.2642 |=================================================================== c . 2.2706 |=================================================================== Neural Magic DeepSparse 1.7 Model: Llama2 Chat 7b Quantized - Scenario: Asynchronous Multi-Stream ms/batch < Lower Is Better a . 961.37 |================================================================= b . 995.84 |=================================================================== c . 980.43 |================================================================== Neural Magic DeepSparse 1.7 Model: Llama2 Chat 7b Quantized - Scenario: Synchronous Single-Stream ms/batch < Lower Is Better a . 183.60 |=================================================================== b . 183.73 |=================================================================== c . 183.73 |=================================================================== Neural Magic DeepSparse 1.7 Model: CV Classification, ResNet-50 ImageNet - Scenario: Asynchronous Multi-Stream ms/batch < Lower Is Better a . 28.64 |==================================================================== b . 28.61 |==================================================================== c . 28.61 |==================================================================== Neural Magic DeepSparse 1.7 Model: CV Classification, ResNet-50 ImageNet - Scenario: Synchronous Single-Stream ms/batch < Lower Is Better a . 13.01 |==================================================================== b . 13.01 |==================================================================== c . 12.98 |==================================================================== Neural Magic DeepSparse 1.7 Model: CV Detection, YOLOv5s COCO, Sparse INT8 - Scenario: Asynchronous Multi-Stream ms/batch < Lower Is Better a . 59.29 |==================================================================== b . 58.74 |=================================================================== c . 58.53 |=================================================================== Neural Magic DeepSparse 1.7 Model: CV Detection, YOLOv5s COCO, Sparse INT8 - Scenario: Synchronous Single-Stream ms/batch < Lower Is Better a . 21.63 |==================================================================== b . 21.41 |=================================================================== c . 21.47 |=================================================================== Neural Magic DeepSparse 1.7 Model: NLP Text Classification, DistilBERT mnli - Scenario: Asynchronous Multi-Stream ms/batch < Lower Is Better a . 40.46 |==================================================================== b . 40.38 |==================================================================== c . 40.54 |==================================================================== Neural Magic DeepSparse 1.7 Model: NLP Text Classification, DistilBERT mnli - Scenario: Synchronous Single-Stream ms/batch < Lower Is Better a . 14.86 |==================================================================== b . 14.89 |==================================================================== c . 14.92 |==================================================================== Neural Magic DeepSparse 1.7 Model: CV Segmentation, 90% Pruned YOLACT Pruned - Scenario: Asynchronous Multi-Stream ms/batch < Lower Is Better a . 211.44 |=================================================================== b . 210.52 |=================================================================== c . 207.89 |================================================================== Neural Magic DeepSparse 1.7 Model: CV Segmentation, 90% Pruned YOLACT Pruned - Scenario: Synchronous Single-Stream ms/batch < Lower Is Better a . 66.60 |==================================================================== b . 66.83 |==================================================================== c . 66.53 |==================================================================== Neural Magic DeepSparse 1.7 Model: BERT-Large, NLP Question Answering, Sparse INT8 - Scenario: Asynchronous Multi-Stream ms/batch < Lower Is Better a . 20.16 |==================================================================== b . 20.21 |==================================================================== c . 20.22 |==================================================================== Neural Magic DeepSparse 1.7 Model: BERT-Large, NLP Question Answering, Sparse INT8 - Scenario: Synchronous Single-Stream ms/batch < Lower Is Better a . 9.7439 |=================================================================== b . 9.7480 |=================================================================== c . 9.7264 |=================================================================== Neural Magic DeepSparse 1.7 Model: NLP Token Classification, BERT base uncased conll2003 - Scenario: Asynchronous Multi-Stream ms/batch < Lower Is Better a . 353.60 |=================================================================== b . 353.58 |=================================================================== c . 353.79 |=================================================================== Neural Magic DeepSparse 1.7 Model: NLP Token Classification, BERT base uncased conll2003 - Scenario: Synchronous Single-Stream ms/batch < Lower Is Better a . 116.67 |=================================================================== b . 116.76 |=================================================================== c . 116.47 |=================================================================== Timed Linux Kernel Compilation 6.8 Build: defconfig Seconds < Lower Is Better a . 104.41 |=================================================================== b . 103.69 |=================================================================== c . 103.08 |================================================================== Timed Linux Kernel Compilation 6.8 Build: allmodconfig Seconds < Lower Is Better a . 1412.03 |================================================================== b . 1410.64 |================================================================== c . 1406.63 |================================================================== Primesieve 12.1 Length: 1e12 Seconds < Lower Is Better a . 15.11 |==================================================================== b . 15.15 |==================================================================== c . 15.09 |==================================================================== Primesieve 12.1 Length: 1e13 Seconds < Lower Is Better a . 186.00 |=================================================================== b . 186.00 |=================================================================== c . 186.83 |===================================================================