eps 2 x AMD EPYC 9684X 96-Core testing with a AMD Titanite_4G (RTI1007B BIOS) and ASPEED on Ubuntu 23.10 via the Phoronix Test Suite. a: Processor: 2 x AMD EPYC 9684X 96-Core @ 2.55GHz (192 Cores / 384 Threads), Motherboard: AMD Titanite_4G (RTI1007B BIOS), Chipset: AMD Device 14a4, Memory: 1520GB, Disk: 3201GB Micron_7450_MTFDKCB3T2TFS, Graphics: ASPEED, Network: Broadcom NetXtreme BCM5720 PCIe OS: Ubuntu 23.10, Kernel: 6.5.0-13-generic (x86_64), Compiler: GCC 13.2.0, File-System: ext4, Screen Resolution: 800x600 b: Processor: 2 x AMD EPYC 9684X 96-Core @ 2.55GHz (192 Cores / 384 Threads), Motherboard: AMD Titanite_4G (RTI1007B BIOS), Chipset: AMD Device 14a4, Memory: 1520GB, Disk: 3201GB Micron_7450_MTFDKCB3T2TFS, Graphics: ASPEED, Network: Broadcom NetXtreme BCM5720 PCIe OS: Ubuntu 23.10, Kernel: 6.5.0-13-generic (x86_64), Compiler: GCC 13.2.0, File-System: ext4, Screen Resolution: 800x600 LeelaChessZero 0.30 Backend: BLAS Nodes Per Second > Higher Is Better a . 853 |===================================================================== b . 871 |====================================================================== LeelaChessZero 0.30 Backend: Eigen Nodes Per Second > Higher Is Better a . 704 |===================================================================== b . 715 |====================================================================== Xmrig 6.21 Variant: KawPow - Hash Count: 1M H/s > Higher Is Better a . 123558.6 |================================================================= b . 123411.1 |================================================================= Xmrig 6.21 Variant: Monero - Hash Count: 1M H/s > Higher Is Better a . 123352.8 |================================================================= b . 122971.0 |================================================================= Xmrig 6.21 Variant: Wownero - Hash Count: 1M H/s > Higher Is Better a . 131141.9 |================================================================= b . 131613.6 |================================================================= Xmrig 6.21 Variant: GhostRider - Hash Count: 1M H/s > Higher Is Better a . 31859.7 |================================================================== b . 31728.9 |================================================================== Xmrig 6.21 Variant: CryptoNight-Heavy - Hash Count: 1M H/s > Higher Is Better a . 123041.6 |================================================================= b . 123777.7 |================================================================= Xmrig 6.21 Variant: CryptoNight-Femto UPX2 - Hash Count: 1M H/s > Higher Is Better a . 123199.0 |================================================================= b . 122070.3 |================================================================ Java SciMark 2.2 Computational Test: Composite Mflops > Higher Is Better a . 3984.62 |================================================================== b . 3996.76 |================================================================== Java SciMark 2.2 Computational Test: Monte Carlo Mflops > Higher Is Better a . 1631.42 |================================================================== b . 1632.45 |================================================================== Java SciMark 2.2 Computational Test: Fast Fourier Transform Mflops > Higher Is Better a . 420.74 |=================================================================== b . 421.91 |=================================================================== Java SciMark 2.2 Computational Test: Sparse Matrix Multiply Mflops > Higher Is Better a . 2809.01 |================================================================== b . 2792.09 |================================================================== Java SciMark 2.2 Computational Test: Dense LU Matrix Factorization Mflops > Higher Is Better a . 13358.53 |================================================================= b . 13434.09 |================================================================= Java SciMark 2.2 Computational Test: Jacobi Successive Over-Relaxation Mflops > Higher Is Better a . 1703.42 |================================================================== b . 1703.25 |================================================================== WebP2 Image Encode 20220823 Encode Settings: Default MP/s > Higher Is Better a . 9.48 |==================================================================== b . 9.63 |===================================================================== WebP2 Image Encode 20220823 Encode Settings: Quality 75, Compression Effort 7 MP/s > Higher Is Better a . 0.83 |===================================================================== b . 0.82 |==================================================================== WebP2 Image Encode 20220823 Encode Settings: Quality 95, Compression Effort 7 MP/s > Higher Is Better a . 0.45 |===================================================================== b . 0.45 |===================================================================== WebP2 Image Encode 20220823 Encode Settings: Quality 100, Compression Effort 5 MP/s > Higher Is Better a . 6.51 |===================================================================== b . 6.28 |=================================================================== WebP2 Image Encode 20220823 Encode Settings: Quality 100, Lossless Compression MP/s > Higher Is Better a . 0.11 |===================================================================== b . 0.11 |===================================================================== SVT-AV1 1.8 Encoder Mode: Preset 4 - Input: Bosphorus 4K Frames Per Second > Higher Is Better a . 8.248 |==================================================================== b . 8.208 |==================================================================== SVT-AV1 1.8 Encoder Mode: Preset 8 - Input: Bosphorus 4K Frames Per Second > Higher Is Better a . 86.43 |==================================================================== b . 86.84 |==================================================================== SVT-AV1 1.8 Encoder Mode: Preset 12 - Input: Bosphorus 4K Frames Per Second > Higher Is Better a . 178.91 |================================================================ b . 186.61 |=================================================================== SVT-AV1 1.8 Encoder Mode: Preset 13 - Input: Bosphorus 4K Frames Per Second > Higher Is Better a . 176.67 |================================================================ b . 184.35 |=================================================================== SVT-AV1 1.8 Encoder Mode: Preset 4 - Input: Bosphorus 1080p Frames Per Second > Higher Is Better a . 21.42 |==================================================================== b . 21.31 |==================================================================== SVT-AV1 1.8 Encoder Mode: Preset 8 - Input: Bosphorus 1080p Frames Per Second > Higher Is Better a . 165.10 |=================================================================== b . 162.56 |================================================================== SVT-AV1 1.8 Encoder Mode: Preset 12 - Input: Bosphorus 1080p Frames Per Second > Higher Is Better a . 571.88 |=================================================================== b . 569.96 |=================================================================== SVT-AV1 1.8 Encoder Mode: Preset 13 - Input: Bosphorus 1080p Frames Per Second > Higher Is Better a . 635.81 |=================================================================== b . 639.09 |=================================================================== OpenSSL Algorithm: SHA256 byte/s > Higher Is Better a . 281869895760 |============================================================= b . 282211175400 |============================================================= OpenSSL Algorithm: SHA512 byte/s > Higher Is Better a . 91630925473 |============================================================== b . 91835961470 |============================================================== OpenSSL Algorithm: RSA4096 sign/s > Higher Is Better a . 98622.0 |================================================================== b . 98528.8 |================================================================== OpenSSL Algorithm: RSA4096 verify/s > Higher Is Better a . 3244390.3 |================================================================ b . 3243345.2 |================================================================ OpenSSL Algorithm: ChaCha20 OpenSSL Algorithm: AES-128-GCM OpenSSL Algorithm: AES-256-GCM OpenSSL Algorithm: ChaCha20-Poly1305 Apache Spark TPC-H 3.5 Scale Factor: 1 - Geometric Mean Of All Queries Seconds < Lower Is Better a . 2.44964916 |============================================================== b . 2.49517747 |=============================================================== Apache Spark TPC-H 3.5 Scale Factor: 1 - Q01 Seconds < Lower Is Better a . 4.32006081 |============================================================= b . 4.44657946 |=============================================================== Apache Spark TPC-H 3.5 Scale Factor: 1 - Q02 Seconds < Lower Is Better a . 2.06179071 |============================================================== b . 2.08224201 |=============================================================== Apache Spark TPC-H 3.5 Scale Factor: 1 - Q03 Seconds < Lower Is Better a . 3.86442184 |=============================================================== b . 3.86610818 |=============================================================== Apache Spark TPC-H 3.5 Scale Factor: 1 - Q04 Seconds < Lower Is Better a . 3.92525745 |=============================================================== b . 3.75427246 |============================================================ Apache Spark TPC-H 3.5 Scale Factor: 1 - Q05 Seconds < Lower Is Better a . 4.13122161 |=============================================================== b . 3.69217634 |======================================================== Apache Spark TPC-H 3.5 Scale Factor: 1 - Q06 Seconds < Lower Is Better a . 0.46822915 |=============================================================== b . 0.35801557 |================================================ Apache Spark TPC-H 3.5 Scale Factor: 1 - Q07 Seconds < Lower Is Better a . 4.01044806 |=============================================================== b . 3.87790275 |============================================================= Apache Spark TPC-H 3.5 Scale Factor: 1 - Q08 Seconds < Lower Is Better a . 2.65584644 |=============================================================== b . 2.60907817 |============================================================== Apache Spark TPC-H 3.5 Scale Factor: 1 - Q09 Seconds < Lower Is Better a . 5.70969407 |============================================================= b . 5.89775848 |=============================================================== Apache Spark TPC-H 3.5 Scale Factor: 1 - Q10 Seconds < Lower Is Better a . 3.81359665 |=============================================================== b . 3.81245542 |=============================================================== Apache Spark TPC-H 3.5 Scale Factor: 1 - Q11 Seconds < Lower Is Better a . 1.27338135 |=============================================================== b . 1.13998687 |======================================================== Apache Spark TPC-H 3.5 Scale Factor: 1 - Q12 Seconds < Lower Is Better a . 2.17542648 |============================================================ b . 2.26641607 |=============================================================== Apache Spark TPC-H 3.5 Scale Factor: 1 - Q13 Seconds < Lower Is Better a . 1.58815936 |========================================================= b . 1.74074161 |=============================================================== Apache Spark TPC-H 3.5 Scale Factor: 1 - Q14 Seconds < Lower Is Better a . 2.06485331 |=========================================================== b . 2.21146965 |=============================================================== Apache Spark TPC-H 3.5 Scale Factor: 1 - Q15 Seconds < Lower Is Better a . 2.50185966 |============================================================= b . 2.58714175 |=============================================================== Apache Spark TPC-H 3.5 Scale Factor: 1 - Q16 Seconds < Lower Is Better a . 1.38147259 |========================================================= b . 1.51779914 |=============================================================== Apache Spark TPC-H 3.5 Scale Factor: 1 - Q17 Seconds < Lower Is Better a . 2.95993924 |=============================================================== b . 2.88348198 |============================================================= Apache Spark TPC-H 3.5 Scale Factor: 1 - Q18 Seconds < Lower Is Better a . 5.62853845 |=============================================================== b . 5.13171148 |========================================================= Apache Spark TPC-H 3.5 Scale Factor: 1 - Q19 Seconds < Lower Is Better a . 0.79092395 |========================================================== b . 0.85797596 |=============================================================== Apache Spark TPC-H 3.5 Scale Factor: 1 - Q20 Seconds < Lower Is Better a . 3.05739617 |=============================================================== b . 3.05001688 |=============================================================== Apache Spark TPC-H 3.5 Scale Factor: 1 - Q21 Seconds < Lower Is Better a . 9.64531231 |=============================================================== b . 9.55909538 |============================================================== Apache Spark TPC-H 3.5 Scale Factor: 1 - Q22 Seconds < Lower Is Better a . 1.00769047 |============================================================ b . 1.06679213 |=============================================================== Apache Spark TPC-H 3.5 Scale Factor: 10 - Geometric Mean Of All Queries Seconds < Lower Is Better a . 10.72 |==================================================================== b . 10.66 |==================================================================== Apache Spark TPC-H 3.5 Scale Factor: 10 - Q01 Seconds < Lower Is Better a . 7.58889151 |=============================================================== b . 7.28826714 |============================================================= Apache Spark TPC-H 3.5 Scale Factor: 10 - Q02 Seconds < Lower Is Better a . 7.43104283 |=============================================================== b . 7.39245987 |=============================================================== Apache Spark TPC-H 3.5 Scale Factor: 10 - Q03 Seconds < Lower Is Better a . 13.97 |================================================================== b . 14.29 |==================================================================== Apache Spark TPC-H 3.5 Scale Factor: 10 - Q04 Seconds < Lower Is Better a . 12.35 |==================================================================== b . 11.26 |============================================================== Apache Spark TPC-H 3.5 Scale Factor: 10 - Q05 Seconds < Lower Is Better a . 16.44 |=========================================================== b . 18.86 |==================================================================== Apache Spark TPC-H 3.5 Scale Factor: 10 - Q06 Seconds < Lower Is Better a . 2.05104745 |=============================================================== b . 1.85595930 |========================================================= Apache Spark TPC-H 3.5 Scale Factor: 10 - Q07 Seconds < Lower Is Better a . 14.65 |=================================================================== b . 14.90 |==================================================================== Apache Spark TPC-H 3.5 Scale Factor: 10 - Q08 Seconds < Lower Is Better a . 15.52 |==================================================================== b . 14.56 |================================================================ Apache Spark TPC-H 3.5 Scale Factor: 10 - Q09 Seconds < Lower Is Better a . 21.91 |================================================================== b . 22.53 |==================================================================== Apache Spark TPC-H 3.5 Scale Factor: 10 - Q10 Seconds < Lower Is Better a . 15.17 |==================================================================== b . 14.78 |================================================================== Apache Spark TPC-H 3.5 Scale Factor: 10 - Q11 Seconds < Lower Is Better a . 8.00292349 |============================================================= b . 8.27814293 |=============================================================== Apache Spark TPC-H 3.5 Scale Factor: 10 - Q12 Seconds < Lower Is Better a . 9.94400438 |============================================================= b . 10.03829002 |============================================================== Apache Spark TPC-H 3.5 Scale Factor: 10 - Q13 Seconds < Lower Is Better a . 7.37728373 |=========================================================== b . 7.94083786 |=============================================================== Apache Spark TPC-H 3.5 Scale Factor: 10 - Q14 Seconds < Lower Is Better a . 7.07622369 |=============================================================== b . 6.90602303 |============================================================= Apache Spark TPC-H 3.5 Scale Factor: 10 - Q15 Seconds < Lower Is Better a . 5.84138076 |=============================================================== b . 5.43870592 |=========================================================== Apache Spark TPC-H 3.5 Scale Factor: 10 - Q16 Seconds < Lower Is Better a . 6.87131294 |============================================================== b . 6.95270681 |=============================================================== Apache Spark TPC-H 3.5 Scale Factor: 10 - Q17 Seconds < Lower Is Better a . 12.77 |=================================================================== b . 13.01 |==================================================================== Apache Spark TPC-H 3.5 Scale Factor: 10 - Q18 Seconds < Lower Is Better a . 18.47 |==================================================================== b . 17.31 |================================================================ Apache Spark TPC-H 3.5 Scale Factor: 10 - Q19 Seconds < Lower Is Better a . 6.20677837 |=============================================================== b . 6.06041670 |============================================================== Apache Spark TPC-H 3.5 Scale Factor: 10 - Q20 Seconds < Lower Is Better a . 11.44 |=================================================================== b . 11.54 |==================================================================== Apache Spark TPC-H 3.5 Scale Factor: 10 - Q21 Seconds < Lower Is Better a . 32.91 |==================================================================== b . 32.70 |==================================================================== Apache Spark TPC-H 3.5 Scale Factor: 10 - Q22 Seconds < Lower Is Better a . 6.05411895 |=============================================================== b . 6.04430914 |=============================================================== Apache Spark TPC-H 3.5 Scale Factor: 50 - Geometric Mean Of All Queries Seconds < Lower Is Better a . 19.59 |==================================================================== b . 19.56 |==================================================================== Apache Spark TPC-H 3.5 Scale Factor: 50 - Q01 Seconds < Lower Is Better a . 12.01 |=============================================================== b . 12.87 |==================================================================== Apache Spark TPC-H 3.5 Scale Factor: 50 - Q02 Seconds < Lower Is Better a . 14.25 |=================================================================== b . 14.53 |==================================================================== Apache Spark TPC-H 3.5 Scale Factor: 50 - Q03 Seconds < Lower Is Better a . 26.19 |============================================================ b . 29.69 |==================================================================== Apache Spark TPC-H 3.5 Scale Factor: 50 - Q04 Seconds < Lower Is Better a . 21.00 |================================================================= b . 21.82 |==================================================================== Apache Spark TPC-H 3.5 Scale Factor: 50 - Q05 Seconds < Lower Is Better a . 29.84 |================================================================= b . 31.20 |==================================================================== Apache Spark TPC-H 3.5 Scale Factor: 50 - Q06 Seconds < Lower Is Better a . 5.90309207 |=============================================================== b . 5.88382483 |=============================================================== Apache Spark TPC-H 3.5 Scale Factor: 50 - Q07 Seconds < Lower Is Better a . 24.86 |================================================================= b . 25.86 |==================================================================== Apache Spark TPC-H 3.5 Scale Factor: 50 - Q08 Seconds < Lower Is Better a . 26.74 |==================================================================== b . 26.63 |==================================================================== Apache Spark TPC-H 3.5 Scale Factor: 50 - Q09 Seconds < Lower Is Better a . 36.66 |==================================================================== b . 36.67 |==================================================================== Apache Spark TPC-H 3.5 Scale Factor: 50 - Q10 Seconds < Lower Is Better a . 24.36 |=================================================================== b . 24.69 |==================================================================== Apache Spark TPC-H 3.5 Scale Factor: 50 - Q11 Seconds < Lower Is Better a . 13.58 |==================================================================== b . 13.31 |=================================================================== Apache Spark TPC-H 3.5 Scale Factor: 50 - Q12 Seconds < Lower Is Better a . 19.41 |==================================================================== b . 17.70 |============================================================== Apache Spark TPC-H 3.5 Scale Factor: 50 - Q13 Seconds < Lower Is Better a . 12.76 |=================================================================== b . 13.04 |==================================================================== Apache Spark TPC-H 3.5 Scale Factor: 50 - Q14 Seconds < Lower Is Better a . 12.70 |==================================================================== b . 12.57 |=================================================================== Apache Spark TPC-H 3.5 Scale Factor: 50 - Q15 Seconds < Lower Is Better a . 9.77733866 |=============================================================== b . 9.48287773 |============================================================= Apache Spark TPC-H 3.5 Scale Factor: 50 - Q16 Seconds < Lower Is Better a . 14.22 |================================================================= b . 14.98 |==================================================================== Apache Spark TPC-H 3.5 Scale Factor: 50 - Q17 Seconds < Lower Is Better a . 24.31 |=================================================================== b . 24.56 |==================================================================== Apache Spark TPC-H 3.5 Scale Factor: 50 - Q18 Seconds < Lower Is Better a . 34.51 |==================================================================== b . 33.74 |================================================================== Apache Spark TPC-H 3.5 Scale Factor: 50 - Q19 Seconds < Lower Is Better a . 10.45 |=========================================================== b . 12.09 |==================================================================== Apache Spark TPC-H 3.5 Scale Factor: 50 - Q20 Seconds < Lower Is Better a . 20.80 |=================================================================== b . 21.05 |==================================================================== Apache Spark TPC-H 3.5 Scale Factor: 50 - Q21 Seconds < Lower Is Better a . 87.90 |==================================================================== b . 77.71 |============================================================ Apache Spark TPC-H 3.5 Scale Factor: 50 - Q22 Seconds < Lower Is Better a . 10.69 |=================================================================== b . 10.87 |==================================================================== PyTorch 2.1 Device: CPU - Batch Size: 1 - Model: ResNet-50 batches/sec > Higher Is Better a . 23.57 |==================================================================== b . 23.12 |=================================================================== PyTorch 2.1 Device: CPU - Batch Size: 1 - Model: ResNet-152 batches/sec > Higher Is Better a . 10.16 |================================================================== b . 10.43 |==================================================================== PyTorch 2.1 Device: CPU - Batch Size: 16 - Model: ResNet-50 batches/sec > Higher Is Better a . 21.16 |=================================================================== b . 21.57 |==================================================================== PyTorch 2.1 Device: CPU - Batch Size: 32 - Model: ResNet-50 batches/sec > Higher Is Better a . 21.00 |==================================================================== b . 21.09 |==================================================================== PyTorch 2.1 Device: CPU - Batch Size: 16 - Model: ResNet-152 batches/sec > Higher Is Better a . 8.93 |===================================================================== b . 8.97 |===================================================================== PyTorch 2.1 Device: CPU - Batch Size: 256 - Model: ResNet-50 batches/sec > Higher Is Better a . 21.29 |==================================================================== b . 20.60 |================================================================== PyTorch 2.1 Device: CPU - Batch Size: 32 - Model: ResNet-152 batches/sec > Higher Is Better a . 8.90 |==================================================================== b . 8.98 |===================================================================== PyTorch 2.1 Device: CPU - Batch Size: 256 - Model: ResNet-152 batches/sec > Higher Is Better a . 8.96 |================================================================ b . 9.65 |===================================================================== PyTorch 2.1 Device: CPU - Batch Size: 1 - Model: Efficientnet_v2_l batches/sec > Higher Is Better a . 6.40 |================================================================== b . 6.74 |===================================================================== PyTorch 2.1 Device: CPU - Batch Size: 16 - Model: Efficientnet_v2_l batches/sec > Higher Is Better a . 2.32 |==================================================================== b . 2.34 |===================================================================== PyTorch 2.1 Device: CPU - Batch Size: 32 - Model: Efficientnet_v2_l batches/sec > Higher Is Better a . 2.32 |===================================================================== b . 2.32 |===================================================================== PyTorch 2.1 Device: CPU - Batch Size: 256 - Model: Efficientnet_v2_l batches/sec > Higher Is Better a . 2.32 |===================================================================== b . 2.28 |==================================================================== Neural Magic DeepSparse 1.6 Model: NLP Document Classification, oBERT base uncased on IMDB - Scenario: Asynchronous Multi-Stream items/sec > Higher Is Better a . 132.66 |=================================================================== b . 132.27 |=================================================================== Neural Magic DeepSparse 1.6 Model: NLP Document Classification, oBERT base uncased on IMDB - Scenario: Asynchronous Multi-Stream ms/batch < Lower Is Better a . 715.04 |=================================================================== b . 717.59 |=================================================================== Neural Magic DeepSparse 1.6 Model: NLP Document Classification, oBERT base uncased on IMDB - Scenario: Synchronous Single-Stream items/sec > Higher Is Better a . 48.45 |==================================================================== b . 48.33 |==================================================================== Neural Magic DeepSparse 1.6 Model: NLP Document Classification, oBERT base uncased on IMDB - Scenario: Synchronous Single-Stream ms/batch < Lower Is Better a . 20.63 |==================================================================== b . 20.69 |==================================================================== Neural Magic DeepSparse 1.6 Model: NLP Text Classification, BERT base uncased SST2, Sparse INT8 - Scenario: Asynchronous Multi-Stream items/sec > Higher Is Better a . 5540.63 |================================================================== b . 5540.52 |================================================================== Neural Magic DeepSparse 1.6 Model: NLP Text Classification, BERT base uncased SST2, Sparse INT8 - Scenario: Asynchronous Multi-Stream ms/batch < Lower Is Better a . 17.30 |==================================================================== b . 17.30 |==================================================================== Neural Magic DeepSparse 1.6 Model: NLP Text Classification, BERT base uncased SST2, Sparse INT8 - Scenario: Synchronous Single-Stream items/sec > Higher Is Better a . 190.80 |=================================================================== b . 191.10 |=================================================================== Neural Magic DeepSparse 1.6 Model: NLP Text Classification, BERT base uncased SST2, Sparse INT8 - Scenario: Synchronous Single-Stream ms/batch < Lower Is Better a . 5.2377 |=================================================================== b . 5.2296 |=================================================================== Neural Magic DeepSparse 1.6 Model: ResNet-50, Baseline - Scenario: Asynchronous Multi-Stream items/sec > Higher Is Better a . 1758.59 |================================================================== b . 1756.56 |================================================================== Neural Magic DeepSparse 1.6 Model: ResNet-50, Baseline - Scenario: Asynchronous Multi-Stream ms/batch < Lower Is Better a . 54.51 |==================================================================== b . 54.55 |==================================================================== Neural Magic DeepSparse 1.6 Model: ResNet-50, Baseline - Scenario: Synchronous Single-Stream items/sec > Higher Is Better a . 209.80 |=================================================================== b . 206.97 |================================================================== Neural Magic DeepSparse 1.6 Model: ResNet-50, Baseline - Scenario: Synchronous Single-Stream ms/batch < Lower Is Better a . 4.7637 |================================================================== b . 4.8290 |=================================================================== Neural Magic DeepSparse 1.6 Model: ResNet-50, Sparse INT8 - Scenario: Asynchronous Multi-Stream items/sec > Higher Is Better a . 17108.46 |================================================================= b . 17047.64 |================================================================= Neural Magic DeepSparse 1.6 Model: ResNet-50, Sparse INT8 - Scenario: Asynchronous Multi-Stream ms/batch < Lower Is Better a . 5.5955 |=================================================================== b . 5.6159 |=================================================================== Neural Magic DeepSparse 1.6 Model: ResNet-50, Sparse INT8 - Scenario: Synchronous Single-Stream items/sec > Higher Is Better a . 804.18 |=================================================================== b . 804.75 |=================================================================== Neural Magic DeepSparse 1.6 Model: ResNet-50, Sparse INT8 - Scenario: Synchronous Single-Stream ms/batch < Lower Is Better a . 1.2413 |=================================================================== b . 1.2404 |=================================================================== Neural Magic DeepSparse 1.6 Model: CV Detection, YOLOv5s COCO - Scenario: Asynchronous Multi-Stream items/sec > Higher Is Better a . 784.52 |=================================================================== b . 786.89 |=================================================================== Neural Magic DeepSparse 1.6 Model: CV Detection, YOLOv5s COCO - Scenario: Asynchronous Multi-Stream ms/batch < Lower Is Better a . 122.03 |=================================================================== b . 121.61 |=================================================================== Neural Magic DeepSparse 1.6 Model: CV Detection, YOLOv5s COCO - Scenario: Synchronous Single-Stream items/sec > Higher Is Better a . 211.74 |=================================================================== b . 211.77 |=================================================================== Neural Magic DeepSparse 1.6 Model: CV Detection, YOLOv5s COCO - Scenario: Synchronous Single-Stream ms/batch < Lower Is Better a . 4.7188 |=================================================================== b . 4.7180 |=================================================================== Neural Magic DeepSparse 1.6 Model: BERT-Large, NLP Question Answering - Scenario: Asynchronous Multi-Stream items/sec > Higher Is Better a . 156.42 |=================================================================== b . 156.43 |=================================================================== Neural Magic DeepSparse 1.6 Model: BERT-Large, NLP Question Answering - Scenario: Asynchronous Multi-Stream ms/batch < Lower Is Better a . 607.57 |=================================================================== b . 607.57 |=================================================================== Neural Magic DeepSparse 1.6 Model: BERT-Large, NLP Question Answering - Scenario: Synchronous Single-Stream items/sec > Higher Is Better a . 32.02 |==================================================================== b . 31.98 |==================================================================== Neural Magic DeepSparse 1.6 Model: BERT-Large, NLP Question Answering - Scenario: Synchronous Single-Stream ms/batch < Lower Is Better a . 31.22 |==================================================================== b . 31.26 |==================================================================== Neural Magic DeepSparse 1.6 Model: CV Classification, ResNet-50 ImageNet - Scenario: Asynchronous Multi-Stream items/sec > Higher Is Better a . 1761.40 |================================================================== b . 1759.07 |================================================================== Neural Magic DeepSparse 1.6 Model: CV Classification, ResNet-50 ImageNet - Scenario: Asynchronous Multi-Stream ms/batch < Lower Is Better a . 54.41 |==================================================================== b . 54.49 |==================================================================== Neural Magic DeepSparse 1.6 Model: CV Classification, ResNet-50 ImageNet - Scenario: Synchronous Single-Stream items/sec > Higher Is Better a . 208.12 |=================================================================== b . 209.20 |=================================================================== Neural Magic DeepSparse 1.6 Model: CV Classification, ResNet-50 ImageNet - Scenario: Synchronous Single-Stream ms/batch < Lower Is Better a . 4.8022 |=================================================================== b . 4.7775 |=================================================================== Neural Magic DeepSparse 1.6 Model: CV Detection, YOLOv5s COCO, Sparse INT8 - Scenario: Asynchronous Multi-Stream items/sec > Higher Is Better a . 796.07 |=================================================================== b . 797.41 |=================================================================== Neural Magic DeepSparse 1.6 Model: CV Detection, YOLOv5s COCO, Sparse INT8 - Scenario: Asynchronous Multi-Stream ms/batch < Lower Is Better a . 120.21 |=================================================================== b . 120.04 |=================================================================== Neural Magic DeepSparse 1.6 Model: CV Detection, YOLOv5s COCO, Sparse INT8 - Scenario: Synchronous Single-Stream items/sec > Higher Is Better a . 212.10 |=================================================================== b . 212.13 |=================================================================== Neural Magic DeepSparse 1.6 Model: CV Detection, YOLOv5s COCO, Sparse INT8 - Scenario: Synchronous Single-Stream ms/batch < Lower Is Better a . 4.7126 |=================================================================== b . 4.7117 |=================================================================== Neural Magic DeepSparse 1.6 Model: NLP Text Classification, DistilBERT mnli - Scenario: Asynchronous Multi-Stream items/sec > Higher Is Better a . 1136.71 |================================================================== b . 1136.64 |================================================================== Neural Magic DeepSparse 1.6 Model: NLP Text Classification, DistilBERT mnli - Scenario: Asynchronous Multi-Stream ms/batch < Lower Is Better a . 84.25 |==================================================================== b . 84.25 |==================================================================== Neural Magic DeepSparse 1.6 Model: NLP Text Classification, DistilBERT mnli - Scenario: Synchronous Single-Stream items/sec > Higher Is Better a . 224.58 |=================================================================== b . 225.40 |=================================================================== Neural Magic DeepSparse 1.6 Model: NLP Text Classification, DistilBERT mnli - Scenario: Synchronous Single-Stream ms/batch < Lower Is Better a . 4.4503 |=================================================================== b . 4.4341 |=================================================================== Neural Magic DeepSparse 1.6 Model: CV Segmentation, 90% Pruned YOLACT Pruned - Scenario: Asynchronous Multi-Stream items/sec > Higher Is Better a . 248.58 |=================================================================== b . 249.50 |=================================================================== Neural Magic DeepSparse 1.6 Model: CV Segmentation, 90% Pruned YOLACT Pruned - Scenario: Asynchronous Multi-Stream ms/batch < Lower Is Better a . 383.20 |=================================================================== b . 382.56 |=================================================================== Neural Magic DeepSparse 1.6 Model: CV Segmentation, 90% Pruned YOLACT Pruned - Scenario: Synchronous Single-Stream items/sec > Higher Is Better a . 65.21 |==================================================================== b . 65.01 |==================================================================== Neural Magic DeepSparse 1.6 Model: CV Segmentation, 90% Pruned YOLACT Pruned - Scenario: Synchronous Single-Stream ms/batch < Lower Is Better a . 15.32 |==================================================================== b . 15.37 |==================================================================== Neural Magic DeepSparse 1.6 Model: BERT-Large, NLP Question Answering, Sparse INT8 - Scenario: Asynchronous Multi-Stream items/sec > Higher Is Better a . 2608.01 |================================================================== b . 2596.10 |================================================================== Neural Magic DeepSparse 1.6 Model: BERT-Large, NLP Question Answering, Sparse INT8 - Scenario: Asynchronous Multi-Stream ms/batch < Lower Is Better a . 36.75 |==================================================================== b . 36.91 |==================================================================== Neural Magic DeepSparse 1.6 Model: BERT-Large, NLP Question Answering, Sparse INT8 - Scenario: Synchronous Single-Stream items/sec > Higher Is Better a . 68.27 |==================================================================== b . 68.26 |==================================================================== Neural Magic DeepSparse 1.6 Model: BERT-Large, NLP Question Answering, Sparse INT8 - Scenario: Synchronous Single-Stream ms/batch < Lower Is Better a . 14.64 |==================================================================== b . 14.64 |==================================================================== Neural Magic DeepSparse 1.6 Model: NLP Token Classification, BERT base uncased conll2003 - Scenario: Asynchronous Multi-Stream items/sec > Higher Is Better a . 132.05 |=================================================================== b . 132.42 |=================================================================== Neural Magic DeepSparse 1.6 Model: NLP Token Classification, BERT base uncased conll2003 - Scenario: Asynchronous Multi-Stream ms/batch < Lower Is Better a . 719.28 |=================================================================== b . 717.98 |=================================================================== Neural Magic DeepSparse 1.6 Model: NLP Token Classification, BERT base uncased conll2003 - Scenario: Synchronous Single-Stream items/sec > Higher Is Better a . 48.49 |==================================================================== b . 48.33 |==================================================================== Neural Magic DeepSparse 1.6 Model: NLP Token Classification, BERT base uncased conll2003 - Scenario: Synchronous Single-Stream ms/batch < Lower Is Better a . 20.62 |==================================================================== b . 20.68 |====================================================================