sdfa AMD Ryzen Threadripper 3990X 64-Core testing with a Gigabyte TRX40 AORUS PRO WIFI (F6 BIOS) and AMD Radeon RX 5700 8GB on Ubuntu 23.10 via the Phoronix Test Suite. a: Processor: AMD Ryzen Threadripper 3990X 64-Core @ 2.90GHz (64 Cores / 128 Threads), Motherboard: Gigabyte TRX40 AORUS PRO WIFI (F6 BIOS), Chipset: AMD Starship/Matisse, Memory: 128GB, Disk: Samsung SSD 970 EVO Plus 500GB, Graphics: AMD Radeon RX 5700 8GB (1750/875MHz), Audio: AMD Navi 10 HDMI Audio, Monitor: DELL P2415Q, Network: Intel I211 + Intel Wi-Fi 6 AX200 OS: Ubuntu 23.10, Kernel: 6.5.0-13-generic (x86_64), Desktop: GNOME Shell 45.0, Display Server: X Server + Wayland, OpenGL: 4.6 Mesa 23.2.1-1ubuntu3 (LLVM 15.0.7 DRM 3.54), Compiler: GCC 13.2.0, File-System: ext4, Screen Resolution: 3840x2160 b: Processor: AMD Ryzen Threadripper 3990X 64-Core @ 2.90GHz (64 Cores / 128 Threads), Motherboard: Gigabyte TRX40 AORUS PRO WIFI (F6 BIOS), Chipset: AMD Starship/Matisse, Memory: 128GB, Disk: Samsung SSD 970 EVO Plus 500GB, Graphics: AMD Radeon RX 5700 8GB (1750/875MHz), Audio: AMD Navi 10 HDMI Audio, Monitor: DELL P2415Q, Network: Intel I211 + Intel Wi-Fi 6 AX200 OS: Ubuntu 23.10, Kernel: 6.5.0-13-generic (x86_64), Desktop: GNOME Shell 45.0, Display Server: X Server + Wayland, OpenGL: 4.6 Mesa 23.2.1-1ubuntu3 (LLVM 15.0.7 DRM 3.54), Compiler: GCC 13.2.0, File-System: ext4, Screen Resolution: 3840x2160 c: Processor: AMD Ryzen Threadripper 3990X 64-Core @ 2.90GHz (64 Cores / 128 Threads), Motherboard: Gigabyte TRX40 AORUS PRO WIFI (F6 BIOS), Chipset: AMD Starship/Matisse, Memory: 128GB, Disk: Samsung SSD 970 EVO Plus 500GB, Graphics: AMD Radeon RX 5700 8GB (1750/875MHz), Audio: AMD Navi 10 HDMI Audio, Monitor: DELL P2415Q, Network: Intel I211 + Intel Wi-Fi 6 AX200 OS: Ubuntu 23.10, Kernel: 6.5.0-13-generic (x86_64), Desktop: GNOME Shell 45.0, Display Server: X Server + Wayland, OpenGL: 4.6 Mesa 23.2.1-1ubuntu3 (LLVM 15.0.7 DRM 3.54), Compiler: GCC 13.2.0, File-System: ext4, Screen Resolution: 3840x2160 d: Processor: AMD Ryzen Threadripper 3990X 64-Core @ 2.90GHz (64 Cores / 128 Threads), Motherboard: Gigabyte TRX40 AORUS PRO WIFI (F6 BIOS), Chipset: AMD Starship/Matisse, Memory: 128GB, Disk: Samsung SSD 970 EVO Plus 500GB, Graphics: AMD Radeon RX 5700 8GB (1750/875MHz), Audio: AMD Navi 10 HDMI Audio, Monitor: DELL P2415Q, Network: Intel I211 + Intel Wi-Fi 6 AX200 OS: Ubuntu 23.10, Kernel: 6.5.0-13-generic (x86_64), Desktop: GNOME Shell 45.0, Display Server: X Server + Wayland, OpenGL: 4.6 Mesa 23.2.1-1ubuntu3 (LLVM 15.0.7 DRM 3.54), Compiler: GCC 13.2.0, File-System: ext4, Screen Resolution: 3840x2160 Apache Spark TPC-H 3.5 Scale Factor: 1 - Geometric Mean Of All Queries Seconds < Lower Is Better a . 2.20328734 |============================================================== b . 2.24531415 |=============================================================== c . 2.23705911 |=============================================================== d . 2.22356446 |============================================================== Apache Spark TPC-H 3.5 Scale Factor: 1 - Q01 Seconds < Lower Is Better a . 4.69511843 |============================================================= b . 4.73092079 |============================================================== c . 4.84034109 |=============================================================== d . 4.84622987 |=============================================================== Apache Spark TPC-H 3.5 Scale Factor: 1 - Q02 Seconds < Lower Is Better a . 2.19916511 |============================================================= b . 2.20726132 |============================================================= c . 2.27846766 |=============================================================== d . 2.27512868 |=============================================================== Apache Spark TPC-H 3.5 Scale Factor: 1 - Q03 Seconds < Lower Is Better a . 3.15890259 |=========================================================== b . 3.23912430 |============================================================= c . 3.34556897 |=============================================================== d . 3.15447322 |=========================================================== Apache Spark TPC-H 3.5 Scale Factor: 1 - Q04 Seconds < Lower Is Better a . 2.96026391 |============================================================ b . 3.09257221 |=============================================================== c . 2.94428595 |============================================================ d . 2.85856040 |========================================================== Apache Spark TPC-H 3.5 Scale Factor: 1 - Q05 Seconds < Lower Is Better a . 3.46156806 |============================================================== b . 3.37768650 |============================================================ c . 3.52418653 |=============================================================== d . 3.30995369 |=========================================================== Apache Spark TPC-H 3.5 Scale Factor: 1 - Q06 Seconds < Lower Is Better a . 0.75870752 |=============================================================== b . 0.73223764 |============================================================= c . 0.71082556 |=========================================================== d . 0.74262398 |============================================================== Apache Spark TPC-H 3.5 Scale Factor: 1 - Q07 Seconds < Lower Is Better a . 3.16296500 |============================================================ b . 3.30348134 |=============================================================== c . 3.18430217 |============================================================= d . 3.25330043 |============================================================== Apache Spark TPC-H 3.5 Scale Factor: 1 - Q08 Seconds < Lower Is Better a . 2.29091269 |============================================================= b . 2.32780766 |============================================================== c . 2.35779921 |=============================================================== d . 2.22529737 |=========================================================== Apache Spark TPC-H 3.5 Scale Factor: 1 - Q09 Seconds < Lower Is Better a . 4.40090740 |============================================================= b . 4.51510048 |============================================================== c . 4.32967218 |============================================================ d . 4.55872504 |=============================================================== Apache Spark TPC-H 3.5 Scale Factor: 1 - Q10 Seconds < Lower Is Better a . 2.83789939 |========================================================= b . 3.07959628 |============================================================== c . 3.02914357 |============================================================= d . 3.14210590 |=============================================================== Apache Spark TPC-H 3.5 Scale Factor: 1 - Q11 Seconds < Lower Is Better a . 1.30849856 |============================================================ b . 1.36447799 |=============================================================== c . 1.28974267 |============================================================ d . 1.29190469 |============================================================ Apache Spark TPC-H 3.5 Scale Factor: 1 - Q12 Seconds < Lower Is Better a . 2.14987269 |=============================================================== b . 2.02941990 |=========================================================== c . 2.04039538 |============================================================ d . 2.06954416 |============================================================= Apache Spark TPC-H 3.5 Scale Factor: 1 - Q13 Seconds < Lower Is Better a . 1.40058664 |============================================================ b . 1.45866323 |=============================================================== c . 1.39991681 |============================================================ d . 1.39225551 |============================================================ Apache Spark TPC-H 3.5 Scale Factor: 1 - Q14 Seconds < Lower Is Better a . 1.83231536 |=============================================================== b . 1.78157055 |============================================================= c . 1.83836599 |=============================================================== d . 1.83219139 |=============================================================== Apache Spark TPC-H 3.5 Scale Factor: 1 - Q15 Seconds < Lower Is Better a . 1.99491587 |======================================================= b . 2.17460775 |============================================================ c . 2.14363774 |=========================================================== d . 2.27715731 |=============================================================== Apache Spark TPC-H 3.5 Scale Factor: 1 - Q16 Seconds < Lower Is Better a . 1.42731291 |========================================================== b . 1.43939853 |=========================================================== c . 1.49448760 |============================================================= d . 1.53928820 |=============================================================== Apache Spark TPC-H 3.5 Scale Factor: 1 - Q17 Seconds < Lower Is Better a . 2.66192222 |============================================================== b . 2.72264194 |=============================================================== c . 2.58527072 |============================================================ d . 2.57773050 |============================================================ Apache Spark TPC-H 3.5 Scale Factor: 1 - Q18 Seconds < Lower Is Better a . 4.00372249 |============================================================== b . 4.04589510 |=============================================================== c . 3.99324067 |============================================================== d . 3.86921136 |============================================================ Apache Spark TPC-H 3.5 Scale Factor: 1 - Q19 Seconds < Lower Is Better a . 1.02847038 |====================================================== b . 1.17394590 |============================================================= c . 1.20837911 |=============================================================== d . 0.99435820 |==================================================== Apache Spark TPC-H 3.5 Scale Factor: 1 - Q20 Seconds < Lower Is Better a . 2.75555176 |=============================================================== b . 2.58762598 |=========================================================== c . 2.75284823 |=============================================================== d . 2.74797662 |=============================================================== Apache Spark TPC-H 3.5 Scale Factor: 1 - Q21 Seconds < Lower Is Better a . 7.39505470 |============================================================ b . 7.70369244 |=============================================================== c . 7.46197589 |============================================================= d . 7.40773551 |============================================================= Apache Spark TPC-H 3.5 Scale Factor: 1 - Q22 Seconds < Lower Is Better a . 1.08502284 |=============================================================== b . 0.98886323 |========================================================= c . 1.05880324 |============================================================= d . 1.06113847 |============================================================== Apache Spark TPC-H 3.5 Scale Factor: 10 - Geometric Mean Of All Queries Seconds < Lower Is Better a . 8.17090667 |============================================================== b . 7.89399455 |============================================================ c . 8.11142575 |============================================================== d . 8.24567273 |=============================================================== Apache Spark TPC-H 3.5 Scale Factor: 10 - Q01 Seconds < Lower Is Better a . 8.67833742 |=============================================================== b . 8.70151806 |=============================================================== c . 8.66681448 |=============================================================== d . 8.73231697 |=============================================================== Apache Spark TPC-H 3.5 Scale Factor: 10 - Q02 Seconds < Lower Is Better a . 4.85099522 |============================================================== b . 4.88495016 |============================================================== c . 4.92897081 |=============================================================== d . 4.79437430 |============================================================= Apache Spark TPC-H 3.5 Scale Factor: 10 - Q03 Seconds < Lower Is Better a . 10.24 |=================================================================== b . 10.32 |==================================================================== c . 10.26 |==================================================================== d . 10.15 |=================================================================== Apache Spark TPC-H 3.5 Scale Factor: 10 - Q04 Seconds < Lower Is Better a . 8.63682970 |============================================================== b . 8.70809078 |=============================================================== c . 8.50457668 |============================================================== d . 8.33958340 |============================================================ Apache Spark TPC-H 3.5 Scale Factor: 10 - Q05 Seconds < Lower Is Better a . 12.48 |================================================================== b . 12.92 |==================================================================== c . 12.66 |=================================================================== d . 12.66 |=================================================================== Apache Spark TPC-H 3.5 Scale Factor: 10 - Q06 Seconds < Lower Is Better a . 4.40131362 |============================================================ b . 4.62162924 |=============================================================== c . 4.30036068 |=========================================================== d . 4.36710962 |============================================================ Apache Spark TPC-H 3.5 Scale Factor: 10 - Q07 Seconds < Lower Is Better a . 10.63 |=================================================================== b . 10.53 |=================================================================== c . 10.62 |=================================================================== d . 10.75 |==================================================================== Apache Spark TPC-H 3.5 Scale Factor: 10 - Q08 Seconds < Lower Is Better a . 10.99 |==================================================================== b . 10.67 |================================================================== c . 10.85 |=================================================================== d . 10.82 |=================================================================== Apache Spark TPC-H 3.5 Scale Factor: 10 - Q09 Seconds < Lower Is Better a . 14.90 |==================================================================== b . 14.18 |================================================================= c . 14.82 |==================================================================== d . 14.92 |==================================================================== Apache Spark TPC-H 3.5 Scale Factor: 10 - Q10 Seconds < Lower Is Better a . 10.27 |=================================================================== b . 10.18 |================================================================== c . 10.42 |==================================================================== d . 10.36 |==================================================================== Apache Spark TPC-H 3.5 Scale Factor: 10 - Q11 Seconds < Lower Is Better a . 4.54931625 |=============================================================== b . 4.26761341 |=========================================================== c . 4.38963429 |============================================================= d . 4.48057460 |============================================================== Apache Spark TPC-H 3.5 Scale Factor: 10 - Q12 Seconds < Lower Is Better a . 8.03465001 |============================================================== b . 8.10859299 |=============================================================== c . 8.17116038 |=============================================================== d . 8.09419775 |============================================================== Apache Spark TPC-H 3.5 Scale Factor: 10 - Q13 Seconds < Lower Is Better a . 4.45708307 |============================================================= b . 4.03698015 |======================================================= c . 4.09777006 |======================================================== d . 4.61340586 |=============================================================== Apache Spark TPC-H 3.5 Scale Factor: 10 - Q14 Seconds < Lower Is Better a . 5.79615275 |============================================================== b . 5.82840872 |=============================================================== c . 5.86177413 |=============================================================== d . 5.73522472 |============================================================== Apache Spark TPC-H 3.5 Scale Factor: 10 - Q15 Seconds < Lower Is Better a . 5.83709431 |=============================================================== b . 5.68089533 |============================================================= c . 5.81021436 |=============================================================== d . 5.85565599 |=============================================================== Apache Spark TPC-H 3.5 Scale Factor: 10 - Q16 Seconds < Lower Is Better a . 4.32050626 |=============================================================== b . 4.15591764 |============================================================ c . 4.17572975 |============================================================= d . 4.34601260 |=============================================================== Apache Spark TPC-H 3.5 Scale Factor: 10 - Q17 Seconds < Lower Is Better a . 12.45 |==================================================================== b . 12.44 |==================================================================== c . 12.42 |==================================================================== d . 12.40 |==================================================================== Apache Spark TPC-H 3.5 Scale Factor: 10 - Q18 Seconds < Lower Is Better a . 14.86 |=============================================================== b . 14.88 |=============================================================== c . 16.11 |==================================================================== d . 14.91 |=============================================================== Apache Spark TPC-H 3.5 Scale Factor: 10 - Q19 Seconds < Lower Is Better a . 7.21627538 |========================================================= b . 6.04250240 |=============================================== c . 6.54328155 |=================================================== d . 8.02795347 |=============================================================== Apache Spark TPC-H 3.5 Scale Factor: 10 - Q20 Seconds < Lower Is Better a . 8.84160964 |=============================================================== b . 8.87663841 |=============================================================== c . 8.91217136 |=============================================================== d . 8.82452583 |============================================================== Apache Spark TPC-H 3.5 Scale Factor: 10 - Q21 Seconds < Lower Is Better a . 29.31 |==================================================================== b . 28.80 |=================================================================== c . 29.03 |=================================================================== d . 28.91 |=================================================================== Apache Spark TPC-H 3.5 Scale Factor: 10 - Q22 Seconds < Lower Is Better a . 3.53586213 |============================================================ b . 3.40110779 |========================================================== c . 3.69966158 |=============================================================== d . 3.59682607 |============================================================= Apache Spark TPC-H 3.5 Scale Factor: 50 - Geometric Mean Of All Queries Seconds < Lower Is Better a . 27.98 |==================================================================== b . 27.79 |=================================================================== c . 28.11 |==================================================================== d . 28.12 |==================================================================== Apache Spark TPC-H 3.5 Scale Factor: 50 - Q01 Seconds < Lower Is Better a . 26.02 |==================================================================== b . 25.99 |==================================================================== c . 26.00 |==================================================================== d . 26.12 |==================================================================== Apache Spark TPC-H 3.5 Scale Factor: 50 - Q02 Seconds < Lower Is Better a . 10.48 |=================================================================== b . 10.45 |=================================================================== c . 10.56 |==================================================================== d . 10.44 |=================================================================== Apache Spark TPC-H 3.5 Scale Factor: 50 - Q03 Seconds < Lower Is Better a . 36.61 |=================================================================== b . 36.03 |================================================================== c . 37.06 |==================================================================== d . 35.99 |================================================================== Apache Spark TPC-H 3.5 Scale Factor: 50 - Q04 Seconds < Lower Is Better a . 30.58 |==================================================================== b . 30.27 |=================================================================== c . 30.22 |=================================================================== d . 30.56 |==================================================================== Apache Spark TPC-H 3.5 Scale Factor: 50 - Q05 Seconds < Lower Is Better a . 41.17 |================================================================= b . 43.25 |==================================================================== c . 39.70 |============================================================== d . 38.99 |============================================================= Apache Spark TPC-H 3.5 Scale Factor: 50 - Q06 Seconds < Lower Is Better a . 20.38 |==================================================================== b . 20.30 |==================================================================== c . 20.34 |==================================================================== d . 20.38 |==================================================================== Apache Spark TPC-H 3.5 Scale Factor: 50 - Q07 Seconds < Lower Is Better a . 35.63 |==================================================================== b . 35.34 |=================================================================== c . 35.46 |==================================================================== d . 35.40 |==================================================================== Apache Spark TPC-H 3.5 Scale Factor: 50 - Q08 Seconds < Lower Is Better a . 38.36 |==================================================================== b . 37.97 |=================================================================== c . 37.81 |=================================================================== d . 38.38 |==================================================================== Apache Spark TPC-H 3.5 Scale Factor: 50 - Q09 Seconds < Lower Is Better a . 46.90 |==================================================================== b . 46.32 |=================================================================== c . 47.11 |==================================================================== d . 47.00 |==================================================================== Apache Spark TPC-H 3.5 Scale Factor: 50 - Q10 Seconds < Lower Is Better a . 35.93 |================================================================== b . 36.27 |=================================================================== c . 36.82 |==================================================================== d . 35.93 |================================================================== Apache Spark TPC-H 3.5 Scale Factor: 50 - Q11 Seconds < Lower Is Better a . 9.06239065 |=============================================================== b . 8.69320774 |============================================================ c . 8.75260639 |============================================================= d . 9.01768939 |=============================================================== Apache Spark TPC-H 3.5 Scale Factor: 50 - Q12 Seconds < Lower Is Better a . 30.43 |=================================================================== b . 30.41 |=================================================================== c . 30.89 |==================================================================== d . 30.04 |================================================================== Apache Spark TPC-H 3.5 Scale Factor: 50 - Q13 Seconds < Lower Is Better a . 12.34 |============================================================= b . 12.13 |============================================================ c . 13.14 |================================================================= d . 13.68 |==================================================================== Apache Spark TPC-H 3.5 Scale Factor: 50 - Q14 Seconds < Lower Is Better a . 25.49 |==================================================================== b . 25.40 |==================================================================== c . 25.46 |==================================================================== d . 25.42 |==================================================================== Apache Spark TPC-H 3.5 Scale Factor: 50 - Q15 Seconds < Lower Is Better a . 23.05 |==================================================================== b . 22.89 |==================================================================== c . 22.95 |==================================================================== d . 23.00 |==================================================================== Apache Spark TPC-H 3.5 Scale Factor: 50 - Q16 Seconds < Lower Is Better a . 10.59 |================================================================== b . 10.53 |================================================================== c . 10.82 |=================================================================== d . 10.91 |==================================================================== Apache Spark TPC-H 3.5 Scale Factor: 50 - Q17 Seconds < Lower Is Better a . 55.61 |==================================================================== b . 55.28 |==================================================================== c . 55.33 |==================================================================== d . 55.42 |==================================================================== Apache Spark TPC-H 3.5 Scale Factor: 50 - Q18 Seconds < Lower Is Better a . 62.51 |==================================================================== b . 61.96 |=================================================================== c . 62.30 |==================================================================== d . 62.73 |==================================================================== Apache Spark TPC-H 3.5 Scale Factor: 50 - Q19 Seconds < Lower Is Better a . 24.72 |=================================================================== b . 25.08 |==================================================================== c . 24.89 |=================================================================== d . 25.03 |==================================================================== Apache Spark TPC-H 3.5 Scale Factor: 50 - Q20 Seconds < Lower Is Better a . 30.71 |==================================================================== b . 30.54 |=================================================================== c . 30.79 |==================================================================== d . 30.52 |=================================================================== Apache Spark TPC-H 3.5 Scale Factor: 50 - Q21 Seconds < Lower Is Better a . 130.87 |=================================================================== b . 131.46 |=================================================================== c . 131.42 |=================================================================== d . 131.01 |=================================================================== Apache Spark TPC-H 3.5 Scale Factor: 50 - Q22 Seconds < Lower Is Better a . 10.67 |==================================================================== b . 10.53 |=================================================================== c . 10.64 |==================================================================== d . 10.27 |================================================================= Apache Spark TPC-H 3.5 Scale Factor: 100 Seconds < Lower Is Better Neural Magic DeepSparse 1.6 Model: NLP Document Classification, oBERT base uncased on IMDB - Scenario: Asynchronous Multi-Stream items/sec > Higher Is Better a . 37.93 |================================================================= b . 39.38 |==================================================================== c . 37.78 |================================================================= d . 38.72 |=================================================================== Neural Magic DeepSparse 1.6 Model: NLP Document Classification, oBERT base uncased on IMDB - Scenario: Asynchronous Multi-Stream ms/batch < Lower Is Better a . 836.92 |=================================================================== b . 806.39 |================================================================ c . 841.34 |=================================================================== d . 820.12 |================================================================= Neural Magic DeepSparse 1.6 Model: NLP Document Classification, oBERT base uncased on IMDB - Scenario: Synchronous Single-Stream items/sec > Higher Is Better a . 13.90 |================================================================== b . 14.40 |==================================================================== c . 13.88 |================================================================== d . 13.90 |================================================================== Neural Magic DeepSparse 1.6 Model: NLP Document Classification, oBERT base uncased on IMDB - Scenario: Synchronous Single-Stream ms/batch < Lower Is Better a . 71.94 |==================================================================== b . 69.42 |================================================================== c . 72.03 |==================================================================== d . 71.92 |==================================================================== Neural Magic DeepSparse 1.6 Model: NLP Text Classification, BERT base uncased SST2, Sparse INT8 - Scenario: Asynchronous Multi-Stream items/sec > Higher Is Better a . 980.45 |================================================================== b . 993.60 |=================================================================== c . 988.49 |=================================================================== d . 989.06 |=================================================================== Neural Magic DeepSparse 1.6 Model: NLP Text Classification, BERT base uncased SST2, Sparse INT8 - Scenario: Asynchronous Multi-Stream ms/batch < Lower Is Better a . 32.60 |==================================================================== b . 32.16 |=================================================================== c . 32.34 |=================================================================== d . 32.31 |=================================================================== Neural Magic DeepSparse 1.6 Model: NLP Text Classification, BERT base uncased SST2, Sparse INT8 - Scenario: Synchronous Single-Stream items/sec > Higher Is Better a . 129.70 |================================================================= b . 131.17 |================================================================= c . 132.10 |================================================================== d . 134.40 |=================================================================== Neural Magic DeepSparse 1.6 Model: NLP Text Classification, BERT base uncased SST2, Sparse INT8 - Scenario: Synchronous Single-Stream ms/batch < Lower Is Better a . 7.7066 |=================================================================== b . 7.6192 |================================================================== c . 7.5689 |================================================================== d . 7.4402 |================================================================= Neural Magic DeepSparse 1.6 Model: ResNet-50, Baseline - Scenario: Asynchronous Multi-Stream items/sec > Higher Is Better a . 441.44 |================================================================== b . 450.81 |=================================================================== c . 445.16 |================================================================== d . 441.39 |================================================================== Neural Magic DeepSparse 1.6 Model: ResNet-50, Baseline - Scenario: Asynchronous Multi-Stream ms/batch < Lower Is Better a . 72.42 |==================================================================== b . 70.94 |=================================================================== c . 71.81 |=================================================================== d . 72.46 |==================================================================== Neural Magic DeepSparse 1.6 Model: ResNet-50, Baseline - Scenario: Synchronous Single-Stream items/sec > Higher Is Better a . 103.56 |================================================================= b . 106.93 |=================================================================== c . 106.03 |================================================================== d . 103.98 |================================================================= Neural Magic DeepSparse 1.6 Model: ResNet-50, Baseline - Scenario: Synchronous Single-Stream ms/batch < Lower Is Better a . 9.6462 |=================================================================== b . 9.3424 |================================================================= c . 9.4215 |================================================================= d . 9.6084 |=================================================================== Neural Magic DeepSparse 1.6 Model: ResNet-50, Sparse INT8 - Scenario: Asynchronous Multi-Stream items/sec > Higher Is Better a . 2555.42 |================================================================ b . 2635.57 |================================================================== c . 2548.38 |================================================================ d . 2543.11 |================================================================ Neural Magic DeepSparse 1.6 Model: ResNet-50, Sparse INT8 - Scenario: Asynchronous Multi-Stream ms/batch < Lower Is Better a . 12.49 |==================================================================== b . 12.11 |================================================================== c . 12.52 |==================================================================== d . 12.55 |==================================================================== Neural Magic DeepSparse 1.6 Model: ResNet-50, Sparse INT8 - Scenario: Synchronous Single-Stream items/sec > Higher Is Better a . 627.75 |=================================================================== b . 599.19 |================================================================ c . 591.26 |=============================================================== d . 620.24 |================================================================== Neural Magic DeepSparse 1.6 Model: ResNet-50, Sparse INT8 - Scenario: Synchronous Single-Stream ms/batch < Lower Is Better a . 1.5897 |=============================================================== b . 1.6656 |================================================================== c . 1.6887 |=================================================================== d . 1.6090 |================================================================ Neural Magic DeepSparse 1.6 Model: CV Detection, YOLOv5s COCO - Scenario: Asynchronous Multi-Stream items/sec > Higher Is Better a . 201.91 |================================================================= b . 206.96 |=================================================================== c . 199.22 |================================================================ d . 201.94 |================================================================= Neural Magic DeepSparse 1.6 Model: CV Detection, YOLOv5s COCO - Scenario: Asynchronous Multi-Stream ms/batch < Lower Is Better a . 158.15 |================================================================== b . 154.30 |================================================================= c . 160.28 |=================================================================== d . 158.11 |================================================================== Neural Magic DeepSparse 1.6 Model: CV Detection, YOLOv5s COCO - Scenario: Synchronous Single-Stream items/sec > Higher Is Better a . 86.38 |=================================================================== b . 88.17 |==================================================================== c . 87.93 |==================================================================== d . 87.47 |=================================================================== Neural Magic DeepSparse 1.6 Model: CV Detection, YOLOv5s COCO - Scenario: Synchronous Single-Stream ms/batch < Lower Is Better a . 11.56 |==================================================================== b . 11.33 |=================================================================== c . 11.36 |=================================================================== d . 11.42 |=================================================================== Neural Magic DeepSparse 1.6 Model: BERT-Large, NLP Question Answering - Scenario: Asynchronous Multi-Stream items/sec > Higher Is Better a . 49.22 |==================================================================== b . 49.33 |==================================================================== c . 49.08 |==================================================================== d . 49.17 |==================================================================== Neural Magic DeepSparse 1.6 Model: BERT-Large, NLP Question Answering - Scenario: Asynchronous Multi-Stream ms/batch < Lower Is Better a . 649.93 |=================================================================== b . 648.52 |=================================================================== c . 649.76 |=================================================================== d . 650.69 |=================================================================== Neural Magic DeepSparse 1.6 Model: BERT-Large, NLP Question Answering - Scenario: Synchronous Single-Stream items/sec > Higher Is Better a . 11.31 |================================================================= b . 11.61 |=================================================================== c . 11.84 |==================================================================== d . 11.74 |=================================================================== Neural Magic DeepSparse 1.6 Model: BERT-Large, NLP Question Answering - Scenario: Synchronous Single-Stream ms/batch < Lower Is Better a . 88.47 |==================================================================== b . 86.15 |================================================================== c . 84.46 |================================================================= d . 85.16 |================================================================= Neural Magic DeepSparse 1.6 Model: CV Classification, ResNet-50 ImageNet - Scenario: Asynchronous Multi-Stream items/sec > Higher Is Better a . 449.31 |=================================================================== b . 448.76 |=================================================================== c . 452.04 |=================================================================== d . 450.06 |=================================================================== Neural Magic DeepSparse 1.6 Model: CV Classification, ResNet-50 ImageNet - Scenario: Asynchronous Multi-Stream ms/batch < Lower Is Better a . 71.17 |==================================================================== b . 71.27 |==================================================================== c . 70.74 |=================================================================== d . 71.05 |==================================================================== Neural Magic DeepSparse 1.6 Model: CV Classification, ResNet-50 ImageNet - Scenario: Synchronous Single-Stream items/sec > Higher Is Better a . 106.44 |=================================================================== b . 106.54 |=================================================================== c . 107.10 |=================================================================== d . 104.29 |================================================================= Neural Magic DeepSparse 1.6 Model: CV Classification, ResNet-50 ImageNet - Scenario: Synchronous Single-Stream ms/batch < Lower Is Better a . 9.3853 |================================================================== b . 9.3770 |================================================================== c . 9.3277 |================================================================= d . 9.5802 |=================================================================== Neural Magic DeepSparse 1.6 Model: CV Detection, YOLOv5s COCO, Sparse INT8 - Scenario: Asynchronous Multi-Stream items/sec > Higher Is Better a . 215.83 |=================================================================== b . 212.76 |================================================================== c . 216.39 |=================================================================== d . 215.29 |=================================================================== Neural Magic DeepSparse 1.6 Model: CV Detection, YOLOv5s COCO, Sparse INT8 - Scenario: Asynchronous Multi-Stream ms/batch < Lower Is Better a . 147.96 |================================================================== b . 150.23 |=================================================================== c . 147.49 |================================================================== d . 148.28 |================================================================== Neural Magic DeepSparse 1.6 Model: CV Detection, YOLOv5s COCO, Sparse INT8 - Scenario: Synchronous Single-Stream items/sec > Higher Is Better a . 91.14 |==================================================================== b . 89.80 |=================================================================== c . 91.16 |==================================================================== d . 89.59 |=================================================================== Neural Magic DeepSparse 1.6 Model: CV Detection, YOLOv5s COCO, Sparse INT8 - Scenario: Synchronous Single-Stream ms/batch < Lower Is Better a . 10.96 |=================================================================== b . 11.13 |==================================================================== c . 10.96 |=================================================================== d . 11.15 |==================================================================== Neural Magic DeepSparse 1.6 Model: NLP Text Classification, DistilBERT mnli - Scenario: Asynchronous Multi-Stream items/sec > Higher Is Better a . 336.19 |=================================================================== b . 337.08 |=================================================================== c . 337.78 |=================================================================== d . 335.10 |================================================================== Neural Magic DeepSparse 1.6 Model: NLP Text Classification, DistilBERT mnli - Scenario: Asynchronous Multi-Stream ms/batch < Lower Is Better a . 95.02 |==================================================================== b . 94.88 |==================================================================== c . 94.68 |=================================================================== d . 95.45 |==================================================================== Neural Magic DeepSparse 1.6 Model: NLP Text Classification, DistilBERT mnli - Scenario: Synchronous Single-Stream items/sec > Higher Is Better a . 56.79 |================================================================= b . 59.01 |==================================================================== c . 57.18 |================================================================== d . 56.86 |================================================================== Neural Magic DeepSparse 1.6 Model: NLP Text Classification, DistilBERT mnli - Scenario: Synchronous Single-Stream ms/batch < Lower Is Better a . 17.60 |==================================================================== b . 16.94 |================================================================= c . 17.48 |==================================================================== d . 17.58 |==================================================================== Neural Magic DeepSparse 1.6 Model: CV Segmentation, 90% Pruned YOLACT Pruned - Scenario: Asynchronous Multi-Stream items/sec > Higher Is Better a . 43.19 |=================================================================== b . 43.84 |==================================================================== c . 43.05 |=================================================================== d . 43.06 |=================================================================== Neural Magic DeepSparse 1.6 Model: CV Segmentation, 90% Pruned YOLACT Pruned - Scenario: Asynchronous Multi-Stream ms/batch < Lower Is Better a . 736.77 |=================================================================== b . 726.25 |================================================================== c . 737.98 |=================================================================== d . 737.85 |=================================================================== Neural Magic DeepSparse 1.6 Model: CV Segmentation, 90% Pruned YOLACT Pruned - Scenario: Synchronous Single-Stream items/sec > Higher Is Better a . 17.07 |================================================================= b . 17.99 |==================================================================== c . 16.99 |================================================================ d . 17.09 |================================================================= Neural Magic DeepSparse 1.6 Model: CV Segmentation, 90% Pruned YOLACT Pruned - Scenario: Synchronous Single-Stream ms/batch < Lower Is Better a . 58.55 |==================================================================== b . 55.56 |================================================================ c . 58.85 |==================================================================== d . 58.50 |==================================================================== Neural Magic DeepSparse 1.6 Model: BERT-Large, NLP Question Answering, Sparse INT8 - Scenario: Asynchronous Multi-Stream items/sec > Higher Is Better a . 503.41 |================================================================== b . 508.59 |=================================================================== c . 506.13 |=================================================================== d . 504.84 |=================================================================== Neural Magic DeepSparse 1.6 Model: BERT-Large, NLP Question Answering, Sparse INT8 - Scenario: Asynchronous Multi-Stream ms/batch < Lower Is Better a . 63.48 |==================================================================== b . 62.81 |=================================================================== c . 63.13 |==================================================================== d . 63.30 |==================================================================== Neural Magic DeepSparse 1.6 Model: BERT-Large, NLP Question Answering, Sparse INT8 - Scenario: Synchronous Single-Stream items/sec > Higher Is Better a . 64.71 |==================================================================== b . 64.40 |==================================================================== c . 64.12 |=================================================================== d . 64.08 |=================================================================== Neural Magic DeepSparse 1.6 Model: BERT-Large, NLP Question Answering, Sparse INT8 - Scenario: Synchronous Single-Stream ms/batch < Lower Is Better a . 15.45 |=================================================================== b . 15.52 |==================================================================== c . 15.59 |==================================================================== d . 15.60 |==================================================================== Neural Magic DeepSparse 1.6 Model: NLP Token Classification, BERT base uncased conll2003 - Scenario: Asynchronous Multi-Stream items/sec > Higher Is Better a . 37.74 |==================================================================== b . 37.95 |==================================================================== c . 37.94 |==================================================================== d . 37.45 |=================================================================== Neural Magic DeepSparse 1.6 Model: NLP Token Classification, BERT base uncased conll2003 - Scenario: Asynchronous Multi-Stream ms/batch < Lower Is Better a . 841.43 |=================================================================== b . 837.46 |================================================================== c . 837.14 |================================================================== d . 845.44 |=================================================================== Neural Magic DeepSparse 1.6 Model: NLP Token Classification, BERT base uncased conll2003 - Scenario: Synchronous Single-Stream items/sec > Higher Is Better a . 14.08 |=================================================================== b . 14.31 |==================================================================== c . 14.10 |=================================================================== d . 14.00 |=================================================================== Neural Magic DeepSparse 1.6 Model: NLP Token Classification, BERT base uncased conll2003 - Scenario: Synchronous Single-Stream ms/batch < Lower Is Better a . 71.04 |==================================================================== b . 69.88 |=================================================================== c . 70.90 |==================================================================== d . 71.42 |====================================================================