new tests thurs AMD Ryzen 9 7950X 16-Core testing with a ASUS ROG CROSSHAIR X670E HERO (0703 BIOS) and AMD Radeon RX 6800/6800 XT / 6900 on Ubuntu 22.10 via the Phoronix Test Suite. A: Processor: AMD Ryzen 9 7950X 16-Core @ 5.88GHz (16 Cores / 32 Threads), Motherboard: ASUS ROG CROSSHAIR X670E HERO (0703 BIOS), Chipset: AMD Device 14d8, Memory: 32GB, Disk: 1000GB Sabrent Rocket 4.0 Plus + 32GB Flash Drive, Graphics: AMD Radeon RX 6800/6800 XT / 6900 (2475/1000MHz), Audio: AMD Navi 21/23, Monitor: ASUS MG28U, Network: Intel I225-V + Intel Wi-Fi 6 AX210/AX211/AX411 OS: Ubuntu 22.10, Kernel: 5.19.0-19-generic (x86_64), Desktop: GNOME Shell 43.0, Display Server: X Server + Wayland, OpenGL: 4.6 Mesa 22.2.0 (LLVM 15.0.1 DRM 3.47), Vulkan: 1.3.224, Compiler: GCC 12.2.0, File-System: ext4, Screen Resolution: 3840x2160 B: Processor: AMD Ryzen 9 7950X 16-Core @ 5.88GHz (16 Cores / 32 Threads), Motherboard: ASUS ROG CROSSHAIR X670E HERO (0703 BIOS), Chipset: AMD Device 14d8, Memory: 32GB, Disk: 1000GB Sabrent Rocket 4.0 Plus + 32GB Flash Drive, Graphics: AMD Radeon RX 6800/6800 XT / 6900 (2475/1000MHz), Audio: AMD Navi 21/23, Monitor: ASUS MG28U, Network: Intel I225-V + Intel Wi-Fi 6 AX210/AX211/AX411 OS: Ubuntu 22.10, Kernel: 5.19.0-19-generic (x86_64), Desktop: GNOME Shell 43.0, Display Server: X Server + Wayland, OpenGL: 4.6 Mesa 22.2.0 (LLVM 15.0.1 DRM 3.47), Vulkan: 1.3.224, Compiler: GCC 12.2.0, File-System: ext4, Screen Resolution: 3840x2160 C: Processor: AMD Ryzen 9 7950X 16-Core @ 5.88GHz (16 Cores / 32 Threads), Motherboard: ASUS ROG CROSSHAIR X670E HERO (0703 BIOS), Chipset: AMD Device 14d8, Memory: 32GB, Disk: 1000GB Sabrent Rocket 4.0 Plus + 32GB Flash Drive, Graphics: AMD Radeon RX 6800/6800 XT / 6900 (2475/1000MHz), Audio: AMD Navi 21/23, Monitor: ASUS MG28U, Network: Intel I225-V + Intel Wi-Fi 6 AX210/AX211/AX411 OS: Ubuntu 22.10, Kernel: 5.19.0-19-generic (x86_64), Desktop: GNOME Shell 43.0, Display Server: X Server + Wayland, OpenGL: 4.6 Mesa 22.2.0 (LLVM 15.0.1 DRM 3.47), Vulkan: 1.3.224, Compiler: GCC 12.2.0, File-System: ext4, Screen Resolution: 3840x2160 Neural Magic DeepSparse 1.1 Model: NLP Document Classification, oBERT base uncased on IMDB - Scenario: Asynchronous Multi-Stream items/sec > Higher Is Better A . 12.45 |==================================================================== B . 12.41 |==================================================================== C . 12.42 |==================================================================== Neural Magic DeepSparse 1.1 Model: NLP Document Classification, oBERT base uncased on IMDB - Scenario: Synchronous Single-Stream items/sec > Higher Is Better A . 11.51 |==================================================================== B . 11.55 |==================================================================== C . 11.56 |==================================================================== Neural Magic DeepSparse 1.1 Model: NLP Question Answering, BERT base uncased SQuaD 12layer Pruned90 - Scenario: Asynchronous Multi-Stream items/sec > Higher Is Better A . 98.07 |==================================================================== B . 98.56 |==================================================================== C . 97.62 |=================================================================== Neural Magic DeepSparse 1.1 Model: NLP Question Answering, BERT base uncased SQuaD 12layer Pruned90 - Scenario: Synchronous Single-Stream items/sec > Higher Is Better A . 63.78 |==================================================================== B . 63.89 |==================================================================== C . 63.62 |==================================================================== Neural Magic DeepSparse 1.1 Model: CV Detection,YOLOv5s COCO - Scenario: Asynchronous Multi-Stream items/sec > Higher Is Better A . 108.86 |=================================================================== B . 108.44 |=================================================================== C . 108.43 |=================================================================== Neural Magic DeepSparse 1.1 Model: CV Detection,YOLOv5s COCO - Scenario: Synchronous Single-Stream items/sec > Higher Is Better A . 94.80 |==================================================================== B . 95.15 |==================================================================== C . 94.99 |==================================================================== Neural Magic DeepSparse 1.1 Model: CV Classification, ResNet-50 ImageNet - Scenario: Asynchronous Multi-Stream items/sec > Higher Is Better A . 292.34 |=================================================================== B . 294.14 |=================================================================== C . 292.46 |=================================================================== Neural Magic DeepSparse 1.1 Model: CV Classification, ResNet-50 ImageNet - Scenario: Synchronous Single-Stream items/sec > Higher Is Better A . 199.45 |=================================================================== B . 199.75 |=================================================================== C . 200.47 |=================================================================== Neural Magic DeepSparse 1.1 Model: NLP Text Classification, DistilBERT mnli - Scenario: Asynchronous Multi-Stream items/sec > Higher Is Better A . 182.48 |=================================================================== B . 183.34 |=================================================================== C . 183.76 |=================================================================== Neural Magic DeepSparse 1.1 Model: NLP Text Classification, DistilBERT mnli - Scenario: Synchronous Single-Stream items/sec > Higher Is Better A . 110.66 |================================================================== B . 112.69 |=================================================================== C . 112.67 |=================================================================== Neural Magic DeepSparse 1.1 Model: NLP Text Classification, BERT base uncased SST2 - Scenario: Asynchronous Multi-Stream items/sec > Higher Is Better A . 95.20 |=================================================================== B . 95.57 |==================================================================== C . 96.06 |==================================================================== Neural Magic DeepSparse 1.1 Model: NLP Text Classification, BERT base uncased SST2 - Scenario: Synchronous Single-Stream items/sec > Higher Is Better A . 59.36 |==================================================================== B . 59.03 |==================================================================== C . 59.41 |==================================================================== Neural Magic DeepSparse 1.1 Model: NLP Token Classification, BERT base uncased conll2003 - Scenario: Asynchronous Multi-Stream items/sec > Higher Is Better A . 12.38 |==================================================================== B . 12.42 |==================================================================== C . 12.42 |==================================================================== Neural Magic DeepSparse 1.1 Model: NLP Token Classification, BERT base uncased conll2003 - Scenario: Synchronous Single-Stream items/sec > Higher Is Better A . 11.50 |==================================================================== B . 11.53 |==================================================================== C . 11.51 |==================================================================== PostgreSQL 15 Scaling Factor: 1 - Clients: 1 - Mode: Read Only TPS > Higher Is Better A . 62767 |=================================================================== B . 63854 |==================================================================== C . 64067 |==================================================================== PostgreSQL 15 Scaling Factor: 1 - Clients: 1 - Mode: Read Write TPS > Higher Is Better A . 2023 |===================================================================== B . 1840 |=============================================================== C . 1780 |============================================================= PostgreSQL 15 Scaling Factor: 1 - Clients: 50 - Mode: Read Only TPS > Higher Is Better A . 920485 |=================================================================== B . 920816 |=================================================================== C . 923688 |=================================================================== PostgreSQL 15 Scaling Factor: 1 - Clients: 100 - Mode: Read Only TPS > Higher Is Better A . 912904 |=================================================================== B . 906853 |================================================================== C . 917912 |=================================================================== PostgreSQL 15 Scaling Factor: 1 - Clients: 50 - Mode: Read Write TPS > Higher Is Better A . 1948 |===================================================================== B . 1946 |===================================================================== C . 1865 |================================================================== PostgreSQL 15 Scaling Factor: 100 - Clients: 1 - Mode: Read Only TPS > Higher Is Better A . 59058 |=================================================================== B . 59808 |==================================================================== C . 55784 |=============================================================== PostgreSQL 15 Scaling Factor: 1 - Clients: 100 - Mode: Read Write TPS > Higher Is Better A . 1783 |===================================================================== B . 1774 |===================================================================== C . 1767 |==================================================================== PostgreSQL 15 Scaling Factor: 100 - Clients: 1 - Mode: Read Write TPS > Higher Is Better A . 1908 |==================================================================== B . 1938 |===================================================================== C . 1770 |=============================================================== PostgreSQL 15 Scaling Factor: 100 - Clients: 50 - Mode: Read Only TPS > Higher Is Better A . 883539 |=================================================================== B . 888448 |=================================================================== C . 881737 |================================================================== PostgreSQL 15 Scaling Factor: 100 - Clients: 100 - Mode: Read Only TPS > Higher Is Better A . 875399 |=================================================================== B . 876999 |=================================================================== C . 881660 |=================================================================== PostgreSQL 15 Scaling Factor: 100 - Clients: 50 - Mode: Read Write TPS > Higher Is Better A . 21799 |==================================================================== B . 13634 |=========================================== C . 13307 |========================================== PostgreSQL 15 Scaling Factor: 100 - Clients: 100 - Mode: Read Write TPS > Higher Is Better A . 16107 |============================================================ B . 16298 |============================================================= C . 18136 |==================================================================== PostgreSQL 15 Scaling Factor: 1 - Clients: 1 - Mode: Read Only - Average Latency ms < Lower Is Better A . 0.016 |==================================================================== B . 0.016 |==================================================================== C . 0.016 |==================================================================== PostgreSQL 15 Scaling Factor: 1 - Clients: 1 - Mode: Read Write - Average Latency ms < Lower Is Better A . 0.494 |============================================================ B . 0.544 |================================================================== C . 0.562 |==================================================================== PostgreSQL 15 Scaling Factor: 1 - Clients: 50 - Mode: Read Only - Average Latency ms < Lower Is Better A . 0.054 |==================================================================== B . 0.054 |==================================================================== C . 0.054 |==================================================================== PostgreSQL 15 Scaling Factor: 1 - Clients: 100 - Mode: Read Only - Average Latency ms < Lower Is Better A . 0.110 |==================================================================== B . 0.110 |==================================================================== C . 0.109 |=================================================================== PostgreSQL 15 Scaling Factor: 1 - Clients: 50 - Mode: Read Write - Average Latency ms < Lower Is Better A . 25.67 |================================================================= B . 25.70 |================================================================= C . 26.81 |==================================================================== PostgreSQL 15 Scaling Factor: 100 - Clients: 1 - Mode: Read Only - Average Latency ms < Lower Is Better A . 0.017 |================================================================ B . 0.017 |================================================================ C . 0.018 |==================================================================== PostgreSQL 15 Scaling Factor: 1 - Clients: 100 - Mode: Read Write - Average Latency ms < Lower Is Better A . 56.09 |=================================================================== B . 56.37 |==================================================================== C . 56.58 |==================================================================== PostgreSQL 15 Scaling Factor: 100 - Clients: 1 - Mode: Read Write - Average Latency ms < Lower Is Better A . 0.524 |=============================================================== B . 0.516 |============================================================== C . 0.565 |==================================================================== PostgreSQL 15 Scaling Factor: 100 - Clients: 50 - Mode: Read Only - Average Latency ms < Lower Is Better A . 0.057 |==================================================================== B . 0.056 |=================================================================== C . 0.057 |==================================================================== PostgreSQL 15 Scaling Factor: 100 - Clients: 100 - Mode: Read Only - Average Latency ms < Lower Is Better A . 0.114 |==================================================================== B . 0.114 |==================================================================== C . 0.113 |=================================================================== PostgreSQL 15 Scaling Factor: 100 - Clients: 50 - Mode: Read Write - Average Latency ms < Lower Is Better A . 2.399 |=========================================== B . 3.667 |================================================================== C . 3.758 |==================================================================== PostgreSQL 15 Scaling Factor: 100 - Clients: 100 - Mode: Read Write - Average Latency ms < Lower Is Better A . 6.223 |==================================================================== B . 6.136 |=================================================================== C . 5.514 |============================================================ Neural Magic DeepSparse 1.1 Model: NLP Document Classification, oBERT base uncased on IMDB - Scenario: Asynchronous Multi-Stream ms/batch < Lower Is Better A . 638.90 |=================================================================== B . 638.18 |=================================================================== C . 639.75 |=================================================================== Neural Magic DeepSparse 1.1 Model: NLP Document Classification, oBERT base uncased on IMDB - Scenario: Synchronous Single-Stream ms/batch < Lower Is Better A . 86.86 |==================================================================== B . 86.55 |==================================================================== C . 86.53 |==================================================================== Neural Magic DeepSparse 1.1 Model: NLP Question Answering, BERT base uncased SQuaD 12layer Pruned90 - Scenario: Asynchronous Multi-Stream ms/batch < Lower Is Better A . 81.55 |==================================================================== B . 81.14 |=================================================================== C . 81.92 |==================================================================== Neural Magic DeepSparse 1.1 Model: NLP Question Answering, BERT base uncased SQuaD 12layer Pruned90 - Scenario: Synchronous Single-Stream ms/batch < Lower Is Better A . 15.67 |==================================================================== B . 15.65 |==================================================================== C . 15.71 |==================================================================== Neural Magic DeepSparse 1.1 Model: CV Detection,YOLOv5s COCO - Scenario: Asynchronous Multi-Stream ms/batch < Lower Is Better A . 73.46 |==================================================================== B . 73.76 |==================================================================== C . 73.76 |==================================================================== Neural Magic DeepSparse 1.1 Model: CV Detection,YOLOv5s COCO - Scenario: Synchronous Single-Stream ms/batch < Lower Is Better A . 10.54 |==================================================================== B . 10.50 |==================================================================== C . 10.52 |==================================================================== Neural Magic DeepSparse 1.1 Model: CV Classification, ResNet-50 ImageNet - Scenario: Asynchronous Multi-Stream ms/batch < Lower Is Better A . 27.35 |==================================================================== B . 27.19 |==================================================================== C . 27.34 |==================================================================== Neural Magic DeepSparse 1.1 Model: CV Classification, ResNet-50 ImageNet - Scenario: Synchronous Single-Stream ms/batch < Lower Is Better A . 5.0093 |=================================================================== B . 5.0011 |=================================================================== C . 4.9834 |=================================================================== Neural Magic DeepSparse 1.1 Model: NLP Text Classification, DistilBERT mnli - Scenario: Asynchronous Multi-Stream ms/batch < Lower Is Better A . 43.83 |==================================================================== B . 43.62 |==================================================================== C . 43.52 |==================================================================== Neural Magic DeepSparse 1.1 Model: NLP Text Classification, DistilBERT mnli - Scenario: Synchronous Single-Stream ms/batch < Lower Is Better A . 9.0330 |=================================================================== B . 8.8697 |================================================================== C . 8.8714 |================================================================== Neural Magic DeepSparse 1.1 Model: NLP Text Classification, BERT base uncased SST2 - Scenario: Asynchronous Multi-Stream ms/batch < Lower Is Better A . 84.03 |==================================================================== B . 83.70 |==================================================================== C . 83.27 |=================================================================== Neural Magic DeepSparse 1.1 Model: NLP Text Classification, BERT base uncased SST2 - Scenario: Synchronous Single-Stream ms/batch < Lower Is Better A . 16.84 |==================================================================== B . 16.94 |==================================================================== C . 16.83 |==================================================================== Neural Magic DeepSparse 1.1 Model: NLP Token Classification, BERT base uncased conll2003 - Scenario: Asynchronous Multi-Stream ms/batch < Lower Is Better A . 642.85 |=================================================================== B . 641.43 |=================================================================== C . 640.53 |=================================================================== Neural Magic DeepSparse 1.1 Model: NLP Token Classification, BERT base uncased conll2003 - Scenario: Synchronous Single-Stream ms/batch < Lower Is Better A . 86.96 |==================================================================== B . 86.72 |==================================================================== C . 86.85 |====================================================================