icl deepsparse openradioss Intel Core i7-1065G7 testing with a Dell 06CDVY (1.0.9 BIOS) and Intel Iris Plus ICL GT2 16GB on Ubuntu 22.04 via the Phoronix Test Suite. A: Processor: Intel Core i7-1065G7 @ 3.90GHz (4 Cores / 8 Threads), Motherboard: Dell 06CDVY (1.0.9 BIOS), Chipset: Intel Ice Lake-LP DRAM, Memory: 16GB, Disk: Toshiba KBG40ZPZ512G NVMe 512GB, Graphics: Intel Iris Plus ICL GT2 16GB (1100MHz), Audio: Realtek ALC289, Network: Intel Ice Lake-LP PCH CNVi WiFi OS: Ubuntu 22.04, Kernel: 5.18.8-051808-generic (x86_64), Desktop: GNOME Shell 42.2, Display Server: X Server + Wayland, OpenGL: 4.6 Mesa 22.0.1, Vulkan: 1.3.204, Compiler: GCC 11.2.0, File-System: ext4, Screen Resolution: 1920x1200 B: Processor: Intel Core i7-1065G7 @ 3.90GHz (4 Cores / 8 Threads), Motherboard: Dell 06CDVY (1.0.9 BIOS), Chipset: Intel Ice Lake-LP DRAM, Memory: 16GB, Disk: Toshiba KBG40ZPZ512G NVMe 512GB, Graphics: Intel Iris Plus ICL GT2 16GB (1100MHz), Audio: Realtek ALC289, Network: Intel Ice Lake-LP PCH CNVi WiFi OS: Ubuntu 22.04, Kernel: 5.18.8-051808-generic (x86_64), Desktop: GNOME Shell 42.2, Display Server: X Server + Wayland, OpenGL: 4.6 Mesa 22.0.1, Vulkan: 1.3.204, Compiler: GCC 11.2.0, File-System: ext4, Screen Resolution: 1920x1200 C: Processor: Intel Core i7-1065G7 @ 3.90GHz (4 Cores / 8 Threads), Motherboard: Dell 06CDVY (1.0.9 BIOS), Chipset: Intel Ice Lake-LP DRAM, Memory: 16GB, Disk: Toshiba KBG40ZPZ512G NVMe 512GB, Graphics: Intel Iris Plus ICL GT2 16GB (1100MHz), Audio: Realtek ALC289, Network: Intel Ice Lake-LP PCH CNVi WiFi OS: Ubuntu 22.04, Kernel: 5.18.8-051808-generic (x86_64), Desktop: GNOME Shell 42.2, Display Server: X Server + Wayland, OpenGL: 4.6 Mesa 22.0.1, Vulkan: 1.3.204, Compiler: GCC 11.2.0, File-System: ext4, Screen Resolution: 1920x1200 Neural Magic DeepSparse 1.1 Model: NLP Document Classification, oBERT base uncased on IMDB - Scenario: Asynchronous Multi-Stream items/sec > Higher Is Better A . 1.9793 |=================================================================== B . 1.9626 |================================================================== C . 1.9124 |================================================================= Neural Magic DeepSparse 1.1 Model: NLP Document Classification, oBERT base uncased on IMDB - Scenario: Asynchronous Multi-Stream ms/batch < Lower Is Better A . 1009.53 |================================================================= B . 1016.67 |================================================================= C . 1032.01 |================================================================== Neural Magic DeepSparse 1.1 Model: NLP Document Classification, oBERT base uncased on IMDB - Scenario: Synchronous Single-Stream items/sec > Higher Is Better A . 1.9812 |=================================================================== B . 1.9699 |=================================================================== C . 1.9550 |================================================================== Neural Magic DeepSparse 1.1 Model: NLP Document Classification, oBERT base uncased on IMDB - Scenario: Synchronous Single-Stream ms/batch < Lower Is Better A . 504.72 |================================================================== B . 507.62 |================================================================== C . 511.49 |=================================================================== Neural Magic DeepSparse 1.1 Model: NLP Question Answering, BERT base uncased SQuaD 12layer Pruned90 - Scenario: Asynchronous Multi-Stream items/sec > Higher Is Better A . 10.73 |==================================================================== B . 10.64 |=================================================================== C . 10.21 |================================================================= Neural Magic DeepSparse 1.1 Model: NLP Question Answering, BERT base uncased SQuaD 12layer Pruned90 - Scenario: Asynchronous Multi-Stream ms/batch < Lower Is Better A . 186.21 |================================================================ B . 187.58 |================================================================ C . 195.74 |=================================================================== Neural Magic DeepSparse 1.1 Model: NLP Question Answering, BERT base uncased SQuaD 12layer Pruned90 - Scenario: Synchronous Single-Stream items/sec > Higher Is Better A . 10.83 |==================================================================== B . 10.70 |=================================================================== C . 10.32 |================================================================= Neural Magic DeepSparse 1.1 Model: NLP Question Answering, BERT base uncased SQuaD 12layer Pruned90 - Scenario: Synchronous Single-Stream ms/batch < Lower Is Better A . 92.31 |================================================================= B . 93.46 |================================================================== C . 96.93 |==================================================================== Neural Magic DeepSparse 1.1 Model: CV Detection,YOLOv5s COCO - Scenario: Asynchronous Multi-Stream items/sec > Higher Is Better A . 13.82 |==================================================================== B . 13.72 |==================================================================== C . 13.66 |=================================================================== Neural Magic DeepSparse 1.1 Model: CV Detection,YOLOv5s COCO - Scenario: Asynchronous Multi-Stream ms/batch < Lower Is Better A . 144.44 |================================================================== B . 145.62 |=================================================================== C . 146.22 |=================================================================== Neural Magic DeepSparse 1.1 Model: CV Detection,YOLOv5s COCO - Scenario: Synchronous Single-Stream items/sec > Higher Is Better A . 12.62 |==================================================================== B . 12.58 |==================================================================== C . 12.57 |==================================================================== Neural Magic DeepSparse 1.1 Model: CV Detection,YOLOv5s COCO - Scenario: Synchronous Single-Stream ms/batch < Lower Is Better A . 79.21 |==================================================================== B . 79.50 |==================================================================== C . 79.57 |==================================================================== Neural Magic DeepSparse 1.1 Model: CV Classification, ResNet-50 ImageNet - Scenario: Asynchronous Multi-Stream items/sec > Higher Is Better A . 34.29 |==================================================================== B . 34.23 |==================================================================== C . 33.68 |=================================================================== Neural Magic DeepSparse 1.1 Model: CV Classification, ResNet-50 ImageNet - Scenario: Asynchronous Multi-Stream ms/batch < Lower Is Better A . 58.27 |=================================================================== B . 58.38 |=================================================================== C . 59.30 |==================================================================== Neural Magic DeepSparse 1.1 Model: CV Classification, ResNet-50 ImageNet - Scenario: Synchronous Single-Stream items/sec > Higher Is Better A . 33.79 |==================================================================== B . 33.74 |==================================================================== C . 32.83 |================================================================== Neural Magic DeepSparse 1.1 Model: CV Classification, ResNet-50 ImageNet - Scenario: Synchronous Single-Stream ms/batch < Lower Is Better A . 29.58 |================================================================== B . 29.63 |================================================================== C . 30.45 |==================================================================== Neural Magic DeepSparse 1.1 Model: NLP Text Classification, DistilBERT mnli - Scenario: Asynchronous Multi-Stream items/sec > Higher Is Better A . 18.61 |==================================================================== B . 18.74 |==================================================================== C . 18.27 |================================================================== Neural Magic DeepSparse 1.1 Model: NLP Text Classification, DistilBERT mnli - Scenario: Asynchronous Multi-Stream ms/batch < Lower Is Better A . 107.35 |================================================================== B . 106.51 |================================================================= C . 109.25 |=================================================================== Neural Magic DeepSparse 1.1 Model: NLP Text Classification, DistilBERT mnli - Scenario: Synchronous Single-Stream items/sec > Higher Is Better A . 17.15 |==================================================================== B . 17.05 |==================================================================== C . 16.88 |=================================================================== Neural Magic DeepSparse 1.1 Model: NLP Text Classification, DistilBERT mnli - Scenario: Synchronous Single-Stream ms/batch < Lower Is Better A . 58.29 |=================================================================== B . 58.63 |=================================================================== C . 59.24 |==================================================================== Neural Magic DeepSparse 1.1 Model: NLP Text Classification, BERT base uncased SST2 - Scenario: Asynchronous Multi-Stream items/sec > Higher Is Better A . 8.6019 |=================================================================== B . 8.5710 |=================================================================== C . 8.4851 |================================================================== Neural Magic DeepSparse 1.1 Model: NLP Text Classification, BERT base uncased SST2 - Scenario: Asynchronous Multi-Stream ms/batch < Lower Is Better A . 232.18 |================================================================== B . 232.61 |================================================================== C . 235.26 |=================================================================== Neural Magic DeepSparse 1.1 Model: NLP Text Classification, BERT base uncased SST2 - Scenario: Synchronous Single-Stream items/sec > Higher Is Better A . 8.0127 |=================================================================== B . 7.9855 |=================================================================== C . 7.8825 |================================================================== Neural Magic DeepSparse 1.1 Model: NLP Text Classification, BERT base uncased SST2 - Scenario: Synchronous Single-Stream ms/batch < Lower Is Better A . 124.79 |================================================================== B . 125.22 |================================================================== C . 126.85 |=================================================================== Neural Magic DeepSparse 1.1 Model: NLP Token Classification, BERT base uncased conll2003 - Scenario: Asynchronous Multi-Stream items/sec > Higher Is Better A . 1.9774 |=================================================================== B . 1.9596 |================================================================== C . 1.9373 |================================================================== Neural Magic DeepSparse 1.1 Model: NLP Token Classification, BERT base uncased conll2003 - Scenario: Asynchronous Multi-Stream ms/batch < Lower Is Better A . 997.69 |================================================================ B . 1018.96 |================================================================= C . 1031.08 |================================================================== Neural Magic DeepSparse 1.1 Model: NLP Token Classification, BERT base uncased conll2003 - Scenario: Synchronous Single-Stream items/sec > Higher Is Better A . 1.9806 |=================================================================== B . 1.9692 |=================================================================== C . 1.9598 |================================================================== Neural Magic DeepSparse 1.1 Model: NLP Token Classification, BERT base uncased conll2003 - Scenario: Synchronous Single-Stream ms/batch < Lower Is Better A . 504.88 |================================================================== B . 507.82 |=================================================================== C . 510.24 |=================================================================== OpenRadioss 2022.10.13 Model: Bumper Beam Seconds < Lower Is Better A . 423.83 |======================================================== B . 424.60 |======================================================== C . 504.27 |=================================================================== OpenRadioss 2022.10.13 Model: Cell Phone Drop Test Seconds < Lower Is Better A . 344.85 |=========================================================== B . 356.11 |============================================================= C . 392.96 |=================================================================== OpenRadioss 2022.10.13 Model: Bird Strike on Windshield Seconds < Lower Is Better A . 923.08 |============================================================ B . 958.10 |============================================================== C . 1018.76 |================================================================== OpenRadioss 2022.10.13 Model: Rubber O-Ring Seal Installation Seconds < Lower Is Better A . 654.41 |============================================================= B . 702.46 |================================================================== C . 714.27 |=================================================================== OpenRadioss 2022.10.13 Model: INIVOL and Fluid Structure Interaction Drop Container Seconds < Lower Is Better A . 1937.97 |============================================================= B . 2040.34 |================================================================ C . 2102.09 |==================================================================