ddddeep Intel Core i7-1280P testing with a MSI Prestige 14Evo A12M MS-14C6 (E14C6IMS.115 BIOS) and MSI Intel ADL GT2 8GB on Ubuntu 23.10 via the Phoronix Test Suite.
Compare your own system(s) to this result file with the
Phoronix Test Suite by running the command:
phoronix-test-suite benchmark 2403158-NE-DDDDEEP7064 a Kernel Notes: Transparent Huge Pages: madviseProcessor Notes: Scaling Governor: intel_pstate powersave (EPP: balance_performance) - CPU Microcode: 0x430 - Thermald 2.5.4Python Notes: Python 3.11.6Security Notes: gather_data_sampling: Not affected + itlb_multihit: Not affected + l1tf: Not affected + mds: Not affected + meltdown: Not affected + mmio_stale_data: Not affected + retbleed: Not affected + spec_rstack_overflow: Not affected + spec_store_bypass: Mitigation of SSB disabled via prctl + spectre_v1: Mitigation of usercopy/swapgs barriers and __user pointer sanitization + spectre_v2: Mitigation of Enhanced / Automatic IBRS IBPB: conditional RSB filling PBRSB-eIBRS: SW sequence + srbds: Not affected + tsx_async_abort: Not affected
b c Processor: Intel Core i7-1280P @ 4.70GHz (14 Cores / 20 Threads), Motherboard: MSI Prestige 14Evo A12M MS-14C6 (E14C6IMS.115 BIOS), Chipset: Intel Alder Lake PCH, Memory: 8 x 2GB LPDDR4-4267MT/s SK Hynix H9HCNNNCPMMLXR-, Disk: 1024GB Micron_3400_MTFDKBA1T0TFH, Graphics: MSI Intel ADL GT2 8GB (1450MHz), Audio: Realtek ALC274, Network: Intel Alder Lake-P PCH CNVi WiFi
OS: Ubuntu 23.10, Kernel: 6.7.0-060700-generic (x86_64), Desktop: GNOME Shell 45.2, Display Server: X Server + Wayland, OpenGL: 4.6 Mesa 24.1~git2401210600.c3a64f~oibaf~m (git-c3a64f8 2024-01-21 mantic-oibaf-ppa), OpenCL: OpenCL 3.0, Compiler: GCC 13.2.0, File-System: ext4, Screen Resolution: 1920x1080
ddddeep OpenBenchmarking.org Phoronix Test Suite Intel Core i7-1280P @ 4.70GHz (14 Cores / 20 Threads) MSI Prestige 14Evo A12M MS-14C6 (E14C6IMS.115 BIOS) Intel Alder Lake PCH 8 x 2GB LPDDR4-4267MT/s SK Hynix H9HCNNNCPMMLXR- 1024GB Micron_3400_MTFDKBA1T0TFH MSI Intel ADL GT2 8GB (1450MHz) Realtek ALC274 Intel Alder Lake-P PCH CNVi WiFi Ubuntu 23.10 6.7.0-060700-generic (x86_64) GNOME Shell 45.2 X Server + Wayland 4.6 Mesa 24.1~git2401210600.c3a64f~oibaf~m (git-c3a64f8 2024-01-21 mantic-oibaf-ppa) OpenCL 3.0 GCC 13.2.0 ext4 1920x1080 Processor Motherboard Chipset Memory Disk Graphics Audio Network OS Kernel Desktop Display Server OpenGL OpenCL Compiler File-System Screen Resolution Ddddeep Benchmarks System Logs - Transparent Huge Pages: madvise - Scaling Governor: intel_pstate powersave (EPP: balance_performance) - CPU Microcode: 0x430 - Thermald 2.5.4 - Python 3.11.6 - gather_data_sampling: Not affected + itlb_multihit: Not affected + l1tf: Not affected + mds: Not affected + meltdown: Not affected + mmio_stale_data: Not affected + retbleed: Not affected + spec_rstack_overflow: Not affected + spec_store_bypass: Mitigation of SSB disabled via prctl + spectre_v1: Mitigation of usercopy/swapgs barriers and __user pointer sanitization + spectre_v2: Mitigation of Enhanced / Automatic IBRS IBPB: conditional RSB filling PBRSB-eIBRS: SW sequence + srbds: Not affected + tsx_async_abort: Not affected
a b c Result Overview Phoronix Test Suite 100% 106% 113% 119% 126% Neural Magic DeepSparse Neural Magic DeepSparse Neural Magic DeepSparse Neural Magic DeepSparse Neural Magic DeepSparse Neural Magic DeepSparse Neural Magic DeepSparse Neural Magic DeepSparse Neural Magic DeepSparse Neural Magic DeepSparse Neural Magic DeepSparse Neural Magic DeepSparse Neural Magic DeepSparse Neural Magic DeepSparse Neural Magic DeepSparse Neural Magic DeepSparse Neural Magic DeepSparse Neural Magic DeepSparse Neural Magic DeepSparse Neural Magic DeepSparse Neural Magic DeepSparse Neural Magic DeepSparse Neural Magic DeepSparse Neural Magic DeepSparse Neural Magic DeepSparse Neural Magic DeepSparse Neural Magic DeepSparse Neural Magic DeepSparse Neural Magic DeepSparse Neural Magic DeepSparse Neural Magic DeepSparse Neural Magic DeepSparse Neural Magic DeepSparse Neural Magic DeepSparse Neural Magic DeepSparse Neural Magic DeepSparse Neural Magic DeepSparse Neural Magic DeepSparse Neural Magic DeepSparse Neural Magic DeepSparse Neural Magic DeepSparse Neural Magic DeepSparse Neural Magic DeepSparse Neural Magic DeepSparse B.L.N.Q.A.S.I - A.M.S B.L.N.Q.A.S.I - A.M.S N.D.C.o.b.u.o.I - S.S.S N.D.C.o.b.u.o.I - S.S.S C.D.Y.C.S.I - A.M.S C.D.Y.C.S.I - A.M.S N.T.C.B.b.u.S.S.I - A.M.S N.T.C.B.b.u.S.S.I - A.M.S N.T.C.B.b.u.c - A.M.S N.T.C.B.b.u.c - A.M.S N.T.C.B.b.u.c - S.S.S N.T.C.B.b.u.c - S.S.S N.D.C.o.b.u.o.I - A.M.S N.D.C.o.b.u.o.I - A.M.S R.5.S.I - A.M.S N.T.C.B.b.u.S.S.I - S.S.S N.T.C.B.b.u.S.S.I - S.S.S R.5.S.I - A.M.S R.5.B - S.S.S R.5.B - S.S.S N.T.C.D.m - A.M.S N.T.C.D.m - A.M.S N.T.C.D.m - S.S.S N.T.C.D.m - S.S.S C.S.9.P.Y.P - S.S.S C.S.9.P.Y.P - S.S.S C.S.9.P.Y.P - A.M.S C.S.9.P.Y.P - A.M.S B.L.N.Q.A.S.I - S.S.S B.L.N.Q.A.S.I - S.S.S R.5.B - A.M.S R.5.B - A.M.S C.C.R.5.I - A.M.S C.C.R.5.I - A.M.S R.5.S.I - S.S.S R.5.S.I - S.S.S C.C.R.5.I - S.S.S C.C.R.5.I - S.S.S L.C.7.Q - S.S.S L.C.7.Q - S.S.S L.C.7.Q - A.M.S L.C.7.Q - A.M.S C.D.Y.C.S.I - S.S.S C.D.Y.C.S.I - S.S.S
ddddeep deepsparse: NLP Document Classification, oBERT base uncased on IMDB - Asynchronous Multi-Stream deepsparse: NLP Document Classification, oBERT base uncased on IMDB - Asynchronous Multi-Stream deepsparse: NLP Document Classification, oBERT base uncased on IMDB - Synchronous Single-Stream deepsparse: NLP Document Classification, oBERT base uncased on IMDB - Synchronous Single-Stream deepsparse: NLP Text Classification, BERT base uncased SST2, Sparse INT8 - Asynchronous Multi-Stream deepsparse: NLP Text Classification, BERT base uncased SST2, Sparse INT8 - Asynchronous Multi-Stream deepsparse: NLP Text Classification, BERT base uncased SST2, Sparse INT8 - Synchronous Single-Stream deepsparse: NLP Text Classification, BERT base uncased SST2, Sparse INT8 - Synchronous Single-Stream deepsparse: ResNet-50, Baseline - Asynchronous Multi-Stream deepsparse: ResNet-50, Baseline - Asynchronous Multi-Stream deepsparse: ResNet-50, Baseline - Synchronous Single-Stream deepsparse: ResNet-50, Baseline - Synchronous Single-Stream deepsparse: ResNet-50, Sparse INT8 - Asynchronous Multi-Stream deepsparse: ResNet-50, Sparse INT8 - Asynchronous Multi-Stream deepsparse: ResNet-50, Sparse INT8 - Synchronous Single-Stream deepsparse: ResNet-50, Sparse INT8 - Synchronous Single-Stream deepsparse: Llama2 Chat 7b Quantized - Asynchronous Multi-Stream deepsparse: Llama2 Chat 7b Quantized - Asynchronous Multi-Stream deepsparse: Llama2 Chat 7b Quantized - Synchronous Single-Stream deepsparse: Llama2 Chat 7b Quantized - Synchronous Single-Stream deepsparse: CV Classification, ResNet-50 ImageNet - Asynchronous Multi-Stream deepsparse: CV Classification, ResNet-50 ImageNet - Asynchronous Multi-Stream deepsparse: CV Classification, ResNet-50 ImageNet - Synchronous Single-Stream deepsparse: CV Classification, ResNet-50 ImageNet - Synchronous Single-Stream deepsparse: CV Detection, YOLOv5s COCO, Sparse INT8 - Asynchronous Multi-Stream deepsparse: CV Detection, YOLOv5s COCO, Sparse INT8 - Asynchronous Multi-Stream deepsparse: CV Detection, YOLOv5s COCO, Sparse INT8 - Synchronous Single-Stream deepsparse: CV Detection, YOLOv5s COCO, Sparse INT8 - Synchronous Single-Stream deepsparse: NLP Text Classification, DistilBERT mnli - Asynchronous Multi-Stream deepsparse: NLP Text Classification, DistilBERT mnli - Asynchronous Multi-Stream deepsparse: NLP Text Classification, DistilBERT mnli - Synchronous Single-Stream deepsparse: NLP Text Classification, DistilBERT mnli - Synchronous Single-Stream deepsparse: CV Segmentation, 90% Pruned YOLACT Pruned - Asynchronous Multi-Stream deepsparse: CV Segmentation, 90% Pruned YOLACT Pruned - Asynchronous Multi-Stream deepsparse: CV Segmentation, 90% Pruned YOLACT Pruned - Synchronous Single-Stream deepsparse: CV Segmentation, 90% Pruned YOLACT Pruned - Synchronous Single-Stream deepsparse: BERT-Large, NLP Question Answering, Sparse INT8 - Asynchronous Multi-Stream deepsparse: BERT-Large, NLP Question Answering, Sparse INT8 - Asynchronous Multi-Stream deepsparse: BERT-Large, NLP Question Answering, Sparse INT8 - Synchronous Single-Stream deepsparse: BERT-Large, NLP Question Answering, Sparse INT8 - Synchronous Single-Stream deepsparse: NLP Token Classification, BERT base uncased conll2003 - Asynchronous Multi-Stream deepsparse: NLP Token Classification, BERT base uncased conll2003 - Asynchronous Multi-Stream deepsparse: NLP Token Classification, BERT base uncased conll2003 - Synchronous Single-Stream deepsparse: NLP Token Classification, BERT base uncased conll2003 - Synchronous Single-Stream a b c 4.6509 644.9876 4.2542 235.052 162.0447 18.4769 128.8096 7.7576 52.9588 56.5933 46.1087 21.6801 403.884 7.3983 335.4037 2.9712 4.6331 642.8633 6.885 145.2236 52.7112 56.8819 43.761 22.8445 24.5281 122.2706 24.7946 40.325 35.7435 83.8992 32.0773 31.1695 4.9333 607.7805 5.0335 198.6491 65.5875 45.702 50.9996 19.5952 3.8107 778.5028 4.0998 243.9057 4.8027 624.614 3.9587 252.6013 150.8361 19.8544 121.9374 8.1944 53.7202 55.7817 46.8078 21.3576 412.6962 7.2381 334.5831 2.9789 4.6873 635.4708 6.7827 147.4126 51.9423 57.7272 44.3564 22.5378 22.5616 132.9137 24.9388 40.091 38.0851 78.7443 33.8164 29.5657 5.2179 574.9023 4.7566 210.2137 61.6926 48.5906 48.955 20.4143 4.0838 730.0217 3.7941 263.5618 5.0006 598.3451 4.3521 229.7648 164.6301 18.1881 130.9919 7.6286 55.003 54.5131 49.4811 20.2029 433.8341 6.8862 342.8475 2.9071 4.6837 635.9719 6.8513 145.937 53.501 56.0394 44.4672 22.4817 24.6313 121.7572 24.9889 40.0104 35.7498 83.8862 31.7852 31.4555 4.9986 600.1209 5.0119 199.5088 77.5952 38.6102 51.2363 19.5064 3.7499 793.8798 4.0441 247.2655 OpenBenchmarking.org
Neural Magic DeepSparse OpenBenchmarking.org items/sec, More Is Better Neural Magic DeepSparse 1.7 Model: NLP Document Classification, oBERT base uncased on IMDB - Scenario: Asynchronous Multi-Stream a b c 1.1251 2.2502 3.3753 4.5004 5.6255 4.6509 4.8027 5.0006
a Kernel Notes: Transparent Huge Pages: madviseProcessor Notes: Scaling Governor: intel_pstate powersave (EPP: balance_performance) - CPU Microcode: 0x430 - Thermald 2.5.4Python Notes: Python 3.11.6Security Notes: gather_data_sampling: Not affected + itlb_multihit: Not affected + l1tf: Not affected + mds: Not affected + meltdown: Not affected + mmio_stale_data: Not affected + retbleed: Not affected + spec_rstack_overflow: Not affected + spec_store_bypass: Mitigation of SSB disabled via prctl + spectre_v1: Mitigation of usercopy/swapgs barriers and __user pointer sanitization + spectre_v2: Mitigation of Enhanced / Automatic IBRS IBPB: conditional RSB filling PBRSB-eIBRS: SW sequence + srbds: Not affected + tsx_async_abort: Not affected
Testing initiated at 15 March 2024 18:59 by user phoronix.
b Kernel Notes: Transparent Huge Pages: madviseProcessor Notes: Scaling Governor: intel_pstate powersave (EPP: balance_performance) - CPU Microcode: 0x430 - Thermald 2.5.4Python Notes: Python 3.11.6Security Notes: gather_data_sampling: Not affected + itlb_multihit: Not affected + l1tf: Not affected + mds: Not affected + meltdown: Not affected + mmio_stale_data: Not affected + retbleed: Not affected + spec_rstack_overflow: Not affected + spec_store_bypass: Mitigation of SSB disabled via prctl + spectre_v1: Mitigation of usercopy/swapgs barriers and __user pointer sanitization + spectre_v2: Mitigation of Enhanced / Automatic IBRS IBPB: conditional RSB filling PBRSB-eIBRS: SW sequence + srbds: Not affected + tsx_async_abort: Not affected
Testing initiated at 15 March 2024 19:20 by user phoronix.
c Processor: Intel Core i7-1280P @ 4.70GHz (14 Cores / 20 Threads), Motherboard: MSI Prestige 14Evo A12M MS-14C6 (E14C6IMS.115 BIOS), Chipset: Intel Alder Lake PCH, Memory: 8 x 2GB LPDDR4-4267MT/s SK Hynix H9HCNNNCPMMLXR-, Disk: 1024GB Micron_3400_MTFDKBA1T0TFH, Graphics: MSI Intel ADL GT2 8GB (1450MHz), Audio: Realtek ALC274, Network: Intel Alder Lake-P PCH CNVi WiFi
OS: Ubuntu 23.10, Kernel: 6.7.0-060700-generic (x86_64), Desktop: GNOME Shell 45.2, Display Server: X Server + Wayland, OpenGL: 4.6 Mesa 24.1~git2401210600.c3a64f~oibaf~m (git-c3a64f8 2024-01-21 mantic-oibaf-ppa), OpenCL: OpenCL 3.0, Compiler: GCC 13.2.0, File-System: ext4, Screen Resolution: 1920x1080
Kernel Notes: Transparent Huge Pages: madviseProcessor Notes: Scaling Governor: intel_pstate powersave (EPP: balance_performance) - CPU Microcode: 0x430 - Thermald 2.5.4Python Notes: Python 3.11.6Security Notes: gather_data_sampling: Not affected + itlb_multihit: Not affected + l1tf: Not affected + mds: Not affected + meltdown: Not affected + mmio_stale_data: Not affected + retbleed: Not affected + spec_rstack_overflow: Not affected + spec_store_bypass: Mitigation of SSB disabled via prctl + spectre_v1: Mitigation of usercopy/swapgs barriers and __user pointer sanitization + spectre_v2: Mitigation of Enhanced / Automatic IBRS IBPB: conditional RSB filling PBRSB-eIBRS: SW sequence + srbds: Not affected + tsx_async_abort: Not affected
Testing initiated at 15 March 2024 19:59 by user phoronix.