deepsparse tigerlake Intel Core i7-1165G7 testing with a Dell 0GG9PT (3.3.0 BIOS) and Intel Xe TGL GT2 3GB on Ubuntu 21.10 via the Phoronix Test Suite.
Compare your own system(s) to this result file with the
Phoronix Test Suite by running the command:
phoronix-test-suite benchmark 2210139-NE-DEEPSPARS89 A Kernel Notes: Transparent Huge Pages: madviseProcessor Notes: Scaling Governor: intel_pstate powersave (EPP: balance_performance) - CPU Microcode: 0xa4 - Thermald 2.4.6Python Notes: Python 3.9.7Security Notes: itlb_multihit: Not affected + l1tf: Not affected + mds: Not affected + meltdown: Not affected + mmio_stale_data: Not affected + spec_store_bypass: Mitigation of SSB disabled via prctl and seccomp + spectre_v1: Mitigation of usercopy/swapgs barriers and __user pointer sanitization + spectre_v2: Mitigation of Enhanced IBRS IBPB: conditional RSB filling + srbds: Not affected + tsx_async_abort: Not affected
B C Processor: Intel Core i7-1165G7 @ 4.70GHz (4 Cores / 8 Threads), Motherboard: Dell 0GG9PT (3.3.0 BIOS), Chipset: Intel Tiger Lake-LP, Memory: 16GB, Disk: Kioxia KBG40ZNS256G NVMe 256GB, Graphics: Intel Xe TGL GT2 3GB (1300MHz), Audio: Realtek ALC289, Network: Intel Wi-Fi 6 AX201
OS: Ubuntu 21.10, Kernel: 5.13.0-52-generic (x86_64), Desktop: GNOME Shell 40.5, Display Server: X Server + Wayland, OpenGL: 4.6 Mesa 21.2.2, Vulkan: 1.2.182, Compiler: GCC 11.2.0, File-System: ext4, Screen Resolution: 1920x1200
deepsparse tigerlake OpenBenchmarking.org Phoronix Test Suite Intel Core i7-1165G7 @ 4.70GHz (4 Cores / 8 Threads) Dell 0GG9PT (3.3.0 BIOS) Intel Tiger Lake-LP 16GB Kioxia KBG40ZNS256G NVMe 256GB Intel Xe TGL GT2 3GB (1300MHz) Realtek ALC289 Intel Wi-Fi 6 AX201 Ubuntu 21.10 5.13.0-52-generic (x86_64) GNOME Shell 40.5 X Server + Wayland 4.6 Mesa 21.2.2 1.2.182 GCC 11.2.0 ext4 1920x1200 Processor Motherboard Chipset Memory Disk Graphics Audio Network OS Kernel Desktop Display Server OpenGL Vulkan Compiler File-System Screen Resolution Deepsparse Tigerlake Benchmarks System Logs - Transparent Huge Pages: madvise - Scaling Governor: intel_pstate powersave (EPP: balance_performance) - CPU Microcode: 0xa4 - Thermald 2.4.6 - Python 3.9.7 - itlb_multihit: Not affected + l1tf: Not affected + mds: Not affected + meltdown: Not affected + mmio_stale_data: Not affected + spec_store_bypass: Mitigation of SSB disabled via prctl and seccomp + spectre_v1: Mitigation of usercopy/swapgs barriers and __user pointer sanitization + spectre_v2: Mitigation of Enhanced IBRS IBPB: conditional RSB filling + srbds: Not affected + tsx_async_abort: Not affected
A B C Result Overview Phoronix Test Suite 100% 106% 113% 119% 125% Neural Magic DeepSparse Neural Magic DeepSparse Neural Magic DeepSparse Neural Magic DeepSparse Neural Magic DeepSparse Neural Magic DeepSparse Neural Magic DeepSparse Neural Magic DeepSparse Neural Magic DeepSparse Neural Magic DeepSparse Neural Magic DeepSparse Neural Magic DeepSparse Neural Magic DeepSparse Neural Magic DeepSparse Neural Magic DeepSparse Neural Magic DeepSparse Neural Magic DeepSparse Neural Magic DeepSparse Neural Magic DeepSparse Neural Magic DeepSparse Neural Magic DeepSparse Neural Magic DeepSparse Neural Magic DeepSparse Neural Magic DeepSparse Neural Magic DeepSparse Neural Magic DeepSparse Neural Magic DeepSparse Neural Magic DeepSparse N.D.C.o.b.u.o.I - A.M.S N.D.C.o.b.u.o.I - A.M.S C.D.Y.C - A.M.S C.D.Y.C - A.M.S N.D.C.o.b.u.o.I - S.S.S N.D.C.o.b.u.o.I - S.S.S N.Q.A.B.b.u.S.1.P - A.M.S N.Q.A.B.b.u.S.1.P - A.M.S C.C.R.5.I - A.M.S C.C.R.5.I - A.M.S N.Q.A.B.b.u.S.1.P - S.S.S N.Q.A.B.b.u.S.1.P - S.S.S N.T.C.B.b.u.c - A.M.S N.T.C.B.b.u.c - A.M.S N.T.C.D.m - A.M.S N.T.C.B.b.u.S - A.M.S N.T.C.D.m - A.M.S N.T.C.B.b.u.S - A.M.S C.C.R.5.I - S.S.S C.C.R.5.I - S.S.S N.T.C.D.m - S.S.S N.T.C.D.m - S.S.S N.T.C.B.b.u.S - S.S.S N.T.C.B.b.u.S - S.S.S C.D.Y.C - S.S.S N.T.C.B.b.u.c - S.S.S N.T.C.B.b.u.c - S.S.S C.D.Y.C - S.S.S
deepsparse tigerlake deepsparse: NLP Document Classification, oBERT base uncased on IMDB - Asynchronous Multi-Stream deepsparse: NLP Document Classification, oBERT base uncased on IMDB - Synchronous Single-Stream deepsparse: NLP Question Answering, BERT base uncased SQuaD 12layer Pruned90 - Asynchronous Multi-Stream deepsparse: NLP Question Answering, BERT base uncased SQuaD 12layer Pruned90 - Synchronous Single-Stream deepsparse: CV Detection,YOLOv5s COCO - Asynchronous Multi-Stream deepsparse: CV Detection,YOLOv5s COCO - Synchronous Single-Stream deepsparse: CV Classification, ResNet-50 ImageNet - Asynchronous Multi-Stream deepsparse: CV Classification, ResNet-50 ImageNet - Synchronous Single-Stream deepsparse: NLP Text Classification, DistilBERT mnli - Asynchronous Multi-Stream deepsparse: NLP Text Classification, DistilBERT mnli - Synchronous Single-Stream deepsparse: NLP Text Classification, BERT base uncased SST2 - Asynchronous Multi-Stream deepsparse: NLP Text Classification, BERT base uncased SST2 - Synchronous Single-Stream deepsparse: NLP Token Classification, BERT base uncased conll2003 - Asynchronous Multi-Stream deepsparse: NLP Token Classification, BERT base uncased conll2003 - Synchronous Single-Stream deepsparse: NLP Document Classification, oBERT base uncased on IMDB - Asynchronous Multi-Stream deepsparse: NLP Document Classification, oBERT base uncased on IMDB - Synchronous Single-Stream deepsparse: NLP Question Answering, BERT base uncased SQuaD 12layer Pruned90 - Asynchronous Multi-Stream deepsparse: NLP Question Answering, BERT base uncased SQuaD 12layer Pruned90 - Synchronous Single-Stream deepsparse: CV Detection,YOLOv5s COCO - Asynchronous Multi-Stream deepsparse: CV Detection,YOLOv5s COCO - Synchronous Single-Stream deepsparse: CV Classification, ResNet-50 ImageNet - Asynchronous Multi-Stream deepsparse: CV Classification, ResNet-50 ImageNet - Synchronous Single-Stream deepsparse: NLP Text Classification, DistilBERT mnli - Asynchronous Multi-Stream deepsparse: NLP Text Classification, DistilBERT mnli - Synchronous Single-Stream deepsparse: NLP Text Classification, BERT base uncased SST2 - Asynchronous Multi-Stream deepsparse: NLP Text Classification, BERT base uncased SST2 - Synchronous Single-Stream deepsparse: NLP Token Classification, BERT base uncased conll2003 - Asynchronous Multi-Stream deepsparse: NLP Token Classification, BERT base uncased conll2003 - Synchronous Single-Stream A B C 3.8206 3.5314 18.07 17.9004 24.557 18.7677 57.4312 53.4965 30.633 25.3909 14.942 12.8288 3.3209 3.0149 522.3633 283.1619 110.5157 55.8547 81.3989 53.2661 34.7921 18.6818 65.2624 39.3757 133.8231 77.9404 602.1996 331.6732 3.1607 3.0965 15.6345 15.9436 20.7752 19.0685 51.1057 49.9473 27.7398 26.5985 13.5625 13.1296 2.9727 3.0819 636.1670 323.6244 127.8716 62.7114 96.3339 52.5447 39.1396 20.0423 72.5088 37.8059 147.8097 76.2820 672.7632 325.0755 3.0716 3.0117 15.7496 16.0589 20.9180 19.2261 50.6621 50.5037 27.5933 26.1379 13.4826 13.3481 3.1063 3.0857 652.8177 332.0357 126.9728 62.3067 95.7682 52.1663 39.4344 19.8473 72.7420 38.3995 148.6594 75.1728 645.2719 324.5840 OpenBenchmarking.org
Neural Magic DeepSparse OpenBenchmarking.org items/sec, More Is Better Neural Magic DeepSparse 1.1 Model: NLP Document Classification, oBERT base uncased on IMDB - Scenario: Asynchronous Multi-Stream A B C 0.8596 1.7192 2.5788 3.4384 4.298 SE +/- 0.0636, N = 15 SE +/- 0.0470, N = 15 3.8206 3.1607 3.0716
OpenBenchmarking.org items/sec, More Is Better Neural Magic DeepSparse 1.1 Model: NLP Document Classification, oBERT base uncased on IMDB - Scenario: Synchronous Single-Stream A B C 0.7946 1.5892 2.3838 3.1784 3.973 SE +/- 0.0393, N = 15 SE +/- 0.0012, N = 3 3.5314 3.0965 3.0117
OpenBenchmarking.org items/sec, More Is Better Neural Magic DeepSparse 1.1 Model: NLP Question Answering, BERT base uncased SQuaD 12layer Pruned90 - Scenario: Asynchronous Multi-Stream A C B 4 8 12 16 20 SE +/- 0.13, N = 9 SE +/- 0.05, N = 3 18.07 15.75 15.63
OpenBenchmarking.org items/sec, More Is Better Neural Magic DeepSparse 1.1 Model: NLP Question Answering, BERT base uncased SQuaD 12layer Pruned90 - Scenario: Synchronous Single-Stream A C B 4 8 12 16 20 SE +/- 0.12, N = 15 SE +/- 0.04, N = 3 17.90 16.06 15.94
OpenBenchmarking.org items/sec, More Is Better Neural Magic DeepSparse 1.1 Model: NLP Token Classification, BERT base uncased conll2003 - Scenario: Asynchronous Multi-Stream A C B 0.7472 1.4944 2.2416 2.9888 3.736 SE +/- 0.0448, N = 15 SE +/- 0.0069, N = 3 3.3209 3.1063 2.9727
OpenBenchmarking.org items/sec, More Is Better Neural Magic DeepSparse 1.1 Model: NLP Token Classification, BERT base uncased conll2003 - Scenario: Synchronous Single-Stream C B A 0.6943 1.3886 2.0829 2.7772 3.4715 SE +/- 0.0339, N = 15 SE +/- 0.0366, N = 15 3.0857 3.0819 3.0149
OpenBenchmarking.org ms/batch, Fewer Is Better Neural Magic DeepSparse 1.1 Model: NLP Document Classification, oBERT base uncased on IMDB - Scenario: Asynchronous Multi-Stream A B C 140 280 420 560 700 SE +/- 12.23, N = 15 SE +/- 9.24, N = 15 522.36 636.17 652.82
OpenBenchmarking.org ms/batch, Fewer Is Better Neural Magic DeepSparse 1.1 Model: NLP Document Classification, oBERT base uncased on IMDB - Scenario: Synchronous Single-Stream A B C 70 140 210 280 350 SE +/- 3.86, N = 15 SE +/- 0.13, N = 3 283.16 323.62 332.04
OpenBenchmarking.org ms/batch, Fewer Is Better Neural Magic DeepSparse 1.1 Model: NLP Question Answering, BERT base uncased SQuaD 12layer Pruned90 - Scenario: Asynchronous Multi-Stream A C B 30 60 90 120 150 SE +/- 0.97, N = 9 SE +/- 0.41, N = 3 110.52 126.97 127.87
OpenBenchmarking.org ms/batch, Fewer Is Better Neural Magic DeepSparse 1.1 Model: NLP Question Answering, BERT base uncased SQuaD 12layer Pruned90 - Scenario: Synchronous Single-Stream A C B 14 28 42 56 70 SE +/- 0.44, N = 15 SE +/- 0.16, N = 3 55.85 62.31 62.71
OpenBenchmarking.org ms/batch, Fewer Is Better Neural Magic DeepSparse 1.1 Model: NLP Text Classification, BERT base uncased SST2 - Scenario: Asynchronous Multi-Stream A B C 30 60 90 120 150 SE +/- 1.95, N = 15 SE +/- 1.90, N = 15 133.82 147.81 148.66
OpenBenchmarking.org ms/batch, Fewer Is Better Neural Magic DeepSparse 1.1 Model: NLP Token Classification, BERT base uncased conll2003 - Scenario: Asynchronous Multi-Stream A C B 150 300 450 600 750 SE +/- 9.13, N = 15 SE +/- 1.54, N = 3 602.20 645.27 672.76
OpenBenchmarking.org ms/batch, Fewer Is Better Neural Magic DeepSparse 1.1 Model: NLP Token Classification, BERT base uncased conll2003 - Scenario: Synchronous Single-Stream C B A 70 140 210 280 350 SE +/- 3.38, N = 15 SE +/- 3.68, N = 15 324.58 325.08 331.67
A Kernel Notes: Transparent Huge Pages: madviseProcessor Notes: Scaling Governor: intel_pstate powersave (EPP: balance_performance) - CPU Microcode: 0xa4 - Thermald 2.4.6Python Notes: Python 3.9.7Security Notes: itlb_multihit: Not affected + l1tf: Not affected + mds: Not affected + meltdown: Not affected + mmio_stale_data: Not affected + spec_store_bypass: Mitigation of SSB disabled via prctl and seccomp + spectre_v1: Mitigation of usercopy/swapgs barriers and __user pointer sanitization + spectre_v2: Mitigation of Enhanced IBRS IBPB: conditional RSB filling + srbds: Not affected + tsx_async_abort: Not affected
Testing initiated at 13 October 2022 14:09 by user phoronix.
B Kernel Notes: Transparent Huge Pages: madviseProcessor Notes: Scaling Governor: intel_pstate powersave (EPP: balance_performance) - CPU Microcode: 0xa4 - Thermald 2.4.6Python Notes: Python 3.9.7Security Notes: itlb_multihit: Not affected + l1tf: Not affected + mds: Not affected + meltdown: Not affected + mmio_stale_data: Not affected + spec_store_bypass: Mitigation of SSB disabled via prctl and seccomp + spectre_v1: Mitigation of usercopy/swapgs barriers and __user pointer sanitization + spectre_v2: Mitigation of Enhanced IBRS IBPB: conditional RSB filling + srbds: Not affected + tsx_async_abort: Not affected
Testing initiated at 13 October 2022 14:24 by user phoronix.
C Processor: Intel Core i7-1165G7 @ 4.70GHz (4 Cores / 8 Threads), Motherboard: Dell 0GG9PT (3.3.0 BIOS), Chipset: Intel Tiger Lake-LP, Memory: 16GB, Disk: Kioxia KBG40ZNS256G NVMe 256GB, Graphics: Intel Xe TGL GT2 3GB (1300MHz), Audio: Realtek ALC289, Network: Intel Wi-Fi 6 AX201
OS: Ubuntu 21.10, Kernel: 5.13.0-52-generic (x86_64), Desktop: GNOME Shell 40.5, Display Server: X Server + Wayland, OpenGL: 4.6 Mesa 21.2.2, Vulkan: 1.2.182, Compiler: GCC 11.2.0, File-System: ext4, Screen Resolution: 1920x1200
Kernel Notes: Transparent Huge Pages: madviseProcessor Notes: Scaling Governor: intel_pstate powersave (EPP: balance_performance) - CPU Microcode: 0xa4 - Thermald 2.4.6Python Notes: Python 3.9.7Security Notes: itlb_multihit: Not affected + l1tf: Not affected + mds: Not affected + meltdown: Not affected + mmio_stale_data: Not affected + spec_store_bypass: Mitigation of SSB disabled via prctl and seccomp + spectre_v1: Mitigation of usercopy/swapgs barriers and __user pointer sanitization + spectre_v2: Mitigation of Enhanced IBRS IBPB: conditional RSB filling + srbds: Not affected + tsx_async_abort: Not affected
Testing initiated at 13 October 2022 16:44 by user phoronix.