Intel Core i7-1065G7 testing with a Dell 06CDVY (1.0.9 BIOS) and Intel Iris Plus ICL GT2 16GB on Ubuntu 22.04 via the Phoronix Test Suite.
Compare your own system(s) to this result file with the
Phoronix Test Suite by running the command:
phoronix-test-suite benchmark 2301233-NE-ICELAKEDE36 icelake deep - Phoronix Test Suite icelake deep Intel Core i7-1065G7 testing with a Dell 06CDVY (1.0.9 BIOS) and Intel Iris Plus ICL GT2 16GB on Ubuntu 22.04 via the Phoronix Test Suite.
HTML result view exported from: https://openbenchmarking.org/result/2301233-NE-ICELAKEDE36&grs&rdt .
icelake deep Processor Motherboard Chipset Memory Disk Graphics Audio Network OS Kernel Desktop Display Server OpenGL OpenCL Vulkan Compiler File-System Screen Resolution a b c Intel Core i7-1065G7 @ 3.90GHz (4 Cores / 8 Threads) Dell 06CDVY (1.0.9 BIOS) Intel Ice Lake-LP DRAM 16GB Toshiba KBG40ZPZ512G NVMe 512GB Intel Iris Plus ICL GT2 16GB (1100MHz) Realtek ALC289 Intel Ice Lake-LP PCH CNVi WiFi Ubuntu 22.04 5.18.8-051808-generic (x86_64) GNOME Shell 42.2 X Server + Wayland 4.6 Mesa 22.0.1 OpenCL 3.0 1.3.204 GCC 11.3.0 ext4 1920x1200 OpenBenchmarking.org Kernel Details - Transparent Huge Pages: madvise Compiler Details - --build=x86_64-linux-gnu --disable-vtable-verify --disable-werror --enable-bootstrap --enable-cet --enable-checking=release --enable-clocale=gnu --enable-default-pie --enable-gnu-unique-object --enable-languages=c,ada,c++,go,brig,d,fortran,objc,obj-c++,m2 --enable-libphobos-checking=release --enable-libstdcxx-debug --enable-libstdcxx-time=yes --enable-link-serialization=2 --enable-multiarch --enable-multilib --enable-nls --enable-objc-gc=auto --enable-offload-targets=nvptx-none=/build/gcc-11-xKiWfi/gcc-11-11.3.0/debian/tmp-nvptx/usr,amdgcn-amdhsa=/build/gcc-11-xKiWfi/gcc-11-11.3.0/debian/tmp-gcn/usr --enable-plugin --enable-shared --enable-threads=posix --host=x86_64-linux-gnu --program-prefix=x86_64-linux-gnu- --target=x86_64-linux-gnu --with-abi=m64 --with-arch-32=i686 --with-build-config=bootstrap-lto-lean --with-default-libstdcxx-abi=new --with-gcc-major-version-only --with-multilib-list=m32,m64,mx32 --with-target-system-zlib=auto --with-tune=generic --without-cuda-driver -v Processor Details - Scaling Governor: intel_pstate powersave (EPP: balance_performance) - CPU Microcode: 0xb2 - Thermald 2.4.9 Python Details - Python 3.10.6 Security Details - itlb_multihit: KVM: Mitigation of VMX disabled + l1tf: Not affected + mds: Not affected + meltdown: Not affected + mmio_stale_data: Mitigation of Clear buffers; SMT vulnerable + spec_store_bypass: Mitigation of SSB disabled via prctl + spectre_v1: Mitigation of usercopy/swapgs barriers and __user pointer sanitization + spectre_v2: Mitigation of Enhanced IBRS IBPB: conditional RSB filling + srbds: Mitigation of Microcode + tsx_async_abort: Not affected
icelake deep keydb: SET - 500 keydb: LPOP - 50 keydb: LPOP - 500 keydb: LPOP - 100 keydb: LPOP - 900 keydb: GET - 900 keydb: SET - 100 keydb: GET - 500 keydb: GET - 100 keydb: SADD - 50 keydb: SET - 900 keydb: HMSET - 50 keydb: SADD - 900 keydb: SADD - 100 keydb: GET - 50 keydb: LPUSH - 50 keydb: SADD - 500 deepsparse: NLP Sentiment Analysis, 80% Pruned Quantized BERT Base Uncased - Asynchronous Multi-Stream deepsparse: NLP Sentiment Analysis, 80% Pruned Quantized BERT Base Uncased - Asynchronous Multi-Stream keydb: HMSET - 100 keydb: LPUSH - 900 keydb: HMSET - 500 keydb: SET - 50 deepsparse: CV Segmentation, 90% Pruned YOLACT Pruned - Asynchronous Multi-Stream deepsparse: CV Segmentation, 90% Pruned YOLACT Pruned - Asynchronous Multi-Stream keydb: LPUSH - 100 keydb: LPUSH - 500 deepsparse: NLP Text Classification, DistilBERT mnli - Asynchronous Multi-Stream deepsparse: NLP Text Classification, DistilBERT mnli - Asynchronous Multi-Stream deepsparse: NLP Question Answering, BERT base uncased SQuaD 12layer Pruned90 - Synchronous Single-Stream deepsparse: NLP Question Answering, BERT base uncased SQuaD 12layer Pruned90 - Synchronous Single-Stream deepsparse: CV Detection, YOLOv5s COCO - Asynchronous Multi-Stream deepsparse: CV Detection, YOLOv5s COCO - Asynchronous Multi-Stream deepsparse: CV Segmentation, 90% Pruned YOLACT Pruned - Synchronous Single-Stream deepsparse: CV Segmentation, 90% Pruned YOLACT Pruned - Synchronous Single-Stream deepsparse: NLP Document Classification, oBERT base uncased on IMDB - Asynchronous Multi-Stream keydb: HMSET - 900 deepsparse: NLP Document Classification, oBERT base uncased on IMDB - Asynchronous Multi-Stream deepsparse: NLP Sentiment Analysis, 80% Pruned Quantized BERT Base Uncased - Synchronous Single-Stream deepsparse: NLP Sentiment Analysis, 80% Pruned Quantized BERT Base Uncased - Synchronous Single-Stream deepsparse: CV Classification, ResNet-50 ImageNet - Asynchronous Multi-Stream deepsparse: CV Classification, ResNet-50 ImageNet - Asynchronous Multi-Stream deepsparse: NLP Token Classification, BERT base uncased conll2003 - Asynchronous Multi-Stream deepsparse: NLP Text Classification, BERT base uncased SST2 - Asynchronous Multi-Stream deepsparse: NLP Text Classification, BERT base uncased SST2 - Asynchronous Multi-Stream deepsparse: NLP Text Classification, DistilBERT mnli - Synchronous Single-Stream deepsparse: NLP Text Classification, DistilBERT mnli - Synchronous Single-Stream deepsparse: CV Detection, YOLOv5s COCO - Synchronous Single-Stream deepsparse: CV Detection, YOLOv5s COCO - Synchronous Single-Stream deepsparse: NLP Token Classification, BERT base uncased conll2003 - Asynchronous Multi-Stream deepsparse: NLP Question Answering, BERT base uncased SQuaD 12layer Pruned90 - Asynchronous Multi-Stream deepsparse: NLP Text Classification, BERT base uncased SST2 - Synchronous Single-Stream deepsparse: NLP Text Classification, BERT base uncased SST2 - Synchronous Single-Stream deepsparse: NLP Document Classification, oBERT base uncased on IMDB - Synchronous Single-Stream deepsparse: NLP Document Classification, oBERT base uncased on IMDB - Synchronous Single-Stream deepsparse: NLP Question Answering, BERT base uncased SQuaD 12layer Pruned90 - Asynchronous Multi-Stream deepsparse: CV Classification, ResNet-50 ImageNet - Synchronous Single-Stream deepsparse: CV Classification, ResNet-50 ImageNet - Synchronous Single-Stream deepsparse: NLP Token Classification, BERT base uncased conll2003 - Synchronous Single-Stream deepsparse: NLP Token Classification, BERT base uncased conll2003 - Synchronous Single-Stream a b c 396372.41 991178.5 906109 910150 809375.81 865980.81 815381.38 889078.56 909570.5 878271.56 703037.06 306581.06 743582.94 814690.5 965959.56 643467.5 739677.81 43.7176 45.7066 245915.34 547585.19 241319.73 791389.69 4.3781 456.7458 605935.75 558752.88 107.1765 18.6457 93.1713 10.7315 146.1429 13.6817 225.1197 4.4416 1.9804 235377.19 1007.5455 41.6174 24.0178 58.2053 34.3354 1.9802 8.6291 231.3358 58.4197 17.1147 80.9337 12.3534 1007.9012 185.64 124.9982 7.9995 502.7354 1.9891 10.7539 33.6523 29.7036 504.4064 1.9825 641659.12 649747.25 604763.12 604353.75 682435.5 700074.25 696563.19 742060 768804.94 768793.12 614001.69 333883.12 696805.88 758575.69 919540.25 618727.69 716722.56 42.468 47.0387 252201.72 535200.12 238656.66 800922.62 4.3125 463.5304 596281.62 553452.44 106.4621 18.7697 93.0114 10.75 146.6365 13.6358 226.366 4.4172 1.9855 234181.08 1005.8884 41.4098 24.1386 58.1527 34.3542 1.9823 8.6367 231.2106 58.6125 17.0586 81.0418 12.3371 1007.6702 186.0784 124.9007 8.0057 503.5758 1.9858 10.7429 33.6395 29.7149 504.1199 1.9836 663182.75 618965.12 596153.62 616424 642269 711207.19 677947.75 752604.06 856061.75 750593 609369.62 300563.84 679338.62 759612.94 908611.81 644712.06 735434.75 43.4887 45.9597 248416.36 537871.56 235957.02 783883.38 4.391 455.3891 603624.19 560846.19 106.2527 18.8072 92.3732 10.8242 147.4004 13.5651 227.0317 4.4042 1.9956 233903.91 1001.2858 41.3882 24.1502 57.9441 34.4836 1.9885 8.6032 232.0997 58.4757 17.0984 81.1917 12.3143 1004.9938 186.122 124.7501 8.0153 503.499 1.9861 10.7395 33.6288 29.7237 504.3356 1.9828 OpenBenchmarking.org
KeyDB Test: SET - Parallel Connections: 500 OpenBenchmarking.org Requests Per Second, More Is Better KeyDB 6.3.2 Test: SET - Parallel Connections: 500 a b c 140K 280K 420K 560K 700K 396372.41 641659.12 663182.75 1. (CXX) g++ options: -ggdb -rdynamic -lm -lz -lcrypto -lbz2 -lzstd -llz4 -lsnappy -latomic -ldl -pthread -lrt -luuid -lcurl -lssl -std=c++14 -pedantic -fno-rtti -O2 -flto -MMD
KeyDB Test: LPOP - Parallel Connections: 50 OpenBenchmarking.org Requests Per Second, More Is Better KeyDB 6.3.2 Test: LPOP - Parallel Connections: 50 a b c 200K 400K 600K 800K 1000K 991178.50 649747.25 618965.12 1. (CXX) g++ options: -ggdb -rdynamic -lm -lz -lcrypto -lbz2 -lzstd -llz4 -lsnappy -latomic -ldl -pthread -lrt -luuid -lcurl -lssl -std=c++14 -pedantic -fno-rtti -O2 -flto -MMD
KeyDB Test: LPOP - Parallel Connections: 500 OpenBenchmarking.org Requests Per Second, More Is Better KeyDB 6.3.2 Test: LPOP - Parallel Connections: 500 a b c 200K 400K 600K 800K 1000K 906109.00 604763.12 596153.62 1. (CXX) g++ options: -ggdb -rdynamic -lm -lz -lcrypto -lbz2 -lzstd -llz4 -lsnappy -latomic -ldl -pthread -lrt -luuid -lcurl -lssl -std=c++14 -pedantic -fno-rtti -O2 -flto -MMD
KeyDB Test: LPOP - Parallel Connections: 100 OpenBenchmarking.org Requests Per Second, More Is Better KeyDB 6.3.2 Test: LPOP - Parallel Connections: 100 a b c 200K 400K 600K 800K 1000K 910150.00 604353.75 616424.00 1. (CXX) g++ options: -ggdb -rdynamic -lm -lz -lcrypto -lbz2 -lzstd -llz4 -lsnappy -latomic -ldl -pthread -lrt -luuid -lcurl -lssl -std=c++14 -pedantic -fno-rtti -O2 -flto -MMD
KeyDB Test: LPOP - Parallel Connections: 900 OpenBenchmarking.org Requests Per Second, More Is Better KeyDB 6.3.2 Test: LPOP - Parallel Connections: 900 a b c 200K 400K 600K 800K 1000K 809375.81 682435.50 642269.00 1. (CXX) g++ options: -ggdb -rdynamic -lm -lz -lcrypto -lbz2 -lzstd -llz4 -lsnappy -latomic -ldl -pthread -lrt -luuid -lcurl -lssl -std=c++14 -pedantic -fno-rtti -O2 -flto -MMD
KeyDB Test: GET - Parallel Connections: 900 OpenBenchmarking.org Requests Per Second, More Is Better KeyDB 6.3.2 Test: GET - Parallel Connections: 900 a b c 200K 400K 600K 800K 1000K 865980.81 700074.25 711207.19 1. (CXX) g++ options: -ggdb -rdynamic -lm -lz -lcrypto -lbz2 -lzstd -llz4 -lsnappy -latomic -ldl -pthread -lrt -luuid -lcurl -lssl -std=c++14 -pedantic -fno-rtti -O2 -flto -MMD
KeyDB Test: SET - Parallel Connections: 100 OpenBenchmarking.org Requests Per Second, More Is Better KeyDB 6.3.2 Test: SET - Parallel Connections: 100 a b c 200K 400K 600K 800K 1000K 815381.38 696563.19 677947.75 1. (CXX) g++ options: -ggdb -rdynamic -lm -lz -lcrypto -lbz2 -lzstd -llz4 -lsnappy -latomic -ldl -pthread -lrt -luuid -lcurl -lssl -std=c++14 -pedantic -fno-rtti -O2 -flto -MMD
KeyDB Test: GET - Parallel Connections: 500 OpenBenchmarking.org Requests Per Second, More Is Better KeyDB 6.3.2 Test: GET - Parallel Connections: 500 a b c 200K 400K 600K 800K 1000K 889078.56 742060.00 752604.06 1. (CXX) g++ options: -ggdb -rdynamic -lm -lz -lcrypto -lbz2 -lzstd -llz4 -lsnappy -latomic -ldl -pthread -lrt -luuid -lcurl -lssl -std=c++14 -pedantic -fno-rtti -O2 -flto -MMD
KeyDB Test: GET - Parallel Connections: 100 OpenBenchmarking.org Requests Per Second, More Is Better KeyDB 6.3.2 Test: GET - Parallel Connections: 100 a b c 200K 400K 600K 800K 1000K 909570.50 768804.94 856061.75 1. (CXX) g++ options: -ggdb -rdynamic -lm -lz -lcrypto -lbz2 -lzstd -llz4 -lsnappy -latomic -ldl -pthread -lrt -luuid -lcurl -lssl -std=c++14 -pedantic -fno-rtti -O2 -flto -MMD
KeyDB Test: SADD - Parallel Connections: 50 OpenBenchmarking.org Requests Per Second, More Is Better KeyDB 6.3.2 Test: SADD - Parallel Connections: 50 a b c 200K 400K 600K 800K 1000K 878271.56 768793.12 750593.00 1. (CXX) g++ options: -ggdb -rdynamic -lm -lz -lcrypto -lbz2 -lzstd -llz4 -lsnappy -latomic -ldl -pthread -lrt -luuid -lcurl -lssl -std=c++14 -pedantic -fno-rtti -O2 -flto -MMD
KeyDB Test: SET - Parallel Connections: 900 OpenBenchmarking.org Requests Per Second, More Is Better KeyDB 6.3.2 Test: SET - Parallel Connections: 900 a b c 150K 300K 450K 600K 750K 703037.06 614001.69 609369.62 1. (CXX) g++ options: -ggdb -rdynamic -lm -lz -lcrypto -lbz2 -lzstd -llz4 -lsnappy -latomic -ldl -pthread -lrt -luuid -lcurl -lssl -std=c++14 -pedantic -fno-rtti -O2 -flto -MMD
KeyDB Test: HMSET - Parallel Connections: 50 OpenBenchmarking.org Requests Per Second, More Is Better KeyDB 6.3.2 Test: HMSET - Parallel Connections: 50 a b c 70K 140K 210K 280K 350K 306581.06 333883.12 300563.84 1. (CXX) g++ options: -ggdb -rdynamic -lm -lz -lcrypto -lbz2 -lzstd -llz4 -lsnappy -latomic -ldl -pthread -lrt -luuid -lcurl -lssl -std=c++14 -pedantic -fno-rtti -O2 -flto -MMD
KeyDB Test: SADD - Parallel Connections: 900 OpenBenchmarking.org Requests Per Second, More Is Better KeyDB 6.3.2 Test: SADD - Parallel Connections: 900 a b c 160K 320K 480K 640K 800K 743582.94 696805.88 679338.62 1. (CXX) g++ options: -ggdb -rdynamic -lm -lz -lcrypto -lbz2 -lzstd -llz4 -lsnappy -latomic -ldl -pthread -lrt -luuid -lcurl -lssl -std=c++14 -pedantic -fno-rtti -O2 -flto -MMD
KeyDB Test: SADD - Parallel Connections: 100 OpenBenchmarking.org Requests Per Second, More Is Better KeyDB 6.3.2 Test: SADD - Parallel Connections: 100 a b c 200K 400K 600K 800K 1000K 814690.50 758575.69 759612.94 1. (CXX) g++ options: -ggdb -rdynamic -lm -lz -lcrypto -lbz2 -lzstd -llz4 -lsnappy -latomic -ldl -pthread -lrt -luuid -lcurl -lssl -std=c++14 -pedantic -fno-rtti -O2 -flto -MMD
KeyDB Test: GET - Parallel Connections: 50 OpenBenchmarking.org Requests Per Second, More Is Better KeyDB 6.3.2 Test: GET - Parallel Connections: 50 a b c 200K 400K 600K 800K 1000K 965959.56 919540.25 908611.81 1. (CXX) g++ options: -ggdb -rdynamic -lm -lz -lcrypto -lbz2 -lzstd -llz4 -lsnappy -latomic -ldl -pthread -lrt -luuid -lcurl -lssl -std=c++14 -pedantic -fno-rtti -O2 -flto -MMD
KeyDB Test: LPUSH - Parallel Connections: 50 OpenBenchmarking.org Requests Per Second, More Is Better KeyDB 6.3.2 Test: LPUSH - Parallel Connections: 50 a b c 140K 280K 420K 560K 700K 643467.50 618727.69 644712.06 1. (CXX) g++ options: -ggdb -rdynamic -lm -lz -lcrypto -lbz2 -lzstd -llz4 -lsnappy -latomic -ldl -pthread -lrt -luuid -lcurl -lssl -std=c++14 -pedantic -fno-rtti -O2 -flto -MMD
KeyDB Test: SADD - Parallel Connections: 500 OpenBenchmarking.org Requests Per Second, More Is Better KeyDB 6.3.2 Test: SADD - Parallel Connections: 500 a b c 160K 320K 480K 640K 800K 739677.81 716722.56 735434.75 1. (CXX) g++ options: -ggdb -rdynamic -lm -lz -lcrypto -lbz2 -lzstd -llz4 -lsnappy -latomic -ldl -pthread -lrt -luuid -lcurl -lssl -std=c++14 -pedantic -fno-rtti -O2 -flto -MMD
Neural Magic DeepSparse Model: NLP Sentiment Analysis, 80% Pruned Quantized BERT Base Uncased - Scenario: Asynchronous Multi-Stream OpenBenchmarking.org items/sec, More Is Better Neural Magic DeepSparse 1.3.2 Model: NLP Sentiment Analysis, 80% Pruned Quantized BERT Base Uncased - Scenario: Asynchronous Multi-Stream a b c 10 20 30 40 50 43.72 42.47 43.49
Neural Magic DeepSparse Model: NLP Sentiment Analysis, 80% Pruned Quantized BERT Base Uncased - Scenario: Asynchronous Multi-Stream OpenBenchmarking.org ms/batch, Fewer Is Better Neural Magic DeepSparse 1.3.2 Model: NLP Sentiment Analysis, 80% Pruned Quantized BERT Base Uncased - Scenario: Asynchronous Multi-Stream a b c 11 22 33 44 55 45.71 47.04 45.96
KeyDB Test: HMSET - Parallel Connections: 100 OpenBenchmarking.org Requests Per Second, More Is Better KeyDB 6.3.2 Test: HMSET - Parallel Connections: 100 a b c 50K 100K 150K 200K 250K 245915.34 252201.72 248416.36 1. (CXX) g++ options: -ggdb -rdynamic -lm -lz -lcrypto -lbz2 -lzstd -llz4 -lsnappy -latomic -ldl -pthread -lrt -luuid -lcurl -lssl -std=c++14 -pedantic -fno-rtti -O2 -flto -MMD
KeyDB Test: LPUSH - Parallel Connections: 900 OpenBenchmarking.org Requests Per Second, More Is Better KeyDB 6.3.2 Test: LPUSH - Parallel Connections: 900 a b c 120K 240K 360K 480K 600K 547585.19 535200.12 537871.56 1. (CXX) g++ options: -ggdb -rdynamic -lm -lz -lcrypto -lbz2 -lzstd -llz4 -lsnappy -latomic -ldl -pthread -lrt -luuid -lcurl -lssl -std=c++14 -pedantic -fno-rtti -O2 -flto -MMD
KeyDB Test: HMSET - Parallel Connections: 500 OpenBenchmarking.org Requests Per Second, More Is Better KeyDB 6.3.2 Test: HMSET - Parallel Connections: 500 a b c 50K 100K 150K 200K 250K 241319.73 238656.66 235957.02 1. (CXX) g++ options: -ggdb -rdynamic -lm -lz -lcrypto -lbz2 -lzstd -llz4 -lsnappy -latomic -ldl -pthread -lrt -luuid -lcurl -lssl -std=c++14 -pedantic -fno-rtti -O2 -flto -MMD
KeyDB Test: SET - Parallel Connections: 50 OpenBenchmarking.org Requests Per Second, More Is Better KeyDB 6.3.2 Test: SET - Parallel Connections: 50 a b c 200K 400K 600K 800K 1000K 791389.69 800922.62 783883.38 1. (CXX) g++ options: -ggdb -rdynamic -lm -lz -lcrypto -lbz2 -lzstd -llz4 -lsnappy -latomic -ldl -pthread -lrt -luuid -lcurl -lssl -std=c++14 -pedantic -fno-rtti -O2 -flto -MMD
Neural Magic DeepSparse Model: CV Segmentation, 90% Pruned YOLACT Pruned - Scenario: Asynchronous Multi-Stream OpenBenchmarking.org items/sec, More Is Better Neural Magic DeepSparse 1.3.2 Model: CV Segmentation, 90% Pruned YOLACT Pruned - Scenario: Asynchronous Multi-Stream a b c 0.988 1.976 2.964 3.952 4.94 4.3781 4.3125 4.3910
Neural Magic DeepSparse Model: CV Segmentation, 90% Pruned YOLACT Pruned - Scenario: Asynchronous Multi-Stream OpenBenchmarking.org ms/batch, Fewer Is Better Neural Magic DeepSparse 1.3.2 Model: CV Segmentation, 90% Pruned YOLACT Pruned - Scenario: Asynchronous Multi-Stream a b c 100 200 300 400 500 456.75 463.53 455.39
KeyDB Test: LPUSH - Parallel Connections: 100 OpenBenchmarking.org Requests Per Second, More Is Better KeyDB 6.3.2 Test: LPUSH - Parallel Connections: 100 a b c 130K 260K 390K 520K 650K 605935.75 596281.62 603624.19 1. (CXX) g++ options: -ggdb -rdynamic -lm -lz -lcrypto -lbz2 -lzstd -llz4 -lsnappy -latomic -ldl -pthread -lrt -luuid -lcurl -lssl -std=c++14 -pedantic -fno-rtti -O2 -flto -MMD
KeyDB Test: LPUSH - Parallel Connections: 500 OpenBenchmarking.org Requests Per Second, More Is Better KeyDB 6.3.2 Test: LPUSH - Parallel Connections: 500 a b c 120K 240K 360K 480K 600K 558752.88 553452.44 560846.19 1. (CXX) g++ options: -ggdb -rdynamic -lm -lz -lcrypto -lbz2 -lzstd -llz4 -lsnappy -latomic -ldl -pthread -lrt -luuid -lcurl -lssl -std=c++14 -pedantic -fno-rtti -O2 -flto -MMD
Neural Magic DeepSparse Model: NLP Text Classification, DistilBERT mnli - Scenario: Asynchronous Multi-Stream OpenBenchmarking.org ms/batch, Fewer Is Better Neural Magic DeepSparse 1.3.2 Model: NLP Text Classification, DistilBERT mnli - Scenario: Asynchronous Multi-Stream a b c 20 40 60 80 100 107.18 106.46 106.25
Neural Magic DeepSparse Model: NLP Text Classification, DistilBERT mnli - Scenario: Asynchronous Multi-Stream OpenBenchmarking.org items/sec, More Is Better Neural Magic DeepSparse 1.3.2 Model: NLP Text Classification, DistilBERT mnli - Scenario: Asynchronous Multi-Stream a b c 5 10 15 20 25 18.65 18.77 18.81
Neural Magic DeepSparse Model: NLP Question Answering, BERT base uncased SQuaD 12layer Pruned90 - Scenario: Synchronous Single-Stream OpenBenchmarking.org ms/batch, Fewer Is Better Neural Magic DeepSparse 1.3.2 Model: NLP Question Answering, BERT base uncased SQuaD 12layer Pruned90 - Scenario: Synchronous Single-Stream a b c 20 40 60 80 100 93.17 93.01 92.37
Neural Magic DeepSparse Model: NLP Question Answering, BERT base uncased SQuaD 12layer Pruned90 - Scenario: Synchronous Single-Stream OpenBenchmarking.org items/sec, More Is Better Neural Magic DeepSparse 1.3.2 Model: NLP Question Answering, BERT base uncased SQuaD 12layer Pruned90 - Scenario: Synchronous Single-Stream a b c 3 6 9 12 15 10.73 10.75 10.82
Neural Magic DeepSparse Model: CV Detection, YOLOv5s COCO - Scenario: Asynchronous Multi-Stream OpenBenchmarking.org ms/batch, Fewer Is Better Neural Magic DeepSparse 1.3.2 Model: CV Detection, YOLOv5s COCO - Scenario: Asynchronous Multi-Stream a b c 30 60 90 120 150 146.14 146.64 147.40
Neural Magic DeepSparse Model: CV Detection, YOLOv5s COCO - Scenario: Asynchronous Multi-Stream OpenBenchmarking.org items/sec, More Is Better Neural Magic DeepSparse 1.3.2 Model: CV Detection, YOLOv5s COCO - Scenario: Asynchronous Multi-Stream a b c 4 8 12 16 20 13.68 13.64 13.57
Neural Magic DeepSparse Model: CV Segmentation, 90% Pruned YOLACT Pruned - Scenario: Synchronous Single-Stream OpenBenchmarking.org ms/batch, Fewer Is Better Neural Magic DeepSparse 1.3.2 Model: CV Segmentation, 90% Pruned YOLACT Pruned - Scenario: Synchronous Single-Stream a b c 50 100 150 200 250 225.12 226.37 227.03
Neural Magic DeepSparse Model: CV Segmentation, 90% Pruned YOLACT Pruned - Scenario: Synchronous Single-Stream OpenBenchmarking.org items/sec, More Is Better Neural Magic DeepSparse 1.3.2 Model: CV Segmentation, 90% Pruned YOLACT Pruned - Scenario: Synchronous Single-Stream a b c 0.9994 1.9988 2.9982 3.9976 4.997 4.4416 4.4172 4.4042
Neural Magic DeepSparse Model: NLP Document Classification, oBERT base uncased on IMDB - Scenario: Asynchronous Multi-Stream OpenBenchmarking.org items/sec, More Is Better Neural Magic DeepSparse 1.3.2 Model: NLP Document Classification, oBERT base uncased on IMDB - Scenario: Asynchronous Multi-Stream a b c 0.449 0.898 1.347 1.796 2.245 1.9804 1.9855 1.9956
KeyDB Test: HMSET - Parallel Connections: 900 OpenBenchmarking.org Requests Per Second, More Is Better KeyDB 6.3.2 Test: HMSET - Parallel Connections: 900 a b c 50K 100K 150K 200K 250K 235377.19 234181.08 233903.91 1. (CXX) g++ options: -ggdb -rdynamic -lm -lz -lcrypto -lbz2 -lzstd -llz4 -lsnappy -latomic -ldl -pthread -lrt -luuid -lcurl -lssl -std=c++14 -pedantic -fno-rtti -O2 -flto -MMD
Neural Magic DeepSparse Model: NLP Document Classification, oBERT base uncased on IMDB - Scenario: Asynchronous Multi-Stream OpenBenchmarking.org ms/batch, Fewer Is Better Neural Magic DeepSparse 1.3.2 Model: NLP Document Classification, oBERT base uncased on IMDB - Scenario: Asynchronous Multi-Stream a b c 200 400 600 800 1000 1007.55 1005.89 1001.29
Neural Magic DeepSparse Model: NLP Sentiment Analysis, 80% Pruned Quantized BERT Base Uncased - Scenario: Synchronous Single-Stream OpenBenchmarking.org items/sec, More Is Better Neural Magic DeepSparse 1.3.2 Model: NLP Sentiment Analysis, 80% Pruned Quantized BERT Base Uncased - Scenario: Synchronous Single-Stream a b c 9 18 27 36 45 41.62 41.41 41.39
Neural Magic DeepSparse Model: NLP Sentiment Analysis, 80% Pruned Quantized BERT Base Uncased - Scenario: Synchronous Single-Stream OpenBenchmarking.org ms/batch, Fewer Is Better Neural Magic DeepSparse 1.3.2 Model: NLP Sentiment Analysis, 80% Pruned Quantized BERT Base Uncased - Scenario: Synchronous Single-Stream a b c 6 12 18 24 30 24.02 24.14 24.15
Neural Magic DeepSparse Model: CV Classification, ResNet-50 ImageNet - Scenario: Asynchronous Multi-Stream OpenBenchmarking.org ms/batch, Fewer Is Better Neural Magic DeepSparse 1.3.2 Model: CV Classification, ResNet-50 ImageNet - Scenario: Asynchronous Multi-Stream a b c 13 26 39 52 65 58.21 58.15 57.94
Neural Magic DeepSparse Model: CV Classification, ResNet-50 ImageNet - Scenario: Asynchronous Multi-Stream OpenBenchmarking.org items/sec, More Is Better Neural Magic DeepSparse 1.3.2 Model: CV Classification, ResNet-50 ImageNet - Scenario: Asynchronous Multi-Stream a b c 8 16 24 32 40 34.34 34.35 34.48
Neural Magic DeepSparse Model: NLP Token Classification, BERT base uncased conll2003 - Scenario: Asynchronous Multi-Stream OpenBenchmarking.org items/sec, More Is Better Neural Magic DeepSparse 1.3.2 Model: NLP Token Classification, BERT base uncased conll2003 - Scenario: Asynchronous Multi-Stream a b c 0.4474 0.8948 1.3422 1.7896 2.237 1.9802 1.9823 1.9885
Neural Magic DeepSparse Model: NLP Text Classification, BERT base uncased SST2 - Scenario: Asynchronous Multi-Stream OpenBenchmarking.org items/sec, More Is Better Neural Magic DeepSparse 1.3.2 Model: NLP Text Classification, BERT base uncased SST2 - Scenario: Asynchronous Multi-Stream a b c 2 4 6 8 10 8.6291 8.6367 8.6032
Neural Magic DeepSparse Model: NLP Text Classification, BERT base uncased SST2 - Scenario: Asynchronous Multi-Stream OpenBenchmarking.org ms/batch, Fewer Is Better Neural Magic DeepSparse 1.3.2 Model: NLP Text Classification, BERT base uncased SST2 - Scenario: Asynchronous Multi-Stream a b c 50 100 150 200 250 231.34 231.21 232.10
Neural Magic DeepSparse Model: NLP Text Classification, DistilBERT mnli - Scenario: Synchronous Single-Stream OpenBenchmarking.org ms/batch, Fewer Is Better Neural Magic DeepSparse 1.3.2 Model: NLP Text Classification, DistilBERT mnli - Scenario: Synchronous Single-Stream a b c 13 26 39 52 65 58.42 58.61 58.48
Neural Magic DeepSparse Model: NLP Text Classification, DistilBERT mnli - Scenario: Synchronous Single-Stream OpenBenchmarking.org items/sec, More Is Better Neural Magic DeepSparse 1.3.2 Model: NLP Text Classification, DistilBERT mnli - Scenario: Synchronous Single-Stream a b c 4 8 12 16 20 17.11 17.06 17.10
Neural Magic DeepSparse Model: CV Detection, YOLOv5s COCO - Scenario: Synchronous Single-Stream OpenBenchmarking.org ms/batch, Fewer Is Better Neural Magic DeepSparse 1.3.2 Model: CV Detection, YOLOv5s COCO - Scenario: Synchronous Single-Stream a b c 20 40 60 80 100 80.93 81.04 81.19
Neural Magic DeepSparse Model: CV Detection, YOLOv5s COCO - Scenario: Synchronous Single-Stream OpenBenchmarking.org items/sec, More Is Better Neural Magic DeepSparse 1.3.2 Model: CV Detection, YOLOv5s COCO - Scenario: Synchronous Single-Stream a b c 3 6 9 12 15 12.35 12.34 12.31
Neural Magic DeepSparse Model: NLP Token Classification, BERT base uncased conll2003 - Scenario: Asynchronous Multi-Stream OpenBenchmarking.org ms/batch, Fewer Is Better Neural Magic DeepSparse 1.3.2 Model: NLP Token Classification, BERT base uncased conll2003 - Scenario: Asynchronous Multi-Stream a b c 200 400 600 800 1000 1007.90 1007.67 1004.99
Neural Magic DeepSparse Model: NLP Question Answering, BERT base uncased SQuaD 12layer Pruned90 - Scenario: Asynchronous Multi-Stream OpenBenchmarking.org ms/batch, Fewer Is Better Neural Magic DeepSparse 1.3.2 Model: NLP Question Answering, BERT base uncased SQuaD 12layer Pruned90 - Scenario: Asynchronous Multi-Stream a b c 40 80 120 160 200 185.64 186.08 186.12
Neural Magic DeepSparse Model: NLP Text Classification, BERT base uncased SST2 - Scenario: Synchronous Single-Stream OpenBenchmarking.org ms/batch, Fewer Is Better Neural Magic DeepSparse 1.3.2 Model: NLP Text Classification, BERT base uncased SST2 - Scenario: Synchronous Single-Stream a b c 30 60 90 120 150 125.00 124.90 124.75
Neural Magic DeepSparse Model: NLP Text Classification, BERT base uncased SST2 - Scenario: Synchronous Single-Stream OpenBenchmarking.org items/sec, More Is Better Neural Magic DeepSparse 1.3.2 Model: NLP Text Classification, BERT base uncased SST2 - Scenario: Synchronous Single-Stream a b c 2 4 6 8 10 7.9995 8.0057 8.0153
Neural Magic DeepSparse Model: NLP Document Classification, oBERT base uncased on IMDB - Scenario: Synchronous Single-Stream OpenBenchmarking.org ms/batch, Fewer Is Better Neural Magic DeepSparse 1.3.2 Model: NLP Document Classification, oBERT base uncased on IMDB - Scenario: Synchronous Single-Stream a b c 110 220 330 440 550 502.74 503.58 503.50
Neural Magic DeepSparse Model: NLP Document Classification, oBERT base uncased on IMDB - Scenario: Synchronous Single-Stream OpenBenchmarking.org items/sec, More Is Better Neural Magic DeepSparse 1.3.2 Model: NLP Document Classification, oBERT base uncased on IMDB - Scenario: Synchronous Single-Stream a b c 0.4475 0.895 1.3425 1.79 2.2375 1.9891 1.9858 1.9861
Neural Magic DeepSparse Model: NLP Question Answering, BERT base uncased SQuaD 12layer Pruned90 - Scenario: Asynchronous Multi-Stream OpenBenchmarking.org items/sec, More Is Better Neural Magic DeepSparse 1.3.2 Model: NLP Question Answering, BERT base uncased SQuaD 12layer Pruned90 - Scenario: Asynchronous Multi-Stream a b c 3 6 9 12 15 10.75 10.74 10.74
Neural Magic DeepSparse Model: CV Classification, ResNet-50 ImageNet - Scenario: Synchronous Single-Stream OpenBenchmarking.org items/sec, More Is Better Neural Magic DeepSparse 1.3.2 Model: CV Classification, ResNet-50 ImageNet - Scenario: Synchronous Single-Stream a b c 8 16 24 32 40 33.65 33.64 33.63
Neural Magic DeepSparse Model: CV Classification, ResNet-50 ImageNet - Scenario: Synchronous Single-Stream OpenBenchmarking.org ms/batch, Fewer Is Better Neural Magic DeepSparse 1.3.2 Model: CV Classification, ResNet-50 ImageNet - Scenario: Synchronous Single-Stream a b c 7 14 21 28 35 29.70 29.71 29.72
Neural Magic DeepSparse Model: NLP Token Classification, BERT base uncased conll2003 - Scenario: Synchronous Single-Stream OpenBenchmarking.org ms/batch, Fewer Is Better Neural Magic DeepSparse 1.3.2 Model: NLP Token Classification, BERT base uncased conll2003 - Scenario: Synchronous Single-Stream a b c 110 220 330 440 550 504.41 504.12 504.34
Neural Magic DeepSparse Model: NLP Token Classification, BERT base uncased conll2003 - Scenario: Synchronous Single-Stream OpenBenchmarking.org items/sec, More Is Better Neural Magic DeepSparse 1.3.2 Model: NLP Token Classification, BERT base uncased conll2003 - Scenario: Synchronous Single-Stream a b c 0.4463 0.8926 1.3389 1.7852 2.2315 1.9825 1.9836 1.9828
Phoronix Test Suite v10.8.5