Intel Core i9-13900K testing with a ASUS PRIME Z790-P WIFI (0602 BIOS) and AMD Radeon RX 6700 XT 12GB on Ubuntu 22.04 via the Phoronix Test Suite.
Compare your own system(s) to this result file with the
Phoronix Test Suite by running the command:
phoronix-test-suite benchmark 2301230-PTS-NEWTESTS60 new tests - Phoronix Test Suite new tests Intel Core i9-13900K testing with a ASUS PRIME Z790-P WIFI (0602 BIOS) and AMD Radeon RX 6700 XT 12GB on Ubuntu 22.04 via the Phoronix Test Suite.
HTML result view exported from: https://openbenchmarking.org/result/2301230-PTS-NEWTESTS60&grr&export=txt&sor&rro .
new tests Processor Motherboard Chipset Memory Disk Graphics Audio Monitor Network OS Kernel Desktop Display Server OpenGL Vulkan Compiler File-System Screen Resolution a b c d Intel Core i9-13900K @ 4.00GHz (24 Cores / 32 Threads) ASUS PRIME Z790-P WIFI (0602 BIOS) Intel Device 7a27 32GB 1000GB Western Digital WDS100T1X0E-00AFY0 AMD Radeon RX 6700 XT 12GB (2855/1000MHz) Realtek ALC897 ASUS VP28U Intel Device 7a70 Ubuntu 22.04 6.0.0-060000-generic (x86_64) GNOME Shell 42.4 X Server 1.21.1.3 + Wayland 4.6 Mesa 22.3.0-devel (git-8945375 2022-10-04 jammy-oibaf-ppa) (LLVM 14.0.6 DRM 3.48) 1.3.230 GCC 11.2.0 ext4 3840x2160 OpenBenchmarking.org Kernel Details - Transparent Huge Pages: madvise Compiler Details - --build=x86_64-linux-gnu --disable-vtable-verify --disable-werror --enable-bootstrap --enable-cet --enable-checking=release --enable-clocale=gnu --enable-default-pie --enable-gnu-unique-object --enable-languages=c,ada,c++,go,brig,d,fortran,objc,obj-c++,m2 --enable-libphobos-checking=release --enable-libstdcxx-debug --enable-libstdcxx-time=yes --enable-link-serialization=2 --enable-multiarch --enable-multilib --enable-nls --enable-objc-gc=auto --enable-offload-targets=nvptx-none=/build/gcc-11-gBFGDP/gcc-11-11.2.0/debian/tmp-nvptx/usr,amdgcn-amdhsa=/build/gcc-11-gBFGDP/gcc-11-11.2.0/debian/tmp-gcn/usr --enable-plugin --enable-shared --enable-threads=posix --host=x86_64-linux-gnu --program-prefix=x86_64-linux-gnu- --target=x86_64-linux-gnu --with-abi=m64 --with-arch-32=i686 --with-build-config=bootstrap-lto-lean --with-default-libstdcxx-abi=new --with-gcc-major-version-only --with-multilib-list=m32,m64,mx32 --with-target-system-zlib=auto --with-tune=generic --without-cuda-driver -v Processor Details - Scaling Governor: intel_pstate powersave (EPP: balance_performance) - CPU Microcode: 0x10e - Thermald 2.4.9 Python Details - Python 3.10.6 Security Details - itlb_multihit: Not affected + l1tf: Not affected + mds: Not affected + meltdown: Not affected + mmio_stale_data: Not affected + retbleed: Not affected + spec_store_bypass: Mitigation of SSB disabled via prctl + spectre_v1: Mitigation of usercopy/swapgs barriers and __user pointer sanitization + spectre_v2: Mitigation of Enhanced IBRS IBPB: conditional RSB filling PBRSB-eIBRS: SW sequence + srbds: Not affected + tsx_async_abort: Not affected
new tests keydb: HMSET - 900 keydb: HMSET - 500 keydb: HMSET - 100 keydb: LPUSH - 500 keydb: LPUSH - 100 keydb: SET - 900 keydb: SADD - 900 keydb: LPOP - 500 keydb: LPOP - 100 keydb: LPOP - 900 keydb: SADD - 500 keydb: GET - 500 keydb: HMSET - 50 deepsparse: NLP Document Classification, oBERT base uncased on IMDB - Asynchronous Multi-Stream deepsparse: NLP Document Classification, oBERT base uncased on IMDB - Asynchronous Multi-Stream deepsparse: NLP Token Classification, BERT base uncased conll2003 - Asynchronous Multi-Stream deepsparse: NLP Token Classification, BERT base uncased conll2003 - Asynchronous Multi-Stream deepsparse: NLP Text Classification, BERT base uncased SST2 - Asynchronous Multi-Stream deepsparse: NLP Text Classification, BERT base uncased SST2 - Asynchronous Multi-Stream keydb: LPUSH - 900 keydb: SADD - 100 deepsparse: CV Segmentation, 90% Pruned YOLACT Pruned - Asynchronous Multi-Stream deepsparse: CV Segmentation, 90% Pruned YOLACT Pruned - Asynchronous Multi-Stream keydb: SET - 100 keydb: SET - 500 deepsparse: NLP Question Answering, BERT base uncased SQuaD 12layer Pruned90 - Asynchronous Multi-Stream deepsparse: NLP Question Answering, BERT base uncased SQuaD 12layer Pruned90 - Asynchronous Multi-Stream deepsparse: NLP Sentiment Analysis, 80% Pruned Quantized BERT Base Uncased - Asynchronous Multi-Stream deepsparse: NLP Sentiment Analysis, 80% Pruned Quantized BERT Base Uncased - Asynchronous Multi-Stream deepsparse: NLP Document Classification, oBERT base uncased on IMDB - Synchronous Single-Stream deepsparse: NLP Document Classification, oBERT base uncased on IMDB - Synchronous Single-Stream deepsparse: NLP Text Classification, BERT base uncased SST2 - Synchronous Single-Stream deepsparse: NLP Text Classification, BERT base uncased SST2 - Synchronous Single-Stream deepsparse: NLP Token Classification, BERT base uncased conll2003 - Synchronous Single-Stream deepsparse: NLP Token Classification, BERT base uncased conll2003 - Synchronous Single-Stream deepsparse: NLP Text Classification, DistilBERT mnli - Asynchronous Multi-Stream deepsparse: NLP Text Classification, DistilBERT mnli - Asynchronous Multi-Stream deepsparse: CV Segmentation, 90% Pruned YOLACT Pruned - Synchronous Single-Stream deepsparse: CV Segmentation, 90% Pruned YOLACT Pruned - Synchronous Single-Stream keydb: GET - 100 keydb: GET - 900 keydb: LPUSH - 50 deepsparse: NLP Sentiment Analysis, 80% Pruned Quantized BERT Base Uncased - Synchronous Single-Stream deepsparse: NLP Sentiment Analysis, 80% Pruned Quantized BERT Base Uncased - Synchronous Single-Stream deepsparse: NLP Question Answering, BERT base uncased SQuaD 12layer Pruned90 - Synchronous Single-Stream deepsparse: NLP Question Answering, BERT base uncased SQuaD 12layer Pruned90 - Synchronous Single-Stream deepsparse: CV Detection, YOLOv5s COCO - Asynchronous Multi-Stream deepsparse: CV Detection, YOLOv5s COCO - Asynchronous Multi-Stream deepsparse: NLP Text Classification, DistilBERT mnli - Synchronous Single-Stream deepsparse: NLP Text Classification, DistilBERT mnli - Synchronous Single-Stream deepsparse: CV Classification, ResNet-50 ImageNet - Asynchronous Multi-Stream deepsparse: CV Classification, ResNet-50 ImageNet - Asynchronous Multi-Stream keydb: SET - 50 deepsparse: CV Detection, YOLOv5s COCO - Synchronous Single-Stream deepsparse: CV Detection, YOLOv5s COCO - Synchronous Single-Stream deepsparse: CV Classification, ResNet-50 ImageNet - Synchronous Single-Stream deepsparse: CV Classification, ResNet-50 ImageNet - Synchronous Single-Stream keydb: LPOP - 50 keydb: SADD - 50 keydb: GET - 50 a b c d 405828.74 540612.07 811193.33 972928.70 1079455.03 1227964.69 1318361.98 1541362.82 1548101.63 1572282.98 1369793.36 1493932.89 770419.02 693.7534 17.0977 694.2030 17.0593 160.4842 74.6763 913352.58 1301147.20 531.4908 22.4583 1277181.12 1242079.00 191.4059 62.4727 59.3747 201.8858 65.9984 15.1510 23.5749 42.4124 65.8471 15.1861 81.6431 146.8362 55.7950 17.9198 1526251.21 1563299.29 1885806.56 10.3007 97.0266 22.5114 44.4092 118.9735 100.5956 11.8712 84.2139 54.4516 220.2242 2051262.03 14.7283 67.8527 7.9509 125.7040 2355768.58 2240005.25 2301457.08 385409.94 575162.19 838926.19 932940.25 1009102.12 1200999.25 1402721.25 1142413.25 1082813.62 1132092.5 1365262.25 1558846.38 827088.81 697.4591 17.0299 693.6961 17.2641 161.8553 73.963 988728.5 1326330.25 530.1764 22.572 1180776.88 1314786.12 192.5598 62.0325 59.0448 202.8721 66.0079 15.1488 23.5757 42.4107 65.7826 15.2008 81.6896 146.7638 55.6313 17.9723 1482228 1582378.62 1849522.75 10.3459 96.6004 22.5427 44.3471 117.1808 102.1854 11.8541 84.3354 53.7491 223.1123 2074172.38 14.607 68.4113 7.8702 126.9919 2032602.88 2210042.5 2203905.25 373870.94 544525.88 858045.06 1020158.31 1053252.5 1209599.38 1331770.75 1166752.25 1133555.5 1171454 1294096.25 1476886.75 822828.56 696.4586 17.1322 695.2334 17.0024 159.443 75.2056 967361.25 1443334.75 528.6955 22.4829 1105998.88 1188354.12 191.3873 62.4335 59.1069 202.7414 66.0331 15.1431 23.4753 42.5922 66.1119 15.1247 80.4175 148.9785 56.0887 17.8257 1488316.62 1527463.75 1964096.25 10.2299 97.6971 22.616 44.2034 118.6662 100.922 11.8147 84.6155 53.8481 222.5816 2007790.25 14.6466 68.2268 7.8877 126.7132 2014748 2215133.75 2258764 405919.94 514398.03 762636.94 925805.94 1161008.62 1305926.38 1264958.12 1101928.38 1110666.75 1168661.12 1296243.38 1553518.62 781066.94 693.8845 17.2573 693.8329 17.2405 161.0202 74.4384 934195.31 1235605.25 523.9449 22.8771 1296008.25 1307531.38 189.9321 62.9368 58.8037 203.7202 65.9748 15.1564 23.57 42.4206 65.8907 15.1758 81.7629 146.7079 55.85 17.902 1484384.38 1510802.12 1816596.38 10.2988 97.0456 22.5261 44.3803 117.9462 101.5808 11.8028 84.7035 54.3784 220.629 2013693.12 14.4335 69.2308 7.8787 126.8482 2016942.25 2186557 2273450.75 OpenBenchmarking.org
KeyDB Test: HMSET - Parallel Connections: 900 OpenBenchmarking.org Requests Per Second, More Is Better KeyDB 6.3.2 Test: HMSET - Parallel Connections: 900 c b a d 90K 180K 270K 360K 450K SE +/- 8190.09, N = 12 373870.94 385409.94 405828.74 405919.94 1. (CXX) g++ options: -ggdb -rdynamic -lm -lz -lcrypto -lbz2 -lzstd -llz4 -lsnappy -latomic -ldl -pthread -lrt -luuid -lcurl -lssl -std=c++14 -pedantic -fno-rtti -O2 -flto -MMD
KeyDB Test: HMSET - Parallel Connections: 500 OpenBenchmarking.org Requests Per Second, More Is Better KeyDB 6.3.2 Test: HMSET - Parallel Connections: 500 d a c b 120K 240K 360K 480K 600K SE +/- 11687.20, N = 15 514398.03 540612.07 544525.88 575162.19 1. (CXX) g++ options: -ggdb -rdynamic -lm -lz -lcrypto -lbz2 -lzstd -llz4 -lsnappy -latomic -ldl -pthread -lrt -luuid -lcurl -lssl -std=c++14 -pedantic -fno-rtti -O2 -flto -MMD
KeyDB Test: HMSET - Parallel Connections: 100 OpenBenchmarking.org Requests Per Second, More Is Better KeyDB 6.3.2 Test: HMSET - Parallel Connections: 100 d a b c 200K 400K 600K 800K 1000K SE +/- 7152.65, N = 15 762636.94 811193.33 838926.19 858045.06 1. (CXX) g++ options: -ggdb -rdynamic -lm -lz -lcrypto -lbz2 -lzstd -llz4 -lsnappy -latomic -ldl -pthread -lrt -luuid -lcurl -lssl -std=c++14 -pedantic -fno-rtti -O2 -flto -MMD
KeyDB Test: LPUSH - Parallel Connections: 500 OpenBenchmarking.org Requests Per Second, More Is Better KeyDB 6.3.2 Test: LPUSH - Parallel Connections: 500 d b a c 200K 400K 600K 800K 1000K SE +/- 11803.30, N = 15 925805.94 932940.25 972928.70 1020158.31 1. (CXX) g++ options: -ggdb -rdynamic -lm -lz -lcrypto -lbz2 -lzstd -llz4 -lsnappy -latomic -ldl -pthread -lrt -luuid -lcurl -lssl -std=c++14 -pedantic -fno-rtti -O2 -flto -MMD
KeyDB Test: LPUSH - Parallel Connections: 100 OpenBenchmarking.org Requests Per Second, More Is Better KeyDB 6.3.2 Test: LPUSH - Parallel Connections: 100 b c a d 200K 400K 600K 800K 1000K SE +/- 19016.70, N = 15 1009102.12 1053252.50 1079455.03 1161008.62 1. (CXX) g++ options: -ggdb -rdynamic -lm -lz -lcrypto -lbz2 -lzstd -llz4 -lsnappy -latomic -ldl -pthread -lrt -luuid -lcurl -lssl -std=c++14 -pedantic -fno-rtti -O2 -flto -MMD
KeyDB Test: SET - Parallel Connections: 900 OpenBenchmarking.org Requests Per Second, More Is Better KeyDB 6.3.2 Test: SET - Parallel Connections: 900 b c a d 300K 600K 900K 1200K 1500K SE +/- 10170.24, N = 15 1200999.25 1209599.38 1227964.69 1305926.38 1. (CXX) g++ options: -ggdb -rdynamic -lm -lz -lcrypto -lbz2 -lzstd -llz4 -lsnappy -latomic -ldl -pthread -lrt -luuid -lcurl -lssl -std=c++14 -pedantic -fno-rtti -O2 -flto -MMD
KeyDB Test: SADD - Parallel Connections: 900 OpenBenchmarking.org Requests Per Second, More Is Better KeyDB 6.3.2 Test: SADD - Parallel Connections: 900 d a c b 300K 600K 900K 1200K 1500K SE +/- 13077.27, N = 15 1264958.12 1318361.98 1331770.75 1402721.25 1. (CXX) g++ options: -ggdb -rdynamic -lm -lz -lcrypto -lbz2 -lzstd -llz4 -lsnappy -latomic -ldl -pthread -lrt -luuid -lcurl -lssl -std=c++14 -pedantic -fno-rtti -O2 -flto -MMD
KeyDB Test: LPOP - Parallel Connections: 500 OpenBenchmarking.org Requests Per Second, More Is Better KeyDB 6.3.2 Test: LPOP - Parallel Connections: 500 d b c a 300K 600K 900K 1200K 1500K SE +/- 15335.07, N = 15 1101928.38 1142413.25 1166752.25 1541362.82 1. (CXX) g++ options: -ggdb -rdynamic -lm -lz -lcrypto -lbz2 -lzstd -llz4 -lsnappy -latomic -ldl -pthread -lrt -luuid -lcurl -lssl -std=c++14 -pedantic -fno-rtti -O2 -flto -MMD
KeyDB Test: LPOP - Parallel Connections: 100 OpenBenchmarking.org Requests Per Second, More Is Better KeyDB 6.3.2 Test: LPOP - Parallel Connections: 100 b d c a 300K 600K 900K 1200K 1500K SE +/- 18631.04, N = 15 1082813.62 1110666.75 1133555.50 1548101.63 1. (CXX) g++ options: -ggdb -rdynamic -lm -lz -lcrypto -lbz2 -lzstd -llz4 -lsnappy -latomic -ldl -pthread -lrt -luuid -lcurl -lssl -std=c++14 -pedantic -fno-rtti -O2 -flto -MMD
KeyDB Test: LPOP - Parallel Connections: 900 OpenBenchmarking.org Requests Per Second, More Is Better KeyDB 6.3.2 Test: LPOP - Parallel Connections: 900 b d c a 300K 600K 900K 1200K 1500K SE +/- 21774.31, N = 15 1132092.50 1168661.12 1171454.00 1572282.98 1. (CXX) g++ options: -ggdb -rdynamic -lm -lz -lcrypto -lbz2 -lzstd -llz4 -lsnappy -latomic -ldl -pthread -lrt -luuid -lcurl -lssl -std=c++14 -pedantic -fno-rtti -O2 -flto -MMD
KeyDB Test: SADD - Parallel Connections: 500 OpenBenchmarking.org Requests Per Second, More Is Better KeyDB 6.3.2 Test: SADD - Parallel Connections: 500 c d b a 300K 600K 900K 1200K 1500K SE +/- 20006.28, N = 12 1294096.25 1296243.38 1365262.25 1369793.36 1. (CXX) g++ options: -ggdb -rdynamic -lm -lz -lcrypto -lbz2 -lzstd -llz4 -lsnappy -latomic -ldl -pthread -lrt -luuid -lcurl -lssl -std=c++14 -pedantic -fno-rtti -O2 -flto -MMD
KeyDB Test: GET - Parallel Connections: 500 OpenBenchmarking.org Requests Per Second, More Is Better KeyDB 6.3.2 Test: GET - Parallel Connections: 500 c a d b 300K 600K 900K 1200K 1500K SE +/- 21838.17, N = 12 1476886.75 1493932.89 1553518.62 1558846.38 1. (CXX) g++ options: -ggdb -rdynamic -lm -lz -lcrypto -lbz2 -lzstd -llz4 -lsnappy -latomic -ldl -pthread -lrt -luuid -lcurl -lssl -std=c++14 -pedantic -fno-rtti -O2 -flto -MMD
KeyDB Test: HMSET - Parallel Connections: 50 OpenBenchmarking.org Requests Per Second, More Is Better KeyDB 6.3.2 Test: HMSET - Parallel Connections: 50 a d c b 200K 400K 600K 800K 1000K SE +/- 10277.06, N = 3 770419.02 781066.94 822828.56 827088.81 1. (CXX) g++ options: -ggdb -rdynamic -lm -lz -lcrypto -lbz2 -lzstd -llz4 -lsnappy -latomic -ldl -pthread -lrt -luuid -lcurl -lssl -std=c++14 -pedantic -fno-rtti -O2 -flto -MMD
Neural Magic DeepSparse Model: NLP Document Classification, oBERT base uncased on IMDB - Scenario: Asynchronous Multi-Stream OpenBenchmarking.org ms/batch, Fewer Is Better Neural Magic DeepSparse 1.3.2 Model: NLP Document Classification, oBERT base uncased on IMDB - Scenario: Asynchronous Multi-Stream b c d a 150 300 450 600 750 SE +/- 1.58, N = 3 697.46 696.46 693.88 693.75
Neural Magic DeepSparse Model: NLP Document Classification, oBERT base uncased on IMDB - Scenario: Asynchronous Multi-Stream OpenBenchmarking.org items/sec, More Is Better Neural Magic DeepSparse 1.3.2 Model: NLP Document Classification, oBERT base uncased on IMDB - Scenario: Asynchronous Multi-Stream b a c d 4 8 12 16 20 SE +/- 0.09, N = 3 17.03 17.10 17.13 17.26
Neural Magic DeepSparse Model: NLP Token Classification, BERT base uncased conll2003 - Scenario: Asynchronous Multi-Stream OpenBenchmarking.org ms/batch, Fewer Is Better Neural Magic DeepSparse 1.3.2 Model: NLP Token Classification, BERT base uncased conll2003 - Scenario: Asynchronous Multi-Stream c a d b 150 300 450 600 750 SE +/- 2.02, N = 3 695.23 694.20 693.83 693.70
Neural Magic DeepSparse Model: NLP Token Classification, BERT base uncased conll2003 - Scenario: Asynchronous Multi-Stream OpenBenchmarking.org items/sec, More Is Better Neural Magic DeepSparse 1.3.2 Model: NLP Token Classification, BERT base uncased conll2003 - Scenario: Asynchronous Multi-Stream c a d b 4 8 12 16 20 SE +/- 0.06, N = 3 17.00 17.06 17.24 17.26
Neural Magic DeepSparse Model: NLP Text Classification, BERT base uncased SST2 - Scenario: Asynchronous Multi-Stream OpenBenchmarking.org ms/batch, Fewer Is Better Neural Magic DeepSparse 1.3.2 Model: NLP Text Classification, BERT base uncased SST2 - Scenario: Asynchronous Multi-Stream b d a c 40 80 120 160 200 SE +/- 0.25, N = 3 161.86 161.02 160.48 159.44
Neural Magic DeepSparse Model: NLP Text Classification, BERT base uncased SST2 - Scenario: Asynchronous Multi-Stream OpenBenchmarking.org items/sec, More Is Better Neural Magic DeepSparse 1.3.2 Model: NLP Text Classification, BERT base uncased SST2 - Scenario: Asynchronous Multi-Stream b d a c 20 40 60 80 100 SE +/- 0.11, N = 3 73.96 74.44 74.68 75.21
KeyDB Test: LPUSH - Parallel Connections: 900 OpenBenchmarking.org Requests Per Second, More Is Better KeyDB 6.3.2 Test: LPUSH - Parallel Connections: 900 a d c b 200K 400K 600K 800K 1000K SE +/- 7796.97, N = 3 913352.58 934195.31 967361.25 988728.50 1. (CXX) g++ options: -ggdb -rdynamic -lm -lz -lcrypto -lbz2 -lzstd -llz4 -lsnappy -latomic -ldl -pthread -lrt -luuid -lcurl -lssl -std=c++14 -pedantic -fno-rtti -O2 -flto -MMD
KeyDB Test: SADD - Parallel Connections: 100 OpenBenchmarking.org Requests Per Second, More Is Better KeyDB 6.3.2 Test: SADD - Parallel Connections: 100 d a b c 300K 600K 900K 1200K 1500K SE +/- 13610.75, N = 5 1235605.25 1301147.20 1326330.25 1443334.75 1. (CXX) g++ options: -ggdb -rdynamic -lm -lz -lcrypto -lbz2 -lzstd -llz4 -lsnappy -latomic -ldl -pthread -lrt -luuid -lcurl -lssl -std=c++14 -pedantic -fno-rtti -O2 -flto -MMD
Neural Magic DeepSparse Model: CV Segmentation, 90% Pruned YOLACT Pruned - Scenario: Asynchronous Multi-Stream OpenBenchmarking.org ms/batch, Fewer Is Better Neural Magic DeepSparse 1.3.2 Model: CV Segmentation, 90% Pruned YOLACT Pruned - Scenario: Asynchronous Multi-Stream a b c d 110 220 330 440 550 SE +/- 0.36, N = 3 531.49 530.18 528.70 523.94
Neural Magic DeepSparse Model: CV Segmentation, 90% Pruned YOLACT Pruned - Scenario: Asynchronous Multi-Stream OpenBenchmarking.org items/sec, More Is Better Neural Magic DeepSparse 1.3.2 Model: CV Segmentation, 90% Pruned YOLACT Pruned - Scenario: Asynchronous Multi-Stream a c b d 5 10 15 20 25 SE +/- 0.04, N = 3 22.46 22.48 22.57 22.88
KeyDB Test: SET - Parallel Connections: 100 OpenBenchmarking.org Requests Per Second, More Is Better KeyDB 6.3.2 Test: SET - Parallel Connections: 100 c b a d 300K 600K 900K 1200K 1500K SE +/- 6390.36, N = 3 1105998.88 1180776.88 1277181.12 1296008.25 1. (CXX) g++ options: -ggdb -rdynamic -lm -lz -lcrypto -lbz2 -lzstd -llz4 -lsnappy -latomic -ldl -pthread -lrt -luuid -lcurl -lssl -std=c++14 -pedantic -fno-rtti -O2 -flto -MMD
KeyDB Test: SET - Parallel Connections: 500 OpenBenchmarking.org Requests Per Second, More Is Better KeyDB 6.3.2 Test: SET - Parallel Connections: 500 c a d b 300K 600K 900K 1200K 1500K SE +/- 7859.44, N = 3 1188354.12 1242079.00 1307531.38 1314786.12 1. (CXX) g++ options: -ggdb -rdynamic -lm -lz -lcrypto -lbz2 -lzstd -llz4 -lsnappy -latomic -ldl -pthread -lrt -luuid -lcurl -lssl -std=c++14 -pedantic -fno-rtti -O2 -flto -MMD
Neural Magic DeepSparse Model: NLP Question Answering, BERT base uncased SQuaD 12layer Pruned90 - Scenario: Asynchronous Multi-Stream OpenBenchmarking.org ms/batch, Fewer Is Better Neural Magic DeepSparse 1.3.2 Model: NLP Question Answering, BERT base uncased SQuaD 12layer Pruned90 - Scenario: Asynchronous Multi-Stream b a c d 40 80 120 160 200 SE +/- 0.42, N = 3 192.56 191.41 191.39 189.93
Neural Magic DeepSparse Model: NLP Question Answering, BERT base uncased SQuaD 12layer Pruned90 - Scenario: Asynchronous Multi-Stream OpenBenchmarking.org items/sec, More Is Better Neural Magic DeepSparse 1.3.2 Model: NLP Question Answering, BERT base uncased SQuaD 12layer Pruned90 - Scenario: Asynchronous Multi-Stream b c a d 14 28 42 56 70 SE +/- 0.15, N = 3 62.03 62.43 62.47 62.94
Neural Magic DeepSparse Model: NLP Sentiment Analysis, 80% Pruned Quantized BERT Base Uncased - Scenario: Asynchronous Multi-Stream OpenBenchmarking.org ms/batch, Fewer Is Better Neural Magic DeepSparse 1.3.2 Model: NLP Sentiment Analysis, 80% Pruned Quantized BERT Base Uncased - Scenario: Asynchronous Multi-Stream a c b d 13 26 39 52 65 SE +/- 0.65, N = 3 59.37 59.11 59.04 58.80
Neural Magic DeepSparse Model: NLP Sentiment Analysis, 80% Pruned Quantized BERT Base Uncased - Scenario: Asynchronous Multi-Stream OpenBenchmarking.org items/sec, More Is Better Neural Magic DeepSparse 1.3.2 Model: NLP Sentiment Analysis, 80% Pruned Quantized BERT Base Uncased - Scenario: Asynchronous Multi-Stream a c b d 40 80 120 160 200 SE +/- 2.21, N = 3 201.89 202.74 202.87 203.72
Neural Magic DeepSparse Model: NLP Document Classification, oBERT base uncased on IMDB - Scenario: Synchronous Single-Stream OpenBenchmarking.org ms/batch, Fewer Is Better Neural Magic DeepSparse 1.3.2 Model: NLP Document Classification, oBERT base uncased on IMDB - Scenario: Synchronous Single-Stream c b a d 15 30 45 60 75 SE +/- 0.08, N = 3 66.03 66.01 66.00 65.97
Neural Magic DeepSparse Model: NLP Document Classification, oBERT base uncased on IMDB - Scenario: Synchronous Single-Stream OpenBenchmarking.org items/sec, More Is Better Neural Magic DeepSparse 1.3.2 Model: NLP Document Classification, oBERT base uncased on IMDB - Scenario: Synchronous Single-Stream c b a d 4 8 12 16 20 SE +/- 0.02, N = 3 15.14 15.15 15.15 15.16
Neural Magic DeepSparse Model: NLP Text Classification, BERT base uncased SST2 - Scenario: Synchronous Single-Stream OpenBenchmarking.org ms/batch, Fewer Is Better Neural Magic DeepSparse 1.3.2 Model: NLP Text Classification, BERT base uncased SST2 - Scenario: Synchronous Single-Stream b a d c 6 12 18 24 30 SE +/- 0.04, N = 3 23.58 23.57 23.57 23.48
Neural Magic DeepSparse Model: NLP Text Classification, BERT base uncased SST2 - Scenario: Synchronous Single-Stream OpenBenchmarking.org items/sec, More Is Better Neural Magic DeepSparse 1.3.2 Model: NLP Text Classification, BERT base uncased SST2 - Scenario: Synchronous Single-Stream b a d c 10 20 30 40 50 SE +/- 0.08, N = 3 42.41 42.41 42.42 42.59
Neural Magic DeepSparse Model: NLP Token Classification, BERT base uncased conll2003 - Scenario: Synchronous Single-Stream OpenBenchmarking.org ms/batch, Fewer Is Better Neural Magic DeepSparse 1.3.2 Model: NLP Token Classification, BERT base uncased conll2003 - Scenario: Synchronous Single-Stream c d a b 15 30 45 60 75 SE +/- 0.18, N = 3 66.11 65.89 65.85 65.78
Neural Magic DeepSparse Model: NLP Token Classification, BERT base uncased conll2003 - Scenario: Synchronous Single-Stream OpenBenchmarking.org items/sec, More Is Better Neural Magic DeepSparse 1.3.2 Model: NLP Token Classification, BERT base uncased conll2003 - Scenario: Synchronous Single-Stream c d a b 4 8 12 16 20 SE +/- 0.04, N = 3 15.12 15.18 15.19 15.20
Neural Magic DeepSparse Model: NLP Text Classification, DistilBERT mnli - Scenario: Asynchronous Multi-Stream OpenBenchmarking.org ms/batch, Fewer Is Better Neural Magic DeepSparse 1.3.2 Model: NLP Text Classification, DistilBERT mnli - Scenario: Asynchronous Multi-Stream d b a c 20 40 60 80 100 SE +/- 0.08, N = 3 81.76 81.69 81.64 80.42
Neural Magic DeepSparse Model: NLP Text Classification, DistilBERT mnli - Scenario: Asynchronous Multi-Stream OpenBenchmarking.org items/sec, More Is Better Neural Magic DeepSparse 1.3.2 Model: NLP Text Classification, DistilBERT mnli - Scenario: Asynchronous Multi-Stream d b a c 30 60 90 120 150 SE +/- 0.16, N = 3 146.71 146.76 146.84 148.98
Neural Magic DeepSparse Model: CV Segmentation, 90% Pruned YOLACT Pruned - Scenario: Synchronous Single-Stream OpenBenchmarking.org ms/batch, Fewer Is Better Neural Magic DeepSparse 1.3.2 Model: CV Segmentation, 90% Pruned YOLACT Pruned - Scenario: Synchronous Single-Stream c d a b 13 26 39 52 65 SE +/- 0.14, N = 3 56.09 55.85 55.80 55.63
Neural Magic DeepSparse Model: CV Segmentation, 90% Pruned YOLACT Pruned - Scenario: Synchronous Single-Stream OpenBenchmarking.org items/sec, More Is Better Neural Magic DeepSparse 1.3.2 Model: CV Segmentation, 90% Pruned YOLACT Pruned - Scenario: Synchronous Single-Stream c d a b 4 8 12 16 20 SE +/- 0.04, N = 3 17.83 17.90 17.92 17.97
KeyDB Test: GET - Parallel Connections: 100 OpenBenchmarking.org Requests Per Second, More Is Better KeyDB 6.3.2 Test: GET - Parallel Connections: 100 b d c a 300K 600K 900K 1200K 1500K SE +/- 12834.74, N = 3 1482228.00 1484384.38 1488316.62 1526251.21 1. (CXX) g++ options: -ggdb -rdynamic -lm -lz -lcrypto -lbz2 -lzstd -llz4 -lsnappy -latomic -ldl -pthread -lrt -luuid -lcurl -lssl -std=c++14 -pedantic -fno-rtti -O2 -flto -MMD
KeyDB Test: GET - Parallel Connections: 900 OpenBenchmarking.org Requests Per Second, More Is Better KeyDB 6.3.2 Test: GET - Parallel Connections: 900 d c a b 300K 600K 900K 1200K 1500K SE +/- 20017.86, N = 3 1510802.12 1527463.75 1563299.29 1582378.62 1. (CXX) g++ options: -ggdb -rdynamic -lm -lz -lcrypto -lbz2 -lzstd -llz4 -lsnappy -latomic -ldl -pthread -lrt -luuid -lcurl -lssl -std=c++14 -pedantic -fno-rtti -O2 -flto -MMD
KeyDB Test: LPUSH - Parallel Connections: 50 OpenBenchmarking.org Requests Per Second, More Is Better KeyDB 6.3.2 Test: LPUSH - Parallel Connections: 50 d b a c 400K 800K 1200K 1600K 2000K SE +/- 22045.54, N = 4 1816596.38 1849522.75 1885806.56 1964096.25 1. (CXX) g++ options: -ggdb -rdynamic -lm -lz -lcrypto -lbz2 -lzstd -llz4 -lsnappy -latomic -ldl -pthread -lrt -luuid -lcurl -lssl -std=c++14 -pedantic -fno-rtti -O2 -flto -MMD
Neural Magic DeepSparse Model: NLP Sentiment Analysis, 80% Pruned Quantized BERT Base Uncased - Scenario: Synchronous Single-Stream OpenBenchmarking.org ms/batch, Fewer Is Better Neural Magic DeepSparse 1.3.2 Model: NLP Sentiment Analysis, 80% Pruned Quantized BERT Base Uncased - Scenario: Synchronous Single-Stream b a d c 3 6 9 12 15 SE +/- 0.02, N = 3 10.35 10.30 10.30 10.23
Neural Magic DeepSparse Model: NLP Sentiment Analysis, 80% Pruned Quantized BERT Base Uncased - Scenario: Synchronous Single-Stream OpenBenchmarking.org items/sec, More Is Better Neural Magic DeepSparse 1.3.2 Model: NLP Sentiment Analysis, 80% Pruned Quantized BERT Base Uncased - Scenario: Synchronous Single-Stream b a d c 20 40 60 80 100 SE +/- 0.18, N = 3 96.60 97.03 97.05 97.70
Neural Magic DeepSparse Model: NLP Question Answering, BERT base uncased SQuaD 12layer Pruned90 - Scenario: Synchronous Single-Stream OpenBenchmarking.org ms/batch, Fewer Is Better Neural Magic DeepSparse 1.3.2 Model: NLP Question Answering, BERT base uncased SQuaD 12layer Pruned90 - Scenario: Synchronous Single-Stream c b d a 5 10 15 20 25 SE +/- 0.04, N = 3 22.62 22.54 22.53 22.51
Neural Magic DeepSparse Model: NLP Question Answering, BERT base uncased SQuaD 12layer Pruned90 - Scenario: Synchronous Single-Stream OpenBenchmarking.org items/sec, More Is Better Neural Magic DeepSparse 1.3.2 Model: NLP Question Answering, BERT base uncased SQuaD 12layer Pruned90 - Scenario: Synchronous Single-Stream c b d a 10 20 30 40 50 SE +/- 0.08, N = 3 44.20 44.35 44.38 44.41
Neural Magic DeepSparse Model: CV Detection, YOLOv5s COCO - Scenario: Asynchronous Multi-Stream OpenBenchmarking.org ms/batch, Fewer Is Better Neural Magic DeepSparse 1.3.2 Model: CV Detection, YOLOv5s COCO - Scenario: Asynchronous Multi-Stream a c d b 30 60 90 120 150 SE +/- 0.61, N = 3 118.97 118.67 117.95 117.18
Neural Magic DeepSparse Model: CV Detection, YOLOv5s COCO - Scenario: Asynchronous Multi-Stream OpenBenchmarking.org items/sec, More Is Better Neural Magic DeepSparse 1.3.2 Model: CV Detection, YOLOv5s COCO - Scenario: Asynchronous Multi-Stream a c d b 20 40 60 80 100 SE +/- 0.45, N = 3 100.60 100.92 101.58 102.19
Neural Magic DeepSparse Model: NLP Text Classification, DistilBERT mnli - Scenario: Synchronous Single-Stream OpenBenchmarking.org ms/batch, Fewer Is Better Neural Magic DeepSparse 1.3.2 Model: NLP Text Classification, DistilBERT mnli - Scenario: Synchronous Single-Stream a b c d 3 6 9 12 15 SE +/- 0.03, N = 3 11.87 11.85 11.81 11.80
Neural Magic DeepSparse Model: NLP Text Classification, DistilBERT mnli - Scenario: Synchronous Single-Stream OpenBenchmarking.org items/sec, More Is Better Neural Magic DeepSparse 1.3.2 Model: NLP Text Classification, DistilBERT mnli - Scenario: Synchronous Single-Stream a b c d 20 40 60 80 100 SE +/- 0.18, N = 3 84.21 84.34 84.62 84.70
Neural Magic DeepSparse Model: CV Classification, ResNet-50 ImageNet - Scenario: Asynchronous Multi-Stream OpenBenchmarking.org ms/batch, Fewer Is Better Neural Magic DeepSparse 1.3.2 Model: CV Classification, ResNet-50 ImageNet - Scenario: Asynchronous Multi-Stream a d c b 12 24 36 48 60 SE +/- 0.36, N = 3 54.45 54.38 53.85 53.75
Neural Magic DeepSparse Model: CV Classification, ResNet-50 ImageNet - Scenario: Asynchronous Multi-Stream OpenBenchmarking.org items/sec, More Is Better Neural Magic DeepSparse 1.3.2 Model: CV Classification, ResNet-50 ImageNet - Scenario: Asynchronous Multi-Stream a d c b 50 100 150 200 250 SE +/- 1.42, N = 3 220.22 220.63 222.58 223.11
KeyDB Test: SET - Parallel Connections: 50 OpenBenchmarking.org Requests Per Second, More Is Better KeyDB 6.3.2 Test: SET - Parallel Connections: 50 c d a b 400K 800K 1200K 1600K 2000K SE +/- 22564.98, N = 4 2007790.25 2013693.12 2051262.03 2074172.38 1. (CXX) g++ options: -ggdb -rdynamic -lm -lz -lcrypto -lbz2 -lzstd -llz4 -lsnappy -latomic -ldl -pthread -lrt -luuid -lcurl -lssl -std=c++14 -pedantic -fno-rtti -O2 -flto -MMD
Neural Magic DeepSparse Model: CV Detection, YOLOv5s COCO - Scenario: Synchronous Single-Stream OpenBenchmarking.org ms/batch, Fewer Is Better Neural Magic DeepSparse 1.3.2 Model: CV Detection, YOLOv5s COCO - Scenario: Synchronous Single-Stream a c b d 4 8 12 16 20 SE +/- 0.07, N = 3 14.73 14.65 14.61 14.43
Neural Magic DeepSparse Model: CV Detection, YOLOv5s COCO - Scenario: Synchronous Single-Stream OpenBenchmarking.org items/sec, More Is Better Neural Magic DeepSparse 1.3.2 Model: CV Detection, YOLOv5s COCO - Scenario: Synchronous Single-Stream a c b d 15 30 45 60 75 SE +/- 0.33, N = 3 67.85 68.23 68.41 69.23
Neural Magic DeepSparse Model: CV Classification, ResNet-50 ImageNet - Scenario: Synchronous Single-Stream OpenBenchmarking.org ms/batch, Fewer Is Better Neural Magic DeepSparse 1.3.2 Model: CV Classification, ResNet-50 ImageNet - Scenario: Synchronous Single-Stream a c d b 2 4 6 8 10 SE +/- 0.0210, N = 3 7.9509 7.8877 7.8787 7.8702
Neural Magic DeepSparse Model: CV Classification, ResNet-50 ImageNet - Scenario: Synchronous Single-Stream OpenBenchmarking.org items/sec, More Is Better Neural Magic DeepSparse 1.3.2 Model: CV Classification, ResNet-50 ImageNet - Scenario: Synchronous Single-Stream a c d b 30 60 90 120 150 SE +/- 0.33, N = 3 125.70 126.71 126.85 126.99
KeyDB Test: LPOP - Parallel Connections: 50 OpenBenchmarking.org Requests Per Second, More Is Better KeyDB 6.3.2 Test: LPOP - Parallel Connections: 50 c d b a 500K 1000K 1500K 2000K 2500K SE +/- 25251.29, N = 3 2014748.00 2016942.25 2032602.88 2355768.58 1. (CXX) g++ options: -ggdb -rdynamic -lm -lz -lcrypto -lbz2 -lzstd -llz4 -lsnappy -latomic -ldl -pthread -lrt -luuid -lcurl -lssl -std=c++14 -pedantic -fno-rtti -O2 -flto -MMD
KeyDB Test: SADD - Parallel Connections: 50 OpenBenchmarking.org Requests Per Second, More Is Better KeyDB 6.3.2 Test: SADD - Parallel Connections: 50 d b c a 500K 1000K 1500K 2000K 2500K SE +/- 14787.63, N = 3 2186557.00 2210042.50 2215133.75 2240005.25 1. (CXX) g++ options: -ggdb -rdynamic -lm -lz -lcrypto -lbz2 -lzstd -llz4 -lsnappy -latomic -ldl -pthread -lrt -luuid -lcurl -lssl -std=c++14 -pedantic -fno-rtti -O2 -flto -MMD
KeyDB Test: GET - Parallel Connections: 50 OpenBenchmarking.org Requests Per Second, More Is Better KeyDB 6.3.2 Test: GET - Parallel Connections: 50 b c d a 500K 1000K 1500K 2000K 2500K SE +/- 12561.87, N = 3 2203905.25 2258764.00 2273450.75 2301457.08 1. (CXX) g++ options: -ggdb -rdynamic -lm -lz -lcrypto -lbz2 -lzstd -llz4 -lsnappy -latomic -ldl -pthread -lrt -luuid -lcurl -lssl -std=c++14 -pedantic -fno-rtti -O2 -flto -MMD
Phoronix Test Suite v10.8.4