new tests

Intel Core i9-13900K testing with a ASUS PRIME Z790-P WIFI (0602 BIOS) and AMD Radeon RX 6700 XT 12GB on Ubuntu 22.04 via the Phoronix Test Suite.

HTML result view exported from: https://openbenchmarking.org/result/2301230-PTS-NEWTESTS60&export=txt&grs&sor&rro.

new testsProcessorMotherboardChipsetMemoryDiskGraphicsAudioMonitorNetworkOSKernelDesktopDisplay ServerOpenGLVulkanCompilerFile-SystemScreen ResolutionabcdIntel Core i9-13900K @ 4.00GHz (24 Cores / 32 Threads)ASUS PRIME Z790-P WIFI (0602 BIOS)Intel Device 7a2732GB1000GB Western Digital WDS100T1X0E-00AFY0AMD Radeon RX 6700 XT 12GB (2855/1000MHz)Realtek ALC897ASUS VP28UIntel Device 7a70Ubuntu 22.046.0.0-060000-generic (x86_64)GNOME Shell 42.4X Server 1.21.1.3 + Wayland4.6 Mesa 22.3.0-devel (git-8945375 2022-10-04 jammy-oibaf-ppa) (LLVM 14.0.6 DRM 3.48)1.3.230GCC 11.2.0ext43840x2160OpenBenchmarking.orgKernel Details- Transparent Huge Pages: madviseCompiler Details- --build=x86_64-linux-gnu --disable-vtable-verify --disable-werror --enable-bootstrap --enable-cet --enable-checking=release --enable-clocale=gnu --enable-default-pie --enable-gnu-unique-object --enable-languages=c,ada,c++,go,brig,d,fortran,objc,obj-c++,m2 --enable-libphobos-checking=release --enable-libstdcxx-debug --enable-libstdcxx-time=yes --enable-link-serialization=2 --enable-multiarch --enable-multilib --enable-nls --enable-objc-gc=auto --enable-offload-targets=nvptx-none=/build/gcc-11-gBFGDP/gcc-11-11.2.0/debian/tmp-nvptx/usr,amdgcn-amdhsa=/build/gcc-11-gBFGDP/gcc-11-11.2.0/debian/tmp-gcn/usr --enable-plugin --enable-shared --enable-threads=posix --host=x86_64-linux-gnu --program-prefix=x86_64-linux-gnu- --target=x86_64-linux-gnu --with-abi=m64 --with-arch-32=i686 --with-build-config=bootstrap-lto-lean --with-default-libstdcxx-abi=new --with-gcc-major-version-only --with-multilib-list=m32,m64,mx32 --with-target-system-zlib=auto --with-tune=generic --without-cuda-driver -v Processor Details- Scaling Governor: intel_pstate powersave (EPP: balance_performance) - CPU Microcode: 0x10e - Thermald 2.4.9Python Details- Python 3.10.6Security Details- itlb_multihit: Not affected + l1tf: Not affected + mds: Not affected + meltdown: Not affected + mmio_stale_data: Not affected + retbleed: Not affected + spec_store_bypass: Mitigation of SSB disabled via prctl + spectre_v1: Mitigation of usercopy/swapgs barriers and __user pointer sanitization + spectre_v2: Mitigation of Enhanced IBRS IBPB: conditional RSB filling PBRSB-eIBRS: SW sequence + srbds: Not affected + tsx_async_abort: Not affected

new testskeydb: LPOP - 100keydb: LPOP - 500keydb: LPOP - 900keydb: SET - 100keydb: LPOP - 50keydb: SADD - 100keydb: HMSET - 100keydb: SADD - 900keydb: SET - 500keydb: LPUSH - 500keydb: SET - 900keydb: LPUSH - 900keydb: LPUSH - 50keydb: HMSET - 50keydb: SADD - 500keydb: GET - 500keydb: GET - 900keydb: GET - 50keydb: SET - 50keydb: GET - 100keydb: SADD - 50deepsparse: CV Detection, YOLOv5s COCO - Synchronous Single-Streamdeepsparse: CV Detection, YOLOv5s COCO - Synchronous Single-Streamdeepsparse: CV Segmentation, 90% Pruned YOLACT Pruned - Asynchronous Multi-Streamdeepsparse: NLP Text Classification, BERT base uncased SST2 - Asynchronous Multi-Streamdeepsparse: NLP Text Classification, DistilBERT mnli - Asynchronous Multi-Streamdeepsparse: CV Detection, YOLOv5s COCO - Asynchronous Multi-Streamdeepsparse: NLP Text Classification, DistilBERT mnli - Asynchronous Multi-Streamdeepsparse: NLP Token Classification, BERT base uncased conll2003 - Asynchronous Multi-Streamdeepsparse: CV Detection, YOLOv5s COCO - Asynchronous Multi-Streamdeepsparse: NLP Text Classification, BERT base uncased SST2 - Asynchronous Multi-Streamdeepsparse: NLP Question Answering, BERT base uncased SQuaD 12layer Pruned90 - Asynchronous Multi-Streamdeepsparse: CV Segmentation, 90% Pruned YOLACT Pruned - Asynchronous Multi-Streamdeepsparse: NLP Question Answering, BERT base uncased SQuaD 12layer Pruned90 - Asynchronous Multi-Streamdeepsparse: NLP Document Classification, oBERT base uncased on IMDB - Asynchronous Multi-Streamdeepsparse: CV Classification, ResNet-50 ImageNet - Asynchronous Multi-Streamdeepsparse: CV Classification, ResNet-50 ImageNet - Asynchronous Multi-Streamdeepsparse: NLP Sentiment Analysis, 80% Pruned Quantized BERT Base Uncased - Synchronous Single-Streamdeepsparse: NLP Sentiment Analysis, 80% Pruned Quantized BERT Base Uncased - Synchronous Single-Streamdeepsparse: CV Classification, ResNet-50 ImageNet - Synchronous Single-Streamdeepsparse: CV Classification, ResNet-50 ImageNet - Synchronous Single-Streamdeepsparse: NLP Sentiment Analysis, 80% Pruned Quantized BERT Base Uncased - Asynchronous Multi-Streamdeepsparse: NLP Sentiment Analysis, 80% Pruned Quantized BERT Base Uncased - Asynchronous Multi-Streamdeepsparse: CV Segmentation, 90% Pruned YOLACT Pruned - Synchronous Single-Streamdeepsparse: CV Segmentation, 90% Pruned YOLACT Pruned - Synchronous Single-Streamdeepsparse: NLP Text Classification, DistilBERT mnli - Synchronous Single-Streamdeepsparse: NLP Text Classification, DistilBERT mnli - Synchronous Single-Streamdeepsparse: NLP Document Classification, oBERT base uncased on IMDB - Asynchronous Multi-Streamdeepsparse: NLP Token Classification, BERT base uncased conll2003 - Synchronous Single-Streamdeepsparse: NLP Token Classification, BERT base uncased conll2003 - Synchronous Single-Streamdeepsparse: NLP Question Answering, BERT base uncased SQuaD 12layer Pruned90 - Synchronous Single-Streamdeepsparse: NLP Question Answering, BERT base uncased SQuaD 12layer Pruned90 - Synchronous Single-Streamdeepsparse: NLP Text Classification, BERT base uncased SST2 - Synchronous Single-Streamdeepsparse: NLP Text Classification, BERT base uncased SST2 - Synchronous Single-Streamdeepsparse: NLP Token Classification, BERT base uncased conll2003 - Asynchronous Multi-Streamdeepsparse: NLP Document Classification, oBERT base uncased on IMDB - Synchronous Single-Streamdeepsparse: NLP Document Classification, oBERT base uncased on IMDB - Synchronous Single-Streamkeydb: LPUSH - 100keydb: HMSET - 900keydb: HMSET - 500abcd1548101.631541362.821572282.981277181.122355768.581301147.20811193.331318361.981242079.00972928.701227964.69913352.581885806.56770419.021369793.361493932.891563299.292301457.082051262.031526251.212240005.2514.728367.852722.458374.676381.6431100.5956146.836217.0593118.9735160.484262.4727531.4908191.405917.0977220.224254.451697.026610.30077.9509125.704059.3747201.885817.919855.795084.213911.8712693.753415.186165.847144.409222.511442.412423.5749694.203065.998415.15101079455.03405828.74540612.071082813.621142413.251132092.51180776.882032602.881326330.25838926.191402721.251314786.12932940.251200999.25988728.51849522.75827088.811365262.251558846.381582378.622203905.252074172.3814822282210042.514.60768.411322.57273.96381.6896102.1854146.763817.2641117.1808161.855362.0325530.1764192.559817.0299223.112353.749196.600410.34597.8702126.991959.0448202.872117.972355.631384.335411.8541697.459115.200865.782644.347122.542742.410723.5757693.696166.007915.14881009102.12385409.94575162.191133555.51166752.2511714541105998.8820147481443334.75858045.061331770.751188354.121020158.311209599.38967361.251964096.25822828.561294096.251476886.751527463.7522587642007790.251488316.622215133.7514.646668.226822.482975.205680.4175100.922148.978517.0024118.6662159.44362.4335528.6955191.387317.1322222.581653.848197.697110.22997.8877126.713259.1069202.741417.825756.088784.615511.8147696.458615.124766.111944.203422.61642.592223.4753695.233466.033115.14311053252.5373870.94544525.881110666.751101928.381168661.121296008.252016942.251235605.25762636.941264958.121307531.38925805.941305926.38934195.311816596.38781066.941296243.381553518.621510802.122273450.752013693.121484384.38218655714.433569.230822.877174.438481.7629101.5808146.707917.2405117.9462161.020262.9368523.9449189.932117.2573220.62954.378497.045610.29887.8787126.848258.8037203.720217.90255.8584.703511.8028693.884515.175865.890744.380322.526142.420623.57693.832965.974815.15641161008.62405919.94514398.03OpenBenchmarking.org

KeyDB

Test: LPOP - Parallel Connections: 100

OpenBenchmarking.orgRequests Per Second, More Is BetterKeyDB 6.3.2Test: LPOP - Parallel Connections: 100bdca300K600K900K1200K1500KSE +/- 18631.04, N = 151082813.621110666.751133555.501548101.631. (CXX) g++ options: -ggdb -rdynamic -lm -lz -lcrypto -lbz2 -lzstd -llz4 -lsnappy -latomic -ldl -pthread -lrt -luuid -lcurl -lssl -std=c++14 -pedantic -fno-rtti -O2 -flto -MMD

KeyDB

Test: LPOP - Parallel Connections: 500

OpenBenchmarking.orgRequests Per Second, More Is BetterKeyDB 6.3.2Test: LPOP - Parallel Connections: 500dbca300K600K900K1200K1500KSE +/- 15335.07, N = 151101928.381142413.251166752.251541362.821. (CXX) g++ options: -ggdb -rdynamic -lm -lz -lcrypto -lbz2 -lzstd -llz4 -lsnappy -latomic -ldl -pthread -lrt -luuid -lcurl -lssl -std=c++14 -pedantic -fno-rtti -O2 -flto -MMD

KeyDB

Test: LPOP - Parallel Connections: 900

OpenBenchmarking.orgRequests Per Second, More Is BetterKeyDB 6.3.2Test: LPOP - Parallel Connections: 900bdca300K600K900K1200K1500KSE +/- 21774.31, N = 151132092.501168661.121171454.001572282.981. (CXX) g++ options: -ggdb -rdynamic -lm -lz -lcrypto -lbz2 -lzstd -llz4 -lsnappy -latomic -ldl -pthread -lrt -luuid -lcurl -lssl -std=c++14 -pedantic -fno-rtti -O2 -flto -MMD

KeyDB

Test: SET - Parallel Connections: 100

OpenBenchmarking.orgRequests Per Second, More Is BetterKeyDB 6.3.2Test: SET - Parallel Connections: 100cbad300K600K900K1200K1500KSE +/- 6390.36, N = 31105998.881180776.881277181.121296008.251. (CXX) g++ options: -ggdb -rdynamic -lm -lz -lcrypto -lbz2 -lzstd -llz4 -lsnappy -latomic -ldl -pthread -lrt -luuid -lcurl -lssl -std=c++14 -pedantic -fno-rtti -O2 -flto -MMD

KeyDB

Test: LPOP - Parallel Connections: 50

OpenBenchmarking.orgRequests Per Second, More Is BetterKeyDB 6.3.2Test: LPOP - Parallel Connections: 50cdba500K1000K1500K2000K2500KSE +/- 25251.29, N = 32014748.002016942.252032602.882355768.581. (CXX) g++ options: -ggdb -rdynamic -lm -lz -lcrypto -lbz2 -lzstd -llz4 -lsnappy -latomic -ldl -pthread -lrt -luuid -lcurl -lssl -std=c++14 -pedantic -fno-rtti -O2 -flto -MMD

KeyDB

Test: SADD - Parallel Connections: 100

OpenBenchmarking.orgRequests Per Second, More Is BetterKeyDB 6.3.2Test: SADD - Parallel Connections: 100dabc300K600K900K1200K1500KSE +/- 13610.75, N = 51235605.251301147.201326330.251443334.751. (CXX) g++ options: -ggdb -rdynamic -lm -lz -lcrypto -lbz2 -lzstd -llz4 -lsnappy -latomic -ldl -pthread -lrt -luuid -lcurl -lssl -std=c++14 -pedantic -fno-rtti -O2 -flto -MMD

KeyDB

Test: HMSET - Parallel Connections: 100

OpenBenchmarking.orgRequests Per Second, More Is BetterKeyDB 6.3.2Test: HMSET - Parallel Connections: 100dabc200K400K600K800K1000KSE +/- 7152.65, N = 15762636.94811193.33838926.19858045.061. (CXX) g++ options: -ggdb -rdynamic -lm -lz -lcrypto -lbz2 -lzstd -llz4 -lsnappy -latomic -ldl -pthread -lrt -luuid -lcurl -lssl -std=c++14 -pedantic -fno-rtti -O2 -flto -MMD

KeyDB

Test: SADD - Parallel Connections: 900

OpenBenchmarking.orgRequests Per Second, More Is BetterKeyDB 6.3.2Test: SADD - Parallel Connections: 900dacb300K600K900K1200K1500KSE +/- 13077.27, N = 151264958.121318361.981331770.751402721.251. (CXX) g++ options: -ggdb -rdynamic -lm -lz -lcrypto -lbz2 -lzstd -llz4 -lsnappy -latomic -ldl -pthread -lrt -luuid -lcurl -lssl -std=c++14 -pedantic -fno-rtti -O2 -flto -MMD

KeyDB

Test: SET - Parallel Connections: 500

OpenBenchmarking.orgRequests Per Second, More Is BetterKeyDB 6.3.2Test: SET - Parallel Connections: 500cadb300K600K900K1200K1500KSE +/- 7859.44, N = 31188354.121242079.001307531.381314786.121. (CXX) g++ options: -ggdb -rdynamic -lm -lz -lcrypto -lbz2 -lzstd -llz4 -lsnappy -latomic -ldl -pthread -lrt -luuid -lcurl -lssl -std=c++14 -pedantic -fno-rtti -O2 -flto -MMD

KeyDB

Test: LPUSH - Parallel Connections: 500

OpenBenchmarking.orgRequests Per Second, More Is BetterKeyDB 6.3.2Test: LPUSH - Parallel Connections: 500dbac200K400K600K800K1000KSE +/- 11803.30, N = 15925805.94932940.25972928.701020158.311. (CXX) g++ options: -ggdb -rdynamic -lm -lz -lcrypto -lbz2 -lzstd -llz4 -lsnappy -latomic -ldl -pthread -lrt -luuid -lcurl -lssl -std=c++14 -pedantic -fno-rtti -O2 -flto -MMD

KeyDB

Test: SET - Parallel Connections: 900

OpenBenchmarking.orgRequests Per Second, More Is BetterKeyDB 6.3.2Test: SET - Parallel Connections: 900bcad300K600K900K1200K1500KSE +/- 10170.24, N = 151200999.251209599.381227964.691305926.381. (CXX) g++ options: -ggdb -rdynamic -lm -lz -lcrypto -lbz2 -lzstd -llz4 -lsnappy -latomic -ldl -pthread -lrt -luuid -lcurl -lssl -std=c++14 -pedantic -fno-rtti -O2 -flto -MMD

KeyDB

Test: LPUSH - Parallel Connections: 900

OpenBenchmarking.orgRequests Per Second, More Is BetterKeyDB 6.3.2Test: LPUSH - Parallel Connections: 900adcb200K400K600K800K1000KSE +/- 7796.97, N = 3913352.58934195.31967361.25988728.501. (CXX) g++ options: -ggdb -rdynamic -lm -lz -lcrypto -lbz2 -lzstd -llz4 -lsnappy -latomic -ldl -pthread -lrt -luuid -lcurl -lssl -std=c++14 -pedantic -fno-rtti -O2 -flto -MMD

KeyDB

Test: LPUSH - Parallel Connections: 50

OpenBenchmarking.orgRequests Per Second, More Is BetterKeyDB 6.3.2Test: LPUSH - Parallel Connections: 50dbac400K800K1200K1600K2000KSE +/- 22045.54, N = 41816596.381849522.751885806.561964096.251. (CXX) g++ options: -ggdb -rdynamic -lm -lz -lcrypto -lbz2 -lzstd -llz4 -lsnappy -latomic -ldl -pthread -lrt -luuid -lcurl -lssl -std=c++14 -pedantic -fno-rtti -O2 -flto -MMD

KeyDB

Test: HMSET - Parallel Connections: 50

OpenBenchmarking.orgRequests Per Second, More Is BetterKeyDB 6.3.2Test: HMSET - Parallel Connections: 50adcb200K400K600K800K1000KSE +/- 10277.06, N = 3770419.02781066.94822828.56827088.811. (CXX) g++ options: -ggdb -rdynamic -lm -lz -lcrypto -lbz2 -lzstd -llz4 -lsnappy -latomic -ldl -pthread -lrt -luuid -lcurl -lssl -std=c++14 -pedantic -fno-rtti -O2 -flto -MMD

KeyDB

Test: SADD - Parallel Connections: 500

OpenBenchmarking.orgRequests Per Second, More Is BetterKeyDB 6.3.2Test: SADD - Parallel Connections: 500cdba300K600K900K1200K1500KSE +/- 20006.28, N = 121294096.251296243.381365262.251369793.361. (CXX) g++ options: -ggdb -rdynamic -lm -lz -lcrypto -lbz2 -lzstd -llz4 -lsnappy -latomic -ldl -pthread -lrt -luuid -lcurl -lssl -std=c++14 -pedantic -fno-rtti -O2 -flto -MMD

KeyDB

Test: GET - Parallel Connections: 500

OpenBenchmarking.orgRequests Per Second, More Is BetterKeyDB 6.3.2Test: GET - Parallel Connections: 500cadb300K600K900K1200K1500KSE +/- 21838.17, N = 121476886.751493932.891553518.621558846.381. (CXX) g++ options: -ggdb -rdynamic -lm -lz -lcrypto -lbz2 -lzstd -llz4 -lsnappy -latomic -ldl -pthread -lrt -luuid -lcurl -lssl -std=c++14 -pedantic -fno-rtti -O2 -flto -MMD

KeyDB

Test: GET - Parallel Connections: 900

OpenBenchmarking.orgRequests Per Second, More Is BetterKeyDB 6.3.2Test: GET - Parallel Connections: 900dcab300K600K900K1200K1500KSE +/- 20017.86, N = 31510802.121527463.751563299.291582378.621. (CXX) g++ options: -ggdb -rdynamic -lm -lz -lcrypto -lbz2 -lzstd -llz4 -lsnappy -latomic -ldl -pthread -lrt -luuid -lcurl -lssl -std=c++14 -pedantic -fno-rtti -O2 -flto -MMD

KeyDB

Test: GET - Parallel Connections: 50

OpenBenchmarking.orgRequests Per Second, More Is BetterKeyDB 6.3.2Test: GET - Parallel Connections: 50bcda500K1000K1500K2000K2500KSE +/- 12561.87, N = 32203905.252258764.002273450.752301457.081. (CXX) g++ options: -ggdb -rdynamic -lm -lz -lcrypto -lbz2 -lzstd -llz4 -lsnappy -latomic -ldl -pthread -lrt -luuid -lcurl -lssl -std=c++14 -pedantic -fno-rtti -O2 -flto -MMD

KeyDB

Test: SET - Parallel Connections: 50

OpenBenchmarking.orgRequests Per Second, More Is BetterKeyDB 6.3.2Test: SET - Parallel Connections: 50cdab400K800K1200K1600K2000KSE +/- 22564.98, N = 42007790.252013693.122051262.032074172.381. (CXX) g++ options: -ggdb -rdynamic -lm -lz -lcrypto -lbz2 -lzstd -llz4 -lsnappy -latomic -ldl -pthread -lrt -luuid -lcurl -lssl -std=c++14 -pedantic -fno-rtti -O2 -flto -MMD

KeyDB

Test: GET - Parallel Connections: 100

OpenBenchmarking.orgRequests Per Second, More Is BetterKeyDB 6.3.2Test: GET - Parallel Connections: 100bdca300K600K900K1200K1500KSE +/- 12834.74, N = 31482228.001484384.381488316.621526251.211. (CXX) g++ options: -ggdb -rdynamic -lm -lz -lcrypto -lbz2 -lzstd -llz4 -lsnappy -latomic -ldl -pthread -lrt -luuid -lcurl -lssl -std=c++14 -pedantic -fno-rtti -O2 -flto -MMD

KeyDB

Test: SADD - Parallel Connections: 50

OpenBenchmarking.orgRequests Per Second, More Is BetterKeyDB 6.3.2Test: SADD - Parallel Connections: 50dbca500K1000K1500K2000K2500KSE +/- 14787.63, N = 32186557.002210042.502215133.752240005.251. (CXX) g++ options: -ggdb -rdynamic -lm -lz -lcrypto -lbz2 -lzstd -llz4 -lsnappy -latomic -ldl -pthread -lrt -luuid -lcurl -lssl -std=c++14 -pedantic -fno-rtti -O2 -flto -MMD

Neural Magic DeepSparse

Model: CV Detection, YOLOv5s COCO - Scenario: Synchronous Single-Stream

OpenBenchmarking.orgms/batch, Fewer Is BetterNeural Magic DeepSparse 1.3.2Model: CV Detection, YOLOv5s COCO - Scenario: Synchronous Single-Streamacbd48121620SE +/- 0.07, N = 314.7314.6514.6114.43

Neural Magic DeepSparse

Model: CV Detection, YOLOv5s COCO - Scenario: Synchronous Single-Stream

OpenBenchmarking.orgitems/sec, More Is BetterNeural Magic DeepSparse 1.3.2Model: CV Detection, YOLOv5s COCO - Scenario: Synchronous Single-Streamacbd1530456075SE +/- 0.33, N = 367.8568.2368.4169.23

Neural Magic DeepSparse

Model: CV Segmentation, 90% Pruned YOLACT Pruned - Scenario: Asynchronous Multi-Stream

OpenBenchmarking.orgitems/sec, More Is BetterNeural Magic DeepSparse 1.3.2Model: CV Segmentation, 90% Pruned YOLACT Pruned - Scenario: Asynchronous Multi-Streamacbd510152025SE +/- 0.04, N = 322.4622.4822.5722.88

Neural Magic DeepSparse

Model: NLP Text Classification, BERT base uncased SST2 - Scenario: Asynchronous Multi-Stream

OpenBenchmarking.orgitems/sec, More Is BetterNeural Magic DeepSparse 1.3.2Model: NLP Text Classification, BERT base uncased SST2 - Scenario: Asynchronous Multi-Streambdac20406080100SE +/- 0.11, N = 373.9674.4474.6875.21

Neural Magic DeepSparse

Model: NLP Text Classification, DistilBERT mnli - Scenario: Asynchronous Multi-Stream

OpenBenchmarking.orgms/batch, Fewer Is BetterNeural Magic DeepSparse 1.3.2Model: NLP Text Classification, DistilBERT mnli - Scenario: Asynchronous Multi-Streamdbac20406080100SE +/- 0.08, N = 381.7681.6981.6480.42

Neural Magic DeepSparse

Model: CV Detection, YOLOv5s COCO - Scenario: Asynchronous Multi-Stream

OpenBenchmarking.orgitems/sec, More Is BetterNeural Magic DeepSparse 1.3.2Model: CV Detection, YOLOv5s COCO - Scenario: Asynchronous Multi-Streamacdb20406080100SE +/- 0.45, N = 3100.60100.92101.58102.19

Neural Magic DeepSparse

Model: NLP Text Classification, DistilBERT mnli - Scenario: Asynchronous Multi-Stream

OpenBenchmarking.orgitems/sec, More Is BetterNeural Magic DeepSparse 1.3.2Model: NLP Text Classification, DistilBERT mnli - Scenario: Asynchronous Multi-Streamdbac306090120150SE +/- 0.16, N = 3146.71146.76146.84148.98

Neural Magic DeepSparse

Model: NLP Token Classification, BERT base uncased conll2003 - Scenario: Asynchronous Multi-Stream

OpenBenchmarking.orgitems/sec, More Is BetterNeural Magic DeepSparse 1.3.2Model: NLP Token Classification, BERT base uncased conll2003 - Scenario: Asynchronous Multi-Streamcadb48121620SE +/- 0.06, N = 317.0017.0617.2417.26

Neural Magic DeepSparse

Model: CV Detection, YOLOv5s COCO - Scenario: Asynchronous Multi-Stream

OpenBenchmarking.orgms/batch, Fewer Is BetterNeural Magic DeepSparse 1.3.2Model: CV Detection, YOLOv5s COCO - Scenario: Asynchronous Multi-Streamacdb306090120150SE +/- 0.61, N = 3118.97118.67117.95117.18

Neural Magic DeepSparse

Model: NLP Text Classification, BERT base uncased SST2 - Scenario: Asynchronous Multi-Stream

OpenBenchmarking.orgms/batch, Fewer Is BetterNeural Magic DeepSparse 1.3.2Model: NLP Text Classification, BERT base uncased SST2 - Scenario: Asynchronous Multi-Streambdac4080120160200SE +/- 0.25, N = 3161.86161.02160.48159.44

Neural Magic DeepSparse

Model: NLP Question Answering, BERT base uncased SQuaD 12layer Pruned90 - Scenario: Asynchronous Multi-Stream

OpenBenchmarking.orgitems/sec, More Is BetterNeural Magic DeepSparse 1.3.2Model: NLP Question Answering, BERT base uncased SQuaD 12layer Pruned90 - Scenario: Asynchronous Multi-Streambcad1428425670SE +/- 0.15, N = 362.0362.4362.4762.94

Neural Magic DeepSparse

Model: CV Segmentation, 90% Pruned YOLACT Pruned - Scenario: Asynchronous Multi-Stream

OpenBenchmarking.orgms/batch, Fewer Is BetterNeural Magic DeepSparse 1.3.2Model: CV Segmentation, 90% Pruned YOLACT Pruned - Scenario: Asynchronous Multi-Streamabcd110220330440550SE +/- 0.36, N = 3531.49530.18528.70523.94

Neural Magic DeepSparse

Model: NLP Question Answering, BERT base uncased SQuaD 12layer Pruned90 - Scenario: Asynchronous Multi-Stream

OpenBenchmarking.orgms/batch, Fewer Is BetterNeural Magic DeepSparse 1.3.2Model: NLP Question Answering, BERT base uncased SQuaD 12layer Pruned90 - Scenario: Asynchronous Multi-Streambacd4080120160200SE +/- 0.42, N = 3192.56191.41191.39189.93

Neural Magic DeepSparse

Model: NLP Document Classification, oBERT base uncased on IMDB - Scenario: Asynchronous Multi-Stream

OpenBenchmarking.orgitems/sec, More Is BetterNeural Magic DeepSparse 1.3.2Model: NLP Document Classification, oBERT base uncased on IMDB - Scenario: Asynchronous Multi-Streambacd48121620SE +/- 0.09, N = 317.0317.1017.1317.26

Neural Magic DeepSparse

Model: CV Classification, ResNet-50 ImageNet - Scenario: Asynchronous Multi-Stream

OpenBenchmarking.orgitems/sec, More Is BetterNeural Magic DeepSparse 1.3.2Model: CV Classification, ResNet-50 ImageNet - Scenario: Asynchronous Multi-Streamadcb50100150200250SE +/- 1.42, N = 3220.22220.63222.58223.11

Neural Magic DeepSparse

Model: CV Classification, ResNet-50 ImageNet - Scenario: Asynchronous Multi-Stream

OpenBenchmarking.orgms/batch, Fewer Is BetterNeural Magic DeepSparse 1.3.2Model: CV Classification, ResNet-50 ImageNet - Scenario: Asynchronous Multi-Streamadcb1224364860SE +/- 0.36, N = 354.4554.3853.8553.75

Neural Magic DeepSparse

Model: NLP Sentiment Analysis, 80% Pruned Quantized BERT Base Uncased - Scenario: Synchronous Single-Stream

OpenBenchmarking.orgitems/sec, More Is BetterNeural Magic DeepSparse 1.3.2Model: NLP Sentiment Analysis, 80% Pruned Quantized BERT Base Uncased - Scenario: Synchronous Single-Streambadc20406080100SE +/- 0.18, N = 396.6097.0397.0597.70

Neural Magic DeepSparse

Model: NLP Sentiment Analysis, 80% Pruned Quantized BERT Base Uncased - Scenario: Synchronous Single-Stream

OpenBenchmarking.orgms/batch, Fewer Is BetterNeural Magic DeepSparse 1.3.2Model: NLP Sentiment Analysis, 80% Pruned Quantized BERT Base Uncased - Scenario: Synchronous Single-Streambadc3691215SE +/- 0.02, N = 310.3510.3010.3010.23

Neural Magic DeepSparse

Model: CV Classification, ResNet-50 ImageNet - Scenario: Synchronous Single-Stream

OpenBenchmarking.orgms/batch, Fewer Is BetterNeural Magic DeepSparse 1.3.2Model: CV Classification, ResNet-50 ImageNet - Scenario: Synchronous Single-Streamacdb246810SE +/- 0.0210, N = 37.95097.88777.87877.8702

Neural Magic DeepSparse

Model: CV Classification, ResNet-50 ImageNet - Scenario: Synchronous Single-Stream

OpenBenchmarking.orgitems/sec, More Is BetterNeural Magic DeepSparse 1.3.2Model: CV Classification, ResNet-50 ImageNet - Scenario: Synchronous Single-Streamacdb306090120150SE +/- 0.33, N = 3125.70126.71126.85126.99

Neural Magic DeepSparse

Model: NLP Sentiment Analysis, 80% Pruned Quantized BERT Base Uncased - Scenario: Asynchronous Multi-Stream

OpenBenchmarking.orgms/batch, Fewer Is BetterNeural Magic DeepSparse 1.3.2Model: NLP Sentiment Analysis, 80% Pruned Quantized BERT Base Uncased - Scenario: Asynchronous Multi-Streamacbd1326395265SE +/- 0.65, N = 359.3759.1159.0458.80

Neural Magic DeepSparse

Model: NLP Sentiment Analysis, 80% Pruned Quantized BERT Base Uncased - Scenario: Asynchronous Multi-Stream

OpenBenchmarking.orgitems/sec, More Is BetterNeural Magic DeepSparse 1.3.2Model: NLP Sentiment Analysis, 80% Pruned Quantized BERT Base Uncased - Scenario: Asynchronous Multi-Streamacbd4080120160200SE +/- 2.21, N = 3201.89202.74202.87203.72

Neural Magic DeepSparse

Model: CV Segmentation, 90% Pruned YOLACT Pruned - Scenario: Synchronous Single-Stream

OpenBenchmarking.orgitems/sec, More Is BetterNeural Magic DeepSparse 1.3.2Model: CV Segmentation, 90% Pruned YOLACT Pruned - Scenario: Synchronous Single-Streamcdab48121620SE +/- 0.04, N = 317.8317.9017.9217.97

Neural Magic DeepSparse

Model: CV Segmentation, 90% Pruned YOLACT Pruned - Scenario: Synchronous Single-Stream

OpenBenchmarking.orgms/batch, Fewer Is BetterNeural Magic DeepSparse 1.3.2Model: CV Segmentation, 90% Pruned YOLACT Pruned - Scenario: Synchronous Single-Streamcdab1326395265SE +/- 0.14, N = 356.0955.8555.8055.63

Neural Magic DeepSparse

Model: NLP Text Classification, DistilBERT mnli - Scenario: Synchronous Single-Stream

OpenBenchmarking.orgitems/sec, More Is BetterNeural Magic DeepSparse 1.3.2Model: NLP Text Classification, DistilBERT mnli - Scenario: Synchronous Single-Streamabcd20406080100SE +/- 0.18, N = 384.2184.3484.6284.70

Neural Magic DeepSparse

Model: NLP Text Classification, DistilBERT mnli - Scenario: Synchronous Single-Stream

OpenBenchmarking.orgms/batch, Fewer Is BetterNeural Magic DeepSparse 1.3.2Model: NLP Text Classification, DistilBERT mnli - Scenario: Synchronous Single-Streamabcd3691215SE +/- 0.03, N = 311.8711.8511.8111.80

Neural Magic DeepSparse

Model: NLP Document Classification, oBERT base uncased on IMDB - Scenario: Asynchronous Multi-Stream

OpenBenchmarking.orgms/batch, Fewer Is BetterNeural Magic DeepSparse 1.3.2Model: NLP Document Classification, oBERT base uncased on IMDB - Scenario: Asynchronous Multi-Streambcda150300450600750SE +/- 1.58, N = 3697.46696.46693.88693.75

Neural Magic DeepSparse

Model: NLP Token Classification, BERT base uncased conll2003 - Scenario: Synchronous Single-Stream

OpenBenchmarking.orgitems/sec, More Is BetterNeural Magic DeepSparse 1.3.2Model: NLP Token Classification, BERT base uncased conll2003 - Scenario: Synchronous Single-Streamcdab48121620SE +/- 0.04, N = 315.1215.1815.1915.20

Neural Magic DeepSparse

Model: NLP Token Classification, BERT base uncased conll2003 - Scenario: Synchronous Single-Stream

OpenBenchmarking.orgms/batch, Fewer Is BetterNeural Magic DeepSparse 1.3.2Model: NLP Token Classification, BERT base uncased conll2003 - Scenario: Synchronous Single-Streamcdab1530456075SE +/- 0.18, N = 366.1165.8965.8565.78

Neural Magic DeepSparse

Model: NLP Question Answering, BERT base uncased SQuaD 12layer Pruned90 - Scenario: Synchronous Single-Stream

OpenBenchmarking.orgitems/sec, More Is BetterNeural Magic DeepSparse 1.3.2Model: NLP Question Answering, BERT base uncased SQuaD 12layer Pruned90 - Scenario: Synchronous Single-Streamcbda1020304050SE +/- 0.08, N = 344.2044.3544.3844.41

Neural Magic DeepSparse

Model: NLP Question Answering, BERT base uncased SQuaD 12layer Pruned90 - Scenario: Synchronous Single-Stream

OpenBenchmarking.orgms/batch, Fewer Is BetterNeural Magic DeepSparse 1.3.2Model: NLP Question Answering, BERT base uncased SQuaD 12layer Pruned90 - Scenario: Synchronous Single-Streamcbda510152025SE +/- 0.04, N = 322.6222.5422.5322.51

Neural Magic DeepSparse

Model: NLP Text Classification, BERT base uncased SST2 - Scenario: Synchronous Single-Stream

OpenBenchmarking.orgitems/sec, More Is BetterNeural Magic DeepSparse 1.3.2Model: NLP Text Classification, BERT base uncased SST2 - Scenario: Synchronous Single-Streambadc1020304050SE +/- 0.08, N = 342.4142.4142.4242.59

Neural Magic DeepSparse

Model: NLP Text Classification, BERT base uncased SST2 - Scenario: Synchronous Single-Stream

OpenBenchmarking.orgms/batch, Fewer Is BetterNeural Magic DeepSparse 1.3.2Model: NLP Text Classification, BERT base uncased SST2 - Scenario: Synchronous Single-Streambadc612182430SE +/- 0.04, N = 323.5823.5723.5723.48

Neural Magic DeepSparse

Model: NLP Token Classification, BERT base uncased conll2003 - Scenario: Asynchronous Multi-Stream

OpenBenchmarking.orgms/batch, Fewer Is BetterNeural Magic DeepSparse 1.3.2Model: NLP Token Classification, BERT base uncased conll2003 - Scenario: Asynchronous Multi-Streamcadb150300450600750SE +/- 2.02, N = 3695.23694.20693.83693.70

Neural Magic DeepSparse

Model: NLP Document Classification, oBERT base uncased on IMDB - Scenario: Synchronous Single-Stream

OpenBenchmarking.orgms/batch, Fewer Is BetterNeural Magic DeepSparse 1.3.2Model: NLP Document Classification, oBERT base uncased on IMDB - Scenario: Synchronous Single-Streamcbad1530456075SE +/- 0.08, N = 366.0366.0166.0065.97

Neural Magic DeepSparse

Model: NLP Document Classification, oBERT base uncased on IMDB - Scenario: Synchronous Single-Stream

OpenBenchmarking.orgitems/sec, More Is BetterNeural Magic DeepSparse 1.3.2Model: NLP Document Classification, oBERT base uncased on IMDB - Scenario: Synchronous Single-Streamcbad48121620SE +/- 0.02, N = 315.1415.1515.1515.16

KeyDB

Test: LPUSH - Parallel Connections: 100

OpenBenchmarking.orgRequests Per Second, More Is BetterKeyDB 6.3.2Test: LPUSH - Parallel Connections: 100bcad200K400K600K800K1000KSE +/- 19016.70, N = 151009102.121053252.501079455.031161008.621. (CXX) g++ options: -ggdb -rdynamic -lm -lz -lcrypto -lbz2 -lzstd -llz4 -lsnappy -latomic -ldl -pthread -lrt -luuid -lcurl -lssl -std=c++14 -pedantic -fno-rtti -O2 -flto -MMD

KeyDB

Test: HMSET - Parallel Connections: 900

OpenBenchmarking.orgRequests Per Second, More Is BetterKeyDB 6.3.2Test: HMSET - Parallel Connections: 900cbad90K180K270K360K450KSE +/- 8190.09, N = 12373870.94385409.94405828.74405919.941. (CXX) g++ options: -ggdb -rdynamic -lm -lz -lcrypto -lbz2 -lzstd -llz4 -lsnappy -latomic -ldl -pthread -lrt -luuid -lcurl -lssl -std=c++14 -pedantic -fno-rtti -O2 -flto -MMD

KeyDB

Test: HMSET - Parallel Connections: 500

OpenBenchmarking.orgRequests Per Second, More Is BetterKeyDB 6.3.2Test: HMSET - Parallel Connections: 500dacb120K240K360K480K600KSE +/- 11687.20, N = 15514398.03540612.07544525.88575162.191. (CXX) g++ options: -ggdb -rdynamic -lm -lz -lcrypto -lbz2 -lzstd -llz4 -lsnappy -latomic -ldl -pthread -lrt -luuid -lcurl -lssl -std=c++14 -pedantic -fno-rtti -O2 -flto -MMD


Phoronix Test Suite v10.8.5