AMD EPYC 7763 64-Core testing with a AMD DAYTONA_X (RYM1009B BIOS) and ASPEED on Ubuntu 22.04 via the Phoronix Test Suite.
Compare your own system(s) to this result file with the
Phoronix Test Suite by running the command:
phoronix-test-suite benchmark 2308059-NE-77632204529 7763 2204 - Phoronix Test Suite 7763 2204 AMD EPYC 7763 64-Core testing with a AMD DAYTONA_X (RYM1009B BIOS) and ASPEED on Ubuntu 22.04 via the Phoronix Test Suite.
HTML result view exported from: https://openbenchmarking.org/result/2308059-NE-77632204529&sor&grr .
7763 2204 Processor Motherboard Chipset Memory Disk Graphics Monitor Network OS Kernel Desktop Display Server Vulkan Compiler File-System Screen Resolution a b c AMD EPYC 7763 64-Core @ 2.45GHz (64 Cores / 128 Threads) AMD DAYTONA_X (RYM1009B BIOS) AMD Starship/Matisse 256GB 800GB INTEL SSDPF21Q800GB ASPEED VE228 2 x Mellanox MT27710 Ubuntu 22.04 6.2.0-phx (x86_64) GNOME Shell 42.5 X Server 1.21.1.3 1.3.224 GCC 11.3.0 + LLVM 14.0.0 ext4 1920x1080 OpenBenchmarking.org Kernel Details - Transparent Huge Pages: madvise Compiler Details - --build=x86_64-linux-gnu --disable-vtable-verify --disable-werror --enable-bootstrap --enable-cet --enable-checking=release --enable-clocale=gnu --enable-default-pie --enable-gnu-unique-object --enable-languages=c,ada,c++,go,brig,d,fortran,objc,obj-c++,m2 --enable-libphobos-checking=release --enable-libstdcxx-debug --enable-libstdcxx-time=yes --enable-link-serialization=2 --enable-multiarch --enable-multilib --enable-nls --enable-objc-gc=auto --enable-offload-targets=nvptx-none=/build/gcc-11-xKiWfi/gcc-11-11.3.0/debian/tmp-nvptx/usr,amdgcn-amdhsa=/build/gcc-11-xKiWfi/gcc-11-11.3.0/debian/tmp-gcn/usr --enable-plugin --enable-shared --enable-threads=posix --host=x86_64-linux-gnu --program-prefix=x86_64-linux-gnu- --target=x86_64-linux-gnu --with-abi=m64 --with-arch-32=i686 --with-build-config=bootstrap-lto-lean --with-default-libstdcxx-abi=new --with-gcc-major-version-only --with-multilib-list=m32,m64,mx32 --with-target-system-zlib=auto --with-tune=generic --without-cuda-driver -v Processor Details - Scaling Governor: acpi-cpufreq performance (Boost: Enabled) - CPU Microcode: 0xa001173 Java Details - OpenJDK Runtime Environment (build 11.0.20+8-post-Ubuntu-1ubuntu122.04) Python Details - Python 3.10.6 Security Details - itlb_multihit: Not affected + l1tf: Not affected + mds: Not affected + meltdown: Not affected + mmio_stale_data: Not affected + retbleed: Not affected + spec_store_bypass: Mitigation of SSB disabled via prctl + spectre_v1: Mitigation of usercopy/swapgs barriers and __user pointer sanitization + spectre_v2: Mitigation of Retpolines IBPB: conditional IBRS_FW STIBP: always-on RSB filling PBRSB-eIBRS: Not affected + srbds: Not affected + tsx_async_abort: Not affected
7763 2204 couchdb: 500 - 3000 - 30 build-gcc: Time To Compile brl-cad: VGR Performance Metric couchdb: 300 - 3000 - 30 couchdb: 100 - 3000 - 30 couchdb: 500 - 1000 - 30 blender: Barbershop - CPU-Only couchdb: 300 - 1000 - 30 deepsparse: BERT-Large, NLP Question Answering - Asynchronous Multi-Stream deepsparse: BERT-Large, NLP Question Answering - Asynchronous Multi-Stream cassandra: Writes deepsparse: BERT-Large, NLP Question Answering, Sparse INT8 - Synchronous Single-Stream deepsparse: BERT-Large, NLP Question Answering, Sparse INT8 - Synchronous Single-Stream deepsparse: BERT-Large, NLP Question Answering - Synchronous Single-Stream deepsparse: BERT-Large, NLP Question Answering - Synchronous Single-Stream vvenc: Bosphorus 4K - Fast couchdb: 100 - 1000 - 30 ncnn: CPU - FastestDet ncnn: CPU - vision_transformer ncnn: CPU - regnety_400m ncnn: CPU - squeezenet_ssd ncnn: CPU - yolov4-tiny ncnn: CPU - resnet50 ncnn: CPU - alexnet ncnn: CPU - resnet18 ncnn: CPU - vgg16 ncnn: CPU - googlenet ncnn: CPU - blazeface ncnn: CPU - efficientnet-b0 ncnn: CPU - mnasnet ncnn: CPU - shufflenet-v2 ncnn: CPU-v3-v3 - mobilenet-v3 ncnn: CPU-v2-v2 - mobilenet-v2 ncnn: CPU - mobilenet deepsparse: BERT-Large, NLP Question Answering, Sparse INT8 - Asynchronous Multi-Stream deepsparse: BERT-Large, NLP Question Answering, Sparse INT8 - Asynchronous Multi-Stream blender: Pabellon Barcelona - CPU-Only apache-iotdb: 500 - 100 - 500 apache-iotdb: 500 - 100 - 500 blender: Classroom - CPU-Only deepsparse: NLP Text Classification, BERT base uncased SST2, Sparse INT8 - Synchronous Single-Stream deepsparse: NLP Text Classification, BERT base uncased SST2, Sparse INT8 - Synchronous Single-Stream deepsparse: NLP Question Answering, BERT base uncased SQuaD 12layer Pruned90 - Asynchronous Multi-Stream deepsparse: NLP Question Answering, BERT base uncased SQuaD 12layer Pruned90 - Asynchronous Multi-Stream vvenc: Bosphorus 4K - Faster deepsparse: NLP Sentiment Analysis, 80% Pruned Quantized BERT Base Uncased - Asynchronous Multi-Stream deepsparse: NLP Sentiment Analysis, 80% Pruned Quantized BERT Base Uncased - Asynchronous Multi-Stream deepsparse: NLP Text Classification, BERT base uncased SST2 - Asynchronous Multi-Stream deepsparse: NLP Text Classification, BERT base uncased SST2 - Asynchronous Multi-Stream deepsparse: CV Segmentation, 90% Pruned YOLACT Pruned - Asynchronous Multi-Stream deepsparse: CV Segmentation, 90% Pruned YOLACT Pruned - Asynchronous Multi-Stream deepsparse: NLP Text Classification, BERT base uncased SST2, Sparse INT8 - Asynchronous Multi-Stream deepsparse: NLP Text Classification, BERT base uncased SST2, Sparse INT8 - Asynchronous Multi-Stream deepsparse: NLP Document Classification, oBERT base uncased on IMDB - Asynchronous Multi-Stream deepsparse: NLP Document Classification, oBERT base uncased on IMDB - Asynchronous Multi-Stream deepsparse: NLP Token Classification, BERT base uncased conll2003 - Asynchronous Multi-Stream deepsparse: NLP Token Classification, BERT base uncased conll2003 - Asynchronous Multi-Stream deepsparse: NLP Text Classification, BERT base uncased SST2 - Synchronous Single-Stream deepsparse: NLP Text Classification, BERT base uncased SST2 - Synchronous Single-Stream deepsparse: NLP Document Classification, oBERT base uncased on IMDB - Synchronous Single-Stream deepsparse: NLP Document Classification, oBERT base uncased on IMDB - Synchronous Single-Stream deepsparse: NLP Token Classification, BERT base uncased conll2003 - Synchronous Single-Stream deepsparse: NLP Token Classification, BERT base uncased conll2003 - Synchronous Single-Stream deepsparse: NLP Question Answering, BERT base uncased SQuaD 12layer Pruned90 - Synchronous Single-Stream deepsparse: NLP Question Answering, BERT base uncased SQuaD 12layer Pruned90 - Synchronous Single-Stream deepsparse: NLP Sentiment Analysis, 80% Pruned Quantized BERT Base Uncased - Synchronous Single-Stream deepsparse: NLP Sentiment Analysis, 80% Pruned Quantized BERT Base Uncased - Synchronous Single-Stream apache-iotdb: 200 - 100 - 500 apache-iotdb: 200 - 100 - 500 deepsparse: CV Segmentation, 90% Pruned YOLACT Pruned - Synchronous Single-Stream deepsparse: CV Segmentation, 90% Pruned YOLACT Pruned - Synchronous Single-Stream deepsparse: NLP Text Classification, DistilBERT mnli - Asynchronous Multi-Stream deepsparse: NLP Text Classification, DistilBERT mnli - Asynchronous Multi-Stream apache-iotdb: 500 - 100 - 200 apache-iotdb: 500 - 100 - 200 srsran: PUSCH Processor Benchmark, Throughput Total deepsparse: NLP Text Classification, DistilBERT mnli - Synchronous Single-Stream deepsparse: NLP Text Classification, DistilBERT mnli - Synchronous Single-Stream vvenc: Bosphorus 1080p - Fast deepsparse: CV Detection, YOLOv5s COCO, Sparse INT8 - Asynchronous Multi-Stream deepsparse: CV Detection, YOLOv5s COCO, Sparse INT8 - Asynchronous Multi-Stream deepsparse: CV Detection, YOLOv5s COCO - Asynchronous Multi-Stream deepsparse: CV Detection, YOLOv5s COCO - Asynchronous Multi-Stream deepsparse: ResNet-50, Sparse INT8 - Synchronous Single-Stream deepsparse: ResNet-50, Sparse INT8 - Synchronous Single-Stream deepsparse: CV Detection, YOLOv5s COCO - Synchronous Single-Stream deepsparse: CV Detection, YOLOv5s COCO - Synchronous Single-Stream deepsparse: CV Detection, YOLOv5s COCO, Sparse INT8 - Synchronous Single-Stream deepsparse: CV Detection, YOLOv5s COCO, Sparse INT8 - Synchronous Single-Stream deepsparse: ResNet-50, Baseline - Asynchronous Multi-Stream deepsparse: ResNet-50, Baseline - Asynchronous Multi-Stream deepsparse: CV Classification, ResNet-50 ImageNet - Asynchronous Multi-Stream deepsparse: CV Classification, ResNet-50 ImageNet - Asynchronous Multi-Stream deepsparse: ResNet-50, Baseline - Synchronous Single-Stream deepsparse: ResNet-50, Baseline - Synchronous Single-Stream deepsparse: CV Classification, ResNet-50 ImageNet - Synchronous Single-Stream deepsparse: CV Classification, ResNet-50 ImageNet - Synchronous Single-Stream deepsparse: ResNet-50, Sparse INT8 - Asynchronous Multi-Stream deepsparse: ResNet-50, Sparse INT8 - Asynchronous Multi-Stream apache-iotdb: 500 - 1 - 500 apache-iotdb: 500 - 1 - 500 blender: Fishy Cat - CPU-Only blender: BMW27 - CPU-Only apache-iotdb: 100 - 100 - 500 apache-iotdb: 100 - 100 - 500 apache-iotdb: 200 - 100 - 200 apache-iotdb: 200 - 100 - 200 apache-iotdb: 200 - 1 - 500 apache-iotdb: 200 - 1 - 500 apache-iotdb: 500 - 1 - 200 apache-iotdb: 500 - 1 - 200 vvenc: Bosphorus 1080p - Faster srsran: Downlink Processor Benchmark apache-iotdb: 100 - 100 - 200 apache-iotdb: 100 - 100 - 200 apache-iotdb: 100 - 1 - 500 apache-iotdb: 100 - 1 - 500 apache-iotdb: 200 - 1 - 200 apache-iotdb: 200 - 1 - 200 apache-iotdb: 100 - 1 - 200 apache-iotdb: 100 - 1 - 200 srsran: PUSCH Processor Benchmark, Throughput Thread a b c 2390.933 1020.133 734386 572.125 346.085 339.967 253.49 169.505 681.2756 46.7204 236650 10.5770 94.5097 40.9048 24.4422 5.991 101.578 10.25 48.79 35.24 14.17 20.66 15.49 5.23 8.50 23.84 14.62 3.97 9.98 6.09 9.09 7.00 6.35 14.11 55.5817 574.9550 84.55 81.81 56935634.55 68.80 5.747 173.9462 223.4715 143.0725 10.646 65.2750 489.7709 192.3546 165.9980 597.9679 53.4831 28.9150 1105.3791 840.5416 37.6144 841.4779 37.6109 18.5858 53.7845 50.1258 19.9469 49.8132 20.0720 25.356 39.4219 11.5378 86.6104 109.38 42048733.22 34.9104 28.6310 97.8711 326.3123 35.05 51341708.85 9682.1 10.2645 97.3561 16.083 140.3979 227.4188 141.7036 225.3104 1.3784 723.7149 8.3423 119.8041 8.3056 120.3645 68.2852 468.1383 68.3188 467.9848 6.2525 159.8492 6.2481 159.9660 8.3665 3814.5194 27.1 1636128.73 33.70 27.27 81.2 51316464.44 35.09 46437377.67 33.5 1232509.19 13.54 1182440.62 29.352 657.7 36.04 39287432.92 34.36 1038515.62 15.24 898967.08 17.45 644019.72 211.1 1020.846 729876 253.77 679.8240 46.5739 238161 10.6299 94.0445 40.6993 24.5655 5.993 9.04 48.49 27.54 14.11 20.51 15.34 5.22 8.42 23.64 14.53 3.43 9.78 5.9 7.93 6.55 6.26 13.97 55.5601 575.2859 84.35 79.83 59505306.55 68.50 5.7863 172.7505 223.3670 143.2058 10.815 65.7359 486.1728 192.1645 166.2231 596.8091 53.5531 28.9451 1104.0377 840.9493 37.5315 840.2573 37.5352 18.6144 53.7023 49.9226 20.0279 49.9144 20.0315 25.2659 39.5631 11.5666 86.3905 110.88 41987111.39 34.9082 28.6329 97.8434 326.5439 36.1 50045888.98 9718.6 10.2380 97.6121 16.090 140.2607 227.6423 141.6185 225.4592 1.3623 732.1101 8.3368 119.8818 8.3182 120.1762 68.2763 467.9696 68.2716 468.1087 6.2566 159.7481 6.2495 159.9238 8.3441 3824.3045 26 1686943.16 33.76 27.5 82.26 50507747.12 33.84 47245476.78 33.86 1226219.88 11.63 1365831.5 29.39 658.1 36.85 38401769.17 32.92 1069145.79 13.6 978176.76 17.28 648308.27 208.2 1020.216 730434 253.43 679.8118 46.9021 234887 10.5720 94.5548 40.7606 24.5284 5.976 8.88 48.43 27.59 14.59 20.80 15.54 5.22 8.51 23.91 14.47 3.48 9.75 5.86 7.60 6.34 6.17 14.03 55.5835 575.1158 84.17 83.14 56463717.54 68.70 5.8092 172.0659 222.8175 143.5714 10.818 66.2837 482.1274 192.2148 166.058 596.5343 53.6166 28.9644 1103.3552 840.1234 37.5814 840.4350 37.5755 18.528 53.952 49.9312 20.0245 49.8580 20.0536 25.2989 39.5117 11.5764 86.3219 106.73 43363203.76 34.8978 28.6416 97.8698 326.4108 37.12 49201448.81 9727.1 10.3146 96.8822 16.055 140.3036 227.5740 141.4775 225.8007 1.3634 731.5099 8.3300 119.9790 8.2903 120.5813 68.2503 468.3293 68.3388 467.6396 6.2238 160.5932 6.2195 160.6985 8.3468 3823.0833 31.4 1446487.7 33.72 27.24 79.16 52464142.83 34.79 46674344.69 32.71 1261385.89 11.83 1367763.49 29.469 619.3 35.01 39945212.99 34.08 1044153.44 16.07 870795.92 16.35 667880.96 210.8 OpenBenchmarking.org
Apache CouchDB Bulk Size: 500 - Inserts: 3000 - Rounds: 30 OpenBenchmarking.org Seconds, Fewer Is Better Apache CouchDB 3.3.2 Bulk Size: 500 - Inserts: 3000 - Rounds: 30 a 500 1000 1500 2000 2500 2390.93 1. (CXX) g++ options: -std=c++17 -lmozjs-78 -lm -lei -fPIC -MMD
Timed GCC Compilation Time To Compile OpenBenchmarking.org Seconds, Fewer Is Better Timed GCC Compilation 13.2 Time To Compile a c b 200 400 600 800 1000 SE +/- 1.81, N = 2 SE +/- 0.66, N = 2 SE +/- 0.03, N = 2 1020.13 1020.22 1020.85
BRL-CAD VGR Performance Metric OpenBenchmarking.org VGR Performance Metric, More Is Better BRL-CAD 7.36 VGR Performance Metric a c b 160K 320K 480K 640K 800K SE +/- 1805.50, N = 2 SE +/- 963.50, N = 2 SE +/- 357.50, N = 2 734386 730434 729876 1. (CXX) g++ options: -std=c++14 -pipe -fvisibility=hidden -fno-strict-aliasing -fno-common -fexceptions -ftemplate-depth-128 -m64 -ggdb3 -O3 -fipa-pta -fstrength-reduce -finline-functions -flto -ltcl8.6 -lregex_brl -lz_brl -lnetpbm -ldl -lm -ltk8.6
Apache CouchDB Bulk Size: 300 - Inserts: 3000 - Rounds: 30 OpenBenchmarking.org Seconds, Fewer Is Better Apache CouchDB 3.3.2 Bulk Size: 300 - Inserts: 3000 - Rounds: 30 a 120 240 360 480 600 SE +/- 0.52, N = 2 572.13 1. (CXX) g++ options: -std=c++17 -lmozjs-78 -lm -lei -fPIC -MMD
Apache CouchDB Bulk Size: 100 - Inserts: 3000 - Rounds: 30 OpenBenchmarking.org Seconds, Fewer Is Better Apache CouchDB 3.3.2 Bulk Size: 100 - Inserts: 3000 - Rounds: 30 a 80 160 240 320 400 SE +/- 0.25, N = 2 346.09 1. (CXX) g++ options: -std=c++17 -lmozjs-78 -lm -lei -fPIC -MMD
Apache CouchDB Bulk Size: 500 - Inserts: 1000 - Rounds: 30 OpenBenchmarking.org Seconds, Fewer Is Better Apache CouchDB 3.3.2 Bulk Size: 500 - Inserts: 1000 - Rounds: 30 a 70 140 210 280 350 SE +/- 8.78, N = 2 339.97 1. (CXX) g++ options: -std=c++17 -lmozjs-78 -lm -lei -fPIC -MMD
Blender Blend File: Barbershop - Compute: CPU-Only OpenBenchmarking.org Seconds, Fewer Is Better Blender 3.6 Blend File: Barbershop - Compute: CPU-Only c a b 60 120 180 240 300 SE +/- 0.52, N = 2 SE +/- 0.11, N = 2 SE +/- 0.23, N = 2 253.43 253.49 253.77
Apache CouchDB Bulk Size: 300 - Inserts: 1000 - Rounds: 30 OpenBenchmarking.org Seconds, Fewer Is Better Apache CouchDB 3.3.2 Bulk Size: 300 - Inserts: 1000 - Rounds: 30 a 40 80 120 160 200 SE +/- 0.64, N = 2 169.51 1. (CXX) g++ options: -std=c++17 -lmozjs-78 -lm -lei -fPIC -MMD
Neural Magic DeepSparse Model: BERT-Large, NLP Question Answering - Scenario: Asynchronous Multi-Stream OpenBenchmarking.org ms/batch, Fewer Is Better Neural Magic DeepSparse 1.5 Model: BERT-Large, NLP Question Answering - Scenario: Asynchronous Multi-Stream c b a 150 300 450 600 750 SE +/- 0.34, N = 2 SE +/- 0.17, N = 2 SE +/- 0.37, N = 2 679.81 679.82 681.28
Neural Magic DeepSparse Model: BERT-Large, NLP Question Answering - Scenario: Asynchronous Multi-Stream OpenBenchmarking.org items/sec, More Is Better Neural Magic DeepSparse 1.5 Model: BERT-Large, NLP Question Answering - Scenario: Asynchronous Multi-Stream c a b 11 22 33 44 55 SE +/- 0.02, N = 2 SE +/- 0.10, N = 2 SE +/- 0.03, N = 2 46.90 46.72 46.57
Apache Cassandra Test: Writes OpenBenchmarking.org Op/s, More Is Better Apache Cassandra 4.1.3 Test: Writes b a c 50K 100K 150K 200K 250K SE +/- 817.50, N = 2 SE +/- 633.50, N = 2 SE +/- 669.00, N = 2 238161 236650 234887
Neural Magic DeepSparse Model: BERT-Large, NLP Question Answering, Sparse INT8 - Scenario: Synchronous Single-Stream OpenBenchmarking.org ms/batch, Fewer Is Better Neural Magic DeepSparse 1.5 Model: BERT-Large, NLP Question Answering, Sparse INT8 - Scenario: Synchronous Single-Stream c a b 3 6 9 12 15 SE +/- 0.02, N = 2 SE +/- 0.01, N = 2 SE +/- 0.07, N = 2 10.57 10.58 10.63
Neural Magic DeepSparse Model: BERT-Large, NLP Question Answering, Sparse INT8 - Scenario: Synchronous Single-Stream OpenBenchmarking.org items/sec, More Is Better Neural Magic DeepSparse 1.5 Model: BERT-Large, NLP Question Answering, Sparse INT8 - Scenario: Synchronous Single-Stream c a b 20 40 60 80 100 SE +/- 0.18, N = 2 SE +/- 0.05, N = 2 SE +/- 0.64, N = 2 94.55 94.51 94.04
Neural Magic DeepSparse Model: BERT-Large, NLP Question Answering - Scenario: Synchronous Single-Stream OpenBenchmarking.org ms/batch, Fewer Is Better Neural Magic DeepSparse 1.5 Model: BERT-Large, NLP Question Answering - Scenario: Synchronous Single-Stream b c a 9 18 27 36 45 SE +/- 0.01, N = 2 SE +/- 0.01, N = 2 SE +/- 0.01, N = 2 40.70 40.76 40.90
Neural Magic DeepSparse Model: BERT-Large, NLP Question Answering - Scenario: Synchronous Single-Stream OpenBenchmarking.org items/sec, More Is Better Neural Magic DeepSparse 1.5 Model: BERT-Large, NLP Question Answering - Scenario: Synchronous Single-Stream b c a 6 12 18 24 30 SE +/- 0.01, N = 2 SE +/- 0.00, N = 2 SE +/- 0.01, N = 2 24.57 24.53 24.44
VVenC Video Input: Bosphorus 4K - Video Preset: Fast OpenBenchmarking.org Frames Per Second, More Is Better VVenC 1.9 Video Input: Bosphorus 4K - Video Preset: Fast b a c 1.3484 2.6968 4.0452 5.3936 6.742 SE +/- 0.006, N = 2 SE +/- 0.001, N = 2 SE +/- 0.002, N = 2 5.993 5.991 5.976 1. (CXX) g++ options: -O3 -flto -fno-fat-lto-objects -flto=auto
Apache CouchDB Bulk Size: 100 - Inserts: 1000 - Rounds: 30 OpenBenchmarking.org Seconds, Fewer Is Better Apache CouchDB 3.3.2 Bulk Size: 100 - Inserts: 1000 - Rounds: 30 a 20 40 60 80 100 SE +/- 0.50, N = 2 101.58 1. (CXX) g++ options: -std=c++17 -lmozjs-78 -lm -lei -fPIC -MMD
NCNN Target: CPU - Model: FastestDet OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: CPU - Model: FastestDet c b a 3 6 9 12 15 SE +/- 0.01, N = 2 SE +/- 0.11, N = 2 SE +/- 0.94, N = 2 8.88 9.04 10.25 MIN: 8.58 / MAX: 13.34 MIN: 8.65 / MAX: 15.19 MIN: 8.95 / MAX: 17.14 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread
NCNN Target: CPU - Model: vision_transformer OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: CPU - Model: vision_transformer c b a 11 22 33 44 55 SE +/- 0.30, N = 2 SE +/- 0.04, N = 2 SE +/- 0.07, N = 2 48.43 48.49 48.79 MIN: 47.33 / MAX: 85.35 MIN: 47.44 / MAX: 58.53 MIN: 47.65 / MAX: 78.36 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread
NCNN Target: CPU - Model: regnety_400m OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: CPU - Model: regnety_400m b c a 8 16 24 32 40 SE +/- 0.15, N = 2 SE +/- 0.07, N = 2 SE +/- 5.99, N = 2 27.54 27.59 35.24 MIN: 26.96 / MAX: 33.56 MIN: 26.64 / MAX: 33.41 MIN: 27.86 / MAX: 47.9 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread
NCNN Target: CPU - Model: squeezenet_ssd OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: CPU - Model: squeezenet_ssd b a c 4 8 12 16 20 SE +/- 0.09, N = 2 SE +/- 0.01, N = 2 SE +/- 0.62, N = 2 14.11 14.17 14.59 MIN: 13.32 / MAX: 18.63 MIN: 13.53 / MAX: 18.45 MIN: 13.37 / MAX: 277.61 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread
NCNN Target: CPU - Model: yolov4-tiny OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: CPU - Model: yolov4-tiny b a c 5 10 15 20 25 SE +/- 0.08, N = 2 SE +/- 0.02, N = 2 SE +/- 0.13, N = 2 20.51 20.66 20.80 MIN: 19.87 / MAX: 24.86 MIN: 20.04 / MAX: 25.04 MIN: 20.01 / MAX: 96.45 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread
NCNN Target: CPU - Model: resnet50 OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: CPU - Model: resnet50 b a c 4 8 12 16 20 SE +/- 0.10, N = 2 SE +/- 0.08, N = 2 SE +/- 0.14, N = 2 15.34 15.49 15.54 MIN: 15.07 / MAX: 21.69 MIN: 15.24 / MAX: 21.82 MIN: 15.15 / MAX: 27.2 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread
NCNN Target: CPU - Model: alexnet OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: CPU - Model: alexnet b c a 1.1768 2.3536 3.5304 4.7072 5.884 SE +/- 0.02, N = 2 SE +/- 0.02, N = 2 SE +/- 0.01, N = 2 5.22 5.22 5.23 MIN: 5.11 / MAX: 5.77 MIN: 5.12 / MAX: 7.84 MIN: 5.12 / MAX: 11.62 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread
NCNN Target: CPU - Model: resnet18 OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: CPU - Model: resnet18 b a c 2 4 6 8 10 SE +/- 0.04, N = 2 SE +/- 0.03, N = 2 SE +/- 0.03, N = 2 8.42 8.50 8.51 MIN: 8.27 / MAX: 14.6 MIN: 8.33 / MAX: 14.69 MIN: 8.3 / MAX: 13.61 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread
NCNN Target: CPU - Model: vgg16 OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: CPU - Model: vgg16 b a c 6 12 18 24 30 SE +/- 0.06, N = 2 SE +/- 0.05, N = 2 SE +/- 0.08, N = 2 23.64 23.84 23.91 MIN: 23.33 / MAX: 28.05 MIN: 23.45 / MAX: 28.55 MIN: 23.48 / MAX: 30.63 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread
NCNN Target: CPU - Model: googlenet OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: CPU - Model: googlenet c b a 4 8 12 16 20 SE +/- 0.06, N = 2 SE +/- 0.02, N = 2 SE +/- 0.03, N = 2 14.47 14.53 14.62 MIN: 14.26 / MAX: 20.57 MIN: 14.31 / MAX: 24.11 MIN: 14.46 / MAX: 25.51 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread
NCNN Target: CPU - Model: blazeface OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: CPU - Model: blazeface b c a 0.8933 1.7866 2.6799 3.5732 4.4665 SE +/- 0.01, N = 2 SE +/- 0.05, N = 2 SE +/- 0.09, N = 2 3.43 3.48 3.97 MIN: 3.35 / MAX: 3.83 MIN: 3.32 / MAX: 8.58 MIN: 3.5 / MAX: 7.61 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread
NCNN Target: CPU - Model: efficientnet-b0 OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: CPU - Model: efficientnet-b0 c b a 3 6 9 12 15 SE +/- 0.04, N = 2 SE +/- 0.01, N = 2 SE +/- 0.03, N = 2 9.75 9.78 9.98 MIN: 9.58 / MAX: 13.38 MIN: 9.64 / MAX: 16.01 MIN: 9.82 / MAX: 10.96 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread
NCNN Target: CPU - Model: mnasnet OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: CPU - Model: mnasnet c b a 2 4 6 8 10 SE +/- 0.05, N = 2 SE +/- 0.00, N = 2 SE +/- 0.11, N = 2 5.86 5.90 6.09 MIN: 5.73 / MAX: 11.67 MIN: 5.81 / MAX: 12.33 MIN: 5.89 / MAX: 10.35 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread
NCNN Target: CPU - Model: shufflenet-v2 OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: CPU - Model: shufflenet-v2 c b a 3 6 9 12 15 SE +/- 0.04, N = 2 SE +/- 0.31, N = 2 SE +/- 1.23, N = 2 7.60 7.93 9.09 MIN: 7.44 / MAX: 11.6 MIN: 7.51 / MAX: 11.45 MIN: 7.74 / MAX: 15.92 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread
NCNN Target: CPU-v3-v3 - Model: mobilenet-v3 OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: CPU-v3-v3 - Model: mobilenet-v3 c b a 2 4 6 8 10 SE +/- 0.04, N = 2 SE +/- 0.18, N = 2 SE +/- 0.55, N = 2 6.34 6.55 7.00 MIN: 6.15 / MAX: 11.57 MIN: 6.24 / MAX: 7.61 MIN: 6.3 / MAX: 10.2 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread
NCNN Target: CPU-v2-v2 - Model: mobilenet-v2 OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: CPU-v2-v2 - Model: mobilenet-v2 c b a 2 4 6 8 10 SE +/- 0.07, N = 2 SE +/- 0.00, N = 2 SE +/- 0.01, N = 2 6.17 6.26 6.35 MIN: 6.02 / MAX: 6.83 MIN: 6.11 / MAX: 12.76 MIN: 6.19 / MAX: 12.45 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread
NCNN Target: CPU - Model: mobilenet OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: CPU - Model: mobilenet b c a 4 8 12 16 20 SE +/- 0.03, N = 2 SE +/- 0.09, N = 2 SE +/- 0.04, N = 2 13.97 14.03 14.11 MIN: 13.64 / MAX: 19.68 MIN: 13.68 / MAX: 18.14 MIN: 13.76 / MAX: 19.75 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread
Neural Magic DeepSparse Model: BERT-Large, NLP Question Answering, Sparse INT8 - Scenario: Asynchronous Multi-Stream OpenBenchmarking.org ms/batch, Fewer Is Better Neural Magic DeepSparse 1.5 Model: BERT-Large, NLP Question Answering, Sparse INT8 - Scenario: Asynchronous Multi-Stream b a c 12 24 36 48 60 SE +/- 0.06, N = 2 SE +/- 0.03, N = 2 SE +/- 0.04, N = 2 55.56 55.58 55.58
Neural Magic DeepSparse Model: BERT-Large, NLP Question Answering, Sparse INT8 - Scenario: Asynchronous Multi-Stream OpenBenchmarking.org items/sec, More Is Better Neural Magic DeepSparse 1.5 Model: BERT-Large, NLP Question Answering, Sparse INT8 - Scenario: Asynchronous Multi-Stream b c a 120 240 360 480 600 SE +/- 0.68, N = 2 SE +/- 0.31, N = 2 SE +/- 0.03, N = 2 575.29 575.12 574.96
Blender Blend File: Pabellon Barcelona - Compute: CPU-Only OpenBenchmarking.org Seconds, Fewer Is Better Blender 3.6 Blend File: Pabellon Barcelona - Compute: CPU-Only c b a 20 40 60 80 100 SE +/- 0.04, N = 2 SE +/- 0.14, N = 2 SE +/- 0.40, N = 2 84.17 84.35 84.55
Apache IoTDB Device Count: 500 - Batch Size Per Write: 100 - Sensor Count: 500 OpenBenchmarking.org Average Latency, More Is Better Apache IoTDB 1.1.2 Device Count: 500 - Batch Size Per Write: 100 - Sensor Count: 500 c a b 20 40 60 80 100 83.14 81.81 79.83 MAX: 2932.1 MAX: 3018.16 MAX: 1607.86
Apache IoTDB Device Count: 500 - Batch Size Per Write: 100 - Sensor Count: 500 OpenBenchmarking.org point/sec, More Is Better Apache IoTDB 1.1.2 Device Count: 500 - Batch Size Per Write: 100 - Sensor Count: 500 b a c 13M 26M 39M 52M 65M 59505306.55 56935634.55 56463717.54
Blender Blend File: Classroom - Compute: CPU-Only OpenBenchmarking.org Seconds, Fewer Is Better Blender 3.6 Blend File: Classroom - Compute: CPU-Only b c a 15 30 45 60 75 SE +/- 0.03, N = 2 SE +/- 0.13, N = 2 SE +/- 0.14, N = 2 68.50 68.70 68.80
Neural Magic DeepSparse Model: NLP Text Classification, BERT base uncased SST2, Sparse INT8 - Scenario: Synchronous Single-Stream OpenBenchmarking.org ms/batch, Fewer Is Better Neural Magic DeepSparse 1.5 Model: NLP Text Classification, BERT base uncased SST2, Sparse INT8 - Scenario: Synchronous Single-Stream a b c 1.3071 2.6142 3.9213 5.2284 6.5355 SE +/- 0.0325, N = 2 SE +/- 0.0212, N = 2 SE +/- 0.0271, N = 2 5.7470 5.7863 5.8092
Neural Magic DeepSparse Model: NLP Text Classification, BERT base uncased SST2, Sparse INT8 - Scenario: Synchronous Single-Stream OpenBenchmarking.org items/sec, More Is Better Neural Magic DeepSparse 1.5 Model: NLP Text Classification, BERT base uncased SST2, Sparse INT8 - Scenario: Synchronous Single-Stream a b c 40 80 120 160 200 SE +/- 0.98, N = 2 SE +/- 0.63, N = 2 SE +/- 0.80, N = 2 173.95 172.75 172.07
Neural Magic DeepSparse Model: NLP Question Answering, BERT base uncased SQuaD 12layer Pruned90 - Scenario: Asynchronous Multi-Stream OpenBenchmarking.org ms/batch, Fewer Is Better Neural Magic DeepSparse 1.5 Model: NLP Question Answering, BERT base uncased SQuaD 12layer Pruned90 - Scenario: Asynchronous Multi-Stream c b a 50 100 150 200 250 SE +/- 0.05, N = 2 SE +/- 0.26, N = 2 SE +/- 0.03, N = 2 222.82 223.37 223.47
Neural Magic DeepSparse Model: NLP Question Answering, BERT base uncased SQuaD 12layer Pruned90 - Scenario: Asynchronous Multi-Stream OpenBenchmarking.org items/sec, More Is Better Neural Magic DeepSparse 1.5 Model: NLP Question Answering, BERT base uncased SQuaD 12layer Pruned90 - Scenario: Asynchronous Multi-Stream c b a 30 60 90 120 150 SE +/- 0.03, N = 2 SE +/- 0.18, N = 2 SE +/- 0.01, N = 2 143.57 143.21 143.07
VVenC Video Input: Bosphorus 4K - Video Preset: Faster OpenBenchmarking.org Frames Per Second, More Is Better VVenC 1.9 Video Input: Bosphorus 4K - Video Preset: Faster c b a 3 6 9 12 15 SE +/- 0.00, N = 2 SE +/- 0.02, N = 2 SE +/- 0.18, N = 2 10.82 10.82 10.65 1. (CXX) g++ options: -O3 -flto -fno-fat-lto-objects -flto=auto
Neural Magic DeepSparse Model: NLP Sentiment Analysis, 80% Pruned Quantized BERT Base Uncased - Scenario: Asynchronous Multi-Stream OpenBenchmarking.org ms/batch, Fewer Is Better Neural Magic DeepSparse 1.5 Model: NLP Sentiment Analysis, 80% Pruned Quantized BERT Base Uncased - Scenario: Asynchronous Multi-Stream a b c 15 30 45 60 75 SE +/- 0.10, N = 2 SE +/- 0.10, N = 2 SE +/- 0.99, N = 2 65.28 65.74 66.28
Neural Magic DeepSparse Model: NLP Sentiment Analysis, 80% Pruned Quantized BERT Base Uncased - Scenario: Asynchronous Multi-Stream OpenBenchmarking.org items/sec, More Is Better Neural Magic DeepSparse 1.5 Model: NLP Sentiment Analysis, 80% Pruned Quantized BERT Base Uncased - Scenario: Asynchronous Multi-Stream a b c 110 220 330 440 550 SE +/- 0.68, N = 2 SE +/- 1.03, N = 2 SE +/- 7.21, N = 2 489.77 486.17 482.13
Neural Magic DeepSparse Model: NLP Text Classification, BERT base uncased SST2 - Scenario: Asynchronous Multi-Stream OpenBenchmarking.org ms/batch, Fewer Is Better Neural Magic DeepSparse 1.5 Model: NLP Text Classification, BERT base uncased SST2 - Scenario: Asynchronous Multi-Stream b c a 40 80 120 160 200 SE +/- 0.00, N = 2 SE +/- 0.21, N = 2 SE +/- 0.06, N = 2 192.16 192.21 192.35
Neural Magic DeepSparse Model: NLP Text Classification, BERT base uncased SST2 - Scenario: Asynchronous Multi-Stream OpenBenchmarking.org items/sec, More Is Better Neural Magic DeepSparse 1.5 Model: NLP Text Classification, BERT base uncased SST2 - Scenario: Asynchronous Multi-Stream b c a 40 80 120 160 200 SE +/- 0.06, N = 2 SE +/- 0.08, N = 2 SE +/- 0.05, N = 2 166.22 166.06 166.00
Neural Magic DeepSparse Model: CV Segmentation, 90% Pruned YOLACT Pruned - Scenario: Asynchronous Multi-Stream OpenBenchmarking.org ms/batch, Fewer Is Better Neural Magic DeepSparse 1.5 Model: CV Segmentation, 90% Pruned YOLACT Pruned - Scenario: Asynchronous Multi-Stream c b a 130 260 390 520 650 SE +/- 0.01, N = 2 SE +/- 0.12, N = 2 SE +/- 0.05, N = 2 596.53 596.81 597.97
Neural Magic DeepSparse Model: CV Segmentation, 90% Pruned YOLACT Pruned - Scenario: Asynchronous Multi-Stream OpenBenchmarking.org items/sec, More Is Better Neural Magic DeepSparse 1.5 Model: CV Segmentation, 90% Pruned YOLACT Pruned - Scenario: Asynchronous Multi-Stream c b a 12 24 36 48 60 SE +/- 0.00, N = 2 SE +/- 0.05, N = 2 SE +/- 0.00, N = 2 53.62 53.55 53.48
Neural Magic DeepSparse Model: NLP Text Classification, BERT base uncased SST2, Sparse INT8 - Scenario: Asynchronous Multi-Stream OpenBenchmarking.org ms/batch, Fewer Is Better Neural Magic DeepSparse 1.5 Model: NLP Text Classification, BERT base uncased SST2, Sparse INT8 - Scenario: Asynchronous Multi-Stream a b c 7 14 21 28 35 SE +/- 0.03, N = 2 SE +/- 0.03, N = 2 SE +/- 0.01, N = 2 28.92 28.95 28.96
Neural Magic DeepSparse Model: NLP Text Classification, BERT base uncased SST2, Sparse INT8 - Scenario: Asynchronous Multi-Stream OpenBenchmarking.org items/sec, More Is Better Neural Magic DeepSparse 1.5 Model: NLP Text Classification, BERT base uncased SST2, Sparse INT8 - Scenario: Asynchronous Multi-Stream a b c 200 400 600 800 1000 SE +/- 1.07, N = 2 SE +/- 1.15, N = 2 SE +/- 0.21, N = 2 1105.38 1104.04 1103.36
Neural Magic DeepSparse Model: NLP Document Classification, oBERT base uncased on IMDB - Scenario: Asynchronous Multi-Stream OpenBenchmarking.org ms/batch, Fewer Is Better Neural Magic DeepSparse 1.5 Model: NLP Document Classification, oBERT base uncased on IMDB - Scenario: Asynchronous Multi-Stream c a b 200 400 600 800 1000 SE +/- 0.59, N = 2 SE +/- 1.02, N = 2 SE +/- 0.16, N = 2 840.12 840.54 840.95
Neural Magic DeepSparse Model: NLP Document Classification, oBERT base uncased on IMDB - Scenario: Asynchronous Multi-Stream OpenBenchmarking.org items/sec, More Is Better Neural Magic DeepSparse 1.5 Model: NLP Document Classification, oBERT base uncased on IMDB - Scenario: Asynchronous Multi-Stream a c b 9 18 27 36 45 SE +/- 0.09, N = 2 SE +/- 0.02, N = 2 SE +/- 0.01, N = 2 37.61 37.58 37.53
Neural Magic DeepSparse Model: NLP Token Classification, BERT base uncased conll2003 - Scenario: Asynchronous Multi-Stream OpenBenchmarking.org ms/batch, Fewer Is Better Neural Magic DeepSparse 1.5 Model: NLP Token Classification, BERT base uncased conll2003 - Scenario: Asynchronous Multi-Stream b c a 200 400 600 800 1000 SE +/- 0.01, N = 2 SE +/- 0.44, N = 2 SE +/- 0.49, N = 2 840.26 840.44 841.48
Neural Magic DeepSparse Model: NLP Token Classification, BERT base uncased conll2003 - Scenario: Asynchronous Multi-Stream OpenBenchmarking.org items/sec, More Is Better Neural Magic DeepSparse 1.5 Model: NLP Token Classification, BERT base uncased conll2003 - Scenario: Asynchronous Multi-Stream a c b 9 18 27 36 45 SE +/- 0.04, N = 2 SE +/- 0.02, N = 2 SE +/- 0.01, N = 2 37.61 37.58 37.54
Neural Magic DeepSparse Model: NLP Text Classification, BERT base uncased SST2 - Scenario: Synchronous Single-Stream OpenBenchmarking.org ms/batch, Fewer Is Better Neural Magic DeepSparse 1.5 Model: NLP Text Classification, BERT base uncased SST2 - Scenario: Synchronous Single-Stream c a b 5 10 15 20 25 SE +/- 0.03, N = 2 SE +/- 0.03, N = 2 SE +/- 0.00, N = 2 18.53 18.59 18.61
Neural Magic DeepSparse Model: NLP Text Classification, BERT base uncased SST2 - Scenario: Synchronous Single-Stream OpenBenchmarking.org items/sec, More Is Better Neural Magic DeepSparse 1.5 Model: NLP Text Classification, BERT base uncased SST2 - Scenario: Synchronous Single-Stream c a b 12 24 36 48 60 SE +/- 0.10, N = 2 SE +/- 0.08, N = 2 SE +/- 0.01, N = 2 53.95 53.78 53.70
Neural Magic DeepSparse Model: NLP Document Classification, oBERT base uncased on IMDB - Scenario: Synchronous Single-Stream OpenBenchmarking.org ms/batch, Fewer Is Better Neural Magic DeepSparse 1.5 Model: NLP Document Classification, oBERT base uncased on IMDB - Scenario: Synchronous Single-Stream b c a 11 22 33 44 55 SE +/- 0.11, N = 2 SE +/- 0.05, N = 2 SE +/- 0.02, N = 2 49.92 49.93 50.13
Neural Magic DeepSparse Model: NLP Document Classification, oBERT base uncased on IMDB - Scenario: Synchronous Single-Stream OpenBenchmarking.org items/sec, More Is Better Neural Magic DeepSparse 1.5 Model: NLP Document Classification, oBERT base uncased on IMDB - Scenario: Synchronous Single-Stream b c a 5 10 15 20 25 SE +/- 0.04, N = 2 SE +/- 0.02, N = 2 SE +/- 0.01, N = 2 20.03 20.02 19.95
Neural Magic DeepSparse Model: NLP Token Classification, BERT base uncased conll2003 - Scenario: Synchronous Single-Stream OpenBenchmarking.org ms/batch, Fewer Is Better Neural Magic DeepSparse 1.5 Model: NLP Token Classification, BERT base uncased conll2003 - Scenario: Synchronous Single-Stream a c b 11 22 33 44 55 SE +/- 0.05, N = 2 SE +/- 0.06, N = 2 SE +/- 0.00, N = 2 49.81 49.86 49.91
Neural Magic DeepSparse Model: NLP Token Classification, BERT base uncased conll2003 - Scenario: Synchronous Single-Stream OpenBenchmarking.org items/sec, More Is Better Neural Magic DeepSparse 1.5 Model: NLP Token Classification, BERT base uncased conll2003 - Scenario: Synchronous Single-Stream a c b 5 10 15 20 25 SE +/- 0.02, N = 2 SE +/- 0.02, N = 2 SE +/- 0.00, N = 2 20.07 20.05 20.03
Neural Magic DeepSparse Model: NLP Question Answering, BERT base uncased SQuaD 12layer Pruned90 - Scenario: Synchronous Single-Stream OpenBenchmarking.org ms/batch, Fewer Is Better Neural Magic DeepSparse 1.5 Model: NLP Question Answering, BERT base uncased SQuaD 12layer Pruned90 - Scenario: Synchronous Single-Stream b c a 6 12 18 24 30 SE +/- 0.07, N = 2 SE +/- 0.01, N = 2 SE +/- 0.04, N = 2 25.27 25.30 25.36
Neural Magic DeepSparse Model: NLP Question Answering, BERT base uncased SQuaD 12layer Pruned90 - Scenario: Synchronous Single-Stream OpenBenchmarking.org items/sec, More Is Better Neural Magic DeepSparse 1.5 Model: NLP Question Answering, BERT base uncased SQuaD 12layer Pruned90 - Scenario: Synchronous Single-Stream b c a 9 18 27 36 45 SE +/- 0.11, N = 2 SE +/- 0.01, N = 2 SE +/- 0.06, N = 2 39.56 39.51 39.42
Neural Magic DeepSparse Model: NLP Sentiment Analysis, 80% Pruned Quantized BERT Base Uncased - Scenario: Synchronous Single-Stream OpenBenchmarking.org ms/batch, Fewer Is Better Neural Magic DeepSparse 1.5 Model: NLP Sentiment Analysis, 80% Pruned Quantized BERT Base Uncased - Scenario: Synchronous Single-Stream a b c 3 6 9 12 15 SE +/- 0.08, N = 2 SE +/- 0.01, N = 2 SE +/- 0.06, N = 2 11.54 11.57 11.58
Neural Magic DeepSparse Model: NLP Sentiment Analysis, 80% Pruned Quantized BERT Base Uncased - Scenario: Synchronous Single-Stream OpenBenchmarking.org items/sec, More Is Better Neural Magic DeepSparse 1.5 Model: NLP Sentiment Analysis, 80% Pruned Quantized BERT Base Uncased - Scenario: Synchronous Single-Stream a b c 20 40 60 80 100 SE +/- 0.59, N = 2 SE +/- 0.08, N = 2 SE +/- 0.42, N = 2 86.61 86.39 86.32
Apache IoTDB Device Count: 200 - Batch Size Per Write: 100 - Sensor Count: 500 OpenBenchmarking.org Average Latency, More Is Better Apache IoTDB 1.1.2 Device Count: 200 - Batch Size Per Write: 100 - Sensor Count: 500 b a c 20 40 60 80 100 110.88 109.38 106.73 MAX: 3569.78 MAX: 3597.09 MAX: 3485.91
Apache IoTDB Device Count: 200 - Batch Size Per Write: 100 - Sensor Count: 500 OpenBenchmarking.org point/sec, More Is Better Apache IoTDB 1.1.2 Device Count: 200 - Batch Size Per Write: 100 - Sensor Count: 500 c a b 9M 18M 27M 36M 45M 43363203.76 42048733.22 41987111.39
Neural Magic DeepSparse Model: CV Segmentation, 90% Pruned YOLACT Pruned - Scenario: Synchronous Single-Stream OpenBenchmarking.org ms/batch, Fewer Is Better Neural Magic DeepSparse 1.5 Model: CV Segmentation, 90% Pruned YOLACT Pruned - Scenario: Synchronous Single-Stream c b a 8 16 24 32 40 SE +/- 0.03, N = 2 SE +/- 0.05, N = 2 SE +/- 0.03, N = 2 34.90 34.91 34.91
Neural Magic DeepSparse Model: CV Segmentation, 90% Pruned YOLACT Pruned - Scenario: Synchronous Single-Stream OpenBenchmarking.org items/sec, More Is Better Neural Magic DeepSparse 1.5 Model: CV Segmentation, 90% Pruned YOLACT Pruned - Scenario: Synchronous Single-Stream c b a 7 14 21 28 35 SE +/- 0.02, N = 2 SE +/- 0.04, N = 2 SE +/- 0.03, N = 2 28.64 28.63 28.63
Neural Magic DeepSparse Model: NLP Text Classification, DistilBERT mnli - Scenario: Asynchronous Multi-Stream OpenBenchmarking.org ms/batch, Fewer Is Better Neural Magic DeepSparse 1.5 Model: NLP Text Classification, DistilBERT mnli - Scenario: Asynchronous Multi-Stream b c a 20 40 60 80 100 SE +/- 0.15, N = 2 SE +/- 0.16, N = 2 SE +/- 0.00, N = 2 97.84 97.87 97.87
Neural Magic DeepSparse Model: NLP Text Classification, DistilBERT mnli - Scenario: Asynchronous Multi-Stream OpenBenchmarking.org items/sec, More Is Better Neural Magic DeepSparse 1.5 Model: NLP Text Classification, DistilBERT mnli - Scenario: Asynchronous Multi-Stream b c a 70 140 210 280 350 SE +/- 0.48, N = 2 SE +/- 0.49, N = 2 SE +/- 0.12, N = 2 326.54 326.41 326.31
Apache IoTDB Device Count: 500 - Batch Size Per Write: 100 - Sensor Count: 200 OpenBenchmarking.org Average Latency, More Is Better Apache IoTDB 1.1.2 Device Count: 500 - Batch Size Per Write: 100 - Sensor Count: 200 c b a 9 18 27 36 45 37.12 36.10 35.05 MAX: 2182.81 MAX: 1990.15 MAX: 2157.23
Apache IoTDB Device Count: 500 - Batch Size Per Write: 100 - Sensor Count: 200 OpenBenchmarking.org point/sec, More Is Better Apache IoTDB 1.1.2 Device Count: 500 - Batch Size Per Write: 100 - Sensor Count: 200 a b c 11M 22M 33M 44M 55M 51341708.85 50045888.98 49201448.81
srsRAN Project Test: PUSCH Processor Benchmark, Throughput Total OpenBenchmarking.org Mbps, More Is Better srsRAN Project 23.5 Test: PUSCH Processor Benchmark, Throughput Total c b a 2K 4K 6K 8K 10K SE +/- 56.45, N = 2 SE +/- 44.35, N = 2 SE +/- 13.30, N = 2 9727.1 9718.6 9682.1 1. (CXX) g++ options: -march=native -mfma -O3 -fno-trapping-math -fno-math-errno -lgtest
Neural Magic DeepSparse Model: NLP Text Classification, DistilBERT mnli - Scenario: Synchronous Single-Stream OpenBenchmarking.org ms/batch, Fewer Is Better Neural Magic DeepSparse 1.5 Model: NLP Text Classification, DistilBERT mnli - Scenario: Synchronous Single-Stream b a c 3 6 9 12 15 SE +/- 0.02, N = 2 SE +/- 0.00, N = 2 SE +/- 0.01, N = 2 10.24 10.26 10.31
Neural Magic DeepSparse Model: NLP Text Classification, DistilBERT mnli - Scenario: Synchronous Single-Stream OpenBenchmarking.org items/sec, More Is Better Neural Magic DeepSparse 1.5 Model: NLP Text Classification, DistilBERT mnli - Scenario: Synchronous Single-Stream b a c 20 40 60 80 100 SE +/- 0.17, N = 2 SE +/- 0.04, N = 2 SE +/- 0.08, N = 2 97.61 97.36 96.88
VVenC Video Input: Bosphorus 1080p - Video Preset: Fast OpenBenchmarking.org Frames Per Second, More Is Better VVenC 1.9 Video Input: Bosphorus 1080p - Video Preset: Fast b a c 4 8 12 16 20 SE +/- 0.02, N = 2 SE +/- 0.02, N = 2 SE +/- 0.02, N = 2 16.09 16.08 16.06 1. (CXX) g++ options: -O3 -flto -fno-fat-lto-objects -flto=auto
Neural Magic DeepSparse Model: CV Detection, YOLOv5s COCO, Sparse INT8 - Scenario: Asynchronous Multi-Stream OpenBenchmarking.org ms/batch, Fewer Is Better Neural Magic DeepSparse 1.5 Model: CV Detection, YOLOv5s COCO, Sparse INT8 - Scenario: Asynchronous Multi-Stream b c a 30 60 90 120 150 SE +/- 0.12, N = 2 SE +/- 0.23, N = 2 SE +/- 0.07, N = 2 140.26 140.30 140.40
Neural Magic DeepSparse Model: CV Detection, YOLOv5s COCO, Sparse INT8 - Scenario: Asynchronous Multi-Stream OpenBenchmarking.org items/sec, More Is Better Neural Magic DeepSparse 1.5 Model: CV Detection, YOLOv5s COCO, Sparse INT8 - Scenario: Asynchronous Multi-Stream b c a 50 100 150 200 250 SE +/- 0.19, N = 2 SE +/- 0.22, N = 2 SE +/- 0.11, N = 2 227.64 227.57 227.42
Neural Magic DeepSparse Model: CV Detection, YOLOv5s COCO - Scenario: Asynchronous Multi-Stream OpenBenchmarking.org ms/batch, Fewer Is Better Neural Magic DeepSparse 1.5 Model: CV Detection, YOLOv5s COCO - Scenario: Asynchronous Multi-Stream c b a 30 60 90 120 150 SE +/- 0.05, N = 2 SE +/- 0.09, N = 2 SE +/- 0.13, N = 2 141.48 141.62 141.70
Neural Magic DeepSparse Model: CV Detection, YOLOv5s COCO - Scenario: Asynchronous Multi-Stream OpenBenchmarking.org items/sec, More Is Better Neural Magic DeepSparse 1.5 Model: CV Detection, YOLOv5s COCO - Scenario: Asynchronous Multi-Stream c b a 50 100 150 200 250 SE +/- 0.22, N = 2 SE +/- 0.10, N = 2 SE +/- 0.21, N = 2 225.80 225.46 225.31
Neural Magic DeepSparse Model: ResNet-50, Sparse INT8 - Scenario: Synchronous Single-Stream OpenBenchmarking.org ms/batch, Fewer Is Better Neural Magic DeepSparse 1.5 Model: ResNet-50, Sparse INT8 - Scenario: Synchronous Single-Stream b c a 0.3101 0.6202 0.9303 1.2404 1.5505 SE +/- 0.0055, N = 2 SE +/- 0.0010, N = 2 SE +/- 0.0205, N = 2 1.3623 1.3634 1.3784
Neural Magic DeepSparse Model: ResNet-50, Sparse INT8 - Scenario: Synchronous Single-Stream OpenBenchmarking.org items/sec, More Is Better Neural Magic DeepSparse 1.5 Model: ResNet-50, Sparse INT8 - Scenario: Synchronous Single-Stream b c a 160 320 480 640 800 SE +/- 2.91, N = 2 SE +/- 0.48, N = 2 SE +/- 10.71, N = 2 732.11 731.51 723.71
Neural Magic DeepSparse Model: CV Detection, YOLOv5s COCO - Scenario: Synchronous Single-Stream OpenBenchmarking.org ms/batch, Fewer Is Better Neural Magic DeepSparse 1.5 Model: CV Detection, YOLOv5s COCO - Scenario: Synchronous Single-Stream c b a 2 4 6 8 10 SE +/- 0.0003, N = 2 SE +/- 0.0012, N = 2 SE +/- 0.0056, N = 2 8.3300 8.3368 8.3423
Neural Magic DeepSparse Model: CV Detection, YOLOv5s COCO - Scenario: Synchronous Single-Stream OpenBenchmarking.org items/sec, More Is Better Neural Magic DeepSparse 1.5 Model: CV Detection, YOLOv5s COCO - Scenario: Synchronous Single-Stream c b a 30 60 90 120 150 SE +/- 0.01, N = 2 SE +/- 0.02, N = 2 SE +/- 0.08, N = 2 119.98 119.88 119.80
Neural Magic DeepSparse Model: CV Detection, YOLOv5s COCO, Sparse INT8 - Scenario: Synchronous Single-Stream OpenBenchmarking.org ms/batch, Fewer Is Better Neural Magic DeepSparse 1.5 Model: CV Detection, YOLOv5s COCO, Sparse INT8 - Scenario: Synchronous Single-Stream c a b 2 4 6 8 10 SE +/- 0.0053, N = 2 SE +/- 0.0127, N = 2 SE +/- 0.0014, N = 2 8.2903 8.3056 8.3182
Neural Magic DeepSparse Model: CV Detection, YOLOv5s COCO, Sparse INT8 - Scenario: Synchronous Single-Stream OpenBenchmarking.org items/sec, More Is Better Neural Magic DeepSparse 1.5 Model: CV Detection, YOLOv5s COCO, Sparse INT8 - Scenario: Synchronous Single-Stream c a b 30 60 90 120 150 SE +/- 0.08, N = 2 SE +/- 0.19, N = 2 SE +/- 0.02, N = 2 120.58 120.36 120.18
Neural Magic DeepSparse Model: ResNet-50, Baseline - Scenario: Asynchronous Multi-Stream OpenBenchmarking.org ms/batch, Fewer Is Better Neural Magic DeepSparse 1.5 Model: ResNet-50, Baseline - Scenario: Asynchronous Multi-Stream c b a 15 30 45 60 75 SE +/- 0.01, N = 2 SE +/- 0.04, N = 2 SE +/- 0.04, N = 2 68.25 68.28 68.29
Neural Magic DeepSparse Model: ResNet-50, Baseline - Scenario: Asynchronous Multi-Stream OpenBenchmarking.org items/sec, More Is Better Neural Magic DeepSparse 1.5 Model: ResNet-50, Baseline - Scenario: Asynchronous Multi-Stream c a b 100 200 300 400 500 SE +/- 0.09, N = 2 SE +/- 0.19, N = 2 SE +/- 0.62, N = 2 468.33 468.14 467.97
Neural Magic DeepSparse Model: CV Classification, ResNet-50 ImageNet - Scenario: Asynchronous Multi-Stream OpenBenchmarking.org ms/batch, Fewer Is Better Neural Magic DeepSparse 1.5 Model: CV Classification, ResNet-50 ImageNet - Scenario: Asynchronous Multi-Stream b a c 15 30 45 60 75 SE +/- 0.13, N = 2 SE +/- 0.04, N = 2 SE +/- 0.01, N = 2 68.27 68.32 68.34
Neural Magic DeepSparse Model: CV Classification, ResNet-50 ImageNet - Scenario: Asynchronous Multi-Stream OpenBenchmarking.org items/sec, More Is Better Neural Magic DeepSparse 1.5 Model: CV Classification, ResNet-50 ImageNet - Scenario: Asynchronous Multi-Stream b a c 100 200 300 400 500 SE +/- 1.23, N = 2 SE +/- 0.36, N = 2 SE +/- 0.26, N = 2 468.11 467.98 467.64
Neural Magic DeepSparse Model: ResNet-50, Baseline - Scenario: Synchronous Single-Stream OpenBenchmarking.org ms/batch, Fewer Is Better Neural Magic DeepSparse 1.5 Model: ResNet-50, Baseline - Scenario: Synchronous Single-Stream c a b 2 4 6 8 10 SE +/- 0.0308, N = 2 SE +/- 0.0166, N = 2 SE +/- 0.0087, N = 2 6.2238 6.2525 6.2566
Neural Magic DeepSparse Model: ResNet-50, Baseline - Scenario: Synchronous Single-Stream OpenBenchmarking.org items/sec, More Is Better Neural Magic DeepSparse 1.5 Model: ResNet-50, Baseline - Scenario: Synchronous Single-Stream c a b 40 80 120 160 200 SE +/- 0.80, N = 2 SE +/- 0.43, N = 2 SE +/- 0.22, N = 2 160.59 159.85 159.75
Neural Magic DeepSparse Model: CV Classification, ResNet-50 ImageNet - Scenario: Synchronous Single-Stream OpenBenchmarking.org ms/batch, Fewer Is Better Neural Magic DeepSparse 1.5 Model: CV Classification, ResNet-50 ImageNet - Scenario: Synchronous Single-Stream c a b 2 4 6 8 10 SE +/- 0.0244, N = 2 SE +/- 0.0097, N = 2 SE +/- 0.0121, N = 2 6.2195 6.2481 6.2495
Neural Magic DeepSparse Model: CV Classification, ResNet-50 ImageNet - Scenario: Synchronous Single-Stream OpenBenchmarking.org items/sec, More Is Better Neural Magic DeepSparse 1.5 Model: CV Classification, ResNet-50 ImageNet - Scenario: Synchronous Single-Stream c a b 40 80 120 160 200 SE +/- 0.62, N = 2 SE +/- 0.24, N = 2 SE +/- 0.32, N = 2 160.70 159.97 159.92
Neural Magic DeepSparse Model: ResNet-50, Sparse INT8 - Scenario: Asynchronous Multi-Stream OpenBenchmarking.org ms/batch, Fewer Is Better Neural Magic DeepSparse 1.5 Model: ResNet-50, Sparse INT8 - Scenario: Asynchronous Multi-Stream b c a 2 4 6 8 10 SE +/- 0.0212, N = 2 SE +/- 0.0366, N = 2 SE +/- 0.0013, N = 2 8.3441 8.3468 8.3665
Neural Magic DeepSparse Model: ResNet-50, Sparse INT8 - Scenario: Asynchronous Multi-Stream OpenBenchmarking.org items/sec, More Is Better Neural Magic DeepSparse 1.5 Model: ResNet-50, Sparse INT8 - Scenario: Asynchronous Multi-Stream b c a 800 1600 2400 3200 4000 SE +/- 10.54, N = 2 SE +/- 17.23, N = 2 SE +/- 0.54, N = 2 3824.30 3823.08 3814.52
Apache IoTDB Device Count: 500 - Batch Size Per Write: 1 - Sensor Count: 500 OpenBenchmarking.org Average Latency, More Is Better Apache IoTDB 1.1.2 Device Count: 500 - Batch Size Per Write: 1 - Sensor Count: 500 c a b 7 14 21 28 35 31.4 27.1 26.0 MAX: 890.05 MAX: 934.45 MAX: 873.88
Apache IoTDB Device Count: 500 - Batch Size Per Write: 1 - Sensor Count: 500 OpenBenchmarking.org point/sec, More Is Better Apache IoTDB 1.1.2 Device Count: 500 - Batch Size Per Write: 1 - Sensor Count: 500 b a c 400K 800K 1200K 1600K 2000K 1686943.16 1636128.73 1446487.70
Blender Blend File: Fishy Cat - Compute: CPU-Only OpenBenchmarking.org Seconds, Fewer Is Better Blender 3.6 Blend File: Fishy Cat - Compute: CPU-Only a c b 8 16 24 32 40 SE +/- 0.02, N = 2 SE +/- 0.01, N = 2 SE +/- 0.25, N = 2 33.70 33.72 33.76
Blender Blend File: BMW27 - Compute: CPU-Only OpenBenchmarking.org Seconds, Fewer Is Better Blender 3.6 Blend File: BMW27 - Compute: CPU-Only c a b 6 12 18 24 30 SE +/- 0.04, N = 2 SE +/- 0.06, N = 2 SE +/- 0.15, N = 2 27.24 27.27 27.50
Apache IoTDB Device Count: 100 - Batch Size Per Write: 100 - Sensor Count: 500 OpenBenchmarking.org Average Latency, More Is Better Apache IoTDB 1.1.2 Device Count: 100 - Batch Size Per Write: 100 - Sensor Count: 500 b a c 20 40 60 80 100 82.26 81.20 79.16 MAX: 864.29 MAX: 1009.28 MAX: 1006.03
Apache IoTDB Device Count: 100 - Batch Size Per Write: 100 - Sensor Count: 500 OpenBenchmarking.org point/sec, More Is Better Apache IoTDB 1.1.2 Device Count: 100 - Batch Size Per Write: 100 - Sensor Count: 500 c a b 11M 22M 33M 44M 55M 52464142.83 51316464.44 50507747.12
Apache IoTDB Device Count: 200 - Batch Size Per Write: 100 - Sensor Count: 200 OpenBenchmarking.org Average Latency, More Is Better Apache IoTDB 1.1.2 Device Count: 200 - Batch Size Per Write: 100 - Sensor Count: 200 a c b 8 16 24 32 40 35.09 34.79 33.84 MAX: 804.64 MAX: 780.01 MAX: 773.52
Apache IoTDB Device Count: 200 - Batch Size Per Write: 100 - Sensor Count: 200 OpenBenchmarking.org point/sec, More Is Better Apache IoTDB 1.1.2 Device Count: 200 - Batch Size Per Write: 100 - Sensor Count: 200 b c a 10M 20M 30M 40M 50M 47245476.78 46674344.69 46437377.67
Apache IoTDB Device Count: 200 - Batch Size Per Write: 1 - Sensor Count: 500 OpenBenchmarking.org Average Latency, More Is Better Apache IoTDB 1.1.2 Device Count: 200 - Batch Size Per Write: 1 - Sensor Count: 500 b a c 8 16 24 32 40 33.86 33.50 32.71 MAX: 659.59 MAX: 690.29 MAX: 725.08
Apache IoTDB Device Count: 200 - Batch Size Per Write: 1 - Sensor Count: 500 OpenBenchmarking.org point/sec, More Is Better Apache IoTDB 1.1.2 Device Count: 200 - Batch Size Per Write: 1 - Sensor Count: 500 c a b 300K 600K 900K 1200K 1500K 1261385.89 1232509.19 1226219.88
Apache IoTDB Device Count: 500 - Batch Size Per Write: 1 - Sensor Count: 200 OpenBenchmarking.org Average Latency, More Is Better Apache IoTDB 1.1.2 Device Count: 500 - Batch Size Per Write: 1 - Sensor Count: 200 a c b 3 6 9 12 15 13.54 11.83 11.63 MAX: 856.65 MAX: 836.9 MAX: 860.78
Apache IoTDB Device Count: 500 - Batch Size Per Write: 1 - Sensor Count: 200 OpenBenchmarking.org point/sec, More Is Better Apache IoTDB 1.1.2 Device Count: 500 - Batch Size Per Write: 1 - Sensor Count: 200 c b a 300K 600K 900K 1200K 1500K 1367763.49 1365831.50 1182440.62
VVenC Video Input: Bosphorus 1080p - Video Preset: Faster OpenBenchmarking.org Frames Per Second, More Is Better VVenC 1.9 Video Input: Bosphorus 1080p - Video Preset: Faster c b a 7 14 21 28 35 SE +/- 0.07, N = 2 SE +/- 0.11, N = 2 SE +/- 0.07, N = 2 29.47 29.39 29.35 1. (CXX) g++ options: -O3 -flto -fno-fat-lto-objects -flto=auto
srsRAN Project Test: Downlink Processor Benchmark OpenBenchmarking.org Mbps, More Is Better srsRAN Project 23.5 Test: Downlink Processor Benchmark b a c 140 280 420 560 700 SE +/- 27.75, N = 2 SE +/- 17.85, N = 2 SE +/- 0.35, N = 2 658.1 657.7 619.3 1. (CXX) g++ options: -march=native -mfma -O3 -fno-trapping-math -fno-math-errno -lgtest
Apache IoTDB Device Count: 100 - Batch Size Per Write: 100 - Sensor Count: 200 OpenBenchmarking.org Average Latency, More Is Better Apache IoTDB 1.1.2 Device Count: 100 - Batch Size Per Write: 100 - Sensor Count: 200 b a c 8 16 24 32 40 36.85 36.04 35.01 MAX: 721.27 MAX: 804.01 MAX: 746.4
Apache IoTDB Device Count: 100 - Batch Size Per Write: 100 - Sensor Count: 200 OpenBenchmarking.org point/sec, More Is Better Apache IoTDB 1.1.2 Device Count: 100 - Batch Size Per Write: 100 - Sensor Count: 200 c a b 9M 18M 27M 36M 45M 39945212.99 39287432.92 38401769.17
Apache IoTDB Device Count: 100 - Batch Size Per Write: 1 - Sensor Count: 500 OpenBenchmarking.org Average Latency, More Is Better Apache IoTDB 1.1.2 Device Count: 100 - Batch Size Per Write: 1 - Sensor Count: 500 a c b 8 16 24 32 40 34.36 34.08 32.92 MAX: 704.53 MAX: 699.28 MAX: 728.63
Apache IoTDB Device Count: 100 - Batch Size Per Write: 1 - Sensor Count: 500 OpenBenchmarking.org point/sec, More Is Better Apache IoTDB 1.1.2 Device Count: 100 - Batch Size Per Write: 1 - Sensor Count: 500 b c a 200K 400K 600K 800K 1000K 1069145.79 1044153.44 1038515.62
Apache IoTDB Device Count: 200 - Batch Size Per Write: 1 - Sensor Count: 200 OpenBenchmarking.org Average Latency, More Is Better Apache IoTDB 1.1.2 Device Count: 200 - Batch Size Per Write: 1 - Sensor Count: 200 c a b 4 8 12 16 20 16.07 15.24 13.60 MAX: 592.48 MAX: 583.94 MAX: 586.94
Apache IoTDB Device Count: 200 - Batch Size Per Write: 1 - Sensor Count: 200 OpenBenchmarking.org point/sec, More Is Better Apache IoTDB 1.1.2 Device Count: 200 - Batch Size Per Write: 1 - Sensor Count: 200 b a c 200K 400K 600K 800K 1000K 978176.76 898967.08 870795.92
Apache IoTDB Device Count: 100 - Batch Size Per Write: 1 - Sensor Count: 200 OpenBenchmarking.org Average Latency, More Is Better Apache IoTDB 1.1.2 Device Count: 100 - Batch Size Per Write: 1 - Sensor Count: 200 a b c 4 8 12 16 20 17.45 17.28 16.35 MAX: 645.35 MAX: 644.33 MAX: 668.86
Apache IoTDB Device Count: 100 - Batch Size Per Write: 1 - Sensor Count: 200 OpenBenchmarking.org point/sec, More Is Better Apache IoTDB 1.1.2 Device Count: 100 - Batch Size Per Write: 1 - Sensor Count: 200 c b a 140K 280K 420K 560K 700K 667880.96 648308.27 644019.72
srsRAN Project Test: PUSCH Processor Benchmark, Throughput Thread OpenBenchmarking.org Mbps, More Is Better srsRAN Project 23.5 Test: PUSCH Processor Benchmark, Throughput Thread a c b 50 100 150 200 250 SE +/- 0.10, N = 2 SE +/- 0.20, N = 2 SE +/- 1.90, N = 2 211.1 210.8 208.2 1. (CXX) g++ options: -march=native -mfma -O3 -fno-trapping-math -fno-math-errno -lgtest
Phoronix Test Suite v10.8.4