2 x AMD EPYC 9654 96-Core testing with a AMD Titanite_4G (RTI1004D BIOS) and llvmpipe on Red Hat Enterprise Linux 9.1 via the Phoronix Test Suite.
Compare your own system(s) to this result file with the
Phoronix Test Suite by running the command:
phoronix-test-suite benchmark 2303114-NE-9654NEW5019 9654 new - Phoronix Test Suite 9654 new 2 x AMD EPYC 9654 96-Core testing with a AMD Titanite_4G (RTI1004D BIOS) and llvmpipe on Red Hat Enterprise Linux 9.1 via the Phoronix Test Suite.
HTML result view exported from: https://openbenchmarking.org/result/2303114-NE-9654NEW5019&grs&export=pdf&sro&rro .
9654 new Processor Motherboard Chipset Memory Disk Graphics Monitor Network OS Kernel Desktop Display Server Compiler File-System Screen Resolution OpenGL a b c no smt a no smt b smt a smt b smt c smt d AMD EPYC 9654 96-Core @ 2.40GHz (96 Cores / 192 Threads) AMD Titanite_4G (RTI1004D BIOS) AMD Device 14a4 768GB 2 x 1920GB SAMSUNG MZWLJ1T9HBJR-00007 ASPEED VGA HDMI Broadcom NetXtreme BCM5720 PCIe Red Hat Enterprise Linux 9.1 5.14.0-162.6.1.el9_1.x86_64 (x86_64) GNOME Shell 40.10 X Server 1.20.11 GCC 11.3.1 20220421 xfs 1600x1200 2 x AMD EPYC 9654 96-Core @ 2.40GHz (192 Cores) 1520GB llvmpipe 4.5 Mesa 22.1.5 (LLVM 14.0.6 256 bits) 1024x768 2 x AMD EPYC 9654 96-Core @ 2.40GHz (192 Cores / 384 Threads) OpenBenchmarking.org Kernel Details - Transparent Huge Pages: always Compiler Details - --build=x86_64-redhat-linux --disable-libunwind-exceptions --enable-__cxa_atexit --enable-bootstrap --enable-cet --enable-checking=release --enable-gnu-indirect-function --enable-gnu-unique-object --enable-host-bind-now --enable-host-pie --enable-initfini-array --enable-languages=c,c++,fortran,lto --enable-link-serialization=1 --enable-multilib --enable-offload-targets=nvptx-none --enable-plugin --enable-shared --enable-threads=posix --mandir=/usr/share/man --with-arch_32=x86-64 --with-arch_64=x86-64-v2 --with-build-config=bootstrap-lto --with-gcc-major-version-only --with-linker-hash-style=gnu --with-tune=generic --without-cuda-driver --without-isl Processor Details - Scaling Governor: acpi-cpufreq performance (Boost: Enabled) - CPU Microcode: 0xa101111 Python Details - Python 3.9.14 Security Details - a: SELinux + itlb_multihit: Not affected + l1tf: Not affected + mds: Not affected + meltdown: Not affected + mmio_stale_data: Not affected + retbleed: Not affected + spec_store_bypass: Mitigation of SSB disabled via prctl + spectre_v1: Mitigation of usercopy/swapgs barriers and __user pointer sanitization + spectre_v2: Mitigation of Retpolines IBPB: conditional IBRS_FW STIBP: always-on RSB filling PBRSB-eIBRS: Not affected + srbds: Not affected + tsx_async_abort: Not affected - b: SELinux + itlb_multihit: Not affected + l1tf: Not affected + mds: Not affected + meltdown: Not affected + mmio_stale_data: Not affected + retbleed: Not affected + spec_store_bypass: Mitigation of SSB disabled via prctl + spectre_v1: Mitigation of usercopy/swapgs barriers and __user pointer sanitization + spectre_v2: Mitigation of Retpolines IBPB: conditional IBRS_FW STIBP: always-on RSB filling PBRSB-eIBRS: Not affected + srbds: Not affected + tsx_async_abort: Not affected - c: SELinux + itlb_multihit: Not affected + l1tf: Not affected + mds: Not affected + meltdown: Not affected + mmio_stale_data: Not affected + retbleed: Not affected + spec_store_bypass: Mitigation of SSB disabled via prctl + spectre_v1: Mitigation of usercopy/swapgs barriers and __user pointer sanitization + spectre_v2: Mitigation of Retpolines IBPB: conditional IBRS_FW STIBP: always-on RSB filling PBRSB-eIBRS: Not affected + srbds: Not affected + tsx_async_abort: Not affected - no smt a: SELinux + itlb_multihit: Not affected + l1tf: Not affected + mds: Not affected + meltdown: Not affected + mmio_stale_data: Not affected + retbleed: Not affected + spec_store_bypass: Mitigation of SSB disabled via prctl + spectre_v1: Mitigation of usercopy/swapgs barriers and __user pointer sanitization + spectre_v2: Mitigation of Retpolines IBPB: conditional IBRS_FW STIBP: disabled RSB filling PBRSB-eIBRS: Not affected + srbds: Not affected + tsx_async_abort: Not affected - no smt b: SELinux + itlb_multihit: Not affected + l1tf: Not affected + mds: Not affected + meltdown: Not affected + mmio_stale_data: Not affected + retbleed: Not affected + spec_store_bypass: Mitigation of SSB disabled via prctl + spectre_v1: Mitigation of usercopy/swapgs barriers and __user pointer sanitization + spectre_v2: Mitigation of Retpolines IBPB: conditional IBRS_FW STIBP: disabled RSB filling PBRSB-eIBRS: Not affected + srbds: Not affected + tsx_async_abort: Not affected - smt a: SELinux + itlb_multihit: Not affected + l1tf: Not affected + mds: Not affected + meltdown: Not affected + mmio_stale_data: Not affected + retbleed: Not affected + spec_store_bypass: Mitigation of SSB disabled via prctl + spectre_v1: Mitigation of usercopy/swapgs barriers and __user pointer sanitization + spectre_v2: Mitigation of Retpolines IBPB: conditional IBRS_FW STIBP: always-on RSB filling PBRSB-eIBRS: Not affected + srbds: Not affected + tsx_async_abort: Not affected - smt b: SELinux + itlb_multihit: Not affected + l1tf: Not affected + mds: Not affected + meltdown: Not affected + mmio_stale_data: Not affected + retbleed: Not affected + spec_store_bypass: Mitigation of SSB disabled via prctl + spectre_v1: Mitigation of usercopy/swapgs barriers and __user pointer sanitization + spectre_v2: Mitigation of Retpolines IBPB: conditional IBRS_FW STIBP: always-on RSB filling PBRSB-eIBRS: Not affected + srbds: Not affected + tsx_async_abort: Not affected - smt c: SELinux + itlb_multihit: Not affected + l1tf: Not affected + mds: Not affected + meltdown: Not affected + mmio_stale_data: Not affected + retbleed: Not affected + spec_store_bypass: Mitigation of SSB disabled via prctl + spectre_v1: Mitigation of usercopy/swapgs barriers and __user pointer sanitization + spectre_v2: Mitigation of Retpolines IBPB: conditional IBRS_FW STIBP: always-on RSB filling PBRSB-eIBRS: Not affected + srbds: Not affected + tsx_async_abort: Not affected - smt d: SELinux + itlb_multihit: Not affected + l1tf: Not affected + mds: Not affected + meltdown: Not affected + mmio_stale_data: Not affected + retbleed: Not affected + spec_store_bypass: Mitigation of SSB disabled via prctl + spectre_v1: Mitigation of usercopy/swapgs barriers and __user pointer sanitization + spectre_v2: Mitigation of Retpolines IBPB: conditional IBRS_FW STIBP: always-on RSB filling PBRSB-eIBRS: Not affected + srbds: Not affected + tsx_async_abort: Not affected
9654 new stress-ng: MMAP onednn: IP Shapes 1D - bf16bf16bf16 - CPU onednn: Recurrent Neural Network Inference - f32 - CPU onednn: IP Shapes 1D - f32 - CPU onednn: Recurrent Neural Network Inference - u8s8f32 - CPU onednn: Recurrent Neural Network Inference - bf16bf16bf16 - CPU onednn: IP Shapes 1D - u8s8f32 - CPU stress-ng: Context Switching stress-ng: NUMA onednn: Deconvolution Batch shapes_1d - f32 - CPU openvino: Vehicle Detection FP16 - CPU openvino: Vehicle Detection FP16 - CPU stress-ng: Pthread rocksdb: Rand Read stress-ng: Matrix Math stress-ng: CPU Cache stress-ng: CPU Stress stress-ng: Vector Math stress-ng: Mutex stress-ng: Crypto onednn: IP Shapes 3D - f32 - CPU stress-ng: SENDFILE stress-ng: Function Call onednn: Matrix Multiply Batch Shapes Transformer - u8s8f32 - CPU stress-ng: Atomic deepsparse: NLP Token Classification, BERT base uncased conll2003 - Asynchronous Multi-Stream stress-ng: Glibc Qsort Data Sorting deepsparse: NLP Document Classification, oBERT base uncased on IMDB - Asynchronous Multi-Stream onednn: Deconvolution Batch shapes_1d - u8s8f32 - CPU stress-ng: Glibc C String Functions deepsparse: CV Classification, ResNet-50 ImageNet - Asynchronous Multi-Stream onednn: Matrix Multiply Batch Shapes Transformer - bf16bf16bf16 - CPU stress-ng: Hash openvino: Face Detection FP16-INT8 - CPU openvino: Face Detection FP16 - CPU openvino: Face Detection FP16 - CPU openvino: Weld Porosity Detection FP16-INT8 - CPU openvino: Face Detection FP16-INT8 - CPU onednn: IP Shapes 3D - bf16bf16bf16 - CPU onednn: Recurrent Neural Network Training - f32 - CPU openvino: Weld Porosity Detection FP16 - CPU openvino: Weld Porosity Detection FP16 - CPU deepsparse: CV Detection, YOLOv5s COCO - Asynchronous Multi-Stream onednn: Recurrent Neural Network Training - u8s8f32 - CPU onednn: Recurrent Neural Network Training - bf16bf16bf16 - CPU deepsparse: CV Segmentation, 90% Pruned YOLACT Pruned - Asynchronous Multi-Stream deepsparse: NLP Text Classification, DistilBERT mnli - Asynchronous Multi-Stream deepsparse: NLP Text Classification, BERT base uncased SST2 - Asynchronous Multi-Stream openvino: Person Detection FP32 - CPU openvino: Vehicle Detection FP16-INT8 - CPU openvino: Vehicle Detection FP16-INT8 - CPU openvino: Person Detection FP32 - CPU deepsparse: NLP Question Answering, BERT base uncased SQuaD 12layer Pruned90 - Asynchronous Multi-Stream stress-ng: Malloc openvino: Person Detection FP16 - CPU openvino: Person Detection FP16 - CPU openvino: Age Gender Recognition Retail 0013 FP16-INT8 - CPU rocksdb: Read While Writing deepsparse: NLP Sentiment Analysis, 80% Pruned Quantized BERT Base Uncased - Asynchronous Multi-Stream onednn: Convolution Batch Shapes Auto - bf16bf16bf16 - CPU openvino: Machine Translation EN To DE FP16 - CPU openvino: Machine Translation EN To DE FP16 - CPU onednn: Matrix Multiply Batch Shapes Transformer - f32 - CPU onednn: IP Shapes 3D - u8s8f32 - CPU onednn: Deconvolution Batch shapes_3d - u8s8f32 - CPU stress-ng: MEMFD stress-ng: Memory Copying stress-ng: Forking stress-ng: Futex gromacs: MPI CPU - water_GMX50_bare rocksdb: Read Rand Write Rand onednn: Deconvolution Batch shapes_3d - f32 - CPU onednn: Convolution Batch Shapes Auto - f32 - CPU openvino: Person Vehicle Bike Detection FP16 - CPU openvino: Person Vehicle Bike Detection FP16 - CPU memcached: 1:5 stress-ng: System V Message Passing onednn: Deconvolution Batch shapes_1d - bf16bf16bf16 - CPU kvazaar: Bosphorus 4K - Slow memcached: 1:10 onednn: Convolution Batch Shapes Auto - u8s8f32 - CPU kvazaar: Bosphorus 4K - Medium uvg266: Bosphorus 4K - Slow openvino: Age Gender Recognition Retail 0013 FP16 - CPU onednn: Deconvolution Batch shapes_3d - bf16bf16bf16 - CPU memcached: 1:100 stress-ng: Semaphores build-linux-kernel: defconfig stress-ng: Poll openvkl: vklBenchmark Scalar deepsparse: NLP Text Classification, DistilBERT mnli - Synchronous Single-Stream deepsparse: NLP Text Classification, DistilBERT mnli - Synchronous Single-Stream uvg266: Bosphorus 4K - Medium deepsparse: NLP Question Answering, BERT base uncased SQuaD 12layer Pruned90 - Synchronous Single-Stream deepsparse: NLP Question Answering, BERT base uncased SQuaD 12layer Pruned90 - Synchronous Single-Stream deepsparse: CV Classification, ResNet-50 ImageNet - Synchronous Single-Stream deepsparse: CV Classification, ResNet-50 ImageNet - Synchronous Single-Stream deepsparse: NLP Text Classification, BERT base uncased SST2 - Synchronous Single-Stream deepsparse: NLP Text Classification, BERT base uncased SST2 - Synchronous Single-Stream uvg266: Bosphorus 1080p - Super Fast uvg266: Bosphorus 1080p - Ultra Fast compress-zstd: 12 - Compression Speed rocksdb: Update Rand rocksdb: Seq Fill deepsparse: CV Segmentation, 90% Pruned YOLACT Pruned - Synchronous Single-Stream deepsparse: CV Segmentation, 90% Pruned YOLACT Pruned - Synchronous Single-Stream uvg266: Bosphorus 1080p - Very Fast clickhouse: 100M Rows Hits Dataset, Second Run rocksdb: Rand Fill clickhouse: 100M Rows Hits Dataset, First Run / Cold Cache deepsparse: CV Detection, YOLOv5s COCO - Synchronous Single-Stream deepsparse: CV Detection, YOLOv5s COCO - Synchronous Single-Stream clickhouse: 100M Rows Hits Dataset, Third Run vpxenc: Speed 5 - Bosphorus 4K openvino: Age Gender Recognition Retail 0013 FP16 - CPU uvg266: Bosphorus 4K - Ultra Fast openvkl: vklBenchmark ISPC compress-zstd: 8 - Compression Speed compress-zstd: 3 - Compression Speed build-ffmpeg: Time To Compile kvazaar: Bosphorus 1080p - Very Fast uvg266: Bosphorus 4K - Very Fast kvazaar: Bosphorus 1080p - Slow kvazaar: Bosphorus 1080p - Super Fast uvg266: Bosphorus 4K - Super Fast deepsparse: NLP Token Classification, BERT base uncased conll2003 - Synchronous Single-Stream deepsparse: NLP Token Classification, BERT base uncased conll2003 - Synchronous Single-Stream compress-zstd: 3, Long Mode - Compression Speed kvazaar: Bosphorus 1080p - Medium deepsparse: NLP Document Classification, oBERT base uncased on IMDB - Synchronous Single-Stream deepsparse: NLP Document Classification, oBERT base uncased on IMDB - Synchronous Single-Stream rocksdb: Rand Fill Sync deepsparse: NLP Document Classification, oBERT base uncased on IMDB - Asynchronous Multi-Stream deepsparse: NLP Token Classification, BERT base uncased conll2003 - Asynchronous Multi-Stream kvazaar: Bosphorus 4K - Very Fast kvazaar: Bosphorus 1080p - Ultra Fast kvazaar: Bosphorus 4K - Super Fast kvazaar: Bosphorus 4K - Ultra Fast openvino: Age Gender Recognition Retail 0013 FP16-INT8 - CPU deepsparse: NLP Sentiment Analysis, 80% Pruned Quantized BERT Base Uncased - Synchronous Single-Stream deepsparse: NLP Sentiment Analysis, 80% Pruned Quantized BERT Base Uncased - Synchronous Single-Stream deepsparse: CV Classification, ResNet-50 ImageNet - Asynchronous Multi-Stream vpxenc: Speed 0 - Bosphorus 4K uvg266: Bosphorus 1080p - Slow uvg266: Bosphorus 1080p - Medium openvino: Weld Porosity Detection FP16-INT8 - CPU compress-zstd: 8, Long Mode - Compression Speed deepsparse: CV Detection, YOLOv5s COCO - Asynchronous Multi-Stream vpxenc: Speed 0 - Bosphorus 1080p compress-zstd: 19, Long Mode - Compression Speed deepsparse: CV Segmentation, 90% Pruned YOLACT Pruned - Asynchronous Multi-Stream deepsparse: NLP Text Classification, DistilBERT mnli - Asynchronous Multi-Stream vvenc: Bosphorus 1080p - Faster compress-zstd: 19 - Compression Speed deepsparse: NLP Text Classification, BERT base uncased SST2 - Asynchronous Multi-Stream deepsparse: NLP Question Answering, BERT base uncased SQuaD 12layer Pruned90 - Asynchronous Multi-Stream stress-ng: Socket Activity compress-zstd: 19 - Decompression Speed compress-zstd: 12 - Decompression Speed vvenc: Bosphorus 4K - Fast compress-zstd: 19, Long Mode - Decompression Speed vvenc: Bosphorus 4K - Faster compress-zstd: 3 - Decompression Speed compress-zstd: 8 - Decompression Speed vpxenc: Speed 5 - Bosphorus 1080p deepsparse: NLP Sentiment Analysis, 80% Pruned Quantized BERT Base Uncased - Asynchronous Multi-Stream vvenc: Bosphorus 1080p - Fast compress-zstd: 8, Long Mode - Decompression Speed compress-zstd: 3, Long Mode - Decompression Speed embree: Pathtracer - Crown a b c no smt a no smt b smt a smt b smt c smt d 1663.08 8.74576 674.088 2.01581 671.69 675.915 3.11497 18941003.97 483.9 7.35746 16.3 2941.47 109397.78 468231434 382305.69 77.51 205134.87 556875.59 59479783.64 203073.15 1.56461 1950323.96 621015.34 0.263997 174.72 42.9581 1122.39 42.9893 0.555277 16257100.25 1009.8486 0.2902 18954936 95.76 962.1 49.72 9494.22 499.32 1.87777 907.489 4914.17 9.76 440.8533 911.245 909.691 149.7446 619.7531 314.8361 27.97 8.11 5908.89 1701.7 401.7766 312709034.05 28.47 1671.46 74495.17 9296185 1566.564 0.415446 540.41 88.71 0.357017 0.53445 0.304972 518.46 20106.51 58156.36 2794694.37 10.569 2926458 1.17004 0.693367 5750.74 8.34 4143044.21 10473084.09 1.83147 40.63 6861088.89 0.311405 41.4 29.37 112346.36 0.711588 4878860 18128283.29 25.768 12653709.64 556 5.1812 192.8806 33.03 99.1493 10.0811 5.3018 188.5378 11.362 87.9631 238.68 240.98 330.8 544384 542256 16.5828 60.2522 234.95 625.91 533927 600.70 9.5962 104.1178 625.16 17.36 0.65 70.56 1089 1233.8 3033.9 12.569 296.93 68.82 139.92 307.49 69.33 28.6233 34.9289 892.7 143.31 34.8262 28.7077 376601 1115.7602 1116.426 80.9 310.73 81.9 83.45 1.09 197.1018 5.0715 47.464 7.68 81.16 91.37 10.1 938.5 108.6775 14.69 9.22 319.9815 77.3363 30.039 19.1 152.1377 119.2067 8873.58 1472.6 1704.8 5.82 1383.7 12.336 1514.8 1669.3 29.37 30.5924 12.44 1677.9 1536.6 1664.75 9.1957 671.654 1.9234 670.202 668.952 4.27858 16313126.86 498.67 7.28243 15.54 3085.53 109356.78 466888888 382304.04 67.21 217072.27 556797.27 60031543.97 203147.15 1.56257 1913590.77 621003.92 0.263373 223.29 42.8348 1132.35 42.897 0.551966 16168655.17 1012.3494 0.291387 18955118.1 95.48 962.63 49.72 9496.95 500.83 1.72975 903.272 4915.42 9.75 439.0373 912.114 917.077 149.8207 619.9442 316.2907 28.23 8.13 5894.27 1685.26 401.2053 314418461.02 28.42 1675.96 74353.41 8620352 1573.7496 0.578466 541.03 88.61 0.356875 0.513068 0.305731 507.74 20340.4 58664.97 2805836.52 10.609 2891962 1.16696 0.689064 5720.9 8.38 4162812.47 10471889.97 1.84172 40.86 6792746.34 0.312721 41.39 29.29 111378.39 0.712495 4876951.36 18100474.36 25.749 12661687.46 557 5.1633 193.5305 33.13 98.6175 10.136 5.498 181.8123 11.1872 89.3355 239.92 240.91 332 545556 545396 16.6745 59.9222 234.68 627.20 534681 610.79 9.5793 104.2908 628.37 17.37 0.66 70.68 1075 1239.3 3095 12.434 290.59 68.79 139.07 303.99 69 28.7137 34.8189 909.4 144.21 35.0962 28.4866 373002 1118.4383 1116.2899 81.61 305.52 80.68 84.77 1.09 196.5414 5.0859 47.3532 7.71 81.41 91.47 10.1 910.8 109.1129 14.67 9.29 319.9648 77.305 29.845 19.1 151.6003 119.3799 8851.65 1467.8 1716.7 5.809 1378.2 12.391 1516.6 1651 29.5 30.4608 12.399 1673.7 1537.5 1668.92 3.83017 669.146 1.88686 671.233 667.95 4.3972 16862185.54 478.05 7.38116 15.1 3174.04 109609.15 468069792 382328.5 97.04 217304.19 556833.5 59929401.4 203095.44 1.56742 1891202.37 621041.93 0.26292 183.33 42.9152 1125.86 42.7433 0.556851 16537965.14 1013.7357 0.285432 18961773.18 95.59 962.15 49.77 9485.65 500.3 1.6386 906.139 4910.13 9.76 438.4372 915.084 911.762 149.6905 625.5512 316.6282 27.89 8.13 5898.79 1704.02 400.0427 313768771.26 28.33 1679.03 74486.08 8316379 1571.8281 0.417188 535.78 89.46 0.353175 0.546991 0.305599 507.97 20297.91 64299.62 2794473.75 10.587 2910023 1.16919 0.692667 5724.81 8.37 4155273.46 10475486.39 1.82264 40.76 6839975.87 0.312289 41.47 29.41 112186.25 0.708563 4852421.67 18088584.67 25.68 12676101.41 549 5.1696 193.3119 33.1 98.5922 10.1384 5.5007 181.7243 11.1605 89.547 237.96 238.77 330.1 543572 544565 16.5185 60.4884 237.44 621.88 536551 614.24 8.8004 113.5057 636.18 17.46 0.66 71.13 1066 1241.1 3049.4 12.465 291.03 69.04 140.1 301.4 69.92 28.5698 34.9937 916.9 143.81 35.1188 28.4687 356922 1122.0584 1117.7323 80.31 309.64 84.55 82.7 1.09 197.6601 5.0573 47.2942 7.7 81.1 91.29 10.11 926.9 109.2467 14.76 9.3 320.0504 76.6181 30.081 19.1 151.3899 119.7026 8876.1 1470.5 1715.6 5.808 1384 12.399 1517.4 1661.9 29.46 30.4911 12.441 1675.4 1540.9 4520.48 6.19257 930.257 2.04003 913.201 973.318 4.87883 47222624.49 20.51 9.25292 6.01 7979.05 68076.03 1209611055 932248.18 40.94 328297.04 920216.27 63581395.67 435615.35 2.19645 3284433.2 829106.63 0.316665 400.03 97.3996 1978.8 97.4442 0.664359 26755146.69 2204.7533 0.316779 27408413.1 208.82 438.29 109.08 20836.1 229.46 2.02286 1148.26 10556.93 4.54 941.2443 1151.25 1147.34 308.4294 1291.9733 649.302 57.18 3.95 12119.66 833.99 813.9832 456657338.84 57.58 828.67 127833.52 7643831 3107.2957 0.291687 1045.81 45.86 0.342836 0.556069 0.160361 303.56 11342.69 43094.96 3746361.52 19.175 2079804 0.664295 0.40359 9784.09 4.9 2444886.82 7402514.74 1.65041 47.13 4244908.83 0.254845 47.97 34.59 160545.41 0.462386 3192061.68 13141129.68 17.606 10458275.24 647 7.5856 131.7586 38.38 72.1298 13.8559 6.0118 166.2725 14.4638 69.1063 181.67 186.24 279.9 462018 465700 20.7385 48.1849 196.25 666.43 478629 635.48 9.8899 101.0095 665.00 14.48 0.57 57.78 1098 1227.5 2804.2 10.597 269.62 57.13 155.56 288.41 59.83 31.9229 31.3187 955.6 159.63 31.9146 31.327 350673 955.1631 956.0088 70.02 278.62 75.5 76.05 1.01 191.2605 5.2262 42.805 7.04 87.15 98.21 9.18 859.9 100.2418 14.28 9.39 304.2075 73.1993 28.689 19.8 145.3548 116.1532 8968.28 1483.1 1728 5.89 1395.8 12.477 1500.4 1669.7 29.62 30.4213 12.357 1684.8 1542.5 3591.71 7.98861 901.044 1.93126 903.358 939.547 4.79302 44683906.68 19.78 9.31751 6.01 7979.82 67451.3 1213540299 925984.24 55.84 326819.72 920642.34 65500297.73 437065.75 2.12379 3282827.5 829445.38 0.293564 395.95 97.3441 1963.75 97.3838 0.670574 28009942.56 2203.1573 0.340498 27422305.53 210.14 438.99 109.04 20849.75 228.01 2.02833 1124.75 10549.24 4.54 938.9084 1119.17 1139.63 308.3033 1292.8038 647.7596 57.19 3.95 12128.38 834.17 817.6313 456651508.04 57.49 830.04 127770.39 7913568 3086.3591 0.291229 1048.74 45.73 0.347231 0.629026 0.164845 308.67 10949.9 45685.28 3802292.6 18.837 2063831 0.651301 0.400325 9758.74 4.91 2460416.33 7372780.72 1.70057 47.43 4220522.49 0.255796 47.93 34.68 162294.47 0.461174 3216133.91 13192391.85 17.586 10393402.01 652 6.4174 155.7357 38.65 71.8878 13.9039 6.065 164.8126 14.4178 69.3266 216.15 209.85 278.2 452228 464044 20.4223 48.9257 183.75 649.52 468210 622.73 9.9395 100.509 662.01 14.5 0.58 57.23 1108 1234.2 2865.9 10.662 268.77 58.09 159.79 280.18 58.33 32.1778 31.07 1032.3 161.4 29.7544 33.6012 344040 955.2386 955.3438 69.94 271.45 74.03 74.37 1.01 175.3199 5.7015 42.8393 7.16 88.13 96.67 9.18 892 100.4198 13.72 9.76 303.7377 73.2087 28.52 19.8 145.3627 115.6369 8924.1 1483.8 1727.8 5.895 1397.6 12.31 1511.5 1667.6 29.71 30.6403 12.374 1682.8 1538.8 8360.76 15.421 3187 7.60566 3269.05 3133.65 11.4326 12895047.73 24.82 20.2695 6.02 7953.21 74978.51 1225662852 946032.88 47.58 487359.39 1291689.33 136339636.91 466292.12 3.39151 4329963.26 1414423.65 0.591738 184.34 97.2916 2564.48 97.0249 1.20049 35687958.37 2230.3099 0.621776 41966139.67 208.13 439.41 108.93 20610.23 230.21 3.38833 1912.16 10544.03 4.54 929.0246 1830.41 1888.1 314.6186 1306.6677 658.2111 57.29 3.95 12117.16 833.15 817.7536 634718750.41 57.72 827.07 148316.88 15317250 3114.6848 0.451472 1049.16 45.71 0.674455 0.975357 0.279638 464.7 15914.1 36266.02 2333781.39 18.818 1787682 0.97859 0.538539 9746.08 4.92 2520253 10103952.13 2.70807 64.63 4530948.62 0.40743 65.56 45.56 173926.92 0.672585 4383314.65 20047519.05 17.376 15359471.73 764 7.5225 132.8655 47.56 77.8827 12.8322 7.1865 139.0973 13.637 73.2924 178.49 216.15 254.1 413199 414168 21.0323 47.506 183.99 524.04 423027 500.13 8.1914 121.9232 534.82 13.9 0.53 57.96 1318 1023.9 2795.1 10.381 250.71 59.3 132.67 296.72 59.06 32.2956 30.9572 1051.4 140.78 31.7592 31.4805 388052 972.5163 970.2202 75.91 302.55 76.64 77.96 0.97 188.5826 5.3004 42.9515 7.24 80.17 89.4 9.28 853.8 103.0888 14.08 9.78 303.3272 73.306 18.8 145.3966 117.1233 8748.99 1495.2 1723.7 1393.5 1515.6 1664.3 29.54 30.7462 1682.5 1539.5 9017.12 19.549 3034.62 7.4227 3249.28 3222.93 7.30147 12221058.34 24.77 20.572 6.03 7949.05 180407.59 1231168093 946660.88 44.38 487987.32 1291704.57 135855921.28 466609.22 3.49453 4329824.38 1413555.44 0.600199 186.64 97.3528 2520.38 96.906 1.18135 35996976.13 2245.8036 0.620907 41989522.68 208.19 439.5 108.89 20610.62 230.06 3.59643 1890.28 10549.15 4.54 927.679 1937.65 1894.07 314.2452 1306.5839 658.341 57.31 3.96 12109.64 832.7 822.1552 634995429.66 57.67 827.03 147582.87 13830689 3114.4293 0.450261 1050.37 45.66 0.675664 0.971704 0.291087 564.05 15430.18 36020.16 2396325.93 19.015 1761416 0.973555 0.551783 9680.44 4.95 2507331.29 8586858.51 2.7614 66.22 4392934.75 0.408074 65.69 46.25 175754.35 0.672768 4357453.76 19927313.52 17.017 15341564.16 810 6.8844 145.1736 46.62 77.4096 12.9106 7.3054 136.8324 14.6356 68.2924 220.79 179.56 259.1 411513 413708 21.7398 45.9642 209.55 527.88 417882 516.09 8.3506 119.5964 538.63 14.27 0.53 57.51 1243 1122.4 2513.4 10.259 274.63 57.86 136.33 267.04 59.2 32.2415 31.0093 1038.9 138.33 31.7806 31.4593 401295 975.504 969.2122 74.11 295.18 76.7 77 0.97 188.4147 5.3053 42.6536 6.97 79.95 89.7 9.28 852.7 103.1902 13.57 9.87 304.0782 73.3496 19.5 145.2745 116.4813 8750.61 1483.5 1726.3 1392.4 1519.1 1671 29.6 30.7508 1676.5 1540.4 7633.16 18.6594 3011.21 9.34981 3153.22 3185.64 12.013 12328933.65 24.79 21.0301 5.99 7993.18 77834.39 1234570512 952461.48 42.55 490300.48 1295970.28 138939958.5 468159.21 3.60194 4351871.12 1422505.51 0.593299 184.24 96.9077 2517.26 96.9975 1.20146 34602827.77 2253.1411 0.634992 41972765.93 210.19 437.51 109.36 20684.29 227.93 3.36535 1888.21 10603.84 4.52 937.992 1926.15 1925.68 315.3318 1305.7137 662.2585 57.71 3.93 12168.05 826.67 818.6037 639757070.52 58.05 821.93 148047.13 13948914 3132.3838 0.44782 1058.57 45.31 0.676124 0.99571 0.275917 413.34 15077.69 34917.39 2067499.58 19.045 1613913 0.972863 0.545118 9773.26 4.9 2517750.47 8609357.68 2.70235 65.49 4549313.42 0.394377 65.59 46.34 173620.31 0.676798 4394194.49 19866440.86 17.053 15228320.07 793 7.355 135.8907 46.57 70.1418 14.2486 6.1057 163.7111 14.2011 70.3828 178.78 218.28 249.8 419985 414282 20.4884 48.772 192.25 524.75 416006 495.85 10.3852 96.1737 530.47 15.38 0.53 57.33 1235 1005.8 2604.6 10.407 243.36 57.44 135.8 288.09 58.34 33.9879 29.4163 1046.6 140.89 31.9277 31.3146 394852 971.8988 971.6677 73.57 303.43 78.72 76.14 0.96 189.4054 5.2773 42.5521 6.92 79.63 89.16 9.25 860.8 102.0999 13.97 9.82 303.2549 73.3407 19.2 144.4661 116.9689 8747.83 1475.2 1732.3 1389 1516.2 1664 29.54 30.5784 1683.7 1540.9 7273.19 11.7573 3349.36 8.26078 3171.15 3172.19 12.1571 12728401.45 24.71 20.9726 6 7990.47 91735.75 1231916197 951700.17 40.56 489998.75 1300693.93 137579874.56 468006 3.50563 4351920.42 1425621.44 0.602831 182.8 97.8908 2516.09 96.7169 1.2334 36111781.82 2255.422 0.627133 41965286.06 209.24 437.68 109.23 20679.68 228.94 2.97427 1957.77 10587.3 4.52 931.8223 1883.76 1912.74 316.0328 1301.1811 660.2132 57.62 3.93 12172.92 828.01 816.7562 640853365.54 58 822.5 149647.68 13098097 3116.0909 0.457554 1057.74 45.34 0.672115 0.975091 0.247825 394.49 13370.33 34685.85 2077119.76 19.077 1752904 0.973984 0.63063 9767.99 4.91 2527084.87 12418451.71 2.67303 66.08 4569476.86 0.389223 65.49 46.25 172228.34 0.680395 4403267.42 19842969.28 17.079 15403597.24 781 7.064 141.4705 46.21 75.524 13.2331 7.149 139.8241 15.126 66.083 218.91 224.31 256.2 420514 414047 21.3593 46.7815 181.83 515.11 415428 500.118340751 9.3656 106.6518 527.80 15.11 0.53 58.76 1238 1024.8 2528.7 10.373 270.76 57.52 135.56 256.45 58.66 32.1758 31.0727 1059 136.42 30.2494 33.0514 404463 974.985 966.7606 73.24 303.63 75.5 78.85 0.97 183.949 5.4341 42.4798 7.15 80 88.91 9.26 879.6 102.7646 13.68 9.81 301.9192 73.6196 19.6 144.9144 117.2219 8864.78 1479.1 1731.2 1391.7 1513.5 1664.2 29.66 30.7247 1677.6 1543.6 OpenBenchmarking.org
Stress-NG Test: MMAP OpenBenchmarking.org Bogo Ops/s, More Is Better Stress-NG 0.15.04 Test: MMAP smt d smt c smt b smt a no smt b no smt a c b a 2K 4K 6K 8K 10K 7273.19 7633.16 9017.12 8360.76 3591.71 4520.48 1668.92 1664.75 1663.08 1. (CC) gcc options: -std=gnu99 -O2 -lm -lc -lcrypt -ldl -lrt -pthread
oneDNN Harness: IP Shapes 1D - Data Type: bf16bf16bf16 - Engine: CPU OpenBenchmarking.org ms, Fewer Is Better oneDNN 3.0 Harness: IP Shapes 1D - Data Type: bf16bf16bf16 - Engine: CPU smt d smt c smt b smt a no smt b no smt a c b a 5 10 15 20 25 11.75730 18.65940 19.54900 15.42100 7.98861 6.19257 3.83017 9.19570 8.74576 MIN: 8.17 MIN: 11.23 MIN: 10.67 MIN: 10.4 MIN: 3.88 MIN: 3.65 MIN: 2.72 MIN: 4.13 MIN: 3.65 1. (CXX) g++ options: -O3 -march=native -fopenmp -msse4.1 -fPIC -pie -ldl -lpthread
oneDNN Harness: Recurrent Neural Network Inference - Data Type: f32 - Engine: CPU OpenBenchmarking.org ms, Fewer Is Better oneDNN 3.0 Harness: Recurrent Neural Network Inference - Data Type: f32 - Engine: CPU smt d smt c smt b smt a no smt b no smt a c b a 700 1400 2100 2800 3500 3349.36 3011.21 3034.62 3187.00 901.04 930.26 669.15 671.65 674.09 MIN: 3325.75 MIN: 2853.26 MIN: 2821.2 MIN: 3034.82 MIN: 864.28 MIN: 898.89 MIN: 661.71 MIN: 664.19 MIN: 667.01 1. (CXX) g++ options: -O3 -march=native -fopenmp -msse4.1 -fPIC -pie -ldl -lpthread
oneDNN Harness: IP Shapes 1D - Data Type: f32 - Engine: CPU OpenBenchmarking.org ms, Fewer Is Better oneDNN 3.0 Harness: IP Shapes 1D - Data Type: f32 - Engine: CPU smt d smt c smt b smt a no smt b no smt a c b a 3 6 9 12 15 8.26078 9.34981 7.42270 7.60566 1.93126 2.04003 1.88686 1.92340 2.01581 MIN: 7.09 MIN: 7.73 MIN: 6.44 MIN: 6.54 MIN: 1.77 MIN: 1.78 MIN: 1.68 MIN: 1.72 MIN: 1.81 1. (CXX) g++ options: -O3 -march=native -fopenmp -msse4.1 -fPIC -pie -ldl -lpthread
oneDNN Harness: Recurrent Neural Network Inference - Data Type: u8s8f32 - Engine: CPU OpenBenchmarking.org ms, Fewer Is Better oneDNN 3.0 Harness: Recurrent Neural Network Inference - Data Type: u8s8f32 - Engine: CPU smt d smt c smt b smt a no smt b no smt a c b a 700 1400 2100 2800 3500 3171.15 3153.22 3249.28 3269.05 903.36 913.20 671.23 670.20 671.69 MIN: 3148.29 MIN: 3059.88 MIN: 3226.78 MIN: 3243.11 MIN: 874.93 MIN: 884.92 MIN: 662.9 MIN: 663.54 MIN: 664.46 1. (CXX) g++ options: -O3 -march=native -fopenmp -msse4.1 -fPIC -pie -ldl -lpthread
oneDNN Harness: Recurrent Neural Network Inference - Data Type: bf16bf16bf16 - Engine: CPU OpenBenchmarking.org ms, Fewer Is Better oneDNN 3.0 Harness: Recurrent Neural Network Inference - Data Type: bf16bf16bf16 - Engine: CPU smt d smt c smt b smt a no smt b no smt a c b a 700 1400 2100 2800 3500 3172.19 3185.64 3222.93 3133.65 939.55 973.32 667.95 668.95 675.92 MIN: 3155.85 MIN: 2949.58 MIN: 2927.41 MIN: 3037.06 MIN: 904.78 MIN: 937.3 MIN: 660.93 MIN: 662.29 MIN: 668.99 1. (CXX) g++ options: -O3 -march=native -fopenmp -msse4.1 -fPIC -pie -ldl -lpthread
oneDNN Harness: IP Shapes 1D - Data Type: u8s8f32 - Engine: CPU OpenBenchmarking.org ms, Fewer Is Better oneDNN 3.0 Harness: IP Shapes 1D - Data Type: u8s8f32 - Engine: CPU smt d smt c smt b smt a no smt b no smt a c b a 3 6 9 12 15 12.15710 12.01300 7.30147 11.43260 4.79302 4.87883 4.39720 4.27858 3.11497 MIN: 8.52 MIN: 8.02 MIN: 5.77 MIN: 7.59 MIN: 3.4 MIN: 3.53 MIN: 3.21 MIN: 3.37 MIN: 2.47 1. (CXX) g++ options: -O3 -march=native -fopenmp -msse4.1 -fPIC -pie -ldl -lpthread
Stress-NG Test: Context Switching OpenBenchmarking.org Bogo Ops/s, More Is Better Stress-NG 0.15.04 Test: Context Switching smt d smt c smt b smt a no smt b no smt a c b a 10M 20M 30M 40M 50M 12728401.45 12328933.65 12221058.34 12895047.73 44683906.68 47222624.49 16862185.54 16313126.86 18941003.97 1. (CC) gcc options: -std=gnu99 -O2 -lm -lc -lcrypt -ldl -lrt -pthread
Stress-NG Test: NUMA OpenBenchmarking.org Bogo Ops/s, More Is Better Stress-NG 0.15.04 Test: NUMA smt d smt c smt b smt a no smt b no smt a c b a 110 220 330 440 550 24.71 24.79 24.77 24.82 19.78 20.51 478.05 498.67 483.90 1. (CC) gcc options: -std=gnu99 -O2 -lm -lc -lcrypt -ldl -lrt -pthread
oneDNN Harness: Deconvolution Batch shapes_1d - Data Type: f32 - Engine: CPU OpenBenchmarking.org ms, Fewer Is Better oneDNN 3.0 Harness: Deconvolution Batch shapes_1d - Data Type: f32 - Engine: CPU smt d smt c smt b smt a no smt b no smt a c b a 5 10 15 20 25 20.97260 21.03010 20.57200 20.26950 9.31751 9.25292 7.38116 7.28243 7.35746 MIN: 18.21 MIN: 17.96 MIN: 18.25 MIN: 17.68 MIN: 8.07 MIN: 7.79 MIN: 6.77 MIN: 6.58 MIN: 4.85 1. (CXX) g++ options: -O3 -march=native -fopenmp -msse4.1 -fPIC -pie -ldl -lpthread
OpenVINO Model: Vehicle Detection FP16 - Device: CPU OpenBenchmarking.org ms, Fewer Is Better OpenVINO 2022.3 Model: Vehicle Detection FP16 - Device: CPU smt d smt c smt b smt a no smt b no smt a c b a 4 8 12 16 20 6.00 5.99 6.03 6.02 6.01 6.01 15.10 15.54 16.30 MIN: 5.21 / MAX: 25.51 MIN: 5.13 / MAX: 31.29 MIN: 5.17 / MAX: 38.35 MIN: 5.27 / MAX: 25.12 MIN: 5.02 / MAX: 36.88 MIN: 5.2 / MAX: 37.8 MIN: 6.94 / MAX: 60.59 MIN: 8.28 / MAX: 57.72 MIN: 7.91 / MAX: 51.62 1. (CXX) g++ options: -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -pie -ldl
OpenVINO Model: Vehicle Detection FP16 - Device: CPU OpenBenchmarking.org FPS, More Is Better OpenVINO 2022.3 Model: Vehicle Detection FP16 - Device: CPU smt d smt c smt b smt a no smt b no smt a c b a 2K 4K 6K 8K 10K 7990.47 7993.18 7949.05 7953.21 7979.82 7979.05 3174.04 3085.53 2941.47 1. (CXX) g++ options: -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -pie -ldl
Stress-NG Test: Pthread OpenBenchmarking.org Bogo Ops/s, More Is Better Stress-NG 0.15.04 Test: Pthread smt d smt c smt b smt a no smt b no smt a c b a 40K 80K 120K 160K 200K 91735.75 77834.39 180407.59 74978.51 67451.30 68076.03 109609.15 109356.78 109397.78 1. (CC) gcc options: -std=gnu99 -O2 -lm -lc -lcrypt -ldl -lrt -pthread
RocksDB Test: Random Read OpenBenchmarking.org Op/s, More Is Better RocksDB 7.9.2 Test: Random Read smt d smt c smt b smt a no smt b no smt a c b a 300M 600M 900M 1200M 1500M 1231916197 1234570512 1231168093 1225662852 1213540299 1209611055 468069792 466888888 468231434 1. (CXX) g++ options: -O3 -march=native -pthread -fno-builtin-memcmp -fno-rtti
Stress-NG Test: Matrix Math OpenBenchmarking.org Bogo Ops/s, More Is Better Stress-NG 0.15.04 Test: Matrix Math smt d smt c smt b smt a no smt b no smt a c b a 200K 400K 600K 800K 1000K 951700.17 952461.48 946660.88 946032.88 925984.24 932248.18 382328.50 382304.04 382305.69 1. (CC) gcc options: -std=gnu99 -O2 -lm -lc -lcrypt -ldl -lrt -pthread
Stress-NG Test: CPU Cache OpenBenchmarking.org Bogo Ops/s, More Is Better Stress-NG 0.15.04 Test: CPU Cache smt d smt c smt b smt a no smt b no smt a c b a 20 40 60 80 100 40.56 42.55 44.38 47.58 55.84 40.94 97.04 67.21 77.51 1. (CC) gcc options: -std=gnu99 -O2 -lm -lc -lcrypt -ldl -lrt -pthread
Stress-NG Test: CPU Stress OpenBenchmarking.org Bogo Ops/s, More Is Better Stress-NG 0.15.04 Test: CPU Stress smt d smt c smt b smt a no smt b no smt a c b a 110K 220K 330K 440K 550K 489998.75 490300.48 487987.32 487359.39 326819.72 328297.04 217304.19 217072.27 205134.87 1. (CC) gcc options: -std=gnu99 -O2 -lm -lc -lcrypt -ldl -lrt -pthread
Stress-NG Test: Vector Math OpenBenchmarking.org Bogo Ops/s, More Is Better Stress-NG 0.15.04 Test: Vector Math smt d smt c smt b smt a no smt b no smt a c b a 300K 600K 900K 1200K 1500K 1300693.93 1295970.28 1291704.57 1291689.33 920642.34 920216.27 556833.50 556797.27 556875.59 1. (CC) gcc options: -std=gnu99 -O2 -lm -lc -lcrypt -ldl -lrt -pthread
Stress-NG Test: Mutex OpenBenchmarking.org Bogo Ops/s, More Is Better Stress-NG 0.15.04 Test: Mutex smt d smt c smt b smt a no smt b no smt a c b a 30M 60M 90M 120M 150M 137579874.56 138939958.50 135855921.28 136339636.91 65500297.73 63581395.67 59929401.40 60031543.97 59479783.64 1. (CC) gcc options: -std=gnu99 -O2 -lm -lc -lcrypt -ldl -lrt -pthread
Stress-NG Test: Crypto OpenBenchmarking.org Bogo Ops/s, More Is Better Stress-NG 0.15.04 Test: Crypto smt d smt c smt b smt a no smt b no smt a c b a 100K 200K 300K 400K 500K 468006.00 468159.21 466609.22 466292.12 437065.75 435615.35 203095.44 203147.15 203073.15 1. (CC) gcc options: -std=gnu99 -O2 -lm -lc -lcrypt -ldl -lrt -pthread
oneDNN Harness: IP Shapes 3D - Data Type: f32 - Engine: CPU OpenBenchmarking.org ms, Fewer Is Better oneDNN 3.0 Harness: IP Shapes 3D - Data Type: f32 - Engine: CPU smt d smt c smt b smt a no smt b no smt a c b a 0.8104 1.6208 2.4312 3.2416 4.052 3.50563 3.60194 3.49453 3.39151 2.12379 2.19645 1.56742 1.56257 1.56461 MIN: 3.19 MIN: 3.08 MIN: 3.11 MIN: 2.93 MIN: 1.9 MIN: 2 MIN: 1.39 MIN: 1.41 MIN: 1.41 1. (CXX) g++ options: -O3 -march=native -fopenmp -msse4.1 -fPIC -pie -ldl -lpthread
Stress-NG Test: SENDFILE OpenBenchmarking.org Bogo Ops/s, More Is Better Stress-NG 0.15.04 Test: SENDFILE smt d smt c smt b smt a no smt b no smt a c b a 900K 1800K 2700K 3600K 4500K 4351920.42 4351871.12 4329824.38 4329963.26 3282827.50 3284433.20 1891202.37 1913590.77 1950323.96 1. (CC) gcc options: -std=gnu99 -O2 -lm -lc -lcrypt -ldl -lrt -pthread
Stress-NG Test: Function Call OpenBenchmarking.org Bogo Ops/s, More Is Better Stress-NG 0.15.04 Test: Function Call smt d smt c smt b smt a no smt b no smt a c b a 300K 600K 900K 1200K 1500K 1425621.44 1422505.51 1413555.44 1414423.65 829445.38 829106.63 621041.93 621003.92 621015.34 1. (CC) gcc options: -std=gnu99 -O2 -lm -lc -lcrypt -ldl -lrt -pthread
oneDNN Harness: Matrix Multiply Batch Shapes Transformer - Data Type: u8s8f32 - Engine: CPU OpenBenchmarking.org ms, Fewer Is Better oneDNN 3.0 Harness: Matrix Multiply Batch Shapes Transformer - Data Type: u8s8f32 - Engine: CPU smt d smt c smt b smt a no smt b no smt a c b a 0.1356 0.2712 0.4068 0.5424 0.678 0.602831 0.593299 0.600199 0.591738 0.293564 0.316665 0.262920 0.263373 0.263997 MIN: 0.48 MIN: 0.39 MIN: 0.38 MIN: 0.43 MIN: 0.22 MIN: 0.24 MIN: 0.2 MIN: 0.2 MIN: 0.18 1. (CXX) g++ options: -O3 -march=native -fopenmp -msse4.1 -fPIC -pie -ldl -lpthread
Stress-NG Test: Atomic OpenBenchmarking.org Bogo Ops/s, More Is Better Stress-NG 0.15.04 Test: Atomic smt d smt c smt b smt a no smt b no smt a c b a 90 180 270 360 450 182.80 184.24 186.64 184.34 395.95 400.03 183.33 223.29 174.72 1. (CC) gcc options: -std=gnu99 -O2 -lm -lc -lcrypt -ldl -lrt -pthread
Neural Magic DeepSparse Model: NLP Token Classification, BERT base uncased conll2003 - Scenario: Asynchronous Multi-Stream OpenBenchmarking.org items/sec, More Is Better Neural Magic DeepSparse 1.3.2 Model: NLP Token Classification, BERT base uncased conll2003 - Scenario: Asynchronous Multi-Stream smt d smt c smt b smt a no smt b no smt a c b a 20 40 60 80 100 97.89 96.91 97.35 97.29 97.34 97.40 42.92 42.83 42.96
Stress-NG Test: Glibc Qsort Data Sorting OpenBenchmarking.org Bogo Ops/s, More Is Better Stress-NG 0.15.04 Test: Glibc Qsort Data Sorting smt d smt c smt b smt a no smt b no smt a c b a 600 1200 1800 2400 3000 2516.09 2517.26 2520.38 2564.48 1963.75 1978.80 1125.86 1132.35 1122.39 1. (CC) gcc options: -std=gnu99 -O2 -lm -lc -lcrypt -ldl -lrt -pthread
Neural Magic DeepSparse Model: NLP Document Classification, oBERT base uncased on IMDB - Scenario: Asynchronous Multi-Stream OpenBenchmarking.org items/sec, More Is Better Neural Magic DeepSparse 1.3.2 Model: NLP Document Classification, oBERT base uncased on IMDB - Scenario: Asynchronous Multi-Stream smt d smt c smt b smt a no smt b no smt a c b a 20 40 60 80 100 96.72 97.00 96.91 97.02 97.38 97.44 42.74 42.90 42.99
oneDNN Harness: Deconvolution Batch shapes_1d - Data Type: u8s8f32 - Engine: CPU OpenBenchmarking.org ms, Fewer Is Better oneDNN 3.0 Harness: Deconvolution Batch shapes_1d - Data Type: u8s8f32 - Engine: CPU smt d smt c smt b smt a no smt b no smt a c b a 0.2775 0.555 0.8325 1.11 1.3875 1.233400 1.201460 1.181350 1.200490 0.670574 0.664359 0.556851 0.551966 0.555277 MIN: 1.08 MIN: 1.08 MIN: 1.08 MIN: 1.04 MIN: 0.55 MIN: 0.56 MIN: 0.53 MIN: 0.49 MIN: 0.53 1. (CXX) g++ options: -O3 -march=native -fopenmp -msse4.1 -fPIC -pie -ldl -lpthread
Stress-NG Test: Glibc C String Functions OpenBenchmarking.org Bogo Ops/s, More Is Better Stress-NG 0.15.04 Test: Glibc C String Functions smt d smt c smt b smt a no smt b no smt a c b a 8M 16M 24M 32M 40M 36111781.82 34602827.77 35996976.13 35687958.37 28009942.56 26755146.69 16537965.14 16168655.17 16257100.25 1. (CC) gcc options: -std=gnu99 -O2 -lm -lc -lcrypt -ldl -lrt -pthread
Neural Magic DeepSparse Model: CV Classification, ResNet-50 ImageNet - Scenario: Asynchronous Multi-Stream OpenBenchmarking.org items/sec, More Is Better Neural Magic DeepSparse 1.3.2 Model: CV Classification, ResNet-50 ImageNet - Scenario: Asynchronous Multi-Stream smt d smt c smt b smt a no smt b no smt a c b a 500 1000 1500 2000 2500 2255.42 2253.14 2245.80 2230.31 2203.16 2204.75 1013.74 1012.35 1009.85
oneDNN Harness: Matrix Multiply Batch Shapes Transformer - Data Type: bf16bf16bf16 - Engine: CPU OpenBenchmarking.org ms, Fewer Is Better oneDNN 3.0 Harness: Matrix Multiply Batch Shapes Transformer - Data Type: bf16bf16bf16 - Engine: CPU smt d smt c smt b smt a no smt b no smt a c b a 0.1429 0.2858 0.4287 0.5716 0.7145 0.627133 0.634992 0.620907 0.621776 0.340498 0.316779 0.285432 0.291387 0.290200 MIN: 0.4 MIN: 0.55 MIN: 0.41 MIN: 0.45 MIN: 0.28 MIN: 0.25 MIN: 0.24 MIN: 0.23 MIN: 0.25 1. (CXX) g++ options: -O3 -march=native -fopenmp -msse4.1 -fPIC -pie -ldl -lpthread
Stress-NG Test: Hash OpenBenchmarking.org Bogo Ops/s, More Is Better Stress-NG 0.15.04 Test: Hash smt d smt c smt b smt a no smt b no smt a c b a 9M 18M 27M 36M 45M 41965286.06 41972765.93 41989522.68 41966139.67 27422305.53 27408413.10 18961773.18 18955118.10 18954936.00 1. (CC) gcc options: -std=gnu99 -O2 -lm -lc -lcrypt -ldl -lrt -pthread
OpenVINO Model: Face Detection FP16-INT8 - Device: CPU OpenBenchmarking.org FPS, More Is Better OpenVINO 2022.3 Model: Face Detection FP16-INT8 - Device: CPU smt d smt c smt b smt a no smt b no smt a c b a 50 100 150 200 250 209.24 210.19 208.19 208.13 210.14 208.82 95.59 95.48 95.76 1. (CXX) g++ options: -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -pie -ldl
OpenVINO Model: Face Detection FP16 - Device: CPU OpenBenchmarking.org ms, Fewer Is Better OpenVINO 2022.3 Model: Face Detection FP16 - Device: CPU smt d smt c smt b smt a no smt b no smt a c b a 200 400 600 800 1000 437.68 437.51 439.50 439.41 438.99 438.29 962.15 962.63 962.10 MIN: 394.89 / MAX: 478.58 MIN: 400.05 / MAX: 473.91 MIN: 424.22 / MAX: 477.66 MIN: 410.81 / MAX: 465.4 MIN: 427.57 / MAX: 484.31 MIN: 416.93 / MAX: 496.86 MIN: 888.7 / MAX: 1017.92 MIN: 893.43 / MAX: 1015.71 MIN: 879.24 / MAX: 1018.81 1. (CXX) g++ options: -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -pie -ldl
OpenVINO Model: Face Detection FP16 - Device: CPU OpenBenchmarking.org FPS, More Is Better OpenVINO 2022.3 Model: Face Detection FP16 - Device: CPU smt d smt c smt b smt a no smt b no smt a c b a 20 40 60 80 100 109.23 109.36 108.89 108.93 109.04 109.08 49.77 49.72 49.72 1. (CXX) g++ options: -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -pie -ldl
OpenVINO Model: Weld Porosity Detection FP16-INT8 - Device: CPU OpenBenchmarking.org FPS, More Is Better OpenVINO 2022.3 Model: Weld Porosity Detection FP16-INT8 - Device: CPU smt d smt c smt b smt a no smt b no smt a c b a 4K 8K 12K 16K 20K 20679.68 20684.29 20610.62 20610.23 20849.75 20836.10 9485.65 9496.95 9494.22 1. (CXX) g++ options: -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -pie -ldl
OpenVINO Model: Face Detection FP16-INT8 - Device: CPU OpenBenchmarking.org ms, Fewer Is Better OpenVINO 2022.3 Model: Face Detection FP16-INT8 - Device: CPU smt d smt c smt b smt a no smt b no smt a c b a 110 220 330 440 550 228.94 227.93 230.06 230.21 228.01 229.46 500.30 500.83 499.32 MIN: 214.92 / MAX: 248.29 MIN: 210.46 / MAX: 252.96 MIN: 210.58 / MAX: 251.42 MIN: 211.56 / MAX: 253.58 MIN: 212.99 / MAX: 267.17 MIN: 217.93 / MAX: 265.81 MIN: 410.04 / MAX: 531.93 MIN: 418.76 / MAX: 546.62 MIN: 264.54 / MAX: 537.26 1. (CXX) g++ options: -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -pie -ldl
oneDNN Harness: IP Shapes 3D - Data Type: bf16bf16bf16 - Engine: CPU OpenBenchmarking.org ms, Fewer Is Better oneDNN 3.0 Harness: IP Shapes 3D - Data Type: bf16bf16bf16 - Engine: CPU smt d smt c smt b smt a no smt b no smt a c b a 0.8092 1.6184 2.4276 3.2368 4.046 2.97427 3.36535 3.59643 3.38833 2.02833 2.02286 1.63860 1.72975 1.87777 MIN: 2.3 MIN: 2.7 MIN: 2.7 MIN: 2.76 MIN: 1.43 MIN: 1.56 MIN: 1.2 MIN: 1.27 MIN: 1.23 1. (CXX) g++ options: -O3 -march=native -fopenmp -msse4.1 -fPIC -pie -ldl -lpthread
oneDNN Harness: Recurrent Neural Network Training - Data Type: f32 - Engine: CPU OpenBenchmarking.org ms, Fewer Is Better oneDNN 3.0 Harness: Recurrent Neural Network Training - Data Type: f32 - Engine: CPU smt d smt c smt b smt a no smt b no smt a c b a 400 800 1200 1600 2000 1957.77 1888.21 1890.28 1912.16 1124.75 1148.26 906.14 903.27 907.49 MIN: 1929.47 MIN: 1860.29 MIN: 1864.29 MIN: 1891.03 MIN: 1091.36 MIN: 1108.42 MIN: 898.41 MIN: 894.76 MIN: 897.95 1. (CXX) g++ options: -O3 -march=native -fopenmp -msse4.1 -fPIC -pie -ldl -lpthread
OpenVINO Model: Weld Porosity Detection FP16 - Device: CPU OpenBenchmarking.org FPS, More Is Better OpenVINO 2022.3 Model: Weld Porosity Detection FP16 - Device: CPU smt d smt c smt b smt a no smt b no smt a c b a 2K 4K 6K 8K 10K 10587.30 10603.84 10549.15 10544.03 10549.24 10556.93 4910.13 4915.42 4914.17 1. (CXX) g++ options: -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -pie -ldl
OpenVINO Model: Weld Porosity Detection FP16 - Device: CPU OpenBenchmarking.org ms, Fewer Is Better OpenVINO 2022.3 Model: Weld Porosity Detection FP16 - Device: CPU smt d smt c smt b smt a no smt b no smt a c b a 3 6 9 12 15 4.52 4.52 4.54 4.54 4.54 4.54 9.76 9.75 9.76 MIN: 4.03 / MAX: 45.52 MIN: 4.12 / MAX: 33.17 MIN: 4.11 / MAX: 42.14 MIN: 4.07 / MAX: 35.16 MIN: 4.09 / MAX: 55.78 MIN: 4.12 / MAX: 27.91 MIN: 4.98 / MAX: 36.12 MIN: 5.25 / MAX: 35.53 MIN: 5.03 / MAX: 28.1 1. (CXX) g++ options: -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -pie -ldl
Neural Magic DeepSparse Model: CV Detection, YOLOv5s COCO - Scenario: Asynchronous Multi-Stream OpenBenchmarking.org items/sec, More Is Better Neural Magic DeepSparse 1.3.2 Model: CV Detection, YOLOv5s COCO - Scenario: Asynchronous Multi-Stream smt d smt c smt b smt a no smt b no smt a c b a 200 400 600 800 1000 931.82 937.99 927.68 929.02 938.91 941.24 438.44 439.04 440.85
oneDNN Harness: Recurrent Neural Network Training - Data Type: u8s8f32 - Engine: CPU OpenBenchmarking.org ms, Fewer Is Better oneDNN 3.0 Harness: Recurrent Neural Network Training - Data Type: u8s8f32 - Engine: CPU smt d smt c smt b smt a no smt b no smt a c b a 400 800 1200 1600 2000 1883.76 1926.15 1937.65 1830.41 1119.17 1151.25 915.08 912.11 911.25 MIN: 1850.75 MIN: 1901.09 MIN: 1908.65 MIN: 1807.97 MIN: 1086.08 MIN: 1070.35 MIN: 906.26 MIN: 902.43 MIN: 903.01 1. (CXX) g++ options: -O3 -march=native -fopenmp -msse4.1 -fPIC -pie -ldl -lpthread
oneDNN Harness: Recurrent Neural Network Training - Data Type: bf16bf16bf16 - Engine: CPU OpenBenchmarking.org ms, Fewer Is Better oneDNN 3.0 Harness: Recurrent Neural Network Training - Data Type: bf16bf16bf16 - Engine: CPU smt d smt c smt b smt a no smt b no smt a c b a 400 800 1200 1600 2000 1912.74 1925.68 1894.07 1888.10 1139.63 1147.34 911.76 917.08 909.69 MIN: 1890.93 MIN: 1901.2 MIN: 1870.66 MIN: 1865.76 MIN: 1100.83 MIN: 1109.85 MIN: 901.88 MIN: 909.14 MIN: 901.1 1. (CXX) g++ options: -O3 -march=native -fopenmp -msse4.1 -fPIC -pie -ldl -lpthread
Neural Magic DeepSparse Model: CV Segmentation, 90% Pruned YOLACT Pruned - Scenario: Asynchronous Multi-Stream OpenBenchmarking.org items/sec, More Is Better Neural Magic DeepSparse 1.3.2 Model: CV Segmentation, 90% Pruned YOLACT Pruned - Scenario: Asynchronous Multi-Stream smt d smt c smt b smt a no smt b no smt a c b a 70 140 210 280 350 316.03 315.33 314.25 314.62 308.30 308.43 149.69 149.82 149.74
Neural Magic DeepSparse Model: NLP Text Classification, DistilBERT mnli - Scenario: Asynchronous Multi-Stream OpenBenchmarking.org items/sec, More Is Better Neural Magic DeepSparse 1.3.2 Model: NLP Text Classification, DistilBERT mnli - Scenario: Asynchronous Multi-Stream smt d smt c smt b smt a no smt b no smt a c b a 300 600 900 1200 1500 1301.18 1305.71 1306.58 1306.67 1292.80 1291.97 625.55 619.94 619.75
Neural Magic DeepSparse Model: NLP Text Classification, BERT base uncased SST2 - Scenario: Asynchronous Multi-Stream OpenBenchmarking.org items/sec, More Is Better Neural Magic DeepSparse 1.3.2 Model: NLP Text Classification, BERT base uncased SST2 - Scenario: Asynchronous Multi-Stream smt d smt c smt b smt a no smt b no smt a c b a 140 280 420 560 700 660.21 662.26 658.34 658.21 647.76 649.30 316.63 316.29 314.84
OpenVINO Model: Person Detection FP32 - Device: CPU OpenBenchmarking.org FPS, More Is Better OpenVINO 2022.3 Model: Person Detection FP32 - Device: CPU smt d smt c smt b smt a no smt b no smt a c b a 13 26 39 52 65 57.62 57.71 57.31 57.29 57.19 57.18 27.89 28.23 27.97 1. (CXX) g++ options: -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -pie -ldl
OpenVINO Model: Vehicle Detection FP16-INT8 - Device: CPU OpenBenchmarking.org ms, Fewer Is Better OpenVINO 2022.3 Model: Vehicle Detection FP16-INT8 - Device: CPU smt d smt c smt b smt a no smt b no smt a c b a 2 4 6 8 10 3.93 3.93 3.96 3.95 3.95 3.95 8.13 8.13 8.11 MIN: 3.61 / MAX: 23.62 MIN: 3.61 / MAX: 42.82 MIN: 3.66 / MAX: 32.83 MIN: 3.61 / MAX: 34.38 MIN: 3.68 / MAX: 42.53 MIN: 3.61 / MAX: 38 MIN: 3.83 / MAX: 59.87 MIN: 5.35 / MAX: 55.84 MIN: 5.39 / MAX: 69.87 1. (CXX) g++ options: -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -pie -ldl
OpenVINO Model: Vehicle Detection FP16-INT8 - Device: CPU OpenBenchmarking.org FPS, More Is Better OpenVINO 2022.3 Model: Vehicle Detection FP16-INT8 - Device: CPU smt d smt c smt b smt a no smt b no smt a c b a 3K 6K 9K 12K 15K 12172.92 12168.05 12109.64 12117.16 12128.38 12119.66 5898.79 5894.27 5908.89 1. (CXX) g++ options: -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -pie -ldl
OpenVINO Model: Person Detection FP32 - Device: CPU OpenBenchmarking.org ms, Fewer Is Better OpenVINO 2022.3 Model: Person Detection FP32 - Device: CPU smt d smt c smt b smt a no smt b no smt a c b a 400 800 1200 1600 2000 828.01 826.67 832.70 833.15 834.17 833.99 1704.02 1685.26 1701.70 MIN: 716.19 / MAX: 1018.34 MIN: 724.1 / MAX: 1006.75 MIN: 723.65 / MAX: 1031.69 MIN: 725.84 / MAX: 1017.38 MIN: 732.01 / MAX: 1006.46 MIN: 723.91 / MAX: 1011.94 MIN: 828.99 / MAX: 1969.02 MIN: 891.16 / MAX: 1979.37 MIN: 1395.71 / MAX: 2063.97 1. (CXX) g++ options: -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -pie -ldl
Neural Magic DeepSparse Model: NLP Question Answering, BERT base uncased SQuaD 12layer Pruned90 - Scenario: Asynchronous Multi-Stream OpenBenchmarking.org items/sec, More Is Better Neural Magic DeepSparse 1.3.2 Model: NLP Question Answering, BERT base uncased SQuaD 12layer Pruned90 - Scenario: Asynchronous Multi-Stream smt d smt c smt b smt a no smt b no smt a c b a 200 400 600 800 1000 816.76 818.60 822.16 817.75 817.63 813.98 400.04 401.21 401.78
Stress-NG Test: Malloc OpenBenchmarking.org Bogo Ops/s, More Is Better Stress-NG 0.15.04 Test: Malloc smt d smt c smt b smt a no smt b no smt a c b a 140M 280M 420M 560M 700M 640853365.54 639757070.52 634995429.66 634718750.41 456651508.04 456657338.84 313768771.26 314418461.02 312709034.05 1. (CC) gcc options: -std=gnu99 -O2 -lm -lc -lcrypt -ldl -lrt -pthread
OpenVINO Model: Person Detection FP16 - Device: CPU OpenBenchmarking.org FPS, More Is Better OpenVINO 2022.3 Model: Person Detection FP16 - Device: CPU smt d smt c smt b smt a no smt b no smt a c b a 13 26 39 52 65 58.00 58.05 57.67 57.72 57.49 57.58 28.33 28.42 28.47 1. (CXX) g++ options: -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -pie -ldl
OpenVINO Model: Person Detection FP16 - Device: CPU OpenBenchmarking.org ms, Fewer Is Better OpenVINO 2022.3 Model: Person Detection FP16 - Device: CPU smt d smt c smt b smt a no smt b no smt a c b a 400 800 1200 1600 2000 822.50 821.93 827.03 827.07 830.04 828.67 1679.03 1675.96 1671.46 MIN: 717.65 / MAX: 997.43 MIN: 725.25 / MAX: 1010.56 MIN: 723.77 / MAX: 1003.25 MIN: 724.84 / MAX: 1037.91 MIN: 722.6 / MAX: 1015.67 MIN: 730.29 / MAX: 1036.1 MIN: 865.58 / MAX: 1995.12 MIN: 1231.52 / MAX: 1967.75 MIN: 924.15 / MAX: 1977.46 1. (CXX) g++ options: -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -pie -ldl
OpenVINO Model: Age Gender Recognition Retail 0013 FP16-INT8 - Device: CPU OpenBenchmarking.org FPS, More Is Better OpenVINO 2022.3 Model: Age Gender Recognition Retail 0013 FP16-INT8 - Device: CPU smt d smt c smt b smt a no smt b no smt a c b a 30K 60K 90K 120K 150K 149647.68 148047.13 147582.87 148316.88 127770.39 127833.52 74486.08 74353.41 74495.17 1. (CXX) g++ options: -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -pie -ldl
RocksDB Test: Read While Writing OpenBenchmarking.org Op/s, More Is Better RocksDB 7.9.2 Test: Read While Writing smt d smt c smt b smt a no smt b no smt a c b a 3M 6M 9M 12M 15M 13098097 13948914 13830689 15317250 7913568 7643831 8316379 8620352 9296185 1. (CXX) g++ options: -O3 -march=native -pthread -fno-builtin-memcmp -fno-rtti
Neural Magic DeepSparse Model: NLP Sentiment Analysis, 80% Pruned Quantized BERT Base Uncased - Scenario: Asynchronous Multi-Stream OpenBenchmarking.org items/sec, More Is Better Neural Magic DeepSparse 1.3.2 Model: NLP Sentiment Analysis, 80% Pruned Quantized BERT Base Uncased - Scenario: Asynchronous Multi-Stream smt d smt c smt b smt a no smt b no smt a c b a 700 1400 2100 2800 3500 3116.09 3132.38 3114.43 3114.68 3086.36 3107.30 1571.83 1573.75 1566.56
oneDNN Harness: Convolution Batch Shapes Auto - Data Type: bf16bf16bf16 - Engine: CPU OpenBenchmarking.org ms, Fewer Is Better oneDNN 3.0 Harness: Convolution Batch Shapes Auto - Data Type: bf16bf16bf16 - Engine: CPU smt d smt c smt b smt a no smt b no smt a c b a 0.1302 0.2604 0.3906 0.5208 0.651 0.457554 0.447820 0.450261 0.451472 0.291229 0.291687 0.417188 0.578466 0.415446 MIN: 0.34 MIN: 0.37 MIN: 0.37 MIN: 0.4 MIN: 0.27 MIN: 0.27 MIN: 0.4 MIN: 0.4 MIN: 0.4 1. (CXX) g++ options: -O3 -march=native -fopenmp -msse4.1 -fPIC -pie -ldl -lpthread
OpenVINO Model: Machine Translation EN To DE FP16 - Device: CPU OpenBenchmarking.org FPS, More Is Better OpenVINO 2022.3 Model: Machine Translation EN To DE FP16 - Device: CPU smt d smt c smt b smt a no smt b no smt a c b a 200 400 600 800 1000 1057.74 1058.57 1050.37 1049.16 1048.74 1045.81 535.78 541.03 540.41 1. (CXX) g++ options: -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -pie -ldl
OpenVINO Model: Machine Translation EN To DE FP16 - Device: CPU OpenBenchmarking.org ms, Fewer Is Better OpenVINO 2022.3 Model: Machine Translation EN To DE FP16 - Device: CPU smt d smt c smt b smt a no smt b no smt a c b a 20 40 60 80 100 45.34 45.31 45.66 45.71 45.73 45.86 89.46 88.61 88.71 MIN: 39.4 / MAX: 73.83 MIN: 39.35 / MAX: 73.19 MIN: 38.82 / MAX: 75.65 MIN: 39.78 / MAX: 73.33 MIN: 39.68 / MAX: 86.83 MIN: 39.29 / MAX: 91.13 MIN: 42.16 / MAX: 123.4 MIN: 47.57 / MAX: 124.67 MIN: 44.2 / MAX: 132.86 1. (CXX) g++ options: -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -pie -ldl
oneDNN Harness: Matrix Multiply Batch Shapes Transformer - Data Type: f32 - Engine: CPU OpenBenchmarking.org ms, Fewer Is Better oneDNN 3.0 Harness: Matrix Multiply Batch Shapes Transformer - Data Type: f32 - Engine: CPU smt d smt c smt b smt a no smt b no smt a c b a 0.1521 0.3042 0.4563 0.6084 0.7605 0.672115 0.676124 0.675664 0.674455 0.347231 0.342836 0.353175 0.356875 0.357017 MIN: 0.49 MIN: 0.49 MIN: 0.55 MIN: 0.49 MIN: 0.3 MIN: 0.28 MIN: 0.31 MIN: 0.31 MIN: 0.31 1. (CXX) g++ options: -O3 -march=native -fopenmp -msse4.1 -fPIC -pie -ldl -lpthread
oneDNN Harness: IP Shapes 3D - Data Type: u8s8f32 - Engine: CPU OpenBenchmarking.org ms, Fewer Is Better oneDNN 3.0 Harness: IP Shapes 3D - Data Type: u8s8f32 - Engine: CPU smt d smt c smt b smt a no smt b no smt a c b a 0.224 0.448 0.672 0.896 1.12 0.975091 0.995710 0.971704 0.975357 0.629026 0.556069 0.546991 0.513068 0.534450 MIN: 0.91 MIN: 0.87 MIN: 0.82 MIN: 0.92 MIN: 0.51 MIN: 0.46 MIN: 0.5 MIN: 0.47 MIN: 0.49 1. (CXX) g++ options: -O3 -march=native -fopenmp -msse4.1 -fPIC -pie -ldl -lpthread
oneDNN Harness: Deconvolution Batch shapes_3d - Data Type: u8s8f32 - Engine: CPU OpenBenchmarking.org ms, Fewer Is Better oneDNN 3.0 Harness: Deconvolution Batch shapes_3d - Data Type: u8s8f32 - Engine: CPU smt d smt c smt b smt a no smt b no smt a c b a 0.0688 0.1376 0.2064 0.2752 0.344 0.247825 0.275917 0.291087 0.279638 0.164845 0.160361 0.305599 0.305731 0.304972 MIN: 0.23 MIN: 0.23 MIN: 0.23 MIN: 0.23 MIN: 0.15 MIN: 0.15 MIN: 0.28 MIN: 0.28 MIN: 0.28 1. (CXX) g++ options: -O3 -march=native -fopenmp -msse4.1 -fPIC -pie -ldl -lpthread
Stress-NG Test: MEMFD OpenBenchmarking.org Bogo Ops/s, More Is Better Stress-NG 0.15.04 Test: MEMFD smt d smt c smt b smt a no smt b no smt a c b a 120 240 360 480 600 394.49 413.34 564.05 464.70 308.67 303.56 507.97 507.74 518.46 1. (CC) gcc options: -std=gnu99 -O2 -lm -lc -lcrypt -ldl -lrt -pthread
Stress-NG Test: Memory Copying OpenBenchmarking.org Bogo Ops/s, More Is Better Stress-NG 0.15.04 Test: Memory Copying smt d smt c smt b smt a no smt b no smt a c b a 4K 8K 12K 16K 20K 13370.33 15077.69 15430.18 15914.10 10949.90 11342.69 20297.91 20340.40 20106.51 1. (CC) gcc options: -std=gnu99 -O2 -lm -lc -lcrypt -ldl -lrt -pthread
Stress-NG Test: Forking OpenBenchmarking.org Bogo Ops/s, More Is Better Stress-NG 0.15.04 Test: Forking smt d smt c smt b smt a no smt b no smt a c b a 14K 28K 42K 56K 70K 34685.85 34917.39 36020.16 36266.02 45685.28 43094.96 64299.62 58664.97 58156.36 1. (CC) gcc options: -std=gnu99 -O2 -lm -lc -lcrypt -ldl -lrt -pthread
Stress-NG Test: Futex OpenBenchmarking.org Bogo Ops/s, More Is Better Stress-NG 0.15.04 Test: Futex smt d smt c smt b smt a no smt b no smt a c b a 800K 1600K 2400K 3200K 4000K 2077119.76 2067499.58 2396325.93 2333781.39 3802292.60 3746361.52 2794473.75 2805836.52 2794694.37 1. (CC) gcc options: -std=gnu99 -O2 -lm -lc -lcrypt -ldl -lrt -pthread
GROMACS Implementation: MPI CPU - Input: water_GMX50_bare OpenBenchmarking.org Ns Per Day, More Is Better GROMACS 2023 Implementation: MPI CPU - Input: water_GMX50_bare smt d smt c smt b smt a no smt b no smt a c b a 5 10 15 20 25 19.08 19.05 19.02 18.82 18.84 19.18 10.59 10.61 10.57 1. (CXX) g++ options: -O3
RocksDB Test: Read Random Write Random OpenBenchmarking.org Op/s, More Is Better RocksDB 7.9.2 Test: Read Random Write Random smt d smt c smt b smt a no smt b no smt a c b a 600K 1200K 1800K 2400K 3000K 1752904 1613913 1761416 1787682 2063831 2079804 2910023 2891962 2926458 1. (CXX) g++ options: -O3 -march=native -pthread -fno-builtin-memcmp -fno-rtti
oneDNN Harness: Deconvolution Batch shapes_3d - Data Type: f32 - Engine: CPU OpenBenchmarking.org ms, Fewer Is Better oneDNN 3.0 Harness: Deconvolution Batch shapes_3d - Data Type: f32 - Engine: CPU smt d smt c smt b smt a no smt b no smt a c b a 0.2633 0.5266 0.7899 1.0532 1.3165 0.973984 0.972863 0.973555 0.978590 0.651301 0.664295 1.169190 1.166960 1.170040 MIN: 0.92 MIN: 0.92 MIN: 0.92 MIN: 0.93 MIN: 0.62 MIN: 0.63 MIN: 1.07 MIN: 1.07 MIN: 1.07 1. (CXX) g++ options: -O3 -march=native -fopenmp -msse4.1 -fPIC -pie -ldl -lpthread
oneDNN Harness: Convolution Batch Shapes Auto - Data Type: f32 - Engine: CPU OpenBenchmarking.org ms, Fewer Is Better oneDNN 3.0 Harness: Convolution Batch Shapes Auto - Data Type: f32 - Engine: CPU smt d smt c smt b smt a no smt b no smt a c b a 0.156 0.312 0.468 0.624 0.78 0.630630 0.545118 0.551783 0.538539 0.400325 0.403590 0.692667 0.689064 0.693367 MIN: 0.48 MIN: 0.45 MIN: 0.49 MIN: 0.48 MIN: 0.38 MIN: 0.38 MIN: 0.65 MIN: 0.64 MIN: 0.64 1. (CXX) g++ options: -O3 -march=native -fopenmp -msse4.1 -fPIC -pie -ldl -lpthread
OpenVINO Model: Person Vehicle Bike Detection FP16 - Device: CPU OpenBenchmarking.org FPS, More Is Better OpenVINO 2022.3 Model: Person Vehicle Bike Detection FP16 - Device: CPU smt d smt c smt b smt a no smt b no smt a c b a 2K 4K 6K 8K 10K 9767.99 9773.26 9680.44 9746.08 9758.74 9784.09 5724.81 5720.90 5750.74 1. (CXX) g++ options: -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -pie -ldl
OpenVINO Model: Person Vehicle Bike Detection FP16 - Device: CPU OpenBenchmarking.org ms, Fewer Is Better OpenVINO 2022.3 Model: Person Vehicle Bike Detection FP16 - Device: CPU smt d smt c smt b smt a no smt b no smt a c b a 2 4 6 8 10 4.91 4.90 4.95 4.92 4.91 4.90 8.37 8.38 8.34 MIN: 4.51 / MAX: 27.27 MIN: 4.5 / MAX: 29.95 MIN: 4.54 / MAX: 24.23 MIN: 4.52 / MAX: 34.8 MIN: 4.49 / MAX: 34.72 MIN: 4.46 / MAX: 56.52 MIN: 6.83 / MAX: 50.24 MIN: 6.4 / MAX: 32.33 MIN: 6.61 / MAX: 55.54 1. (CXX) g++ options: -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -pie -ldl
Memcached Set To Get Ratio: 1:5 OpenBenchmarking.org Ops/sec, More Is Better Memcached 1.6.18 Set To Get Ratio: 1:5 smt d smt c smt b smt a no smt b no smt a c b a 900K 1800K 2700K 3600K 4500K 2527084.87 2517750.47 2507331.29 2520253.00 2460416.33 2444886.82 4155273.46 4162812.47 4143044.21 1. (CXX) g++ options: -O2 -levent_openssl -levent -lcrypto -lssl -lpthread -lz -lpcre
Stress-NG Test: System V Message Passing OpenBenchmarking.org Bogo Ops/s, More Is Better Stress-NG 0.15.04 Test: System V Message Passing smt d smt c smt b smt a no smt b no smt a c b a 3M 6M 9M 12M 15M 12418451.71 8609357.68 8586858.51 10103952.13 7372780.72 7402514.74 10475486.39 10471889.97 10473084.09 1. (CC) gcc options: -std=gnu99 -O2 -lm -lc -lcrypt -ldl -lrt -pthread
oneDNN Harness: Deconvolution Batch shapes_1d - Data Type: bf16bf16bf16 - Engine: CPU OpenBenchmarking.org ms, Fewer Is Better oneDNN 3.0 Harness: Deconvolution Batch shapes_1d - Data Type: bf16bf16bf16 - Engine: CPU smt d smt c smt b smt a no smt b no smt a c b a 0.6213 1.2426 1.8639 2.4852 3.1065 2.67303 2.70235 2.76140 2.70807 1.70057 1.65041 1.82264 1.84172 1.83147 MIN: 2.09 MIN: 2.27 MIN: 2.46 MIN: 2.09 MIN: 1.5 MIN: 1.41 MIN: 1.71 MIN: 1.76 MIN: 1.73 1. (CXX) g++ options: -O3 -march=native -fopenmp -msse4.1 -fPIC -pie -ldl -lpthread
Kvazaar Video Input: Bosphorus 4K - Video Preset: Slow OpenBenchmarking.org Frames Per Second, More Is Better Kvazaar 2.2 Video Input: Bosphorus 4K - Video Preset: Slow smt d smt c smt b smt a no smt b no smt a c b a 15 30 45 60 75 66.08 65.49 66.22 64.63 47.43 47.13 40.76 40.86 40.63 1. (CC) gcc options: -pthread -ftree-vectorize -fvisibility=hidden -O2 -lpthread -lm -lrt
Memcached Set To Get Ratio: 1:10 OpenBenchmarking.org Ops/sec, More Is Better Memcached 1.6.18 Set To Get Ratio: 1:10 smt d smt c smt b smt a no smt b no smt a c b a 1.5M 3M 4.5M 6M 7.5M 4569476.86 4549313.42 4392934.75 4530948.62 4220522.49 4244908.83 6839975.87 6792746.34 6861088.89 1. (CXX) g++ options: -O2 -levent_openssl -levent -lcrypto -lssl -lpthread -lz -lpcre
oneDNN Harness: Convolution Batch Shapes Auto - Data Type: u8s8f32 - Engine: CPU OpenBenchmarking.org ms, Fewer Is Better oneDNN 3.0 Harness: Convolution Batch Shapes Auto - Data Type: u8s8f32 - Engine: CPU smt d smt c smt b smt a no smt b no smt a c b a 0.0918 0.1836 0.2754 0.3672 0.459 0.389223 0.394377 0.408074 0.407430 0.255796 0.254845 0.312289 0.312721 0.311405 MIN: 0.29 MIN: 0.29 MIN: 0.27 MIN: 0.27 MIN: 0.18 MIN: 0.18 MIN: 0.28 MIN: 0.28 MIN: 0.3 1. (CXX) g++ options: -O3 -march=native -fopenmp -msse4.1 -fPIC -pie -ldl -lpthread
Kvazaar Video Input: Bosphorus 4K - Video Preset: Medium OpenBenchmarking.org Frames Per Second, More Is Better Kvazaar 2.2 Video Input: Bosphorus 4K - Video Preset: Medium smt d smt c smt b smt a no smt b no smt a c b a 15 30 45 60 75 65.49 65.59 65.69 65.56 47.93 47.97 41.47 41.39 41.40 1. (CC) gcc options: -pthread -ftree-vectorize -fvisibility=hidden -O2 -lpthread -lm -lrt
uvg266 Video Input: Bosphorus 4K - Video Preset: Slow OpenBenchmarking.org Frames Per Second, More Is Better uvg266 0.4.1 Video Input: Bosphorus 4K - Video Preset: Slow smt d smt c smt b smt a no smt b no smt a c b a 11 22 33 44 55 46.25 46.34 46.25 45.56 34.68 34.59 29.41 29.29 29.37
OpenVINO Model: Age Gender Recognition Retail 0013 FP16 - Device: CPU OpenBenchmarking.org FPS, More Is Better OpenVINO 2022.3 Model: Age Gender Recognition Retail 0013 FP16 - Device: CPU smt d smt c smt b smt a no smt b no smt a c b a 40K 80K 120K 160K 200K 172228.34 173620.31 175754.35 173926.92 162294.47 160545.41 112186.25 111378.39 112346.36 1. (CXX) g++ options: -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -pie -ldl
oneDNN Harness: Deconvolution Batch shapes_3d - Data Type: bf16bf16bf16 - Engine: CPU OpenBenchmarking.org ms, Fewer Is Better oneDNN 3.0 Harness: Deconvolution Batch shapes_3d - Data Type: bf16bf16bf16 - Engine: CPU smt d smt c smt b smt a no smt b no smt a c b a 0.1603 0.3206 0.4809 0.6412 0.8015 0.680395 0.676798 0.672768 0.672585 0.461174 0.462386 0.708563 0.712495 0.711588 MIN: 0.53 MIN: 0.53 MIN: 0.53 MIN: 0.52 MIN: 0.43 MIN: 0.42 MIN: 0.67 MIN: 0.68 MIN: 0.68 1. (CXX) g++ options: -O3 -march=native -fopenmp -msse4.1 -fPIC -pie -ldl -lpthread
Memcached Set To Get Ratio: 1:100 OpenBenchmarking.org Ops/sec, More Is Better Memcached 1.6.18 Set To Get Ratio: 1:100 smt d smt c smt b smt a no smt b no smt a c b a 1000K 2000K 3000K 4000K 5000K 4403267.42 4394194.49 4357453.76 4383314.65 3216133.91 3192061.68 4852421.67 4876951.36 4878860.00 1. (CXX) g++ options: -O2 -levent_openssl -levent -lcrypto -lssl -lpthread -lz -lpcre
Stress-NG Test: Semaphores OpenBenchmarking.org Bogo Ops/s, More Is Better Stress-NG 0.15.04 Test: Semaphores smt d smt c smt b smt a no smt b no smt a c b a 4M 8M 12M 16M 20M 19842969.28 19866440.86 19927313.52 20047519.05 13192391.85 13141129.68 18088584.67 18100474.36 18128283.29 1. (CC) gcc options: -std=gnu99 -O2 -lm -lc -lcrypt -ldl -lrt -pthread
Timed Linux Kernel Compilation Build: defconfig OpenBenchmarking.org Seconds, Fewer Is Better Timed Linux Kernel Compilation 6.1 Build: defconfig smt d smt c smt b smt a no smt b no smt a c b a 6 12 18 24 30 17.08 17.05 17.02 17.38 17.59 17.61 25.68 25.75 25.77
Stress-NG Test: Poll OpenBenchmarking.org Bogo Ops/s, More Is Better Stress-NG 0.15.04 Test: Poll smt d smt c smt b smt a no smt b no smt a c b a 3M 6M 9M 12M 15M 15403597.24 15228320.07 15341564.16 15359471.73 10393402.01 10458275.24 12676101.41 12661687.46 12653709.64 1. (CC) gcc options: -std=gnu99 -O2 -lm -lc -lcrypt -ldl -lrt -pthread
OpenVKL Benchmark: vklBenchmark Scalar OpenBenchmarking.org Items / Sec, More Is Better OpenVKL 1.3.1 Benchmark: vklBenchmark Scalar smt d smt c smt b smt a no smt b no smt a c b a 200 400 600 800 1000 781 793 810 764 652 647 549 557 556 MIN: 139 / MAX: 3776 MIN: 139 / MAX: 3583 MIN: 138 / MAX: 3650 MIN: 139 / MAX: 3808 MIN: 102 / MAX: 3990 MIN: 101 / MAX: 4024 MIN: 60 / MAX: 5475 MIN: 61 / MAX: 6019 MIN: 61 / MAX: 5994
Neural Magic DeepSparse Model: NLP Text Classification, DistilBERT mnli - Scenario: Synchronous Single-Stream OpenBenchmarking.org ms/batch, Fewer Is Better Neural Magic DeepSparse 1.3.2 Model: NLP Text Classification, DistilBERT mnli - Scenario: Synchronous Single-Stream smt d smt c smt b smt a no smt b no smt a c b a 2 4 6 8 10 7.0640 7.3550 6.8844 7.5225 6.4174 7.5856 5.1696 5.1633 5.1812
Neural Magic DeepSparse Model: NLP Text Classification, DistilBERT mnli - Scenario: Synchronous Single-Stream OpenBenchmarking.org items/sec, More Is Better Neural Magic DeepSparse 1.3.2 Model: NLP Text Classification, DistilBERT mnli - Scenario: Synchronous Single-Stream smt d smt c smt b smt a no smt b no smt a c b a 40 80 120 160 200 141.47 135.89 145.17 132.87 155.74 131.76 193.31 193.53 192.88
uvg266 Video Input: Bosphorus 4K - Video Preset: Medium OpenBenchmarking.org Frames Per Second, More Is Better uvg266 0.4.1 Video Input: Bosphorus 4K - Video Preset: Medium smt d smt c smt b smt a no smt b no smt a c b a 11 22 33 44 55 46.21 46.57 46.62 47.56 38.65 38.38 33.10 33.13 33.03
Neural Magic DeepSparse Model: NLP Question Answering, BERT base uncased SQuaD 12layer Pruned90 - Scenario: Synchronous Single-Stream OpenBenchmarking.org items/sec, More Is Better Neural Magic DeepSparse 1.3.2 Model: NLP Question Answering, BERT base uncased SQuaD 12layer Pruned90 - Scenario: Synchronous Single-Stream smt d smt c smt b smt a no smt b no smt a c b a 20 40 60 80 100 75.52 70.14 77.41 77.88 71.89 72.13 98.59 98.62 99.15
Neural Magic DeepSparse Model: NLP Question Answering, BERT base uncased SQuaD 12layer Pruned90 - Scenario: Synchronous Single-Stream OpenBenchmarking.org ms/batch, Fewer Is Better Neural Magic DeepSparse 1.3.2 Model: NLP Question Answering, BERT base uncased SQuaD 12layer Pruned90 - Scenario: Synchronous Single-Stream smt d smt c smt b smt a no smt b no smt a c b a 4 8 12 16 20 13.23 14.25 12.91 12.83 13.90 13.86 10.14 10.14 10.08
Neural Magic DeepSparse Model: CV Classification, ResNet-50 ImageNet - Scenario: Synchronous Single-Stream OpenBenchmarking.org ms/batch, Fewer Is Better Neural Magic DeepSparse 1.3.2 Model: CV Classification, ResNet-50 ImageNet - Scenario: Synchronous Single-Stream smt d smt c smt b smt a no smt b no smt a c b a 2 4 6 8 10 7.1490 6.1057 7.3054 7.1865 6.0650 6.0118 5.5007 5.4980 5.3018
Neural Magic DeepSparse Model: CV Classification, ResNet-50 ImageNet - Scenario: Synchronous Single-Stream OpenBenchmarking.org items/sec, More Is Better Neural Magic DeepSparse 1.3.2 Model: CV Classification, ResNet-50 ImageNet - Scenario: Synchronous Single-Stream smt d smt c smt b smt a no smt b no smt a c b a 40 80 120 160 200 139.82 163.71 136.83 139.10 164.81 166.27 181.72 181.81 188.54
Neural Magic DeepSparse Model: NLP Text Classification, BERT base uncased SST2 - Scenario: Synchronous Single-Stream OpenBenchmarking.org ms/batch, Fewer Is Better Neural Magic DeepSparse 1.3.2 Model: NLP Text Classification, BERT base uncased SST2 - Scenario: Synchronous Single-Stream smt d smt c smt b smt a no smt b no smt a c b a 4 8 12 16 20 15.13 14.20 14.64 13.64 14.42 14.46 11.16 11.19 11.36
Neural Magic DeepSparse Model: NLP Text Classification, BERT base uncased SST2 - Scenario: Synchronous Single-Stream OpenBenchmarking.org items/sec, More Is Better Neural Magic DeepSparse 1.3.2 Model: NLP Text Classification, BERT base uncased SST2 - Scenario: Synchronous Single-Stream smt d smt c smt b smt a no smt b no smt a c b a 20 40 60 80 100 66.08 70.38 68.29 73.29 69.33 69.11 89.55 89.34 87.96
uvg266 Video Input: Bosphorus 1080p - Video Preset: Super Fast OpenBenchmarking.org Frames Per Second, More Is Better uvg266 0.4.1 Video Input: Bosphorus 1080p - Video Preset: Super Fast smt d smt c smt b smt a no smt b no smt a c b a 50 100 150 200 250 218.91 178.78 220.79 178.49 216.15 181.67 237.96 239.92 238.68
uvg266 Video Input: Bosphorus 1080p - Video Preset: Ultra Fast OpenBenchmarking.org Frames Per Second, More Is Better uvg266 0.4.1 Video Input: Bosphorus 1080p - Video Preset: Ultra Fast smt d smt c smt b smt a no smt b no smt a c b a 50 100 150 200 250 224.31 218.28 179.56 216.15 209.85 186.24 238.77 240.91 240.98
Zstd Compression Compression Level: 12 - Compression Speed OpenBenchmarking.org MB/s, More Is Better Zstd Compression 1.5.4 Compression Level: 12 - Compression Speed smt d smt c smt b smt a no smt b no smt a c b a 70 140 210 280 350 256.2 249.8 259.1 254.1 278.2 279.9 330.1 332.0 330.8 1. (CC) gcc options: -O3 -pthread -lz
RocksDB Test: Update Random OpenBenchmarking.org Op/s, More Is Better RocksDB 7.9.2 Test: Update Random smt d smt c smt b smt a no smt b no smt a c b a 120K 240K 360K 480K 600K 420514 419985 411513 413199 452228 462018 543572 545556 544384 1. (CXX) g++ options: -O3 -march=native -pthread -fno-builtin-memcmp -fno-rtti
RocksDB Test: Sequential Fill OpenBenchmarking.org Op/s, More Is Better RocksDB 7.9.2 Test: Sequential Fill smt d smt c smt b smt a no smt b no smt a c b a 120K 240K 360K 480K 600K 414047 414282 413708 414168 464044 465700 544565 545396 542256 1. (CXX) g++ options: -O3 -march=native -pthread -fno-builtin-memcmp -fno-rtti
Neural Magic DeepSparse Model: CV Segmentation, 90% Pruned YOLACT Pruned - Scenario: Synchronous Single-Stream OpenBenchmarking.org ms/batch, Fewer Is Better Neural Magic DeepSparse 1.3.2 Model: CV Segmentation, 90% Pruned YOLACT Pruned - Scenario: Synchronous Single-Stream smt d smt c smt b smt a no smt b no smt a c b a 5 10 15 20 25 21.36 20.49 21.74 21.03 20.42 20.74 16.52 16.67 16.58
Neural Magic DeepSparse Model: CV Segmentation, 90% Pruned YOLACT Pruned - Scenario: Synchronous Single-Stream OpenBenchmarking.org items/sec, More Is Better Neural Magic DeepSparse 1.3.2 Model: CV Segmentation, 90% Pruned YOLACT Pruned - Scenario: Synchronous Single-Stream smt d smt c smt b smt a no smt b no smt a c b a 14 28 42 56 70 46.78 48.77 45.96 47.51 48.93 48.18 60.49 59.92 60.25
uvg266 Video Input: Bosphorus 1080p - Video Preset: Very Fast OpenBenchmarking.org Frames Per Second, More Is Better uvg266 0.4.1 Video Input: Bosphorus 1080p - Video Preset: Very Fast smt d smt c smt b smt a no smt b no smt a c b a 50 100 150 200 250 181.83 192.25 209.55 183.99 183.75 196.25 237.44 234.68 234.95
ClickHouse 100M Rows Hits Dataset, Second Run OpenBenchmarking.org Queries Per Minute, Geo Mean, More Is Better ClickHouse 22.12.3.5 100M Rows Hits Dataset, Second Run smt d smt c smt b smt a no smt b no smt a c b a 140 280 420 560 700 515.11 524.75 527.88 524.04 649.52 666.43 621.88 627.20 625.91 MIN: 88.5 / MAX: 6000 MIN: 90.23 / MAX: 6000 MIN: 90.63 / MAX: 6666.67 MIN: 90.5 / MAX: 6666.67 MIN: 86.46 / MAX: 6000 MIN: 85.11 / MAX: 6666.67 MIN: 58.54 / MAX: 6666.67 MIN: 58.54 / MAX: 6000 MIN: 56.13 / MAX: 7500
RocksDB Test: Random Fill OpenBenchmarking.org Op/s, More Is Better RocksDB 7.9.2 Test: Random Fill smt d smt c smt b smt a no smt b no smt a c b a 110K 220K 330K 440K 550K 415428 416006 417882 423027 468210 478629 536551 534681 533927 1. (CXX) g++ options: -O3 -march=native -pthread -fno-builtin-memcmp -fno-rtti
ClickHouse 100M Rows Hits Dataset, First Run / Cold Cache OpenBenchmarking.org Queries Per Minute, Geo Mean, More Is Better ClickHouse 22.12.3.5 100M Rows Hits Dataset, First Run / Cold Cache smt d smt c smt b smt a no smt b no smt a c b a 140 280 420 560 700 500.12 495.85 516.09 500.13 622.73 635.48 614.24 610.79 600.70 MIN: 69.61 / MAX: 6000 MIN: 51.06 / MAX: 6000 MIN: 89.55 / MAX: 6000 MIN: 65.15 / MAX: 6000 MIN: 83.22 / MAX: 5454.55 MIN: 85.35 / MAX: 6000 MIN: 57.14 / MAX: 6666.67 MIN: 58.14 / MAX: 6666.67 MIN: 57.75 / MAX: 6000
Neural Magic DeepSparse Model: CV Detection, YOLOv5s COCO - Scenario: Synchronous Single-Stream OpenBenchmarking.org ms/batch, Fewer Is Better Neural Magic DeepSparse 1.3.2 Model: CV Detection, YOLOv5s COCO - Scenario: Synchronous Single-Stream smt d smt c smt b smt a no smt b no smt a c b a 3 6 9 12 15 9.3656 10.3852 8.3506 8.1914 9.9395 9.8899 8.8004 9.5793 9.5962
Neural Magic DeepSparse Model: CV Detection, YOLOv5s COCO - Scenario: Synchronous Single-Stream OpenBenchmarking.org items/sec, More Is Better Neural Magic DeepSparse 1.3.2 Model: CV Detection, YOLOv5s COCO - Scenario: Synchronous Single-Stream smt d smt c smt b smt a no smt b no smt a c b a 30 60 90 120 150 106.65 96.17 119.60 121.92 100.51 101.01 113.51 104.29 104.12
ClickHouse 100M Rows Hits Dataset, Third Run OpenBenchmarking.org Queries Per Minute, Geo Mean, More Is Better ClickHouse 22.12.3.5 100M Rows Hits Dataset, Third Run smt d smt c smt b smt a no smt b no smt a c b a 140 280 420 560 700 527.80 530.47 538.63 534.82 662.01 665.00 636.18 628.37 625.16 MIN: 81.52 / MAX: 6000 MIN: 92.02 / MAX: 5454.55 MIN: 89.82 / MAX: 6000 MIN: 91.32 / MAX: 6000 MIN: 86.33 / MAX: 6000 MIN: 85.84 / MAX: 6000 MIN: 58.03 / MAX: 6666.67 MIN: 58.37 / MAX: 6666.67 MIN: 57.69 / MAX: 6000
VP9 libvpx Encoding Speed: Speed 5 - Input: Bosphorus 4K OpenBenchmarking.org Frames Per Second, More Is Better VP9 libvpx Encoding 1.13 Speed: Speed 5 - Input: Bosphorus 4K smt d smt c smt b smt a no smt b no smt a c b a 4 8 12 16 20 15.11 15.38 14.27 13.90 14.50 14.48 17.46 17.37 17.36 1. (CXX) g++ options: -m64 -lm -lpthread -O3 -fPIC -U_FORTIFY_SOURCE -std=gnu++11
OpenVINO Model: Age Gender Recognition Retail 0013 FP16 - Device: CPU OpenBenchmarking.org ms, Fewer Is Better OpenVINO 2022.3 Model: Age Gender Recognition Retail 0013 FP16 - Device: CPU smt d smt c smt b smt a no smt b no smt a c b a 0.1485 0.297 0.4455 0.594 0.7425 0.53 0.53 0.53 0.53 0.58 0.57 0.66 0.66 0.65 MIN: 0.5 / MAX: 8.86 MIN: 0.5 / MAX: 9.15 MIN: 0.5 / MAX: 9.61 MIN: 0.5 / MAX: 24.2 MIN: 0.5 / MAX: 12.8 MIN: 0.5 / MAX: 13.02 MIN: 0.31 / MAX: 22.67 MIN: 0.32 / MAX: 19.99 MIN: 0.32 / MAX: 20.68 1. (CXX) g++ options: -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -pie -ldl
uvg266 Video Input: Bosphorus 4K - Video Preset: Ultra Fast OpenBenchmarking.org Frames Per Second, More Is Better uvg266 0.4.1 Video Input: Bosphorus 4K - Video Preset: Ultra Fast smt d smt c smt b smt a no smt b no smt a c b a 16 32 48 64 80 58.76 57.33 57.51 57.96 57.23 57.78 71.13 70.68 70.56
OpenVKL Benchmark: vklBenchmark ISPC OpenBenchmarking.org Items / Sec, More Is Better OpenVKL 1.3.1 Benchmark: vklBenchmark ISPC smt d smt c smt b smt a no smt b no smt a c b a 300 600 900 1200 1500 1238 1235 1243 1318 1108 1098 1066 1075 1089 MIN: 392 / MAX: 3738 MIN: 392 / MAX: 4332 MIN: 390 / MAX: 3501 MIN: 391 / MAX: 3855 MIN: 297 / MAX: 4208 MIN: 297 / MAX: 4284 MIN: 180 / MAX: 6427 MIN: 179 / MAX: 6442 MIN: 179 / MAX: 7194
Zstd Compression Compression Level: 8 - Compression Speed OpenBenchmarking.org MB/s, More Is Better Zstd Compression 1.5.4 Compression Level: 8 - Compression Speed smt d smt c smt b smt a no smt b no smt a c b a 300 600 900 1200 1500 1024.8 1005.8 1122.4 1023.9 1234.2 1227.5 1241.1 1239.3 1233.8 1. (CC) gcc options: -O3 -pthread -lz
Zstd Compression Compression Level: 3 - Compression Speed OpenBenchmarking.org MB/s, More Is Better Zstd Compression 1.5.4 Compression Level: 3 - Compression Speed smt d smt c smt b smt a no smt b no smt a c b a 700 1400 2100 2800 3500 2528.7 2604.6 2513.4 2795.1 2865.9 2804.2 3049.4 3095.0 3033.9 1. (CC) gcc options: -O3 -pthread -lz
Timed FFmpeg Compilation Time To Compile OpenBenchmarking.org Seconds, Fewer Is Better Timed FFmpeg Compilation 6.0 Time To Compile smt d smt c smt b smt a no smt b no smt a c b a 3 6 9 12 15 10.37 10.41 10.26 10.38 10.66 10.60 12.47 12.43 12.57
Kvazaar Video Input: Bosphorus 1080p - Video Preset: Very Fast OpenBenchmarking.org Frames Per Second, More Is Better Kvazaar 2.2 Video Input: Bosphorus 1080p - Video Preset: Very Fast smt d smt c smt b smt a no smt b no smt a c b a 60 120 180 240 300 270.76 243.36 274.63 250.71 268.77 269.62 291.03 290.59 296.93 1. (CC) gcc options: -pthread -ftree-vectorize -fvisibility=hidden -O2 -lpthread -lm -lrt
uvg266 Video Input: Bosphorus 4K - Video Preset: Very Fast OpenBenchmarking.org Frames Per Second, More Is Better uvg266 0.4.1 Video Input: Bosphorus 4K - Video Preset: Very Fast smt d smt c smt b smt a no smt b no smt a c b a 15 30 45 60 75 57.52 57.44 57.86 59.30 58.09 57.13 69.04 68.79 68.82
Kvazaar Video Input: Bosphorus 1080p - Video Preset: Slow OpenBenchmarking.org Frames Per Second, More Is Better Kvazaar 2.2 Video Input: Bosphorus 1080p - Video Preset: Slow smt d smt c smt b smt a no smt b no smt a c b a 40 80 120 160 200 135.56 135.80 136.33 132.67 159.79 155.56 140.10 139.07 139.92 1. (CC) gcc options: -pthread -ftree-vectorize -fvisibility=hidden -O2 -lpthread -lm -lrt
Kvazaar Video Input: Bosphorus 1080p - Video Preset: Super Fast OpenBenchmarking.org Frames Per Second, More Is Better Kvazaar 2.2 Video Input: Bosphorus 1080p - Video Preset: Super Fast smt d smt c smt b smt a no smt b no smt a c b a 70 140 210 280 350 256.45 288.09 267.04 296.72 280.18 288.41 301.40 303.99 307.49 1. (CC) gcc options: -pthread -ftree-vectorize -fvisibility=hidden -O2 -lpthread -lm -lrt
uvg266 Video Input: Bosphorus 4K - Video Preset: Super Fast OpenBenchmarking.org Frames Per Second, More Is Better uvg266 0.4.1 Video Input: Bosphorus 4K - Video Preset: Super Fast smt d smt c smt b smt a no smt b no smt a c b a 16 32 48 64 80 58.66 58.34 59.20 59.06 58.33 59.83 69.92 69.00 69.33
Neural Magic DeepSparse Model: NLP Token Classification, BERT base uncased conll2003 - Scenario: Synchronous Single-Stream OpenBenchmarking.org ms/batch, Fewer Is Better Neural Magic DeepSparse 1.3.2 Model: NLP Token Classification, BERT base uncased conll2003 - Scenario: Synchronous Single-Stream smt d smt c smt b smt a no smt b no smt a c b a 8 16 24 32 40 32.18 33.99 32.24 32.30 32.18 31.92 28.57 28.71 28.62
Neural Magic DeepSparse Model: NLP Token Classification, BERT base uncased conll2003 - Scenario: Synchronous Single-Stream OpenBenchmarking.org items/sec, More Is Better Neural Magic DeepSparse 1.3.2 Model: NLP Token Classification, BERT base uncased conll2003 - Scenario: Synchronous Single-Stream smt d smt c smt b smt a no smt b no smt a c b a 8 16 24 32 40 31.07 29.42 31.01 30.96 31.07 31.32 34.99 34.82 34.93
Zstd Compression Compression Level: 3, Long Mode - Compression Speed OpenBenchmarking.org MB/s, More Is Better Zstd Compression 1.5.4 Compression Level: 3, Long Mode - Compression Speed smt d smt c smt b smt a no smt b no smt a c b a 200 400 600 800 1000 1059.0 1046.6 1038.9 1051.4 1032.3 955.6 916.9 909.4 892.7 1. (CC) gcc options: -O3 -pthread -lz
Kvazaar Video Input: Bosphorus 1080p - Video Preset: Medium OpenBenchmarking.org Frames Per Second, More Is Better Kvazaar 2.2 Video Input: Bosphorus 1080p - Video Preset: Medium smt d smt c smt b smt a no smt b no smt a c b a 40 80 120 160 200 136.42 140.89 138.33 140.78 161.40 159.63 143.81 144.21 143.31 1. (CC) gcc options: -pthread -ftree-vectorize -fvisibility=hidden -O2 -lpthread -lm -lrt
Neural Magic DeepSparse Model: NLP Document Classification, oBERT base uncased on IMDB - Scenario: Synchronous Single-Stream OpenBenchmarking.org items/sec, More Is Better Neural Magic DeepSparse 1.3.2 Model: NLP Document Classification, oBERT base uncased on IMDB - Scenario: Synchronous Single-Stream smt d smt c smt b smt a no smt b no smt a c b a 8 16 24 32 40 30.25 31.93 31.78 31.76 29.75 31.91 35.12 35.10 34.83
Neural Magic DeepSparse Model: NLP Document Classification, oBERT base uncased on IMDB - Scenario: Synchronous Single-Stream OpenBenchmarking.org ms/batch, Fewer Is Better Neural Magic DeepSparse 1.3.2 Model: NLP Document Classification, oBERT base uncased on IMDB - Scenario: Synchronous Single-Stream smt d smt c smt b smt a no smt b no smt a c b a 8 16 24 32 40 33.05 31.31 31.46 31.48 33.60 31.33 28.47 28.49 28.71
RocksDB Test: Random Fill Sync OpenBenchmarking.org Op/s, More Is Better RocksDB 7.9.2 Test: Random Fill Sync smt d smt c smt b smt a no smt b no smt a c b a 90K 180K 270K 360K 450K 404463 394852 401295 388052 344040 350673 356922 373002 376601 1. (CXX) g++ options: -O3 -march=native -pthread -fno-builtin-memcmp -fno-rtti
Neural Magic DeepSparse Model: NLP Document Classification, oBERT base uncased on IMDB - Scenario: Asynchronous Multi-Stream OpenBenchmarking.org ms/batch, Fewer Is Better Neural Magic DeepSparse 1.3.2 Model: NLP Document Classification, oBERT base uncased on IMDB - Scenario: Asynchronous Multi-Stream smt d smt c smt b smt a no smt b no smt a c b a 200 400 600 800 1000 974.99 971.90 975.50 972.52 955.24 955.16 1122.06 1118.44 1115.76
Neural Magic DeepSparse Model: NLP Token Classification, BERT base uncased conll2003 - Scenario: Asynchronous Multi-Stream OpenBenchmarking.org ms/batch, Fewer Is Better Neural Magic DeepSparse 1.3.2 Model: NLP Token Classification, BERT base uncased conll2003 - Scenario: Asynchronous Multi-Stream smt d smt c smt b smt a no smt b no smt a c b a 200 400 600 800 1000 966.76 971.67 969.21 970.22 955.34 956.01 1117.73 1116.29 1116.43
Kvazaar Video Input: Bosphorus 4K - Video Preset: Very Fast OpenBenchmarking.org Frames Per Second, More Is Better Kvazaar 2.2 Video Input: Bosphorus 4K - Video Preset: Very Fast smt d smt c smt b smt a no smt b no smt a c b a 20 40 60 80 100 73.24 73.57 74.11 75.91 69.94 70.02 80.31 81.61 80.90 1. (CC) gcc options: -pthread -ftree-vectorize -fvisibility=hidden -O2 -lpthread -lm -lrt
Kvazaar Video Input: Bosphorus 1080p - Video Preset: Ultra Fast OpenBenchmarking.org Frames Per Second, More Is Better Kvazaar 2.2 Video Input: Bosphorus 1080p - Video Preset: Ultra Fast smt d smt c smt b smt a no smt b no smt a c b a 70 140 210 280 350 303.63 303.43 295.18 302.55 271.45 278.62 309.64 305.52 310.73 1. (CC) gcc options: -pthread -ftree-vectorize -fvisibility=hidden -O2 -lpthread -lm -lrt
Kvazaar Video Input: Bosphorus 4K - Video Preset: Super Fast OpenBenchmarking.org Frames Per Second, More Is Better Kvazaar 2.2 Video Input: Bosphorus 4K - Video Preset: Super Fast smt d smt c smt b smt a no smt b no smt a c b a 20 40 60 80 100 75.50 78.72 76.70 76.64 74.03 75.50 84.55 80.68 81.90 1. (CC) gcc options: -pthread -ftree-vectorize -fvisibility=hidden -O2 -lpthread -lm -lrt
Kvazaar Video Input: Bosphorus 4K - Video Preset: Ultra Fast OpenBenchmarking.org Frames Per Second, More Is Better Kvazaar 2.2 Video Input: Bosphorus 4K - Video Preset: Ultra Fast smt d smt c smt b smt a no smt b no smt a c b a 20 40 60 80 100 78.85 76.14 77.00 77.96 74.37 76.05 82.70 84.77 83.45 1. (CC) gcc options: -pthread -ftree-vectorize -fvisibility=hidden -O2 -lpthread -lm -lrt
OpenVINO Model: Age Gender Recognition Retail 0013 FP16-INT8 - Device: CPU OpenBenchmarking.org ms, Fewer Is Better OpenVINO 2022.3 Model: Age Gender Recognition Retail 0013 FP16-INT8 - Device: CPU smt d smt c smt b smt a no smt b no smt a c b a 0.2453 0.4906 0.7359 0.9812 1.2265 0.97 0.96 0.97 0.97 1.01 1.01 1.09 1.09 1.09 MIN: 0.86 / MAX: 13.31 MIN: 0.87 / MAX: 9.72 MIN: 0.87 / MAX: 10.96 MIN: 0.87 / MAX: 9.78 MIN: 0.86 / MAX: 8.72 MIN: 0.86 / MAX: 8.97 MIN: 0.49 / MAX: 22.06 MIN: 0.48 / MAX: 25.82 MIN: 0.49 / MAX: 19.01 1. (CXX) g++ options: -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -pie -ldl
Neural Magic DeepSparse Model: NLP Sentiment Analysis, 80% Pruned Quantized BERT Base Uncased - Scenario: Synchronous Single-Stream OpenBenchmarking.org items/sec, More Is Better Neural Magic DeepSparse 1.3.2 Model: NLP Sentiment Analysis, 80% Pruned Quantized BERT Base Uncased - Scenario: Synchronous Single-Stream smt d smt c smt b smt a no smt b no smt a c b a 40 80 120 160 200 183.95 189.41 188.41 188.58 175.32 191.26 197.66 196.54 197.10
Neural Magic DeepSparse Model: NLP Sentiment Analysis, 80% Pruned Quantized BERT Base Uncased - Scenario: Synchronous Single-Stream OpenBenchmarking.org ms/batch, Fewer Is Better Neural Magic DeepSparse 1.3.2 Model: NLP Sentiment Analysis, 80% Pruned Quantized BERT Base Uncased - Scenario: Synchronous Single-Stream smt d smt c smt b smt a no smt b no smt a c b a 1.2828 2.5656 3.8484 5.1312 6.414 5.4341 5.2773 5.3053 5.3004 5.7015 5.2262 5.0573 5.0859 5.0715
Neural Magic DeepSparse Model: CV Classification, ResNet-50 ImageNet - Scenario: Asynchronous Multi-Stream OpenBenchmarking.org ms/batch, Fewer Is Better Neural Magic DeepSparse 1.3.2 Model: CV Classification, ResNet-50 ImageNet - Scenario: Asynchronous Multi-Stream smt d smt c smt b smt a no smt b no smt a c b a 11 22 33 44 55 42.48 42.55 42.65 42.95 42.84 42.81 47.29 47.35 47.46
VP9 libvpx Encoding Speed: Speed 0 - Input: Bosphorus 4K OpenBenchmarking.org Frames Per Second, More Is Better VP9 libvpx Encoding 1.13 Speed: Speed 0 - Input: Bosphorus 4K smt d smt c smt b smt a no smt b no smt a c b a 2 4 6 8 10 7.15 6.92 6.97 7.24 7.16 7.04 7.70 7.71 7.68 1. (CXX) g++ options: -m64 -lm -lpthread -O3 -fPIC -U_FORTIFY_SOURCE -std=gnu++11
uvg266 Video Input: Bosphorus 1080p - Video Preset: Slow OpenBenchmarking.org Frames Per Second, More Is Better uvg266 0.4.1 Video Input: Bosphorus 1080p - Video Preset: Slow smt d smt c smt b smt a no smt b no smt a c b a 20 40 60 80 100 80.00 79.63 79.95 80.17 88.13 87.15 81.10 81.41 81.16
uvg266 Video Input: Bosphorus 1080p - Video Preset: Medium OpenBenchmarking.org Frames Per Second, More Is Better uvg266 0.4.1 Video Input: Bosphorus 1080p - Video Preset: Medium smt d smt c smt b smt a no smt b no smt a c b a 20 40 60 80 100 88.91 89.16 89.70 89.40 96.67 98.21 91.29 91.47 91.37
OpenVINO Model: Weld Porosity Detection FP16-INT8 - Device: CPU OpenBenchmarking.org ms, Fewer Is Better OpenVINO 2022.3 Model: Weld Porosity Detection FP16-INT8 - Device: CPU smt d smt c smt b smt a no smt b no smt a c b a 3 6 9 12 15 9.26 9.25 9.28 9.28 9.18 9.18 10.11 10.10 10.10 MIN: 8.12 / MAX: 30.29 MIN: 8.12 / MAX: 19.04 MIN: 8.13 / MAX: 24.28 MIN: 8.1 / MAX: 22.87 MIN: 8.21 / MAX: 24.31 MIN: 8.17 / MAX: 31.1 MIN: 5.43 / MAX: 36.27 MIN: 5.47 / MAX: 28.16 MIN: 5.46 / MAX: 35.41 1. (CXX) g++ options: -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -pie -ldl
Zstd Compression Compression Level: 8, Long Mode - Compression Speed OpenBenchmarking.org MB/s, More Is Better Zstd Compression 1.5.4 Compression Level: 8, Long Mode - Compression Speed smt d smt c smt b smt a no smt b no smt a c b a 200 400 600 800 1000 879.6 860.8 852.7 853.8 892.0 859.9 926.9 910.8 938.5 1. (CC) gcc options: -O3 -pthread -lz
Neural Magic DeepSparse Model: CV Detection, YOLOv5s COCO - Scenario: Asynchronous Multi-Stream OpenBenchmarking.org ms/batch, Fewer Is Better Neural Magic DeepSparse 1.3.2 Model: CV Detection, YOLOv5s COCO - Scenario: Asynchronous Multi-Stream smt d smt c smt b smt a no smt b no smt a c b a 20 40 60 80 100 102.76 102.10 103.19 103.09 100.42 100.24 109.25 109.11 108.68
VP9 libvpx Encoding Speed: Speed 0 - Input: Bosphorus 1080p OpenBenchmarking.org Frames Per Second, More Is Better VP9 libvpx Encoding 1.13 Speed: Speed 0 - Input: Bosphorus 1080p smt d smt c smt b smt a no smt b no smt a c b a 4 8 12 16 20 13.68 13.97 13.57 14.08 13.72 14.28 14.76 14.67 14.69 1. (CXX) g++ options: -m64 -lm -lpthread -O3 -fPIC -U_FORTIFY_SOURCE -std=gnu++11
Zstd Compression Compression Level: 19, Long Mode - Compression Speed OpenBenchmarking.org MB/s, More Is Better Zstd Compression 1.5.4 Compression Level: 19, Long Mode - Compression Speed smt d smt c smt b smt a no smt b no smt a c b a 3 6 9 12 15 9.81 9.82 9.87 9.78 9.76 9.39 9.30 9.29 9.22 1. (CC) gcc options: -O3 -pthread -lz
Neural Magic DeepSparse Model: CV Segmentation, 90% Pruned YOLACT Pruned - Scenario: Asynchronous Multi-Stream OpenBenchmarking.org ms/batch, Fewer Is Better Neural Magic DeepSparse 1.3.2 Model: CV Segmentation, 90% Pruned YOLACT Pruned - Scenario: Asynchronous Multi-Stream smt d smt c smt b smt a no smt b no smt a c b a 70 140 210 280 350 301.92 303.25 304.08 303.33 303.74 304.21 320.05 319.96 319.98
Neural Magic DeepSparse Model: NLP Text Classification, DistilBERT mnli - Scenario: Asynchronous Multi-Stream OpenBenchmarking.org ms/batch, Fewer Is Better Neural Magic DeepSparse 1.3.2 Model: NLP Text Classification, DistilBERT mnli - Scenario: Asynchronous Multi-Stream smt d smt c smt b smt a no smt b no smt a c b a 20 40 60 80 100 73.62 73.34 73.35 73.31 73.21 73.20 76.62 77.31 77.34
VVenC Video Input: Bosphorus 1080p - Video Preset: Faster OpenBenchmarking.org Frames Per Second, More Is Better VVenC 1.7 Video Input: Bosphorus 1080p - Video Preset: Faster no smt b no smt a c b a 7 14 21 28 35 28.52 28.69 30.08 29.85 30.04 1. (CXX) g++ options: -O3 -flto -fno-fat-lto-objects -flto=auto
Zstd Compression Compression Level: 19 - Compression Speed OpenBenchmarking.org MB/s, More Is Better Zstd Compression 1.5.4 Compression Level: 19 - Compression Speed smt d smt c smt b smt a no smt b no smt a c b a 5 10 15 20 25 19.6 19.2 19.5 18.8 19.8 19.8 19.1 19.1 19.1 1. (CC) gcc options: -O3 -pthread -lz
Neural Magic DeepSparse Model: NLP Text Classification, BERT base uncased SST2 - Scenario: Asynchronous Multi-Stream OpenBenchmarking.org ms/batch, Fewer Is Better Neural Magic DeepSparse 1.3.2 Model: NLP Text Classification, BERT base uncased SST2 - Scenario: Asynchronous Multi-Stream smt d smt c smt b smt a no smt b no smt a c b a 30 60 90 120 150 144.91 144.47 145.27 145.40 145.36 145.35 151.39 151.60 152.14
Neural Magic DeepSparse Model: NLP Question Answering, BERT base uncased SQuaD 12layer Pruned90 - Scenario: Asynchronous Multi-Stream OpenBenchmarking.org ms/batch, Fewer Is Better Neural Magic DeepSparse 1.3.2 Model: NLP Question Answering, BERT base uncased SQuaD 12layer Pruned90 - Scenario: Asynchronous Multi-Stream smt d smt c smt b smt a no smt b no smt a c b a 30 60 90 120 150 117.22 116.97 116.48 117.12 115.64 116.15 119.70 119.38 119.21
Stress-NG Test: Socket Activity OpenBenchmarking.org Bogo Ops/s, More Is Better Stress-NG 0.15.04 Test: Socket Activity smt d smt c smt b smt a no smt b no smt a c b a 2K 4K 6K 8K 10K 8864.78 8747.83 8750.61 8748.99 8924.10 8968.28 8876.10 8851.65 8873.58 1. (CC) gcc options: -std=gnu99 -O2 -lm -lc -lcrypt -ldl -lrt -pthread
Zstd Compression Compression Level: 19 - Decompression Speed OpenBenchmarking.org MB/s, More Is Better Zstd Compression 1.5.4 Compression Level: 19 - Decompression Speed smt d smt c smt b smt a no smt b no smt a c b a 300 600 900 1200 1500 1479.1 1475.2 1483.5 1495.2 1483.8 1483.1 1470.5 1467.8 1472.6 1. (CC) gcc options: -O3 -pthread -lz
Zstd Compression Compression Level: 12 - Decompression Speed OpenBenchmarking.org MB/s, More Is Better Zstd Compression 1.5.4 Compression Level: 12 - Decompression Speed smt d smt c smt b smt a no smt b no smt a c b a 400 800 1200 1600 2000 1731.2 1732.3 1726.3 1723.7 1727.8 1728.0 1715.6 1716.7 1704.8 1. (CC) gcc options: -O3 -pthread -lz
VVenC Video Input: Bosphorus 4K - Video Preset: Fast OpenBenchmarking.org Frames Per Second, More Is Better VVenC 1.7 Video Input: Bosphorus 4K - Video Preset: Fast no smt b no smt a c b a 1.3264 2.6528 3.9792 5.3056 6.632 5.895 5.890 5.808 5.809 5.820 1. (CXX) g++ options: -O3 -flto -fno-fat-lto-objects -flto=auto
Zstd Compression Compression Level: 19, Long Mode - Decompression Speed OpenBenchmarking.org MB/s, More Is Better Zstd Compression 1.5.4 Compression Level: 19, Long Mode - Decompression Speed smt d smt c smt b smt a no smt b no smt a c b a 300 600 900 1200 1500 1391.7 1389.0 1392.4 1393.5 1397.6 1395.8 1384.0 1378.2 1383.7 1. (CC) gcc options: -O3 -pthread -lz
VVenC Video Input: Bosphorus 4K - Video Preset: Faster OpenBenchmarking.org Frames Per Second, More Is Better VVenC 1.7 Video Input: Bosphorus 4K - Video Preset: Faster no smt b no smt a c b a 3 6 9 12 15 12.31 12.48 12.40 12.39 12.34 1. (CXX) g++ options: -O3 -flto -fno-fat-lto-objects -flto=auto
Zstd Compression Compression Level: 3 - Decompression Speed OpenBenchmarking.org MB/s, More Is Better Zstd Compression 1.5.4 Compression Level: 3 - Decompression Speed smt d smt c smt b smt a no smt b no smt a c b a 300 600 900 1200 1500 1513.5 1516.2 1519.1 1515.6 1511.5 1500.4 1517.4 1516.6 1514.8 1. (CC) gcc options: -O3 -pthread -lz
Zstd Compression Compression Level: 8 - Decompression Speed OpenBenchmarking.org MB/s, More Is Better Zstd Compression 1.5.4 Compression Level: 8 - Decompression Speed smt d smt c smt b smt a no smt b no smt a c b a 400 800 1200 1600 2000 1664.2 1664.0 1671.0 1664.3 1667.6 1669.7 1661.9 1651.0 1669.3 1. (CC) gcc options: -O3 -pthread -lz
VP9 libvpx Encoding Speed: Speed 5 - Input: Bosphorus 1080p OpenBenchmarking.org Frames Per Second, More Is Better VP9 libvpx Encoding 1.13 Speed: Speed 5 - Input: Bosphorus 1080p smt d smt c smt b smt a no smt b no smt a c b a 7 14 21 28 35 29.66 29.54 29.60 29.54 29.71 29.62 29.46 29.50 29.37 1. (CXX) g++ options: -m64 -lm -lpthread -O3 -fPIC -U_FORTIFY_SOURCE -std=gnu++11
Neural Magic DeepSparse Model: NLP Sentiment Analysis, 80% Pruned Quantized BERT Base Uncased - Scenario: Asynchronous Multi-Stream OpenBenchmarking.org ms/batch, Fewer Is Better Neural Magic DeepSparse 1.3.2 Model: NLP Sentiment Analysis, 80% Pruned Quantized BERT Base Uncased - Scenario: Asynchronous Multi-Stream smt d smt c smt b smt a no smt b no smt a c b a 7 14 21 28 35 30.72 30.58 30.75 30.75 30.64 30.42 30.49 30.46 30.59
VVenC Video Input: Bosphorus 1080p - Video Preset: Fast OpenBenchmarking.org Frames Per Second, More Is Better VVenC 1.7 Video Input: Bosphorus 1080p - Video Preset: Fast no smt b no smt a c b a 3 6 9 12 15 12.37 12.36 12.44 12.40 12.44 1. (CXX) g++ options: -O3 -flto -fno-fat-lto-objects -flto=auto
Zstd Compression Compression Level: 8, Long Mode - Decompression Speed OpenBenchmarking.org MB/s, More Is Better Zstd Compression 1.5.4 Compression Level: 8, Long Mode - Decompression Speed smt d smt c smt b smt a no smt b no smt a c b a 400 800 1200 1600 2000 1677.6 1683.7 1676.5 1682.5 1682.8 1684.8 1675.4 1673.7 1677.9 1. (CC) gcc options: -O3 -pthread -lz
Zstd Compression Compression Level: 3, Long Mode - Decompression Speed OpenBenchmarking.org MB/s, More Is Better Zstd Compression 1.5.4 Compression Level: 3, Long Mode - Decompression Speed smt d smt c smt b smt a no smt b no smt a c b a 300 600 900 1200 1500 1543.6 1540.9 1540.4 1539.5 1538.8 1542.5 1540.9 1537.5 1536.6 1. (CC) gcc options: -O3 -pthread -lz
Phoronix Test Suite v10.8.4