sss

AMD Ryzen 9 5950X 16-Core testing with a ASUS ROG CROSSHAIR VIII HERO (WI-FI) (4006 BIOS) and llvmpipe on Ubuntu 20.04 via the Phoronix Test Suite.

HTML result view exported from: https://openbenchmarking.org/result/2210145-NE-SSS86725756&sor&grs.

sssProcessorMotherboardChipsetMemoryDiskGraphicsAudioMonitorNetworkOSKernelDesktopDisplay ServerOpenGLOpenCLVulkanCompilerFile-SystemScreen ResolutionABCAMD Ryzen 9 5950X 16-Core @ 3.40GHz (16 Cores / 32 Threads)ASUS ROG CROSSHAIR VIII HERO (WI-FI) (4006 BIOS)AMD Starship/Matisse32GB500GB Western Digital WDS500G3X0C-00SJG0llvmpipe (2450MHz)Intel Device 4f92ASUS MG28URealtek RTL8125 2.5GbE + Intel I211 + Intel Wi-Fi 6 AX200Ubuntu 20.046.0.0-060000rc5daily20220915-generic (x86_64)GNOME Shell 3.36.9X Server 1.20.134.5 Mesa 21.2.6 (LLVM 12.0.0 256 bits)OpenCL 3.01.1.182GCC 9.4.0ext43840x2160OpenBenchmarking.orgKernel Details- i915.force_probe=56a5 - Transparent Huge Pages: madviseCompiler Details- --build=x86_64-linux-gnu --disable-vtable-verify --disable-werror --enable-checking=release --enable-clocale=gnu --enable-default-pie --enable-gnu-unique-object --enable-languages=c,ada,c++,go,brig,d,fortran,objc,obj-c++,gm2 --enable-libstdcxx-debug --enable-libstdcxx-time=yes --enable-multiarch --enable-multilib --enable-nls --enable-objc-gc=auto --enable-offload-targets=nvptx-none=/build/gcc-9-Av3uEd/gcc-9-9.4.0/debian/tmp-nvptx/usr,hsa --enable-plugin --enable-shared --enable-threads=posix --host=x86_64-linux-gnu --program-prefix=x86_64-linux-gnu- --target=x86_64-linux-gnu --with-abi=m64 --with-arch-32=i686 --with-default-libstdcxx-abi=new --with-gcc-major-version-only --with-multilib-list=m32,m64,mx32 --with-target-system-zlib=auto --with-tune=generic --without-cuda-driver -v Processor Details- Scaling Governor: acpi-cpufreq ondemand (Boost: Enabled) - CPU Microcode: 0xa201016 Python Details- Python 2.7.18 + Python 3.8.10Security Details- itlb_multihit: Not affected + l1tf: Not affected + mds: Not affected + meltdown: Not affected + mmio_stale_data: Not affected + retbleed: Not affected + spec_store_bypass: Mitigation of SSB disabled via prctl + spectre_v1: Mitigation of usercopy/swapgs barriers and __user pointer sanitization + spectre_v2: Mitigation of Retpolines IBPB: conditional IBRS_FW STIBP: always-on RSB filling PBRSB-eIBRS: Not affected + srbds: Not affected + tsx_async_abort: Not affected

sssonednn: IP Shapes 3D - f32 - CPUonednn: Recurrent Neural Network Inference - f32 - CPUsmhasher: wyhashsmhasher: Spooky32smhasher: fasthash32onednn: Recurrent Neural Network Inference - bf16bf16bf16 - CPUonednn: Convolution Batch Shapes Auto - u8s8f32 - CPUsmhasher: t1ha0_aes_avx2 x86_64onednn: IP Shapes 3D - u8s8f32 - CPUonednn: Recurrent Neural Network Training - f32 - CPUonednn: IP Shapes 1D - f32 - CPUdeepsparse: NLP Question Answering, BERT base uncased SQuaD 12layer Pruned90 - Asynchronous Multi-Streamdeepsparse: NLP Question Answering, BERT base uncased SQuaD 12layer Pruned90 - Asynchronous Multi-Streamopenradioss: Rubber O-Ring Seal Installationsmhasher: SHA3-256openradioss: Bumper Beamsmhasher: FarmHash128onednn: Convolution Batch Shapes Auto - f32 - CPUonednn: Deconvolution Batch shapes_3d - u8s8f32 - CPUonednn: Deconvolution Batch shapes_3d - f32 - CPUsmhasher: t1ha2_atonceonednn: Recurrent Neural Network Training - u8s8f32 - CPUopenradioss: Cell Phone Drop Testonednn: Recurrent Neural Network Inference - u8s8f32 - CPUsmhasher: MeowHash x86_64 AES-NIopenradioss: INIVOL and Fluid Structure Interaction Drop Containerdeepsparse: CV Detection,YOLOv5s COCO - Asynchronous Multi-Streamdeepsparse: CV Detection,YOLOv5s COCO - Asynchronous Multi-Streamonednn: Matrix Multiply Batch Shapes Transformer - f32 - CPUonednn: Deconvolution Batch shapes_1d - f32 - CPUdeepsparse: NLP Token Classification, BERT base uncased conll2003 - Synchronous Single-Streamdeepsparse: NLP Token Classification, BERT base uncased conll2003 - Synchronous Single-Streamquadray: 3 - 4Ktensorflow: CPU - 16 - AlexNetopenradioss: Bird Strike on Windshielddeepsparse: NLP Document Classification, oBERT base uncased on IMDB - Asynchronous Multi-Streamquadray: 5 - 1080pdeepsparse: CV Classification, ResNet-50 ImageNet - Asynchronous Multi-Streamquadray: 2 - 1080pdeepsparse: CV Classification, ResNet-50 ImageNet - Asynchronous Multi-Streamdeepsparse: NLP Token Classification, BERT base uncased conll2003 - Asynchronous Multi-Streamdeepsparse: CV Detection,YOLOv5s COCO - Synchronous Single-Streamdeepsparse: CV Detection,YOLOv5s COCO - Synchronous Single-Streamdeepsparse: NLP Token Classification, BERT base uncased conll2003 - Asynchronous Multi-Streamspacy: en_core_web_trfquadray: 1 - 1080ptensorflow: CPU - 32 - AlexNetquadray: 2 - 4Kdeepsparse: NLP Text Classification, DistilBERT mnli - Synchronous Single-Streamdeepsparse: NLP Text Classification, DistilBERT mnli - Synchronous Single-Streamdeepsparse: NLP Document Classification, oBERT base uncased on IMDB - Asynchronous Multi-Streamtensorflow: CPU - 32 - GoogLeNettensorflow: CPU - 16 - GoogLeNettensorflow: CPU - 16 - ResNet-50onednn: Deconvolution Batch shapes_1d - u8s8f32 - CPUonednn: Recurrent Neural Network Training - bf16bf16bf16 - CPUsmhasher: FarmHash32 x86_64 AVXy-cruncher: 500Mdeepsparse: NLP Text Classification, DistilBERT mnli - Asynchronous Multi-Streamdeepsparse: NLP Text Classification, DistilBERT mnli - Asynchronous Multi-Streamonednn: IP Shapes 1D - u8s8f32 - CPUdeepsparse: NLP Document Classification, oBERT base uncased on IMDB - Synchronous Single-Streamdeepsparse: NLP Document Classification, oBERT base uncased on IMDB - Synchronous Single-Streamy-cruncher: 1Bdeepsparse: NLP Text Classification, BERT base uncased SST2 - Synchronous Single-Streamdeepsparse: NLP Text Classification, BERT base uncased SST2 - Synchronous Single-Streamtensorflow: CPU - 32 - ResNet-50deepsparse: NLP Question Answering, BERT base uncased SQuaD 12layer Pruned90 - Synchronous Single-Streamdeepsparse: NLP Question Answering, BERT base uncased SQuaD 12layer Pruned90 - Synchronous Single-Streamdeepsparse: NLP Text Classification, BERT base uncased SST2 - Asynchronous Multi-Streamdeepsparse: CV Classification, ResNet-50 ImageNet - Synchronous Single-Streamdeepsparse: CV Classification, ResNet-50 ImageNet - Synchronous Single-Streamspacy: en_core_web_lgdeepsparse: NLP Text Classification, BERT base uncased SST2 - Asynchronous Multi-Streamonednn: Matrix Multiply Batch Shapes Transformer - u8s8f32 - CPUquadray: 1 - 4Kquadray: 3 - 1080pquadray: 5 - 4Ksmhasher: MeowHash x86_64 AES-NIsmhasher: t1ha0_aes_avx2 x86_64smhasher: FarmHash32 x86_64 AVXsmhasher: t1ha2_atoncesmhasher: FarmHash128smhasher: fasthash32smhasher: Spooky32smhasher: SHA3-256smhasher: wyhashABC8.180951647.7328600.2419879.938321.971637.8718.422483813.930.4676692602.443.86625192.355741.58292.08209.85110.0620617.6817.03321.659823.5780518113.852594.5987.551641.1546922.83512.2567.8059117.84570.7862434.4075211.429587.48752.8756.17225.54622.83843.5450.102113.44159.543626.728617.775856.227812.7051107446.0380.013.4180.496112.41812.734632.3634.0511.791.071592608.1233990.6918.058119.366666.99230.77611988.011411.361438.8224.180541.346411.531.326531.9148144.47338.8777112.55721600055.29550.68193711.111.440.8953.59122.09728.72522.87852.11423.5129.9441852.22514.9687.957401666.7927320.4919280.727974.211679.6418.231382904.440.4599092624.273.92386192.825141.480793.05204.91110.2920202.8116.81221.686783.6344518237.822590.1087.661655.4746726.11511.3868.3274116.96550.7954734.3600211.307588.43112.9056.67226.96625.54653.5550.424213.33158.5903626.008717.751256.304412.6875107645.9479.673.3980.327712.443812.736632.4834.1611.831.072452597.6734077.2018.126119.284967.03150.77750788.095911.350638.81924.212741.291811.4931.360031.8808144.70118.8852112.46321602155.24200.68274711.1111.440.8953.46221.95028.52323.11151.83224.69030.8151898.73816.3159.085801744.7827857.6719028.978089.021703.1118.928381425.190.4731132670.403.96322197.096340.582094.30206.29107.7620364.5017.12131.670463.6037018384.272627.8588.811664.4447376.73518.2967.4911118.40570.7899104.3950411.346088.13292.9056.51227.52628.28563.5750.523713.36158.2496631.115417.876855.909612.6218106945.7579.543.3980.043212.488012.666232.5334.2011.781.076022598.1733941.3318.118118.959867.20810.77505588.276011.327438.71024.248641.230711.4731.406031.8345144.77348.8930112.36451601655.22320.68222811.1111.440.8953.36122.09728.56022.88051.63024.31631.1981889.90515.420OpenBenchmarking.org

oneDNN

Harness: IP Shapes 3D - Data Type: f32 - Engine: CPU

OpenBenchmarking.orgms, Fewer Is BetteroneDNN 2.7Harness: IP Shapes 3D - Data Type: f32 - Engine: CPUBAC3691215SE +/- 0.05903, N = 3SE +/- 0.00928, N = 37.957408.180959.08580MIN: 7.47MIN: 7.86MIN: 8.891. (CXX) g++ options: -O3 -march=native -fopenmp -msse4.1 -fPIC -pie -lpthread -ldl

oneDNN

Harness: Recurrent Neural Network Inference - Data Type: f32 - Engine: CPU

OpenBenchmarking.orgms, Fewer Is BetteroneDNN 2.7Harness: Recurrent Neural Network Inference - Data Type: f32 - Engine: CPUABC400800120016002000SE +/- 19.69, N = 3SE +/- 19.41, N = 31647.731666.791744.78MIN: 1635.21MIN: 1620.68MIN: 1706.541. (CXX) g++ options: -O3 -march=native -fopenmp -msse4.1 -fPIC -pie -lpthread -ldl

SMHasher

Hash: wyhash

OpenBenchmarking.orgMiB/sec, More Is BetterSMHasher 2022-08-22Hash: wyhashACB6K12K18K24K30KSE +/- 110.01, N = 3SE +/- 37.17, N = 328600.2427857.6727320.491. (CXX) g++ options: -march=native -O3 -flto -fno-fat-lto-objects -lpthread

SMHasher

Hash: Spooky32

OpenBenchmarking.orgMiB/sec, More Is BetterSMHasher 2022-08-22Hash: Spooky32ABC4K8K12K16K20KSE +/- 10.48, N = 3SE +/- 43.76, N = 319879.9319280.7219028.971. (CXX) g++ options: -march=native -O3 -flto -fno-fat-lto-objects -lpthread

SMHasher

Hash: fasthash32

OpenBenchmarking.orgMiB/sec, More Is BetterSMHasher 2022-08-22Hash: fasthash32ACB2K4K6K8K10KSE +/- 59.27, N = 3SE +/- 27.99, N = 38321.978089.027974.211. (CXX) g++ options: -march=native -O3 -flto -fno-fat-lto-objects -lpthread

oneDNN

Harness: Recurrent Neural Network Inference - Data Type: bf16bf16bf16 - Engine: CPU

OpenBenchmarking.orgms, Fewer Is BetteroneDNN 2.7Harness: Recurrent Neural Network Inference - Data Type: bf16bf16bf16 - Engine: CPUABC400800120016002000SE +/- 12.64, N = 11SE +/- 15.79, N = 61637.871679.641703.11MIN: 1622.93MIN: 1619.79MIN: 1634.781. (CXX) g++ options: -O3 -march=native -fopenmp -msse4.1 -fPIC -pie -lpthread -ldl

oneDNN

Harness: Convolution Batch Shapes Auto - Data Type: u8s8f32 - Engine: CPU

OpenBenchmarking.orgms, Fewer Is BetteroneDNN 2.7Harness: Convolution Batch Shapes Auto - Data Type: u8s8f32 - Engine: CPUBAC510152025SE +/- 0.06, N = 3SE +/- 0.05, N = 318.2318.4218.93MIN: 17.9MIN: 18.06MIN: 18.481. (CXX) g++ options: -O3 -march=native -fopenmp -msse4.1 -fPIC -pie -lpthread -ldl

SMHasher

Hash: t1ha0_aes_avx2 x86_64

OpenBenchmarking.orgMiB/sec, More Is BetterSMHasher 2022-08-22Hash: t1ha0_aes_avx2 x86_64ABC20K40K60K80K100KSE +/- 679.73, N = 9SE +/- 940.93, N = 383813.9382904.4481425.191. (CXX) g++ options: -march=native -O3 -flto -fno-fat-lto-objects -lpthread

oneDNN

Harness: IP Shapes 3D - Data Type: u8s8f32 - Engine: CPU

OpenBenchmarking.orgms, Fewer Is BetteroneDNN 2.7Harness: IP Shapes 3D - Data Type: u8s8f32 - Engine: CPUBAC0.10650.2130.31950.4260.5325SE +/- 0.004656, N = 3SE +/- 0.001374, N = 30.4599090.4676690.473113MIN: 0.41MIN: 0.43MIN: 0.441. (CXX) g++ options: -O3 -march=native -fopenmp -msse4.1 -fPIC -pie -lpthread -ldl

oneDNN

Harness: Recurrent Neural Network Training - Data Type: f32 - Engine: CPU

OpenBenchmarking.orgms, Fewer Is BetteroneDNN 2.7Harness: Recurrent Neural Network Training - Data Type: f32 - Engine: CPUABC6001200180024003000SE +/- 11.17, N = 3SE +/- 28.26, N = 52602.442624.272670.40MIN: 2590.62MIN: 2596.85MIN: 2578.831. (CXX) g++ options: -O3 -march=native -fopenmp -msse4.1 -fPIC -pie -lpthread -ldl

oneDNN

Harness: IP Shapes 1D - Data Type: f32 - Engine: CPU

OpenBenchmarking.orgms, Fewer Is BetteroneDNN 2.7Harness: IP Shapes 1D - Data Type: f32 - Engine: CPUABC0.89171.78342.67513.56684.4585SE +/- 0.03093, N = 3SE +/- 0.00111, N = 33.866253.923863.96322MIN: 3.66MIN: 3.68MIN: 3.731. (CXX) g++ options: -O3 -march=native -fopenmp -msse4.1 -fPIC -pie -lpthread -ldl

Neural Magic DeepSparse

Model: NLP Question Answering, BERT base uncased SQuaD 12layer Pruned90 - Scenario: Asynchronous Multi-Stream

OpenBenchmarking.orgms/batch, Fewer Is BetterNeural Magic DeepSparse 1.1Model: NLP Question Answering, BERT base uncased SQuaD 12layer Pruned90 - Scenario: Asynchronous Multi-StreamABC4080120160200SE +/- 0.11, N = 3SE +/- 0.14, N = 3192.36192.83197.10

Neural Magic DeepSparse

Model: NLP Question Answering, BERT base uncased SQuaD 12layer Pruned90 - Scenario: Asynchronous Multi-Stream

OpenBenchmarking.orgitems/sec, More Is BetterNeural Magic DeepSparse 1.1Model: NLP Question Answering, BERT base uncased SQuaD 12layer Pruned90 - Scenario: Asynchronous Multi-StreamABC918273645SE +/- 0.02, N = 3SE +/- 0.03, N = 341.5841.4840.58

OpenRadioss

Model: Rubber O-Ring Seal Installation

OpenBenchmarking.orgSeconds, Fewer Is BetterOpenRadioss 2022.10.13Model: Rubber O-Ring Seal InstallationABC20406080100SE +/- 0.43, N = 3SE +/- 0.41, N = 392.0893.0594.30

SMHasher

Hash: SHA3-256

OpenBenchmarking.orgMiB/sec, More Is BetterSMHasher 2022-08-22Hash: SHA3-256ACB50100150200250SE +/- 0.30, N = 3SE +/- 1.57, N = 3209.85206.29204.911. (CXX) g++ options: -march=native -O3 -flto -fno-fat-lto-objects -lpthread

OpenRadioss

Model: Bumper Beam

OpenBenchmarking.orgSeconds, Fewer Is BetterOpenRadioss 2022.10.13Model: Bumper BeamCAB20406080100SE +/- 0.29, N = 3SE +/- 0.53, N = 3107.76110.06110.29

SMHasher

Hash: FarmHash128

OpenBenchmarking.orgMiB/sec, More Is BetterSMHasher 2022-08-22Hash: FarmHash128ACB4K8K12K16K20KSE +/- 92.08, N = 3SE +/- 147.41, N = 320617.6820364.5020202.811. (CXX) g++ options: -march=native -O3 -flto -fno-fat-lto-objects -lpthread

oneDNN

Harness: Convolution Batch Shapes Auto - Data Type: f32 - Engine: CPU

OpenBenchmarking.orgms, Fewer Is BetteroneDNN 2.7Harness: Convolution Batch Shapes Auto - Data Type: f32 - Engine: CPUBAC48121620SE +/- 0.04, N = 3SE +/- 0.03, N = 316.8117.0317.12MIN: 16.43MIN: 16.72MIN: 16.811. (CXX) g++ options: -O3 -march=native -fopenmp -msse4.1 -fPIC -pie -lpthread -ldl

oneDNN

Harness: Deconvolution Batch shapes_3d - Data Type: u8s8f32 - Engine: CPU

OpenBenchmarking.orgms, Fewer Is BetteroneDNN 2.7Harness: Deconvolution Batch shapes_3d - Data Type: u8s8f32 - Engine: CPUACB0.37950.7591.13851.5181.8975SE +/- 0.00419, N = 3SE +/- 0.01458, N = 31.659821.670461.68678MIN: 1.53MIN: 1.54MIN: 1.521. (CXX) g++ options: -O3 -march=native -fopenmp -msse4.1 -fPIC -pie -lpthread -ldl

oneDNN

Harness: Deconvolution Batch shapes_3d - Data Type: f32 - Engine: CPU

OpenBenchmarking.orgms, Fewer Is BetteroneDNN 2.7Harness: Deconvolution Batch shapes_3d - Data Type: f32 - Engine: CPUACB0.81781.63562.45343.27124.089SE +/- 0.01247, N = 3SE +/- 0.00327, N = 33.578053.603703.63445MIN: 3.41MIN: 3.4MIN: 3.411. (CXX) g++ options: -O3 -march=native -fopenmp -msse4.1 -fPIC -pie -lpthread -ldl

SMHasher

Hash: t1ha2_atonce

OpenBenchmarking.orgMiB/sec, More Is BetterSMHasher 2022-08-22Hash: t1ha2_atonceCBA4K8K12K16K20KSE +/- 219.35, N = 4SE +/- 55.92, N = 318384.2718237.8218113.851. (CXX) g++ options: -march=native -O3 -flto -fno-fat-lto-objects -lpthread

oneDNN

Harness: Recurrent Neural Network Training - Data Type: u8s8f32 - Engine: CPU

OpenBenchmarking.orgms, Fewer Is BetteroneDNN 2.7Harness: Recurrent Neural Network Training - Data Type: u8s8f32 - Engine: CPUBAC6001200180024003000SE +/- 6.69, N = 3SE +/- 31.84, N = 32590.102594.592627.85MIN: 2567.75MIN: 2584.09MIN: 2582.81. (CXX) g++ options: -O3 -march=native -fopenmp -msse4.1 -fPIC -pie -lpthread -ldl

OpenRadioss

Model: Cell Phone Drop Test

OpenBenchmarking.orgSeconds, Fewer Is BetterOpenRadioss 2022.10.13Model: Cell Phone Drop TestABC20406080100SE +/- 0.05, N = 3SE +/- 0.52, N = 387.5587.6688.81

oneDNN

Harness: Recurrent Neural Network Inference - Data Type: u8s8f32 - Engine: CPU

OpenBenchmarking.orgms, Fewer Is BetteroneDNN 2.7Harness: Recurrent Neural Network Inference - Data Type: u8s8f32 - Engine: CPUABC400800120016002000SE +/- 15.95, N = 3SE +/- 17.63, N = 51641.151655.471664.44MIN: 1622.41MIN: 1622.09MIN: 1618.121. (CXX) g++ options: -O3 -march=native -fopenmp -msse4.1 -fPIC -pie -lpthread -ldl

SMHasher

Hash: MeowHash x86_64 AES-NI

OpenBenchmarking.orgMiB/sec, More Is BetterSMHasher 2022-08-22Hash: MeowHash x86_64 AES-NICAB10K20K30K40K50KSE +/- 226.78, N = 3SE +/- 324.08, N = 347376.7346922.8346726.111. (CXX) g++ options: -march=native -O3 -flto -fno-fat-lto-objects -lpthread

OpenRadioss

Model: INIVOL and Fluid Structure Interaction Drop Container

OpenBenchmarking.orgSeconds, Fewer Is BetterOpenRadioss 2022.10.13Model: INIVOL and Fluid Structure Interaction Drop ContainerBAC110220330440550SE +/- 0.30, N = 3SE +/- 4.41, N = 8511.38512.25518.29

Neural Magic DeepSparse

Model: CV Detection,YOLOv5s COCO - Scenario: Asynchronous Multi-Stream

OpenBenchmarking.orgitems/sec, More Is BetterNeural Magic DeepSparse 1.1Model: CV Detection,YOLOv5s COCO - Scenario: Asynchronous Multi-StreamBAC1530456075SE +/- 0.91, N = 3SE +/- 0.13, N = 368.3367.8167.49

Neural Magic DeepSparse

Model: CV Detection,YOLOv5s COCO - Scenario: Asynchronous Multi-Stream

OpenBenchmarking.orgms/batch, Fewer Is BetterNeural Magic DeepSparse 1.1Model: CV Detection,YOLOv5s COCO - Scenario: Asynchronous Multi-StreamBAC306090120150SE +/- 1.56, N = 3SE +/- 0.21, N = 3116.97117.85118.41

oneDNN

Harness: Matrix Multiply Batch Shapes Transformer - Data Type: f32 - Engine: CPU

OpenBenchmarking.orgms, Fewer Is BetteroneDNN 2.7Harness: Matrix Multiply Batch Shapes Transformer - Data Type: f32 - Engine: CPUACB0.1790.3580.5370.7160.895SE +/- 0.002635, N = 3SE +/- 0.008536, N = 30.7862430.7899100.795473MIN: 0.7MIN: 0.7MIN: 0.71. (CXX) g++ options: -O3 -march=native -fopenmp -msse4.1 -fPIC -pie -lpthread -ldl

oneDNN

Harness: Deconvolution Batch shapes_1d - Data Type: f32 - Engine: CPU

OpenBenchmarking.orgms, Fewer Is BetteroneDNN 2.7Harness: Deconvolution Batch shapes_1d - Data Type: f32 - Engine: CPUBCA0.99171.98342.97513.96684.9585SE +/- 0.03778, N = 8SE +/- 0.01829, N = 34.360024.395044.40752MIN: 3.52MIN: 3.54MIN: 3.51. (CXX) g++ options: -O3 -march=native -fopenmp -msse4.1 -fPIC -pie -lpthread -ldl

Neural Magic DeepSparse

Model: NLP Token Classification, BERT base uncased conll2003 - Scenario: Synchronous Single-Stream

OpenBenchmarking.orgitems/sec, More Is BetterNeural Magic DeepSparse 1.1Model: NLP Token Classification, BERT base uncased conll2003 - Scenario: Synchronous Single-StreamACB3691215SE +/- 0.03, N = 3SE +/- 0.01, N = 311.4311.3511.31

Neural Magic DeepSparse

Model: NLP Token Classification, BERT base uncased conll2003 - Scenario: Synchronous Single-Stream

OpenBenchmarking.orgms/batch, Fewer Is BetterNeural Magic DeepSparse 1.1Model: NLP Token Classification, BERT base uncased conll2003 - Scenario: Synchronous Single-StreamACB20406080100SE +/- 0.22, N = 3SE +/- 0.09, N = 387.4988.1388.43

QuadRay

Scene: 3 - Resolution: 4K

OpenBenchmarking.orgFPS, More Is BetterQuadRay 2022.05.25Scene: 3 - Resolution: 4KCBA0.65251.3051.95752.613.2625SE +/- 0.01, N = 3SE +/- 0.01, N = 32.902.902.871. (CXX) g++ options: -O3 -pthread -lm -lstdc++ -lX11 -lXext -lpthread

TensorFlow

Device: CPU - Batch Size: 16 - Model: AlexNet

OpenBenchmarking.orgimages/sec, More Is BetterTensorFlow 2.10Device: CPU - Batch Size: 16 - Model: AlexNetBCA1326395265SE +/- 0.14, N = 3SE +/- 0.15, N = 356.6756.5156.17

OpenRadioss

Model: Bird Strike on Windshield

OpenBenchmarking.orgSeconds, Fewer Is BetterOpenRadioss 2022.10.13Model: Bird Strike on WindshieldABC50100150200250SE +/- 0.53, N = 3SE +/- 0.78, N = 3225.54226.96227.52

Neural Magic DeepSparse

Model: NLP Document Classification, oBERT base uncased on IMDB - Scenario: Asynchronous Multi-Stream

OpenBenchmarking.orgms/batch, Fewer Is BetterNeural Magic DeepSparse 1.1Model: NLP Document Classification, oBERT base uncased on IMDB - Scenario: Asynchronous Multi-StreamABC140280420560700SE +/- 2.50, N = 3SE +/- 2.78, N = 3622.84625.55628.29

QuadRay

Scene: 5 - Resolution: 1080p

OpenBenchmarking.orgFPS, More Is BetterQuadRay 2022.05.25Scene: 5 - Resolution: 1080pCBA0.80331.60662.40993.21324.0165SE +/- 0.00, N = 3SE +/- 0.01, N = 33.573.553.541. (CXX) g++ options: -O3 -pthread -lm -lstdc++ -lX11 -lXext -lpthread

Neural Magic DeepSparse

Model: CV Classification, ResNet-50 ImageNet - Scenario: Asynchronous Multi-Stream

OpenBenchmarking.orgms/batch, Fewer Is BetterNeural Magic DeepSparse 1.1Model: CV Classification, ResNet-50 ImageNet - Scenario: Asynchronous Multi-StreamABC1122334455SE +/- 0.23, N = 3SE +/- 0.18, N = 350.1050.4250.52

QuadRay

Scene: 2 - Resolution: 1080p

OpenBenchmarking.orgFPS, More Is BetterQuadRay 2022.05.25Scene: 2 - Resolution: 1080pACB3691215SE +/- 0.04, N = 3SE +/- 0.02, N = 313.4413.3613.331. (CXX) g++ options: -O3 -pthread -lm -lstdc++ -lX11 -lXext -lpthread

Neural Magic DeepSparse

Model: CV Classification, ResNet-50 ImageNet - Scenario: Asynchronous Multi-Stream

OpenBenchmarking.orgitems/sec, More Is BetterNeural Magic DeepSparse 1.1Model: CV Classification, ResNet-50 ImageNet - Scenario: Asynchronous Multi-StreamABC4080120160200SE +/- 0.71, N = 3SE +/- 0.56, N = 3159.54158.59158.25

Neural Magic DeepSparse

Model: NLP Token Classification, BERT base uncased conll2003 - Scenario: Asynchronous Multi-Stream

OpenBenchmarking.orgms/batch, Fewer Is BetterNeural Magic DeepSparse 1.1Model: NLP Token Classification, BERT base uncased conll2003 - Scenario: Asynchronous Multi-StreamBAC140280420560700SE +/- 1.55, N = 3SE +/- 1.66, N = 3626.01626.73631.12

Neural Magic DeepSparse

Model: CV Detection,YOLOv5s COCO - Scenario: Synchronous Single-Stream

OpenBenchmarking.orgms/batch, Fewer Is BetterNeural Magic DeepSparse 1.1Model: CV Detection,YOLOv5s COCO - Scenario: Synchronous Single-StreamBAC48121620SE +/- 0.01, N = 3SE +/- 0.01, N = 317.7517.7817.88

Neural Magic DeepSparse

Model: CV Detection,YOLOv5s COCO - Scenario: Synchronous Single-Stream

OpenBenchmarking.orgitems/sec, More Is BetterNeural Magic DeepSparse 1.1Model: CV Detection,YOLOv5s COCO - Scenario: Synchronous Single-StreamBAC1326395265SE +/- 0.05, N = 3SE +/- 0.03, N = 356.3056.2355.91

Neural Magic DeepSparse

Model: NLP Token Classification, BERT base uncased conll2003 - Scenario: Asynchronous Multi-Stream

OpenBenchmarking.orgitems/sec, More Is BetterNeural Magic DeepSparse 1.1Model: NLP Token Classification, BERT base uncased conll2003 - Scenario: Asynchronous Multi-StreamABC3691215SE +/- 0.02, N = 3SE +/- 0.03, N = 312.7112.6912.62

spaCy

Model: en_core_web_trf

OpenBenchmarking.orgtokens/sec, More Is BetterspaCy 3.4.1Model: en_core_web_trfBAC2004006008001000SE +/- 3.48, N = 3SE +/- 4.93, N = 3107610741069

QuadRay

Scene: 1 - Resolution: 1080p

OpenBenchmarking.orgFPS, More Is BetterQuadRay 2022.05.25Scene: 1 - Resolution: 1080pABC1020304050SE +/- 0.09, N = 3SE +/- 0.04, N = 346.0345.9445.751. (CXX) g++ options: -O3 -pthread -lm -lstdc++ -lX11 -lXext -lpthread

TensorFlow

Device: CPU - Batch Size: 32 - Model: AlexNet

OpenBenchmarking.orgimages/sec, More Is BetterTensorFlow 2.10Device: CPU - Batch Size: 32 - Model: AlexNetABC20406080100SE +/- 0.15, N = 3SE +/- 0.12, N = 380.0179.6779.54

QuadRay

Scene: 2 - Resolution: 4K

OpenBenchmarking.orgFPS, More Is BetterQuadRay 2022.05.25Scene: 2 - Resolution: 4KACB0.76731.53462.30193.06923.8365SE +/- 0.01, N = 3SE +/- 0.01, N = 33.413.393.391. (CXX) g++ options: -O3 -pthread -lm -lstdc++ -lX11 -lXext -lpthread

Neural Magic DeepSparse

Model: NLP Text Classification, DistilBERT mnli - Scenario: Synchronous Single-Stream

OpenBenchmarking.orgitems/sec, More Is BetterNeural Magic DeepSparse 1.1Model: NLP Text Classification, DistilBERT mnli - Scenario: Synchronous Single-StreamABC20406080100SE +/- 0.08, N = 3SE +/- 0.18, N = 380.5080.3380.04

Neural Magic DeepSparse

Model: NLP Text Classification, DistilBERT mnli - Scenario: Synchronous Single-Stream

OpenBenchmarking.orgms/batch, Fewer Is BetterNeural Magic DeepSparse 1.1Model: NLP Text Classification, DistilBERT mnli - Scenario: Synchronous Single-StreamABC3691215SE +/- 0.01, N = 3SE +/- 0.03, N = 312.4212.4412.49

Neural Magic DeepSparse

Model: NLP Document Classification, oBERT base uncased on IMDB - Scenario: Asynchronous Multi-Stream

OpenBenchmarking.orgitems/sec, More Is BetterNeural Magic DeepSparse 1.1Model: NLP Document Classification, oBERT base uncased on IMDB - Scenario: Asynchronous Multi-StreamBAC3691215SE +/- 0.07, N = 3SE +/- 0.04, N = 312.7412.7312.67

TensorFlow

Device: CPU - Batch Size: 32 - Model: GoogLeNet

OpenBenchmarking.orgimages/sec, More Is BetterTensorFlow 2.10Device: CPU - Batch Size: 32 - Model: GoogLeNetCBA816243240SE +/- 0.02, N = 3SE +/- 0.02, N = 332.5332.4832.36

TensorFlow

Device: CPU - Batch Size: 16 - Model: GoogLeNet

OpenBenchmarking.orgimages/sec, More Is BetterTensorFlow 2.10Device: CPU - Batch Size: 16 - Model: GoogLeNetCBA816243240SE +/- 0.02, N = 3SE +/- 0.02, N = 334.2034.1634.05

TensorFlow

Device: CPU - Batch Size: 16 - Model: ResNet-50

OpenBenchmarking.orgimages/sec, More Is BetterTensorFlow 2.10Device: CPU - Batch Size: 16 - Model: ResNet-50BAC3691215SE +/- 0.01, N = 3SE +/- 0.03, N = 311.8311.7911.78

oneDNN

Harness: Deconvolution Batch shapes_1d - Data Type: u8s8f32 - Engine: CPU

OpenBenchmarking.orgms, Fewer Is BetteroneDNN 2.7Harness: Deconvolution Batch shapes_1d - Data Type: u8s8f32 - Engine: CPUABC0.24210.48420.72630.96841.2105SE +/- 0.00134, N = 3SE +/- 0.00183, N = 31.071591.072451.07602MIN: 0.97MIN: 0.97MIN: 0.971. (CXX) g++ options: -O3 -march=native -fopenmp -msse4.1 -fPIC -pie -lpthread -ldl

oneDNN

Harness: Recurrent Neural Network Training - Data Type: bf16bf16bf16 - Engine: CPU

OpenBenchmarking.orgms, Fewer Is BetteroneDNN 2.7Harness: Recurrent Neural Network Training - Data Type: bf16bf16bf16 - Engine: CPUBCA6001200180024003000SE +/- 8.68, N = 3SE +/- 7.36, N = 32597.672598.172608.12MIN: 2576.33MIN: 2573.5MIN: 2597.21. (CXX) g++ options: -O3 -march=native -fopenmp -msse4.1 -fPIC -pie -lpthread -ldl

SMHasher

Hash: FarmHash32 x86_64 AVX

OpenBenchmarking.orgMiB/sec, More Is BetterSMHasher 2022-08-22Hash: FarmHash32 x86_64 AVXBAC7K14K21K28K35KSE +/- 102.23, N = 3SE +/- 52.01, N = 334077.2033990.6933941.331. (CXX) g++ options: -march=native -O3 -flto -fno-fat-lto-objects -lpthread

Y-Cruncher

Pi Digits To Calculate: 500M

OpenBenchmarking.orgSeconds, Fewer Is BetterY-Cruncher 0.7.10.9513Pi Digits To Calculate: 500MACB48121620SE +/- 0.01, N = 3SE +/- 0.02, N = 318.0618.1218.13

Neural Magic DeepSparse

Model: NLP Text Classification, DistilBERT mnli - Scenario: Asynchronous Multi-Stream

OpenBenchmarking.orgitems/sec, More Is BetterNeural Magic DeepSparse 1.1Model: NLP Text Classification, DistilBERT mnli - Scenario: Asynchronous Multi-StreamABC306090120150SE +/- 0.08, N = 3SE +/- 0.06, N = 3119.37119.28118.96

Neural Magic DeepSparse

Model: NLP Text Classification, DistilBERT mnli - Scenario: Asynchronous Multi-Stream

OpenBenchmarking.orgms/batch, Fewer Is BetterNeural Magic DeepSparse 1.1Model: NLP Text Classification, DistilBERT mnli - Scenario: Asynchronous Multi-StreamABC1530456075SE +/- 0.03, N = 3SE +/- 0.02, N = 366.9967.0367.21

oneDNN

Harness: IP Shapes 1D - Data Type: u8s8f32 - Engine: CPU

OpenBenchmarking.orgms, Fewer Is BetteroneDNN 2.7Harness: IP Shapes 1D - Data Type: u8s8f32 - Engine: CPUCAB0.17490.34980.52470.69960.8745SE +/- 0.001575, N = 3SE +/- 0.000399, N = 30.7750550.7761190.777507MIN: 0.7MIN: 0.7MIN: 0.711. (CXX) g++ options: -O3 -march=native -fopenmp -msse4.1 -fPIC -pie -lpthread -ldl

Neural Magic DeepSparse

Model: NLP Document Classification, oBERT base uncased on IMDB - Scenario: Synchronous Single-Stream

OpenBenchmarking.orgms/batch, Fewer Is BetterNeural Magic DeepSparse 1.1Model: NLP Document Classification, oBERT base uncased on IMDB - Scenario: Synchronous Single-StreamABC20406080100SE +/- 0.13, N = 3SE +/- 0.06, N = 388.0188.1088.28

Neural Magic DeepSparse

Model: NLP Document Classification, oBERT base uncased on IMDB - Scenario: Synchronous Single-Stream

OpenBenchmarking.orgitems/sec, More Is BetterNeural Magic DeepSparse 1.1Model: NLP Document Classification, oBERT base uncased on IMDB - Scenario: Synchronous Single-StreamABC3691215SE +/- 0.02, N = 3SE +/- 0.01, N = 311.3611.3511.33

Y-Cruncher

Pi Digits To Calculate: 1B

OpenBenchmarking.orgSeconds, Fewer Is BetterY-Cruncher 0.7.10.9513Pi Digits To Calculate: 1BCBA918273645SE +/- 0.09, N = 3SE +/- 0.13, N = 338.7138.8238.82

Neural Magic DeepSparse

Model: NLP Text Classification, BERT base uncased SST2 - Scenario: Synchronous Single-Stream

OpenBenchmarking.orgms/batch, Fewer Is BetterNeural Magic DeepSparse 1.1Model: NLP Text Classification, BERT base uncased SST2 - Scenario: Synchronous Single-StreamABC612182430SE +/- 0.01, N = 3SE +/- 0.03, N = 324.1824.2124.25

Neural Magic DeepSparse

Model: NLP Text Classification, BERT base uncased SST2 - Scenario: Synchronous Single-Stream

OpenBenchmarking.orgitems/sec, More Is BetterNeural Magic DeepSparse 1.1Model: NLP Text Classification, BERT base uncased SST2 - Scenario: Synchronous Single-StreamABC918273645SE +/- 0.02, N = 3SE +/- 0.05, N = 341.3541.2941.23

TensorFlow

Device: CPU - Batch Size: 32 - Model: ResNet-50

OpenBenchmarking.orgimages/sec, More Is BetterTensorFlow 2.10Device: CPU - Batch Size: 32 - Model: ResNet-50ABC3691215SE +/- 0.01, N = 3SE +/- 0.02, N = 311.5011.4911.47

Neural Magic DeepSparse

Model: NLP Question Answering, BERT base uncased SQuaD 12layer Pruned90 - Scenario: Synchronous Single-Stream

OpenBenchmarking.orgms/batch, Fewer Is BetterNeural Magic DeepSparse 1.1Model: NLP Question Answering, BERT base uncased SQuaD 12layer Pruned90 - Scenario: Synchronous Single-StreamABC714212835SE +/- 0.06, N = 3SE +/- 0.10, N = 331.3331.3631.41

Neural Magic DeepSparse

Model: NLP Question Answering, BERT base uncased SQuaD 12layer Pruned90 - Scenario: Synchronous Single-Stream

OpenBenchmarking.orgitems/sec, More Is BetterNeural Magic DeepSparse 1.1Model: NLP Question Answering, BERT base uncased SQuaD 12layer Pruned90 - Scenario: Synchronous Single-StreamABC714212835SE +/- 0.06, N = 3SE +/- 0.10, N = 331.9131.8831.83

Neural Magic DeepSparse

Model: NLP Text Classification, BERT base uncased SST2 - Scenario: Asynchronous Multi-Stream

OpenBenchmarking.orgms/batch, Fewer Is BetterNeural Magic DeepSparse 1.1Model: NLP Text Classification, BERT base uncased SST2 - Scenario: Asynchronous Multi-StreamABC306090120150SE +/- 0.23, N = 3SE +/- 0.12, N = 3144.47144.70144.77

Neural Magic DeepSparse

Model: CV Classification, ResNet-50 ImageNet - Scenario: Synchronous Single-Stream

OpenBenchmarking.orgms/batch, Fewer Is BetterNeural Magic DeepSparse 1.1Model: CV Classification, ResNet-50 ImageNet - Scenario: Synchronous Single-StreamABC246810SE +/- 0.0017, N = 3SE +/- 0.0051, N = 38.87778.88528.8930

Neural Magic DeepSparse

Model: CV Classification, ResNet-50 ImageNet - Scenario: Synchronous Single-Stream

OpenBenchmarking.orgitems/sec, More Is BetterNeural Magic DeepSparse 1.1Model: CV Classification, ResNet-50 ImageNet - Scenario: Synchronous Single-StreamABC306090120150SE +/- 0.02, N = 3SE +/- 0.06, N = 3112.56112.46112.36

spaCy

Model: en_core_web_lg

OpenBenchmarking.orgtokens/sec, More Is BetterspaCy 3.4.1Model: en_core_web_lgBCA3K6K9K12K15KSE +/- 18.22, N = 3SE +/- 4.58, N = 3160211601616000

Neural Magic DeepSparse

Model: NLP Text Classification, BERT base uncased SST2 - Scenario: Asynchronous Multi-Stream

OpenBenchmarking.orgitems/sec, More Is BetterNeural Magic DeepSparse 1.1Model: NLP Text Classification, BERT base uncased SST2 - Scenario: Asynchronous Multi-StreamABC1224364860SE +/- 0.08, N = 3SE +/- 0.05, N = 355.3055.2455.22

oneDNN

Harness: Matrix Multiply Batch Shapes Transformer - Data Type: u8s8f32 - Engine: CPU

OpenBenchmarking.orgms, Fewer Is BetteroneDNN 2.7Harness: Matrix Multiply Batch Shapes Transformer - Data Type: u8s8f32 - Engine: CPUACB0.15360.30720.46080.61440.768SE +/- 0.002683, N = 3SE +/- 0.002859, N = 30.6819370.6822280.682747MIN: 0.59MIN: 0.58MIN: 0.581. (CXX) g++ options: -O3 -march=native -fopenmp -msse4.1 -fPIC -pie -lpthread -ldl

QuadRay

Scene: 1 - Resolution: 4K

OpenBenchmarking.orgFPS, More Is BetterQuadRay 2022.05.25Scene: 1 - Resolution: 4KCBA3691215SE +/- 0.02, N = 3SE +/- 0.03, N = 311.1111.1111.101. (CXX) g++ options: -O3 -pthread -lm -lstdc++ -lX11 -lXext -lpthread

QuadRay

Scene: 3 - Resolution: 1080p

OpenBenchmarking.orgFPS, More Is BetterQuadRay 2022.05.25Scene: 3 - Resolution: 1080pCBA3691215SE +/- 0.01, N = 3SE +/- 0.03, N = 311.4411.4411.441. (CXX) g++ options: -O3 -pthread -lm -lstdc++ -lX11 -lXext -lpthread

QuadRay

Scene: 5 - Resolution: 4K

OpenBenchmarking.orgFPS, More Is BetterQuadRay 2022.05.25Scene: 5 - Resolution: 4KCBA0.20030.40060.60090.80121.0015SE +/- 0.00, N = 3SE +/- 0.00, N = 30.890.890.891. (CXX) g++ options: -O3 -pthread -lm -lstdc++ -lX11 -lXext -lpthread

SMHasher

Hash: MeowHash x86_64 AES-NI

OpenBenchmarking.orgcycles/hash, Fewer Is BetterSMHasher 2022-08-22Hash: MeowHash x86_64 AES-NICBA1224364860SE +/- 0.30, N = 3SE +/- 0.18, N = 353.3653.4653.591. (CXX) g++ options: -march=native -O3 -flto -fno-fat-lto-objects -lpthread

SMHasher

Hash: t1ha0_aes_avx2 x86_64

OpenBenchmarking.orgcycles/hash, Fewer Is BetterSMHasher 2022-08-22Hash: t1ha0_aes_avx2 x86_64BAC510152025SE +/- 0.17, N = 9SE +/- 0.00, N = 321.9522.1022.101. (CXX) g++ options: -march=native -O3 -flto -fno-fat-lto-objects -lpthread

SMHasher

Hash: FarmHash32 x86_64 AVX

OpenBenchmarking.orgcycles/hash, Fewer Is BetterSMHasher 2022-08-22Hash: FarmHash32 x86_64 AVXBCA714212835SE +/- 0.01, N = 3SE +/- 0.02, N = 328.5228.5628.731. (CXX) g++ options: -march=native -O3 -flto -fno-fat-lto-objects -lpthread

SMHasher

Hash: t1ha2_atonce

OpenBenchmarking.orgcycles/hash, Fewer Is BetterSMHasher 2022-08-22Hash: t1ha2_atonceACB612182430SE +/- 0.43, N = 4SE +/- 0.27, N = 322.8822.8823.111. (CXX) g++ options: -march=native -O3 -flto -fno-fat-lto-objects -lpthread

SMHasher

Hash: FarmHash128

OpenBenchmarking.orgcycles/hash, Fewer Is BetterSMHasher 2022-08-22Hash: FarmHash128CBA1224364860SE +/- 0.45, N = 3SE +/- 0.44, N = 351.6351.8352.111. (CXX) g++ options: -march=native -O3 -flto -fno-fat-lto-objects -lpthread

SMHasher

Hash: fasthash32

OpenBenchmarking.orgcycles/hash, Fewer Is BetterSMHasher 2022-08-22Hash: fasthash32ACB612182430SE +/- 0.16, N = 3SE +/- 0.10, N = 323.5124.3224.691. (CXX) g++ options: -march=native -O3 -flto -fno-fat-lto-objects -lpthread

SMHasher

Hash: Spooky32

OpenBenchmarking.orgcycles/hash, Fewer Is BetterSMHasher 2022-08-22Hash: Spooky32ABC714212835SE +/- 0.09, N = 3SE +/- 0.11, N = 329.9430.8231.201. (CXX) g++ options: -march=native -O3 -flto -fno-fat-lto-objects -lpthread

SMHasher

Hash: SHA3-256

OpenBenchmarking.orgcycles/hash, Fewer Is BetterSMHasher 2022-08-22Hash: SHA3-256ACB400800120016002000SE +/- 3.87, N = 3SE +/- 14.44, N = 31852.231889.911898.741. (CXX) g++ options: -march=native -O3 -flto -fno-fat-lto-objects -lpthread

SMHasher

Hash: wyhash

OpenBenchmarking.orgcycles/hash, Fewer Is BetterSMHasher 2022-08-22Hash: wyhashACB48121620SE +/- 0.16, N = 3SE +/- 0.35, N = 314.9715.4216.321. (CXX) g++ options: -march=native -O3 -flto -fno-fat-lto-objects -lpthread

Phoronix Test Suite v10.8.4