2023-02-25-0937

Docker testing on Ubuntu 20.04.4 LTS via the Phoronix Test Suite.

HTML result view exported from: https://openbenchmarking.org/result/2302251-NE-20230225022&gru.

2023-02-25-0937ProcessorMotherboardChipsetMemoryDiskGraphicsAudioMonitorNetworkOSKernelOpenCLVulkanCompilerFile-SystemScreen ResolutionSystem LayerASPEED - AMD Ryzen Threadripper PRO 3955WX 16-CoresAMD Ryzen Threadripper PRO 3955WX 16-Cores @ 3.90GHz (16 Cores / 32 Threads)ASUS Pro WS WRX80E-SAGE SE WIFI (0602 BIOS)AMD Starship/Matisse256GB4 x 2000GB Samsung SSD 970 EVO Plus 2TB + 1000GB Samsung SSD 970 EVO Plus 1TB + 500GB Western Digital WDS500G2B0BASPEED 16GB (2450MHz)Intel Device 4f922 x DELL U2312HM + 28E8502 x Intel 10G X550TUbuntu 20.04.4 LTS6.2.0-1275.native (x86_64)OpenCL 2.1 AMD-APP (3513.0)1.2.182GCC 9.4.0overlayfs1920x1080DockerOpenBenchmarking.org- amdgpu.ppfeaturemask=0xffffffff amdgpu.vm_block_size=10 amdgpu.vm_size=1024 - Transparent Huge Pages: always- --build=x86_64-linux-gnu --disable-vtable-verify --disable-werror --enable-checking=release --enable-clocale=gnu --enable-default-pie --enable-gnu-unique-object --enable-languages=c,ada,c++,go,brig,d,fortran,objc,obj-c++,gm2 --enable-libstdcxx-debug --enable-libstdcxx-time=yes --enable-multiarch --enable-multilib --enable-nls --enable-objc-gc=auto --enable-offload-targets=nvptx-none=/build/gcc-9-Av3uEd/gcc-9-9.4.0/debian/tmp-nvptx/usr,hsa --enable-plugin --enable-shared --enable-threads=posix --host=x86_64-linux-gnu --program-prefix=x86_64-linux-gnu- --target=x86_64-linux-gnu --with-abi=m64 --with-arch-32=i686 --with-default-libstdcxx-abi=new --with-gcc-major-version-only --with-multilib-list=m32,m64,mx32 --with-target-system-zlib=auto --with-tune=generic --without-cuda-driver -v - Scaling Governor: acpi-cpufreq performance (Boost: Enabled) - CPU Microcode: 0x8301055- Python 3.8.10- itlb_multihit: Not affected + l1tf: Not affected + mds: Not affected + meltdown: Not affected + mmio_stale_data: Not affected + retbleed: Vulnerable + spec_store_bypass: Vulnerable + spectre_v1: Vulnerable: __user pointer sanitization and usercopy barriers only; no swapgs barriers + spectre_v2: Vulnerable IBPB: disabled STIBP: disabled PBRSB-eIBRS: Not affected + srbds: Not affected + tsx_async_abort: Not affected

2023-02-25-0937plaidml: No - Inference - VGG16 - CPUplaidml: No - Inference - ResNet 50 - CPUopenvino: Face Detection FP16 - CPUopenvino: Person Detection FP16 - CPUopenvino: Person Detection FP32 - CPUopenvino: Vehicle Detection FP16 - CPUopenvino: Face Detection FP16-INT8 - CPUopenvino: Vehicle Detection FP16-INT8 - CPUopenvino: Weld Porosity Detection FP16 - CPUopenvino: Machine Translation EN To DE FP16 - CPUopenvino: Weld Porosity Detection FP16-INT8 - CPUopenvino: Person Vehicle Bike Detection FP16 - CPUopenvino: Age Gender Recognition Retail 0013 FP16 - CPUopenvino: Age Gender Recognition Retail 0013 FP16-INT8 - CPUshoc: OpenCL - Triadshoc: OpenCL - Reductionshoc: OpenCL - Bus Speed Downloadshoc: OpenCL - Bus Speed Readbackshoc: OpenCL - Texture Read Bandwidthshoc: OpenCL - S3Dshoc: OpenCL - FFT SPshoc: OpenCL - GEMM SGEMM_Nshoc: OpenCL - Max SP Flopsshoc: OpenCL - MD5 Hashdeepsparse: NLP Document Classification, oBERT base uncased on IMDB - Asynchronous Multi-Streamdeepsparse: NLP Document Classification, oBERT base uncased on IMDB - Synchronous Single-Streamdeepsparse: NLP Sentiment Analysis, 80% Pruned Quantized BERT Base Uncased - Asynchronous Multi-Streamdeepsparse: NLP Sentiment Analysis, 80% Pruned Quantized BERT Base Uncased - Synchronous Single-Streamdeepsparse: NLP Question Answering, BERT base uncased SQuaD 12layer Pruned90 - Asynchronous Multi-Streamdeepsparse: NLP Question Answering, BERT base uncased SQuaD 12layer Pruned90 - Synchronous Single-Streamdeepsparse: CV Detection, YOLOv5s COCO - Asynchronous Multi-Streamdeepsparse: CV Detection, YOLOv5s COCO - Synchronous Single-Streamdeepsparse: CV Classification, ResNet-50 ImageNet - Asynchronous Multi-Streamdeepsparse: CV Classification, ResNet-50 ImageNet - Synchronous Single-Streamdeepsparse: NLP Text Classification, DistilBERT mnli - Asynchronous Multi-Streamdeepsparse: NLP Text Classification, DistilBERT mnli - Synchronous Single-Streamdeepsparse: CV Segmentation, 90% Pruned YOLACT Pruned - Asynchronous Multi-Streamdeepsparse: CV Segmentation, 90% Pruned YOLACT Pruned - Synchronous Single-Streamdeepsparse: NLP Text Classification, BERT base uncased SST2 - Asynchronous Multi-Streamdeepsparse: NLP Text Classification, BERT base uncased SST2 - Synchronous Single-Streamdeepsparse: NLP Token Classification, BERT base uncased conll2003 - Asynchronous Multi-Streamdeepsparse: NLP Token Classification, BERT base uncased conll2003 - Synchronous Single-Streamlczero: BLASnumpy: spacy: en_core_web_lgspacy: en_core_web_trfcaffe: AlexNet - CPU - 100caffe: AlexNet - CPU - 200caffe: AlexNet - CPU - 1000caffe: GoogleNet - CPU - 100caffe: GoogleNet - CPU - 200caffe: GoogleNet - CPU - 1000onednn: IP Shapes 1D - f32 - CPUonednn: IP Shapes 3D - f32 - CPUonednn: IP Shapes 1D - u8s8f32 - CPUonednn: IP Shapes 3D - u8s8f32 - CPUonednn: Convolution Batch Shapes Auto - f32 - CPUonednn: Deconvolution Batch shapes_1d - f32 - CPUonednn: Deconvolution Batch shapes_3d - f32 - CPUonednn: Convolution Batch Shapes Auto - u8s8f32 - CPUonednn: Deconvolution Batch shapes_1d - u8s8f32 - CPUonednn: Deconvolution Batch shapes_3d - u8s8f32 - CPUonednn: Recurrent Neural Network Training - f32 - CPUonednn: Recurrent Neural Network Inference - f32 - CPUonednn: Recurrent Neural Network Training - u8s8f32 - CPUonednn: Recurrent Neural Network Inference - u8s8f32 - CPUonednn: Matrix Multiply Batch Shapes Transformer - f32 - CPUonednn: Recurrent Neural Network Training - bf16bf16bf16 - CPUonednn: Recurrent Neural Network Inference - bf16bf16bf16 - CPUonednn: Matrix Multiply Batch Shapes Transformer - u8s8f32 - CPUmnn: nasnetmnn: mobilenetV3mnn: squeezenetv1.1mnn: resnet-v2-50mnn: SqueezeNetV1.0mnn: MobileNetV2_224mnn: mobilenet-v1-1.0mnn: inception-v3ncnn: CPU - mobilenetncnn: CPU-v2-v2 - mobilenet-v2ncnn: CPU-v3-v3 - mobilenet-v3ncnn: CPU - shufflenet-v2ncnn: CPU - efficientnet-b0ncnn: CPU - blazefacencnn: CPU - googlenetncnn: CPU - vgg16ncnn: CPU - resnet18ncnn: CPU - alexnetncnn: CPU - resnet50ncnn: CPU - yolov4-tinyncnn: CPU - squeezenet_ssdncnn: CPU - regnety_400mncnn: CPU - vision_transformerncnn: CPU - FastestDetncnn: CPU - mnasnetncnn: Vulkan GPU - mobilenetncnn: Vulkan GPU-v2-v2 - mobilenet-v2ncnn: Vulkan GPU-v3-v3 - mobilenet-v3ncnn: Vulkan GPU - mnasnetncnn: Vulkan GPU - efficientnet-b0ncnn: Vulkan GPU - blazefacencnn: Vulkan GPU - googlenetncnn: Vulkan GPU - vgg16ncnn: Vulkan GPU - resnet18ncnn: Vulkan GPU - alexnetncnn: Vulkan GPU - resnet50ncnn: Vulkan GPU - yolov4-tinyncnn: Vulkan GPU - squeezenet_ssdncnn: Vulkan GPU - regnety_400mncnn: Vulkan GPU - vision_transformerncnn: Vulkan GPU - FastestDetncnn: Vulkan GPU - shufflenet-v2tnn: CPU - DenseNettnn: CPU - MobileNet v2tnn: CPU - SqueezeNet v2tnn: CPU - SqueezeNet v1.1openvino: Face Detection FP16 - CPUopenvino: Person Detection FP16 - CPUopenvino: Person Detection FP32 - CPUopenvino: Vehicle Detection FP16 - CPUopenvino: Face Detection FP16-INT8 - CPUopenvino: Vehicle Detection FP16-INT8 - CPUopenvino: Weld Porosity Detection FP16 - CPUopenvino: Machine Translation EN To DE FP16 - CPUopenvino: Weld Porosity Detection FP16-INT8 - CPUopenvino: Person Vehicle Bike Detection FP16 - CPUopenvino: Age Gender Recognition Retail 0013 FP16 - CPUopenvino: Age Gender Recognition Retail 0013 FP16-INT8 - CPUopencv: DNN - Deep Neural Networkdeepsparse: NLP Document Classification, oBERT base uncased on IMDB - Asynchronous Multi-Streamdeepsparse: NLP Document Classification, oBERT base uncased on IMDB - Synchronous Single-Streamdeepsparse: NLP Sentiment Analysis, 80% Pruned Quantized BERT Base Uncased - Asynchronous Multi-Streamdeepsparse: NLP Sentiment Analysis, 80% Pruned Quantized BERT Base Uncased - Synchronous Single-Streamdeepsparse: NLP Question Answering, BERT base uncased SQuaD 12layer Pruned90 - Asynchronous Multi-Streamdeepsparse: NLP Question Answering, BERT base uncased SQuaD 12layer Pruned90 - Synchronous Single-Streamdeepsparse: CV Detection, YOLOv5s COCO - Asynchronous Multi-Streamdeepsparse: CV Detection, YOLOv5s COCO - Synchronous Single-Streamdeepsparse: CV Classification, ResNet-50 ImageNet - Asynchronous Multi-Streamdeepsparse: CV Classification, ResNet-50 ImageNet - Synchronous Single-Streamdeepsparse: NLP Text Classification, DistilBERT mnli - Asynchronous Multi-Streamdeepsparse: NLP Text Classification, DistilBERT mnli - Synchronous Single-Streamdeepsparse: CV Segmentation, 90% Pruned YOLACT Pruned - Asynchronous Multi-Streamdeepsparse: CV Segmentation, 90% Pruned YOLACT Pruned - Synchronous Single-Streamdeepsparse: NLP Text Classification, BERT base uncased SST2 - Asynchronous Multi-Streamdeepsparse: NLP Text Classification, BERT base uncased SST2 - Synchronous Single-Streamdeepsparse: NLP Token Classification, BERT base uncased conll2003 - Asynchronous Multi-Streamdeepsparse: NLP Token Classification, BERT base uncased conll2003 - Synchronous Single-Streamdeepspeech: CPUrbenchmark: rnnoise: numenta-nab: KNN CADnumenta-nab: Windowed Gaussiannumenta-nab: Earthgecko Skylinenumenta-nab: Contextual Anomaly Detector OSEmlpack: scikit_icamlpack: scikit_qdamlpack: scikit_svmmlpack: scikit_linearridgeregressionscikit-learn: Sparse Rand Projections, 100 IterationsASPEED - AMD Ryzen Threadripper PRO 3955WX 16-Cores32.096.975.123.423.38440.865.88495.20485.0253.70582.32694.0314801.7316201.4012.6200309.97413.893914.1144447.313286.9802417.735888.79807877316.836914.074610.3485158.888165.707155.599626.613479.173750.2138169.1345109.9966113.608176.365918.172114.601157.592538.963414.069910.32841237428.191273711836094212145060328415151530246215070372.605153.789931.327300.4658605.098644.068263.610815.931091.885252.542832284.141408.742287.001398.040.8937742287.071399.200.88857317.2502.7694.70924.7397.3754.9853.33927.33914.275.665.036.387.772.7713.4728.088.996.4316.7020.6615.3719.57149.358.004.988.542.463.272.748.711.815.954.792.741.985.4510.095.794.67166.513.112.392551.153255.67364.261233.1701548.992312.142329.1418.131353.7616.1516.48148.8827.4611.511.070.987037566.954296.623450.294615.2098143.657437.5650100.905619.901247.24729.081370.342513.0867439.425668.4716138.732625.6567568.026296.812457.592290.201217.873118.7115.01973.57943.18150.8637.3619.961.51159.005OpenBenchmarking.org

PlaidML

FP16: No - Mode: Inference - Network: VGG16 - Device: CPU

OpenBenchmarking.orgFPS, More Is BetterPlaidMLFP16: No - Mode: Inference - Network: VGG16 - Device: CPUASPEED - AMD Ryzen Threadripper PRO 3955WX 16-Cores714212835SE +/- 0.10, N = 332.09

PlaidML

FP16: No - Mode: Inference - Network: ResNet 50 - Device: CPU

OpenBenchmarking.orgFPS, More Is BetterPlaidMLFP16: No - Mode: Inference - Network: ResNet 50 - Device: CPUASPEED - AMD Ryzen Threadripper PRO 3955WX 16-Cores246810SE +/- 0.00, N = 36.97

OpenVINO

Model: Face Detection FP16 - Device: CPU

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2022.3Model: Face Detection FP16 - Device: CPUASPEED - AMD Ryzen Threadripper PRO 3955WX 16-Cores1.1522.3043.4564.6085.76SE +/- 0.03, N = 35.121. (CXX) g++ options: -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -pie -pthread -ldl

OpenVINO

Model: Person Detection FP16 - Device: CPU

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2022.3Model: Person Detection FP16 - Device: CPUASPEED - AMD Ryzen Threadripper PRO 3955WX 16-Cores0.76951.5392.30853.0783.8475SE +/- 0.01, N = 33.421. (CXX) g++ options: -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -pie -pthread -ldl

OpenVINO

Model: Person Detection FP32 - Device: CPU

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2022.3Model: Person Detection FP32 - Device: CPUASPEED - AMD Ryzen Threadripper PRO 3955WX 16-Cores0.76051.5212.28153.0423.8025SE +/- 0.01, N = 33.381. (CXX) g++ options: -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -pie -pthread -ldl

OpenVINO

Model: Vehicle Detection FP16 - Device: CPU

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2022.3Model: Vehicle Detection FP16 - Device: CPUASPEED - AMD Ryzen Threadripper PRO 3955WX 16-Cores100200300400500SE +/- 0.83, N = 3440.861. (CXX) g++ options: -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -pie -pthread -ldl

OpenVINO

Model: Face Detection FP16-INT8 - Device: CPU

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2022.3Model: Face Detection FP16-INT8 - Device: CPUASPEED - AMD Ryzen Threadripper PRO 3955WX 16-Cores1.3232.6463.9695.2926.615SE +/- 0.00, N = 35.881. (CXX) g++ options: -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -pie -pthread -ldl

OpenVINO

Model: Vehicle Detection FP16-INT8 - Device: CPU

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2022.3Model: Vehicle Detection FP16-INT8 - Device: CPUASPEED - AMD Ryzen Threadripper PRO 3955WX 16-Cores110220330440550SE +/- 1.15, N = 3495.201. (CXX) g++ options: -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -pie -pthread -ldl

OpenVINO

Model: Weld Porosity Detection FP16 - Device: CPU

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2022.3Model: Weld Porosity Detection FP16 - Device: CPUASPEED - AMD Ryzen Threadripper PRO 3955WX 16-Cores100200300400500SE +/- 1.25, N = 3485.021. (CXX) g++ options: -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -pie -pthread -ldl

OpenVINO

Model: Machine Translation EN To DE FP16 - Device: CPU

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2022.3Model: Machine Translation EN To DE FP16 - Device: CPUASPEED - AMD Ryzen Threadripper PRO 3955WX 16-Cores1224364860SE +/- 0.16, N = 353.701. (CXX) g++ options: -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -pie -pthread -ldl

OpenVINO

Model: Weld Porosity Detection FP16-INT8 - Device: CPU

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2022.3Model: Weld Porosity Detection FP16-INT8 - Device: CPUASPEED - AMD Ryzen Threadripper PRO 3955WX 16-Cores130260390520650SE +/- 0.36, N = 3582.321. (CXX) g++ options: -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -pie -pthread -ldl

OpenVINO

Model: Person Vehicle Bike Detection FP16 - Device: CPU

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2022.3Model: Person Vehicle Bike Detection FP16 - Device: CPUASPEED - AMD Ryzen Threadripper PRO 3955WX 16-Cores150300450600750SE +/- 1.05, N = 3694.031. (CXX) g++ options: -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -pie -pthread -ldl

OpenVINO

Model: Age Gender Recognition Retail 0013 FP16 - Device: CPU

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2022.3Model: Age Gender Recognition Retail 0013 FP16 - Device: CPUASPEED - AMD Ryzen Threadripper PRO 3955WX 16-Cores3K6K9K12K15KSE +/- 86.33, N = 314801.731. (CXX) g++ options: -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -pie -pthread -ldl

OpenVINO

Model: Age Gender Recognition Retail 0013 FP16-INT8 - Device: CPU

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2022.3Model: Age Gender Recognition Retail 0013 FP16-INT8 - Device: CPUASPEED - AMD Ryzen Threadripper PRO 3955WX 16-Cores3K6K9K12K15KSE +/- 41.51, N = 316201.401. (CXX) g++ options: -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -pie -pthread -ldl

SHOC Scalable HeterOgeneous Computing

Target: OpenCL - Benchmark: Triad

OpenBenchmarking.orgGB/s, More Is BetterSHOC Scalable HeterOgeneous Computing 2020-04-17Target: OpenCL - Benchmark: TriadASPEED - AMD Ryzen Threadripper PRO 3955WX 16-Cores3691215SE +/- 0.03, N = 312.621. (CXX) g++ options: -O2 -lSHOCCommonMPI -lSHOCCommonOpenCL -lSHOCCommon -lOpenCL -lrt -pthread -lmpi_cxx -lmpi

SHOC Scalable HeterOgeneous Computing

Target: OpenCL - Benchmark: Reduction

OpenBenchmarking.orgGB/s, More Is BetterSHOC Scalable HeterOgeneous Computing 2020-04-17Target: OpenCL - Benchmark: ReductionASPEED - AMD Ryzen Threadripper PRO 3955WX 16-Cores70140210280350SE +/- 0.60, N = 3309.971. (CXX) g++ options: -O2 -lSHOCCommonMPI -lSHOCCommonOpenCL -lSHOCCommon -lOpenCL -lrt -pthread -lmpi_cxx -lmpi

SHOC Scalable HeterOgeneous Computing

Target: OpenCL - Benchmark: Bus Speed Download

OpenBenchmarking.orgGB/s, More Is BetterSHOC Scalable HeterOgeneous Computing 2020-04-17Target: OpenCL - Benchmark: Bus Speed DownloadASPEED - AMD Ryzen Threadripper PRO 3955WX 16-Cores48121620SE +/- 0.00, N = 313.891. (CXX) g++ options: -O2 -lSHOCCommonMPI -lSHOCCommonOpenCL -lSHOCCommon -lOpenCL -lrt -pthread -lmpi_cxx -lmpi

SHOC Scalable HeterOgeneous Computing

Target: OpenCL - Benchmark: Bus Speed Readback

OpenBenchmarking.orgGB/s, More Is BetterSHOC Scalable HeterOgeneous Computing 2020-04-17Target: OpenCL - Benchmark: Bus Speed ReadbackASPEED - AMD Ryzen Threadripper PRO 3955WX 16-Cores48121620SE +/- 0.00, N = 314.111. (CXX) g++ options: -O2 -lSHOCCommonMPI -lSHOCCommonOpenCL -lSHOCCommon -lOpenCL -lrt -pthread -lmpi_cxx -lmpi

SHOC Scalable HeterOgeneous Computing

Target: OpenCL - Benchmark: Texture Read Bandwidth

OpenBenchmarking.orgGB/s, More Is BetterSHOC Scalable HeterOgeneous Computing 2020-04-17Target: OpenCL - Benchmark: Texture Read BandwidthASPEED - AMD Ryzen Threadripper PRO 3955WX 16-Cores100200300400500SE +/- 2.64, N = 3447.311. (CXX) g++ options: -O2 -lSHOCCommonMPI -lSHOCCommonOpenCL -lSHOCCommon -lOpenCL -lrt -pthread -lmpi_cxx -lmpi

SHOC Scalable HeterOgeneous Computing

Target: OpenCL - Benchmark: S3D

OpenBenchmarking.orgGFLOPS, More Is BetterSHOC Scalable HeterOgeneous Computing 2020-04-17Target: OpenCL - Benchmark: S3DASPEED - AMD Ryzen Threadripper PRO 3955WX 16-Cores60120180240300SE +/- 0.29, N = 3286.981. (CXX) g++ options: -O2 -lSHOCCommonMPI -lSHOCCommonOpenCL -lSHOCCommon -lOpenCL -lrt -pthread -lmpi_cxx -lmpi

SHOC Scalable HeterOgeneous Computing

Target: OpenCL - Benchmark: FFT SP

OpenBenchmarking.orgGFLOPS, More Is BetterSHOC Scalable HeterOgeneous Computing 2020-04-17Target: OpenCL - Benchmark: FFT SPASPEED - AMD Ryzen Threadripper PRO 3955WX 16-Cores5001000150020002500SE +/- 0.71, N = 32417.731. (CXX) g++ options: -O2 -lSHOCCommonMPI -lSHOCCommonOpenCL -lSHOCCommon -lOpenCL -lrt -pthread -lmpi_cxx -lmpi

SHOC Scalable HeterOgeneous Computing

Target: OpenCL - Benchmark: GEMM SGEMM_N

OpenBenchmarking.orgGFLOPS, More Is BetterSHOC Scalable HeterOgeneous Computing 2020-04-17Target: OpenCL - Benchmark: GEMM SGEMM_NASPEED - AMD Ryzen Threadripper PRO 3955WX 16-Cores13002600390052006500SE +/- 4.69, N = 35888.791. (CXX) g++ options: -O2 -lSHOCCommonMPI -lSHOCCommonOpenCL -lSHOCCommon -lOpenCL -lrt -pthread -lmpi_cxx -lmpi

SHOC Scalable HeterOgeneous Computing

Target: OpenCL - Benchmark: Max SP Flops

OpenBenchmarking.orgGFLOPS, More Is BetterSHOC Scalable HeterOgeneous Computing 2020-04-17Target: OpenCL - Benchmark: Max SP FlopsASPEED - AMD Ryzen Threadripper PRO 3955WX 16-Cores2M4M6M8M10MSE +/- 61430.57, N = 380787731. (CXX) g++ options: -O2 -lSHOCCommonMPI -lSHOCCommonOpenCL -lSHOCCommon -lOpenCL -lrt -pthread -lmpi_cxx -lmpi

SHOC Scalable HeterOgeneous Computing

Target: OpenCL - Benchmark: MD5 Hash

OpenBenchmarking.orgGHash/s, More Is BetterSHOC Scalable HeterOgeneous Computing 2020-04-17Target: OpenCL - Benchmark: MD5 HashASPEED - AMD Ryzen Threadripper PRO 3955WX 16-Cores48121620SE +/- 0.00, N = 316.841. (CXX) g++ options: -O2 -lSHOCCommonMPI -lSHOCCommonOpenCL -lSHOCCommon -lOpenCL -lrt -pthread -lmpi_cxx -lmpi

Neural Magic DeepSparse

Model: NLP Document Classification, oBERT base uncased on IMDB - Scenario: Asynchronous Multi-Stream

OpenBenchmarking.orgitems/sec, More Is BetterNeural Magic DeepSparse 1.3.2Model: NLP Document Classification, oBERT base uncased on IMDB - Scenario: Asynchronous Multi-StreamASPEED - AMD Ryzen Threadripper PRO 3955WX 16-Cores48121620SE +/- 0.00, N = 314.07

Neural Magic DeepSparse

Model: NLP Document Classification, oBERT base uncased on IMDB - Scenario: Synchronous Single-Stream

OpenBenchmarking.orgitems/sec, More Is BetterNeural Magic DeepSparse 1.3.2Model: NLP Document Classification, oBERT base uncased on IMDB - Scenario: Synchronous Single-StreamASPEED - AMD Ryzen Threadripper PRO 3955WX 16-Cores3691215SE +/- 0.00, N = 310.35

Neural Magic DeepSparse

Model: NLP Sentiment Analysis, 80% Pruned Quantized BERT Base Uncased - Scenario: Asynchronous Multi-Stream

OpenBenchmarking.orgitems/sec, More Is BetterNeural Magic DeepSparse 1.3.2Model: NLP Sentiment Analysis, 80% Pruned Quantized BERT Base Uncased - Scenario: Asynchronous Multi-StreamASPEED - AMD Ryzen Threadripper PRO 3955WX 16-Cores4080120160200SE +/- 0.15, N = 3158.89

Neural Magic DeepSparse

Model: NLP Sentiment Analysis, 80% Pruned Quantized BERT Base Uncased - Scenario: Synchronous Single-Stream

OpenBenchmarking.orgitems/sec, More Is BetterNeural Magic DeepSparse 1.3.2Model: NLP Sentiment Analysis, 80% Pruned Quantized BERT Base Uncased - Scenario: Synchronous Single-StreamASPEED - AMD Ryzen Threadripper PRO 3955WX 16-Cores1530456075SE +/- 0.22, N = 365.71

Neural Magic DeepSparse

Model: NLP Question Answering, BERT base uncased SQuaD 12layer Pruned90 - Scenario: Asynchronous Multi-Stream

OpenBenchmarking.orgitems/sec, More Is BetterNeural Magic DeepSparse 1.3.2Model: NLP Question Answering, BERT base uncased SQuaD 12layer Pruned90 - Scenario: Asynchronous Multi-StreamASPEED - AMD Ryzen Threadripper PRO 3955WX 16-Cores1224364860SE +/- 0.21, N = 355.60

Neural Magic DeepSparse

Model: NLP Question Answering, BERT base uncased SQuaD 12layer Pruned90 - Scenario: Synchronous Single-Stream

OpenBenchmarking.orgitems/sec, More Is BetterNeural Magic DeepSparse 1.3.2Model: NLP Question Answering, BERT base uncased SQuaD 12layer Pruned90 - Scenario: Synchronous Single-StreamASPEED - AMD Ryzen Threadripper PRO 3955WX 16-Cores612182430SE +/- 0.09, N = 326.61

Neural Magic DeepSparse

Model: CV Detection, YOLOv5s COCO - Scenario: Asynchronous Multi-Stream

OpenBenchmarking.orgitems/sec, More Is BetterNeural Magic DeepSparse 1.3.2Model: CV Detection, YOLOv5s COCO - Scenario: Asynchronous Multi-StreamASPEED - AMD Ryzen Threadripper PRO 3955WX 16-Cores20406080100SE +/- 0.16, N = 379.17

Neural Magic DeepSparse

Model: CV Detection, YOLOv5s COCO - Scenario: Synchronous Single-Stream

OpenBenchmarking.orgitems/sec, More Is BetterNeural Magic DeepSparse 1.3.2Model: CV Detection, YOLOv5s COCO - Scenario: Synchronous Single-StreamASPEED - AMD Ryzen Threadripper PRO 3955WX 16-Cores1122334455SE +/- 0.09, N = 350.21

Neural Magic DeepSparse

Model: CV Classification, ResNet-50 ImageNet - Scenario: Asynchronous Multi-Stream

OpenBenchmarking.orgitems/sec, More Is BetterNeural Magic DeepSparse 1.3.2Model: CV Classification, ResNet-50 ImageNet - Scenario: Asynchronous Multi-StreamASPEED - AMD Ryzen Threadripper PRO 3955WX 16-Cores4080120160200SE +/- 0.13, N = 3169.13

Neural Magic DeepSparse

Model: CV Classification, ResNet-50 ImageNet - Scenario: Synchronous Single-Stream

OpenBenchmarking.orgitems/sec, More Is BetterNeural Magic DeepSparse 1.3.2Model: CV Classification, ResNet-50 ImageNet - Scenario: Synchronous Single-StreamASPEED - AMD Ryzen Threadripper PRO 3955WX 16-Cores20406080100SE +/- 0.36, N = 3110.00

Neural Magic DeepSparse

Model: NLP Text Classification, DistilBERT mnli - Scenario: Asynchronous Multi-Stream

OpenBenchmarking.orgitems/sec, More Is BetterNeural Magic DeepSparse 1.3.2Model: NLP Text Classification, DistilBERT mnli - Scenario: Asynchronous Multi-StreamASPEED - AMD Ryzen Threadripper PRO 3955WX 16-Cores306090120150SE +/- 0.08, N = 3113.61

Neural Magic DeepSparse

Model: NLP Text Classification, DistilBERT mnli - Scenario: Synchronous Single-Stream

OpenBenchmarking.orgitems/sec, More Is BetterNeural Magic DeepSparse 1.3.2Model: NLP Text Classification, DistilBERT mnli - Scenario: Synchronous Single-StreamASPEED - AMD Ryzen Threadripper PRO 3955WX 16-Cores20406080100SE +/- 0.17, N = 376.37

Neural Magic DeepSparse

Model: CV Segmentation, 90% Pruned YOLACT Pruned - Scenario: Asynchronous Multi-Stream

OpenBenchmarking.orgitems/sec, More Is BetterNeural Magic DeepSparse 1.3.2Model: CV Segmentation, 90% Pruned YOLACT Pruned - Scenario: Asynchronous Multi-StreamASPEED - AMD Ryzen Threadripper PRO 3955WX 16-Cores48121620SE +/- 0.01, N = 318.17

Neural Magic DeepSparse

Model: CV Segmentation, 90% Pruned YOLACT Pruned - Scenario: Synchronous Single-Stream

OpenBenchmarking.orgitems/sec, More Is BetterNeural Magic DeepSparse 1.3.2Model: CV Segmentation, 90% Pruned YOLACT Pruned - Scenario: Synchronous Single-StreamASPEED - AMD Ryzen Threadripper PRO 3955WX 16-Cores48121620SE +/- 0.02, N = 314.60

Neural Magic DeepSparse

Model: NLP Text Classification, BERT base uncased SST2 - Scenario: Asynchronous Multi-Stream

OpenBenchmarking.orgitems/sec, More Is BetterNeural Magic DeepSparse 1.3.2Model: NLP Text Classification, BERT base uncased SST2 - Scenario: Asynchronous Multi-StreamASPEED - AMD Ryzen Threadripper PRO 3955WX 16-Cores1326395265SE +/- 0.06, N = 357.59

Neural Magic DeepSparse

Model: NLP Text Classification, BERT base uncased SST2 - Scenario: Synchronous Single-Stream

OpenBenchmarking.orgitems/sec, More Is BetterNeural Magic DeepSparse 1.3.2Model: NLP Text Classification, BERT base uncased SST2 - Scenario: Synchronous Single-StreamASPEED - AMD Ryzen Threadripper PRO 3955WX 16-Cores918273645SE +/- 0.12, N = 338.96

Neural Magic DeepSparse

Model: NLP Token Classification, BERT base uncased conll2003 - Scenario: Asynchronous Multi-Stream

OpenBenchmarking.orgitems/sec, More Is BetterNeural Magic DeepSparse 1.3.2Model: NLP Token Classification, BERT base uncased conll2003 - Scenario: Asynchronous Multi-StreamASPEED - AMD Ryzen Threadripper PRO 3955WX 16-Cores48121620SE +/- 0.01, N = 314.07

Neural Magic DeepSparse

Model: NLP Token Classification, BERT base uncased conll2003 - Scenario: Synchronous Single-Stream

OpenBenchmarking.orgitems/sec, More Is BetterNeural Magic DeepSparse 1.3.2Model: NLP Token Classification, BERT base uncased conll2003 - Scenario: Synchronous Single-StreamASPEED - AMD Ryzen Threadripper PRO 3955WX 16-Cores3691215SE +/- 0.01, N = 310.33

LeelaChessZero

Backend: BLAS

OpenBenchmarking.orgNodes Per Second, More Is BetterLeelaChessZero 0.28Backend: BLASASPEED - AMD Ryzen Threadripper PRO 3955WX 16-Cores30060090012001500SE +/- 8.45, N = 312371. (CXX) g++ options: -flto -pthread

Numpy Benchmark

OpenBenchmarking.orgScore, More Is BetterNumpy BenchmarkASPEED - AMD Ryzen Threadripper PRO 3955WX 16-Cores90180270360450SE +/- 0.13, N = 3428.19

spaCy

Model: en_core_web_lg

OpenBenchmarking.orgtokens/sec, More Is BetterspaCy 3.4.1Model: en_core_web_lgASPEED - AMD Ryzen Threadripper PRO 3955WX 16-Cores3K6K9K12K15KSE +/- 32.15, N = 312737

spaCy

Model: en_core_web_trf

OpenBenchmarking.orgtokens/sec, More Is BetterspaCy 3.4.1Model: en_core_web_trfASPEED - AMD Ryzen Threadripper PRO 3955WX 16-Cores30060090012001500SE +/- 3.79, N = 31183

Caffe

Model: AlexNet - Acceleration: CPU - Iterations: 100

OpenBenchmarking.orgMilli-Seconds, Fewer Is BetterCaffe 2020-02-13Model: AlexNet - Acceleration: CPU - Iterations: 100ASPEED - AMD Ryzen Threadripper PRO 3955WX 16-Cores13K26K39K52K65KSE +/- 142.99, N = 3609421. (CXX) g++ options: -fPIC -O3 -rdynamic -lglog -lgflags -lprotobuf -lpthread -lsz -lz -ldl -lm -llmdb -lopenblas

Caffe

Model: AlexNet - Acceleration: CPU - Iterations: 200

OpenBenchmarking.orgMilli-Seconds, Fewer Is BetterCaffe 2020-02-13Model: AlexNet - Acceleration: CPU - Iterations: 200ASPEED - AMD Ryzen Threadripper PRO 3955WX 16-Cores30K60K90K120K150KSE +/- 323.13, N = 31214501. (CXX) g++ options: -fPIC -O3 -rdynamic -lglog -lgflags -lprotobuf -lpthread -lsz -lz -ldl -lm -llmdb -lopenblas

Caffe

Model: AlexNet - Acceleration: CPU - Iterations: 1000

OpenBenchmarking.orgMilli-Seconds, Fewer Is BetterCaffe 2020-02-13Model: AlexNet - Acceleration: CPU - Iterations: 1000ASPEED - AMD Ryzen Threadripper PRO 3955WX 16-Cores130K260K390K520K650KSE +/- 1629.47, N = 36032841. (CXX) g++ options: -fPIC -O3 -rdynamic -lglog -lgflags -lprotobuf -lpthread -lsz -lz -ldl -lm -llmdb -lopenblas

Caffe

Model: GoogleNet - Acceleration: CPU - Iterations: 100

OpenBenchmarking.orgMilli-Seconds, Fewer Is BetterCaffe 2020-02-13Model: GoogleNet - Acceleration: CPU - Iterations: 100ASPEED - AMD Ryzen Threadripper PRO 3955WX 16-Cores30K60K90K120K150KSE +/- 206.25, N = 31515151. (CXX) g++ options: -fPIC -O3 -rdynamic -lglog -lgflags -lprotobuf -lpthread -lsz -lz -ldl -lm -llmdb -lopenblas

Caffe

Model: GoogleNet - Acceleration: CPU - Iterations: 200

OpenBenchmarking.orgMilli-Seconds, Fewer Is BetterCaffe 2020-02-13Model: GoogleNet - Acceleration: CPU - Iterations: 200ASPEED - AMD Ryzen Threadripper PRO 3955WX 16-Cores60K120K180K240K300KSE +/- 159.69, N = 33024621. (CXX) g++ options: -fPIC -O3 -rdynamic -lglog -lgflags -lprotobuf -lpthread -lsz -lz -ldl -lm -llmdb -lopenblas

Caffe

Model: GoogleNet - Acceleration: CPU - Iterations: 1000

OpenBenchmarking.orgMilli-Seconds, Fewer Is BetterCaffe 2020-02-13Model: GoogleNet - Acceleration: CPU - Iterations: 1000ASPEED - AMD Ryzen Threadripper PRO 3955WX 16-Cores300K600K900K1200K1500KSE +/- 1946.22, N = 315070371. (CXX) g++ options: -fPIC -O3 -rdynamic -lglog -lgflags -lprotobuf -lpthread -lsz -lz -ldl -lm -llmdb -lopenblas

oneDNN

Harness: IP Shapes 1D - Data Type: f32 - Engine: CPU

OpenBenchmarking.orgms, Fewer Is BetteroneDNN 3.0Harness: IP Shapes 1D - Data Type: f32 - Engine: CPUASPEED - AMD Ryzen Threadripper PRO 3955WX 16-Cores0.58621.17241.75862.34482.931SE +/- 0.01318, N = 32.60515MIN: 2.31. (CXX) g++ options: -O3 -march=native -fopenmp -msse4.1 -fPIC -pie -lpthread -ldl

oneDNN

Harness: IP Shapes 3D - Data Type: f32 - Engine: CPU

OpenBenchmarking.orgms, Fewer Is BetteroneDNN 3.0Harness: IP Shapes 3D - Data Type: f32 - Engine: CPUASPEED - AMD Ryzen Threadripper PRO 3955WX 16-Cores0.85271.70542.55813.41084.2635SE +/- 0.00629, N = 33.78993MIN: 3.641. (CXX) g++ options: -O3 -march=native -fopenmp -msse4.1 -fPIC -pie -lpthread -ldl

oneDNN

Harness: IP Shapes 1D - Data Type: u8s8f32 - Engine: CPU

OpenBenchmarking.orgms, Fewer Is BetteroneDNN 3.0Harness: IP Shapes 1D - Data Type: u8s8f32 - Engine: CPUASPEED - AMD Ryzen Threadripper PRO 3955WX 16-Cores0.29860.59720.89581.19441.493SE +/- 0.00076, N = 31.32730MIN: 1.291. (CXX) g++ options: -O3 -march=native -fopenmp -msse4.1 -fPIC -pie -lpthread -ldl

oneDNN

Harness: IP Shapes 3D - Data Type: u8s8f32 - Engine: CPU

OpenBenchmarking.orgms, Fewer Is BetteroneDNN 3.0Harness: IP Shapes 3D - Data Type: u8s8f32 - Engine: CPUASPEED - AMD Ryzen Threadripper PRO 3955WX 16-Cores0.10480.20960.31440.41920.524SE +/- 0.000464, N = 30.465860MIN: 0.431. (CXX) g++ options: -O3 -march=native -fopenmp -msse4.1 -fPIC -pie -lpthread -ldl

oneDNN

Harness: Convolution Batch Shapes Auto - Data Type: f32 - Engine: CPU

OpenBenchmarking.orgms, Fewer Is BetteroneDNN 3.0Harness: Convolution Batch Shapes Auto - Data Type: f32 - Engine: CPUASPEED - AMD Ryzen Threadripper PRO 3955WX 16-Cores1.14722.29443.44164.58885.736SE +/- 0.00163, N = 35.09864MIN: 5.021. (CXX) g++ options: -O3 -march=native -fopenmp -msse4.1 -fPIC -pie -lpthread -ldl

oneDNN

Harness: Deconvolution Batch shapes_1d - Data Type: f32 - Engine: CPU

OpenBenchmarking.orgms, Fewer Is BetteroneDNN 3.0Harness: Deconvolution Batch shapes_1d - Data Type: f32 - Engine: CPUASPEED - AMD Ryzen Threadripper PRO 3955WX 16-Cores0.91541.83082.74623.66164.577SE +/- 0.03564, N = 154.06826MIN: 3.41. (CXX) g++ options: -O3 -march=native -fopenmp -msse4.1 -fPIC -pie -lpthread -ldl

oneDNN

Harness: Deconvolution Batch shapes_3d - Data Type: f32 - Engine: CPU

OpenBenchmarking.orgms, Fewer Is BetteroneDNN 3.0Harness: Deconvolution Batch shapes_3d - Data Type: f32 - Engine: CPUASPEED - AMD Ryzen Threadripper PRO 3955WX 16-Cores0.81241.62482.43723.24964.062SE +/- 0.01772, N = 33.61081MIN: 3.421. (CXX) g++ options: -O3 -march=native -fopenmp -msse4.1 -fPIC -pie -lpthread -ldl

oneDNN

Harness: Convolution Batch Shapes Auto - Data Type: u8s8f32 - Engine: CPU

OpenBenchmarking.orgms, Fewer Is BetteroneDNN 3.0Harness: Convolution Batch Shapes Auto - Data Type: u8s8f32 - Engine: CPUASPEED - AMD Ryzen Threadripper PRO 3955WX 16-Cores1.33452.6694.00355.3386.6725SE +/- 0.01049, N = 35.93109MIN: 5.81. (CXX) g++ options: -O3 -march=native -fopenmp -msse4.1 -fPIC -pie -lpthread -ldl

oneDNN

Harness: Deconvolution Batch shapes_1d - Data Type: u8s8f32 - Engine: CPU

OpenBenchmarking.orgms, Fewer Is BetteroneDNN 3.0Harness: Deconvolution Batch shapes_1d - Data Type: u8s8f32 - Engine: CPUASPEED - AMD Ryzen Threadripper PRO 3955WX 16-Cores0.42420.84841.27261.69682.121SE +/- 0.00284, N = 31.88525MIN: 1.811. (CXX) g++ options: -O3 -march=native -fopenmp -msse4.1 -fPIC -pie -lpthread -ldl

oneDNN

Harness: Deconvolution Batch shapes_3d - Data Type: u8s8f32 - Engine: CPU

OpenBenchmarking.orgms, Fewer Is BetteroneDNN 3.0Harness: Deconvolution Batch shapes_3d - Data Type: u8s8f32 - Engine: CPUASPEED - AMD Ryzen Threadripper PRO 3955WX 16-Cores0.57211.14421.71632.28842.8605SE +/- 0.00091, N = 32.54283MIN: 2.411. (CXX) g++ options: -O3 -march=native -fopenmp -msse4.1 -fPIC -pie -lpthread -ldl

oneDNN

Harness: Recurrent Neural Network Training - Data Type: f32 - Engine: CPU

OpenBenchmarking.orgms, Fewer Is BetteroneDNN 3.0Harness: Recurrent Neural Network Training - Data Type: f32 - Engine: CPUASPEED - AMD Ryzen Threadripper PRO 3955WX 16-Cores5001000150020002500SE +/- 6.20, N = 32284.14MIN: 2251.421. (CXX) g++ options: -O3 -march=native -fopenmp -msse4.1 -fPIC -pie -lpthread -ldl

oneDNN

Harness: Recurrent Neural Network Inference - Data Type: f32 - Engine: CPU

OpenBenchmarking.orgms, Fewer Is BetteroneDNN 3.0Harness: Recurrent Neural Network Inference - Data Type: f32 - Engine: CPUASPEED - AMD Ryzen Threadripper PRO 3955WX 16-Cores30060090012001500SE +/- 4.60, N = 31408.74MIN: 1385.131. (CXX) g++ options: -O3 -march=native -fopenmp -msse4.1 -fPIC -pie -lpthread -ldl

oneDNN

Harness: Recurrent Neural Network Training - Data Type: u8s8f32 - Engine: CPU

OpenBenchmarking.orgms, Fewer Is BetteroneDNN 3.0Harness: Recurrent Neural Network Training - Data Type: u8s8f32 - Engine: CPUASPEED - AMD Ryzen Threadripper PRO 3955WX 16-Cores5001000150020002500SE +/- 3.22, N = 32287.00MIN: 2262.471. (CXX) g++ options: -O3 -march=native -fopenmp -msse4.1 -fPIC -pie -lpthread -ldl

oneDNN

Harness: Recurrent Neural Network Inference - Data Type: u8s8f32 - Engine: CPU

OpenBenchmarking.orgms, Fewer Is BetteroneDNN 3.0Harness: Recurrent Neural Network Inference - Data Type: u8s8f32 - Engine: CPUASPEED - AMD Ryzen Threadripper PRO 3955WX 16-Cores30060090012001500SE +/- 6.53, N = 31398.04MIN: 1368.381. (CXX) g++ options: -O3 -march=native -fopenmp -msse4.1 -fPIC -pie -lpthread -ldl

oneDNN

Harness: Matrix Multiply Batch Shapes Transformer - Data Type: f32 - Engine: CPU

OpenBenchmarking.orgms, Fewer Is BetteroneDNN 3.0Harness: Matrix Multiply Batch Shapes Transformer - Data Type: f32 - Engine: CPUASPEED - AMD Ryzen Threadripper PRO 3955WX 16-Cores0.20110.40220.60330.80441.0055SE +/- 0.002397, N = 30.893774MIN: 0.791. (CXX) g++ options: -O3 -march=native -fopenmp -msse4.1 -fPIC -pie -lpthread -ldl

oneDNN

Harness: Recurrent Neural Network Training - Data Type: bf16bf16bf16 - Engine: CPU

OpenBenchmarking.orgms, Fewer Is BetteroneDNN 3.0Harness: Recurrent Neural Network Training - Data Type: bf16bf16bf16 - Engine: CPUASPEED - AMD Ryzen Threadripper PRO 3955WX 16-Cores5001000150020002500SE +/- 1.32, N = 32287.07MIN: 2268.861. (CXX) g++ options: -O3 -march=native -fopenmp -msse4.1 -fPIC -pie -lpthread -ldl

oneDNN

Harness: Recurrent Neural Network Inference - Data Type: bf16bf16bf16 - Engine: CPU

OpenBenchmarking.orgms, Fewer Is BetteroneDNN 3.0Harness: Recurrent Neural Network Inference - Data Type: bf16bf16bf16 - Engine: CPUASPEED - AMD Ryzen Threadripper PRO 3955WX 16-Cores30060090012001500SE +/- 2.96, N = 31399.20MIN: 1380.771. (CXX) g++ options: -O3 -march=native -fopenmp -msse4.1 -fPIC -pie -lpthread -ldl

oneDNN

Harness: Matrix Multiply Batch Shapes Transformer - Data Type: u8s8f32 - Engine: CPU

OpenBenchmarking.orgms, Fewer Is BetteroneDNN 3.0Harness: Matrix Multiply Batch Shapes Transformer - Data Type: u8s8f32 - Engine: CPUASPEED - AMD Ryzen Threadripper PRO 3955WX 16-Cores0.19990.39980.59970.79960.9995SE +/- 0.001776, N = 30.888573MIN: 0.791. (CXX) g++ options: -O3 -march=native -fopenmp -msse4.1 -fPIC -pie -lpthread -ldl

Mobile Neural Network

Model: nasnet

OpenBenchmarking.orgms, Fewer Is BetterMobile Neural Network 2.1Model: nasnetASPEED - AMD Ryzen Threadripper PRO 3955WX 16-Cores48121620SE +/- 0.13, N = 1017.25MIN: 16.25 / MAX: 291. (CXX) g++ options: -std=c++11 -O3 -fvisibility=hidden -fomit-frame-pointer -fstrict-aliasing -ffunction-sections -fdata-sections -ffast-math -fno-rtti -fno-exceptions -rdynamic -pthread -ldl

Mobile Neural Network

Model: mobilenetV3

OpenBenchmarking.orgms, Fewer Is BetterMobile Neural Network 2.1Model: mobilenetV3ASPEED - AMD Ryzen Threadripper PRO 3955WX 16-Cores0.6231.2461.8692.4923.115SE +/- 0.019, N = 102.769MIN: 2.58 / MAX: 12.751. (CXX) g++ options: -std=c++11 -O3 -fvisibility=hidden -fomit-frame-pointer -fstrict-aliasing -ffunction-sections -fdata-sections -ffast-math -fno-rtti -fno-exceptions -rdynamic -pthread -ldl

Mobile Neural Network

Model: squeezenetv1.1

OpenBenchmarking.orgms, Fewer Is BetterMobile Neural Network 2.1Model: squeezenetv1.1ASPEED - AMD Ryzen Threadripper PRO 3955WX 16-Cores1.05952.1193.17854.2385.2975SE +/- 0.078, N = 104.709MIN: 4.09 / MAX: 14.591. (CXX) g++ options: -std=c++11 -O3 -fvisibility=hidden -fomit-frame-pointer -fstrict-aliasing -ffunction-sections -fdata-sections -ffast-math -fno-rtti -fno-exceptions -rdynamic -pthread -ldl

Mobile Neural Network

Model: resnet-v2-50

OpenBenchmarking.orgms, Fewer Is BetterMobile Neural Network 2.1Model: resnet-v2-50ASPEED - AMD Ryzen Threadripper PRO 3955WX 16-Cores612182430SE +/- 0.05, N = 1024.74MIN: 24.1 / MAX: 45.21. (CXX) g++ options: -std=c++11 -O3 -fvisibility=hidden -fomit-frame-pointer -fstrict-aliasing -ffunction-sections -fdata-sections -ffast-math -fno-rtti -fno-exceptions -rdynamic -pthread -ldl

Mobile Neural Network

Model: SqueezeNetV1.0

OpenBenchmarking.orgms, Fewer Is BetterMobile Neural Network 2.1Model: SqueezeNetV1.0ASPEED - AMD Ryzen Threadripper PRO 3955WX 16-Cores246810SE +/- 0.053, N = 107.375MIN: 6.6 / MAX: 18.711. (CXX) g++ options: -std=c++11 -O3 -fvisibility=hidden -fomit-frame-pointer -fstrict-aliasing -ffunction-sections -fdata-sections -ffast-math -fno-rtti -fno-exceptions -rdynamic -pthread -ldl

Mobile Neural Network

Model: MobileNetV2_224

OpenBenchmarking.orgms, Fewer Is BetterMobile Neural Network 2.1Model: MobileNetV2_224ASPEED - AMD Ryzen Threadripper PRO 3955WX 16-Cores1.12162.24323.36484.48645.608SE +/- 0.055, N = 104.985MIN: 4.47 / MAX: 15.171. (CXX) g++ options: -std=c++11 -O3 -fvisibility=hidden -fomit-frame-pointer -fstrict-aliasing -ffunction-sections -fdata-sections -ffast-math -fno-rtti -fno-exceptions -rdynamic -pthread -ldl

Mobile Neural Network

Model: mobilenet-v1-1.0

OpenBenchmarking.orgms, Fewer Is BetterMobile Neural Network 2.1Model: mobilenet-v1-1.0ASPEED - AMD Ryzen Threadripper PRO 3955WX 16-Cores0.75131.50262.25393.00523.7565SE +/- 0.020, N = 103.339MIN: 3.19 / MAX: 13.411. (CXX) g++ options: -std=c++11 -O3 -fvisibility=hidden -fomit-frame-pointer -fstrict-aliasing -ffunction-sections -fdata-sections -ffast-math -fno-rtti -fno-exceptions -rdynamic -pthread -ldl

Mobile Neural Network

Model: inception-v3

OpenBenchmarking.orgms, Fewer Is BetterMobile Neural Network 2.1Model: inception-v3ASPEED - AMD Ryzen Threadripper PRO 3955WX 16-Cores612182430SE +/- 0.05, N = 1027.34MIN: 26.3 / MAX: 39.961. (CXX) g++ options: -std=c++11 -O3 -fvisibility=hidden -fomit-frame-pointer -fstrict-aliasing -ffunction-sections -fdata-sections -ffast-math -fno-rtti -fno-exceptions -rdynamic -pthread -ldl

NCNN

Target: CPU - Model: mobilenet

OpenBenchmarking.orgms, Fewer Is BetterNCNN 20220729Target: CPU - Model: mobilenetASPEED - AMD Ryzen Threadripper PRO 3955WX 16-Cores48121620SE +/- 0.12, N = 814.27MIN: 13.56 / MAX: 86.271. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread -pthread

NCNN

Target: CPU-v2-v2 - Model: mobilenet-v2

OpenBenchmarking.orgms, Fewer Is BetterNCNN 20220729Target: CPU-v2-v2 - Model: mobilenet-v2ASPEED - AMD Ryzen Threadripper PRO 3955WX 16-Cores1.27352.5473.82055.0946.3675SE +/- 0.06, N = 85.66MIN: 5.34 / MAX: 101.471. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread -pthread

NCNN

Target: CPU-v3-v3 - Model: mobilenet-v3

OpenBenchmarking.orgms, Fewer Is BetterNCNN 20220729Target: CPU-v3-v3 - Model: mobilenet-v3ASPEED - AMD Ryzen Threadripper PRO 3955WX 16-Cores1.13182.26363.39544.52725.659SE +/- 0.05, N = 85.03MIN: 4.78 / MAX: 96.841. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread -pthread

NCNN

Target: CPU - Model: shufflenet-v2

OpenBenchmarking.orgms, Fewer Is BetterNCNN 20220729Target: CPU - Model: shufflenet-v2ASPEED - AMD Ryzen Threadripper PRO 3955WX 16-Cores246810SE +/- 0.04, N = 86.38MIN: 6.08 / MAX: 89.141. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread -pthread

NCNN

Target: CPU - Model: efficientnet-b0

OpenBenchmarking.orgms, Fewer Is BetterNCNN 20220729Target: CPU - Model: efficientnet-b0ASPEED - AMD Ryzen Threadripper PRO 3955WX 16-Cores246810SE +/- 0.05, N = 87.77MIN: 7.41 / MAX: 100.471. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread -pthread

NCNN

Target: CPU - Model: blazeface

OpenBenchmarking.orgms, Fewer Is BetterNCNN 20220729Target: CPU - Model: blazefaceASPEED - AMD Ryzen Threadripper PRO 3955WX 16-Cores0.62331.24661.86992.49323.1165SE +/- 0.01, N = 82.77MIN: 2.26 / MAX: 57.641. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread -pthread

NCNN

Target: CPU - Model: googlenet

OpenBenchmarking.orgms, Fewer Is BetterNCNN 20220729Target: CPU - Model: googlenetASPEED - AMD Ryzen Threadripper PRO 3955WX 16-Cores3691215SE +/- 0.07, N = 813.47MIN: 12.67 / MAX: 93.341. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread -pthread

NCNN

Target: CPU - Model: vgg16

OpenBenchmarking.orgms, Fewer Is BetterNCNN 20220729Target: CPU - Model: vgg16ASPEED - AMD Ryzen Threadripper PRO 3955WX 16-Cores714212835SE +/- 0.23, N = 828.08MIN: 26.45 / MAX: 128.511. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread -pthread

NCNN

Target: CPU - Model: resnet18

OpenBenchmarking.orgms, Fewer Is BetterNCNN 20220729Target: CPU - Model: resnet18ASPEED - AMD Ryzen Threadripper PRO 3955WX 16-Cores3691215SE +/- 0.07, N = 88.99MIN: 8.66 / MAX: 65.81. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread -pthread

NCNN

Target: CPU - Model: alexnet

OpenBenchmarking.orgms, Fewer Is BetterNCNN 20220729Target: CPU - Model: alexnetASPEED - AMD Ryzen Threadripper PRO 3955WX 16-Cores246810SE +/- 0.02, N = 86.43MIN: 6.12 / MAX: 69.881. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread -pthread

NCNN

Target: CPU - Model: resnet50

OpenBenchmarking.orgms, Fewer Is BetterNCNN 20220729Target: CPU - Model: resnet50ASPEED - AMD Ryzen Threadripper PRO 3955WX 16-Cores48121620SE +/- 0.09, N = 816.70MIN: 16.11 / MAX: 83.221. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread -pthread

NCNN

Target: CPU - Model: yolov4-tiny

OpenBenchmarking.orgms, Fewer Is BetterNCNN 20220729Target: CPU - Model: yolov4-tinyASPEED - AMD Ryzen Threadripper PRO 3955WX 16-Cores510152025SE +/- 0.13, N = 820.66MIN: 19.76 / MAX: 123.681. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread -pthread

NCNN

Target: CPU - Model: squeezenet_ssd

OpenBenchmarking.orgms, Fewer Is BetterNCNN 20220729Target: CPU - Model: squeezenet_ssdASPEED - AMD Ryzen Threadripper PRO 3955WX 16-Cores48121620SE +/- 0.09, N = 815.37MIN: 14.62 / MAX: 86.951. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread -pthread

NCNN

Target: CPU - Model: regnety_400m

OpenBenchmarking.orgms, Fewer Is BetterNCNN 20220729Target: CPU - Model: regnety_400mASPEED - AMD Ryzen Threadripper PRO 3955WX 16-Cores510152025SE +/- 0.14, N = 819.57MIN: 18.75 / MAX: 103.271. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread -pthread

NCNN

Target: CPU - Model: vision_transformer

OpenBenchmarking.orgms, Fewer Is BetterNCNN 20220729Target: CPU - Model: vision_transformerASPEED - AMD Ryzen Threadripper PRO 3955WX 16-Cores306090120150SE +/- 0.17, N = 8149.35MIN: 147.12 / MAX: 186.181. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread -pthread

NCNN

Target: CPU - Model: FastestDet

OpenBenchmarking.orgms, Fewer Is BetterNCNN 20220729Target: CPU - Model: FastestDetASPEED - AMD Ryzen Threadripper PRO 3955WX 16-Cores246810SE +/- 0.07, N = 88.00MIN: 7.32 / MAX: 79.371. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread -pthread

NCNN

Target: CPU - Model: mnasnet

OpenBenchmarking.orgms, Fewer Is BetterNCNN 20220729Target: CPU - Model: mnasnetASPEED - AMD Ryzen Threadripper PRO 3955WX 16-Cores1.12052.2413.36154.4825.6025SE +/- 0.02, N = 74.98MIN: 4.8 / MAX: 36.861. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread -pthread

NCNN

Target: Vulkan GPU - Model: mobilenet

OpenBenchmarking.orgms, Fewer Is BetterNCNN 20220729Target: Vulkan GPU - Model: mobilenetASPEED - AMD Ryzen Threadripper PRO 3955WX 16-Cores246810SE +/- 0.02, N = 38.54MIN: 8.25 / MAX: 22.671. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread -pthread

NCNN

Target: Vulkan GPU-v2-v2 - Model: mobilenet-v2

OpenBenchmarking.orgms, Fewer Is BetterNCNN 20220729Target: Vulkan GPU-v2-v2 - Model: mobilenet-v2ASPEED - AMD Ryzen Threadripper PRO 3955WX 16-Cores0.55351.1071.66052.2142.7675SE +/- 0.03, N = 32.46MIN: 2.3 / MAX: 7.231. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread -pthread

NCNN

Target: Vulkan GPU-v3-v3 - Model: mobilenet-v3

OpenBenchmarking.orgms, Fewer Is BetterNCNN 20220729Target: Vulkan GPU-v3-v3 - Model: mobilenet-v3ASPEED - AMD Ryzen Threadripper PRO 3955WX 16-Cores0.73581.47162.20742.94323.679SE +/- 0.01, N = 33.27MIN: 3.05 / MAX: 7.561. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread -pthread

NCNN

Target: Vulkan GPU - Model: mnasnet

OpenBenchmarking.orgms, Fewer Is BetterNCNN 20220729Target: Vulkan GPU - Model: mnasnetASPEED - AMD Ryzen Threadripper PRO 3955WX 16-Cores0.61651.2331.84952.4663.0825SE +/- 0.02, N = 32.74MIN: 2.57 / MAX: 7.271. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread -pthread

NCNN

Target: Vulkan GPU - Model: efficientnet-b0

OpenBenchmarking.orgms, Fewer Is BetterNCNN 20220729Target: Vulkan GPU - Model: efficientnet-b0ASPEED - AMD Ryzen Threadripper PRO 3955WX 16-Cores246810SE +/- 0.02, N = 38.71MIN: 8.58 / MAX: 20.281. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread -pthread

NCNN

Target: Vulkan GPU - Model: blazeface

OpenBenchmarking.orgms, Fewer Is BetterNCNN 20220729Target: Vulkan GPU - Model: blazefaceASPEED - AMD Ryzen Threadripper PRO 3955WX 16-Cores0.40730.81461.22191.62922.0365SE +/- 0.00, N = 31.81MIN: 1.78 / MAX: 2.371. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread -pthread

NCNN

Target: Vulkan GPU - Model: googlenet

OpenBenchmarking.orgms, Fewer Is BetterNCNN 20220729Target: Vulkan GPU - Model: googlenetASPEED - AMD Ryzen Threadripper PRO 3955WX 16-Cores1.33882.67764.01645.35526.694SE +/- 0.02, N = 35.95MIN: 5.86 / MAX: 16.181. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread -pthread

NCNN

Target: Vulkan GPU - Model: vgg16

OpenBenchmarking.orgms, Fewer Is BetterNCNN 20220729Target: Vulkan GPU - Model: vgg16ASPEED - AMD Ryzen Threadripper PRO 3955WX 16-Cores1.07782.15563.23344.31125.389SE +/- 0.02, N = 34.79MIN: 4.73 / MAX: 9.551. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread -pthread

NCNN

Target: Vulkan GPU - Model: resnet18

OpenBenchmarking.orgms, Fewer Is BetterNCNN 20220729Target: Vulkan GPU - Model: resnet18ASPEED - AMD Ryzen Threadripper PRO 3955WX 16-Cores0.61651.2331.84952.4663.0825SE +/- 0.00, N = 32.74MIN: 2.65 / MAX: 3.661. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread -pthread

NCNN

Target: Vulkan GPU - Model: alexnet

OpenBenchmarking.orgms, Fewer Is BetterNCNN 20220729Target: Vulkan GPU - Model: alexnetASPEED - AMD Ryzen Threadripper PRO 3955WX 16-Cores0.44550.8911.33651.7822.2275SE +/- 0.05, N = 31.98MIN: 1.79 / MAX: 2.651. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread -pthread

NCNN

Target: Vulkan GPU - Model: resnet50

OpenBenchmarking.orgms, Fewer Is BetterNCNN 20220729Target: Vulkan GPU - Model: resnet50ASPEED - AMD Ryzen Threadripper PRO 3955WX 16-Cores1.22632.45263.67894.90526.1315SE +/- 0.01, N = 35.45MIN: 5.26 / MAX: 10.831. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread -pthread

NCNN

Target: Vulkan GPU - Model: yolov4-tiny

OpenBenchmarking.orgms, Fewer Is BetterNCNN 20220729Target: Vulkan GPU - Model: yolov4-tinyASPEED - AMD Ryzen Threadripper PRO 3955WX 16-Cores3691215SE +/- 0.03, N = 310.09MIN: 9.65 / MAX: 43.221. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread -pthread

NCNN

Target: Vulkan GPU - Model: squeezenet_ssd

OpenBenchmarking.orgms, Fewer Is BetterNCNN 20220729Target: Vulkan GPU - Model: squeezenet_ssdASPEED - AMD Ryzen Threadripper PRO 3955WX 16-Cores1.30282.60563.90845.21126.514SE +/- 0.02, N = 35.79MIN: 4.91 / MAX: 11.021. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread -pthread

NCNN

Target: Vulkan GPU - Model: regnety_400m

OpenBenchmarking.orgms, Fewer Is BetterNCNN 20220729Target: Vulkan GPU - Model: regnety_400mASPEED - AMD Ryzen Threadripper PRO 3955WX 16-Cores1.05082.10163.15244.20325.254SE +/- 0.02, N = 34.67MIN: 4.59 / MAX: 10.71. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread -pthread

NCNN

Target: Vulkan GPU - Model: vision_transformer

OpenBenchmarking.orgms, Fewer Is BetterNCNN 20220729Target: Vulkan GPU - Model: vision_transformerASPEED - AMD Ryzen Threadripper PRO 3955WX 16-Cores4080120160200SE +/- 0.58, N = 3166.51MIN: 163.13 / MAX: 180.551. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread -pthread

NCNN

Target: Vulkan GPU - Model: FastestDet

OpenBenchmarking.orgms, Fewer Is BetterNCNN 20220729Target: Vulkan GPU - Model: FastestDetASPEED - AMD Ryzen Threadripper PRO 3955WX 16-Cores0.69981.39962.09942.79923.499SE +/- 0.01, N = 33.11MIN: 2.64 / MAX: 9.481. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread -pthread

NCNN

Target: Vulkan GPU - Model: shufflenet-v2

OpenBenchmarking.orgms, Fewer Is BetterNCNN 20220729Target: Vulkan GPU - Model: shufflenet-v2ASPEED - AMD Ryzen Threadripper PRO 3955WX 16-Cores0.53781.07561.61342.15122.689SE +/- 0.01, N = 22.39MIN: 2.34 / MAX: 2.951. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread -pthread

TNN

Target: CPU - Model: DenseNet

OpenBenchmarking.orgms, Fewer Is BetterTNN 0.3Target: CPU - Model: DenseNetASPEED - AMD Ryzen Threadripper PRO 3955WX 16-Cores5001000150020002500SE +/- 0.54, N = 32551.15MIN: 2472.01 / MAX: 2685.911. (CXX) g++ options: -fopenmp -pthread -fvisibility=hidden -fvisibility=default -O3 -rdynamic -ldl

TNN

Target: CPU - Model: MobileNet v2

OpenBenchmarking.orgms, Fewer Is BetterTNN 0.3Target: CPU - Model: MobileNet v2ASPEED - AMD Ryzen Threadripper PRO 3955WX 16-Cores60120180240300SE +/- 0.06, N = 3255.67MIN: 253.22 / MAX: 264.21. (CXX) g++ options: -fopenmp -pthread -fvisibility=hidden -fvisibility=default -O3 -rdynamic -ldl

TNN

Target: CPU - Model: SqueezeNet v2

OpenBenchmarking.orgms, Fewer Is BetterTNN 0.3Target: CPU - Model: SqueezeNet v2ASPEED - AMD Ryzen Threadripper PRO 3955WX 16-Cores1428425670SE +/- 0.30, N = 364.26MIN: 63.33 / MAX: 65.391. (CXX) g++ options: -fopenmp -pthread -fvisibility=hidden -fvisibility=default -O3 -rdynamic -ldl

TNN

Target: CPU - Model: SqueezeNet v1.1

OpenBenchmarking.orgms, Fewer Is BetterTNN 0.3Target: CPU - Model: SqueezeNet v1.1ASPEED - AMD Ryzen Threadripper PRO 3955WX 16-Cores50100150200250SE +/- 0.05, N = 3233.17MIN: 232.51 / MAX: 236.391. (CXX) g++ options: -fopenmp -pthread -fvisibility=hidden -fvisibility=default -O3 -rdynamic -ldl

OpenVINO

Model: Face Detection FP16 - Device: CPU

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2022.3Model: Face Detection FP16 - Device: CPUASPEED - AMD Ryzen Threadripper PRO 3955WX 16-Cores30060090012001500SE +/- 10.70, N = 31548.99MIN: 823.66 / MAX: 1631.071. (CXX) g++ options: -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -pie -pthread -ldl

OpenVINO

Model: Person Detection FP16 - Device: CPU

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2022.3Model: Person Detection FP16 - Device: CPUASPEED - AMD Ryzen Threadripper PRO 3955WX 16-Cores5001000150020002500SE +/- 5.81, N = 32312.14MIN: 2105.5 / MAX: 2440.891. (CXX) g++ options: -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -pie -pthread -ldl

OpenVINO

Model: Person Detection FP32 - Device: CPU

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2022.3Model: Person Detection FP32 - Device: CPUASPEED - AMD Ryzen Threadripper PRO 3955WX 16-Cores5001000150020002500SE +/- 5.13, N = 32329.14MIN: 1204.05 / MAX: 2474.771. (CXX) g++ options: -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -pie -pthread -ldl

OpenVINO

Model: Vehicle Detection FP16 - Device: CPU

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2022.3Model: Vehicle Detection FP16 - Device: CPUASPEED - AMD Ryzen Threadripper PRO 3955WX 16-Cores48121620SE +/- 0.04, N = 318.13MIN: 9.86 / MAX: 31.711. (CXX) g++ options: -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -pie -pthread -ldl

OpenVINO

Model: Face Detection FP16-INT8 - Device: CPU

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2022.3Model: Face Detection FP16-INT8 - Device: CPUASPEED - AMD Ryzen Threadripper PRO 3955WX 16-Cores30060090012001500SE +/- 0.42, N = 31353.76MIN: 1282.18 / MAX: 1368.21. (CXX) g++ options: -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -pie -pthread -ldl

OpenVINO

Model: Vehicle Detection FP16-INT8 - Device: CPU

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2022.3Model: Vehicle Detection FP16-INT8 - Device: CPUASPEED - AMD Ryzen Threadripper PRO 3955WX 16-Cores48121620SE +/- 0.04, N = 316.15MIN: 9.33 / MAX: 26.861. (CXX) g++ options: -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -pie -pthread -ldl

OpenVINO

Model: Weld Porosity Detection FP16 - Device: CPU

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2022.3Model: Weld Porosity Detection FP16 - Device: CPUASPEED - AMD Ryzen Threadripper PRO 3955WX 16-Cores48121620SE +/- 0.04, N = 316.48MIN: 8.96 / MAX: 29.71. (CXX) g++ options: -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -pie -pthread -ldl

OpenVINO

Model: Machine Translation EN To DE FP16 - Device: CPU

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2022.3Model: Machine Translation EN To DE FP16 - Device: CPUASPEED - AMD Ryzen Threadripper PRO 3955WX 16-Cores306090120150SE +/- 0.43, N = 3148.88MIN: 73.69 / MAX: 174.871. (CXX) g++ options: -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -pie -pthread -ldl

OpenVINO

Model: Weld Porosity Detection FP16-INT8 - Device: CPU

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2022.3Model: Weld Porosity Detection FP16-INT8 - Device: CPUASPEED - AMD Ryzen Threadripper PRO 3955WX 16-Cores612182430SE +/- 0.02, N = 327.46MIN: 15.54 / MAX: 38.781. (CXX) g++ options: -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -pie -pthread -ldl

OpenVINO

Model: Person Vehicle Bike Detection FP16 - Device: CPU

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2022.3Model: Person Vehicle Bike Detection FP16 - Device: CPUASPEED - AMD Ryzen Threadripper PRO 3955WX 16-Cores3691215SE +/- 0.02, N = 311.51MIN: 7.34 / MAX: 24.371. (CXX) g++ options: -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -pie -pthread -ldl

OpenVINO

Model: Age Gender Recognition Retail 0013 FP16 - Device: CPU

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2022.3Model: Age Gender Recognition Retail 0013 FP16 - Device: CPUASPEED - AMD Ryzen Threadripper PRO 3955WX 16-Cores0.24080.48160.72240.96321.204SE +/- 0.01, N = 31.07MIN: 0.59 / MAX: 13.91. (CXX) g++ options: -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -pie -pthread -ldl

OpenVINO

Model: Age Gender Recognition Retail 0013 FP16-INT8 - Device: CPU

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2022.3Model: Age Gender Recognition Retail 0013 FP16-INT8 - Device: CPUASPEED - AMD Ryzen Threadripper PRO 3955WX 16-Cores0.22050.4410.66150.8821.1025SE +/- 0.00, N = 30.98MIN: 0.57 / MAX: 13.641. (CXX) g++ options: -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -pie -pthread -ldl

OpenCV

Test: DNN - Deep Neural Network

OpenBenchmarking.orgms, Fewer Is BetterOpenCV 4.6Test: DNN - Deep Neural NetworkASPEED - AMD Ryzen Threadripper PRO 3955WX 16-Cores15003000450060007500SE +/- 26.03, N = 370371. (CXX) g++ options: -fPIC -fsigned-char -pthread -fomit-frame-pointer -ffunction-sections -fdata-sections -msse -msse2 -msse3 -fvisibility=hidden -O3 -shared

Neural Magic DeepSparse

Model: NLP Document Classification, oBERT base uncased on IMDB - Scenario: Asynchronous Multi-Stream

OpenBenchmarking.orgms/batch, Fewer Is BetterNeural Magic DeepSparse 1.3.2Model: NLP Document Classification, oBERT base uncased on IMDB - Scenario: Asynchronous Multi-StreamASPEED - AMD Ryzen Threadripper PRO 3955WX 16-Cores120240360480600SE +/- 0.69, N = 3566.95

Neural Magic DeepSparse

Model: NLP Document Classification, oBERT base uncased on IMDB - Scenario: Synchronous Single-Stream

OpenBenchmarking.orgms/batch, Fewer Is BetterNeural Magic DeepSparse 1.3.2Model: NLP Document Classification, oBERT base uncased on IMDB - Scenario: Synchronous Single-StreamASPEED - AMD Ryzen Threadripper PRO 3955WX 16-Cores20406080100SE +/- 0.02, N = 396.62

Neural Magic DeepSparse

Model: NLP Sentiment Analysis, 80% Pruned Quantized BERT Base Uncased - Scenario: Asynchronous Multi-Stream

OpenBenchmarking.orgms/batch, Fewer Is BetterNeural Magic DeepSparse 1.3.2Model: NLP Sentiment Analysis, 80% Pruned Quantized BERT Base Uncased - Scenario: Asynchronous Multi-StreamASPEED - AMD Ryzen Threadripper PRO 3955WX 16-Cores1122334455SE +/- 0.05, N = 350.29

Neural Magic DeepSparse

Model: NLP Sentiment Analysis, 80% Pruned Quantized BERT Base Uncased - Scenario: Synchronous Single-Stream

OpenBenchmarking.orgms/batch, Fewer Is BetterNeural Magic DeepSparse 1.3.2Model: NLP Sentiment Analysis, 80% Pruned Quantized BERT Base Uncased - Scenario: Synchronous Single-StreamASPEED - AMD Ryzen Threadripper PRO 3955WX 16-Cores48121620SE +/- 0.05, N = 315.21

Neural Magic DeepSparse

Model: NLP Question Answering, BERT base uncased SQuaD 12layer Pruned90 - Scenario: Asynchronous Multi-Stream

OpenBenchmarking.orgms/batch, Fewer Is BetterNeural Magic DeepSparse 1.3.2Model: NLP Question Answering, BERT base uncased SQuaD 12layer Pruned90 - Scenario: Asynchronous Multi-StreamASPEED - AMD Ryzen Threadripper PRO 3955WX 16-Cores306090120150SE +/- 0.54, N = 3143.66

Neural Magic DeepSparse

Model: NLP Question Answering, BERT base uncased SQuaD 12layer Pruned90 - Scenario: Synchronous Single-Stream

OpenBenchmarking.orgms/batch, Fewer Is BetterNeural Magic DeepSparse 1.3.2Model: NLP Question Answering, BERT base uncased SQuaD 12layer Pruned90 - Scenario: Synchronous Single-StreamASPEED - AMD Ryzen Threadripper PRO 3955WX 16-Cores918273645SE +/- 0.13, N = 337.57

Neural Magic DeepSparse

Model: CV Detection, YOLOv5s COCO - Scenario: Asynchronous Multi-Stream

OpenBenchmarking.orgms/batch, Fewer Is BetterNeural Magic DeepSparse 1.3.2Model: CV Detection, YOLOv5s COCO - Scenario: Asynchronous Multi-StreamASPEED - AMD Ryzen Threadripper PRO 3955WX 16-Cores20406080100SE +/- 0.17, N = 3100.91

Neural Magic DeepSparse

Model: CV Detection, YOLOv5s COCO - Scenario: Synchronous Single-Stream

OpenBenchmarking.orgms/batch, Fewer Is BetterNeural Magic DeepSparse 1.3.2Model: CV Detection, YOLOv5s COCO - Scenario: Synchronous Single-StreamASPEED - AMD Ryzen Threadripper PRO 3955WX 16-Cores510152025SE +/- 0.04, N = 319.90

Neural Magic DeepSparse

Model: CV Classification, ResNet-50 ImageNet - Scenario: Asynchronous Multi-Stream

OpenBenchmarking.orgms/batch, Fewer Is BetterNeural Magic DeepSparse 1.3.2Model: CV Classification, ResNet-50 ImageNet - Scenario: Asynchronous Multi-StreamASPEED - AMD Ryzen Threadripper PRO 3955WX 16-Cores1122334455SE +/- 0.04, N = 347.25

Neural Magic DeepSparse

Model: CV Classification, ResNet-50 ImageNet - Scenario: Synchronous Single-Stream

OpenBenchmarking.orgms/batch, Fewer Is BetterNeural Magic DeepSparse 1.3.2Model: CV Classification, ResNet-50 ImageNet - Scenario: Synchronous Single-StreamASPEED - AMD Ryzen Threadripper PRO 3955WX 16-Cores3691215SE +/- 0.0294, N = 39.0813

Neural Magic DeepSparse

Model: NLP Text Classification, DistilBERT mnli - Scenario: Asynchronous Multi-Stream

OpenBenchmarking.orgms/batch, Fewer Is BetterNeural Magic DeepSparse 1.3.2Model: NLP Text Classification, DistilBERT mnli - Scenario: Asynchronous Multi-StreamASPEED - AMD Ryzen Threadripper PRO 3955WX 16-Cores1632486480SE +/- 0.04, N = 370.34

Neural Magic DeepSparse

Model: NLP Text Classification, DistilBERT mnli - Scenario: Synchronous Single-Stream

OpenBenchmarking.orgms/batch, Fewer Is BetterNeural Magic DeepSparse 1.3.2Model: NLP Text Classification, DistilBERT mnli - Scenario: Synchronous Single-StreamASPEED - AMD Ryzen Threadripper PRO 3955WX 16-Cores3691215SE +/- 0.03, N = 313.09

Neural Magic DeepSparse

Model: CV Segmentation, 90% Pruned YOLACT Pruned - Scenario: Asynchronous Multi-Stream

OpenBenchmarking.orgms/batch, Fewer Is BetterNeural Magic DeepSparse 1.3.2Model: CV Segmentation, 90% Pruned YOLACT Pruned - Scenario: Asynchronous Multi-StreamASPEED - AMD Ryzen Threadripper PRO 3955WX 16-Cores100200300400500SE +/- 0.19, N = 3439.43

Neural Magic DeepSparse

Model: CV Segmentation, 90% Pruned YOLACT Pruned - Scenario: Synchronous Single-Stream

OpenBenchmarking.orgms/batch, Fewer Is BetterNeural Magic DeepSparse 1.3.2Model: CV Segmentation, 90% Pruned YOLACT Pruned - Scenario: Synchronous Single-StreamASPEED - AMD Ryzen Threadripper PRO 3955WX 16-Cores1530456075SE +/- 0.10, N = 368.47

Neural Magic DeepSparse

Model: NLP Text Classification, BERT base uncased SST2 - Scenario: Asynchronous Multi-Stream

OpenBenchmarking.orgms/batch, Fewer Is BetterNeural Magic DeepSparse 1.3.2Model: NLP Text Classification, BERT base uncased SST2 - Scenario: Asynchronous Multi-StreamASPEED - AMD Ryzen Threadripper PRO 3955WX 16-Cores306090120150SE +/- 0.08, N = 3138.73

Neural Magic DeepSparse

Model: NLP Text Classification, BERT base uncased SST2 - Scenario: Synchronous Single-Stream

OpenBenchmarking.orgms/batch, Fewer Is BetterNeural Magic DeepSparse 1.3.2Model: NLP Text Classification, BERT base uncased SST2 - Scenario: Synchronous Single-StreamASPEED - AMD Ryzen Threadripper PRO 3955WX 16-Cores612182430SE +/- 0.08, N = 325.66

Neural Magic DeepSparse

Model: NLP Token Classification, BERT base uncased conll2003 - Scenario: Asynchronous Multi-Stream

OpenBenchmarking.orgms/batch, Fewer Is BetterNeural Magic DeepSparse 1.3.2Model: NLP Token Classification, BERT base uncased conll2003 - Scenario: Asynchronous Multi-StreamASPEED - AMD Ryzen Threadripper PRO 3955WX 16-Cores120240360480600SE +/- 0.27, N = 3568.03

Neural Magic DeepSparse

Model: NLP Token Classification, BERT base uncased conll2003 - Scenario: Synchronous Single-Stream

OpenBenchmarking.orgms/batch, Fewer Is BetterNeural Magic DeepSparse 1.3.2Model: NLP Token Classification, BERT base uncased conll2003 - Scenario: Synchronous Single-StreamASPEED - AMD Ryzen Threadripper PRO 3955WX 16-Cores20406080100SE +/- 0.10, N = 396.81

DeepSpeech

Acceleration: CPU

OpenBenchmarking.orgSeconds, Fewer Is BetterDeepSpeech 0.6Acceleration: CPUASPEED - AMD Ryzen Threadripper PRO 3955WX 16-Cores1326395265SE +/- 0.10, N = 357.59

R Benchmark

OpenBenchmarking.orgSeconds, Fewer Is BetterR BenchmarkASPEED - AMD Ryzen Threadripper PRO 3955WX 16-Cores0.04530.09060.13590.18120.2265SE +/- 0.0005, N = 30.20121. R scripting front-end version 3.6.3 (2020-02-29)

RNNoise

OpenBenchmarking.orgSeconds, Fewer Is BetterRNNoise 2020-06-28ASPEED - AMD Ryzen Threadripper PRO 3955WX 16-Cores48121620SE +/- 0.04, N = 317.871. (CC) gcc options: -O2 -pedantic -fvisibility=hidden

Numenta Anomaly Benchmark

Detector: KNN CAD

OpenBenchmarking.orgSeconds, Fewer Is BetterNumenta Anomaly Benchmark 1.1Detector: KNN CADASPEED - AMD Ryzen Threadripper PRO 3955WX 16-Cores306090120150SE +/- 1.26, N = 3118.71

Numenta Anomaly Benchmark

Detector: Windowed Gaussian

OpenBenchmarking.orgSeconds, Fewer Is BetterNumenta Anomaly Benchmark 1.1Detector: Windowed GaussianASPEED - AMD Ryzen Threadripper PRO 3955WX 16-Cores1.12932.25863.38794.51725.6465SE +/- 0.030, N = 35.019

Numenta Anomaly Benchmark

Detector: Earthgecko Skyline

OpenBenchmarking.orgSeconds, Fewer Is BetterNumenta Anomaly Benchmark 1.1Detector: Earthgecko SkylineASPEED - AMD Ryzen Threadripper PRO 3955WX 16-Cores1632486480SE +/- 0.46, N = 373.58

Numenta Anomaly Benchmark

Detector: Contextual Anomaly Detector OSE

OpenBenchmarking.orgSeconds, Fewer Is BetterNumenta Anomaly Benchmark 1.1Detector: Contextual Anomaly Detector OSEASPEED - AMD Ryzen Threadripper PRO 3955WX 16-Cores1020304050SE +/- 0.29, N = 343.18

Mlpack Benchmark

Benchmark: scikit_ica

OpenBenchmarking.orgSeconds, Fewer Is BetterMlpack BenchmarkBenchmark: scikit_icaASPEED - AMD Ryzen Threadripper PRO 3955WX 16-Cores1122334455SE +/- 0.32, N = 350.86

Mlpack Benchmark

Benchmark: scikit_qda

OpenBenchmarking.orgSeconds, Fewer Is BetterMlpack BenchmarkBenchmark: scikit_qdaASPEED - AMD Ryzen Threadripper PRO 3955WX 16-Cores918273645SE +/- 0.10, N = 337.36

Mlpack Benchmark

Benchmark: scikit_svm

OpenBenchmarking.orgSeconds, Fewer Is BetterMlpack BenchmarkBenchmark: scikit_svmASPEED - AMD Ryzen Threadripper PRO 3955WX 16-Cores510152025SE +/- 0.02, N = 319.96

Mlpack Benchmark

Benchmark: scikit_linearridgeregression

OpenBenchmarking.orgSeconds, Fewer Is BetterMlpack BenchmarkBenchmark: scikit_linearridgeregressionASPEED - AMD Ryzen Threadripper PRO 3955WX 16-Cores0.33980.67961.01941.35921.699SE +/- 0.01, N = 31.51

Scikit-Learn

Benchmark: Sparse Random Projections, 100 Iterations

OpenBenchmarking.orgSeconds, Fewer Is BetterScikit-Learn 1.1.3Benchmark: Sparse Random Projections, 100 IterationsASPEED - AMD Ryzen Threadripper PRO 3955WX 16-Cores4080120160200SE +/- 0.43, N = 3159.01

Phoronix Test Suite v10.8.4