ddddx

AMD Ryzen Threadripper PRO 5965WX 24-Cores testing with a ASUS Pro WS WRX80E-SAGE SE WIFI (1201 BIOS) and ASUS NVIDIA NV106 2GB on Ubuntu 23.10 via the Phoronix Test Suite.

HTML result view exported from: https://openbenchmarking.org/result/2403218-NE-DDDDX513530&sor&grs.

ddddxProcessorMotherboardChipsetMemoryDiskGraphicsAudioMonitorNetworkOSKernelDesktopDisplay ServerDisplay DriverOpenGLCompilerFile-SystemScreen ResolutionabcdAMD Ryzen Threadripper PRO 5965WX 24-Cores @ 3.80GHz (24 Cores / 48 Threads)ASUS Pro WS WRX80E-SAGE SE WIFI (1201 BIOS)AMD Starship/Matisse8 x 16GB DDR4-2133MT/s Corsair CMK32GX4M2E3200C162048GB SOLIDIGM SSDPFKKW020X7ASUS NVIDIA NV106 2GBAMD Starship/MatisseVA24312 x Intel X550 + Intel Wi-Fi 6 AX200Ubuntu 23.106.5.0-15-generic (x86_64)GNOME Shell 45.0X Server + Waylandnouveau4.3 Mesa 23.2.1-1ubuntu3GCC 13.2.0ext41920x1080OpenBenchmarking.orgKernel Details- Transparent Huge Pages: madviseCompiler Details- --build=x86_64-linux-gnu --disable-vtable-verify --disable-werror --enable-bootstrap --enable-cet --enable-checking=release --enable-clocale=gnu --enable-default-pie --enable-gnu-unique-object --enable-languages=c,ada,c++,go,d,fortran,objc,obj-c++,m2 --enable-libphobos-checking=release --enable-libstdcxx-debug --enable-libstdcxx-time=yes --enable-link-serialization=2 --enable-multiarch --enable-multilib --enable-nls --enable-objc-gc=auto --enable-offload-defaulted --enable-offload-targets=nvptx-none=/build/gcc-13-XYspKM/gcc-13-13.2.0/debian/tmp-nvptx/usr,amdgcn-amdhsa=/build/gcc-13-XYspKM/gcc-13-13.2.0/debian/tmp-gcn/usr --enable-plugin --enable-shared --enable-threads=posix --host=x86_64-linux-gnu --program-prefix=x86_64-linux-gnu- --target=x86_64-linux-gnu --with-abi=m64 --with-arch-32=i686 --with-build-config=bootstrap-lto-lean --with-default-libstdcxx-abi=new --with-gcc-major-version-only --with-multilib-list=m32,m64,mx32 --with-target-system-zlib=auto --with-tune=generic --without-cuda-driver -v Processor Details- Scaling Governor: acpi-cpufreq schedutil (Boost: Enabled) - CPU Microcode: 0xa008205Python Details- Python 3.11.6Security Details- gather_data_sampling: Not affected + itlb_multihit: Not affected + l1tf: Not affected + mds: Not affected + meltdown: Not affected + mmio_stale_data: Not affected + retbleed: Not affected + spec_rstack_overflow: Mitigation of safe RET no microcode + spec_store_bypass: Mitigation of SSB disabled via prctl + spectre_v1: Mitigation of usercopy/swapgs barriers and __user pointer sanitization + spectre_v2: Mitigation of Retpolines IBPB: conditional IBRS_FW STIBP: always-on RSB filling PBRSB-eIBRS: Not affected + srbds: Not affected + tsx_async_abort: Not affected

ddddxjpegxl: PNG - 80jpegxl: JPEG - 80jpegxl-decode: Allonednn: Deconvolution Batch shapes_1d - CPUjpegxl: JPEG - 90jpegxl: PNG - 90svt-av1: Preset 13 - Bosphorus 4Ksrsran: PDSCH Processor Benchmark, Throughput Totalsvt-av1: Preset 13 - Bosphorus 1080pcompress-pbzip2: FreeBSD-13.0-RELEASE-amd64-memstick.img Compressionsvt-av1: Preset 12 - Bosphorus 4Kprimesieve: 1e12build-linux-kernel: defconfigrocksdb: Rand Fill Syncrocksdb: Overwritesvt-av1: Preset 12 - Bosphorus 1080pbrl-cad: VGR Performance Metricrocksdb: Read Rand Write Randsvt-av1: Preset 8 - Bosphorus 4Ksvt-av1: Preset 4 - Bosphorus 1080psvt-av1: Preset 4 - Bosphorus 4Krocksdb: Rand Fillvvenc: Bosphorus 1080p - Fastervvenc: Bosphorus 4K - Fasterjpegxl-decode: 1deepsparse: ResNet-50, Sparse INT8 - Asynchronous Multi-Streamdeepsparse: ResNet-50, Sparse INT8 - Asynchronous Multi-Streamrocksdb: Update Randopenvino: Vehicle Detection FP16 - CPUopenvino: Vehicle Detection FP16 - CPUsrsran: PUSCH Processor Benchmark, Throughput Totalsvt-av1: Preset 8 - Bosphorus 1080procksdb: Read While Writingospray-studio: 1 - 4K - 16 - Path Tracer - CPUdeepsparse: NLP Text Classification, BERT base uncased SST2, Sparse INT8 - Synchronous Single-Streamdeepsparse: NLP Text Classification, BERT base uncased SST2, Sparse INT8 - Synchronous Single-Streamrocksdb: Seq Fillospray-studio: 3 - 4K - 32 - Path Tracer - CPUospray-studio: 2 - 4K - 32 - Path Tracer - CPUsrsran: PDSCH Processor Benchmark, Throughput Threadonednn: IP Shapes 1D - CPUopenvino: Noise Suppression Poconet-Like FP16 - CPUopenvino: Face Detection FP16 - CPUjpegxl: PNG - 100openvino: Noise Suppression Poconet-Like FP16 - CPUospray-studio: 2 - 1080p - 32 - Path Tracer - CPUopenvino: Handwritten English Recognition FP16-INT8 - CPUopenvino: Handwritten English Recognition FP16-INT8 - CPUdraco: Church Facadeopenvino: Age Gender Recognition Retail 0013 FP16 - CPUopenvino: Face Detection FP16 - CPUonednn: Convolution Batch Shapes Auto - CPUonednn: Deconvolution Batch shapes_3d - CPUdeepsparse: BERT-Large, NLP Question Answering, Sparse INT8 - Asynchronous Multi-Streamdeepsparse: BERT-Large, NLP Question Answering, Sparse INT8 - Asynchronous Multi-Streamdeepsparse: CV Detection, YOLOv5s COCO, Sparse INT8 - Asynchronous Multi-Streamospray: particle_volume/pathtracer/real_timeonednn: Recurrent Neural Network Inference - CPUvvenc: Bosphorus 1080p - Fastdeepsparse: CV Detection, YOLOv5s COCO, Sparse INT8 - Asynchronous Multi-Streamdeepsparse: NLP Text Classification, DistilBERT mnli - Synchronous Single-Streamdeepsparse: NLP Text Classification, DistilBERT mnli - Synchronous Single-Streamospray-studio: 2 - 4K - 16 - Path Tracer - CPUv-ray: CPUjpegxl: JPEG - 100openvino: Road Segmentation ADAS FP16 - CPUopenvino: Road Segmentation ADAS FP16 - CPUdeepsparse: NLP Document Classification, oBERT base uncased on IMDB - Synchronous Single-Streamdeepsparse: NLP Document Classification, oBERT base uncased on IMDB - Synchronous Single-Streamdeepsparse: ResNet-50, Sparse INT8 - Synchronous Single-Streamopenvino: Handwritten English Recognition FP16 - CPUopenvino: Handwritten English Recognition FP16 - CPUdeepsparse: CV Classification, ResNet-50 ImageNet - Synchronous Single-Streamrocksdb: Rand Readdeepsparse: ResNet-50, Sparse INT8 - Synchronous Single-Streamdeepsparse: CV Classification, ResNet-50 ImageNet - Synchronous Single-Streamospray-studio: 3 - 1080p - 32 - Path Tracer - CPUospray-studio: 2 - 1080p - 1 - Path Tracer - CPUdraco: Lionopenvino: Age Gender Recognition Retail 0013 FP16 - CPUopenvino: Person Vehicle Bike Detection FP16 - CPUopenvino: Person Vehicle Bike Detection FP16 - CPUopenvino: Weld Porosity Detection FP16 - CPUopenvino: Age Gender Recognition Retail 0013 FP16-INT8 - CPUopenvino: Weld Porosity Detection FP16 - CPUsrsran: PUSCH Processor Benchmark, Throughput Threadospray-studio: 3 - 1080p - 1 - Path Tracer - CPUdeepsparse: NLP Token Classification, BERT base uncased conll2003 - Asynchronous Multi-Streamospray-studio: 2 - 1080p - 16 - Path Tracer - CPUopenvino: Person Detection FP16 - CPUdeepsparse: ResNet-50, Baseline - Synchronous Single-Streamopenvino: Person Detection FP16 - CPUdeepsparse: BERT-Large, NLP Question Answering, Sparse INT8 - Synchronous Single-Streamdeepsparse: ResNet-50, Baseline - Synchronous Single-Streamdeepsparse: BERT-Large, NLP Question Answering, Sparse INT8 - Synchronous Single-Streamopenvino: Face Detection Retail FP16-INT8 - CPUdeepsparse: CV Detection, YOLOv5s COCO, Sparse INT8 - Synchronous Single-Streamdeepsparse: CV Detection, YOLOv5s COCO, Sparse INT8 - Synchronous Single-Streamdeepsparse: NLP Text Classification, DistilBERT mnli - Asynchronous Multi-Streamdeepsparse: NLP Text Classification, DistilBERT mnli - Asynchronous Multi-Streamdeepsparse: Llama2 Chat 7b Quantized - Asynchronous Multi-Streamopenvino: Face Detection Retail FP16-INT8 - CPUdeepsparse: Llama2 Chat 7b Quantized - Asynchronous Multi-Streamospray: particle_volume/ao/real_timeopenvino: Weld Porosity Detection FP16-INT8 - CPUopenvino: Face Detection FP16-INT8 - CPUospray: gravity_spheres_volume/dim_512/scivis/real_timeopenvino: Weld Porosity Detection FP16-INT8 - CPUospray: gravity_spheres_volume/dim_512/ao/real_timeopenvino: Person Detection FP32 - CPUospray: particle_volume/scivis/real_timeospray-studio: 1 - 4K - 32 - Path Tracer - CPUopenvino: Person Detection FP32 - CPUprimesieve: 1e13ospray: gravity_spheres_volume/dim_512/pathtracer/real_timeopenvino: Vehicle Detection FP16-INT8 - CPUopenvino: Vehicle Detection FP16-INT8 - CPUonednn: IP Shapes 3D - CPUopenvino: Road Segmentation ADAS FP16-INT8 - CPUopenvino: Road Segmentation ADAS FP16-INT8 - CPUopenvino: Face Detection FP16-INT8 - CPUospray-studio: 1 - 1080p - 1 - Path Tracer - CPUdeepsparse: NLP Document Classification, oBERT base uncased on IMDB - Asynchronous Multi-Streamdeepsparse: CV Segmentation, 90% Pruned YOLACT Pruned - Asynchronous Multi-Streamopenvino: Machine Translation EN To DE FP16 - CPUospray-studio: 3 - 4K - 16 - Path Tracer - CPUdeepsparse: NLP Document Classification, oBERT base uncased on IMDB - Asynchronous Multi-Streamopenvino: Machine Translation EN To DE FP16 - CPUospray-studio: 1 - 4K - 1 - Path Tracer - CPUdeepsparse: NLP Token Classification, BERT base uncased conll2003 - Asynchronous Multi-Streamopenvino: Person Re-Identification Retail FP16 - CPUospray-studio: 3 - 1080p - 16 - Path Tracer - CPUopenvino: Person Re-Identification Retail FP16 - CPUospray-studio: 1 - 1080p - 16 - Path Tracer - CPUopenvino: Face Detection Retail FP16 - CPUdeepsparse: NLP Text Classification, BERT base uncased SST2, Sparse INT8 - Asynchronous Multi-Streamdeepsparse: NLP Text Classification, BERT base uncased SST2, Sparse INT8 - Asynchronous Multi-Streamdeepsparse: CV Segmentation, 90% Pruned YOLACT Pruned - Synchronous Single-Streamdeepsparse: CV Segmentation, 90% Pruned YOLACT Pruned - Synchronous Single-Streamdeepsparse: NLP Token Classification, BERT base uncased conll2003 - Synchronous Single-Streamdeepsparse: NLP Token Classification, BERT base uncased conll2003 - Synchronous Single-Streambuild-linux-kernel: allmodconfigospray-studio: 3 - 4K - 1 - Path Tracer - CPUvvenc: Bosphorus 4K - Fastencode-wavpack: WAV To WavPackospray-studio: 2 - 4K - 1 - Path Tracer - CPUospray-studio: 1 - 1080p - 32 - Path Tracer - CPUdeepsparse: CV Segmentation, 90% Pruned YOLACT Pruned - Asynchronous Multi-Streamopenvino: Face Detection Retail FP16 - CPUonednn: Recurrent Neural Network Training - CPUdeepsparse: Llama2 Chat 7b Quantized - Synchronous Single-Streamdeepsparse: Llama2 Chat 7b Quantized - Synchronous Single-Streamdeepsparse: CV Classification, ResNet-50 ImageNet - Asynchronous Multi-Streamdeepsparse: CV Classification, ResNet-50 ImageNet - Asynchronous Multi-Streamdeepsparse: ResNet-50, Baseline - Asynchronous Multi-Streamdeepsparse: ResNet-50, Baseline - Asynchronous Multi-Streamopenvino: Age Gender Recognition Retail 0013 FP16-INT8 - CPUstockfish: Chess Benchmarkabcd44.74846.584483.4155.5108242.50239.403151.95811710.3587.483.230305154.176.24454.13345528781166489.431430387287978762.95419.2066.64779060341.28314.56664.0321994.3896.00367112119.94600.941888.2131.7215546801780615.1435194.2938920430177445152653603.71.314741033.677.627.6511.544183956.87421.7270230.971558.812.738042.36511334.388735.850979.4548156.396638.12819.624150.82838.8921112.3308787884428727.52870.48170.154.416718.3721.249163.46377.84156.4322145962891798.47476.3846474951158532824478.3113.38895.2516.6944518.2718.2177.6133626.74111854370.016.3927171.2213.1742156.209975.8763.7199.334410.0615224.164153.48125873.27013224.641.877210.283514.5216.664.598821652.054.90257171.9210.138615037569.7377.1487.4661410.861104.033.52569428.4327.99715.22113826.692630.468313690362448.709188.134528446.840610214491198.64182032192.0917.5108684.565146.058221.700918.490854.0673597.03453307.064.433461641167393.66065.461254.68165.39996.044839.0528306.970239.0535306.99740.535260752843.24542.66482.5775.3392939.89237.44145.9712063.4602.9683.314749150.8096.08154.20845974779236499.14424641285933963.89318.9166.78778396740.46114.82363.3012032.95675.889866671520.04598.031889.2131.9285586539769955.2115191.7617908177175496153552600.21.311011046.37.6827.57211.414159856.3426.0370340.971547.312.711352.34602337.452535.539178.8208155.373636.76819.489151.98118.9317111.8371783044463427.32370.23170.6854.013918.5091.246863.08380.22155.3135145893753800.04746.4293478271150536524434.5113.47889.3616.6544461.02719.93178.7133826.85541847169.956.4286171.4313.1312155.341376.12223.6999.222810.0731224.197153.47145904.27883241.351.867710.246614.4916.734.577061654.724.88053171.1610.143315004070.0176.9187.4483710.831106.443.52584428.1828713.72113826.603530.4851135.790131447.653588.334521446.23759.98213981201.22181812197.517.4858685.448945.960121.747118.523253.9731597.89853427.0754.438460941125393.07615.451255.67165.55196.039339.0344307.089339.0501307.01880.536100827039.90442.353470.1595.3736740.76138.511150.52211723.5606.3603.285082150.1336.11352.81246701777804487.658420528286868162.49819.1706.71177438440.91514.77763.0381998.60735.988866097820.21592.961886.9133.2705592760775115.1787192.9770908882175767152510605.51.327431041.677.5927.48711.464205056.64423.4670920.981563.152.731922.36138334.747235.809279.2452155.833637.17319.460151.25598.9189112.0022789194437527.32170.66169.6554.329218.40181.247863.40378.34155.7940144921763799.41516.4103476861152536324329.2113.39894.5716.7644306.10715.28178.6133326.69561857969.836.4155171.6713.1431155.680376.05753.7198.800110.1159223.002853.75535896.59333228.191.870510.280414.5616.664.592661647.134.88608171.2910.147915038169.9977.2147.4382110.821107.643.53323427.0228.08716.16113826.616530.3866136.1590083449.118788.054535447.58299.97214091202.09182292197.9817.5229684.015946.076321.692518.486654.0787596.50653417.0664.435461541189393.82065.451256.43165.46516.042539.0572306.946439.0664306.91590.535523757342.17643.502439.6135.0364139.59538.265152.46611654.9587.9713.221993151.6386.12354.16346299762175488.521425778281451362.93119.3186.667771154114.85962.8172014.57665.941765852019.84603.961857133.7845633613770735.1408194.384919148175129151620607.81.313331043.827.6427.33811.444195956.9421.5470960.971553.542.733522.34346334.718135.813579.5069155.049642.19419.529150.73448.9625111.4506783544463327.31970.76169.4154.340218.39781.255963.54377.49155.8985145516874794.39026.4059475971153534724496.8413.4894.4116.7244596.35716.84177.9133026.80791848869.616.4001172.1813.2048156.041575.69913.7199.054710.0902223.55453.62595897.27913230.251.870110.298514.5116.744.580811652.914.89928171.5410.183315069469.8677.067.452210.861103.63.53851428.5527.98713.64113426.694130.488135.9690380448.92788.144528447.17439.97213851202.22181772198.1117.476685.815946.016621.720618.47954.1018597.82753317.0624.442461841111393.3945.451254.43165.60616.037339.0746306.899739.0595306.94360.5353082452OpenBenchmarking.org

JPEG-XL libjxl

Input: PNG - Quality: 80

OpenBenchmarking.orgMP/s, More Is BetterJPEG-XL libjxl 0.10.1Input: PNG - Quality: 80abdc1020304050SE +/- 0.29, N = 344.7543.2542.1839.901. (CXX) g++ options: -fno-rtti -O3 -fPIE -pie -lm

JPEG-XL libjxl

Input: JPEG - Quality: 80

OpenBenchmarking.orgMP/s, More Is BetterJPEG-XL libjxl 0.10.1Input: JPEG - Quality: 80adbc1122334455SE +/- 0.32, N = 346.5843.5042.6642.351. (CXX) g++ options: -fno-rtti -O3 -fPIE -pie -lm

JPEG-XL Decoding libjxl

CPU Threads: All

OpenBenchmarking.orgMP/s, More Is BetterJPEG-XL Decoding libjxl 0.10.1CPU Threads: Allabcd100200300400500SE +/- 1.28, N = 3483.42482.58470.16439.61

oneDNN

Harness: Deconvolution Batch shapes_1d - Engine: CPU

OpenBenchmarking.orgms, Fewer Is BetteroneDNN 3.4Harness: Deconvolution Batch shapes_1d - Engine: CPUdbca1.23992.47983.71974.95966.1995SE +/- 0.03672, N = 35.036415.339295.373675.51082MIN: 3.91MIN: 3.86MIN: 3.84MIN: 3.91. (CXX) g++ options: -O3 -march=native -fopenmp -msse4.1 -fPIC -pie -ldl

JPEG-XL libjxl

Input: JPEG - Quality: 90

OpenBenchmarking.orgMP/s, More Is BetterJPEG-XL libjxl 0.10.1Input: JPEG - Quality: 90acbd1020304050SE +/- 0.39, N = 1542.5040.7639.8939.601. (CXX) g++ options: -fno-rtti -O3 -fPIE -pie -lm

JPEG-XL libjxl

Input: PNG - Quality: 90

OpenBenchmarking.orgMP/s, More Is BetterJPEG-XL libjxl 0.10.1Input: PNG - Quality: 90acdb918273645SE +/- 0.27, N = 1539.4038.5138.2737.441. (CXX) g++ options: -fno-rtti -O3 -fPIE -pie -lm

SVT-AV1

Encoder Mode: Preset 13 - Input: Bosphorus 4K

OpenBenchmarking.orgFrames Per Second, More Is BetterSVT-AV1 2.0Encoder Mode: Preset 13 - Input: Bosphorus 4Kdacb306090120150SE +/- 1.34, N = 3152.47151.96150.52145.971. (CXX) g++ options: -march=native -mno-avx -mavx2 -mavx512f -mavx512bw -mavx512dq

srsRAN Project

Test: PDSCH Processor Benchmark, Throughput Total

OpenBenchmarking.orgMbps, More Is BettersrsRAN Project 23.10.1-20240219Test: PDSCH Processor Benchmark, Throughput Totalbcad3K6K9K12K15KSE +/- 27.21, N = 312063.411723.511710.311654.91. (CXX) g++ options: -march=native -mavx2 -mavx -msse4.1 -mfma -O3 -fno-trapping-math -fno-math-errno -ldl

SVT-AV1

Encoder Mode: Preset 13 - Input: Bosphorus 1080p

OpenBenchmarking.orgFrames Per Second, More Is BetterSVT-AV1 2.0Encoder Mode: Preset 13 - Input: Bosphorus 1080pcbda130260390520650SE +/- 4.87, N = 3606.36602.97587.97587.481. (CXX) g++ options: -march=native -mno-avx -mavx2 -mavx512f -mavx512bw -mavx512dq

Parallel BZIP2 Compression

FreeBSD-13.0-RELEASE-amd64-memstick.img Compression

OpenBenchmarking.orgSeconds, Fewer Is BetterParallel BZIP2 Compression 1.1.13FreeBSD-13.0-RELEASE-amd64-memstick.img Compressiondacb0.74581.49162.23742.98323.729SE +/- 0.043942, N = 123.2219933.2303053.2850823.3147491. (CXX) g++ options: -O2 -pthread -lbz2 -lpthread

SVT-AV1

Encoder Mode: Preset 12 - Input: Bosphorus 4K

OpenBenchmarking.orgFrames Per Second, More Is BetterSVT-AV1 2.0Encoder Mode: Preset 12 - Input: Bosphorus 4Kadbc306090120150SE +/- 0.99, N = 3154.17151.64150.81150.131. (CXX) g++ options: -march=native -mno-avx -mavx2 -mavx512f -mavx512bw -mavx512dq

Primesieve

Length: 1e12

OpenBenchmarking.orgSeconds, Fewer Is BetterPrimesieve 12.1Length: 1e12bcda246810SE +/- 0.056, N = 36.0816.1136.1236.2441. (CXX) g++ options: -O3

Timed Linux Kernel Compilation

Build: defconfig

OpenBenchmarking.orgSeconds, Fewer Is BetterTimed Linux Kernel Compilation 6.8Build: defconfigcadb1224364860SE +/- 0.59, N = 352.8154.1354.1654.21

RocksDB

Test: Random Fill Sync

OpenBenchmarking.orgOp/s, More Is BetterRocksDB 9.0Test: Random Fill Synccdba10K20K30K40K50KSE +/- 32.60, N = 3467014629945974455281. (CXX) g++ options: -O3 -march=native -pthread -fno-builtin-memcmp -fno-rtti -lpthread

RocksDB

Test: Overwrite

OpenBenchmarking.orgOp/s, More Is BetterRocksDB 9.0Test: Overwriteabcd200K400K600K800K1000KSE +/- 4178.44, N = 37811667792367778047621751. (CXX) g++ options: -O3 -march=native -pthread -fno-builtin-memcmp -fno-rtti -lpthread

SVT-AV1

Encoder Mode: Preset 12 - Input: Bosphorus 1080p

OpenBenchmarking.orgFrames Per Second, More Is BetterSVT-AV1 2.0Encoder Mode: Preset 12 - Input: Bosphorus 1080pbadc110220330440550SE +/- 6.37, N = 3499.14489.43488.52487.661. (CXX) g++ options: -march=native -mno-avx -mavx2 -mavx512f -mavx512bw -mavx512dq

BRL-CAD

VGR Performance Metric

OpenBenchmarking.orgVGR Performance Metric, More Is BetterBRL-CAD 7.38.2VGR Performance Metricadbc90K180K270K360K450K4303874257784246414205281. (CXX) g++ options: -std=c++17 -pipe -fvisibility=hidden -fno-strict-aliasing -fno-common -fexceptions -ftemplate-depth-128 -m64 -ggdb3 -O3 -fipa-pta -fstrength-reduce -finline-functions -flto -ltcl8.6 -lnetpbm -lregex_brl -lz_brl -lassimp -ldl -lm -ltk8.6

RocksDB

Test: Read Random Write Random

OpenBenchmarking.orgOp/s, More Is BetterRocksDB 9.0Test: Read Random Write Randomacbd600K1200K1800K2400K3000KSE +/- 5471.11, N = 328797872868681285933928145131. (CXX) g++ options: -O3 -march=native -pthread -fno-builtin-memcmp -fno-rtti -lpthread

SVT-AV1

Encoder Mode: Preset 8 - Input: Bosphorus 4K

OpenBenchmarking.orgFrames Per Second, More Is BetterSVT-AV1 2.0Encoder Mode: Preset 8 - Input: Bosphorus 4Kbadc1428425670SE +/- 0.26, N = 363.8962.9562.9362.501. (CXX) g++ options: -march=native -mno-avx -mavx2 -mavx512f -mavx512bw -mavx512dq

SVT-AV1

Encoder Mode: Preset 4 - Input: Bosphorus 1080p

OpenBenchmarking.orgFrames Per Second, More Is BetterSVT-AV1 2.0Encoder Mode: Preset 4 - Input: Bosphorus 1080pdacb510152025SE +/- 0.04, N = 319.3219.2119.1718.921. (CXX) g++ options: -march=native -mno-avx -mavx2 -mavx512f -mavx512bw -mavx512dq

SVT-AV1

Encoder Mode: Preset 4 - Input: Bosphorus 4K

OpenBenchmarking.orgFrames Per Second, More Is BetterSVT-AV1 2.0Encoder Mode: Preset 4 - Input: Bosphorus 4Kbcda246810SE +/- 0.005, N = 36.7876.7116.6606.6471. (CXX) g++ options: -march=native -mno-avx -mavx2 -mavx512f -mavx512bw -mavx512dq

RocksDB

Test: Random Fill

OpenBenchmarking.orgOp/s, More Is BetterRocksDB 9.0Test: Random Fillabdc200K400K600K800K1000KSE +/- 1329.24, N = 37906037839677771157743841. (CXX) g++ options: -O3 -march=native -pthread -fno-builtin-memcmp -fno-rtti -lpthread

VVenC

Video Input: Bosphorus 1080p - Video Preset: Faster

OpenBenchmarking.orgFrames Per Second, More Is BetterVVenC 1.11Video Input: Bosphorus 1080p - Video Preset: Fasteradcb918273645SE +/- 0.15, N = 341.2841.0040.9240.461. (CXX) g++ options: -O3 -flto=auto -fno-fat-lto-objects

VVenC

Video Input: Bosphorus 4K - Video Preset: Faster

OpenBenchmarking.orgFrames Per Second, More Is BetterVVenC 1.11Video Input: Bosphorus 4K - Video Preset: Fasterdbca48121620SE +/- 0.03, N = 314.8614.8214.7814.571. (CXX) g++ options: -O3 -flto=auto -fno-fat-lto-objects

JPEG-XL Decoding libjxl

CPU Threads: 1

OpenBenchmarking.orgMP/s, More Is BetterJPEG-XL Decoding libjxl 0.10.1CPU Threads: 1abcd1428425670SE +/- 0.12, N = 364.0363.3063.0462.82

Neural Magic DeepSparse

Model: ResNet-50, Sparse INT8 - Scenario: Asynchronous Multi-Stream

OpenBenchmarking.orgitems/sec, More Is BetterNeural Magic DeepSparse 1.7Model: ResNet-50, Sparse INT8 - Scenario: Asynchronous Multi-Streambdca400800120016002000SE +/- 6.40, N = 32032.962014.581998.611994.39

Neural Magic DeepSparse

Model: ResNet-50, Sparse INT8 - Scenario: Asynchronous Multi-Stream

OpenBenchmarking.orgms/batch, Fewer Is BetterNeural Magic DeepSparse 1.7Model: ResNet-50, Sparse INT8 - Scenario: Asynchronous Multi-Streambdca246810SE +/- 0.0186, N = 35.88985.94175.98886.0030

RocksDB

Test: Update Random

OpenBenchmarking.orgOp/s, More Is BetterRocksDB 9.0Test: Update Randomabcd140K280K420K560K700KSE +/- 1458.06, N = 36711216667156609786585201. (CXX) g++ options: -O3 -march=native -pthread -fno-builtin-memcmp -fno-rtti -lpthread

OpenVINO

Model: Vehicle Detection FP16 - Device: CPU

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2024.0Model: Vehicle Detection FP16 - Device: CPUdabc510152025SE +/- 0.01, N = 319.8419.9420.0420.21MIN: 11.38 / MAX: 34.24MIN: 8.87 / MAX: 42.42MIN: 12.81 / MAX: 37.51MIN: 9.28 / MAX: 49.091. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl

OpenVINO

Model: Vehicle Detection FP16 - Device: CPU

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2024.0Model: Vehicle Detection FP16 - Device: CPUdabc130260390520650SE +/- 0.39, N = 3603.96600.94598.03592.961. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl

srsRAN Project

Test: PUSCH Processor Benchmark, Throughput Total

OpenBenchmarking.orgMbps, More Is BettersrsRAN Project 23.10.1-20240219Test: PUSCH Processor Benchmark, Throughput Totalbacd400800120016002000SE +/- 1.62, N = 31889.21888.21886.91857.0MIN: 1137.6MIN: 1136.3MIN: 1135 / MAX: 1889.2MIN: 1145.41. (CXX) g++ options: -march=native -mavx2 -mavx -msse4.1 -mfma -O3 -fno-trapping-math -fno-math-errno -ldl

SVT-AV1

Encoder Mode: Preset 8 - Input: Bosphorus 1080p

OpenBenchmarking.orgFrames Per Second, More Is BetterSVT-AV1 2.0Encoder Mode: Preset 8 - Input: Bosphorus 1080pdcba306090120150SE +/- 0.72, N = 3133.78133.27131.93131.721. (CXX) g++ options: -march=native -mno-avx -mavx2 -mavx512f -mavx512bw -mavx512dq

RocksDB

Test: Read While Writing

OpenBenchmarking.orgOp/s, More Is BetterRocksDB 9.0Test: Read While Writingdcba1.2M2.4M3.6M4.8M6MSE +/- 38492.46, N = 356336135592760558653955468011. (CXX) g++ options: -O3 -march=native -pthread -fno-builtin-memcmp -fno-rtti -lpthread

OSPRay Studio

Camera: 1 - Resolution: 4K - Samples Per Pixel: 16 - Renderer: Path Tracer - Acceleration: CPU

OpenBenchmarking.orgms, Fewer Is BetterOSPRay Studio 1.0Camera: 1 - Resolution: 4K - Samples Per Pixel: 16 - Renderer: Path Tracer - Acceleration: CPUbdca20K40K60K80K100KSE +/- 155.86, N = 376995770737751178061

Neural Magic DeepSparse

Model: NLP Text Classification, BERT base uncased SST2, Sparse INT8 - Scenario: Synchronous Single-Stream

OpenBenchmarking.orgms/batch, Fewer Is BetterNeural Magic DeepSparse 1.7Model: NLP Text Classification, BERT base uncased SST2, Sparse INT8 - Scenario: Synchronous Single-Streamdacb1.17262.34523.51784.69045.863SE +/- 0.0247, N = 35.14085.14355.17875.2115

Neural Magic DeepSparse

Model: NLP Text Classification, BERT base uncased SST2, Sparse INT8 - Scenario: Synchronous Single-Stream

OpenBenchmarking.orgitems/sec, More Is BetterNeural Magic DeepSparse 1.7Model: NLP Text Classification, BERT base uncased SST2, Sparse INT8 - Scenario: Synchronous Single-Streamdacb4080120160200SE +/- 0.92, N = 3194.38194.29192.98191.76

RocksDB

Test: Sequential Fill

OpenBenchmarking.orgOp/s, More Is BetterRocksDB 9.0Test: Sequential Filladcb200K400K600K800K1000KSE +/- 4090.17, N = 39204309191489088829081771. (CXX) g++ options: -O3 -march=native -pthread -fno-builtin-memcmp -fno-rtti -lpthread

OSPRay Studio

Camera: 3 - Resolution: 4K - Samples Per Pixel: 32 - Renderer: Path Tracer - Acceleration: CPU

OpenBenchmarking.orgms, Fewer Is BetterOSPRay Studio 1.0Camera: 3 - Resolution: 4K - Samples Per Pixel: 32 - Renderer: Path Tracer - Acceleration: CPUdbca40K80K120K160K200KSE +/- 268.88, N = 3175129175496175767177445

OSPRay Studio

Camera: 2 - Resolution: 4K - Samples Per Pixel: 32 - Renderer: Path Tracer - Acceleration: CPU

OpenBenchmarking.orgms, Fewer Is BetterOSPRay Studio 1.0Camera: 2 - Resolution: 4K - Samples Per Pixel: 32 - Renderer: Path Tracer - Acceleration: CPUdcab30K60K90K120K150KSE +/- 88.33, N = 3151620152510152653153552

srsRAN Project

Test: PDSCH Processor Benchmark, Throughput Thread

OpenBenchmarking.orgMbps, More Is BettersrsRAN Project 23.10.1-20240219Test: PDSCH Processor Benchmark, Throughput Threaddcab130260390520650SE +/- 1.71, N = 3607.8605.5603.7600.21. (CXX) g++ options: -march=native -mavx2 -mavx -msse4.1 -mfma -O3 -fno-trapping-math -fno-math-errno -ldl

oneDNN

Harness: IP Shapes 1D - Engine: CPU

OpenBenchmarking.orgms, Fewer Is BetteroneDNN 3.4Harness: IP Shapes 1D - Engine: CPUbdac0.29870.59740.89611.19481.4935SE +/- 0.00673, N = 31.311011.313331.314741.32743MIN: 1.27MIN: 1.27MIN: 1.27MIN: 1.261. (CXX) g++ options: -O3 -march=native -fopenmp -msse4.1 -fPIC -pie -ldl

OpenVINO

Model: Noise Suppression Poconet-Like FP16 - Device: CPU

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2024.0Model: Noise Suppression Poconet-Like FP16 - Device: CPUbdca2004006008001000SE +/- 3.03, N = 31046.301043.821041.671033.671. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl

OpenVINO

Model: Face Detection FP16 - Device: CPU

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2024.0Model: Face Detection FP16 - Device: CPUbdac246810SE +/- 0.01, N = 37.687.647.607.591. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl

JPEG-XL libjxl

Input: PNG - Quality: 100

OpenBenchmarking.orgMP/s, More Is BetterJPEG-XL libjxl 0.10.1Input: PNG - Quality: 100abcd714212835SE +/- 0.01, N = 327.6527.5727.4927.341. (CXX) g++ options: -fno-rtti -O3 -fPIE -pie -lm

OpenVINO

Model: Noise Suppression Poconet-Like FP16 - Device: CPU

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2024.0Model: Noise Suppression Poconet-Like FP16 - Device: CPUbdca3691215SE +/- 0.03, N = 311.4111.4411.4611.54MIN: 7.86 / MAX: 31.96MIN: 9.06 / MAX: 31.81MIN: 6.19 / MAX: 32.08MIN: 6.61 / MAX: 30.791. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl

OSPRay Studio

Camera: 2 - Resolution: 1080p - Samples Per Pixel: 32 - Renderer: Path Tracer - Acceleration: CPU

OpenBenchmarking.orgms, Fewer Is BetterOSPRay Studio 1.0Camera: 2 - Resolution: 1080p - Samples Per Pixel: 32 - Renderer: Path Tracer - Acceleration: CPUbadc9K18K27K36K45KSE +/- 250.02, N = 341598418394195942050

OpenVINO

Model: Handwritten English Recognition FP16-INT8 - Device: CPU

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2024.0Model: Handwritten English Recognition FP16-INT8 - Device: CPUbcad1326395265SE +/- 0.01, N = 356.3056.6456.8756.90MIN: 52.17 / MAX: 69.13MIN: 36.21 / MAX: 73.18MIN: 35.77 / MAX: 72.74MIN: 51.72 / MAX: 72.171. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl

OpenVINO

Model: Handwritten English Recognition FP16-INT8 - Device: CPU

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2024.0Model: Handwritten English Recognition FP16-INT8 - Device: CPUbcad90180270360450SE +/- 0.10, N = 3426.03423.46421.72421.541. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl

Google Draco

Model: Church Facade

OpenBenchmarking.orgms, Fewer Is BetterGoogle Draco 1.5.6Model: Church Facadeabcd15003000450060007500SE +/- 9.91, N = 370237034709270961. (CXX) g++ options: -O3

OpenVINO

Model: Age Gender Recognition Retail 0013 FP16 - Device: CPU

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2024.0Model: Age Gender Recognition Retail 0013 FP16 - Device: CPUabdc0.22050.4410.66150.8821.1025SE +/- 0.00, N = 30.970.970.970.98MIN: 0.66 / MAX: 15.15MIN: 0.57 / MAX: 14.08MIN: 0.55 / MAX: 13.67MIN: 0.53 / MAX: 16.951. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl

OpenVINO

Model: Face Detection FP16 - Device: CPU

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2024.0Model: Face Detection FP16 - Device: CPUbdac30060090012001500SE +/- 0.80, N = 31547.311553.541558.811563.15MIN: 1403.59 / MAX: 1636.72MIN: 1365.71 / MAX: 1635.16MIN: 1416.22 / MAX: 1644.19MIN: 1369.79 / MAX: 1663.371. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl

oneDNN

Harness: Convolution Batch Shapes Auto - Engine: CPU

OpenBenchmarking.orgms, Fewer Is BetteroneDNN 3.4Harness: Convolution Batch Shapes Auto - Engine: CPUbcda0.61611.23221.84832.46443.0805SE +/- 0.00421, N = 32.711352.731922.733522.73804MIN: 2.65MIN: 2.66MIN: 2.67MIN: 2.661. (CXX) g++ options: -O3 -march=native -fopenmp -msse4.1 -fPIC -pie -ldl

oneDNN

Harness: Deconvolution Batch shapes_3d - Engine: CPU

OpenBenchmarking.orgms, Fewer Is BetteroneDNN 3.4Harness: Deconvolution Batch shapes_3d - Engine: CPUdbca0.53211.06421.59632.12842.6605SE +/- 0.00819, N = 32.343462.346022.361382.36511MIN: 2.25MIN: 2.28MIN: 2.25MIN: 2.281. (CXX) g++ options: -O3 -march=native -fopenmp -msse4.1 -fPIC -pie -ldl

Neural Magic DeepSparse

Model: BERT-Large, NLP Question Answering, Sparse INT8 - Scenario: Asynchronous Multi-Stream

OpenBenchmarking.orgitems/sec, More Is BetterNeural Magic DeepSparse 1.7Model: BERT-Large, NLP Question Answering, Sparse INT8 - Scenario: Asynchronous Multi-Streambcda70140210280350SE +/- 0.53, N = 3337.45334.75334.72334.39

Neural Magic DeepSparse

Model: BERT-Large, NLP Question Answering, Sparse INT8 - Scenario: Asynchronous Multi-Stream

OpenBenchmarking.orgms/batch, Fewer Is BetterNeural Magic DeepSparse 1.7Model: BERT-Large, NLP Question Answering, Sparse INT8 - Scenario: Asynchronous Multi-Streambcda816243240SE +/- 0.06, N = 335.5435.8135.8135.85

Neural Magic DeepSparse

Model: CV Detection, YOLOv5s COCO, Sparse INT8 - Scenario: Asynchronous Multi-Stream

OpenBenchmarking.orgms/batch, Fewer Is BetterNeural Magic DeepSparse 1.7Model: CV Detection, YOLOv5s COCO, Sparse INT8 - Scenario: Asynchronous Multi-Streambcad20406080100SE +/- 0.16, N = 378.8279.2579.4579.51

OSPRay

Benchmark: particle_volume/pathtracer/real_time

OpenBenchmarking.orgItems Per Second, More Is BetterOSPRay 3.1Benchmark: particle_volume/pathtracer/real_timeacbd306090120150SE +/- 0.22, N = 3156.40155.83155.37155.05

oneDNN

Harness: Recurrent Neural Network Inference - Engine: CPU

OpenBenchmarking.orgms, Fewer Is BetteroneDNN 3.4Harness: Recurrent Neural Network Inference - Engine: CPUbcad140280420560700SE +/- 0.39, N = 3636.77637.17638.13642.19MIN: 632.58MIN: 632.47MIN: 634.67MIN: 633.41. (CXX) g++ options: -O3 -march=native -fopenmp -msse4.1 -fPIC -pie -ldl

VVenC

Video Input: Bosphorus 1080p - Video Preset: Fast

OpenBenchmarking.orgFrames Per Second, More Is BetterVVenC 1.11Video Input: Bosphorus 1080p - Video Preset: Fastadbc510152025SE +/- 0.03, N = 319.6219.5319.4919.461. (CXX) g++ options: -O3 -flto=auto -fno-fat-lto-objects

Neural Magic DeepSparse

Model: CV Detection, YOLOv5s COCO, Sparse INT8 - Scenario: Asynchronous Multi-Stream

OpenBenchmarking.orgitems/sec, More Is BetterNeural Magic DeepSparse 1.7Model: CV Detection, YOLOv5s COCO, Sparse INT8 - Scenario: Asynchronous Multi-Streambcad306090120150SE +/- 0.31, N = 3151.98151.26150.83150.73

Neural Magic DeepSparse

Model: NLP Text Classification, DistilBERT mnli - Scenario: Synchronous Single-Stream

OpenBenchmarking.orgms/batch, Fewer Is BetterNeural Magic DeepSparse 1.7Model: NLP Text Classification, DistilBERT mnli - Scenario: Synchronous Single-Streamacbd3691215SE +/- 0.0057, N = 38.89218.91898.93178.9625

Neural Magic DeepSparse

Model: NLP Text Classification, DistilBERT mnli - Scenario: Synchronous Single-Stream

OpenBenchmarking.orgitems/sec, More Is BetterNeural Magic DeepSparse 1.7Model: NLP Text Classification, DistilBERT mnli - Scenario: Synchronous Single-Streamacbd306090120150SE +/- 0.07, N = 3112.33112.00111.84111.45

OSPRay Studio

Camera: 2 - Resolution: 4K - Samples Per Pixel: 16 - Renderer: Path Tracer - Acceleration: CPU

OpenBenchmarking.orgms, Fewer Is BetterOSPRay Studio 1.0Camera: 2 - Resolution: 4K - Samples Per Pixel: 16 - Renderer: Path Tracer - Acceleration: CPUbdac20K40K60K80K100KSE +/- 110.73, N = 378304783547878878919

Chaos Group V-RAY

Mode: CPU

OpenBenchmarking.orgvsamples, More Is BetterChaos Group V-RAY 6.0Mode: CPUbdca10K20K30K40K50KSE +/- 195.74, N = 344634446334437544287

JPEG-XL libjxl

Input: JPEG - Quality: 100

OpenBenchmarking.orgMP/s, More Is BetterJPEG-XL libjxl 0.10.1Input: JPEG - Quality: 100abcd612182430SE +/- 0.03, N = 327.5327.3227.3227.321. (CXX) g++ options: -fno-rtti -O3 -fPIE -pie -lm

OpenVINO

Model: Road Segmentation ADAS FP16 - Device: CPU

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2024.0Model: Road Segmentation ADAS FP16 - Device: CPUbacd1632486480SE +/- 0.02, N = 370.2370.4870.6670.76MIN: 43.23 / MAX: 122.75MIN: 43.7 / MAX: 128.66MIN: 24.84 / MAX: 127.05MIN: 41.53 / MAX: 122.261. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl

OpenVINO

Model: Road Segmentation ADAS FP16 - Device: CPU

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2024.0Model: Road Segmentation ADAS FP16 - Device: CPUbacd4080120160200SE +/- 0.04, N = 3170.68170.10169.65169.411. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl

Neural Magic DeepSparse

Model: NLP Document Classification, oBERT base uncased on IMDB - Scenario: Synchronous Single-Stream

OpenBenchmarking.orgms/batch, Fewer Is BetterNeural Magic DeepSparse 1.7Model: NLP Document Classification, oBERT base uncased on IMDB - Scenario: Synchronous Single-Streambcda1224364860SE +/- 0.05, N = 354.0154.3354.3454.42

Neural Magic DeepSparse

Model: NLP Document Classification, oBERT base uncased on IMDB - Scenario: Synchronous Single-Stream

OpenBenchmarking.orgitems/sec, More Is BetterNeural Magic DeepSparse 1.7Model: NLP Document Classification, oBERT base uncased on IMDB - Scenario: Synchronous Single-Streambcda510152025SE +/- 0.02, N = 318.5118.4018.4018.37

Neural Magic DeepSparse

Model: ResNet-50, Sparse INT8 - Scenario: Synchronous Single-Stream

OpenBenchmarking.orgms/batch, Fewer Is BetterNeural Magic DeepSparse 1.7Model: ResNet-50, Sparse INT8 - Scenario: Synchronous Single-Streambcad0.28260.56520.84781.13041.413SE +/- 0.0030, N = 31.24681.24781.24911.2559

OpenVINO

Model: Handwritten English Recognition FP16 - Device: CPU

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2024.0Model: Handwritten English Recognition FP16 - Device: CPUbcad1428425670SE +/- 0.11, N = 363.0863.4063.4663.54MIN: 42.84 / MAX: 83.6MIN: 37.56 / MAX: 90.19MIN: 45.07 / MAX: 85.2MIN: 41.89 / MAX: 89.711. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl

OpenVINO

Model: Handwritten English Recognition FP16 - Device: CPU

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2024.0Model: Handwritten English Recognition FP16 - Device: CPUbcad80160240320400SE +/- 0.65, N = 3380.22378.34377.84377.491. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl

Neural Magic DeepSparse

Model: CV Classification, ResNet-50 ImageNet - Scenario: Synchronous Single-Stream

OpenBenchmarking.orgitems/sec, More Is BetterNeural Magic DeepSparse 1.7Model: CV Classification, ResNet-50 ImageNet - Scenario: Synchronous Single-Streamadcb306090120150SE +/- 0.15, N = 3156.43155.90155.79155.31

RocksDB

Test: Random Read

OpenBenchmarking.orgOp/s, More Is BetterRocksDB 9.0Test: Random Readabdc30M60M90M120M150MSE +/- 374266.35, N = 31459628911458937531455168741449217631. (CXX) g++ options: -O3 -march=native -pthread -fno-builtin-memcmp -fno-rtti -lpthread

Neural Magic DeepSparse

Model: ResNet-50, Sparse INT8 - Scenario: Synchronous Single-Stream

OpenBenchmarking.orgitems/sec, More Is BetterNeural Magic DeepSparse 1.7Model: ResNet-50, Sparse INT8 - Scenario: Synchronous Single-Streambcad2004006008001000SE +/- 1.90, N = 3800.05799.42798.47794.39

Neural Magic DeepSparse

Model: CV Classification, ResNet-50 ImageNet - Scenario: Synchronous Single-Stream

OpenBenchmarking.orgms/batch, Fewer Is BetterNeural Magic DeepSparse 1.7Model: CV Classification, ResNet-50 ImageNet - Scenario: Synchronous Single-Streamadcb246810SE +/- 0.0061, N = 36.38466.40596.41036.4293

OSPRay Studio

Camera: 3 - Resolution: 1080p - Samples Per Pixel: 32 - Renderer: Path Tracer - Acceleration: CPU

OpenBenchmarking.orgms, Fewer Is BetterOSPRay Studio 1.0Camera: 3 - Resolution: 1080p - Samples Per Pixel: 32 - Renderer: Path Tracer - Acceleration: CPUadcb10K20K30K40K50KSE +/- 129.36, N = 347495475974768647827

OSPRay Studio

Camera: 2 - Resolution: 1080p - Samples Per Pixel: 1 - Renderer: Path Tracer - Acceleration: CPU

OpenBenchmarking.orgms, Fewer Is BetterOSPRay Studio 1.0Camera: 2 - Resolution: 1080p - Samples Per Pixel: 1 - Renderer: Path Tracer - Acceleration: CPUbcda2004006008001000SE +/- 0.67, N = 31150115211531158

Google Draco

Model: Lion

OpenBenchmarking.orgms, Fewer Is BetterGoogle Draco 1.5.6Model: Lionadcb12002400360048006000SE +/- 15.72, N = 353285347536353651. (CXX) g++ options: -O3

OpenVINO

Model: Age Gender Recognition Retail 0013 FP16 - Device: CPU

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2024.0Model: Age Gender Recognition Retail 0013 FP16 - Device: CPUdabc5K10K15K20K25KSE +/- 23.00, N = 324496.8424478.3124434.5124329.211. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl

OpenVINO

Model: Person Vehicle Bike Detection FP16 - Device: CPU

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2024.0Model: Person Vehicle Bike Detection FP16 - Device: CPUacdb3691215SE +/- 0.05, N = 313.3813.3913.4013.47MIN: 7.23 / MAX: 35.43MIN: 7.1 / MAX: 31.9MIN: 8.15 / MAX: 34.65MIN: 7.33 / MAX: 31.021. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl

OpenVINO

Model: Person Vehicle Bike Detection FP16 - Device: CPU

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2024.0Model: Person Vehicle Bike Detection FP16 - Device: CPUacdb2004006008001000SE +/- 3.73, N = 3895.25894.57894.41889.361. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl

OpenVINO

Model: Weld Porosity Detection FP16 - Device: CPU

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2024.0Model: Weld Porosity Detection FP16 - Device: CPUbadc48121620SE +/- 0.01, N = 316.6516.6916.7216.76MIN: 10 / MAX: 33.75MIN: 13.14 / MAX: 33.56MIN: 9.04 / MAX: 25.27MIN: 8.74 / MAX: 34.171. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl

OpenVINO

Model: Age Gender Recognition Retail 0013 FP16-INT8 - Device: CPU

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2024.0Model: Age Gender Recognition Retail 0013 FP16-INT8 - Device: CPUdabc10K20K30K40K50KSE +/- 27.92, N = 344596.3544518.2044461.0244306.101. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl

OpenVINO

Model: Weld Porosity Detection FP16 - Device: CPU

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2024.0Model: Weld Porosity Detection FP16 - Device: CPUbadc160320480640800SE +/- 0.61, N = 3719.93718.20716.84715.281. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl

srsRAN Project

Test: PUSCH Processor Benchmark, Throughput Thread

OpenBenchmarking.orgMbps, More Is BettersrsRAN Project 23.10.1-20240219Test: PUSCH Processor Benchmark, Throughput Threadbcda4080120160200SE +/- 0.70, N = 3178.7178.6177.9177.6MIN: 112.2MIN: 112.6 / MAX: 179.4MIN: 110.9MIN: 113.31. (CXX) g++ options: -march=native -mavx2 -mavx -msse4.1 -mfma -O3 -fno-trapping-math -fno-math-errno -ldl

OSPRay Studio

Camera: 3 - Resolution: 1080p - Samples Per Pixel: 1 - Renderer: Path Tracer - Acceleration: CPU

OpenBenchmarking.orgms, Fewer Is BetterOSPRay Studio 1.0Camera: 3 - Resolution: 1080p - Samples Per Pixel: 1 - Renderer: Path Tracer - Acceleration: CPUdcab30060090012001500SE +/- 4.41, N = 31330133313361338

Neural Magic DeepSparse

Model: NLP Token Classification, BERT base uncased conll2003 - Scenario: Asynchronous Multi-Stream

OpenBenchmarking.orgitems/sec, More Is BetterNeural Magic DeepSparse 1.7Model: NLP Token Classification, BERT base uncased conll2003 - Scenario: Asynchronous Multi-Streambdac612182430SE +/- 0.03, N = 326.8626.8126.7426.70

OSPRay Studio

Camera: 2 - Resolution: 1080p - Samples Per Pixel: 16 - Renderer: Path Tracer - Acceleration: CPU

OpenBenchmarking.orgms, Fewer Is BetterOSPRay Studio 1.0Camera: 2 - Resolution: 1080p - Samples Per Pixel: 16 - Renderer: Path Tracer - Acceleration: CPUbdac4K8K12K16K20KSE +/- 16.33, N = 318471184881854318579

OpenVINO

Model: Person Detection FP16 - Device: CPU

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2024.0Model: Person Detection FP16 - Device: CPUabcd1632486480SE +/- 0.04, N = 370.0169.9569.8369.611. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl

Neural Magic DeepSparse

Model: ResNet-50, Baseline - Scenario: Synchronous Single-Stream

OpenBenchmarking.orgms/batch, Fewer Is BetterNeural Magic DeepSparse 1.7Model: ResNet-50, Baseline - Scenario: Synchronous Single-Streamadcb246810SE +/- 0.0148, N = 36.39276.40016.41556.4286

OpenVINO

Model: Person Detection FP16 - Device: CPU

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2024.0Model: Person Detection FP16 - Device: CPUabcd4080120160200SE +/- 0.11, N = 3171.22171.43171.67172.18MIN: 130.32 / MAX: 233.99MIN: 140.26 / MAX: 224.54MIN: 132.19 / MAX: 231.16MIN: 138.53 / MAX: 224.81. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl

Neural Magic DeepSparse

Model: BERT-Large, NLP Question Answering, Sparse INT8 - Scenario: Synchronous Single-Stream

OpenBenchmarking.orgms/batch, Fewer Is BetterNeural Magic DeepSparse 1.7Model: BERT-Large, NLP Question Answering, Sparse INT8 - Scenario: Synchronous Single-Streambcad3691215SE +/- 0.02, N = 313.1313.1413.1713.20

Neural Magic DeepSparse

Model: ResNet-50, Baseline - Scenario: Synchronous Single-Stream

OpenBenchmarking.orgitems/sec, More Is BetterNeural Magic DeepSparse 1.7Model: ResNet-50, Baseline - Scenario: Synchronous Single-Streamadcb306090120150SE +/- 0.37, N = 3156.21156.04155.68155.34

Neural Magic DeepSparse

Model: BERT-Large, NLP Question Answering, Sparse INT8 - Scenario: Synchronous Single-Stream

OpenBenchmarking.orgitems/sec, More Is BetterNeural Magic DeepSparse 1.7Model: BERT-Large, NLP Question Answering, Sparse INT8 - Scenario: Synchronous Single-Streambcad20406080100SE +/- 0.14, N = 376.1276.0675.8875.70

OpenVINO

Model: Face Detection Retail FP16-INT8 - Device: CPU

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2024.0Model: Face Detection Retail FP16-INT8 - Device: CPUbacd0.83481.66962.50443.33924.174SE +/- 0.01, N = 33.693.713.713.71MIN: 2.28 / MAX: 15.95MIN: 2.14 / MAX: 26.73MIN: 2.23 / MAX: 18MIN: 2.38 / MAX: 15.941. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl

Neural Magic DeepSparse

Model: CV Detection, YOLOv5s COCO, Sparse INT8 - Scenario: Synchronous Single-Stream

OpenBenchmarking.orgitems/sec, More Is BetterNeural Magic DeepSparse 1.7Model: CV Detection, YOLOv5s COCO, Sparse INT8 - Scenario: Synchronous Single-Streamabdc20406080100SE +/- 0.05, N = 399.3399.2299.0598.80

Neural Magic DeepSparse

Model: CV Detection, YOLOv5s COCO, Sparse INT8 - Scenario: Synchronous Single-Stream

OpenBenchmarking.orgms/batch, Fewer Is BetterNeural Magic DeepSparse 1.7Model: CV Detection, YOLOv5s COCO, Sparse INT8 - Scenario: Synchronous Single-Streamabdc3691215SE +/- 0.01, N = 310.0610.0710.0910.12

Neural Magic DeepSparse

Model: NLP Text Classification, DistilBERT mnli - Scenario: Asynchronous Multi-Stream

OpenBenchmarking.orgitems/sec, More Is BetterNeural Magic DeepSparse 1.7Model: NLP Text Classification, DistilBERT mnli - Scenario: Asynchronous Multi-Streambadc50100150200250SE +/- 0.20, N = 3224.20224.16223.55223.00

Neural Magic DeepSparse

Model: NLP Text Classification, DistilBERT mnli - Scenario: Asynchronous Multi-Stream

OpenBenchmarking.orgms/batch, Fewer Is BetterNeural Magic DeepSparse 1.7Model: NLP Text Classification, DistilBERT mnli - Scenario: Asynchronous Multi-Streambadc1224364860SE +/- 0.05, N = 353.4753.4853.6353.76

Neural Magic DeepSparse

Model: Llama2 Chat 7b Quantized - Scenario: Asynchronous Multi-Stream

OpenBenchmarking.orgms/batch, Fewer Is BetterNeural Magic DeepSparse 1.7Model: Llama2 Chat 7b Quantized - Scenario: Asynchronous Multi-Streamacdb13002600390052006500SE +/- 9.69, N = 35873.275896.595897.285904.28

OpenVINO

Model: Face Detection Retail FP16-INT8 - Device: CPU

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2024.0Model: Face Detection Retail FP16-INT8 - Device: CPUbdca7001400210028003500SE +/- 3.42, N = 33241.353230.253228.193224.641. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl

Neural Magic DeepSparse

Model: Llama2 Chat 7b Quantized - Scenario: Asynchronous Multi-Stream

OpenBenchmarking.orgitems/sec, More Is BetterNeural Magic DeepSparse 1.7Model: Llama2 Chat 7b Quantized - Scenario: Asynchronous Multi-Streamacdb0.42240.84481.26721.68962.112SE +/- 0.0031, N = 31.87721.87051.87011.8677

OSPRay

Benchmark: particle_volume/ao/real_time

OpenBenchmarking.orgItems Per Second, More Is BetterOSPRay 3.1Benchmark: particle_volume/ao/real_timedacb3691215SE +/- 0.01, N = 310.3010.2810.2810.25

OpenVINO

Model: Weld Porosity Detection FP16-INT8 - Device: CPU

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2024.0Model: Weld Porosity Detection FP16-INT8 - Device: CPUbdac48121620SE +/- 0.01, N = 314.4914.5114.5214.56MIN: 8.36 / MAX: 28.02MIN: 8.58 / MAX: 29.8MIN: 8.14 / MAX: 28.07MIN: 8.28 / MAX: 27.621. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl

OpenVINO

Model: Face Detection FP16-INT8 - Device: CPU

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2024.0Model: Face Detection FP16-INT8 - Device: CPUdbca48121620SE +/- 0.01, N = 316.7416.7316.6616.661. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl

OSPRay

Benchmark: gravity_spheres_volume/dim_512/scivis/real_time

OpenBenchmarking.orgItems Per Second, More Is BetterOSPRay 3.1Benchmark: gravity_spheres_volume/dim_512/scivis/real_timeacdb1.03472.06943.10414.13885.1735SE +/- 0.00963, N = 34.598824.592664.580814.57706

OpenVINO

Model: Weld Porosity Detection FP16-INT8 - Device: CPU

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2024.0Model: Weld Porosity Detection FP16-INT8 - Device: CPUbdac400800120016002000SE +/- 0.87, N = 31654.721652.911652.051647.131. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl

OSPRay

Benchmark: gravity_spheres_volume/dim_512/ao/real_time

OpenBenchmarking.orgItems Per Second, More Is BetterOSPRay 3.1Benchmark: gravity_spheres_volume/dim_512/ao/real_timeadcb1.10312.20623.30934.41245.5155SE +/- 0.01074, N = 34.902574.899284.886084.88053

OpenVINO

Model: Person Detection FP32 - Device: CPU

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2024.0Model: Person Detection FP32 - Device: CPUbcda4080120160200SE +/- 0.12, N = 3171.16171.29171.54171.92MIN: 129.54 / MAX: 225.82MIN: 134.25 / MAX: 225.4MIN: 135.7 / MAX: 226.04MIN: 129.51 / MAX: 227.571. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl

OSPRay

Benchmark: particle_volume/scivis/real_time

OpenBenchmarking.orgItems Per Second, More Is BetterOSPRay 3.1Benchmark: particle_volume/scivis/real_timedcba3691215SE +/- 0.01, N = 310.1810.1510.1410.14

OSPRay Studio

Camera: 1 - Resolution: 4K - Samples Per Pixel: 32 - Renderer: Path Tracer - Acceleration: CPU

OpenBenchmarking.orgms, Fewer Is BetterOSPRay Studio 1.0Camera: 1 - Resolution: 4K - Samples Per Pixel: 32 - Renderer: Path Tracer - Acceleration: CPUbacd30K60K90K120K150KSE +/- 64.70, N = 3150040150375150381150694

OpenVINO

Model: Person Detection FP32 - Device: CPU

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2024.0Model: Person Detection FP32 - Device: CPUbcda1632486480SE +/- 0.06, N = 370.0169.9969.8669.731. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl

Primesieve

Length: 1e13

OpenBenchmarking.orgSeconds, Fewer Is BetterPrimesieve 12.1Length: 1e13bdac20406080100SE +/- 0.05, N = 376.9277.0677.1577.211. (CXX) g++ options: -O3

OSPRay

Benchmark: gravity_spheres_volume/dim_512/pathtracer/real_time

OpenBenchmarking.orgItems Per Second, More Is BetterOSPRay 3.1Benchmark: gravity_spheres_volume/dim_512/pathtracer/real_timeadbc246810SE +/- 0.00473, N = 37.466147.452207.448377.43821

OpenVINO

Model: Vehicle Detection FP16-INT8 - Device: CPU

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2024.0Model: Vehicle Detection FP16-INT8 - Device: CPUcbad3691215SE +/- 0.02, N = 310.8210.8310.8610.86MIN: 5.79 / MAX: 27.55MIN: 6.67 / MAX: 24.71MIN: 7.32 / MAX: 25.15MIN: 6.39 / MAX: 25.761. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl

OpenVINO

Model: Vehicle Detection FP16-INT8 - Device: CPU

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2024.0Model: Vehicle Detection FP16-INT8 - Device: CPUcbad2004006008001000SE +/- 1.89, N = 31107.641106.441104.031103.601. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl

oneDNN

Harness: IP Shapes 3D - Engine: CPU

OpenBenchmarking.orgms, Fewer Is BetteroneDNN 3.4Harness: IP Shapes 3D - Engine: CPUabcd0.79621.59242.38863.18483.981SE +/- 0.00386, N = 33.525693.525843.533233.53851MIN: 3.48MIN: 3.48MIN: 3.47MIN: 3.491. (CXX) g++ options: -O3 -march=native -fopenmp -msse4.1 -fPIC -pie -ldl

OpenVINO

Model: Road Segmentation ADAS FP16-INT8 - Device: CPU

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2024.0Model: Road Segmentation ADAS FP16-INT8 - Device: CPUdabc90180270360450SE +/- 0.36, N = 3428.55428.43428.18427.021. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl

OpenVINO

Model: Road Segmentation ADAS FP16-INT8 - Device: CPU

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2024.0Model: Road Segmentation ADAS FP16-INT8 - Device: CPUdabc714212835SE +/- 0.02, N = 327.9827.9928.0028.08MIN: 14.99 / MAX: 46.26MIN: 18.95 / MAX: 38.76MIN: 18.81 / MAX: 38.86MIN: 14.29 / MAX: 41.571. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl

OpenVINO

Model: Face Detection FP16-INT8 - Device: CPU

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2024.0Model: Face Detection FP16-INT8 - Device: CPUdbac150300450600750SE +/- 0.26, N = 3713.64713.72715.22716.16MIN: 667.56 / MAX: 731.59MIN: 658.61 / MAX: 738.08MIN: 664.62 / MAX: 729.04MIN: 661.6 / MAX: 732.111. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl

OSPRay Studio

Camera: 1 - Resolution: 1080p - Samples Per Pixel: 1 - Renderer: Path Tracer - Acceleration: CPU

OpenBenchmarking.orgms, Fewer Is BetterOSPRay Studio 1.0Camera: 1 - Resolution: 1080p - Samples Per Pixel: 1 - Renderer: Path Tracer - Acceleration: CPUdabc2004006008001000SE +/- 1.00, N = 31134113811381138

Neural Magic DeepSparse

Model: NLP Document Classification, oBERT base uncased on IMDB - Scenario: Asynchronous Multi-Stream

OpenBenchmarking.orgitems/sec, More Is BetterNeural Magic DeepSparse 1.7Model: NLP Document Classification, oBERT base uncased on IMDB - Scenario: Asynchronous Multi-Streamdacb612182430SE +/- 0.04, N = 326.6926.6926.6226.60

Neural Magic DeepSparse

Model: CV Segmentation, 90% Pruned YOLACT Pruned - Scenario: Asynchronous Multi-Stream

OpenBenchmarking.orgitems/sec, More Is BetterNeural Magic DeepSparse 1.7Model: CV Segmentation, 90% Pruned YOLACT Pruned - Scenario: Asynchronous Multi-Streamdbac714212835SE +/- 0.09, N = 330.4930.4930.4730.39

OpenVINO

Model: Machine Translation EN To DE FP16 - Device: CPU

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2024.0Model: Machine Translation EN To DE FP16 - Device: CPUbdac306090120150SE +/- 0.06, N = 3135.70135.96136.00136.15MIN: 109.99 / MAX: 155.71MIN: 110.6 / MAX: 155.59MIN: 118.61 / MAX: 153.61MIN: 73.11 / MAX: 161.851. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl

OSPRay Studio

Camera: 3 - Resolution: 4K - Samples Per Pixel: 16 - Renderer: Path Tracer - Acceleration: CPU

OpenBenchmarking.orgms, Fewer Is BetterOSPRay Studio 1.0Camera: 3 - Resolution: 4K - Samples Per Pixel: 16 - Renderer: Path Tracer - Acceleration: CPUcbad20K40K60K80K100KSE +/- 228.07, N = 390083901319036290380

Neural Magic DeepSparse

Model: NLP Document Classification, oBERT base uncased on IMDB - Scenario: Asynchronous Multi-Stream

OpenBenchmarking.orgms/batch, Fewer Is BetterNeural Magic DeepSparse 1.7Model: NLP Document Classification, oBERT base uncased on IMDB - Scenario: Asynchronous Multi-Streambadc100200300400500SE +/- 0.38, N = 3447.65448.71448.93449.12

OpenVINO

Model: Machine Translation EN To DE FP16 - Device: CPU

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2024.0Model: Machine Translation EN To DE FP16 - Device: CPUbdac20406080100SE +/- 0.03, N = 388.3388.1488.1388.051. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl

OSPRay Studio

Camera: 1 - Resolution: 4K - Samples Per Pixel: 1 - Renderer: Path Tracer - Acceleration: CPU

OpenBenchmarking.orgms, Fewer Is BetterOSPRay Studio 1.0Camera: 1 - Resolution: 4K - Samples Per Pixel: 1 - Renderer: Path Tracer - Acceleration: CPUbadc10002000300040005000SE +/- 4.41, N = 34521452845284535

Neural Magic DeepSparse

Model: NLP Token Classification, BERT base uncased conll2003 - Scenario: Asynchronous Multi-Stream

OpenBenchmarking.orgms/batch, Fewer Is BetterNeural Magic DeepSparse 1.7Model: NLP Token Classification, BERT base uncased conll2003 - Scenario: Asynchronous Multi-Streambadc100200300400500SE +/- 0.66, N = 3446.24446.84447.17447.58

OpenVINO

Model: Person Re-Identification Retail FP16 - Device: CPU

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2024.0Model: Person Re-Identification Retail FP16 - Device: CPUcdba3691215SE +/- 0.01, N = 39.979.979.9810.00MIN: 5.91 / MAX: 41.34MIN: 5.63 / MAX: 25.55MIN: 5.6 / MAX: 24.1MIN: 6.87 / MAX: 16.11. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl

OSPRay Studio

Camera: 3 - Resolution: 1080p - Samples Per Pixel: 16 - Renderer: Path Tracer - Acceleration: CPU

OpenBenchmarking.orgms, Fewer Is BetterOSPRay Studio 1.0Camera: 3 - Resolution: 1080p - Samples Per Pixel: 16 - Renderer: Path Tracer - Acceleration: CPUdbca5K10K15K20K25KSE +/- 35.14, N = 321385213982140921449

OpenVINO

Model: Person Re-Identification Retail FP16 - Device: CPU

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2024.0Model: Person Re-Identification Retail FP16 - Device: CPUdcba30060090012001500SE +/- 0.87, N = 31202.221202.091201.221198.641. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl

OSPRay Studio

Camera: 1 - Resolution: 1080p - Samples Per Pixel: 16 - Renderer: Path Tracer - Acceleration: CPU

OpenBenchmarking.orgms, Fewer Is BetterOSPRay Studio 1.0Camera: 1 - Resolution: 1080p - Samples Per Pixel: 16 - Renderer: Path Tracer - Acceleration: CPUdbac4K8K12K16K20KSE +/- 39.50, N = 318177181811820318229

OpenVINO

Model: Face Detection Retail FP16 - Device: CPU

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2024.0Model: Face Detection Retail FP16 - Device: CPUdcba5001000150020002500SE +/- 4.26, N = 32198.112197.982197.502192.091. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl

Neural Magic DeepSparse

Model: NLP Text Classification, BERT base uncased SST2, Sparse INT8 - Scenario: Asynchronous Multi-Stream

OpenBenchmarking.orgms/batch, Fewer Is BetterNeural Magic DeepSparse 1.7Model: NLP Text Classification, BERT base uncased SST2, Sparse INT8 - Scenario: Asynchronous Multi-Streamdbac48121620SE +/- 0.01, N = 317.4817.4917.5117.52

Neural Magic DeepSparse

Model: NLP Text Classification, BERT base uncased SST2, Sparse INT8 - Scenario: Asynchronous Multi-Stream

OpenBenchmarking.orgitems/sec, More Is BetterNeural Magic DeepSparse 1.7Model: NLP Text Classification, BERT base uncased SST2, Sparse INT8 - Scenario: Asynchronous Multi-Streamdbac150300450600750SE +/- 0.35, N = 3685.82685.45684.57684.02

Neural Magic DeepSparse

Model: CV Segmentation, 90% Pruned YOLACT Pruned - Scenario: Synchronous Single-Stream

OpenBenchmarking.orgms/batch, Fewer Is BetterNeural Magic DeepSparse 1.7Model: CV Segmentation, 90% Pruned YOLACT Pruned - Scenario: Synchronous Single-Streambdac1020304050SE +/- 0.03, N = 345.9646.0246.0646.08

Neural Magic DeepSparse

Model: CV Segmentation, 90% Pruned YOLACT Pruned - Scenario: Synchronous Single-Stream

OpenBenchmarking.orgitems/sec, More Is BetterNeural Magic DeepSparse 1.7Model: CV Segmentation, 90% Pruned YOLACT Pruned - Scenario: Synchronous Single-Streambdac510152025SE +/- 0.01, N = 321.7521.7221.7021.69

Neural Magic DeepSparse

Model: NLP Token Classification, BERT base uncased conll2003 - Scenario: Synchronous Single-Stream

OpenBenchmarking.orgitems/sec, More Is BetterNeural Magic DeepSparse 1.7Model: NLP Token Classification, BERT base uncased conll2003 - Scenario: Synchronous Single-Streambacd510152025SE +/- 0.01, N = 318.5218.4918.4918.48

Neural Magic DeepSparse

Model: NLP Token Classification, BERT base uncased conll2003 - Scenario: Synchronous Single-Stream

OpenBenchmarking.orgms/batch, Fewer Is BetterNeural Magic DeepSparse 1.7Model: NLP Token Classification, BERT base uncased conll2003 - Scenario: Synchronous Single-Streambacd1224364860SE +/- 0.02, N = 353.9754.0754.0854.10

Timed Linux Kernel Compilation

Build: allmodconfig

OpenBenchmarking.orgSeconds, Fewer Is BetterTimed Linux Kernel Compilation 6.8Build: allmodconfigcadb130260390520650SE +/- 0.92, N = 3596.51597.03597.83597.90

OSPRay Studio

Camera: 3 - Resolution: 4K - Samples Per Pixel: 1 - Renderer: Path Tracer - Acceleration: CPU

OpenBenchmarking.orgms, Fewer Is BetterOSPRay Studio 1.0Camera: 3 - Resolution: 4K - Samples Per Pixel: 1 - Renderer: Path Tracer - Acceleration: CPUadcb11002200330044005500SE +/- 4.04, N = 35330533153415342

VVenC

Video Input: Bosphorus 4K - Video Preset: Fast

OpenBenchmarking.orgFrames Per Second, More Is BetterVVenC 1.11Video Input: Bosphorus 4K - Video Preset: Fastbcda246810SE +/- 0.031, N = 37.0757.0667.0627.0601. (CXX) g++ options: -O3 -flto=auto -fno-fat-lto-objects

WavPack Audio Encoding

WAV To WavPack

OpenBenchmarking.orgSeconds, Fewer Is BetterWavPack Audio Encoding 5.7WAV To WavPackacbd0.99951.9992.99853.9984.9975SE +/- 0.000, N = 54.4334.4354.4384.442

OSPRay Studio

Camera: 2 - Resolution: 4K - Samples Per Pixel: 1 - Renderer: Path Tracer - Acceleration: CPU

OpenBenchmarking.orgms, Fewer Is BetterOSPRay Studio 1.0Camera: 2 - Resolution: 4K - Samples Per Pixel: 1 - Renderer: Path Tracer - Acceleration: CPUbcad10002000300040005000SE +/- 7.22, N = 34609461546164618

OSPRay Studio

Camera: 1 - Resolution: 1080p - Samples Per Pixel: 32 - Renderer: Path Tracer - Acceleration: CPU

OpenBenchmarking.orgms, Fewer Is BetterOSPRay Studio 1.0Camera: 1 - Resolution: 1080p - Samples Per Pixel: 32 - Renderer: Path Tracer - Acceleration: CPUdbac9K18K27K36K45KSE +/- 58.89, N = 341111411254116741189

Neural Magic DeepSparse

Model: CV Segmentation, 90% Pruned YOLACT Pruned - Scenario: Asynchronous Multi-Stream

OpenBenchmarking.orgms/batch, Fewer Is BetterNeural Magic DeepSparse 1.7Model: CV Segmentation, 90% Pruned YOLACT Pruned - Scenario: Asynchronous Multi-Streambdac90180270360450SE +/- 0.32, N = 3393.08393.39393.66393.82

OpenVINO

Model: Face Detection Retail FP16 - Device: CPU

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2024.0Model: Face Detection Retail FP16 - Device: CPUbcda1.22852.4573.68554.9146.1425SE +/- 0.01, N = 35.455.455.455.46MIN: 2.89 / MAX: 21.85MIN: 2.8 / MAX: 22.36MIN: 2.8 / MAX: 28.34MIN: 2.82 / MAX: 21.641. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl

oneDNN

Harness: Recurrent Neural Network Training - Engine: CPU

OpenBenchmarking.orgms, Fewer Is BetteroneDNN 3.4Harness: Recurrent Neural Network Training - Engine: CPUdabc30060090012001500SE +/- 0.77, N = 31254.431254.681255.671256.43MIN: 1249.27MIN: 1250.31MIN: 1250.99MIN: 1249.441. (CXX) g++ options: -O3 -march=native -fopenmp -msse4.1 -fPIC -pie -ldl

Neural Magic DeepSparse

Model: Llama2 Chat 7b Quantized - Scenario: Synchronous Single-Stream

OpenBenchmarking.orgms/batch, Fewer Is BetterNeural Magic DeepSparse 1.7Model: Llama2 Chat 7b Quantized - Scenario: Synchronous Single-Streamacbd4080120160200SE +/- 0.11, N = 3165.40165.47165.55165.61

Neural Magic DeepSparse

Model: Llama2 Chat 7b Quantized - Scenario: Synchronous Single-Stream

OpenBenchmarking.orgitems/sec, More Is BetterNeural Magic DeepSparse 1.7Model: Llama2 Chat 7b Quantized - Scenario: Synchronous Single-Streamacbd246810SE +/- 0.0038, N = 36.04486.04256.03936.0373

Neural Magic DeepSparse

Model: CV Classification, ResNet-50 ImageNet - Scenario: Asynchronous Multi-Stream

OpenBenchmarking.orgms/batch, Fewer Is BetterNeural Magic DeepSparse 1.7Model: CV Classification, ResNet-50 ImageNet - Scenario: Asynchronous Multi-Streambacd918273645SE +/- 0.00, N = 339.0339.0539.0639.07

Neural Magic DeepSparse

Model: CV Classification, ResNet-50 ImageNet - Scenario: Asynchronous Multi-Stream

OpenBenchmarking.orgitems/sec, More Is BetterNeural Magic DeepSparse 1.7Model: CV Classification, ResNet-50 ImageNet - Scenario: Asynchronous Multi-Streambacd70140210280350SE +/- 0.03, N = 3307.09306.97306.95306.90

Neural Magic DeepSparse

Model: ResNet-50, Baseline - Scenario: Asynchronous Multi-Stream

OpenBenchmarking.orgms/batch, Fewer Is BetterNeural Magic DeepSparse 1.7Model: ResNet-50, Baseline - Scenario: Asynchronous Multi-Streambadc918273645SE +/- 0.00, N = 339.0539.0539.0639.07

Neural Magic DeepSparse

Model: ResNet-50, Baseline - Scenario: Asynchronous Multi-Stream

OpenBenchmarking.orgitems/sec, More Is BetterNeural Magic DeepSparse 1.7Model: ResNet-50, Baseline - Scenario: Asynchronous Multi-Streambadc70140210280350SE +/- 0.00, N = 3307.02307.00306.94306.92

OpenVINO

Model: Age Gender Recognition Retail 0013 FP16-INT8 - Device: CPU

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2024.0Model: Age Gender Recognition Retail 0013 FP16-INT8 - Device: CPUabcd0.11930.23860.35790.47720.5965SE +/- 0.00, N = 30.530.530.530.53MIN: 0.3 / MAX: 12.95MIN: 0.3 / MAX: 12.53MIN: 0.3 / MAX: 13.83MIN: 0.31 / MAX: 12.661. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl

Stockfish

Chess Benchmark

OpenBenchmarking.orgNodes Per Second, More Is BetterStockfish 16.1Chess Benchmarkbcda13M26M39M52M65MSE +/- 1129127.95, N = 15610082705523757353082452526075281. (CXX) g++ options: -lgcov -m64 -lpthread -fno-exceptions -std=c++17 -fno-peel-loops -fno-tracer -pedantic -O3 -funroll-loops -msse -msse3 -mpopcnt -mavx2 -mbmi -msse4.1 -mssse3 -msse2 -mbmi2 -flto -flto-partition=one -flto=jobserver


Phoronix Test Suite v10.8.5