df2888

Intel Xeon E E-2488 testing with a Supermicro Super Server X13SCL-F v0123456789 (1.1 BIOS) and llvmpipe on Ubuntu 22.04 via the Phoronix Test Suite.

HTML result view exported from: https://openbenchmarking.org/result/2403242-NE-DF288805368&sro&grs.

df2888ProcessorMotherboardChipsetMemoryDiskGraphicsMonitorNetworkOSKernelDesktopDisplay ServerVulkanCompilerFile-SystemScreen ResolutionabcIntel Xeon E E-2488 @ 3.20GHz (8 Cores / 16 Threads)Supermicro Super Server X13SCL-F v0123456789 (1.1 BIOS)Intel Device 7a272 x 32GB DRAM-4400MT/s Micron MTC20C2085S1EC48BA1960GB Micron_7450_MTFDKBA960TFR + 3201GB Micron_7450_MTFDKCC3T2TFSllvmpipeVE2282 x Intel I210Ubuntu 22.046.2.0-26-generic (x86_64)GNOME Shell 42.9X Server 1.21.1.41.3.238GCC 11.4.0ext41920x1080OpenBenchmarking.orgKernel Details- Transparent Huge Pages: madviseCompiler Details- --build=x86_64-linux-gnu --disable-vtable-verify --disable-werror --enable-bootstrap --enable-cet --enable-checking=release --enable-clocale=gnu --enable-default-pie --enable-gnu-unique-object --enable-languages=c,ada,c++,go,brig,d,fortran,objc,obj-c++,m2 --enable-libphobos-checking=release --enable-libstdcxx-debug --enable-libstdcxx-time=yes --enable-link-serialization=2 --enable-multiarch --enable-multilib --enable-nls --enable-objc-gc=auto --enable-offload-targets=nvptx-none=/build/gcc-11-XeT9lY/gcc-11-11.4.0/debian/tmp-nvptx/usr,amdgcn-amdhsa=/build/gcc-11-XeT9lY/gcc-11-11.4.0/debian/tmp-gcn/usr --enable-plugin --enable-shared --enable-threads=posix --host=x86_64-linux-gnu --program-prefix=x86_64-linux-gnu- --target=x86_64-linux-gnu --with-abi=m64 --with-arch-32=i686 --with-build-config=bootstrap-lto-lean --with-default-libstdcxx-abi=new --with-gcc-major-version-only --with-multilib-list=m32,m64,mx32 --with-target-system-zlib=auto --with-tune=generic --without-cuda-driver -v Processor Details- Scaling Governor: acpi-cpufreq performance (Boost: Enabled) - CPU Microcode: 0x11f - Thermald 2.4.9 Python Details- Python 3.10.12Security Details- itlb_multihit: Not affected + l1tf: Not affected + mds: Not affected + meltdown: Not affected + mmio_stale_data: Not affected + retbleed: Not affected + spec_store_bypass: Mitigation of SSB disabled via prctl + spectre_v1: Mitigation of usercopy/swapgs barriers and __user pointer sanitization + spectre_v2: Mitigation of Enhanced IBRS IBPB: conditional RSB filling PBRSB-eIBRS: SW sequence + srbds: Not affected + tsx_async_abort: Not affected

df2888openvino: Road Segmentation ADAS FP16 - CPUopenvino: Road Segmentation ADAS FP16 - CPUpgbench: 1 - 800 - Read Onlypgbench: 1 - 800 - Read Only - Average Latencysvt-vp9: Visual Quality Optimized - Bosphorus 4Ksvt-av1: Preset 12 - Bosphorus 4Kpgbench: 1000 - 800 - Read Writepgbench: 1000 - 800 - Read Write - Average Latencyuvg266: Bosphorus 1080p - Super Fastsvt-av1: Preset 8 - Bosphorus 1080ppgbench: 1000 - 1000 - Read Writepgbench: 1000 - 1000 - Read Write - Average Latencypgbench: 1 - 800 - Read Writepgbench: 1 - 800 - Read Write - Average Latencyuvg266: Bosphorus 1080p - Very Fastx265: Bosphorus 1080psvt-vp9: PSNR/SSIM Optimized - Bosphorus 4Kpgbench: 100 - 800 - Read Only - Average Latencydeepsparse: NLP Document Classification, oBERT base uncased on IMDB - Asynchronous Multi-Streamdeepsparse: NLP Document Classification, oBERT base uncased on IMDB - Asynchronous Multi-Streampgbench: 100 - 800 - Read Onlysvt-av1: Preset 13 - Bosphorus 4Kpgbench: 1 - 1000 - Read Writepgbench: 1 - 1000 - Read Only - Average Latencypgbench: 1 - 1000 - Read Onlysvt-hevc: 7 - Bosphorus 4Kpgbench: 1000 - 800 - Read Onlypgbench: 1000 - 800 - Read Only - Average Latencyspecfem3d: Layered Halfspaceuvg266: Bosphorus 1080p - Ultra Fastbuild-ffmpeg: Time To Compileopenvino: Person Detection FP32 - CPUopenvino: Person Detection FP32 - CPUdeepsparse: NLP Token Classification, BERT base uncased conll2003 - Synchronous Single-Streamdeepsparse: NLP Token Classification, BERT base uncased conll2003 - Synchronous Single-Streamdeepsparse: NLP Document Classification, oBERT base uncased on IMDB - Synchronous Single-Streamdeepsparse: NLP Document Classification, oBERT base uncased on IMDB - Synchronous Single-Streamcompress-7zip: Decompression Ratingpgbench: 1000 - 1000 - Read Onlypgbench: 1000 - 1000 - Read Only - Average Latencydeepsparse: ResNet-50, Sparse INT8 - Asynchronous Multi-Streamdeepsparse: ResNet-50, Sparse INT8 - Asynchronous Multi-Streamdeepsparse: CV Detection, YOLOv5s COCO, Sparse INT8 - Synchronous Single-Streamdeepsparse: CV Detection, YOLOv5s COCO, Sparse INT8 - Synchronous Single-Streamdeepsparse: CV Classification, ResNet-50 ImageNet - Asynchronous Multi-Streamdeepsparse: CV Classification, ResNet-50 ImageNet - Asynchronous Multi-Streamdeepsparse: BERT-Large, NLP Question Answering, Sparse INT8 - Asynchronous Multi-Streamdeepsparse: BERT-Large, NLP Question Answering, Sparse INT8 - Asynchronous Multi-Streamsvt-hevc: 10 - Bosphorus 4Kclickhouse: 100M Rows Hits Dataset, First Run / Cold Cachex264: Bosphorus 4Kpgbench: 100 - 1000 - Read Only - Average Latencypgbench: 100 - 1000 - Read Onlysvt-av1: Preset 4 - Bosphorus 1080puvg266: Bosphorus 4K - Super Fastpgbench: 100 - 800 - Read Writepgbench: 100 - 800 - Read Write - Average Latencyopenvino: Face Detection Retail FP16 - CPUopenvino: Face Detection Retail FP16 - CPUdeepsparse: ResNet-50, Baseline - Asynchronous Multi-Streamdeepsparse: ResNet-50, Baseline - Asynchronous Multi-Streamsvt-vp9: VMAF Optimized - Bosphorus 1080pdeepsparse: ResNet-50, Baseline - Synchronous Single-Streamdeepsparse: ResNet-50, Baseline - Synchronous Single-Streamclickhouse: 100M Rows Hits Dataset, Second Runvvenc: Bosphorus 1080p - Fasteropenvino: Person Detection FP16 - CPUnamd: ATPase with 327,506 Atomsopenvino: Person Detection FP16 - CPUdeepsparse: CV Classification, ResNet-50 ImageNet - Synchronous Single-Streamdeepsparse: CV Classification, ResNet-50 ImageNet - Synchronous Single-Streamdeepsparse: NLP Text Classification, DistilBERT mnli - Asynchronous Multi-Streamdeepsparse: NLP Text Classification, DistilBERT mnli - Asynchronous Multi-Streamclickhouse: 100M Rows Hits Dataset, Third Rundeepsparse: NLP Text Classification, BERT base uncased SST2, Sparse INT8 - Asynchronous Multi-Streamdeepsparse: NLP Text Classification, BERT base uncased SST2, Sparse INT8 - Asynchronous Multi-Streamspecfem3d: Tomographic Modelsvt-av1: Preset 8 - Bosphorus 4Kdeepsparse: CV Segmentation, 90% Pruned YOLACT Pruned - Synchronous Single-Streamdeepsparse: NLP Text Classification, DistilBERT mnli - Synchronous Single-Streamdeepsparse: CV Segmentation, 90% Pruned YOLACT Pruned - Synchronous Single-Streamdeepsparse: CV Segmentation, 90% Pruned YOLACT Pruned - Asynchronous Multi-Streamdeepsparse: CV Segmentation, 90% Pruned YOLACT Pruned - Asynchronous Multi-Streamdeepsparse: NLP Text Classification, DistilBERT mnli - Synchronous Single-Streamdeepsparse: ResNet-50, Sparse INT8 - Synchronous Single-Streamdeepsparse: BERT-Large, NLP Question Answering, Sparse INT8 - Synchronous Single-Streamdeepsparse: BERT-Large, NLP Question Answering, Sparse INT8 - Synchronous Single-Streamdeepsparse: ResNet-50, Sparse INT8 - Synchronous Single-Streamspecfem3d: Mount St. Helenssvt-vp9: VMAF Optimized - Bosphorus 4Knamd: STMV with 1,066,628 Atomsdeepsparse: NLP Text Classification, BERT base uncased SST2, Sparse INT8 - Synchronous Single-Streamdeepsparse: NLP Text Classification, BERT base uncased SST2, Sparse INT8 - Synchronous Single-Streambuild-linux-kernel: defconfigopenvino: Weld Porosity Detection FP16-INT8 - CPUopenvino: Road Segmentation ADAS FP16-INT8 - CPUopenvino: Road Segmentation ADAS FP16-INT8 - CPUdeepsparse: CV Detection, YOLOv5s COCO, Sparse INT8 - Asynchronous Multi-Streamdeepsparse: CV Detection, YOLOv5s COCO, Sparse INT8 - Asynchronous Multi-Streamopenvino: Weld Porosity Detection FP16-INT8 - CPUuvg266: Bosphorus 1080p - Slowgromacs: MPI CPU - water_GMX50_barecompress-7zip: Compression Ratingdeepsparse: NLP Token Classification, BERT base uncased conll2003 - Asynchronous Multi-Streamdeepsparse: NLP Token Classification, BERT base uncased conll2003 - Asynchronous Multi-Streambrl-cad: VGR Performance Metricopenvino: Face Detection FP16-INT8 - CPUuvg266: Bosphorus 4K - Ultra Fastsvt-hevc: 1 - Bosphorus 1080popenvino: Face Detection FP16-INT8 - CPUuvg266: Bosphorus 1080p - Mediumopenvino: Age Gender Recognition Retail 0013 FP16 - CPUopenvino: Age Gender Recognition Retail 0013 FP16-INT8 - CPUpgbench: 100 - 1000 - Read Write - Average Latencypgbench: 100 - 1000 - Read Writebuild-llvm: Unix Makefilesbuild-llvm: Ninjaopenfoam: drivaerFastback, Small Mesh Size - Mesh Timebuild-godot: Time To Compileopenvino: Age Gender Recognition Retail 0013 FP16 - CPUvvenc: Bosphorus 4K - Fasterspecfem3d: Water-layered Halfspacex265: Bosphorus 4Kvvenc: Bosphorus 1080p - Fastopenvino: Face Detection FP16 - CPUopenvino: Vehicle Detection FP16 - CPUsvt-av1: Preset 4 - Bosphorus 4Kopenvino: Vehicle Detection FP16 - CPUdeepsparse: Llama2 Chat 7b Quantized - Asynchronous Multi-Streamdeepsparse: Llama2 Chat 7b Quantized - Asynchronous Multi-Streamopenvino: Noise Suppression Poconet-Like FP16 - CPUopenvino: Machine Translation EN To DE FP16 - CPUblender: BMW27 - CPU-Onlyopenvino: Machine Translation EN To DE FP16 - CPUopenvino: Noise Suppression Poconet-Like FP16 - CPUsvt-vp9: Visual Quality Optimized - Bosphorus 1080pblender: Fishy Cat - CPU-Onlyopenvino: Vehicle Detection FP16-INT8 - CPUopenvino: Weld Porosity Detection FP16 - CPUopenvino: Weld Porosity Detection FP16 - CPUopenvino: Vehicle Detection FP16-INT8 - CPUopenvino: Face Detection FP16 - CPUopenvino: Person Vehicle Bike Detection FP16 - CPUopenvino: Handwritten English Recognition FP16-INT8 - CPUopenvino: Handwritten English Recognition FP16-INT8 - CPUuvg266: Bosphorus 4K - Mediumopenvino: Person Vehicle Bike Detection FP16 - CPUsvt-hevc: 1 - Bosphorus 4Kuvg266: Bosphorus 4K - Very Fastopenvino: Handwritten English Recognition FP16 - CPUopenvino: Handwritten English Recognition FP16 - CPUopenvino: Person Re-Identification Retail FP16 - CPUopenvino: Person Re-Identification Retail FP16 - CPUblender: Classroom - CPU-Onlybuild-nodejs: Time To Compileopenfoam: drivaerFastback, Medium Mesh Size - Mesh Timesvt-vp9: PSNR/SSIM Optimized - Bosphorus 1080popenvino: Face Detection Retail FP16-INT8 - CPUblender: Barbershop - CPU-Onlybuild-linux-kernel: allmodconfigsvt-hevc: 10 - Bosphorus 1080pbuild-gcc: Time To Compileopenvino: Face Detection Retail FP16-INT8 - CPUuvg266: Bosphorus 4K - Slowopenfoam: drivaerFastback, Small Mesh Size - Execution Timevvenc: Bosphorus 4K - Fastdeepsparse: Llama2 Chat 7b Quantized - Synchronous Single-Streamdeepsparse: Llama2 Chat 7b Quantized - Synchronous Single-Streamsvt-av1: Preset 12 - Bosphorus 1080pblender: Pabellon Barcelona - CPU-Onlysvt-av1: Preset 13 - Bosphorus 1080popenfoam: drivaerFastback, Medium Mesh Size - Execution Timeopenvino: Age Gender Recognition Retail 0013 FP16-INT8 - CPUpgbench: 1 - 1000 - Read Write - Average Latencyx264: Bosphorus 1080psvt-hevc: 7 - Bosphorus 1080pspecfem3d: Homogeneous Halfspaceabc44.8089.409658130.82859.67105.7832754129.10598.26136.0882518939.7252347341.05291.5884.8583.760.952446.58878.9566840809106.95317431.16485951448.386197381.292150.981664264120.8842.18628.53140.16109.83189.1063111.06309.0036651805995141.668802.07404.972719.234251.9778109.270036.595926.4927150.8967103.05226.1031.001.41170888716.22620.313497422.8759.88808.93109.338336.5726281.0810.269597.3359234.7140.203137.850.6720829.0110.271397.317851.320477.9417243.23334.711911.938356.66165837135.47195.444714.828310.4758362.274311.041167.43131.49269.8776101.1551667.993056.75468608781.760.209373.8825257.331791.8507.68210.1938.0472.630955.06411040.6832.901.38795609445.64028.9756189384371.8224.899.5410.7437.150.9321990.1933.39629945646.617636.00030.329564307.1158563.7611.604139.21004261819.4815.4482.6816.604.354240.785.0711781.119515.1933.68123.52118.74524.22224.63165.95497.2729.22273.6216.081495.1010.3843.92182.106.68384.902.3819.3152.04153.6618.18439.76336.83523.986221.90624291.271634.311297.101311.874381.27791.2664.895.97226.483294.954124.61958.0234532.480414.01644.2812182.83870.36575.609155.70171.8972.92466691439.65100.8310783770.74265.11117.0322597830.795108.39135.7052473240.4342236357.81198.6491.0389.470.948419.60259.5325844015109.00818461.10890221050.985881991.36151.953370514126.9140.22229.06137.62106.1979.416106.58219.3819679345810011.721813.73364.901218.553853.8839112.977735.394825.5887156.2254101.69226.1930.621.38672151715.72120.93438323.26810.06794.56112.375835.5844274.2710.02199.7371240.3840.747135.480.6673329.510.060199.327250.255479.5793239.55340.821411.724356.61069074436.02493.845414.550710.6542355.52711.250268.70721.48029.7105102.886673.15357.0672556280.440.209273.8228261.269290.4937.71213.2237.571.69955.7791036.6932.491.40696616442.90289.0311187148374.6424.729.5910.6736.750.9321792.1333.50929843639.853629.4730.018778306.6568538.5911.696139.33030273719.515.5662.6616.694.336239.555.1063775.759615.0933.7122.72118.68527.54225.96165.79499.4629.35272.4616.011502.5510.4243.88182.216.68383.452.3719.2351.83154.2818.17439.93335.62522.107222.69332291.651639.461298.171308.543381.92790.8684.885.98226.111034.946124.47198.0329531.906413.66644.7562184.34250.36541.625157.78166.272.25806154845.8787.1610578780.75666.15110.5692868527.889104.61124.1882305843.372413331.59296.7289.7288.881.009441.68929.0559793236113.6318151.17185391749.075947781.345144.466011134122.2840.38329.87133.88111.05899.0038111.31078.9835673345765861.734831.54434.796518.628353.6674109.109736.649426.3635151.6354105.17218.8031.631.43169905015.81520.263397623.5469.78817.16110.662836.1354274.4810.045599.4907237.9839.798138.640.6568328.8510.055399.393950.382979.3779238.31334.201711.955955.57467586336.15693.656814.557710.6756356.213411.228568.6721.46659.7892102.0626679.280956.14319272880.610.206013.8275260.955590.5057.6211.4337.8272.731354.98751051.2232.941.40396892439.84179.0939188833370.2724.69.6510.7936.90.9421757.7733.15630160643.708630.04130.039036304.5828495.0411.65138.25437520219.6315.4942.6616.574.367241.225.0995776.831315.1133.48122.81119.45527.2224.72165.05499.9229.37272.25161499.5710.4344.09181.396.71383.232.3719.2751.99153.8118.24438.29336.08523.423222.19301292.251636.641294.721308.925382.17789.5944.885.98226.417394.952124.55378.0276532.511413.87644.6952183.09580.36551.009152.15185.2471.570884588OpenBenchmarking.org

OpenVINO

Model: Road Segmentation ADAS FP16 - Device: CPU

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2024.0Model: Road Segmentation ADAS FP16 - Device: CPUabc1020304050SE +/- 0.49, N = 1544.8039.6545.87MIN: 17.6 / MAX: 66.04MIN: 31.4 / MAX: 49.78MIN: 28.97 / MAX: 51.941. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl

OpenVINO

Model: Road Segmentation ADAS FP16 - Device: CPU

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2024.0Model: Road Segmentation ADAS FP16 - Device: CPUabc20406080100SE +/- 1.01, N = 1589.40100.8387.161. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl

PostgreSQL

Scaling Factor: 1 - Clients: 800 - Mode: Read Only

OpenBenchmarking.orgTPS, More Is BetterPostgreSQL 16Scaling Factor: 1 - Clients: 800 - Mode: Read Onlyabc200K400K600K800K1000KSE +/- 2188.42, N = 3965813107837710578781. (CC) gcc options: -fno-strict-aliasing -fwrapv -O2 -lm -licui18n -licuuc -licudata

PostgreSQL

Scaling Factor: 1 - Clients: 800 - Mode: Read Only - Average Latency

OpenBenchmarking.orgms, Fewer Is BetterPostgreSQL 16Scaling Factor: 1 - Clients: 800 - Mode: Read Only - Average Latencyabc0.18630.37260.55890.74520.9315SE +/- 0.002, N = 30.8280.7420.7561. (CC) gcc options: -fno-strict-aliasing -fwrapv -O2 -lm -licui18n -licuuc -licudata

SVT-VP9

Tuning: Visual Quality Optimized - Input: Bosphorus 4K

OpenBenchmarking.orgFrames Per Second, More Is BetterSVT-VP9 0.3Tuning: Visual Quality Optimized - Input: Bosphorus 4Kabc1530456075SE +/- 0.69, N = 359.6765.1166.151. (CC) gcc options: -O3 -fcommon -fPIE -fPIC -fvisibility=hidden -pie -rdynamic -lpthread -lrt -lm

SVT-AV1

Encoder Mode: Preset 12 - Input: Bosphorus 4K

OpenBenchmarking.orgFrames Per Second, More Is BetterSVT-AV1 2.0Encoder Mode: Preset 12 - Input: Bosphorus 4Kabc306090120150SE +/- 0.98, N = 15105.78117.03110.571. (CXX) g++ options: -march=native -mno-avx -mavx2 -mavx512f -mavx512bw -mavx512dq

PostgreSQL

Scaling Factor: 1000 - Clients: 800 - Mode: Read Write

OpenBenchmarking.orgTPS, More Is BetterPostgreSQL 16Scaling Factor: 1000 - Clients: 800 - Mode: Read Writeabc6K12K18K24K30KSE +/- 372.60, N = 122754125978286851. (CC) gcc options: -fno-strict-aliasing -fwrapv -O2 -lm -licui18n -licuuc -licudata

PostgreSQL

Scaling Factor: 1000 - Clients: 800 - Mode: Read Write - Average Latency

OpenBenchmarking.orgms, Fewer Is BetterPostgreSQL 16Scaling Factor: 1000 - Clients: 800 - Mode: Read Write - Average Latencyabc714212835SE +/- 0.39, N = 1229.1130.8027.891. (CC) gcc options: -fno-strict-aliasing -fwrapv -O2 -lm -licui18n -licuuc -licudata

uvg266

Video Input: Bosphorus 1080p - Video Preset: Super Fast

OpenBenchmarking.orgFrames Per Second, More Is Betteruvg266 0.4.1Video Input: Bosphorus 1080p - Video Preset: Super Fastabc20406080100SE +/- 1.28, N = 1298.26108.39104.61

SVT-AV1

Encoder Mode: Preset 8 - Input: Bosphorus 1080p

OpenBenchmarking.orgFrames Per Second, More Is BetterSVT-AV1 2.0Encoder Mode: Preset 8 - Input: Bosphorus 1080pabc306090120150SE +/- 0.21, N = 3136.09135.71124.191. (CXX) g++ options: -march=native -mno-avx -mavx2 -mavx512f -mavx512bw -mavx512dq

PostgreSQL

Scaling Factor: 1000 - Clients: 1000 - Mode: Read Write

OpenBenchmarking.orgTPS, More Is BetterPostgreSQL 16Scaling Factor: 1000 - Clients: 1000 - Mode: Read Writeabc5K10K15K20K25KSE +/- 190.45, N = 122518924732230581. (CC) gcc options: -fno-strict-aliasing -fwrapv -O2 -lm -licui18n -licuuc -licudata

PostgreSQL

Scaling Factor: 1000 - Clients: 1000 - Mode: Read Write - Average Latency

OpenBenchmarking.orgms, Fewer Is BetterPostgreSQL 16Scaling Factor: 1000 - Clients: 1000 - Mode: Read Write - Average Latencyabc1020304050SE +/- 0.30, N = 1239.7340.4343.371. (CC) gcc options: -fno-strict-aliasing -fwrapv -O2 -lm -licui18n -licuuc -licudata

PostgreSQL

Scaling Factor: 1 - Clients: 800 - Mode: Read Write

OpenBenchmarking.orgTPS, More Is BetterPostgreSQL 16Scaling Factor: 1 - Clients: 800 - Mode: Read Writeabc5001000150020002500SE +/- 33.78, N = 32347223624131. (CC) gcc options: -fno-strict-aliasing -fwrapv -O2 -lm -licui18n -licuuc -licudata

PostgreSQL

Scaling Factor: 1 - Clients: 800 - Mode: Read Write - Average Latency

OpenBenchmarking.orgms, Fewer Is BetterPostgreSQL 16Scaling Factor: 1 - Clients: 800 - Mode: Read Write - Average Latencyabc80160240320400SE +/- 4.95, N = 3341.05357.81331.591. (CC) gcc options: -fno-strict-aliasing -fwrapv -O2 -lm -licui18n -licuuc -licudata

uvg266

Video Input: Bosphorus 1080p - Video Preset: Very Fast

OpenBenchmarking.orgFrames Per Second, More Is Betteruvg266 0.4.1Video Input: Bosphorus 1080p - Video Preset: Very Fastabc20406080100SE +/- 1.03, N = 1591.5898.6496.72

x265

Video Input: Bosphorus 1080p

OpenBenchmarking.orgFrames Per Second, More Is Betterx265 3.4Video Input: Bosphorus 1080pabc20406080100SE +/- 0.71, N = 1584.8591.0389.721. (CXX) g++ options: -O3 -rdynamic -lpthread -lrt -ldl -lnuma

SVT-VP9

Tuning: PSNR/SSIM Optimized - Input: Bosphorus 4K

OpenBenchmarking.orgFrames Per Second, More Is BetterSVT-VP9 0.3Tuning: PSNR/SSIM Optimized - Input: Bosphorus 4Kabc20406080100SE +/- 0.78, N = 1583.7689.4788.881. (CC) gcc options: -O3 -fcommon -fPIE -fPIC -fvisibility=hidden -pie -rdynamic -lpthread -lrt -lm

PostgreSQL

Scaling Factor: 100 - Clients: 800 - Mode: Read Only - Average Latency

OpenBenchmarking.orgms, Fewer Is BetterPostgreSQL 16Scaling Factor: 100 - Clients: 800 - Mode: Read Only - Average Latencyabc0.2270.4540.6810.9081.135SE +/- 0.010, N = 30.9520.9481.0091. (CC) gcc options: -fno-strict-aliasing -fwrapv -O2 -lm -licui18n -licuuc -licudata

Neural Magic DeepSparse

Model: NLP Document Classification, oBERT base uncased on IMDB - Scenario: Asynchronous Multi-Stream

OpenBenchmarking.orgms/batch, Fewer Is BetterNeural Magic DeepSparse 1.7Model: NLP Document Classification, oBERT base uncased on IMDB - Scenario: Asynchronous Multi-Streamabc100200300400500SE +/- 0.58, N = 3446.59419.60441.69

Neural Magic DeepSparse

Model: NLP Document Classification, oBERT base uncased on IMDB - Scenario: Asynchronous Multi-Stream

OpenBenchmarking.orgitems/sec, More Is BetterNeural Magic DeepSparse 1.7Model: NLP Document Classification, oBERT base uncased on IMDB - Scenario: Asynchronous Multi-Streamabc3691215SE +/- 0.0116, N = 38.95669.53259.0559

PostgreSQL

Scaling Factor: 100 - Clients: 800 - Mode: Read Only

OpenBenchmarking.orgTPS, More Is BetterPostgreSQL 16Scaling Factor: 100 - Clients: 800 - Mode: Read Onlyabc200K400K600K800K1000KSE +/- 8610.85, N = 38408098440157932361. (CC) gcc options: -fno-strict-aliasing -fwrapv -O2 -lm -licui18n -licuuc -licudata

SVT-AV1

Encoder Mode: Preset 13 - Input: Bosphorus 4K

OpenBenchmarking.orgFrames Per Second, More Is BetterSVT-AV1 2.0Encoder Mode: Preset 13 - Input: Bosphorus 4Kabc306090120150SE +/- 1.39, N = 15106.95109.01113.631. (CXX) g++ options: -march=native -mno-avx -mavx2 -mavx512f -mavx512bw -mavx512dq

PostgreSQL

Scaling Factor: 1 - Clients: 1000 - Mode: Read Write

OpenBenchmarking.orgTPS, More Is BetterPostgreSQL 16Scaling Factor: 1 - Clients: 1000 - Mode: Read Writeabc400800120016002000SE +/- 28.21, N = 121743184618151. (CC) gcc options: -fno-strict-aliasing -fwrapv -O2 -lm -licui18n -licuuc -licudata

PostgreSQL

Scaling Factor: 1 - Clients: 1000 - Mode: Read Only - Average Latency

OpenBenchmarking.orgms, Fewer Is BetterPostgreSQL 16Scaling Factor: 1 - Clients: 1000 - Mode: Read Only - Average Latencyabc0.26350.5270.79051.0541.3175SE +/- 0.010, N = 31.1641.1081.1711. (CC) gcc options: -fno-strict-aliasing -fwrapv -O2 -lm -licui18n -licuuc -licudata

PostgreSQL

Scaling Factor: 1 - Clients: 1000 - Mode: Read Only

OpenBenchmarking.orgTPS, More Is BetterPostgreSQL 16Scaling Factor: 1 - Clients: 1000 - Mode: Read Onlyabc200K400K600K800K1000KSE +/- 6896.35, N = 38595149022108539171. (CC) gcc options: -fno-strict-aliasing -fwrapv -O2 -lm -licui18n -licuuc -licudata

SVT-HEVC

Tuning: 7 - Input: Bosphorus 4K

OpenBenchmarking.orgFrames Per Second, More Is BetterSVT-HEVC 1.5.0Tuning: 7 - Input: Bosphorus 4Kabc1224364860SE +/- 0.07, N = 348.3850.9849.071. (CC) gcc options: -fPIE -fPIC -O3 -O2 -pie -rdynamic -lpthread -lrt

PostgreSQL

Scaling Factor: 1000 - Clients: 800 - Mode: Read Only

OpenBenchmarking.orgTPS, More Is BetterPostgreSQL 16Scaling Factor: 1000 - Clients: 800 - Mode: Read Onlyabc130K260K390K520K650KSE +/- 5061.88, N = 96197385881995947781. (CC) gcc options: -fno-strict-aliasing -fwrapv -O2 -lm -licui18n -licuuc -licudata

PostgreSQL

Scaling Factor: 1000 - Clients: 800 - Mode: Read Only - Average Latency

OpenBenchmarking.orgms, Fewer Is BetterPostgreSQL 16Scaling Factor: 1000 - Clients: 800 - Mode: Read Only - Average Latencyabc0.3060.6120.9181.2241.53SE +/- 0.011, N = 91.2921.3601.3451. (CC) gcc options: -fno-strict-aliasing -fwrapv -O2 -lm -licui18n -licuuc -licudata

SPECFEM3D

Model: Layered Halfspace

OpenBenchmarking.orgSeconds, Fewer Is BetterSPECFEM3D 4.0Model: Layered Halfspaceabc306090120150SE +/- 1.25, N = 3150.98151.95144.471. (F9X) gfortran options: -O2 -fopenmp -std=f2003 -fimplicit-none -fmax-errors=10 -pedantic -pedantic-errors -O3 -finline-functions -lmpi_usempif08 -lmpi_mpifh -lmpi -lopen-rte -lopen-pal -lhwloc -levent_core -levent_pthreads -lm -lz

uvg266

Video Input: Bosphorus 1080p - Video Preset: Ultra Fast

OpenBenchmarking.orgFrames Per Second, More Is Betteruvg266 0.4.1Video Input: Bosphorus 1080p - Video Preset: Ultra Fastabc306090120150SE +/- 0.11, N = 3120.88126.91122.28

Timed FFmpeg Compilation

Time To Compile

OpenBenchmarking.orgSeconds, Fewer Is BetterTimed FFmpeg Compilation 6.1Time To Compileabc1020304050SE +/- 0.05, N = 342.1940.2240.38

OpenVINO

Model: Person Detection FP32 - Device: CPU

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2024.0Model: Person Detection FP32 - Device: CPUabc714212835SE +/- 0.07, N = 328.5329.0629.871. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl

OpenVINO

Model: Person Detection FP32 - Device: CPU

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2024.0Model: Person Detection FP32 - Device: CPUabc306090120150SE +/- 0.35, N = 3140.16137.62133.88MIN: 114.55 / MAX: 222.64MIN: 115.29 / MAX: 146.55MIN: 115.87 / MAX: 217.541. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl

Neural Magic DeepSparse

Model: NLP Token Classification, BERT base uncased conll2003 - Scenario: Synchronous Single-Stream

OpenBenchmarking.orgms/batch, Fewer Is BetterNeural Magic DeepSparse 1.7Model: NLP Token Classification, BERT base uncased conll2003 - Scenario: Synchronous Single-Streamabc20406080100SE +/- 1.11, N = 3109.83106.20111.06

Neural Magic DeepSparse

Model: NLP Token Classification, BERT base uncased conll2003 - Scenario: Synchronous Single-Stream

OpenBenchmarking.orgitems/sec, More Is BetterNeural Magic DeepSparse 1.7Model: NLP Token Classification, BERT base uncased conll2003 - Scenario: Synchronous Single-Streamabc3691215SE +/- 0.0932, N = 39.10639.41609.0038

Neural Magic DeepSparse

Model: NLP Document Classification, oBERT base uncased on IMDB - Scenario: Synchronous Single-Stream

OpenBenchmarking.orgms/batch, Fewer Is BetterNeural Magic DeepSparse 1.7Model: NLP Document Classification, oBERT base uncased on IMDB - Scenario: Synchronous Single-Streamabc20406080100SE +/- 0.25, N = 3111.06106.58111.31

Neural Magic DeepSparse

Model: NLP Document Classification, oBERT base uncased on IMDB - Scenario: Synchronous Single-Stream

OpenBenchmarking.orgitems/sec, More Is BetterNeural Magic DeepSparse 1.7Model: NLP Document Classification, oBERT base uncased on IMDB - Scenario: Synchronous Single-Streamabc3691215SE +/- 0.0199, N = 39.00369.38198.9835

7-Zip Compression

Test: Decompression Rating

OpenBenchmarking.orgMIPS, More Is Better7-Zip Compression 22.01Test: Decompression Ratingabc15K30K45K60K75KSE +/- 855.85, N = 36518067934673341. (CXX) g++ options: -lpthread -ldl -O2 -fPIC

PostgreSQL

Scaling Factor: 1000 - Clients: 1000 - Mode: Read Only

OpenBenchmarking.orgTPS, More Is BetterPostgreSQL 16Scaling Factor: 1000 - Clients: 1000 - Mode: Read Onlyabc130K260K390K520K650KSE +/- 5543.80, N = 35995145810015765861. (CC) gcc options: -fno-strict-aliasing -fwrapv -O2 -lm -licui18n -licuuc -licudata

PostgreSQL

Scaling Factor: 1000 - Clients: 1000 - Mode: Read Only - Average Latency

OpenBenchmarking.orgms, Fewer Is BetterPostgreSQL 16Scaling Factor: 1000 - Clients: 1000 - Mode: Read Only - Average Latencyabc0.39020.78041.17061.56081.951SE +/- 0.016, N = 31.6681.7211.7341. (CC) gcc options: -fno-strict-aliasing -fwrapv -O2 -lm -licui18n -licuuc -licudata

Neural Magic DeepSparse

Model: ResNet-50, Sparse INT8 - Scenario: Asynchronous Multi-Stream

OpenBenchmarking.orgitems/sec, More Is BetterNeural Magic DeepSparse 1.7Model: ResNet-50, Sparse INT8 - Scenario: Asynchronous Multi-Streamabc2004006008001000SE +/- 1.78, N = 3802.07813.73831.54

Neural Magic DeepSparse

Model: ResNet-50, Sparse INT8 - Scenario: Asynchronous Multi-Stream

OpenBenchmarking.orgms/batch, Fewer Is BetterNeural Magic DeepSparse 1.7Model: ResNet-50, Sparse INT8 - Scenario: Asynchronous Multi-Streamabc1.11892.23783.35674.47565.5945SE +/- 0.0110, N = 34.97274.90124.7965

Neural Magic DeepSparse

Model: CV Detection, YOLOv5s COCO, Sparse INT8 - Scenario: Synchronous Single-Stream

OpenBenchmarking.orgms/batch, Fewer Is BetterNeural Magic DeepSparse 1.7Model: CV Detection, YOLOv5s COCO, Sparse INT8 - Scenario: Synchronous Single-Streamabc510152025SE +/- 0.02, N = 319.2318.5518.63

Neural Magic DeepSparse

Model: CV Detection, YOLOv5s COCO, Sparse INT8 - Scenario: Synchronous Single-Stream

OpenBenchmarking.orgitems/sec, More Is BetterNeural Magic DeepSparse 1.7Model: CV Detection, YOLOv5s COCO, Sparse INT8 - Scenario: Synchronous Single-Streamabc1224364860SE +/- 0.05, N = 351.9853.8853.67

Neural Magic DeepSparse

Model: CV Classification, ResNet-50 ImageNet - Scenario: Asynchronous Multi-Stream

OpenBenchmarking.orgitems/sec, More Is BetterNeural Magic DeepSparse 1.7Model: CV Classification, ResNet-50 ImageNet - Scenario: Asynchronous Multi-Streamabc306090120150SE +/- 0.18, N = 3109.27112.98109.11

Neural Magic DeepSparse

Model: CV Classification, ResNet-50 ImageNet - Scenario: Asynchronous Multi-Stream

OpenBenchmarking.orgms/batch, Fewer Is BetterNeural Magic DeepSparse 1.7Model: CV Classification, ResNet-50 ImageNet - Scenario: Asynchronous Multi-Streamabc816243240SE +/- 0.06, N = 336.6035.3936.65

Neural Magic DeepSparse

Model: BERT-Large, NLP Question Answering, Sparse INT8 - Scenario: Asynchronous Multi-Stream

OpenBenchmarking.orgms/batch, Fewer Is BetterNeural Magic DeepSparse 1.7Model: BERT-Large, NLP Question Answering, Sparse INT8 - Scenario: Asynchronous Multi-Streamabc612182430SE +/- 0.03, N = 326.4925.5926.36

Neural Magic DeepSparse

Model: BERT-Large, NLP Question Answering, Sparse INT8 - Scenario: Asynchronous Multi-Stream

OpenBenchmarking.orgitems/sec, More Is BetterNeural Magic DeepSparse 1.7Model: BERT-Large, NLP Question Answering, Sparse INT8 - Scenario: Asynchronous Multi-Streamabc306090120150SE +/- 0.16, N = 3150.90156.23151.64

SVT-HEVC

Tuning: 10 - Input: Bosphorus 4K

OpenBenchmarking.orgFrames Per Second, More Is BetterSVT-HEVC 1.5.0Tuning: 10 - Input: Bosphorus 4Kabc20406080100SE +/- 1.12, N = 14103.05101.69105.171. (CC) gcc options: -fPIE -fPIC -O3 -O2 -pie -rdynamic -lpthread -lrt

ClickHouse

100M Rows Hits Dataset, First Run / Cold Cache

OpenBenchmarking.orgQueries Per Minute, Geo Mean, More Is BetterClickHouse 22.12.3.5100M Rows Hits Dataset, First Run / Cold Cacheabc50100150200250SE +/- 0.26, N = 3226.10226.19218.80MIN: 9.15 / MAX: 10000MIN: 9.16 / MAX: 8571.43MIN: 9.11 / MAX: 10000

x264

Video Input: Bosphorus 4K

OpenBenchmarking.orgFrames Per Second, More Is Betterx264 2022-02-22Video Input: Bosphorus 4Kabc714212835SE +/- 0.38, N = 331.0030.6231.631. (CC) gcc options: -ldl -lavformat -lavcodec -lavutil -lswscale -m64 -lm -lpthread -O3 -flto

PostgreSQL

Scaling Factor: 100 - Clients: 1000 - Mode: Read Only - Average Latency

OpenBenchmarking.orgms, Fewer Is BetterPostgreSQL 16Scaling Factor: 100 - Clients: 1000 - Mode: Read Only - Average Latencyabc0.3220.6440.9661.2881.61SE +/- 0.016, N = 31.4111.3861.4311. (CC) gcc options: -fno-strict-aliasing -fwrapv -O2 -lm -licui18n -licuuc -licudata

PostgreSQL

Scaling Factor: 100 - Clients: 1000 - Mode: Read Only

OpenBenchmarking.orgTPS, More Is BetterPostgreSQL 16Scaling Factor: 100 - Clients: 1000 - Mode: Read Onlyabc150K300K450K600K750KSE +/- 8095.75, N = 37088877215176990501. (CC) gcc options: -fno-strict-aliasing -fwrapv -O2 -lm -licui18n -licuuc -licudata

SVT-AV1

Encoder Mode: Preset 4 - Input: Bosphorus 1080p

OpenBenchmarking.orgFrames Per Second, More Is BetterSVT-AV1 2.0Encoder Mode: Preset 4 - Input: Bosphorus 1080pabc48121620SE +/- 0.12, N = 1516.2315.7215.821. (CXX) g++ options: -march=native -mno-avx -mavx2 -mavx512f -mavx512bw -mavx512dq

uvg266

Video Input: Bosphorus 4K - Video Preset: Super Fast

OpenBenchmarking.orgFrames Per Second, More Is Betteruvg266 0.4.1Video Input: Bosphorus 4K - Video Preset: Super Fastabc510152025SE +/- 0.02, N = 320.3120.9020.26

PostgreSQL

Scaling Factor: 100 - Clients: 800 - Mode: Read Write

OpenBenchmarking.orgTPS, More Is BetterPostgreSQL 16Scaling Factor: 100 - Clients: 800 - Mode: Read Writeabc7K14K21K28K35KSE +/- 96.04, N = 33497434383339761. (CC) gcc options: -fno-strict-aliasing -fwrapv -O2 -lm -licui18n -licuuc -licudata

PostgreSQL

Scaling Factor: 100 - Clients: 800 - Mode: Read Write - Average Latency

OpenBenchmarking.orgms, Fewer Is BetterPostgreSQL 16Scaling Factor: 100 - Clients: 800 - Mode: Read Write - Average Latencyabc612182430SE +/- 0.06, N = 322.8823.2723.551. (CC) gcc options: -fno-strict-aliasing -fwrapv -O2 -lm -licui18n -licuuc -licudata

OpenVINO

Model: Face Detection Retail FP16 - Device: CPU

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2024.0Model: Face Detection Retail FP16 - Device: CPUabc3691215SE +/- 0.11, N = 39.8810.069.78MIN: 5.09 / MAX: 39.75MIN: 4.92 / MAX: 36.16MIN: 6.25 / MAX: 37.551. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl

OpenVINO

Model: Face Detection Retail FP16 - Device: CPU

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2024.0Model: Face Detection Retail FP16 - Device: CPUabc2004006008001000SE +/- 9.03, N = 3808.93794.56817.161. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl

Neural Magic DeepSparse

Model: ResNet-50, Baseline - Scenario: Asynchronous Multi-Stream

OpenBenchmarking.orgitems/sec, More Is BetterNeural Magic DeepSparse 1.7Model: ResNet-50, Baseline - Scenario: Asynchronous Multi-Streamabc306090120150SE +/- 0.05, N = 3109.34112.38110.66

Neural Magic DeepSparse

Model: ResNet-50, Baseline - Scenario: Asynchronous Multi-Stream

OpenBenchmarking.orgms/batch, Fewer Is BetterNeural Magic DeepSparse 1.7Model: ResNet-50, Baseline - Scenario: Asynchronous Multi-Streamabc816243240SE +/- 0.02, N = 336.5735.5836.14

SVT-VP9

Tuning: VMAF Optimized - Input: Bosphorus 1080p

OpenBenchmarking.orgFrames Per Second, More Is BetterSVT-VP9 0.3Tuning: VMAF Optimized - Input: Bosphorus 1080pabc60120180240300SE +/- 4.04, N = 3281.08274.27274.481. (CC) gcc options: -O3 -fcommon -fPIE -fPIC -fvisibility=hidden -pie -rdynamic -lpthread -lrt -lm

Neural Magic DeepSparse

Model: ResNet-50, Baseline - Scenario: Synchronous Single-Stream

OpenBenchmarking.orgms/batch, Fewer Is BetterNeural Magic DeepSparse 1.7Model: ResNet-50, Baseline - Scenario: Synchronous Single-Streamabc3691215SE +/- 0.10, N = 310.2710.0210.05

Neural Magic DeepSparse

Model: ResNet-50, Baseline - Scenario: Synchronous Single-Stream

OpenBenchmarking.orgitems/sec, More Is BetterNeural Magic DeepSparse 1.7Model: ResNet-50, Baseline - Scenario: Synchronous Single-Streamabc20406080100SE +/- 0.95, N = 397.3499.7499.49

ClickHouse

100M Rows Hits Dataset, Second Run

OpenBenchmarking.orgQueries Per Minute, Geo Mean, More Is BetterClickHouse 22.12.3.5100M Rows Hits Dataset, Second Runabc50100150200250SE +/- 0.63, N = 3234.71240.38237.98MIN: 9.36 / MAX: 10000MIN: 9.44 / MAX: 12000MIN: 9.26 / MAX: 8571.43

VVenC

Video Input: Bosphorus 1080p - Video Preset: Faster

OpenBenchmarking.orgFrames Per Second, More Is BetterVVenC 1.11Video Input: Bosphorus 1080p - Video Preset: Fasterabc918273645SE +/- 0.28, N = 1540.2040.7539.801. (CXX) g++ options: -O3 -flto -fno-fat-lto-objects -flto=auto

OpenVINO

Model: Person Detection FP16 - Device: CPU

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2024.0Model: Person Detection FP16 - Device: CPUabc306090120150SE +/- 1.56, N = 3137.85135.48138.64MIN: 112.51 / MAX: 221.34MIN: 115.39 / MAX: 224.16MIN: 114.61 / MAX: 225.731. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl

NAMD

Input: ATPase with 327,506 Atoms

OpenBenchmarking.orgns/day, More Is BetterNAMD 3.0b6Input: ATPase with 327,506 Atomsabc0.15120.30240.45360.60480.756SE +/- 0.00274, N = 30.672080.667330.65683

OpenVINO

Model: Person Detection FP16 - Device: CPU

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2024.0Model: Person Detection FP16 - Device: CPUabc714212835SE +/- 0.32, N = 329.0129.5028.851. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl

Neural Magic DeepSparse

Model: CV Classification, ResNet-50 ImageNet - Scenario: Synchronous Single-Stream

OpenBenchmarking.orgms/batch, Fewer Is BetterNeural Magic DeepSparse 1.7Model: CV Classification, ResNet-50 ImageNet - Scenario: Synchronous Single-Streamabc3691215SE +/- 0.11, N = 310.2710.0610.06

Neural Magic DeepSparse

Model: CV Classification, ResNet-50 ImageNet - Scenario: Synchronous Single-Stream

OpenBenchmarking.orgitems/sec, More Is BetterNeural Magic DeepSparse 1.7Model: CV Classification, ResNet-50 ImageNet - Scenario: Synchronous Single-Streamabc20406080100SE +/- 1.02, N = 397.3299.3399.39

Neural Magic DeepSparse

Model: NLP Text Classification, DistilBERT mnli - Scenario: Asynchronous Multi-Stream

OpenBenchmarking.orgms/batch, Fewer Is BetterNeural Magic DeepSparse 1.7Model: NLP Text Classification, DistilBERT mnli - Scenario: Asynchronous Multi-Streamabc1224364860SE +/- 0.49, N = 351.3250.2650.38

Neural Magic DeepSparse

Model: NLP Text Classification, DistilBERT mnli - Scenario: Asynchronous Multi-Stream

OpenBenchmarking.orgitems/sec, More Is BetterNeural Magic DeepSparse 1.7Model: NLP Text Classification, DistilBERT mnli - Scenario: Asynchronous Multi-Streamabc20406080100SE +/- 0.75, N = 377.9479.5879.38

ClickHouse

100M Rows Hits Dataset, Third Run

OpenBenchmarking.orgQueries Per Minute, Geo Mean, More Is BetterClickHouse 22.12.3.5100M Rows Hits Dataset, Third Runabc50100150200250SE +/- 1.48, N = 3243.23239.55238.31MIN: 9.39 / MAX: 10000MIN: 9.46 / MAX: 10000MIN: 9.23 / MAX: 8571.43

Neural Magic DeepSparse

Model: NLP Text Classification, BERT base uncased SST2, Sparse INT8 - Scenario: Asynchronous Multi-Stream

OpenBenchmarking.orgitems/sec, More Is BetterNeural Magic DeepSparse 1.7Model: NLP Text Classification, BERT base uncased SST2, Sparse INT8 - Scenario: Asynchronous Multi-Streamabc70140210280350SE +/- 0.48, N = 3334.71340.82334.20

Neural Magic DeepSparse

Model: NLP Text Classification, BERT base uncased SST2, Sparse INT8 - Scenario: Asynchronous Multi-Stream

OpenBenchmarking.orgms/batch, Fewer Is BetterNeural Magic DeepSparse 1.7Model: NLP Text Classification, BERT base uncased SST2, Sparse INT8 - Scenario: Asynchronous Multi-Streamabc3691215SE +/- 0.02, N = 311.9411.7211.96

SPECFEM3D

Model: Tomographic Model

OpenBenchmarking.orgSeconds, Fewer Is BetterSPECFEM3D 4.0Model: Tomographic Modelabc1326395265SE +/- 0.29, N = 356.6656.6155.571. (F9X) gfortran options: -O2 -fopenmp -std=f2003 -fimplicit-none -fmax-errors=10 -pedantic -pedantic-errors -O3 -finline-functions -lmpi_usempif08 -lmpi_mpifh -lmpi -lopen-rte -lopen-pal -lhwloc -levent_core -levent_pthreads -lm -lz

SVT-AV1

Encoder Mode: Preset 8 - Input: Bosphorus 4K

OpenBenchmarking.orgFrames Per Second, More Is BetterSVT-AV1 2.0Encoder Mode: Preset 8 - Input: Bosphorus 4Kabc816243240SE +/- 0.15, N = 335.4736.0236.161. (CXX) g++ options: -march=native -mno-avx -mavx2 -mavx512f -mavx512bw -mavx512dq

Neural Magic DeepSparse

Model: CV Segmentation, 90% Pruned YOLACT Pruned - Scenario: Synchronous Single-Stream

OpenBenchmarking.orgms/batch, Fewer Is BetterNeural Magic DeepSparse 1.7Model: CV Segmentation, 90% Pruned YOLACT Pruned - Scenario: Synchronous Single-Streamabc20406080100SE +/- 0.24, N = 395.4493.8593.66

Neural Magic DeepSparse

Model: NLP Text Classification, DistilBERT mnli - Scenario: Synchronous Single-Stream

OpenBenchmarking.orgms/batch, Fewer Is BetterNeural Magic DeepSparse 1.7Model: NLP Text Classification, DistilBERT mnli - Scenario: Synchronous Single-Streamabc48121620SE +/- 0.14, N = 314.8314.5514.56

Neural Magic DeepSparse

Model: CV Segmentation, 90% Pruned YOLACT Pruned - Scenario: Synchronous Single-Stream

OpenBenchmarking.orgitems/sec, More Is BetterNeural Magic DeepSparse 1.7Model: CV Segmentation, 90% Pruned YOLACT Pruned - Scenario: Synchronous Single-Streamabc3691215SE +/- 0.03, N = 310.4810.6510.68

Neural Magic DeepSparse

Model: CV Segmentation, 90% Pruned YOLACT Pruned - Scenario: Asynchronous Multi-Stream

OpenBenchmarking.orgms/batch, Fewer Is BetterNeural Magic DeepSparse 1.7Model: CV Segmentation, 90% Pruned YOLACT Pruned - Scenario: Asynchronous Multi-Streamabc80160240320400SE +/- 1.69, N = 3362.27355.53356.21

Neural Magic DeepSparse

Model: CV Segmentation, 90% Pruned YOLACT Pruned - Scenario: Asynchronous Multi-Stream

OpenBenchmarking.orgitems/sec, More Is BetterNeural Magic DeepSparse 1.7Model: CV Segmentation, 90% Pruned YOLACT Pruned - Scenario: Asynchronous Multi-Streamabc3691215SE +/- 0.05, N = 311.0411.2511.23

Neural Magic DeepSparse

Model: NLP Text Classification, DistilBERT mnli - Scenario: Synchronous Single-Stream

OpenBenchmarking.orgitems/sec, More Is BetterNeural Magic DeepSparse 1.7Model: NLP Text Classification, DistilBERT mnli - Scenario: Synchronous Single-Streamabc1530456075SE +/- 0.62, N = 367.4368.7168.67

Neural Magic DeepSparse

Model: ResNet-50, Sparse INT8 - Scenario: Synchronous Single-Stream

OpenBenchmarking.orgms/batch, Fewer Is BetterNeural Magic DeepSparse 1.7Model: ResNet-50, Sparse INT8 - Scenario: Synchronous Single-Streamabc0.33580.67161.00741.34321.679SE +/- 0.0116, N = 101.49261.48021.4665

Neural Magic DeepSparse

Model: BERT-Large, NLP Question Answering, Sparse INT8 - Scenario: Synchronous Single-Stream

OpenBenchmarking.orgms/batch, Fewer Is BetterNeural Magic DeepSparse 1.7Model: BERT-Large, NLP Question Answering, Sparse INT8 - Scenario: Synchronous Single-Streamabc3691215SE +/- 0.0846, N = 39.87769.71059.7892

Neural Magic DeepSparse

Model: BERT-Large, NLP Question Answering, Sparse INT8 - Scenario: Synchronous Single-Stream

OpenBenchmarking.orgitems/sec, More Is BetterNeural Magic DeepSparse 1.7Model: BERT-Large, NLP Question Answering, Sparse INT8 - Scenario: Synchronous Single-Streamabc20406080100SE +/- 0.86, N = 3101.16102.89102.06

Neural Magic DeepSparse

Model: ResNet-50, Sparse INT8 - Scenario: Synchronous Single-Stream

OpenBenchmarking.orgitems/sec, More Is BetterNeural Magic DeepSparse 1.7Model: ResNet-50, Sparse INT8 - Scenario: Synchronous Single-Streamabc150300450600750SE +/- 5.27, N = 10667.99673.15679.28

SPECFEM3D

Model: Mount St. Helens

OpenBenchmarking.orgSeconds, Fewer Is BetterSPECFEM3D 4.0Model: Mount St. Helensabc1326395265SE +/- 0.60, N = 556.7557.0756.141. (F9X) gfortran options: -O2 -fopenmp -std=f2003 -fimplicit-none -fmax-errors=10 -pedantic -pedantic-errors -O3 -finline-functions -lmpi_usempif08 -lmpi_mpifh -lmpi -lopen-rte -lopen-pal -lhwloc -levent_core -levent_pthreads -lm -lz

SVT-VP9

Tuning: VMAF Optimized - Input: Bosphorus 4K

OpenBenchmarking.orgFrames Per Second, More Is BetterSVT-VP9 0.3Tuning: VMAF Optimized - Input: Bosphorus 4Kabc20406080100SE +/- 0.99, N = 481.7680.4480.611. (CC) gcc options: -O3 -fcommon -fPIE -fPIC -fvisibility=hidden -pie -rdynamic -lpthread -lrt -lm

NAMD

Input: STMV with 1,066,628 Atoms

OpenBenchmarking.orgns/day, More Is BetterNAMD 3.0b6Input: STMV with 1,066,628 Atomsabc0.04710.09420.14130.18840.2355SE +/- 0.00013, N = 30.209370.209270.20601

Neural Magic DeepSparse

Model: NLP Text Classification, BERT base uncased SST2, Sparse INT8 - Scenario: Synchronous Single-Stream

OpenBenchmarking.orgms/batch, Fewer Is BetterNeural Magic DeepSparse 1.7Model: NLP Text Classification, BERT base uncased SST2, Sparse INT8 - Scenario: Synchronous Single-Streamabc0.87361.74722.62083.49444.368SE +/- 0.0378, N = 33.88253.82283.8275

Neural Magic DeepSparse

Model: NLP Text Classification, BERT base uncased SST2, Sparse INT8 - Scenario: Synchronous Single-Stream

OpenBenchmarking.orgitems/sec, More Is BetterNeural Magic DeepSparse 1.7Model: NLP Text Classification, BERT base uncased SST2, Sparse INT8 - Scenario: Synchronous Single-Streamabc60120180240300SE +/- 2.52, N = 3257.33261.27260.96

Timed Linux Kernel Compilation

Build: defconfig

OpenBenchmarking.orgSeconds, Fewer Is BetterTimed Linux Kernel Compilation 6.8Build: defconfigabc20406080100SE +/- 0.18, N = 391.8590.4990.51

OpenVINO

Model: Weld Porosity Detection FP16-INT8 - Device: CPU

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2024.0Model: Weld Porosity Detection FP16-INT8 - Device: CPUabc246810SE +/- 0.03, N = 37.687.717.60MIN: 4.73 / MAX: 36.9MIN: 4.97 / MAX: 32.25MIN: 5.3 / MAX: 31.741. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl

OpenVINO

Model: Road Segmentation ADAS FP16-INT8 - Device: CPU

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2024.0Model: Road Segmentation ADAS FP16-INT8 - Device: CPUabc50100150200250SE +/- 0.70, N = 3210.19213.22211.431. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl

OpenVINO

Model: Road Segmentation ADAS FP16-INT8 - Device: CPU

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2024.0Model: Road Segmentation ADAS FP16-INT8 - Device: CPUabc918273645SE +/- 0.13, N = 338.0437.5037.82MIN: 21.24 / MAX: 90.73MIN: 26.78 / MAX: 87.54MIN: 21.82 / MAX: 84.81. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl

Neural Magic DeepSparse

Model: CV Detection, YOLOv5s COCO, Sparse INT8 - Scenario: Asynchronous Multi-Stream

OpenBenchmarking.orgms/batch, Fewer Is BetterNeural Magic DeepSparse 1.7Model: CV Detection, YOLOv5s COCO, Sparse INT8 - Scenario: Asynchronous Multi-Streamabc1632486480SE +/- 0.13, N = 372.6371.7072.73

Neural Magic DeepSparse

Model: CV Detection, YOLOv5s COCO, Sparse INT8 - Scenario: Asynchronous Multi-Stream

OpenBenchmarking.orgitems/sec, More Is BetterNeural Magic DeepSparse 1.7Model: CV Detection, YOLOv5s COCO, Sparse INT8 - Scenario: Asynchronous Multi-Streamabc1326395265SE +/- 0.10, N = 355.0655.7854.99

OpenVINO

Model: Weld Porosity Detection FP16-INT8 - Device: CPU

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2024.0Model: Weld Porosity Detection FP16-INT8 - Device: CPUabc2004006008001000SE +/- 3.64, N = 31040.681036.691051.221. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl

uvg266

Video Input: Bosphorus 1080p - Video Preset: Slow

OpenBenchmarking.orgFrames Per Second, More Is Betteruvg266 0.4.1Video Input: Bosphorus 1080p - Video Preset: Slowabc816243240SE +/- 0.40, N = 332.9032.4932.94

GROMACS

Implementation: MPI CPU - Input: water_GMX50_bare

OpenBenchmarking.orgNs Per Day, More Is BetterGROMACS 2024Implementation: MPI CPU - Input: water_GMX50_bareabc0.31640.63280.94921.26561.582SE +/- 0.015, N = 51.3871.4061.4031. (CXX) g++ options: -O3 -lm

7-Zip Compression

Test: Compression Rating

OpenBenchmarking.orgMIPS, More Is Better7-Zip Compression 22.01Test: Compression Ratingabc20K40K60K80K100KSE +/- 806.12, N = 39560996616968921. (CXX) g++ options: -lpthread -ldl -O2 -fPIC

Neural Magic DeepSparse

Model: NLP Token Classification, BERT base uncased conll2003 - Scenario: Asynchronous Multi-Stream

OpenBenchmarking.orgms/batch, Fewer Is BetterNeural Magic DeepSparse 1.7Model: NLP Token Classification, BERT base uncased conll2003 - Scenario: Asynchronous Multi-Streamabc100200300400500SE +/- 0.33, N = 3445.64442.90439.84

Neural Magic DeepSparse

Model: NLP Token Classification, BERT base uncased conll2003 - Scenario: Asynchronous Multi-Stream

OpenBenchmarking.orgitems/sec, More Is BetterNeural Magic DeepSparse 1.7Model: NLP Token Classification, BERT base uncased conll2003 - Scenario: Asynchronous Multi-Streamabc3691215SE +/- 0.0066, N = 38.97569.03119.0939

BRL-CAD

VGR Performance Metric

OpenBenchmarking.orgVGR Performance Metric, More Is BetterBRL-CAD 7.38.2VGR Performance Metricabc40K80K120K160K200K1893841871481888331. (CXX) g++ options: -std=c++17 -pipe -fvisibility=hidden -fno-strict-aliasing -fno-common -fexceptions -ftemplate-depth-128 -m64 -ggdb3 -O3 -fipa-pta -fstrength-reduce -finline-functions -flto -ltcl8.6 -lnetpbm -lregex_brl -lz_brl -lassimp -ldl -lm -ltk8.6

OpenVINO

Model: Face Detection FP16-INT8 - Device: CPU

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2024.0Model: Face Detection FP16-INT8 - Device: CPUabc80160240320400SE +/- 1.42, N = 3371.82374.64370.27MIN: 283.89 / MAX: 476.9MIN: 283.87 / MAX: 471.9MIN: 284.02 / MAX: 518.251. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl

uvg266

Video Input: Bosphorus 4K - Video Preset: Ultra Fast

OpenBenchmarking.orgFrames Per Second, More Is Betteruvg266 0.4.1Video Input: Bosphorus 4K - Video Preset: Ultra Fastabc612182430SE +/- 0.14, N = 324.8924.7224.60

SVT-HEVC

Tuning: 1 - Input: Bosphorus 1080p

OpenBenchmarking.orgFrames Per Second, More Is BetterSVT-HEVC 1.5.0Tuning: 1 - Input: Bosphorus 1080pabc3691215SE +/- 0.03, N = 39.549.599.651. (CC) gcc options: -fPIE -fPIC -O3 -O2 -pie -rdynamic -lpthread -lrt

OpenVINO

Model: Face Detection FP16-INT8 - Device: CPU

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2024.0Model: Face Detection FP16-INT8 - Device: CPUabc3691215SE +/- 0.05, N = 310.7410.6710.791. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl

uvg266

Video Input: Bosphorus 1080p - Video Preset: Medium

OpenBenchmarking.orgFrames Per Second, More Is Betteruvg266 0.4.1Video Input: Bosphorus 1080p - Video Preset: Mediumabc918273645SE +/- 0.23, N = 337.1536.7536.90

OpenVINO

Model: Age Gender Recognition Retail 0013 FP16 - Device: CPU

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2024.0Model: Age Gender Recognition Retail 0013 FP16 - Device: CPUabc0.21150.4230.63450.8461.0575SE +/- 0.00, N = 30.930.930.94MIN: 0.5 / MAX: 10.02MIN: 0.55 / MAX: 9.6MIN: 0.57 / MAX: 17.211. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl

OpenVINO

Model: Age Gender Recognition Retail 0013 FP16-INT8 - Device: CPU

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2024.0Model: Age Gender Recognition Retail 0013 FP16-INT8 - Device: CPUabc5K10K15K20K25KSE +/- 20.16, N = 321990.1921792.1321757.771. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl

PostgreSQL

Scaling Factor: 100 - Clients: 1000 - Mode: Read Write - Average Latency

OpenBenchmarking.orgms, Fewer Is BetterPostgreSQL 16Scaling Factor: 100 - Clients: 1000 - Mode: Read Write - Average Latencyabc816243240SE +/- 0.14, N = 333.4033.5133.161. (CC) gcc options: -fno-strict-aliasing -fwrapv -O2 -lm -licui18n -licuuc -licudata

PostgreSQL

Scaling Factor: 100 - Clients: 1000 - Mode: Read Write

OpenBenchmarking.orgTPS, More Is BetterPostgreSQL 16Scaling Factor: 100 - Clients: 1000 - Mode: Read Writeabc6K12K18K24K30KSE +/- 122.20, N = 32994529843301601. (CC) gcc options: -fno-strict-aliasing -fwrapv -O2 -lm -licui18n -licuuc -licudata

Timed LLVM Compilation

Build System: Unix Makefiles

OpenBenchmarking.orgSeconds, Fewer Is BetterTimed LLVM Compilation 16.0Build System: Unix Makefilesabc140280420560700SE +/- 2.26, N = 3646.62639.85643.71

Timed LLVM Compilation

Build System: Ninja

OpenBenchmarking.orgSeconds, Fewer Is BetterTimed LLVM Compilation 16.0Build System: Ninjaabc140280420560700SE +/- 2.72, N = 3636.00629.47630.04

OpenFOAM

Input: drivaerFastback, Small Mesh Size - Mesh Time

OpenBenchmarking.orgSeconds, Fewer Is BetterOpenFOAM 10Input: drivaerFastback, Small Mesh Size - Mesh Timeabc71421283530.3330.0230.041. (CXX) g++ options: -std=c++14 -m64 -O3 -ftemplate-depth-100 -fPIC -fuse-ld=bfd -Xlinker --add-needed --no-as-needed -ldynamicMesh -lgenericPatchFields -lOpenFOAM -ldl -lm

Timed Godot Game Engine Compilation

Time To Compile

OpenBenchmarking.orgSeconds, Fewer Is BetterTimed Godot Game Engine Compilation 4.0Time To Compileabc70140210280350SE +/- 0.18, N = 3307.12306.66304.58

OpenVINO

Model: Age Gender Recognition Retail 0013 FP16 - Device: CPU

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2024.0Model: Age Gender Recognition Retail 0013 FP16 - Device: CPUabc2K4K6K8K10KSE +/- 16.19, N = 38563.768538.598495.041. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl

VVenC

Video Input: Bosphorus 4K - Video Preset: Faster

OpenBenchmarking.orgFrames Per Second, More Is BetterVVenC 1.11Video Input: Bosphorus 4K - Video Preset: Fasterabc3691215SE +/- 0.01, N = 311.6011.7011.651. (CXX) g++ options: -O3 -flto -fno-fat-lto-objects -flto=auto

SPECFEM3D

Model: Water-layered Halfspace

OpenBenchmarking.orgSeconds, Fewer Is BetterSPECFEM3D 4.0Model: Water-layered Halfspaceabc306090120150SE +/- 0.57, N = 3139.21139.33138.251. (F9X) gfortran options: -O2 -fopenmp -std=f2003 -fimplicit-none -fmax-errors=10 -pedantic -pedantic-errors -O3 -finline-functions -lmpi_usempif08 -lmpi_mpifh -lmpi -lopen-rte -lopen-pal -lhwloc -levent_core -levent_pthreads -lm -lz

x265

Video Input: Bosphorus 4K

OpenBenchmarking.orgFrames Per Second, More Is Betterx265 3.4Video Input: Bosphorus 4Kabc510152025SE +/- 0.08, N = 319.4819.5019.631. (CXX) g++ options: -O3 -rdynamic -lpthread -lrt -ldl -lnuma

VVenC

Video Input: Bosphorus 1080p - Video Preset: Fast

OpenBenchmarking.orgFrames Per Second, More Is BetterVVenC 1.11Video Input: Bosphorus 1080p - Video Preset: Fastabc48121620SE +/- 0.04, N = 315.4515.5715.491. (CXX) g++ options: -O3 -flto -fno-fat-lto-objects -flto=auto

OpenVINO

Model: Face Detection FP16 - Device: CPU

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2024.0Model: Face Detection FP16 - Device: CPUabc0.6031.2061.8092.4123.015SE +/- 0.01, N = 32.682.662.661. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl

OpenVINO

Model: Vehicle Detection FP16 - Device: CPU

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2024.0Model: Vehicle Detection FP16 - Device: CPUabc48121620SE +/- 0.04, N = 316.6016.6916.57MIN: 8.7 / MAX: 42.87MIN: 11.64 / MAX: 52.91MIN: 7.95 / MAX: 34.051. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl

SVT-AV1

Encoder Mode: Preset 4 - Input: Bosphorus 4K

OpenBenchmarking.orgFrames Per Second, More Is BetterSVT-AV1 2.0Encoder Mode: Preset 4 - Input: Bosphorus 4Kabc0.98261.96522.94783.93044.913SE +/- 0.009, N = 34.3544.3364.3671. (CXX) g++ options: -march=native -mno-avx -mavx2 -mavx512f -mavx512bw -mavx512dq

OpenVINO

Model: Vehicle Detection FP16 - Device: CPU

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2024.0Model: Vehicle Detection FP16 - Device: CPUabc50100150200250SE +/- 0.65, N = 3240.78239.55241.221. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl

Neural Magic DeepSparse

Model: Llama2 Chat 7b Quantized - Scenario: Asynchronous Multi-Stream

OpenBenchmarking.orgitems/sec, More Is BetterNeural Magic DeepSparse 1.7Model: Llama2 Chat 7b Quantized - Scenario: Asynchronous Multi-Streamabc1.14892.29783.44674.59565.7445SE +/- 0.0090, N = 35.07115.10635.0995

Neural Magic DeepSparse

Model: Llama2 Chat 7b Quantized - Scenario: Asynchronous Multi-Stream

OpenBenchmarking.orgms/batch, Fewer Is BetterNeural Magic DeepSparse 1.7Model: Llama2 Chat 7b Quantized - Scenario: Asynchronous Multi-Streamabc2004006008001000SE +/- 1.38, N = 3781.12775.76776.83

OpenVINO

Model: Noise Suppression Poconet-Like FP16 - Device: CPU

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2024.0Model: Noise Suppression Poconet-Like FP16 - Device: CPUabc48121620SE +/- 0.03, N = 315.1915.0915.11MIN: 9.31 / MAX: 55.49MIN: 9.56 / MAX: 53.46MIN: 10.08 / MAX: 54.611. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl

OpenVINO

Model: Machine Translation EN To DE FP16 - Device: CPU

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2024.0Model: Machine Translation EN To DE FP16 - Device: CPUabc816243240SE +/- 0.06, N = 333.6833.7033.481. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl

Blender

Blend File: BMW27 - Compute: CPU-Only

OpenBenchmarking.orgSeconds, Fewer Is BetterBlender 4.0Blend File: BMW27 - Compute: CPU-Onlyabc306090120150SE +/- 0.23, N = 3123.52122.72122.81

OpenVINO

Model: Machine Translation EN To DE FP16 - Device: CPU

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2024.0Model: Machine Translation EN To DE FP16 - Device: CPUabc306090120150SE +/- 0.23, N = 3118.74118.68119.45MIN: 92.44 / MAX: 207.3MIN: 90.15 / MAX: 196.79MIN: 92.57 / MAX: 198.031. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl

OpenVINO

Model: Noise Suppression Poconet-Like FP16 - Device: CPU

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2024.0Model: Noise Suppression Poconet-Like FP16 - Device: CPUabc110220330440550SE +/- 0.86, N = 3524.22527.54527.201. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl

SVT-VP9

Tuning: Visual Quality Optimized - Input: Bosphorus 1080p

OpenBenchmarking.orgFrames Per Second, More Is BetterSVT-VP9 0.3Tuning: Visual Quality Optimized - Input: Bosphorus 1080pabc50100150200250SE +/- 0.62, N = 3224.63225.96224.721. (CC) gcc options: -O3 -fcommon -fPIE -fPIC -fvisibility=hidden -pie -rdynamic -lpthread -lrt -lm

Blender

Blend File: Fishy Cat - Compute: CPU-Only

OpenBenchmarking.orgSeconds, Fewer Is BetterBlender 4.0Blend File: Fishy Cat - Compute: CPU-Onlyabc4080120160200SE +/- 0.37, N = 3165.95165.79165.05

OpenVINO

Model: Vehicle Detection FP16-INT8 - Device: CPU

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2024.0Model: Vehicle Detection FP16-INT8 - Device: CPUabc110220330440550SE +/- 2.47, N = 3497.27499.46499.921. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl

OpenVINO

Model: Weld Porosity Detection FP16 - Device: CPU

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2024.0Model: Weld Porosity Detection FP16 - Device: CPUabc714212835SE +/- 0.09, N = 329.2229.3529.37MIN: 15.45 / MAX: 72.82MIN: 15.67 / MAX: 72.92MIN: 15.64 / MAX: 69.571. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl

OpenVINO

Model: Weld Porosity Detection FP16 - Device: CPU

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2024.0Model: Weld Porosity Detection FP16 - Device: CPUabc60120180240300SE +/- 0.87, N = 3273.62272.46272.251. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl

OpenVINO

Model: Vehicle Detection FP16-INT8 - Device: CPU

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2024.0Model: Vehicle Detection FP16-INT8 - Device: CPUabc48121620SE +/- 0.08, N = 316.0816.0116.00MIN: 10.36 / MAX: 57.94MIN: 10.53 / MAX: 53.92MIN: 10.21 / MAX: 59.041. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl

OpenVINO

Model: Face Detection FP16 - Device: CPU

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2024.0Model: Face Detection FP16 - Device: CPUabc30060090012001500SE +/- 5.23, N = 31495.101502.551499.57MIN: 1179.18 / MAX: 1678.08MIN: 1181.84 / MAX: 1684.45MIN: 1217.27 / MAX: 1680.471. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl

OpenVINO

Model: Person Vehicle Bike Detection FP16 - Device: CPU

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2024.0Model: Person Vehicle Bike Detection FP16 - Device: CPUabc3691215SE +/- 0.01, N = 310.3810.4210.43MIN: 6.09 / MAX: 31.65MIN: 6.07 / MAX: 30.65MIN: 7.42 / MAX: 34.611. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl

OpenVINO

Model: Handwritten English Recognition FP16-INT8 - Device: CPU

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2024.0Model: Handwritten English Recognition FP16-INT8 - Device: CPUabc1020304050SE +/- 0.23, N = 343.9243.8844.09MIN: 32.41 / MAX: 102.38MIN: 32.8 / MAX: 111.1MIN: 32.81 / MAX: 104.251. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl

OpenVINO

Model: Handwritten English Recognition FP16-INT8 - Device: CPU

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2024.0Model: Handwritten English Recognition FP16-INT8 - Device: CPUabc4080120160200SE +/- 0.96, N = 3182.10182.21181.391. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl

uvg266

Video Input: Bosphorus 4K - Video Preset: Medium

OpenBenchmarking.orgFrames Per Second, More Is Betteruvg266 0.4.1Video Input: Bosphorus 4K - Video Preset: Mediumabc246810SE +/- 0.00, N = 36.686.686.71

OpenVINO

Model: Person Vehicle Bike Detection FP16 - Device: CPU

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2024.0Model: Person Vehicle Bike Detection FP16 - Device: CPUabc80160240320400SE +/- 0.46, N = 3384.90383.45383.231. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl

SVT-HEVC

Tuning: 1 - Input: Bosphorus 4K

OpenBenchmarking.orgFrames Per Second, More Is BetterSVT-HEVC 1.5.0Tuning: 1 - Input: Bosphorus 4Kabc0.53551.0711.60652.1422.6775SE +/- 0.00, N = 32.382.372.371. (CC) gcc options: -fPIE -fPIC -O3 -O2 -pie -rdynamic -lpthread -lrt

uvg266

Video Input: Bosphorus 4K - Video Preset: Very Fast

OpenBenchmarking.orgFrames Per Second, More Is Betteruvg266 0.4.1Video Input: Bosphorus 4K - Video Preset: Very Fastabc510152025SE +/- 0.12, N = 319.3119.2319.27

OpenVINO

Model: Handwritten English Recognition FP16 - Device: CPU

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2024.0Model: Handwritten English Recognition FP16 - Device: CPUabc1224364860SE +/- 0.22, N = 352.0451.8351.99MIN: 38.84 / MAX: 116.68MIN: 39.59 / MAX: 122.23MIN: 39.27 / MAX: 121.441. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl

OpenVINO

Model: Handwritten English Recognition FP16 - Device: CPU

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2024.0Model: Handwritten English Recognition FP16 - Device: CPUabc306090120150SE +/- 0.64, N = 3153.66154.28153.811. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl

OpenVINO

Model: Person Re-Identification Retail FP16 - Device: CPU

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2024.0Model: Person Re-Identification Retail FP16 - Device: CPUabc48121620SE +/- 0.03, N = 318.1818.1718.24MIN: 9.82 / MAX: 46.59MIN: 9.62 / MAX: 52.96MIN: 9.91 / MAX: 48.241. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl

OpenVINO

Model: Person Re-Identification Retail FP16 - Device: CPU

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2024.0Model: Person Re-Identification Retail FP16 - Device: CPUabc100200300400500SE +/- 0.80, N = 3439.76439.93438.291. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl

Blender

Blend File: Classroom - Compute: CPU-Only

OpenBenchmarking.orgSeconds, Fewer Is BetterBlender 4.0Blend File: Classroom - Compute: CPU-Onlyabc70140210280350SE +/- 0.33, N = 3336.83335.62336.08

Timed Node.js Compilation

Time To Compile

OpenBenchmarking.orgSeconds, Fewer Is BetterTimed Node.js Compilation 19.8.1Time To Compileabc110220330440550SE +/- 0.47, N = 3523.99522.11523.42

OpenFOAM

Input: drivaerFastback, Medium Mesh Size - Mesh Time

OpenBenchmarking.orgSeconds, Fewer Is BetterOpenFOAM 10Input: drivaerFastback, Medium Mesh Size - Mesh Timeabc50100150200250221.91222.69222.191. (CXX) g++ options: -std=c++14 -m64 -O3 -ftemplate-depth-100 -fPIC -fuse-ld=bfd -Xlinker --add-needed --no-as-needed -ldynamicMesh -lgenericPatchFields -lOpenFOAM -ldl -lm

SVT-VP9

Tuning: PSNR/SSIM Optimized - Input: Bosphorus 1080p

OpenBenchmarking.orgFrames Per Second, More Is BetterSVT-VP9 0.3Tuning: PSNR/SSIM Optimized - Input: Bosphorus 1080pabc60120180240300SE +/- 0.30, N = 3291.27291.65292.251. (CC) gcc options: -O3 -fcommon -fPIE -fPIC -fvisibility=hidden -pie -rdynamic -lpthread -lrt -lm

OpenVINO

Model: Face Detection Retail FP16-INT8 - Device: CPU

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2024.0Model: Face Detection Retail FP16-INT8 - Device: CPUabc400800120016002000SE +/- 5.93, N = 31634.311639.461636.641. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl

Blender

Blend File: Barbershop - Compute: CPU-Only

OpenBenchmarking.orgSeconds, Fewer Is BetterBlender 4.0Blend File: Barbershop - Compute: CPU-Onlyabc30060090012001500SE +/- 0.90, N = 31297.101298.171294.72

Timed Linux Kernel Compilation

Build: allmodconfig

OpenBenchmarking.orgSeconds, Fewer Is BetterTimed Linux Kernel Compilation 6.8Build: allmodconfigabc30060090012001500SE +/- 0.39, N = 31311.871308.541308.93

SVT-HEVC

Tuning: 10 - Input: Bosphorus 1080p

OpenBenchmarking.orgFrames Per Second, More Is BetterSVT-HEVC 1.5.0Tuning: 10 - Input: Bosphorus 1080pabc80160240320400SE +/- 0.21, N = 3381.27381.92382.171. (CC) gcc options: -fPIE -fPIC -O3 -O2 -pie -rdynamic -lpthread -lrt

Timed GCC Compilation

Time To Compile

OpenBenchmarking.orgSeconds, Fewer Is BetterTimed GCC Compilation 13.2Time To Compileabc2004006008001000SE +/- 1.92, N = 3791.27790.87789.59

OpenVINO

Model: Face Detection Retail FP16-INT8 - Device: CPU

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2024.0Model: Face Detection Retail FP16-INT8 - Device: CPUabc1.10032.20063.30094.40125.5015SE +/- 0.02, N = 34.894.884.88MIN: 3.4 / MAX: 23.02MIN: 3.54 / MAX: 21.77MIN: 3.52 / MAX: 27.881. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl

uvg266

Video Input: Bosphorus 4K - Video Preset: Slow

OpenBenchmarking.orgFrames Per Second, More Is Betteruvg266 0.4.1Video Input: Bosphorus 4K - Video Preset: Slowabc1.34552.6914.03655.3826.7275SE +/- 0.01, N = 35.975.985.98

OpenFOAM

Input: drivaerFastback, Small Mesh Size - Execution Time

OpenBenchmarking.orgSeconds, Fewer Is BetterOpenFOAM 10Input: drivaerFastback, Small Mesh Size - Execution Timeabc50100150200250226.48226.11226.421. (CXX) g++ options: -std=c++14 -m64 -O3 -ftemplate-depth-100 -fPIC -fuse-ld=bfd -Xlinker --add-needed --no-as-needed -ldynamicMesh -lgenericPatchFields -lOpenFOAM -ldl -lm

VVenC

Video Input: Bosphorus 4K - Video Preset: Fast

OpenBenchmarking.orgFrames Per Second, More Is BetterVVenC 1.11Video Input: Bosphorus 4K - Video Preset: Fastabc1.11472.22943.34414.45885.5735SE +/- 0.024, N = 34.9544.9464.9521. (CXX) g++ options: -O3 -flto -fno-fat-lto-objects -flto=auto

Neural Magic DeepSparse

Model: Llama2 Chat 7b Quantized - Scenario: Synchronous Single-Stream

OpenBenchmarking.orgms/batch, Fewer Is BetterNeural Magic DeepSparse 1.7Model: Llama2 Chat 7b Quantized - Scenario: Synchronous Single-Streamabc306090120150SE +/- 0.06, N = 3124.62124.47124.55

Neural Magic DeepSparse

Model: Llama2 Chat 7b Quantized - Scenario: Synchronous Single-Stream

OpenBenchmarking.orgitems/sec, More Is BetterNeural Magic DeepSparse 1.7Model: Llama2 Chat 7b Quantized - Scenario: Synchronous Single-Streamabc246810SE +/- 0.0042, N = 38.02348.03298.0276

SVT-AV1

Encoder Mode: Preset 12 - Input: Bosphorus 1080p

OpenBenchmarking.orgFrames Per Second, More Is BetterSVT-AV1 2.0Encoder Mode: Preset 12 - Input: Bosphorus 1080pabc120240360480600SE +/- 1.61, N = 3532.48531.91532.511. (CXX) g++ options: -march=native -mno-avx -mavx2 -mavx512f -mavx512bw -mavx512dq

Blender

Blend File: Pabellon Barcelona - Compute: CPU-Only

OpenBenchmarking.orgSeconds, Fewer Is BetterBlender 4.0Blend File: Pabellon Barcelona - Compute: CPU-Onlyabc90180270360450SE +/- 0.40, N = 3414.01413.66413.87

SVT-AV1

Encoder Mode: Preset 13 - Input: Bosphorus 1080p

OpenBenchmarking.orgFrames Per Second, More Is BetterSVT-AV1 2.0Encoder Mode: Preset 13 - Input: Bosphorus 1080pabc140280420560700SE +/- 0.52, N = 3644.28644.76644.701. (CXX) g++ options: -march=native -mno-avx -mavx2 -mavx512f -mavx512bw -mavx512dq

OpenFOAM

Input: drivaerFastback, Medium Mesh Size - Execution Time

OpenBenchmarking.orgSeconds, Fewer Is BetterOpenFOAM 10Input: drivaerFastback, Medium Mesh Size - Execution Timeabc50010001500200025002182.842184.342183.101. (CXX) g++ options: -std=c++14 -m64 -O3 -ftemplate-depth-100 -fPIC -fuse-ld=bfd -Xlinker --add-needed --no-as-needed -ldynamicMesh -lgenericPatchFields -lOpenFOAM -ldl -lm

OpenVINO

Model: Age Gender Recognition Retail 0013 FP16-INT8 - Device: CPU

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2024.0Model: Age Gender Recognition Retail 0013 FP16-INT8 - Device: CPUabc0.0810.1620.2430.3240.405SE +/- 0.00, N = 30.360.360.36MIN: 0.19 / MAX: 8.34MIN: 0.21 / MAX: 7.95MIN: 0.19 / MAX: 8.11. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl

PostgreSQL

Scaling Factor: 1 - Clients: 1000 - Mode: Read Write - Average Latency

OpenBenchmarking.orgms, Fewer Is BetterPostgreSQL 16Scaling Factor: 1 - Clients: 1000 - Mode: Read Write - Average Latencyabc120240360480600SE +/- 10.12, N = 12575.61541.63551.011. (CC) gcc options: -fno-strict-aliasing -fwrapv -O2 -lm -licui18n -licuuc -licudata

x264

Video Input: Bosphorus 1080p

OpenBenchmarking.orgFrames Per Second, More Is Betterx264 2022-02-22Video Input: Bosphorus 1080pabc306090120150SE +/- 2.82, N = 15155.70157.78152.151. (CC) gcc options: -ldl -lavformat -lavcodec -lavutil -lswscale -m64 -lm -lpthread -O3 -flto

SVT-HEVC

Tuning: 7 - Input: Bosphorus 1080p

OpenBenchmarking.orgFrames Per Second, More Is BetterSVT-HEVC 1.5.0Tuning: 7 - Input: Bosphorus 1080pabc4080120160200SE +/- 3.25, N = 15171.89166.20185.241. (CC) gcc options: -fPIE -fPIC -O3 -O2 -pie -rdynamic -lpthread -lrt

SPECFEM3D

Model: Homogeneous Halfspace

OpenBenchmarking.orgSeconds, Fewer Is BetterSPECFEM3D 4.0Model: Homogeneous Halfspaceabc1632486480SE +/- 1.31, N = 1272.9272.2671.571. (F9X) gfortran options: -O2 -fopenmp -std=f2003 -fimplicit-none -fmax-errors=10 -pedantic -pedantic-errors -O3 -finline-functions -lmpi_usempif08 -lmpi_mpifh -lmpi -lopen-rte -lopen-pal -lhwloc -levent_core -levent_pthreads -lm -lz

Phoronix Test Suite v10.8.5