AMD EPYC Turin 2025 New AVX-512 Benchmarks

AMD EPYC 9655P AVX-512 on/off benchmarks by Michael Larabel for a future article.

HTML result view exported from: https://openbenchmarking.org/result/2501295-NE-AMDEPYCTU09&grt&sro.

ProcessorMotherboardChipsetMemoryDiskGraphicsNetworkOSKernelDesktopDisplay ServerCompilerFile-SystemScreen ResolutionEPYC Turin AVX-512 Enabled AVX-512 DisabledAMD EPYC 9655P 96-Core @ 2.60GHz (96 Cores / 192 Threads)Supermicro Super Server H13SSL-N v1.01 (3.0 BIOS)AMD 1Ah12 x 64GB DDR5-6000MT/s Micron MTC40F2046S1RC64BDY QSFF3201GB Micron_7450_MTFDKCB3T2TFSASPEED2 x Broadcom NetXtreme BCM5720 PCIeUbuntu 24.106.13.0-rc4-phx-stock (x86_64)GNOME Shell 47.0X ServerGCC 14.2.0ext41024x768OpenBenchmarking.orgKernel Details- Transparent Huge Pages: madviseEnvironment Details- CXXFLAGS="-O3 -march=znver5 -mprefer-vector-width=512 -flto" CFLAGS="-O3 -march=znver5 -mprefer-vector-width=512 -flto" Compiler Details- --build=x86_64-linux-gnu --disable-vtable-verify --disable-werror --enable-bootstrap --enable-cet --enable-checking=release --enable-clocale=gnu --enable-default-pie --enable-gnu-unique-object --enable-languages=c,ada,c++,go,d,fortran,objc,obj-c++,m2,rust --enable-libphobos-checking=release --enable-libstdcxx-backtrace --enable-libstdcxx-debug --enable-libstdcxx-time=yes --enable-link-serialization=2 --enable-multiarch --enable-multilib --enable-nls --enable-objc-gc=auto --enable-offload-defaulted --enable-offload-targets=nvptx-none=/build/gcc-14-zdkDXv/gcc-14-14.2.0/debian/tmp-nvptx/usr,amdgcn-amdhsa=/build/gcc-14-zdkDXv/gcc-14-14.2.0/debian/tmp-gcn/usr --enable-plugin --enable-shared --enable-threads=posix --host=x86_64-linux-gnu --program-prefix=x86_64-linux-gnu- --target=x86_64-linux-gnu --with-abi=m64 --with-arch-32=i686 --with-build-config=bootstrap-lto-lean --with-default-libstdcxx-abi=new --with-gcc-major-version-only --with-multilib-list=m32,m64,mx32 --with-target-system-zlib=auto --with-tune=generic --without-cuda-driver -v Processor Details- Scaling Governor: acpi-cpufreq performance (Boost: Enabled) - CPU Microcode: 0xb002116 Python Details- Python 3.12.7Security Details- gather_data_sampling: Not affected + itlb_multihit: Not affected + l1tf: Not affected + mds: Not affected + meltdown: Not affected + mmio_stale_data: Not affected + reg_file_data_sampling: Not affected + retbleed: Not affected + spec_rstack_overflow: Not affected + spec_store_bypass: Mitigation of SSB disabled via prctl + spectre_v1: Mitigation of usercopy/swapgs barriers and __user pointer sanitization + spectre_v2: Mitigation of Enhanced / Automatic IBRS; IBPB: conditional; STIBP: always-on; RSB filling; PBRSB-eIBRS: Not affected; BHI: Not affected + srbds: Not affected + tsx_async_abort: Not affected

mt-dgemm: Sustained Floating-Point Ratecpuminer-opt: scryptcpuminer-opt: Skeincoincpuminer-opt: LBC, LBRY Creditsembree: Pathtracer ISPC - Asian Dragonembree: Pathtracer ISPC - Asian Dragon Objembree: Pathtracer ISPC - Crowngromacs: MPI CPU - water_GMX50_barelaghos: Sedov Blast Wave, ube_922_hex.meshlibxsmm: 128llama-cpp: CPU BLAS - Llama-3.1-Tulu-3-8B-Q8_0 - Prompt Processing 512llama-cpp: CPU BLAS - Llama-3.1-Tulu-3-8B-Q8_0 - Prompt Processing 1024llama-cpp: CPU BLAS - Llama-3.1-Tulu-3-8B-Q8_0 - Prompt Processing 2048llama-cpp: CPU BLAS - granite-3.0-3b-a800m-instruct-Q8_0 - Prompt Processing 512llama-cpp: CPU BLAS - Mistral-7B-Instruct-v0.3-Q8_0 - Prompt Processing 512llama-cpp: CPU BLAS - Mistral-7B-Instruct-v0.3-Q8_0 - Prompt Processing 1024llama-cpp: CPU BLAS - Mistral-7B-Instruct-v0.3-Q8_0 - Prompt Processing 2048minibude: OpenMP - BM1minibude: OpenMP - BM1minibude: OpenMP - BM2minibude: OpenMP - BM2mnn: mobilenetV3mnn: resnet-v2-50mnn: SqueezeNetV1.0numpy: onednn: Deconvolution Batch shapes_1d - CPUonednn: Deconvolution Batch shapes_3d - CPUonednn: IP Shapes 1D - CPUonednn: IP Shapes 3D - CPUonednn: Recurrent Neural Network Training - CPUonednn: Recurrent Neural Network Inference - CPUonnx: fcn-resnet101-11 - CPU - Standardonnx: fcn-resnet101-11 - CPU - Standardonnx: super-resolution-10 - CPU - Standardonnx: super-resolution-10 - CPU - Standardonnx: bertsquad-12 - CPU - Standardonnx: bertsquad-12 - CPU - Standardonnx: GPT-2 - CPU - Standardonnx: GPT-2 - CPU - Standardonnx: ArcFace ResNet-100 - CPU - Standardonnx: ArcFace ResNet-100 - CPU - Standardonnx: Faster R-CNN R-50-FPN-int8 - CPU - Standardonnx: Faster R-CNN R-50-FPN-int8 - CPU - Standardonnx: T5 Encoder - CPU - Standardonnx: T5 Encoder - CPU - Standardonnx: ResNet101_DUC_HDC-12 - CPU - Standardonnx: ResNet101_DUC_HDC-12 - CPU - Standardopenvino: Face Detection FP16-INT8 - CPUopenvino: Face Detection FP16-INT8 - CPUopenvino: Age Gender Recognition Retail 0013 FP16-INT8 - CPUopenvino: Age Gender Recognition Retail 0013 FP16-INT8 - CPUopenvino: Person Detection FP16 - CPUopenvino: Person Detection FP16 - CPUopenvino: Weld Porosity Detection FP16 - CPUopenvino: Weld Porosity Detection FP16 - CPUopenvino: Person Vehicle Bike Detection FP16 - CPUopenvino: Person Vehicle Bike Detection FP16 - CPUopenvino: Machine Translation EN To DE FP16 - CPUopenvino: Machine Translation EN To DE FP16 - CPUopenvino: Face Detection Retail FP16 - CPUopenvino: Face Detection Retail FP16 - CPUopenvino: Handwritten English Recognition FP16 - CPUopenvino: Handwritten English Recognition FP16 - CPUopenvino: Road Segmentation ADAS FP16 - CPUopenvino: Road Segmentation ADAS FP16 - CPUopenvino: Person Re-Identification Retail FP16 - CPUopenvino: Person Re-Identification Retail FP16 - CPUopenvino: Noise Suppression Poconet-Like FP16 - CPUopenvino: Noise Suppression Poconet-Like FP16 - CPUopenvino-genai: TinyLlama-1.1B-Chat-v1.0 - CPUopenvino-genai: TinyLlama-1.1B-Chat-v1.0 - CPU - Time To First Tokenopenvino-genai: TinyLlama-1.1B-Chat-v1.0 - CPU - Time Per Output Tokenopenvkl: vklBenchmarkCPU ISPCospray: gravity_spheres_volume/dim_512/ao/real_timeospray: gravity_spheres_volume/dim_512/scivis/real_timeospray: gravity_spheres_volume/dim_512/pathtracer/real_timeospray-studio: 1 - 4K - 1 - Path Tracer - CPUospray-studio: 1 - 4K - 16 - Path Tracer - CPUospray-studio: 1 - 4K - 32 - Path Tracer - CPUospray-studio: 2 - 4K - 1 - Path Tracer - CPUospray-studio: 2 - 4K - 16 - Path Tracer - CPUospray-studio: 2 - 4K - 32 - Path Tracer - CPUospray-studio: 3 - 4K - 1 - Path Tracer - CPUospray-studio: 3 - 4K - 16 - Path Tracer - CPUospray-studio: 3 - 4K - 32 - Path Tracer - CPUpytorch: CPU - 256 - ResNet-50pytorch: CPU - 512 - ResNet-50smhasher: FarmHash32 x86_64 AVXsmhasher: FarmHash32 x86_64 AVXsrsran: PDSCH Processor Benchmark, Throughput Totalsvt-av1: Preset 13 - Bosphorus 4Ksvt-av1: Preset 8 - Bosphorus 4Ksvt-av1: Preset 5 - Bosphorus 4Ksvt-av1: Preset 3 - Bosphorus 4Ktensorflow: CPU - 256 - ResNet-50tensorflow: CPU - 512 - ResNet-50y-cruncher: 5By-cruncher: 10BEPYC Turin AVX-512 Enabled AVX-512 Disabled4225.4328972937.261286043750993172.9627148.3243136.415517.416603.923422.0107.84106.71107.34356.76109.25107.32108.746368.737254.7507162.269286.4911.7367.4093.121882.006.724660.7169790.5334140.265927428.500278.3109.56433104.8273215.1624.6475022.433144.5813196.3485.0910848.781720.503064.066115.6086244.9894.081146.50795153.707147.06325.74162706.140.31710.0467.477611.1712.536775.467.03857.1655.9516580.182.823488.2327.502102.0322.7810524.044.526825.0913.4766.5718.5915.03281735.284934.689138.1975868138233095887313901310771024162973595550.5250.8437638.6222.808124852.3416.846184.79156.47715.649212.82241.2221.84744.05370.732517846.44747190245007159.2583138.2788126.631714.125590.603004.695.0393.4595.47347.1096.1594.9095.273416.427136.6573666.094146.6441.8039.9503.327828.6712.86011.354390.6462113.07554491.687307.0506.58603151.849170.9495.8509617.759056.3086182.8105.4682040.554824.669437.152026.9708238.6404.189724.30914235.70585.54558.74129762.910.53241.77198.172189.0643.733132.5015.26295.88162.026688.987.141200.3779.90713.9667.123605.7513.302516.8938.0591.4027.0110.97235324.028023.167831.5810966153823466897415503344801135180983978543.3343.4436139.9522.658110897.9412.131171.28951.73514.521152.47161.2733.08568.129OpenBenchmarking.org

ACES DGEMM

Sustained Floating-Point Rate

OpenBenchmarking.orgGFLOP/s, More Is BetterACES DGEMM 1.0Sustained Floating-Point RateAVX-512 DisabledAVX-512 Enabled9001800270036004500SE +/- 0.03, N = 3SE +/- 8.70, N = 470.734225.431. (CC) gcc options: -ffast-math -O3 -march=znver5 -flto -mavx2 -fopenmp -lopenblas

ACES DGEMM

CPU Peak Freq (Highest CPU Core Frequency) Monitor

OpenBenchmarking.orgMegahertz, More Is BetterACES DGEMM 1.0CPU Peak Freq (Highest CPU Core Frequency) MonitorAVX-512 DisabledAVX-512 Enabled8001600240032004000Min: 2600 / Avg: 4512.78 / Max: 4573Min: 2187 / Avg: 2495.27 / Max: 4544

ACES DGEMM

CPU Power Consumption Monitor

OpenBenchmarking.orgWatts, Fewer Is BetterACES DGEMM 1.0CPU Power Consumption MonitorAVX-512 DisabledAVX-512 Enabled50100150200250Min: 36.88 / Avg: 85.07 / Max: 92.11Min: 36.7 / Avg: 194.6 / Max: 256.01

ACES DGEMM

CPU Temperature Monitor

OpenBenchmarking.orgCelsius, Fewer Is BetterACES DGEMM 1.0CPU Temperature MonitorAVX-512 DisabledAVX-512 Enabled1122334455Min: 41.25 / Avg: 49.42 / Max: 57.38Min: 46 / Avg: 48.6 / Max: 50.5

Cpuminer-Opt

Algorithm: scrypt

OpenBenchmarking.orgkH/s, More Is BetterCpuminer-Opt 24.3Algorithm: scryptAVX-512 DisabledAVX-512 Enabled6001200180024003000SE +/- 0.72, N = 3SE +/- 3.40, N = 3846.442937.26-mno-avx512f1. (CXX) g++ options: -O3 -march=znver5 -flto -lcurl -lz -lpthread -lgmp

Cpuminer-Opt

CPU Peak Freq (Highest CPU Core Frequency) Monitor

OpenBenchmarking.orgMegahertz, More Is BetterCpuminer-Opt 24.3CPU Peak Freq (Highest CPU Core Frequency) MonitorAVX-512 DisabledAVX-512 Enabled8001600240032004000Min: 2600 / Avg: 3767.56 / Max: 4534Min: 2600 / Avg: 3550.91 / Max: 4551

Cpuminer-Opt

CPU Power Consumption Monitor

OpenBenchmarking.orgWatts, Fewer Is BetterCpuminer-Opt 24.3CPU Power Consumption MonitorAVX-512 DisabledAVX-512 Enabled70140210280350Min: 18.63 / Avg: 330.99 / Max: 379.47Min: 37.17 / Avg: 332.45 / Max: 374.99

Cpuminer-Opt

CPU Temperature Monitor

OpenBenchmarking.orgCelsius, Fewer Is BetterCpuminer-Opt 24.3CPU Temperature MonitorAVX-512 DisabledAVX-512 Enabled1122334455Min: 28.25 / Avg: 48.52 / Max: 53.13Min: 26.75 / Avg: 49.05 / Max: 54.88

Cpuminer-Opt

Algorithm: Skeincoin

OpenBenchmarking.orgkH/s, More Is BetterCpuminer-Opt 24.3Algorithm: SkeincoinAVX-512 DisabledAVX-512 Enabled300K600K900K1200K1500KSE +/- 2345.30, N = 3SE +/- 6599.64, N = 37471901286043-mno-avx512f1. (CXX) g++ options: -O3 -march=znver5 -flto -lcurl -lz -lpthread -lgmp

Cpuminer-Opt

CPU Peak Freq (Highest CPU Core Frequency) Monitor

OpenBenchmarking.orgMegahertz, More Is BetterCpuminer-Opt 24.3CPU Peak Freq (Highest CPU Core Frequency) MonitorAVX-512 DisabledAVX-512 Enabled8001600240032004000Min: 2600 / Avg: 3492.86 / Max: 4537Min: 2600 / Avg: 3208.97 / Max: 4575

Cpuminer-Opt

CPU Power Consumption Monitor

OpenBenchmarking.orgWatts, Fewer Is BetterCpuminer-Opt 24.3CPU Power Consumption MonitorAVX-512 DisabledAVX-512 Enabled70140210280350Min: 7.98 / Avg: 318.09 / Max: 366.04Min: 0.73 / Avg: 312.01 / Max: 355.44

Cpuminer-Opt

CPU Temperature Monitor

OpenBenchmarking.orgCelsius, Fewer Is BetterCpuminer-Opt 24.3CPU Temperature MonitorAVX-512 DisabledAVX-512 Enabled1224364860Min: 40.75 / Avg: 53.68 / Max: 56.5Min: 42.75 / Avg: 55.68 / Max: 58.88

Cpuminer-Opt

Algorithm: LBC, LBRY Credits

OpenBenchmarking.orgkH/s, More Is BetterCpuminer-Opt 24.3Algorithm: LBC, LBRY CreditsAVX-512 DisabledAVX-512 Enabled160K320K480K640K800KSE +/- 2841.66, N = 3SE +/- 622.53, N = 3245007750993-mno-avx512f1. (CXX) g++ options: -O3 -march=znver5 -flto -lcurl -lz -lpthread -lgmp

Cpuminer-Opt

CPU Peak Freq (Highest CPU Core Frequency) Monitor

OpenBenchmarking.orgMegahertz, More Is BetterCpuminer-Opt 24.3CPU Peak Freq (Highest CPU Core Frequency) MonitorAVX-512 DisabledAVX-512 Enabled8001600240032004000Min: 2600 / Avg: 3245.21 / Max: 4555Min: 2600 / Avg: 3198.91 / Max: 4544

Cpuminer-Opt

CPU Power Consumption Monitor

OpenBenchmarking.orgWatts, Fewer Is BetterCpuminer-Opt 24.3CPU Power Consumption MonitorAVX-512 DisabledAVX-512 Enabled60120180240300Min: 36.66 / Avg: 314.81 / Max: 357.18Min: 37.18 / Avg: 311.96 / Max: 353.17

Cpuminer-Opt

CPU Temperature Monitor

OpenBenchmarking.orgCelsius, Fewer Is BetterCpuminer-Opt 24.3CPU Temperature MonitorAVX-512 DisabledAVX-512 Enabled1122334455Min: 40.5 / Avg: 52.67 / Max: 56.13Min: 42.88 / Avg: 55.37 / Max: 58

Embree

Binary: Pathtracer ISPC - Model: Asian Dragon

OpenBenchmarking.orgFrames Per Second, More Is BetterEmbree 4.3Binary: Pathtracer ISPC - Model: Asian DragonAVX-512 DisabledAVX-512 Enabled4080120160200SE +/- 0.10, N = 7SE +/- 0.06, N = 8159.26172.96MIN: 154.75 / MAX: 162.47MIN: 169.32 / MAX: 176.68

Embree

CPU Peak Freq (Highest CPU Core Frequency) Monitor

OpenBenchmarking.orgMegahertz, More Is BetterEmbree 4.3CPU Peak Freq (Highest CPU Core Frequency) MonitorAVX-512 DisabledAVX-512 Enabled8001600240032004000Min: 2600 / Avg: 3040.09 / Max: 4547Min: 2600 / Avg: 3128.57 / Max: 4560

Embree

CPU Power Consumption Monitor

OpenBenchmarking.orgWatts, Fewer Is BetterEmbree 4.3CPU Power Consumption MonitorAVX-512 DisabledAVX-512 Enabled60120180240300Min: 0.18 / Avg: 204.2 / Max: 361.91Min: 1.68 / Avg: 193.84 / Max: 362.36

Embree

CPU Temperature Monitor

OpenBenchmarking.orgCelsius, Fewer Is BetterEmbree 4.3CPU Temperature MonitorAVX-512 DisabledAVX-512 Enabled918273645Min: 30.25 / Avg: 41.89 / Max: 46Min: 33.5 / Avg: 42.23 / Max: 46

Embree

Binary: Pathtracer ISPC - Model: Asian Dragon Obj

OpenBenchmarking.orgFrames Per Second, More Is BetterEmbree 4.3Binary: Pathtracer ISPC - Model: Asian Dragon ObjAVX-512 DisabledAVX-512 Enabled306090120150SE +/- 0.18, N = 4SE +/- 0.11, N = 5138.28148.32MIN: 134.48 / MAX: 141.24MIN: 144.93 / MAX: 151.79

Embree

CPU Peak Freq (Highest CPU Core Frequency) Monitor

OpenBenchmarking.orgMegahertz, More Is BetterEmbree 4.3CPU Peak Freq (Highest CPU Core Frequency) MonitorAVX-512 DisabledAVX-512 Enabled8001600240032004000Min: 2600 / Avg: 3719.73 / Max: 4565Min: 2600 / Avg: 3694.68 / Max: 4564

Embree

CPU Power Consumption Monitor

OpenBenchmarking.orgWatts, Fewer Is BetterEmbree 4.3CPU Power Consumption MonitorAVX-512 DisabledAVX-512 Enabled60120180240300Min: 0.78 / Avg: 165.7 / Max: 361.58Min: 36.52 / Avg: 163.55 / Max: 361.39

Embree

CPU Temperature Monitor

OpenBenchmarking.orgCelsius, Fewer Is BetterEmbree 4.3CPU Temperature MonitorAVX-512 DisabledAVX-512 Enabled1122334455Min: 35.88 / Avg: 48.67 / Max: 53.88Min: 35.38 / Avg: 48.97 / Max: 54.63

Embree

Binary: Pathtracer ISPC - Model: Crown

OpenBenchmarking.orgFrames Per Second, More Is BetterEmbree 4.3Binary: Pathtracer ISPC - Model: CrownAVX-512 DisabledAVX-512 Enabled306090120150SE +/- 0.04, N = 7SE +/- 0.09, N = 7126.63136.42MIN: 123.57 / MAX: 130.86MIN: 132.5 / MAX: 141.56

Embree

CPU Peak Freq (Highest CPU Core Frequency) Monitor

OpenBenchmarking.orgMegahertz, More Is BetterEmbree 4.3CPU Peak Freq (Highest CPU Core Frequency) MonitorAVX-512 DisabledAVX-512 Enabled8001600240032004000Min: 2600 / Avg: 3055.85 / Max: 4561Min: 2600 / Avg: 3105.48 / Max: 4563

Embree

CPU Power Consumption Monitor

OpenBenchmarking.orgWatts, Fewer Is BetterEmbree 4.3CPU Power Consumption MonitorAVX-512 DisabledAVX-512 Enabled70140210280350Min: 1.25 / Avg: 226.37 / Max: 364.61Min: 0.33 / Avg: 219.77 / Max: 362.2

Embree

CPU Temperature Monitor

OpenBenchmarking.orgCelsius, Fewer Is BetterEmbree 4.3CPU Temperature MonitorAVX-512 DisabledAVX-512 Enabled1020304050Min: 36.25 / Avg: 45.23 / Max: 48.75Min: 36.13 / Avg: 45.47 / Max: 49.38

GROMACS

Implementation: MPI CPU - Input: water_GMX50_bare

OpenBenchmarking.orgNs Per Day, More Is BetterGROMACS 2024Implementation: MPI CPU - Input: water_GMX50_bareAVX-512 DisabledAVX-512 Enabled48121620SE +/- 0.04, N = 3SE +/- 0.02, N = 314.1317.42-mno-avx512f1. (CXX) g++ options: -O3 -march=znver5 -flto -lm

GROMACS

CPU Peak Freq (Highest CPU Core Frequency) Monitor

OpenBenchmarking.orgMegahertz, More Is BetterGROMACS 2024CPU Peak Freq (Highest CPU Core Frequency) MonitorAVX-512 DisabledAVX-512 Enabled8001600240032004000Min: 2600 / Avg: 3308.87 / Max: 4572Min: 2600 / Avg: 3374.13 / Max: 4568

GROMACS

CPU Power Consumption Monitor

OpenBenchmarking.orgWatts, Fewer Is BetterGROMACS 2024CPU Power Consumption MonitorAVX-512 DisabledAVX-512 Enabled60120180240300Min: 36.56 / Avg: 275.27 / Max: 358.87Min: 36.33 / Avg: 271.46 / Max: 353.96

GROMACS

CPU Temperature Monitor

OpenBenchmarking.orgCelsius, Fewer Is BetterGROMACS 2024CPU Temperature MonitorAVX-512 DisabledAVX-512 Enabled1122334455Min: 25.88 / Avg: 43.94 / Max: 49.75Min: 35.5 / Avg: 48.78 / Max: 54.38

Laghos

Test: Sedov Blast Wave, ube_922_hex.mesh

OpenBenchmarking.orgMajor Kernels Total Rate, More Is BetterLaghos 3.1Test: Sedov Blast Wave, ube_922_hex.meshAVX-512 DisabledAVX-512 Enabled130260390520650SE +/- 3.13, N = 3SE +/- 2.72, N = 3590.60603.92-mno-avx512f1. (CXX) g++ options: -O3 -march=znver5 -flto -lmfem -lHYPRE -lmetis -lrt -lmpi_cxx -lmpi

Laghos

CPU Peak Freq (Highest CPU Core Frequency) Monitor

OpenBenchmarking.orgMegahertz, More Is BetterLaghos 3.1CPU Peak Freq (Highest CPU Core Frequency) MonitorAVX-512 DisabledAVX-512 Enabled8001600240032004000Min: 2600 / Avg: 3421.94 / Max: 4586Min: 2600 / Avg: 3459.12 / Max: 4564

Laghos

CPU Power Consumption Monitor

OpenBenchmarking.orgWatts, Fewer Is BetterLaghos 3.1CPU Power Consumption MonitorAVX-512 DisabledAVX-512 Enabled60120180240300Min: 9.49 / Avg: 309.54 / Max: 345.92Min: 1.13 / Avg: 321.97 / Max: 352.64

Laghos

CPU Temperature Monitor

OpenBenchmarking.orgCelsius, Fewer Is BetterLaghos 3.1CPU Temperature MonitorAVX-512 DisabledAVX-512 Enabled1224364860Min: 31.63 / Avg: 47.78 / Max: 51.38Min: 46.88 / Avg: 59.01 / Max: 60.5

libxsmm

M N K: 128

OpenBenchmarking.orgGFLOPS/s, More Is Betterlibxsmm 2-1.17-3645M N K: 128AVX-512 DisabledAVX-512 Enabled7001400210028003500SE +/- 10.56, N = 3SE +/- 5.19, N = 33004.63422.01. (CXX) g++ options: -dynamic -Bstatic -static-libgcc -lgomp -lm -lrt -ldl -lquadmath -lstdc++ -pthread -fPIC -std=c++14 -O2 -fopenmp-simd -funroll-loops -ftree-vectorize -fdata-sections -ffunction-sections -fvisibility=hidden -msse4.2

libxsmm

CPU Peak Freq (Highest CPU Core Frequency) Monitor

OpenBenchmarking.orgMegahertz, More Is Betterlibxsmm 2-1.17-3645CPU Peak Freq (Highest CPU Core Frequency) MonitorAVX-512 DisabledAVX-512 Enabled8001600240032004000Min: 2600 / Avg: 4357.28 / Max: 4572Min: 2600 / Avg: 4316.37 / Max: 4568

libxsmm

CPU Power Consumption Monitor

OpenBenchmarking.orgWatts, Fewer Is Betterlibxsmm 2-1.17-3645CPU Power Consumption MonitorAVX-512 DisabledAVX-512 Enabled70140210280350Min: 0.98 / Avg: 171.17 / Max: 366.29Min: 0.42 / Avg: 176.21 / Max: 341.37

libxsmm

CPU Temperature Monitor

OpenBenchmarking.orgCelsius, Fewer Is Betterlibxsmm 2-1.17-3645CPU Temperature MonitorAVX-512 DisabledAVX-512 Enabled1122334455Min: 38.5 / Avg: 47.97 / Max: 50.75Min: 40 / Avg: 51.39 / Max: 53.5

Llama.cpp

Backend: CPU BLAS - Model: Llama-3.1-Tulu-3-8B-Q8_0 - Test: Prompt Processing 512

OpenBenchmarking.orgTokens Per Second, More Is BetterLlama.cpp b4397Backend: CPU BLAS - Model: Llama-3.1-Tulu-3-8B-Q8_0 - Test: Prompt Processing 512AVX-512 DisabledAVX-512 Enabled20406080100SE +/- 0.94, N = 3SE +/- 0.76, N = 395.03107.84-mno-avx512f1. (CXX) g++ options: -O3 -march=znver5 -flto

Llama.cpp

CPU Peak Freq (Highest CPU Core Frequency) Monitor

OpenBenchmarking.orgMegahertz, More Is BetterLlama.cpp b4397CPU Peak Freq (Highest CPU Core Frequency) MonitorAVX-512 DisabledAVX-512 Enabled8001600240032004000Min: 2600 / Avg: 3577.05 / Max: 4540Min: 2600 / Avg: 3527.61 / Max: 4565

Llama.cpp

CPU Power Consumption Monitor

OpenBenchmarking.orgWatts, Fewer Is BetterLlama.cpp b4397CPU Power Consumption MonitorAVX-512 DisabledAVX-512 Enabled50100150200250Min: 2.83 / Avg: 238.62 / Max: 295Min: 0.58 / Avg: 223.48 / Max: 272.26

Llama.cpp

CPU Temperature Monitor

OpenBenchmarking.orgCelsius, Fewer Is BetterLlama.cpp b4397CPU Temperature MonitorAVX-512 DisabledAVX-512 Enabled1020304050Min: 36.13 / Avg: 45.03 / Max: 47Min: 38.88 / Avg: 46.74 / Max: 48.63

Llama.cpp

Backend: CPU BLAS - Model: Llama-3.1-Tulu-3-8B-Q8_0 - Test: Prompt Processing 1024

OpenBenchmarking.orgTokens Per Second, More Is BetterLlama.cpp b4397Backend: CPU BLAS - Model: Llama-3.1-Tulu-3-8B-Q8_0 - Test: Prompt Processing 1024AVX-512 DisabledAVX-512 Enabled20406080100SE +/- 0.60, N = 3SE +/- 0.46, N = 393.45106.71-mno-avx512f1. (CXX) g++ options: -O3 -march=znver5 -flto

Llama.cpp

CPU Peak Freq (Highest CPU Core Frequency) Monitor

OpenBenchmarking.orgMegahertz, More Is BetterLlama.cpp b4397CPU Peak Freq (Highest CPU Core Frequency) MonitorAVX-512 DisabledAVX-512 Enabled8001600240032004000Min: 2600 / Avg: 3623.36 / Max: 4547Min: 2600 / Avg: 3527.95 / Max: 4580

Llama.cpp

CPU Power Consumption Monitor

OpenBenchmarking.orgWatts, Fewer Is BetterLlama.cpp b4397CPU Power Consumption MonitorAVX-512 DisabledAVX-512 Enabled50100150200250Min: 8.48 / Avg: 252.08 / Max: 292.53Min: 1.13 / Avg: 236.52 / Max: 282.16

Llama.cpp

CPU Temperature Monitor

OpenBenchmarking.orgCelsius, Fewer Is BetterLlama.cpp b4397CPU Temperature MonitorAVX-512 DisabledAVX-512 Enabled1020304050Min: 36.25 / Avg: 46.07 / Max: 47.63Min: 38.38 / Avg: 47.57 / Max: 48.88

Llama.cpp

Backend: CPU BLAS - Model: Llama-3.1-Tulu-3-8B-Q8_0 - Test: Prompt Processing 2048

OpenBenchmarking.orgTokens Per Second, More Is BetterLlama.cpp b4397Backend: CPU BLAS - Model: Llama-3.1-Tulu-3-8B-Q8_0 - Test: Prompt Processing 2048AVX-512 DisabledAVX-512 Enabled20406080100SE +/- 0.54, N = 3SE +/- 0.56, N = 395.47107.34-mno-avx512f1. (CXX) g++ options: -O3 -march=znver5 -flto

Llama.cpp

CPU Peak Freq (Highest CPU Core Frequency) Monitor

OpenBenchmarking.orgMegahertz, More Is BetterLlama.cpp b4397CPU Peak Freq (Highest CPU Core Frequency) MonitorAVX-512 DisabledAVX-512 Enabled8001600240032004000Min: 2600 / Avg: 3613.53 / Max: 4521Min: 2600 / Avg: 3527.95 / Max: 4566

Llama.cpp

CPU Power Consumption Monitor

OpenBenchmarking.orgWatts, Fewer Is BetterLlama.cpp b4397CPU Power Consumption MonitorAVX-512 DisabledAVX-512 Enabled50100150200250Min: 5.84 / Avg: 258.55 / Max: 290.38Min: 0.65 / Avg: 244.86 / Max: 269.27

Llama.cpp

CPU Temperature Monitor

OpenBenchmarking.orgCelsius, Fewer Is BetterLlama.cpp b4397CPU Temperature MonitorAVX-512 DisabledAVX-512 Enabled1020304050Min: 36.63 / Avg: 46.96 / Max: 48.75Min: 38.75 / Avg: 48.39 / Max: 49.63

Llama.cpp

Backend: CPU BLAS - Model: granite-3.0-3b-a800m-instruct-Q8_0 - Test: Prompt Processing 512

OpenBenchmarking.orgTokens Per Second, More Is BetterLlama.cpp b4397Backend: CPU BLAS - Model: granite-3.0-3b-a800m-instruct-Q8_0 - Test: Prompt Processing 512AVX-512 DisabledAVX-512 Enabled80160240320400SE +/- 2.08, N = 5SE +/- 2.40, N = 5347.10356.76-mno-avx512f1. (CXX) g++ options: -O3 -march=znver5 -flto

Llama.cpp

CPU Peak Freq (Highest CPU Core Frequency) Monitor

OpenBenchmarking.orgMegahertz, More Is BetterLlama.cpp b4397CPU Peak Freq (Highest CPU Core Frequency) MonitorAVX-512 DisabledAVX-512 Enabled8001600240032004000Min: 2600 / Avg: 3454.73 / Max: 4569Min: 2600 / Avg: 3430.29 / Max: 4544

Llama.cpp

CPU Power Consumption Monitor

OpenBenchmarking.orgWatts, Fewer Is BetterLlama.cpp b4397CPU Power Consumption MonitorAVX-512 DisabledAVX-512 Enabled50100150200250Min: 3 / Avg: 178.29 / Max: 256.87Min: 36.59 / Avg: 178.87 / Max: 253.24

Llama.cpp

CPU Temperature Monitor

OpenBenchmarking.orgCelsius, Fewer Is BetterLlama.cpp b4397CPU Temperature MonitorAVX-512 DisabledAVX-512 Enabled918273645Min: 34.75 / Avg: 40.31 / Max: 42Min: 37 / Avg: 42.78 / Max: 44.63

Llama.cpp

Backend: CPU BLAS - Model: Mistral-7B-Instruct-v0.3-Q8_0 - Test: Prompt Processing 512

OpenBenchmarking.orgTokens Per Second, More Is BetterLlama.cpp b4397Backend: CPU BLAS - Model: Mistral-7B-Instruct-v0.3-Q8_0 - Test: Prompt Processing 512AVX-512 DisabledAVX-512 Enabled20406080100SE +/- 1.01, N = 4SE +/- 1.17, N = 596.15109.25-mno-avx512f1. (CXX) g++ options: -O3 -march=znver5 -flto

Llama.cpp

CPU Peak Freq (Highest CPU Core Frequency) Monitor

OpenBenchmarking.orgMegahertz, More Is BetterLlama.cpp b4397CPU Peak Freq (Highest CPU Core Frequency) MonitorAVX-512 DisabledAVX-512 Enabled8001600240032004000Min: 2600 / Avg: 3594.14 / Max: 4556Min: 2600 / Avg: 3512.12 / Max: 4558

Llama.cpp

CPU Power Consumption Monitor

OpenBenchmarking.orgWatts, Fewer Is BetterLlama.cpp b4397CPU Power Consumption MonitorAVX-512 DisabledAVX-512 Enabled50100150200250Min: 4.3 / Avg: 242.37 / Max: 295Min: 0.17 / Avg: 226.72 / Max: 292.5

Llama.cpp

CPU Temperature Monitor

OpenBenchmarking.orgCelsius, Fewer Is BetterLlama.cpp b4397CPU Temperature MonitorAVX-512 DisabledAVX-512 Enabled1020304050Min: 36.5 / Avg: 45.61 / Max: 47.25Min: 38.63 / Avg: 46.34 / Max: 48.13

Llama.cpp

Backend: CPU BLAS - Model: Mistral-7B-Instruct-v0.3-Q8_0 - Test: Prompt Processing 1024

OpenBenchmarking.orgTokens Per Second, More Is BetterLlama.cpp b4397Backend: CPU BLAS - Model: Mistral-7B-Instruct-v0.3-Q8_0 - Test: Prompt Processing 1024AVX-512 DisabledAVX-512 Enabled20406080100SE +/- 0.77, N = 3SE +/- 1.15, N = 394.90107.32-mno-avx512f1. (CXX) g++ options: -O3 -march=znver5 -flto

Llama.cpp

CPU Peak Freq (Highest CPU Core Frequency) Monitor

OpenBenchmarking.orgMegahertz, More Is BetterLlama.cpp b4397CPU Peak Freq (Highest CPU Core Frequency) MonitorAVX-512 DisabledAVX-512 Enabled8001600240032004000Min: 2600 / Avg: 3628.57 / Max: 4544Min: 2600 / Avg: 3535.35 / Max: 4545

Llama.cpp

CPU Power Consumption Monitor

OpenBenchmarking.orgWatts, Fewer Is BetterLlama.cpp b4397CPU Power Consumption MonitorAVX-512 DisabledAVX-512 Enabled50100150200250Min: 36.63 / Avg: 255.53 / Max: 287.29Min: 1.04 / Avg: 238.56 / Max: 271.81

Llama.cpp

CPU Temperature Monitor

OpenBenchmarking.orgCelsius, Fewer Is BetterLlama.cpp b4397CPU Temperature MonitorAVX-512 DisabledAVX-512 Enabled1020304050Min: 36.75 / Avg: 46.77 / Max: 48.88Min: 38.38 / Avg: 47.27 / Max: 49.13

Llama.cpp

Backend: CPU BLAS - Model: Mistral-7B-Instruct-v0.3-Q8_0 - Test: Prompt Processing 2048

OpenBenchmarking.orgTokens Per Second, More Is BetterLlama.cpp b4397Backend: CPU BLAS - Model: Mistral-7B-Instruct-v0.3-Q8_0 - Test: Prompt Processing 2048AVX-512 DisabledAVX-512 Enabled20406080100SE +/- 0.65, N = 3SE +/- 0.94, N = 395.27108.74-mno-avx512f1. (CXX) g++ options: -O3 -march=znver5 -flto

Llama.cpp

CPU Peak Freq (Highest CPU Core Frequency) Monitor

OpenBenchmarking.orgMegahertz, More Is BetterLlama.cpp b4397CPU Peak Freq (Highest CPU Core Frequency) MonitorAVX-512 DisabledAVX-512 Enabled8001600240032004000Min: 2600 / Avg: 3605.26 / Max: 4568Min: 2600 / Avg: 3524.34 / Max: 4584

Llama.cpp

CPU Power Consumption Monitor

OpenBenchmarking.orgWatts, Fewer Is BetterLlama.cpp b4397CPU Power Consumption MonitorAVX-512 DisabledAVX-512 Enabled50100150200250Min: 7.92 / Avg: 259.65 / Max: 287.46Min: 0.57 / Avg: 245.43 / Max: 278.75

Llama.cpp

CPU Temperature Monitor

OpenBenchmarking.orgCelsius, Fewer Is BetterLlama.cpp b4397CPU Temperature MonitorAVX-512 DisabledAVX-512 Enabled1020304050Min: 37.38 / Avg: 47.47 / Max: 49.13Min: 38.5 / Avg: 48.02 / Max: 49.63

miniBUDE

Implementation: OpenMP - Input Deck: BM1

OpenBenchmarking.orgGFInst/s, More Is BetterminiBUDE 20210901Implementation: OpenMP - Input Deck: BM1AVX-512 DisabledAVX-512 Enabled14002800420056007000SE +/- 55.43, N = 15SE +/- 149.28, N = 153416.436368.741. (CC) gcc options: -std=c99 -Ofast -ffast-math -fopenmp -march=native -lm

miniBUDE

Implementation: OpenMP - Input Deck: BM1

OpenBenchmarking.orgBillion Interactions/s, More Is BetterminiBUDE 20210901Implementation: OpenMP - Input Deck: BM1AVX-512 DisabledAVX-512 Enabled60120180240300SE +/- 2.22, N = 15SE +/- 5.97, N = 15136.66254.751. (CC) gcc options: -std=c99 -Ofast -ffast-math -fopenmp -march=native -lm

miniBUDE

CPU Peak Freq (Highest CPU Core Frequency) Monitor

OpenBenchmarking.orgMegahertz, More Is BetterminiBUDE 20210901CPU Peak Freq (Highest CPU Core Frequency) MonitorAVX-512 DisabledAVX-512 Enabled8001600240032004000Min: 2600 / Avg: 3205.72 / Max: 4570Min: 2600 / Avg: 2984.55 / Max: 4564

miniBUDE

CPU Power Consumption Monitor

OpenBenchmarking.orgWatts, Fewer Is BetterminiBUDE 20210901CPU Power Consumption MonitorAVX-512 DisabledAVX-512 Enabled60120180240300Min: 36.55 / Avg: 215.04 / Max: 347.21Min: 36.47 / Avg: 167.14 / Max: 343.87

miniBUDE

CPU Temperature Monitor

OpenBenchmarking.orgCelsius, Fewer Is BetterminiBUDE 20210901CPU Temperature MonitorAVX-512 DisabledAVX-512 Enabled1020304050Min: 28 / Avg: 43.7 / Max: 48.25Min: 26.38 / Avg: 41.78 / Max: 46.38

miniBUDE

Implementation: OpenMP - Input Deck: BM2

OpenBenchmarking.orgGFInst/s, More Is BetterminiBUDE 20210901Implementation: OpenMP - Input Deck: BM2AVX-512 DisabledAVX-512 Enabled15003000450060007500SE +/- 2.24, N = 3SE +/- 54.17, N = 33666.097162.271. (CC) gcc options: -std=c99 -Ofast -ffast-math -fopenmp -march=native -lm

miniBUDE

Implementation: OpenMP - Input Deck: BM2

OpenBenchmarking.orgBillion Interactions/s, More Is BetterminiBUDE 20210901Implementation: OpenMP - Input Deck: BM2AVX-512 DisabledAVX-512 Enabled60120180240300SE +/- 0.09, N = 3SE +/- 2.17, N = 3146.64286.491. (CC) gcc options: -std=c99 -Ofast -ffast-math -fopenmp -march=native -lm

miniBUDE

CPU Peak Freq (Highest CPU Core Frequency) Monitor

OpenBenchmarking.orgMegahertz, More Is BetterminiBUDE 20210901CPU Peak Freq (Highest CPU Core Frequency) MonitorAVX-512 DisabledAVX-512 Enabled8001600240032004000Min: 2600 / Avg: 3405.75 / Max: 4550Min: 2600 / Avg: 3170.12 / Max: 4554

miniBUDE

CPU Power Consumption Monitor

OpenBenchmarking.orgWatts, Fewer Is BetterminiBUDE 20210901CPU Power Consumption MonitorAVX-512 DisabledAVX-512 Enabled60120180240300Min: 36.57 / Avg: 308.5 / Max: 348.32Min: 36.07 / Avg: 283.87 / Max: 350.74

miniBUDE

CPU Temperature Monitor

OpenBenchmarking.orgCelsius, Fewer Is BetterminiBUDE 20210901CPU Temperature MonitorAVX-512 DisabledAVX-512 Enabled1122334455Min: 33.38 / Avg: 52.1 / Max: 56.88Min: 33 / Avg: 51.41 / Max: 57.75

Mobile Neural Network

Model: mobilenetV3

OpenBenchmarking.orgms, Fewer Is BetterMobile Neural Network 3.0Model: mobilenetV3AVX-512 DisabledAVX-512 Enabled0.40570.81141.21711.62282.0285SE +/- 0.010, N = 3SE +/- 0.010, N = 31.8031.736MIN: 1.68 / MAX: 2.88-march=znver5 - MIN: 1.59 / MAX: 2.211. (CXX) g++ options: -O3 -flto -std=c++11 -fvisibility=hidden -fomit-frame-pointer -fstrict-aliasing -ffunction-sections -fdata-sections -ffast-math -fno-rtti -fno-exceptions -pthread -ldl

Mobile Neural Network

Model: resnet-v2-50

OpenBenchmarking.orgms, Fewer Is BetterMobile Neural Network 3.0Model: resnet-v2-50AVX-512 DisabledAVX-512 Enabled3691215SE +/- 0.086, N = 3SE +/- 0.117, N = 39.9507.409MIN: 9.4 / MAX: 12.2-march=znver5 - MIN: 7.08 / MAX: 9.541. (CXX) g++ options: -O3 -flto -std=c++11 -fvisibility=hidden -fomit-frame-pointer -fstrict-aliasing -ffunction-sections -fdata-sections -ffast-math -fno-rtti -fno-exceptions -pthread -ldl

Mobile Neural Network

Model: SqueezeNetV1.0

OpenBenchmarking.orgms, Fewer Is BetterMobile Neural Network 3.0Model: SqueezeNetV1.0AVX-512 DisabledAVX-512 Enabled0.74861.49722.24582.99443.743SE +/- 0.030, N = 3SE +/- 0.011, N = 33.3273.121MIN: 3.21 / MAX: 5.6-march=znver5 - MIN: 3.05 / MAX: 5.81. (CXX) g++ options: -O3 -flto -std=c++11 -fvisibility=hidden -fomit-frame-pointer -fstrict-aliasing -ffunction-sections -fdata-sections -ffast-math -fno-rtti -fno-exceptions -pthread -ldl

Numpy Benchmark

OpenBenchmarking.orgScore, More Is BetterNumpy BenchmarkAVX-512 DisabledAVX-512 Enabled2004006008001000SE +/- 0.83, N = 3SE +/- 1.81, N = 3828.67882.00

Numpy Benchmark

CPU Peak Freq (Highest CPU Core Frequency) Monitor

OpenBenchmarking.orgMegahertz, More Is BetterNumpy BenchmarkCPU Peak Freq (Highest CPU Core Frequency) MonitorAVX-512 DisabledAVX-512 Enabled8001600240032004000Min: 2600 / Avg: 4473.89 / Max: 4563Min: 2600 / Avg: 4481.19 / Max: 4567

Numpy Benchmark

CPU Power Consumption Monitor

OpenBenchmarking.orgWatts, Fewer Is BetterNumpy BenchmarkCPU Power Consumption MonitorAVX-512 DisabledAVX-512 Enabled20406080100Min: 1.18 / Avg: 80.05 / Max: 87.29Min: 0.54 / Avg: 79.85 / Max: 87.65

Numpy Benchmark

CPU Temperature Monitor

OpenBenchmarking.orgCelsius, Fewer Is BetterNumpy BenchmarkCPU Temperature MonitorAVX-512 DisabledAVX-512 Enabled1122334455Min: 40.75 / Avg: 52.43 / Max: 57.5Min: 39.75 / Avg: 52.09 / Max: 57.5

oneDNN

Harness: Deconvolution Batch shapes_1d - Engine: CPU

OpenBenchmarking.orgms, Fewer Is BetteroneDNN 3.6Harness: Deconvolution Batch shapes_1d - Engine: CPUAVX-512 DisabledAVX-512 Enabled3691215SE +/- 0.00410, N = 3SE +/- 0.01312, N = 312.860106.72466-mno-avx512f - MIN: 9.8MIN: 4.21. (CXX) g++ options: -O3 -march=native -march=znver5 -flto -fopenmp -msse4.1 -fPIC -fcf-protection=full -pie -ldl

oneDNN

CPU Peak Freq (Highest CPU Core Frequency) Monitor

OpenBenchmarking.orgMegahertz, More Is BetteroneDNN 3.6CPU Peak Freq (Highest CPU Core Frequency) MonitorAVX-512 DisabledAVX-512 Enabled8001600240032004000Min: 2600 / Avg: 3408.08 / Max: 4620Min: 2583 / Avg: 3215.51 / Max: 4567

oneDNN

CPU Power Consumption Monitor

OpenBenchmarking.orgWatts, Fewer Is BetteroneDNN 3.6CPU Power Consumption MonitorAVX-512 DisabledAVX-512 Enabled50100150200250Min: 7.6 / Avg: 200.15 / Max: 283.44Min: 1.53 / Avg: 183.78 / Max: 258.76

oneDNN

CPU Temperature Monitor

OpenBenchmarking.orgCelsius, Fewer Is BetteroneDNN 3.6CPU Temperature MonitorAVX-512 DisabledAVX-512 Enabled918273645Min: 40.13 / Avg: 43.41 / Max: 46Min: 33.38 / Avg: 40.12 / Max: 42.88

oneDNN

Harness: Deconvolution Batch shapes_3d - Engine: CPU

OpenBenchmarking.orgms, Fewer Is BetteroneDNN 3.6Harness: Deconvolution Batch shapes_3d - Engine: CPUAVX-512 DisabledAVX-512 Enabled0.30470.60940.91411.21881.5235SE +/- 0.001116, N = 9SE +/- 0.001216, N = 91.3543900.716979-mno-avx512f - MIN: 1.31MIN: 0.621. (CXX) g++ options: -O3 -march=native -march=znver5 -flto -fopenmp -msse4.1 -fPIC -fcf-protection=full -pie -ldl

oneDNN

CPU Peak Freq (Highest CPU Core Frequency) Monitor

OpenBenchmarking.orgMegahertz, More Is BetteroneDNN 3.6CPU Peak Freq (Highest CPU Core Frequency) MonitorAVX-512 DisabledAVX-512 Enabled8001600240032004000Min: 2600 / Avg: 3065.11 / Max: 4539Min: 2448 / Avg: 2613.84 / Max: 4541

oneDNN

CPU Power Consumption Monitor

OpenBenchmarking.orgWatts, Fewer Is BetteroneDNN 3.6CPU Power Consumption MonitorAVX-512 DisabledAVX-512 Enabled50100150200250Min: 11.96 / Avg: 136.6 / Max: 273.71Min: 0.2 / Avg: 112.35 / Max: 220.99

oneDNN

CPU Temperature Monitor

OpenBenchmarking.orgCelsius, Fewer Is BetteroneDNN 3.6CPU Temperature MonitorAVX-512 DisabledAVX-512 Enabled1020304050Min: 35.13 / Avg: 44.03 / Max: 47.38Min: 34.13 / Avg: 38.62 / Max: 40.38

oneDNN

Harness: IP Shapes 1D - Engine: CPU

OpenBenchmarking.orgms, Fewer Is BetteroneDNN 3.6Harness: IP Shapes 1D - Engine: CPUAVX-512 DisabledAVX-512 Enabled0.14540.29080.43620.58160.727SE +/- 0.000257, N = 4SE +/- 0.001169, N = 40.6462110.533414-mno-avx512f - MIN: 0.59MIN: 0.491. (CXX) g++ options: -O3 -march=native -march=znver5 -flto -fopenmp -msse4.1 -fPIC -fcf-protection=full -pie -ldl

oneDNN

CPU Peak Freq (Highest CPU Core Frequency) Monitor

OpenBenchmarking.orgMegahertz, More Is BetteroneDNN 3.6CPU Peak Freq (Highest CPU Core Frequency) MonitorAVX-512 DisabledAVX-512 Enabled8001600240032004000Min: 2557 / Avg: 3100.03 / Max: 4539Min: 2374 / Avg: 3087.78 / Max: 4541

oneDNN

CPU Power Consumption Monitor

OpenBenchmarking.orgWatts, Fewer Is BetteroneDNN 3.6CPU Power Consumption MonitorAVX-512 DisabledAVX-512 Enabled50100150200250Min: 14.09 / Avg: 163.35 / Max: 251.52Min: 0.41 / Avg: 152.19 / Max: 222.77

oneDNN

CPU Temperature Monitor

OpenBenchmarking.orgCelsius, Fewer Is BetteroneDNN 3.6CPU Temperature MonitorAVX-512 DisabledAVX-512 Enabled918273645Min: 34 / Avg: 39.16 / Max: 42.25Min: 32.5 / Avg: 37.99 / Max: 40.25

oneDNN

Harness: IP Shapes 3D - Engine: CPU

OpenBenchmarking.orgms, Fewer Is BetteroneDNN 3.6Harness: IP Shapes 3D - Engine: CPUAVX-512 DisabledAVX-512 Enabled0.6921.3842.0762.7683.46SE +/- 0.002544, N = 5SE +/- 0.000384, N = 53.0755400.265927-mno-avx512f - MIN: 3.021. (CXX) g++ options: -O3 -march=native -march=znver5 -flto -fopenmp -msse4.1 -fPIC -fcf-protection=full -pie -ldl

oneDNN

CPU Peak Freq (Highest CPU Core Frequency) Monitor

OpenBenchmarking.orgMegahertz, More Is BetteroneDNN 3.6CPU Peak Freq (Highest CPU Core Frequency) MonitorAVX-512 DisabledAVX-512 Enabled8001600240032004000Min: 2600 / Avg: 3798.58 / Max: 4549Min: 2539 / Avg: 3522.54 / Max: 4553

oneDNN

CPU Power Consumption Monitor

OpenBenchmarking.orgWatts, Fewer Is BetteroneDNN 3.6CPU Power Consumption MonitorAVX-512 DisabledAVX-512 Enabled50100150200250Min: 0.69 / Avg: 138.48 / Max: 275.92Min: 0.74 / Avg: 168.39 / Max: 247.9

oneDNN

CPU Temperature Monitor

OpenBenchmarking.orgCelsius, Fewer Is BetteroneDNN 3.6CPU Temperature MonitorAVX-512 DisabledAVX-512 Enabled1020304050Min: 32.38 / Avg: 43.48 / Max: 49.88Min: 32.88 / Avg: 40.62 / Max: 44.5

oneDNN

Harness: Recurrent Neural Network Training - Engine: CPU

OpenBenchmarking.orgms, Fewer Is BetteroneDNN 3.6Harness: Recurrent Neural Network Training - Engine: CPUAVX-512 DisabledAVX-512 Enabled110220330440550SE +/- 0.56, N = 3SE +/- 0.35, N = 3491.69428.50-mno-avx512f - MIN: 486.04MIN: 421.371. (CXX) g++ options: -O3 -march=native -march=znver5 -flto -fopenmp -msse4.1 -fPIC -fcf-protection=full -pie -ldl

oneDNN

CPU Peak Freq (Highest CPU Core Frequency) Monitor

OpenBenchmarking.orgMegahertz, More Is BetteroneDNN 3.6CPU Peak Freq (Highest CPU Core Frequency) MonitorAVX-512 DisabledAVX-512 Enabled8001600240032004000Min: 2299 / Avg: 3036.42 / Max: 4549Min: 1783 / Avg: 3106.58 / Max: 4569

oneDNN

CPU Power Consumption Monitor

OpenBenchmarking.orgWatts, Fewer Is BetteroneDNN 3.6CPU Power Consumption MonitorAVX-512 DisabledAVX-512 Enabled60120180240300Min: 3.06 / Avg: 245.86 / Max: 314.05Min: 1.05 / Avg: 209.55 / Max: 281.38

oneDNN

CPU Temperature Monitor

OpenBenchmarking.orgCelsius, Fewer Is BetteroneDNN 3.6CPU Temperature MonitorAVX-512 DisabledAVX-512 Enabled1020304050Min: 33.38 / Avg: 43.76 / Max: 46.88Min: 34.25 / Avg: 45.28 / Max: 47.5

oneDNN

Harness: Recurrent Neural Network Inference - Engine: CPU

OpenBenchmarking.orgms, Fewer Is BetteroneDNN 3.6Harness: Recurrent Neural Network Inference - Engine: CPUAVX-512 DisabledAVX-512 Enabled70140210280350SE +/- 0.47, N = 3SE +/- 0.34, N = 3307.05278.31-mno-avx512f - MIN: 301.26MIN: 271.121. (CXX) g++ options: -O3 -march=native -march=znver5 -flto -fopenmp -msse4.1 -fPIC -fcf-protection=full -pie -ldl

oneDNN

CPU Peak Freq (Highest CPU Core Frequency) Monitor

OpenBenchmarking.orgMegahertz, More Is BetteroneDNN 3.6CPU Peak Freq (Highest CPU Core Frequency) MonitorAVX-512 DisabledAVX-512 Enabled8001600240032004000Min: 1753 / Avg: 3103.6 / Max: 4545Min: 1575 / Avg: 3008.31 / Max: 4572

oneDNN

CPU Power Consumption Monitor

OpenBenchmarking.orgWatts, Fewer Is BetteroneDNN 3.6CPU Power Consumption MonitorAVX-512 DisabledAVX-512 Enabled60120180240300Min: 1.24 / Avg: 223.45 / Max: 310.66Min: 0.63 / Avg: 202.71 / Max: 294.11

oneDNN

CPU Temperature Monitor

OpenBenchmarking.orgCelsius, Fewer Is BetteroneDNN 3.6CPU Temperature MonitorAVX-512 DisabledAVX-512 Enabled918273645Min: 36.5 / Avg: 43.53 / Max: 46.38Min: 36.5 / Avg: 44.46 / Max: 46.38

ONNX Runtime

Model: fcn-resnet101-11 - Device: CPU - Executor: Standard

OpenBenchmarking.orgInferences Per Second, More Is BetterONNX Runtime 1.19Model: fcn-resnet101-11 - Device: CPU - Executor: StandardAVX-512 DisabledAVX-512 Enabled3691215SE +/- 0.04636, N = 3SE +/- 0.13137, N = 156.586039.564331. (CXX) g++ options: -O3 -march=native -march=znver5 -flto -ffunction-sections -fdata-sections -mtune=native -flto=auto -fno-fat-lto-objects -ldl -lrt

ONNX Runtime

Model: fcn-resnet101-11 - Device: CPU - Executor: Standard

OpenBenchmarking.orgInference Time Cost (ms), Fewer Is BetterONNX Runtime 1.19Model: fcn-resnet101-11 - Device: CPU - Executor: StandardAVX-512 DisabledAVX-512 Enabled306090120150SE +/- 1.07, N = 3SE +/- 1.43, N = 15151.85104.831. (CXX) g++ options: -O3 -march=native -march=znver5 -flto -ffunction-sections -fdata-sections -mtune=native -flto=auto -fno-fat-lto-objects -ldl -lrt

ONNX Runtime

CPU Peak Freq (Highest CPU Core Frequency) Monitor

OpenBenchmarking.orgMegahertz, More Is BetterONNX Runtime 1.19CPU Peak Freq (Highest CPU Core Frequency) MonitorAVX-512 DisabledAVX-512 Enabled8001600240032004000Min: 2600 / Avg: 3268.36 / Max: 4548Min: 2600 / Avg: 2988 / Max: 4545

ONNX Runtime

CPU Power Consumption Monitor

OpenBenchmarking.orgWatts, Fewer Is BetterONNX Runtime 1.19CPU Power Consumption MonitorAVX-512 DisabledAVX-512 Enabled50100150200250Min: 1.64 / Avg: 246.05 / Max: 287.32Min: 0.88 / Avg: 231.28 / Max: 253.21

ONNX Runtime

CPU Temperature Monitor

OpenBenchmarking.orgCelsius, Fewer Is BetterONNX Runtime 1.19CPU Temperature MonitorAVX-512 DisabledAVX-512 Enabled1020304050Min: 37.63 / Avg: 46.6 / Max: 49.13Min: 41.38 / Avg: 46.81 / Max: 48.38

ONNX Runtime

Model: super-resolution-10 - Device: CPU - Executor: Standard

OpenBenchmarking.orgInferences Per Second, More Is BetterONNX Runtime 1.19Model: super-resolution-10 - Device: CPU - Executor: StandardAVX-512 DisabledAVX-512 Enabled50100150200250SE +/- 1.92, N = 3SE +/- 0.59, N = 3170.95215.161. (CXX) g++ options: -O3 -march=native -march=znver5 -flto -ffunction-sections -fdata-sections -mtune=native -flto=auto -fno-fat-lto-objects -ldl -lrt

ONNX Runtime

Model: super-resolution-10 - Device: CPU - Executor: Standard

OpenBenchmarking.orgInference Time Cost (ms), Fewer Is BetterONNX Runtime 1.19Model: super-resolution-10 - Device: CPU - Executor: StandardAVX-512 DisabledAVX-512 Enabled1.31652.6333.94955.2666.5825SE +/- 0.06614, N = 3SE +/- 0.01281, N = 35.850964.647501. (CXX) g++ options: -O3 -march=native -march=znver5 -flto -ffunction-sections -fdata-sections -mtune=native -flto=auto -fno-fat-lto-objects -ldl -lrt

ONNX Runtime

CPU Peak Freq (Highest CPU Core Frequency) Monitor

OpenBenchmarking.orgMegahertz, More Is BetterONNX Runtime 1.19CPU Peak Freq (Highest CPU Core Frequency) MonitorAVX-512 DisabledAVX-512 Enabled8001600240032004000Min: 2600 / Avg: 4425.08 / Max: 4547Min: 2600 / Avg: 4421.13 / Max: 4559

ONNX Runtime

CPU Power Consumption Monitor

OpenBenchmarking.orgWatts, Fewer Is BetterONNX Runtime 1.19CPU Power Consumption MonitorAVX-512 DisabledAVX-512 Enabled4080120160200Min: 1.15 / Avg: 191.45 / Max: 204.65Min: 0.02 / Avg: 190.61 / Max: 202.75

ONNX Runtime

CPU Temperature Monitor

OpenBenchmarking.orgCelsius, Fewer Is BetterONNX Runtime 1.19CPU Temperature MonitorAVX-512 DisabledAVX-512 Enabled1326395265Min: 37.5 / Avg: 59.48 / Max: 65.88Min: 38.5 / Avg: 58.83 / Max: 60.25

ONNX Runtime

Model: bertsquad-12 - Device: CPU - Executor: Standard

OpenBenchmarking.orgInferences Per Second, More Is BetterONNX Runtime 1.19Model: bertsquad-12 - Device: CPU - Executor: StandardAVX-512 DisabledAVX-512 Enabled510152025SE +/- 0.06, N = 3SE +/- 0.19, N = 317.7622.431. (CXX) g++ options: -O3 -march=native -march=znver5 -flto -ffunction-sections -fdata-sections -mtune=native -flto=auto -fno-fat-lto-objects -ldl -lrt

ONNX Runtime

Model: bertsquad-12 - Device: CPU - Executor: Standard

OpenBenchmarking.orgInference Time Cost (ms), Fewer Is BetterONNX Runtime 1.19Model: bertsquad-12 - Device: CPU - Executor: StandardAVX-512 DisabledAVX-512 Enabled1326395265SE +/- 0.18, N = 3SE +/- 0.38, N = 356.3144.581. (CXX) g++ options: -O3 -march=native -march=znver5 -flto -ffunction-sections -fdata-sections -mtune=native -flto=auto -fno-fat-lto-objects -ldl -lrt

ONNX Runtime

CPU Peak Freq (Highest CPU Core Frequency) Monitor

OpenBenchmarking.orgMegahertz, More Is BetterONNX Runtime 1.19CPU Peak Freq (Highest CPU Core Frequency) MonitorAVX-512 DisabledAVX-512 Enabled8001600240032004000Min: 2600 / Avg: 4080.57 / Max: 4569Min: 2600 / Avg: 4001.58 / Max: 4561

ONNX Runtime

CPU Power Consumption Monitor

OpenBenchmarking.orgWatts, Fewer Is BetterONNX Runtime 1.19CPU Power Consumption MonitorAVX-512 DisabledAVX-512 Enabled50100150200250Min: 0.36 / Avg: 256.73 / Max: 276.98Min: 0.84 / Avg: 240.74 / Max: 259.52

ONNX Runtime

CPU Temperature Monitor

OpenBenchmarking.orgCelsius, Fewer Is BetterONNX Runtime 1.19CPU Temperature MonitorAVX-512 DisabledAVX-512 Enabled1122334455Min: 38.38 / Avg: 50.61 / Max: 52.5Min: 39.38 / Avg: 49.58 / Max: 51.13

ONNX Runtime

Model: GPT-2 - Device: CPU - Executor: Standard

OpenBenchmarking.orgInferences Per Second, More Is BetterONNX Runtime 1.19Model: GPT-2 - Device: CPU - Executor: StandardAVX-512 DisabledAVX-512 Enabled4080120160200SE +/- 0.35, N = 3SE +/- 0.28, N = 3182.81196.351. (CXX) g++ options: -O3 -march=native -march=znver5 -flto -ffunction-sections -fdata-sections -mtune=native -flto=auto -fno-fat-lto-objects -ldl -lrt

ONNX Runtime

Model: GPT-2 - Device: CPU - Executor: Standard

OpenBenchmarking.orgInference Time Cost (ms), Fewer Is BetterONNX Runtime 1.19Model: GPT-2 - Device: CPU - Executor: StandardAVX-512 DisabledAVX-512 Enabled1.23032.46063.69094.92126.1515SE +/- 0.01048, N = 3SE +/- 0.00735, N = 35.468205.091081. (CXX) g++ options: -O3 -march=native -march=znver5 -flto -ffunction-sections -fdata-sections -mtune=native -flto=auto -fno-fat-lto-objects -ldl -lrt

ONNX Runtime

CPU Peak Freq (Highest CPU Core Frequency) Monitor

OpenBenchmarking.orgMegahertz, More Is BetterONNX Runtime 1.19CPU Peak Freq (Highest CPU Core Frequency) MonitorAVX-512 DisabledAVX-512 Enabled8001600240032004000Min: 2600 / Avg: 4426.17 / Max: 4546Min: 2600 / Avg: 4416.61 / Max: 4569

ONNX Runtime

CPU Power Consumption Monitor

OpenBenchmarking.orgWatts, Fewer Is BetterONNX Runtime 1.19CPU Power Consumption MonitorAVX-512 DisabledAVX-512 Enabled306090120150Min: 36.49 / Avg: 147.92 / Max: 156.91Min: 0.44 / Avg: 146.99 / Max: 156.52

ONNX Runtime

CPU Temperature Monitor

OpenBenchmarking.orgCelsius, Fewer Is BetterONNX Runtime 1.19CPU Temperature MonitorAVX-512 DisabledAVX-512 Enabled918273645Min: 38.5 / Avg: 44.07 / Max: 45.5Min: 38.88 / Avg: 44.72 / Max: 46

ONNX Runtime

Model: ArcFace ResNet-100 - Device: CPU - Executor: Standard

OpenBenchmarking.orgInferences Per Second, More Is BetterONNX Runtime 1.19Model: ArcFace ResNet-100 - Device: CPU - Executor: StandardAVX-512 DisabledAVX-512 Enabled1122334455SE +/- 0.36, N = 8SE +/- 0.53, N = 340.5548.781. (CXX) g++ options: -O3 -march=native -march=znver5 -flto -ffunction-sections -fdata-sections -mtune=native -flto=auto -fno-fat-lto-objects -ldl -lrt

ONNX Runtime

Model: ArcFace ResNet-100 - Device: CPU - Executor: Standard

OpenBenchmarking.orgInference Time Cost (ms), Fewer Is BetterONNX Runtime 1.19Model: ArcFace ResNet-100 - Device: CPU - Executor: StandardAVX-512 DisabledAVX-512 Enabled612182430SE +/- 0.22, N = 8SE +/- 0.22, N = 324.6720.501. (CXX) g++ options: -O3 -march=native -march=znver5 -flto -ffunction-sections -fdata-sections -mtune=native -flto=auto -fno-fat-lto-objects -ldl -lrt

ONNX Runtime

CPU Peak Freq (Highest CPU Core Frequency) Monitor

OpenBenchmarking.orgMegahertz, More Is BetterONNX Runtime 1.19CPU Peak Freq (Highest CPU Core Frequency) MonitorAVX-512 DisabledAVX-512 Enabled8001600240032004000Min: 2600 / Avg: 4096.82 / Max: 4554Min: 2600 / Avg: 3936.11 / Max: 4564

ONNX Runtime

CPU Power Consumption Monitor

OpenBenchmarking.orgWatts, Fewer Is BetterONNX Runtime 1.19CPU Power Consumption MonitorAVX-512 DisabledAVX-512 Enabled60120180240300Min: 0.74 / Avg: 291.22 / Max: 317.17Min: 1.07 / Avg: 259.86 / Max: 281.69

ONNX Runtime

CPU Temperature Monitor

OpenBenchmarking.orgCelsius, Fewer Is BetterONNX Runtime 1.19CPU Temperature MonitorAVX-512 DisabledAVX-512 Enabled1020304050Min: 33.88 / Avg: 48.88 / Max: 52.25Min: 34.88 / Avg: 45.81 / Max: 47.63

ONNX Runtime

Model: Faster R-CNN R-50-FPN-int8 - Device: CPU - Executor: Standard

OpenBenchmarking.orgInferences Per Second, More Is BetterONNX Runtime 1.19Model: Faster R-CNN R-50-FPN-int8 - Device: CPU - Executor: StandardAVX-512 DisabledAVX-512 Enabled1428425670SE +/- 0.45, N = 15SE +/- 0.50, N = 337.1564.071. (CXX) g++ options: -O3 -march=native -march=znver5 -flto -ffunction-sections -fdata-sections -mtune=native -flto=auto -fno-fat-lto-objects -ldl -lrt

ONNX Runtime

Model: Faster R-CNN R-50-FPN-int8 - Device: CPU - Executor: Standard

OpenBenchmarking.orgInference Time Cost (ms), Fewer Is BetterONNX Runtime 1.19Model: Faster R-CNN R-50-FPN-int8 - Device: CPU - Executor: StandardAVX-512 DisabledAVX-512 Enabled612182430SE +/- 0.33, N = 15SE +/- 0.12, N = 326.9715.611. (CXX) g++ options: -O3 -march=native -march=znver5 -flto -ffunction-sections -fdata-sections -mtune=native -flto=auto -fno-fat-lto-objects -ldl -lrt

ONNX Runtime

CPU Peak Freq (Highest CPU Core Frequency) Monitor

OpenBenchmarking.orgMegahertz, More Is BetterONNX Runtime 1.19CPU Peak Freq (Highest CPU Core Frequency) MonitorAVX-512 DisabledAVX-512 Enabled8001600240032004000Min: 2600 / Avg: 4278.95 / Max: 4568Min: 2600 / Avg: 4262.82 / Max: 4568

ONNX Runtime

CPU Power Consumption Monitor

OpenBenchmarking.orgWatts, Fewer Is BetterONNX Runtime 1.19CPU Power Consumption MonitorAVX-512 DisabledAVX-512 Enabled50100150200250Min: 0.9 / Avg: 239.2 / Max: 267.87Min: 0.4 / Avg: 268.32 / Max: 289.36

ONNX Runtime

CPU Temperature Monitor

OpenBenchmarking.orgCelsius, Fewer Is BetterONNX Runtime 1.19CPU Temperature MonitorAVX-512 DisabledAVX-512 Enabled1224364860Min: 37 / Avg: 55.24 / Max: 57.63Min: 39.25 / Avg: 58.41 / Max: 62.75

ONNX Runtime

Model: T5 Encoder - Device: CPU - Executor: Standard

OpenBenchmarking.orgInferences Per Second, More Is BetterONNX Runtime 1.19Model: T5 Encoder - Device: CPU - Executor: StandardAVX-512 DisabledAVX-512 Enabled50100150200250SE +/- 0.75, N = 3SE +/- 1.84, N = 3238.64244.991. (CXX) g++ options: -O3 -march=native -march=znver5 -flto -ffunction-sections -fdata-sections -mtune=native -flto=auto -fno-fat-lto-objects -ldl -lrt

ONNX Runtime

Model: T5 Encoder - Device: CPU - Executor: Standard

OpenBenchmarking.orgInference Time Cost (ms), Fewer Is BetterONNX Runtime 1.19Model: T5 Encoder - Device: CPU - Executor: StandardAVX-512 DisabledAVX-512 Enabled0.94271.88542.82813.77084.7135SE +/- 0.01312, N = 3SE +/- 0.03056, N = 34.189724.081141. (CXX) g++ options: -O3 -march=native -march=znver5 -flto -ffunction-sections -fdata-sections -mtune=native -flto=auto -fno-fat-lto-objects -ldl -lrt

ONNX Runtime

CPU Peak Freq (Highest CPU Core Frequency) Monitor

OpenBenchmarking.orgMegahertz, More Is BetterONNX Runtime 1.19CPU Peak Freq (Highest CPU Core Frequency) MonitorAVX-512 DisabledAVX-512 Enabled8001600240032004000Min: 2600 / Avg: 4427.57 / Max: 4563Min: 2600 / Avg: 4416 / Max: 4542

ONNX Runtime

CPU Power Consumption Monitor

OpenBenchmarking.orgWatts, Fewer Is BetterONNX Runtime 1.19CPU Power Consumption MonitorAVX-512 DisabledAVX-512 Enabled306090120150Min: 0.11 / Avg: 138.67 / Max: 148.1Min: 0.13 / Avg: 140.78 / Max: 152.77

ONNX Runtime

CPU Temperature Monitor

OpenBenchmarking.orgCelsius, Fewer Is BetterONNX Runtime 1.19CPU Temperature MonitorAVX-512 DisabledAVX-512 Enabled918273645Min: 38.75 / Avg: 42.4 / Max: 43.63Min: 41 / Avg: 44.03 / Max: 46

ONNX Runtime

Model: ResNet101_DUC_HDC-12 - Device: CPU - Executor: Standard

OpenBenchmarking.orgInferences Per Second, More Is BetterONNX Runtime 1.19Model: ResNet101_DUC_HDC-12 - Device: CPU - Executor: StandardAVX-512 DisabledAVX-512 Enabled246810SE +/- 0.15591, N = 12SE +/- 0.06862, N = 44.309146.507951. (CXX) g++ options: -O3 -march=native -march=znver5 -flto -ffunction-sections -fdata-sections -mtune=native -flto=auto -fno-fat-lto-objects -ldl -lrt

ONNX Runtime

Model: ResNet101_DUC_HDC-12 - Device: CPU - Executor: Standard

OpenBenchmarking.orgInference Time Cost (ms), Fewer Is BetterONNX Runtime 1.19Model: ResNet101_DUC_HDC-12 - Device: CPU - Executor: StandardAVX-512 DisabledAVX-512 Enabled50100150200250SE +/- 9.18, N = 12SE +/- 1.61, N = 4235.71153.711. (CXX) g++ options: -O3 -march=native -march=znver5 -flto -ffunction-sections -fdata-sections -mtune=native -flto=auto -fno-fat-lto-objects -ldl -lrt

ONNX Runtime

CPU Peak Freq (Highest CPU Core Frequency) Monitor

OpenBenchmarking.orgMegahertz, More Is BetterONNX Runtime 1.19CPU Peak Freq (Highest CPU Core Frequency) MonitorAVX-512 DisabledAVX-512 Enabled8001600240032004000Min: 2600 / Avg: 3239.62 / Max: 4575Min: 2600 / Avg: 2916.93 / Max: 4543

ONNX Runtime

CPU Power Consumption Monitor

OpenBenchmarking.orgWatts, Fewer Is BetterONNX Runtime 1.19CPU Power Consumption MonitorAVX-512 DisabledAVX-512 Enabled60120180240300Min: 0.55 / Avg: 286.1 / Max: 324.37Min: 0.49 / Avg: 258.72 / Max: 282.16

ONNX Runtime

CPU Temperature Monitor

OpenBenchmarking.orgCelsius, Fewer Is BetterONNX Runtime 1.19CPU Temperature MonitorAVX-512 DisabledAVX-512 Enabled1122334455Min: 39.5 / Avg: 51.21 / Max: 53.25Min: 39.75 / Avg: 48.18 / Max: 49.63

OpenVINO

Model: Face Detection FP16-INT8 - Device: CPU

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2024.5Model: Face Detection FP16-INT8 - Device: CPUAVX-512 DisabledAVX-512 Enabled306090120150SE +/- 0.04, N = 3SE +/- 0.11, N = 385.54147.06-march=znver51. (CXX) g++ options: -fPIC -O3 -flto -fsigned-char -ffunction-sections -fdata-sections -fno-strict-overflow -fwrapv -shared -ldl -lstdc++fs

OpenVINO

Model: Face Detection FP16-INT8 - Device: CPU

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2024.5Model: Face Detection FP16-INT8 - Device: CPUAVX-512 DisabledAVX-512 Enabled120240360480600SE +/- 0.19, N = 3SE +/- 0.27, N = 3558.74325.74MIN: 273.34 / MAX: 587.28-march=znver5 - MIN: 255.71 / MAX: 363.131. (CXX) g++ options: -fPIC -O3 -flto -fsigned-char -ffunction-sections -fdata-sections -fno-strict-overflow -fwrapv -shared -ldl -lstdc++fs

OpenVINO

CPU Peak Freq (Highest CPU Core Frequency) Monitor

OpenBenchmarking.orgMegahertz, More Is BetterOpenVINO 2024.5CPU Peak Freq (Highest CPU Core Frequency) MonitorAVX-512 DisabledAVX-512 Enabled8001600240032004000Min: 2574 / Avg: 2722.94 / Max: 4564Min: 2239 / Avg: 2473.08 / Max: 4567

OpenVINO

CPU Power Consumption Monitor

OpenBenchmarking.orgWatts, Fewer Is BetterOpenVINO 2024.5CPU Power Consumption MonitorAVX-512 DisabledAVX-512 Enabled60120180240300Min: 1.61 / Avg: 295.57 / Max: 326.2Min: 1.25 / Avg: 283.24 / Max: 311.62

OpenVINO

CPU Temperature Monitor

OpenBenchmarking.orgCelsius, Fewer Is BetterOpenVINO 2024.5CPU Temperature MonitorAVX-512 DisabledAVX-512 Enabled1122334455Min: 40.38 / Avg: 52.66 / Max: 55.88Min: 42.5 / Avg: 52.78 / Max: 54.38

OpenVINO

Model: Age Gender Recognition Retail 0013 FP16-INT8 - Device: CPU

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2024.5Model: Age Gender Recognition Retail 0013 FP16-INT8 - Device: CPUAVX-512 DisabledAVX-512 Enabled30K60K90K120K150KSE +/- 170.89, N = 3SE +/- 180.45, N = 3129762.91162706.14-march=znver51. (CXX) g++ options: -fPIC -O3 -flto -fsigned-char -ffunction-sections -fdata-sections -fno-strict-overflow -fwrapv -shared -ldl -lstdc++fs

OpenVINO

Model: Age Gender Recognition Retail 0013 FP16-INT8 - Device: CPU

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2024.5Model: Age Gender Recognition Retail 0013 FP16-INT8 - Device: CPUAVX-512 DisabledAVX-512 Enabled0.11930.23860.35790.47720.5965SE +/- 0.00, N = 3SE +/- 0.00, N = 30.530.31MIN: 0.18 / MAX: 13.44-march=znver5 - MIN: 0.12 / MAX: 25.731. (CXX) g++ options: -fPIC -O3 -flto -fsigned-char -ffunction-sections -fdata-sections -fno-strict-overflow -fwrapv -shared -ldl -lstdc++fs

OpenVINO

CPU Peak Freq (Highest CPU Core Frequency) Monitor

OpenBenchmarking.orgMegahertz, More Is BetterOpenVINO 2024.5CPU Peak Freq (Highest CPU Core Frequency) MonitorAVX-512 DisabledAVX-512 Enabled8001600240032004000Min: 2600 / Avg: 2860.53 / Max: 4567Min: 2600 / Avg: 2876.42 / Max: 4546

OpenVINO

CPU Power Consumption Monitor

OpenBenchmarking.orgWatts, Fewer Is BetterOpenVINO 2024.5CPU Power Consumption MonitorAVX-512 DisabledAVX-512 Enabled50100150200250Min: 1.45 / Avg: 271.96 / Max: 293.6Min: 1.05 / Avg: 246.58 / Max: 270.54

OpenVINO

CPU Temperature Monitor

OpenBenchmarking.orgCelsius, Fewer Is BetterOpenVINO 2024.5CPU Temperature MonitorAVX-512 DisabledAVX-512 Enabled1020304050Min: 41.75 / Avg: 49.3 / Max: 51Min: 42.75 / Avg: 48.39 / Max: 50.5

OpenVINO

Model: Person Detection FP16 - Device: CPU

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2024.5Model: Person Detection FP16 - Device: CPUAVX-512 DisabledAVX-512 Enabled150300450600750SE +/- 0.82, N = 3SE +/- 0.91, N = 3241.77710.04-march=znver51. (CXX) g++ options: -fPIC -O3 -flto -fsigned-char -ffunction-sections -fdata-sections -fno-strict-overflow -fwrapv -shared -ldl -lstdc++fs

OpenVINO

Model: Person Detection FP16 - Device: CPU

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2024.5Model: Person Detection FP16 - Device: CPUAVX-512 DisabledAVX-512 Enabled4080120160200SE +/- 0.68, N = 3SE +/- 0.09, N = 3198.1767.47MIN: 94.52 / MAX: 339.84-march=znver5 - MIN: 29.95 / MAX: 185.481. (CXX) g++ options: -fPIC -O3 -flto -fsigned-char -ffunction-sections -fdata-sections -fno-strict-overflow -fwrapv -shared -ldl -lstdc++fs

OpenVINO

CPU Peak Freq (Highest CPU Core Frequency) Monitor

OpenBenchmarking.orgMegahertz, More Is BetterOpenVINO 2024.5CPU Peak Freq (Highest CPU Core Frequency) MonitorAVX-512 DisabledAVX-512 Enabled8001600240032004000Min: 2595 / Avg: 2827.98 / Max: 4555Min: 2394 / Avg: 2594.63 / Max: 4554

OpenVINO

CPU Power Consumption Monitor

OpenBenchmarking.orgWatts, Fewer Is BetterOpenVINO 2024.5CPU Power Consumption MonitorAVX-512 DisabledAVX-512 Enabled60120180240300Min: 1.07 / Avg: 320.76 / Max: 358.34Min: 0.62 / Avg: 326.41 / Max: 360.67

OpenVINO

CPU Temperature Monitor

OpenBenchmarking.orgCelsius, Fewer Is BetterOpenVINO 2024.5CPU Temperature MonitorAVX-512 DisabledAVX-512 Enabled1122334455Min: 40 / Avg: 53.07 / Max: 55.25Min: 40.5 / Avg: 54.34 / Max: 57.13

OpenVINO

Model: Weld Porosity Detection FP16 - Device: CPU

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2024.5Model: Weld Porosity Detection FP16 - Device: CPUAVX-512 DisabledAVX-512 Enabled16003200480064008000SE +/- 1.32, N = 3SE +/- 5.62, N = 32189.067611.17-march=znver51. (CXX) g++ options: -fPIC -O3 -flto -fsigned-char -ffunction-sections -fdata-sections -fno-strict-overflow -fwrapv -shared -ldl -lstdc++fs

OpenVINO

Model: Weld Porosity Detection FP16 - Device: CPU

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2024.5Model: Weld Porosity Detection FP16 - Device: CPUAVX-512 DisabledAVX-512 Enabled1020304050SE +/- 0.03, N = 3SE +/- 0.01, N = 343.7312.53MIN: 23.02 / MAX: 62.81-march=znver5 - MIN: 5.9 / MAX: 31.11. (CXX) g++ options: -fPIC -O3 -flto -fsigned-char -ffunction-sections -fdata-sections -fno-strict-overflow -fwrapv -shared -ldl -lstdc++fs

OpenVINO

CPU Peak Freq (Highest CPU Core Frequency) Monitor

OpenBenchmarking.orgMegahertz, More Is BetterOpenVINO 2024.5CPU Peak Freq (Highest CPU Core Frequency) MonitorAVX-512 DisabledAVX-512 Enabled8001600240032004000Min: 2470 / Avg: 2551.48 / Max: 4563Min: 2270 / Avg: 2390.53 / Max: 4551

OpenVINO

CPU Power Consumption Monitor

OpenBenchmarking.orgWatts, Fewer Is BetterOpenVINO 2024.5CPU Power Consumption MonitorAVX-512 DisabledAVX-512 Enabled60120180240300Min: 1.05 / Avg: 304.33 / Max: 329.28Min: 0.62 / Avg: 316.71 / Max: 341.63

OpenVINO

CPU Temperature Monitor

OpenBenchmarking.orgCelsius, Fewer Is BetterOpenVINO 2024.5CPU Temperature MonitorAVX-512 DisabledAVX-512 Enabled1122334455Min: 42.25 / Avg: 52.62 / Max: 53.88Min: 43.88 / Avg: 55.64 / Max: 57.25

OpenVINO

Model: Person Vehicle Bike Detection FP16 - Device: CPU

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2024.5Model: Person Vehicle Bike Detection FP16 - Device: CPUAVX-512 DisabledAVX-512 Enabled15003000450060007500SE +/- 2.95, N = 3SE +/- 17.52, N = 33132.506775.46-march=znver51. (CXX) g++ options: -fPIC -O3 -flto -fsigned-char -ffunction-sections -fdata-sections -fno-strict-overflow -fwrapv -shared -ldl -lstdc++fs

OpenVINO

Model: Person Vehicle Bike Detection FP16 - Device: CPU

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2024.5Model: Person Vehicle Bike Detection FP16 - Device: CPUAVX-512 DisabledAVX-512 Enabled48121620SE +/- 0.01, N = 3SE +/- 0.02, N = 315.267.03MIN: 7.72 / MAX: 48.83-march=znver5 - MIN: 3.43 / MAX: 24.511. (CXX) g++ options: -fPIC -O3 -flto -fsigned-char -ffunction-sections -fdata-sections -fno-strict-overflow -fwrapv -shared -ldl -lstdc++fs

OpenVINO

CPU Peak Freq (Highest CPU Core Frequency) Monitor

OpenBenchmarking.orgMegahertz, More Is BetterOpenVINO 2024.5CPU Peak Freq (Highest CPU Core Frequency) MonitorAVX-512 DisabledAVX-512 Enabled8001600240032004000Min: 2600 / Avg: 2695.03 / Max: 4545Min: 2216 / Avg: 2363.03 / Max: 4569

OpenVINO

CPU Power Consumption Monitor

OpenBenchmarking.orgWatts, Fewer Is BetterOpenVINO 2024.5CPU Power Consumption MonitorAVX-512 DisabledAVX-512 Enabled60120180240300Min: 0.87 / Avg: 304.37 / Max: 332.97Min: 0.53 / Avg: 264.6 / Max: 289.09

OpenVINO

CPU Temperature Monitor

OpenBenchmarking.orgCelsius, Fewer Is BetterOpenVINO 2024.5CPU Temperature MonitorAVX-512 DisabledAVX-512 Enabled1122334455Min: 41.75 / Avg: 51.96 / Max: 53.5Min: 44.38 / Avg: 50.87 / Max: 51.63

OpenVINO

Model: Machine Translation EN To DE FP16 - Device: CPU

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2024.5Model: Machine Translation EN To DE FP16 - Device: CPUAVX-512 DisabledAVX-512 Enabled2004006008001000SE +/- 0.09, N = 3SE +/- 0.97, N = 3295.88857.16-march=znver51. (CXX) g++ options: -fPIC -O3 -flto -fsigned-char -ffunction-sections -fdata-sections -fno-strict-overflow -fwrapv -shared -ldl -lstdc++fs

OpenVINO

Model: Machine Translation EN To DE FP16 - Device: CPU

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2024.5Model: Machine Translation EN To DE FP16 - Device: CPUAVX-512 DisabledAVX-512 Enabled4080120160200SE +/- 0.05, N = 3SE +/- 0.07, N = 3162.0255.95MIN: 75.62 / MAX: 264.14-march=znver5 - MIN: 26.76 / MAX: 125.181. (CXX) g++ options: -fPIC -O3 -flto -fsigned-char -ffunction-sections -fdata-sections -fno-strict-overflow -fwrapv -shared -ldl -lstdc++fs

OpenVINO

CPU Peak Freq (Highest CPU Core Frequency) Monitor

OpenBenchmarking.orgMegahertz, More Is BetterOpenVINO 2024.5CPU Peak Freq (Highest CPU Core Frequency) MonitorAVX-512 DisabledAVX-512 Enabled8001600240032004000Min: 2480 / Avg: 2633.44 / Max: 4568Min: 2411 / Avg: 2576.28 / Max: 4563

OpenVINO

CPU Power Consumption Monitor

OpenBenchmarking.orgWatts, Fewer Is BetterOpenVINO 2024.5CPU Power Consumption MonitorAVX-512 DisabledAVX-512 Enabled60120180240300Min: 0.71 / Avg: 320.19 / Max: 351.89Min: 0.21 / Avg: 324.19 / Max: 363.41

OpenVINO

CPU Temperature Monitor

OpenBenchmarking.orgCelsius, Fewer Is BetterOpenVINO 2024.5CPU Temperature MonitorAVX-512 DisabledAVX-512 Enabled1122334455Min: 41.75 / Avg: 53.37 / Max: 55.38Min: 41.38 / Avg: 55.24 / Max: 57.63

OpenVINO

Model: Face Detection Retail FP16 - Device: CPU

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2024.5Model: Face Detection Retail FP16 - Device: CPUAVX-512 DisabledAVX-512 Enabled4K8K12K16K20KSE +/- 8.72, N = 3SE +/- 12.77, N = 36688.9816580.18-march=znver51. (CXX) g++ options: -fPIC -O3 -flto -fsigned-char -ffunction-sections -fdata-sections -fno-strict-overflow -fwrapv -shared -ldl -lstdc++fs

OpenVINO

Model: Face Detection Retail FP16 - Device: CPU

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2024.5Model: Face Detection Retail FP16 - Device: CPUAVX-512 DisabledAVX-512 Enabled246810SE +/- 0.01, N = 3SE +/- 0.00, N = 37.142.82MIN: 2.91 / MAX: 32.04-march=znver5 - MIN: 1.14 / MAX: 18.121. (CXX) g++ options: -fPIC -O3 -flto -fsigned-char -ffunction-sections -fdata-sections -fno-strict-overflow -fwrapv -shared -ldl -lstdc++fs

OpenVINO

CPU Peak Freq (Highest CPU Core Frequency) Monitor

OpenBenchmarking.orgMegahertz, More Is BetterOpenVINO 2024.5CPU Peak Freq (Highest CPU Core Frequency) MonitorAVX-512 DisabledAVX-512 Enabled8001600240032004000Min: 2469 / Avg: 2591.17 / Max: 4530Min: 2269 / Avg: 2338.6 / Max: 4541

OpenVINO

CPU Power Consumption Monitor

OpenBenchmarking.orgWatts, Fewer Is BetterOpenVINO 2024.5CPU Power Consumption MonitorAVX-512 DisabledAVX-512 Enabled60120180240300Min: 0.69 / Avg: 315.06 / Max: 340.78Min: 0.21 / Avg: 299.8 / Max: 324.2

OpenVINO

CPU Temperature Monitor

OpenBenchmarking.orgCelsius, Fewer Is BetterOpenVINO 2024.5CPU Temperature MonitorAVX-512 DisabledAVX-512 Enabled1122334455Min: 42.38 / Avg: 53.28 / Max: 54.88Min: 44.38 / Avg: 54.39 / Max: 55.75

OpenVINO

Model: Handwritten English Recognition FP16 - Device: CPU

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2024.5Model: Handwritten English Recognition FP16 - Device: CPUAVX-512 DisabledAVX-512 Enabled7001400210028003500SE +/- 0.28, N = 3SE +/- 4.77, N = 31200.373488.23-march=znver51. (CXX) g++ options: -fPIC -O3 -flto -fsigned-char -ffunction-sections -fdata-sections -fno-strict-overflow -fwrapv -shared -ldl -lstdc++fs

OpenVINO

Model: Handwritten English Recognition FP16 - Device: CPU

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2024.5Model: Handwritten English Recognition FP16 - Device: CPUAVX-512 DisabledAVX-512 Enabled20406080100SE +/- 0.02, N = 3SE +/- 0.04, N = 379.9027.50MIN: 35.69 / MAX: 151.02-march=znver5 - MIN: 13.39 / MAX: 61.161. (CXX) g++ options: -fPIC -O3 -flto -fsigned-char -ffunction-sections -fdata-sections -fno-strict-overflow -fwrapv -shared -ldl -lstdc++fs

OpenVINO

CPU Peak Freq (Highest CPU Core Frequency) Monitor

OpenBenchmarking.orgMegahertz, More Is BetterOpenVINO 2024.5CPU Peak Freq (Highest CPU Core Frequency) MonitorAVX-512 DisabledAVX-512 Enabled8001600240032004000Min: 2447 / Avg: 2590.5 / Max: 4548Min: 2434 / Avg: 2573.7 / Max: 4562

OpenVINO

CPU Power Consumption Monitor

OpenBenchmarking.orgWatts, Fewer Is BetterOpenVINO 2024.5CPU Power Consumption MonitorAVX-512 DisabledAVX-512 Enabled60120180240300Min: 0.57 / Avg: 322.31 / Max: 347.06Min: 0.04 / Avg: 335.84 / Max: 363.56

OpenVINO

CPU Temperature Monitor

OpenBenchmarking.orgCelsius, Fewer Is BetterOpenVINO 2024.5CPU Temperature MonitorAVX-512 DisabledAVX-512 Enabled1224364860Min: 42.5 / Avg: 53.6 / Max: 55.5Min: 43.75 / Avg: 56.13 / Max: 58.25

OpenVINO

Model: Road Segmentation ADAS FP16 - Device: CPU

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2024.5Model: Road Segmentation ADAS FP16 - Device: CPUAVX-512 DisabledAVX-512 Enabled5001000150020002500SE +/- 1.87, N = 3SE +/- 6.46, N = 3713.962102.03-march=znver51. (CXX) g++ options: -fPIC -O3 -flto -fsigned-char -ffunction-sections -fdata-sections -fno-strict-overflow -fwrapv -shared -ldl -lstdc++fs

OpenVINO

Model: Road Segmentation ADAS FP16 - Device: CPU

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2024.5Model: Road Segmentation ADAS FP16 - Device: CPUAVX-512 DisabledAVX-512 Enabled1530456075SE +/- 0.18, N = 3SE +/- 0.07, N = 367.1222.78MIN: 17.27 / MAX: 133.16-march=znver5 - MIN: 11.57 / MAX: 67.031. (CXX) g++ options: -fPIC -O3 -flto -fsigned-char -ffunction-sections -fdata-sections -fno-strict-overflow -fwrapv -shared -ldl -lstdc++fs

OpenVINO

CPU Peak Freq (Highest CPU Core Frequency) Monitor

OpenBenchmarking.orgMegahertz, More Is BetterOpenVINO 2024.5CPU Peak Freq (Highest CPU Core Frequency) MonitorAVX-512 DisabledAVX-512 Enabled8001600240032004000Min: 2595 / Avg: 2730.19 / Max: 4576Min: 2323 / Avg: 2457.52 / Max: 4559

OpenVINO

CPU Power Consumption Monitor

OpenBenchmarking.orgWatts, Fewer Is BetterOpenVINO 2024.5CPU Power Consumption MonitorAVX-512 DisabledAVX-512 Enabled60120180240300Min: 0.49 / Avg: 279.46 / Max: 305.97Min: 0.04 / Avg: 305.32 / Max: 336.48

OpenVINO

CPU Temperature Monitor

OpenBenchmarking.orgCelsius, Fewer Is BetterOpenVINO 2024.5CPU Temperature MonitorAVX-512 DisabledAVX-512 Enabled1122334455Min: 42.75 / Avg: 49.82 / Max: 50.88Min: 45.25 / Avg: 54.03 / Max: 55.38

OpenVINO

Model: Person Re-Identification Retail FP16 - Device: CPU

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2024.5Model: Person Re-Identification Retail FP16 - Device: CPUAVX-512 DisabledAVX-512 Enabled2K4K6K8K10KSE +/- 2.23, N = 3SE +/- 6.44, N = 33605.7510524.04-march=znver51. (CXX) g++ options: -fPIC -O3 -flto -fsigned-char -ffunction-sections -fdata-sections -fno-strict-overflow -fwrapv -shared -ldl -lstdc++fs

OpenVINO

Model: Person Re-Identification Retail FP16 - Device: CPU

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2024.5Model: Person Re-Identification Retail FP16 - Device: CPUAVX-512 DisabledAVX-512 Enabled3691215SE +/- 0.01, N = 3SE +/- 0.00, N = 313.304.52MIN: 7.56 / MAX: 31.8-march=znver5 - MIN: 2.34 / MAX: 21.711. (CXX) g++ options: -fPIC -O3 -flto -fsigned-char -ffunction-sections -fdata-sections -fno-strict-overflow -fwrapv -shared -ldl -lstdc++fs

OpenVINO

CPU Peak Freq (Highest CPU Core Frequency) Monitor

OpenBenchmarking.orgMegahertz, More Is BetterOpenVINO 2024.5CPU Peak Freq (Highest CPU Core Frequency) MonitorAVX-512 DisabledAVX-512 Enabled8001600240032004000Min: 2522 / Avg: 2627.11 / Max: 4561Min: 2362 / Avg: 2549.18 / Max: 4548

OpenVINO

CPU Power Consumption Monitor

OpenBenchmarking.orgWatts, Fewer Is BetterOpenVINO 2024.5CPU Power Consumption MonitorAVX-512 DisabledAVX-512 Enabled60120180240300Min: 0.19 / Avg: 302.22 / Max: 329.27Min: 37.27 / Avg: 311.68 / Max: 337.81

OpenVINO

CPU Temperature Monitor

OpenBenchmarking.orgCelsius, Fewer Is BetterOpenVINO 2024.5CPU Temperature MonitorAVX-512 DisabledAVX-512 Enabled1122334455Min: 40.63 / Avg: 53.04 / Max: 54.88Min: 44.25 / Avg: 56.07 / Max: 57.63

OpenVINO

Model: Noise Suppression Poconet-Like FP16 - Device: CPU

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2024.5Model: Noise Suppression Poconet-Like FP16 - Device: CPUAVX-512 DisabledAVX-512 Enabled15003000450060007500SE +/- 6.13, N = 3SE +/- 13.37, N = 32516.896825.09-march=znver51. (CXX) g++ options: -fPIC -O3 -flto -fsigned-char -ffunction-sections -fdata-sections -fno-strict-overflow -fwrapv -shared -ldl -lstdc++fs

OpenVINO

Model: Noise Suppression Poconet-Like FP16 - Device: CPU

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2024.5Model: Noise Suppression Poconet-Like FP16 - Device: CPUAVX-512 DisabledAVX-512 Enabled918273645SE +/- 0.09, N = 3SE +/- 0.03, N = 338.0513.47MIN: 16.64 / MAX: 61.98-march=znver5 - MIN: 7.72 / MAX: 51.081. (CXX) g++ options: -fPIC -O3 -flto -fsigned-char -ffunction-sections -fdata-sections -fno-strict-overflow -fwrapv -shared -ldl -lstdc++fs

OpenVINO

CPU Peak Freq (Highest CPU Core Frequency) Monitor

OpenBenchmarking.orgMegahertz, More Is BetterOpenVINO 2024.5CPU Peak Freq (Highest CPU Core Frequency) MonitorAVX-512 DisabledAVX-512 Enabled8001600240032004000Min: 2600 / Avg: 3003.36 / Max: 4566Min: 2528 / Avg: 2682.43 / Max: 4556

OpenVINO

CPU Power Consumption Monitor

OpenBenchmarking.orgWatts, Fewer Is BetterOpenVINO 2024.5CPU Power Consumption MonitorAVX-512 DisabledAVX-512 Enabled60120180240300Min: 0.02 / Avg: 263.3 / Max: 290.12Min: 36.85 / Avg: 290.71 / Max: 317.82

OpenVINO

CPU Temperature Monitor

OpenBenchmarking.orgCelsius, Fewer Is BetterOpenVINO 2024.5CPU Temperature MonitorAVX-512 DisabledAVX-512 Enabled1122334455Min: 42.5 / Avg: 48.86 / Max: 49.75Min: 44.63 / Avg: 51.63 / Max: 52.75

OpenVINO GenAI

Model: TinyLlama-1.1B-Chat-v1.0 - Device: CPU

OpenBenchmarking.orgtokens/s, More Is BetterOpenVINO GenAI 2024.5Model: TinyLlama-1.1B-Chat-v1.0 - Device: CPUAVX-512 DisabledAVX-512 Enabled20406080100SE +/- 1.17, N = 15SE +/- 0.94, N = 391.4066.57

OpenVINO GenAI

Model: TinyLlama-1.1B-Chat-v1.0 - Device: CPU - Time To First Token

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO GenAI 2024.5Model: TinyLlama-1.1B-Chat-v1.0 - Device: CPU - Time To First TokenAVX-512 DisabledAVX-512 Enabled612182430SE +/- 0.58, N = 15SE +/- 0.18, N = 327.0118.59

OpenVINO GenAI

Model: TinyLlama-1.1B-Chat-v1.0 - Device: CPU - Time Per Output Token

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO GenAI 2024.5Model: TinyLlama-1.1B-Chat-v1.0 - Device: CPU - Time Per Output TokenAVX-512 DisabledAVX-512 Enabled48121620SE +/- 0.14, N = 15SE +/- 0.21, N = 310.9715.03

OpenVINO GenAI

CPU Peak Freq (Highest CPU Core Frequency) Monitor

OpenBenchmarking.orgMegahertz, More Is BetterOpenVINO GenAI 2024.5CPU Peak Freq (Highest CPU Core Frequency) MonitorAVX-512 DisabledAVX-512 Enabled8001600240032004000Min: 2600 / Avg: 3933.96 / Max: 4565Min: 2600 / Avg: 3645.71 / Max: 4548

OpenVINO GenAI

CPU Power Consumption Monitor

OpenBenchmarking.orgWatts, Fewer Is BetterOpenVINO GenAI 2024.5CPU Power Consumption MonitorAVX-512 DisabledAVX-512 Enabled60120180240300Min: 1.57 / Avg: 239.85 / Max: 314.54Min: 1.23 / Avg: 214.02 / Max: 268.91

OpenVINO GenAI

CPU Temperature Monitor

OpenBenchmarking.orgCelsius, Fewer Is BetterOpenVINO GenAI 2024.5CPU Temperature MonitorAVX-512 DisabledAVX-512 Enabled1020304050Min: 39.75 / Avg: 47.62 / Max: 49.13Min: 42.63 / Avg: 48.16 / Max: 49.88

OpenVKL

Benchmark: vklBenchmarkCPU ISPC

OpenBenchmarking.orgItems / Sec, More Is BetterOpenVKL 2.0.0Benchmark: vklBenchmarkCPU ISPCAVX-512 DisabledAVX-512 Enabled6001200180024003000SE +/- 0.58, N = 3SE +/- 0.58, N = 323532817MIN: 179 / MAX: 30230MIN: 217 / MAX: 36244

OpenVKL

CPU Peak Freq (Highest CPU Core Frequency) Monitor

OpenBenchmarking.orgMegahertz, More Is BetterOpenVKL 2.0.0CPU Peak Freq (Highest CPU Core Frequency) MonitorAVX-512 DisabledAVX-512 Enabled8001600240032004000Min: 2600 / Avg: 3532.47 / Max: 4576Min: 2600 / Avg: 3612.44 / Max: 4571

OpenVKL

CPU Power Consumption Monitor

OpenBenchmarking.orgWatts, Fewer Is BetterOpenVKL 2.0.0CPU Power Consumption MonitorAVX-512 DisabledAVX-512 Enabled70140210280350Min: 1.97 / Avg: 306.93 / Max: 377.3Min: 0.88 / Avg: 308.36 / Max: 372.72

OpenVKL

CPU Temperature Monitor

OpenBenchmarking.orgCelsius, Fewer Is BetterOpenVKL 2.0.0CPU Temperature MonitorAVX-512 DisabledAVX-512 Enabled1326395265Min: 37.5 / Avg: 54.39 / Max: 61.88Min: 37.38 / Avg: 57.47 / Max: 65.38

OSPRay

Benchmark: gravity_spheres_volume/dim_512/ao/real_time

OpenBenchmarking.orgItems Per Second, More Is BetterOSPRay 3.2Benchmark: gravity_spheres_volume/dim_512/ao/real_timeAVX-512 DisabledAVX-512 Enabled816243240SE +/- 0.01, N = 3SE +/- 0.02, N = 324.0335.28

OSPRay

CPU Peak Freq (Highest CPU Core Frequency) Monitor

OpenBenchmarking.orgMegahertz, More Is BetterOSPRay 3.2CPU Peak Freq (Highest CPU Core Frequency) MonitorAVX-512 DisabledAVX-512 Enabled8001600240032004000Min: 2600 / Avg: 3299.52 / Max: 4562Min: 2600 / Avg: 3311.38 / Max: 4571

OSPRay

CPU Power Consumption Monitor

OpenBenchmarking.orgWatts, Fewer Is BetterOSPRay 3.2CPU Power Consumption MonitorAVX-512 DisabledAVX-512 Enabled70140210280350Min: 1.33 / Avg: 311.01 / Max: 355.31Min: 37.02 / Avg: 330.88 / Max: 364.25

OSPRay

CPU Temperature Monitor

OpenBenchmarking.orgCelsius, Fewer Is BetterOSPRay 3.2CPU Temperature MonitorAVX-512 DisabledAVX-512 Enabled1326395265Min: 41.63 / Avg: 55.83 / Max: 63.13Min: 44.63 / Avg: 60.49 / Max: 69

OSPRay

Benchmark: gravity_spheres_volume/dim_512/scivis/real_time

OpenBenchmarking.orgItems Per Second, More Is BetterOSPRay 3.2Benchmark: gravity_spheres_volume/dim_512/scivis/real_timeAVX-512 DisabledAVX-512 Enabled816243240SE +/- 0.01, N = 3SE +/- 0.01, N = 323.1734.69

OSPRay

CPU Peak Freq (Highest CPU Core Frequency) Monitor

OpenBenchmarking.orgMegahertz, More Is BetterOSPRay 3.2CPU Peak Freq (Highest CPU Core Frequency) MonitorAVX-512 DisabledAVX-512 Enabled8001600240032004000Min: 2600 / Avg: 3317.14 / Max: 4562Min: 2600 / Avg: 3312.61 / Max: 4560

OSPRay

CPU Power Consumption Monitor

OpenBenchmarking.orgWatts, Fewer Is BetterOSPRay 3.2CPU Power Consumption MonitorAVX-512 DisabledAVX-512 Enabled70140210280350Min: 0.75 / Avg: 313.05 / Max: 359.7Min: 0.39 / Avg: 330.26 / Max: 366.06

OSPRay

CPU Temperature Monitor

OpenBenchmarking.orgCelsius, Fewer Is BetterOSPRay 3.2CPU Temperature MonitorAVX-512 DisabledAVX-512 Enabled1326395265Min: 42.5 / Avg: 56.43 / Max: 64.25Min: 46.38 / Avg: 60.94 / Max: 67.5

OSPRay

Benchmark: gravity_spheres_volume/dim_512/pathtracer/real_time

OpenBenchmarking.orgItems Per Second, More Is BetterOSPRay 3.2Benchmark: gravity_spheres_volume/dim_512/pathtracer/real_timeAVX-512 DisabledAVX-512 Enabled918273645SE +/- 0.04, N = 3SE +/- 0.02, N = 331.5838.20

OSPRay

CPU Peak Freq (Highest CPU Core Frequency) Monitor

OpenBenchmarking.orgMegahertz, More Is BetterOSPRay 3.2CPU Peak Freq (Highest CPU Core Frequency) MonitorAVX-512 DisabledAVX-512 Enabled8001600240032004000Min: 2600 / Avg: 3173.58 / Max: 4560Min: 2600 / Avg: 3170.5 / Max: 4566

OSPRay

CPU Power Consumption Monitor

OpenBenchmarking.orgWatts, Fewer Is BetterOSPRay 3.2CPU Power Consumption MonitorAVX-512 DisabledAVX-512 Enabled60120180240300Min: 0.2 / Avg: 305.49 / Max: 349.64Min: 0.93 / Avg: 315.73 / Max: 348.88

OSPRay

CPU Temperature Monitor

OpenBenchmarking.orgCelsius, Fewer Is BetterOSPRay 3.2CPU Temperature MonitorAVX-512 DisabledAVX-512 Enabled1326395265Min: 42.88 / Avg: 56.25 / Max: 63.25Min: 46.63 / Avg: 59.53 / Max: 66.25

OSPRay Studio

Camera: 1 - Resolution: 4K - Samples Per Pixel: 1 - Renderer: Path Tracer - Acceleration: CPU

OpenBenchmarking.orgms, Fewer Is BetterOSPRay Studio 1.0Camera: 1 - Resolution: 4K - Samples Per Pixel: 1 - Renderer: Path Tracer - Acceleration: CPUAVX-512 DisabledAVX-512 Enabled2004006008001000SE +/- 0.33, N = 3SE +/- 0.33, N = 3966868

OSPRay Studio

CPU Peak Freq (Highest CPU Core Frequency) Monitor

OpenBenchmarking.orgMegahertz, More Is BetterOSPRay Studio 1.0CPU Peak Freq (Highest CPU Core Frequency) MonitorAVX-512 DisabledAVX-512 Enabled8001600240032004000Min: 2600 / Avg: 3210.72 / Max: 4566Min: 2600 / Avg: 3207.7 / Max: 4551

OSPRay Studio

CPU Power Consumption Monitor

OpenBenchmarking.orgWatts, Fewer Is BetterOSPRay Studio 1.0CPU Power Consumption MonitorAVX-512 DisabledAVX-512 Enabled60120180240300Min: 0.81 / Avg: 285.31 / Max: 348.45Min: 0.92 / Avg: 282.47 / Max: 349.36

OSPRay Studio

CPU Temperature Monitor

OpenBenchmarking.orgCelsius, Fewer Is BetterOSPRay Studio 1.0CPU Temperature MonitorAVX-512 DisabledAVX-512 Enabled1326395265Min: 39.13 / Avg: 53.19 / Max: 62.75Min: 42.75 / Avg: 56.59 / Max: 66.88

OSPRay Studio

Camera: 1 - Resolution: 4K - Samples Per Pixel: 16 - Renderer: Path Tracer - Acceleration: CPU

OpenBenchmarking.orgms, Fewer Is BetterOSPRay Studio 1.0Camera: 1 - Resolution: 4K - Samples Per Pixel: 16 - Renderer: Path Tracer - Acceleration: CPUAVX-512 DisabledAVX-512 Enabled3K6K9K12K15KSE +/- 6.36, N = 3SE +/- 15.50, N = 31538213823

OSPRay Studio

CPU Peak Freq (Highest CPU Core Frequency) Monitor

OpenBenchmarking.orgMegahertz, More Is BetterOSPRay Studio 1.0CPU Peak Freq (Highest CPU Core Frequency) MonitorAVX-512 DisabledAVX-512 Enabled8001600240032004000Min: 2600 / Avg: 3221.06 / Max: 4547Min: 2600 / Avg: 3129.96 / Max: 4569

OSPRay Studio

CPU Power Consumption Monitor

OpenBenchmarking.orgWatts, Fewer Is BetterOSPRay Studio 1.0CPU Power Consumption MonitorAVX-512 DisabledAVX-512 Enabled60120180240300Min: 0.5 / Avg: 279.19 / Max: 346.69Min: 0.62 / Avg: 303.81 / Max: 348.8

OSPRay Studio

CPU Temperature Monitor

OpenBenchmarking.orgCelsius, Fewer Is BetterOSPRay Studio 1.0CPU Temperature MonitorAVX-512 DisabledAVX-512 Enabled1326395265Min: 41.13 / Avg: 53.68 / Max: 64Min: 44.25 / Avg: 57.72 / Max: 69.13

OSPRay Studio

Camera: 1 - Resolution: 4K - Samples Per Pixel: 32 - Renderer: Path Tracer - Acceleration: CPU

OpenBenchmarking.orgms, Fewer Is BetterOSPRay Studio 1.0Camera: 1 - Resolution: 4K - Samples Per Pixel: 32 - Renderer: Path Tracer - Acceleration: CPUAVX-512 DisabledAVX-512 Enabled7K14K21K28K35KSE +/- 88.93, N = 3SE +/- 80.19, N = 33466830958

OSPRay Studio

CPU Peak Freq (Highest CPU Core Frequency) Monitor

OpenBenchmarking.orgMegahertz, More Is BetterOSPRay Studio 1.0CPU Peak Freq (Highest CPU Core Frequency) MonitorAVX-512 DisabledAVX-512 Enabled8001600240032004000Min: 2600 / Avg: 3292.85 / Max: 4565Min: 2600 / Avg: 3314.8 / Max: 4569

OSPRay Studio

CPU Power Consumption Monitor

OpenBenchmarking.orgWatts, Fewer Is BetterOSPRay Studio 1.0CPU Power Consumption MonitorAVX-512 DisabledAVX-512 Enabled60120180240300Min: 36.53 / Avg: 256.86 / Max: 346.05Min: 1.32 / Avg: 250.81 / Max: 347.38

OSPRay Studio

CPU Temperature Monitor

OpenBenchmarking.orgCelsius, Fewer Is BetterOSPRay Studio 1.0CPU Temperature MonitorAVX-512 DisabledAVX-512 Enabled1326395265Min: 40.75 / Avg: 52.93 / Max: 60Min: 45.25 / Avg: 57.63 / Max: 68.25

OSPRay Studio

Camera: 2 - Resolution: 4K - Samples Per Pixel: 1 - Renderer: Path Tracer - Acceleration: CPU

OpenBenchmarking.orgms, Fewer Is BetterOSPRay Studio 1.0Camera: 2 - Resolution: 4K - Samples Per Pixel: 1 - Renderer: Path Tracer - Acceleration: CPUAVX-512 DisabledAVX-512 Enabled2004006008001000SE +/- 0.67, N = 3SE +/- 0.67, N = 3974873

OSPRay Studio

CPU Peak Freq (Highest CPU Core Frequency) Monitor

OpenBenchmarking.orgMegahertz, More Is BetterOSPRay Studio 1.0CPU Peak Freq (Highest CPU Core Frequency) MonitorAVX-512 DisabledAVX-512 Enabled8001600240032004000Min: 2600 / Avg: 3247.36 / Max: 4575Min: 2600 / Avg: 3233.67 / Max: 4563

OSPRay Studio

CPU Power Consumption Monitor

OpenBenchmarking.orgWatts, Fewer Is BetterOSPRay Studio 1.0CPU Power Consumption MonitorAVX-512 DisabledAVX-512 Enabled60120180240300Min: 1.24 / Avg: 286.84 / Max: 349.23Min: 0.31 / Avg: 283.34 / Max: 345.47

OSPRay Studio

CPU Temperature Monitor

OpenBenchmarking.orgCelsius, Fewer Is BetterOSPRay Studio 1.0CPU Temperature MonitorAVX-512 DisabledAVX-512 Enabled1326395265Min: 40.25 / Avg: 53.39 / Max: 62.63Min: 43.38 / Avg: 56.47 / Max: 66.13

OSPRay Studio

Camera: 2 - Resolution: 4K - Samples Per Pixel: 16 - Renderer: Path Tracer - Acceleration: CPU

OpenBenchmarking.orgms, Fewer Is BetterOSPRay Studio 1.0Camera: 2 - Resolution: 4K - Samples Per Pixel: 16 - Renderer: Path Tracer - Acceleration: CPUAVX-512 DisabledAVX-512 Enabled3K6K9K12K15KSE +/- 17.35, N = 3SE +/- 18.02, N = 31550313901

OSPRay Studio

CPU Peak Freq (Highest CPU Core Frequency) Monitor

OpenBenchmarking.orgMegahertz, More Is BetterOSPRay Studio 1.0CPU Peak Freq (Highest CPU Core Frequency) MonitorAVX-512 DisabledAVX-512 Enabled8001600240032004000Min: 2600 / Avg: 3230.91 / Max: 4572Min: 2600 / Avg: 3138.26 / Max: 4559

OSPRay Studio

CPU Power Consumption Monitor

OpenBenchmarking.orgWatts, Fewer Is BetterOSPRay Studio 1.0CPU Power Consumption MonitorAVX-512 DisabledAVX-512 Enabled60120180240300Min: 0.93 / Avg: 282.37 / Max: 350.52Min: 0.03 / Avg: 301.64 / Max: 351.17

OSPRay Studio

CPU Temperature Monitor

OpenBenchmarking.orgCelsius, Fewer Is BetterOSPRay Studio 1.0CPU Temperature MonitorAVX-512 DisabledAVX-512 Enabled1326395265Min: 41.25 / Avg: 53.71 / Max: 62.75Min: 44.13 / Avg: 57.23 / Max: 68.38

OSPRay Studio

Camera: 2 - Resolution: 4K - Samples Per Pixel: 32 - Renderer: Path Tracer - Acceleration: CPU

OpenBenchmarking.orgms, Fewer Is BetterOSPRay Studio 1.0Camera: 2 - Resolution: 4K - Samples Per Pixel: 32 - Renderer: Path Tracer - Acceleration: CPUAVX-512 DisabledAVX-512 Enabled7K14K21K28K35KSE +/- 52.20, N = 3SE +/- 30.12, N = 33448031077

OSPRay Studio

CPU Peak Freq (Highest CPU Core Frequency) Monitor

OpenBenchmarking.orgMegahertz, More Is BetterOSPRay Studio 1.0CPU Peak Freq (Highest CPU Core Frequency) MonitorAVX-512 DisabledAVX-512 Enabled8001600240032004000Min: 2600 / Avg: 3313.46 / Max: 4563Min: 2600 / Avg: 3343.68 / Max: 4571

OSPRay Studio

CPU Power Consumption Monitor

OpenBenchmarking.orgWatts, Fewer Is BetterOSPRay Studio 1.0CPU Power Consumption MonitorAVX-512 DisabledAVX-512 Enabled60120180240300Min: 0.43 / Avg: 259.93 / Max: 349.27Min: 0.71 / Avg: 250.67 / Max: 343.32

OSPRay Studio

CPU Temperature Monitor

OpenBenchmarking.orgCelsius, Fewer Is BetterOSPRay Studio 1.0CPU Temperature MonitorAVX-512 DisabledAVX-512 Enabled1326395265Min: 41.38 / Avg: 53.67 / Max: 62.63Min: 45 / Avg: 57.53 / Max: 68

OSPRay Studio

Camera: 3 - Resolution: 4K - Samples Per Pixel: 1 - Renderer: Path Tracer - Acceleration: CPU

OpenBenchmarking.orgms, Fewer Is BetterOSPRay Studio 1.0Camera: 3 - Resolution: 4K - Samples Per Pixel: 1 - Renderer: Path Tracer - Acceleration: CPUAVX-512 DisabledAVX-512 Enabled2004006008001000SE +/- 0.33, N = 3SE +/- 1.15, N = 311351024

OSPRay Studio

CPU Peak Freq (Highest CPU Core Frequency) Monitor

OpenBenchmarking.orgMegahertz, More Is BetterOSPRay Studio 1.0CPU Peak Freq (Highest CPU Core Frequency) MonitorAVX-512 DisabledAVX-512 Enabled8001600240032004000Min: 2600 / Avg: 3232.49 / Max: 4565Min: 2600 / Avg: 3218.88 / Max: 4545

OSPRay Studio

CPU Power Consumption Monitor

OpenBenchmarking.orgWatts, Fewer Is BetterOSPRay Studio 1.0CPU Power Consumption MonitorAVX-512 DisabledAVX-512 Enabled60120180240300Min: 1.69 / Avg: 286.42 / Max: 348.82Min: 1.64 / Avg: 284.65 / Max: 349.24

OSPRay Studio

CPU Temperature Monitor

OpenBenchmarking.orgCelsius, Fewer Is BetterOSPRay Studio 1.0CPU Temperature MonitorAVX-512 DisabledAVX-512 Enabled1326395265Min: 40.38 / Avg: 53.32 / Max: 62.25Min: 43 / Avg: 56.47 / Max: 65.88

OSPRay Studio

Camera: 3 - Resolution: 4K - Samples Per Pixel: 16 - Renderer: Path Tracer - Acceleration: CPU

OpenBenchmarking.orgms, Fewer Is BetterOSPRay Studio 1.0Camera: 3 - Resolution: 4K - Samples Per Pixel: 16 - Renderer: Path Tracer - Acceleration: CPUAVX-512 DisabledAVX-512 Enabled4K8K12K16K20KSE +/- 34.35, N = 3SE +/- 1.45, N = 31809816297

OSPRay Studio

CPU Peak Freq (Highest CPU Core Frequency) Monitor

OpenBenchmarking.orgMegahertz, More Is BetterOSPRay Studio 1.0CPU Peak Freq (Highest CPU Core Frequency) MonitorAVX-512 DisabledAVX-512 Enabled8001600240032004000Min: 2600 / Avg: 3195.19 / Max: 4571Min: 2600 / Avg: 3212.44 / Max: 4562

OSPRay Studio

CPU Power Consumption Monitor

OpenBenchmarking.orgWatts, Fewer Is BetterOSPRay Studio 1.0CPU Power Consumption MonitorAVX-512 DisabledAVX-512 Enabled60120180240300Min: 1.44 / Avg: 288.83 / Max: 349.3Min: 1.41 / Avg: 279.53 / Max: 349.12

OSPRay Studio

CPU Temperature Monitor

OpenBenchmarking.orgCelsius, Fewer Is BetterOSPRay Studio 1.0CPU Temperature MonitorAVX-512 DisabledAVX-512 Enabled1326395265Min: 41.38 / Avg: 53.9 / Max: 63.38Min: 44 / Avg: 56.51 / Max: 66.25

OSPRay Studio

Camera: 3 - Resolution: 4K - Samples Per Pixel: 32 - Renderer: Path Tracer - Acceleration: CPU

OpenBenchmarking.orgms, Fewer Is BetterOSPRay Studio 1.0Camera: 3 - Resolution: 4K - Samples Per Pixel: 32 - Renderer: Path Tracer - Acceleration: CPUAVX-512 DisabledAVX-512 Enabled9K18K27K36K45KSE +/- 59.00, N = 3SE +/- 50.45, N = 33978535955

OSPRay Studio

CPU Peak Freq (Highest CPU Core Frequency) Monitor

OpenBenchmarking.orgMegahertz, More Is BetterOSPRay Studio 1.0CPU Peak Freq (Highest CPU Core Frequency) MonitorAVX-512 DisabledAVX-512 Enabled8001600240032004000Min: 2600 / Avg: 3282.21 / Max: 4552Min: 2600 / Avg: 3291.96 / Max: 4561

OSPRay Studio

CPU Power Consumption Monitor

OpenBenchmarking.orgWatts, Fewer Is BetterOSPRay Studio 1.0CPU Power Consumption MonitorAVX-512 DisabledAVX-512 Enabled60120180240300Min: 1.22 / Avg: 266.98 / Max: 349.17Min: 1.05 / Avg: 258.37 / Max: 349.72

OSPRay Studio

CPU Temperature Monitor

OpenBenchmarking.orgCelsius, Fewer Is BetterOSPRay Studio 1.0CPU Temperature MonitorAVX-512 DisabledAVX-512 Enabled1326395265Min: 41.5 / Avg: 53.34 / Max: 61.5Min: 43.63 / Avg: 56.16 / Max: 64.88

PyTorch

Device: CPU - Batch Size: 256 - Model: ResNet-50

OpenBenchmarking.orgbatches/sec, More Is BetterPyTorch 2.2.1Device: CPU - Batch Size: 256 - Model: ResNet-50AVX-512 DisabledAVX-512 Enabled1122334455SE +/- 0.21, N = 3SE +/- 0.31, N = 343.3350.52MIN: 38.8 / MAX: 44.27MIN: 43.73 / MAX: 51.63

PyTorch

CPU Peak Freq (Highest CPU Core Frequency) Monitor

OpenBenchmarking.orgMegahertz, More Is BetterPyTorch 2.2.1CPU Peak Freq (Highest CPU Core Frequency) MonitorAVX-512 DisabledAVX-512 Enabled8001600240032004000Min: 2600 / Avg: 4212.95 / Max: 4574Min: 2600 / Avg: 4209.69 / Max: 4572

PyTorch

CPU Power Consumption Monitor

OpenBenchmarking.orgWatts, Fewer Is BetterPyTorch 2.2.1CPU Power Consumption MonitorAVX-512 DisabledAVX-512 Enabled50100150200250Min: 0.77 / Avg: 256.39 / Max: 297.62Min: 0.15 / Avg: 249.75 / Max: 294.5

PyTorch

CPU Temperature Monitor

OpenBenchmarking.orgCelsius, Fewer Is BetterPyTorch 2.2.1CPU Temperature MonitorAVX-512 DisabledAVX-512 Enabled1122334455Min: 27.13 / Avg: 43.57 / Max: 50.75Min: 36.38 / Avg: 47.66 / Max: 53.5

PyTorch

Device: CPU - Batch Size: 512 - Model: ResNet-50

OpenBenchmarking.orgbatches/sec, More Is BetterPyTorch 2.2.1Device: CPU - Batch Size: 512 - Model: ResNet-50AVX-512 DisabledAVX-512 Enabled1122334455SE +/- 0.14, N = 3SE +/- 0.28, N = 343.4450.84MIN: 38.59 / MAX: 44.4MIN: 44.51 / MAX: 52.23

PyTorch

CPU Peak Freq (Highest CPU Core Frequency) Monitor

OpenBenchmarking.orgMegahertz, More Is BetterPyTorch 2.2.1CPU Peak Freq (Highest CPU Core Frequency) MonitorAVX-512 DisabledAVX-512 Enabled8001600240032004000Min: 2600 / Avg: 4212.34 / Max: 4571Min: 2600 / Avg: 4217.46 / Max: 4566

PyTorch

CPU Power Consumption Monitor

OpenBenchmarking.orgWatts, Fewer Is BetterPyTorch 2.2.1CPU Power Consumption MonitorAVX-512 DisabledAVX-512 Enabled50100150200250Min: 36.76 / Avg: 259.81 / Max: 297.93Min: 1.08 / Avg: 250.98 / Max: 296.3

PyTorch

CPU Temperature Monitor

OpenBenchmarking.orgCelsius, Fewer Is BetterPyTorch 2.2.1CPU Temperature MonitorAVX-512 DisabledAVX-512 Enabled1122334455Min: 36 / Avg: 46 / Max: 52.13Min: 38.63 / Avg: 47.96 / Max: 54

SMHasher

Hash: FarmHash32 x86_64 AVX

OpenBenchmarking.orgMiB/sec, More Is BetterSMHasher 2022-08-22Hash: FarmHash32 x86_64 AVXAVX-512 DisabledAVX-512 Enabled8K16K24K32K40KSE +/- 21.63, N = 6SE +/- 10.36, N = 636139.9537638.621. (CXX) g++ options: -O3 -march=znver5 -flto -march=native -flto=auto -fno-fat-lto-objects

SMHasher

Hash: FarmHash32 x86_64 AVX

OpenBenchmarking.orgcycles/hash, Fewer Is BetterSMHasher 2022-08-22Hash: FarmHash32 x86_64 AVXAVX-512 DisabledAVX-512 Enabled510152025SE +/- 0.02, N = 6SE +/- 0.03, N = 622.6622.811. (CXX) g++ options: -O3 -march=znver5 -flto -march=native -flto=auto -fno-fat-lto-objects

SMHasher

CPU Peak Freq (Highest CPU Core Frequency) Monitor

OpenBenchmarking.orgMegahertz, More Is BetterSMHasher 2022-08-22CPU Peak Freq (Highest CPU Core Frequency) MonitorAVX-512 DisabledAVX-512 Enabled8001600240032004000Min: 2600 / Avg: 3893.07 / Max: 4550Min: 2600 / Avg: 3897.49 / Max: 4573

SMHasher

CPU Power Consumption Monitor

OpenBenchmarking.orgWatts, Fewer Is BetterSMHasher 2022-08-22CPU Power Consumption MonitorAVX-512 DisabledAVX-512 Enabled1530456075Min: 0.22 / Avg: 61.63 / Max: 78.15Min: 0.95 / Avg: 62.02 / Max: 78.27

SMHasher

CPU Temperature Monitor

OpenBenchmarking.orgCelsius, Fewer Is BetterSMHasher 2022-08-22CPU Temperature MonitorAVX-512 DisabledAVX-512 Enabled918273645Min: 30.25 / Avg: 36.44 / Max: 38.63Min: 32.75 / Avg: 38.67 / Max: 41.25

srsRAN Project

Test: PDSCH Processor Benchmark, Throughput Total

OpenBenchmarking.orgMbps, More Is BettersrsRAN Project 24.10Test: PDSCH Processor Benchmark, Throughput TotalAVX-512 DisabledAVX-512 Enabled30K60K90K120K150KSE +/- 1008.70, N = 3SE +/- 1542.84, N = 3110897.9124852.31. (CXX) g++ options: -O3 -march=native -mtune=generic -fno-trapping-math -fno-math-errno -ldl

srsRAN Project

CPU Peak Freq (Highest CPU Core Frequency) Monitor

OpenBenchmarking.orgMegahertz, More Is BettersrsRAN Project 24.10CPU Peak Freq (Highest CPU Core Frequency) MonitorAVX-512 DisabledAVX-512 Enabled8001600240032004000Min: 2600 / Avg: 2811.32 / Max: 4544Min: 2600 / Avg: 2762.93 / Max: 4542

srsRAN Project

CPU Power Consumption Monitor

OpenBenchmarking.orgWatts, Fewer Is BettersrsRAN Project 24.10CPU Power Consumption MonitorAVX-512 DisabledAVX-512 Enabled60120180240300Min: 1.13 / Avg: 292.55 / Max: 349.7Min: 0.15 / Avg: 271.4 / Max: 328.99

srsRAN Project

CPU Temperature Monitor

OpenBenchmarking.orgCelsius, Fewer Is BettersrsRAN Project 24.10CPU Temperature MonitorAVX-512 DisabledAVX-512 Enabled1020304050Min: 39.5 / Avg: 49.62 / Max: 52.38Min: 41.63 / Avg: 50.03 / Max: 52.38

SVT-AV1

Encoder Mode: Preset 13 - Input: Bosphorus 4K

OpenBenchmarking.orgFrames Per Second, More Is BetterSVT-AV1 2.3Encoder Mode: Preset 13 - Input: Bosphorus 4KAVX-512 DisabledAVX-512 Enabled90180270360450SE +/- 1.74, N = 6SE +/- 2.58, N = 6412.13416.851. (CXX) g++ options: -O3 -march=znver5 -flto -march=native -mno-avx -mavx2 -mavx512f -mavx512bw -mavx512dq

SVT-AV1

CPU Peak Freq (Highest CPU Core Frequency) Monitor

OpenBenchmarking.orgMegahertz, More Is BetterSVT-AV1 2.3CPU Peak Freq (Highest CPU Core Frequency) MonitorAVX-512 DisabledAVX-512 Enabled8001600240032004000Min: 2600 / Avg: 3774.89 / Max: 4562Min: 2600 / Avg: 3734.19 / Max: 4566

SVT-AV1

CPU Power Consumption Monitor

OpenBenchmarking.orgWatts, Fewer Is BetterSVT-AV1 2.3CPU Power Consumption MonitorAVX-512 DisabledAVX-512 Enabled50100150200250Min: 0.29 / Avg: 170.74 / Max: 305.74Min: 1.05 / Avg: 163.02 / Max: 295.39

SVT-AV1

CPU Temperature Monitor

OpenBenchmarking.orgCelsius, Fewer Is BetterSVT-AV1 2.3CPU Temperature MonitorAVX-512 DisabledAVX-512 Enabled1020304050Min: 34.38 / Avg: 44.92 / Max: 49.5Min: 36.13 / Avg: 45.83 / Max: 50.25

SVT-AV1

Encoder Mode: Preset 8 - Input: Bosphorus 4K

OpenBenchmarking.orgFrames Per Second, More Is BetterSVT-AV1 2.3Encoder Mode: Preset 8 - Input: Bosphorus 4KAVX-512 DisabledAVX-512 Enabled4080120160200SE +/- 1.07, N = 4SE +/- 1.02, N = 4171.29184.791. (CXX) g++ options: -O3 -march=znver5 -flto -march=native -mno-avx -mavx2 -mavx512f -mavx512bw -mavx512dq

SVT-AV1

CPU Peak Freq (Highest CPU Core Frequency) Monitor

OpenBenchmarking.orgMegahertz, More Is BetterSVT-AV1 2.3CPU Peak Freq (Highest CPU Core Frequency) MonitorAVX-512 DisabledAVX-512 Enabled8001600240032004000Min: 2600 / Avg: 3904.67 / Max: 4570Min: 2600 / Avg: 3914.47 / Max: 4570

SVT-AV1

CPU Power Consumption Monitor

OpenBenchmarking.orgWatts, Fewer Is BetterSVT-AV1 2.3CPU Power Consumption MonitorAVX-512 DisabledAVX-512 Enabled50100150200250Min: 1.23 / Avg: 188.91 / Max: 270.08Min: 0.59 / Avg: 185.34 / Max: 269.37

SVT-AV1

CPU Temperature Monitor

OpenBenchmarking.orgCelsius, Fewer Is BetterSVT-AV1 2.3CPU Temperature MonitorAVX-512 DisabledAVX-512 Enabled1020304050Min: 33.75 / Avg: 45.25 / Max: 49Min: 35.25 / Avg: 45.48 / Max: 50.63

SVT-AV1

Encoder Mode: Preset 5 - Input: Bosphorus 4K

OpenBenchmarking.orgFrames Per Second, More Is BetterSVT-AV1 2.3Encoder Mode: Preset 5 - Input: Bosphorus 4KAVX-512 DisabledAVX-512 Enabled1326395265SE +/- 0.29, N = 3SE +/- 0.23, N = 351.7456.481. (CXX) g++ options: -O3 -march=znver5 -flto -march=native -mno-avx -mavx2 -mavx512f -mavx512bw -mavx512dq

SVT-AV1

CPU Peak Freq (Highest CPU Core Frequency) Monitor

OpenBenchmarking.orgMegahertz, More Is BetterSVT-AV1 2.3CPU Peak Freq (Highest CPU Core Frequency) MonitorAVX-512 DisabledAVX-512 Enabled8001600240032004000Min: 2600 / Avg: 4239.94 / Max: 4571Min: 2600 / Avg: 4297 / Max: 4570

SVT-AV1

CPU Power Consumption Monitor

OpenBenchmarking.orgWatts, Fewer Is BetterSVT-AV1 2.3CPU Power Consumption MonitorAVX-512 DisabledAVX-512 Enabled50100150200250Min: 1.28 / Avg: 230.74 / Max: 300.94Min: 0.6 / Avg: 232.94 / Max: 291.22

SVT-AV1

CPU Temperature Monitor

OpenBenchmarking.orgCelsius, Fewer Is BetterSVT-AV1 2.3CPU Temperature MonitorAVX-512 DisabledAVX-512 Enabled1224364860Min: 35.5 / Avg: 53.19 / Max: 57.38Min: 36.88 / Avg: 54.65 / Max: 58.75

SVT-AV1

Encoder Mode: Preset 3 - Input: Bosphorus 4K

OpenBenchmarking.orgFrames Per Second, More Is BetterSVT-AV1 2.3Encoder Mode: Preset 3 - Input: Bosphorus 4KAVX-512 DisabledAVX-512 Enabled48121620SE +/- 0.05, N = 3SE +/- 0.03, N = 314.5215.651. (CXX) g++ options: -O3 -march=znver5 -flto -march=native -mno-avx -mavx2 -mavx512f -mavx512bw -mavx512dq

SVT-AV1

CPU Peak Freq (Highest CPU Core Frequency) Monitor

OpenBenchmarking.orgMegahertz, More Is BetterSVT-AV1 2.3CPU Peak Freq (Highest CPU Core Frequency) MonitorAVX-512 DisabledAVX-512 Enabled8001600240032004000Min: 2600 / Avg: 4391.15 / Max: 4583Min: 2600 / Avg: 4350.52 / Max: 4664

SVT-AV1

CPU Power Consumption Monitor

OpenBenchmarking.orgWatts, Fewer Is BetterSVT-AV1 2.3CPU Power Consumption MonitorAVX-512 DisabledAVX-512 Enabled60120180240300Min: 0.07 / Avg: 262.75 / Max: 330.69Min: 0.7 / Avg: 254.79 / Max: 334.43

SVT-AV1

CPU Temperature Monitor

OpenBenchmarking.orgCelsius, Fewer Is BetterSVT-AV1 2.3CPU Temperature MonitorAVX-512 DisabledAVX-512 Enabled1224364860Min: 36.63 / Avg: 58.79 / Max: 62.75Min: 37.88 / Avg: 59.2 / Max: 62.88

TensorFlow

Device: CPU - Batch Size: 256 - Model: ResNet-50

OpenBenchmarking.orgimages/sec, More Is BetterTensorFlow 2.16.1Device: CPU - Batch Size: 256 - Model: ResNet-50AVX-512 DisabledAVX-512 Enabled50100150200250SE +/- 0.30, N = 3SE +/- 0.23, N = 3152.47212.82

TensorFlow

CPU Peak Freq (Highest CPU Core Frequency) Monitor

OpenBenchmarking.orgMegahertz, More Is BetterTensorFlow 2.16.1CPU Peak Freq (Highest CPU Core Frequency) MonitorAVX-512 DisabledAVX-512 Enabled8001600240032004000Min: 2372 / Avg: 2955.16 / Max: 4564Min: 2065 / Avg: 2823.71 / Max: 4564

TensorFlow

CPU Power Consumption Monitor

OpenBenchmarking.orgWatts, Fewer Is BetterTensorFlow 2.16.1CPU Power Consumption MonitorAVX-512 DisabledAVX-512 Enabled50100150200250Min: 1.33 / Avg: 267.26 / Max: 298.64Min: 0.27 / Avg: 248.57 / Max: 270.85

TensorFlow

CPU Temperature Monitor

OpenBenchmarking.orgCelsius, Fewer Is BetterTensorFlow 2.16.1CPU Temperature MonitorAVX-512 DisabledAVX-512 Enabled1122334455Min: 37.38 / Avg: 48 / Max: 53.5Min: 38.63 / Avg: 47.61 / Max: 54.5

TensorFlow

Device: CPU - Batch Size: 512 - Model: ResNet-50

OpenBenchmarking.orgimages/sec, More Is BetterTensorFlow 2.16.1Device: CPU - Batch Size: 512 - Model: ResNet-50AVX-512 DisabledAVX-512 Enabled50100150200250SE +/- 0.06, N = 3SE +/- 0.01, N = 3161.27241.22

TensorFlow

CPU Peak Freq (Highest CPU Core Frequency) Monitor

OpenBenchmarking.orgMegahertz, More Is BetterTensorFlow 2.16.1CPU Peak Freq (Highest CPU Core Frequency) MonitorAVX-512 DisabledAVX-512 Enabled8001600240032004000Min: 2253 / Avg: 2945.32 / Max: 4568Min: 2036 / Avg: 2748.97 / Max: 4564

TensorFlow

CPU Power Consumption Monitor

OpenBenchmarking.orgWatts, Fewer Is BetterTensorFlow 2.16.1CPU Power Consumption MonitorAVX-512 DisabledAVX-512 Enabled60120180240300Min: 0.16 / Avg: 277.42 / Max: 315.43Min: 1.32 / Avg: 264.53 / Max: 295.65

TensorFlow

CPU Temperature Monitor

OpenBenchmarking.orgCelsius, Fewer Is BetterTensorFlow 2.16.1CPU Temperature MonitorAVX-512 DisabledAVX-512 Enabled1122334455Min: 39 / Avg: 50.44 / Max: 57.38Min: 40.5 / Avg: 50.21 / Max: 57.38

Y-Cruncher

Pi Digits To Calculate: 5B

OpenBenchmarking.orgSeconds, Fewer Is BetterY-Cruncher 0.8.5Pi Digits To Calculate: 5BAVX-512 DisabledAVX-512 Enabled816243240SE +/- 0.00, N = 3SE +/- 0.04, N = 333.0921.85

Y-Cruncher

CPU Peak Freq (Highest CPU Core Frequency) Monitor

OpenBenchmarking.orgMegahertz, More Is BetterY-Cruncher 0.8.5CPU Peak Freq (Highest CPU Core Frequency) MonitorAVX-512 DisabledAVX-512 Enabled8001600240032004000Min: 2387 / Avg: 3380.84 / Max: 4566Min: 2075 / Avg: 3308.94 / Max: 4567

Y-Cruncher

CPU Power Consumption Monitor

OpenBenchmarking.orgWatts, Fewer Is BetterY-Cruncher 0.8.5CPU Power Consumption MonitorAVX-512 DisabledAVX-512 Enabled70140210280350Min: 0.37 / Avg: 257.56 / Max: 387.52Min: 0.21 / Avg: 238.17 / Max: 383.47

Y-Cruncher

CPU Temperature Monitor

OpenBenchmarking.orgCelsius, Fewer Is BetterY-Cruncher 0.8.5CPU Temperature MonitorAVX-512 DisabledAVX-512 Enabled1122334455Min: 40.88 / Avg: 52.98 / Max: 57.13Min: 42.63 / Avg: 51.7 / Max: 57.13

Y-Cruncher

Pi Digits To Calculate: 10B

OpenBenchmarking.orgSeconds, Fewer Is BetterY-Cruncher 0.8.5Pi Digits To Calculate: 10BAVX-512 DisabledAVX-512 Enabled1530456075SE +/- 0.19, N = 3SE +/- 0.01, N = 368.1344.05

Y-Cruncher

CPU Peak Freq (Highest CPU Core Frequency) Monitor

OpenBenchmarking.orgMegahertz, More Is BetterY-Cruncher 0.8.5CPU Peak Freq (Highest CPU Core Frequency) MonitorAVX-512 DisabledAVX-512 Enabled8001600240032004000Min: 2378 / Avg: 3385.41 / Max: 4566Min: 2056 / Avg: 3198.93 / Max: 4561

Y-Cruncher

CPU Power Consumption Monitor

OpenBenchmarking.orgWatts, Fewer Is BetterY-Cruncher 0.8.5CPU Power Consumption MonitorAVX-512 DisabledAVX-512 Enabled70140210280350Min: 1.56 / Avg: 280.62 / Max: 389.7Min: 0.9 / Avg: 265.18 / Max: 384.27

Y-Cruncher

CPU Temperature Monitor

OpenBenchmarking.orgCelsius, Fewer Is BetterY-Cruncher 0.8.5CPU Temperature MonitorAVX-512 DisabledAVX-512 Enabled1224364860Min: 40.38 / Avg: 54.9 / Max: 61.38Min: 41.75 / Avg: 53.32 / Max: 61.5

Phoronix Test Suite v10.8.5