AMD EPYC Turin 2025 New AVX-512 Benchmarks

AMD EPYC 9655P AVX-512 on/off benchmarks by Michael Larabel for a future article.

HTML result view exported from: https://openbenchmarking.org/result/2501295-NE-AMDEPYCTU09&grt.

ProcessorMotherboardChipsetMemoryDiskGraphicsNetworkOSKernelDesktopDisplay ServerCompilerFile-SystemScreen ResolutionEPYC Turin AVX-512 Enabled AVX-512 DisabledAMD EPYC 9655P 96-Core @ 2.60GHz (96 Cores / 192 Threads)Supermicro Super Server H13SSL-N v1.01 (3.0 BIOS)AMD 1Ah12 x 64GB DDR5-6000MT/s Micron MTC40F2046S1RC64BDY QSFF3201GB Micron_7450_MTFDKCB3T2TFSASPEED2 x Broadcom NetXtreme BCM5720 PCIeUbuntu 24.106.13.0-rc4-phx-stock (x86_64)GNOME Shell 47.0X ServerGCC 14.2.0ext41024x768OpenBenchmarking.orgKernel Details- Transparent Huge Pages: madviseEnvironment Details- CXXFLAGS="-O3 -march=znver5 -mprefer-vector-width=512 -flto" CFLAGS="-O3 -march=znver5 -mprefer-vector-width=512 -flto" Compiler Details- --build=x86_64-linux-gnu --disable-vtable-verify --disable-werror --enable-bootstrap --enable-cet --enable-checking=release --enable-clocale=gnu --enable-default-pie --enable-gnu-unique-object --enable-languages=c,ada,c++,go,d,fortran,objc,obj-c++,m2,rust --enable-libphobos-checking=release --enable-libstdcxx-backtrace --enable-libstdcxx-debug --enable-libstdcxx-time=yes --enable-link-serialization=2 --enable-multiarch --enable-multilib --enable-nls --enable-objc-gc=auto --enable-offload-defaulted --enable-offload-targets=nvptx-none=/build/gcc-14-zdkDXv/gcc-14-14.2.0/debian/tmp-nvptx/usr,amdgcn-amdhsa=/build/gcc-14-zdkDXv/gcc-14-14.2.0/debian/tmp-gcn/usr --enable-plugin --enable-shared --enable-threads=posix --host=x86_64-linux-gnu --program-prefix=x86_64-linux-gnu- --target=x86_64-linux-gnu --with-abi=m64 --with-arch-32=i686 --with-build-config=bootstrap-lto-lean --with-default-libstdcxx-abi=new --with-gcc-major-version-only --with-multilib-list=m32,m64,mx32 --with-target-system-zlib=auto --with-tune=generic --without-cuda-driver -v Processor Details- Scaling Governor: acpi-cpufreq performance (Boost: Enabled) - CPU Microcode: 0xb002116 Python Details- Python 3.12.7Security Details- gather_data_sampling: Not affected + itlb_multihit: Not affected + l1tf: Not affected + mds: Not affected + meltdown: Not affected + mmio_stale_data: Not affected + reg_file_data_sampling: Not affected + retbleed: Not affected + spec_rstack_overflow: Not affected + spec_store_bypass: Mitigation of SSB disabled via prctl + spectre_v1: Mitigation of usercopy/swapgs barriers and __user pointer sanitization + spectre_v2: Mitigation of Enhanced / Automatic IBRS; IBPB: conditional; STIBP: always-on; RSB filling; PBRSB-eIBRS: Not affected; BHI: Not affected + srbds: Not affected + tsx_async_abort: Not affected

mt-dgemm: Sustained Floating-Point Ratecpuminer-opt: scryptcpuminer-opt: Skeincoincpuminer-opt: LBC, LBRY Creditsembree: Pathtracer ISPC - Asian Dragonembree: Pathtracer ISPC - Asian Dragon Objembree: Pathtracer ISPC - Crowngromacs: MPI CPU - water_GMX50_barelaghos: Sedov Blast Wave, ube_922_hex.meshlibxsmm: 128llama-cpp: CPU BLAS - Llama-3.1-Tulu-3-8B-Q8_0 - Prompt Processing 512llama-cpp: CPU BLAS - Llama-3.1-Tulu-3-8B-Q8_0 - Prompt Processing 1024llama-cpp: CPU BLAS - Llama-3.1-Tulu-3-8B-Q8_0 - Prompt Processing 2048llama-cpp: CPU BLAS - granite-3.0-3b-a800m-instruct-Q8_0 - Prompt Processing 512llama-cpp: CPU BLAS - Mistral-7B-Instruct-v0.3-Q8_0 - Prompt Processing 512llama-cpp: CPU BLAS - Mistral-7B-Instruct-v0.3-Q8_0 - Prompt Processing 1024llama-cpp: CPU BLAS - Mistral-7B-Instruct-v0.3-Q8_0 - Prompt Processing 2048minibude: OpenMP - BM1minibude: OpenMP - BM1minibude: OpenMP - BM2minibude: OpenMP - BM2mnn: mobilenetV3mnn: resnet-v2-50mnn: SqueezeNetV1.0numpy: onednn: Deconvolution Batch shapes_1d - CPUonednn: Deconvolution Batch shapes_3d - CPUonednn: IP Shapes 1D - CPUonednn: IP Shapes 3D - CPUonednn: Recurrent Neural Network Training - CPUonednn: Recurrent Neural Network Inference - CPUonnx: fcn-resnet101-11 - CPU - Standardonnx: fcn-resnet101-11 - CPU - Standardonnx: super-resolution-10 - CPU - Standardonnx: super-resolution-10 - CPU - Standardonnx: bertsquad-12 - CPU - Standardonnx: bertsquad-12 - CPU - Standardonnx: GPT-2 - CPU - Standardonnx: GPT-2 - CPU - Standardonnx: ArcFace ResNet-100 - CPU - Standardonnx: ArcFace ResNet-100 - CPU - Standardonnx: Faster R-CNN R-50-FPN-int8 - CPU - Standardonnx: Faster R-CNN R-50-FPN-int8 - CPU - Standardonnx: T5 Encoder - CPU - Standardonnx: T5 Encoder - CPU - Standardonnx: ResNet101_DUC_HDC-12 - CPU - Standardonnx: ResNet101_DUC_HDC-12 - CPU - Standardopenvino: Face Detection FP16-INT8 - CPUopenvino: Face Detection FP16-INT8 - CPUopenvino: Age Gender Recognition Retail 0013 FP16-INT8 - CPUopenvino: Age Gender Recognition Retail 0013 FP16-INT8 - CPUopenvino: Person Detection FP16 - CPUopenvino: Person Detection FP16 - CPUopenvino: Weld Porosity Detection FP16 - CPUopenvino: Weld Porosity Detection FP16 - CPUopenvino: Person Vehicle Bike Detection FP16 - CPUopenvino: Person Vehicle Bike Detection FP16 - CPUopenvino: Machine Translation EN To DE FP16 - CPUopenvino: Machine Translation EN To DE FP16 - CPUopenvino: Face Detection Retail FP16 - CPUopenvino: Face Detection Retail FP16 - CPUopenvino: Handwritten English Recognition FP16 - CPUopenvino: Handwritten English Recognition FP16 - CPUopenvino: Road Segmentation ADAS FP16 - CPUopenvino: Road Segmentation ADAS FP16 - CPUopenvino: Person Re-Identification Retail FP16 - CPUopenvino: Person Re-Identification Retail FP16 - CPUopenvino: Noise Suppression Poconet-Like FP16 - CPUopenvino: Noise Suppression Poconet-Like FP16 - CPUopenvino-genai: TinyLlama-1.1B-Chat-v1.0 - CPUopenvino-genai: TinyLlama-1.1B-Chat-v1.0 - CPU - Time To First Tokenopenvino-genai: TinyLlama-1.1B-Chat-v1.0 - CPU - Time Per Output Tokenopenvkl: vklBenchmarkCPU ISPCospray: gravity_spheres_volume/dim_512/ao/real_timeospray: gravity_spheres_volume/dim_512/scivis/real_timeospray: gravity_spheres_volume/dim_512/pathtracer/real_timeospray-studio: 1 - 4K - 1 - Path Tracer - CPUospray-studio: 1 - 4K - 16 - Path Tracer - CPUospray-studio: 1 - 4K - 32 - Path Tracer - CPUospray-studio: 2 - 4K - 1 - Path Tracer - CPUospray-studio: 2 - 4K - 16 - Path Tracer - CPUospray-studio: 2 - 4K - 32 - Path Tracer - CPUospray-studio: 3 - 4K - 1 - Path Tracer - CPUospray-studio: 3 - 4K - 16 - Path Tracer - CPUospray-studio: 3 - 4K - 32 - Path Tracer - CPUpytorch: CPU - 256 - ResNet-50pytorch: CPU - 512 - ResNet-50smhasher: FarmHash32 x86_64 AVXsmhasher: FarmHash32 x86_64 AVXsrsran: PDSCH Processor Benchmark, Throughput Totalsvt-av1: Preset 13 - Bosphorus 4Ksvt-av1: Preset 8 - Bosphorus 4Ksvt-av1: Preset 5 - Bosphorus 4Ksvt-av1: Preset 3 - Bosphorus 4Ktensorflow: CPU - 256 - ResNet-50tensorflow: CPU - 512 - ResNet-50y-cruncher: 5By-cruncher: 10BEPYC Turin AVX-512 Enabled AVX-512 Disabled4225.4328972937.261286043750993172.9627148.3243136.415517.416603.923422.0107.84106.71107.34356.76109.25107.32108.746368.737254.7507162.269286.4911.7367.4093.121882.006.724660.7169790.5334140.265927428.500278.3109.56433104.8273215.1624.6475022.433144.5813196.3485.0910848.781720.503064.066115.6086244.9894.081146.50795153.707147.06325.74162706.140.31710.0467.477611.1712.536775.467.03857.1655.9516580.182.823488.2327.502102.0322.7810524.044.526825.0913.4766.5718.5915.03281735.284934.689138.1975868138233095887313901310771024162973595550.5250.8437638.6222.808124852.3416.846184.79156.47715.649212.82241.2221.84744.05370.732517846.44747190245007159.2583138.2788126.631714.125590.603004.695.0393.4595.47347.1096.1594.9095.273416.427136.6573666.094146.6441.8039.9503.327828.6712.86011.354390.6462113.07554491.687307.0506.58603151.849170.9495.8509617.759056.3086182.8105.4682040.554824.669437.152026.9708238.6404.189724.30914235.70585.54558.74129762.910.53241.77198.172189.0643.733132.5015.26295.88162.026688.987.141200.3779.90713.9667.123605.7513.302516.8938.0591.4027.0110.97235324.028023.167831.5810966153823466897415503344801135180983978543.3343.4436139.9522.658110897.9412.131171.28951.73514.521152.47161.2733.08568.129OpenBenchmarking.org

ACES DGEMM

Sustained Floating-Point Rate

OpenBenchmarking.orgGFLOP/s, More Is BetterACES DGEMM 1.0Sustained Floating-Point RateAVX-512 EnabledAVX-512 Disabled9001800270036004500SE +/- 8.70, N = 4SE +/- 0.03, N = 34225.4370.731. (CC) gcc options: -ffast-math -O3 -march=znver5 -flto -mavx2 -fopenmp -lopenblas

ACES DGEMM

CPU Peak Freq (Highest CPU Core Frequency) Monitor

OpenBenchmarking.orgMegahertz, More Is BetterACES DGEMM 1.0CPU Peak Freq (Highest CPU Core Frequency) MonitorAVX-512 EnabledAVX-512 Disabled8001600240032004000Min: 2187 / Avg: 2495.27 / Max: 4544Min: 2600 / Avg: 4512.78 / Max: 4573

ACES DGEMM

CPU Power Consumption Monitor

OpenBenchmarking.orgWatts, Fewer Is BetterACES DGEMM 1.0CPU Power Consumption MonitorAVX-512 EnabledAVX-512 Disabled50100150200250Min: 36.7 / Avg: 194.6 / Max: 256.01Min: 36.88 / Avg: 85.07 / Max: 92.11

ACES DGEMM

CPU Temperature Monitor

OpenBenchmarking.orgCelsius, Fewer Is BetterACES DGEMM 1.0CPU Temperature MonitorAVX-512 EnabledAVX-512 Disabled1122334455Min: 46 / Avg: 48.6 / Max: 50.5Min: 41.25 / Avg: 49.42 / Max: 57.38

Cpuminer-Opt

Algorithm: scrypt

OpenBenchmarking.orgkH/s, More Is BetterCpuminer-Opt 24.3Algorithm: scryptAVX-512 EnabledAVX-512 Disabled6001200180024003000SE +/- 3.40, N = 3SE +/- 0.72, N = 32937.26846.44-mno-avx512f1. (CXX) g++ options: -O3 -march=znver5 -flto -lcurl -lz -lpthread -lgmp

Cpuminer-Opt

CPU Peak Freq (Highest CPU Core Frequency) Monitor

OpenBenchmarking.orgMegahertz, More Is BetterCpuminer-Opt 24.3CPU Peak Freq (Highest CPU Core Frequency) MonitorAVX-512 EnabledAVX-512 Disabled8001600240032004000Min: 2600 / Avg: 3550.91 / Max: 4551Min: 2600 / Avg: 3767.56 / Max: 4534

Cpuminer-Opt

CPU Power Consumption Monitor

OpenBenchmarking.orgWatts, Fewer Is BetterCpuminer-Opt 24.3CPU Power Consumption MonitorAVX-512 EnabledAVX-512 Disabled70140210280350Min: 37.17 / Avg: 332.45 / Max: 374.99Min: 18.63 / Avg: 330.99 / Max: 379.47

Cpuminer-Opt

CPU Temperature Monitor

OpenBenchmarking.orgCelsius, Fewer Is BetterCpuminer-Opt 24.3CPU Temperature MonitorAVX-512 EnabledAVX-512 Disabled1122334455Min: 26.75 / Avg: 49.05 / Max: 54.88Min: 28.25 / Avg: 48.52 / Max: 53.13

Cpuminer-Opt

Algorithm: Skeincoin

OpenBenchmarking.orgkH/s, More Is BetterCpuminer-Opt 24.3Algorithm: SkeincoinAVX-512 EnabledAVX-512 Disabled300K600K900K1200K1500KSE +/- 6599.64, N = 3SE +/- 2345.30, N = 31286043747190-mno-avx512f1. (CXX) g++ options: -O3 -march=znver5 -flto -lcurl -lz -lpthread -lgmp

Cpuminer-Opt

CPU Peak Freq (Highest CPU Core Frequency) Monitor

OpenBenchmarking.orgMegahertz, More Is BetterCpuminer-Opt 24.3CPU Peak Freq (Highest CPU Core Frequency) MonitorAVX-512 EnabledAVX-512 Disabled8001600240032004000Min: 2600 / Avg: 3208.97 / Max: 4575Min: 2600 / Avg: 3492.86 / Max: 4537

Cpuminer-Opt

CPU Power Consumption Monitor

OpenBenchmarking.orgWatts, Fewer Is BetterCpuminer-Opt 24.3CPU Power Consumption MonitorAVX-512 EnabledAVX-512 Disabled70140210280350Min: 0.73 / Avg: 312.01 / Max: 355.44Min: 7.98 / Avg: 318.09 / Max: 366.04

Cpuminer-Opt

CPU Temperature Monitor

OpenBenchmarking.orgCelsius, Fewer Is BetterCpuminer-Opt 24.3CPU Temperature MonitorAVX-512 EnabledAVX-512 Disabled1224364860Min: 42.75 / Avg: 55.68 / Max: 58.88Min: 40.75 / Avg: 53.68 / Max: 56.5

Cpuminer-Opt

Algorithm: LBC, LBRY Credits

OpenBenchmarking.orgkH/s, More Is BetterCpuminer-Opt 24.3Algorithm: LBC, LBRY CreditsAVX-512 EnabledAVX-512 Disabled160K320K480K640K800KSE +/- 622.53, N = 3SE +/- 2841.66, N = 3750993245007-mno-avx512f1. (CXX) g++ options: -O3 -march=znver5 -flto -lcurl -lz -lpthread -lgmp

Cpuminer-Opt

CPU Peak Freq (Highest CPU Core Frequency) Monitor

OpenBenchmarking.orgMegahertz, More Is BetterCpuminer-Opt 24.3CPU Peak Freq (Highest CPU Core Frequency) MonitorAVX-512 EnabledAVX-512 Disabled8001600240032004000Min: 2600 / Avg: 3198.91 / Max: 4544Min: 2600 / Avg: 3245.21 / Max: 4555

Cpuminer-Opt

CPU Power Consumption Monitor

OpenBenchmarking.orgWatts, Fewer Is BetterCpuminer-Opt 24.3CPU Power Consumption MonitorAVX-512 EnabledAVX-512 Disabled60120180240300Min: 37.18 / Avg: 311.96 / Max: 353.17Min: 36.66 / Avg: 314.81 / Max: 357.18

Cpuminer-Opt

CPU Temperature Monitor

OpenBenchmarking.orgCelsius, Fewer Is BetterCpuminer-Opt 24.3CPU Temperature MonitorAVX-512 EnabledAVX-512 Disabled1122334455Min: 42.88 / Avg: 55.37 / Max: 58Min: 40.5 / Avg: 52.67 / Max: 56.13

Embree

Binary: Pathtracer ISPC - Model: Asian Dragon

OpenBenchmarking.orgFrames Per Second, More Is BetterEmbree 4.3Binary: Pathtracer ISPC - Model: Asian DragonAVX-512 EnabledAVX-512 Disabled4080120160200SE +/- 0.06, N = 8SE +/- 0.10, N = 7172.96159.26MIN: 169.32 / MAX: 176.68MIN: 154.75 / MAX: 162.47

Embree

CPU Peak Freq (Highest CPU Core Frequency) Monitor

OpenBenchmarking.orgMegahertz, More Is BetterEmbree 4.3CPU Peak Freq (Highest CPU Core Frequency) MonitorAVX-512 EnabledAVX-512 Disabled8001600240032004000Min: 2600 / Avg: 3128.57 / Max: 4560Min: 2600 / Avg: 3040.09 / Max: 4547

Embree

CPU Power Consumption Monitor

OpenBenchmarking.orgWatts, Fewer Is BetterEmbree 4.3CPU Power Consumption MonitorAVX-512 EnabledAVX-512 Disabled60120180240300Min: 1.68 / Avg: 193.84 / Max: 362.36Min: 0.18 / Avg: 204.2 / Max: 361.91

Embree

CPU Temperature Monitor

OpenBenchmarking.orgCelsius, Fewer Is BetterEmbree 4.3CPU Temperature MonitorAVX-512 EnabledAVX-512 Disabled918273645Min: 33.5 / Avg: 42.23 / Max: 46Min: 30.25 / Avg: 41.89 / Max: 46

Embree

Binary: Pathtracer ISPC - Model: Asian Dragon Obj

OpenBenchmarking.orgFrames Per Second, More Is BetterEmbree 4.3Binary: Pathtracer ISPC - Model: Asian Dragon ObjAVX-512 EnabledAVX-512 Disabled306090120150SE +/- 0.11, N = 5SE +/- 0.18, N = 4148.32138.28MIN: 144.93 / MAX: 151.79MIN: 134.48 / MAX: 141.24

Embree

CPU Peak Freq (Highest CPU Core Frequency) Monitor

OpenBenchmarking.orgMegahertz, More Is BetterEmbree 4.3CPU Peak Freq (Highest CPU Core Frequency) MonitorAVX-512 EnabledAVX-512 Disabled8001600240032004000Min: 2600 / Avg: 3694.68 / Max: 4564Min: 2600 / Avg: 3719.73 / Max: 4565

Embree

CPU Power Consumption Monitor

OpenBenchmarking.orgWatts, Fewer Is BetterEmbree 4.3CPU Power Consumption MonitorAVX-512 EnabledAVX-512 Disabled60120180240300Min: 36.52 / Avg: 163.55 / Max: 361.39Min: 0.78 / Avg: 165.7 / Max: 361.58

Embree

CPU Temperature Monitor

OpenBenchmarking.orgCelsius, Fewer Is BetterEmbree 4.3CPU Temperature MonitorAVX-512 EnabledAVX-512 Disabled1122334455Min: 35.38 / Avg: 48.97 / Max: 54.63Min: 35.88 / Avg: 48.67 / Max: 53.88

Embree

Binary: Pathtracer ISPC - Model: Crown

OpenBenchmarking.orgFrames Per Second, More Is BetterEmbree 4.3Binary: Pathtracer ISPC - Model: CrownAVX-512 EnabledAVX-512 Disabled306090120150SE +/- 0.09, N = 7SE +/- 0.04, N = 7136.42126.63MIN: 132.5 / MAX: 141.56MIN: 123.57 / MAX: 130.86

Embree

CPU Peak Freq (Highest CPU Core Frequency) Monitor

OpenBenchmarking.orgMegahertz, More Is BetterEmbree 4.3CPU Peak Freq (Highest CPU Core Frequency) MonitorAVX-512 EnabledAVX-512 Disabled8001600240032004000Min: 2600 / Avg: 3105.48 / Max: 4563Min: 2600 / Avg: 3055.85 / Max: 4561

Embree

CPU Power Consumption Monitor

OpenBenchmarking.orgWatts, Fewer Is BetterEmbree 4.3CPU Power Consumption MonitorAVX-512 EnabledAVX-512 Disabled70140210280350Min: 0.33 / Avg: 219.77 / Max: 362.2Min: 1.25 / Avg: 226.37 / Max: 364.61

Embree

CPU Temperature Monitor

OpenBenchmarking.orgCelsius, Fewer Is BetterEmbree 4.3CPU Temperature MonitorAVX-512 EnabledAVX-512 Disabled1020304050Min: 36.13 / Avg: 45.47 / Max: 49.38Min: 36.25 / Avg: 45.23 / Max: 48.75

GROMACS

Implementation: MPI CPU - Input: water_GMX50_bare

OpenBenchmarking.orgNs Per Day, More Is BetterGROMACS 2024Implementation: MPI CPU - Input: water_GMX50_bareAVX-512 EnabledAVX-512 Disabled48121620SE +/- 0.02, N = 3SE +/- 0.04, N = 317.4214.13-mno-avx512f1. (CXX) g++ options: -O3 -march=znver5 -flto -lm

GROMACS

CPU Peak Freq (Highest CPU Core Frequency) Monitor

OpenBenchmarking.orgMegahertz, More Is BetterGROMACS 2024CPU Peak Freq (Highest CPU Core Frequency) MonitorAVX-512 EnabledAVX-512 Disabled8001600240032004000Min: 2600 / Avg: 3374.13 / Max: 4568Min: 2600 / Avg: 3308.87 / Max: 4572

GROMACS

CPU Power Consumption Monitor

OpenBenchmarking.orgWatts, Fewer Is BetterGROMACS 2024CPU Power Consumption MonitorAVX-512 EnabledAVX-512 Disabled60120180240300Min: 36.33 / Avg: 271.46 / Max: 353.96Min: 36.56 / Avg: 275.27 / Max: 358.87

GROMACS

CPU Temperature Monitor

OpenBenchmarking.orgCelsius, Fewer Is BetterGROMACS 2024CPU Temperature MonitorAVX-512 EnabledAVX-512 Disabled1122334455Min: 35.5 / Avg: 48.78 / Max: 54.38Min: 25.88 / Avg: 43.94 / Max: 49.75

Laghos

Test: Sedov Blast Wave, ube_922_hex.mesh

OpenBenchmarking.orgMajor Kernels Total Rate, More Is BetterLaghos 3.1Test: Sedov Blast Wave, ube_922_hex.meshAVX-512 EnabledAVX-512 Disabled130260390520650SE +/- 2.72, N = 3SE +/- 3.13, N = 3603.92590.60-mno-avx512f1. (CXX) g++ options: -O3 -march=znver5 -flto -lmfem -lHYPRE -lmetis -lrt -lmpi_cxx -lmpi

Laghos

CPU Peak Freq (Highest CPU Core Frequency) Monitor

OpenBenchmarking.orgMegahertz, More Is BetterLaghos 3.1CPU Peak Freq (Highest CPU Core Frequency) MonitorAVX-512 EnabledAVX-512 Disabled8001600240032004000Min: 2600 / Avg: 3459.12 / Max: 4564Min: 2600 / Avg: 3421.94 / Max: 4586

Laghos

CPU Power Consumption Monitor

OpenBenchmarking.orgWatts, Fewer Is BetterLaghos 3.1CPU Power Consumption MonitorAVX-512 EnabledAVX-512 Disabled60120180240300Min: 1.13 / Avg: 321.97 / Max: 352.64Min: 9.49 / Avg: 309.54 / Max: 345.92

Laghos

CPU Temperature Monitor

OpenBenchmarking.orgCelsius, Fewer Is BetterLaghos 3.1CPU Temperature MonitorAVX-512 EnabledAVX-512 Disabled1224364860Min: 46.88 / Avg: 59.01 / Max: 60.5Min: 31.63 / Avg: 47.78 / Max: 51.38

libxsmm

M N K: 128

OpenBenchmarking.orgGFLOPS/s, More Is Betterlibxsmm 2-1.17-3645M N K: 128AVX-512 EnabledAVX-512 Disabled7001400210028003500SE +/- 5.19, N = 3SE +/- 10.56, N = 33422.03004.61. (CXX) g++ options: -dynamic -Bstatic -static-libgcc -lgomp -lm -lrt -ldl -lquadmath -lstdc++ -pthread -fPIC -std=c++14 -O2 -fopenmp-simd -funroll-loops -ftree-vectorize -fdata-sections -ffunction-sections -fvisibility=hidden -msse4.2

libxsmm

CPU Peak Freq (Highest CPU Core Frequency) Monitor

OpenBenchmarking.orgMegahertz, More Is Betterlibxsmm 2-1.17-3645CPU Peak Freq (Highest CPU Core Frequency) MonitorAVX-512 EnabledAVX-512 Disabled8001600240032004000Min: 2600 / Avg: 4316.37 / Max: 4568Min: 2600 / Avg: 4357.28 / Max: 4572

libxsmm

CPU Power Consumption Monitor

OpenBenchmarking.orgWatts, Fewer Is Betterlibxsmm 2-1.17-3645CPU Power Consumption MonitorAVX-512 EnabledAVX-512 Disabled70140210280350Min: 0.42 / Avg: 176.21 / Max: 341.37Min: 0.98 / Avg: 171.17 / Max: 366.29

libxsmm

CPU Temperature Monitor

OpenBenchmarking.orgCelsius, Fewer Is Betterlibxsmm 2-1.17-3645CPU Temperature MonitorAVX-512 EnabledAVX-512 Disabled1122334455Min: 40 / Avg: 51.39 / Max: 53.5Min: 38.5 / Avg: 47.97 / Max: 50.75

Llama.cpp

Backend: CPU BLAS - Model: Llama-3.1-Tulu-3-8B-Q8_0 - Test: Prompt Processing 512

OpenBenchmarking.orgTokens Per Second, More Is BetterLlama.cpp b4397Backend: CPU BLAS - Model: Llama-3.1-Tulu-3-8B-Q8_0 - Test: Prompt Processing 512AVX-512 EnabledAVX-512 Disabled20406080100SE +/- 0.76, N = 3SE +/- 0.94, N = 3107.8495.03-mno-avx512f1. (CXX) g++ options: -O3 -march=znver5 -flto

Llama.cpp

CPU Peak Freq (Highest CPU Core Frequency) Monitor

OpenBenchmarking.orgMegahertz, More Is BetterLlama.cpp b4397CPU Peak Freq (Highest CPU Core Frequency) MonitorAVX-512 EnabledAVX-512 Disabled8001600240032004000Min: 2600 / Avg: 3527.61 / Max: 4565Min: 2600 / Avg: 3577.05 / Max: 4540

Llama.cpp

CPU Power Consumption Monitor

OpenBenchmarking.orgWatts, Fewer Is BetterLlama.cpp b4397CPU Power Consumption MonitorAVX-512 EnabledAVX-512 Disabled50100150200250Min: 0.58 / Avg: 223.48 / Max: 272.26Min: 2.83 / Avg: 238.62 / Max: 295

Llama.cpp

CPU Temperature Monitor

OpenBenchmarking.orgCelsius, Fewer Is BetterLlama.cpp b4397CPU Temperature MonitorAVX-512 EnabledAVX-512 Disabled1020304050Min: 38.88 / Avg: 46.74 / Max: 48.63Min: 36.13 / Avg: 45.03 / Max: 47

Llama.cpp

Backend: CPU BLAS - Model: Llama-3.1-Tulu-3-8B-Q8_0 - Test: Prompt Processing 1024

OpenBenchmarking.orgTokens Per Second, More Is BetterLlama.cpp b4397Backend: CPU BLAS - Model: Llama-3.1-Tulu-3-8B-Q8_0 - Test: Prompt Processing 1024AVX-512 EnabledAVX-512 Disabled20406080100SE +/- 0.46, N = 3SE +/- 0.60, N = 3106.7193.45-mno-avx512f1. (CXX) g++ options: -O3 -march=znver5 -flto

Llama.cpp

CPU Peak Freq (Highest CPU Core Frequency) Monitor

OpenBenchmarking.orgMegahertz, More Is BetterLlama.cpp b4397CPU Peak Freq (Highest CPU Core Frequency) MonitorAVX-512 EnabledAVX-512 Disabled8001600240032004000Min: 2600 / Avg: 3527.95 / Max: 4580Min: 2600 / Avg: 3623.36 / Max: 4547

Llama.cpp

CPU Power Consumption Monitor

OpenBenchmarking.orgWatts, Fewer Is BetterLlama.cpp b4397CPU Power Consumption MonitorAVX-512 EnabledAVX-512 Disabled50100150200250Min: 1.13 / Avg: 236.52 / Max: 282.16Min: 8.48 / Avg: 252.08 / Max: 292.53

Llama.cpp

CPU Temperature Monitor

OpenBenchmarking.orgCelsius, Fewer Is BetterLlama.cpp b4397CPU Temperature MonitorAVX-512 EnabledAVX-512 Disabled1020304050Min: 38.38 / Avg: 47.57 / Max: 48.88Min: 36.25 / Avg: 46.07 / Max: 47.63

Llama.cpp

Backend: CPU BLAS - Model: Llama-3.1-Tulu-3-8B-Q8_0 - Test: Prompt Processing 2048

OpenBenchmarking.orgTokens Per Second, More Is BetterLlama.cpp b4397Backend: CPU BLAS - Model: Llama-3.1-Tulu-3-8B-Q8_0 - Test: Prompt Processing 2048AVX-512 EnabledAVX-512 Disabled20406080100SE +/- 0.56, N = 3SE +/- 0.54, N = 3107.3495.47-mno-avx512f1. (CXX) g++ options: -O3 -march=znver5 -flto

Llama.cpp

CPU Peak Freq (Highest CPU Core Frequency) Monitor

OpenBenchmarking.orgMegahertz, More Is BetterLlama.cpp b4397CPU Peak Freq (Highest CPU Core Frequency) MonitorAVX-512 EnabledAVX-512 Disabled8001600240032004000Min: 2600 / Avg: 3527.95 / Max: 4566Min: 2600 / Avg: 3613.53 / Max: 4521

Llama.cpp

CPU Power Consumption Monitor

OpenBenchmarking.orgWatts, Fewer Is BetterLlama.cpp b4397CPU Power Consumption MonitorAVX-512 EnabledAVX-512 Disabled50100150200250Min: 0.65 / Avg: 244.86 / Max: 269.27Min: 5.84 / Avg: 258.55 / Max: 290.38

Llama.cpp

CPU Temperature Monitor

OpenBenchmarking.orgCelsius, Fewer Is BetterLlama.cpp b4397CPU Temperature MonitorAVX-512 EnabledAVX-512 Disabled1020304050Min: 38.75 / Avg: 48.39 / Max: 49.63Min: 36.63 / Avg: 46.96 / Max: 48.75

Llama.cpp

Backend: CPU BLAS - Model: granite-3.0-3b-a800m-instruct-Q8_0 - Test: Prompt Processing 512

OpenBenchmarking.orgTokens Per Second, More Is BetterLlama.cpp b4397Backend: CPU BLAS - Model: granite-3.0-3b-a800m-instruct-Q8_0 - Test: Prompt Processing 512AVX-512 EnabledAVX-512 Disabled80160240320400SE +/- 2.40, N = 5SE +/- 2.08, N = 5356.76347.10-mno-avx512f1. (CXX) g++ options: -O3 -march=znver5 -flto

Llama.cpp

CPU Peak Freq (Highest CPU Core Frequency) Monitor

OpenBenchmarking.orgMegahertz, More Is BetterLlama.cpp b4397CPU Peak Freq (Highest CPU Core Frequency) MonitorAVX-512 EnabledAVX-512 Disabled8001600240032004000Min: 2600 / Avg: 3430.29 / Max: 4544Min: 2600 / Avg: 3454.73 / Max: 4569

Llama.cpp

CPU Power Consumption Monitor

OpenBenchmarking.orgWatts, Fewer Is BetterLlama.cpp b4397CPU Power Consumption MonitorAVX-512 EnabledAVX-512 Disabled50100150200250Min: 36.59 / Avg: 178.87 / Max: 253.24Min: 3 / Avg: 178.29 / Max: 256.87

Llama.cpp

CPU Temperature Monitor

OpenBenchmarking.orgCelsius, Fewer Is BetterLlama.cpp b4397CPU Temperature MonitorAVX-512 EnabledAVX-512 Disabled918273645Min: 37 / Avg: 42.78 / Max: 44.63Min: 34.75 / Avg: 40.31 / Max: 42

Llama.cpp

Backend: CPU BLAS - Model: Mistral-7B-Instruct-v0.3-Q8_0 - Test: Prompt Processing 512

OpenBenchmarking.orgTokens Per Second, More Is BetterLlama.cpp b4397Backend: CPU BLAS - Model: Mistral-7B-Instruct-v0.3-Q8_0 - Test: Prompt Processing 512AVX-512 EnabledAVX-512 Disabled20406080100SE +/- 1.17, N = 5SE +/- 1.01, N = 4109.2596.15-mno-avx512f1. (CXX) g++ options: -O3 -march=znver5 -flto

Llama.cpp

CPU Peak Freq (Highest CPU Core Frequency) Monitor

OpenBenchmarking.orgMegahertz, More Is BetterLlama.cpp b4397CPU Peak Freq (Highest CPU Core Frequency) MonitorAVX-512 EnabledAVX-512 Disabled8001600240032004000Min: 2600 / Avg: 3512.12 / Max: 4558Min: 2600 / Avg: 3594.14 / Max: 4556

Llama.cpp

CPU Power Consumption Monitor

OpenBenchmarking.orgWatts, Fewer Is BetterLlama.cpp b4397CPU Power Consumption MonitorAVX-512 EnabledAVX-512 Disabled50100150200250Min: 0.17 / Avg: 226.72 / Max: 292.5Min: 4.3 / Avg: 242.37 / Max: 295

Llama.cpp

CPU Temperature Monitor

OpenBenchmarking.orgCelsius, Fewer Is BetterLlama.cpp b4397CPU Temperature MonitorAVX-512 EnabledAVX-512 Disabled1020304050Min: 38.63 / Avg: 46.34 / Max: 48.13Min: 36.5 / Avg: 45.61 / Max: 47.25

Llama.cpp

Backend: CPU BLAS - Model: Mistral-7B-Instruct-v0.3-Q8_0 - Test: Prompt Processing 1024

OpenBenchmarking.orgTokens Per Second, More Is BetterLlama.cpp b4397Backend: CPU BLAS - Model: Mistral-7B-Instruct-v0.3-Q8_0 - Test: Prompt Processing 1024AVX-512 EnabledAVX-512 Disabled20406080100SE +/- 1.15, N = 3SE +/- 0.77, N = 3107.3294.90-mno-avx512f1. (CXX) g++ options: -O3 -march=znver5 -flto

Llama.cpp

CPU Peak Freq (Highest CPU Core Frequency) Monitor

OpenBenchmarking.orgMegahertz, More Is BetterLlama.cpp b4397CPU Peak Freq (Highest CPU Core Frequency) MonitorAVX-512 EnabledAVX-512 Disabled8001600240032004000Min: 2600 / Avg: 3535.35 / Max: 4545Min: 2600 / Avg: 3628.57 / Max: 4544

Llama.cpp

CPU Power Consumption Monitor

OpenBenchmarking.orgWatts, Fewer Is BetterLlama.cpp b4397CPU Power Consumption MonitorAVX-512 EnabledAVX-512 Disabled50100150200250Min: 1.04 / Avg: 238.56 / Max: 271.81Min: 36.63 / Avg: 255.53 / Max: 287.29

Llama.cpp

CPU Temperature Monitor

OpenBenchmarking.orgCelsius, Fewer Is BetterLlama.cpp b4397CPU Temperature MonitorAVX-512 EnabledAVX-512 Disabled1020304050Min: 38.38 / Avg: 47.27 / Max: 49.13Min: 36.75 / Avg: 46.77 / Max: 48.88

Llama.cpp

Backend: CPU BLAS - Model: Mistral-7B-Instruct-v0.3-Q8_0 - Test: Prompt Processing 2048

OpenBenchmarking.orgTokens Per Second, More Is BetterLlama.cpp b4397Backend: CPU BLAS - Model: Mistral-7B-Instruct-v0.3-Q8_0 - Test: Prompt Processing 2048AVX-512 EnabledAVX-512 Disabled20406080100SE +/- 0.94, N = 3SE +/- 0.65, N = 3108.7495.27-mno-avx512f1. (CXX) g++ options: -O3 -march=znver5 -flto

Llama.cpp

CPU Peak Freq (Highest CPU Core Frequency) Monitor

OpenBenchmarking.orgMegahertz, More Is BetterLlama.cpp b4397CPU Peak Freq (Highest CPU Core Frequency) MonitorAVX-512 EnabledAVX-512 Disabled8001600240032004000Min: 2600 / Avg: 3524.34 / Max: 4584Min: 2600 / Avg: 3605.26 / Max: 4568

Llama.cpp

CPU Power Consumption Monitor

OpenBenchmarking.orgWatts, Fewer Is BetterLlama.cpp b4397CPU Power Consumption MonitorAVX-512 EnabledAVX-512 Disabled50100150200250Min: 0.57 / Avg: 245.43 / Max: 278.75Min: 7.92 / Avg: 259.65 / Max: 287.46

Llama.cpp

CPU Temperature Monitor

OpenBenchmarking.orgCelsius, Fewer Is BetterLlama.cpp b4397CPU Temperature MonitorAVX-512 EnabledAVX-512 Disabled1020304050Min: 38.5 / Avg: 48.02 / Max: 49.63Min: 37.38 / Avg: 47.47 / Max: 49.13

miniBUDE

Implementation: OpenMP - Input Deck: BM1

OpenBenchmarking.orgGFInst/s, More Is BetterminiBUDE 20210901Implementation: OpenMP - Input Deck: BM1AVX-512 EnabledAVX-512 Disabled14002800420056007000SE +/- 149.28, N = 15SE +/- 55.43, N = 156368.743416.431. (CC) gcc options: -std=c99 -Ofast -ffast-math -fopenmp -march=native -lm

miniBUDE

Implementation: OpenMP - Input Deck: BM1

OpenBenchmarking.orgBillion Interactions/s, More Is BetterminiBUDE 20210901Implementation: OpenMP - Input Deck: BM1AVX-512 EnabledAVX-512 Disabled60120180240300SE +/- 5.97, N = 15SE +/- 2.22, N = 15254.75136.661. (CC) gcc options: -std=c99 -Ofast -ffast-math -fopenmp -march=native -lm

miniBUDE

CPU Peak Freq (Highest CPU Core Frequency) Monitor

OpenBenchmarking.orgMegahertz, More Is BetterminiBUDE 20210901CPU Peak Freq (Highest CPU Core Frequency) MonitorAVX-512 EnabledAVX-512 Disabled8001600240032004000Min: 2600 / Avg: 2984.55 / Max: 4564Min: 2600 / Avg: 3205.72 / Max: 4570

miniBUDE

CPU Power Consumption Monitor

OpenBenchmarking.orgWatts, Fewer Is BetterminiBUDE 20210901CPU Power Consumption MonitorAVX-512 EnabledAVX-512 Disabled60120180240300Min: 36.47 / Avg: 167.14 / Max: 343.87Min: 36.55 / Avg: 215.04 / Max: 347.21

miniBUDE

CPU Temperature Monitor

OpenBenchmarking.orgCelsius, Fewer Is BetterminiBUDE 20210901CPU Temperature MonitorAVX-512 EnabledAVX-512 Disabled1020304050Min: 26.38 / Avg: 41.78 / Max: 46.38Min: 28 / Avg: 43.7 / Max: 48.25

miniBUDE

Implementation: OpenMP - Input Deck: BM2

OpenBenchmarking.orgGFInst/s, More Is BetterminiBUDE 20210901Implementation: OpenMP - Input Deck: BM2AVX-512 EnabledAVX-512 Disabled15003000450060007500SE +/- 54.17, N = 3SE +/- 2.24, N = 37162.273666.091. (CC) gcc options: -std=c99 -Ofast -ffast-math -fopenmp -march=native -lm

miniBUDE

Implementation: OpenMP - Input Deck: BM2

OpenBenchmarking.orgBillion Interactions/s, More Is BetterminiBUDE 20210901Implementation: OpenMP - Input Deck: BM2AVX-512 EnabledAVX-512 Disabled60120180240300SE +/- 2.17, N = 3SE +/- 0.09, N = 3286.49146.641. (CC) gcc options: -std=c99 -Ofast -ffast-math -fopenmp -march=native -lm

miniBUDE

CPU Peak Freq (Highest CPU Core Frequency) Monitor

OpenBenchmarking.orgMegahertz, More Is BetterminiBUDE 20210901CPU Peak Freq (Highest CPU Core Frequency) MonitorAVX-512 EnabledAVX-512 Disabled8001600240032004000Min: 2600 / Avg: 3170.12 / Max: 4554Min: 2600 / Avg: 3405.75 / Max: 4550

miniBUDE

CPU Power Consumption Monitor

OpenBenchmarking.orgWatts, Fewer Is BetterminiBUDE 20210901CPU Power Consumption MonitorAVX-512 EnabledAVX-512 Disabled60120180240300Min: 36.07 / Avg: 283.87 / Max: 350.74Min: 36.57 / Avg: 308.5 / Max: 348.32

miniBUDE

CPU Temperature Monitor

OpenBenchmarking.orgCelsius, Fewer Is BetterminiBUDE 20210901CPU Temperature MonitorAVX-512 EnabledAVX-512 Disabled1122334455Min: 33 / Avg: 51.41 / Max: 57.75Min: 33.38 / Avg: 52.1 / Max: 56.88

Mobile Neural Network

Model: mobilenetV3

OpenBenchmarking.orgms, Fewer Is BetterMobile Neural Network 3.0Model: mobilenetV3AVX-512 EnabledAVX-512 Disabled0.40570.81141.21711.62282.0285SE +/- 0.010, N = 3SE +/- 0.010, N = 31.7361.803-march=znver5 - MIN: 1.59 / MAX: 2.21MIN: 1.68 / MAX: 2.881. (CXX) g++ options: -O3 -flto -std=c++11 -fvisibility=hidden -fomit-frame-pointer -fstrict-aliasing -ffunction-sections -fdata-sections -ffast-math -fno-rtti -fno-exceptions -pthread -ldl

Mobile Neural Network

Model: resnet-v2-50

OpenBenchmarking.orgms, Fewer Is BetterMobile Neural Network 3.0Model: resnet-v2-50AVX-512 EnabledAVX-512 Disabled3691215SE +/- 0.117, N = 3SE +/- 0.086, N = 37.4099.950-march=znver5 - MIN: 7.08 / MAX: 9.54MIN: 9.4 / MAX: 12.21. (CXX) g++ options: -O3 -flto -std=c++11 -fvisibility=hidden -fomit-frame-pointer -fstrict-aliasing -ffunction-sections -fdata-sections -ffast-math -fno-rtti -fno-exceptions -pthread -ldl

Mobile Neural Network

Model: SqueezeNetV1.0

OpenBenchmarking.orgms, Fewer Is BetterMobile Neural Network 3.0Model: SqueezeNetV1.0AVX-512 EnabledAVX-512 Disabled0.74861.49722.24582.99443.743SE +/- 0.011, N = 3SE +/- 0.030, N = 33.1213.327-march=znver5 - MIN: 3.05 / MAX: 5.8MIN: 3.21 / MAX: 5.61. (CXX) g++ options: -O3 -flto -std=c++11 -fvisibility=hidden -fomit-frame-pointer -fstrict-aliasing -ffunction-sections -fdata-sections -ffast-math -fno-rtti -fno-exceptions -pthread -ldl

Numpy Benchmark

OpenBenchmarking.orgScore, More Is BetterNumpy BenchmarkAVX-512 EnabledAVX-512 Disabled2004006008001000SE +/- 1.81, N = 3SE +/- 0.83, N = 3882.00828.67

Numpy Benchmark

CPU Peak Freq (Highest CPU Core Frequency) Monitor

OpenBenchmarking.orgMegahertz, More Is BetterNumpy BenchmarkCPU Peak Freq (Highest CPU Core Frequency) MonitorAVX-512 EnabledAVX-512 Disabled8001600240032004000Min: 2600 / Avg: 4481.19 / Max: 4567Min: 2600 / Avg: 4473.89 / Max: 4563

Numpy Benchmark

CPU Power Consumption Monitor

OpenBenchmarking.orgWatts, Fewer Is BetterNumpy BenchmarkCPU Power Consumption MonitorAVX-512 EnabledAVX-512 Disabled20406080100Min: 0.54 / Avg: 79.85 / Max: 87.65Min: 1.18 / Avg: 80.05 / Max: 87.29

Numpy Benchmark

CPU Temperature Monitor

OpenBenchmarking.orgCelsius, Fewer Is BetterNumpy BenchmarkCPU Temperature MonitorAVX-512 EnabledAVX-512 Disabled1122334455Min: 39.75 / Avg: 52.09 / Max: 57.5Min: 40.75 / Avg: 52.43 / Max: 57.5

oneDNN

Harness: Deconvolution Batch shapes_1d - Engine: CPU

OpenBenchmarking.orgms, Fewer Is BetteroneDNN 3.6Harness: Deconvolution Batch shapes_1d - Engine: CPUAVX-512 EnabledAVX-512 Disabled3691215SE +/- 0.01312, N = 3SE +/- 0.00410, N = 36.7246612.86010MIN: 4.2-mno-avx512f - MIN: 9.81. (CXX) g++ options: -O3 -march=native -march=znver5 -flto -fopenmp -msse4.1 -fPIC -fcf-protection=full -pie -ldl

oneDNN

CPU Peak Freq (Highest CPU Core Frequency) Monitor

OpenBenchmarking.orgMegahertz, More Is BetteroneDNN 3.6CPU Peak Freq (Highest CPU Core Frequency) MonitorAVX-512 EnabledAVX-512 Disabled8001600240032004000Min: 2583 / Avg: 3215.51 / Max: 4567Min: 2600 / Avg: 3408.08 / Max: 4620

oneDNN

CPU Power Consumption Monitor

OpenBenchmarking.orgWatts, Fewer Is BetteroneDNN 3.6CPU Power Consumption MonitorAVX-512 EnabledAVX-512 Disabled50100150200250Min: 1.53 / Avg: 183.78 / Max: 258.76Min: 7.6 / Avg: 200.15 / Max: 283.44

oneDNN

CPU Temperature Monitor

OpenBenchmarking.orgCelsius, Fewer Is BetteroneDNN 3.6CPU Temperature MonitorAVX-512 EnabledAVX-512 Disabled918273645Min: 33.38 / Avg: 40.12 / Max: 42.88Min: 40.13 / Avg: 43.41 / Max: 46

oneDNN

Harness: Deconvolution Batch shapes_3d - Engine: CPU

OpenBenchmarking.orgms, Fewer Is BetteroneDNN 3.6Harness: Deconvolution Batch shapes_3d - Engine: CPUAVX-512 EnabledAVX-512 Disabled0.30470.60940.91411.21881.5235SE +/- 0.001216, N = 9SE +/- 0.001116, N = 90.7169791.354390MIN: 0.62-mno-avx512f - MIN: 1.311. (CXX) g++ options: -O3 -march=native -march=znver5 -flto -fopenmp -msse4.1 -fPIC -fcf-protection=full -pie -ldl

oneDNN

CPU Peak Freq (Highest CPU Core Frequency) Monitor

OpenBenchmarking.orgMegahertz, More Is BetteroneDNN 3.6CPU Peak Freq (Highest CPU Core Frequency) MonitorAVX-512 EnabledAVX-512 Disabled8001600240032004000Min: 2448 / Avg: 2613.84 / Max: 4541Min: 2600 / Avg: 3065.11 / Max: 4539

oneDNN

CPU Power Consumption Monitor

OpenBenchmarking.orgWatts, Fewer Is BetteroneDNN 3.6CPU Power Consumption MonitorAVX-512 EnabledAVX-512 Disabled50100150200250Min: 0.2 / Avg: 112.35 / Max: 220.99Min: 11.96 / Avg: 136.6 / Max: 273.71

oneDNN

CPU Temperature Monitor

OpenBenchmarking.orgCelsius, Fewer Is BetteroneDNN 3.6CPU Temperature MonitorAVX-512 EnabledAVX-512 Disabled1020304050Min: 34.13 / Avg: 38.62 / Max: 40.38Min: 35.13 / Avg: 44.03 / Max: 47.38

oneDNN

Harness: IP Shapes 1D - Engine: CPU

OpenBenchmarking.orgms, Fewer Is BetteroneDNN 3.6Harness: IP Shapes 1D - Engine: CPUAVX-512 EnabledAVX-512 Disabled0.14540.29080.43620.58160.727SE +/- 0.001169, N = 4SE +/- 0.000257, N = 40.5334140.646211MIN: 0.49-mno-avx512f - MIN: 0.591. (CXX) g++ options: -O3 -march=native -march=znver5 -flto -fopenmp -msse4.1 -fPIC -fcf-protection=full -pie -ldl

oneDNN

CPU Peak Freq (Highest CPU Core Frequency) Monitor

OpenBenchmarking.orgMegahertz, More Is BetteroneDNN 3.6CPU Peak Freq (Highest CPU Core Frequency) MonitorAVX-512 EnabledAVX-512 Disabled8001600240032004000Min: 2374 / Avg: 3087.78 / Max: 4541Min: 2557 / Avg: 3100.03 / Max: 4539

oneDNN

CPU Power Consumption Monitor

OpenBenchmarking.orgWatts, Fewer Is BetteroneDNN 3.6CPU Power Consumption MonitorAVX-512 EnabledAVX-512 Disabled50100150200250Min: 0.41 / Avg: 152.19 / Max: 222.77Min: 14.09 / Avg: 163.35 / Max: 251.52

oneDNN

CPU Temperature Monitor

OpenBenchmarking.orgCelsius, Fewer Is BetteroneDNN 3.6CPU Temperature MonitorAVX-512 EnabledAVX-512 Disabled918273645Min: 32.5 / Avg: 37.99 / Max: 40.25Min: 34 / Avg: 39.16 / Max: 42.25

oneDNN

Harness: IP Shapes 3D - Engine: CPU

OpenBenchmarking.orgms, Fewer Is BetteroneDNN 3.6Harness: IP Shapes 3D - Engine: CPUAVX-512 EnabledAVX-512 Disabled0.6921.3842.0762.7683.46SE +/- 0.000384, N = 5SE +/- 0.002544, N = 50.2659273.075540-mno-avx512f - MIN: 3.021. (CXX) g++ options: -O3 -march=native -march=znver5 -flto -fopenmp -msse4.1 -fPIC -fcf-protection=full -pie -ldl

oneDNN

CPU Peak Freq (Highest CPU Core Frequency) Monitor

OpenBenchmarking.orgMegahertz, More Is BetteroneDNN 3.6CPU Peak Freq (Highest CPU Core Frequency) MonitorAVX-512 EnabledAVX-512 Disabled8001600240032004000Min: 2539 / Avg: 3522.54 / Max: 4553Min: 2600 / Avg: 3798.58 / Max: 4549

oneDNN

CPU Power Consumption Monitor

OpenBenchmarking.orgWatts, Fewer Is BetteroneDNN 3.6CPU Power Consumption MonitorAVX-512 EnabledAVX-512 Disabled50100150200250Min: 0.74 / Avg: 168.39 / Max: 247.9Min: 0.69 / Avg: 138.48 / Max: 275.92

oneDNN

CPU Temperature Monitor

OpenBenchmarking.orgCelsius, Fewer Is BetteroneDNN 3.6CPU Temperature MonitorAVX-512 EnabledAVX-512 Disabled1020304050Min: 32.88 / Avg: 40.62 / Max: 44.5Min: 32.38 / Avg: 43.48 / Max: 49.88

oneDNN

Harness: Recurrent Neural Network Training - Engine: CPU

OpenBenchmarking.orgms, Fewer Is BetteroneDNN 3.6Harness: Recurrent Neural Network Training - Engine: CPUAVX-512 EnabledAVX-512 Disabled110220330440550SE +/- 0.35, N = 3SE +/- 0.56, N = 3428.50491.69MIN: 421.37-mno-avx512f - MIN: 486.041. (CXX) g++ options: -O3 -march=native -march=znver5 -flto -fopenmp -msse4.1 -fPIC -fcf-protection=full -pie -ldl

oneDNN

CPU Peak Freq (Highest CPU Core Frequency) Monitor

OpenBenchmarking.orgMegahertz, More Is BetteroneDNN 3.6CPU Peak Freq (Highest CPU Core Frequency) MonitorAVX-512 EnabledAVX-512 Disabled8001600240032004000Min: 1783 / Avg: 3106.58 / Max: 4569Min: 2299 / Avg: 3036.42 / Max: 4549

oneDNN

CPU Power Consumption Monitor

OpenBenchmarking.orgWatts, Fewer Is BetteroneDNN 3.6CPU Power Consumption MonitorAVX-512 EnabledAVX-512 Disabled60120180240300Min: 1.05 / Avg: 209.55 / Max: 281.38Min: 3.06 / Avg: 245.86 / Max: 314.05

oneDNN

CPU Temperature Monitor

OpenBenchmarking.orgCelsius, Fewer Is BetteroneDNN 3.6CPU Temperature MonitorAVX-512 EnabledAVX-512 Disabled1020304050Min: 34.25 / Avg: 45.28 / Max: 47.5Min: 33.38 / Avg: 43.76 / Max: 46.88

oneDNN

Harness: Recurrent Neural Network Inference - Engine: CPU

OpenBenchmarking.orgms, Fewer Is BetteroneDNN 3.6Harness: Recurrent Neural Network Inference - Engine: CPUAVX-512 EnabledAVX-512 Disabled70140210280350SE +/- 0.34, N = 3SE +/- 0.47, N = 3278.31307.05MIN: 271.12-mno-avx512f - MIN: 301.261. (CXX) g++ options: -O3 -march=native -march=znver5 -flto -fopenmp -msse4.1 -fPIC -fcf-protection=full -pie -ldl

oneDNN

CPU Peak Freq (Highest CPU Core Frequency) Monitor

OpenBenchmarking.orgMegahertz, More Is BetteroneDNN 3.6CPU Peak Freq (Highest CPU Core Frequency) MonitorAVX-512 EnabledAVX-512 Disabled8001600240032004000Min: 1575 / Avg: 3008.31 / Max: 4572Min: 1753 / Avg: 3103.6 / Max: 4545

oneDNN

CPU Power Consumption Monitor

OpenBenchmarking.orgWatts, Fewer Is BetteroneDNN 3.6CPU Power Consumption MonitorAVX-512 EnabledAVX-512 Disabled60120180240300Min: 0.63 / Avg: 202.71 / Max: 294.11Min: 1.24 / Avg: 223.45 / Max: 310.66

oneDNN

CPU Temperature Monitor

OpenBenchmarking.orgCelsius, Fewer Is BetteroneDNN 3.6CPU Temperature MonitorAVX-512 EnabledAVX-512 Disabled918273645Min: 36.5 / Avg: 44.46 / Max: 46.38Min: 36.5 / Avg: 43.53 / Max: 46.38

ONNX Runtime

Model: fcn-resnet101-11 - Device: CPU - Executor: Standard

OpenBenchmarking.orgInferences Per Second, More Is BetterONNX Runtime 1.19Model: fcn-resnet101-11 - Device: CPU - Executor: StandardAVX-512 EnabledAVX-512 Disabled3691215SE +/- 0.13137, N = 15SE +/- 0.04636, N = 39.564336.586031. (CXX) g++ options: -O3 -march=native -march=znver5 -flto -ffunction-sections -fdata-sections -mtune=native -flto=auto -fno-fat-lto-objects -ldl -lrt

ONNX Runtime

Model: fcn-resnet101-11 - Device: CPU - Executor: Standard

OpenBenchmarking.orgInference Time Cost (ms), Fewer Is BetterONNX Runtime 1.19Model: fcn-resnet101-11 - Device: CPU - Executor: StandardAVX-512 EnabledAVX-512 Disabled306090120150SE +/- 1.43, N = 15SE +/- 1.07, N = 3104.83151.851. (CXX) g++ options: -O3 -march=native -march=znver5 -flto -ffunction-sections -fdata-sections -mtune=native -flto=auto -fno-fat-lto-objects -ldl -lrt

ONNX Runtime

CPU Peak Freq (Highest CPU Core Frequency) Monitor

OpenBenchmarking.orgMegahertz, More Is BetterONNX Runtime 1.19CPU Peak Freq (Highest CPU Core Frequency) MonitorAVX-512 EnabledAVX-512 Disabled8001600240032004000Min: 2600 / Avg: 2988 / Max: 4545Min: 2600 / Avg: 3268.36 / Max: 4548

ONNX Runtime

CPU Power Consumption Monitor

OpenBenchmarking.orgWatts, Fewer Is BetterONNX Runtime 1.19CPU Power Consumption MonitorAVX-512 EnabledAVX-512 Disabled50100150200250Min: 0.88 / Avg: 231.28 / Max: 253.21Min: 1.64 / Avg: 246.05 / Max: 287.32

ONNX Runtime

CPU Temperature Monitor

OpenBenchmarking.orgCelsius, Fewer Is BetterONNX Runtime 1.19CPU Temperature MonitorAVX-512 EnabledAVX-512 Disabled1020304050Min: 41.38 / Avg: 46.81 / Max: 48.38Min: 37.63 / Avg: 46.6 / Max: 49.13

ONNX Runtime

Model: super-resolution-10 - Device: CPU - Executor: Standard

OpenBenchmarking.orgInferences Per Second, More Is BetterONNX Runtime 1.19Model: super-resolution-10 - Device: CPU - Executor: StandardAVX-512 EnabledAVX-512 Disabled50100150200250SE +/- 0.59, N = 3SE +/- 1.92, N = 3215.16170.951. (CXX) g++ options: -O3 -march=native -march=znver5 -flto -ffunction-sections -fdata-sections -mtune=native -flto=auto -fno-fat-lto-objects -ldl -lrt

ONNX Runtime

Model: super-resolution-10 - Device: CPU - Executor: Standard

OpenBenchmarking.orgInference Time Cost (ms), Fewer Is BetterONNX Runtime 1.19Model: super-resolution-10 - Device: CPU - Executor: StandardAVX-512 EnabledAVX-512 Disabled1.31652.6333.94955.2666.5825SE +/- 0.01281, N = 3SE +/- 0.06614, N = 34.647505.850961. (CXX) g++ options: -O3 -march=native -march=znver5 -flto -ffunction-sections -fdata-sections -mtune=native -flto=auto -fno-fat-lto-objects -ldl -lrt

ONNX Runtime

CPU Peak Freq (Highest CPU Core Frequency) Monitor

OpenBenchmarking.orgMegahertz, More Is BetterONNX Runtime 1.19CPU Peak Freq (Highest CPU Core Frequency) MonitorAVX-512 EnabledAVX-512 Disabled8001600240032004000Min: 2600 / Avg: 4421.13 / Max: 4559Min: 2600 / Avg: 4425.08 / Max: 4547

ONNX Runtime

CPU Power Consumption Monitor

OpenBenchmarking.orgWatts, Fewer Is BetterONNX Runtime 1.19CPU Power Consumption MonitorAVX-512 EnabledAVX-512 Disabled4080120160200Min: 0.02 / Avg: 190.61 / Max: 202.75Min: 1.15 / Avg: 191.45 / Max: 204.65

ONNX Runtime

CPU Temperature Monitor

OpenBenchmarking.orgCelsius, Fewer Is BetterONNX Runtime 1.19CPU Temperature MonitorAVX-512 EnabledAVX-512 Disabled1326395265Min: 38.5 / Avg: 58.83 / Max: 60.25Min: 37.5 / Avg: 59.48 / Max: 65.88

ONNX Runtime

Model: bertsquad-12 - Device: CPU - Executor: Standard

OpenBenchmarking.orgInferences Per Second, More Is BetterONNX Runtime 1.19Model: bertsquad-12 - Device: CPU - Executor: StandardAVX-512 EnabledAVX-512 Disabled510152025SE +/- 0.19, N = 3SE +/- 0.06, N = 322.4317.761. (CXX) g++ options: -O3 -march=native -march=znver5 -flto -ffunction-sections -fdata-sections -mtune=native -flto=auto -fno-fat-lto-objects -ldl -lrt

ONNX Runtime

Model: bertsquad-12 - Device: CPU - Executor: Standard

OpenBenchmarking.orgInference Time Cost (ms), Fewer Is BetterONNX Runtime 1.19Model: bertsquad-12 - Device: CPU - Executor: StandardAVX-512 EnabledAVX-512 Disabled1326395265SE +/- 0.38, N = 3SE +/- 0.18, N = 344.5856.311. (CXX) g++ options: -O3 -march=native -march=znver5 -flto -ffunction-sections -fdata-sections -mtune=native -flto=auto -fno-fat-lto-objects -ldl -lrt

ONNX Runtime

CPU Peak Freq (Highest CPU Core Frequency) Monitor

OpenBenchmarking.orgMegahertz, More Is BetterONNX Runtime 1.19CPU Peak Freq (Highest CPU Core Frequency) MonitorAVX-512 EnabledAVX-512 Disabled8001600240032004000Min: 2600 / Avg: 4001.58 / Max: 4561Min: 2600 / Avg: 4080.57 / Max: 4569

ONNX Runtime

CPU Power Consumption Monitor

OpenBenchmarking.orgWatts, Fewer Is BetterONNX Runtime 1.19CPU Power Consumption MonitorAVX-512 EnabledAVX-512 Disabled50100150200250Min: 0.84 / Avg: 240.74 / Max: 259.52Min: 0.36 / Avg: 256.73 / Max: 276.98

ONNX Runtime

CPU Temperature Monitor

OpenBenchmarking.orgCelsius, Fewer Is BetterONNX Runtime 1.19CPU Temperature MonitorAVX-512 EnabledAVX-512 Disabled1122334455Min: 39.38 / Avg: 49.58 / Max: 51.13Min: 38.38 / Avg: 50.61 / Max: 52.5

ONNX Runtime

Model: GPT-2 - Device: CPU - Executor: Standard

OpenBenchmarking.orgInferences Per Second, More Is BetterONNX Runtime 1.19Model: GPT-2 - Device: CPU - Executor: StandardAVX-512 EnabledAVX-512 Disabled4080120160200SE +/- 0.28, N = 3SE +/- 0.35, N = 3196.35182.811. (CXX) g++ options: -O3 -march=native -march=znver5 -flto -ffunction-sections -fdata-sections -mtune=native -flto=auto -fno-fat-lto-objects -ldl -lrt

ONNX Runtime

Model: GPT-2 - Device: CPU - Executor: Standard

OpenBenchmarking.orgInference Time Cost (ms), Fewer Is BetterONNX Runtime 1.19Model: GPT-2 - Device: CPU - Executor: StandardAVX-512 EnabledAVX-512 Disabled1.23032.46063.69094.92126.1515SE +/- 0.00735, N = 3SE +/- 0.01048, N = 35.091085.468201. (CXX) g++ options: -O3 -march=native -march=znver5 -flto -ffunction-sections -fdata-sections -mtune=native -flto=auto -fno-fat-lto-objects -ldl -lrt

ONNX Runtime

CPU Peak Freq (Highest CPU Core Frequency) Monitor

OpenBenchmarking.orgMegahertz, More Is BetterONNX Runtime 1.19CPU Peak Freq (Highest CPU Core Frequency) MonitorAVX-512 EnabledAVX-512 Disabled8001600240032004000Min: 2600 / Avg: 4416.61 / Max: 4569Min: 2600 / Avg: 4426.17 / Max: 4546

ONNX Runtime

CPU Power Consumption Monitor

OpenBenchmarking.orgWatts, Fewer Is BetterONNX Runtime 1.19CPU Power Consumption MonitorAVX-512 EnabledAVX-512 Disabled306090120150Min: 0.44 / Avg: 146.99 / Max: 156.52Min: 36.49 / Avg: 147.92 / Max: 156.91

ONNX Runtime

CPU Temperature Monitor

OpenBenchmarking.orgCelsius, Fewer Is BetterONNX Runtime 1.19CPU Temperature MonitorAVX-512 EnabledAVX-512 Disabled918273645Min: 38.88 / Avg: 44.72 / Max: 46Min: 38.5 / Avg: 44.07 / Max: 45.5

ONNX Runtime

Model: ArcFace ResNet-100 - Device: CPU - Executor: Standard

OpenBenchmarking.orgInferences Per Second, More Is BetterONNX Runtime 1.19Model: ArcFace ResNet-100 - Device: CPU - Executor: StandardAVX-512 EnabledAVX-512 Disabled1122334455SE +/- 0.53, N = 3SE +/- 0.36, N = 848.7840.551. (CXX) g++ options: -O3 -march=native -march=znver5 -flto -ffunction-sections -fdata-sections -mtune=native -flto=auto -fno-fat-lto-objects -ldl -lrt

ONNX Runtime

Model: ArcFace ResNet-100 - Device: CPU - Executor: Standard

OpenBenchmarking.orgInference Time Cost (ms), Fewer Is BetterONNX Runtime 1.19Model: ArcFace ResNet-100 - Device: CPU - Executor: StandardAVX-512 EnabledAVX-512 Disabled612182430SE +/- 0.22, N = 3SE +/- 0.22, N = 820.5024.671. (CXX) g++ options: -O3 -march=native -march=znver5 -flto -ffunction-sections -fdata-sections -mtune=native -flto=auto -fno-fat-lto-objects -ldl -lrt

ONNX Runtime

CPU Peak Freq (Highest CPU Core Frequency) Monitor

OpenBenchmarking.orgMegahertz, More Is BetterONNX Runtime 1.19CPU Peak Freq (Highest CPU Core Frequency) MonitorAVX-512 EnabledAVX-512 Disabled8001600240032004000Min: 2600 / Avg: 3936.11 / Max: 4564Min: 2600 / Avg: 4096.82 / Max: 4554

ONNX Runtime

CPU Power Consumption Monitor

OpenBenchmarking.orgWatts, Fewer Is BetterONNX Runtime 1.19CPU Power Consumption MonitorAVX-512 EnabledAVX-512 Disabled60120180240300Min: 1.07 / Avg: 259.86 / Max: 281.69Min: 0.74 / Avg: 291.22 / Max: 317.17

ONNX Runtime

CPU Temperature Monitor

OpenBenchmarking.orgCelsius, Fewer Is BetterONNX Runtime 1.19CPU Temperature MonitorAVX-512 EnabledAVX-512 Disabled1020304050Min: 34.88 / Avg: 45.81 / Max: 47.63Min: 33.88 / Avg: 48.88 / Max: 52.25

ONNX Runtime

Model: Faster R-CNN R-50-FPN-int8 - Device: CPU - Executor: Standard

OpenBenchmarking.orgInferences Per Second, More Is BetterONNX Runtime 1.19Model: Faster R-CNN R-50-FPN-int8 - Device: CPU - Executor: StandardAVX-512 EnabledAVX-512 Disabled1428425670SE +/- 0.50, N = 3SE +/- 0.45, N = 1564.0737.151. (CXX) g++ options: -O3 -march=native -march=znver5 -flto -ffunction-sections -fdata-sections -mtune=native -flto=auto -fno-fat-lto-objects -ldl -lrt

ONNX Runtime

Model: Faster R-CNN R-50-FPN-int8 - Device: CPU - Executor: Standard

OpenBenchmarking.orgInference Time Cost (ms), Fewer Is BetterONNX Runtime 1.19Model: Faster R-CNN R-50-FPN-int8 - Device: CPU - Executor: StandardAVX-512 EnabledAVX-512 Disabled612182430SE +/- 0.12, N = 3SE +/- 0.33, N = 1515.6126.971. (CXX) g++ options: -O3 -march=native -march=znver5 -flto -ffunction-sections -fdata-sections -mtune=native -flto=auto -fno-fat-lto-objects -ldl -lrt

ONNX Runtime

CPU Peak Freq (Highest CPU Core Frequency) Monitor

OpenBenchmarking.orgMegahertz, More Is BetterONNX Runtime 1.19CPU Peak Freq (Highest CPU Core Frequency) MonitorAVX-512 EnabledAVX-512 Disabled8001600240032004000Min: 2600 / Avg: 4262.82 / Max: 4568Min: 2600 / Avg: 4278.95 / Max: 4568

ONNX Runtime

CPU Power Consumption Monitor

OpenBenchmarking.orgWatts, Fewer Is BetterONNX Runtime 1.19CPU Power Consumption MonitorAVX-512 EnabledAVX-512 Disabled50100150200250Min: 0.4 / Avg: 268.32 / Max: 289.36Min: 0.9 / Avg: 239.2 / Max: 267.87

ONNX Runtime

CPU Temperature Monitor

OpenBenchmarking.orgCelsius, Fewer Is BetterONNX Runtime 1.19CPU Temperature MonitorAVX-512 EnabledAVX-512 Disabled1224364860Min: 39.25 / Avg: 58.41 / Max: 62.75Min: 37 / Avg: 55.24 / Max: 57.63

ONNX Runtime

Model: T5 Encoder - Device: CPU - Executor: Standard

OpenBenchmarking.orgInferences Per Second, More Is BetterONNX Runtime 1.19Model: T5 Encoder - Device: CPU - Executor: StandardAVX-512 EnabledAVX-512 Disabled50100150200250SE +/- 1.84, N = 3SE +/- 0.75, N = 3244.99238.641. (CXX) g++ options: -O3 -march=native -march=znver5 -flto -ffunction-sections -fdata-sections -mtune=native -flto=auto -fno-fat-lto-objects -ldl -lrt

ONNX Runtime

Model: T5 Encoder - Device: CPU - Executor: Standard

OpenBenchmarking.orgInference Time Cost (ms), Fewer Is BetterONNX Runtime 1.19Model: T5 Encoder - Device: CPU - Executor: StandardAVX-512 EnabledAVX-512 Disabled0.94271.88542.82813.77084.7135SE +/- 0.03056, N = 3SE +/- 0.01312, N = 34.081144.189721. (CXX) g++ options: -O3 -march=native -march=znver5 -flto -ffunction-sections -fdata-sections -mtune=native -flto=auto -fno-fat-lto-objects -ldl -lrt

ONNX Runtime

CPU Peak Freq (Highest CPU Core Frequency) Monitor

OpenBenchmarking.orgMegahertz, More Is BetterONNX Runtime 1.19CPU Peak Freq (Highest CPU Core Frequency) MonitorAVX-512 EnabledAVX-512 Disabled8001600240032004000Min: 2600 / Avg: 4416 / Max: 4542Min: 2600 / Avg: 4427.57 / Max: 4563

ONNX Runtime

CPU Power Consumption Monitor

OpenBenchmarking.orgWatts, Fewer Is BetterONNX Runtime 1.19CPU Power Consumption MonitorAVX-512 EnabledAVX-512 Disabled306090120150Min: 0.13 / Avg: 140.78 / Max: 152.77Min: 0.11 / Avg: 138.67 / Max: 148.1

ONNX Runtime

CPU Temperature Monitor

OpenBenchmarking.orgCelsius, Fewer Is BetterONNX Runtime 1.19CPU Temperature MonitorAVX-512 EnabledAVX-512 Disabled918273645Min: 41 / Avg: 44.03 / Max: 46Min: 38.75 / Avg: 42.4 / Max: 43.63

ONNX Runtime

Model: ResNet101_DUC_HDC-12 - Device: CPU - Executor: Standard

OpenBenchmarking.orgInferences Per Second, More Is BetterONNX Runtime 1.19Model: ResNet101_DUC_HDC-12 - Device: CPU - Executor: StandardAVX-512 EnabledAVX-512 Disabled246810SE +/- 0.06862, N = 4SE +/- 0.15591, N = 126.507954.309141. (CXX) g++ options: -O3 -march=native -march=znver5 -flto -ffunction-sections -fdata-sections -mtune=native -flto=auto -fno-fat-lto-objects -ldl -lrt

ONNX Runtime

Model: ResNet101_DUC_HDC-12 - Device: CPU - Executor: Standard

OpenBenchmarking.orgInference Time Cost (ms), Fewer Is BetterONNX Runtime 1.19Model: ResNet101_DUC_HDC-12 - Device: CPU - Executor: StandardAVX-512 EnabledAVX-512 Disabled50100150200250SE +/- 1.61, N = 4SE +/- 9.18, N = 12153.71235.711. (CXX) g++ options: -O3 -march=native -march=znver5 -flto -ffunction-sections -fdata-sections -mtune=native -flto=auto -fno-fat-lto-objects -ldl -lrt

ONNX Runtime

CPU Peak Freq (Highest CPU Core Frequency) Monitor

OpenBenchmarking.orgMegahertz, More Is BetterONNX Runtime 1.19CPU Peak Freq (Highest CPU Core Frequency) MonitorAVX-512 EnabledAVX-512 Disabled8001600240032004000Min: 2600 / Avg: 2916.93 / Max: 4543Min: 2600 / Avg: 3239.62 / Max: 4575

ONNX Runtime

CPU Power Consumption Monitor

OpenBenchmarking.orgWatts, Fewer Is BetterONNX Runtime 1.19CPU Power Consumption MonitorAVX-512 EnabledAVX-512 Disabled60120180240300Min: 0.49 / Avg: 258.72 / Max: 282.16Min: 0.55 / Avg: 286.1 / Max: 324.37

ONNX Runtime

CPU Temperature Monitor

OpenBenchmarking.orgCelsius, Fewer Is BetterONNX Runtime 1.19CPU Temperature MonitorAVX-512 EnabledAVX-512 Disabled1122334455Min: 39.75 / Avg: 48.18 / Max: 49.63Min: 39.5 / Avg: 51.21 / Max: 53.25

OpenVINO

Model: Face Detection FP16-INT8 - Device: CPU

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2024.5Model: Face Detection FP16-INT8 - Device: CPUAVX-512 EnabledAVX-512 Disabled306090120150SE +/- 0.11, N = 3SE +/- 0.04, N = 3147.0685.54-march=znver51. (CXX) g++ options: -fPIC -O3 -flto -fsigned-char -ffunction-sections -fdata-sections -fno-strict-overflow -fwrapv -shared -ldl -lstdc++fs

OpenVINO

Model: Face Detection FP16-INT8 - Device: CPU

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2024.5Model: Face Detection FP16-INT8 - Device: CPUAVX-512 EnabledAVX-512 Disabled120240360480600SE +/- 0.27, N = 3SE +/- 0.19, N = 3325.74558.74-march=znver5 - MIN: 255.71 / MAX: 363.13MIN: 273.34 / MAX: 587.281. (CXX) g++ options: -fPIC -O3 -flto -fsigned-char -ffunction-sections -fdata-sections -fno-strict-overflow -fwrapv -shared -ldl -lstdc++fs

OpenVINO

CPU Peak Freq (Highest CPU Core Frequency) Monitor

OpenBenchmarking.orgMegahertz, More Is BetterOpenVINO 2024.5CPU Peak Freq (Highest CPU Core Frequency) MonitorAVX-512 EnabledAVX-512 Disabled8001600240032004000Min: 2239 / Avg: 2473.08 / Max: 4567Min: 2574 / Avg: 2722.94 / Max: 4564

OpenVINO

CPU Power Consumption Monitor

OpenBenchmarking.orgWatts, Fewer Is BetterOpenVINO 2024.5CPU Power Consumption MonitorAVX-512 EnabledAVX-512 Disabled60120180240300Min: 1.25 / Avg: 283.24 / Max: 311.62Min: 1.61 / Avg: 295.57 / Max: 326.2

OpenVINO

CPU Temperature Monitor

OpenBenchmarking.orgCelsius, Fewer Is BetterOpenVINO 2024.5CPU Temperature MonitorAVX-512 EnabledAVX-512 Disabled1122334455Min: 42.5 / Avg: 52.78 / Max: 54.38Min: 40.38 / Avg: 52.66 / Max: 55.88

OpenVINO

Model: Age Gender Recognition Retail 0013 FP16-INT8 - Device: CPU

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2024.5Model: Age Gender Recognition Retail 0013 FP16-INT8 - Device: CPUAVX-512 EnabledAVX-512 Disabled30K60K90K120K150KSE +/- 180.45, N = 3SE +/- 170.89, N = 3162706.14129762.91-march=znver51. (CXX) g++ options: -fPIC -O3 -flto -fsigned-char -ffunction-sections -fdata-sections -fno-strict-overflow -fwrapv -shared -ldl -lstdc++fs

OpenVINO

Model: Age Gender Recognition Retail 0013 FP16-INT8 - Device: CPU

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2024.5Model: Age Gender Recognition Retail 0013 FP16-INT8 - Device: CPUAVX-512 EnabledAVX-512 Disabled0.11930.23860.35790.47720.5965SE +/- 0.00, N = 3SE +/- 0.00, N = 30.310.53-march=znver5 - MIN: 0.12 / MAX: 25.73MIN: 0.18 / MAX: 13.441. (CXX) g++ options: -fPIC -O3 -flto -fsigned-char -ffunction-sections -fdata-sections -fno-strict-overflow -fwrapv -shared -ldl -lstdc++fs

OpenVINO

CPU Peak Freq (Highest CPU Core Frequency) Monitor

OpenBenchmarking.orgMegahertz, More Is BetterOpenVINO 2024.5CPU Peak Freq (Highest CPU Core Frequency) MonitorAVX-512 EnabledAVX-512 Disabled8001600240032004000Min: 2600 / Avg: 2876.42 / Max: 4546Min: 2600 / Avg: 2860.53 / Max: 4567

OpenVINO

CPU Power Consumption Monitor

OpenBenchmarking.orgWatts, Fewer Is BetterOpenVINO 2024.5CPU Power Consumption MonitorAVX-512 EnabledAVX-512 Disabled50100150200250Min: 1.05 / Avg: 246.58 / Max: 270.54Min: 1.45 / Avg: 271.96 / Max: 293.6

OpenVINO

CPU Temperature Monitor

OpenBenchmarking.orgCelsius, Fewer Is BetterOpenVINO 2024.5CPU Temperature MonitorAVX-512 EnabledAVX-512 Disabled1020304050Min: 42.75 / Avg: 48.39 / Max: 50.5Min: 41.75 / Avg: 49.3 / Max: 51

OpenVINO

Model: Person Detection FP16 - Device: CPU

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2024.5Model: Person Detection FP16 - Device: CPUAVX-512 EnabledAVX-512 Disabled150300450600750SE +/- 0.91, N = 3SE +/- 0.82, N = 3710.04241.77-march=znver51. (CXX) g++ options: -fPIC -O3 -flto -fsigned-char -ffunction-sections -fdata-sections -fno-strict-overflow -fwrapv -shared -ldl -lstdc++fs

OpenVINO

Model: Person Detection FP16 - Device: CPU

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2024.5Model: Person Detection FP16 - Device: CPUAVX-512 EnabledAVX-512 Disabled4080120160200SE +/- 0.09, N = 3SE +/- 0.68, N = 367.47198.17-march=znver5 - MIN: 29.95 / MAX: 185.48MIN: 94.52 / MAX: 339.841. (CXX) g++ options: -fPIC -O3 -flto -fsigned-char -ffunction-sections -fdata-sections -fno-strict-overflow -fwrapv -shared -ldl -lstdc++fs

OpenVINO

CPU Peak Freq (Highest CPU Core Frequency) Monitor

OpenBenchmarking.orgMegahertz, More Is BetterOpenVINO 2024.5CPU Peak Freq (Highest CPU Core Frequency) MonitorAVX-512 EnabledAVX-512 Disabled8001600240032004000Min: 2394 / Avg: 2594.63 / Max: 4554Min: 2595 / Avg: 2827.98 / Max: 4555

OpenVINO

CPU Power Consumption Monitor

OpenBenchmarking.orgWatts, Fewer Is BetterOpenVINO 2024.5CPU Power Consumption MonitorAVX-512 EnabledAVX-512 Disabled60120180240300Min: 0.62 / Avg: 326.41 / Max: 360.67Min: 1.07 / Avg: 320.76 / Max: 358.34

OpenVINO

CPU Temperature Monitor

OpenBenchmarking.orgCelsius, Fewer Is BetterOpenVINO 2024.5CPU Temperature MonitorAVX-512 EnabledAVX-512 Disabled1122334455Min: 40.5 / Avg: 54.34 / Max: 57.13Min: 40 / Avg: 53.07 / Max: 55.25

OpenVINO

Model: Weld Porosity Detection FP16 - Device: CPU

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2024.5Model: Weld Porosity Detection FP16 - Device: CPUAVX-512 EnabledAVX-512 Disabled16003200480064008000SE +/- 5.62, N = 3SE +/- 1.32, N = 37611.172189.06-march=znver51. (CXX) g++ options: -fPIC -O3 -flto -fsigned-char -ffunction-sections -fdata-sections -fno-strict-overflow -fwrapv -shared -ldl -lstdc++fs

OpenVINO

Model: Weld Porosity Detection FP16 - Device: CPU

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2024.5Model: Weld Porosity Detection FP16 - Device: CPUAVX-512 EnabledAVX-512 Disabled1020304050SE +/- 0.01, N = 3SE +/- 0.03, N = 312.5343.73-march=znver5 - MIN: 5.9 / MAX: 31.1MIN: 23.02 / MAX: 62.811. (CXX) g++ options: -fPIC -O3 -flto -fsigned-char -ffunction-sections -fdata-sections -fno-strict-overflow -fwrapv -shared -ldl -lstdc++fs

OpenVINO

CPU Peak Freq (Highest CPU Core Frequency) Monitor

OpenBenchmarking.orgMegahertz, More Is BetterOpenVINO 2024.5CPU Peak Freq (Highest CPU Core Frequency) MonitorAVX-512 EnabledAVX-512 Disabled8001600240032004000Min: 2270 / Avg: 2390.53 / Max: 4551Min: 2470 / Avg: 2551.48 / Max: 4563

OpenVINO

CPU Power Consumption Monitor

OpenBenchmarking.orgWatts, Fewer Is BetterOpenVINO 2024.5CPU Power Consumption MonitorAVX-512 EnabledAVX-512 Disabled60120180240300Min: 0.62 / Avg: 316.71 / Max: 341.63Min: 1.05 / Avg: 304.33 / Max: 329.28

OpenVINO

CPU Temperature Monitor

OpenBenchmarking.orgCelsius, Fewer Is BetterOpenVINO 2024.5CPU Temperature MonitorAVX-512 EnabledAVX-512 Disabled1122334455Min: 43.88 / Avg: 55.64 / Max: 57.25Min: 42.25 / Avg: 52.62 / Max: 53.88

OpenVINO

Model: Person Vehicle Bike Detection FP16 - Device: CPU

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2024.5Model: Person Vehicle Bike Detection FP16 - Device: CPUAVX-512 EnabledAVX-512 Disabled15003000450060007500SE +/- 17.52, N = 3SE +/- 2.95, N = 36775.463132.50-march=znver51. (CXX) g++ options: -fPIC -O3 -flto -fsigned-char -ffunction-sections -fdata-sections -fno-strict-overflow -fwrapv -shared -ldl -lstdc++fs

OpenVINO

Model: Person Vehicle Bike Detection FP16 - Device: CPU

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2024.5Model: Person Vehicle Bike Detection FP16 - Device: CPUAVX-512 EnabledAVX-512 Disabled48121620SE +/- 0.02, N = 3SE +/- 0.01, N = 37.0315.26-march=znver5 - MIN: 3.43 / MAX: 24.51MIN: 7.72 / MAX: 48.831. (CXX) g++ options: -fPIC -O3 -flto -fsigned-char -ffunction-sections -fdata-sections -fno-strict-overflow -fwrapv -shared -ldl -lstdc++fs

OpenVINO

CPU Peak Freq (Highest CPU Core Frequency) Monitor

OpenBenchmarking.orgMegahertz, More Is BetterOpenVINO 2024.5CPU Peak Freq (Highest CPU Core Frequency) MonitorAVX-512 EnabledAVX-512 Disabled8001600240032004000Min: 2216 / Avg: 2363.03 / Max: 4569Min: 2600 / Avg: 2695.03 / Max: 4545

OpenVINO

CPU Power Consumption Monitor

OpenBenchmarking.orgWatts, Fewer Is BetterOpenVINO 2024.5CPU Power Consumption MonitorAVX-512 EnabledAVX-512 Disabled60120180240300Min: 0.53 / Avg: 264.6 / Max: 289.09Min: 0.87 / Avg: 304.37 / Max: 332.97

OpenVINO

CPU Temperature Monitor

OpenBenchmarking.orgCelsius, Fewer Is BetterOpenVINO 2024.5CPU Temperature MonitorAVX-512 EnabledAVX-512 Disabled1122334455Min: 44.38 / Avg: 50.87 / Max: 51.63Min: 41.75 / Avg: 51.96 / Max: 53.5

OpenVINO

Model: Machine Translation EN To DE FP16 - Device: CPU

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2024.5Model: Machine Translation EN To DE FP16 - Device: CPUAVX-512 EnabledAVX-512 Disabled2004006008001000SE +/- 0.97, N = 3SE +/- 0.09, N = 3857.16295.88-march=znver51. (CXX) g++ options: -fPIC -O3 -flto -fsigned-char -ffunction-sections -fdata-sections -fno-strict-overflow -fwrapv -shared -ldl -lstdc++fs

OpenVINO

Model: Machine Translation EN To DE FP16 - Device: CPU

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2024.5Model: Machine Translation EN To DE FP16 - Device: CPUAVX-512 EnabledAVX-512 Disabled4080120160200SE +/- 0.07, N = 3SE +/- 0.05, N = 355.95162.02-march=znver5 - MIN: 26.76 / MAX: 125.18MIN: 75.62 / MAX: 264.141. (CXX) g++ options: -fPIC -O3 -flto -fsigned-char -ffunction-sections -fdata-sections -fno-strict-overflow -fwrapv -shared -ldl -lstdc++fs

OpenVINO

CPU Peak Freq (Highest CPU Core Frequency) Monitor

OpenBenchmarking.orgMegahertz, More Is BetterOpenVINO 2024.5CPU Peak Freq (Highest CPU Core Frequency) MonitorAVX-512 EnabledAVX-512 Disabled8001600240032004000Min: 2411 / Avg: 2576.28 / Max: 4563Min: 2480 / Avg: 2633.44 / Max: 4568

OpenVINO

CPU Power Consumption Monitor

OpenBenchmarking.orgWatts, Fewer Is BetterOpenVINO 2024.5CPU Power Consumption MonitorAVX-512 EnabledAVX-512 Disabled60120180240300Min: 0.21 / Avg: 324.19 / Max: 363.41Min: 0.71 / Avg: 320.19 / Max: 351.89

OpenVINO

CPU Temperature Monitor

OpenBenchmarking.orgCelsius, Fewer Is BetterOpenVINO 2024.5CPU Temperature MonitorAVX-512 EnabledAVX-512 Disabled1122334455Min: 41.38 / Avg: 55.24 / Max: 57.63Min: 41.75 / Avg: 53.37 / Max: 55.38

OpenVINO

Model: Face Detection Retail FP16 - Device: CPU

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2024.5Model: Face Detection Retail FP16 - Device: CPUAVX-512 EnabledAVX-512 Disabled4K8K12K16K20KSE +/- 12.77, N = 3SE +/- 8.72, N = 316580.186688.98-march=znver51. (CXX) g++ options: -fPIC -O3 -flto -fsigned-char -ffunction-sections -fdata-sections -fno-strict-overflow -fwrapv -shared -ldl -lstdc++fs

OpenVINO

Model: Face Detection Retail FP16 - Device: CPU

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2024.5Model: Face Detection Retail FP16 - Device: CPUAVX-512 EnabledAVX-512 Disabled246810SE +/- 0.00, N = 3SE +/- 0.01, N = 32.827.14-march=znver5 - MIN: 1.14 / MAX: 18.12MIN: 2.91 / MAX: 32.041. (CXX) g++ options: -fPIC -O3 -flto -fsigned-char -ffunction-sections -fdata-sections -fno-strict-overflow -fwrapv -shared -ldl -lstdc++fs

OpenVINO

CPU Peak Freq (Highest CPU Core Frequency) Monitor

OpenBenchmarking.orgMegahertz, More Is BetterOpenVINO 2024.5CPU Peak Freq (Highest CPU Core Frequency) MonitorAVX-512 EnabledAVX-512 Disabled8001600240032004000Min: 2269 / Avg: 2338.6 / Max: 4541Min: 2469 / Avg: 2591.17 / Max: 4530

OpenVINO

CPU Power Consumption Monitor

OpenBenchmarking.orgWatts, Fewer Is BetterOpenVINO 2024.5CPU Power Consumption MonitorAVX-512 EnabledAVX-512 Disabled60120180240300Min: 0.21 / Avg: 299.8 / Max: 324.2Min: 0.69 / Avg: 315.06 / Max: 340.78

OpenVINO

CPU Temperature Monitor

OpenBenchmarking.orgCelsius, Fewer Is BetterOpenVINO 2024.5CPU Temperature MonitorAVX-512 EnabledAVX-512 Disabled1122334455Min: 44.38 / Avg: 54.39 / Max: 55.75Min: 42.38 / Avg: 53.28 / Max: 54.88

OpenVINO

Model: Handwritten English Recognition FP16 - Device: CPU

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2024.5Model: Handwritten English Recognition FP16 - Device: CPUAVX-512 EnabledAVX-512 Disabled7001400210028003500SE +/- 4.77, N = 3SE +/- 0.28, N = 33488.231200.37-march=znver51. (CXX) g++ options: -fPIC -O3 -flto -fsigned-char -ffunction-sections -fdata-sections -fno-strict-overflow -fwrapv -shared -ldl -lstdc++fs

OpenVINO

Model: Handwritten English Recognition FP16 - Device: CPU

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2024.5Model: Handwritten English Recognition FP16 - Device: CPUAVX-512 EnabledAVX-512 Disabled20406080100SE +/- 0.04, N = 3SE +/- 0.02, N = 327.5079.90-march=znver5 - MIN: 13.39 / MAX: 61.16MIN: 35.69 / MAX: 151.021. (CXX) g++ options: -fPIC -O3 -flto -fsigned-char -ffunction-sections -fdata-sections -fno-strict-overflow -fwrapv -shared -ldl -lstdc++fs

OpenVINO

CPU Peak Freq (Highest CPU Core Frequency) Monitor

OpenBenchmarking.orgMegahertz, More Is BetterOpenVINO 2024.5CPU Peak Freq (Highest CPU Core Frequency) MonitorAVX-512 EnabledAVX-512 Disabled8001600240032004000Min: 2434 / Avg: 2573.7 / Max: 4562Min: 2447 / Avg: 2590.5 / Max: 4548

OpenVINO

CPU Power Consumption Monitor

OpenBenchmarking.orgWatts, Fewer Is BetterOpenVINO 2024.5CPU Power Consumption MonitorAVX-512 EnabledAVX-512 Disabled60120180240300Min: 0.04 / Avg: 335.84 / Max: 363.56Min: 0.57 / Avg: 322.31 / Max: 347.06

OpenVINO

CPU Temperature Monitor

OpenBenchmarking.orgCelsius, Fewer Is BetterOpenVINO 2024.5CPU Temperature MonitorAVX-512 EnabledAVX-512 Disabled1224364860Min: 43.75 / Avg: 56.13 / Max: 58.25Min: 42.5 / Avg: 53.6 / Max: 55.5

OpenVINO

Model: Road Segmentation ADAS FP16 - Device: CPU

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2024.5Model: Road Segmentation ADAS FP16 - Device: CPUAVX-512 EnabledAVX-512 Disabled5001000150020002500SE +/- 6.46, N = 3SE +/- 1.87, N = 32102.03713.96-march=znver51. (CXX) g++ options: -fPIC -O3 -flto -fsigned-char -ffunction-sections -fdata-sections -fno-strict-overflow -fwrapv -shared -ldl -lstdc++fs

OpenVINO

Model: Road Segmentation ADAS FP16 - Device: CPU

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2024.5Model: Road Segmentation ADAS FP16 - Device: CPUAVX-512 EnabledAVX-512 Disabled1530456075SE +/- 0.07, N = 3SE +/- 0.18, N = 322.7867.12-march=znver5 - MIN: 11.57 / MAX: 67.03MIN: 17.27 / MAX: 133.161. (CXX) g++ options: -fPIC -O3 -flto -fsigned-char -ffunction-sections -fdata-sections -fno-strict-overflow -fwrapv -shared -ldl -lstdc++fs

OpenVINO

CPU Peak Freq (Highest CPU Core Frequency) Monitor

OpenBenchmarking.orgMegahertz, More Is BetterOpenVINO 2024.5CPU Peak Freq (Highest CPU Core Frequency) MonitorAVX-512 EnabledAVX-512 Disabled8001600240032004000Min: 2323 / Avg: 2457.52 / Max: 4559Min: 2595 / Avg: 2730.19 / Max: 4576

OpenVINO

CPU Power Consumption Monitor

OpenBenchmarking.orgWatts, Fewer Is BetterOpenVINO 2024.5CPU Power Consumption MonitorAVX-512 EnabledAVX-512 Disabled60120180240300Min: 0.04 / Avg: 305.32 / Max: 336.48Min: 0.49 / Avg: 279.46 / Max: 305.97

OpenVINO

CPU Temperature Monitor

OpenBenchmarking.orgCelsius, Fewer Is BetterOpenVINO 2024.5CPU Temperature MonitorAVX-512 EnabledAVX-512 Disabled1122334455Min: 45.25 / Avg: 54.03 / Max: 55.38Min: 42.75 / Avg: 49.82 / Max: 50.88

OpenVINO

Model: Person Re-Identification Retail FP16 - Device: CPU

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2024.5Model: Person Re-Identification Retail FP16 - Device: CPUAVX-512 EnabledAVX-512 Disabled2K4K6K8K10KSE +/- 6.44, N = 3SE +/- 2.23, N = 310524.043605.75-march=znver51. (CXX) g++ options: -fPIC -O3 -flto -fsigned-char -ffunction-sections -fdata-sections -fno-strict-overflow -fwrapv -shared -ldl -lstdc++fs

OpenVINO

Model: Person Re-Identification Retail FP16 - Device: CPU

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2024.5Model: Person Re-Identification Retail FP16 - Device: CPUAVX-512 EnabledAVX-512 Disabled3691215SE +/- 0.00, N = 3SE +/- 0.01, N = 34.5213.30-march=znver5 - MIN: 2.34 / MAX: 21.71MIN: 7.56 / MAX: 31.81. (CXX) g++ options: -fPIC -O3 -flto -fsigned-char -ffunction-sections -fdata-sections -fno-strict-overflow -fwrapv -shared -ldl -lstdc++fs

OpenVINO

CPU Peak Freq (Highest CPU Core Frequency) Monitor

OpenBenchmarking.orgMegahertz, More Is BetterOpenVINO 2024.5CPU Peak Freq (Highest CPU Core Frequency) MonitorAVX-512 EnabledAVX-512 Disabled8001600240032004000Min: 2362 / Avg: 2549.18 / Max: 4548Min: 2522 / Avg: 2627.11 / Max: 4561

OpenVINO

CPU Power Consumption Monitor

OpenBenchmarking.orgWatts, Fewer Is BetterOpenVINO 2024.5CPU Power Consumption MonitorAVX-512 EnabledAVX-512 Disabled60120180240300Min: 37.27 / Avg: 311.68 / Max: 337.81Min: 0.19 / Avg: 302.22 / Max: 329.27

OpenVINO

CPU Temperature Monitor

OpenBenchmarking.orgCelsius, Fewer Is BetterOpenVINO 2024.5CPU Temperature MonitorAVX-512 EnabledAVX-512 Disabled1122334455Min: 44.25 / Avg: 56.07 / Max: 57.63Min: 40.63 / Avg: 53.04 / Max: 54.88

OpenVINO

Model: Noise Suppression Poconet-Like FP16 - Device: CPU

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2024.5Model: Noise Suppression Poconet-Like FP16 - Device: CPUAVX-512 EnabledAVX-512 Disabled15003000450060007500SE +/- 13.37, N = 3SE +/- 6.13, N = 36825.092516.89-march=znver51. (CXX) g++ options: -fPIC -O3 -flto -fsigned-char -ffunction-sections -fdata-sections -fno-strict-overflow -fwrapv -shared -ldl -lstdc++fs

OpenVINO

Model: Noise Suppression Poconet-Like FP16 - Device: CPU

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2024.5Model: Noise Suppression Poconet-Like FP16 - Device: CPUAVX-512 EnabledAVX-512 Disabled918273645SE +/- 0.03, N = 3SE +/- 0.09, N = 313.4738.05-march=znver5 - MIN: 7.72 / MAX: 51.08MIN: 16.64 / MAX: 61.981. (CXX) g++ options: -fPIC -O3 -flto -fsigned-char -ffunction-sections -fdata-sections -fno-strict-overflow -fwrapv -shared -ldl -lstdc++fs

OpenVINO

CPU Peak Freq (Highest CPU Core Frequency) Monitor

OpenBenchmarking.orgMegahertz, More Is BetterOpenVINO 2024.5CPU Peak Freq (Highest CPU Core Frequency) MonitorAVX-512 EnabledAVX-512 Disabled8001600240032004000Min: 2528 / Avg: 2682.43 / Max: 4556Min: 2600 / Avg: 3003.36 / Max: 4566

OpenVINO

CPU Power Consumption Monitor

OpenBenchmarking.orgWatts, Fewer Is BetterOpenVINO 2024.5CPU Power Consumption MonitorAVX-512 EnabledAVX-512 Disabled60120180240300Min: 36.85 / Avg: 290.71 / Max: 317.82Min: 0.02 / Avg: 263.3 / Max: 290.12

OpenVINO

CPU Temperature Monitor

OpenBenchmarking.orgCelsius, Fewer Is BetterOpenVINO 2024.5CPU Temperature MonitorAVX-512 EnabledAVX-512 Disabled1122334455Min: 44.63 / Avg: 51.63 / Max: 52.75Min: 42.5 / Avg: 48.86 / Max: 49.75

OpenVINO GenAI

Model: TinyLlama-1.1B-Chat-v1.0 - Device: CPU

OpenBenchmarking.orgtokens/s, More Is BetterOpenVINO GenAI 2024.5Model: TinyLlama-1.1B-Chat-v1.0 - Device: CPUAVX-512 EnabledAVX-512 Disabled20406080100SE +/- 0.94, N = 3SE +/- 1.17, N = 1566.5791.40

OpenVINO GenAI

Model: TinyLlama-1.1B-Chat-v1.0 - Device: CPU - Time To First Token

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO GenAI 2024.5Model: TinyLlama-1.1B-Chat-v1.0 - Device: CPU - Time To First TokenAVX-512 EnabledAVX-512 Disabled612182430SE +/- 0.18, N = 3SE +/- 0.58, N = 1518.5927.01

OpenVINO GenAI

Model: TinyLlama-1.1B-Chat-v1.0 - Device: CPU - Time Per Output Token

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO GenAI 2024.5Model: TinyLlama-1.1B-Chat-v1.0 - Device: CPU - Time Per Output TokenAVX-512 EnabledAVX-512 Disabled48121620SE +/- 0.21, N = 3SE +/- 0.14, N = 1515.0310.97

OpenVINO GenAI

CPU Peak Freq (Highest CPU Core Frequency) Monitor

OpenBenchmarking.orgMegahertz, More Is BetterOpenVINO GenAI 2024.5CPU Peak Freq (Highest CPU Core Frequency) MonitorAVX-512 EnabledAVX-512 Disabled8001600240032004000Min: 2600 / Avg: 3645.71 / Max: 4548Min: 2600 / Avg: 3933.96 / Max: 4565

OpenVINO GenAI

CPU Power Consumption Monitor

OpenBenchmarking.orgWatts, Fewer Is BetterOpenVINO GenAI 2024.5CPU Power Consumption MonitorAVX-512 EnabledAVX-512 Disabled60120180240300Min: 1.23 / Avg: 214.02 / Max: 268.91Min: 1.57 / Avg: 239.85 / Max: 314.54

OpenVINO GenAI

CPU Temperature Monitor

OpenBenchmarking.orgCelsius, Fewer Is BetterOpenVINO GenAI 2024.5CPU Temperature MonitorAVX-512 EnabledAVX-512 Disabled1020304050Min: 42.63 / Avg: 48.16 / Max: 49.88Min: 39.75 / Avg: 47.62 / Max: 49.13

OpenVKL

Benchmark: vklBenchmarkCPU ISPC

OpenBenchmarking.orgItems / Sec, More Is BetterOpenVKL 2.0.0Benchmark: vklBenchmarkCPU ISPCAVX-512 EnabledAVX-512 Disabled6001200180024003000SE +/- 0.58, N = 3SE +/- 0.58, N = 328172353MIN: 217 / MAX: 36244MIN: 179 / MAX: 30230

OpenVKL

CPU Peak Freq (Highest CPU Core Frequency) Monitor

OpenBenchmarking.orgMegahertz, More Is BetterOpenVKL 2.0.0CPU Peak Freq (Highest CPU Core Frequency) MonitorAVX-512 EnabledAVX-512 Disabled8001600240032004000Min: 2600 / Avg: 3612.44 / Max: 4571Min: 2600 / Avg: 3532.47 / Max: 4576

OpenVKL

CPU Power Consumption Monitor

OpenBenchmarking.orgWatts, Fewer Is BetterOpenVKL 2.0.0CPU Power Consumption MonitorAVX-512 EnabledAVX-512 Disabled70140210280350Min: 0.88 / Avg: 308.36 / Max: 372.72Min: 1.97 / Avg: 306.93 / Max: 377.3

OpenVKL

CPU Temperature Monitor

OpenBenchmarking.orgCelsius, Fewer Is BetterOpenVKL 2.0.0CPU Temperature MonitorAVX-512 EnabledAVX-512 Disabled1326395265Min: 37.38 / Avg: 57.47 / Max: 65.38Min: 37.5 / Avg: 54.39 / Max: 61.88

OSPRay

Benchmark: gravity_spheres_volume/dim_512/ao/real_time

OpenBenchmarking.orgItems Per Second, More Is BetterOSPRay 3.2Benchmark: gravity_spheres_volume/dim_512/ao/real_timeAVX-512 EnabledAVX-512 Disabled816243240SE +/- 0.02, N = 3SE +/- 0.01, N = 335.2824.03

OSPRay

CPU Peak Freq (Highest CPU Core Frequency) Monitor

OpenBenchmarking.orgMegahertz, More Is BetterOSPRay 3.2CPU Peak Freq (Highest CPU Core Frequency) MonitorAVX-512 EnabledAVX-512 Disabled8001600240032004000Min: 2600 / Avg: 3311.38 / Max: 4571Min: 2600 / Avg: 3299.52 / Max: 4562

OSPRay

CPU Power Consumption Monitor

OpenBenchmarking.orgWatts, Fewer Is BetterOSPRay 3.2CPU Power Consumption MonitorAVX-512 EnabledAVX-512 Disabled70140210280350Min: 37.02 / Avg: 330.88 / Max: 364.25Min: 1.33 / Avg: 311.01 / Max: 355.31

OSPRay

CPU Temperature Monitor

OpenBenchmarking.orgCelsius, Fewer Is BetterOSPRay 3.2CPU Temperature MonitorAVX-512 EnabledAVX-512 Disabled1326395265Min: 44.63 / Avg: 60.49 / Max: 69Min: 41.63 / Avg: 55.83 / Max: 63.13

OSPRay

Benchmark: gravity_spheres_volume/dim_512/scivis/real_time

OpenBenchmarking.orgItems Per Second, More Is BetterOSPRay 3.2Benchmark: gravity_spheres_volume/dim_512/scivis/real_timeAVX-512 EnabledAVX-512 Disabled816243240SE +/- 0.01, N = 3SE +/- 0.01, N = 334.6923.17

OSPRay

CPU Peak Freq (Highest CPU Core Frequency) Monitor

OpenBenchmarking.orgMegahertz, More Is BetterOSPRay 3.2CPU Peak Freq (Highest CPU Core Frequency) MonitorAVX-512 EnabledAVX-512 Disabled8001600240032004000Min: 2600 / Avg: 3312.61 / Max: 4560Min: 2600 / Avg: 3317.14 / Max: 4562

OSPRay

CPU Power Consumption Monitor

OpenBenchmarking.orgWatts, Fewer Is BetterOSPRay 3.2CPU Power Consumption MonitorAVX-512 EnabledAVX-512 Disabled70140210280350Min: 0.39 / Avg: 330.26 / Max: 366.06Min: 0.75 / Avg: 313.05 / Max: 359.7

OSPRay

CPU Temperature Monitor

OpenBenchmarking.orgCelsius, Fewer Is BetterOSPRay 3.2CPU Temperature MonitorAVX-512 EnabledAVX-512 Disabled1326395265Min: 46.38 / Avg: 60.94 / Max: 67.5Min: 42.5 / Avg: 56.43 / Max: 64.25

OSPRay

Benchmark: gravity_spheres_volume/dim_512/pathtracer/real_time

OpenBenchmarking.orgItems Per Second, More Is BetterOSPRay 3.2Benchmark: gravity_spheres_volume/dim_512/pathtracer/real_timeAVX-512 EnabledAVX-512 Disabled918273645SE +/- 0.02, N = 3SE +/- 0.04, N = 338.2031.58

OSPRay

CPU Peak Freq (Highest CPU Core Frequency) Monitor

OpenBenchmarking.orgMegahertz, More Is BetterOSPRay 3.2CPU Peak Freq (Highest CPU Core Frequency) MonitorAVX-512 EnabledAVX-512 Disabled8001600240032004000Min: 2600 / Avg: 3170.5 / Max: 4566Min: 2600 / Avg: 3173.58 / Max: 4560

OSPRay

CPU Power Consumption Monitor

OpenBenchmarking.orgWatts, Fewer Is BetterOSPRay 3.2CPU Power Consumption MonitorAVX-512 EnabledAVX-512 Disabled60120180240300Min: 0.93 / Avg: 315.73 / Max: 348.88Min: 0.2 / Avg: 305.49 / Max: 349.64

OSPRay

CPU Temperature Monitor

OpenBenchmarking.orgCelsius, Fewer Is BetterOSPRay 3.2CPU Temperature MonitorAVX-512 EnabledAVX-512 Disabled1326395265Min: 46.63 / Avg: 59.53 / Max: 66.25Min: 42.88 / Avg: 56.25 / Max: 63.25

OSPRay Studio

Camera: 1 - Resolution: 4K - Samples Per Pixel: 1 - Renderer: Path Tracer - Acceleration: CPU

OpenBenchmarking.orgms, Fewer Is BetterOSPRay Studio 1.0Camera: 1 - Resolution: 4K - Samples Per Pixel: 1 - Renderer: Path Tracer - Acceleration: CPUAVX-512 EnabledAVX-512 Disabled2004006008001000SE +/- 0.33, N = 3SE +/- 0.33, N = 3868966

OSPRay Studio

CPU Peak Freq (Highest CPU Core Frequency) Monitor

OpenBenchmarking.orgMegahertz, More Is BetterOSPRay Studio 1.0CPU Peak Freq (Highest CPU Core Frequency) MonitorAVX-512 EnabledAVX-512 Disabled8001600240032004000Min: 2600 / Avg: 3207.7 / Max: 4551Min: 2600 / Avg: 3210.72 / Max: 4566

OSPRay Studio

CPU Power Consumption Monitor

OpenBenchmarking.orgWatts, Fewer Is BetterOSPRay Studio 1.0CPU Power Consumption MonitorAVX-512 EnabledAVX-512 Disabled60120180240300Min: 0.92 / Avg: 282.47 / Max: 349.36Min: 0.81 / Avg: 285.31 / Max: 348.45

OSPRay Studio

CPU Temperature Monitor

OpenBenchmarking.orgCelsius, Fewer Is BetterOSPRay Studio 1.0CPU Temperature MonitorAVX-512 EnabledAVX-512 Disabled1326395265Min: 42.75 / Avg: 56.59 / Max: 66.88Min: 39.13 / Avg: 53.19 / Max: 62.75

OSPRay Studio

Camera: 1 - Resolution: 4K - Samples Per Pixel: 16 - Renderer: Path Tracer - Acceleration: CPU

OpenBenchmarking.orgms, Fewer Is BetterOSPRay Studio 1.0Camera: 1 - Resolution: 4K - Samples Per Pixel: 16 - Renderer: Path Tracer - Acceleration: CPUAVX-512 EnabledAVX-512 Disabled3K6K9K12K15KSE +/- 15.50, N = 3SE +/- 6.36, N = 31382315382

OSPRay Studio

CPU Peak Freq (Highest CPU Core Frequency) Monitor

OpenBenchmarking.orgMegahertz, More Is BetterOSPRay Studio 1.0CPU Peak Freq (Highest CPU Core Frequency) MonitorAVX-512 EnabledAVX-512 Disabled8001600240032004000Min: 2600 / Avg: 3129.96 / Max: 4569Min: 2600 / Avg: 3221.06 / Max: 4547

OSPRay Studio

CPU Power Consumption Monitor

OpenBenchmarking.orgWatts, Fewer Is BetterOSPRay Studio 1.0CPU Power Consumption MonitorAVX-512 EnabledAVX-512 Disabled60120180240300Min: 0.62 / Avg: 303.81 / Max: 348.8Min: 0.5 / Avg: 279.19 / Max: 346.69

OSPRay Studio

CPU Temperature Monitor

OpenBenchmarking.orgCelsius, Fewer Is BetterOSPRay Studio 1.0CPU Temperature MonitorAVX-512 EnabledAVX-512 Disabled1326395265Min: 44.25 / Avg: 57.72 / Max: 69.13Min: 41.13 / Avg: 53.68 / Max: 64

OSPRay Studio

Camera: 1 - Resolution: 4K - Samples Per Pixel: 32 - Renderer: Path Tracer - Acceleration: CPU

OpenBenchmarking.orgms, Fewer Is BetterOSPRay Studio 1.0Camera: 1 - Resolution: 4K - Samples Per Pixel: 32 - Renderer: Path Tracer - Acceleration: CPUAVX-512 EnabledAVX-512 Disabled7K14K21K28K35KSE +/- 80.19, N = 3SE +/- 88.93, N = 33095834668

OSPRay Studio

CPU Peak Freq (Highest CPU Core Frequency) Monitor

OpenBenchmarking.orgMegahertz, More Is BetterOSPRay Studio 1.0CPU Peak Freq (Highest CPU Core Frequency) MonitorAVX-512 EnabledAVX-512 Disabled8001600240032004000Min: 2600 / Avg: 3314.8 / Max: 4569Min: 2600 / Avg: 3292.85 / Max: 4565

OSPRay Studio

CPU Power Consumption Monitor

OpenBenchmarking.orgWatts, Fewer Is BetterOSPRay Studio 1.0CPU Power Consumption MonitorAVX-512 EnabledAVX-512 Disabled60120180240300Min: 1.32 / Avg: 250.81 / Max: 347.38Min: 36.53 / Avg: 256.86 / Max: 346.05

OSPRay Studio

CPU Temperature Monitor

OpenBenchmarking.orgCelsius, Fewer Is BetterOSPRay Studio 1.0CPU Temperature MonitorAVX-512 EnabledAVX-512 Disabled1326395265Min: 45.25 / Avg: 57.63 / Max: 68.25Min: 40.75 / Avg: 52.93 / Max: 60

OSPRay Studio

Camera: 2 - Resolution: 4K - Samples Per Pixel: 1 - Renderer: Path Tracer - Acceleration: CPU

OpenBenchmarking.orgms, Fewer Is BetterOSPRay Studio 1.0Camera: 2 - Resolution: 4K - Samples Per Pixel: 1 - Renderer: Path Tracer - Acceleration: CPUAVX-512 EnabledAVX-512 Disabled2004006008001000SE +/- 0.67, N = 3SE +/- 0.67, N = 3873974

OSPRay Studio

CPU Peak Freq (Highest CPU Core Frequency) Monitor

OpenBenchmarking.orgMegahertz, More Is BetterOSPRay Studio 1.0CPU Peak Freq (Highest CPU Core Frequency) MonitorAVX-512 EnabledAVX-512 Disabled8001600240032004000Min: 2600 / Avg: 3233.67 / Max: 4563Min: 2600 / Avg: 3247.36 / Max: 4575

OSPRay Studio

CPU Power Consumption Monitor

OpenBenchmarking.orgWatts, Fewer Is BetterOSPRay Studio 1.0CPU Power Consumption MonitorAVX-512 EnabledAVX-512 Disabled60120180240300Min: 0.31 / Avg: 283.34 / Max: 345.47Min: 1.24 / Avg: 286.84 / Max: 349.23

OSPRay Studio

CPU Temperature Monitor

OpenBenchmarking.orgCelsius, Fewer Is BetterOSPRay Studio 1.0CPU Temperature MonitorAVX-512 EnabledAVX-512 Disabled1326395265Min: 43.38 / Avg: 56.47 / Max: 66.13Min: 40.25 / Avg: 53.39 / Max: 62.63

OSPRay Studio

Camera: 2 - Resolution: 4K - Samples Per Pixel: 16 - Renderer: Path Tracer - Acceleration: CPU

OpenBenchmarking.orgms, Fewer Is BetterOSPRay Studio 1.0Camera: 2 - Resolution: 4K - Samples Per Pixel: 16 - Renderer: Path Tracer - Acceleration: CPUAVX-512 EnabledAVX-512 Disabled3K6K9K12K15KSE +/- 18.02, N = 3SE +/- 17.35, N = 31390115503

OSPRay Studio

CPU Peak Freq (Highest CPU Core Frequency) Monitor

OpenBenchmarking.orgMegahertz, More Is BetterOSPRay Studio 1.0CPU Peak Freq (Highest CPU Core Frequency) MonitorAVX-512 EnabledAVX-512 Disabled8001600240032004000Min: 2600 / Avg: 3138.26 / Max: 4559Min: 2600 / Avg: 3230.91 / Max: 4572

OSPRay Studio

CPU Power Consumption Monitor

OpenBenchmarking.orgWatts, Fewer Is BetterOSPRay Studio 1.0CPU Power Consumption MonitorAVX-512 EnabledAVX-512 Disabled60120180240300Min: 0.03 / Avg: 301.64 / Max: 351.17Min: 0.93 / Avg: 282.37 / Max: 350.52

OSPRay Studio

CPU Temperature Monitor

OpenBenchmarking.orgCelsius, Fewer Is BetterOSPRay Studio 1.0CPU Temperature MonitorAVX-512 EnabledAVX-512 Disabled1326395265Min: 44.13 / Avg: 57.23 / Max: 68.38Min: 41.25 / Avg: 53.71 / Max: 62.75

OSPRay Studio

Camera: 2 - Resolution: 4K - Samples Per Pixel: 32 - Renderer: Path Tracer - Acceleration: CPU

OpenBenchmarking.orgms, Fewer Is BetterOSPRay Studio 1.0Camera: 2 - Resolution: 4K - Samples Per Pixel: 32 - Renderer: Path Tracer - Acceleration: CPUAVX-512 EnabledAVX-512 Disabled7K14K21K28K35KSE +/- 30.12, N = 3SE +/- 52.20, N = 33107734480

OSPRay Studio

CPU Peak Freq (Highest CPU Core Frequency) Monitor

OpenBenchmarking.orgMegahertz, More Is BetterOSPRay Studio 1.0CPU Peak Freq (Highest CPU Core Frequency) MonitorAVX-512 EnabledAVX-512 Disabled8001600240032004000Min: 2600 / Avg: 3343.68 / Max: 4571Min: 2600 / Avg: 3313.46 / Max: 4563

OSPRay Studio

CPU Power Consumption Monitor

OpenBenchmarking.orgWatts, Fewer Is BetterOSPRay Studio 1.0CPU Power Consumption MonitorAVX-512 EnabledAVX-512 Disabled60120180240300Min: 0.71 / Avg: 250.67 / Max: 343.32Min: 0.43 / Avg: 259.93 / Max: 349.27

OSPRay Studio

CPU Temperature Monitor

OpenBenchmarking.orgCelsius, Fewer Is BetterOSPRay Studio 1.0CPU Temperature MonitorAVX-512 EnabledAVX-512 Disabled1326395265Min: 45 / Avg: 57.53 / Max: 68Min: 41.38 / Avg: 53.67 / Max: 62.63

OSPRay Studio

Camera: 3 - Resolution: 4K - Samples Per Pixel: 1 - Renderer: Path Tracer - Acceleration: CPU

OpenBenchmarking.orgms, Fewer Is BetterOSPRay Studio 1.0Camera: 3 - Resolution: 4K - Samples Per Pixel: 1 - Renderer: Path Tracer - Acceleration: CPUAVX-512 EnabledAVX-512 Disabled2004006008001000SE +/- 1.15, N = 3SE +/- 0.33, N = 310241135

OSPRay Studio

CPU Peak Freq (Highest CPU Core Frequency) Monitor

OpenBenchmarking.orgMegahertz, More Is BetterOSPRay Studio 1.0CPU Peak Freq (Highest CPU Core Frequency) MonitorAVX-512 EnabledAVX-512 Disabled8001600240032004000Min: 2600 / Avg: 3218.88 / Max: 4545Min: 2600 / Avg: 3232.49 / Max: 4565

OSPRay Studio

CPU Power Consumption Monitor

OpenBenchmarking.orgWatts, Fewer Is BetterOSPRay Studio 1.0CPU Power Consumption MonitorAVX-512 EnabledAVX-512 Disabled60120180240300Min: 1.64 / Avg: 284.65 / Max: 349.24Min: 1.69 / Avg: 286.42 / Max: 348.82

OSPRay Studio

CPU Temperature Monitor

OpenBenchmarking.orgCelsius, Fewer Is BetterOSPRay Studio 1.0CPU Temperature MonitorAVX-512 EnabledAVX-512 Disabled1326395265Min: 43 / Avg: 56.47 / Max: 65.88Min: 40.38 / Avg: 53.32 / Max: 62.25

OSPRay Studio

Camera: 3 - Resolution: 4K - Samples Per Pixel: 16 - Renderer: Path Tracer - Acceleration: CPU

OpenBenchmarking.orgms, Fewer Is BetterOSPRay Studio 1.0Camera: 3 - Resolution: 4K - Samples Per Pixel: 16 - Renderer: Path Tracer - Acceleration: CPUAVX-512 EnabledAVX-512 Disabled4K8K12K16K20KSE +/- 1.45, N = 3SE +/- 34.35, N = 31629718098

OSPRay Studio

CPU Peak Freq (Highest CPU Core Frequency) Monitor

OpenBenchmarking.orgMegahertz, More Is BetterOSPRay Studio 1.0CPU Peak Freq (Highest CPU Core Frequency) MonitorAVX-512 EnabledAVX-512 Disabled8001600240032004000Min: 2600 / Avg: 3212.44 / Max: 4562Min: 2600 / Avg: 3195.19 / Max: 4571

OSPRay Studio

CPU Power Consumption Monitor

OpenBenchmarking.orgWatts, Fewer Is BetterOSPRay Studio 1.0CPU Power Consumption MonitorAVX-512 EnabledAVX-512 Disabled60120180240300Min: 1.41 / Avg: 279.53 / Max: 349.12Min: 1.44 / Avg: 288.83 / Max: 349.3

OSPRay Studio

CPU Temperature Monitor

OpenBenchmarking.orgCelsius, Fewer Is BetterOSPRay Studio 1.0CPU Temperature MonitorAVX-512 EnabledAVX-512 Disabled1326395265Min: 44 / Avg: 56.51 / Max: 66.25Min: 41.38 / Avg: 53.9 / Max: 63.38

OSPRay Studio

Camera: 3 - Resolution: 4K - Samples Per Pixel: 32 - Renderer: Path Tracer - Acceleration: CPU

OpenBenchmarking.orgms, Fewer Is BetterOSPRay Studio 1.0Camera: 3 - Resolution: 4K - Samples Per Pixel: 32 - Renderer: Path Tracer - Acceleration: CPUAVX-512 EnabledAVX-512 Disabled9K18K27K36K45KSE +/- 50.45, N = 3SE +/- 59.00, N = 33595539785

OSPRay Studio

CPU Peak Freq (Highest CPU Core Frequency) Monitor

OpenBenchmarking.orgMegahertz, More Is BetterOSPRay Studio 1.0CPU Peak Freq (Highest CPU Core Frequency) MonitorAVX-512 EnabledAVX-512 Disabled8001600240032004000Min: 2600 / Avg: 3291.96 / Max: 4561Min: 2600 / Avg: 3282.21 / Max: 4552

OSPRay Studio

CPU Power Consumption Monitor

OpenBenchmarking.orgWatts, Fewer Is BetterOSPRay Studio 1.0CPU Power Consumption MonitorAVX-512 EnabledAVX-512 Disabled60120180240300Min: 1.05 / Avg: 258.37 / Max: 349.72Min: 1.22 / Avg: 266.98 / Max: 349.17

OSPRay Studio

CPU Temperature Monitor

OpenBenchmarking.orgCelsius, Fewer Is BetterOSPRay Studio 1.0CPU Temperature MonitorAVX-512 EnabledAVX-512 Disabled1326395265Min: 43.63 / Avg: 56.16 / Max: 64.88Min: 41.5 / Avg: 53.34 / Max: 61.5

PyTorch

Device: CPU - Batch Size: 256 - Model: ResNet-50

OpenBenchmarking.orgbatches/sec, More Is BetterPyTorch 2.2.1Device: CPU - Batch Size: 256 - Model: ResNet-50AVX-512 EnabledAVX-512 Disabled1122334455SE +/- 0.31, N = 3SE +/- 0.21, N = 350.5243.33MIN: 43.73 / MAX: 51.63MIN: 38.8 / MAX: 44.27

PyTorch

CPU Peak Freq (Highest CPU Core Frequency) Monitor

OpenBenchmarking.orgMegahertz, More Is BetterPyTorch 2.2.1CPU Peak Freq (Highest CPU Core Frequency) MonitorAVX-512 EnabledAVX-512 Disabled8001600240032004000Min: 2600 / Avg: 4209.69 / Max: 4572Min: 2600 / Avg: 4212.95 / Max: 4574

PyTorch

CPU Power Consumption Monitor

OpenBenchmarking.orgWatts, Fewer Is BetterPyTorch 2.2.1CPU Power Consumption MonitorAVX-512 EnabledAVX-512 Disabled50100150200250Min: 0.15 / Avg: 249.75 / Max: 294.5Min: 0.77 / Avg: 256.39 / Max: 297.62

PyTorch

CPU Temperature Monitor

OpenBenchmarking.orgCelsius, Fewer Is BetterPyTorch 2.2.1CPU Temperature MonitorAVX-512 EnabledAVX-512 Disabled1122334455Min: 36.38 / Avg: 47.66 / Max: 53.5Min: 27.13 / Avg: 43.57 / Max: 50.75

PyTorch

Device: CPU - Batch Size: 512 - Model: ResNet-50

OpenBenchmarking.orgbatches/sec, More Is BetterPyTorch 2.2.1Device: CPU - Batch Size: 512 - Model: ResNet-50AVX-512 EnabledAVX-512 Disabled1122334455SE +/- 0.28, N = 3SE +/- 0.14, N = 350.8443.44MIN: 44.51 / MAX: 52.23MIN: 38.59 / MAX: 44.4

PyTorch

CPU Peak Freq (Highest CPU Core Frequency) Monitor

OpenBenchmarking.orgMegahertz, More Is BetterPyTorch 2.2.1CPU Peak Freq (Highest CPU Core Frequency) MonitorAVX-512 EnabledAVX-512 Disabled8001600240032004000Min: 2600 / Avg: 4217.46 / Max: 4566Min: 2600 / Avg: 4212.34 / Max: 4571

PyTorch

CPU Power Consumption Monitor

OpenBenchmarking.orgWatts, Fewer Is BetterPyTorch 2.2.1CPU Power Consumption MonitorAVX-512 EnabledAVX-512 Disabled50100150200250Min: 1.08 / Avg: 250.98 / Max: 296.3Min: 36.76 / Avg: 259.81 / Max: 297.93

PyTorch

CPU Temperature Monitor

OpenBenchmarking.orgCelsius, Fewer Is BetterPyTorch 2.2.1CPU Temperature MonitorAVX-512 EnabledAVX-512 Disabled1122334455Min: 38.63 / Avg: 47.96 / Max: 54Min: 36 / Avg: 46 / Max: 52.13

SMHasher

Hash: FarmHash32 x86_64 AVX

OpenBenchmarking.orgMiB/sec, More Is BetterSMHasher 2022-08-22Hash: FarmHash32 x86_64 AVXAVX-512 EnabledAVX-512 Disabled8K16K24K32K40KSE +/- 10.36, N = 6SE +/- 21.63, N = 637638.6236139.951. (CXX) g++ options: -O3 -march=znver5 -flto -march=native -flto=auto -fno-fat-lto-objects

SMHasher

Hash: FarmHash32 x86_64 AVX

OpenBenchmarking.orgcycles/hash, Fewer Is BetterSMHasher 2022-08-22Hash: FarmHash32 x86_64 AVXAVX-512 EnabledAVX-512 Disabled510152025SE +/- 0.03, N = 6SE +/- 0.02, N = 622.8122.661. (CXX) g++ options: -O3 -march=znver5 -flto -march=native -flto=auto -fno-fat-lto-objects

SMHasher

CPU Peak Freq (Highest CPU Core Frequency) Monitor

OpenBenchmarking.orgMegahertz, More Is BetterSMHasher 2022-08-22CPU Peak Freq (Highest CPU Core Frequency) MonitorAVX-512 EnabledAVX-512 Disabled8001600240032004000Min: 2600 / Avg: 3897.49 / Max: 4573Min: 2600 / Avg: 3893.07 / Max: 4550

SMHasher

CPU Power Consumption Monitor

OpenBenchmarking.orgWatts, Fewer Is BetterSMHasher 2022-08-22CPU Power Consumption MonitorAVX-512 EnabledAVX-512 Disabled1530456075Min: 0.95 / Avg: 62.02 / Max: 78.27Min: 0.22 / Avg: 61.63 / Max: 78.15

SMHasher

CPU Temperature Monitor

OpenBenchmarking.orgCelsius, Fewer Is BetterSMHasher 2022-08-22CPU Temperature MonitorAVX-512 EnabledAVX-512 Disabled918273645Min: 32.75 / Avg: 38.67 / Max: 41.25Min: 30.25 / Avg: 36.44 / Max: 38.63

srsRAN Project

Test: PDSCH Processor Benchmark, Throughput Total

OpenBenchmarking.orgMbps, More Is BettersrsRAN Project 24.10Test: PDSCH Processor Benchmark, Throughput TotalAVX-512 EnabledAVX-512 Disabled30K60K90K120K150KSE +/- 1542.84, N = 3SE +/- 1008.70, N = 3124852.3110897.91. (CXX) g++ options: -O3 -march=native -mtune=generic -fno-trapping-math -fno-math-errno -ldl

srsRAN Project

CPU Peak Freq (Highest CPU Core Frequency) Monitor

OpenBenchmarking.orgMegahertz, More Is BettersrsRAN Project 24.10CPU Peak Freq (Highest CPU Core Frequency) MonitorAVX-512 EnabledAVX-512 Disabled8001600240032004000Min: 2600 / Avg: 2762.93 / Max: 4542Min: 2600 / Avg: 2811.32 / Max: 4544

srsRAN Project

CPU Power Consumption Monitor

OpenBenchmarking.orgWatts, Fewer Is BettersrsRAN Project 24.10CPU Power Consumption MonitorAVX-512 EnabledAVX-512 Disabled60120180240300Min: 0.15 / Avg: 271.4 / Max: 328.99Min: 1.13 / Avg: 292.55 / Max: 349.7

srsRAN Project

CPU Temperature Monitor

OpenBenchmarking.orgCelsius, Fewer Is BettersrsRAN Project 24.10CPU Temperature MonitorAVX-512 EnabledAVX-512 Disabled1020304050Min: 41.63 / Avg: 50.03 / Max: 52.38Min: 39.5 / Avg: 49.62 / Max: 52.38

SVT-AV1

Encoder Mode: Preset 13 - Input: Bosphorus 4K

OpenBenchmarking.orgFrames Per Second, More Is BetterSVT-AV1 2.3Encoder Mode: Preset 13 - Input: Bosphorus 4KAVX-512 EnabledAVX-512 Disabled90180270360450SE +/- 2.58, N = 6SE +/- 1.74, N = 6416.85412.131. (CXX) g++ options: -O3 -march=znver5 -flto -march=native -mno-avx -mavx2 -mavx512f -mavx512bw -mavx512dq

SVT-AV1

CPU Peak Freq (Highest CPU Core Frequency) Monitor

OpenBenchmarking.orgMegahertz, More Is BetterSVT-AV1 2.3CPU Peak Freq (Highest CPU Core Frequency) MonitorAVX-512 EnabledAVX-512 Disabled8001600240032004000Min: 2600 / Avg: 3734.19 / Max: 4566Min: 2600 / Avg: 3774.89 / Max: 4562

SVT-AV1

CPU Power Consumption Monitor

OpenBenchmarking.orgWatts, Fewer Is BetterSVT-AV1 2.3CPU Power Consumption MonitorAVX-512 EnabledAVX-512 Disabled50100150200250Min: 1.05 / Avg: 163.02 / Max: 295.39Min: 0.29 / Avg: 170.74 / Max: 305.74

SVT-AV1

CPU Temperature Monitor

OpenBenchmarking.orgCelsius, Fewer Is BetterSVT-AV1 2.3CPU Temperature MonitorAVX-512 EnabledAVX-512 Disabled1020304050Min: 36.13 / Avg: 45.83 / Max: 50.25Min: 34.38 / Avg: 44.92 / Max: 49.5

SVT-AV1

Encoder Mode: Preset 8 - Input: Bosphorus 4K

OpenBenchmarking.orgFrames Per Second, More Is BetterSVT-AV1 2.3Encoder Mode: Preset 8 - Input: Bosphorus 4KAVX-512 EnabledAVX-512 Disabled4080120160200SE +/- 1.02, N = 4SE +/- 1.07, N = 4184.79171.291. (CXX) g++ options: -O3 -march=znver5 -flto -march=native -mno-avx -mavx2 -mavx512f -mavx512bw -mavx512dq

SVT-AV1

CPU Peak Freq (Highest CPU Core Frequency) Monitor

OpenBenchmarking.orgMegahertz, More Is BetterSVT-AV1 2.3CPU Peak Freq (Highest CPU Core Frequency) MonitorAVX-512 EnabledAVX-512 Disabled8001600240032004000Min: 2600 / Avg: 3914.47 / Max: 4570Min: 2600 / Avg: 3904.67 / Max: 4570

SVT-AV1

CPU Power Consumption Monitor

OpenBenchmarking.orgWatts, Fewer Is BetterSVT-AV1 2.3CPU Power Consumption MonitorAVX-512 EnabledAVX-512 Disabled50100150200250Min: 0.59 / Avg: 185.34 / Max: 269.37Min: 1.23 / Avg: 188.91 / Max: 270.08

SVT-AV1

CPU Temperature Monitor

OpenBenchmarking.orgCelsius, Fewer Is BetterSVT-AV1 2.3CPU Temperature MonitorAVX-512 EnabledAVX-512 Disabled1020304050Min: 35.25 / Avg: 45.48 / Max: 50.63Min: 33.75 / Avg: 45.25 / Max: 49

SVT-AV1

Encoder Mode: Preset 5 - Input: Bosphorus 4K

OpenBenchmarking.orgFrames Per Second, More Is BetterSVT-AV1 2.3Encoder Mode: Preset 5 - Input: Bosphorus 4KAVX-512 EnabledAVX-512 Disabled1326395265SE +/- 0.23, N = 3SE +/- 0.29, N = 356.4851.741. (CXX) g++ options: -O3 -march=znver5 -flto -march=native -mno-avx -mavx2 -mavx512f -mavx512bw -mavx512dq

SVT-AV1

CPU Peak Freq (Highest CPU Core Frequency) Monitor

OpenBenchmarking.orgMegahertz, More Is BetterSVT-AV1 2.3CPU Peak Freq (Highest CPU Core Frequency) MonitorAVX-512 EnabledAVX-512 Disabled8001600240032004000Min: 2600 / Avg: 4297 / Max: 4570Min: 2600 / Avg: 4239.94 / Max: 4571

SVT-AV1

CPU Power Consumption Monitor

OpenBenchmarking.orgWatts, Fewer Is BetterSVT-AV1 2.3CPU Power Consumption MonitorAVX-512 EnabledAVX-512 Disabled50100150200250Min: 0.6 / Avg: 232.94 / Max: 291.22Min: 1.28 / Avg: 230.74 / Max: 300.94

SVT-AV1

CPU Temperature Monitor

OpenBenchmarking.orgCelsius, Fewer Is BetterSVT-AV1 2.3CPU Temperature MonitorAVX-512 EnabledAVX-512 Disabled1224364860Min: 36.88 / Avg: 54.65 / Max: 58.75Min: 35.5 / Avg: 53.19 / Max: 57.38

SVT-AV1

Encoder Mode: Preset 3 - Input: Bosphorus 4K

OpenBenchmarking.orgFrames Per Second, More Is BetterSVT-AV1 2.3Encoder Mode: Preset 3 - Input: Bosphorus 4KAVX-512 EnabledAVX-512 Disabled48121620SE +/- 0.03, N = 3SE +/- 0.05, N = 315.6514.521. (CXX) g++ options: -O3 -march=znver5 -flto -march=native -mno-avx -mavx2 -mavx512f -mavx512bw -mavx512dq

SVT-AV1

CPU Peak Freq (Highest CPU Core Frequency) Monitor

OpenBenchmarking.orgMegahertz, More Is BetterSVT-AV1 2.3CPU Peak Freq (Highest CPU Core Frequency) MonitorAVX-512 EnabledAVX-512 Disabled8001600240032004000Min: 2600 / Avg: 4350.52 / Max: 4664Min: 2600 / Avg: 4391.15 / Max: 4583

SVT-AV1

CPU Power Consumption Monitor

OpenBenchmarking.orgWatts, Fewer Is BetterSVT-AV1 2.3CPU Power Consumption MonitorAVX-512 EnabledAVX-512 Disabled60120180240300Min: 0.7 / Avg: 254.79 / Max: 334.43Min: 0.07 / Avg: 262.75 / Max: 330.69

SVT-AV1

CPU Temperature Monitor

OpenBenchmarking.orgCelsius, Fewer Is BetterSVT-AV1 2.3CPU Temperature MonitorAVX-512 EnabledAVX-512 Disabled1224364860Min: 37.88 / Avg: 59.2 / Max: 62.88Min: 36.63 / Avg: 58.79 / Max: 62.75

TensorFlow

Device: CPU - Batch Size: 256 - Model: ResNet-50

OpenBenchmarking.orgimages/sec, More Is BetterTensorFlow 2.16.1Device: CPU - Batch Size: 256 - Model: ResNet-50AVX-512 EnabledAVX-512 Disabled50100150200250SE +/- 0.23, N = 3SE +/- 0.30, N = 3212.82152.47

TensorFlow

CPU Peak Freq (Highest CPU Core Frequency) Monitor

OpenBenchmarking.orgMegahertz, More Is BetterTensorFlow 2.16.1CPU Peak Freq (Highest CPU Core Frequency) MonitorAVX-512 EnabledAVX-512 Disabled8001600240032004000Min: 2065 / Avg: 2823.71 / Max: 4564Min: 2372 / Avg: 2955.16 / Max: 4564

TensorFlow

CPU Power Consumption Monitor

OpenBenchmarking.orgWatts, Fewer Is BetterTensorFlow 2.16.1CPU Power Consumption MonitorAVX-512 EnabledAVX-512 Disabled50100150200250Min: 0.27 / Avg: 248.57 / Max: 270.85Min: 1.33 / Avg: 267.26 / Max: 298.64

TensorFlow

CPU Temperature Monitor

OpenBenchmarking.orgCelsius, Fewer Is BetterTensorFlow 2.16.1CPU Temperature MonitorAVX-512 EnabledAVX-512 Disabled1122334455Min: 38.63 / Avg: 47.61 / Max: 54.5Min: 37.38 / Avg: 48 / Max: 53.5

TensorFlow

Device: CPU - Batch Size: 512 - Model: ResNet-50

OpenBenchmarking.orgimages/sec, More Is BetterTensorFlow 2.16.1Device: CPU - Batch Size: 512 - Model: ResNet-50AVX-512 EnabledAVX-512 Disabled50100150200250SE +/- 0.01, N = 3SE +/- 0.06, N = 3241.22161.27

TensorFlow

CPU Peak Freq (Highest CPU Core Frequency) Monitor

OpenBenchmarking.orgMegahertz, More Is BetterTensorFlow 2.16.1CPU Peak Freq (Highest CPU Core Frequency) MonitorAVX-512 EnabledAVX-512 Disabled8001600240032004000Min: 2036 / Avg: 2748.97 / Max: 4564Min: 2253 / Avg: 2945.32 / Max: 4568

TensorFlow

CPU Power Consumption Monitor

OpenBenchmarking.orgWatts, Fewer Is BetterTensorFlow 2.16.1CPU Power Consumption MonitorAVX-512 EnabledAVX-512 Disabled60120180240300Min: 1.32 / Avg: 264.53 / Max: 295.65Min: 0.16 / Avg: 277.42 / Max: 315.43

TensorFlow

CPU Temperature Monitor

OpenBenchmarking.orgCelsius, Fewer Is BetterTensorFlow 2.16.1CPU Temperature MonitorAVX-512 EnabledAVX-512 Disabled1122334455Min: 40.5 / Avg: 50.21 / Max: 57.38Min: 39 / Avg: 50.44 / Max: 57.38

Y-Cruncher

Pi Digits To Calculate: 5B

OpenBenchmarking.orgSeconds, Fewer Is BetterY-Cruncher 0.8.5Pi Digits To Calculate: 5BAVX-512 EnabledAVX-512 Disabled816243240SE +/- 0.04, N = 3SE +/- 0.00, N = 321.8533.09

Y-Cruncher

CPU Peak Freq (Highest CPU Core Frequency) Monitor

OpenBenchmarking.orgMegahertz, More Is BetterY-Cruncher 0.8.5CPU Peak Freq (Highest CPU Core Frequency) MonitorAVX-512 EnabledAVX-512 Disabled8001600240032004000Min: 2075 / Avg: 3308.94 / Max: 4567Min: 2387 / Avg: 3380.84 / Max: 4566

Y-Cruncher

CPU Power Consumption Monitor

OpenBenchmarking.orgWatts, Fewer Is BetterY-Cruncher 0.8.5CPU Power Consumption MonitorAVX-512 EnabledAVX-512 Disabled70140210280350Min: 0.21 / Avg: 238.17 / Max: 383.47Min: 0.37 / Avg: 257.56 / Max: 387.52

Y-Cruncher

CPU Temperature Monitor

OpenBenchmarking.orgCelsius, Fewer Is BetterY-Cruncher 0.8.5CPU Temperature MonitorAVX-512 EnabledAVX-512 Disabled1122334455Min: 42.63 / Avg: 51.7 / Max: 57.13Min: 40.88 / Avg: 52.98 / Max: 57.13

Y-Cruncher

Pi Digits To Calculate: 10B

OpenBenchmarking.orgSeconds, Fewer Is BetterY-Cruncher 0.8.5Pi Digits To Calculate: 10BAVX-512 EnabledAVX-512 Disabled1530456075SE +/- 0.01, N = 3SE +/- 0.19, N = 344.0568.13

Y-Cruncher

CPU Peak Freq (Highest CPU Core Frequency) Monitor

OpenBenchmarking.orgMegahertz, More Is BetterY-Cruncher 0.8.5CPU Peak Freq (Highest CPU Core Frequency) MonitorAVX-512 EnabledAVX-512 Disabled8001600240032004000Min: 2056 / Avg: 3198.93 / Max: 4561Min: 2378 / Avg: 3385.41 / Max: 4566

Y-Cruncher

CPU Power Consumption Monitor

OpenBenchmarking.orgWatts, Fewer Is BetterY-Cruncher 0.8.5CPU Power Consumption MonitorAVX-512 EnabledAVX-512 Disabled70140210280350Min: 0.9 / Avg: 265.18 / Max: 384.27Min: 1.56 / Avg: 280.62 / Max: 389.7

Y-Cruncher

CPU Temperature Monitor

OpenBenchmarking.orgCelsius, Fewer Is BetterY-Cruncher 0.8.5CPU Temperature MonitorAVX-512 EnabledAVX-512 Disabled1224364860Min: 41.75 / Avg: 53.32 / Max: 61.5Min: 40.38 / Avg: 54.9 / Max: 61.38

Phoronix Test Suite v10.8.5