AMD EPYC 9754 Bergamo AVX-512

AMD EPYC 9754 1P benchmarks with AVX-512 benchmarking and then AVX-512 disabled. Tests by Michael Larabel for a future article.

Compare your own system(s) to this result file with the Phoronix Test Suite by running the command: phoronix-test-suite benchmark 2307197-NE-AMDBERGAM43
Jump To Table - Results

View

Do Not Show Noisy Results
Do Not Show Results With Incomplete Data
Do Not Show Results With Little Change/Spread
List Notable Results

Limit displaying results to tests within:

CPU Massive 3 Tests
Creator Workloads 5 Tests
HPC - High Performance Computing 5 Tests
Machine Learning 4 Tests
Multi-Core 6 Tests
Intel oneAPI 5 Tests
Python Tests 3 Tests
Server CPU Tests 2 Tests

Statistics

Show Overall Harmonic Mean(s)
Show Overall Geometric Mean
Show Geometric Means Per-Suite/Category
Show Wins / Losses Counts (Pie Chart)
Normalize Results
Remove Outliers Before Calculating Averages

Graph Settings

Force Line Graphs Where Applicable
Convert To Scalar Where Applicable
Prefer Vertical Bar Graphs
No Box Plots
On Line Graphs With Missing Data, Connect The Line Gaps

Multi-Way Comparison

Condense Multi-Option Tests Into Single Result Graphs
Condense Test Profiles With Multiple Version Results Into Single Result Graphs

Table

Show Detailed System Result Table

Run Management

Highlight
Result
Hide
Result
Result
Identifier
View Logs
Performance Per
Dollar
Date
Run
  Test
  Duration
AVX512 On
July 16 2023
  7 Hours, 54 Minutes
AVX512 Off
July 16 2023
  11 Hours, 27 Minutes
Invert Hiding All Results Option
  9 Hours, 40 Minutes
Only show results matching title/arguments (delimit multiple options with a comma):
Do not show results matching title/arguments (delimit multiple options with a comma):

AVX512 On

Kernel Notes: Transparent Huge Pages: madvise
Compiler Notes: --build=x86_64-linux-gnu --disable-vtable-verify --disable-werror --enable-bootstrap --enable-cet --enable-checking=release --enable-clocale=gnu --enable-default-pie --enable-gnu-unique-object --enable-languages=c,ada,c++,go,brig,d,fortran,objc,obj-c++,m2 --enable-libphobos-checking=release --enable-libstdcxx-debug --enable-libstdcxx-time=yes --enable-link-serialization=2 --enable-multiarch --enable-multilib --enable-nls --enable-objc-gc=auto --enable-offload-targets=nvptx-none=/build/gcc-11-xKiWfi/gcc-11-11.3.0/debian/tmp-nvptx/usr,amdgcn-amdhsa=/build/gcc-11-xKiWfi/gcc-11-11.3.0/debian/tmp-gcn/usr --enable-plugin --enable-shared --enable-threads=posix --host=x86_64-linux-gnu --program-prefix=x86_64-linux-gnu- --target=x86_64-linux-gnu --with-abi=m64 --with-arch-32=i686 --with-build-config=bootstrap-lto-lean --with-default-libstdcxx-abi=new --with-gcc-major-version-only --with-multilib-list=m32,m64,mx32 --with-target-system-zlib=auto --with-tune=generic --without-cuda-driver -v
Processor Notes: Scaling Governor: acpi-cpufreq performance (Boost: Enabled) - CPU Microcode: 0xaa0010b
Python Notes: Python 3.10.6
Security Notes: itlb_multihit: Not affected + l1tf: Not affected + mds: Not affected + meltdown: Not affected + mmio_stale_data: Not affected + retbleed: Not affected + spec_store_bypass: Mitigation of SSB disabled via prctl + spectre_v1: Mitigation of usercopy/swapgs barriers and __user pointer sanitization + spectre_v2: Mitigation of Retpolines IBPB: conditional IBRS_FW STIBP: always-on RSB filling PBRSB-eIBRS: Not affected + srbds: Not affected + tsx_async_abort: Not affected

AVX512 Off

Processor: AMD EPYC 9754 128-Core @ 2.25GHz (128 Cores / 256 Threads), Motherboard: AMD Titanite_4G (RTI1007B BIOS), Chipset: AMD Device 14a4, Memory: 768GB, Disk: 2 x 1920GB SAMSUNG MZWLJ1T9HBJR-00007, Graphics: ASPEED, Network: Broadcom NetXtreme BCM5720 PCIe

OS: Ubuntu 22.04, Kernel: 5.19.0-41-generic (x86_64), Desktop: GNOME Shell 42.5, Display Server: X Server 1.21.1.4, Vulkan: 1.3.224, Compiler: GCC 11.3.0, File-System: ext4, Screen Resolution: 1024x768

AMD EPYC 9754 Bergamo AVX-512OpenBenchmarking.orgPhoronix Test SuiteAMD EPYC 9754 128-Core @ 2.25GHz (128 Cores / 256 Threads)AMD Titanite_4G (RTI1007B BIOS)AMD Device 14a4768GB2 x 1920GB SAMSUNG MZWLJ1T9HBJR-00007ASPEEDBroadcom NetXtreme BCM5720 PCIeUbuntu 22.045.19.0-41-generic (x86_64)GNOME Shell 42.5X Server 1.21.1.41.3.224GCC 11.3.0ext41024x768ProcessorMotherboardChipsetMemoryDiskGraphicsNetworkOSKernelDesktopDisplay ServerVulkanCompilerFile-SystemScreen ResolutionAMD EPYC 9754 Bergamo AVX-512 BenchmarksSystem Logs- Transparent Huge Pages: madvise- --build=x86_64-linux-gnu --disable-vtable-verify --disable-werror --enable-bootstrap --enable-cet --enable-checking=release --enable-clocale=gnu --enable-default-pie --enable-gnu-unique-object --enable-languages=c,ada,c++,go,brig,d,fortran,objc,obj-c++,m2 --enable-libphobos-checking=release --enable-libstdcxx-debug --enable-libstdcxx-time=yes --enable-link-serialization=2 --enable-multiarch --enable-multilib --enable-nls --enable-objc-gc=auto --enable-offload-targets=nvptx-none=/build/gcc-11-xKiWfi/gcc-11-11.3.0/debian/tmp-nvptx/usr,amdgcn-amdhsa=/build/gcc-11-xKiWfi/gcc-11-11.3.0/debian/tmp-gcn/usr --enable-plugin --enable-shared --enable-threads=posix --host=x86_64-linux-gnu --program-prefix=x86_64-linux-gnu- --target=x86_64-linux-gnu --with-abi=m64 --with-arch-32=i686 --with-build-config=bootstrap-lto-lean --with-default-libstdcxx-abi=new --with-gcc-major-version-only --with-multilib-list=m32,m64,mx32 --with-target-system-zlib=auto --with-tune=generic --without-cuda-driver -v - Scaling Governor: acpi-cpufreq performance (Boost: Enabled) - CPU Microcode: 0xaa0010b - Python 3.10.6- itlb_multihit: Not affected + l1tf: Not affected + mds: Not affected + meltdown: Not affected + mmio_stale_data: Not affected + retbleed: Not affected + spec_store_bypass: Mitigation of SSB disabled via prctl + spectre_v1: Mitigation of usercopy/swapgs barriers and __user pointer sanitization + spectre_v2: Mitigation of Retpolines IBPB: conditional IBRS_FW STIBP: always-on RSB filling PBRSB-eIBRS: Not affected + srbds: Not affected + tsx_async_abort: Not affected

AVX512 On vs. AVX512 Off ComparisonPhoronix Test SuiteBaseline+346.4%+346.4%+692.8%+692.8%+1039.2%+1039.2%CPU - 512 - GoogLeNet789.3%CPU - 64 - AlexNet778.7%CPU - 32 - AlexNet562.1%CPU - 256 - ResNet-50499.9%CPU - 64 - GoogLeNet499%CPU - 512 - ResNet-50491%CPU - 64 - ResNet-50423.7%CPU - 16 - AlexNet389.1%CPU - 32 - ResNet-50309.7%CPU - 32 - GoogLeNet306.6%CPU - 16 - ResNet-50171%CPU - 16 - GoogLeNet164.3%W.P.D.F - CPU138.2%W.P.D.F - CPU138%F.D.F - CPU132%F.D.F - CPU131.2%M.T.E.T.D.F - CPU130.3%M.T.E.T.D.F - CPU130.2%W.P.D.F.I - CPU107.7%W.P.D.F.I - CPU107.6%F.D.F.I - CPU103.6%F.D.F.I - CPU102.6%P.V.B.D.F - CPU100.2%P.V.B.D.F - CPU100.1%CPU - 512 - AlexNet1385.6%CPU - 256 - AlexNet1238.3%CPU - 256 - GoogLeNet986.6%LBC, LBRY Credits98.6%A.G.R.R.0.F - CPU92.9%N.S.A.8.P.Q.B.B.U - A.M.S92.3%N.S.A.8.P.Q.B.B.U - A.M.S92.2%C.S.9.P.Y.P - A.M.S83.3%C.S.9.P.Y.P - A.M.S81.9%P.D.F - CPU80.2%P.D.F - CPU79.8%P.D.F - CPU78.2%P.D.F - CPU77.9%A.G.R.R.0.F - CPU76.2%gravity_spheres_volume/dim_512/scivis/real_time75.8%Q.S.2.P71.1%gravity_spheres_volume/dim_512/ao/real_time70.8%x25x65.4%Blake-2 S58.9%scrypt47.2%V.D.F.I - CPU43.9%V.D.F.I - CPU43.6%gravity_spheres_volume/dim_512/pathtracer/real_time43.1%25638.4%Garlicoin34.5%OpenMP - BM231.9%OpenMP - BM231.9%OpenMP - BM126.7%OpenMP - BM126.7%Skeincoin25.3%N.T.C.B.b.u.S - A.M.S21.2%N.T.C.B.b.u.S - A.M.S21.1%Myriad-Groestl20.7%V.D.F - CPU20.1%V.D.F - CPU20%N.D.C.o.b.u.o.I - A.M.S19.8%N.T.C.D.m - A.M.S19.7%N.T.C.D.m - A.M.S19.6%N.T.C.B.b.u.c - A.M.S19.6%N.T.C.B.b.u.c - A.M.S19.3%N.D.C.o.b.u.o.I - A.M.S19.2%vklBenchmark ISPC18.6%Pathtracer ISPC - Asian Dragon18.6%Pathtracer ISPC - Asian Dragon Obj17%N.Q.A.B.b.u.S.1.P - A.M.S16.1%N.Q.A.B.b.u.S.1.P - A.M.S14.8%R.N.N.T - bf16bf16bf16 - CPU12.2%Pathtracer ISPC - Crown11.9%A.G.R.R.0.F.I - CPU11.4%C.C.R.5.I - A.M.S11.4%C.C.R.5.I - A.M.S11.3%A.G.R.R.0.F.I - CPU10.6%1284.6%TensorFlowTensorFlowTensorFlowTensorFlowTensorFlowTensorFlowTensorFlowTensorFlowTensorFlowTensorFlowTensorFlowTensorFlowOpenVINOOpenVINOOpenVINOOpenVINOOpenVINOOpenVINOOpenVINOOpenVINOOpenVINOOpenVINOOpenVINOOpenVINOTensorFlowTensorFlowTensorFlowCpuminer-OptOpenVINONeural Magic DeepSparseNeural Magic DeepSparseNeural Magic DeepSparseNeural Magic DeepSparseOpenVINOOpenVINOOpenVINOOpenVINOOpenVINOOSPRayCpuminer-OptOSPRayCpuminer-OptCpuminer-OptCpuminer-OptOpenVINOOpenVINOOSPRaylibxsmmCpuminer-OptminiBUDEminiBUDEminiBUDEminiBUDECpuminer-OptNeural Magic DeepSparseNeural Magic DeepSparseCpuminer-OptOpenVINOOpenVINONeural Magic DeepSparseNeural Magic DeepSparseNeural Magic DeepSparseNeural Magic DeepSparseNeural Magic DeepSparseNeural Magic DeepSparseOpenVKLEmbreeEmbreeNeural Magic DeepSparseNeural Magic DeepSparseoneDNNEmbreeOpenVINONeural Magic DeepSparseNeural Magic DeepSparseOpenVINOlibxsmmAVX512 OnAVX512 Off

TensorFlow

This is a benchmark of the TensorFlow deep learning framework using the TensorFlow reference benchmarks (tensorflow/benchmarks with tf_cnn_benchmarks.py). Note with the Phoronix Test Suite there is also pts/tensorflow-lite for benchmarking the TensorFlow Lite binaries if desired for complementary metrics. Learn more via the OpenBenchmarking.org test page.

OpenBenchmarking.orgimages/sec, More Is BetterTensorFlow 2.12Device: CPU - Batch Size: 512 - Model: ResNet-50AVX512 OnAVX512 Off306090120150SE +/- 0.99, N = 3SE +/- 0.04, N = 3122.8120.78
OpenBenchmarking.orgimages/sec Per Watt, More Is BetterTensorFlow 2.12Device: CPU - Batch Size: 512 - Model: ResNet-50AVX512 OnAVX512 Off0.1220.2440.3660.4880.610.5420.148
OpenBenchmarking.orgMegahertz, More Is BetterTensorFlow 2.12CPU Peak Freq (Highest CPU Core Frequency) MonitorAVX512 OnAVX512 Off6001200180024003000Min: 2250 / Avg: 2958.12 / Max: 3370Min: 2243 / Avg: 3085.93 / Max: 3295
OpenBenchmarking.orgWatts, Fewer Is BetterTensorFlow 2.12CPU Power Consumption MonitorAVX512 OnAVX512 Off50100150200250Min: 20.5 / Avg: 226.78 / Max: 289.01Min: 20.22 / Avg: 140.06 / Max: 209.421. AVX512 On: Approximate power consumption of 102427 Joules per run.2. AVX512 Off: Approximate power consumption of 378535 Joules per run.
OpenBenchmarking.orgCelsius, Fewer Is BetterTensorFlow 2.12CPU Temperature MonitorAVX512 OnAVX512 Off1122334455Min: 35.38 / Avg: 48.79 / Max: 57.5Min: 29.38 / Avg: 39.11 / Max: 54.13

OpenBenchmarking.orgimages/sec, More Is BetterTensorFlow 2.12Device: CPU - Batch Size: 256 - Model: ResNet-50AVX512 OnAVX512 Off306090120150SE +/- 1.01, N = 12SE +/- 0.02, N = 3119.3219.89
OpenBenchmarking.orgimages/sec Per Watt, More Is BetterTensorFlow 2.12Device: CPU - Batch Size: 256 - Model: ResNet-50AVX512 OnAVX512 Off0.12080.24160.36240.48320.6040.5370.143
OpenBenchmarking.orgMegahertz, More Is BetterTensorFlow 2.12CPU Peak Freq (Highest CPU Core Frequency) MonitorAVX512 OnAVX512 Off6001200180024003000Min: 2250 / Avg: 2956.33 / Max: 3308Min: 2250 / Avg: 3085.27 / Max: 3559
OpenBenchmarking.orgWatts, Fewer Is BetterTensorFlow 2.12CPU Power Consumption MonitorAVX512 OnAVX512 Off50100150200250Min: 20.59 / Avg: 222.08 / Max: 272.77Min: 20.06 / Avg: 139.18 / Max: 198.981. AVX512 On: Approximate power consumption of 52984 Joules per run.2. AVX512 Off: Approximate power consumption of 197083 Joules per run.
OpenBenchmarking.orgCelsius, Fewer Is BetterTensorFlow 2.12CPU Temperature MonitorAVX512 OnAVX512 Off1122334455Min: 37.63 / Avg: 48.39 / Max: 57.63Min: 29 / Avg: 38.33 / Max: 53.13

OpenBenchmarking.orgimages/sec, More Is BetterTensorFlow 2.12Device: CPU - Batch Size: 512 - Model: GoogLeNetAVX512 OnAVX512 Off90180270360450SE +/- 5.30, N = 12SE +/- 0.05, N = 3417.2546.92
OpenBenchmarking.orgimages/sec Per Watt, More Is BetterTensorFlow 2.12Device: CPU - Batch Size: 512 - Model: GoogLeNetAVX512 OnAVX512 Off0.39740.79481.19221.58961.9871.7660.369
OpenBenchmarking.orgMegahertz, More Is BetterTensorFlow 2.12CPU Peak Freq (Highest CPU Core Frequency) MonitorAVX512 OnAVX512 Off6001200180024003000Min: 2250 / Avg: 2973.8 / Max: 3207Min: 2250 / Avg: 3083.42 / Max: 3336
OpenBenchmarking.orgWatts, Fewer Is BetterTensorFlow 2.12CPU Power Consumption MonitorAVX512 OnAVX512 Off50100150200250Min: 10.84 / Avg: 236.33 / Max: 283.4Min: 19.97 / Avg: 127.28 / Max: 218.041. AVX512 On: Approximate power consumption of 32654 Joules per run.2. AVX512 Off: Approximate power consumption of 152954 Joules per run.
OpenBenchmarking.orgCelsius, Fewer Is BetterTensorFlow 2.12CPU Temperature MonitorAVX512 OnAVX512 Off1224364860Min: 34.63 / Avg: 50.93 / Max: 58.38Min: 30.13 / Avg: 36.83 / Max: 51.13

libxsmm

Libxsmm is an open-source library for specialized dense and sparse matrix operations and deep learning primitives. Libxsmm supports making use of Intel AMX, AVX-512, and other modern CPU instruction set capabilities. Learn more via the OpenBenchmarking.org test page.

OpenBenchmarking.orgGFLOPS/s, More Is Betterlibxsmm 2-1.17-3645M N K: 128AVX512 OnAVX512 Off6001200180024003000SE +/- 12.20, N = 3SE +/- 2.70, N = 32690.72573.31. (CXX) g++ options: -dynamic -Bstatic -static-libgcc -lgomp -lm -lrt -ldl -lquadmath -lstdc++ -pthread -fPIC -std=c++14 -O2 -fopenmp-simd -funroll-loops -ftree-vectorize -fdata-sections -ffunction-sections -fvisibility=hidden -msse4.2
OpenBenchmarking.orgGFLOPS/s Per Watt, More Is Betterlibxsmm 2-1.17-3645M N K: 128AVX512 OnAVX512 Off369121513.0712.53
OpenBenchmarking.orgMegahertz, More Is Betterlibxsmm 2-1.17-3645CPU Peak Freq (Highest CPU Core Frequency) MonitorAVX512 OnAVX512 Off5001000150020002500Min: 2250 / Avg: 3087.64 / Max: 3133Min: 2250 / Avg: 3086 / Max: 3117
OpenBenchmarking.orgWatts, Fewer Is Betterlibxsmm 2-1.17-3645CPU Power Consumption MonitorAVX512 OnAVX512 Off60120180240300Min: 20.28 / Avg: 205.93 / Max: 329.89Min: 20.58 / Avg: 205.43 / Max: 327.081. AVX512 On: Approximate power consumption of 106125 Joules per run.2. AVX512 Off: Approximate power consumption of 105932 Joules per run.
OpenBenchmarking.orgCelsius, Fewer Is Betterlibxsmm 2-1.17-3645CPU Temperature MonitorAVX512 OnAVX512 Off1122334455Min: 35.75 / Avg: 50.34 / Max: 53.25Min: 35.5 / Avg: 49.14 / Max: 50.88

OpenVKL

OpenVKL is the Intel Open Volume Kernel Library that offers high-performance volume computation kernels and part of the Intel oneAPI rendering toolkit. Learn more via the OpenBenchmarking.org test page.

OpenBenchmarking.orgItems / Sec, More Is BetterOpenVKL 1.3.1Benchmark: vklBenchmark ISPCAVX512 OnAVX512 Off30060090012001500SE +/- 2.65, N = 3SE +/- 0.33, N = 313981179MIN: 229 / MAX: 11779MIN: 178 / MAX: 10473
OpenBenchmarking.orgItems / Sec Per Watt, More Is BetterOpenVKL 1.3.1Benchmark: vklBenchmark ISPCAVX512 OnAVX512 Off1.31812.63623.95435.27246.59055.8584.569
OpenBenchmarking.orgMegahertz, More Is BetterOpenVKL 1.3.1CPU Peak Freq (Highest CPU Core Frequency) MonitorAVX512 OnAVX512 Off5001000150020002500Min: 2250 / Avg: 2983.53 / Max: 3118Min: 2250 / Avg: 2958.42 / Max: 3128
MinAvgMaxAVX512 On19.9238.7322.2AVX512 Off10.2258.1348.2OpenBenchmarking.orgWatts, Fewer Is BetterOpenVKL 1.3.1CPU Power Consumption Monitor1002003004005001. AVX512 On: Approximate power consumption of 95462 Joules per run.2. AVX512 Off: Approximate power consumption of 105723 Joules per run.
OpenBenchmarking.orgCelsius, Fewer Is BetterOpenVKL 1.3.1CPU Temperature MonitorAVX512 OnAVX512 Off1326395265Min: 30.63 / Avg: 53.11 / Max: 62.25Min: 30.38 / Avg: 54.49 / Max: 64.75

TensorFlow

This is a benchmark of the TensorFlow deep learning framework using the TensorFlow reference benchmarks (tensorflow/benchmarks with tf_cnn_benchmarks.py). Note with the Phoronix Test Suite there is also pts/tensorflow-lite for benchmarking the TensorFlow Lite binaries if desired for complementary metrics. Learn more via the OpenBenchmarking.org test page.

OpenBenchmarking.orgimages/sec, More Is BetterTensorFlow 2.12Device: CPU - Batch Size: 256 - Model: GoogLeNetAVX512 OnAVX512 Off110220330440550SE +/- 4.92, N = 3SE +/- 0.06, N = 3501.6746.17
OpenBenchmarking.orgimages/sec Per Watt, More Is BetterTensorFlow 2.12Device: CPU - Batch Size: 256 - Model: GoogLeNetAVX512 OnAVX512 Off0.46760.93521.40281.87042.3382.0780.362
MinAvgMaxAVX512 On225029163157AVX512 Off225030863417OpenBenchmarking.orgMegahertz, More Is BetterTensorFlow 2.12CPU Peak Freq (Highest CPU Core Frequency) Monitor8001600240032004000
MinAvgMaxAVX512 On19.9241.5277.9AVX512 Off20.1127.4214.3OpenBenchmarking.orgWatts, Fewer Is BetterTensorFlow 2.12CPU Power Consumption Monitor701402102803501. AVX512 On: Approximate power consumption of 15453 Joules per run.2. AVX512 Off: Approximate power consumption of 78367 Joules per run.
MinAvgMaxAVX512 On34.450.958.9AVX512 Off29.836.452.1OpenBenchmarking.orgCelsius, Fewer Is BetterTensorFlow 2.12CPU Temperature Monitor1632486480

OpenVINO

This is a test of the Intel OpenVINO, a toolkit around neural networks, using its built-in benchmarking support and analyzing the throughput and latency for various models. Learn more via the OpenBenchmarking.org test page.

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2022.3Model: Machine Translation EN To DE FP16 - Device: CPUAVX512 OnAVX512 Off60120180240300SE +/- 1.22, N = 15SE +/- 2.83, N = 15110.35254.01MIN: 49.7 / MAX: 183.59MIN: 116.71 / MAX: 398.881. (CXX) g++ options: -isystem -fsigned-char -ffunction-sections -fdata-sections -msse4.1 -msse4.2 -O3 -fno-strict-overflow -fwrapv -fPIC -fvisibility=hidden -Os -std=c++11 -MD -MT -MF
MinAvgMaxAVX512 On225026613111AVX512 Off221425373101OpenBenchmarking.orgMegahertz, More Is BetterOpenVINO 2022.3CPU Peak Freq (Highest CPU Core Frequency) Monitor8001600240032004000
MinAvgMaxAVX512 On10.9295.1345.9AVX512 Off21.2292.9331.3OpenBenchmarking.orgWatts, Fewer Is BetterOpenVINO 2022.3CPU Power Consumption Monitor1002003004005001. AVX512 On: Approximate power consumption of 19339 Joules per run.2. AVX512 Off: Approximate power consumption of 19276 Joules per run.
MinAvgMaxAVX512 On40.557.365.8AVX512 Off41.056.764.6OpenBenchmarking.orgCelsius, Fewer Is BetterOpenVINO 2022.3CPU Temperature Monitor20406080100

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2022.3Model: Machine Translation EN To DE FP16 - Device: CPUAVX512 OnAVX512 Off130260390520650SE +/- 7.10, N = 15SE +/- 3.15, N = 15580.40251.971. (CXX) g++ options: -isystem -fsigned-char -ffunction-sections -fdata-sections -msse4.1 -msse4.2 -O3 -fno-strict-overflow -fwrapv -fPIC -fvisibility=hidden -Os -std=c++11 -MD -MT -MF

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2022.3Model: Vehicle Detection FP16 - Device: CPUAVX512 OnAVX512 Off1224364860SE +/- 0.60, N = 15SE +/- 0.62, N = 1444.8453.79MIN: 8.1 / MAX: 137.01MIN: 13.85 / MAX: 136.831. (CXX) g++ options: -isystem -fsigned-char -ffunction-sections -fdata-sections -msse4.1 -msse4.2 -O3 -fno-strict-overflow -fwrapv -fPIC -fvisibility=hidden -Os -std=c++11 -MD -MT -MF
MinAvgMaxAVX512 On225030293257AVX512 Off224129183105OpenBenchmarking.orgMegahertz, More Is BetterOpenVINO 2022.3CPU Peak Freq (Highest CPU Core Frequency) Monitor8001600240032004000
MinAvgMaxAVX512 On20.7256.0333.9AVX512 Off21.0266.8326.8OpenBenchmarking.orgWatts, Fewer Is BetterOpenVINO 2022.3CPU Power Consumption Monitor801602403204001. AVX512 On: Approximate power consumption of 16419 Joules per run.2. AVX512 Off: Approximate power consumption of 17056 Joules per run.
MinAvgMaxAVX512 On36.851.757.5AVX512 Off36.853.962.3OpenBenchmarking.orgCelsius, Fewer Is BetterOpenVINO 2022.3CPU Temperature Monitor20406080100

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2022.3Model: Vehicle Detection FP16 - Device: CPUAVX512 OnAVX512 Off30060090012001500SE +/- 22.93, N = 15SE +/- 15.59, N = 141430.451190.911. (CXX) g++ options: -isystem -fsigned-char -ffunction-sections -fdata-sections -msse4.1 -msse4.2 -O3 -fno-strict-overflow -fwrapv -fPIC -fvisibility=hidden -Os -std=c++11 -MD -MT -MF

TensorFlow

This is a benchmark of the TensorFlow deep learning framework using the TensorFlow reference benchmarks (tensorflow/benchmarks with tf_cnn_benchmarks.py). Note with the Phoronix Test Suite there is also pts/tensorflow-lite for benchmarking the TensorFlow Lite binaries if desired for complementary metrics. Learn more via the OpenBenchmarking.org test page.

OpenBenchmarking.orgimages/sec, More Is BetterTensorFlow 2.12Device: CPU - Batch Size: 512 - Model: AlexNetAVX512 OnAVX512 Off400800120016002000SE +/- 1.53, N = 3SE +/- 0.06, N = 31632.40109.88
OpenBenchmarking.orgimages/sec Per Watt, More Is BetterTensorFlow 2.12Device: CPU - Batch Size: 512 - Model: AlexNetAVX512 OnAVX512 Off2468107.0970.865
MinAvgMaxAVX512 On225027543095AVX512 Off220330793359OpenBenchmarking.orgMegahertz, More Is BetterTensorFlow 2.12CPU Peak Freq (Highest CPU Core Frequency) Monitor8001600240032004000
MinAvgMaxAVX512 On20.3230.0283.1AVX512 Off20.3127.0189.9OpenBenchmarking.orgWatts, Fewer Is BetterTensorFlow 2.12CPU Power Consumption Monitor701402102803501. AVX512 On: Approximate power consumption of 9354 Joules per run.2. AVX512 Off: Approximate power consumption of 65553 Joules per run.
MinAvgMaxAVX512 On33.949.855.8AVX512 Off28.836.247.4OpenBenchmarking.orgCelsius, Fewer Is BetterTensorFlow 2.12CPU Temperature Monitor1632486480

OpenBenchmarking.orgimages/sec, More Is BetterTensorFlow 2.12Device: CPU - Batch Size: 64 - Model: ResNet-50AVX512 OnAVX512 Off20406080100SE +/- 0.06, N = 3SE +/- 0.03, N = 396.6318.45
OpenBenchmarking.orgimages/sec Per Watt, More Is BetterTensorFlow 2.12Device: CPU - Batch Size: 64 - Model: ResNet-50AVX512 OnAVX512 Off0.10870.21740.32610.43480.54350.4830.131
MinAvgMaxAVX512 On225030213283AVX512 Off225030903249OpenBenchmarking.orgMegahertz, More Is BetterTensorFlow 2.12CPU Peak Freq (Highest CPU Core Frequency) Monitor8001600240032004000
MinAvgMaxAVX512 On20.6200.2230.6AVX512 Off20.1140.4179.7OpenBenchmarking.orgWatts, Fewer Is BetterTensorFlow 2.12CPU Power Consumption Monitor601201802403001. AVX512 On: Approximate power consumption of 16285 Joules per run.2. AVX512 Off: Approximate power consumption of 54525 Joules per run.
MinAvgMaxAVX512 On33.545.353.3AVX512 Off28.838.151.6OpenBenchmarking.orgCelsius, Fewer Is BetterTensorFlow 2.12CPU Temperature Monitor1530456075

OpenVINO

This is a test of the Intel OpenVINO, a toolkit around neural networks, using its built-in benchmarking support and analyzing the throughput and latency for various models. Learn more via the OpenBenchmarking.org test page.

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2022.3Model: Person Detection FP32 - Device: CPUAVX512 OnAVX512 Off9001800270036004500SE +/- 14.83, N = 12SE +/- 41.79, N = 52339.814170.49MIN: 1045.35 / MAX: 3232.69MIN: 1759.65 / MAX: 4969.51. (CXX) g++ options: -isystem -fsigned-char -ffunction-sections -fdata-sections -msse4.1 -msse4.2 -O3 -fno-strict-overflow -fwrapv -fPIC -fvisibility=hidden -Os -std=c++11 -MD -MT -MF
MinAvgMaxAVX512 On225029363426AVX512 Off225026453103OpenBenchmarking.orgMegahertz, More Is BetterOpenVINO 2022.3CPU Peak Freq (Highest CPU Core Frequency) Monitor8001600240032004000
MinAvgMaxAVX512 On20.8306.2358.4AVX512 Off21.0292.7340.3OpenBenchmarking.orgWatts, Fewer Is BetterOpenVINO 2022.3CPU Power Consumption Monitor1002003004005001. AVX512 On: Approximate power consumption of 21691 Joules per run.2. AVX512 Off: Approximate power consumption of 21659 Joules per run.
MinAvgMaxAVX512 On37.660.672.9AVX512 Off36.558.072.4OpenBenchmarking.orgCelsius, Fewer Is BetterOpenVINO 2022.3CPU Temperature Monitor20406080100

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2022.3Model: Person Detection FP32 - Device: CPUAVX512 OnAVX512 Off612182430SE +/- 0.18, N = 12SE +/- 0.16, N = 527.0114.991. (CXX) g++ options: -isystem -fsigned-char -ffunction-sections -fdata-sections -msse4.1 -msse4.2 -O3 -fno-strict-overflow -fwrapv -fPIC -fvisibility=hidden -Os -std=c++11 -MD -MT -MF

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2022.3Model: Person Vehicle Bike Detection FP16 - Device: CPUAVX512 OnAVX512 Off510152025SE +/- 0.02, N = 3SE +/- 0.14, N = 159.6319.28MIN: 6.4 / MAX: 33.26MIN: 10.31 / MAX: 50.611. (CXX) g++ options: -isystem -fsigned-char -ffunction-sections -fdata-sections -msse4.1 -msse4.2 -O3 -fno-strict-overflow -fwrapv -fPIC -fvisibility=hidden -Os -std=c++11 -MD -MT -MF
MinAvgMaxAVX512 On225026793096AVX512 Off225024833131OpenBenchmarking.orgMegahertz, More Is BetterOpenVINO 2022.3CPU Peak Freq (Highest CPU Core Frequency) Monitor8001600240032004000
MinAvgMaxAVX512 On21.6308.5346.7AVX512 Off21.1293.4328.1OpenBenchmarking.orgWatts, Fewer Is BetterOpenVINO 2022.3CPU Power Consumption Monitor1002003004005001. AVX512 On: Approximate power consumption of 20465 Joules per run.2. AVX512 Off: Approximate power consumption of 19264 Joules per run.
MinAvgMaxAVX512 On40.560.666.6AVX512 Off41.057.164.8OpenBenchmarking.orgCelsius, Fewer Is BetterOpenVINO 2022.3CPU Temperature Monitor20406080100

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2022.3Model: Person Vehicle Bike Detection FP16 - Device: CPUAVX512 OnAVX512 Off14002800420056007000SE +/- 13.51, N = 3SE +/- 26.07, N = 156638.713317.341. (CXX) g++ options: -isystem -fsigned-char -ffunction-sections -fdata-sections -msse4.1 -msse4.2 -O3 -fno-strict-overflow -fwrapv -fPIC -fvisibility=hidden -Os -std=c++11 -MD -MT -MF

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2022.3Model: Vehicle Detection FP16-INT8 - Device: CPUAVX512 OnAVX512 Off48121620SE +/- 0.15, N = 15SE +/- 0.01, N = 311.2616.17MIN: 4.37 / MAX: 45.06MIN: 8.44 / MAX: 56.881. (CXX) g++ options: -isystem -fsigned-char -ffunction-sections -fdata-sections -msse4.1 -msse4.2 -O3 -fno-strict-overflow -fwrapv -fPIC -fvisibility=hidden -Os -std=c++11 -MD -MT -MF
MinAvgMaxAVX512 On225027053121AVX512 Off225026593098OpenBenchmarking.orgMegahertz, More Is BetterOpenVINO 2022.3CPU Peak Freq (Highest CPU Core Frequency) Monitor8001600240032004000
MinAvgMaxAVX512 On20.8306.3336.2AVX512 Off21.2330.4358.0OpenBenchmarking.orgWatts, Fewer Is BetterOpenVINO 2022.3CPU Power Consumption Monitor1002003004005001. AVX512 On: Approximate power consumption of 19708 Joules per run.2. AVX512 Off: Approximate power consumption of 21257 Joules per run.
MinAvgMaxAVX512 On40.059.266.6AVX512 Off42.167.772.8OpenBenchmarking.orgCelsius, Fewer Is BetterOpenVINO 2022.3CPU Temperature Monitor20406080100

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2022.3Model: Vehicle Detection FP16-INT8 - Device: CPUAVX512 OnAVX512 Off12002400360048006000SE +/- 89.26, N = 15SE +/- 2.12, N = 35690.343954.901. (CXX) g++ options: -isystem -fsigned-char -ffunction-sections -fdata-sections -msse4.1 -msse4.2 -O3 -fno-strict-overflow -fwrapv -fPIC -fvisibility=hidden -Os -std=c++11 -MD -MT -MF

libxsmm

Libxsmm is an open-source library for specialized dense and sparse matrix operations and deep learning primitives. Libxsmm supports making use of Intel AMX, AVX-512, and other modern CPU instruction set capabilities. Learn more via the OpenBenchmarking.org test page.

OpenBenchmarking.orgGFLOPS/s, More Is Betterlibxsmm 2-1.17-3645M N K: 256AVX512 OnAVX512 Off7001400210028003500SE +/- 5.78, N = 3SE +/- 6.53, N = 33342.52415.31. (CXX) g++ options: -dynamic -Bstatic -static-libgcc -lgomp -lm -lrt -ldl -lquadmath -lstdc++ -pthread -fPIC -std=c++14 -O2 -fopenmp-simd -funroll-loops -ftree-vectorize -fdata-sections -ffunction-sections -fvisibility=hidden -msse4.2
OpenBenchmarking.orgGFLOPS/s Per Watt, More Is Betterlibxsmm 2-1.17-3645M N K: 256AVX512 OnAVX512 Off4812162015.9011.40
MinAvgMaxAVX512 On225030543135AVX512 Off223930383143OpenBenchmarking.orgMegahertz, More Is Betterlibxsmm 2-1.17-3645CPU Peak Freq (Highest CPU Core Frequency) Monitor8001600240032004000
MinAvgMaxAVX512 On20.5210.2340.2AVX512 Off20.7211.9335.7OpenBenchmarking.orgWatts, Fewer Is Betterlibxsmm 2-1.17-3645CPU Power Consumption Monitor801602403204001. AVX512 On: Approximate power consumption of 38260 Joules per run.2. AVX512 Off: Approximate power consumption of 39198 Joules per run.
MinAvgMaxAVX512 On34.449.754.3AVX512 Off33.549.453.3OpenBenchmarking.orgCelsius, Fewer Is Betterlibxsmm 2-1.17-3645CPU Temperature Monitor1530456075

OpenVINO

This is a test of the Intel OpenVINO, a toolkit around neural networks, using its built-in benchmarking support and analyzing the throughput and latency for various models. Learn more via the OpenBenchmarking.org test page.

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2022.3Model: Person Detection FP16 - Device: CPUAVX512 OnAVX512 Off9001800270036004500SE +/- 22.79, N = 12SE +/- 59.43, N = 32334.294153.59MIN: 1017.83 / MAX: 3101.05MIN: 1975.28 / MAX: 5152.661. (CXX) g++ options: -isystem -fsigned-char -ffunction-sections -fdata-sections -msse4.1 -msse4.2 -O3 -fno-strict-overflow -fwrapv -fPIC -fvisibility=hidden -Os -std=c++11 -MD -MT -MF
MinAvgMaxAVX512 On225029313185AVX512 Off225026493132OpenBenchmarking.orgMegahertz, More Is BetterOpenVINO 2022.3CPU Peak Freq (Highest CPU Core Frequency) Monitor8001600240032004000
MinAvgMaxAVX512 On19.8306.5359.5AVX512 Off21.3283.3329.0OpenBenchmarking.orgWatts, Fewer Is BetterOpenVINO 2022.3CPU Power Consumption Monitor1002003004005001. AVX512 On: Approximate power consumption of 21785 Joules per run.2. AVX512 Off: Approximate power consumption of 21245 Joules per run.
MinAvgMaxAVX512 On41.361.273.4AVX512 Off37.657.467.6OpenBenchmarking.orgCelsius, Fewer Is BetterOpenVINO 2022.3CPU Temperature Monitor20406080100

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2022.3Model: Person Detection FP16 - Device: CPUAVX512 OnAVX512 Off612182430SE +/- 0.30, N = 12SE +/- 0.21, N = 327.0815.061. (CXX) g++ options: -isystem -fsigned-char -ffunction-sections -fdata-sections -msse4.1 -msse4.2 -O3 -fno-strict-overflow -fwrapv -fPIC -fvisibility=hidden -Os -std=c++11 -MD -MT -MF

TensorFlow

This is a benchmark of the TensorFlow deep learning framework using the TensorFlow reference benchmarks (tensorflow/benchmarks with tf_cnn_benchmarks.py). Note with the Phoronix Test Suite there is also pts/tensorflow-lite for benchmarking the TensorFlow Lite binaries if desired for complementary metrics. Learn more via the OpenBenchmarking.org test page.

OpenBenchmarking.orgimages/sec, More Is BetterTensorFlow 2.12Device: CPU - Batch Size: 64 - Model: GoogLeNetAVX512 OnAVX512 Off60120180240300SE +/- 2.62, N = 15SE +/- 0.18, N = 3277.2446.28
OpenBenchmarking.orgimages/sec Per Watt, More Is BetterTensorFlow 2.12Device: CPU - Batch Size: 64 - Model: GoogLeNetAVX512 OnAVX512 Off0.33210.66420.99631.32841.66051.4760.369
MinAvgMaxAVX512 On225029953532AVX512 Off225030753309OpenBenchmarking.orgMegahertz, More Is BetterTensorFlow 2.12CPU Peak Freq (Highest CPU Core Frequency) Monitor10002000300040005000
MinAvgMaxAVX512 On20.1187.9236.9AVX512 Off19.9125.3152.5OpenBenchmarking.orgWatts, Fewer Is BetterTensorFlow 2.12CPU Power Consumption Monitor601201802403001. AVX512 On: Approximate power consumption of 6163 Joules per run.2. AVX512 Off: Approximate power consumption of 19961 Joules per run.
MinAvgMaxAVX512 On32.544.352.8AVX512 Off29.935.949.0OpenBenchmarking.orgCelsius, Fewer Is BetterTensorFlow 2.12CPU Temperature Monitor1530456075

OpenBenchmarking.orgimages/sec, More Is BetterTensorFlow 2.12Device: CPU - Batch Size: 256 - Model: AlexNetAVX512 OnAVX512 Off30060090012001500SE +/- 6.55, N = 3SE +/- 0.24, N = 31422.36106.28
OpenBenchmarking.orgimages/sec Per Watt, More Is BetterTensorFlow 2.12Device: CPU - Batch Size: 256 - Model: AlexNetAVX512 OnAVX512 Off2468106.9840.839
MinAvgMaxAVX512 On225027423102AVX512 Off225030733166OpenBenchmarking.orgMegahertz, More Is BetterTensorFlow 2.12CPU Peak Freq (Highest CPU Core Frequency) Monitor8001600240032004000
MinAvgMaxAVX512 On20.1203.7257.0AVX512 Off20.0126.7155.1OpenBenchmarking.orgWatts, Fewer Is BetterTensorFlow 2.12CPU Power Consumption Monitor701402102803501. AVX512 On: Approximate power consumption of 5160 Joules per run.2. AVX512 Off: Approximate power consumption of 34203 Joules per run.
MinAvgMaxAVX512 On30.944.852.3AVX512 Off28.836.047.9OpenBenchmarking.orgCelsius, Fewer Is BetterTensorFlow 2.12CPU Temperature Monitor1530456075

Neural Magic DeepSparse

This is a benchmark of Neural Magic's DeepSparse using its built-in deepsparse.benchmark utility and various models from their SparseZoo (https://sparsezoo.neuralmagic.com/). Learn more via the OpenBenchmarking.org test page.

OpenBenchmarking.orgms/batch, Fewer Is BetterNeural Magic DeepSparse 1.5Model: NLP Question Answering, BERT base uncased SQuaD 12layer Pruned90 - Scenario: Asynchronous Multi-StreamAVX512 OnAVX512 Off60120180240300SE +/- 6.02, N = 15SE +/- 0.24, N = 3259.65298.06
MinAvgMaxAVX512 On225030023416AVX512 Off225030233152OpenBenchmarking.orgMegahertz, More Is BetterNeural Magic DeepSparse 1.5CPU Peak Freq (Highest CPU Core Frequency) Monitor8001600240032004000
MinAvgMaxAVX512 On10.5234.1342.4AVX512 Off21.0228.6329.0OpenBenchmarking.orgWatts, Fewer Is BetterNeural Magic DeepSparse 1.5CPU Power Consumption Monitor801602403204001. AVX512 On: Approximate power consumption of 11484 Joules per run.2. AVX512 Off: Approximate power consumption of 14249 Joules per run.
MinAvgMaxAVX512 On38.653.166.5AVX512 Off38.152.965.8OpenBenchmarking.orgCelsius, Fewer Is BetterNeural Magic DeepSparse 1.5CPU Temperature Monitor20406080100

OpenBenchmarking.orgitems/sec, More Is BetterNeural Magic DeepSparse 1.5Model: NLP Question Answering, BERT base uncased SQuaD 12layer Pruned90 - Scenario: Asynchronous Multi-StreamAVX512 OnAVX512 Off50100150200250SE +/- 7.49, N = 15SE +/- 0.14, N = 3247.97213.64

TensorFlow

This is a benchmark of the TensorFlow deep learning framework using the TensorFlow reference benchmarks (tensorflow/benchmarks with tf_cnn_benchmarks.py). Note with the Phoronix Test Suite there is also pts/tensorflow-lite for benchmarking the TensorFlow Lite binaries if desired for complementary metrics. Learn more via the OpenBenchmarking.org test page.

OpenBenchmarking.orgimages/sec, More Is BetterTensorFlow 2.12Device: CPU - Batch Size: 32 - Model: ResNet-50AVX512 OnAVX512 Off1632486480SE +/- 0.23, N = 3SE +/- 0.04, N = 371.6217.48
OpenBenchmarking.orgimages/sec Per Watt, More Is BetterTensorFlow 2.12Device: CPU - Batch Size: 32 - Model: ResNet-50AVX512 OnAVX512 Off0.09160.18320.27480.36640.4580.4070.124
MinAvgMaxAVX512 On225030663438AVX512 Off225030843221OpenBenchmarking.orgMegahertz, More Is BetterTensorFlow 2.12CPU Peak Freq (Highest CPU Core Frequency) Monitor8001600240032004000
MinAvgMaxAVX512 On20.0176.0204.8AVX512 Off20.1141.1172.9OpenBenchmarking.orgWatts, Fewer Is BetterTensorFlow 2.12CPU Power Consumption Monitor501001502002501. AVX512 On: Approximate power consumption of 10151 Joules per run.2. AVX512 Off: Approximate power consumption of 29590 Joules per run.
MinAvgMaxAVX512 On31.341.848.6AVX512 Off28.337.551.6OpenBenchmarking.orgCelsius, Fewer Is BetterTensorFlow 2.12CPU Temperature Monitor1530456075

Cpuminer-Opt

Cpuminer-Opt is a fork of cpuminer-multi that carries a wide range of CPU performance optimizations for measuring the potential cryptocurrency mining performance of the CPU/processor with a wide variety of cryptocurrencies. The benchmark reports the hash speed for the CPU mining performance for the selected cryptocurrency. Learn more via the OpenBenchmarking.org test page.

OpenBenchmarking.orgkH/s, More Is BetterCpuminer-Opt 3.20.3Algorithm: Myriad-GroestlAVX512 OnAVX512 Off2K4K6K8K10KSE +/- 340.56, N = 15SE +/- 21.55, N = 38628.767149.951. (CXX) g++ options: -O2 -lcurl -lz -lpthread -lssl -lcrypto -lgmp
OpenBenchmarking.orgkH/s Per Watt, More Is BetterCpuminer-Opt 3.20.3Algorithm: Myriad-GroestlAVX512 OnAVX512 Off122436486051.6544.67
MinAvgMaxAVX512 On225030213097AVX512 Off225030463099OpenBenchmarking.orgMegahertz, More Is BetterCpuminer-Opt 3.20.3CPU Peak Freq (Highest CPU Core Frequency) Monitor8001600240032004000
MinAvgMaxAVX512 On19.9167.1199.7AVX512 Off20.4160.1173.0OpenBenchmarking.orgWatts, Fewer Is BetterCpuminer-Opt 3.20.3CPU Power Consumption Monitor501001502002501. AVX512 On: Approximate power consumption of 6282 Joules per run.2. AVX512 Off: Approximate power consumption of 9871 Joules per run.
MinAvgMaxAVX512 On34.942.247.0AVX512 Off33.940.944.6OpenBenchmarking.orgCelsius, Fewer Is BetterCpuminer-Opt 3.20.3CPU Temperature Monitor1428425670

TensorFlow

This is a benchmark of the TensorFlow deep learning framework using the TensorFlow reference benchmarks (tensorflow/benchmarks with tf_cnn_benchmarks.py). Note with the Phoronix Test Suite there is also pts/tensorflow-lite for benchmarking the TensorFlow Lite binaries if desired for complementary metrics. Learn more via the OpenBenchmarking.org test page.

OpenBenchmarking.orgimages/sec, More Is BetterTensorFlow 2.12Device: CPU - Batch Size: 32 - Model: GoogLeNetAVX512 OnAVX512 Off4080120160200SE +/- 3.09, N = 15SE +/- 0.10, N = 3180.7744.46
OpenBenchmarking.orgimages/sec Per Watt, More Is BetterTensorFlow 2.12Device: CPU - Batch Size: 32 - Model: GoogLeNetAVX512 OnAVX512 Off0.25610.51220.76831.02441.28051.1380.360
MinAvgMaxAVX512 On225029923472AVX512 Off225030643197OpenBenchmarking.orgMegahertz, More Is BetterTensorFlow 2.12CPU Peak Freq (Highest CPU Core Frequency) Monitor8001600240032004000
MinAvgMaxAVX512 On20.3158.9208.4AVX512 Off20.2123.6149.7OpenBenchmarking.orgWatts, Fewer Is BetterTensorFlow 2.12CPU Power Consumption Monitor601201802403001. AVX512 On: Approximate power consumption of 4258 Joules per run.2. AVX512 Off: Approximate power consumption of 10709 Joules per run.
MinAvgMaxAVX512 On30.840.349.9AVX512 Off29.335.347.1OpenBenchmarking.orgCelsius, Fewer Is BetterTensorFlow 2.12CPU Temperature Monitor1428425670

oneDNN

This is a test of the Intel oneDNN as an Intel-optimized library for Deep Neural Networks and making use of its built-in benchdnn functionality. The result is the total perf time reported. Intel oneDNN was formerly known as DNNL (Deep Neural Network Library) and MKL-DNN before being rebranded as part of the Intel oneAPI toolkit. Learn more via the OpenBenchmarking.org test page.

OpenBenchmarking.orgms, Fewer Is BetteroneDNN 3.1Harness: Recurrent Neural Network Training - Data Type: bf16bf16bf16 - Engine: CPUAVX512 OnAVX512 Off30060090012001500SE +/- 12.60, N = 4SE +/- 1.80, N = 31174.751317.57MIN: 1143.88MIN: 1299.381. (CXX) g++ options: -O3 -march=native -fopenmp -msse4.1 -fPIC -pie -ldl -lpthread
MinAvgMaxAVX512 On225029983097AVX512 Off225028243100OpenBenchmarking.orgMegahertz, More Is BetteroneDNN 3.1CPU Peak Freq (Highest CPU Core Frequency) Monitor8001600240032004000
MinAvgMaxAVX512 On20.0176.7277.9AVX512 Off19.8202.0272.5OpenBenchmarking.orgWatts, Fewer Is BetteroneDNN 3.1CPU Power Consumption Monitor701402102803501. AVX512 On: Approximate power consumption of 15289 Joules per run.2. AVX512 Off: Approximate power consumption of 17643 Joules per run.
MinAvgMaxAVX512 On30.439.245.3AVX512 Off24.539.845.1OpenBenchmarking.orgCelsius, Fewer Is BetteroneDNN 3.1CPU Temperature Monitor1224364860

Cpuminer-Opt

Cpuminer-Opt is a fork of cpuminer-multi that carries a wide range of CPU performance optimizations for measuring the potential cryptocurrency mining performance of the CPU/processor with a wide variety of cryptocurrencies. The benchmark reports the hash speed for the CPU mining performance for the selected cryptocurrency. Learn more via the OpenBenchmarking.org test page.

OpenBenchmarking.orgkH/s, More Is BetterCpuminer-Opt 3.20.3Algorithm: Blake-2 SAVX512 OnAVX512 Off1.6M3.2M4.8M6.4M8MSE +/- 3854.05, N = 3SE +/- 37514.23, N = 15723865045555801. (CXX) g++ options: -O2 -lcurl -lz -lpthread -lssl -lcrypto -lgmp
OpenBenchmarking.orgkH/s Per Watt, More Is BetterCpuminer-Opt 3.20.3Algorithm: Blake-2 SAVX512 OnAVX512 Off5K10K15K20K25K22160.8214551.24
MinAvgMaxAVX512 On225029253002AVX512 Off225027793096OpenBenchmarking.orgMegahertz, More Is BetterCpuminer-Opt 3.20.3CPU Peak Freq (Highest CPU Core Frequency) Monitor8001600240032004000
MinAvgMaxAVX512 On20.4326.6376.0AVX512 Off21.0313.1358.1OpenBenchmarking.orgWatts, Fewer Is BetterCpuminer-Opt 3.20.3CPU Power Consumption Monitor1002003004005001. AVX512 On: Approximate power consumption of 10997 Joules per run.2. AVX512 Off: Approximate power consumption of 10519 Joules per run.
MinAvgMaxAVX512 On33.964.072.8AVX512 Off31.164.070.0OpenBenchmarking.orgCelsius, Fewer Is BetterCpuminer-Opt 3.20.3CPU Temperature Monitor20406080100

OpenBenchmarking.orgkH/s, More Is BetterCpuminer-Opt 3.20.3Algorithm: GarlicoinAVX512 OnAVX512 Off11K22K33K44K55KSE +/- 110.15, N = 3SE +/- 102.69, N = 353090394731. (CXX) g++ options: -O2 -lcurl -lz -lpthread -lssl -lcrypto -lgmp
OpenBenchmarking.orgkH/s Per Watt, More Is BetterCpuminer-Opt 3.20.3Algorithm: GarlicoinAVX512 OnAVX512 Off60120180240300272.43202.32
MinAvgMaxAVX512 On225030583102AVX512 Off225030653102OpenBenchmarking.orgMegahertz, More Is BetterCpuminer-Opt 3.20.3CPU Peak Freq (Highest CPU Core Frequency) Monitor8001600240032004000
MinAvgMaxAVX512 On20.7194.9210.1AVX512 Off20.8195.1204.9OpenBenchmarking.orgWatts, Fewer Is BetterCpuminer-Opt 3.20.3CPU Power Consumption Monitor601201802403001. AVX512 On: Approximate power consumption of 13966 Joules per run.2. AVX512 Off: Approximate power consumption of 22112 Joules per run.
MinAvgMaxAVX512 On38.145.447.9AVX512 Off36.843.746.6OpenBenchmarking.orgCelsius, Fewer Is BetterCpuminer-Opt 3.20.3CPU Temperature Monitor1428425670

TensorFlow

This is a benchmark of the TensorFlow deep learning framework using the TensorFlow reference benchmarks (tensorflow/benchmarks with tf_cnn_benchmarks.py). Note with the Phoronix Test Suite there is also pts/tensorflow-lite for benchmarking the TensorFlow Lite binaries if desired for complementary metrics. Learn more via the OpenBenchmarking.org test page.

OpenBenchmarking.orgimages/sec, More Is BetterTensorFlow 2.12Device: CPU - Batch Size: 16 - Model: ResNet-50AVX512 OnAVX512 Off1020304050SE +/- 0.03, N = 3SE +/- 0.05, N = 343.1115.91
OpenBenchmarking.orgimages/sec Per Watt, More Is BetterTensorFlow 2.12Device: CPU - Batch Size: 16 - Model: ResNet-50AVX512 OnAVX512 Off0.06620.13240.19860.26480.3310.2940.113
MinAvgMaxAVX512 On225030413348AVX512 Off225030753217OpenBenchmarking.orgMegahertz, More Is BetterTensorFlow 2.12CPU Peak Freq (Highest CPU Core Frequency) Monitor8001600240032004000
MinAvgMaxAVX512 On20.1146.6171.4AVX512 Off20.2140.3151.8OpenBenchmarking.orgWatts, Fewer Is BetterTensorFlow 2.12CPU Power Consumption Monitor501001502002501. AVX512 On: Approximate power consumption of 7281 Joules per run.2. AVX512 Off: Approximate power consumption of 16692 Joules per run.
MinAvgMaxAVX512 On30.438.144.6AVX512 Off28.837.248.8OpenBenchmarking.orgCelsius, Fewer Is BetterTensorFlow 2.12CPU Temperature Monitor1428425670

OpenBenchmarking.orgimages/sec, More Is BetterTensorFlow 2.12Device: CPU - Batch Size: 16 - Model: GoogLeNetAVX512 OnAVX512 Off20406080100SE +/- 1.96, N = 15SE +/- 0.17, N = 3104.7739.64
OpenBenchmarking.orgimages/sec Per Watt, More Is BetterTensorFlow 2.12Device: CPU - Batch Size: 16 - Model: GoogLeNetAVX512 OnAVX512 Off0.17420.34840.52260.69680.8710.7740.335
MinAvgMaxAVX512 On225029773320AVX512 Off225030323208OpenBenchmarking.orgMegahertz, More Is BetterTensorFlow 2.12CPU Peak Freq (Highest CPU Core Frequency) Monitor8001600240032004000
MinAvgMaxAVX512 On19.8135.4179.1AVX512 Off20.1118.4140.5OpenBenchmarking.orgWatts, Fewer Is BetterTensorFlow 2.12CPU Power Consumption Monitor501001502002501. AVX512 On: Approximate power consumption of 3259 Joules per run.2. AVX512 Off: Approximate power consumption of 6158 Joules per run.
MinAvgMaxAVX512 On33.038.545.8AVX512 Off29.135.747.9OpenBenchmarking.orgCelsius, Fewer Is BetterTensorFlow 2.12CPU Temperature Monitor1428425670

OpenVINO

This is a test of the Intel OpenVINO, a toolkit around neural networks, using its built-in benchmarking support and analyzing the throughput and latency for various models. Learn more via the OpenBenchmarking.org test page.

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2022.3Model: Face Detection FP16 - Device: CPUAVX512 OnAVX512 Off5001000150020002500SE +/- 0.42, N = 3SE +/- 26.84, N = 31048.372423.39MIN: 508.1 / MAX: 1159.4MIN: 1130.42 / MAX: 2937.221. (CXX) g++ options: -isystem -fsigned-char -ffunction-sections -fdata-sections -msse4.1 -msse4.2 -O3 -fno-strict-overflow -fwrapv -fPIC -fvisibility=hidden -Os -std=c++11 -MD -MT -MF
MinAvgMaxAVX512 On225027083101AVX512 Off225024913096OpenBenchmarking.orgMegahertz, More Is BetterOpenVINO 2022.3CPU Peak Freq (Highest CPU Core Frequency) Monitor8001600240032004000
MinAvgMaxAVX512 On21.1315.2358.6AVX512 Off21.1286.8325.8OpenBenchmarking.orgWatts, Fewer Is BetterOpenVINO 2022.3CPU Power Consumption Monitor1002003004005001. AVX512 On: Approximate power consumption of 21962 Joules per run.2. AVX512 Off: Approximate power consumption of 20458 Joules per run.
MinAvgMaxAVX512 On38.663.170.0AVX512 Off39.859.368.1OpenBenchmarking.orgCelsius, Fewer Is BetterOpenVINO 2022.3CPU Temperature Monitor20406080100

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2022.3Model: Face Detection FP16 - Device: CPUAVX512 OnAVX512 Off1428425670SE +/- 0.06, N = 3SE +/- 0.36, N = 360.7326.181. (CXX) g++ options: -isystem -fsigned-char -ffunction-sections -fdata-sections -msse4.1 -msse4.2 -O3 -fno-strict-overflow -fwrapv -fPIC -fvisibility=hidden -Os -std=c++11 -MD -MT -MF

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2022.3Model: Face Detection FP16-INT8 - Device: CPUAVX512 OnAVX512 Off2004006008001000SE +/- 0.07, N = 3SE +/- 0.34, N = 3540.351094.52MIN: 257.81 / MAX: 586.56MIN: 509.48 / MAX: 1179.631. (CXX) g++ options: -isystem -fsigned-char -ffunction-sections -fdata-sections -msse4.1 -msse4.2 -O3 -fno-strict-overflow -fwrapv -fPIC -fvisibility=hidden -Os -std=c++11 -MD -MT -MF
MinAvgMaxAVX512 On225026963116AVX512 Off225026883124OpenBenchmarking.orgMegahertz, More Is BetterOpenVINO 2022.3CPU Peak Freq (Highest CPU Core Frequency) Monitor8001600240032004000
MinAvgMaxAVX512 On20.5295.4336.3AVX512 Off21.2323.5369.0OpenBenchmarking.orgWatts, Fewer Is BetterOpenVINO 2022.3CPU Power Consumption Monitor1002003004005001. AVX512 On: Approximate power consumption of 20678 Joules per run.2. AVX512 Off: Approximate power consumption of 22863 Joules per run.
MinAvgMaxAVX512 On34.459.570.5AVX512 Off35.865.576.1OpenBenchmarking.orgCelsius, Fewer Is BetterOpenVINO 2022.3CPU Temperature Monitor20406080100

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2022.3Model: Face Detection FP16-INT8 - Device: CPUAVX512 OnAVX512 Off306090120150SE +/- 0.02, N = 3SE +/- 0.03, N = 3118.0057.951. (CXX) g++ options: -isystem -fsigned-char -ffunction-sections -fdata-sections -msse4.1 -msse4.2 -O3 -fno-strict-overflow -fwrapv -fPIC -fvisibility=hidden -Os -std=c++11 -MD -MT -MF

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2022.3Model: Age Gender Recognition Retail 0013 FP16 - Device: CPUAVX512 OnAVX512 Off0.42980.85961.28941.71922.149SE +/- 0.00, N = 3SE +/- 0.01, N = 30.991.91MIN: 0.35 / MAX: 19.47MIN: 0.67 / MAX: 19.481. (CXX) g++ options: -isystem -fsigned-char -ffunction-sections -fdata-sections -msse4.1 -msse4.2 -O3 -fno-strict-overflow -fwrapv -fPIC -fvisibility=hidden -Os -std=c++11 -MD -MT -MF
MinAvgMaxAVX512 On225028923097AVX512 Off225025903101OpenBenchmarking.orgMegahertz, More Is BetterOpenVINO 2022.3CPU Peak Freq (Highest CPU Core Frequency) Monitor8001600240032004000
MinAvgMaxAVX512 On21.2316.4347.6AVX512 Off21.0298.9332.9OpenBenchmarking.orgWatts, Fewer Is BetterOpenVINO 2022.3CPU Power Consumption Monitor1002003004005001. AVX512 On: Approximate power consumption of 20569 Joules per run.2. AVX512 Off: Approximate power consumption of 19729 Joules per run.
MinAvgMaxAVX512 On41.064.670.0AVX512 Off37.361.868.0OpenBenchmarking.orgCelsius, Fewer Is BetterOpenVINO 2022.3CPU Temperature Monitor20406080100

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2022.3Model: Age Gender Recognition Retail 0013 FP16 - Device: CPUAVX512 OnAVX512 Off20K40K60K80K100KSE +/- 314.35, N = 3SE +/- 278.13, N = 3110240.8962564.161. (CXX) g++ options: -isystem -fsigned-char -ffunction-sections -fdata-sections -msse4.1 -msse4.2 -O3 -fno-strict-overflow -fwrapv -fPIC -fvisibility=hidden -Os -std=c++11 -MD -MT -MF

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2022.3Model: Age Gender Recognition Retail 0013 FP16-INT8 - Device: CPUAVX512 OnAVX512 Off0.3960.7921.1881.5841.98SE +/- 0.00, N = 3SE +/- 0.00, N = 31.581.76MIN: 0.55 / MAX: 17.78MIN: 0.64 / MAX: 19.61. (CXX) g++ options: -isystem -fsigned-char -ffunction-sections -fdata-sections -msse4.1 -msse4.2 -O3 -fno-strict-overflow -fwrapv -fPIC -fvisibility=hidden -Os -std=c++11 -MD -MT -MF
MinAvgMaxAVX512 On225027613119AVX512 Off225026263099OpenBenchmarking.orgMegahertz, More Is BetterOpenVINO 2022.3CPU Peak Freq (Highest CPU Core Frequency) Monitor8001600240032004000
MinAvgMaxAVX512 On21.4311.4336.9AVX512 Off21.2308.2334.1OpenBenchmarking.orgWatts, Fewer Is BetterOpenVINO 2022.3CPU Power Consumption Monitor801602403204001. AVX512 On: Approximate power consumption of 20032 Joules per run.2. AVX512 Off: Approximate power consumption of 19931 Joules per run.
MinAvgMaxAVX512 On41.463.468.1AVX512 Off41.063.969.0OpenBenchmarking.orgCelsius, Fewer Is BetterOpenVINO 2022.3CPU Temperature Monitor20406080100

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2022.3Model: Age Gender Recognition Retail 0013 FP16-INT8 - Device: CPUAVX512 OnAVX512 Off16K32K48K64K80KSE +/- 95.74, N = 3SE +/- 20.10, N = 373970.1266895.491. (CXX) g++ options: -isystem -fsigned-char -ffunction-sections -fdata-sections -msse4.1 -msse4.2 -O3 -fno-strict-overflow -fwrapv -fPIC -fvisibility=hidden -Os -std=c++11 -MD -MT -MF

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2022.3Model: Weld Porosity Detection FP16-INT8 - Device: CPUAVX512 OnAVX512 Off510152025SE +/- 0.00, N = 3SE +/- 0.01, N = 310.8222.47MIN: 5 / MAX: 31.44MIN: 10.87 / MAX: 43.51. (CXX) g++ options: -isystem -fsigned-char -ffunction-sections -fdata-sections -msse4.1 -msse4.2 -O3 -fno-strict-overflow -fwrapv -fPIC -fvisibility=hidden -Os -std=c++11 -MD -MT -MF
MinAvgMaxAVX512 On225026193137AVX512 Off225026713102OpenBenchmarking.orgMegahertz, More Is BetterOpenVINO 2022.3CPU Peak Freq (Highest CPU Core Frequency) Monitor8001600240032004000
MinAvgMaxAVX512 On21.1306.8333.5AVX512 Off21.1336.6365.0OpenBenchmarking.orgWatts, Fewer Is BetterOpenVINO 2022.3CPU Power Consumption Monitor1002003004005001. AVX512 On: Approximate power consumption of 19739 Joules per run.2. AVX512 Off: Approximate power consumption of 21653 Joules per run.
MinAvgMaxAVX512 On36.861.366.1AVX512 Off36.867.473.3OpenBenchmarking.orgCelsius, Fewer Is BetterOpenVINO 2022.3CPU Temperature Monitor20406080100

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2022.3Model: Weld Porosity Detection FP16-INT8 - Device: CPUAVX512 OnAVX512 Off3K6K9K12K15KSE +/- 1.17, N = 3SE +/- 1.65, N = 311818.335692.991. (CXX) g++ options: -isystem -fsigned-char -ffunction-sections -fdata-sections -msse4.1 -msse4.2 -O3 -fno-strict-overflow -fwrapv -fPIC -fvisibility=hidden -Os -std=c++11 -MD -MT -MF

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2022.3Model: Weld Porosity Detection FP16 - Device: CPUAVX512 OnAVX512 Off612182430SE +/- 0.00, N = 3SE +/- 0.01, N = 310.5225.06MIN: 5.11 / MAX: 34.37MIN: 13.08 / MAX: 57.021. (CXX) g++ options: -isystem -fsigned-char -ffunction-sections -fdata-sections -msse4.1 -msse4.2 -O3 -fno-strict-overflow -fwrapv -fPIC -fvisibility=hidden -Os -std=c++11 -MD -MT -MF
MinAvgMaxAVX512 On225027183095AVX512 Off225023733112OpenBenchmarking.orgMegahertz, More Is BetterOpenVINO 2022.3CPU Peak Freq (Highest CPU Core Frequency) Monitor8001600240032004000
MinAvgMaxAVX512 On21.2327.7354.4AVX512 Off21.3288.2311.1OpenBenchmarking.orgWatts, Fewer Is BetterOpenVINO 2022.3CPU Power Consumption Monitor1002003004005001. AVX512 On: Approximate power consumption of 20970 Joules per run.2. AVX512 Off: Approximate power consumption of 18446 Joules per run.
MinAvgMaxAVX512 On36.864.269.9AVX512 Off41.961.164.3OpenBenchmarking.orgCelsius, Fewer Is BetterOpenVINO 2022.3CPU Temperature Monitor20406080100

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2022.3Model: Weld Porosity Detection FP16 - Device: CPUAVX512 OnAVX512 Off13002600390052006500SE +/- 1.32, N = 3SE +/- 0.56, N = 36073.222551.701. (CXX) g++ options: -isystem -fsigned-char -ffunction-sections -fdata-sections -msse4.1 -msse4.2 -O3 -fno-strict-overflow -fwrapv -fPIC -fvisibility=hidden -Os -std=c++11 -MD -MT -MF

Neural Magic DeepSparse

This is a benchmark of Neural Magic's DeepSparse using its built-in deepsparse.benchmark utility and various models from their SparseZoo (https://sparsezoo.neuralmagic.com/). Learn more via the OpenBenchmarking.org test page.

OpenBenchmarking.orgms/batch, Fewer Is BetterNeural Magic DeepSparse 1.5Model: CV Segmentation, 90% Pruned YOLACT Pruned - Scenario: Asynchronous Multi-StreamAVX512 OnAVX512 Off2004006008001000SE +/- 0.16, N = 3SE +/- 0.94, N = 3498.48906.94
MinAvgMaxAVX512 On225030063115AVX512 Off225030483116OpenBenchmarking.orgMegahertz, More Is BetterNeural Magic DeepSparse 1.5CPU Peak Freq (Highest CPU Core Frequency) Monitor8001600240032004000
MinAvgMaxAVX512 On20.6228.3324.5AVX512 Off21.5235.2340.9OpenBenchmarking.orgWatts, Fewer Is BetterNeural Magic DeepSparse 1.5CPU Power Consumption Monitor801602403204001. AVX512 On: Approximate power consumption of 13469 Joules per run.2. AVX512 Off: Approximate power consumption of 15600 Joules per run.
MinAvgMaxAVX512 On38.153.166.1AVX512 Off40.054.069.1OpenBenchmarking.orgCelsius, Fewer Is BetterNeural Magic DeepSparse 1.5CPU Temperature Monitor20406080100

OpenBenchmarking.orgitems/sec, More Is BetterNeural Magic DeepSparse 1.5Model: CV Segmentation, 90% Pruned YOLACT Pruned - Scenario: Asynchronous Multi-StreamAVX512 OnAVX512 Off306090120150SE +/- 0.01, N = 3SE +/- 0.03, N = 3127.0669.33

OSPRay

Intel OSPRay is a portable ray-tracing engine for high-performance, high-fidelity scientific visualizations. OSPRay builds off Intel's Embree and Intel SPMD Program Compiler (ISPC) components as part of the oneAPI rendering toolkit. Learn more via the OpenBenchmarking.org test page.

OpenBenchmarking.orgItems Per Second, More Is BetterOSPRay 2.12Benchmark: gravity_spheres_volume/dim_512/scivis/real_timeAVX512 OnAVX512 Off714212835SE +/- 0.02, N = 3SE +/- 0.00, N = 331.6818.02
OpenBenchmarking.orgItems Per Second Per Watt, More Is BetterOSPRay 2.12Benchmark: gravity_spheres_volume/dim_512/scivis/real_timeAVX512 OnAVX512 Off0.02570.05140.07710.10280.12850.1140.062
MinAvgMaxAVX512 On225028753096AVX512 Off225029253101OpenBenchmarking.orgMegahertz, More Is BetterOSPRay 2.12CPU Peak Freq (Highest CPU Core Frequency) Monitor8001600240032004000
MinAvgMaxAVX512 On20.9278.8332.5AVX512 Off20.9292.2347.0OpenBenchmarking.orgWatts, Fewer Is BetterOSPRay 2.12CPU Power Consumption Monitor1002003004005001. AVX512 On: Approximate power consumption of 15522 Joules per run.2. AVX512 Off: Approximate power consumption of 17047 Joules per run.
MinAvgMaxAVX512 On39.457.264.3AVX512 Off39.459.766.1OpenBenchmarking.orgCelsius, Fewer Is BetterOSPRay 2.12CPU Temperature Monitor20406080100

OpenBenchmarking.orgItems Per Second, More Is BetterOSPRay 2.12Benchmark: gravity_spheres_volume/dim_512/pathtracer/real_timeAVX512 OnAVX512 Off714212835SE +/- 0.01, N = 3SE +/- 0.01, N = 327.9719.55
OpenBenchmarking.orgItems Per Second Per Watt, More Is BetterOSPRay 2.12Benchmark: gravity_spheres_volume/dim_512/pathtracer/real_timeAVX512 OnAVX512 Off0.02390.04780.07170.09560.11950.1060.072
MinAvgMaxAVX512 On225030243099AVX512 Off225030523098OpenBenchmarking.orgMegahertz, More Is BetterOSPRay 2.12CPU Peak Freq (Highest CPU Core Frequency) Monitor8001600240032004000
MinAvgMaxAVX512 On21.1263.0313.6AVX512 Off21.2271.7321.8OpenBenchmarking.orgWatts, Fewer Is BetterOSPRay 2.12CPU Power Consumption Monitor801602403204001. AVX512 On: Approximate power consumption of 14640 Joules per run.2. AVX512 Off: Approximate power consumption of 15576 Joules per run.
MinAvgMaxAVX512 On40.055.661.3AVX512 Off40.057.062.8OpenBenchmarking.orgCelsius, Fewer Is BetterOSPRay 2.12CPU Temperature Monitor20406080100

OpenBenchmarking.orgItems Per Second, More Is BetterOSPRay 2.12Benchmark: gravity_spheres_volume/dim_512/ao/real_timeAVX512 OnAVX512 Off816243240SE +/- 0.01, N = 3SE +/- 0.02, N = 332.7619.17
OpenBenchmarking.orgItems Per Second Per Watt, More Is BetterOSPRay 2.12Benchmark: gravity_spheres_volume/dim_512/ao/real_timeAVX512 OnAVX512 Off0.02660.05320.07980.10640.1330.1180.066
MinAvgMaxAVX512 On225028853099AVX512 Off225029263096OpenBenchmarking.orgMegahertz, More Is BetterOSPRay 2.12CPU Peak Freq (Highest CPU Core Frequency) Monitor8001600240032004000
MinAvgMaxAVX512 On20.1276.9329.8AVX512 Off20.8290.6346.2OpenBenchmarking.orgWatts, Fewer Is BetterOSPRay 2.12CPU Power Consumption Monitor1002003004005001. AVX512 On: Approximate power consumption of 15324 Joules per run.2. AVX512 Off: Approximate power consumption of 16755 Joules per run.
MinAvgMaxAVX512 On33.555.163.3AVX512 Off32.057.565.3OpenBenchmarking.orgCelsius, Fewer Is BetterOSPRay 2.12CPU Temperature Monitor20406080100

Neural Magic DeepSparse

This is a benchmark of Neural Magic's DeepSparse using its built-in deepsparse.benchmark utility and various models from their SparseZoo (https://sparsezoo.neuralmagic.com/). Learn more via the OpenBenchmarking.org test page.

OpenBenchmarking.orgms/batch, Fewer Is BetterNeural Magic DeepSparse 1.5Model: NLP Token Classification, BERT base uncased conll2003 - Scenario: Asynchronous Multi-StreamAVX512 OnAVX512 Off2004006008001000SE +/- 0.45, N = 3SE +/- 1.31, N = 3859.711025.48
MinAvgMaxAVX512 On225030063135AVX512 Off225029523146OpenBenchmarking.orgMegahertz, More Is BetterNeural Magic DeepSparse 1.5CPU Peak Freq (Highest CPU Core Frequency) Monitor8001600240032004000
MinAvgMaxAVX512 On20.7244.3357.9AVX512 Off21.2245.5344.8OpenBenchmarking.orgWatts, Fewer Is BetterNeural Magic DeepSparse 1.5CPU Power Consumption Monitor1002003004005001. AVX512 On: Approximate power consumption of 13764 Joules per run.2. AVX512 Off: Approximate power consumption of 13586 Joules per run.
MinAvgMaxAVX512 On37.357.670.9AVX512 Off38.958.370.9OpenBenchmarking.orgCelsius, Fewer Is BetterNeural Magic DeepSparse 1.5CPU Temperature Monitor20406080100

OpenBenchmarking.orgitems/sec, More Is BetterNeural Magic DeepSparse 1.5Model: NLP Token Classification, BERT base uncased conll2003 - Scenario: Asynchronous Multi-StreamAVX512 OnAVX512 Off1632486480SE +/- 0.19, N = 3SE +/- 0.13, N = 373.1561.18

OpenBenchmarking.orgms/batch, Fewer Is BetterNeural Magic DeepSparse 1.5Model: NLP Document Classification, oBERT base uncased on IMDB - Scenario: Asynchronous Multi-StreamAVX512 OnAVX512 Off2004006008001000SE +/- 0.33, N = 3SE +/- 1.40, N = 3858.401023.17
MinAvgMaxAVX512 On225030113136AVX512 Off225029333136OpenBenchmarking.orgMegahertz, More Is BetterNeural Magic DeepSparse 1.5CPU Peak Freq (Highest CPU Core Frequency) Monitor8001600240032004000
MinAvgMaxAVX512 On20.4253.4358.2AVX512 Off20.7244.1344.4OpenBenchmarking.orgWatts, Fewer Is BetterNeural Magic DeepSparse 1.5CPU Power Consumption Monitor1002003004005001. AVX512 On: Approximate power consumption of 13937 Joules per run.2. AVX512 Off: Approximate power consumption of 13506 Joules per run.
MinAvgMaxAVX512 On34.657.571.4AVX512 Off30.654.570.5OpenBenchmarking.orgCelsius, Fewer Is BetterNeural Magic DeepSparse 1.5CPU Temperature Monitor20406080100

OpenBenchmarking.orgitems/sec, More Is BetterNeural Magic DeepSparse 1.5Model: NLP Document Classification, oBERT base uncased on IMDB - Scenario: Asynchronous Multi-StreamAVX512 OnAVX512 Off1632486480SE +/- 0.14, N = 3SE +/- 0.02, N = 373.5461.37

OpenBenchmarking.orgms/batch, Fewer Is BetterNeural Magic DeepSparse 1.5Model: NLP Text Classification, BERT base uncased SST2 - Scenario: Asynchronous Multi-StreamAVX512 OnAVX512 Off50100150200250SE +/- 0.47, N = 3SE +/- 0.64, N = 3201.60244.15
MinAvgMaxAVX512 On225030133127AVX512 Off225029403122OpenBenchmarking.orgMegahertz, More Is BetterNeural Magic DeepSparse 1.5CPU Peak Freq (Highest CPU Core Frequency) Monitor8001600240032004000
MinAvgMaxAVX512 On20.6247.7358.6AVX512 Off20.8247.7347.3OpenBenchmarking.orgWatts, Fewer Is BetterNeural Magic DeepSparse 1.5CPU Power Consumption Monitor1002003004005001. AVX512 On: Approximate power consumption of 13294 Joules per run.2. AVX512 Off: Approximate power consumption of 13127 Joules per run.
MinAvgMaxAVX512 On36.057.770.9AVX512 Off38.458.370.5OpenBenchmarking.orgCelsius, Fewer Is BetterNeural Magic DeepSparse 1.5CPU Temperature Monitor20406080100

OpenBenchmarking.orgitems/sec, More Is BetterNeural Magic DeepSparse 1.5Model: NLP Text Classification, BERT base uncased SST2 - Scenario: Asynchronous Multi-StreamAVX512 OnAVX512 Off70140210280350SE +/- 0.78, N = 3SE +/- 0.63, N = 3316.17260.77

OpenBenchmarking.orgms/batch, Fewer Is BetterNeural Magic DeepSparse 1.5Model: NLP Sentiment Analysis, 80% Pruned Quantized BERT Base Uncased - Scenario: Asynchronous Multi-StreamAVX512 OnAVX512 Off20406080100SE +/- 0.05, N = 3SE +/- 0.67, N = 346.2688.91
MinAvgMaxAVX512 On225030263124AVX512 Off225030153127OpenBenchmarking.orgMegahertz, More Is BetterNeural Magic DeepSparse 1.5CPU Peak Freq (Highest CPU Core Frequency) Monitor8001600240032004000
MinAvgMaxAVX512 On21.1250.2321.3AVX512 Off21.3234.7326.7OpenBenchmarking.orgWatts, Fewer Is BetterNeural Magic DeepSparse 1.5CPU Power Consumption Monitor801602403204001. AVX512 On: Approximate power consumption of 12009 Joules per run.2. AVX512 Off: Approximate power consumption of 12985 Joules per run.
MinAvgMaxAVX512 On39.458.167.6AVX512 Off38.153.864.3OpenBenchmarking.orgCelsius, Fewer Is BetterNeural Magic DeepSparse 1.5CPU Temperature Monitor20406080100

OpenBenchmarking.orgitems/sec, More Is BetterNeural Magic DeepSparse 1.5Model: NLP Sentiment Analysis, 80% Pruned Quantized BERT Base Uncased - Scenario: Asynchronous Multi-StreamAVX512 OnAVX512 Off30060090012001500SE +/- 1.58, N = 3SE +/- 5.33, N = 31381.57718.30

TensorFlow

This is a benchmark of the TensorFlow deep learning framework using the TensorFlow reference benchmarks (tensorflow/benchmarks with tf_cnn_benchmarks.py). Note with the Phoronix Test Suite there is also pts/tensorflow-lite for benchmarking the TensorFlow Lite binaries if desired for complementary metrics. Learn more via the OpenBenchmarking.org test page.

OpenBenchmarking.orgimages/sec, More Is BetterTensorFlow 2.12Device: CPU - Batch Size: 64 - Model: AlexNetAVX512 OnAVX512 Off2004006008001000SE +/- 1.99, N = 5SE +/- 0.08, N = 3857.5597.59
OpenBenchmarking.orgimages/sec Per Watt, More Is BetterTensorFlow 2.12Device: CPU - Batch Size: 64 - Model: AlexNetAVX512 OnAVX512 Off1.33612.67224.00835.34446.68055.9380.778
MinAvgMaxAVX512 On225027513111AVX512 Off225030573377OpenBenchmarking.orgMegahertz, More Is BetterTensorFlow 2.12CPU Peak Freq (Highest CPU Core Frequency) Monitor8001600240032004000
MinAvgMaxAVX512 On20.2144.4224.5AVX512 Off19.9125.4141.0OpenBenchmarking.orgWatts, Fewer Is BetterTensorFlow 2.12CPU Power Consumption Monitor601201802403001. AVX512 On: Approximate power consumption of 2079 Joules per run.2. AVX512 Off: Approximate power consumption of 9780 Joules per run.
MinAvgMaxAVX512 On30.638.345.5AVX512 Off30.436.546.4OpenBenchmarking.orgCelsius, Fewer Is BetterTensorFlow 2.12CPU Temperature Monitor1428425670

Neural Magic DeepSparse

This is a benchmark of Neural Magic's DeepSparse using its built-in deepsparse.benchmark utility and various models from their SparseZoo (https://sparsezoo.neuralmagic.com/). Learn more via the OpenBenchmarking.org test page.

OpenBenchmarking.orgms/batch, Fewer Is BetterNeural Magic DeepSparse 1.5Model: NLP Text Classification, DistilBERT mnli - Scenario: Asynchronous Multi-StreamAVX512 OnAVX512 Off306090120150SE +/- 0.07, N = 3SE +/- 0.15, N = 3102.22122.23
MinAvgMaxAVX512 On225030343099AVX512 Off225029313101OpenBenchmarking.orgMegahertz, More Is BetterNeural Magic DeepSparse 1.5CPU Peak Freq (Highest CPU Core Frequency) Monitor8001600240032004000
MinAvgMaxAVX512 On20.8255.7320.6AVX512 Off21.0256.2320.4OpenBenchmarking.orgWatts, Fewer Is BetterNeural Magic DeepSparse 1.5CPU Power Consumption Monitor801602403204001. AVX512 On: Approximate power consumption of 11253 Joules per run.2. AVX512 Off: Approximate power consumption of 11359 Joules per run.
MinAvgMaxAVX512 On39.160.467.1AVX512 Off41.060.768.5OpenBenchmarking.orgCelsius, Fewer Is BetterNeural Magic DeepSparse 1.5CPU Temperature Monitor20406080100

OpenBenchmarking.orgitems/sec, More Is BetterNeural Magic DeepSparse 1.5Model: NLP Text Classification, DistilBERT mnli - Scenario: Asynchronous Multi-StreamAVX512 OnAVX512 Off130260390520650SE +/- 0.52, N = 3SE +/- 0.58, N = 3624.74522.11

OpenBenchmarking.orgms/batch, Fewer Is BetterNeural Magic DeepSparse 1.5Model: CV Classification, ResNet-50 ImageNet - Scenario: Asynchronous Multi-StreamAVX512 OnAVX512 Off1632486480SE +/- 0.03, N = 3SE +/- 0.04, N = 365.8973.35
MinAvgMaxAVX512 On225030123134AVX512 Off225029623101OpenBenchmarking.orgMegahertz, More Is BetterNeural Magic DeepSparse 1.5CPU Peak Freq (Highest CPU Core Frequency) Monitor8001600240032004000
MinAvgMaxAVX512 On21.1279.2332.9AVX512 Off21.0284.5342.1OpenBenchmarking.orgWatts, Fewer Is BetterNeural Magic DeepSparse 1.5CPU Power Consumption Monitor801602403204001. AVX512 On: Approximate power consumption of 11726 Joules per run.2. AVX512 Off: Approximate power consumption of 11855 Joules per run.
MinAvgMaxAVX512 On39.862.870.4AVX512 Off40.065.272.9OpenBenchmarking.orgCelsius, Fewer Is BetterNeural Magic DeepSparse 1.5CPU Temperature Monitor20406080100

OpenBenchmarking.orgitems/sec, More Is BetterNeural Magic DeepSparse 1.5Model: CV Classification, ResNet-50 ImageNet - Scenario: Asynchronous Multi-StreamAVX512 OnAVX512 Off2004006008001000SE +/- 0.42, N = 3SE +/- 0.45, N = 3970.06870.95

Cpuminer-Opt

Cpuminer-Opt is a fork of cpuminer-multi that carries a wide range of CPU performance optimizations for measuring the potential cryptocurrency mining performance of the CPU/processor with a wide variety of cryptocurrencies. The benchmark reports the hash speed for the CPU mining performance for the selected cryptocurrency. Learn more via the OpenBenchmarking.org test page.

OpenBenchmarking.orgkH/s, More Is BetterCpuminer-Opt 3.20.3Algorithm: x25xAVX512 OnAVX512 Off11002200330044005500SE +/- 15.71, N = 3SE +/- 35.48, N = 44977.893010.371. (CXX) g++ options: -O2 -lcurl -lz -lpthread -lssl -lcrypto -lgmp
OpenBenchmarking.orgkH/s Per Watt, More Is BetterCpuminer-Opt 3.20.3Algorithm: x25xAVX512 OnAVX512 Off4812162016.8510.81
MinAvgMaxAVX512 On225029093099AVX512 Off225030193099OpenBenchmarking.orgMegahertz, More Is BetterCpuminer-Opt 3.20.3CPU Peak Freq (Highest CPU Core Frequency) Monitor8001600240032004000
MinAvgMaxAVX512 On20.4295.4341.0AVX512 Off21.0278.5321.6OpenBenchmarking.orgWatts, Fewer Is BetterCpuminer-Opt 3.20.3CPU Power Consumption Monitor801602403204001. AVX512 On: Approximate power consumption of 10042 Joules per run.2. AVX512 Off: Approximate power consumption of 9328 Joules per run.
MinAvgMaxAVX512 On39.458.764.3AVX512 Off37.655.259.5OpenBenchmarking.orgCelsius, Fewer Is BetterCpuminer-Opt 3.20.3CPU Temperature Monitor20406080100

TensorFlow

This is a benchmark of the TensorFlow deep learning framework using the TensorFlow reference benchmarks (tensorflow/benchmarks with tf_cnn_benchmarks.py). Note with the Phoronix Test Suite there is also pts/tensorflow-lite for benchmarking the TensorFlow Lite binaries if desired for complementary metrics. Learn more via the OpenBenchmarking.org test page.

OpenBenchmarking.orgimages/sec, More Is BetterTensorFlow 2.12Device: CPU - Batch Size: 32 - Model: AlexNetAVX512 OnAVX512 Off120240360480600SE +/- 2.06, N = 6SE +/- 0.16, N = 3562.4884.96
OpenBenchmarking.orgimages/sec Per Watt, More Is BetterTensorFlow 2.12Device: CPU - Batch Size: 32 - Model: AlexNetAVX512 OnAVX512 Off0.98461.96922.95383.93844.9234.3760.691
MinAvgMaxAVX512 On225028363101AVX512 Off225030393146OpenBenchmarking.orgMegahertz, More Is BetterTensorFlow 2.12CPU Peak Freq (Highest CPU Core Frequency) Monitor8001600240032004000
MinAvgMaxAVX512 On20.3128.6210.9AVX512 Off20.5123.0139.4OpenBenchmarking.orgWatts, Fewer Is BetterTensorFlow 2.12CPU Power Consumption Monitor601201802403001. AVX512 On: Approximate power consumption of 1585 Joules per run.2. AVX512 Off: Approximate power consumption of 5861 Joules per run.
MinAvgMaxAVX512 On32.037.845.3AVX512 Off30.537.145.6OpenBenchmarking.orgCelsius, Fewer Is BetterTensorFlow 2.12CPU Temperature Monitor1224364860

Cpuminer-Opt

Cpuminer-Opt is a fork of cpuminer-multi that carries a wide range of CPU performance optimizations for measuring the potential cryptocurrency mining performance of the CPU/processor with a wide variety of cryptocurrencies. The benchmark reports the hash speed for the CPU mining performance for the selected cryptocurrency. Learn more via the OpenBenchmarking.org test page.

OpenBenchmarking.orgkH/s, More Is BetterCpuminer-Opt 3.20.3Algorithm: SkeincoinAVX512 OnAVX512 Off300K600K900K1200K1500KSE +/- 4577.90, N = 3SE +/- 1811.04, N = 311749539379771. (CXX) g++ options: -O2 -lcurl -lz -lpthread -lssl -lcrypto -lgmp
OpenBenchmarking.orgkH/s Per Watt, More Is BetterCpuminer-Opt 3.20.3Algorithm: SkeincoinAVX512 OnAVX512 Off80016002400320040003691.562892.87
MinAvgMaxAVX512 On225028043143AVX512 Off225028993096OpenBenchmarking.orgMegahertz, More Is BetterCpuminer-Opt 3.20.3CPU Peak Freq (Highest CPU Core Frequency) Monitor8001600240032004000
MinAvgMaxAVX512 On20.2318.3370.8AVX512 Off20.2324.2376.0OpenBenchmarking.orgWatts, Fewer Is BetterCpuminer-Opt 3.20.3CPU Power Consumption Monitor1002003004005001. AVX512 On: Approximate power consumption of 10822 Joules per run.2. AVX512 Off: Approximate power consumption of 11024 Joules per run.
MinAvgMaxAVX512 On34.463.471.9AVX512 Off33.062.971.9OpenBenchmarking.orgCelsius, Fewer Is BetterCpuminer-Opt 3.20.3CPU Temperature Monitor20406080100

OpenBenchmarking.orgkH/s, More Is BetterCpuminer-Opt 3.20.3Algorithm: scryptAVX512 OnAVX512 Off6001200180024003000SE +/- 1.66, N = 3SE +/- 0.24, N = 32993.212033.151. (CXX) g++ options: -O2 -lcurl -lz -lpthread -lssl -lcrypto -lgmp
OpenBenchmarking.orgkH/s Per Watt, More Is BetterCpuminer-Opt 3.20.3Algorithm: scryptAVX512 OnAVX512 Off2468108.9136.476
MinAvgMaxAVX512 On225029593095AVX512 Off225029523091OpenBenchmarking.orgMegahertz, More Is BetterCpuminer-Opt 3.20.3CPU Peak Freq (Highest CPU Core Frequency) Monitor8001600240032004000
MinAvgMaxAVX512 On20.9335.8385.3AVX512 Off20.9314.0361.0OpenBenchmarking.orgWatts, Fewer Is BetterCpuminer-Opt 3.20.3CPU Power Consumption Monitor1002003004005001. AVX512 On: Approximate power consumption of 11306 Joules per run.2. AVX512 Off: Approximate power consumption of 10675 Joules per run.
MinAvgMaxAVX512 On39.566.473.3AVX512 Off38.163.670.0OpenBenchmarking.orgCelsius, Fewer Is BetterCpuminer-Opt 3.20.3CPU Temperature Monitor20406080100

OpenBenchmarking.orgkH/s, More Is BetterCpuminer-Opt 3.20.3Algorithm: LBC, LBRY CreditsAVX512 OnAVX512 Off140K280K420K560K700KSE +/- 76.38, N = 3SE +/- 141.93, N = 36606603326671. (CXX) g++ options: -O2 -lcurl -lz -lpthread -lssl -lcrypto -lgmp
OpenBenchmarking.orgkH/s Per Watt, More Is BetterCpuminer-Opt 3.20.3Algorithm: LBC, LBRY CreditsAVX512 OnAVX512 Off4008001200160020002003.501052.19
MinAvgMaxAVX512 On225028863096AVX512 Off225027463096OpenBenchmarking.orgMegahertz, More Is BetterCpuminer-Opt 3.20.3CPU Peak Freq (Highest CPU Core Frequency) Monitor8001600240032004000
MinAvgMaxAVX512 On20.3329.8380.7AVX512 Off20.7316.2362.0OpenBenchmarking.orgWatts, Fewer Is BetterCpuminer-Opt 3.20.3CPU Power Consumption Monitor1002003004005001. AVX512 On: Approximate power consumption of 10992 Joules per run.2. AVX512 Off: Approximate power consumption of 10750 Joules per run.
MinAvgMaxAVX512 On32.063.272.8AVX512 Off30.659.571.9OpenBenchmarking.orgCelsius, Fewer Is BetterCpuminer-Opt 3.20.3CPU Temperature Monitor20406080100

OpenBenchmarking.orgkH/s, More Is BetterCpuminer-Opt 3.20.3Algorithm: Quad SHA-256, PyriteAVX512 OnAVX512 Off300K600K900K1200K1500KSE +/- 3455.26, N = 3SE +/- 2198.34, N = 314989378761371. (CXX) g++ options: -O2 -lcurl -lz -lpthread -lssl -lcrypto -lgmp
OpenBenchmarking.orgkH/s Per Watt, More Is BetterCpuminer-Opt 3.20.3Algorithm: Quad SHA-256, PyriteAVX512 OnAVX512 Off90018002700360045004347.582825.31
MinAvgMaxAVX512 On225029663080AVX512 Off225029193027OpenBenchmarking.orgMegahertz, More Is BetterCpuminer-Opt 3.20.3CPU Peak Freq (Highest CPU Core Frequency) Monitor8001600240032004000
MinAvgMaxAVX512 On21.1344.8398.4AVX512 Off11.0310.1352.7OpenBenchmarking.orgWatts, Fewer Is BetterCpuminer-Opt 3.20.3CPU Power Consumption Monitor1102203304405501. AVX512 On: Approximate power consumption of 11722 Joules per run.2. AVX512 Off: Approximate power consumption of 10440 Joules per run.
MinAvgMaxAVX512 On40.568.774.3AVX512 Off38.662.068.1OpenBenchmarking.orgCelsius, Fewer Is BetterCpuminer-Opt 3.20.3CPU Temperature Monitor20406080100

miniBUDE

MiniBUDE is a mini application for the the core computation of the Bristol University Docking Engine (BUDE). This test profile currently makes use of the OpenMP implementation of miniBUDE for CPU benchmarking. Learn more via the OpenBenchmarking.org test page.

OpenBenchmarking.orgBillion Interactions/s, More Is BetterminiBUDE 20210901Implementation: OpenMP - Input Deck: BM2AVX512 OnAVX512 Off50100150200250SE +/- 0.02, N = 3SE +/- 0.62, N = 3238.89181.131. (CC) gcc options: -std=c99 -Ofast -ffast-math -fopenmp -march=native -lm
OpenBenchmarking.orgBillion Interactions/s Per Watt, More Is BetterminiBUDE 20210901Implementation: OpenMP - Input Deck: BM2AVX512 OnAVX512 Off0.20030.40060.60090.80121.00150.890.62
MinAvgMaxAVX512 On225027963014AVX512 Off225026963098OpenBenchmarking.orgMegahertz, More Is BetterminiBUDE 20210901CPU Peak Freq (Highest CPU Core Frequency) Monitor8001600240032004000
MinAvgMaxAVX512 On19.7268.3323.2AVX512 Off19.7292.3341.7OpenBenchmarking.orgWatts, Fewer Is BetterminiBUDE 20210901CPU Power Consumption Monitor801602403204001. AVX512 On: Approximate power consumption of 6886 Joules per run.2. AVX512 Off: Approximate power consumption of 9353 Joules per run.
MinAvgMaxAVX512 On29.353.462.3AVX512 Off27.955.164.1OpenBenchmarking.orgCelsius, Fewer Is BetterminiBUDE 20210901CPU Temperature Monitor20406080100

OpenBenchmarking.orgGFInst/s, More Is BetterminiBUDE 20210901Implementation: OpenMP - Input Deck: BM2AVX512 OnAVX512 Off13002600390052006500SE +/- 0.44, N = 3SE +/- 15.49, N = 35972.194528.311. (CC) gcc options: -std=c99 -Ofast -ffast-math -fopenmp -march=native -lm

TensorFlow

This is a benchmark of the TensorFlow deep learning framework using the TensorFlow reference benchmarks (tensorflow/benchmarks with tf_cnn_benchmarks.py). Note with the Phoronix Test Suite there is also pts/tensorflow-lite for benchmarking the TensorFlow Lite binaries if desired for complementary metrics. Learn more via the OpenBenchmarking.org test page.

OpenBenchmarking.orgimages/sec, More Is BetterTensorFlow 2.12Device: CPU - Batch Size: 16 - Model: AlexNetAVX512 OnAVX512 Off70140210280350SE +/- 0.70, N = 6SE +/- 0.29, N = 3342.8870.10
OpenBenchmarking.orgimages/sec Per Watt, More Is BetterTensorFlow 2.12Device: CPU - Batch Size: 16 - Model: AlexNetAVX512 OnAVX512 Off0.66741.33482.00222.66963.3372.9660.576
MinAvgMaxAVX512 On225028013254AVX512 Off225029923136OpenBenchmarking.orgMegahertz, More Is BetterTensorFlow 2.12CPU Peak Freq (Highest CPU Core Frequency) Monitor8001600240032004000
MinAvgMaxAVX512 On10.4115.6199.6AVX512 Off20.6121.7145.6OpenBenchmarking.orgWatts, Fewer Is BetterTensorFlow 2.12CPU Power Consumption Monitor501001502002501. AVX512 On: Approximate power consumption of 1291 Joules per run.2. AVX512 Off: Approximate power consumption of 3812 Joules per run.
MinAvgMaxAVX512 On34.941.149.4AVX512 Off33.940.349.4OpenBenchmarking.orgCelsius, Fewer Is BetterTensorFlow 2.12CPU Temperature Monitor1428425670

Embree

Intel Embree is a collection of high-performance ray-tracing kernels for execution on CPUs (and GPUs via SYCL) and supporting instruction sets such as SSE, AVX, AVX2, and AVX-512. Embree also supports making use of the Intel SPMD Program Compiler (ISPC). Learn more via the OpenBenchmarking.org test page.

OpenBenchmarking.orgFrames Per Second, More Is BetterEmbree 4.1Binary: Pathtracer ISPC - Model: Asian Dragon ObjAVX512 OnAVX512 Off306090120150SE +/- 0.16, N = 4SE +/- 0.08, N = 4134.84115.27MIN: 132.49 / MAX: 139MIN: 113.48 / MAX: 119.15
OpenBenchmarking.orgFrames Per Second Per Watt, More Is BetterEmbree 4.1Binary: Pathtracer ISPC - Model: Asian Dragon ObjAVX512 OnAVX512 Off0.24080.48160.72240.96321.2041.0700.842
MinAvgMaxAVX512 On225028413101AVX512 Off225028553098OpenBenchmarking.orgMegahertz, More Is BetterEmbree 4.1CPU Peak Freq (Highest CPU Core Frequency) Monitor8001600240032004000
MinAvgMaxAVX512 On20.3126.0325.6AVX512 Off20.2136.9330.7OpenBenchmarking.orgWatts, Fewer Is BetterEmbree 4.1CPU Power Consumption Monitor801602403204001. AVX512 On: Approximate power consumption of 2426 Joules per run.2. AVX512 Off: Approximate power consumption of 2773 Joules per run.
MinAvgMaxAVX512 On32.539.347.0AVX512 Off32.536.747.0OpenBenchmarking.orgCelsius, Fewer Is BetterEmbree 4.1CPU Temperature Monitor1428425670

OpenBenchmarking.orgFrames Per Second, More Is BetterEmbree 4.1Binary: Pathtracer ISPC - Model: CrownAVX512 OnAVX512 Off306090120150SE +/- 0.09, N = 7SE +/- 0.11, N = 7125.54112.23MIN: 122.35 / MAX: 131.95MIN: 109.49 / MAX: 117.02
OpenBenchmarking.orgFrames Per Second Per Watt, More Is BetterEmbree 4.1Binary: Pathtracer ISPC - Model: CrownAVX512 OnAVX512 Off0.15120.30240.45360.60480.7560.6720.560
MinAvgMaxAVX512 On225025113101AVX512 Off225026273097OpenBenchmarking.orgMegahertz, More Is BetterEmbree 4.1CPU Peak Freq (Highest CPU Core Frequency) Monitor8001600240032004000
MinAvgMaxAVX512 On20.3186.8333.7AVX512 Off20.6200.3339.8OpenBenchmarking.orgWatts, Fewer Is BetterEmbree 4.1CPU Power Consumption Monitor801602403204001. AVX512 On: Approximate power consumption of 1761 Joules per run.2. AVX512 Off: Approximate power consumption of 2061 Joules per run.
MinAvgMaxAVX512 On33.543.251.3AVX512 Off33.344.252.8OpenBenchmarking.orgCelsius, Fewer Is BetterEmbree 4.1CPU Temperature Monitor1530456075

OpenBenchmarking.orgFrames Per Second, More Is BetterEmbree 4.1Binary: Pathtracer ISPC - Model: Asian DragonAVX512 OnAVX512 Off306090120150SE +/- 0.09, N = 8SE +/- 0.11, N = 7157.65132.95MIN: 155.33 / MAX: 162.92MIN: 130.4 / MAX: 138.16
OpenBenchmarking.orgFrames Per Second Per Watt, More Is BetterEmbree 4.1Binary: Pathtracer ISPC - Model: Asian DragonAVX512 OnAVX512 Off0.2070.4140.6210.8281.0350.9200.715
MinAvgMaxAVX512 On225025653098AVX512 Off225025843096OpenBenchmarking.orgMegahertz, More Is BetterEmbree 4.1CPU Peak Freq (Highest CPU Core Frequency) Monitor8001600240032004000
MinAvgMaxAVX512 On20.1171.3327.2AVX512 Off20.4186.1333.2OpenBenchmarking.orgWatts, Fewer Is BetterEmbree 4.1CPU Power Consumption Monitor801602403204001. AVX512 On: Approximate power consumption of 1478 Joules per run.2. AVX512 Off: Approximate power consumption of 1754 Joules per run.
MinAvgMaxAVX512 On33.541.448.9AVX512 Off33.542.250.0OpenBenchmarking.orgCelsius, Fewer Is BetterEmbree 4.1CPU Temperature Monitor1428425670

miniBUDE

MiniBUDE is a mini application for the the core computation of the Bristol University Docking Engine (BUDE). This test profile currently makes use of the OpenMP implementation of miniBUDE for CPU benchmarking. Learn more via the OpenBenchmarking.org test page.

OpenBenchmarking.orgBillion Interactions/s, More Is BetterminiBUDE 20210901Implementation: OpenMP - Input Deck: BM1AVX512 OnAVX512 Off50100150200250SE +/- 0.09, N = 9SE +/- 0.72, N = 8237.03187.111. (CC) gcc options: -std=c99 -Ofast -ffast-math -fopenmp -march=native -lm
OpenBenchmarking.orgBillion Interactions/s Per Watt, More Is BetterminiBUDE 20210901Implementation: OpenMP - Input Deck: BM1AVX512 OnAVX512 Off0.34880.69761.04641.39521.7441.5501.026
MinAvgMaxAVX512 On225025973098AVX512 Off225025293096OpenBenchmarking.orgMegahertz, More Is BetterminiBUDE 20210901CPU Peak Freq (Highest CPU Core Frequency) Monitor8001600240032004000
MinAvgMaxAVX512 On19.2152.9317.5AVX512 Off19.5182.4337.5OpenBenchmarking.orgWatts, Fewer Is BetterminiBUDE 20210901CPU Power Consumption Monitor801602403204001. AVX512 On: Approximate power consumption of 1138 Joules per run.2. AVX512 Off: Approximate power consumption of 1505 Joules per run.
MinAvgMaxAVX512 On23.336.350.8AVX512 Off20.837.052.8OpenBenchmarking.orgCelsius, Fewer Is BetterminiBUDE 20210901CPU Temperature Monitor1530456075

OpenBenchmarking.orgGFInst/s, More Is BetterminiBUDE 20210901Implementation: OpenMP - Input Deck: BM1AVX512 OnAVX512 Off13002600390052006500SE +/- 2.27, N = 9SE +/- 18.03, N = 85925.674677.681. (CC) gcc options: -std=c99 -Ofast -ffast-math -fopenmp -march=native -lm

CPU Temperature Monitor

OpenBenchmarking.orgCelsiusCPU Temperature MonitorPhoronix Test Suite System MonitoringAVX512 OnAVX512 Off1530456075Min: 23.25 / Avg: 51.4 / Max: 74.25Min: 20.75 / Avg: 44.22 / Max: 76.13

CPU Power Consumption Monitor

OpenBenchmarking.orgWattsCPU Power Consumption MonitorPhoronix Test Suite System MonitoringAVX512 OnAVX512 Off70140210280350Min: 10.25 / Avg: 231.36 / Max: 398.39Min: 10.15 / Avg: 179.15 / Max: 378.14

CPU Peak Freq (Highest CPU Core Frequency) Monitor

OpenBenchmarking.orgMegahertzCPU Peak Freq (Highest CPU Core Frequency) MonitorPhoronix Test Suite System MonitoringAVX512 OnAVX512 Off6001200180024003000Min: 2250 / Avg: 2918.06 / Max: 3532Min: 2203 / Avg: 2979.69 / Max: 3559

80 Results Shown

TensorFlow:
  CPU - 512 - ResNet-50
  CPU - 256 - ResNet-50
  CPU - 512 - GoogLeNet
libxsmm
OpenVKL
TensorFlow
OpenVINO:
  Machine Translation EN To DE FP16 - CPU:
    ms
    FPS
  Vehicle Detection FP16 - CPU:
    ms
    FPS
TensorFlow:
  CPU - 512 - AlexNet
  CPU - 64 - ResNet-50
OpenVINO:
  Person Detection FP32 - CPU:
    ms
    FPS
  Person Vehicle Bike Detection FP16 - CPU:
    ms
    FPS
  Vehicle Detection FP16-INT8 - CPU:
    ms
    FPS
libxsmm
OpenVINO:
  Person Detection FP16 - CPU:
    ms
    FPS
TensorFlow:
  CPU - 64 - GoogLeNet
  CPU - 256 - AlexNet
Neural Magic DeepSparse:
  NLP Question Answering, BERT base uncased SQuaD 12layer Pruned90 - Asynchronous Multi-Stream:
    ms/batch
    items/sec
TensorFlow
Cpuminer-Opt
TensorFlow
oneDNN
Cpuminer-Opt:
  Blake-2 S
  Garlicoin
TensorFlow:
  CPU - 16 - ResNet-50
  CPU - 16 - GoogLeNet
OpenVINO:
  Face Detection FP16 - CPU:
    ms
    FPS
  Face Detection FP16-INT8 - CPU:
    ms
    FPS
  Age Gender Recognition Retail 0013 FP16 - CPU:
    ms
    FPS
  Age Gender Recognition Retail 0013 FP16-INT8 - CPU:
    ms
    FPS
  Weld Porosity Detection FP16-INT8 - CPU:
    ms
    FPS
  Weld Porosity Detection FP16 - CPU:
    ms
    FPS
Neural Magic DeepSparse:
  CV Segmentation, 90% Pruned YOLACT Pruned - Asynchronous Multi-Stream:
    ms/batch
    items/sec
OSPRay:
  gravity_spheres_volume/dim_512/scivis/real_time
  gravity_spheres_volume/dim_512/pathtracer/real_time
  gravity_spheres_volume/dim_512/ao/real_time
Neural Magic DeepSparse:
  NLP Token Classification, BERT base uncased conll2003 - Asynchronous Multi-Stream:
    ms/batch
    items/sec
  NLP Document Classification, oBERT base uncased on IMDB - Asynchronous Multi-Stream:
    ms/batch
    items/sec
  NLP Text Classification, BERT base uncased SST2 - Asynchronous Multi-Stream:
    ms/batch
    items/sec
  NLP Sentiment Analysis, 80% Pruned Quantized BERT Base Uncased - Asynchronous Multi-Stream:
    ms/batch
    items/sec
TensorFlow
Neural Magic DeepSparse:
  NLP Text Classification, DistilBERT mnli - Asynchronous Multi-Stream:
    ms/batch
    items/sec
  CV Classification, ResNet-50 ImageNet - Asynchronous Multi-Stream:
    ms/batch
    items/sec
Cpuminer-Opt
TensorFlow
Cpuminer-Opt:
  Skeincoin
  scrypt
  LBC, LBRY Credits
  Quad SHA-256, Pyrite
miniBUDE:
  OpenMP - BM2:
    Billion Interactions/s
    GFInst/s
TensorFlow
Embree:
  Pathtracer ISPC - Asian Dragon Obj
  Pathtracer ISPC - Crown
  Pathtracer ISPC - Asian Dragon
miniBUDE:
  OpenMP - BM1:
    Billion Interactions/s
    GFInst/s
CPU Temperature Monitor:
  Phoronix Test Suite System Monitoring:
    Celsius
    Watts
    Megahertz

AVX512 On

Kernel Notes: Transparent Huge Pages: madvise
Compiler Notes: --build=x86_64-linux-gnu --disable-vtable-verify --disable-werror --enable-bootstrap --enable-cet --enable-checking=release --enable-clocale=gnu --enable-default-pie --enable-gnu-unique-object --enable-languages=c,ada,c++,go,brig,d,fortran,objc,obj-c++,m2 --enable-libphobos-checking=release --enable-libstdcxx-debug --enable-libstdcxx-time=yes --enable-link-serialization=2 --enable-multiarch --enable-multilib --enable-nls --enable-objc-gc=auto --enable-offload-targets=nvptx-none=/build/gcc-11-xKiWfi/gcc-11-11.3.0/debian/tmp-nvptx/usr,amdgcn-amdhsa=/build/gcc-11-xKiWfi/gcc-11-11.3.0/debian/tmp-gcn/usr --enable-plugin --enable-shared --enable-threads=posix --host=x86_64-linux-gnu --program-prefix=x86_64-linux-gnu- --target=x86_64-linux-gnu --with-abi=m64 --with-arch-32=i686 --with-build-config=bootstrap-lto-lean --with-default-libstdcxx-abi=new --with-gcc-major-version-only --with-multilib-list=m32,m64,mx32 --with-target-system-zlib=auto --with-tune=generic --without-cuda-driver -v
Processor Notes: Scaling Governor: acpi-cpufreq performance (Boost: Enabled) - CPU Microcode: 0xaa0010b
Python Notes: Python 3.10.6
Security Notes: itlb_multihit: Not affected + l1tf: Not affected + mds: Not affected + meltdown: Not affected + mmio_stale_data: Not affected + retbleed: Not affected + spec_store_bypass: Mitigation of SSB disabled via prctl + spectre_v1: Mitigation of usercopy/swapgs barriers and __user pointer sanitization + spectre_v2: Mitigation of Retpolines IBPB: conditional IBRS_FW STIBP: always-on RSB filling PBRSB-eIBRS: Not affected + srbds: Not affected + tsx_async_abort: Not affected

Testing initiated at 16 July 2023 06:38 by user phoronix.

AVX512 Off

Processor: AMD EPYC 9754 128-Core @ 2.25GHz (128 Cores / 256 Threads), Motherboard: AMD Titanite_4G (RTI1007B BIOS), Chipset: AMD Device 14a4, Memory: 768GB, Disk: 2 x 1920GB SAMSUNG MZWLJ1T9HBJR-00007, Graphics: ASPEED, Network: Broadcom NetXtreme BCM5720 PCIe

OS: Ubuntu 22.04, Kernel: 5.19.0-41-generic (x86_64), Desktop: GNOME Shell 42.5, Display Server: X Server 1.21.1.4, Vulkan: 1.3.224, Compiler: GCC 11.3.0, File-System: ext4, Screen Resolution: 1024x768

Kernel Notes: Transparent Huge Pages: madvise
Compiler Notes: --build=x86_64-linux-gnu --disable-vtable-verify --disable-werror --enable-bootstrap --enable-cet --enable-checking=release --enable-clocale=gnu --enable-default-pie --enable-gnu-unique-object --enable-languages=c,ada,c++,go,brig,d,fortran,objc,obj-c++,m2 --enable-libphobos-checking=release --enable-libstdcxx-debug --enable-libstdcxx-time=yes --enable-link-serialization=2 --enable-multiarch --enable-multilib --enable-nls --enable-objc-gc=auto --enable-offload-targets=nvptx-none=/build/gcc-11-xKiWfi/gcc-11-11.3.0/debian/tmp-nvptx/usr,amdgcn-amdhsa=/build/gcc-11-xKiWfi/gcc-11-11.3.0/debian/tmp-gcn/usr --enable-plugin --enable-shared --enable-threads=posix --host=x86_64-linux-gnu --program-prefix=x86_64-linux-gnu- --target=x86_64-linux-gnu --with-abi=m64 --with-arch-32=i686 --with-build-config=bootstrap-lto-lean --with-default-libstdcxx-abi=new --with-gcc-major-version-only --with-multilib-list=m32,m64,mx32 --with-target-system-zlib=auto --with-tune=generic --without-cuda-driver -v
Processor Notes: Scaling Governor: acpi-cpufreq performance (Boost: Enabled) - CPU Microcode: 0xaa0010b
Python Notes: Python 3.10.6
Security Notes: itlb_multihit: Not affected + l1tf: Not affected + mds: Not affected + meltdown: Not affected + mmio_stale_data: Not affected + retbleed: Not affected + spec_store_bypass: Mitigation of SSB disabled via prctl + spectre_v1: Mitigation of usercopy/swapgs barriers and __user pointer sanitization + spectre_v2: Mitigation of Retpolines IBPB: conditional IBRS_FW STIBP: always-on RSB filling PBRSB-eIBRS: Not affected + srbds: Not affected + tsx_async_abort: Not affected

Testing initiated at 16 July 2023 14:04 by user phoronix.