AMD EPYC 9754 1P benchmarks with AVX-512 benchmarking and then AVX-512 disabled. Tests by Michael Larabel for a future article.
Compare your own system(s) to this result file with the
Phoronix Test Suite by running the command:
phoronix-test-suite benchmark 2307197-NE-AMDBERGAM43 AMD EPYC 9754 Bergamo AVX-512 - Phoronix Test Suite AMD EPYC 9754 Bergamo AVX-512 AMD EPYC 9754 1P benchmarks with AVX-512 benchmarking and then AVX-512 disabled. Tests by Michael Larabel for a future article.
HTML result view exported from: https://openbenchmarking.org/result/2307197-NE-AMDBERGAM43&grt&sor .
AMD EPYC 9754 Bergamo AVX-512 Processor Motherboard Chipset Memory Disk Graphics Network OS Kernel Desktop Display Server Vulkan Compiler File-System Screen Resolution AVX512 On AVX512 Off AMD EPYC 9754 128-Core @ 2.25GHz (128 Cores / 256 Threads) AMD Titanite_4G (RTI1007B BIOS) AMD Device 14a4 768GB 2 x 1920GB SAMSUNG MZWLJ1T9HBJR-00007 ASPEED Broadcom NetXtreme BCM5720 PCIe Ubuntu 22.04 5.19.0-41-generic (x86_64) GNOME Shell 42.5 X Server 1.21.1.4 1.3.224 GCC 11.3.0 ext4 1024x768 OpenBenchmarking.org Kernel Details - Transparent Huge Pages: madvise Compiler Details - --build=x86_64-linux-gnu --disable-vtable-verify --disable-werror --enable-bootstrap --enable-cet --enable-checking=release --enable-clocale=gnu --enable-default-pie --enable-gnu-unique-object --enable-languages=c,ada,c++,go,brig,d,fortran,objc,obj-c++,m2 --enable-libphobos-checking=release --enable-libstdcxx-debug --enable-libstdcxx-time=yes --enable-link-serialization=2 --enable-multiarch --enable-multilib --enable-nls --enable-objc-gc=auto --enable-offload-targets=nvptx-none=/build/gcc-11-xKiWfi/gcc-11-11.3.0/debian/tmp-nvptx/usr,amdgcn-amdhsa=/build/gcc-11-xKiWfi/gcc-11-11.3.0/debian/tmp-gcn/usr --enable-plugin --enable-shared --enable-threads=posix --host=x86_64-linux-gnu --program-prefix=x86_64-linux-gnu- --target=x86_64-linux-gnu --with-abi=m64 --with-arch-32=i686 --with-build-config=bootstrap-lto-lean --with-default-libstdcxx-abi=new --with-gcc-major-version-only --with-multilib-list=m32,m64,mx32 --with-target-system-zlib=auto --with-tune=generic --without-cuda-driver -v Processor Details - Scaling Governor: acpi-cpufreq performance (Boost: Enabled) - CPU Microcode: 0xaa0010b Python Details - Python 3.10.6 Security Details - itlb_multihit: Not affected + l1tf: Not affected + mds: Not affected + meltdown: Not affected + mmio_stale_data: Not affected + retbleed: Not affected + spec_store_bypass: Mitigation of SSB disabled via prctl + spectre_v1: Mitigation of usercopy/swapgs barriers and __user pointer sanitization + spectre_v2: Mitigation of Retpolines IBPB: conditional IBRS_FW STIBP: always-on RSB filling PBRSB-eIBRS: Not affected + srbds: Not affected + tsx_async_abort: Not affected
AMD EPYC 9754 Bergamo AVX-512 cpuminer-opt: x25x cpuminer-opt: scrypt cpuminer-opt: Blake-2 S cpuminer-opt: Garlicoin cpuminer-opt: Skeincoin cpuminer-opt: Myriad-Groestl cpuminer-opt: LBC, LBRY Credits cpuminer-opt: Quad SHA-256, Pyrite embree: Pathtracer ISPC - Crown embree: Pathtracer ISPC - Asian Dragon embree: Pathtracer ISPC - Asian Dragon Obj libxsmm: 128 libxsmm: 256 minibude: OpenMP - BM1 minibude: OpenMP - BM1 minibude: OpenMP - BM2 minibude: OpenMP - BM2 deepsparse: NLP Document Classification, oBERT base uncased on IMDB - Asynchronous Multi-Stream deepsparse: NLP Document Classification, oBERT base uncased on IMDB - Asynchronous Multi-Stream deepsparse: NLP Sentiment Analysis, 80% Pruned Quantized BERT Base Uncased - Asynchronous Multi-Stream deepsparse: NLP Sentiment Analysis, 80% Pruned Quantized BERT Base Uncased - Asynchronous Multi-Stream deepsparse: NLP Question Answering, BERT base uncased SQuaD 12layer Pruned90 - Asynchronous Multi-Stream deepsparse: NLP Question Answering, BERT base uncased SQuaD 12layer Pruned90 - Asynchronous Multi-Stream deepsparse: CV Classification, ResNet-50 ImageNet - Asynchronous Multi-Stream deepsparse: CV Classification, ResNet-50 ImageNet - Asynchronous Multi-Stream deepsparse: NLP Text Classification, DistilBERT mnli - Asynchronous Multi-Stream deepsparse: NLP Text Classification, DistilBERT mnli - Asynchronous Multi-Stream deepsparse: CV Segmentation, 90% Pruned YOLACT Pruned - Asynchronous Multi-Stream deepsparse: CV Segmentation, 90% Pruned YOLACT Pruned - Asynchronous Multi-Stream deepsparse: NLP Text Classification, BERT base uncased SST2 - Asynchronous Multi-Stream deepsparse: NLP Text Classification, BERT base uncased SST2 - Asynchronous Multi-Stream deepsparse: NLP Token Classification, BERT base uncased conll2003 - Asynchronous Multi-Stream deepsparse: NLP Token Classification, BERT base uncased conll2003 - Asynchronous Multi-Stream onednn: Recurrent Neural Network Training - bf16bf16bf16 - CPU openvino: Face Detection FP16 - CPU openvino: Face Detection FP16 - CPU openvino: Person Detection FP16 - CPU openvino: Person Detection FP16 - CPU openvino: Person Detection FP32 - CPU openvino: Person Detection FP32 - CPU openvino: Vehicle Detection FP16 - CPU openvino: Vehicle Detection FP16 - CPU openvino: Face Detection FP16-INT8 - CPU openvino: Face Detection FP16-INT8 - CPU openvino: Vehicle Detection FP16-INT8 - CPU openvino: Vehicle Detection FP16-INT8 - CPU openvino: Weld Porosity Detection FP16 - CPU openvino: Weld Porosity Detection FP16 - CPU openvino: Machine Translation EN To DE FP16 - CPU openvino: Machine Translation EN To DE FP16 - CPU openvino: Weld Porosity Detection FP16-INT8 - CPU openvino: Weld Porosity Detection FP16-INT8 - CPU openvino: Person Vehicle Bike Detection FP16 - CPU openvino: Person Vehicle Bike Detection FP16 - CPU openvino: Age Gender Recognition Retail 0013 FP16 - CPU openvino: Age Gender Recognition Retail 0013 FP16 - CPU openvino: Age Gender Recognition Retail 0013 FP16-INT8 - CPU openvino: Age Gender Recognition Retail 0013 FP16-INT8 - CPU openvkl: vklBenchmark ISPC ospray: gravity_spheres_volume/dim_512/ao/real_time ospray: gravity_spheres_volume/dim_512/scivis/real_time ospray: gravity_spheres_volume/dim_512/pathtracer/real_time tensorflow: CPU - 16 - AlexNet tensorflow: CPU - 32 - AlexNet tensorflow: CPU - 64 - AlexNet tensorflow: CPU - 256 - AlexNet tensorflow: CPU - 512 - AlexNet tensorflow: CPU - 16 - GoogLeNet tensorflow: CPU - 16 - ResNet-50 tensorflow: CPU - 32 - GoogLeNet tensorflow: CPU - 32 - ResNet-50 tensorflow: CPU - 64 - GoogLeNet tensorflow: CPU - 64 - ResNet-50 tensorflow: CPU - 256 - GoogLeNet tensorflow: CPU - 256 - ResNet-50 tensorflow: CPU - 512 - GoogLeNet tensorflow: CPU - 512 - ResNet-50 AVX512 On AVX512 Off 4977.89 2993.21 7238650 53090 1174953 8628.76 660660 1498937 125.5414 157.6450 134.8396 2690.7 3342.5 5925.670 237.027 5972.187 238.887 73.5393 858.4029 1381.5669 46.2608 247.9653 259.6468 970.0568 65.8909 624.7370 102.2238 127.0611 498.4779 316.1679 201.5964 73.1459 859.7119 1174.75 60.73 1048.37 27.08 2334.29 27.01 2339.81 1430.45 44.84 118.00 540.35 5690.34 11.26 6073.22 10.52 580.40 110.35 11818.33 10.82 6638.71 9.63 110240.89 0.99 73970.12 1.58 1398 32.7557 31.6753 27.9715 342.88 562.48 857.55 1422.36 1632.40 104.77 43.11 180.77 71.62 277.24 96.63 501.67 119.32 417.25 122.81 3010.37 2033.15 4555580 39473 937977 7149.95 332667 876137 112.2274 132.9504 115.2682 2573.3 2415.3 4677.682 187.107 4528.305 181.132 61.3689 1023.1690 718.3032 88.9093 213.6393 298.0628 870.9541 73.3502 522.1104 122.2295 69.3341 906.9414 260.7671 244.1513 61.1755 1025.4806 1317.57 26.18 2423.39 15.06 4153.59 14.99 4170.49 1190.91 53.79 57.95 1094.52 3954.90 16.17 2551.70 25.06 251.97 254.01 5692.99 22.47 3317.34 19.28 62564.16 1.91 66895.49 1.76 1179 19.1731 18.0203 19.5463 70.10 84.96 97.59 106.28 109.88 39.64 15.91 44.46 17.48 46.28 18.45 46.17 19.89 46.92 20.78 OpenBenchmarking.org
CPU Peak Freq (Highest CPU Core Frequency) Monitor Phoronix Test Suite System Monitoring OpenBenchmarking.org Megahertz CPU Peak Freq (Highest CPU Core Frequency) Monitor Phoronix Test Suite System Monitoring AVX512 On AVX512 Off 600 1200 1800 2400 3000 Min: 2250 / Avg: 2918.06 / Max: 3532 Min: 2203 / Avg: 2979.69 / Max: 3559
CPU Power Consumption Monitor Phoronix Test Suite System Monitoring OpenBenchmarking.org Watts CPU Power Consumption Monitor Phoronix Test Suite System Monitoring AVX512 Off AVX512 On 70 140 210 280 350 Min: 10.15 / Avg: 179.15 / Max: 378.14 Min: 10.25 / Avg: 231.36 / Max: 398.39
CPU Temperature Monitor Phoronix Test Suite System Monitoring OpenBenchmarking.org Celsius CPU Temperature Monitor Phoronix Test Suite System Monitoring AVX512 Off AVX512 On 15 30 45 60 75 Min: 20.75 / Avg: 44.22 / Max: 76.13 Min: 23.25 / Avg: 51.4 / Max: 74.25
Cpuminer-Opt Algorithm: x25x OpenBenchmarking.org kH/s, More Is Better Cpuminer-Opt 3.20.3 Algorithm: x25x AVX512 On AVX512 Off 1100 2200 3300 4400 5500 SE +/- 15.71, N = 3 SE +/- 35.48, N = 4 4977.89 3010.37 1. (CXX) g++ options: -O2 -lcurl -lz -lpthread -lssl -lcrypto -lgmp
Cpuminer-Opt Algorithm: x25x OpenBenchmarking.org kH/s Per Watt, More Is Better Cpuminer-Opt 3.20.3 Algorithm: x25x AVX512 On AVX512 Off 4 8 12 16 20 16.85 10.81
Cpuminer-Opt CPU Peak Freq (Highest CPU Core Frequency) Monitor Min Avg Max AVX512 Off 2250 3019 3099 AVX512 On 2250 2909 3099 OpenBenchmarking.org Megahertz, More Is Better Cpuminer-Opt 3.20.3 CPU Peak Freq (Highest CPU Core Frequency) Monitor 800 1600 2400 3200 4000
Cpuminer-Opt CPU Power Consumption Monitor Min Avg Max AVX512 Off 21.0 278.5 321.6 AVX512 On 20.4 295.4 341.0 OpenBenchmarking.org Watts, Fewer Is Better Cpuminer-Opt 3.20.3 CPU Power Consumption Monitor 80 160 240 320 400
Cpuminer-Opt CPU Temperature Monitor Min Avg Max AVX512 Off 37.6 55.2 59.5 AVX512 On 39.4 58.7 64.3 OpenBenchmarking.org Celsius, Fewer Is Better Cpuminer-Opt 3.20.3 CPU Temperature Monitor 20 40 60 80 100
Cpuminer-Opt Algorithm: scrypt OpenBenchmarking.org kH/s, More Is Better Cpuminer-Opt 3.20.3 Algorithm: scrypt AVX512 On AVX512 Off 600 1200 1800 2400 3000 SE +/- 1.66, N = 3 SE +/- 0.24, N = 3 2993.21 2033.15 1. (CXX) g++ options: -O2 -lcurl -lz -lpthread -lssl -lcrypto -lgmp
Cpuminer-Opt Algorithm: scrypt OpenBenchmarking.org kH/s Per Watt, More Is Better Cpuminer-Opt 3.20.3 Algorithm: scrypt AVX512 On AVX512 Off 2 4 6 8 10 8.913 6.476
Cpuminer-Opt CPU Peak Freq (Highest CPU Core Frequency) Monitor Min Avg Max AVX512 On 2250 2959 3095 AVX512 Off 2250 2952 3091 OpenBenchmarking.org Megahertz, More Is Better Cpuminer-Opt 3.20.3 CPU Peak Freq (Highest CPU Core Frequency) Monitor 800 1600 2400 3200 4000
Cpuminer-Opt CPU Power Consumption Monitor Min Avg Max AVX512 Off 20.9 314.0 361.0 AVX512 On 20.9 335.8 385.3 OpenBenchmarking.org Watts, Fewer Is Better Cpuminer-Opt 3.20.3 CPU Power Consumption Monitor 100 200 300 400 500
Cpuminer-Opt CPU Temperature Monitor Min Avg Max AVX512 Off 38.1 63.6 70.0 AVX512 On 39.5 66.4 73.3 OpenBenchmarking.org Celsius, Fewer Is Better Cpuminer-Opt 3.20.3 CPU Temperature Monitor 20 40 60 80 100
Cpuminer-Opt Algorithm: Blake-2 S OpenBenchmarking.org kH/s, More Is Better Cpuminer-Opt 3.20.3 Algorithm: Blake-2 S AVX512 On AVX512 Off 1.6M 3.2M 4.8M 6.4M 8M SE +/- 3854.05, N = 3 SE +/- 37514.23, N = 15 7238650 4555580 1. (CXX) g++ options: -O2 -lcurl -lz -lpthread -lssl -lcrypto -lgmp
Cpuminer-Opt Algorithm: Blake-2 S OpenBenchmarking.org kH/s Per Watt, More Is Better Cpuminer-Opt 3.20.3 Algorithm: Blake-2 S AVX512 On AVX512 Off 5K 10K 15K 20K 25K 22160.82 14551.24
Cpuminer-Opt CPU Peak Freq (Highest CPU Core Frequency) Monitor Min Avg Max AVX512 On 2250 2925 3002 AVX512 Off 2250 2779 3096 OpenBenchmarking.org Megahertz, More Is Better Cpuminer-Opt 3.20.3 CPU Peak Freq (Highest CPU Core Frequency) Monitor 800 1600 2400 3200 4000
Cpuminer-Opt CPU Power Consumption Monitor Min Avg Max AVX512 Off 21.0 313.1 358.1 AVX512 On 20.4 326.6 376.0 OpenBenchmarking.org Watts, Fewer Is Better Cpuminer-Opt 3.20.3 CPU Power Consumption Monitor 100 200 300 400 500
Cpuminer-Opt CPU Temperature Monitor Min Avg Max AVX512 Off 31.1 64.0 70.0 AVX512 On 33.9 64.0 72.8 OpenBenchmarking.org Celsius, Fewer Is Better Cpuminer-Opt 3.20.3 CPU Temperature Monitor 20 40 60 80 100
Cpuminer-Opt Algorithm: Garlicoin OpenBenchmarking.org kH/s, More Is Better Cpuminer-Opt 3.20.3 Algorithm: Garlicoin AVX512 On AVX512 Off 11K 22K 33K 44K 55K SE +/- 110.15, N = 3 SE +/- 102.69, N = 3 53090 39473 1. (CXX) g++ options: -O2 -lcurl -lz -lpthread -lssl -lcrypto -lgmp
Cpuminer-Opt Algorithm: Garlicoin OpenBenchmarking.org kH/s Per Watt, More Is Better Cpuminer-Opt 3.20.3 Algorithm: Garlicoin AVX512 On AVX512 Off 60 120 180 240 300 272.43 202.32
Cpuminer-Opt CPU Peak Freq (Highest CPU Core Frequency) Monitor Min Avg Max AVX512 Off 2250 3065 3102 AVX512 On 2250 3058 3102 OpenBenchmarking.org Megahertz, More Is Better Cpuminer-Opt 3.20.3 CPU Peak Freq (Highest CPU Core Frequency) Monitor 800 1600 2400 3200 4000
Cpuminer-Opt CPU Power Consumption Monitor Min Avg Max AVX512 On 20.7 194.9 210.1 AVX512 Off 20.8 195.1 204.9 OpenBenchmarking.org Watts, Fewer Is Better Cpuminer-Opt 3.20.3 CPU Power Consumption Monitor 60 120 180 240 300
Cpuminer-Opt CPU Temperature Monitor Min Avg Max AVX512 Off 36.8 43.7 46.6 AVX512 On 38.1 45.4 47.9 OpenBenchmarking.org Celsius, Fewer Is Better Cpuminer-Opt 3.20.3 CPU Temperature Monitor 14 28 42 56 70
Cpuminer-Opt Algorithm: Skeincoin OpenBenchmarking.org kH/s, More Is Better Cpuminer-Opt 3.20.3 Algorithm: Skeincoin AVX512 On AVX512 Off 300K 600K 900K 1200K 1500K SE +/- 4577.90, N = 3 SE +/- 1811.04, N = 3 1174953 937977 1. (CXX) g++ options: -O2 -lcurl -lz -lpthread -lssl -lcrypto -lgmp
Cpuminer-Opt Algorithm: Skeincoin OpenBenchmarking.org kH/s Per Watt, More Is Better Cpuminer-Opt 3.20.3 Algorithm: Skeincoin AVX512 On AVX512 Off 800 1600 2400 3200 4000 3691.56 2892.87
Cpuminer-Opt CPU Peak Freq (Highest CPU Core Frequency) Monitor Min Avg Max AVX512 Off 2250 2899 3096 AVX512 On 2250 2804 3143 OpenBenchmarking.org Megahertz, More Is Better Cpuminer-Opt 3.20.3 CPU Peak Freq (Highest CPU Core Frequency) Monitor 800 1600 2400 3200 4000
Cpuminer-Opt CPU Power Consumption Monitor Min Avg Max AVX512 On 20.2 318.3 370.8 AVX512 Off 20.2 324.2 376.0 OpenBenchmarking.org Watts, Fewer Is Better Cpuminer-Opt 3.20.3 CPU Power Consumption Monitor 100 200 300 400 500
Cpuminer-Opt CPU Temperature Monitor Min Avg Max AVX512 Off 33.0 62.9 71.9 AVX512 On 34.4 63.4 71.9 OpenBenchmarking.org Celsius, Fewer Is Better Cpuminer-Opt 3.20.3 CPU Temperature Monitor 20 40 60 80 100
Cpuminer-Opt Algorithm: Myriad-Groestl OpenBenchmarking.org kH/s, More Is Better Cpuminer-Opt 3.20.3 Algorithm: Myriad-Groestl AVX512 On AVX512 Off 2K 4K 6K 8K 10K SE +/- 340.56, N = 15 SE +/- 21.55, N = 3 8628.76 7149.95 1. (CXX) g++ options: -O2 -lcurl -lz -lpthread -lssl -lcrypto -lgmp
Cpuminer-Opt Algorithm: Myriad-Groestl OpenBenchmarking.org kH/s Per Watt, More Is Better Cpuminer-Opt 3.20.3 Algorithm: Myriad-Groestl AVX512 On AVX512 Off 12 24 36 48 60 51.65 44.67
Cpuminer-Opt CPU Peak Freq (Highest CPU Core Frequency) Monitor Min Avg Max AVX512 Off 2250 3046 3099 AVX512 On 2250 3021 3097 OpenBenchmarking.org Megahertz, More Is Better Cpuminer-Opt 3.20.3 CPU Peak Freq (Highest CPU Core Frequency) Monitor 800 1600 2400 3200 4000
Cpuminer-Opt CPU Power Consumption Monitor Min Avg Max AVX512 Off 20.4 160.1 173.0 AVX512 On 19.9 167.1 199.7 OpenBenchmarking.org Watts, Fewer Is Better Cpuminer-Opt 3.20.3 CPU Power Consumption Monitor 50 100 150 200 250
Cpuminer-Opt CPU Temperature Monitor Min Avg Max AVX512 Off 33.9 40.9 44.6 AVX512 On 34.9 42.2 47.0 OpenBenchmarking.org Celsius, Fewer Is Better Cpuminer-Opt 3.20.3 CPU Temperature Monitor 14 28 42 56 70
Cpuminer-Opt Algorithm: LBC, LBRY Credits OpenBenchmarking.org kH/s, More Is Better Cpuminer-Opt 3.20.3 Algorithm: LBC, LBRY Credits AVX512 On AVX512 Off 140K 280K 420K 560K 700K SE +/- 76.38, N = 3 SE +/- 141.93, N = 3 660660 332667 1. (CXX) g++ options: -O2 -lcurl -lz -lpthread -lssl -lcrypto -lgmp
Cpuminer-Opt Algorithm: LBC, LBRY Credits OpenBenchmarking.org kH/s Per Watt, More Is Better Cpuminer-Opt 3.20.3 Algorithm: LBC, LBRY Credits AVX512 On AVX512 Off 400 800 1200 1600 2000 2003.50 1052.19
Cpuminer-Opt CPU Peak Freq (Highest CPU Core Frequency) Monitor Min Avg Max AVX512 On 2250 2886 3096 AVX512 Off 2250 2746 3096 OpenBenchmarking.org Megahertz, More Is Better Cpuminer-Opt 3.20.3 CPU Peak Freq (Highest CPU Core Frequency) Monitor 800 1600 2400 3200 4000
Cpuminer-Opt CPU Power Consumption Monitor Min Avg Max AVX512 Off 20.7 316.2 362.0 AVX512 On 20.3 329.8 380.7 OpenBenchmarking.org Watts, Fewer Is Better Cpuminer-Opt 3.20.3 CPU Power Consumption Monitor 100 200 300 400 500
Cpuminer-Opt CPU Temperature Monitor Min Avg Max AVX512 Off 30.6 59.5 71.9 AVX512 On 32.0 63.2 72.8 OpenBenchmarking.org Celsius, Fewer Is Better Cpuminer-Opt 3.20.3 CPU Temperature Monitor 20 40 60 80 100
Cpuminer-Opt Algorithm: Quad SHA-256, Pyrite OpenBenchmarking.org kH/s, More Is Better Cpuminer-Opt 3.20.3 Algorithm: Quad SHA-256, Pyrite AVX512 On AVX512 Off 300K 600K 900K 1200K 1500K SE +/- 3455.26, N = 3 SE +/- 2198.34, N = 3 1498937 876137 1. (CXX) g++ options: -O2 -lcurl -lz -lpthread -lssl -lcrypto -lgmp
Cpuminer-Opt Algorithm: Quad SHA-256, Pyrite OpenBenchmarking.org kH/s Per Watt, More Is Better Cpuminer-Opt 3.20.3 Algorithm: Quad SHA-256, Pyrite AVX512 On AVX512 Off 900 1800 2700 3600 4500 4347.58 2825.31
Cpuminer-Opt CPU Peak Freq (Highest CPU Core Frequency) Monitor Min Avg Max AVX512 On 2250 2966 3080 AVX512 Off 2250 2919 3027 OpenBenchmarking.org Megahertz, More Is Better Cpuminer-Opt 3.20.3 CPU Peak Freq (Highest CPU Core Frequency) Monitor 800 1600 2400 3200 4000
Cpuminer-Opt CPU Power Consumption Monitor Min Avg Max AVX512 Off 11.0 310.1 352.7 AVX512 On 21.1 344.8 398.4 OpenBenchmarking.org Watts, Fewer Is Better Cpuminer-Opt 3.20.3 CPU Power Consumption Monitor 110 220 330 440 550
Cpuminer-Opt CPU Temperature Monitor Min Avg Max AVX512 Off 38.6 62.0 68.1 AVX512 On 40.5 68.7 74.3 OpenBenchmarking.org Celsius, Fewer Is Better Cpuminer-Opt 3.20.3 CPU Temperature Monitor 20 40 60 80 100
Embree Binary: Pathtracer ISPC - Model: Crown OpenBenchmarking.org Frames Per Second, More Is Better Embree 4.1 Binary: Pathtracer ISPC - Model: Crown AVX512 On AVX512 Off 30 60 90 120 150 SE +/- 0.09, N = 7 SE +/- 0.11, N = 7 125.54 112.23 MIN: 122.35 / MAX: 131.95 MIN: 109.49 / MAX: 117.02
Embree Binary: Pathtracer ISPC - Model: Crown OpenBenchmarking.org Frames Per Second Per Watt, More Is Better Embree 4.1 Binary: Pathtracer ISPC - Model: Crown AVX512 On AVX512 Off 0.1512 0.3024 0.4536 0.6048 0.756 0.672 0.560
Embree CPU Peak Freq (Highest CPU Core Frequency) Monitor Min Avg Max AVX512 Off 2250 2627 3097 AVX512 On 2250 2511 3101 OpenBenchmarking.org Megahertz, More Is Better Embree 4.1 CPU Peak Freq (Highest CPU Core Frequency) Monitor 800 1600 2400 3200 4000
Embree CPU Power Consumption Monitor Min Avg Max AVX512 On 20.3 186.8 333.7 AVX512 Off 20.6 200.3 339.8 OpenBenchmarking.org Watts, Fewer Is Better Embree 4.1 CPU Power Consumption Monitor 80 160 240 320 400
Embree CPU Temperature Monitor Min Avg Max AVX512 On 33.5 43.2 51.3 AVX512 Off 33.3 44.2 52.8 OpenBenchmarking.org Celsius, Fewer Is Better Embree 4.1 CPU Temperature Monitor 15 30 45 60 75
Embree Binary: Pathtracer ISPC - Model: Asian Dragon OpenBenchmarking.org Frames Per Second, More Is Better Embree 4.1 Binary: Pathtracer ISPC - Model: Asian Dragon AVX512 On AVX512 Off 30 60 90 120 150 SE +/- 0.09, N = 8 SE +/- 0.11, N = 7 157.65 132.95 MIN: 155.33 / MAX: 162.92 MIN: 130.4 / MAX: 138.16
Embree Binary: Pathtracer ISPC - Model: Asian Dragon OpenBenchmarking.org Frames Per Second Per Watt, More Is Better Embree 4.1 Binary: Pathtracer ISPC - Model: Asian Dragon AVX512 On AVX512 Off 0.207 0.414 0.621 0.828 1.035 0.920 0.715
Embree CPU Peak Freq (Highest CPU Core Frequency) Monitor Min Avg Max AVX512 Off 2250 2584 3096 AVX512 On 2250 2565 3098 OpenBenchmarking.org Megahertz, More Is Better Embree 4.1 CPU Peak Freq (Highest CPU Core Frequency) Monitor 800 1600 2400 3200 4000
Embree CPU Power Consumption Monitor Min Avg Max AVX512 On 20.1 171.3 327.2 AVX512 Off 20.4 186.1 333.2 OpenBenchmarking.org Watts, Fewer Is Better Embree 4.1 CPU Power Consumption Monitor 80 160 240 320 400
Embree CPU Temperature Monitor Min Avg Max AVX512 On 33.5 41.4 48.9 AVX512 Off 33.5 42.2 50.0 OpenBenchmarking.org Celsius, Fewer Is Better Embree 4.1 CPU Temperature Monitor 14 28 42 56 70
Embree Binary: Pathtracer ISPC - Model: Asian Dragon Obj OpenBenchmarking.org Frames Per Second, More Is Better Embree 4.1 Binary: Pathtracer ISPC - Model: Asian Dragon Obj AVX512 On AVX512 Off 30 60 90 120 150 SE +/- 0.16, N = 4 SE +/- 0.08, N = 4 134.84 115.27 MIN: 132.49 / MAX: 139 MIN: 113.48 / MAX: 119.15
Embree Binary: Pathtracer ISPC - Model: Asian Dragon Obj OpenBenchmarking.org Frames Per Second Per Watt, More Is Better Embree 4.1 Binary: Pathtracer ISPC - Model: Asian Dragon Obj AVX512 On AVX512 Off 0.2408 0.4816 0.7224 0.9632 1.204 1.070 0.842
Embree CPU Peak Freq (Highest CPU Core Frequency) Monitor Min Avg Max AVX512 Off 2250 2855 3098 AVX512 On 2250 2841 3101 OpenBenchmarking.org Megahertz, More Is Better Embree 4.1 CPU Peak Freq (Highest CPU Core Frequency) Monitor 800 1600 2400 3200 4000
Embree CPU Power Consumption Monitor Min Avg Max AVX512 On 20.3 126.0 325.6 AVX512 Off 20.2 136.9 330.7 OpenBenchmarking.org Watts, Fewer Is Better Embree 4.1 CPU Power Consumption Monitor 80 160 240 320 400
Embree CPU Temperature Monitor Min Avg Max AVX512 Off 32.5 36.7 47.0 AVX512 On 32.5 39.3 47.0 OpenBenchmarking.org Celsius, Fewer Is Better Embree 4.1 CPU Temperature Monitor 14 28 42 56 70
libxsmm M N K: 128 OpenBenchmarking.org GFLOPS/s, More Is Better libxsmm 2-1.17-3645 M N K: 128 AVX512 On AVX512 Off 600 1200 1800 2400 3000 SE +/- 12.20, N = 3 SE +/- 2.70, N = 3 2690.7 2573.3 1. (CXX) g++ options: -dynamic -Bstatic -static-libgcc -lgomp -lm -lrt -ldl -lquadmath -lstdc++ -pthread -fPIC -std=c++14 -O2 -fopenmp-simd -funroll-loops -ftree-vectorize -fdata-sections -ffunction-sections -fvisibility=hidden -msse4.2
libxsmm M N K: 128 OpenBenchmarking.org GFLOPS/s Per Watt, More Is Better libxsmm 2-1.17-3645 M N K: 128 AVX512 On AVX512 Off 3 6 9 12 15 13.07 12.53
libxsmm CPU Peak Freq (Highest CPU Core Frequency) Monitor OpenBenchmarking.org Megahertz, More Is Better libxsmm 2-1.17-3645 CPU Peak Freq (Highest CPU Core Frequency) Monitor AVX512 On AVX512 Off 500 1000 1500 2000 2500 Min: 2250 / Avg: 3087.64 / Max: 3133 Min: 2250 / Avg: 3086 / Max: 3117
libxsmm CPU Power Consumption Monitor OpenBenchmarking.org Watts, Fewer Is Better libxsmm 2-1.17-3645 CPU Power Consumption Monitor AVX512 Off AVX512 On 60 120 180 240 300 Min: 20.58 / Avg: 205.43 / Max: 327.08 Min: 20.28 / Avg: 205.93 / Max: 329.89
libxsmm CPU Temperature Monitor OpenBenchmarking.org Celsius, Fewer Is Better libxsmm 2-1.17-3645 CPU Temperature Monitor AVX512 Off AVX512 On 11 22 33 44 55 Min: 35.5 / Avg: 49.14 / Max: 50.88 Min: 35.75 / Avg: 50.34 / Max: 53.25
libxsmm M N K: 256 OpenBenchmarking.org GFLOPS/s, More Is Better libxsmm 2-1.17-3645 M N K: 256 AVX512 On AVX512 Off 700 1400 2100 2800 3500 SE +/- 5.78, N = 3 SE +/- 6.53, N = 3 3342.5 2415.3 1. (CXX) g++ options: -dynamic -Bstatic -static-libgcc -lgomp -lm -lrt -ldl -lquadmath -lstdc++ -pthread -fPIC -std=c++14 -O2 -fopenmp-simd -funroll-loops -ftree-vectorize -fdata-sections -ffunction-sections -fvisibility=hidden -msse4.2
libxsmm M N K: 256 OpenBenchmarking.org GFLOPS/s Per Watt, More Is Better libxsmm 2-1.17-3645 M N K: 256 AVX512 On AVX512 Off 4 8 12 16 20 15.90 11.40
libxsmm CPU Peak Freq (Highest CPU Core Frequency) Monitor Min Avg Max AVX512 On 2250 3054 3135 AVX512 Off 2239 3038 3143 OpenBenchmarking.org Megahertz, More Is Better libxsmm 2-1.17-3645 CPU Peak Freq (Highest CPU Core Frequency) Monitor 800 1600 2400 3200 4000
libxsmm CPU Power Consumption Monitor Min Avg Max AVX512 On 20.5 210.2 340.2 AVX512 Off 20.7 211.9 335.7 OpenBenchmarking.org Watts, Fewer Is Better libxsmm 2-1.17-3645 CPU Power Consumption Monitor 80 160 240 320 400
libxsmm CPU Temperature Monitor Min Avg Max AVX512 Off 33.5 49.4 53.3 AVX512 On 34.4 49.7 54.3 OpenBenchmarking.org Celsius, Fewer Is Better libxsmm 2-1.17-3645 CPU Temperature Monitor 15 30 45 60 75
miniBUDE Implementation: OpenMP - Input Deck: BM1 OpenBenchmarking.org GFInst/s, More Is Better miniBUDE 20210901 Implementation: OpenMP - Input Deck: BM1 AVX512 On AVX512 Off 1300 2600 3900 5200 6500 SE +/- 2.27, N = 9 SE +/- 18.03, N = 8 5925.67 4677.68 1. (CC) gcc options: -std=c99 -Ofast -ffast-math -fopenmp -march=native -lm
miniBUDE Implementation: OpenMP - Input Deck: BM1 OpenBenchmarking.org Billion Interactions/s, More Is Better miniBUDE 20210901 Implementation: OpenMP - Input Deck: BM1 AVX512 On AVX512 Off 50 100 150 200 250 SE +/- 0.09, N = 9 SE +/- 0.72, N = 8 237.03 187.11 1. (CC) gcc options: -std=c99 -Ofast -ffast-math -fopenmp -march=native -lm
miniBUDE Implementation: OpenMP - Input Deck: BM1 OpenBenchmarking.org Billion Interactions/s Per Watt, More Is Better miniBUDE 20210901 Implementation: OpenMP - Input Deck: BM1 AVX512 On AVX512 Off 0.3488 0.6976 1.0464 1.3952 1.744 1.550 1.026
miniBUDE CPU Peak Freq (Highest CPU Core Frequency) Monitor Min Avg Max AVX512 On 2250 2597 3098 AVX512 Off 2250 2529 3096 OpenBenchmarking.org Megahertz, More Is Better miniBUDE 20210901 CPU Peak Freq (Highest CPU Core Frequency) Monitor 800 1600 2400 3200 4000
miniBUDE CPU Power Consumption Monitor Min Avg Max AVX512 On 19.2 152.9 317.5 AVX512 Off 19.5 182.4 337.5 OpenBenchmarking.org Watts, Fewer Is Better miniBUDE 20210901 CPU Power Consumption Monitor 80 160 240 320 400
miniBUDE CPU Temperature Monitor Min Avg Max AVX512 On 23.3 36.3 50.8 AVX512 Off 20.8 37.0 52.8 OpenBenchmarking.org Celsius, Fewer Is Better miniBUDE 20210901 CPU Temperature Monitor 15 30 45 60 75
miniBUDE Implementation: OpenMP - Input Deck: BM2 OpenBenchmarking.org GFInst/s, More Is Better miniBUDE 20210901 Implementation: OpenMP - Input Deck: BM2 AVX512 On AVX512 Off 1300 2600 3900 5200 6500 SE +/- 0.44, N = 3 SE +/- 15.49, N = 3 5972.19 4528.31 1. (CC) gcc options: -std=c99 -Ofast -ffast-math -fopenmp -march=native -lm
miniBUDE Implementation: OpenMP - Input Deck: BM2 OpenBenchmarking.org Billion Interactions/s, More Is Better miniBUDE 20210901 Implementation: OpenMP - Input Deck: BM2 AVX512 On AVX512 Off 50 100 150 200 250 SE +/- 0.02, N = 3 SE +/- 0.62, N = 3 238.89 181.13 1. (CC) gcc options: -std=c99 -Ofast -ffast-math -fopenmp -march=native -lm
miniBUDE Implementation: OpenMP - Input Deck: BM2 OpenBenchmarking.org Billion Interactions/s Per Watt, More Is Better miniBUDE 20210901 Implementation: OpenMP - Input Deck: BM2 AVX512 On AVX512 Off 0.2003 0.4006 0.6009 0.8012 1.0015 0.89 0.62
miniBUDE CPU Peak Freq (Highest CPU Core Frequency) Monitor Min Avg Max AVX512 On 2250 2796 3014 AVX512 Off 2250 2696 3098 OpenBenchmarking.org Megahertz, More Is Better miniBUDE 20210901 CPU Peak Freq (Highest CPU Core Frequency) Monitor 800 1600 2400 3200 4000
miniBUDE CPU Power Consumption Monitor Min Avg Max AVX512 On 19.7 268.3 323.2 AVX512 Off 19.7 292.3 341.7 OpenBenchmarking.org Watts, Fewer Is Better miniBUDE 20210901 CPU Power Consumption Monitor 80 160 240 320 400
miniBUDE CPU Temperature Monitor Min Avg Max AVX512 On 29.3 53.4 62.3 AVX512 Off 27.9 55.1 64.1 OpenBenchmarking.org Celsius, Fewer Is Better miniBUDE 20210901 CPU Temperature Monitor 20 40 60 80 100
Neural Magic DeepSparse Model: NLP Document Classification, oBERT base uncased on IMDB - Scenario: Asynchronous Multi-Stream OpenBenchmarking.org items/sec, More Is Better Neural Magic DeepSparse 1.5 Model: NLP Document Classification, oBERT base uncased on IMDB - Scenario: Asynchronous Multi-Stream AVX512 On AVX512 Off 16 32 48 64 80 SE +/- 0.14, N = 3 SE +/- 0.02, N = 3 73.54 61.37
Neural Magic DeepSparse Model: NLP Document Classification, oBERT base uncased on IMDB - Scenario: Asynchronous Multi-Stream OpenBenchmarking.org ms/batch, Fewer Is Better Neural Magic DeepSparse 1.5 Model: NLP Document Classification, oBERT base uncased on IMDB - Scenario: Asynchronous Multi-Stream AVX512 On AVX512 Off 200 400 600 800 1000 SE +/- 0.33, N = 3 SE +/- 1.40, N = 3 858.40 1023.17
Neural Magic DeepSparse CPU Peak Freq (Highest CPU Core Frequency) Monitor Min Avg Max AVX512 On 2250 3011 3136 AVX512 Off 2250 2933 3136 OpenBenchmarking.org Megahertz, More Is Better Neural Magic DeepSparse 1.5 CPU Peak Freq (Highest CPU Core Frequency) Monitor 800 1600 2400 3200 4000
Neural Magic DeepSparse CPU Power Consumption Monitor Min Avg Max AVX512 Off 20.7 244.1 344.4 AVX512 On 20.4 253.4 358.2 OpenBenchmarking.org Watts, Fewer Is Better Neural Magic DeepSparse 1.5 CPU Power Consumption Monitor 100 200 300 400 500
Neural Magic DeepSparse CPU Temperature Monitor Min Avg Max AVX512 Off 30.6 54.5 70.5 AVX512 On 34.6 57.5 71.4 OpenBenchmarking.org Celsius, Fewer Is Better Neural Magic DeepSparse 1.5 CPU Temperature Monitor 20 40 60 80 100
Neural Magic DeepSparse Model: NLP Sentiment Analysis, 80% Pruned Quantized BERT Base Uncased - Scenario: Asynchronous Multi-Stream OpenBenchmarking.org items/sec, More Is Better Neural Magic DeepSparse 1.5 Model: NLP Sentiment Analysis, 80% Pruned Quantized BERT Base Uncased - Scenario: Asynchronous Multi-Stream AVX512 On AVX512 Off 300 600 900 1200 1500 SE +/- 1.58, N = 3 SE +/- 5.33, N = 3 1381.57 718.30
Neural Magic DeepSparse Model: NLP Sentiment Analysis, 80% Pruned Quantized BERT Base Uncased - Scenario: Asynchronous Multi-Stream OpenBenchmarking.org ms/batch, Fewer Is Better Neural Magic DeepSparse 1.5 Model: NLP Sentiment Analysis, 80% Pruned Quantized BERT Base Uncased - Scenario: Asynchronous Multi-Stream AVX512 On AVX512 Off 20 40 60 80 100 SE +/- 0.05, N = 3 SE +/- 0.67, N = 3 46.26 88.91
Neural Magic DeepSparse CPU Peak Freq (Highest CPU Core Frequency) Monitor Min Avg Max AVX512 On 2250 3026 3124 AVX512 Off 2250 3015 3127 OpenBenchmarking.org Megahertz, More Is Better Neural Magic DeepSparse 1.5 CPU Peak Freq (Highest CPU Core Frequency) Monitor 800 1600 2400 3200 4000
Neural Magic DeepSparse CPU Power Consumption Monitor Min Avg Max AVX512 Off 21.3 234.7 326.7 AVX512 On 21.1 250.2 321.3 OpenBenchmarking.org Watts, Fewer Is Better Neural Magic DeepSparse 1.5 CPU Power Consumption Monitor 80 160 240 320 400
Neural Magic DeepSparse CPU Temperature Monitor Min Avg Max AVX512 Off 38.1 53.8 64.3 AVX512 On 39.4 58.1 67.6 OpenBenchmarking.org Celsius, Fewer Is Better Neural Magic DeepSparse 1.5 CPU Temperature Monitor 20 40 60 80 100
Neural Magic DeepSparse Model: NLP Question Answering, BERT base uncased SQuaD 12layer Pruned90 - Scenario: Asynchronous Multi-Stream OpenBenchmarking.org items/sec, More Is Better Neural Magic DeepSparse 1.5 Model: NLP Question Answering, BERT base uncased SQuaD 12layer Pruned90 - Scenario: Asynchronous Multi-Stream AVX512 On AVX512 Off 50 100 150 200 250 SE +/- 7.49, N = 15 SE +/- 0.14, N = 3 247.97 213.64
Neural Magic DeepSparse Model: NLP Question Answering, BERT base uncased SQuaD 12layer Pruned90 - Scenario: Asynchronous Multi-Stream OpenBenchmarking.org ms/batch, Fewer Is Better Neural Magic DeepSparse 1.5 Model: NLP Question Answering, BERT base uncased SQuaD 12layer Pruned90 - Scenario: Asynchronous Multi-Stream AVX512 On AVX512 Off 60 120 180 240 300 SE +/- 6.02, N = 15 SE +/- 0.24, N = 3 259.65 298.06
Neural Magic DeepSparse CPU Peak Freq (Highest CPU Core Frequency) Monitor Min Avg Max AVX512 Off 2250 3023 3152 AVX512 On 2250 3002 3416 OpenBenchmarking.org Megahertz, More Is Better Neural Magic DeepSparse 1.5 CPU Peak Freq (Highest CPU Core Frequency) Monitor 800 1600 2400 3200 4000
Neural Magic DeepSparse CPU Power Consumption Monitor Min Avg Max AVX512 Off 21.0 228.6 329.0 AVX512 On 10.5 234.1 342.4 OpenBenchmarking.org Watts, Fewer Is Better Neural Magic DeepSparse 1.5 CPU Power Consumption Monitor 80 160 240 320 400
Neural Magic DeepSparse CPU Temperature Monitor Min Avg Max AVX512 Off 38.1 52.9 65.8 AVX512 On 38.6 53.1 66.5 OpenBenchmarking.org Celsius, Fewer Is Better Neural Magic DeepSparse 1.5 CPU Temperature Monitor 20 40 60 80 100
Neural Magic DeepSparse Model: CV Classification, ResNet-50 ImageNet - Scenario: Asynchronous Multi-Stream OpenBenchmarking.org items/sec, More Is Better Neural Magic DeepSparse 1.5 Model: CV Classification, ResNet-50 ImageNet - Scenario: Asynchronous Multi-Stream AVX512 On AVX512 Off 200 400 600 800 1000 SE +/- 0.42, N = 3 SE +/- 0.45, N = 3 970.06 870.95
Neural Magic DeepSparse Model: CV Classification, ResNet-50 ImageNet - Scenario: Asynchronous Multi-Stream OpenBenchmarking.org ms/batch, Fewer Is Better Neural Magic DeepSparse 1.5 Model: CV Classification, ResNet-50 ImageNet - Scenario: Asynchronous Multi-Stream AVX512 On AVX512 Off 16 32 48 64 80 SE +/- 0.03, N = 3 SE +/- 0.04, N = 3 65.89 73.35
Neural Magic DeepSparse CPU Peak Freq (Highest CPU Core Frequency) Monitor Min Avg Max AVX512 On 2250 3012 3134 AVX512 Off 2250 2962 3101 OpenBenchmarking.org Megahertz, More Is Better Neural Magic DeepSparse 1.5 CPU Peak Freq (Highest CPU Core Frequency) Monitor 800 1600 2400 3200 4000
Neural Magic DeepSparse CPU Power Consumption Monitor Min Avg Max AVX512 On 21.1 279.2 332.9 AVX512 Off 21.0 284.5 342.1 OpenBenchmarking.org Watts, Fewer Is Better Neural Magic DeepSparse 1.5 CPU Power Consumption Monitor 80 160 240 320 400
Neural Magic DeepSparse CPU Temperature Monitor Min Avg Max AVX512 On 39.8 62.8 70.4 AVX512 Off 40.0 65.2 72.9 OpenBenchmarking.org Celsius, Fewer Is Better Neural Magic DeepSparse 1.5 CPU Temperature Monitor 20 40 60 80 100
Neural Magic DeepSparse Model: NLP Text Classification, DistilBERT mnli - Scenario: Asynchronous Multi-Stream OpenBenchmarking.org items/sec, More Is Better Neural Magic DeepSparse 1.5 Model: NLP Text Classification, DistilBERT mnli - Scenario: Asynchronous Multi-Stream AVX512 On AVX512 Off 130 260 390 520 650 SE +/- 0.52, N = 3 SE +/- 0.58, N = 3 624.74 522.11
Neural Magic DeepSparse Model: NLP Text Classification, DistilBERT mnli - Scenario: Asynchronous Multi-Stream OpenBenchmarking.org ms/batch, Fewer Is Better Neural Magic DeepSparse 1.5 Model: NLP Text Classification, DistilBERT mnli - Scenario: Asynchronous Multi-Stream AVX512 On AVX512 Off 30 60 90 120 150 SE +/- 0.07, N = 3 SE +/- 0.15, N = 3 102.22 122.23
Neural Magic DeepSparse CPU Peak Freq (Highest CPU Core Frequency) Monitor Min Avg Max AVX512 On 2250 3034 3099 AVX512 Off 2250 2931 3101 OpenBenchmarking.org Megahertz, More Is Better Neural Magic DeepSparse 1.5 CPU Peak Freq (Highest CPU Core Frequency) Monitor 800 1600 2400 3200 4000
Neural Magic DeepSparse CPU Power Consumption Monitor Min Avg Max AVX512 On 20.8 255.7 320.6 AVX512 Off 21.0 256.2 320.4 OpenBenchmarking.org Watts, Fewer Is Better Neural Magic DeepSparse 1.5 CPU Power Consumption Monitor 80 160 240 320 400
Neural Magic DeepSparse CPU Temperature Monitor Min Avg Max AVX512 On 39.1 60.4 67.1 AVX512 Off 41.0 60.7 68.5 OpenBenchmarking.org Celsius, Fewer Is Better Neural Magic DeepSparse 1.5 CPU Temperature Monitor 20 40 60 80 100
Neural Magic DeepSparse Model: CV Segmentation, 90% Pruned YOLACT Pruned - Scenario: Asynchronous Multi-Stream OpenBenchmarking.org items/sec, More Is Better Neural Magic DeepSparse 1.5 Model: CV Segmentation, 90% Pruned YOLACT Pruned - Scenario: Asynchronous Multi-Stream AVX512 On AVX512 Off 30 60 90 120 150 SE +/- 0.01, N = 3 SE +/- 0.03, N = 3 127.06 69.33
Neural Magic DeepSparse Model: CV Segmentation, 90% Pruned YOLACT Pruned - Scenario: Asynchronous Multi-Stream OpenBenchmarking.org ms/batch, Fewer Is Better Neural Magic DeepSparse 1.5 Model: CV Segmentation, 90% Pruned YOLACT Pruned - Scenario: Asynchronous Multi-Stream AVX512 On AVX512 Off 200 400 600 800 1000 SE +/- 0.16, N = 3 SE +/- 0.94, N = 3 498.48 906.94
Neural Magic DeepSparse CPU Peak Freq (Highest CPU Core Frequency) Monitor Min Avg Max AVX512 Off 2250 3048 3116 AVX512 On 2250 3006 3115 OpenBenchmarking.org Megahertz, More Is Better Neural Magic DeepSparse 1.5 CPU Peak Freq (Highest CPU Core Frequency) Monitor 800 1600 2400 3200 4000
Neural Magic DeepSparse CPU Power Consumption Monitor Min Avg Max AVX512 On 20.6 228.3 324.5 AVX512 Off 21.5 235.2 340.9 OpenBenchmarking.org Watts, Fewer Is Better Neural Magic DeepSparse 1.5 CPU Power Consumption Monitor 80 160 240 320 400
Neural Magic DeepSparse CPU Temperature Monitor Min Avg Max AVX512 On 38.1 53.1 66.1 AVX512 Off 40.0 54.0 69.1 OpenBenchmarking.org Celsius, Fewer Is Better Neural Magic DeepSparse 1.5 CPU Temperature Monitor 20 40 60 80 100
Neural Magic DeepSparse Model: NLP Text Classification, BERT base uncased SST2 - Scenario: Asynchronous Multi-Stream OpenBenchmarking.org items/sec, More Is Better Neural Magic DeepSparse 1.5 Model: NLP Text Classification, BERT base uncased SST2 - Scenario: Asynchronous Multi-Stream AVX512 On AVX512 Off 70 140 210 280 350 SE +/- 0.78, N = 3 SE +/- 0.63, N = 3 316.17 260.77
Neural Magic DeepSparse Model: NLP Text Classification, BERT base uncased SST2 - Scenario: Asynchronous Multi-Stream OpenBenchmarking.org ms/batch, Fewer Is Better Neural Magic DeepSparse 1.5 Model: NLP Text Classification, BERT base uncased SST2 - Scenario: Asynchronous Multi-Stream AVX512 On AVX512 Off 50 100 150 200 250 SE +/- 0.47, N = 3 SE +/- 0.64, N = 3 201.60 244.15
Neural Magic DeepSparse CPU Peak Freq (Highest CPU Core Frequency) Monitor Min Avg Max AVX512 On 2250 3013 3127 AVX512 Off 2250 2940 3122 OpenBenchmarking.org Megahertz, More Is Better Neural Magic DeepSparse 1.5 CPU Peak Freq (Highest CPU Core Frequency) Monitor 800 1600 2400 3200 4000
Neural Magic DeepSparse CPU Power Consumption Monitor Min Avg Max AVX512 Off 20.8 247.7 347.3 AVX512 On 20.6 247.7 358.6 OpenBenchmarking.org Watts, Fewer Is Better Neural Magic DeepSparse 1.5 CPU Power Consumption Monitor 100 200 300 400 500
Neural Magic DeepSparse CPU Temperature Monitor Min Avg Max AVX512 On 36.0 57.7 70.9 AVX512 Off 38.4 58.3 70.5 OpenBenchmarking.org Celsius, Fewer Is Better Neural Magic DeepSparse 1.5 CPU Temperature Monitor 20 40 60 80 100
Neural Magic DeepSparse Model: NLP Token Classification, BERT base uncased conll2003 - Scenario: Asynchronous Multi-Stream OpenBenchmarking.org items/sec, More Is Better Neural Magic DeepSparse 1.5 Model: NLP Token Classification, BERT base uncased conll2003 - Scenario: Asynchronous Multi-Stream AVX512 On AVX512 Off 16 32 48 64 80 SE +/- 0.19, N = 3 SE +/- 0.13, N = 3 73.15 61.18
Neural Magic DeepSparse Model: NLP Token Classification, BERT base uncased conll2003 - Scenario: Asynchronous Multi-Stream OpenBenchmarking.org ms/batch, Fewer Is Better Neural Magic DeepSparse 1.5 Model: NLP Token Classification, BERT base uncased conll2003 - Scenario: Asynchronous Multi-Stream AVX512 On AVX512 Off 200 400 600 800 1000 SE +/- 0.45, N = 3 SE +/- 1.31, N = 3 859.71 1025.48
Neural Magic DeepSparse CPU Peak Freq (Highest CPU Core Frequency) Monitor Min Avg Max AVX512 On 2250 3006 3135 AVX512 Off 2250 2952 3146 OpenBenchmarking.org Megahertz, More Is Better Neural Magic DeepSparse 1.5 CPU Peak Freq (Highest CPU Core Frequency) Monitor 800 1600 2400 3200 4000
Neural Magic DeepSparse CPU Power Consumption Monitor Min Avg Max AVX512 On 20.7 244.3 357.9 AVX512 Off 21.2 245.5 344.8 OpenBenchmarking.org Watts, Fewer Is Better Neural Magic DeepSparse 1.5 CPU Power Consumption Monitor 100 200 300 400 500
Neural Magic DeepSparse CPU Temperature Monitor Min Avg Max AVX512 On 37.3 57.6 70.9 AVX512 Off 38.9 58.3 70.9 OpenBenchmarking.org Celsius, Fewer Is Better Neural Magic DeepSparse 1.5 CPU Temperature Monitor 20 40 60 80 100
oneDNN Harness: Recurrent Neural Network Training - Data Type: bf16bf16bf16 - Engine: CPU OpenBenchmarking.org ms, Fewer Is Better oneDNN 3.1 Harness: Recurrent Neural Network Training - Data Type: bf16bf16bf16 - Engine: CPU AVX512 On AVX512 Off 300 600 900 1200 1500 SE +/- 12.60, N = 4 SE +/- 1.80, N = 3 1174.75 1317.57 MIN: 1143.88 MIN: 1299.38 1. (CXX) g++ options: -O3 -march=native -fopenmp -msse4.1 -fPIC -pie -ldl -lpthread
oneDNN CPU Peak Freq (Highest CPU Core Frequency) Monitor Min Avg Max AVX512 On 2250 2998 3097 AVX512 Off 2250 2824 3100 OpenBenchmarking.org Megahertz, More Is Better oneDNN 3.1 CPU Peak Freq (Highest CPU Core Frequency) Monitor 800 1600 2400 3200 4000
oneDNN CPU Power Consumption Monitor Min Avg Max AVX512 On 20.0 176.7 277.9 AVX512 Off 19.8 202.0 272.5 OpenBenchmarking.org Watts, Fewer Is Better oneDNN 3.1 CPU Power Consumption Monitor 70 140 210 280 350
oneDNN CPU Temperature Monitor Min Avg Max AVX512 On 30.4 39.2 45.3 AVX512 Off 24.5 39.8 45.1 OpenBenchmarking.org Celsius, Fewer Is Better oneDNN 3.1 CPU Temperature Monitor 12 24 36 48 60
OpenVINO Model: Face Detection FP16 - Device: CPU OpenBenchmarking.org FPS, More Is Better OpenVINO 2022.3 Model: Face Detection FP16 - Device: CPU AVX512 On AVX512 Off 14 28 42 56 70 SE +/- 0.06, N = 3 SE +/- 0.36, N = 3 60.73 26.18 1. (CXX) g++ options: -isystem -fsigned-char -ffunction-sections -fdata-sections -msse4.1 -msse4.2 -O3 -fno-strict-overflow -fwrapv -fPIC -fvisibility=hidden -Os -std=c++11 -MD -MT -MF
OpenVINO Model: Face Detection FP16 - Device: CPU OpenBenchmarking.org ms, Fewer Is Better OpenVINO 2022.3 Model: Face Detection FP16 - Device: CPU AVX512 On AVX512 Off 500 1000 1500 2000 2500 SE +/- 0.42, N = 3 SE +/- 26.84, N = 3 1048.37 2423.39 MIN: 508.1 / MAX: 1159.4 MIN: 1130.42 / MAX: 2937.22 1. (CXX) g++ options: -isystem -fsigned-char -ffunction-sections -fdata-sections -msse4.1 -msse4.2 -O3 -fno-strict-overflow -fwrapv -fPIC -fvisibility=hidden -Os -std=c++11 -MD -MT -MF
OpenVINO CPU Peak Freq (Highest CPU Core Frequency) Monitor Min Avg Max AVX512 On 2250 2708 3101 AVX512 Off 2250 2491 3096 OpenBenchmarking.org Megahertz, More Is Better OpenVINO 2022.3 CPU Peak Freq (Highest CPU Core Frequency) Monitor 800 1600 2400 3200 4000
OpenVINO CPU Power Consumption Monitor Min Avg Max AVX512 Off 21.1 286.8 325.8 AVX512 On 21.1 315.2 358.6 OpenBenchmarking.org Watts, Fewer Is Better OpenVINO 2022.3 CPU Power Consumption Monitor 100 200 300 400 500
OpenVINO CPU Temperature Monitor Min Avg Max AVX512 Off 39.8 59.3 68.1 AVX512 On 38.6 63.1 70.0 OpenBenchmarking.org Celsius, Fewer Is Better OpenVINO 2022.3 CPU Temperature Monitor 20 40 60 80 100
OpenVINO Model: Person Detection FP16 - Device: CPU OpenBenchmarking.org FPS, More Is Better OpenVINO 2022.3 Model: Person Detection FP16 - Device: CPU AVX512 On AVX512 Off 6 12 18 24 30 SE +/- 0.30, N = 12 SE +/- 0.21, N = 3 27.08 15.06 1. (CXX) g++ options: -isystem -fsigned-char -ffunction-sections -fdata-sections -msse4.1 -msse4.2 -O3 -fno-strict-overflow -fwrapv -fPIC -fvisibility=hidden -Os -std=c++11 -MD -MT -MF
OpenVINO Model: Person Detection FP16 - Device: CPU OpenBenchmarking.org ms, Fewer Is Better OpenVINO 2022.3 Model: Person Detection FP16 - Device: CPU AVX512 On AVX512 Off 900 1800 2700 3600 4500 SE +/- 22.79, N = 12 SE +/- 59.43, N = 3 2334.29 4153.59 MIN: 1017.83 / MAX: 3101.05 MIN: 1975.28 / MAX: 5152.66 1. (CXX) g++ options: -isystem -fsigned-char -ffunction-sections -fdata-sections -msse4.1 -msse4.2 -O3 -fno-strict-overflow -fwrapv -fPIC -fvisibility=hidden -Os -std=c++11 -MD -MT -MF
OpenVINO CPU Peak Freq (Highest CPU Core Frequency) Monitor Min Avg Max AVX512 On 2250 2931 3185 AVX512 Off 2250 2649 3132 OpenBenchmarking.org Megahertz, More Is Better OpenVINO 2022.3 CPU Peak Freq (Highest CPU Core Frequency) Monitor 800 1600 2400 3200 4000
OpenVINO CPU Power Consumption Monitor Min Avg Max AVX512 Off 21.3 283.3 329.0 AVX512 On 19.8 306.5 359.5 OpenBenchmarking.org Watts, Fewer Is Better OpenVINO 2022.3 CPU Power Consumption Monitor 100 200 300 400 500
OpenVINO CPU Temperature Monitor Min Avg Max AVX512 Off 37.6 57.4 67.6 AVX512 On 41.3 61.2 73.4 OpenBenchmarking.org Celsius, Fewer Is Better OpenVINO 2022.3 CPU Temperature Monitor 20 40 60 80 100
OpenVINO Model: Person Detection FP32 - Device: CPU OpenBenchmarking.org FPS, More Is Better OpenVINO 2022.3 Model: Person Detection FP32 - Device: CPU AVX512 On AVX512 Off 6 12 18 24 30 SE +/- 0.18, N = 12 SE +/- 0.16, N = 5 27.01 14.99 1. (CXX) g++ options: -isystem -fsigned-char -ffunction-sections -fdata-sections -msse4.1 -msse4.2 -O3 -fno-strict-overflow -fwrapv -fPIC -fvisibility=hidden -Os -std=c++11 -MD -MT -MF
OpenVINO Model: Person Detection FP32 - Device: CPU OpenBenchmarking.org ms, Fewer Is Better OpenVINO 2022.3 Model: Person Detection FP32 - Device: CPU AVX512 On AVX512 Off 900 1800 2700 3600 4500 SE +/- 14.83, N = 12 SE +/- 41.79, N = 5 2339.81 4170.49 MIN: 1045.35 / MAX: 3232.69 MIN: 1759.65 / MAX: 4969.5 1. (CXX) g++ options: -isystem -fsigned-char -ffunction-sections -fdata-sections -msse4.1 -msse4.2 -O3 -fno-strict-overflow -fwrapv -fPIC -fvisibility=hidden -Os -std=c++11 -MD -MT -MF
OpenVINO CPU Peak Freq (Highest CPU Core Frequency) Monitor Min Avg Max AVX512 On 2250 2936 3426 AVX512 Off 2250 2645 3103 OpenBenchmarking.org Megahertz, More Is Better OpenVINO 2022.3 CPU Peak Freq (Highest CPU Core Frequency) Monitor 800 1600 2400 3200 4000
OpenVINO CPU Power Consumption Monitor Min Avg Max AVX512 Off 21.0 292.7 340.3 AVX512 On 20.8 306.2 358.4 OpenBenchmarking.org Watts, Fewer Is Better OpenVINO 2022.3 CPU Power Consumption Monitor 100 200 300 400 500
OpenVINO CPU Temperature Monitor Min Avg Max AVX512 Off 36.5 58.0 72.4 AVX512 On 37.6 60.6 72.9 OpenBenchmarking.org Celsius, Fewer Is Better OpenVINO 2022.3 CPU Temperature Monitor 20 40 60 80 100
OpenVINO Model: Vehicle Detection FP16 - Device: CPU OpenBenchmarking.org FPS, More Is Better OpenVINO 2022.3 Model: Vehicle Detection FP16 - Device: CPU AVX512 On AVX512 Off 300 600 900 1200 1500 SE +/- 22.93, N = 15 SE +/- 15.59, N = 14 1430.45 1190.91 1. (CXX) g++ options: -isystem -fsigned-char -ffunction-sections -fdata-sections -msse4.1 -msse4.2 -O3 -fno-strict-overflow -fwrapv -fPIC -fvisibility=hidden -Os -std=c++11 -MD -MT -MF
OpenVINO Model: Vehicle Detection FP16 - Device: CPU OpenBenchmarking.org ms, Fewer Is Better OpenVINO 2022.3 Model: Vehicle Detection FP16 - Device: CPU AVX512 On AVX512 Off 12 24 36 48 60 SE +/- 0.60, N = 15 SE +/- 0.62, N = 14 44.84 53.79 MIN: 8.1 / MAX: 137.01 MIN: 13.85 / MAX: 136.83 1. (CXX) g++ options: -isystem -fsigned-char -ffunction-sections -fdata-sections -msse4.1 -msse4.2 -O3 -fno-strict-overflow -fwrapv -fPIC -fvisibility=hidden -Os -std=c++11 -MD -MT -MF
OpenVINO CPU Peak Freq (Highest CPU Core Frequency) Monitor Min Avg Max AVX512 On 2250 3029 3257 AVX512 Off 2241 2918 3105 OpenBenchmarking.org Megahertz, More Is Better OpenVINO 2022.3 CPU Peak Freq (Highest CPU Core Frequency) Monitor 800 1600 2400 3200 4000
OpenVINO CPU Power Consumption Monitor Min Avg Max AVX512 On 20.7 256.0 333.9 AVX512 Off 21.0 266.8 326.8 OpenBenchmarking.org Watts, Fewer Is Better OpenVINO 2022.3 CPU Power Consumption Monitor 80 160 240 320 400
OpenVINO CPU Temperature Monitor Min Avg Max AVX512 On 36.8 51.7 57.5 AVX512 Off 36.8 53.9 62.3 OpenBenchmarking.org Celsius, Fewer Is Better OpenVINO 2022.3 CPU Temperature Monitor 20 40 60 80 100
OpenVINO Model: Face Detection FP16-INT8 - Device: CPU OpenBenchmarking.org FPS, More Is Better OpenVINO 2022.3 Model: Face Detection FP16-INT8 - Device: CPU AVX512 On AVX512 Off 30 60 90 120 150 SE +/- 0.02, N = 3 SE +/- 0.03, N = 3 118.00 57.95 1. (CXX) g++ options: -isystem -fsigned-char -ffunction-sections -fdata-sections -msse4.1 -msse4.2 -O3 -fno-strict-overflow -fwrapv -fPIC -fvisibility=hidden -Os -std=c++11 -MD -MT -MF
OpenVINO Model: Face Detection FP16-INT8 - Device: CPU OpenBenchmarking.org ms, Fewer Is Better OpenVINO 2022.3 Model: Face Detection FP16-INT8 - Device: CPU AVX512 On AVX512 Off 200 400 600 800 1000 SE +/- 0.07, N = 3 SE +/- 0.34, N = 3 540.35 1094.52 MIN: 257.81 / MAX: 586.56 MIN: 509.48 / MAX: 1179.63 1. (CXX) g++ options: -isystem -fsigned-char -ffunction-sections -fdata-sections -msse4.1 -msse4.2 -O3 -fno-strict-overflow -fwrapv -fPIC -fvisibility=hidden -Os -std=c++11 -MD -MT -MF
OpenVINO CPU Peak Freq (Highest CPU Core Frequency) Monitor Min Avg Max AVX512 On 2250 2696 3116 AVX512 Off 2250 2688 3124 OpenBenchmarking.org Megahertz, More Is Better OpenVINO 2022.3 CPU Peak Freq (Highest CPU Core Frequency) Monitor 800 1600 2400 3200 4000
OpenVINO CPU Power Consumption Monitor Min Avg Max AVX512 On 20.5 295.4 336.3 AVX512 Off 21.2 323.5 369.0 OpenBenchmarking.org Watts, Fewer Is Better OpenVINO 2022.3 CPU Power Consumption Monitor 100 200 300 400 500
OpenVINO CPU Temperature Monitor Min Avg Max AVX512 On 34.4 59.5 70.5 AVX512 Off 35.8 65.5 76.1 OpenBenchmarking.org Celsius, Fewer Is Better OpenVINO 2022.3 CPU Temperature Monitor 20 40 60 80 100
OpenVINO Model: Vehicle Detection FP16-INT8 - Device: CPU OpenBenchmarking.org FPS, More Is Better OpenVINO 2022.3 Model: Vehicle Detection FP16-INT8 - Device: CPU AVX512 On AVX512 Off 1200 2400 3600 4800 6000 SE +/- 89.26, N = 15 SE +/- 2.12, N = 3 5690.34 3954.90 1. (CXX) g++ options: -isystem -fsigned-char -ffunction-sections -fdata-sections -msse4.1 -msse4.2 -O3 -fno-strict-overflow -fwrapv -fPIC -fvisibility=hidden -Os -std=c++11 -MD -MT -MF
OpenVINO Model: Vehicle Detection FP16-INT8 - Device: CPU OpenBenchmarking.org ms, Fewer Is Better OpenVINO 2022.3 Model: Vehicle Detection FP16-INT8 - Device: CPU AVX512 On AVX512 Off 4 8 12 16 20 SE +/- 0.15, N = 15 SE +/- 0.01, N = 3 11.26 16.17 MIN: 4.37 / MAX: 45.06 MIN: 8.44 / MAX: 56.88 1. (CXX) g++ options: -isystem -fsigned-char -ffunction-sections -fdata-sections -msse4.1 -msse4.2 -O3 -fno-strict-overflow -fwrapv -fPIC -fvisibility=hidden -Os -std=c++11 -MD -MT -MF
OpenVINO CPU Peak Freq (Highest CPU Core Frequency) Monitor Min Avg Max AVX512 On 2250 2705 3121 AVX512 Off 2250 2659 3098 OpenBenchmarking.org Megahertz, More Is Better OpenVINO 2022.3 CPU Peak Freq (Highest CPU Core Frequency) Monitor 800 1600 2400 3200 4000
OpenVINO CPU Power Consumption Monitor Min Avg Max AVX512 On 20.8 306.3 336.2 AVX512 Off 21.2 330.4 358.0 OpenBenchmarking.org Watts, Fewer Is Better OpenVINO 2022.3 CPU Power Consumption Monitor 100 200 300 400 500
OpenVINO CPU Temperature Monitor Min Avg Max AVX512 On 40.0 59.2 66.6 AVX512 Off 42.1 67.7 72.8 OpenBenchmarking.org Celsius, Fewer Is Better OpenVINO 2022.3 CPU Temperature Monitor 20 40 60 80 100
OpenVINO Model: Weld Porosity Detection FP16 - Device: CPU OpenBenchmarking.org FPS, More Is Better OpenVINO 2022.3 Model: Weld Porosity Detection FP16 - Device: CPU AVX512 On AVX512 Off 1300 2600 3900 5200 6500 SE +/- 1.32, N = 3 SE +/- 0.56, N = 3 6073.22 2551.70 1. (CXX) g++ options: -isystem -fsigned-char -ffunction-sections -fdata-sections -msse4.1 -msse4.2 -O3 -fno-strict-overflow -fwrapv -fPIC -fvisibility=hidden -Os -std=c++11 -MD -MT -MF
OpenVINO Model: Weld Porosity Detection FP16 - Device: CPU OpenBenchmarking.org ms, Fewer Is Better OpenVINO 2022.3 Model: Weld Porosity Detection FP16 - Device: CPU AVX512 On AVX512 Off 6 12 18 24 30 SE +/- 0.00, N = 3 SE +/- 0.01, N = 3 10.52 25.06 MIN: 5.11 / MAX: 34.37 MIN: 13.08 / MAX: 57.02 1. (CXX) g++ options: -isystem -fsigned-char -ffunction-sections -fdata-sections -msse4.1 -msse4.2 -O3 -fno-strict-overflow -fwrapv -fPIC -fvisibility=hidden -Os -std=c++11 -MD -MT -MF
OpenVINO CPU Peak Freq (Highest CPU Core Frequency) Monitor Min Avg Max AVX512 On 2250 2718 3095 AVX512 Off 2250 2373 3112 OpenBenchmarking.org Megahertz, More Is Better OpenVINO 2022.3 CPU Peak Freq (Highest CPU Core Frequency) Monitor 800 1600 2400 3200 4000
OpenVINO CPU Power Consumption Monitor Min Avg Max AVX512 Off 21.3 288.2 311.1 AVX512 On 21.2 327.7 354.4 OpenBenchmarking.org Watts, Fewer Is Better OpenVINO 2022.3 CPU Power Consumption Monitor 100 200 300 400 500
OpenVINO CPU Temperature Monitor Min Avg Max AVX512 Off 41.9 61.1 64.3 AVX512 On 36.8 64.2 69.9 OpenBenchmarking.org Celsius, Fewer Is Better OpenVINO 2022.3 CPU Temperature Monitor 20 40 60 80 100
OpenVINO Model: Machine Translation EN To DE FP16 - Device: CPU OpenBenchmarking.org FPS, More Is Better OpenVINO 2022.3 Model: Machine Translation EN To DE FP16 - Device: CPU AVX512 On AVX512 Off 130 260 390 520 650 SE +/- 7.10, N = 15 SE +/- 3.15, N = 15 580.40 251.97 1. (CXX) g++ options: -isystem -fsigned-char -ffunction-sections -fdata-sections -msse4.1 -msse4.2 -O3 -fno-strict-overflow -fwrapv -fPIC -fvisibility=hidden -Os -std=c++11 -MD -MT -MF
OpenVINO Model: Machine Translation EN To DE FP16 - Device: CPU OpenBenchmarking.org ms, Fewer Is Better OpenVINO 2022.3 Model: Machine Translation EN To DE FP16 - Device: CPU AVX512 On AVX512 Off 60 120 180 240 300 SE +/- 1.22, N = 15 SE +/- 2.83, N = 15 110.35 254.01 MIN: 49.7 / MAX: 183.59 MIN: 116.71 / MAX: 398.88 1. (CXX) g++ options: -isystem -fsigned-char -ffunction-sections -fdata-sections -msse4.1 -msse4.2 -O3 -fno-strict-overflow -fwrapv -fPIC -fvisibility=hidden -Os -std=c++11 -MD -MT -MF
OpenVINO CPU Peak Freq (Highest CPU Core Frequency) Monitor Min Avg Max AVX512 On 2250 2661 3111 AVX512 Off 2214 2537 3101 OpenBenchmarking.org Megahertz, More Is Better OpenVINO 2022.3 CPU Peak Freq (Highest CPU Core Frequency) Monitor 800 1600 2400 3200 4000
OpenVINO CPU Power Consumption Monitor Min Avg Max AVX512 Off 21.2 292.9 331.3 AVX512 On 10.9 295.1 345.9 OpenBenchmarking.org Watts, Fewer Is Better OpenVINO 2022.3 CPU Power Consumption Monitor 100 200 300 400 500
OpenVINO CPU Temperature Monitor Min Avg Max AVX512 Off 41.0 56.7 64.6 AVX512 On 40.5 57.3 65.8 OpenBenchmarking.org Celsius, Fewer Is Better OpenVINO 2022.3 CPU Temperature Monitor 20 40 60 80 100
OpenVINO Model: Weld Porosity Detection FP16-INT8 - Device: CPU OpenBenchmarking.org FPS, More Is Better OpenVINO 2022.3 Model: Weld Porosity Detection FP16-INT8 - Device: CPU AVX512 On AVX512 Off 3K 6K 9K 12K 15K SE +/- 1.17, N = 3 SE +/- 1.65, N = 3 11818.33 5692.99 1. (CXX) g++ options: -isystem -fsigned-char -ffunction-sections -fdata-sections -msse4.1 -msse4.2 -O3 -fno-strict-overflow -fwrapv -fPIC -fvisibility=hidden -Os -std=c++11 -MD -MT -MF
OpenVINO Model: Weld Porosity Detection FP16-INT8 - Device: CPU OpenBenchmarking.org ms, Fewer Is Better OpenVINO 2022.3 Model: Weld Porosity Detection FP16-INT8 - Device: CPU AVX512 On AVX512 Off 5 10 15 20 25 SE +/- 0.00, N = 3 SE +/- 0.01, N = 3 10.82 22.47 MIN: 5 / MAX: 31.44 MIN: 10.87 / MAX: 43.5 1. (CXX) g++ options: -isystem -fsigned-char -ffunction-sections -fdata-sections -msse4.1 -msse4.2 -O3 -fno-strict-overflow -fwrapv -fPIC -fvisibility=hidden -Os -std=c++11 -MD -MT -MF
OpenVINO CPU Peak Freq (Highest CPU Core Frequency) Monitor Min Avg Max AVX512 Off 2250 2671 3102 AVX512 On 2250 2619 3137 OpenBenchmarking.org Megahertz, More Is Better OpenVINO 2022.3 CPU Peak Freq (Highest CPU Core Frequency) Monitor 800 1600 2400 3200 4000
OpenVINO CPU Power Consumption Monitor Min Avg Max AVX512 On 21.1 306.8 333.5 AVX512 Off 21.1 336.6 365.0 OpenBenchmarking.org Watts, Fewer Is Better OpenVINO 2022.3 CPU Power Consumption Monitor 100 200 300 400 500
OpenVINO CPU Temperature Monitor Min Avg Max AVX512 On 36.8 61.3 66.1 AVX512 Off 36.8 67.4 73.3 OpenBenchmarking.org Celsius, Fewer Is Better OpenVINO 2022.3 CPU Temperature Monitor 20 40 60 80 100
OpenVINO Model: Person Vehicle Bike Detection FP16 - Device: CPU OpenBenchmarking.org FPS, More Is Better OpenVINO 2022.3 Model: Person Vehicle Bike Detection FP16 - Device: CPU AVX512 On AVX512 Off 1400 2800 4200 5600 7000 SE +/- 13.51, N = 3 SE +/- 26.07, N = 15 6638.71 3317.34 1. (CXX) g++ options: -isystem -fsigned-char -ffunction-sections -fdata-sections -msse4.1 -msse4.2 -O3 -fno-strict-overflow -fwrapv -fPIC -fvisibility=hidden -Os -std=c++11 -MD -MT -MF
OpenVINO Model: Person Vehicle Bike Detection FP16 - Device: CPU OpenBenchmarking.org ms, Fewer Is Better OpenVINO 2022.3 Model: Person Vehicle Bike Detection FP16 - Device: CPU AVX512 On AVX512 Off 5 10 15 20 25 SE +/- 0.02, N = 3 SE +/- 0.14, N = 15 9.63 19.28 MIN: 6.4 / MAX: 33.26 MIN: 10.31 / MAX: 50.61 1. (CXX) g++ options: -isystem -fsigned-char -ffunction-sections -fdata-sections -msse4.1 -msse4.2 -O3 -fno-strict-overflow -fwrapv -fPIC -fvisibility=hidden -Os -std=c++11 -MD -MT -MF
OpenVINO CPU Peak Freq (Highest CPU Core Frequency) Monitor Min Avg Max AVX512 On 2250 2679 3096 AVX512 Off 2250 2483 3131 OpenBenchmarking.org Megahertz, More Is Better OpenVINO 2022.3 CPU Peak Freq (Highest CPU Core Frequency) Monitor 800 1600 2400 3200 4000
OpenVINO CPU Power Consumption Monitor Min Avg Max AVX512 Off 21.1 293.4 328.1 AVX512 On 21.6 308.5 346.7 OpenBenchmarking.org Watts, Fewer Is Better OpenVINO 2022.3 CPU Power Consumption Monitor 100 200 300 400 500
OpenVINO CPU Temperature Monitor Min Avg Max AVX512 Off 41.0 57.1 64.8 AVX512 On 40.5 60.6 66.6 OpenBenchmarking.org Celsius, Fewer Is Better OpenVINO 2022.3 CPU Temperature Monitor 20 40 60 80 100
OpenVINO Model: Age Gender Recognition Retail 0013 FP16 - Device: CPU OpenBenchmarking.org FPS, More Is Better OpenVINO 2022.3 Model: Age Gender Recognition Retail 0013 FP16 - Device: CPU AVX512 On AVX512 Off 20K 40K 60K 80K 100K SE +/- 314.35, N = 3 SE +/- 278.13, N = 3 110240.89 62564.16 1. (CXX) g++ options: -isystem -fsigned-char -ffunction-sections -fdata-sections -msse4.1 -msse4.2 -O3 -fno-strict-overflow -fwrapv -fPIC -fvisibility=hidden -Os -std=c++11 -MD -MT -MF
OpenVINO Model: Age Gender Recognition Retail 0013 FP16 - Device: CPU OpenBenchmarking.org ms, Fewer Is Better OpenVINO 2022.3 Model: Age Gender Recognition Retail 0013 FP16 - Device: CPU AVX512 On AVX512 Off 0.4298 0.8596 1.2894 1.7192 2.149 SE +/- 0.00, N = 3 SE +/- 0.01, N = 3 0.99 1.91 MIN: 0.35 / MAX: 19.47 MIN: 0.67 / MAX: 19.48 1. (CXX) g++ options: -isystem -fsigned-char -ffunction-sections -fdata-sections -msse4.1 -msse4.2 -O3 -fno-strict-overflow -fwrapv -fPIC -fvisibility=hidden -Os -std=c++11 -MD -MT -MF
OpenVINO CPU Peak Freq (Highest CPU Core Frequency) Monitor Min Avg Max AVX512 On 2250 2892 3097 AVX512 Off 2250 2590 3101 OpenBenchmarking.org Megahertz, More Is Better OpenVINO 2022.3 CPU Peak Freq (Highest CPU Core Frequency) Monitor 800 1600 2400 3200 4000
OpenVINO CPU Power Consumption Monitor Min Avg Max AVX512 Off 21.0 298.9 332.9 AVX512 On 21.2 316.4 347.6 OpenBenchmarking.org Watts, Fewer Is Better OpenVINO 2022.3 CPU Power Consumption Monitor 100 200 300 400 500
OpenVINO CPU Temperature Monitor Min Avg Max AVX512 Off 37.3 61.8 68.0 AVX512 On 41.0 64.6 70.0 OpenBenchmarking.org Celsius, Fewer Is Better OpenVINO 2022.3 CPU Temperature Monitor 20 40 60 80 100
OpenVINO Model: Age Gender Recognition Retail 0013 FP16-INT8 - Device: CPU OpenBenchmarking.org FPS, More Is Better OpenVINO 2022.3 Model: Age Gender Recognition Retail 0013 FP16-INT8 - Device: CPU AVX512 On AVX512 Off 16K 32K 48K 64K 80K SE +/- 95.74, N = 3 SE +/- 20.10, N = 3 73970.12 66895.49 1. (CXX) g++ options: -isystem -fsigned-char -ffunction-sections -fdata-sections -msse4.1 -msse4.2 -O3 -fno-strict-overflow -fwrapv -fPIC -fvisibility=hidden -Os -std=c++11 -MD -MT -MF
OpenVINO Model: Age Gender Recognition Retail 0013 FP16-INT8 - Device: CPU OpenBenchmarking.org ms, Fewer Is Better OpenVINO 2022.3 Model: Age Gender Recognition Retail 0013 FP16-INT8 - Device: CPU AVX512 On AVX512 Off 0.396 0.792 1.188 1.584 1.98 SE +/- 0.00, N = 3 SE +/- 0.00, N = 3 1.58 1.76 MIN: 0.55 / MAX: 17.78 MIN: 0.64 / MAX: 19.6 1. (CXX) g++ options: -isystem -fsigned-char -ffunction-sections -fdata-sections -msse4.1 -msse4.2 -O3 -fno-strict-overflow -fwrapv -fPIC -fvisibility=hidden -Os -std=c++11 -MD -MT -MF
OpenVINO CPU Peak Freq (Highest CPU Core Frequency) Monitor Min Avg Max AVX512 On 2250 2761 3119 AVX512 Off 2250 2626 3099 OpenBenchmarking.org Megahertz, More Is Better OpenVINO 2022.3 CPU Peak Freq (Highest CPU Core Frequency) Monitor 800 1600 2400 3200 4000
OpenVINO CPU Power Consumption Monitor Min Avg Max AVX512 Off 21.2 308.2 334.1 AVX512 On 21.4 311.4 336.9 OpenBenchmarking.org Watts, Fewer Is Better OpenVINO 2022.3 CPU Power Consumption Monitor 80 160 240 320 400
OpenVINO CPU Temperature Monitor Min Avg Max AVX512 On 41.4 63.4 68.1 AVX512 Off 41.0 63.9 69.0 OpenBenchmarking.org Celsius, Fewer Is Better OpenVINO 2022.3 CPU Temperature Monitor 20 40 60 80 100
OpenVKL Benchmark: vklBenchmark ISPC OpenBenchmarking.org Items / Sec, More Is Better OpenVKL 1.3.1 Benchmark: vklBenchmark ISPC AVX512 On AVX512 Off 300 600 900 1200 1500 SE +/- 2.65, N = 3 SE +/- 0.33, N = 3 1398 1179 MIN: 229 / MAX: 11779 MIN: 178 / MAX: 10473
OpenVKL Benchmark: vklBenchmark ISPC OpenBenchmarking.org Items / Sec Per Watt, More Is Better OpenVKL 1.3.1 Benchmark: vklBenchmark ISPC AVX512 On AVX512 Off 1.3181 2.6362 3.9543 5.2724 6.5905 5.858 4.569
OpenVKL CPU Peak Freq (Highest CPU Core Frequency) Monitor OpenBenchmarking.org Megahertz, More Is Better OpenVKL 1.3.1 CPU Peak Freq (Highest CPU Core Frequency) Monitor AVX512 On AVX512 Off 500 1000 1500 2000 2500 Min: 2250 / Avg: 2983.53 / Max: 3118 Min: 2250 / Avg: 2958.42 / Max: 3128
OpenVKL CPU Power Consumption Monitor Min Avg Max AVX512 On 19.9 238.7 322.2 AVX512 Off 10.2 258.1 348.2 OpenBenchmarking.org Watts, Fewer Is Better OpenVKL 1.3.1 CPU Power Consumption Monitor 100 200 300 400 500
OpenVKL CPU Temperature Monitor OpenBenchmarking.org Celsius, Fewer Is Better OpenVKL 1.3.1 CPU Temperature Monitor AVX512 On AVX512 Off 13 26 39 52 65 Min: 30.63 / Avg: 53.11 / Max: 62.25 Min: 30.38 / Avg: 54.49 / Max: 64.75
OSPRay Benchmark: gravity_spheres_volume/dim_512/ao/real_time OpenBenchmarking.org Items Per Second, More Is Better OSPRay 2.12 Benchmark: gravity_spheres_volume/dim_512/ao/real_time AVX512 On AVX512 Off 8 16 24 32 40 SE +/- 0.01, N = 3 SE +/- 0.02, N = 3 32.76 19.17
OSPRay Benchmark: gravity_spheres_volume/dim_512/ao/real_time OpenBenchmarking.org Items Per Second Per Watt, More Is Better OSPRay 2.12 Benchmark: gravity_spheres_volume/dim_512/ao/real_time AVX512 On AVX512 Off 0.0266 0.0532 0.0798 0.1064 0.133 0.118 0.066
OSPRay CPU Peak Freq (Highest CPU Core Frequency) Monitor Min Avg Max AVX512 Off 2250 2926 3096 AVX512 On 2250 2885 3099 OpenBenchmarking.org Megahertz, More Is Better OSPRay 2.12 CPU Peak Freq (Highest CPU Core Frequency) Monitor 800 1600 2400 3200 4000
OSPRay CPU Power Consumption Monitor Min Avg Max AVX512 On 20.1 276.9 329.8 AVX512 Off 20.8 290.6 346.2 OpenBenchmarking.org Watts, Fewer Is Better OSPRay 2.12 CPU Power Consumption Monitor 100 200 300 400 500
OSPRay CPU Temperature Monitor Min Avg Max AVX512 On 33.5 55.1 63.3 AVX512 Off 32.0 57.5 65.3 OpenBenchmarking.org Celsius, Fewer Is Better OSPRay 2.12 CPU Temperature Monitor 20 40 60 80 100
OSPRay Benchmark: gravity_spheres_volume/dim_512/scivis/real_time OpenBenchmarking.org Items Per Second, More Is Better OSPRay 2.12 Benchmark: gravity_spheres_volume/dim_512/scivis/real_time AVX512 On AVX512 Off 7 14 21 28 35 SE +/- 0.02, N = 3 SE +/- 0.00, N = 3 31.68 18.02
OSPRay Benchmark: gravity_spheres_volume/dim_512/scivis/real_time OpenBenchmarking.org Items Per Second Per Watt, More Is Better OSPRay 2.12 Benchmark: gravity_spheres_volume/dim_512/scivis/real_time AVX512 On AVX512 Off 0.0257 0.0514 0.0771 0.1028 0.1285 0.114 0.062
OSPRay CPU Peak Freq (Highest CPU Core Frequency) Monitor Min Avg Max AVX512 Off 2250 2925 3101 AVX512 On 2250 2875 3096 OpenBenchmarking.org Megahertz, More Is Better OSPRay 2.12 CPU Peak Freq (Highest CPU Core Frequency) Monitor 800 1600 2400 3200 4000
OSPRay CPU Power Consumption Monitor Min Avg Max AVX512 On 20.9 278.8 332.5 AVX512 Off 20.9 292.2 347.0 OpenBenchmarking.org Watts, Fewer Is Better OSPRay 2.12 CPU Power Consumption Monitor 100 200 300 400 500
OSPRay CPU Temperature Monitor Min Avg Max AVX512 On 39.4 57.2 64.3 AVX512 Off 39.4 59.7 66.1 OpenBenchmarking.org Celsius, Fewer Is Better OSPRay 2.12 CPU Temperature Monitor 20 40 60 80 100
OSPRay Benchmark: gravity_spheres_volume/dim_512/pathtracer/real_time OpenBenchmarking.org Items Per Second, More Is Better OSPRay 2.12 Benchmark: gravity_spheres_volume/dim_512/pathtracer/real_time AVX512 On AVX512 Off 7 14 21 28 35 SE +/- 0.01, N = 3 SE +/- 0.01, N = 3 27.97 19.55
OSPRay Benchmark: gravity_spheres_volume/dim_512/pathtracer/real_time OpenBenchmarking.org Items Per Second Per Watt, More Is Better OSPRay 2.12 Benchmark: gravity_spheres_volume/dim_512/pathtracer/real_time AVX512 On AVX512 Off 0.0239 0.0478 0.0717 0.0956 0.1195 0.106 0.072
OSPRay CPU Peak Freq (Highest CPU Core Frequency) Monitor Min Avg Max AVX512 Off 2250 3052 3098 AVX512 On 2250 3024 3099 OpenBenchmarking.org Megahertz, More Is Better OSPRay 2.12 CPU Peak Freq (Highest CPU Core Frequency) Monitor 800 1600 2400 3200 4000
OSPRay CPU Power Consumption Monitor Min Avg Max AVX512 On 21.1 263.0 313.6 AVX512 Off 21.2 271.7 321.8 OpenBenchmarking.org Watts, Fewer Is Better OSPRay 2.12 CPU Power Consumption Monitor 80 160 240 320 400
OSPRay CPU Temperature Monitor Min Avg Max AVX512 On 40.0 55.6 61.3 AVX512 Off 40.0 57.0 62.8 OpenBenchmarking.org Celsius, Fewer Is Better OSPRay 2.12 CPU Temperature Monitor 20 40 60 80 100
TensorFlow Device: CPU - Batch Size: 16 - Model: AlexNet OpenBenchmarking.org images/sec, More Is Better TensorFlow 2.12 Device: CPU - Batch Size: 16 - Model: AlexNet AVX512 On AVX512 Off 70 140 210 280 350 SE +/- 0.70, N = 6 SE +/- 0.29, N = 3 342.88 70.10
TensorFlow Device: CPU - Batch Size: 16 - Model: AlexNet OpenBenchmarking.org images/sec Per Watt, More Is Better TensorFlow 2.12 Device: CPU - Batch Size: 16 - Model: AlexNet AVX512 On AVX512 Off 0.6674 1.3348 2.0022 2.6696 3.337 2.966 0.576
TensorFlow CPU Peak Freq (Highest CPU Core Frequency) Monitor Min Avg Max AVX512 Off 2250 2992 3136 AVX512 On 2250 2801 3254 OpenBenchmarking.org Megahertz, More Is Better TensorFlow 2.12 CPU Peak Freq (Highest CPU Core Frequency) Monitor 800 1600 2400 3200 4000
TensorFlow CPU Power Consumption Monitor Min Avg Max AVX512 On 10.4 115.6 199.6 AVX512 Off 20.6 121.7 145.6 OpenBenchmarking.org Watts, Fewer Is Better TensorFlow 2.12 CPU Power Consumption Monitor 50 100 150 200 250
TensorFlow CPU Temperature Monitor Min Avg Max AVX512 Off 33.9 40.3 49.4 AVX512 On 34.9 41.1 49.4 OpenBenchmarking.org Celsius, Fewer Is Better TensorFlow 2.12 CPU Temperature Monitor 14 28 42 56 70
TensorFlow Device: CPU - Batch Size: 32 - Model: AlexNet OpenBenchmarking.org images/sec, More Is Better TensorFlow 2.12 Device: CPU - Batch Size: 32 - Model: AlexNet AVX512 On AVX512 Off 120 240 360 480 600 SE +/- 2.06, N = 6 SE +/- 0.16, N = 3 562.48 84.96
TensorFlow Device: CPU - Batch Size: 32 - Model: AlexNet OpenBenchmarking.org images/sec Per Watt, More Is Better TensorFlow 2.12 Device: CPU - Batch Size: 32 - Model: AlexNet AVX512 On AVX512 Off 0.9846 1.9692 2.9538 3.9384 4.923 4.376 0.691
TensorFlow CPU Peak Freq (Highest CPU Core Frequency) Monitor Min Avg Max AVX512 Off 2250 3039 3146 AVX512 On 2250 2836 3101 OpenBenchmarking.org Megahertz, More Is Better TensorFlow 2.12 CPU Peak Freq (Highest CPU Core Frequency) Monitor 800 1600 2400 3200 4000
TensorFlow CPU Power Consumption Monitor Min Avg Max AVX512 Off 20.5 123.0 139.4 AVX512 On 20.3 128.6 210.9 OpenBenchmarking.org Watts, Fewer Is Better TensorFlow 2.12 CPU Power Consumption Monitor 60 120 180 240 300
TensorFlow CPU Temperature Monitor Min Avg Max AVX512 Off 30.5 37.1 45.6 AVX512 On 32.0 37.8 45.3 OpenBenchmarking.org Celsius, Fewer Is Better TensorFlow 2.12 CPU Temperature Monitor 12 24 36 48 60
TensorFlow Device: CPU - Batch Size: 64 - Model: AlexNet OpenBenchmarking.org images/sec, More Is Better TensorFlow 2.12 Device: CPU - Batch Size: 64 - Model: AlexNet AVX512 On AVX512 Off 200 400 600 800 1000 SE +/- 1.99, N = 5 SE +/- 0.08, N = 3 857.55 97.59
TensorFlow Device: CPU - Batch Size: 64 - Model: AlexNet OpenBenchmarking.org images/sec Per Watt, More Is Better TensorFlow 2.12 Device: CPU - Batch Size: 64 - Model: AlexNet AVX512 On AVX512 Off 1.3361 2.6722 4.0083 5.3444 6.6805 5.938 0.778
TensorFlow CPU Peak Freq (Highest CPU Core Frequency) Monitor Min Avg Max AVX512 Off 2250 3057 3377 AVX512 On 2250 2751 3111 OpenBenchmarking.org Megahertz, More Is Better TensorFlow 2.12 CPU Peak Freq (Highest CPU Core Frequency) Monitor 800 1600 2400 3200 4000
TensorFlow CPU Power Consumption Monitor Min Avg Max AVX512 Off 19.9 125.4 141.0 AVX512 On 20.2 144.4 224.5 OpenBenchmarking.org Watts, Fewer Is Better TensorFlow 2.12 CPU Power Consumption Monitor 60 120 180 240 300
TensorFlow CPU Temperature Monitor Min Avg Max AVX512 Off 30.4 36.5 46.4 AVX512 On 30.6 38.3 45.5 OpenBenchmarking.org Celsius, Fewer Is Better TensorFlow 2.12 CPU Temperature Monitor 14 28 42 56 70
TensorFlow Device: CPU - Batch Size: 256 - Model: AlexNet OpenBenchmarking.org images/sec, More Is Better TensorFlow 2.12 Device: CPU - Batch Size: 256 - Model: AlexNet AVX512 On AVX512 Off 300 600 900 1200 1500 SE +/- 6.55, N = 3 SE +/- 0.24, N = 3 1422.36 106.28
TensorFlow Device: CPU - Batch Size: 256 - Model: AlexNet OpenBenchmarking.org images/sec Per Watt, More Is Better TensorFlow 2.12 Device: CPU - Batch Size: 256 - Model: AlexNet AVX512 On AVX512 Off 2 4 6 8 10 6.984 0.839
TensorFlow CPU Peak Freq (Highest CPU Core Frequency) Monitor Min Avg Max AVX512 Off 2250 3073 3166 AVX512 On 2250 2742 3102 OpenBenchmarking.org Megahertz, More Is Better TensorFlow 2.12 CPU Peak Freq (Highest CPU Core Frequency) Monitor 800 1600 2400 3200 4000
TensorFlow CPU Power Consumption Monitor Min Avg Max AVX512 Off 20.0 126.7 155.1 AVX512 On 20.1 203.7 257.0 OpenBenchmarking.org Watts, Fewer Is Better TensorFlow 2.12 CPU Power Consumption Monitor 70 140 210 280 350
TensorFlow CPU Temperature Monitor Min Avg Max AVX512 Off 28.8 36.0 47.9 AVX512 On 30.9 44.8 52.3 OpenBenchmarking.org Celsius, Fewer Is Better TensorFlow 2.12 CPU Temperature Monitor 15 30 45 60 75
TensorFlow Device: CPU - Batch Size: 512 - Model: AlexNet OpenBenchmarking.org images/sec, More Is Better TensorFlow 2.12 Device: CPU - Batch Size: 512 - Model: AlexNet AVX512 On AVX512 Off 400 800 1200 1600 2000 SE +/- 1.53, N = 3 SE +/- 0.06, N = 3 1632.40 109.88
TensorFlow Device: CPU - Batch Size: 512 - Model: AlexNet OpenBenchmarking.org images/sec Per Watt, More Is Better TensorFlow 2.12 Device: CPU - Batch Size: 512 - Model: AlexNet AVX512 On AVX512 Off 2 4 6 8 10 7.097 0.865
TensorFlow CPU Peak Freq (Highest CPU Core Frequency) Monitor OpenBenchmarking.org Megahertz, More Is Better TensorFlow 2.12 CPU Peak Freq (Highest CPU Core Frequency) Monitor AVX512 Off AVX512 On 600 1200 1800 2400 3000 Min: 2203 / Avg: 3078.92 / Max: 3359 Min: 2250 / Avg: 2753.56 / Max: 3095
TensorFlow CPU Power Consumption Monitor OpenBenchmarking.org Watts, Fewer Is Better TensorFlow 2.12 CPU Power Consumption Monitor AVX512 Off AVX512 On 50 100 150 200 250 Min: 20.3 / Avg: 127.04 / Max: 189.92 Min: 20.26 / Avg: 230.01 / Max: 283.14
TensorFlow CPU Temperature Monitor OpenBenchmarking.org Celsius, Fewer Is Better TensorFlow 2.12 CPU Temperature Monitor AVX512 Off AVX512 On 11 22 33 44 55 Min: 28.75 / Avg: 36.18 / Max: 47.38 Min: 33.88 / Avg: 49.84 / Max: 55.75
TensorFlow Device: CPU - Batch Size: 16 - Model: GoogLeNet OpenBenchmarking.org images/sec, More Is Better TensorFlow 2.12 Device: CPU - Batch Size: 16 - Model: GoogLeNet AVX512 On AVX512 Off 20 40 60 80 100 SE +/- 1.96, N = 15 SE +/- 0.17, N = 3 104.77 39.64
TensorFlow Device: CPU - Batch Size: 16 - Model: GoogLeNet OpenBenchmarking.org images/sec Per Watt, More Is Better TensorFlow 2.12 Device: CPU - Batch Size: 16 - Model: GoogLeNet AVX512 On AVX512 Off 0.1742 0.3484 0.5226 0.6968 0.871 0.774 0.335
TensorFlow CPU Peak Freq (Highest CPU Core Frequency) Monitor Min Avg Max AVX512 Off 2250 3032 3208 AVX512 On 2250 2977 3320 OpenBenchmarking.org Megahertz, More Is Better TensorFlow 2.12 CPU Peak Freq (Highest CPU Core Frequency) Monitor 800 1600 2400 3200 4000
TensorFlow CPU Power Consumption Monitor Min Avg Max AVX512 Off 20.1 118.4 140.5 AVX512 On 19.8 135.4 179.1 OpenBenchmarking.org Watts, Fewer Is Better TensorFlow 2.12 CPU Power Consumption Monitor 50 100 150 200 250
TensorFlow CPU Temperature Monitor Min Avg Max AVX512 Off 29.1 35.7 47.9 AVX512 On 33.0 38.5 45.8 OpenBenchmarking.org Celsius, Fewer Is Better TensorFlow 2.12 CPU Temperature Monitor 14 28 42 56 70
TensorFlow Device: CPU - Batch Size: 16 - Model: ResNet-50 OpenBenchmarking.org images/sec, More Is Better TensorFlow 2.12 Device: CPU - Batch Size: 16 - Model: ResNet-50 AVX512 On AVX512 Off 10 20 30 40 50 SE +/- 0.03, N = 3 SE +/- 0.05, N = 3 43.11 15.91
TensorFlow Device: CPU - Batch Size: 16 - Model: ResNet-50 OpenBenchmarking.org images/sec Per Watt, More Is Better TensorFlow 2.12 Device: CPU - Batch Size: 16 - Model: ResNet-50 AVX512 On AVX512 Off 0.0662 0.1324 0.1986 0.2648 0.331 0.294 0.113
TensorFlow CPU Peak Freq (Highest CPU Core Frequency) Monitor Min Avg Max AVX512 Off 2250 3075 3217 AVX512 On 2250 3041 3348 OpenBenchmarking.org Megahertz, More Is Better TensorFlow 2.12 CPU Peak Freq (Highest CPU Core Frequency) Monitor 800 1600 2400 3200 4000
TensorFlow CPU Power Consumption Monitor Min Avg Max AVX512 Off 20.2 140.3 151.8 AVX512 On 20.1 146.6 171.4 OpenBenchmarking.org Watts, Fewer Is Better TensorFlow 2.12 CPU Power Consumption Monitor 50 100 150 200 250
TensorFlow CPU Temperature Monitor Min Avg Max AVX512 Off 28.8 37.2 48.8 AVX512 On 30.4 38.1 44.6 OpenBenchmarking.org Celsius, Fewer Is Better TensorFlow 2.12 CPU Temperature Monitor 14 28 42 56 70
TensorFlow Device: CPU - Batch Size: 32 - Model: GoogLeNet OpenBenchmarking.org images/sec, More Is Better TensorFlow 2.12 Device: CPU - Batch Size: 32 - Model: GoogLeNet AVX512 On AVX512 Off 40 80 120 160 200 SE +/- 3.09, N = 15 SE +/- 0.10, N = 3 180.77 44.46
TensorFlow Device: CPU - Batch Size: 32 - Model: GoogLeNet OpenBenchmarking.org images/sec Per Watt, More Is Better TensorFlow 2.12 Device: CPU - Batch Size: 32 - Model: GoogLeNet AVX512 On AVX512 Off 0.2561 0.5122 0.7683 1.0244 1.2805 1.138 0.360
TensorFlow CPU Peak Freq (Highest CPU Core Frequency) Monitor Min Avg Max AVX512 Off 2250 3064 3197 AVX512 On 2250 2992 3472 OpenBenchmarking.org Megahertz, More Is Better TensorFlow 2.12 CPU Peak Freq (Highest CPU Core Frequency) Monitor 800 1600 2400 3200 4000
TensorFlow CPU Power Consumption Monitor Min Avg Max AVX512 Off 20.2 123.6 149.7 AVX512 On 20.3 158.9 208.4 OpenBenchmarking.org Watts, Fewer Is Better TensorFlow 2.12 CPU Power Consumption Monitor 60 120 180 240 300
TensorFlow CPU Temperature Monitor Min Avg Max AVX512 Off 29.3 35.3 47.1 AVX512 On 30.8 40.3 49.9 OpenBenchmarking.org Celsius, Fewer Is Better TensorFlow 2.12 CPU Temperature Monitor 14 28 42 56 70
TensorFlow Device: CPU - Batch Size: 32 - Model: ResNet-50 OpenBenchmarking.org images/sec, More Is Better TensorFlow 2.12 Device: CPU - Batch Size: 32 - Model: ResNet-50 AVX512 On AVX512 Off 16 32 48 64 80 SE +/- 0.23, N = 3 SE +/- 0.04, N = 3 71.62 17.48
TensorFlow Device: CPU - Batch Size: 32 - Model: ResNet-50 OpenBenchmarking.org images/sec Per Watt, More Is Better TensorFlow 2.12 Device: CPU - Batch Size: 32 - Model: ResNet-50 AVX512 On AVX512 Off 0.0916 0.1832 0.2748 0.3664 0.458 0.407 0.124
TensorFlow CPU Peak Freq (Highest CPU Core Frequency) Monitor Min Avg Max AVX512 Off 2250 3084 3221 AVX512 On 2250 3066 3438 OpenBenchmarking.org Megahertz, More Is Better TensorFlow 2.12 CPU Peak Freq (Highest CPU Core Frequency) Monitor 800 1600 2400 3200 4000
TensorFlow CPU Power Consumption Monitor Min Avg Max AVX512 Off 20.1 141.1 172.9 AVX512 On 20.0 176.0 204.8 OpenBenchmarking.org Watts, Fewer Is Better TensorFlow 2.12 CPU Power Consumption Monitor 50 100 150 200 250
TensorFlow CPU Temperature Monitor Min Avg Max AVX512 Off 28.3 37.5 51.6 AVX512 On 31.3 41.8 48.6 OpenBenchmarking.org Celsius, Fewer Is Better TensorFlow 2.12 CPU Temperature Monitor 15 30 45 60 75
TensorFlow Device: CPU - Batch Size: 64 - Model: GoogLeNet OpenBenchmarking.org images/sec, More Is Better TensorFlow 2.12 Device: CPU - Batch Size: 64 - Model: GoogLeNet AVX512 On AVX512 Off 60 120 180 240 300 SE +/- 2.62, N = 15 SE +/- 0.18, N = 3 277.24 46.28
TensorFlow Device: CPU - Batch Size: 64 - Model: GoogLeNet OpenBenchmarking.org images/sec Per Watt, More Is Better TensorFlow 2.12 Device: CPU - Batch Size: 64 - Model: GoogLeNet AVX512 On AVX512 Off 0.3321 0.6642 0.9963 1.3284 1.6605 1.476 0.369
TensorFlow CPU Peak Freq (Highest CPU Core Frequency) Monitor Min Avg Max AVX512 Off 2250 3075 3309 AVX512 On 2250 2995 3532 OpenBenchmarking.org Megahertz, More Is Better TensorFlow 2.12 CPU Peak Freq (Highest CPU Core Frequency) Monitor 1000 2000 3000 4000 5000
TensorFlow CPU Power Consumption Monitor Min Avg Max AVX512 Off 19.9 125.3 152.5 AVX512 On 20.1 187.9 236.9 OpenBenchmarking.org Watts, Fewer Is Better TensorFlow 2.12 CPU Power Consumption Monitor 60 120 180 240 300
TensorFlow CPU Temperature Monitor Min Avg Max AVX512 Off 29.9 35.9 49.0 AVX512 On 32.5 44.3 52.8 OpenBenchmarking.org Celsius, Fewer Is Better TensorFlow 2.12 CPU Temperature Monitor 15 30 45 60 75
TensorFlow Device: CPU - Batch Size: 64 - Model: ResNet-50 OpenBenchmarking.org images/sec, More Is Better TensorFlow 2.12 Device: CPU - Batch Size: 64 - Model: ResNet-50 AVX512 On AVX512 Off 20 40 60 80 100 SE +/- 0.06, N = 3 SE +/- 0.03, N = 3 96.63 18.45
TensorFlow Device: CPU - Batch Size: 64 - Model: ResNet-50 OpenBenchmarking.org images/sec Per Watt, More Is Better TensorFlow 2.12 Device: CPU - Batch Size: 64 - Model: ResNet-50 AVX512 On AVX512 Off 0.1087 0.2174 0.3261 0.4348 0.5435 0.483 0.131
TensorFlow CPU Peak Freq (Highest CPU Core Frequency) Monitor Min Avg Max AVX512 Off 2250 3090 3249 AVX512 On 2250 3021 3283 OpenBenchmarking.org Megahertz, More Is Better TensorFlow 2.12 CPU Peak Freq (Highest CPU Core Frequency) Monitor 800 1600 2400 3200 4000
TensorFlow CPU Power Consumption Monitor Min Avg Max AVX512 Off 20.1 140.4 179.7 AVX512 On 20.6 200.2 230.6 OpenBenchmarking.org Watts, Fewer Is Better TensorFlow 2.12 CPU Power Consumption Monitor 60 120 180 240 300
TensorFlow CPU Temperature Monitor Min Avg Max AVX512 Off 28.8 38.1 51.6 AVX512 On 33.5 45.3 53.3 OpenBenchmarking.org Celsius, Fewer Is Better TensorFlow 2.12 CPU Temperature Monitor 15 30 45 60 75
TensorFlow Device: CPU - Batch Size: 256 - Model: GoogLeNet OpenBenchmarking.org images/sec, More Is Better TensorFlow 2.12 Device: CPU - Batch Size: 256 - Model: GoogLeNet AVX512 On AVX512 Off 110 220 330 440 550 SE +/- 4.92, N = 3 SE +/- 0.06, N = 3 501.67 46.17
TensorFlow Device: CPU - Batch Size: 256 - Model: GoogLeNet OpenBenchmarking.org images/sec Per Watt, More Is Better TensorFlow 2.12 Device: CPU - Batch Size: 256 - Model: GoogLeNet AVX512 On AVX512 Off 0.4676 0.9352 1.4028 1.8704 2.338 2.078 0.362
TensorFlow CPU Peak Freq (Highest CPU Core Frequency) Monitor OpenBenchmarking.org Megahertz, More Is Better TensorFlow 2.12 CPU Peak Freq (Highest CPU Core Frequency) Monitor AVX512 Off AVX512 On 600 1200 1800 2400 3000 Min: 2250 / Avg: 3085.78 / Max: 3417 Min: 2250 / Avg: 2916.06 / Max: 3157
TensorFlow CPU Power Consumption Monitor OpenBenchmarking.org Watts, Fewer Is Better TensorFlow 2.12 CPU Power Consumption Monitor AVX512 Off AVX512 On 50 100 150 200 250 Min: 20.1 / Avg: 127.43 / Max: 214.25 Min: 19.85 / Avg: 241.46 / Max: 277.92
TensorFlow CPU Temperature Monitor OpenBenchmarking.org Celsius, Fewer Is Better TensorFlow 2.12 CPU Temperature Monitor AVX512 Off AVX512 On 12 24 36 48 60 Min: 29.75 / Avg: 36.4 / Max: 52.13 Min: 34.38 / Avg: 50.91 / Max: 58.88
TensorFlow Device: CPU - Batch Size: 256 - Model: ResNet-50 OpenBenchmarking.org images/sec, More Is Better TensorFlow 2.12 Device: CPU - Batch Size: 256 - Model: ResNet-50 AVX512 On AVX512 Off 30 60 90 120 150 SE +/- 1.01, N = 12 SE +/- 0.02, N = 3 119.32 19.89
TensorFlow Device: CPU - Batch Size: 256 - Model: ResNet-50 OpenBenchmarking.org images/sec Per Watt, More Is Better TensorFlow 2.12 Device: CPU - Batch Size: 256 - Model: ResNet-50 AVX512 On AVX512 Off 0.1208 0.2416 0.3624 0.4832 0.604 0.537 0.143
TensorFlow CPU Peak Freq (Highest CPU Core Frequency) Monitor OpenBenchmarking.org Megahertz, More Is Better TensorFlow 2.12 CPU Peak Freq (Highest CPU Core Frequency) Monitor AVX512 Off AVX512 On 600 1200 1800 2400 3000 Min: 2250 / Avg: 3085.27 / Max: 3559 Min: 2250 / Avg: 2956.33 / Max: 3308
TensorFlow CPU Power Consumption Monitor OpenBenchmarking.org Watts, Fewer Is Better TensorFlow 2.12 CPU Power Consumption Monitor AVX512 Off AVX512 On 50 100 150 200 250 Min: 20.06 / Avg: 139.18 / Max: 198.98 Min: 20.59 / Avg: 222.08 / Max: 272.77
TensorFlow CPU Temperature Monitor OpenBenchmarking.org Celsius, Fewer Is Better TensorFlow 2.12 CPU Temperature Monitor AVX512 Off AVX512 On 11 22 33 44 55 Min: 29 / Avg: 38.33 / Max: 53.13 Min: 37.63 / Avg: 48.39 / Max: 57.63
TensorFlow Device: CPU - Batch Size: 512 - Model: GoogLeNet OpenBenchmarking.org images/sec, More Is Better TensorFlow 2.12 Device: CPU - Batch Size: 512 - Model: GoogLeNet AVX512 On AVX512 Off 90 180 270 360 450 SE +/- 5.30, N = 12 SE +/- 0.05, N = 3 417.25 46.92
TensorFlow Device: CPU - Batch Size: 512 - Model: GoogLeNet OpenBenchmarking.org images/sec Per Watt, More Is Better TensorFlow 2.12 Device: CPU - Batch Size: 512 - Model: GoogLeNet AVX512 On AVX512 Off 0.3974 0.7948 1.1922 1.5896 1.987 1.766 0.369
TensorFlow CPU Peak Freq (Highest CPU Core Frequency) Monitor OpenBenchmarking.org Megahertz, More Is Better TensorFlow 2.12 CPU Peak Freq (Highest CPU Core Frequency) Monitor AVX512 Off AVX512 On 600 1200 1800 2400 3000 Min: 2250 / Avg: 3083.42 / Max: 3336 Min: 2250 / Avg: 2973.8 / Max: 3207
TensorFlow CPU Power Consumption Monitor OpenBenchmarking.org Watts, Fewer Is Better TensorFlow 2.12 CPU Power Consumption Monitor AVX512 Off AVX512 On 50 100 150 200 250 Min: 19.97 / Avg: 127.28 / Max: 218.04 Min: 10.84 / Avg: 236.33 / Max: 283.4
TensorFlow CPU Temperature Monitor OpenBenchmarking.org Celsius, Fewer Is Better TensorFlow 2.12 CPU Temperature Monitor AVX512 Off AVX512 On 12 24 36 48 60 Min: 30.13 / Avg: 36.83 / Max: 51.13 Min: 34.63 / Avg: 50.93 / Max: 58.38
TensorFlow Device: CPU - Batch Size: 512 - Model: ResNet-50 OpenBenchmarking.org images/sec, More Is Better TensorFlow 2.12 Device: CPU - Batch Size: 512 - Model: ResNet-50 AVX512 On AVX512 Off 30 60 90 120 150 SE +/- 0.99, N = 3 SE +/- 0.04, N = 3 122.81 20.78
TensorFlow Device: CPU - Batch Size: 512 - Model: ResNet-50 OpenBenchmarking.org images/sec Per Watt, More Is Better TensorFlow 2.12 Device: CPU - Batch Size: 512 - Model: ResNet-50 AVX512 On AVX512 Off 0.122 0.244 0.366 0.488 0.61 0.542 0.148
TensorFlow CPU Peak Freq (Highest CPU Core Frequency) Monitor OpenBenchmarking.org Megahertz, More Is Better TensorFlow 2.12 CPU Peak Freq (Highest CPU Core Frequency) Monitor AVX512 Off AVX512 On 600 1200 1800 2400 3000 Min: 2243 / Avg: 3085.93 / Max: 3295 Min: 2250 / Avg: 2958.12 / Max: 3370
TensorFlow CPU Power Consumption Monitor OpenBenchmarking.org Watts, Fewer Is Better TensorFlow 2.12 CPU Power Consumption Monitor AVX512 Off AVX512 On 50 100 150 200 250 Min: 20.22 / Avg: 140.06 / Max: 209.42 Min: 20.5 / Avg: 226.78 / Max: 289.01
TensorFlow CPU Temperature Monitor OpenBenchmarking.org Celsius, Fewer Is Better TensorFlow 2.12 CPU Temperature Monitor AVX512 Off AVX512 On 11 22 33 44 55 Min: 29.38 / Avg: 39.11 / Max: 54.13 Min: 35.38 / Avg: 48.79 / Max: 57.5
Phoronix Test Suite v10.8.4