oneDNN 3.0 Intel Sapphire Rapids AMX

2 x Intel Xeon Platinum 8490H oneDNN Intel AMX Sapphire Rapids benchmarks by Michael Larabel for a future article.

Compare your own system(s) to this result file with the Phoronix Test Suite by running the command: phoronix-test-suite benchmark 2301158-NE-ONEDNN30I77
Jump To Table - Results

View

Do Not Show Noisy Results
Do Not Show Results With Incomplete Data
Do Not Show Results With Little Change/Spread
List Notable Results

Statistics

Show Overall Harmonic Mean(s)
Show Overall Geometric Mean
Show Wins / Losses Counts (Pie Chart)
Normalize Results
Remove Outliers Before Calculating Averages

Graph Settings

Force Line Graphs Where Applicable
Convert To Scalar Where Applicable
Prefer Vertical Bar Graphs
No Box Plots
On Line Graphs With Missing Data, Connect The Line Gaps

Multi-Way Comparison

Condense Multi-Option Tests Into Single Result Graphs
Condense Test Profiles With Multiple Version Results Into Single Result Graphs

Table

Show Detailed System Result Table

Run Management

Highlight
Result
Hide
Result
Result
Identifier
Performance Per
Dollar
Date
Run
  Test
  Duration
AVX512_CORE_AMX
January 15 2023
  12 Minutes
AVX512_CORE_FP16
January 15 2023
  23 Minutes
AVX512_CORE_BF16
January 15 2023
  27 Minutes
AVX512_CORE_VNNI
January 15 2023
  27 Minutes
AVX512_CORE
January 15 2023
  29 Minutes
Invert Hiding All Results Option
  23 Minutes

Only show results where is faster than
Only show results matching title/arguments (delimit multiple options with a comma):
Do not show results matching title/arguments (delimit multiple options with a comma):

AVX512_CORE_AMX

Kernel Notes: Transparent Huge Pages: madvise
Compiler Notes: --build=x86_64-linux-gnu --disable-vtable-verify --disable-werror --enable-bootstrap --enable-cet --enable-checking=release --enable-clocale=gnu --enable-default-pie --enable-gnu-unique-object --enable-languages=c,ada,c++,go,brig,d,fortran,objc,obj-c++,m2 --enable-libphobos-checking=release --enable-libstdcxx-debug --enable-libstdcxx-time=yes --enable-link-serialization=2 --enable-multiarch --enable-multilib --enable-nls --enable-objc-gc=auto --enable-offload-targets=nvptx-none=/build/gcc-11-xKiWfi/gcc-11-11.3.0/debian/tmp-nvptx/usr,amdgcn-amdhsa=/build/gcc-11-xKiWfi/gcc-11-11.3.0/debian/tmp-gcn/usr --enable-plugin --enable-shared --enable-threads=posix --host=x86_64-linux-gnu --program-prefix=x86_64-linux-gnu- --target=x86_64-linux-gnu --with-abi=m64 --with-arch-32=i686 --with-build-config=bootstrap-lto-lean --with-default-libstdcxx-abi=new --with-gcc-major-version-only --with-multilib-list=m32,m64,mx32 --with-target-system-zlib=auto --with-tune=generic --without-cuda-driver -v
Processor Notes: Scaling Governor: intel_pstate performance (EPP: performance) - CPU Microcode: 0x2b0000c0
Security Notes: itlb_multihit: Not affected + l1tf: Not affected + mds: Not affected + meltdown: Not affected + mmio_stale_data: Not affected + retbleed: Not affected + spec_store_bypass: Mitigation of SSB disabled via prctl + spectre_v1: Mitigation of usercopy/swapgs barriers and __user pointer sanitization + spectre_v2: Mitigation of Enhanced IBRS IBPB: conditional RSB filling PBRSB-eIBRS: SW sequence + srbds: Not affected + tsx_async_abort: Not affected

AVX512_CORE_FP16

AVX512_CORE_BF16

AVX512_CORE_VNNI

AVX512_CORE

Processor: 2 x Intel Xeon Platinum 8490H @ 3.50GHz (120 Cores / 240 Threads), Motherboard: Quanta Cloud S6Q-MB-MPS (3A10.uh BIOS), Chipset: Intel Device 1bce, Memory: 16 x 64 GB 4800MT/s Samsung M321R8GA0BB0-CQKEG, Disk: 2 x 1920GB SAMSUNG MZWLJ1T9HBJR-00007 + 3841GB Micron_9300_MTFDHAL3T8TDP + 960GB INTEL SSDSC2KG96, Graphics: ASPEED, Monitor: VGA HDMI, Network: 4 x Intel E810-C for QSFP + 2 x Intel X710 for 10GBASE-T

OS: Ubuntu 22.04, Kernel: 6.1.4-060104-generic (x86_64), Desktop: GNOME Shell 42.2, Display Server: X Server 1.21.1.3, Vulkan: 1.2.204, Compiler: GCC 11.3.0 + Clang 14.0.0-1ubuntu1, File-System: ext4, Screen Resolution: 1920x1080

oneDNN 3.0 Intel Sapphire Rapids AMXOpenBenchmarking.orgPhoronix Test Suite2 x Intel Xeon Platinum 8490H @ 3.50GHz (120 Cores / 240 Threads)Quanta Cloud S6Q-MB-MPS (3A10.uh BIOS)Intel Device 1bce16 x 64 GB 4800MT/s Samsung M321R8GA0BB0-CQKEG2 x 1920GB SAMSUNG MZWLJ1T9HBJR-00007 + 3841GB Micron_9300_MTFDHAL3T8TDP + 960GB INTEL SSDSC2KG96ASPEEDVGA HDMI4 x Intel E810-C for QSFP + 2 x Intel X710 for 10GBASE-TUbuntu 22.046.1.4-060104-generic (x86_64)GNOME Shell 42.2X Server 1.21.1.31.2.204GCC 11.3.0 + Clang 14.0.0-1ubuntu1ext41920x1080ProcessorMotherboardChipsetMemoryDiskGraphicsMonitorNetworkOSKernelDesktopDisplay ServerVulkanCompilerFile-SystemScreen ResolutionOneDNN 3.0 Intel Sapphire Rapids AMX BenchmarksSystem Logs- Transparent Huge Pages: madvise- --build=x86_64-linux-gnu --disable-vtable-verify --disable-werror --enable-bootstrap --enable-cet --enable-checking=release --enable-clocale=gnu --enable-default-pie --enable-gnu-unique-object --enable-languages=c,ada,c++,go,brig,d,fortran,objc,obj-c++,m2 --enable-libphobos-checking=release --enable-libstdcxx-debug --enable-libstdcxx-time=yes --enable-link-serialization=2 --enable-multiarch --enable-multilib --enable-nls --enable-objc-gc=auto --enable-offload-targets=nvptx-none=/build/gcc-11-xKiWfi/gcc-11-11.3.0/debian/tmp-nvptx/usr,amdgcn-amdhsa=/build/gcc-11-xKiWfi/gcc-11-11.3.0/debian/tmp-gcn/usr --enable-plugin --enable-shared --enable-threads=posix --host=x86_64-linux-gnu --program-prefix=x86_64-linux-gnu- --target=x86_64-linux-gnu --with-abi=m64 --with-arch-32=i686 --with-build-config=bootstrap-lto-lean --with-default-libstdcxx-abi=new --with-gcc-major-version-only --with-multilib-list=m32,m64,mx32 --with-target-system-zlib=auto --with-tune=generic --without-cuda-driver -v - Scaling Governor: intel_pstate performance (EPP: performance) - CPU Microcode: 0x2b0000c0- itlb_multihit: Not affected + l1tf: Not affected + mds: Not affected + meltdown: Not affected + mmio_stale_data: Not affected + retbleed: Not affected + spec_store_bypass: Mitigation of SSB disabled via prctl + spectre_v1: Mitigation of usercopy/swapgs barriers and __user pointer sanitization + spectre_v2: Mitigation of Enhanced IBRS IBPB: conditional RSB filling PBRSB-eIBRS: SW sequence + srbds: Not affected + tsx_async_abort: Not affected

AVX512_CORE_AMXAVX512_CORE_FP16AVX512_CORE_BF16AVX512_CORE_VNNIAVX512_CORELogarithmic Result OverviewPhoronix Test SuiteoneDNNoneDNNoneDNNoneDNNoneDNNM.M.B.S.T - bf16bf16bf16 - CPUD.B.s - bf16bf16bf16 - CPUD.B.s - bf16bf16bf16 - CPUIP Shapes 1D - bf16bf16bf16 - CPUR.N.N.I - bf16bf16bf16 - CPU

oneDNN 3.0 Intel Sapphire Rapids AMXonednn: Deconvolution Batch shapes_1d - bf16bf16bf16 - CPUonednn: Deconvolution Batch shapes_3d - bf16bf16bf16 - CPUonednn: IP Shapes 1D - bf16bf16bf16 - CPUonednn: Matrix Multiply Batch Shapes Transformer - bf16bf16bf16 - CPUonednn: Recurrent Neural Network Inference - bf16bf16bf16 - CPUAVX512_CORE_AMXAVX512_CORE_FP16AVX512_CORE_BF16AVX512_CORE_VNNIAVX512_CORE0.5806460.4457795.737170.329961850.3462.170731.045116.483660.383785875.2842.171091.048366.588410.386636864.4593.067212.378926.8243316.5751862.4213.056122.383616.6481616.4589856.896OpenBenchmarking.org

CPU Temperature Monitor

OpenBenchmarking.orgCelsiusCPU Temperature MonitorPhoronix Test Suite System MonitoringAVX512_CORE_AMXAVX512_CORE_FP16AVX512_CORE_BF16AVX512_CORE_VNNIAVX512_CORE1122334455Min: 29 / Avg: 48.95 / Max: 54Min: 33 / Avg: 49.66 / Max: 56Min: 28 / Avg: 49 / Max: 54Min: 29 / Avg: 47.07 / Max: 53Min: 30 / Avg: 46.72 / Max: 53

CPU Peak Freq (Highest CPU Core Frequency) Monitor

OpenBenchmarking.orgMegahertzCPU Peak Freq (Highest CPU Core Frequency) MonitorPhoronix Test Suite System MonitoringAVX512_CORE_AMXAVX512_CORE_FP16AVX512_CORE_BF16AVX512_CORE_VNNIAVX512_CORE9001800270036004500Min: 1900 / Avg: 2940.37 / Max: 5340Min: 1900 / Avg: 2938.87 / Max: 5424Min: 1900 / Avg: 2941.89 / Max: 3518Min: 1900 / Avg: 2941.72 / Max: 3515Min: 1900 / Avg: 2933.61 / Max: 3515

CPU Power Consumption Monitor

OpenBenchmarking.orgWattsCPU Power Consumption MonitorPhoronix Test Suite System MonitoringAVX512_CORE_AMXAVX512_CORE_FP16AVX512_CORE_BF16AVX512_CORE_VNNIAVX512_CORE130260390520650Min: 106.17 / Avg: 565.15 / Max: 756.46Min: 193.64 / Avg: 570.71 / Max: 756.46Min: 105.24 / Avg: 567.97 / Max: 756.24Min: 104.8 / Avg: 550.33 / Max: 744.38Min: 195.36 / Avg: 545.16 / Max: 744.31

oneDNN

MinAvgMaxAVX512_CORE_AMX190027663501AVX512_CORE_FP16190028443506AVX512_CORE_BF16190027913502AVX512_CORE_VNNI190029173502AVX512_CORE190028583501OpenBenchmarking.orgMegahertz, More Is BetteroneDNN 3.0CPU Peak Freq (Highest CPU Core Frequency) Monitor10002000300040005000

MinAvgMaxAVX512_CORE_AMX190029183508AVX512_CORE_FP16190027553504AVX512_CORE_BF16190031103502AVX512_CORE_VNNI190029523506AVX512_CORE190028673510OpenBenchmarking.orgMegahertz, More Is BetteroneDNN 3.0CPU Peak Freq (Highest CPU Core Frequency) Monitor10002000300040005000

MinAvgMaxAVX512_CORE_AMX190029853512AVX512_CORE_FP16190029243510AVX512_CORE_BF16190029333508AVX512_CORE_VNNI190029403515AVX512_CORE190029263513OpenBenchmarking.orgMegahertz, More Is BetteroneDNN 3.0CPU Peak Freq (Highest CPU Core Frequency) Monitor10002000300040005000

MinAvgMaxAVX512_CORE_AMX190028775340AVX512_CORE_FP16190028773502AVX512_CORE_BF16190027663509AVX512_CORE_VNNI190030143512AVX512_CORE190029593508OpenBenchmarking.orgMegahertz, More Is BetteroneDNN 3.0CPU Peak Freq (Highest CPU Core Frequency) Monitor13002600390052006500

MinAvgMaxAVX512_CORE_AMX190029273518AVX512_CORE_FP16190029233516AVX512_CORE_BF16190029233515AVX512_CORE_VNNI190029243514AVX512_CORE190029263515OpenBenchmarking.orgMegahertz, More Is BetteroneDNN 3.0CPU Peak Freq (Highest CPU Core Frequency) Monitor10002000300040005000

MinAvgMaxAVX512_CORE_AMX29.041.447.0AVX512_CORE_FP1633.046.052.0AVX512_CORE_BF1628.043.151.0AVX512_CORE_VNNI29.043.751.0AVX512_CORE30.044.952.0OpenBenchmarking.orgCelsius, Fewer Is BetteroneDNN 3.0CPU Temperature Monitor1530456075

MinAvgMaxAVX512_CORE_AMX35.040.245.0AVX512_CORE_FP1636.040.946.0AVX512_CORE_BF1635.040.245.0AVX512_CORE_VNNI35.039.846.0AVX512_CORE35.040.747.0OpenBenchmarking.orgCelsius, Fewer Is BetteroneDNN 3.0CPU Temperature Monitor1428425670

MinAvgMaxAVX512_CORE_AMX34.043.447.0AVX512_CORE_FP1634.044.548.0AVX512_CORE_BF1634.044.247.0AVX512_CORE_VNNI34.045.150.0AVX512_CORE34.045.249.0OpenBenchmarking.orgCelsius, Fewer Is BetteroneDNN 3.0CPU Temperature Monitor1428425670

MinAvgMaxAVX512_CORE_AMX35.042.546.0AVX512_CORE_FP1636.044.248.0AVX512_CORE_BF1635.043.748.0AVX512_CORE_VNNI36.042.749.0AVX512_CORE35.042.846.0OpenBenchmarking.orgCelsius, Fewer Is BetteroneDNN 3.0CPU Temperature Monitor1428425670

MinAvgMaxAVX512_CORE_AMX37.048.953.0AVX512_CORE_FP1637.049.655.0AVX512_CORE_BF1638.049.754.0AVX512_CORE_VNNI35.048.753.0AVX512_CORE34.048.653.0OpenBenchmarking.orgCelsius, Fewer Is BetteroneDNN 3.0CPU Temperature Monitor1530456075

OpenBenchmarking.orgms, Fewer Is BetteroneDNN 3.0Harness: Deconvolution Batch shapes_1d - Data Type: bf16bf16bf16 - Engine: CPUAVX512_CORE_AMXAVX512_CORE_FP16AVX512_CORE_BF16AVX512_CORE_VNNIAVX512_CORE0.69011.38022.07032.76043.4505SE +/- 0.006207, N = 4SE +/- 0.004682, N = 3SE +/- 0.008260, N = 3SE +/- 0.014891, N = 3SE +/- 0.025853, N = 30.5806462.1707302.1710903.0672103.056120MIN: 0.47MIN: 2.01MIN: 2MIN: 2.85MIN: 2.841. (CXX) g++ options: -O3 -march=native -fopenmp -msse4.1 -fPIC -pie -ldl -lpthread
MinAvgMaxAVX512_CORE_AMX190027663501AVX512_CORE_FP16190028443506AVX512_CORE_BF16190027913502AVX512_CORE_VNNI190029173502AVX512_CORE190028583501OpenBenchmarking.orgMegahertz, More Is BetteroneDNN 3.0CPU Peak Freq (Highest CPU Core Frequency) Monitor10002000300040005000
MinAvgMaxAVX512_CORE_AMX194534637AVX512_CORE_FP16194586721AVX512_CORE_BF16199591722AVX512_CORE_VNNI201596744AVX512_CORE197596744OpenBenchmarking.orgWatts, Fewer Is BetteroneDNN 3.0CPU Power Consumption Monitor20040060080010001. AVX512_CORE_AMX: Approximate power consumption of 13344 Joules per run.2. AVX512_CORE_FP16: Approximate power consumption of 14837 Joules per run.3. AVX512_CORE_BF16: Approximate power consumption of 14775 Joules per run.4. AVX512_CORE_VNNI: Approximate power consumption of 15101 Joules per run.5. AVX512_CORE: Approximate power consumption of 15094 Joules per run.
MinAvgMaxAVX512_CORE_AMX29.041.447.0AVX512_CORE_FP1633.046.052.0AVX512_CORE_BF1628.043.151.0AVX512_CORE_VNNI29.043.751.0AVX512_CORE30.044.952.0OpenBenchmarking.orgCelsius, Fewer Is BetteroneDNN 3.0CPU Temperature Monitor1530456075
OpenBenchmarking.orgms, Fewer Is BetteroneDNN 3.0Harness: Deconvolution Batch shapes_1d - Data Type: bf16bf16bf16 - Engine: CPUAVX512_CORE_AMXAVX512_CORE_FP16AVX512_CORE_BF16AVX512_CORE_VNNIAVX512_CORE246810Min: 0.57 / Avg: 0.58 / Max: 0.6Min: 2.16 / Avg: 2.17 / Max: 2.18Min: 2.16 / Avg: 2.17 / Max: 2.18Min: 3.04 / Avg: 3.07 / Max: 3.09Min: 3.02 / Avg: 3.06 / Max: 3.11. (CXX) g++ options: -O3 -march=native -fopenmp -msse4.1 -fPIC -pie -ldl -lpthread

OpenBenchmarking.orgms, Fewer Is BetteroneDNN 3.0Harness: Deconvolution Batch shapes_3d - Data Type: bf16bf16bf16 - Engine: CPUAVX512_CORE_AMXAVX512_CORE_FP16AVX512_CORE_BF16AVX512_CORE_VNNIAVX512_CORE0.53631.07261.60892.14522.6815SE +/- 0.001544, N = 9SE +/- 0.003634, N = 9SE +/- 0.005254, N = 9SE +/- 0.007894, N = 9SE +/- 0.008552, N = 90.4457791.0451101.0483602.3789202.383610MIN: 0.38MIN: 1.01MIN: 1.01MIN: 2.3MIN: 2.291. (CXX) g++ options: -O3 -march=native -fopenmp -msse4.1 -fPIC -pie -ldl -lpthread
MinAvgMaxAVX512_CORE_AMX190029183508AVX512_CORE_FP16190027553504AVX512_CORE_BF16190031103502AVX512_CORE_VNNI190029523506AVX512_CORE190028673510OpenBenchmarking.orgMegahertz, More Is BetteroneDNN 3.0CPU Peak Freq (Highest CPU Core Frequency) Monitor10002000300040005000
MinAvgMaxAVX512_CORE_AMX194440756AVX512_CORE_FP16196416702AVX512_CORE_BF16195425705AVX512_CORE_VNNI198428722AVX512_CORE200430721OpenBenchmarking.orgWatts, Fewer Is BetteroneDNN 3.0CPU Power Consumption Monitor20040060080010001. AVX512_CORE_AMX: Approximate power consumption of 3131 Joules per run.2. AVX512_CORE_FP16: Approximate power consumption of 2913 Joules per run.3. AVX512_CORE_BF16: Approximate power consumption of 2972 Joules per run.4. AVX512_CORE_VNNI: Approximate power consumption of 2995 Joules per run.5. AVX512_CORE: Approximate power consumption of 3012 Joules per run.
MinAvgMaxAVX512_CORE_AMX35.040.245.0AVX512_CORE_FP1636.040.946.0AVX512_CORE_BF1635.040.245.0AVX512_CORE_VNNI35.039.846.0AVX512_CORE35.040.747.0OpenBenchmarking.orgCelsius, Fewer Is BetteroneDNN 3.0CPU Temperature Monitor1428425670
OpenBenchmarking.orgms, Fewer Is BetteroneDNN 3.0Harness: Deconvolution Batch shapes_3d - Data Type: bf16bf16bf16 - Engine: CPUAVX512_CORE_AMXAVX512_CORE_FP16AVX512_CORE_BF16AVX512_CORE_VNNIAVX512_CORE246810Min: 0.44 / Avg: 0.45 / Max: 0.45Min: 1.03 / Avg: 1.05 / Max: 1.06Min: 1.04 / Avg: 1.05 / Max: 1.08Min: 2.36 / Avg: 2.38 / Max: 2.42Min: 2.36 / Avg: 2.38 / Max: 2.421. (CXX) g++ options: -O3 -march=native -fopenmp -msse4.1 -fPIC -pie -ldl -lpthread

OpenBenchmarking.orgms, Fewer Is BetteroneDNN 3.0Harness: IP Shapes 1D - Data Type: bf16bf16bf16 - Engine: CPUAVX512_CORE_AMXAVX512_CORE_FP16AVX512_CORE_BF16AVX512_CORE_VNNIAVX512_CORE246810SE +/- 0.11275, N = 15SE +/- 0.27236, N = 15SE +/- 0.33773, N = 15SE +/- 0.31537, N = 15SE +/- 0.26087, N = 155.737176.483666.588416.824336.64816MIN: 3.75MIN: 3.65MIN: 3.66MIN: 4.26MIN: 3.811. (CXX) g++ options: -O3 -march=native -fopenmp -msse4.1 -fPIC -pie -ldl -lpthread
MinAvgMaxAVX512_CORE_AMX190029853512AVX512_CORE_FP16190029243510AVX512_CORE_BF16190029333508AVX512_CORE_VNNI190029403515AVX512_CORE190029263513OpenBenchmarking.orgMegahertz, More Is BetteroneDNN 3.0CPU Peak Freq (Highest CPU Core Frequency) Monitor10002000300040005000
MinAvgMaxAVX512_CORE_AMX106493604AVX512_CORE_FP16195508657AVX512_CORE_BF16197510652AVX512_CORE_VNNI193534705AVX512_CORE200536710OpenBenchmarking.orgWatts, Fewer Is BetteroneDNN 3.0CPU Power Consumption Monitor20040060080010001. AVX512_CORE_AMX: Approximate power consumption of 9540 Joules per run.2. AVX512_CORE_FP16: Approximate power consumption of 9682 Joules per run.3. AVX512_CORE_BF16: Approximate power consumption of 9766 Joules per run.4. AVX512_CORE_VNNI: Approximate power consumption of 10219 Joules per run.5. AVX512_CORE: Approximate power consumption of 10249 Joules per run.
MinAvgMaxAVX512_CORE_AMX34.043.447.0AVX512_CORE_FP1634.044.548.0AVX512_CORE_BF1634.044.247.0AVX512_CORE_VNNI34.045.150.0AVX512_CORE34.045.249.0OpenBenchmarking.orgCelsius, Fewer Is BetteroneDNN 3.0CPU Temperature Monitor1428425670
OpenBenchmarking.orgms, Fewer Is BetteroneDNN 3.0Harness: IP Shapes 1D - Data Type: bf16bf16bf16 - Engine: CPUAVX512_CORE_AMXAVX512_CORE_FP16AVX512_CORE_BF16AVX512_CORE_VNNIAVX512_CORE3691215Min: 4.79 / Avg: 5.74 / Max: 6.62Min: 5.14 / Avg: 6.48 / Max: 8.68Min: 5.04 / Avg: 6.59 / Max: 9.49Min: 5.53 / Avg: 6.82 / Max: 9.06Min: 5.2 / Avg: 6.65 / Max: 9.251. (CXX) g++ options: -O3 -march=native -fopenmp -msse4.1 -fPIC -pie -ldl -lpthread

OpenBenchmarking.orgms, Fewer Is BetteroneDNN 3.0Harness: Matrix Multiply Batch Shapes Transformer - Data Type: bf16bf16bf16 - Engine: CPUAVX512_CORE_AMXAVX512_CORE_FP16AVX512_CORE_BF16AVX512_CORE_VNNIAVX512_CORE48121620SE +/- 0.002541, N = 4SE +/- 0.001848, N = 4SE +/- 0.003559, N = 4SE +/- 0.135276, N = 4SE +/- 0.370395, N = 150.3299610.3837850.38663616.57510016.458900MIN: 15.34MIN: 12.811. (CXX) g++ options: -O3 -march=native -fopenmp -msse4.1 -fPIC -pie -ldl -lpthread
MinAvgMaxAVX512_CORE_AMX190028775340AVX512_CORE_FP16190028773502AVX512_CORE_BF16190027663509AVX512_CORE_VNNI190030143512AVX512_CORE190029593508OpenBenchmarking.orgMegahertz, More Is BetteroneDNN 3.0CPU Peak Freq (Highest CPU Core Frequency) Monitor13002600390052006500
MinAvgMaxAVX512_CORE_AMX197485593AVX512_CORE_FP16199512647AVX512_CORE_BF16196507647AVX512_CORE_VNNI199486601AVX512_CORE197487609OpenBenchmarking.orgWatts, Fewer Is BetteroneDNN 3.0CPU Power Consumption Monitor20040060080010001. AVX512_CORE_AMX: Approximate power consumption of 7883 Joules per run.2. AVX512_CORE_FP16: Approximate power consumption of 8313 Joules per run.3. AVX512_CORE_BF16: Approximate power consumption of 8107 Joules per run.4. AVX512_CORE_VNNI: Approximate power consumption of 7897 Joules per run.5. AVX512_CORE: Approximate power consumption of 7864 Joules per run.
MinAvgMaxAVX512_CORE_AMX35.042.546.0AVX512_CORE_FP1636.044.248.0AVX512_CORE_BF1635.043.748.0AVX512_CORE_VNNI36.042.749.0AVX512_CORE35.042.846.0OpenBenchmarking.orgCelsius, Fewer Is BetteroneDNN 3.0CPU Temperature Monitor1428425670
OpenBenchmarking.orgms, Fewer Is BetteroneDNN 3.0Harness: Matrix Multiply Batch Shapes Transformer - Data Type: bf16bf16bf16 - Engine: CPUAVX512_CORE_AMXAVX512_CORE_FP16AVX512_CORE_BF16AVX512_CORE_VNNIAVX512_CORE48121620Min: 0.33 / Avg: 0.33 / Max: 0.34Min: 0.38 / Avg: 0.38 / Max: 0.39Min: 0.38 / Avg: 0.39 / Max: 0.39Min: 16.4 / Avg: 16.58 / Max: 16.97Min: 13.77 / Avg: 16.46 / Max: 18.641. (CXX) g++ options: -O3 -march=native -fopenmp -msse4.1 -fPIC -pie -ldl -lpthread

OpenBenchmarking.orgms, Fewer Is BetteroneDNN 3.0Harness: Recurrent Neural Network Inference - Data Type: bf16bf16bf16 - Engine: CPUAVX512_CORE_AMXAVX512_CORE_FP16AVX512_CORE_BF16AVX512_CORE_VNNIAVX512_CORE2004006008001000SE +/- 9.48, N = 4SE +/- 12.20, N = 12SE +/- 10.05, N = 15SE +/- 13.04, N = 15SE +/- 10.59, N = 15850.35875.28864.46862.42856.90MIN: 797.61MIN: 777.12MIN: 758.74MIN: 767.35MIN: 741.531. (CXX) g++ options: -O3 -march=native -fopenmp -msse4.1 -fPIC -pie -ldl -lpthread
MinAvgMaxAVX512_CORE_AMX190029273518AVX512_CORE_FP16190029233516AVX512_CORE_BF16190029233515AVX512_CORE_VNNI190029243514AVX512_CORE190029263515OpenBenchmarking.orgMegahertz, More Is BetteroneDNN 3.0CPU Peak Freq (Highest CPU Core Frequency) Monitor10002000300040005000
MinAvgMaxAVX512_CORE_AMX196576732AVX512_CORE_FP16194576732AVX512_CORE_BF16197577733AVX512_CORE_VNNI197576733AVX512_CORE201576734OpenBenchmarking.orgWatts, Fewer Is BetteroneDNN 3.0CPU Power Consumption Monitor20040060080010001. AVX512_CORE_AMX: Approximate power consumption of 48077 Joules per run.2. AVX512_CORE_FP16: Approximate power consumption of 48220 Joules per run.3. AVX512_CORE_BF16: Approximate power consumption of 48195 Joules per run.4. AVX512_CORE_VNNI: Approximate power consumption of 48162 Joules per run.5. AVX512_CORE: Approximate power consumption of 48127 Joules per run.
MinAvgMaxAVX512_CORE_AMX37.048.953.0AVX512_CORE_FP1637.049.655.0AVX512_CORE_BF1638.049.754.0AVX512_CORE_VNNI35.048.753.0AVX512_CORE34.048.653.0OpenBenchmarking.orgCelsius, Fewer Is BetteroneDNN 3.0CPU Temperature Monitor1530456075
OpenBenchmarking.orgms, Fewer Is BetteroneDNN 3.0Harness: Recurrent Neural Network Inference - Data Type: bf16bf16bf16 - Engine: CPUAVX512_CORE_AMXAVX512_CORE_FP16AVX512_CORE_BF16AVX512_CORE_VNNIAVX512_CORE150300450600750Min: 822.36 / Avg: 850.35 / Max: 864.19Min: 813.47 / Avg: 875.28 / Max: 963.03Min: 781.77 / Avg: 864.46 / Max: 943.67Min: 786.04 / Avg: 862.42 / Max: 959.13Min: 765.16 / Avg: 856.9 / Max: 905.911. (CXX) g++ options: -O3 -march=native -fopenmp -msse4.1 -fPIC -pie -ldl -lpthread

18 Results Shown

CPU Temperature Monitor:
  Phoronix Test Suite System Monitoring:
    Celsius
    Megahertz
    Watts
  CPU Peak Freq (Highest CPU Core Frequency) Monitor:
    Megahertz
    Megahertz
    Megahertz
    Megahertz
    Megahertz
  CPU Temp Monitor:
    Celsius
    Celsius
    Celsius
    Celsius
    Celsius
oneDNN:
  Deconvolution Batch shapes_1d - bf16bf16bf16 - CPU
  Deconvolution Batch shapes_3d - bf16bf16bf16 - CPU
  IP Shapes 1D - bf16bf16bf16 - CPU
  Matrix Multiply Batch Shapes Transformer - bf16bf16bf16 - CPU
  Recurrent Neural Network Inference - bf16bf16bf16 - CPU

AVX512_CORE_AMX

Kernel Notes: Transparent Huge Pages: madvise
Compiler Notes: --build=x86_64-linux-gnu --disable-vtable-verify --disable-werror --enable-bootstrap --enable-cet --enable-checking=release --enable-clocale=gnu --enable-default-pie --enable-gnu-unique-object --enable-languages=c,ada,c++,go,brig,d,fortran,objc,obj-c++,m2 --enable-libphobos-checking=release --enable-libstdcxx-debug --enable-libstdcxx-time=yes --enable-link-serialization=2 --enable-multiarch --enable-multilib --enable-nls --enable-objc-gc=auto --enable-offload-targets=nvptx-none=/build/gcc-11-xKiWfi/gcc-11-11.3.0/debian/tmp-nvptx/usr,amdgcn-amdhsa=/build/gcc-11-xKiWfi/gcc-11-11.3.0/debian/tmp-gcn/usr --enable-plugin --enable-shared --enable-threads=posix --host=x86_64-linux-gnu --program-prefix=x86_64-linux-gnu- --target=x86_64-linux-gnu --with-abi=m64 --with-arch-32=i686 --with-build-config=bootstrap-lto-lean --with-default-libstdcxx-abi=new --with-gcc-major-version-only --with-multilib-list=m32,m64,mx32 --with-target-system-zlib=auto --with-tune=generic --without-cuda-driver -v
Processor Notes: Scaling Governor: intel_pstate performance (EPP: performance) - CPU Microcode: 0x2b0000c0
Security Notes: itlb_multihit: Not affected + l1tf: Not affected + mds: Not affected + meltdown: Not affected + mmio_stale_data: Not affected + retbleed: Not affected + spec_store_bypass: Mitigation of SSB disabled via prctl + spectre_v1: Mitigation of usercopy/swapgs barriers and __user pointer sanitization + spectre_v2: Mitigation of Enhanced IBRS IBPB: conditional RSB filling PBRSB-eIBRS: SW sequence + srbds: Not affected + tsx_async_abort: Not affected

Testing initiated at 15 January 2023 05:14 by user root.

AVX512_CORE_FP16

Kernel Notes: Transparent Huge Pages: madvise
Compiler Notes: --build=x86_64-linux-gnu --disable-vtable-verify --disable-werror --enable-bootstrap --enable-cet --enable-checking=release --enable-clocale=gnu --enable-default-pie --enable-gnu-unique-object --enable-languages=c,ada,c++,go,brig,d,fortran,objc,obj-c++,m2 --enable-libphobos-checking=release --enable-libstdcxx-debug --enable-libstdcxx-time=yes --enable-link-serialization=2 --enable-multiarch --enable-multilib --enable-nls --enable-objc-gc=auto --enable-offload-targets=nvptx-none=/build/gcc-11-xKiWfi/gcc-11-11.3.0/debian/tmp-nvptx/usr,amdgcn-amdhsa=/build/gcc-11-xKiWfi/gcc-11-11.3.0/debian/tmp-gcn/usr --enable-plugin --enable-shared --enable-threads=posix --host=x86_64-linux-gnu --program-prefix=x86_64-linux-gnu- --target=x86_64-linux-gnu --with-abi=m64 --with-arch-32=i686 --with-build-config=bootstrap-lto-lean --with-default-libstdcxx-abi=new --with-gcc-major-version-only --with-multilib-list=m32,m64,mx32 --with-target-system-zlib=auto --with-tune=generic --without-cuda-driver -v
Processor Notes: Scaling Governor: intel_pstate performance (EPP: performance) - CPU Microcode: 0x2b0000c0
Security Notes: itlb_multihit: Not affected + l1tf: Not affected + mds: Not affected + meltdown: Not affected + mmio_stale_data: Not affected + retbleed: Not affected + spec_store_bypass: Mitigation of SSB disabled via prctl + spectre_v1: Mitigation of usercopy/swapgs barriers and __user pointer sanitization + spectre_v2: Mitigation of Enhanced IBRS IBPB: conditional RSB filling PBRSB-eIBRS: SW sequence + srbds: Not affected + tsx_async_abort: Not affected

Testing initiated at 15 January 2023 06:46 by user root.

AVX512_CORE_BF16

Kernel Notes: Transparent Huge Pages: madvise
Compiler Notes: --build=x86_64-linux-gnu --disable-vtable-verify --disable-werror --enable-bootstrap --enable-cet --enable-checking=release --enable-clocale=gnu --enable-default-pie --enable-gnu-unique-object --enable-languages=c,ada,c++,go,brig,d,fortran,objc,obj-c++,m2 --enable-libphobos-checking=release --enable-libstdcxx-debug --enable-libstdcxx-time=yes --enable-link-serialization=2 --enable-multiarch --enable-multilib --enable-nls --enable-objc-gc=auto --enable-offload-targets=nvptx-none=/build/gcc-11-xKiWfi/gcc-11-11.3.0/debian/tmp-nvptx/usr,amdgcn-amdhsa=/build/gcc-11-xKiWfi/gcc-11-11.3.0/debian/tmp-gcn/usr --enable-plugin --enable-shared --enable-threads=posix --host=x86_64-linux-gnu --program-prefix=x86_64-linux-gnu- --target=x86_64-linux-gnu --with-abi=m64 --with-arch-32=i686 --with-build-config=bootstrap-lto-lean --with-default-libstdcxx-abi=new --with-gcc-major-version-only --with-multilib-list=m32,m64,mx32 --with-target-system-zlib=auto --with-tune=generic --without-cuda-driver -v
Processor Notes: Scaling Governor: intel_pstate performance (EPP: performance) - CPU Microcode: 0x2b0000c0
Security Notes: itlb_multihit: Not affected + l1tf: Not affected + mds: Not affected + meltdown: Not affected + mmio_stale_data: Not affected + retbleed: Not affected + spec_store_bypass: Mitigation of SSB disabled via prctl + spectre_v1: Mitigation of usercopy/swapgs barriers and __user pointer sanitization + spectre_v2: Mitigation of Enhanced IBRS IBPB: conditional RSB filling PBRSB-eIBRS: SW sequence + srbds: Not affected + tsx_async_abort: Not affected

Testing initiated at 15 January 2023 09:23 by user root.

AVX512_CORE_VNNI

Kernel Notes: Transparent Huge Pages: madvise
Compiler Notes: --build=x86_64-linux-gnu --disable-vtable-verify --disable-werror --enable-bootstrap --enable-cet --enable-checking=release --enable-clocale=gnu --enable-default-pie --enable-gnu-unique-object --enable-languages=c,ada,c++,go,brig,d,fortran,objc,obj-c++,m2 --enable-libphobos-checking=release --enable-libstdcxx-debug --enable-libstdcxx-time=yes --enable-link-serialization=2 --enable-multiarch --enable-multilib --enable-nls --enable-objc-gc=auto --enable-offload-targets=nvptx-none=/build/gcc-11-xKiWfi/gcc-11-11.3.0/debian/tmp-nvptx/usr,amdgcn-amdhsa=/build/gcc-11-xKiWfi/gcc-11-11.3.0/debian/tmp-gcn/usr --enable-plugin --enable-shared --enable-threads=posix --host=x86_64-linux-gnu --program-prefix=x86_64-linux-gnu- --target=x86_64-linux-gnu --with-abi=m64 --with-arch-32=i686 --with-build-config=bootstrap-lto-lean --with-default-libstdcxx-abi=new --with-gcc-major-version-only --with-multilib-list=m32,m64,mx32 --with-target-system-zlib=auto --with-tune=generic --without-cuda-driver -v
Processor Notes: Scaling Governor: intel_pstate performance (EPP: performance) - CPU Microcode: 0x2b0000c0
Security Notes: itlb_multihit: Not affected + l1tf: Not affected + mds: Not affected + meltdown: Not affected + mmio_stale_data: Not affected + retbleed: Not affected + spec_store_bypass: Mitigation of SSB disabled via prctl + spectre_v1: Mitigation of usercopy/swapgs barriers and __user pointer sanitization + spectre_v2: Mitigation of Enhanced IBRS IBPB: conditional RSB filling PBRSB-eIBRS: SW sequence + srbds: Not affected + tsx_async_abort: Not affected

Testing initiated at 15 January 2023 11:21 by user root.

AVX512_CORE

Processor: 2 x Intel Xeon Platinum 8490H @ 3.50GHz (120 Cores / 240 Threads), Motherboard: Quanta Cloud S6Q-MB-MPS (3A10.uh BIOS), Chipset: Intel Device 1bce, Memory: 16 x 64 GB 4800MT/s Samsung M321R8GA0BB0-CQKEG, Disk: 2 x 1920GB SAMSUNG MZWLJ1T9HBJR-00007 + 3841GB Micron_9300_MTFDHAL3T8TDP + 960GB INTEL SSDSC2KG96, Graphics: ASPEED, Monitor: VGA HDMI, Network: 4 x Intel E810-C for QSFP + 2 x Intel X710 for 10GBASE-T

OS: Ubuntu 22.04, Kernel: 6.1.4-060104-generic (x86_64), Desktop: GNOME Shell 42.2, Display Server: X Server 1.21.1.3, Vulkan: 1.2.204, Compiler: GCC 11.3.0 + Clang 14.0.0-1ubuntu1, File-System: ext4, Screen Resolution: 1920x1080

Kernel Notes: Transparent Huge Pages: madvise
Compiler Notes: --build=x86_64-linux-gnu --disable-vtable-verify --disable-werror --enable-bootstrap --enable-cet --enable-checking=release --enable-clocale=gnu --enable-default-pie --enable-gnu-unique-object --enable-languages=c,ada,c++,go,brig,d,fortran,objc,obj-c++,m2 --enable-libphobos-checking=release --enable-libstdcxx-debug --enable-libstdcxx-time=yes --enable-link-serialization=2 --enable-multiarch --enable-multilib --enable-nls --enable-objc-gc=auto --enable-offload-targets=nvptx-none=/build/gcc-11-xKiWfi/gcc-11-11.3.0/debian/tmp-nvptx/usr,amdgcn-amdhsa=/build/gcc-11-xKiWfi/gcc-11-11.3.0/debian/tmp-gcn/usr --enable-plugin --enable-shared --enable-threads=posix --host=x86_64-linux-gnu --program-prefix=x86_64-linux-gnu- --target=x86_64-linux-gnu --with-abi=m64 --with-arch-32=i686 --with-build-config=bootstrap-lto-lean --with-default-libstdcxx-abi=new --with-gcc-major-version-only --with-multilib-list=m32,m64,mx32 --with-target-system-zlib=auto --with-tune=generic --without-cuda-driver -v
Processor Notes: Scaling Governor: intel_pstate performance (EPP: performance) - CPU Microcode: 0x2b0000c0
Security Notes: itlb_multihit: Not affected + l1tf: Not affected + mds: Not affected + meltdown: Not affected + mmio_stale_data: Not affected + retbleed: Not affected + spec_store_bypass: Mitigation of SSB disabled via prctl + spectre_v1: Mitigation of usercopy/swapgs barriers and __user pointer sanitization + spectre_v2: Mitigation of Enhanced IBRS IBPB: conditional RSB filling PBRSB-eIBRS: SW sequence + srbds: Not affected + tsx_async_abort: Not affected

Testing initiated at 15 January 2023 11:53 by user root.