AMD EPYC 4th Gen AVX-512 Comparison

AMD EPYC 9654 Genoa AVX-512 benchmark comparison by Michael Larabel for a future article.

Compare your own system(s) to this result file with the Phoronix Test Suite by running the command: phoronix-test-suite benchmark 2212195-NE-AVXCOMPAR69
Jump To Table - Results

View

Do Not Show Noisy Results
Do Not Show Results With Incomplete Data
Do Not Show Results With Little Change/Spread
List Notable Results
Show Result Confidence Charts
Allow Limiting Results To Certain Suite(s)

Statistics

Show Overall Harmonic Mean(s)
Show Overall Geometric Mean
Show Wins / Losses Counts (Pie Chart)
Normalize Results
Remove Outliers Before Calculating Averages

Graph Settings

Force Line Graphs Where Applicable
Convert To Scalar Where Applicable
Prefer Vertical Bar Graphs
No Box Plots
On Line Graphs With Missing Data, Connect The Line Gaps

Multi-Way Comparison

Condense Multi-Option Tests Into Single Result Graphs
Condense Test Profiles With Multiple Version Results Into Single Result Graphs

Table

Show Detailed System Result Table

Run Management

Highlight
Result
Toggle/Hide
Result
Result
Identifier
Performance Per
Dollar
Date
Run
  Test
  Duration
AVX-512 On
December 18 2022
  20 Hours, 2 Minutes
AVX-512 Off
December 18 2022
  15 Hours, 29 Minutes
Invert Behavior (Only Show Selected Data)
  17 Hours, 45 Minutes
Only show results matching title/arguments (delimit multiple options with a comma):
Do not show results matching title/arguments (delimit multiple options with a comma):


AMD EPYC 4th Gen AVX-512 ComparisonOpenBenchmarking.orgPhoronix Test Suite2 x AMD EPYC 9654 96-Core @ 2.40GHz (192 Cores / 384 Threads)AMD Titanite_4G (RTI1002E BIOS)AMD Device 14a41520GB800GB INTEL SSDPF21Q800GBASPEEDVGA HDMIBroadcom NetXtreme BCM5720 PCIeUbuntu 22.106.1.0-phx (x86_64)GNOME Shell 43.0X Server 1.21.1.41.3.224GCC 12.2.0 + Clang 15.0.2-1ext41920x1080ProcessorMotherboardChipsetMemoryDiskGraphicsMonitorNetworkOSKernelDesktopDisplay ServerVulkanCompilerFile-SystemScreen ResolutionAMD EPYC 4th Gen AVX-512 Comparison BenchmarksSystem Logs- Transparent Huge Pages: madvise- AVX-512 On: CXXFLAGS="-O3 -march=native -mavx512f -mavx512cd -mavx512vl -mavx512bw -mavx512dq -mavx512ifma -mavx512vbmi -mprefer-vector-width=512" CFLAGS="-O3 -march=native -mavx512f -mavx512cd -mavx512vl -mavx512bw -mavx512dq -mavx512ifma -mavx512vbmi -mprefer-vector-width=512" - AVX-512 Off: CXXFLAGS="-O3 -march=native -mno-avx512f" CFLAGS="-O3 -march=native -mno-avx512f" - --build=x86_64-linux-gnu --disable-vtable-verify --disable-werror --enable-cet --enable-checking=release --enable-clocale=gnu --enable-default-pie --enable-gnu-unique-object --enable-languages=c,ada,c++,go,d,fortran,objc,obj-c++,m2 --enable-libphobos-checking=release --enable-libstdcxx-debug --enable-libstdcxx-time=yes --enable-multiarch --enable-multilib --enable-nls --enable-objc-gc=auto --enable-offload-defaulted --enable-offload-targets=nvptx-none=/build/gcc-12-U8K4Qv/gcc-12-12.2.0/debian/tmp-nvptx/usr,amdgcn-amdhsa=/build/gcc-12-U8K4Qv/gcc-12-12.2.0/debian/tmp-gcn/usr --enable-plugin --enable-shared --enable-threads=posix --host=x86_64-linux-gnu --program-prefix=x86_64-linux-gnu- --target=x86_64-linux-gnu --with-abi=m64 --with-arch-32=i686 --with-default-libstdcxx-abi=new --with-gcc-major-version-only --with-multilib-list=m32,m64,mx32 --with-target-system-zlib=auto --with-tune=generic --without-cuda-driver -v - Scaling Governor: acpi-cpufreq performance (Boost: Enabled) - CPU Microcode: 0xa10110d - Python 3.10.7- itlb_multihit: Not affected + l1tf: Not affected + mds: Not affected + meltdown: Not affected + mmio_stale_data: Not affected + retbleed: Not affected + spec_store_bypass: Mitigation of SSB disabled via prctl + spectre_v1: Mitigation of usercopy/swapgs barriers and __user pointer sanitization + spectre_v2: Mitigation of Retpolines IBPB: conditional IBRS_FW STIBP: always-on RSB filling PBRSB-eIBRS: Not affected + srbds: Not affected + tsx_async_abort: Not affected

AVX-512 On vs. AVX-512 Off ComparisonPhoronix Test SuiteBaseline+46.4%+46.4%+92.8%+92.8%+139.2%+139.2%50.5%37.6%3.6%CPU - 16 - AlexNet185.4%R.N.N.T - bf16bf16bf16 - CPU155.1%D.T.S153.4%R.N.N.T - f32 - CPU152.5%W.P.D.F - CPU143.4%W.P.D.F - CPU143%F.D.F - CPU132.2%F.D.F - CPU131.9%A.G.R.R.0.F - CPU120%LBC, LBRY Credits117%Device AI Score115.1%M.T.E.T.D.F - CPU114.8%M.T.E.T.D.F - CPU114.7%W.P.D.F.I - CPU104.7%W.P.D.F.I - CPU104.5%F.D.F.I - CPU104.2%F.D.F.I - CPU103.9%V.D.F - CPU97.1%V.D.F - CPU97%CPU - 16 - GoogLeNet93.3%D.I.S92.8%V.D.F.I - CPU84.7%V.D.F.I - CPU84.6%D.B.s - f32 - CPU78.2%P.V.B.D.F - CPU76.5%P.V.B.D.F - CPU76.4%CPU - 16 - ResNet-5073.7%gravity_spheres_volume/dim_512/scivis/real_time71.4%Q.S.2.P70.9%P.D.F - CPU70%P.D.F - CPU69.5%P.D.F - CPU68.4%P.D.F - CPU67.7%gravity_spheres_volume/dim_512/ao/real_time67.4%scrypt62.9%A.G.R.R.0.F.I - CPU58.3%resnet-v2-5056.1%N.Q.A.B.b.u.S.1.P - A.M.S55.2%N.Q.A.B.b.u.S.1.P - A.M.S55.2%inception-v3Garlicoin46.5%OpenMP - BM144.1%OpenMP - BM144.1%N.Q.A.B.b.u.S.1.P - S.S.S41.5%N.Q.A.B.b.u.S.1.P - S.S.S41.5%C.C.R.5.I - A.M.S38.5%C.C.R.5.I - A.M.S38.4%R.N.N.I - bf16bf16bf16 - CPUA.G.R.R.0.F - CPU37.4%Skeincoin36.7%OpenMP - BM235.4%OpenMP - BM235.4%gravity_spheres_volume/dim_512/pathtracer/real_time33.8%x25x33.1%DistinctUserID30.7%PartialTweets30.4%C.D.Y.C - A.M.S24.1%C.D.Y.C - A.M.S24%Kostya23.3%2 - 1080p - 32 - Path Tracer23%2 - 4K - 16 - Path Tracer22.8%2 - 1080p - 16 - Path Tracer22.7%1 - 1080p - 32 - Path Tracer22.4%1 - 1080p - 1 - Path Tracer22.3%2 - 4K - 32 - Path Tracer22.2%1 - 4K - 1 - Path Tracer22.2%1 - 4K - 32 - Path Tracer22.1%1 - 4K - 16 - Path Tracer22.1%1 - 1080p - 16 - Path Tracer22%3 - 1080p - 16 - Path Tracer21.7%3 - 1080p - 32 - Path Tracer21.6%TopTweet21.2%3 - 4K - 1 - Path Tracer21.2%LargeRand21.2%N.T.C.B.b.u.S - A.M.S21%3 - 4K - 16 - Path Tracer21%N.T.C.B.b.u.S - A.M.S20.9%3 - 1080p - 1 - Path Tracer20.9%3 - 4K - 32 - Path Tracer20.7%Pathtracer ISPC - Asian Dragon20.4%Pathtracer ISPC - Asian Dragon Obj20.2%C.D.Y.C - S.S.S20.1%C.D.Y.C - S.S.S20.1%Pathtracer ISPC - Crown19.8%2 - 4K - 1 - Path Tracer19.5%2 - 1080p - 1 - Path Tracer19.5%N.T.C.D.m - A.M.S19%d.M.M.S - Execution Time19%N.T.C.D.m - A.M.S19%OpenMP - Points2Image18.7%super-resolution-10 - CPU - Standard17.7%N.T.C.B.b.u.S - S.S.S15.6%N.T.C.B.b.u.S - S.S.S15.6%vklBenchmark ISPC15.3%CPU - vision_transformer15%ArcFace ResNet-100 - CPU - Standard14.5%A.G.R.R.0.F.I - CPU12.8%N.T.C.B.b.u.c - S.S.S12.8%N.T.C.B.b.u.c - S.S.S12.8%N.D.C.o.b.u.o.I - S.S.S12.2%N.D.C.o.b.u.o.I - S.S.S12.2%CPU - blazeface12.1%Eigen11.4%CPU - regnety_400m9.2%F.x.A9.1%CPU - efficientnet-b08.9%fcn-resnet101-11 - CPU - Standard8.4%BLAS7.8%Fayalite-FIST6.7%SqueezeNetV1.06.6%d.M.M.S - Mesh Time6.3%N.T.C.D.m - S.S.S5.4%N.T.C.D.m - S.S.S5.4%Preset 12 - Bosphorus 4K5%4.9%JPEG - 904.6%CPU - googlenet4.6%PNG - 904.5%CPU - mnasnet4.5%B.C4.3%C.B.S.A - f32 - CPU4.2%RTLightmap.hdr.4096x4096bertsquad-12 - CPU - Standard3.4%Windowed Gaussian3.3%CPU - FastestDet3%Preset 13 - Bosphorus 4K2.8%JPEG - 1002.8%OpenMP - NDT Mapping2.7%TensorFlowoneDNNAI Benchmark AlphaoneDNNOpenVINOOpenVINOOpenVINOOpenVINOOpenVINOCpuminer-OptAI Benchmark AlphaOpenVINOOpenVINOOpenVINOOpenVINOOpenVINOOpenVINOOpenVINOOpenVINOTensorFlowAI Benchmark AlphaOpenVINOOpenVINOoneDNNOpenVINOOpenVINOTensorFlowOSPRayCpuminer-OptOpenVINOOpenVINOOpenVINOOpenVINOOSPRayCpuminer-OptOpenVINOMobile Neural NetworkNeural Magic DeepSparseNeural Magic DeepSparseMobile Neural NetworkCpuminer-OptminiBUDEminiBUDENeural Magic DeepSparseNeural Magic DeepSparseNeural Magic DeepSparseNeural Magic DeepSparseoneDNNOpenVINOCpuminer-OptminiBUDEminiBUDEOSPRayCpuminer-OptsimdjsonsimdjsonNeural Magic DeepSparseNeural Magic DeepSparsesimdjsonOSPRay StudioOSPRay StudioOSPRay StudioOSPRay StudioOSPRay StudioOSPRay StudioOSPRay StudioOSPRay StudioOSPRay StudioOSPRay StudioOSPRay StudioOSPRay StudiosimdjsonOSPRay StudiosimdjsonNeural Magic DeepSparseOSPRay StudioNeural Magic DeepSparseOSPRay StudioOSPRay StudioEmbreeEmbreeNeural Magic DeepSparseNeural Magic DeepSparseEmbreeOSPRay StudioOSPRay StudioNeural Magic DeepSparseOpenFOAMNeural Magic DeepSparseDarmstadt Automotive Parallel Heterogeneous SuiteONNX RuntimeNeural Magic DeepSparseNeural Magic DeepSparseOpenVKLNCNNONNX RuntimeOpenVINONeural Magic DeepSparseNeural Magic DeepSparseNeural Magic DeepSparseNeural Magic DeepSparseNCNNLeelaChessZeroNCNNSMHasherNCNNONNX RuntimeLeelaChessZeroCP2K Molecular DynamicsMobile Neural NetworkOpenFOAMNeural Magic DeepSparseNeural Magic DeepSparseSVT-AV1Numpy BenchmarkJPEG XL libjxlNCNNJPEG XL libjxlNCNNNumenta Anomaly BenchmarkoneDNNIntel Open Image DenoiseONNX RuntimeNumenta Anomaly BenchmarkNCNNSVT-AV1JPEG XL libjxlDarmstadt Automotive Parallel Heterogeneous SuiteAVX-512 OnAVX-512 Off

AMD EPYC 4th Gen AVX-512 Comparisonai-benchmark: Device Inference Scoreai-benchmark: Device Training Scoreai-benchmark: Device AI Scoredeepsparse: NLP Text Classification, BERT base uncased SST2 - Synchronous Single-Streamdeepsparse: NLP Text Classification, BERT base uncased SST2 - Synchronous Single-Streamdeepsparse: NLP Text Classification, BERT base uncased SST2 - Asynchronous Multi-Streamdeepsparse: NLP Text Classification, BERT base uncased SST2 - Asynchronous Multi-Streamdeepsparse: NLP Text Classification, DistilBERT mnli - Synchronous Single-Streamdeepsparse: NLP Text Classification, DistilBERT mnli - Synchronous Single-Streamdeepsparse: NLP Text Classification, DistilBERT mnli - Asynchronous Multi-Streamdeepsparse: NLP Text Classification, DistilBERT mnli - Asynchronous Multi-Streamdeepsparse: CV Classification, ResNet-50 ImageNet - Asynchronous Multi-Streamdeepsparse: CV Classification, ResNet-50 ImageNet - Asynchronous Multi-Streamdeepsparse: NLP Token Classification, BERT base uncased conll2003 - Synchronous Single-Streamdeepsparse: NLP Token Classification, BERT base uncased conll2003 - Synchronous Single-Streamdeepsparse: NLP Question Answering, BERT base uncased SQuaD 12layer Pruned90 - Synchronous Single-Streamdeepsparse: NLP Question Answering, BERT base uncased SQuaD 12layer Pruned90 - Synchronous Single-Streamdeepsparse: NLP Question Answering, BERT base uncased SQuaD 12layer Pruned90 - Asynchronous Multi-Streamdeepsparse: NLP Question Answering, BERT base uncased SQuaD 12layer Pruned90 - Asynchronous Multi-Streamdeepsparse: CV Detection,YOLOv5s COCO - Synchronous Single-Streamdeepsparse: CV Detection,YOLOv5s COCO - Synchronous Single-Streamdeepsparse: CV Detection,YOLOv5s COCO - Asynchronous Multi-Streamdeepsparse: CV Detection,YOLOv5s COCO - Asynchronous Multi-Streamdeepsparse: NLP Document Classification, oBERT base uncased on IMDB - Synchronous Single-Streamdeepsparse: NLP Document Classification, oBERT base uncased on IMDB - Synchronous Single-Streamtensorflow: CPU - 16 - ResNet-50tensorflow: CPU - 16 - AlexNettensorflow: CPU - 16 - GoogLeNetlczero: BLASlczero: Eigenembree: Pathtracer ISPC - Asian Dragonembree: Pathtracer ISPC - Asian Dragon Objembree: Pathtracer ISPC - Crownopenvkl: vklBenchmark ISPCospray: gravity_spheres_volume/dim_512/ao/real_timeospray: gravity_spheres_volume/dim_512/scivis/real_timeospray: gravity_spheres_volume/dim_512/pathtracer/real_timeospray-studio: 1 - 1080p - 1 - Path Tracerospray-studio: 1 - 1080p - 16 - Path Tracerospray-studio: 1 - 1080p - 32 - Path Tracerospray-studio: 1 - 4K - 1 - Path Tracerospray-studio: 1 - 4K - 16 - Path Tracerospray-studio: 1 - 4K - 32 - Path Tracerospray-studio: 2 - 1080p - 1 - Path Tracerospray-studio: 2 - 1080p - 16 - Path Tracerospray-studio: 2 - 1080p - 32 - Path Tracerospray-studio: 2 - 4K - 1 - Path Tracerospray-studio: 2 - 4K - 16 - Path Tracerospray-studio: 2 - 4K - 32 - Path Tracerospray-studio: 3 - 1080p - 1 - Path Tracerospray-studio: 3 - 1080p - 16 - Path Tracerospray-studio: 3 - 1080p - 32 - Path Tracerospray-studio: 3 - 4K - 1 - Path Tracerospray-studio: 3 - 4K - 16 - Path Tracerospray-studio: 3 - 4K - 32 - Path Traceronednn: Convolution Batch Shapes Auto - f32 - CPUonednn: Deconvolution Batch shapes_1d - f32 - CPUonednn: Deconvolution Batch shapes_3d - f32 - CPUonednn: Recurrent Neural Network Training - f32 - CPUonednn: Recurrent Neural Network Training - bf16bf16bf16 - CPUonednn: Recurrent Neural Network Inference - bf16bf16bf16 - CPUmnn: resnet-v2-50mnn: SqueezeNetV1.0mnn: inception-v3cpuminer-opt: scryptcpuminer-opt: Quad SHA-256, Pyritecpuminer-opt: x25xcpuminer-opt: Garlicoincpuminer-opt: Skeincoincpuminer-opt: LBC, LBRY Creditsncnn: CPU - mnasnetncnn: CPU - efficientnet-b0ncnn: CPU - blazefacencnn: CPU - googlenetncnn: CPU - resnet50ncnn: CPU - regnety_400mncnn: CPU - vision_transformerncnn: CPU - FastestDetopenvino: Face Detection FP16 - CPUopenvino: Face Detection FP16 - CPUopenvino: Face Detection FP16-INT8 - CPUopenvino: Face Detection FP16-INT8 - CPUopenvino: Age Gender Recognition Retail 0013 FP16 - CPUopenvino: Age Gender Recognition Retail 0013 FP16 - CPUopenvino: Age Gender Recognition Retail 0013 FP16-INT8 - CPUopenvino: Age Gender Recognition Retail 0013 FP16-INT8 - CPUopenvino: Person Detection FP16 - CPUopenvino: Person Detection FP16 - CPUopenvino: Person Detection FP32 - CPUopenvino: Person Detection FP32 - CPUopenvino: Weld Porosity Detection FP16-INT8 - CPUopenvino: Weld Porosity Detection FP16-INT8 - CPUopenvino: Weld Porosity Detection FP16 - CPUopenvino: Weld Porosity Detection FP16 - CPUopenvino: Vehicle Detection FP16-INT8 - CPUopenvino: Vehicle Detection FP16-INT8 - CPUopenvino: Vehicle Detection FP16 - CPUopenvino: Vehicle Detection FP16 - CPUopenvino: Person Vehicle Bike Detection FP16 - CPUopenvino: Person Vehicle Bike Detection FP16 - CPUopenvino: Machine Translation EN To DE FP16 - CPUopenvino: Machine Translation EN To DE FP16 - CPUgromacs: MPI CPU - water_GMX50_bareonnx: fcn-resnet101-11 - CPU - Standardonnx: super-resolution-10 - CPU - Standardonnx: bertsquad-12 - CPU - Standardonnx: ArcFace ResNet-100 - CPU - Standardnumpy: minibude: OpenMP - BM1minibude: OpenMP - BM1minibude: OpenMP - BM2minibude: OpenMP - BM2svt-av1: Preset 13 - Bosphorus 4Ksvt-av1: Preset 12 - Bosphorus 4Knumenta-nab: Bayesian Changepointnumenta-nab: Windowed Gaussiannumenta-nab: Relative Entropyoidn: RT.hdr_alb_nrm.3840x2160oidn: RT.ldr_alb_nrm.3840x2160oidn: RTLightmap.hdr.4096x4096daphne: OpenMP - NDT Mappingdaphne: OpenMP - Points2Imagesimdjson: PartialTweetssimdjson: LargeRandsimdjson: Kostyasimdjson: DistinctUserIDsimdjson: TopTweetopenfoam: drivaerFastback, Medium Mesh Size - Mesh Timeopenfoam: drivaerFastback, Medium Mesh Size - Execution Timecp2k: Fayalite-FISTsmhasher: FarmHash32 x86_64 AVXsmhasher: FarmHash32 x86_64 AVXjpegxl: JPEG - 90jpegxl: JPEG - 100jpegxl: PNG - 90AVX-512 OnAVX-512 Off35102701621184.998811.7586616.6804155.3462187.08375.34131195.936780.08261953.025549.032834.520628.9609100.54589.9408761.2241125.7860189.56115.2712857.2265111.728234.251829.189222.15157.2960.1790779096212.9364184.4701180.9315133244.156643.071154.218714823534698581927118546154238447646049385188081772804560869411070221840.51616722.58750.9895911962.061955.392361.2715.4418.57945.8124800.6122606807696.06715531951330106695042.3057.7125.8872.8066.34247.3274.9358.95102.04469.29193.93246.98148967.980.55170652.710.3643.341100.2543.341101.0019800.209.639988.444.7911202.624.287452.966.439065.345.29956.8850.1118.76427174015161051575.667299.545291.9828652.005346.081246.572245.55516.6774.7279.8983.513.511.661371.1813677.7644948436.701.264.186.866.52135.77418113.637331122.72539794.4426.3879.550.749.9518211066288773.538013.5907509.4462187.8540177.53875.62821005.161595.33471410.805167.911830.606232.664771.043714.0655490.3658195.1656157.8196.3313690.8722138.586130.526732.750512.7555.1131.1284238162176.8831153.4476151.0558115526.379425.135440.51171812871574971011321226471842924586172211524229852143413682084113398267850.53768222.83541.763364953.984989.131715.7624.0989.14630.4362946.0813229605784.2548827142754349174044.1962.8629.0076.1367.48270.1286.1660.7343.941088.5094.97503.50108449.491.21151239.600.5725.501864.9325.741845.959672.9419.694110.4911.666065.767.903782.0812.675135.569.33445.59107.6218.4672506288499918548.835065.096202.6046391.519255.661239.881233.94417.3894.88210.0353.503.511.721335.6311521.385.141.043.395.255.38144.3478135.216481198.40836483.7026.4129.130.729.52OpenBenchmarking.org

AI Benchmark Alpha

AI Benchmark Alpha is a Python library for evaluating artificial intelligence (AI) performance on diverse hardware platforms and relies upon the TensorFlow machine learning library. Learn more via the OpenBenchmarking.org test page.

OpenBenchmarking.orgScore, More Is BetterAI Benchmark Alpha 0.1.2Device Inference ScoreAVX-512 OnAVX-512 Off800160024003200400035101821

OpenBenchmarking.orgScore, More Is BetterAI Benchmark Alpha 0.1.2Device Training ScoreAVX-512 OnAVX-512 Off600120018002400300027011066

OpenBenchmarking.orgScore, More Is BetterAI Benchmark Alpha 0.1.2Device AI ScoreAVX-512 OnAVX-512 Off1300260039005200650062112887

Neural Magic DeepSparse

OpenBenchmarking.orgitems/sec, More Is BetterNeural Magic DeepSparse 1.1Model: NLP Text Classification, BERT base uncased SST2 - Scenario: Synchronous Single-StreamAVX-512 OnAVX-512 Off20406080100SE +/- 0.45, N = 3SE +/- 0.27, N = 385.0073.54

OpenBenchmarking.orgms/batch, Fewer Is BetterNeural Magic DeepSparse 1.1Model: NLP Text Classification, BERT base uncased SST2 - Scenario: Synchronous Single-StreamAVX-512 OnAVX-512 Off3691215SE +/- 0.06, N = 3SE +/- 0.05, N = 311.7613.59

OpenBenchmarking.orgitems/sec, More Is BetterNeural Magic DeepSparse 1.1Model: NLP Text Classification, BERT base uncased SST2 - Scenario: Asynchronous Multi-StreamAVX-512 OnAVX-512 Off130260390520650SE +/- 1.92, N = 3SE +/- 0.33, N = 3616.68509.45

OpenBenchmarking.orgms/batch, Fewer Is BetterNeural Magic DeepSparse 1.1Model: NLP Text Classification, BERT base uncased SST2 - Scenario: Asynchronous Multi-StreamAVX-512 OnAVX-512 Off4080120160200SE +/- 0.47, N = 3SE +/- 0.10, N = 3155.35187.85

OpenBenchmarking.orgitems/sec, More Is BetterNeural Magic DeepSparse 1.1Model: NLP Text Classification, DistilBERT mnli - Scenario: Synchronous Single-StreamAVX-512 OnAVX-512 Off4080120160200SE +/- 0.11, N = 3SE +/- 0.29, N = 3187.08177.54

OpenBenchmarking.orgms/batch, Fewer Is BetterNeural Magic DeepSparse 1.1Model: NLP Text Classification, DistilBERT mnli - Scenario: Synchronous Single-StreamAVX-512 OnAVX-512 Off1.26632.53263.79895.06526.3315SE +/- 0.0031, N = 3SE +/- 0.0091, N = 35.34135.6282

OpenBenchmarking.orgitems/sec, More Is BetterNeural Magic DeepSparse 1.1Model: NLP Text Classification, DistilBERT mnli - Scenario: Asynchronous Multi-StreamAVX-512 OnAVX-512 Off30060090012001500SE +/- 2.39, N = 3SE +/- 0.78, N = 31195.941005.16

OpenBenchmarking.orgms/batch, Fewer Is BetterNeural Magic DeepSparse 1.1Model: NLP Text Classification, DistilBERT mnli - Scenario: Asynchronous Multi-StreamAVX-512 OnAVX-512 Off20406080100SE +/- 0.14, N = 3SE +/- 0.05, N = 380.0895.33

OpenBenchmarking.orgitems/sec, More Is BetterNeural Magic DeepSparse 1.1Model: CV Classification, ResNet-50 ImageNet - Scenario: Asynchronous Multi-StreamAVX-512 OnAVX-512 Off400800120016002000SE +/- 4.32, N = 3SE +/- 0.17, N = 31953.031410.81

OpenBenchmarking.orgms/batch, Fewer Is BetterNeural Magic DeepSparse 1.1Model: CV Classification, ResNet-50 ImageNet - Scenario: Asynchronous Multi-StreamAVX-512 OnAVX-512 Off1530456075SE +/- 0.11, N = 3SE +/- 0.02, N = 349.0367.91

OpenBenchmarking.orgitems/sec, More Is BetterNeural Magic DeepSparse 1.1Model: NLP Token Classification, BERT base uncased conll2003 - Scenario: Synchronous Single-StreamAVX-512 OnAVX-512 Off816243240SE +/- 0.03, N = 3SE +/- 0.03, N = 334.5230.61

OpenBenchmarking.orgms/batch, Fewer Is BetterNeural Magic DeepSparse 1.1Model: NLP Token Classification, BERT base uncased conll2003 - Scenario: Synchronous Single-StreamAVX-512 OnAVX-512 Off816243240SE +/- 0.03, N = 3SE +/- 0.04, N = 328.9632.66

OpenBenchmarking.orgitems/sec, More Is BetterNeural Magic DeepSparse 1.1Model: NLP Question Answering, BERT base uncased SQuaD 12layer Pruned90 - Scenario: Synchronous Single-StreamAVX-512 OnAVX-512 Off20406080100SE +/- 0.17, N = 3SE +/- 0.17, N = 3100.5571.04

OpenBenchmarking.orgms/batch, Fewer Is BetterNeural Magic DeepSparse 1.1Model: NLP Question Answering, BERT base uncased SQuaD 12layer Pruned90 - Scenario: Synchronous Single-StreamAVX-512 OnAVX-512 Off48121620SE +/- 0.0172, N = 3SE +/- 0.0334, N = 39.940814.0655

OpenBenchmarking.orgitems/sec, More Is BetterNeural Magic DeepSparse 1.1Model: NLP Question Answering, BERT base uncased SQuaD 12layer Pruned90 - Scenario: Asynchronous Multi-StreamAVX-512 OnAVX-512 Off160320480640800SE +/- 0.79, N = 3SE +/- 0.61, N = 3761.22490.37

OpenBenchmarking.orgms/batch, Fewer Is BetterNeural Magic DeepSparse 1.1Model: NLP Question Answering, BERT base uncased SQuaD 12layer Pruned90 - Scenario: Asynchronous Multi-StreamAVX-512 OnAVX-512 Off4080120160200SE +/- 0.11, N = 3SE +/- 0.23, N = 3125.79195.17

OpenBenchmarking.orgitems/sec, More Is BetterNeural Magic DeepSparse 1.1Model: CV Detection,YOLOv5s COCO - Scenario: Synchronous Single-StreamAVX-512 OnAVX-512 Off4080120160200SE +/- 0.22, N = 3SE +/- 0.17, N = 3189.56157.82

OpenBenchmarking.orgms/batch, Fewer Is BetterNeural Magic DeepSparse 1.1Model: CV Detection,YOLOv5s COCO - Scenario: Synchronous Single-StreamAVX-512 OnAVX-512 Off246810SE +/- 0.0061, N = 3SE +/- 0.0068, N = 35.27126.3313

OpenBenchmarking.orgitems/sec, More Is BetterNeural Magic DeepSparse 1.1Model: CV Detection,YOLOv5s COCO - Scenario: Asynchronous Multi-StreamAVX-512 OnAVX-512 Off2004006008001000SE +/- 0.77, N = 3SE +/- 1.12, N = 3857.23690.87

OpenBenchmarking.orgms/batch, Fewer Is BetterNeural Magic DeepSparse 1.1Model: CV Detection,YOLOv5s COCO - Scenario: Asynchronous Multi-StreamAVX-512 OnAVX-512 Off306090120150SE +/- 0.12, N = 3SE +/- 0.25, N = 3111.73138.59

OpenBenchmarking.orgitems/sec, More Is BetterNeural Magic DeepSparse 1.1Model: NLP Document Classification, oBERT base uncased on IMDB - Scenario: Synchronous Single-StreamAVX-512 OnAVX-512 Off816243240SE +/- 0.08, N = 3SE +/- 0.10, N = 334.2530.53

OpenBenchmarking.orgms/batch, Fewer Is BetterNeural Magic DeepSparse 1.1Model: NLP Document Classification, oBERT base uncased on IMDB - Scenario: Synchronous Single-StreamAVX-512 OnAVX-512 Off816243240SE +/- 0.07, N = 3SE +/- 0.11, N = 329.1932.75

TensorFlow

This is a benchmark of the TensorFlow deep learning framework using the TensorFlow reference benchmarks (tensorflow/benchmarks with tf_cnn_benchmarks.py). Note with the Phoronix Test Suite there is also pts/tensorflow-lite for benchmarking the TensorFlow Lite binaries too. Learn more via the OpenBenchmarking.org test page.

OpenBenchmarking.orgimages/sec, More Is BetterTensorFlow 2.10Device: CPU - Batch Size: 16 - Model: ResNet-50AVX-512 OnAVX-512 Off510152025SE +/- 0.11, N = 3SE +/- 0.01, N = 322.1512.75

OpenBenchmarking.orgimages/sec, More Is BetterTensorFlow 2.10Device: CPU - Batch Size: 16 - Model: AlexNetAVX-512 OnAVX-512 Off306090120150SE +/- 2.35, N = 12SE +/- 0.19, N = 3157.2955.11

OpenBenchmarking.orgimages/sec, More Is BetterTensorFlow 2.10Device: CPU - Batch Size: 16 - Model: GoogLeNetAVX-512 OnAVX-512 Off1326395265SE +/- 1.01, N = 15SE +/- 0.25, N = 360.1731.12

LeelaChessZero

LeelaChessZero (lc0 / lczero) is a chess engine automated vian neural networks. This test profile can be used for OpenCL, CUDA + cuDNN, and BLAS (CPU-based) benchmarking. Learn more via the OpenBenchmarking.org test page.

OpenBenchmarking.orgNodes Per Second, More Is BetterLeelaChessZero 0.28Backend: BLASAVX-512 OnAVX-512 Off2K4K6K8K10KSE +/- 103.04, N = 4SE +/- 21.53, N = 390778423-mavx512f -mavx512cd -mavx512vl -mavx512bw -mavx512dq -mavx512ifma -mavx512vbmi-mno-avx512f1. (CXX) g++ options: -flto -O3 -march=native -pthread

OpenBenchmarking.orgNodes Per Second, More Is BetterLeelaChessZero 0.28Backend: EigenAVX-512 OnAVX-512 Off2K4K6K8K10KSE +/- 45.37, N = 3SE +/- 43.59, N = 390968162-mavx512f -mavx512cd -mavx512vl -mavx512bw -mavx512dq -mavx512ifma -mavx512vbmi-mno-avx512f1. (CXX) g++ options: -flto -O3 -march=native -pthread

Embree

Intel Embree is a collection of high-performance ray-tracing kernels for execution on CPUs and supporting instruction sets such as SSE, AVX, AVX2, and AVX-512. Embree also supports making use of the Intel SPMD Program Compiler (ISPC). Learn more via the OpenBenchmarking.org test page.

OpenBenchmarking.orgFrames Per Second, More Is BetterEmbree 3.13Binary: Pathtracer ISPC - Model: Asian DragonAVX-512 OnAVX-512 Off50100150200250SE +/- 0.22, N = 9SE +/- 0.21, N = 8212.94176.88MIN: 207.72 / MAX: 227.11MIN: 170.19 / MAX: 190

OpenBenchmarking.orgFrames Per Second, More Is BetterEmbree 3.13Binary: Pathtracer ISPC - Model: Asian Dragon ObjAVX-512 OnAVX-512 Off4080120160200SE +/- 0.62, N = 4SE +/- 0.36, N = 4184.47153.45MIN: 178.83 / MAX: 196.34MIN: 130.31 / MAX: 166.17

OpenBenchmarking.orgFrames Per Second, More Is BetterEmbree 3.13Binary: Pathtracer ISPC - Model: CrownAVX-512 OnAVX-512 Off4080120160200SE +/- 0.62, N = 8SE +/- 0.25, N = 7180.93151.06MIN: 124.41 / MAX: 209.74MIN: 114.4 / MAX: 176.43

OpenVKL

OpenVKL is the Intel Open Volume Kernel Library that offers high-performance volume computation kernels and part of the Intel oneAPI rendering toolkit. Learn more via the OpenBenchmarking.org test page.

OpenBenchmarking.orgItems / Sec, More Is BetterOpenVKL 1.3.1Benchmark: vklBenchmark ISPCAVX-512 OnAVX-512 Off30060090012001500SE +/- 13.58, N = 3SE +/- 8.11, N = 313321155MIN: 329 / MAX: 4770MIN: 251 / MAX: 5181

OSPRay

Intel OSPRay is a portable ray-tracing engine for high-performance, high-fidelity scientific visualizations. OSPRay builds off Intel's Embree and Intel SPMD Program Compiler (ISPC) components as part of the oneAPI rendering toolkit. Learn more via the OpenBenchmarking.org test page.

OpenBenchmarking.orgItems Per Second, More Is BetterOSPRay 2.10Benchmark: gravity_spheres_volume/dim_512/ao/real_timeAVX-512 OnAVX-512 Off1020304050SE +/- 0.21, N = 3SE +/- 0.02, N = 344.1626.38

OpenBenchmarking.orgItems Per Second, More Is BetterOSPRay 2.10Benchmark: gravity_spheres_volume/dim_512/scivis/real_timeAVX-512 OnAVX-512 Off1020304050SE +/- 0.10, N = 3SE +/- 0.04, N = 343.0725.14

OpenBenchmarking.orgItems Per Second, More Is BetterOSPRay 2.10Benchmark: gravity_spheres_volume/dim_512/pathtracer/real_timeAVX-512 OnAVX-512 Off1224364860SE +/- 0.05, N = 3SE +/- 0.03, N = 354.2240.51

OSPRay Studio

Intel OSPRay Studio is an open-source, interactive visualization and ray-tracing software package. OSPRay Studio makes use of Intel OSPRay, a portable ray-tracing engine for high-performance, high-fidelity visualizations. OSPRay builds off Intel's Embree and Intel SPMD Program Compiler (ISPC) components as part of the oneAPI rendering toolkit. Learn more via the OpenBenchmarking.org test page.

OpenBenchmarking.orgms, Fewer Is BetterOSPRay Studio 0.11Camera: 1 - Resolution: 1080p - Samples Per Pixel: 1 - Renderer: Path TracerAVX-512 OnAVX-512 Off4080120160200SE +/- 0.33, N = 3SE +/- 0.33, N = 3148181-mavx512f -mavx512cd -mavx512vl -mavx512bw -mavx512dq -mavx512ifma -mavx512vbmi-mno-avx512f1. (CXX) g++ options: -O3 -march=native -ldl

OpenBenchmarking.orgms, Fewer Is BetterOSPRay Studio 0.11Camera: 1 - Resolution: 1080p - Samples Per Pixel: 16 - Renderer: Path TracerAVX-512 OnAVX-512 Off6001200180024003000SE +/- 2.33, N = 3SE +/- 1.20, N = 323532871-mavx512f -mavx512cd -mavx512vl -mavx512bw -mavx512dq -mavx512ifma -mavx512vbmi-mno-avx512f1. (CXX) g++ options: -O3 -march=native -ldl

OpenBenchmarking.orgms, Fewer Is BetterOSPRay Studio 0.11Camera: 1 - Resolution: 1080p - Samples Per Pixel: 32 - Renderer: Path TracerAVX-512 OnAVX-512 Off12002400360048006000SE +/- 3.71, N = 3SE +/- 6.56, N = 346985749-mavx512f -mavx512cd -mavx512vl -mavx512bw -mavx512dq -mavx512ifma -mavx512vbmi-mno-avx512f1. (CXX) g++ options: -O3 -march=native -ldl

OpenBenchmarking.orgms, Fewer Is BetterOSPRay Studio 0.11Camera: 1 - Resolution: 4K - Samples Per Pixel: 1 - Renderer: Path TracerAVX-512 OnAVX-512 Off150300450600750SE +/- 1.00, N = 3SE +/- 2.08, N = 3581710-mavx512f -mavx512cd -mavx512vl -mavx512bw -mavx512dq -mavx512ifma -mavx512vbmi-mno-avx512f1. (CXX) g++ options: -O3 -march=native -ldl

OpenBenchmarking.orgms, Fewer Is BetterOSPRay Studio 0.11Camera: 1 - Resolution: 4K - Samples Per Pixel: 16 - Renderer: Path TracerAVX-512 OnAVX-512 Off2K4K6K8K10KSE +/- 10.17, N = 3SE +/- 24.46, N = 3927111321-mavx512f -mavx512cd -mavx512vl -mavx512bw -mavx512dq -mavx512ifma -mavx512vbmi-mno-avx512f1. (CXX) g++ options: -O3 -march=native -ldl

OpenBenchmarking.orgms, Fewer Is BetterOSPRay Studio 0.11Camera: 1 - Resolution: 4K - Samples Per Pixel: 32 - Renderer: Path TracerAVX-512 OnAVX-512 Off5K10K15K20K25KSE +/- 13.75, N = 3SE +/- 27.41, N = 31854622647-mavx512f -mavx512cd -mavx512vl -mavx512bw -mavx512dq -mavx512ifma -mavx512vbmi-mno-avx512f1. (CXX) g++ options: -O3 -march=native -ldl

OpenBenchmarking.orgms, Fewer Is BetterOSPRay Studio 0.11Camera: 2 - Resolution: 1080p - Samples Per Pixel: 1 - Renderer: Path TracerAVX-512 OnAVX-512 Off4080120160200SE +/- 0.00, N = 3SE +/- 0.33, N = 3154184-mavx512f -mavx512cd -mavx512vl -mavx512bw -mavx512dq -mavx512ifma -mavx512vbmi-mno-avx512f1. (CXX) g++ options: -O3 -march=native -ldl

OpenBenchmarking.orgms, Fewer Is BetterOSPRay Studio 0.11Camera: 2 - Resolution: 1080p - Samples Per Pixel: 16 - Renderer: Path TracerAVX-512 OnAVX-512 Off6001200180024003000SE +/- 9.17, N = 3SE +/- 0.88, N = 323842924-mavx512f -mavx512cd -mavx512vl -mavx512bw -mavx512dq -mavx512ifma -mavx512vbmi-mno-avx512f1. (CXX) g++ options: -O3 -march=native -ldl

OpenBenchmarking.orgms, Fewer Is BetterOSPRay Studio 0.11Camera: 2 - Resolution: 1080p - Samples Per Pixel: 32 - Renderer: Path TracerAVX-512 OnAVX-512 Off13002600390052006500SE +/- 6.24, N = 3SE +/- 8.25, N = 347645861-mavx512f -mavx512cd -mavx512vl -mavx512bw -mavx512dq -mavx512ifma -mavx512vbmi-mno-avx512f1. (CXX) g++ options: -O3 -march=native -ldl

OpenBenchmarking.orgms, Fewer Is BetterOSPRay Studio 0.11Camera: 2 - Resolution: 4K - Samples Per Pixel: 1 - Renderer: Path TracerAVX-512 OnAVX-512 Off160320480640800SE +/- 1.00, N = 3SE +/- 0.67, N = 3604722-mavx512f -mavx512cd -mavx512vl -mavx512bw -mavx512dq -mavx512ifma -mavx512vbmi-mno-avx512f1. (CXX) g++ options: -O3 -march=native -ldl

OpenBenchmarking.orgms, Fewer Is BetterOSPRay Studio 0.11Camera: 2 - Resolution: 4K - Samples Per Pixel: 16 - Renderer: Path TracerAVX-512 OnAVX-512 Off2K4K6K8K10KSE +/- 2.19, N = 3SE +/- 15.51, N = 3938511524-mavx512f -mavx512cd -mavx512vl -mavx512bw -mavx512dq -mavx512ifma -mavx512vbmi-mno-avx512f1. (CXX) g++ options: -O3 -march=native -ldl

OpenBenchmarking.orgms, Fewer Is BetterOSPRay Studio 0.11Camera: 2 - Resolution: 4K - Samples Per Pixel: 32 - Renderer: Path TracerAVX-512 OnAVX-512 Off5K10K15K20K25KSE +/- 20.34, N = 3SE +/- 30.02, N = 31880822985-mavx512f -mavx512cd -mavx512vl -mavx512bw -mavx512dq -mavx512ifma -mavx512vbmi-mno-avx512f1. (CXX) g++ options: -O3 -march=native -ldl

OpenBenchmarking.orgms, Fewer Is BetterOSPRay Studio 0.11Camera: 3 - Resolution: 1080p - Samples Per Pixel: 1 - Renderer: Path TracerAVX-512 OnAVX-512 Off50100150200250SE +/- 0.00, N = 3SE +/- 0.58, N = 3177214-mavx512f -mavx512cd -mavx512vl -mavx512bw -mavx512dq -mavx512ifma -mavx512vbmi-mno-avx512f1. (CXX) g++ options: -O3 -march=native -ldl

OpenBenchmarking.orgms, Fewer Is BetterOSPRay Studio 0.11Camera: 3 - Resolution: 1080p - Samples Per Pixel: 16 - Renderer: Path TracerAVX-512 OnAVX-512 Off7001400210028003500SE +/- 3.06, N = 3SE +/- 7.00, N = 328043413-mavx512f -mavx512cd -mavx512vl -mavx512bw -mavx512dq -mavx512ifma -mavx512vbmi-mno-avx512f1. (CXX) g++ options: -O3 -march=native -ldl

OpenBenchmarking.orgms, Fewer Is BetterOSPRay Studio 0.11Camera: 3 - Resolution: 1080p - Samples Per Pixel: 32 - Renderer: Path TracerAVX-512 OnAVX-512 Off15003000450060007500SE +/- 9.54, N = 3SE +/- 5.51, N = 356086820-mavx512f -mavx512cd -mavx512vl -mavx512bw -mavx512dq -mavx512ifma -mavx512vbmi-mno-avx512f1. (CXX) g++ options: -O3 -march=native -ldl

OpenBenchmarking.orgms, Fewer Is BetterOSPRay Studio 0.11Camera: 3 - Resolution: 4K - Samples Per Pixel: 1 - Renderer: Path TracerAVX-512 OnAVX-512 Off2004006008001000SE +/- 1.53, N = 3SE +/- 1.15, N = 3694841-mavx512f -mavx512cd -mavx512vl -mavx512bw -mavx512dq -mavx512ifma -mavx512vbmi-mno-avx512f1. (CXX) g++ options: -O3 -march=native -ldl

OpenBenchmarking.orgms, Fewer Is BetterOSPRay Studio 0.11Camera: 3 - Resolution: 4K - Samples Per Pixel: 16 - Renderer: Path TracerAVX-512 OnAVX-512 Off3K6K9K12K15KSE +/- 27.14, N = 3SE +/- 22.81, N = 31107013398-mavx512f -mavx512cd -mavx512vl -mavx512bw -mavx512dq -mavx512ifma -mavx512vbmi-mno-avx512f1. (CXX) g++ options: -O3 -march=native -ldl

OpenBenchmarking.orgms, Fewer Is BetterOSPRay Studio 0.11Camera: 3 - Resolution: 4K - Samples Per Pixel: 32 - Renderer: Path TracerAVX-512 OnAVX-512 Off6K12K18K24K30KSE +/- 21.63, N = 3SE +/- 15.62, N = 32218426785-mavx512f -mavx512cd -mavx512vl -mavx512bw -mavx512dq -mavx512ifma -mavx512vbmi-mno-avx512f1. (CXX) g++ options: -O3 -march=native -ldl

oneDNN

OpenBenchmarking.orgms, Fewer Is BetteroneDNN 2.7Harness: Convolution Batch Shapes Auto - Data Type: f32 - Engine: CPUAVX-512 OnAVX-512 Off0.1210.2420.3630.4840.605SE +/- 0.004067, N = 7SE +/- 0.004200, N = 150.5161670.537682-mavx512f -mavx512cd -mavx512vl -mavx512bw -mavx512dq -mavx512ifma -mavx512vbmi - MIN: 0.42-mno-avx512f - MIN: 0.421. (CXX) g++ options: -O3 -march=native -fopenmp -msse4.1 -fPIC -pie -ldl -lpthread

OpenBenchmarking.orgms, Fewer Is BetteroneDNN 2.7Harness: Deconvolution Batch shapes_1d - Data Type: f32 - Engine: CPUAVX-512 OnAVX-512 Off510152025SE +/- 0.16, N = 3SE +/- 0.06, N = 322.5922.84-mavx512f -mavx512cd -mavx512vl -mavx512bw -mavx512dq -mavx512ifma -mavx512vbmi - MIN: 19.6-mno-avx512f - MIN: 20.221. (CXX) g++ options: -O3 -march=native -fopenmp -msse4.1 -fPIC -pie -ldl -lpthread

OpenBenchmarking.orgms, Fewer Is BetteroneDNN 2.7Harness: Deconvolution Batch shapes_3d - Data Type: f32 - Engine: CPUAVX-512 OnAVX-512 Off0.39680.79361.19041.58721.984SE +/- 0.001906, N = 9SE +/- 0.004550, N = 90.9895911.763360-mavx512f -mavx512cd -mavx512vl -mavx512bw -mavx512dq -mavx512ifma -mavx512vbmi - MIN: 0.9-mno-avx512f - MIN: 1.551. (CXX) g++ options: -O3 -march=native -fopenmp -msse4.1 -fPIC -pie -ldl -lpthread

OpenBenchmarking.orgms, Fewer Is BetteroneDNN 2.7Harness: Recurrent Neural Network Training - Data Type: f32 - Engine: CPUAVX-512 OnAVX-512 Off11002200330044005500SE +/- 20.13, N = 5SE +/- 60.12, N = 41962.064953.98-mavx512f -mavx512cd -mavx512vl -mavx512bw -mavx512dq -mavx512ifma -mavx512vbmi - MIN: 1899.34-mno-avx512f - MIN: 4690.851. (CXX) g++ options: -O3 -march=native -fopenmp -msse4.1 -fPIC -pie -ldl -lpthread

OpenBenchmarking.orgms, Fewer Is BetteroneDNN 2.7Harness: Recurrent Neural Network Training - Data Type: bf16bf16bf16 - Engine: CPUAVX-512 OnAVX-512 Off11002200330044005500SE +/- 19.67, N = 15SE +/- 41.71, N = 31955.394989.13-mavx512f -mavx512cd -mavx512vl -mavx512bw -mavx512dq -mavx512ifma -mavx512vbmi - MIN: 1769.7-mno-avx512f - MIN: 4767.221. (CXX) g++ options: -O3 -march=native -fopenmp -msse4.1 -fPIC -pie -ldl -lpthread

OpenBenchmarking.orgms, Fewer Is BetteroneDNN 2.7Harness: Recurrent Neural Network Inference - Data Type: bf16bf16bf16 - Engine: CPUAVX-512 OnAVX-512 Off5001000150020002500SE +/- 24.41, N = 4SE +/- 22.32, N = 32361.271715.76-mavx512f -mavx512cd -mavx512vl -mavx512bw -mavx512dq -mavx512ifma -mavx512vbmi - MIN: 2265.7-mno-avx512f - MIN: 1607.131. (CXX) g++ options: -O3 -march=native -fopenmp -msse4.1 -fPIC -pie -ldl -lpthread

Mobile Neural Network

MNN is the Mobile Neural Network as a highly efficient, lightweight deep learning framework developed by Alibaba. This MNN test profile is building the OpenMP / CPU threaded version for processor benchmarking and not any GPU-accelerated test. MNN does allow making use of AVX-512 extensions. Learn more via the OpenBenchmarking.org test page.

OpenBenchmarking.orgms, Fewer Is BetterMobile Neural Network 2.1Model: resnet-v2-50AVX-512 OnAVX-512 Off612182430SE +/- 0.08, N = 9SE +/- 0.08, N = 815.4424.10-mavx512f -mavx512cd -mavx512vl -mavx512bw -mavx512dq -mavx512ifma -mavx512vbmi - MIN: 14.79 / MAX: 54.05-mno-avx512f - MIN: 23.44 / MAX: 71.321. (CXX) g++ options: -O3 -march=native -std=c++11 -fvisibility=hidden -fomit-frame-pointer -fstrict-aliasing -ffunction-sections -fdata-sections -ffast-math -fno-rtti -fno-exceptions -rdynamic -pthread -ldl

OpenBenchmarking.orgms, Fewer Is BetterMobile Neural Network 2.1Model: SqueezeNetV1.0AVX-512 OnAVX-512 Off3691215SE +/- 0.147, N = 9SE +/- 0.091, N = 88.5799.146-mavx512f -mavx512cd -mavx512vl -mavx512bw -mavx512dq -mavx512ifma -mavx512vbmi - MIN: 6.67 / MAX: 21.5-mno-avx512f - MIN: 7.72 / MAX: 19.031. (CXX) g++ options: -O3 -march=native -std=c++11 -fvisibility=hidden -fomit-frame-pointer -fstrict-aliasing -ffunction-sections -fdata-sections -ffast-math -fno-rtti -fno-exceptions -rdynamic -pthread -ldl

OpenBenchmarking.orgms, Fewer Is BetterMobile Neural Network 2.1Model: inception-v3AVX-512 OnAVX-512 Off1020304050SE +/- 0.23, N = 9SE +/- 0.13, N = 845.8130.44-mavx512f -mavx512cd -mavx512vl -mavx512bw -mavx512dq -mavx512ifma -mavx512vbmi - MIN: 44.13 / MAX: 87.28-mno-avx512f - MIN: 28.75 / MAX: 109.221. (CXX) g++ options: -O3 -march=native -std=c++11 -fvisibility=hidden -fomit-frame-pointer -fstrict-aliasing -ffunction-sections -fdata-sections -ffast-math -fno-rtti -fno-exceptions -rdynamic -pthread -ldl

Cpuminer-Opt

Cpuminer-Opt is a fork of cpuminer-multi that carries a wide range of CPU performance optimizations for measuring the potential cryptocurrency mining performance of the CPU/processor with a wide variety of cryptocurrencies. The benchmark reports the hash speed for the CPU mining performance for the selected cryptocurrency. Learn more via the OpenBenchmarking.org test page.

OpenBenchmarking.orgkH/s, More Is BetterCpuminer-Opt 3.20.3Algorithm: scryptAVX-512 OnAVX-512 Off10002000300040005000SE +/- 0.45, N = 3SE +/- 0.31, N = 34800.612946.08-mavx512f -mavx512cd -mavx512vl -mavx512bw -mavx512dq -mavx512ifma -mavx512vbmi-mno-avx512f1. (CXX) g++ options: -O3 -march=native -lcurl -lz -lpthread -lssl -lcrypto -lgmp

OpenBenchmarking.orgkH/s, More Is BetterCpuminer-Opt 3.20.3Algorithm: Quad SHA-256, PyriteAVX-512 OnAVX-512 Off500K1000K1500K2000K2500KSE +/- 9040.25, N = 3SE +/- 276.83, N = 322606801322960-mavx512f -mavx512cd -mavx512vl -mavx512bw -mavx512dq -mavx512ifma -mavx512vbmi-mno-avx512f1. (CXX) g++ options: -O3 -march=native -lcurl -lz -lpthread -lssl -lcrypto -lgmp

OpenBenchmarking.orgkH/s, More Is BetterCpuminer-Opt 3.20.3Algorithm: x25xAVX-512 OnAVX-512 Off16003200480064008000SE +/- 16.58, N = 3SE +/- 14.41, N = 37696.065784.25-mavx512f -mavx512cd -mavx512vl -mavx512bw -mavx512dq -mavx512ifma -mavx512vbmi-mno-avx512f1. (CXX) g++ options: -O3 -march=native -lcurl -lz -lpthread -lssl -lcrypto -lgmp

OpenBenchmarking.orgkH/s, More Is BetterCpuminer-Opt 3.20.3Algorithm: GarlicoinAVX-512 OnAVX-512 Off15K30K45K60K75KSE +/- 89.69, N = 3SE +/- 13.33, N = 37155348827-mavx512f -mavx512cd -mavx512vl -mavx512bw -mavx512dq -mavx512ifma -mavx512vbmi-mno-avx512f1. (CXX) g++ options: -O3 -march=native -lcurl -lz -lpthread -lssl -lcrypto -lgmp

OpenBenchmarking.orgkH/s, More Is BetterCpuminer-Opt 3.20.3Algorithm: SkeincoinAVX-512 OnAVX-512 Off400K800K1200K1600K2000KSE +/- 13005.55, N = 3SE +/- 3964.01, N = 319513301427543-mavx512f -mavx512cd -mavx512vl -mavx512bw -mavx512dq -mavx512ifma -mavx512vbmi-mno-avx512f1. (CXX) g++ options: -O3 -march=native -lcurl -lz -lpthread -lssl -lcrypto -lgmp

OpenBenchmarking.orgkH/s, More Is BetterCpuminer-Opt 3.20.3Algorithm: LBC, LBRY CreditsAVX-512 OnAVX-512 Off200K400K600K800K1000KSE +/- 7645.47, N = 3SE +/- 667.26, N = 31066950491740-mavx512f -mavx512cd -mavx512vl -mavx512bw -mavx512dq -mavx512ifma -mavx512vbmi-mno-avx512f1. (CXX) g++ options: -O3 -march=native -lcurl -lz -lpthread -lssl -lcrypto -lgmp

NCNN

NCNN is a high performance neural network inference framework optimized for mobile and other platforms developed by Tencent. Learn more via the OpenBenchmarking.org test page.

OpenBenchmarking.orgms, Fewer Is BetterNCNN 20220729Target: CPU - Model: mnasnetAVX-512 OnAVX-512 Off1020304050SE +/- 0.26, N = 8SE +/- 0.49, N = 342.3044.19-mavx512f -mavx512cd -mavx512vl -mavx512bw -mavx512dq -mavx512ifma -mavx512vbmi - MIN: 39.64 / MAX: 571.78-mno-avx512f - MIN: 41.56 / MAX: 148.981. (CXX) g++ options: -O3 -march=native -rdynamic -lgomp -lpthread

OpenBenchmarking.orgms, Fewer Is BetterNCNN 20220729Target: CPU - Model: efficientnet-b0AVX-512 OnAVX-512 Off1428425670SE +/- 0.35, N = 8SE +/- 0.31, N = 357.7162.86-mavx512f -mavx512cd -mavx512vl -mavx512bw -mavx512dq -mavx512ifma -mavx512vbmi - MIN: 54.52 / MAX: 522.74-mno-avx512f - MIN: 59.46 / MAX: 154.821. (CXX) g++ options: -O3 -march=native -rdynamic -lgomp -lpthread

OpenBenchmarking.orgms, Fewer Is BetterNCNN 20220729Target: CPU - Model: blazefaceAVX-512 OnAVX-512 Off714212835SE +/- 0.18, N = 8SE +/- 0.54, N = 325.8829.00-mavx512f -mavx512cd -mavx512vl -mavx512bw -mavx512dq -mavx512ifma -mavx512vbmi - MIN: 24.45 / MAX: 112.37-mno-avx512f - MIN: 26.03 / MAX: 144.121. (CXX) g++ options: -O3 -march=native -rdynamic -lgomp -lpthread

OpenBenchmarking.orgms, Fewer Is BetterNCNN 20220729Target: CPU - Model: googlenetAVX-512 OnAVX-512 Off20406080100SE +/- 0.76, N = 8SE +/- 1.35, N = 372.8076.13-mavx512f -mavx512cd -mavx512vl -mavx512bw -mavx512dq -mavx512ifma -mavx512vbmi - MIN: 67.13 / MAX: 388.52-mno-avx512f - MIN: 70.13 / MAX: 155.611. (CXX) g++ options: -O3 -march=native -rdynamic -lgomp -lpthread

OpenBenchmarking.orgms, Fewer Is BetterNCNN 20220729Target: CPU - Model: resnet50AVX-512 OnAVX-512 Off1530456075SE +/- 0.50, N = 8SE +/- 0.89, N = 366.3467.48-mavx512f -mavx512cd -mavx512vl -mavx512bw -mavx512dq -mavx512ifma -mavx512vbmi - MIN: 62.92 / MAX: 194.74-mno-avx512f - MIN: 63.4 / MAX: 170.141. (CXX) g++ options: -O3 -march=native -rdynamic -lgomp -lpthread

OpenBenchmarking.orgms, Fewer Is BetterNCNN 20220729Target: CPU - Model: regnety_400mAVX-512 OnAVX-512 Off60120180240300SE +/- 2.25, N = 8SE +/- 3.75, N = 3247.32270.12-mavx512f -mavx512cd -mavx512vl -mavx512bw -mavx512dq -mavx512ifma -mavx512vbmi - MIN: 232.61 / MAX: 506.55-mno-avx512f - MIN: 245.47 / MAX: 498.81. (CXX) g++ options: -O3 -march=native -rdynamic -lgomp -lpthread

OpenBenchmarking.orgms, Fewer Is BetterNCNN 20220729Target: CPU - Model: vision_transformerAVX-512 OnAVX-512 Off20406080100SE +/- 1.59, N = 8SE +/- 5.82, N = 374.9386.16-mavx512f -mavx512cd -mavx512vl -mavx512bw -mavx512dq -mavx512ifma -mavx512vbmi - MIN: 65.04 / MAX: 2154.62-mno-avx512f - MIN: 73.72 / MAX: 1760.741. (CXX) g++ options: -O3 -march=native -rdynamic -lgomp -lpthread

OpenBenchmarking.orgms, Fewer Is BetterNCNN 20220729Target: CPU - Model: FastestDetAVX-512 OnAVX-512 Off1428425670SE +/- 0.49, N = 8SE +/- 2.90, N = 358.9560.73-mavx512f -mavx512cd -mavx512vl -mavx512bw -mavx512dq -mavx512ifma -mavx512vbmi - MIN: 55.11 / MAX: 282.72-mno-avx512f - MIN: 52.89 / MAX: 236.851. (CXX) g++ options: -O3 -march=native -rdynamic -lgomp -lpthread

OpenVINO

This is a test of the Intel OpenVINO, a toolkit around neural networks, using its built-in benchmarking support and analyzing the throughput and latency for various models. Learn more via the OpenBenchmarking.org test page.

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2022.2.devModel: Face Detection FP16 - Device: CPUAVX-512 OnAVX-512 Off20406080100SE +/- 0.07, N = 3SE +/- 0.01, N = 3102.0443.94-mavx512f -mavx512cd -mavx512vl -mavx512bw -mavx512dq -mavx512ifma -mavx512vbmi-mno-avx512f1. (CXX) g++ options: -fPIC -O3 -march=native -fsigned-char -ffunction-sections -fdata-sections -fno-strict-overflow -fwrapv -flto -shared

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2022.2.devModel: Face Detection FP16 - Device: CPUAVX-512 OnAVX-512 Off2004006008001000SE +/- 0.24, N = 3SE +/- 0.51, N = 3469.291088.50-mavx512f -mavx512cd -mavx512vl -mavx512bw -mavx512dq -mavx512ifma -mavx512vbmi - MIN: 410.93 / MAX: 547.33-mno-avx512f - MIN: 937.5 / MAX: 1231.551. (CXX) g++ options: -fPIC -O3 -march=native -fsigned-char -ffunction-sections -fdata-sections -fno-strict-overflow -fwrapv -flto -shared

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2022.2.devModel: Face Detection FP16-INT8 - Device: CPUAVX-512 OnAVX-512 Off4080120160200SE +/- 0.05, N = 3SE +/- 0.02, N = 3193.9394.97-mavx512f -mavx512cd -mavx512vl -mavx512bw -mavx512dq -mavx512ifma -mavx512vbmi-mno-avx512f1. (CXX) g++ options: -fPIC -O3 -march=native -fsigned-char -ffunction-sections -fdata-sections -fno-strict-overflow -fwrapv -flto -shared

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2022.2.devModel: Face Detection FP16-INT8 - Device: CPUAVX-512 OnAVX-512 Off110220330440550SE +/- 0.01, N = 3SE +/- 0.07, N = 3246.98503.50-mavx512f -mavx512cd -mavx512vl -mavx512bw -mavx512dq -mavx512ifma -mavx512vbmi - MIN: 204.5 / MAX: 299.54-mno-avx512f - MIN: 402.47 / MAX: 569.651. (CXX) g++ options: -fPIC -O3 -march=native -fsigned-char -ffunction-sections -fdata-sections -fno-strict-overflow -fwrapv -flto -shared

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2022.2.devModel: Age Gender Recognition Retail 0013 FP16 - Device: CPUAVX-512 OnAVX-512 Off30K60K90K120K150KSE +/- 1328.20, N = 3SE +/- 1130.85, N = 4148967.98108449.49-mavx512f -mavx512cd -mavx512vl -mavx512bw -mavx512dq -mavx512ifma -mavx512vbmi-mno-avx512f1. (CXX) g++ options: -fPIC -O3 -march=native -fsigned-char -ffunction-sections -fdata-sections -fno-strict-overflow -fwrapv -flto -shared

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2022.2.devModel: Age Gender Recognition Retail 0013 FP16 - Device: CPUAVX-512 OnAVX-512 Off0.27230.54460.81691.08921.3615SE +/- 0.00, N = 3SE +/- 0.01, N = 40.551.21-mavx512f -mavx512cd -mavx512vl -mavx512bw -mavx512dq -mavx512ifma -mavx512vbmi - MIN: 0.5 / MAX: 29.57-mno-avx512f - MIN: 0.98 / MAX: 47.371. (CXX) g++ options: -fPIC -O3 -march=native -fsigned-char -ffunction-sections -fdata-sections -fno-strict-overflow -fwrapv -flto -shared

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2022.2.devModel: Age Gender Recognition Retail 0013 FP16-INT8 - Device: CPUAVX-512 OnAVX-512 Off40K80K120K160K200KSE +/- 127.59, N = 3SE +/- 1455.37, N = 3170652.71151239.60-mavx512f -mavx512cd -mavx512vl -mavx512bw -mavx512dq -mavx512ifma -mavx512vbmi-mno-avx512f1. (CXX) g++ options: -fPIC -O3 -march=native -fsigned-char -ffunction-sections -fdata-sections -fno-strict-overflow -fwrapv -flto -shared

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2022.2.devModel: Age Gender Recognition Retail 0013 FP16-INT8 - Device: CPUAVX-512 OnAVX-512 Off0.12830.25660.38490.51320.6415SE +/- 0.00, N = 3SE +/- 0.00, N = 30.360.57-mavx512f -mavx512cd -mavx512vl -mavx512bw -mavx512dq -mavx512ifma -mavx512vbmi - MIN: 0.34 / MAX: 33.7-mno-avx512f - MIN: 0.52 / MAX: 47.931. (CXX) g++ options: -fPIC -O3 -march=native -fsigned-char -ffunction-sections -fdata-sections -fno-strict-overflow -fwrapv -flto -shared

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2022.2.devModel: Person Detection FP16 - Device: CPUAVX-512 OnAVX-512 Off1020304050SE +/- 0.20, N = 3SE +/- 0.22, N = 343.3425.50-mavx512f -mavx512cd -mavx512vl -mavx512bw -mavx512dq -mavx512ifma -mavx512vbmi-mno-avx512f1. (CXX) g++ options: -fPIC -O3 -march=native -fsigned-char -ffunction-sections -fdata-sections -fno-strict-overflow -fwrapv -flto -shared

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2022.2.devModel: Person Detection FP16 - Device: CPUAVX-512 OnAVX-512 Off400800120016002000SE +/- 4.83, N = 3SE +/- 15.65, N = 31100.251864.93-mavx512f -mavx512cd -mavx512vl -mavx512bw -mavx512dq -mavx512ifma -mavx512vbmi - MIN: 783.9 / MAX: 1824.92-mno-avx512f - MIN: 1371.29 / MAX: 2532.741. (CXX) g++ options: -fPIC -O3 -march=native -fsigned-char -ffunction-sections -fdata-sections -fno-strict-overflow -fwrapv -flto -shared

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2022.2.devModel: Person Detection FP32 - Device: CPUAVX-512 OnAVX-512 Off1020304050SE +/- 0.18, N = 3SE +/- 0.04, N = 343.3425.74-mavx512f -mavx512cd -mavx512vl -mavx512bw -mavx512dq -mavx512ifma -mavx512vbmi-mno-avx512f1. (CXX) g++ options: -fPIC -O3 -march=native -fsigned-char -ffunction-sections -fdata-sections -fno-strict-overflow -fwrapv -flto -shared

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2022.2.devModel: Person Detection FP32 - Device: CPUAVX-512 OnAVX-512 Off400800120016002000SE +/- 4.44, N = 3SE +/- 3.32, N = 31101.001845.95-mavx512f -mavx512cd -mavx512vl -mavx512bw -mavx512dq -mavx512ifma -mavx512vbmi - MIN: 840.82 / MAX: 1792.51-mno-avx512f - MIN: 1386.31 / MAX: 27981. (CXX) g++ options: -fPIC -O3 -march=native -fsigned-char -ffunction-sections -fdata-sections -fno-strict-overflow -fwrapv -flto -shared

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2022.2.devModel: Weld Porosity Detection FP16-INT8 - Device: CPUAVX-512 OnAVX-512 Off4K8K12K16K20KSE +/- 12.72, N = 3SE +/- 1.81, N = 319800.209672.94-mavx512f -mavx512cd -mavx512vl -mavx512bw -mavx512dq -mavx512ifma -mavx512vbmi-mno-avx512f1. (CXX) g++ options: -fPIC -O3 -march=native -fsigned-char -ffunction-sections -fdata-sections -fno-strict-overflow -fwrapv -flto -shared

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2022.2.devModel: Weld Porosity Detection FP16-INT8 - Device: CPUAVX-512 OnAVX-512 Off510152025SE +/- 0.01, N = 3SE +/- 0.01, N = 39.6319.69-mavx512f -mavx512cd -mavx512vl -mavx512bw -mavx512dq -mavx512ifma -mavx512vbmi - MIN: 8.29 / MAX: 57.53-mno-avx512f - MIN: 16.4 / MAX: 74.851. (CXX) g++ options: -fPIC -O3 -march=native -fsigned-char -ffunction-sections -fdata-sections -fno-strict-overflow -fwrapv -flto -shared

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2022.2.devModel: Weld Porosity Detection FP16 - Device: CPUAVX-512 OnAVX-512 Off2K4K6K8K10KSE +/- 6.96, N = 3SE +/- 4.29, N = 39988.444110.49-mavx512f -mavx512cd -mavx512vl -mavx512bw -mavx512dq -mavx512ifma -mavx512vbmi-mno-avx512f1. (CXX) g++ options: -fPIC -O3 -march=native -fsigned-char -ffunction-sections -fdata-sections -fno-strict-overflow -fwrapv -flto -shared

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2022.2.devModel: Weld Porosity Detection FP16 - Device: CPUAVX-512 OnAVX-512 Off3691215SE +/- 0.01, N = 3SE +/- 0.01, N = 34.7911.66-mavx512f -mavx512cd -mavx512vl -mavx512bw -mavx512dq -mavx512ifma -mavx512vbmi - MIN: 3.98 / MAX: 28.79-mno-avx512f - MIN: 9.72 / MAX: 43.71. (CXX) g++ options: -fPIC -O3 -march=native -fsigned-char -ffunction-sections -fdata-sections -fno-strict-overflow -fwrapv -flto -shared

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2022.2.devModel: Vehicle Detection FP16-INT8 - Device: CPUAVX-512 OnAVX-512 Off2K4K6K8K10KSE +/- 2.53, N = 3SE +/- 1.39, N = 311202.626065.76-mavx512f -mavx512cd -mavx512vl -mavx512bw -mavx512dq -mavx512ifma -mavx512vbmi-mno-avx512f1. (CXX) g++ options: -fPIC -O3 -march=native -fsigned-char -ffunction-sections -fdata-sections -fno-strict-overflow -fwrapv -flto -shared

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2022.2.devModel: Vehicle Detection FP16-INT8 - Device: CPUAVX-512 OnAVX-512 Off246810SE +/- 0.00, N = 3SE +/- 0.00, N = 34.287.90-mavx512f -mavx512cd -mavx512vl -mavx512bw -mavx512dq -mavx512ifma -mavx512vbmi - MIN: 3.5 / MAX: 42.62-mno-avx512f - MIN: 6.37 / MAX: 41.531. (CXX) g++ options: -fPIC -O3 -march=native -fsigned-char -ffunction-sections -fdata-sections -fno-strict-overflow -fwrapv -flto -shared

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2022.2.devModel: Vehicle Detection FP16 - Device: CPUAVX-512 OnAVX-512 Off16003200480064008000SE +/- 3.52, N = 3SE +/- 10.17, N = 37452.963782.08-mavx512f -mavx512cd -mavx512vl -mavx512bw -mavx512dq -mavx512ifma -mavx512vbmi-mno-avx512f1. (CXX) g++ options: -fPIC -O3 -march=native -fsigned-char -ffunction-sections -fdata-sections -fno-strict-overflow -fwrapv -flto -shared

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2022.2.devModel: Vehicle Detection FP16 - Device: CPUAVX-512 OnAVX-512 Off3691215SE +/- 0.00, N = 3SE +/- 0.03, N = 36.4312.67-mavx512f -mavx512cd -mavx512vl -mavx512bw -mavx512dq -mavx512ifma -mavx512vbmi - MIN: 5.22 / MAX: 61.26-mno-avx512f - MIN: 9.58 / MAX: 77.911. (CXX) g++ options: -fPIC -O3 -march=native -fsigned-char -ffunction-sections -fdata-sections -fno-strict-overflow -fwrapv -flto -shared

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2022.2.devModel: Person Vehicle Bike Detection FP16 - Device: CPUAVX-512 OnAVX-512 Off2K4K6K8K10KSE +/- 4.01, N = 3SE +/- 2.17, N = 39065.345135.56-mavx512f -mavx512cd -mavx512vl -mavx512bw -mavx512dq -mavx512ifma -mavx512vbmi-mno-avx512f1. (CXX) g++ options: -fPIC -O3 -march=native -fsigned-char -ffunction-sections -fdata-sections -fno-strict-overflow -fwrapv -flto -shared

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2022.2.devModel: Person Vehicle Bike Detection FP16 - Device: CPUAVX-512 OnAVX-512 Off3691215SE +/- 0.00, N = 3SE +/- 0.00, N = 35.299.33-mavx512f -mavx512cd -mavx512vl -mavx512bw -mavx512dq -mavx512ifma -mavx512vbmi - MIN: 4.36 / MAX: 45.25-mno-avx512f - MIN: 7.39 / MAX: 57.571. (CXX) g++ options: -fPIC -O3 -march=native -fsigned-char -ffunction-sections -fdata-sections -fno-strict-overflow -fwrapv -flto -shared

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2022.2.devModel: Machine Translation EN To DE FP16 - Device: CPUAVX-512 OnAVX-512 Off2004006008001000SE +/- 4.00, N = 3SE +/- 3.04, N = 3956.88445.59-mavx512f -mavx512cd -mavx512vl -mavx512bw -mavx512dq -mavx512ifma -mavx512vbmi-mno-avx512f1. (CXX) g++ options: -fPIC -O3 -march=native -fsigned-char -ffunction-sections -fdata-sections -fno-strict-overflow -fwrapv -flto -shared

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2022.2.devModel: Machine Translation EN To DE FP16 - Device: CPUAVX-512 OnAVX-512 Off20406080100SE +/- 0.21, N = 3SE +/- 0.74, N = 350.11107.62-mavx512f -mavx512cd -mavx512vl -mavx512bw -mavx512dq -mavx512ifma -mavx512vbmi - MIN: 38.98 / MAX: 166.69-mno-avx512f - MIN: 83.48 / MAX: 216.071. (CXX) g++ options: -fPIC -O3 -march=native -fsigned-char -ffunction-sections -fdata-sections -fno-strict-overflow -fwrapv -flto -shared

GROMACS

The GROMACS (GROningen MAchine for Chemical Simulations) molecular dynamics package testing with the water_GMX50 data. This test profile allows selecting between CPU and GPU-based GROMACS builds. Learn more via the OpenBenchmarking.org test page.

OpenBenchmarking.orgNs Per Day, More Is BetterGROMACS 2022.1Implementation: MPI CPU - Input: water_GMX50_bareAVX-512 OnAVX-512 Off510152025SE +/- 0.24, N = 3SE +/- 0.10, N = 318.7618.47-mavx512f -mavx512cd -mavx512vl -mavx512bw -mavx512dq -mavx512ifma -mavx512vbmi-mno-avx512f1. (CXX) g++ options: -O3 -march=native

ONNX Runtime

ONNX Runtime is developed by Microsoft and partners as a open-source, cross-platform, high performance machine learning inferencing and training accelerator. This test profile runs the ONNX Runtime with various models available from the ONNX Zoo. Learn more via the OpenBenchmarking.org test page.

OpenBenchmarking.orgInferences Per Minute, More Is BetterONNX Runtime 1.11Model: fcn-resnet101-11 - Device: CPU - Executor: StandardAVX-512 OnAVX-512 Off60120180240300SE +/- 0.44, N = 3SE +/- 2.36, N = 12271250-mavx512f -mavx512cd -mavx512vl -mavx512bw -mavx512dq -mavx512ifma -mavx512vbmi-mno-avx512f1. (CXX) g++ options: -O3 -march=native -ffunction-sections -fdata-sections -mtune=native -flto=auto -fno-fat-lto-objects -ldl -lrt

OpenBenchmarking.orgInferences Per Minute, More Is BetterONNX Runtime 1.11Model: super-resolution-10 - Device: CPU - Executor: StandardAVX-512 OnAVX-512 Off16003200480064008000SE +/- 5.46, N = 3SE +/- 260.54, N = 1274016288-mavx512f -mavx512cd -mavx512vl -mavx512bw -mavx512dq -mavx512ifma -mavx512vbmi-mno-avx512f1. (CXX) g++ options: -O3 -march=native -ffunction-sections -fdata-sections -mtune=native -flto=auto -fno-fat-lto-objects -ldl -lrt

OpenBenchmarking.orgInferences Per Minute, More Is BetterONNX Runtime 1.11Model: bertsquad-12 - Device: CPU - Executor: StandardAVX-512 OnAVX-512 Off110220330440550SE +/- 1.86, N = 3SE +/- 3.11, N = 3516499-mavx512f -mavx512cd -mavx512vl -mavx512bw -mavx512dq -mavx512ifma -mavx512vbmi-mno-avx512f1. (CXX) g++ options: -O3 -march=native -ffunction-sections -fdata-sections -mtune=native -flto=auto -fno-fat-lto-objects -ldl -lrt

OpenBenchmarking.orgInferences Per Minute, More Is BetterONNX Runtime 1.11Model: ArcFace ResNet-100 - Device: CPU - Executor: StandardAVX-512 OnAVX-512 Off2004006008001000SE +/- 39.92, N = 12SE +/- 20.83, N = 121051918-mavx512f -mavx512cd -mavx512vl -mavx512bw -mavx512dq -mavx512ifma -mavx512vbmi-mno-avx512f1. (CXX) g++ options: -O3 -march=native -ffunction-sections -fdata-sections -mtune=native -flto=auto -fno-fat-lto-objects -ldl -lrt

Numpy Benchmark

This is a test to obtain the general Numpy performance. Learn more via the OpenBenchmarking.org test page.

OpenBenchmarking.orgScore, More Is BetterNumpy BenchmarkAVX-512 OnAVX-512 Off120240360480600SE +/- 2.18, N = 3SE +/- 0.59, N = 3575.66548.83

miniBUDE

MiniBUDE is a mini application for the the core computation of the Bristol University Docking Engine (BUDE). This test profile currently makes use of the OpenMP implementation of miniBUDE for CPU benchmarking. Learn more via the OpenBenchmarking.org test page.

OpenBenchmarking.orgGFInst/s, More Is BetterminiBUDE 20210901Implementation: OpenMP - Input Deck: BM1AVX-512 OnAVX-512 Off16003200480064008000SE +/- 5.79, N = 10SE +/- 26.42, N = 87299.555065.101. (CC) gcc options: -std=c99 -Ofast -ffast-math -fopenmp -march=native -lm

OpenBenchmarking.orgBillion Interactions/s, More Is BetterminiBUDE 20210901Implementation: OpenMP - Input Deck: BM1AVX-512 OnAVX-512 Off60120180240300SE +/- 0.23, N = 10SE +/- 1.06, N = 8291.98202.601. (CC) gcc options: -std=c99 -Ofast -ffast-math -fopenmp -march=native -lm

OpenBenchmarking.orgGFInst/s, More Is BetterminiBUDE 20210901Implementation: OpenMP - Input Deck: BM2AVX-512 OnAVX-512 Off2K4K6K8K10KSE +/- 11.62, N = 4SE +/- 31.65, N = 38652.016391.521. (CC) gcc options: -std=c99 -Ofast -ffast-math -fopenmp -march=native -lm

OpenBenchmarking.orgBillion Interactions/s, More Is BetterminiBUDE 20210901Implementation: OpenMP - Input Deck: BM2AVX-512 OnAVX-512 Off80160240320400SE +/- 0.46, N = 4SE +/- 1.27, N = 3346.08255.661. (CC) gcc options: -std=c99 -Ofast -ffast-math -fopenmp -march=native -lm

SVT-AV1

This is a benchmark of the SVT-AV1 open-source video encoder/decoder. SVT-AV1 was originally developed by Intel as part of their Open Visual Cloud / Scalable Video Technology (SVT). Development of SVT-AV1 has since moved to the Alliance for Open Media as part of upstream AV1 development. SVT-AV1 is a CPU-based multi-threaded video encoder for the AV1 video format with a sample YUV video file. Learn more via the OpenBenchmarking.org test page.

OpenBenchmarking.orgFrames Per Second, More Is BetterSVT-AV1 1.4Encoder Mode: Preset 13 - Input: Bosphorus 4KAVX-512 OnAVX-512 Off50100150200250SE +/- 4.74, N = 15SE +/- 4.01, N = 15246.57239.88

OpenBenchmarking.orgFrames Per Second, More Is BetterSVT-AV1 1.4Encoder Mode: Preset 12 - Input: Bosphorus 4KAVX-512 OnAVX-512 Off50100150200250SE +/- 4.78, N = 15SE +/- 3.57, N = 15245.56233.94

Numenta Anomaly Benchmark

Numenta Anomaly Benchmark (NAB) is a benchmark for evaluating algorithms for anomaly detection in streaming, real-time applications. It is comprised of over 50 labeled real-world and artificial time-series data files plus a novel scoring mechanism designed for real-time applications. This test profile currently measures the time to run various detectors. Learn more via the OpenBenchmarking.org test page.

OpenBenchmarking.orgSeconds, Fewer Is BetterNumenta Anomaly Benchmark 1.1Detector: Bayesian ChangepointAVX-512 OnAVX-512 Off48121620SE +/- 0.21, N = 4SE +/- 0.19, N = 516.6817.39

OpenBenchmarking.orgSeconds, Fewer Is BetterNumenta Anomaly Benchmark 1.1Detector: Windowed GaussianAVX-512 OnAVX-512 Off1.09852.1973.29554.3945.4925SE +/- 0.032, N = 15SE +/- 0.044, N = 74.7274.882

OpenBenchmarking.orgSeconds, Fewer Is BetterNumenta Anomaly Benchmark 1.1Detector: Relative EntropyAVX-512 OnAVX-512 Off3691215SE +/- 0.089, N = 5SE +/- 0.082, N = 59.89810.035

Intel Open Image Denoise

Open Image Denoise is a denoising library for ray-tracing and part of the oneAPI rendering toolkit. Learn more via the OpenBenchmarking.org test page.

OpenBenchmarking.orgImages / Sec, More Is BetterIntel Open Image Denoise 1.4.0Run: RT.hdr_alb_nrm.3840x2160AVX-512 OnAVX-512 Off0.78981.57962.36943.15923.949SE +/- 0.01, N = 5SE +/- 0.03, N = 53.513.50

OpenBenchmarking.orgImages / Sec, More Is BetterIntel Open Image Denoise 1.4.0Run: RT.ldr_alb_nrm.3840x2160AVX-512 OnAVX-512 Off0.78981.57962.36943.15923.949SE +/- 0.01, N = 5SE +/- 0.01, N = 53.513.51

OpenBenchmarking.orgImages / Sec, More Is BetterIntel Open Image Denoise 1.4.0Run: RTLightmap.hdr.4096x4096AVX-512 OnAVX-512 Off0.3870.7741.1611.5481.935SE +/- 0.01, N = 3SE +/- 0.01, N = 31.661.72

Darmstadt Automotive Parallel Heterogeneous Suite

DAPHNE is the Darmstadt Automotive Parallel HeterogeNEous Benchmark Suite with OpenCL / CUDA / OpenMP test cases for these automotive benchmarks for evaluating programming models in context to vehicle autonomous driving capabilities. Learn more via the OpenBenchmarking.org test page.

OpenBenchmarking.orgTest Cases Per Minute, More Is BetterDarmstadt Automotive Parallel Heterogeneous SuiteBackend: OpenMP - Kernel: NDT MappingAVX-512 OnAVX-512 Off30060090012001500SE +/- 5.94, N = 3SE +/- 8.66, N = 31371.181335.631. (CXX) g++ options: -O3 -std=c++11 -fopenmp

OpenBenchmarking.orgTest Cases Per Minute, More Is BetterDarmstadt Automotive Parallel Heterogeneous SuiteBackend: OpenMP - Kernel: Points2ImageAVX-512 OnAVX-512 Off3K6K9K12K15KSE +/- 126.17, N = 15SE +/- 269.82, N = 1513677.7611521.381. (CXX) g++ options: -O3 -std=c++11 -fopenmp

simdjson

This is a benchmark of SIMDJSON, a high performance JSON parser. SIMDJSON aims to be the fastest JSON parser and is used by projects like Microsoft FishStore, Yandex ClickHouse, Shopify, and others. Learn more via the OpenBenchmarking.org test page.

OpenBenchmarking.orgGB/s, More Is Bettersimdjson 2.0Throughput Test: PartialTweetsAVX-512 OnAVX-512 Off246810SE +/- 0.04, N = 3SE +/- 0.01, N = 36.705.14-mavx512f -mavx512cd -mavx512vl -mavx512bw -mavx512dq -mavx512ifma -mavx512vbmi-mno-avx512f1. (CXX) g++ options: -O3 -march=native

OpenBenchmarking.orgGB/s, More Is Bettersimdjson 2.0Throughput Test: LargeRandomAVX-512 OnAVX-512 Off0.28350.5670.85051.1341.4175SE +/- 0.00, N = 3SE +/- 0.00, N = 31.261.04-mavx512f -mavx512cd -mavx512vl -mavx512bw -mavx512dq -mavx512ifma -mavx512vbmi-mno-avx512f1. (CXX) g++ options: -O3 -march=native

OpenBenchmarking.orgGB/s, More Is Bettersimdjson 2.0Throughput Test: KostyaAVX-512 OnAVX-512 Off0.94051.8812.82153.7624.7025SE +/- 0.02, N = 3SE +/- 0.00, N = 34.183.39-mavx512f -mavx512cd -mavx512vl -mavx512bw -mavx512dq -mavx512ifma -mavx512vbmi-mno-avx512f1. (CXX) g++ options: -O3 -march=native

OpenBenchmarking.orgGB/s, More Is Bettersimdjson 2.0Throughput Test: DistinctUserIDAVX-512 OnAVX-512 Off246810SE +/- 0.02, N = 3SE +/- 0.01, N = 36.865.25-mavx512f -mavx512cd -mavx512vl -mavx512bw -mavx512dq -mavx512ifma -mavx512vbmi-mno-avx512f1. (CXX) g++ options: -O3 -march=native

OpenBenchmarking.orgGB/s, More Is Bettersimdjson 2.0Throughput Test: TopTweetAVX-512 OnAVX-512 Off246810SE +/- 0.08, N = 4SE +/- 0.01, N = 36.525.38-mavx512f -mavx512cd -mavx512vl -mavx512bw -mavx512dq -mavx512ifma -mavx512vbmi-mno-avx512f1. (CXX) g++ options: -O3 -march=native

OpenFOAM

OpenFOAM is the leading free, open-source software for computational fluid dynamics (CFD). This test profile currently uses the drivaerFastback test case for analyzing automotive aerodynamics or alternatively the older motorBike input. Learn more via the OpenBenchmarking.org test page.

OpenBenchmarking.orgSeconds, Fewer Is BetterOpenFOAM 10Input: drivaerFastback, Medium Mesh Size - Mesh TimeAVX-512 OnAVX-512 Off306090120150135.77144.351. (CXX) g++ options: -std=c++14 -m64 -O3 -ftemplate-depth-100 -fPIC -fuse-ld=bfd -Xlinker --add-needed --no-as-needed -lfiniteVolume -lmeshTools -lparallel -llagrangian -lregionModels -lgenericPatchFields -lOpenFOAM -ldl -lm

OpenBenchmarking.orgSeconds, Fewer Is BetterOpenFOAM 10Input: drivaerFastback, Medium Mesh Size - Execution TimeAVX-512 OnAVX-512 Off306090120150113.64135.221. (CXX) g++ options: -std=c++14 -m64 -O3 -ftemplate-depth-100 -fPIC -fuse-ld=bfd -Xlinker --add-needed --no-as-needed -lfiniteVolume -lmeshTools -lparallel -llagrangian -lregionModels -lgenericPatchFields -lOpenFOAM -ldl -lm

CP2K Molecular Dynamics

CP2K is an open-source molecular dynamics software package focused on quantum chemistry and solid-state physics. This test profile currently uses the SSMP (OpenMP) version of cp2k. Learn more via the OpenBenchmarking.org test page.

OpenBenchmarking.orgSeconds, Fewer Is BetterCP2K Molecular Dynamics 8.2Input: Fayalite-FISTAVX-512 OnAVX-512 Off300600900120015001122.731198.41

SMHasher

SMHasher is a hash function tester supporting various algorithms and able to make use of AVX and other modern CPU instruction set extensions. Learn more via the OpenBenchmarking.org test page.

OpenBenchmarking.orgMiB/sec, More Is BetterSMHasher 2022-08-22Hash: FarmHash32 x86_64 AVXAVX-512 OnAVX-512 Off9K18K27K36K45KSE +/- 0.46, N = 5SE +/- 28.78, N = 539794.4436483.70-mavx512f -mavx512cd -mavx512vl -mavx512bw -mavx512dq -mavx512ifma -mavx512vbmi-mno-avx512f1. (CXX) g++ options: -O3 -march=native -flto=auto -fno-fat-lto-objects

JPEG XL libjxl

The JPEG XL Image Coding System is designed to provide next-generation JPEG image capabilities with JPEG XL offering better image quality and compression over legacy JPEG. This test profile is currently focused on the multi-threaded JPEG XL image encode performance using the reference libjxl library. Learn more via the OpenBenchmarking.org test page.

OpenBenchmarking.orgMP/s, More Is BetterJPEG XL libjxl 0.7Input: JPEG - Quality: 90AVX-512 OnAVX-512 Off3691215SE +/- 0.04, N = 3SE +/- 0.03, N = 39.559.13-mavx512f -mavx512cd -mavx512vl -mavx512bw -mavx512dq -mavx512ifma -mavx512vbmi-mno-avx512f1. (CXX) g++ options: -O3 -march=native -fno-rtti -funwind-tables -O2 -fPIE -pie -lm -latomic

OpenBenchmarking.orgMP/s, More Is BetterJPEG XL libjxl 0.7Input: JPEG - Quality: 100AVX-512 OnAVX-512 Off0.16650.3330.49950.6660.8325SE +/- 0.01, N = 9SE +/- 0.01, N = 90.740.72-mavx512f -mavx512cd -mavx512vl -mavx512bw -mavx512dq -mavx512ifma -mavx512vbmi-mno-avx512f1. (CXX) g++ options: -O3 -march=native -fno-rtti -funwind-tables -O2 -fPIE -pie -lm -latomic

OpenBenchmarking.orgMP/s, More Is BetterJPEG XL libjxl 0.7Input: PNG - Quality: 90AVX-512 OnAVX-512 Off3691215SE +/- 0.02, N = 3SE +/- 0.02, N = 39.959.52-mavx512f -mavx512cd -mavx512vl -mavx512bw -mavx512dq -mavx512ifma -mavx512vbmi-mno-avx512f1. (CXX) g++ options: -O3 -march=native -fno-rtti -funwind-tables -O2 -fPIE -pie -lm -latomic

CPU Power Consumption Monitor

OpenBenchmarking.orgWattsCPU Power Consumption MonitorPhoronix Test Suite System MonitoringAVX-512 OnAVX-512 Off130260390520650Min: 26.37 / Avg: 434.8 / Max: 766.01Min: 106.95 / Avg: 449.58 / Max: 735.32

CPU Temperature Monitor

OpenBenchmarking.orgCelsiusCPU Temperature MonitorPhoronix Test Suite System MonitoringAVX-512 OnAVX-512 Off1428425670Min: 30.13 / Avg: 49.97 / Max: 73.38Min: 35.5 / Avg: 51.26 / Max: 73.75

136 Results Shown

AI Benchmark Alpha:
  Device Inference Score
  Device Training Score
  Device AI Score
Neural Magic DeepSparse:
  NLP Text Classification, BERT base uncased SST2 - Synchronous Single-Stream:
    items/sec
    ms/batch
  NLP Text Classification, BERT base uncased SST2 - Asynchronous Multi-Stream:
    items/sec
    ms/batch
  NLP Text Classification, DistilBERT mnli - Synchronous Single-Stream:
    items/sec
    ms/batch
  NLP Text Classification, DistilBERT mnli - Asynchronous Multi-Stream:
    items/sec
    ms/batch
  CV Classification, ResNet-50 ImageNet - Asynchronous Multi-Stream:
    items/sec
    ms/batch
  NLP Token Classification, BERT base uncased conll2003 - Synchronous Single-Stream:
    items/sec
    ms/batch
  NLP Question Answering, BERT base uncased SQuaD 12layer Pruned90 - Synchronous Single-Stream:
    items/sec
    ms/batch
  NLP Question Answering, BERT base uncased SQuaD 12layer Pruned90 - Asynchronous Multi-Stream:
    items/sec
    ms/batch
  CV Detection,YOLOv5s COCO - Synchronous Single-Stream:
    items/sec
    ms/batch
  CV Detection,YOLOv5s COCO - Asynchronous Multi-Stream:
    items/sec
    ms/batch
  NLP Document Classification, oBERT base uncased on IMDB - Synchronous Single-Stream:
    items/sec
    ms/batch
TensorFlow:
  CPU - 16 - ResNet-50
  CPU - 16 - AlexNet
  CPU - 16 - GoogLeNet
LeelaChessZero:
  BLAS
  Eigen
Embree:
  Pathtracer ISPC - Asian Dragon
  Pathtracer ISPC - Asian Dragon Obj
  Pathtracer ISPC - Crown
OpenVKL
OSPRay:
  gravity_spheres_volume/dim_512/ao/real_time
  gravity_spheres_volume/dim_512/scivis/real_time
  gravity_spheres_volume/dim_512/pathtracer/real_time
OSPRay Studio:
  1 - 1080p - 1 - Path Tracer
  1 - 1080p - 16 - Path Tracer
  1 - 1080p - 32 - Path Tracer
  1 - 4K - 1 - Path Tracer
  1 - 4K - 16 - Path Tracer
  1 - 4K - 32 - Path Tracer
  2 - 1080p - 1 - Path Tracer
  2 - 1080p - 16 - Path Tracer
  2 - 1080p - 32 - Path Tracer
  2 - 4K - 1 - Path Tracer
  2 - 4K - 16 - Path Tracer
  2 - 4K - 32 - Path Tracer
  3 - 1080p - 1 - Path Tracer
  3 - 1080p - 16 - Path Tracer
  3 - 1080p - 32 - Path Tracer
  3 - 4K - 1 - Path Tracer
  3 - 4K - 16 - Path Tracer
  3 - 4K - 32 - Path Tracer
oneDNN:
  Convolution Batch Shapes Auto - f32 - CPU
  Deconvolution Batch shapes_1d - f32 - CPU
  Deconvolution Batch shapes_3d - f32 - CPU
  Recurrent Neural Network Training - f32 - CPU
  Recurrent Neural Network Training - bf16bf16bf16 - CPU
  Recurrent Neural Network Inference - bf16bf16bf16 - CPU
Mobile Neural Network:
  resnet-v2-50
  SqueezeNetV1.0
  inception-v3
Cpuminer-Opt:
  scrypt
  Quad SHA-256, Pyrite
  x25x
  Garlicoin
  Skeincoin
  LBC, LBRY Credits
NCNN:
  CPU - mnasnet
  CPU - efficientnet-b0
  CPU - blazeface
  CPU - googlenet
  CPU - resnet50
  CPU - regnety_400m
  CPU - vision_transformer
  CPU - FastestDet
OpenVINO:
  Face Detection FP16 - CPU:
    FPS
    ms
  Face Detection FP16-INT8 - CPU:
    FPS
    ms
  Age Gender Recognition Retail 0013 FP16 - CPU:
    FPS
    ms
  Age Gender Recognition Retail 0013 FP16-INT8 - CPU:
    FPS
    ms
  Person Detection FP16 - CPU:
    FPS
    ms
  Person Detection FP32 - CPU:
    FPS
    ms
  Weld Porosity Detection FP16-INT8 - CPU:
    FPS
    ms
  Weld Porosity Detection FP16 - CPU:
    FPS
    ms
  Vehicle Detection FP16-INT8 - CPU:
    FPS
    ms
  Vehicle Detection FP16 - CPU:
    FPS
    ms
  Person Vehicle Bike Detection FP16 - CPU:
    FPS
    ms
  Machine Translation EN To DE FP16 - CPU:
    FPS
    ms
GROMACS
ONNX Runtime:
  fcn-resnet101-11 - CPU - Standard
  super-resolution-10 - CPU - Standard
  bertsquad-12 - CPU - Standard
  ArcFace ResNet-100 - CPU - Standard
Numpy Benchmark
miniBUDE:
  OpenMP - BM1:
    GFInst/s
    Billion Interactions/s
  OpenMP - BM2:
    GFInst/s
    Billion Interactions/s
SVT-AV1:
  Preset 13 - Bosphorus 4K
  Preset 12 - Bosphorus 4K
Numenta Anomaly Benchmark:
  Bayesian Changepoint
  Windowed Gaussian
  Relative Entropy
Intel Open Image Denoise:
  RT.hdr_alb_nrm.3840x2160
  RT.ldr_alb_nrm.3840x2160
  RTLightmap.hdr.4096x4096
Darmstadt Automotive Parallel Heterogeneous Suite:
  OpenMP - NDT Mapping
  OpenMP - Points2Image
simdjson:
  PartialTweets
  LargeRand
  Kostya
  DistinctUserID
  TopTweet
OpenFOAM:
  drivaerFastback, Medium Mesh Size - Mesh Time
  drivaerFastback, Medium Mesh Size - Execution Time
CP2K Molecular Dynamics
SMHasher
JPEG XL libjxl:
  JPEG - 90
  JPEG - 100
  PNG - 90
CPU Power Consumption Monitor:
  Phoronix Test Suite System Monitoring:
    Watts
    Celsius