Intel Core i3-12100 testing with a ASRock H610M-HDV/M.2 R2.0 (6.03 BIOS) and Intel ADL-S GT1 3GB on Ubuntu 20.04 via the Phoronix Test Suite.
Compare your own system(s) to this result file with the
Phoronix Test Suite by running the command:
phoronix-test-suite benchmark 2408228-HERT-H610I3171 H610-i312100-1 - Phoronix Test Suite H610-i312100-1 Intel Core i3-12100 testing with a ASRock H610M-HDV/M.2 R2.0 (6.03 BIOS) and Intel ADL-S GT1 3GB on Ubuntu 20.04 via the Phoronix Test Suite.
HTML result view exported from: https://openbenchmarking.org/result/2408228-HERT-H610I3171&grs .
H610-i312100-1 Processor Motherboard Chipset Memory Disk Graphics Audio Network OS Kernel Desktop Display Server OpenGL Vulkan Compiler File-System Screen Resolution Intel ADL-S GT1 - Intel Core i3-12100 Intel Core i3-12100 @ 4.30GHz (4 Cores / 8 Threads) ASRock H610M-HDV/M.2 R2.0 (6.03 BIOS) Intel Device 7aa7 4096MB 1000GB Western Digital WDS100T2B0A Intel ADL-S GT1 3GB (1400MHz) Realtek ALC897 Realtek RTL8111/8168/8411 Ubuntu 20.04 5.15.0-89-generic (x86_64) GNOME Shell 3.36.9 X Server 1.20.13 4.6 Mesa 21.2.6 1.2.182 GCC 9.4.0 ext4 1366x768 OpenBenchmarking.org - Transparent Huge Pages: madvise - --build=x86_64-linux-gnu --disable-vtable-verify --disable-werror --enable-checking=release --enable-clocale=gnu --enable-default-pie --enable-gnu-unique-object --enable-languages=c,ada,c++,go,brig,d,fortran,objc,obj-c++,gm2 --enable-libstdcxx-debug --enable-libstdcxx-time=yes --enable-multiarch --enable-multilib --enable-nls --enable-objc-gc=auto --enable-offload-targets=nvptx-none=/build/gcc-9-9QDOt0/gcc-9-9.4.0/debian/tmp-nvptx/usr,hsa --enable-plugin --enable-shared --enable-threads=posix --host=x86_64-linux-gnu --program-prefix=x86_64-linux-gnu- --target=x86_64-linux-gnu --with-abi=m64 --with-arch-32=i686 --with-default-libstdcxx-abi=new --with-gcc-major-version-only --with-multilib-list=m32,m64,mx32 --with-target-system-zlib=auto --with-tune=generic --without-cuda-driver -v - Scaling Governor: intel_pstate powersave (EPP: balance_performance) - CPU Microcode: 0x2e - Python 3.8.10 - gather_data_sampling: Not affected + itlb_multihit: Not affected + l1tf: Not affected + mds: Not affected + meltdown: Not affected + mmio_stale_data: Not affected + retbleed: Not affected + spec_rstack_overflow: Not affected + spec_store_bypass: Mitigation of SSB disabled via prctl and seccomp + spectre_v1: Mitigation of usercopy/swapgs barriers and __user pointer sanitization + spectre_v2: Mitigation of Enhanced IBRS IBPB: conditional RSB filling PBRSB-eIBRS: SW sequence + srbds: Not affected + tsx_async_abort: Not affected
H610-i312100-1 whisper-cpp: ggml-medium.en - 2016 State of the Union whisper-cpp: ggml-small.en - 2016 State of the Union whisper-cpp: ggml-base.en - 2016 State of the Union scikit-learn: Sparse Rand Projections / 100 Iterations scikit-learn: Kernel PCA Solvers / Time vs. N Samples scikit-learn: Hist Gradient Boosting Categorical Only scikit-learn: Plot Polynomial Kernel Approximation scikit-learn: 20 Newsgroups / Logistic Regression scikit-learn: Hist Gradient Boosting Higgs Boson scikit-learn: Plot Singular Value Decomposition scikit-learn: Hist Gradient Boosting Threading scikit-learn: Hist Gradient Boosting Adult scikit-learn: Covertype Dataset Benchmark scikit-learn: Sample Without Replacement scikit-learn: Hist Gradient Boosting scikit-learn: Plot Incremental PCA scikit-learn: TSNE MNIST Dataset scikit-learn: LocalOutlierFactor scikit-learn: Feature Expansions scikit-learn: Plot OMP vs. LARS scikit-learn: Plot Hierarchical scikit-learn: Text Vectorizers scikit-learn: Plot Lasso Path scikit-learn: SGD Regression scikit-learn: Plot Neighbors scikit-learn: MNIST Dataset scikit-learn: Plot Ward scikit-learn: Sparsify scikit-learn: Lasso scikit-learn: Tree scikit-learn: SAGA scikit-learn: GLM mlpack: scikit_svm mlpack: scikit_ica onnx: Faster R-CNN R-50-FPN-int8 - CPU - Standard onnx: Faster R-CNN R-50-FPN-int8 - CPU - Parallel onnx: super-resolution-10 - CPU - Parallel onnx: ResNet50 v1-12-int8 - CPU - Standard onnx: ResNet50 v1-12-int8 - CPU - Parallel onnx: ArcFace ResNet-100 - CPU - Standard onnx: ArcFace ResNet-100 - CPU - Parallel onnx: CaffeNet 12-int8 - CPU - Standard onnx: CaffeNet 12-int8 - CPU - Parallel onnx: bertsquad-12 - CPU - Parallel onnx: T5 Encoder - CPU - Standard onnx: T5 Encoder - CPU - Parallel onnx: GPT-2 - CPU - Standard onnx: GPT-2 - CPU - Parallel numenta-nab: Contextual Anomaly Detector OSE numenta-nab: Bayesian Changepoint numenta-nab: Earthgecko Skyline numenta-nab: Windowed Gaussian numenta-nab: Relative Entropy numenta-nab: KNN CAD openvino: Age Gender Recognition Retail 0013 FP16-INT8 - CPU openvino: Age Gender Recognition Retail 0013 FP16-INT8 - CPU openvino: Handwritten English Recognition FP16-INT8 - CPU openvino: Handwritten English Recognition FP16-INT8 - CPU openvino: Age Gender Recognition Retail 0013 FP16 - CPU openvino: Age Gender Recognition Retail 0013 FP16 - CPU openvino: Person Re-Identification Retail FP16 - CPU openvino: Person Re-Identification Retail FP16 - CPU openvino: Handwritten English Recognition FP16 - CPU openvino: Handwritten English Recognition FP16 - CPU openvino: Noise Suppression Poconet-Like FP16 - CPU openvino: Noise Suppression Poconet-Like FP16 - CPU openvino: Person Vehicle Bike Detection FP16 - CPU openvino: Person Vehicle Bike Detection FP16 - CPU openvino: Weld Porosity Detection FP16-INT8 - CPU openvino: Weld Porosity Detection FP16-INT8 - CPU openvino: Machine Translation EN To DE FP16 - CPU openvino: Machine Translation EN To DE FP16 - CPU openvino: Road Segmentation ADAS FP16-INT8 - CPU openvino: Road Segmentation ADAS FP16-INT8 - CPU openvino: Face Detection Retail FP16-INT8 - CPU openvino: Face Detection Retail FP16-INT8 - CPU openvino: Weld Porosity Detection FP16 - CPU openvino: Weld Porosity Detection FP16 - CPU openvino: Vehicle Detection FP16-INT8 - CPU openvino: Vehicle Detection FP16-INT8 - CPU openvino: Road Segmentation ADAS FP16 - CPU openvino: Road Segmentation ADAS FP16 - CPU openvino: Face Detection Retail FP16 - CPU openvino: Face Detection Retail FP16 - CPU openvino: Face Detection FP16-INT8 - CPU openvino: Face Detection FP16-INT8 - CPU openvino: Vehicle Detection FP16 - CPU openvino: Vehicle Detection FP16 - CPU openvino: Person Detection FP32 - CPU openvino: Person Detection FP32 - CPU openvino: Person Detection FP16 - CPU openvino: Person Detection FP16 - CPU openvino: Face Detection FP16 - CPU openvino: Face Detection FP16 - CPU plaidml: No - Inference - ResNet 50 - CPU plaidml: No - Inference - VGG16 - CPU xnnpack: QU8MobileNetV3Small xnnpack: QU8MobileNetV3Large xnnpack: QU8MobileNetV2 xnnpack: FP16MobileNetV3Small xnnpack: FP16MobileNetV3Large xnnpack: FP16MobileNetV2 xnnpack: FP32MobileNetV3Small xnnpack: FP32MobileNetV3Large xnnpack: FP32MobileNetV2 tnn: CPU - SqueezeNet v1.1 tnn: CPU - SqueezeNet v2 tnn: CPU - MobileNet v2 tnn: CPU - DenseNet ncnn: Vulkan GPU - FastestDet ncnn: Vulkan GPU - vision_transformer ncnn: Vulkan GPU - regnety_400m ncnn: Vulkan GPU - squeezenet_ssd ncnn: Vulkan GPU - yolov4-tiny ncnn: Vulkan GPU - resnet50 ncnn: Vulkan GPU - alexnet ncnn: Vulkan GPU - resnet18 ncnn: Vulkan GPU - vgg16 ncnn: Vulkan GPU - googlenet ncnn: Vulkan GPU - blazeface ncnn: Vulkan GPU - efficientnet-b0 ncnn: Vulkan GPU - mnasnet ncnn: Vulkan GPU - shufflenet-v2 ncnn: Vulkan GPU-v3-v3 - mobilenet-v3 ncnn: Vulkan GPU-v2-v2 - mobilenet-v2 ncnn: Vulkan GPU - mobilenet ncnn: CPU - FastestDet ncnn: CPU - vision_transformer ncnn: CPU - regnety_400m ncnn: CPU - squeezenet_ssd ncnn: CPU - yolov4-tiny ncnn: CPU - resnet50 ncnn: CPU - alexnet ncnn: CPU - resnet18 ncnn: CPU - vgg16 ncnn: CPU - googlenet ncnn: CPU - blazeface ncnn: CPU - efficientnet-b0 ncnn: CPU - shufflenet-v2 ncnn: CPU-v3-v3 - mobilenet-v3 ncnn: CPU-v2-v2 - mobilenet-v2 ncnn: CPU - mobilenet mnn: inception-v3 mnn: mobilenet-v1-1.0 mnn: MobileNetV2_224 mnn: SqueezeNetV1.0 mnn: resnet-v2-50 mnn: squeezenetv1.1 mnn: nasnet caffe: GoogleNet - CPU - 1000 caffe: GoogleNet - CPU - 200 caffe: GoogleNet - CPU - 100 caffe: AlexNet - CPU - 1000 caffe: AlexNet - CPU - 200 caffe: AlexNet - CPU - 100 deepsparse: NLP Token Classification, BERT base uncased conll2003 - Synchronous Single-Stream deepsparse: NLP Token Classification, BERT base uncased conll2003 - Synchronous Single-Stream deepsparse: NLP Token Classification, BERT base uncased conll2003 - Asynchronous Multi-Stream deepsparse: NLP Token Classification, BERT base uncased conll2003 - Asynchronous Multi-Stream deepsparse: BERT-Large, NLP Question Answering, Sparse INT8 - Synchronous Single-Stream deepsparse: BERT-Large, NLP Question Answering, Sparse INT8 - Synchronous Single-Stream deepsparse: BERT-Large, NLP Question Answering, Sparse INT8 - Asynchronous Multi-Stream deepsparse: BERT-Large, NLP Question Answering, Sparse INT8 - Asynchronous Multi-Stream deepsparse: CV Segmentation, 90% Pruned YOLACT Pruned - Synchronous Single-Stream deepsparse: CV Segmentation, 90% Pruned YOLACT Pruned - Synchronous Single-Stream deepsparse: CV Segmentation, 90% Pruned YOLACT Pruned - Asynchronous Multi-Stream deepsparse: CV Segmentation, 90% Pruned YOLACT Pruned - Asynchronous Multi-Stream deepsparse: NLP Text Classification, DistilBERT mnli - Synchronous Single-Stream deepsparse: NLP Text Classification, DistilBERT mnli - Synchronous Single-Stream deepsparse: NLP Text Classification, DistilBERT mnli - Asynchronous Multi-Stream deepsparse: NLP Text Classification, DistilBERT mnli - Asynchronous Multi-Stream deepsparse: CV Detection, YOLOv5s COCO, Sparse INT8 - Synchronous Single-Stream deepsparse: CV Detection, YOLOv5s COCO, Sparse INT8 - Synchronous Single-Stream deepsparse: CV Detection, YOLOv5s COCO, Sparse INT8 - Asynchronous Multi-Stream deepsparse: CV Detection, YOLOv5s COCO, Sparse INT8 - Asynchronous Multi-Stream deepsparse: CV Classification, ResNet-50 ImageNet - Synchronous Single-Stream deepsparse: CV Classification, ResNet-50 ImageNet - Synchronous Single-Stream deepsparse: CV Classification, ResNet-50 ImageNet - Asynchronous Multi-Stream deepsparse: CV Classification, ResNet-50 ImageNet - Asynchronous Multi-Stream deepsparse: Llama2 Chat 7b Quantized - Synchronous Single-Stream deepsparse: Llama2 Chat 7b Quantized - Synchronous Single-Stream deepsparse: Llama2 Chat 7b Quantized - Asynchronous Multi-Stream deepsparse: Llama2 Chat 7b Quantized - Asynchronous Multi-Stream deepsparse: ResNet-50, Sparse INT8 - Synchronous Single-Stream deepsparse: ResNet-50, Sparse INT8 - Synchronous Single-Stream deepsparse: ResNet-50, Sparse INT8 - Asynchronous Multi-Stream deepsparse: ResNet-50, Sparse INT8 - Asynchronous Multi-Stream deepsparse: ResNet-50, Baseline - Synchronous Single-Stream deepsparse: ResNet-50, Baseline - Synchronous Single-Stream deepsparse: ResNet-50, Baseline - Asynchronous Multi-Stream deepsparse: ResNet-50, Baseline - Asynchronous Multi-Stream deepsparse: NLP Text Classification, BERT base uncased SST2, Sparse INT8 - Synchronous Single-Stream deepsparse: NLP Text Classification, BERT base uncased SST2, Sparse INT8 - Synchronous Single-Stream deepsparse: NLP Text Classification, BERT base uncased SST2, Sparse INT8 - Asynchronous Multi-Stream deepsparse: NLP Text Classification, BERT base uncased SST2, Sparse INT8 - Asynchronous Multi-Stream deepsparse: NLP Document Classification, oBERT base uncased on IMDB - Synchronous Single-Stream deepsparse: NLP Document Classification, oBERT base uncased on IMDB - Synchronous Single-Stream deepsparse: NLP Document Classification, oBERT base uncased on IMDB - Asynchronous Multi-Stream deepsparse: NLP Document Classification, oBERT base uncased on IMDB - Asynchronous Multi-Stream pytorch: CPU - 512 - Efficientnet_v2_l pytorch: CPU - 256 - Efficientnet_v2_l pytorch: CPU - 64 - Efficientnet_v2_l pytorch: CPU - 32 - Efficientnet_v2_l pytorch: CPU - 16 - Efficientnet_v2_l pytorch: CPU - 1 - Efficientnet_v2_l pytorch: CPU - 512 - ResNet-152 pytorch: CPU - 256 - ResNet-152 pytorch: CPU - 64 - ResNet-152 pytorch: CPU - 512 - ResNet-50 pytorch: CPU - 32 - ResNet-152 pytorch: CPU - 256 - ResNet-50 pytorch: CPU - 16 - ResNet-152 pytorch: CPU - 64 - ResNet-50 pytorch: CPU - 32 - ResNet-50 pytorch: CPU - 16 - ResNet-50 pytorch: CPU - 1 - ResNet-152 pytorch: CPU - 1 - ResNet-50 tensorflow-lite: Inception ResNet V2 tensorflow-lite: Mobilenet Quant tensorflow-lite: Mobilenet Float tensorflow-lite: NASNet Mobile tensorflow-lite: Inception V4 tensorflow-lite: SqueezeNet rbenchmark: deepspeech: CPU numpy: onednn: Recurrent Neural Network Inference - CPU onednn: Recurrent Neural Network Training - CPU onednn: Deconvolution Batch shapes_3d - CPU onednn: Deconvolution Batch shapes_1d - CPU onednn: Convolution Batch Shapes Auto - CPU onednn: IP Shapes 3D - CPU onednn: IP Shapes 1D - CPU opencv: DNN - Deep Neural Network scikit-learn: Kernel PCA Solvers / Time vs. N Components onnx: Faster R-CNN R-50-FPN-int8 - CPU - Standard onnx: Faster R-CNN R-50-FPN-int8 - CPU - Parallel onnx: super-resolution-10 - CPU - Standard onnx: super-resolution-10 - CPU - Standard onnx: super-resolution-10 - CPU - Parallel onnx: ResNet50 v1-12-int8 - CPU - Standard onnx: ResNet50 v1-12-int8 - CPU - Parallel onnx: ArcFace ResNet-100 - CPU - Standard onnx: ArcFace ResNet-100 - CPU - Parallel onnx: fcn-resnet101-11 - CPU - Standard onnx: fcn-resnet101-11 - CPU - Standard onnx: fcn-resnet101-11 - CPU - Parallel onnx: fcn-resnet101-11 - CPU - Parallel onnx: CaffeNet 12-int8 - CPU - Standard onnx: CaffeNet 12-int8 - CPU - Parallel onnx: bertsquad-12 - CPU - Standard onnx: bertsquad-12 - CPU - Standard onnx: bertsquad-12 - CPU - Parallel onnx: T5 Encoder - CPU - Standard onnx: T5 Encoder - CPU - Parallel onnx: GPT-2 - CPU - Standard onnx: GPT-2 - CPU - Parallel ncnn: CPU - mnasnet mnn: mobilenetV3 lczero: BLAS Intel ADL-S GT1 - Intel Core i3-12100 2280.50283 753.36065 246.28971 905.082 276.878 14.585 233.561 49.524 138.505 199.818 336.099 60.214 446.723 95.571 96.579 45.043 324.552 83.873 241.475 134.664 186.554 51.999 226.078 119.971 105.444 64.562 52.219 74.982 492.674 39.661 728.144 611.356 14.51 64.74 5.10110 4.43575 38.0422 142.738 122.008 17.3121 11.5402 233.254 209.447 5.34494 57.5920 56.0251 51.1188 50.3149 52.860 47.477 209.161 11.696 20.596 245.088 0.37 10592.67 45.22 88.41 1.14 3481.92 19.80 201.90 60.21 66.40 19.10 208.84 23.99 166.71 7.79 513.04 267.44 14.95 63.27 63.19 4.80 833.34 29.24 136.74 18.66 214.21 219.10 18.24 12.09 330.42 759.24 5.26 63.85 62.61 398.50 10.04 397.47 10.06 3213.28 1.24 4.62 8.86 728 1974 2163 965 3396 3444 1302 5484 5545 166.131 46.205 201.109 2446.578 3.30 136.23 6.53 17.49 39.80 26.97 12.54 12.25 111.12 15.83 0.61 7.99 3.19 2.21 3.19 5.29 21.05 3.31 135.49 6.57 17.42 40.12 26.71 12.63 12.34 109.92 16.13 0.62 8.15 2.25 3.18 5.35 21.50 33.507 3.373 2.705 4.744 26.312 2.900 9.072 1038320 206462 103411 445933 90229 44622 230.8279 4.3321 485.3757 4.1179 15.4107 64.8531 25.5563 78.1891 207.6729 4.8149 399.5003 5.0039 31.0112 32.2416 58.0973 34.4105 42.6304 23.4544 83.7416 23.8766 21.0587 47.4739 38.7596 51.5732 108889.8452 0.0092 211288.5780 0.0070 3.3220 300.1121 5.3924 369.3583 21.0243 47.5516 38.6775 51.6867 6.8777 145.2740 12.1290 164.6514 231.8064 4.3138 490.7977 4.0749 4.48 4.50 4.52 4.55 4.51 8.10 6.17 6.22 6.23 12.76 6.24 12.91 6.33 12.69 12.56 12.57 11.34 24.89 60949.0 5198.07 3574.97 12005.2 64931.8 4322.71 0.1847 87.51840 463.17 3071.31 5708.97 10.3718 10.7533 44.1438 26.6998 6.00601 27185 319.472 196.432 225.465 19.2123 53.2627 26.2946 7.00536 8.19737 57.7900 86.6977 1353.24 0.742532 1975.36 0.512476 4.28771 4.77472 130.814 7.91502 187.689 17.3627 17.8485 19.5609 19.8719 3.53 1.188 OpenBenchmarking.org
Whisper.cpp Model: ggml-medium.en - Input: 2016 State of the Union OpenBenchmarking.org Seconds, Fewer Is Better Whisper.cpp 1.6.2 Model: ggml-medium.en - Input: 2016 State of the Union Intel ADL-S GT1 - Intel Core i3-12100 500 1000 1500 2000 2500 SE +/- 3.55, N = 3 2280.50 1. (CXX) g++ options: -O3 -std=c++11 -fPIC -pthread -msse3 -mssse3 -mavx -mf16c -mfma -mavx2
Whisper.cpp Model: ggml-small.en - Input: 2016 State of the Union OpenBenchmarking.org Seconds, Fewer Is Better Whisper.cpp 1.6.2 Model: ggml-small.en - Input: 2016 State of the Union Intel ADL-S GT1 - Intel Core i3-12100 160 320 480 640 800 SE +/- 0.45, N = 3 753.36 1. (CXX) g++ options: -O3 -std=c++11 -fPIC -pthread -msse3 -mssse3 -mavx -mf16c -mfma -mavx2
Whisper.cpp Model: ggml-base.en - Input: 2016 State of the Union OpenBenchmarking.org Seconds, Fewer Is Better Whisper.cpp 1.6.2 Model: ggml-base.en - Input: 2016 State of the Union Intel ADL-S GT1 - Intel Core i3-12100 50 100 150 200 250 SE +/- 0.15, N = 3 246.29 1. (CXX) g++ options: -O3 -std=c++11 -fPIC -pthread -msse3 -mssse3 -mavx -mf16c -mfma -mavx2
Scikit-Learn Benchmark: Sparse Random Projections / 100 Iterations OpenBenchmarking.org Seconds, Fewer Is Better Scikit-Learn 1.2.2 Benchmark: Sparse Random Projections / 100 Iterations Intel ADL-S GT1 - Intel Core i3-12100 200 400 600 800 1000 SE +/- 0.87, N = 3 905.08 1. (F9X) gfortran options: -O0
Scikit-Learn Benchmark: Kernel PCA Solvers / Time vs. N Samples OpenBenchmarking.org Seconds, Fewer Is Better Scikit-Learn 1.2.2 Benchmark: Kernel PCA Solvers / Time vs. N Samples Intel ADL-S GT1 - Intel Core i3-12100 60 120 180 240 300 SE +/- 0.57, N = 3 276.88 1. (F9X) gfortran options: -O0
Scikit-Learn Benchmark: Hist Gradient Boosting Categorical Only OpenBenchmarking.org Seconds, Fewer Is Better Scikit-Learn 1.2.2 Benchmark: Hist Gradient Boosting Categorical Only Intel ADL-S GT1 - Intel Core i3-12100 4 8 12 16 20 SE +/- 0.10, N = 3 14.59 1. (F9X) gfortran options: -O0
Scikit-Learn Benchmark: Plot Polynomial Kernel Approximation OpenBenchmarking.org Seconds, Fewer Is Better Scikit-Learn 1.2.2 Benchmark: Plot Polynomial Kernel Approximation Intel ADL-S GT1 - Intel Core i3-12100 50 100 150 200 250 SE +/- 1.21, N = 3 233.56 1. (F9X) gfortran options: -O0
Scikit-Learn Benchmark: 20 Newsgroups / Logistic Regression OpenBenchmarking.org Seconds, Fewer Is Better Scikit-Learn 1.2.2 Benchmark: 20 Newsgroups / Logistic Regression Intel ADL-S GT1 - Intel Core i3-12100 11 22 33 44 55 SE +/- 0.02, N = 3 49.52 1. (F9X) gfortran options: -O0
Scikit-Learn Benchmark: Hist Gradient Boosting Higgs Boson OpenBenchmarking.org Seconds, Fewer Is Better Scikit-Learn 1.2.2 Benchmark: Hist Gradient Boosting Higgs Boson Intel ADL-S GT1 - Intel Core i3-12100 30 60 90 120 150 SE +/- 1.02, N = 12 138.51 1. (F9X) gfortran options: -O0
Scikit-Learn Benchmark: Plot Singular Value Decomposition OpenBenchmarking.org Seconds, Fewer Is Better Scikit-Learn 1.2.2 Benchmark: Plot Singular Value Decomposition Intel ADL-S GT1 - Intel Core i3-12100 40 80 120 160 200 SE +/- 0.02, N = 3 199.82 1. (F9X) gfortran options: -O0
Scikit-Learn Benchmark: Hist Gradient Boosting Threading OpenBenchmarking.org Seconds, Fewer Is Better Scikit-Learn 1.2.2 Benchmark: Hist Gradient Boosting Threading Intel ADL-S GT1 - Intel Core i3-12100 70 140 210 280 350 SE +/- 3.75, N = 3 336.10 1. (F9X) gfortran options: -O0
Scikit-Learn Benchmark: Hist Gradient Boosting Adult OpenBenchmarking.org Seconds, Fewer Is Better Scikit-Learn 1.2.2 Benchmark: Hist Gradient Boosting Adult Intel ADL-S GT1 - Intel Core i3-12100 13 26 39 52 65 SE +/- 0.09, N = 3 60.21 1. (F9X) gfortran options: -O0
Scikit-Learn Benchmark: Covertype Dataset Benchmark OpenBenchmarking.org Seconds, Fewer Is Better Scikit-Learn 1.2.2 Benchmark: Covertype Dataset Benchmark Intel ADL-S GT1 - Intel Core i3-12100 100 200 300 400 500 SE +/- 0.14, N = 3 446.72 1. (F9X) gfortran options: -O0
Scikit-Learn Benchmark: Sample Without Replacement OpenBenchmarking.org Seconds, Fewer Is Better Scikit-Learn 1.2.2 Benchmark: Sample Without Replacement Intel ADL-S GT1 - Intel Core i3-12100 20 40 60 80 100 SE +/- 0.11, N = 3 95.57 1. (F9X) gfortran options: -O0
Scikit-Learn Benchmark: Hist Gradient Boosting OpenBenchmarking.org Seconds, Fewer Is Better Scikit-Learn 1.2.2 Benchmark: Hist Gradient Boosting Intel ADL-S GT1 - Intel Core i3-12100 20 40 60 80 100 SE +/- 0.18, N = 3 96.58 1. (F9X) gfortran options: -O0
Scikit-Learn Benchmark: Plot Incremental PCA OpenBenchmarking.org Seconds, Fewer Is Better Scikit-Learn 1.2.2 Benchmark: Plot Incremental PCA Intel ADL-S GT1 - Intel Core i3-12100 10 20 30 40 50 SE +/- 0.33, N = 3 45.04 1. (F9X) gfortran options: -O0
Scikit-Learn Benchmark: TSNE MNIST Dataset OpenBenchmarking.org Seconds, Fewer Is Better Scikit-Learn 1.2.2 Benchmark: TSNE MNIST Dataset Intel ADL-S GT1 - Intel Core i3-12100 70 140 210 280 350 SE +/- 0.43, N = 3 324.55 1. (F9X) gfortran options: -O0
Scikit-Learn Benchmark: LocalOutlierFactor OpenBenchmarking.org Seconds, Fewer Is Better Scikit-Learn 1.2.2 Benchmark: LocalOutlierFactor Intel ADL-S GT1 - Intel Core i3-12100 20 40 60 80 100 SE +/- 0.08, N = 3 83.87 1. (F9X) gfortran options: -O0
Scikit-Learn Benchmark: Feature Expansions OpenBenchmarking.org Seconds, Fewer Is Better Scikit-Learn 1.2.2 Benchmark: Feature Expansions Intel ADL-S GT1 - Intel Core i3-12100 50 100 150 200 250 SE +/- 0.42, N = 3 241.48 1. (F9X) gfortran options: -O0
Scikit-Learn Benchmark: Plot OMP vs. LARS OpenBenchmarking.org Seconds, Fewer Is Better Scikit-Learn 1.2.2 Benchmark: Plot OMP vs. LARS Intel ADL-S GT1 - Intel Core i3-12100 30 60 90 120 150 SE +/- 0.10, N = 3 134.66 1. (F9X) gfortran options: -O0
Scikit-Learn Benchmark: Plot Hierarchical OpenBenchmarking.org Seconds, Fewer Is Better Scikit-Learn 1.2.2 Benchmark: Plot Hierarchical Intel ADL-S GT1 - Intel Core i3-12100 40 80 120 160 200 SE +/- 0.37, N = 3 186.55 1. (F9X) gfortran options: -O0
Scikit-Learn Benchmark: Text Vectorizers OpenBenchmarking.org Seconds, Fewer Is Better Scikit-Learn 1.2.2 Benchmark: Text Vectorizers Intel ADL-S GT1 - Intel Core i3-12100 12 24 36 48 60 SE +/- 0.05, N = 3 52.00 1. (F9X) gfortran options: -O0
Scikit-Learn Benchmark: Plot Lasso Path OpenBenchmarking.org Seconds, Fewer Is Better Scikit-Learn 1.2.2 Benchmark: Plot Lasso Path Intel ADL-S GT1 - Intel Core i3-12100 50 100 150 200 250 SE +/- 1.40, N = 3 226.08 1. (F9X) gfortran options: -O0
Scikit-Learn Benchmark: SGD Regression OpenBenchmarking.org Seconds, Fewer Is Better Scikit-Learn 1.2.2 Benchmark: SGD Regression Intel ADL-S GT1 - Intel Core i3-12100 30 60 90 120 150 SE +/- 0.56, N = 3 119.97 1. (F9X) gfortran options: -O0
Scikit-Learn Benchmark: Plot Neighbors OpenBenchmarking.org Seconds, Fewer Is Better Scikit-Learn 1.2.2 Benchmark: Plot Neighbors Intel ADL-S GT1 - Intel Core i3-12100 20 40 60 80 100 SE +/- 0.15, N = 3 105.44 1. (F9X) gfortran options: -O0
Scikit-Learn Benchmark: MNIST Dataset OpenBenchmarking.org Seconds, Fewer Is Better Scikit-Learn 1.2.2 Benchmark: MNIST Dataset Intel ADL-S GT1 - Intel Core i3-12100 14 28 42 56 70 SE +/- 0.04, N = 3 64.56 1. (F9X) gfortran options: -O0
Scikit-Learn Benchmark: Plot Ward OpenBenchmarking.org Seconds, Fewer Is Better Scikit-Learn 1.2.2 Benchmark: Plot Ward Intel ADL-S GT1 - Intel Core i3-12100 12 24 36 48 60 SE +/- 0.10, N = 3 52.22 1. (F9X) gfortran options: -O0
Scikit-Learn Benchmark: Sparsify OpenBenchmarking.org Seconds, Fewer Is Better Scikit-Learn 1.2.2 Benchmark: Sparsify Intel ADL-S GT1 - Intel Core i3-12100 20 40 60 80 100 SE +/- 0.27, N = 3 74.98 1. (F9X) gfortran options: -O0
Scikit-Learn Benchmark: Lasso OpenBenchmarking.org Seconds, Fewer Is Better Scikit-Learn 1.2.2 Benchmark: Lasso Intel ADL-S GT1 - Intel Core i3-12100 110 220 330 440 550 SE +/- 1.04, N = 3 492.67 1. (F9X) gfortran options: -O0
Scikit-Learn Benchmark: Tree OpenBenchmarking.org Seconds, Fewer Is Better Scikit-Learn 1.2.2 Benchmark: Tree Intel ADL-S GT1 - Intel Core i3-12100 9 18 27 36 45 SE +/- 0.30, N = 15 39.66 1. (F9X) gfortran options: -O0
Scikit-Learn Benchmark: SAGA OpenBenchmarking.org Seconds, Fewer Is Better Scikit-Learn 1.2.2 Benchmark: SAGA Intel ADL-S GT1 - Intel Core i3-12100 160 320 480 640 800 SE +/- 0.42, N = 3 728.14 1. (F9X) gfortran options: -O0
Scikit-Learn Benchmark: GLM OpenBenchmarking.org Seconds, Fewer Is Better Scikit-Learn 1.2.2 Benchmark: GLM Intel ADL-S GT1 - Intel Core i3-12100 130 260 390 520 650 SE +/- 0.96, N = 3 611.36 1. (F9X) gfortran options: -O0
Mlpack Benchmark Benchmark: scikit_svm OpenBenchmarking.org Seconds, Fewer Is Better Mlpack Benchmark Benchmark: scikit_svm Intel ADL-S GT1 - Intel Core i3-12100 4 8 12 16 20 SE +/- 0.03, N = 3 14.51
Mlpack Benchmark Benchmark: scikit_ica OpenBenchmarking.org Seconds, Fewer Is Better Mlpack Benchmark Benchmark: scikit_ica Intel ADL-S GT1 - Intel Core i3-12100 14 28 42 56 70 SE +/- 0.72, N = 4 64.74
ONNX Runtime Model: Faster R-CNN R-50-FPN-int8 - Device: CPU - Executor: Standard OpenBenchmarking.org Inferences Per Second, More Is Better ONNX Runtime 1.17 Model: Faster R-CNN R-50-FPN-int8 - Device: CPU - Executor: Standard Intel ADL-S GT1 - Intel Core i3-12100 1.1477 2.2954 3.4431 4.5908 5.7385 SE +/- 0.06535, N = 12 5.10110 1. (CXX) g++ options: -O3 -march=native -ffunction-sections -fdata-sections -mtune=native -flto -fno-fat-lto-objects -ldl -lrt -lpthread -pthread
ONNX Runtime Model: Faster R-CNN R-50-FPN-int8 - Device: CPU - Executor: Parallel OpenBenchmarking.org Inferences Per Second, More Is Better ONNX Runtime 1.17 Model: Faster R-CNN R-50-FPN-int8 - Device: CPU - Executor: Parallel Intel ADL-S GT1 - Intel Core i3-12100 0.998 1.996 2.994 3.992 4.99 SE +/- 0.03335, N = 3 4.43575 1. (CXX) g++ options: -O3 -march=native -ffunction-sections -fdata-sections -mtune=native -flto -fno-fat-lto-objects -ldl -lrt -lpthread -pthread
ONNX Runtime Model: super-resolution-10 - Device: CPU - Executor: Parallel OpenBenchmarking.org Inferences Per Second, More Is Better ONNX Runtime 1.17 Model: super-resolution-10 - Device: CPU - Executor: Parallel Intel ADL-S GT1 - Intel Core i3-12100 9 18 27 36 45 SE +/- 0.42, N = 4 38.04 1. (CXX) g++ options: -O3 -march=native -ffunction-sections -fdata-sections -mtune=native -flto -fno-fat-lto-objects -ldl -lrt -lpthread -pthread
ONNX Runtime Model: ResNet50 v1-12-int8 - Device: CPU - Executor: Standard OpenBenchmarking.org Inferences Per Second, More Is Better ONNX Runtime 1.17 Model: ResNet50 v1-12-int8 - Device: CPU - Executor: Standard Intel ADL-S GT1 - Intel Core i3-12100 30 60 90 120 150 SE +/- 0.84, N = 3 142.74 1. (CXX) g++ options: -O3 -march=native -ffunction-sections -fdata-sections -mtune=native -flto -fno-fat-lto-objects -ldl -lrt -lpthread -pthread
ONNX Runtime Model: ResNet50 v1-12-int8 - Device: CPU - Executor: Parallel OpenBenchmarking.org Inferences Per Second, More Is Better ONNX Runtime 1.17 Model: ResNet50 v1-12-int8 - Device: CPU - Executor: Parallel Intel ADL-S GT1 - Intel Core i3-12100 30 60 90 120 150 SE +/- 1.34, N = 3 122.01 1. (CXX) g++ options: -O3 -march=native -ffunction-sections -fdata-sections -mtune=native -flto -fno-fat-lto-objects -ldl -lrt -lpthread -pthread
ONNX Runtime Model: ArcFace ResNet-100 - Device: CPU - Executor: Standard OpenBenchmarking.org Inferences Per Second, More Is Better ONNX Runtime 1.17 Model: ArcFace ResNet-100 - Device: CPU - Executor: Standard Intel ADL-S GT1 - Intel Core i3-12100 4 8 12 16 20 SE +/- 0.19, N = 5 17.31 1. (CXX) g++ options: -O3 -march=native -ffunction-sections -fdata-sections -mtune=native -flto -fno-fat-lto-objects -ldl -lrt -lpthread -pthread
ONNX Runtime Model: ArcFace ResNet-100 - Device: CPU - Executor: Parallel OpenBenchmarking.org Inferences Per Second, More Is Better ONNX Runtime 1.17 Model: ArcFace ResNet-100 - Device: CPU - Executor: Parallel Intel ADL-S GT1 - Intel Core i3-12100 3 6 9 12 15 SE +/- 0.12, N = 6 11.54 1. (CXX) g++ options: -O3 -march=native -ffunction-sections -fdata-sections -mtune=native -flto -fno-fat-lto-objects -ldl -lrt -lpthread -pthread
ONNX Runtime Model: CaffeNet 12-int8 - Device: CPU - Executor: Standard OpenBenchmarking.org Inferences Per Second, More Is Better ONNX Runtime 1.17 Model: CaffeNet 12-int8 - Device: CPU - Executor: Standard Intel ADL-S GT1 - Intel Core i3-12100 50 100 150 200 250 SE +/- 2.63, N = 3 233.25 1. (CXX) g++ options: -O3 -march=native -ffunction-sections -fdata-sections -mtune=native -flto -fno-fat-lto-objects -ldl -lrt -lpthread -pthread
ONNX Runtime Model: CaffeNet 12-int8 - Device: CPU - Executor: Parallel OpenBenchmarking.org Inferences Per Second, More Is Better ONNX Runtime 1.17 Model: CaffeNet 12-int8 - Device: CPU - Executor: Parallel Intel ADL-S GT1 - Intel Core i3-12100 50 100 150 200 250 SE +/- 2.29, N = 3 209.45 1. (CXX) g++ options: -O3 -march=native -ffunction-sections -fdata-sections -mtune=native -flto -fno-fat-lto-objects -ldl -lrt -lpthread -pthread
ONNX Runtime Model: bertsquad-12 - Device: CPU - Executor: Parallel OpenBenchmarking.org Inferences Per Second, More Is Better ONNX Runtime 1.17 Model: bertsquad-12 - Device: CPU - Executor: Parallel Intel ADL-S GT1 - Intel Core i3-12100 1.2026 2.4052 3.6078 4.8104 6.013 SE +/- 0.07908, N = 15 5.34494 1. (CXX) g++ options: -O3 -march=native -ffunction-sections -fdata-sections -mtune=native -flto -fno-fat-lto-objects -ldl -lrt -lpthread -pthread
ONNX Runtime Model: T5 Encoder - Device: CPU - Executor: Standard OpenBenchmarking.org Inferences Per Second, More Is Better ONNX Runtime 1.17 Model: T5 Encoder - Device: CPU - Executor: Standard Intel ADL-S GT1 - Intel Core i3-12100 13 26 39 52 65 SE +/- 0.02, N = 3 57.59 1. (CXX) g++ options: -O3 -march=native -ffunction-sections -fdata-sections -mtune=native -flto -fno-fat-lto-objects -ldl -lrt -lpthread -pthread
ONNX Runtime Model: T5 Encoder - Device: CPU - Executor: Parallel OpenBenchmarking.org Inferences Per Second, More Is Better ONNX Runtime 1.17 Model: T5 Encoder - Device: CPU - Executor: Parallel Intel ADL-S GT1 - Intel Core i3-12100 13 26 39 52 65 SE +/- 0.04, N = 3 56.03 1. (CXX) g++ options: -O3 -march=native -ffunction-sections -fdata-sections -mtune=native -flto -fno-fat-lto-objects -ldl -lrt -lpthread -pthread
ONNX Runtime Model: GPT-2 - Device: CPU - Executor: Standard OpenBenchmarking.org Inferences Per Second, More Is Better ONNX Runtime 1.17 Model: GPT-2 - Device: CPU - Executor: Standard Intel ADL-S GT1 - Intel Core i3-12100 12 24 36 48 60 SE +/- 0.51, N = 3 51.12 1. (CXX) g++ options: -O3 -march=native -ffunction-sections -fdata-sections -mtune=native -flto -fno-fat-lto-objects -ldl -lrt -lpthread -pthread
ONNX Runtime Model: GPT-2 - Device: CPU - Executor: Parallel OpenBenchmarking.org Inferences Per Second, More Is Better ONNX Runtime 1.17 Model: GPT-2 - Device: CPU - Executor: Parallel Intel ADL-S GT1 - Intel Core i3-12100 11 22 33 44 55 SE +/- 0.08, N = 3 50.31 1. (CXX) g++ options: -O3 -march=native -ffunction-sections -fdata-sections -mtune=native -flto -fno-fat-lto-objects -ldl -lrt -lpthread -pthread
Numenta Anomaly Benchmark Detector: Contextual Anomaly Detector OSE OpenBenchmarking.org Seconds, Fewer Is Better Numenta Anomaly Benchmark 1.1 Detector: Contextual Anomaly Detector OSE Intel ADL-S GT1 - Intel Core i3-12100 12 24 36 48 60 SE +/- 0.27, N = 3 52.86
Numenta Anomaly Benchmark Detector: Bayesian Changepoint OpenBenchmarking.org Seconds, Fewer Is Better Numenta Anomaly Benchmark 1.1 Detector: Bayesian Changepoint Intel ADL-S GT1 - Intel Core i3-12100 11 22 33 44 55 SE +/- 0.32, N = 13 47.48
Numenta Anomaly Benchmark Detector: Earthgecko Skyline OpenBenchmarking.org Seconds, Fewer Is Better Numenta Anomaly Benchmark 1.1 Detector: Earthgecko Skyline Intel ADL-S GT1 - Intel Core i3-12100 50 100 150 200 250 SE +/- 1.51, N = 3 209.16
Numenta Anomaly Benchmark Detector: Windowed Gaussian OpenBenchmarking.org Seconds, Fewer Is Better Numenta Anomaly Benchmark 1.1 Detector: Windowed Gaussian Intel ADL-S GT1 - Intel Core i3-12100 3 6 9 12 15 SE +/- 0.02, N = 3 11.70
Numenta Anomaly Benchmark Detector: Relative Entropy OpenBenchmarking.org Seconds, Fewer Is Better Numenta Anomaly Benchmark 1.1 Detector: Relative Entropy Intel ADL-S GT1 - Intel Core i3-12100 5 10 15 20 25 SE +/- 0.18, N = 8 20.60
Numenta Anomaly Benchmark Detector: KNN CAD OpenBenchmarking.org Seconds, Fewer Is Better Numenta Anomaly Benchmark 1.1 Detector: KNN CAD Intel ADL-S GT1 - Intel Core i3-12100 50 100 150 200 250 SE +/- 1.93, N = 3 245.09
OpenVINO Model: Age Gender Recognition Retail 0013 FP16-INT8 - Device: CPU OpenBenchmarking.org ms, Fewer Is Better OpenVINO 2024.0 Model: Age Gender Recognition Retail 0013 FP16-INT8 - Device: CPU Intel ADL-S GT1 - Intel Core i3-12100 0.0833 0.1666 0.2499 0.3332 0.4165 SE +/- 0.00, N = 3 0.37 MIN: 0.24 / MAX: 45.25 1. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl
OpenVINO Model: Age Gender Recognition Retail 0013 FP16-INT8 - Device: CPU OpenBenchmarking.org FPS, More Is Better OpenVINO 2024.0 Model: Age Gender Recognition Retail 0013 FP16-INT8 - Device: CPU Intel ADL-S GT1 - Intel Core i3-12100 2K 4K 6K 8K 10K SE +/- 11.73, N = 3 10592.67 1. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl
OpenVINO Model: Handwritten English Recognition FP16-INT8 - Device: CPU OpenBenchmarking.org ms, Fewer Is Better OpenVINO 2024.0 Model: Handwritten English Recognition FP16-INT8 - Device: CPU Intel ADL-S GT1 - Intel Core i3-12100 10 20 30 40 50 SE +/- 0.20, N = 3 45.22 MIN: 31.25 / MAX: 119.06 1. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl
OpenVINO Model: Handwritten English Recognition FP16-INT8 - Device: CPU OpenBenchmarking.org FPS, More Is Better OpenVINO 2024.0 Model: Handwritten English Recognition FP16-INT8 - Device: CPU Intel ADL-S GT1 - Intel Core i3-12100 20 40 60 80 100 SE +/- 0.38, N = 3 88.41 1. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl
OpenVINO Model: Age Gender Recognition Retail 0013 FP16 - Device: CPU OpenBenchmarking.org ms, Fewer Is Better OpenVINO 2024.0 Model: Age Gender Recognition Retail 0013 FP16 - Device: CPU Intel ADL-S GT1 - Intel Core i3-12100 0.2565 0.513 0.7695 1.026 1.2825 SE +/- 0.01, N = 3 1.14 MIN: 0.58 / MAX: 29.63 1. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl
OpenVINO Model: Age Gender Recognition Retail 0013 FP16 - Device: CPU OpenBenchmarking.org FPS, More Is Better OpenVINO 2024.0 Model: Age Gender Recognition Retail 0013 FP16 - Device: CPU Intel ADL-S GT1 - Intel Core i3-12100 700 1400 2100 2800 3500 SE +/- 28.02, N = 3 3481.92 1. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl
OpenVINO Model: Person Re-Identification Retail FP16 - Device: CPU OpenBenchmarking.org ms, Fewer Is Better OpenVINO 2024.0 Model: Person Re-Identification Retail FP16 - Device: CPU Intel ADL-S GT1 - Intel Core i3-12100 5 10 15 20 25 SE +/- 0.16, N = 3 19.80 MIN: 9.77 / MAX: 70.87 1. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl
OpenVINO Model: Person Re-Identification Retail FP16 - Device: CPU OpenBenchmarking.org FPS, More Is Better OpenVINO 2024.0 Model: Person Re-Identification Retail FP16 - Device: CPU Intel ADL-S GT1 - Intel Core i3-12100 40 80 120 160 200 SE +/- 1.60, N = 3 201.90 1. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl
OpenVINO Model: Handwritten English Recognition FP16 - Device: CPU OpenBenchmarking.org ms, Fewer Is Better OpenVINO 2024.0 Model: Handwritten English Recognition FP16 - Device: CPU Intel ADL-S GT1 - Intel Core i3-12100 13 26 39 52 65 SE +/- 0.22, N = 3 60.21 MIN: 40.96 / MAX: 136.76 1. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl
OpenVINO Model: Handwritten English Recognition FP16 - Device: CPU OpenBenchmarking.org FPS, More Is Better OpenVINO 2024.0 Model: Handwritten English Recognition FP16 - Device: CPU Intel ADL-S GT1 - Intel Core i3-12100 15 30 45 60 75 SE +/- 0.24, N = 3 66.40 1. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl
OpenVINO Model: Noise Suppression Poconet-Like FP16 - Device: CPU OpenBenchmarking.org ms, Fewer Is Better OpenVINO 2024.0 Model: Noise Suppression Poconet-Like FP16 - Device: CPU Intel ADL-S GT1 - Intel Core i3-12100 5 10 15 20 25 SE +/- 0.06, N = 3 19.10 MIN: 10.05 / MAX: 69.6 1. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl
OpenVINO Model: Noise Suppression Poconet-Like FP16 - Device: CPU OpenBenchmarking.org FPS, More Is Better OpenVINO 2024.0 Model: Noise Suppression Poconet-Like FP16 - Device: CPU Intel ADL-S GT1 - Intel Core i3-12100 50 100 150 200 250 SE +/- 0.74, N = 3 208.84 1. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl
OpenVINO Model: Person Vehicle Bike Detection FP16 - Device: CPU OpenBenchmarking.org ms, Fewer Is Better OpenVINO 2024.0 Model: Person Vehicle Bike Detection FP16 - Device: CPU Intel ADL-S GT1 - Intel Core i3-12100 6 12 18 24 30 SE +/- 0.32, N = 3 23.99 MIN: 13.37 / MAX: 116.89 1. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl
OpenVINO Model: Person Vehicle Bike Detection FP16 - Device: CPU OpenBenchmarking.org FPS, More Is Better OpenVINO 2024.0 Model: Person Vehicle Bike Detection FP16 - Device: CPU Intel ADL-S GT1 - Intel Core i3-12100 40 80 120 160 200 SE +/- 2.19, N = 3 166.71 1. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl
OpenVINO Model: Weld Porosity Detection FP16-INT8 - Device: CPU OpenBenchmarking.org ms, Fewer Is Better OpenVINO 2024.0 Model: Weld Porosity Detection FP16-INT8 - Device: CPU Intel ADL-S GT1 - Intel Core i3-12100 2 4 6 8 10 SE +/- 0.01, N = 3 7.79 MIN: 4.62 / MAX: 55.08 1. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl
OpenVINO Model: Weld Porosity Detection FP16-INT8 - Device: CPU OpenBenchmarking.org FPS, More Is Better OpenVINO 2024.0 Model: Weld Porosity Detection FP16-INT8 - Device: CPU Intel ADL-S GT1 - Intel Core i3-12100 110 220 330 440 550 SE +/- 0.62, N = 3 513.04 1. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl
OpenVINO Model: Machine Translation EN To DE FP16 - Device: CPU OpenBenchmarking.org ms, Fewer Is Better OpenVINO 2024.0 Model: Machine Translation EN To DE FP16 - Device: CPU Intel ADL-S GT1 - Intel Core i3-12100 60 120 180 240 300 SE +/- 0.35, N = 3 267.44 MIN: 221.5 / MAX: 472.42 1. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl
OpenVINO Model: Machine Translation EN To DE FP16 - Device: CPU OpenBenchmarking.org FPS, More Is Better OpenVINO 2024.0 Model: Machine Translation EN To DE FP16 - Device: CPU Intel ADL-S GT1 - Intel Core i3-12100 4 8 12 16 20 SE +/- 0.02, N = 3 14.95 1. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl
OpenVINO Model: Road Segmentation ADAS FP16-INT8 - Device: CPU OpenBenchmarking.org ms, Fewer Is Better OpenVINO 2024.0 Model: Road Segmentation ADAS FP16-INT8 - Device: CPU Intel ADL-S GT1 - Intel Core i3-12100 14 28 42 56 70 SE +/- 0.22, N = 3 63.27 MIN: 31.99 / MAX: 132.5 1. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl
OpenVINO Model: Road Segmentation ADAS FP16-INT8 - Device: CPU OpenBenchmarking.org FPS, More Is Better OpenVINO 2024.0 Model: Road Segmentation ADAS FP16-INT8 - Device: CPU Intel ADL-S GT1 - Intel Core i3-12100 14 28 42 56 70 SE +/- 0.23, N = 3 63.19 1. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl
OpenVINO Model: Face Detection Retail FP16-INT8 - Device: CPU OpenBenchmarking.org ms, Fewer Is Better OpenVINO 2024.0 Model: Face Detection Retail FP16-INT8 - Device: CPU Intel ADL-S GT1 - Intel Core i3-12100 1.08 2.16 3.24 4.32 5.4 SE +/- 0.02, N = 3 4.80 MIN: 3.51 / MAX: 52.64 1. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl
OpenVINO Model: Face Detection Retail FP16-INT8 - Device: CPU OpenBenchmarking.org FPS, More Is Better OpenVINO 2024.0 Model: Face Detection Retail FP16-INT8 - Device: CPU Intel ADL-S GT1 - Intel Core i3-12100 200 400 600 800 1000 SE +/- 3.58, N = 3 833.34 1. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl
OpenVINO Model: Weld Porosity Detection FP16 - Device: CPU OpenBenchmarking.org ms, Fewer Is Better OpenVINO 2024.0 Model: Weld Porosity Detection FP16 - Device: CPU Intel ADL-S GT1 - Intel Core i3-12100 7 14 21 28 35 SE +/- 0.06, N = 3 29.24 MIN: 19.89 / MAX: 66.26 1. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl
OpenVINO Model: Weld Porosity Detection FP16 - Device: CPU OpenBenchmarking.org FPS, More Is Better OpenVINO 2024.0 Model: Weld Porosity Detection FP16 - Device: CPU Intel ADL-S GT1 - Intel Core i3-12100 30 60 90 120 150 SE +/- 0.25, N = 3 136.74 1. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl
OpenVINO Model: Vehicle Detection FP16-INT8 - Device: CPU OpenBenchmarking.org ms, Fewer Is Better OpenVINO 2024.0 Model: Vehicle Detection FP16-INT8 - Device: CPU Intel ADL-S GT1 - Intel Core i3-12100 5 10 15 20 25 SE +/- 0.07, N = 3 18.66 MIN: 9.8 / MAX: 90.77 1. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl
OpenVINO Model: Vehicle Detection FP16-INT8 - Device: CPU OpenBenchmarking.org FPS, More Is Better OpenVINO 2024.0 Model: Vehicle Detection FP16-INT8 - Device: CPU Intel ADL-S GT1 - Intel Core i3-12100 50 100 150 200 250 SE +/- 0.79, N = 3 214.21 1. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl
OpenVINO Model: Road Segmentation ADAS FP16 - Device: CPU OpenBenchmarking.org ms, Fewer Is Better OpenVINO 2024.0 Model: Road Segmentation ADAS FP16 - Device: CPU Intel ADL-S GT1 - Intel Core i3-12100 50 100 150 200 250 SE +/- 0.53, N = 3 219.10 MIN: 120.84 / MAX: 389.85 1. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl
OpenVINO Model: Road Segmentation ADAS FP16 - Device: CPU OpenBenchmarking.org FPS, More Is Better OpenVINO 2024.0 Model: Road Segmentation ADAS FP16 - Device: CPU Intel ADL-S GT1 - Intel Core i3-12100 4 8 12 16 20 SE +/- 0.04, N = 3 18.24 1. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl
OpenVINO Model: Face Detection Retail FP16 - Device: CPU OpenBenchmarking.org ms, Fewer Is Better OpenVINO 2024.0 Model: Face Detection Retail FP16 - Device: CPU Intel ADL-S GT1 - Intel Core i3-12100 3 6 9 12 15 SE +/- 0.09, N = 3 12.09 MIN: 5.92 / MAX: 79.84 1. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl
OpenVINO Model: Face Detection Retail FP16 - Device: CPU OpenBenchmarking.org FPS, More Is Better OpenVINO 2024.0 Model: Face Detection Retail FP16 - Device: CPU Intel ADL-S GT1 - Intel Core i3-12100 70 140 210 280 350 SE +/- 2.36, N = 3 330.42 1. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl
OpenVINO Model: Face Detection FP16-INT8 - Device: CPU OpenBenchmarking.org ms, Fewer Is Better OpenVINO 2024.0 Model: Face Detection FP16-INT8 - Device: CPU Intel ADL-S GT1 - Intel Core i3-12100 160 320 480 640 800 SE +/- 1.39, N = 3 759.24 MIN: 716.66 / MAX: 920.51 1. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl
OpenVINO Model: Face Detection FP16-INT8 - Device: CPU OpenBenchmarking.org FPS, More Is Better OpenVINO 2024.0 Model: Face Detection FP16-INT8 - Device: CPU Intel ADL-S GT1 - Intel Core i3-12100 1.1835 2.367 3.5505 4.734 5.9175 SE +/- 0.01, N = 3 5.26 1. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl
OpenVINO Model: Vehicle Detection FP16 - Device: CPU OpenBenchmarking.org ms, Fewer Is Better OpenVINO 2024.0 Model: Vehicle Detection FP16 - Device: CPU Intel ADL-S GT1 - Intel Core i3-12100 14 28 42 56 70 SE +/- 0.06, N = 3 63.85 MIN: 30.33 / MAX: 201.51 1. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl
OpenVINO Model: Vehicle Detection FP16 - Device: CPU OpenBenchmarking.org FPS, More Is Better OpenVINO 2024.0 Model: Vehicle Detection FP16 - Device: CPU Intel ADL-S GT1 - Intel Core i3-12100 14 28 42 56 70 SE +/- 0.06, N = 3 62.61 1. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl
OpenVINO Model: Person Detection FP32 - Device: CPU OpenBenchmarking.org ms, Fewer Is Better OpenVINO 2024.0 Model: Person Detection FP32 - Device: CPU Intel ADL-S GT1 - Intel Core i3-12100 90 180 270 360 450 SE +/- 0.71, N = 3 398.50 MIN: 293.03 / MAX: 564.35 1. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl
OpenVINO Model: Person Detection FP32 - Device: CPU OpenBenchmarking.org FPS, More Is Better OpenVINO 2024.0 Model: Person Detection FP32 - Device: CPU Intel ADL-S GT1 - Intel Core i3-12100 3 6 9 12 15 SE +/- 0.02, N = 3 10.04 1. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl
OpenVINO Model: Person Detection FP16 - Device: CPU OpenBenchmarking.org ms, Fewer Is Better OpenVINO 2024.0 Model: Person Detection FP16 - Device: CPU Intel ADL-S GT1 - Intel Core i3-12100 90 180 270 360 450 SE +/- 1.13, N = 3 397.47 MIN: 223.86 / MAX: 566.44 1. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl
OpenVINO Model: Person Detection FP16 - Device: CPU OpenBenchmarking.org FPS, More Is Better OpenVINO 2024.0 Model: Person Detection FP16 - Device: CPU Intel ADL-S GT1 - Intel Core i3-12100 3 6 9 12 15 SE +/- 0.03, N = 3 10.06 1. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl
OpenVINO Model: Face Detection FP16 - Device: CPU OpenBenchmarking.org ms, Fewer Is Better OpenVINO 2024.0 Model: Face Detection FP16 - Device: CPU Intel ADL-S GT1 - Intel Core i3-12100 700 1400 2100 2800 3500 SE +/- 1.80, N = 3 3213.28 MIN: 2964.37 / MAX: 3356.75 1. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl
OpenVINO Model: Face Detection FP16 - Device: CPU OpenBenchmarking.org FPS, More Is Better OpenVINO 2024.0 Model: Face Detection FP16 - Device: CPU Intel ADL-S GT1 - Intel Core i3-12100 0.279 0.558 0.837 1.116 1.395 SE +/- 0.00, N = 3 1.24 1. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl
PlaidML FP16: No - Mode: Inference - Network: ResNet 50 - Device: CPU OpenBenchmarking.org FPS, More Is Better PlaidML FP16: No - Mode: Inference - Network: ResNet 50 - Device: CPU Intel ADL-S GT1 - Intel Core i3-12100 1.0395 2.079 3.1185 4.158 5.1975 SE +/- 0.01, N = 3 4.62
PlaidML FP16: No - Mode: Inference - Network: VGG16 - Device: CPU OpenBenchmarking.org FPS, More Is Better PlaidML FP16: No - Mode: Inference - Network: VGG16 - Device: CPU Intel ADL-S GT1 - Intel Core i3-12100 2 4 6 8 10 SE +/- 0.09, N = 3 8.86
XNNPACK Model: QU8MobileNetV3Small OpenBenchmarking.org us, Fewer Is Better XNNPACK 2cd86b Model: QU8MobileNetV3Small Intel ADL-S GT1 - Intel Core i3-12100 160 320 480 640 800 SE +/- 0.67, N = 3 728 1. (CXX) g++ options: -O3 -lrt -pthread -lpthread -lm
XNNPACK Model: QU8MobileNetV3Large OpenBenchmarking.org us, Fewer Is Better XNNPACK 2cd86b Model: QU8MobileNetV3Large Intel ADL-S GT1 - Intel Core i3-12100 400 800 1200 1600 2000 SE +/- 20.30, N = 3 1974 1. (CXX) g++ options: -O3 -lrt -pthread -lpthread -lm
XNNPACK Model: QU8MobileNetV2 OpenBenchmarking.org us, Fewer Is Better XNNPACK 2cd86b Model: QU8MobileNetV2 Intel ADL-S GT1 - Intel Core i3-12100 500 1000 1500 2000 2500 SE +/- 21.07, N = 3 2163 1. (CXX) g++ options: -O3 -lrt -pthread -lpthread -lm
XNNPACK Model: FP16MobileNetV3Small OpenBenchmarking.org us, Fewer Is Better XNNPACK 2cd86b Model: FP16MobileNetV3Small Intel ADL-S GT1 - Intel Core i3-12100 200 400 600 800 1000 SE +/- 5.84, N = 3 965 1. (CXX) g++ options: -O3 -lrt -pthread -lpthread -lm
XNNPACK Model: FP16MobileNetV3Large OpenBenchmarking.org us, Fewer Is Better XNNPACK 2cd86b Model: FP16MobileNetV3Large Intel ADL-S GT1 - Intel Core i3-12100 700 1400 2100 2800 3500 SE +/- 40.00, N = 3 3396 1. (CXX) g++ options: -O3 -lrt -pthread -lpthread -lm
XNNPACK Model: FP16MobileNetV2 OpenBenchmarking.org us, Fewer Is Better XNNPACK 2cd86b Model: FP16MobileNetV2 Intel ADL-S GT1 - Intel Core i3-12100 700 1400 2100 2800 3500 3444 1. (CXX) g++ options: -O3 -lrt -pthread -lpthread -lm
XNNPACK Model: FP32MobileNetV3Small OpenBenchmarking.org us, Fewer Is Better XNNPACK 2cd86b Model: FP32MobileNetV3Small Intel ADL-S GT1 - Intel Core i3-12100 300 600 900 1200 1500 SE +/- 14.19, N = 3 1302 1. (CXX) g++ options: -O3 -lrt -pthread -lpthread -lm
XNNPACK Model: FP32MobileNetV3Large OpenBenchmarking.org us, Fewer Is Better XNNPACK 2cd86b Model: FP32MobileNetV3Large Intel ADL-S GT1 - Intel Core i3-12100 1200 2400 3600 4800 6000 SE +/- 28.00, N = 3 5484 1. (CXX) g++ options: -O3 -lrt -pthread -lpthread -lm
XNNPACK Model: FP32MobileNetV2 OpenBenchmarking.org us, Fewer Is Better XNNPACK 2cd86b Model: FP32MobileNetV2 Intel ADL-S GT1 - Intel Core i3-12100 1200 2400 3600 4800 6000 SE +/- 15.31, N = 3 5545 1. (CXX) g++ options: -O3 -lrt -pthread -lpthread -lm
TNN Target: CPU - Model: SqueezeNet v1.1 OpenBenchmarking.org ms, Fewer Is Better TNN 0.3 Target: CPU - Model: SqueezeNet v1.1 Intel ADL-S GT1 - Intel Core i3-12100 40 80 120 160 200 SE +/- 0.09, N = 3 166.13 MIN: 162.86 / MAX: 170.17 1. (CXX) g++ options: -fopenmp -pthread -fvisibility=hidden -fvisibility=default -O3 -rdynamic -ldl
TNN Target: CPU - Model: SqueezeNet v2 OpenBenchmarking.org ms, Fewer Is Better TNN 0.3 Target: CPU - Model: SqueezeNet v2 Intel ADL-S GT1 - Intel Core i3-12100 10 20 30 40 50 SE +/- 0.07, N = 3 46.21 MIN: 45.29 / MAX: 48.72 1. (CXX) g++ options: -fopenmp -pthread -fvisibility=hidden -fvisibility=default -O3 -rdynamic -ldl
TNN Target: CPU - Model: MobileNet v2 OpenBenchmarking.org ms, Fewer Is Better TNN 0.3 Target: CPU - Model: MobileNet v2 Intel ADL-S GT1 - Intel Core i3-12100 40 80 120 160 200 SE +/- 0.11, N = 3 201.11 MIN: 198.48 / MAX: 216.2 1. (CXX) g++ options: -fopenmp -pthread -fvisibility=hidden -fvisibility=default -O3 -rdynamic -ldl
TNN Target: CPU - Model: DenseNet OpenBenchmarking.org ms, Fewer Is Better TNN 0.3 Target: CPU - Model: DenseNet Intel ADL-S GT1 - Intel Core i3-12100 500 1000 1500 2000 2500 SE +/- 1.75, N = 3 2446.58 MIN: 2392.91 / MAX: 2547.47 1. (CXX) g++ options: -fopenmp -pthread -fvisibility=hidden -fvisibility=default -O3 -rdynamic -ldl
NCNN Target: Vulkan GPU - Model: FastestDet OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: Vulkan GPU - Model: FastestDet Intel ADL-S GT1 - Intel Core i3-12100 0.7425 1.485 2.2275 2.97 3.7125 SE +/- 0.02, N = 3 3.30 MIN: 3.23 / MAX: 3.39 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread -pthread
NCNN Target: Vulkan GPU - Model: vision_transformer OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: Vulkan GPU - Model: vision_transformer Intel ADL-S GT1 - Intel Core i3-12100 30 60 90 120 150 SE +/- 0.09, N = 3 136.23 MIN: 134.17 / MAX: 308.71 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread -pthread
NCNN Target: Vulkan GPU - Model: regnety_400m OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: Vulkan GPU - Model: regnety_400m Intel ADL-S GT1 - Intel Core i3-12100 2 4 6 8 10 SE +/- 0.00, N = 3 6.53 MIN: 6.48 / MAX: 6.72 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread -pthread
NCNN Target: Vulkan GPU - Model: squeezenet_ssd OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: Vulkan GPU - Model: squeezenet_ssd Intel ADL-S GT1 - Intel Core i3-12100 4 8 12 16 20 SE +/- 0.03, N = 3 17.49 MIN: 17.21 / MAX: 58.42 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread -pthread
NCNN Target: Vulkan GPU - Model: yolov4-tiny OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: Vulkan GPU - Model: yolov4-tiny Intel ADL-S GT1 - Intel Core i3-12100 9 18 27 36 45 SE +/- 0.14, N = 3 39.80 MIN: 39.37 / MAX: 78.33 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread -pthread
NCNN Target: Vulkan GPU - Model: resnet50 OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: Vulkan GPU - Model: resnet50 Intel ADL-S GT1 - Intel Core i3-12100 6 12 18 24 30 SE +/- 0.29, N = 3 26.97 MIN: 26.39 / MAX: 123.4 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread -pthread
NCNN Target: Vulkan GPU - Model: alexnet OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: Vulkan GPU - Model: alexnet Intel ADL-S GT1 - Intel Core i3-12100 3 6 9 12 15 SE +/- 0.02, N = 3 12.54 MIN: 12.43 / MAX: 14.78 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread -pthread
NCNN Target: Vulkan GPU - Model: resnet18 OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: Vulkan GPU - Model: resnet18 Intel ADL-S GT1 - Intel Core i3-12100 3 6 9 12 15 SE +/- 0.02, N = 3 12.25 MIN: 12.15 / MAX: 12.69 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread -pthread
NCNN Target: Vulkan GPU - Model: vgg16 OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: Vulkan GPU - Model: vgg16 Intel ADL-S GT1 - Intel Core i3-12100 20 40 60 80 100 SE +/- 0.29, N = 3 111.12 MIN: 109.05 / MAX: 253.29 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread -pthread
NCNN Target: Vulkan GPU - Model: googlenet OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: Vulkan GPU - Model: googlenet Intel ADL-S GT1 - Intel Core i3-12100 4 8 12 16 20 SE +/- 0.04, N = 3 15.83 MIN: 15.64 / MAX: 54.42 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread -pthread
NCNN Target: Vulkan GPU - Model: blazeface OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: Vulkan GPU - Model: blazeface Intel ADL-S GT1 - Intel Core i3-12100 0.1373 0.2746 0.4119 0.5492 0.6865 SE +/- 0.00, N = 3 0.61 MIN: 0.59 / MAX: 0.65 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread -pthread
NCNN Target: Vulkan GPU - Model: efficientnet-b0 OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: Vulkan GPU - Model: efficientnet-b0 Intel ADL-S GT1 - Intel Core i3-12100 2 4 6 8 10 SE +/- 0.02, N = 3 7.99 MIN: 7.85 / MAX: 10.37 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread -pthread
NCNN Target: Vulkan GPU - Model: mnasnet OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: Vulkan GPU - Model: mnasnet Intel ADL-S GT1 - Intel Core i3-12100 0.7178 1.4356 2.1534 2.8712 3.589 SE +/- 0.02, N = 2 3.19 MIN: 3.12 / MAX: 4.45 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread -pthread
NCNN Target: Vulkan GPU - Model: shufflenet-v2 OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: Vulkan GPU - Model: shufflenet-v2 Intel ADL-S GT1 - Intel Core i3-12100 0.4973 0.9946 1.4919 1.9892 2.4865 SE +/- 0.01, N = 3 2.21 MIN: 2.16 / MAX: 3.84 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread -pthread
NCNN Target: Vulkan GPU-v3-v3 - Model: mobilenet-v3 OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: Vulkan GPU-v3-v3 - Model: mobilenet-v3 Intel ADL-S GT1 - Intel Core i3-12100 0.7178 1.4356 2.1534 2.8712 3.589 SE +/- 0.00, N = 3 3.19 MIN: 3.11 / MAX: 4.76 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread -pthread
NCNN Target: Vulkan GPU-v2-v2 - Model: mobilenet-v2 OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: Vulkan GPU-v2-v2 - Model: mobilenet-v2 Intel ADL-S GT1 - Intel Core i3-12100 1.1903 2.3806 3.5709 4.7612 5.9515 SE +/- 0.02, N = 3 5.29 MIN: 5.08 / MAX: 7.09 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread -pthread
NCNN Target: Vulkan GPU - Model: mobilenet OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: Vulkan GPU - Model: mobilenet Intel ADL-S GT1 - Intel Core i3-12100 5 10 15 20 25 SE +/- 0.03, N = 3 21.05 MIN: 20.81 / MAX: 24.47 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread -pthread
NCNN Target: CPU - Model: FastestDet OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: CPU - Model: FastestDet Intel ADL-S GT1 - Intel Core i3-12100 0.7448 1.4896 2.2344 2.9792 3.724 SE +/- 0.05, N = 4 3.31 MIN: 3.17 / MAX: 7.41 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread -pthread
NCNN Target: CPU - Model: vision_transformer OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: CPU - Model: vision_transformer Intel ADL-S GT1 - Intel Core i3-12100 30 60 90 120 150 SE +/- 0.38, N = 4 135.49 MIN: 134.44 / MAX: 253.67 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread -pthread
NCNN Target: CPU - Model: regnety_400m OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: CPU - Model: regnety_400m Intel ADL-S GT1 - Intel Core i3-12100 2 4 6 8 10 SE +/- 0.03, N = 4 6.57 MIN: 6.47 / MAX: 15.95 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread -pthread
NCNN Target: CPU - Model: squeezenet_ssd OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: CPU - Model: squeezenet_ssd Intel ADL-S GT1 - Intel Core i3-12100 4 8 12 16 20 SE +/- 0.08, N = 4 17.42 MIN: 17.17 / MAX: 33.61 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread -pthread
NCNN Target: CPU - Model: yolov4-tiny OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: CPU - Model: yolov4-tiny Intel ADL-S GT1 - Intel Core i3-12100 9 18 27 36 45 SE +/- 0.06, N = 4 40.12 MIN: 39.72 / MAX: 83.43 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread -pthread
NCNN Target: CPU - Model: resnet50 OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: CPU - Model: resnet50 Intel ADL-S GT1 - Intel Core i3-12100 6 12 18 24 30 SE +/- 0.11, N = 4 26.71 MIN: 26.31 / MAX: 69.35 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread -pthread
NCNN Target: CPU - Model: alexnet OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: CPU - Model: alexnet Intel ADL-S GT1 - Intel Core i3-12100 3 6 9 12 15 SE +/- 0.02, N = 4 12.63 MIN: 12.45 / MAX: 50.58 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread -pthread
NCNN Target: CPU - Model: resnet18 OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: CPU - Model: resnet18 Intel ADL-S GT1 - Intel Core i3-12100 3 6 9 12 15 SE +/- 0.02, N = 4 12.34 MIN: 12.22 / MAX: 14.13 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread -pthread
NCNN Target: CPU - Model: vgg16 OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: CPU - Model: vgg16 Intel ADL-S GT1 - Intel Core i3-12100 20 40 60 80 100 SE +/- 0.09, N = 4 109.92 MIN: 108.76 / MAX: 150.98 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread -pthread
NCNN Target: CPU - Model: googlenet OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: CPU - Model: googlenet Intel ADL-S GT1 - Intel Core i3-12100 4 8 12 16 20 SE +/- 0.28, N = 4 16.13 MIN: 15.65 / MAX: 75.15 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread -pthread
NCNN Target: CPU - Model: blazeface OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: CPU - Model: blazeface Intel ADL-S GT1 - Intel Core i3-12100 0.1395 0.279 0.4185 0.558 0.6975 SE +/- 0.00, N = 4 0.62 MIN: 0.59 / MAX: 0.68 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread -pthread
NCNN Target: CPU - Model: efficientnet-b0 OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: CPU - Model: efficientnet-b0 Intel ADL-S GT1 - Intel Core i3-12100 2 4 6 8 10 SE +/- 0.06, N = 4 8.15 MIN: 7.89 / MAX: 41.7 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread -pthread
NCNN Target: CPU - Model: shufflenet-v2 OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: CPU - Model: shufflenet-v2 Intel ADL-S GT1 - Intel Core i3-12100 0.5063 1.0126 1.5189 2.0252 2.5315 SE +/- 0.03, N = 4 2.25 MIN: 2.16 / MAX: 18.02 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread -pthread
NCNN Target: CPU-v3-v3 - Model: mobilenet-v3 OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: CPU-v3-v3 - Model: mobilenet-v3 Intel ADL-S GT1 - Intel Core i3-12100 0.7155 1.431 2.1465 2.862 3.5775 SE +/- 0.02, N = 4 3.18 MIN: 3.1 / MAX: 5.1 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread -pthread
NCNN Target: CPU-v2-v2 - Model: mobilenet-v2 OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: CPU-v2-v2 - Model: mobilenet-v2 Intel ADL-S GT1 - Intel Core i3-12100 1.2038 2.4076 3.6114 4.8152 6.019 SE +/- 0.04, N = 4 5.35 MIN: 5.1 / MAX: 16.42 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread -pthread
NCNN Target: CPU - Model: mobilenet OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: CPU - Model: mobilenet Intel ADL-S GT1 - Intel Core i3-12100 5 10 15 20 25 SE +/- 0.26, N = 4 21.50 MIN: 20.87 / MAX: 110.78 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread -pthread
Mobile Neural Network Model: inception-v3 OpenBenchmarking.org ms, Fewer Is Better Mobile Neural Network 2.9.b11b7037d Model: inception-v3 Intel ADL-S GT1 - Intel Core i3-12100 8 16 24 32 40 SE +/- 0.07, N = 3 33.51 MIN: 32.99 / MAX: 109 1. (CXX) g++ options: -std=c++11 -O3 -fvisibility=hidden -fomit-frame-pointer -fstrict-aliasing -ffunction-sections -fdata-sections -ffast-math -fno-rtti -fno-exceptions -pthread -ldl
Mobile Neural Network Model: mobilenet-v1-1.0 OpenBenchmarking.org ms, Fewer Is Better Mobile Neural Network 2.9.b11b7037d Model: mobilenet-v1-1.0 Intel ADL-S GT1 - Intel Core i3-12100 0.7589 1.5178 2.2767 3.0356 3.7945 SE +/- 0.048, N = 3 3.373 MIN: 3.24 / MAX: 29.19 1. (CXX) g++ options: -std=c++11 -O3 -fvisibility=hidden -fomit-frame-pointer -fstrict-aliasing -ffunction-sections -fdata-sections -ffast-math -fno-rtti -fno-exceptions -pthread -ldl
Mobile Neural Network Model: MobileNetV2_224 OpenBenchmarking.org ms, Fewer Is Better Mobile Neural Network 2.9.b11b7037d Model: MobileNetV2_224 Intel ADL-S GT1 - Intel Core i3-12100 0.6086 1.2172 1.8258 2.4344 3.043 SE +/- 0.007, N = 3 2.705 MIN: 2.65 / MAX: 4.63 1. (CXX) g++ options: -std=c++11 -O3 -fvisibility=hidden -fomit-frame-pointer -fstrict-aliasing -ffunction-sections -fdata-sections -ffast-math -fno-rtti -fno-exceptions -pthread -ldl
Mobile Neural Network Model: SqueezeNetV1.0 OpenBenchmarking.org ms, Fewer Is Better Mobile Neural Network 2.9.b11b7037d Model: SqueezeNetV1.0 Intel ADL-S GT1 - Intel Core i3-12100 1.0674 2.1348 3.2022 4.2696 5.337 SE +/- 0.009, N = 3 4.744 MIN: 4.68 / MAX: 20.9 1. (CXX) g++ options: -std=c++11 -O3 -fvisibility=hidden -fomit-frame-pointer -fstrict-aliasing -ffunction-sections -fdata-sections -ffast-math -fno-rtti -fno-exceptions -pthread -ldl
Mobile Neural Network Model: resnet-v2-50 OpenBenchmarking.org ms, Fewer Is Better Mobile Neural Network 2.9.b11b7037d Model: resnet-v2-50 Intel ADL-S GT1 - Intel Core i3-12100 6 12 18 24 30 SE +/- 0.04, N = 3 26.31 MIN: 25.83 / MAX: 99.47 1. (CXX) g++ options: -std=c++11 -O3 -fvisibility=hidden -fomit-frame-pointer -fstrict-aliasing -ffunction-sections -fdata-sections -ffast-math -fno-rtti -fno-exceptions -pthread -ldl
Mobile Neural Network Model: squeezenetv1.1 OpenBenchmarking.org ms, Fewer Is Better Mobile Neural Network 2.9.b11b7037d Model: squeezenetv1.1 Intel ADL-S GT1 - Intel Core i3-12100 0.6525 1.305 1.9575 2.61 3.2625 SE +/- 0.015, N = 3 2.900 MIN: 2.83 / MAX: 42.33 1. (CXX) g++ options: -std=c++11 -O3 -fvisibility=hidden -fomit-frame-pointer -fstrict-aliasing -ffunction-sections -fdata-sections -ffast-math -fno-rtti -fno-exceptions -pthread -ldl
Mobile Neural Network Model: nasnet OpenBenchmarking.org ms, Fewer Is Better Mobile Neural Network 2.9.b11b7037d Model: nasnet Intel ADL-S GT1 - Intel Core i3-12100 3 6 9 12 15 SE +/- 0.022, N = 3 9.072 MIN: 8.93 / MAX: 29.56 1. (CXX) g++ options: -std=c++11 -O3 -fvisibility=hidden -fomit-frame-pointer -fstrict-aliasing -ffunction-sections -fdata-sections -ffast-math -fno-rtti -fno-exceptions -pthread -ldl
Caffe Model: GoogleNet - Acceleration: CPU - Iterations: 1000 OpenBenchmarking.org Milli-Seconds, Fewer Is Better Caffe 2020-02-13 Model: GoogleNet - Acceleration: CPU - Iterations: 1000 Intel ADL-S GT1 - Intel Core i3-12100 200K 400K 600K 800K 1000K SE +/- 3248.77, N = 3 1038320 1. (CXX) g++ options: -fPIC -O3 -rdynamic -lglog -lgflags -lprotobuf -lpthread -lsz -lz -ldl -lm -llmdb -lopenblas
Caffe Model: GoogleNet - Acceleration: CPU - Iterations: 200 OpenBenchmarking.org Milli-Seconds, Fewer Is Better Caffe 2020-02-13 Model: GoogleNet - Acceleration: CPU - Iterations: 200 Intel ADL-S GT1 - Intel Core i3-12100 40K 80K 120K 160K 200K SE +/- 44.14, N = 3 206462 1. (CXX) g++ options: -fPIC -O3 -rdynamic -lglog -lgflags -lprotobuf -lpthread -lsz -lz -ldl -lm -llmdb -lopenblas
Caffe Model: GoogleNet - Acceleration: CPU - Iterations: 100 OpenBenchmarking.org Milli-Seconds, Fewer Is Better Caffe 2020-02-13 Model: GoogleNet - Acceleration: CPU - Iterations: 100 Intel ADL-S GT1 - Intel Core i3-12100 20K 40K 60K 80K 100K SE +/- 148.50, N = 3 103411 1. (CXX) g++ options: -fPIC -O3 -rdynamic -lglog -lgflags -lprotobuf -lpthread -lsz -lz -ldl -lm -llmdb -lopenblas
Caffe Model: AlexNet - Acceleration: CPU - Iterations: 1000 OpenBenchmarking.org Milli-Seconds, Fewer Is Better Caffe 2020-02-13 Model: AlexNet - Acceleration: CPU - Iterations: 1000 Intel ADL-S GT1 - Intel Core i3-12100 100K 200K 300K 400K 500K SE +/- 4664.71, N = 3 445933 1. (CXX) g++ options: -fPIC -O3 -rdynamic -lglog -lgflags -lprotobuf -lpthread -lsz -lz -ldl -lm -llmdb -lopenblas
Caffe Model: AlexNet - Acceleration: CPU - Iterations: 200 OpenBenchmarking.org Milli-Seconds, Fewer Is Better Caffe 2020-02-13 Model: AlexNet - Acceleration: CPU - Iterations: 200 Intel ADL-S GT1 - Intel Core i3-12100 20K 40K 60K 80K 100K SE +/- 881.92, N = 3 90229 1. (CXX) g++ options: -fPIC -O3 -rdynamic -lglog -lgflags -lprotobuf -lpthread -lsz -lz -ldl -lm -llmdb -lopenblas
Caffe Model: AlexNet - Acceleration: CPU - Iterations: 100 OpenBenchmarking.org Milli-Seconds, Fewer Is Better Caffe 2020-02-13 Model: AlexNet - Acceleration: CPU - Iterations: 100 Intel ADL-S GT1 - Intel Core i3-12100 10K 20K 30K 40K 50K SE +/- 405.55, N = 3 44622 1. (CXX) g++ options: -fPIC -O3 -rdynamic -lglog -lgflags -lprotobuf -lpthread -lsz -lz -ldl -lm -llmdb -lopenblas
Neural Magic DeepSparse Model: NLP Token Classification, BERT base uncased conll2003 - Scenario: Synchronous Single-Stream OpenBenchmarking.org ms/batch, Fewer Is Better Neural Magic DeepSparse 1.7 Model: NLP Token Classification, BERT base uncased conll2003 - Scenario: Synchronous Single-Stream Intel ADL-S GT1 - Intel Core i3-12100 50 100 150 200 250 SE +/- 0.31, N = 3 230.83
Neural Magic DeepSparse Model: NLP Token Classification, BERT base uncased conll2003 - Scenario: Synchronous Single-Stream OpenBenchmarking.org items/sec, More Is Better Neural Magic DeepSparse 1.7 Model: NLP Token Classification, BERT base uncased conll2003 - Scenario: Synchronous Single-Stream Intel ADL-S GT1 - Intel Core i3-12100 0.9747 1.9494 2.9241 3.8988 4.8735 SE +/- 0.0059, N = 3 4.3321
Neural Magic DeepSparse Model: NLP Token Classification, BERT base uncased conll2003 - Scenario: Asynchronous Multi-Stream OpenBenchmarking.org ms/batch, Fewer Is Better Neural Magic DeepSparse 1.7 Model: NLP Token Classification, BERT base uncased conll2003 - Scenario: Asynchronous Multi-Stream Intel ADL-S GT1 - Intel Core i3-12100 110 220 330 440 550 SE +/- 2.32, N = 3 485.38
Neural Magic DeepSparse Model: NLP Token Classification, BERT base uncased conll2003 - Scenario: Asynchronous Multi-Stream OpenBenchmarking.org items/sec, More Is Better Neural Magic DeepSparse 1.7 Model: NLP Token Classification, BERT base uncased conll2003 - Scenario: Asynchronous Multi-Stream Intel ADL-S GT1 - Intel Core i3-12100 0.9265 1.853 2.7795 3.706 4.6325 SE +/- 0.0219, N = 3 4.1179
Neural Magic DeepSparse Model: BERT-Large, NLP Question Answering, Sparse INT8 - Scenario: Synchronous Single-Stream OpenBenchmarking.org ms/batch, Fewer Is Better Neural Magic DeepSparse 1.7 Model: BERT-Large, NLP Question Answering, Sparse INT8 - Scenario: Synchronous Single-Stream Intel ADL-S GT1 - Intel Core i3-12100 4 8 12 16 20 SE +/- 0.01, N = 3 15.41
Neural Magic DeepSparse Model: BERT-Large, NLP Question Answering, Sparse INT8 - Scenario: Synchronous Single-Stream OpenBenchmarking.org items/sec, More Is Better Neural Magic DeepSparse 1.7 Model: BERT-Large, NLP Question Answering, Sparse INT8 - Scenario: Synchronous Single-Stream Intel ADL-S GT1 - Intel Core i3-12100 14 28 42 56 70 SE +/- 0.06, N = 3 64.85
Neural Magic DeepSparse Model: BERT-Large, NLP Question Answering, Sparse INT8 - Scenario: Asynchronous Multi-Stream OpenBenchmarking.org ms/batch, Fewer Is Better Neural Magic DeepSparse 1.7 Model: BERT-Large, NLP Question Answering, Sparse INT8 - Scenario: Asynchronous Multi-Stream Intel ADL-S GT1 - Intel Core i3-12100 6 12 18 24 30 SE +/- 0.02, N = 3 25.56
Neural Magic DeepSparse Model: BERT-Large, NLP Question Answering, Sparse INT8 - Scenario: Asynchronous Multi-Stream OpenBenchmarking.org items/sec, More Is Better Neural Magic DeepSparse 1.7 Model: BERT-Large, NLP Question Answering, Sparse INT8 - Scenario: Asynchronous Multi-Stream Intel ADL-S GT1 - Intel Core i3-12100 20 40 60 80 100 SE +/- 0.07, N = 3 78.19
Neural Magic DeepSparse Model: CV Segmentation, 90% Pruned YOLACT Pruned - Scenario: Synchronous Single-Stream OpenBenchmarking.org ms/batch, Fewer Is Better Neural Magic DeepSparse 1.7 Model: CV Segmentation, 90% Pruned YOLACT Pruned - Scenario: Synchronous Single-Stream Intel ADL-S GT1 - Intel Core i3-12100 50 100 150 200 250 SE +/- 0.45, N = 3 207.67
Neural Magic DeepSparse Model: CV Segmentation, 90% Pruned YOLACT Pruned - Scenario: Synchronous Single-Stream OpenBenchmarking.org items/sec, More Is Better Neural Magic DeepSparse 1.7 Model: CV Segmentation, 90% Pruned YOLACT Pruned - Scenario: Synchronous Single-Stream Intel ADL-S GT1 - Intel Core i3-12100 1.0834 2.1668 3.2502 4.3336 5.417 SE +/- 0.0104, N = 3 4.8149
Neural Magic DeepSparse Model: CV Segmentation, 90% Pruned YOLACT Pruned - Scenario: Asynchronous Multi-Stream OpenBenchmarking.org ms/batch, Fewer Is Better Neural Magic DeepSparse 1.7 Model: CV Segmentation, 90% Pruned YOLACT Pruned - Scenario: Asynchronous Multi-Stream Intel ADL-S GT1 - Intel Core i3-12100 90 180 270 360 450 SE +/- 3.23, N = 3 399.50
Neural Magic DeepSparse Model: CV Segmentation, 90% Pruned YOLACT Pruned - Scenario: Asynchronous Multi-Stream OpenBenchmarking.org items/sec, More Is Better Neural Magic DeepSparse 1.7 Model: CV Segmentation, 90% Pruned YOLACT Pruned - Scenario: Asynchronous Multi-Stream Intel ADL-S GT1 - Intel Core i3-12100 1.1259 2.2518 3.3777 4.5036 5.6295 SE +/- 0.0404, N = 3 5.0039
Neural Magic DeepSparse Model: NLP Text Classification, DistilBERT mnli - Scenario: Synchronous Single-Stream OpenBenchmarking.org ms/batch, Fewer Is Better Neural Magic DeepSparse 1.7 Model: NLP Text Classification, DistilBERT mnli - Scenario: Synchronous Single-Stream Intel ADL-S GT1 - Intel Core i3-12100 7 14 21 28 35 SE +/- 0.04, N = 3 31.01
Neural Magic DeepSparse Model: NLP Text Classification, DistilBERT mnli - Scenario: Synchronous Single-Stream OpenBenchmarking.org items/sec, More Is Better Neural Magic DeepSparse 1.7 Model: NLP Text Classification, DistilBERT mnli - Scenario: Synchronous Single-Stream Intel ADL-S GT1 - Intel Core i3-12100 7 14 21 28 35 SE +/- 0.05, N = 3 32.24
Neural Magic DeepSparse Model: NLP Text Classification, DistilBERT mnli - Scenario: Asynchronous Multi-Stream OpenBenchmarking.org ms/batch, Fewer Is Better Neural Magic DeepSparse 1.7 Model: NLP Text Classification, DistilBERT mnli - Scenario: Asynchronous Multi-Stream Intel ADL-S GT1 - Intel Core i3-12100 13 26 39 52 65 SE +/- 0.19, N = 3 58.10
Neural Magic DeepSparse Model: NLP Text Classification, DistilBERT mnli - Scenario: Asynchronous Multi-Stream OpenBenchmarking.org items/sec, More Is Better Neural Magic DeepSparse 1.7 Model: NLP Text Classification, DistilBERT mnli - Scenario: Asynchronous Multi-Stream Intel ADL-S GT1 - Intel Core i3-12100 8 16 24 32 40 SE +/- 0.12, N = 3 34.41
Neural Magic DeepSparse Model: CV Detection, YOLOv5s COCO, Sparse INT8 - Scenario: Synchronous Single-Stream OpenBenchmarking.org ms/batch, Fewer Is Better Neural Magic DeepSparse 1.7 Model: CV Detection, YOLOv5s COCO, Sparse INT8 - Scenario: Synchronous Single-Stream Intel ADL-S GT1 - Intel Core i3-12100 10 20 30 40 50 SE +/- 0.02, N = 3 42.63
Neural Magic DeepSparse Model: CV Detection, YOLOv5s COCO, Sparse INT8 - Scenario: Synchronous Single-Stream OpenBenchmarking.org items/sec, More Is Better Neural Magic DeepSparse 1.7 Model: CV Detection, YOLOv5s COCO, Sparse INT8 - Scenario: Synchronous Single-Stream Intel ADL-S GT1 - Intel Core i3-12100 6 12 18 24 30 SE +/- 0.01, N = 3 23.45
Neural Magic DeepSparse Model: CV Detection, YOLOv5s COCO, Sparse INT8 - Scenario: Asynchronous Multi-Stream OpenBenchmarking.org ms/batch, Fewer Is Better Neural Magic DeepSparse 1.7 Model: CV Detection, YOLOv5s COCO, Sparse INT8 - Scenario: Asynchronous Multi-Stream Intel ADL-S GT1 - Intel Core i3-12100 20 40 60 80 100 SE +/- 0.10, N = 3 83.74
Neural Magic DeepSparse Model: CV Detection, YOLOv5s COCO, Sparse INT8 - Scenario: Asynchronous Multi-Stream OpenBenchmarking.org items/sec, More Is Better Neural Magic DeepSparse 1.7 Model: CV Detection, YOLOv5s COCO, Sparse INT8 - Scenario: Asynchronous Multi-Stream Intel ADL-S GT1 - Intel Core i3-12100 6 12 18 24 30 SE +/- 0.03, N = 3 23.88
Neural Magic DeepSparse Model: CV Classification, ResNet-50 ImageNet - Scenario: Synchronous Single-Stream OpenBenchmarking.org ms/batch, Fewer Is Better Neural Magic DeepSparse 1.7 Model: CV Classification, ResNet-50 ImageNet - Scenario: Synchronous Single-Stream Intel ADL-S GT1 - Intel Core i3-12100 5 10 15 20 25 SE +/- 0.02, N = 3 21.06
Neural Magic DeepSparse Model: CV Classification, ResNet-50 ImageNet - Scenario: Synchronous Single-Stream OpenBenchmarking.org items/sec, More Is Better Neural Magic DeepSparse 1.7 Model: CV Classification, ResNet-50 ImageNet - Scenario: Synchronous Single-Stream Intel ADL-S GT1 - Intel Core i3-12100 11 22 33 44 55 SE +/- 0.04, N = 3 47.47
Neural Magic DeepSparse Model: CV Classification, ResNet-50 ImageNet - Scenario: Asynchronous Multi-Stream OpenBenchmarking.org ms/batch, Fewer Is Better Neural Magic DeepSparse 1.7 Model: CV Classification, ResNet-50 ImageNet - Scenario: Asynchronous Multi-Stream Intel ADL-S GT1 - Intel Core i3-12100 9 18 27 36 45 SE +/- 0.11, N = 3 38.76
Neural Magic DeepSparse Model: CV Classification, ResNet-50 ImageNet - Scenario: Asynchronous Multi-Stream OpenBenchmarking.org items/sec, More Is Better Neural Magic DeepSparse 1.7 Model: CV Classification, ResNet-50 ImageNet - Scenario: Asynchronous Multi-Stream Intel ADL-S GT1 - Intel Core i3-12100 12 24 36 48 60 SE +/- 0.15, N = 3 51.57
Neural Magic DeepSparse Model: Llama2 Chat 7b Quantized - Scenario: Synchronous Single-Stream OpenBenchmarking.org ms/batch, Fewer Is Better Neural Magic DeepSparse 1.7 Model: Llama2 Chat 7b Quantized - Scenario: Synchronous Single-Stream Intel ADL-S GT1 - Intel Core i3-12100 20K 40K 60K 80K 100K SE +/- 1162.79, N = 9 108889.85
Neural Magic DeepSparse Model: Llama2 Chat 7b Quantized - Scenario: Synchronous Single-Stream OpenBenchmarking.org items/sec, More Is Better Neural Magic DeepSparse 1.7 Model: Llama2 Chat 7b Quantized - Scenario: Synchronous Single-Stream Intel ADL-S GT1 - Intel Core i3-12100 0.0021 0.0042 0.0063 0.0084 0.0105 SE +/- 0.0001, N = 9 0.0092
Neural Magic DeepSparse Model: Llama2 Chat 7b Quantized - Scenario: Asynchronous Multi-Stream OpenBenchmarking.org ms/batch, Fewer Is Better Neural Magic DeepSparse 1.7 Model: Llama2 Chat 7b Quantized - Scenario: Asynchronous Multi-Stream Intel ADL-S GT1 - Intel Core i3-12100 50K 100K 150K 200K 250K SE +/- 2251.32, N = 9 211288.58
Neural Magic DeepSparse Model: Llama2 Chat 7b Quantized - Scenario: Asynchronous Multi-Stream OpenBenchmarking.org items/sec, More Is Better Neural Magic DeepSparse 1.7 Model: Llama2 Chat 7b Quantized - Scenario: Asynchronous Multi-Stream Intel ADL-S GT1 - Intel Core i3-12100 0.0016 0.0032 0.0048 0.0064 0.008 SE +/- 0.0001, N = 9 0.0070
Neural Magic DeepSparse Model: ResNet-50, Sparse INT8 - Scenario: Synchronous Single-Stream OpenBenchmarking.org ms/batch, Fewer Is Better Neural Magic DeepSparse 1.7 Model: ResNet-50, Sparse INT8 - Scenario: Synchronous Single-Stream Intel ADL-S GT1 - Intel Core i3-12100 0.7475 1.495 2.2425 2.99 3.7375 SE +/- 0.0131, N = 3 3.3220
Neural Magic DeepSparse Model: ResNet-50, Sparse INT8 - Scenario: Synchronous Single-Stream OpenBenchmarking.org items/sec, More Is Better Neural Magic DeepSparse 1.7 Model: ResNet-50, Sparse INT8 - Scenario: Synchronous Single-Stream Intel ADL-S GT1 - Intel Core i3-12100 70 140 210 280 350 SE +/- 1.17, N = 3 300.11
Neural Magic DeepSparse Model: ResNet-50, Sparse INT8 - Scenario: Asynchronous Multi-Stream OpenBenchmarking.org ms/batch, Fewer Is Better Neural Magic DeepSparse 1.7 Model: ResNet-50, Sparse INT8 - Scenario: Asynchronous Multi-Stream Intel ADL-S GT1 - Intel Core i3-12100 1.2133 2.4266 3.6399 4.8532 6.0665 SE +/- 0.0167, N = 3 5.3924
Neural Magic DeepSparse Model: ResNet-50, Sparse INT8 - Scenario: Asynchronous Multi-Stream OpenBenchmarking.org items/sec, More Is Better Neural Magic DeepSparse 1.7 Model: ResNet-50, Sparse INT8 - Scenario: Asynchronous Multi-Stream Intel ADL-S GT1 - Intel Core i3-12100 80 160 240 320 400 SE +/- 1.14, N = 3 369.36
Neural Magic DeepSparse Model: ResNet-50, Baseline - Scenario: Synchronous Single-Stream OpenBenchmarking.org ms/batch, Fewer Is Better Neural Magic DeepSparse 1.7 Model: ResNet-50, Baseline - Scenario: Synchronous Single-Stream Intel ADL-S GT1 - Intel Core i3-12100 5 10 15 20 25 SE +/- 0.02, N = 3 21.02
Neural Magic DeepSparse Model: ResNet-50, Baseline - Scenario: Synchronous Single-Stream OpenBenchmarking.org items/sec, More Is Better Neural Magic DeepSparse 1.7 Model: ResNet-50, Baseline - Scenario: Synchronous Single-Stream Intel ADL-S GT1 - Intel Core i3-12100 11 22 33 44 55 SE +/- 0.05, N = 3 47.55
Neural Magic DeepSparse Model: ResNet-50, Baseline - Scenario: Asynchronous Multi-Stream OpenBenchmarking.org ms/batch, Fewer Is Better Neural Magic DeepSparse 1.7 Model: ResNet-50, Baseline - Scenario: Asynchronous Multi-Stream Intel ADL-S GT1 - Intel Core i3-12100 9 18 27 36 45 SE +/- 0.14, N = 3 38.68
Neural Magic DeepSparse Model: ResNet-50, Baseline - Scenario: Asynchronous Multi-Stream OpenBenchmarking.org items/sec, More Is Better Neural Magic DeepSparse 1.7 Model: ResNet-50, Baseline - Scenario: Asynchronous Multi-Stream Intel ADL-S GT1 - Intel Core i3-12100 12 24 36 48 60 SE +/- 0.19, N = 3 51.69
Neural Magic DeepSparse Model: NLP Text Classification, BERT base uncased SST2, Sparse INT8 - Scenario: Synchronous Single-Stream OpenBenchmarking.org ms/batch, Fewer Is Better Neural Magic DeepSparse 1.7 Model: NLP Text Classification, BERT base uncased SST2, Sparse INT8 - Scenario: Synchronous Single-Stream Intel ADL-S GT1 - Intel Core i3-12100 2 4 6 8 10 SE +/- 0.0159, N = 3 6.8777
Neural Magic DeepSparse Model: NLP Text Classification, BERT base uncased SST2, Sparse INT8 - Scenario: Synchronous Single-Stream OpenBenchmarking.org items/sec, More Is Better Neural Magic DeepSparse 1.7 Model: NLP Text Classification, BERT base uncased SST2, Sparse INT8 - Scenario: Synchronous Single-Stream Intel ADL-S GT1 - Intel Core i3-12100 30 60 90 120 150 SE +/- 0.34, N = 3 145.27
Neural Magic DeepSparse Model: NLP Text Classification, BERT base uncased SST2, Sparse INT8 - Scenario: Asynchronous Multi-Stream OpenBenchmarking.org ms/batch, Fewer Is Better Neural Magic DeepSparse 1.7 Model: NLP Text Classification, BERT base uncased SST2, Sparse INT8 - Scenario: Asynchronous Multi-Stream Intel ADL-S GT1 - Intel Core i3-12100 3 6 9 12 15 SE +/- 0.00, N = 3 12.13
Neural Magic DeepSparse Model: NLP Text Classification, BERT base uncased SST2, Sparse INT8 - Scenario: Asynchronous Multi-Stream OpenBenchmarking.org items/sec, More Is Better Neural Magic DeepSparse 1.7 Model: NLP Text Classification, BERT base uncased SST2, Sparse INT8 - Scenario: Asynchronous Multi-Stream Intel ADL-S GT1 - Intel Core i3-12100 40 80 120 160 200 SE +/- 0.07, N = 3 164.65
Neural Magic DeepSparse Model: NLP Document Classification, oBERT base uncased on IMDB - Scenario: Synchronous Single-Stream OpenBenchmarking.org ms/batch, Fewer Is Better Neural Magic DeepSparse 1.7 Model: NLP Document Classification, oBERT base uncased on IMDB - Scenario: Synchronous Single-Stream Intel ADL-S GT1 - Intel Core i3-12100 50 100 150 200 250 SE +/- 0.21, N = 3 231.81
Neural Magic DeepSparse Model: NLP Document Classification, oBERT base uncased on IMDB - Scenario: Synchronous Single-Stream OpenBenchmarking.org items/sec, More Is Better Neural Magic DeepSparse 1.7 Model: NLP Document Classification, oBERT base uncased on IMDB - Scenario: Synchronous Single-Stream Intel ADL-S GT1 - Intel Core i3-12100 0.9706 1.9412 2.9118 3.8824 4.853 SE +/- 0.0039, N = 3 4.3138
Neural Magic DeepSparse Model: NLP Document Classification, oBERT base uncased on IMDB - Scenario: Asynchronous Multi-Stream OpenBenchmarking.org ms/batch, Fewer Is Better Neural Magic DeepSparse 1.7 Model: NLP Document Classification, oBERT base uncased on IMDB - Scenario: Asynchronous Multi-Stream Intel ADL-S GT1 - Intel Core i3-12100 110 220 330 440 550 SE +/- 1.50, N = 3 490.80
Neural Magic DeepSparse Model: NLP Document Classification, oBERT base uncased on IMDB - Scenario: Asynchronous Multi-Stream OpenBenchmarking.org items/sec, More Is Better Neural Magic DeepSparse 1.7 Model: NLP Document Classification, oBERT base uncased on IMDB - Scenario: Asynchronous Multi-Stream Intel ADL-S GT1 - Intel Core i3-12100 0.9169 1.8338 2.7507 3.6676 4.5845 SE +/- 0.0124, N = 3 4.0749
PyTorch Device: CPU - Batch Size: 512 - Model: Efficientnet_v2_l OpenBenchmarking.org batches/sec, More Is Better PyTorch 2.2.1 Device: CPU - Batch Size: 512 - Model: Efficientnet_v2_l Intel ADL-S GT1 - Intel Core i3-12100 1.008 2.016 3.024 4.032 5.04 SE +/- 0.04, N = 3 4.48 MIN: 3.37 / MAX: 4.59
PyTorch Device: CPU - Batch Size: 256 - Model: Efficientnet_v2_l OpenBenchmarking.org batches/sec, More Is Better PyTorch 2.2.1 Device: CPU - Batch Size: 256 - Model: Efficientnet_v2_l Intel ADL-S GT1 - Intel Core i3-12100 1.0125 2.025 3.0375 4.05 5.0625 SE +/- 0.02, N = 3 4.50 MIN: 3.35 / MAX: 4.57
PyTorch Device: CPU - Batch Size: 64 - Model: Efficientnet_v2_l OpenBenchmarking.org batches/sec, More Is Better PyTorch 2.2.1 Device: CPU - Batch Size: 64 - Model: Efficientnet_v2_l Intel ADL-S GT1 - Intel Core i3-12100 1.017 2.034 3.051 4.068 5.085 SE +/- 0.01, N = 3 4.52 MIN: 3.17 / MAX: 4.6
PyTorch Device: CPU - Batch Size: 32 - Model: Efficientnet_v2_l OpenBenchmarking.org batches/sec, More Is Better PyTorch 2.2.1 Device: CPU - Batch Size: 32 - Model: Efficientnet_v2_l Intel ADL-S GT1 - Intel Core i3-12100 1.0238 2.0476 3.0714 4.0952 5.119 SE +/- 0.01, N = 3 4.55 MIN: 3.32 / MAX: 4.61
PyTorch Device: CPU - Batch Size: 16 - Model: Efficientnet_v2_l OpenBenchmarking.org batches/sec, More Is Better PyTorch 2.2.1 Device: CPU - Batch Size: 16 - Model: Efficientnet_v2_l Intel ADL-S GT1 - Intel Core i3-12100 1.0148 2.0296 3.0444 4.0592 5.074 SE +/- 0.04, N = 3 4.51 MIN: 3.16 / MAX: 4.61
PyTorch Device: CPU - Batch Size: 1 - Model: Efficientnet_v2_l OpenBenchmarking.org batches/sec, More Is Better PyTorch 2.2.1 Device: CPU - Batch Size: 1 - Model: Efficientnet_v2_l Intel ADL-S GT1 - Intel Core i3-12100 2 4 6 8 10 SE +/- 0.08, N = 5 8.10 MIN: 5.47 / MAX: 8.32
PyTorch Device: CPU - Batch Size: 512 - Model: ResNet-152 OpenBenchmarking.org batches/sec, More Is Better PyTorch 2.2.1 Device: CPU - Batch Size: 512 - Model: ResNet-152 Intel ADL-S GT1 - Intel Core i3-12100 2 4 6 8 10 SE +/- 0.03, N = 3 6.17 MIN: 4.17 / MAX: 6.34
PyTorch Device: CPU - Batch Size: 256 - Model: ResNet-152 OpenBenchmarking.org batches/sec, More Is Better PyTorch 2.2.1 Device: CPU - Batch Size: 256 - Model: ResNet-152 Intel ADL-S GT1 - Intel Core i3-12100 2 4 6 8 10 SE +/- 0.02, N = 3 6.22 MIN: 4.3 / MAX: 6.36
PyTorch Device: CPU - Batch Size: 64 - Model: ResNet-152 OpenBenchmarking.org batches/sec, More Is Better PyTorch 2.2.1 Device: CPU - Batch Size: 64 - Model: ResNet-152 Intel ADL-S GT1 - Intel Core i3-12100 2 4 6 8 10 SE +/- 0.03, N = 3 6.23 MIN: 3.95 / MAX: 6.36
PyTorch Device: CPU - Batch Size: 512 - Model: ResNet-50 OpenBenchmarking.org batches/sec, More Is Better PyTorch 2.2.1 Device: CPU - Batch Size: 512 - Model: ResNet-50 Intel ADL-S GT1 - Intel Core i3-12100 3 6 9 12 15 SE +/- 0.18, N = 3 12.76 MIN: 7.94 / MAX: 13.36
PyTorch Device: CPU - Batch Size: 32 - Model: ResNet-152 OpenBenchmarking.org batches/sec, More Is Better PyTorch 2.2.1 Device: CPU - Batch Size: 32 - Model: ResNet-152 Intel ADL-S GT1 - Intel Core i3-12100 2 4 6 8 10 SE +/- 0.03, N = 3 6.24 MIN: 4.36 / MAX: 6.39
PyTorch Device: CPU - Batch Size: 256 - Model: ResNet-50 OpenBenchmarking.org batches/sec, More Is Better PyTorch 2.2.1 Device: CPU - Batch Size: 256 - Model: ResNet-50 Intel ADL-S GT1 - Intel Core i3-12100 3 6 9 12 15 SE +/- 0.09, N = 3 12.91 MIN: 8.58 / MAX: 13.25
PyTorch Device: CPU - Batch Size: 16 - Model: ResNet-152 OpenBenchmarking.org batches/sec, More Is Better PyTorch 2.2.1 Device: CPU - Batch Size: 16 - Model: ResNet-152 Intel ADL-S GT1 - Intel Core i3-12100 2 4 6 8 10 SE +/- 0.04, N = 3 6.33 MIN: 4.35 / MAX: 6.45
PyTorch Device: CPU - Batch Size: 64 - Model: ResNet-50 OpenBenchmarking.org batches/sec, More Is Better PyTorch 2.2.1 Device: CPU - Batch Size: 64 - Model: ResNet-50 Intel ADL-S GT1 - Intel Core i3-12100 3 6 9 12 15 SE +/- 0.12, N = 3 12.69 MIN: 8.56 / MAX: 13.22
PyTorch Device: CPU - Batch Size: 32 - Model: ResNet-50 OpenBenchmarking.org batches/sec, More Is Better PyTorch 2.2.1 Device: CPU - Batch Size: 32 - Model: ResNet-50 Intel ADL-S GT1 - Intel Core i3-12100 3 6 9 12 15 SE +/- 0.11, N = 8 12.56 MIN: 7.8 / MAX: 13.33
PyTorch Device: CPU - Batch Size: 16 - Model: ResNet-50 OpenBenchmarking.org batches/sec, More Is Better PyTorch 2.2.1 Device: CPU - Batch Size: 16 - Model: ResNet-50 Intel ADL-S GT1 - Intel Core i3-12100 3 6 9 12 15 SE +/- 0.11, N = 3 12.57 MIN: 8.58 / MAX: 12.97
PyTorch Device: CPU - Batch Size: 1 - Model: ResNet-152 OpenBenchmarking.org batches/sec, More Is Better PyTorch 2.2.1 Device: CPU - Batch Size: 1 - Model: ResNet-152 Intel ADL-S GT1 - Intel Core i3-12100 3 6 9 12 15 SE +/- 0.03, N = 3 11.34 MIN: 7.58 / MAX: 11.54
PyTorch Device: CPU - Batch Size: 1 - Model: ResNet-50 OpenBenchmarking.org batches/sec, More Is Better PyTorch 2.2.1 Device: CPU - Batch Size: 1 - Model: ResNet-50 Intel ADL-S GT1 - Intel Core i3-12100 6 12 18 24 30 SE +/- 0.26, N = 4 24.89 MIN: 11.71 / MAX: 25.9
TensorFlow Lite Model: Inception ResNet V2 OpenBenchmarking.org Microseconds, Fewer Is Better TensorFlow Lite 2022-05-18 Model: Inception ResNet V2 Intel ADL-S GT1 - Intel Core i3-12100 13K 26K 39K 52K 65K SE +/- 49.41, N = 3 60949.0
TensorFlow Lite Model: Mobilenet Quant OpenBenchmarking.org Microseconds, Fewer Is Better TensorFlow Lite 2022-05-18 Model: Mobilenet Quant Intel ADL-S GT1 - Intel Core i3-12100 1100 2200 3300 4400 5500 SE +/- 9.03, N = 3 5198.07
TensorFlow Lite Model: Mobilenet Float OpenBenchmarking.org Microseconds, Fewer Is Better TensorFlow Lite 2022-05-18 Model: Mobilenet Float Intel ADL-S GT1 - Intel Core i3-12100 800 1600 2400 3200 4000 SE +/- 12.76, N = 3 3574.97
TensorFlow Lite Model: NASNet Mobile OpenBenchmarking.org Microseconds, Fewer Is Better TensorFlow Lite 2022-05-18 Model: NASNet Mobile Intel ADL-S GT1 - Intel Core i3-12100 3K 6K 9K 12K 15K SE +/- 3.59, N = 3 12005.2
TensorFlow Lite Model: Inception V4 OpenBenchmarking.org Microseconds, Fewer Is Better TensorFlow Lite 2022-05-18 Model: Inception V4 Intel ADL-S GT1 - Intel Core i3-12100 14K 28K 42K 56K 70K SE +/- 110.99, N = 3 64931.8
TensorFlow Lite Model: SqueezeNet OpenBenchmarking.org Microseconds, Fewer Is Better TensorFlow Lite 2022-05-18 Model: SqueezeNet Intel ADL-S GT1 - Intel Core i3-12100 900 1800 2700 3600 4500 SE +/- 12.96, N = 3 4322.71
R Benchmark OpenBenchmarking.org Seconds, Fewer Is Better R Benchmark Intel ADL-S GT1 - Intel Core i3-12100 0.0416 0.0832 0.1248 0.1664 0.208 SE +/- 0.0015, N = 3 0.1847 1. R scripting front-end version 3.6.3 (2020-02-29)
DeepSpeech Acceleration: CPU OpenBenchmarking.org Seconds, Fewer Is Better DeepSpeech 0.6 Acceleration: CPU Intel ADL-S GT1 - Intel Core i3-12100 20 40 60 80 100 SE +/- 0.60, N = 3 87.52
Numpy Benchmark OpenBenchmarking.org Score, More Is Better Numpy Benchmark Intel ADL-S GT1 - Intel Core i3-12100 100 200 300 400 500 SE +/- 0.39, N = 3 463.17
oneDNN Harness: Recurrent Neural Network Inference - Engine: CPU OpenBenchmarking.org ms, Fewer Is Better oneDNN 3.4 Harness: Recurrent Neural Network Inference - Engine: CPU Intel ADL-S GT1 - Intel Core i3-12100 700 1400 2100 2800 3500 SE +/- 12.33, N = 3 3071.31 MIN: 3037.61 1. (CXX) g++ options: -O3 -march=native -fopenmp -msse4.1 -fPIC -pie -lpthread -ldl
oneDNN Harness: Recurrent Neural Network Training - Engine: CPU OpenBenchmarking.org ms, Fewer Is Better oneDNN 3.4 Harness: Recurrent Neural Network Training - Engine: CPU Intel ADL-S GT1 - Intel Core i3-12100 1200 2400 3600 4800 6000 SE +/- 4.76, N = 3 5708.97 MIN: 5677.08 1. (CXX) g++ options: -O3 -march=native -fopenmp -msse4.1 -fPIC -pie -lpthread -ldl
oneDNN Harness: Deconvolution Batch shapes_3d - Engine: CPU OpenBenchmarking.org ms, Fewer Is Better oneDNN 3.4 Harness: Deconvolution Batch shapes_3d - Engine: CPU Intel ADL-S GT1 - Intel Core i3-12100 3 6 9 12 15 SE +/- 0.09, N = 3 10.37 MIN: 10.1 1. (CXX) g++ options: -O3 -march=native -fopenmp -msse4.1 -fPIC -pie -lpthread -ldl
oneDNN Harness: Deconvolution Batch shapes_1d - Engine: CPU OpenBenchmarking.org ms, Fewer Is Better oneDNN 3.4 Harness: Deconvolution Batch shapes_1d - Engine: CPU Intel ADL-S GT1 - Intel Core i3-12100 3 6 9 12 15 SE +/- 0.10, N = 15 10.75 MIN: 8.01 1. (CXX) g++ options: -O3 -march=native -fopenmp -msse4.1 -fPIC -pie -lpthread -ldl
oneDNN Harness: Convolution Batch Shapes Auto - Engine: CPU OpenBenchmarking.org ms, Fewer Is Better oneDNN 3.4 Harness: Convolution Batch Shapes Auto - Engine: CPU Intel ADL-S GT1 - Intel Core i3-12100 10 20 30 40 50 SE +/- 0.43, N = 3 44.14 MIN: 43.16 1. (CXX) g++ options: -O3 -march=native -fopenmp -msse4.1 -fPIC -pie -lpthread -ldl
oneDNN Harness: IP Shapes 3D - Engine: CPU OpenBenchmarking.org ms, Fewer Is Better oneDNN 3.4 Harness: IP Shapes 3D - Engine: CPU Intel ADL-S GT1 - Intel Core i3-12100 6 12 18 24 30 SE +/- 0.03, N = 3 26.70 MIN: 26.52 1. (CXX) g++ options: -O3 -march=native -fopenmp -msse4.1 -fPIC -pie -lpthread -ldl
oneDNN Harness: IP Shapes 1D - Engine: CPU OpenBenchmarking.org ms, Fewer Is Better oneDNN 3.4 Harness: IP Shapes 1D - Engine: CPU Intel ADL-S GT1 - Intel Core i3-12100 2 4 6 8 10 SE +/- 0.01732, N = 3 6.00601 MIN: 5.51 1. (CXX) g++ options: -O3 -march=native -fopenmp -msse4.1 -fPIC -pie -lpthread -ldl
OpenCV Test: DNN - Deep Neural Network OpenBenchmarking.org ms, Fewer Is Better OpenCV 4.7 Test: DNN - Deep Neural Network Intel ADL-S GT1 - Intel Core i3-12100 6K 12K 18K 24K 30K SE +/- 832.87, N = 12 27185 1. (CXX) g++ options: -fPIC -fsigned-char -pthread -fomit-frame-pointer -ffunction-sections -fdata-sections -msse -msse2 -msse3 -fvisibility=hidden -O3 -shared
Scikit-Learn Benchmark: Kernel PCA Solvers / Time vs. N Components OpenBenchmarking.org Seconds, Fewer Is Better Scikit-Learn 1.2.2 Benchmark: Kernel PCA Solvers / Time vs. N Components Intel ADL-S GT1 - Intel Core i3-12100 70 140 210 280 350 SE +/- 14.09, N = 9 319.47 1. (F9X) gfortran options: -O0
ONNX Runtime Model: Faster R-CNN R-50-FPN-int8 - Device: CPU - Executor: Standard OpenBenchmarking.org Inference Time Cost (ms), Fewer Is Better ONNX Runtime 1.17 Model: Faster R-CNN R-50-FPN-int8 - Device: CPU - Executor: Standard Intel ADL-S GT1 - Intel Core i3-12100 40 80 120 160 200 SE +/- 2.83, N = 12 196.43 1. (CXX) g++ options: -O3 -march=native -ffunction-sections -fdata-sections -mtune=native -flto -fno-fat-lto-objects -ldl -lrt -lpthread -pthread
ONNX Runtime Model: Faster R-CNN R-50-FPN-int8 - Device: CPU - Executor: Parallel OpenBenchmarking.org Inference Time Cost (ms), Fewer Is Better ONNX Runtime 1.17 Model: Faster R-CNN R-50-FPN-int8 - Device: CPU - Executor: Parallel Intel ADL-S GT1 - Intel Core i3-12100 50 100 150 200 250 SE +/- 1.70, N = 3 225.47 1. (CXX) g++ options: -O3 -march=native -ffunction-sections -fdata-sections -mtune=native -flto -fno-fat-lto-objects -ldl -lrt -lpthread -pthread
ONNX Runtime Model: super-resolution-10 - Device: CPU - Executor: Standard OpenBenchmarking.org Inference Time Cost (ms), Fewer Is Better ONNX Runtime 1.17 Model: super-resolution-10 - Device: CPU - Executor: Standard Intel ADL-S GT1 - Intel Core i3-12100 5 10 15 20 25 SE +/- 0.90, N = 15 19.21 1. (CXX) g++ options: -O3 -march=native -ffunction-sections -fdata-sections -mtune=native -flto -fno-fat-lto-objects -ldl -lrt -lpthread -pthread
ONNX Runtime Model: super-resolution-10 - Device: CPU - Executor: Standard OpenBenchmarking.org Inferences Per Second, More Is Better ONNX Runtime 1.17 Model: super-resolution-10 - Device: CPU - Executor: Standard Intel ADL-S GT1 - Intel Core i3-12100 12 24 36 48 60 SE +/- 1.87, N = 15 53.26 1. (CXX) g++ options: -O3 -march=native -ffunction-sections -fdata-sections -mtune=native -flto -fno-fat-lto-objects -ldl -lrt -lpthread -pthread
ONNX Runtime Model: super-resolution-10 - Device: CPU - Executor: Parallel OpenBenchmarking.org Inference Time Cost (ms), Fewer Is Better ONNX Runtime 1.17 Model: super-resolution-10 - Device: CPU - Executor: Parallel Intel ADL-S GT1 - Intel Core i3-12100 6 12 18 24 30 SE +/- 0.28, N = 4 26.29 1. (CXX) g++ options: -O3 -march=native -ffunction-sections -fdata-sections -mtune=native -flto -fno-fat-lto-objects -ldl -lrt -lpthread -pthread
ONNX Runtime Model: ResNet50 v1-12-int8 - Device: CPU - Executor: Standard OpenBenchmarking.org Inference Time Cost (ms), Fewer Is Better ONNX Runtime 1.17 Model: ResNet50 v1-12-int8 - Device: CPU - Executor: Standard Intel ADL-S GT1 - Intel Core i3-12100 2 4 6 8 10 SE +/- 0.04100, N = 3 7.00536 1. (CXX) g++ options: -O3 -march=native -ffunction-sections -fdata-sections -mtune=native -flto -fno-fat-lto-objects -ldl -lrt -lpthread -pthread
ONNX Runtime Model: ResNet50 v1-12-int8 - Device: CPU - Executor: Parallel OpenBenchmarking.org Inference Time Cost (ms), Fewer Is Better ONNX Runtime 1.17 Model: ResNet50 v1-12-int8 - Device: CPU - Executor: Parallel Intel ADL-S GT1 - Intel Core i3-12100 2 4 6 8 10 SE +/- 0.09036, N = 3 8.19737 1. (CXX) g++ options: -O3 -march=native -ffunction-sections -fdata-sections -mtune=native -flto -fno-fat-lto-objects -ldl -lrt -lpthread -pthread
ONNX Runtime Model: ArcFace ResNet-100 - Device: CPU - Executor: Standard OpenBenchmarking.org Inference Time Cost (ms), Fewer Is Better ONNX Runtime 1.17 Model: ArcFace ResNet-100 - Device: CPU - Executor: Standard Intel ADL-S GT1 - Intel Core i3-12100 13 26 39 52 65 SE +/- 0.65, N = 5 57.79 1. (CXX) g++ options: -O3 -march=native -ffunction-sections -fdata-sections -mtune=native -flto -fno-fat-lto-objects -ldl -lrt -lpthread -pthread
ONNX Runtime Model: ArcFace ResNet-100 - Device: CPU - Executor: Parallel OpenBenchmarking.org Inference Time Cost (ms), Fewer Is Better ONNX Runtime 1.17 Model: ArcFace ResNet-100 - Device: CPU - Executor: Parallel Intel ADL-S GT1 - Intel Core i3-12100 20 40 60 80 100 SE +/- 0.89, N = 6 86.70 1. (CXX) g++ options: -O3 -march=native -ffunction-sections -fdata-sections -mtune=native -flto -fno-fat-lto-objects -ldl -lrt -lpthread -pthread
ONNX Runtime Model: fcn-resnet101-11 - Device: CPU - Executor: Standard OpenBenchmarking.org Inference Time Cost (ms), Fewer Is Better ONNX Runtime 1.17 Model: fcn-resnet101-11 - Device: CPU - Executor: Standard Intel ADL-S GT1 - Intel Core i3-12100 300 600 900 1200 1500 SE +/- 26.80, N = 15 1353.24 1. (CXX) g++ options: -O3 -march=native -ffunction-sections -fdata-sections -mtune=native -flto -fno-fat-lto-objects -ldl -lrt -lpthread -pthread
ONNX Runtime Model: fcn-resnet101-11 - Device: CPU - Executor: Standard OpenBenchmarking.org Inferences Per Second, More Is Better ONNX Runtime 1.17 Model: fcn-resnet101-11 - Device: CPU - Executor: Standard Intel ADL-S GT1 - Intel Core i3-12100 0.1671 0.3342 0.5013 0.6684 0.8355 SE +/- 0.012881, N = 15 0.742532 1. (CXX) g++ options: -O3 -march=native -ffunction-sections -fdata-sections -mtune=native -flto -fno-fat-lto-objects -ldl -lrt -lpthread -pthread
ONNX Runtime Model: fcn-resnet101-11 - Device: CPU - Executor: Parallel OpenBenchmarking.org Inference Time Cost (ms), Fewer Is Better ONNX Runtime 1.17 Model: fcn-resnet101-11 - Device: CPU - Executor: Parallel Intel ADL-S GT1 - Intel Core i3-12100 400 800 1200 1600 2000 SE +/- 57.61, N = 15 1975.36 1. (CXX) g++ options: -O3 -march=native -ffunction-sections -fdata-sections -mtune=native -flto -fno-fat-lto-objects -ldl -lrt -lpthread -pthread
ONNX Runtime Model: fcn-resnet101-11 - Device: CPU - Executor: Parallel OpenBenchmarking.org Inferences Per Second, More Is Better ONNX Runtime 1.17 Model: fcn-resnet101-11 - Device: CPU - Executor: Parallel Intel ADL-S GT1 - Intel Core i3-12100 0.1153 0.2306 0.3459 0.4612 0.5765 SE +/- 0.015328, N = 15 0.512476 1. (CXX) g++ options: -O3 -march=native -ffunction-sections -fdata-sections -mtune=native -flto -fno-fat-lto-objects -ldl -lrt -lpthread -pthread
ONNX Runtime Model: CaffeNet 12-int8 - Device: CPU - Executor: Standard OpenBenchmarking.org Inference Time Cost (ms), Fewer Is Better ONNX Runtime 1.17 Model: CaffeNet 12-int8 - Device: CPU - Executor: Standard Intel ADL-S GT1 - Intel Core i3-12100 0.9647 1.9294 2.8941 3.8588 4.8235 SE +/- 0.04898, N = 3 4.28771 1. (CXX) g++ options: -O3 -march=native -ffunction-sections -fdata-sections -mtune=native -flto -fno-fat-lto-objects -ldl -lrt -lpthread -pthread
ONNX Runtime Model: CaffeNet 12-int8 - Device: CPU - Executor: Parallel OpenBenchmarking.org Inference Time Cost (ms), Fewer Is Better ONNX Runtime 1.17 Model: CaffeNet 12-int8 - Device: CPU - Executor: Parallel Intel ADL-S GT1 - Intel Core i3-12100 1.0743 2.1486 3.2229 4.2972 5.3715 SE +/- 0.05286, N = 3 4.77472 1. (CXX) g++ options: -O3 -march=native -ffunction-sections -fdata-sections -mtune=native -flto -fno-fat-lto-objects -ldl -lrt -lpthread -pthread
ONNX Runtime Model: bertsquad-12 - Device: CPU - Executor: Standard OpenBenchmarking.org Inference Time Cost (ms), Fewer Is Better ONNX Runtime 1.17 Model: bertsquad-12 - Device: CPU - Executor: Standard Intel ADL-S GT1 - Intel Core i3-12100 30 60 90 120 150 SE +/- 8.08, N = 14 130.81 1. (CXX) g++ options: -O3 -march=native -ffunction-sections -fdata-sections -mtune=native -flto -fno-fat-lto-objects -ldl -lrt -lpthread -pthread
ONNX Runtime Model: bertsquad-12 - Device: CPU - Executor: Standard OpenBenchmarking.org Inferences Per Second, More Is Better ONNX Runtime 1.17 Model: bertsquad-12 - Device: CPU - Executor: Standard Intel ADL-S GT1 - Intel Core i3-12100 2 4 6 8 10 SE +/- 0.33732, N = 14 7.91502 1. (CXX) g++ options: -O3 -march=native -ffunction-sections -fdata-sections -mtune=native -flto -fno-fat-lto-objects -ldl -lrt -lpthread -pthread
ONNX Runtime Model: bertsquad-12 - Device: CPU - Executor: Parallel OpenBenchmarking.org Inference Time Cost (ms), Fewer Is Better ONNX Runtime 1.17 Model: bertsquad-12 - Device: CPU - Executor: Parallel Intel ADL-S GT1 - Intel Core i3-12100 40 80 120 160 200 SE +/- 2.89, N = 15 187.69 1. (CXX) g++ options: -O3 -march=native -ffunction-sections -fdata-sections -mtune=native -flto -fno-fat-lto-objects -ldl -lrt -lpthread -pthread
ONNX Runtime Model: T5 Encoder - Device: CPU - Executor: Standard OpenBenchmarking.org Inference Time Cost (ms), Fewer Is Better ONNX Runtime 1.17 Model: T5 Encoder - Device: CPU - Executor: Standard Intel ADL-S GT1 - Intel Core i3-12100 4 8 12 16 20 SE +/- 0.01, N = 3 17.36 1. (CXX) g++ options: -O3 -march=native -ffunction-sections -fdata-sections -mtune=native -flto -fno-fat-lto-objects -ldl -lrt -lpthread -pthread
ONNX Runtime Model: T5 Encoder - Device: CPU - Executor: Parallel OpenBenchmarking.org Inference Time Cost (ms), Fewer Is Better ONNX Runtime 1.17 Model: T5 Encoder - Device: CPU - Executor: Parallel Intel ADL-S GT1 - Intel Core i3-12100 4 8 12 16 20 SE +/- 0.01, N = 3 17.85 1. (CXX) g++ options: -O3 -march=native -ffunction-sections -fdata-sections -mtune=native -flto -fno-fat-lto-objects -ldl -lrt -lpthread -pthread
ONNX Runtime Model: GPT-2 - Device: CPU - Executor: Standard OpenBenchmarking.org Inference Time Cost (ms), Fewer Is Better ONNX Runtime 1.17 Model: GPT-2 - Device: CPU - Executor: Standard Intel ADL-S GT1 - Intel Core i3-12100 5 10 15 20 25 SE +/- 0.19, N = 3 19.56 1. (CXX) g++ options: -O3 -march=native -ffunction-sections -fdata-sections -mtune=native -flto -fno-fat-lto-objects -ldl -lrt -lpthread -pthread
ONNX Runtime Model: GPT-2 - Device: CPU - Executor: Parallel OpenBenchmarking.org Inference Time Cost (ms), Fewer Is Better ONNX Runtime 1.17 Model: GPT-2 - Device: CPU - Executor: Parallel Intel ADL-S GT1 - Intel Core i3-12100 5 10 15 20 25 SE +/- 0.03, N = 3 19.87 1. (CXX) g++ options: -O3 -march=native -ffunction-sections -fdata-sections -mtune=native -flto -fno-fat-lto-objects -ldl -lrt -lpthread -pthread
NCNN Target: CPU - Model: mnasnet OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: CPU - Model: mnasnet Intel ADL-S GT1 - Intel Core i3-12100 0.7943 1.5886 2.3829 3.1772 3.9715 SE +/- 0.25, N = 4 3.53 MIN: 3.13 / MAX: 39.25 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread -pthread
Mobile Neural Network Model: mobilenetV3 OpenBenchmarking.org ms, Fewer Is Better Mobile Neural Network 2.9.b11b7037d Model: mobilenetV3 Intel ADL-S GT1 - Intel Core i3-12100 0.2673 0.5346 0.8019 1.0692 1.3365 SE +/- 0.047, N = 3 1.188 MIN: 1.12 / MAX: 49.7 1. (CXX) g++ options: -std=c++11 -O3 -fvisibility=hidden -fomit-frame-pointer -fstrict-aliasing -ffunction-sections -fdata-sections -ffast-math -fno-rtti -fno-exceptions -pthread -ldl
Phoronix Test Suite v10.8.5