Intel Core i3-12100 testing with a ASRock H610M-HDV/M.2 R2.0 (6.03 BIOS) and Intel ADL-S GT1 3GB on Ubuntu 20.04 via the Phoronix Test Suite.
Compare your own system(s) to this result file with the
Phoronix Test Suite by running the command:
phoronix-test-suite benchmark 2408228-HERT-H610I3171 H610-i312100-1 - Phoronix Test Suite H610-i312100-1 Intel Core i3-12100 testing with a ASRock H610M-HDV/M.2 R2.0 (6.03 BIOS) and Intel ADL-S GT1 3GB on Ubuntu 20.04 via the Phoronix Test Suite.
HTML result view exported from: https://openbenchmarking.org/result/2408228-HERT-H610I3171&grw .
H610-i312100-1 Processor Motherboard Chipset Memory Disk Graphics Audio Network OS Kernel Desktop Display Server OpenGL Vulkan Compiler File-System Screen Resolution Intel ADL-S GT1 - Intel Core i3-12100 Intel Core i3-12100 @ 4.30GHz (4 Cores / 8 Threads) ASRock H610M-HDV/M.2 R2.0 (6.03 BIOS) Intel Device 7aa7 4096MB 1000GB Western Digital WDS100T2B0A Intel ADL-S GT1 3GB (1400MHz) Realtek ALC897 Realtek RTL8111/8168/8411 Ubuntu 20.04 5.15.0-89-generic (x86_64) GNOME Shell 3.36.9 X Server 1.20.13 4.6 Mesa 21.2.6 1.2.182 GCC 9.4.0 ext4 1366x768 OpenBenchmarking.org - Transparent Huge Pages: madvise - --build=x86_64-linux-gnu --disable-vtable-verify --disable-werror --enable-checking=release --enable-clocale=gnu --enable-default-pie --enable-gnu-unique-object --enable-languages=c,ada,c++,go,brig,d,fortran,objc,obj-c++,gm2 --enable-libstdcxx-debug --enable-libstdcxx-time=yes --enable-multiarch --enable-multilib --enable-nls --enable-objc-gc=auto --enable-offload-targets=nvptx-none=/build/gcc-9-9QDOt0/gcc-9-9.4.0/debian/tmp-nvptx/usr,hsa --enable-plugin --enable-shared --enable-threads=posix --host=x86_64-linux-gnu --program-prefix=x86_64-linux-gnu- --target=x86_64-linux-gnu --with-abi=m64 --with-arch-32=i686 --with-default-libstdcxx-abi=new --with-gcc-major-version-only --with-multilib-list=m32,m64,mx32 --with-target-system-zlib=auto --with-tune=generic --without-cuda-driver -v - Scaling Governor: intel_pstate powersave (EPP: balance_performance) - CPU Microcode: 0x2e - Python 3.8.10 - gather_data_sampling: Not affected + itlb_multihit: Not affected + l1tf: Not affected + mds: Not affected + meltdown: Not affected + mmio_stale_data: Not affected + retbleed: Not affected + spec_rstack_overflow: Not affected + spec_store_bypass: Mitigation of SSB disabled via prctl and seccomp + spectre_v1: Mitigation of usercopy/swapgs barriers and __user pointer sanitization + spectre_v2: Mitigation of Enhanced IBRS IBPB: conditional RSB filling PBRSB-eIBRS: SW sequence + srbds: Not affected + tsx_async_abort: Not affected
H610-i312100-1 plaidml: No - Inference - VGG16 - CPU plaidml: No - Inference - ResNet 50 - CPU numenta-nab: KNN CAD numenta-nab: Relative Entropy numenta-nab: Windowed Gaussian numenta-nab: Earthgecko Skyline numenta-nab: Bayesian Changepoint numenta-nab: Contextual Anomaly Detector OSE scikit-learn: GLM scikit-learn: SAGA scikit-learn: Tree scikit-learn: Lasso scikit-learn: Sparsify scikit-learn: Plot Ward scikit-learn: MNIST Dataset scikit-learn: Plot Neighbors scikit-learn: SGD Regression scikit-learn: Plot Lasso Path scikit-learn: Text Vectorizers scikit-learn: Plot Hierarchical scikit-learn: Plot OMP vs. LARS scikit-learn: Feature Expansions scikit-learn: LocalOutlierFactor scikit-learn: TSNE MNIST Dataset scikit-learn: Plot Incremental PCA scikit-learn: Hist Gradient Boosting scikit-learn: Sample Without Replacement scikit-learn: Covertype Dataset Benchmark scikit-learn: Hist Gradient Boosting Adult scikit-learn: Hist Gradient Boosting Threading scikit-learn: Plot Singular Value Decomposition scikit-learn: Hist Gradient Boosting Higgs Boson scikit-learn: 20 Newsgroups / Logistic Regression scikit-learn: Plot Polynomial Kernel Approximation scikit-learn: Hist Gradient Boosting Categorical Only scikit-learn: Kernel PCA Solvers / Time vs. N Samples scikit-learn: Kernel PCA Solvers / Time vs. N Components scikit-learn: Sparse Rand Projections / 100 Iterations rbenchmark: numpy: deepspeech: CPU mnn: nasnet mnn: mobilenetV3 mnn: squeezenetv1.1 mnn: resnet-v2-50 mnn: SqueezeNetV1.0 mnn: MobileNetV2_224 mnn: mobilenet-v1-1.0 mnn: inception-v3 deepsparse: NLP Document Classification, oBERT base uncased on IMDB - Asynchronous Multi-Stream deepsparse: NLP Document Classification, oBERT base uncased on IMDB - Asynchronous Multi-Stream deepsparse: NLP Document Classification, oBERT base uncased on IMDB - Synchronous Single-Stream deepsparse: NLP Document Classification, oBERT base uncased on IMDB - Synchronous Single-Stream deepsparse: NLP Text Classification, BERT base uncased SST2, Sparse INT8 - Asynchronous Multi-Stream deepsparse: NLP Text Classification, BERT base uncased SST2, Sparse INT8 - Asynchronous Multi-Stream deepsparse: NLP Text Classification, BERT base uncased SST2, Sparse INT8 - Synchronous Single-Stream deepsparse: NLP Text Classification, BERT base uncased SST2, Sparse INT8 - Synchronous Single-Stream deepsparse: ResNet-50, Baseline - Asynchronous Multi-Stream deepsparse: ResNet-50, Baseline - Asynchronous Multi-Stream deepsparse: ResNet-50, Baseline - Synchronous Single-Stream deepsparse: ResNet-50, Baseline - Synchronous Single-Stream deepsparse: ResNet-50, Sparse INT8 - Asynchronous Multi-Stream deepsparse: ResNet-50, Sparse INT8 - Asynchronous Multi-Stream deepsparse: ResNet-50, Sparse INT8 - Synchronous Single-Stream deepsparse: ResNet-50, Sparse INT8 - Synchronous Single-Stream deepsparse: Llama2 Chat 7b Quantized - Asynchronous Multi-Stream deepsparse: Llama2 Chat 7b Quantized - Asynchronous Multi-Stream deepsparse: Llama2 Chat 7b Quantized - Synchronous Single-Stream deepsparse: Llama2 Chat 7b Quantized - Synchronous Single-Stream deepsparse: CV Classification, ResNet-50 ImageNet - Asynchronous Multi-Stream deepsparse: CV Classification, ResNet-50 ImageNet - Asynchronous Multi-Stream deepsparse: CV Classification, ResNet-50 ImageNet - Synchronous Single-Stream deepsparse: CV Classification, ResNet-50 ImageNet - Synchronous Single-Stream deepsparse: CV Detection, YOLOv5s COCO, Sparse INT8 - Asynchronous Multi-Stream deepsparse: CV Detection, YOLOv5s COCO, Sparse INT8 - Asynchronous Multi-Stream deepsparse: CV Detection, YOLOv5s COCO, Sparse INT8 - Synchronous Single-Stream deepsparse: CV Detection, YOLOv5s COCO, Sparse INT8 - Synchronous Single-Stream deepsparse: NLP Text Classification, DistilBERT mnli - Asynchronous Multi-Stream deepsparse: NLP Text Classification, DistilBERT mnli - Asynchronous Multi-Stream deepsparse: NLP Text Classification, DistilBERT mnli - Synchronous Single-Stream deepsparse: NLP Text Classification, DistilBERT mnli - Synchronous Single-Stream deepsparse: CV Segmentation, 90% Pruned YOLACT Pruned - Asynchronous Multi-Stream deepsparse: CV Segmentation, 90% Pruned YOLACT Pruned - Asynchronous Multi-Stream deepsparse: CV Segmentation, 90% Pruned YOLACT Pruned - Synchronous Single-Stream deepsparse: CV Segmentation, 90% Pruned YOLACT Pruned - Synchronous Single-Stream deepsparse: BERT-Large, NLP Question Answering, Sparse INT8 - Asynchronous Multi-Stream deepsparse: BERT-Large, NLP Question Answering, Sparse INT8 - Asynchronous Multi-Stream deepsparse: BERT-Large, NLP Question Answering, Sparse INT8 - Synchronous Single-Stream deepsparse: BERT-Large, NLP Question Answering, Sparse INT8 - Synchronous Single-Stream deepsparse: NLP Token Classification, BERT base uncased conll2003 - Asynchronous Multi-Stream deepsparse: NLP Token Classification, BERT base uncased conll2003 - Asynchronous Multi-Stream deepsparse: NLP Token Classification, BERT base uncased conll2003 - Synchronous Single-Stream deepsparse: NLP Token Classification, BERT base uncased conll2003 - Synchronous Single-Stream onnx: GPT-2 - CPU - Parallel onnx: GPT-2 - CPU - Standard onnx: T5 Encoder - CPU - Parallel onnx: T5 Encoder - CPU - Standard onnx: bertsquad-12 - CPU - Parallel onnx: bertsquad-12 - CPU - Standard onnx: CaffeNet 12-int8 - CPU - Parallel onnx: CaffeNet 12-int8 - CPU - Standard onnx: fcn-resnet101-11 - CPU - Parallel onnx: fcn-resnet101-11 - CPU - Standard onnx: ArcFace ResNet-100 - CPU - Parallel onnx: ArcFace ResNet-100 - CPU - Standard onnx: ResNet50 v1-12-int8 - CPU - Parallel onnx: ResNet50 v1-12-int8 - CPU - Standard onnx: super-resolution-10 - CPU - Parallel onnx: super-resolution-10 - CPU - Standard onnx: Faster R-CNN R-50-FPN-int8 - CPU - Parallel onnx: Faster R-CNN R-50-FPN-int8 - CPU - Standard opencv: DNN - Deep Neural Network pytorch: CPU - 1 - ResNet-50 pytorch: CPU - 1 - ResNet-152 pytorch: CPU - 16 - ResNet-50 pytorch: CPU - 32 - ResNet-50 pytorch: CPU - 64 - ResNet-50 pytorch: CPU - 16 - ResNet-152 pytorch: CPU - 256 - ResNet-50 pytorch: CPU - 32 - ResNet-152 pytorch: CPU - 512 - ResNet-50 pytorch: CPU - 64 - ResNet-152 pytorch: CPU - 256 - ResNet-152 pytorch: CPU - 512 - ResNet-152 pytorch: CPU - 1 - Efficientnet_v2_l pytorch: CPU - 16 - Efficientnet_v2_l pytorch: CPU - 32 - Efficientnet_v2_l pytorch: CPU - 64 - Efficientnet_v2_l pytorch: CPU - 256 - Efficientnet_v2_l pytorch: CPU - 512 - Efficientnet_v2_l tensorflow-lite: SqueezeNet tensorflow-lite: Inception V4 tensorflow-lite: NASNet Mobile tensorflow-lite: Mobilenet Float tensorflow-lite: Mobilenet Quant tensorflow-lite: Inception ResNet V2 tnn: CPU - DenseNet tnn: CPU - MobileNet v2 tnn: CPU - SqueezeNet v2 tnn: CPU - SqueezeNet v1.1 whisper-cpp: ggml-base.en - 2016 State of the Union whisper-cpp: ggml-small.en - 2016 State of the Union whisper-cpp: ggml-medium.en - 2016 State of the Union xnnpack: FP32MobileNetV2 xnnpack: FP32MobileNetV3Large xnnpack: FP32MobileNetV3Small xnnpack: FP16MobileNetV2 xnnpack: FP16MobileNetV3Large xnnpack: FP16MobileNetV3Small xnnpack: QU8MobileNetV2 xnnpack: QU8MobileNetV3Large xnnpack: QU8MobileNetV3Small caffe: AlexNet - CPU - 100 caffe: AlexNet - CPU - 200 caffe: AlexNet - CPU - 1000 caffe: GoogleNet - CPU - 100 caffe: GoogleNet - CPU - 200 caffe: GoogleNet - CPU - 1000 ncnn: CPU - mobilenet ncnn: CPU-v2-v2 - mobilenet-v2 ncnn: CPU-v3-v3 - mobilenet-v3 ncnn: CPU - shufflenet-v2 ncnn: CPU - mnasnet ncnn: CPU - efficientnet-b0 ncnn: CPU - blazeface ncnn: CPU - googlenet ncnn: CPU - vgg16 ncnn: CPU - resnet18 ncnn: CPU - alexnet ncnn: CPU - resnet50 ncnn: CPU - yolov4-tiny ncnn: CPU - squeezenet_ssd ncnn: CPU - regnety_400m ncnn: CPU - vision_transformer ncnn: CPU - FastestDet ncnn: Vulkan GPU - mobilenet ncnn: Vulkan GPU-v2-v2 - mobilenet-v2 ncnn: Vulkan GPU-v3-v3 - mobilenet-v3 ncnn: Vulkan GPU - shufflenet-v2 ncnn: Vulkan GPU - mnasnet ncnn: Vulkan GPU - efficientnet-b0 ncnn: Vulkan GPU - blazeface ncnn: Vulkan GPU - googlenet ncnn: Vulkan GPU - vgg16 ncnn: Vulkan GPU - resnet18 ncnn: Vulkan GPU - alexnet ncnn: Vulkan GPU - resnet50 ncnn: Vulkan GPU - yolov4-tiny ncnn: Vulkan GPU - squeezenet_ssd ncnn: Vulkan GPU - regnety_400m ncnn: Vulkan GPU - vision_transformer ncnn: Vulkan GPU - FastestDet mlpack: scikit_ica mlpack: scikit_svm onednn: IP Shapes 1D - CPU onednn: IP Shapes 3D - CPU onednn: Convolution Batch Shapes Auto - CPU onednn: Deconvolution Batch shapes_1d - CPU onednn: Deconvolution Batch shapes_3d - CPU onednn: Recurrent Neural Network Training - CPU onednn: Recurrent Neural Network Inference - CPU openvino: Face Detection FP16 - CPU openvino: Face Detection FP16 - CPU openvino: Person Detection FP16 - CPU openvino: Person Detection FP16 - CPU openvino: Person Detection FP32 - CPU openvino: Person Detection FP32 - CPU openvino: Vehicle Detection FP16 - CPU openvino: Vehicle Detection FP16 - CPU openvino: Face Detection FP16-INT8 - CPU openvino: Face Detection FP16-INT8 - CPU openvino: Face Detection Retail FP16 - CPU openvino: Face Detection Retail FP16 - CPU openvino: Road Segmentation ADAS FP16 - CPU openvino: Road Segmentation ADAS FP16 - CPU openvino: Vehicle Detection FP16-INT8 - CPU openvino: Vehicle Detection FP16-INT8 - CPU openvino: Weld Porosity Detection FP16 - CPU openvino: Weld Porosity Detection FP16 - CPU openvino: Face Detection Retail FP16-INT8 - CPU openvino: Face Detection Retail FP16-INT8 - CPU openvino: Road Segmentation ADAS FP16-INT8 - CPU openvino: Road Segmentation ADAS FP16-INT8 - CPU openvino: Machine Translation EN To DE FP16 - CPU openvino: Machine Translation EN To DE FP16 - CPU openvino: Weld Porosity Detection FP16-INT8 - CPU openvino: Weld Porosity Detection FP16-INT8 - CPU openvino: Person Vehicle Bike Detection FP16 - CPU openvino: Person Vehicle Bike Detection FP16 - CPU openvino: Noise Suppression Poconet-Like FP16 - CPU openvino: Noise Suppression Poconet-Like FP16 - CPU openvino: Handwritten English Recognition FP16 - CPU openvino: Handwritten English Recognition FP16 - CPU openvino: Person Re-Identification Retail FP16 - CPU openvino: Person Re-Identification Retail FP16 - CPU openvino: Age Gender Recognition Retail 0013 FP16 - CPU openvino: Age Gender Recognition Retail 0013 FP16 - CPU openvino: Handwritten English Recognition FP16-INT8 - CPU openvino: Handwritten English Recognition FP16-INT8 - CPU openvino: Age Gender Recognition Retail 0013 FP16-INT8 - CPU openvino: Age Gender Recognition Retail 0013 FP16-INT8 - CPU onnx: GPT-2 - CPU - Parallel onnx: GPT-2 - CPU - Standard onnx: T5 Encoder - CPU - Parallel onnx: T5 Encoder - CPU - Standard onnx: bertsquad-12 - CPU - Parallel onnx: bertsquad-12 - CPU - Standard onnx: CaffeNet 12-int8 - CPU - Parallel onnx: CaffeNet 12-int8 - CPU - Standard onnx: fcn-resnet101-11 - CPU - Parallel onnx: fcn-resnet101-11 - CPU - Standard onnx: ArcFace ResNet-100 - CPU - Parallel onnx: ArcFace ResNet-100 - CPU - Standard onnx: ResNet50 v1-12-int8 - CPU - Parallel onnx: ResNet50 v1-12-int8 - CPU - Standard onnx: super-resolution-10 - CPU - Parallel onnx: super-resolution-10 - CPU - Standard onnx: Faster R-CNN R-50-FPN-int8 - CPU - Parallel onnx: Faster R-CNN R-50-FPN-int8 - CPU - Standard Intel ADL-S GT1 - Intel Core i3-12100 8.86 4.62 245.088 20.596 11.696 209.161 47.477 52.860 611.356 728.144 39.661 492.674 74.982 52.219 64.562 105.444 119.971 226.078 51.999 186.554 134.664 241.475 83.873 324.552 45.043 96.579 95.571 446.723 60.214 336.099 199.818 138.505 49.524 233.561 14.585 276.878 319.472 905.082 0.1847 463.17 87.51840 9.072 1.188 2.900 26.312 4.744 2.705 3.373 33.507 4.0749 490.7977 4.3138 231.8064 164.6514 12.1290 145.2740 6.8777 51.6867 38.6775 47.5516 21.0243 369.3583 5.3924 300.1121 3.3220 0.0070 211288.5780 0.0092 108889.8452 51.5732 38.7596 47.4739 21.0587 23.8766 83.7416 23.4544 42.6304 34.4105 58.0973 32.2416 31.0112 5.0039 399.5003 4.8149 207.6729 78.1891 25.5563 64.8531 15.4107 4.1179 485.3757 4.3321 230.8279 50.3149 51.1188 56.0251 57.5920 5.34494 7.91502 209.447 233.254 0.512476 0.742532 11.5402 17.3121 122.008 142.738 38.0422 53.2627 4.43575 5.10110 27185 24.89 11.34 12.57 12.56 12.69 6.33 12.91 6.24 12.76 6.23 6.22 6.17 8.10 4.51 4.55 4.52 4.50 4.48 4322.71 64931.8 12005.2 3574.97 5198.07 60949.0 2446.578 201.109 46.205 166.131 246.28971 753.36065 2280.50283 5545 5484 1302 3444 3396 965 2163 1974 728 44622 90229 445933 103411 206462 1038320 21.50 5.35 3.18 2.25 3.53 8.15 0.62 16.13 109.92 12.34 12.63 26.71 40.12 17.42 6.57 135.49 3.31 21.05 5.29 3.19 2.21 3.19 7.99 0.61 15.83 111.12 12.25 12.54 26.97 39.80 17.49 6.53 136.23 3.30 64.74 14.51 6.00601 26.6998 44.1438 10.7533 10.3718 5708.97 3071.31 1.24 3213.28 10.06 397.47 10.04 398.50 62.61 63.85 5.26 759.24 330.42 12.09 18.24 219.10 214.21 18.66 136.74 29.24 833.34 4.80 63.19 63.27 14.95 267.44 513.04 7.79 166.71 23.99 208.84 19.10 66.40 60.21 201.90 19.80 3481.92 1.14 88.41 45.22 10592.67 0.37 19.8719 19.5609 17.8485 17.3627 187.689 130.814 4.77472 4.28771 1975.36 1353.24 86.6977 57.7900 8.19737 7.00536 26.2946 19.2123 225.465 196.432 OpenBenchmarking.org
PlaidML FP16: No - Mode: Inference - Network: VGG16 - Device: CPU OpenBenchmarking.org FPS, More Is Better PlaidML FP16: No - Mode: Inference - Network: VGG16 - Device: CPU Intel ADL-S GT1 - Intel Core i3-12100 2 4 6 8 10 SE +/- 0.09, N = 3 8.86
PlaidML FP16: No - Mode: Inference - Network: ResNet 50 - Device: CPU OpenBenchmarking.org FPS, More Is Better PlaidML FP16: No - Mode: Inference - Network: ResNet 50 - Device: CPU Intel ADL-S GT1 - Intel Core i3-12100 1.0395 2.079 3.1185 4.158 5.1975 SE +/- 0.01, N = 3 4.62
Numenta Anomaly Benchmark Detector: KNN CAD OpenBenchmarking.org Seconds, Fewer Is Better Numenta Anomaly Benchmark 1.1 Detector: KNN CAD Intel ADL-S GT1 - Intel Core i3-12100 50 100 150 200 250 SE +/- 1.93, N = 3 245.09
Numenta Anomaly Benchmark Detector: Relative Entropy OpenBenchmarking.org Seconds, Fewer Is Better Numenta Anomaly Benchmark 1.1 Detector: Relative Entropy Intel ADL-S GT1 - Intel Core i3-12100 5 10 15 20 25 SE +/- 0.18, N = 8 20.60
Numenta Anomaly Benchmark Detector: Windowed Gaussian OpenBenchmarking.org Seconds, Fewer Is Better Numenta Anomaly Benchmark 1.1 Detector: Windowed Gaussian Intel ADL-S GT1 - Intel Core i3-12100 3 6 9 12 15 SE +/- 0.02, N = 3 11.70
Numenta Anomaly Benchmark Detector: Earthgecko Skyline OpenBenchmarking.org Seconds, Fewer Is Better Numenta Anomaly Benchmark 1.1 Detector: Earthgecko Skyline Intel ADL-S GT1 - Intel Core i3-12100 50 100 150 200 250 SE +/- 1.51, N = 3 209.16
Numenta Anomaly Benchmark Detector: Bayesian Changepoint OpenBenchmarking.org Seconds, Fewer Is Better Numenta Anomaly Benchmark 1.1 Detector: Bayesian Changepoint Intel ADL-S GT1 - Intel Core i3-12100 11 22 33 44 55 SE +/- 0.32, N = 13 47.48
Numenta Anomaly Benchmark Detector: Contextual Anomaly Detector OSE OpenBenchmarking.org Seconds, Fewer Is Better Numenta Anomaly Benchmark 1.1 Detector: Contextual Anomaly Detector OSE Intel ADL-S GT1 - Intel Core i3-12100 12 24 36 48 60 SE +/- 0.27, N = 3 52.86
Scikit-Learn Benchmark: GLM OpenBenchmarking.org Seconds, Fewer Is Better Scikit-Learn 1.2.2 Benchmark: GLM Intel ADL-S GT1 - Intel Core i3-12100 130 260 390 520 650 SE +/- 0.96, N = 3 611.36 1. (F9X) gfortran options: -O0
Scikit-Learn Benchmark: SAGA OpenBenchmarking.org Seconds, Fewer Is Better Scikit-Learn 1.2.2 Benchmark: SAGA Intel ADL-S GT1 - Intel Core i3-12100 160 320 480 640 800 SE +/- 0.42, N = 3 728.14 1. (F9X) gfortran options: -O0
Scikit-Learn Benchmark: Tree OpenBenchmarking.org Seconds, Fewer Is Better Scikit-Learn 1.2.2 Benchmark: Tree Intel ADL-S GT1 - Intel Core i3-12100 9 18 27 36 45 SE +/- 0.30, N = 15 39.66 1. (F9X) gfortran options: -O0
Scikit-Learn Benchmark: Lasso OpenBenchmarking.org Seconds, Fewer Is Better Scikit-Learn 1.2.2 Benchmark: Lasso Intel ADL-S GT1 - Intel Core i3-12100 110 220 330 440 550 SE +/- 1.04, N = 3 492.67 1. (F9X) gfortran options: -O0
Scikit-Learn Benchmark: Sparsify OpenBenchmarking.org Seconds, Fewer Is Better Scikit-Learn 1.2.2 Benchmark: Sparsify Intel ADL-S GT1 - Intel Core i3-12100 20 40 60 80 100 SE +/- 0.27, N = 3 74.98 1. (F9X) gfortran options: -O0
Scikit-Learn Benchmark: Plot Ward OpenBenchmarking.org Seconds, Fewer Is Better Scikit-Learn 1.2.2 Benchmark: Plot Ward Intel ADL-S GT1 - Intel Core i3-12100 12 24 36 48 60 SE +/- 0.10, N = 3 52.22 1. (F9X) gfortran options: -O0
Scikit-Learn Benchmark: MNIST Dataset OpenBenchmarking.org Seconds, Fewer Is Better Scikit-Learn 1.2.2 Benchmark: MNIST Dataset Intel ADL-S GT1 - Intel Core i3-12100 14 28 42 56 70 SE +/- 0.04, N = 3 64.56 1. (F9X) gfortran options: -O0
Scikit-Learn Benchmark: Plot Neighbors OpenBenchmarking.org Seconds, Fewer Is Better Scikit-Learn 1.2.2 Benchmark: Plot Neighbors Intel ADL-S GT1 - Intel Core i3-12100 20 40 60 80 100 SE +/- 0.15, N = 3 105.44 1. (F9X) gfortran options: -O0
Scikit-Learn Benchmark: SGD Regression OpenBenchmarking.org Seconds, Fewer Is Better Scikit-Learn 1.2.2 Benchmark: SGD Regression Intel ADL-S GT1 - Intel Core i3-12100 30 60 90 120 150 SE +/- 0.56, N = 3 119.97 1. (F9X) gfortran options: -O0
Scikit-Learn Benchmark: Plot Lasso Path OpenBenchmarking.org Seconds, Fewer Is Better Scikit-Learn 1.2.2 Benchmark: Plot Lasso Path Intel ADL-S GT1 - Intel Core i3-12100 50 100 150 200 250 SE +/- 1.40, N = 3 226.08 1. (F9X) gfortran options: -O0
Scikit-Learn Benchmark: Text Vectorizers OpenBenchmarking.org Seconds, Fewer Is Better Scikit-Learn 1.2.2 Benchmark: Text Vectorizers Intel ADL-S GT1 - Intel Core i3-12100 12 24 36 48 60 SE +/- 0.05, N = 3 52.00 1. (F9X) gfortran options: -O0
Scikit-Learn Benchmark: Plot Hierarchical OpenBenchmarking.org Seconds, Fewer Is Better Scikit-Learn 1.2.2 Benchmark: Plot Hierarchical Intel ADL-S GT1 - Intel Core i3-12100 40 80 120 160 200 SE +/- 0.37, N = 3 186.55 1. (F9X) gfortran options: -O0
Scikit-Learn Benchmark: Plot OMP vs. LARS OpenBenchmarking.org Seconds, Fewer Is Better Scikit-Learn 1.2.2 Benchmark: Plot OMP vs. LARS Intel ADL-S GT1 - Intel Core i3-12100 30 60 90 120 150 SE +/- 0.10, N = 3 134.66 1. (F9X) gfortran options: -O0
Scikit-Learn Benchmark: Feature Expansions OpenBenchmarking.org Seconds, Fewer Is Better Scikit-Learn 1.2.2 Benchmark: Feature Expansions Intel ADL-S GT1 - Intel Core i3-12100 50 100 150 200 250 SE +/- 0.42, N = 3 241.48 1. (F9X) gfortran options: -O0
Scikit-Learn Benchmark: LocalOutlierFactor OpenBenchmarking.org Seconds, Fewer Is Better Scikit-Learn 1.2.2 Benchmark: LocalOutlierFactor Intel ADL-S GT1 - Intel Core i3-12100 20 40 60 80 100 SE +/- 0.08, N = 3 83.87 1. (F9X) gfortran options: -O0
Scikit-Learn Benchmark: TSNE MNIST Dataset OpenBenchmarking.org Seconds, Fewer Is Better Scikit-Learn 1.2.2 Benchmark: TSNE MNIST Dataset Intel ADL-S GT1 - Intel Core i3-12100 70 140 210 280 350 SE +/- 0.43, N = 3 324.55 1. (F9X) gfortran options: -O0
Scikit-Learn Benchmark: Plot Incremental PCA OpenBenchmarking.org Seconds, Fewer Is Better Scikit-Learn 1.2.2 Benchmark: Plot Incremental PCA Intel ADL-S GT1 - Intel Core i3-12100 10 20 30 40 50 SE +/- 0.33, N = 3 45.04 1. (F9X) gfortran options: -O0
Scikit-Learn Benchmark: Hist Gradient Boosting OpenBenchmarking.org Seconds, Fewer Is Better Scikit-Learn 1.2.2 Benchmark: Hist Gradient Boosting Intel ADL-S GT1 - Intel Core i3-12100 20 40 60 80 100 SE +/- 0.18, N = 3 96.58 1. (F9X) gfortran options: -O0
Scikit-Learn Benchmark: Sample Without Replacement OpenBenchmarking.org Seconds, Fewer Is Better Scikit-Learn 1.2.2 Benchmark: Sample Without Replacement Intel ADL-S GT1 - Intel Core i3-12100 20 40 60 80 100 SE +/- 0.11, N = 3 95.57 1. (F9X) gfortran options: -O0
Scikit-Learn Benchmark: Covertype Dataset Benchmark OpenBenchmarking.org Seconds, Fewer Is Better Scikit-Learn 1.2.2 Benchmark: Covertype Dataset Benchmark Intel ADL-S GT1 - Intel Core i3-12100 100 200 300 400 500 SE +/- 0.14, N = 3 446.72 1. (F9X) gfortran options: -O0
Scikit-Learn Benchmark: Hist Gradient Boosting Adult OpenBenchmarking.org Seconds, Fewer Is Better Scikit-Learn 1.2.2 Benchmark: Hist Gradient Boosting Adult Intel ADL-S GT1 - Intel Core i3-12100 13 26 39 52 65 SE +/- 0.09, N = 3 60.21 1. (F9X) gfortran options: -O0
Scikit-Learn Benchmark: Hist Gradient Boosting Threading OpenBenchmarking.org Seconds, Fewer Is Better Scikit-Learn 1.2.2 Benchmark: Hist Gradient Boosting Threading Intel ADL-S GT1 - Intel Core i3-12100 70 140 210 280 350 SE +/- 3.75, N = 3 336.10 1. (F9X) gfortran options: -O0
Scikit-Learn Benchmark: Plot Singular Value Decomposition OpenBenchmarking.org Seconds, Fewer Is Better Scikit-Learn 1.2.2 Benchmark: Plot Singular Value Decomposition Intel ADL-S GT1 - Intel Core i3-12100 40 80 120 160 200 SE +/- 0.02, N = 3 199.82 1. (F9X) gfortran options: -O0
Scikit-Learn Benchmark: Hist Gradient Boosting Higgs Boson OpenBenchmarking.org Seconds, Fewer Is Better Scikit-Learn 1.2.2 Benchmark: Hist Gradient Boosting Higgs Boson Intel ADL-S GT1 - Intel Core i3-12100 30 60 90 120 150 SE +/- 1.02, N = 12 138.51 1. (F9X) gfortran options: -O0
Scikit-Learn Benchmark: 20 Newsgroups / Logistic Regression OpenBenchmarking.org Seconds, Fewer Is Better Scikit-Learn 1.2.2 Benchmark: 20 Newsgroups / Logistic Regression Intel ADL-S GT1 - Intel Core i3-12100 11 22 33 44 55 SE +/- 0.02, N = 3 49.52 1. (F9X) gfortran options: -O0
Scikit-Learn Benchmark: Plot Polynomial Kernel Approximation OpenBenchmarking.org Seconds, Fewer Is Better Scikit-Learn 1.2.2 Benchmark: Plot Polynomial Kernel Approximation Intel ADL-S GT1 - Intel Core i3-12100 50 100 150 200 250 SE +/- 1.21, N = 3 233.56 1. (F9X) gfortran options: -O0
Scikit-Learn Benchmark: Hist Gradient Boosting Categorical Only OpenBenchmarking.org Seconds, Fewer Is Better Scikit-Learn 1.2.2 Benchmark: Hist Gradient Boosting Categorical Only Intel ADL-S GT1 - Intel Core i3-12100 4 8 12 16 20 SE +/- 0.10, N = 3 14.59 1. (F9X) gfortran options: -O0
Scikit-Learn Benchmark: Kernel PCA Solvers / Time vs. N Samples OpenBenchmarking.org Seconds, Fewer Is Better Scikit-Learn 1.2.2 Benchmark: Kernel PCA Solvers / Time vs. N Samples Intel ADL-S GT1 - Intel Core i3-12100 60 120 180 240 300 SE +/- 0.57, N = 3 276.88 1. (F9X) gfortran options: -O0
Scikit-Learn Benchmark: Kernel PCA Solvers / Time vs. N Components OpenBenchmarking.org Seconds, Fewer Is Better Scikit-Learn 1.2.2 Benchmark: Kernel PCA Solvers / Time vs. N Components Intel ADL-S GT1 - Intel Core i3-12100 70 140 210 280 350 SE +/- 14.09, N = 9 319.47 1. (F9X) gfortran options: -O0
Scikit-Learn Benchmark: Sparse Random Projections / 100 Iterations OpenBenchmarking.org Seconds, Fewer Is Better Scikit-Learn 1.2.2 Benchmark: Sparse Random Projections / 100 Iterations Intel ADL-S GT1 - Intel Core i3-12100 200 400 600 800 1000 SE +/- 0.87, N = 3 905.08 1. (F9X) gfortran options: -O0
R Benchmark OpenBenchmarking.org Seconds, Fewer Is Better R Benchmark Intel ADL-S GT1 - Intel Core i3-12100 0.0416 0.0832 0.1248 0.1664 0.208 SE +/- 0.0015, N = 3 0.1847 1. R scripting front-end version 3.6.3 (2020-02-29)
Numpy Benchmark OpenBenchmarking.org Score, More Is Better Numpy Benchmark Intel ADL-S GT1 - Intel Core i3-12100 100 200 300 400 500 SE +/- 0.39, N = 3 463.17
DeepSpeech Acceleration: CPU OpenBenchmarking.org Seconds, Fewer Is Better DeepSpeech 0.6 Acceleration: CPU Intel ADL-S GT1 - Intel Core i3-12100 20 40 60 80 100 SE +/- 0.60, N = 3 87.52
Mobile Neural Network Model: nasnet OpenBenchmarking.org ms, Fewer Is Better Mobile Neural Network 2.9.b11b7037d Model: nasnet Intel ADL-S GT1 - Intel Core i3-12100 3 6 9 12 15 SE +/- 0.022, N = 3 9.072 MIN: 8.93 / MAX: 29.56 1. (CXX) g++ options: -std=c++11 -O3 -fvisibility=hidden -fomit-frame-pointer -fstrict-aliasing -ffunction-sections -fdata-sections -ffast-math -fno-rtti -fno-exceptions -pthread -ldl
Mobile Neural Network Model: mobilenetV3 OpenBenchmarking.org ms, Fewer Is Better Mobile Neural Network 2.9.b11b7037d Model: mobilenetV3 Intel ADL-S GT1 - Intel Core i3-12100 0.2673 0.5346 0.8019 1.0692 1.3365 SE +/- 0.047, N = 3 1.188 MIN: 1.12 / MAX: 49.7 1. (CXX) g++ options: -std=c++11 -O3 -fvisibility=hidden -fomit-frame-pointer -fstrict-aliasing -ffunction-sections -fdata-sections -ffast-math -fno-rtti -fno-exceptions -pthread -ldl
Mobile Neural Network Model: squeezenetv1.1 OpenBenchmarking.org ms, Fewer Is Better Mobile Neural Network 2.9.b11b7037d Model: squeezenetv1.1 Intel ADL-S GT1 - Intel Core i3-12100 0.6525 1.305 1.9575 2.61 3.2625 SE +/- 0.015, N = 3 2.900 MIN: 2.83 / MAX: 42.33 1. (CXX) g++ options: -std=c++11 -O3 -fvisibility=hidden -fomit-frame-pointer -fstrict-aliasing -ffunction-sections -fdata-sections -ffast-math -fno-rtti -fno-exceptions -pthread -ldl
Mobile Neural Network Model: resnet-v2-50 OpenBenchmarking.org ms, Fewer Is Better Mobile Neural Network 2.9.b11b7037d Model: resnet-v2-50 Intel ADL-S GT1 - Intel Core i3-12100 6 12 18 24 30 SE +/- 0.04, N = 3 26.31 MIN: 25.83 / MAX: 99.47 1. (CXX) g++ options: -std=c++11 -O3 -fvisibility=hidden -fomit-frame-pointer -fstrict-aliasing -ffunction-sections -fdata-sections -ffast-math -fno-rtti -fno-exceptions -pthread -ldl
Mobile Neural Network Model: SqueezeNetV1.0 OpenBenchmarking.org ms, Fewer Is Better Mobile Neural Network 2.9.b11b7037d Model: SqueezeNetV1.0 Intel ADL-S GT1 - Intel Core i3-12100 1.0674 2.1348 3.2022 4.2696 5.337 SE +/- 0.009, N = 3 4.744 MIN: 4.68 / MAX: 20.9 1. (CXX) g++ options: -std=c++11 -O3 -fvisibility=hidden -fomit-frame-pointer -fstrict-aliasing -ffunction-sections -fdata-sections -ffast-math -fno-rtti -fno-exceptions -pthread -ldl
Mobile Neural Network Model: MobileNetV2_224 OpenBenchmarking.org ms, Fewer Is Better Mobile Neural Network 2.9.b11b7037d Model: MobileNetV2_224 Intel ADL-S GT1 - Intel Core i3-12100 0.6086 1.2172 1.8258 2.4344 3.043 SE +/- 0.007, N = 3 2.705 MIN: 2.65 / MAX: 4.63 1. (CXX) g++ options: -std=c++11 -O3 -fvisibility=hidden -fomit-frame-pointer -fstrict-aliasing -ffunction-sections -fdata-sections -ffast-math -fno-rtti -fno-exceptions -pthread -ldl
Mobile Neural Network Model: mobilenet-v1-1.0 OpenBenchmarking.org ms, Fewer Is Better Mobile Neural Network 2.9.b11b7037d Model: mobilenet-v1-1.0 Intel ADL-S GT1 - Intel Core i3-12100 0.7589 1.5178 2.2767 3.0356 3.7945 SE +/- 0.048, N = 3 3.373 MIN: 3.24 / MAX: 29.19 1. (CXX) g++ options: -std=c++11 -O3 -fvisibility=hidden -fomit-frame-pointer -fstrict-aliasing -ffunction-sections -fdata-sections -ffast-math -fno-rtti -fno-exceptions -pthread -ldl
Mobile Neural Network Model: inception-v3 OpenBenchmarking.org ms, Fewer Is Better Mobile Neural Network 2.9.b11b7037d Model: inception-v3 Intel ADL-S GT1 - Intel Core i3-12100 8 16 24 32 40 SE +/- 0.07, N = 3 33.51 MIN: 32.99 / MAX: 109 1. (CXX) g++ options: -std=c++11 -O3 -fvisibility=hidden -fomit-frame-pointer -fstrict-aliasing -ffunction-sections -fdata-sections -ffast-math -fno-rtti -fno-exceptions -pthread -ldl
Neural Magic DeepSparse Model: NLP Document Classification, oBERT base uncased on IMDB - Scenario: Asynchronous Multi-Stream OpenBenchmarking.org items/sec, More Is Better Neural Magic DeepSparse 1.7 Model: NLP Document Classification, oBERT base uncased on IMDB - Scenario: Asynchronous Multi-Stream Intel ADL-S GT1 - Intel Core i3-12100 0.9169 1.8338 2.7507 3.6676 4.5845 SE +/- 0.0124, N = 3 4.0749
Neural Magic DeepSparse Model: NLP Document Classification, oBERT base uncased on IMDB - Scenario: Asynchronous Multi-Stream OpenBenchmarking.org ms/batch, Fewer Is Better Neural Magic DeepSparse 1.7 Model: NLP Document Classification, oBERT base uncased on IMDB - Scenario: Asynchronous Multi-Stream Intel ADL-S GT1 - Intel Core i3-12100 110 220 330 440 550 SE +/- 1.50, N = 3 490.80
Neural Magic DeepSparse Model: NLP Document Classification, oBERT base uncased on IMDB - Scenario: Synchronous Single-Stream OpenBenchmarking.org items/sec, More Is Better Neural Magic DeepSparse 1.7 Model: NLP Document Classification, oBERT base uncased on IMDB - Scenario: Synchronous Single-Stream Intel ADL-S GT1 - Intel Core i3-12100 0.9706 1.9412 2.9118 3.8824 4.853 SE +/- 0.0039, N = 3 4.3138
Neural Magic DeepSparse Model: NLP Document Classification, oBERT base uncased on IMDB - Scenario: Synchronous Single-Stream OpenBenchmarking.org ms/batch, Fewer Is Better Neural Magic DeepSparse 1.7 Model: NLP Document Classification, oBERT base uncased on IMDB - Scenario: Synchronous Single-Stream Intel ADL-S GT1 - Intel Core i3-12100 50 100 150 200 250 SE +/- 0.21, N = 3 231.81
Neural Magic DeepSparse Model: NLP Text Classification, BERT base uncased SST2, Sparse INT8 - Scenario: Asynchronous Multi-Stream OpenBenchmarking.org items/sec, More Is Better Neural Magic DeepSparse 1.7 Model: NLP Text Classification, BERT base uncased SST2, Sparse INT8 - Scenario: Asynchronous Multi-Stream Intel ADL-S GT1 - Intel Core i3-12100 40 80 120 160 200 SE +/- 0.07, N = 3 164.65
Neural Magic DeepSparse Model: NLP Text Classification, BERT base uncased SST2, Sparse INT8 - Scenario: Asynchronous Multi-Stream OpenBenchmarking.org ms/batch, Fewer Is Better Neural Magic DeepSparse 1.7 Model: NLP Text Classification, BERT base uncased SST2, Sparse INT8 - Scenario: Asynchronous Multi-Stream Intel ADL-S GT1 - Intel Core i3-12100 3 6 9 12 15 SE +/- 0.00, N = 3 12.13
Neural Magic DeepSparse Model: NLP Text Classification, BERT base uncased SST2, Sparse INT8 - Scenario: Synchronous Single-Stream OpenBenchmarking.org items/sec, More Is Better Neural Magic DeepSparse 1.7 Model: NLP Text Classification, BERT base uncased SST2, Sparse INT8 - Scenario: Synchronous Single-Stream Intel ADL-S GT1 - Intel Core i3-12100 30 60 90 120 150 SE +/- 0.34, N = 3 145.27
Neural Magic DeepSparse Model: NLP Text Classification, BERT base uncased SST2, Sparse INT8 - Scenario: Synchronous Single-Stream OpenBenchmarking.org ms/batch, Fewer Is Better Neural Magic DeepSparse 1.7 Model: NLP Text Classification, BERT base uncased SST2, Sparse INT8 - Scenario: Synchronous Single-Stream Intel ADL-S GT1 - Intel Core i3-12100 2 4 6 8 10 SE +/- 0.0159, N = 3 6.8777
Neural Magic DeepSparse Model: ResNet-50, Baseline - Scenario: Asynchronous Multi-Stream OpenBenchmarking.org items/sec, More Is Better Neural Magic DeepSparse 1.7 Model: ResNet-50, Baseline - Scenario: Asynchronous Multi-Stream Intel ADL-S GT1 - Intel Core i3-12100 12 24 36 48 60 SE +/- 0.19, N = 3 51.69
Neural Magic DeepSparse Model: ResNet-50, Baseline - Scenario: Asynchronous Multi-Stream OpenBenchmarking.org ms/batch, Fewer Is Better Neural Magic DeepSparse 1.7 Model: ResNet-50, Baseline - Scenario: Asynchronous Multi-Stream Intel ADL-S GT1 - Intel Core i3-12100 9 18 27 36 45 SE +/- 0.14, N = 3 38.68
Neural Magic DeepSparse Model: ResNet-50, Baseline - Scenario: Synchronous Single-Stream OpenBenchmarking.org items/sec, More Is Better Neural Magic DeepSparse 1.7 Model: ResNet-50, Baseline - Scenario: Synchronous Single-Stream Intel ADL-S GT1 - Intel Core i3-12100 11 22 33 44 55 SE +/- 0.05, N = 3 47.55
Neural Magic DeepSparse Model: ResNet-50, Baseline - Scenario: Synchronous Single-Stream OpenBenchmarking.org ms/batch, Fewer Is Better Neural Magic DeepSparse 1.7 Model: ResNet-50, Baseline - Scenario: Synchronous Single-Stream Intel ADL-S GT1 - Intel Core i3-12100 5 10 15 20 25 SE +/- 0.02, N = 3 21.02
Neural Magic DeepSparse Model: ResNet-50, Sparse INT8 - Scenario: Asynchronous Multi-Stream OpenBenchmarking.org items/sec, More Is Better Neural Magic DeepSparse 1.7 Model: ResNet-50, Sparse INT8 - Scenario: Asynchronous Multi-Stream Intel ADL-S GT1 - Intel Core i3-12100 80 160 240 320 400 SE +/- 1.14, N = 3 369.36
Neural Magic DeepSparse Model: ResNet-50, Sparse INT8 - Scenario: Asynchronous Multi-Stream OpenBenchmarking.org ms/batch, Fewer Is Better Neural Magic DeepSparse 1.7 Model: ResNet-50, Sparse INT8 - Scenario: Asynchronous Multi-Stream Intel ADL-S GT1 - Intel Core i3-12100 1.2133 2.4266 3.6399 4.8532 6.0665 SE +/- 0.0167, N = 3 5.3924
Neural Magic DeepSparse Model: ResNet-50, Sparse INT8 - Scenario: Synchronous Single-Stream OpenBenchmarking.org items/sec, More Is Better Neural Magic DeepSparse 1.7 Model: ResNet-50, Sparse INT8 - Scenario: Synchronous Single-Stream Intel ADL-S GT1 - Intel Core i3-12100 70 140 210 280 350 SE +/- 1.17, N = 3 300.11
Neural Magic DeepSparse Model: ResNet-50, Sparse INT8 - Scenario: Synchronous Single-Stream OpenBenchmarking.org ms/batch, Fewer Is Better Neural Magic DeepSparse 1.7 Model: ResNet-50, Sparse INT8 - Scenario: Synchronous Single-Stream Intel ADL-S GT1 - Intel Core i3-12100 0.7475 1.495 2.2425 2.99 3.7375 SE +/- 0.0131, N = 3 3.3220
Neural Magic DeepSparse Model: Llama2 Chat 7b Quantized - Scenario: Asynchronous Multi-Stream OpenBenchmarking.org items/sec, More Is Better Neural Magic DeepSparse 1.7 Model: Llama2 Chat 7b Quantized - Scenario: Asynchronous Multi-Stream Intel ADL-S GT1 - Intel Core i3-12100 0.0016 0.0032 0.0048 0.0064 0.008 SE +/- 0.0001, N = 9 0.0070
Neural Magic DeepSparse Model: Llama2 Chat 7b Quantized - Scenario: Asynchronous Multi-Stream OpenBenchmarking.org ms/batch, Fewer Is Better Neural Magic DeepSparse 1.7 Model: Llama2 Chat 7b Quantized - Scenario: Asynchronous Multi-Stream Intel ADL-S GT1 - Intel Core i3-12100 50K 100K 150K 200K 250K SE +/- 2251.32, N = 9 211288.58
Neural Magic DeepSparse Model: Llama2 Chat 7b Quantized - Scenario: Synchronous Single-Stream OpenBenchmarking.org items/sec, More Is Better Neural Magic DeepSparse 1.7 Model: Llama2 Chat 7b Quantized - Scenario: Synchronous Single-Stream Intel ADL-S GT1 - Intel Core i3-12100 0.0021 0.0042 0.0063 0.0084 0.0105 SE +/- 0.0001, N = 9 0.0092
Neural Magic DeepSparse Model: Llama2 Chat 7b Quantized - Scenario: Synchronous Single-Stream OpenBenchmarking.org ms/batch, Fewer Is Better Neural Magic DeepSparse 1.7 Model: Llama2 Chat 7b Quantized - Scenario: Synchronous Single-Stream Intel ADL-S GT1 - Intel Core i3-12100 20K 40K 60K 80K 100K SE +/- 1162.79, N = 9 108889.85
Neural Magic DeepSparse Model: CV Classification, ResNet-50 ImageNet - Scenario: Asynchronous Multi-Stream OpenBenchmarking.org items/sec, More Is Better Neural Magic DeepSparse 1.7 Model: CV Classification, ResNet-50 ImageNet - Scenario: Asynchronous Multi-Stream Intel ADL-S GT1 - Intel Core i3-12100 12 24 36 48 60 SE +/- 0.15, N = 3 51.57
Neural Magic DeepSparse Model: CV Classification, ResNet-50 ImageNet - Scenario: Asynchronous Multi-Stream OpenBenchmarking.org ms/batch, Fewer Is Better Neural Magic DeepSparse 1.7 Model: CV Classification, ResNet-50 ImageNet - Scenario: Asynchronous Multi-Stream Intel ADL-S GT1 - Intel Core i3-12100 9 18 27 36 45 SE +/- 0.11, N = 3 38.76
Neural Magic DeepSparse Model: CV Classification, ResNet-50 ImageNet - Scenario: Synchronous Single-Stream OpenBenchmarking.org items/sec, More Is Better Neural Magic DeepSparse 1.7 Model: CV Classification, ResNet-50 ImageNet - Scenario: Synchronous Single-Stream Intel ADL-S GT1 - Intel Core i3-12100 11 22 33 44 55 SE +/- 0.04, N = 3 47.47
Neural Magic DeepSparse Model: CV Classification, ResNet-50 ImageNet - Scenario: Synchronous Single-Stream OpenBenchmarking.org ms/batch, Fewer Is Better Neural Magic DeepSparse 1.7 Model: CV Classification, ResNet-50 ImageNet - Scenario: Synchronous Single-Stream Intel ADL-S GT1 - Intel Core i3-12100 5 10 15 20 25 SE +/- 0.02, N = 3 21.06
Neural Magic DeepSparse Model: CV Detection, YOLOv5s COCO, Sparse INT8 - Scenario: Asynchronous Multi-Stream OpenBenchmarking.org items/sec, More Is Better Neural Magic DeepSparse 1.7 Model: CV Detection, YOLOv5s COCO, Sparse INT8 - Scenario: Asynchronous Multi-Stream Intel ADL-S GT1 - Intel Core i3-12100 6 12 18 24 30 SE +/- 0.03, N = 3 23.88
Neural Magic DeepSparse Model: CV Detection, YOLOv5s COCO, Sparse INT8 - Scenario: Asynchronous Multi-Stream OpenBenchmarking.org ms/batch, Fewer Is Better Neural Magic DeepSparse 1.7 Model: CV Detection, YOLOv5s COCO, Sparse INT8 - Scenario: Asynchronous Multi-Stream Intel ADL-S GT1 - Intel Core i3-12100 20 40 60 80 100 SE +/- 0.10, N = 3 83.74
Neural Magic DeepSparse Model: CV Detection, YOLOv5s COCO, Sparse INT8 - Scenario: Synchronous Single-Stream OpenBenchmarking.org items/sec, More Is Better Neural Magic DeepSparse 1.7 Model: CV Detection, YOLOv5s COCO, Sparse INT8 - Scenario: Synchronous Single-Stream Intel ADL-S GT1 - Intel Core i3-12100 6 12 18 24 30 SE +/- 0.01, N = 3 23.45
Neural Magic DeepSparse Model: CV Detection, YOLOv5s COCO, Sparse INT8 - Scenario: Synchronous Single-Stream OpenBenchmarking.org ms/batch, Fewer Is Better Neural Magic DeepSparse 1.7 Model: CV Detection, YOLOv5s COCO, Sparse INT8 - Scenario: Synchronous Single-Stream Intel ADL-S GT1 - Intel Core i3-12100 10 20 30 40 50 SE +/- 0.02, N = 3 42.63
Neural Magic DeepSparse Model: NLP Text Classification, DistilBERT mnli - Scenario: Asynchronous Multi-Stream OpenBenchmarking.org items/sec, More Is Better Neural Magic DeepSparse 1.7 Model: NLP Text Classification, DistilBERT mnli - Scenario: Asynchronous Multi-Stream Intel ADL-S GT1 - Intel Core i3-12100 8 16 24 32 40 SE +/- 0.12, N = 3 34.41
Neural Magic DeepSparse Model: NLP Text Classification, DistilBERT mnli - Scenario: Asynchronous Multi-Stream OpenBenchmarking.org ms/batch, Fewer Is Better Neural Magic DeepSparse 1.7 Model: NLP Text Classification, DistilBERT mnli - Scenario: Asynchronous Multi-Stream Intel ADL-S GT1 - Intel Core i3-12100 13 26 39 52 65 SE +/- 0.19, N = 3 58.10
Neural Magic DeepSparse Model: NLP Text Classification, DistilBERT mnli - Scenario: Synchronous Single-Stream OpenBenchmarking.org items/sec, More Is Better Neural Magic DeepSparse 1.7 Model: NLP Text Classification, DistilBERT mnli - Scenario: Synchronous Single-Stream Intel ADL-S GT1 - Intel Core i3-12100 7 14 21 28 35 SE +/- 0.05, N = 3 32.24
Neural Magic DeepSparse Model: NLP Text Classification, DistilBERT mnli - Scenario: Synchronous Single-Stream OpenBenchmarking.org ms/batch, Fewer Is Better Neural Magic DeepSparse 1.7 Model: NLP Text Classification, DistilBERT mnli - Scenario: Synchronous Single-Stream Intel ADL-S GT1 - Intel Core i3-12100 7 14 21 28 35 SE +/- 0.04, N = 3 31.01
Neural Magic DeepSparse Model: CV Segmentation, 90% Pruned YOLACT Pruned - Scenario: Asynchronous Multi-Stream OpenBenchmarking.org items/sec, More Is Better Neural Magic DeepSparse 1.7 Model: CV Segmentation, 90% Pruned YOLACT Pruned - Scenario: Asynchronous Multi-Stream Intel ADL-S GT1 - Intel Core i3-12100 1.1259 2.2518 3.3777 4.5036 5.6295 SE +/- 0.0404, N = 3 5.0039
Neural Magic DeepSparse Model: CV Segmentation, 90% Pruned YOLACT Pruned - Scenario: Asynchronous Multi-Stream OpenBenchmarking.org ms/batch, Fewer Is Better Neural Magic DeepSparse 1.7 Model: CV Segmentation, 90% Pruned YOLACT Pruned - Scenario: Asynchronous Multi-Stream Intel ADL-S GT1 - Intel Core i3-12100 90 180 270 360 450 SE +/- 3.23, N = 3 399.50
Neural Magic DeepSparse Model: CV Segmentation, 90% Pruned YOLACT Pruned - Scenario: Synchronous Single-Stream OpenBenchmarking.org items/sec, More Is Better Neural Magic DeepSparse 1.7 Model: CV Segmentation, 90% Pruned YOLACT Pruned - Scenario: Synchronous Single-Stream Intel ADL-S GT1 - Intel Core i3-12100 1.0834 2.1668 3.2502 4.3336 5.417 SE +/- 0.0104, N = 3 4.8149
Neural Magic DeepSparse Model: CV Segmentation, 90% Pruned YOLACT Pruned - Scenario: Synchronous Single-Stream OpenBenchmarking.org ms/batch, Fewer Is Better Neural Magic DeepSparse 1.7 Model: CV Segmentation, 90% Pruned YOLACT Pruned - Scenario: Synchronous Single-Stream Intel ADL-S GT1 - Intel Core i3-12100 50 100 150 200 250 SE +/- 0.45, N = 3 207.67
Neural Magic DeepSparse Model: BERT-Large, NLP Question Answering, Sparse INT8 - Scenario: Asynchronous Multi-Stream OpenBenchmarking.org items/sec, More Is Better Neural Magic DeepSparse 1.7 Model: BERT-Large, NLP Question Answering, Sparse INT8 - Scenario: Asynchronous Multi-Stream Intel ADL-S GT1 - Intel Core i3-12100 20 40 60 80 100 SE +/- 0.07, N = 3 78.19
Neural Magic DeepSparse Model: BERT-Large, NLP Question Answering, Sparse INT8 - Scenario: Asynchronous Multi-Stream OpenBenchmarking.org ms/batch, Fewer Is Better Neural Magic DeepSparse 1.7 Model: BERT-Large, NLP Question Answering, Sparse INT8 - Scenario: Asynchronous Multi-Stream Intel ADL-S GT1 - Intel Core i3-12100 6 12 18 24 30 SE +/- 0.02, N = 3 25.56
Neural Magic DeepSparse Model: BERT-Large, NLP Question Answering, Sparse INT8 - Scenario: Synchronous Single-Stream OpenBenchmarking.org items/sec, More Is Better Neural Magic DeepSparse 1.7 Model: BERT-Large, NLP Question Answering, Sparse INT8 - Scenario: Synchronous Single-Stream Intel ADL-S GT1 - Intel Core i3-12100 14 28 42 56 70 SE +/- 0.06, N = 3 64.85
Neural Magic DeepSparse Model: BERT-Large, NLP Question Answering, Sparse INT8 - Scenario: Synchronous Single-Stream OpenBenchmarking.org ms/batch, Fewer Is Better Neural Magic DeepSparse 1.7 Model: BERT-Large, NLP Question Answering, Sparse INT8 - Scenario: Synchronous Single-Stream Intel ADL-S GT1 - Intel Core i3-12100 4 8 12 16 20 SE +/- 0.01, N = 3 15.41
Neural Magic DeepSparse Model: NLP Token Classification, BERT base uncased conll2003 - Scenario: Asynchronous Multi-Stream OpenBenchmarking.org items/sec, More Is Better Neural Magic DeepSparse 1.7 Model: NLP Token Classification, BERT base uncased conll2003 - Scenario: Asynchronous Multi-Stream Intel ADL-S GT1 - Intel Core i3-12100 0.9265 1.853 2.7795 3.706 4.6325 SE +/- 0.0219, N = 3 4.1179
Neural Magic DeepSparse Model: NLP Token Classification, BERT base uncased conll2003 - Scenario: Asynchronous Multi-Stream OpenBenchmarking.org ms/batch, Fewer Is Better Neural Magic DeepSparse 1.7 Model: NLP Token Classification, BERT base uncased conll2003 - Scenario: Asynchronous Multi-Stream Intel ADL-S GT1 - Intel Core i3-12100 110 220 330 440 550 SE +/- 2.32, N = 3 485.38
Neural Magic DeepSparse Model: NLP Token Classification, BERT base uncased conll2003 - Scenario: Synchronous Single-Stream OpenBenchmarking.org items/sec, More Is Better Neural Magic DeepSparse 1.7 Model: NLP Token Classification, BERT base uncased conll2003 - Scenario: Synchronous Single-Stream Intel ADL-S GT1 - Intel Core i3-12100 0.9747 1.9494 2.9241 3.8988 4.8735 SE +/- 0.0059, N = 3 4.3321
Neural Magic DeepSparse Model: NLP Token Classification, BERT base uncased conll2003 - Scenario: Synchronous Single-Stream OpenBenchmarking.org ms/batch, Fewer Is Better Neural Magic DeepSparse 1.7 Model: NLP Token Classification, BERT base uncased conll2003 - Scenario: Synchronous Single-Stream Intel ADL-S GT1 - Intel Core i3-12100 50 100 150 200 250 SE +/- 0.31, N = 3 230.83
ONNX Runtime Model: GPT-2 - Device: CPU - Executor: Parallel OpenBenchmarking.org Inferences Per Second, More Is Better ONNX Runtime 1.17 Model: GPT-2 - Device: CPU - Executor: Parallel Intel ADL-S GT1 - Intel Core i3-12100 11 22 33 44 55 SE +/- 0.08, N = 3 50.31 1. (CXX) g++ options: -O3 -march=native -ffunction-sections -fdata-sections -mtune=native -flto -fno-fat-lto-objects -ldl -lrt -lpthread -pthread
ONNX Runtime Model: GPT-2 - Device: CPU - Executor: Standard OpenBenchmarking.org Inferences Per Second, More Is Better ONNX Runtime 1.17 Model: GPT-2 - Device: CPU - Executor: Standard Intel ADL-S GT1 - Intel Core i3-12100 12 24 36 48 60 SE +/- 0.51, N = 3 51.12 1. (CXX) g++ options: -O3 -march=native -ffunction-sections -fdata-sections -mtune=native -flto -fno-fat-lto-objects -ldl -lrt -lpthread -pthread
ONNX Runtime Model: T5 Encoder - Device: CPU - Executor: Parallel OpenBenchmarking.org Inferences Per Second, More Is Better ONNX Runtime 1.17 Model: T5 Encoder - Device: CPU - Executor: Parallel Intel ADL-S GT1 - Intel Core i3-12100 13 26 39 52 65 SE +/- 0.04, N = 3 56.03 1. (CXX) g++ options: -O3 -march=native -ffunction-sections -fdata-sections -mtune=native -flto -fno-fat-lto-objects -ldl -lrt -lpthread -pthread
ONNX Runtime Model: T5 Encoder - Device: CPU - Executor: Standard OpenBenchmarking.org Inferences Per Second, More Is Better ONNX Runtime 1.17 Model: T5 Encoder - Device: CPU - Executor: Standard Intel ADL-S GT1 - Intel Core i3-12100 13 26 39 52 65 SE +/- 0.02, N = 3 57.59 1. (CXX) g++ options: -O3 -march=native -ffunction-sections -fdata-sections -mtune=native -flto -fno-fat-lto-objects -ldl -lrt -lpthread -pthread
ONNX Runtime Model: bertsquad-12 - Device: CPU - Executor: Parallel OpenBenchmarking.org Inferences Per Second, More Is Better ONNX Runtime 1.17 Model: bertsquad-12 - Device: CPU - Executor: Parallel Intel ADL-S GT1 - Intel Core i3-12100 1.2026 2.4052 3.6078 4.8104 6.013 SE +/- 0.07908, N = 15 5.34494 1. (CXX) g++ options: -O3 -march=native -ffunction-sections -fdata-sections -mtune=native -flto -fno-fat-lto-objects -ldl -lrt -lpthread -pthread
ONNX Runtime Model: bertsquad-12 - Device: CPU - Executor: Standard OpenBenchmarking.org Inferences Per Second, More Is Better ONNX Runtime 1.17 Model: bertsquad-12 - Device: CPU - Executor: Standard Intel ADL-S GT1 - Intel Core i3-12100 2 4 6 8 10 SE +/- 0.33732, N = 14 7.91502 1. (CXX) g++ options: -O3 -march=native -ffunction-sections -fdata-sections -mtune=native -flto -fno-fat-lto-objects -ldl -lrt -lpthread -pthread
ONNX Runtime Model: CaffeNet 12-int8 - Device: CPU - Executor: Parallel OpenBenchmarking.org Inferences Per Second, More Is Better ONNX Runtime 1.17 Model: CaffeNet 12-int8 - Device: CPU - Executor: Parallel Intel ADL-S GT1 - Intel Core i3-12100 50 100 150 200 250 SE +/- 2.29, N = 3 209.45 1. (CXX) g++ options: -O3 -march=native -ffunction-sections -fdata-sections -mtune=native -flto -fno-fat-lto-objects -ldl -lrt -lpthread -pthread
ONNX Runtime Model: CaffeNet 12-int8 - Device: CPU - Executor: Standard OpenBenchmarking.org Inferences Per Second, More Is Better ONNX Runtime 1.17 Model: CaffeNet 12-int8 - Device: CPU - Executor: Standard Intel ADL-S GT1 - Intel Core i3-12100 50 100 150 200 250 SE +/- 2.63, N = 3 233.25 1. (CXX) g++ options: -O3 -march=native -ffunction-sections -fdata-sections -mtune=native -flto -fno-fat-lto-objects -ldl -lrt -lpthread -pthread
ONNX Runtime Model: fcn-resnet101-11 - Device: CPU - Executor: Parallel OpenBenchmarking.org Inferences Per Second, More Is Better ONNX Runtime 1.17 Model: fcn-resnet101-11 - Device: CPU - Executor: Parallel Intel ADL-S GT1 - Intel Core i3-12100 0.1153 0.2306 0.3459 0.4612 0.5765 SE +/- 0.015328, N = 15 0.512476 1. (CXX) g++ options: -O3 -march=native -ffunction-sections -fdata-sections -mtune=native -flto -fno-fat-lto-objects -ldl -lrt -lpthread -pthread
ONNX Runtime Model: fcn-resnet101-11 - Device: CPU - Executor: Standard OpenBenchmarking.org Inferences Per Second, More Is Better ONNX Runtime 1.17 Model: fcn-resnet101-11 - Device: CPU - Executor: Standard Intel ADL-S GT1 - Intel Core i3-12100 0.1671 0.3342 0.5013 0.6684 0.8355 SE +/- 0.012881, N = 15 0.742532 1. (CXX) g++ options: -O3 -march=native -ffunction-sections -fdata-sections -mtune=native -flto -fno-fat-lto-objects -ldl -lrt -lpthread -pthread
ONNX Runtime Model: ArcFace ResNet-100 - Device: CPU - Executor: Parallel OpenBenchmarking.org Inferences Per Second, More Is Better ONNX Runtime 1.17 Model: ArcFace ResNet-100 - Device: CPU - Executor: Parallel Intel ADL-S GT1 - Intel Core i3-12100 3 6 9 12 15 SE +/- 0.12, N = 6 11.54 1. (CXX) g++ options: -O3 -march=native -ffunction-sections -fdata-sections -mtune=native -flto -fno-fat-lto-objects -ldl -lrt -lpthread -pthread
ONNX Runtime Model: ArcFace ResNet-100 - Device: CPU - Executor: Standard OpenBenchmarking.org Inferences Per Second, More Is Better ONNX Runtime 1.17 Model: ArcFace ResNet-100 - Device: CPU - Executor: Standard Intel ADL-S GT1 - Intel Core i3-12100 4 8 12 16 20 SE +/- 0.19, N = 5 17.31 1. (CXX) g++ options: -O3 -march=native -ffunction-sections -fdata-sections -mtune=native -flto -fno-fat-lto-objects -ldl -lrt -lpthread -pthread
ONNX Runtime Model: ResNet50 v1-12-int8 - Device: CPU - Executor: Parallel OpenBenchmarking.org Inferences Per Second, More Is Better ONNX Runtime 1.17 Model: ResNet50 v1-12-int8 - Device: CPU - Executor: Parallel Intel ADL-S GT1 - Intel Core i3-12100 30 60 90 120 150 SE +/- 1.34, N = 3 122.01 1. (CXX) g++ options: -O3 -march=native -ffunction-sections -fdata-sections -mtune=native -flto -fno-fat-lto-objects -ldl -lrt -lpthread -pthread
ONNX Runtime Model: ResNet50 v1-12-int8 - Device: CPU - Executor: Standard OpenBenchmarking.org Inferences Per Second, More Is Better ONNX Runtime 1.17 Model: ResNet50 v1-12-int8 - Device: CPU - Executor: Standard Intel ADL-S GT1 - Intel Core i3-12100 30 60 90 120 150 SE +/- 0.84, N = 3 142.74 1. (CXX) g++ options: -O3 -march=native -ffunction-sections -fdata-sections -mtune=native -flto -fno-fat-lto-objects -ldl -lrt -lpthread -pthread
ONNX Runtime Model: super-resolution-10 - Device: CPU - Executor: Parallel OpenBenchmarking.org Inferences Per Second, More Is Better ONNX Runtime 1.17 Model: super-resolution-10 - Device: CPU - Executor: Parallel Intel ADL-S GT1 - Intel Core i3-12100 9 18 27 36 45 SE +/- 0.42, N = 4 38.04 1. (CXX) g++ options: -O3 -march=native -ffunction-sections -fdata-sections -mtune=native -flto -fno-fat-lto-objects -ldl -lrt -lpthread -pthread
ONNX Runtime Model: super-resolution-10 - Device: CPU - Executor: Standard OpenBenchmarking.org Inferences Per Second, More Is Better ONNX Runtime 1.17 Model: super-resolution-10 - Device: CPU - Executor: Standard Intel ADL-S GT1 - Intel Core i3-12100 12 24 36 48 60 SE +/- 1.87, N = 15 53.26 1. (CXX) g++ options: -O3 -march=native -ffunction-sections -fdata-sections -mtune=native -flto -fno-fat-lto-objects -ldl -lrt -lpthread -pthread
ONNX Runtime Model: Faster R-CNN R-50-FPN-int8 - Device: CPU - Executor: Parallel OpenBenchmarking.org Inferences Per Second, More Is Better ONNX Runtime 1.17 Model: Faster R-CNN R-50-FPN-int8 - Device: CPU - Executor: Parallel Intel ADL-S GT1 - Intel Core i3-12100 0.998 1.996 2.994 3.992 4.99 SE +/- 0.03335, N = 3 4.43575 1. (CXX) g++ options: -O3 -march=native -ffunction-sections -fdata-sections -mtune=native -flto -fno-fat-lto-objects -ldl -lrt -lpthread -pthread
ONNX Runtime Model: Faster R-CNN R-50-FPN-int8 - Device: CPU - Executor: Standard OpenBenchmarking.org Inferences Per Second, More Is Better ONNX Runtime 1.17 Model: Faster R-CNN R-50-FPN-int8 - Device: CPU - Executor: Standard Intel ADL-S GT1 - Intel Core i3-12100 1.1477 2.2954 3.4431 4.5908 5.7385 SE +/- 0.06535, N = 12 5.10110 1. (CXX) g++ options: -O3 -march=native -ffunction-sections -fdata-sections -mtune=native -flto -fno-fat-lto-objects -ldl -lrt -lpthread -pthread
OpenCV Test: DNN - Deep Neural Network OpenBenchmarking.org ms, Fewer Is Better OpenCV 4.7 Test: DNN - Deep Neural Network Intel ADL-S GT1 - Intel Core i3-12100 6K 12K 18K 24K 30K SE +/- 832.87, N = 12 27185 1. (CXX) g++ options: -fPIC -fsigned-char -pthread -fomit-frame-pointer -ffunction-sections -fdata-sections -msse -msse2 -msse3 -fvisibility=hidden -O3 -shared
PyTorch Device: CPU - Batch Size: 1 - Model: ResNet-50 OpenBenchmarking.org batches/sec, More Is Better PyTorch 2.2.1 Device: CPU - Batch Size: 1 - Model: ResNet-50 Intel ADL-S GT1 - Intel Core i3-12100 6 12 18 24 30 SE +/- 0.26, N = 4 24.89 MIN: 11.71 / MAX: 25.9
PyTorch Device: CPU - Batch Size: 1 - Model: ResNet-152 OpenBenchmarking.org batches/sec, More Is Better PyTorch 2.2.1 Device: CPU - Batch Size: 1 - Model: ResNet-152 Intel ADL-S GT1 - Intel Core i3-12100 3 6 9 12 15 SE +/- 0.03, N = 3 11.34 MIN: 7.58 / MAX: 11.54
PyTorch Device: CPU - Batch Size: 16 - Model: ResNet-50 OpenBenchmarking.org batches/sec, More Is Better PyTorch 2.2.1 Device: CPU - Batch Size: 16 - Model: ResNet-50 Intel ADL-S GT1 - Intel Core i3-12100 3 6 9 12 15 SE +/- 0.11, N = 3 12.57 MIN: 8.58 / MAX: 12.97
PyTorch Device: CPU - Batch Size: 32 - Model: ResNet-50 OpenBenchmarking.org batches/sec, More Is Better PyTorch 2.2.1 Device: CPU - Batch Size: 32 - Model: ResNet-50 Intel ADL-S GT1 - Intel Core i3-12100 3 6 9 12 15 SE +/- 0.11, N = 8 12.56 MIN: 7.8 / MAX: 13.33
PyTorch Device: CPU - Batch Size: 64 - Model: ResNet-50 OpenBenchmarking.org batches/sec, More Is Better PyTorch 2.2.1 Device: CPU - Batch Size: 64 - Model: ResNet-50 Intel ADL-S GT1 - Intel Core i3-12100 3 6 9 12 15 SE +/- 0.12, N = 3 12.69 MIN: 8.56 / MAX: 13.22
PyTorch Device: CPU - Batch Size: 16 - Model: ResNet-152 OpenBenchmarking.org batches/sec, More Is Better PyTorch 2.2.1 Device: CPU - Batch Size: 16 - Model: ResNet-152 Intel ADL-S GT1 - Intel Core i3-12100 2 4 6 8 10 SE +/- 0.04, N = 3 6.33 MIN: 4.35 / MAX: 6.45
PyTorch Device: CPU - Batch Size: 256 - Model: ResNet-50 OpenBenchmarking.org batches/sec, More Is Better PyTorch 2.2.1 Device: CPU - Batch Size: 256 - Model: ResNet-50 Intel ADL-S GT1 - Intel Core i3-12100 3 6 9 12 15 SE +/- 0.09, N = 3 12.91 MIN: 8.58 / MAX: 13.25
PyTorch Device: CPU - Batch Size: 32 - Model: ResNet-152 OpenBenchmarking.org batches/sec, More Is Better PyTorch 2.2.1 Device: CPU - Batch Size: 32 - Model: ResNet-152 Intel ADL-S GT1 - Intel Core i3-12100 2 4 6 8 10 SE +/- 0.03, N = 3 6.24 MIN: 4.36 / MAX: 6.39
PyTorch Device: CPU - Batch Size: 512 - Model: ResNet-50 OpenBenchmarking.org batches/sec, More Is Better PyTorch 2.2.1 Device: CPU - Batch Size: 512 - Model: ResNet-50 Intel ADL-S GT1 - Intel Core i3-12100 3 6 9 12 15 SE +/- 0.18, N = 3 12.76 MIN: 7.94 / MAX: 13.36
PyTorch Device: CPU - Batch Size: 64 - Model: ResNet-152 OpenBenchmarking.org batches/sec, More Is Better PyTorch 2.2.1 Device: CPU - Batch Size: 64 - Model: ResNet-152 Intel ADL-S GT1 - Intel Core i3-12100 2 4 6 8 10 SE +/- 0.03, N = 3 6.23 MIN: 3.95 / MAX: 6.36
PyTorch Device: CPU - Batch Size: 256 - Model: ResNet-152 OpenBenchmarking.org batches/sec, More Is Better PyTorch 2.2.1 Device: CPU - Batch Size: 256 - Model: ResNet-152 Intel ADL-S GT1 - Intel Core i3-12100 2 4 6 8 10 SE +/- 0.02, N = 3 6.22 MIN: 4.3 / MAX: 6.36
PyTorch Device: CPU - Batch Size: 512 - Model: ResNet-152 OpenBenchmarking.org batches/sec, More Is Better PyTorch 2.2.1 Device: CPU - Batch Size: 512 - Model: ResNet-152 Intel ADL-S GT1 - Intel Core i3-12100 2 4 6 8 10 SE +/- 0.03, N = 3 6.17 MIN: 4.17 / MAX: 6.34
PyTorch Device: CPU - Batch Size: 1 - Model: Efficientnet_v2_l OpenBenchmarking.org batches/sec, More Is Better PyTorch 2.2.1 Device: CPU - Batch Size: 1 - Model: Efficientnet_v2_l Intel ADL-S GT1 - Intel Core i3-12100 2 4 6 8 10 SE +/- 0.08, N = 5 8.10 MIN: 5.47 / MAX: 8.32
PyTorch Device: CPU - Batch Size: 16 - Model: Efficientnet_v2_l OpenBenchmarking.org batches/sec, More Is Better PyTorch 2.2.1 Device: CPU - Batch Size: 16 - Model: Efficientnet_v2_l Intel ADL-S GT1 - Intel Core i3-12100 1.0148 2.0296 3.0444 4.0592 5.074 SE +/- 0.04, N = 3 4.51 MIN: 3.16 / MAX: 4.61
PyTorch Device: CPU - Batch Size: 32 - Model: Efficientnet_v2_l OpenBenchmarking.org batches/sec, More Is Better PyTorch 2.2.1 Device: CPU - Batch Size: 32 - Model: Efficientnet_v2_l Intel ADL-S GT1 - Intel Core i3-12100 1.0238 2.0476 3.0714 4.0952 5.119 SE +/- 0.01, N = 3 4.55 MIN: 3.32 / MAX: 4.61
PyTorch Device: CPU - Batch Size: 64 - Model: Efficientnet_v2_l OpenBenchmarking.org batches/sec, More Is Better PyTorch 2.2.1 Device: CPU - Batch Size: 64 - Model: Efficientnet_v2_l Intel ADL-S GT1 - Intel Core i3-12100 1.017 2.034 3.051 4.068 5.085 SE +/- 0.01, N = 3 4.52 MIN: 3.17 / MAX: 4.6
PyTorch Device: CPU - Batch Size: 256 - Model: Efficientnet_v2_l OpenBenchmarking.org batches/sec, More Is Better PyTorch 2.2.1 Device: CPU - Batch Size: 256 - Model: Efficientnet_v2_l Intel ADL-S GT1 - Intel Core i3-12100 1.0125 2.025 3.0375 4.05 5.0625 SE +/- 0.02, N = 3 4.50 MIN: 3.35 / MAX: 4.57
PyTorch Device: CPU - Batch Size: 512 - Model: Efficientnet_v2_l OpenBenchmarking.org batches/sec, More Is Better PyTorch 2.2.1 Device: CPU - Batch Size: 512 - Model: Efficientnet_v2_l Intel ADL-S GT1 - Intel Core i3-12100 1.008 2.016 3.024 4.032 5.04 SE +/- 0.04, N = 3 4.48 MIN: 3.37 / MAX: 4.59
TensorFlow Lite Model: SqueezeNet OpenBenchmarking.org Microseconds, Fewer Is Better TensorFlow Lite 2022-05-18 Model: SqueezeNet Intel ADL-S GT1 - Intel Core i3-12100 900 1800 2700 3600 4500 SE +/- 12.96, N = 3 4322.71
TensorFlow Lite Model: Inception V4 OpenBenchmarking.org Microseconds, Fewer Is Better TensorFlow Lite 2022-05-18 Model: Inception V4 Intel ADL-S GT1 - Intel Core i3-12100 14K 28K 42K 56K 70K SE +/- 110.99, N = 3 64931.8
TensorFlow Lite Model: NASNet Mobile OpenBenchmarking.org Microseconds, Fewer Is Better TensorFlow Lite 2022-05-18 Model: NASNet Mobile Intel ADL-S GT1 - Intel Core i3-12100 3K 6K 9K 12K 15K SE +/- 3.59, N = 3 12005.2
TensorFlow Lite Model: Mobilenet Float OpenBenchmarking.org Microseconds, Fewer Is Better TensorFlow Lite 2022-05-18 Model: Mobilenet Float Intel ADL-S GT1 - Intel Core i3-12100 800 1600 2400 3200 4000 SE +/- 12.76, N = 3 3574.97
TensorFlow Lite Model: Mobilenet Quant OpenBenchmarking.org Microseconds, Fewer Is Better TensorFlow Lite 2022-05-18 Model: Mobilenet Quant Intel ADL-S GT1 - Intel Core i3-12100 1100 2200 3300 4400 5500 SE +/- 9.03, N = 3 5198.07
TensorFlow Lite Model: Inception ResNet V2 OpenBenchmarking.org Microseconds, Fewer Is Better TensorFlow Lite 2022-05-18 Model: Inception ResNet V2 Intel ADL-S GT1 - Intel Core i3-12100 13K 26K 39K 52K 65K SE +/- 49.41, N = 3 60949.0
TNN Target: CPU - Model: DenseNet OpenBenchmarking.org ms, Fewer Is Better TNN 0.3 Target: CPU - Model: DenseNet Intel ADL-S GT1 - Intel Core i3-12100 500 1000 1500 2000 2500 SE +/- 1.75, N = 3 2446.58 MIN: 2392.91 / MAX: 2547.47 1. (CXX) g++ options: -fopenmp -pthread -fvisibility=hidden -fvisibility=default -O3 -rdynamic -ldl
TNN Target: CPU - Model: MobileNet v2 OpenBenchmarking.org ms, Fewer Is Better TNN 0.3 Target: CPU - Model: MobileNet v2 Intel ADL-S GT1 - Intel Core i3-12100 40 80 120 160 200 SE +/- 0.11, N = 3 201.11 MIN: 198.48 / MAX: 216.2 1. (CXX) g++ options: -fopenmp -pthread -fvisibility=hidden -fvisibility=default -O3 -rdynamic -ldl
TNN Target: CPU - Model: SqueezeNet v2 OpenBenchmarking.org ms, Fewer Is Better TNN 0.3 Target: CPU - Model: SqueezeNet v2 Intel ADL-S GT1 - Intel Core i3-12100 10 20 30 40 50 SE +/- 0.07, N = 3 46.21 MIN: 45.29 / MAX: 48.72 1. (CXX) g++ options: -fopenmp -pthread -fvisibility=hidden -fvisibility=default -O3 -rdynamic -ldl
TNN Target: CPU - Model: SqueezeNet v1.1 OpenBenchmarking.org ms, Fewer Is Better TNN 0.3 Target: CPU - Model: SqueezeNet v1.1 Intel ADL-S GT1 - Intel Core i3-12100 40 80 120 160 200 SE +/- 0.09, N = 3 166.13 MIN: 162.86 / MAX: 170.17 1. (CXX) g++ options: -fopenmp -pthread -fvisibility=hidden -fvisibility=default -O3 -rdynamic -ldl
Whisper.cpp Model: ggml-base.en - Input: 2016 State of the Union OpenBenchmarking.org Seconds, Fewer Is Better Whisper.cpp 1.6.2 Model: ggml-base.en - Input: 2016 State of the Union Intel ADL-S GT1 - Intel Core i3-12100 50 100 150 200 250 SE +/- 0.15, N = 3 246.29 1. (CXX) g++ options: -O3 -std=c++11 -fPIC -pthread -msse3 -mssse3 -mavx -mf16c -mfma -mavx2
Whisper.cpp Model: ggml-small.en - Input: 2016 State of the Union OpenBenchmarking.org Seconds, Fewer Is Better Whisper.cpp 1.6.2 Model: ggml-small.en - Input: 2016 State of the Union Intel ADL-S GT1 - Intel Core i3-12100 160 320 480 640 800 SE +/- 0.45, N = 3 753.36 1. (CXX) g++ options: -O3 -std=c++11 -fPIC -pthread -msse3 -mssse3 -mavx -mf16c -mfma -mavx2
Whisper.cpp Model: ggml-medium.en - Input: 2016 State of the Union OpenBenchmarking.org Seconds, Fewer Is Better Whisper.cpp 1.6.2 Model: ggml-medium.en - Input: 2016 State of the Union Intel ADL-S GT1 - Intel Core i3-12100 500 1000 1500 2000 2500 SE +/- 3.55, N = 3 2280.50 1. (CXX) g++ options: -O3 -std=c++11 -fPIC -pthread -msse3 -mssse3 -mavx -mf16c -mfma -mavx2
XNNPACK Model: FP32MobileNetV2 OpenBenchmarking.org us, Fewer Is Better XNNPACK 2cd86b Model: FP32MobileNetV2 Intel ADL-S GT1 - Intel Core i3-12100 1200 2400 3600 4800 6000 SE +/- 15.31, N = 3 5545 1. (CXX) g++ options: -O3 -lrt -pthread -lpthread -lm
XNNPACK Model: FP32MobileNetV3Large OpenBenchmarking.org us, Fewer Is Better XNNPACK 2cd86b Model: FP32MobileNetV3Large Intel ADL-S GT1 - Intel Core i3-12100 1200 2400 3600 4800 6000 SE +/- 28.00, N = 3 5484 1. (CXX) g++ options: -O3 -lrt -pthread -lpthread -lm
XNNPACK Model: FP32MobileNetV3Small OpenBenchmarking.org us, Fewer Is Better XNNPACK 2cd86b Model: FP32MobileNetV3Small Intel ADL-S GT1 - Intel Core i3-12100 300 600 900 1200 1500 SE +/- 14.19, N = 3 1302 1. (CXX) g++ options: -O3 -lrt -pthread -lpthread -lm
XNNPACK Model: FP16MobileNetV2 OpenBenchmarking.org us, Fewer Is Better XNNPACK 2cd86b Model: FP16MobileNetV2 Intel ADL-S GT1 - Intel Core i3-12100 700 1400 2100 2800 3500 3444 1. (CXX) g++ options: -O3 -lrt -pthread -lpthread -lm
XNNPACK Model: FP16MobileNetV3Large OpenBenchmarking.org us, Fewer Is Better XNNPACK 2cd86b Model: FP16MobileNetV3Large Intel ADL-S GT1 - Intel Core i3-12100 700 1400 2100 2800 3500 SE +/- 40.00, N = 3 3396 1. (CXX) g++ options: -O3 -lrt -pthread -lpthread -lm
XNNPACK Model: FP16MobileNetV3Small OpenBenchmarking.org us, Fewer Is Better XNNPACK 2cd86b Model: FP16MobileNetV3Small Intel ADL-S GT1 - Intel Core i3-12100 200 400 600 800 1000 SE +/- 5.84, N = 3 965 1. (CXX) g++ options: -O3 -lrt -pthread -lpthread -lm
XNNPACK Model: QU8MobileNetV2 OpenBenchmarking.org us, Fewer Is Better XNNPACK 2cd86b Model: QU8MobileNetV2 Intel ADL-S GT1 - Intel Core i3-12100 500 1000 1500 2000 2500 SE +/- 21.07, N = 3 2163 1. (CXX) g++ options: -O3 -lrt -pthread -lpthread -lm
XNNPACK Model: QU8MobileNetV3Large OpenBenchmarking.org us, Fewer Is Better XNNPACK 2cd86b Model: QU8MobileNetV3Large Intel ADL-S GT1 - Intel Core i3-12100 400 800 1200 1600 2000 SE +/- 20.30, N = 3 1974 1. (CXX) g++ options: -O3 -lrt -pthread -lpthread -lm
XNNPACK Model: QU8MobileNetV3Small OpenBenchmarking.org us, Fewer Is Better XNNPACK 2cd86b Model: QU8MobileNetV3Small Intel ADL-S GT1 - Intel Core i3-12100 160 320 480 640 800 SE +/- 0.67, N = 3 728 1. (CXX) g++ options: -O3 -lrt -pthread -lpthread -lm
Caffe Model: AlexNet - Acceleration: CPU - Iterations: 100 OpenBenchmarking.org Milli-Seconds, Fewer Is Better Caffe 2020-02-13 Model: AlexNet - Acceleration: CPU - Iterations: 100 Intel ADL-S GT1 - Intel Core i3-12100 10K 20K 30K 40K 50K SE +/- 405.55, N = 3 44622 1. (CXX) g++ options: -fPIC -O3 -rdynamic -lglog -lgflags -lprotobuf -lpthread -lsz -lz -ldl -lm -llmdb -lopenblas
Caffe Model: AlexNet - Acceleration: CPU - Iterations: 200 OpenBenchmarking.org Milli-Seconds, Fewer Is Better Caffe 2020-02-13 Model: AlexNet - Acceleration: CPU - Iterations: 200 Intel ADL-S GT1 - Intel Core i3-12100 20K 40K 60K 80K 100K SE +/- 881.92, N = 3 90229 1. (CXX) g++ options: -fPIC -O3 -rdynamic -lglog -lgflags -lprotobuf -lpthread -lsz -lz -ldl -lm -llmdb -lopenblas
Caffe Model: AlexNet - Acceleration: CPU - Iterations: 1000 OpenBenchmarking.org Milli-Seconds, Fewer Is Better Caffe 2020-02-13 Model: AlexNet - Acceleration: CPU - Iterations: 1000 Intel ADL-S GT1 - Intel Core i3-12100 100K 200K 300K 400K 500K SE +/- 4664.71, N = 3 445933 1. (CXX) g++ options: -fPIC -O3 -rdynamic -lglog -lgflags -lprotobuf -lpthread -lsz -lz -ldl -lm -llmdb -lopenblas
Caffe Model: GoogleNet - Acceleration: CPU - Iterations: 100 OpenBenchmarking.org Milli-Seconds, Fewer Is Better Caffe 2020-02-13 Model: GoogleNet - Acceleration: CPU - Iterations: 100 Intel ADL-S GT1 - Intel Core i3-12100 20K 40K 60K 80K 100K SE +/- 148.50, N = 3 103411 1. (CXX) g++ options: -fPIC -O3 -rdynamic -lglog -lgflags -lprotobuf -lpthread -lsz -lz -ldl -lm -llmdb -lopenblas
Caffe Model: GoogleNet - Acceleration: CPU - Iterations: 200 OpenBenchmarking.org Milli-Seconds, Fewer Is Better Caffe 2020-02-13 Model: GoogleNet - Acceleration: CPU - Iterations: 200 Intel ADL-S GT1 - Intel Core i3-12100 40K 80K 120K 160K 200K SE +/- 44.14, N = 3 206462 1. (CXX) g++ options: -fPIC -O3 -rdynamic -lglog -lgflags -lprotobuf -lpthread -lsz -lz -ldl -lm -llmdb -lopenblas
Caffe Model: GoogleNet - Acceleration: CPU - Iterations: 1000 OpenBenchmarking.org Milli-Seconds, Fewer Is Better Caffe 2020-02-13 Model: GoogleNet - Acceleration: CPU - Iterations: 1000 Intel ADL-S GT1 - Intel Core i3-12100 200K 400K 600K 800K 1000K SE +/- 3248.77, N = 3 1038320 1. (CXX) g++ options: -fPIC -O3 -rdynamic -lglog -lgflags -lprotobuf -lpthread -lsz -lz -ldl -lm -llmdb -lopenblas
NCNN Target: CPU - Model: mobilenet OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: CPU - Model: mobilenet Intel ADL-S GT1 - Intel Core i3-12100 5 10 15 20 25 SE +/- 0.26, N = 4 21.50 MIN: 20.87 / MAX: 110.78 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread -pthread
NCNN Target: CPU-v2-v2 - Model: mobilenet-v2 OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: CPU-v2-v2 - Model: mobilenet-v2 Intel ADL-S GT1 - Intel Core i3-12100 1.2038 2.4076 3.6114 4.8152 6.019 SE +/- 0.04, N = 4 5.35 MIN: 5.1 / MAX: 16.42 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread -pthread
NCNN Target: CPU-v3-v3 - Model: mobilenet-v3 OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: CPU-v3-v3 - Model: mobilenet-v3 Intel ADL-S GT1 - Intel Core i3-12100 0.7155 1.431 2.1465 2.862 3.5775 SE +/- 0.02, N = 4 3.18 MIN: 3.1 / MAX: 5.1 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread -pthread
NCNN Target: CPU - Model: shufflenet-v2 OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: CPU - Model: shufflenet-v2 Intel ADL-S GT1 - Intel Core i3-12100 0.5063 1.0126 1.5189 2.0252 2.5315 SE +/- 0.03, N = 4 2.25 MIN: 2.16 / MAX: 18.02 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread -pthread
NCNN Target: CPU - Model: mnasnet OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: CPU - Model: mnasnet Intel ADL-S GT1 - Intel Core i3-12100 0.7943 1.5886 2.3829 3.1772 3.9715 SE +/- 0.25, N = 4 3.53 MIN: 3.13 / MAX: 39.25 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread -pthread
NCNN Target: CPU - Model: efficientnet-b0 OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: CPU - Model: efficientnet-b0 Intel ADL-S GT1 - Intel Core i3-12100 2 4 6 8 10 SE +/- 0.06, N = 4 8.15 MIN: 7.89 / MAX: 41.7 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread -pthread
NCNN Target: CPU - Model: blazeface OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: CPU - Model: blazeface Intel ADL-S GT1 - Intel Core i3-12100 0.1395 0.279 0.4185 0.558 0.6975 SE +/- 0.00, N = 4 0.62 MIN: 0.59 / MAX: 0.68 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread -pthread
NCNN Target: CPU - Model: googlenet OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: CPU - Model: googlenet Intel ADL-S GT1 - Intel Core i3-12100 4 8 12 16 20 SE +/- 0.28, N = 4 16.13 MIN: 15.65 / MAX: 75.15 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread -pthread
NCNN Target: CPU - Model: vgg16 OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: CPU - Model: vgg16 Intel ADL-S GT1 - Intel Core i3-12100 20 40 60 80 100 SE +/- 0.09, N = 4 109.92 MIN: 108.76 / MAX: 150.98 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread -pthread
NCNN Target: CPU - Model: resnet18 OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: CPU - Model: resnet18 Intel ADL-S GT1 - Intel Core i3-12100 3 6 9 12 15 SE +/- 0.02, N = 4 12.34 MIN: 12.22 / MAX: 14.13 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread -pthread
NCNN Target: CPU - Model: alexnet OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: CPU - Model: alexnet Intel ADL-S GT1 - Intel Core i3-12100 3 6 9 12 15 SE +/- 0.02, N = 4 12.63 MIN: 12.45 / MAX: 50.58 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread -pthread
NCNN Target: CPU - Model: resnet50 OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: CPU - Model: resnet50 Intel ADL-S GT1 - Intel Core i3-12100 6 12 18 24 30 SE +/- 0.11, N = 4 26.71 MIN: 26.31 / MAX: 69.35 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread -pthread
NCNN Target: CPU - Model: yolov4-tiny OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: CPU - Model: yolov4-tiny Intel ADL-S GT1 - Intel Core i3-12100 9 18 27 36 45 SE +/- 0.06, N = 4 40.12 MIN: 39.72 / MAX: 83.43 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread -pthread
NCNN Target: CPU - Model: squeezenet_ssd OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: CPU - Model: squeezenet_ssd Intel ADL-S GT1 - Intel Core i3-12100 4 8 12 16 20 SE +/- 0.08, N = 4 17.42 MIN: 17.17 / MAX: 33.61 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread -pthread
NCNN Target: CPU - Model: regnety_400m OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: CPU - Model: regnety_400m Intel ADL-S GT1 - Intel Core i3-12100 2 4 6 8 10 SE +/- 0.03, N = 4 6.57 MIN: 6.47 / MAX: 15.95 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread -pthread
NCNN Target: CPU - Model: vision_transformer OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: CPU - Model: vision_transformer Intel ADL-S GT1 - Intel Core i3-12100 30 60 90 120 150 SE +/- 0.38, N = 4 135.49 MIN: 134.44 / MAX: 253.67 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread -pthread
NCNN Target: CPU - Model: FastestDet OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: CPU - Model: FastestDet Intel ADL-S GT1 - Intel Core i3-12100 0.7448 1.4896 2.2344 2.9792 3.724 SE +/- 0.05, N = 4 3.31 MIN: 3.17 / MAX: 7.41 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread -pthread
NCNN Target: Vulkan GPU - Model: mobilenet OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: Vulkan GPU - Model: mobilenet Intel ADL-S GT1 - Intel Core i3-12100 5 10 15 20 25 SE +/- 0.03, N = 3 21.05 MIN: 20.81 / MAX: 24.47 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread -pthread
NCNN Target: Vulkan GPU-v2-v2 - Model: mobilenet-v2 OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: Vulkan GPU-v2-v2 - Model: mobilenet-v2 Intel ADL-S GT1 - Intel Core i3-12100 1.1903 2.3806 3.5709 4.7612 5.9515 SE +/- 0.02, N = 3 5.29 MIN: 5.08 / MAX: 7.09 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread -pthread
NCNN Target: Vulkan GPU-v3-v3 - Model: mobilenet-v3 OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: Vulkan GPU-v3-v3 - Model: mobilenet-v3 Intel ADL-S GT1 - Intel Core i3-12100 0.7178 1.4356 2.1534 2.8712 3.589 SE +/- 0.00, N = 3 3.19 MIN: 3.11 / MAX: 4.76 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread -pthread
NCNN Target: Vulkan GPU - Model: shufflenet-v2 OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: Vulkan GPU - Model: shufflenet-v2 Intel ADL-S GT1 - Intel Core i3-12100 0.4973 0.9946 1.4919 1.9892 2.4865 SE +/- 0.01, N = 3 2.21 MIN: 2.16 / MAX: 3.84 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread -pthread
NCNN Target: Vulkan GPU - Model: mnasnet OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: Vulkan GPU - Model: mnasnet Intel ADL-S GT1 - Intel Core i3-12100 0.7178 1.4356 2.1534 2.8712 3.589 SE +/- 0.02, N = 2 3.19 MIN: 3.12 / MAX: 4.45 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread -pthread
NCNN Target: Vulkan GPU - Model: efficientnet-b0 OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: Vulkan GPU - Model: efficientnet-b0 Intel ADL-S GT1 - Intel Core i3-12100 2 4 6 8 10 SE +/- 0.02, N = 3 7.99 MIN: 7.85 / MAX: 10.37 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread -pthread
NCNN Target: Vulkan GPU - Model: blazeface OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: Vulkan GPU - Model: blazeface Intel ADL-S GT1 - Intel Core i3-12100 0.1373 0.2746 0.4119 0.5492 0.6865 SE +/- 0.00, N = 3 0.61 MIN: 0.59 / MAX: 0.65 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread -pthread
NCNN Target: Vulkan GPU - Model: googlenet OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: Vulkan GPU - Model: googlenet Intel ADL-S GT1 - Intel Core i3-12100 4 8 12 16 20 SE +/- 0.04, N = 3 15.83 MIN: 15.64 / MAX: 54.42 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread -pthread
NCNN Target: Vulkan GPU - Model: vgg16 OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: Vulkan GPU - Model: vgg16 Intel ADL-S GT1 - Intel Core i3-12100 20 40 60 80 100 SE +/- 0.29, N = 3 111.12 MIN: 109.05 / MAX: 253.29 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread -pthread
NCNN Target: Vulkan GPU - Model: resnet18 OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: Vulkan GPU - Model: resnet18 Intel ADL-S GT1 - Intel Core i3-12100 3 6 9 12 15 SE +/- 0.02, N = 3 12.25 MIN: 12.15 / MAX: 12.69 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread -pthread
NCNN Target: Vulkan GPU - Model: alexnet OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: Vulkan GPU - Model: alexnet Intel ADL-S GT1 - Intel Core i3-12100 3 6 9 12 15 SE +/- 0.02, N = 3 12.54 MIN: 12.43 / MAX: 14.78 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread -pthread
NCNN Target: Vulkan GPU - Model: resnet50 OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: Vulkan GPU - Model: resnet50 Intel ADL-S GT1 - Intel Core i3-12100 6 12 18 24 30 SE +/- 0.29, N = 3 26.97 MIN: 26.39 / MAX: 123.4 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread -pthread
NCNN Target: Vulkan GPU - Model: yolov4-tiny OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: Vulkan GPU - Model: yolov4-tiny Intel ADL-S GT1 - Intel Core i3-12100 9 18 27 36 45 SE +/- 0.14, N = 3 39.80 MIN: 39.37 / MAX: 78.33 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread -pthread
NCNN Target: Vulkan GPU - Model: squeezenet_ssd OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: Vulkan GPU - Model: squeezenet_ssd Intel ADL-S GT1 - Intel Core i3-12100 4 8 12 16 20 SE +/- 0.03, N = 3 17.49 MIN: 17.21 / MAX: 58.42 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread -pthread
NCNN Target: Vulkan GPU - Model: regnety_400m OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: Vulkan GPU - Model: regnety_400m Intel ADL-S GT1 - Intel Core i3-12100 2 4 6 8 10 SE +/- 0.00, N = 3 6.53 MIN: 6.48 / MAX: 6.72 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread -pthread
NCNN Target: Vulkan GPU - Model: vision_transformer OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: Vulkan GPU - Model: vision_transformer Intel ADL-S GT1 - Intel Core i3-12100 30 60 90 120 150 SE +/- 0.09, N = 3 136.23 MIN: 134.17 / MAX: 308.71 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread -pthread
NCNN Target: Vulkan GPU - Model: FastestDet OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: Vulkan GPU - Model: FastestDet Intel ADL-S GT1 - Intel Core i3-12100 0.7425 1.485 2.2275 2.97 3.7125 SE +/- 0.02, N = 3 3.30 MIN: 3.23 / MAX: 3.39 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread -pthread
Mlpack Benchmark Benchmark: scikit_ica OpenBenchmarking.org Seconds, Fewer Is Better Mlpack Benchmark Benchmark: scikit_ica Intel ADL-S GT1 - Intel Core i3-12100 14 28 42 56 70 SE +/- 0.72, N = 4 64.74
Mlpack Benchmark Benchmark: scikit_svm OpenBenchmarking.org Seconds, Fewer Is Better Mlpack Benchmark Benchmark: scikit_svm Intel ADL-S GT1 - Intel Core i3-12100 4 8 12 16 20 SE +/- 0.03, N = 3 14.51
oneDNN Harness: IP Shapes 1D - Engine: CPU OpenBenchmarking.org ms, Fewer Is Better oneDNN 3.4 Harness: IP Shapes 1D - Engine: CPU Intel ADL-S GT1 - Intel Core i3-12100 2 4 6 8 10 SE +/- 0.01732, N = 3 6.00601 MIN: 5.51 1. (CXX) g++ options: -O3 -march=native -fopenmp -msse4.1 -fPIC -pie -lpthread -ldl
oneDNN Harness: IP Shapes 3D - Engine: CPU OpenBenchmarking.org ms, Fewer Is Better oneDNN 3.4 Harness: IP Shapes 3D - Engine: CPU Intel ADL-S GT1 - Intel Core i3-12100 6 12 18 24 30 SE +/- 0.03, N = 3 26.70 MIN: 26.52 1. (CXX) g++ options: -O3 -march=native -fopenmp -msse4.1 -fPIC -pie -lpthread -ldl
oneDNN Harness: Convolution Batch Shapes Auto - Engine: CPU OpenBenchmarking.org ms, Fewer Is Better oneDNN 3.4 Harness: Convolution Batch Shapes Auto - Engine: CPU Intel ADL-S GT1 - Intel Core i3-12100 10 20 30 40 50 SE +/- 0.43, N = 3 44.14 MIN: 43.16 1. (CXX) g++ options: -O3 -march=native -fopenmp -msse4.1 -fPIC -pie -lpthread -ldl
oneDNN Harness: Deconvolution Batch shapes_1d - Engine: CPU OpenBenchmarking.org ms, Fewer Is Better oneDNN 3.4 Harness: Deconvolution Batch shapes_1d - Engine: CPU Intel ADL-S GT1 - Intel Core i3-12100 3 6 9 12 15 SE +/- 0.10, N = 15 10.75 MIN: 8.01 1. (CXX) g++ options: -O3 -march=native -fopenmp -msse4.1 -fPIC -pie -lpthread -ldl
oneDNN Harness: Deconvolution Batch shapes_3d - Engine: CPU OpenBenchmarking.org ms, Fewer Is Better oneDNN 3.4 Harness: Deconvolution Batch shapes_3d - Engine: CPU Intel ADL-S GT1 - Intel Core i3-12100 3 6 9 12 15 SE +/- 0.09, N = 3 10.37 MIN: 10.1 1. (CXX) g++ options: -O3 -march=native -fopenmp -msse4.1 -fPIC -pie -lpthread -ldl
oneDNN Harness: Recurrent Neural Network Training - Engine: CPU OpenBenchmarking.org ms, Fewer Is Better oneDNN 3.4 Harness: Recurrent Neural Network Training - Engine: CPU Intel ADL-S GT1 - Intel Core i3-12100 1200 2400 3600 4800 6000 SE +/- 4.76, N = 3 5708.97 MIN: 5677.08 1. (CXX) g++ options: -O3 -march=native -fopenmp -msse4.1 -fPIC -pie -lpthread -ldl
oneDNN Harness: Recurrent Neural Network Inference - Engine: CPU OpenBenchmarking.org ms, Fewer Is Better oneDNN 3.4 Harness: Recurrent Neural Network Inference - Engine: CPU Intel ADL-S GT1 - Intel Core i3-12100 700 1400 2100 2800 3500 SE +/- 12.33, N = 3 3071.31 MIN: 3037.61 1. (CXX) g++ options: -O3 -march=native -fopenmp -msse4.1 -fPIC -pie -lpthread -ldl
OpenVINO Model: Face Detection FP16 - Device: CPU OpenBenchmarking.org FPS, More Is Better OpenVINO 2024.0 Model: Face Detection FP16 - Device: CPU Intel ADL-S GT1 - Intel Core i3-12100 0.279 0.558 0.837 1.116 1.395 SE +/- 0.00, N = 3 1.24 1. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl
OpenVINO Model: Face Detection FP16 - Device: CPU OpenBenchmarking.org ms, Fewer Is Better OpenVINO 2024.0 Model: Face Detection FP16 - Device: CPU Intel ADL-S GT1 - Intel Core i3-12100 700 1400 2100 2800 3500 SE +/- 1.80, N = 3 3213.28 MIN: 2964.37 / MAX: 3356.75 1. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl
OpenVINO Model: Person Detection FP16 - Device: CPU OpenBenchmarking.org FPS, More Is Better OpenVINO 2024.0 Model: Person Detection FP16 - Device: CPU Intel ADL-S GT1 - Intel Core i3-12100 3 6 9 12 15 SE +/- 0.03, N = 3 10.06 1. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl
OpenVINO Model: Person Detection FP16 - Device: CPU OpenBenchmarking.org ms, Fewer Is Better OpenVINO 2024.0 Model: Person Detection FP16 - Device: CPU Intel ADL-S GT1 - Intel Core i3-12100 90 180 270 360 450 SE +/- 1.13, N = 3 397.47 MIN: 223.86 / MAX: 566.44 1. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl
OpenVINO Model: Person Detection FP32 - Device: CPU OpenBenchmarking.org FPS, More Is Better OpenVINO 2024.0 Model: Person Detection FP32 - Device: CPU Intel ADL-S GT1 - Intel Core i3-12100 3 6 9 12 15 SE +/- 0.02, N = 3 10.04 1. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl
OpenVINO Model: Person Detection FP32 - Device: CPU OpenBenchmarking.org ms, Fewer Is Better OpenVINO 2024.0 Model: Person Detection FP32 - Device: CPU Intel ADL-S GT1 - Intel Core i3-12100 90 180 270 360 450 SE +/- 0.71, N = 3 398.50 MIN: 293.03 / MAX: 564.35 1. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl
OpenVINO Model: Vehicle Detection FP16 - Device: CPU OpenBenchmarking.org FPS, More Is Better OpenVINO 2024.0 Model: Vehicle Detection FP16 - Device: CPU Intel ADL-S GT1 - Intel Core i3-12100 14 28 42 56 70 SE +/- 0.06, N = 3 62.61 1. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl
OpenVINO Model: Vehicle Detection FP16 - Device: CPU OpenBenchmarking.org ms, Fewer Is Better OpenVINO 2024.0 Model: Vehicle Detection FP16 - Device: CPU Intel ADL-S GT1 - Intel Core i3-12100 14 28 42 56 70 SE +/- 0.06, N = 3 63.85 MIN: 30.33 / MAX: 201.51 1. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl
OpenVINO Model: Face Detection FP16-INT8 - Device: CPU OpenBenchmarking.org FPS, More Is Better OpenVINO 2024.0 Model: Face Detection FP16-INT8 - Device: CPU Intel ADL-S GT1 - Intel Core i3-12100 1.1835 2.367 3.5505 4.734 5.9175 SE +/- 0.01, N = 3 5.26 1. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl
OpenVINO Model: Face Detection FP16-INT8 - Device: CPU OpenBenchmarking.org ms, Fewer Is Better OpenVINO 2024.0 Model: Face Detection FP16-INT8 - Device: CPU Intel ADL-S GT1 - Intel Core i3-12100 160 320 480 640 800 SE +/- 1.39, N = 3 759.24 MIN: 716.66 / MAX: 920.51 1. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl
OpenVINO Model: Face Detection Retail FP16 - Device: CPU OpenBenchmarking.org FPS, More Is Better OpenVINO 2024.0 Model: Face Detection Retail FP16 - Device: CPU Intel ADL-S GT1 - Intel Core i3-12100 70 140 210 280 350 SE +/- 2.36, N = 3 330.42 1. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl
OpenVINO Model: Face Detection Retail FP16 - Device: CPU OpenBenchmarking.org ms, Fewer Is Better OpenVINO 2024.0 Model: Face Detection Retail FP16 - Device: CPU Intel ADL-S GT1 - Intel Core i3-12100 3 6 9 12 15 SE +/- 0.09, N = 3 12.09 MIN: 5.92 / MAX: 79.84 1. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl
OpenVINO Model: Road Segmentation ADAS FP16 - Device: CPU OpenBenchmarking.org FPS, More Is Better OpenVINO 2024.0 Model: Road Segmentation ADAS FP16 - Device: CPU Intel ADL-S GT1 - Intel Core i3-12100 4 8 12 16 20 SE +/- 0.04, N = 3 18.24 1. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl
OpenVINO Model: Road Segmentation ADAS FP16 - Device: CPU OpenBenchmarking.org ms, Fewer Is Better OpenVINO 2024.0 Model: Road Segmentation ADAS FP16 - Device: CPU Intel ADL-S GT1 - Intel Core i3-12100 50 100 150 200 250 SE +/- 0.53, N = 3 219.10 MIN: 120.84 / MAX: 389.85 1. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl
OpenVINO Model: Vehicle Detection FP16-INT8 - Device: CPU OpenBenchmarking.org FPS, More Is Better OpenVINO 2024.0 Model: Vehicle Detection FP16-INT8 - Device: CPU Intel ADL-S GT1 - Intel Core i3-12100 50 100 150 200 250 SE +/- 0.79, N = 3 214.21 1. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl
OpenVINO Model: Vehicle Detection FP16-INT8 - Device: CPU OpenBenchmarking.org ms, Fewer Is Better OpenVINO 2024.0 Model: Vehicle Detection FP16-INT8 - Device: CPU Intel ADL-S GT1 - Intel Core i3-12100 5 10 15 20 25 SE +/- 0.07, N = 3 18.66 MIN: 9.8 / MAX: 90.77 1. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl
OpenVINO Model: Weld Porosity Detection FP16 - Device: CPU OpenBenchmarking.org FPS, More Is Better OpenVINO 2024.0 Model: Weld Porosity Detection FP16 - Device: CPU Intel ADL-S GT1 - Intel Core i3-12100 30 60 90 120 150 SE +/- 0.25, N = 3 136.74 1. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl
OpenVINO Model: Weld Porosity Detection FP16 - Device: CPU OpenBenchmarking.org ms, Fewer Is Better OpenVINO 2024.0 Model: Weld Porosity Detection FP16 - Device: CPU Intel ADL-S GT1 - Intel Core i3-12100 7 14 21 28 35 SE +/- 0.06, N = 3 29.24 MIN: 19.89 / MAX: 66.26 1. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl
OpenVINO Model: Face Detection Retail FP16-INT8 - Device: CPU OpenBenchmarking.org FPS, More Is Better OpenVINO 2024.0 Model: Face Detection Retail FP16-INT8 - Device: CPU Intel ADL-S GT1 - Intel Core i3-12100 200 400 600 800 1000 SE +/- 3.58, N = 3 833.34 1. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl
OpenVINO Model: Face Detection Retail FP16-INT8 - Device: CPU OpenBenchmarking.org ms, Fewer Is Better OpenVINO 2024.0 Model: Face Detection Retail FP16-INT8 - Device: CPU Intel ADL-S GT1 - Intel Core i3-12100 1.08 2.16 3.24 4.32 5.4 SE +/- 0.02, N = 3 4.80 MIN: 3.51 / MAX: 52.64 1. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl
OpenVINO Model: Road Segmentation ADAS FP16-INT8 - Device: CPU OpenBenchmarking.org FPS, More Is Better OpenVINO 2024.0 Model: Road Segmentation ADAS FP16-INT8 - Device: CPU Intel ADL-S GT1 - Intel Core i3-12100 14 28 42 56 70 SE +/- 0.23, N = 3 63.19 1. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl
OpenVINO Model: Road Segmentation ADAS FP16-INT8 - Device: CPU OpenBenchmarking.org ms, Fewer Is Better OpenVINO 2024.0 Model: Road Segmentation ADAS FP16-INT8 - Device: CPU Intel ADL-S GT1 - Intel Core i3-12100 14 28 42 56 70 SE +/- 0.22, N = 3 63.27 MIN: 31.99 / MAX: 132.5 1. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl
OpenVINO Model: Machine Translation EN To DE FP16 - Device: CPU OpenBenchmarking.org FPS, More Is Better OpenVINO 2024.0 Model: Machine Translation EN To DE FP16 - Device: CPU Intel ADL-S GT1 - Intel Core i3-12100 4 8 12 16 20 SE +/- 0.02, N = 3 14.95 1. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl
OpenVINO Model: Machine Translation EN To DE FP16 - Device: CPU OpenBenchmarking.org ms, Fewer Is Better OpenVINO 2024.0 Model: Machine Translation EN To DE FP16 - Device: CPU Intel ADL-S GT1 - Intel Core i3-12100 60 120 180 240 300 SE +/- 0.35, N = 3 267.44 MIN: 221.5 / MAX: 472.42 1. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl
OpenVINO Model: Weld Porosity Detection FP16-INT8 - Device: CPU OpenBenchmarking.org FPS, More Is Better OpenVINO 2024.0 Model: Weld Porosity Detection FP16-INT8 - Device: CPU Intel ADL-S GT1 - Intel Core i3-12100 110 220 330 440 550 SE +/- 0.62, N = 3 513.04 1. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl
OpenVINO Model: Weld Porosity Detection FP16-INT8 - Device: CPU OpenBenchmarking.org ms, Fewer Is Better OpenVINO 2024.0 Model: Weld Porosity Detection FP16-INT8 - Device: CPU Intel ADL-S GT1 - Intel Core i3-12100 2 4 6 8 10 SE +/- 0.01, N = 3 7.79 MIN: 4.62 / MAX: 55.08 1. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl
OpenVINO Model: Person Vehicle Bike Detection FP16 - Device: CPU OpenBenchmarking.org FPS, More Is Better OpenVINO 2024.0 Model: Person Vehicle Bike Detection FP16 - Device: CPU Intel ADL-S GT1 - Intel Core i3-12100 40 80 120 160 200 SE +/- 2.19, N = 3 166.71 1. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl
OpenVINO Model: Person Vehicle Bike Detection FP16 - Device: CPU OpenBenchmarking.org ms, Fewer Is Better OpenVINO 2024.0 Model: Person Vehicle Bike Detection FP16 - Device: CPU Intel ADL-S GT1 - Intel Core i3-12100 6 12 18 24 30 SE +/- 0.32, N = 3 23.99 MIN: 13.37 / MAX: 116.89 1. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl
OpenVINO Model: Noise Suppression Poconet-Like FP16 - Device: CPU OpenBenchmarking.org FPS, More Is Better OpenVINO 2024.0 Model: Noise Suppression Poconet-Like FP16 - Device: CPU Intel ADL-S GT1 - Intel Core i3-12100 50 100 150 200 250 SE +/- 0.74, N = 3 208.84 1. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl
OpenVINO Model: Noise Suppression Poconet-Like FP16 - Device: CPU OpenBenchmarking.org ms, Fewer Is Better OpenVINO 2024.0 Model: Noise Suppression Poconet-Like FP16 - Device: CPU Intel ADL-S GT1 - Intel Core i3-12100 5 10 15 20 25 SE +/- 0.06, N = 3 19.10 MIN: 10.05 / MAX: 69.6 1. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl
OpenVINO Model: Handwritten English Recognition FP16 - Device: CPU OpenBenchmarking.org FPS, More Is Better OpenVINO 2024.0 Model: Handwritten English Recognition FP16 - Device: CPU Intel ADL-S GT1 - Intel Core i3-12100 15 30 45 60 75 SE +/- 0.24, N = 3 66.40 1. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl
OpenVINO Model: Handwritten English Recognition FP16 - Device: CPU OpenBenchmarking.org ms, Fewer Is Better OpenVINO 2024.0 Model: Handwritten English Recognition FP16 - Device: CPU Intel ADL-S GT1 - Intel Core i3-12100 13 26 39 52 65 SE +/- 0.22, N = 3 60.21 MIN: 40.96 / MAX: 136.76 1. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl
OpenVINO Model: Person Re-Identification Retail FP16 - Device: CPU OpenBenchmarking.org FPS, More Is Better OpenVINO 2024.0 Model: Person Re-Identification Retail FP16 - Device: CPU Intel ADL-S GT1 - Intel Core i3-12100 40 80 120 160 200 SE +/- 1.60, N = 3 201.90 1. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl
OpenVINO Model: Person Re-Identification Retail FP16 - Device: CPU OpenBenchmarking.org ms, Fewer Is Better OpenVINO 2024.0 Model: Person Re-Identification Retail FP16 - Device: CPU Intel ADL-S GT1 - Intel Core i3-12100 5 10 15 20 25 SE +/- 0.16, N = 3 19.80 MIN: 9.77 / MAX: 70.87 1. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl
OpenVINO Model: Age Gender Recognition Retail 0013 FP16 - Device: CPU OpenBenchmarking.org FPS, More Is Better OpenVINO 2024.0 Model: Age Gender Recognition Retail 0013 FP16 - Device: CPU Intel ADL-S GT1 - Intel Core i3-12100 700 1400 2100 2800 3500 SE +/- 28.02, N = 3 3481.92 1. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl
OpenVINO Model: Age Gender Recognition Retail 0013 FP16 - Device: CPU OpenBenchmarking.org ms, Fewer Is Better OpenVINO 2024.0 Model: Age Gender Recognition Retail 0013 FP16 - Device: CPU Intel ADL-S GT1 - Intel Core i3-12100 0.2565 0.513 0.7695 1.026 1.2825 SE +/- 0.01, N = 3 1.14 MIN: 0.58 / MAX: 29.63 1. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl
OpenVINO Model: Handwritten English Recognition FP16-INT8 - Device: CPU OpenBenchmarking.org FPS, More Is Better OpenVINO 2024.0 Model: Handwritten English Recognition FP16-INT8 - Device: CPU Intel ADL-S GT1 - Intel Core i3-12100 20 40 60 80 100 SE +/- 0.38, N = 3 88.41 1. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl
OpenVINO Model: Handwritten English Recognition FP16-INT8 - Device: CPU OpenBenchmarking.org ms, Fewer Is Better OpenVINO 2024.0 Model: Handwritten English Recognition FP16-INT8 - Device: CPU Intel ADL-S GT1 - Intel Core i3-12100 10 20 30 40 50 SE +/- 0.20, N = 3 45.22 MIN: 31.25 / MAX: 119.06 1. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl
OpenVINO Model: Age Gender Recognition Retail 0013 FP16-INT8 - Device: CPU OpenBenchmarking.org FPS, More Is Better OpenVINO 2024.0 Model: Age Gender Recognition Retail 0013 FP16-INT8 - Device: CPU Intel ADL-S GT1 - Intel Core i3-12100 2K 4K 6K 8K 10K SE +/- 11.73, N = 3 10592.67 1. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl
OpenVINO Model: Age Gender Recognition Retail 0013 FP16-INT8 - Device: CPU OpenBenchmarking.org ms, Fewer Is Better OpenVINO 2024.0 Model: Age Gender Recognition Retail 0013 FP16-INT8 - Device: CPU Intel ADL-S GT1 - Intel Core i3-12100 0.0833 0.1666 0.2499 0.3332 0.4165 SE +/- 0.00, N = 3 0.37 MIN: 0.24 / MAX: 45.25 1. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl
ONNX Runtime Model: GPT-2 - Device: CPU - Executor: Parallel OpenBenchmarking.org Inference Time Cost (ms), Fewer Is Better ONNX Runtime 1.17 Model: GPT-2 - Device: CPU - Executor: Parallel Intel ADL-S GT1 - Intel Core i3-12100 5 10 15 20 25 SE +/- 0.03, N = 3 19.87 1. (CXX) g++ options: -O3 -march=native -ffunction-sections -fdata-sections -mtune=native -flto -fno-fat-lto-objects -ldl -lrt -lpthread -pthread
ONNX Runtime Model: GPT-2 - Device: CPU - Executor: Standard OpenBenchmarking.org Inference Time Cost (ms), Fewer Is Better ONNX Runtime 1.17 Model: GPT-2 - Device: CPU - Executor: Standard Intel ADL-S GT1 - Intel Core i3-12100 5 10 15 20 25 SE +/- 0.19, N = 3 19.56 1. (CXX) g++ options: -O3 -march=native -ffunction-sections -fdata-sections -mtune=native -flto -fno-fat-lto-objects -ldl -lrt -lpthread -pthread
ONNX Runtime Model: T5 Encoder - Device: CPU - Executor: Parallel OpenBenchmarking.org Inference Time Cost (ms), Fewer Is Better ONNX Runtime 1.17 Model: T5 Encoder - Device: CPU - Executor: Parallel Intel ADL-S GT1 - Intel Core i3-12100 4 8 12 16 20 SE +/- 0.01, N = 3 17.85 1. (CXX) g++ options: -O3 -march=native -ffunction-sections -fdata-sections -mtune=native -flto -fno-fat-lto-objects -ldl -lrt -lpthread -pthread
ONNX Runtime Model: T5 Encoder - Device: CPU - Executor: Standard OpenBenchmarking.org Inference Time Cost (ms), Fewer Is Better ONNX Runtime 1.17 Model: T5 Encoder - Device: CPU - Executor: Standard Intel ADL-S GT1 - Intel Core i3-12100 4 8 12 16 20 SE +/- 0.01, N = 3 17.36 1. (CXX) g++ options: -O3 -march=native -ffunction-sections -fdata-sections -mtune=native -flto -fno-fat-lto-objects -ldl -lrt -lpthread -pthread
ONNX Runtime Model: bertsquad-12 - Device: CPU - Executor: Parallel OpenBenchmarking.org Inference Time Cost (ms), Fewer Is Better ONNX Runtime 1.17 Model: bertsquad-12 - Device: CPU - Executor: Parallel Intel ADL-S GT1 - Intel Core i3-12100 40 80 120 160 200 SE +/- 2.89, N = 15 187.69 1. (CXX) g++ options: -O3 -march=native -ffunction-sections -fdata-sections -mtune=native -flto -fno-fat-lto-objects -ldl -lrt -lpthread -pthread
ONNX Runtime Model: bertsquad-12 - Device: CPU - Executor: Standard OpenBenchmarking.org Inference Time Cost (ms), Fewer Is Better ONNX Runtime 1.17 Model: bertsquad-12 - Device: CPU - Executor: Standard Intel ADL-S GT1 - Intel Core i3-12100 30 60 90 120 150 SE +/- 8.08, N = 14 130.81 1. (CXX) g++ options: -O3 -march=native -ffunction-sections -fdata-sections -mtune=native -flto -fno-fat-lto-objects -ldl -lrt -lpthread -pthread
ONNX Runtime Model: CaffeNet 12-int8 - Device: CPU - Executor: Parallel OpenBenchmarking.org Inference Time Cost (ms), Fewer Is Better ONNX Runtime 1.17 Model: CaffeNet 12-int8 - Device: CPU - Executor: Parallel Intel ADL-S GT1 - Intel Core i3-12100 1.0743 2.1486 3.2229 4.2972 5.3715 SE +/- 0.05286, N = 3 4.77472 1. (CXX) g++ options: -O3 -march=native -ffunction-sections -fdata-sections -mtune=native -flto -fno-fat-lto-objects -ldl -lrt -lpthread -pthread
ONNX Runtime Model: CaffeNet 12-int8 - Device: CPU - Executor: Standard OpenBenchmarking.org Inference Time Cost (ms), Fewer Is Better ONNX Runtime 1.17 Model: CaffeNet 12-int8 - Device: CPU - Executor: Standard Intel ADL-S GT1 - Intel Core i3-12100 0.9647 1.9294 2.8941 3.8588 4.8235 SE +/- 0.04898, N = 3 4.28771 1. (CXX) g++ options: -O3 -march=native -ffunction-sections -fdata-sections -mtune=native -flto -fno-fat-lto-objects -ldl -lrt -lpthread -pthread
ONNX Runtime Model: fcn-resnet101-11 - Device: CPU - Executor: Parallel OpenBenchmarking.org Inference Time Cost (ms), Fewer Is Better ONNX Runtime 1.17 Model: fcn-resnet101-11 - Device: CPU - Executor: Parallel Intel ADL-S GT1 - Intel Core i3-12100 400 800 1200 1600 2000 SE +/- 57.61, N = 15 1975.36 1. (CXX) g++ options: -O3 -march=native -ffunction-sections -fdata-sections -mtune=native -flto -fno-fat-lto-objects -ldl -lrt -lpthread -pthread
ONNX Runtime Model: fcn-resnet101-11 - Device: CPU - Executor: Standard OpenBenchmarking.org Inference Time Cost (ms), Fewer Is Better ONNX Runtime 1.17 Model: fcn-resnet101-11 - Device: CPU - Executor: Standard Intel ADL-S GT1 - Intel Core i3-12100 300 600 900 1200 1500 SE +/- 26.80, N = 15 1353.24 1. (CXX) g++ options: -O3 -march=native -ffunction-sections -fdata-sections -mtune=native -flto -fno-fat-lto-objects -ldl -lrt -lpthread -pthread
ONNX Runtime Model: ArcFace ResNet-100 - Device: CPU - Executor: Parallel OpenBenchmarking.org Inference Time Cost (ms), Fewer Is Better ONNX Runtime 1.17 Model: ArcFace ResNet-100 - Device: CPU - Executor: Parallel Intel ADL-S GT1 - Intel Core i3-12100 20 40 60 80 100 SE +/- 0.89, N = 6 86.70 1. (CXX) g++ options: -O3 -march=native -ffunction-sections -fdata-sections -mtune=native -flto -fno-fat-lto-objects -ldl -lrt -lpthread -pthread
ONNX Runtime Model: ArcFace ResNet-100 - Device: CPU - Executor: Standard OpenBenchmarking.org Inference Time Cost (ms), Fewer Is Better ONNX Runtime 1.17 Model: ArcFace ResNet-100 - Device: CPU - Executor: Standard Intel ADL-S GT1 - Intel Core i3-12100 13 26 39 52 65 SE +/- 0.65, N = 5 57.79 1. (CXX) g++ options: -O3 -march=native -ffunction-sections -fdata-sections -mtune=native -flto -fno-fat-lto-objects -ldl -lrt -lpthread -pthread
ONNX Runtime Model: ResNet50 v1-12-int8 - Device: CPU - Executor: Parallel OpenBenchmarking.org Inference Time Cost (ms), Fewer Is Better ONNX Runtime 1.17 Model: ResNet50 v1-12-int8 - Device: CPU - Executor: Parallel Intel ADL-S GT1 - Intel Core i3-12100 2 4 6 8 10 SE +/- 0.09036, N = 3 8.19737 1. (CXX) g++ options: -O3 -march=native -ffunction-sections -fdata-sections -mtune=native -flto -fno-fat-lto-objects -ldl -lrt -lpthread -pthread
ONNX Runtime Model: ResNet50 v1-12-int8 - Device: CPU - Executor: Standard OpenBenchmarking.org Inference Time Cost (ms), Fewer Is Better ONNX Runtime 1.17 Model: ResNet50 v1-12-int8 - Device: CPU - Executor: Standard Intel ADL-S GT1 - Intel Core i3-12100 2 4 6 8 10 SE +/- 0.04100, N = 3 7.00536 1. (CXX) g++ options: -O3 -march=native -ffunction-sections -fdata-sections -mtune=native -flto -fno-fat-lto-objects -ldl -lrt -lpthread -pthread
ONNX Runtime Model: super-resolution-10 - Device: CPU - Executor: Parallel OpenBenchmarking.org Inference Time Cost (ms), Fewer Is Better ONNX Runtime 1.17 Model: super-resolution-10 - Device: CPU - Executor: Parallel Intel ADL-S GT1 - Intel Core i3-12100 6 12 18 24 30 SE +/- 0.28, N = 4 26.29 1. (CXX) g++ options: -O3 -march=native -ffunction-sections -fdata-sections -mtune=native -flto -fno-fat-lto-objects -ldl -lrt -lpthread -pthread
ONNX Runtime Model: super-resolution-10 - Device: CPU - Executor: Standard OpenBenchmarking.org Inference Time Cost (ms), Fewer Is Better ONNX Runtime 1.17 Model: super-resolution-10 - Device: CPU - Executor: Standard Intel ADL-S GT1 - Intel Core i3-12100 5 10 15 20 25 SE +/- 0.90, N = 15 19.21 1. (CXX) g++ options: -O3 -march=native -ffunction-sections -fdata-sections -mtune=native -flto -fno-fat-lto-objects -ldl -lrt -lpthread -pthread
ONNX Runtime Model: Faster R-CNN R-50-FPN-int8 - Device: CPU - Executor: Parallel OpenBenchmarking.org Inference Time Cost (ms), Fewer Is Better ONNX Runtime 1.17 Model: Faster R-CNN R-50-FPN-int8 - Device: CPU - Executor: Parallel Intel ADL-S GT1 - Intel Core i3-12100 50 100 150 200 250 SE +/- 1.70, N = 3 225.47 1. (CXX) g++ options: -O3 -march=native -ffunction-sections -fdata-sections -mtune=native -flto -fno-fat-lto-objects -ldl -lrt -lpthread -pthread
ONNX Runtime Model: Faster R-CNN R-50-FPN-int8 - Device: CPU - Executor: Standard OpenBenchmarking.org Inference Time Cost (ms), Fewer Is Better ONNX Runtime 1.17 Model: Faster R-CNN R-50-FPN-int8 - Device: CPU - Executor: Standard Intel ADL-S GT1 - Intel Core i3-12100 40 80 120 160 200 SE +/- 2.83, N = 12 196.43 1. (CXX) g++ options: -O3 -march=native -ffunction-sections -fdata-sections -mtune=native -flto -fno-fat-lto-objects -ldl -lrt -lpthread -pthread
Phoronix Test Suite v10.8.5