ML Intel Core i7-12700H testing with a Intel NUC12SNKi72 (SNADL357.0055.2022.0923.1555 BIOS) and Intel Arctm A770M DG2 16GB on Ubuntu 22.04 via the Phoronix Test Suite. Intel Arctm A770M DG2: Processor: Intel Core i7-12700H @ 4.60GHz (14 Cores / 20 Threads), Motherboard: Intel NUC12SNKi72 (SNADL357.0055.2022.0923.1555 BIOS), Chipset: Intel Alder Lake PCH, Memory: 16GB, Disk: 1024GB SAMSUNG MZVL21T0HCLR-00A00, Graphics: Intel Arctm A770M DG2 16GB (1400MHz), Audio: Realtek ALC274, Monitor: S27H85x, Network: Intel I225-LM + Intel Alder Lake-P PCH CNVi WiFi OS: Ubuntu 22.04, Kernel: 5.19.0-42-generic (x86_64), Desktop: GNOME Shell 42.5, Display Server: X Server 1.21.1.3 + Wayland, OpenGL: 4.6 Mesa 23.1.0-devel (git-722bcd7973), OpenCL: OpenCL 3.0 + OpenCL 3.0, Vulkan: 1.3.238, Compiler: GCC 11.3.0, File-System: ext4, Screen Resolution: 2560x1440 AI Benchmark Alpha 0.1.2 Device Inference Score Score > Higher Is Better Intel Arctm A770M DG2 . 1089 |================================================= AI Benchmark Alpha 0.1.2 Device Training Score Score > Higher Is Better Intel Arctm A770M DG2 . 1631 |================================================= AI Benchmark Alpha 0.1.2 Device AI Score Score > Higher Is Better Intel Arctm A770M DG2 . 2720 |================================================= Caffe 2020-02-13 Model: AlexNet - Acceleration: CPU - Iterations: 100 Milli-Seconds < Lower Is Better Intel Arctm A770M DG2 . 33092 |================================================ Caffe 2020-02-13 Model: AlexNet - Acceleration: CPU - Iterations: 200 Milli-Seconds < Lower Is Better Intel Arctm A770M DG2 . 67449 |================================================ Caffe 2020-02-13 Model: AlexNet - Acceleration: CPU - Iterations: 1000 Milli-Seconds < Lower Is Better Intel Arctm A770M DG2 . 341794 |=============================================== Caffe 2020-02-13 Model: GoogleNet - Acceleration: CPU - Iterations: 100 Milli-Seconds < Lower Is Better Intel Arctm A770M DG2 . 93045 |================================================ Caffe 2020-02-13 Model: GoogleNet - Acceleration: CPU - Iterations: 200 Milli-Seconds < Lower Is Better Intel Arctm A770M DG2 . 187086 |=============================================== Caffe 2020-02-13 Model: GoogleNet - Acceleration: CPU - Iterations: 1000 Milli-Seconds < Lower Is Better Intel Arctm A770M DG2 . 940482 |=============================================== DeepSpeech 0.6 Acceleration: CPU Seconds < Lower Is Better Intel Arctm A770M DG2 . 65.91 |================================================ ECP-CANDLE 0.4 Benchmark: P1B2 Seconds < Lower Is Better ECP-CANDLE 0.4 Benchmark: P3B1 Seconds < Lower Is Better ECP-CANDLE 0.4 Benchmark: P3B2 Seconds < Lower Is Better LeelaChessZero 0.28 Backend: BLAS Nodes Per Second > Higher Is Better Intel Arctm A770M DG2 . 1132 |================================================= Mlpack Benchmark Benchmark: scikit_ica Seconds < Lower Is Better Intel Arctm A770M DG2 . 39.71 |================================================ Mlpack Benchmark Benchmark: scikit_qda Seconds < Lower Is Better Intel Arctm A770M DG2 . 48.44 |================================================ Mlpack Benchmark Benchmark: scikit_svm Seconds < Lower Is Better Intel Arctm A770M DG2 . 12.54 |================================================ Mlpack Benchmark Benchmark: scikit_linearridgeregression Seconds < Lower Is Better Intel Arctm A770M DG2 . 2.09 |================================================= Mobile Neural Network 2.1 Model: nasnet ms < Lower Is Better Intel Arctm A770M DG2 . 10.93 |================================================ Mobile Neural Network 2.1 Model: mobilenetV3 ms < Lower Is Better Intel Arctm A770M DG2 . 1.508 |================================================ Mobile Neural Network 2.1 Model: squeezenetv1.1 ms < Lower Is Better Intel Arctm A770M DG2 . 3.703 |================================================ Mobile Neural Network 2.1 Model: resnet-v2-50 ms < Lower Is Better Intel Arctm A770M DG2 . 28.04 |================================================ Mobile Neural Network 2.1 Model: SqueezeNetV1.0 ms < Lower Is Better Intel Arctm A770M DG2 . 5.823 |================================================ Mobile Neural Network 2.1 Model: MobileNetV2_224 ms < Lower Is Better Intel Arctm A770M DG2 . 3.181 |================================================ Mobile Neural Network 2.1 Model: mobilenet-v1-1.0 ms < Lower Is Better Intel Arctm A770M DG2 . 3.817 |================================================ Mobile Neural Network 2.1 Model: inception-v3 ms < Lower Is Better Intel Arctm A770M DG2 . 30.91 |================================================ NCNN 20220729 Target: CPU - Model: mobilenet ms < Lower Is Better Intel Arctm A770M DG2 . 11.85 |================================================ NCNN 20220729 Target: CPU-v2-v2 - Model: mobilenet-v2 ms < Lower Is Better Intel Arctm A770M DG2 . 3.27 |================================================= NCNN 20220729 Target: CPU-v3-v3 - Model: mobilenet-v3 ms < Lower Is Better Intel Arctm A770M DG2 . 2.79 |================================================= NCNN 20220729 Target: CPU - Model: shufflenet-v2 ms < Lower Is Better Intel Arctm A770M DG2 . 2.94 |================================================= NCNN 20220729 Target: CPU - Model: mnasnet ms < Lower Is Better Intel Arctm A770M DG2 . 3.15 |================================================= NCNN 20220729 Target: CPU - Model: efficientnet-b0 ms < Lower Is Better Intel Arctm A770M DG2 . 5.77 |================================================= NCNN 20220729 Target: CPU - Model: blazeface ms < Lower Is Better Intel Arctm A770M DG2 . 1.14 |================================================= NCNN 20220729 Target: CPU - Model: googlenet ms < Lower Is Better Intel Arctm A770M DG2 . 9.32 |================================================= NCNN 20220729 Target: CPU - Model: vgg16 ms < Lower Is Better Intel Arctm A770M DG2 . 40.44 |================================================ NCNN 20220729 Target: CPU - Model: resnet18 ms < Lower Is Better Intel Arctm A770M DG2 . 8.66 |================================================= NCNN 20220729 Target: CPU - Model: alexnet ms < Lower Is Better Intel Arctm A770M DG2 . 6.69 |================================================= NCNN 20220729 Target: CPU - Model: resnet50 ms < Lower Is Better Intel Arctm A770M DG2 . 15.57 |================================================ NCNN 20220729 Target: CPU - Model: yolov4-tiny ms < Lower Is Better Intel Arctm A770M DG2 . 18.37 |================================================ NCNN 20220729 Target: CPU - Model: squeezenet_ssd ms < Lower Is Better Intel Arctm A770M DG2 . 12.46 |================================================ NCNN 20220729 Target: CPU - Model: regnety_400m ms < Lower Is Better Intel Arctm A770M DG2 . 9.41 |================================================= NCNN 20220729 Target: CPU - Model: vision_transformer ms < Lower Is Better Intel Arctm A770M DG2 . 203.13 |=============================================== NCNN 20220729 Target: CPU - Model: FastestDet ms < Lower Is Better Intel Arctm A770M DG2 . 4.08 |================================================= NCNN 20220729 Target: Vulkan GPU - Model: mobilenet ms < Lower Is Better Intel Arctm A770M DG2 . 21.28 |================================================ NCNN 20220729 Target: Vulkan GPU-v2-v2 - Model: mobilenet-v2 ms < Lower Is Better Intel Arctm A770M DG2 . 5.49 |================================================= NCNN 20220729 Target: Vulkan GPU-v3-v3 - Model: mobilenet-v3 ms < Lower Is Better Intel Arctm A770M DG2 . 7.71 |================================================= NCNN 20220729 Target: Vulkan GPU - Model: shufflenet-v2 ms < Lower Is Better Intel Arctm A770M DG2 . 3.64 |================================================= NCNN 20220729 Target: Vulkan GPU - Model: mnasnet ms < Lower Is Better Intel Arctm A770M DG2 . 4.98 |================================================= NCNN 20220729 Target: Vulkan GPU - Model: efficientnet-b0 ms < Lower Is Better Intel Arctm A770M DG2 . 17.10 |================================================ NCNN 20220729 Target: Vulkan GPU - Model: blazeface ms < Lower Is Better Intel Arctm A770M DG2 . 1.59 |================================================= NCNN 20220729 Target: Vulkan GPU - Model: googlenet ms < Lower Is Better Intel Arctm A770M DG2 . 20.59 |================================================ NCNN 20220729 Target: Vulkan GPU - Model: vgg16 ms < Lower Is Better Intel Arctm A770M DG2 . 16.79 |================================================ NCNN 20220729 Target: Vulkan GPU - Model: resnet18 ms < Lower Is Better Intel Arctm A770M DG2 . 17.54 |================================================ NCNN 20220729 Target: Vulkan GPU - Model: alexnet ms < Lower Is Better Intel Arctm A770M DG2 . 3.22 |================================================= NCNN 20220729 Target: Vulkan GPU - Model: resnet50 ms < Lower Is Better Intel Arctm A770M DG2 . 22.53 |================================================ NCNN 20220729 Target: Vulkan GPU - Model: yolov4-tiny ms < Lower Is Better Intel Arctm A770M DG2 . 24.44 |================================================ NCNN 20220729 Target: Vulkan GPU - Model: squeezenet_ssd ms < Lower Is Better Intel Arctm A770M DG2 . 21.15 |================================================ NCNN 20220729 Target: Vulkan GPU - Model: regnety_400m ms < Lower Is Better Intel Arctm A770M DG2 . 10.44 |================================================ NCNN 20220729 Target: Vulkan GPU - Model: vision_transformer ms < Lower Is Better Intel Arctm A770M DG2 . 1487.89 |============================================== NCNN 20220729 Target: Vulkan GPU - Model: FastestDet ms < Lower Is Better Intel Arctm A770M DG2 . 20.43 |================================================ Neural Magic DeepSparse 1.3.2 Model: NLP Document Classification, oBERT base uncased on IMDB - Scenario: Asynchronous Multi-Stream items/sec > Higher Is Better Intel Arctm A770M DG2 . 7.7657 |=============================================== Neural Magic DeepSparse 1.3.2 Model: NLP Document Classification, oBERT base uncased on IMDB - Scenario: Asynchronous Multi-Stream ms/batch < Lower Is Better Intel Arctm A770M DG2 . 895.03 |=============================================== Neural Magic DeepSparse 1.3.2 Model: NLP Document Classification, oBERT base uncased on IMDB - Scenario: Synchronous Single-Stream items/sec > Higher Is Better Intel Arctm A770M DG2 . 7.3377 |=============================================== Neural Magic DeepSparse 1.3.2 Model: NLP Document Classification, oBERT base uncased on IMDB - Scenario: Synchronous Single-Stream ms/batch < Lower Is Better Intel Arctm A770M DG2 . 136.28 |=============================================== Neural Magic DeepSparse 1.3.2 Model: NLP Sentiment Analysis, 80% Pruned Quantized BERT Base Uncased - Scenario: Asynchronous Multi-Stream items/sec > Higher Is Better Intel Arctm A770M DG2 . 94.93 |================================================ Neural Magic DeepSparse 1.3.2 Model: NLP Sentiment Analysis, 80% Pruned Quantized BERT Base Uncased - Scenario: Asynchronous Multi-Stream ms/batch < Lower Is Better Intel Arctm A770M DG2 . 73.65 |================================================ Neural Magic DeepSparse 1.3.2 Model: NLP Sentiment Analysis, 80% Pruned Quantized BERT Base Uncased - Scenario: Synchronous Single-Stream items/sec > Higher Is Better Intel Arctm A770M DG2 . 61.07 |================================================ Neural Magic DeepSparse 1.3.2 Model: NLP Sentiment Analysis, 80% Pruned Quantized BERT Base Uncased - Scenario: Synchronous Single-Stream ms/batch < Lower Is Better Intel Arctm A770M DG2 . 16.37 |================================================ Neural Magic DeepSparse 1.3.2 Model: NLP Question Answering, BERT base uncased SQuaD 12layer Pruned90 - Scenario: Asynchronous Multi-Stream items/sec > Higher Is Better Intel Arctm A770M DG2 . 29.30 |================================================ Neural Magic DeepSparse 1.3.2 Model: NLP Question Answering, BERT base uncased SQuaD 12layer Pruned90 - Scenario: Asynchronous Multi-Stream ms/batch < Lower Is Better Intel Arctm A770M DG2 . 238.12 |=============================================== Neural Magic DeepSparse 1.3.2 Model: NLP Question Answering, BERT base uncased SQuaD 12layer Pruned90 - Scenario: Synchronous Single-Stream items/sec > Higher Is Better Intel Arctm A770M DG2 . 22.50 |================================================ Neural Magic DeepSparse 1.3.2 Model: NLP Question Answering, BERT base uncased SQuaD 12layer Pruned90 - Scenario: Synchronous Single-Stream ms/batch < Lower Is Better Intel Arctm A770M DG2 . 44.44 |================================================ Neural Magic DeepSparse 1.3.2 Model: CV Detection, YOLOv5s COCO - Scenario: Asynchronous Multi-Stream items/sec > Higher Is Better Intel Arctm A770M DG2 . 48.19 |================================================ Neural Magic DeepSparse 1.3.2 Model: CV Detection, YOLOv5s COCO - Scenario: Asynchronous Multi-Stream ms/batch < Lower Is Better Intel Arctm A770M DG2 . 144.97 |=============================================== Neural Magic DeepSparse 1.3.2 Model: CV Detection, YOLOv5s COCO - Scenario: Synchronous Single-Stream items/sec > Higher Is Better Intel Arctm A770M DG2 . 39.52 |================================================ Neural Magic DeepSparse 1.3.2 Model: CV Detection, YOLOv5s COCO - Scenario: Synchronous Single-Stream ms/batch < Lower Is Better Intel Arctm A770M DG2 . 25.29 |================================================ Neural Magic DeepSparse 1.3.2 Model: CV Classification, ResNet-50 ImageNet - Scenario: Asynchronous Multi-Stream items/sec > Higher Is Better Intel Arctm A770M DG2 . 104.15 |=============================================== Neural Magic DeepSparse 1.3.2 Model: CV Classification, ResNet-50 ImageNet - Scenario: Asynchronous Multi-Stream ms/batch < Lower Is Better Intel Arctm A770M DG2 . 67.15 |================================================ Neural Magic DeepSparse 1.3.2 Model: CV Classification, ResNet-50 ImageNet - Scenario: Synchronous Single-Stream items/sec > Higher Is Better Intel Arctm A770M DG2 . 70.13 |================================================ Neural Magic DeepSparse 1.3.2 Model: CV Classification, ResNet-50 ImageNet - Scenario: Synchronous Single-Stream ms/batch < Lower Is Better Intel Arctm A770M DG2 . 14.25 |================================================ Neural Magic DeepSparse 1.3.2 Model: NLP Text Classification, DistilBERT mnli - Scenario: Asynchronous Multi-Stream items/sec > Higher Is Better Intel Arctm A770M DG2 . 66.00 |================================================ Neural Magic DeepSparse 1.3.2 Model: NLP Text Classification, DistilBERT mnli - Scenario: Asynchronous Multi-Stream ms/batch < Lower Is Better Intel Arctm A770M DG2 . 105.86 |=============================================== Neural Magic DeepSparse 1.3.2 Model: NLP Text Classification, DistilBERT mnli - Scenario: Synchronous Single-Stream items/sec > Higher Is Better Intel Arctm A770M DG2 . 52.11 |================================================ Neural Magic DeepSparse 1.3.2 Model: NLP Text Classification, DistilBERT mnli - Scenario: Synchronous Single-Stream ms/batch < Lower Is Better Intel Arctm A770M DG2 . 19.19 |================================================ Neural Magic DeepSparse 1.3.2 Model: CV Segmentation, 90% Pruned YOLACT Pruned - Scenario: Asynchronous Multi-Stream items/sec > Higher Is Better Intel Arctm A770M DG2 . 10.17 |================================================ Neural Magic DeepSparse 1.3.2 Model: CV Segmentation, 90% Pruned YOLACT Pruned - Scenario: Asynchronous Multi-Stream ms/batch < Lower Is Better Intel Arctm A770M DG2 . 684.00 |=============================================== Neural Magic DeepSparse 1.3.2 Model: CV Segmentation, 90% Pruned YOLACT Pruned - Scenario: Synchronous Single-Stream items/sec > Higher Is Better Intel Arctm A770M DG2 . 9.6843 |=============================================== Neural Magic DeepSparse 1.3.2 Model: CV Segmentation, 90% Pruned YOLACT Pruned - Scenario: Synchronous Single-Stream ms/batch < Lower Is Better Intel Arctm A770M DG2 . 103.25 |=============================================== Neural Magic DeepSparse 1.3.2 Model: NLP Text Classification, BERT base uncased SST2 - Scenario: Asynchronous Multi-Stream items/sec > Higher Is Better Intel Arctm A770M DG2 . 32.76 |================================================ Neural Magic DeepSparse 1.3.2 Model: NLP Text Classification, BERT base uncased SST2 - Scenario: Asynchronous Multi-Stream ms/batch < Lower Is Better Intel Arctm A770M DG2 . 212.59 |=============================================== Neural Magic DeepSparse 1.3.2 Model: NLP Text Classification, BERT base uncased SST2 - Scenario: Synchronous Single-Stream items/sec > Higher Is Better Intel Arctm A770M DG2 . 26.11 |================================================ Neural Magic DeepSparse 1.3.2 Model: NLP Text Classification, BERT base uncased SST2 - Scenario: Synchronous Single-Stream ms/batch < Lower Is Better Intel Arctm A770M DG2 . 38.29 |================================================ Neural Magic DeepSparse 1.3.2 Model: NLP Token Classification, BERT base uncased conll2003 - Scenario: Asynchronous Multi-Stream items/sec > Higher Is Better Intel Arctm A770M DG2 . 7.7507 |=============================================== Neural Magic DeepSparse 1.3.2 Model: NLP Token Classification, BERT base uncased conll2003 - Scenario: Asynchronous Multi-Stream ms/batch < Lower Is Better Intel Arctm A770M DG2 . 897.71 |=============================================== Neural Magic DeepSparse 1.3.2 Model: NLP Token Classification, BERT base uncased conll2003 - Scenario: Synchronous Single-Stream items/sec > Higher Is Better Intel Arctm A770M DG2 . 7.3345 |=============================================== Neural Magic DeepSparse 1.3.2 Model: NLP Token Classification, BERT base uncased conll2003 - Scenario: Synchronous Single-Stream ms/batch < Lower Is Better Intel Arctm A770M DG2 . 136.34 |=============================================== Numenta Anomaly Benchmark 1.1 Detector: KNN CAD Seconds < Lower Is Better Intel Arctm A770M DG2 . 189.84 |=============================================== Numenta Anomaly Benchmark 1.1 Detector: Relative Entropy Seconds < Lower Is Better Intel Arctm A770M DG2 . 14.04 |================================================ Numenta Anomaly Benchmark 1.1 Detector: Windowed Gaussian Seconds < Lower Is Better Intel Arctm A770M DG2 . 8.107 |================================================ Numenta Anomaly Benchmark 1.1 Detector: Earthgecko Skyline Seconds < Lower Is Better Intel Arctm A770M DG2 . 91.99 |================================================ Numenta Anomaly Benchmark 1.1 Detector: Bayesian Changepoint Seconds < Lower Is Better Intel Arctm A770M DG2 . 22.23 |================================================ Numenta Anomaly Benchmark 1.1 Detector: Contextual Anomaly Detector OSE Seconds < Lower Is Better Intel Arctm A770M DG2 . 33.23 |================================================ Numpy Benchmark Score > Higher Is Better Intel Arctm A770M DG2 . 568.41 |=============================================== oneDNN 3.1 Harness: IP Shapes 1D - Data Type: f32 - Engine: CPU ms < Lower Is Better Intel Arctm A770M DG2 . 3.94803 |============================================== oneDNN 3.1 Harness: IP Shapes 3D - Data Type: f32 - Engine: CPU ms < Lower Is Better Intel Arctm A770M DG2 . 11.30 |================================================ oneDNN 3.1 Harness: IP Shapes 1D - Data Type: u8s8f32 - Engine: CPU ms < Lower Is Better Intel Arctm A770M DG2 . 1.40659 |============================================== oneDNN 3.1 Harness: IP Shapes 3D - Data Type: u8s8f32 - Engine: CPU ms < Lower Is Better Intel Arctm A770M DG2 . 2.48025 |============================================== oneDNN 3.1 Harness: IP Shapes 1D - Data Type: bf16bf16bf16 - Engine: CPU ms < Lower Is Better oneDNN 3.1 Harness: IP Shapes 3D - Data Type: bf16bf16bf16 - Engine: CPU ms < Lower Is Better oneDNN 3.1 Harness: Convolution Batch Shapes Auto - Data Type: f32 - Engine: CPU ms < Lower Is Better Intel Arctm A770M DG2 . 15.87 |================================================ oneDNN 3.1 Harness: Deconvolution Batch shapes_1d - Data Type: f32 - Engine: CPU ms < Lower Is Better Intel Arctm A770M DG2 . 9.12010 |============================================== oneDNN 3.1 Harness: Deconvolution Batch shapes_3d - Data Type: f32 - Engine: CPU ms < Lower Is Better Intel Arctm A770M DG2 . 7.64277 |============================================== oneDNN 3.1 Harness: Convolution Batch Shapes Auto - Data Type: u8s8f32 - Engine: CPU ms < Lower Is Better Intel Arctm A770M DG2 . 15.62 |================================================ oneDNN 3.1 Harness: Deconvolution Batch shapes_1d - Data Type: u8s8f32 - Engine: CPU ms < Lower Is Better Intel Arctm A770M DG2 . 1.89282 |============================================== oneDNN 3.1 Harness: Deconvolution Batch shapes_3d - Data Type: u8s8f32 - Engine: CPU ms < Lower Is Better Intel Arctm A770M DG2 . 3.21314 |============================================== oneDNN 3.1 Harness: Recurrent Neural Network Training - Data Type: f32 - Engine: CPU ms < Lower Is Better Intel Arctm A770M DG2 . 4363.75 |============================================== oneDNN 3.1 Harness: Recurrent Neural Network Inference - Data Type: f32 - Engine: CPU ms < Lower Is Better Intel Arctm A770M DG2 . 2199.96 |============================================== oneDNN 3.1 Harness: Recurrent Neural Network Training - Data Type: u8s8f32 - Engine: CPU ms < Lower Is Better Intel Arctm A770M DG2 . 4377.60 |============================================== oneDNN 3.1 Harness: Convolution Batch Shapes Auto - Data Type: bf16bf16bf16 - Engine: CPU ms < Lower Is Better oneDNN 3.1 Harness: Deconvolution Batch shapes_1d - Data Type: bf16bf16bf16 - Engine: CPU ms < Lower Is Better oneDNN 3.1 Harness: Deconvolution Batch shapes_3d - Data Type: bf16bf16bf16 - Engine: CPU ms < Lower Is Better oneDNN 3.1 Harness: Recurrent Neural Network Inference - Data Type: u8s8f32 - Engine: CPU ms < Lower Is Better Intel Arctm A770M DG2 . 2190.97 |============================================== oneDNN 3.1 Harness: Recurrent Neural Network Training - Data Type: bf16bf16bf16 - Engine: CPU ms < Lower Is Better Intel Arctm A770M DG2 . 4387.13 |============================================== oneDNN 3.1 Harness: Recurrent Neural Network Inference - Data Type: bf16bf16bf16 - Engine: CPU ms < Lower Is Better Intel Arctm A770M DG2 . 2198.98 |============================================== ONNX Runtime 1.14 Model: GPT-2 - Device: CPU - Executor: Parallel Inferences Per Second > Higher Is Better ONNX Runtime 1.14 Model: GPT-2 - Device: CPU - Executor: Standard Inferences Per Second > Higher Is Better ONNX Runtime 1.14 Model: yolov4 - Device: CPU - Executor: Parallel Inferences Per Second > Higher Is Better ONNX Runtime 1.14 Model: yolov4 - Device: CPU - Executor: Standard Inferences Per Second > Higher Is Better ONNX Runtime 1.14 Model: bertsquad-12 - Device: CPU - Executor: Parallel Inferences Per Second > Higher Is Better ONNX Runtime 1.14 Model: bertsquad-12 - Device: CPU - Executor: Standard Inferences Per Second > Higher Is Better ONNX Runtime 1.14 Model: CaffeNet 12-int8 - Device: CPU - Executor: Parallel Inferences Per Second > Higher Is Better ONNX Runtime 1.14 Model: CaffeNet 12-int8 - Device: CPU - Executor: Standard Inferences Per Second > Higher Is Better ONNX Runtime 1.14 Model: fcn-resnet101-11 - Device: CPU - Executor: Parallel Inferences Per Second > Higher Is Better ONNX Runtime 1.14 Model: fcn-resnet101-11 - Device: CPU - Executor: Standard Inferences Per Second > Higher Is Better ONNX Runtime 1.14 Model: ArcFace ResNet-100 - Device: CPU - Executor: Parallel Inferences Per Second > Higher Is Better ONNX Runtime 1.14 Model: ArcFace ResNet-100 - Device: CPU - Executor: Standard Inferences Per Second > Higher Is Better ONNX Runtime 1.14 Model: ResNet50 v1-12-int8 - Device: CPU - Executor: Parallel Inferences Per Second > Higher Is Better ONNX Runtime 1.14 Model: ResNet50 v1-12-int8 - Device: CPU - Executor: Standard Inferences Per Second > Higher Is Better ONNX Runtime 1.14 Model: super-resolution-10 - Device: CPU - Executor: Parallel Inferences Per Second > Higher Is Better ONNX Runtime 1.14 Model: super-resolution-10 - Device: CPU - Executor: Standard Inferences Per Second > Higher Is Better ONNX Runtime 1.14 Model: Faster R-CNN R-50-FPN-int8 - Device: CPU - Executor: Parallel Inferences Per Second > Higher Is Better ONNX Runtime 1.14 Model: Faster R-CNN R-50-FPN-int8 - Device: CPU - Executor: Standard Inferences Per Second > Higher Is Better OpenCV 4.7 Test: DNN - Deep Neural Network ms < Lower Is Better Intel Arctm A770M DG2 . 33936 |================================================ OpenVINO 2022.3 Model: Face Detection FP16 - Device: CPU FPS > Higher Is Better Intel Arctm A770M DG2 . 2.51 |================================================= OpenVINO 2022.3 Model: Face Detection FP16 - Device: CPU ms < Lower Is Better Intel Arctm A770M DG2 . 2364.50 |============================================== OpenVINO 2022.3 Model: Person Detection FP16 - Device: CPU FPS > Higher Is Better Intel Arctm A770M DG2 . 1.57 |================================================= OpenVINO 2022.3 Model: Person Detection FP16 - Device: CPU ms < Lower Is Better Intel Arctm A770M DG2 . 3708.47 |============================================== OpenVINO 2022.3 Model: Person Detection FP32 - Device: CPU FPS > Higher Is Better Intel Arctm A770M DG2 . 1.56 |================================================= OpenVINO 2022.3 Model: Person Detection FP32 - Device: CPU ms < Lower Is Better Intel Arctm A770M DG2 . 3747.64 |============================================== OpenVINO 2022.3 Model: Vehicle Detection FP16 - Device: CPU FPS > Higher Is Better Intel Arctm A770M DG2 . 174.86 |=============================================== OpenVINO 2022.3 Model: Vehicle Detection FP16 - Device: CPU ms < Lower Is Better Intel Arctm A770M DG2 . 34.24 |================================================ OpenVINO 2022.3 Model: Face Detection FP16-INT8 - Device: CPU FPS > Higher Is Better Intel Arctm A770M DG2 . 8.96 |================================================= OpenVINO 2022.3 Model: Face Detection FP16-INT8 - Device: CPU ms < Lower Is Better Intel Arctm A770M DG2 . 662.97 |=============================================== OpenVINO 2022.3 Model: Vehicle Detection FP16-INT8 - Device: CPU FPS > Higher Is Better Intel Arctm A770M DG2 . 463.39 |=============================================== OpenVINO 2022.3 Model: Vehicle Detection FP16-INT8 - Device: CPU ms < Lower Is Better Intel Arctm A770M DG2 . 12.90 |================================================ OpenVINO 2022.3 Model: Weld Porosity Detection FP16 - Device: CPU FPS > Higher Is Better Intel Arctm A770M DG2 . 264.48 |=============================================== OpenVINO 2022.3 Model: Weld Porosity Detection FP16 - Device: CPU ms < Lower Is Better Intel Arctm A770M DG2 . 57.49 |================================================ OpenVINO 2022.3 Model: Machine Translation EN To DE FP16 - Device: CPU FPS > Higher Is Better Intel Arctm A770M DG2 . 28.49 |================================================ OpenVINO 2022.3 Model: Machine Translation EN To DE FP16 - Device: CPU ms < Lower Is Better Intel Arctm A770M DG2 . 210.29 |=============================================== OpenVINO 2022.3 Model: Weld Porosity Detection FP16-INT8 - Device: CPU FPS > Higher Is Better Intel Arctm A770M DG2 . 929.62 |=============================================== OpenVINO 2022.3 Model: Weld Porosity Detection FP16-INT8 - Device: CPU ms < Lower Is Better Intel Arctm A770M DG2 . 21.50 |================================================ OpenVINO 2022.3 Model: Person Vehicle Bike Detection FP16 - Device: CPU FPS > Higher Is Better Intel Arctm A770M DG2 . 371.68 |=============================================== OpenVINO 2022.3 Model: Person Vehicle Bike Detection FP16 - Device: CPU ms < Lower Is Better Intel Arctm A770M DG2 . 16.10 |================================================ OpenVINO 2022.3 Model: Age Gender Recognition Retail 0013 FP16 - Device: CPU FPS > Higher Is Better Intel Arctm A770M DG2 . 8722.19 |============================================== OpenVINO 2022.3 Model: Age Gender Recognition Retail 0013 FP16 - Device: CPU ms < Lower Is Better Intel Arctm A770M DG2 . 2.29 |================================================= OpenVINO 2022.3 Model: Age Gender Recognition Retail 0013 FP16-INT8 - Device: CPU FPS > Higher Is Better Intel Arctm A770M DG2 . 9443.09 |============================================== OpenVINO 2022.3 Model: Age Gender Recognition Retail 0013 FP16-INT8 - Device: CPU ms < Lower Is Better Intel Arctm A770M DG2 . 2.11 |================================================= PlaidML FP16: No - Mode: Inference - Network: VGG16 - Device: CPU Examples Per Second > Higher Is Better PlaidML FP16: No - Mode: Inference - Network: ResNet 50 - Device: CPU Examples Per Second > Higher Is Better R Benchmark Seconds < Lower Is Better Intel Arctm A770M DG2 . 0.1057 |=============================================== RNNoise 2020-06-28 Seconds < Lower Is Better Intel Arctm A770M DG2 . 14.77 |================================================ Scikit-Learn 1.2.2 Benchmark: GLM Seconds < Lower Is Better Scikit-Learn 1.2.2 Benchmark: SAGA Seconds < Lower Is Better Scikit-Learn 1.2.2 Benchmark: Tree Seconds < Lower Is Better Scikit-Learn 1.2.2 Benchmark: Lasso Seconds < Lower Is Better Scikit-Learn 1.2.2 Benchmark: Glmnet Seconds < Lower Is Better Scikit-Learn 1.2.2 Benchmark: Sparsify Seconds < Lower Is Better Scikit-Learn 1.2.2 Benchmark: Plot Ward Seconds < Lower Is Better Scikit-Learn 1.2.2 Benchmark: MNIST Dataset Seconds < Lower Is Better Scikit-Learn 1.2.2 Benchmark: Plot Neighbors Seconds < Lower Is Better Scikit-Learn 1.2.2 Benchmark: SGD Regression Seconds < Lower Is Better Scikit-Learn 1.2.2 Benchmark: SGDOneClassSVM Seconds < Lower Is Better Scikit-Learn 1.2.2 Benchmark: Plot Lasso Path Seconds < Lower Is Better Scikit-Learn 1.2.2 Benchmark: Isolation Forest Seconds < Lower Is Better Scikit-Learn 1.2.2 Benchmark: Plot Fast KMeans Seconds < Lower Is Better Scikit-Learn 1.2.2 Benchmark: Text Vectorizers Seconds < Lower Is Better Scikit-Learn 1.2.2 Benchmark: Plot Hierarchical Seconds < Lower Is Better Scikit-Learn 1.2.2 Benchmark: Plot OMP vs. LARS Seconds < Lower Is Better Scikit-Learn 1.2.2 Benchmark: Feature Expansions Seconds < Lower Is Better Scikit-Learn 1.2.2 Benchmark: LocalOutlierFactor Seconds < Lower Is Better Scikit-Learn 1.2.2 Benchmark: TSNE MNIST Dataset Seconds < Lower Is Better Scikit-Learn 1.2.2 Benchmark: Isotonic / Logistic Seconds < Lower Is Better Scikit-Learn 1.2.2 Benchmark: Plot Incremental PCA Seconds < Lower Is Better Scikit-Learn 1.2.2 Benchmark: Hist Gradient Boosting Seconds < Lower Is Better Scikit-Learn 1.2.2 Benchmark: Plot Parallel Pairwise Seconds < Lower Is Better Scikit-Learn 1.2.2 Benchmark: Isotonic / Pathological Seconds < Lower Is Better Scikit-Learn 1.2.2 Benchmark: RCV1 Logreg Convergencet Seconds < Lower Is Better Scikit-Learn 1.2.2 Benchmark: Sample Without Replacement Seconds < Lower Is Better Scikit-Learn 1.2.2 Benchmark: Covertype Dataset Benchmark Seconds < Lower Is Better Scikit-Learn 1.2.2 Benchmark: Hist Gradient Boosting Adult Seconds < Lower Is Better Scikit-Learn 1.2.2 Benchmark: Isotonic / Perturbed Logarithm Seconds < Lower Is Better Scikit-Learn 1.2.2 Benchmark: Hist Gradient Boosting Threading Seconds < Lower Is Better Scikit-Learn 1.2.2 Benchmark: Plot Singular Value Decomposition Seconds < Lower Is Better Scikit-Learn 1.2.2 Benchmark: Hist Gradient Boosting Higgs Boson Seconds < Lower Is Better Scikit-Learn 1.2.2 Benchmark: 20 Newsgroups / Logistic Regression Seconds < Lower Is Better Scikit-Learn 1.2.2 Benchmark: Plot Polynomial Kernel Approximation Seconds < Lower Is Better Scikit-Learn 1.2.2 Benchmark: Plot Non-Negative Matrix Factorization Seconds < Lower Is Better Scikit-Learn 1.2.2 Benchmark: Hist Gradient Boosting Categorical Only Seconds < Lower Is Better Scikit-Learn 1.2.2 Benchmark: Kernel PCA Solvers / Time vs. N Samples Seconds < Lower Is Better Scikit-Learn 1.2.2 Benchmark: Kernel PCA Solvers / Time vs. N Components Seconds < Lower Is Better Scikit-Learn 1.2.2 Benchmark: Sparse Random Projections / 100 Iterations Seconds < Lower Is Better SHOC Scalable HeterOgeneous Computing 2020-04-17 Target: OpenCL - Benchmark: S3D GFLOPS > Higher Is Better Intel Arctm A770M DG2 . 74.68 |================================================ SHOC Scalable HeterOgeneous Computing 2020-04-17 Target: OpenCL - Benchmark: Triad GB/s > Higher Is Better Intel Arctm A770M DG2 . 11.00 |================================================ SHOC Scalable HeterOgeneous Computing 2020-04-17 Target: OpenCL - Benchmark: FFT SP GFLOPS > Higher Is Better Intel Arctm A770M DG2 . 1253.70 |============================================== SHOC Scalable HeterOgeneous Computing 2020-04-17 Target: OpenCL - Benchmark: MD5 Hash GHash/s > Higher Is Better Intel Arctm A770M DG2 . 18.26 |================================================ SHOC Scalable HeterOgeneous Computing 2020-04-17 Target: OpenCL - Benchmark: Reduction GB/s > Higher Is Better Intel Arctm A770M DG2 . 275.03 |=============================================== SHOC Scalable HeterOgeneous Computing 2020-04-17 Target: OpenCL - Benchmark: GEMM SGEMM_N GFLOPS > Higher Is Better Intel Arctm A770M DG2 . 2186.71 |============================================== SHOC Scalable HeterOgeneous Computing 2020-04-17 Target: OpenCL - Benchmark: Max SP Flops GFLOPS > Higher Is Better Intel Arctm A770M DG2 . 1439586 |============================================== SHOC Scalable HeterOgeneous Computing 2020-04-17 Target: OpenCL - Benchmark: Bus Speed Download GB/s > Higher Is Better Intel Arctm A770M DG2 . 11.76 |================================================ SHOC Scalable HeterOgeneous Computing 2020-04-17 Target: OpenCL - Benchmark: Bus Speed Readback GB/s > Higher Is Better Intel Arctm A770M DG2 . 11.34 |================================================ SHOC Scalable HeterOgeneous Computing 2020-04-17 Target: OpenCL - Benchmark: Texture Read Bandwidth GB/s > Higher Is Better Intel Arctm A770M DG2 . 1124.02 |============================================== spaCy 3.4.1 Model: en_core_web_lg tokens/sec > Higher Is Better Intel Arctm A770M DG2 . 16538 |================================================ spaCy 3.4.1 Model: en_core_web_trf tokens/sec > Higher Is Better Intel Arctm A770M DG2 . 1232 |================================================= TensorFlow 2.12 Device: CPU - Batch Size: 16 - Model: VGG-16 images/sec > Higher Is Better Intel Arctm A770M DG2 . 5.36 |================================================= TensorFlow 2.12 Device: CPU - Batch Size: 32 - Model: VGG-16 images/sec > Higher Is Better Intel Arctm A770M DG2 . 5.53 |================================================= TensorFlow 2.12 Device: CPU - Batch Size: 64 - Model: VGG-16 images/sec > Higher Is Better Intel Arctm A770M DG2 . 5.54 |================================================= TensorFlow 2.12 Device: CPU - Batch Size: 16 - Model: AlexNet images/sec > Higher Is Better Intel Arctm A770M DG2 . 76.22 |================================================ TensorFlow 2.12 Device: CPU - Batch Size: 256 - Model: VGG-16 images/sec > Higher Is Better TensorFlow 2.12 Device: CPU - Batch Size: 32 - Model: AlexNet images/sec > Higher Is Better Intel Arctm A770M DG2 . 100.35 |=============================================== TensorFlow 2.12 Device: CPU - Batch Size: 512 - Model: VGG-16 images/sec > Higher Is Better TensorFlow 2.12 Device: CPU - Batch Size: 64 - Model: AlexNet images/sec > Higher Is Better Intel Arctm A770M DG2 . 118.53 |=============================================== TensorFlow 2.12 Device: CPU - Batch Size: 256 - Model: AlexNet images/sec > Higher Is Better Intel Arctm A770M DG2 . 144.49 |=============================================== TensorFlow 2.12 Device: CPU - Batch Size: 512 - Model: AlexNet images/sec > Higher Is Better Intel Arctm A770M DG2 . 154.08 |=============================================== TensorFlow 2.12 Device: CPU - Batch Size: 16 - Model: GoogLeNet images/sec > Higher Is Better Intel Arctm A770M DG2 . 55.09 |================================================ TensorFlow 2.12 Device: CPU - Batch Size: 16 - Model: ResNet-50 images/sec > Higher Is Better Intel Arctm A770M DG2 . 16.27 |================================================ TensorFlow 2.12 Device: CPU - Batch Size: 32 - Model: GoogLeNet images/sec > Higher Is Better Intel Arctm A770M DG2 . 54.91 |================================================ TensorFlow 2.12 Device: CPU - Batch Size: 32 - Model: ResNet-50 images/sec > Higher Is Better Intel Arctm A770M DG2 . 16.75 |================================================ TensorFlow 2.12 Device: CPU - Batch Size: 64 - Model: GoogLeNet images/sec > Higher Is Better Intel Arctm A770M DG2 . 56.82 |================================================ TensorFlow 2.12 Device: CPU - Batch Size: 64 - Model: ResNet-50 images/sec > Higher Is Better Intel Arctm A770M DG2 . 16.99 |================================================ TensorFlow 2.12 Device: CPU - Batch Size: 256 - Model: GoogLeNet images/sec > Higher Is Better Intel Arctm A770M DG2 . 58.43 |================================================ TensorFlow 2.12 Device: CPU - Batch Size: 256 - Model: ResNet-50 images/sec > Higher Is Better TensorFlow 2.12 Device: CPU - Batch Size: 512 - Model: GoogLeNet images/sec > Higher Is Better Intel Arctm A770M DG2 . 59.45 |================================================ TensorFlow 2.12 Device: CPU - Batch Size: 512 - Model: ResNet-50 images/sec > Higher Is Better TensorFlow Lite 2022-05-18 Model: SqueezeNet Microseconds < Lower Is Better Intel Arctm A770M DG2 . 2642.68 |============================================== TensorFlow Lite 2022-05-18 Model: Inception V4 Microseconds < Lower Is Better Intel Arctm A770M DG2 . 37029.6 |============================================== TensorFlow Lite 2022-05-18 Model: NASNet Mobile Microseconds < Lower Is Better Intel Arctm A770M DG2 . 114426.0 |============================================= TensorFlow Lite 2022-05-18 Model: Mobilenet Float Microseconds < Lower Is Better Intel Arctm A770M DG2 . 1916.03 |============================================== TensorFlow Lite 2022-05-18 Model: Mobilenet Quant Microseconds < Lower Is Better Intel Arctm A770M DG2 . 3248.53 |============================================== TensorFlow Lite 2022-05-18 Model: Inception ResNet V2 Microseconds < Lower Is Better Intel Arctm A770M DG2 . 129929.4 |============================================= TNN 0.3 Target: CPU - Model: DenseNet ms < Lower Is Better Intel Arctm A770M DG2 . 1992.47 |============================================== TNN 0.3 Target: CPU - Model: MobileNet v2 ms < Lower Is Better Intel Arctm A770M DG2 . 191.61 |=============================================== TNN 0.3 Target: CPU - Model: SqueezeNet v2 ms < Lower Is Better Intel Arctm A770M DG2 . 43.23 |================================================ TNN 0.3 Target: CPU - Model: SqueezeNet v1.1 ms < Lower Is Better Intel Arctm A770M DG2 . 149.23 |===============================================