TestRunNewCMake Intel Pentium Gold G6400 testing with a ASRock H510M-HDV/M.2 SE (P1.60 BIOS) and Intel UHD 610 CML GT1 3GB on Ubuntu 20.04 via the Phoronix Test Suite. Intel UHD 610 CML GT1: Processor: Intel Pentium Gold G6400 @ 4.00GHz (2 Cores / 4 Threads), Motherboard: ASRock H510M-HDV/M.2 SE (P1.60 BIOS), Chipset: Intel Comet Lake PCH, Memory: 3584MB, Disk: 1000GB Western Digital WDS100T2B0A, Graphics: Intel UHD 610 CML GT1 3GB (1050MHz), Audio: Realtek ALC897, Monitor: G185BGEL01, Network: Realtek RTL8111/8168/8411 OS: Ubuntu 20.04, Kernel: 5.15.0-86-generic (x86_64), Desktop: GNOME Shell 3.36.9, Display Server: X Server 1.20.13, OpenGL: 4.6 Mesa 21.2.6, Vulkan: 1.2.182, Compiler: GCC 9.4.0, File-System: ext4, Screen Resolution: 1368x768 Whisper.cpp 1.4 Model: ggml-medium.en - Input: 2016 State of the Union Seconds < Lower Is Better Intel UHD 610 CML GT1 . 41069.32 |============================================= Whisper.cpp 1.4 Model: ggml-small.en - Input: 2016 State of the Union Seconds < Lower Is Better Intel UHD 610 CML GT1 . 11708.17 |============================================= Scikit-Learn 1.2.2 Benchmark: SGDOneClassSVM Seconds < Lower Is Better Scikit-Learn 1.2.2 Benchmark: Isolation Forest Seconds < Lower Is Better Scikit-Learn 1.2.2 Benchmark: Sparse Random Projections / 100 Iterations Seconds < Lower Is Better Intel UHD 610 CML GT1 . 3104.85 |============================================== Scikit-Learn 1.2.2 Benchmark: Plot Fast KMeans Seconds < Lower Is Better Whisper.cpp 1.4 Model: ggml-base.en - Input: 2016 State of the Union Seconds < Lower Is Better Intel UHD 610 CML GT1 . 3200.85 |============================================== Caffe 2020-02-13 Model: GoogleNet - Acceleration: CPU - Iterations: 1000 Milli-Seconds < Lower Is Better Intel UHD 610 CML GT1 . 2142790 |============================================== Scikit-Learn 1.2.2 Benchmark: GLM Seconds < Lower Is Better Intel UHD 610 CML GT1 . 1129.13 |============================================== Scikit-Learn 1.2.2 Benchmark: Lasso Seconds < Lower Is Better Intel UHD 610 CML GT1 . 1084.14 |============================================== Scikit-Learn 1.2.2 Benchmark: SAGA Seconds < Lower Is Better Intel UHD 610 CML GT1 . 1078.49 |============================================== PlaidML FP16: No - Mode: Inference - Network: VGG16 - Device: CPU FPS > Higher Is Better Intel UHD 610 CML GT1 . 1.59 |================================================= Scikit-Learn 1.2.2 Benchmark: Kernel PCA Solvers / Time vs. N Components Seconds < Lower Is Better Intel UHD 610 CML GT1 . 327.83 |=============================================== Scikit-Learn 1.2.2 Benchmark: TSNE MNIST Dataset Seconds < Lower Is Better Intel UHD 610 CML GT1 . 807.65 |=============================================== Caffe 2020-02-13 Model: AlexNet - Acceleration: CPU - Iterations: 1000 Milli-Seconds < Lower Is Better Intel UHD 610 CML GT1 . 986440 |=============================================== PlaidML FP16: No - Mode: Inference - Network: ResNet 50 - Device: CPU FPS > Higher Is Better Intel UHD 610 CML GT1 . 2.40 |================================================= Mobile Neural Network 2.1 Model: inception-v3 ms < Lower Is Better Intel UHD 610 CML GT1 . 158.59 |=============================================== Mobile Neural Network 2.1 Model: mobilenet-v1-1.0 ms < Lower Is Better Intel UHD 610 CML GT1 . 20.61 |================================================ Mobile Neural Network 2.1 Model: MobileNetV2_224 ms < Lower Is Better Intel UHD 610 CML GT1 . 13.32 |================================================ Mobile Neural Network 2.1 Model: SqueezeNetV1.0 ms < Lower Is Better Intel UHD 610 CML GT1 . 22.63 |================================================ Mobile Neural Network 2.1 Model: resnet-v2-50 ms < Lower Is Better Intel UHD 610 CML GT1 . 119.02 |=============================================== Mobile Neural Network 2.1 Model: squeezenetv1.1 ms < Lower Is Better Intel UHD 610 CML GT1 . 12.23 |================================================ Mobile Neural Network 2.1 Model: mobilenetV3 ms < Lower Is Better Intel UHD 610 CML GT1 . 3.861 |================================================ Mobile Neural Network 2.1 Model: nasnet ms < Lower Is Better Intel UHD 610 CML GT1 . 30.01 |================================================ Scikit-Learn 1.2.2 Benchmark: Covertype Dataset Benchmark Seconds < Lower Is Better Intel UHD 610 CML GT1 . 584.05 |=============================================== Numenta Anomaly Benchmark 1.1 Detector: KNN CAD Seconds < Lower Is Better Intel UHD 610 CML GT1 . 726.70 |=============================================== Scikit-Learn 1.2.2 Benchmark: Plot Lasso Path Seconds < Lower Is Better Intel UHD 610 CML GT1 . 476.23 |=============================================== Scikit-Learn 1.2.2 Benchmark: Kernel PCA Solvers / Time vs. N Samples Seconds < Lower Is Better Intel UHD 610 CML GT1 . 463.43 |=============================================== Scikit-Learn 1.2.2 Benchmark: LocalOutlierFactor Seconds < Lower Is Better Intel UHD 610 CML GT1 . 456.45 |=============================================== Scikit-Learn 1.2.2 Benchmark: Hist Gradient Boosting Threading Seconds < Lower Is Better Intel UHD 610 CML GT1 . 445.70 |=============================================== NCNN 20230517 Target: Vulkan GPU - Model: FastestDet ms < Lower Is Better Intel UHD 610 CML GT1 . 10.32 |================================================ NCNN 20230517 Target: Vulkan GPU - Model: vision_transformer ms < Lower Is Better Intel UHD 610 CML GT1 . 922.08 |=============================================== NCNN 20230517 Target: Vulkan GPU - Model: regnety_400m ms < Lower Is Better Intel UHD 610 CML GT1 . 23.61 |================================================ NCNN 20230517 Target: Vulkan GPU - Model: squeezenet_ssd ms < Lower Is Better Intel UHD 610 CML GT1 . 38.59 |================================================ NCNN 20230517 Target: Vulkan GPU - Model: yolov4-tiny ms < Lower Is Better Intel UHD 610 CML GT1 . 93.72 |================================================ NCNN 20230517 Target: Vulkan GPU - Model: resnet50 ms < Lower Is Better Intel UHD 610 CML GT1 . 125.65 |=============================================== NCNN 20230517 Target: Vulkan GPU - Model: alexnet ms < Lower Is Better Intel UHD 610 CML GT1 . 37.88 |================================================ NCNN 20230517 Target: Vulkan GPU - Model: resnet18 ms < Lower Is Better Intel UHD 610 CML GT1 . 46.11 |================================================ NCNN 20230517 Target: Vulkan GPU - Model: vgg16 ms < Lower Is Better Intel UHD 610 CML GT1 . 269.82 |=============================================== NCNN 20230517 Target: Vulkan GPU - Model: googlenet ms < Lower Is Better Intel UHD 610 CML GT1 . 55.09 |================================================ NCNN 20230517 Target: Vulkan GPU - Model: blazeface ms < Lower Is Better Intel UHD 610 CML GT1 . 2.45 |================================================= NCNN 20230517 Target: Vulkan GPU - Model: efficientnet-b0 ms < Lower Is Better Intel UHD 610 CML GT1 . 29.19 |================================================ NCNN 20230517 Target: Vulkan GPU - Model: mnasnet ms < Lower Is Better Intel UHD 610 CML GT1 . 17.45 |================================================ NCNN 20230517 Target: Vulkan GPU - Model: shufflenet-v2 ms < Lower Is Better Intel UHD 610 CML GT1 . 8.63 |================================================= NCNN 20230517 Target: Vulkan GPU-v3-v3 - Model: mobilenet-v3 ms < Lower Is Better Intel UHD 610 CML GT1 . 15.15 |================================================ NCNN 20230517 Target: Vulkan GPU-v2-v2 - Model: mobilenet-v2 ms < Lower Is Better Intel UHD 610 CML GT1 . 20.77 |================================================ NCNN 20230517 Target: Vulkan GPU - Model: mobilenet ms < Lower Is Better Intel UHD 610 CML GT1 . 75.75 |================================================ NCNN 20230517 Target: CPU - Model: FastestDet ms < Lower Is Better Intel UHD 610 CML GT1 . 10.31 |================================================ NCNN 20230517 Target: CPU - Model: vision_transformer ms < Lower Is Better Intel UHD 610 CML GT1 . 922.27 |=============================================== NCNN 20230517 Target: CPU - Model: regnety_400m ms < Lower Is Better Intel UHD 610 CML GT1 . 23.14 |================================================ NCNN 20230517 Target: CPU - Model: squeezenet_ssd ms < Lower Is Better Intel UHD 610 CML GT1 . 38.64 |================================================ NCNN 20230517 Target: CPU - Model: yolov4-tiny ms < Lower Is Better Intel UHD 610 CML GT1 . 93.85 |================================================ NCNN 20230517 Target: CPU - Model: resnet50 ms < Lower Is Better Intel UHD 610 CML GT1 . 125.87 |=============================================== NCNN 20230517 Target: CPU - Model: alexnet ms < Lower Is Better Intel UHD 610 CML GT1 . 37.89 |================================================ NCNN 20230517 Target: CPU - Model: resnet18 ms < Lower Is Better Intel UHD 610 CML GT1 . 46.04 |================================================ NCNN 20230517 Target: CPU - Model: vgg16 ms < Lower Is Better Intel UHD 610 CML GT1 . 268.68 |=============================================== NCNN 20230517 Target: CPU - Model: googlenet ms < Lower Is Better Intel UHD 610 CML GT1 . 54.88 |================================================ NCNN 20230517 Target: CPU - Model: blazeface ms < Lower Is Better Intel UHD 610 CML GT1 . 2.46 |================================================= NCNN 20230517 Target: CPU - Model: efficientnet-b0 ms < Lower Is Better Intel UHD 610 CML GT1 . 29.21 |================================================ NCNN 20230517 Target: CPU - Model: mnasnet ms < Lower Is Better Intel UHD 610 CML GT1 . 17.42 |================================================ NCNN 20230517 Target: CPU - Model: shufflenet-v2 ms < Lower Is Better Intel UHD 610 CML GT1 . 8.65 |================================================= NCNN 20230517 Target: CPU-v3-v3 - Model: mobilenet-v3 ms < Lower Is Better Intel UHD 610 CML GT1 . 15.09 |================================================ NCNN 20230517 Target: CPU-v2-v2 - Model: mobilenet-v2 ms < Lower Is Better Intel UHD 610 CML GT1 . 20.69 |================================================ NCNN 20230517 Target: CPU - Model: mobilenet ms < Lower Is Better Intel UHD 610 CML GT1 . 75.80 |================================================ ONNX Runtime 1.14 Model: fcn-resnet101-11 - Device: CPU - Executor: Parallel Inference Time Cost (ms) < Lower Is Better Intel UHD 610 CML GT1 . 13223.7 |============================================== ONNX Runtime 1.14 Model: fcn-resnet101-11 - Device: CPU - Executor: Parallel Inferences Per Second > Higher Is Better Intel UHD 610 CML GT1 . 0.0793033 |============================================ Numenta Anomaly Benchmark 1.1 Detector: Earthgecko Skyline Seconds < Lower Is Better Intel UHD 610 CML GT1 . 534.68 |=============================================== Scikit-Learn 1.2.2 Benchmark: Plot Singular Value Decomposition Seconds < Lower Is Better Intel UHD 610 CML GT1 . 357.66 |=============================================== ONNX Runtime 1.14 Model: Faster R-CNN R-50-FPN-int8 - Device: CPU - Executor: Parallel Inference Time Cost (ms) < Lower Is Better Intel UHD 610 CML GT1 . 1090.13 |============================================== ONNX Runtime 1.14 Model: Faster R-CNN R-50-FPN-int8 - Device: CPU - Executor: Parallel Inferences Per Second > Higher Is Better Intel UHD 610 CML GT1 . 0.933197 |============================================= ONNX Runtime 1.14 Model: bertsquad-12 - Device: CPU - Executor: Standard Inference Time Cost (ms) < Lower Is Better Intel UHD 610 CML GT1 . 851.92 |=============================================== ONNX Runtime 1.14 Model: bertsquad-12 - Device: CPU - Executor: Standard Inferences Per Second > Higher Is Better Intel UHD 610 CML GT1 . 1.17517 |============================================== ONNX Runtime 1.14 Model: super-resolution-10 - Device: CPU - Executor: Parallel Inference Time Cost (ms) < Lower Is Better Intel UHD 610 CML GT1 . 135.21 |=============================================== ONNX Runtime 1.14 Model: super-resolution-10 - Device: CPU - Executor: Parallel Inferences Per Second > Higher Is Better Intel UHD 610 CML GT1 . 7.64505 |============================================== Caffe 2020-02-13 Model: GoogleNet - Acceleration: CPU - Iterations: 200 Milli-Seconds < Lower Is Better Intel UHD 610 CML GT1 . 429063 |=============================================== Scikit-Learn 1.2.2 Benchmark: Plot Polynomial Kernel Approximation Seconds < Lower Is Better Intel UHD 610 CML GT1 . 301.90 |=============================================== Mlpack Benchmark Benchmark: scikit_qda Seconds < Lower Is Better TNN 0.3 Target: CPU - Model: DenseNet ms < Lower Is Better Intel UHD 610 CML GT1 . 5394.68 |============================================== Mlpack Benchmark Benchmark: scikit_linearridgeregression Seconds < Lower Is Better Scikit-Learn 1.2.2 Benchmark: Plot Hierarchical Seconds < Lower Is Better Intel UHD 610 CML GT1 . 278.83 |=============================================== LeelaChessZero 0.28 Backend: BLAS Nodes Per Second > Higher Is Better Intel UHD 610 CML GT1 . 137 |================================================== ONNX Runtime 1.14 Model: GPT-2 - Device: CPU - Executor: Parallel Inference Time Cost (ms) < Lower Is Better Intel UHD 610 CML GT1 . 35.98 |================================================ ONNX Runtime 1.14 Model: GPT-2 - Device: CPU - Executor: Parallel Inferences Per Second > Higher Is Better Intel UHD 610 CML GT1 . 27.88 |================================================ ONNX Runtime 1.14 Model: super-resolution-10 - Device: CPU - Executor: Standard Inference Time Cost (ms) < Lower Is Better Intel UHD 610 CML GT1 . 112.53 |=============================================== ONNX Runtime 1.14 Model: super-resolution-10 - Device: CPU - Executor: Standard Inferences Per Second > Higher Is Better Intel UHD 610 CML GT1 . 8.91625 |============================================== Scikit-Learn 1.2.2 Benchmark: Plot Neighbors Seconds < Lower Is Better Intel UHD 610 CML GT1 . 267.82 |=============================================== Scikit-Learn 1.2.2 Benchmark: Plot OMP vs. LARS Seconds < Lower Is Better Intel UHD 610 CML GT1 . 253.19 |=============================================== ONNX Runtime 1.14 Model: CaffeNet 12-int8 - Device: CPU - Executor: Standard Inference Time Cost (ms) < Lower Is Better Intel UHD 610 CML GT1 . 25.73 |================================================ ONNX Runtime 1.14 Model: CaffeNet 12-int8 - Device: CPU - Executor: Standard Inferences Per Second > Higher Is Better Intel UHD 610 CML GT1 . 38.89 |================================================ Scikit-Learn 1.2.2 Benchmark: Isotonic / Pathological Seconds < Lower Is Better Scikit-Learn 1.2.2 Benchmark: Isotonic / Logistic Seconds < Lower Is Better Scikit-Learn 1.2.2 Benchmark: SGD Regression Seconds < Lower Is Better Intel UHD 610 CML GT1 . 228.72 |=============================================== oneDNN 3.3 Harness: Recurrent Neural Network Training - Data Type: f32 - Engine: CPU ms < Lower Is Better Intel UHD 610 CML GT1 . 42198.7 |============================================== oneDNN 3.3 Harness: Recurrent Neural Network Training - Data Type: u8s8f32 - Engine: CPU ms < Lower Is Better Intel UHD 610 CML GT1 . 42203.1 |============================================== oneDNN 3.3 Harness: Recurrent Neural Network Training - Data Type: bf16bf16bf16 - Engine: CPU ms < Lower Is Better Intel UHD 610 CML GT1 . 42196.3 |============================================== Scikit-Learn 1.2.2 Benchmark: Feature Expansions Seconds < Lower Is Better Intel UHD 610 CML GT1 . 211.25 |=============================================== Scikit-Learn 1.2.2 Benchmark: Hist Gradient Boosting Seconds < Lower Is Better Intel UHD 610 CML GT1 . 199.38 |=============================================== Scikit-Learn 1.2.2 Benchmark: Hist Gradient Boosting Higgs Boson Seconds < Lower Is Better Intel UHD 610 CML GT1 . 194.07 |=============================================== Scikit-Learn 1.2.2 Benchmark: Tree Seconds < Lower Is Better Intel UHD 610 CML GT1 . 48.19 |================================================ Scikit-Learn 1.2.2 Benchmark: Isotonic / Perturbed Logarithm Seconds < Lower Is Better Caffe 2020-02-13 Model: GoogleNet - Acceleration: CPU - Iterations: 100 Milli-Seconds < Lower Is Better Intel UHD 610 CML GT1 . 214399 |=============================================== Numpy Benchmark Score > Higher Is Better Intel UHD 610 CML GT1 . 290.97 |=============================================== Caffe 2020-02-13 Model: AlexNet - Acceleration: CPU - Iterations: 200 Milli-Seconds < Lower Is Better Intel UHD 610 CML GT1 . 197396 |=============================================== Scikit-Learn 1.2.2 Benchmark: Sample Without Replacement Seconds < Lower Is Better Intel UHD 610 CML GT1 . 146.72 |=============================================== oneDNN 3.3 Harness: Recurrent Neural Network Inference - Data Type: u8s8f32 - Engine: CPU ms < Lower Is Better Intel UHD 610 CML GT1 . 21214.8 |============================================== oneDNN 3.3 Harness: Recurrent Neural Network Inference - Data Type: bf16bf16bf16 - Engine: CPU ms < Lower Is Better Intel UHD 610 CML GT1 . 21213.8 |============================================== oneDNN 3.3 Harness: Recurrent Neural Network Inference - Data Type: f32 - Engine: CPU ms < Lower Is Better Intel UHD 610 CML GT1 . 21191.3 |============================================== Numenta Anomaly Benchmark 1.1 Detector: Bayesian Changepoint Seconds < Lower Is Better Intel UHD 610 CML GT1 . 168.69 |=============================================== Scikit-Learn 1.2.2 Benchmark: Sparsify Seconds < Lower Is Better Intel UHD 610 CML GT1 . 107.14 |=============================================== Scikit-Learn 1.2.2 Benchmark: Hist Gradient Boosting Adult Seconds < Lower Is Better Intel UHD 610 CML GT1 . 104.54 |=============================================== Numenta Anomaly Benchmark 1.1 Detector: Contextual Anomaly Detector OSE Seconds < Lower Is Better Intel UHD 610 CML GT1 . 139.35 |=============================================== Scikit-Learn 1.2.2 Benchmark: Plot Ward Seconds < Lower Is Better Intel UHD 610 CML GT1 . 94.53 |================================================ Mlpack Benchmark Benchmark: scikit_ica Seconds < Lower Is Better Intel UHD 610 CML GT1 . 118.31 |=============================================== Scikit-Learn 1.2.2 Benchmark: MNIST Dataset Seconds < Lower Is Better Intel UHD 610 CML GT1 . 87.88 |================================================ Scikit-Learn 1.2.2 Benchmark: Plot Incremental PCA Seconds < Lower Is Better Intel UHD 610 CML GT1 . 85.50 |================================================ Scikit-Learn 1.2.2 Benchmark: Text Vectorizers Seconds < Lower Is Better Intel UHD 610 CML GT1 . 81.39 |================================================ ONNX Runtime 1.14 Model: fcn-resnet101-11 - Device: CPU - Executor: Standard Inference Time Cost (ms) < Lower Is Better Intel UHD 610 CML GT1 . 9433.67 |============================================== ONNX Runtime 1.14 Model: fcn-resnet101-11 - Device: CPU - Executor: Standard Inferences Per Second > Higher Is Better Intel UHD 610 CML GT1 . 0.106003 |============================================= Caffe 2020-02-13 Model: AlexNet - Acceleration: CPU - Iterations: 100 Milli-Seconds < Lower Is Better Intel UHD 610 CML GT1 . 98849 |================================================ ONNX Runtime 1.14 Model: bertsquad-12 - Device: CPU - Executor: Parallel Inference Time Cost (ms) < Lower Is Better Intel UHD 610 CML GT1 . 914.99 |=============================================== ONNX Runtime 1.14 Model: bertsquad-12 - Device: CPU - Executor: Parallel Inferences Per Second > Higher Is Better Intel UHD 610 CML GT1 . 1.09291 |============================================== ONNX Runtime 1.14 Model: Faster R-CNN R-50-FPN-int8 - Device: CPU - Executor: Standard Inference Time Cost (ms) < Lower Is Better Intel UHD 610 CML GT1 . 750.73 |=============================================== ONNX Runtime 1.14 Model: Faster R-CNN R-50-FPN-int8 - Device: CPU - Executor: Standard Inferences Per Second > Higher Is Better Intel UHD 610 CML GT1 . 1.33205 |============================================== ONNX Runtime 1.14 Model: ArcFace ResNet-100 - Device: CPU - Executor: Standard Inference Time Cost (ms) < Lower Is Better Intel UHD 610 CML GT1 . 410.75 |=============================================== ONNX Runtime 1.14 Model: ArcFace ResNet-100 - Device: CPU - Executor: Standard Inferences Per Second > Higher Is Better Intel UHD 610 CML GT1 . 2.43458 |============================================== ONNX Runtime 1.14 Model: ArcFace ResNet-100 - Device: CPU - Executor: Parallel Inference Time Cost (ms) < Lower Is Better Intel UHD 610 CML GT1 . 461.30 |=============================================== ONNX Runtime 1.14 Model: ArcFace ResNet-100 - Device: CPU - Executor: Parallel Inferences Per Second > Higher Is Better Intel UHD 610 CML GT1 . 2.16783 |============================================== ONNX Runtime 1.14 Model: GPT-2 - Device: CPU - Executor: Standard Inference Time Cost (ms) < Lower Is Better Intel UHD 610 CML GT1 . 33.95 |================================================ ONNX Runtime 1.14 Model: GPT-2 - Device: CPU - Executor: Standard Inferences Per Second > Higher Is Better Intel UHD 610 CML GT1 . 29.45 |================================================ OpenVINO 2023.1 Model: Face Detection FP16 - Device: CPU ms < Lower Is Better Intel UHD 610 CML GT1 . 11809.47 |============================================= OpenVINO 2023.1 Model: Face Detection FP16 - Device: CPU FPS > Higher Is Better Intel UHD 610 CML GT1 . 0.17 |================================================= ONNX Runtime 1.14 Model: ResNet50 v1-12-int8 - Device: CPU - Executor: Parallel Inference Time Cost (ms) < Lower Is Better Intel UHD 610 CML GT1 . 112.39 |=============================================== ONNX Runtime 1.14 Model: ResNet50 v1-12-int8 - Device: CPU - Executor: Parallel Inferences Per Second > Higher Is Better Intel UHD 610 CML GT1 . 8.89770 |============================================== ONNX Runtime 1.14 Model: ResNet50 v1-12-int8 - Device: CPU - Executor: Standard Inference Time Cost (ms) < Lower Is Better Intel UHD 610 CML GT1 . 96.62 |================================================ ONNX Runtime 1.14 Model: ResNet50 v1-12-int8 - Device: CPU - Executor: Standard Inferences Per Second > Higher Is Better Intel UHD 610 CML GT1 . 10.35 |================================================ ONNX Runtime 1.14 Model: CaffeNet 12-int8 - Device: CPU - Executor: Parallel Inference Time Cost (ms) < Lower Is Better Intel UHD 610 CML GT1 . 31.29 |================================================ ONNX Runtime 1.14 Model: CaffeNet 12-int8 - Device: CPU - Executor: Parallel Inferences Per Second > Higher Is Better Intel UHD 610 CML GT1 . 31.95 |================================================ Scikit-Learn 1.2.2 Benchmark: 20 Newsgroups / Logistic Regression Seconds < Lower Is Better Intel UHD 610 CML GT1 . 61.60 |================================================ Scikit-Learn 1.2.2 Benchmark: Plot Non-Negative Matrix Factorization Seconds < Lower Is Better OpenVINO 2023.1 Model: Face Detection FP16-INT8 - Device: CPU ms < Lower Is Better Intel UHD 610 CML GT1 . 3863.39 |============================================== OpenVINO 2023.1 Model: Face Detection FP16-INT8 - Device: CPU FPS > Higher Is Better Intel UHD 610 CML GT1 . 0.52 |================================================= Numenta Anomaly Benchmark 1.1 Detector: Relative Entropy Seconds < Lower Is Better Intel UHD 610 CML GT1 . 68.53 |================================================ OpenVINO 2023.1 Model: Machine Translation EN To DE FP16 - Device: CPU ms < Lower Is Better Intel UHD 610 CML GT1 . 810.07 |=============================================== OpenVINO 2023.1 Model: Machine Translation EN To DE FP16 - Device: CPU FPS > Higher Is Better Intel UHD 610 CML GT1 . 2.47 |================================================= TensorFlow Lite 2022-05-18 Model: Mobilenet Quant Microseconds < Lower Is Better Intel UHD 610 CML GT1 . 576680 |=============================================== TensorFlow Lite 2022-05-18 Model: Inception V4 Microseconds < Lower Is Better Intel UHD 610 CML GT1 . 426984 |=============================================== TensorFlow Lite 2022-05-18 Model: Inception ResNet V2 Microseconds < Lower Is Better Intel UHD 610 CML GT1 . 397245 |=============================================== OpenVINO 2023.1 Model: Person Detection FP16 - Device: CPU ms < Lower Is Better Intel UHD 610 CML GT1 . 926.63 |=============================================== OpenVINO 2023.1 Model: Person Detection FP16 - Device: CPU FPS > Higher Is Better Intel UHD 610 CML GT1 . 2.15 |================================================= OpenVINO 2023.1 Model: Person Detection FP32 - Device: CPU ms < Lower Is Better Intel UHD 610 CML GT1 . 941.81 |=============================================== OpenVINO 2023.1 Model: Person Detection FP32 - Device: CPU FPS > Higher Is Better Intel UHD 610 CML GT1 . 2.12 |================================================= OpenVINO 2023.1 Model: Road Segmentation ADAS FP16-INT8 - Device: CPU ms < Lower Is Better Intel UHD 610 CML GT1 . 119.47 |=============================================== OpenVINO 2023.1 Model: Road Segmentation ADAS FP16-INT8 - Device: CPU FPS > Higher Is Better Intel UHD 610 CML GT1 . 16.74 |================================================ OpenVINO 2023.1 Model: Road Segmentation ADAS FP16 - Device: CPU ms < Lower Is Better Intel UHD 610 CML GT1 . 238.38 |=============================================== OpenVINO 2023.1 Model: Road Segmentation ADAS FP16 - Device: CPU FPS > Higher Is Better Intel UHD 610 CML GT1 . 8.39 |================================================= TensorFlow Lite 2022-05-18 Model: NASNet Mobile Microseconds < Lower Is Better Intel UHD 610 CML GT1 . 41500.3 |============================================== OpenVINO 2023.1 Model: Handwritten English Recognition FP16-INT8 - Device: CPU ms < Lower Is Better Intel UHD 610 CML GT1 . 125.63 |=============================================== OpenVINO 2023.1 Model: Handwritten English Recognition FP16-INT8 - Device: CPU FPS > Higher Is Better Intel UHD 610 CML GT1 . 15.91 |================================================ OpenVINO 2023.1 Model: Handwritten English Recognition FP16 - Device: CPU ms < Lower Is Better Intel UHD 610 CML GT1 . 158.68 |=============================================== OpenVINO 2023.1 Model: Handwritten English Recognition FP16 - Device: CPU FPS > Higher Is Better Intel UHD 610 CML GT1 . 12.60 |================================================ TensorFlow Lite 2022-05-18 Model: Mobilenet Float Microseconds < Lower Is Better Intel UHD 610 CML GT1 . 22665.5 |============================================== TensorFlow Lite 2022-05-18 Model: SqueezeNet Microseconds < Lower Is Better Intel UHD 610 CML GT1 . 31464.6 |============================================== OpenVINO 2023.1 Model: Vehicle Detection FP16-INT8 - Device: CPU ms < Lower Is Better Intel UHD 610 CML GT1 . 59.63 |================================================ OpenVINO 2023.1 Model: Vehicle Detection FP16-INT8 - Device: CPU FPS > Higher Is Better Intel UHD 610 CML GT1 . 33.53 |================================================ OpenVINO 2023.1 Model: Weld Porosity Detection FP16 - Device: CPU ms < Lower Is Better Intel UHD 610 CML GT1 . 120.87 |=============================================== OpenVINO 2023.1 Model: Weld Porosity Detection FP16 - Device: CPU FPS > Higher Is Better Intel UHD 610 CML GT1 . 16.54 |================================================ OpenVINO 2023.1 Model: Vehicle Detection FP16 - Device: CPU ms < Lower Is Better Intel UHD 610 CML GT1 . 128.29 |=============================================== OpenVINO 2023.1 Model: Vehicle Detection FP16 - Device: CPU FPS > Higher Is Better Intel UHD 610 CML GT1 . 15.58 |================================================ OpenVINO 2023.1 Model: Face Detection Retail FP16-INT8 - Device: CPU ms < Lower Is Better Intel UHD 610 CML GT1 . 18.84 |================================================ OpenVINO 2023.1 Model: Face Detection Retail FP16-INT8 - Device: CPU FPS > Higher Is Better Intel UHD 610 CML GT1 . 106.11 |=============================================== OpenVINO 2023.1 Model: Weld Porosity Detection FP16-INT8 - Device: CPU ms < Lower Is Better Intel UHD 610 CML GT1 . 38.50 |================================================ OpenVINO 2023.1 Model: Weld Porosity Detection FP16-INT8 - Device: CPU FPS > Higher Is Better Intel UHD 610 CML GT1 . 51.93 |================================================ OpenVINO 2023.1 Model: Face Detection Retail FP16 - Device: CPU ms < Lower Is Better Intel UHD 610 CML GT1 . 38.92 |================================================ OpenVINO 2023.1 Model: Face Detection Retail FP16 - Device: CPU FPS > Higher Is Better Intel UHD 610 CML GT1 . 51.36 |================================================ OpenVINO 2023.1 Model: Age Gender Recognition Retail 0013 FP16 - Device: CPU ms < Lower Is Better Intel UHD 610 CML GT1 . 3.96 |================================================= OpenVINO 2023.1 Model: Age Gender Recognition Retail 0013 FP16 - Device: CPU FPS > Higher Is Better Intel UHD 610 CML GT1 . 504.02 |=============================================== OpenVINO 2023.1 Model: Age Gender Recognition Retail 0013 FP16-INT8 - Device: CPU ms < Lower Is Better Intel UHD 610 CML GT1 . 1.53 |================================================= OpenVINO 2023.1 Model: Age Gender Recognition Retail 0013 FP16-INT8 - Device: CPU FPS > Higher Is Better Intel UHD 610 CML GT1 . 1300.47 |============================================== Scikit-Learn 1.2.2 Benchmark: Hist Gradient Boosting Categorical Only Seconds < Lower Is Better Intel UHD 610 CML GT1 . 27.72 |================================================ R Benchmark Seconds < Lower Is Better Intel UHD 610 CML GT1 . 0.3572 |=============================================== Numenta Anomaly Benchmark 1.1 Detector: Windowed Gaussian Seconds < Lower Is Better Intel UHD 610 CML GT1 . 34.11 |================================================ Mlpack Benchmark Benchmark: scikit_svm Seconds < Lower Is Better Intel UHD 610 CML GT1 . 27.75 |================================================ RNNoise 2020-06-28 Seconds < Lower Is Better Intel UHD 610 CML GT1 . 27.00 |================================================ TNN 0.3 Target: CPU - Model: MobileNet v2 ms < Lower Is Better Intel UHD 610 CML GT1 . 374.72 |=============================================== TNN 0.3 Target: CPU - Model: SqueezeNet v1.1 ms < Lower Is Better Intel UHD 610 CML GT1 . 335.26 |=============================================== oneDNN 3.3 Harness: Deconvolution Batch shapes_1d - Data Type: f32 - Engine: CPU ms < Lower Is Better Intel UHD 610 CML GT1 . 78.29 |================================================ oneDNN 3.3 Harness: Deconvolution Batch shapes_1d - Data Type: u8s8f32 - Engine: CPU ms < Lower Is Better Intel UHD 610 CML GT1 . 19.79 |================================================ oneDNN 3.3 Harness: IP Shapes 1D - Data Type: f32 - Engine: CPU ms < Lower Is Better Intel UHD 610 CML GT1 . 37.76 |================================================ oneDNN 3.3 Harness: IP Shapes 1D - Data Type: u8s8f32 - Engine: CPU ms < Lower Is Better Intel UHD 610 CML GT1 . 12.28 |================================================ Scikit-Learn 1.2.2 Benchmark: RCV1 Logreg Convergencet Seconds < Lower Is Better oneDNN 3.3 Harness: IP Shapes 3D - Data Type: f32 - Engine: CPU ms < Lower Is Better Intel UHD 610 CML GT1 . 38.02 |================================================ OpenCV 4.7 Test: DNN - Deep Neural Network ms < Lower Is Better oneDNN 3.3 Harness: IP Shapes 3D - Data Type: u8s8f32 - Engine: CPU ms < Lower Is Better Intel UHD 610 CML GT1 . 5.90947 |============================================== oneDNN 3.3 Harness: Convolution Batch Shapes Auto - Data Type: f32 - Engine: CPU ms < Lower Is Better Intel UHD 610 CML GT1 . 60.25 |================================================ oneDNN 3.3 Harness: Convolution Batch Shapes Auto - Data Type: u8s8f32 - Engine: CPU ms < Lower Is Better Intel UHD 610 CML GT1 . 48.78 |================================================ TNN 0.3 Target: CPU - Model: SqueezeNet v2 ms < Lower Is Better Intel UHD 610 CML GT1 . 75.16 |================================================ oneDNN 3.3 Harness: Deconvolution Batch shapes_3d - Data Type: f32 - Engine: CPU ms < Lower Is Better Intel UHD 610 CML GT1 . 92.42 |================================================ oneDNN 3.3 Harness: Deconvolution Batch shapes_3d - Data Type: u8s8f32 - Engine: CPU ms < Lower Is Better Intel UHD 610 CML GT1 . 26.95 |================================================ TensorFlow 2.12 Device: CPU - Batch Size: 16 - Model: VGG-16 images/sec > Higher Is Better TensorFlow 2.12 Device: CPU - Batch Size: 256 - Model: ResNet-50 images/sec > Higher Is Better TensorFlow 2.12 Device: CPU - Batch Size: 256 - Model: AlexNet images/sec > Higher Is Better TensorFlow 2.12 Device: CPU - Batch Size: 512 - Model: ResNet-50 images/sec > Higher Is Better TensorFlow 2.12 Device: CPU - Batch Size: 32 - Model: ResNet-50 images/sec > Higher Is Better TensorFlow 2.12 Device: CPU - Batch Size: 16 - Model: GoogLeNet images/sec > Higher Is Better TensorFlow 2.12 Device: CPU - Batch Size: 32 - Model: AlexNet images/sec > Higher Is Better TensorFlow 2.12 Device: CPU - Batch Size: 256 - Model: VGG-16 images/sec > Higher Is Better TensorFlow 2.12 Device: CPU - Batch Size: 64 - Model: VGG-16 images/sec > Higher Is Better TensorFlow 2.12 Device: CPU - Batch Size: 16 - Model: AlexNet images/sec > Higher Is Better TensorFlow 2.12 Device: CPU - Batch Size: 32 - Model: VGG-16 images/sec > Higher Is Better TensorFlow 2.12 Device: CPU - Batch Size: 512 - Model: VGG-16 images/sec > Higher Is Better TensorFlow 2.12 Device: CPU - Batch Size: 512 - Model: AlexNet images/sec > Higher Is Better TensorFlow 2.12 Device: CPU - Batch Size: 256 - Model: GoogLeNet images/sec > Higher Is Better TensorFlow 2.12 Device: CPU - Batch Size: 64 - Model: ResNet-50 images/sec > Higher Is Better TensorFlow 2.12 Device: CPU - Batch Size: 16 - Model: ResNet-50 images/sec > Higher Is Better TensorFlow 2.12 Device: CPU - Batch Size: 512 - Model: GoogLeNet images/sec > Higher Is Better TensorFlow 2.12 Device: CPU - Batch Size: 32 - Model: GoogLeNet images/sec > Higher Is Better TensorFlow 2.12 Device: CPU - Batch Size: 64 - Model: AlexNet images/sec > Higher Is Better TensorFlow 2.12 Device: CPU - Batch Size: 64 - Model: GoogLeNet images/sec > Higher Is Better Neural Magic DeepSparse 1.5 Model: BERT-Large, NLP Question Answering - Scenario: Synchronous Single-Stream items/sec > Higher Is Better Neural Magic DeepSparse 1.5 Model: CV Classification, ResNet-50 ImageNet - Scenario: Synchronous Single-Stream items/sec > Higher Is Better Neural Magic DeepSparse 1.5 Model: CV Detection, YOLOv5s COCO, Sparse INT8 - Scenario: Synchronous Single-Stream items/sec > Higher Is Better Neural Magic DeepSparse 1.5 Model: BERT-Large, NLP Question Answering - Scenario: Asynchronous Multi-Stream items/sec > Higher Is Better Neural Magic DeepSparse 1.5 Model: NLP Text Classification, DistilBERT mnli - Scenario: Synchronous Single-Stream items/sec > Higher Is Better Neural Magic DeepSparse 1.5 Model: NLP Text Classification, DistilBERT mnli - Scenario: Asynchronous Multi-Stream items/sec > Higher Is Better Neural Magic DeepSparse 1.5 Model: CV Classification, ResNet-50 ImageNet - Scenario: Asynchronous Multi-Stream items/sec > Higher Is Better Neural Magic DeepSparse 1.5 Model: CV Detection, YOLOv5s COCO, Sparse INT8 - Scenario: Asynchronous Multi-Stream items/sec > Higher Is Better Scikit-Learn 1.2.2 Benchmark: Plot Parallel Pairwise Seconds < Lower Is Better Neural Magic DeepSparse 1.5 Model: CV Segmentation, 90% Pruned YOLACT Pruned - Scenario: Asynchronous Multi-Stream items/sec > Higher Is Better Neural Magic DeepSparse 1.5 Model: BERT-Large, NLP Question Answering, Sparse INT8 - Scenario: Asynchronous Multi-Stream items/sec > Higher Is Better Neural Magic DeepSparse 1.5 Model: CV Detection, YOLOv5s COCO - Scenario: Synchronous Single-Stream items/sec > Higher Is Better Neural Magic DeepSparse 1.5 Model: BERT-Large, NLP Question Answering, Sparse INT8 - Scenario: Synchronous Single-Stream items/sec > Higher Is Better Neural Magic DeepSparse 1.5 Model: CV Segmentation, 90% Pruned YOLACT Pruned - Scenario: Synchronous Single-Stream items/sec > Higher Is Better Neural Magic DeepSparse 1.5 Model: NLP Document Classification, oBERT base uncased on IMDB - Scenario: Asynchronous Multi-Stream items/sec > Higher Is Better Neural Magic DeepSparse 1.5 Model: NLP Text Classification, BERT base uncased SST2, Sparse INT8 - Scenario: Synchronous Single-Stream items/sec > Higher Is Better Neural Magic DeepSparse 1.5 Model: ResNet-50, Sparse INT8 - Scenario: Synchronous Single-Stream items/sec > Higher Is Better Neural Magic DeepSparse 1.5 Model: ResNet-50, Baseline - Scenario: Asynchronous Multi-Stream items/sec > Higher Is Better Neural Magic DeepSparse 1.5 Model: NLP Question Answering, BERT base uncased SQuaD 12layer Pruned90 - Scenario: Synchronous Single-Stream items/sec > Higher Is Better Neural Magic DeepSparse 1.5 Model: NLP Sentiment Analysis, 80% Pruned Quantized BERT Base Uncased - Scenario: Synchronous Single-Stream items/sec > Higher Is Better Neural Magic DeepSparse 1.5 Model: NLP Text Classification, BERT base uncased SST2 - Scenario: Synchronous Single-Stream items/sec > Higher Is Better Neural Magic DeepSparse 1.5 Model: ResNet-50, Sparse INT8 - Scenario: Asynchronous Multi-Stream items/sec > Higher Is Better Neural Magic DeepSparse 1.5 Model: ResNet-50, Baseline - Scenario: Synchronous Single-Stream items/sec > Higher Is Better Neural Magic DeepSparse 1.5 Model: NLP Token Classification, BERT base uncased conll2003 - Scenario: Synchronous Single-Stream items/sec > Higher Is Better Neural Magic DeepSparse 1.5 Model: NLP Sentiment Analysis, 80% Pruned Quantized BERT Base Uncased - Scenario: Asynchronous Multi-Stream items/sec > Higher Is Better Neural Magic DeepSparse 1.5 Model: NLP Document Classification, oBERT base uncased on IMDB - Scenario: Synchronous Single-Stream items/sec > Higher Is Better Neural Magic DeepSparse 1.5 Model: CV Detection, YOLOv5s COCO - Scenario: Asynchronous Multi-Stream items/sec > Higher Is Better Neural Magic DeepSparse 1.5 Model: NLP Question Answering, BERT base uncased SQuaD 12layer Pruned90 - Scenario: Asynchronous Multi-Stream items/sec > Higher Is Better Neural Magic DeepSparse 1.5 Model: NLP Text Classification, BERT base uncased SST2 - Scenario: Asynchronous Multi-Stream items/sec > Higher Is Better Neural Magic DeepSparse 1.5 Model: NLP Text Classification, BERT base uncased SST2, Sparse INT8 - Scenario: Asynchronous Multi-Stream items/sec > Higher Is Better Neural Magic DeepSparse 1.5 Model: NLP Token Classification, BERT base uncased conll2003 - Scenario: Asynchronous Multi-Stream items/sec > Higher Is Better Scikit-Learn 1.2.2 Benchmark: Glmnet Seconds < Lower Is Better OpenVINO 2023.1 Model: Person Vehicle Bike Detection FP16 - Device: CPU FPS > Higher Is Better spaCy 3.4.1 tokens/sec > Higher Is Better ECP-CANDLE 0.4 Benchmark: P1B2 Seconds < Lower Is Better DeepSpeech 0.6 Acceleration: CPU Seconds < Lower Is Better ECP-CANDLE 0.4 Benchmark: P3B1 Seconds < Lower Is Better ECP-CANDLE 0.4 Benchmark: P3B2 Seconds < Lower Is Better AI Benchmark Alpha 0.1.2 Score > Higher Is Better ONNX Runtime 1.14 Model: yolov4 - Device: CPU - Executor: Standard Inferences Per Second > Higher Is Better ONNX Runtime 1.14 Model: yolov4 - Device: CPU - Executor: Parallel Inferences Per Second > Higher Is Better oneDNN 3.3 Harness: Deconvolution Batch shapes_1d - Data Type: bf16bf16bf16 - Engine: CPU ms < Lower Is Better oneDNN 3.3 Harness: Convolution Batch Shapes Auto - Data Type: bf16bf16bf16 - Engine: CPU ms < Lower Is Better oneDNN 3.3 Harness: Deconvolution Batch shapes_3d - Data Type: bf16bf16bf16 - Engine: CPU ms < Lower Is Better oneDNN 3.3 Harness: IP Shapes 3D - Data Type: bf16bf16bf16 - Engine: CPU ms < Lower Is Better oneDNN 3.3 Harness: IP Shapes 1D - Data Type: bf16bf16bf16 - Engine: CPU ms < Lower Is Better