24.03.13.Pop.2204.ML.test1 AMD Ryzen 9 7950X 16-Core testing with a ASUS ProArt X670E-CREATOR WIFI (1710 BIOS) and Zotac NVIDIA GeForce RTX 4070 Ti 12GB on Pop 22.04 via the Phoronix Test Suite. Initial test 1 No water cool: Processor: AMD Ryzen 9 7950X 16-Core @ 5.88GHz (16 Cores / 32 Threads), Motherboard: ASUS ProArt X670E-CREATOR WIFI (1710 BIOS), Chipset: AMD Device 14d8, Memory: 2 x 16 GB DDR5-4800MT/s G Skill F5-6000J3636F16G, Disk: 1000GB PNY CS2130 1TB SSD, Graphics: Zotac NVIDIA GeForce RTX 4070 Ti 12GB, Audio: NVIDIA Device 22bc, Monitor: 2 x DELL 2001FP, Network: Intel I225-V + Aquantia AQtion AQC113CS NBase-T/IEEE + MEDIATEK MT7922 802.11ax PCI OS: Pop 22.04, Kernel: 6.6.10-76060610-generic (x86_64), Desktop: GNOME Shell 42.5, Display Server: X Server 1.21.1.4, Display Driver: NVIDIA 550.54.14, OpenGL: 4.6.0, OpenCL: OpenCL 3.0 CUDA 12.4.89, Vulkan: 1.3.277, Compiler: GCC 11.4.0, File-System: ext4, Screen Resolution: 3200x1200 SHOC Scalable HeterOgeneous Computing 2020-04-17 Target: OpenCL - Benchmark: S3D GFLOPS > Higher Is Better Initial test 1 No water cool . 299.47 |======================================== SHOC Scalable HeterOgeneous Computing 2020-04-17 Target: OpenCL - Benchmark: Triad GB/s > Higher Is Better Initial test 1 No water cool . 25.46 |========================================= SHOC Scalable HeterOgeneous Computing 2020-04-17 Target: OpenCL - Benchmark: FFT SP GFLOPS > Higher Is Better Initial test 1 No water cool . 1292.53 |======================================= SHOC Scalable HeterOgeneous Computing 2020-04-17 Target: OpenCL - Benchmark: MD5 Hash GHash/s > Higher Is Better Initial test 1 No water cool . 47.90 |========================================= SHOC Scalable HeterOgeneous Computing 2020-04-17 Target: OpenCL - Benchmark: Reduction GB/s > Higher Is Better Initial test 1 No water cool . 388.93 |======================================== SHOC Scalable HeterOgeneous Computing 2020-04-17 Target: OpenCL - Benchmark: GEMM SGEMM_N GFLOPS > Higher Is Better Initial test 1 No water cool . 13212.0 |======================================= SHOC Scalable HeterOgeneous Computing 2020-04-17 Target: OpenCL - Benchmark: Max SP Flops GFLOPS > Higher Is Better Initial test 1 No water cool . 43074.9 |======================================= SHOC Scalable HeterOgeneous Computing 2020-04-17 Target: OpenCL - Benchmark: Bus Speed Download GB/s > Higher Is Better Initial test 1 No water cool . 26.83 |========================================= SHOC Scalable HeterOgeneous Computing 2020-04-17 Target: OpenCL - Benchmark: Bus Speed Readback GB/s > Higher Is Better Initial test 1 No water cool . 27.07 |========================================= SHOC Scalable HeterOgeneous Computing 2020-04-17 Target: OpenCL - Benchmark: Texture Read Bandwidth GB/s > Higher Is Better Initial test 1 No water cool . 2985.70 |======================================= LeelaChessZero 0.30 Backend: BLAS Nodes Per Second > Higher Is Better oneDNN 3.4 Harness: IP Shapes 1D - Engine: CPU ms < Lower Is Better Initial test 1 No water cool . 1.17351 |======================================= oneDNN 3.4 Harness: IP Shapes 3D - Engine: CPU ms < Lower Is Better Initial test 1 No water cool . 4.42170 |======================================= oneDNN 3.4 Harness: Convolution Batch Shapes Auto - Engine: CPU ms < Lower Is Better Initial test 1 No water cool . 7.16631 |======================================= oneDNN 3.4 Harness: Deconvolution Batch shapes_1d - Engine: CPU ms < Lower Is Better Initial test 1 No water cool . 3.06179 |======================================= oneDNN 3.4 Harness: Deconvolution Batch shapes_3d - Engine: CPU ms < Lower Is Better Initial test 1 No water cool . 2.56519 |======================================= oneDNN 3.4 Harness: Recurrent Neural Network Training - Engine: CPU ms < Lower Is Better Initial test 1 No water cool . 1452.96 |======================================= oneDNN 3.4 Harness: Recurrent Neural Network Inference - Engine: CPU ms < Lower Is Better Initial test 1 No water cool . 747.50 |======================================== Numpy Benchmark Score > Higher Is Better Initial test 1 No water cool . 704.52 |======================================== DeepSpeech 0.6 Acceleration: CPU Seconds < Lower Is Better Initial test 1 No water cool . 47.04 |========================================= R Benchmark Seconds < Lower Is Better RNNoise 2020-06-28 Seconds < Lower Is Better Initial test 1 No water cool . 13.71 |========================================= TensorFlow Lite 2022-05-18 Model: SqueezeNet Microseconds < Lower Is Better Initial test 1 No water cool . 1716.04 |======================================= TensorFlow Lite 2022-05-18 Model: Inception V4 Microseconds < Lower Is Better Initial test 1 No water cool . 21139.4 |======================================= TensorFlow Lite 2022-05-18 Model: NASNet Mobile Microseconds < Lower Is Better Initial test 1 No water cool . 10099.3 |======================================= TensorFlow Lite 2022-05-18 Model: Mobilenet Float Microseconds < Lower Is Better Initial test 1 No water cool . 1214.11 |======================================= TensorFlow Lite 2022-05-18 Model: Mobilenet Quant Microseconds < Lower Is Better Initial test 1 No water cool . 1861.53 |======================================= TensorFlow Lite 2022-05-18 Model: Inception ResNet V2 Microseconds < Lower Is Better Initial test 1 No water cool . 21857.0 |======================================= PyTorch 2.1 Device: CPU - Batch Size: 1 - Model: ResNet-50 batches/sec > Higher Is Better Initial test 1 No water cool . 64.81 |========================================= PyTorch 2.1 Device: CPU - Batch Size: 1 - Model: ResNet-152 batches/sec > Higher Is Better Initial test 1 No water cool . 25.64 |========================================= PyTorch 2.1 Device: CPU - Batch Size: 16 - Model: ResNet-50 batches/sec > Higher Is Better Initial test 1 No water cool . 44.08 |========================================= PyTorch 2.1 Device: CPU - Batch Size: 32 - Model: ResNet-50 batches/sec > Higher Is Better Initial test 1 No water cool . 44.08 |========================================= PyTorch 2.1 Device: CPU - Batch Size: 64 - Model: ResNet-50 batches/sec > Higher Is Better Initial test 1 No water cool . 43.35 |========================================= PyTorch 2.1 Device: CPU - Batch Size: 16 - Model: ResNet-152 batches/sec > Higher Is Better Initial test 1 No water cool . 17.66 |========================================= PyTorch 2.1 Device: CPU - Batch Size: 256 - Model: ResNet-50 batches/sec > Higher Is Better Initial test 1 No water cool . 43.49 |========================================= PyTorch 2.1 Device: CPU - Batch Size: 32 - Model: ResNet-152 batches/sec > Higher Is Better Initial test 1 No water cool . 17.64 |========================================= PyTorch 2.1 Device: CPU - Batch Size: 512 - Model: ResNet-50 batches/sec > Higher Is Better Initial test 1 No water cool . 42.91 |========================================= PyTorch 2.1 Device: CPU - Batch Size: 64 - Model: ResNet-152 batches/sec > Higher Is Better Initial test 1 No water cool . 17.66 |========================================= PyTorch 2.1 Device: CPU - Batch Size: 256 - Model: ResNet-152 batches/sec > Higher Is Better Initial test 1 No water cool . 17.69 |========================================= PyTorch 2.1 Device: CPU - Batch Size: 512 - Model: ResNet-152 batches/sec > Higher Is Better Initial test 1 No water cool . 17.59 |========================================= PyTorch 2.1 Device: CPU - Batch Size: 1 - Model: Efficientnet_v2_l batches/sec > Higher Is Better Initial test 1 No water cool . 14.14 |========================================= PyTorch 2.1 Device: CPU - Batch Size: 16 - Model: Efficientnet_v2_l batches/sec > Higher Is Better Initial test 1 No water cool . 10.46 |========================================= PyTorch 2.1 Device: CPU - Batch Size: 32 - Model: Efficientnet_v2_l batches/sec > Higher Is Better Initial test 1 No water cool . 10.59 |========================================= PyTorch 2.1 Device: CPU - Batch Size: 64 - Model: Efficientnet_v2_l batches/sec > Higher Is Better Initial test 1 No water cool . 10.58 |========================================= PyTorch 2.1 Device: CPU - Batch Size: 256 - Model: Efficientnet_v2_l batches/sec > Higher Is Better Initial test 1 No water cool . 10.63 |========================================= PyTorch 2.1 Device: CPU - Batch Size: 512 - Model: Efficientnet_v2_l batches/sec > Higher Is Better Initial test 1 No water cool . 10.44 |========================================= PyTorch 2.1 Device: NVIDIA CUDA GPU - Batch Size: 1 - Model: ResNet-50 batches/sec > Higher Is Better Initial test 1 No water cool . 387.06 |======================================== PyTorch 2.1 Device: NVIDIA CUDA GPU - Batch Size: 1 - Model: ResNet-152 batches/sec > Higher Is Better Initial test 1 No water cool . 137.39 |======================================== PyTorch 2.1 Device: NVIDIA CUDA GPU - Batch Size: 16 - Model: ResNet-50 batches/sec > Higher Is Better Initial test 1 No water cool . 380.67 |======================================== PyTorch 2.1 Device: NVIDIA CUDA GPU - Batch Size: 32 - Model: ResNet-50 batches/sec > Higher Is Better Initial test 1 No water cool . 380.74 |======================================== PyTorch 2.1 Device: NVIDIA CUDA GPU - Batch Size: 64 - Model: ResNet-50 batches/sec > Higher Is Better Initial test 1 No water cool . 379.98 |======================================== PyTorch 2.1 Device: NVIDIA CUDA GPU - Batch Size: 16 - Model: ResNet-152 batches/sec > Higher Is Better Initial test 1 No water cool . 138.78 |======================================== PyTorch 2.1 Device: NVIDIA CUDA GPU - Batch Size: 256 - Model: ResNet-50 batches/sec > Higher Is Better Initial test 1 No water cool . 380.34 |======================================== PyTorch 2.1 Device: NVIDIA CUDA GPU - Batch Size: 32 - Model: ResNet-152 batches/sec > Higher Is Better Initial test 1 No water cool . 139.41 |======================================== PyTorch 2.1 Device: NVIDIA CUDA GPU - Batch Size: 512 - Model: ResNet-50 batches/sec > Higher Is Better Initial test 1 No water cool . 383.56 |======================================== PyTorch 2.1 Device: NVIDIA CUDA GPU - Batch Size: 64 - Model: ResNet-152 batches/sec > Higher Is Better Initial test 1 No water cool . 138.72 |======================================== PyTorch 2.1 Device: NVIDIA CUDA GPU - Batch Size: 256 - Model: ResNet-152 batches/sec > Higher Is Better Initial test 1 No water cool . 138.62 |======================================== PyTorch 2.1 Device: NVIDIA CUDA GPU - Batch Size: 512 - Model: ResNet-152 batches/sec > Higher Is Better Initial test 1 No water cool . 140.41 |======================================== PyTorch 2.1 Device: NVIDIA CUDA GPU - Batch Size: 1 - Model: Efficientnet_v2_l batches/sec > Higher Is Better Initial test 1 No water cool . 71.98 |========================================= PyTorch 2.1 Device: NVIDIA CUDA GPU - Batch Size: 16 - Model: Efficientnet_v2_l batches/sec > Higher Is Better Initial test 1 No water cool . 70.63 |========================================= PyTorch 2.1 Device: NVIDIA CUDA GPU - Batch Size: 32 - Model: Efficientnet_v2_l batches/sec > Higher Is Better Initial test 1 No water cool . 70.52 |========================================= PyTorch 2.1 Device: NVIDIA CUDA GPU - Batch Size: 64 - Model: Efficientnet_v2_l batches/sec > Higher Is Better Initial test 1 No water cool . 69.80 |========================================= PyTorch 2.1 Device: NVIDIA CUDA GPU - Batch Size: 256 - Model: Efficientnet_v2_l batches/sec > Higher Is Better Initial test 1 No water cool . 70.81 |========================================= PyTorch 2.1 Device: NVIDIA CUDA GPU - Batch Size: 512 - Model: Efficientnet_v2_l batches/sec > Higher Is Better Initial test 1 No water cool . 69.97 |========================================= TensorFlow 2.12 Device: CPU - Batch Size: 1 - Model: VGG-16 images/sec > Higher Is Better Initial test 1 No water cool . 4.74 |========================================== TensorFlow 2.12 Device: GPU - Batch Size: 1 - Model: VGG-16 images/sec > Higher Is Better Initial test 1 No water cool . 1.46 |========================================== TensorFlow 2.12 Device: CPU - Batch Size: 1 - Model: AlexNet images/sec > Higher Is Better Initial test 1 No water cool . 13.00 |========================================= TensorFlow 2.12 Device: CPU - Batch Size: 16 - Model: VGG-16 images/sec > Higher Is Better Initial test 1 No water cool . 16.09 |========================================= TensorFlow 2.12 Device: CPU - Batch Size: 32 - Model: VGG-16 images/sec > Higher Is Better Initial test 1 No water cool . 16.89 |========================================= TensorFlow 2.12 Device: CPU - Batch Size: 64 - Model: VGG-16 images/sec > Higher Is Better Initial test 1 No water cool . 17.44 |========================================= TensorFlow 2.12 Device: GPU - Batch Size: 1 - Model: AlexNet images/sec > Higher Is Better Initial test 1 No water cool . 12.58 |========================================= TensorFlow 2.12 Device: GPU - Batch Size: 16 - Model: VGG-16 images/sec > Higher Is Better Initial test 1 No water cool . 1.70 |========================================== TensorFlow 2.12 Device: GPU - Batch Size: 32 - Model: VGG-16 images/sec > Higher Is Better Initial test 1 No water cool . 1.72 |========================================== TensorFlow 2.12 Device: GPU - Batch Size: 64 - Model: VGG-16 images/sec > Higher Is Better Initial test 1 No water cool . 1.73 |========================================== TensorFlow 2.12 Device: CPU - Batch Size: 16 - Model: AlexNet images/sec > Higher Is Better Initial test 1 No water cool . 148.72 |======================================== TensorFlow 2.12 Device: CPU - Batch Size: 256 - Model: VGG-16 images/sec > Higher Is Better Initial test 1 No water cool . 18.12 |========================================= TensorFlow 2.12 Device: CPU - Batch Size: 32 - Model: AlexNet images/sec > Higher Is Better Initial test 1 No water cool . 224.56 |======================================== TensorFlow 2.12 Device: CPU - Batch Size: 512 - Model: VGG-16 images/sec > Higher Is Better TensorFlow 2.12 Device: CPU - Batch Size: 64 - Model: AlexNet images/sec > Higher Is Better Initial test 1 No water cool . 305.81 |======================================== TensorFlow 2.12 Device: GPU - Batch Size: 16 - Model: AlexNet images/sec > Higher Is Better Initial test 1 No water cool . 30.67 |========================================= TensorFlow 2.12 Device: GPU - Batch Size: 256 - Model: VGG-16 images/sec > Higher Is Better Initial test 1 No water cool . 1.77 |========================================== TensorFlow 2.12 Device: GPU - Batch Size: 32 - Model: AlexNet images/sec > Higher Is Better Initial test 1 No water cool . 33.39 |========================================= TensorFlow 2.12 Device: GPU - Batch Size: 512 - Model: VGG-16 images/sec > Higher Is Better TensorFlow 2.12 Device: GPU - Batch Size: 64 - Model: AlexNet images/sec > Higher Is Better Initial test 1 No water cool . 34.84 |========================================= TensorFlow 2.12 Device: CPU - Batch Size: 1 - Model: GoogLeNet images/sec > Higher Is Better Initial test 1 No water cool . 47.21 |========================================= TensorFlow 2.12 Device: CPU - Batch Size: 1 - Model: ResNet-50 images/sec > Higher Is Better Initial test 1 No water cool . 12.70 |========================================= TensorFlow 2.12 Device: CPU - Batch Size: 256 - Model: AlexNet images/sec > Higher Is Better Initial test 1 No water cool . 388.40 |======================================== TensorFlow 2.12 Device: CPU - Batch Size: 512 - Model: AlexNet images/sec > Higher Is Better Initial test 1 No water cool . 392.16 |======================================== TensorFlow 2.12 Device: GPU - Batch Size: 1 - Model: GoogLeNet images/sec > Higher Is Better Initial test 1 No water cool . 12.36 |========================================= TensorFlow 2.12 Device: GPU - Batch Size: 1 - Model: ResNet-50 images/sec > Higher Is Better Initial test 1 No water cool . 4.25 |========================================== TensorFlow 2.12 Device: GPU - Batch Size: 256 - Model: AlexNet images/sec > Higher Is Better Initial test 1 No water cool . 35.82 |========================================= TensorFlow 2.12 Device: GPU - Batch Size: 512 - Model: AlexNet images/sec > Higher Is Better Initial test 1 No water cool . 35.93 |========================================= TensorFlow 2.12 Device: CPU - Batch Size: 16 - Model: GoogLeNet images/sec > Higher Is Better Initial test 1 No water cool . 125.83 |======================================== TensorFlow 2.12 Device: CPU - Batch Size: 16 - Model: ResNet-50 images/sec > Higher Is Better Initial test 1 No water cool . 36.43 |========================================= TensorFlow 2.12 Device: CPU - Batch Size: 32 - Model: GoogLeNet images/sec > Higher Is Better Initial test 1 No water cool . 122.39 |======================================== TensorFlow 2.12 Device: CPU - Batch Size: 32 - Model: ResNet-50 images/sec > Higher Is Better Initial test 1 No water cool . 36.74 |========================================= TensorFlow 2.12 Device: CPU - Batch Size: 64 - Model: GoogLeNet images/sec > Higher Is Better Initial test 1 No water cool . 119.04 |======================================== TensorFlow 2.12 Device: CPU - Batch Size: 64 - Model: ResNet-50 images/sec > Higher Is Better Initial test 1 No water cool . 36.36 |========================================= TensorFlow 2.12 Device: GPU - Batch Size: 16 - Model: GoogLeNet images/sec > Higher Is Better Initial test 1 No water cool . 15.10 |========================================= TensorFlow 2.12 Device: GPU - Batch Size: 16 - Model: ResNet-50 images/sec > Higher Is Better Initial test 1 No water cool . 5.42 |========================================== TensorFlow 2.12 Device: GPU - Batch Size: 32 - Model: GoogLeNet images/sec > Higher Is Better Initial test 1 No water cool . 15.45 |========================================= TensorFlow 2.12 Device: GPU - Batch Size: 32 - Model: ResNet-50 images/sec > Higher Is Better Initial test 1 No water cool . 5.49 |========================================== TensorFlow 2.12 Device: GPU - Batch Size: 64 - Model: GoogLeNet images/sec > Higher Is Better Initial test 1 No water cool . 15.61 |========================================= TensorFlow 2.12 Device: GPU - Batch Size: 64 - Model: ResNet-50 images/sec > Higher Is Better Initial test 1 No water cool . 5.51 |========================================== TensorFlow 2.12 Device: CPU - Batch Size: 256 - Model: GoogLeNet images/sec > Higher Is Better Initial test 1 No water cool . 116.33 |======================================== TensorFlow 2.12 Device: CPU - Batch Size: 256 - Model: ResNet-50 images/sec > Higher Is Better Initial test 1 No water cool . 36.15 |========================================= TensorFlow 2.12 Device: CPU - Batch Size: 512 - Model: GoogLeNet images/sec > Higher Is Better Initial test 1 No water cool . 115.70 |======================================== TensorFlow 2.12 Device: CPU - Batch Size: 512 - Model: ResNet-50 images/sec > Higher Is Better TensorFlow 2.12 Device: GPU - Batch Size: 256 - Model: GoogLeNet images/sec > Higher Is Better Initial test 1 No water cool . 15.76 |========================================= TensorFlow 2.12 Device: GPU - Batch Size: 256 - Model: ResNet-50 images/sec > Higher Is Better Initial test 1 No water cool . 5.56 |========================================== TensorFlow 2.12 Device: GPU - Batch Size: 512 - Model: GoogLeNet images/sec > Higher Is Better Initial test 1 No water cool . 15.90 |========================================= TensorFlow 2.12 Device: GPU - Batch Size: 512 - Model: ResNet-50 images/sec > Higher Is Better Neural Magic DeepSparse 1.6 Model: NLP Document Classification, oBERT base uncased on IMDB - Scenario: Asynchronous Multi-Stream items/sec > Higher Is Better Initial test 1 No water cool . 20.09 |========================================= Neural Magic DeepSparse 1.6 Model: NLP Document Classification, oBERT base uncased on IMDB - Scenario: Asynchronous Multi-Stream ms/batch < Lower Is Better Initial test 1 No water cool . 397.58 |======================================== Neural Magic DeepSparse 1.6 Model: NLP Document Classification, oBERT base uncased on IMDB - Scenario: Synchronous Single-Stream items/sec > Higher Is Better Initial test 1 No water cool . 17.36 |========================================= Neural Magic DeepSparse 1.6 Model: NLP Document Classification, oBERT base uncased on IMDB - Scenario: Synchronous Single-Stream ms/batch < Lower Is Better Initial test 1 No water cool . 57.60 |========================================= Neural Magic DeepSparse 1.6 Model: NLP Text Classification, BERT base uncased SST2, Sparse INT8 - Scenario: Asynchronous Multi-Stream items/sec > Higher Is Better Initial test 1 No water cool . 890.19 |======================================== Neural Magic DeepSparse 1.6 Model: NLP Text Classification, BERT base uncased SST2, Sparse INT8 - Scenario: Asynchronous Multi-Stream ms/batch < Lower Is Better Initial test 1 No water cool . 8.9714 |======================================== Neural Magic DeepSparse 1.6 Model: NLP Text Classification, BERT base uncased SST2, Sparse INT8 - Scenario: Synchronous Single-Stream items/sec > Higher Is Better Initial test 1 No water cool . 278.31 |======================================== Neural Magic DeepSparse 1.6 Model: NLP Text Classification, BERT base uncased SST2, Sparse INT8 - Scenario: Synchronous Single-Stream ms/batch < Lower Is Better Initial test 1 No water cool . 3.5898 |======================================== Neural Magic DeepSparse 1.6 Model: ResNet-50, Baseline - Scenario: Asynchronous Multi-Stream items/sec > Higher Is Better Initial test 1 No water cool . 264.38 |======================================== Neural Magic DeepSparse 1.6 Model: ResNet-50, Baseline - Scenario: Asynchronous Multi-Stream ms/batch < Lower Is Better Initial test 1 No water cool . 30.24 |========================================= Neural Magic DeepSparse 1.6 Model: ResNet-50, Baseline - Scenario: Synchronous Single-Stream items/sec > Higher Is Better Initial test 1 No water cool . 173.38 |======================================== Neural Magic DeepSparse 1.6 Model: ResNet-50, Baseline - Scenario: Synchronous Single-Stream ms/batch < Lower Is Better Initial test 1 No water cool . 5.7601 |======================================== Neural Magic DeepSparse 1.6 Model: ResNet-50, Sparse INT8 - Scenario: Asynchronous Multi-Stream items/sec > Higher Is Better Initial test 1 No water cool . 2031.88 |======================================= Neural Magic DeepSparse 1.6 Model: ResNet-50, Sparse INT8 - Scenario: Asynchronous Multi-Stream ms/batch < Lower Is Better Initial test 1 No water cool . 3.9240 |======================================== Neural Magic DeepSparse 1.6 Model: ResNet-50, Sparse INT8 - Scenario: Synchronous Single-Stream items/sec > Higher Is Better Initial test 1 No water cool . 1214.24 |======================================= Neural Magic DeepSparse 1.6 Model: ResNet-50, Sparse INT8 - Scenario: Synchronous Single-Stream ms/batch < Lower Is Better Initial test 1 No water cool . 0.8214 |======================================== Neural Magic DeepSparse 1.6 Model: CV Detection, YOLOv5s COCO - Scenario: Asynchronous Multi-Stream items/sec > Higher Is Better Initial test 1 No water cool . 110.99 |======================================== Neural Magic DeepSparse 1.6 Model: CV Detection, YOLOv5s COCO - Scenario: Asynchronous Multi-Stream ms/batch < Lower Is Better Initial test 1 No water cool . 72.04 |========================================= Neural Magic DeepSparse 1.6 Model: CV Detection, YOLOv5s COCO - Scenario: Synchronous Single-Stream items/sec > Higher Is Better Initial test 1 No water cool . 90.36 |========================================= Neural Magic DeepSparse 1.6 Model: CV Detection, YOLOv5s COCO - Scenario: Synchronous Single-Stream ms/batch < Lower Is Better Initial test 1 No water cool . 11.06 |========================================= Neural Magic DeepSparse 1.6 Model: BERT-Large, NLP Question Answering - Scenario: Asynchronous Multi-Stream items/sec > Higher Is Better Initial test 1 No water cool . 26.33 |========================================= Neural Magic DeepSparse 1.6 Model: BERT-Large, NLP Question Answering - Scenario: Asynchronous Multi-Stream ms/batch < Lower Is Better Initial test 1 No water cool . 303.33 |======================================== Neural Magic DeepSparse 1.6 Model: BERT-Large, NLP Question Answering - Scenario: Synchronous Single-Stream items/sec > Higher Is Better Initial test 1 No water cool . 18.51 |========================================= Neural Magic DeepSparse 1.6 Model: BERT-Large, NLP Question Answering - Scenario: Synchronous Single-Stream ms/batch < Lower Is Better Initial test 1 No water cool . 54.00 |========================================= Neural Magic DeepSparse 1.6 Model: CV Classification, ResNet-50 ImageNet - Scenario: Asynchronous Multi-Stream items/sec > Higher Is Better Initial test 1 No water cool . 265.20 |======================================== Neural Magic DeepSparse 1.6 Model: CV Classification, ResNet-50 ImageNet - Scenario: Asynchronous Multi-Stream ms/batch < Lower Is Better Initial test 1 No water cool . 30.15 |========================================= Neural Magic DeepSparse 1.6 Model: CV Classification, ResNet-50 ImageNet - Scenario: Synchronous Single-Stream items/sec > Higher Is Better Initial test 1 No water cool . 173.90 |======================================== Neural Magic DeepSparse 1.6 Model: CV Classification, ResNet-50 ImageNet - Scenario: Synchronous Single-Stream ms/batch < Lower Is Better Initial test 1 No water cool . 5.7436 |======================================== Neural Magic DeepSparse 1.6 Model: CV Detection, YOLOv5s COCO, Sparse INT8 - Scenario: Asynchronous Multi-Stream items/sec > Higher Is Better Initial test 1 No water cool . 115.11 |======================================== Neural Magic DeepSparse 1.6 Model: CV Detection, YOLOv5s COCO, Sparse INT8 - Scenario: Asynchronous Multi-Stream ms/batch < Lower Is Better Initial test 1 No water cool . 69.44 |========================================= Neural Magic DeepSparse 1.6 Model: CV Detection, YOLOv5s COCO, Sparse INT8 - Scenario: Synchronous Single-Stream items/sec > Higher Is Better Initial test 1 No water cool . 92.51 |========================================= Neural Magic DeepSparse 1.6 Model: CV Detection, YOLOv5s COCO, Sparse INT8 - Scenario: Synchronous Single-Stream ms/batch < Lower Is Better Initial test 1 No water cool . 10.80 |========================================= Neural Magic DeepSparse 1.6 Model: NLP Text Classification, DistilBERT mnli - Scenario: Asynchronous Multi-Stream items/sec > Higher Is Better Initial test 1 No water cool . 182.62 |======================================== Neural Magic DeepSparse 1.6 Model: NLP Text Classification, DistilBERT mnli - Scenario: Asynchronous Multi-Stream ms/batch < Lower Is Better Initial test 1 No water cool . 43.78 |========================================= Neural Magic DeepSparse 1.6 Model: NLP Text Classification, DistilBERT mnli - Scenario: Synchronous Single-Stream items/sec > Higher Is Better Initial test 1 No water cool . 96.47 |========================================= Neural Magic DeepSparse 1.6 Model: NLP Text Classification, DistilBERT mnli - Scenario: Synchronous Single-Stream ms/batch < Lower Is Better Initial test 1 No water cool . 10.36 |========================================= Neural Magic DeepSparse 1.6 Model: CV Segmentation, 90% Pruned YOLACT Pruned - Scenario: Asynchronous Multi-Stream items/sec > Higher Is Better Initial test 1 No water cool . 33.81 |========================================= Neural Magic DeepSparse 1.6 Model: CV Segmentation, 90% Pruned YOLACT Pruned - Scenario: Asynchronous Multi-Stream ms/batch < Lower Is Better Initial test 1 No water cool . 236.31 |======================================== Neural Magic DeepSparse 1.6 Model: CV Segmentation, 90% Pruned YOLACT Pruned - Scenario: Synchronous Single-Stream items/sec > Higher Is Better Initial test 1 No water cool . 27.53 |========================================= Neural Magic DeepSparse 1.6 Model: CV Segmentation, 90% Pruned YOLACT Pruned - Scenario: Synchronous Single-Stream ms/batch < Lower Is Better Initial test 1 No water cool . 36.31 |========================================= Neural Magic DeepSparse 1.6 Model: BERT-Large, NLP Question Answering, Sparse INT8 - Scenario: Asynchronous Multi-Stream items/sec > Higher Is Better Initial test 1 No water cool . 417.17 |======================================== Neural Magic DeepSparse 1.6 Model: BERT-Large, NLP Question Answering, Sparse INT8 - Scenario: Asynchronous Multi-Stream ms/batch < Lower Is Better Initial test 1 No water cool . 19.16 |========================================= Neural Magic DeepSparse 1.6 Model: BERT-Large, NLP Question Answering, Sparse INT8 - Scenario: Synchronous Single-Stream items/sec > Higher Is Better Initial test 1 No water cool . 98.08 |========================================= Neural Magic DeepSparse 1.6 Model: BERT-Large, NLP Question Answering, Sparse INT8 - Scenario: Synchronous Single-Stream ms/batch < Lower Is Better Initial test 1 No water cool . 10.19 |========================================= Neural Magic DeepSparse 1.6 Model: NLP Token Classification, BERT base uncased conll2003 - Scenario: Asynchronous Multi-Stream items/sec > Higher Is Better Initial test 1 No water cool . 19.91 |========================================= Neural Magic DeepSparse 1.6 Model: NLP Token Classification, BERT base uncased conll2003 - Scenario: Asynchronous Multi-Stream ms/batch < Lower Is Better Initial test 1 No water cool . 400.39 |======================================== Neural Magic DeepSparse 1.6 Model: NLP Token Classification, BERT base uncased conll2003 - Scenario: Synchronous Single-Stream items/sec > Higher Is Better Initial test 1 No water cool . 17.29 |========================================= Neural Magic DeepSparse 1.6 Model: NLP Token Classification, BERT base uncased conll2003 - Scenario: Synchronous Single-Stream ms/batch < Lower Is Better Initial test 1 No water cool . 57.83 |========================================= spaCy 3.4.1 Model: en_core_web_lg tokens/sec > Higher Is Better Initial test 1 No water cool . 18557 |========================================= spaCy 3.4.1 Model: en_core_web_trf tokens/sec > Higher Is Better Initial test 1 No water cool . 2415 |========================================== Caffe 2020-02-13 Model: AlexNet - Acceleration: CPU - Iterations: 100 Milli-Seconds < Lower Is Better Caffe 2020-02-13 Model: AlexNet - Acceleration: CPU - Iterations: 200 Milli-Seconds < Lower Is Better Caffe 2020-02-13 Model: AlexNet - Acceleration: CPU - Iterations: 1000 Milli-Seconds < Lower Is Better Caffe 2020-02-13 Model: GoogleNet - Acceleration: CPU - Iterations: 100 Milli-Seconds < Lower Is Better Caffe 2020-02-13 Model: GoogleNet - Acceleration: CPU - Iterations: 200 Milli-Seconds < Lower Is Better Caffe 2020-02-13 Model: GoogleNet - Acceleration: CPU - Iterations: 1000 Milli-Seconds < Lower Is Better Mobile Neural Network 2.1 Model: nasnet ms < Lower Is Better Initial test 1 No water cool . 11.30 |========================================= Mobile Neural Network 2.1 Model: mobilenetV3 ms < Lower Is Better Initial test 1 No water cool . 1.638 |========================================= Mobile Neural Network 2.1 Model: squeezenetv1.1 ms < Lower Is Better Initial test 1 No water cool . 2.542 |========================================= Mobile Neural Network 2.1 Model: resnet-v2-50 ms < Lower Is Better Initial test 1 No water cool . 12.12 |========================================= Mobile Neural Network 2.1 Model: SqueezeNetV1.0 ms < Lower Is Better Initial test 1 No water cool . 4.141 |========================================= Mobile Neural Network 2.1 Model: MobileNetV2_224 ms < Lower Is Better Initial test 1 No water cool . 3.410 |========================================= Mobile Neural Network 2.1 Model: mobilenet-v1-1.0 ms < Lower Is Better Initial test 1 No water cool . 2.456 |========================================= Mobile Neural Network 2.1 Model: inception-v3 ms < Lower Is Better Initial test 1 No water cool . 23.42 |========================================= NCNN 20230517 Target: CPU - Model: mobilenet ms < Lower Is Better Initial test 1 No water cool . 9.80 |========================================== NCNN 20230517 Target: CPU-v2-v2 - Model: mobilenet-v2 ms < Lower Is Better Initial test 1 No water cool . 3.65 |========================================== NCNN 20230517 Target: CPU-v3-v3 - Model: mobilenet-v3 ms < Lower Is Better Initial test 1 No water cool . 3.69 |========================================== NCNN 20230517 Target: CPU - Model: shufflenet-v2 ms < Lower Is Better Initial test 1 No water cool . 3.93 |========================================== NCNN 20230517 Target: CPU - Model: mnasnet ms < Lower Is Better Initial test 1 No water cool . 3.46 |========================================== NCNN 20230517 Target: CPU - Model: efficientnet-b0 ms < Lower Is Better Initial test 1 No water cool . 4.50 |========================================== NCNN 20230517 Target: CPU - Model: blazeface ms < Lower Is Better Initial test 1 No water cool . 1.60 |========================================== NCNN 20230517 Target: CPU - Model: googlenet ms < Lower Is Better Initial test 1 No water cool . 9.56 |========================================== NCNN 20230517 Target: CPU - Model: vgg16 ms < Lower Is Better Initial test 1 No water cool . 32.60 |========================================= NCNN 20230517 Target: CPU - Model: resnet18 ms < Lower Is Better Initial test 1 No water cool . 6.62 |========================================== NCNN 20230517 Target: CPU - Model: alexnet ms < Lower Is Better Initial test 1 No water cool . 5.52 |========================================== NCNN 20230517 Target: CPU - Model: resnet50 ms < Lower Is Better Initial test 1 No water cool . 13.48 |========================================= NCNN 20230517 Target: CPUv2-yolov3v2-yolov3 - Model: mobilenetv2-yolov3 ms < Lower Is Better Initial test 1 No water cool . 9.80 |========================================== NCNN 20230517 Target: CPU - Model: yolov4-tiny ms < Lower Is Better Initial test 1 No water cool . 16.28 |========================================= NCNN 20230517 Target: CPU - Model: squeezenet_ssd ms < Lower Is Better Initial test 1 No water cool . 8.49 |========================================== NCNN 20230517 Target: CPU - Model: regnety_400m ms < Lower Is Better Initial test 1 No water cool . 9.87 |========================================== NCNN 20230517 Target: CPU - Model: vision_transformer ms < Lower Is Better Initial test 1 No water cool . 37.92 |========================================= NCNN 20230517 Target: CPU - Model: FastestDet ms < Lower Is Better Initial test 1 No water cool . 4.69 |========================================== NCNN 20230517 Target: Vulkan GPU - Model: mobilenet ms < Lower Is Better Initial test 1 No water cool . 9.36 |========================================== NCNN 20230517 Target: Vulkan GPU-v2-v2 - Model: mobilenet-v2 ms < Lower Is Better Initial test 1 No water cool . 3.69 |========================================== NCNN 20230517 Target: Vulkan GPU-v3-v3 - Model: mobilenet-v3 ms < Lower Is Better Initial test 1 No water cool . 3.72 |========================================== NCNN 20230517 Target: Vulkan GPU - Model: shufflenet-v2 ms < Lower Is Better Initial test 1 No water cool . 3.90 |========================================== NCNN 20230517 Target: Vulkan GPU - Model: mnasnet ms < Lower Is Better Initial test 1 No water cool . 3.45 |========================================== NCNN 20230517 Target: Vulkan GPU - Model: efficientnet-b0 ms < Lower Is Better Initial test 1 No water cool . 4.57 |========================================== NCNN 20230517 Target: Vulkan GPU - Model: blazeface ms < Lower Is Better Initial test 1 No water cool . 1.62 |========================================== NCNN 20230517 Target: Vulkan GPU - Model: googlenet ms < Lower Is Better Initial test 1 No water cool . 9.69 |========================================== NCNN 20230517 Target: Vulkan GPU - Model: vgg16 ms < Lower Is Better Initial test 1 No water cool . 32.31 |========================================= NCNN 20230517 Target: Vulkan GPU - Model: resnet18 ms < Lower Is Better Initial test 1 No water cool . 6.65 |========================================== NCNN 20230517 Target: Vulkan GPU - Model: alexnet ms < Lower Is Better Initial test 1 No water cool . 5.67 |========================================== NCNN 20230517 Target: Vulkan GPU - Model: resnet50 ms < Lower Is Better Initial test 1 No water cool . 13.16 |========================================= NCNN 20230517 Target: Vulkan GPUv2-yolov3v2-yolov3 - Model: mobilenetv2-yolov3 ms < Lower Is Better Initial test 1 No water cool . 9.36 |========================================== NCNN 20230517 Target: Vulkan GPU - Model: yolov4-tiny ms < Lower Is Better Initial test 1 No water cool . 15.84 |========================================= NCNN 20230517 Target: Vulkan GPU - Model: squeezenet_ssd ms < Lower Is Better Initial test 1 No water cool . 8.38 |========================================== NCNN 20230517 Target: Vulkan GPU - Model: regnety_400m ms < Lower Is Better Initial test 1 No water cool . 9.83 |========================================== NCNN 20230517 Target: Vulkan GPU - Model: vision_transformer ms < Lower Is Better Initial test 1 No water cool . 38.12 |========================================= NCNN 20230517 Target: Vulkan GPU - Model: FastestDet ms < Lower Is Better Initial test 1 No water cool . 4.30 |========================================== TNN 0.3 Target: CPU - Model: DenseNet ms < Lower Is Better Initial test 1 No water cool . 2005.61 |======================================= TNN 0.3 Target: CPU - Model: MobileNet v2 ms < Lower Is Better Initial test 1 No water cool . 183.28 |======================================== TNN 0.3 Target: CPU - Model: SqueezeNet v2 ms < Lower Is Better Initial test 1 No water cool . 42.22 |========================================= TNN 0.3 Target: CPU - Model: SqueezeNet v1.1 ms < Lower Is Better Initial test 1 No water cool . 179.68 |======================================== PlaidML FP16: No - Mode: Inference - Network: VGG16 - Device: CPU Examples Per Second > Higher Is Better PlaidML FP16: No - Mode: Inference - Network: ResNet 50 - Device: CPU Examples Per Second > Higher Is Better OpenVINO 2024.0 Model: Face Detection FP16 - Device: CPU FPS > Higher Is Better Initial test 1 No water cool . 12.75 |========================================= OpenVINO 2024.0 Model: Face Detection FP16 - Device: CPU ms < Lower Is Better Initial test 1 No water cool . 625.66 |======================================== OpenVINO 2024.0 Model: Person Detection FP16 - Device: CPU FPS > Higher Is Better Initial test 1 No water cool . 76.65 |========================================= OpenVINO 2024.0 Model: Person Detection FP16 - Device: CPU ms < Lower Is Better Initial test 1 No water cool . 104.27 |======================================== OpenVINO 2024.0 Model: Person Detection FP32 - Device: CPU FPS > Higher Is Better Initial test 1 No water cool . 77.54 |========================================= OpenVINO 2024.0 Model: Person Detection FP32 - Device: CPU ms < Lower Is Better Initial test 1 No water cool . 103.09 |======================================== OpenVINO 2024.0 Model: Vehicle Detection FP16 - Device: CPU FPS > Higher Is Better Initial test 1 No water cool . 618.41 |======================================== OpenVINO 2024.0 Model: Vehicle Detection FP16 - Device: CPU ms < Lower Is Better Initial test 1 No water cool . 12.91 |========================================= OpenVINO 2024.0 Model: Face Detection FP16-INT8 - Device: CPU FPS > Higher Is Better Initial test 1 No water cool . 24.71 |========================================= OpenVINO 2024.0 Model: Face Detection FP16-INT8 - Device: CPU ms < Lower Is Better Initial test 1 No water cool . 323.12 |======================================== OpenVINO 2024.0 Model: Face Detection Retail FP16 - Device: CPU FPS > Higher Is Better Initial test 1 No water cool . 3062.63 |======================================= OpenVINO 2024.0 Model: Face Detection Retail FP16 - Device: CPU ms < Lower Is Better Initial test 1 No water cool . 2.53 |========================================== OpenVINO 2024.0 Model: Road Segmentation ADAS FP16 - Device: CPU FPS > Higher Is Better Initial test 1 No water cool . 272.36 |======================================== OpenVINO 2024.0 Model: Road Segmentation ADAS FP16 - Device: CPU ms < Lower Is Better Initial test 1 No water cool . 29.32 |========================================= OpenVINO 2024.0 Model: Vehicle Detection FP16-INT8 - Device: CPU FPS > Higher Is Better Initial test 1 No water cool . 1538.27 |======================================= OpenVINO 2024.0 Model: Vehicle Detection FP16-INT8 - Device: CPU ms < Lower Is Better Initial test 1 No water cool . 5.18 |========================================== OpenVINO 2024.0 Model: Weld Porosity Detection FP16 - Device: CPU FPS > Higher Is Better Initial test 1 No water cool . 1266.85 |======================================= OpenVINO 2024.0 Model: Weld Porosity Detection FP16 - Device: CPU ms < Lower Is Better Initial test 1 No water cool . 12.61 |========================================= OpenVINO 2024.0 Model: Face Detection Retail FP16-INT8 - Device: CPU FPS > Higher Is Better Initial test 1 No water cool . 4335.66 |======================================= OpenVINO 2024.0 Model: Face Detection Retail FP16-INT8 - Device: CPU ms < Lower Is Better Initial test 1 No water cool . 3.61 |========================================== OpenVINO 2024.0 Model: Road Segmentation ADAS FP16-INT8 - Device: CPU FPS > Higher Is Better Initial test 1 No water cool . 448.28 |======================================== OpenVINO 2024.0 Model: Road Segmentation ADAS FP16-INT8 - Device: CPU ms < Lower Is Better Initial test 1 No water cool . 17.81 |========================================= OpenVINO 2024.0 Model: Machine Translation EN To DE FP16 - Device: CPU FPS > Higher Is Better Initial test 1 No water cool . 121.96 |======================================== OpenVINO 2024.0 Model: Machine Translation EN To DE FP16 - Device: CPU ms < Lower Is Better Initial test 1 No water cool . 65.52 |========================================= OpenVINO 2024.0 Model: Weld Porosity Detection FP16-INT8 - Device: CPU FPS > Higher Is Better Initial test 1 No water cool . 2470.86 |======================================= OpenVINO 2024.0 Model: Weld Porosity Detection FP16-INT8 - Device: CPU ms < Lower Is Better Initial test 1 No water cool . 6.44 |========================================== OpenVINO 2024.0 Model: Person Vehicle Bike Detection FP16 - Device: CPU FPS > Higher Is Better Initial test 1 No water cool . 1442.07 |======================================= OpenVINO 2024.0 Model: Person Vehicle Bike Detection FP16 - Device: CPU ms < Lower Is Better Initial test 1 No water cool . 5.53 |========================================== OpenVINO 2024.0 Model: Noise Suppression Poconet-Like FP16 - Device: CPU FPS > Higher Is Better Initial test 1 No water cool . 1386.93 |======================================= OpenVINO 2024.0 Model: Noise Suppression Poconet-Like FP16 - Device: CPU ms < Lower Is Better Initial test 1 No water cool . 11.33 |========================================= OpenVINO 2024.0 Model: Handwritten English Recognition FP16 - Device: CPU FPS > Higher Is Better Initial test 1 No water cool . 667.29 |======================================== OpenVINO 2024.0 Model: Handwritten English Recognition FP16 - Device: CPU ms < Lower Is Better Initial test 1 No water cool . 23.94 |========================================= OpenVINO 2024.0 Model: Person Re-Identification Retail FP16 - Device: CPU FPS > Higher Is Better Initial test 1 No water cool . 1785.48 |======================================= OpenVINO 2024.0 Model: Person Re-Identification Retail FP16 - Device: CPU ms < Lower Is Better Initial test 1 No water cool . 4.46 |========================================== OpenVINO 2024.0 Model: Age Gender Recognition Retail 0013 FP16 - Device: CPU FPS > Higher Is Better Initial test 1 No water cool . 32402.42 |====================================== OpenVINO 2024.0 Model: Age Gender Recognition Retail 0013 FP16 - Device: CPU ms < Lower Is Better Initial test 1 No water cool . 0.45 |========================================== OpenVINO 2024.0 Model: Handwritten English Recognition FP16-INT8 - Device: CPU FPS > Higher Is Better Initial test 1 No water cool . 729.99 |======================================== OpenVINO 2024.0 Model: Handwritten English Recognition FP16-INT8 - Device: CPU ms < Lower Is Better Initial test 1 No water cool . 21.88 |========================================= OpenVINO 2024.0 Model: Age Gender Recognition Retail 0013 FP16-INT8 - Device: CPU FPS > Higher Is Better Initial test 1 No water cool . 46025.94 |====================================== OpenVINO 2024.0 Model: Age Gender Recognition Retail 0013 FP16-INT8 - Device: CPU ms < Lower Is Better Initial test 1 No water cool . 0.31 |========================================== Numenta Anomaly Benchmark 1.1 Detector: KNN CAD Seconds < Lower Is Better Initial test 1 No water cool . 105.00 |======================================== Numenta Anomaly Benchmark 1.1 Detector: Relative Entropy Seconds < Lower Is Better Initial test 1 No water cool . 8.281 |========================================= Numenta Anomaly Benchmark 1.1 Detector: Windowed Gaussian Seconds < Lower Is Better Initial test 1 No water cool . 4.984 |========================================= Numenta Anomaly Benchmark 1.1 Detector: Earthgecko Skyline Seconds < Lower Is Better Initial test 1 No water cool . 55.57 |========================================= Numenta Anomaly Benchmark 1.1 Detector: Bayesian Changepoint Seconds < Lower Is Better Initial test 1 No water cool . 13.21 |========================================= Numenta Anomaly Benchmark 1.1 Detector: Contextual Anomaly Detector OSE Seconds < Lower Is Better Initial test 1 No water cool . 25.40 |========================================= ONNX Runtime 1.17 Model: GPT-2 - Device: CPU - Executor: Parallel Inferences Per Second > Higher Is Better ONNX Runtime 1.17 Model: GPT-2 - Device: CPU - Executor: Standard Inferences Per Second > Higher Is Better ONNX Runtime 1.17 Model: yolov4 - Device: CPU - Executor: Parallel Inferences Per Second > Higher Is Better ONNX Runtime 1.17 Model: yolov4 - Device: CPU - Executor: Standard Inferences Per Second > Higher Is Better ONNX Runtime 1.17 Model: T5 Encoder - Device: CPU - Executor: Parallel Inferences Per Second > Higher Is Better ONNX Runtime 1.17 Model: T5 Encoder - Device: CPU - Executor: Standard Inferences Per Second > Higher Is Better ONNX Runtime 1.17 Model: bertsquad-12 - Device: CPU - Executor: Parallel Inferences Per Second > Higher Is Better ONNX Runtime 1.17 Model: bertsquad-12 - Device: CPU - Executor: Standard Inferences Per Second > Higher Is Better ONNX Runtime 1.17 Model: CaffeNet 12-int8 - Device: CPU - Executor: Parallel Inferences Per Second > Higher Is Better ONNX Runtime 1.17 Model: CaffeNet 12-int8 - Device: CPU - Executor: Standard Inferences Per Second > Higher Is Better ONNX Runtime 1.17 Model: fcn-resnet101-11 - Device: CPU - Executor: Parallel Inferences Per Second > Higher Is Better ONNX Runtime 1.17 Model: fcn-resnet101-11 - Device: CPU - Executor: Standard Inferences Per Second > Higher Is Better ONNX Runtime 1.17 Model: ArcFace ResNet-100 - Device: CPU - Executor: Parallel Inferences Per Second > Higher Is Better ONNX Runtime 1.17 Model: ArcFace ResNet-100 - Device: CPU - Executor: Standard Inferences Per Second > Higher Is Better ONNX Runtime 1.17 Model: ResNet50 v1-12-int8 - Device: CPU - Executor: Parallel Inferences Per Second > Higher Is Better ONNX Runtime 1.17 Model: ResNet50 v1-12-int8 - Device: CPU - Executor: Standard Inferences Per Second > Higher Is Better ONNX Runtime 1.17 Model: super-resolution-10 - Device: CPU - Executor: Parallel Inferences Per Second > Higher Is Better ONNX Runtime 1.17 Model: super-resolution-10 - Device: CPU - Executor: Standard Inferences Per Second > Higher Is Better ONNX Runtime 1.17 Model: Faster R-CNN R-50-FPN-int8 - Device: CPU - Executor: Parallel Inferences Per Second > Higher Is Better ONNX Runtime 1.17 Model: Faster R-CNN R-50-FPN-int8 - Device: CPU - Executor: Standard Inferences Per Second > Higher Is Better AI Benchmark Alpha 0.1.2 Device Inference Score Score > Higher Is Better Initial test 1 No water cool . 2900 |========================================== AI Benchmark Alpha 0.1.2 Device Training Score Score > Higher Is Better Initial test 1 No water cool . 3573 |========================================== AI Benchmark Alpha 0.1.2 Device AI Score Score > Higher Is Better Initial test 1 No water cool . 6473 |========================================== Mlpack Benchmark Benchmark: scikit_ica Seconds < Lower Is Better Initial test 1 No water cool . 30.12 |========================================= Mlpack Benchmark Benchmark: scikit_qda Seconds < Lower Is Better Initial test 1 No water cool . 34.07 |========================================= Mlpack Benchmark Benchmark: scikit_svm Seconds < Lower Is Better Initial test 1 No water cool . 15.12 |========================================= Mlpack Benchmark Benchmark: scikit_linearridgeregression Seconds < Lower Is Better Initial test 1 No water cool . 1.03 |========================================== Scikit-Learn 1.2.2 Benchmark: GLM Seconds < Lower Is Better Scikit-Learn 1.2.2 Benchmark: SAGA Seconds < Lower Is Better Scikit-Learn 1.2.2 Benchmark: Tree Seconds < Lower Is Better Scikit-Learn 1.2.2 Benchmark: Lasso Seconds < Lower Is Better Scikit-Learn 1.2.2 Benchmark: Glmnet Seconds < Lower Is Better Scikit-Learn 1.2.2 Benchmark: Sparsify Seconds < Lower Is Better Scikit-Learn 1.2.2 Benchmark: Plot Ward Seconds < Lower Is Better Scikit-Learn 1.2.2 Benchmark: MNIST Dataset Seconds < Lower Is Better Scikit-Learn 1.2.2 Benchmark: Plot Neighbors Seconds < Lower Is Better Scikit-Learn 1.2.2 Benchmark: SGD Regression Seconds < Lower Is Better Scikit-Learn 1.2.2 Benchmark: SGDOneClassSVM Seconds < Lower Is Better Scikit-Learn 1.2.2 Benchmark: Plot Lasso Path Seconds < Lower Is Better Scikit-Learn 1.2.2 Benchmark: Isolation Forest Seconds < Lower Is Better Scikit-Learn 1.2.2 Benchmark: Plot Fast KMeans Seconds < Lower Is Better Scikit-Learn 1.2.2 Benchmark: Text Vectorizers Seconds < Lower Is Better Scikit-Learn 1.2.2 Benchmark: Plot Hierarchical Seconds < Lower Is Better Scikit-Learn 1.2.2 Benchmark: Plot OMP vs. LARS Seconds < Lower Is Better Scikit-Learn 1.2.2 Benchmark: Feature Expansions Seconds < Lower Is Better Scikit-Learn 1.2.2 Benchmark: LocalOutlierFactor Seconds < Lower Is Better Scikit-Learn 1.2.2 Benchmark: TSNE MNIST Dataset Seconds < Lower Is Better Scikit-Learn 1.2.2 Benchmark: Isotonic / Logistic Seconds < Lower Is Better Scikit-Learn 1.2.2 Benchmark: Plot Incremental PCA Seconds < Lower Is Better Scikit-Learn 1.2.2 Benchmark: Hist Gradient Boosting Seconds < Lower Is Better Scikit-Learn 1.2.2 Benchmark: Plot Parallel Pairwise Seconds < Lower Is Better Scikit-Learn 1.2.2 Benchmark: Isotonic / Pathological Seconds < Lower Is Better Scikit-Learn 1.2.2 Benchmark: RCV1 Logreg Convergencet Seconds < Lower Is Better Scikit-Learn 1.2.2 Benchmark: Sample Without Replacement Seconds < Lower Is Better Scikit-Learn 1.2.2 Benchmark: Covertype Dataset Benchmark Seconds < Lower Is Better Scikit-Learn 1.2.2 Benchmark: Hist Gradient Boosting Adult Seconds < Lower Is Better Scikit-Learn 1.2.2 Benchmark: Isotonic / Perturbed Logarithm Seconds < Lower Is Better Scikit-Learn 1.2.2 Benchmark: Hist Gradient Boosting Threading Seconds < Lower Is Better Scikit-Learn 1.2.2 Benchmark: Plot Singular Value Decomposition Seconds < Lower Is Better Scikit-Learn 1.2.2 Benchmark: Hist Gradient Boosting Higgs Boson Seconds < Lower Is Better Scikit-Learn 1.2.2 Benchmark: 20 Newsgroups / Logistic Regression Seconds < Lower Is Better Scikit-Learn 1.2.2 Benchmark: Plot Polynomial Kernel Approximation Seconds < Lower Is Better Scikit-Learn 1.2.2 Benchmark: Plot Non-Negative Matrix Factorization Seconds < Lower Is Better Scikit-Learn 1.2.2 Benchmark: Hist Gradient Boosting Categorical Only Seconds < Lower Is Better Scikit-Learn 1.2.2 Benchmark: Kernel PCA Solvers / Time vs. N Samples Seconds < Lower Is Better Scikit-Learn 1.2.2 Benchmark: Kernel PCA Solvers / Time vs. N Components Seconds < Lower Is Better Scikit-Learn 1.2.2 Benchmark: Sparse Random Projections / 100 Iterations Seconds < Lower Is Better Whisper.cpp 1.4 Model: ggml-base.en - Input: 2016 State of the Union Seconds < Lower Is Better Initial test 1 No water cool . 0.15350 |======================================= Whisper.cpp 1.4 Model: ggml-small.en - Input: 2016 State of the Union Seconds < Lower Is Better Initial test 1 No water cool . 0.34881 |======================================= Whisper.cpp 1.4 Model: ggml-medium.en - Input: 2016 State of the Union Seconds < Lower Is Better Initial test 1 No water cool . 0.86615 |======================================= Llama.cpp b1808 Model: llama-2-7b.Q4_0.gguf Tokens Per Second > Higher Is Better Llama.cpp b1808 Model: llama-2-13b.Q4_0.gguf Tokens Per Second > Higher Is Better Llama.cpp b1808 Model: llama-2-70b-chat.Q5_0.gguf Tokens Per Second > Higher Is Better Llamafile 0.6 Test: llava-v1.5-7b-q4 - Acceleration: CPU Tokens Per Second > Higher Is Better Llamafile 0.6 Test: mistral-7b-instruct-v0.2.Q8_0 - Acceleration: CPU Tokens Per Second > Higher Is Better Llamafile 0.6 Test: wizardcoder-python-34b-v1.0.Q6_K - Acceleration: CPU Tokens Per Second > Higher Is Better OpenCV 4.7 Test: DNN - Deep Neural Network ms < Lower Is Better Initial test 1 No water cool . 30277 |=========================================