102424machinelearningtest
Intel Core i9-12900K testing with a ASUS PRIME Z790-V AX (1802 BIOS) and ASUS NVIDIA w Dual GeForce RTX 3090 24GB w NVLink on Ubuntu 24.04 via the Phoronix Test Suite.


ASUS NVIDIA GeForce RTX 3090: 

	Processor: Intel Core i9-12900K @ 5.10GHz (16 Cores / 24 Threads), Motherboard: ASUS PRIME Z790-V AX (1802 BIOS), Chipset: Intel Raptor Lake-S PCH, Memory: 96GB, Disk: 2000GB Samsung SSD 970 EVO Plus 2TB, Graphics: ASUS NVIDIA GeForce RTX 3090 24GB, Audio: Intel Raptor Lake HD Audio, Monitor: S24F350, Network: Realtek RTL8111/8168/8211/8411 + Realtek Device b851

	OS: Ubuntu 24.04, Kernel: 6.8.0-47-generic (x86_64), Desktop: GNOME Shell 46.0, Display Server: X Server + Wayland, Display Driver: NVIDIA 560.35.03, OpenGL: 4.6.0, OpenCL: OpenCL 3.0 CUDA 12.6.65, Compiler: GCC 13.2.0 + CUDA 12.5, File-System: ext4, Screen Resolution: 1920x1080


SHOC Scalable HeterOgeneous Computing 2020-04-17
Target: OpenCL - Benchmark: S3D
GFLOPS > Higher Is Better
ASUS NVIDIA GeForce RTX 3090 . 455.96 |========================================


SHOC Scalable HeterOgeneous Computing 2020-04-17
Target: OpenCL - Benchmark: Triad
GB/s > Higher Is Better
ASUS NVIDIA GeForce RTX 3090 . 6.6057 |========================================


SHOC Scalable HeterOgeneous Computing 2020-04-17
Target: OpenCL - Benchmark: FFT SP
GFLOPS > Higher Is Better
ASUS NVIDIA GeForce RTX 3090 . 2510.98 |=======================================


SHOC Scalable HeterOgeneous Computing 2020-04-17
Target: OpenCL - Benchmark: MD5 Hash
GHash/s > Higher Is Better
ASUS NVIDIA GeForce RTX 3090 . 45.45 |=========================================


SHOC Scalable HeterOgeneous Computing 2020-04-17
Target: OpenCL - Benchmark: Reduction
GB/s > Higher Is Better
ASUS NVIDIA GeForce RTX 3090 . 406.41 |========================================


SHOC Scalable HeterOgeneous Computing 2020-04-17
Target: OpenCL - Benchmark: GEMM SGEMM_N
GFLOPS > Higher Is Better
ASUS NVIDIA GeForce RTX 3090 . 8456.99 |=======================================


SHOC Scalable HeterOgeneous Computing 2020-04-17
Target: OpenCL - Benchmark: Max SP Flops
GFLOPS > Higher Is Better
ASUS NVIDIA GeForce RTX 3090 . 41982.2 |=======================================


SHOC Scalable HeterOgeneous Computing 2020-04-17
Target: OpenCL - Benchmark: Bus Speed Download
GB/s > Higher Is Better
ASUS NVIDIA GeForce RTX 3090 . 6.6413 |========================================


SHOC Scalable HeterOgeneous Computing 2020-04-17
Target: OpenCL - Benchmark: Bus Speed Readback
GB/s > Higher Is Better
ASUS NVIDIA GeForce RTX 3090 . 6.7652 |========================================


SHOC Scalable HeterOgeneous Computing 2020-04-17
Target: OpenCL - Benchmark: Texture Read Bandwidth
GB/s > Higher Is Better
ASUS NVIDIA GeForce RTX 3090 . 2178.63 |=======================================


LeelaChessZero 0.31.1
Backend: BLAS
Nodes Per Second > Higher Is Better
ASUS NVIDIA GeForce RTX 3090 . 166 |===========================================


oneDNN 3.6
Harness: IP Shapes 1D - Engine: CPU
ms < Lower Is Better
ASUS NVIDIA GeForce RTX 3090 . 2.89253 |=======================================


oneDNN 3.6
Harness: IP Shapes 3D - Engine: CPU
ms < Lower Is Better
ASUS NVIDIA GeForce RTX 3090 . 9.32676 |=======================================


oneDNN 3.6
Harness: Convolution Batch Shapes Auto - Engine: CPU
ms < Lower Is Better
ASUS NVIDIA GeForce RTX 3090 . 8.24915 |=======================================


oneDNN 3.6
Harness: Deconvolution Batch shapes_1d - Engine: CPU
ms < Lower Is Better
ASUS NVIDIA GeForce RTX 3090 . 4.47259 |=======================================


oneDNN 3.6
Harness: Deconvolution Batch shapes_3d - Engine: CPU
ms < Lower Is Better
ASUS NVIDIA GeForce RTX 3090 . 5.57785 |=======================================


oneDNN 3.6
Harness: Recurrent Neural Network Training - Engine: CPU
ms < Lower Is Better
ASUS NVIDIA GeForce RTX 3090 . 2702.00 |=======================================


oneDNN 3.6
Harness: Recurrent Neural Network Inference - Engine: CPU
ms < Lower Is Better
ASUS NVIDIA GeForce RTX 3090 . 1452.76 |=======================================


Numpy Benchmark

Score > Higher Is Better
ASUS NVIDIA GeForce RTX 3090 . 644.54 |========================================


DeepSpeech 0.6
Acceleration: CPU
Seconds < Lower Is Better
ASUS NVIDIA GeForce RTX 3090 . 54.63 |=========================================


R Benchmark

Seconds < Lower Is Better
ASUS NVIDIA GeForce RTX 3090 . 0.0970 |========================================


RNNoise 0.2
Input: 26 Minute Long Talking Sample
Seconds < Lower Is Better
ASUS NVIDIA GeForce RTX 3090 . 7.245 |=========================================


TensorFlow Lite 2022-05-18
Model: SqueezeNet
Microseconds < Lower Is Better
ASUS NVIDIA GeForce RTX 3090 . 1904.01 |=======================================


TensorFlow Lite 2022-05-18
Model: Inception V4
Microseconds < Lower Is Better
ASUS NVIDIA GeForce RTX 3090 . 26961.0 |=======================================


TensorFlow Lite 2022-05-18
Model: NASNet Mobile
Microseconds < Lower Is Better
ASUS NVIDIA GeForce RTX 3090 . 311542 |========================================


TensorFlow Lite 2022-05-18
Model: Mobilenet Float
Microseconds < Lower Is Better
ASUS NVIDIA GeForce RTX 3090 . 1383.02 |=======================================


TensorFlow Lite 2022-05-18
Model: Mobilenet Quant
Microseconds < Lower Is Better
ASUS NVIDIA GeForce RTX 3090 . 2317.67 |=======================================


TensorFlow Lite 2022-05-18
Model: Inception ResNet V2
Microseconds < Lower Is Better
ASUS NVIDIA GeForce RTX 3090 . 128179 |========================================


PyTorch 2.2.1
Device: CPU - Batch Size: 1 - Model: ResNet-50
batches/sec > Higher Is Better
ASUS NVIDIA GeForce RTX 3090 . 49.49 |=========================================


PyTorch 2.2.1
Device: CPU - Batch Size: 1 - Model: ResNet-152
batches/sec > Higher Is Better
ASUS NVIDIA GeForce RTX 3090 . 18.80 |=========================================


PyTorch 2.2.1
Device: CPU - Batch Size: 16 - Model: ResNet-50
batches/sec > Higher Is Better
ASUS NVIDIA GeForce RTX 3090 . 30.07 |=========================================


PyTorch 2.2.1
Device: CPU - Batch Size: 32 - Model: ResNet-50
batches/sec > Higher Is Better
ASUS NVIDIA GeForce RTX 3090 . 30.27 |=========================================


PyTorch 2.2.1
Device: CPU - Batch Size: 64 - Model: ResNet-50
batches/sec > Higher Is Better
ASUS NVIDIA GeForce RTX 3090 . 29.52 |=========================================


PyTorch 2.2.1
Device: CPU - Batch Size: 16 - Model: ResNet-152
batches/sec > Higher Is Better
ASUS NVIDIA GeForce RTX 3090 . 11.49 |=========================================


PyTorch 2.2.1
Device: CPU - Batch Size: 256 - Model: ResNet-50
batches/sec > Higher Is Better
ASUS NVIDIA GeForce RTX 3090 . 29.33 |=========================================


PyTorch 2.2.1
Device: CPU - Batch Size: 32 - Model: ResNet-152
batches/sec > Higher Is Better
ASUS NVIDIA GeForce RTX 3090 . 10.16 |=========================================


PyTorch 2.2.1
Device: CPU - Batch Size: 512 - Model: ResNet-50
batches/sec > Higher Is Better
ASUS NVIDIA GeForce RTX 3090 . 30.14 |=========================================


PyTorch 2.2.1
Device: CPU - Batch Size: 64 - Model: ResNet-152
batches/sec > Higher Is Better
ASUS NVIDIA GeForce RTX 3090 . 11.66 |=========================================


PyTorch 2.2.1
Device: CPU - Batch Size: 256 - Model: ResNet-152
batches/sec > Higher Is Better
ASUS NVIDIA GeForce RTX 3090 . 11.45 |=========================================


PyTorch 2.2.1
Device: CPU - Batch Size: 512 - Model: ResNet-152
batches/sec > Higher Is Better
ASUS NVIDIA GeForce RTX 3090 . 11.46 |=========================================


PyTorch 2.2.1
Device: CPU - Batch Size: 1 - Model: Efficientnet_v2_l
batches/sec > Higher Is Better
ASUS NVIDIA GeForce RTX 3090 . 10.55 |=========================================


PyTorch 2.2.1
Device: CPU - Batch Size: 16 - Model: Efficientnet_v2_l
batches/sec > Higher Is Better
ASUS NVIDIA GeForce RTX 3090 . 6.97 |==========================================


PyTorch 2.2.1
Device: CPU - Batch Size: 32 - Model: Efficientnet_v2_l
batches/sec > Higher Is Better
ASUS NVIDIA GeForce RTX 3090 . 7.92 |==========================================


PyTorch 2.2.1
Device: CPU - Batch Size: 64 - Model: Efficientnet_v2_l
batches/sec > Higher Is Better
ASUS NVIDIA GeForce RTX 3090 . 7.82 |==========================================


PyTorch 2.2.1
Device: CPU - Batch Size: 256 - Model: Efficientnet_v2_l
batches/sec > Higher Is Better
ASUS NVIDIA GeForce RTX 3090 . 6.48 |==========================================


PyTorch 2.2.1
Device: CPU - Batch Size: 512 - Model: Efficientnet_v2_l
batches/sec > Higher Is Better
ASUS NVIDIA GeForce RTX 3090 . 7.26 |==========================================


PyTorch 2.2.1
Device: NVIDIA CUDA GPU - Batch Size: 1 - Model: ResNet-50
batches/sec > Higher Is Better
ASUS NVIDIA GeForce RTX 3090 . 469.71 |========================================


PyTorch 2.2.1
Device: NVIDIA CUDA GPU - Batch Size: 1 - Model: ResNet-152
batches/sec > Higher Is Better
ASUS NVIDIA GeForce RTX 3090 . 172.05 |========================================


PyTorch 2.2.1
Device: NVIDIA CUDA GPU - Batch Size: 16 - Model: ResNet-50
batches/sec > Higher Is Better
ASUS NVIDIA GeForce RTX 3090 . 414.29 |========================================


PyTorch 2.2.1
Device: NVIDIA CUDA GPU - Batch Size: 32 - Model: ResNet-50
batches/sec > Higher Is Better
ASUS NVIDIA GeForce RTX 3090 . 413.19 |========================================


PyTorch 2.2.1
Device: NVIDIA CUDA GPU - Batch Size: 64 - Model: ResNet-50
batches/sec > Higher Is Better
ASUS NVIDIA GeForce RTX 3090 . 413.29 |========================================


PyTorch 2.2.1
Device: NVIDIA CUDA GPU - Batch Size: 16 - Model: ResNet-152
batches/sec > Higher Is Better
ASUS NVIDIA GeForce RTX 3090 . 162.59 |========================================


PyTorch 2.2.1
Device: NVIDIA CUDA GPU - Batch Size: 256 - Model: ResNet-50
batches/sec > Higher Is Better
ASUS NVIDIA GeForce RTX 3090 . 413.12 |========================================


PyTorch 2.2.1
Device: NVIDIA CUDA GPU - Batch Size: 32 - Model: ResNet-152
batches/sec > Higher Is Better
ASUS NVIDIA GeForce RTX 3090 . 162.45 |========================================


PyTorch 2.2.1
Device: NVIDIA CUDA GPU - Batch Size: 512 - Model: ResNet-50
batches/sec > Higher Is Better
ASUS NVIDIA GeForce RTX 3090 . 414.51 |========================================


PyTorch 2.2.1
Device: NVIDIA CUDA GPU - Batch Size: 64 - Model: ResNet-152
batches/sec > Higher Is Better
ASUS NVIDIA GeForce RTX 3090 . 162.69 |========================================


PyTorch 2.2.1
Device: NVIDIA CUDA GPU - Batch Size: 256 - Model: ResNet-152
batches/sec > Higher Is Better
ASUS NVIDIA GeForce RTX 3090 . 162.68 |========================================


PyTorch 2.2.1
Device: NVIDIA CUDA GPU - Batch Size: 512 - Model: ResNet-152
batches/sec > Higher Is Better
ASUS NVIDIA GeForce RTX 3090 . 162.35 |========================================


PyTorch 2.2.1
Device: NVIDIA CUDA GPU - Batch Size: 1 - Model: Efficientnet_v2_l
batches/sec > Higher Is Better
ASUS NVIDIA GeForce RTX 3090 . 88.41 |=========================================


PyTorch 2.2.1
Device: NVIDIA CUDA GPU - Batch Size: 16 - Model: Efficientnet_v2_l
batches/sec > Higher Is Better
ASUS NVIDIA GeForce RTX 3090 . 85.74 |=========================================


PyTorch 2.2.1
Device: NVIDIA CUDA GPU - Batch Size: 32 - Model: Efficientnet_v2_l
batches/sec > Higher Is Better
ASUS NVIDIA GeForce RTX 3090 . 84.09 |=========================================


PyTorch 2.2.1
Device: NVIDIA CUDA GPU - Batch Size: 64 - Model: Efficientnet_v2_l
batches/sec > Higher Is Better
ASUS NVIDIA GeForce RTX 3090 . 85.37 |=========================================


PyTorch 2.2.1
Device: NVIDIA CUDA GPU - Batch Size: 256 - Model: Efficientnet_v2_l
batches/sec > Higher Is Better
ASUS NVIDIA GeForce RTX 3090 . 85.57 |=========================================


PyTorch 2.2.1
Device: NVIDIA CUDA GPU - Batch Size: 512 - Model: Efficientnet_v2_l
batches/sec > Higher Is Better
ASUS NVIDIA GeForce RTX 3090 . 86.27 |=========================================


TensorFlow 2.16.1
Device: CPU - Batch Size: 1 - Model: VGG-16
images/sec > Higher Is Better
ASUS NVIDIA GeForce RTX 3090 . 4.12 |==========================================


TensorFlow 2.16.1
Device: GPU - Batch Size: 1 - Model: VGG-16
images/sec > Higher Is Better
ASUS NVIDIA GeForce RTX 3090 . 1.57 |==========================================


TensorFlow 2.16.1
Device: CPU - Batch Size: 1 - Model: AlexNet
images/sec > Higher Is Better
ASUS NVIDIA GeForce RTX 3090 . 15.83 |=========================================


TensorFlow 2.16.1
Device: CPU - Batch Size: 16 - Model: VGG-16
images/sec > Higher Is Better
ASUS NVIDIA GeForce RTX 3090 . 8.97 |==========================================


TensorFlow 2.16.1
Device: CPU - Batch Size: 32 - Model: VGG-16
images/sec > Higher Is Better
ASUS NVIDIA GeForce RTX 3090 . 9.22 |==========================================


TensorFlow 2.16.1
Device: CPU - Batch Size: 64 - Model: VGG-16
images/sec > Higher Is Better
ASUS NVIDIA GeForce RTX 3090 . 9.31 |==========================================


TensorFlow 2.16.1
Device: GPU - Batch Size: 1 - Model: AlexNet
images/sec > Higher Is Better
ASUS NVIDIA GeForce RTX 3090 . 14.14 |=========================================


TensorFlow 2.16.1
Device: GPU - Batch Size: 16 - Model: VGG-16
images/sec > Higher Is Better
ASUS NVIDIA GeForce RTX 3090 . 2.25 |==========================================


TensorFlow 2.16.1
Device: GPU - Batch Size: 32 - Model: VGG-16
images/sec > Higher Is Better
ASUS NVIDIA GeForce RTX 3090 . 2.27 |==========================================


TensorFlow 2.16.1
Device: GPU - Batch Size: 64 - Model: VGG-16
images/sec > Higher Is Better
ASUS NVIDIA GeForce RTX 3090 . 2.27 |==========================================


TensorFlow 2.16.1
Device: CPU - Batch Size: 16 - Model: AlexNet
images/sec > Higher Is Better
ASUS NVIDIA GeForce RTX 3090 . 129.70 |========================================


TensorFlow 2.16.1
Device: CPU - Batch Size: 256 - Model: VGG-16
images/sec > Higher Is Better
ASUS NVIDIA GeForce RTX 3090 . 9.11 |==========================================


TensorFlow 2.16.1
Device: CPU - Batch Size: 32 - Model: AlexNet
images/sec > Higher Is Better
ASUS NVIDIA GeForce RTX 3090 . 167.17 |========================================


TensorFlow 2.16.1
Device: CPU - Batch Size: 512 - Model: VGG-16
images/sec > Higher Is Better
ASUS NVIDIA GeForce RTX 3090 . 8.95 |==========================================


TensorFlow 2.16.1
Device: CPU - Batch Size: 64 - Model: AlexNet
images/sec > Higher Is Better
ASUS NVIDIA GeForce RTX 3090 . 192.13 |========================================


TensorFlow 2.16.1
Device: GPU - Batch Size: 16 - Model: AlexNet
images/sec > Higher Is Better
ASUS NVIDIA GeForce RTX 3090 . 38.93 |=========================================


TensorFlow 2.16.1
Device: GPU - Batch Size: 256 - Model: VGG-16
images/sec > Higher Is Better
ASUS NVIDIA GeForce RTX 3090 . 2.28 |==========================================


TensorFlow 2.16.1
Device: GPU - Batch Size: 32 - Model: AlexNet
images/sec > Higher Is Better
ASUS NVIDIA GeForce RTX 3090 . 41.45 |=========================================


TensorFlow 2.16.1
Device: GPU - Batch Size: 512 - Model: VGG-16
images/sec > Higher Is Better
ASUS NVIDIA GeForce RTX 3090 . 2.29 |==========================================


TensorFlow 2.16.1
Device: GPU - Batch Size: 64 - Model: AlexNet
images/sec > Higher Is Better
ASUS NVIDIA GeForce RTX 3090 . 42.49 |=========================================


TensorFlow 2.16.1
Device: CPU - Batch Size: 1 - Model: GoogLeNet
images/sec > Higher Is Better
ASUS NVIDIA GeForce RTX 3090 . 47.92 |=========================================


TensorFlow 2.16.1
Device: CPU - Batch Size: 1 - Model: ResNet-50
images/sec > Higher Is Better
ASUS NVIDIA GeForce RTX 3090 . 14.04 |=========================================


TensorFlow 2.16.1
Device: CPU - Batch Size: 256 - Model: AlexNet
images/sec > Higher Is Better
ASUS NVIDIA GeForce RTX 3090 . 227.40 |========================================


TensorFlow 2.16.1
Device: CPU - Batch Size: 512 - Model: AlexNet
images/sec > Higher Is Better
ASUS NVIDIA GeForce RTX 3090 . 245.67 |========================================


TensorFlow 2.16.1
Device: GPU - Batch Size: 1 - Model: GoogLeNet
images/sec > Higher Is Better
ASUS NVIDIA GeForce RTX 3090 . 12.00 |=========================================


TensorFlow 2.16.1
Device: GPU - Batch Size: 1 - Model: ResNet-50
images/sec > Higher Is Better
ASUS NVIDIA GeForce RTX 3090 . 5.62 |==========================================


TensorFlow 2.16.1
Device: GPU - Batch Size: 256 - Model: AlexNet
images/sec > Higher Is Better
ASUS NVIDIA GeForce RTX 3090 . 44.05 |=========================================


TensorFlow 2.16.1
Device: GPU - Batch Size: 512 - Model: AlexNet
images/sec > Higher Is Better
ASUS NVIDIA GeForce RTX 3090 . 44.64 |=========================================


TensorFlow 2.16.1
Device: CPU - Batch Size: 16 - Model: GoogLeNet
images/sec > Higher Is Better
ASUS NVIDIA GeForce RTX 3090 . 101.50 |========================================


TensorFlow 2.16.1
Device: CPU - Batch Size: 16 - Model: ResNet-50
images/sec > Higher Is Better
ASUS NVIDIA GeForce RTX 3090 . 27.80 |=========================================


TensorFlow 2.16.1
Device: CPU - Batch Size: 32 - Model: GoogLeNet
images/sec > Higher Is Better
ASUS NVIDIA GeForce RTX 3090 . 96.69 |=========================================


TensorFlow 2.16.1
Device: CPU - Batch Size: 32 - Model: ResNet-50
images/sec > Higher Is Better
ASUS NVIDIA GeForce RTX 3090 . 26.82 |=========================================


TensorFlow 2.16.1
Device: CPU - Batch Size: 64 - Model: GoogLeNet
images/sec > Higher Is Better
ASUS NVIDIA GeForce RTX 3090 . 94.21 |=========================================


TensorFlow 2.16.1
Device: CPU - Batch Size: 64 - Model: ResNet-50
images/sec > Higher Is Better
ASUS NVIDIA GeForce RTX 3090 . 26.52 |=========================================


TensorFlow 2.16.1
Device: GPU - Batch Size: 16 - Model: GoogLeNet
images/sec > Higher Is Better
ASUS NVIDIA GeForce RTX 3090 . 25.99 |=========================================


TensorFlow 2.16.1
Device: GPU - Batch Size: 16 - Model: ResNet-50
images/sec > Higher Is Better
ASUS NVIDIA GeForce RTX 3090 . 7.93 |==========================================


TensorFlow 2.16.1
Device: GPU - Batch Size: 32 - Model: GoogLeNet
images/sec > Higher Is Better
ASUS NVIDIA GeForce RTX 3090 . 26.63 |=========================================


TensorFlow 2.16.1
Device: GPU - Batch Size: 32 - Model: ResNet-50
images/sec > Higher Is Better
ASUS NVIDIA GeForce RTX 3090 . 7.98 |==========================================


TensorFlow 2.16.1
Device: GPU - Batch Size: 64 - Model: GoogLeNet
images/sec > Higher Is Better
ASUS NVIDIA GeForce RTX 3090 . 26.91 |=========================================


TensorFlow 2.16.1
Device: GPU - Batch Size: 64 - Model: ResNet-50
images/sec > Higher Is Better
ASUS NVIDIA GeForce RTX 3090 . 8.00 |==========================================


TensorFlow 2.16.1
Device: CPU - Batch Size: 256 - Model: GoogLeNet
images/sec > Higher Is Better
ASUS NVIDIA GeForce RTX 3090 . 93.45 |=========================================


TensorFlow 2.16.1
Device: CPU - Batch Size: 256 - Model: ResNet-50
images/sec > Higher Is Better
ASUS NVIDIA GeForce RTX 3090 . 27.49 |=========================================


TensorFlow 2.16.1
Device: CPU - Batch Size: 512 - Model: GoogLeNet
images/sec > Higher Is Better
ASUS NVIDIA GeForce RTX 3090 . 93.78 |=========================================


TensorFlow 2.16.1
Device: CPU - Batch Size: 512 - Model: ResNet-50
images/sec > Higher Is Better
ASUS NVIDIA GeForce RTX 3090 . 28.13 |=========================================


TensorFlow 2.16.1
Device: GPU - Batch Size: 256 - Model: GoogLeNet
images/sec > Higher Is Better
ASUS NVIDIA GeForce RTX 3090 . 27.04 |=========================================


TensorFlow 2.16.1
Device: GPU - Batch Size: 256 - Model: ResNet-50
images/sec > Higher Is Better
ASUS NVIDIA GeForce RTX 3090 . 8.04 |==========================================


TensorFlow 2.16.1
Device: GPU - Batch Size: 512 - Model: GoogLeNet
images/sec > Higher Is Better
ASUS NVIDIA GeForce RTX 3090 . 27.08 |=========================================


TensorFlow 2.16.1
Device: GPU - Batch Size: 512 - Model: ResNet-50
images/sec > Higher Is Better
ASUS NVIDIA GeForce RTX 3090 . 8.04 |==========================================


Neural Magic DeepSparse 1.7
Model: NLP Document Classification, oBERT base uncased on IMDB - Scenario: Asynchronous Multi-Stream
items/sec > Higher Is Better


Neural Magic DeepSparse 1.7
Model: NLP Document Classification, oBERT base uncased on IMDB - Scenario: Synchronous Single-Stream
items/sec > Higher Is Better


Neural Magic DeepSparse 1.7
Model: NLP Text Classification, BERT base uncased SST2, Sparse INT8 - Scenario: Asynchronous Multi-Stream
items/sec > Higher Is Better


Neural Magic DeepSparse 1.7
Model: NLP Text Classification, BERT base uncased SST2, Sparse INT8 - Scenario: Synchronous Single-Stream
items/sec > Higher Is Better


Neural Magic DeepSparse 1.7
Model: ResNet-50, Baseline - Scenario: Asynchronous Multi-Stream
items/sec > Higher Is Better


Neural Magic DeepSparse 1.7
Model: ResNet-50, Baseline - Scenario: Synchronous Single-Stream
items/sec > Higher Is Better


Neural Magic DeepSparse 1.7
Model: ResNet-50, Sparse INT8 - Scenario: Asynchronous Multi-Stream
items/sec > Higher Is Better


Neural Magic DeepSparse 1.7
Model: ResNet-50, Sparse INT8 - Scenario: Synchronous Single-Stream
items/sec > Higher Is Better


Neural Magic DeepSparse 1.7
Model: Llama2 Chat 7b Quantized - Scenario: Asynchronous Multi-Stream
items/sec > Higher Is Better


Neural Magic DeepSparse 1.7
Model: Llama2 Chat 7b Quantized - Scenario: Synchronous Single-Stream
items/sec > Higher Is Better


Neural Magic DeepSparse 1.7
Model: CV Classification, ResNet-50 ImageNet - Scenario: Asynchronous Multi-Stream
items/sec > Higher Is Better


Neural Magic DeepSparse 1.7
Model: CV Classification, ResNet-50 ImageNet - Scenario: Synchronous Single-Stream
items/sec > Higher Is Better


Neural Magic DeepSparse 1.7
Model: CV Detection, YOLOv5s COCO, Sparse INT8 - Scenario: Asynchronous Multi-Stream
items/sec > Higher Is Better


Neural Magic DeepSparse 1.7
Model: CV Detection, YOLOv5s COCO, Sparse INT8 - Scenario: Synchronous Single-Stream
items/sec > Higher Is Better


Neural Magic DeepSparse 1.7
Model: NLP Text Classification, DistilBERT mnli - Scenario: Asynchronous Multi-Stream
items/sec > Higher Is Better


Neural Magic DeepSparse 1.7
Model: NLP Text Classification, DistilBERT mnli - Scenario: Synchronous Single-Stream
items/sec > Higher Is Better


Neural Magic DeepSparse 1.7
Model: CV Segmentation, 90% Pruned YOLACT Pruned - Scenario: Asynchronous Multi-Stream
items/sec > Higher Is Better


Neural Magic DeepSparse 1.7
Model: CV Segmentation, 90% Pruned YOLACT Pruned - Scenario: Synchronous Single-Stream
items/sec > Higher Is Better


Neural Magic DeepSparse 1.7
Model: BERT-Large, NLP Question Answering, Sparse INT8 - Scenario: Asynchronous Multi-Stream
items/sec > Higher Is Better


Neural Magic DeepSparse 1.7
Model: BERT-Large, NLP Question Answering, Sparse INT8 - Scenario: Synchronous Single-Stream
items/sec > Higher Is Better


Neural Magic DeepSparse 1.7
Model: NLP Token Classification, BERT base uncased conll2003 - Scenario: Asynchronous Multi-Stream
items/sec > Higher Is Better


Neural Magic DeepSparse 1.7
Model: NLP Token Classification, BERT base uncased conll2003 - Scenario: Synchronous Single-Stream
items/sec > Higher Is Better


spaCy 3.4.1

tokens/sec > Higher Is Better


Caffe 2020-02-13
Model: AlexNet - Acceleration: CPU - Iterations: 100
Milli-Seconds < Lower Is Better


Caffe 2020-02-13
Model: AlexNet - Acceleration: CPU - Iterations: 200
Milli-Seconds < Lower Is Better


Caffe 2020-02-13
Model: AlexNet - Acceleration: CPU - Iterations: 1000
Milli-Seconds < Lower Is Better


Caffe 2020-02-13
Model: GoogleNet - Acceleration: CPU - Iterations: 100
Milli-Seconds < Lower Is Better


Caffe 2020-02-13
Model: GoogleNet - Acceleration: CPU - Iterations: 200
Milli-Seconds < Lower Is Better


Caffe 2020-02-13
Model: GoogleNet - Acceleration: CPU - Iterations: 1000
Milli-Seconds < Lower Is Better


Mobile Neural Network 2.9.b11b7037d
Model: nasnet
ms < Lower Is Better
ASUS NVIDIA GeForce RTX 3090 . 6.956 |=========================================


Mobile Neural Network 2.9.b11b7037d
Model: mobilenetV3
ms < Lower Is Better
ASUS NVIDIA GeForce RTX 3090 . 1.012 |=========================================


Mobile Neural Network 2.9.b11b7037d
Model: squeezenetv1.1
ms < Lower Is Better
ASUS NVIDIA GeForce RTX 3090 . 1.836 |=========================================


Mobile Neural Network 2.9.b11b7037d
Model: resnet-v2-50
ms < Lower Is Better
ASUS NVIDIA GeForce RTX 3090 . 14.36 |=========================================


Mobile Neural Network 2.9.b11b7037d
Model: SqueezeNetV1.0
ms < Lower Is Better
ASUS NVIDIA GeForce RTX 3090 . 2.935 |=========================================


Mobile Neural Network 2.9.b11b7037d
Model: MobileNetV2_224
ms < Lower Is Better
ASUS NVIDIA GeForce RTX 3090 . 1.882 |=========================================


Mobile Neural Network 2.9.b11b7037d
Model: mobilenet-v1-1.0
ms < Lower Is Better
ASUS NVIDIA GeForce RTX 3090 . 2.084 |=========================================


Mobile Neural Network 2.9.b11b7037d
Model: inception-v3
ms < Lower Is Better
ASUS NVIDIA GeForce RTX 3090 . 21.37 |=========================================


NCNN 20230517
Target: CPU - Model: mobilenet
ms < Lower Is Better
ASUS NVIDIA GeForce RTX 3090 . 16.46 |=========================================


NCNN 20230517
Target: CPU-v2-v2 - Model: mobilenet-v2
ms < Lower Is Better
ASUS NVIDIA GeForce RTX 3090 . 5.53 |==========================================


NCNN 20230517
Target: CPU-v3-v3 - Model: mobilenet-v3
ms < Lower Is Better
ASUS NVIDIA GeForce RTX 3090 . 8.28 |==========================================


NCNN 20230517
Target: CPU - Model: shufflenet-v2
ms < Lower Is Better
ASUS NVIDIA GeForce RTX 3090 . 11.05 |=========================================


NCNN 20230517
Target: CPU - Model: mnasnet
ms < Lower Is Better
ASUS NVIDIA GeForce RTX 3090 . 5.46 |==========================================


NCNN 20230517
Target: CPU - Model: efficientnet-b0
ms < Lower Is Better
ASUS NVIDIA GeForce RTX 3090 . 19.35 |=========================================


NCNN 20230517
Target: CPU - Model: blazeface
ms < Lower Is Better
ASUS NVIDIA GeForce RTX 3090 . 5.95 |==========================================


NCNN 20230517
Target: CPU - Model: googlenet
ms < Lower Is Better
ASUS NVIDIA GeForce RTX 3090 . 19.65 |=========================================


NCNN 20230517
Target: CPU - Model: vgg16
ms < Lower Is Better
ASUS NVIDIA GeForce RTX 3090 . 28.60 |=========================================


NCNN 20230517
Target: CPU - Model: resnet18
ms < Lower Is Better
ASUS NVIDIA GeForce RTX 3090 . 7.50 |==========================================


NCNN 20230517
Target: CPU - Model: alexnet
ms < Lower Is Better
ASUS NVIDIA GeForce RTX 3090 . 5.58 |==========================================


NCNN 20230517
Target: CPU - Model: resnet50
ms < Lower Is Better
ASUS NVIDIA GeForce RTX 3090 . 22.09 |=========================================


NCNN 20230517
Target: CPU - Model: yolov4-tiny
ms < Lower Is Better
ASUS NVIDIA GeForce RTX 3090 . 18.40 |=========================================


NCNN 20230517
Target: CPU - Model: squeezenet_ssd
ms < Lower Is Better
ASUS NVIDIA GeForce RTX 3090 . 13.62 |=========================================


NCNN 20230517
Target: CPU - Model: regnety_400m
ms < Lower Is Better
ASUS NVIDIA GeForce RTX 3090 . 82.63 |=========================================


NCNN 20230517
Target: CPU - Model: vision_transformer
ms < Lower Is Better
ASUS NVIDIA GeForce RTX 3090 . 70.73 |=========================================


NCNN 20230517
Target: CPU - Model: FastestDet
ms < Lower Is Better
ASUS NVIDIA GeForce RTX 3090 . 6.47 |==========================================


NCNN 20230517
Target: Vulkan GPU - Model: mobilenet
ms < Lower Is Better
ASUS NVIDIA GeForce RTX 3090 . 17.34 |=========================================


NCNN 20230517
Target: Vulkan GPU-v2-v2 - Model: mobilenet-v2
ms < Lower Is Better
ASUS NVIDIA GeForce RTX 3090 . 5.72 |==========================================


NCNN 20230517
Target: Vulkan GPU-v3-v3 - Model: mobilenet-v3
ms < Lower Is Better
ASUS NVIDIA GeForce RTX 3090 . 7.93 |==========================================


NCNN 20230517
Target: Vulkan GPU - Model: shufflenet-v2
ms < Lower Is Better
ASUS NVIDIA GeForce RTX 3090 . 10.51 |=========================================


NCNN 20230517
Target: Vulkan GPU - Model: mnasnet
ms < Lower Is Better
ASUS NVIDIA GeForce RTX 3090 . 5.43 |==========================================


NCNN 20230517
Target: Vulkan GPU - Model: efficientnet-b0
ms < Lower Is Better
ASUS NVIDIA GeForce RTX 3090 . 18.73 |=========================================


NCNN 20230517
Target: Vulkan GPU - Model: blazeface
ms < Lower Is Better
ASUS NVIDIA GeForce RTX 3090 . 6.00 |==========================================


NCNN 20230517
Target: Vulkan GPU - Model: googlenet
ms < Lower Is Better
ASUS NVIDIA GeForce RTX 3090 . 19.43 |=========================================


NCNN 20230517
Target: Vulkan GPU - Model: vgg16
ms < Lower Is Better
ASUS NVIDIA GeForce RTX 3090 . 28.61 |=========================================


NCNN 20230517
Target: Vulkan GPU - Model: resnet18
ms < Lower Is Better
ASUS NVIDIA GeForce RTX 3090 . 7.29 |==========================================


NCNN 20230517
Target: Vulkan GPU - Model: alexnet
ms < Lower Is Better
ASUS NVIDIA GeForce RTX 3090 . 5.49 |==========================================


NCNN 20230517
Target: Vulkan GPU - Model: resnet50
ms < Lower Is Better
ASUS NVIDIA GeForce RTX 3090 . 22.13 |=========================================


NCNN 20230517
Target: Vulkan GPU - Model: yolov4-tiny
ms < Lower Is Better
ASUS NVIDIA GeForce RTX 3090 . 18.17 |=========================================


NCNN 20230517
Target: Vulkan GPU - Model: squeezenet_ssd
ms < Lower Is Better
ASUS NVIDIA GeForce RTX 3090 . 14.25 |=========================================


NCNN 20230517
Target: Vulkan GPU - Model: regnety_400m
ms < Lower Is Better
ASUS NVIDIA GeForce RTX 3090 . 84.85 |=========================================


NCNN 20230517
Target: Vulkan GPU - Model: vision_transformer
ms < Lower Is Better
ASUS NVIDIA GeForce RTX 3090 . 70.75 |=========================================


NCNN 20230517
Target: Vulkan GPU - Model: FastestDet
ms < Lower Is Better
ASUS NVIDIA GeForce RTX 3090 . 6.56 |==========================================


TNN 0.3
Target: CPU - Model: DenseNet
ms < Lower Is Better


TNN 0.3
Target: CPU - Model: MobileNet v2
ms < Lower Is Better


TNN 0.3
Target: CPU - Model: SqueezeNet v2
ms < Lower Is Better


TNN 0.3
Target: CPU - Model: SqueezeNet v1.1
ms < Lower Is Better


XNNPACK b7b048
Model: FP32MobileNetV1
us < Lower Is Better
ASUS NVIDIA GeForce RTX 3090 . 1394 |==========================================


XNNPACK b7b048
Model: FP32MobileNetV2
us < Lower Is Better
ASUS NVIDIA GeForce RTX 3090 . 1125 |==========================================


XNNPACK b7b048
Model: FP32MobileNetV3Large
us < Lower Is Better
ASUS NVIDIA GeForce RTX 3090 . 1585 |==========================================


XNNPACK b7b048
Model: FP32MobileNetV3Small
us < Lower Is Better
ASUS NVIDIA GeForce RTX 3090 . 820 |===========================================


XNNPACK b7b048
Model: FP16MobileNetV1
us < Lower Is Better
ASUS NVIDIA GeForce RTX 3090 . 2197 |==========================================


XNNPACK b7b048
Model: FP16MobileNetV2
us < Lower Is Better
ASUS NVIDIA GeForce RTX 3090 . 1646 |==========================================


XNNPACK b7b048
Model: FP16MobileNetV3Large
us < Lower Is Better
ASUS NVIDIA GeForce RTX 3090 . 1834 |==========================================


XNNPACK b7b048
Model: FP16MobileNetV3Small
us < Lower Is Better
ASUS NVIDIA GeForce RTX 3090 . 928 |===========================================


XNNPACK b7b048
Model: QS8MobileNetV2
us < Lower Is Better
ASUS NVIDIA GeForce RTX 3090 . 904 |===========================================


PlaidML
FP16: No - Mode: Inference - Network: VGG16 - Device: CPU
Examples Per Second > Higher Is Better


PlaidML
FP16: No - Mode: Inference - Network: ResNet 50 - Device: CPU
Examples Per Second > Higher Is Better


OpenVINO 2024.0
Model: Face Detection FP16 - Device: CPU
FPS > Higher Is Better
ASUS NVIDIA GeForce RTX 3090 . 4.33 |==========================================


OpenVINO 2024.0
Model: Face Detection FP16 - Device: CPU
ms < Lower Is Better
ASUS NVIDIA GeForce RTX 3090 . 1375.14 |=======================================


OpenVINO 2024.0
Model: Person Detection FP16 - Device: CPU
FPS > Higher Is Better
ASUS NVIDIA GeForce RTX 3090 . 40.98 |=========================================


OpenVINO 2024.0
Model: Person Detection FP16 - Device: CPU
ms < Lower Is Better
ASUS NVIDIA GeForce RTX 3090 . 146.25 |========================================


OpenVINO 2024.0
Model: Person Detection FP32 - Device: CPU
FPS > Higher Is Better
ASUS NVIDIA GeForce RTX 3090 . 41.47 |=========================================


OpenVINO 2024.0
Model: Person Detection FP32 - Device: CPU
ms < Lower Is Better
ASUS NVIDIA GeForce RTX 3090 . 144.47 |========================================


OpenVINO 2024.0
Model: Vehicle Detection FP16 - Device: CPU
FPS > Higher Is Better
ASUS NVIDIA GeForce RTX 3090 . 296.62 |========================================


OpenVINO 2024.0
Model: Vehicle Detection FP16 - Device: CPU
ms < Lower Is Better
ASUS NVIDIA GeForce RTX 3090 . 20.16 |=========================================


OpenVINO 2024.0
Model: Face Detection FP16-INT8 - Device: CPU
FPS > Higher Is Better
ASUS NVIDIA GeForce RTX 3090 . 15.61 |=========================================


OpenVINO 2024.0
Model: Face Detection FP16-INT8 - Device: CPU
ms < Lower Is Better
ASUS NVIDIA GeForce RTX 3090 . 382.74 |========================================


OpenVINO 2024.0
Model: Face Detection Retail FP16 - Device: CPU
FPS > Higher Is Better
ASUS NVIDIA GeForce RTX 3090 . 1180.46 |=======================================


OpenVINO 2024.0
Model: Face Detection Retail FP16 - Device: CPU
ms < Lower Is Better
ASUS NVIDIA GeForce RTX 3090 . 5.02 |==========================================


OpenVINO 2024.0
Model: Road Segmentation ADAS FP16 - Device: CPU
FPS > Higher Is Better
ASUS NVIDIA GeForce RTX 3090 . 86.71 |=========================================


OpenVINO 2024.0
Model: Road Segmentation ADAS FP16 - Device: CPU
ms < Lower Is Better
ASUS NVIDIA GeForce RTX 3090 . 69.09 |=========================================


OpenVINO 2024.0
Model: Vehicle Detection FP16-INT8 - Device: CPU
FPS > Higher Is Better
ASUS NVIDIA GeForce RTX 3090 . 712.75 |========================================


OpenVINO 2024.0
Model: Vehicle Detection FP16-INT8 - Device: CPU
ms < Lower Is Better
ASUS NVIDIA GeForce RTX 3090 . 8.35 |==========================================


OpenVINO 2024.0
Model: Weld Porosity Detection FP16 - Device: CPU
FPS > Higher Is Better
ASUS NVIDIA GeForce RTX 3090 . 450.44 |========================================


OpenVINO 2024.0
Model: Weld Porosity Detection FP16 - Device: CPU
ms < Lower Is Better
ASUS NVIDIA GeForce RTX 3090 . 44.26 |=========================================


OpenVINO 2024.0
Model: Face Detection Retail FP16-INT8 - Device: CPU
FPS > Higher Is Better
ASUS NVIDIA GeForce RTX 3090 . 1948.78 |=======================================


OpenVINO 2024.0
Model: Face Detection Retail FP16-INT8 - Device: CPU
ms < Lower Is Better
ASUS NVIDIA GeForce RTX 3090 . 2.97 |==========================================


OpenVINO 2024.0
Model: Road Segmentation ADAS FP16-INT8 - Device: CPU
FPS > Higher Is Better
ASUS NVIDIA GeForce RTX 3090 . 283.34 |========================================


OpenVINO 2024.0
Model: Road Segmentation ADAS FP16-INT8 - Device: CPU
ms < Lower Is Better
ASUS NVIDIA GeForce RTX 3090 . 21.09 |=========================================


OpenVINO 2024.0
Model: Machine Translation EN To DE FP16 - Device: CPU
FPS > Higher Is Better
ASUS NVIDIA GeForce RTX 3090 . 51.69 |=========================================


OpenVINO 2024.0
Model: Machine Translation EN To DE FP16 - Device: CPU
ms < Lower Is Better
ASUS NVIDIA GeForce RTX 3090 . 115.87 |========================================


OpenVINO 2024.0
Model: Weld Porosity Detection FP16-INT8 - Device: CPU
FPS > Higher Is Better
ASUS NVIDIA GeForce RTX 3090 . 1544.54 |=======================================


OpenVINO 2024.0
Model: Weld Porosity Detection FP16-INT8 - Device: CPU
ms < Lower Is Better
ASUS NVIDIA GeForce RTX 3090 . 12.81 |=========================================


OpenVINO 2024.0
Model: Person Vehicle Bike Detection FP16 - Device: CPU
FPS > Higher Is Better
ASUS NVIDIA GeForce RTX 3090 . 528.13 |========================================


OpenVINO 2024.0
Model: Person Vehicle Bike Detection FP16 - Device: CPU
ms < Lower Is Better
ASUS NVIDIA GeForce RTX 3090 . 11.29 |=========================================


OpenVINO 2024.0
Model: Noise Suppression Poconet-Like FP16 - Device: CPU
FPS > Higher Is Better
ASUS NVIDIA GeForce RTX 3090 . 656.86 |========================================


OpenVINO 2024.0
Model: Noise Suppression Poconet-Like FP16 - Device: CPU
ms < Lower Is Better
ASUS NVIDIA GeForce RTX 3090 . 9.02 |==========================================


OpenVINO 2024.0
Model: Handwritten English Recognition FP16 - Device: CPU
FPS > Higher Is Better
ASUS NVIDIA GeForce RTX 3090 . 225.32 |========================================


OpenVINO 2024.0
Model: Handwritten English Recognition FP16 - Device: CPU
ms < Lower Is Better
ASUS NVIDIA GeForce RTX 3090 . 88.65 |=========================================


OpenVINO 2024.0
Model: Person Re-Identification Retail FP16 - Device: CPU
FPS > Higher Is Better
ASUS NVIDIA GeForce RTX 3090 . 659.72 |========================================


OpenVINO 2024.0
Model: Person Re-Identification Retail FP16 - Device: CPU
ms < Lower Is Better
ASUS NVIDIA GeForce RTX 3090 . 9.01 |==========================================


OpenVINO 2024.0
Model: Age Gender Recognition Retail 0013 FP16 - Device: CPU
FPS > Higher Is Better
ASUS NVIDIA GeForce RTX 3090 . 11244.68 |======================================


OpenVINO 2024.0
Model: Age Gender Recognition Retail 0013 FP16 - Device: CPU
ms < Lower Is Better
ASUS NVIDIA GeForce RTX 3090 . 1.66 |==========================================


OpenVINO 2024.0
Model: Handwritten English Recognition FP16-INT8 - Device: CPU
FPS > Higher Is Better
ASUS NVIDIA GeForce RTX 3090 . 266.87 |========================================


OpenVINO 2024.0
Model: Handwritten English Recognition FP16-INT8 - Device: CPU
ms < Lower Is Better
ASUS NVIDIA GeForce RTX 3090 . 74.85 |=========================================


OpenVINO 2024.0
Model: Age Gender Recognition Retail 0013 FP16-INT8 - Device: CPU
FPS > Higher Is Better
ASUS NVIDIA GeForce RTX 3090 . 29697.31 |======================================


OpenVINO 2024.0
Model: Age Gender Recognition Retail 0013 FP16-INT8 - Device: CPU
ms < Lower Is Better
ASUS NVIDIA GeForce RTX 3090 . 0.62 |==========================================


ECP-CANDLE 0.4
Benchmark: P1B2
Seconds < Lower Is Better


ECP-CANDLE 0.4
Benchmark: P3B1
Seconds < Lower Is Better


ECP-CANDLE 0.4
Benchmark: P3B2
Seconds < Lower Is Better


Numenta Anomaly Benchmark 1.1
Detector: KNN CAD
Seconds < Lower Is Better


Numenta Anomaly Benchmark 1.1
Detector: Relative Entropy
Seconds < Lower Is Better


Numenta Anomaly Benchmark 1.1
Detector: Windowed Gaussian
Seconds < Lower Is Better


Numenta Anomaly Benchmark 1.1
Detector: Earthgecko Skyline
Seconds < Lower Is Better


Numenta Anomaly Benchmark 1.1
Detector: Bayesian Changepoint
Seconds < Lower Is Better


Numenta Anomaly Benchmark 1.1
Detector: Contextual Anomaly Detector OSE
Seconds < Lower Is Better


ONNX Runtime 1.19
Model: GPT-2 - Device: CPU - Executor: Parallel
Inferences Per Second > Higher Is Better


ONNX Runtime 1.19
Model: GPT-2 - Device: CPU - Executor: Standard
Inferences Per Second > Higher Is Better


ONNX Runtime 1.19
Model: yolov4 - Device: CPU - Executor: Parallel
Inferences Per Second > Higher Is Better
ASUS NVIDIA GeForce RTX 3090 . 10.49 |=========================================


ONNX Runtime 1.19
Model: yolov4 - Device: CPU - Executor: Parallel
Inference Time Cost (ms) < Lower Is Better
ASUS NVIDIA GeForce RTX 3090 . 95.56 |=========================================


ONNX Runtime 1.19
Model: yolov4 - Device: CPU - Executor: Standard
Inferences Per Second > Higher Is Better
ASUS NVIDIA GeForce RTX 3090 . 11.15 |=========================================


ONNX Runtime 1.19
Model: yolov4 - Device: CPU - Executor: Standard
Inference Time Cost (ms) < Lower Is Better
ASUS NVIDIA GeForce RTX 3090 . 89.76 |=========================================


ONNX Runtime 1.19
Model: ZFNet-512 - Device: CPU - Executor: Parallel
Inferences Per Second > Higher Is Better
ASUS NVIDIA GeForce RTX 3090 . 79.43 |=========================================


ONNX Runtime 1.19
Model: ZFNet-512 - Device: CPU - Executor: Parallel
Inference Time Cost (ms) < Lower Is Better
ASUS NVIDIA GeForce RTX 3090 . 12.59 |=========================================


ONNX Runtime 1.19
Model: ZFNet-512 - Device: CPU - Executor: Standard
Inferences Per Second > Higher Is Better
ASUS NVIDIA GeForce RTX 3090 . 92.47 |=========================================


ONNX Runtime 1.19
Model: ZFNet-512 - Device: CPU - Executor: Standard
Inference Time Cost (ms) < Lower Is Better
ASUS NVIDIA GeForce RTX 3090 . 10.81 |=========================================


ONNX Runtime 1.19
Model: T5 Encoder - Device: CPU - Executor: Parallel
Inferences Per Second > Higher Is Better
ASUS NVIDIA GeForce RTX 3090 . 115.17 |========================================


ONNX Runtime 1.19
Model: T5 Encoder - Device: CPU - Executor: Parallel
Inference Time Cost (ms) < Lower Is Better
ASUS NVIDIA GeForce RTX 3090 . 8.68614 |=======================================


ONNX Runtime 1.19
Model: T5 Encoder - Device: CPU - Executor: Standard
Inferences Per Second > Higher Is Better
ASUS NVIDIA GeForce RTX 3090 . 152.80 |========================================


ONNX Runtime 1.19
Model: T5 Encoder - Device: CPU - Executor: Standard
Inference Time Cost (ms) < Lower Is Better
ASUS NVIDIA GeForce RTX 3090 . 6.54304 |=======================================


ONNX Runtime 1.19
Model: bertsquad-12 - Device: CPU - Executor: Parallel
Inferences Per Second > Higher Is Better


ONNX Runtime 1.19
Model: bertsquad-12 - Device: CPU - Executor: Standard
Inferences Per Second > Higher Is Better


ONNX Runtime 1.19
Model: CaffeNet 12-int8 - Device: CPU - Executor: Parallel
Inferences Per Second > Higher Is Better
ASUS NVIDIA GeForce RTX 3090 . 307.37 |========================================


ONNX Runtime 1.19
Model: CaffeNet 12-int8 - Device: CPU - Executor: Parallel
Inference Time Cost (ms) < Lower Is Better
ASUS NVIDIA GeForce RTX 3090 . 3.25179 |=======================================


ONNX Runtime 1.19
Model: CaffeNet 12-int8 - Device: CPU - Executor: Standard
Inferences Per Second > Higher Is Better
ASUS NVIDIA GeForce RTX 3090 . 646.11 |========================================


ONNX Runtime 1.19
Model: CaffeNet 12-int8 - Device: CPU - Executor: Standard
Inference Time Cost (ms) < Lower Is Better
ASUS NVIDIA GeForce RTX 3090 . 1.54657 |=======================================


ONNX Runtime 1.19
Model: fcn-resnet101-11 - Device: CPU - Executor: Parallel
Inferences Per Second > Higher Is Better
ASUS NVIDIA GeForce RTX 3090 . 1.65652 |=======================================


ONNX Runtime 1.19
Model: fcn-resnet101-11 - Device: CPU - Executor: Parallel
Inference Time Cost (ms) < Lower Is Better
ASUS NVIDIA GeForce RTX 3090 . 604.55 |========================================


ONNX Runtime 1.19
Model: fcn-resnet101-11 - Device: CPU - Executor: Standard
Inferences Per Second > Higher Is Better
ASUS NVIDIA GeForce RTX 3090 . 2.49779 |=======================================


ONNX Runtime 1.19
Model: fcn-resnet101-11 - Device: CPU - Executor: Standard
Inference Time Cost (ms) < Lower Is Better
ASUS NVIDIA GeForce RTX 3090 . 400.35 |========================================


ONNX Runtime 1.19
Model: ArcFace ResNet-100 - Device: CPU - Executor: Parallel
Inferences Per Second > Higher Is Better


ONNX Runtime 1.19
Model: ArcFace ResNet-100 - Device: CPU - Executor: Standard
Inferences Per Second > Higher Is Better


ONNX Runtime 1.19
Model: ResNet50 v1-12-int8 - Device: CPU - Executor: Parallel
Inferences Per Second > Higher Is Better
ASUS NVIDIA GeForce RTX 3090 . 162.89 |========================================


ONNX Runtime 1.19
Model: ResNet50 v1-12-int8 - Device: CPU - Executor: Parallel
Inference Time Cost (ms) < Lower Is Better
ASUS NVIDIA GeForce RTX 3090 . 6.14425 |=======================================


ONNX Runtime 1.19
Model: ResNet50 v1-12-int8 - Device: CPU - Executor: Standard
Inferences Per Second > Higher Is Better
ASUS NVIDIA GeForce RTX 3090 . 313.24 |========================================


ONNX Runtime 1.19
Model: ResNet50 v1-12-int8 - Device: CPU - Executor: Standard
Inference Time Cost (ms) < Lower Is Better
ASUS NVIDIA GeForce RTX 3090 . 3.19151 |=======================================


ONNX Runtime 1.19
Model: super-resolution-10 - Device: CPU - Executor: Parallel
Inferences Per Second > Higher Is Better
ASUS NVIDIA GeForce RTX 3090 . 71.81 |=========================================


ONNX Runtime 1.19
Model: super-resolution-10 - Device: CPU - Executor: Parallel
Inference Time Cost (ms) < Lower Is Better
ASUS NVIDIA GeForce RTX 3090 . 13.94 |=========================================


ONNX Runtime 1.19
Model: super-resolution-10 - Device: CPU - Executor: Standard
Inferences Per Second > Higher Is Better
ASUS NVIDIA GeForce RTX 3090 . 81.78 |=========================================


ONNX Runtime 1.19
Model: super-resolution-10 - Device: CPU - Executor: Standard
Inference Time Cost (ms) < Lower Is Better
ASUS NVIDIA GeForce RTX 3090 . 12.23 |=========================================


ONNX Runtime 1.19
Model: ResNet101_DUC_HDC-12 - Device: CPU - Executor: Parallel
Inferences Per Second > Higher Is Better
ASUS NVIDIA GeForce RTX 3090 . 0.743436 |======================================


ONNX Runtime 1.19
Model: ResNet101_DUC_HDC-12 - Device: CPU - Executor: Parallel
Inference Time Cost (ms) < Lower Is Better
ASUS NVIDIA GeForce RTX 3090 . 1345.66 |=======================================


ONNX Runtime 1.19
Model: ResNet101_DUC_HDC-12 - Device: CPU - Executor: Standard
Inferences Per Second > Higher Is Better
ASUS NVIDIA GeForce RTX 3090 . 0.943867 |======================================


ONNX Runtime 1.19
Model: ResNet101_DUC_HDC-12 - Device: CPU - Executor: Standard
Inference Time Cost (ms) < Lower Is Better
ASUS NVIDIA GeForce RTX 3090 . 1059.49 |=======================================


ONNX Runtime 1.19
Model: Faster R-CNN R-50-FPN-int8 - Device: CPU - Executor: Parallel
Inferences Per Second > Higher Is Better


ONNX Runtime 1.19
Model: Faster R-CNN R-50-FPN-int8 - Device: CPU - Executor: Standard
Inferences Per Second > Higher Is Better


AI Benchmark Alpha 0.1.2

Score > Higher Is Better


Mlpack Benchmark
Benchmark: scikit_ica
Seconds < Lower Is Better


Mlpack Benchmark
Benchmark: scikit_qda
Seconds < Lower Is Better


Mlpack Benchmark
Benchmark: scikit_svm
Seconds < Lower Is Better


Mlpack Benchmark
Benchmark: scikit_linearridgeregression
Seconds < Lower Is Better


Scikit-Learn 1.2.2
Benchmark: GLM
Seconds < Lower Is Better
ASUS NVIDIA GeForce RTX 3090 . 183.48 |========================================


Scikit-Learn 1.2.2
Benchmark: SAGA
Seconds < Lower Is Better
ASUS NVIDIA GeForce RTX 3090 . 521.31 |========================================


Scikit-Learn 1.2.2
Benchmark: Tree
Seconds < Lower Is Better
ASUS NVIDIA GeForce RTX 3090 . 42.77 |=========================================


Scikit-Learn 1.2.2
Benchmark: Lasso
Seconds < Lower Is Better
ASUS NVIDIA GeForce RTX 3090 . 258.72 |========================================


Scikit-Learn 1.2.2
Benchmark: Glmnet
Seconds < Lower Is Better


Scikit-Learn 1.2.2
Benchmark: Sparsify
Seconds < Lower Is Better
ASUS NVIDIA GeForce RTX 3090 . 85.98 |=========================================


Scikit-Learn 1.2.2
Benchmark: Plot Ward
Seconds < Lower Is Better
ASUS NVIDIA GeForce RTX 3090 . 43.62 |=========================================


Scikit-Learn 1.2.2
Benchmark: MNIST Dataset
Seconds < Lower Is Better
ASUS NVIDIA GeForce RTX 3090 . 53.54 |=========================================


Scikit-Learn 1.2.2
Benchmark: Plot Neighbors
Seconds < Lower Is Better
ASUS NVIDIA GeForce RTX 3090 . 96.76 |=========================================


Scikit-Learn 1.2.2
Benchmark: SGD Regression
Seconds < Lower Is Better
ASUS NVIDIA GeForce RTX 3090 . 63.79 |=========================================


Scikit-Learn 1.2.2
Benchmark: SGDOneClassSVM
Seconds < Lower Is Better
ASUS NVIDIA GeForce RTX 3090 . 196.47 |========================================


Scikit-Learn 1.2.2
Benchmark: Plot Lasso Path
Seconds < Lower Is Better


Scikit-Learn 1.2.2
Benchmark: Isolation Forest
Seconds < Lower Is Better
ASUS NVIDIA GeForce RTX 3090 . 147.52 |========================================


Scikit-Learn 1.2.2
Benchmark: Plot Fast KMeans
Seconds < Lower Is Better


Scikit-Learn 1.2.2
Benchmark: Text Vectorizers
Seconds < Lower Is Better
ASUS NVIDIA GeForce RTX 3090 . 38.91 |=========================================


Scikit-Learn 1.2.2
Benchmark: Plot Hierarchical
Seconds < Lower Is Better
ASUS NVIDIA GeForce RTX 3090 . 153.06 |========================================


Scikit-Learn 1.2.2
Benchmark: Plot OMP vs. LARS
Seconds < Lower Is Better
ASUS NVIDIA GeForce RTX 3090 . 45.62 |=========================================


Scikit-Learn 1.2.2
Benchmark: Feature Expansions
Seconds < Lower Is Better
ASUS NVIDIA GeForce RTX 3090 . 120.33 |========================================


Scikit-Learn 1.2.2
Benchmark: LocalOutlierFactor
Seconds < Lower Is Better
ASUS NVIDIA GeForce RTX 3090 . 35.65 |=========================================


Scikit-Learn 1.2.2
Benchmark: TSNE MNIST Dataset
Seconds < Lower Is Better
ASUS NVIDIA GeForce RTX 3090 . 188.33 |========================================


Scikit-Learn 1.2.2
Benchmark: Isotonic / Logistic
Seconds < Lower Is Better
ASUS NVIDIA GeForce RTX 3090 . 1447.69 |=======================================


Scikit-Learn 1.2.2
Benchmark: Plot Incremental PCA
Seconds < Lower Is Better
ASUS NVIDIA GeForce RTX 3090 . 21.31 |=========================================


Scikit-Learn 1.2.2
Benchmark: Hist Gradient Boosting
Seconds < Lower Is Better
ASUS NVIDIA GeForce RTX 3090 . 1120.04 |=======================================


Scikit-Learn 1.2.2
Benchmark: Plot Parallel Pairwise
Seconds < Lower Is Better
ASUS NVIDIA GeForce RTX 3090 . 332.19 |========================================


Scikit-Learn 1.2.2
Benchmark: Isotonic / Pathological
Seconds < Lower Is Better


Scikit-Learn 1.2.2
Benchmark: RCV1 Logreg Convergencet
Seconds < Lower Is Better


Scikit-Learn 1.2.2
Benchmark: Sample Without Replacement
Seconds < Lower Is Better
ASUS NVIDIA GeForce RTX 3090 . 84.27 |=========================================


Scikit-Learn 1.2.2
Benchmark: Covertype Dataset Benchmark
Seconds < Lower Is Better
ASUS NVIDIA GeForce RTX 3090 . 317.46 |========================================


Scikit-Learn 1.2.2
Benchmark: Hist Gradient Boosting Adult
Seconds < Lower Is Better
ASUS NVIDIA GeForce RTX 3090 . 1173.89 |=======================================


Scikit-Learn 1.2.2
Benchmark: Isotonic / Perturbed Logarithm
Seconds < Lower Is Better
ASUS NVIDIA GeForce RTX 3090 . 1469.79 |=======================================


Scikit-Learn 1.2.2
Benchmark: Hist Gradient Boosting Threading
Seconds < Lower Is Better
ASUS NVIDIA GeForce RTX 3090 . 140.75 |========================================


Scikit-Learn 1.2.2
Benchmark: Plot Singular Value Decomposition
Seconds < Lower Is Better


Scikit-Learn 1.2.2
Benchmark: Hist Gradient Boosting Higgs Boson
Seconds < Lower Is Better
ASUS NVIDIA GeForce RTX 3090 . 190.58 |========================================


Scikit-Learn 1.2.2
Benchmark: 20 Newsgroups / Logistic Regression
Seconds < Lower Is Better
ASUS NVIDIA GeForce RTX 3090 . 9.614 |=========================================


Scikit-Learn 1.2.2
Benchmark: Plot Polynomial Kernel Approximation
Seconds < Lower Is Better
ASUS NVIDIA GeForce RTX 3090 . 133.12 |========================================


Scikit-Learn 1.2.2
Benchmark: Plot Non-Negative Matrix Factorization
Seconds < Lower Is Better


Scikit-Learn 1.2.2
Benchmark: Hist Gradient Boosting Categorical Only
Seconds < Lower Is Better
ASUS NVIDIA GeForce RTX 3090 . 195.30 |========================================


Scikit-Learn 1.2.2
Benchmark: Kernel PCA Solvers / Time vs. N Samples
Seconds < Lower Is Better
ASUS NVIDIA GeForce RTX 3090 . 69.63 |=========================================


Scikit-Learn 1.2.2
Benchmark: Kernel PCA Solvers / Time vs. N Components
Seconds < Lower Is Better
ASUS NVIDIA GeForce RTX 3090 . 25.59 |=========================================


Scikit-Learn 1.2.2
Benchmark: Sparse Random Projections / 100 Iterations
Seconds < Lower Is Better
ASUS NVIDIA GeForce RTX 3090 . 587.84 |========================================


Whisper.cpp 1.6.2
Model: ggml-base.en - Input: 2016 State of the Union
Seconds < Lower Is Better
ASUS NVIDIA GeForce RTX 3090 . 209.02 |========================================


Whisper.cpp 1.6.2
Model: ggml-small.en - Input: 2016 State of the Union
Seconds < Lower Is Better
ASUS NVIDIA GeForce RTX 3090 . 638.01 |========================================


Whisper.cpp 1.6.2
Model: ggml-medium.en - Input: 2016 State of the Union
Seconds < Lower Is Better
ASUS NVIDIA GeForce RTX 3090 . 2000.10 |=======================================


Llama.cpp b3067
Model: llama-2-7b.Q4_0.gguf
Tokens Per Second > Higher Is Better


Llama.cpp b3067
Model: llama-2-13b.Q4_0.gguf
Tokens Per Second > Higher Is Better


Llama.cpp b3067
Model: llama-2-70b-chat.Q5_0.gguf
Tokens Per Second > Higher Is Better


Llamafile 0.8.6
Test: llava-v1.5-7b-q4 - Acceleration: CPU
Tokens Per Second > Higher Is Better


Llamafile 0.8.6
Test: mistral-7b-instruct-v0.2.Q8_0 - Acceleration: CPU
Tokens Per Second > Higher Is Better


Llamafile 0.8.6
Test: wizardcoder-python-34b-v1.0.Q6_K - Acceleration: CPU
Tokens Per Second > Higher Is Better


OpenCV 4.7
Test: DNN - Deep Neural Network
ms < Lower Is Better
ASUS NVIDIA GeForce RTX 3090 . 25552 |=========================================