102424machinelearningtest Intel Core i9-12900K testing with a ASUS PRIME Z790-V AX (1802 BIOS) and ASUS NVIDIA w Dual GeForce RTX 3090 24GB w NVLink on Ubuntu 24.04 via the Phoronix Test Suite. ASUS NVIDIA GeForce RTX 3090: Processor: Intel Core i9-12900K @ 5.10GHz (16 Cores / 24 Threads), Motherboard: ASUS PRIME Z790-V AX (1802 BIOS), Chipset: Intel Raptor Lake-S PCH, Memory: 96GB, Disk: 2000GB Samsung SSD 970 EVO Plus 2TB, Graphics: ASUS NVIDIA GeForce RTX 3090 24GB, Audio: Intel Raptor Lake HD Audio, Monitor: S24F350, Network: Realtek RTL8111/8168/8211/8411 + Realtek Device b851 OS: Ubuntu 24.04, Kernel: 6.8.0-47-generic (x86_64), Desktop: GNOME Shell 46.0, Display Server: X Server + Wayland, Display Driver: NVIDIA 560.35.03, OpenGL: 4.6.0, OpenCL: OpenCL 3.0 CUDA 12.6.65, Compiler: GCC 13.2.0 + CUDA 12.5, File-System: ext4, Screen Resolution: 1920x1080 AI Benchmark Alpha 0.1.2 Score > Higher Is Better Caffe 2020-02-13 Model: AlexNet - Acceleration: CPU - Iterations: 100 Milli-Seconds < Lower Is Better Caffe 2020-02-13 Model: AlexNet - Acceleration: CPU - Iterations: 200 Milli-Seconds < Lower Is Better Caffe 2020-02-13 Model: AlexNet - Acceleration: CPU - Iterations: 1000 Milli-Seconds < Lower Is Better Caffe 2020-02-13 Model: GoogleNet - Acceleration: CPU - Iterations: 100 Milli-Seconds < Lower Is Better Caffe 2020-02-13 Model: GoogleNet - Acceleration: CPU - Iterations: 200 Milli-Seconds < Lower Is Better Caffe 2020-02-13 Model: GoogleNet - Acceleration: CPU - Iterations: 1000 Milli-Seconds < Lower Is Better DeepSpeech 0.6 Acceleration: CPU Seconds < Lower Is Better ASUS NVIDIA GeForce RTX 3090 . 54.63 |========================================= ECP-CANDLE 0.4 Benchmark: P1B2 Seconds < Lower Is Better ECP-CANDLE 0.4 Benchmark: P3B1 Seconds < Lower Is Better ECP-CANDLE 0.4 Benchmark: P3B2 Seconds < Lower Is Better LeelaChessZero 0.31.1 Backend: BLAS Nodes Per Second > Higher Is Better ASUS NVIDIA GeForce RTX 3090 . 166 |=========================================== Llama.cpp b3067 Model: llama-2-7b.Q4_0.gguf Tokens Per Second > Higher Is Better Llama.cpp b3067 Model: llama-2-13b.Q4_0.gguf Tokens Per Second > Higher Is Better Llama.cpp b3067 Model: llama-2-70b-chat.Q5_0.gguf Tokens Per Second > Higher Is Better Llamafile 0.8.6 Test: llava-v1.5-7b-q4 - Acceleration: CPU Tokens Per Second > Higher Is Better Llamafile 0.8.6 Test: mistral-7b-instruct-v0.2.Q8_0 - Acceleration: CPU Tokens Per Second > Higher Is Better Llamafile 0.8.6 Test: wizardcoder-python-34b-v1.0.Q6_K - Acceleration: CPU Tokens Per Second > Higher Is Better Mlpack Benchmark Benchmark: scikit_ica Seconds < Lower Is Better Mlpack Benchmark Benchmark: scikit_qda Seconds < Lower Is Better Mlpack Benchmark Benchmark: scikit_svm Seconds < Lower Is Better Mlpack Benchmark Benchmark: scikit_linearridgeregression Seconds < Lower Is Better Mobile Neural Network 2.9.b11b7037d Model: nasnet ms < Lower Is Better ASUS NVIDIA GeForce RTX 3090 . 6.956 |========================================= Mobile Neural Network 2.9.b11b7037d Model: mobilenetV3 ms < Lower Is Better ASUS NVIDIA GeForce RTX 3090 . 1.012 |========================================= Mobile Neural Network 2.9.b11b7037d Model: squeezenetv1.1 ms < Lower Is Better ASUS NVIDIA GeForce RTX 3090 . 1.836 |========================================= Mobile Neural Network 2.9.b11b7037d Model: resnet-v2-50 ms < Lower Is Better ASUS NVIDIA GeForce RTX 3090 . 14.36 |========================================= Mobile Neural Network 2.9.b11b7037d Model: SqueezeNetV1.0 ms < Lower Is Better ASUS NVIDIA GeForce RTX 3090 . 2.935 |========================================= Mobile Neural Network 2.9.b11b7037d Model: MobileNetV2_224 ms < Lower Is Better ASUS NVIDIA GeForce RTX 3090 . 1.882 |========================================= Mobile Neural Network 2.9.b11b7037d Model: mobilenet-v1-1.0 ms < Lower Is Better ASUS NVIDIA GeForce RTX 3090 . 2.084 |========================================= Mobile Neural Network 2.9.b11b7037d Model: inception-v3 ms < Lower Is Better ASUS NVIDIA GeForce RTX 3090 . 21.37 |========================================= NCNN 20230517 Target: CPU - Model: mobilenet ms < Lower Is Better ASUS NVIDIA GeForce RTX 3090 . 16.46 |========================================= NCNN 20230517 Target: CPU-v2-v2 - Model: mobilenet-v2 ms < Lower Is Better ASUS NVIDIA GeForce RTX 3090 . 5.53 |========================================== NCNN 20230517 Target: CPU-v3-v3 - Model: mobilenet-v3 ms < Lower Is Better ASUS NVIDIA GeForce RTX 3090 . 8.28 |========================================== NCNN 20230517 Target: CPU - Model: shufflenet-v2 ms < Lower Is Better ASUS NVIDIA GeForce RTX 3090 . 11.05 |========================================= NCNN 20230517 Target: CPU - Model: mnasnet ms < Lower Is Better ASUS NVIDIA GeForce RTX 3090 . 5.46 |========================================== NCNN 20230517 Target: CPU - Model: efficientnet-b0 ms < Lower Is Better ASUS NVIDIA GeForce RTX 3090 . 19.35 |========================================= NCNN 20230517 Target: CPU - Model: blazeface ms < Lower Is Better ASUS NVIDIA GeForce RTX 3090 . 5.95 |========================================== NCNN 20230517 Target: CPU - Model: googlenet ms < Lower Is Better ASUS NVIDIA GeForce RTX 3090 . 19.65 |========================================= NCNN 20230517 Target: CPU - Model: vgg16 ms < Lower Is Better ASUS NVIDIA GeForce RTX 3090 . 28.60 |========================================= NCNN 20230517 Target: CPU - Model: resnet18 ms < Lower Is Better ASUS NVIDIA GeForce RTX 3090 . 7.50 |========================================== NCNN 20230517 Target: CPU - Model: alexnet ms < Lower Is Better ASUS NVIDIA GeForce RTX 3090 . 5.58 |========================================== NCNN 20230517 Target: CPU - Model: resnet50 ms < Lower Is Better ASUS NVIDIA GeForce RTX 3090 . 22.09 |========================================= NCNN 20230517 Target: CPU - Model: yolov4-tiny ms < Lower Is Better ASUS NVIDIA GeForce RTX 3090 . 18.40 |========================================= NCNN 20230517 Target: CPU - Model: squeezenet_ssd ms < Lower Is Better ASUS NVIDIA GeForce RTX 3090 . 13.62 |========================================= NCNN 20230517 Target: CPU - Model: regnety_400m ms < Lower Is Better ASUS NVIDIA GeForce RTX 3090 . 82.63 |========================================= NCNN 20230517 Target: CPU - Model: vision_transformer ms < Lower Is Better ASUS NVIDIA GeForce RTX 3090 . 70.73 |========================================= NCNN 20230517 Target: CPU - Model: FastestDet ms < Lower Is Better ASUS NVIDIA GeForce RTX 3090 . 6.47 |========================================== NCNN 20230517 Target: Vulkan GPU - Model: mobilenet ms < Lower Is Better ASUS NVIDIA GeForce RTX 3090 . 17.34 |========================================= NCNN 20230517 Target: Vulkan GPU-v2-v2 - Model: mobilenet-v2 ms < Lower Is Better ASUS NVIDIA GeForce RTX 3090 . 5.72 |========================================== NCNN 20230517 Target: Vulkan GPU-v3-v3 - Model: mobilenet-v3 ms < Lower Is Better ASUS NVIDIA GeForce RTX 3090 . 7.93 |========================================== NCNN 20230517 Target: Vulkan GPU - Model: shufflenet-v2 ms < Lower Is Better ASUS NVIDIA GeForce RTX 3090 . 10.51 |========================================= NCNN 20230517 Target: Vulkan GPU - Model: mnasnet ms < Lower Is Better ASUS NVIDIA GeForce RTX 3090 . 5.43 |========================================== NCNN 20230517 Target: Vulkan GPU - Model: efficientnet-b0 ms < Lower Is Better ASUS NVIDIA GeForce RTX 3090 . 18.73 |========================================= NCNN 20230517 Target: Vulkan GPU - Model: blazeface ms < Lower Is Better ASUS NVIDIA GeForce RTX 3090 . 6.00 |========================================== NCNN 20230517 Target: Vulkan GPU - Model: googlenet ms < Lower Is Better ASUS NVIDIA GeForce RTX 3090 . 19.43 |========================================= NCNN 20230517 Target: Vulkan GPU - Model: vgg16 ms < Lower Is Better ASUS NVIDIA GeForce RTX 3090 . 28.61 |========================================= NCNN 20230517 Target: Vulkan GPU - Model: resnet18 ms < Lower Is Better ASUS NVIDIA GeForce RTX 3090 . 7.29 |========================================== NCNN 20230517 Target: Vulkan GPU - Model: alexnet ms < Lower Is Better ASUS NVIDIA GeForce RTX 3090 . 5.49 |========================================== NCNN 20230517 Target: Vulkan GPU - Model: resnet50 ms < Lower Is Better ASUS NVIDIA GeForce RTX 3090 . 22.13 |========================================= NCNN 20230517 Target: Vulkan GPU - Model: yolov4-tiny ms < Lower Is Better ASUS NVIDIA GeForce RTX 3090 . 18.17 |========================================= NCNN 20230517 Target: Vulkan GPU - Model: squeezenet_ssd ms < Lower Is Better ASUS NVIDIA GeForce RTX 3090 . 14.25 |========================================= NCNN 20230517 Target: Vulkan GPU - Model: regnety_400m ms < Lower Is Better ASUS NVIDIA GeForce RTX 3090 . 84.85 |========================================= NCNN 20230517 Target: Vulkan GPU - Model: vision_transformer ms < Lower Is Better ASUS NVIDIA GeForce RTX 3090 . 70.75 |========================================= NCNN 20230517 Target: Vulkan GPU - Model: FastestDet ms < Lower Is Better ASUS NVIDIA GeForce RTX 3090 . 6.56 |========================================== Neural Magic DeepSparse 1.7 Model: NLP Document Classification, oBERT base uncased on IMDB - Scenario: Asynchronous Multi-Stream items/sec > Higher Is Better Neural Magic DeepSparse 1.7 Model: NLP Document Classification, oBERT base uncased on IMDB - Scenario: Synchronous Single-Stream items/sec > Higher Is Better Neural Magic DeepSparse 1.7 Model: NLP Text Classification, BERT base uncased SST2, Sparse INT8 - Scenario: Asynchronous Multi-Stream items/sec > Higher Is Better Neural Magic DeepSparse 1.7 Model: NLP Text Classification, BERT base uncased SST2, Sparse INT8 - Scenario: Synchronous Single-Stream items/sec > Higher Is Better Neural Magic DeepSparse 1.7 Model: ResNet-50, Baseline - Scenario: Asynchronous Multi-Stream items/sec > Higher Is Better Neural Magic DeepSparse 1.7 Model: ResNet-50, Baseline - Scenario: Synchronous Single-Stream items/sec > Higher Is Better Neural Magic DeepSparse 1.7 Model: ResNet-50, Sparse INT8 - Scenario: Asynchronous Multi-Stream items/sec > Higher Is Better Neural Magic DeepSparse 1.7 Model: ResNet-50, Sparse INT8 - Scenario: Synchronous Single-Stream items/sec > Higher Is Better Neural Magic DeepSparse 1.7 Model: Llama2 Chat 7b Quantized - Scenario: Asynchronous Multi-Stream items/sec > Higher Is Better Neural Magic DeepSparse 1.7 Model: Llama2 Chat 7b Quantized - Scenario: Synchronous Single-Stream items/sec > Higher Is Better Neural Magic DeepSparse 1.7 Model: CV Classification, ResNet-50 ImageNet - Scenario: Asynchronous Multi-Stream items/sec > Higher Is Better Neural Magic DeepSparse 1.7 Model: CV Classification, ResNet-50 ImageNet - Scenario: Synchronous Single-Stream items/sec > Higher Is Better Neural Magic DeepSparse 1.7 Model: CV Detection, YOLOv5s COCO, Sparse INT8 - Scenario: Asynchronous Multi-Stream items/sec > Higher Is Better Neural Magic DeepSparse 1.7 Model: CV Detection, YOLOv5s COCO, Sparse INT8 - Scenario: Synchronous Single-Stream items/sec > Higher Is Better Neural Magic DeepSparse 1.7 Model: NLP Text Classification, DistilBERT mnli - Scenario: Asynchronous Multi-Stream items/sec > Higher Is Better Neural Magic DeepSparse 1.7 Model: NLP Text Classification, DistilBERT mnli - Scenario: Synchronous Single-Stream items/sec > Higher Is Better Neural Magic DeepSparse 1.7 Model: CV Segmentation, 90% Pruned YOLACT Pruned - Scenario: Asynchronous Multi-Stream items/sec > Higher Is Better Neural Magic DeepSparse 1.7 Model: CV Segmentation, 90% Pruned YOLACT Pruned - Scenario: Synchronous Single-Stream items/sec > Higher Is Better Neural Magic DeepSparse 1.7 Model: BERT-Large, NLP Question Answering, Sparse INT8 - Scenario: Asynchronous Multi-Stream items/sec > Higher Is Better Neural Magic DeepSparse 1.7 Model: BERT-Large, NLP Question Answering, Sparse INT8 - Scenario: Synchronous Single-Stream items/sec > Higher Is Better Neural Magic DeepSparse 1.7 Model: NLP Token Classification, BERT base uncased conll2003 - Scenario: Asynchronous Multi-Stream items/sec > Higher Is Better Neural Magic DeepSparse 1.7 Model: NLP Token Classification, BERT base uncased conll2003 - Scenario: Synchronous Single-Stream items/sec > Higher Is Better Numenta Anomaly Benchmark 1.1 Detector: KNN CAD Seconds < Lower Is Better Numenta Anomaly Benchmark 1.1 Detector: Relative Entropy Seconds < Lower Is Better Numenta Anomaly Benchmark 1.1 Detector: Windowed Gaussian Seconds < Lower Is Better Numenta Anomaly Benchmark 1.1 Detector: Earthgecko Skyline Seconds < Lower Is Better Numenta Anomaly Benchmark 1.1 Detector: Bayesian Changepoint Seconds < Lower Is Better Numenta Anomaly Benchmark 1.1 Detector: Contextual Anomaly Detector OSE Seconds < Lower Is Better Numpy Benchmark Score > Higher Is Better ASUS NVIDIA GeForce RTX 3090 . 644.54 |======================================== oneDNN 3.6 Harness: IP Shapes 1D - Engine: CPU ms < Lower Is Better ASUS NVIDIA GeForce RTX 3090 . 2.89253 |======================================= oneDNN 3.6 Harness: IP Shapes 3D - Engine: CPU ms < Lower Is Better ASUS NVIDIA GeForce RTX 3090 . 9.32676 |======================================= oneDNN 3.6 Harness: Convolution Batch Shapes Auto - Engine: CPU ms < Lower Is Better ASUS NVIDIA GeForce RTX 3090 . 8.24915 |======================================= oneDNN 3.6 Harness: Deconvolution Batch shapes_1d - Engine: CPU ms < Lower Is Better ASUS NVIDIA GeForce RTX 3090 . 4.47259 |======================================= oneDNN 3.6 Harness: Deconvolution Batch shapes_3d - Engine: CPU ms < Lower Is Better ASUS NVIDIA GeForce RTX 3090 . 5.57785 |======================================= oneDNN 3.6 Harness: Recurrent Neural Network Training - Engine: CPU ms < Lower Is Better ASUS NVIDIA GeForce RTX 3090 . 2702.00 |======================================= oneDNN 3.6 Harness: Recurrent Neural Network Inference - Engine: CPU ms < Lower Is Better ASUS NVIDIA GeForce RTX 3090 . 1452.76 |======================================= ONNX Runtime 1.19 Model: GPT-2 - Device: CPU - Executor: Parallel Inferences Per Second > Higher Is Better ONNX Runtime 1.19 Model: GPT-2 - Device: CPU - Executor: Standard Inferences Per Second > Higher Is Better ONNX Runtime 1.19 Model: yolov4 - Device: CPU - Executor: Parallel Inferences Per Second > Higher Is Better ASUS NVIDIA GeForce RTX 3090 . 10.49 |========================================= ONNX Runtime 1.19 Model: yolov4 - Device: CPU - Executor: Parallel Inference Time Cost (ms) < Lower Is Better ASUS NVIDIA GeForce RTX 3090 . 95.56 |========================================= ONNX Runtime 1.19 Model: yolov4 - Device: CPU - Executor: Standard Inferences Per Second > Higher Is Better ASUS NVIDIA GeForce RTX 3090 . 11.15 |========================================= ONNX Runtime 1.19 Model: yolov4 - Device: CPU - Executor: Standard Inference Time Cost (ms) < Lower Is Better ASUS NVIDIA GeForce RTX 3090 . 89.76 |========================================= ONNX Runtime 1.19 Model: ZFNet-512 - Device: CPU - Executor: Parallel Inferences Per Second > Higher Is Better ASUS NVIDIA GeForce RTX 3090 . 79.43 |========================================= ONNX Runtime 1.19 Model: ZFNet-512 - Device: CPU - Executor: Parallel Inference Time Cost (ms) < Lower Is Better ASUS NVIDIA GeForce RTX 3090 . 12.59 |========================================= ONNX Runtime 1.19 Model: ZFNet-512 - Device: CPU - Executor: Standard Inferences Per Second > Higher Is Better ASUS NVIDIA GeForce RTX 3090 . 92.47 |========================================= ONNX Runtime 1.19 Model: ZFNet-512 - Device: CPU - Executor: Standard Inference Time Cost (ms) < Lower Is Better ASUS NVIDIA GeForce RTX 3090 . 10.81 |========================================= ONNX Runtime 1.19 Model: T5 Encoder - Device: CPU - Executor: Parallel Inferences Per Second > Higher Is Better ASUS NVIDIA GeForce RTX 3090 . 115.17 |======================================== ONNX Runtime 1.19 Model: T5 Encoder - Device: CPU - Executor: Parallel Inference Time Cost (ms) < Lower Is Better ASUS NVIDIA GeForce RTX 3090 . 8.68614 |======================================= ONNX Runtime 1.19 Model: T5 Encoder - Device: CPU - Executor: Standard Inferences Per Second > Higher Is Better ASUS NVIDIA GeForce RTX 3090 . 152.80 |======================================== ONNX Runtime 1.19 Model: T5 Encoder - Device: CPU - Executor: Standard Inference Time Cost (ms) < Lower Is Better ASUS NVIDIA GeForce RTX 3090 . 6.54304 |======================================= ONNX Runtime 1.19 Model: bertsquad-12 - Device: CPU - Executor: Parallel Inferences Per Second > Higher Is Better ONNX Runtime 1.19 Model: bertsquad-12 - Device: CPU - Executor: Standard Inferences Per Second > Higher Is Better ONNX Runtime 1.19 Model: CaffeNet 12-int8 - Device: CPU - Executor: Parallel Inferences Per Second > Higher Is Better ASUS NVIDIA GeForce RTX 3090 . 307.37 |======================================== ONNX Runtime 1.19 Model: CaffeNet 12-int8 - Device: CPU - Executor: Parallel Inference Time Cost (ms) < Lower Is Better ASUS NVIDIA GeForce RTX 3090 . 3.25179 |======================================= ONNX Runtime 1.19 Model: CaffeNet 12-int8 - Device: CPU - Executor: Standard Inferences Per Second > Higher Is Better ASUS NVIDIA GeForce RTX 3090 . 646.11 |======================================== ONNX Runtime 1.19 Model: CaffeNet 12-int8 - Device: CPU - Executor: Standard Inference Time Cost (ms) < Lower Is Better ASUS NVIDIA GeForce RTX 3090 . 1.54657 |======================================= ONNX Runtime 1.19 Model: fcn-resnet101-11 - Device: CPU - Executor: Parallel Inferences Per Second > Higher Is Better ASUS NVIDIA GeForce RTX 3090 . 1.65652 |======================================= ONNX Runtime 1.19 Model: fcn-resnet101-11 - Device: CPU - Executor: Parallel Inference Time Cost (ms) < Lower Is Better ASUS NVIDIA GeForce RTX 3090 . 604.55 |======================================== ONNX Runtime 1.19 Model: fcn-resnet101-11 - Device: CPU - Executor: Standard Inferences Per Second > Higher Is Better ASUS NVIDIA GeForce RTX 3090 . 2.49779 |======================================= ONNX Runtime 1.19 Model: fcn-resnet101-11 - Device: CPU - Executor: Standard Inference Time Cost (ms) < Lower Is Better ASUS NVIDIA GeForce RTX 3090 . 400.35 |======================================== ONNX Runtime 1.19 Model: ArcFace ResNet-100 - Device: CPU - Executor: Parallel Inferences Per Second > Higher Is Better ONNX Runtime 1.19 Model: ArcFace ResNet-100 - Device: CPU - Executor: Standard Inferences Per Second > Higher Is Better ONNX Runtime 1.19 Model: ResNet50 v1-12-int8 - Device: CPU - Executor: Parallel Inferences Per Second > Higher Is Better ASUS NVIDIA GeForce RTX 3090 . 162.89 |======================================== ONNX Runtime 1.19 Model: ResNet50 v1-12-int8 - Device: CPU - Executor: Parallel Inference Time Cost (ms) < Lower Is Better ASUS NVIDIA GeForce RTX 3090 . 6.14425 |======================================= ONNX Runtime 1.19 Model: ResNet50 v1-12-int8 - Device: CPU - Executor: Standard Inferences Per Second > Higher Is Better ASUS NVIDIA GeForce RTX 3090 . 313.24 |======================================== ONNX Runtime 1.19 Model: ResNet50 v1-12-int8 - Device: CPU - Executor: Standard Inference Time Cost (ms) < Lower Is Better ASUS NVIDIA GeForce RTX 3090 . 3.19151 |======================================= ONNX Runtime 1.19 Model: super-resolution-10 - Device: CPU - Executor: Parallel Inferences Per Second > Higher Is Better ASUS NVIDIA GeForce RTX 3090 . 71.81 |========================================= ONNX Runtime 1.19 Model: super-resolution-10 - Device: CPU - Executor: Parallel Inference Time Cost (ms) < Lower Is Better ASUS NVIDIA GeForce RTX 3090 . 13.94 |========================================= ONNX Runtime 1.19 Model: super-resolution-10 - Device: CPU - Executor: Standard Inferences Per Second > Higher Is Better ASUS NVIDIA GeForce RTX 3090 . 81.78 |========================================= ONNX Runtime 1.19 Model: super-resolution-10 - Device: CPU - Executor: Standard Inference Time Cost (ms) < Lower Is Better ASUS NVIDIA GeForce RTX 3090 . 12.23 |========================================= ONNX Runtime 1.19 Model: ResNet101_DUC_HDC-12 - Device: CPU - Executor: Parallel Inferences Per Second > Higher Is Better ASUS NVIDIA GeForce RTX 3090 . 0.743436 |====================================== ONNX Runtime 1.19 Model: ResNet101_DUC_HDC-12 - Device: CPU - Executor: Parallel Inference Time Cost (ms) < Lower Is Better ASUS NVIDIA GeForce RTX 3090 . 1345.66 |======================================= ONNX Runtime 1.19 Model: ResNet101_DUC_HDC-12 - Device: CPU - Executor: Standard Inferences Per Second > Higher Is Better ASUS NVIDIA GeForce RTX 3090 . 0.943867 |====================================== ONNX Runtime 1.19 Model: ResNet101_DUC_HDC-12 - Device: CPU - Executor: Standard Inference Time Cost (ms) < Lower Is Better ASUS NVIDIA GeForce RTX 3090 . 1059.49 |======================================= ONNX Runtime 1.19 Model: Faster R-CNN R-50-FPN-int8 - Device: CPU - Executor: Parallel Inferences Per Second > Higher Is Better ONNX Runtime 1.19 Model: Faster R-CNN R-50-FPN-int8 - Device: CPU - Executor: Standard Inferences Per Second > Higher Is Better OpenCV 4.7 Test: DNN - Deep Neural Network ms < Lower Is Better ASUS NVIDIA GeForce RTX 3090 . 25552 |========================================= OpenVINO 2024.0 Model: Face Detection FP16 - Device: CPU FPS > Higher Is Better ASUS NVIDIA GeForce RTX 3090 . 4.33 |========================================== OpenVINO 2024.0 Model: Face Detection FP16 - Device: CPU ms < Lower Is Better ASUS NVIDIA GeForce RTX 3090 . 1375.14 |======================================= OpenVINO 2024.0 Model: Person Detection FP16 - Device: CPU FPS > Higher Is Better ASUS NVIDIA GeForce RTX 3090 . 40.98 |========================================= OpenVINO 2024.0 Model: Person Detection FP16 - Device: CPU ms < Lower Is Better ASUS NVIDIA GeForce RTX 3090 . 146.25 |======================================== OpenVINO 2024.0 Model: Person Detection FP32 - Device: CPU FPS > Higher Is Better ASUS NVIDIA GeForce RTX 3090 . 41.47 |========================================= OpenVINO 2024.0 Model: Person Detection FP32 - Device: CPU ms < Lower Is Better ASUS NVIDIA GeForce RTX 3090 . 144.47 |======================================== OpenVINO 2024.0 Model: Vehicle Detection FP16 - Device: CPU FPS > Higher Is Better ASUS NVIDIA GeForce RTX 3090 . 296.62 |======================================== OpenVINO 2024.0 Model: Vehicle Detection FP16 - Device: CPU ms < Lower Is Better ASUS NVIDIA GeForce RTX 3090 . 20.16 |========================================= OpenVINO 2024.0 Model: Face Detection FP16-INT8 - Device: CPU FPS > Higher Is Better ASUS NVIDIA GeForce RTX 3090 . 15.61 |========================================= OpenVINO 2024.0 Model: Face Detection FP16-INT8 - Device: CPU ms < Lower Is Better ASUS NVIDIA GeForce RTX 3090 . 382.74 |======================================== OpenVINO 2024.0 Model: Face Detection Retail FP16 - Device: CPU FPS > Higher Is Better ASUS NVIDIA GeForce RTX 3090 . 1180.46 |======================================= OpenVINO 2024.0 Model: Face Detection Retail FP16 - Device: CPU ms < Lower Is Better ASUS NVIDIA GeForce RTX 3090 . 5.02 |========================================== OpenVINO 2024.0 Model: Road Segmentation ADAS FP16 - Device: CPU FPS > Higher Is Better ASUS NVIDIA GeForce RTX 3090 . 86.71 |========================================= OpenVINO 2024.0 Model: Road Segmentation ADAS FP16 - Device: CPU ms < Lower Is Better ASUS NVIDIA GeForce RTX 3090 . 69.09 |========================================= OpenVINO 2024.0 Model: Vehicle Detection FP16-INT8 - Device: CPU FPS > Higher Is Better ASUS NVIDIA GeForce RTX 3090 . 712.75 |======================================== OpenVINO 2024.0 Model: Vehicle Detection FP16-INT8 - Device: CPU ms < Lower Is Better ASUS NVIDIA GeForce RTX 3090 . 8.35 |========================================== OpenVINO 2024.0 Model: Weld Porosity Detection FP16 - Device: CPU FPS > Higher Is Better ASUS NVIDIA GeForce RTX 3090 . 450.44 |======================================== OpenVINO 2024.0 Model: Weld Porosity Detection FP16 - Device: CPU ms < Lower Is Better ASUS NVIDIA GeForce RTX 3090 . 44.26 |========================================= OpenVINO 2024.0 Model: Face Detection Retail FP16-INT8 - Device: CPU FPS > Higher Is Better ASUS NVIDIA GeForce RTX 3090 . 1948.78 |======================================= OpenVINO 2024.0 Model: Face Detection Retail FP16-INT8 - Device: CPU ms < Lower Is Better ASUS NVIDIA GeForce RTX 3090 . 2.97 |========================================== OpenVINO 2024.0 Model: Road Segmentation ADAS FP16-INT8 - Device: CPU FPS > Higher Is Better ASUS NVIDIA GeForce RTX 3090 . 283.34 |======================================== OpenVINO 2024.0 Model: Road Segmentation ADAS FP16-INT8 - Device: CPU ms < Lower Is Better ASUS NVIDIA GeForce RTX 3090 . 21.09 |========================================= OpenVINO 2024.0 Model: Machine Translation EN To DE FP16 - Device: CPU FPS > Higher Is Better ASUS NVIDIA GeForce RTX 3090 . 51.69 |========================================= OpenVINO 2024.0 Model: Machine Translation EN To DE FP16 - Device: CPU ms < Lower Is Better ASUS NVIDIA GeForce RTX 3090 . 115.87 |======================================== OpenVINO 2024.0 Model: Weld Porosity Detection FP16-INT8 - Device: CPU FPS > Higher Is Better ASUS NVIDIA GeForce RTX 3090 . 1544.54 |======================================= OpenVINO 2024.0 Model: Weld Porosity Detection FP16-INT8 - Device: CPU ms < Lower Is Better ASUS NVIDIA GeForce RTX 3090 . 12.81 |========================================= OpenVINO 2024.0 Model: Person Vehicle Bike Detection FP16 - Device: CPU FPS > Higher Is Better ASUS NVIDIA GeForce RTX 3090 . 528.13 |======================================== OpenVINO 2024.0 Model: Person Vehicle Bike Detection FP16 - Device: CPU ms < Lower Is Better ASUS NVIDIA GeForce RTX 3090 . 11.29 |========================================= OpenVINO 2024.0 Model: Noise Suppression Poconet-Like FP16 - Device: CPU FPS > Higher Is Better ASUS NVIDIA GeForce RTX 3090 . 656.86 |======================================== OpenVINO 2024.0 Model: Noise Suppression Poconet-Like FP16 - Device: CPU ms < Lower Is Better ASUS NVIDIA GeForce RTX 3090 . 9.02 |========================================== OpenVINO 2024.0 Model: Handwritten English Recognition FP16 - Device: CPU FPS > Higher Is Better ASUS NVIDIA GeForce RTX 3090 . 225.32 |======================================== OpenVINO 2024.0 Model: Handwritten English Recognition FP16 - Device: CPU ms < Lower Is Better ASUS NVIDIA GeForce RTX 3090 . 88.65 |========================================= OpenVINO 2024.0 Model: Person Re-Identification Retail FP16 - Device: CPU FPS > Higher Is Better ASUS NVIDIA GeForce RTX 3090 . 659.72 |======================================== OpenVINO 2024.0 Model: Person Re-Identification Retail FP16 - Device: CPU ms < Lower Is Better ASUS NVIDIA GeForce RTX 3090 . 9.01 |========================================== OpenVINO 2024.0 Model: Age Gender Recognition Retail 0013 FP16 - Device: CPU FPS > Higher Is Better ASUS NVIDIA GeForce RTX 3090 . 11244.68 |====================================== OpenVINO 2024.0 Model: Age Gender Recognition Retail 0013 FP16 - Device: CPU ms < Lower Is Better ASUS NVIDIA GeForce RTX 3090 . 1.66 |========================================== OpenVINO 2024.0 Model: Handwritten English Recognition FP16-INT8 - Device: CPU FPS > Higher Is Better ASUS NVIDIA GeForce RTX 3090 . 266.87 |======================================== OpenVINO 2024.0 Model: Handwritten English Recognition FP16-INT8 - Device: CPU ms < Lower Is Better ASUS NVIDIA GeForce RTX 3090 . 74.85 |========================================= OpenVINO 2024.0 Model: Age Gender Recognition Retail 0013 FP16-INT8 - Device: CPU FPS > Higher Is Better ASUS NVIDIA GeForce RTX 3090 . 29697.31 |====================================== OpenVINO 2024.0 Model: Age Gender Recognition Retail 0013 FP16-INT8 - Device: CPU ms < Lower Is Better ASUS NVIDIA GeForce RTX 3090 . 0.62 |========================================== PlaidML FP16: No - Mode: Inference - Network: VGG16 - Device: CPU Examples Per Second > Higher Is Better PlaidML FP16: No - Mode: Inference - Network: ResNet 50 - Device: CPU Examples Per Second > Higher Is Better PyTorch 2.2.1 Device: CPU - Batch Size: 1 - Model: ResNet-50 batches/sec > Higher Is Better ASUS NVIDIA GeForce RTX 3090 . 49.49 |========================================= PyTorch 2.2.1 Device: CPU - Batch Size: 1 - Model: ResNet-152 batches/sec > Higher Is Better ASUS NVIDIA GeForce RTX 3090 . 18.80 |========================================= PyTorch 2.2.1 Device: CPU - Batch Size: 16 - Model: ResNet-50 batches/sec > Higher Is Better ASUS NVIDIA GeForce RTX 3090 . 30.07 |========================================= PyTorch 2.2.1 Device: CPU - Batch Size: 32 - Model: ResNet-50 batches/sec > Higher Is Better ASUS NVIDIA GeForce RTX 3090 . 30.27 |========================================= PyTorch 2.2.1 Device: CPU - Batch Size: 64 - Model: ResNet-50 batches/sec > Higher Is Better ASUS NVIDIA GeForce RTX 3090 . 29.52 |========================================= PyTorch 2.2.1 Device: CPU - Batch Size: 16 - Model: ResNet-152 batches/sec > Higher Is Better ASUS NVIDIA GeForce RTX 3090 . 11.49 |========================================= PyTorch 2.2.1 Device: CPU - Batch Size: 256 - Model: ResNet-50 batches/sec > Higher Is Better ASUS NVIDIA GeForce RTX 3090 . 29.33 |========================================= PyTorch 2.2.1 Device: CPU - Batch Size: 32 - Model: ResNet-152 batches/sec > Higher Is Better ASUS NVIDIA GeForce RTX 3090 . 10.16 |========================================= PyTorch 2.2.1 Device: CPU - Batch Size: 512 - Model: ResNet-50 batches/sec > Higher Is Better ASUS NVIDIA GeForce RTX 3090 . 30.14 |========================================= PyTorch 2.2.1 Device: CPU - Batch Size: 64 - Model: ResNet-152 batches/sec > Higher Is Better ASUS NVIDIA GeForce RTX 3090 . 11.66 |========================================= PyTorch 2.2.1 Device: CPU - Batch Size: 256 - Model: ResNet-152 batches/sec > Higher Is Better ASUS NVIDIA GeForce RTX 3090 . 11.45 |========================================= PyTorch 2.2.1 Device: CPU - Batch Size: 512 - Model: ResNet-152 batches/sec > Higher Is Better ASUS NVIDIA GeForce RTX 3090 . 11.46 |========================================= PyTorch 2.2.1 Device: CPU - Batch Size: 1 - Model: Efficientnet_v2_l batches/sec > Higher Is Better ASUS NVIDIA GeForce RTX 3090 . 10.55 |========================================= PyTorch 2.2.1 Device: CPU - Batch Size: 16 - Model: Efficientnet_v2_l batches/sec > Higher Is Better ASUS NVIDIA GeForce RTX 3090 . 6.97 |========================================== PyTorch 2.2.1 Device: CPU - Batch Size: 32 - Model: Efficientnet_v2_l batches/sec > Higher Is Better ASUS NVIDIA GeForce RTX 3090 . 7.92 |========================================== PyTorch 2.2.1 Device: CPU - Batch Size: 64 - Model: Efficientnet_v2_l batches/sec > Higher Is Better ASUS NVIDIA GeForce RTX 3090 . 7.82 |========================================== PyTorch 2.2.1 Device: CPU - Batch Size: 256 - Model: Efficientnet_v2_l batches/sec > Higher Is Better ASUS NVIDIA GeForce RTX 3090 . 6.48 |========================================== PyTorch 2.2.1 Device: CPU - Batch Size: 512 - Model: Efficientnet_v2_l batches/sec > Higher Is Better ASUS NVIDIA GeForce RTX 3090 . 7.26 |========================================== PyTorch 2.2.1 Device: NVIDIA CUDA GPU - Batch Size: 1 - Model: ResNet-50 batches/sec > Higher Is Better ASUS NVIDIA GeForce RTX 3090 . 469.71 |======================================== PyTorch 2.2.1 Device: NVIDIA CUDA GPU - Batch Size: 1 - Model: ResNet-152 batches/sec > Higher Is Better ASUS NVIDIA GeForce RTX 3090 . 172.05 |======================================== PyTorch 2.2.1 Device: NVIDIA CUDA GPU - Batch Size: 16 - Model: ResNet-50 batches/sec > Higher Is Better ASUS NVIDIA GeForce RTX 3090 . 414.29 |======================================== PyTorch 2.2.1 Device: NVIDIA CUDA GPU - Batch Size: 32 - Model: ResNet-50 batches/sec > Higher Is Better ASUS NVIDIA GeForce RTX 3090 . 413.19 |======================================== PyTorch 2.2.1 Device: NVIDIA CUDA GPU - Batch Size: 64 - Model: ResNet-50 batches/sec > Higher Is Better ASUS NVIDIA GeForce RTX 3090 . 413.29 |======================================== PyTorch 2.2.1 Device: NVIDIA CUDA GPU - Batch Size: 16 - Model: ResNet-152 batches/sec > Higher Is Better ASUS NVIDIA GeForce RTX 3090 . 162.59 |======================================== PyTorch 2.2.1 Device: NVIDIA CUDA GPU - Batch Size: 256 - Model: ResNet-50 batches/sec > Higher Is Better ASUS NVIDIA GeForce RTX 3090 . 413.12 |======================================== PyTorch 2.2.1 Device: NVIDIA CUDA GPU - Batch Size: 32 - Model: ResNet-152 batches/sec > Higher Is Better ASUS NVIDIA GeForce RTX 3090 . 162.45 |======================================== PyTorch 2.2.1 Device: NVIDIA CUDA GPU - Batch Size: 512 - Model: ResNet-50 batches/sec > Higher Is Better ASUS NVIDIA GeForce RTX 3090 . 414.51 |======================================== PyTorch 2.2.1 Device: NVIDIA CUDA GPU - Batch Size: 64 - Model: ResNet-152 batches/sec > Higher Is Better ASUS NVIDIA GeForce RTX 3090 . 162.69 |======================================== PyTorch 2.2.1 Device: NVIDIA CUDA GPU - Batch Size: 256 - Model: ResNet-152 batches/sec > Higher Is Better ASUS NVIDIA GeForce RTX 3090 . 162.68 |======================================== PyTorch 2.2.1 Device: NVIDIA CUDA GPU - Batch Size: 512 - Model: ResNet-152 batches/sec > Higher Is Better ASUS NVIDIA GeForce RTX 3090 . 162.35 |======================================== PyTorch 2.2.1 Device: NVIDIA CUDA GPU - Batch Size: 1 - Model: Efficientnet_v2_l batches/sec > Higher Is Better ASUS NVIDIA GeForce RTX 3090 . 88.41 |========================================= PyTorch 2.2.1 Device: NVIDIA CUDA GPU - Batch Size: 16 - Model: Efficientnet_v2_l batches/sec > Higher Is Better ASUS NVIDIA GeForce RTX 3090 . 85.74 |========================================= PyTorch 2.2.1 Device: NVIDIA CUDA GPU - Batch Size: 32 - Model: Efficientnet_v2_l batches/sec > Higher Is Better ASUS NVIDIA GeForce RTX 3090 . 84.09 |========================================= PyTorch 2.2.1 Device: NVIDIA CUDA GPU - Batch Size: 64 - Model: Efficientnet_v2_l batches/sec > Higher Is Better ASUS NVIDIA GeForce RTX 3090 . 85.37 |========================================= PyTorch 2.2.1 Device: NVIDIA CUDA GPU - Batch Size: 256 - Model: Efficientnet_v2_l batches/sec > Higher Is Better ASUS NVIDIA GeForce RTX 3090 . 85.57 |========================================= PyTorch 2.2.1 Device: NVIDIA CUDA GPU - Batch Size: 512 - Model: Efficientnet_v2_l batches/sec > Higher Is Better ASUS NVIDIA GeForce RTX 3090 . 86.27 |========================================= R Benchmark Seconds < Lower Is Better ASUS NVIDIA GeForce RTX 3090 . 0.0970 |======================================== RNNoise 0.2 Input: 26 Minute Long Talking Sample Seconds < Lower Is Better ASUS NVIDIA GeForce RTX 3090 . 7.245 |========================================= Scikit-Learn 1.2.2 Benchmark: GLM Seconds < Lower Is Better ASUS NVIDIA GeForce RTX 3090 . 183.48 |======================================== Scikit-Learn 1.2.2 Benchmark: SAGA Seconds < Lower Is Better ASUS NVIDIA GeForce RTX 3090 . 521.31 |======================================== Scikit-Learn 1.2.2 Benchmark: Tree Seconds < Lower Is Better ASUS NVIDIA GeForce RTX 3090 . 42.77 |========================================= Scikit-Learn 1.2.2 Benchmark: Lasso Seconds < Lower Is Better ASUS NVIDIA GeForce RTX 3090 . 258.72 |======================================== Scikit-Learn 1.2.2 Benchmark: Glmnet Seconds < Lower Is Better Scikit-Learn 1.2.2 Benchmark: Sparsify Seconds < Lower Is Better ASUS NVIDIA GeForce RTX 3090 . 85.98 |========================================= Scikit-Learn 1.2.2 Benchmark: Plot Ward Seconds < Lower Is Better ASUS NVIDIA GeForce RTX 3090 . 43.62 |========================================= Scikit-Learn 1.2.2 Benchmark: MNIST Dataset Seconds < Lower Is Better ASUS NVIDIA GeForce RTX 3090 . 53.54 |========================================= Scikit-Learn 1.2.2 Benchmark: Plot Neighbors Seconds < Lower Is Better ASUS NVIDIA GeForce RTX 3090 . 96.76 |========================================= Scikit-Learn 1.2.2 Benchmark: SGD Regression Seconds < Lower Is Better ASUS NVIDIA GeForce RTX 3090 . 63.79 |========================================= Scikit-Learn 1.2.2 Benchmark: SGDOneClassSVM Seconds < Lower Is Better ASUS NVIDIA GeForce RTX 3090 . 196.47 |======================================== Scikit-Learn 1.2.2 Benchmark: Plot Lasso Path Seconds < Lower Is Better Scikit-Learn 1.2.2 Benchmark: Isolation Forest Seconds < Lower Is Better ASUS NVIDIA GeForce RTX 3090 . 147.52 |======================================== Scikit-Learn 1.2.2 Benchmark: Plot Fast KMeans Seconds < Lower Is Better Scikit-Learn 1.2.2 Benchmark: Text Vectorizers Seconds < Lower Is Better ASUS NVIDIA GeForce RTX 3090 . 38.91 |========================================= Scikit-Learn 1.2.2 Benchmark: Plot Hierarchical Seconds < Lower Is Better ASUS NVIDIA GeForce RTX 3090 . 153.06 |======================================== Scikit-Learn 1.2.2 Benchmark: Plot OMP vs. LARS Seconds < Lower Is Better ASUS NVIDIA GeForce RTX 3090 . 45.62 |========================================= Scikit-Learn 1.2.2 Benchmark: Feature Expansions Seconds < Lower Is Better ASUS NVIDIA GeForce RTX 3090 . 120.33 |======================================== Scikit-Learn 1.2.2 Benchmark: LocalOutlierFactor Seconds < Lower Is Better ASUS NVIDIA GeForce RTX 3090 . 35.65 |========================================= Scikit-Learn 1.2.2 Benchmark: TSNE MNIST Dataset Seconds < Lower Is Better ASUS NVIDIA GeForce RTX 3090 . 188.33 |======================================== Scikit-Learn 1.2.2 Benchmark: Isotonic / Logistic Seconds < Lower Is Better ASUS NVIDIA GeForce RTX 3090 . 1447.69 |======================================= Scikit-Learn 1.2.2 Benchmark: Plot Incremental PCA Seconds < Lower Is Better ASUS NVIDIA GeForce RTX 3090 . 21.31 |========================================= Scikit-Learn 1.2.2 Benchmark: Hist Gradient Boosting Seconds < Lower Is Better ASUS NVIDIA GeForce RTX 3090 . 1120.04 |======================================= Scikit-Learn 1.2.2 Benchmark: Plot Parallel Pairwise Seconds < Lower Is Better ASUS NVIDIA GeForce RTX 3090 . 332.19 |======================================== Scikit-Learn 1.2.2 Benchmark: Isotonic / Pathological Seconds < Lower Is Better Scikit-Learn 1.2.2 Benchmark: RCV1 Logreg Convergencet Seconds < Lower Is Better Scikit-Learn 1.2.2 Benchmark: Sample Without Replacement Seconds < Lower Is Better ASUS NVIDIA GeForce RTX 3090 . 84.27 |========================================= Scikit-Learn 1.2.2 Benchmark: Covertype Dataset Benchmark Seconds < Lower Is Better ASUS NVIDIA GeForce RTX 3090 . 317.46 |======================================== Scikit-Learn 1.2.2 Benchmark: Hist Gradient Boosting Adult Seconds < Lower Is Better ASUS NVIDIA GeForce RTX 3090 . 1173.89 |======================================= Scikit-Learn 1.2.2 Benchmark: Isotonic / Perturbed Logarithm Seconds < Lower Is Better ASUS NVIDIA GeForce RTX 3090 . 1469.79 |======================================= Scikit-Learn 1.2.2 Benchmark: Hist Gradient Boosting Threading Seconds < Lower Is Better ASUS NVIDIA GeForce RTX 3090 . 140.75 |======================================== Scikit-Learn 1.2.2 Benchmark: Plot Singular Value Decomposition Seconds < Lower Is Better Scikit-Learn 1.2.2 Benchmark: Hist Gradient Boosting Higgs Boson Seconds < Lower Is Better ASUS NVIDIA GeForce RTX 3090 . 190.58 |======================================== Scikit-Learn 1.2.2 Benchmark: 20 Newsgroups / Logistic Regression Seconds < Lower Is Better ASUS NVIDIA GeForce RTX 3090 . 9.614 |========================================= Scikit-Learn 1.2.2 Benchmark: Plot Polynomial Kernel Approximation Seconds < Lower Is Better ASUS NVIDIA GeForce RTX 3090 . 133.12 |======================================== Scikit-Learn 1.2.2 Benchmark: Plot Non-Negative Matrix Factorization Seconds < Lower Is Better Scikit-Learn 1.2.2 Benchmark: Hist Gradient Boosting Categorical Only Seconds < Lower Is Better ASUS NVIDIA GeForce RTX 3090 . 195.30 |======================================== Scikit-Learn 1.2.2 Benchmark: Kernel PCA Solvers / Time vs. N Samples Seconds < Lower Is Better ASUS NVIDIA GeForce RTX 3090 . 69.63 |========================================= Scikit-Learn 1.2.2 Benchmark: Kernel PCA Solvers / Time vs. N Components Seconds < Lower Is Better ASUS NVIDIA GeForce RTX 3090 . 25.59 |========================================= Scikit-Learn 1.2.2 Benchmark: Sparse Random Projections / 100 Iterations Seconds < Lower Is Better ASUS NVIDIA GeForce RTX 3090 . 587.84 |======================================== SHOC Scalable HeterOgeneous Computing 2020-04-17 Target: OpenCL - Benchmark: S3D GFLOPS > Higher Is Better ASUS NVIDIA GeForce RTX 3090 . 455.96 |======================================== SHOC Scalable HeterOgeneous Computing 2020-04-17 Target: OpenCL - Benchmark: Triad GB/s > Higher Is Better ASUS NVIDIA GeForce RTX 3090 . 6.6057 |======================================== SHOC Scalable HeterOgeneous Computing 2020-04-17 Target: OpenCL - Benchmark: FFT SP GFLOPS > Higher Is Better ASUS NVIDIA GeForce RTX 3090 . 2510.98 |======================================= SHOC Scalable HeterOgeneous Computing 2020-04-17 Target: OpenCL - Benchmark: MD5 Hash GHash/s > Higher Is Better ASUS NVIDIA GeForce RTX 3090 . 45.45 |========================================= SHOC Scalable HeterOgeneous Computing 2020-04-17 Target: OpenCL - Benchmark: Reduction GB/s > Higher Is Better ASUS NVIDIA GeForce RTX 3090 . 406.41 |======================================== SHOC Scalable HeterOgeneous Computing 2020-04-17 Target: OpenCL - Benchmark: GEMM SGEMM_N GFLOPS > Higher Is Better ASUS NVIDIA GeForce RTX 3090 . 8456.99 |======================================= SHOC Scalable HeterOgeneous Computing 2020-04-17 Target: OpenCL - Benchmark: Max SP Flops GFLOPS > Higher Is Better ASUS NVIDIA GeForce RTX 3090 . 41982.2 |======================================= SHOC Scalable HeterOgeneous Computing 2020-04-17 Target: OpenCL - Benchmark: Bus Speed Download GB/s > Higher Is Better ASUS NVIDIA GeForce RTX 3090 . 6.6413 |======================================== SHOC Scalable HeterOgeneous Computing 2020-04-17 Target: OpenCL - Benchmark: Bus Speed Readback GB/s > Higher Is Better ASUS NVIDIA GeForce RTX 3090 . 6.7652 |======================================== SHOC Scalable HeterOgeneous Computing 2020-04-17 Target: OpenCL - Benchmark: Texture Read Bandwidth GB/s > Higher Is Better ASUS NVIDIA GeForce RTX 3090 . 2178.63 |======================================= spaCy 3.4.1 tokens/sec > Higher Is Better TensorFlow 2.16.1 Device: CPU - Batch Size: 1 - Model: VGG-16 images/sec > Higher Is Better ASUS NVIDIA GeForce RTX 3090 . 4.12 |========================================== TensorFlow 2.16.1 Device: GPU - Batch Size: 1 - Model: VGG-16 images/sec > Higher Is Better ASUS NVIDIA GeForce RTX 3090 . 1.57 |========================================== TensorFlow 2.16.1 Device: CPU - Batch Size: 1 - Model: AlexNet images/sec > Higher Is Better ASUS NVIDIA GeForce RTX 3090 . 15.83 |========================================= TensorFlow 2.16.1 Device: CPU - Batch Size: 16 - Model: VGG-16 images/sec > Higher Is Better ASUS NVIDIA GeForce RTX 3090 . 8.97 |========================================== TensorFlow 2.16.1 Device: CPU - Batch Size: 32 - Model: VGG-16 images/sec > Higher Is Better ASUS NVIDIA GeForce RTX 3090 . 9.22 |========================================== TensorFlow 2.16.1 Device: CPU - Batch Size: 64 - Model: VGG-16 images/sec > Higher Is Better ASUS NVIDIA GeForce RTX 3090 . 9.31 |========================================== TensorFlow 2.16.1 Device: GPU - Batch Size: 1 - Model: AlexNet images/sec > Higher Is Better ASUS NVIDIA GeForce RTX 3090 . 14.14 |========================================= TensorFlow 2.16.1 Device: GPU - Batch Size: 16 - Model: VGG-16 images/sec > Higher Is Better ASUS NVIDIA GeForce RTX 3090 . 2.25 |========================================== TensorFlow 2.16.1 Device: GPU - Batch Size: 32 - Model: VGG-16 images/sec > Higher Is Better ASUS NVIDIA GeForce RTX 3090 . 2.27 |========================================== TensorFlow 2.16.1 Device: GPU - Batch Size: 64 - Model: VGG-16 images/sec > Higher Is Better ASUS NVIDIA GeForce RTX 3090 . 2.27 |========================================== TensorFlow 2.16.1 Device: CPU - Batch Size: 16 - Model: AlexNet images/sec > Higher Is Better ASUS NVIDIA GeForce RTX 3090 . 129.70 |======================================== TensorFlow 2.16.1 Device: CPU - Batch Size: 256 - Model: VGG-16 images/sec > Higher Is Better ASUS NVIDIA GeForce RTX 3090 . 9.11 |========================================== TensorFlow 2.16.1 Device: CPU - Batch Size: 32 - Model: AlexNet images/sec > Higher Is Better ASUS NVIDIA GeForce RTX 3090 . 167.17 |======================================== TensorFlow 2.16.1 Device: CPU - Batch Size: 512 - Model: VGG-16 images/sec > Higher Is Better ASUS NVIDIA GeForce RTX 3090 . 8.95 |========================================== TensorFlow 2.16.1 Device: CPU - Batch Size: 64 - Model: AlexNet images/sec > Higher Is Better ASUS NVIDIA GeForce RTX 3090 . 192.13 |======================================== TensorFlow 2.16.1 Device: GPU - Batch Size: 16 - Model: AlexNet images/sec > Higher Is Better ASUS NVIDIA GeForce RTX 3090 . 38.93 |========================================= TensorFlow 2.16.1 Device: GPU - Batch Size: 256 - Model: VGG-16 images/sec > Higher Is Better ASUS NVIDIA GeForce RTX 3090 . 2.28 |========================================== TensorFlow 2.16.1 Device: GPU - Batch Size: 32 - Model: AlexNet images/sec > Higher Is Better ASUS NVIDIA GeForce RTX 3090 . 41.45 |========================================= TensorFlow 2.16.1 Device: GPU - Batch Size: 512 - Model: VGG-16 images/sec > Higher Is Better ASUS NVIDIA GeForce RTX 3090 . 2.29 |========================================== TensorFlow 2.16.1 Device: GPU - Batch Size: 64 - Model: AlexNet images/sec > Higher Is Better ASUS NVIDIA GeForce RTX 3090 . 42.49 |========================================= TensorFlow 2.16.1 Device: CPU - Batch Size: 1 - Model: GoogLeNet images/sec > Higher Is Better ASUS NVIDIA GeForce RTX 3090 . 47.92 |========================================= TensorFlow 2.16.1 Device: CPU - Batch Size: 1 - Model: ResNet-50 images/sec > Higher Is Better ASUS NVIDIA GeForce RTX 3090 . 14.04 |========================================= TensorFlow 2.16.1 Device: CPU - Batch Size: 256 - Model: AlexNet images/sec > Higher Is Better ASUS NVIDIA GeForce RTX 3090 . 227.40 |======================================== TensorFlow 2.16.1 Device: CPU - Batch Size: 512 - Model: AlexNet images/sec > Higher Is Better ASUS NVIDIA GeForce RTX 3090 . 245.67 |======================================== TensorFlow 2.16.1 Device: GPU - Batch Size: 1 - Model: GoogLeNet images/sec > Higher Is Better ASUS NVIDIA GeForce RTX 3090 . 12.00 |========================================= TensorFlow 2.16.1 Device: GPU - Batch Size: 1 - Model: ResNet-50 images/sec > Higher Is Better ASUS NVIDIA GeForce RTX 3090 . 5.62 |========================================== TensorFlow 2.16.1 Device: GPU - Batch Size: 256 - Model: AlexNet images/sec > Higher Is Better ASUS NVIDIA GeForce RTX 3090 . 44.05 |========================================= TensorFlow 2.16.1 Device: GPU - Batch Size: 512 - Model: AlexNet images/sec > Higher Is Better ASUS NVIDIA GeForce RTX 3090 . 44.64 |========================================= TensorFlow 2.16.1 Device: CPU - Batch Size: 16 - Model: GoogLeNet images/sec > Higher Is Better ASUS NVIDIA GeForce RTX 3090 . 101.50 |======================================== TensorFlow 2.16.1 Device: CPU - Batch Size: 16 - Model: ResNet-50 images/sec > Higher Is Better ASUS NVIDIA GeForce RTX 3090 . 27.80 |========================================= TensorFlow 2.16.1 Device: CPU - Batch Size: 32 - Model: GoogLeNet images/sec > Higher Is Better ASUS NVIDIA GeForce RTX 3090 . 96.69 |========================================= TensorFlow 2.16.1 Device: CPU - Batch Size: 32 - Model: ResNet-50 images/sec > Higher Is Better ASUS NVIDIA GeForce RTX 3090 . 26.82 |========================================= TensorFlow 2.16.1 Device: CPU - Batch Size: 64 - Model: GoogLeNet images/sec > Higher Is Better ASUS NVIDIA GeForce RTX 3090 . 94.21 |========================================= TensorFlow 2.16.1 Device: CPU - Batch Size: 64 - Model: ResNet-50 images/sec > Higher Is Better ASUS NVIDIA GeForce RTX 3090 . 26.52 |========================================= TensorFlow 2.16.1 Device: GPU - Batch Size: 16 - Model: GoogLeNet images/sec > Higher Is Better ASUS NVIDIA GeForce RTX 3090 . 25.99 |========================================= TensorFlow 2.16.1 Device: GPU - Batch Size: 16 - Model: ResNet-50 images/sec > Higher Is Better ASUS NVIDIA GeForce RTX 3090 . 7.93 |========================================== TensorFlow 2.16.1 Device: GPU - Batch Size: 32 - Model: GoogLeNet images/sec > Higher Is Better ASUS NVIDIA GeForce RTX 3090 . 26.63 |========================================= TensorFlow 2.16.1 Device: GPU - Batch Size: 32 - Model: ResNet-50 images/sec > Higher Is Better ASUS NVIDIA GeForce RTX 3090 . 7.98 |========================================== TensorFlow 2.16.1 Device: GPU - Batch Size: 64 - Model: GoogLeNet images/sec > Higher Is Better ASUS NVIDIA GeForce RTX 3090 . 26.91 |========================================= TensorFlow 2.16.1 Device: GPU - Batch Size: 64 - Model: ResNet-50 images/sec > Higher Is Better ASUS NVIDIA GeForce RTX 3090 . 8.00 |========================================== TensorFlow 2.16.1 Device: CPU - Batch Size: 256 - Model: GoogLeNet images/sec > Higher Is Better ASUS NVIDIA GeForce RTX 3090 . 93.45 |========================================= TensorFlow 2.16.1 Device: CPU - Batch Size: 256 - Model: ResNet-50 images/sec > Higher Is Better ASUS NVIDIA GeForce RTX 3090 . 27.49 |========================================= TensorFlow 2.16.1 Device: CPU - Batch Size: 512 - Model: GoogLeNet images/sec > Higher Is Better ASUS NVIDIA GeForce RTX 3090 . 93.78 |========================================= TensorFlow 2.16.1 Device: CPU - Batch Size: 512 - Model: ResNet-50 images/sec > Higher Is Better ASUS NVIDIA GeForce RTX 3090 . 28.13 |========================================= TensorFlow 2.16.1 Device: GPU - Batch Size: 256 - Model: GoogLeNet images/sec > Higher Is Better ASUS NVIDIA GeForce RTX 3090 . 27.04 |========================================= TensorFlow 2.16.1 Device: GPU - Batch Size: 256 - Model: ResNet-50 images/sec > Higher Is Better ASUS NVIDIA GeForce RTX 3090 . 8.04 |========================================== TensorFlow 2.16.1 Device: GPU - Batch Size: 512 - Model: GoogLeNet images/sec > Higher Is Better ASUS NVIDIA GeForce RTX 3090 . 27.08 |========================================= TensorFlow 2.16.1 Device: GPU - Batch Size: 512 - Model: ResNet-50 images/sec > Higher Is Better ASUS NVIDIA GeForce RTX 3090 . 8.04 |========================================== TensorFlow Lite 2022-05-18 Model: SqueezeNet Microseconds < Lower Is Better ASUS NVIDIA GeForce RTX 3090 . 1904.01 |======================================= TensorFlow Lite 2022-05-18 Model: Inception V4 Microseconds < Lower Is Better ASUS NVIDIA GeForce RTX 3090 . 26961.0 |======================================= TensorFlow Lite 2022-05-18 Model: NASNet Mobile Microseconds < Lower Is Better ASUS NVIDIA GeForce RTX 3090 . 311542 |======================================== TensorFlow Lite 2022-05-18 Model: Mobilenet Float Microseconds < Lower Is Better ASUS NVIDIA GeForce RTX 3090 . 1383.02 |======================================= TensorFlow Lite 2022-05-18 Model: Mobilenet Quant Microseconds < Lower Is Better ASUS NVIDIA GeForce RTX 3090 . 2317.67 |======================================= TensorFlow Lite 2022-05-18 Model: Inception ResNet V2 Microseconds < Lower Is Better ASUS NVIDIA GeForce RTX 3090 . 128179 |======================================== TNN 0.3 Target: CPU - Model: DenseNet ms < Lower Is Better TNN 0.3 Target: CPU - Model: MobileNet v2 ms < Lower Is Better TNN 0.3 Target: CPU - Model: SqueezeNet v2 ms < Lower Is Better TNN 0.3 Target: CPU - Model: SqueezeNet v1.1 ms < Lower Is Better Whisper.cpp 1.6.2 Model: ggml-base.en - Input: 2016 State of the Union Seconds < Lower Is Better ASUS NVIDIA GeForce RTX 3090 . 209.02 |======================================== Whisper.cpp 1.6.2 Model: ggml-small.en - Input: 2016 State of the Union Seconds < Lower Is Better ASUS NVIDIA GeForce RTX 3090 . 638.01 |======================================== Whisper.cpp 1.6.2 Model: ggml-medium.en - Input: 2016 State of the Union Seconds < Lower Is Better ASUS NVIDIA GeForce RTX 3090 . 2000.10 |======================================= XNNPACK b7b048 Model: FP32MobileNetV1 us < Lower Is Better ASUS NVIDIA GeForce RTX 3090 . 1394 |========================================== XNNPACK b7b048 Model: FP32MobileNetV2 us < Lower Is Better ASUS NVIDIA GeForce RTX 3090 . 1125 |========================================== XNNPACK b7b048 Model: FP32MobileNetV3Large us < Lower Is Better ASUS NVIDIA GeForce RTX 3090 . 1585 |========================================== XNNPACK b7b048 Model: FP32MobileNetV3Small us < Lower Is Better ASUS NVIDIA GeForce RTX 3090 . 820 |=========================================== XNNPACK b7b048 Model: FP16MobileNetV1 us < Lower Is Better ASUS NVIDIA GeForce RTX 3090 . 2197 |========================================== XNNPACK b7b048 Model: FP16MobileNetV2 us < Lower Is Better ASUS NVIDIA GeForce RTX 3090 . 1646 |========================================== XNNPACK b7b048 Model: FP16MobileNetV3Large us < Lower Is Better ASUS NVIDIA GeForce RTX 3090 . 1834 |========================================== XNNPACK b7b048 Model: FP16MobileNetV3Small us < Lower Is Better ASUS NVIDIA GeForce RTX 3090 . 928 |=========================================== XNNPACK b7b048 Model: QS8MobileNetV2 us < Lower Is Better ASUS NVIDIA GeForce RTX 3090 . 904 |===========================================