AMD Ryzen Threadripper 7960X 24-Cores testing with a Gigabyte TRX50 AERO D (FA BIOS) and Sapphire AMD Radeon RX 7900 XTX 24GB on Ubuntu 24.04 via the Phoronix Test Suite.
OpenBenchmarking.org batches/sec, More Is Better PyTorch 2.2.1 Device: CPU - Batch Size: 1 - Model: ResNet-152 phoronix-ml.txt 6 12 18 24 30 SE +/- 0.14, N = 3 23.19 MIN: 19.02 / MAX: 24.35
OpenBenchmarking.org batches/sec, More Is Better PyTorch 2.2.1 Device: CPU - Batch Size: 16 - Model: ResNet-50 phoronix-ml.txt 10 20 30 40 50 SE +/- 0.20, N = 3 45.59 MIN: 38.82 / MAX: 46.87
OpenBenchmarking.org batches/sec, More Is Better PyTorch 2.2.1 Device: CPU - Batch Size: 32 - Model: ResNet-50 phoronix-ml.txt 11 22 33 44 55 SE +/- 0.47, N = 3 46.42 MIN: 41.66 / MAX: 47.49
OpenBenchmarking.org batches/sec, More Is Better PyTorch 2.2.1 Device: CPU - Batch Size: 64 - Model: ResNet-50 phoronix-ml.txt 11 22 33 44 55 SE +/- 0.10, N = 3 46.69 MIN: 42.52 / MAX: 47.36
OpenBenchmarking.org batches/sec, More Is Better PyTorch 2.2.1 Device: CPU - Batch Size: 16 - Model: ResNet-152 phoronix-ml.txt 4 8 12 16 20 SE +/- 0.07, N = 3 17.97 MIN: 14.56 / MAX: 18.28
OpenBenchmarking.org batches/sec, More Is Better PyTorch 2.2.1 Device: CPU - Batch Size: 256 - Model: ResNet-50 phoronix-ml.txt 10 20 30 40 50 SE +/- 0.23, N = 3 46.06 MIN: 38.74 / MAX: 46.92
OpenBenchmarking.org batches/sec, More Is Better PyTorch 2.2.1 Device: CPU - Batch Size: 32 - Model: ResNet-152 phoronix-ml.txt 4 8 12 16 20 SE +/- 0.03, N = 3 17.78 MIN: 15.02 / MAX: 18.13
OpenBenchmarking.org batches/sec, More Is Better PyTorch 2.2.1 Device: CPU - Batch Size: 512 - Model: ResNet-50 phoronix-ml.txt 10 20 30 40 50 SE +/- 0.29, N = 3 45.75 MIN: 41.27 / MAX: 46.82
OpenBenchmarking.org batches/sec, More Is Better PyTorch 2.2.1 Device: CPU - Batch Size: 64 - Model: ResNet-152 phoronix-ml.txt 4 8 12 16 20 SE +/- 0.08, N = 3 17.64 MIN: 14.41 / MAX: 18.1
OpenBenchmarking.org batches/sec, More Is Better PyTorch 2.2.1 Device: CPU - Batch Size: 256 - Model: ResNet-152 phoronix-ml.txt 4 8 12 16 20 SE +/- 0.06, N = 3 17.92 MIN: 14.68 / MAX: 18.36
OpenBenchmarking.org batches/sec, More Is Better PyTorch 2.2.1 Device: CPU - Batch Size: 512 - Model: ResNet-152 phoronix-ml.txt 4 8 12 16 20 SE +/- 0.07, N = 3 17.99 MIN: 14.67 / MAX: 18.38
OpenBenchmarking.org batches/sec, More Is Better PyTorch 2.2.1 Device: CPU - Batch Size: 1 - Model: Efficientnet_v2_l phoronix-ml.txt 4 8 12 16 20 SE +/- 0.19, N = 3 14.18 MIN: 12.11 / MAX: 14.81
OpenBenchmarking.org batches/sec, More Is Better PyTorch 2.2.1 Device: CPU - Batch Size: 16 - Model: Efficientnet_v2_l phoronix-ml.txt 3 6 9 12 15 SE +/- 0.09, N = 3 9.90 MIN: 8.07 / MAX: 10.35
OpenBenchmarking.org batches/sec, More Is Better PyTorch 2.2.1 Device: CPU - Batch Size: 32 - Model: Efficientnet_v2_l phoronix-ml.txt 3 6 9 12 15 SE +/- 0.07, N = 3 9.85 MIN: 8.27 / MAX: 10.3
OpenBenchmarking.org batches/sec, More Is Better PyTorch 2.2.1 Device: CPU - Batch Size: 64 - Model: Efficientnet_v2_l phoronix-ml.txt 3 6 9 12 15 SE +/- 0.05, N = 3 9.98 MIN: 7.98 / MAX: 10.29
OpenBenchmarking.org batches/sec, More Is Better PyTorch 2.2.1 Device: CPU - Batch Size: 256 - Model: Efficientnet_v2_l phoronix-ml.txt 3 6 9 12 15 SE +/- 0.05, N = 3 10.11 MIN: 8.31 / MAX: 10.38
OpenBenchmarking.org batches/sec, More Is Better PyTorch 2.2.1 Device: CPU - Batch Size: 512 - Model: Efficientnet_v2_l phoronix-ml.txt 3 6 9 12 15 SE +/- 0.09, N = 12 9.94 MIN: 7.81 / MAX: 10.45
PlaidML This test profile uses PlaidML deep learning framework developed by Intel for offering up various benchmarks. Learn more via the OpenBenchmarking.org test page.
FP16: No - Mode: Inference - Network: VGG16 - Device: CPU
phoronix-ml.txt: The test quit with a non-zero exit status. E: ModuleNotFoundError: No module named 'tensorflow'
FP16: No - Mode: Inference - Network: ResNet 50 - Device: CPU
phoronix-ml.txt: The test quit with a non-zero exit status. E: ModuleNotFoundError: No module named 'tensorflow'
OpenVINO This is a test of the Intel OpenVINO, a toolkit around neural networks, using its built-in benchmarking support and analyzing the throughput and latency for various models. Learn more via the OpenBenchmarking.org test page.
OpenBenchmarking.org FPS, More Is Better OpenVINO 2024.0 Model: Face Detection FP16 - Device: CPU phoronix-ml.txt 5 10 15 20 25 SE +/- 0.09, N = 3 19.69 1. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl
OpenBenchmarking.org FPS, More Is Better OpenVINO 2024.0 Model: Person Detection FP16 - Device: CPU phoronix-ml.txt 30 60 90 120 150 SE +/- 3.20, N = 15 129.45 1. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl
OpenBenchmarking.org FPS, More Is Better OpenVINO 2024.0 Model: Person Detection FP32 - Device: CPU phoronix-ml.txt 30 60 90 120 150 SE +/- 1.07, N = 15 119.41 1. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl
OpenBenchmarking.org FPS, More Is Better OpenVINO 2024.0 Model: Vehicle Detection FP16 - Device: CPU phoronix-ml.txt 200 400 600 800 1000 SE +/- 15.55, N = 15 1077.87 1. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl
OpenBenchmarking.org FPS, More Is Better OpenVINO 2024.0 Model: Face Detection FP16-INT8 - Device: CPU phoronix-ml.txt 9 18 27 36 45 SE +/- 0.02, N = 3 37.36 1. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl
OpenBenchmarking.org FPS, More Is Better OpenVINO 2024.0 Model: Face Detection Retail FP16 - Device: CPU phoronix-ml.txt 900 1800 2700 3600 4500 SE +/- 13.60, N = 3 4273.09 1. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl
OpenBenchmarking.org FPS, More Is Better OpenVINO 2024.0 Model: Road Segmentation ADAS FP16 - Device: CPU phoronix-ml.txt 100 200 300 400 500 SE +/- 9.05, N = 15 465.86 1. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl
OpenBenchmarking.org FPS, More Is Better OpenVINO 2024.0 Model: Vehicle Detection FP16-INT8 - Device: CPU phoronix-ml.txt 500 1000 1500 2000 2500 SE +/- 3.34, N = 3 2160.57 1. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl
OpenBenchmarking.org FPS, More Is Better OpenVINO 2024.0 Model: Weld Porosity Detection FP16 - Device: CPU phoronix-ml.txt 400 800 1200 1600 2000 SE +/- 1.95, N = 3 1947.67 1. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl
OpenBenchmarking.org FPS, More Is Better OpenVINO 2024.0 Model: Face Detection Retail FP16-INT8 - Device: CPU phoronix-ml.txt 1400 2800 4200 5600 7000 SE +/- 8.36, N = 3 6458.38 1. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl
OpenBenchmarking.org FPS, More Is Better OpenVINO 2024.0 Model: Road Segmentation ADAS FP16-INT8 - Device: CPU phoronix-ml.txt 150 300 450 600 750 SE +/- 2.47, N = 3 679.66 1. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl
OpenBenchmarking.org FPS, More Is Better OpenVINO 2024.0 Model: Machine Translation EN To DE FP16 - Device: CPU phoronix-ml.txt 40 80 120 160 200 SE +/- 3.37, N = 12 187.88 1. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl
OpenBenchmarking.org FPS, More Is Better OpenVINO 2024.0 Model: Weld Porosity Detection FP16-INT8 - Device: CPU phoronix-ml.txt 800 1600 2400 3200 4000 SE +/- 3.08, N = 3 3742.77 1. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl
OpenBenchmarking.org FPS, More Is Better OpenVINO 2024.0 Model: Person Vehicle Bike Detection FP16 - Device: CPU phoronix-ml.txt 400 800 1200 1600 2000 SE +/- 3.67, N = 3 1930.08 1. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl
OpenBenchmarking.org FPS, More Is Better OpenVINO 2024.0 Model: Noise Suppression Poconet-Like FP16 - Device: CPU phoronix-ml.txt 400 800 1200 1600 2000 SE +/- 35.59, N = 15 2052.02 1. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl
OpenBenchmarking.org FPS, More Is Better OpenVINO 2024.0 Model: Handwritten English Recognition FP16 - Device: CPU phoronix-ml.txt 200 400 600 800 1000 SE +/- 4.12, N = 3 1035.29 1. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl
OpenBenchmarking.org FPS, More Is Better OpenVINO 2024.0 Model: Person Re-Identification Retail FP16 - Device: CPU phoronix-ml.txt 500 1000 1500 2000 2500 SE +/- 3.45, N = 3 2523.59 1. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl
OpenBenchmarking.org FPS, More Is Better OpenVINO 2024.0 Model: Age Gender Recognition Retail 0013 FP16 - Device: CPU phoronix-ml.txt 10K 20K 30K 40K 50K SE +/- 17.34, N = 3 48433.92 1. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl
OpenBenchmarking.org FPS, More Is Better OpenVINO 2024.0 Model: Handwritten English Recognition FP16-INT8 - Device: CPU phoronix-ml.txt 200 400 600 800 1000 SE +/- 4.28, N = 3 1108.65 1. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl
OpenBenchmarking.org FPS, More Is Better OpenVINO 2024.0 Model: Age Gender Recognition Retail 0013 FP16-INT8 - Device: CPU phoronix-ml.txt 14K 28K 42K 56K 70K SE +/- 38.55, N = 3 67537.77 1. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl
TensorFlow This is a benchmark of the TensorFlow deep learning framework using the TensorFlow reference benchmarks (tensorflow/benchmarks with tf_cnn_benchmarks.py). Note with the Phoronix Test Suite there is also pts/tensorflow-lite for benchmarking the TensorFlow Lite binaries if desired for complementary metrics. Learn more via the OpenBenchmarking.org test page.
OpenBenchmarking.org images/sec, More Is Better TensorFlow 2.16.1 Device: CPU - Batch Size: 1 - Model: VGG-16 phoronix-ml.txt 3 6 9 12 15 SE +/- 0.00, N = 3 9.70
ONNX Runtime Model: GPT-2 - Device: CPU - Executor: Parallel
phoronix-ml.txt: The test quit with a non-zero exit status. E: onnxruntime/onnxruntime/test/onnx/onnx_model_info.cc:45 void OnnxModelInfo::InitOnnxModelInfo(const std::filesystem::__cxx11::path&) open file "GPT2/model.onnx" failed: No such file or directory
Model: GPT-2 - Device: CPU - Executor: Standard
phoronix-ml.txt: The test quit with a non-zero exit status. E: onnxruntime/onnxruntime/test/onnx/onnx_model_info.cc:45 void OnnxModelInfo::InitOnnxModelInfo(const std::filesystem::__cxx11::path&) open file "GPT2/model.onnx" failed: No such file or directory
Model: bertsquad-12 - Device: CPU - Executor: Parallel
phoronix-ml.txt: The test quit with a non-zero exit status. E: onnxruntime/onnxruntime/test/onnx/onnx_model_info.cc:45 void OnnxModelInfo::InitOnnxModelInfo(const std::filesystem::__cxx11::path&) open file "bertsquad-12/bertsquad-12.onnx" failed: No such file or directory
Model: bertsquad-12 - Device: CPU - Executor: Standard
phoronix-ml.txt: The test quit with a non-zero exit status. E: onnxruntime/onnxruntime/test/onnx/onnx_model_info.cc:45 void OnnxModelInfo::InitOnnxModelInfo(const std::filesystem::__cxx11::path&) open file "bertsquad-12/bertsquad-12.onnx" failed: No such file or directory
Model: ArcFace ResNet-100 - Device: CPU - Executor: Parallel
phoronix-ml.txt: The test quit with a non-zero exit status. E: onnxruntime/onnxruntime/test/onnx/onnx_model_info.cc:45 void OnnxModelInfo::InitOnnxModelInfo(const std::filesystem::__cxx11::path&) open file "resnet100/resnet100.onnx" failed: No such file or directory
Model: ArcFace ResNet-100 - Device: CPU - Executor: Standard
phoronix-ml.txt: The test quit with a non-zero exit status. E: onnxruntime/onnxruntime/test/onnx/onnx_model_info.cc:45 void OnnxModelInfo::InitOnnxModelInfo(const std::filesystem::__cxx11::path&) open file "resnet100/resnet100.onnx" failed: No such file or directory
Model: Faster R-CNN R-50-FPN-int8 - Device: CPU - Executor: Parallel
phoronix-ml.txt: The test quit with a non-zero exit status. E: onnxruntime/onnxruntime/test/onnx/onnx_model_info.cc:45 void OnnxModelInfo::InitOnnxModelInfo(const std::filesystem::__cxx11::path&) open file "FasterRCNN-12-int8/FasterRCNN-12-int8.onnx" failed: No such file or directory
Model: Faster R-CNN R-50-FPN-int8 - Device: CPU - Executor: Standard
phoronix-ml.txt: The test quit with a non-zero exit status. E: onnxruntime/onnxruntime/test/onnx/onnx_model_info.cc:45 void OnnxModelInfo::InitOnnxModelInfo(const std::filesystem::__cxx11::path&) open file "FasterRCNN-12-int8/FasterRCNN-12-int8.onnx" failed: No such file or directory
Neural Magic DeepSparse This is a benchmark of Neural Magic's DeepSparse using its built-in deepsparse.benchmark utility and various models from their SparseZoo (https://sparsezoo.neuralmagic.com/). Learn more via the OpenBenchmarking.org test page.
Model: NLP Document Classification, oBERT base uncased on IMDB - Scenario: Asynchronous Multi-Stream
phoronix-ml.txt: The test quit with a non-zero exit status. E: ./deepsparse: 2: /.local/bin/deepsparse.benchmark: not found
Model: NLP Document Classification, oBERT base uncased on IMDB - Scenario: Synchronous Single-Stream
phoronix-ml.txt: The test quit with a non-zero exit status. E: ./deepsparse: 2: /.local/bin/deepsparse.benchmark: not found
Model: NLP Text Classification, BERT base uncased SST2, Sparse INT8 - Scenario: Asynchronous Multi-Stream
phoronix-ml.txt: The test quit with a non-zero exit status. E: ./deepsparse: 2: /.local/bin/deepsparse.benchmark: not found
Model: NLP Text Classification, BERT base uncased SST2, Sparse INT8 - Scenario: Synchronous Single-Stream
phoronix-ml.txt: The test quit with a non-zero exit status. E: ./deepsparse: 2: /.local/bin/deepsparse.benchmark: not found
Model: ResNet-50, Baseline - Scenario: Asynchronous Multi-Stream
phoronix-ml.txt: The test quit with a non-zero exit status. E: ./deepsparse: 2: /.local/bin/deepsparse.benchmark: not found
Model: ResNet-50, Baseline - Scenario: Synchronous Single-Stream
phoronix-ml.txt: The test quit with a non-zero exit status. E: ./deepsparse: 2: /.local/bin/deepsparse.benchmark: not found
Model: ResNet-50, Sparse INT8 - Scenario: Asynchronous Multi-Stream
phoronix-ml.txt: The test quit with a non-zero exit status. E: ./deepsparse: 2: /.local/bin/deepsparse.benchmark: not found
Model: ResNet-50, Sparse INT8 - Scenario: Synchronous Single-Stream
phoronix-ml.txt: The test quit with a non-zero exit status. E: ./deepsparse: 2: /.local/bin/deepsparse.benchmark: not found
Model: Llama2 Chat 7b Quantized - Scenario: Asynchronous Multi-Stream
phoronix-ml.txt: The test quit with a non-zero exit status. E: ./deepsparse: 2: /.local/bin/deepsparse.benchmark: not found
Model: Llama2 Chat 7b Quantized - Scenario: Synchronous Single-Stream
phoronix-ml.txt: The test quit with a non-zero exit status. E: ./deepsparse: 2: /.local/bin/deepsparse.benchmark: not found
Model: CV Classification, ResNet-50 ImageNet - Scenario: Asynchronous Multi-Stream
phoronix-ml.txt: The test quit with a non-zero exit status. E: ./deepsparse: 2: /.local/bin/deepsparse.benchmark: not found
Model: CV Classification, ResNet-50 ImageNet - Scenario: Synchronous Single-Stream
phoronix-ml.txt: The test quit with a non-zero exit status. E: ./deepsparse: 2: /.local/bin/deepsparse.benchmark: not found
Model: CV Detection, YOLOv5s COCO, Sparse INT8 - Scenario: Asynchronous Multi-Stream
phoronix-ml.txt: The test quit with a non-zero exit status. E: ./deepsparse: 2: /.local/bin/deepsparse.benchmark: not found
Model: CV Detection, YOLOv5s COCO, Sparse INT8 - Scenario: Synchronous Single-Stream
phoronix-ml.txt: The test quit with a non-zero exit status. E: ./deepsparse: 2: /.local/bin/deepsparse.benchmark: not found
Model: NLP Text Classification, DistilBERT mnli - Scenario: Asynchronous Multi-Stream
phoronix-ml.txt: The test quit with a non-zero exit status. E: ./deepsparse: 2: /.local/bin/deepsparse.benchmark: not found
Model: NLP Text Classification, DistilBERT mnli - Scenario: Synchronous Single-Stream
phoronix-ml.txt: The test quit with a non-zero exit status. E: ./deepsparse: 2: /.local/bin/deepsparse.benchmark: not found
Model: CV Segmentation, 90% Pruned YOLACT Pruned - Scenario: Asynchronous Multi-Stream
phoronix-ml.txt: The test quit with a non-zero exit status. E: ./deepsparse: 2: /.local/bin/deepsparse.benchmark: not found
Model: CV Segmentation, 90% Pruned YOLACT Pruned - Scenario: Synchronous Single-Stream
phoronix-ml.txt: The test quit with a non-zero exit status. E: ./deepsparse: 2: /.local/bin/deepsparse.benchmark: not found
Model: BERT-Large, NLP Question Answering, Sparse INT8 - Scenario: Asynchronous Multi-Stream
phoronix-ml.txt: The test quit with a non-zero exit status. E: ./deepsparse: 2: /.local/bin/deepsparse.benchmark: not found
Model: BERT-Large, NLP Question Answering, Sparse INT8 - Scenario: Synchronous Single-Stream
phoronix-ml.txt: The test quit with a non-zero exit status. E: ./deepsparse: 2: /.local/bin/deepsparse.benchmark: not found
Model: NLP Token Classification, BERT base uncased conll2003 - Scenario: Asynchronous Multi-Stream
phoronix-ml.txt: The test quit with a non-zero exit status. E: ./deepsparse: 2: /.local/bin/deepsparse.benchmark: not found
Model: NLP Token Classification, BERT base uncased conll2003 - Scenario: Synchronous Single-Stream
phoronix-ml.txt: The test quit with a non-zero exit status. E: ./deepsparse: 2: /.local/bin/deepsparse.benchmark: not found
AI Benchmark Alpha AI Benchmark Alpha is a Python library for evaluating artificial intelligence (AI) performance on diverse hardware platforms and relies upon the TensorFlow machine learning library. Learn more via the OpenBenchmarking.org test page.
phoronix-ml.txt: The test quit with a non-zero exit status. E: ModuleNotFoundError: No module named 'tensorflow'
Llama.cpp Model: llama-2-7b.Q4_0.gguf
phoronix-ml.txt: The test quit with a non-zero exit status. E: main: error: unable to load model
Model: llama-2-13b.Q4_0.gguf
phoronix-ml.txt: The test quit with a non-zero exit status. E: main: error: unable to load model
Model: llama-2-70b-chat.Q5_0.gguf
phoronix-ml.txt: The test quit with a non-zero exit status. E: main: error: unable to load model
Llamafile Test: llava-v1.5-7b-q4 - Acceleration: CPU
phoronix-ml.txt: The test quit with a non-zero exit status. E: ./run-llava: line 2: ./llava-v1.6-mistral-7b.Q8_0.llamafile.86: No such file or directory
Test: mistral-7b-instruct-v0.2.Q8_0 - Acceleration: CPU
phoronix-ml.txt: The test quit with a non-zero exit status. E: ./run-mistral: line 2: ./mistral-7b-instruct-v0.2.Q5_K_M.llamafile.86: No such file or directory
Test: wizardcoder-python-34b-v1.0.Q6_K - Acceleration: CPU
phoronix-ml.txt: The test quit with a non-zero exit status. E: ./run-wizardcoder: line 2: ./wizardcoder-python-34b-v1.0.Q6_K.llamafile.86: No such file or directory
spaCy The spaCy library is an open-source solution for advanced neural language processing (NLP). The spaCy library leverages Python and is a leading neural language processing solution. This test profile times the spaCy CPU performance with various models. Learn more via the OpenBenchmarking.org test page.
phoronix-ml.txt: The test quit with a non-zero exit status. E: ModuleNotFoundError: No module named 'tqdm'
Caffe This is a benchmark of the Caffe deep learning framework and currently supports the AlexNet and Googlenet model and execution on both CPUs and NVIDIA GPUs. Learn more via the OpenBenchmarking.org test page.
Model: AlexNet - Acceleration: CPU - Iterations: 100
phoronix-ml.txt: The test quit with a non-zero exit status. E: ./caffe: 3: ./tools/caffe: not found
Model: AlexNet - Acceleration: CPU - Iterations: 200
phoronix-ml.txt: The test quit with a non-zero exit status. E: ./caffe: 3: ./tools/caffe: not found
Model: AlexNet - Acceleration: CPU - Iterations: 1000
phoronix-ml.txt: The test quit with a non-zero exit status. E: ./caffe: 3: ./tools/caffe: not found
Model: GoogleNet - Acceleration: CPU - Iterations: 100
phoronix-ml.txt: The test quit with a non-zero exit status. E: ./caffe: 3: ./tools/caffe: not found
Model: GoogleNet - Acceleration: CPU - Iterations: 200
phoronix-ml.txt: The test quit with a non-zero exit status. E: ./caffe: 3: ./tools/caffe: not found
Model: GoogleNet - Acceleration: CPU - Iterations: 1000
phoronix-ml.txt: The test quit with a non-zero exit status. E: ./caffe: 3: ./tools/caffe: not found
oneDNN OpenBenchmarking.org ms, Fewer Is Better oneDNN 3.6 Harness: IP Shapes 1D - Engine: CPU phoronix-ml.txt 0.2557 0.5114 0.7671 1.0228 1.2785 SE +/- 0.00979, N = 15 1.13657 MIN: 1.01 1. (CXX) g++ options: -O3 -march=native -fopenmp -msse4.1 -fPIC -fcf-protection=full -pie -ldl
OpenBenchmarking.org ms, Fewer Is Better oneDNN 3.6 Harness: IP Shapes 3D - Engine: CPU phoronix-ml.txt 0.3141 0.6282 0.9423 1.2564 1.5705 SE +/- 0.01618, N = 15 1.39591 MIN: 1.16 1. (CXX) g++ options: -O3 -march=native -fopenmp -msse4.1 -fPIC -fcf-protection=full -pie -ldl
OpenBenchmarking.org ms, Fewer Is Better oneDNN 3.6 Harness: Convolution Batch Shapes Auto - Engine: CPU phoronix-ml.txt 0.5317 1.0634 1.5951 2.1268 2.6585 SE +/- 0.02737, N = 4 2.36317 MIN: 1.97 1. (CXX) g++ options: -O3 -march=native -fopenmp -msse4.1 -fPIC -fcf-protection=full -pie -ldl
OpenBenchmarking.org ms, Fewer Is Better oneDNN 3.6 Harness: Deconvolution Batch shapes_1d - Engine: CPU phoronix-ml.txt 0.8495 1.699 2.5485 3.398 4.2475 SE +/- 0.00801, N = 3 3.77567 MIN: 2.81 1. (CXX) g++ options: -O3 -march=native -fopenmp -msse4.1 -fPIC -fcf-protection=full -pie -ldl
OpenBenchmarking.org ms, Fewer Is Better oneDNN 3.6 Harness: Deconvolution Batch shapes_3d - Engine: CPU phoronix-ml.txt 0.4167 0.8334 1.2501 1.6668 2.0835 SE +/- 0.01905, N = 4 1.85206 MIN: 1.73 1. (CXX) g++ options: -O3 -march=native -fopenmp -msse4.1 -fPIC -fcf-protection=full -pie -ldl
OpenBenchmarking.org ms, Fewer Is Better oneDNN 3.6 Harness: Recurrent Neural Network Training - Engine: CPU phoronix-ml.txt 300 600 900 1200 1500 SE +/- 9.63, N = 3 1261.40 MIN: 1196.78 1. (CXX) g++ options: -O3 -march=native -fopenmp -msse4.1 -fPIC -fcf-protection=full -pie -ldl
OpenBenchmarking.org ms, Fewer Is Better oneDNN 3.6 Harness: Recurrent Neural Network Inference - Engine: CPU phoronix-ml.txt 160 320 480 640 800 SE +/- 8.66, N = 3 736.40 MIN: 639.28 1. (CXX) g++ options: -O3 -march=native -fopenmp -msse4.1 -fPIC -fcf-protection=full -pie -ldl
Mobile Neural Network OpenBenchmarking.org ms, Fewer Is Better Mobile Neural Network 2.9.b11b7037d Model: nasnet phoronix-ml.txt 4 8 12 16 20 SE +/- 0.02, N = 3 15.30 MIN: 14.65 / MAX: 21.59 1. (CXX) g++ options: -std=c++11 -O3 -fvisibility=hidden -fomit-frame-pointer -fstrict-aliasing -ffunction-sections -fdata-sections -ffast-math -fno-rtti -fno-exceptions -pthread -ldl
OpenBenchmarking.org ms, Fewer Is Better Mobile Neural Network 2.9.b11b7037d Model: mobilenetV3 phoronix-ml.txt 0.5706 1.1412 1.7118 2.2824 2.853 SE +/- 0.008, N = 3 2.536 MIN: 2.4 / MAX: 3.47 1. (CXX) g++ options: -std=c++11 -O3 -fvisibility=hidden -fomit-frame-pointer -fstrict-aliasing -ffunction-sections -fdata-sections -ffast-math -fno-rtti -fno-exceptions -pthread -ldl
OpenBenchmarking.org ms, Fewer Is Better Mobile Neural Network 2.9.b11b7037d Model: squeezenetv1.1 phoronix-ml.txt 0.9736 1.9472 2.9208 3.8944 4.868 SE +/- 0.117, N = 3 4.327 MIN: 3.96 / MAX: 6.65 1. (CXX) g++ options: -std=c++11 -O3 -fvisibility=hidden -fomit-frame-pointer -fstrict-aliasing -ffunction-sections -fdata-sections -ffast-math -fno-rtti -fno-exceptions -pthread -ldl
OpenBenchmarking.org ms, Fewer Is Better Mobile Neural Network 2.9.b11b7037d Model: resnet-v2-50 phoronix-ml.txt 5 10 15 20 25 SE +/- 0.11, N = 3 18.53 MIN: 18.26 / MAX: 28.9 1. (CXX) g++ options: -std=c++11 -O3 -fvisibility=hidden -fomit-frame-pointer -fstrict-aliasing -ffunction-sections -fdata-sections -ffast-math -fno-rtti -fno-exceptions -pthread -ldl
OpenBenchmarking.org ms, Fewer Is Better Mobile Neural Network 2.9.b11b7037d Model: SqueezeNetV1.0 phoronix-ml.txt 2 4 6 8 10 SE +/- 0.200, N = 3 6.429 MIN: 5.97 / MAX: 7.14 1. (CXX) g++ options: -std=c++11 -O3 -fvisibility=hidden -fomit-frame-pointer -fstrict-aliasing -ffunction-sections -fdata-sections -ffast-math -fno-rtti -fno-exceptions -pthread -ldl
OpenBenchmarking.org ms, Fewer Is Better Mobile Neural Network 2.9.b11b7037d Model: MobileNetV2_224 phoronix-ml.txt 0.7353 1.4706 2.2059 2.9412 3.6765 SE +/- 0.042, N = 3 3.268 MIN: 3.15 / MAX: 5.05 1. (CXX) g++ options: -std=c++11 -O3 -fvisibility=hidden -fomit-frame-pointer -fstrict-aliasing -ffunction-sections -fdata-sections -ffast-math -fno-rtti -fno-exceptions -pthread -ldl
OpenBenchmarking.org ms, Fewer Is Better Mobile Neural Network 2.9.b11b7037d Model: mobilenet-v1-1.0 phoronix-ml.txt 0.8514 1.7028 2.5542 3.4056 4.257 SE +/- 0.007, N = 3 3.784 MIN: 3.71 / MAX: 6.58 1. (CXX) g++ options: -std=c++11 -O3 -fvisibility=hidden -fomit-frame-pointer -fstrict-aliasing -ffunction-sections -fdata-sections -ffast-math -fno-rtti -fno-exceptions -pthread -ldl
OpenBenchmarking.org ms, Fewer Is Better Mobile Neural Network 2.9.b11b7037d Model: inception-v3 phoronix-ml.txt 8 16 24 32 40 SE +/- 0.04, N = 3 36.45 MIN: 36.16 / MAX: 50.97 1. (CXX) g++ options: -std=c++11 -O3 -fvisibility=hidden -fomit-frame-pointer -fstrict-aliasing -ffunction-sections -fdata-sections -ffast-math -fno-rtti -fno-exceptions -pthread -ldl
OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: CPU-v2-v2 - Model: mobilenet-v2 phoronix-ml.txt 2 4 6 8 10 SE +/- 0.05, N = 15 6.30 MIN: 5.63 / MAX: 33.15 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread
OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: CPU-v3-v3 - Model: mobilenet-v3 phoronix-ml.txt 2 4 6 8 10 SE +/- 0.04, N = 15 6.45 MIN: 5.96 / MAX: 63.77 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread
OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: CPU - Model: shufflenet-v2 phoronix-ml.txt 2 4 6 8 10 SE +/- 0.09, N = 15 8.15 MIN: 7.52 / MAX: 291.18 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread
OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: CPU - Model: mnasnet phoronix-ml.txt 2 4 6 8 10 SE +/- 0.11, N = 15 6.11 MIN: 5.26 / MAX: 321.29 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread
OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: CPU - Model: efficientnet-b0 phoronix-ml.txt 2 4 6 8 10 SE +/- 0.09, N = 15 8.28 MIN: 7.57 / MAX: 296.15 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread
OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: CPU - Model: blazeface phoronix-ml.txt 0.6998 1.3996 2.0994 2.7992 3.499 SE +/- 0.02, N = 15 3.11 MIN: 2.85 / MAX: 11.53 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread
OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: CPU - Model: googlenet phoronix-ml.txt 4 8 12 16 20 SE +/- 0.15, N = 15 16.42 MIN: 15.38 / MAX: 271.34 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread
OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: CPU - Model: vgg16 phoronix-ml.txt 6 12 18 24 30 SE +/- 0.34, N = 15 25.71 MIN: 22.56 / MAX: 344.2 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread
OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: CPU - Model: resnet18 phoronix-ml.txt 2 4 6 8 10 SE +/- 0.07, N = 15 8.02 MIN: 7.54 / MAX: 17.96 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread
OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: CPU - Model: alexnet phoronix-ml.txt 1.251 2.502 3.753 5.004 6.255 SE +/- 0.07, N = 15 5.56 MIN: 5.07 / MAX: 35.35 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread
OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: CPU - Model: resnet50 phoronix-ml.txt 3 6 9 12 15 SE +/- 0.13, N = 15 13.33 MIN: 11.94 / MAX: 281.3 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread
OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: CPUv2-yolov3v2-yolov3 - Model: mobilenetv2-yolov3 phoronix-ml.txt 4 8 12 16 20 SE +/- 0.12, N = 15 13.85 MIN: 12.83 / MAX: 247.13 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread
OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: CPU - Model: yolov4-tiny phoronix-ml.txt 6 12 18 24 30 SE +/- 0.11, N = 15 24.14 MIN: 21.58 / MAX: 105.12 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread
OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: CPU - Model: squeezenet_ssd phoronix-ml.txt 4 8 12 16 20 SE +/- 0.12, N = 15 14.34 MIN: 13.13 / MAX: 263.27 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread
OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: CPU - Model: regnety_400m phoronix-ml.txt 5 10 15 20 25 SE +/- 0.12, N = 15 18.58 MIN: 17.32 / MAX: 295.21 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread
OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: CPU - Model: vision_transformer phoronix-ml.txt 9 18 27 36 45 SE +/- 0.18, N = 15 40.59 MIN: 37.83 / MAX: 299.43 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread
OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: CPU - Model: FastestDet phoronix-ml.txt 3 6 9 12 15 SE +/- 0.30, N = 15 9.80 MIN: 6.73 / MAX: 273.49 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread
OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: Vulkan GPU - Model: mobilenet phoronix-ml.txt 4 8 12 16 20 SE +/- 0.10, N = 3 13.79 MIN: 13.15 / MAX: 23.26 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread
OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: Vulkan GPU-v2-v2 - Model: mobilenet-v2 phoronix-ml.txt 2 4 6 8 10 SE +/- 0.02, N = 3 6.31 MIN: 5.98 / MAX: 14.56 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread
OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: Vulkan GPU-v3-v3 - Model: mobilenet-v3 phoronix-ml.txt 2 4 6 8 10 SE +/- 0.03, N = 3 6.49 MIN: 6.21 / MAX: 15.75 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread
OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: Vulkan GPU - Model: shufflenet-v2 phoronix-ml.txt 2 4 6 8 10 SE +/- 0.05, N = 3 8.06 MIN: 7.83 / MAX: 15.08 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread
OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: Vulkan GPU - Model: mnasnet phoronix-ml.txt 1.3478 2.6956 4.0434 5.3912 6.739 SE +/- 0.01, N = 3 5.99 MIN: 5.68 / MAX: 14.25 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread
OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: Vulkan GPU - Model: efficientnet-b0 phoronix-ml.txt 2 4 6 8 10 SE +/- 0.37, N = 3 8.68 MIN: 7.76 / MAX: 202.36 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread
OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: Vulkan GPU - Model: blazeface phoronix-ml.txt 0.7065 1.413 2.1195 2.826 3.5325 SE +/- 0.00, N = 3 3.14 MIN: 3.01 / MAX: 8.5 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread
OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: Vulkan GPU - Model: googlenet phoronix-ml.txt 4 8 12 16 20 SE +/- 0.04, N = 3 16.01 MIN: 15.51 / MAX: 26.58 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread
OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: Vulkan GPU - Model: vgg16 phoronix-ml.txt 6 12 18 24 30 SE +/- 0.31, N = 3 25.13 MIN: 23.1 / MAX: 139.05 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread
OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: Vulkan GPU - Model: resnet18 phoronix-ml.txt 2 4 6 8 10 SE +/- 0.08, N = 3 7.86 MIN: 7.54 / MAX: 15.18 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread
OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: Vulkan GPU - Model: alexnet phoronix-ml.txt 1.188 2.376 3.564 4.752 5.94 SE +/- 0.02, N = 3 5.28 MIN: 5.1 / MAX: 15.43 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread
OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: Vulkan GPU - Model: resnet50 phoronix-ml.txt 4 8 12 16 20 SE +/- 1.16, N = 3 14.65 MIN: 12.41 / MAX: 377.47 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread
OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: Vulkan GPUv2-yolov3v2-yolov3 - Model: mobilenetv2-yolov3 phoronix-ml.txt 4 8 12 16 20 SE +/- 0.10, N = 3 13.79 MIN: 13.15 / MAX: 23.26 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread
OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: Vulkan GPU - Model: yolov4-tiny phoronix-ml.txt 6 12 18 24 30 SE +/- 0.41, N = 3 23.64 MIN: 22.17 / MAX: 39.94 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread
OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: Vulkan GPU - Model: squeezenet_ssd phoronix-ml.txt 4 8 12 16 20 SE +/- 1.72, N = 3 16.04 MIN: 13.26 / MAX: 580.79 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread
OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: Vulkan GPU - Model: regnety_400m phoronix-ml.txt 5 10 15 20 25 SE +/- 0.09, N = 3 18.63 MIN: 18.07 / MAX: 108.4 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread
OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: Vulkan GPU - Model: vision_transformer phoronix-ml.txt 9 18 27 36 45 SE +/- 0.27, N = 3 41.05 MIN: 38.99 / MAX: 101.63 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread
OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: Vulkan GPU - Model: FastestDet phoronix-ml.txt 3 6 9 12 15 SE +/- 0.39, N = 3 9.82 MIN: 8.74 / MAX: 19.33 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread
TNN TNN is an open-source deep learning reasoning framework developed by Tencent. Learn more via the OpenBenchmarking.org test page.
Target: CPU - Model: DenseNet
phoronix-ml.txt: The test quit with a non-zero exit status. E: ./tnn: 3: ./test/TNNTest: not found
Target: CPU - Model: MobileNet v2
phoronix-ml.txt: The test quit with a non-zero exit status. E: ./tnn: 3: ./test/TNNTest: not found
Target: CPU - Model: SqueezeNet v2
phoronix-ml.txt: The test quit with a non-zero exit status. E: ./tnn: 3: ./test/TNNTest: not found
Target: CPU - Model: SqueezeNet v1.1
phoronix-ml.txt: The test quit with a non-zero exit status. E: ./tnn: 3: ./test/TNNTest: not found
OpenVINO This is a test of the Intel OpenVINO, a toolkit around neural networks, using its built-in benchmarking support and analyzing the throughput and latency for various models. Learn more via the OpenBenchmarking.org test page.
OpenBenchmarking.org ms, Fewer Is Better OpenVINO 2024.0 Model: Face Detection FP16 - Device: CPU phoronix-ml.txt 130 260 390 520 650 SE +/- 2.89, N = 3 607.48 MIN: 575.29 / MAX: 657.06 1. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl
OpenBenchmarking.org ms, Fewer Is Better OpenVINO 2024.0 Model: Person Detection FP16 - Device: CPU phoronix-ml.txt 20 40 60 80 100 SE +/- 2.03, N = 15 93.29 MIN: 31.12 / MAX: 185.66 1. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl
OpenBenchmarking.org ms, Fewer Is Better OpenVINO 2024.0 Model: Person Detection FP32 - Device: CPU phoronix-ml.txt 20 40 60 80 100 SE +/- 0.82, N = 15 100.46 MIN: 32.5 / MAX: 161.81 1. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl
OpenBenchmarking.org ms, Fewer Is Better OpenVINO 2024.0 Model: Vehicle Detection FP16 - Device: CPU phoronix-ml.txt 3 6 9 12 15 SE +/- 0.14, N = 15 11.12 MIN: 4.52 / MAX: 34.07 1. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl
OpenBenchmarking.org ms, Fewer Is Better OpenVINO 2024.0 Model: Face Detection FP16-INT8 - Device: CPU phoronix-ml.txt 70 140 210 280 350 SE +/- 0.15, N = 3 320.42 MIN: 299.06 / MAX: 380.18 1. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl
OpenBenchmarking.org ms, Fewer Is Better OpenVINO 2024.0 Model: Face Detection Retail FP16 - Device: CPU phoronix-ml.txt 0.621 1.242 1.863 2.484 3.105 SE +/- 0.01, N = 3 2.76 MIN: 1.41 / MAX: 15.61 1. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl
OpenBenchmarking.org ms, Fewer Is Better OpenVINO 2024.0 Model: Road Segmentation ADAS FP16 - Device: CPU phoronix-ml.txt 6 12 18 24 30 SE +/- 0.43, N = 15 25.83 MIN: 10.2 / MAX: 57.07 1. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl
OpenBenchmarking.org ms, Fewer Is Better OpenVINO 2024.0 Model: Vehicle Detection FP16-INT8 - Device: CPU phoronix-ml.txt 1.2398 2.4796 3.7194 4.9592 6.199 SE +/- 0.01, N = 3 5.51 MIN: 2.97 / MAX: 19.15 1. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl
OpenBenchmarking.org ms, Fewer Is Better OpenVINO 2024.0 Model: Weld Porosity Detection FP16 - Device: CPU phoronix-ml.txt 3 6 9 12 15 SE +/- 0.01, N = 3 12.25 MIN: 6.32 / MAX: 26.6 1. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl
OpenBenchmarking.org ms, Fewer Is Better OpenVINO 2024.0 Model: Face Detection Retail FP16-INT8 - Device: CPU phoronix-ml.txt 0.8055 1.611 2.4165 3.222 4.0275 SE +/- 0.00, N = 3 3.58 MIN: 1.95 / MAX: 16.91 1. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl
OpenBenchmarking.org ms, Fewer Is Better OpenVINO 2024.0 Model: Road Segmentation ADAS FP16-INT8 - Device: CPU phoronix-ml.txt 4 8 12 16 20 SE +/- 0.06, N = 3 17.62 MIN: 9.01 / MAX: 33.69 1. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl
OpenBenchmarking.org ms, Fewer Is Better OpenVINO 2024.0 Model: Machine Translation EN To DE FP16 - Device: CPU phoronix-ml.txt 14 28 42 56 70 SE +/- 1.08, N = 12 63.99 MIN: 29.61 / MAX: 110.09 1. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl
OpenBenchmarking.org ms, Fewer Is Better OpenVINO 2024.0 Model: Weld Porosity Detection FP16-INT8 - Device: CPU phoronix-ml.txt 2 4 6 8 10 SE +/- 0.01, N = 3 6.31 MIN: 3.31 / MAX: 21.37 1. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl
OpenBenchmarking.org ms, Fewer Is Better OpenVINO 2024.0 Model: Person Vehicle Bike Detection FP16 - Device: CPU phoronix-ml.txt 2 4 6 8 10 SE +/- 0.01, N = 3 6.18 MIN: 3.57 / MAX: 20.46 1. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl
OpenBenchmarking.org ms, Fewer Is Better OpenVINO 2024.0 Model: Noise Suppression Poconet-Like FP16 - Device: CPU phoronix-ml.txt 3 6 9 12 15 SE +/- 0.18, N = 15 11.53 MIN: 5.76 / MAX: 42.76 1. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl
OpenBenchmarking.org ms, Fewer Is Better OpenVINO 2024.0 Model: Handwritten English Recognition FP16 - Device: CPU phoronix-ml.txt 6 12 18 24 30 SE +/- 0.09, N = 3 23.12 MIN: 14.9 / MAX: 38.92 1. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl
OpenBenchmarking.org ms, Fewer Is Better OpenVINO 2024.0 Model: Person Re-Identification Retail FP16 - Device: CPU phoronix-ml.txt 1.062 2.124 3.186 4.248 5.31 SE +/- 0.01, N = 3 4.72 MIN: 2.72 / MAX: 17.21 1. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl
OpenBenchmarking.org ms, Fewer Is Better OpenVINO 2024.0 Model: Age Gender Recognition Retail 0013 FP16 - Device: CPU phoronix-ml.txt 0.0968 0.1936 0.2904 0.3872 0.484 SE +/- 0.00, N = 3 0.43 MIN: 0.23 / MAX: 11.96 1. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl
OpenBenchmarking.org ms, Fewer Is Better OpenVINO 2024.0 Model: Handwritten English Recognition FP16-INT8 - Device: CPU phoronix-ml.txt 5 10 15 20 25 SE +/- 0.08, N = 3 21.58 MIN: 16.4 / MAX: 43.47 1. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl
OpenBenchmarking.org ms, Fewer Is Better OpenVINO 2024.0 Model: Age Gender Recognition Retail 0013 FP16-INT8 - Device: CPU phoronix-ml.txt 0.0675 0.135 0.2025 0.27 0.3375 SE +/- 0.00, N = 3 0.3 MIN: 0.17 / MAX: 9.44 1. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl
OpenCV This is a benchmark of the OpenCV (Computer Vision) library's built-in performance tests. Learn more via the OpenBenchmarking.org test page.
OpenBenchmarking.org ms, Fewer Is Better OpenCV 4.7 Test: DNN - Deep Neural Network phoronix-ml.txt 7K 14K 21K 28K 35K SE +/- 1066.17, N = 15 33080 1. (CXX) g++ options: -fsigned-char -pthread -fomit-frame-pointer -ffunction-sections -fdata-sections -msse -msse2 -msse3 -fvisibility=hidden -O3 -ldl -lm -lpthread -lrt
DeepSpeech Mozilla DeepSpeech is a speech-to-text engine powered by TensorFlow for machine learning and derived from Baidu's Deep Speech research paper. This test profile times the speech-to-text process for a roughly three minute audio recording. Learn more via the OpenBenchmarking.org test page.
OpenBenchmarking.org Seconds, Fewer Is Better DeepSpeech 0.6 Acceleration: CPU phoronix-ml.txt 10 20 30 40 50 SE +/- 0.15, N = 3 46.23
RNNoise RNNoise is a recurrent neural network for audio noise reduction developed by Mozilla and Xiph.Org. This test profile is a single-threaded test measuring the time to denoise a sample 26 minute long 16-bit RAW audio file using this recurrent neural network noise suppression library. Learn more via the OpenBenchmarking.org test page.
OpenBenchmarking.org Seconds, Fewer Is Better RNNoise 0.2 Input: 26 Minute Long Talking Sample phoronix-ml.txt 2 4 6 8 10 SE +/- 0.019, N = 3 7.852 1. (CC) gcc options: -O2 -pedantic -fvisibility=hidden
Numenta Anomaly Benchmark Numenta Anomaly Benchmark (NAB) is a benchmark for evaluating algorithms for anomaly detection in streaming, real-time applications. It is comprised of over 50 labeled real-world and artificial time-series data files plus a novel scoring mechanism designed for real-time applications. This test profile currently measures the time to run various detectors. Learn more via the OpenBenchmarking.org test page.
Detector: KNN CAD
phoronix-ml.txt: The test quit with a non-zero exit status. E: ModuleNotFoundError: No module named 'pandas'
Detector: Relative Entropy
phoronix-ml.txt: The test quit with a non-zero exit status. E: ModuleNotFoundError: No module named 'pandas'
Detector: Windowed Gaussian
phoronix-ml.txt: The test quit with a non-zero exit status. E: ModuleNotFoundError: No module named 'pandas'
Detector: Earthgecko Skyline
phoronix-ml.txt: The test quit with a non-zero exit status. E: ModuleNotFoundError: No module named 'pandas'
Detector: Bayesian Changepoint
phoronix-ml.txt: The test quit with a non-zero exit status. E: ModuleNotFoundError: No module named 'pandas'
Detector: Contextual Anomaly Detector OSE
phoronix-ml.txt: The test quit with a non-zero exit status. E: ModuleNotFoundError: No module named 'pandas'
Benchmark: scikit_qda
phoronix-ml.txt: The test quit with a non-zero exit status. E: ModuleNotFoundError: No module named 'imp'
Benchmark: scikit_svm
phoronix-ml.txt: The test quit with a non-zero exit status. E: ModuleNotFoundError: No module named 'imp'
Benchmark: scikit_linearridgeregression
phoronix-ml.txt: The test quit with a non-zero exit status. E: ModuleNotFoundError: No module named 'imp'
Benchmark: Glmnet
phoronix-ml.txt: The test quit with a non-zero exit status. E: ModuleNotFoundError: No module named 'glmnet'
Benchmark: Plot Lasso Path
phoronix-ml.txt: The test quit with a non-zero exit status. E: ModuleNotFoundError: No module named 'matplotlib.tri.triangulation'
Benchmark: Plot Fast KMeans
phoronix-ml.txt: The test quit with a non-zero exit status. E: ModuleNotFoundError: No module named 'matplotlib.tri.triangulation'
Benchmark: RCV1 Logreg Convergencet
phoronix-ml.txt: The test quit with a non-zero exit status. E: IndexError: list index out of range
OpenBenchmarking.org Seconds, Fewer Is Better Scikit-Learn 1.2.2 Benchmark: Isotonic / Perturbed Logarithm phoronix-ml.txt 300 600 900 1200 1500 SE +/- 2.36, N = 3 1528.97 1. (F9X) gfortran options: -O0
Benchmark: Plot Singular Value Decomposition
phoronix-ml.txt: The test quit with a non-zero exit status. E: ModuleNotFoundError: No module named 'matplotlib.tri.triangulation'
Benchmark: Plot Non-Negative Matrix Factorization
phoronix-ml.txt: The test quit with a non-zero exit status. E: KeyError:
OpenBenchmarking.org Seconds, Fewer Is Better Scikit-Learn 1.2.2 Benchmark: Hist Gradient Boosting Categorical Only phoronix-ml.txt 7 14 21 28 35 SE +/- 0.30, N = 15 30.19 1. (F9X) gfortran options: -O0
OpenBenchmarking.org Seconds, Fewer Is Better Scikit-Learn 1.2.2 Benchmark: Kernel PCA Solvers / Time vs. N Samples phoronix-ml.txt 14 28 42 56 70 SE +/- 0.24, N = 3 61.61 1. (F9X) gfortran options: -O0
OpenBenchmarking.org Seconds, Fewer Is Better Scikit-Learn 1.2.2 Benchmark: Kernel PCA Solvers / Time vs. N Components phoronix-ml.txt 7 14 21 28 35 SE +/- 0.33, N = 3 31.04 1. (F9X) gfortran options: -O0
OpenBenchmarking.org Seconds, Fewer Is Better Scikit-Learn 1.2.2 Benchmark: Sparse Random Projections / 100 Iterations phoronix-ml.txt 110 220 330 440 550 SE +/- 2.62, N = 3 504.83 1. (F9X) gfortran options: -O0
Whisper.cpp OpenBenchmarking.org Seconds, Fewer Is Better Whisper.cpp 1.6.2 Model: ggml-base.en - Input: 2016 State of the Union phoronix-ml.txt 20 40 60 80 100 SE +/- 0.44, N = 3 92.75 1. (CXX) g++ options: -O3 -std=c++11 -fPIC -pthread -msse3 -mssse3 -mavx -mf16c -mfma -mavx2 -mavx512f -mavx512cd -mavx512vl -mavx512dq -mavx512bw -mavx512vbmi -mavx512vnni
OpenBenchmarking.org Seconds, Fewer Is Better Whisper.cpp 1.6.2 Model: ggml-small.en - Input: 2016 State of the Union phoronix-ml.txt 50 100 150 200 250 SE +/- 0.41, N = 3 218.19 1. (CXX) g++ options: -O3 -std=c++11 -fPIC -pthread -msse3 -mssse3 -mavx -mf16c -mfma -mavx2 -mavx512f -mavx512cd -mavx512vl -mavx512dq -mavx512bw -mavx512vbmi -mavx512vnni
OpenBenchmarking.org Seconds, Fewer Is Better Whisper.cpp 1.6.2 Model: ggml-medium.en - Input: 2016 State of the Union phoronix-ml.txt 130 260 390 520 650 SE +/- 1.41, N = 3 579.17 1. (CXX) g++ options: -O3 -std=c++11 -fPIC -pthread -msse3 -mssse3 -mavx -mf16c -mfma -mavx2 -mavx512f -mavx512cd -mavx512vl -mavx512dq -mavx512bw -mavx512vbmi -mavx512vnni
XNNPACK OpenBenchmarking.org us, Fewer Is Better XNNPACK b7b048 Model: FP32MobileNetV1 phoronix-ml.txt 300 600 900 1200 1500 SE +/- 2.52, N = 3 1233 1. (CXX) g++ options: -O3 -lrt -lm
OpenBenchmarking.org us, Fewer Is Better XNNPACK b7b048 Model: FP32MobileNetV3Large phoronix-ml.txt 500 1000 1500 2000 2500 SE +/- 12.67, N = 3 2465 1. (CXX) g++ options: -O3 -lrt -lm
OpenBenchmarking.org us, Fewer Is Better XNNPACK b7b048 Model: FP16MobileNetV3Large phoronix-ml.txt 500 1000 1500 2000 2500 SE +/- 6.66, N = 3 2128 1. (CXX) g++ options: -O3 -lrt -lm
phoronix-ml.txt Processor: AMD Ryzen Threadripper 7960X 24-Cores @ 7.79GHz (24 Cores / 48 Threads), Motherboard: Gigabyte TRX50 AERO D (FA BIOS), Chipset: AMD Device 14a4, Memory: 4 x 32GB DDR5-5200MT/s Micron MTC20F1045S1RC56BG1, Disk: 1000GB GIGABYTE AG512K1TB, Graphics: Sapphire AMD Radeon RX 7900 XTX 24GB, Audio: AMD Device 14cc, Monitor: HP E273, Network: Aquantia AQC113C NBase-T/IEEE + Realtek RTL8125 2.5GbE + Qualcomm WCN785x Wi-Fi 7
OS: Ubuntu 24.04, Kernel: 6.8.0-48-generic (x86_64), Desktop: GNOME Shell 46.0, Display Server: X Server + Wayland, OpenGL: 4.6 Mesa 24.2.0-devel (LLVM 18.1.7 DRM 3.58), OpenCL: OpenCL 2.1 AMD-APP (3625.0), Compiler: GCC 13.2.0, File-System: ext4, Screen Resolution: 1920x1080
Kernel Notes: Transparent Huge Pages: madviseCompiler Notes: --build=x86_64-linux-gnu --disable-vtable-verify --disable-werror --enable-cet --enable-checking=release --enable-clocale=gnu --enable-default-pie --enable-gnu-unique-object --enable-languages=c,ada,c++,go,d,fortran,objc,obj-c++,m2 --enable-libphobos-checking=release --enable-libstdcxx-backtrace --enable-libstdcxx-debug --enable-libstdcxx-time=yes --enable-multiarch --enable-multilib --enable-nls --enable-objc-gc=auto --enable-offload-defaulted --enable-offload-targets=nvptx-none=/build/gcc-13-uJ7kn6/gcc-13-13.2.0/debian/tmp-nvptx/usr,amdgcn-amdhsa=/build/gcc-13-uJ7kn6/gcc-13-13.2.0/debian/tmp-gcn/usr --enable-plugin --enable-shared --enable-threads=posix --host=x86_64-linux-gnu --program-prefix=x86_64-linux-gnu- --target=x86_64-linux-gnu --with-abi=m64 --with-arch-32=i686 --with-default-libstdcxx-abi=new --with-gcc-major-version-only --with-multilib-list=m32,m64,mx32 --with-target-system-zlib=auto --with-tune=generic --without-cuda-driver -vProcessor Notes: Scaling Governor: amd-pstate-epp powersave (EPP: balance_performance) - CPU Microcode: 0xa108105Graphics Notes: BAR1 / Visible vRAM Size: 24560 MBPython Notes: Python 3.12.3Security Notes: gather_data_sampling: Not affected + itlb_multihit: Not affected + l1tf: Not affected + mds: Not affected + meltdown: Not affected + mmio_stale_data: Not affected + reg_file_data_sampling: Not affected + retbleed: Not affected + spec_rstack_overflow: Mitigation of Safe RET + spec_store_bypass: Mitigation of SSB disabled via prctl + spectre_v1: Mitigation of usercopy/swapgs barriers and __user pointer sanitization + spectre_v2: Mitigation of Enhanced / Automatic IBRS; IBPB: conditional; STIBP: always-on; RSB filling; PBRSB-eIBRS: Not affected; BHI: Not affected + srbds: Not affected + tsx_async_abort: Not affected
Testing initiated at 10 November 2024 13:29 by user amol.