Llama.cpp NVIDIA GeForce RTX 5090

Benchmarks by Michael Larabel for a future article on Phoronix.

HTML result view exported from: https://openbenchmarking.org/result/2501264-PTS-LLAMACPP76&gru.

Llama.cpp

Backend: NVIDIA CUDA - Model: Llama-3.1-Tulu-3-8B-Q8_0 - Test: Text Generation 128

Llama.cpp

Backend: NVIDIA CUDA - Model: Llama-3.1-Tulu-3-8B-Q8_0 - Test: Prompt Processing 512

Llama.cpp

Backend: NVIDIA CUDA - Model: Llama-3.1-Tulu-3-8B-Q8_0 - Test: Prompt Processing 1024

Llama.cpp

Backend: NVIDIA CUDA - Model: Llama-3.1-Tulu-3-8B-Q8_0 - Test: Prompt Processing 2048

Llama.cpp

Backend: NVIDIA CUDA - Model: Mistral-7B-Instruct-v0.3-Q8_0 - Test: Text Generation 128

Llama.cpp

Backend: NVIDIA CUDA - Model: Mistral-7B-Instruct-v0.3-Q8_0 - Test: Prompt Processing 512

Llama.cpp

Backend: NVIDIA CUDA - Model: Mistral-7B-Instruct-v0.3-Q8_0 - Test: Prompt Processing 1024

Llama.cpp

Backend: NVIDIA CUDA - Model: Mistral-7B-Instruct-v0.3-Q8_0 - Test: Prompt Processing 2048

Llama.cpp

GPU Temperature Monitor

Llama.cpp

GPU Temperature Monitor

Llama.cpp

GPU Temperature Monitor

Llama.cpp

GPU Temperature Monitor

Llama.cpp

GPU Temperature Monitor

Llama.cpp

GPU Temperature Monitor

Llama.cpp

GPU Temperature Monitor

Llama.cpp

GPU Temperature Monitor

Llama.cpp

GPU Power Consumption Monitor

Llama.cpp

GPU Power Consumption Monitor

Llama.cpp

GPU Power Consumption Monitor

Llama.cpp

GPU Power Consumption Monitor

Llama.cpp

GPU Power Consumption Monitor

Llama.cpp

GPU Power Consumption Monitor

Llama.cpp

GPU Power Consumption Monitor

Llama.cpp

GPU Power Consumption Monitor

Phoronix Test Suite v10.8.5