Llama.cpp NVIDIA GeForce RTX 5090

Benchmarks by Michael Larabel for a future article on Phoronix.

HTML result view exported from: https://openbenchmarking.org/result/2501264-PTS-LLAMACPP76.

Llama.cpp

Backend: NVIDIA CUDA - Model: Llama-3.1-Tulu-3-8B-Q8_0 - Test: Text Generation 128

Llama.cpp

GPU Power Consumption Monitor

Llama.cpp

GPU Temperature Monitor

Llama.cpp

Backend: NVIDIA CUDA - Model: Llama-3.1-Tulu-3-8B-Q8_0 - Test: Prompt Processing 512

Llama.cpp

GPU Power Consumption Monitor

Llama.cpp

GPU Temperature Monitor

Llama.cpp

Backend: NVIDIA CUDA - Model: Llama-3.1-Tulu-3-8B-Q8_0 - Test: Prompt Processing 1024

Llama.cpp

GPU Power Consumption Monitor

Llama.cpp

GPU Temperature Monitor

Llama.cpp

Backend: NVIDIA CUDA - Model: Llama-3.1-Tulu-3-8B-Q8_0 - Test: Prompt Processing 2048

Llama.cpp

GPU Power Consumption Monitor

Llama.cpp

GPU Temperature Monitor

Llama.cpp

Backend: NVIDIA CUDA - Model: Mistral-7B-Instruct-v0.3-Q8_0 - Test: Text Generation 128

Llama.cpp

GPU Power Consumption Monitor

Llama.cpp

GPU Temperature Monitor

Llama.cpp

Backend: NVIDIA CUDA - Model: Mistral-7B-Instruct-v0.3-Q8_0 - Test: Prompt Processing 512

Llama.cpp

GPU Power Consumption Monitor

Llama.cpp

GPU Temperature Monitor

Llama.cpp

Backend: NVIDIA CUDA - Model: Mistral-7B-Instruct-v0.3-Q8_0 - Test: Prompt Processing 1024

Llama.cpp

GPU Power Consumption Monitor

Llama.cpp

GPU Temperature Monitor

Llama.cpp

Backend: NVIDIA CUDA - Model: Mistral-7B-Instruct-v0.3-Q8_0 - Test: Prompt Processing 2048

Llama.cpp

GPU Power Consumption Monitor

Llama.cpp

GPU Temperature Monitor

Phoronix Test Suite v10.8.5