llama fun AMD Ryzen Threadripper 7980X 64-Cores testing with a System76 Thelio Major (FA Z5 BIOS) and AMD Radeon RX 6700 XT 12GB on Ubuntu 24.04 via the Phoronix Test Suite. AMD Ryzen Threadripper 7980X 64-Cores - AMD Radeon RX: Processor: AMD Ryzen Threadripper 7980X 64-Cores @ 7.79GHz (64 Cores / 128 Threads), Motherboard: System76 Thelio Major (FA Z5 BIOS), Chipset: AMD Device 14a4, Memory: 4 x 32GB DDR5-4800MT/s Micron MTC20F1045S1RC48BA2, Disk: 1000GB CT1000T700SSD5, Graphics: AMD Radeon RX 6700 XT 12GB, Audio: AMD Device 14cc, Monitor: DELL P2415Q, Network: Aquantia AQC113C NBase-T/IEEE + Realtek RTL8125 2.5GbE + Intel Wi-Fi 6E OS: Ubuntu 24.04, Kernel: 6.8.0-48-generic (x86_64), Desktop: GNOME Shell 46.0, Display Server: X Server + Wayland, OpenGL: 4.6 Mesa 24.0.9-0ubuntu0.2 (LLVM 17.0.6 DRM 3.57), Compiler: GCC 13.2.0, File-System: ext4, Screen Resolution: 1920x1200 Llama.cpp b4154 Backend: CPU BLAS - Model: Llama-3.1-Tulu-3-8B-Q8_0 - Test: Text Generation 128 Tokens Per Second > Higher Is Better AMD Ryzen Threadripper 7980X 64-Cores - AMD Radeon RX . 15.51 |================ Llama.cpp b4154 CPU Peak Freq (Highest CPU Core Frequency) Monitor Megahertz > Higher Is Better AMD Ryzen Threadripper 7980X 64-Cores - AMD Radeon RX . MIN: 3905 AVG: 4737 MAX: 5280 Llama.cpp b4154 CPU Temperature Monitor Celsius < Lower Is Better AMD Ryzen Threadripper 7980X 64-Cores - AMD Radeon RX . MIN: 37.5 AVG: 52.7 MAX: 61.9 Llama.cpp b4154 Drive Temperature (nvme0n1) Monitor Celsius < Lower Is Better AMD Ryzen Threadripper 7980X 64-Cores - AMD Radeon RX . MIN: 39.9 AVG: 39.9 MAX: 39.9 Llama.cpp b4154 Backend: CPU BLAS - Model: Llama-3.1-Tulu-3-8B-Q8_0 - Test: Prompt Processing 512 Tokens Per Second > Higher Is Better AMD Ryzen Threadripper 7980X 64-Cores - AMD Radeon RX . 104.06 |=============== Llama.cpp b4154 CPU Peak Freq (Highest CPU Core Frequency) Monitor Megahertz > Higher Is Better AMD Ryzen Threadripper 7980X 64-Cores - AMD Radeon RX . MIN: 2030 AVG: 4379 MAX: 5290 Llama.cpp b4154 CPU Temperature Monitor Celsius < Lower Is Better AMD Ryzen Threadripper 7980X 64-Cores - AMD Radeon RX . MIN: 46.0 AVG: 63.0 MAX: 73.0 Llama.cpp b4154 Drive Temperature (nvme0n1) Monitor Celsius < Lower Is Better AMD Ryzen Threadripper 7980X 64-Cores - AMD Radeon RX . MIN: 39.9 AVG: 40.7 MAX: 40.9 Llama.cpp b4154 Backend: CPU BLAS - Model: Llama-3.1-Tulu-3-8B-Q8_0 - Test: Prompt Processing 1024 Tokens Per Second > Higher Is Better AMD Ryzen Threadripper 7980X 64-Cores - AMD Radeon RX . 123.05 |=============== Llama.cpp b4154 CPU Peak Freq (Highest CPU Core Frequency) Monitor Megahertz > Higher Is Better AMD Ryzen Threadripper 7980X 64-Cores - AMD Radeon RX . MIN: 3965 AVG: 4339 MAX: 5342 Llama.cpp b4154 CPU Temperature Monitor Celsius < Lower Is Better AMD Ryzen Threadripper 7980X 64-Cores - AMD Radeon RX . MIN: 47.9 AVG: 64.6 MAX: 72.4 Llama.cpp b4154 Drive Temperature (nvme0n1) Monitor Celsius < Lower Is Better AMD Ryzen Threadripper 7980X 64-Cores - AMD Radeon RX . MIN: 40.9 AVG: 41.5 MAX: 41.9 Llama.cpp b4154 Backend: CPU BLAS - Model: Llama-3.1-Tulu-3-8B-Q8_0 - Test: Prompt Processing 2048 Tokens Per Second > Higher Is Better AMD Ryzen Threadripper 7980X 64-Cores - AMD Radeon RX . 151.91 |=============== Llama.cpp b4154 CPU Peak Freq (Highest CPU Core Frequency) Monitor Megahertz > Higher Is Better AMD Ryzen Threadripper 7980X 64-Cores - AMD Radeon RX . MIN: 1946 AVG: 4220 MAX: 5274 Llama.cpp b4154 CPU Temperature Monitor Celsius < Lower Is Better AMD Ryzen Threadripper 7980X 64-Cores - AMD Radeon RX . MIN: 48 AVG: 67 MAX: 82 Llama.cpp b4154 Drive Temperature (nvme0n1) Monitor Celsius < Lower Is Better AMD Ryzen Threadripper 7980X 64-Cores - AMD Radeon RX . MIN: 41.9 AVG: 42.6 MAX: 43.9 Llama.cpp b4154 Backend: CPU BLAS - Model: granite-3.0-3b-a800m-instruct-Q8_0 - Test: Text Generation 128 Tokens Per Second > Higher Is Better AMD Ryzen Threadripper 7980X 64-Cores - AMD Radeon RX . 76.72 |================ Llama.cpp b4154 CPU Peak Freq (Highest CPU Core Frequency) Monitor Megahertz > Higher Is Better AMD Ryzen Threadripper 7980X 64-Cores - AMD Radeon RX . MIN: 4043 AVG: 4600 MAX: 5288 Llama.cpp b4154 CPU Temperature Monitor Celsius < Lower Is Better AMD Ryzen Threadripper 7980X 64-Cores - AMD Radeon RX . MIN: 50.5 AVG: 57.8 MAX: 68.1 Llama.cpp b4154 Drive Temperature (nvme0n1) Monitor Celsius < Lower Is Better AMD Ryzen Threadripper 7980X 64-Cores - AMD Radeon RX . MIN: 43.9 AVG: 43.9 MAX: 43.9 Llama.cpp b4154 Backend: CPU BLAS - Model: granite-3.0-3b-a800m-instruct-Q8_0 - Test: Prompt Processing 512 Tokens Per Second > Higher Is Better AMD Ryzen Threadripper 7980X 64-Cores - AMD Radeon RX . 247.17 |=============== Llama.cpp b4154 CPU Peak Freq (Highest CPU Core Frequency) Monitor Megahertz > Higher Is Better AMD Ryzen Threadripper 7980X 64-Cores - AMD Radeon RX . MIN: 3296 AVG: 4460 MAX: 5316 Llama.cpp b4154 CPU Temperature Monitor Celsius < Lower Is Better AMD Ryzen Threadripper 7980X 64-Cores - AMD Radeon RX . MIN: 47.9 AVG: 63.8 MAX: 73.9 Llama.cpp b4154 Drive Temperature (nvme0n1) Monitor Celsius < Lower Is Better AMD Ryzen Threadripper 7980X 64-Cores - AMD Radeon RX . MIN: 42.9 AVG: 43.8 MAX: 43.9 Llama.cpp b4154 Backend: CPU BLAS - Model: granite-3.0-3b-a800m-instruct-Q8_0 - Test: Prompt Processing 1024 Tokens Per Second > Higher Is Better AMD Ryzen Threadripper 7980X 64-Cores - AMD Radeon RX . 331.37 |=============== Llama.cpp b4154 CPU Peak Freq (Highest CPU Core Frequency) Monitor Megahertz > Higher Is Better AMD Ryzen Threadripper 7980X 64-Cores - AMD Radeon RX . MIN: 2178 AVG: 4404 MAX: 5258 Llama.cpp b4154 CPU Temperature Monitor Celsius < Lower Is Better AMD Ryzen Threadripper 7980X 64-Cores - AMD Radeon RX . MIN: 47 AVG: 65 MAX: 79 Llama.cpp b4154 Drive Temperature (nvme0n1) Monitor Celsius < Lower Is Better AMD Ryzen Threadripper 7980X 64-Cores - AMD Radeon RX . MIN: 42.9 AVG: 42.9 MAX: 43.9 Llama.cpp b4154 Backend: CPU BLAS - Model: granite-3.0-3b-a800m-instruct-Q8_0 - Test: Prompt Processing 2048 Tokens Per Second > Higher Is Better AMD Ryzen Threadripper 7980X 64-Cores - AMD Radeon RX . 415.85 |=============== Llama.cpp b4154 CPU Peak Freq (Highest CPU Core Frequency) Monitor Megahertz > Higher Is Better AMD Ryzen Threadripper 7980X 64-Cores - AMD Radeon RX . MIN: 3503 AVG: 4324 MAX: 5284 Llama.cpp b4154 CPU Temperature Monitor Celsius < Lower Is Better AMD Ryzen Threadripper 7980X 64-Cores - AMD Radeon RX . MIN: 48.3 AVG: 65.7 MAX: 73.9 Llama.cpp b4154 Drive Temperature (nvme0n1) Monitor Celsius < Lower Is Better AMD Ryzen Threadripper 7980X 64-Cores - AMD Radeon RX . MIN: 41.9 AVG: 42.7 MAX: 42.9 Llama.cpp b4154 Backend: CPU BLAS - Model: Mistral-7B-Instruct-v0.3-Q8_0 - Test: Text Generation 128 Tokens Per Second > Higher Is Better AMD Ryzen Threadripper 7980X 64-Cores - AMD Radeon RX . 16.34 |================ Llama.cpp b4154 CPU Peak Freq (Highest CPU Core Frequency) Monitor Megahertz > Higher Is Better AMD Ryzen Threadripper 7980X 64-Cores - AMD Radeon RX . MIN: 3366 AVG: 4760 MAX: 5339 Llama.cpp b4154 CPU Temperature Monitor Celsius < Lower Is Better AMD Ryzen Threadripper 7980X 64-Cores - AMD Radeon RX . MIN: 48.8 AVG: 60.0 MAX: 66.3 Llama.cpp b4154 Drive Temperature (nvme0n1) Monitor Celsius < Lower Is Better AMD Ryzen Threadripper 7980X 64-Cores - AMD Radeon RX . MIN: 41.9 AVG: 43.1 MAX: 45.9 Llama.cpp b4154 Backend: CPU BLAS - Model: Mistral-7B-Instruct-v0.3-Q8_0 - Test: Prompt Processing 512 Tokens Per Second > Higher Is Better AMD Ryzen Threadripper 7980X 64-Cores - AMD Radeon RX . 101.35 |=============== Llama.cpp b4154 CPU Peak Freq (Highest CPU Core Frequency) Monitor Megahertz > Higher Is Better AMD Ryzen Threadripper 7980X 64-Cores - AMD Radeon RX . MIN: 2348 AVG: 4350 MAX: 5338 Llama.cpp b4154 CPU Temperature Monitor Celsius < Lower Is Better AMD Ryzen Threadripper 7980X 64-Cores - AMD Radeon RX . MIN: 50.5 AVG: 65.0 MAX: 73.8 Llama.cpp b4154 Drive Temperature (nvme0n1) Monitor Celsius < Lower Is Better AMD Ryzen Threadripper 7980X 64-Cores - AMD Radeon RX . MIN: 42.9 AVG: 43.2 MAX: 44.9 Llama.cpp b4154 Backend: CPU BLAS - Model: Mistral-7B-Instruct-v0.3-Q8_0 - Test: Prompt Processing 1024 Tokens Per Second > Higher Is Better AMD Ryzen Threadripper 7980X 64-Cores - AMD Radeon RX . 125.64 |=============== Llama.cpp b4154 CPU Peak Freq (Highest CPU Core Frequency) Monitor Megahertz > Higher Is Better AMD Ryzen Threadripper 7980X 64-Cores - AMD Radeon RX . MIN: 2255 AVG: 4302 MAX: 5341 Llama.cpp b4154 CPU Temperature Monitor Celsius < Lower Is Better AMD Ryzen Threadripper 7980X 64-Cores - AMD Radeon RX . MIN: 49 AVG: 66 MAX: 81 Llama.cpp b4154 Drive Temperature (nvme0n1) Monitor Celsius < Lower Is Better AMD Ryzen Threadripper 7980X 64-Cores - AMD Radeon RX . MIN: 42.9 AVG: 42.8 MAX: 42.9 Llama.cpp b4154 Backend: CPU BLAS - Model: Mistral-7B-Instruct-v0.3-Q8_0 - Test: Prompt Processing 2048 Tokens Per Second > Higher Is Better AMD Ryzen Threadripper 7980X 64-Cores - AMD Radeon RX . 151.40 |=============== Llama.cpp b4154 CPU Peak Freq (Highest CPU Core Frequency) Monitor Megahertz > Higher Is Better AMD Ryzen Threadripper 7980X 64-Cores - AMD Radeon RX . MIN: 3718 AVG: 4213 MAX: 5243 Llama.cpp b4154 CPU Temperature Monitor Celsius < Lower Is Better AMD Ryzen Threadripper 7980X 64-Cores - AMD Radeon RX . MIN: 49.6 AVG: 67.6 MAX: 75.6 Llama.cpp b4154 Drive Temperature (nvme0n1) Monitor Celsius < Lower Is Better AMD Ryzen Threadripper 7980X 64-Cores - AMD Radeon RX . MIN: 42.9 AVG: 43.5 MAX: 43.9