llama fun

AMD Ryzen Threadripper 7980X 64-Cores testing with a System76 Thelio Major (FA Z5 BIOS) and AMD Radeon RX 6700 XT 12GB on Ubuntu 24.04 via the Phoronix Test Suite.

HTML result view exported from: https://openbenchmarking.org/result/2411243-PTS-LLAMAFUN93.

llama funProcessorMotherboardChipsetMemoryDiskGraphicsAudioMonitorNetworkOSKernelDesktopDisplay ServerOpenGLCompilerFile-SystemScreen ResolutionAMD Ryzen Threadripper 7980X 64-Cores - AMD Radeon RXAMD Ryzen Threadripper 7980X 64-Cores @ 7.79GHz (64 Cores / 128 Threads)System76 Thelio Major (FA Z5 BIOS)AMD Device 14a44 x 32GB DDR5-4800MT/s Micron MTC20F1045S1RC48BA21000GB CT1000T700SSD5AMD Radeon RX 6700 XT 12GBAMD Device 14ccDELL P2415QAquantia AQC113C NBase-T/IEEE + Realtek RTL8125 2.5GbE + Intel Wi-Fi 6EUbuntu 24.046.8.0-48-generic (x86_64)GNOME Shell 46.0X Server + Wayland4.6 Mesa 24.0.9-0ubuntu0.2 (LLVM 17.0.6 DRM 3.57)GCC 13.2.0ext41920x1200OpenBenchmarking.org- Transparent Huge Pages: madvise- --build=x86_64-linux-gnu --disable-vtable-verify --disable-werror --enable-cet --enable-checking=release --enable-clocale=gnu --enable-default-pie --enable-gnu-unique-object --enable-languages=c,ada,c++,go,d,fortran,objc,obj-c++,m2 --enable-libphobos-checking=release --enable-libstdcxx-backtrace --enable-libstdcxx-debug --enable-libstdcxx-time=yes --enable-multiarch --enable-multilib --enable-nls --enable-objc-gc=auto --enable-offload-defaulted --enable-offload-targets=nvptx-none=/build/gcc-13-uJ7kn6/gcc-13-13.2.0/debian/tmp-nvptx/usr,amdgcn-amdhsa=/build/gcc-13-uJ7kn6/gcc-13-13.2.0/debian/tmp-gcn/usr --enable-plugin --enable-shared --enable-threads=posix --host=x86_64-linux-gnu --program-prefix=x86_64-linux-gnu- --target=x86_64-linux-gnu --with-abi=m64 --with-arch-32=i686 --with-default-libstdcxx-abi=new --with-gcc-major-version-only --with-multilib-list=m32,m64,mx32 --with-target-system-zlib=auto --with-tune=generic --without-cuda-driver -v - Scaling Governor: amd-pstate-epp powersave (EPP: balance_performance) - CPU Microcode: 0xa108105 - gather_data_sampling: Not affected + itlb_multihit: Not affected + l1tf: Not affected + mds: Not affected + meltdown: Not affected + mmio_stale_data: Not affected + reg_file_data_sampling: Not affected + retbleed: Not affected + spec_rstack_overflow: Mitigation of Safe RET + spec_store_bypass: Mitigation of SSB disabled via prctl + spectre_v1: Mitigation of usercopy/swapgs barriers and __user pointer sanitization + spectre_v2: Mitigation of Enhanced / Automatic IBRS; IBPB: conditional; STIBP: always-on; RSB filling; PBRSB-eIBRS: Not affected; BHI: Not affected + srbds: Not affected + tsx_async_abort: Not affected

llama funllama-cpp: CPU BLAS - Llama-3.1-Tulu-3-8B-Q8_0 - Text Generation 128llama-cpp: CPU BLAS - Llama-3.1-Tulu-3-8B-Q8_0 - Prompt Processing 512llama-cpp: CPU BLAS - Llama-3.1-Tulu-3-8B-Q8_0 - Prompt Processing 1024llama-cpp: CPU BLAS - Llama-3.1-Tulu-3-8B-Q8_0 - Prompt Processing 2048llama-cpp: CPU BLAS - granite-3.0-3b-a800m-instruct-Q8_0 - Text Generation 128llama-cpp: CPU BLAS - granite-3.0-3b-a800m-instruct-Q8_0 - Prompt Processing 512llama-cpp: CPU BLAS - granite-3.0-3b-a800m-instruct-Q8_0 - Prompt Processing 1024llama-cpp: CPU BLAS - granite-3.0-3b-a800m-instruct-Q8_0 - Prompt Processing 2048llama-cpp: CPU BLAS - Mistral-7B-Instruct-v0.3-Q8_0 - Text Generation 128llama-cpp: CPU BLAS - Mistral-7B-Instruct-v0.3-Q8_0 - Prompt Processing 512llama-cpp: CPU BLAS - Mistral-7B-Instruct-v0.3-Q8_0 - Prompt Processing 1024llama-cpp: CPU BLAS - Mistral-7B-Instruct-v0.3-Q8_0 - Prompt Processing 2048AMD Ryzen Threadripper 7980X 64-Cores - AMD Radeon RX15.51104.06123.05151.9176.72247.17331.37415.8516.34101.35125.64151.40OpenBenchmarking.org

Llama.cpp

Backend: CPU BLAS - Model: Llama-3.1-Tulu-3-8B-Q8_0 - Test: Text Generation 128

OpenBenchmarking.orgTokens Per Second, More Is BetterLlama.cpp b4154Backend: CPU BLAS - Model: Llama-3.1-Tulu-3-8B-Q8_0 - Test: Text Generation 128AMD Ryzen Threadripper 7980X 64-Cores - AMD Radeon RX48121620SE +/- 0.05, N = 315.511. (CXX) g++ options: -std=c++11 -fPIC -O3 -pthread -fopenmp -march=native -mtune=native -lopenblas

Llama.cpp

CPU Peak Freq (Highest CPU Core Frequency) Monitor

MinAvgMaxAMD Ryzen Threadripper 7980X 64-Cores - AMD Radeon RX390547375280OpenBenchmarking.orgMegahertz, More Is BetterLlama.cpp b4154CPU Peak Freq (Highest CPU Core Frequency) Monitor13002600390052006500

Llama.cpp

CPU Temperature Monitor

MinAvgMaxAMD Ryzen Threadripper 7980X 64-Cores - AMD Radeon RX37.552.761.9OpenBenchmarking.orgCelsius, Fewer Is BetterLlama.cpp b4154CPU Temperature Monitor20406080100

Llama.cpp

Drive Temperature (nvme0n1) Monitor

OpenBenchmarking.orgCelsius, Fewer Is BetterLlama.cpp b4154Drive Temperature (nvme0n1) MonitorAMD Ryzen Threadripper 7980X 64-Cores - AMD Radeon RX91827364539.85

Llama.cpp

Backend: CPU BLAS - Model: Llama-3.1-Tulu-3-8B-Q8_0 - Test: Prompt Processing 512

OpenBenchmarking.orgTokens Per Second, More Is BetterLlama.cpp b4154Backend: CPU BLAS - Model: Llama-3.1-Tulu-3-8B-Q8_0 - Test: Prompt Processing 512AMD Ryzen Threadripper 7980X 64-Cores - AMD Radeon RX20406080100SE +/- 1.41, N = 15104.061. (CXX) g++ options: -std=c++11 -fPIC -O3 -pthread -fopenmp -march=native -mtune=native -lopenblas

Llama.cpp

CPU Peak Freq (Highest CPU Core Frequency) Monitor

MinAvgMaxAMD Ryzen Threadripper 7980X 64-Cores - AMD Radeon RX203043795290OpenBenchmarking.orgMegahertz, More Is BetterLlama.cpp b4154CPU Peak Freq (Highest CPU Core Frequency) Monitor13002600390052006500

Llama.cpp

CPU Temperature Monitor

MinAvgMaxAMD Ryzen Threadripper 7980X 64-Cores - AMD Radeon RX46.063.073.0OpenBenchmarking.orgCelsius, Fewer Is BetterLlama.cpp b4154CPU Temperature Monitor20406080100

Llama.cpp

Drive Temperature (nvme0n1) Monitor

MinAvgMaxAMD Ryzen Threadripper 7980X 64-Cores - AMD Radeon RX39.940.740.9OpenBenchmarking.orgCelsius, Fewer Is BetterLlama.cpp b4154Drive Temperature (nvme0n1) Monitor1122334455

Llama.cpp

Backend: CPU BLAS - Model: Llama-3.1-Tulu-3-8B-Q8_0 - Test: Prompt Processing 1024

OpenBenchmarking.orgTokens Per Second, More Is BetterLlama.cpp b4154Backend: CPU BLAS - Model: Llama-3.1-Tulu-3-8B-Q8_0 - Test: Prompt Processing 1024AMD Ryzen Threadripper 7980X 64-Cores - AMD Radeon RX306090120150SE +/- 1.50, N = 3123.051. (CXX) g++ options: -std=c++11 -fPIC -O3 -pthread -fopenmp -march=native -mtune=native -lopenblas

Llama.cpp

CPU Peak Freq (Highest CPU Core Frequency) Monitor

MinAvgMaxAMD Ryzen Threadripper 7980X 64-Cores - AMD Radeon RX396543395342OpenBenchmarking.orgMegahertz, More Is BetterLlama.cpp b4154CPU Peak Freq (Highest CPU Core Frequency) Monitor13002600390052006500

Llama.cpp

CPU Temperature Monitor

MinAvgMaxAMD Ryzen Threadripper 7980X 64-Cores - AMD Radeon RX47.964.672.4OpenBenchmarking.orgCelsius, Fewer Is BetterLlama.cpp b4154CPU Temperature Monitor20406080100

Llama.cpp

Drive Temperature (nvme0n1) Monitor

MinAvgMaxAMD Ryzen Threadripper 7980X 64-Cores - AMD Radeon RX40.941.541.9OpenBenchmarking.orgCelsius, Fewer Is BetterLlama.cpp b4154Drive Temperature (nvme0n1) Monitor1224364860

Llama.cpp

Backend: CPU BLAS - Model: Llama-3.1-Tulu-3-8B-Q8_0 - Test: Prompt Processing 2048

OpenBenchmarking.orgTokens Per Second, More Is BetterLlama.cpp b4154Backend: CPU BLAS - Model: Llama-3.1-Tulu-3-8B-Q8_0 - Test: Prompt Processing 2048AMD Ryzen Threadripper 7980X 64-Cores - AMD Radeon RX306090120150SE +/- 1.39, N = 15151.911. (CXX) g++ options: -std=c++11 -fPIC -O3 -pthread -fopenmp -march=native -mtune=native -lopenblas

Llama.cpp

CPU Peak Freq (Highest CPU Core Frequency) Monitor

OpenBenchmarking.orgMegahertz, More Is BetterLlama.cpp b4154CPU Peak Freq (Highest CPU Core Frequency) MonitorAMD Ryzen Threadripper 7980X 64-Cores - AMD Radeon RX9001800270036004500Min: 1946 / Avg: 4219.67 / Max: 5274

Llama.cpp

CPU Temperature Monitor

OpenBenchmarking.orgCelsius, Fewer Is BetterLlama.cpp b4154CPU Temperature MonitorAMD Ryzen Threadripper 7980X 64-Cores - AMD Radeon RX1632486480Min: 48.25 / Avg: 67.02 / Max: 81.75

Llama.cpp

Drive Temperature (nvme0n1) Monitor

OpenBenchmarking.orgCelsius, Fewer Is BetterLlama.cpp b4154Drive Temperature (nvme0n1) MonitorAMD Ryzen Threadripper 7980X 64-Cores - AMD Radeon RX918273645Min: 41.85 / Avg: 42.56 / Max: 43.85

Llama.cpp

Backend: CPU BLAS - Model: granite-3.0-3b-a800m-instruct-Q8_0 - Test: Text Generation 128

OpenBenchmarking.orgTokens Per Second, More Is BetterLlama.cpp b4154Backend: CPU BLAS - Model: granite-3.0-3b-a800m-instruct-Q8_0 - Test: Text Generation 128AMD Ryzen Threadripper 7980X 64-Cores - AMD Radeon RX20406080100SE +/- 0.24, N = 576.721. (CXX) g++ options: -std=c++11 -fPIC -O3 -pthread -fopenmp -march=native -mtune=native -lopenblas

Llama.cpp

CPU Peak Freq (Highest CPU Core Frequency) Monitor

MinAvgMaxAMD Ryzen Threadripper 7980X 64-Cores - AMD Radeon RX404346005288OpenBenchmarking.orgMegahertz, More Is BetterLlama.cpp b4154CPU Peak Freq (Highest CPU Core Frequency) Monitor13002600390052006500

Llama.cpp

CPU Temperature Monitor

MinAvgMaxAMD Ryzen Threadripper 7980X 64-Cores - AMD Radeon RX50.557.868.1OpenBenchmarking.orgCelsius, Fewer Is BetterLlama.cpp b4154CPU Temperature Monitor20406080100

Llama.cpp

Drive Temperature (nvme0n1) Monitor

OpenBenchmarking.orgCelsius, Fewer Is BetterLlama.cpp b4154Drive Temperature (nvme0n1) MonitorAMD Ryzen Threadripper 7980X 64-Cores - AMD Radeon RX102030405043.85

Llama.cpp

Backend: CPU BLAS - Model: granite-3.0-3b-a800m-instruct-Q8_0 - Test: Prompt Processing 512

OpenBenchmarking.orgTokens Per Second, More Is BetterLlama.cpp b4154Backend: CPU BLAS - Model: granite-3.0-3b-a800m-instruct-Q8_0 - Test: Prompt Processing 512AMD Ryzen Threadripper 7980X 64-Cores - AMD Radeon RX50100150200250SE +/- 2.17, N = 15247.171. (CXX) g++ options: -std=c++11 -fPIC -O3 -pthread -fopenmp -march=native -mtune=native -lopenblas

Llama.cpp

CPU Peak Freq (Highest CPU Core Frequency) Monitor

MinAvgMaxAMD Ryzen Threadripper 7980X 64-Cores - AMD Radeon RX329644605316OpenBenchmarking.orgMegahertz, More Is BetterLlama.cpp b4154CPU Peak Freq (Highest CPU Core Frequency) Monitor13002600390052006500

Llama.cpp

CPU Temperature Monitor

MinAvgMaxAMD Ryzen Threadripper 7980X 64-Cores - AMD Radeon RX47.963.873.9OpenBenchmarking.orgCelsius, Fewer Is BetterLlama.cpp b4154CPU Temperature Monitor20406080100

Llama.cpp

Drive Temperature (nvme0n1) Monitor

MinAvgMaxAMD Ryzen Threadripper 7980X 64-Cores - AMD Radeon RX42.943.843.9OpenBenchmarking.orgCelsius, Fewer Is BetterLlama.cpp b4154Drive Temperature (nvme0n1) Monitor1224364860

Llama.cpp

Backend: CPU BLAS - Model: granite-3.0-3b-a800m-instruct-Q8_0 - Test: Prompt Processing 1024

OpenBenchmarking.orgTokens Per Second, More Is BetterLlama.cpp b4154Backend: CPU BLAS - Model: granite-3.0-3b-a800m-instruct-Q8_0 - Test: Prompt Processing 1024AMD Ryzen Threadripper 7980X 64-Cores - AMD Radeon RX70140210280350SE +/- 4.15, N = 15331.371. (CXX) g++ options: -std=c++11 -fPIC -O3 -pthread -fopenmp -march=native -mtune=native -lopenblas

Llama.cpp

CPU Peak Freq (Highest CPU Core Frequency) Monitor

MinAvgMaxAMD Ryzen Threadripper 7980X 64-Cores - AMD Radeon RX217844045258OpenBenchmarking.orgMegahertz, More Is BetterLlama.cpp b4154CPU Peak Freq (Highest CPU Core Frequency) Monitor13002600390052006500

Llama.cpp

CPU Temperature Monitor

MinAvgMaxAMD Ryzen Threadripper 7980X 64-Cores - AMD Radeon RX47.465.378.8OpenBenchmarking.orgCelsius, Fewer Is BetterLlama.cpp b4154CPU Temperature Monitor20406080100

Llama.cpp

Drive Temperature (nvme0n1) Monitor

MinAvgMaxAMD Ryzen Threadripper 7980X 64-Cores - AMD Radeon RX42.942.943.9OpenBenchmarking.orgCelsius, Fewer Is BetterLlama.cpp b4154Drive Temperature (nvme0n1) Monitor1224364860

Llama.cpp

Backend: CPU BLAS - Model: granite-3.0-3b-a800m-instruct-Q8_0 - Test: Prompt Processing 2048

OpenBenchmarking.orgTokens Per Second, More Is BetterLlama.cpp b4154Backend: CPU BLAS - Model: granite-3.0-3b-a800m-instruct-Q8_0 - Test: Prompt Processing 2048AMD Ryzen Threadripper 7980X 64-Cores - AMD Radeon RX90180270360450SE +/- 3.55, N = 8415.851. (CXX) g++ options: -std=c++11 -fPIC -O3 -pthread -fopenmp -march=native -mtune=native -lopenblas

Llama.cpp

CPU Peak Freq (Highest CPU Core Frequency) Monitor

MinAvgMaxAMD Ryzen Threadripper 7980X 64-Cores - AMD Radeon RX350343245284OpenBenchmarking.orgMegahertz, More Is BetterLlama.cpp b4154CPU Peak Freq (Highest CPU Core Frequency) Monitor13002600390052006500

Llama.cpp

CPU Temperature Monitor

MinAvgMaxAMD Ryzen Threadripper 7980X 64-Cores - AMD Radeon RX48.365.773.9OpenBenchmarking.orgCelsius, Fewer Is BetterLlama.cpp b4154CPU Temperature Monitor20406080100

Llama.cpp

Drive Temperature (nvme0n1) Monitor

MinAvgMaxAMD Ryzen Threadripper 7980X 64-Cores - AMD Radeon RX41.942.742.9OpenBenchmarking.orgCelsius, Fewer Is BetterLlama.cpp b4154Drive Temperature (nvme0n1) Monitor1224364860

Llama.cpp

Backend: CPU BLAS - Model: Mistral-7B-Instruct-v0.3-Q8_0 - Test: Text Generation 128

OpenBenchmarking.orgTokens Per Second, More Is BetterLlama.cpp b4154Backend: CPU BLAS - Model: Mistral-7B-Instruct-v0.3-Q8_0 - Test: Text Generation 128AMD Ryzen Threadripper 7980X 64-Cores - AMD Radeon RX48121620SE +/- 0.08, N = 316.341. (CXX) g++ options: -std=c++11 -fPIC -O3 -pthread -fopenmp -march=native -mtune=native -lopenblas

Llama.cpp

CPU Peak Freq (Highest CPU Core Frequency) Monitor

MinAvgMaxAMD Ryzen Threadripper 7980X 64-Cores - AMD Radeon RX336647605339OpenBenchmarking.orgMegahertz, More Is BetterLlama.cpp b4154CPU Peak Freq (Highest CPU Core Frequency) Monitor13002600390052006500

Llama.cpp

CPU Temperature Monitor

MinAvgMaxAMD Ryzen Threadripper 7980X 64-Cores - AMD Radeon RX48.860.066.3OpenBenchmarking.orgCelsius, Fewer Is BetterLlama.cpp b4154CPU Temperature Monitor20406080100

Llama.cpp

Drive Temperature (nvme0n1) Monitor

MinAvgMaxAMD Ryzen Threadripper 7980X 64-Cores - AMD Radeon RX41.943.145.9OpenBenchmarking.orgCelsius, Fewer Is BetterLlama.cpp b4154Drive Temperature (nvme0n1) Monitor1224364860

Llama.cpp

Backend: CPU BLAS - Model: Mistral-7B-Instruct-v0.3-Q8_0 - Test: Prompt Processing 512

OpenBenchmarking.orgTokens Per Second, More Is BetterLlama.cpp b4154Backend: CPU BLAS - Model: Mistral-7B-Instruct-v0.3-Q8_0 - Test: Prompt Processing 512AMD Ryzen Threadripper 7980X 64-Cores - AMD Radeon RX20406080100SE +/- 1.28, N = 15101.351. (CXX) g++ options: -std=c++11 -fPIC -O3 -pthread -fopenmp -march=native -mtune=native -lopenblas

Llama.cpp

CPU Peak Freq (Highest CPU Core Frequency) Monitor

MinAvgMaxAMD Ryzen Threadripper 7980X 64-Cores - AMD Radeon RX234843505338OpenBenchmarking.orgMegahertz, More Is BetterLlama.cpp b4154CPU Peak Freq (Highest CPU Core Frequency) Monitor13002600390052006500

Llama.cpp

CPU Temperature Monitor

MinAvgMaxAMD Ryzen Threadripper 7980X 64-Cores - AMD Radeon RX50.565.073.8OpenBenchmarking.orgCelsius, Fewer Is BetterLlama.cpp b4154CPU Temperature Monitor20406080100

Llama.cpp

Drive Temperature (nvme0n1) Monitor

MinAvgMaxAMD Ryzen Threadripper 7980X 64-Cores - AMD Radeon RX42.943.244.9OpenBenchmarking.orgCelsius, Fewer Is BetterLlama.cpp b4154Drive Temperature (nvme0n1) Monitor1224364860

Llama.cpp

Backend: CPU BLAS - Model: Mistral-7B-Instruct-v0.3-Q8_0 - Test: Prompt Processing 1024

OpenBenchmarking.orgTokens Per Second, More Is BetterLlama.cpp b4154Backend: CPU BLAS - Model: Mistral-7B-Instruct-v0.3-Q8_0 - Test: Prompt Processing 1024AMD Ryzen Threadripper 7980X 64-Cores - AMD Radeon RX306090120150SE +/- 1.22, N = 15125.641. (CXX) g++ options: -std=c++11 -fPIC -O3 -pthread -fopenmp -march=native -mtune=native -lopenblas

Llama.cpp

CPU Peak Freq (Highest CPU Core Frequency) Monitor

MinAvgMaxAMD Ryzen Threadripper 7980X 64-Cores - AMD Radeon RX225543025341OpenBenchmarking.orgMegahertz, More Is BetterLlama.cpp b4154CPU Peak Freq (Highest CPU Core Frequency) Monitor13002600390052006500

Llama.cpp

CPU Temperature Monitor

MinAvgMaxAMD Ryzen Threadripper 7980X 64-Cores - AMD Radeon RX48.866.581.3OpenBenchmarking.orgCelsius, Fewer Is BetterLlama.cpp b4154CPU Temperature Monitor20406080100

Llama.cpp

Drive Temperature (nvme0n1) Monitor

OpenBenchmarking.orgCelsius, Fewer Is BetterLlama.cpp b4154Drive Temperature (nvme0n1) MonitorAMD Ryzen Threadripper 7980X 64-Cores - AMD Radeon RX102030405042.85

Llama.cpp

Backend: CPU BLAS - Model: Mistral-7B-Instruct-v0.3-Q8_0 - Test: Prompt Processing 2048

OpenBenchmarking.orgTokens Per Second, More Is BetterLlama.cpp b4154Backend: CPU BLAS - Model: Mistral-7B-Instruct-v0.3-Q8_0 - Test: Prompt Processing 2048AMD Ryzen Threadripper 7980X 64-Cores - AMD Radeon RX306090120150SE +/- 1.25, N = 15151.401. (CXX) g++ options: -std=c++11 -fPIC -O3 -pthread -fopenmp -march=native -mtune=native -lopenblas

Llama.cpp

CPU Peak Freq (Highest CPU Core Frequency) Monitor

OpenBenchmarking.orgMegahertz, More Is BetterLlama.cpp b4154CPU Peak Freq (Highest CPU Core Frequency) MonitorAMD Ryzen Threadripper 7980X 64-Cores - AMD Radeon RX9001800270036004500Min: 3718 / Avg: 4213.28 / Max: 5243

Llama.cpp

CPU Temperature Monitor

OpenBenchmarking.orgCelsius, Fewer Is BetterLlama.cpp b4154CPU Temperature MonitorAMD Ryzen Threadripper 7980X 64-Cores - AMD Radeon RX1530456075Min: 49.63 / Avg: 67.59 / Max: 75.63

Llama.cpp

Drive Temperature (nvme0n1) Monitor

OpenBenchmarking.orgCelsius, Fewer Is BetterLlama.cpp b4154Drive Temperature (nvme0n1) MonitorAMD Ryzen Threadripper 7980X 64-Cores - AMD Radeon RX918273645Min: 42.85 / Avg: 43.47 / Max: 43.85


Phoronix Test Suite v10.8.5