newaa AMD Ryzen Threadripper 7980X 64-Cores testing with a System76 Thelio Major (FA Z5 BIOS) and AMD Radeon RX 6700 XT 12GB on Ubuntu 24.04 via the Phoronix Test Suite. a: Processor: AMD Ryzen Threadripper 7980X 64-Cores @ 7.79GHz (64 Cores / 128 Threads), Motherboard: System76 Thelio Major (FA Z5 BIOS), Chipset: AMD Device 14a4, Memory: 4 x 32GB DDR5-4800MT/s Micron MTC20F1045S1RC48BA2, Disk: 1000GB CT1000T700SSD5, Graphics: AMD Radeon RX 6700 XT 12GB, Audio: AMD Device 14cc, Monitor: DELL P2415Q, Network: Aquantia AQC113C NBase-T/IEEE + Realtek RTL8125 2.5GbE + Intel Wi-Fi 6E OS: Ubuntu 24.04, Kernel: 6.8.0-49-generic (x86_64), Desktop: GNOME Shell 46.0, Display Server: X Server + Wayland, OpenGL: 4.6 Mesa 24.0.9-0ubuntu0.2 (LLVM 17.0.6 DRM 3.57), Compiler: GCC 13.2.0, File-System: ext4, Screen Resolution: 1920x1200 b: Processor: AMD Ryzen Threadripper 7980X 64-Cores @ 7.79GHz (64 Cores / 128 Threads), Motherboard: System76 Thelio Major (FA Z5 BIOS), Chipset: AMD Device 14a4, Memory: 4 x 32GB DDR5-4800MT/s Micron MTC20F1045S1RC48BA2, Disk: 1000GB CT1000T700SSD5, Graphics: AMD Radeon RX 6700 XT 12GB, Audio: AMD Device 14cc, Monitor: DELL P2415Q, Network: Aquantia AQC113C NBase-T/IEEE + Realtek RTL8125 2.5GbE + Intel Wi-Fi 6E OS: Ubuntu 24.04, Kernel: 6.8.0-49-generic (x86_64), Desktop: GNOME Shell 46.0, Display Server: X Server + Wayland, OpenGL: 4.6 Mesa 24.0.9-0ubuntu0.2 (LLVM 17.0.6 DRM 3.57), Compiler: GCC 13.2.0, File-System: ext4, Screen Resolution: 1920x1200 c: Processor: AMD Ryzen Threadripper 7980X 64-Cores @ 7.79GHz (64 Cores / 128 Threads), Motherboard: System76 Thelio Major (FA Z5 BIOS), Chipset: AMD Device 14a4, Memory: 4 x 32GB DDR5-4800MT/s Micron MTC20F1045S1RC48BA2, Disk: 1000GB CT1000T700SSD5, Graphics: AMD Radeon RX 6700 XT 12GB, Audio: AMD Device 14cc, Monitor: DELL P2415Q, Network: Aquantia AQC113C NBase-T/IEEE + Realtek RTL8125 2.5GbE + Intel Wi-Fi 6E OS: Ubuntu 24.04, Kernel: 6.8.0-49-generic (x86_64), Desktop: GNOME Shell 46.0, Display Server: X Server + Wayland, OpenGL: 4.6 Mesa 24.0.9-0ubuntu0.2 (LLVM 17.0.6 DRM 3.57), Compiler: GCC 13.2.0, File-System: ext4, Screen Resolution: 1920x1200 d: Processor: AMD Ryzen Threadripper 7980X 64-Cores @ 7.79GHz (64 Cores / 128 Threads), Motherboard: System76 Thelio Major (FA Z5 BIOS), Chipset: AMD Device 14a4, Memory: 4 x 32GB DDR5-4800MT/s Micron MTC20F1045S1RC48BA2, Disk: 1000GB CT1000T700SSD5, Graphics: AMD Radeon RX 6700 XT 12GB, Audio: AMD Device 14cc, Monitor: DELL P2415Q, Network: Aquantia AQC113C NBase-T/IEEE + Realtek RTL8125 2.5GbE + Intel Wi-Fi 6E OS: Ubuntu 24.04, Kernel: 6.8.0-49-generic (x86_64), Desktop: GNOME Shell 46.0, Display Server: X Server + Wayland, OpenGL: 4.6 Mesa 24.0.9-0ubuntu0.2 (LLVM 17.0.6 DRM 3.57), Compiler: GCC 13.2.0, File-System: ext4, Screen Resolution: 1920x1200 RELION 5.0 Test: Basic - Device: CPU Seconds < Lower Is Better a . 411.76 |================================================================== b . 366.92 |=========================================================== c . 401.05 |================================================================ d . 418.27 |=================================================================== Llamafile 0.8.16 Model: wizardcoder-python-34b-v1.0.Q6_K - Test: Text Generation 128 Tokens Per Second > Higher Is Better a . 4.45 |================================================================ b . 4.77 |===================================================================== c . 4.80 |===================================================================== d . 4.78 |===================================================================== Llamafile 0.8.16 Model: wizardcoder-python-34b-v1.0.Q6_K - Test: Prompt Processing 2048 Tokens Per Second > Higher Is Better a . 12288 |==================================================================== b . 12288 |==================================================================== c . 12288 |==================================================================== d . 12288 |==================================================================== Llamafile 0.8.16 Model: mistral-7b-instruct-v0.2.Q5_K_M - Test: Prompt Processing 2048 Tokens Per Second > Higher Is Better a . 32768 |==================================================================== b . 32768 |==================================================================== c . 32768 |==================================================================== d . 32768 |==================================================================== Llamafile 0.8.16 Model: mistral-7b-instruct-v0.2.Q5_K_M - Test: Text Generation 128 Tokens Per Second > Higher Is Better a . 23.42 |=================================================================== b . 23.47 |=================================================================== c . 23.64 |==================================================================== d . 23.68 |==================================================================== Llamafile 0.8.16 Model: wizardcoder-python-34b-v1.0.Q6_K - Test: Prompt Processing 1024 Tokens Per Second > Higher Is Better a . 6144 |===================================================================== b . 6144 |===================================================================== c . 6144 |===================================================================== d . 6144 |===================================================================== Llamafile 0.8.16 Model: Llama-3.2-3B-Instruct.Q6_K - Test: Text Generation 128 Tokens Per Second > Higher Is Better a . 42.08 |==================================================================== b . 42.01 |==================================================================== c . 41.61 |=================================================================== d . 41.62 |=================================================================== Llamafile 0.8.16 Model: wizardcoder-python-34b-v1.0.Q6_K - Test: Text Generation 16 Tokens Per Second > Higher Is Better a . 4.66 |=================================================================== b . 3.70 |===================================================== c . 4.77 |===================================================================== d . 4.78 |===================================================================== Llamafile 0.8.16 Model: Llama-3.2-3B-Instruct.Q6_K - Test: Prompt Processing 2048 Tokens Per Second > Higher Is Better a . 32768 |==================================================================== b . 32768 |==================================================================== c . 32768 |==================================================================== d . 32768 |==================================================================== Llamafile 0.8.16 Model: mistral-7b-instruct-v0.2.Q5_K_M - Test: Prompt Processing 1024 Tokens Per Second > Higher Is Better a . 16384 |==================================================================== b . 16384 |==================================================================== c . 16384 |==================================================================== d . 16384 |==================================================================== Llamafile 0.8.16 Model: TinyLlama-1.1B-Chat-v1.0.BF16 - Test: Text Generation 128 Tokens Per Second > Higher Is Better a . 57.43 |==================================================================== b . 56.58 |=================================================================== c . 56.47 |=================================================================== d . 56.97 |=================================================================== Llamafile 0.8.16 Model: wizardcoder-python-34b-v1.0.Q6_K - Test: Prompt Processing 512 Tokens Per Second > Higher Is Better a . 3072 |===================================================================== b . 3072 |===================================================================== c . 3072 |===================================================================== d . 3072 |===================================================================== Llamafile 0.8.16 Model: mistral-7b-instruct-v0.2.Q5_K_M - Test: Text Generation 16 Tokens Per Second > Higher Is Better a . 23.17 |=================================================================== b . 21.34 |============================================================== c . 23.47 |==================================================================== d . 23.38 |==================================================================== Llamafile 0.8.16 Model: TinyLlama-1.1B-Chat-v1.0.BF16 - Test: Prompt Processing 2048 Tokens Per Second > Higher Is Better a . 32768 |==================================================================== b . 32768 |==================================================================== c . 32768 |==================================================================== d . 32768 |==================================================================== Llamafile 0.8.16 Model: mistral-7b-instruct-v0.2.Q5_K_M - Test: Prompt Processing 512 Tokens Per Second > Higher Is Better a . 8192 |===================================================================== b . 8192 |===================================================================== c . 8192 |===================================================================== d . 8192 |===================================================================== Llamafile 0.8.16 Model: Llama-3.2-3B-Instruct.Q6_K - Test: Prompt Processing 1024 Tokens Per Second > Higher Is Better a . 16384 |==================================================================== b . 16384 |==================================================================== c . 16384 |==================================================================== d . 16384 |==================================================================== Llamafile 0.8.16 Model: wizardcoder-python-34b-v1.0.Q6_K - Test: Prompt Processing 256 Tokens Per Second > Higher Is Better a . 1536 |===================================================================== b . 1536 |===================================================================== c . 1536 |===================================================================== d . 1536 |===================================================================== Llamafile 0.8.16 Model: TinyLlama-1.1B-Chat-v1.0.BF16 - Test: Prompt Processing 1024 Tokens Per Second > Higher Is Better a . 16384 |==================================================================== b . 16384 |==================================================================== c . 16384 |==================================================================== d . 16384 |==================================================================== Llamafile 0.8.16 Model: mistral-7b-instruct-v0.2.Q5_K_M - Test: Prompt Processing 256 Tokens Per Second > Higher Is Better a . 4096 |===================================================================== b . 4096 |===================================================================== c . 4096 |===================================================================== d . 4096 |===================================================================== Llamafile 0.8.16 Model: Llama-3.2-3B-Instruct.Q6_K - Test: Prompt Processing 512 Tokens Per Second > Higher Is Better a . 8192 |===================================================================== b . 8192 |===================================================================== c . 8192 |===================================================================== d . 8192 |===================================================================== Llamafile 0.8.16 Model: Llama-3.2-3B-Instruct.Q6_K - Test: Text Generation 16 Tokens Per Second > Higher Is Better a . 41.72 |=================================================================== b . 37.23 |============================================================ c . 41.92 |==================================================================== d . 42.21 |==================================================================== Llamafile 0.8.16 Model: TinyLlama-1.1B-Chat-v1.0.BF16 - Test: Text Generation 16 Tokens Per Second > Higher Is Better a . 57.19 |==================================================================== b . 49.89 |=========================================================== c . 56.40 |=================================================================== d . 56.72 |=================================================================== Llamafile 0.8.16 Model: Llama-3.2-3B-Instruct.Q6_K - Test: Prompt Processing 256 Tokens Per Second > Higher Is Better a . 4096 |===================================================================== b . 4096 |===================================================================== c . 4096 |===================================================================== d . 4096 |===================================================================== Llamafile 0.8.16 Model: TinyLlama-1.1B-Chat-v1.0.BF16 - Test: Prompt Processing 512 Tokens Per Second > Higher Is Better a . 8192 |===================================================================== b . 8192 |===================================================================== c . 8192 |===================================================================== d . 8192 |===================================================================== Llamafile 0.8.16 Model: TinyLlama-1.1B-Chat-v1.0.BF16 - Test: Prompt Processing 256 Tokens Per Second > Higher Is Better a . 4096 |===================================================================== b . 4096 |===================================================================== c . 4096 |===================================================================== d . 4096 |=====================================================================