llamafile threadripper amd
AMD Ryzen Threadripper 7980X 64-Cores testing with a System76 Thelio Major (FA Z5 BIOS) and AMD Radeon Pro W7900 on Ubuntu 24.04 via the Phoronix Test Suite.


a: 

	Processor: AMD Ryzen Threadripper 7980X 64-Cores @ 7.79GHz (64 Cores / 128 Threads), Motherboard: System76 Thelio Major (FA Z5 BIOS), Chipset: AMD Device 14a4, Memory: 4 x 32GB DRAM-4800MT/s Micron MTC20F1045S1RC48BA2, Disk: 1000GB CT1000T700SSD5, Graphics: AMD Radeon Pro W7900, Audio: AMD Device 14cc, Monitor: DELL P2415Q, Network: Aquantia AQC113C NBase-T/IEEE + Realtek RTL8125 2.5GbE + Intel Wi-Fi 6E

	OS: Ubuntu 24.04, Kernel: 6.8.0-11-generic (x86_64), Desktop: GNOME Shell 45.3, Display Server: X Server + Wayland, File-System: ext4, Screen Resolution: 1920x1080

b: 

	Processor: AMD Ryzen Threadripper 7980X 64-Cores @ 7.79GHz (64 Cores / 128 Threads), Motherboard: System76 Thelio Major (FA Z5 BIOS), Chipset: AMD Device 14a4, Memory: 4 x 32GB DRAM-4800MT/s Micron MTC20F1045S1RC48BA2, Disk: 1000GB CT1000T700SSD5, Graphics: AMD Radeon Pro W7900, Audio: AMD Device 14cc, Monitor: DELL P2415Q, Network: Aquantia AQC113C NBase-T/IEEE + Realtek RTL8125 2.5GbE + Intel Wi-Fi 6E

	OS: Ubuntu 24.04, Kernel: 6.8.0-11-generic (x86_64), Desktop: GNOME Shell 45.3, Display Server: X Server + Wayland, File-System: ext4, Screen Resolution: 1920x1080

c: 

	Processor: AMD Ryzen Threadripper 7980X 64-Cores @ 7.79GHz (64 Cores / 128 Threads), Motherboard: System76 Thelio Major (FA Z5 BIOS), Chipset: AMD Device 14a4, Memory: 4 x 32GB DRAM-4800MT/s Micron MTC20F1045S1RC48BA2, Disk: 1000GB CT1000T700SSD5, Graphics: AMD Radeon Pro W7900, Audio: AMD Device 14cc, Monitor: DELL P2415Q, Network: Aquantia AQC113C NBase-T/IEEE + Realtek RTL8125 2.5GbE + Intel Wi-Fi 6E

	OS: Ubuntu 24.04, Kernel: 6.8.0-11-generic (x86_64), Desktop: GNOME Shell 45.3, Display Server: X Server + Wayland, File-System: ext4, Screen Resolution: 1920x1080

d: 

	Processor: AMD Ryzen Threadripper 7980X 64-Cores @ 7.79GHz (64 Cores / 128 Threads), Motherboard: System76 Thelio Major (FA Z5 BIOS), Chipset: AMD Device 14a4, Memory: 4 x 32GB DRAM-4800MT/s Micron MTC20F1045S1RC48BA2, Disk: 1000GB CT1000T700SSD5, Graphics: AMD Radeon Pro W7900, Audio: AMD Device 14cc, Monitor: DELL P2415Q, Network: Aquantia AQC113C NBase-T/IEEE + Realtek RTL8125 2.5GbE + Intel Wi-Fi 6E

	OS: Ubuntu 24.04, Kernel: 6.8.0-11-generic (x86_64), Desktop: GNOME Shell 45.3, Display Server: X Server + Wayland, File-System: ext4, Screen Resolution: 1920x1080


Llamafile 0.7
Test: llava-v1.5-7b-q4 - Acceleration: CPU
Tokens Per Second > Higher Is Better
a . 20.39 |====================================================================
b . 20.44 |====================================================================
c . 20.38 |====================================================================
d . 20.36 |====================================================================


Llamafile 0.7
Test: mistral-7b-instruct-v0.2.Q8_0 - Acceleration: CPU
Tokens Per Second > Higher Is Better
a . 13.97 |====================================================================
b . 13.95 |====================================================================
c . 13.97 |====================================================================
d . 13.95 |====================================================================


Llamafile 0.7
Test: wizardcoder-python-34b-v1.0.Q6_K - Acceleration: CPU
Tokens Per Second > Higher Is Better
a . 4.35 |=====================================================================
b . 4.35 |=====================================================================
c . 4.34 |=====================================================================
d . 4.35 |=====================================================================