llamafile ripper
AMD Ryzen Threadripper 3990X 64-Core testing with a Gigabyte TRX40 AORUS PRO WIFI (F6 BIOS) and AMD Radeon RX 5700 8GB on Pop 22.04 via the Phoronix Test Suite.


a: 

	Processor: AMD Ryzen Threadripper 3990X 64-Core @ 2.90GHz (64 Cores / 128 Threads), Motherboard: Gigabyte TRX40 AORUS PRO WIFI (F6 BIOS), Chipset: AMD Starship/Matisse, Memory: 4 x 32 GB DRAM-3000MT/s CMK64GX4M2D3000C16, Disk: Samsung SSD 970 EVO Plus 500GB, Graphics: AMD Radeon RX 5700 8GB (1750/875MHz), Audio: AMD Navi 10 HDMI Audio, Monitor: DELL P2415Q, Network: Intel I211 + Intel Wi-Fi 6 AX200

	OS: Pop 22.04, Kernel: 6.6.6-76060606-generic (x86_64), Desktop: GNOME Shell 42.5, Display Server: X Server 1.21.1.4, OpenGL: 4.6 Mesa 23.3.2-1pop0~1704238321~22.04~36f1d0e (LLVM 15.0.7 DRM 3.54), Vulkan: 1.3.267, Compiler: GCC 11.4.0, File-System: ext4, Screen Resolution: 3840x2160

b: 

	Processor: AMD Ryzen Threadripper 3990X 64-Core @ 2.90GHz (64 Cores / 128 Threads), Motherboard: Gigabyte TRX40 AORUS PRO WIFI (F6 BIOS), Chipset: AMD Starship/Matisse, Memory: 4 x 32 GB DRAM-3000MT/s CMK64GX4M2D3000C16, Disk: Samsung SSD 970 EVO Plus 500GB, Graphics: AMD Radeon RX 5700 8GB (1750/875MHz), Audio: AMD Navi 10 HDMI Audio, Monitor: DELL P2415Q, Network: Intel I211 + Intel Wi-Fi 6 AX200

	OS: Pop 22.04, Kernel: 6.6.6-76060606-generic (x86_64), Desktop: GNOME Shell 42.5, Display Server: X Server 1.21.1.4, OpenGL: 4.6 Mesa 23.3.2-1pop0~1704238321~22.04~36f1d0e (LLVM 15.0.7 DRM 3.54), Vulkan: 1.3.267, Compiler: GCC 11.4.0, File-System: ext4, Screen Resolution: 3840x2160

c: 

	Processor: AMD Ryzen Threadripper 3990X 64-Core @ 2.90GHz (64 Cores / 128 Threads), Motherboard: Gigabyte TRX40 AORUS PRO WIFI (F6 BIOS), Chipset: AMD Starship/Matisse, Memory: 4 x 32 GB DRAM-3000MT/s CMK64GX4M2D3000C16, Disk: Samsung SSD 970 EVO Plus 500GB, Graphics: AMD Radeon RX 5700 8GB (1750/875MHz), Audio: AMD Navi 10 HDMI Audio, Monitor: DELL P2415Q, Network: Intel I211 + Intel Wi-Fi 6 AX200

	OS: Pop 22.04, Kernel: 6.6.6-76060606-generic (x86_64), Desktop: GNOME Shell 42.5, Display Server: X Server 1.21.1.4, OpenGL: 4.6 Mesa 23.3.2-1pop0~1704238321~22.04~36f1d0e (LLVM 15.0.7 DRM 3.54), Vulkan: 1.3.267, Compiler: GCC 11.4.0, File-System: ext4, Screen Resolution: 3840x2160

d: 

	Processor: AMD Ryzen Threadripper 3990X 64-Core @ 2.90GHz (64 Cores / 128 Threads), Motherboard: Gigabyte TRX40 AORUS PRO WIFI (F6 BIOS), Chipset: AMD Starship/Matisse, Memory: 4 x 32 GB DRAM-3000MT/s CMK64GX4M2D3000C16, Disk: Samsung SSD 970 EVO Plus 500GB, Graphics: AMD Radeon RX 5700 8GB (1750/875MHz), Audio: AMD Navi 10 HDMI Audio, Monitor: DELL P2415Q, Network: Intel I211 + Intel Wi-Fi 6 AX200

	OS: Pop 22.04, Kernel: 6.6.6-76060606-generic (x86_64), Desktop: GNOME Shell 42.5, Display Server: X Server 1.21.1.4, OpenGL: 4.6 Mesa 23.3.2-1pop0~1704238321~22.04~36f1d0e (LLVM 15.0.7 DRM 3.54), Vulkan: 1.3.267, Compiler: GCC 11.4.0, File-System: ext4, Screen Resolution: 3840x2160


Llamafile 0.6
Test: llava-v1.5-7b-q4 - Acceleration: CPU
Tokens Per Second > Higher Is Better
a . 11.81 |====================================================================
b . 11.64 |===================================================================
c . 11.57 |===================================================================
d . 11.57 |===================================================================


Llamafile 0.6
Test: mistral-7b-instruct-v0.2.Q8_0 - Acceleration: CPU
Tokens Per Second > Higher Is Better
a . 7.65 |====================================================================
b . 7.71 |=====================================================================
c . 7.68 |=====================================================================
d . 7.65 |====================================================================


Llamafile 0.6
Test: wizardcoder-python-34b-v1.0.Q6_K - Acceleration: CPU
Tokens Per Second > Higher Is Better
a . 2.41 |=====================================================================
b . 2.40 |=====================================================================
c . 2.41 |=====================================================================
d . 2.40 |=====================================================================