llamafile 0.7
AMD Ryzen 9 7950X 16-Core testing with a ASUS ROG STRIX X670E-E GAMING WIFI (1905 BIOS) and NVIDIA GeForce RTX 3080 10GB on Ubuntu 23.10 via the Phoronix Test Suite.


a: 

	Processor: AMD Ryzen 9 7950X 16-Core @ 5.88GHz (16 Cores / 32 Threads), Motherboard: ASUS ROG STRIX X670E-E GAMING WIFI (1905 BIOS), Chipset: AMD Device 14d8, Memory: 2 x 16GB DRAM-6000MT/s G Skill F5-6000J3038F16G, Disk: 2000GB Samsung SSD 980 PRO 2TB, Graphics: NVIDIA GeForce RTX 3080 10GB, Audio: NVIDIA GA102 HD Audio, Monitor: DELL U2723QE, Network: Intel I225-V + Intel Wi-Fi 6 AX210/AX211/AX411

	OS: Ubuntu 23.10, Kernel: 6.7.0-060700-generic (x86_64), Desktop: GNOME Shell 45.2, Display Server: X Server 1.21.1.7, Display Driver: NVIDIA 550.54.14, OpenGL: 4.6.0, OpenCL: OpenCL 3.0 CUDA 12.4.89, Compiler: GCC 13.2.0, File-System: ext4, Screen Resolution: 3840x2160

b: 

	Processor: AMD Ryzen 9 7950X 16-Core @ 5.88GHz (16 Cores / 32 Threads), Motherboard: ASUS ROG STRIX X670E-E GAMING WIFI (1905 BIOS), Chipset: AMD Device 14d8, Memory: 2 x 16GB DRAM-6000MT/s G Skill F5-6000J3038F16G, Disk: 2000GB Samsung SSD 980 PRO 2TB, Graphics: NVIDIA GeForce RTX 3080 10GB, Audio: NVIDIA GA102 HD Audio, Monitor: DELL U2723QE, Network: Intel I225-V + Intel Wi-Fi 6 AX210/AX211/AX411

	OS: Ubuntu 23.10, Kernel: 6.7.0-060700-generic (x86_64), Desktop: GNOME Shell 45.2, Display Server: X Server 1.21.1.7, Display Driver: NVIDIA 550.54.14, OpenGL: 4.6.0, OpenCL: OpenCL 3.0 CUDA 12.4.89, Compiler: GCC 13.2.0, File-System: ext4, Screen Resolution: 3840x2160

c: 

	Processor: AMD Ryzen 9 7950X 16-Core @ 5.88GHz (16 Cores / 32 Threads), Motherboard: ASUS ROG STRIX X670E-E GAMING WIFI (1905 BIOS), Chipset: AMD Device 14d8, Memory: 2 x 16GB DRAM-6000MT/s G Skill F5-6000J3038F16G, Disk: 2000GB Samsung SSD 980 PRO 2TB, Graphics: NVIDIA GeForce RTX 3080 10GB, Audio: NVIDIA GA102 HD Audio, Monitor: DELL U2723QE, Network: Intel I225-V + Intel Wi-Fi 6 AX210/AX211/AX411

	OS: Ubuntu 23.10, Kernel: 6.7.0-060700-generic (x86_64), Desktop: GNOME Shell 45.2, Display Server: X Server 1.21.1.7, Display Driver: NVIDIA 550.54.14, OpenGL: 4.6.0, OpenCL: OpenCL 3.0 CUDA 12.4.89, Compiler: GCC 13.2.0, File-System: ext4, Screen Resolution: 3840x2160

d: 

	Processor: AMD Ryzen 9 7950X 16-Core @ 5.88GHz (16 Cores / 32 Threads), Motherboard: ASUS ROG STRIX X670E-E GAMING WIFI (1905 BIOS), Chipset: AMD Device 14d8, Memory: 2 x 16GB DRAM-6000MT/s G Skill F5-6000J3038F16G, Disk: 2000GB Samsung SSD 980 PRO 2TB, Graphics: NVIDIA GeForce RTX 3080 10GB, Audio: NVIDIA GA102 HD Audio, Monitor: DELL U2723QE, Network: Intel I225-V + Intel Wi-Fi 6 AX210/AX211/AX411

	OS: Ubuntu 23.10, Kernel: 6.7.0-060700-generic (x86_64), Desktop: GNOME Shell 45.2, Display Server: X Server 1.21.1.7, Display Driver: NVIDIA 550.54.14, OpenGL: 4.6.0, OpenCL: OpenCL 3.0 CUDA 12.4.89, Compiler: GCC 13.2.0, File-System: ext4, Screen Resolution: 3840x2160


Llamafile 0.7
Test: mistral-7b-instruct-v0.2.Q8_0 - Acceleration: CPU
Tokens Per Second > Higher Is Better
a . 8.29 |=====================================================================
b . 8.27 |=====================================================================
c . 8.26 |=====================================================================
d . 8.25 |=====================================================================


Llamafile 0.7
Test: llava-v1.5-7b-q4 - Acceleration: CPU
Tokens Per Second > Higher Is Better
a . 12.93 |====================================================================
b . 12.95 |====================================================================
c . 12.99 |====================================================================
d . 12.95 |====================================================================


Llamafile 0.7
Test: wizardcoder-python-34b-v1.0.Q6_K - Acceleration: CPU
Tokens Per Second > Higher Is Better
a . 2.35 |=====================================================================
b . 2.35 |=====================================================================
c . 2.34 |=====================================================================
d . 2.35 |=====================================================================