ai ai ai
AMD Ryzen Threadripper 7980X 64-Cores testing with a System76 Thelio Major (FA Z5 BIOS) and AMD Radeon Pro W7900 45GB on Ubuntu 24.04 via the Phoronix Test Suite.


a: 

	Processor: AMD Ryzen Threadripper 7980X 64-Cores @ 7.79GHz (64 Cores / 128 Threads), Motherboard: System76 Thelio Major (FA Z5 BIOS), Chipset: AMD Device 14a4, Memory: 4 x 32GB DDR5-4800MT/s Micron MTC20F1045S1RC48BA2, Disk: 1000GB CT1000T700SSD5, Graphics: AMD Radeon Pro W7900 45GB, Audio: AMD Device 14cc, Monitor: DELL P2415Q, Network: Aquantia AQC113C NBase-T/IEEE + Realtek RTL8125 2.5GbE + Intel Wi-Fi 6E

	OS: Ubuntu 24.04, Kernel: 6.8.0-060800-generic (x86_64), Desktop: GNOME Shell 46.0, Display Server: X Server + Wayland, OpenGL: 4.6 Mesa 24.0.5-1ubuntu1 (LLVM 17.0.6 DRM 3.57), Compiler: GCC 13.2.0, File-System: ext4, Screen Resolution: 1920x1200

b: 

	Processor: AMD Ryzen Threadripper 7980X 64-Cores @ 7.79GHz (64 Cores / 128 Threads), Motherboard: System76 Thelio Major (FA Z5 BIOS), Chipset: AMD Device 14a4, Memory: 4 x 32GB DDR5-4800MT/s Micron MTC20F1045S1RC48BA2, Disk: 1000GB CT1000T700SSD5, Graphics: AMD Radeon Pro W7900 45GB, Audio: AMD Device 14cc, Monitor: DELL P2415Q, Network: Aquantia AQC113C NBase-T/IEEE + Realtek RTL8125 2.5GbE + Intel Wi-Fi 6E

	OS: Ubuntu 24.04, Kernel: 6.8.0-060800-generic (x86_64), Desktop: GNOME Shell 46.0, Display Server: X Server + Wayland, OpenGL: 4.6 Mesa 24.0.5-1ubuntu1 (LLVM 17.0.6 DRM 3.57), Compiler: GCC 13.2.0, File-System: ext4, Screen Resolution: 1920x1200

c: 

	Processor: AMD Ryzen Threadripper 7980X 64-Cores @ 7.79GHz (64 Cores / 128 Threads), Motherboard: System76 Thelio Major (FA Z5 BIOS), Chipset: AMD Device 14a4, Memory: 4 x 32GB DDR5-4800MT/s Micron MTC20F1045S1RC48BA2, Disk: 1000GB CT1000T700SSD5, Graphics: AMD Radeon Pro W7900 45GB, Audio: AMD Device 14cc, Monitor: DELL P2415Q, Network: Aquantia AQC113C NBase-T/IEEE + Realtek RTL8125 2.5GbE + Intel Wi-Fi 6E

	OS: Ubuntu 24.04, Kernel: 6.8.0-060800-generic (x86_64), Desktop: GNOME Shell 46.0, Display Server: X Server + Wayland, OpenGL: 4.6 Mesa 24.0.5-1ubuntu1 (LLVM 17.0.6 DRM 3.57), Compiler: GCC 13.2.0, File-System: ext4, Screen Resolution: 1920x1200

d: 

	Processor: AMD Ryzen Threadripper 7980X 64-Cores @ 7.79GHz (64 Cores / 128 Threads), Motherboard: System76 Thelio Major (FA Z5 BIOS), Chipset: AMD Device 14a4, Memory: 4 x 32GB DDR5-4800MT/s Micron MTC20F1045S1RC48BA2, Disk: 1000GB CT1000T700SSD5, Graphics: AMD Radeon Pro W7900 45GB, Audio: AMD Device 14cc, Monitor: DELL P2415Q, Network: Aquantia AQC113C NBase-T/IEEE + Realtek RTL8125 2.5GbE + Intel Wi-Fi 6E

	OS: Ubuntu 24.04, Kernel: 6.8.0-060800-generic (x86_64), Desktop: GNOME Shell 46.0, Display Server: X Server + Wayland, OpenGL: 4.6 Mesa 24.0.5-1ubuntu1 (LLVM 17.0.6 DRM 3.57), Compiler: GCC 13.2.0, File-System: ext4, Screen Resolution: 1920x1200

e: 

	Processor: AMD Ryzen Threadripper 7980X 64-Cores @ 7.79GHz (64 Cores / 128 Threads), Motherboard: System76 Thelio Major (FA Z5 BIOS), Chipset: AMD Device 14a4, Memory: 4 x 32GB DDR5-4800MT/s Micron MTC20F1045S1RC48BA2, Disk: 1000GB CT1000T700SSD5, Graphics: AMD Radeon Pro W7900 45GB, Audio: AMD Device 14cc, Monitor: DELL P2415Q, Network: Aquantia AQC113C NBase-T/IEEE + Realtek RTL8125 2.5GbE + Intel Wi-Fi 6E

	OS: Ubuntu 24.04, Kernel: 6.8.0-060800-generic (x86_64), Desktop: GNOME Shell 46.0, Display Server: X Server + Wayland, OpenGL: 4.6 Mesa 24.0.5-1ubuntu1 (LLVM 17.0.6 DRM 3.57), Compiler: GCC 13.2.0, File-System: ext4, Screen Resolution: 1920x1200


Llamafile 0.8.6
Test: TinyLlama-1.1B-Chat-v1.0.BF16 - Acceleration: CPU
Tokens Per Second > Higher Is Better
a . 52.94 |====================================================================
b . 50.74 |=================================================================
c . 52.41 |===================================================================
e . 52.94 |====================================================================


Whisper.cpp 1.6.2
Model: ggml-base.en - Input: 2016 State of the Union
Seconds < Lower Is Better
a . 118.87 |===================================================================
b . 118.65 |===================================================================
c . 118.26 |==================================================================
d . 118.41 |==================================================================
e . 119.42 |===================================================================


Whisper.cpp 1.6.2
Model: ggml-medium.en - Input: 2016 State of the Union
Seconds < Lower Is Better
a . 558.14 |===================================================================
b . 555.70 |==================================================================
c . 559.57 |===================================================================
d . 560.64 |===================================================================
e . 559.88 |===================================================================


Whisper.cpp 1.6.2
Model: ggml-small.en - Input: 2016 State of the Union
Seconds < Lower Is Better
a . 232.20 |==================================================================
b . 232.16 |==================================================================
c . 234.12 |===================================================================
d . 232.77 |===================================================================
e . 233.88 |===================================================================


Llamafile 0.8.6
Test: wizardcoder-python-34b-v1.0.Q6_K - Acceleration: CPU
Tokens Per Second > Higher Is Better
a . 4.02 |=====================================================================
b . 4.00 |=====================================================================
c . 4.00 |=====================================================================
e . 3.99 |====================================================================


Llamafile 0.8.6
Test: mistral-7b-instruct-v0.2.Q5_K_M - Acceleration: CPU
Tokens Per Second > Higher Is Better
a . 22.18 |====================================================================
b . 22.03 |====================================================================
c . 22.17 |====================================================================
e . 22.08 |====================================================================


Llama.cpp b3067
Model: Meta-Llama-3-8B-Instruct-Q8_0.gguf
Tokens Per Second > Higher Is Better
a . 13.09 |====================================================================
b . 13.10 |====================================================================
c . 13.14 |====================================================================
d . 13.09 |====================================================================
e . 13.08 |====================================================================


Llamafile 0.8.6
Test: Meta-Llama-3-8B-Instruct.F16 - Acceleration: CPU
Tokens Per Second > Higher Is Better
a . 8.65 |=====================================================================
e . 8.63 |=====================================================================


Llamafile 0.8.6
Test: llava-v1.6-mistral-7b.Q8_0 - Acceleration: CPU
Tokens Per Second > Higher Is Better