llama tr
AMD Ryzen Threadripper 3990X 64-Core testing with a Gigabyte TRX40 AORUS PRO WIFI (F6 BIOS) and AMD Radeon RX 5700 8GB on Ubuntu 23.10 via the Phoronix Test Suite.


a: 

	Processor: AMD Ryzen Threadripper 3990X 64-Core @ 2.90GHz (64 Cores / 128 Threads), Motherboard: Gigabyte TRX40 AORUS PRO WIFI (F6 BIOS), Chipset: AMD Starship/Matisse, Memory: 128GB, Disk: Samsung SSD 970 EVO Plus 500GB, Graphics: AMD Radeon RX 5700 8GB (1750/875MHz), Audio: AMD Navi 10 HDMI Audio, Monitor: DELL P2415Q, Network: Intel I211 + Intel Wi-Fi 6 AX200

	OS: Ubuntu 23.10, Kernel: 6.5.0-14-generic (x86_64), Desktop: GNOME Shell 45.0, Display Server: X Server + Wayland, OpenGL: 4.6 Mesa 23.2.1-1ubuntu3 (LLVM 15.0.7 DRM 3.54), Compiler: GCC 13.2.0, File-System: ext4, Screen Resolution: 3840x2160

b: 

	Processor: AMD Ryzen Threadripper 3990X 64-Core @ 2.90GHz (64 Cores / 128 Threads), Motherboard: Gigabyte TRX40 AORUS PRO WIFI (F6 BIOS), Chipset: AMD Starship/Matisse, Memory: 128GB, Disk: Samsung SSD 970 EVO Plus 500GB, Graphics: AMD Radeon RX 5700 8GB (1750/875MHz), Audio: AMD Navi 10 HDMI Audio, Monitor: DELL P2415Q, Network: Intel I211 + Intel Wi-Fi 6 AX200

	OS: Ubuntu 23.10, Kernel: 6.5.0-14-generic (x86_64), Desktop: GNOME Shell 45.0, Display Server: X Server + Wayland, OpenGL: 4.6 Mesa 23.2.1-1ubuntu3 (LLVM 15.0.7 DRM 3.54), Compiler: GCC 13.2.0, File-System: ext4, Screen Resolution: 3840x2160

c: 

	Processor: AMD Ryzen Threadripper 3990X 64-Core @ 2.90GHz (64 Cores / 128 Threads), Motherboard: Gigabyte TRX40 AORUS PRO WIFI (F6 BIOS), Chipset: AMD Starship/Matisse, Memory: 128GB, Disk: Samsung SSD 970 EVO Plus 500GB, Graphics: AMD Radeon RX 5700 8GB (1750/875MHz), Audio: AMD Navi 10 HDMI Audio, Monitor: DELL P2415Q, Network: Intel I211 + Intel Wi-Fi 6 AX200

	OS: Ubuntu 23.10, Kernel: 6.5.0-14-generic (x86_64), Desktop: GNOME Shell 45.0, Display Server: X Server + Wayland, OpenGL: 4.6 Mesa 23.2.1-1ubuntu3 (LLVM 15.0.7 DRM 3.54), Compiler: GCC 13.2.0, File-System: ext4, Screen Resolution: 3840x2160

f: 

	Processor: AMD Ryzen Threadripper 3990X 64-Core @ 2.90GHz (64 Cores / 128 Threads), Motherboard: Gigabyte TRX40 AORUS PRO WIFI (F6 BIOS), Chipset: AMD Starship/Matisse, Memory: 128GB, Disk: Samsung SSD 970 EVO Plus 500GB, Graphics: AMD Radeon RX 5700 8GB (1750/875MHz), Audio: AMD Navi 10 HDMI Audio, Monitor: DELL P2415Q, Network: Intel I211 + Intel Wi-Fi 6 AX200

	OS: Ubuntu 23.10, Kernel: 6.5.0-14-generic (x86_64), Desktop: GNOME Shell 45.0, Display Server: X Server + Wayland, OpenGL: 4.6 Mesa 23.2.1-1ubuntu3 (LLVM 15.0.7 DRM 3.54), Compiler: GCC 13.2.0, File-System: ext4, Screen Resolution: 3840x2160


Llama.cpp b1808
Model: llama-2-70b-chat.Q5_0.gguf
Tokens Per Second > Higher Is Better
a . 1.45 |=====================================================================
b . 1.45 |=====================================================================
c . 1.45 |=====================================================================
f . 1.46 |=====================================================================


CacheBench
Test: Read
MB/s > Higher Is Better
a . 10914.54 |=================================================================
b . 10857.24 |================================================================
c . 10863.42 |================================================================
f . 10954.50 |=================================================================


CacheBench
Test: Write
MB/s > Higher Is Better
a . 63817.97 |=================================================================
b . 64184.07 |=================================================================
c . 63783.85 |=================================================================
f . 64095.09 |=================================================================


CacheBench
Test: Read / Modify / Write
MB/s > Higher Is Better
a . 115970.80 |===============================================================
b . 117095.82 |================================================================
c . 116206.00 |===============================================================
f . 117248.30 |================================================================


Llama.cpp b1808
Model: llama-2-13b.Q4_0.gguf
Tokens Per Second > Higher Is Better
a . 8.21 |=====================================================================
b . 8.01 |===================================================================
c . 8.02 |===================================================================
f . 8.00 |===================================================================


Llama.cpp b1808
Model: llama-2-7b.Q4_0.gguf
Tokens Per Second > Higher Is Better
a . 14.23 |=================================================================
b . 14.23 |=================================================================
c . 14.99 |====================================================================
f . 14.19 |================================================================