AMD Ryzen Threadripper 3990X 64-Core testing with a Gigabyte TRX40 AORUS PRO WIFI (F6 BIOS) and AMD Radeon RX 5700 8GB on Pop 22.04 via the Phoronix Test Suite.
Compare your own system(s) to this result file with the
Phoronix Test Suite by running the command:
phoronix-test-suite benchmark 2401191-NE-LLAMAFILE37
llamafile ripper
AMD Ryzen Threadripper 3990X 64-Core testing with a Gigabyte TRX40 AORUS PRO WIFI (F6 BIOS) and AMD Radeon RX 5700 8GB on Pop 22.04 via the Phoronix Test Suite.
a:
Processor: AMD Ryzen Threadripper 3990X 64-Core @ 2.90GHz (64 Cores / 128 Threads), Motherboard: Gigabyte TRX40 AORUS PRO WIFI (F6 BIOS), Chipset: AMD Starship/Matisse, Memory: 4 x 32 GB DRAM-3000MT/s CMK64GX4M2D3000C16, Disk: Samsung SSD 970 EVO Plus 500GB, Graphics: AMD Radeon RX 5700 8GB (1750/875MHz), Audio: AMD Navi 10 HDMI Audio, Monitor: DELL P2415Q, Network: Intel I211 + Intel Wi-Fi 6 AX200
OS: Pop 22.04, Kernel: 6.6.6-76060606-generic (x86_64), Desktop: GNOME Shell 42.5, Display Server: X Server 1.21.1.4, OpenGL: 4.6 Mesa 23.3.2-1pop0~1704238321~22.04~36f1d0e (LLVM 15.0.7 DRM 3.54), Vulkan: 1.3.267, Compiler: GCC 11.4.0, File-System: ext4, Screen Resolution: 3840x2160
b:
Processor: AMD Ryzen Threadripper 3990X 64-Core @ 2.90GHz (64 Cores / 128 Threads), Motherboard: Gigabyte TRX40 AORUS PRO WIFI (F6 BIOS), Chipset: AMD Starship/Matisse, Memory: 4 x 32 GB DRAM-3000MT/s CMK64GX4M2D3000C16, Disk: Samsung SSD 970 EVO Plus 500GB, Graphics: AMD Radeon RX 5700 8GB (1750/875MHz), Audio: AMD Navi 10 HDMI Audio, Monitor: DELL P2415Q, Network: Intel I211 + Intel Wi-Fi 6 AX200
OS: Pop 22.04, Kernel: 6.6.6-76060606-generic (x86_64), Desktop: GNOME Shell 42.5, Display Server: X Server 1.21.1.4, OpenGL: 4.6 Mesa 23.3.2-1pop0~1704238321~22.04~36f1d0e (LLVM 15.0.7 DRM 3.54), Vulkan: 1.3.267, Compiler: GCC 11.4.0, File-System: ext4, Screen Resolution: 3840x2160
c:
Processor: AMD Ryzen Threadripper 3990X 64-Core @ 2.90GHz (64 Cores / 128 Threads), Motherboard: Gigabyte TRX40 AORUS PRO WIFI (F6 BIOS), Chipset: AMD Starship/Matisse, Memory: 4 x 32 GB DRAM-3000MT/s CMK64GX4M2D3000C16, Disk: Samsung SSD 970 EVO Plus 500GB, Graphics: AMD Radeon RX 5700 8GB (1750/875MHz), Audio: AMD Navi 10 HDMI Audio, Monitor: DELL P2415Q, Network: Intel I211 + Intel Wi-Fi 6 AX200
OS: Pop 22.04, Kernel: 6.6.6-76060606-generic (x86_64), Desktop: GNOME Shell 42.5, Display Server: X Server 1.21.1.4, OpenGL: 4.6 Mesa 23.3.2-1pop0~1704238321~22.04~36f1d0e (LLVM 15.0.7 DRM 3.54), Vulkan: 1.3.267, Compiler: GCC 11.4.0, File-System: ext4, Screen Resolution: 3840x2160
d:
Processor: AMD Ryzen Threadripper 3990X 64-Core @ 2.90GHz (64 Cores / 128 Threads), Motherboard: Gigabyte TRX40 AORUS PRO WIFI (F6 BIOS), Chipset: AMD Starship/Matisse, Memory: 4 x 32 GB DRAM-3000MT/s CMK64GX4M2D3000C16, Disk: Samsung SSD 970 EVO Plus 500GB, Graphics: AMD Radeon RX 5700 8GB (1750/875MHz), Audio: AMD Navi 10 HDMI Audio, Monitor: DELL P2415Q, Network: Intel I211 + Intel Wi-Fi 6 AX200
OS: Pop 22.04, Kernel: 6.6.6-76060606-generic (x86_64), Desktop: GNOME Shell 42.5, Display Server: X Server 1.21.1.4, OpenGL: 4.6 Mesa 23.3.2-1pop0~1704238321~22.04~36f1d0e (LLVM 15.0.7 DRM 3.54), Vulkan: 1.3.267, Compiler: GCC 11.4.0, File-System: ext4, Screen Resolution: 3840x2160
Llamafile 0.6
Test: llava-v1.5-7b-q4 - Acceleration: CPU
Tokens Per Second > Higher Is Better
a . 11.81 |====================================================================
b . 11.64 |===================================================================
c . 11.57 |===================================================================
d . 11.57 |===================================================================
Llamafile 0.6
Test: mistral-7b-instruct-v0.2.Q8_0 - Acceleration: CPU
Tokens Per Second > Higher Is Better
a . 7.65 |====================================================================
b . 7.71 |=====================================================================
c . 7.68 |=====================================================================
d . 7.65 |====================================================================
Llamafile 0.6
Test: wizardcoder-python-34b-v1.0.Q6_K - Acceleration: CPU
Tokens Per Second > Higher Is Better
a . 2.41 |=====================================================================
b . 2.40 |=====================================================================
c . 2.41 |=====================================================================
d . 2.40 |=====================================================================