llamafile sienna
AMD EPYC 8534P 64-Core testing with a AMD Cinnabar (RCB1009C BIOS) and ASPEED on Ubuntu 23.10 via the Phoronix Test Suite.


a: 

	Processor: AMD EPYC 8534P 64-Core @ 2.30GHz (64 Cores / 128 Threads), Motherboard: AMD Cinnabar (RCB1009C BIOS), Chipset: AMD Device 14a4, Memory: 6 x 32GB DRAM-4800MT/s Samsung M321R4GA0BB0-CQKMG, Disk: 3201GB Micron_7450_MTFDKCB3T2TFS + 2 x 1920GB KIOXIA KCD8XPUG1T92, Graphics: ASPEED, Network: 2 x Broadcom NetXtreme BCM5720 PCIe

	OS: Ubuntu 23.10, Kernel: 6.8.1-060801-generic (x86_64), Desktop: GNOME Shell 45.2, Display Server: X Server 1.21.1.7, Compiler: GCC 13.2.0, File-System: ext4, Screen Resolution: 1920x1200

b: 

	Processor: AMD EPYC 8534P 64-Core @ 2.30GHz (64 Cores / 128 Threads), Motherboard: AMD Cinnabar (RCB1009C BIOS), Chipset: AMD Device 14a4, Memory: 6 x 32GB DRAM-4800MT/s Samsung M321R4GA0BB0-CQKMG, Disk: 3201GB Micron_7450_MTFDKCB3T2TFS + 2 x 1920GB KIOXIA KCD8XPUG1T92, Graphics: ASPEED, Network: 2 x Broadcom NetXtreme BCM5720 PCIe

	OS: Ubuntu 23.10, Kernel: 6.8.1-060801-generic (x86_64), Desktop: GNOME Shell 45.2, Display Server: X Server 1.21.1.7, Compiler: GCC 13.2.0, File-System: ext4, Screen Resolution: 1920x1200

c: 

	Processor: AMD EPYC 8534P 64-Core @ 2.30GHz (64 Cores / 128 Threads), Motherboard: AMD Cinnabar (RCB1009C BIOS), Chipset: AMD Device 14a4, Memory: 6 x 32GB DRAM-4800MT/s Samsung M321R4GA0BB0-CQKMG, Disk: 3201GB Micron_7450_MTFDKCB3T2TFS + 2 x 1920GB KIOXIA KCD8XPUG1T92, Graphics: ASPEED, Network: 2 x Broadcom NetXtreme BCM5720 PCIe

	OS: Ubuntu 23.10, Kernel: 6.8.1-060801-generic (x86_64), Desktop: GNOME Shell 45.2, Display Server: X Server 1.21.1.7, Compiler: GCC 13.2.0, File-System: ext4, Screen Resolution: 1920x1200

d: 

	Processor: AMD EPYC 8534P 64-Core @ 2.30GHz (64 Cores / 128 Threads), Motherboard: AMD Cinnabar (RCB1009C BIOS), Chipset: AMD Device 14a4, Memory: 6 x 32GB DRAM-4800MT/s Samsung M321R4GA0BB0-CQKMG, Disk: 3201GB Micron_7450_MTFDKCB3T2TFS + 2 x 1920GB KIOXIA KCD8XPUG1T92, Graphics: ASPEED, Network: 2 x Broadcom NetXtreme BCM5720 PCIe

	OS: Ubuntu 23.10, Kernel: 6.8.1-060801-generic (x86_64), Desktop: GNOME Shell 45.2, Display Server: X Server 1.21.1.7, Compiler: GCC 13.2.0, File-System: ext4, Screen Resolution: 1920x1200


Llamafile 0.7
Test: llava-v1.5-7b-q4 - Acceleration: CPU
Tokens Per Second > Higher Is Better
a . 20.83 |====================================================================
b . 20.67 |===================================================================
c . 20.84 |====================================================================
d . 20.74 |====================================================================


Llamafile 0.7
Test: mistral-7b-instruct-v0.2.Q8_0 - Acceleration: CPU
Tokens Per Second > Higher Is Better
a . 16.00 |====================================================================
b . 15.99 |====================================================================
c . 16.01 |====================================================================
d . 15.93 |====================================================================


Llamafile 0.7
Test: wizardcoder-python-34b-v1.0.Q6_K - Acceleration: CPU
Tokens Per Second > Higher Is Better
a . 5.20 |=====================================================================
b . 5.18 |=====================================================================
c . 5.21 |=====================================================================
d . 5.21 |=====================================================================