llamafile xeon
Intel Xeon E E-2488 testing with a Supermicro Super Server X13SCL-F v0123456789 (1.1 BIOS) and ASPEED on Ubuntu 24.04 via the Phoronix Test Suite.


a: 

	Processor: Intel Xeon E E-2488 @ 3.20GHz (8 Cores / 16 Threads), Motherboard: Supermicro Super Server X13SCL-F v0123456789 (1.1 BIOS), Chipset: Intel Raptor Lake-S PCH, Memory: 2 x 32GB DRAM-4800MT/s Micron MTC20C2085S1EC48BA1, Disk: 3201GB Micron_7450_MTFDKCC3T2TFS + 960GB Micron_7450_MTFDKBA960TFR, Graphics: ASPEED, Monitor: VE228, Network: 2 x Intel I210

	OS: Ubuntu 24.04, Kernel: 6.8.0-11-generic (x86_64), Desktop: GNOME Shell 45.3, Display Server: X Server 1.21.1.11, Compiler: GCC 13.2.0, File-System: ext4, Screen Resolution: 1920x1080

b: 

	Processor: Intel Xeon E E-2488 @ 3.20GHz (8 Cores / 16 Threads), Motherboard: Supermicro Super Server X13SCL-F v0123456789 (1.1 BIOS), Chipset: Intel Raptor Lake-S PCH, Memory: 2 x 32GB DRAM-4800MT/s Micron MTC20C2085S1EC48BA1, Disk: 3201GB Micron_7450_MTFDKCC3T2TFS + 960GB Micron_7450_MTFDKBA960TFR, Graphics: ASPEED, Monitor: VE228, Network: 2 x Intel I210

	OS: Ubuntu 24.04, Kernel: 6.8.0-11-generic (x86_64), Desktop: GNOME Shell 45.3, Display Server: X Server 1.21.1.11, Compiler: GCC 13.2.0, File-System: ext4, Screen Resolution: 1920x1080

c: 

	Processor: Intel Xeon E E-2488 @ 3.20GHz (8 Cores / 16 Threads), Motherboard: Supermicro Super Server X13SCL-F v0123456789 (1.1 BIOS), Chipset: Intel Raptor Lake-S PCH, Memory: 2 x 32GB DRAM-4800MT/s Micron MTC20C2085S1EC48BA1, Disk: 3201GB Micron_7450_MTFDKCC3T2TFS + 960GB Micron_7450_MTFDKBA960TFR, Graphics: ASPEED, Monitor: VE228, Network: 2 x Intel I210

	OS: Ubuntu 24.04, Kernel: 6.8.0-11-generic (x86_64), Desktop: GNOME Shell 45.3, Display Server: X Server 1.21.1.11, Compiler: GCC 13.2.0, File-System: ext4, Screen Resolution: 1920x1080

d: 

	Processor: Intel Xeon E E-2488 @ 3.20GHz (8 Cores / 16 Threads), Motherboard: Supermicro Super Server X13SCL-F v0123456789 (1.1 BIOS), Chipset: Intel Raptor Lake-S PCH, Memory: 2 x 32GB DRAM-4800MT/s Micron MTC20C2085S1EC48BA1, Disk: 3201GB Micron_7450_MTFDKCC3T2TFS + 960GB Micron_7450_MTFDKBA960TFR, Graphics: ASPEED, Monitor: VE228, Network: 2 x Intel I210

	OS: Ubuntu 24.04, Kernel: 6.8.0-11-generic (x86_64), Desktop: GNOME Shell 45.3, Display Server: X Server 1.21.1.11, Compiler: GCC 13.2.0, File-System: ext4, Screen Resolution: 1920x1080


Llamafile 0.7
Test: llava-v1.5-7b-q4 - Acceleration: CPU
Tokens Per Second > Higher Is Better
a . 13.28 |====================================================================
b . 13.18 |===================================================================
c . 13.24 |====================================================================
d . 13.22 |====================================================================


Llamafile 0.7
Test: mistral-7b-instruct-v0.2.Q8_0 - Acceleration: CPU
Tokens Per Second > Higher Is Better
a . 8.36 |=====================================================================
b . 8.33 |=====================================================================
c . 8.33 |=====================================================================
d . 8.32 |=====================================================================


Llamafile 0.7
Test: wizardcoder-python-34b-v1.0.Q6_K - Acceleration: CPU
Tokens Per Second > Higher Is Better
a . 2.17 |=====================================================================
b . 2.17 |=====================================================================
c . 2.17 |=====================================================================
d . 2.17 |=====================================================================