2 x Intel Xeon Max 9480 testing with a Supermicro SYS-221H-TNR X13DEM v1.10 (1.3 BIOS) and ASPEED on Ubuntu 23.10 via the Phoronix Test Suite.
Compare your own system(s) to this result file with the
Phoronix Test Suite by running the command:
phoronix-test-suite benchmark 2404031-NE-XEONMAXLL89
xeon max llamafile
2 x Intel Xeon Max 9480 testing with a Supermicro SYS-221H-TNR X13DEM v1.10 (1.3 BIOS) and ASPEED on Ubuntu 23.10 via the Phoronix Test Suite.
a:
Processor: 2 x Intel Xeon Max 9480 @ 3.50GHz (112 Cores / 224 Threads), Motherboard: Supermicro SYS-221H-TNR X13DEM v1.10 (1.3 BIOS), Chipset: Intel Device 1bce, Memory: 512GB, Disk: 7682GB INTEL SSDPF2KX076TZ, Graphics: ASPEED, Network: 2 x Broadcom BCM57508 NetXtreme-E 10Gb/25Gb/40Gb/50Gb/100Gb/200Gb
OS: Ubuntu 23.10, Kernel: 6.9.0-060900rc2-generic (x86_64), Desktop: GNOME Shell 45.0, Display Server: X Server 1.21.1.7, Compiler: GCC 13.2.0, File-System: ext4, Screen Resolution: 1024x768
b:
Processor: 2 x Intel Xeon Max 9480 @ 3.50GHz (112 Cores / 224 Threads), Motherboard: Supermicro SYS-221H-TNR X13DEM v1.10 (1.3 BIOS), Chipset: Intel Device 1bce, Memory: 512GB, Disk: 7682GB INTEL SSDPF2KX076TZ, Graphics: ASPEED, Network: 2 x Broadcom BCM57508 NetXtreme-E 10Gb/25Gb/40Gb/50Gb/100Gb/200Gb
OS: Ubuntu 23.10, Kernel: 6.9.0-060900rc2-generic (x86_64), Desktop: GNOME Shell 45.0, Display Server: X Server 1.21.1.7, Compiler: GCC 13.2.0, File-System: ext4, Screen Resolution: 1024x768
Llamafile 0.7
Test: llava-v1.5-7b-q4 - Acceleration: CPU
Tokens Per Second > Higher Is Better
a . 1.51 |=============================================
b . 2.32 |=====================================================================
Llamafile 0.7
Test: mistral-7b-instruct-v0.2.Q8_0 - Acceleration: CPU
Tokens Per Second > Higher Is Better
a . 3.66 |====================================================================
b . 3.70 |=====================================================================
Llamafile 0.7
Test: wizardcoder-python-34b-v1.0.Q6_K - Acceleration: CPU
Tokens Per Second > Higher Is Better
a . 0.59 |=====================================================================