Intel Xeon E E-2488 testing with a Supermicro Super Server X13SCL-F v0123456789 (1.1 BIOS) and ASPEED on Ubuntu 24.04 via the Phoronix Test Suite.
Compare your own system(s) to this result file with the
Phoronix Test Suite by running the command:
phoronix-test-suite benchmark 2404035-NE-LLAMAFILE41 llamafile xeon - Phoronix Test Suite llamafile xeon Intel Xeon E E-2488 testing with a Supermicro Super Server X13SCL-F v0123456789 (1.1 BIOS) and ASPEED on Ubuntu 24.04 via the Phoronix Test Suite.
HTML result view exported from: https://openbenchmarking.org/result/2404035-NE-LLAMAFILE41&rdt&grr .
llamafile xeon Processor Motherboard Chipset Memory Disk Graphics Monitor Network OS Kernel Desktop Display Server Compiler File-System Screen Resolution a b c d Intel Xeon E E-2488 @ 3.20GHz (8 Cores / 16 Threads) Supermicro Super Server X13SCL-F v0123456789 (1.1 BIOS) Intel Raptor Lake-S PCH 2 x 32GB DRAM-4800MT/s Micron MTC20C2085S1EC48BA1 3201GB Micron_7450_MTFDKCC3T2TFS + 960GB Micron_7450_MTFDKBA960TFR ASPEED VE228 2 x Intel I210 Ubuntu 24.04 6.8.0-11-generic (x86_64) GNOME Shell 45.3 X Server 1.21.1.11 GCC 13.2.0 ext4 1920x1080 OpenBenchmarking.org Kernel Details - Transparent Huge Pages: madvise Processor Details - Scaling Governor: acpi-cpufreq performance (Boost: Enabled) - CPU Microcode: 0x122 - Thermald 2.5.6 Security Details - gather_data_sampling: Not affected + itlb_multihit: Not affected + l1tf: Not affected + mds: Not affected + meltdown: Not affected + mmio_stale_data: Not affected + retbleed: Not affected + spec_rstack_overflow: Not affected + spec_store_bypass: Mitigation of SSB disabled via prctl + spectre_v1: Mitigation of usercopy/swapgs barriers and __user pointer sanitization + spectre_v2: Mitigation of Enhanced / Automatic IBRS IBPB: conditional RSB filling PBRSB-eIBRS: SW sequence + srbds: Not affected + tsx_async_abort: Not affected
llamafile xeon llamafile: wizardcoder-python-34b-v1.0.Q6_K - CPU llamafile: mistral-7b-instruct-v0.2.Q8_0 - CPU llamafile: llava-v1.5-7b-q4 - CPU a b c d 2.17 8.36 13.28 2.17 8.33 13.18 2.17 8.33 13.24 2.17 8.32 13.22 OpenBenchmarking.org
Llamafile Test: wizardcoder-python-34b-v1.0.Q6_K - Acceleration: CPU OpenBenchmarking.org Tokens Per Second, More Is Better Llamafile 0.7 Test: wizardcoder-python-34b-v1.0.Q6_K - Acceleration: CPU a b c d 0.4883 0.9766 1.4649 1.9532 2.4415 SE +/- 0.00, N = 3 SE +/- 0.00, N = 3 SE +/- 0.00, N = 3 SE +/- 0.00, N = 3 2.17 2.17 2.17 2.17
Llamafile Test: mistral-7b-instruct-v0.2.Q8_0 - Acceleration: CPU OpenBenchmarking.org Tokens Per Second, More Is Better Llamafile 0.7 Test: mistral-7b-instruct-v0.2.Q8_0 - Acceleration: CPU a b c d 2 4 6 8 10 SE +/- 0.01, N = 3 SE +/- 0.02, N = 3 SE +/- 0.01, N = 3 SE +/- 0.01, N = 3 8.36 8.33 8.33 8.32
Llamafile Test: llava-v1.5-7b-q4 - Acceleration: CPU OpenBenchmarking.org Tokens Per Second, More Is Better Llamafile 0.7 Test: llava-v1.5-7b-q4 - Acceleration: CPU a b c d 3 6 9 12 15 SE +/- 0.02, N = 3 SE +/- 0.04, N = 3 SE +/- 0.08, N = 3 SE +/- 0.03, N = 3 13.28 13.18 13.24 13.22
Phoronix Test Suite v10.8.4