AMD EPYC 8534P 64-Core testing with a AMD Cinnabar (RCB1009C BIOS) and ASPEED on Ubuntu 23.10 via the Phoronix Test Suite.
Compare your own system(s) to this result file with the
Phoronix Test Suite by running the command:
phoronix-test-suite benchmark 2404045-NE-LLAMAFILE27 llamafile sienna - Phoronix Test Suite llamafile sienna AMD EPYC 8534P 64-Core testing with a AMD Cinnabar (RCB1009C BIOS) and ASPEED on Ubuntu 23.10 via the Phoronix Test Suite.
HTML result view exported from: https://openbenchmarking.org/result/2404045-NE-LLAMAFILE27&export=txt&grr&rdt&rro .
llamafile sienna Processor Motherboard Chipset Memory Disk Graphics Network OS Kernel Desktop Display Server Compiler File-System Screen Resolution a b c d AMD EPYC 8534P 64-Core @ 2.30GHz (64 Cores / 128 Threads) AMD Cinnabar (RCB1009C BIOS) AMD Device 14a4 6 x 32GB DRAM-4800MT/s Samsung M321R4GA0BB0-CQKMG 3201GB Micron_7450_MTFDKCB3T2TFS + 2 x 1920GB KIOXIA KCD8XPUG1T92 ASPEED 2 x Broadcom NetXtreme BCM5720 PCIe Ubuntu 23.10 6.8.1-060801-generic (x86_64) GNOME Shell 45.2 X Server 1.21.1.7 GCC 13.2.0 ext4 1920x1200 OpenBenchmarking.org Kernel Details - Transparent Huge Pages: madvise Processor Details - Scaling Governor: acpi-cpufreq schedutil (Boost: Enabled) - CPU Microcode: 0xaa00212 Security Details - gather_data_sampling: Not affected + itlb_multihit: Not affected + l1tf: Not affected + mds: Not affected + meltdown: Not affected + mmio_stale_data: Not affected + reg_file_data_sampling: Not affected + retbleed: Not affected + spec_rstack_overflow: Mitigation of Safe RET + spec_store_bypass: Mitigation of SSB disabled via prctl + spectre_v1: Mitigation of usercopy/swapgs barriers and __user pointer sanitization + spectre_v2: Mitigation of Enhanced / Automatic IBRS IBPB: conditional STIBP: always-on RSB filling PBRSB-eIBRS: Not affected + srbds: Not affected + tsx_async_abort: Not affected
llamafile sienna llamafile: wizardcoder-python-34b-v1.0.Q6_K - CPU llamafile: mistral-7b-instruct-v0.2.Q8_0 - CPU llamafile: llava-v1.5-7b-q4 - CPU a b c d 5.20 16.00 20.83 5.18 15.99 20.67 5.21 16.01 20.84 5.21 15.93 20.74 OpenBenchmarking.org
Llamafile Test: wizardcoder-python-34b-v1.0.Q6_K - Acceleration: CPU OpenBenchmarking.org Tokens Per Second, More Is Better Llamafile 0.7 Test: wizardcoder-python-34b-v1.0.Q6_K - Acceleration: CPU d c b a 1.1723 2.3446 3.5169 4.6892 5.8615 SE +/- 0.00, N = 3 SE +/- 0.01, N = 3 SE +/- 0.01, N = 3 SE +/- 0.01, N = 3 5.21 5.21 5.18 5.20
Llamafile Test: mistral-7b-instruct-v0.2.Q8_0 - Acceleration: CPU OpenBenchmarking.org Tokens Per Second, More Is Better Llamafile 0.7 Test: mistral-7b-instruct-v0.2.Q8_0 - Acceleration: CPU d c b a 4 8 12 16 20 SE +/- 0.04, N = 3 SE +/- 0.04, N = 3 SE +/- 0.02, N = 3 SE +/- 0.01, N = 3 15.93 16.01 15.99 16.00
Llamafile Test: llava-v1.5-7b-q4 - Acceleration: CPU OpenBenchmarking.org Tokens Per Second, More Is Better Llamafile 0.7 Test: llava-v1.5-7b-q4 - Acceleration: CPU d c b a 5 10 15 20 25 SE +/- 0.01, N = 3 SE +/- 0.08, N = 3 SE +/- 0.05, N = 3 SE +/- 0.06, N = 3 20.74 20.84 20.67 20.83
Phoronix Test Suite v10.8.4