AMD Ryzen Threadripper 7980X 64-Cores testing with a System76 Thelio Major (FA Z5 BIOS) and NAVI31 45GB on Pop 22.04 via the Phoronix Test Suite.
Compare your own system(s) to this result file with the
Phoronix Test Suite by running the command:
phoronix-test-suite benchmark 2401195-PTS-LLAMAFIL46
llamafile thelio,
"Llamafile 0.6 - Test: mistral-7b-instruct-v0.2.Q8_0 - Acceleration: CPU",
Higher Results Are Better
"a",13.07,13.04,13.1
"b",13.13,13,13.1
"c",13.06,13.04,13
"d",13.08,13.02,13.02
"e",13.06,13.05,13.04
"Llamafile 0.6 - Test: wizardcoder-python-34b-v1.0.Q6_K - Acceleration: CPU",
Higher Results Are Better
"a",4.41,4.4,4.4
"b",4.41,4.4,4.4
"c",4.41,4.41,4.41
"d",4.42,4.4,4.41
"e",4.41,4.39,4.39
"Llamafile 0.6 - Test: llava-v1.5-7b-q4 - Acceleration: CPU",
Higher Results Are Better
"a",21.52,21.23,21.15
"b",21.59,21.35,21.18
"c",21.55,21.32,21.36
"d",21.44,21.31,21.22
"e",21.55,21.49,21.31