AMD Ryzen Threadripper PRO 7995WX 96-Cores testing with a HP Z6 G5 A Workstation 8B24 (U65 Ver. 01.01.04 BIOS) and NVIDIA RTX A4000 16GB on Ubuntu 23.10 via the Phoronix Test Suite.
Compare your own system(s) to this result file with the
Phoronix Test Suite by running the command:
phoronix-test-suite benchmark 2401223-PTS-KKK7127595
kkk,
"Llamafile 0.6 - Test: mistral-7b-instruct-v0.2.Q8_0 - Acceleration: CPU",
Higher Results Are Better
"a",17.96,17.82,17.89
"b",18.05,18.07,17.91
"c",18.02,17.91,17.95
"d",18.05,17.94,18.06
"e",17.88,18.04,18.08
"Llamafile 0.6 - Test: wizardcoder-python-34b-v1.0.Q6_K - Acceleration: CPU",
Higher Results Are Better
"a",7.67,7.64,7.67
"b",7.68,7.62,7.66
"c",7.66,7.63,7.6
"d",7.66,7.6,7.62
"e",7.68,7.65,7.64
"LZ4 Compression 1.9.4 - Compression Level: 9 - Decompression Speed",
Higher Results Are Better
"a",5703.1,5660.5,5682.2
"b",5574,5655,5541.5
"c",5546.6,5650.5,5545.1
"d",5695.1,5577.3,5587.5
"e",5715.7,5645.7,5629.7
"LZ4 Compression 1.9.4 - Compression Level: 9 - Compression Speed",
Higher Results Are Better
"a",48.25,47.9,48.02
"b",47.35,47.87,47.03
"c",47.21,47.66,47.08
"d",48.16,47.4,47.2
"e",48.31,47.94,47.65
"LZ4 Compression 1.9.4 - Compression Level: 1 - Decompression Speed",
Higher Results Are Better
"a",6065.6,6049.6,6043.5
"b",6000.7,5992.3,6023.1
"c",5931.5,6071.4,6041.1
"d",6070.9,5865.8,5914.5
"e",5920.8,5928.6,5943.5
"LZ4 Compression 1.9.4 - Compression Level: 1 - Compression Speed",
Higher Results Are Better
"a",958.19,954.01,950.62
"b",938.99,938.59,947.7
"c",932.44,950.67,953.7
"d",957.26,924.86,932.83
"e",930.39,934.63,938.45
"LZ4 Compression 1.9.4 - Compression Level: 3 - Decompression Speed",
Higher Results Are Better
"a",5415.6,5376.5,5404.1
"b",5315.7,5549.5,5323.7
"c",5303.7,5330.6,5423.3
"d",5488.8,5469.9,5360.4
"e",5493.3,5357.8,5379.3
"LZ4 Compression 1.9.4 - Compression Level: 3 - Compression Speed",
Higher Results Are Better
"a",144.52,143.63,144.16
"b",142.01,147.24,141.93
"c",141.9,142.34,144.66
"d",146.55,145.82,142.63
"e",147.12,142.89,143.46
"Llamafile 0.6 - Test: llava-v1.5-7b-q4 - Acceleration: CPU",
Higher Results Are Better
"a",27.91,27.85,27.54
"b",28.16,27.82,27.64
"c",28.21,27.82,27.71
"d",28.08,27.95,27.58
"e",27.91,27.82,27.28