llamafile xeon

Intel Xeon E E-2488 testing with a Supermicro Super Server X13SCL-F v0123456789 (1.1 BIOS) and ASPEED on Ubuntu 24.04 via the Phoronix Test Suite.

Compare your own system(s) to this result file with the Phoronix Test Suite by running the command: phoronix-test-suite benchmark 2404035-NE-LLAMAFILE41
Jump To Table - Results

View

Do Not Show Noisy Results
Do Not Show Results With Incomplete Data
Do Not Show Results With Little Change/Spread
List Notable Results

Statistics

Show Overall Harmonic Mean(s)
Show Overall Geometric Mean
Show Wins / Losses Counts (Pie Chart)
Normalize Results
Remove Outliers Before Calculating Averages

Graph Settings

Force Line Graphs Where Applicable
Convert To Scalar Where Applicable
Prefer Vertical Bar Graphs

Multi-Way Comparison

Condense Multi-Option Tests Into Single Result Graphs

Table

Show Detailed System Result Table

Run Management

Highlight
Result
Hide
Result
Result
Identifier
Performance Per
Dollar
Date
Run
  Test
  Duration
a
April 03
  17 Minutes
b
April 03
  17 Minutes
c
April 03
  17 Minutes
d
April 03
  18 Minutes
Invert Hiding All Results Option
  18 Minutes

Only show results where is faster than
Only show results matching title/arguments (delimit multiple options with a comma):
Do not show results matching title/arguments (delimit multiple options with a comma):

{ "title": "llamafile xeon", "last_modified": "2024-04-04 00:38:33", "description": "Intel Xeon E E-2488 testing with a Supermicro Super Server X13SCL-F v0123456789 (1.1 BIOS) and ASPEED on Ubuntu 24.04 via the Phoronix Test Suite.", "systems": { "a": { "identifier": "a", "hardware": { "Processor": "Intel Xeon E E-2488 @ 3.20GHz (8 Cores \/ 16 Threads)", "Motherboard": "Supermicro Super Server X13SCL-F v0123456789 (1.1 BIOS)", "Chipset": "Intel Raptor Lake-S PCH", "Memory": "2 x 32GB DRAM-4800MT\/s Micron MTC20C2085S1EC48BA1", "Disk": "3201GB Micron_7450_MTFDKCC3T2TFS + 960GB Micron_7450_MTFDKBA960TFR", "Graphics": "ASPEED", "Monitor": "VE228", "Network": "2 x Intel I210" }, "software": { "OS": "Ubuntu 24.04", "Kernel": "6.8.0-11-generic (x86_64)", "Desktop": "GNOME Shell 45.3", "Display Server": "X Server 1.21.1.11", "Compiler": "GCC 13.2.0", "File-System": "ext4", "Screen Resolution": "1920x1080" }, "user": "phoronix", "timestamp": "2024-04-03 21:05:35", "client_version": "10.8.4", "data": { "cpu-scaling-governor": "acpi-cpufreq performance (Boost: Enabled)", "cpu-microcode": "0x122", "cpu-thermald": "2.5.6", "kernel-extra-details": "Transparent Huge Pages: madvise", "security": "gather_data_sampling: Not affected + itlb_multihit: Not affected + l1tf: Not affected + mds: Not affected + meltdown: Not affected + mmio_stale_data: Not affected + retbleed: Not affected + spec_rstack_overflow: Not affected + spec_store_bypass: Mitigation of SSB disabled via prctl + spectre_v1: Mitigation of usercopy\/swapgs barriers and __user pointer sanitization + spectre_v2: Mitigation of Enhanced \/ Automatic IBRS IBPB: conditional RSB filling PBRSB-eIBRS: SW sequence + srbds: Not affected + tsx_async_abort: Not affected" } }, "b": { "identifier": "b", "hardware": { "Processor": "Intel Xeon E E-2488 @ 3.20GHz (8 Cores \/ 16 Threads)", "Motherboard": "Supermicro Super Server X13SCL-F v0123456789 (1.1 BIOS)", "Chipset": "Intel Raptor Lake-S PCH", "Memory": "2 x 32GB DRAM-4800MT\/s Micron MTC20C2085S1EC48BA1", "Disk": "3201GB Micron_7450_MTFDKCC3T2TFS + 960GB Micron_7450_MTFDKBA960TFR", "Graphics": "ASPEED", "Monitor": "VE228", "Network": "2 x Intel I210" }, "software": { "OS": "Ubuntu 24.04", "Kernel": "6.8.0-11-generic (x86_64)", "Desktop": "GNOME Shell 45.3", "Display Server": "X Server 1.21.1.11", "Compiler": "GCC 13.2.0", "File-System": "ext4", "Screen Resolution": "1920x1080" }, "user": "phoronix", "timestamp": "2024-04-03 21:49:26", "client_version": "10.8.4", "data": { "cpu-scaling-governor": "acpi-cpufreq performance (Boost: Enabled)", "cpu-microcode": "0x122", "cpu-thermald": "2.5.6", "kernel-extra-details": "Transparent Huge Pages: madvise", "security": "gather_data_sampling: Not affected + itlb_multihit: Not affected + l1tf: Not affected + mds: Not affected + meltdown: Not affected + mmio_stale_data: Not affected + retbleed: Not affected + spec_rstack_overflow: Not affected + spec_store_bypass: Mitigation of SSB disabled via prctl + spectre_v1: Mitigation of usercopy\/swapgs barriers and __user pointer sanitization + spectre_v2: Mitigation of Enhanced \/ Automatic IBRS IBPB: conditional RSB filling PBRSB-eIBRS: SW sequence + srbds: Not affected + tsx_async_abort: Not affected" } }, "c": { "identifier": "c", "hardware": { "Processor": "Intel Xeon E E-2488 @ 3.20GHz (8 Cores \/ 16 Threads)", "Motherboard": "Supermicro Super Server X13SCL-F v0123456789 (1.1 BIOS)", "Chipset": "Intel Raptor Lake-S PCH", "Memory": "2 x 32GB DRAM-4800MT\/s Micron MTC20C2085S1EC48BA1", "Disk": "3201GB Micron_7450_MTFDKCC3T2TFS + 960GB Micron_7450_MTFDKBA960TFR", "Graphics": "ASPEED", "Monitor": "VE228", "Network": "2 x Intel I210" }, "software": { "OS": "Ubuntu 24.04", "Kernel": "6.8.0-11-generic (x86_64)", "Desktop": "GNOME Shell 45.3", "Display Server": "X Server 1.21.1.11", "Compiler": "GCC 13.2.0", "File-System": "ext4", "Screen Resolution": "1920x1080" }, "user": "phoronix", "timestamp": "2024-04-03 23:36:02", "client_version": "10.8.4", "data": { "cpu-scaling-governor": "acpi-cpufreq performance (Boost: Enabled)", "cpu-microcode": "0x122", "cpu-thermald": "2.5.6", "kernel-extra-details": "Transparent Huge Pages: madvise", "security": "gather_data_sampling: Not affected + itlb_multihit: Not affected + l1tf: Not affected + mds: Not affected + meltdown: Not affected + mmio_stale_data: Not affected + retbleed: Not affected + spec_rstack_overflow: Not affected + spec_store_bypass: Mitigation of SSB disabled via prctl + spectre_v1: Mitigation of usercopy\/swapgs barriers and __user pointer sanitization + spectre_v2: Mitigation of Enhanced \/ Automatic IBRS IBPB: conditional RSB filling PBRSB-eIBRS: SW sequence + srbds: Not affected + tsx_async_abort: Not affected" } }, "d": { "identifier": "d", "hardware": { "Processor": "Intel Xeon E E-2488 @ 3.20GHz (8 Cores \/ 16 Threads)", "Motherboard": "Supermicro Super Server X13SCL-F v0123456789 (1.1 BIOS)", "Chipset": "Intel Raptor Lake-S PCH", "Memory": "2 x 32GB DRAM-4800MT\/s Micron MTC20C2085S1EC48BA1", "Disk": "3201GB Micron_7450_MTFDKCC3T2TFS + 960GB Micron_7450_MTFDKBA960TFR", "Graphics": "ASPEED", "Monitor": "VE228", "Network": "2 x Intel I210" }, "software": { "OS": "Ubuntu 24.04", "Kernel": "6.8.0-11-generic (x86_64)", "Desktop": "GNOME Shell 45.3", "Display Server": "X Server 1.21.1.11", "Compiler": "GCC 13.2.0", "File-System": "ext4", "Screen Resolution": "1920x1080" }, "user": "phoronix", "timestamp": "2024-04-03 23:55:39", "client_version": "10.8.4", "data": { "cpu-scaling-governor": "acpi-cpufreq performance (Boost: Enabled)", "cpu-microcode": "0x122", "cpu-thermald": "2.5.6", "kernel-extra-details": "Transparent Huge Pages: madvise", "security": "gather_data_sampling: Not affected + itlb_multihit: Not affected + l1tf: Not affected + mds: Not affected + meltdown: Not affected + mmio_stale_data: Not affected + retbleed: Not affected + spec_rstack_overflow: Not affected + spec_store_bypass: Mitigation of SSB disabled via prctl + spectre_v1: Mitigation of usercopy\/swapgs barriers and __user pointer sanitization + spectre_v2: Mitigation of Enhanced \/ Automatic IBRS IBPB: conditional RSB filling PBRSB-eIBRS: SW sequence + srbds: Not affected + tsx_async_abort: Not affected" } } }, "results": { "f9b6d7041c6abe4265ea58f78e1994894b1c0df8": { "identifier": "pts\/llamafile-1.1.0", "title": "Llamafile", "app_version": "0.7", "arguments": "run-llava --gpu DISABLE", "description": "Test: llava-v1.5-7b-q4 - Acceleration: CPU", "scale": "Tokens Per Second", "proportion": "HIB", "display_format": "BAR_GRAPH", "results": { "a": { "value": 13.2799999999999993605115378159098327159881591796875, "raw_values": [ 13.25, 13.32000000000000028421709430404007434844970703125, 13.2599999999999997868371792719699442386627197265625 ], "test_run_times": [ 31.1400000000000005684341886080801486968994140625, 27.530000000000001136868377216160297393798828125, 32.24000000000000198951966012828052043914794921875 ] }, "b": { "value": 13.17999999999999971578290569595992565155029296875, "raw_values": [ 13.2599999999999997868371792719699442386627197265625, 13.1699999999999999289457264239899814128875732421875, 13.1199999999999992184029906638897955417633056640625 ], "test_run_times": [ 29.620000000000000994759830064140260219573974609375, 32.46000000000000085265128291212022304534912109375, 31.910000000000000142108547152020037174224853515625 ] }, "c": { "value": 13.2400000000000002131628207280300557613372802734375, "raw_values": [ 13.199999999999999289457264239899814128875732421875, 13.1300000000000007815970093361102044582366943359375, 13.3900000000000005684341886080801486968994140625 ], "test_run_times": [ 28.239999999999998436805981327779591083526611328125, 29.17999999999999971578290569595992565155029296875, 26.160000000000000142108547152020037174224853515625 ] }, "d": { "value": 13.2200000000000006394884621840901672840118408203125, "raw_values": [ 13.1699999999999999289457264239899814128875732421875, 13.2599999999999997868371792719699442386627197265625, 13.230000000000000426325641456060111522674560546875 ], "test_run_times": [ 34.10000000000000142108547152020037174224853515625, 29.3599999999999994315658113919198513031005859375, 29.35000000000000142108547152020037174224853515625 ] } } }, "88400dbd1e8e7a5a3eef10e38cd834537b0aa992": { "identifier": "pts\/llamafile-1.1.0", "title": "Llamafile", "app_version": "0.7", "arguments": "run-mistral --gpu DISABLE", "description": "Test: mistral-7b-instruct-v0.2.Q8_0 - Acceleration: CPU", "scale": "Tokens Per Second", "proportion": "HIB", "display_format": "BAR_GRAPH", "results": { "a": { "value": 8.3599999999999994315658113919198513031005859375, "raw_values": [ 8.3699999999999992184029906638897955417633056640625, 8.3699999999999992184029906638897955417633056640625, 8.3300000000000000710542735760100185871124267578125 ], "test_run_times": [ 107.4200000000000017053025658242404460906982421875, 87.56999999999999317878973670303821563720703125, 131.580000000000012505552149377763271331787109375 ] }, "b": { "value": 8.3300000000000000710542735760100185871124267578125, "raw_values": [ 8.300000000000000710542735760100185871124267578125, 8.3499999999999996447286321199499070644378662109375, 8.339999999999999857891452847979962825775146484375 ], "test_run_times": [ 132.969999999999998863131622783839702606201171875, 99.159999999999996589394868351519107818603515625, 106.2999999999999971578290569595992565155029296875 ] }, "c": { "value": 8.3300000000000000710542735760100185871124267578125, "raw_values": [ 8.3100000000000004973799150320701301097869873046875, 8.3499999999999996447286321199499070644378662109375, 8.32000000000000028421709430404007434844970703125 ], "test_run_times": [ 129.400000000000005684341886080801486968994140625, 96.7399999999999948840923025272786617279052734375, 123.650000000000005684341886080801486968994140625 ] }, "d": { "value": 8.32000000000000028421709430404007434844970703125, "raw_values": [ 8.3100000000000004973799150320701301097869873046875, 8.3100000000000004973799150320701301097869873046875, 8.339999999999999857891452847979962825775146484375 ], "test_run_times": [ 134.18000000000000682121026329696178436279296875, 125.7399999999999948840923025272786617279052734375, 110.7699999999999960209606797434389591217041015625 ] } } }, "55efe9890511064c53225eab6117f9ccd4cbe0d6": { "identifier": "pts\/llamafile-1.1.0", "title": "Llamafile", "app_version": "0.7", "arguments": "run-wizardcoder --gpu DISABLE", "description": "Test: wizardcoder-python-34b-v1.0.Q6_K - Acceleration: CPU", "scale": "Tokens Per Second", "proportion": "HIB", "display_format": "BAR_GRAPH", "results": { "a": { "value": 2.1699999999999999289457264239899814128875732421875, "raw_values": [ 2.1699999999999999289457264239899814128875732421875, 2.160000000000000142108547152020037174224853515625, 2.1699999999999999289457264239899814128875732421875 ], "test_run_times": [ 207.490000000000009094947017729282379150390625, 194.539999999999992041921359486877918243408203125, 193.150000000000005684341886080801486968994140625 ] }, "b": { "value": 2.1699999999999999289457264239899814128875732421875, "raw_values": [ 2.1699999999999999289457264239899814128875732421875, 2.1699999999999999289457264239899814128875732421875, 2.1699999999999999289457264239899814128875732421875 ], "test_run_times": [ 194.3899999999999863575794734060764312744140625, 193.280000000000001136868377216160297393798828125, 193.490000000000009094947017729282379150390625 ] }, "c": { "value": 2.1699999999999999289457264239899814128875732421875, "raw_values": [ 2.1699999999999999289457264239899814128875732421875, 2.1699999999999999289457264239899814128875732421875, 2.1699999999999999289457264239899814128875732421875 ], "test_run_times": [ 194.81000000000000227373675443232059478759765625, 193.18000000000000682121026329696178436279296875, 193.25 ] }, "d": { "value": 2.1699999999999999289457264239899814128875732421875, "raw_values": [ 2.1699999999999999289457264239899814128875732421875, 2.1699999999999999289457264239899814128875732421875, 2.160000000000000142108547152020037174224853515625 ], "test_run_times": [ 196.169999999999987494447850622236728668212890625, 193.719999999999998863131622783839702606201171875, 194.56999999999999317878973670303821563720703125 ] } } } } }