llama

AMD Ryzen 7 9800X3D 8-Core testing with a ASRock X870E Taichi (3.12.AS02 BIOS) and AMD Radeon PRO W7500 8GB on Ubuntu 24.04 via the Phoronix Test Suite.

HTML result view exported from: https://openbenchmarking.org/result/2412052-PTS-LLAMA11353&grs&rdt.

llamaProcessorMotherboardChipsetMemoryDiskGraphicsAudioMonitorNetworkOSKernelDesktopDisplay ServerOpenGLCompilerFile-SystemScreen ResolutionabAMD Ryzen 7 9800X3D 8-Core - AMD Radeon PRO W7500 8GBcAMD Ryzen 7 9800X3D 8-Core @ 6.22GHz (8 Cores / 16 Threads)ASRock X870E Taichi (3.12.AS02 BIOS)AMD Device 14d82 x 16GB DDR5-6000MT/s F5-6000J2836G16GWestern Digital WD_BLACK SN850X 2000GBAMD Radeon PRO W7500 8GBAMD Navi 31 HDMI/DPDELL U2723QERealtek Device 8126 + MEDIATEK Device 0717Ubuntu 24.046.8.0-49-generic (x86_64)GNOME Shell 46.0X Server 1.21.1.11 + Wayland4.6 Mesa 24.2.0-devel (LLVM 18.1.7 DRM 3.58)GCC 13.2.0ext43840x2160OpenBenchmarking.orgKernel Details- Transparent Huge Pages: madviseProcessor Details- Scaling Governor: amd-pstate-epp powersave (EPP: balance_performance) - CPU Microcode: 0xb404023Security Details- gather_data_sampling: Not affected + itlb_multihit: Not affected + l1tf: Not affected + mds: Not affected + meltdown: Not affected + mmio_stale_data: Not affected + reg_file_data_sampling: Not affected + retbleed: Not affected + spec_rstack_overflow: Not affected + spec_store_bypass: Mitigation of SSB disabled via prctl + spectre_v1: Mitigation of usercopy/swapgs barriers and __user pointer sanitization + spectre_v2: Mitigation of Enhanced / Automatic IBRS; IBPB: conditional; STIBP: always-on; RSB filling; PBRSB-eIBRS: Not affected; BHI: Not affected + srbds: Not affected + tsx_async_abort: Not affected

llamallamafile: wizardcoder-python-34b-v1.0.Q6_K - Text Generation 16llamafile: TinyLlama-1.1B-Chat-v1.0.BF16 - Text Generation 16llamafile: Llama-3.2-3B-Instruct.Q6_K - Text Generation 16llamafile: mistral-7b-instruct-v0.2.Q5_K_M - Text Generation 16llamafile: TinyLlama-1.1B-Chat-v1.0.BF16 - Text Generation 128llamafile: mistral-7b-instruct-v0.2.Q5_K_M - Text Generation 128llamafile: Llama-3.2-3B-Instruct.Q6_K - Text Generation 128llamafile: wizardcoder-python-34b-v1.0.Q6_K - Prompt Processing 2048llamafile: wizardcoder-python-34b-v1.0.Q6_K - Prompt Processing 1024llamafile: wizardcoder-python-34b-v1.0.Q6_K - Prompt Processing 512llamafile: wizardcoder-python-34b-v1.0.Q6_K - Prompt Processing 256llamafile: mistral-7b-instruct-v0.2.Q5_K_M - Prompt Processing 2048llamafile: mistral-7b-instruct-v0.2.Q5_K_M - Prompt Processing 1024llamafile: mistral-7b-instruct-v0.2.Q5_K_M - Prompt Processing 512llamafile: mistral-7b-instruct-v0.2.Q5_K_M - Prompt Processing 256llamafile: wizardcoder-python-34b-v1.0.Q6_K - Text Generation 128llamafile: TinyLlama-1.1B-Chat-v1.0.BF16 - Prompt Processing 2048llamafile: TinyLlama-1.1B-Chat-v1.0.BF16 - Prompt Processing 1024llamafile: TinyLlama-1.1B-Chat-v1.0.BF16 - Prompt Processing 512llamafile: TinyLlama-1.1B-Chat-v1.0.BF16 - Prompt Processing 256llamafile: Llama-3.2-3B-Instruct.Q6_K - Prompt Processing 2048llamafile: Llama-3.2-3B-Instruct.Q6_K - Prompt Processing 1024llamafile: Llama-3.2-3B-Instruct.Q6_K - Prompt Processing 512llamafile: Llama-3.2-3B-Instruct.Q6_K - Prompt Processing 256abAMD Ryzen 7 9800X3D 8-Core - AMD Radeon PRO W7500 8GBc2.0927.5421.4711.1927.5711.3121.42122886144307215363276816384819240962.1327681638481924096327681638481924096226.2820.7810.9427.5111.321.42122886144307215363276816384819240962.13276816384819240963276816384819240961.9826.4220.7710.9427.5511.321.43122886144307215363276816384819240962.13276816384819240963276816384819240961.9626.5120.7910.9527.5211.321.43122886144307215363276816384819240962.1327681638481924096327681638481924096OpenBenchmarking.org

Llamafile

Model: wizardcoder-python-34b-v1.0.Q6_K - Test: Text Generation 16

OpenBenchmarking.orgTokens Per Second, More Is BetterLlamafile 0.8.16Model: wizardcoder-python-34b-v1.0.Q6_K - Test: Text Generation 16abAMD Ryzen 7 9800X3D 8-Core - AMD Radeon PRO W7500 8GBc0.47030.94061.41091.88122.3515SE +/- 0.02, N = 122.092.001.981.96

Llamafile

Model: TinyLlama-1.1B-Chat-v1.0.BF16 - Test: Text Generation 16

OpenBenchmarking.orgTokens Per Second, More Is BetterLlamafile 0.8.16Model: TinyLlama-1.1B-Chat-v1.0.BF16 - Test: Text Generation 16abAMD Ryzen 7 9800X3D 8-Core - AMD Radeon PRO W7500 8GBc612182430SE +/- 0.00, N = 327.5426.2826.4226.51

Llamafile

Model: Llama-3.2-3B-Instruct.Q6_K - Test: Text Generation 16

OpenBenchmarking.orgTokens Per Second, More Is BetterLlamafile 0.8.16Model: Llama-3.2-3B-Instruct.Q6_K - Test: Text Generation 16abAMD Ryzen 7 9800X3D 8-Core - AMD Radeon PRO W7500 8GBc510152025SE +/- 0.01, N = 321.4720.7820.7720.79

Llamafile

Model: mistral-7b-instruct-v0.2.Q5_K_M - Test: Text Generation 16

OpenBenchmarking.orgTokens Per Second, More Is BetterLlamafile 0.8.16Model: mistral-7b-instruct-v0.2.Q5_K_M - Test: Text Generation 16abAMD Ryzen 7 9800X3D 8-Core - AMD Radeon PRO W7500 8GBc3691215SE +/- 0.13, N = 311.1910.9410.9410.95

Llamafile

Model: TinyLlama-1.1B-Chat-v1.0.BF16 - Test: Text Generation 128

OpenBenchmarking.orgTokens Per Second, More Is BetterLlamafile 0.8.16Model: TinyLlama-1.1B-Chat-v1.0.BF16 - Test: Text Generation 128abAMD Ryzen 7 9800X3D 8-Core - AMD Radeon PRO W7500 8GBc612182430SE +/- 0.02, N = 327.5727.5127.5527.52

Llamafile

Model: mistral-7b-instruct-v0.2.Q5_K_M - Test: Text Generation 128

OpenBenchmarking.orgTokens Per Second, More Is BetterLlamafile 0.8.16Model: mistral-7b-instruct-v0.2.Q5_K_M - Test: Text Generation 128abAMD Ryzen 7 9800X3D 8-Core - AMD Radeon PRO W7500 8GBc3691215SE +/- 0.00, N = 311.3111.3011.3011.30

Llamafile

Model: Llama-3.2-3B-Instruct.Q6_K - Test: Text Generation 128

OpenBenchmarking.orgTokens Per Second, More Is BetterLlamafile 0.8.16Model: Llama-3.2-3B-Instruct.Q6_K - Test: Text Generation 128abAMD Ryzen 7 9800X3D 8-Core - AMD Radeon PRO W7500 8GBc510152025SE +/- 0.00, N = 321.4221.4221.4321.43

Llamafile

Model: wizardcoder-python-34b-v1.0.Q6_K - Test: Prompt Processing 2048

OpenBenchmarking.orgTokens Per Second, More Is BetterLlamafile 0.8.16Model: wizardcoder-python-34b-v1.0.Q6_K - Test: Prompt Processing 2048abAMD Ryzen 7 9800X3D 8-Core - AMD Radeon PRO W7500 8GBc3K6K9K12K15KSE +/- 0.00, N = 312288122881228812288

Llamafile

Model: wizardcoder-python-34b-v1.0.Q6_K - Test: Prompt Processing 1024

OpenBenchmarking.orgTokens Per Second, More Is BetterLlamafile 0.8.16Model: wizardcoder-python-34b-v1.0.Q6_K - Test: Prompt Processing 1024abAMD Ryzen 7 9800X3D 8-Core - AMD Radeon PRO W7500 8GBc13002600390052006500SE +/- 0.00, N = 36144614461446144

Llamafile

Model: wizardcoder-python-34b-v1.0.Q6_K - Test: Prompt Processing 512

OpenBenchmarking.orgTokens Per Second, More Is BetterLlamafile 0.8.16Model: wizardcoder-python-34b-v1.0.Q6_K - Test: Prompt Processing 512abAMD Ryzen 7 9800X3D 8-Core - AMD Radeon PRO W7500 8GBc7001400210028003500SE +/- 0.00, N = 33072307230723072

Llamafile

Model: wizardcoder-python-34b-v1.0.Q6_K - Test: Prompt Processing 256

OpenBenchmarking.orgTokens Per Second, More Is BetterLlamafile 0.8.16Model: wizardcoder-python-34b-v1.0.Q6_K - Test: Prompt Processing 256abAMD Ryzen 7 9800X3D 8-Core - AMD Radeon PRO W7500 8GBc30060090012001500SE +/- 0.00, N = 31536153615361536

Llamafile

Model: mistral-7b-instruct-v0.2.Q5_K_M - Test: Prompt Processing 2048

OpenBenchmarking.orgTokens Per Second, More Is BetterLlamafile 0.8.16Model: mistral-7b-instruct-v0.2.Q5_K_M - Test: Prompt Processing 2048abAMD Ryzen 7 9800X3D 8-Core - AMD Radeon PRO W7500 8GBc7K14K21K28K35KSE +/- 0.00, N = 332768327683276832768

Llamafile

Model: mistral-7b-instruct-v0.2.Q5_K_M - Test: Prompt Processing 1024

OpenBenchmarking.orgTokens Per Second, More Is BetterLlamafile 0.8.16Model: mistral-7b-instruct-v0.2.Q5_K_M - Test: Prompt Processing 1024abAMD Ryzen 7 9800X3D 8-Core - AMD Radeon PRO W7500 8GBc4K8K12K16K20KSE +/- 0.00, N = 316384163841638416384

Llamafile

Model: mistral-7b-instruct-v0.2.Q5_K_M - Test: Prompt Processing 512

OpenBenchmarking.orgTokens Per Second, More Is BetterLlamafile 0.8.16Model: mistral-7b-instruct-v0.2.Q5_K_M - Test: Prompt Processing 512abAMD Ryzen 7 9800X3D 8-Core - AMD Radeon PRO W7500 8GBc2K4K6K8K10KSE +/- 0.00, N = 38192819281928192

Llamafile

Model: mistral-7b-instruct-v0.2.Q5_K_M - Test: Prompt Processing 256

OpenBenchmarking.orgTokens Per Second, More Is BetterLlamafile 0.8.16Model: mistral-7b-instruct-v0.2.Q5_K_M - Test: Prompt Processing 256abAMD Ryzen 7 9800X3D 8-Core - AMD Radeon PRO W7500 8GBc9001800270036004500SE +/- 0.00, N = 34096409640964096

Llamafile

Model: wizardcoder-python-34b-v1.0.Q6_K - Test: Text Generation 128

OpenBenchmarking.orgTokens Per Second, More Is BetterLlamafile 0.8.16Model: wizardcoder-python-34b-v1.0.Q6_K - Test: Text Generation 128abAMD Ryzen 7 9800X3D 8-Core - AMD Radeon PRO W7500 8GBc0.47250.9451.41751.892.3625SE +/- 0.00, N = 32.12.12.12.1

Llamafile

Model: TinyLlama-1.1B-Chat-v1.0.BF16 - Test: Prompt Processing 2048

OpenBenchmarking.orgTokens Per Second, More Is BetterLlamafile 0.8.16Model: TinyLlama-1.1B-Chat-v1.0.BF16 - Test: Prompt Processing 2048abAMD Ryzen 7 9800X3D 8-Core - AMD Radeon PRO W7500 8GBc7K14K21K28K35KSE +/- 0.00, N = 332768327683276832768

Llamafile

Model: TinyLlama-1.1B-Chat-v1.0.BF16 - Test: Prompt Processing 1024

OpenBenchmarking.orgTokens Per Second, More Is BetterLlamafile 0.8.16Model: TinyLlama-1.1B-Chat-v1.0.BF16 - Test: Prompt Processing 1024abAMD Ryzen 7 9800X3D 8-Core - AMD Radeon PRO W7500 8GBc4K8K12K16K20KSE +/- 0.00, N = 316384163841638416384

Llamafile

Model: TinyLlama-1.1B-Chat-v1.0.BF16 - Test: Prompt Processing 512

OpenBenchmarking.orgTokens Per Second, More Is BetterLlamafile 0.8.16Model: TinyLlama-1.1B-Chat-v1.0.BF16 - Test: Prompt Processing 512abAMD Ryzen 7 9800X3D 8-Core - AMD Radeon PRO W7500 8GBc2K4K6K8K10KSE +/- 0.00, N = 38192819281928192

Llamafile

Model: TinyLlama-1.1B-Chat-v1.0.BF16 - Test: Prompt Processing 256

OpenBenchmarking.orgTokens Per Second, More Is BetterLlamafile 0.8.16Model: TinyLlama-1.1B-Chat-v1.0.BF16 - Test: Prompt Processing 256abAMD Ryzen 7 9800X3D 8-Core - AMD Radeon PRO W7500 8GBc9001800270036004500SE +/- 0.00, N = 34096409640964096

Llamafile

Model: Llama-3.2-3B-Instruct.Q6_K - Test: Prompt Processing 2048

OpenBenchmarking.orgTokens Per Second, More Is BetterLlamafile 0.8.16Model: Llama-3.2-3B-Instruct.Q6_K - Test: Prompt Processing 2048abAMD Ryzen 7 9800X3D 8-Core - AMD Radeon PRO W7500 8GBc7K14K21K28K35KSE +/- 0.00, N = 332768327683276832768

Llamafile

Model: Llama-3.2-3B-Instruct.Q6_K - Test: Prompt Processing 1024

OpenBenchmarking.orgTokens Per Second, More Is BetterLlamafile 0.8.16Model: Llama-3.2-3B-Instruct.Q6_K - Test: Prompt Processing 1024abAMD Ryzen 7 9800X3D 8-Core - AMD Radeon PRO W7500 8GBc4K8K12K16K20KSE +/- 0.00, N = 316384163841638416384

Llamafile

Model: Llama-3.2-3B-Instruct.Q6_K - Test: Prompt Processing 512

OpenBenchmarking.orgTokens Per Second, More Is BetterLlamafile 0.8.16Model: Llama-3.2-3B-Instruct.Q6_K - Test: Prompt Processing 512abAMD Ryzen 7 9800X3D 8-Core - AMD Radeon PRO W7500 8GBc2K4K6K8K10KSE +/- 0.00, N = 38192819281928192

Llamafile

Model: Llama-3.2-3B-Instruct.Q6_K - Test: Prompt Processing 256

OpenBenchmarking.orgTokens Per Second, More Is BetterLlamafile 0.8.16Model: Llama-3.2-3B-Instruct.Q6_K - Test: Prompt Processing 256abAMD Ryzen 7 9800X3D 8-Core - AMD Radeon PRO W7500 8GBc9001800270036004500SE +/- 0.00, N = 34096409640964096


Phoronix Test Suite v10.8.5