llama ncnn 9950X AMD Ryzen 9 9950X 16-Core testing with a ASRock X870E Taichi (3.12.AS02 BIOS) and AMD Radeon RX 7800 XT 16GB on Ubuntu 24.04 via the Phoronix Test Suite. a: Processor: AMD Ryzen 9 9950X 16-Core @ 5.75GHz (16 Cores / 32 Threads), Motherboard: ASRock X870E Taichi (3.12.AS02 BIOS), Chipset: AMD Device 14d8, Memory: 2 x 16GB DDR5-6000MT/s F5-6000J2836G16G, Disk: Western Digital WD_BLACK SN850X 2000GB + 32GB Flash Drive, Graphics: AMD Radeon RX 7800 XT 16GB, Audio: AMD Navi 31 HDMI/DP, Monitor: DELL U2723QE, Network: Realtek Device 8126 + MEDIATEK Device 0717 OS: Ubuntu 24.04, Kernel: 6.12.3-061203-generic (x86_64), Desktop: GNOME Shell 46.0, Display Server: X Server 1.21.1.11 + Wayland, OpenGL: 4.6 Mesa 24.2.0-devel (LLVM 18.1.7 DRM 3.59), Compiler: GCC 13.3.0, File-System: ext4, Screen Resolution: 3840x2160 b: Processor: AMD Ryzen 9 9950X 16-Core @ 5.75GHz (16 Cores / 32 Threads), Motherboard: ASRock X870E Taichi (3.12.AS02 BIOS), Chipset: AMD Device 14d8, Memory: 2 x 16GB DDR5-6000MT/s F5-6000J2836G16G, Disk: Western Digital WD_BLACK SN850X 2000GB + 32GB Flash Drive, Graphics: AMD Radeon RX 7800 XT 16GB, Audio: AMD Navi 31 HDMI/DP, Monitor: DELL U2723QE, Network: Realtek Device 8126 + MEDIATEK Device 0717 OS: Ubuntu 24.04, Kernel: 6.12.3-061203-generic (x86_64), Desktop: GNOME Shell 46.0, Display Server: X Server 1.21.1.11 + Wayland, OpenGL: 4.6 Mesa 24.2.0-devel (LLVM 18.1.7 DRM 3.59), Compiler: GCC 13.3.0, File-System: ext4, Screen Resolution: 3840x2160 c: Processor: AMD Ryzen 9 9950X 16-Core @ 5.75GHz (16 Cores / 32 Threads), Motherboard: ASRock X870E Taichi (3.12.AS02 BIOS), Chipset: AMD Device 14d8, Memory: 2 x 16GB DDR5-6000MT/s F5-6000J2836G16G, Disk: Western Digital WD_BLACK SN850X 2000GB + 32GB Flash Drive, Graphics: AMD Radeon RX 7800 XT 16GB, Audio: AMD Navi 31 HDMI/DP, Monitor: DELL U2723QE, Network: Realtek Device 8126 + MEDIATEK Device 0717 OS: Ubuntu 24.04, Kernel: 6.12.3-061203-generic (x86_64), Desktop: GNOME Shell 46.0, Display Server: X Server 1.21.1.11 + Wayland, OpenGL: 4.6 Mesa 24.2.0-devel (LLVM 18.1.7 DRM 3.59), Compiler: GCC 13.3.0, File-System: ext4, Screen Resolution: 3840x2160 d: Processor: AMD Ryzen 9 9950X 16-Core @ 5.75GHz (16 Cores / 32 Threads), Motherboard: ASRock X870E Taichi (3.12.AS02 BIOS), Chipset: AMD Device 14d8, Memory: 2 x 16GB DDR5-6000MT/s F5-6000J2836G16G, Disk: Western Digital WD_BLACK SN850X 2000GB + 32GB Flash Drive, Graphics: AMD Radeon RX 7800 XT 16GB, Audio: AMD Navi 31 HDMI/DP, Monitor: DELL U2723QE, Network: Realtek Device 8126 + MEDIATEK Device 0717 OS: Ubuntu 24.04, Kernel: 6.12.3-061203-generic (x86_64), Desktop: GNOME Shell 46.0, Display Server: X Server 1.21.1.11 + Wayland, OpenGL: 4.6 Mesa 24.2.0-devel (LLVM 18.1.7 DRM 3.59), Compiler: GCC 13.3.0, File-System: ext4, Screen Resolution: 3840x2160 NCNN 20241226 Target: CPU - Model: mobilenet ms < Lower Is Better a . 6.84 |==================================================================== b . 6.77 |=================================================================== c . 6.94 |===================================================================== d . 6.79 |==================================================================== NCNN 20241226 Target: CPU-v2-v2 - Model: mobilenet-v2 ms < Lower Is Better a . 2.45 |================================================================= b . 2.47 |================================================================== c . 2.56 |==================================================================== d . 2.59 |===================================================================== NCNN 20241226 Target: CPU-v3-v3 - Model: mobilenet-v3 ms < Lower Is Better a . 2.37 |=================================================================== b . 2.34 |================================================================== c . 2.40 |==================================================================== d . 2.45 |===================================================================== NCNN 20241226 Target: CPU - Model: shufflenet-v2 ms < Lower Is Better a . 2.19 |=================================================================== b . 2.20 |==================================================================== c . 2.23 |===================================================================== d . 2.24 |===================================================================== NCNN 20241226 Target: CPU - Model: mnasnet ms < Lower Is Better a . 2.20 |============================================================ b . 2.32 |=============================================================== c . 2.43 |================================================================== d . 2.53 |===================================================================== NCNN 20241226 Target: CPU - Model: efficientnet-b0 ms < Lower Is Better a . 3.09 |================================================================== b . 3.13 |=================================================================== c . 3.21 |===================================================================== d . 3.23 |===================================================================== NCNN 20241226 Target: CPU - Model: blazeface ms < Lower Is Better a . 0.92 |================================================================ b . 0.94 |================================================================== c . 0.96 |=================================================================== d . 0.99 |===================================================================== NCNN 20241226 Target: CPU - Model: googlenet ms < Lower Is Better a . 5.78 |=================================================================== b . 5.84 |==================================================================== c . 5.85 |==================================================================== d . 5.95 |===================================================================== NCNN 20241226 Target: CPU - Model: vgg16 ms < Lower Is Better a . 23.65 |================================================================= b . 23.52 |================================================================= c . 24.77 |==================================================================== d . 23.64 |================================================================= NCNN 20241226 Target: CPU - Model: resnet18 ms < Lower Is Better a . 4.05 |===================================================================== b . 4.00 |==================================================================== c . 3.99 |==================================================================== d . 4.05 |===================================================================== NCNN 20241226 Target: CPU - Model: alexnet ms < Lower Is Better a . 3.30 |===================================================================== b . 3.26 |==================================================================== c . 3.20 |=================================================================== d . 3.30 |===================================================================== NCNN 20241226 Target: CPU - Model: resnet50 ms < Lower Is Better a . 9.01 |==================================================================== b . 9.01 |==================================================================== c . 9.10 |===================================================================== d . 9.02 |==================================================================== NCNN 20241226 Target: CPUv2-yolov3v2-yolov3 - Model: mobilenetv2-yolov3 ms < Lower Is Better a . 6.84 |==================================================================== b . 6.77 |=================================================================== c . 6.94 |===================================================================== d . 6.79 |==================================================================== NCNN 20241226 Target: CPU - Model: yolov4-tiny ms < Lower Is Better a . 10.46 |================================================================== b . 10.54 |=================================================================== c . 10.41 |================================================================== d . 10.73 |==================================================================== NCNN 20241226 Target: CPU - Model: squeezenet_ssd ms < Lower Is Better a . 5.48 |================================================================== b . 5.49 |================================================================== c . 5.76 |===================================================================== d . 5.67 |==================================================================== NCNN 20241226 Target: CPU - Model: regnety_400m ms < Lower Is Better a . 6.34 |================================================================== b . 6.34 |================================================================== c . 6.62 |===================================================================== d . 6.64 |===================================================================== NCNN 20241226 Target: CPU - Model: vision_transformer ms < Lower Is Better a . 26.57 |=================================================================== b . 26.82 |==================================================================== c . 26.56 |=================================================================== d . 26.49 |=================================================================== NCNN 20241226 Target: CPU - Model: FastestDet ms < Lower Is Better a . 2.46 |================================================================ b . 2.66 |===================================================================== c . 1.80 |=============================================== d . 2.64 |==================================================================== NCNN 20241226 Target: Vulkan GPU - Model: mobilenet ms < Lower Is Better a . 6.92 |===================================================================== b . 6.79 |==================================================================== c . 6.41 |================================================================ d . 6.81 |==================================================================== NCNN 20241226 Target: Vulkan GPU-v2-v2 - Model: mobilenet-v2 ms < Lower Is Better a . 2.56 |==================================================================== b . 2.53 |=================================================================== c . 2.57 |==================================================================== d . 2.61 |===================================================================== NCNN 20241226 Target: Vulkan GPU-v3-v3 - Model: mobilenet-v3 ms < Lower Is Better a . 2.40 |==================================================================== b . 2.40 |==================================================================== c . 2.42 |===================================================================== d . 2.41 |===================================================================== NCNN 20241226 Target: Vulkan GPU - Model: shufflenet-v2 ms < Lower Is Better a . 2.27 |===================================================================== b . 2.25 |==================================================================== c . 2.26 |===================================================================== d . 2.27 |===================================================================== NCNN 20241226 Target: Vulkan GPU - Model: mnasnet ms < Lower Is Better a . 2.42 |==================================================================== b . 2.40 |==================================================================== c . 2.43 |==================================================================== d . 2.45 |===================================================================== NCNN 20241226 Target: Vulkan GPU - Model: efficientnet-b0 ms < Lower Is Better a . 3.19 |=================================================================== b . 3.21 |=================================================================== c . 3.20 |=================================================================== d . 3.30 |===================================================================== NCNN 20241226 Target: Vulkan GPU - Model: blazeface ms < Lower Is Better a . 0.97 |==================================================================== b . 0.95 |=================================================================== c . 0.98 |===================================================================== d . 0.97 |==================================================================== NCNN 20241226 Target: Vulkan GPU - Model: googlenet ms < Lower Is Better a . 6.01 |===================================================================== b . 5.91 |==================================================================== c . 5.94 |==================================================================== d . 5.97 |===================================================================== NCNN 20241226 Target: Vulkan GPU - Model: vgg16 ms < Lower Is Better a . 24.01 |==================================================================== b . 23.62 |=================================================================== c . 23.98 |==================================================================== d . 24.15 |==================================================================== NCNN 20241226 Target: Vulkan GPU - Model: resnet18 ms < Lower Is Better a . 3.99 |==================================================================== b . 4.04 |===================================================================== c . 3.98 |==================================================================== d . 4.00 |==================================================================== NCNN 20241226 Target: Vulkan GPU - Model: alexnet ms < Lower Is Better a . 3.20 |================================================================== b . 3.29 |==================================================================== c . 3.19 |================================================================== d . 3.33 |===================================================================== NCNN 20241226 Target: Vulkan GPU - Model: resnet50 ms < Lower Is Better a . 8.97 |===================================================================== b . 9.01 |===================================================================== c . 8.91 |==================================================================== d . 8.94 |==================================================================== NCNN 20241226 Target: Vulkan GPUv2-yolov3v2-yolov3 - Model: mobilenetv2-yolov3 ms < Lower Is Better a . 6.92 |===================================================================== b . 6.79 |==================================================================== c . 6.41 |================================================================ d . 6.81 |==================================================================== NCNN 20241226 Target: Vulkan GPU - Model: yolov4-tiny ms < Lower Is Better a . 10.88 |==================================================================== b . 10.59 |================================================================== c . 10.25 |================================================================ d . 9.73 |============================================================= NCNN 20241226 Target: Vulkan GPU - Model: squeezenet_ssd ms < Lower Is Better a . 5.53 |==================================================================== b . 5.47 |==================================================================== c . 5.13 |=============================================================== d . 5.58 |===================================================================== NCNN 20241226 Target: Vulkan GPU - Model: regnety_400m ms < Lower Is Better a . 6.51 |===================================================================== b . 6.44 |==================================================================== c . 6.46 |==================================================================== d . 6.52 |===================================================================== NCNN 20241226 Target: Vulkan GPU - Model: vision_transformer ms < Lower Is Better a . 27.89 |==================================================================== b . 26.59 |================================================================= c . 26.85 |================================================================= d . 26.31 |================================================================ NCNN 20241226 Target: Vulkan GPU - Model: FastestDet ms < Lower Is Better a . 2.74 |===================================================================== b . 2.48 |============================================================== c . 2.07 |==================================================== d . 1.81 |============================================== Llama.cpp b4397 Backend: CPU BLAS - Model: Llama-3.1-Tulu-3-8B-Q8_0 - Test: Text Generation 128 Tokens Per Second > Higher Is Better a . 9.25 |===================================================================== b . 9.25 |===================================================================== c . 9.24 |===================================================================== d . 9.26 |===================================================================== Llama.cpp b4397 Backend: CPU BLAS - Model: Llama-3.1-Tulu-3-8B-Q8_0 - Test: Prompt Processing 512 Tokens Per Second > Higher Is Better a . 90.47 |================================================================== b . 91.73 |=================================================================== c . 87.44 |================================================================ d . 92.82 |==================================================================== Llama.cpp b4397 Backend: CPU BLAS - Model: Llama-3.1-Tulu-3-8B-Q8_0 - Test: Prompt Processing 1024 Tokens Per Second > Higher Is Better a . 91.86 |================================================================== b . 90.59 |================================================================= c . 94.63 |==================================================================== d . 92.60 |=================================================================== Llama.cpp b4397 Backend: CPU BLAS - Model: Llama-3.1-Tulu-3-8B-Q8_0 - Test: Prompt Processing 2048 Tokens Per Second > Higher Is Better a . 88.94 |=================================================================== b . 89.40 |=================================================================== c . 89.60 |=================================================================== d . 90.50 |==================================================================== Llama.cpp b4397 Backend: CPU BLAS - Model: Mistral-7B-Instruct-v0.3-Q8_0 - Test: Text Generation 128 Tokens Per Second > Higher Is Better a . 9.74 |===================================================================== b . 9.75 |===================================================================== c . 9.80 |===================================================================== d . 9.75 |===================================================================== Llama.cpp b4397 Backend: CPU BLAS - Model: Mistral-7B-Instruct-v0.3-Q8_0 - Test: Prompt Processing 512 Tokens Per Second > Higher Is Better a . 89.96 |================================================================== b . 89.92 |================================================================== c . 90.09 |=================================================================== d . 92.08 |==================================================================== Llama.cpp b4397 Backend: CPU BLAS - Model: Mistral-7B-Instruct-v0.3-Q8_0 - Test: Prompt Processing 1024 Tokens Per Second > Higher Is Better a . 91.63 |=================================================================== b . 92.01 |=================================================================== c . 91.73 |=================================================================== d . 93.49 |==================================================================== Llama.cpp b4397 Backend: CPU BLAS - Model: Mistral-7B-Instruct-v0.3-Q8_0 - Test: Prompt Processing 2048 Tokens Per Second > Higher Is Better a . 88.19 |================================================================== b . 90.55 |==================================================================== c . 90.44 |==================================================================== d . 89.71 |=================================================================== Llama.cpp b4397 Backend: CPU BLAS - Model: granite-3.0-3b-a800m-instruct-Q8_0 - Test: Text Generation 128 Tokens Per Second > Higher Is Better a . 65.97 |==================================================================== b . 65.92 |==================================================================== c . 66.05 |==================================================================== d . 65.86 |==================================================================== Llama.cpp b4397 Backend: CPU BLAS - Model: granite-3.0-3b-a800m-instruct-Q8_0 - Test: Prompt Processing 512 Tokens Per Second > Higher Is Better a . 412.57 |================================================================== b . 418.20 |=================================================================== c . 414.72 |================================================================== d . 410.23 |================================================================== Llama.cpp b4397 Backend: CPU BLAS - Model: granite-3.0-3b-a800m-instruct-Q8_0 - Test: Prompt Processing 1024 Tokens Per Second > Higher Is Better a . 396.01 |=================================================================== b . 395.63 |=================================================================== c . 391.87 |================================================================== d . 397.20 |=================================================================== Llama.cpp b4397 Backend: CPU BLAS - Model: granite-3.0-3b-a800m-instruct-Q8_0 - Test: Prompt Processing 2048 Tokens Per Second > Higher Is Better a . 372.75 |=================================================================== b . 372.04 |=================================================================== c . 371.89 |=================================================================== d . 371.23 |===================================================================