9950X EOY 2024 AMD Ryzen 9 9950X 16-Core testing with a ASRock X870E Taichi (3.12.AS02 BIOS) and AMD Radeon RX 7800 XT 16GB on Ubuntu 24.04 via the Phoronix Test Suite. a: Processor: AMD Ryzen 9 9950X 16-Core @ 5.75GHz (16 Cores / 32 Threads), Motherboard: ASRock X870E Taichi (3.12.AS02 BIOS), Chipset: AMD Device 14d8, Memory: 2 x 16GB DDR5-6000MT/s F5-6000J2836G16G, Disk: Western Digital WD_BLACK SN850X 2000GB, Graphics: AMD Radeon RX 7800 XT 16GB (2200/3000MHz), Audio: AMD Navi 31 HDMI/DP, Monitor: DELL U2723QE, Network: Realtek Device 8126 + MEDIATEK Device 0717 OS: Ubuntu 24.04, Kernel: 6.8.0-50-generic (x86_64), Desktop: GNOME Shell 46.0, Display Server: X Server 1.21.1.11 + Wayland, OpenGL: 4.6 Mesa 24.2.0-devel (LLVM 18.1.7 DRM 3.58), Compiler: GCC 13.3.0, File-System: ext4, Screen Resolution: 3840x2160 b: Processor: AMD Ryzen 9 9950X 16-Core @ 5.75GHz (16 Cores / 32 Threads), Motherboard: ASRock X870E Taichi (3.12.AS02 BIOS), Chipset: AMD Device 14d8, Memory: 2 x 16GB DDR5-6000MT/s F5-6000J2836G16G, Disk: Western Digital WD_BLACK SN850X 2000GB, Graphics: AMD Radeon RX 7800 XT 16GB (2200/3000MHz), Audio: AMD Navi 31 HDMI/DP, Monitor: DELL U2723QE, Network: Realtek Device 8126 + MEDIATEK Device 0717 OS: Ubuntu 24.04, Kernel: 6.8.0-50-generic (x86_64), Desktop: GNOME Shell 46.0, Display Server: X Server 1.21.1.11 + Wayland, OpenGL: 4.6 Mesa 24.2.0-devel (LLVM 18.1.7 DRM 3.58), Compiler: GCC 13.3.0, File-System: ext4, Screen Resolution: 3840x2160 bb: Processor: AMD Ryzen 9 9950X 16-Core @ 5.75GHz (16 Cores / 32 Threads), Motherboard: ASRock X870E Taichi (3.12.AS02 BIOS), Chipset: AMD Device 14d8, Memory: 2 x 16GB DDR5-6000MT/s F5-6000J2836G16G, Disk: Western Digital WD_BLACK SN850X 2000GB, Graphics: AMD Radeon RX 7800 XT 16GB (2200/3000MHz), Audio: AMD Navi 31 HDMI/DP, Monitor: DELL U2723QE, Network: Realtek Device 8126 + MEDIATEK Device 0717 OS: Ubuntu 24.04, Kernel: 6.8.0-50-generic (x86_64), Desktop: GNOME Shell 46.0, Display Server: X Server 1.21.1.11 + Wayland, OpenGL: 4.6 Mesa 24.2.0-devel (LLVM 18.1.7 DRM 3.58), Compiler: GCC 13.3.0, File-System: ext4, Screen Resolution: 3840x2160 Laghos 3.1 Test: Triple Point Problem Major Kernels Total Rate > Higher Is Better a .. 238.80 |================================================================== b .. 237.25 |================================================================== bb . 234.27 |================================================================= Laghos 3.1 Test: Sedov Blast Wave, ube_922_hex.mesh Major Kernels Total Rate > Higher Is Better a .. 256.49 |================================================================== b .. 257.21 |================================================================== bb . 253.08 |================================================================= Palabos 2.3 Grid Size: 100 Mega Site Updates Per Second > Higher Is Better a .. 108.79 |================================================================= b .. 110.07 |================================================================== bb . 107.16 |================================================================ Palabos 2.3 Grid Size: 400 Mega Site Updates Per Second > Higher Is Better a .. 123.45 |================================================================== b .. 123.32 |================================================================== bb . 123.46 |================================================================== Palabos 2.3 Grid Size: 500 Mega Site Updates Per Second > Higher Is Better a .. 127.21 |================================================================== b .. 127.28 |================================================================== bb . 127.02 |================================================================== RELION 5.0 Test: Basic - Device: CPU Seconds < Lower Is Better a .. 711.39 |================================================================ b .. 729.78 |================================================================== bb . 733.99 |================================================================== Renaissance 0.16 Test: Scala Dotty ms < Lower Is Better a .. 345.7 |================================================================ b .. 341.2 |=============================================================== bb . 362.5 |=================================================================== Renaissance 0.16 Test: Random Forest ms < Lower Is Better a .. 326.1 |=============================================================== b .. 345.0 |=================================================================== bb . 330.5 |================================================================ Renaissance 0.16 Test: ALS Movie Lens ms < Lower Is Better a .. 7965.4 |================================================================ b .. 8191.4 |================================================================== bb . 8155.8 |================================================================== Renaissance 0.16 Test: Apache Spark ALS ms < Lower Is Better Renaissance 0.16 Test: Apache Spark Bayes ms < Lower Is Better a .. 438.8 |=================================================================== b .. 409.4 |=============================================================== bb . 410.2 |=============================================================== Renaissance 0.16 Test: Savina Reactors.IO ms < Lower Is Better a .. 4289.1 |================================================================== b .. 4190.5 |================================================================ bb . 4257.9 |================================================================== Renaissance 0.16 Test: Apache Spark PageRank ms < Lower Is Better a .. 1914.9 |================================================================== b .. 1859.5 |================================================================ bb . 1853.2 |================================================================ Renaissance 0.16 Test: Finagle HTTP Requests ms < Lower Is Better a .. 1157.0 |================================================================= b .. 1169.6 |================================================================== bb . 1165.0 |================================================================== Renaissance 0.16 Test: Gaussian Mixture Model ms < Lower Is Better a .. 3017.1 |================================================================== b .. 3018.1 |================================================================== bb . 3016.9 |================================================================== Renaissance 0.16 Test: In-Memory Database Shootout ms < Lower Is Better a .. 3328.8 |=============================================================== b .. 3329.0 |=============================================================== bb . 3478.3 |================================================================== Renaissance 0.16 Test: Akka Unbalanced Cobwebbed Tree ms < Lower Is Better a .. 3782.4 |================================================================= b .. 3821.4 |================================================================== bb . 3842.9 |================================================================== Renaissance 0.16 Test: Genetic Algorithm Using Jenetics + Futures ms < Lower Is Better a .. 699.2 |=================================================================== b .. 691.4 |================================================================== bb . 683.8 |================================================================== srsRAN Project 24.10 Test: PDSCH Processor Benchmark, Throughput Total Mbps > Higher Is Better a .. 44425.8 |================================================================ b .. 44552.3 |================================================================= bb . 44781.9 |================================================================= srsRAN Project 24.10 Test: PUSCH Processor Benchmark, Throughput Total Mbps > Higher Is Better a .. 3391.0 |================================================================== b .. 3391.8 |================================================================== bb . 3391.2 |================================================================== srsRAN Project 24.10 Test: PDSCH Processor Benchmark, Throughput Thread Mbps > Higher Is Better a .. 2303.1 |================================================================= b .. 2332.2 |================================================================== bb . 2276.9 |================================================================ srsRAN Project 24.10 Test: PUSCH Processor Benchmark, Throughput Thread Mbps > Higher Is Better a .. 317.7 |=================================================================== b .. 317.5 |=================================================================== bb . 317.7 |=================================================================== SVT-AV1 2.3 Encoder Mode: Preset 3 - Input: Bosphorus 4K Frames Per Second > Higher Is Better a .. 12.24 |=================================================================== b .. 12.23 |=================================================================== bb . 12.22 |=================================================================== SVT-AV1 2.3 Encoder Mode: Preset 5 - Input: Bosphorus 4K Frames Per Second > Higher Is Better a .. 44.46 |=================================================================== b .. 44.48 |=================================================================== bb . 44.61 |=================================================================== SVT-AV1 2.3 Encoder Mode: Preset 8 - Input: Bosphorus 4K Frames Per Second > Higher Is Better a .. 129.41 |================================================================== b .. 129.57 |================================================================== bb . 129.60 |================================================================== SVT-AV1 2.3 Encoder Mode: Preset 13 - Input: Bosphorus 4K Frames Per Second > Higher Is Better a .. 258.82 |================================================================== b .. 259.39 |================================================================== bb . 259.28 |================================================================== SVT-AV1 2.3 Encoder Mode: Preset 3 - Input: Bosphorus 1080p Frames Per Second > Higher Is Better a .. 37.73 |=================================================================== b .. 37.78 |=================================================================== bb . 37.90 |=================================================================== SVT-AV1 2.3 Encoder Mode: Preset 5 - Input: Bosphorus 1080p Frames Per Second > Higher Is Better a .. 128.82 |================================================================= b .. 128.43 |================================================================= bb . 129.87 |================================================================== SVT-AV1 2.3 Encoder Mode: Preset 8 - Input: Bosphorus 1080p Frames Per Second > Higher Is Better a .. 427.83 |================================================================= b .. 430.11 |================================================================== bb . 431.41 |================================================================== SVT-AV1 2.3 Encoder Mode: Preset 13 - Input: Bosphorus 1080p Frames Per Second > Higher Is Better a .. 1018.40 |================================================================= b .. 1018.46 |================================================================= bb . 1018.59 |================================================================= SVT-AV1 2.3 Encoder Mode: Preset 3 - Input: Beauty 4K 10-bit Frames Per Second > Higher Is Better a . 1.681 |==================================================================== b . 1.688 |==================================================================== SVT-AV1 2.3 Encoder Mode: Preset 5 - Input: Beauty 4K 10-bit Frames Per Second > Higher Is Better a . 7.459 |==================================================================== b . 7.356 |=================================================================== SVT-AV1 2.3 Encoder Mode: Preset 8 - Input: Beauty 4K 10-bit Frames Per Second > Higher Is Better a . 14.15 |==================================================================== b . 13.86 |=================================================================== SVT-AV1 2.3 Encoder Mode: Preset 13 - Input: Beauty 4K 10-bit Frames Per Second > Higher Is Better a . 19.50 |==================================================================== b . 19.50 |==================================================================== VVenC 1.13 Video Input: Bosphorus 4K - Video Preset: Fast Frames Per Second > Higher Is Better a . 10.64 |==================================================================== b . 10.59 |==================================================================== VVenC 1.13 Video Input: Bosphorus 4K - Video Preset: Faster Frames Per Second > Higher Is Better a . 23.18 |==================================================================== b . 23.00 |=================================================================== VVenC 1.13 Video Input: Bosphorus 1080p - Video Preset: Fast Frames Per Second > Higher Is Better a . 29.68 |==================================================================== b . 29.59 |==================================================================== VVenC 1.13 Video Input: Bosphorus 1080p - Video Preset: Faster Frames Per Second > Higher Is Better a . 62.47 |==================================================================== b . 61.33 |=================================================================== x265 4.1 Video Input: Bosphorus 4K Frames Per Second > Higher Is Better a . 41.13 |==================================================================== b . 41.33 |==================================================================== x265 4.1 Video Input: Bosphorus 1080p Frames Per Second > Higher Is Better a . 141.82 |=================================================================== b . 140.75 |================================================================== ACES DGEMM 1.0 Sustained Floating-Point Rate GFLOP/s > Higher Is Better a . 1782.78 |================================================================== b . 1786.58 |================================================================== Primesieve 12.6 Length: 1e12 Seconds < Lower Is Better a . 5.931 |==================================================================== b . 5.960 |==================================================================== Primesieve 12.6 Length: 1e13 Seconds < Lower Is Better a . 74.52 |==================================================================== b . 74.54 |==================================================================== Timed Eigen Compilation 3.4.0 Time To Compile Seconds < Lower Is Better ASTC Encoder 5.0 Preset: Fast MT/s > Higher Is Better a . 445.80 |=================================================================== b . 446.93 |=================================================================== ASTC Encoder 5.0 Preset: Medium MT/s > Higher Is Better a . 175.81 |=================================================================== b . 176.05 |=================================================================== ASTC Encoder 5.0 Preset: Thorough MT/s > Higher Is Better a . 23.06 |==================================================================== b . 23.14 |==================================================================== ASTC Encoder 5.0 Preset: Exhaustive MT/s > Higher Is Better a . 1.9466 |=================================================================== b . 1.9436 |=================================================================== ASTC Encoder 5.0 Preset: Very Thorough MT/s > Higher Is Better a . 3.1661 |=================================================================== b . 3.1636 |=================================================================== Blender 4.3 Blend File: BMW27 - Compute: CPU-Only Seconds < Lower Is Better a . 46.16 |=================================================================== b . 46.54 |==================================================================== Blender 4.3 Blend File: Junkshop - Compute: CPU-Only Seconds < Lower Is Better a . 61.86 |=================================================================== b . 62.37 |==================================================================== Blender 4.3 Blend File: Classroom - Compute: CPU-Only Seconds < Lower Is Better a . 129.62 |================================================================== b . 130.68 |=================================================================== Blender 4.3 Blend File: Fishy Cat - Compute: CPU-Only Seconds < Lower Is Better a . 65.06 |==================================================================== b . 65.21 |==================================================================== Blender 4.3 Blend File: Barbershop - Compute: CPU-Only Seconds < Lower Is Better a . 461.30 |================================================================== b . 464.99 |=================================================================== Blender 4.3 Blend File: Pabellon Barcelona - Compute: CPU-Only Seconds < Lower Is Better a . 145.91 |=================================================================== b . 146.58 |=================================================================== OpenVINO 2024.5 Model: Face Detection FP16 - Device: CPU FPS > Higher Is Better a . 23.23 |==================================================================== b . 23.40 |==================================================================== OpenVINO 2024.5 Model: Face Detection FP16 - Device: CPU ms < Lower Is Better a . 343.55 |=================================================================== b . 341.22 |=================================================================== OpenVINO 2024.5 Model: Person Detection FP16 - Device: CPU FPS > Higher Is Better a . 109.86 |=================================================================== b . 109.45 |=================================================================== OpenVINO 2024.5 Model: Person Detection FP16 - Device: CPU ms < Lower Is Better a . 72.73 |==================================================================== b . 73.00 |==================================================================== OpenVINO 2024.5 Model: Person Detection FP32 - Device: CPU FPS > Higher Is Better a . 110.17 |=================================================================== b . 109.87 |=================================================================== OpenVINO 2024.5 Model: Person Detection FP32 - Device: CPU ms < Lower Is Better a . 72.53 |==================================================================== b . 72.73 |==================================================================== OpenVINO 2024.5 Model: Vehicle Detection FP16 - Device: CPU FPS > Higher Is Better a . 898.70 |=================================================================== b . 886.76 |================================================================== OpenVINO 2024.5 Model: Vehicle Detection FP16 - Device: CPU ms < Lower Is Better a . 8.87 |==================================================================== b . 8.99 |===================================================================== OpenVINO 2024.5 Model: Face Detection FP16-INT8 - Device: CPU FPS > Higher Is Better a . 47.02 |==================================================================== b . 46.95 |==================================================================== OpenVINO 2024.5 Model: Face Detection FP16-INT8 - Device: CPU ms < Lower Is Better a . 169.95 |=================================================================== b . 170.22 |=================================================================== OpenVINO 2024.5 Model: Face Detection Retail FP16 - Device: CPU FPS > Higher Is Better a . 4839.87 |================================================================== b . 4825.11 |================================================================== OpenVINO 2024.5 Model: Face Detection Retail FP16 - Device: CPU ms < Lower Is Better a . 1.55 |===================================================================== b . 1.55 |===================================================================== OpenVINO 2024.5 Model: Road Segmentation ADAS FP16 - Device: CPU FPS > Higher Is Better a . 373.69 |================================================================== b . 377.31 |=================================================================== OpenVINO 2024.5 Model: Road Segmentation ADAS FP16 - Device: CPU ms < Lower Is Better a . 21.36 |==================================================================== b . 21.16 |=================================================================== OpenVINO 2024.5 Model: Vehicle Detection FP16-INT8 - Device: CPU FPS > Higher Is Better a . 2522.84 |================================================================== b . 2530.77 |================================================================== OpenVINO 2024.5 Model: Vehicle Detection FP16-INT8 - Device: CPU ms < Lower Is Better a . 3.13 |===================================================================== b . 3.12 |===================================================================== OpenVINO 2024.5 Model: Weld Porosity Detection FP16 - Device: CPU FPS > Higher Is Better a . 2303.98 |================================================================== b . 2301.68 |================================================================== OpenVINO 2024.5 Model: Weld Porosity Detection FP16 - Device: CPU ms < Lower Is Better a . 6.88 |===================================================================== b . 6.89 |===================================================================== OpenVINO 2024.5 Model: Face Detection Retail FP16-INT8 - Device: CPU FPS > Higher Is Better a . 7122.61 |================================================================== b . 7153.86 |================================================================== OpenVINO 2024.5 Model: Face Detection Retail FP16-INT8 - Device: CPU ms < Lower Is Better a . 2.14 |===================================================================== b . 2.13 |===================================================================== OpenVINO 2024.5 Model: Road Segmentation ADAS FP16-INT8 - Device: CPU FPS > Higher Is Better a . 603.24 |=================================================================== b . 602.01 |=================================================================== OpenVINO 2024.5 Model: Road Segmentation ADAS FP16-INT8 - Device: CPU ms < Lower Is Better a . 13.22 |==================================================================== b . 13.25 |==================================================================== OpenVINO 2024.5 Model: Machine Translation EN To DE FP16 - Device: CPU FPS > Higher Is Better a . 187.14 |=================================================================== b . 186.20 |=================================================================== OpenVINO 2024.5 Model: Machine Translation EN To DE FP16 - Device: CPU ms < Lower Is Better a . 42.69 |==================================================================== b . 42.90 |==================================================================== OpenVINO 2024.5 Model: Weld Porosity Detection FP16-INT8 - Device: CPU FPS > Higher Is Better a . 4524.64 |================================================================== b . 4516.85 |================================================================== OpenVINO 2024.5 Model: Weld Porosity Detection FP16-INT8 - Device: CPU ms < Lower Is Better a . 3.44 |===================================================================== b . 3.44 |===================================================================== OpenVINO 2024.5 Model: Person Vehicle Bike Detection FP16 - Device: CPU FPS > Higher Is Better a . 2127.08 |================================================================== b . 2128.36 |================================================================== OpenVINO 2024.5 Model: Person Vehicle Bike Detection FP16 - Device: CPU ms < Lower Is Better a . 3.73 |===================================================================== b . 3.73 |===================================================================== OpenVINO 2024.5 Model: Noise Suppression Poconet-Like FP16 - Device: CPU FPS > Higher Is Better a . 1145.37 |================================================================== b . 1145.81 |================================================================== OpenVINO 2024.5 Model: Noise Suppression Poconet-Like FP16 - Device: CPU ms < Lower Is Better a . 13.81 |==================================================================== b . 13.80 |==================================================================== OpenVINO 2024.5 Model: Handwritten English Recognition FP16 - Device: CPU FPS > Higher Is Better a . 908.55 |=================================================================== b . 899.22 |================================================================== OpenVINO 2024.5 Model: Handwritten English Recognition FP16 - Device: CPU ms < Lower Is Better a . 17.53 |=================================================================== b . 17.71 |==================================================================== OpenVINO 2024.5 Model: Person Re-Identification Retail FP16 - Device: CPU FPS > Higher Is Better a . 2968.38 |================================================================== b . 2959.34 |================================================================== OpenVINO 2024.5 Model: Person Re-Identification Retail FP16 - Device: CPU ms < Lower Is Better a . 2.64 |===================================================================== b . 2.65 |===================================================================== OpenVINO 2024.5 Model: Age Gender Recognition Retail 0013 FP16 - Device: CPU FPS > Higher Is Better a . 55602.53 |================================================================= b . 55559.65 |================================================================= OpenVINO 2024.5 Model: Age Gender Recognition Retail 0013 FP16 - Device: CPU ms < Lower Is Better a . 0.25 |===================================================================== b . 0.25 |===================================================================== OpenVINO 2024.5 Model: Handwritten English Recognition FP16-INT8 - Device: CPU FPS > Higher Is Better a . 990.07 |=================================================================== b . 985.48 |=================================================================== OpenVINO 2024.5 Model: Handwritten English Recognition FP16-INT8 - Device: CPU ms < Lower Is Better a . 16.09 |==================================================================== b . 16.15 |==================================================================== OpenVINO 2024.5 Model: Age Gender Recognition Retail 0013 FP16-INT8 - Device: CPU FPS > Higher Is Better a . 74229.17 |================================================================= b . 74013.33 |================================================================= OpenVINO 2024.5 Model: Age Gender Recognition Retail 0013 FP16-INT8 - Device: CPU ms < Lower Is Better a . 0.19 |===================================================================== b . 0.19 |===================================================================== Llama.cpp b4154 Backend: CPU BLAS - Model: Llama-3.1-Tulu-3-8B-Q8_0 - Test: Text Generation 128 Tokens Per Second > Higher Is Better a . 9.29 |===================================================================== b . 9.29 |===================================================================== Llama.cpp b4154 Backend: CPU BLAS - Model: Llama-3.1-Tulu-3-8B-Q8_0 - Test: Prompt Processing 512 Tokens Per Second > Higher Is Better a . 146.74 |=================================================================== b . 144.99 |================================================================== Llama.cpp b4154 Backend: CPU BLAS - Model: Llama-3.1-Tulu-3-8B-Q8_0 - Test: Prompt Processing 1024 Tokens Per Second > Higher Is Better a . 141.61 |=================================================================== b . 141.27 |=================================================================== Llama.cpp b4154 Backend: CPU BLAS - Model: Llama-3.1-Tulu-3-8B-Q8_0 - Test: Prompt Processing 2048 Tokens Per Second > Higher Is Better a . 133.60 |=================================================================== b . 134.15 |=================================================================== Llama.cpp b4154 Backend: CPU BLAS - Model: Mistral-7B-Instruct-v0.3-Q8_0 - Test: Text Generation 128 Tokens Per Second > Higher Is Better a . 9.77 |===================================================================== b . 9.76 |===================================================================== Llama.cpp b4154 Backend: CPU BLAS - Model: Mistral-7B-Instruct-v0.3-Q8_0 - Test: Prompt Processing 512 Tokens Per Second > Higher Is Better a . 142.82 |================================================================== b . 144.46 |=================================================================== Llama.cpp b4154 Backend: CPU BLAS - Model: Mistral-7B-Instruct-v0.3-Q8_0 - Test: Prompt Processing 1024 Tokens Per Second > Higher Is Better a . 141.85 |=================================================================== b . 139.95 |================================================================== Llama.cpp b4154 Backend: CPU BLAS - Model: Mistral-7B-Instruct-v0.3-Q8_0 - Test: Prompt Processing 2048 Tokens Per Second > Higher Is Better a . 132.57 |================================================================== b . 135.51 |=================================================================== Llama.cpp b4154 Backend: CPU BLAS - Model: granite-3.0-3b-a800m-instruct-Q8_0 - Test: Text Generation 128 Tokens Per Second > Higher Is Better a . 64.94 |==================================================================== b . 65.27 |==================================================================== Llama.cpp b4154 Backend: CPU BLAS - Model: granite-3.0-3b-a800m-instruct-Q8_0 - Test: Prompt Processing 512 Tokens Per Second > Higher Is Better a . 455.75 |================================================================== b . 463.68 |=================================================================== Llama.cpp b4154 Backend: CPU BLAS - Model: granite-3.0-3b-a800m-instruct-Q8_0 - Test: Prompt Processing 1024 Tokens Per Second > Higher Is Better a . 467.93 |=================================================================== b . 466.88 |=================================================================== Llama.cpp b4154 Backend: CPU BLAS - Model: granite-3.0-3b-a800m-instruct-Q8_0 - Test: Prompt Processing 2048 Tokens Per Second > Higher Is Better a . 400.00 |=================================================================== b . 388.99 |================================================================= Llamafile 0.8.16 Model: Llama-3.2-3B-Instruct.Q6_K - Test: Text Generation 16 Tokens Per Second > Higher Is Better a . 25.35 |==================================================================== b . 25.31 |==================================================================== Llamafile 0.8.16 Model: Llama-3.2-3B-Instruct.Q6_K - Test: Text Generation 128 Tokens Per Second > Higher Is Better a . 26.71 |==================================================================== b . 26.67 |==================================================================== Llamafile 0.8.16 Model: TinyLlama-1.1B-Chat-v1.0.BF16 - Test: Text Generation 16 Tokens Per Second > Higher Is Better a . 31.02 |==================================================================== b . 31.19 |==================================================================== Llamafile 0.8.16 Model: TinyLlama-1.1B-Chat-v1.0.BF16 - Test: Text Generation 128 Tokens Per Second > Higher Is Better a . 32.84 |==================================================================== b . 32.83 |==================================================================== Llamafile 0.8.16 Model: mistral-7b-instruct-v0.2.Q5_K_M - Test: Text Generation 16 Tokens Per Second > Higher Is Better a . 13.51 |==================================================================== b . 13.49 |==================================================================== Llamafile 0.8.16 Model: mistral-7b-instruct-v0.2.Q5_K_M - Test: Text Generation 128 Tokens Per Second > Higher Is Better a . 14.22 |==================================================================== b . 14.25 |==================================================================== Llamafile 0.8.16 Model: wizardcoder-python-34b-v1.0.Q6_K - Test: Text Generation 16 Tokens Per Second > Higher Is Better a . 2.36 |===================================================================== b . 2.36 |===================================================================== Llamafile 0.8.16 Model: wizardcoder-python-34b-v1.0.Q6_K - Test: Text Generation 128 Tokens Per Second > Higher Is Better a . 2.67 |===================================================================== b . 2.67 |===================================================================== OpenVINO GenAI 2024.5 Model: Gemma-7b-int4-ov - Device: CPU tokens/s > Higher Is Better a . 12.69 |==================================================================== b . 12.73 |==================================================================== OpenVINO GenAI 2024.5 Model: Gemma-7b-int4-ov - Device: CPU - Time To First Token ms < Lower Is Better a . 83.05 |==================================================================== b . 83.21 |==================================================================== OpenVINO GenAI 2024.5 Model: Gemma-7b-int4-ov - Device: CPU - Time Per Output Token ms < Lower Is Better a . 78.79 |==================================================================== b . 78.58 |==================================================================== OpenVINO GenAI 2024.5 Model: TinyLlama-1.1B-Chat-v1.0 - Device: CPU tokens/s > Higher Is Better a . 30.78 |==================================================================== b . 30.81 |==================================================================== OpenVINO GenAI 2024.5 Model: TinyLlama-1.1B-Chat-v1.0 - Device: CPU - Time To First Token ms < Lower Is Better a . 33.59 |==================================================================== b . 33.47 |==================================================================== OpenVINO GenAI 2024.5 Model: TinyLlama-1.1B-Chat-v1.0 - Device: CPU - Time Per Output Token ms < Lower Is Better a . 32.49 |==================================================================== b . 32.46 |==================================================================== OpenVINO GenAI 2024.5 Model: Falcon-7b-instruct-int4-ov - Device: CPU tokens/s > Higher Is Better a . 16.70 |==================================================================== b . 16.71 |==================================================================== OpenVINO GenAI 2024.5 Model: Falcon-7b-instruct-int4-ov - Device: CPU - Time To First Token ms < Lower Is Better a . 65.11 |==================================================================== b . 65.12 |==================================================================== OpenVINO GenAI 2024.5 Model: Falcon-7b-instruct-int4-ov - Device: CPU - Time Per Output Token ms < Lower Is Better a . 59.89 |==================================================================== b . 59.84 |==================================================================== OpenVINO GenAI 2024.5 Model: Phi-3-mini-128k-instruct-int4-ov - Device: CPU tokens/s > Higher Is Better a . 25.23 |==================================================================== b . 25.25 |==================================================================== OpenVINO GenAI 2024.5 Model: Phi-3-mini-128k-instruct-int4-ov - Device: CPU - Time To First Token ms < Lower Is Better a . 43.16 |==================================================================== b . 43.29 |==================================================================== OpenVINO GenAI 2024.5 Model: Phi-3-mini-128k-instruct-int4-ov - Device: CPU - Time Per Output Token ms < Lower Is Better a . 39.63 |==================================================================== b . 39.61 |====================================================================