ampere arm tests ARMv8 Neoverse-N1 testing with a System76 Thelio Astra (3.02 BIOS) and NVIDIA RTX A400/PCIe 4GB on Ubuntu 24.04 via the Phoronix Test Suite. a: Processor: ARMv8 Neoverse-N1 @ 3.00GHz (128 Cores), Motherboard: System76 Thelio Astra (3.02 BIOS), Chipset: Ampere Computing LLC Altra PCI Root Complex A, Memory: 8 x 32GB DDR4-3200MT/s Micron 18ASF4G72PDZ-3G2F1, Disk: 1024GB KINGSTON SKC3000S1024G, Graphics: NVIDIA RTX A400/PCIe 4GB, Audio: NVIDIA Device 2291, Monitor: DELL P2415Q, Network: 2 x Intel X550 + Intel I210 OS: Ubuntu 24.04, Kernel: 6.8.0-48-generic-64k (aarch64), Desktop: GNOME Shell 46.0, Display Server: X Server, Display Driver: NVIDIA 550.120, OpenGL: 4.6.0, Compiler: GCC 13.2.0, File-System: ext4, Screen Resolution: 3840x2160 b: Processor: ARMv8 Neoverse-N1 @ 3.00GHz (128 Cores), Motherboard: System76 Thelio Astra (3.02 BIOS), Chipset: Ampere Computing LLC Altra PCI Root Complex A, Memory: 8 x 32GB DDR4-3200MT/s Micron 18ASF4G72PDZ-3G2F1, Disk: 1024GB KINGSTON SKC3000S1024G, Graphics: NVIDIA RTX A400/PCIe 4GB, Audio: NVIDIA Device 2291, Monitor: DELL P2415Q, Network: 2 x Intel X550 + Intel I210 OS: Ubuntu 24.04, Kernel: 6.8.0-48-generic-64k (aarch64), Desktop: GNOME Shell 46.0, Display Server: X Server, Display Driver: NVIDIA 550.120, OpenGL: 4.6.0, Compiler: GCC 13.2.0, File-System: ext4, Screen Resolution: 3840x2160 OpenVINO 2024.5 Model: Face Detection FP16 - Device: CPU FPS > Higher Is Better a . 6.89 |===================================================================== b . 6.89 |===================================================================== OpenVINO 2024.5 Model: Person Detection FP16 - Device: CPU FPS > Higher Is Better a . 34.03 |==================================================================== b . 33.97 |==================================================================== OpenVINO 2024.5 Model: Person Detection FP32 - Device: CPU FPS > Higher Is Better a . 33.99 |==================================================================== b . 34.01 |==================================================================== OpenVINO 2024.5 Model: Vehicle Detection FP16 - Device: CPU FPS > Higher Is Better a . 524.99 |================================================================ b . 549.61 |=================================================================== OpenVINO 2024.5 Model: Face Detection FP16-INT8 - Device: CPU FPS > Higher Is Better a . 6.53 |===================================================================== b . 6.53 |===================================================================== OpenVINO 2024.5 Model: Face Detection Retail FP16 - Device: CPU FPS > Higher Is Better a . 1224.24 |================================================================== b . 1154.89 |============================================================== OpenVINO 2024.5 Model: Road Segmentation ADAS FP16 - Device: CPU FPS > Higher Is Better a . 230.52 |=================================================================== b . 232.20 |=================================================================== OpenVINO 2024.5 Model: Vehicle Detection FP16-INT8 - Device: CPU FPS > Higher Is Better a . 86.55 |==================================================================== b . 86.94 |==================================================================== OpenVINO 2024.5 Model: Weld Porosity Detection FP16 - Device: CPU FPS > Higher Is Better a . 506.71 |=================================================================== b . 506.24 |=================================================================== OpenVINO 2024.5 Model: Face Detection Retail FP16-INT8 - Device: CPU FPS > Higher Is Better a . 392.46 |=================================================================== b . 392.98 |=================================================================== OpenVINO 2024.5 Model: Road Segmentation ADAS FP16-INT8 - Device: CPU FPS > Higher Is Better a . 31.09 |==================================================================== b . 31.10 |==================================================================== OpenVINO 2024.5 Model: Machine Translation EN To DE FP16 - Device: CPU FPS > Higher Is Better a . 299.07 |=================================================================== b . 300.92 |=================================================================== OpenVINO 2024.5 Model: Weld Porosity Detection FP16-INT8 - Device: CPU FPS > Higher Is Better a . 521.97 |=================================================================== b . 525.64 |=================================================================== OpenVINO 2024.5 Model: Person Vehicle Bike Detection FP16 - Device: CPU FPS > Higher Is Better a . 297.98 |=================================================================== b . 294.35 |================================================================== OpenVINO 2024.5 Model: Noise Suppression Poconet-Like FP16 - Device: CPU FPS > Higher Is Better a . 91.23 |==================================================================== b . 91.86 |==================================================================== OpenVINO 2024.5 Model: Handwritten English Recognition FP16 - Device: CPU FPS > Higher Is Better a . 177.16 |=================================================================== b . 177.13 |=================================================================== OpenVINO 2024.5 Model: Person Re-Identification Retail FP16 - Device: CPU FPS > Higher Is Better a . 179.01 |================================================================== b . 181.87 |=================================================================== OpenVINO 2024.5 Model: Age Gender Recognition Retail 0013 FP16 - Device: CPU FPS > Higher Is Better a . 1848.45 |================================================================== b . 1818.46 |================================================================= OpenVINO 2024.5 Model: Handwritten English Recognition FP16-INT8 - Device: CPU FPS > Higher Is Better a . 153.15 |=================================================================== b . 153.23 |=================================================================== OpenVINO 2024.5 Model: Age Gender Recognition Retail 0013 FP16-INT8 - Device: CPU FPS > Higher Is Better a . 1850.02 |============================================================== b . 1970.51 |================================================================== Llama.cpp b4154 Backend: CPU BLAS - Model: Llama-3.1-Tulu-3-8B-Q8_0 - Test: Text Generation 128 Tokens Per Second > Higher Is Better a . 19.04 |==================================================================== b . 19.00 |==================================================================== Llama.cpp b4154 Backend: CPU BLAS - Model: Llama-3.1-Tulu-3-8B-Q8_0 - Test: Prompt Processing 512 Tokens Per Second > Higher Is Better a . 96.79 |==================================================================== b . 94.36 |================================================================== Llama.cpp b4154 Backend: CPU BLAS - Model: Llama-3.1-Tulu-3-8B-Q8_0 - Test: Prompt Processing 1024 Tokens Per Second > Higher Is Better a . 103.45 |=================================================================== b . 103.70 |=================================================================== Llama.cpp b4154 Backend: CPU BLAS - Model: Llama-3.1-Tulu-3-8B-Q8_0 - Test: Prompt Processing 2048 Tokens Per Second > Higher Is Better a . 104.84 |=================================================================== b . 104.02 |================================================================== Llama.cpp b4154 Backend: CPU BLAS - Model: Mistral-7B-Instruct-v0.3-Q8_0 - Test: Text Generation 128 Tokens Per Second > Higher Is Better a . 15.38 |================================================================ b . 16.24 |==================================================================== Llama.cpp b4154 Backend: CPU BLAS - Model: Mistral-7B-Instruct-v0.3-Q8_0 - Test: Prompt Processing 512 Tokens Per Second > Higher Is Better a . 96.79 |==================================================================== b . 96.90 |==================================================================== Llama.cpp b4154 Backend: CPU BLAS - Model: Mistral-7B-Instruct-v0.3-Q8_0 - Test: Prompt Processing 1024 Tokens Per Second > Higher Is Better a . 101.01 |================================================================= b . 103.43 |=================================================================== Llama.cpp b4154 Backend: CPU BLAS - Model: Mistral-7B-Instruct-v0.3-Q8_0 - Test: Prompt Processing 2048 Tokens Per Second > Higher Is Better a . 105.45 |=================================================================== b . 105.16 |=================================================================== Llama.cpp b4154 Backend: CPU BLAS - Model: granite-3.0-3b-a800m-instruct-Q8_0 - Test: Text Generation 128 Tokens Per Second > Higher Is Better a . 36.58 |================================================================ b . 38.61 |==================================================================== Llama.cpp b4154 Backend: CPU BLAS - Model: granite-3.0-3b-a800m-instruct-Q8_0 - Test: Prompt Processing 512 Tokens Per Second > Higher Is Better a . 199.65 |=================================================================== b . 194.76 |================================================================= Llama.cpp b4154 Backend: CPU BLAS - Model: granite-3.0-3b-a800m-instruct-Q8_0 - Test: Prompt Processing 1024 Tokens Per Second > Higher Is Better a . 239.94 |=================================================================== b . 229.86 |================================================================ Llama.cpp b4154 Backend: CPU BLAS - Model: granite-3.0-3b-a800m-instruct-Q8_0 - Test: Prompt Processing 2048 Tokens Per Second > Higher Is Better a . 261.20 |=================================================================== b . 260.03 |=================================================================== OpenVINO GenAI 2024.5 Model: Gemma-7b-int4-ov - Device: CPU tokens/s > Higher Is Better a . 5.92 |================================================================== b . 6.23 |===================================================================== OpenVINO GenAI 2024.5 Model: TinyLlama-1.1B-Chat-v1.0 - Device: CPU tokens/s > Higher Is Better a . 18.14 |=========================================================== b . 20.91 |==================================================================== OpenVINO GenAI 2024.5 Model: Falcon-7b-instruct-int4-ov - Device: CPU tokens/s > Higher Is Better a . 9.88 |===================================================================== b . 9.86 |===================================================================== OpenVINO GenAI 2024.5 Model: Phi-3-mini-128k-instruct-int4-ov - Device: CPU tokens/s > Higher Is Better a . 9.58 |==================================================================== b . 9.66 |===================================================================== Renaissance 0.16 Test: Scala Dotty ms < Lower Is Better a . 1360.1 |=================================================================== b . 1318.9 |================================================================= Renaissance 0.16 Test: Random Forest ms < Lower Is Better a . 905.4 |==================================================================== b . 907.6 |==================================================================== Renaissance 0.16 Test: ALS Movie Lens ms < Lower Is Better a . 21263.0 |================================================================== b . 20800.9 |================================================================= Renaissance 0.16 Test: Apache Spark Bayes ms < Lower Is Better a . 331.4 |==================================================================== b . 330.7 |==================================================================== Renaissance 0.16 Test: Savina Reactors.IO ms < Lower Is Better a . 11584.0 |================================================================= b . 11761.5 |================================================================== Renaissance 0.16 Test: Apache Spark PageRank ms < Lower Is Better a . 3933.2 |=================================================================== b . 3959.6 |=================================================================== Renaissance 0.16 Test: Finagle HTTP Requests ms < Lower Is Better a . 6084.3 |=================================================================== b . 5918.8 |================================================================= Renaissance 0.16 Test: Gaussian Mixture Model ms < Lower Is Better a . 6255.5 |=================================================================== b . 6162.5 |================================================================== Renaissance 0.16 Test: In-Memory Database Shootout ms < Lower Is Better a . 14065.8 |================================================================== b . 14026.9 |================================================================== Renaissance 0.16 Test: Akka Unbalanced Cobwebbed Tree ms < Lower Is Better a . 43100.5 |================================================================ b . 44671.6 |================================================================== Renaissance 0.16 Test: Genetic Algorithm Using Jenetics + Futures ms < Lower Is Better a . 2583.2 |=================================================================== b . 2587.8 |=================================================================== OpenVINO 2024.5 Model: Face Detection FP16 - Device: CPU ms < Lower Is Better a . 4567.63 |================================================================== b . 4571.10 |================================================================== OpenVINO 2024.5 Model: Person Detection FP16 - Device: CPU ms < Lower Is Better a . 937.32 |=================================================================== b . 938.79 |=================================================================== OpenVINO 2024.5 Model: Person Detection FP32 - Device: CPU ms < Lower Is Better a . 938.50 |=================================================================== b . 937.77 |=================================================================== OpenVINO 2024.5 Model: Vehicle Detection FP16 - Device: CPU ms < Lower Is Better a . 60.90 |==================================================================== b . 58.17 |================================================================= OpenVINO 2024.5 Model: Face Detection FP16-INT8 - Device: CPU ms < Lower Is Better a . 4788.61 |================================================================== b . 4793.12 |================================================================== OpenVINO 2024.5 Model: Face Detection Retail FP16 - Device: CPU ms < Lower Is Better a . 26.12 |================================================================ b . 27.69 |==================================================================== OpenVINO 2024.5 Model: Road Segmentation ADAS FP16 - Device: CPU ms < Lower Is Better a . 138.60 |=================================================================== b . 137.56 |================================================================== OpenVINO 2024.5 Model: Vehicle Detection FP16-INT8 - Device: CPU ms < Lower Is Better a . 368.95 |=================================================================== b . 367.30 |=================================================================== OpenVINO 2024.5 Model: Weld Porosity Detection FP16 - Device: CPU ms < Lower Is Better a . 63.11 |==================================================================== b . 63.17 |==================================================================== OpenVINO 2024.5 Model: Face Detection Retail FP16-INT8 - Device: CPU ms < Lower Is Better a . 81.42 |==================================================================== b . 81.32 |==================================================================== OpenVINO 2024.5 Model: Road Segmentation ADAS FP16-INT8 - Device: CPU ms < Lower Is Better a . 1021.92 |================================================================== b . 1021.49 |================================================================== OpenVINO 2024.5 Model: Machine Translation EN To DE FP16 - Device: CPU ms < Lower Is Better a . 106.86 |=================================================================== b . 106.21 |=================================================================== OpenVINO 2024.5 Model: Weld Porosity Detection FP16-INT8 - Device: CPU ms < Lower Is Better a . 61.26 |==================================================================== b . 60.83 |==================================================================== OpenVINO 2024.5 Model: Person Vehicle Bike Detection FP16 - Device: CPU ms < Lower Is Better a . 107.33 |================================================================== b . 108.65 |=================================================================== OpenVINO 2024.5 Model: Noise Suppression Poconet-Like FP16 - Device: CPU ms < Lower Is Better a . 350.23 |=================================================================== b . 347.78 |=================================================================== OpenVINO 2024.5 Model: Handwritten English Recognition FP16 - Device: CPU ms < Lower Is Better a . 180.27 |=================================================================== b . 180.37 |=================================================================== OpenVINO 2024.5 Model: Person Re-Identification Retail FP16 - Device: CPU ms < Lower Is Better a . 178.63 |=================================================================== b . 175.83 |================================================================== OpenVINO 2024.5 Model: Age Gender Recognition Retail 0013 FP16 - Device: CPU ms < Lower Is Better a . 17.30 |=================================================================== b . 17.59 |==================================================================== OpenVINO 2024.5 Model: Handwritten English Recognition FP16-INT8 - Device: CPU ms < Lower Is Better a . 208.51 |=================================================================== b . 208.37 |=================================================================== OpenVINO 2024.5 Model: Age Gender Recognition Retail 0013 FP16-INT8 - Device: CPU ms < Lower Is Better a . 17.28 |==================================================================== b . 16.23 |================================================================ OpenVINO GenAI 2024.5 Model: Gemma-7b-int4-ov - Device: CPU - Time To First Token ms < Lower Is Better a . 201.31 |=================================================================== b . 184.88 |============================================================== OpenVINO GenAI 2024.5 Model: Gemma-7b-int4-ov - Device: CPU - Time Per Output Token ms < Lower Is Better a . 168.98 |=================================================================== b . 160.47 |================================================================ OpenVINO GenAI 2024.5 Model: TinyLlama-1.1B-Chat-v1.0 - Device: CPU - Time To First Token ms < Lower Is Better a . 82.43 |==================================================================== b . 68.65 |========================================================= OpenVINO GenAI 2024.5 Model: TinyLlama-1.1B-Chat-v1.0 - Device: CPU - Time Per Output Token ms < Lower Is Better a . 55.13 |==================================================================== b . 47.83 |=========================================================== OpenVINO GenAI 2024.5 Model: Falcon-7b-instruct-int4-ov - Device: CPU - Time To First Token ms < Lower Is Better a . 121.27 |=================================================================== b . 122.07 |=================================================================== OpenVINO GenAI 2024.5 Model: Falcon-7b-instruct-int4-ov - Device: CPU - Time Per Output Token ms < Lower Is Better a . 101.20 |=================================================================== b . 101.42 |=================================================================== OpenVINO GenAI 2024.5 Model: Phi-3-mini-128k-instruct-int4-ov - Device: CPU - Time To First Token ms < Lower Is Better a . 131.88 |=================================================================== b . 131.64 |=================================================================== OpenVINO GenAI 2024.5 Model: Phi-3-mini-128k-instruct-int4-ov - Device: CPU - Time Per Output Token ms < Lower Is Better a . 104.37 |=================================================================== b . 103.51 |================================================================== Primesieve 12.6 Length: 1e12 Seconds < Lower Is Better a . 2.575 |=================================================================== b . 2.598 |==================================================================== Primesieve 12.6 Length: 1e13 Seconds < Lower Is Better a . 41.05 |================================================================= b . 42.81 |====================================================================