eoy2024 Benchmarks for a future article. AMD EPYC 4484PX 12-Core testing with a Supermicro AS-3015A-I H13SAE-MF v1.00 (2.1 BIOS) and ASPEED on Ubuntu 24.04 via the Phoronix Test Suite. a: Processor: AMD EPYC 4564P 16-Core @ 5.88GHz (16 Cores / 32 Threads), Motherboard: Supermicro AS-3015A-I H13SAE-MF v1.00 (2.1 BIOS), Chipset: AMD Device 14d8, Memory: 2 x 32GB DRAM-4800MT/s Micron MTC20C2085S1EC48BA1 BC, Disk: 3201GB Micron_7450_MTFDKCC3T2TFS + 960GB SAMSUNG MZ1L2960HCJR-00A07, Graphics: ASPEED, Audio: AMD Rembrandt Radeon HD Audio, Monitor: VA2431, Network: 2 x Intel I210 OS: Ubuntu 24.04, Kernel: 6.8.0-11-generic (x86_64), Desktop: GNOME Shell 45.3, Display Server: X Server 1.21.1.11, Compiler: GCC 13.2.0, File-System: ext4, Screen Resolution: 1024x768 4484PX: Processor: AMD EPYC 4484PX 12-Core @ 5.66GHz (12 Cores / 24 Threads), Motherboard: Supermicro AS-3015A-I H13SAE-MF v1.00 (2.1 BIOS), Chipset: AMD Device 14d8, Memory: 2 x 32GB DRAM-4800MT/s Micron MTC20C2085S1EC48BA1 BC, Disk: 3201GB Micron_7450_MTFDKCC3T2TFS + 960GB SAMSUNG MZ1L2960HCJR-00A07, Graphics: ASPEED, Audio: AMD Rembrandt Radeon HD Audio, Monitor: VA2431, Network: 2 x Intel I210 OS: Ubuntu 24.04, Kernel: 6.12.2-061202-generic (x86_64), Desktop: GNOME Shell 45.3, Display Server: X Server 1.21.1.11, Compiler: GCC 13.2.0, File-System: ext4, Screen Resolution: 1024x768 px: Processor: AMD EPYC 4484PX 12-Core @ 5.66GHz (12 Cores / 24 Threads), Motherboard: Supermicro AS-3015A-I H13SAE-MF v1.00 (2.1 BIOS), Chipset: AMD Device 14d8, Memory: 2 x 32GB DRAM-4800MT/s Micron MTC20C2085S1EC48BA1 BC, Disk: 3201GB Micron_7450_MTFDKCC3T2TFS + 960GB SAMSUNG MZ1L2960HCJR-00A07, Graphics: ASPEED, Audio: AMD Rembrandt Radeon HD Audio, Monitor: VA2431, Network: 2 x Intel I210 OS: Ubuntu 24.04, Kernel: 6.12.2-061202-generic (x86_64), Desktop: GNOME Shell 45.3, Display Server: X Server 1.21.1.11, Compiler: GCC 13.2.0, File-System: ext4, Screen Resolution: 1024x768 Etcpak 2.0 Benchmark: Multi-Threaded - Configuration: ETC2 Mpx/s > Higher Is Better a ...... 577.82 |============================================================== 4484PX . 410.73 |============================================ px ..... 409.88 |============================================ CP2K Molecular Dynamics 2024.3 Input: H20-64 Seconds < Lower Is Better a ...... 58.19 |=============================================================== 4484PX . 53.01 |========================================================= px ..... 52.72 |========================================================= CP2K Molecular Dynamics 2024.3 Input: H20-256 Seconds < Lower Is Better a ...... 592.86 |========================================================== 4484PX . 628.10 |============================================================== px ..... 631.31 |============================================================== CP2K Molecular Dynamics 2024.3 Input: Fayalite-FIST Seconds < Lower Is Better a ...... 94.03 |============================================================== 4484PX . 92.21 |============================================================= px ..... 94.90 |=============================================================== NAMD 3.0 Input: ATPase with 327,506 Atoms ns/day > Higher Is Better a ...... 2.79632 |============================================================= 4484PX . 2.38124 |==================================================== px ..... 2.35379 |=================================================== NAMD 3.0 Input: STMV with 1,066,628 Atoms ns/day > Higher Is Better a ...... 0.75656 |============================================================= 4484PX . 0.65119 |===================================================== px ..... 0.65448 |===================================================== QuantLib 1.35-dev Size: S tasks/s > Higher Is Better a ...... 12.75 |=============================================================== 4484PX . 11.86 |=========================================================== px ..... 11.84 |=========================================================== QuantLib 1.35-dev Size: XXS tasks/s > Higher Is Better a ...... 13.43 |=============================================================== 4484PX . 12.12 |========================================================= px ..... 12.11 |========================================================= RELION 5.0 Test: Basic - Device: CPU Seconds < Lower Is Better a ...... 944.27 |============================================================== 4484PX . 729.40 |================================================ px ..... 733.02 |================================================ simdjson 3.10 Throughput Test: Kostya GB/s > Higher Is Better a ...... 5.97 |=============================================================== 4484PX . 6.11 |================================================================ px ..... 5.45 |========================================================= simdjson 3.10 Throughput Test: TopTweet GB/s > Higher Is Better a ...... 10.46 |============================================================= 4484PX . 10.82 |=============================================================== px ..... 10.51 |============================================================= simdjson 3.10 Throughput Test: LargeRandom GB/s > Higher Is Better a ...... 1.83 |================================================================ 4484PX . 1.84 |================================================================ px ..... 1.84 |================================================================ simdjson 3.10 Throughput Test: PartialTweets GB/s > Higher Is Better a ...... 9.76 |============================================================= 4484PX . 10.10 |=============================================================== px ..... 8.35 |==================================================== simdjson 3.10 Throughput Test: DistinctUserID GB/s > Higher Is Better a ...... 10.46 |============================================================= 4484PX . 10.76 |=============================================================== px ..... 8.97 |===================================================== Renaissance 0.16 Test: Scala Dotty ms < Lower Is Better a ...... 477.0 |=============================================================== 4484PX . 428.6 |========================================================= px ..... 436.2 |========================================================== Renaissance 0.16 Test: Random Forest ms < Lower Is Better a ...... 414.4 |========================================================== 4484PX . 422.0 |=========================================================== px ..... 453.2 |=============================================================== Renaissance 0.16 Test: ALS Movie Lens ms < Lower Is Better a ...... 9805.7 |============================================================== 4484PX . 9378.8 |=========================================================== px ..... 9275.7 |=========================================================== Renaissance 0.16 Test: Apache Spark ALS ms < Lower Is Better Renaissance 0.16 Test: Apache Spark Bayes ms < Lower Is Better a ...... 490.0 |============================================================ 4484PX . 513.2 |=============================================================== px ..... 474.9 |========================================================== Renaissance 0.16 Test: Savina Reactors.IO ms < Lower Is Better a ...... 3506.4 |=========================================================== 4484PX . 3655.8 |============================================================== px ..... 3676.0 |============================================================== Renaissance 0.16 Test: Apache Spark PageRank ms < Lower Is Better a ...... 2412.2 |============================================================== 4484PX . 2138.1 |======================================================= px ..... 2229.7 |========================================================= Renaissance 0.16 Test: Finagle HTTP Requests ms < Lower Is Better a ...... 2319.4 |========================================================== 4484PX . 2492.2 |============================================================== px ..... 2483.1 |============================================================== Renaissance 0.16 Test: Gaussian Mixture Model ms < Lower Is Better a ...... 3399.5 |======================================================= 4484PX . 3860.6 |============================================================== px ..... 3815.2 |============================================================= Renaissance 0.16 Test: In-Memory Database Shootout ms < Lower Is Better a ...... 3256.1 |============================================================== 4484PX . 3241.5 |============================================================== px ..... 3175.6 |============================================================ Renaissance 0.16 Test: Akka Unbalanced Cobwebbed Tree ms < Lower Is Better a ...... 4403.8 |============================================================== 4484PX . 4038.4 |========================================================= px ..... 4002.3 |======================================================== Renaissance 0.16 Test: Genetic Algorithm Using Jenetics + Futures ms < Lower Is Better a ...... 732.8 |================================================== 4484PX . 904.0 |============================================================== px ..... 920.7 |=============================================================== BYTE Unix Benchmark 5.1.3-git Computational Test: Pipe LPS > Higher Is Better a ...... 48806257.1 |========================================================== 4484PX . 33443359.2 |======================================== px ..... 33381363.1 |======================================== BYTE Unix Benchmark 5.1.3-git Computational Test: Dhrystone 2 LPS > Higher Is Better a ...... 1866536062.7 |======================================================== 4484PX . 1346521770.3 |======================================== px ..... 1340340196.6 |======================================== BYTE Unix Benchmark 5.1.3-git Computational Test: System Call LPS > Higher Is Better a ...... 49140426.6 |========================================================== 4484PX . 30761218.9 |==================================== px ..... 30701622.8 |==================================== BYTE Unix Benchmark 5.1.3-git Computational Test: Whetstone Double MWIPS > Higher Is Better a ...... 343491.9 |============================================================ 4484PX . 244075.3 |=========================================== px ..... 244131.0 |=========================================== SVT-AV1 2.3 Encoder Mode: Preset 3 - Input: Bosphorus 4K Frames Per Second > Higher Is Better a ...... 9.590 |=============================================================== 4484PX . 7.684 |================================================== px ..... 7.646 |================================================== SVT-AV1 2.3 Encoder Mode: Preset 5 - Input: Bosphorus 4K Frames Per Second > Higher Is Better a ...... 34.54 |=============================================================== 4484PX . 29.09 |===================================================== px ..... 28.82 |===================================================== SVT-AV1 2.3 Encoder Mode: Preset 8 - Input: Bosphorus 4K Frames Per Second > Higher Is Better a ...... 102.01 |============================================================== 4484PX . 85.20 |==================================================== px ..... 85.00 |==================================================== SVT-AV1 2.3 Encoder Mode: Preset 13 - Input: Bosphorus 4K Frames Per Second > Higher Is Better a ...... 212.52 |============================================================== 4484PX . 198.11 |========================================================== px ..... 194.02 |========================================================= SVT-AV1 2.3 Encoder Mode: Preset 3 - Input: Bosphorus 1080p Frames Per Second > Higher Is Better a ...... 29.57 |=============================================================== 4484PX . 25.45 |====================================================== px ..... 25.45 |====================================================== SVT-AV1 2.3 Encoder Mode: Preset 5 - Input: Bosphorus 1080p Frames Per Second > Higher Is Better a ...... 101.97 |============================================================== 4484PX . 88.42 |====================================================== px ..... 88.27 |====================================================== SVT-AV1 2.3 Encoder Mode: Preset 8 - Input: Bosphorus 1080p Frames Per Second > Higher Is Better a ...... 339.02 |============================================================== 4484PX . 287.05 |==================================================== px ..... 286.96 |==================================================== SVT-AV1 2.3 Encoder Mode: Preset 13 - Input: Bosphorus 1080p Frames Per Second > Higher Is Better a ...... 842.56 |============================================================== 4484PX . 776.12 |========================================================= px ..... 769.82 |========================================================= SVT-AV1 2.3 Encoder Mode: Preset 3 - Input: Beauty 4K 10-bit Frames Per Second > Higher Is Better a ...... 1.422 |=============================================================== 4484PX . 1.188 |===================================================== px ..... 1.184 |==================================================== SVT-AV1 2.3 Encoder Mode: Preset 5 - Input: Beauty 4K 10-bit Frames Per Second > Higher Is Better a ...... 6.504 |=============================================================== 4484PX . 5.602 |====================================================== px ..... 5.551 |====================================================== SVT-AV1 2.3 Encoder Mode: Preset 8 - Input: Beauty 4K 10-bit Frames Per Second > Higher Is Better a ...... 12.47 |=============================================================== 4484PX . 10.97 |======================================================= px ..... 10.86 |======================================================= SVT-AV1 2.3 Encoder Mode: Preset 13 - Input: Beauty 4K 10-bit Frames Per Second > Higher Is Better a ...... 18.59 |=============================================================== 4484PX . 17.41 |=========================================================== px ..... 17.36 |=========================================================== x265 Video Input: Bosphorus 4K Frames Per Second > Higher Is Better a ...... 32.57 |=============================================================== 4484PX . 27.16 |===================================================== px ..... 26.94 |==================================================== x265 Video Input: Bosphorus 1080p Frames Per Second > Higher Is Better a ...... 114.45 |============================================================== 4484PX . 101.37 |======================================================= px ..... 101.25 |======================================================= ACES DGEMM 1.0 Sustained Floating-Point Rate GFLOP/s > Higher Is Better a ...... 1141.19 |============================================================= 4484PX . 842.73 |============================================= px ..... 842.01 |============================================= OSPRay 3.2 Benchmark: particle_volume/ao/real_time Items Per Second > Higher Is Better a ...... 9.00917 |============================================================= 4484PX . 6.52776 |============================================ px ..... 6.52206 |============================================ OSPRay 3.2 Benchmark: particle_volume/scivis/real_time Items Per Second > Higher Is Better a ...... 8.98486 |============================================================= 4484PX . 6.44913 |============================================ px ..... 6.52304 |============================================ OSPRay 3.2 Benchmark: particle_volume/pathtracer/real_time Items Per Second > Higher Is Better a ...... 236.25 |============================================================== 4484PX . 199.02 |==================================================== px ..... 197.20 |==================================================== OSPRay 3.2 Benchmark: gravity_spheres_volume/dim_512/ao/real_time Items Per Second > Higher Is Better a ...... 7.63944 |============================================================= 4484PX . 5.63122 |============================================= px ..... 5.71084 |============================================== OSPRay 3.2 Benchmark: gravity_spheres_volume/dim_512/scivis/real_time Items Per Second > Higher Is Better a ...... 7.58789 |============================================================= 4484PX . 5.54888 |============================================= px ..... 5.61470 |============================================= OSPRay 3.2 Benchmark: gravity_spheres_volume/dim_512/pathtracer/real_time Items Per Second > Higher Is Better a ...... 8.82093 |============================================================= 4484PX . 6.41198 |============================================ px ..... 6.40740 |============================================ 7-Zip Compression Test: Compression Rating MIPS > Higher Is Better a ...... 163859 |============================================================== 4484PX . 141263 |===================================================== px ..... 142213 |====================================================== 7-Zip Compression Test: Decompression Rating MIPS > Higher Is Better a ...... 165916 |============================================================== 4484PX . 125698 |=============================================== px ..... 125605 |=============================================== Stockfish 17 Chess Benchmark Nodes Per Second > Higher Is Better a ...... 54752796 |============================================================ 4484PX . 45267546 |================================================== px ..... 42973396 |=============================================== Stockfish Chess Benchmark Nodes Per Second > Higher Is Better a ...... 46507038 |============================================================ 4484PX . 33702298 |=========================================== px ..... 33871595 |============================================ Build2 0.17 Time To Compile Seconds < Lower Is Better a ...... 92.05 |================================================== 4484PX . 111.65 |============================================================= px ..... 113.78 |============================================================== Primesieve 12.6 Length: 1e12 Seconds < Lower Is Better a ...... 6.347 |============================================ 4484PX . 9.116 |=============================================================== px ..... 9.147 |=============================================================== Primesieve 12.6 Length: 1e13 Seconds < Lower Is Better a ...... 78.50 |============================================ 4484PX . 110.61 |============================================================== px ..... 110.71 |============================================================== Y-Cruncher 0.8.5 Pi Digits To Calculate: 1B Seconds < Lower Is Better a ...... 18.49 |=============================================================== 4484PX . 18.38 |=============================================================== px ..... 18.37 |=============================================================== Y-Cruncher 0.8.5 Pi Digits To Calculate: 500M Seconds < Lower Is Better a ...... 8.772 |=============================================================== 4484PX . 8.688 |============================================================== px ..... 8.623 |============================================================== POV-Ray Trace Time Seconds < Lower Is Better a ...... 18.54 |============================================== 4484PX . 25.26 |=============================================================== px ..... 25.33 |=============================================================== oneDNN 3.6 Harness: IP Shapes 1D - Engine: CPU ms < Lower Is Better a ...... 1.12573 |=================================== 4484PX . 1.93806 |============================================================= px ..... 1.93913 |============================================================= oneDNN 3.6 Harness: IP Shapes 3D - Engine: CPU ms < Lower Is Better a ...... 4.05800 |============================================================= 4484PX . 2.73072 |========================================= px ..... 2.72942 |========================================= oneDNN 3.6 Harness: Convolution Batch Shapes Auto - Engine: CPU ms < Lower Is Better a ...... 6.67287 |============================================================= 4484PX . 4.11551 |====================================== px ..... 4.13321 |====================================== oneDNN 3.6 Harness: Deconvolution Batch shapes_1d - Engine: CPU ms < Lower Is Better a ...... 2.97612 |===================================================== 4484PX . 3.40293 |============================================================= px ..... 3.40628 |============================================================= oneDNN 3.6 Harness: Deconvolution Batch shapes_3d - Engine: CPU ms < Lower Is Better a ...... 2.41294 |========================================== 4484PX . 3.50840 |============================================================= px ..... 3.51243 |============================================================= oneDNN 3.6 Harness: Recurrent Neural Network Training - Engine: CPU ms < Lower Is Better a ...... 1372.03 |============================================ 4484PX . 1898.36 |============================================================= px ..... 1895.68 |============================================================= oneDNN 3.6 Harness: Recurrent Neural Network Inference - Engine: CPU ms < Lower Is Better a ...... 700.86 |============================================= 4484PX . 965.02 |============================================================== px ..... 966.01 |============================================================== Numpy Benchmark Score > Higher Is Better a ...... 775.75 |========================================================== 4484PX . 745.59 |======================================================== px ..... 831.42 |============================================================== Timed Eigen Compilation 3.4.0 Time To Compile Seconds < Lower Is Better a ...... 58.66 |======================================================= 4484PX . 67.36 |=============================================================== px ..... 67.08 |=============================================================== Gcrypt Library 1.10.3 Seconds < Lower Is Better a ...... 162.13 |=========================================================== 4484PX . 171.02 |============================================================== px ..... 163.84 |=========================================================== Rustls 0.23.17 Benchmark: handshake - Suite: TLS13_CHACHA20_POLY1305_SHA256 handshakes/s > Higher Is Better a ...... 76454.45 |============================================================ 4484PX . 57716.64 |============================================= px ..... 57688.08 |============================================= Rustls 0.23.17 Benchmark: handshake - Suite: TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 handshakes/s > Higher Is Better a ...... 80462.60 |============================================================ 4484PX . 59308.75 |============================================ px ..... 59206.34 |============================================ Rustls 0.23.17 Benchmark: handshake-resume - Suite: TLS13_CHACHA20_POLY1305_SHA256 handshakes/s > Higher Is Better a ...... 388077.69 |=========================================================== 4484PX . 333882.92 |=================================================== px ..... 333574.30 |=================================================== Rustls 0.23.17 Benchmark: handshake-ticket - Suite: TLS13_CHACHA20_POLY1305_SHA256 handshakes/s > Higher Is Better a ...... 404263.45 |=========================================================== 4484PX . 344296.24 |================================================== px ..... 342775.29 |================================================== Rustls 0.23.17 Benchmark: handshake - Suite: TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384 handshakes/s > Higher Is Better a ...... 423535.68 |=========================================================== 4484PX . 306153.20 |=========================================== px ..... 304060.28 |========================================== Rustls 0.23.17 Benchmark: handshake-resume - Suite: TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 handshakes/s > Higher Is Better a ...... 3563852.57 |========================================================== 4484PX . 3035330.21 |================================================= px ..... 3038723.48 |================================================= Rustls 0.23.17 Benchmark: handshake-ticket - Suite: TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 handshakes/s > Higher Is Better a ...... 2620332.00 |========================================================== 4484PX . 2282729.64 |=================================================== px ..... 2292879.44 |=================================================== Rustls 0.23.17 Benchmark: handshake-resume - Suite: TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384 handshakes/s > Higher Is Better a ...... 1820810.21 |========================================================== 4484PX . 1586292.42 |=================================================== px ..... 1572010.68 |================================================== Rustls 0.23.17 Benchmark: handshake-ticket - Suite: TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384 handshakes/s > Higher Is Better a ...... 1553632.14 |========================================================== 4484PX . 1329363.10 |================================================== px ..... 1340712.85 |================================================== OpenSSL Algorithm: SHA256 OpenSSL Algorithm: SHA512 OpenSSL Algorithm: RSA4096 OpenSSL Algorithm: ChaCha20 byte/s > Higher Is Better a ...... 130588495050 |======================================================== 4484PX . 97105235690 |========================================== px ..... 97019897450 |========================================== OpenSSL Algorithm: AES-128-GCM byte/s > Higher Is Better a ...... 104784522170 |======================================================== 4484PX . 76496336760 |========================================= px ..... 76184405610 |========================================= OpenSSL Algorithm: AES-256-GCM byte/s > Higher Is Better a ...... 97172751700 |========================================================= 4484PX . 71160291870 |========================================== px ..... 70902656480 |========================================== OpenSSL Algorithm: ChaCha20-Poly1305 byte/s > Higher Is Better a ...... 92393529340 |========================================================= 4484PX . 68816544020 |========================================== px ..... 68678955550 |========================================== Apache CouchDB 3.4.1 Bulk Size: 100 - Inserts: 1000 - Rounds: 30 Seconds < Lower Is Better a ...... 69.93 |========================================================== 4484PX . 75.90 |=============================================================== px ..... 76.39 |=============================================================== Apache CouchDB 3.4.1 Bulk Size: 100 - Inserts: 3000 - Rounds: 30 Seconds < Lower Is Better a ...... 232.19 |========================================================= 4484PX . 253.99 |============================================================== px ..... 254.73 |============================================================== Apache CouchDB 3.4.1 Bulk Size: 300 - Inserts: 1000 - Rounds: 30 Seconds < Lower Is Better a ...... 106.13 |======================================================= 4484PX . 117.57 |============================================================= px ..... 119.35 |============================================================== Apache CouchDB 3.4.1 Bulk Size: 300 - Inserts: 3000 - Rounds: 30 Seconds < Lower Is Better a ...... 367.83 |======================================================== 4484PX . 406.12 |============================================================== px ..... 408.48 |============================================================== Apache CouchDB 3.4.1 Bulk Size: 500 - Inserts: 1000 - Rounds: 30 Seconds < Lower Is Better a ...... 148.05 |======================================================== 4484PX . 164.47 |============================================================== px ..... 164.81 |============================================================== Apache CouchDB 3.4.1 Bulk Size: 500 - Inserts: 3000 - Rounds: 30 Seconds < Lower Is Better a ...... 511.78 |========================================================= 4484PX . 559.35 |============================================================== px ..... 560.70 |============================================================== FinanceBench 2016-07-25 Benchmark: Repo OpenMP ms < Lower Is Better a ...... 21418.45 |========================================================== 4484PX . 22320.33 |============================================================ px ..... 22318.74 |============================================================ FinanceBench 2016-07-25 Benchmark: Bonds OpenMP ms < Lower Is Better a ...... 33061.22 |========================================================= 4484PX . 34600.77 |=========================================================== px ..... 34896.84 |============================================================ ASTC Encoder 5.0 Preset: Fast MT/s > Higher Is Better a ...... 396.65 |============================================================== 4484PX . 278.24 |=========================================== px ..... 277.30 |=========================================== ASTC Encoder 5.0 Preset: Medium MT/s > Higher Is Better a ...... 156.22 |============================================================== 4484PX . 109.03 |=========================================== px ..... 108.86 |=========================================== ASTC Encoder 5.0 Preset: Thorough MT/s > Higher Is Better a ...... 20.30 |=============================================================== 4484PX . 14.17 |============================================ px ..... 14.15 |============================================ ASTC Encoder 5.0 Preset: Exhaustive MT/s > Higher Is Better a ...... 1.6844 |============================================================== 4484PX . 1.1887 |============================================ px ..... 1.1862 |============================================ ASTC Encoder 5.0 Preset: Very Thorough MT/s > Higher Is Better a ...... 2.7410 |============================================================== 4484PX . 1.9412 |============================================ px ..... 1.9391 |============================================ GROMACS Input: water_GMX50_bare Ns Per Day > Higher Is Better a ...... 1.692 |=============================================================== 4484PX . 1.577 |=========================================================== px ..... 1.575 |=========================================================== LiteRT 2024-10-15 Model: DeepLab V3 Microseconds < Lower Is Better a ...... 3579.67 |============================================================= 4484PX . 2343.38 |======================================== px ..... 2359.99 |======================================== LiteRT 2024-10-15 Model: SqueezeNet Microseconds < Lower Is Better a ...... 1794.11 |============================================================ 4484PX . 1809.18 |============================================================= px ..... 1821.35 |============================================================= LiteRT 2024-10-15 Model: Inception V4 Microseconds < Lower Is Better a ...... 21477.8 |========================================================== 4484PX . 22083.3 |=========================================================== px ..... 22752.4 |============================================================= LiteRT 2024-10-15 Model: NASNet Mobile Microseconds < Lower Is Better a ...... 16936.00 |============================================================ 4484PX . 8057.56 |============================= px ..... 7931.64 |============================ LiteRT 2024-10-15 Model: Mobilenet Float Microseconds < Lower Is Better a ...... 1211.48 |=========================================================== 4484PX . 1244.70 |============================================================= px ..... 1244.51 |============================================================= LiteRT 2024-10-15 Model: Mobilenet Quant Microseconds < Lower Is Better a ...... 823.17 |============================================================ 4484PX . 848.94 |============================================================== px ..... 849.21 |============================================================== LiteRT 2024-10-15 Model: Inception ResNet V2 Microseconds < Lower Is Better a ...... 19530.2 |============================================================= 4484PX . 19477.8 |============================================================= px ..... 19490.7 |============================================================= LiteRT 2024-10-15 Model: Quantized COCO SSD MobileNet v1 Microseconds < Lower Is Better a ...... 2129.52 |============================================================= 4484PX . 1420.15 |========================================= px ..... 1417.35 |========================================= XNNPACK b7b048 Model: FP32MobileNetV1 us < Lower Is Better a ...... 1252 |=============================================================== 4484PX . 1257 |=============================================================== px ..... 1272 |================================================================ XNNPACK b7b048 Model: FP32MobileNetV2 us < Lower Is Better a ...... 1495 |================================================================ 4484PX . 1365 |========================================================== px ..... 1368 |=========================================================== XNNPACK b7b048 Model: FP32MobileNetV3Large us < Lower Is Better a ...... 1810 |================================================================ 4484PX . 1515 |====================================================== px ..... 1574 |======================================================== XNNPACK b7b048 Model: FP32MobileNetV3Small us < Lower Is Better a ...... 979 |================================================================= 4484PX . 809 |====================================================== px ..... 837 |======================================================== XNNPACK b7b048 Model: FP16MobileNetV1 us < Lower Is Better a ...... 1143 |===================================================== 4484PX . 1383 |================================================================ px ..... 1386 |================================================================ XNNPACK b7b048 Model: FP16MobileNetV2 us < Lower Is Better a ...... 1190 |============================================================= 4484PX . 1217 |============================================================== px ..... 1248 |================================================================ XNNPACK b7b048 Model: FP16MobileNetV3Large us < Lower Is Better a ...... 1498 |=============================================================== 4484PX . 1467 |============================================================= px ..... 1527 |================================================================ XNNPACK b7b048 Model: FP16MobileNetV3Small us < Lower Is Better a ...... 920 |================================================================= 4484PX . 779 |======================================================= px ..... 798 |======================================================== XNNPACK b7b048 Model: QS8MobileNetV2 us < Lower Is Better a ...... 844 |================================================================= 4484PX . 717 |======================================================= px ..... 723 |======================================================== Blender 4.3 Blend File: BMW27 - Compute: CPU-Only Seconds < Lower Is Better a ...... 53.55 |============================================== 4484PX . 74.08 |=============================================================== px ..... 73.16 |============================================================== Blender 4.3 Blend File: Junkshop - Compute: CPU-Only Seconds < Lower Is Better a ...... 73.56 |================================================ 4484PX . 97.01 |=============================================================== px ..... 97.10 |=============================================================== Blender 4.3 Blend File: Classroom - Compute: CPU-Only Seconds < Lower Is Better a ...... 143.36 |============================================= 4484PX . 197.20 |============================================================== px ..... 197.53 |============================================================== Blender 4.3 Blend File: Fishy Cat - Compute: CPU-Only Seconds < Lower Is Better a ...... 71.35 |============================================== 4484PX . 96.67 |=============================================================== px ..... 97.09 |=============================================================== Blender 4.3 Blend File: Barbershop - Compute: CPU-Only Seconds < Lower Is Better a ...... 506.20 |============================================== 4484PX . 679.34 |============================================================== px ..... 678.40 |============================================================== Blender 4.3 Blend File: Pabellon Barcelona - Compute: CPU-Only Seconds < Lower Is Better a ...... 166.12 |============================================== 4484PX . 226.34 |============================================================== px ..... 224.64 |============================================================== Apache Cassandra 5.0 Test: Writes Op/s > Higher Is Better a ...... 271333 |============================================================== 4484PX . 174960 |======================================== px ..... 173946 |======================================== PyPerformance 1.11 Benchmark: go Milliseconds < Lower Is Better a ...... 77.8 |=============================================================== 4484PX . 78.6 |=============================================================== px ..... 79.4 |================================================================ PyPerformance 1.11 Benchmark: chaos Milliseconds < Lower Is Better a ...... 38.2 |============================================================== 4484PX . 39.7 |================================================================ px ..... 39.4 |================================================================ PyPerformance 1.11 Benchmark: float Milliseconds < Lower Is Better a ...... 50.7 |=============================================================== 4484PX . 51.3 |================================================================ px ..... 50.8 |=============================================================== PyPerformance 1.11 Benchmark: nbody Milliseconds < Lower Is Better a ...... 59.0 |=============================================================== 4484PX . 59.5 |================================================================ px ..... 59.2 |================================================================ PyPerformance 1.11 Benchmark: pathlib Milliseconds < Lower Is Better a ...... 14.2 |=============================================================== 4484PX . 14.4 |================================================================ px ..... 14.4 |================================================================ PyPerformance 1.11 Benchmark: raytrace Milliseconds < Lower Is Better a ...... 175 |=============================================================== 4484PX . 182 |================================================================= px ..... 182 |================================================================= PyPerformance 1.11 Benchmark: xml_etree Milliseconds < Lower Is Better a ...... 35.8 |============================================================== 4484PX . 36.8 |================================================================ px ..... 36.5 |=============================================================== PyPerformance 1.11 Benchmark: gc_collect Milliseconds < Lower Is Better a ...... 677 |============================================================== 4484PX . 699 |================================================================ px ..... 706 |================================================================= PyPerformance 1.11 Benchmark: json_loads Milliseconds < Lower Is Better a ...... 12.1 |============================================================== 4484PX . 12.4 |=============================================================== px ..... 12.5 |================================================================ PyPerformance 1.11 Benchmark: crypto_pyaes Milliseconds < Lower Is Better a ...... 41.7 |============================================================== 4484PX . 43.1 |================================================================ px ..... 43.3 |================================================================ PyPerformance 1.11 Benchmark: async_tree_io Milliseconds < Lower Is Better a ...... 755 |================================================================= 4484PX . 666 |========================================================= px ..... 656 |======================================================== PyPerformance 1.11 Benchmark: regex_compile Milliseconds < Lower Is Better a ...... 69.8 |============================================================== 4484PX . 71.7 |=============================================================== px ..... 72.5 |================================================================ PyPerformance 1.11 Benchmark: python_startup Milliseconds < Lower Is Better a ...... 5.77 |============================================================= 4484PX . 6.08 |================================================================ px ..... 6.09 |================================================================ PyPerformance 1.11 Benchmark: asyncio_tcp_ssl Milliseconds < Lower Is Better a ...... 645 |================================================================= 4484PX . 590 |=========================================================== px ..... 590 |=========================================================== PyPerformance 1.11 Benchmark: django_template Milliseconds < Lower Is Better a ...... 20.7 |============================================================== 4484PX . 21.0 |=============================================================== px ..... 21.2 |================================================================ PyPerformance 1.11 Benchmark: asyncio_websockets Milliseconds < Lower Is Better a ...... 315 |================================================================ 4484PX . 321 |================================================================= px ..... 322 |================================================================= PyPerformance 1.11 Benchmark: pickle_pure_python Milliseconds < Lower Is Better a ...... 165 |=============================================================== 4484PX . 169 |================================================================= px ..... 168 |================================================================= ONNX Runtime 1.19 Model: GPT-2 - Device: CPU - Executor: Standard Inferences Per Second > Higher Is Better a ...... 134.60 |==================================================== 4484PX . 159.71 |============================================================== px ..... 157.89 |============================================================= ONNX Runtime 1.19 Model: GPT-2 - Device: CPU - Executor: Standard Inference Time Cost (ms) < Lower Is Better a ...... 7.42776 |============================================================= 4484PX . 6.25815 |=================================================== px ..... 6.33034 |==================================================== ONNX Runtime 1.19 Model: yolov4 - Device: CPU - Executor: Standard Inferences Per Second > Higher Is Better a ...... 11.06 |=============================================================== 4484PX . 10.73 |============================================================= px ..... 10.71 |============================================================= ONNX Runtime 1.19 Model: yolov4 - Device: CPU - Executor: Standard Inference Time Cost (ms) < Lower Is Better a ...... 90.45 |============================================================= 4484PX . 93.16 |=============================================================== px ..... 93.34 |=============================================================== ONNX Runtime 1.19 Model: ZFNet-512 - Device: CPU - Executor: Standard Inferences Per Second > Higher Is Better a ...... 102.33 |========================================================= 4484PX . 110.94 |============================================================== px ..... 110.89 |============================================================== ONNX Runtime 1.19 Model: ZFNet-512 - Device: CPU - Executor: Standard Inference Time Cost (ms) < Lower Is Better a ...... 9.76985 |============================================================= 4484PX . 9.01322 |======================================================== px ..... 9.01687 |======================================================== ONNX Runtime 1.19 Model: T5 Encoder - Device: CPU - Executor: Standard Inferences Per Second > Higher Is Better a ...... 156.45 |=============================================== 4484PX . 208.17 |============================================================== px ..... 206.09 |============================================================= ONNX Runtime 1.19 Model: T5 Encoder - Device: CPU - Executor: Standard Inference Time Cost (ms) < Lower Is Better a ...... 6.39112 |============================================================= 4484PX . 4.80287 |============================================== px ..... 4.85142 |============================================== ONNX Runtime 1.19 Model: bertsquad-12 - Device: CPU - Executor: Standard Inferences Per Second > Higher Is Better a ...... 15.59 |=============================================================== 4484PX . 14.51 |=========================================================== px ..... 14.57 |=========================================================== ONNX Runtime 1.19 Model: bertsquad-12 - Device: CPU - Executor: Standard Inference Time Cost (ms) < Lower Is Better a ...... 64.14 |=========================================================== 4484PX . 68.91 |=============================================================== px ..... 68.61 |=============================================================== ONNX Runtime 1.19 Model: CaffeNet 12-int8 - Device: CPU - Executor: Standard Inferences Per Second > Higher Is Better a ...... 636.32 |========================================== 4484PX . 941.40 |============================================================== px ..... 937.78 |============================================================== ONNX Runtime 1.19 Model: CaffeNet 12-int8 - Device: CPU - Executor: Standard Inference Time Cost (ms) < Lower Is Better a ...... 1.57084 |============================================================= 4484PX . 1.06188 |========================================= px ..... 1.06600 |========================================= ONNX Runtime 1.19 Model: fcn-resnet101-11 - Device: CPU - Executor: Standard Inferences Per Second > Higher Is Better a ...... 3.21670 |============================================================= 4484PX . 2.81093 |===================================================== px ..... 2.79638 |===================================================== ONNX Runtime 1.19 Model: fcn-resnet101-11 - Device: CPU - Executor: Standard Inference Time Cost (ms) < Lower Is Better a ...... 310.88 |====================================================== 4484PX . 355.75 |============================================================== px ..... 357.60 |============================================================== ONNX Runtime 1.19 Model: ArcFace ResNet-100 - Device: CPU - Executor: Standard Inferences Per Second > Higher Is Better a ...... 42.45 |=============================================================== 4484PX . 37.38 |======================================================= px ..... 37.10 |======================================================= ONNX Runtime 1.19 Model: ArcFace ResNet-100 - Device: CPU - Executor: Standard Inference Time Cost (ms) < Lower Is Better a ...... 23.55 |======================================================= 4484PX . 26.75 |=============================================================== px ..... 26.95 |=============================================================== ONNX Runtime 1.19 Model: ResNet50 v1-12-int8 - Device: CPU - Executor: Standard Inferences Per Second > Higher Is Better a ...... 390.60 |============================================================== 4484PX . 356.41 |========================================================= px ..... 356.19 |========================================================= ONNX Runtime 1.19 Model: ResNet50 v1-12-int8 - Device: CPU - Executor: Standard Inference Time Cost (ms) < Lower Is Better a ...... 2.55898 |======================================================== 4484PX . 2.80544 |============================================================= px ..... 2.80695 |============================================================= ONNX Runtime 1.19 Model: super-resolution-10 - Device: CPU - Executor: Standard Inferences Per Second > Higher Is Better a ...... 141.12 |============================================================== 4484PX . 125.17 |======================================================= px ..... 125.08 |======================================================= ONNX Runtime 1.19 Model: super-resolution-10 - Device: CPU - Executor: Standard Inference Time Cost (ms) < Lower Is Better a ...... 7.08601 |====================================================== 4484PX . 7.98873 |============================================================= px ..... 7.99486 |============================================================= ONNX Runtime 1.19 Model: ResNet101_DUC_HDC-12 - Device: CPU - Executor: Standard Inferences Per Second > Higher Is Better a ...... 1.54196 |============================================================= 4484PX . 1.17627 |=============================================== px ..... 1.17050 |============================================== ONNX Runtime 1.19 Model: ResNet101_DUC_HDC-12 - Device: CPU - Executor: Standard Inference Time Cost (ms) < Lower Is Better a ...... 648.52 |=============================================== 4484PX . 850.14 |============================================================== px ..... 854.33 |============================================================== ONNX Runtime 1.19 Model: Faster R-CNN R-50-FPN-int8 - Device: CPU - Executor: Standard Inferences Per Second > Higher Is Better a ...... 47.07 |=============================================================== 4484PX . 40.09 |====================================================== px ..... 43.36 |========================================================== ONNX Runtime 1.19 Model: Faster R-CNN R-50-FPN-int8 - Device: CPU - Executor: Standard Inference Time Cost (ms) < Lower Is Better a ...... 21.24 |====================================================== 4484PX . 24.94 |=============================================================== px ..... 23.06 |========================================================== Whisper.cpp 1.6.2 Model: ggml-base.en - Input: 2016 State of the Union Seconds < Lower Is Better a ...... 87.49 |=========================================================== 4484PX . 92.71 |=============================================================== px ..... 93.45 |=============================================================== Whisper.cpp 1.6.2 Model: ggml-small.en - Input: 2016 State of the Union Seconds < Lower Is Better a ...... 245.08 |========================================================= 4484PX . 268.24 |============================================================== px ..... 266.81 |============================================================== Whisper.cpp 1.6.2 Model: ggml-medium.en - Input: 2016 State of the Union Seconds < Lower Is Better a ...... 700.91 |====================================================== 4484PX . 809.79 |============================================================== px ..... 809.49 |============================================================== Whisperfile 20Aug24 Model Size: Tiny Seconds < Lower Is Better a ...... 41.71 |=============================================================== 4484PX . 37.13 |======================================================== px ..... 38.72 |========================================================== Whisperfile 20Aug24 Model Size: Small Seconds < Lower Is Better a ...... 195.42 |============================================================== 4484PX . 173.38 |======================================================= px ..... 167.89 |===================================================== Whisperfile 20Aug24 Model Size: Medium Seconds < Lower Is Better a ...... 534.92 |============================================================== 4484PX . 473.55 |======================================================= px ..... 475.51 |======================================================= Llama.cpp b4154 Backend: CPU BLAS - Model: Llama-3.1-Tulu-3-8B-Q8_0 - Test: Text Generation 128 Tokens Per Second > Higher Is Better a ...... 6.88 |============================================================== 4484PX . 7.11 |================================================================ px ..... 7.12 |================================================================ Llama.cpp b4154 Backend: CPU BLAS - Model: Llama-3.1-Tulu-3-8B-Q8_0 - Test: Prompt Processing 512 Tokens Per Second > Higher Is Better a ...... 70.76 |=============================================================== 4484PX . 69.11 |============================================================== px ..... 67.95 |============================================================ Llama.cpp b4154 Backend: CPU BLAS - Model: Llama-3.1-Tulu-3-8B-Q8_0 - Test: Prompt Processing 1024 Tokens Per Second > Higher Is Better a ...... 70.85 |=============================================================== 4484PX . 66.57 |=========================================================== px ..... 66.35 |=========================================================== Llama.cpp b4154 Backend: CPU BLAS - Model: Llama-3.1-Tulu-3-8B-Q8_0 - Test: Prompt Processing 2048 Tokens Per Second > Higher Is Better a ...... 63.09 |============================================================== 4484PX . 63.80 |=============================================================== px ..... 63.79 |=============================================================== Llama.cpp b4154 Backend: CPU BLAS - Model: Mistral-7B-Instruct-v0.3-Q8_0 - Test: Text Generation 128 Tokens Per Second > Higher Is Better a ...... 7.24 |============================================================== 4484PX . 7.41 |================================================================ px ..... 7.44 |================================================================ Llama.cpp b4154 Backend: CPU BLAS - Model: Mistral-7B-Instruct-v0.3-Q8_0 - Test: Prompt Processing 512 Tokens Per Second > Higher Is Better a ...... 68.40 |=============================================================== 4484PX . 68.20 |============================================================== px ..... 68.81 |=============================================================== Llama.cpp b4154 Backend: CPU BLAS - Model: Mistral-7B-Instruct-v0.3-Q8_0 - Test: Prompt Processing 1024 Tokens Per Second > Higher Is Better a ...... 69.26 |=============================================================== 4484PX . 66.85 |============================================================= px ..... 66.52 |============================================================= Llama.cpp b4154 Backend: CPU BLAS - Model: Mistral-7B-Instruct-v0.3-Q8_0 - Test: Prompt Processing 2048 Tokens Per Second > Higher Is Better a ...... 62.97 |============================================================== 4484PX . 63.61 |=============================================================== px ..... 63.41 |=============================================================== Llama.cpp b4154 Backend: CPU BLAS - Model: granite-3.0-3b-a800m-instruct-Q8_0 - Test: Text Generation 128 Tokens Per Second > Higher Is Better a ...... 47.72 |========================================================= 4484PX . 52.30 |=============================================================== px ..... 52.37 |=============================================================== Llama.cpp b4154 Backend: CPU BLAS - Model: granite-3.0-3b-a800m-instruct-Q8_0 - Test: Prompt Processing 512 Tokens Per Second > Higher Is Better a ...... 327.30 |============================================================== 4484PX . 243.14 |============================================== px ..... 232.86 |============================================ Llama.cpp b4154 Backend: CPU BLAS - Model: granite-3.0-3b-a800m-instruct-Q8_0 - Test: Prompt Processing 1024 Tokens Per Second > Higher Is Better a ...... 355.09 |============================================================== 4484PX . 232.26 |========================================= px ..... 244.77 |=========================================== Llama.cpp b4154 Backend: CPU BLAS - Model: granite-3.0-3b-a800m-instruct-Q8_0 - Test: Prompt Processing 2048 Tokens Per Second > Higher Is Better a ...... 279.04 |============================================================== 4484PX . 222.75 |================================================= px ..... 208.99 |============================================== Llamafile 0.8.16 Model: Llama-3.2-3B-Instruct.Q6_K - Test: Text Generation 16 Tokens Per Second > Higher Is Better a ...... 19.03 |============================================================= 4484PX . 19.49 |=============================================================== px ..... 19.50 |=============================================================== Llamafile 0.8.16 Model: Llama-3.2-3B-Instruct.Q6_K - Test: Text Generation 128 Tokens Per Second > Higher Is Better a ...... 20.13 |============================================================== 4484PX . 20.39 |=============================================================== px ..... 20.51 |=============================================================== Llamafile 0.8.16 Model: Llama-3.2-3B-Instruct.Q6_K - Test: Prompt Processing 256 Tokens Per Second > Higher Is Better a ...... 4096 |================================================================ 4484PX . 4096 |================================================================ px ..... 4096 |================================================================ Llamafile 0.8.16 Model: Llama-3.2-3B-Instruct.Q6_K - Test: Prompt Processing 512 Tokens Per Second > Higher Is Better a ...... 8192 |================================================================ 4484PX . 8192 |================================================================ px ..... 8192 |================================================================ Llamafile 0.8.16 Model: TinyLlama-1.1B-Chat-v1.0.BF16 - Test: Text Generation 16 Tokens Per Second > Higher Is Better a ...... 24.59 |============================================================ 4484PX . 25.86 |=============================================================== px ..... 25.94 |=============================================================== Llamafile 0.8.16 Model: Llama-3.2-3B-Instruct.Q6_K - Test: Prompt Processing 1024 Tokens Per Second > Higher Is Better a ...... 16384 |=============================================================== 4484PX . 16384 |=============================================================== px ..... 16384 |=============================================================== Llamafile 0.8.16 Model: Llama-3.2-3B-Instruct.Q6_K - Test: Prompt Processing 2048 Tokens Per Second > Higher Is Better a ...... 32768 |=============================================================== 4484PX . 32768 |=============================================================== px ..... 32768 |=============================================================== Llamafile 0.8.16 Model: TinyLlama-1.1B-Chat-v1.0.BF16 - Test: Text Generation 128 Tokens Per Second > Higher Is Better a ...... 26.28 |============================================================ 4484PX . 27.59 |=============================================================== px ..... 27.80 |=============================================================== Llamafile 0.8.16 Model: mistral-7b-instruct-v0.2.Q5_K_M - Test: Text Generation 16 Tokens Per Second > Higher Is Better a ...... 10.22 |============================================================== 4484PX . 10.45 |=============================================================== px ..... 10.45 |=============================================================== Llamafile 0.8.16 Model: TinyLlama-1.1B-Chat-v1.0.BF16 - Test: Prompt Processing 256 Tokens Per Second > Higher Is Better a ...... 4096 |================================================================ 4484PX . 4096 |================================================================ px ..... 4096 |================================================================ Llamafile 0.8.16 Model: TinyLlama-1.1B-Chat-v1.0.BF16 - Test: Prompt Processing 512 Tokens Per Second > Higher Is Better a ...... 8192 |================================================================ 4484PX . 8192 |================================================================ px ..... 8192 |================================================================ Llamafile 0.8.16 Model: mistral-7b-instruct-v0.2.Q5_K_M - Test: Text Generation 128 Tokens Per Second > Higher Is Better a ...... 10.47 |============================================================ 4484PX . 10.91 |=============================================================== px ..... 10.93 |=============================================================== Llamafile 0.8.16 Model: wizardcoder-python-34b-v1.0.Q6_K - Test: Text Generation 16 Tokens Per Second > Higher Is Better a ...... 1.78 |============================================================== 4484PX . 1.83 |================================================================ px ..... 1.84 |================================================================ Llamafile 0.8.16 Model: TinyLlama-1.1B-Chat-v1.0.BF16 - Test: Prompt Processing 1024 Tokens Per Second > Higher Is Better a ...... 16384 |=============================================================== 4484PX . 16384 |=============================================================== px ..... 16384 |=============================================================== Llamafile 0.8.16 Model: TinyLlama-1.1B-Chat-v1.0.BF16 - Test: Prompt Processing 2048 Tokens Per Second > Higher Is Better a ...... 32768 |=============================================================== 4484PX . 32768 |=============================================================== px ..... 32768 |=============================================================== Llamafile 0.8.16 Model: wizardcoder-python-34b-v1.0.Q6_K - Test: Text Generation 128 Tokens Per Second > Higher Is Better a ...... 1.99 |============================================================== 4484PX . 2.05 |================================================================ px ..... 2.05 |================================================================ Llamafile 0.8.16 Model: mistral-7b-instruct-v0.2.Q5_K_M - Test: Prompt Processing 256 Tokens Per Second > Higher Is Better a ...... 4096 |================================================================ 4484PX . 4096 |================================================================ px ..... 4096 |================================================================ Llamafile 0.8.16 Model: mistral-7b-instruct-v0.2.Q5_K_M - Test: Prompt Processing 512 Tokens Per Second > Higher Is Better a ...... 8192 |================================================================ 4484PX . 8192 |================================================================ px ..... 8192 |================================================================ Llamafile 0.8.16 Model: mistral-7b-instruct-v0.2.Q5_K_M - Test: Prompt Processing 1024 Tokens Per Second > Higher Is Better a ...... 16384 |=============================================================== 4484PX . 16384 |=============================================================== px ..... 16384 |=============================================================== Llamafile 0.8.16 Model: mistral-7b-instruct-v0.2.Q5_K_M - Test: Prompt Processing 2048 Tokens Per Second > Higher Is Better a ...... 32768 |=============================================================== 4484PX . 32768 |=============================================================== px ..... 32768 |=============================================================== Llamafile 0.8.16 Model: wizardcoder-python-34b-v1.0.Q6_K - Test: Prompt Processing 256 Tokens Per Second > Higher Is Better a ...... 1536 |================================================================ 4484PX . 1536 |================================================================ px ..... 1536 |================================================================ Llamafile 0.8.16 Model: wizardcoder-python-34b-v1.0.Q6_K - Test: Prompt Processing 512 Tokens Per Second > Higher Is Better a ...... 3072 |================================================================ 4484PX . 3072 |================================================================ px ..... 3072 |================================================================ Llamafile 0.8.16 Model: wizardcoder-python-34b-v1.0.Q6_K - Test: Prompt Processing 1024 Tokens Per Second > Higher Is Better a ...... 6144 |================================================================ 4484PX . 6144 |================================================================ px ..... 6144 |================================================================ Llamafile 0.8.16 Model: wizardcoder-python-34b-v1.0.Q6_K - Test: Prompt Processing 2048 Tokens Per Second > Higher Is Better a ...... 12288 |=============================================================== 4484PX . 12288 |=============================================================== px ..... 12288 |=============================================================== OpenVINO GenAI 2024.5 Model: Gemma-7b-int4-ov - Device: CPU tokens/s > Higher Is Better a ...... 9.83 |============================================================ 4484PX . 10.23 |=============================================================== px ..... 10.24 |=============================================================== OpenVINO GenAI 2024.5 Model: Gemma-7b-int4-ov - Device: CPU - Time To First Token ms < Lower Is Better a ...... 106.62 |====================================================== 4484PX . 121.48 |============================================================== px ..... 122.30 |============================================================== OpenVINO GenAI 2024.5 Model: Gemma-7b-int4-ov - Device: CPU - Time Per Output Token ms < Lower Is Better a ...... 101.72 |============================================================== 4484PX . 97.79 |============================================================ px ..... 97.61 |=========================================================== OpenVINO GenAI 2024.5 Model: TinyLlama-1.1B-Chat-v1.0 - Device: CPU tokens/s > Higher Is Better OpenVINO GenAI 2024.5 Model: Falcon-7b-instruct-int4-ov - Device: CPU tokens/s > Higher Is Better a ...... 12.93 |============================================================= 4484PX . 13.40 |=============================================================== px ..... 13.41 |=============================================================== OpenVINO GenAI 2024.5 Model: Falcon-7b-instruct-int4-ov - Device: CPU - Time To First Token ms < Lower Is Better a ...... 86.06 |========================================================== 4484PX . 93.01 |=============================================================== px ..... 93.00 |=============================================================== OpenVINO GenAI 2024.5 Model: Falcon-7b-instruct-int4-ov - Device: CPU - Time Per Output Token ms < Lower Is Better a ...... 77.34 |=============================================================== 4484PX . 74.65 |============================================================= px ..... 74.54 |============================================================= OpenVINO GenAI 2024.5 Model: Phi-3-mini-128k-instruct-int4-ov - Device: CPU tokens/s > Higher Is Better a ...... 19.28 |============================================================ 4484PX . 20.28 |=============================================================== px ..... 20.29 |=============================================================== OpenVINO GenAI 2024.5 Model: Phi-3-mini-128k-instruct-int4-ov - Device: CPU - Time To First Token ms < Lower Is Better a ...... 55.93 |============================================================ 4484PX . 58.91 |=============================================================== px ..... 58.86 |=============================================================== OpenVINO GenAI 2024.5 Model: Phi-3-mini-128k-instruct-int4-ov - Device: CPU - Time Per Output Token ms < Lower Is Better a ...... 51.86 |=============================================================== 4484PX . 49.31 |============================================================ px ..... 49.28 |============================================================