Linux 6.13 AMD EPYC Performance Benchmarks for a future article. AMD EPYC 9575F 64-Core testing with a Supermicro Super Server H13SSL-N v1.01 (3.0 BIOS) and ASPEED on Ubuntu 24.10 via the Phoronix Test Suite. v6.12: Processor: AMD EPYC 9575F 64-Core @ 3.30GHz (64 Cores / 128 Threads), Motherboard: Supermicro Super Server H13SSL-N v1.01 (3.0 BIOS), Chipset: AMD 1Ah, Memory: 12 x 64GB DDR5-6000MT/s Micron MTC40F2046S1RC64BDY QSFF, Disk: 3201GB Micron_7450_MTFDKCB3T2TFS, Graphics: ASPEED, Network: 2 x Broadcom NetXtreme BCM5720 PCIe OS: Ubuntu 24.10, Kernel: 6.12.0-phx (x86_64), Desktop: GNOME Shell 47.0, Display Server: X Server, Compiler: GCC 14.2.0, File-System: ext4, Screen Resolution: 1024x768 v6.11: Processor: AMD EPYC 9575F 64-Core @ 3.30GHz (64 Cores / 128 Threads), Motherboard: Supermicro Super Server H13SSL-N v1.01 (3.0 BIOS), Chipset: AMD 1Ah, Memory: 12 x 64GB DDR5-6000MT/s Micron MTC40F2046S1RC64BDY QSFF, Disk: 3201GB Micron_7450_MTFDKCB3T2TFS, Graphics: ASPEED, Network: 2 x Broadcom NetXtreme BCM5720 PCIe OS: Ubuntu 24.10, Kernel: 6.11.0-phx (x86_64), Desktop: GNOME Shell 47.0, Display Server: X Server, Compiler: GCC 14.2.0, File-System: ext4, Screen Resolution: 1920x1200 v6.13 7 Dec: Processor: AMD EPYC 9575F 64-Core @ 3.30GHz (64 Cores / 128 Threads), Motherboard: Supermicro Super Server H13SSL-N v1.01 (3.0 BIOS), Chipset: AMD 1Ah, Memory: 12 x 64GB DDR5-6000MT/s Micron MTC40F2046S1RC64BDY QSFF, Disk: 3201GB Micron_7450_MTFDKCB3T2TFS, Graphics: ASPEED, Network: 2 x Broadcom NetXtreme BCM5720 PCIe OS: Ubuntu 24.10, Kernel: 6.13.0-rc1-phx (x86_64), Desktop: GNOME Shell 47.0, Display Server: X Server, Compiler: GCC 14.2.0, File-System: ext4, Screen Resolution: 1024x768 Stress-NG 0.17.08 Test: Context Switching Bogo Ops/s > Higher Is Better v6.12 ....... 55755486.32 |==================================================== v6.11 ....... 36748330.54 |================================== v6.13 7 Dec . 53038526.00 |================================================= MariaDB 11.5 Test: oltp_update_index - Threads: 128 Queries Per Second > Higher Is Better v6.12 ....... 183537 |====================================================== v6.11 ....... 192098 |========================================================= v6.13 7 Dec . 158304 |=============================================== DaCapo Benchmark 23.11 Java Test: Spring Boot msec < Lower Is Better v6.12 ....... 2259 |================================================= v6.11 ....... 2724 |=========================================================== v6.13 7 Dec . 2651 |========================================================= DaCapo Benchmark 23.11 Java Test: Apache Lucene Search Engine msec < Lower Is Better v6.12 ....... 4326 |=========================================================== v6.11 ....... 4120 |======================================================== v6.13 7 Dec . 3631 |================================================== Stress-NG 0.17.08 Test: NUMA Bogo Ops/s > Higher Is Better v6.12 ....... 2298.65 |====================================================== v6.11 ....... 2071.98 |================================================= v6.13 7 Dec . 2385.59 |======================================================== DaCapo Benchmark 23.11 Java Test: Zxing 1D/2D Barcode Image Processing msec < Lower Is Better v6.12 ....... 551 |============================================================ v6.11 ....... 522 |========================================================= v6.13 7 Dec . 493 |====================================================== Stress-NG 0.17.08 Test: Semaphores Bogo Ops/s > Higher Is Better v6.12 ....... 251549880.01 |=================================================== v6.11 ....... 233159294.14 |=============================================== v6.13 7 Dec . 246944948.46 |================================================== PostgreSQL 17 Scaling Factor: 100 - Clients: 1000 - Mode: Read Write - Average Latency ms < Lower Is Better v6.12 ....... 8.927 |========================================================= v6.11 ....... 8.498 |====================================================== v6.13 7 Dec . 9.082 |========================================================== PostgreSQL 17 Scaling Factor: 100 - Clients: 1000 - Mode: Read Write TPS > Higher Is Better v6.12 ....... 112025 |====================================================== v6.11 ....... 117677 |========================================================= v6.13 7 Dec . 110112 |===================================================== MariaDB 11.5 Test: oltp_write_only - Threads: 128 Queries Per Second > Higher Is Better v6.12 ....... 399417 |======================================================= v6.11 ....... 412967 |========================================================= v6.13 7 Dec . 386983 |===================================================== Memcached 1.6.19 Set To Get Ratio: 1:100 Ops/sec > Higher Is Better v6.12 ....... 13294800.59 |=================================================== v6.11 ....... 12696519.85 |================================================= v6.13 7 Dec . 13514659.67 |==================================================== Stress-NG 0.17.08 Test: Futex Bogo Ops/s > Higher Is Better v6.12 ....... 4313954.94 |==================================================== v6.11 ....... 4154651.85 |================================================== v6.13 7 Dec . 4398887.77 |===================================================== Stress-NG 0.17.08 Test: MEMFD Bogo Ops/s > Higher Is Better v6.12 ....... 2010.99 |===================================================== v6.11 ....... 2054.70 |====================================================== v6.13 7 Dec . 2125.13 |======================================================== PostgreSQL 17 Scaling Factor: 100 - Clients: 800 - Mode: Read Write - Average Latency ms < Lower Is Better v6.12 ....... 6.550 |========================================================= v6.11 ....... 6.305 |======================================================= v6.13 7 Dec . 6.628 |========================================================== Stress-NG 0.17.08 Test: Mutex Bogo Ops/s > Higher Is Better v6.12 ....... 45398166.14 |==================================================== v6.11 ....... 45376718.94 |==================================================== v6.13 7 Dec . 43186401.84 |================================================= PostgreSQL 17 Scaling Factor: 100 - Clients: 800 - Mode: Read Write TPS > Higher Is Better v6.12 ....... 122181 |======================================================= v6.11 ....... 126877 |========================================================= v6.13 7 Dec . 120703 |====================================================== DaCapo Benchmark 23.11 Java Test: H2 Database Engine msec < Lower Is Better v6.12 ....... 2049 |========================================================= v6.11 ....... 2104 |=========================================================== v6.13 7 Dec . 2010 |======================================================== Llamafile 0.8.16 Model: mistral-7b-instruct-v0.2.Q5_K_M - Test: Text Generation 128 Tokens Per Second > Higher Is Better v6.12 ....... 67.71 |========================================================== v6.11 ....... 68.13 |========================================================== v6.13 7 Dec . 65.45 |======================================================== Apache CouchDB 3.4.1 Bulk Size: 300 - Inserts: 1000 - Rounds: 30 Seconds < Lower Is Better v6.12 ....... 104.16 |========================================================= v6.11 ....... 103.23 |======================================================== v6.13 7 Dec . 100.14 |======================================================= Stress-NG 0.17.08 Test: Mixed Scheduler Bogo Ops/s > Higher Is Better v6.12 ....... 82674.98 |======================================================= v6.11 ....... 79773.37 |===================================================== v6.13 7 Dec . 81582.58 |====================================================== DaCapo Benchmark 23.11 Java Test: Avrora AVR Simulation Framework msec < Lower Is Better v6.12 ....... 2303 |========================================================== v6.11 ....... 2354 |=========================================================== v6.13 7 Dec . 2283 |========================================================= SVT-AV1 2.3 Encoder Mode: Preset 13 - Input: Bosphorus 4K Frames Per Second > Higher Is Better v6.12 ....... 468.67 |======================================================== v6.11 ....... 466.61 |======================================================= v6.13 7 Dec . 480.91 |========================================================= PostgreSQL 17 Scaling Factor: 100 - Clients: 1000 - Mode: Read Only - Average Latency ms < Lower Is Better v6.12 ....... 0.211 |======================================================== v6.11 ....... 0.217 |========================================================== v6.13 7 Dec . 0.211 |======================================================== Llamafile 0.8.16 Model: Llama-3.2-3B-Instruct.Q6_K - Test: Text Generation 128 Tokens Per Second > Higher Is Better v6.12 ....... 111.98 |========================================================= v6.11 ....... 112.56 |========================================================= v6.13 7 Dec . 109.60 |======================================================== PostgreSQL 17 Scaling Factor: 100 - Clients: 1000 - Mode: Read Only TPS > Higher Is Better v6.12 ....... 4732753 |======================================================== v6.11 ....... 4621379 |======================================================= v6.13 7 Dec . 4735578 |======================================================== MariaDB 11.5 Test: oltp_read_write - Threads: 128 Queries Per Second > Higher Is Better v6.12 ....... 181192 |======================================================== v6.11 ....... 183693 |========================================================= v6.13 7 Dec . 179909 |======================================================== Llamafile 0.8.16 Model: TinyLlama-1.1B-Chat-v1.0.BF16 - Test: Text Generation 128 Tokens Per Second > Higher Is Better v6.12 ....... 159.60 |========================================================= v6.11 ....... 159.90 |========================================================= v6.13 7 Dec . 156.82 |======================================================== MariaDB 11.5 Test: oltp_read_only - Threads: 128 Queries Per Second > Higher Is Better v6.12 ....... 43581 |========================================================== v6.11 ....... 43688 |========================================================== v6.13 7 Dec . 42936 |========================================================= Laghos 3.1 Test: Triple Point Problem Major Kernels Total Rate > Higher Is Better v6.12 ....... 303.11 |========================================================= v6.11 ....... 297.99 |======================================================== v6.13 7 Dec . 300.72 |========================================================= Memcached 1.6.19 Set To Get Ratio: 1:10 Ops/sec > Higher Is Better v6.12 ....... 7297642.21 |==================================================== v6.11 ....... 7421710.24 |===================================================== v6.13 7 Dec . 7400996.98 |===================================================== DaCapo Benchmark 23.11 Java Test: Apache Tomcat msec < Lower Is Better v6.12 ....... 1038 |========================================================== v6.11 ....... 1053 |=========================================================== v6.13 7 Dec . 1055 |=========================================================== DaCapo Benchmark 23.11 Java Test: Batik SVG Toolkit msec < Lower Is Better v6.12 ....... 937 |============================================================ v6.11 ....... 923 |=========================================================== v6.13 7 Dec . 938 |============================================================ DaCapo Benchmark 23.11 Java Test: Apache Lucene Search Index msec < Lower Is Better v6.12 ....... 2258 |========================================================== v6.11 ....... 2294 |=========================================================== v6.13 7 Dec . 2278 |=========================================================== Apache Cassandra 5.0 Test: Writes Op/s > Higher Is Better v6.12 ....... 491186 |========================================================= v6.11 ....... 483952 |======================================================== v6.13 7 Dec . 487743 |========================================================= Blender 4.3 Blend File: Fishy Cat - Compute: CPU-Only Seconds < Lower Is Better v6.12 ....... 21.49 |========================================================== v6.11 ....... 21.66 |========================================================== v6.13 7 Dec . 21.35 |========================================================= ClickHouse 22.12.3.5 100M Rows Hits Dataset, First Run / Cold Cache Queries Per Minute, Geo Mean > Higher Is Better v6.12 ....... 765.59 |======================================================== v6.12 ....... 776.70 |========================================================= v6.11 ....... 770.47 |========================================================= v6.13 7 Dec . 774.01 |========================================================= DaCapo Benchmark 23.11 Java Test: Tradesoap msec < Lower Is Better v6.12 ....... 3035 |========================================================== v6.11 ....... 3077 |=========================================================== v6.13 7 Dec . 3049 |========================================================== Blender 4.3 Blend File: Classroom - Compute: CPU-Only Seconds < Lower Is Better v6.12 ....... 41.74 |========================================================== v6.11 ....... 42.03 |========================================================== v6.13 7 Dec . 41.49 |========================================================= DaCapo Benchmark 23.11 Java Test: BioJava Biological Data Framework msec < Lower Is Better v6.12 ....... 4071 |=========================================================== v6.11 ....... 4058 |=========================================================== v6.13 7 Dec . 4019 |========================================================== Memcached 1.6.19 Set To Get Ratio: 1:5 Ops/sec > Higher Is Better v6.12 ....... 4064048.14 |==================================================== v6.11 ....... 4115352.49 |===================================================== v6.13 7 Dec . 4099640.31 |===================================================== PostgreSQL 17 Scaling Factor: 100 - Clients: 800 - Mode: Read Only - Average Latency ms < Lower Is Better v6.12 ....... 0.166 |========================================================== v6.11 ....... 0.167 |========================================================== v6.13 7 Dec . 0.165 |========================================================= DaCapo Benchmark 23.11 Java Test: FOP Print Formatter msec < Lower Is Better v6.12 ....... 335 |=========================================================== v6.11 ....... 339 |============================================================ v6.13 7 Dec . 338 |============================================================ Blender 4.3 Blend File: BMW27 - Compute: CPU-Only Seconds < Lower Is Better v6.12 ....... 15.08 |========================================================== v6.11 ....... 15.19 |========================================================== v6.13 7 Dec . 15.02 |========================================================= DaCapo Benchmark 23.11 Java Test: Eclipse msec < Lower Is Better v6.12 ....... 6280 |========================================================== v6.11 ....... 6349 |=========================================================== v6.13 7 Dec . 6331 |=========================================================== SVT-AV1 2.3 Encoder Mode: Preset 5 - Input: Bosphorus 4K Frames Per Second > Higher Is Better v6.12 ....... 60.38 |========================================================== v6.11 ....... 60.17 |========================================================= v6.13 7 Dec . 60.82 |========================================================== Blender 4.3 Blend File: Pabellon Barcelona - Compute: CPU-Only Seconds < Lower Is Better v6.12 ....... 47.70 |========================================================== v6.11 ....... 47.90 |========================================================== v6.13 7 Dec . 47.39 |========================================================= ClickHouse 22.12.3.5 100M Rows Hits Dataset, Second Run Queries Per Minute, Geo Mean > Higher Is Better v6.12 ....... 801.90 |========================================================= v6.12 ....... 795.68 |======================================================== v6.11 ....... 804.06 |========================================================= v6.13 7 Dec . 803.49 |========================================================= PostgreSQL 17 Scaling Factor: 100 - Clients: 800 - Mode: Read Only TPS > Higher Is Better v6.12 ....... 4815625 |======================================================== v6.11 ....... 4783648 |======================================================= v6.13 7 Dec . 4829255 |======================================================== OpenVINO GenAI 2024.5 Model: Falcon-7b-instruct-int4-ov - Device: CPU tokens/s > Higher Is Better v6.12 ....... 56.21 |========================================================= v6.11 ....... 56.73 |========================================================== v6.13 7 Dec . 56.28 |========================================================== DaCapo Benchmark 23.11 Java Test: GraphChi msec < Lower Is Better v6.12 ....... 2083 |========================================================== v6.11 ....... 2093 |=========================================================== v6.13 7 Dec . 2102 |=========================================================== ClickHouse 22.12.3.5 100M Rows Hits Dataset, Third Run Queries Per Minute, Geo Mean > Higher Is Better v6.12 ....... 799.99 |======================================================== v6.12 ....... 800.56 |========================================================= v6.11 ....... 801.19 |========================================================= v6.13 7 Dec . 807.27 |========================================================= OpenSSL Algorithm: ChaCha20 byte/s > Higher Is Better v6.12 ....... 721616676910 |=================================================== v6.12 ....... 724611832580 |=================================================== v6.11 ....... 718106876590 |=================================================== v6.13 7 Dec . 719980763180 |=================================================== OpenVINO GenAI 2024.5 Model: TinyLlama-1.1B-Chat-v1.0 - Device: CPU tokens/s > Higher Is Better v6.12 ....... 78.18 |========================================================== v6.11 ....... 78.36 |========================================================== v6.13 7 Dec . 77.67 |========================================================= OpenVINO GenAI 2024.5 Model: Gemma-7b-int4-ov - Device: CPU tokens/s > Higher Is Better v6.12 ....... 49.09 |========================================================== v6.11 ....... 49.13 |========================================================== v6.13 7 Dec . 48.70 |========================================================= Llamafile 0.8.16 Model: wizardcoder-python-34b-v1.0.Q6_K - Test: Text Generation 128 Tokens Per Second > Higher Is Better v6.12 ....... 15.82 |========================================================== v6.11 ....... 15.90 |========================================================== v6.13 7 Dec . 15.77 |========================================================== SVT-AV1 2.3 Encoder Mode: Preset 8 - Input: Bosphorus 4K Frames Per Second > Higher Is Better v6.12 ....... 200.32 |========================================================= v6.11 ....... 199.21 |========================================================= v6.13 7 Dec . 200.82 |========================================================= Blender 4.3 Blend File: Barbershop - Compute: CPU-Only Seconds < Lower Is Better v6.12 ....... 148.82 |========================================================= v6.11 ....... 149.30 |========================================================= v6.13 7 Dec . 148.16 |========================================================= SVT-AV1 2.3 Encoder Mode: Preset 3 - Input: Bosphorus 4K Frames Per Second > Higher Is Better v6.12 ....... 16.99 |========================================================== v6.11 ....... 16.88 |========================================================== v6.13 7 Dec . 17.01 |========================================================== Timed Linux Kernel Compilation 6.8 Build: allmodconfig Seconds < Lower Is Better v6.12 ....... 193.20 |========================================================= v6.11 ....... 194.34 |========================================================= v6.13 7 Dec . 192.92 |========================================================= DaCapo Benchmark 23.11 Java Test: Jython msec < Lower Is Better v6.12 ....... 3755 |=========================================================== v6.11 ....... 3782 |=========================================================== v6.13 7 Dec . 3763 |=========================================================== Laghos 3.1 Test: Sedov Blast Wave, ube_922_hex.mesh Major Kernels Total Rate > Higher Is Better v6.12 ....... 565.10 |========================================================= v6.11 ....... 561.91 |========================================================= v6.13 7 Dec . 565.14 |========================================================= Stress-NG 0.17.08 Test: Socket Activity Bogo Ops/s > Higher Is Better v6.12 ....... 47501.04 |======================================================= v6.11 ....... 47424.87 |======================================================= v6.13 7 Dec . 47238.70 |======================================================= Primesieve 12.6 Length: 1e12 Seconds < Lower Is Better v6.12 ....... 1.990 |========================================================== v6.11 ....... 1.999 |========================================================== v6.13 7 Dec . 1.988 |========================================================== OpenSSL Algorithm: ChaCha20-Poly1305 byte/s > Higher Is Better v6.12 ....... 490511085203 |=================================================== v6.12 ....... 489904220937 |=================================================== v6.11 ....... 487834985450 |=================================================== v6.13 7 Dec . 489486102253 |=================================================== DaCapo Benchmark 23.11 Java Test: Apache Kafka msec < Lower Is Better v6.12 ....... 5045 |=========================================================== v6.11 ....... 5070 |=========================================================== v6.13 7 Dec . 5057 |=========================================================== Timed LLVM Compilation 16.0 Build System: Unix Makefiles Seconds < Lower Is Better v6.12 ....... 160.30 |========================================================= v6.11 ....... 159.82 |========================================================= v6.13 7 Dec . 160.56 |========================================================= OpenSSL Algorithm: AES-128-GCM byte/s > Higher Is Better v6.12 ....... 361564053870 |=================================================== v6.12 ....... 361513614630 |=================================================== v6.11 ....... 359931303710 |=================================================== v6.13 7 Dec . 361324888880 |=================================================== OpenSSL Algorithm: AES-256-GCM byte/s > Higher Is Better v6.12 ....... 334486696117 |=================================================== v6.12 ....... 334279722140 |=================================================== v6.11 ....... 333162995897 |=================================================== v6.13 7 Dec . 334428157450 |=================================================== Primesieve 12.6 Length: 1e13 Seconds < Lower Is Better v6.12 ....... 24.97 |========================================================== v6.11 ....... 25.04 |========================================================== v6.13 7 Dec . 24.94 |========================================================== Timed Linux Kernel Compilation 6.8 Build: defconfig Seconds < Lower Is Better v6.12 ....... 21.85 |========================================================== v6.11 ....... 21.79 |========================================================== v6.13 7 Dec . 21.86 |========================================================== DaCapo Benchmark 23.11 Java Test: Apache Xalan XSLT msec < Lower Is Better v6.12 ....... 741 |============================================================ v6.11 ....... 743 |============================================================ v6.13 7 Dec . 742 |============================================================ Timed LLVM Compilation 16.0 Build System: Ninja Seconds < Lower Is Better v6.12 ....... 101.60 |========================================================= v6.11 ....... 101.82 |========================================================= v6.13 7 Dec . 101.73 |========================================================= Blender 4.3 Blend File: Junkshop - Compute: CPU-Only Seconds < Lower Is Better v6.12 ....... 20.21 |========================================================== v6.11 ....... 20.24 |========================================================== v6.13 7 Dec . 20.20 |========================================================== OpenVINO GenAI 2024.5 Model: Phi-3-mini-128k-instruct-int4-ov - Device: CPU tokens/s > Higher Is Better v6.12 ....... 65.82 |========================================================== v6.11 ....... 65.89 |========================================================== v6.13 7 Dec . 65.91 |========================================================== DaCapo Benchmark 23.11 Java Test: jMonkeyEngine msec < Lower Is Better v6.12 ....... 6797 |=========================================================== v6.11 ....... 6791 |=========================================================== v6.13 7 Dec . 6793 |=========================================================== Llamafile 0.8.16 Model: wizardcoder-python-34b-v1.0.Q6_K - Test: Prompt Processing 2048 Tokens Per Second > Higher Is Better v6.12 ....... 12288 |========================================================== v6.11 ....... 12288 |========================================================== v6.13 7 Dec . 12288 |========================================================== Llamafile 0.8.16 Model: wizardcoder-python-34b-v1.0.Q6_K - Test: Prompt Processing 1024 Tokens Per Second > Higher Is Better v6.12 ....... 6144 |=========================================================== v6.11 ....... 6144 |=========================================================== v6.13 7 Dec . 6144 |=========================================================== Llamafile 0.8.16 Model: wizardcoder-python-34b-v1.0.Q6_K - Test: Prompt Processing 512 Tokens Per Second > Higher Is Better v6.12 ....... 3072 |=========================================================== v6.11 ....... 3072 |=========================================================== v6.13 7 Dec . 3072 |=========================================================== Llamafile 0.8.16 Model: wizardcoder-python-34b-v1.0.Q6_K - Test: Prompt Processing 256 Tokens Per Second > Higher Is Better v6.12 ....... 1536 |=========================================================== v6.11 ....... 1536 |=========================================================== v6.13 7 Dec . 1536 |=========================================================== Llamafile 0.8.16 Model: mistral-7b-instruct-v0.2.Q5_K_M - Test: Prompt Processing 2048 Tokens Per Second > Higher Is Better v6.12 ....... 32768 |========================================================== v6.11 ....... 32768 |========================================================== v6.13 7 Dec . 32768 |========================================================== Llamafile 0.8.16 Model: mistral-7b-instruct-v0.2.Q5_K_M - Test: Prompt Processing 1024 Tokens Per Second > Higher Is Better v6.12 ....... 16384 |========================================================== v6.11 ....... 16384 |========================================================== v6.13 7 Dec . 16384 |========================================================== Llamafile 0.8.16 Model: mistral-7b-instruct-v0.2.Q5_K_M - Test: Prompt Processing 512 Tokens Per Second > Higher Is Better v6.12 ....... 8192 |=========================================================== v6.11 ....... 8192 |=========================================================== v6.13 7 Dec . 8192 |=========================================================== Llamafile 0.8.16 Model: mistral-7b-instruct-v0.2.Q5_K_M - Test: Prompt Processing 256 Tokens Per Second > Higher Is Better v6.12 ....... 4096 |=========================================================== v6.11 ....... 4096 |=========================================================== v6.13 7 Dec . 4096 |=========================================================== Llamafile 0.8.16 Model: TinyLlama-1.1B-Chat-v1.0.BF16 - Test: Prompt Processing 2048 Tokens Per Second > Higher Is Better v6.12 ....... 32768 |========================================================== v6.11 ....... 32768 |========================================================== v6.13 7 Dec . 32768 |========================================================== Llamafile 0.8.16 Model: TinyLlama-1.1B-Chat-v1.0.BF16 - Test: Prompt Processing 1024 Tokens Per Second > Higher Is Better v6.12 ....... 16384 |========================================================== v6.11 ....... 16384 |========================================================== v6.13 7 Dec . 16384 |========================================================== Llamafile 0.8.16 Model: TinyLlama-1.1B-Chat-v1.0.BF16 - Test: Prompt Processing 512 Tokens Per Second > Higher Is Better v6.12 ....... 8192 |=========================================================== v6.11 ....... 8192 |=========================================================== v6.13 7 Dec . 8192 |=========================================================== Llamafile 0.8.16 Model: TinyLlama-1.1B-Chat-v1.0.BF16 - Test: Prompt Processing 256 Tokens Per Second > Higher Is Better v6.12 ....... 4096 |=========================================================== v6.11 ....... 4096 |=========================================================== v6.13 7 Dec . 4096 |=========================================================== Llamafile 0.8.16 Model: Llama-3.2-3B-Instruct.Q6_K - Test: Prompt Processing 2048 Tokens Per Second > Higher Is Better v6.12 ....... 32768 |========================================================== v6.11 ....... 32768 |========================================================== v6.13 7 Dec . 32768 |========================================================== Llamafile 0.8.16 Model: Llama-3.2-3B-Instruct.Q6_K - Test: Prompt Processing 1024 Tokens Per Second > Higher Is Better v6.12 ....... 16384 |========================================================== v6.11 ....... 16384 |========================================================== v6.13 7 Dec . 16384 |========================================================== Llamafile 0.8.16 Model: Llama-3.2-3B-Instruct.Q6_K - Test: Prompt Processing 512 Tokens Per Second > Higher Is Better v6.12 ....... 8192 |=========================================================== v6.11 ....... 8192 |=========================================================== v6.13 7 Dec . 8192 |=========================================================== Llamafile 0.8.16 Model: Llama-3.2-3B-Instruct.Q6_K - Test: Prompt Processing 256 Tokens Per Second > Higher Is Better v6.12 ....... 4096 |=========================================================== v6.11 ....... 4096 |=========================================================== v6.13 7 Dec . 4096 |=========================================================== OpenVINO GenAI 2024.5 Model: Phi-3-mini-128k-instruct-int4-ov - Device: CPU - Time Per Output Token ms < Lower Is Better v6.12 ....... 15.19 |========================================================== v6.11 ....... 15.18 |========================================================== v6.13 7 Dec . 15.17 |========================================================== OpenVINO GenAI 2024.5 Model: Phi-3-mini-128k-instruct-int4-ov - Device: CPU - Time To First Token ms < Lower Is Better v6.12 ....... 22.73 |========================================================== v6.11 ....... 22.66 |========================================================== v6.13 7 Dec . 22.38 |========================================================= OpenVINO GenAI 2024.5 Model: Falcon-7b-instruct-int4-ov - Device: CPU - Time Per Output Token ms < Lower Is Better v6.12 ....... 17.80 |========================================================== v6.11 ....... 17.63 |========================================================= v6.13 7 Dec . 17.77 |========================================================== OpenVINO GenAI 2024.5 Model: Falcon-7b-instruct-int4-ov - Device: CPU - Time To First Token ms < Lower Is Better v6.12 ....... 30.00 |========================================================= v6.11 ....... 30.39 |========================================================== v6.13 7 Dec . 30.18 |========================================================== OpenVINO GenAI 2024.5 Model: TinyLlama-1.1B-Chat-v1.0 - Device: CPU - Time Per Output Token ms < Lower Is Better v6.12 ....... 12.79 |========================================================== v6.11 ....... 12.76 |========================================================= v6.13 7 Dec . 12.88 |========================================================== OpenVINO GenAI 2024.5 Model: TinyLlama-1.1B-Chat-v1.0 - Device: CPU - Time To First Token ms < Lower Is Better v6.12 ....... 15.89 |========================================================= v6.11 ....... 15.86 |========================================================= v6.13 7 Dec . 16.05 |========================================================== OpenVINO GenAI 2024.5 Model: Gemma-7b-int4-ov - Device: CPU - Time Per Output Token ms < Lower Is Better v6.12 ....... 20.37 |========================================================== v6.11 ....... 20.36 |========================================================= v6.13 7 Dec . 20.54 |========================================================== OpenVINO GenAI 2024.5 Model: Gemma-7b-int4-ov - Device: CPU - Time To First Token ms < Lower Is Better v6.12 ....... 30.27 |========================================================== v6.11 ....... 30.39 |========================================================== v6.13 7 Dec . 30.48 |========================================================== Llamafile 0.8.16 Model: wizardcoder-python-34b-v1.0.Q6_K - Test: Text Generation 16 Tokens Per Second > Higher Is Better v6.12 ....... 15.01 |========================================================== v6.11 ....... 14.98 |========================================================== v6.13 7 Dec . 14.90 |========================================================== Llamafile 0.8.16 Model: mistral-7b-instruct-v0.2.Q5_K_M - Test: Text Generation 16 Tokens Per Second > Higher Is Better v6.12 ....... 64.80 |========================================================== v6.11 ....... 64.87 |========================================================== v6.13 7 Dec . 64.15 |========================================================= Llamafile 0.8.16 Model: TinyLlama-1.1B-Chat-v1.0.BF16 - Test: Text Generation 16 Tokens Per Second > Higher Is Better v6.12 ....... 152.89 |========================================================= v6.11 ....... 152.07 |========================================================= v6.13 7 Dec . 147.98 |======================================================= Llamafile 0.8.16 Model: Llama-3.2-3B-Instruct.Q6_K - Test: Text Generation 16 Tokens Per Second > Higher Is Better v6.12 ....... 107.95 |========================================================= v6.11 ....... 108.22 |========================================================= v6.13 7 Dec . 102.21 |====================================================== DaCapo Benchmark 23.11 Java Test: PMD Source Code Analyzer msec < Lower Is Better v6.12 ....... 1167 |=========================================================== v6.11 ....... 1085 |======================================================= v6.13 7 Dec . 1089 |======================================================= Apache CouchDB 3.4.1 Bulk Size: 500 - Inserts: 1000 - Rounds: 30 Seconds < Lower Is Better v6.12 ....... 333.18 |====================================================== v6.11 ....... 354.53 |========================================================= v6.13 7 Dec . 311.04 |================================================== Apache CouchDB 3.4.1 Bulk Size: 300 - Inserts: 3000 - Rounds: 30 Seconds < Lower Is Better v6.12 ....... 371.29 |========================================================= v6.11 ....... 363.53 |======================================================== v6.13 7 Dec . 360.83 |======================================================= DaCapo Benchmark 23.11 Java Test: Tradebeans msec < Lower Is Better v6.12 ....... 4968 |=========================================================== v6.11 ....... 4999 |=========================================================== v6.13 7 Dec . 4957 |=========================================================== RELION 5.0 Test: Basic - Device: CPU Seconds < Lower Is Better v6.12 ....... 179.87 |========================================================= v6.11 ....... 176.67 |======================================================== v6.13 7 Dec . 172.48 |======================================================= Palabos 2.3 Grid Size: 1000 Mega Site Updates Per Second > Higher Is Better v6.12 ....... 655.18 |========================================================= v6.11 ....... 481.49 |========================================== v6.13 7 Dec . 509.13 |============================================ Palabos 2.3 Grid Size: 500 Mega Site Updates Per Second > Higher Is Better v6.12 ....... 775.11 |========================================================= v6.11 ....... 529.23 |======================================= v6.13 7 Dec . 764.02 |========================================================