Linux 6.13 AMD EPYC Performance Benchmarks for a future article. AMD EPYC 9575F 64-Core testing with a Supermicro Super Server H13SSL-N v1.01 (3.0 BIOS) and ASPEED on Ubuntu 24.10 via the Phoronix Test Suite.
HTML result view exported from: https://openbenchmarking.org/result/2412073-NE-LINUX613A93&sro .
Linux 6.13 AMD EPYC Performance Processor Motherboard Chipset Memory Disk Graphics Network OS Kernel Desktop Display Server Compiler File-System Screen Resolution v6.11 v6.12 v6.13 7 Dec AMD EPYC 9575F 64-Core @ 3.30GHz (64 Cores / 128 Threads) Supermicro Super Server H13SSL-N v1.01 (3.0 BIOS) AMD 1Ah 12 x 64GB DDR5-6000MT/s Micron MTC40F2046S1RC64BDY QSFF 3201GB Micron_7450_MTFDKCB3T2TFS ASPEED 2 x Broadcom NetXtreme BCM5720 PCIe Ubuntu 24.10 6.11.0-phx (x86_64) GNOME Shell 47.0 X Server GCC 14.2.0 ext4 1920x1200 6.12.0-phx (x86_64) 1024x768 6.13.0-rc1-phx (x86_64) OpenBenchmarking.org Kernel Details - Transparent Huge Pages: madvise Compiler Details - --build=x86_64-linux-gnu --disable-vtable-verify --disable-werror --enable-bootstrap --enable-cet --enable-checking=release --enable-clocale=gnu --enable-default-pie --enable-gnu-unique-object --enable-languages=c,ada,c++,go,d,fortran,objc,obj-c++,m2,rust --enable-libphobos-checking=release --enable-libstdcxx-backtrace --enable-libstdcxx-debug --enable-libstdcxx-time=yes --enable-link-serialization=2 --enable-multiarch --enable-multilib --enable-nls --enable-objc-gc=auto --enable-offload-defaulted --enable-offload-targets=nvptx-none=/build/gcc-14-zdkDXv/gcc-14-14.2.0/debian/tmp-nvptx/usr,amdgcn-amdhsa=/build/gcc-14-zdkDXv/gcc-14-14.2.0/debian/tmp-gcn/usr --enable-plugin --enable-shared --enable-threads=posix --host=x86_64-linux-gnu --program-prefix=x86_64-linux-gnu- --target=x86_64-linux-gnu --with-abi=m64 --with-arch-32=i686 --with-build-config=bootstrap-lto-lean --with-default-libstdcxx-abi=new --with-gcc-major-version-only --with-multilib-list=m32,m64,mx32 --with-target-system-zlib=auto --with-tune=generic --without-cuda-driver -v Processor Details - Scaling Governor: acpi-cpufreq performance (Boost: Enabled) - CPU Microcode: 0xb002116 Java Details - OpenJDK Runtime Environment (build 21.0.5+11-Ubuntu-1ubuntu124.10) Python Details - Python 3.12.7 Security Details - gather_data_sampling: Not affected + itlb_multihit: Not affected + l1tf: Not affected + mds: Not affected + meltdown: Not affected + mmio_stale_data: Not affected + reg_file_data_sampling: Not affected + retbleed: Not affected + spec_rstack_overflow: Not affected + spec_store_bypass: Mitigation of SSB disabled via prctl + spectre_v1: Mitigation of usercopy/swapgs barriers and __user pointer sanitization + spectre_v2: Mitigation of Enhanced / Automatic IBRS; IBPB: conditional; STIBP: always-on; RSB filling; PBRSB-eIBRS: Not affected; BHI: Not affected + srbds: Not affected + tsx_async_abort: Not affected
Linux 6.13 AMD EPYC Performance laghos: Triple Point Problem laghos: Sedov Blast Wave, ube_922_hex.mesh palabos: 500 palabos: 1000 relion: Basic - CPU dacapobench: Jython dacapobench: GraphChi dacapobench: Tradesoap dacapobench: Tradebeans dacapobench: Apache Tomcat dacapobench: Apache Xalan XSLT dacapobench: H2 Database Engine svt-av1: Preset 3 - Bosphorus 4K svt-av1: Preset 5 - Bosphorus 4K svt-av1: Preset 8 - Bosphorus 4K svt-av1: Preset 13 - Bosphorus 4K build-linux-kernel: defconfig build-linux-kernel: allmodconfig build-llvm: Ninja build-llvm: Unix Makefiles primesieve: 1e12 primesieve: 1e13 openssl: ChaCha20 openssl: AES-128-GCM openssl: AES-256-GCM openssl: ChaCha20-Poly1305 clickhouse: 100M Rows Hits Dataset, First Run / Cold Cache clickhouse: 100M Rows Hits Dataset, Second Run clickhouse: 100M Rows Hits Dataset, Third Run couchdb: 300 - 1000 - 30 couchdb: 300 - 3000 - 30 couchdb: 500 - 1000 - 30 dacapobench: Eclipse dacapobench: Spring Boot dacapobench: Apache Kafka dacapobench: jMonkeyEngine dacapobench: Batik SVG Toolkit dacapobench: FOP Print Formatter dacapobench: PMD Source Code Analyzer dacapobench: Apache Lucene Search Index dacapobench: Apache Lucene Search Engine dacapobench: Avrora AVR Simulation Framework dacapobench: BioJava Biological Data Framework dacapobench: Zxing 1D/2D Barcode Image Processing memcached: 1:5 memcached: 1:10 memcached: 1:100 mariadb: oltp_read_only - 128 mariadb: oltp_read_write - 128 mariadb: oltp_write_only - 128 mariadb: oltp_update_index - 128 pgbench: 100 - 800 - Read Only pgbench: 100 - 800 - Read Only - Average Latency pgbench: 100 - 1000 - Read Only pgbench: 100 - 1000 - Read Only - Average Latency pgbench: 100 - 800 - Read Write pgbench: 100 - 800 - Read Write - Average Latency pgbench: 100 - 1000 - Read Write pgbench: 100 - 1000 - Read Write - Average Latency stress-ng: NUMA stress-ng: Futex stress-ng: MEMFD stress-ng: Mutex stress-ng: Semaphores stress-ng: Mixed Scheduler stress-ng: Socket Activity stress-ng: Context Switching blender: BMW27 - CPU-Only blender: Junkshop - CPU-Only blender: Classroom - CPU-Only blender: Fishy Cat - CPU-Only blender: Barbershop - CPU-Only blender: Pabellon Barcelona - CPU-Only cassandra: Writes llamafile: Llama-3.2-3B-Instruct.Q6_K - Text Generation 16 llamafile: Llama-3.2-3B-Instruct.Q6_K - Text Generation 128 llamafile: Llama-3.2-3B-Instruct.Q6_K - Prompt Processing 256 llamafile: Llama-3.2-3B-Instruct.Q6_K - Prompt Processing 512 llamafile: TinyLlama-1.1B-Chat-v1.0.BF16 - Text Generation 16 llamafile: Llama-3.2-3B-Instruct.Q6_K - Prompt Processing 1024 llamafile: Llama-3.2-3B-Instruct.Q6_K - Prompt Processing 2048 llamafile: TinyLlama-1.1B-Chat-v1.0.BF16 - Text Generation 128 llamafile: mistral-7b-instruct-v0.2.Q5_K_M - Text Generation 16 llamafile: TinyLlama-1.1B-Chat-v1.0.BF16 - Prompt Processing 256 llamafile: TinyLlama-1.1B-Chat-v1.0.BF16 - Prompt Processing 512 llamafile: mistral-7b-instruct-v0.2.Q5_K_M - Text Generation 128 llamafile: wizardcoder-python-34b-v1.0.Q6_K - Text Generation 16 llamafile: TinyLlama-1.1B-Chat-v1.0.BF16 - Prompt Processing 1024 llamafile: TinyLlama-1.1B-Chat-v1.0.BF16 - Prompt Processing 2048 llamafile: wizardcoder-python-34b-v1.0.Q6_K - Text Generation 128 llamafile: mistral-7b-instruct-v0.2.Q5_K_M - Prompt Processing 256 llamafile: mistral-7b-instruct-v0.2.Q5_K_M - Prompt Processing 512 llamafile: mistral-7b-instruct-v0.2.Q5_K_M - Prompt Processing 1024 llamafile: mistral-7b-instruct-v0.2.Q5_K_M - Prompt Processing 2048 llamafile: wizardcoder-python-34b-v1.0.Q6_K - Prompt Processing 256 llamafile: wizardcoder-python-34b-v1.0.Q6_K - Prompt Processing 512 llamafile: wizardcoder-python-34b-v1.0.Q6_K - Prompt Processing 1024 llamafile: wizardcoder-python-34b-v1.0.Q6_K - Prompt Processing 2048 openvino-genai: Gemma-7b-int4-ov - CPU openvino-genai: Gemma-7b-int4-ov - CPU - Time To First Token openvino-genai: Gemma-7b-int4-ov - CPU - Time Per Output Token openvino-genai: TinyLlama-1.1B-Chat-v1.0 - CPU openvino-genai: TinyLlama-1.1B-Chat-v1.0 - CPU - Time To First Token openvino-genai: TinyLlama-1.1B-Chat-v1.0 - CPU - Time Per Output Token openvino-genai: Falcon-7b-instruct-int4-ov - CPU openvino-genai: Falcon-7b-instruct-int4-ov - CPU - Time To First Token openvino-genai: Falcon-7b-instruct-int4-ov - CPU - Time Per Output Token openvino-genai: Phi-3-mini-128k-instruct-int4-ov - CPU openvino-genai: Phi-3-mini-128k-instruct-int4-ov - CPU - Time To First Token openvino-genai: Phi-3-mini-128k-instruct-int4-ov - CPU - Time Per Output Token v6.11 v6.12 v6.13 7 Dec 297.99 561.91 529.232 481.489 176.673 3782 2093 3077 4999 1053 743 2104 16.880 60.171 199.209 466.614 21.792 194.338 101.821 159.819 1.999 25.036 718106876590 359931303710 333162995897 487834985450 770.47 804.06 801.19 103.228 363.527 354.526 6349 2724 5070 6791 923 339 1085 2294 4120 2354 4058 522 4115352.49 7421710.24 12696519.85 43688 183693 412967 192098 4783648 0.167 4621379 0.217 126877 6.305 117677 8.498 2071.98 4154651.85 2054.70 45376718.94 233159294.14 79773.37 47424.87 36748330.54 15.19 20.24 42.03 21.66 149.30 47.90 483952 108.22 112.56 4096 8192 152.07 16384 32768 159.90 64.87 4096 8192 68.13 14.98 16384 32768 15.90 4096 8192 16384 32768 1536 3072 6144 12288 49.13 30.39 20.36 78.36 15.86 12.76 56.73 30.39 17.63 65.89 22.66 15.18 303.11 565.10 775.110 655.184 179.868 3755 2083 3035 4968 1038 741 2049 16.990 60.381 200.316 468.666 21.848 193.202 101.599 160.302 1.990 24.972 721616676910 361564053870 334486696117 490511085203 765.59 801.90 799.99 104.163 371.288 333.178 6280 2259 5045 6797 937 335 1167 2258 4326 2303 4071 551 4064048.14 7297642.21 13294800.59 43581 181192 399417 183537 4815625 0.166 4732753 0.211 122181 6.550 112025 8.927 2298.65 4313954.94 2010.99 45398166.14 251549880.01 82674.98 47501.04 55755486.32 15.08 20.21 41.74 21.49 148.82 47.70 491186 107.95 111.98 4096 8192 152.89 16384 32768 159.60 64.80 4096 8192 67.71 15.01 16384 32768 15.82 4096 8192 16384 32768 1536 3072 6144 12288 49.09 30.27 20.37 78.18 15.89 12.79 56.21 30.00 17.80 65.82 22.73 15.19 300.72 565.14 764.024 509.126 172.482 3763 2102 3049 4957 1055 742 2010 17.008 60.821 200.823 480.912 21.855 192.923 101.725 160.559 1.988 24.939 719980763180 361324888880 334428157450 489486102253 774.01 803.49 807.27 100.140 360.826 311.038 6331 2651 5057 6793 938 338 1089 2278 3631 2283 4019 493 4099640.31 7400996.98 13514659.67 42936 179909 386983 158304 4829255 0.165 4735578 0.211 120703 6.628 110112 9.082 2385.59 4398887.77 2125.13 43186401.84 246944948.46 81582.58 47238.70 53038526.00 15.02 20.20 41.49 21.35 148.16 47.39 487743 102.21 109.60 4096 8192 147.98 16384 32768 156.82 64.15 4096 8192 65.45 14.90 16384 32768 15.77 4096 8192 16384 32768 1536 3072 6144 12288 48.70 30.48 20.54 77.67 16.05 12.88 56.28 30.18 17.77 65.91 22.38 15.17 OpenBenchmarking.org
Laghos Test: Triple Point Problem OpenBenchmarking.org Major Kernels Total Rate, More Is Better Laghos 3.1 Test: Triple Point Problem v6.11 v6.12 v6.13 7 Dec 70 140 210 280 350 SE +/- 3.08, N = 3 SE +/- 1.53, N = 3 SE +/- 3.23, N = 3 297.99 303.11 300.72 1. (CXX) g++ options: -O3 -std=c++11 -lmfem -lHYPRE -lmetis -lrt -lmpi_cxx -lmpi
Laghos Test: Sedov Blast Wave, ube_922_hex.mesh OpenBenchmarking.org Major Kernels Total Rate, More Is Better Laghos 3.1 Test: Sedov Blast Wave, ube_922_hex.mesh v6.11 v6.12 v6.13 7 Dec 120 240 360 480 600 SE +/- 3.27, N = 3 SE +/- 2.47, N = 3 SE +/- 2.95, N = 3 561.91 565.10 565.14 1. (CXX) g++ options: -O3 -std=c++11 -lmfem -lHYPRE -lmetis -lrt -lmpi_cxx -lmpi
Palabos Grid Size: 500 OpenBenchmarking.org Mega Site Updates Per Second, More Is Better Palabos 2.3 Grid Size: 500 v6.11 v6.12 v6.13 7 Dec 200 400 600 800 1000 SE +/- 40.67, N = 12 SE +/- 1.19, N = 3 SE +/- 5.28, N = 3 529.23 775.11 764.02 1. (CXX) g++ options: -std=c++17 -pedantic -O3 -rdynamic -lcrypto -lcurl -lsz -lz -ldl -lm
Palabos Grid Size: 1000 OpenBenchmarking.org Mega Site Updates Per Second, More Is Better Palabos 2.3 Grid Size: 1000 v6.11 v6.12 v6.13 7 Dec 140 280 420 560 700 SE +/- 3.07, N = 15 SE +/- 36.69, N = 12 SE +/- 21.98, N = 15 481.49 655.18 509.13 1. (CXX) g++ options: -std=c++17 -pedantic -O3 -rdynamic -lcrypto -lcurl -lsz -lz -ldl -lm
RELION Test: Basic - Device: CPU OpenBenchmarking.org Seconds, Fewer Is Better RELION 5.0 Test: Basic - Device: CPU v6.11 v6.12 v6.13 7 Dec 40 80 120 160 200 SE +/- 1.95, N = 3 SE +/- 0.79, N = 3 SE +/- 3.31, N = 12 176.67 179.87 172.48 1. (CXX) g++ options: -fPIC -std=c++14 -fopenmp -O3 -rdynamic -ldl -ltiff -lfftw3f -lfftw3 -lpng -ljpeg -lmpi_cxx -lmpi
DaCapo Benchmark Java Test: Jython OpenBenchmarking.org msec, Fewer Is Better DaCapo Benchmark 23.11 Java Test: Jython v6.11 v6.12 v6.13 7 Dec 800 1600 2400 3200 4000 SE +/- 6.67, N = 3 SE +/- 5.33, N = 3 SE +/- 11.00, N = 3 3782 3755 3763
DaCapo Benchmark Java Test: GraphChi OpenBenchmarking.org msec, Fewer Is Better DaCapo Benchmark 23.11 Java Test: GraphChi v6.11 v6.12 v6.13 7 Dec 500 1000 1500 2000 2500 SE +/- 13.86, N = 3 SE +/- 5.51, N = 3 SE +/- 6.00, N = 3 2093 2083 2102
DaCapo Benchmark Java Test: Tradesoap OpenBenchmarking.org msec, Fewer Is Better DaCapo Benchmark 23.11 Java Test: Tradesoap v6.11 v6.12 v6.13 7 Dec 700 1400 2100 2800 3500 SE +/- 24.92, N = 3 SE +/- 21.97, N = 15 SE +/- 31.71, N = 5 3077 3035 3049
DaCapo Benchmark Java Test: Tradebeans OpenBenchmarking.org msec, Fewer Is Better DaCapo Benchmark 23.11 Java Test: Tradebeans v6.11 v6.12 v6.13 7 Dec 1100 2200 3300 4400 5500 SE +/- 48.79, N = 15 SE +/- 76.59, N = 15 SE +/- 100.21, N = 12 4999 4968 4957
DaCapo Benchmark Java Test: Apache Tomcat OpenBenchmarking.org msec, Fewer Is Better DaCapo Benchmark 23.11 Java Test: Apache Tomcat v6.11 v6.12 v6.13 7 Dec 200 400 600 800 1000 SE +/- 7.81, N = 3 SE +/- 3.51, N = 3 SE +/- 2.00, N = 3 1053 1038 1055
DaCapo Benchmark Java Test: Apache Xalan XSLT OpenBenchmarking.org msec, Fewer Is Better DaCapo Benchmark 23.11 Java Test: Apache Xalan XSLT v6.11 v6.12 v6.13 7 Dec 160 320 480 640 800 SE +/- 6.06, N = 3 SE +/- 1.76, N = 3 SE +/- 8.54, N = 3 743 741 742
DaCapo Benchmark Java Test: H2 Database Engine OpenBenchmarking.org msec, Fewer Is Better DaCapo Benchmark 23.11 Java Test: H2 Database Engine v6.11 v6.12 v6.13 7 Dec 500 1000 1500 2000 2500 SE +/- 28.09, N = 3 SE +/- 13.65, N = 3 SE +/- 18.21, N = 3 2104 2049 2010
SVT-AV1 Encoder Mode: Preset 3 - Input: Bosphorus 4K OpenBenchmarking.org Frames Per Second, More Is Better SVT-AV1 2.3 Encoder Mode: Preset 3 - Input: Bosphorus 4K v6.11 v6.12 v6.13 7 Dec 4 8 12 16 20 SE +/- 0.08, N = 3 SE +/- 0.04, N = 3 SE +/- 0.02, N = 3 16.88 16.99 17.01 1. (CXX) g++ options: -march=native -mno-avx -mavx2 -mavx512f -mavx512bw -mavx512dq
SVT-AV1 Encoder Mode: Preset 5 - Input: Bosphorus 4K OpenBenchmarking.org Frames Per Second, More Is Better SVT-AV1 2.3 Encoder Mode: Preset 5 - Input: Bosphorus 4K v6.11 v6.12 v6.13 7 Dec 14 28 42 56 70 SE +/- 0.07, N = 3 SE +/- 0.12, N = 3 SE +/- 0.08, N = 3 60.17 60.38 60.82 1. (CXX) g++ options: -march=native -mno-avx -mavx2 -mavx512f -mavx512bw -mavx512dq
SVT-AV1 Encoder Mode: Preset 8 - Input: Bosphorus 4K OpenBenchmarking.org Frames Per Second, More Is Better SVT-AV1 2.3 Encoder Mode: Preset 8 - Input: Bosphorus 4K v6.11 v6.12 v6.13 7 Dec 40 80 120 160 200 SE +/- 1.84, N = 3 SE +/- 1.77, N = 3 SE +/- 0.64, N = 3 199.21 200.32 200.82 1. (CXX) g++ options: -march=native -mno-avx -mavx2 -mavx512f -mavx512bw -mavx512dq
SVT-AV1 Encoder Mode: Preset 13 - Input: Bosphorus 4K OpenBenchmarking.org Frames Per Second, More Is Better SVT-AV1 2.3 Encoder Mode: Preset 13 - Input: Bosphorus 4K v6.11 v6.12 v6.13 7 Dec 100 200 300 400 500 SE +/- 3.40, N = 3 SE +/- 1.95, N = 3 SE +/- 0.44, N = 3 466.61 468.67 480.91 1. (CXX) g++ options: -march=native -mno-avx -mavx2 -mavx512f -mavx512bw -mavx512dq
Timed Linux Kernel Compilation Build: defconfig OpenBenchmarking.org Seconds, Fewer Is Better Timed Linux Kernel Compilation 6.8 Build: defconfig v6.11 v6.12 v6.13 7 Dec 5 10 15 20 25 SE +/- 0.25, N = 4 SE +/- 0.31, N = 3 SE +/- 0.26, N = 4 21.79 21.85 21.86
Timed Linux Kernel Compilation Build: allmodconfig OpenBenchmarking.org Seconds, Fewer Is Better Timed Linux Kernel Compilation 6.8 Build: allmodconfig v6.11 v6.12 v6.13 7 Dec 40 80 120 160 200 SE +/- 0.21, N = 3 SE +/- 0.34, N = 3 SE +/- 0.25, N = 3 194.34 193.20 192.92
Timed LLVM Compilation Build System: Ninja OpenBenchmarking.org Seconds, Fewer Is Better Timed LLVM Compilation 16.0 Build System: Ninja v6.11 v6.12 v6.13 7 Dec 20 40 60 80 100 SE +/- 0.11, N = 3 SE +/- 0.06, N = 3 SE +/- 0.19, N = 3 101.82 101.60 101.73
Timed LLVM Compilation Build System: Unix Makefiles OpenBenchmarking.org Seconds, Fewer Is Better Timed LLVM Compilation 16.0 Build System: Unix Makefiles v6.11 v6.12 v6.13 7 Dec 40 80 120 160 200 SE +/- 0.34, N = 3 SE +/- 0.33, N = 3 SE +/- 0.86, N = 3 159.82 160.30 160.56
Primesieve Length: 1e12 OpenBenchmarking.org Seconds, Fewer Is Better Primesieve 12.6 Length: 1e12 v6.11 v6.12 v6.13 7 Dec 0.4498 0.8996 1.3494 1.7992 2.249 SE +/- 0.003, N = 3 SE +/- 0.001, N = 3 SE +/- 0.002, N = 3 1.999 1.990 1.988 1. (CXX) g++ options: -O3
Primesieve Length: 1e13 OpenBenchmarking.org Seconds, Fewer Is Better Primesieve 12.6 Length: 1e13 v6.11 v6.12 v6.13 7 Dec 6 12 18 24 30 SE +/- 0.04, N = 3 SE +/- 0.03, N = 3 SE +/- 0.02, N = 3 25.04 24.97 24.94 1. (CXX) g++ options: -O3
OpenSSL Algorithm: ChaCha20 OpenBenchmarking.org byte/s, More Is Better OpenSSL Algorithm: ChaCha20 v6.11 v6.12 v6.13 7 Dec 160000M 320000M 480000M 640000M 800000M SE +/- 652717270.91, N = 3 SE +/- 1474303291.59, N = 3 SE +/- 626958911.68, N = 3 718106876590 721616676910 719980763180 1. OpenSSL 3.3.1 4 Jun 2024 (Library: OpenSSL 3.3.1 4 Jun 2024) - Additional Parameters: -engine qatengine -async_jobs 8
OpenSSL Algorithm: AES-128-GCM OpenBenchmarking.org byte/s, More Is Better OpenSSL Algorithm: AES-128-GCM v6.11 v6.12 v6.13 7 Dec 80000M 160000M 240000M 320000M 400000M SE +/- 130266226.11, N = 3 SE +/- 210850977.29, N = 3 SE +/- 88875000.56, N = 3 359931303710 361513614630 361324888880 1. OpenSSL 3.3.1 4 Jun 2024 (Library: OpenSSL 3.3.1 4 Jun 2024) - Additional Parameters: -engine qatengine -async_jobs 8
OpenSSL Algorithm: AES-256-GCM OpenBenchmarking.org byte/s, More Is Better OpenSSL Algorithm: AES-256-GCM v6.11 v6.12 v6.13 7 Dec 70000M 140000M 210000M 280000M 350000M SE +/- 105448439.59, N = 3 SE +/- 189485712.03, N = 3 SE +/- 134252719.45, N = 3 333162995897 334279722140 334428157450 1. OpenSSL 3.3.1 4 Jun 2024 (Library: OpenSSL 3.3.1 4 Jun 2024) - Additional Parameters: -engine qatengine -async_jobs 8
OpenSSL Algorithm: ChaCha20-Poly1305 OpenBenchmarking.org byte/s, More Is Better OpenSSL Algorithm: ChaCha20-Poly1305 v6.11 v6.12 v6.13 7 Dec 110000M 220000M 330000M 440000M 550000M SE +/- 271433364.08, N = 3 SE +/- 45557628.93, N = 3 SE +/- 158903312.00, N = 3 487834985450 489904220937 489486102253 1. OpenSSL 3.3.1 4 Jun 2024 (Library: OpenSSL 3.3.1 4 Jun 2024) - Additional Parameters: -engine qatengine -async_jobs 8
ClickHouse 100M Rows Hits Dataset, First Run / Cold Cache OpenBenchmarking.org Queries Per Minute, Geo Mean, More Is Better ClickHouse 22.12.3.5 100M Rows Hits Dataset, First Run / Cold Cache v6.11 v6.12 v6.13 7 Dec 200 400 600 800 1000 SE +/- 2.38, N = 3 SE +/- 5.98, N = 3 SE +/- 2.84, N = 3 770.47 765.59 774.01 MIN: 65.15 / MAX: 8571.43 MIN: 63.29 / MAX: 7500 MIN: 65.86 / MAX: 7500
ClickHouse 100M Rows Hits Dataset, Second Run OpenBenchmarking.org Queries Per Minute, Geo Mean, More Is Better ClickHouse 22.12.3.5 100M Rows Hits Dataset, Second Run v6.11 v6.12 v6.13 7 Dec 200 400 600 800 1000 SE +/- 4.73, N = 3 SE +/- 7.42, N = 3 SE +/- 8.62, N = 3 804.06 795.68 803.49 MIN: 65.15 / MAX: 10000 MIN: 65.29 / MAX: 7500 MIN: 66.08 / MAX: 10000
ClickHouse 100M Rows Hits Dataset, Third Run OpenBenchmarking.org Queries Per Minute, Geo Mean, More Is Better ClickHouse 22.12.3.5 100M Rows Hits Dataset, Third Run v6.11 v6.12 v6.13 7 Dec 200 400 600 800 1000 SE +/- 4.79, N = 3 SE +/- 7.60, N = 3 SE +/- 2.96, N = 3 801.19 799.99 807.27 MIN: 65.5 / MAX: 8571.43 MIN: 64.94 / MAX: 10000 MIN: 66.01 / MAX: 8571.43
Apache CouchDB Bulk Size: 300 - Inserts: 1000 - Rounds: 30 OpenBenchmarking.org Seconds, Fewer Is Better Apache CouchDB 3.4.1 Bulk Size: 300 - Inserts: 1000 - Rounds: 30 v6.11 v6.12 v6.13 7 Dec 20 40 60 80 100 SE +/- 1.11, N = 3 SE +/- 0.71, N = 3 SE +/- 0.87, N = 7 103.23 104.16 100.14 1. (CXX) g++ options: -flto -lstdc++ -shared -lei
Apache CouchDB Bulk Size: 300 - Inserts: 3000 - Rounds: 30 OpenBenchmarking.org Seconds, Fewer Is Better Apache CouchDB 3.4.1 Bulk Size: 300 - Inserts: 3000 - Rounds: 30 v6.11 v6.12 v6.13 7 Dec 80 160 240 320 400 SE +/- 6.26, N = 9 SE +/- 12.48, N = 9 SE +/- 3.45, N = 9 363.53 371.29 360.83 1. (CXX) g++ options: -flto -lstdc++ -shared -lei
Apache CouchDB Bulk Size: 500 - Inserts: 1000 - Rounds: 30 OpenBenchmarking.org Seconds, Fewer Is Better Apache CouchDB 3.4.1 Bulk Size: 500 - Inserts: 1000 - Rounds: 30 v6.11 v6.12 v6.13 7 Dec 80 160 240 320 400 SE +/- 62.71, N = 8 SE +/- 36.20, N = 9 SE +/- 29.09, N = 7 354.53 333.18 311.04 1. (CXX) g++ options: -flto -lstdc++ -shared -lei
DaCapo Benchmark Java Test: Eclipse OpenBenchmarking.org msec, Fewer Is Better DaCapo Benchmark 23.11 Java Test: Eclipse v6.11 v6.12 v6.13 7 Dec 1400 2800 4200 5600 7000 SE +/- 14.75, N = 3 SE +/- 33.39, N = 3 SE +/- 23.78, N = 3 6349 6280 6331
DaCapo Benchmark Java Test: Spring Boot OpenBenchmarking.org msec, Fewer Is Better DaCapo Benchmark 23.11 Java Test: Spring Boot v6.11 v6.12 v6.13 7 Dec 600 1200 1800 2400 3000 SE +/- 17.89, N = 3 SE +/- 20.60, N = 3 SE +/- 26.77, N = 3 2724 2259 2651
DaCapo Benchmark Java Test: Apache Kafka OpenBenchmarking.org msec, Fewer Is Better DaCapo Benchmark 23.11 Java Test: Apache Kafka v6.11 v6.12 v6.13 7 Dec 1100 2200 3300 4400 5500 SE +/- 7.33, N = 3 SE +/- 26.64, N = 3 SE +/- 3.53, N = 3 5070 5045 5057
DaCapo Benchmark Java Test: jMonkeyEngine OpenBenchmarking.org msec, Fewer Is Better DaCapo Benchmark 23.11 Java Test: jMonkeyEngine v6.11 v6.12 v6.13 7 Dec 1500 3000 4500 6000 7500 SE +/- 1.86, N = 3 SE +/- 4.84, N = 3 SE +/- 1.86, N = 3 6791 6797 6793
DaCapo Benchmark Java Test: Batik SVG Toolkit OpenBenchmarking.org msec, Fewer Is Better DaCapo Benchmark 23.11 Java Test: Batik SVG Toolkit v6.11 v6.12 v6.13 7 Dec 200 400 600 800 1000 SE +/- 3.61, N = 3 SE +/- 4.00, N = 3 SE +/- 7.55, N = 3 923 937 938
DaCapo Benchmark Java Test: FOP Print Formatter OpenBenchmarking.org msec, Fewer Is Better DaCapo Benchmark 23.11 Java Test: FOP Print Formatter v6.11 v6.12 v6.13 7 Dec 70 140 210 280 350 SE +/- 0.67, N = 3 SE +/- 2.08, N = 3 SE +/- 1.20, N = 3 339 335 338
DaCapo Benchmark Java Test: PMD Source Code Analyzer OpenBenchmarking.org msec, Fewer Is Better DaCapo Benchmark 23.11 Java Test: PMD Source Code Analyzer v6.11 v6.12 v6.13 7 Dec 300 600 900 1200 1500 SE +/- 5.90, N = 3 SE +/- 23.97, N = 15 SE +/- 18.53, N = 15 1085 1167 1089
DaCapo Benchmark Java Test: Apache Lucene Search Index OpenBenchmarking.org msec, Fewer Is Better DaCapo Benchmark 23.11 Java Test: Apache Lucene Search Index v6.11 v6.12 v6.13 7 Dec 500 1000 1500 2000 2500 SE +/- 16.64, N = 3 SE +/- 8.39, N = 3 SE +/- 25.32, N = 3 2294 2258 2278
DaCapo Benchmark Java Test: Apache Lucene Search Engine OpenBenchmarking.org msec, Fewer Is Better DaCapo Benchmark 23.11 Java Test: Apache Lucene Search Engine v6.11 v6.12 v6.13 7 Dec 900 1800 2700 3600 4500 SE +/- 43.31, N = 15 SE +/- 47.07, N = 3 SE +/- 42.57, N = 3 4120 4326 3631
DaCapo Benchmark Java Test: Avrora AVR Simulation Framework OpenBenchmarking.org msec, Fewer Is Better DaCapo Benchmark 23.11 Java Test: Avrora AVR Simulation Framework v6.11 v6.12 v6.13 7 Dec 500 1000 1500 2000 2500 SE +/- 16.90, N = 3 SE +/- 15.56, N = 13 SE +/- 20.72, N = 15 2354 2303 2283
DaCapo Benchmark Java Test: BioJava Biological Data Framework OpenBenchmarking.org msec, Fewer Is Better DaCapo Benchmark 23.11 Java Test: BioJava Biological Data Framework v6.11 v6.12 v6.13 7 Dec 900 1800 2700 3600 4500 SE +/- 11.89, N = 3 SE +/- 18.98, N = 3 SE +/- 22.00, N = 3 4058 4071 4019
DaCapo Benchmark Java Test: Zxing 1D/2D Barcode Image Processing OpenBenchmarking.org msec, Fewer Is Better DaCapo Benchmark 23.11 Java Test: Zxing 1D/2D Barcode Image Processing v6.11 v6.12 v6.13 7 Dec 120 240 360 480 600 SE +/- 4.26, N = 3 SE +/- 4.38, N = 9 SE +/- 5.55, N = 3 522 551 493
Memcached Set To Get Ratio: 1:5 OpenBenchmarking.org Ops/sec, More Is Better Memcached 1.6.19 Set To Get Ratio: 1:5 v6.11 v6.12 v6.13 7 Dec 900K 1800K 2700K 3600K 4500K SE +/- 26467.49, N = 3 SE +/- 16085.40, N = 3 SE +/- 28528.57, N = 3 4115352.49 4064048.14 4099640.31 1. (CXX) g++ options: -O2 -levent_openssl -levent -lcrypto -lssl -lpthread -lz -lpcre
Memcached Set To Get Ratio: 1:10 OpenBenchmarking.org Ops/sec, More Is Better Memcached 1.6.19 Set To Get Ratio: 1:10 v6.11 v6.12 v6.13 7 Dec 1.6M 3.2M 4.8M 6.4M 8M SE +/- 35717.40, N = 3 SE +/- 30399.87, N = 3 SE +/- 37597.18, N = 3 7421710.24 7297642.21 7400996.98 1. (CXX) g++ options: -O2 -levent_openssl -levent -lcrypto -lssl -lpthread -lz -lpcre
Memcached Set To Get Ratio: 1:100 OpenBenchmarking.org Ops/sec, More Is Better Memcached 1.6.19 Set To Get Ratio: 1:100 v6.11 v6.12 v6.13 7 Dec 3M 6M 9M 12M 15M SE +/- 13756.93, N = 3 SE +/- 15384.68, N = 3 SE +/- 24482.38, N = 3 12696519.85 13294800.59 13514659.67 1. (CXX) g++ options: -O2 -levent_openssl -levent -lcrypto -lssl -lpthread -lz -lpcre
MariaDB Test: oltp_read_only - Threads: 128 OpenBenchmarking.org Queries Per Second, More Is Better MariaDB 11.5 Test: oltp_read_only - Threads: 128 v6.11 v6.12 v6.13 7 Dec 9K 18K 27K 36K 45K SE +/- 14.78, N = 3 SE +/- 15.19, N = 3 SE +/- 18.45, N = 3 43688 43581 42936 1. (CXX) g++ options: -pie -fPIC -fstack-protector -O3 -lnuma -lpcre2-8 -lcrypt -laio -lz -lm -lssl -lcrypto -lpthread -ldl
MariaDB Test: oltp_read_write - Threads: 128 OpenBenchmarking.org Queries Per Second, More Is Better MariaDB 11.5 Test: oltp_read_write - Threads: 128 v6.11 v6.12 v6.13 7 Dec 40K 80K 120K 160K 200K SE +/- 155.52, N = 3 SE +/- 60.34, N = 3 SE +/- 77.99, N = 3 183693 181192 179909 1. (CXX) g++ options: -pie -fPIC -fstack-protector -O3 -lnuma -lpcre2-8 -lcrypt -laio -lz -lm -lssl -lcrypto -lpthread -ldl
MariaDB Test: oltp_write_only - Threads: 128 OpenBenchmarking.org Queries Per Second, More Is Better MariaDB 11.5 Test: oltp_write_only - Threads: 128 v6.11 v6.12 v6.13 7 Dec 90K 180K 270K 360K 450K SE +/- 286.92, N = 3 SE +/- 474.96, N = 3 SE +/- 154.44, N = 3 412967 399417 386983 1. (CXX) g++ options: -pie -fPIC -fstack-protector -O3 -lnuma -lpcre2-8 -lcrypt -laio -lz -lm -lssl -lcrypto -lpthread -ldl
MariaDB Test: oltp_update_index - Threads: 128 OpenBenchmarking.org Queries Per Second, More Is Better MariaDB 11.5 Test: oltp_update_index - Threads: 128 v6.11 v6.12 v6.13 7 Dec 40K 80K 120K 160K 200K SE +/- 123.94, N = 3 SE +/- 76.99, N = 3 SE +/- 112.30, N = 3 192098 183537 158304 1. (CXX) g++ options: -pie -fPIC -fstack-protector -O3 -lnuma -lpcre2-8 -lcrypt -laio -lz -lm -lssl -lcrypto -lpthread -ldl
PostgreSQL Scaling Factor: 100 - Clients: 800 - Mode: Read Only OpenBenchmarking.org TPS, More Is Better PostgreSQL 17 Scaling Factor: 100 - Clients: 800 - Mode: Read Only v6.11 v6.12 v6.13 7 Dec 1000K 2000K 3000K 4000K 5000K SE +/- 11484.93, N = 3 SE +/- 9902.78, N = 3 SE +/- 8981.60, N = 3 4783648 4815625 4829255 1. (CC) gcc options: -fno-strict-aliasing -fwrapv -O2 -lpq -lpgcommon -lpgport -lm
PostgreSQL Scaling Factor: 100 - Clients: 800 - Mode: Read Only - Average Latency OpenBenchmarking.org ms, Fewer Is Better PostgreSQL 17 Scaling Factor: 100 - Clients: 800 - Mode: Read Only - Average Latency v6.11 v6.12 v6.13 7 Dec 0.0376 0.0752 0.1128 0.1504 0.188 SE +/- 0.001, N = 3 SE +/- 0.000, N = 3 SE +/- 0.000, N = 3 0.167 0.166 0.165 1. (CC) gcc options: -fno-strict-aliasing -fwrapv -O2 -lpq -lpgcommon -lpgport -lm
PostgreSQL Scaling Factor: 100 - Clients: 1000 - Mode: Read Only OpenBenchmarking.org TPS, More Is Better PostgreSQL 17 Scaling Factor: 100 - Clients: 1000 - Mode: Read Only v6.11 v6.12 v6.13 7 Dec 1000K 2000K 3000K 4000K 5000K SE +/- 23812.59, N = 3 SE +/- 18553.35, N = 3 SE +/- 9098.55, N = 3 4621379 4732753 4735578 1. (CC) gcc options: -fno-strict-aliasing -fwrapv -O2 -lpq -lpgcommon -lpgport -lm
PostgreSQL Scaling Factor: 100 - Clients: 1000 - Mode: Read Only - Average Latency OpenBenchmarking.org ms, Fewer Is Better PostgreSQL 17 Scaling Factor: 100 - Clients: 1000 - Mode: Read Only - Average Latency v6.11 v6.12 v6.13 7 Dec 0.0488 0.0976 0.1464 0.1952 0.244 SE +/- 0.001, N = 3 SE +/- 0.001, N = 3 SE +/- 0.001, N = 3 0.217 0.211 0.211 1. (CC) gcc options: -fno-strict-aliasing -fwrapv -O2 -lpq -lpgcommon -lpgport -lm
PostgreSQL Scaling Factor: 100 - Clients: 800 - Mode: Read Write OpenBenchmarking.org TPS, More Is Better PostgreSQL 17 Scaling Factor: 100 - Clients: 800 - Mode: Read Write v6.11 v6.12 v6.13 7 Dec 30K 60K 90K 120K 150K SE +/- 594.42, N = 3 SE +/- 1549.82, N = 3 SE +/- 536.64, N = 3 126877 122181 120703 1. (CC) gcc options: -fno-strict-aliasing -fwrapv -O2 -lpq -lpgcommon -lpgport -lm
PostgreSQL Scaling Factor: 100 - Clients: 800 - Mode: Read Write - Average Latency OpenBenchmarking.org ms, Fewer Is Better PostgreSQL 17 Scaling Factor: 100 - Clients: 800 - Mode: Read Write - Average Latency v6.11 v6.12 v6.13 7 Dec 2 4 6 8 10 SE +/- 0.030, N = 3 SE +/- 0.082, N = 3 SE +/- 0.030, N = 3 6.305 6.550 6.628 1. (CC) gcc options: -fno-strict-aliasing -fwrapv -O2 -lpq -lpgcommon -lpgport -lm
PostgreSQL Scaling Factor: 100 - Clients: 1000 - Mode: Read Write OpenBenchmarking.org TPS, More Is Better PostgreSQL 17 Scaling Factor: 100 - Clients: 1000 - Mode: Read Write v6.11 v6.12 v6.13 7 Dec 30K 60K 90K 120K 150K SE +/- 274.82, N = 3 SE +/- 604.26, N = 3 SE +/- 515.38, N = 3 117677 112025 110112 1. (CC) gcc options: -fno-strict-aliasing -fwrapv -O2 -lpq -lpgcommon -lpgport -lm
PostgreSQL Scaling Factor: 100 - Clients: 1000 - Mode: Read Write - Average Latency OpenBenchmarking.org ms, Fewer Is Better PostgreSQL 17 Scaling Factor: 100 - Clients: 1000 - Mode: Read Write - Average Latency v6.11 v6.12 v6.13 7 Dec 3 6 9 12 15 SE +/- 0.020, N = 3 SE +/- 0.048, N = 3 SE +/- 0.042, N = 3 8.498 8.927 9.082 1. (CC) gcc options: -fno-strict-aliasing -fwrapv -O2 -lpq -lpgcommon -lpgport -lm
Stress-NG Test: NUMA OpenBenchmarking.org Bogo Ops/s, More Is Better Stress-NG 0.17.08 Test: NUMA v6.11 v6.12 v6.13 7 Dec 500 1000 1500 2000 2500 SE +/- 6.78, N = 3 SE +/- 1.02, N = 3 SE +/- 10.25, N = 3 2071.98 2298.65 2385.59 1. (CXX) g++ options: -O2 -std=gnu99 -lc -lm
Stress-NG Test: Futex OpenBenchmarking.org Bogo Ops/s, More Is Better Stress-NG 0.17.08 Test: Futex v6.11 v6.12 v6.13 7 Dec 900K 1800K 2700K 3600K 4500K SE +/- 17548.84, N = 3 SE +/- 15124.18, N = 3 SE +/- 31680.82, N = 3 4154651.85 4313954.94 4398887.77 1. (CXX) g++ options: -O2 -std=gnu99 -lc -lm
Stress-NG Test: MEMFD OpenBenchmarking.org Bogo Ops/s, More Is Better Stress-NG 0.17.08 Test: MEMFD v6.11 v6.12 v6.13 7 Dec 500 1000 1500 2000 2500 SE +/- 4.79, N = 3 SE +/- 2.72, N = 3 SE +/- 3.27, N = 3 2054.70 2010.99 2125.13 1. (CXX) g++ options: -O2 -std=gnu99 -lc -lm
Stress-NG Test: Mutex OpenBenchmarking.org Bogo Ops/s, More Is Better Stress-NG 0.17.08 Test: Mutex v6.11 v6.12 v6.13 7 Dec 10M 20M 30M 40M 50M SE +/- 181454.22, N = 3 SE +/- 442253.04, N = 3 SE +/- 188378.96, N = 3 45376718.94 45398166.14 43186401.84 1. (CXX) g++ options: -O2 -std=gnu99 -lc -lm
Stress-NG Test: Semaphores OpenBenchmarking.org Bogo Ops/s, More Is Better Stress-NG 0.17.08 Test: Semaphores v6.11 v6.12 v6.13 7 Dec 50M 100M 150M 200M 250M SE +/- 2150155.18, N = 15 SE +/- 3074547.79, N = 3 SE +/- 1084579.30, N = 3 233159294.14 251549880.01 246944948.46 1. (CXX) g++ options: -O2 -std=gnu99 -lc -lm
Stress-NG Test: Mixed Scheduler OpenBenchmarking.org Bogo Ops/s, More Is Better Stress-NG 0.17.08 Test: Mixed Scheduler v6.11 v6.12 v6.13 7 Dec 20K 40K 60K 80K 100K SE +/- 161.54, N = 3 SE +/- 215.02, N = 3 SE +/- 282.83, N = 3 79773.37 82674.98 81582.58 1. (CXX) g++ options: -O2 -std=gnu99 -lc -lm
Stress-NG Test: Socket Activity OpenBenchmarking.org Bogo Ops/s, More Is Better Stress-NG 0.17.08 Test: Socket Activity v6.11 v6.12 v6.13 7 Dec 10K 20K 30K 40K 50K SE +/- 29.82, N = 3 SE +/- 68.22, N = 3 SE +/- 42.06, N = 3 47424.87 47501.04 47238.70 1. (CXX) g++ options: -O2 -std=gnu99 -lc -lm
Stress-NG Test: Context Switching OpenBenchmarking.org Bogo Ops/s, More Is Better Stress-NG 0.17.08 Test: Context Switching v6.11 v6.12 v6.13 7 Dec 12M 24M 36M 48M 60M SE +/- 247154.69, N = 13 SE +/- 64836.17, N = 3 SE +/- 159927.59, N = 3 36748330.54 55755486.32 53038526.00 1. (CXX) g++ options: -O2 -std=gnu99 -lc -lm
Blender Blend File: BMW27 - Compute: CPU-Only OpenBenchmarking.org Seconds, Fewer Is Better Blender 4.3 Blend File: BMW27 - Compute: CPU-Only v6.11 v6.12 v6.13 7 Dec 4 8 12 16 20 SE +/- 0.00, N = 3 SE +/- 0.05, N = 3 SE +/- 0.04, N = 3 15.19 15.08 15.02
Blender Blend File: Junkshop - Compute: CPU-Only OpenBenchmarking.org Seconds, Fewer Is Better Blender 4.3 Blend File: Junkshop - Compute: CPU-Only v6.11 v6.12 v6.13 7 Dec 5 10 15 20 25 SE +/- 0.04, N = 3 SE +/- 0.06, N = 3 SE +/- 0.06, N = 3 20.24 20.21 20.20
Blender Blend File: Classroom - Compute: CPU-Only OpenBenchmarking.org Seconds, Fewer Is Better Blender 4.3 Blend File: Classroom - Compute: CPU-Only v6.11 v6.12 v6.13 7 Dec 10 20 30 40 50 SE +/- 0.01, N = 3 SE +/- 0.05, N = 3 SE +/- 0.04, N = 3 42.03 41.74 41.49
Blender Blend File: Fishy Cat - Compute: CPU-Only OpenBenchmarking.org Seconds, Fewer Is Better Blender 4.3 Blend File: Fishy Cat - Compute: CPU-Only v6.11 v6.12 v6.13 7 Dec 5 10 15 20 25 SE +/- 0.12, N = 3 SE +/- 0.06, N = 3 SE +/- 0.06, N = 3 21.66 21.49 21.35
Blender Blend File: Barbershop - Compute: CPU-Only OpenBenchmarking.org Seconds, Fewer Is Better Blender 4.3 Blend File: Barbershop - Compute: CPU-Only v6.11 v6.12 v6.13 7 Dec 30 60 90 120 150 SE +/- 0.08, N = 3 SE +/- 0.18, N = 3 SE +/- 0.23, N = 3 149.30 148.82 148.16
Blender Blend File: Pabellon Barcelona - Compute: CPU-Only OpenBenchmarking.org Seconds, Fewer Is Better Blender 4.3 Blend File: Pabellon Barcelona - Compute: CPU-Only v6.11 v6.12 v6.13 7 Dec 11 22 33 44 55 SE +/- 0.05, N = 3 SE +/- 0.08, N = 3 SE +/- 0.02, N = 3 47.90 47.70 47.39
Apache Cassandra Test: Writes OpenBenchmarking.org Op/s, More Is Better Apache Cassandra 5.0 Test: Writes v6.11 v6.12 v6.13 7 Dec 110K 220K 330K 440K 550K SE +/- 5140.53, N = 3 SE +/- 1282.82, N = 3 SE +/- 4648.88, N = 3 483952 491186 487743
Llamafile Model: Llama-3.2-3B-Instruct.Q6_K - Test: Text Generation 16 OpenBenchmarking.org Tokens Per Second, More Is Better Llamafile 0.8.16 Model: Llama-3.2-3B-Instruct.Q6_K - Test: Text Generation 16 v6.11 v6.12 v6.13 7 Dec 20 40 60 80 100 SE +/- 2.48, N = 12 SE +/- 2.55, N = 12 SE +/- 2.52, N = 14 108.22 107.95 102.21
Llamafile Model: Llama-3.2-3B-Instruct.Q6_K - Test: Text Generation 128 OpenBenchmarking.org Tokens Per Second, More Is Better Llamafile 0.8.16 Model: Llama-3.2-3B-Instruct.Q6_K - Test: Text Generation 128 v6.11 v6.12 v6.13 7 Dec 30 60 90 120 150 SE +/- 0.87, N = 3 SE +/- 1.04, N = 3 SE +/- 1.53, N = 3 112.56 111.98 109.60
Llamafile Model: Llama-3.2-3B-Instruct.Q6_K - Test: Prompt Processing 256 OpenBenchmarking.org Tokens Per Second, More Is Better Llamafile 0.8.16 Model: Llama-3.2-3B-Instruct.Q6_K - Test: Prompt Processing 256 v6.11 v6.12 v6.13 7 Dec 900 1800 2700 3600 4500 SE +/- 0.00, N = 3 SE +/- 0.00, N = 3 SE +/- 0.00, N = 3 4096 4096 4096
Llamafile Model: Llama-3.2-3B-Instruct.Q6_K - Test: Prompt Processing 512 OpenBenchmarking.org Tokens Per Second, More Is Better Llamafile 0.8.16 Model: Llama-3.2-3B-Instruct.Q6_K - Test: Prompt Processing 512 v6.11 v6.12 v6.13 7 Dec 2K 4K 6K 8K 10K SE +/- 0.00, N = 3 SE +/- 0.00, N = 3 SE +/- 0.00, N = 3 8192 8192 8192
Llamafile Model: TinyLlama-1.1B-Chat-v1.0.BF16 - Test: Text Generation 16 OpenBenchmarking.org Tokens Per Second, More Is Better Llamafile 0.8.16 Model: TinyLlama-1.1B-Chat-v1.0.BF16 - Test: Text Generation 16 v6.11 v6.12 v6.13 7 Dec 30 60 90 120 150 SE +/- 4.56, N = 12 SE +/- 4.63, N = 12 SE +/- 4.35, N = 12 152.07 152.89 147.98
Llamafile Model: Llama-3.2-3B-Instruct.Q6_K - Test: Prompt Processing 1024 OpenBenchmarking.org Tokens Per Second, More Is Better Llamafile 0.8.16 Model: Llama-3.2-3B-Instruct.Q6_K - Test: Prompt Processing 1024 v6.11 v6.12 v6.13 7 Dec 4K 8K 12K 16K 20K SE +/- 0.00, N = 3 SE +/- 0.00, N = 3 SE +/- 0.00, N = 3 16384 16384 16384
Llamafile Model: Llama-3.2-3B-Instruct.Q6_K - Test: Prompt Processing 2048 OpenBenchmarking.org Tokens Per Second, More Is Better Llamafile 0.8.16 Model: Llama-3.2-3B-Instruct.Q6_K - Test: Prompt Processing 2048 v6.11 v6.12 v6.13 7 Dec 7K 14K 21K 28K 35K SE +/- 0.00, N = 3 SE +/- 0.00, N = 3 SE +/- 0.00, N = 3 32768 32768 32768
Llamafile Model: TinyLlama-1.1B-Chat-v1.0.BF16 - Test: Text Generation 128 OpenBenchmarking.org Tokens Per Second, More Is Better Llamafile 0.8.16 Model: TinyLlama-1.1B-Chat-v1.0.BF16 - Test: Text Generation 128 v6.11 v6.12 v6.13 7 Dec 40 80 120 160 200 SE +/- 1.11, N = 3 SE +/- 1.26, N = 3 SE +/- 1.45, N = 15 159.90 159.60 156.82
Llamafile Model: mistral-7b-instruct-v0.2.Q5_K_M - Test: Text Generation 16 OpenBenchmarking.org Tokens Per Second, More Is Better Llamafile 0.8.16 Model: mistral-7b-instruct-v0.2.Q5_K_M - Test: Text Generation 16 v6.11 v6.12 v6.13 7 Dec 14 28 42 56 70 SE +/- 1.73, N = 12 SE +/- 1.75, N = 12 SE +/- 1.70, N = 12 64.87 64.80 64.15
Llamafile Model: TinyLlama-1.1B-Chat-v1.0.BF16 - Test: Prompt Processing 256 OpenBenchmarking.org Tokens Per Second, More Is Better Llamafile 0.8.16 Model: TinyLlama-1.1B-Chat-v1.0.BF16 - Test: Prompt Processing 256 v6.11 v6.12 v6.13 7 Dec 900 1800 2700 3600 4500 SE +/- 0.00, N = 3 SE +/- 0.00, N = 3 SE +/- 0.00, N = 3 4096 4096 4096
Llamafile Model: TinyLlama-1.1B-Chat-v1.0.BF16 - Test: Prompt Processing 512 OpenBenchmarking.org Tokens Per Second, More Is Better Llamafile 0.8.16 Model: TinyLlama-1.1B-Chat-v1.0.BF16 - Test: Prompt Processing 512 v6.11 v6.12 v6.13 7 Dec 2K 4K 6K 8K 10K SE +/- 0.00, N = 3 SE +/- 0.00, N = 3 SE +/- 0.00, N = 3 8192 8192 8192
Llamafile Model: mistral-7b-instruct-v0.2.Q5_K_M - Test: Text Generation 128 OpenBenchmarking.org Tokens Per Second, More Is Better Llamafile 0.8.16 Model: mistral-7b-instruct-v0.2.Q5_K_M - Test: Text Generation 128 v6.11 v6.12 v6.13 7 Dec 15 30 45 60 75 SE +/- 0.37, N = 3 SE +/- 0.47, N = 3 SE +/- 0.36, N = 3 68.13 67.71 65.45
Llamafile Model: wizardcoder-python-34b-v1.0.Q6_K - Test: Text Generation 16 OpenBenchmarking.org Tokens Per Second, More Is Better Llamafile 0.8.16 Model: wizardcoder-python-34b-v1.0.Q6_K - Test: Text Generation 16 v6.11 v6.12 v6.13 7 Dec 4 8 12 16 20 SE +/- 0.74, N = 12 SE +/- 0.74, N = 12 SE +/- 0.73, N = 12 14.98 15.01 14.90
Llamafile Model: TinyLlama-1.1B-Chat-v1.0.BF16 - Test: Prompt Processing 1024 OpenBenchmarking.org Tokens Per Second, More Is Better Llamafile 0.8.16 Model: TinyLlama-1.1B-Chat-v1.0.BF16 - Test: Prompt Processing 1024 v6.11 v6.12 v6.13 7 Dec 4K 8K 12K 16K 20K SE +/- 0.00, N = 3 SE +/- 0.00, N = 3 SE +/- 0.00, N = 3 16384 16384 16384
Llamafile Model: TinyLlama-1.1B-Chat-v1.0.BF16 - Test: Prompt Processing 2048 OpenBenchmarking.org Tokens Per Second, More Is Better Llamafile 0.8.16 Model: TinyLlama-1.1B-Chat-v1.0.BF16 - Test: Prompt Processing 2048 v6.11 v6.12 v6.13 7 Dec 7K 14K 21K 28K 35K SE +/- 0.00, N = 3 SE +/- 0.00, N = 3 SE +/- 0.00, N = 3 32768 32768 32768
Llamafile Model: wizardcoder-python-34b-v1.0.Q6_K - Test: Text Generation 128 OpenBenchmarking.org Tokens Per Second, More Is Better Llamafile 0.8.16 Model: wizardcoder-python-34b-v1.0.Q6_K - Test: Text Generation 128 v6.11 v6.12 v6.13 7 Dec 4 8 12 16 20 SE +/- 0.02, N = 3 SE +/- 0.04, N = 3 SE +/- 0.02, N = 3 15.90 15.82 15.77
Llamafile Model: mistral-7b-instruct-v0.2.Q5_K_M - Test: Prompt Processing 256 OpenBenchmarking.org Tokens Per Second, More Is Better Llamafile 0.8.16 Model: mistral-7b-instruct-v0.2.Q5_K_M - Test: Prompt Processing 256 v6.11 v6.12 v6.13 7 Dec 900 1800 2700 3600 4500 SE +/- 0.00, N = 3 SE +/- 0.00, N = 3 SE +/- 0.00, N = 3 4096 4096 4096
Llamafile Model: mistral-7b-instruct-v0.2.Q5_K_M - Test: Prompt Processing 512 OpenBenchmarking.org Tokens Per Second, More Is Better Llamafile 0.8.16 Model: mistral-7b-instruct-v0.2.Q5_K_M - Test: Prompt Processing 512 v6.11 v6.12 v6.13 7 Dec 2K 4K 6K 8K 10K SE +/- 0.00, N = 3 SE +/- 0.00, N = 3 SE +/- 0.00, N = 3 8192 8192 8192
Llamafile Model: mistral-7b-instruct-v0.2.Q5_K_M - Test: Prompt Processing 1024 OpenBenchmarking.org Tokens Per Second, More Is Better Llamafile 0.8.16 Model: mistral-7b-instruct-v0.2.Q5_K_M - Test: Prompt Processing 1024 v6.11 v6.12 v6.13 7 Dec 4K 8K 12K 16K 20K SE +/- 0.00, N = 3 SE +/- 0.00, N = 3 SE +/- 0.00, N = 3 16384 16384 16384
Llamafile Model: mistral-7b-instruct-v0.2.Q5_K_M - Test: Prompt Processing 2048 OpenBenchmarking.org Tokens Per Second, More Is Better Llamafile 0.8.16 Model: mistral-7b-instruct-v0.2.Q5_K_M - Test: Prompt Processing 2048 v6.11 v6.12 v6.13 7 Dec 7K 14K 21K 28K 35K SE +/- 0.00, N = 3 SE +/- 0.00, N = 3 SE +/- 0.00, N = 3 32768 32768 32768
Llamafile Model: wizardcoder-python-34b-v1.0.Q6_K - Test: Prompt Processing 256 OpenBenchmarking.org Tokens Per Second, More Is Better Llamafile 0.8.16 Model: wizardcoder-python-34b-v1.0.Q6_K - Test: Prompt Processing 256 v6.11 v6.12 v6.13 7 Dec 300 600 900 1200 1500 SE +/- 0.00, N = 3 SE +/- 0.00, N = 3 SE +/- 0.00, N = 3 1536 1536 1536
Llamafile Model: wizardcoder-python-34b-v1.0.Q6_K - Test: Prompt Processing 512 OpenBenchmarking.org Tokens Per Second, More Is Better Llamafile 0.8.16 Model: wizardcoder-python-34b-v1.0.Q6_K - Test: Prompt Processing 512 v6.11 v6.12 v6.13 7 Dec 700 1400 2100 2800 3500 SE +/- 0.00, N = 3 SE +/- 0.00, N = 3 SE +/- 0.00, N = 3 3072 3072 3072
Llamafile Model: wizardcoder-python-34b-v1.0.Q6_K - Test: Prompt Processing 1024 OpenBenchmarking.org Tokens Per Second, More Is Better Llamafile 0.8.16 Model: wizardcoder-python-34b-v1.0.Q6_K - Test: Prompt Processing 1024 v6.11 v6.12 v6.13 7 Dec 1300 2600 3900 5200 6500 SE +/- 0.00, N = 3 SE +/- 0.00, N = 3 SE +/- 0.00, N = 3 6144 6144 6144
Llamafile Model: wizardcoder-python-34b-v1.0.Q6_K - Test: Prompt Processing 2048 OpenBenchmarking.org Tokens Per Second, More Is Better Llamafile 0.8.16 Model: wizardcoder-python-34b-v1.0.Q6_K - Test: Prompt Processing 2048 v6.11 v6.12 v6.13 7 Dec 3K 6K 9K 12K 15K SE +/- 0.00, N = 3 SE +/- 0.00, N = 3 SE +/- 0.00, N = 3 12288 12288 12288
OpenVINO GenAI Model: Gemma-7b-int4-ov - Device: CPU OpenBenchmarking.org tokens/s, More Is Better OpenVINO GenAI 2024.5 Model: Gemma-7b-int4-ov - Device: CPU v6.11 v6.12 v6.13 7 Dec 11 22 33 44 55 SE +/- 0.15, N = 3 SE +/- 0.10, N = 3 SE +/- 0.27, N = 3 49.13 49.09 48.70
OpenVINO GenAI Model: Gemma-7b-int4-ov - Device: CPU - Time To First Token OpenBenchmarking.org ms, Fewer Is Better OpenVINO GenAI 2024.5 Model: Gemma-7b-int4-ov - Device: CPU - Time To First Token v6.11 v6.12 v6.13 7 Dec 7 14 21 28 35 SE +/- 0.08, N = 3 SE +/- 0.04, N = 3 SE +/- 0.18, N = 3 30.39 30.27 30.48
OpenVINO GenAI Model: Gemma-7b-int4-ov - Device: CPU - Time Per Output Token OpenBenchmarking.org ms, Fewer Is Better OpenVINO GenAI 2024.5 Model: Gemma-7b-int4-ov - Device: CPU - Time Per Output Token v6.11 v6.12 v6.13 7 Dec 5 10 15 20 25 SE +/- 0.06, N = 3 SE +/- 0.04, N = 3 SE +/- 0.11, N = 3 20.36 20.37 20.54
OpenVINO GenAI Model: TinyLlama-1.1B-Chat-v1.0 - Device: CPU OpenBenchmarking.org tokens/s, More Is Better OpenVINO GenAI 2024.5 Model: TinyLlama-1.1B-Chat-v1.0 - Device: CPU v6.11 v6.12 v6.13 7 Dec 20 40 60 80 100 SE +/- 0.57, N = 3 SE +/- 0.23, N = 3 SE +/- 0.28, N = 3 78.36 78.18 77.67
OpenVINO GenAI Model: TinyLlama-1.1B-Chat-v1.0 - Device: CPU - Time To First Token OpenBenchmarking.org ms, Fewer Is Better OpenVINO GenAI 2024.5 Model: TinyLlama-1.1B-Chat-v1.0 - Device: CPU - Time To First Token v6.11 v6.12 v6.13 7 Dec 4 8 12 16 20 SE +/- 0.19, N = 3 SE +/- 0.05, N = 3 SE +/- 0.08, N = 3 15.86 15.89 16.05
OpenVINO GenAI Model: TinyLlama-1.1B-Chat-v1.0 - Device: CPU - Time Per Output Token OpenBenchmarking.org ms, Fewer Is Better OpenVINO GenAI 2024.5 Model: TinyLlama-1.1B-Chat-v1.0 - Device: CPU - Time Per Output Token v6.11 v6.12 v6.13 7 Dec 3 6 9 12 15 SE +/- 0.09, N = 3 SE +/- 0.04, N = 3 SE +/- 0.04, N = 3 12.76 12.79 12.88
OpenVINO GenAI Model: Falcon-7b-instruct-int4-ov - Device: CPU OpenBenchmarking.org tokens/s, More Is Better OpenVINO GenAI 2024.5 Model: Falcon-7b-instruct-int4-ov - Device: CPU v6.11 v6.12 v6.13 7 Dec 13 26 39 52 65 SE +/- 0.63, N = 3 SE +/- 0.55, N = 6 SE +/- 0.54, N = 3 56.73 56.21 56.28
OpenVINO GenAI Model: Falcon-7b-instruct-int4-ov - Device: CPU - Time To First Token OpenBenchmarking.org ms, Fewer Is Better OpenVINO GenAI 2024.5 Model: Falcon-7b-instruct-int4-ov - Device: CPU - Time To First Token v6.11 v6.12 v6.13 7 Dec 7 14 21 28 35 SE +/- 0.27, N = 3 SE +/- 0.26, N = 6 SE +/- 0.44, N = 3 30.39 30.00 30.18
OpenVINO GenAI Model: Falcon-7b-instruct-int4-ov - Device: CPU - Time Per Output Token OpenBenchmarking.org ms, Fewer Is Better OpenVINO GenAI 2024.5 Model: Falcon-7b-instruct-int4-ov - Device: CPU - Time Per Output Token v6.11 v6.12 v6.13 7 Dec 4 8 12 16 20 SE +/- 0.19, N = 3 SE +/- 0.17, N = 6 SE +/- 0.17, N = 3 17.63 17.80 17.77
OpenVINO GenAI Model: Phi-3-mini-128k-instruct-int4-ov - Device: CPU OpenBenchmarking.org tokens/s, More Is Better OpenVINO GenAI 2024.5 Model: Phi-3-mini-128k-instruct-int4-ov - Device: CPU v6.11 v6.12 v6.13 7 Dec 15 30 45 60 75 SE +/- 0.41, N = 3 SE +/- 0.10, N = 3 SE +/- 0.79, N = 3 65.89 65.82 65.91
OpenVINO GenAI Model: Phi-3-mini-128k-instruct-int4-ov - Device: CPU - Time To First Token OpenBenchmarking.org ms, Fewer Is Better OpenVINO GenAI 2024.5 Model: Phi-3-mini-128k-instruct-int4-ov - Device: CPU - Time To First Token v6.11 v6.12 v6.13 7 Dec 5 10 15 20 25 SE +/- 0.24, N = 3 SE +/- 0.37, N = 3 SE +/- 0.43, N = 3 22.66 22.73 22.38
OpenVINO GenAI Model: Phi-3-mini-128k-instruct-int4-ov - Device: CPU - Time Per Output Token OpenBenchmarking.org ms, Fewer Is Better OpenVINO GenAI 2024.5 Model: Phi-3-mini-128k-instruct-int4-ov - Device: CPU - Time Per Output Token v6.11 v6.12 v6.13 7 Dec 4 8 12 16 20 SE +/- 0.10, N = 3 SE +/- 0.02, N = 3 SE +/- 0.18, N = 3 15.18 15.19 15.17
Phoronix Test Suite v10.8.5