Linux 6.13 AMD EPYC Performance

Benchmarks for a future article. AMD EPYC 9575F 64-Core testing with a Supermicro Super Server H13SSL-N v1.01 (3.0 BIOS) and ASPEED on Ubuntu 24.10 via the Phoronix Test Suite.

HTML result view exported from: https://openbenchmarking.org/result/2412073-NE-LINUX613A93&sor.

Linux 6.13 AMD EPYC PerformanceProcessorMotherboardChipsetMemoryDiskGraphicsNetworkOSKernelDesktopDisplay ServerCompilerFile-SystemScreen Resolutionv6.11v6.12v6.13 7 DecAMD EPYC 9575F 64-Core @ 3.30GHz (64 Cores / 128 Threads)Supermicro Super Server H13SSL-N v1.01 (3.0 BIOS)AMD 1Ah12 x 64GB DDR5-6000MT/s Micron MTC40F2046S1RC64BDY QSFF3201GB Micron_7450_MTFDKCB3T2TFSASPEED2 x Broadcom NetXtreme BCM5720 PCIeUbuntu 24.106.11.0-phx (x86_64)GNOME Shell 47.0X ServerGCC 14.2.0ext41920x12006.12.0-phx (x86_64)1024x7686.13.0-rc1-phx (x86_64)OpenBenchmarking.orgKernel Details- Transparent Huge Pages: madviseCompiler Details- --build=x86_64-linux-gnu --disable-vtable-verify --disable-werror --enable-bootstrap --enable-cet --enable-checking=release --enable-clocale=gnu --enable-default-pie --enable-gnu-unique-object --enable-languages=c,ada,c++,go,d,fortran,objc,obj-c++,m2,rust --enable-libphobos-checking=release --enable-libstdcxx-backtrace --enable-libstdcxx-debug --enable-libstdcxx-time=yes --enable-link-serialization=2 --enable-multiarch --enable-multilib --enable-nls --enable-objc-gc=auto --enable-offload-defaulted --enable-offload-targets=nvptx-none=/build/gcc-14-zdkDXv/gcc-14-14.2.0/debian/tmp-nvptx/usr,amdgcn-amdhsa=/build/gcc-14-zdkDXv/gcc-14-14.2.0/debian/tmp-gcn/usr --enable-plugin --enable-shared --enable-threads=posix --host=x86_64-linux-gnu --program-prefix=x86_64-linux-gnu- --target=x86_64-linux-gnu --with-abi=m64 --with-arch-32=i686 --with-build-config=bootstrap-lto-lean --with-default-libstdcxx-abi=new --with-gcc-major-version-only --with-multilib-list=m32,m64,mx32 --with-target-system-zlib=auto --with-tune=generic --without-cuda-driver -v Processor Details- Scaling Governor: acpi-cpufreq performance (Boost: Enabled) - CPU Microcode: 0xb002116Java Details- OpenJDK Runtime Environment (build 21.0.5+11-Ubuntu-1ubuntu124.10)Python Details- Python 3.12.7Security Details- gather_data_sampling: Not affected + itlb_multihit: Not affected + l1tf: Not affected + mds: Not affected + meltdown: Not affected + mmio_stale_data: Not affected + reg_file_data_sampling: Not affected + retbleed: Not affected + spec_rstack_overflow: Not affected + spec_store_bypass: Mitigation of SSB disabled via prctl + spectre_v1: Mitigation of usercopy/swapgs barriers and __user pointer sanitization + spectre_v2: Mitigation of Enhanced / Automatic IBRS; IBPB: conditional; STIBP: always-on; RSB filling; PBRSB-eIBRS: Not affected; BHI: Not affected + srbds: Not affected + tsx_async_abort: Not affected

Linux 6.13 AMD EPYC Performancelaghos: Triple Point Problemlaghos: Sedov Blast Wave, ube_922_hex.meshpalabos: 500palabos: 1000relion: Basic - CPUdacapobench: Jythondacapobench: GraphChidacapobench: Tradesoapdacapobench: Tradebeansdacapobench: Apache Tomcatdacapobench: Apache Xalan XSLTdacapobench: H2 Database Enginesvt-av1: Preset 3 - Bosphorus 4Ksvt-av1: Preset 5 - Bosphorus 4Ksvt-av1: Preset 8 - Bosphorus 4Ksvt-av1: Preset 13 - Bosphorus 4Kbuild-linux-kernel: defconfigbuild-linux-kernel: allmodconfigbuild-llvm: Ninjabuild-llvm: Unix Makefilesprimesieve: 1e12primesieve: 1e13openssl: ChaCha20openssl: AES-128-GCMopenssl: AES-256-GCMopenssl: ChaCha20-Poly1305clickhouse: 100M Rows Hits Dataset, First Run / Cold Cacheclickhouse: 100M Rows Hits Dataset, Second Runclickhouse: 100M Rows Hits Dataset, Third Runcouchdb: 300 - 1000 - 30couchdb: 300 - 3000 - 30couchdb: 500 - 1000 - 30dacapobench: Eclipsedacapobench: Spring Bootdacapobench: Apache Kafkadacapobench: jMonkeyEnginedacapobench: Batik SVG Toolkitdacapobench: FOP Print Formatterdacapobench: PMD Source Code Analyzerdacapobench: Apache Lucene Search Indexdacapobench: Apache Lucene Search Enginedacapobench: Avrora AVR Simulation Frameworkdacapobench: BioJava Biological Data Frameworkdacapobench: Zxing 1D/2D Barcode Image Processingmemcached: 1:5memcached: 1:10memcached: 1:100mariadb: oltp_read_only - 128mariadb: oltp_read_write - 128mariadb: oltp_write_only - 128mariadb: oltp_update_index - 128pgbench: 100 - 800 - Read Onlypgbench: 100 - 800 - Read Only - Average Latencypgbench: 100 - 1000 - Read Onlypgbench: 100 - 1000 - Read Only - Average Latencypgbench: 100 - 800 - Read Writepgbench: 100 - 800 - Read Write - Average Latencypgbench: 100 - 1000 - Read Writepgbench: 100 - 1000 - Read Write - Average Latencystress-ng: NUMAstress-ng: Futexstress-ng: MEMFDstress-ng: Mutexstress-ng: Semaphoresstress-ng: Mixed Schedulerstress-ng: Socket Activitystress-ng: Context Switchingblender: BMW27 - CPU-Onlyblender: Junkshop - CPU-Onlyblender: Classroom - CPU-Onlyblender: Fishy Cat - CPU-Onlyblender: Barbershop - CPU-Onlyblender: Pabellon Barcelona - CPU-Onlycassandra: Writesllamafile: Llama-3.2-3B-Instruct.Q6_K - Text Generation 16llamafile: Llama-3.2-3B-Instruct.Q6_K - Text Generation 128llamafile: Llama-3.2-3B-Instruct.Q6_K - Prompt Processing 256llamafile: Llama-3.2-3B-Instruct.Q6_K - Prompt Processing 512llamafile: TinyLlama-1.1B-Chat-v1.0.BF16 - Text Generation 16llamafile: Llama-3.2-3B-Instruct.Q6_K - Prompt Processing 1024llamafile: Llama-3.2-3B-Instruct.Q6_K - Prompt Processing 2048llamafile: TinyLlama-1.1B-Chat-v1.0.BF16 - Text Generation 128llamafile: mistral-7b-instruct-v0.2.Q5_K_M - Text Generation 16llamafile: TinyLlama-1.1B-Chat-v1.0.BF16 - Prompt Processing 256llamafile: TinyLlama-1.1B-Chat-v1.0.BF16 - Prompt Processing 512llamafile: mistral-7b-instruct-v0.2.Q5_K_M - Text Generation 128llamafile: wizardcoder-python-34b-v1.0.Q6_K - Text Generation 16llamafile: TinyLlama-1.1B-Chat-v1.0.BF16 - Prompt Processing 1024llamafile: TinyLlama-1.1B-Chat-v1.0.BF16 - Prompt Processing 2048llamafile: wizardcoder-python-34b-v1.0.Q6_K - Text Generation 128llamafile: mistral-7b-instruct-v0.2.Q5_K_M - Prompt Processing 256llamafile: mistral-7b-instruct-v0.2.Q5_K_M - Prompt Processing 512llamafile: mistral-7b-instruct-v0.2.Q5_K_M - Prompt Processing 1024llamafile: mistral-7b-instruct-v0.2.Q5_K_M - Prompt Processing 2048llamafile: wizardcoder-python-34b-v1.0.Q6_K - Prompt Processing 256llamafile: wizardcoder-python-34b-v1.0.Q6_K - Prompt Processing 512llamafile: wizardcoder-python-34b-v1.0.Q6_K - Prompt Processing 1024llamafile: wizardcoder-python-34b-v1.0.Q6_K - Prompt Processing 2048openvino-genai: Gemma-7b-int4-ov - CPUopenvino-genai: Gemma-7b-int4-ov - CPU - Time To First Tokenopenvino-genai: Gemma-7b-int4-ov - CPU - Time Per Output Tokenopenvino-genai: TinyLlama-1.1B-Chat-v1.0 - CPUopenvino-genai: TinyLlama-1.1B-Chat-v1.0 - CPU - Time To First Tokenopenvino-genai: TinyLlama-1.1B-Chat-v1.0 - CPU - Time Per Output Tokenopenvino-genai: Falcon-7b-instruct-int4-ov - CPUopenvino-genai: Falcon-7b-instruct-int4-ov - CPU - Time To First Tokenopenvino-genai: Falcon-7b-instruct-int4-ov - CPU - Time Per Output Tokenopenvino-genai: Phi-3-mini-128k-instruct-int4-ov - CPUopenvino-genai: Phi-3-mini-128k-instruct-int4-ov - CPU - Time To First Tokenopenvino-genai: Phi-3-mini-128k-instruct-int4-ov - CPU - Time Per Output Tokenv6.11v6.12v6.13 7 Dec297.99561.91529.232481.489176.67337822093307749991053743210416.88060.171199.209466.61421.792194.338101.821159.8191.99925.036718106876590359931303710333162995897487834985450770.47804.06801.19103.228363.527354.5266349272450706791923339108522944120235440585224115352.497421710.2412696519.854368818369341296719209847836480.16746213790.2171268776.3051176778.4982071.984154651.852054.7045376718.94233159294.1479773.3747424.8736748330.5415.1920.2442.0321.66149.3047.90483952108.22112.5640968192152.071638432768159.9064.874096819268.1314.98163843276815.904096819216384327681536307261441228849.1330.3920.3678.3615.8612.7656.7330.3917.6365.8922.6615.18303.11565.10775.110655.184179.86837552083303549681038741204916.99060.381200.316468.66621.848193.202101.599160.3021.99024.972721616676910361564053870334486696117490511085203765.59801.90799.99104.163371.288333.1786280225950456797937335116722584326230340715514064048.147297642.2113294800.594358118119239941718353748156250.16647327530.2111221816.5501120258.9272298.654313954.942010.9945398166.14251549880.0182674.9847501.0455755486.3215.0820.2141.7421.49148.8247.70491186107.95111.9840968192152.891638432768159.6064.804096819267.7115.01163843276815.824096819216384327681536307261441228849.0930.2720.3778.1815.8912.7956.2130.0017.8065.8222.7315.19300.72565.14764.024509.126172.48237632102304949571055742201017.00860.821200.823480.91221.855192.923101.725160.5591.98824.939719980763180361324888880334428157450489486102253774.01803.49807.27100.140360.826311.0386331265150576793938338108922783631228340194934099640.317400996.9813514659.674293617990938698315830448292550.16547355780.2111207036.6281101129.0822385.594398887.772125.1343186401.84246944948.4681582.5847238.7053038526.0015.0220.2041.4921.35148.1647.39487743102.21109.6040968192147.981638432768156.8264.154096819265.4514.90163843276815.774096819216384327681536307261441228848.7030.4820.5477.6716.0512.8856.2830.1817.7765.9122.3815.17OpenBenchmarking.org

Laghos

Test: Triple Point Problem

OpenBenchmarking.orgMajor Kernels Total Rate, More Is BetterLaghos 3.1Test: Triple Point Problemv6.12v6.13 7 Decv6.1170140210280350SE +/- 1.53, N = 3SE +/- 3.23, N = 3SE +/- 3.08, N = 3303.11300.72297.991. (CXX) g++ options: -O3 -std=c++11 -lmfem -lHYPRE -lmetis -lrt -lmpi_cxx -lmpi

Laghos

Test: Sedov Blast Wave, ube_922_hex.mesh

OpenBenchmarking.orgMajor Kernels Total Rate, More Is BetterLaghos 3.1Test: Sedov Blast Wave, ube_922_hex.meshv6.13 7 Decv6.12v6.11120240360480600SE +/- 2.95, N = 3SE +/- 2.47, N = 3SE +/- 3.27, N = 3565.14565.10561.911. (CXX) g++ options: -O3 -std=c++11 -lmfem -lHYPRE -lmetis -lrt -lmpi_cxx -lmpi

Palabos

Grid Size: 500

OpenBenchmarking.orgMega Site Updates Per Second, More Is BetterPalabos 2.3Grid Size: 500v6.12v6.13 7 Decv6.112004006008001000SE +/- 1.19, N = 3SE +/- 5.28, N = 3SE +/- 40.67, N = 12775.11764.02529.231. (CXX) g++ options: -std=c++17 -pedantic -O3 -rdynamic -lcrypto -lcurl -lsz -lz -ldl -lm

Palabos

Grid Size: 1000

OpenBenchmarking.orgMega Site Updates Per Second, More Is BetterPalabos 2.3Grid Size: 1000v6.12v6.13 7 Decv6.11140280420560700SE +/- 36.69, N = 12SE +/- 21.98, N = 15SE +/- 3.07, N = 15655.18509.13481.491. (CXX) g++ options: -std=c++17 -pedantic -O3 -rdynamic -lcrypto -lcurl -lsz -lz -ldl -lm

RELION

Test: Basic - Device: CPU

OpenBenchmarking.orgSeconds, Fewer Is BetterRELION 5.0Test: Basic - Device: CPUv6.13 7 Decv6.11v6.124080120160200SE +/- 3.31, N = 12SE +/- 1.95, N = 3SE +/- 0.79, N = 3172.48176.67179.871. (CXX) g++ options: -fPIC -std=c++14 -fopenmp -O3 -rdynamic -ldl -ltiff -lfftw3f -lfftw3 -lpng -ljpeg -lmpi_cxx -lmpi

DaCapo Benchmark

Java Test: Jython

OpenBenchmarking.orgmsec, Fewer Is BetterDaCapo Benchmark 23.11Java Test: Jythonv6.12v6.13 7 Decv6.118001600240032004000SE +/- 5.33, N = 3SE +/- 11.00, N = 3SE +/- 6.67, N = 3375537633782

DaCapo Benchmark

Java Test: GraphChi

OpenBenchmarking.orgmsec, Fewer Is BetterDaCapo Benchmark 23.11Java Test: GraphChiv6.12v6.11v6.13 7 Dec5001000150020002500SE +/- 5.51, N = 3SE +/- 13.86, N = 3SE +/- 6.00, N = 3208320932102

DaCapo Benchmark

Java Test: Tradesoap

OpenBenchmarking.orgmsec, Fewer Is BetterDaCapo Benchmark 23.11Java Test: Tradesoapv6.12v6.13 7 Decv6.117001400210028003500SE +/- 21.97, N = 15SE +/- 31.71, N = 5SE +/- 24.92, N = 3303530493077

DaCapo Benchmark

Java Test: Tradebeans

OpenBenchmarking.orgmsec, Fewer Is BetterDaCapo Benchmark 23.11Java Test: Tradebeansv6.13 7 Decv6.12v6.1111002200330044005500SE +/- 100.21, N = 12SE +/- 76.59, N = 15SE +/- 48.79, N = 15495749684999

DaCapo Benchmark

Java Test: Apache Tomcat

OpenBenchmarking.orgmsec, Fewer Is BetterDaCapo Benchmark 23.11Java Test: Apache Tomcatv6.12v6.11v6.13 7 Dec2004006008001000SE +/- 3.51, N = 3SE +/- 7.81, N = 3SE +/- 2.00, N = 3103810531055

DaCapo Benchmark

Java Test: Apache Xalan XSLT

OpenBenchmarking.orgmsec, Fewer Is BetterDaCapo Benchmark 23.11Java Test: Apache Xalan XSLTv6.12v6.13 7 Decv6.11160320480640800SE +/- 1.76, N = 3SE +/- 8.54, N = 3SE +/- 6.06, N = 3741742743

DaCapo Benchmark

Java Test: H2 Database Engine

OpenBenchmarking.orgmsec, Fewer Is BetterDaCapo Benchmark 23.11Java Test: H2 Database Enginev6.13 7 Decv6.12v6.115001000150020002500SE +/- 18.21, N = 3SE +/- 13.65, N = 3SE +/- 28.09, N = 3201020492104

SVT-AV1

Encoder Mode: Preset 3 - Input: Bosphorus 4K

OpenBenchmarking.orgFrames Per Second, More Is BetterSVT-AV1 2.3Encoder Mode: Preset 3 - Input: Bosphorus 4Kv6.13 7 Decv6.12v6.1148121620SE +/- 0.02, N = 3SE +/- 0.04, N = 3SE +/- 0.08, N = 317.0116.9916.881. (CXX) g++ options: -march=native -mno-avx -mavx2 -mavx512f -mavx512bw -mavx512dq

SVT-AV1

Encoder Mode: Preset 5 - Input: Bosphorus 4K

OpenBenchmarking.orgFrames Per Second, More Is BetterSVT-AV1 2.3Encoder Mode: Preset 5 - Input: Bosphorus 4Kv6.13 7 Decv6.12v6.111428425670SE +/- 0.08, N = 3SE +/- 0.12, N = 3SE +/- 0.07, N = 360.8260.3860.171. (CXX) g++ options: -march=native -mno-avx -mavx2 -mavx512f -mavx512bw -mavx512dq

SVT-AV1

Encoder Mode: Preset 8 - Input: Bosphorus 4K

OpenBenchmarking.orgFrames Per Second, More Is BetterSVT-AV1 2.3Encoder Mode: Preset 8 - Input: Bosphorus 4Kv6.13 7 Decv6.12v6.114080120160200SE +/- 0.64, N = 3SE +/- 1.77, N = 3SE +/- 1.84, N = 3200.82200.32199.211. (CXX) g++ options: -march=native -mno-avx -mavx2 -mavx512f -mavx512bw -mavx512dq

SVT-AV1

Encoder Mode: Preset 13 - Input: Bosphorus 4K

OpenBenchmarking.orgFrames Per Second, More Is BetterSVT-AV1 2.3Encoder Mode: Preset 13 - Input: Bosphorus 4Kv6.13 7 Decv6.12v6.11100200300400500SE +/- 0.44, N = 3SE +/- 1.95, N = 3SE +/- 3.40, N = 3480.91468.67466.611. (CXX) g++ options: -march=native -mno-avx -mavx2 -mavx512f -mavx512bw -mavx512dq

Timed Linux Kernel Compilation

Build: defconfig

OpenBenchmarking.orgSeconds, Fewer Is BetterTimed Linux Kernel Compilation 6.8Build: defconfigv6.11v6.12v6.13 7 Dec510152025SE +/- 0.25, N = 4SE +/- 0.31, N = 3SE +/- 0.26, N = 421.7921.8521.86

Timed Linux Kernel Compilation

Build: allmodconfig

OpenBenchmarking.orgSeconds, Fewer Is BetterTimed Linux Kernel Compilation 6.8Build: allmodconfigv6.13 7 Decv6.12v6.114080120160200SE +/- 0.25, N = 3SE +/- 0.34, N = 3SE +/- 0.21, N = 3192.92193.20194.34

Timed LLVM Compilation

Build System: Ninja

OpenBenchmarking.orgSeconds, Fewer Is BetterTimed LLVM Compilation 16.0Build System: Ninjav6.12v6.13 7 Decv6.1120406080100SE +/- 0.06, N = 3SE +/- 0.19, N = 3SE +/- 0.11, N = 3101.60101.73101.82

Timed LLVM Compilation

Build System: Unix Makefiles

OpenBenchmarking.orgSeconds, Fewer Is BetterTimed LLVM Compilation 16.0Build System: Unix Makefilesv6.11v6.12v6.13 7 Dec4080120160200SE +/- 0.34, N = 3SE +/- 0.33, N = 3SE +/- 0.86, N = 3159.82160.30160.56

Primesieve

Length: 1e12

OpenBenchmarking.orgSeconds, Fewer Is BetterPrimesieve 12.6Length: 1e12v6.13 7 Decv6.12v6.110.44980.89961.34941.79922.249SE +/- 0.002, N = 3SE +/- 0.001, N = 3SE +/- 0.003, N = 31.9881.9901.9991. (CXX) g++ options: -O3

Primesieve

Length: 1e13

OpenBenchmarking.orgSeconds, Fewer Is BetterPrimesieve 12.6Length: 1e13v6.13 7 Decv6.12v6.11612182430SE +/- 0.02, N = 3SE +/- 0.03, N = 3SE +/- 0.04, N = 324.9424.9725.041. (CXX) g++ options: -O3

OpenSSL

Algorithm: ChaCha20

OpenBenchmarking.orgbyte/s, More Is BetterOpenSSLAlgorithm: ChaCha20v6.12v6.13 7 Decv6.11160000M320000M480000M640000M800000MSE +/- 2136131841.14, N = 3SE +/- 626958911.68, N = 3SE +/- 652717270.91, N = 37246118325807199807631807181068765901. OpenSSL 3.3.1 4 Jun 2024 (Library: OpenSSL 3.3.1 4 Jun 2024) - Additional Parameters: -engine qatengine -async_jobs 8

OpenSSL

Algorithm: AES-128-GCM

OpenBenchmarking.orgbyte/s, More Is BetterOpenSSLAlgorithm: AES-128-GCMv6.12v6.13 7 Decv6.1180000M160000M240000M320000M400000MSE +/- 158067306.31, N = 3SE +/- 88875000.56, N = 3SE +/- 130266226.11, N = 33615640538703613248888803599313037101. OpenSSL 3.3.1 4 Jun 2024 (Library: OpenSSL 3.3.1 4 Jun 2024) - Additional Parameters: -engine qatengine -async_jobs 8

OpenSSL

Algorithm: AES-256-GCM

OpenBenchmarking.orgbyte/s, More Is BetterOpenSSLAlgorithm: AES-256-GCMv6.12v6.13 7 Decv6.1170000M140000M210000M280000M350000MSE +/- 47580595.33, N = 3SE +/- 134252719.45, N = 3SE +/- 105448439.59, N = 33344866961173344281574503331629958971. OpenSSL 3.3.1 4 Jun 2024 (Library: OpenSSL 3.3.1 4 Jun 2024) - Additional Parameters: -engine qatengine -async_jobs 8

OpenSSL

Algorithm: ChaCha20-Poly1305

OpenBenchmarking.orgbyte/s, More Is BetterOpenSSLAlgorithm: ChaCha20-Poly1305v6.12v6.13 7 Decv6.11110000M220000M330000M440000M550000MSE +/- 118571563.57, N = 3SE +/- 158903312.00, N = 3SE +/- 271433364.08, N = 34905110852034894861022534878349854501. OpenSSL 3.3.1 4 Jun 2024 (Library: OpenSSL 3.3.1 4 Jun 2024) - Additional Parameters: -engine qatengine -async_jobs 8

ClickHouse

100M Rows Hits Dataset, First Run / Cold Cache

OpenBenchmarking.orgQueries Per Minute, Geo Mean, More Is BetterClickHouse 22.12.3.5100M Rows Hits Dataset, First Run / Cold Cachev6.12v6.13 7 Decv6.112004006008001000SE +/- 3.20, N = 3SE +/- 2.84, N = 3SE +/- 2.38, N = 3776.70774.01770.47MIN: 65.22 / MAX: 8571.43MIN: 65.86 / MAX: 7500MIN: 65.15 / MAX: 8571.43

ClickHouse

100M Rows Hits Dataset, Second Run

OpenBenchmarking.orgQueries Per Minute, Geo Mean, More Is BetterClickHouse 22.12.3.5100M Rows Hits Dataset, Second Runv6.11v6.13 7 Decv6.122004006008001000SE +/- 4.73, N = 3SE +/- 8.62, N = 3SE +/- 3.55, N = 3804.06803.49801.90MIN: 65.15 / MAX: 10000MIN: 66.08 / MAX: 10000MIN: 65.79 / MAX: 8571.43

ClickHouse

100M Rows Hits Dataset, Third Run

OpenBenchmarking.orgQueries Per Minute, Geo Mean, More Is BetterClickHouse 22.12.3.5100M Rows Hits Dataset, Third Runv6.13 7 Decv6.11v6.122004006008001000SE +/- 2.96, N = 3SE +/- 4.79, N = 3SE +/- 3.72, N = 3807.27801.19800.56MIN: 66.01 / MAX: 8571.43MIN: 65.5 / MAX: 8571.43MIN: 63.63 / MAX: 10000

Apache CouchDB

Bulk Size: 300 - Inserts: 1000 - Rounds: 30

OpenBenchmarking.orgSeconds, Fewer Is BetterApache CouchDB 3.4.1Bulk Size: 300 - Inserts: 1000 - Rounds: 30v6.13 7 Decv6.11v6.1220406080100SE +/- 0.87, N = 7SE +/- 1.11, N = 3SE +/- 0.71, N = 3100.14103.23104.161. (CXX) g++ options: -flto -lstdc++ -shared -lei

Apache CouchDB

Bulk Size: 300 - Inserts: 3000 - Rounds: 30

OpenBenchmarking.orgSeconds, Fewer Is BetterApache CouchDB 3.4.1Bulk Size: 300 - Inserts: 3000 - Rounds: 30v6.13 7 Decv6.11v6.1280160240320400SE +/- 3.45, N = 9SE +/- 6.26, N = 9SE +/- 12.48, N = 9360.83363.53371.291. (CXX) g++ options: -flto -lstdc++ -shared -lei

Apache CouchDB

Bulk Size: 500 - Inserts: 1000 - Rounds: 30

OpenBenchmarking.orgSeconds, Fewer Is BetterApache CouchDB 3.4.1Bulk Size: 500 - Inserts: 1000 - Rounds: 30v6.13 7 Decv6.12v6.1180160240320400SE +/- 29.09, N = 7SE +/- 36.20, N = 9SE +/- 62.71, N = 8311.04333.18354.531. (CXX) g++ options: -flto -lstdc++ -shared -lei

DaCapo Benchmark

Java Test: Eclipse

OpenBenchmarking.orgmsec, Fewer Is BetterDaCapo Benchmark 23.11Java Test: Eclipsev6.12v6.13 7 Decv6.1114002800420056007000SE +/- 33.39, N = 3SE +/- 23.78, N = 3SE +/- 14.75, N = 3628063316349

DaCapo Benchmark

Java Test: Spring Boot

OpenBenchmarking.orgmsec, Fewer Is BetterDaCapo Benchmark 23.11Java Test: Spring Bootv6.12v6.13 7 Decv6.116001200180024003000SE +/- 20.60, N = 3SE +/- 26.77, N = 3SE +/- 17.89, N = 3225926512724

DaCapo Benchmark

Java Test: Apache Kafka

OpenBenchmarking.orgmsec, Fewer Is BetterDaCapo Benchmark 23.11Java Test: Apache Kafkav6.12v6.13 7 Decv6.1111002200330044005500SE +/- 26.64, N = 3SE +/- 3.53, N = 3SE +/- 7.33, N = 3504550575070

DaCapo Benchmark

Java Test: jMonkeyEngine

OpenBenchmarking.orgmsec, Fewer Is BetterDaCapo Benchmark 23.11Java Test: jMonkeyEnginev6.11v6.13 7 Decv6.1215003000450060007500SE +/- 1.86, N = 3SE +/- 1.86, N = 3SE +/- 4.84, N = 3679167936797

DaCapo Benchmark

Java Test: Batik SVG Toolkit

OpenBenchmarking.orgmsec, Fewer Is BetterDaCapo Benchmark 23.11Java Test: Batik SVG Toolkitv6.11v6.12v6.13 7 Dec2004006008001000SE +/- 3.61, N = 3SE +/- 4.00, N = 3SE +/- 7.55, N = 3923937938

DaCapo Benchmark

Java Test: FOP Print Formatter

OpenBenchmarking.orgmsec, Fewer Is BetterDaCapo Benchmark 23.11Java Test: FOP Print Formatterv6.12v6.13 7 Decv6.1170140210280350SE +/- 2.08, N = 3SE +/- 1.20, N = 3SE +/- 0.67, N = 3335338339

DaCapo Benchmark

Java Test: PMD Source Code Analyzer

OpenBenchmarking.orgmsec, Fewer Is BetterDaCapo Benchmark 23.11Java Test: PMD Source Code Analyzerv6.11v6.13 7 Decv6.1230060090012001500SE +/- 5.90, N = 3SE +/- 18.53, N = 15SE +/- 23.97, N = 15108510891167

DaCapo Benchmark

Java Test: Apache Lucene Search Index

OpenBenchmarking.orgmsec, Fewer Is BetterDaCapo Benchmark 23.11Java Test: Apache Lucene Search Indexv6.12v6.13 7 Decv6.115001000150020002500SE +/- 8.39, N = 3SE +/- 25.32, N = 3SE +/- 16.64, N = 3225822782294

DaCapo Benchmark

Java Test: Apache Lucene Search Engine

OpenBenchmarking.orgmsec, Fewer Is BetterDaCapo Benchmark 23.11Java Test: Apache Lucene Search Enginev6.13 7 Decv6.11v6.129001800270036004500SE +/- 42.57, N = 3SE +/- 43.31, N = 15SE +/- 47.07, N = 3363141204326

DaCapo Benchmark

Java Test: Avrora AVR Simulation Framework

OpenBenchmarking.orgmsec, Fewer Is BetterDaCapo Benchmark 23.11Java Test: Avrora AVR Simulation Frameworkv6.13 7 Decv6.12v6.115001000150020002500SE +/- 20.72, N = 15SE +/- 15.56, N = 13SE +/- 16.90, N = 3228323032354

DaCapo Benchmark

Java Test: BioJava Biological Data Framework

OpenBenchmarking.orgmsec, Fewer Is BetterDaCapo Benchmark 23.11Java Test: BioJava Biological Data Frameworkv6.13 7 Decv6.11v6.129001800270036004500SE +/- 22.00, N = 3SE +/- 11.89, N = 3SE +/- 18.98, N = 3401940584071

DaCapo Benchmark

Java Test: Zxing 1D/2D Barcode Image Processing

OpenBenchmarking.orgmsec, Fewer Is BetterDaCapo Benchmark 23.11Java Test: Zxing 1D/2D Barcode Image Processingv6.13 7 Decv6.11v6.12120240360480600SE +/- 5.55, N = 3SE +/- 4.26, N = 3SE +/- 4.38, N = 9493522551

Memcached

Set To Get Ratio: 1:5

OpenBenchmarking.orgOps/sec, More Is BetterMemcached 1.6.19Set To Get Ratio: 1:5v6.11v6.13 7 Decv6.12900K1800K2700K3600K4500KSE +/- 26467.49, N = 3SE +/- 28528.57, N = 3SE +/- 16085.40, N = 34115352.494099640.314064048.141. (CXX) g++ options: -O2 -levent_openssl -levent -lcrypto -lssl -lpthread -lz -lpcre

Memcached

Set To Get Ratio: 1:10

OpenBenchmarking.orgOps/sec, More Is BetterMemcached 1.6.19Set To Get Ratio: 1:10v6.11v6.13 7 Decv6.121.6M3.2M4.8M6.4M8MSE +/- 35717.40, N = 3SE +/- 37597.18, N = 3SE +/- 30399.87, N = 37421710.247400996.987297642.211. (CXX) g++ options: -O2 -levent_openssl -levent -lcrypto -lssl -lpthread -lz -lpcre

Memcached

Set To Get Ratio: 1:100

OpenBenchmarking.orgOps/sec, More Is BetterMemcached 1.6.19Set To Get Ratio: 1:100v6.13 7 Decv6.12v6.113M6M9M12M15MSE +/- 24482.38, N = 3SE +/- 15384.68, N = 3SE +/- 13756.93, N = 313514659.6713294800.5912696519.851. (CXX) g++ options: -O2 -levent_openssl -levent -lcrypto -lssl -lpthread -lz -lpcre

MariaDB

Test: oltp_read_only - Threads: 128

OpenBenchmarking.orgQueries Per Second, More Is BetterMariaDB 11.5Test: oltp_read_only - Threads: 128v6.11v6.12v6.13 7 Dec9K18K27K36K45KSE +/- 14.78, N = 3SE +/- 15.19, N = 3SE +/- 18.45, N = 34368843581429361. (CXX) g++ options: -pie -fPIC -fstack-protector -O3 -lnuma -lpcre2-8 -lcrypt -laio -lz -lm -lssl -lcrypto -lpthread -ldl

MariaDB

Test: oltp_read_write - Threads: 128

OpenBenchmarking.orgQueries Per Second, More Is BetterMariaDB 11.5Test: oltp_read_write - Threads: 128v6.11v6.12v6.13 7 Dec40K80K120K160K200KSE +/- 155.52, N = 3SE +/- 60.34, N = 3SE +/- 77.99, N = 31836931811921799091. (CXX) g++ options: -pie -fPIC -fstack-protector -O3 -lnuma -lpcre2-8 -lcrypt -laio -lz -lm -lssl -lcrypto -lpthread -ldl

MariaDB

Test: oltp_write_only - Threads: 128

OpenBenchmarking.orgQueries Per Second, More Is BetterMariaDB 11.5Test: oltp_write_only - Threads: 128v6.11v6.12v6.13 7 Dec90K180K270K360K450KSE +/- 286.92, N = 3SE +/- 474.96, N = 3SE +/- 154.44, N = 34129673994173869831. (CXX) g++ options: -pie -fPIC -fstack-protector -O3 -lnuma -lpcre2-8 -lcrypt -laio -lz -lm -lssl -lcrypto -lpthread -ldl

MariaDB

Test: oltp_update_index - Threads: 128

OpenBenchmarking.orgQueries Per Second, More Is BetterMariaDB 11.5Test: oltp_update_index - Threads: 128v6.11v6.12v6.13 7 Dec40K80K120K160K200KSE +/- 123.94, N = 3SE +/- 76.99, N = 3SE +/- 112.30, N = 31920981835371583041. (CXX) g++ options: -pie -fPIC -fstack-protector -O3 -lnuma -lpcre2-8 -lcrypt -laio -lz -lm -lssl -lcrypto -lpthread -ldl

PostgreSQL

Scaling Factor: 100 - Clients: 800 - Mode: Read Only

OpenBenchmarking.orgTPS, More Is BetterPostgreSQL 17Scaling Factor: 100 - Clients: 800 - Mode: Read Onlyv6.13 7 Decv6.12v6.111000K2000K3000K4000K5000KSE +/- 8981.60, N = 3SE +/- 9902.78, N = 3SE +/- 11484.93, N = 34829255481562547836481. (CC) gcc options: -fno-strict-aliasing -fwrapv -O2 -lpq -lpgcommon -lpgport -lm

PostgreSQL

Scaling Factor: 100 - Clients: 800 - Mode: Read Only - Average Latency

OpenBenchmarking.orgms, Fewer Is BetterPostgreSQL 17Scaling Factor: 100 - Clients: 800 - Mode: Read Only - Average Latencyv6.13 7 Decv6.12v6.110.03760.07520.11280.15040.188SE +/- 0.000, N = 3SE +/- 0.000, N = 3SE +/- 0.001, N = 30.1650.1660.1671. (CC) gcc options: -fno-strict-aliasing -fwrapv -O2 -lpq -lpgcommon -lpgport -lm

PostgreSQL

Scaling Factor: 100 - Clients: 1000 - Mode: Read Only

OpenBenchmarking.orgTPS, More Is BetterPostgreSQL 17Scaling Factor: 100 - Clients: 1000 - Mode: Read Onlyv6.13 7 Decv6.12v6.111000K2000K3000K4000K5000KSE +/- 9098.55, N = 3SE +/- 18553.35, N = 3SE +/- 23812.59, N = 34735578473275346213791. (CC) gcc options: -fno-strict-aliasing -fwrapv -O2 -lpq -lpgcommon -lpgport -lm

PostgreSQL

Scaling Factor: 100 - Clients: 1000 - Mode: Read Only - Average Latency

OpenBenchmarking.orgms, Fewer Is BetterPostgreSQL 17Scaling Factor: 100 - Clients: 1000 - Mode: Read Only - Average Latencyv6.12v6.13 7 Decv6.110.04880.09760.14640.19520.244SE +/- 0.001, N = 3SE +/- 0.001, N = 3SE +/- 0.001, N = 30.2110.2110.2171. (CC) gcc options: -fno-strict-aliasing -fwrapv -O2 -lpq -lpgcommon -lpgport -lm

PostgreSQL

Scaling Factor: 100 - Clients: 800 - Mode: Read Write

OpenBenchmarking.orgTPS, More Is BetterPostgreSQL 17Scaling Factor: 100 - Clients: 800 - Mode: Read Writev6.11v6.12v6.13 7 Dec30K60K90K120K150KSE +/- 594.42, N = 3SE +/- 1549.82, N = 3SE +/- 536.64, N = 31268771221811207031. (CC) gcc options: -fno-strict-aliasing -fwrapv -O2 -lpq -lpgcommon -lpgport -lm

PostgreSQL

Scaling Factor: 100 - Clients: 800 - Mode: Read Write - Average Latency

OpenBenchmarking.orgms, Fewer Is BetterPostgreSQL 17Scaling Factor: 100 - Clients: 800 - Mode: Read Write - Average Latencyv6.11v6.12v6.13 7 Dec246810SE +/- 0.030, N = 3SE +/- 0.082, N = 3SE +/- 0.030, N = 36.3056.5506.6281. (CC) gcc options: -fno-strict-aliasing -fwrapv -O2 -lpq -lpgcommon -lpgport -lm

PostgreSQL

Scaling Factor: 100 - Clients: 1000 - Mode: Read Write

OpenBenchmarking.orgTPS, More Is BetterPostgreSQL 17Scaling Factor: 100 - Clients: 1000 - Mode: Read Writev6.11v6.12v6.13 7 Dec30K60K90K120K150KSE +/- 274.82, N = 3SE +/- 604.26, N = 3SE +/- 515.38, N = 31176771120251101121. (CC) gcc options: -fno-strict-aliasing -fwrapv -O2 -lpq -lpgcommon -lpgport -lm

PostgreSQL

Scaling Factor: 100 - Clients: 1000 - Mode: Read Write - Average Latency

OpenBenchmarking.orgms, Fewer Is BetterPostgreSQL 17Scaling Factor: 100 - Clients: 1000 - Mode: Read Write - Average Latencyv6.11v6.12v6.13 7 Dec3691215SE +/- 0.020, N = 3SE +/- 0.048, N = 3SE +/- 0.042, N = 38.4988.9279.0821. (CC) gcc options: -fno-strict-aliasing -fwrapv -O2 -lpq -lpgcommon -lpgport -lm

Stress-NG

Test: NUMA

OpenBenchmarking.orgBogo Ops/s, More Is BetterStress-NG 0.17.08Test: NUMAv6.13 7 Decv6.12v6.115001000150020002500SE +/- 10.25, N = 3SE +/- 1.02, N = 3SE +/- 6.78, N = 32385.592298.652071.981. (CXX) g++ options: -O2 -std=gnu99 -lc -lm

Stress-NG

Test: Futex

OpenBenchmarking.orgBogo Ops/s, More Is BetterStress-NG 0.17.08Test: Futexv6.13 7 Decv6.12v6.11900K1800K2700K3600K4500KSE +/- 31680.82, N = 3SE +/- 15124.18, N = 3SE +/- 17548.84, N = 34398887.774313954.944154651.851. (CXX) g++ options: -O2 -std=gnu99 -lc -lm

Stress-NG

Test: MEMFD

OpenBenchmarking.orgBogo Ops/s, More Is BetterStress-NG 0.17.08Test: MEMFDv6.13 7 Decv6.11v6.125001000150020002500SE +/- 3.27, N = 3SE +/- 4.79, N = 3SE +/- 2.72, N = 32125.132054.702010.991. (CXX) g++ options: -O2 -std=gnu99 -lc -lm

Stress-NG

Test: Mutex

OpenBenchmarking.orgBogo Ops/s, More Is BetterStress-NG 0.17.08Test: Mutexv6.12v6.11v6.13 7 Dec10M20M30M40M50MSE +/- 442253.04, N = 3SE +/- 181454.22, N = 3SE +/- 188378.96, N = 345398166.1445376718.9443186401.841. (CXX) g++ options: -O2 -std=gnu99 -lc -lm

Stress-NG

Test: Semaphores

OpenBenchmarking.orgBogo Ops/s, More Is BetterStress-NG 0.17.08Test: Semaphoresv6.12v6.13 7 Decv6.1150M100M150M200M250MSE +/- 3074547.79, N = 3SE +/- 1084579.30, N = 3SE +/- 2150155.18, N = 15251549880.01246944948.46233159294.141. (CXX) g++ options: -O2 -std=gnu99 -lc -lm

Stress-NG

Test: Mixed Scheduler

OpenBenchmarking.orgBogo Ops/s, More Is BetterStress-NG 0.17.08Test: Mixed Schedulerv6.12v6.13 7 Decv6.1120K40K60K80K100KSE +/- 215.02, N = 3SE +/- 282.83, N = 3SE +/- 161.54, N = 382674.9881582.5879773.371. (CXX) g++ options: -O2 -std=gnu99 -lc -lm

Stress-NG

Test: Socket Activity

OpenBenchmarking.orgBogo Ops/s, More Is BetterStress-NG 0.17.08Test: Socket Activityv6.12v6.11v6.13 7 Dec10K20K30K40K50KSE +/- 68.22, N = 3SE +/- 29.82, N = 3SE +/- 42.06, N = 347501.0447424.8747238.701. (CXX) g++ options: -O2 -std=gnu99 -lc -lm

Stress-NG

Test: Context Switching

OpenBenchmarking.orgBogo Ops/s, More Is BetterStress-NG 0.17.08Test: Context Switchingv6.12v6.13 7 Decv6.1112M24M36M48M60MSE +/- 64836.17, N = 3SE +/- 159927.59, N = 3SE +/- 247154.69, N = 1355755486.3253038526.0036748330.541. (CXX) g++ options: -O2 -std=gnu99 -lc -lm

Blender

Blend File: BMW27 - Compute: CPU-Only

OpenBenchmarking.orgSeconds, Fewer Is BetterBlender 4.3Blend File: BMW27 - Compute: CPU-Onlyv6.13 7 Decv6.12v6.1148121620SE +/- 0.04, N = 3SE +/- 0.05, N = 3SE +/- 0.00, N = 315.0215.0815.19

Blender

Blend File: Junkshop - Compute: CPU-Only

OpenBenchmarking.orgSeconds, Fewer Is BetterBlender 4.3Blend File: Junkshop - Compute: CPU-Onlyv6.13 7 Decv6.12v6.11510152025SE +/- 0.06, N = 3SE +/- 0.06, N = 3SE +/- 0.04, N = 320.2020.2120.24

Blender

Blend File: Classroom - Compute: CPU-Only

OpenBenchmarking.orgSeconds, Fewer Is BetterBlender 4.3Blend File: Classroom - Compute: CPU-Onlyv6.13 7 Decv6.12v6.111020304050SE +/- 0.04, N = 3SE +/- 0.05, N = 3SE +/- 0.01, N = 341.4941.7442.03

Blender

Blend File: Fishy Cat - Compute: CPU-Only

OpenBenchmarking.orgSeconds, Fewer Is BetterBlender 4.3Blend File: Fishy Cat - Compute: CPU-Onlyv6.13 7 Decv6.12v6.11510152025SE +/- 0.06, N = 3SE +/- 0.06, N = 3SE +/- 0.12, N = 321.3521.4921.66

Blender

Blend File: Barbershop - Compute: CPU-Only

OpenBenchmarking.orgSeconds, Fewer Is BetterBlender 4.3Blend File: Barbershop - Compute: CPU-Onlyv6.13 7 Decv6.12v6.11306090120150SE +/- 0.23, N = 3SE +/- 0.18, N = 3SE +/- 0.08, N = 3148.16148.82149.30

Blender

Blend File: Pabellon Barcelona - Compute: CPU-Only

OpenBenchmarking.orgSeconds, Fewer Is BetterBlender 4.3Blend File: Pabellon Barcelona - Compute: CPU-Onlyv6.13 7 Decv6.12v6.111122334455SE +/- 0.02, N = 3SE +/- 0.08, N = 3SE +/- 0.05, N = 347.3947.7047.90

Apache Cassandra

Test: Writes

OpenBenchmarking.orgOp/s, More Is BetterApache Cassandra 5.0Test: Writesv6.12v6.13 7 Decv6.11110K220K330K440K550KSE +/- 1282.82, N = 3SE +/- 4648.88, N = 3SE +/- 5140.53, N = 3491186487743483952

Llamafile

Model: Llama-3.2-3B-Instruct.Q6_K - Test: Text Generation 16

OpenBenchmarking.orgTokens Per Second, More Is BetterLlamafile 0.8.16Model: Llama-3.2-3B-Instruct.Q6_K - Test: Text Generation 16v6.11v6.12v6.13 7 Dec20406080100SE +/- 2.48, N = 12SE +/- 2.55, N = 12SE +/- 2.52, N = 14108.22107.95102.21

Llamafile

Model: Llama-3.2-3B-Instruct.Q6_K - Test: Text Generation 128

OpenBenchmarking.orgTokens Per Second, More Is BetterLlamafile 0.8.16Model: Llama-3.2-3B-Instruct.Q6_K - Test: Text Generation 128v6.11v6.12v6.13 7 Dec306090120150SE +/- 0.87, N = 3SE +/- 1.04, N = 3SE +/- 1.53, N = 3112.56111.98109.60

Llamafile

Model: Llama-3.2-3B-Instruct.Q6_K - Test: Prompt Processing 256

OpenBenchmarking.orgTokens Per Second, More Is BetterLlamafile 0.8.16Model: Llama-3.2-3B-Instruct.Q6_K - Test: Prompt Processing 256v6.13 7 Decv6.12v6.119001800270036004500SE +/- 0.00, N = 3SE +/- 0.00, N = 3SE +/- 0.00, N = 3409640964096

Llamafile

Model: Llama-3.2-3B-Instruct.Q6_K - Test: Prompt Processing 512

OpenBenchmarking.orgTokens Per Second, More Is BetterLlamafile 0.8.16Model: Llama-3.2-3B-Instruct.Q6_K - Test: Prompt Processing 512v6.13 7 Decv6.12v6.112K4K6K8K10KSE +/- 0.00, N = 3SE +/- 0.00, N = 3SE +/- 0.00, N = 3819281928192

Llamafile

Model: TinyLlama-1.1B-Chat-v1.0.BF16 - Test: Text Generation 16

OpenBenchmarking.orgTokens Per Second, More Is BetterLlamafile 0.8.16Model: TinyLlama-1.1B-Chat-v1.0.BF16 - Test: Text Generation 16v6.12v6.11v6.13 7 Dec306090120150SE +/- 4.63, N = 12SE +/- 4.56, N = 12SE +/- 4.35, N = 12152.89152.07147.98

Llamafile

Model: Llama-3.2-3B-Instruct.Q6_K - Test: Prompt Processing 1024

OpenBenchmarking.orgTokens Per Second, More Is BetterLlamafile 0.8.16Model: Llama-3.2-3B-Instruct.Q6_K - Test: Prompt Processing 1024v6.13 7 Decv6.12v6.114K8K12K16K20KSE +/- 0.00, N = 3SE +/- 0.00, N = 3SE +/- 0.00, N = 3163841638416384

Llamafile

Model: Llama-3.2-3B-Instruct.Q6_K - Test: Prompt Processing 2048

OpenBenchmarking.orgTokens Per Second, More Is BetterLlamafile 0.8.16Model: Llama-3.2-3B-Instruct.Q6_K - Test: Prompt Processing 2048v6.13 7 Decv6.12v6.117K14K21K28K35KSE +/- 0.00, N = 3SE +/- 0.00, N = 3SE +/- 0.00, N = 3327683276832768

Llamafile

Model: TinyLlama-1.1B-Chat-v1.0.BF16 - Test: Text Generation 128

OpenBenchmarking.orgTokens Per Second, More Is BetterLlamafile 0.8.16Model: TinyLlama-1.1B-Chat-v1.0.BF16 - Test: Text Generation 128v6.11v6.12v6.13 7 Dec4080120160200SE +/- 1.11, N = 3SE +/- 1.26, N = 3SE +/- 1.45, N = 15159.90159.60156.82

Llamafile

Model: mistral-7b-instruct-v0.2.Q5_K_M - Test: Text Generation 16

OpenBenchmarking.orgTokens Per Second, More Is BetterLlamafile 0.8.16Model: mistral-7b-instruct-v0.2.Q5_K_M - Test: Text Generation 16v6.11v6.12v6.13 7 Dec1428425670SE +/- 1.73, N = 12SE +/- 1.75, N = 12SE +/- 1.70, N = 1264.8764.8064.15

Llamafile

Model: TinyLlama-1.1B-Chat-v1.0.BF16 - Test: Prompt Processing 256

OpenBenchmarking.orgTokens Per Second, More Is BetterLlamafile 0.8.16Model: TinyLlama-1.1B-Chat-v1.0.BF16 - Test: Prompt Processing 256v6.13 7 Decv6.12v6.119001800270036004500SE +/- 0.00, N = 3SE +/- 0.00, N = 3SE +/- 0.00, N = 3409640964096

Llamafile

Model: TinyLlama-1.1B-Chat-v1.0.BF16 - Test: Prompt Processing 512

OpenBenchmarking.orgTokens Per Second, More Is BetterLlamafile 0.8.16Model: TinyLlama-1.1B-Chat-v1.0.BF16 - Test: Prompt Processing 512v6.13 7 Decv6.12v6.112K4K6K8K10KSE +/- 0.00, N = 3SE +/- 0.00, N = 3SE +/- 0.00, N = 3819281928192

Llamafile

Model: mistral-7b-instruct-v0.2.Q5_K_M - Test: Text Generation 128

OpenBenchmarking.orgTokens Per Second, More Is BetterLlamafile 0.8.16Model: mistral-7b-instruct-v0.2.Q5_K_M - Test: Text Generation 128v6.11v6.12v6.13 7 Dec1530456075SE +/- 0.37, N = 3SE +/- 0.47, N = 3SE +/- 0.36, N = 368.1367.7165.45

Llamafile

Model: wizardcoder-python-34b-v1.0.Q6_K - Test: Text Generation 16

OpenBenchmarking.orgTokens Per Second, More Is BetterLlamafile 0.8.16Model: wizardcoder-python-34b-v1.0.Q6_K - Test: Text Generation 16v6.12v6.11v6.13 7 Dec48121620SE +/- 0.74, N = 12SE +/- 0.74, N = 12SE +/- 0.73, N = 1215.0114.9814.90

Llamafile

Model: TinyLlama-1.1B-Chat-v1.0.BF16 - Test: Prompt Processing 1024

OpenBenchmarking.orgTokens Per Second, More Is BetterLlamafile 0.8.16Model: TinyLlama-1.1B-Chat-v1.0.BF16 - Test: Prompt Processing 1024v6.13 7 Decv6.12v6.114K8K12K16K20KSE +/- 0.00, N = 3SE +/- 0.00, N = 3SE +/- 0.00, N = 3163841638416384

Llamafile

Model: TinyLlama-1.1B-Chat-v1.0.BF16 - Test: Prompt Processing 2048

OpenBenchmarking.orgTokens Per Second, More Is BetterLlamafile 0.8.16Model: TinyLlama-1.1B-Chat-v1.0.BF16 - Test: Prompt Processing 2048v6.13 7 Decv6.12v6.117K14K21K28K35KSE +/- 0.00, N = 3SE +/- 0.00, N = 3SE +/- 0.00, N = 3327683276832768

Llamafile

Model: wizardcoder-python-34b-v1.0.Q6_K - Test: Text Generation 128

OpenBenchmarking.orgTokens Per Second, More Is BetterLlamafile 0.8.16Model: wizardcoder-python-34b-v1.0.Q6_K - Test: Text Generation 128v6.11v6.12v6.13 7 Dec48121620SE +/- 0.02, N = 3SE +/- 0.04, N = 3SE +/- 0.02, N = 315.9015.8215.77

Llamafile

Model: mistral-7b-instruct-v0.2.Q5_K_M - Test: Prompt Processing 256

OpenBenchmarking.orgTokens Per Second, More Is BetterLlamafile 0.8.16Model: mistral-7b-instruct-v0.2.Q5_K_M - Test: Prompt Processing 256v6.13 7 Decv6.12v6.119001800270036004500SE +/- 0.00, N = 3SE +/- 0.00, N = 3SE +/- 0.00, N = 3409640964096

Llamafile

Model: mistral-7b-instruct-v0.2.Q5_K_M - Test: Prompt Processing 512

OpenBenchmarking.orgTokens Per Second, More Is BetterLlamafile 0.8.16Model: mistral-7b-instruct-v0.2.Q5_K_M - Test: Prompt Processing 512v6.13 7 Decv6.12v6.112K4K6K8K10KSE +/- 0.00, N = 3SE +/- 0.00, N = 3SE +/- 0.00, N = 3819281928192

Llamafile

Model: mistral-7b-instruct-v0.2.Q5_K_M - Test: Prompt Processing 1024

OpenBenchmarking.orgTokens Per Second, More Is BetterLlamafile 0.8.16Model: mistral-7b-instruct-v0.2.Q5_K_M - Test: Prompt Processing 1024v6.13 7 Decv6.12v6.114K8K12K16K20KSE +/- 0.00, N = 3SE +/- 0.00, N = 3SE +/- 0.00, N = 3163841638416384

Llamafile

Model: mistral-7b-instruct-v0.2.Q5_K_M - Test: Prompt Processing 2048

OpenBenchmarking.orgTokens Per Second, More Is BetterLlamafile 0.8.16Model: mistral-7b-instruct-v0.2.Q5_K_M - Test: Prompt Processing 2048v6.13 7 Decv6.12v6.117K14K21K28K35KSE +/- 0.00, N = 3SE +/- 0.00, N = 3SE +/- 0.00, N = 3327683276832768

Llamafile

Model: wizardcoder-python-34b-v1.0.Q6_K - Test: Prompt Processing 256

OpenBenchmarking.orgTokens Per Second, More Is BetterLlamafile 0.8.16Model: wizardcoder-python-34b-v1.0.Q6_K - Test: Prompt Processing 256v6.13 7 Decv6.12v6.1130060090012001500SE +/- 0.00, N = 3SE +/- 0.00, N = 3SE +/- 0.00, N = 3153615361536

Llamafile

Model: wizardcoder-python-34b-v1.0.Q6_K - Test: Prompt Processing 512

OpenBenchmarking.orgTokens Per Second, More Is BetterLlamafile 0.8.16Model: wizardcoder-python-34b-v1.0.Q6_K - Test: Prompt Processing 512v6.13 7 Decv6.12v6.117001400210028003500SE +/- 0.00, N = 3SE +/- 0.00, N = 3SE +/- 0.00, N = 3307230723072

Llamafile

Model: wizardcoder-python-34b-v1.0.Q6_K - Test: Prompt Processing 1024

OpenBenchmarking.orgTokens Per Second, More Is BetterLlamafile 0.8.16Model: wizardcoder-python-34b-v1.0.Q6_K - Test: Prompt Processing 1024v6.13 7 Decv6.12v6.1113002600390052006500SE +/- 0.00, N = 3SE +/- 0.00, N = 3SE +/- 0.00, N = 3614461446144

Llamafile

Model: wizardcoder-python-34b-v1.0.Q6_K - Test: Prompt Processing 2048

OpenBenchmarking.orgTokens Per Second, More Is BetterLlamafile 0.8.16Model: wizardcoder-python-34b-v1.0.Q6_K - Test: Prompt Processing 2048v6.13 7 Decv6.12v6.113K6K9K12K15KSE +/- 0.00, N = 3SE +/- 0.00, N = 3SE +/- 0.00, N = 3122881228812288

OpenVINO GenAI

Model: Gemma-7b-int4-ov - Device: CPU

OpenBenchmarking.orgtokens/s, More Is BetterOpenVINO GenAI 2024.5Model: Gemma-7b-int4-ov - Device: CPUv6.11v6.12v6.13 7 Dec1122334455SE +/- 0.15, N = 3SE +/- 0.10, N = 3SE +/- 0.27, N = 349.1349.0948.70

OpenVINO GenAI

Model: Gemma-7b-int4-ov - Device: CPU - Time To First Token

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO GenAI 2024.5Model: Gemma-7b-int4-ov - Device: CPU - Time To First Tokenv6.12v6.11v6.13 7 Dec714212835SE +/- 0.04, N = 3SE +/- 0.08, N = 3SE +/- 0.18, N = 330.2730.3930.48

OpenVINO GenAI

Model: Gemma-7b-int4-ov - Device: CPU - Time Per Output Token

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO GenAI 2024.5Model: Gemma-7b-int4-ov - Device: CPU - Time Per Output Tokenv6.11v6.12v6.13 7 Dec510152025SE +/- 0.06, N = 3SE +/- 0.04, N = 3SE +/- 0.11, N = 320.3620.3720.54

OpenVINO GenAI

Model: TinyLlama-1.1B-Chat-v1.0 - Device: CPU

OpenBenchmarking.orgtokens/s, More Is BetterOpenVINO GenAI 2024.5Model: TinyLlama-1.1B-Chat-v1.0 - Device: CPUv6.11v6.12v6.13 7 Dec20406080100SE +/- 0.57, N = 3SE +/- 0.23, N = 3SE +/- 0.28, N = 378.3678.1877.67

OpenVINO GenAI

Model: TinyLlama-1.1B-Chat-v1.0 - Device: CPU - Time To First Token

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO GenAI 2024.5Model: TinyLlama-1.1B-Chat-v1.0 - Device: CPU - Time To First Tokenv6.11v6.12v6.13 7 Dec48121620SE +/- 0.19, N = 3SE +/- 0.05, N = 3SE +/- 0.08, N = 315.8615.8916.05

OpenVINO GenAI

Model: TinyLlama-1.1B-Chat-v1.0 - Device: CPU - Time Per Output Token

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO GenAI 2024.5Model: TinyLlama-1.1B-Chat-v1.0 - Device: CPU - Time Per Output Tokenv6.11v6.12v6.13 7 Dec3691215SE +/- 0.09, N = 3SE +/- 0.04, N = 3SE +/- 0.04, N = 312.7612.7912.88

OpenVINO GenAI

Model: Falcon-7b-instruct-int4-ov - Device: CPU

OpenBenchmarking.orgtokens/s, More Is BetterOpenVINO GenAI 2024.5Model: Falcon-7b-instruct-int4-ov - Device: CPUv6.11v6.13 7 Decv6.121326395265SE +/- 0.63, N = 3SE +/- 0.54, N = 3SE +/- 0.55, N = 656.7356.2856.21

OpenVINO GenAI

Model: Falcon-7b-instruct-int4-ov - Device: CPU - Time To First Token

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO GenAI 2024.5Model: Falcon-7b-instruct-int4-ov - Device: CPU - Time To First Tokenv6.12v6.13 7 Decv6.11714212835SE +/- 0.26, N = 6SE +/- 0.44, N = 3SE +/- 0.27, N = 330.0030.1830.39

OpenVINO GenAI

Model: Falcon-7b-instruct-int4-ov - Device: CPU - Time Per Output Token

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO GenAI 2024.5Model: Falcon-7b-instruct-int4-ov - Device: CPU - Time Per Output Tokenv6.11v6.13 7 Decv6.1248121620SE +/- 0.19, N = 3SE +/- 0.17, N = 3SE +/- 0.17, N = 617.6317.7717.80

OpenVINO GenAI

Model: Phi-3-mini-128k-instruct-int4-ov - Device: CPU

OpenBenchmarking.orgtokens/s, More Is BetterOpenVINO GenAI 2024.5Model: Phi-3-mini-128k-instruct-int4-ov - Device: CPUv6.13 7 Decv6.11v6.121530456075SE +/- 0.79, N = 3SE +/- 0.41, N = 3SE +/- 0.10, N = 365.9165.8965.82

OpenVINO GenAI

Model: Phi-3-mini-128k-instruct-int4-ov - Device: CPU - Time To First Token

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO GenAI 2024.5Model: Phi-3-mini-128k-instruct-int4-ov - Device: CPU - Time To First Tokenv6.13 7 Decv6.11v6.12510152025SE +/- 0.43, N = 3SE +/- 0.24, N = 3SE +/- 0.37, N = 322.3822.6622.73

OpenVINO GenAI

Model: Phi-3-mini-128k-instruct-int4-ov - Device: CPU - Time Per Output Token

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO GenAI 2024.5Model: Phi-3-mini-128k-instruct-int4-ov - Device: CPU - Time Per Output Tokenv6.13 7 Decv6.11v6.1248121620SE +/- 0.18, N = 3SE +/- 0.10, N = 3SE +/- 0.02, N = 315.1715.1815.19

Phoronix Test Suite v10.8.5