eoy2024

Tests for a future article. AMD EPYC 4564P 16-Core testing with a Supermicro AS-3015A-I H13SAE-MF v1.00 (2.1 BIOS) and ASPEED on Ubuntu 24.04 via the Phoronix Test Suite.

Compare your own system(s) to this result file with the Phoronix Test Suite by running the command: phoronix-test-suite benchmark 2412068-NE-EOY20244373
Jump To Table - Results

View

Do Not Show Noisy Results
Do Not Show Results With Incomplete Data
Do Not Show Results With Little Change/Spread
List Notable Results
Show Result Confidence Charts
Allow Limiting Results To Certain Suite(s)

Statistics

Show Overall Harmonic Mean(s)
Show Overall Geometric Mean
Show Wins / Losses Counts (Pie Chart)
Normalize Results
Remove Outliers Before Calculating Averages

Graph Settings

Force Line Graphs Where Applicable
Convert To Scalar Where Applicable
Prefer Vertical Bar Graphs

Multi-Way Comparison

Condense Multi-Option Tests Into Single Result Graphs

Table

Show Detailed System Result Table

Run Management

Highlight
Result
Toggle/Hide
Result
Result
Identifier
Performance Per
Dollar
Date
Run
  Test
  Duration
a
December 05
  6 Hours, 48 Minutes
b
December 06
  6 Hours, 49 Minutes
c
December 06
  2 Hours, 24 Minutes
Invert Behavior (Only Show Selected Data)
  5 Hours, 20 Minutes

Only show results where is faster than
Only show results matching title/arguments (delimit multiple options with a comma):
Do not show results matching title/arguments (delimit multiple options with a comma):


eoy2024OpenBenchmarking.orgPhoronix Test SuiteAMD EPYC 4564P 16-Core @ 5.88GHz (16 Cores / 32 Threads)Supermicro AS-3015A-I H13SAE-MF v1.00 (2.1 BIOS)AMD Device 14d82 x 32GB DRAM-4800MT/s Micron MTC20C2085S1EC48BA1 BC3201GB Micron_7450_MTFDKCC3T2TFS + 960GB SAMSUNG MZ1L2960HCJR-00A07ASPEEDAMD Rembrandt Radeon HD AudioVA24312 x Intel I210Ubuntu 24.046.8.0-11-generic (x86_64)GNOME Shell 45.3X Server 1.21.1.11GCC 13.2.0ext41024x768ProcessorMotherboardChipsetMemoryDiskGraphicsAudioMonitorNetworkOSKernelDesktopDisplay ServerCompilerFile-SystemScreen ResolutionEoy2024 BenchmarksSystem Logs- Transparent Huge Pages: madvise- --build=x86_64-linux-gnu --disable-vtable-verify --disable-werror --enable-cet --enable-checking=release --enable-clocale=gnu --enable-default-pie --enable-gnu-unique-object --enable-languages=c,ada,c++,go,d,fortran,objc,obj-c++,m2 --enable-libphobos-checking=release --enable-libstdcxx-backtrace --enable-libstdcxx-debug --enable-libstdcxx-time=yes --enable-multiarch --enable-multilib --enable-nls --enable-objc-gc=auto --enable-offload-defaulted --enable-offload-targets=nvptx-none=/build/gcc-13-fxIygj/gcc-13-13.2.0/debian/tmp-nvptx/usr,amdgcn-amdhsa=/build/gcc-13-fxIygj/gcc-13-13.2.0/debian/tmp-gcn/usr --enable-plugin --enable-shared --enable-threads=posix --host=x86_64-linux-gnu --program-prefix=x86_64-linux-gnu- --target=x86_64-linux-gnu --with-abi=m64 --with-arch-32=i686 --with-default-libstdcxx-abi=new --with-gcc-major-version-only --with-multilib-list=m32,m64,mx32 --with-target-system-zlib=auto --with-tune=generic --without-cuda-driver -v - Scaling Governor: amd-pstate-epp performance (EPP: performance) - CPU Microcode: 0xa601209- OpenJDK Runtime Environment (build 21.0.2+13-Ubuntu-2)- Python 3.12.3- gather_data_sampling: Not affected + itlb_multihit: Not affected + l1tf: Not affected + mds: Not affected + meltdown: Not affected + mmio_stale_data: Not affected + retbleed: Not affected + spec_rstack_overflow: Mitigation of Safe RET + spec_store_bypass: Mitigation of SSB disabled via prctl + spectre_v1: Mitigation of usercopy/swapgs barriers and __user pointer sanitization + spectre_v2: Mitigation of Enhanced / Automatic IBRS IBPB: conditional STIBP: always-on RSB filling PBRSB-eIBRS: Not affected + srbds: Not affected + tsx_async_abort: Not affected

abcResult OverviewPhoronix Test Suite100%103%105%108%110%StockfishRELIONCP2K Molecular DynamicsRenaissancex265simdjsonACES DGEMMSVT-AV1NAMDEtcpakOSPRay7-Zip CompressionQuantLibBYTE Unix Benchmark

eoy2024litert: Quantized COCO SSD MobileNet v1litert: NASNet Mobilelitert: DeepLab V3litert: Mobilenet Quantcp2k: Fayalite-FISTllama-cpp: CPU BLAS - Mistral-7B-Instruct-v0.3-Q8_0 - Prompt Processing 512stockfish: Chess Benchmarkrelion: Basic - CPUlitert: Inception V4llama-cpp: CPU BLAS - granite-3.0-3b-a800m-instruct-Q8_0 - Prompt Processing 1024renaissance: Apache Spark Bayesllama-cpp: CPU BLAS - Llama-3.1-Tulu-3-8B-Q8_0 - Prompt Processing 512llama-cpp: CPU BLAS - Llama-3.1-Tulu-3-8B-Q8_0 - Prompt Processing 1024litert: Mobilenet Floatrenaissance: In-Memory Database Shootoutrenaissance: Scala Dottycp2k: H20-256renaissance: Rand Forestllama-cpp: CPU BLAS - Llama-3.1-Tulu-3-8B-Q8_0 - Prompt Processing 2048rustls: handshake - TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384simdjson: LargeRandgcrypt: xnnpack: FP16MobileNetV2onednn: Deconvolution Batch shapes_1d - CPUlitert: Inception ResNet V2xnnpack: FP32MobileNetV2simdjson: Kostyapyperformance: asyncio_tcp_sslllamafile: Llama-3.2-3B-Instruct.Q6_K - Text Generation 16xnnpack: FP32MobileNetV3Largelitert: SqueezeNetllama-cpp: CPU BLAS - Mistral-7B-Instruct-v0.3-Q8_0 - Prompt Processing 2048renaissance: Genetic Algorithm Using Jenetics + Futuresxnnpack: FP16MobileNetV3Largesimdjson: TopTweetllama-cpp: CPU BLAS - granite-3.0-3b-a800m-instruct-Q8_0 - Text Generation 128xnnpack: FP32MobileNetV1renaissance: Gaussian Mixture Modelxnnpack: FP16MobileNetV1xnnpack: FP32MobileNetV3Smallrenaissance: Savina Reactors.IOrenaissance: Akka Unbalanced Cobwebbed Treesvt-av1: Preset 8 - Bosphorus 1080psvt-av1: Preset 8 - Bosphorus 4Konednn: IP Shapes 3D - CPUrenaissance: Finagle HTTP Requestsonednn: IP Shapes 1D - CPUllama-cpp: CPU BLAS - granite-3.0-3b-a800m-instruct-Q8_0 - Prompt Processing 2048cp2k: H20-64onednn: Convolution Batch Shapes Auto - CPUx265: Bosphorus 4Ksvt-av1: Preset 13 - Bosphorus 1080psvt-av1: Preset 5 - Beauty 4K 10-bitbuild-eigen: Time To Compileonednn: Deconvolution Batch shapes_3d - CPUrustls: handshake-resume - TLS13_CHACHA20_POLY1305_SHA256whisper-cpp: ggml-small.en - 2016 State of the Unionrustls: handshake-ticket - TLS13_CHACHA20_POLY1305_SHA256onnx: bertsquad-12 - CPU - Standardllamafile: TinyLlama-1.1B-Chat-v1.0.BF16 - Text Generation 128rustls: handshake - TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256rustls: handshake-resume - TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256stockfish: Chess Benchmarkpovray: Trace Timellamafile: mistral-7b-instruct-v0.2.Q5_K_M - Text Generation 128llamafile: Llama-3.2-3B-Instruct.Q6_K - Text Generation 128renaissance: ALS Movie Lenssvt-av1: Preset 13 - Bosphorus 4Konednn: Recurrent Neural Network Inference - CPUonnx: ZFNet-512 - CPU - Standardx265: Bosphorus 1080ponnx: fcn-resnet101-11 - CPU - Standardsimdjson: PartialTweetswhisperfile: Smallonnx: Faster R-CNN R-50-FPN-int8 - CPU - Standardnamd: ATPase with 327,506 Atomsonnx: ResNet101_DUC_HDC-12 - CPU - Standardcouchdb: 100 - 3000 - 30numpy: pyperformance: chaosmt-dgemm: Sustained Floating-Point Ratesvt-av1: Preset 5 - Bosphorus 1080ppyperformance: floatxnnpack: FP16MobileNetV3Smallrustls: handshake-ticket - TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256xnnpack: QS8MobileNetV2whisperfile: Tinyonnx: ArcFace ResNet-100 - CPU - Standardrenaissance: Apache Spark PageRanksvt-av1: Preset 8 - Beauty 4K 10-bitpyperformance: raytracerustls: handshake-ticket - TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384financebench: Bonds OpenMPonnx: CaffeNet 12-int8 - CPU - Standardcouchdb: 500 - 3000 - 30onnx: ResNet50 v1-12-int8 - CPU - Standardpyperformance: gosvt-av1: Preset 3 - Bosphorus 4Kcouchdb: 300 - 1000 - 30llama-cpp: CPU BLAS - granite-3.0-3b-a800m-instruct-Q8_0 - Prompt Processing 512blender: Junkshop - CPU-Onlysvt-av1: Preset 5 - Bosphorus 4Kospray: gravity_spheres_volume/dim_512/ao/real_timeonnx: yolov4 - CPU - Standardcompress-7zip: Decompression Ratingonednn: Recurrent Neural Network Training - CPUopenvino-genai: Gemma-7b-int4-ov - CPUsvt-av1: Preset 13 - Beauty 4K 10-bitospray: gravity_spheres_volume/dim_512/scivis/real_timesvt-av1: Preset 3 - Beauty 4K 10-bitgromacs: water_GMX50_baresimdjson: DistinctUserIDblender: Classroom - CPU-Onlypyperformance: pathlibllama-cpp: CPU BLAS - Mistral-7B-Instruct-v0.3-Q8_0 - Text Generation 128astcenc: Exhaustiveastcenc: Thoroughetcpak: Multi-Threaded - ETC2pyperformance: nbodysvt-av1: Preset 3 - Bosphorus 1080pllama-cpp: CPU BLAS - Mistral-7B-Instruct-v0.3-Q8_0 - Prompt Processing 1024couchdb: 500 - 1000 - 30astcenc: Mediumblender: Barbershop - CPU-Onlypyperformance: pickle_pure_pythonastcenc: Very Thoroughpyperformance: gc_collectospray: particle_volume/scivis/real_timeprimesieve: 1e13pyperformance: regex_compilellamafile: wizardcoder-python-34b-v1.0.Q6_K - Text Generation 16ospray: particle_volume/pathtracer/real_timepyperformance: async_tree_iollamafile: wizardcoder-python-34b-v1.0.Q6_K - Text Generation 128blender: Fishy Cat - CPU-Onlyprimesieve: 1e12rustls: handshake - TLS13_CHACHA20_POLY1305_SHA256financebench: Repo OpenMPpyperformance: django_templateospray: particle_volume/ao/real_timebyte: Dhrystone 2quantlib: XXSy-cruncher: 1Bllama-cpp: CPU BLAS - Llama-3.1-Tulu-3-8B-Q8_0 - Text Generation 128openvino-genai: Phi-3-mini-128k-instruct-int4-ov - CPUllamafile: TinyLlama-1.1B-Chat-v1.0.BF16 - Text Generation 16whisperfile: Mediumbyte: Pipellamafile: mistral-7b-instruct-v0.2.Q5_K_M - Text Generation 16blender: BMW27 - CPU-Onlyopenssl: AES-128-GCMopenssl: AES-256-GCMpyperformance: python_startupospray: gravity_spheres_volume/dim_512/pathtracer/real_timewhisper-cpp: ggml-medium.en - 2016 State of the Unionpyperformance: asyncio_websocketsopenvino-genai: Falcon-7b-instruct-int4-ov - CPUquantlib: Spyperformance: xml_etreecompress-7zip: Compression Ratingcouchdb: 100 - 1000 - 30build2: Time To Compilebyte: System Cally-cruncher: 500Mwhisper-cpp: ggml-base.en - 2016 State of the Uniononnx: T5 Encoder - CPU - Standardpyperformance: crypto_pyaesnamd: STMV with 1,066,628 Atomscouchdb: 300 - 3000 - 30onnx: GPT-2 - CPU - Standardopenssl: ChaCha20-Poly1305openssl: ChaCha20byte: Whetstone Doubleonnx: super-resolution-10 - CPU - Standardblender: Pabellon Barcelona - CPU-Onlyastcenc: Fastrustls: handshake-resume - TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384cassandra: Writesllamafile: wizardcoder-python-34b-v1.0.Q6_K - Prompt Processing 2048llamafile: wizardcoder-python-34b-v1.0.Q6_K - Prompt Processing 1024llamafile: wizardcoder-python-34b-v1.0.Q6_K - Prompt Processing 512llamafile: wizardcoder-python-34b-v1.0.Q6_K - Prompt Processing 256llamafile: mistral-7b-instruct-v0.2.Q5_K_M - Prompt Processing 2048llamafile: mistral-7b-instruct-v0.2.Q5_K_M - Prompt Processing 1024llamafile: mistral-7b-instruct-v0.2.Q5_K_M - Prompt Processing 512llamafile: mistral-7b-instruct-v0.2.Q5_K_M - Prompt Processing 256llamafile: TinyLlama-1.1B-Chat-v1.0.BF16 - Prompt Processing 2048llamafile: TinyLlama-1.1B-Chat-v1.0.BF16 - Prompt Processing 1024llamafile: TinyLlama-1.1B-Chat-v1.0.BF16 - Prompt Processing 512llamafile: TinyLlama-1.1B-Chat-v1.0.BF16 - Prompt Processing 256llamafile: Llama-3.2-3B-Instruct.Q6_K - Prompt Processing 2048llamafile: Llama-3.2-3B-Instruct.Q6_K - Prompt Processing 1024llamafile: Llama-3.2-3B-Instruct.Q6_K - Prompt Processing 512llamafile: Llama-3.2-3B-Instruct.Q6_K - Prompt Processing 256pyperformance: json_loadsopenvino-genai: Phi-3-mini-128k-instruct-int4-ov - CPU - Time Per Output Tokenopenvino-genai: Phi-3-mini-128k-instruct-int4-ov - CPU - Time To First Tokenopenvino-genai: Falcon-7b-instruct-int4-ov - CPU - Time Per Output Tokenopenvino-genai: Falcon-7b-instruct-int4-ov - CPU - Time To First Tokenopenvino-genai: Gemma-7b-int4-ov - CPU - Time Per Output Tokenopenvino-genai: Gemma-7b-int4-ov - CPU - Time To First Tokenonnx: Faster R-CNN R-50-FPN-int8 - CPU - Standardonnx: ResNet101_DUC_HDC-12 - CPU - Standardonnx: super-resolution-10 - CPU - Standardonnx: ResNet50 v1-12-int8 - CPU - Standardonnx: ArcFace ResNet-100 - CPU - Standardonnx: fcn-resnet101-11 - CPU - Standardonnx: CaffeNet 12-int8 - CPU - Standardonnx: bertsquad-12 - CPU - Standardonnx: T5 Encoder - CPU - Standardonnx: ZFNet-512 - CPU - Standardonnx: yolov4 - CPU - Standardonnx: GPT-2 - CPU - Standardrenaissance: Apache Spark ALSabc2129.52169363579.67823.1794.03268.454752796944.2721477.8355.09490.070.7670.851211.483256.1477.0592.857414.463.09423535.681.83162.12511902.9761219530.214955.9764519.0318101794.1162.97732.8149810.4647.7212523399.511439793506.44403.8339.023102.0054.0582319.41.12573279.0458.1916.6728732.57842.5586.50458.6552.41294388077.69245.07838404263.4515.589926.2880462.63563852.574650703818.54210.4720.139805.7212.52700.859102.331114.453.21679.76195.4164247.06912.796321.54196232.188775.7538.21141.194104101.97150.7920262033284441.7093542.45372412.212.4681751553632.1433061.21875636.318511.775390.59777.89.59106.13327.373.5634.5387.6394411.05521659161372.039.8318.5887.587891.4221.69210.46143.3614.27.241.684420.3025577.8175929.57369.26148.049156.2217506.21652.7416778.9848678.49869.81.78236.2457551.9971.356.34776454.4521418.44531220.79.009171866536062.713.43218.4856.8819.2824.59534.91948806257.110.2253.55104784522170971727517005.778.82093700.9131512.9312.747635.816385969.92992.05349140426.68.77287.48973156.45341.70.75656367.83134.59692393529340130588495050343491.9141.117166.12396.64951820810.212713331228861443072153632768163848192409632768163848192409632768163848192409612.151.8655.9377.3486.06101.72106.6221.2429648.5227.086012.5589823.553310.8751.5708464.1416.391129.7698590.45237.427762958.4821468.74287.06933.176102.41861.6259130265867.31523265.4328.47529.575.96661295.513081.5447.0629.557398.159.84402625.061.81154.5312473.112620375.715595.9367218.3218771860.3565.27744.3154910.846.2812903494.8117410053594.34439.9330.8799.5544.156822264.71.15274285.7157.3476.8175432.04824.8086.37159.8732.46279380493.86240.59909397022.415.318225.8379085.83504511.314575174718.84610.6419.819958.3209.773711.433103.862112.853.17059.82192.6780847.72932.790251.52109235.345765.3538.71137.394602100.89350.19312589637.9285442.2004941.96392439.912.6111771536355.933432.636719629.655517.149386.577779.554107.182324.2174.2634.2257.5740810.96031668431383.649.9118.4427.528751.4151.67910.38144.4114.37.191.672820.1644575.0259.429.37568.8149.028155.2665509.31662.72486818.9324578.94970.21.79235.325759271.76.37876083.7321522.06640620.88.966321857795366.113.429218.4036.8519.224.69532.8074448718087.110.2653.75104404347840968217370605.798.79096703.2218831612.9712.709835.716405070.11492.29249062324.18.79487.27256156.83341.80.75634368.664134.31192216350580130359884190343113141.003166.25396.42611821261.882713731228861443072153632768163848192409632768163848192409632768163848192409612.152.156.2677.1384.39100.94107.0320.9488657.427.091722.5858923.8276315.4051.5874665.27876.375669.625991.23597.4434105.22153623108939.897500.33046.8458.5624.571420.81.745.73719.110.793472.43567.84331.7338.653100.8932296.658.64732.73838.1686.3749907.4212.945114.529.682.829251127.270287102.1092439.212.5979.49534.4487.6428216732118.5567.557911.41110.43573.91429.4658.97005234.9718.985861862548305.413.49148613927.98.8119912.724216431349016743.60.75813343187OpenBenchmarking.org

LiteRT

OpenBenchmarking.orgMicroseconds, Fewer Is BetterLiteRT 2024-10-15Model: Quantized COCO SSD MobileNet v1ab60012001800240030002129.522958.48

OpenBenchmarking.orgMicroseconds, Fewer Is BetterLiteRT 2024-10-15Model: NASNet Mobileab5K10K15K20K25K16936.021468.7

OpenBenchmarking.orgMicroseconds, Fewer Is BetterLiteRT 2024-10-15Model: DeepLab V3ab90018002700360045003579.674287.06

OpenBenchmarking.orgMicroseconds, Fewer Is BetterLiteRT 2024-10-15Model: Mobilenet Quantab2004006008001000823.17933.18

CP2K Molecular Dynamics

OpenBenchmarking.orgSeconds, Fewer Is BetterCP2K Molecular Dynamics 2024.3Input: Fayalite-FISTabc2040608010094.03102.42105.221. (F9X) gfortran options: -fopenmp -march=native -mtune=native -O3 -funroll-loops -fbacktrace -ffree-form -fimplicit-none -std=f2008 -lcp2kstart -lcp2kmc -lcp2kswarm -lcp2kmotion -lcp2kthermostat -lcp2kemd -lcp2ktmc -lcp2kmain -lcp2kdbt -lcp2ktas -lcp2kgrid -lcp2kgriddgemm -lcp2kgridcpu -lcp2kgridref -lcp2kgridcommon -ldbcsrarnoldi -ldbcsrx -lcp2kdbx -lcp2kdbm -lcp2kshg_int -lcp2keri_mme -lcp2kminimax -lcp2khfxbase -lcp2ksubsys -lcp2kxc -lcp2kao -lcp2kpw_env -lcp2kinput -lcp2kpw -lcp2kgpu -lcp2kfft -lcp2kfpga -lcp2kfm -lcp2kcommon -lcp2koffload -lcp2kmpiwrap -lcp2kbase -ldbcsr -lsirius -lspla -lspfft -lsymspg -lvdwxc -l:libhdf5_fortran.a -l:libhdf5.a -lz -lgsl -lelpa_openmp -lcosma -lcosta -lscalapack -lxsmmf -lxsmm -ldl -lpthread -llibgrpp -lxcf03 -lxc -lint2 -lfftw3_mpi -lfftw3 -lfftw3_omp -lmpi_cxx -lmpi -l:libopenblas.a -lvori -lstdc++ -lmpi_usempif08 -lmpi_mpifh -lopen-rte -lopen-pal -lhwloc -levent_core -levent_pthreads -lm

Llama.cpp

OpenBenchmarking.orgTokens Per Second, More Is BetterLlama.cpp b4154Backend: CPU BLAS - Model: Mistral-7B-Instruct-v0.3-Q8_0 - Test: Prompt Processing 512ab153045607568.4061.621. (CXX) g++ options: -std=c++11 -fPIC -O3 -pthread -fopenmp -march=native -mtune=native -lopenblas

Stockfish

OpenBenchmarking.orgNodes Per Second, More Is BetterStockfish 17Chess Benchmarkbac13M26M39M52M65M5913026554752796536231081. (CXX) g++ options: -lgcov -m64 -lpthread -fno-exceptions -std=c++17 -fno-peel-loops -fno-tracer -pedantic -O3 -funroll-loops -msse -msse3 -mpopcnt -mavx2 -mbmi -mavx512f -mavx512bw -mavx512vnni -mavx512dq -mavx512vl -msse4.1 -mssse3 -msse2 -mbmi2 -flto -flto-partition=one -flto=jobserver

RELION

OpenBenchmarking.orgSeconds, Fewer Is BetterRELION 5.0Test: Basic - Device: CPUbca2004006008001000867.32939.90944.271. (CXX) g++ options: -fPIC -std=c++14 -fopenmp -O3 -rdynamic -lfftw3f -lfftw3 -ldl -ltiff -lpng -ljpeg -lmpi_cxx -lmpi

LiteRT

OpenBenchmarking.orgMicroseconds, Fewer Is BetterLiteRT 2024-10-15Model: Inception V4ab5K10K15K20K25K21477.823265.4

Llama.cpp

OpenBenchmarking.orgTokens Per Second, More Is BetterLlama.cpp b4154Backend: CPU BLAS - Model: granite-3.0-3b-a800m-instruct-Q8_0 - Test: Prompt Processing 1024ab80160240320400355.09328.471. (CXX) g++ options: -std=c++11 -fPIC -O3 -pthread -fopenmp -march=native -mtune=native -lopenblas

Renaissance

OpenBenchmarking.orgms, Fewer Is BetterRenaissance 0.16Test: Apache Spark Bayesacb110220330440550490.0500.3529.5MIN: 459.29 / MAX: 580.9MIN: 460.66 / MAX: 542.36MIN: 458.39 / MAX: 562.09

Llama.cpp

OpenBenchmarking.orgTokens Per Second, More Is BetterLlama.cpp b4154Backend: CPU BLAS - Model: Llama-3.1-Tulu-3-8B-Q8_0 - Test: Prompt Processing 512ba2040608010075.9670.761. (CXX) g++ options: -std=c++11 -fPIC -O3 -pthread -fopenmp -march=native -mtune=native -lopenblas

OpenBenchmarking.orgTokens Per Second, More Is BetterLlama.cpp b4154Backend: CPU BLAS - Model: Llama-3.1-Tulu-3-8B-Q8_0 - Test: Prompt Processing 1024ab163248648070.8566.001. (CXX) g++ options: -std=c++11 -fPIC -O3 -pthread -fopenmp -march=native -mtune=native -lopenblas

LiteRT

OpenBenchmarking.orgMicroseconds, Fewer Is BetterLiteRT 2024-10-15Model: Mobilenet Floatab300600900120015001211.481295.51

Renaissance

OpenBenchmarking.orgms, Fewer Is BetterRenaissance 0.16Test: In-Memory Database Shootoutcba70014002100280035003046.83081.53256.1MIN: 2814.66 / MAX: 3304.16MIN: 2836.52 / MAX: 3397.02MIN: 3019.89 / MAX: 3599.5

OpenBenchmarking.orgms, Fewer Is BetterRenaissance 0.16Test: Scala Dottybca100200300400500447.0458.5477.0MIN: 402.95 / MAX: 718.21MIN: 406.93 / MAX: 746.39MIN: 371.54 / MAX: 736.5

CP2K Molecular Dynamics

OpenBenchmarking.orgSeconds, Fewer Is BetterCP2K Molecular Dynamics 2024.3Input: H20-256acb140280420560700592.86624.57629.561. (F9X) gfortran options: -fopenmp -march=native -mtune=native -O3 -funroll-loops -fbacktrace -ffree-form -fimplicit-none -std=f2008 -lcp2kstart -lcp2kmc -lcp2kswarm -lcp2kmotion -lcp2kthermostat -lcp2kemd -lcp2ktmc -lcp2kmain -lcp2kdbt -lcp2ktas -lcp2kgrid -lcp2kgriddgemm -lcp2kgridcpu -lcp2kgridref -lcp2kgridcommon -ldbcsrarnoldi -ldbcsrx -lcp2kdbx -lcp2kdbm -lcp2kshg_int -lcp2keri_mme -lcp2kminimax -lcp2khfxbase -lcp2ksubsys -lcp2kxc -lcp2kao -lcp2kpw_env -lcp2kinput -lcp2kpw -lcp2kgpu -lcp2kfft -lcp2kfpga -lcp2kfm -lcp2kcommon -lcp2koffload -lcp2kmpiwrap -lcp2kbase -ldbcsr -lsirius -lspla -lspfft -lsymspg -lvdwxc -l:libhdf5_fortran.a -l:libhdf5.a -lz -lgsl -lelpa_openmp -lcosma -lcosta -lscalapack -lxsmmf -lxsmm -ldl -lpthread -llibgrpp -lxcf03 -lxc -lint2 -lfftw3_mpi -lfftw3 -lfftw3_omp -lmpi_cxx -lmpi -l:libopenblas.a -lvori -lstdc++ -lmpi_usempif08 -lmpi_mpifh -lopen-rte -lopen-pal -lhwloc -levent_core -levent_pthreads -lm

Renaissance

OpenBenchmarking.orgms, Fewer Is BetterRenaissance 0.16Test: Random Forestbac90180270360450398.1414.4420.8MIN: 343.09 / MAX: 475.62MIN: 322.79 / MAX: 466.1MIN: 316.29 / MAX: 556.39

Llama.cpp

OpenBenchmarking.orgTokens Per Second, More Is BetterLlama.cpp b4154Backend: CPU BLAS - Model: Llama-3.1-Tulu-3-8B-Q8_0 - Test: Prompt Processing 2048ab142842567063.0959.841. (CXX) g++ options: -std=c++11 -fPIC -O3 -pthread -fopenmp -march=native -mtune=native -lopenblas

Rustls

OpenBenchmarking.orghandshakes/s, More Is BetterRustls 0.23.17Benchmark: handshake - Suite: TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384ab90K180K270K360K450K423535.68402625.061. (CC) gcc options: -m64 -lgcc_s -lutil -lrt -lpthread -lm -ldl -lc -pie -nodefaultlibs

simdjson

OpenBenchmarking.orgGB/s, More Is Bettersimdjson 3.10Throughput Test: LargeRandomabc0.41180.82361.23541.64722.0591.831.811.741. (CXX) g++ options: -O3 -lrt

Gcrypt Library

OpenBenchmarking.orgSeconds, Fewer Is BetterGcrypt Library 1.10.3ba4080120160200154.53162.131. (CC) gcc options: -O2 -fvisibility=hidden

XNNPACK

OpenBenchmarking.orgus, Fewer Is BetterXNNPACK b7b048Model: FP16MobileNetV2ab30060090012001500119012471. (CXX) g++ options: -O3 -lrt -lm

oneDNN

OpenBenchmarking.orgms, Fewer Is BetteroneDNN 3.6Harness: Deconvolution Batch shapes_1d - Engine: CPUab0.70031.40062.10092.80123.50152.976123.11260MIN: 2.42MIN: 2.41. (CXX) g++ options: -O3 -march=native -fopenmp -msse4.1 -fPIC -fcf-protection=full -pie -ldl

LiteRT

OpenBenchmarking.orgMicroseconds, Fewer Is BetterLiteRT 2024-10-15Model: Inception ResNet V2ab4K8K12K16K20K19530.220375.7

XNNPACK

OpenBenchmarking.orgus, Fewer Is BetterXNNPACK b7b048Model: FP32MobileNetV2ab30060090012001500149515591. (CXX) g++ options: -O3 -lrt -lm

simdjson

OpenBenchmarking.orgGB/s, More Is Bettersimdjson 3.10Throughput Test: Kostyaabc1.34332.68664.02995.37326.71655.975.935.731. (CXX) g++ options: -O3 -lrt

PyPerformance

OpenBenchmarking.orgMilliseconds, Fewer Is BetterPyPerformance 1.11Benchmark: asyncio_tcp_sslab150300450600750645672

Llamafile

OpenBenchmarking.orgTokens Per Second, More Is BetterLlamafile 0.8.16Model: Llama-3.2-3B-Instruct.Q6_K - Test: Text Generation 16ab51015202519.0318.32

XNNPACK

OpenBenchmarking.orgus, Fewer Is BetterXNNPACK b7b048Model: FP32MobileNetV3Largeab400800120016002000181018771. (CXX) g++ options: -O3 -lrt -lm

LiteRT

OpenBenchmarking.orgMicroseconds, Fewer Is BetterLiteRT 2024-10-15Model: SqueezeNetab4008001200160020001794.111860.35

Llama.cpp

OpenBenchmarking.orgTokens Per Second, More Is BetterLlama.cpp b4154Backend: CPU BLAS - Model: Mistral-7B-Instruct-v0.3-Q8_0 - Test: Prompt Processing 2048ba153045607565.2762.971. (CXX) g++ options: -std=c++11 -fPIC -O3 -pthread -fopenmp -march=native -mtune=native -lopenblas

Renaissance

OpenBenchmarking.orgms, Fewer Is BetterRenaissance 0.16Test: Genetic Algorithm Using Jenetics + Futurescab160320480640800719.1732.8744.3MIN: 670.9 / MAX: 764.9MIN: 713.67 / MAX: 813.49MIN: 714.12 / MAX: 802.66

XNNPACK

OpenBenchmarking.orgus, Fewer Is BetterXNNPACK b7b048Model: FP16MobileNetV3Largeab30060090012001500149815491. (CXX) g++ options: -O3 -lrt -lm

simdjson

OpenBenchmarking.orgGB/s, More Is Bettersimdjson 3.10Throughput Test: TopTweetbca369121510.8010.7910.461. (CXX) g++ options: -O3 -lrt

Llama.cpp

OpenBenchmarking.orgTokens Per Second, More Is BetterLlama.cpp b4154Backend: CPU BLAS - Model: granite-3.0-3b-a800m-instruct-Q8_0 - Test: Text Generation 128ab112233445547.7246.281. (CXX) g++ options: -std=c++11 -fPIC -O3 -pthread -fopenmp -march=native -mtune=native -lopenblas

XNNPACK

OpenBenchmarking.orgus, Fewer Is BetterXNNPACK b7b048Model: FP32MobileNetV1ab30060090012001500125212901. (CXX) g++ options: -O3 -lrt -lm

Renaissance

OpenBenchmarking.orgms, Fewer Is BetterRenaissance 0.16Test: Gaussian Mixture Modelacb70014002100280035003399.53472.43494.8MIN: 2471.52MIN: 2469.6MIN: 2520.23

XNNPACK

OpenBenchmarking.orgus, Fewer Is BetterXNNPACK b7b048Model: FP16MobileNetV1ab30060090012001500114311741. (CXX) g++ options: -O3 -lrt -lm

OpenBenchmarking.orgus, Fewer Is BetterXNNPACK b7b048Model: FP32MobileNetV3Smallab200400600800100097910051. (CXX) g++ options: -O3 -lrt -lm

Renaissance

OpenBenchmarking.orgms, Fewer Is BetterRenaissance 0.16Test: Savina Reactors.IOacb80016002400320040003506.43567.83594.3MIN: 3506.38 / MAX: 4329.37MAX: 5162.74MIN: 3594.26 / MAX: 4599.09

OpenBenchmarking.orgms, Fewer Is BetterRenaissance 0.16Test: Akka Unbalanced Cobwebbed Treecab100020003000400050004331.74403.84439.9MIN: 4331.69 / MAX: 5601.8MAX: 5719.11MAX: 5696.46

SVT-AV1

OpenBenchmarking.orgFrames Per Second, More Is BetterSVT-AV1 2.3Encoder Mode: Preset 8 - Input: Bosphorus 1080pacb70140210280350339.02338.65330.871. (CXX) g++ options: -march=native -mno-avx -mavx2 -mavx512f -mavx512bw -mavx512dq

OpenBenchmarking.orgFrames Per Second, More Is BetterSVT-AV1 2.3Encoder Mode: Preset 8 - Input: Bosphorus 4Kacb20406080100102.01100.8999.551. (CXX) g++ options: -march=native -mno-avx -mavx2 -mavx512f -mavx512bw -mavx512dq

oneDNN

OpenBenchmarking.orgms, Fewer Is BetteroneDNN 3.6Harness: IP Shapes 3D - Engine: CPUab0.93531.87062.80593.74124.67654.058004.15682MIN: 3.75MIN: 3.751. (CXX) g++ options: -O3 -march=native -fopenmp -msse4.1 -fPIC -fcf-protection=full -pie -ldl

Renaissance

OpenBenchmarking.orgms, Fewer Is BetterRenaissance 0.16Test: Finagle HTTP Requestsbca50010001500200025002264.72296.62319.4MIN: 1788.41 / MAX: 2264.71MIN: 1805.17MIN: 1832.84

oneDNN

OpenBenchmarking.orgms, Fewer Is BetteroneDNN 3.6Harness: IP Shapes 1D - Engine: CPUab0.25940.51880.77821.03761.2971.125731.15274MIN: 1.03MIN: 1.031. (CXX) g++ options: -O3 -march=native -fopenmp -msse4.1 -fPIC -fcf-protection=full -pie -ldl

Llama.cpp

OpenBenchmarking.orgTokens Per Second, More Is BetterLlama.cpp b4154Backend: CPU BLAS - Model: granite-3.0-3b-a800m-instruct-Q8_0 - Test: Prompt Processing 2048ba60120180240300285.71279.041. (CXX) g++ options: -std=c++11 -fPIC -O3 -pthread -fopenmp -march=native -mtune=native -lopenblas

CP2K Molecular Dynamics

OpenBenchmarking.orgSeconds, Fewer Is BetterCP2K Molecular Dynamics 2024.3Input: H20-64bac132639526557.3558.1958.651. (F9X) gfortran options: -fopenmp -march=native -mtune=native -O3 -funroll-loops -fbacktrace -ffree-form -fimplicit-none -std=f2008 -lcp2kstart -lcp2kmc -lcp2kswarm -lcp2kmotion -lcp2kthermostat -lcp2kemd -lcp2ktmc -lcp2kmain -lcp2kdbt -lcp2ktas -lcp2kgrid -lcp2kgriddgemm -lcp2kgridcpu -lcp2kgridref -lcp2kgridcommon -ldbcsrarnoldi -ldbcsrx -lcp2kdbx -lcp2kdbm -lcp2kshg_int -lcp2keri_mme -lcp2kminimax -lcp2khfxbase -lcp2ksubsys -lcp2kxc -lcp2kao -lcp2kpw_env -lcp2kinput -lcp2kpw -lcp2kgpu -lcp2kfft -lcp2kfpga -lcp2kfm -lcp2kcommon -lcp2koffload -lcp2kmpiwrap -lcp2kbase -ldbcsr -lsirius -lspla -lspfft -lsymspg -lvdwxc -l:libhdf5_fortran.a -l:libhdf5.a -lz -lgsl -lelpa_openmp -lcosma -lcosta -lscalapack -lxsmmf -lxsmm -ldl -lpthread -llibgrpp -lxcf03 -lxc -lint2 -lfftw3_mpi -lfftw3 -lfftw3_omp -lmpi_cxx -lmpi -l:libopenblas.a -lvori -lstdc++ -lmpi_usempif08 -lmpi_mpifh -lopen-rte -lopen-pal -lhwloc -levent_core -levent_pthreads -lm

oneDNN

OpenBenchmarking.orgms, Fewer Is BetteroneDNN 3.6Harness: Convolution Batch Shapes Auto - Engine: CPUab2468106.672876.81754MIN: 6.2MIN: 6.21. (CXX) g++ options: -O3 -march=native -fopenmp -msse4.1 -fPIC -fcf-protection=full -pie -ldl

x265

OpenBenchmarking.orgFrames Per Second, More Is Betterx265Video Input: Bosphorus 4Kcab81624324032.7332.5732.041. x265 [info]: HEVC encoder version 3.5+1-f0c1022b6

SVT-AV1

OpenBenchmarking.orgFrames Per Second, More Is BetterSVT-AV1 2.3Encoder Mode: Preset 13 - Input: Bosphorus 1080pacb2004006008001000842.56838.17824.811. (CXX) g++ options: -march=native -mno-avx -mavx2 -mavx512f -mavx512bw -mavx512dq

OpenBenchmarking.orgFrames Per Second, More Is BetterSVT-AV1 2.3Encoder Mode: Preset 5 - Input: Beauty 4K 10-bitacb2468106.5046.3746.3711. (CXX) g++ options: -march=native -mno-avx -mavx2 -mavx512f -mavx512bw -mavx512dq

Timed Eigen Compilation

OpenBenchmarking.orgSeconds, Fewer Is BetterTimed Eigen Compilation 3.4.0Time To Compileab132639526558.6659.87

oneDNN

OpenBenchmarking.orgms, Fewer Is BetteroneDNN 3.6Harness: Deconvolution Batch shapes_3d - Engine: CPUab0.55411.10821.66232.21642.77052.412942.46279MIN: 2.34MIN: 2.351. (CXX) g++ options: -O3 -march=native -fopenmp -msse4.1 -fPIC -fcf-protection=full -pie -ldl

Rustls

OpenBenchmarking.orghandshakes/s, More Is BetterRustls 0.23.17Benchmark: handshake-resume - Suite: TLS13_CHACHA20_POLY1305_SHA256ab80K160K240K320K400K388077.69380493.861. (CC) gcc options: -m64 -lgcc_s -lutil -lrt -lpthread -lm -ldl -lc -pie -nodefaultlibs

Whisper.cpp

OpenBenchmarking.orgSeconds, Fewer Is BetterWhisper.cpp 1.6.2Model: ggml-small.en - Input: 2016 State of the Unionba50100150200250240.60245.081. (CXX) g++ options: -O3 -std=c++11 -fPIC -pthread -msse3 -mssse3 -mavx -mf16c -mfma -mavx2 -mavx512f -mavx512cd -mavx512vl -mavx512dq -mavx512bw -mavx512vbmi -mavx512vnni

Rustls

OpenBenchmarking.orghandshakes/s, More Is BetterRustls 0.23.17Benchmark: handshake-ticket - Suite: TLS13_CHACHA20_POLY1305_SHA256ab90K180K270K360K450K404263.45397022.401. (CC) gcc options: -m64 -lgcc_s -lutil -lrt -lpthread -lm -ldl -lc -pie -nodefaultlibs

ONNX Runtime

OpenBenchmarking.orgInferences Per Second, More Is BetterONNX Runtime 1.19Model: bertsquad-12 - Device: CPU - Executor: Standardab4812162015.5915.321. (CXX) g++ options: -O3 -march=native -ffunction-sections -fdata-sections -mtune=native -flto=auto -fno-fat-lto-objects -ldl -lrt

Llamafile

OpenBenchmarking.orgTokens Per Second, More Is BetterLlamafile 0.8.16Model: TinyLlama-1.1B-Chat-v1.0.BF16 - Test: Text Generation 128ab61218243026.2825.83

Rustls

OpenBenchmarking.orghandshakes/s, More Is BetterRustls 0.23.17Benchmark: handshake - Suite: TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256ab20K40K60K80K100K80462.679085.81. (CC) gcc options: -m64 -lgcc_s -lutil -lrt -lpthread -lm -ldl -lc -pie -nodefaultlibs

OpenBenchmarking.orghandshakes/s, More Is BetterRustls 0.23.17Benchmark: handshake-resume - Suite: TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256ab800K1600K2400K3200K4000K3563852.573504511.311. (CC) gcc options: -m64 -lgcc_s -lutil -lrt -lpthread -lm -ldl -lc -pie -nodefaultlibs

Stockfish

OpenBenchmarking.orgNodes Per Second, More Is BetterStockfishChess Benchmarkab10M20M30M40M50M46507038457517471. Stockfish 16 by the Stockfish developers (see AUTHORS file)

POV-Ray

OpenBenchmarking.orgSeconds, Fewer Is BetterPOV-RayTrace Timeab51015202518.5418.851. POV-Ray 3.7.0.10.unofficial

Llamafile

OpenBenchmarking.orgTokens Per Second, More Is BetterLlamafile 0.8.16Model: mistral-7b-instruct-v0.2.Q5_K_M - Test: Text Generation 128ba369121510.6410.47

OpenBenchmarking.orgTokens Per Second, More Is BetterLlamafile 0.8.16Model: Llama-3.2-3B-Instruct.Q6_K - Test: Text Generation 128ab51015202520.1319.81

Renaissance

OpenBenchmarking.orgms, Fewer Is BetterRenaissance 0.16Test: ALS Movie Lensacb2K4K6K8K10K9805.79907.49958.3MIN: 9253.4 / MAX: 10057.61MIN: 9393.64 / MAX: 10087.8MIN: 9305.94 / MAX: 10040.58

SVT-AV1

OpenBenchmarking.orgFrames Per Second, More Is BetterSVT-AV1 2.3Encoder Mode: Preset 13 - Input: Bosphorus 4Kcab50100150200250212.95212.52209.771. (CXX) g++ options: -march=native -mno-avx -mavx2 -mavx512f -mavx512bw -mavx512dq

oneDNN

OpenBenchmarking.orgms, Fewer Is BetteroneDNN 3.6Harness: Recurrent Neural Network Inference - Engine: CPUab150300450600750700.86711.43MIN: 679.89MIN: 684.031. (CXX) g++ options: -O3 -march=native -fopenmp -msse4.1 -fPIC -fcf-protection=full -pie -ldl

ONNX Runtime

OpenBenchmarking.orgInferences Per Second, More Is BetterONNX Runtime 1.19Model: ZFNet-512 - Device: CPU - Executor: Standardba20406080100103.86102.331. (CXX) g++ options: -O3 -march=native -ffunction-sections -fdata-sections -mtune=native -flto=auto -fno-fat-lto-objects -ldl -lrt

x265

OpenBenchmarking.orgFrames Per Second, More Is Betterx265Video Input: Bosphorus 1080pcab306090120150114.52114.45112.851. x265 [info]: HEVC encoder version 3.5+1-f0c1022b6

ONNX Runtime

OpenBenchmarking.orgInferences Per Second, More Is BetterONNX Runtime 1.19Model: fcn-resnet101-11 - Device: CPU - Executor: Standardab0.72381.44762.17142.89523.6193.21673.17051. (CXX) g++ options: -O3 -march=native -ffunction-sections -fdata-sections -mtune=native -flto=auto -fno-fat-lto-objects -ldl -lrt

simdjson

OpenBenchmarking.orgGB/s, More Is Bettersimdjson 3.10Throughput Test: PartialTweetsbac36912159.829.769.681. (CXX) g++ options: -O3 -lrt

Whisperfile

OpenBenchmarking.orgSeconds, Fewer Is BetterWhisperfile 20Aug24Model Size: Smallba4080120160200192.68195.42

ONNX Runtime

OpenBenchmarking.orgInferences Per Second, More Is BetterONNX Runtime 1.19Model: Faster R-CNN R-50-FPN-int8 - Device: CPU - Executor: Standardba112233445547.7347.071. (CXX) g++ options: -O3 -march=native -ffunction-sections -fdata-sections -mtune=native -flto=auto -fno-fat-lto-objects -ldl -lrt

NAMD

OpenBenchmarking.orgns/day, More Is BetterNAMD 3.0Input: ATPase with 327,506 Atomscab0.63661.27321.90982.54643.1832.829252.796322.79025

ONNX Runtime

OpenBenchmarking.orgInferences Per Second, More Is BetterONNX Runtime 1.19Model: ResNet101_DUC_HDC-12 - Device: CPU - Executor: Standardab0.34690.69381.04071.38761.73451.541961.521091. (CXX) g++ options: -O3 -march=native -ffunction-sections -fdata-sections -mtune=native -flto=auto -fno-fat-lto-objects -ldl -lrt

Apache CouchDB

OpenBenchmarking.orgSeconds, Fewer Is BetterApache CouchDB 3.4.1Bulk Size: 100 - Inserts: 3000 - Rounds: 30ab50100150200250232.19235.351. (CXX) g++ options: -flto -lstdc++ -shared -lei

Numpy Benchmark

This is a test to obtain the general Numpy performance. Learn more via the OpenBenchmarking.org test page.

OpenBenchmarking.orgScore, More Is BetterNumpy Benchmarkab2004006008001000775.75765.35

PyPerformance

OpenBenchmarking.orgMilliseconds, Fewer Is BetterPyPerformance 1.11Benchmark: chaosab91827364538.238.7

ACES DGEMM

OpenBenchmarking.orgGFLOP/s, More Is BetterACES DGEMM 1.0Sustained Floating-Point Rateabc20040060080010001141.191137.391127.271. (CC) gcc options: -ffast-math -mavx2 -O3 -fopenmp -lopenblas

SVT-AV1

OpenBenchmarking.orgFrames Per Second, More Is BetterSVT-AV1 2.3Encoder Mode: Preset 5 - Input: Bosphorus 1080pcab20406080100102.11101.97100.891. (CXX) g++ options: -march=native -mno-avx -mavx2 -mavx512f -mavx512bw -mavx512dq

PyPerformance

OpenBenchmarking.orgMilliseconds, Fewer Is BetterPyPerformance 1.11Benchmark: floatba112233445550.150.7

XNNPACK

OpenBenchmarking.orgus, Fewer Is BetterXNNPACK b7b048Model: FP16MobileNetV3Smallab20040060080010009209311. (CXX) g++ options: -O3 -lrt -lm

Rustls

OpenBenchmarking.orghandshakes/s, More Is BetterRustls 0.23.17Benchmark: handshake-ticket - Suite: TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256ab600K1200K1800K2400K3000K2620332.002589637.921. (CC) gcc options: -m64 -lgcc_s -lutil -lrt -lpthread -lm -ldl -lc -pie -nodefaultlibs

XNNPACK

OpenBenchmarking.orgus, Fewer Is BetterXNNPACK b7b048Model: QS8MobileNetV2ab20040060080010008448541. (CXX) g++ options: -O3 -lrt -lm

Whisperfile

OpenBenchmarking.orgSeconds, Fewer Is BetterWhisperfile 20Aug24Model Size: Tinyab102030405041.7142.20

ONNX Runtime

OpenBenchmarking.orgInferences Per Second, More Is BetterONNX Runtime 1.19Model: ArcFace ResNet-100 - Device: CPU - Executor: Standardab102030405042.4541.961. (CXX) g++ options: -O3 -march=native -ffunction-sections -fdata-sections -mtune=native -flto=auto -fno-fat-lto-objects -ldl -lrt

Renaissance

OpenBenchmarking.orgms, Fewer Is BetterRenaissance 0.16Test: Apache Spark PageRankacb50010001500200025002412.22439.22439.9MIN: 1691.04MIN: 1679.36 / MAX: 2439.21MIN: 1684.02 / MAX: 2439.95

SVT-AV1

OpenBenchmarking.orgFrames Per Second, More Is BetterSVT-AV1 2.3Encoder Mode: Preset 8 - Input: Beauty 4K 10-bitbca369121512.6112.6012.471. (CXX) g++ options: -march=native -mno-avx -mavx2 -mavx512f -mavx512bw -mavx512dq

PyPerformance

OpenBenchmarking.orgMilliseconds, Fewer Is BetterPyPerformance 1.11Benchmark: raytraceab4080120160200175177

Rustls

OpenBenchmarking.orghandshakes/s, More Is BetterRustls 0.23.17Benchmark: handshake-ticket - Suite: TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384ab300K600K900K1200K1500K1553632.141536355.901. (CC) gcc options: -m64 -lgcc_s -lutil -lrt -lpthread -lm -ldl -lc -pie -nodefaultlibs

FinanceBench

FinanceBench is a collection of financial program benchmarks with support for benchmarking on the GPU via OpenCL and CPU benchmarking with OpenMP. The FinanceBench test cases are focused on Black-Sholes-Merton Process with Analytic European Option engine, QMC (Sobol) Monte-Carlo method (Equity Option Example), Bonds Fixed-rate bond with flat forward curve, and Repo Securities repurchase agreement. FinanceBench was originally written by the Cavazos Lab at University of Delaware. Learn more via the OpenBenchmarking.org test page.

OpenBenchmarking.orgms, Fewer Is BetterFinanceBench 2016-07-25Benchmark: Bonds OpenMPab7K14K21K28K35K33061.2233432.641. (CXX) g++ options: -O3 -march=native -fopenmp

ONNX Runtime

OpenBenchmarking.orgInferences Per Second, More Is BetterONNX Runtime 1.19Model: CaffeNet 12-int8 - Device: CPU - Executor: Standardab140280420560700636.32629.661. (CXX) g++ options: -O3 -march=native -ffunction-sections -fdata-sections -mtune=native -flto=auto -fno-fat-lto-objects -ldl -lrt

Apache CouchDB

OpenBenchmarking.orgSeconds, Fewer Is BetterApache CouchDB 3.4.1Bulk Size: 500 - Inserts: 3000 - Rounds: 30ab110220330440550511.78517.151. (CXX) g++ options: -flto -lstdc++ -shared -lei

ONNX Runtime

OpenBenchmarking.orgInferences Per Second, More Is BetterONNX Runtime 1.19Model: ResNet50 v1-12-int8 - Device: CPU - Executor: Standardab80160240320400390.60386.581. (CXX) g++ options: -O3 -march=native -ffunction-sections -fdata-sections -mtune=native -flto=auto -fno-fat-lto-objects -ldl -lrt

PyPerformance

OpenBenchmarking.orgMilliseconds, Fewer Is BetterPyPerformance 1.11Benchmark: goba2040608010077.077.8

SVT-AV1

OpenBenchmarking.orgFrames Per Second, More Is BetterSVT-AV1 2.3Encoder Mode: Preset 3 - Input: Bosphorus 4Kabc36912159.5909.5549.4951. (CXX) g++ options: -march=native -mno-avx -mavx2 -mavx512f -mavx512bw -mavx512dq

Apache CouchDB

OpenBenchmarking.orgSeconds, Fewer Is BetterApache CouchDB 3.4.1Bulk Size: 300 - Inserts: 1000 - Rounds: 30ab20406080100106.13107.181. (CXX) g++ options: -flto -lstdc++ -shared -lei

Llama.cpp

OpenBenchmarking.orgTokens Per Second, More Is BetterLlama.cpp b4154Backend: CPU BLAS - Model: granite-3.0-3b-a800m-instruct-Q8_0 - Test: Prompt Processing 512ab70140210280350327.30324.211. (CXX) g++ options: -std=c++11 -fPIC -O3 -pthread -fopenmp -march=native -mtune=native -lopenblas

Blender

OpenBenchmarking.orgSeconds, Fewer Is BetterBlender 4.3Blend File: Junkshop - Compute: CPU-Onlyab163248648073.5674.26

SVT-AV1

OpenBenchmarking.orgFrames Per Second, More Is BetterSVT-AV1 2.3Encoder Mode: Preset 5 - Input: Bosphorus 4Kacb81624324034.5434.4534.231. (CXX) g++ options: -march=native -mno-avx -mavx2 -mavx512f -mavx512bw -mavx512dq

OSPRay

OpenBenchmarking.orgItems Per Second, More Is BetterOSPRay 3.2Benchmark: gravity_spheres_volume/dim_512/ao/real_timecab2468107.642827.639447.57408

ONNX Runtime

OpenBenchmarking.orgInferences Per Second, More Is BetterONNX Runtime 1.19Model: yolov4 - Device: CPU - Executor: Standardab369121511.0610.961. (CXX) g++ options: -O3 -march=native -ffunction-sections -fdata-sections -mtune=native -flto=auto -fno-fat-lto-objects -ldl -lrt

7-Zip Compression

OpenBenchmarking.orgMIPS, More Is Better7-Zip CompressionTest: Decompression Ratingcba40K80K120K160K200K1673211668431659161. 7-Zip 23.01 (x64) : Copyright (c) 1999-2023 Igor Pavlov : 2023-06-20

oneDNN

OpenBenchmarking.orgms, Fewer Is BetteroneDNN 3.6Harness: Recurrent Neural Network Training - Engine: CPUab300600900120015001372.031383.64MIN: 1342.06MIN: 1333.571. (CXX) g++ options: -O3 -march=native -fopenmp -msse4.1 -fPIC -fcf-protection=full -pie -ldl

OpenVINO GenAI

OpenBenchmarking.orgtokens/s, More Is BetterOpenVINO GenAI 2024.5Model: Gemma-7b-int4-ov - Device: CPUba36912159.919.83

SVT-AV1

OpenBenchmarking.orgFrames Per Second, More Is BetterSVT-AV1 2.3Encoder Mode: Preset 13 - Input: Beauty 4K 10-bitacb51015202518.5918.5618.441. (CXX) g++ options: -march=native -mno-avx -mavx2 -mavx512f -mavx512bw -mavx512dq

OSPRay

OpenBenchmarking.orgItems Per Second, More Is BetterOSPRay 3.2Benchmark: gravity_spheres_volume/dim_512/scivis/real_timeacb2468107.587897.557917.52875

SVT-AV1

OpenBenchmarking.orgFrames Per Second, More Is BetterSVT-AV1 2.3Encoder Mode: Preset 3 - Input: Beauty 4K 10-bitabc0.320.640.961.281.61.4221.4151.4111. (CXX) g++ options: -march=native -mno-avx -mavx2 -mavx512f -mavx512bw -mavx512dq

GROMACS

OpenBenchmarking.orgNs Per Day, More Is BetterGROMACSInput: water_GMX50_bareab0.38070.76141.14211.52281.90351.6921.6791. GROMACS version: 2023.3-Ubuntu_2023.3_1ubuntu3

simdjson

OpenBenchmarking.orgGB/s, More Is Bettersimdjson 3.10Throughput Test: DistinctUserIDacb369121510.4610.4310.381. (CXX) g++ options: -O3 -lrt

Blender

OpenBenchmarking.orgSeconds, Fewer Is BetterBlender 4.3Blend File: Classroom - Compute: CPU-Onlyab306090120150143.36144.41

PyPerformance

OpenBenchmarking.orgMilliseconds, Fewer Is BetterPyPerformance 1.11Benchmark: pathlibab4812162014.214.3

Llama.cpp

OpenBenchmarking.orgTokens Per Second, More Is BetterLlama.cpp b4154Backend: CPU BLAS - Model: Mistral-7B-Instruct-v0.3-Q8_0 - Test: Text Generation 128ab2468107.247.191. (CXX) g++ options: -std=c++11 -fPIC -O3 -pthread -fopenmp -march=native -mtune=native -lopenblas

ASTC Encoder

OpenBenchmarking.orgMT/s, More Is BetterASTC Encoder 5.0Preset: Exhaustiveab0.3790.7581.1371.5161.8951.68441.67281. (CXX) g++ options: -O3 -flto -pthread

OpenBenchmarking.orgMT/s, More Is BetterASTC Encoder 5.0Preset: Thoroughab51015202520.3020.161. (CXX) g++ options: -O3 -flto -pthread

Etcpak

OpenBenchmarking.orgMpx/s, More Is BetterEtcpak 2.0Benchmark: Multi-Threaded - Configuration: ETC2abc120240360480600577.82575.02573.911. (CXX) g++ options: -flto -pthread

PyPerformance

OpenBenchmarking.orgMilliseconds, Fewer Is BetterPyPerformance 1.11Benchmark: nbodyab132639526559.059.4

SVT-AV1

OpenBenchmarking.orgFrames Per Second, More Is BetterSVT-AV1 2.3Encoder Mode: Preset 3 - Input: Bosphorus 1080pacb71421283529.5729.4729.381. (CXX) g++ options: -march=native -mno-avx -mavx2 -mavx512f -mavx512bw -mavx512dq

Llama.cpp

OpenBenchmarking.orgTokens Per Second, More Is BetterLlama.cpp b4154Backend: CPU BLAS - Model: Mistral-7B-Instruct-v0.3-Q8_0 - Test: Prompt Processing 1024ab153045607569.2668.801. (CXX) g++ options: -std=c++11 -fPIC -O3 -pthread -fopenmp -march=native -mtune=native -lopenblas

Apache CouchDB

OpenBenchmarking.orgSeconds, Fewer Is BetterApache CouchDB 3.4.1Bulk Size: 500 - Inserts: 1000 - Rounds: 30ab306090120150148.05149.031. (CXX) g++ options: -flto -lstdc++ -shared -lei

ASTC Encoder

OpenBenchmarking.orgMT/s, More Is BetterASTC Encoder 5.0Preset: Mediumab306090120150156.22155.271. (CXX) g++ options: -O3 -flto -pthread

Blender

OpenBenchmarking.orgSeconds, Fewer Is BetterBlender 4.3Blend File: Barbershop - Compute: CPU-Onlyab110220330440550506.2509.3

PyPerformance

OpenBenchmarking.orgMilliseconds, Fewer Is BetterPyPerformance 1.11Benchmark: pickle_pure_pythonab4080120160200165166

ASTC Encoder

OpenBenchmarking.orgMT/s, More Is BetterASTC Encoder 5.0Preset: Very Thoroughab0.61671.23341.85012.46683.08352.74102.72481. (CXX) g++ options: -O3 -flto -pthread

PyPerformance

OpenBenchmarking.orgMilliseconds, Fewer Is BetterPyPerformance 1.11Benchmark: gc_collectab150300450600750677681

OSPRay

OpenBenchmarking.orgItems Per Second, More Is BetterOSPRay 3.2Benchmark: particle_volume/scivis/real_timeacb36912158.984868.970058.93245

Primesieve

OpenBenchmarking.orgSeconds, Fewer Is BetterPrimesieve 12.6Length: 1e13ab2040608010078.5078.951. (CXX) g++ options: -O3

PyPerformance

OpenBenchmarking.orgMilliseconds, Fewer Is BetterPyPerformance 1.11Benchmark: regex_compileab163248648069.870.2

Llamafile

OpenBenchmarking.orgTokens Per Second, More Is BetterLlamafile 0.8.16Model: wizardcoder-python-34b-v1.0.Q6_K - Test: Text Generation 16ba0.40280.80561.20841.61122.0141.791.78

OSPRay

OpenBenchmarking.orgItems Per Second, More Is BetterOSPRay 3.2Benchmark: particle_volume/pathtracer/real_timeabc50100150200250236.25235.33234.97

PyPerformance

OpenBenchmarking.orgMilliseconds, Fewer Is BetterPyPerformance 1.11Benchmark: async_tree_ioab160320480640800755759

Llamafile

OpenBenchmarking.orgTokens Per Second, More Is BetterLlamafile 0.8.16Model: wizardcoder-python-34b-v1.0.Q6_K - Test: Text Generation 128ba0.450.91.351.82.252.001.99

Blender

OpenBenchmarking.orgSeconds, Fewer Is BetterBlender 4.3Blend File: Fishy Cat - Compute: CPU-Onlyab163248648071.3571.70

Primesieve

OpenBenchmarking.orgSeconds, Fewer Is BetterPrimesieve 12.6Length: 1e12ab2468106.3476.3781. (CXX) g++ options: -O3

Rustls

OpenBenchmarking.orghandshakes/s, More Is BetterRustls 0.23.17Benchmark: handshake - Suite: TLS13_CHACHA20_POLY1305_SHA256ab16K32K48K64K80K76454.4576083.731. (CC) gcc options: -m64 -lgcc_s -lutil -lrt -lpthread -lm -ldl -lc -pie -nodefaultlibs

FinanceBench

FinanceBench is a collection of financial program benchmarks with support for benchmarking on the GPU via OpenCL and CPU benchmarking with OpenMP. The FinanceBench test cases are focused on Black-Sholes-Merton Process with Analytic European Option engine, QMC (Sobol) Monte-Carlo method (Equity Option Example), Bonds Fixed-rate bond with flat forward curve, and Repo Securities repurchase agreement. FinanceBench was originally written by the Cavazos Lab at University of Delaware. Learn more via the OpenBenchmarking.org test page.

OpenBenchmarking.orgms, Fewer Is BetterFinanceBench 2016-07-25Benchmark: Repo OpenMPab5K10K15K20K25K21418.4521522.071. (CXX) g++ options: -O3 -march=native -fopenmp

PyPerformance

OpenBenchmarking.orgMilliseconds, Fewer Is BetterPyPerformance 1.11Benchmark: django_templateab51015202520.720.8

OSPRay

OpenBenchmarking.orgItems Per Second, More Is BetterOSPRay 3.2Benchmark: particle_volume/ao/real_timeacb36912159.009178.985868.96632

BYTE Unix Benchmark

OpenBenchmarking.orgLPS, More Is BetterBYTE Unix Benchmark 5.1.3-gitComputational Test: Dhrystone 2acb400M800M1200M1600M2000M1866536062.71862548305.41857795366.11. (CC) gcc options: -pedantic -O3 -ffast-math -march=native -mtune=native -lm

QuantLib

OpenBenchmarking.orgtasks/s, More Is BetterQuantLib 1.35-devSize: XXScab369121513.4913.4313.431. (CXX) g++ options: -O3 -march=native -fPIE -pie

Y-Cruncher

OpenBenchmarking.orgSeconds, Fewer Is BetterY-Cruncher 0.8.5Pi Digits To Calculate: 1Bba51015202518.4018.49

Llama.cpp

OpenBenchmarking.orgTokens Per Second, More Is BetterLlama.cpp b4154Backend: CPU BLAS - Model: Llama-3.1-Tulu-3-8B-Q8_0 - Test: Text Generation 128ab2468106.886.851. (CXX) g++ options: -std=c++11 -fPIC -O3 -pthread -fopenmp -march=native -mtune=native -lopenblas

OpenVINO GenAI

OpenBenchmarking.orgtokens/s, More Is BetterOpenVINO GenAI 2024.5Model: Phi-3-mini-128k-instruct-int4-ov - Device: CPUab51015202519.2819.20

Llamafile

OpenBenchmarking.orgTokens Per Second, More Is BetterLlamafile 0.8.16Model: TinyLlama-1.1B-Chat-v1.0.BF16 - Test: Text Generation 16ba61218243024.6924.59

Whisperfile

OpenBenchmarking.orgSeconds, Fewer Is BetterWhisperfile 20Aug24Model Size: Mediumba120240360480600532.81534.92

BYTE Unix Benchmark

OpenBenchmarking.orgLPS, More Is BetterBYTE Unix Benchmark 5.1.3-gitComputational Test: Pipeabc10M20M30M40M50M48806257.148718087.148613927.91. (CC) gcc options: -pedantic -O3 -ffast-math -march=native -mtune=native -lm

Llamafile

OpenBenchmarking.orgTokens Per Second, More Is BetterLlamafile 0.8.16Model: mistral-7b-instruct-v0.2.Q5_K_M - Test: Text Generation 16ba369121510.2610.22

Blender

OpenBenchmarking.orgSeconds, Fewer Is BetterBlender 4.3Blend File: BMW27 - Compute: CPU-Onlyab122436486053.5553.75

OpenSSL

OpenBenchmarking.orgbyte/s, More Is BetterOpenSSLAlgorithm: AES-128-GCMab20000M40000M60000M80000M100000M1047845221701044043478401. OpenSSL 3.0.13 30 Jan 2024 (Library: OpenSSL 3.0.13 30 Jan 2024) - Additional Parameters: -engine qatengine -async_jobs 8

OpenBenchmarking.orgbyte/s, More Is BetterOpenSSLAlgorithm: AES-256-GCMab20000M40000M60000M80000M100000M97172751700968217370601. OpenSSL 3.0.13 30 Jan 2024 (Library: OpenSSL 3.0.13 30 Jan 2024) - Additional Parameters: -engine qatengine -async_jobs 8

PyPerformance

OpenBenchmarking.orgMilliseconds, Fewer Is BetterPyPerformance 1.11Benchmark: python_startupab1.30282.60563.90845.21126.5145.775.79

OSPRay

OpenBenchmarking.orgItems Per Second, More Is BetterOSPRay 3.2Benchmark: gravity_spheres_volume/dim_512/pathtracer/real_timeacb2468108.820938.811998.79096

Whisper.cpp

OpenBenchmarking.orgSeconds, Fewer Is BetterWhisper.cpp 1.6.2Model: ggml-medium.en - Input: 2016 State of the Unionab150300450600750700.91703.221. (CXX) g++ options: -O3 -std=c++11 -fPIC -pthread -msse3 -mssse3 -mavx -mf16c -mfma -mavx2 -mavx512f -mavx512cd -mavx512vl -mavx512dq -mavx512bw -mavx512vbmi -mavx512vnni

PyPerformance

OpenBenchmarking.orgMilliseconds, Fewer Is BetterPyPerformance 1.11Benchmark: asyncio_websocketsab70140210280350315316

OpenVINO GenAI

OpenBenchmarking.orgtokens/s, More Is BetterOpenVINO GenAI 2024.5Model: Falcon-7b-instruct-int4-ov - Device: CPUba369121512.9712.93

QuantLib

OpenBenchmarking.orgtasks/s, More Is BetterQuantLib 1.35-devSize: Sacb369121512.7512.7212.711. (CXX) g++ options: -O3 -march=native -fPIE -pie

PyPerformance

OpenBenchmarking.orgMilliseconds, Fewer Is BetterPyPerformance 1.11Benchmark: xml_etreeba81624324035.735.8

7-Zip Compression

OpenBenchmarking.orgMIPS, More Is Better7-Zip CompressionTest: Compression Ratingcba40K80K120K160K200K1643131640501638591. 7-Zip 23.01 (x64) : Copyright (c) 1999-2023 Igor Pavlov : 2023-06-20

Apache CouchDB

OpenBenchmarking.orgSeconds, Fewer Is BetterApache CouchDB 3.4.1Bulk Size: 100 - Inserts: 1000 - Rounds: 30ab163248648069.9370.111. (CXX) g++ options: -flto -lstdc++ -shared -lei

Build2

OpenBenchmarking.orgSeconds, Fewer Is BetterBuild2 0.17Time To Compileab2040608010092.0592.29

BYTE Unix Benchmark

OpenBenchmarking.orgLPS, More Is BetterBYTE Unix Benchmark 5.1.3-gitComputational Test: System Callabc11M22M33M44M55M49140426.649062324.149016743.61. (CC) gcc options: -pedantic -O3 -ffast-math -march=native -mtune=native -lm

Y-Cruncher

OpenBenchmarking.orgSeconds, Fewer Is BetterY-Cruncher 0.8.5Pi Digits To Calculate: 500Mab2468108.7728.794

Whisper.cpp

OpenBenchmarking.orgSeconds, Fewer Is BetterWhisper.cpp 1.6.2Model: ggml-base.en - Input: 2016 State of the Unionba2040608010087.2787.491. (CXX) g++ options: -O3 -std=c++11 -fPIC -pthread -msse3 -mssse3 -mavx -mf16c -mfma -mavx2 -mavx512f -mavx512cd -mavx512vl -mavx512dq -mavx512bw -mavx512vbmi -mavx512vnni

ONNX Runtime

OpenBenchmarking.orgInferences Per Second, More Is BetterONNX Runtime 1.19Model: T5 Encoder - Device: CPU - Executor: Standardba306090120150156.83156.451. (CXX) g++ options: -O3 -march=native -ffunction-sections -fdata-sections -mtune=native -flto=auto -fno-fat-lto-objects -ldl -lrt

PyPerformance

OpenBenchmarking.orgMilliseconds, Fewer Is BetterPyPerformance 1.11Benchmark: crypto_pyaesab102030405041.741.8

NAMD

OpenBenchmarking.orgns/day, More Is BetterNAMD 3.0Input: STMV with 1,066,628 Atomscab0.17060.34120.51180.68240.8530.758130.756560.75634

Apache CouchDB

OpenBenchmarking.orgSeconds, Fewer Is BetterApache CouchDB 3.4.1Bulk Size: 300 - Inserts: 3000 - Rounds: 30ab80160240320400367.83368.661. (CXX) g++ options: -flto -lstdc++ -shared -lei

ONNX Runtime

OpenBenchmarking.orgInferences Per Second, More Is BetterONNX Runtime 1.19Model: GPT-2 - Device: CPU - Executor: Standardab306090120150134.60134.311. (CXX) g++ options: -O3 -march=native -ffunction-sections -fdata-sections -mtune=native -flto=auto -fno-fat-lto-objects -ldl -lrt

OpenSSL

OpenBenchmarking.orgbyte/s, More Is BetterOpenSSLAlgorithm: ChaCha20-Poly1305ab20000M40000M60000M80000M100000M92393529340922163505801. OpenSSL 3.0.13 30 Jan 2024 (Library: OpenSSL 3.0.13 30 Jan 2024) - Additional Parameters: -engine qatengine -async_jobs 8

OpenBenchmarking.orgbyte/s, More Is BetterOpenSSLAlgorithm: ChaCha20ab30000M60000M90000M120000M150000M1305884950501303598841901. OpenSSL 3.0.13 30 Jan 2024 (Library: OpenSSL 3.0.13 30 Jan 2024) - Additional Parameters: -engine qatengine -async_jobs 8

BYTE Unix Benchmark

OpenBenchmarking.orgMWIPS, More Is BetterBYTE Unix Benchmark 5.1.3-gitComputational Test: Whetstone Doubleacb70K140K210K280K350K343491.9343187.0343113.01. (CC) gcc options: -pedantic -O3 -ffast-math -march=native -mtune=native -lm

ONNX Runtime

OpenBenchmarking.orgInferences Per Second, More Is BetterONNX Runtime 1.19Model: super-resolution-10 - Device: CPU - Executor: Standardab306090120150141.12141.001. (CXX) g++ options: -O3 -march=native -ffunction-sections -fdata-sections -mtune=native -flto=auto -fno-fat-lto-objects -ldl -lrt

Blender

OpenBenchmarking.orgSeconds, Fewer Is BetterBlender 4.3Blend File: Pabellon Barcelona - Compute: CPU-Onlyab4080120160200166.12166.25

ASTC Encoder

OpenBenchmarking.orgMT/s, More Is BetterASTC Encoder 5.0Preset: Fastab90180270360450396.65396.431. (CXX) g++ options: -O3 -flto -pthread

Rustls

OpenBenchmarking.orghandshakes/s, More Is BetterRustls 0.23.17Benchmark: handshake-resume - Suite: TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384ba400K800K1200K1600K2000K1821261.881820810.211. (CC) gcc options: -m64 -lgcc_s -lutil -lrt -lpthread -lm -ldl -lc -pie -nodefaultlibs

Apache Cassandra

OpenBenchmarking.orgOp/s, More Is BetterApache Cassandra 5.0Test: Writesba60K120K180K240K300K271373271333

Llamafile

OpenBenchmarking.orgTokens Per Second, More Is BetterLlamafile 0.8.16Model: wizardcoder-python-34b-v1.0.Q6_K - Test: Prompt Processing 2048ba3K6K9K12K15K1228812288

OpenBenchmarking.orgTokens Per Second, More Is BetterLlamafile 0.8.16Model: wizardcoder-python-34b-v1.0.Q6_K - Test: Prompt Processing 1024ba1300260039005200650061446144

OpenBenchmarking.orgTokens Per Second, More Is BetterLlamafile 0.8.16Model: wizardcoder-python-34b-v1.0.Q6_K - Test: Prompt Processing 512ba700140021002800350030723072

OpenBenchmarking.orgTokens Per Second, More Is BetterLlamafile 0.8.16Model: wizardcoder-python-34b-v1.0.Q6_K - Test: Prompt Processing 256ba3006009001200150015361536

OpenBenchmarking.orgTokens Per Second, More Is BetterLlamafile 0.8.16Model: mistral-7b-instruct-v0.2.Q5_K_M - Test: Prompt Processing 2048ba7K14K21K28K35K3276832768

OpenBenchmarking.orgTokens Per Second, More Is BetterLlamafile 0.8.16Model: mistral-7b-instruct-v0.2.Q5_K_M - Test: Prompt Processing 1024ba4K8K12K16K20K1638416384

OpenBenchmarking.orgTokens Per Second, More Is BetterLlamafile 0.8.16Model: mistral-7b-instruct-v0.2.Q5_K_M - Test: Prompt Processing 512ba2K4K6K8K10K81928192

OpenBenchmarking.orgTokens Per Second, More Is BetterLlamafile 0.8.16Model: mistral-7b-instruct-v0.2.Q5_K_M - Test: Prompt Processing 256ba900180027003600450040964096

OpenBenchmarking.orgTokens Per Second, More Is BetterLlamafile 0.8.16Model: TinyLlama-1.1B-Chat-v1.0.BF16 - Test: Prompt Processing 2048ba7K14K21K28K35K3276832768

OpenBenchmarking.orgTokens Per Second, More Is BetterLlamafile 0.8.16Model: TinyLlama-1.1B-Chat-v1.0.BF16 - Test: Prompt Processing 1024ba4K8K12K16K20K1638416384

OpenBenchmarking.orgTokens Per Second, More Is BetterLlamafile 0.8.16Model: TinyLlama-1.1B-Chat-v1.0.BF16 - Test: Prompt Processing 512ba2K4K6K8K10K81928192

OpenBenchmarking.orgTokens Per Second, More Is BetterLlamafile 0.8.16Model: TinyLlama-1.1B-Chat-v1.0.BF16 - Test: Prompt Processing 256ba900180027003600450040964096

OpenBenchmarking.orgTokens Per Second, More Is BetterLlamafile 0.8.16Model: Llama-3.2-3B-Instruct.Q6_K - Test: Prompt Processing 2048ba7K14K21K28K35K3276832768

OpenBenchmarking.orgTokens Per Second, More Is BetterLlamafile 0.8.16Model: Llama-3.2-3B-Instruct.Q6_K - Test: Prompt Processing 1024ba4K8K12K16K20K1638416384

OpenBenchmarking.orgTokens Per Second, More Is BetterLlamafile 0.8.16Model: Llama-3.2-3B-Instruct.Q6_K - Test: Prompt Processing 512ba2K4K6K8K10K81928192

OpenBenchmarking.orgTokens Per Second, More Is BetterLlamafile 0.8.16Model: Llama-3.2-3B-Instruct.Q6_K - Test: Prompt Processing 256ba900180027003600450040964096

PyPerformance

OpenBenchmarking.orgMilliseconds, Fewer Is BetterPyPerformance 1.11Benchmark: json_loadsab369121512.112.1

OpenVINO GenAI

Model: TinyLlama-1.1B-Chat-v1.0 - Device: CPU

a: The test quit with a non-zero exit status. E: RuntimeError: Exception from src/inference/src/cpp/core.cpp:90:

b: The test quit with a non-zero exit status. E: RuntimeError: Exception from src/inference/src/cpp/core.cpp:90:

OpenSSL

Algorithm: RSA4096

a: The test quit with a non-zero exit status.

b: The test quit with a non-zero exit status.

Algorithm: SHA512

a: The test quit with a non-zero exit status.

b: The test quit with a non-zero exit status.

Algorithm: SHA256

a: The test quit with a non-zero exit status.

b: The test quit with a non-zero exit status.

Renaissance

Test: Apache Spark ALS

a: The test quit with a non-zero exit status.

b: The test quit with a non-zero exit status.

c: The test quit with a non-zero exit status.

195 Results Shown

LiteRT:
  Quantized COCO SSD MobileNet v1
  NASNet Mobile
  DeepLab V3
  Mobilenet Quant
CP2K Molecular Dynamics
Llama.cpp
Stockfish
RELION
LiteRT
Llama.cpp
Renaissance
Llama.cpp:
  CPU BLAS - Llama-3.1-Tulu-3-8B-Q8_0 - Prompt Processing 512
  CPU BLAS - Llama-3.1-Tulu-3-8B-Q8_0 - Prompt Processing 1024
LiteRT
Renaissance:
  In-Memory Database Shootout
  Scala Dotty
CP2K Molecular Dynamics
Renaissance
Llama.cpp
Rustls
simdjson
Gcrypt Library
XNNPACK
oneDNN
LiteRT
XNNPACK
simdjson
PyPerformance
Llamafile
XNNPACK
LiteRT
Llama.cpp
Renaissance
XNNPACK
simdjson
Llama.cpp
XNNPACK
Renaissance
XNNPACK:
  FP16MobileNetV1
  FP32MobileNetV3Small
Renaissance:
  Savina Reactors.IO
  Akka Unbalanced Cobwebbed Tree
SVT-AV1:
  Preset 8 - Bosphorus 1080p
  Preset 8 - Bosphorus 4K
oneDNN
Renaissance
oneDNN
Llama.cpp
CP2K Molecular Dynamics
oneDNN
x265
SVT-AV1:
  Preset 13 - Bosphorus 1080p
  Preset 5 - Beauty 4K 10-bit
Timed Eigen Compilation
oneDNN
Rustls
Whisper.cpp
Rustls
ONNX Runtime
Llamafile
Rustls:
  handshake - TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256
  handshake-resume - TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256
Stockfish
POV-Ray
Llamafile:
  mistral-7b-instruct-v0.2.Q5_K_M - Text Generation 128
  Llama-3.2-3B-Instruct.Q6_K - Text Generation 128
Renaissance
SVT-AV1
oneDNN
ONNX Runtime
x265
ONNX Runtime
simdjson
Whisperfile
ONNX Runtime
NAMD
ONNX Runtime
Apache CouchDB
Numpy Benchmark
PyPerformance
ACES DGEMM
SVT-AV1
PyPerformance
XNNPACK
Rustls
XNNPACK
Whisperfile
ONNX Runtime
Renaissance
SVT-AV1
PyPerformance
Rustls
FinanceBench
ONNX Runtime
Apache CouchDB
ONNX Runtime
PyPerformance
SVT-AV1
Apache CouchDB
Llama.cpp
Blender
SVT-AV1
OSPRay
ONNX Runtime
7-Zip Compression
oneDNN
OpenVINO GenAI
SVT-AV1
OSPRay
SVT-AV1
GROMACS
simdjson
Blender
PyPerformance
Llama.cpp
ASTC Encoder:
  Exhaustive
  Thorough
Etcpak
PyPerformance
SVT-AV1
Llama.cpp
Apache CouchDB
ASTC Encoder
Blender
PyPerformance
ASTC Encoder
PyPerformance
OSPRay
Primesieve
PyPerformance
Llamafile
OSPRay
PyPerformance
Llamafile
Blender
Primesieve
Rustls
FinanceBench
PyPerformance
OSPRay
BYTE Unix Benchmark
QuantLib
Y-Cruncher
Llama.cpp
OpenVINO GenAI
Llamafile
Whisperfile
BYTE Unix Benchmark
Llamafile
Blender
OpenSSL:
  AES-128-GCM
  AES-256-GCM
PyPerformance
OSPRay
Whisper.cpp
PyPerformance
OpenVINO GenAI
QuantLib
PyPerformance
7-Zip Compression
Apache CouchDB
Build2
BYTE Unix Benchmark
Y-Cruncher
Whisper.cpp
ONNX Runtime
PyPerformance
NAMD
Apache CouchDB
ONNX Runtime
OpenSSL:
  ChaCha20-Poly1305
  ChaCha20
BYTE Unix Benchmark
ONNX Runtime
Blender
ASTC Encoder
Rustls
Apache Cassandra
Llamafile:
  wizardcoder-python-34b-v1.0.Q6_K - Prompt Processing 2048
  wizardcoder-python-34b-v1.0.Q6_K - Prompt Processing 1024
  wizardcoder-python-34b-v1.0.Q6_K - Prompt Processing 512
  wizardcoder-python-34b-v1.0.Q6_K - Prompt Processing 256
  mistral-7b-instruct-v0.2.Q5_K_M - Prompt Processing 2048
  mistral-7b-instruct-v0.2.Q5_K_M - Prompt Processing 1024
  mistral-7b-instruct-v0.2.Q5_K_M - Prompt Processing 512
  mistral-7b-instruct-v0.2.Q5_K_M - Prompt Processing 256
  TinyLlama-1.1B-Chat-v1.0.BF16 - Prompt Processing 2048
  TinyLlama-1.1B-Chat-v1.0.BF16 - Prompt Processing 1024
  TinyLlama-1.1B-Chat-v1.0.BF16 - Prompt Processing 512
  TinyLlama-1.1B-Chat-v1.0.BF16 - Prompt Processing 256
  Llama-3.2-3B-Instruct.Q6_K - Prompt Processing 2048
  Llama-3.2-3B-Instruct.Q6_K - Prompt Processing 1024
  Llama-3.2-3B-Instruct.Q6_K - Prompt Processing 512
  Llama-3.2-3B-Instruct.Q6_K - Prompt Processing 256
PyPerformance