9950X EOY 2024

AMD Ryzen 9 9950X 16-Core testing with a ASRock X870E Taichi (3.12.AS02 BIOS) and AMD Radeon RX 7800 XT 16GB on Ubuntu 24.04 via the Phoronix Test Suite.

Compare your own system(s) to this result file with the Phoronix Test Suite by running the command: phoronix-test-suite benchmark 2412132-PTS-9950XEOY73
Jump To Table - Results

View

Do Not Show Noisy Results
Do Not Show Results With Incomplete Data
Do Not Show Results With Little Change/Spread
List Notable Results
Show Result Confidence Charts
Allow Limiting Results To Certain Suite(s)

Statistics

Show Overall Harmonic Mean(s)
Show Overall Geometric Mean
Show Wins / Losses Counts (Pie Chart)
Normalize Results
Remove Outliers Before Calculating Averages

Graph Settings

Force Line Graphs Where Applicable
Convert To Scalar Where Applicable
Prefer Vertical Bar Graphs

Multi-Way Comparison

Condense Multi-Option Tests Into Single Result Graphs

Table

Show Detailed System Result Table

Run Management

Highlight
Result
Toggle/Hide
Result
Result
Identifier
Performance Per
Dollar
Date
Run
  Test
  Duration
a
December 13
  2 Hours, 30 Minutes
b
December 13
  2 Hours, 30 Minutes
bb
December 13
  40 Minutes
Invert Behavior (Only Show Selected Data)
  1 Hour, 53 Minutes

Only show results where is faster than
Only show results matching title/arguments (delimit multiple options with a comma):
Do not show results matching title/arguments (delimit multiple options with a comma):


9950X EOY 2024OpenBenchmarking.orgPhoronix Test SuiteAMD Ryzen 9 9950X 16-Core @ 5.75GHz (16 Cores / 32 Threads)ASRock X870E Taichi (3.12.AS02 BIOS)AMD Device 14d82 x 16GB DDR5-6000MT/s F5-6000J2836G16GWestern Digital WD_BLACK SN850X 2000GBAMD Radeon RX 7800 XT 16GB (2200/3000MHz)AMD Navi 31 HDMI/DPDELL U2723QERealtek Device 8126 + MEDIATEK Device 0717Ubuntu 24.046.8.0-50-generic (x86_64)GNOME Shell 46.0X Server 1.21.1.11 + Wayland4.6 Mesa 24.2.0-devel (LLVM 18.1.7 DRM 3.58)GCC 13.3.0ext43840x2160ProcessorMotherboardChipsetMemoryDiskGraphicsAudioMonitorNetworkOSKernelDesktopDisplay ServerOpenGLCompilerFile-SystemScreen Resolution9950X EOY 2024 BenchmarksSystem Logs- Transparent Huge Pages: madvise- --build=x86_64-linux-gnu --disable-vtable-verify --disable-werror --enable-bootstrap --enable-cet --enable-checking=release --enable-clocale=gnu --enable-default-pie --enable-gnu-unique-object --enable-languages=c,ada,c++,go,d,fortran,objc,obj-c++,m2 --enable-libphobos-checking=release --enable-libstdcxx-backtrace --enable-libstdcxx-debug --enable-libstdcxx-time=yes --enable-link-serialization=2 --enable-multiarch --enable-multilib --enable-nls --enable-objc-gc=auto --enable-offload-defaulted --enable-offload-targets=nvptx-none=/build/gcc-13-fG75Ri/gcc-13-13.3.0/debian/tmp-nvptx/usr,amdgcn-amdhsa=/build/gcc-13-fG75Ri/gcc-13-13.3.0/debian/tmp-gcn/usr --enable-plugin --enable-shared --enable-threads=posix --host=x86_64-linux-gnu --program-prefix=x86_64-linux-gnu- --target=x86_64-linux-gnu --with-abi=m64 --with-arch-32=i686 --with-build-config=bootstrap-lto-lean --with-default-libstdcxx-abi=new --with-gcc-major-version-only --with-multilib-list=m32,m64,mx32 --with-target-system-zlib=auto --with-tune=generic --without-cuda-driver -v - Scaling Governor: amd-pstate-epp powersave (EPP: balance_performance) - CPU Microcode: 0xb404023 - OpenJDK Runtime Environment (build 21.0.5+11-Ubuntu-1ubuntu124.04)- Python 3.12.3- gather_data_sampling: Not affected + itlb_multihit: Not affected + l1tf: Not affected + mds: Not affected + meltdown: Not affected + mmio_stale_data: Not affected + reg_file_data_sampling: Not affected + retbleed: Not affected + spec_rstack_overflow: Not affected + spec_store_bypass: Mitigation of SSB disabled via prctl + spectre_v1: Mitigation of usercopy/swapgs barriers and __user pointer sanitization + spectre_v2: Mitigation of Enhanced / Automatic IBRS; IBPB: conditional; STIBP: always-on; RSB filling; PBRSB-eIBRS: Not affected; BHI: Not affected + srbds: Not affected + tsx_async_abort: Not affected

abbbResult OverviewPhoronix Test Suite100%101%102%103%104%RELIONLaghosPalabossrsRAN ProjectSVT-AV1Renaissance

9950X EOY 2024renaissance: Apache Spark Bayesrenaissance: Scala Dottyrenaissance: Rand Forestrenaissance: In-Memory Database Shootoutrenaissance: Apache Spark PageRankrelion: Basic - CPUrenaissance: ALS Movie Lensllama-cpp: CPU BLAS - granite-3.0-3b-a800m-instruct-Q8_0 - Prompt Processing 2048palabos: 100srsran: PDSCH Processor Benchmark, Throughput Threadrenaissance: Savina Reactors.IOrenaissance: Genetic Algorithm Using Jenetics + Futuresllama-cpp: CPU BLAS - Mistral-7B-Instruct-v0.3-Q8_0 - Prompt Processing 2048svt-av1: Preset 8 - Beauty 4K 10-bitlaghos: Triple Point Problemvvenc: Bosphorus 1080p - Fasterllama-cpp: CPU BLAS - granite-3.0-3b-a800m-instruct-Q8_0 - Prompt Processing 512laghos: Sedov Blast Wave, ube_922_hex.meshrenaissance: Akka Unbalanced Cobwebbed Treesvt-av1: Preset 5 - Beauty 4K 10-bitllama-cpp: CPU BLAS - Mistral-7B-Instruct-v0.3-Q8_0 - Prompt Processing 1024openvino: Vehicle Detection FP16 - CPUopenvino: Vehicle Detection FP16 - CPUllama-cpp: CPU BLAS - Llama-3.1-Tulu-3-8B-Q8_0 - Prompt Processing 512llama-cpp: CPU BLAS - Mistral-7B-Instruct-v0.3-Q8_0 - Prompt Processing 512svt-av1: Preset 5 - Bosphorus 1080prenaissance: Finagle HTTP Requestsopenvino: Handwritten English Recognition FP16 - CPUopenvino: Handwritten English Recognition FP16 - CPUopenvino: Road Segmentation ADAS FP16 - CPUopenvino: Road Segmentation ADAS FP16 - CPUsvt-av1: Preset 8 - Bosphorus 1080pblender: Junkshop - CPU-Onlyblender: BMW27 - CPU-Onlyblender: Classroom - CPU-Onlysrsran: PDSCH Processor Benchmark, Throughput Totalblender: Barbershop - CPU-Onlyvvenc: Bosphorus 4K - Fasterx265: Bosphorus 1080popenvino: Face Detection FP16 - CPUopenvino: Face Detection FP16 - CPUllamafile: TinyLlama-1.1B-Chat-v1.0.BF16 - Text Generation 16llama-cpp: CPU BLAS - granite-3.0-3b-a800m-instruct-Q8_0 - Text Generation 128openvino: Machine Translation EN To DE FP16 - CPUvvenc: Bosphorus 4K - Fastopenvino: Machine Translation EN To DE FP16 - CPUprimesieve: 1e12x265: Bosphorus 4Kopenvino: Face Detection Retail FP16-INT8 - CPUopenvino: Handwritten English Recognition FP16-INT8 - CPUblender: Pabellon Barcelona - CPU-Onlysvt-av1: Preset 3 - Bosphorus 1080popenvino: Face Detection Retail FP16-INT8 - CPUsvt-av1: Preset 3 - Beauty 4K 10-bitllama-cpp: CPU BLAS - Llama-3.1-Tulu-3-8B-Q8_0 - Prompt Processing 2048openvino: Person Re-Identification Retail FP16 - CPUopenvino: Person Detection FP16 - CPUopenvino: Handwritten English Recognition FP16-INT8 - CPUopenvino: Person Detection FP16 - CPUsvt-av1: Preset 5 - Bosphorus 4Kastcenc: Thoroughopenvino: Vehicle Detection FP16-INT8 - CPUopenvino-genai: Gemma-7b-int4-ov - CPUopenvino: Vehicle Detection FP16-INT8 - CPUopenvino: Face Detection Retail FP16 - CPUopenvino: Person Re-Identification Retail FP16 - CPUopenvino: Age Gender Recognition Retail 0013 FP16-INT8 - CPUvvenc: Bosphorus 1080p - Fastopenvino: Person Detection FP32 - CPUopenvino: Person Detection FP32 - CPUastcenc: Fastllama-cpp: CPU BLAS - Llama-3.1-Tulu-3-8B-Q8_0 - Prompt Processing 1024blender: Fishy Cat - CPU-Onlyopenvino: Road Segmentation ADAS FP16-INT8 - CPUllama-cpp: CPU BLAS - granite-3.0-3b-a800m-instruct-Q8_0 - Prompt Processing 1024svt-av1: Preset 13 - Bosphorus 4Kmt-dgemm: Sustained Floating-Point Ratellamafile: mistral-7b-instruct-v0.2.Q5_K_M - Text Generation 128openvino: Road Segmentation ADAS FP16-INT8 - CPUpalabos: 500openvino: Weld Porosity Detection FP16-INT8 - CPUopenvino: Face Detection FP16-INT8 - CPUllamafile: Llama-3.2-3B-Instruct.Q6_K - Text Generation 16astcenc: Exhaustivellamafile: Llama-3.2-3B-Instruct.Q6_K - Text Generation 128openvino: Face Detection FP16-INT8 - CPUllamafile: mistral-7b-instruct-v0.2.Q5_K_M - Text Generation 16openvino: Weld Porosity Detection FP16 - CPUsvt-av1: Preset 8 - Bosphorus 4Kastcenc: Mediumsvt-av1: Preset 3 - Bosphorus 4Kpalabos: 400llama-cpp: CPU BLAS - Mistral-7B-Instruct-v0.3-Q8_0 - Text Generation 128openvino: Weld Porosity Detection FP16 - CPUopenvino-genai: TinyLlama-1.1B-Chat-v1.0 - CPUopenvino-genai: Phi-3-mini-128k-instruct-int4-ov - CPUastcenc: Very Thoroughopenvino: Age Gender Recognition Retail 0013 FP16 - CPUopenvino: Noise Suppression Poconet-Like FP16 - CPUsrsran: PUSCH Processor Benchmark, Throughput Threadopenvino: Person Vehicle Bike Detection FP16 - CPUopenvino-genai: Falcon-7b-instruct-int4-ov - CPUrenaissance: Gaussian Mixture Modelopenvino: Noise Suppression Poconet-Like FP16 - CPUllamafile: TinyLlama-1.1B-Chat-v1.0.BF16 - Text Generation 128srsran: PUSCH Processor Benchmark, Throughput Totalprimesieve: 1e13svt-av1: Preset 13 - Beauty 4K 10-bitsvt-av1: Preset 13 - Bosphorus 1080pllamafile: wizardcoder-python-34b-v1.0.Q6_K - Text Generation 128llamafile: wizardcoder-python-34b-v1.0.Q6_K - Text Generation 16llama-cpp: CPU BLAS - Llama-3.1-Tulu-3-8B-Q8_0 - Text Generation 128openvino: Age Gender Recognition Retail 0013 FP16-INT8 - CPUopenvino: Age Gender Recognition Retail 0013 FP16 - CPUopenvino: Person Vehicle Bike Detection FP16 - CPUopenvino: Weld Porosity Detection FP16-INT8 - CPUopenvino: Face Detection Retail FP16 - CPUopenvino-genai: Phi-3-mini-128k-instruct-int4-ov - CPU - Time Per Output Tokenopenvino-genai: Phi-3-mini-128k-instruct-int4-ov - CPU - Time To First Tokenopenvino-genai: Falcon-7b-instruct-int4-ov - CPU - Time Per Output Tokenopenvino-genai: Falcon-7b-instruct-int4-ov - CPU - Time To First Tokenopenvino-genai: TinyLlama-1.1B-Chat-v1.0 - CPU - Time Per Output Tokenopenvino-genai: TinyLlama-1.1B-Chat-v1.0 - CPU - Time To First Tokenopenvino-genai: Gemma-7b-int4-ov - CPU - Time Per Output Tokenopenvino-genai: Gemma-7b-int4-ov - CPU - Time To First Tokenrenaissance: Apache Spark ALSabbb438.8345.7326.13328.81914.9711.397965.4400108.7852303.14289.1699.2132.5714.148238.8062.474455.75256.493782.47.459141.858.87898.7146.74142.82128.8191157.0908.5517.53373.6921.36427.82761.8646.16129.6244425.8461.323.179141.8223.23343.5531.0264.94187.1410.64142.695.93141.132.14990.07145.9137.7267122.611.681133.62.64109.8616.0972.7344.45923.05693.1312.692522.844839.872968.3874229.1729.67572.53110.17445.7994141.6165.0613.22467.93258.8211782.7754414.22603.24127.2064524.64169.9525.351.946626.7147.0213.516.88129.411175.811412.238123.459.772303.9830.7825.233.166155602.5313.81317.72127.0816.73017.11145.3732.84339174.52419.5031018.42.672.369.290.190.253.733.441.5539.6343.1659.8965.1132.4933.5978.7983.05409.4341.2345.03329.01859.5729.7818191.4388.99110.0672332.24190.5691.4135.5113.86237.2561.331463.68257.213821.47.356139.958.99886.76144.99144.46128.4261169.6899.2217.71377.3121.16430.1162.3746.54130.6844552.3464.9923.002140.7523.4341.2231.1965.27186.210.58842.95.9641.332.13985.48146.5837.787153.861.688134.152.65109.4516.157344.47923.1353.1212.732530.774825.112959.3474013.3329.59372.73109.87446.9276141.2765.2113.25466.88259.3871786.58455114.25602.01127.2774516.85170.2225.311.943626.6746.9513.496.89129.566176.048512.234123.3169.762301.6830.8125.253.163655559.6513.8317.52128.3616.713018.11145.8132.833391.874.5419.4991018.4622.672.369.290.190.253.733.441.5539.6143.2959.8465.1232.4633.4778.5883.21410.2362.5330.53478.31853.2733.9898155.8107.162276.94257.9683.8234.27253.083842.9129.8721165.0431.41144781.937.89644.613259.275127.021129.59712.222123.461317.73016.93391.21018.587OpenBenchmarking.org

Renaissance

OpenBenchmarking.orgms, Fewer Is BetterRenaissance 0.16Test: Apache Spark Bayesbbba100200300400500410.2409.4438.8MIN: 400.29 / MAX: 445.18MIN: 399.38 / MAX: 440.5MIN: 398.56 / MAX: 461.45

OpenBenchmarking.orgms, Fewer Is BetterRenaissance 0.16Test: Scala Dottybbba80160240320400362.5341.2345.7MIN: 309.18 / MAX: 514.62MIN: 309.54 / MAX: 551.22MIN: 306.83 / MAX: 560.4

OpenBenchmarking.orgms, Fewer Is BetterRenaissance 0.16Test: Random Forestbbba70140210280350330.5345.0326.1MIN: 300.54 / MAX: 379.53MIN: 293.77 / MAX: 390.67MIN: 297.46 / MAX: 381.81

OpenBenchmarking.orgms, Fewer Is BetterRenaissance 0.16Test: In-Memory Database Shootoutbbba70014002100280035003478.33329.03328.8MIN: 2469.88 / MAX: 3478.31MIN: 2393.6MIN: 2448.19 / MAX: 3328.84

OpenBenchmarking.orgms, Fewer Is BetterRenaissance 0.16Test: Apache Spark PageRankbbba4008001200160020001853.21859.51914.9MIN: 1402.26 / MAX: 1853.21MIN: 1396.15 / MAX: 1859.51MIN: 1449.57 / MAX: 1914.91

RELION

OpenBenchmarking.orgSeconds, Fewer Is BetterRELION 5.0Test: Basic - Device: CPUbbba160320480640800733.99729.78711.391. (CXX) g++ options: -fPIC -std=c++14 -fopenmp -O3 -rdynamic -lfftw3f -lfftw3 -ldl -ltiff -lpng -ljpeg -lmpi_cxx -lmpi

Renaissance

OpenBenchmarking.orgms, Fewer Is BetterRenaissance 0.16Test: ALS Movie Lensbbba2K4K6K8K10K8155.88191.47965.4MIN: 7829.44 / MAX: 8248.83MIN: 7897.59 / MAX: 8229.51MIN: 7733.74 / MAX: 8117.3

Llama.cpp

OpenBenchmarking.orgTokens Per Second, More Is BetterLlama.cpp b4154Backend: CPU BLAS - Model: granite-3.0-3b-a800m-instruct-Q8_0 - Test: Prompt Processing 2048ba90180270360450388.99400.001. (CXX) g++ options: -std=c++11 -fPIC -O3 -pthread -fopenmp -march=native -mtune=native -lopenblas

Palabos

OpenBenchmarking.orgMega Site Updates Per Second, More Is BetterPalabos 2.3Grid Size: 100bbba20406080100107.16110.07108.791. (CXX) g++ options: -std=c++17 -pedantic -O3 -rdynamic -lcrypto -lcurl -lsz -lz -ldl -lm

srsRAN Project

OpenBenchmarking.orgMbps, More Is BettersrsRAN Project 24.10Test: PDSCH Processor Benchmark, Throughput Threadbbba50010001500200025002276.92332.22303.11. (CXX) g++ options: -O3 -march=native -mtune=generic -fno-trapping-math -fno-math-errno -ldl

Renaissance

OpenBenchmarking.orgms, Fewer Is BetterRenaissance 0.16Test: Savina Reactors.IObbba90018002700360045004257.94190.54289.1MAX: 5711.68MIN: 4190.46 / MAX: 5753.64MAX: 5880.97

OpenBenchmarking.orgms, Fewer Is BetterRenaissance 0.16Test: Genetic Algorithm Using Jenetics + Futuresbbba150300450600750683.8691.4699.2MIN: 670.11 / MAX: 694.81MIN: 679.67 / MAX: 702.85MIN: 686.61 / MAX: 709.54

Llama.cpp

OpenBenchmarking.orgTokens Per Second, More Is BetterLlama.cpp b4154Backend: CPU BLAS - Model: Mistral-7B-Instruct-v0.3-Q8_0 - Test: Prompt Processing 2048ba306090120150135.51132.571. (CXX) g++ options: -std=c++11 -fPIC -O3 -pthread -fopenmp -march=native -mtune=native -lopenblas

SVT-AV1

OpenBenchmarking.orgFrames Per Second, More Is BetterSVT-AV1 2.3Encoder Mode: Preset 8 - Input: Beauty 4K 10-bitba4812162013.8614.151. (CXX) g++ options: -march=native -mno-avx -mavx2 -mavx512f -mavx512bw -mavx512dq

Laghos

OpenBenchmarking.orgMajor Kernels Total Rate, More Is BetterLaghos 3.1Test: Triple Point Problembbba50100150200250234.27237.25238.801. (CXX) g++ options: -O3 -std=c++11 -lmfem -lHYPRE -lmetis -lrt -lmpi_cxx -lmpi

VVenC

OpenBenchmarking.orgFrames Per Second, More Is BetterVVenC 1.13Video Input: Bosphorus 1080p - Video Preset: Fasterba142842567061.3362.471. (CXX) g++ options: -O3 -flto=auto -fno-fat-lto-objects

Llama.cpp

OpenBenchmarking.orgTokens Per Second, More Is BetterLlama.cpp b4154Backend: CPU BLAS - Model: granite-3.0-3b-a800m-instruct-Q8_0 - Test: Prompt Processing 512ba100200300400500463.68455.751. (CXX) g++ options: -std=c++11 -fPIC -O3 -pthread -fopenmp -march=native -mtune=native -lopenblas

Laghos

OpenBenchmarking.orgMajor Kernels Total Rate, More Is BetterLaghos 3.1Test: Sedov Blast Wave, ube_922_hex.meshbbba60120180240300253.08257.21256.491. (CXX) g++ options: -O3 -std=c++11 -lmfem -lHYPRE -lmetis -lrt -lmpi_cxx -lmpi

Renaissance

OpenBenchmarking.orgms, Fewer Is BetterRenaissance 0.16Test: Akka Unbalanced Cobwebbed Treebbba80016002400320040003842.93821.43782.4MIN: 3842.86 / MAX: 4792.11MIN: 3821.38 / MAX: 4731.87MIN: 3782.36 / MAX: 4681.87

SVT-AV1

OpenBenchmarking.orgFrames Per Second, More Is BetterSVT-AV1 2.3Encoder Mode: Preset 5 - Input: Beauty 4K 10-bitba2468107.3567.4591. (CXX) g++ options: -march=native -mno-avx -mavx2 -mavx512f -mavx512bw -mavx512dq

Llama.cpp

OpenBenchmarking.orgTokens Per Second, More Is BetterLlama.cpp b4154Backend: CPU BLAS - Model: Mistral-7B-Instruct-v0.3-Q8_0 - Test: Prompt Processing 1024ba306090120150139.95141.851. (CXX) g++ options: -std=c++11 -fPIC -O3 -pthread -fopenmp -march=native -mtune=native -lopenblas

OpenVINO

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2024.5Model: Vehicle Detection FP16 - Device: CPUba36912158.998.87MIN: 3.62 / MAX: 15.22MIN: 3.58 / MAX: 14.771. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl -lstdc++fs

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2024.5Model: Vehicle Detection FP16 - Device: CPUba2004006008001000886.76898.701. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl -lstdc++fs

Llama.cpp

OpenBenchmarking.orgTokens Per Second, More Is BetterLlama.cpp b4154Backend: CPU BLAS - Model: Llama-3.1-Tulu-3-8B-Q8_0 - Test: Prompt Processing 512ba306090120150144.99146.741. (CXX) g++ options: -std=c++11 -fPIC -O3 -pthread -fopenmp -march=native -mtune=native -lopenblas

OpenBenchmarking.orgTokens Per Second, More Is BetterLlama.cpp b4154Backend: CPU BLAS - Model: Mistral-7B-Instruct-v0.3-Q8_0 - Test: Prompt Processing 512ba306090120150144.46142.821. (CXX) g++ options: -std=c++11 -fPIC -O3 -pthread -fopenmp -march=native -mtune=native -lopenblas

SVT-AV1

OpenBenchmarking.orgFrames Per Second, More Is BetterSVT-AV1 2.3Encoder Mode: Preset 5 - Input: Bosphorus 1080pbbba306090120150129.87128.43128.821. (CXX) g++ options: -march=native -mno-avx -mavx2 -mavx512f -mavx512bw -mavx512dq

Renaissance

OpenBenchmarking.orgms, Fewer Is BetterRenaissance 0.16Test: Finagle HTTP Requestsbbba300600900120015001165.01169.61157.0MIN: 1141.47 / MAX: 1186.1MIN: 1125.98 / MAX: 1186.46MIN: 1106.11 / MAX: 1180.82

OpenVINO

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2024.5Model: Handwritten English Recognition FP16 - Device: CPUba2004006008001000899.22908.551. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl -lstdc++fs

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2024.5Model: Handwritten English Recognition FP16 - Device: CPUba4812162017.7117.53MIN: 13.82 / MAX: 26.83MIN: 10.08 / MAX: 26.141. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl -lstdc++fs

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2024.5Model: Road Segmentation ADAS FP16 - Device: CPUba80160240320400377.31373.691. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl -lstdc++fs

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2024.5Model: Road Segmentation ADAS FP16 - Device: CPUba51015202521.1621.36MIN: 8.31 / MAX: 32.46MIN: 13.18 / MAX: 32.371. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl -lstdc++fs

SVT-AV1

OpenBenchmarking.orgFrames Per Second, More Is BetterSVT-AV1 2.3Encoder Mode: Preset 8 - Input: Bosphorus 1080pbbba90180270360450431.41430.11427.831. (CXX) g++ options: -march=native -mno-avx -mavx2 -mavx512f -mavx512bw -mavx512dq

Blender

OpenBenchmarking.orgSeconds, Fewer Is BetterBlender 4.3Blend File: Junkshop - Compute: CPU-Onlyba142842567062.3761.86

OpenBenchmarking.orgSeconds, Fewer Is BetterBlender 4.3Blend File: BMW27 - Compute: CPU-Onlyba112233445546.5446.16

OpenBenchmarking.orgSeconds, Fewer Is BetterBlender 4.3Blend File: Classroom - Compute: CPU-Onlyba306090120150130.68129.62

srsRAN Project

OpenBenchmarking.orgMbps, More Is BettersrsRAN Project 24.10Test: PDSCH Processor Benchmark, Throughput Totalbbba10K20K30K40K50K44781.944552.344425.81. (CXX) g++ options: -O3 -march=native -mtune=generic -fno-trapping-math -fno-math-errno -ldl

Blender

OpenBenchmarking.orgSeconds, Fewer Is BetterBlender 4.3Blend File: Barbershop - Compute: CPU-Onlyba100200300400500464.99461.30

VVenC

OpenBenchmarking.orgFrames Per Second, More Is BetterVVenC 1.13Video Input: Bosphorus 4K - Video Preset: Fasterba61218243023.0023.181. (CXX) g++ options: -O3 -flto=auto -fno-fat-lto-objects

x265

OpenBenchmarking.orgFrames Per Second, More Is Betterx265 4.1Video Input: Bosphorus 1080pba306090120150140.75141.821. (CXX) g++ options: -O3 -rdynamic -lpthread -lrt -ldl -lnuma

OpenVINO

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2024.5Model: Face Detection FP16 - Device: CPUba61218243023.4023.231. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl -lstdc++fs

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2024.5Model: Face Detection FP16 - Device: CPUba70140210280350341.22343.55MIN: 168.69 / MAX: 373.62MIN: 320.73 / MAX: 376.331. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl -lstdc++fs

Llamafile

OpenBenchmarking.orgTokens Per Second, More Is BetterLlamafile 0.8.16Model: TinyLlama-1.1B-Chat-v1.0.BF16 - Test: Text Generation 16ba71421283531.1931.02

Llama.cpp

OpenBenchmarking.orgTokens Per Second, More Is BetterLlama.cpp b4154Backend: CPU BLAS - Model: granite-3.0-3b-a800m-instruct-Q8_0 - Test: Text Generation 128ba153045607565.2764.941. (CXX) g++ options: -std=c++11 -fPIC -O3 -pthread -fopenmp -march=native -mtune=native -lopenblas

OpenVINO

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2024.5Model: Machine Translation EN To DE FP16 - Device: CPUba4080120160200186.20187.141. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl -lstdc++fs

VVenC

OpenBenchmarking.orgFrames Per Second, More Is BetterVVenC 1.13Video Input: Bosphorus 4K - Video Preset: Fastba369121510.5910.641. (CXX) g++ options: -O3 -flto=auto -fno-fat-lto-objects

OpenVINO

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2024.5Model: Machine Translation EN To DE FP16 - Device: CPUba102030405042.9042.69MIN: 22.59 / MAX: 57.04MIN: 23.42 / MAX: 55.141. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl -lstdc++fs

Primesieve

OpenBenchmarking.orgSeconds, Fewer Is BetterPrimesieve 12.6Length: 1e12ba1.3412.6824.0235.3646.7055.9605.9311. (CXX) g++ options: -O3

x265

OpenBenchmarking.orgFrames Per Second, More Is Betterx265 4.1Video Input: Bosphorus 4Kba91827364541.3341.131. (CXX) g++ options: -O3 -rdynamic -lpthread -lrt -ldl -lnuma

OpenVINO

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2024.5Model: Face Detection Retail FP16-INT8 - Device: CPUba0.48150.9631.44451.9262.40752.132.14MIN: 1.17 / MAX: 6.16MIN: 1.17 / MAX: 4.441. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl -lstdc++fs

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2024.5Model: Handwritten English Recognition FP16-INT8 - Device: CPUba2004006008001000985.48990.071. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl -lstdc++fs

Blender

OpenBenchmarking.orgSeconds, Fewer Is BetterBlender 4.3Blend File: Pabellon Barcelona - Compute: CPU-Onlyba306090120150146.58145.91

SVT-AV1

OpenBenchmarking.orgFrames Per Second, More Is BetterSVT-AV1 2.3Encoder Mode: Preset 3 - Input: Bosphorus 1080pbbba91827364537.9037.7837.731. (CXX) g++ options: -march=native -mno-avx -mavx2 -mavx512f -mavx512bw -mavx512dq

OpenVINO

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2024.5Model: Face Detection Retail FP16-INT8 - Device: CPUba150030004500600075007153.867122.611. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl -lstdc++fs

SVT-AV1

OpenBenchmarking.orgFrames Per Second, More Is BetterSVT-AV1 2.3Encoder Mode: Preset 3 - Input: Beauty 4K 10-bitba0.37980.75961.13941.51921.8991.6881.6811. (CXX) g++ options: -march=native -mno-avx -mavx2 -mavx512f -mavx512bw -mavx512dq

Llama.cpp

OpenBenchmarking.orgTokens Per Second, More Is BetterLlama.cpp b4154Backend: CPU BLAS - Model: Llama-3.1-Tulu-3-8B-Q8_0 - Test: Prompt Processing 2048ba306090120150134.15133.601. (CXX) g++ options: -std=c++11 -fPIC -O3 -pthread -fopenmp -march=native -mtune=native -lopenblas

OpenVINO

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2024.5Model: Person Re-Identification Retail FP16 - Device: CPUba0.59631.19261.78892.38522.98152.652.64MIN: 1.45 / MAX: 6.9MIN: 1.5 / MAX: 5.661. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl -lstdc++fs

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2024.5Model: Person Detection FP16 - Device: CPUba20406080100109.45109.861. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl -lstdc++fs

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2024.5Model: Handwritten English Recognition FP16-INT8 - Device: CPUba4812162016.1516.09MIN: 10.42 / MAX: 21.28MIN: 10.41 / MAX: 26.191. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl -lstdc++fs

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2024.5Model: Person Detection FP16 - Device: CPUba163248648073.0072.73MIN: 42.63 / MAX: 99.34MIN: 44.19 / MAX: 98.241. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl -lstdc++fs

SVT-AV1

OpenBenchmarking.orgFrames Per Second, More Is BetterSVT-AV1 2.3Encoder Mode: Preset 5 - Input: Bosphorus 4Kbbba102030405044.6144.4844.461. (CXX) g++ options: -march=native -mno-avx -mavx2 -mavx512f -mavx512bw -mavx512dq

ASTC Encoder

OpenBenchmarking.orgMT/s, More Is BetterASTC Encoder 5.0Preset: Thoroughba61218243023.1423.061. (CXX) g++ options: -O3 -flto -pthread

OpenVINO

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2024.5Model: Vehicle Detection FP16-INT8 - Device: CPUba0.70431.40862.11292.81723.52153.123.13MIN: 1.77 / MAX: 14.15MIN: 1.75 / MAX: 13.21. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl -lstdc++fs

OpenVINO GenAI

OpenBenchmarking.orgtokens/s, More Is BetterOpenVINO GenAI 2024.5Model: Gemma-7b-int4-ov - Device: CPUba369121512.7312.69

OpenVINO

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2024.5Model: Vehicle Detection FP16-INT8 - Device: CPUba50010001500200025002530.772522.841. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl -lstdc++fs

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2024.5Model: Face Detection Retail FP16 - Device: CPUba100020003000400050004825.114839.871. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl -lstdc++fs

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2024.5Model: Person Re-Identification Retail FP16 - Device: CPUba60012001800240030002959.342968.381. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl -lstdc++fs

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2024.5Model: Age Gender Recognition Retail 0013 FP16-INT8 - Device: CPUba16K32K48K64K80K74013.3374229.171. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl -lstdc++fs

VVenC

OpenBenchmarking.orgFrames Per Second, More Is BetterVVenC 1.13Video Input: Bosphorus 1080p - Video Preset: Fastba71421283529.5929.681. (CXX) g++ options: -O3 -flto=auto -fno-fat-lto-objects

OpenVINO

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2024.5Model: Person Detection FP32 - Device: CPUba163248648072.7372.53MIN: 57.9 / MAX: 97.75MIN: 52.33 / MAX: 98.911. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl -lstdc++fs

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2024.5Model: Person Detection FP32 - Device: CPUba20406080100109.87110.171. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl -lstdc++fs

ASTC Encoder

OpenBenchmarking.orgMT/s, More Is BetterASTC Encoder 5.0Preset: Fastba100200300400500446.93445.801. (CXX) g++ options: -O3 -flto -pthread

Llama.cpp

OpenBenchmarking.orgTokens Per Second, More Is BetterLlama.cpp b4154Backend: CPU BLAS - Model: Llama-3.1-Tulu-3-8B-Q8_0 - Test: Prompt Processing 1024ba306090120150141.27141.611. (CXX) g++ options: -std=c++11 -fPIC -O3 -pthread -fopenmp -march=native -mtune=native -lopenblas

Blender

OpenBenchmarking.orgSeconds, Fewer Is BetterBlender 4.3Blend File: Fishy Cat - Compute: CPU-Onlyba153045607565.2165.06

OpenVINO

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2024.5Model: Road Segmentation ADAS FP16-INT8 - Device: CPUba369121513.2513.22MIN: 7.29 / MAX: 18.74MIN: 5.84 / MAX: 19.261. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl -lstdc++fs

Llama.cpp

OpenBenchmarking.orgTokens Per Second, More Is BetterLlama.cpp b4154Backend: CPU BLAS - Model: granite-3.0-3b-a800m-instruct-Q8_0 - Test: Prompt Processing 1024ba100200300400500466.88467.931. (CXX) g++ options: -std=c++11 -fPIC -O3 -pthread -fopenmp -march=native -mtune=native -lopenblas

SVT-AV1

OpenBenchmarking.orgFrames Per Second, More Is BetterSVT-AV1 2.3Encoder Mode: Preset 13 - Input: Bosphorus 4Kbbba60120180240300259.28259.39258.821. (CXX) g++ options: -march=native -mno-avx -mavx2 -mavx512f -mavx512bw -mavx512dq

ACES DGEMM

OpenBenchmarking.orgGFLOP/s, More Is BetterACES DGEMM 1.0Sustained Floating-Point Rateba4008001200160020001786.581782.781. (CC) gcc options: -ffast-math -mavx2 -O3 -fopenmp -lopenblas

Llamafile

OpenBenchmarking.orgTokens Per Second, More Is BetterLlamafile 0.8.16Model: mistral-7b-instruct-v0.2.Q5_K_M - Test: Text Generation 128ba4812162014.2514.22

OpenVINO

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2024.5Model: Road Segmentation ADAS FP16-INT8 - Device: CPUba130260390520650602.01603.241. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl -lstdc++fs

Palabos

OpenBenchmarking.orgMega Site Updates Per Second, More Is BetterPalabos 2.3Grid Size: 500bbba306090120150127.02127.28127.211. (CXX) g++ options: -std=c++17 -pedantic -O3 -rdynamic -lcrypto -lcurl -lsz -lz -ldl -lm

OpenVINO

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2024.5Model: Weld Porosity Detection FP16-INT8 - Device: CPUba100020003000400050004516.854524.641. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl -lstdc++fs

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2024.5Model: Face Detection FP16-INT8 - Device: CPUba4080120160200170.22169.95MIN: 154.83 / MAX: 179.25MIN: 160.42 / MAX: 178.71. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl -lstdc++fs

Llamafile

OpenBenchmarking.orgTokens Per Second, More Is BetterLlamafile 0.8.16Model: Llama-3.2-3B-Instruct.Q6_K - Test: Text Generation 16ba61218243025.3125.35

ASTC Encoder

OpenBenchmarking.orgMT/s, More Is BetterASTC Encoder 5.0Preset: Exhaustiveba0.4380.8761.3141.7522.191.94361.94661. (CXX) g++ options: -O3 -flto -pthread

Llamafile

OpenBenchmarking.orgTokens Per Second, More Is BetterLlamafile 0.8.16Model: Llama-3.2-3B-Instruct.Q6_K - Test: Text Generation 128ba61218243026.6726.71

OpenVINO

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2024.5Model: Face Detection FP16-INT8 - Device: CPUba112233445546.9547.021. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl -lstdc++fs

Llamafile

OpenBenchmarking.orgTokens Per Second, More Is BetterLlamafile 0.8.16Model: mistral-7b-instruct-v0.2.Q5_K_M - Test: Text Generation 16ba369121513.4913.51

OpenVINO

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2024.5Model: Weld Porosity Detection FP16 - Device: CPUba2468106.896.88MIN: 3.41 / MAX: 13.56MIN: 3.78 / MAX: 10.741. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl -lstdc++fs

SVT-AV1

OpenBenchmarking.orgFrames Per Second, More Is BetterSVT-AV1 2.3Encoder Mode: Preset 8 - Input: Bosphorus 4Kbbba306090120150129.60129.57129.411. (CXX) g++ options: -march=native -mno-avx -mavx2 -mavx512f -mavx512bw -mavx512dq

ASTC Encoder

OpenBenchmarking.orgMT/s, More Is BetterASTC Encoder 5.0Preset: Mediumba4080120160200176.05175.811. (CXX) g++ options: -O3 -flto -pthread

SVT-AV1

OpenBenchmarking.orgFrames Per Second, More Is BetterSVT-AV1 2.3Encoder Mode: Preset 3 - Input: Bosphorus 4Kbbba369121512.2212.2312.241. (CXX) g++ options: -march=native -mno-avx -mavx2 -mavx512f -mavx512bw -mavx512dq

Palabos

OpenBenchmarking.orgMega Site Updates Per Second, More Is BetterPalabos 2.3Grid Size: 400bbba306090120150123.46123.32123.451. (CXX) g++ options: -std=c++17 -pedantic -O3 -rdynamic -lcrypto -lcurl -lsz -lz -ldl -lm

Llama.cpp

OpenBenchmarking.orgTokens Per Second, More Is BetterLlama.cpp b4154Backend: CPU BLAS - Model: Mistral-7B-Instruct-v0.3-Q8_0 - Test: Text Generation 128ba36912159.769.771. (CXX) g++ options: -std=c++11 -fPIC -O3 -pthread -fopenmp -march=native -mtune=native -lopenblas

OpenVINO

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2024.5Model: Weld Porosity Detection FP16 - Device: CPUba50010001500200025002301.682303.981. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl -lstdc++fs

OpenVINO GenAI

OpenBenchmarking.orgtokens/s, More Is BetterOpenVINO GenAI 2024.5Model: TinyLlama-1.1B-Chat-v1.0 - Device: CPUba71421283530.8130.78

OpenBenchmarking.orgtokens/s, More Is BetterOpenVINO GenAI 2024.5Model: Phi-3-mini-128k-instruct-int4-ov - Device: CPUba61218243025.2525.23

ASTC Encoder

OpenBenchmarking.orgMT/s, More Is BetterASTC Encoder 5.0Preset: Very Thoroughba0.71241.42482.13722.84963.5623.16363.16611. (CXX) g++ options: -O3 -flto -pthread

OpenVINO

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2024.5Model: Age Gender Recognition Retail 0013 FP16 - Device: CPUba12K24K36K48K60K55559.6555602.531. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl -lstdc++fs

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2024.5Model: Noise Suppression Poconet-Like FP16 - Device: CPUba4812162013.8013.81MIN: 6.88 / MAX: 21.43MIN: 6.02 / MAX: 17.661. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl -lstdc++fs

srsRAN Project

OpenBenchmarking.orgMbps, More Is BettersrsRAN Project 24.10Test: PUSCH Processor Benchmark, Throughput Threadbbba70140210280350317.7317.5317.71. (CXX) g++ options: -O3 -march=native -mtune=generic -fno-trapping-math -fno-math-errno -ldl

OpenVINO

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2024.5Model: Person Vehicle Bike Detection FP16 - Device: CPUba50010001500200025002128.362127.081. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl -lstdc++fs

OpenVINO GenAI

OpenBenchmarking.orgtokens/s, More Is BetterOpenVINO GenAI 2024.5Model: Falcon-7b-instruct-int4-ov - Device: CPUba4812162016.7116.70

Renaissance

OpenBenchmarking.orgms, Fewer Is BetterRenaissance 0.16Test: Gaussian Mixture Modelbbba60012001800240030003016.93018.13017.1MIN: 2266.96MIN: 2269.89 / MAX: 3018.14MIN: 2281.04

OpenVINO

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2024.5Model: Noise Suppression Poconet-Like FP16 - Device: CPUba20040060080010001145.811145.371. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl -lstdc++fs

Llamafile

OpenBenchmarking.orgTokens Per Second, More Is BetterLlamafile 0.8.16Model: TinyLlama-1.1B-Chat-v1.0.BF16 - Test: Text Generation 128ba81624324032.8332.84

srsRAN Project

OpenBenchmarking.orgMbps, More Is BettersrsRAN Project 24.10Test: PUSCH Processor Benchmark, Throughput Totalbbba70014002100280035003391.23391.83391.01. (CXX) g++ options: -O3 -march=native -mtune=generic -fno-trapping-math -fno-math-errno -ldl

Primesieve

OpenBenchmarking.orgSeconds, Fewer Is BetterPrimesieve 12.6Length: 1e13ba2040608010074.5474.521. (CXX) g++ options: -O3

SVT-AV1

OpenBenchmarking.orgFrames Per Second, More Is BetterSVT-AV1 2.3Encoder Mode: Preset 13 - Input: Beauty 4K 10-bitba51015202519.5019.501. (CXX) g++ options: -march=native -mno-avx -mavx2 -mavx512f -mavx512bw -mavx512dq

OpenBenchmarking.orgFrames Per Second, More Is BetterSVT-AV1 2.3Encoder Mode: Preset 13 - Input: Bosphorus 1080pbbba20040060080010001018.591018.461018.401. (CXX) g++ options: -march=native -mno-avx -mavx2 -mavx512f -mavx512bw -mavx512dq

Llamafile

OpenBenchmarking.orgTokens Per Second, More Is BetterLlamafile 0.8.16Model: wizardcoder-python-34b-v1.0.Q6_K - Test: Text Generation 128ba0.60081.20161.80242.40323.0042.672.67

OpenBenchmarking.orgTokens Per Second, More Is BetterLlamafile 0.8.16Model: wizardcoder-python-34b-v1.0.Q6_K - Test: Text Generation 16ba0.5311.0621.5932.1242.6552.362.36

Llama.cpp

OpenBenchmarking.orgTokens Per Second, More Is BetterLlama.cpp b4154Backend: CPU BLAS - Model: Llama-3.1-Tulu-3-8B-Q8_0 - Test: Text Generation 128ba36912159.299.291. (CXX) g++ options: -std=c++11 -fPIC -O3 -pthread -fopenmp -march=native -mtune=native -lopenblas

OpenVINO

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2024.5Model: Age Gender Recognition Retail 0013 FP16-INT8 - Device: CPUba0.04280.08560.12840.17120.2140.190.19MIN: 0.11 / MAX: 7.02MIN: 0.11 / MAX: 6.861. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl -lstdc++fs

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2024.5Model: Age Gender Recognition Retail 0013 FP16 - Device: CPUba0.05630.11260.16890.22520.28150.250.25MIN: 0.13 / MAX: 3.61MIN: 0.13 / MAX: 3.511. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl -lstdc++fs

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2024.5Model: Person Vehicle Bike Detection FP16 - Device: CPUba0.83931.67862.51793.35724.19653.733.73MIN: 2.51 / MAX: 6.3MIN: 2.56 / MAX: 6.411. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl -lstdc++fs

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2024.5Model: Weld Porosity Detection FP16-INT8 - Device: CPUba0.7741.5482.3223.0963.873.443.44MIN: 1.67 / MAX: 6.9MIN: 1.68 / MAX: 8.021. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl -lstdc++fs

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2024.5Model: Face Detection Retail FP16 - Device: CPUba0.34880.69761.04641.39521.7441.551.55MIN: 0.81 / MAX: 5.04MIN: 0.76 / MAX: 4.551. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl -lstdc++fs

Timed Eigen Compilation

Time To Compile

a: The test quit with a non-zero exit status. E: LLVM ERROR: inconsistency in registered CommandLine options

b: The test quit with a non-zero exit status. E: LLVM ERROR: inconsistency in registered CommandLine options

Renaissance

Test: Apache Spark ALS

a: The test quit with a non-zero exit status.

b: The test quit with a non-zero exit status.

bb: The test quit with a non-zero exit status.

117 Results Shown

Renaissance:
  Apache Spark Bayes
  Scala Dotty
  Rand Forest
  In-Memory Database Shootout
  Apache Spark PageRank
RELION
Renaissance
Llama.cpp
Palabos
srsRAN Project
Renaissance:
  Savina Reactors.IO
  Genetic Algorithm Using Jenetics + Futures
Llama.cpp
SVT-AV1
Laghos
VVenC
Llama.cpp
Laghos
Renaissance
SVT-AV1
Llama.cpp
OpenVINO:
  Vehicle Detection FP16 - CPU:
    ms
    FPS
Llama.cpp:
  CPU BLAS - Llama-3.1-Tulu-3-8B-Q8_0 - Prompt Processing 512
  CPU BLAS - Mistral-7B-Instruct-v0.3-Q8_0 - Prompt Processing 512
SVT-AV1
Renaissance
OpenVINO:
  Handwritten English Recognition FP16 - CPU:
    FPS
    ms
  Road Segmentation ADAS FP16 - CPU:
    FPS
    ms
SVT-AV1
Blender:
  Junkshop - CPU-Only
  BMW27 - CPU-Only
  Classroom - CPU-Only
srsRAN Project
Blender
VVenC
x265
OpenVINO:
  Face Detection FP16 - CPU:
    FPS
    ms
Llamafile
Llama.cpp
OpenVINO
VVenC
OpenVINO
Primesieve
x265
OpenVINO:
  Face Detection Retail FP16-INT8 - CPU
  Handwritten English Recognition FP16-INT8 - CPU
Blender
SVT-AV1
OpenVINO
SVT-AV1
Llama.cpp
OpenVINO:
  Person Re-Identification Retail FP16 - CPU
  Person Detection FP16 - CPU
  Handwritten English Recognition FP16-INT8 - CPU
  Person Detection FP16 - CPU
SVT-AV1
ASTC Encoder
OpenVINO
OpenVINO GenAI
OpenVINO:
  Vehicle Detection FP16-INT8 - CPU
  Face Detection Retail FP16 - CPU
  Person Re-Identification Retail FP16 - CPU
  Age Gender Recognition Retail 0013 FP16-INT8 - CPU
VVenC
OpenVINO:
  Person Detection FP32 - CPU:
    ms
    FPS
ASTC Encoder
Llama.cpp
Blender
OpenVINO
Llama.cpp
SVT-AV1
ACES DGEMM
Llamafile
OpenVINO
Palabos
OpenVINO:
  Weld Porosity Detection FP16-INT8 - CPU
  Face Detection FP16-INT8 - CPU
Llamafile
ASTC Encoder
Llamafile
OpenVINO
Llamafile
OpenVINO
SVT-AV1
ASTC Encoder
SVT-AV1
Palabos
Llama.cpp
OpenVINO
OpenVINO GenAI:
  TinyLlama-1.1B-Chat-v1.0 - CPU
  Phi-3-mini-128k-instruct-int4-ov - CPU
ASTC Encoder
OpenVINO:
  Age Gender Recognition Retail 0013 FP16 - CPU
  Noise Suppression Poconet-Like FP16 - CPU
srsRAN Project
OpenVINO
OpenVINO GenAI
Renaissance
OpenVINO
Llamafile
srsRAN Project
Primesieve
SVT-AV1:
  Preset 13 - Beauty 4K 10-bit
  Preset 13 - Bosphorus 1080p
Llamafile:
  wizardcoder-python-34b-v1.0.Q6_K - Text Generation 128
  wizardcoder-python-34b-v1.0.Q6_K - Text Generation 16
Llama.cpp
OpenVINO:
  Age Gender Recognition Retail 0013 FP16-INT8 - CPU
  Age Gender Recognition Retail 0013 FP16 - CPU
  Person Vehicle Bike Detection FP16 - CPU
  Weld Porosity Detection FP16-INT8 - CPU
  Face Detection Retail FP16 - CPU