9575F smoke

AMD EPYC 9575F 64-Core testing with a Supermicro Super Server H13SSL-N v1.01 (3.0 BIOS) and ASPEED on Ubuntu 24.10 via the Phoronix Test Suite.

Compare your own system(s) to this result file with the Phoronix Test Suite by running the command: phoronix-test-suite benchmark 2411289-NE-9575FSMOK66
Jump To Table - Results

View

Do Not Show Noisy Results
Do Not Show Results With Incomplete Data
Do Not Show Results With Little Change/Spread
List Notable Results
Show Result Confidence Charts
Allow Limiting Results To Certain Suite(s)

Statistics

Show Overall Harmonic Mean(s)
Show Overall Geometric Mean
Show Wins / Losses Counts (Pie Chart)
Normalize Results
Remove Outliers Before Calculating Averages

Graph Settings

Force Line Graphs Where Applicable
Convert To Scalar Where Applicable
Prefer Vertical Bar Graphs

Multi-Way Comparison

Condense Multi-Option Tests Into Single Result Graphs

Table

Show Detailed System Result Table

Run Management

Highlight
Result
Toggle/Hide
Result
Result
Identifier
View Logs
Performance Per
Dollar
Date
Run
  Test
  Duration
a
November 28
  2 Hours, 4 Minutes
b
November 29
  2 Hours, 4 Minutes
Invert Behavior (Only Show Selected Data)
  2 Hours, 4 Minutes
Only show results matching title/arguments (delimit multiple options with a comma):
Do not show results matching title/arguments (delimit multiple options with a comma):


9575F smokeOpenBenchmarking.orgPhoronix Test SuiteAMD EPYC 9575F 64-Core @ 3.30GHz (64 Cores / 128 Threads)Supermicro Super Server H13SSL-N v1.01 (3.0 BIOS)AMD 1Ah12 x 64GB DDR5-6000MT/s Micron MTC40F2046S1RC64BDY QSFF3201GB Micron_7450_MTFDKCB3T2TFSASPEED2 x Broadcom NetXtreme BCM5720 PCIeUbuntu 24.106.12.0-rc7-linux-pm-next-phx (x86_64)GNOME Shell 47.0X ServerGCC 14.2.0ext41024x768ProcessorMotherboardChipsetMemoryDiskGraphicsNetworkOSKernelDesktopDisplay ServerCompilerFile-SystemScreen Resolution9575F Smoke BenchmarksSystem Logs- Transparent Huge Pages: madvise- --build=x86_64-linux-gnu --disable-vtable-verify --disable-werror --enable-bootstrap --enable-cet --enable-checking=release --enable-clocale=gnu --enable-default-pie --enable-gnu-unique-object --enable-languages=c,ada,c++,go,d,fortran,objc,obj-c++,m2,rust --enable-libphobos-checking=release --enable-libstdcxx-backtrace --enable-libstdcxx-debug --enable-libstdcxx-time=yes --enable-link-serialization=2 --enable-multiarch --enable-multilib --enable-nls --enable-objc-gc=auto --enable-offload-defaulted --enable-offload-targets=nvptx-none=/build/gcc-14-zdkDXv/gcc-14-14.2.0/debian/tmp-nvptx/usr,amdgcn-amdhsa=/build/gcc-14-zdkDXv/gcc-14-14.2.0/debian/tmp-gcn/usr --enable-plugin --enable-shared --enable-threads=posix --host=x86_64-linux-gnu --program-prefix=x86_64-linux-gnu- --target=x86_64-linux-gnu --with-abi=m64 --with-arch-32=i686 --with-build-config=bootstrap-lto-lean --with-default-libstdcxx-abi=new --with-gcc-major-version-only --with-multilib-list=m32,m64,mx32 --with-target-system-zlib=auto --with-tune=generic --without-cuda-driver -v - Scaling Governor: acpi-cpufreq performance (Boost: Enabled) - CPU Microcode: 0xb002116 - OpenJDK Runtime Environment (build 21.0.5+11-Ubuntu-1ubuntu124.10)- Python 3.12.7- gather_data_sampling: Not affected + itlb_multihit: Not affected + l1tf: Not affected + mds: Not affected + meltdown: Not affected + mmio_stale_data: Not affected + reg_file_data_sampling: Not affected + retbleed: Not affected + spec_rstack_overflow: Not affected + spec_store_bypass: Mitigation of SSB disabled via prctl + spectre_v1: Mitigation of usercopy/swapgs barriers and __user pointer sanitization + spectre_v2: Mitigation of Enhanced / Automatic IBRS; IBPB: conditional; STIBP: always-on; RSB filling; PBRSB-eIBRS: Not affected; BHI: Not affected + srbds: Not affected + tsx_async_abort: Not affected

a vs. b ComparisonPhoronix Test SuiteBaseline+2.3%+2.3%+4.6%+4.6%+6.9%+6.9%+9.2%+9.2%6.3%5.9%5.3%2.9%2.7%2.4%2.4%6.5%6.9%DeepLab V39.2%Mobilenet QuantWritesCPU BLAS - Llama-3.1-Tulu-3-8B-Q8_0 - P.P.55.9%FastT.P.P4.5%Q.C.S.M.v4.2%Falcon-7b-instruct-int4-ov - CPU3.7%I.R.VCPU BLAS - Llama-3.1-Tulu-3-8B-Q8_0 - P.P.12.7%CPU BLAS - Llama-3.1-Tulu-3-8B-Q8_0 - P.P.2FP16MobileNetV22.5%R.S.A.F - CPUR.S.A.F - CPUS.B.W.u.m2%Phi-3-mini-128k-instruct-int4-ov - CPU - T.T.F.TFalcon-7b-instruct-int4-ov - CPU - T.P.O.T3.6%Falcon-7b-instruct-int4-ov - CPU - T.T.F.TLiteRTLiteRTApache CassandraLlama.cppASTC EncoderLaghosLiteRTOpenVINO GenAILiteRTLlama.cppLlama.cppXNNPACKOpenVINOOpenVINOLaghosOpenVINO GenAIOpenVINO GenAIOpenVINO GenAIab

9575F smokelitert: DeepLab V3litert: Mobilenet Quantcassandra: Writesllama-cpp: CPU BLAS - Llama-3.1-Tulu-3-8B-Q8_0 - Prompt Processing 512astcenc: Fastlaghos: Triple Point Problemlitert: Quantized COCO SSD MobileNet v1openvino-genai: Falcon-7b-instruct-int4-ov - CPUlitert: Inception ResNet V2llama-cpp: CPU BLAS - Llama-3.1-Tulu-3-8B-Q8_0 - Prompt Processing 1024llama-cpp: CPU BLAS - Llama-3.1-Tulu-3-8B-Q8_0 - Prompt Processing 2048xnnpack: FP16MobileNetV2openvino: Road Segmentation ADAS FP16 - CPUopenvino: Road Segmentation ADAS FP16 - CPUlaghos: Sedov Blast Wave, ube_922_hex.meshllama-cpp: CPU BLAS - Mistral-7B-Instruct-v0.3-Q8_0 - Prompt Processing 2048llama-cpp: CPU BLAS - granite-3.0-3b-a800m-instruct-Q8_0 - Prompt Processing 512litert: NASNet Mobilesvt-av1: Preset 13 - Bosphorus 1080pllama-cpp: CPU BLAS - Mistral-7B-Instruct-v0.3-Q8_0 - Prompt Processing 1024openvino: Age Gender Recognition Retail 0013 FP16-INT8 - CPUnamd: STMV with 1,066,628 Atomslitert: SqueezeNetpalabos: 100blender: BMW27 - CPU-Onlyopenvino-genai: Gemma-7b-int4-ov - CPUsvt-av1: Preset 8 - Bosphorus 1080popenvino-genai: Phi-3-mini-128k-instruct-int4-ov - CPUopenvino: Person Detection FP16 - CPUopenvino: Person Detection FP16 - CPUopenvino: Face Detection Retail FP16 - CPUsvt-av1: Preset 13 - Bosphorus 4Kxnnpack: FP32MobileNetV1blender: Classroom - CPU-Onlyblender: Junkshop - CPU-Onlyopenvino: Vehicle Detection FP16 - CPUopenvino: Vehicle Detection FP16 - CPUxnnpack: FP16MobileNetV3Smallbyte: Dhrystone 2openvino: Face Detection Retail FP16 - CPUpalabos: 500namd: ATPase with 327,506 Atomsopenvino: Road Segmentation ADAS FP16-INT8 - CPUopenvino: Face Detection Retail FP16-INT8 - CPUllama-cpp: CPU BLAS - Mistral-7B-Instruct-v0.3-Q8_0 - Text Generation 128xnnpack: FP32MobileNetV2llama-cpp: CPU BLAS - Llama-3.1-Tulu-3-8B-Q8_0 - Text Generation 128openvino: Road Segmentation ADAS FP16-INT8 - CPUllama-cpp: CPU BLAS - granite-3.0-3b-a800m-instruct-Q8_0 - Prompt Processing 2048build-eigen: Time To Compilellama-cpp: CPU BLAS - granite-3.0-3b-a800m-instruct-Q8_0 - Text Generation 128xnnpack: QS8MobileNetV2blender: Pabellon Barcelona - CPU-Onlyxnnpack: FP32MobileNetV3Largeopenvino: Age Gender Recognition Retail 0013 FP16 - CPUopenvino: Person Re-Identification Retail FP16 - CPUopenvino: Vehicle Detection FP16-INT8 - CPUbyte: Pipepalabos: 400xnnpack: FP16MobileNetV3Largeopenvino: Vehicle Detection FP16-INT8 - CPUlitert: Inception V4openvino: Weld Porosity Detection FP16-INT8 - CPUprimesieve: 1e13openvino: Handwritten English Recognition FP16-INT8 - CPUsvt-av1: Preset 3 - Bosphorus 1080plitert: Mobilenet Floatopenvino: Handwritten English Recognition FP16-INT8 - CPUsvt-av1: Preset 5 - Bosphorus 1080popenvino: Weld Porosity Detection FP16-INT8 - CPUopenvino: Person Re-Identification Retail FP16 - CPUsvt-av1: Preset 8 - Bosphorus 4Kxnnpack: FP32MobileNetV3Smallopenvino: Face Detection FP16 - CPUopenvino: Face Detection FP16-INT8 - CPUastcenc: Exhaustiveopenvino: Noise Suppression Poconet-Like FP16 - CPUprimesieve: 1e12openvino: Noise Suppression Poconet-Like FP16 - CPUastcenc: Thoroughopenvino: Handwritten English Recognition FP16 - CPUllama-cpp: CPU BLAS - granite-3.0-3b-a800m-instruct-Q8_0 - Prompt Processing 1024llama-cpp: CPU BLAS - Mistral-7B-Instruct-v0.3-Q8_0 - Prompt Processing 512openvino: Weld Porosity Detection FP16 - CPUopenvino: Face Detection FP16-INT8 - CPUastcenc: Mediumopenvino: Person Detection FP32 - CPUopenvino: Person Detection FP32 - CPUopenvino: Face Detection FP16 - CPUsvt-av1: Preset 3 - Bosphorus 4Ksvt-av1: Preset 5 - Bosphorus 4Kopenvino: Handwritten English Recognition FP16 - CPUmt-dgemm: Sustained Floating-Point Rateblender: Fishy Cat - CPU-Onlyxnnpack: FP16MobileNetV1openvino: Person Vehicle Bike Detection FP16 - CPUopenvino: Machine Translation EN To DE FP16 - CPUastcenc: Very Thoroughopenvino-genai: TinyLlama-1.1B-Chat-v1.0 - CPUopenvino: Machine Translation EN To DE FP16 - CPUbyte: System Callblender: Barbershop - CPU-Onlybyte: Whetstone Doubleopenvino: Age Gender Recognition Retail 0013 FP16-INT8 - CPUopenvino: Age Gender Recognition Retail 0013 FP16 - CPUopenvino: Person Vehicle Bike Detection FP16 - CPUopenvino: Face Detection Retail FP16-INT8 - CPUopenvino: Weld Porosity Detection FP16 - CPUopenvino-genai: Phi-3-mini-128k-instruct-int4-ov - CPU - Time Per Output Tokenopenvino-genai: Phi-3-mini-128k-instruct-int4-ov - CPU - Time To First Tokenopenvino-genai: Falcon-7b-instruct-int4-ov - CPU - Time Per Output Tokenopenvino-genai: Falcon-7b-instruct-int4-ov - CPU - Time To First Tokenopenvino-genai: TinyLlama-1.1B-Chat-v1.0 - CPU - Time Per Output Tokenopenvino-genai: TinyLlama-1.1B-Chat-v1.0 - CPU - Time To First Tokenopenvino-genai: Gemma-7b-int4-ov - CPU - Time Per Output Tokenopenvino-genai: Gemma-7b-int4-ov - CPU - Time To First Tokenab16007.519415464181313.791077.777308.928744.9157.8239371.7334.84319.54469217.61812.32579.090715221321.821013.032181971299.731335.62166477.753.753603814.42812.28515.1549.16529.4765.33656.2948.6815364.86481.968241741.7120.136.864634.7554466028934961.92.03776.3112.588882316.7522034.4954.23472551.8413.76965.4428.465120.87504147.437242137029.683.417561.85379708473.5734.24270994.2124673.34.6424.7820.5544.8092429.063111.82147.92613617.589322.35202.154541370.11140.596.21619.561.9736485.9772.409720.931055.54317.196826.87227.21514.667548.88653.6455.2317.03360.7293054.875201.05592421.2624706219.36781.0910.122877.2740.94360905676.5148.591440313.30.260.345.092.839.3115.3123.3417.331.2812.9415.8220.3430.3617473.418266.4491661296.411135.1168295.659116.4755.7838276.5325.9328.07480917.181856.61567.84327.81998.22211841316.946331.38164424.123.797373857.68804.11915.0149.61534.21265.91651.8649.0115466.24478.84243241.4620.256.94609.754176060068455.12.02772.49712.646142326.5821941.8154.01474351.6513.71962.0728.37121.27505747.587220137431.923.427540.16378638216.8732.40670824.2224727.44.6324.8320.5944.7222424.823106.45148.16313596.149309.66202.429540670.02140.426.22339.551.9756492.4372.340420.951056.46317.466821.09227.4514.238948.92653.07455.5617.04460.76730535203.82668421.2724716221.54781.3210.125577.2540.93360867935.4148.61440301.80.260.345.092.839.3115.1721.9117.9329.2712.9515.920.1630.16OpenBenchmarking.org

LiteRT

OpenBenchmarking.orgMicroseconds, Fewer Is BetterLiteRT 2024-10-15Model: DeepLab V3ba4K8K12K16K20K17473.416007.5

OpenBenchmarking.orgMicroseconds, Fewer Is BetterLiteRT 2024-10-15Model: Mobilenet Quantba4K8K12K16K20K18266.419415.0

Apache Cassandra

OpenBenchmarking.orgOp/s, More Is BetterApache Cassandra 5.0Test: Writesba110K220K330K440K550K491661464181

Llama.cpp

OpenBenchmarking.orgTokens Per Second, More Is BetterLlama.cpp b4154Backend: CPU BLAS - Model: Llama-3.1-Tulu-3-8B-Q8_0 - Test: Prompt Processing 512ba70140210280350296.41313.791. (CXX) g++ options: -std=c++11 -fPIC -O3 -pthread -fopenmp -march=native -mtune=native -lopenblas

ASTC Encoder

OpenBenchmarking.orgMT/s, More Is BetterASTC Encoder 5.0Preset: Fastba20040060080010001135.121077.781. (CXX) g++ options: -O3 -flto -pthread

Laghos

OpenBenchmarking.orgMajor Kernels Total Rate, More Is BetterLaghos 3.1Test: Triple Point Problemba70140210280350295.65308.921. (CXX) g++ options: -O3 -std=c++11 -lmfem -lHYPRE -lmetis -lrt -lmpi_cxx -lmpi

LiteRT

OpenBenchmarking.orgMicroseconds, Fewer Is BetterLiteRT 2024-10-15Model: Quantized COCO SSD MobileNet v1ba2K4K6K8K10K9116.478744.91

OpenVINO GenAI

OpenBenchmarking.orgtokens/s, More Is BetterOpenVINO GenAI 2024.5Model: Falcon-7b-instruct-int4-ov - Device: CPUba132639526555.7857.82

LiteRT

OpenBenchmarking.orgMicroseconds, Fewer Is BetterLiteRT 2024-10-15Model: Inception ResNet V2ba8K16K24K32K40K38276.539371.7

Llama.cpp

OpenBenchmarking.orgTokens Per Second, More Is BetterLlama.cpp b4154Backend: CPU BLAS - Model: Llama-3.1-Tulu-3-8B-Q8_0 - Test: Prompt Processing 1024ba70140210280350325.90334.841. (CXX) g++ options: -std=c++11 -fPIC -O3 -pthread -fopenmp -march=native -mtune=native -lopenblas

OpenBenchmarking.orgTokens Per Second, More Is BetterLlama.cpp b4154Backend: CPU BLAS - Model: Llama-3.1-Tulu-3-8B-Q8_0 - Test: Prompt Processing 2048ba70140210280350328.07319.541. (CXX) g++ options: -std=c++11 -fPIC -O3 -pthread -fopenmp -march=native -mtune=native -lopenblas

XNNPACK

OpenBenchmarking.orgus, Fewer Is BetterXNNPACK b7b048Model: FP16MobileNetV2ba10002000300040005000480946921. (CXX) g++ options: -O3 -lrt -lm

OpenVINO

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2024.5Model: Road Segmentation ADAS FP16 - Device: CPUba4812162017.1817.60MIN: 11.6 / MAX: 32.37MIN: 8.57 / MAX: 30.921. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl -lstdc++fs

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2024.5Model: Road Segmentation ADAS FP16 - Device: CPUba4008001200160020001856.611812.321. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl -lstdc++fs

Laghos

OpenBenchmarking.orgMajor Kernels Total Rate, More Is BetterLaghos 3.1Test: Sedov Blast Wave, ube_922_hex.meshba130260390520650567.84579.091. (CXX) g++ options: -O3 -std=c++11 -lmfem -lHYPRE -lmetis -lrt -lmpi_cxx -lmpi

Llama.cpp

OpenBenchmarking.orgTokens Per Second, More Is BetterLlama.cpp b4154Backend: CPU BLAS - Model: Mistral-7B-Instruct-v0.3-Q8_0 - Test: Prompt Processing 2048ba70140210280350327.81321.821. (CXX) g++ options: -std=c++11 -fPIC -O3 -pthread -fopenmp -march=native -mtune=native -lopenblas

OpenBenchmarking.orgTokens Per Second, More Is BetterLlama.cpp b4154Backend: CPU BLAS - Model: granite-3.0-3b-a800m-instruct-Q8_0 - Test: Prompt Processing 512ba2004006008001000998.201013.031. (CXX) g++ options: -std=c++11 -fPIC -O3 -pthread -fopenmp -march=native -mtune=native -lopenblas

LiteRT

OpenBenchmarking.orgMicroseconds, Fewer Is BetterLiteRT 2024-10-15Model: NASNet Mobileba50K100K150K200K250K221184218197

SVT-AV1

OpenBenchmarking.orgFrames Per Second, More Is BetterSVT-AV1 2.3Encoder Mode: Preset 13 - Input: Bosphorus 1080pba300600900120015001316.951299.731. (CXX) g++ options: -march=native -mno-avx -mavx2 -mavx512f -mavx512bw -mavx512dq

Llama.cpp

OpenBenchmarking.orgTokens Per Second, More Is BetterLlama.cpp b4154Backend: CPU BLAS - Model: Mistral-7B-Instruct-v0.3-Q8_0 - Test: Prompt Processing 1024ba70140210280350331.38335.621. (CXX) g++ options: -std=c++11 -fPIC -O3 -pthread -fopenmp -march=native -mtune=native -lopenblas

OpenVINO

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2024.5Model: Age Gender Recognition Retail 0013 FP16-INT8 - Device: CPUba40K80K120K160K200K164424.12166477.751. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl -lstdc++fs

NAMD

OpenBenchmarking.orgns/day, More Is BetterNAMD 3.0Input: STMV with 1,066,628 Atomsba0.85441.70882.56323.41764.2723.797373.75360

LiteRT

OpenBenchmarking.orgMicroseconds, Fewer Is BetterLiteRT 2024-10-15Model: SqueezeNetba80016002400320040003857.683814.42

Palabos

OpenBenchmarking.orgMega Site Updates Per Second, More Is BetterPalabos 2.3Grid Size: 100ba2004006008001000804.12812.291. (CXX) g++ options: -std=c++17 -pedantic -O3 -rdynamic -lcrypto -lcurl -lsz -lz -ldl -lm

Blender

OpenBenchmarking.orgSeconds, Fewer Is BetterBlender 4.3Blend File: BMW27 - Compute: CPU-Onlyba4812162015.0115.15

OpenVINO GenAI

OpenBenchmarking.orgtokens/s, More Is BetterOpenVINO GenAI 2024.5Model: Gemma-7b-int4-ov - Device: CPUba112233445549.6149.16

SVT-AV1

OpenBenchmarking.orgFrames Per Second, More Is BetterSVT-AV1 2.3Encoder Mode: Preset 8 - Input: Bosphorus 1080pba120240360480600534.21529.471. (CXX) g++ options: -march=native -mno-avx -mavx2 -mavx512f -mavx512bw -mavx512dq

OpenVINO GenAI

OpenBenchmarking.orgtokens/s, More Is BetterOpenVINO GenAI 2024.5Model: Phi-3-mini-128k-instruct-int4-ov - Device: CPUba153045607565.9165.33

OpenVINO

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2024.5Model: Person Detection FP16 - Device: CPUba140280420560700651.86656.291. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl -lstdc++fs

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2024.5Model: Person Detection FP16 - Device: CPUba112233445549.0148.68MIN: 37.75 / MAX: 78.79MIN: 35.62 / MAX: 80.71. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl -lstdc++fs

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2024.5Model: Face Detection Retail FP16 - Device: CPUba3K6K9K12K15K15466.2415364.861. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl -lstdc++fs

SVT-AV1

OpenBenchmarking.orgFrames Per Second, More Is BetterSVT-AV1 2.3Encoder Mode: Preset 13 - Input: Bosphorus 4Kba100200300400500478.84481.971. (CXX) g++ options: -march=native -mno-avx -mavx2 -mavx512f -mavx512bw -mavx512dq

XNNPACK

OpenBenchmarking.orgus, Fewer Is BetterXNNPACK b7b048Model: FP32MobileNetV1ba5001000150020002500243224171. (CXX) g++ options: -O3 -lrt -lm

Blender

OpenBenchmarking.orgSeconds, Fewer Is BetterBlender 4.3Blend File: Classroom - Compute: CPU-Onlyba102030405041.4641.71

OpenBenchmarking.orgSeconds, Fewer Is BetterBlender 4.3Blend File: Junkshop - Compute: CPU-Onlyba51015202520.2520.13

OpenVINO

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2024.5Model: Vehicle Detection FP16 - Device: CPUba2468106.906.86MIN: 2.86 / MAX: 15.25MIN: 3.31 / MAX: 17.141. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl -lstdc++fs

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2024.5Model: Vehicle Detection FP16 - Device: CPUba100020003000400050004609.704634.751. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl -lstdc++fs

XNNPACK

OpenBenchmarking.orgus, Fewer Is BetterXNNPACK b7b048Model: FP16MobileNetV3Smallba12002400360048006000541754461. (CXX) g++ options: -O3 -lrt -lm

BYTE Unix Benchmark

OpenBenchmarking.orgLPS, More Is BetterBYTE Unix Benchmark 5.1.3-gitComputational Test: Dhrystone 2ba1300M2600M3900M5200M6500M6060068455.16028934961.91. (CC) gcc options: -pedantic -O3 -ffast-math -march=native -mtune=native -lm

OpenVINO

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2024.5Model: Face Detection Retail FP16 - Device: CPUba0.45680.91361.37041.82722.2842.022.03MIN: 1.16 / MAX: 5.27MIN: 1.15 / MAX: 5.231. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl -lstdc++fs

Palabos

OpenBenchmarking.orgMega Site Updates Per Second, More Is BetterPalabos 2.3Grid Size: 500ba2004006008001000772.50776.311. (CXX) g++ options: -std=c++17 -pedantic -O3 -rdynamic -lcrypto -lcurl -lsz -lz -ldl -lm

NAMD

OpenBenchmarking.orgns/day, More Is BetterNAMD 3.0Input: ATPase with 327,506 Atomsba369121512.6512.59

OpenVINO

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2024.5Model: Road Segmentation ADAS FP16-INT8 - Device: CPUba50010001500200025002326.582316.751. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl -lstdc++fs

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2024.5Model: Face Detection Retail FP16-INT8 - Device: CPUba5K10K15K20K25K21941.8122034.491. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl -lstdc++fs

Llama.cpp

OpenBenchmarking.orgTokens Per Second, More Is BetterLlama.cpp b4154Backend: CPU BLAS - Model: Mistral-7B-Instruct-v0.3-Q8_0 - Test: Text Generation 128ba122436486054.0154.231. (CXX) g++ options: -std=c++11 -fPIC -O3 -pthread -fopenmp -march=native -mtune=native -lopenblas

XNNPACK

OpenBenchmarking.orgus, Fewer Is BetterXNNPACK b7b048Model: FP32MobileNetV2ba10002000300040005000474347251. (CXX) g++ options: -O3 -lrt -lm

Llama.cpp

OpenBenchmarking.orgTokens Per Second, More Is BetterLlama.cpp b4154Backend: CPU BLAS - Model: Llama-3.1-Tulu-3-8B-Q8_0 - Test: Text Generation 128ba122436486051.6551.841. (CXX) g++ options: -std=c++11 -fPIC -O3 -pthread -fopenmp -march=native -mtune=native -lopenblas

OpenVINO

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2024.5Model: Road Segmentation ADAS FP16-INT8 - Device: CPUba4812162013.7113.76MIN: 8.65 / MAX: 29.04MIN: 7.69 / MAX: 27.141. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl -lstdc++fs

Llama.cpp

OpenBenchmarking.orgTokens Per Second, More Is BetterLlama.cpp b4154Backend: CPU BLAS - Model: granite-3.0-3b-a800m-instruct-Q8_0 - Test: Prompt Processing 2048ba2004006008001000962.07965.441. (CXX) g++ options: -std=c++11 -fPIC -O3 -pthread -fopenmp -march=native -mtune=native -lopenblas

Timed Eigen Compilation

OpenBenchmarking.orgSeconds, Fewer Is BetterTimed Eigen Compilation 3.4.0Time To Compileba71421283528.3728.47

Llama.cpp

OpenBenchmarking.orgTokens Per Second, More Is BetterLlama.cpp b4154Backend: CPU BLAS - Model: granite-3.0-3b-a800m-instruct-Q8_0 - Test: Text Generation 128ba306090120150121.27120.871. (CXX) g++ options: -std=c++11 -fPIC -O3 -pthread -fopenmp -march=native -mtune=native -lopenblas

XNNPACK

OpenBenchmarking.orgus, Fewer Is BetterXNNPACK b7b048Model: QS8MobileNetV2ba11002200330044005500505750411. (CXX) g++ options: -O3 -lrt -lm

Blender

OpenBenchmarking.orgSeconds, Fewer Is BetterBlender 4.3Blend File: Pabellon Barcelona - Compute: CPU-Onlyba112233445547.5847.43

XNNPACK

OpenBenchmarking.orgus, Fewer Is BetterXNNPACK b7b048Model: FP32MobileNetV3Largeba16003200480064008000722072421. (CXX) g++ options: -O3 -lrt -lm

OpenVINO

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2024.5Model: Age Gender Recognition Retail 0013 FP16 - Device: CPUba30K60K90K120K150K137431.92137029.681. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl -lstdc++fs

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2024.5Model: Person Re-Identification Retail FP16 - Device: CPUba0.76951.5392.30853.0783.84753.423.41MIN: 1.64 / MAX: 10.95MIN: 2.12 / MAX: 10.971. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl -lstdc++fs

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2024.5Model: Vehicle Detection FP16-INT8 - Device: CPUba160032004800640080007540.167561.851. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl -lstdc++fs

BYTE Unix Benchmark

OpenBenchmarking.orgLPS, More Is BetterBYTE Unix Benchmark 5.1.3-gitComputational Test: Pipeba80M160M240M320M400M378638216.8379708473.51. (CC) gcc options: -pedantic -O3 -ffast-math -march=native -mtune=native -lm

Palabos

OpenBenchmarking.orgMega Site Updates Per Second, More Is BetterPalabos 2.3Grid Size: 400ba160320480640800732.41734.241. (CXX) g++ options: -std=c++17 -pedantic -O3 -rdynamic -lcrypto -lcurl -lsz -lz -ldl -lm

XNNPACK

OpenBenchmarking.orgus, Fewer Is BetterXNNPACK b7b048Model: FP16MobileNetV3Largeba15003000450060007500708270991. (CXX) g++ options: -O3 -lrt -lm

OpenVINO

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2024.5Model: Vehicle Detection FP16-INT8 - Device: CPUba0.94951.8992.84853.7984.74754.224.21MIN: 2.1 / MAX: 17.49MIN: 2.44 / MAX: 16.231. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl -lstdc++fs

LiteRT

OpenBenchmarking.orgMicroseconds, Fewer Is BetterLiteRT 2024-10-15Model: Inception V4ba5K10K15K20K25K24727.424673.3

OpenVINO

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2024.5Model: Weld Porosity Detection FP16-INT8 - Device: CPUba1.0442.0883.1324.1765.224.634.64MIN: 2.41 / MAX: 14.26MIN: 2.41 / MAX: 12.381. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl -lstdc++fs

Primesieve

OpenBenchmarking.orgSeconds, Fewer Is BetterPrimesieve 12.6Length: 1e13ba61218243024.8324.781. (CXX) g++ options: -O3

OpenVINO

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2024.5Model: Handwritten English Recognition FP16-INT8 - Device: CPUba51015202520.5920.55MIN: 13.03 / MAX: 36.12MIN: 13.3 / MAX: 28.621. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl -lstdc++fs

SVT-AV1

OpenBenchmarking.orgFrames Per Second, More Is BetterSVT-AV1 2.3Encoder Mode: Preset 3 - Input: Bosphorus 1080pba102030405044.7244.811. (CXX) g++ options: -march=native -mno-avx -mavx2 -mavx512f -mavx512bw -mavx512dq

LiteRT

OpenBenchmarking.orgMicroseconds, Fewer Is BetterLiteRT 2024-10-15Model: Mobilenet Floatba50010001500200025002424.822429.06

OpenVINO

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2024.5Model: Handwritten English Recognition FP16-INT8 - Device: CPUba70014002100280035003106.453111.821. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl -lstdc++fs

SVT-AV1

OpenBenchmarking.orgFrames Per Second, More Is BetterSVT-AV1 2.3Encoder Mode: Preset 5 - Input: Bosphorus 1080pba306090120150148.16147.931. (CXX) g++ options: -march=native -mno-avx -mavx2 -mavx512f -mavx512bw -mavx512dq

OpenVINO

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2024.5Model: Weld Porosity Detection FP16-INT8 - Device: CPUba3K6K9K12K15K13596.1413617.581. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl -lstdc++fs

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2024.5Model: Person Re-Identification Retail FP16 - Device: CPUba2K4K6K8K10K9309.669322.351. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl -lstdc++fs

SVT-AV1

OpenBenchmarking.orgFrames Per Second, More Is BetterSVT-AV1 2.3Encoder Mode: Preset 8 - Input: Bosphorus 4Kba4080120160200202.43202.151. (CXX) g++ options: -march=native -mno-avx -mavx2 -mavx512f -mavx512bw -mavx512dq

XNNPACK

OpenBenchmarking.orgus, Fewer Is BetterXNNPACK b7b048Model: FP32MobileNetV3Smallba12002400360048006000540654131. (CXX) g++ options: -O3 -lrt -lm

OpenVINO

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2024.5Model: Face Detection FP16 - Device: CPUba163248648070.0270.111. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl -lstdc++fs

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2024.5Model: Face Detection FP16-INT8 - Device: CPUba306090120150140.42140.591. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl -lstdc++fs

ASTC Encoder

OpenBenchmarking.orgMT/s, More Is BetterASTC Encoder 5.0Preset: Exhaustiveba2468106.22336.21611. (CXX) g++ options: -O3 -flto -pthread

OpenVINO

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2024.5Model: Noise Suppression Poconet-Like FP16 - Device: CPUba36912159.559.56MIN: 6.24 / MAX: 20.75MIN: 6.01 / MAX: 21.581. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl -lstdc++fs

Primesieve

OpenBenchmarking.orgSeconds, Fewer Is BetterPrimesieve 12.6Length: 1e12ba0.44440.88881.33321.77762.2221.9751.9731. (CXX) g++ options: -O3

OpenVINO

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2024.5Model: Noise Suppression Poconet-Like FP16 - Device: CPUba140028004200560070006492.436485.971. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl -lstdc++fs

ASTC Encoder

OpenBenchmarking.orgMT/s, More Is BetterASTC Encoder 5.0Preset: Thoroughba163248648072.3472.411. (CXX) g++ options: -O3 -flto -pthread

OpenVINO

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2024.5Model: Handwritten English Recognition FP16 - Device: CPUba51015202520.9520.93MIN: 13.73 / MAX: 32.59MIN: 15.01 / MAX: 31.51. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl -lstdc++fs

Llama.cpp

OpenBenchmarking.orgTokens Per Second, More Is BetterLlama.cpp b4154Backend: CPU BLAS - Model: granite-3.0-3b-a800m-instruct-Q8_0 - Test: Prompt Processing 1024ba20040060080010001056.461055.541. (CXX) g++ options: -std=c++11 -fPIC -O3 -pthread -fopenmp -march=native -mtune=native -lopenblas

OpenBenchmarking.orgTokens Per Second, More Is BetterLlama.cpp b4154Backend: CPU BLAS - Model: Mistral-7B-Instruct-v0.3-Q8_0 - Test: Prompt Processing 512ba70140210280350317.46317.191. (CXX) g++ options: -std=c++11 -fPIC -O3 -pthread -fopenmp -march=native -mtune=native -lopenblas

OpenVINO

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2024.5Model: Weld Porosity Detection FP16 - Device: CPUba150030004500600075006821.096826.871. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl -lstdc++fs

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2024.5Model: Face Detection FP16-INT8 - Device: CPUba50100150200250227.40227.21MIN: 190.49 / MAX: 248.73MIN: 213.43 / MAX: 250.311. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl -lstdc++fs

ASTC Encoder

OpenBenchmarking.orgMT/s, More Is BetterASTC Encoder 5.0Preset: Mediumba110220330440550514.24514.671. (CXX) g++ options: -O3 -flto -pthread

OpenVINO

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2024.5Model: Person Detection FP32 - Device: CPUba112233445548.9248.88MIN: 36.36 / MAX: 80.16MIN: 21.94 / MAX: 80.521. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl -lstdc++fs

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2024.5Model: Person Detection FP32 - Device: CPUba140280420560700653.07653.601. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl -lstdc++fs

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2024.5Model: Face Detection FP16 - Device: CPUba100200300400500455.56455.23MIN: 358.53 / MAX: 467.53MIN: 379.27 / MAX: 470.621. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl -lstdc++fs

SVT-AV1

OpenBenchmarking.orgFrames Per Second, More Is BetterSVT-AV1 2.3Encoder Mode: Preset 3 - Input: Bosphorus 4Kba4812162017.0417.031. (CXX) g++ options: -march=native -mno-avx -mavx2 -mavx512f -mavx512bw -mavx512dq

OpenBenchmarking.orgFrames Per Second, More Is BetterSVT-AV1 2.3Encoder Mode: Preset 5 - Input: Bosphorus 4Kba142842567060.7760.731. (CXX) g++ options: -march=native -mno-avx -mavx2 -mavx512f -mavx512bw -mavx512dq

OpenVINO

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2024.5Model: Handwritten English Recognition FP16 - Device: CPUba70014002100280035003053.003054.871. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl -lstdc++fs

ACES DGEMM

OpenBenchmarking.orgGFLOP/s, More Is BetterACES DGEMM 1.0Sustained Floating-Point Rateba110022003300440055005203.835201.061. (CC) gcc options: -ffast-math -mavx2 -O3 -fopenmp -lopenblas

Blender

OpenBenchmarking.orgSeconds, Fewer Is BetterBlender 4.3Blend File: Fishy Cat - Compute: CPU-Onlyba51015202521.2721.26

XNNPACK

OpenBenchmarking.orgus, Fewer Is BetterXNNPACK b7b048Model: FP16MobileNetV1ba5001000150020002500247124701. (CXX) g++ options: -O3 -lrt -lm

OpenVINO

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2024.5Model: Person Vehicle Bike Detection FP16 - Device: CPUba130026003900520065006221.546219.361. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl -lstdc++fs

OpenBenchmarking.orgFPS, More Is BetterOpenVINO 2024.5Model: Machine Translation EN To DE FP16 - Device: CPUba2004006008001000781.32781.091. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl -lstdc++fs

ASTC Encoder

OpenBenchmarking.orgMT/s, More Is BetterASTC Encoder 5.0Preset: Very Thoroughba369121510.1310.121. (CXX) g++ options: -O3 -flto -pthread

OpenVINO GenAI

OpenBenchmarking.orgtokens/s, More Is BetterOpenVINO GenAI 2024.5Model: TinyLlama-1.1B-Chat-v1.0 - Device: CPUba2040608010077.2577.27

OpenVINO

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2024.5Model: Machine Translation EN To DE FP16 - Device: CPUba91827364540.9340.94MIN: 22.52 / MAX: 59.28MIN: 22.77 / MAX: 58.681. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl -lstdc++fs

BYTE Unix Benchmark

OpenBenchmarking.orgLPS, More Is BetterBYTE Unix Benchmark 5.1.3-gitComputational Test: System Callba80M160M240M320M400M360867935.4360905676.51. (CC) gcc options: -pedantic -O3 -ffast-math -march=native -mtune=native -lm

Blender

OpenBenchmarking.orgSeconds, Fewer Is BetterBlender 4.3Blend File: Barbershop - Compute: CPU-Onlyba306090120150148.60148.59

BYTE Unix Benchmark

OpenBenchmarking.orgMWIPS, More Is BetterBYTE Unix Benchmark 5.1.3-gitComputational Test: Whetstone Doubleba300K600K900K1200K1500K1440301.81440313.31. (CC) gcc options: -pedantic -O3 -ffast-math -march=native -mtune=native -lm

OpenVINO

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2024.5Model: Age Gender Recognition Retail 0013 FP16-INT8 - Device: CPUba0.05850.1170.17550.2340.29250.260.26MIN: 0.11 / MAX: 19.06MIN: 0.11 / MAX: 18.011. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl -lstdc++fs

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2024.5Model: Age Gender Recognition Retail 0013 FP16 - Device: CPUba0.07650.1530.22950.3060.38250.340.34MIN: 0.15 / MAX: 10.78MIN: 0.16 / MAX: 10.051. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl -lstdc++fs

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2024.5Model: Person Vehicle Bike Detection FP16 - Device: CPUba1.14532.29063.43594.58125.72655.095.09MIN: 3.24 / MAX: 12.86MIN: 3.2 / MAX: 15.271. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl -lstdc++fs

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2024.5Model: Face Detection Retail FP16-INT8 - Device: CPUba0.63681.27361.91042.54723.1842.832.83MIN: 1.62 / MAX: 9.1MIN: 1.66 / MAX: 11.711. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl -lstdc++fs

OpenBenchmarking.orgms, Fewer Is BetterOpenVINO 2024.5Model: Weld Porosity Detection FP16 - Device: CPUba36912159.319.31MIN: 4.31 / MAX: 25.14MIN: 4.57 / MAX: 24.341. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl -lstdc++fs

108 Results Shown

LiteRT:
  DeepLab V3
  Mobilenet Quant
Apache Cassandra
Llama.cpp
ASTC Encoder
Laghos
LiteRT
OpenVINO GenAI
LiteRT
Llama.cpp:
  CPU BLAS - Llama-3.1-Tulu-3-8B-Q8_0 - Prompt Processing 1024
  CPU BLAS - Llama-3.1-Tulu-3-8B-Q8_0 - Prompt Processing 2048
XNNPACK
OpenVINO:
  Road Segmentation ADAS FP16 - CPU:
    ms
    FPS
Laghos
Llama.cpp:
  CPU BLAS - Mistral-7B-Instruct-v0.3-Q8_0 - Prompt Processing 2048
  CPU BLAS - granite-3.0-3b-a800m-instruct-Q8_0 - Prompt Processing 512
LiteRT
SVT-AV1
Llama.cpp
OpenVINO
NAMD
LiteRT
Palabos
Blender
OpenVINO GenAI
SVT-AV1
OpenVINO GenAI
OpenVINO:
  Person Detection FP16 - CPU:
    FPS
    ms
  Face Detection Retail FP16 - CPU:
    FPS
SVT-AV1
XNNPACK
Blender:
  Classroom - CPU-Only
  Junkshop - CPU-Only
OpenVINO:
  Vehicle Detection FP16 - CPU:
    ms
    FPS
XNNPACK
BYTE Unix Benchmark
OpenVINO
Palabos
NAMD
OpenVINO:
  Road Segmentation ADAS FP16-INT8 - CPU
  Face Detection Retail FP16-INT8 - CPU
Llama.cpp
XNNPACK
Llama.cpp
OpenVINO
Llama.cpp
Timed Eigen Compilation
Llama.cpp
XNNPACK
Blender
XNNPACK
OpenVINO:
  Age Gender Recognition Retail 0013 FP16 - CPU
  Person Re-Identification Retail FP16 - CPU
  Vehicle Detection FP16-INT8 - CPU
BYTE Unix Benchmark
Palabos
XNNPACK
OpenVINO
LiteRT
OpenVINO
Primesieve
OpenVINO
SVT-AV1
LiteRT
OpenVINO
SVT-AV1
OpenVINO:
  Weld Porosity Detection FP16-INT8 - CPU
  Person Re-Identification Retail FP16 - CPU
SVT-AV1
XNNPACK
OpenVINO:
  Face Detection FP16 - CPU
  Face Detection FP16-INT8 - CPU
ASTC Encoder
OpenVINO
Primesieve
OpenVINO
ASTC Encoder
OpenVINO
Llama.cpp:
  CPU BLAS - granite-3.0-3b-a800m-instruct-Q8_0 - Prompt Processing 1024
  CPU BLAS - Mistral-7B-Instruct-v0.3-Q8_0 - Prompt Processing 512
OpenVINO:
  Weld Porosity Detection FP16 - CPU
  Face Detection FP16-INT8 - CPU
ASTC Encoder
OpenVINO:
  Person Detection FP32 - CPU:
    ms
    FPS
  Face Detection FP16 - CPU:
    ms
SVT-AV1:
  Preset 3 - Bosphorus 4K
  Preset 5 - Bosphorus 4K
OpenVINO
ACES DGEMM
Blender
XNNPACK
OpenVINO:
  Person Vehicle Bike Detection FP16 - CPU
  Machine Translation EN To DE FP16 - CPU
ASTC Encoder
OpenVINO GenAI
OpenVINO
BYTE Unix Benchmark
Blender
BYTE Unix Benchmark
OpenVINO:
  Age Gender Recognition Retail 0013 FP16-INT8 - CPU
  Age Gender Recognition Retail 0013 FP16 - CPU
  Person Vehicle Bike Detection FP16 - CPU
  Face Detection Retail FP16-INT8 - CPU
  Weld Porosity Detection FP16 - CPU