new-sun Intel Core Ultra 9 285K testing with a ASUS ROG MAXIMUS Z890 HERO (0806 BIOS) and ASUS NVIDIA GeForce RTX 4070 Ti SUPER 16GB on Ubuntu 24.10 via the Phoronix Test Suite.
HTML result view exported from: https://openbenchmarking.org/result/2411250-PTS-NEWSUN2905&sro&grs .
new-sun Processor Motherboard Chipset Memory Disk Graphics Audio Monitor Network OS Kernel Desktop Display Server Display Driver OpenGL OpenCL Compiler File-System Screen Resolution a aa b c Intel Core Ultra 9 285K @ 5.10GHz (24 Cores) ASUS ROG MAXIMUS Z890 HERO (0806 BIOS) Intel Device ae7f 2 x 16GB DDR5-6400MT/s Micron CP16G64C38U5B.M8D1 4001GB Western Digital WD_BLACK SN850X 4000GB + 1000GB Western Digital WDS100T1X0E-00AFY0 ASUS NVIDIA GeForce RTX 4070 Ti SUPER 16GB Intel Device 7f50 ASUS VP28U Realtek Device 8126 + Intel I226-V + Intel Wi-Fi 7 Ubuntu 24.10 6.11.0-9-generic (x86_64) GNOME Shell 47.0 X Server 1.21.1.13 NVIDIA 565.57.01 4.6.0 OpenCL 3.0 CUDA 12.7.33 GCC 14.2.0 ext4 3840x2160 OpenBenchmarking.org Kernel Details - nouveau.modeset=0 - Transparent Huge Pages: madvise Compiler Details - --build=x86_64-linux-gnu --disable-vtable-verify --disable-werror --enable-bootstrap --enable-cet --enable-checking=release --enable-clocale=gnu --enable-default-pie --enable-gnu-unique-object --enable-languages=c,ada,c++,go,d,fortran,objc,obj-c++,m2,rust --enable-libphobos-checking=release --enable-libstdcxx-backtrace --enable-libstdcxx-debug --enable-libstdcxx-time=yes --enable-link-serialization=2 --enable-multiarch --enable-multilib --enable-nls --enable-objc-gc=auto --enable-offload-defaulted --enable-offload-targets=nvptx-none=/build/gcc-14-zdkDXv/gcc-14-14.2.0/debian/tmp-nvptx/usr,amdgcn-amdhsa=/build/gcc-14-zdkDXv/gcc-14-14.2.0/debian/tmp-gcn/usr --enable-plugin --enable-shared --enable-threads=posix --host=x86_64-linux-gnu --program-prefix=x86_64-linux-gnu- --target=x86_64-linux-gnu --with-abi=m64 --with-arch-32=i686 --with-build-config=bootstrap-lto-lean --with-default-libstdcxx-abi=new --with-gcc-major-version-only --with-multilib-list=m32,m64,mx32 --with-target-system-zlib=auto --with-tune=generic --without-cuda-driver -v Processor Details - Scaling Governor: intel_pstate performance (EPP: default) - CPU Microcode: 0x110 - Thermald 2.5.8 Graphics Details - BAR1 / Visible vRAM Size: 16384 MiB - vBIOS Version: 95.03.45.00.9c Java Details - OpenJDK Runtime Environment (build 21.0.5+11-Ubuntu-1ubuntu124.10) Python Details - Python 3.12.7 Security Details - gather_data_sampling: Not affected + itlb_multihit: Not affected + l1tf: Not affected + mds: Not affected + meltdown: Not affected + mmio_stale_data: Not affected + reg_file_data_sampling: Not affected + retbleed: Not affected + spec_rstack_overflow: Not affected + spec_store_bypass: Mitigation of SSB disabled via prctl + spectre_v1: Mitigation of usercopy/swapgs barriers and __user pointer sanitization + spectre_v2: Mitigation of Enhanced / Automatic IBRS; IBPB: conditional; RSB filling; PBRSB-eIBRS: Not affected; BHI: Not affected + srbds: Not affected + tsx_async_abort: Not affected
new-sun renaissance: Scala Dotty renaissance: Finagle HTTP Requests openvino: Face Detection Retail FP16-INT8 - CPU renaissance: In-Memory Database Shootout openvino: Face Detection Retail FP16-INT8 - CPU openvino: Age Gender Recognition Retail 0013 FP16-INT8 - CPU renaissance: Apache Spark PageRank renaissance: Apache Spark Bayes renaissance: Savina Reactors.IO primesieve: 1e12 openvino: Age Gender Recognition Retail 0013 FP16-INT8 - CPU openvino: Age Gender Recognition Retail 0013 FP16 - CPU renaissance: Akka Unbalanced Cobwebbed Tree openvino: Age Gender Recognition Retail 0013 FP16 - CPU renaissance: ALS Movie Lens renaissance: Rand Forest blender: Barbershop - NVIDIA CUDA blender: BMW27 - CPU-Only blender: Classroom - NVIDIA OptiX primesieve: 1e13 blender: Barbershop - CPU-Only renaissance: Gaussian Mixture Model openvino: Person Detection FP16 - CPU openvino: Person Detection FP16 - CPU openvino: Vehicle Detection FP16-INT8 - CPU blender: BMW27 - NVIDIA OptiX openvino: Road Segmentation ADAS FP16 - CPU openvino: Machine Translation EN To DE FP16 - CPU openvino: Machine Translation EN To DE FP16 - CPU openvino: Road Segmentation ADAS FP16 - CPU openvino: Road Segmentation ADAS FP16-INT8 - CPU openvino: Road Segmentation ADAS FP16-INT8 - CPU openvino: Vehicle Detection FP16-INT8 - CPU blender: Junkshop - NVIDIA OptiX openvino: Face Detection FP16 - CPU openvino: Face Detection FP16-INT8 - CPU renaissance: Genetic Algorithm Using Jenetics + Futures openvino-genai: Gemma-7b-int4-ov - CPU openvino: Handwritten English Recognition FP16-INT8 - CPU openvino: Face Detection FP16-INT8 - CPU openvino: Handwritten English Recognition FP16-INT8 - CPU openvino: Person Vehicle Bike Detection FP16 - CPU blender: Fishy Cat - CPU-Only openvino: Person Vehicle Bike Detection FP16 - CPU openvino: Weld Porosity Detection FP16-INT8 - CPU blender: Classroom - CPU-Only openvino: Person Re-Identification Retail FP16 - CPU openvino: Face Detection FP16 - CPU openvino-genai: Phi-3-mini-128k-instruct-int4-ov - CPU openvino: Weld Porosity Detection FP16-INT8 - CPU openvino: Person Re-Identification Retail FP16 - CPU blender: BMW27 - NVIDIA CUDA blender: Pabellon Barcelona - CPU-Only openvino: Weld Porosity Detection FP16 - CPU openvino: Weld Porosity Detection FP16 - CPU openvino: Noise Suppression Poconet-Like FP16 - CPU openvino: Vehicle Detection FP16 - CPU openvino: Vehicle Detection FP16 - CPU openvino: Noise Suppression Poconet-Like FP16 - CPU openvino: Handwritten English Recognition FP16 - CPU openvino: Handwritten English Recognition FP16 - CPU openvino: Face Detection Retail FP16 - CPU blender: Pabellon Barcelona - NVIDIA OptiX openvino: Person Detection FP32 - CPU openvino: Face Detection Retail FP16 - CPU openvino: Person Detection FP32 - CPU blender: Junkshop - CPU-Only blender: Barbershop - NVIDIA OptiX vkpeak: int32-scalar vkpeak: int32-vec4 blender: Junkshop - NVIDIA CUDA blender: Classroom - NVIDIA CUDA openvino-genai: Falcon-7b-instruct-int4-ov - CPU vkpeak: fp16-scalar vkpeak: int16-scalar blender: Fishy Cat - NVIDIA CUDA blender: Fishy Cat - NVIDIA OptiX vkpeak: fp16-vec4 blender: Pabellon Barcelona - NVIDIA CUDA openvino-genai: TinyLlama-1.1B-Chat-v1.0 - CPU vkpeak: int16-vec4 vkpeak: fp32-scalar vkpeak: fp64-scalar vkpeak: fp64-vec4 vkpeak: fp32-vec4 openvino-genai: Phi-3-mini-128k-instruct-int4-ov - CPU - Time Per Output Token openvino-genai: Phi-3-mini-128k-instruct-int4-ov - CPU - Time To First Token openvino-genai: Falcon-7b-instruct-int4-ov - CPU - Time Per Output Token openvino-genai: Falcon-7b-instruct-int4-ov - CPU - Time To First Token openvino-genai: TinyLlama-1.1B-Chat-v1.0 - CPU - Time Per Output Token openvino-genai: TinyLlama-1.1B-Chat-v1.0 - CPU - Time To First Token openvino-genai: Gemma-7b-int4-ov - CPU - Time Per Output Token openvino-genai: Gemma-7b-int4-ov - CPU - Time To First Token renaissance: Apache Spark ALS a aa b c 356.7 1046.3 3005.7 2218.6 353.2 3855.6 6.637 3472.9 8326.4 393.1 48.64 86.406 3761.7 695.7 74.46 139.63 70.06 23401.3 23392.46 23384.11 15292.55 26846.85 17140.62 23473.4 735.97 735.92 30993.17 374.1 1043.0 2922.03 2990.0 1.96 0.35 2191.9 345.6 3802.0 6.783 39936.10 0.96 3503.8 15086.16 8417.6 391.2 69.34 49.32 11.77 87.499 511.92 3748.3 118.34 50.64 1239.77 5.18 167.54 79.99 74.91 35.76 12.21 488.58 4.78 9.80 6.51 252.76 699.3 14.10 379.60 23.69 42.11 7.43 74.79 802.60 2429.20 140.07 6.11 915.49 27.48 6.48 967.78 8.93 166.82 24.52 648.96 961.01 450.03 13.27 16.44 49.66 321.98 1760.28 13.00 118.80 3.33 50.43 69.93 47.02 23383.52 23363.19 15.32 16.92 20.70 23411.57 15286.16 16.98 8.78 26874.59 37.46 37.64 17127.56 23465.92 735.77 735.95 30988.00 36.39 139.57 48.30 87.61 26.57 37.76 70.94 348.75 380.1 1011.8 2943.91 3052.0 1.95 0.35 2158.4 352.1 3820.3 6.654 40039.83 0.96 3514.0 15058.2 8354.2 395.1 70.15 49.18 11.92 86.826 512.02 3722.9 118.01 50.78 1243.81 5.17 167.22 80.17 74.76 35.82 12.13 491.68 4.77 9.74 6.52 251.23 695.3 14.18 378.84 23.82 42.2 7.44 74.74 801.35 2437.82 140.2 6.1 913.66 27.61 6.47 969.67 8.91 166.41 24.5 649.83 959.74 451.38 13.24 16.46 49.57 322.54 1754.47 13.02 118.83 3.34 50.44 70.11 46.95 23364.4 23347.45 15.32 16.95 20.69 23384.23 15280.32 16.97 8.77 26846.25 37.43 37.67 17139.21 23476.74 735.98 735.92 30992.05 36.22 123.13 48.33 87.56 26.55 37.61 70.5 308.91 372.1 1060.1 3021.51 3085.3 1.9 0.34 2209.4 343.9 3899.9 6.626 40787.25 0.94 3542.8 15355.43 8459.8 397.3 69.11 48.82 11.76 86.554 505.73 3739.4 117.3 51.07 1249.45 5.21 166.27 80.59 74.36 36.02 12.13 491.35 4.75 9.74 6.55 251.7 698.6 14.18 380.92 23.77 41.97 7.47 74.39 798.32 2441.91 139.5 6.08 911.06 27.52 6.45 972.15 8.89 166.11 24.42 651.59 963.53 449.67 13.29 16.4 49.48 323.1 1754.47 12.98 118.47 3.34 50.56 69.93 46.9 23345.26 23345.11 15.29 16.95 20.72 23381.88 15299.24 16.96 8.78 26846.89 37.43 37.67 17137.18 23478.38 735.59 735.82 30990.96 36.34 121.96 48.25 87.16 26.55 37.41 70.5 295.64 OpenBenchmarking.org
Renaissance Test: Scala Dotty OpenBenchmarking.org ms, Fewer Is Better Renaissance 0.16 Test: Scala Dotty a aa b c 80 160 240 320 400 SE +/- 4.82, N = 15 356.7 374.1 380.1 372.1 MIN: 314.79 / MAX: 572.12 MIN: 310.63 / MAX: 579.42 MIN: 311.26 / MAX: 564.35 MIN: 328.33 / MAX: 541.75
Renaissance Test: Finagle HTTP Requests OpenBenchmarking.org ms, Fewer Is Better Renaissance 0.16 Test: Finagle HTTP Requests a aa b c 200 400 600 800 1000 SE +/- 12.36, N = 3 1046.3 1043.0 1011.8 1060.1 MIN: 970.98 / MAX: 1047.12 MIN: 947.76 / MAX: 1063.57 MIN: 958.73 / MAX: 1028.7 MIN: 966.02 / MAX: 1072.83
OpenVINO Model: Face Detection Retail FP16-INT8 - Device: CPU OpenBenchmarking.org FPS, More Is Better OpenVINO 2024.5 Model: Face Detection Retail FP16-INT8 - Device: CPU aa b c 600 1200 1800 2400 3000 SE +/- 1.66, N = 3 2922.03 2943.91 3021.51 1. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl -lstdc++fs
Renaissance Test: In-Memory Database Shootout OpenBenchmarking.org ms, Fewer Is Better Renaissance 0.16 Test: In-Memory Database Shootout a aa b c 700 1400 2100 2800 3500 SE +/- 34.58, N = 4 3005.7 2990.0 3052.0 3085.3 MIN: 2879.35 / MAX: 3300.02 MIN: 2681.96 / MAX: 3324.34 MIN: 2934.84 / MAX: 3173.17 MIN: 2964.43 / MAX: 3292.26
OpenVINO Model: Face Detection Retail FP16-INT8 - Device: CPU OpenBenchmarking.org ms, Fewer Is Better OpenVINO 2024.5 Model: Face Detection Retail FP16-INT8 - Device: CPU aa b c 0.441 0.882 1.323 1.764 2.205 SE +/- 0.00, N = 3 1.96 1.95 1.90 MIN: 1.03 / MAX: 13.23 MIN: 1.03 / MAX: 8.64 MIN: 1.03 / MAX: 8.09 1. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl -lstdc++fs
OpenVINO Model: Age Gender Recognition Retail 0013 FP16-INT8 - Device: CPU OpenBenchmarking.org ms, Fewer Is Better OpenVINO 2024.5 Model: Age Gender Recognition Retail 0013 FP16-INT8 - Device: CPU aa b c 0.0788 0.1576 0.2364 0.3152 0.394 SE +/- 0.00, N = 3 0.35 0.35 0.34 MIN: 0.23 / MAX: 7.42 MIN: 0.23 / MAX: 7.74 MIN: 0.23 / MAX: 7.38 1. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl -lstdc++fs
Renaissance Test: Apache Spark PageRank OpenBenchmarking.org ms, Fewer Is Better Renaissance 0.16 Test: Apache Spark PageRank a aa b c 500 1000 1500 2000 2500 SE +/- 10.75, N = 3 2218.6 2191.9 2158.4 2209.4 MIN: 1633.15 MIN: 1595.5 / MAX: 2209.7 MIN: 1595.87 / MAX: 2158.42 MIN: 1619.41
Renaissance Test: Apache Spark Bayes OpenBenchmarking.org ms, Fewer Is Better Renaissance 0.16 Test: Apache Spark Bayes a aa b c 80 160 240 320 400 SE +/- 2.29, N = 3 353.2 345.6 352.1 343.9 MIN: 330.84 / MAX: 398.86 MIN: 330.79 / MAX: 376.95 MIN: 332.91 / MAX: 374.03 MIN: 331.51 / MAX: 372.63
Renaissance Test: Savina Reactors.IO OpenBenchmarking.org ms, Fewer Is Better Renaissance 0.16 Test: Savina Reactors.IO a aa b c 800 1600 2400 3200 4000 SE +/- 8.00, N = 3 3855.6 3802.0 3820.3 3899.9 MIN: 3855.59 / MAX: 5136.5 MIN: 3792.83 / MAX: 5101.04 MAX: 4954.7 MAX: 5066.66
Primesieve Length: 1e12 OpenBenchmarking.org Seconds, Fewer Is Better Primesieve 12.6 Length: 1e12 a aa b c 2 4 6 8 10 SE +/- 0.066, N = 3 6.637 6.783 6.654 6.626 1. (CXX) g++ options: -O3
OpenVINO Model: Age Gender Recognition Retail 0013 FP16-INT8 - Device: CPU OpenBenchmarking.org FPS, More Is Better OpenVINO 2024.5 Model: Age Gender Recognition Retail 0013 FP16-INT8 - Device: CPU aa b c 9K 18K 27K 36K 45K SE +/- 185.94, N = 3 39936.10 40039.83 40787.25 1. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl -lstdc++fs
OpenVINO Model: Age Gender Recognition Retail 0013 FP16 - Device: CPU OpenBenchmarking.org ms, Fewer Is Better OpenVINO 2024.5 Model: Age Gender Recognition Retail 0013 FP16 - Device: CPU aa b c 0.216 0.432 0.648 0.864 1.08 SE +/- 0.00, N = 3 0.96 0.96 0.94 MIN: 0.59 / MAX: 8.81 MIN: 0.59 / MAX: 14.25 MIN: 0.59 / MAX: 8 1. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl -lstdc++fs
Renaissance Test: Akka Unbalanced Cobwebbed Tree OpenBenchmarking.org ms, Fewer Is Better Renaissance 0.16 Test: Akka Unbalanced Cobwebbed Tree a aa b c 800 1600 2400 3200 4000 SE +/- 14.16, N = 3 3472.9 3503.8 3514.0 3542.8 MAX: 4304.43 MIN: 3475.86 / MAX: 4427.25 MIN: 3513.99 / MAX: 4408.04 MAX: 4342.51
OpenVINO Model: Age Gender Recognition Retail 0013 FP16 - Device: CPU OpenBenchmarking.org FPS, More Is Better OpenVINO 2024.5 Model: Age Gender Recognition Retail 0013 FP16 - Device: CPU aa b c 3K 6K 9K 12K 15K SE +/- 24.22, N = 3 15086.16 15058.20 15355.43 1. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl -lstdc++fs
Renaissance Test: ALS Movie Lens OpenBenchmarking.org ms, Fewer Is Better Renaissance 0.16 Test: ALS Movie Lens a aa b c 2K 4K 6K 8K 10K SE +/- 72.41, N = 3 8326.4 8417.6 8354.2 8459.8 MIN: 7832.53 / MAX: 8748.58 MIN: 7832.38 / MAX: 8701.39 MIN: 7901.13 / MAX: 8415.78 MIN: 8160.64 / MAX: 8649.1
Renaissance Test: Random Forest OpenBenchmarking.org ms, Fewer Is Better Renaissance 0.16 Test: Random Forest a aa b c 90 180 270 360 450 SE +/- 4.00, N = 3 393.1 391.2 395.1 397.3 MIN: 348.04 / MAX: 437.69 MIN: 339.52 / MAX: 449.68 MIN: 343.19 / MAX: 463.11 MIN: 359.4 / MAX: 454.67
Blender Blend File: Barbershop - Compute: NVIDIA CUDA OpenBenchmarking.org Seconds, Fewer Is Better Blender 4.3 Blend File: Barbershop - Compute: NVIDIA CUDA aa b c 16 32 48 64 80 SE +/- 0.05, N = 3 69.34 70.15 69.11
Blender Blend File: BMW27 - Compute: CPU-Only OpenBenchmarking.org Seconds, Fewer Is Better Blender 4.3 Blend File: BMW27 - Compute: CPU-Only a aa b c 11 22 33 44 55 SE +/- 0.10, N = 3 48.64 49.32 49.18 48.82
Blender Blend File: Classroom - Compute: NVIDIA OptiX OpenBenchmarking.org Seconds, Fewer Is Better Blender 4.3 Blend File: Classroom - Compute: NVIDIA OptiX aa b c 3 6 9 12 15 SE +/- 0.02, N = 3 11.77 11.92 11.76
Primesieve Length: 1e13 OpenBenchmarking.org Seconds, Fewer Is Better Primesieve 12.6 Length: 1e13 a aa b c 20 40 60 80 100 SE +/- 0.19, N = 3 86.41 87.50 86.83 86.55 1. (CXX) g++ options: -O3
Blender Blend File: Barbershop - Compute: CPU-Only OpenBenchmarking.org Seconds, Fewer Is Better Blender 4.3 Blend File: Barbershop - Compute: CPU-Only aa b c 110 220 330 440 550 SE +/- 0.48, N = 3 511.92 512.02 505.73
Renaissance Test: Gaussian Mixture Model OpenBenchmarking.org ms, Fewer Is Better Renaissance 0.16 Test: Gaussian Mixture Model a aa b c 800 1600 2400 3200 4000 SE +/- 17.46, N = 3 3761.7 3748.3 3722.9 3739.4 MIN: 2716.5 / MAX: 3761.71 MIN: 2645.79 / MAX: 3781.01 MIN: 2677.26 / MAX: 3722.94 MIN: 2686.88 / MAX: 3739.42
OpenVINO Model: Person Detection FP16 - Device: CPU OpenBenchmarking.org ms, Fewer Is Better OpenVINO 2024.5 Model: Person Detection FP16 - Device: CPU aa b c 30 60 90 120 150 SE +/- 0.09, N = 3 118.34 118.01 117.30 MIN: 61.51 / MAX: 185.26 MIN: 65.25 / MAX: 177.75 MIN: 61.85 / MAX: 184.09 1. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl -lstdc++fs
OpenVINO Model: Person Detection FP16 - Device: CPU OpenBenchmarking.org FPS, More Is Better OpenVINO 2024.5 Model: Person Detection FP16 - Device: CPU aa b c 12 24 36 48 60 SE +/- 0.04, N = 3 50.64 50.78 51.07 1. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl -lstdc++fs
OpenVINO Model: Vehicle Detection FP16-INT8 - Device: CPU OpenBenchmarking.org FPS, More Is Better OpenVINO 2024.5 Model: Vehicle Detection FP16-INT8 - Device: CPU aa b c 300 600 900 1200 1500 SE +/- 4.93, N = 3 1239.77 1243.81 1249.45 1. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl -lstdc++fs
Blender Blend File: BMW27 - Compute: NVIDIA OptiX OpenBenchmarking.org Seconds, Fewer Is Better Blender 4.3 Blend File: BMW27 - Compute: NVIDIA OptiX aa b c 1.1723 2.3446 3.5169 4.6892 5.8615 SE +/- 0.04, N = 13 5.18 5.17 5.21
OpenVINO Model: Road Segmentation ADAS FP16 - Device: CPU OpenBenchmarking.org FPS, More Is Better OpenVINO 2024.5 Model: Road Segmentation ADAS FP16 - Device: CPU aa b c 40 80 120 160 200 SE +/- 0.10, N = 3 167.54 167.22 166.27 1. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl -lstdc++fs
OpenVINO Model: Machine Translation EN To DE FP16 - Device: CPU OpenBenchmarking.org FPS, More Is Better OpenVINO 2024.5 Model: Machine Translation EN To DE FP16 - Device: CPU aa b c 20 40 60 80 100 SE +/- 0.11, N = 3 79.99 80.17 80.59 1. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl -lstdc++fs
OpenVINO Model: Machine Translation EN To DE FP16 - Device: CPU OpenBenchmarking.org ms, Fewer Is Better OpenVINO 2024.5 Model: Machine Translation EN To DE FP16 - Device: CPU aa b c 20 40 60 80 100 SE +/- 0.10, N = 3 74.91 74.76 74.36 MIN: 47.45 / MAX: 125.1 MIN: 47.89 / MAX: 124.4 MIN: 47.32 / MAX: 118.58 1. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl -lstdc++fs
OpenVINO Model: Road Segmentation ADAS FP16 - Device: CPU OpenBenchmarking.org ms, Fewer Is Better OpenVINO 2024.5 Model: Road Segmentation ADAS FP16 - Device: CPU aa b c 8 16 24 32 40 SE +/- 0.02, N = 3 35.76 35.82 36.02 MIN: 13.85 / MAX: 68.74 MIN: 14.06 / MAX: 65.07 MIN: 14 / MAX: 64.19 1. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl -lstdc++fs
OpenVINO Model: Road Segmentation ADAS FP16-INT8 - Device: CPU OpenBenchmarking.org ms, Fewer Is Better OpenVINO 2024.5 Model: Road Segmentation ADAS FP16-INT8 - Device: CPU aa b c 3 6 9 12 15 SE +/- 0.05, N = 3 12.21 12.13 12.13 MIN: 7.15 / MAX: 34.14 MIN: 7.17 / MAX: 30.85 MIN: 7.17 / MAX: 30.12 1. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl -lstdc++fs
OpenVINO Model: Road Segmentation ADAS FP16-INT8 - Device: CPU OpenBenchmarking.org FPS, More Is Better OpenVINO 2024.5 Model: Road Segmentation ADAS FP16-INT8 - Device: CPU aa b c 110 220 330 440 550 SE +/- 1.99, N = 3 488.58 491.68 491.35 1. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl -lstdc++fs
OpenVINO Model: Vehicle Detection FP16-INT8 - Device: CPU OpenBenchmarking.org ms, Fewer Is Better OpenVINO 2024.5 Model: Vehicle Detection FP16-INT8 - Device: CPU aa b c 1.0755 2.151 3.2265 4.302 5.3775 SE +/- 0.02, N = 3 4.78 4.77 4.75 MIN: 2.72 / MAX: 19.75 MIN: 2.73 / MAX: 14.77 MIN: 2.72 / MAX: 13.93 1. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl -lstdc++fs
Blender Blend File: Junkshop - Compute: NVIDIA OptiX OpenBenchmarking.org Seconds, Fewer Is Better Blender 4.3 Blend File: Junkshop - Compute: NVIDIA OptiX aa b c 3 6 9 12 15 SE +/- 0.11, N = 4 9.80 9.74 9.74
OpenVINO Model: Face Detection FP16 - Device: CPU OpenBenchmarking.org FPS, More Is Better OpenVINO 2024.5 Model: Face Detection FP16 - Device: CPU aa b c 2 4 6 8 10 SE +/- 0.03, N = 3 6.51 6.52 6.55 1. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl -lstdc++fs
OpenVINO Model: Face Detection FP16-INT8 - Device: CPU OpenBenchmarking.org ms, Fewer Is Better OpenVINO 2024.5 Model: Face Detection FP16-INT8 - Device: CPU aa b c 60 120 180 240 300 SE +/- 0.70, N = 3 252.76 251.23 251.70 MIN: 162.35 / MAX: 383.62 MIN: 164.3 / MAX: 390.35 MIN: 159.88 / MAX: 505.02 1. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl -lstdc++fs
Renaissance Test: Genetic Algorithm Using Jenetics + Futures OpenBenchmarking.org ms, Fewer Is Better Renaissance 0.16 Test: Genetic Algorithm Using Jenetics + Futures a aa b c 150 300 450 600 750 SE +/- 2.18, N = 3 695.7 699.3 695.3 698.6 MIN: 678.55 / MAX: 727.82 MIN: 680.86 / MAX: 729.65 MIN: 679.26 / MAX: 721.02 MIN: 686.88 / MAX: 715.07
OpenVINO GenAI Model: Gemma-7b-int4-ov - Device: CPU OpenBenchmarking.org tokens/s, More Is Better OpenVINO GenAI 2024.5 Model: Gemma-7b-int4-ov - Device: CPU aa b c 4 8 12 16 20 SE +/- 0.02, N = 3 14.10 14.18 14.18
OpenVINO Model: Handwritten English Recognition FP16-INT8 - Device: CPU OpenBenchmarking.org FPS, More Is Better OpenVINO 2024.5 Model: Handwritten English Recognition FP16-INT8 - Device: CPU aa b c 80 160 240 320 400 SE +/- 0.35, N = 3 379.60 378.84 380.92 1. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl -lstdc++fs
OpenVINO Model: Face Detection FP16-INT8 - Device: CPU OpenBenchmarking.org FPS, More Is Better OpenVINO 2024.5 Model: Face Detection FP16-INT8 - Device: CPU aa b c 6 12 18 24 30 SE +/- 0.07, N = 3 23.69 23.82 23.77 1. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl -lstdc++fs
OpenVINO Model: Handwritten English Recognition FP16-INT8 - Device: CPU OpenBenchmarking.org ms, Fewer Is Better OpenVINO 2024.5 Model: Handwritten English Recognition FP16-INT8 - Device: CPU aa b c 10 20 30 40 50 SE +/- 0.04, N = 3 42.11 42.20 41.97 MIN: 34.54 / MAX: 67.63 MIN: 34.58 / MAX: 72.64 MIN: 34.55 / MAX: 61.29 1. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl -lstdc++fs
OpenVINO Model: Person Vehicle Bike Detection FP16 - Device: CPU OpenBenchmarking.org ms, Fewer Is Better OpenVINO 2024.5 Model: Person Vehicle Bike Detection FP16 - Device: CPU aa b c 2 4 6 8 10 SE +/- 0.01, N = 3 7.43 7.44 7.47 MIN: 4.78 / MAX: 22.13 MIN: 4.83 / MAX: 16.13 MIN: 4.79 / MAX: 16.87 1. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl -lstdc++fs
Blender Blend File: Fishy Cat - Compute: CPU-Only OpenBenchmarking.org Seconds, Fewer Is Better Blender 4.3 Blend File: Fishy Cat - Compute: CPU-Only a aa b c 20 40 60 80 100 SE +/- 0.19, N = 3 74.46 74.79 74.74 74.39
OpenVINO Model: Person Vehicle Bike Detection FP16 - Device: CPU OpenBenchmarking.org FPS, More Is Better OpenVINO 2024.5 Model: Person Vehicle Bike Detection FP16 - Device: CPU aa b c 200 400 600 800 1000 SE +/- 1.01, N = 3 802.60 801.35 798.32 1. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl -lstdc++fs
OpenVINO Model: Weld Porosity Detection FP16-INT8 - Device: CPU OpenBenchmarking.org FPS, More Is Better OpenVINO 2024.5 Model: Weld Porosity Detection FP16-INT8 - Device: CPU aa b c 500 1000 1500 2000 2500 SE +/- 2.42, N = 3 2429.20 2437.82 2441.91 1. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl -lstdc++fs
Blender Blend File: Classroom - Compute: CPU-Only OpenBenchmarking.org Seconds, Fewer Is Better Blender 4.3 Blend File: Classroom - Compute: CPU-Only a aa b c 30 60 90 120 150 SE +/- 0.44, N = 3 139.63 140.07 140.20 139.50
OpenVINO Model: Person Re-Identification Retail FP16 - Device: CPU OpenBenchmarking.org ms, Fewer Is Better OpenVINO 2024.5 Model: Person Re-Identification Retail FP16 - Device: CPU aa b c 2 4 6 8 10 SE +/- 0.00, N = 3 6.11 6.10 6.08 MIN: 4.06 / MAX: 15.69 MIN: 4.08 / MAX: 15.79 MIN: 4.07 / MAX: 16.28 1. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl -lstdc++fs
OpenVINO Model: Face Detection FP16 - Device: CPU OpenBenchmarking.org ms, Fewer Is Better OpenVINO 2024.5 Model: Face Detection FP16 - Device: CPU aa b c 200 400 600 800 1000 SE +/- 3.02, N = 3 915.49 913.66 911.06 MIN: 627.29 / MAX: 1324.9 MIN: 624.01 / MAX: 1326.84 MIN: 624.05 / MAX: 1319.97 1. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl -lstdc++fs
OpenVINO GenAI Model: Phi-3-mini-128k-instruct-int4-ov - Device: CPU OpenBenchmarking.org tokens/s, More Is Better OpenVINO GenAI 2024.5 Model: Phi-3-mini-128k-instruct-int4-ov - Device: CPU aa b c 6 12 18 24 30 SE +/- 0.04, N = 3 27.48 27.61 27.52
OpenVINO Model: Weld Porosity Detection FP16-INT8 - Device: CPU OpenBenchmarking.org ms, Fewer Is Better OpenVINO 2024.5 Model: Weld Porosity Detection FP16-INT8 - Device: CPU aa b c 2 4 6 8 10 SE +/- 0.01, N = 3 6.48 6.47 6.45 MIN: 4.94 / MAX: 20.98 MIN: 4.93 / MAX: 19.26 MIN: 4.94 / MAX: 15.49 1. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl -lstdc++fs
OpenVINO Model: Person Re-Identification Retail FP16 - Device: CPU OpenBenchmarking.org FPS, More Is Better OpenVINO 2024.5 Model: Person Re-Identification Retail FP16 - Device: CPU aa b c 200 400 600 800 1000 SE +/- 0.36, N = 3 967.78 969.67 972.15 1. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl -lstdc++fs
Blender Blend File: BMW27 - Compute: NVIDIA CUDA OpenBenchmarking.org Seconds, Fewer Is Better Blender 4.3 Blend File: BMW27 - Compute: NVIDIA CUDA aa b c 2 4 6 8 10 SE +/- 0.02, N = 3 8.93 8.91 8.89
Blender Blend File: Pabellon Barcelona - Compute: CPU-Only OpenBenchmarking.org Seconds, Fewer Is Better Blender 4.3 Blend File: Pabellon Barcelona - Compute: CPU-Only aa b c 40 80 120 160 200 SE +/- 0.30, N = 3 166.82 166.41 166.11
OpenVINO Model: Weld Porosity Detection FP16 - Device: CPU OpenBenchmarking.org ms, Fewer Is Better OpenVINO 2024.5 Model: Weld Porosity Detection FP16 - Device: CPU aa b c 6 12 18 24 30 SE +/- 0.02, N = 3 24.52 24.50 24.42 MIN: 18.82 / MAX: 39.72 MIN: 18.99 / MAX: 38.98 MIN: 18.97 / MAX: 42.61 1. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl -lstdc++fs
OpenVINO Model: Weld Porosity Detection FP16 - Device: CPU OpenBenchmarking.org FPS, More Is Better OpenVINO 2024.5 Model: Weld Porosity Detection FP16 - Device: CPU aa b c 140 280 420 560 700 SE +/- 0.78, N = 3 648.96 649.83 651.59 1. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl -lstdc++fs
OpenVINO Model: Noise Suppression Poconet-Like FP16 - Device: CPU OpenBenchmarking.org FPS, More Is Better OpenVINO 2024.5 Model: Noise Suppression Poconet-Like FP16 - Device: CPU aa b c 200 400 600 800 1000 SE +/- 1.81, N = 3 961.01 959.74 963.53 1. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl -lstdc++fs
OpenVINO Model: Vehicle Detection FP16 - Device: CPU OpenBenchmarking.org FPS, More Is Better OpenVINO 2024.5 Model: Vehicle Detection FP16 - Device: CPU aa b c 100 200 300 400 500 SE +/- 0.62, N = 3 450.03 451.38 449.67 1. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl -lstdc++fs
OpenVINO Model: Vehicle Detection FP16 - Device: CPU OpenBenchmarking.org ms, Fewer Is Better OpenVINO 2024.5 Model: Vehicle Detection FP16 - Device: CPU aa b c 3 6 9 12 15 SE +/- 0.02, N = 3 13.27 13.24 13.29 MIN: 6.33 / MAX: 29.46 MIN: 6.36 / MAX: 26.02 MIN: 6.36 / MAX: 24.82 1. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl -lstdc++fs
OpenVINO Model: Noise Suppression Poconet-Like FP16 - Device: CPU OpenBenchmarking.org ms, Fewer Is Better OpenVINO 2024.5 Model: Noise Suppression Poconet-Like FP16 - Device: CPU aa b c 4 8 12 16 20 SE +/- 0.03, N = 3 16.44 16.46 16.40 MIN: 12.76 / MAX: 28.04 MIN: 13.06 / MAX: 29.49 MIN: 12.8 / MAX: 27.92 1. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl -lstdc++fs
OpenVINO Model: Handwritten English Recognition FP16 - Device: CPU OpenBenchmarking.org ms, Fewer Is Better OpenVINO 2024.5 Model: Handwritten English Recognition FP16 - Device: CPU aa b c 11 22 33 44 55 SE +/- 0.08, N = 3 49.66 49.57 49.48 MIN: 39.94 / MAX: 79.52 MIN: 40.31 / MAX: 72.84 MIN: 39.94 / MAX: 72.51 1. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl -lstdc++fs
OpenVINO Model: Handwritten English Recognition FP16 - Device: CPU OpenBenchmarking.org FPS, More Is Better OpenVINO 2024.5 Model: Handwritten English Recognition FP16 - Device: CPU aa b c 70 140 210 280 350 SE +/- 0.53, N = 3 321.98 322.54 323.10 1. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl -lstdc++fs
OpenVINO Model: Face Detection Retail FP16 - Device: CPU OpenBenchmarking.org FPS, More Is Better OpenVINO 2024.5 Model: Face Detection Retail FP16 - Device: CPU aa b c 400 800 1200 1600 2000 SE +/- 0.98, N = 3 1760.28 1754.47 1754.47 1. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl -lstdc++fs
Blender Blend File: Pabellon Barcelona - Compute: NVIDIA OptiX OpenBenchmarking.org Seconds, Fewer Is Better Blender 4.3 Blend File: Pabellon Barcelona - Compute: NVIDIA OptiX aa b c 3 6 9 12 15 SE +/- 0.02, N = 3 13.00 13.02 12.98
OpenVINO Model: Person Detection FP32 - Device: CPU OpenBenchmarking.org ms, Fewer Is Better OpenVINO 2024.5 Model: Person Detection FP32 - Device: CPU aa b c 30 60 90 120 150 SE +/- 0.13, N = 3 118.80 118.83 118.47 MIN: 60.56 / MAX: 183.24 MIN: 61.78 / MAX: 181.12 MIN: 60.72 / MAX: 182.58 1. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl -lstdc++fs
OpenVINO Model: Face Detection Retail FP16 - Device: CPU OpenBenchmarking.org ms, Fewer Is Better OpenVINO 2024.5 Model: Face Detection Retail FP16 - Device: CPU aa b c 0.7515 1.503 2.2545 3.006 3.7575 SE +/- 0.00, N = 3 3.33 3.34 3.34 MIN: 2.03 / MAX: 17.62 MIN: 2.03 / MAX: 10.68 MIN: 2.03 / MAX: 10.57 1. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl -lstdc++fs
OpenVINO Model: Person Detection FP32 - Device: CPU OpenBenchmarking.org FPS, More Is Better OpenVINO 2024.5 Model: Person Detection FP32 - Device: CPU aa b c 11 22 33 44 55 SE +/- 0.06, N = 3 50.43 50.44 50.56 1. (CXX) g++ options: -fPIC -fsigned-char -ffunction-sections -fdata-sections -O3 -fno-strict-overflow -fwrapv -shared -ldl -lstdc++fs
Blender Blend File: Junkshop - Compute: CPU-Only OpenBenchmarking.org Seconds, Fewer Is Better Blender 4.3 Blend File: Junkshop - Compute: CPU-Only a aa b c 16 32 48 64 80 SE +/- 0.20, N = 3 70.06 69.93 70.11 69.93
Blender Blend File: Barbershop - Compute: NVIDIA OptiX OpenBenchmarking.org Seconds, Fewer Is Better Blender 4.3 Blend File: Barbershop - Compute: NVIDIA OptiX aa b c 11 22 33 44 55 SE +/- 0.05, N = 3 47.02 46.95 46.90
vkpeak int32-scalar OpenBenchmarking.org GIOPS, More Is Better vkpeak 20240505 int32-scalar a aa b c 5K 10K 15K 20K 25K SE +/- 18.61, N = 3 23401.30 23383.52 23364.40 23345.26
vkpeak int32-vec4 OpenBenchmarking.org GIOPS, More Is Better vkpeak 20240505 int32-vec4 a aa b c 5K 10K 15K 20K 25K SE +/- 15.15, N = 3 23392.46 23363.19 23347.45 23345.11
Blender Blend File: Junkshop - Compute: NVIDIA CUDA OpenBenchmarking.org Seconds, Fewer Is Better Blender 4.3 Blend File: Junkshop - Compute: NVIDIA CUDA aa b c 4 8 12 16 20 SE +/- 0.01, N = 3 15.32 15.32 15.29
Blender Blend File: Classroom - Compute: NVIDIA CUDA OpenBenchmarking.org Seconds, Fewer Is Better Blender 4.3 Blend File: Classroom - Compute: NVIDIA CUDA aa b c 4 8 12 16 20 SE +/- 0.01, N = 3 16.92 16.95 16.95
OpenVINO GenAI Model: Falcon-7b-instruct-int4-ov - Device: CPU OpenBenchmarking.org tokens/s, More Is Better OpenVINO GenAI 2024.5 Model: Falcon-7b-instruct-int4-ov - Device: CPU aa b c 5 10 15 20 25 SE +/- 0.02, N = 3 20.70 20.69 20.72
vkpeak fp16-scalar OpenBenchmarking.org GFLOPS, More Is Better vkpeak 20240505 fp16-scalar a aa b c 5K 10K 15K 20K 25K SE +/- 28.91, N = 3 23384.11 23411.57 23384.23 23381.88
vkpeak int16-scalar OpenBenchmarking.org GIOPS, More Is Better vkpeak 20240505 int16-scalar a aa b c 3K 6K 9K 12K 15K SE +/- 2.13, N = 3 15292.55 15286.16 15280.32 15299.24
Blender Blend File: Fishy Cat - Compute: NVIDIA CUDA OpenBenchmarking.org Seconds, Fewer Is Better Blender 4.3 Blend File: Fishy Cat - Compute: NVIDIA CUDA aa b c 4 8 12 16 20 SE +/- 0.01, N = 3 16.98 16.97 16.96
Blender Blend File: Fishy Cat - Compute: NVIDIA OptiX OpenBenchmarking.org Seconds, Fewer Is Better Blender 4.3 Blend File: Fishy Cat - Compute: NVIDIA OptiX aa b c 2 4 6 8 10 SE +/- 0.01, N = 3 8.78 8.77 8.78
vkpeak fp16-vec4 OpenBenchmarking.org GFLOPS, More Is Better vkpeak 20240505 fp16-vec4 a aa b c 6K 12K 18K 24K 30K SE +/- 24.97, N = 3 26846.85 26874.59 26846.25 26846.89
Blender Blend File: Pabellon Barcelona - Compute: NVIDIA CUDA OpenBenchmarking.org Seconds, Fewer Is Better Blender 4.3 Blend File: Pabellon Barcelona - Compute: NVIDIA CUDA aa b c 9 18 27 36 45 SE +/- 0.04, N = 3 37.46 37.43 37.43
OpenVINO GenAI Model: TinyLlama-1.1B-Chat-v1.0 - Device: CPU OpenBenchmarking.org tokens/s, More Is Better OpenVINO GenAI 2024.5 Model: TinyLlama-1.1B-Chat-v1.0 - Device: CPU aa b c 9 18 27 36 45 SE +/- 0.01, N = 3 37.64 37.67 37.67
vkpeak int16-vec4 OpenBenchmarking.org GIOPS, More Is Better vkpeak 20240505 int16-vec4 a aa b c 4K 8K 12K 16K 20K SE +/- 8.92, N = 3 17140.62 17127.56 17139.21 17137.18
vkpeak fp32-scalar OpenBenchmarking.org GFLOPS, More Is Better vkpeak 20240505 fp32-scalar a aa b c 5K 10K 15K 20K 25K SE +/- 5.72, N = 3 23473.40 23465.92 23476.74 23478.38
vkpeak fp64-scalar OpenBenchmarking.org GFLOPS, More Is Better vkpeak 20240505 fp64-scalar a aa b c 160 320 480 640 800 SE +/- 0.09, N = 3 735.97 735.77 735.98 735.59
vkpeak fp64-vec4 OpenBenchmarking.org GFLOPS, More Is Better vkpeak 20240505 fp64-vec4 a aa b c 160 320 480 640 800 SE +/- 0.01, N = 3 735.92 735.95 735.92 735.82
vkpeak fp32-vec4 OpenBenchmarking.org GFLOPS, More Is Better vkpeak 20240505 fp32-vec4 a aa b c 7K 14K 21K 28K 35K SE +/- 0.50, N = 3 30993.17 30988.00 30992.05 30990.96
OpenVINO GenAI Model: Phi-3-mini-128k-instruct-int4-ov - Device: CPU - Time Per Output Token OpenBenchmarking.org ms, Fewer Is Better OpenVINO GenAI 2024.5 Model: Phi-3-mini-128k-instruct-int4-ov - Device: CPU - Time Per Output Token aa b c 8 16 24 32 40 SE +/- 0.06, N = 3 36.39 36.22 36.34
OpenVINO GenAI Model: Phi-3-mini-128k-instruct-int4-ov - Device: CPU - Time To First Token OpenBenchmarking.org ms, Fewer Is Better OpenVINO GenAI 2024.5 Model: Phi-3-mini-128k-instruct-int4-ov - Device: CPU - Time To First Token aa b c 30 60 90 120 150 SE +/- 8.87, N = 3 139.57 123.13 121.96
OpenVINO GenAI Model: Falcon-7b-instruct-int4-ov - Device: CPU - Time Per Output Token OpenBenchmarking.org ms, Fewer Is Better OpenVINO GenAI 2024.5 Model: Falcon-7b-instruct-int4-ov - Device: CPU - Time Per Output Token aa b c 11 22 33 44 55 SE +/- 0.04, N = 3 48.30 48.33 48.25
OpenVINO GenAI Model: Falcon-7b-instruct-int4-ov - Device: CPU - Time To First Token OpenBenchmarking.org ms, Fewer Is Better OpenVINO GenAI 2024.5 Model: Falcon-7b-instruct-int4-ov - Device: CPU - Time To First Token aa b c 20 40 60 80 100 SE +/- 0.86, N = 3 87.61 87.56 87.16
OpenVINO GenAI Model: TinyLlama-1.1B-Chat-v1.0 - Device: CPU - Time Per Output Token OpenBenchmarking.org ms, Fewer Is Better OpenVINO GenAI 2024.5 Model: TinyLlama-1.1B-Chat-v1.0 - Device: CPU - Time Per Output Token aa b c 6 12 18 24 30 SE +/- 0.01, N = 3 26.57 26.55 26.55
OpenVINO GenAI Model: TinyLlama-1.1B-Chat-v1.0 - Device: CPU - Time To First Token OpenBenchmarking.org ms, Fewer Is Better OpenVINO GenAI 2024.5 Model: TinyLlama-1.1B-Chat-v1.0 - Device: CPU - Time To First Token aa b c 9 18 27 36 45 SE +/- 0.21, N = 3 37.76 37.61 37.41
OpenVINO GenAI Model: Gemma-7b-int4-ov - Device: CPU - Time Per Output Token OpenBenchmarking.org ms, Fewer Is Better OpenVINO GenAI 2024.5 Model: Gemma-7b-int4-ov - Device: CPU - Time Per Output Token aa b c 16 32 48 64 80 SE +/- 0.09, N = 3 70.94 70.50 70.50
OpenVINO GenAI Model: Gemma-7b-int4-ov - Device: CPU - Time To First Token OpenBenchmarking.org ms, Fewer Is Better OpenVINO GenAI 2024.5 Model: Gemma-7b-int4-ov - Device: CPU - Time To First Token aa b c 80 160 240 320 400 SE +/- 61.61, N = 3 348.75 308.91 295.64
Phoronix Test Suite v10.8.5