vulkan-benchmarks AMD Ryzen 9 7950X 16-Core testing with a ASUS ROG STRIX X670E-E GAMING WIFI (1416 BIOS) and NVIDIA GeForce RTX 4090 24GB on Ubuntu 23.04 via the Phoronix Test Suite.
HTML result view exported from: https://openbenchmarking.org/result/2308069-PTS-VULKANBE16&gru&sor&rro .
vulkan-benchmarks Processor Motherboard Chipset Memory Disk Graphics Audio Monitor Network OS Kernel Desktop Display Server OpenGL Compiler File-System Screen Resolution Display Driver a b c d e f g h i 4080 4080 rep 4080 xxx 4080 zzz 3090 3090 rep 3070 RTX 3070 Ti 4090 4090 rep nv 4090 AMD Ryzen 9 7950X 16-Core @ 4.50GHz (16 Cores / 32 Threads) ASUS ROG STRIX X670E-E GAMING WIFI (1416 BIOS) AMD Device 14d8 32GB Western Digital WD_BLACK SN850X 1000GB + 4001GB AMD Radeon RX 6700 XT (2855/1000MHz) AMD Navi 21/23 ASUS MG28U Intel I225-V + Intel Wi-Fi 6 AX210/AX211/AX411 Ubuntu 23.04 6.4.6-060406-generic (x86_64) GNOME Shell 44.2 X Server 1.21.1.7 + Wayland 4.6 Mesa 23.3~git2307260600.87109c~oibaf~l (git-87109c3 2023-07-26 lunar-oibaf-ppa) (LLVM 15.0.7 DRM 3.52) GCC 12.2.0 ext4 3840x2160 MSI NVIDIA GeForce RTX 4060 8GB NVIDIA Device 22be X Server 1.21.1.7 NVIDIA 535.86.05 4.6.0 eVGA NVIDIA GeForce RTX 3060 12GB NVIDIA GA106 HD Audio NVIDIA GeForce RTX 3060 Ti 8GB NVIDIA GA104 HD Audio 2560x1440 NVIDIA GeForce RTX 4080 16GB NVIDIA Device 22bb 3840x2160 NVIDIA GeForce RTX 3090 24GB NVIDIA GA102 HD Audio NVIDIA GeForce RTX 3070 8GB NVIDIA GA104 HD Audio 2560x1440 NVIDIA GeForce RTX 3070 Ti 8GB NVIDIA GeForce RTX 4090 24GB NVIDIA AD102 HD Audio 3840x2160 OpenBenchmarking.org Kernel Details - Transparent Huge Pages: madvise Compiler Details - --build=x86_64-linux-gnu --disable-vtable-verify --disable-werror --enable-cet --enable-checking=release --enable-clocale=gnu --enable-default-pie --enable-gnu-unique-object --enable-languages=c,ada,c++,go,d,fortran,objc,obj-c++,m2 --enable-libphobos-checking=release --enable-libstdcxx-debug --enable-libstdcxx-time=yes --enable-multiarch --enable-multilib --enable-nls --enable-objc-gc=auto --enable-offload-defaulted --enable-offload-targets=nvptx-none=/build/gcc-12-Pa930Z/gcc-12-12.2.0/debian/tmp-nvptx/usr,amdgcn-amdhsa=/build/gcc-12-Pa930Z/gcc-12-12.2.0/debian/tmp-gcn/usr --enable-plugin --enable-shared --enable-threads=posix --host=x86_64-linux-gnu --program-prefix=x86_64-linux-gnu- --target=x86_64-linux-gnu --with-abi=m64 --with-arch-32=i686 --with-default-libstdcxx-abi=new --with-gcc-major-version-only --with-multilib-list=m32,m64,mx32 --with-target-system-zlib=auto --with-tune=generic --without-cuda-driver -v Processor Details - a: Scaling Governor: acpi-cpufreq schedutil (Boost: Enabled) - CPU Microcode: 0xa601203 - b: Scaling Governor: acpi-cpufreq schedutil (Boost: Enabled) - CPU Microcode: 0xa601203 - c: Scaling Governor: acpi-cpufreq schedutil (Boost: Enabled) - CPU Microcode: 0xa601203 - d: Scaling Governor: acpi-cpufreq performance (Boost: Enabled) - CPU Microcode: 0xa601203 - e: Scaling Governor: acpi-cpufreq performance (Boost: Enabled) - CPU Microcode: 0xa601203 - f: Scaling Governor: acpi-cpufreq performance (Boost: Enabled) - CPU Microcode: 0xa601203 - g: Scaling Governor: acpi-cpufreq performance (Boost: Enabled) - CPU Microcode: 0xa601203 - h: Scaling Governor: acpi-cpufreq performance (Boost: Enabled) - CPU Microcode: 0xa601203 - i: Scaling Governor: acpi-cpufreq performance (Boost: Enabled) - CPU Microcode: 0xa601203 - 4080: Scaling Governor: acpi-cpufreq performance (Boost: Enabled) - CPU Microcode: 0xa601203 - 4080 rep: Scaling Governor: acpi-cpufreq performance (Boost: Enabled) - CPU Microcode: 0xa601203 - 4080 xxx: Scaling Governor: acpi-cpufreq performance (Boost: Enabled) - CPU Microcode: 0xa601203 - 4080 zzz: Scaling Governor: acpi-cpufreq performance (Boost: Enabled) - CPU Microcode: 0xa601203 - 3090: Scaling Governor: acpi-cpufreq performance (Boost: Enabled) - CPU Microcode: 0xa601203 - 3090 rep: Scaling Governor: acpi-cpufreq performance (Boost: Enabled) - CPU Microcode: 0xa601203 - 3070: Scaling Governor: acpi-cpufreq performance (Boost: Enabled) - CPU Microcode: 0xa601203 - RTX 3070 Ti: Scaling Governor: acpi-cpufreq performance (Boost: Enabled) - CPU Microcode: 0xa601203 - 4090: Scaling Governor: acpi-cpufreq performance (Boost: Enabled) - CPU Microcode: 0xa601203 - 4090 rep: Scaling Governor: acpi-cpufreq performance (Boost: Enabled) - CPU Microcode: 0xa601203 - nv 4090: Scaling Governor: acpi-cpufreq performance (Boost: Enabled) - CPU Microcode: 0xa601203 Graphics Details - a: BAR1 / Visible vRAM Size: 12272 MB - vBIOS Version: 113-D5121100-101 - b: BAR1 / Visible vRAM Size: 12272 MB - vBIOS Version: 113-D5121100-101 - c: BAR1 / Visible vRAM Size: 12272 MB - vBIOS Version: 113-D5121100-101 - d: BAR1 / Visible vRAM Size: 8192 MiB - vBIOS Version: 95.07.31.00.e3 - e: BAR1 / Visible vRAM Size: 8192 MiB - vBIOS Version: 95.07.31.00.e3 - f: BAR1 / Visible vRAM Size: 16384 MiB - vBIOS Version: 94.06.14.40.46 - g: BAR1 / Visible vRAM Size: 16384 MiB - vBIOS Version: 94.06.14.40.46 - h: BAR1 / Visible vRAM Size: 16384 MiB - vBIOS Version: 94.06.14.40.46 - i: BAR1 / Visible vRAM Size: 8192 MiB - vBIOS Version: 94.04.25.00.2c - 4080: BAR1 / Visible vRAM Size: 16384 MiB - vBIOS Version: 95.03.0e.00.04 - 4080 rep: BAR1 / Visible vRAM Size: 16384 MiB - vBIOS Version: 95.03.0e.00.04 - 4080 xxx: BAR1 / Visible vRAM Size: 16384 MiB - vBIOS Version: 95.03.0e.00.04 - 4080 zzz: BAR1 / Visible vRAM Size: 16384 MiB - vBIOS Version: 95.03.0e.00.04 - 3090: BAR1 / Visible vRAM Size: 32768 MiB - vBIOS Version: 94.02.27.00.02 - 3090 rep: BAR1 / Visible vRAM Size: 32768 MiB - vBIOS Version: 94.02.27.00.02 - 3070: BAR1 / Visible vRAM Size: 8192 MiB - vBIOS Version: 94.04.25.00.2b - RTX 3070 Ti: BAR1 / Visible vRAM Size: 8192 MiB - vBIOS Version: 94.04.5b.00.02 - 4090: BAR1 / Visible vRAM Size: 32768 MiB - vBIOS Version: 95.02.20.00.01 - 4090 rep: BAR1 / Visible vRAM Size: 32768 MiB - vBIOS Version: 95.02.20.00.01 - nv 4090: BAR1 / Visible vRAM Size: 32768 MiB - vBIOS Version: 95.02.20.00.01 Security Details - itlb_multihit: Not affected + l1tf: Not affected + mds: Not affected + meltdown: Not affected + mmio_stale_data: Not affected + retbleed: Not affected + spec_store_bypass: Mitigation of SSB disabled via prctl + spectre_v1: Mitigation of usercopy/swapgs barriers and __user pointer sanitization + spectre_v2: Mitigation of Enhanced / Automatic IBRS IBPB: conditional RSB filling PBRSB-eIBRS: Not affected + srbds: Not affected + tsx_async_abort: Not affected
vulkan-benchmarks vkfft: FFT + iFFT R2C / C2R vkfft: FFT + iFFT C2C 1D batched in half precision vkfft: FFT + iFFT C2C Bluestein in single precision vkfft: FFT + iFFT C2C 1D batched in double precision vkfft: FFT + iFFT C2C 1D batched in single precision vkfft: FFT + iFFT C2C multidimensional in single precision vkfft: FFT + iFFT C2C Bluestein benchmark in double precision vkfft: FFT + iFFT C2C 1D batched in single precision, no reshuffling vkpeak: fp32-scalar vkpeak: fp32-vec4 vkpeak: fp16-scalar vkpeak: fp16-vec4 vkpeak: fp64-scalar vkpeak: fp64-vec4 vkpeak: int32-scalar vkpeak: int32-vec4 vkpeak: int16-scalar vkpeak: int16-vec4 vkresample: 2x - Single ncnn: CPU - mobilenet ncnn: CPU-v2-v2 - mobilenet-v2 ncnn: CPU - shufflenet-v2 ncnn: CPU - mnasnet ncnn: CPU - efficientnet-b0 ncnn: CPU - blazeface ncnn: CPU - googlenet ncnn: CPU - vgg16 ncnn: CPU - resnet18 ncnn: CPU - alexnet ncnn: CPU - resnet50 ncnn: CPU - yolov4-tiny ncnn: CPU - squeezenet_ssd ncnn: CPU - regnety_400m ncnn: CPU - vision_transformer ncnn: CPU - FastestDet ncnn: CPU-v3-v3 - mobilenet-v3 ncnn: Vulkan GPU - mobilenet ncnn: Vulkan GPU-v2-v2 - mobilenet-v2 ncnn: Vulkan GPU-v3-v3 - mobilenet-v3 ncnn: Vulkan GPU - shufflenet-v2 ncnn: Vulkan GPU - mnasnet ncnn: Vulkan GPU - efficientnet-b0 ncnn: Vulkan GPU - blazeface ncnn: Vulkan GPU - googlenet ncnn: Vulkan GPU - vgg16 ncnn: Vulkan GPU - resnet18 ncnn: Vulkan GPU - alexnet ncnn: Vulkan GPU - resnet50 ncnn: Vulkan GPU - yolov4-tiny ncnn: Vulkan GPU - squeezenet_ssd ncnn: Vulkan GPU - regnety_400m ncnn: Vulkan GPU - vision_transformer ncnn: Vulkan GPU - FastestDet ncnn: CPU-v3-v3-v3 - mobilenet ncnn: CPU-v3-v3-v3-v2-v2 - mobilenet-v2 ncnn: CPU-v3-v3-v3-v3-v3 - mobilenet-v3 ncnn: CPU-v3-v3-v3 - shufflenet-v2 ncnn: CPU-v3-v3-v3 - mnasnet ncnn: CPU-v3-v3-v3 - efficientnet-b0 ncnn: CPU-v3-v3-v3 - blazeface ncnn: CPU-v3-v3-v3 - googlenet ncnn: CPU-v3-v3-v3 - vgg16 ncnn: CPU-v3-v3-v3 - resnet18 ncnn: CPU-v3-v3-v3 - alexnet ncnn: CPU-v3-v3-v3 - resnet50 ncnn: CPU-v3-v3-v3 - yolov4-tiny ncnn: CPU-v3-v3-v3 - squeezenet_ssd ncnn: CPU-v3-v3-v3 - regnety_400m ncnn: CPU-v3-v3-v3 - vision_transformer ncnn: CPU-v3-v3-v3 - FastestDet ncnn: Vulkan GPU-v3-v3-v3 - mobilenet ncnn: Vulkan GPU-v3-v3-v3-v2-v2 - mobilenet-v2 ncnn: Vulkan GPU-v3-v3-v3-v3-v3 - mobilenet-v3 ncnn: Vulkan GPU-v3-v3-v3 - shufflenet-v2 ncnn: Vulkan GPU-v3-v3-v3 - mnasnet ncnn: Vulkan GPU-v3-v3-v3 - efficientnet-b0 ncnn: Vulkan GPU-v3-v3-v3 - blazeface ncnn: Vulkan GPU-v3-v3-v3 - googlenet ncnn: Vulkan GPU-v3-v3-v3 - vgg16 ncnn: Vulkan GPU-v3-v3-v3 - resnet18 ncnn: Vulkan GPU-v3-v3-v3 - alexnet ncnn: Vulkan GPU-v3-v3-v3 - resnet50 ncnn: Vulkan GPU-v3-v3-v3 - yolov4-tiny ncnn: Vulkan GPU-v3-v3-v3 - squeezenet_ssd ncnn: Vulkan GPU-v3-v3-v3 - regnety_400m ncnn: Vulkan GPU-v3-v3-v3 - vision_transformer ncnn: Vulkan GPU-v3-v3-v3 - FastestDet vkresample: 2x - Double ncnn: Vulkan GPU-v3-v3-v3-v3-v3-v3 - mobilenet ncnn: Vulkan GPU-v3-v3-v3-v3-v3-v3-v2-v2 - mobilenet-v2 ncnn: Vulkan GPU-v3-v3-v3-v3-v3-v3-v3-v3 - mobilenet-v3 ncnn: Vulkan GPU-v3-v3-v3-v3-v3-v3 - shufflenet-v2 ncnn: Vulkan GPU-v3-v3-v3-v3-v3-v3 - mnasnet ncnn: Vulkan GPU-v3-v3-v3-v3-v3-v3 - efficientnet-b0 ncnn: Vulkan GPU-v3-v3-v3-v3-v3-v3 - blazeface ncnn: Vulkan GPU-v3-v3-v3-v3-v3-v3 - googlenet ncnn: Vulkan GPU-v3-v3-v3-v3-v3-v3 - vgg16 ncnn: Vulkan GPU-v3-v3-v3-v3-v3-v3 - resnet18 ncnn: Vulkan GPU-v3-v3-v3-v3-v3-v3 - alexnet ncnn: Vulkan GPU-v3-v3-v3-v3-v3-v3 - resnet50 ncnn: Vulkan GPU-v3-v3-v3-v3-v3-v3 - yolov4-tiny ncnn: Vulkan GPU-v3-v3-v3-v3-v3-v3 - squeezenet_ssd ncnn: Vulkan GPU-v3-v3-v3-v3-v3-v3 - regnety_400m ncnn: Vulkan GPU-v3-v3-v3-v3-v3-v3 - vision_transformer ncnn: Vulkan GPU-v3-v3-v3-v3-v3-v3 - FastestDet ncnn: Vulkan GPU-v3-v3-v3-v3-v3-v3-v3-v3-v3 - mobilenet ncnn: Vulkan GPU-v3-v3-v3-v3-v3-v3-v3-v3-v3-v2-v2 - mobilenet-v2 ncnn: Vulkan GPU-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3 - mobilenet-v3 ncnn: Vulkan GPU-v3-v3-v3-v3-v3-v3-v3-v3-v3 - shufflenet-v2 ncnn: Vulkan GPU-v3-v3-v3-v3-v3-v3-v3-v3-v3 - mnasnet ncnn: Vulkan GPU-v3-v3-v3-v3-v3-v3-v3-v3-v3 - efficientnet-b0 ncnn: Vulkan GPU-v3-v3-v3-v3-v3-v3-v3-v3-v3 - blazeface ncnn: Vulkan GPU-v3-v3-v3-v3-v3-v3-v3-v3-v3 - googlenet ncnn: Vulkan GPU-v3-v3-v3-v3-v3-v3-v3-v3-v3 - vgg16 ncnn: Vulkan GPU-v3-v3-v3-v3-v3-v3-v3-v3-v3 - resnet18 ncnn: Vulkan GPU-v3-v3-v3-v3-v3-v3-v3-v3-v3 - alexnet ncnn: Vulkan GPU-v3-v3-v3-v3-v3-v3-v3-v3-v3 - resnet50 ncnn: Vulkan GPU-v3-v3-v3-v3-v3-v3-v3-v3-v3 - yolov4-tiny ncnn: Vulkan GPU-v3-v3-v3-v3-v3-v3-v3-v3-v3 - squeezenet_ssd ncnn: Vulkan GPU-v3-v3-v3-v3-v3-v3-v3-v3-v3 - regnety_400m ncnn: Vulkan GPU-v3-v3-v3-v3-v3-v3-v3-v3-v3 - vision_transformer ncnn: Vulkan GPU-v3-v3-v3-v3-v3-v3-v3-v3-v3 - FastestDet ncnn: CPU-v3-v3-v3-v3-v3-v3 - mobilenet ncnn: CPU-v3-v3-v3-v3-v3-v3-v2-v2 - mobilenet-v2 ncnn: CPU-v3-v3-v3-v3-v3-v3-v3-v3 - mobilenet-v3 ncnn: CPU-v3-v3-v3-v3-v3-v3 - shufflenet-v2 ncnn: CPU-v3-v3-v3-v3-v3-v3 - mnasnet ncnn: CPU-v3-v3-v3-v3-v3-v3 - efficientnet-b0 ncnn: CPU-v3-v3-v3-v3-v3-v3 - blazeface ncnn: CPU-v3-v3-v3-v3-v3-v3 - googlenet ncnn: CPU-v3-v3-v3-v3-v3-v3 - vgg16 ncnn: CPU-v3-v3-v3-v3-v3-v3 - resnet18 ncnn: CPU-v3-v3-v3-v3-v3-v3 - alexnet ncnn: CPU-v3-v3-v3-v3-v3-v3 - resnet50 ncnn: CPU-v3-v3-v3-v3-v3-v3 - yolov4-tiny ncnn: CPU-v3-v3-v3-v3-v3-v3 - squeezenet_ssd ncnn: CPU-v3-v3-v3-v3-v3-v3 - regnety_400m ncnn: CPU-v3-v3-v3-v3-v3-v3 - vision_transformer ncnn: CPU-v3-v3-v3-v3-v3-v3 - FastestDet ncnn: CPU-v3-v3-v3-v3-v3-v3-v3-v3-v3 - mobilenet ncnn: CPU-v3-v3-v3-v3-v3-v3-v3-v3-v3-v2-v2 - mobilenet-v2 ncnn: CPU-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3 - mobilenet-v3 ncnn: CPU-v3-v3-v3-v3-v3-v3-v3-v3-v3 - shufflenet-v2 ncnn: CPU-v3-v3-v3-v3-v3-v3-v3-v3-v3 - mnasnet ncnn: CPU-v3-v3-v3-v3-v3-v3-v3-v3-v3 - efficientnet-b0 ncnn: CPU-v3-v3-v3-v3-v3-v3-v3-v3-v3 - blazeface ncnn: CPU-v3-v3-v3-v3-v3-v3-v3-v3-v3 - googlenet ncnn: CPU-v3-v3-v3-v3-v3-v3-v3-v3-v3 - vgg16 ncnn: CPU-v3-v3-v3-v3-v3-v3-v3-v3-v3 - resnet18 ncnn: CPU-v3-v3-v3-v3-v3-v3-v3-v3-v3 - alexnet ncnn: CPU-v3-v3-v3-v3-v3-v3-v3-v3-v3 - resnet50 ncnn: CPU-v3-v3-v3-v3-v3-v3-v3-v3-v3 - yolov4-tiny ncnn: CPU-v3-v3-v3-v3-v3-v3-v3-v3-v3 - squeezenet_ssd ncnn: CPU-v3-v3-v3-v3-v3-v3-v3-v3-v3 - regnety_400m ncnn: CPU-v3-v3-v3-v3-v3-v3-v3-v3-v3 - vision_transformer ncnn: CPU-v3-v3-v3-v3-v3-v3-v3-v3-v3 - FastestDet ncnn: CPU-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3 - mobilenet ncnn: CPU-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v2-v2 - mobilenet-v2 ncnn: CPU-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3 - mobilenet-v3 ncnn: CPU-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3 - shufflenet-v2 ncnn: CPU-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3 - mnasnet ncnn: CPU-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3 - efficientnet-b0 ncnn: CPU-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3 - blazeface ncnn: CPU-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3 - googlenet ncnn: CPU-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3 - vgg16 ncnn: CPU-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3 - resnet18 ncnn: CPU-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3 - alexnet ncnn: CPU-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3 - resnet50 ncnn: CPU-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3 - yolov4-tiny ncnn: CPU-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3 - squeezenet_ssd ncnn: CPU-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3 - regnety_400m ncnn: CPU-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3 - vision_transformer ncnn: CPU-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3 - FastestDet a b c d e f g h i 4080 4080 rep 4080 xxx 4080 zzz 3090 3090 rep 3070 RTX 3070 Ti 4090 4090 rep nv 4090 42105 91597 11340 20816 47887 33001 4717 50504 13190.09 12730.08 13154.15 23232.42 841.40 841.80 2272.62 2658.73 13102.75 23123.77 11.686 8.05 3.16 3.34 2.97 3.90 1.38 7.94 23.75 5.29 4.41 10.20 12.90 7.07 8.18 32.49 3.62 3.18 8.05 3.17 3.17 3.35 2.98 3.86 1.38 7.90 23.51 5.28 4.31 10.01 12.84 7.09 8.16 31.88 4.1 42163 91812 11273 20822 47948 32751 4695 50643 12807.06 12808.59 13145.19 23390.44 839.2 836.55 2269.25 2640.08 13070.81 23396.59 11.69 7.97 3.16 3.34 2.97 3.85 1.38 7.82 23.49 5.2 4.32 10.01 12.74 7.06 8.18 31.95 4.05 8.04 3.14 3.33 2.95 3.82 1.37 7.85 23.56 5.23 4.33 10 12.87 7.07 8.21 31.85 4.07 8.01 3.15 3.16 3.33 2.97 3.82 1.37 7.84 23.5 5.21 4.29 10.01 12.98 7.03 8.05 31.65 4.07 8 3.15 3.16 3.33 2.96 3.85 1.37 7.97 23.42 5.42 4.42 9.87 12.77 7.14 8.27 31.71 4.06 43021 91744 11311 20847 47971 32812 4670 50596 12860.56 12822.01 13136.79 23387.26 839.01 836.16 2269.06 2638.69 13063.86 23385.44 11.688 8.02 3.18 3.35 2.99 3.88 1.39 7.93 23.45 5.24 4.31 10.11 12.81 7.1 8.27 31.77 4.11 3.17 8.03 3.13 3.2 3.32 2.96 3.83 1.37 7.8 23.54 5.21 4.33 10 12.81 7.06 8 31.79 4.09 8 3.14 3.16 3.34 2.97 3.89 1.38 7.88 23.99 5.26 4.28 10.33 12.89 7.04 8.14 31.78 4.08 7.95 3.15 3.17 3.33 2.96 3.82 1.36 7.83 23.54 5.23 4.3 10.03 12.86 7.07 7.98 31.66 3.69 35399 85181 10719 12143 42645 36328 2346 43365 8531.96 11251.17 8412.33 16864.47 267.43 267.74 8520.02 8465.82 5676.02 7352.85 32.855 8.10 3.17 3.35 2.98 3.85 1.38 7.85 23.51 5.23 4.30 10.00 12.95 7.09 8.23 32.43 4.08 3.17 8.02 3.16 3.17 3.35 2.97 3.87 1.38 7.85 23.56 5.23 4.31 10.10 12.85 7.08 8.17 32.12 4.11 500.014 35304 85191 10560 12168 42651 37090 2343 43365 8515.58 11231.72 8397.80 16865.29 267.41 267.25 8505.20 8465.71 5675.99 7336.25 32.850 8.04 3.14 3.18 3.33 2.96 3.84 1.38 7.85 23.60 5.22 4.31 10.10 12.87 7.05 8.10 31.93 4.08 500.016 26593 104146 7571 10561 56476 26238 1814 57110 6837.94 9006.57 6812.52 13440.97 214.17 214.23 6827.92 6800.17 4480.59 5959.75 26.738 8.45 3.15 3.55 2.97 3.87 1.37 7.92 24.19 5.69 4.36 11.05 13.32 7.23 8.08 33.56 4.22 8.27 3.13 3.14 3.4 2.97 3.86 1.38 8.15 24.55 5.48 4.64 10.26 13.17 7.08 8.34 32.92 4.24 8.56 3.16 3.15 3.4 3.12 4.04 1.43 8.07 24.45 6.13 4.83 11.05 13.07 6.97 8.34 33.47 3.85 8.65 3.16 3.15 3.33 2.96 3.85 1.37 7.94 24.12 5.3 4.35 10.25 14.34 7.09 8.5 33.36 4.2 500.01 26638 104171 7574 10548 56455 26541 1818 57094 6832.74 9003.12 6810.55 13438.4 213.96 213.95 6824.21 6795.39 4478.41 5956.24 26.769 22.74 3.16 3.59 3 3.91 1.38 7.98 23.78 5.55 4.32 10.72 17.23 7.13 8.3 32.73 2.57 3.14 8.17 3.18 3.35 2.98 3.86 1.41 8.96 24.2 6.22 4.87 10.34 13.64 7.1 8.36 32.42 4.07 8.98 3.15 3.16 3.38 3.05 4.14 1.37 9.15 24.92 5.48 4.71 11.25 13.08 7.26 8.07 33.39 3.97 8.04 3.14 3.15 3.33 2.95 3.85 1.37 7.94 23.82 5.3 4.35 10.18 12.89 7.19 7.99 32.38 4.06 500.011 8.5 3.17 3.16 3.34 2.97 3.84 1.38 7.96 24.04 5.28 4.35 10.33 13.14 7.14 8.38 33.32 3.92 26524 104298 7622 10572 56431 6810.73 9036.17 6838.32 13490.24 213.37 210.96 6800.6 6772.98 4495.98 5978.38 33727 132270 10061 14780 69738 34686 2417 71163 20.93 8.37 3.52 5.03 2.74 4.05 1.4 10.3 30.96 5.6 5.3 12.09 14.65 8.96 9.94 37.8 2.66 3.26 10.4 3.29 4.87 3.49 3.2 5.88 1.4 8.75 27.83 5.82 6.53 12.96 15.16 7.46 9.88 36.42 5.14 10.08 3.3 3.29 3.52 3.39 4.68 1.28 10.17 29.12 5.86 5.01 14.05 15.11 8.16 8.21 36.55 5.69 9.05 3.28 3.26 3.36 2.99 4.21 1.25 10.19 29.07 5.85 4.99 11.15 15.43 8.33 7.99 38.33 4.43 500.006 10.02 3.29 3.26 3.43 3.07 4.19 1.41 10.47 27.43 5.88 5.1 13.1 13.77 7.21 8.46 38.01 3.83 66473 211076 17121 34974 104556 65869 5579 106210 13.136 8.73 3.28 3.41 3.05 3.99 1.4 8.42 25.37 5.69 4.75 11.16 13.85 7.73 8.24 35.07 4.42 3.24 8.44 3.26 3.43 3.07 4.01 1.41 8.42 25 5.67 4.62 10.81 13.79 7.58 8.39 35.56 4.2 8.43 3.31 3.28 3.46 3.08 4.02 1.44 8.45 25.1 5.7 4.66 11.11 13.81 7.64 8.33 34.2 4.2 8.43 3.29 3.27 3.44 3.09 4.05 1.42 8.49 25.04 5.65 4.69 10.95 13.79 7.66 8.45 34.13 4.2 288.201 8.84 3.28 3.46 3.06 4.06 1.43 8.4 25.48 5.61 4.61 11.4 13.86 7.66 8.61 34.91 4.28 8.43 3.29 3.26 3.48 3.1 4.04 1.44 8.79 25.67 5.92 4.98 11.48 13.93 7.71 8.67 35.6 4.19 68279 211058 17287 35038 104491 70068 5583 106205 13.136 8.41 3.28 3.43 3.06 4.09 1.42 8.52 26.11 5.68 4.67 11.76 14.03 7.86 8.67 35.28 4.34 3.26 8.57 3.29 3.44 3.08 4.05 1.41 8.4 25.04 5.61 4.65 10.84 13.67 7.64 8.56 35.07 4.21 8.48 3.27 3.43 3.09 4.02 1.42 8.52 24.91 5.63 4.65 10.79 13.55 7.59 8.57 34.27 4.18 8.46 3.3 3.27 3.44 3.07 4.04 1.45 8.58 25.04 5.69 4.69 10.84 13.68 7.63 8.44 34.29 4.09 288.166 8.4 3.27 3.24 3.39 3.03 3.98 1.41 8.49 25.05 5.61 4.72 10.8 13.55 7.55 8.24 34.1 4.14 8.38 3.27 3.31 3.44 3.06 4.01 1.42 8.42 25.56 5.67 4.64 11.07 13.73 7.62 8.35 33.93 4.17 8.45 3.3 3.28 3.47 3.09 4.07 1.43 8.52 25.01 5.64 4.68 10.86 13.71 7.67 8.72 34.22 4.2 69068 210713 17343 35071 104528 67887 5587 106099 13.137 8.37 3.28 3.45 3.06 4.04 1.42 8.38 25.03 5.56 4.68 10.82 13.6 7.62 8.45 34.27 4.17 3.27 8.37 3.26 3.5 3.07 4.04 1.42 8.42 25.01 5.62 4.68 10.94 13.65 7.62 8.56 34.19 4.2 8.44 3.3 3.31 3.47 3.07 4.06 1.42 8.5 25 5.66 4.65 10.91 13.69 7.64 8.75 34.37 4.19 8.88 3.4 3.33 3.51 3.13 4.22 1.42 8.99 26.08 5.89 5.21 11.5 13.95 7.7 8.58 35.4 4.31 288.039 8.46 3.27 3.26 3.43 3.05 4.02 1.42 8.43 25.4 5.67 4.67 10.91 13.62 7.67 8.52 34.23 4.17 8.31 3.14 3.05 3.34 2.98 3.97 1.31 8.26 25.33 5.65 4.71 11.22 13.52 7.27 8.25 33.9 3.75 8.34 3.2 3.08 3.4 3 4.01 1.32 8.32 25.44 5.78 4.69 11.26 13.63 7.27 8.38 34.14 3.8 67689 210991 17185 35058 104543 70040 5584 105926 13.126 8.38 3.25 3.42 3.04 3.99 1.4 8.4 25.4 5.63 4.69 11.1 13.63 7.55 8.37 34.1 4.16 3.24 8.47 3.29 3.28 3.46 3.06 4.04 1.42 8.41 25.82 5.59 4.68 11.07 13.8 7.63 8.58 34.32 4.79 8.4 3.28 3.27 3.43 3.06 4.03 1.41 8.55 25.45 5.71 4.67 11.21 13.83 7.35 8.47 34.47 4.04 8.38 3.23 3.2 3.37 3.01 3.95 1.39 8.37 25.26 5.59 4.65 11.09 13.62 7.51 8.1 34.05 4.12 288.028 8.46 3.28 3.24 3.43 3.08 4.01 1.42 8.42 25.16 5.6 4.68 10.91 13.61 7.62 8.49 34.1 4.2 9.19 3.28 3.26 3.44 3.08 4.05 1.41 8.55 26.09 5.74 4.7 12.5 15.26 8.06 8.37 35.36 4.61 8.25 3.16 3.06 3.36 2.96 3.95 1.31 8.29 25.26 5.77 4.66 11.1 13.42 7.25 8.34 34.47 3.82 55347 255207 14406 30945 141357 51005 4282 143969 21269.72 27797.8 20845.09 41149.1 653.13 653.15 20909.02 20820.09 13710.88 16886.66 10.399 8.6 3.17 3.34 2.99 3.87 1.38 7.86 23.43 5.21 4.3 10.3 14.26 7.52 8.25 33.01 4.21 3.18 8.07 3.18 3.19 3.39 2.99 3.88 1.39 7.87 23.55 5.27 4.35 10.1 12.88 7.16 8.38 31.94 4.11 8.06 3.14 3.15 3.32 2.95 3.83 1.36 7.83 23.5 5.19 4.31 10.05 12.87 7.04 7.95 31.89 4.04 8 3.16 3.34 2.97 3.85 1.36 7.82 23.5 5.23 4.3 9.97 12.88 7.04 8.01 31.86 4.04 371.699 8.11 3.15 3.16 3.33 2.96 3.83 1.36 7.86 23.55 5.19 4.31 10.38 13.1 7.04 7.99 33.22 4.03 8.03 3.12 3.13 3.32 2.94 3.88 1.38 7.84 23.51 5.21 4.32 10.07 12.97 7.05 8.33 31.94 3.83 8.07 3.16 3.36 2.98 3.88 1.39 7.9 23.58 5.2 4.33 10.03 12.82 7.12 8.22 32.1 4.1 8.01 3.15 3.16 3.36 2.97 3.86 1.39 7.83 23.5 5.2 4.3 10.03 12.86 7.05 8.2 32.16 4.08 54432 265171 14449 31122 141437 54814 4289 143956 20708.84 27393.2 20640.67 40876.12 648.71 20613.41 20517.45 13606.79 16878.2 10.428 8.01 3.17 3.35 2.97 3.86 1.38 7.9 23.43 5.29 4.31 9.95 12.77 7.12 8.24 31.8 4.08 3.19 8.03 3.17 3.16 3.36 2.97 3.86 1.37 7.82 23.48 5.2 4.31 10.06 12.83 7.06 8.02 31.91 4.08 8.03 3.17 3.15 3.33 2.96 3.85 1.37 7.85 23.54 5.2 4.3 10.07 12.82 7.09 8.09 31.93 4.11 8.01 3.17 3.15 3.36 2.97 3.85 1.38 7.85 23.43 5.24 4.3 10.01 12.84 7.08 8.25 32.11 4.1 371.422 8.04 3.19 3.37 2.99 3.87 1.39 7.89 23.52 5.22 4.31 10.04 12.86 7.09 8.34 32.09 4.08 8.05 3.17 3.18 3.34 2.97 3.85 1.37 7.86 23.38 5.2 4.3 9.98 12.9 7.08 8.07 31.97 4.07 8.06 3.16 3.17 3.33 2.97 3.85 1.38 7.91 23.4 5.3 4.31 10.06 12.81 7.09 8.03 31.85 4.07 8.05 3.17 3.36 2.98 3.85 1.38 7.82 23.47 5.2 4.3 10.04 12.89 7.07 8.19 32.13 4.1 8.03 3.15 3.19 3.32 2.96 3.84 1.38 7.86 23.72 5.27 4.31 10.27 12.92 7.07 8.06 31.94 4.07 22.064 17.81 9.67 6.82 6.88 9.23 2.98 18.25 56.64 14.03 9.62 21.5 27.66 13.2 18 75.34 8.65 7.52 21.11 7.81 6.6 7.07 5.09 8.99 3.98 19.49 48.29 12.68 10.88 23.48 28.59 15.82 16.22 70.76 8.41 17.82 9.19 5.38 8.13 6.87 9.01 3.03 17 49.75 11.14 11 24.07 29.34 17.75 19.66 81.77 9.18 16.34 7.24 8.06 4.89 6.02 7.81 3.18 20.72 55.42 13.38 9.86 23.11 29.49 16.15 17.23 73.51 8.63 24.745 18.39 8.35 6.56 8 4.59 8.41 1.77 18.6 55.48 12.14 10.08 23.59 29.8 15.4 17.88 71.08 6.93 17.09 5.46 5.99 5.59 6.06 9.81 2.99 16.97 49.7 11.3 11.89 23.44 28.73 18.83 17.61 70.29 6.71 16.52 7.22 6.43 7.81 6.07 9.19 2.53 19.2 50.32 12.64 10.59 22.19 28.41 14.27 18.25 65.41 7.12 18.54 5.49 5.97 6.3 8.15 9.53 3.57 18.66 51.28 13.34 10.69 23.54 26.33 15.46 18.24 69.48 4.48 17.06 5.92 7.34 5.89 8.55 6.63 2.69 18.8 53.48 12.13 11.43 22.15 29.38 15.32 17.02 70.53 7.23 27.183 9.43 3.76 3.77 3.26 4.72 1.60 9.69 28.36 6.08 5.49 12.73 15.20 8.13 8.83 37.91 3.94 3.61 9.35 3.56 3.64 3.98 3.40 4.53 1.60 9.87 29.06 6.28 5.25 12.60 15.54 8.47 9.07 38.03 4.26 9.62 3.41 3.76 4.09 3.11 4.78 1.79 9.65 28.40 6.18 5.55 12.11 15.42 8.39 9.02 37.86 4.33 9.62 3.66 3.65 3.95 3.37 4.73 1.71 9.86 28.40 6.23 5.67 12.35 15.44 8.29 8.89 38.29 4.26 24.805 9.62 3.66 3.62 3.75 3.34 4.60 1.49 9.84 28.63 6.57 5.53 12.73 15.21 8.28 9.19 37.88 4.41 9.52 3.66 3.44 3.89 3.10 4.37 1.34 9.90 28.53 6.40 5.34 12.42 15.00 8.65 9.05 38.27 4.32 10.03 3.91 3.70 4.02 3.24 4.74 2.48 9.68 27.98 6.22 6.17 12.81 14.57 7.57 8.42 38.04 4.18 9.98 3.69 3.52 3.92 3.25 4.55 1.51 9.58 28.53 6.69 5.41 12.52 15.56 8.31 9.10 38.32 4.25 10.02 3.83 3.24 3.48 3.12 4.17 1.40 9.97 27.86 5.94 6.25 13.15 14.64 7.45 9.14 38.50 4.14 84351 290342 20373 55214 153896 81406 8039 152656 9.284 10.08 3.3 3.55 3.12 4.23 1.27 10.27 27.75 6 5.14 14.1 13.68 7.86 8.13 38.25 5.48 3.53 10.55 3.3 3.28 3.45 3.18 4.34 1.39 10.62 28.82 5.69 4.64 14.13 13.97 7.83 8.64 38.76 4.39 8.96 3.46 3.25 5.18 3.19 4.09 1.17 9.97 28.55 5.81 4.94 11.39 15.3 9.32 8.13 38.79 2.93 8.81 4.99 3.12 3.34 3 4.18 1.33 8.87 28.21 5.97 6.11 14.58 15.44 7.93 9.6 39.01 3.94 172.883 8.96 3.41 3.34 5.17 3.1 4.24 1.43 11.3 29.05 5.78 4.72 11.53 15.55 9.28 10.1 39.06 4.13 9.16 3.48 3.62 3.52 4.93 4.15 1.42 9.05 27.44 7.52 4.67 12.4 15.95 9.81 9.87 38.62 4.45 9.04 3.36 3.33 3.48 5.19 4.47 1.3 8.38 30.16 7.74 4.99 11.72 15.85 9.51 10.09 38.76 2.85 8.46 5.25 3.36 3.47 3.19 4.14 1.45 8.91 27.31 7.78 4.94 12.98 15.69 7.4 10.05 38.82 2.82 10.56 4.75 3.36 3.56 3.23 4.63 1.35 8.55 27.32 6.96 5.14 13 16.05 7.43 10.11 39.35 4.62 81329 287651 20404 55383 153939 80999 8119 155936 8.962 10.75 3.3 3.47 3.18 4.3 1.45 10.86 29.24 6.77 5.23 13.73 16.79 7.98 8.78 37.05 5.27 3.31 10.23 3.31 3.3 3.49 3.15 4.09 1.4 10.65 27.04 6.01 6.79 13.08 15.72 8.22 8.48 37.59 4.59 8.74 3.45 3.53 3.59 3.23 4.34 1.34 9.29 27.25 7.75 5.27 13.82 15.34 9.3 17.15 39.12 4.16 9.54 3.31 4.9 3.4 3.1 6.28 1.41 8.9 27.59 5.81 6.58 13.57 16.39 7.81 10.34 38.73 4.16 173.043 8.37 3.34 3.33 5.27 3.13 4.04 1.46 10.38 29.12 6.05 5.33 12.17 15.45 9.16 8.64 38.17 3.96 9.02 3.36 3.34 3.48 4.99 4.41 1.41 10.39 29.17 5.9 5.25 11.51 15.4 9.46 10.69 39.03 4.11 10.61 3.44 3.3 3.42 5.11 4.35 1.42 8.97 30.74 8.14 5.45 12.47 13.88 9.38 10.23 38.79 3.12 8.83 3.6 3.44 5.18 3.28 4.44 1.38 10.18 29.35 5.84 5.16 11.24 16.6 9.34 8.45 38.69 3.91 8.22 3.38 3.35 5.23 3.12 4.1 1.42 10.47 29.85 5.87 5.34 10.96 15.41 9.44 8.7 38.65 4.59 84887 292768 20601 54950 152170 82875 8132 155148 8.967 8.15 3.6 3.45 4.77 4.37 1.33 8.93 29.54 7.82 5.18 11.41 15.4 9.11 10.17 38.9 4.51 3.47 8.45 3.43 3.36 3.51 4.7 4.1 1.4 8.7 29.29 7.44 5.2 12.45 15.26 9.11 9.81 39.04 3.93 8.91 3.39 3.35 3.46 4.61 4.04 1.16 9.02 27.04 7.61 4.67 13.68 15.62 9.37 9.55 38.99 4.06 10.54 4.45 2.61 3.17 2.54 5.26 1.07 10.01 27.77 5.84 6.11 13.13 16.61 7.72 7.73 38.58 5.92 172.887 9.41 5.1 3.26 3.51 3.16 4.12 1.18 8.61 27.89 8.16 5.14 13.63 17.3 9.21 10.09 39.18 2.81 8.93 3.42 3.17 3.5 3.07 4.1 1.26 8.35 28.14 7.38 4.69 13.13 15.55 7.02 10.03 38.46 2.64 12.12 3.29 4.97 3.32 3.12 5.94 1.42 10.75 27.61 6.07 6.62 13.29 17.67 7.48 8.25 38.95 3.93 10.15 3.27 4.81 3.37 3.1 5.88 2.91 8.85 29.4 5.97 6.54 13.46 15.67 7.72 8.37 38.58 4.01 10.64 3.29 4.96 3.43 3.1 5.82 1.4 10.14 27.25 5.58 6.32 13.25 16.3 8.26 8.34 37.13 5.86 OpenBenchmarking.org
VkFFT Test: FFT + iFFT R2C / C2R OpenBenchmarking.org Benchmark Score, More Is Better VkFFT 1.2.31 Test: FFT + iFFT R2C / C2R h f g i e d a b c 3090 rep 3090 4080 4080 zzz 4080 rep 4080 xxx 4090 rep 4090 nv 4090 20K 40K 60K 80K 100K SE +/- 3.71, N = 3 SE +/- 118.74, N = 3 SE +/- 200.55, N = 3 SE +/- 796.66, N = 3 26524 26593 26638 33727 35304 35399 42105 42163 43021 54432 55347 66473 67689 68279 69068 81329 84351 84887 1. (CXX) g++ options: -O3
VkFFT Test: FFT + iFFT C2C 1D batched in half precision OpenBenchmarking.org Benchmark Score, More Is Better VkFFT 1.2.31 Test: FFT + iFFT C2C 1D batched in half precision d e a c b f g h i 4080 xxx 4080 zzz 4080 rep 4080 3090 3090 rep 4090 rep 4090 nv 4090 60K 120K 180K 240K 300K SE +/- 18.50, N = 3 SE +/- 26.03, N = 3 SE +/- 83.55, N = 3 SE +/- 133.47, N = 3 85181 85191 91597 91744 91812 104146 104171 104298 132270 210713 210991 211058 211076 255207 265171 287651 290342 292768 1. (CXX) g++ options: -O3
VkFFT Test: FFT + iFFT C2C Bluestein in single precision OpenBenchmarking.org Benchmark Score, More Is Better VkFFT 1.2.31 Test: FFT + iFFT C2C Bluestein in single precision f g h i e d b c a 3090 3090 rep 4080 4080 zzz 4080 rep 4080 xxx 4090 4090 rep nv 4090 4K 8K 12K 16K 20K SE +/- 75.16, N = 15 SE +/- 72.34, N = 3 SE +/- 62.67, N = 3 SE +/- 83.38, N = 3 7571 7574 7622 10061 10560 10719 11273 11311 11340 14406 14449 17121 17185 17287 17343 20373 20404 20601 1. (CXX) g++ options: -O3
VkFFT Test: FFT + iFFT C2C 1D batched in double precision OpenBenchmarking.org Benchmark Score, More Is Better VkFFT 1.2.31 Test: FFT + iFFT C2C 1D batched in double precision g f h d e i a b c 3090 3090 rep 4080 4080 rep 4080 zzz 4080 xxx nv 4090 4090 4090 rep 12K 24K 36K 48K 60K SE +/- 10.58, N = 3 SE +/- 12.42, N = 3 SE +/- 11.67, N = 3 SE +/- 14.62, N = 3 10548 10561 10572 12143 12168 14780 20816 20822 20847 30945 31122 34974 35038 35058 35071 54950 55214 55383 1. (CXX) g++ options: -O3
VkFFT Test: FFT + iFFT C2C 1D batched in single precision OpenBenchmarking.org Benchmark Score, More Is Better VkFFT 1.2.31 Test: FFT + iFFT C2C 1D batched in single precision d e a b c h g f i 4080 rep 4080 xxx 4080 zzz 4080 3090 3090 rep nv 4090 4090 4090 rep 30K 60K 90K 120K 150K SE +/- 2.73, N = 3 SE +/- 1.67, N = 3 SE +/- 9.54, N = 3 SE +/- 25.50, N = 3 42645 42651 47887 47948 47971 56431 56455 56476 69738 104491 104528 104543 104556 141357 141437 152170 153896 153939 1. (CXX) g++ options: -O3
VkFFT Test: FFT + iFFT C2C multidimensional in single precision OpenBenchmarking.org Benchmark Score, More Is Better VkFFT 1.2.31 Test: FFT + iFFT C2C multidimensional in single precision f g b c a i d e 3090 3090 rep 4080 4080 xxx 4080 zzz 4080 rep 4090 rep 4090 nv 4090 20K 40K 60K 80K 100K SE +/- 57.83, N = 3 SE +/- 116.12, N = 3 SE +/- 437.33, N = 3 SE +/- 555.86, N = 3 26238 26541 32751 32812 33001 34686 36328 37090 51005 54814 65869 67887 70040 70068 80999 81406 82875 1. (CXX) g++ options: -O3
VkFFT Test: FFT + iFFT C2C Bluestein benchmark in double precision OpenBenchmarking.org Benchmark Score, More Is Better VkFFT 1.2.31 Test: FFT + iFFT C2C Bluestein benchmark in double precision f g e d i 3090 3090 rep c b a 4080 4080 rep 4080 zzz 4080 xxx 4090 4090 rep nv 4090 2K 4K 6K 8K 10K SE +/- 11.20, N = 3 SE +/- 4.37, N = 3 SE +/- 0.33, N = 3 1814 1818 2343 2346 2417 4282 4289 4670 4695 4717 5579 5583 5584 5587 8039 8119 8132 1. (CXX) g++ options: -O3
VkFFT Test: FFT + iFFT C2C 1D batched in single precision, no reshuffling OpenBenchmarking.org Benchmark Score, More Is Better VkFFT 1.2.31 Test: FFT + iFFT C2C 1D batched in single precision, no reshuffling d e a c b g f i 4080 zzz 4080 xxx 4080 rep 4080 3090 rep 3090 4090 nv 4090 4090 rep 30K 60K 90K 120K 150K SE +/- 2.33, N = 3 SE +/- 2.08, N = 3 SE +/- 8.89, N = 3 43365 43365 50504 50596 50643 57094 57110 71163 105926 106099 106205 106210 143956 143969 152656 155148 155936 1. (CXX) g++ options: -O3
vkpeak fp32-scalar OpenBenchmarking.org GFLOPS, More Is Better vkpeak 20230730 fp32-scalar h g f e d b c a 3090 rep 3090 5K 10K 15K 20K 25K SE +/- 0.30, N = 3 SE +/- 16.18, N = 3 SE +/- 4.18, N = 3 6810.73 6812.99 6837.94 8515.58 8531.96 12807.06 12860.56 13190.09 20708.84 21269.72
vkpeak fp32-vec4 OpenBenchmarking.org GFLOPS, More Is Better vkpeak 20230730 fp32-vec4 g f h e d a b c 3090 rep 3090 6K 12K 18K 24K 30K SE +/- 2.57, N = 3 SE +/- 19.37, N = 3 SE +/- 1.81, N = 3 9002.59 9006.57 9036.17 11231.72 11251.17 12730.08 12808.59 12822.01 27393.20 27797.80
vkpeak fp16-scalar OpenBenchmarking.org GFLOPS, More Is Better vkpeak 20230730 fp16-scalar g f h e d c b a 3090 rep 3090 4K 8K 12K 16K 20K SE +/- 5.09, N = 3 SE +/- 13.46, N = 3 SE +/- 4.01, N = 3 6810.55 6812.52 6838.32 8397.80 8412.33 13136.79 13145.19 13154.15 20640.67 20845.09
vkpeak fp16-vec4 OpenBenchmarking.org GFLOPS, More Is Better vkpeak 20230730 fp16-vec4 g f h d e a c b 3090 rep 3090 9K 18K 27K 36K 45K SE +/- 0.37, N = 3 SE +/- 0.36, N = 3 SE +/- 5.96, N = 3 13438.40 13440.97 13490.24 16864.47 16865.29 23232.42 23387.26 23390.44 40876.12 41149.10
vkpeak fp64-scalar OpenBenchmarking.org GFLOPS, More Is Better vkpeak 20230730 fp64-scalar h g f e d 3090 rep 3090 c b a 200 400 600 800 1000 SE +/- 0.00, N = 3 SE +/- 0.01, N = 3 SE +/- 0.22, N = 3 213.37 213.96 214.17 267.41 267.43 648.71 653.13 839.01 839.20 841.40
vkpeak fp64-vec4 OpenBenchmarking.org GFLOPS, More Is Better vkpeak 20230730 fp64-vec4 h g f e d 3090 c b a 200 400 600 800 1000 SE +/- 0.00, N = 3 SE +/- 0.48, N = 3 SE +/- 0.32, N = 3 210.96 213.95 214.23 267.25 267.74 653.15 836.16 836.55 841.80
vkpeak int32-scalar OpenBenchmarking.org GIOPS, More Is Better vkpeak 20230730 int32-scalar c b a h g f e d 3090 rep 3090 4K 8K 12K 16K 20K SE +/- 0.34, N = 3 SE +/- 0.03, N = 3 SE +/- 15.02, N = 3 2269.06 2269.25 2272.62 6800.60 6824.21 6827.92 8505.20 8520.02 20613.41 20909.02
vkpeak int32-vec4 OpenBenchmarking.org GIOPS, More Is Better vkpeak 20230730 int32-vec4 c b a h g f e d 3090 rep 3090 4K 8K 12K 16K 20K SE +/- 0.26, N = 3 SE +/- 0.05, N = 3 SE +/- 0.19, N = 3 2638.69 2640.08 2658.73 6772.98 6794.92 6800.17 8465.71 8465.82 20517.45 20820.09
vkpeak int16-scalar OpenBenchmarking.org GIOPS, More Is Better vkpeak 20230730 int16-scalar g f h e d c b a 3090 rep 3090 3K 6K 9K 12K 15K SE +/- 0.02, N = 3 SE +/- 0.09, N = 3 SE +/- 1.30, N = 3 4478.41 4480.59 4495.98 5675.99 5676.02 13063.86 13070.81 13102.75 13606.79 13710.88
vkpeak int16-vec4 OpenBenchmarking.org GIOPS, More Is Better vkpeak 20230730 int16-vec4 g f h e d 3090 rep 3090 a c b 5K 10K 15K 20K 25K SE +/- 0.31, N = 3 SE +/- 17.33, N = 3 SE +/- 21.55, N = 3 5956.24 5959.75 5978.38 7336.25 7352.85 16878.20 16886.66 23123.77 23385.44 23396.59
VkResample Upscale: 2x - Precision: Single OpenBenchmarking.org ms, Fewer Is Better VkResample 1.0 Upscale: 2x - Precision: Single d e RTX 3070 Ti g f 3070 i 4080 xxx 4080 rep 4080 4080 zzz b c a 3090 rep 3090 4090 nv 4090 4090 rep 8 16 24 32 40 SE +/- 0.004, N = 3 SE +/- 0.000, N = 3 SE +/- 0.029, N = 3 SE +/- 0.001, N = 3 32.855 32.850 27.183 26.769 26.738 22.064 20.930 13.137 13.136 13.136 13.126 11.690 11.688 11.686 10.428 10.399 9.284 8.967 8.962 1. (CXX) g++ options: -O3
NCNN Target: CPU - Model: mobilenet OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: CPU - Model: mobilenet g 3070 4090 rep 4090 RTX 3070 Ti 4080 3090 f 4080 rep 4080 zzz 4080 xxx i nv 4090 d a c 3090 rep b 5 10 15 20 25 SE +/- 0.22, N = 15 SE +/- 0.06, N = 3 SE +/- 0.03, N = 3 22.74 17.81 10.75 10.08 9.43 8.73 8.60 8.45 8.41 8.38 8.37 8.37 8.15 8.10 8.05 8.02 8.01 7.97 MIN: 8.24 / MAX: 1264.67 MIN: 8.05 / MAX: 159.41 MIN: 8.24 / MAX: 287.14 MIN: 8.1 / MAX: 118.32 MIN: 7.95 / MAX: 398.1 MIN: 8.15 / MAX: 10.96 MIN: 8.5 / MAX: 13.72 MIN: 8.37 / MAX: 9.44 MIN: 8.14 / MAX: 11.03 MIN: 7.94 / MAX: 10.16 MIN: 7.96 / MAX: 9.72 MIN: 8.15 / MAX: 9.75 MIN: 7.73 / MAX: 9.34 MIN: 7.94 / MAX: 14.4 MIN: 7.97 / MAX: 9.07 MIN: 7.98 / MAX: 8.33 MIN: 7.96 / MAX: 9.85 MIN: 7.94 / MAX: 8.26 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread
NCNN Target: CPU-v2-v2 - Model: mobilenet-v2 OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: CPU-v2-v2 - Model: mobilenet-v2 3070 4090 rep RTX 3070 Ti nv 4090 i 4090 4080 xxx 4080 rep 4080 4080 zzz c 3090 rep 3090 d g b a f 3 6 9 12 15 SE +/- 0.17, N = 15 SE +/- 0.02, N = 3 SE +/- 0.00, N = 3 9.67 4.74 3.76 3.60 3.52 3.30 3.28 3.28 3.28 3.25 3.18 3.17 3.17 3.17 3.16 3.16 3.16 3.15 MIN: 3.19 / MAX: 225.84 MIN: 3.09 / MAX: 140.79 MIN: 2.6 / MAX: 364.73 MIN: 3.43 / MAX: 4.62 MIN: 3.29 / MAX: 19.18 MIN: 3.11 / MAX: 4.81 MIN: 3.1 / MAX: 4.05 MIN: 3.11 / MAX: 4 MIN: 3.11 / MAX: 3.88 MIN: 3.09 / MAX: 4.51 MIN: 3.13 / MAX: 3.84 MIN: 3.11 / MAX: 4.94 MIN: 3.12 / MAX: 4.05 MIN: 3.1 / MAX: 8.86 MIN: 3.11 / MAX: 3.83 MIN: 3.11 / MAX: 3.61 MIN: 3.1 / MAX: 3.8 MIN: 3.1 / MAX: 3.65 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread
NCNN Target: CPU - Model: shufflenet-v2 OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: CPU - Model: shufflenet-v2 3070 i RTX 3070 Ti g 4090 f 4090 rep nv 4090 4080 xxx 4080 rep 4080 zzz 4080 3090 rep d c 3090 b a 2 4 6 8 10 SE +/- 0.19, N = 15 SE +/- 0.01, N = 3 SE +/- 0.01, N = 3 6.82 5.03 3.77 3.59 3.55 3.55 3.51 3.45 3.45 3.43 3.42 3.41 3.35 3.35 3.35 3.34 3.34 3.34 MIN: 3.16 / MAX: 64.72 MIN: 3.07 / MAX: 228.55 MIN: 3.02 / MAX: 511.95 MIN: 3.3 / MAX: 25.28 MIN: 3.39 / MAX: 5.48 MIN: 3.27 / MAX: 22.86 MIN: 3.38 / MAX: 5.4 MIN: 3.32 / MAX: 4.91 MIN: 3.32 / MAX: 3.85 MIN: 3.3 / MAX: 4.15 MIN: 3.28 / MAX: 4.19 MIN: 3.28 / MAX: 4.87 MIN: 3.31 / MAX: 3.68 MIN: 3.3 / MAX: 3.82 MIN: 3.31 / MAX: 3.8 MIN: 3.3 / MAX: 4.19 MIN: 3.31 / MAX: 3.77 MIN: 3.3 / MAX: 3.85 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread
NCNN Target: CPU - Model: mnasnet OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: CPU - Model: mnasnet 3070 nv 4090 RTX 3070 Ti 4090 rep 4090 4080 xxx 4080 rep 4080 4080 zzz g 3090 c d 3090 rep f b a i 2 4 6 8 10 SE +/- 0.14, N = 15 SE +/- 0.01, N = 3 SE +/- 0.01, N = 3 6.88 4.77 3.26 3.22 3.12 3.06 3.06 3.05 3.04 3.00 2.99 2.99 2.98 2.97 2.97 2.97 2.97 2.74 MIN: 3.05 / MAX: 110.25 MIN: 3.07 / MAX: 97.57 MIN: 2.46 / MAX: 277.54 MIN: 3.11 / MAX: 3.71 MIN: 2.98 / MAX: 3.79 MIN: 2.94 / MAX: 4.45 MIN: 2.94 / MAX: 4.51 MIN: 2.92 / MAX: 3.82 MIN: 2.91 / MAX: 4.47 MIN: 2.96 / MAX: 3.68 MIN: 2.95 / MAX: 3.88 MIN: 2.96 / MAX: 3.44 MIN: 2.94 / MAX: 3.83 MIN: 2.93 / MAX: 3.28 MIN: 2.93 / MAX: 3.95 MIN: 2.93 / MAX: 3.45 MIN: 2.92 / MAX: 3.48 MIN: 2.62 / MAX: 4.22 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread
NCNN Target: CPU - Model: efficientnet-b0 OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: CPU - Model: efficientnet-b0 3070 RTX 3070 Ti nv 4090 4090 rep 4090 4080 rep i 4080 xxx 4080 zzz 4080 g a c 3090 f 3090 rep d b 3 6 9 12 15 SE +/- 0.19, N = 15 SE +/- 0.05, N = 3 SE +/- 0.00, N = 3 9.23 4.72 4.37 4.30 4.23 4.09 4.05 4.04 3.99 3.99 3.91 3.90 3.88 3.87 3.87 3.86 3.85 3.85 MIN: 3.43 / MAX: 156.19 MIN: 3.37 / MAX: 486.93 MIN: 4.15 / MAX: 5.96 MIN: 4.08 / MAX: 5.07 MIN: 3.98 / MAX: 12.23 MIN: 3.86 / MAX: 5.59 MIN: 3.78 / MAX: 5.45 MIN: 3.83 / MAX: 5.71 MIN: 3.8 / MAX: 5.69 MIN: 3.79 / MAX: 5.83 MIN: 3.85 / MAX: 4.64 MIN: 3.82 / MAX: 4.51 MIN: 3.84 / MAX: 4.41 MIN: 3.83 / MAX: 4.69 MIN: 3.81 / MAX: 4.97 MIN: 3.81 / MAX: 4.75 MIN: 3.81 / MAX: 4.46 MIN: 3.81 / MAX: 4.42 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread
NCNN Target: CPU - Model: blazeface OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: CPU - Model: blazeface 3070 RTX 3070 Ti 4090 rep 4080 xxx 4080 rep 4080 zzz 4080 i c 3090 rep 3090 g d b a f nv 4090 4090 0.6705 1.341 2.0115 2.682 3.3525 SE +/- 0.14, N = 15 SE +/- 0.01, N = 3 SE +/- 0.00, N = 3 2.98 1.60 1.45 1.42 1.42 1.40 1.40 1.40 1.39 1.38 1.38 1.38 1.38 1.38 1.38 1.37 1.33 1.27 MIN: 1.29 / MAX: 144.96 MIN: 0.95 / MAX: 433.24 MIN: 1.38 / MAX: 2.96 MIN: 1.36 / MAX: 2.02 MIN: 1.35 / MAX: 1.88 MIN: 1.34 / MAX: 2.1 MIN: 1.34 / MAX: 2.15 MIN: 1.34 / MAX: 2 MIN: 1.36 / MAX: 1.53 MIN: 1.36 / MAX: 1.71 MIN: 1.35 / MAX: 2.23 MIN: 1.36 / MAX: 1.62 MIN: 1.35 / MAX: 2.05 MIN: 1.35 / MAX: 1.67 MIN: 1.35 / MAX: 2.06 MIN: 1.34 / MAX: 2.11 MIN: 1.27 / MAX: 1.77 MIN: 1.21 / MAX: 1.95 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread
NCNN Target: CPU - Model: googlenet OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: CPU - Model: googlenet 3070 4090 rep i 4090 RTX 3070 Ti nv 4090 4080 rep 4080 4080 zzz 4080 xxx g a c f 3090 rep 3090 d b 4 8 12 16 20 SE +/- 0.22, N = 15 SE +/- 0.11, N = 3 SE +/- 0.01, N = 3 18.25 10.86 10.30 10.27 9.69 8.93 8.52 8.42 8.40 8.38 7.98 7.94 7.93 7.92 7.90 7.86 7.85 7.82 MIN: 7.5 / MAX: 267.89 MIN: 8.12 / MAX: 189.87 MIN: 8.19 / MAX: 349.57 MIN: 7.95 / MAX: 115.68 MIN: 7.29 / MAX: 407.61 MIN: 8.27 / MAX: 10.68 MIN: 7.84 / MAX: 10.21 MIN: 7.75 / MAX: 9.96 MIN: 7.72 / MAX: 10.5 MIN: 7.72 / MAX: 10.05 MIN: 7.86 / MAX: 8.78 MIN: 7.71 / MAX: 8.73 MIN: 7.82 / MAX: 8.91 MIN: 7.8 / MAX: 8.96 MIN: 7.79 / MAX: 8.74 MIN: 7.76 / MAX: 8.74 MIN: 7.71 / MAX: 8.83 MIN: 7.73 / MAX: 8.65 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread
NCNN Target: CPU - Model: vgg16 OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: CPU - Model: vgg16 3070 i nv 4090 4090 rep RTX 3070 Ti 4090 4080 rep 4080 zzz 4080 4080 xxx f g a d b c 3090 rep 3090 13 26 39 52 65 SE +/- 0.23, N = 15 SE +/- 0.30, N = 3 SE +/- 0.05, N = 3 56.64 30.96 29.54 29.24 28.36 27.75 26.11 25.40 25.37 25.03 24.19 23.78 23.75 23.51 23.49 23.45 23.43 23.43 MIN: 25.75 / MAX: 367.74 MIN: 25.92 / MAX: 328.63 MIN: 24.77 / MAX: 364.86 MIN: 26.51 / MAX: 270.71 MIN: 24.13 / MAX: 449.57 MIN: 24.58 / MAX: 282.59 MIN: 24.54 / MAX: 30.29 MIN: 24.09 / MAX: 32.86 MIN: 24.26 / MAX: 36.52 MIN: 23.85 / MAX: 28.9 MIN: 23.99 / MAX: 30.98 MIN: 23.52 / MAX: 24.89 MIN: 23.31 / MAX: 25.12 MIN: 23.19 / MAX: 24.68 MIN: 23.36 / MAX: 24.62 MIN: 23.26 / MAX: 24.51 MIN: 23.23 / MAX: 24.39 MIN: 23.2 / MAX: 24.1 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread
NCNN Target: CPU - Model: resnet18 OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: CPU - Model: resnet18 3070 4090 rep nv 4090 RTX 3070 Ti 4090 4080 f 4080 rep 4080 zzz i 4080 xxx g 3090 rep a c d 3090 b 4 8 12 16 20 SE +/- 0.17, N = 15 SE +/- 0.07, N = 3 SE +/- 0.02, N = 3 14.03 8.07 7.82 6.08 6.00 5.69 5.69 5.68 5.63 5.60 5.56 5.55 5.29 5.29 5.24 5.23 5.21 5.20 MIN: 5 / MAX: 303.38 MIN: 5.86 / MAX: 121.03 MIN: 5.54 / MAX: 303.05 MIN: 4.97 / MAX: 245.95 MIN: 5.47 / MAX: 7.29 MIN: 5.16 / MAX: 7.68 MIN: 5.22 / MAX: 92.59 MIN: 5.17 / MAX: 7.45 MIN: 5.08 / MAX: 7.55 MIN: 5.13 / MAX: 6.83 MIN: 5.09 / MAX: 6.84 MIN: 5.19 / MAX: 25.4 MIN: 5.18 / MAX: 6.19 MIN: 5.09 / MAX: 6.29 MIN: 5.15 / MAX: 6.09 MIN: 5.1 / MAX: 6.28 MIN: 5.09 / MAX: 6.04 MIN: 5.1 / MAX: 5.9 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread
NCNN Target: CPU - Model: alexnet OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: CPU - Model: alexnet 3070 RTX 3070 Ti i 4090 rep nv 4090 4090 4080 4080 zzz 4080 xxx 4080 rep a f g b 3090 rep c 3090 d 3 6 9 12 15 SE +/- 0.21, N = 14 SE +/- 0.11, N = 3 SE +/- 0.01, N = 3 9.62 5.49 5.30 5.23 5.18 5.14 4.75 4.69 4.68 4.67 4.41 4.36 4.32 4.32 4.31 4.31 4.30 4.30 MIN: 4.31 / MAX: 147.6 MIN: 4.26 / MAX: 363.39 MIN: 4.92 / MAX: 7.18 MIN: 4.78 / MAX: 7.33 MIN: 4.75 / MAX: 7.12 MIN: 4.73 / MAX: 6.32 MIN: 4.31 / MAX: 13.88 MIN: 4.29 / MAX: 5.78 MIN: 4.28 / MAX: 6.37 MIN: 4.27 / MAX: 5.88 MIN: 4.24 / MAX: 5.16 MIN: 4.29 / MAX: 5.7 MIN: 4.25 / MAX: 5.17 MIN: 4.26 / MAX: 5.15 MIN: 4.26 / MAX: 5.18 MIN: 4.26 / MAX: 4.98 MIN: 4.25 / MAX: 4.83 MIN: 4.23 / MAX: 5.32 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread
NCNN Target: CPU - Model: resnet50 OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: CPU - Model: resnet50 3070 4090 4090 rep RTX 3070 Ti i 4080 rep nv 4090 4080 4080 zzz f 4080 xxx g 3090 a c b d 3090 rep 5 10 15 20 25 SE +/- 0.26, N = 15 SE +/- 0.23, N = 3 SE +/- 0.01, N = 3 21.50 14.10 13.73 12.73 12.09 11.76 11.41 11.16 11.10 11.05 10.82 10.72 10.30 10.20 10.11 10.01 10.00 9.95 MIN: 10.24 / MAX: 116.85 MIN: 10.27 / MAX: 287 MIN: 10.4 / MAX: 137.78 MIN: 10.18 / MAX: 541.92 MIN: 11.16 / MAX: 13.48 MIN: 10.68 / MAX: 44.94 MIN: 10.57 / MAX: 12.22 MIN: 10.29 / MAX: 15.03 MIN: 10.2 / MAX: 13.06 MIN: 10.14 / MAX: 162.88 MIN: 9.9 / MAX: 12.26 MIN: 10.1 / MAX: 108.3 MIN: 9.82 / MAX: 17.56 MIN: 9.84 / MAX: 12.48 MIN: 9.95 / MAX: 16.18 MIN: 9.85 / MAX: 11.06 MIN: 9.86 / MAX: 11.02 MIN: 9.85 / MAX: 10.72 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread
NCNN Target: CPU - Model: yolov4-tiny OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: CPU - Model: yolov4-tiny 3070 g 4090 rep nv 4090 RTX 3070 Ti i 3090 4080 rep 4080 4090 4080 zzz 4080 xxx f d a c 3090 rep b 7 14 21 28 35 SE +/- 0.18, N = 15 SE +/- 0.05, N = 3 SE +/- 0.11, N = 3 27.66 17.23 16.79 15.40 15.20 14.65 14.26 14.03 13.85 13.68 13.63 13.60 13.32 12.95 12.90 12.81 12.77 12.74 MIN: 12.74 / MAX: 294.9 MIN: 12.99 / MAX: 196.66 MIN: 14.1 / MAX: 273.41 MIN: 12.35 / MAX: 321.43 MIN: 12.69 / MAX: 431.37 MIN: 12.44 / MAX: 202.68 MIN: 14.17 / MAX: 14.53 MIN: 13.15 / MAX: 15.97 MIN: 12.84 / MAX: 16.75 MIN: 12.83 / MAX: 14.63 MIN: 12.77 / MAX: 15.36 MIN: 12.8 / MAX: 16.23 MIN: 12.95 / MAX: 35.49 MIN: 12.75 / MAX: 18.88 MIN: 12.69 / MAX: 15.88 MIN: 12.74 / MAX: 13.2 MIN: 12.7 / MAX: 13.02 MIN: 12.66 / MAX: 13.28 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread
NCNN Target: CPU - Model: squeezenet_ssd OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: CPU - Model: squeezenet_ssd 3070 nv 4090 i RTX 3070 Ti 4090 rep 4090 4080 rep 4080 4080 xxx 4080 zzz 3090 f g 3090 rep c d a b 3 6 9 12 15 SE +/- 0.24, N = 15 SE +/- 0.01, N = 3 SE +/- 0.01, N = 3 13.20 9.11 8.96 8.13 7.98 7.86 7.86 7.73 7.62 7.55 7.52 7.23 7.13 7.12 7.10 7.09 7.07 7.06 MIN: 6.9 / MAX: 68.61 MIN: 6.77 / MAX: 101.58 MIN: 6.92 / MAX: 244.02 MIN: 6.37 / MAX: 399.11 MIN: 7.32 / MAX: 16.07 MIN: 7.25 / MAX: 8.98 MIN: 7.22 / MAX: 10.84 MIN: 7.13 / MAX: 9.7 MIN: 7.01 / MAX: 8.84 MIN: 7 / MAX: 8.72 MIN: 7.45 / MAX: 7.74 MIN: 7.15 / MAX: 8.02 MIN: 7.04 / MAX: 8.43 MIN: 7.05 / MAX: 7.63 MIN: 7.05 / MAX: 7.65 MIN: 6.99 / MAX: 9.39 MIN: 7.01 / MAX: 8.07 MIN: 7.01 / MAX: 7.55 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread
NCNN Target: CPU - Model: regnety_400m OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: CPU - Model: regnety_400m 3070 nv 4090 i 4090 rep RTX 3070 Ti 4080 rep 4080 xxx 4080 zzz g c 3090 3090 rep 4080 d b a 4090 f 4 8 12 16 20 SE +/- 0.19, N = 15 SE +/- 0.06, N = 3 SE +/- 0.04, N = 3 18.00 10.17 9.94 9.66 8.83 8.67 8.45 8.37 8.30 8.27 8.25 8.24 8.24 8.23 8.18 8.18 8.13 8.08 MIN: 7.91 / MAX: 176.28 MIN: 8.12 / MAX: 209.53 MIN: 7.43 / MAX: 166.02 MIN: 7.78 / MAX: 95.3 MIN: 7.65 / MAX: 351.08 MIN: 8.22 / MAX: 15.29 MIN: 8.12 / MAX: 9.68 MIN: 8.05 / MAX: 10.19 MIN: 8.22 / MAX: 9.1 MIN: 8.22 / MAX: 9.18 MIN: 8.17 / MAX: 8.9 MIN: 8.17 / MAX: 8.84 MIN: 7.89 / MAX: 9.52 MIN: 8.03 / MAX: 8.9 MIN: 8.12 / MAX: 8.86 MIN: 8.07 / MAX: 9.68 MIN: 7.78 / MAX: 9.98 MIN: 7.98 / MAX: 10.87 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread
NCNN Target: CPU - Model: vision_transformer OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: CPU - Model: vision_transformer 3070 nv 4090 4090 RTX 3070 Ti 4090 rep i 4080 rep 4080 4080 xxx 4080 zzz f 3090 g a d b 3090 rep c 20 40 60 80 100 SE +/- 0.12, N = 15 SE +/- 0.29, N = 3 SE +/- 0.39, N = 3 75.34 38.90 38.25 37.91 37.81 37.80 35.28 35.07 34.27 34.10 33.56 33.01 32.73 32.49 32.43 31.95 31.80 31.77 MIN: 38.72 / MAX: 418.01 MIN: 34.2 / MAX: 300.84 MIN: 33.04 / MAX: 447.7 MIN: 32.08 / MAX: 541.11 MIN: 32.66 / MAX: 453.44 MIN: 33.74 / MAX: 321.51 MIN: 33.9 / MAX: 38.67 MIN: 33.14 / MAX: 43.26 MIN: 32.82 / MAX: 39.79 MIN: 32.65 / MAX: 37.64 MIN: 32.98 / MAX: 51.93 MIN: 32.88 / MAX: 33.42 MIN: 31.44 / MAX: 81.32 MIN: 31.67 / MAX: 40.11 MIN: 31.56 / MAX: 37.69 MIN: 31.79 / MAX: 32.33 MIN: 31.66 / MAX: 32.23 MIN: 31.61 / MAX: 35.68 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread
NCNN Target: CPU - Model: FastestDet OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: CPU - Model: FastestDet 3070 4090 4090 rep nv 4090 4080 4080 rep f 3090 4080 xxx 4080 zzz c 3090 rep d b RTX 3070 Ti a i g 2 4 6 8 10 SE +/- 0.01, N = 3 SE +/- 0.23, N = 15 SE +/- 0.45, N = 3 8.65 5.48 5.27 4.51 4.42 4.34 4.22 4.21 4.17 4.16 4.11 4.08 4.08 4.05 3.94 3.62 2.66 2.57 MIN: 3.94 / MAX: 185.21 MIN: 2.67 / MAX: 259.34 MIN: 4.05 / MAX: 247.02 MIN: 4.34 / MAX: 5.96 MIN: 4.25 / MAX: 6.71 MIN: 4.19 / MAX: 5.77 MIN: 4.18 / MAX: 4.97 MIN: 4.19 / MAX: 4.41 MIN: 4.05 / MAX: 4.74 MIN: 4 / MAX: 4.69 MIN: 4.08 / MAX: 4.4 MIN: 4.05 / MAX: 4.84 MIN: 4.02 / MAX: 4.28 MIN: 4.02 / MAX: 4.35 MIN: 2.43 / MAX: 267.02 MIN: 2.7 / MAX: 4.54 MIN: 2.54 / MAX: 3.41 MIN: 2.53 / MAX: 3.21 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread
NCNN Target: CPU-v3-v3 - Model: mobilenet-v3 OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: CPU-v3-v3 - Model: mobilenet-v3 3070 RTX 3070 Ti 4090 nv 4090 4090 rep 4080 xxx 4080 rep i 4080 zzz 4080 3090 rep 3090 a d c g 2 4 6 8 10 SE +/- 0.21, N = 14 SE +/- 0.00, N = 2 SE +/- 0.00, N = 3 7.52 3.61 3.53 3.47 3.41 3.27 3.26 3.26 3.24 3.24 3.19 3.18 3.18 3.17 3.17 3.14 MIN: 2.94 / MAX: 215 MIN: 2.51 / MAX: 502.85 MIN: 3.39 / MAX: 4.31 MIN: 3.32 / MAX: 4.91 MIN: 3.27 / MAX: 5.24 MIN: 3.13 / MAX: 3.85 MIN: 3.09 / MAX: 3.96 MIN: 3.14 / MAX: 3.9 MIN: 3.11 / MAX: 4.47 MIN: 3.09 / MAX: 4.73 MIN: 3.15 / MAX: 3.72 MIN: 3.14 / MAX: 4.14 MIN: 3.14 / MAX: 3.82 MIN: 3.12 / MAX: 3.96 MIN: 3.15 / MAX: 3.74 MIN: 3.1 / MAX: 3.81 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread
NCNN Target: Vulkan GPU - Model: mobilenet OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: Vulkan GPU - Model: mobilenet 3070 4090 i 4090 rep RTX 3070 Ti 4080 rep 4080 zzz nv 4090 4080 4080 xxx f g 3090 a e b 3090 rep c d 5 10 15 20 25 SE +/- 0.25, N = 15 SE +/- 0.02, N = 3 SE +/- 0.02, N = 3 SE +/- 0.00, N = 3 21.11 10.55 10.40 10.23 9.35 8.57 8.47 8.45 8.44 8.37 8.27 8.17 8.07 8.05 8.04 8.04 8.03 8.03 8.02 MIN: 7.98 / MAX: 322.43 MIN: 8.22 / MAX: 303.1 MIN: 7.97 / MAX: 455.46 MIN: 8.13 / MAX: 386.42 MIN: 7.49 / MAX: 474.12 MIN: 7.98 / MAX: 10 MIN: 8.04 / MAX: 10.17 MIN: 8.03 / MAX: 12.61 MIN: 7.98 / MAX: 10.55 MIN: 7.97 / MAX: 16.09 MIN: 8.17 / MAX: 9.04 MIN: 8.08 / MAX: 9.37 MIN: 7.99 / MAX: 8.8 MIN: 7.95 / MAX: 8.89 MIN: 7.95 / MAX: 9.09 MIN: 7.95 / MAX: 14.33 MIN: 7.96 / MAX: 8.77 MIN: 7.98 / MAX: 8.84 MIN: 7.95 / MAX: 9.81 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread
NCNN Target: Vulkan GPU-v2-v2 - Model: mobilenet-v2 OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: Vulkan GPU-v2-v2 - Model: mobilenet-v2 3070 RTX 3070 Ti nv 4090 4090 rep 4090 4080 zzz 4080 rep i 4080 xxx 4080 3090 g 3090 rep a d e b f c 2 4 6 8 10 SE +/- 0.14, N = 15 SE +/- 0.01, N = 3 SE +/- 0.01, N = 3 SE +/- 0.01, N = 3 7.81 3.56 3.43 3.31 3.30 3.29 3.29 3.29 3.26 3.26 3.18 3.18 3.17 3.17 3.16 3.14 3.14 3.13 3.13 MIN: 3.07 / MAX: 154.75 MIN: 3.09 / MAX: 345.01 MIN: 3.25 / MAX: 4.81 MIN: 3.14 / MAX: 4.92 MIN: 3.12 / MAX: 4.82 MIN: 3.11 / MAX: 3.98 MIN: 3.12 / MAX: 4.14 MIN: 3.12 / MAX: 3.93 MIN: 3.1 / MAX: 3.87 MIN: 3.1 / MAX: 4.12 MIN: 3.14 / MAX: 3.63 MIN: 3.13 / MAX: 3.9 MIN: 3.12 / MAX: 3.64 MIN: 3.09 / MAX: 3.78 MIN: 3.09 / MAX: 3.92 MIN: 3.08 / MAX: 4.06 MIN: 3.1 / MAX: 3.73 MIN: 3.07 / MAX: 3.82 MIN: 3.08 / MAX: 3.85 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread
NCNN Target: Vulkan GPU-v3-v3 - Model: mobilenet-v3 OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: Vulkan GPU-v3-v3 - Model: mobilenet-v3 3070 i RTX 3070 Ti nv 4090 4090 rep 4090 4080 zzz c 3090 e d a 3090 rep f 2 4 6 8 10 SE +/- 0.20, N = 14 SE +/- 0.02, N = 3 SE +/- 0.00, N = 2 SE +/- 0.02, N = 3 6.60 4.87 3.64 3.36 3.30 3.28 3.28 3.20 3.19 3.18 3.17 3.17 3.16 3.14 MIN: 2.98 / MAX: 166.19 MIN: 3.14 / MAX: 278.98 MIN: 2.87 / MAX: 429.02 MIN: 3.21 / MAX: 4.3 MIN: 3.15 / MAX: 3.92 MIN: 3.15 / MAX: 3.9 MIN: 3.13 / MAX: 4.65 MIN: 3.16 / MAX: 3.68 MIN: 3.14 / MAX: 3.48 MIN: 3.11 / MAX: 3.78 MIN: 3.1 / MAX: 3.83 MIN: 3.11 / MAX: 3.73 MIN: 3.11 / MAX: 3.62 MIN: 3.09 / MAX: 3.54 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread
NCNN Target: Vulkan GPU - Model: shufflenet-v2 OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: Vulkan GPU - Model: shufflenet-v2 3070 RTX 3070 Ti nv 4090 4080 xxx 4090 rep i 4080 zzz 4090 4080 rep 4080 f 3090 3090 rep g d a e b c 2 4 6 8 10 SE +/- 0.20, N = 15 SE +/- 0.01, N = 3 SE +/- 0.01, N = 3 SE +/- 0.01, N = 3 7.07 3.98 3.51 3.50 3.49 3.49 3.46 3.45 3.44 3.43 3.40 3.39 3.36 3.35 3.35 3.35 3.33 3.33 3.32 MIN: 3.25 / MAX: 243.32 MIN: 3.14 / MAX: 529.82 MIN: 3.37 / MAX: 4 MIN: 3.37 / MAX: 4.85 MIN: 3.36 / MAX: 4.33 MIN: 3.35 / MAX: 4.24 MIN: 3.32 / MAX: 5.24 MIN: 3.32 / MAX: 3.99 MIN: 3.3 / MAX: 5.36 MIN: 3.3 / MAX: 4.22 MIN: 3.35 / MAX: 5.89 MIN: 3.35 / MAX: 3.69 MIN: 3.32 / MAX: 4.06 MIN: 3.3 / MAX: 4.02 MIN: 3.3 / MAX: 3.82 MIN: 3.29 / MAX: 3.85 MIN: 3.28 / MAX: 4.14 MIN: 3.3 / MAX: 3.59 MIN: 3.29 / MAX: 4.19 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread
NCNN Target: Vulkan GPU - Model: mnasnet OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: Vulkan GPU - Model: mnasnet 3070 nv 4090 RTX 3070 Ti i 4090 4090 rep 4080 rep 4080 xxx 4080 4080 zzz 3090 g a 3090 rep f d e c b 1.1453 2.2906 3.4359 4.5812 5.7265 SE +/- 0.16, N = 15 SE +/- 0.01, N = 3 SE +/- 0.01, N = 3 SE +/- 0.01, N = 3 5.09 4.70 3.40 3.20 3.18 3.15 3.08 3.07 3.07 3.06 2.99 2.98 2.98 2.97 2.97 2.97 2.96 2.96 2.95 MIN: 2.86 / MAX: 53.75 MIN: 3 / MAX: 188.08 MIN: 2.72 / MAX: 432.18 MIN: 3.07 / MAX: 3.86 MIN: 3.05 / MAX: 4.64 MIN: 3 / MAX: 4.54 MIN: 2.94 / MAX: 3.67 MIN: 2.95 / MAX: 4.19 MIN: 2.93 / MAX: 4.63 MIN: 2.92 / MAX: 3.73 MIN: 2.96 / MAX: 3.14 MIN: 2.94 / MAX: 3.65 MIN: 2.92 / MAX: 4.03 MIN: 2.94 / MAX: 3.28 MIN: 2.93 / MAX: 3.66 MIN: 2.92 / MAX: 3.34 MIN: 2.91 / MAX: 5.9 MIN: 2.93 / MAX: 3.41 MIN: 2.92 / MAX: 3.42 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread
NCNN Target: Vulkan GPU - Model: efficientnet-b0 OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: Vulkan GPU - Model: efficientnet-b0 3070 i RTX 3070 Ti 4090 nv 4090 4090 rep 4080 rep 4080 zzz 4080 xxx 4080 3090 d 3090 rep g f a e c b 3 6 9 12 15 SE +/- 0.18, N = 15 SE +/- 0.02, N = 3 SE +/- 0.01, N = 3 SE +/- 0.01, N = 3 8.99 5.88 4.53 4.34 4.10 4.09 4.05 4.04 4.04 4.01 3.88 3.87 3.86 3.86 3.86 3.86 3.84 3.83 3.82 MIN: 3.71 / MAX: 129.99 MIN: 4.04 / MAX: 364.21 MIN: 3.75 / MAX: 396.62 MIN: 4.14 / MAX: 5.84 MIN: 3.86 / MAX: 5.46 MIN: 3.87 / MAX: 5.46 MIN: 3.83 / MAX: 6.11 MIN: 3.8 / MAX: 5.31 MIN: 3.82 / MAX: 5.33 MIN: 3.78 / MAX: 5.34 MIN: 3.84 / MAX: 4.39 MIN: 3.77 / MAX: 9.91 MIN: 3.82 / MAX: 4.34 MIN: 3.82 / MAX: 4.22 MIN: 3.78 / MAX: 10.45 MIN: 3.8 / MAX: 4.6 MIN: 3.79 / MAX: 4.76 MIN: 3.79 / MAX: 4.61 MIN: 3.78 / MAX: 4.39 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread
NCNN Target: Vulkan GPU - Model: blazeface OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: Vulkan GPU - Model: blazeface 3070 RTX 3070 Ti 4080 zzz 4080 xxx 4080 rep 4080 g nv 4090 4090 rep i 4090 3090 f e d a 3090 rep c b 0.8955 1.791 2.6865 3.582 4.4775 SE +/- 0.16, N = 15 SE +/- 0.01, N = 3 SE +/- 0.01, N = 3 SE +/- 0.01, N = 3 3.98 1.60 1.42 1.42 1.41 1.41 1.41 1.40 1.40 1.40 1.39 1.39 1.38 1.38 1.38 1.38 1.37 1.37 1.37 MIN: 1.31 / MAX: 228.4 MIN: 1.11 / MAX: 436.01 MIN: 1.36 / MAX: 2.01 MIN: 1.36 / MAX: 1.93 MIN: 1.35 / MAX: 1.9 MIN: 1.35 / MAX: 2.01 MIN: 1.38 / MAX: 2.09 MIN: 1.34 / MAX: 1.87 MIN: 1.33 / MAX: 1.93 MIN: 1.33 / MAX: 2 MIN: 1.33 / MAX: 1.94 MIN: 1.37 / MAX: 1.82 MIN: 1.35 / MAX: 2.08 MIN: 1.34 / MAX: 1.88 MIN: 1.34 / MAX: 2.25 MIN: 1.34 / MAX: 1.85 MIN: 1.36 / MAX: 1.46 MIN: 1.35 / MAX: 1.82 MIN: 1.35 / MAX: 1.75 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread
NCNN Target: Vulkan GPU - Model: googlenet OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: Vulkan GPU - Model: googlenet 3070 4090 rep 4090 RTX 3070 Ti g i nv 4090 4080 xxx 4080 4080 zzz 4080 rep f a 3090 e d b 3090 rep c 5 10 15 20 25 SE +/- 0.22, N = 15 SE +/- 0.02, N = 3 SE +/- 0.01, N = 3 SE +/- 0.04, N = 3 19.49 10.65 10.62 9.87 8.96 8.75 8.70 8.42 8.42 8.41 8.40 8.15 7.90 7.87 7.85 7.85 7.85 7.82 7.80 MIN: 7.4 / MAX: 200.01 MIN: 8.29 / MAX: 236.11 MIN: 7.83 / MAX: 323.31 MIN: 7.33 / MAX: 399.24 MIN: 8.82 / MAX: 9.87 MIN: 8.08 / MAX: 16.01 MIN: 7.96 / MAX: 10.01 MIN: 7.73 / MAX: 10.06 MIN: 7.79 / MAX: 10.01 MIN: 7.72 / MAX: 9.9 MIN: 7.77 / MAX: 9.78 MIN: 8.02 / MAX: 9.02 MIN: 7.74 / MAX: 9.54 MIN: 7.76 / MAX: 10.36 MIN: 7.71 / MAX: 8.76 MIN: 7.71 / MAX: 8.85 MIN: 7.76 / MAX: 8.76 MIN: 7.69 / MAX: 8.61 MIN: 7.72 / MAX: 8.74 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread
NCNN Target: Vulkan GPU - Model: vgg16 OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: Vulkan GPU - Model: vgg16 3070 nv 4090 RTX 3070 Ti 4090 i 4090 rep 4080 zzz 4080 rep 4080 xxx 4080 f g e d b 3090 c a 3090 rep 11 22 33 44 55 SE +/- 0.24, N = 15 SE +/- 0.14, N = 3 SE +/- 0.10, N = 3 SE +/- 0.04, N = 3 48.29 29.29 29.06 28.82 27.83 27.04 25.82 25.04 25.01 25.00 24.55 24.20 23.60 23.56 23.56 23.55 23.54 23.51 23.48 MIN: 24.97 / MAX: 183.12 MIN: 24.63 / MAX: 296.95 MIN: 24.11 / MAX: 541.55 MIN: 24.35 / MAX: 214.1 MIN: 24.98 / MAX: 262.23 MIN: 24.22 / MAX: 296.13 MIN: 24.35 / MAX: 62.94 MIN: 24.06 / MAX: 27.35 MIN: 23.8 / MAX: 26.41 MIN: 23.93 / MAX: 26.69 MIN: 23.62 / MAX: 97.69 MIN: 23.56 / MAX: 58.31 MIN: 23.17 / MAX: 24.71 MIN: 23.24 / MAX: 24.78 MIN: 23.34 / MAX: 24.72 MIN: 23.3 / MAX: 24.45 MIN: 23.33 / MAX: 24.61 MIN: 23.29 / MAX: 24.68 MIN: 23.24 / MAX: 29.21 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread
NCNN Target: Vulkan GPU - Model: resnet18 OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: Vulkan GPU - Model: resnet18 3070 nv 4090 RTX 3070 Ti g 4090 rep i 4090 4080 4080 xxx 4080 rep 4080 zzz f a 3090 d b e c 3090 rep 3 6 9 12 15 SE +/- 0.20, N = 15 SE +/- 0.01, N = 3 SE +/- 0.03, N = 3 SE +/- 0.01, N = 3 12.68 7.44 6.28 6.22 6.01 5.82 5.69 5.67 5.62 5.61 5.59 5.48 5.28 5.27 5.23 5.23 5.22 5.21 5.20 MIN: 5.39 / MAX: 262.62 MIN: 5.29 / MAX: 320.54 MIN: 4.94 / MAX: 298.06 MIN: 6.11 / MAX: 7 MIN: 5.44 / MAX: 8.18 MIN: 5.28 / MAX: 7.02 MIN: 5.16 / MAX: 8.22 MIN: 5.18 / MAX: 7.22 MIN: 5.1 / MAX: 7.65 MIN: 5.11 / MAX: 7.44 MIN: 5.06 / MAX: 6.95 MIN: 5.33 / MAX: 6.16 MIN: 5.17 / MAX: 6.16 MIN: 5.15 / MAX: 6.19 MIN: 5.08 / MAX: 6.28 MIN: 5.13 / MAX: 6.18 MIN: 5.09 / MAX: 11.15 MIN: 5.11 / MAX: 6.04 MIN: 5.09 / MAX: 5.98 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread
NCNN Target: Vulkan GPU - Model: alexnet OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: Vulkan GPU - Model: alexnet 3070 4090 rep i RTX 3070 Ti nv 4090 g 4080 zzz 4080 xxx 4080 rep 4090 f 4080 3090 c b 3090 rep e d a 3 6 9 12 15 SE +/- 0.18, N = 15 SE +/- 0.00, N = 3 SE +/- 0.01, N = 3 SE +/- 0.01, N = 3 10.88 6.79 6.53 5.25 5.20 4.87 4.68 4.68 4.65 4.64 4.64 4.62 4.35 4.33 4.33 4.31 4.31 4.31 4.31 MIN: 4.38 / MAX: 52.99 MIN: 4.23 / MAX: 262.43 MIN: 4.57 / MAX: 242.16 MIN: 4.23 / MAX: 375.94 MIN: 4.82 / MAX: 7.07 MIN: 4.8 / MAX: 5.62 MIN: 4.26 / MAX: 6.23 MIN: 4.26 / MAX: 6.61 MIN: 4.26 / MAX: 6.53 MIN: 4.26 / MAX: 5.98 MIN: 4.57 / MAX: 5.49 MIN: 4.26 / MAX: 6.15 MIN: 4.28 / MAX: 7.49 MIN: 4.26 / MAX: 10.59 MIN: 4.28 / MAX: 5.16 MIN: 4.26 / MAX: 5.26 MIN: 4.23 / MAX: 11.03 MIN: 4.25 / MAX: 5.28 MIN: 4.24 / MAX: 5.2 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread
NCNN Target: Vulkan GPU - Model: resnet50 OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: Vulkan GPU - Model: resnet50 3070 4090 4090 rep i RTX 3070 Ti nv 4090 4080 zzz 4080 xxx 4080 rep 4080 g f 3090 e d 3090 rep a c b 6 12 18 24 30 SE +/- 0.26, N = 15 SE +/- 0.12, N = 3 SE +/- 0.09, N = 3 SE +/- 0.02, N = 3 23.48 14.13 13.08 12.96 12.60 12.45 11.07 10.94 10.84 10.81 10.34 10.26 10.10 10.10 10.10 10.06 10.01 10.00 10.00 MIN: 10.06 / MAX: 112.91 MIN: 10.63 / MAX: 167.28 MIN: 10.11 / MAX: 444.45 MIN: 10.23 / MAX: 424.46 MIN: 9.82 / MAX: 418.4 MIN: 11.55 / MAX: 14.48 MIN: 10.1 / MAX: 13.23 MIN: 9.95 / MAX: 12.7 MIN: 9.93 / MAX: 12.81 MIN: 9.95 / MAX: 12.78 MIN: 10.14 / MAX: 11.37 MIN: 10.09 / MAX: 11.22 MIN: 9.97 / MAX: 11.42 MIN: 9.84 / MAX: 11.72 MIN: 9.86 / MAX: 11.08 MIN: 9.95 / MAX: 11.04 MIN: 9.88 / MAX: 11.4 MIN: 9.91 / MAX: 11.15 MIN: 9.92 / MAX: 12.35 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread
NCNN Target: Vulkan GPU - Model: yolov4-tiny OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: Vulkan GPU - Model: yolov4-tiny 3070 4090 rep RTX 3070 Ti nv 4090 i 4090 4080 zzz 4080 4080 rep 4080 xxx g f 3090 e b d a 3090 rep c 7 14 21 28 35 SE +/- 0.18, N = 15 SE +/- 0.04, N = 3 SE +/- 0.01, N = 3 SE +/- 0.02, N = 3 28.59 15.72 15.54 15.26 15.16 13.97 13.80 13.79 13.67 13.65 13.64 13.17 12.88 12.87 12.87 12.85 12.84 12.83 12.81 MIN: 12.87 / MAX: 325.37 MIN: 13.2 / MAX: 301.81 MIN: 12.15 / MAX: 492.01 MIN: 12.87 / MAX: 132.82 MIN: 12.86 / MAX: 248.64 MIN: 13.11 / MAX: 16.15 MIN: 12.76 / MAX: 15.76 MIN: 12.75 / MAX: 19.63 MIN: 12.71 / MAX: 14.88 MIN: 12.71 / MAX: 14.99 MIN: 13.04 / MAX: 76.32 MIN: 13.03 / MAX: 14.1 MIN: 12.76 / MAX: 13.67 MIN: 12.68 / MAX: 13.84 MIN: 12.76 / MAX: 13.73 MIN: 12.72 / MAX: 13.93 MIN: 12.69 / MAX: 15.33 MIN: 12.74 / MAX: 13.59 MIN: 12.73 / MAX: 13.08 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread
NCNN Target: Vulkan GPU - Model: squeezenet_ssd OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: Vulkan GPU - Model: squeezenet_ssd 3070 nv 4090 RTX 3070 Ti 4090 rep 4090 4080 rep 4080 zzz 4080 xxx 4080 i 3090 g a f d b 3090 rep c e 4 8 12 16 20 SE +/- 0.26, N = 15 SE +/- 0.02, N = 3 SE +/- 0.02, N = 3 SE +/- 0.02, N = 3 15.82 9.11 8.47 8.22 7.83 7.64 7.63 7.62 7.58 7.46 7.16 7.10 7.09 7.08 7.08 7.07 7.06 7.06 7.05 MIN: 6.99 / MAX: 82.57 MIN: 6.35 / MAX: 130.38 MIN: 6.29 / MAX: 533.92 MIN: 7.56 / MAX: 9.8 MIN: 7.21 / MAX: 9.32 MIN: 7.05 / MAX: 9.12 MIN: 7 / MAX: 9.17 MIN: 7.01 / MAX: 9.28 MIN: 6.98 / MAX: 9.05 MIN: 6.9 / MAX: 8.9 MIN: 7.05 / MAX: 13.55 MIN: 6.99 / MAX: 8.59 MIN: 6.98 / MAX: 7.95 MIN: 6.98 / MAX: 8.07 MIN: 6.97 / MAX: 7.99 MIN: 7 / MAX: 8.07 MIN: 7 / MAX: 7.82 MIN: 7 / MAX: 8.03 MIN: 6.95 / MAX: 8 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread
NCNN Target: Vulkan GPU - Model: regnety_400m OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: Vulkan GPU - Model: regnety_400m 3070 i nv 4090 RTX 3070 Ti 4090 4080 zzz 4080 xxx 4080 rep 4090 rep 4080 3090 g f b d a e 3090 rep c 4 8 12 16 20 SE +/- 0.21, N = 15 SE +/- 0.06, N = 3 SE +/- 0.11, N = 3 SE +/- 0.02, N = 3 16.22 9.88 9.81 9.07 8.64 8.58 8.56 8.56 8.48 8.39 8.38 8.36 8.34 8.21 8.17 8.16 8.10 8.02 8.00 MIN: 7.74 / MAX: 314.84 MIN: 8.14 / MAX: 251.77 MIN: 7.82 / MAX: 241.19 MIN: 7.61 / MAX: 402.49 MIN: 8.28 / MAX: 10.42 MIN: 8.13 / MAX: 9.78 MIN: 8.15 / MAX: 9.8 MIN: 8.17 / MAX: 10.28 MIN: 8.09 / MAX: 9.64 MIN: 8 / MAX: 10.29 MIN: 8.31 / MAX: 8.86 MIN: 8.27 / MAX: 9.08 MIN: 7.99 / MAX: 26.72 MIN: 8.14 / MAX: 8.84 MIN: 7.99 / MAX: 8.97 MIN: 7.9 / MAX: 8.99 MIN: 7.98 / MAX: 8.84 MIN: 7.95 / MAX: 8.63 MIN: 7.94 / MAX: 8.88 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread
NCNN Target: Vulkan GPU - Model: vision_transformer OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: Vulkan GPU - Model: vision_transformer 3070 nv 4090 4090 RTX 3070 Ti 4090 rep i 4080 4080 rep 4080 zzz 4080 xxx f g d 3090 e 3090 rep a b c 16 32 48 64 80 SE +/- 0.16, N = 15 SE +/- 0.21, N = 3 SE +/- 0.07, N = 3 SE +/- 0.09, N = 3 70.76 39.04 38.76 38.03 37.59 36.42 35.56 35.07 34.32 34.19 32.92 32.42 32.12 31.94 31.93 31.91 31.88 31.85 31.79 MIN: 38.81 / MAX: 250.01 MIN: 33.83 / MAX: 463.88 MIN: 33.12 / MAX: 539.58 MIN: 32.66 / MAX: 467.28 MIN: 34.45 / MAX: 457.98 MIN: 33.49 / MAX: 224.86 MIN: 33.19 / MAX: 40.43 MIN: 33.66 / MAX: 39.36 MIN: 32.58 / MAX: 41.88 MIN: 32.72 / MAX: 36.79 MIN: 32.67 / MAX: 36.93 MIN: 31.89 / MAX: 65.47 MIN: 31.66 / MAX: 46.9 MIN: 31.73 / MAX: 34.21 MIN: 31.62 / MAX: 35.85 MIN: 31.74 / MAX: 34.28 MIN: 31.55 / MAX: 37.47 MIN: 31.69 / MAX: 33.06 MIN: 31.63 / MAX: 35.57 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread
NCNN Target: Vulkan GPU - Model: FastestDet OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: Vulkan GPU - Model: FastestDet 3070 i 4080 zzz 4090 rep 4090 RTX 3070 Ti f 4080 rep 4080 xxx 4080 3090 d a c 3090 rep e g b nv 4090 2 4 6 8 10 SE +/- 0.29, N = 15 SE +/- 0.02, N = 3 SE +/- 0.00, N = 3 SE +/- 0.00, N = 3 8.41 5.14 4.79 4.59 4.39 4.26 4.24 4.21 4.20 4.20 4.11 4.11 4.10 4.09 4.08 4.08 4.07 4.07 3.93 MIN: 2.89 / MAX: 487.78 MIN: 3.7 / MAX: 81.79 MIN: 4.64 / MAX: 6.21 MIN: 2.62 / MAX: 232.18 MIN: 4.25 / MAX: 5.86 MIN: 2.5 / MAX: 396.93 MIN: 3.88 / MAX: 24.21 MIN: 4.04 / MAX: 4.97 MIN: 4.03 / MAX: 6.49 MIN: 4.02 / MAX: 4.97 MIN: 4.07 / MAX: 4.29 MIN: 4.01 / MAX: 9.72 MIN: 4.06 / MAX: 4.81 MIN: 4.05 / MAX: 5.5 MIN: 4.04 / MAX: 4.35 MIN: 4.03 / MAX: 5.29 MIN: 4.02 / MAX: 4.82 MIN: 4.04 / MAX: 4.53 MIN: 3.8 / MAX: 5.4 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread
NCNN Target: CPU-v3-v3-v3 - Model: mobilenet OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: CPU-v3-v3-v3 - Model: mobilenet 3070 i RTX 3070 Ti g 4090 nv 4090 4090 rep f 4080 rep 4080 xxx 4080 4080 zzz 3090 3090 rep b c 4 8 12 16 20 SE +/- 0.23, N = 15 17.82 10.08 9.62 8.98 8.96 8.91 8.74 8.56 8.48 8.44 8.43 8.40 8.06 8.03 8.01 8.00 MIN: 7.57 / MAX: 211.62 MIN: 8.08 / MAX: 286.28 MIN: 7.76 / MAX: 454.91 MIN: 8.1 / MAX: 124.43 MIN: 8.39 / MAX: 10.77 MIN: 8.33 / MAX: 10.07 MIN: 8.25 / MAX: 10.5 MIN: 8.04 / MAX: 75.44 MIN: 7.96 / MAX: 10.32 MIN: 7.97 / MAX: 10.71 MIN: 7.99 / MAX: 10.44 MIN: 8.12 / MAX: 10.11 MIN: 7.94 / MAX: 13.92 MIN: 7.98 / MAX: 8.77 MIN: 7.95 / MAX: 8.95 MIN: 7.96 / MAX: 8.63 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread
NCNN Target: CPU-v3-v3-v3-v2-v2 - Model: mobilenet-v2 OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: CPU-v3-v3-v3-v2-v2 - Model: mobilenet-v2 3070 4090 4090 rep RTX 3070 Ti nv 4090 4080 4080 xxx i 4080 zzz 4080 rep 3090 rep f g b 3090 c 3 6 9 12 15 SE +/- 0.10, N = 15 9.19 3.46 3.45 3.41 3.39 3.31 3.30 3.30 3.28 3.27 3.17 3.16 3.15 3.15 3.14 3.14 MIN: 3.04 / MAX: 232.12 MIN: 3.29 / MAX: 4.38 MIN: 3.23 / MAX: 4.55 MIN: 2.99 / MAX: 184.91 MIN: 3.21 / MAX: 4.24 MIN: 3.12 / MAX: 4.76 MIN: 3.12 / MAX: 4.03 MIN: 3.14 / MAX: 4.82 MIN: 3.1 / MAX: 4 MIN: 3.1 / MAX: 4.34 MIN: 3.11 / MAX: 4.5 MIN: 3.09 / MAX: 3.89 MIN: 3.1 / MAX: 3.63 MIN: 3.1 / MAX: 3.68 MIN: 3.08 / MAX: 3.7 MIN: 3.1 / MAX: 3.67 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread
NCNN Target: CPU-v3-v3-v3-v3-v3 - Model: mobilenet-v3 OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: CPU-v3-v3-v3-v3-v3 - Model: mobilenet-v3 3070 RTX 3070 Ti 4090 rep nv 4090 4080 xxx i 4080 4080 zzz 4090 g c b 3090 rep 3090 f 1.2105 2.421 3.6315 4.842 6.0525 SE +/- 0.22, N = 14 5.38 3.76 3.53 3.35 3.31 3.29 3.28 3.27 3.25 3.16 3.16 3.16 3.15 3.15 3.15 MIN: 2.74 / MAX: 121.29 MIN: 2.89 / MAX: 366.04 MIN: 3.2 / MAX: 40.81 MIN: 3.21 / MAX: 5.23 MIN: 3.16 / MAX: 5.3 MIN: 3.15 / MAX: 4.32 MIN: 3.14 / MAX: 3.89 MIN: 3.14 / MAX: 4.63 MIN: 3.11 / MAX: 4.74 MIN: 3.11 / MAX: 3.93 MIN: 3.12 / MAX: 3.7 MIN: 3.12 / MAX: 3.69 MIN: 3.11 / MAX: 3.6 MIN: 3.11 / MAX: 3.71 MIN: 3.11 / MAX: 3.48 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread
NCNN Target: CPU-v3-v3-v3 - Model: shufflenet-v2 OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: CPU-v3-v3-v3 - Model: shufflenet-v2 3070 4090 RTX 3070 Ti 4090 rep i 4080 xxx nv 4090 4080 4080 zzz 4080 rep f g c 3090 rep b 3090 2 4 6 8 10 SE +/- 0.21, N = 15 8.13 5.18 4.09 3.59 3.52 3.47 3.46 3.46 3.43 3.43 3.40 3.38 3.34 3.33 3.33 3.32 MIN: 3.09 / MAX: 147.21 MIN: 3.34 / MAX: 283.54 MIN: 3.12 / MAX: 435.28 MIN: 3.46 / MAX: 4.09 MIN: 3.39 / MAX: 4.05 MIN: 3.33 / MAX: 5.01 MIN: 3.32 / MAX: 5.2 MIN: 3.34 / MAX: 3.93 MIN: 3.31 / MAX: 3.94 MIN: 3.3 / MAX: 4.03 MIN: 3.35 / MAX: 4.17 MIN: 3.34 / MAX: 4.15 MIN: 3.32 / MAX: 3.79 MIN: 3.3 / MAX: 3.67 MIN: 3.3 / MAX: 3.79 MIN: 3.28 / MAX: 3.66 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread
NCNN Target: CPU-v3-v3-v3 - Model: mnasnet OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: CPU-v3-v3-v3 - Model: mnasnet 3070 nv 4090 i 4090 rep 4090 f RTX 3070 Ti 4080 rep 4080 4080 xxx 4080 zzz g c b 3090 rep 3090 2 4 6 8 10 SE +/- 0.04, N = 15 6.87 4.61 3.39 3.23 3.19 3.12 3.11 3.09 3.08 3.07 3.06 3.05 2.97 2.97 2.96 2.95 MIN: 2.93 / MAX: 216.41 MIN: 2.78 / MAX: 222.99 MIN: 3.26 / MAX: 4.86 MIN: 3.1 / MAX: 3.75 MIN: 3.06 / MAX: 3.75 MIN: 3.08 / MAX: 3.86 MIN: 2.8 / MAX: 4.98 MIN: 2.95 / MAX: 4.52 MIN: 2.94 / MAX: 4.52 MIN: 2.94 / MAX: 3.6 MIN: 2.93 / MAX: 3.64 MIN: 3.01 / MAX: 3.88 MIN: 2.94 / MAX: 3.43 MIN: 2.94 / MAX: 3.43 MIN: 2.94 / MAX: 3.38 MIN: 2.92 / MAX: 3.29 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread
NCNN Target: CPU-v3-v3-v3 - Model: efficientnet-b0 OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: CPU-v3-v3-v3 - Model: efficientnet-b0 3070 RTX 3070 Ti i 4090 rep g 4090 4080 xxx nv 4090 f 4080 zzz 4080 rep 4080 c 3090 rep 3090 b 3 6 9 12 15 SE +/- 0.22, N = 15 9.01 4.78 4.68 4.34 4.14 4.09 4.06 4.04 4.04 4.03 4.02 4.02 3.89 3.85 3.83 3.82 MIN: 3.98 / MAX: 188.57 MIN: 3.82 / MAX: 411.19 MIN: 4.48 / MAX: 6.02 MIN: 4.16 / MAX: 5.28 MIN: 4.09 / MAX: 5.13 MIN: 3.86 / MAX: 4.83 MIN: 3.83 / MAX: 5.55 MIN: 3.78 / MAX: 4.9 MIN: 3.99 / MAX: 4.82 MIN: 3.82 / MAX: 5.43 MIN: 3.82 / MAX: 5.39 MIN: 3.82 / MAX: 5.66 MIN: 3.83 / MAX: 9.72 MIN: 3.81 / MAX: 4.53 MIN: 3.78 / MAX: 4.41 MIN: 3.79 / MAX: 4.34 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread
NCNN Target: CPU-v3-v3-v3 - Model: blazeface OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: CPU-v3-v3-v3 - Model: blazeface 3070 RTX 3070 Ti 4080 f 4080 xxx 4080 rep 4080 zzz c 3090 rep g b 3090 4090 rep i 4090 nv 4090 0.6818 1.3636 2.0454 2.7272 3.409 SE +/- 0.19, N = 15 3.03 1.79 1.44 1.43 1.42 1.42 1.41 1.38 1.37 1.37 1.37 1.36 1.34 1.28 1.17 1.16 MIN: 1.28 / MAX: 96.94 MIN: 1.13 / MAX: 312.12 MIN: 1.37 / MAX: 3.45 MIN: 1.4 / MAX: 1.77 MIN: 1.36 / MAX: 1.92 MIN: 1.36 / MAX: 2.2 MIN: 1.34 / MAX: 1.91 MIN: 1.36 / MAX: 1.58 MIN: 1.35 / MAX: 1.46 MIN: 1.34 / MAX: 2.07 MIN: 1.35 / MAX: 1.52 MIN: 1.34 / MAX: 1.46 MIN: 1.27 / MAX: 1.95 MIN: 1.23 / MAX: 1.73 MIN: 1.11 / MAX: 1.9 MIN: 1.11 / MAX: 1.67 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread
NCNN Target: CPU-v3-v3-v3 - Model: googlenet OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: CPU-v3-v3-v3 - Model: googlenet 3070 i 4090 RTX 3070 Ti 4090 rep g nv 4090 4080 zzz 4080 rep 4080 xxx 4080 f c 3090 rep b 3090 4 8 12 16 20 SE +/- 0.24, N = 15 17.00 10.17 9.97 9.65 9.29 9.15 9.02 8.55 8.52 8.50 8.45 8.07 7.88 7.85 7.84 7.83 MIN: 7.35 / MAX: 277.79 MIN: 7.94 / MAX: 150.01 MIN: 7.67 / MAX: 258.52 MIN: 7.59 / MAX: 472.81 MIN: 7.98 / MAX: 83.03 MIN: 7.84 / MAX: 198.46 MIN: 8.41 / MAX: 11.08 MIN: 7.85 / MAX: 10.35 MIN: 7.81 / MAX: 10.78 MIN: 7.79 / MAX: 9.94 MIN: 7.79 / MAX: 10.32 MIN: 7.92 / MAX: 8.86 MIN: 7.79 / MAX: 8.78 MIN: 7.75 / MAX: 8.69 MIN: 7.74 / MAX: 8.7 MIN: 7.71 / MAX: 8.8 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread
NCNN Target: CPU-v3-v3-v3 - Model: vgg16 OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: CPU-v3-v3-v3 - Model: vgg16 3070 i 4090 RTX 3070 Ti 4090 rep nv 4090 4080 zzz 4080 4080 xxx g 4080 rep f c 3090 rep 3090 b 11 22 33 44 55 SE +/- 0.24, N = 15 49.75 29.12 28.55 28.40 27.25 27.04 25.45 25.10 25.00 24.92 24.91 24.45 23.99 23.54 23.50 23.50 MIN: 25.45 / MAX: 273.86 MIN: 26.33 / MAX: 310.23 MIN: 24.05 / MAX: 201.8 MIN: 24.12 / MAX: 509.06 MIN: 24.14 / MAX: 379.93 MIN: 24.33 / MAX: 215.56 MIN: 24.22 / MAX: 27.73 MIN: 24.12 / MAX: 27.57 MIN: 23.91 / MAX: 27.99 MIN: 24.58 / MAX: 31.89 MIN: 23.8 / MAX: 26.87 MIN: 24.26 / MAX: 25.26 MIN: 23.72 / MAX: 24.98 MIN: 23.33 / MAX: 24.41 MIN: 23.17 / MAX: 24.44 MIN: 23.3 / MAX: 24.41 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread
NCNN Target: CPU-v3-v3-v3 - Model: resnet18 OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: CPU-v3-v3-v3 - Model: resnet18 3070 4090 rep nv 4090 RTX 3070 Ti f i 4090 4080 zzz 4080 4080 xxx 4080 rep g c b 3090 rep 3090 3 6 9 12 15 SE +/- 0.19, N = 15 11.14 7.75 7.61 6.18 6.13 5.86 5.81 5.71 5.70 5.66 5.63 5.48 5.26 5.21 5.20 5.19 MIN: 4.79 / MAX: 65.12 MIN: 5.57 / MAX: 125.43 MIN: 5.23 / MAX: 90.18 MIN: 5.17 / MAX: 262.79 MIN: 5.41 / MAX: 151.51 MIN: 5.35 / MAX: 7.79 MIN: 5.27 / MAX: 7.16 MIN: 5.12 / MAX: 8.19 MIN: 5.15 / MAX: 7.9 MIN: 5.14 / MAX: 7.49 MIN: 5.09 / MAX: 7.75 MIN: 5.37 / MAX: 6.51 MIN: 5.18 / MAX: 6.27 MIN: 5.12 / MAX: 6.22 MIN: 5.1 / MAX: 5.97 MIN: 5.09 / MAX: 6.13 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread
NCNN Target: CPU-v3-v3-v3 - Model: alexnet OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: CPU-v3-v3-v3 - Model: alexnet 3070 RTX 3070 Ti 4090 rep i 4090 f g nv 4090 4080 zzz 4080 4080 xxx 4080 rep 3090 3090 rep b c 3 6 9 12 15 SE +/- 0.23, N = 15 11.00 5.55 5.27 5.01 4.94 4.83 4.71 4.67 4.67 4.66 4.65 4.65 4.31 4.30 4.29 4.28 MIN: 4.33 / MAX: 199.92 MIN: 4.2 / MAX: 281.58 MIN: 4.78 / MAX: 7.7 MIN: 4.6 / MAX: 6.68 MIN: 4.51 / MAX: 6.64 MIN: 4.76 / MAX: 5.74 MIN: 4.65 / MAX: 5.57 MIN: 4.28 / MAX: 5.7 MIN: 4.28 / MAX: 6.29 MIN: 4.29 / MAX: 6.1 MIN: 4.28 / MAX: 6.42 MIN: 4.26 / MAX: 6.13 MIN: 4.25 / MAX: 5.13 MIN: 4.24 / MAX: 4.99 MIN: 4.24 / MAX: 5.64 MIN: 4.24 / MAX: 5.12 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread
NCNN Target: CPU-v3-v3-v3 - Model: resnet50 OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: CPU-v3-v3-v3 - Model: resnet50 3070 i 4090 rep nv 4090 RTX 3070 Ti 4090 g 4080 zzz 4080 f 4080 xxx 4080 rep c 3090 rep 3090 b 6 12 18 24 30 SE +/- 0.22, N = 15 24.07 14.05 13.82 13.68 12.11 11.39 11.25 11.21 11.11 11.05 10.91 10.79 10.33 10.07 10.05 10.01 MIN: 10.02 / MAX: 218.35 MIN: 11.69 / MAX: 252.21 MIN: 10.34 / MAX: 245.6 MIN: 10.25 / MAX: 566.67 MIN: 10.16 / MAX: 382.56 MIN: 10.48 / MAX: 13.29 MIN: 10.55 / MAX: 118.12 MIN: 10.3 / MAX: 13.25 MIN: 10.19 / MAX: 13.03 MIN: 10.46 / MAX: 112.6 MIN: 9.91 / MAX: 13.1 MIN: 9.91 / MAX: 12.75 MIN: 10.16 / MAX: 13.97 MIN: 9.94 / MAX: 11.06 MIN: 9.85 / MAX: 12.64 MIN: 9.89 / MAX: 10.86 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread
NCNN Target: CPU-v3-v3-v3 - Model: yolov4-tiny OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: CPU-v3-v3-v3 - Model: yolov4-tiny 3070 nv 4090 RTX 3070 Ti 4090 rep 4090 i 4080 zzz 4080 4080 xxx 4080 rep g f b c 3090 3090 rep 7 14 21 28 35 SE +/- 0.14, N = 15 29.34 15.62 15.42 15.34 15.30 15.11 13.83 13.81 13.69 13.55 13.08 13.07 12.98 12.89 12.87 12.82 MIN: 12.17 / MAX: 245.34 MIN: 12.99 / MAX: 184 MIN: 12.21 / MAX: 414.81 MIN: 12.94 / MAX: 157.95 MIN: 12.87 / MAX: 144.73 MIN: 12.93 / MAX: 151.45 MIN: 12.89 / MAX: 15.4 MIN: 12.84 / MAX: 15.1 MIN: 12.73 / MAX: 15.68 MIN: 12.75 / MAX: 14.74 MIN: 12.96 / MAX: 13.83 MIN: 12.95 / MAX: 14.55 MIN: 12.73 / MAX: 35.55 MIN: 12.84 / MAX: 13.19 MIN: 12.75 / MAX: 13.58 MIN: 12.72 / MAX: 13.48 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread
NCNN Target: CPU-v3-v3-v3 - Model: squeezenet_ssd OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: CPU-v3-v3-v3 - Model: squeezenet_ssd 3070 nv 4090 4090 4090 rep RTX 3070 Ti i 4080 xxx 4080 4080 rep 4080 zzz g 3090 rep 3090 c b f 4 8 12 16 20 SE +/- 0.24, N = 15 17.75 9.37 9.32 9.30 8.39 8.16 7.64 7.64 7.59 7.35 7.26 7.09 7.04 7.04 7.03 6.97 MIN: 6.47 / MAX: 272.11 MIN: 7.07 / MAX: 281.92 MIN: 7.1 / MAX: 172.56 MIN: 6.92 / MAX: 310.91 MIN: 6.53 / MAX: 436.05 MIN: 7.51 / MAX: 9.94 MIN: 7.03 / MAX: 9.19 MIN: 7.05 / MAX: 9.9 MIN: 7.02 / MAX: 8.87 MIN: 6.79 / MAX: 9.82 MIN: 7.14 / MAX: 8.59 MIN: 7.02 / MAX: 7.99 MIN: 6.96 / MAX: 7.74 MIN: 6.96 / MAX: 7.83 MIN: 6.97 / MAX: 7.88 MIN: 6.83 / MAX: 13.87 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread
NCNN Target: CPU-v3-v3-v3 - Model: regnety_400m OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: CPU-v3-v3-v3 - Model: regnety_400m 3070 4090 rep nv 4090 RTX 3070 Ti 4080 xxx 4080 rep 4080 zzz f 4080 i c 4090 3090 rep g b 3090 5 10 15 20 25 SE +/- 0.21, N = 15 19.66 17.15 9.55 9.02 8.75 8.57 8.47 8.34 8.33 8.21 8.14 8.13 8.09 8.07 8.05 7.95 MIN: 7.5 / MAX: 235.36 MIN: 8.02 / MAX: 773.45 MIN: 7.5 / MAX: 193.79 MIN: 7.69 / MAX: 501.76 MIN: 8.35 / MAX: 10.08 MIN: 8.21 / MAX: 10.39 MIN: 8.13 / MAX: 10.27 MIN: 8.26 / MAX: 9.3 MIN: 8.02 / MAX: 9.64 MIN: 7.9 / MAX: 9.99 MIN: 8.08 / MAX: 8.69 MIN: 7.75 / MAX: 10.05 MIN: 7.99 / MAX: 14.25 MIN: 7.97 / MAX: 8.81 MIN: 8 / MAX: 8.58 MIN: 7.88 / MAX: 8.67 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread
NCNN Target: CPU-v3-v3-v3 - Model: vision_transformer OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: CPU-v3-v3-v3 - Model: vision_transformer 3070 4090 rep nv 4090 4090 RTX 3070 Ti i 4080 zzz 4080 xxx 4080 rep 4080 f g 3090 rep 3090 c b 20 40 60 80 100 SE +/- 0.18, N = 15 81.77 39.12 38.99 38.79 37.86 36.55 34.47 34.37 34.27 34.20 33.47 33.39 31.93 31.89 31.78 31.65 MIN: 44.4 / MAX: 460.28 MIN: 33.92 / MAX: 465.83 MIN: 34.17 / MAX: 473.06 MIN: 33.95 / MAX: 457.41 MIN: 32.9 / MAX: 463.9 MIN: 33 / MAX: 209.38 MIN: 33.32 / MAX: 37.42 MIN: 33.01 / MAX: 38.7 MIN: 33.07 / MAX: 37.01 MIN: 32.92 / MAX: 36.19 MIN: 32.89 / MAX: 74.09 MIN: 32.73 / MAX: 88.83 MIN: 31.76 / MAX: 33.09 MIN: 31.66 / MAX: 39.97 MIN: 31.64 / MAX: 34.51 MIN: 31.53 / MAX: 32.23 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread
NCNN Target: CPU-v3-v3-v3 - Model: FastestDet OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: CPU-v3-v3-v3 - Model: FastestDet 3070 i RTX 3070 Ti 4080 4080 xxx 4080 rep 4090 rep 3090 rep c b nv 4090 3090 4080 zzz g f 4090 3 6 9 12 15 SE +/- 0.27, N = 14 9.18 5.69 4.33 4.20 4.19 4.18 4.16 4.11 4.08 4.07 4.06 4.04 4.04 3.97 3.85 2.93 MIN: 3.64 / MAX: 122.65 MIN: 3.69 / MAX: 261.71 MIN: 2.59 / MAX: 433.58 MIN: 4.06 / MAX: 4.86 MIN: 4.04 / MAX: 5.47 MIN: 4.03 / MAX: 5.07 MIN: 4 / MAX: 5.58 MIN: 4.07 / MAX: 4.21 MIN: 4.05 / MAX: 4.36 MIN: 4.03 / MAX: 5.83 MIN: 3.91 / MAX: 5.78 MIN: 4.01 / MAX: 4.15 MIN: 3.89 / MAX: 5.01 MIN: 3.92 / MAX: 4.75 MIN: 3.8 / MAX: 4.65 MIN: 2.84 / MAX: 3.38 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread
NCNN Target: Vulkan GPU-v3-v3-v3 - Model: mobilenet OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: Vulkan GPU-v3-v3-v3 - Model: mobilenet 3070 nv 4090 RTX 3070 Ti 4090 rep i 4080 xxx 4090 f 4080 rep 4080 4080 zzz g 3090 rep 3090 b c 4 8 12 16 20 SE +/- 0.26, N = 15 16.34 10.54 9.62 9.54 9.05 8.88 8.81 8.65 8.46 8.43 8.38 8.20 8.01 8.00 8.00 7.95 MIN: 8.13 / MAX: 80.69 MIN: 8.41 / MAX: 134.08 MIN: 7.76 / MAX: 502.83 MIN: 8.94 / MAX: 10.54 MIN: 8.48 / MAX: 11.28 MIN: 8.31 / MAX: 10.01 MIN: 8.32 / MAX: 10.7 MIN: 8.55 / MAX: 9.53 MIN: 7.99 / MAX: 10.62 MIN: 7.99 / MAX: 10.66 MIN: 7.95 / MAX: 10.41 MIN: 8.12 / MAX: 9.4 MIN: 7.95 / MAX: 8.35 MIN: 7.94 / MAX: 8.78 MIN: 7.95 / MAX: 8.99 MIN: 7.89 / MAX: 8.79 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread
NCNN Target: Vulkan GPU-v3-v3-v3-v2-v2 - Model: mobilenet-v2 OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: Vulkan GPU-v3-v3-v3-v2-v2 - Model: mobilenet-v2 3070 4090 nv 4090 RTX 3070 Ti 4080 xxx 4090 rep 4080 rep 4080 i 4080 zzz 3090 rep g 3090 f c b 2 4 6 8 10 SE +/- 0.18, N = 15 7.24 4.99 4.45 3.66 3.40 3.31 3.30 3.29 3.28 3.23 3.17 3.17 3.16 3.16 3.15 3.15 MIN: 3.04 / MAX: 261.68 MIN: 3.1 / MAX: 201.8 MIN: 2.65 / MAX: 216.76 MIN: 3.01 / MAX: 437.59 MIN: 3.23 / MAX: 4.8 MIN: 3.12 / MAX: 4.6 MIN: 3.12 / MAX: 4.7 MIN: 3.12 / MAX: 4.64 MIN: 3.09 / MAX: 5.28 MIN: 3.06 / MAX: 4.66 MIN: 3.12 / MAX: 4.03 MIN: 3.13 / MAX: 3.58 MIN: 3.11 / MAX: 3.51 MIN: 3.1 / MAX: 3.71 MIN: 3.11 / MAX: 3.85 MIN: 3.11 / MAX: 3.88 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread
NCNN Target: Vulkan GPU-v3-v3-v3-v3-v3 - Model: mobilenet-v3 OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: Vulkan GPU-v3-v3-v3-v3-v3 - Model: mobilenet-v3 3070 4090 rep RTX 3070 Ti 4080 xxx 4080 rep 4080 i 4080 zzz c b 3090 rep g f 4090 nv 4090 2 4 6 8 10 SE +/- 0.18, N = 15 8.06 4.90 3.65 3.33 3.27 3.27 3.26 3.20 3.17 3.16 3.15 3.15 3.15 3.12 2.61 MIN: 2.96 / MAX: 219.87 MIN: 3.17 / MAX: 120.84 MIN: 2.87 / MAX: 347.75 MIN: 3.19 / MAX: 4.2 MIN: 3.14 / MAX: 3.99 MIN: 3.12 / MAX: 5.24 MIN: 3.12 / MAX: 4.19 MIN: 3.06 / MAX: 3.84 MIN: 3.11 / MAX: 8.89 MIN: 3.11 / MAX: 3.75 MIN: 3.11 / MAX: 3.83 MIN: 3.1 / MAX: 3.87 MIN: 3.1 / MAX: 3.8 MIN: 2.99 / MAX: 5.09 MIN: 2.5 / MAX: 3.12 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread
NCNN Target: Vulkan GPU-v3-v3-v3 - Model: shufflenet-v2 OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: Vulkan GPU-v3-v3-v3 - Model: shufflenet-v2 3070 RTX 3070 Ti 4080 xxx 4080 rep 4080 4090 rep 4080 zzz 3090 rep i g 4090 3090 f c b nv 4090 1.1003 2.2006 3.3009 4.4012 5.5015 SE +/- 0.22, N = 15 4.89 3.95 3.51 3.44 3.44 3.40 3.37 3.36 3.36 3.35 3.34 3.34 3.33 3.33 3.33 3.17 MIN: 3.04 / MAX: 18.32 MIN: 3.19 / MAX: 410.41 MIN: 3.37 / MAX: 4.26 MIN: 3.32 / MAX: 4.16 MIN: 3.31 / MAX: 4.88 MIN: 3.26 / MAX: 4.84 MIN: 3.25 / MAX: 3.95 MIN: 3.32 / MAX: 3.7 MIN: 3.25 / MAX: 4.02 MIN: 3.31 / MAX: 4.01 MIN: 3.23 / MAX: 4.78 MIN: 3.31 / MAX: 3.6 MIN: 3.29 / MAX: 3.99 MIN: 3.31 / MAX: 3.81 MIN: 3.3 / MAX: 3.77 MIN: 3.04 / MAX: 3.78 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread
NCNN Target: Vulkan GPU-v3-v3-v3 - Model: mnasnet OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: Vulkan GPU-v3-v3-v3 - Model: mnasnet 3070 RTX 3070 Ti 4080 xxx 4090 rep 4080 4080 rep 4080 zzz 4090 i g 3090 rep 3090 f c b nv 4090 2 4 6 8 10 SE +/- 0.16, N = 14 6.02 3.37 3.13 3.10 3.09 3.07 3.01 3.00 2.99 2.98 2.97 2.97 2.96 2.96 2.96 2.54 MIN: 2.79 / MAX: 50.49 MIN: 2.86 / MAX: 278.87 MIN: 3 / MAX: 5.1 MIN: 2.97 / MAX: 3.72 MIN: 2.94 / MAX: 3.79 MIN: 2.94 / MAX: 3.72 MIN: 2.91 / MAX: 3.6 MIN: 2.89 / MAX: 3.46 MIN: 2.86 / MAX: 4.38 MIN: 2.95 / MAX: 3.63 MIN: 2.94 / MAX: 3.39 MIN: 2.92 / MAX: 3.28 MIN: 2.92 / MAX: 3.81 MIN: 2.93 / MAX: 3.41 MIN: 2.93 / MAX: 3.4 MIN: 2.44 / MAX: 3.58 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread
NCNN Target: Vulkan GPU-v3-v3-v3 - Model: efficientnet-b0 OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: Vulkan GPU-v3-v3-v3 - Model: efficientnet-b0 3070 4090 rep nv 4090 RTX 3070 Ti g 4080 xxx i 4090 4080 4080 rep 4080 zzz 3090 rep 3090 f b c 2 4 6 8 10 SE +/- 0.18, N = 15 7.81 6.28 5.26 4.73 4.63 4.22 4.21 4.18 4.05 4.04 3.95 3.85 3.85 3.85 3.85 3.82 MIN: 3.73 / MAX: 159.47 MIN: 3.91 / MAX: 337.73 MIN: 3.48 / MAX: 250.88 MIN: 3.79 / MAX: 418.72 MIN: 3.8 / MAX: 159.43 MIN: 4 / MAX: 5.58 MIN: 3.96 / MAX: 4.94 MIN: 4 / MAX: 5.25 MIN: 3.83 / MAX: 5 MIN: 3.81 / MAX: 5.08 MIN: 3.76 / MAX: 4.84 MIN: 3.81 / MAX: 4.62 MIN: 3.8 / MAX: 4.43 MIN: 3.8 / MAX: 4.6 MIN: 3.82 / MAX: 4.48 MIN: 3.78 / MAX: 4.53 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread
NCNN Target: Vulkan GPU-v3-v3-v3 - Model: blazeface OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: Vulkan GPU-v3-v3-v3 - Model: blazeface 3070 RTX 3070 Ti 4080 rep 4080 xxx 4080 4090 rep 4080 zzz 3090 rep g f b 3090 c 4090 i nv 4090 0.7155 1.431 2.1465 2.862 3.5775 SE +/- 0.18, N = 15 3.18 1.71 1.45 1.42 1.42 1.41 1.39 1.38 1.38 1.37 1.37 1.36 1.36 1.33 1.25 1.07 MIN: 1.31 / MAX: 185.03 MIN: 1.09 / MAX: 448.17 MIN: 1.36 / MAX: 8.73 MIN: 1.36 / MAX: 1.92 MIN: 1.35 / MAX: 2.15 MIN: 1.35 / MAX: 1.89 MIN: 1.34 / MAX: 1.89 MIN: 1.36 / MAX: 1.9 MIN: 1.36 / MAX: 1.76 MIN: 1.35 / MAX: 1.62 MIN: 1.35 / MAX: 1.39 MIN: 1.34 / MAX: 1.61 MIN: 1.34 / MAX: 1.44 MIN: 1.27 / MAX: 1.98 MIN: 1.19 / MAX: 2.61 MIN: 1.02 / MAX: 1.52 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread
NCNN Target: Vulkan GPU-v3-v3-v3 - Model: googlenet OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: Vulkan GPU-v3-v3-v3 - Model: googlenet 3070 i nv 4090 RTX 3070 Ti 4080 xxx 4090 rep 4090 4080 rep 4080 4080 zzz g b f 3090 rep c 3090 5 10 15 20 25 SE +/- 0.21, N = 15 20.72 10.19 10.01 9.86 8.99 8.90 8.87 8.58 8.49 8.37 8.35 7.97 7.94 7.85 7.83 7.82 MIN: 7.49 / MAX: 355.33 MIN: 7.73 / MAX: 212.36 MIN: 7.29 / MAX: 259.11 MIN: 7.54 / MAX: 396.21 MIN: 8.25 / MAX: 10.27 MIN: 8.22 / MAX: 11.07 MIN: 8.18 / MAX: 11.09 MIN: 7.79 / MAX: 10.48 MIN: 7.82 / MAX: 11.98 MIN: 7.76 / MAX: 10.31 MIN: 8.2 / MAX: 9.39 MIN: 7.89 / MAX: 8.7 MIN: 7.8 / MAX: 8.78 MIN: 7.75 / MAX: 8.64 MIN: 7.74 / MAX: 8.61 MIN: 7.69 / MAX: 8.6 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread
NCNN Target: Vulkan GPU-v3-v3-v3 - Model: vgg16 OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: Vulkan GPU-v3-v3-v3 - Model: vgg16 3070 i RTX 3070 Ti 4090 nv 4090 4090 rep 4080 xxx 4080 zzz 4080 rep 4080 g f c 3090 3090 rep b 12 24 36 48 60 SE +/- 0.27, N = 15 55.42 29.07 28.40 28.21 27.77 27.59 26.08 25.26 25.04 25.04 24.71 24.12 23.54 23.50 23.43 23.42 MIN: 25.32 / MAX: 281.46 MIN: 24.45 / MAX: 263.33 MIN: 23.98 / MAX: 456 MIN: 24.57 / MAX: 270.76 MIN: 24.82 / MAX: 264.66 MIN: 24.34 / MAX: 396.09 MIN: 24.52 / MAX: 27.73 MIN: 24.14 / MAX: 27.73 MIN: 23.81 / MAX: 27.15 MIN: 23.87 / MAX: 28.04 MIN: 23.88 / MAX: 119.23 MIN: 23.57 / MAX: 46.44 MIN: 23.32 / MAX: 24.54 MIN: 23.23 / MAX: 24.26 MIN: 23.26 / MAX: 24.3 MIN: 23.27 / MAX: 24.32 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread
NCNN Target: Vulkan GPU-v3-v3-v3 - Model: resnet18 OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: Vulkan GPU-v3-v3-v3 - Model: resnet18 3070 RTX 3070 Ti 4090 4080 xxx i nv 4090 4090 rep 4080 rep 4080 4080 zzz g b f 3090 rep 3090 c 3 6 9 12 15 SE +/- 0.16, N = 15 13.38 6.23 5.97 5.89 5.85 5.84 5.81 5.69 5.65 5.59 5.50 5.42 5.30 5.24 5.23 5.23 MIN: 5.43 / MAX: 208.42 MIN: 4.99 / MAX: 309.18 MIN: 5.46 / MAX: 7.02 MIN: 5.36 / MAX: 7.53 MIN: 5.3 / MAX: 8.27 MIN: 5.35 / MAX: 7.72 MIN: 5.3 / MAX: 6.82 MIN: 5.11 / MAX: 6.94 MIN: 5.14 / MAX: 6.93 MIN: 5.09 / MAX: 7.7 MIN: 5.4 / MAX: 6.38 MIN: 5.36 / MAX: 6.27 MIN: 5.17 / MAX: 5.93 MIN: 5.14 / MAX: 5.99 MIN: 5.1 / MAX: 6.07 MIN: 5.11 / MAX: 6.03 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread
NCNN Target: Vulkan GPU-v3-v3-v3 - Model: alexnet OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: Vulkan GPU-v3-v3-v3 - Model: alexnet 3070 4090 rep nv 4090 4090 RTX 3070 Ti 4080 xxx i g 4080 rep 4080 4080 zzz b f 3090 rep 3090 c 3 6 9 12 15 SE +/- 0.23, N = 15 9.86 6.58 6.11 6.11 5.67 5.21 4.99 4.86 4.69 4.69 4.65 4.42 4.35 4.30 4.30 4.30 MIN: 4.25 / MAX: 157.02 MIN: 4.61 / MAX: 91.07 MIN: 4.83 / MAX: 124.76 MIN: 4.73 / MAX: 81.72 MIN: 4.21 / MAX: 365.75 MIN: 4.79 / MAX: 6.66 MIN: 4.59 / MAX: 6.56 MIN: 4.8 / MAX: 6.37 MIN: 4.26 / MAX: 7.17 MIN: 4.26 / MAX: 6.15 MIN: 4.26 / MAX: 5.97 MIN: 4.32 / MAX: 5.1 MIN: 4.27 / MAX: 5.16 MIN: 4.25 / MAX: 4.7 MIN: 4.25 / MAX: 5.08 MIN: 4.26 / MAX: 5.16 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread
NCNN Target: Vulkan GPU-v3-v3-v3 - Model: resnet50 OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: Vulkan GPU-v3-v3-v3 - Model: resnet50 3070 4090 4090 rep nv 4090 RTX 3070 Ti 4080 xxx i 4080 zzz 4080 4080 rep g f c 3090 rep 3090 b 6 12 18 24 30 SE +/- 0.27, N = 15 23.11 14.58 13.57 13.13 12.35 11.50 11.15 11.09 10.95 10.84 10.43 10.25 10.03 10.01 9.97 9.87 MIN: 10.22 / MAX: 140.41 MIN: 10.67 / MAX: 324.82 MIN: 10.45 / MAX: 199.55 MIN: 10.56 / MAX: 323.44 MIN: 9.83 / MAX: 424.28 MIN: 10.5 / MAX: 13.47 MIN: 10.31 / MAX: 12.97 MIN: 10.18 / MAX: 13.12 MIN: 9.91 / MAX: 17.11 MIN: 9.93 / MAX: 12.83 MIN: 10.19 / MAX: 11.32 MIN: 10.05 / MAX: 11.08 MIN: 9.93 / MAX: 10.96 MIN: 9.91 / MAX: 10.74 MIN: 9.86 / MAX: 10.84 MIN: 9.79 / MAX: 10.73 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread
NCNN Target: Vulkan GPU-v3-v3-v3 - Model: yolov4-tiny OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: Vulkan GPU-v3-v3-v3 - Model: yolov4-tiny 3070 nv 4090 4090 rep 4090 RTX 3070 Ti i f 4080 xxx 4080 4080 rep 4080 zzz g 3090 c 3090 rep b 7 14 21 28 35 SE +/- 0.23, N = 15 29.49 16.61 16.39 15.44 15.44 15.43 14.34 13.95 13.79 13.68 13.62 13.35 12.88 12.86 12.84 12.77 MIN: 13.03 / MAX: 182.99 MIN: 12.32 / MAX: 375.99 MIN: 12.97 / MAX: 369.64 MIN: 12.92 / MAX: 211.43 MIN: 12.61 / MAX: 387.62 MIN: 13.1 / MAX: 210.2 MIN: 14.23 / MAX: 15.12 MIN: 13.03 / MAX: 15.9 MIN: 12.79 / MAX: 15.92 MIN: 12.77 / MAX: 15.57 MIN: 12.75 / MAX: 15.79 MIN: 12.87 / MAX: 58.52 MIN: 12.75 / MAX: 13.79 MIN: 12.76 / MAX: 13.98 MIN: 12.76 / MAX: 13.7 MIN: 12.69 / MAX: 13.71 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread
NCNN Target: Vulkan GPU-v3-v3-v3 - Model: squeezenet_ssd OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: Vulkan GPU-v3-v3-v3 - Model: squeezenet_ssd 3070 i RTX 3070 Ti 4090 4090 rep nv 4090 4080 xxx 4080 4080 rep 4080 zzz g b f 3090 rep c 3090 4 8 12 16 20 SE +/- 0.19, N = 15 16.15 8.33 8.29 7.93 7.81 7.72 7.70 7.66 7.63 7.51 7.31 7.14 7.09 7.08 7.07 7.04 MIN: 7.25 / MAX: 210.69 MIN: 6.32 / MAX: 222.03 MIN: 6.37 / MAX: 448.22 MIN: 7.31 / MAX: 9.45 MIN: 7.24 / MAX: 9.04 MIN: 7.12 / MAX: 23.25 MIN: 7.11 / MAX: 9.19 MIN: 7.02 / MAX: 9.08 MIN: 7.02 / MAX: 9.71 MIN: 6.94 / MAX: 9.51 MIN: 6.96 / MAX: 30.1 MIN: 7.06 / MAX: 7.95 MIN: 6.98 / MAX: 8.01 MIN: 7.01 / MAX: 7.93 MIN: 7.01 / MAX: 7.75 MIN: 6.97 / MAX: 7.76 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread
NCNN Target: Vulkan GPU-v3-v3-v3 - Model: regnety_400m OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: Vulkan GPU-v3-v3-v3 - Model: regnety_400m 3070 4090 rep 4090 RTX 3070 Ti 4080 xxx f 4080 4080 rep b 3090 rep 4080 zzz 3090 i g c nv 4090 4 8 12 16 20 SE +/- 0.25, N = 14 17.23 10.34 9.60 8.89 8.58 8.50 8.45 8.44 8.27 8.25 8.10 8.01 7.99 7.99 7.98 7.73 MIN: 7.8 / MAX: 193.14 MIN: 8.21 / MAX: 214.16 MIN: 7.66 / MAX: 210.23 MIN: 7.74 / MAX: 476.28 MIN: 8.23 / MAX: 10.39 MIN: 8.04 / MAX: 30.12 MIN: 8.05 / MAX: 10.3 MIN: 8.04 / MAX: 10.17 MIN: 8.22 / MAX: 9.01 MIN: 8.12 / MAX: 14 MIN: 7.77 / MAX: 15.42 MIN: 7.93 / MAX: 8.35 MIN: 7.62 / MAX: 9.27 MIN: 7.91 / MAX: 8.8 MIN: 7.93 / MAX: 8.65 MIN: 7.43 / MAX: 9.41 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread
NCNN Target: Vulkan GPU-v3-v3-v3 - Model: vision_transformer OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: Vulkan GPU-v3-v3-v3 - Model: vision_transformer 3070 4090 4090 rep nv 4090 i RTX 3070 Ti 4080 xxx 4080 rep 4080 4080 zzz f g 3090 rep 3090 b c 16 32 48 64 80 SE +/- 0.13, N = 15 73.51 39.01 38.73 38.58 38.33 38.29 35.40 34.29 34.13 34.05 33.36 32.68 32.11 31.86 31.71 31.66 MIN: 39.27 / MAX: 288.2 MIN: 33.91 / MAX: 411.66 MIN: 33.81 / MAX: 362.17 MIN: 33.77 / MAX: 476.18 MIN: 34.14 / MAX: 246.43 MIN: 32.31 / MAX: 557.38 MIN: 33.93 / MAX: 39.3 MIN: 33.11 / MAX: 40.12 MIN: 32.98 / MAX: 36.11 MIN: 32.83 / MAX: 38.57 MIN: 32.83 / MAX: 76.21 MIN: 32.02 / MAX: 87.72 MIN: 31.94 / MAX: 33.01 MIN: 31.58 / MAX: 35.84 MIN: 31.56 / MAX: 33.03 MIN: 31.52 / MAX: 32.14 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread
NCNN Target: Vulkan GPU-v3-v3-v3 - Model: FastestDet OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: Vulkan GPU-v3-v3-v3 - Model: FastestDet 3070 nv 4090 i 4080 xxx RTX 3070 Ti 4080 f 4090 rep 4080 zzz 3090 rep 4080 rep g b 3090 4090 c 2 4 6 8 10 SE +/- 0.15, N = 15 8.63 5.92 4.43 4.31 4.26 4.20 4.20 4.16 4.12 4.10 4.09 4.06 4.06 4.04 3.94 3.69 MIN: 4.27 / MAX: 144.3 MIN: 4.25 / MAX: 103.26 MIN: 4.28 / MAX: 5.01 MIN: 4.14 / MAX: 6.11 MIN: 2.71 / MAX: 347.03 MIN: 4.04 / MAX: 5.82 MIN: 4.15 / MAX: 4.92 MIN: 4.03 / MAX: 4.73 MIN: 3.97 / MAX: 6.99 MIN: 4.06 / MAX: 4.21 MIN: 3.92 / MAX: 5.5 MIN: 4.01 / MAX: 4.78 MIN: 4.03 / MAX: 4.3 MIN: 4 / MAX: 4.15 MIN: 3.8 / MAX: 5.41 MIN: 3.66 / MAX: 3.92 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread
VkResample Upscale: 2x - Precision: Double OpenBenchmarking.org ms, Fewer Is Better VkResample 1.0 Upscale: 2x - Precision: Double e d g f i 3090 3090 rep 4080 4080 rep 4080 xxx 4080 zzz 4090 rep nv 4090 4090 RTX 3070 Ti 3070 110 220 330 440 550 SE +/- 0.00, N = 3 SE +/- 0.00, N = 3 SE +/- 0.01, N = 3 500.02 500.01 500.01 500.01 500.01 371.70 371.42 288.20 288.17 288.04 288.03 173.04 172.89 172.88 24.81 24.75 1. (CXX) g++ options: -O3
NCNN Target: Vulkan GPU-v3-v3-v3-v3-v3-v3 - Model: mobilenet OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: Vulkan GPU-v3-v3-v3-v3-v3-v3 - Model: mobilenet 3070 4090 i RTX 3070 Ti nv 4090 4080 g 4080 zzz 4080 xxx 4080 rep 4090 rep 3090 3090 rep 5 10 15 20 25 SE +/- 0.27, N = 15 18.39 10.18 10.02 9.62 9.41 8.84 8.50 8.46 8.46 8.40 8.37 8.11 8.04 MIN: 7.92 / MAX: 173.39 MIN: 8.18 / MAX: 235.56 MIN: 8.07 / MAX: 266.25 MIN: 7.71 / MAX: 449.11 MIN: 8.98 / MAX: 11.38 MIN: 8.31 / MAX: 10.98 MIN: 8.42 / MAX: 9.29 MIN: 7.97 / MAX: 10.56 MIN: 7.95 / MAX: 10.34 MIN: 7.93 / MAX: 15.25 MIN: 7.98 / MAX: 10.71 MIN: 8.02 / MAX: 14.2 MIN: 7.96 / MAX: 9.01 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread
NCNN Target: Vulkan GPU-v3-v3-v3-v3-v3-v3-v2-v2 - Model: mobilenet-v2 OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: Vulkan GPU-v3-v3-v3-v3-v3-v3-v2-v2 - Model: mobilenet-v2 3070 nv 4090 RTX 3070 Ti 4090 4090 rep i 4080 zzz 4080 4080 xxx 4080 rep 3090 rep g 3090 2 4 6 8 10 SE +/- 0.15, N = 15 8.35 5.10 3.66 3.41 3.34 3.29 3.28 3.28 3.27 3.27 3.19 3.17 3.15 MIN: 3.08 / MAX: 103.38 MIN: 3.14 / MAX: 138.88 MIN: 3.01 / MAX: 311.25 MIN: 3.24 / MAX: 5.42 MIN: 3.14 / MAX: 4.45 MIN: 3.1 / MAX: 3.96 MIN: 3.09 / MAX: 4.98 MIN: 3.11 / MAX: 4.16 MIN: 3.11 / MAX: 4.73 MIN: 3.08 / MAX: 5.18 MIN: 3.13 / MAX: 4 MIN: 3.1 / MAX: 5.03 MIN: 3.1 / MAX: 3.68 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread
NCNN Target: Vulkan GPU-v3-v3-v3-v3-v3-v3-v3-v3 - Model: mobilenet-v3 OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: Vulkan GPU-v3-v3-v3-v3-v3-v3-v3-v3 - Model: mobilenet-v3 3070 RTX 3070 Ti 4090 4090 rep nv 4090 4080 xxx i 4080 zzz 4080 rep 3090 g 2 4 6 8 10 SE +/- 0.18, N = 15 6.56 3.62 3.34 3.33 3.26 3.26 3.26 3.24 3.24 3.16 3.16 MIN: 3.07 / MAX: 110.87 MIN: 3 / MAX: 469.9 MIN: 3.21 / MAX: 4.31 MIN: 3.19 / MAX: 4.79 MIN: 3.13 / MAX: 3.96 MIN: 3.13 / MAX: 4.08 MIN: 3.11 / MAX: 4.7 MIN: 3.1 / MAX: 3.88 MIN: 3.11 / MAX: 4.37 MIN: 3.11 / MAX: 3.77 MIN: 3.12 / MAX: 3.58 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread
NCNN Target: Vulkan GPU-v3-v3-v3-v3-v3-v3 - Model: shufflenet-v2 OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: Vulkan GPU-v3-v3-v3-v3-v3-v3 - Model: shufflenet-v2 3070 4090 rep 4090 RTX 3070 Ti nv 4090 4080 4080 zzz 4080 xxx i 4080 rep 3090 rep g 3090 2 4 6 8 10 SE +/- 0.16, N = 15 8.00 5.27 5.17 3.75 3.51 3.46 3.43 3.43 3.43 3.39 3.37 3.34 3.33 MIN: 3.16 / MAX: 190.15 MIN: 3.27 / MAX: 191.55 MIN: 3.22 / MAX: 208.13 MIN: 3.2 / MAX: 361.52 MIN: 3.38 / MAX: 4.05 MIN: 3.3 / MAX: 5.74 MIN: 3.29 / MAX: 3.87 MIN: 3.31 / MAX: 3.95 MIN: 3.3 / MAX: 4.89 MIN: 3.26 / MAX: 3.91 MIN: 3.33 / MAX: 3.8 MIN: 3.31 / MAX: 4.05 MIN: 3.29 / MAX: 3.67 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread
NCNN Target: Vulkan GPU-v3-v3-v3-v3-v3-v3 - Model: mnasnet OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: Vulkan GPU-v3-v3-v3-v3-v3-v3 - Model: mnasnet 3070 RTX 3070 Ti 4090 nv 4090 4090 rep 4080 zzz i 4080 4080 xxx 4080 rep 3090 rep g 3090 1.0328 2.0656 3.0984 4.1312 5.164 SE +/- 0.14, N = 15 4.59 3.34 3.17 3.16 3.13 3.08 3.07 3.06 3.05 3.03 2.99 2.97 2.96 MIN: 2.88 / MAX: 20.12 MIN: 2.68 / MAX: 393.6 MIN: 3.03 / MAX: 3.66 MIN: 3.02 / MAX: 4.6 MIN: 3.01 / MAX: 3.62 MIN: 2.93 / MAX: 4.42 MIN: 2.93 / MAX: 3.84 MIN: 2.94 / MAX: 3.67 MIN: 2.91 / MAX: 3.67 MIN: 2.91 / MAX: 4.45 MIN: 2.96 / MAX: 3.32 MIN: 2.93 / MAX: 3.88 MIN: 2.93 / MAX: 3.31 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread
NCNN Target: Vulkan GPU-v3-v3-v3-v3-v3-v3 - Model: efficientnet-b0 OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: Vulkan GPU-v3-v3-v3-v3-v3-v3 - Model: efficientnet-b0 3070 RTX 3070 Ti 4090 i nv 4090 4080 4090 rep 4080 xxx 4080 zzz 4080 rep 3090 rep g 3090 2 4 6 8 10 SE +/- 0.19, N = 15 8.41 4.60 4.36 4.19 4.12 4.06 4.04 4.02 4.01 3.98 3.87 3.84 3.83 MIN: 3.76 / MAX: 67.73 MIN: 3.79 / MAX: 336.2 MIN: 4.14 / MAX: 5.24 MIN: 4.01 / MAX: 5.09 MIN: 3.86 / MAX: 5.39 MIN: 3.85 / MAX: 4.97 MIN: 3.85 / MAX: 4.9 MIN: 3.8 / MAX: 5.14 MIN: 3.79 / MAX: 5.39 MIN: 3.77 / MAX: 5.44 MIN: 3.81 / MAX: 4.62 MIN: 3.78 / MAX: 4.57 MIN: 3.78 / MAX: 4.4 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread
NCNN Target: Vulkan GPU-v3-v3-v3-v3-v3-v3 - Model: blazeface OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: Vulkan GPU-v3-v3-v3-v3-v3-v3 - Model: blazeface 3070 RTX 3070 Ti 4090 rep 4090 4080 4080 zzz 4080 xxx 4080 rep i 3090 rep g 3090 nv 4090 0.3983 0.7966 1.1949 1.5932 1.9915 SE +/- 0.12, N = 14 1.77 1.49 1.46 1.43 1.43 1.42 1.42 1.41 1.41 1.39 1.38 1.36 1.18 MIN: 1.08 / MAX: 12.53 MIN: 1.05 / MAX: 379.08 MIN: 1.39 / MAX: 2.91 MIN: 1.36 / MAX: 2.04 MIN: 1.36 / MAX: 2.06 MIN: 1.34 / MAX: 2.84 MIN: 1.35 / MAX: 2 MIN: 1.34 / MAX: 2.1 MIN: 1.35 / MAX: 2.02 MIN: 1.37 / MAX: 1.52 MIN: 1.35 / MAX: 2.09 MIN: 1.34 / MAX: 1.46 MIN: 1.11 / MAX: 1.85 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread
NCNN Target: Vulkan GPU-v3-v3-v3-v3-v3-v3 - Model: googlenet OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: Vulkan GPU-v3-v3-v3-v3-v3-v3 - Model: googlenet 3070 4090 i 4090 rep RTX 3070 Ti nv 4090 4080 rep 4080 xxx 4080 zzz 4080 g 3090 rep 3090 5 10 15 20 25 SE +/- 0.24, N = 15 18.60 11.30 10.47 10.38 9.84 8.61 8.49 8.43 8.42 8.40 7.96 7.89 7.86 MIN: 8.02 / MAX: 292.16 MIN: 7.95 / MAX: 477.54 MIN: 8.21 / MAX: 350.07 MIN: 7.96 / MAX: 255.68 MIN: 7.3 / MAX: 438.04 MIN: 7.95 / MAX: 10.07 MIN: 7.74 / MAX: 10.76 MIN: 7.77 / MAX: 10.4 MIN: 7.78 / MAX: 10.7 MIN: 7.71 / MAX: 10.64 MIN: 7.81 / MAX: 9.05 MIN: 7.79 / MAX: 8.84 MIN: 7.74 / MAX: 8.62 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread
NCNN Target: Vulkan GPU-v3-v3-v3-v3-v3-v3 - Model: vgg16 OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: Vulkan GPU-v3-v3-v3-v3-v3-v3 - Model: vgg16 3070 4090 4090 rep RTX 3070 Ti nv 4090 i 4080 4080 xxx 4080 zzz 4080 rep g 3090 3090 rep 12 24 36 48 60 SE +/- 0.30, N = 15 55.48 31.57 29.12 28.63 27.89 27.43 25.48 25.40 25.16 25.05 24.04 23.55 23.52 MIN: 25.94 / MAX: 298.67 MIN: 26.09 / MAX: 318.58 MIN: 24.62 / MAX: 266.39 MIN: 24.13 / MAX: 500.18 MIN: 24.5 / MAX: 463.23 MIN: 24.65 / MAX: 251.37 MIN: 23.88 / MAX: 51.68 MIN: 24.05 / MAX: 27.09 MIN: 23.97 / MAX: 27.81 MIN: 23.78 / MAX: 26.95 MIN: 23.48 / MAX: 73.3 MIN: 23.31 / MAX: 24.48 MIN: 23.33 / MAX: 25.08 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread
NCNN Target: Vulkan GPU-v3-v3-v3-v3-v3-v3 - Model: resnet18 OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: Vulkan GPU-v3-v3-v3-v3-v3-v3 - Model: resnet18 3070 nv 4090 4090 RTX 3070 Ti 4090 rep i 4080 xxx 4080 rep 4080 4080 zzz g 3090 rep 3090 3 6 9 12 15 SE +/- 0.23, N = 15 12.14 8.16 6.58 6.57 6.05 5.88 5.67 5.61 5.61 5.60 5.28 5.22 5.19 MIN: 5.28 / MAX: 151.53 MIN: 5.39 / MAX: 397.44 MIN: 6.04 / MAX: 7.81 MIN: 4.91 / MAX: 391.33 MIN: 5.53 / MAX: 7.66 MIN: 5.36 / MAX: 8.2 MIN: 5.1 / MAX: 8.06 MIN: 5.07 / MAX: 7.08 MIN: 5.09 / MAX: 7.91 MIN: 5.09 / MAX: 7.51 MIN: 5.16 / MAX: 6.09 MIN: 5.13 / MAX: 6.1 MIN: 5.09 / MAX: 6 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread
NCNN Target: Vulkan GPU-v3-v3-v3-v3-v3-v3 - Model: alexnet OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: Vulkan GPU-v3-v3-v3-v3-v3-v3 - Model: alexnet 3070 RTX 3070 Ti 4090 rep nv 4090 4090 i 4080 rep 4080 zzz 4080 xxx 4080 g 3090 rep 3090 3 6 9 12 15 SE +/- 0.22, N = 15 10.08 5.53 5.33 5.14 5.14 5.10 4.72 4.68 4.67 4.61 4.35 4.31 4.31 MIN: 4.36 / MAX: 225.66 MIN: 4.22 / MAX: 362.62 MIN: 4.83 / MAX: 6.6 MIN: 4.65 / MAX: 6.81 MIN: 4.76 / MAX: 6.16 MIN: 4.75 / MAX: 6.12 MIN: 4.25 / MAX: 7.3 MIN: 4.26 / MAX: 6.8 MIN: 4.27 / MAX: 6.36 MIN: 4.24 / MAX: 7.25 MIN: 4.28 / MAX: 5.1 MIN: 4.26 / MAX: 5.07 MIN: 4.25 / MAX: 4.94 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread
NCNN Target: Vulkan GPU-v3-v3-v3-v3-v3-v3 - Model: resnet50 OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: Vulkan GPU-v3-v3-v3-v3-v3-v3 - Model: resnet50 3070 4090 nv 4090 i RTX 3070 Ti 4090 rep 4080 4080 zzz 4080 xxx 4080 rep 3090 g 3090 rep 6 12 18 24 30 SE +/- 0.23, N = 15 23.59 14.08 13.63 13.10 12.73 12.17 11.40 10.91 10.91 10.80 10.38 10.33 10.04 MIN: 9.96 / MAX: 177.63 MIN: 10.29 / MAX: 247.29 MIN: 10.52 / MAX: 488.94 MIN: 10.59 / MAX: 267.95 MIN: 9.84 / MAX: 518.97 MIN: 11.25 / MAX: 13.79 MIN: 10.5 / MAX: 13.51 MIN: 9.94 / MAX: 14.83 MIN: 9.91 / MAX: 13.07 MIN: 9.89 / MAX: 12.54 MIN: 9.88 / MAX: 18.75 MIN: 10.2 / MAX: 11.18 MIN: 9.94 / MAX: 10.89 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread
NCNN Target: Vulkan GPU-v3-v3-v3-v3-v3-v3 - Model: yolov4-tiny OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: Vulkan GPU-v3-v3-v3-v3-v3-v3 - Model: yolov4-tiny 3070 nv 4090 4090 4090 rep RTX 3070 Ti 4080 i 4080 xxx 4080 zzz 4080 rep g 3090 3090 rep 7 14 21 28 35 SE +/- 0.28, N = 15 29.80 17.30 15.55 15.45 15.21 13.86 13.77 13.62 13.61 13.55 13.14 13.10 12.86 MIN: 12.85 / MAX: 216.34 MIN: 14.66 / MAX: 441.3 MIN: 13.11 / MAX: 307.2 MIN: 12.65 / MAX: 445.76 MIN: 12.34 / MAX: 380.51 MIN: 13.04 / MAX: 15.04 MIN: 12.96 / MAX: 14.66 MIN: 12.71 / MAX: 15.65 MIN: 12.67 / MAX: 19.72 MIN: 12.72 / MAX: 15.51 MIN: 13 / MAX: 14.02 MIN: 13.01 / MAX: 14.17 MIN: 12.76 / MAX: 13.73 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread
NCNN Target: Vulkan GPU-v3-v3-v3-v3-v3-v3 - Model: squeezenet_ssd OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: Vulkan GPU-v3-v3-v3-v3-v3-v3 - Model: squeezenet_ssd 3070 4090 nv 4090 4090 rep RTX 3070 Ti 4080 xxx 4080 4080 zzz 4080 rep i g 3090 rep 3090 4 8 12 16 20 SE +/- 0.25, N = 14 15.40 9.28 9.21 9.16 8.28 7.67 7.66 7.62 7.55 7.21 7.14 7.09 7.04 MIN: 6.64 / MAX: 132.68 MIN: 6.92 / MAX: 355.6 MIN: 6.83 / MAX: 203.62 MIN: 6.73 / MAX: 423.75 MIN: 6.38 / MAX: 381.81 MIN: 7.04 / MAX: 9.1 MIN: 7.09 / MAX: 8.97 MIN: 7 / MAX: 9.93 MIN: 6.99 / MAX: 9.08 MIN: 6.73 / MAX: 8.82 MIN: 7.03 / MAX: 7.99 MIN: 7.01 / MAX: 7.97 MIN: 6.96 / MAX: 7.7 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread
NCNN Target: Vulkan GPU-v3-v3-v3-v3-v3-v3 - Model: regnety_400m OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: Vulkan GPU-v3-v3-v3-v3-v3-v3 - Model: regnety_400m 3070 4090 nv 4090 RTX 3070 Ti 4090 rep 4080 4080 xxx 4080 zzz i g 3090 rep 4080 rep 3090 4 8 12 16 20 SE +/- 0.21, N = 15 17.88 10.10 10.09 9.19 8.64 8.61 8.52 8.49 8.46 8.38 8.34 8.24 7.99 MIN: 7.38 / MAX: 190.77 MIN: 7.93 / MAX: 156.75 MIN: 7.84 / MAX: 366.66 MIN: 7.44 / MAX: 524.66 MIN: 8.3 / MAX: 10.51 MIN: 8.21 / MAX: 10.07 MIN: 8.13 / MAX: 9.73 MIN: 8.08 / MAX: 9.72 MIN: 8.08 / MAX: 10.33 MIN: 8.05 / MAX: 27.34 MIN: 8.26 / MAX: 9.09 MIN: 7.91 / MAX: 9.53 MIN: 7.92 / MAX: 8.78 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread
NCNN Target: Vulkan GPU-v3-v3-v3-v3-v3-v3 - Model: vision_transformer OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: Vulkan GPU-v3-v3-v3-v3-v3-v3 - Model: vision_transformer 3070 nv 4090 4090 4090 rep i RTX 3070 Ti 4080 4080 xxx 4080 zzz 4080 rep g 3090 3090 rep 16 32 48 64 80 SE +/- 0.20, N = 15 71.08 39.18 39.06 38.17 38.01 37.88 34.91 34.23 34.10 34.10 33.32 33.22 32.09 MIN: 38.84 / MAX: 374.68 MIN: 33.74 / MAX: 520.24 MIN: 34.16 / MAX: 481.28 MIN: 32.97 / MAX: 462.63 MIN: 32.96 / MAX: 388.09 MIN: 32.46 / MAX: 518.57 MIN: 33.72 / MAX: 36.82 MIN: 33.08 / MAX: 37.43 MIN: 32.32 / MAX: 38.54 MIN: 32.43 / MAX: 38.75 MIN: 31.83 / MAX: 104.12 MIN: 33.04 / MAX: 36.99 MIN: 31.84 / MAX: 32.77 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread
NCNN Target: Vulkan GPU-v3-v3-v3-v3-v3-v3 - Model: FastestDet OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: Vulkan GPU-v3-v3-v3-v3-v3-v3 - Model: FastestDet 3070 RTX 3070 Ti 4080 4080 zzz 4080 xxx 4080 rep 4090 3090 rep 3090 4090 rep g i nv 4090 2 4 6 8 10 SE +/- 0.20, N = 15 6.93 4.41 4.28 4.20 4.17 4.14 4.13 4.08 4.03 3.96 3.92 3.83 2.81 MIN: 2.57 / MAX: 163.84 MIN: 2.06 / MAX: 295.24 MIN: 4.13 / MAX: 4.85 MIN: 4.01 / MAX: 11.47 MIN: 4.03 / MAX: 5.63 MIN: 4 / MAX: 5.6 MIN: 3.99 / MAX: 4.67 MIN: 4.04 / MAX: 4.29 MIN: 3.99 / MAX: 4.22 MIN: 3.79 / MAX: 11.36 MIN: 3.88 / MAX: 4.72 MIN: 3.7 / MAX: 4.57 MIN: 2.68 / MAX: 4.38 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread
NCNN Target: Vulkan GPU-v3-v3-v3-v3-v3-v3-v3-v3-v3 - Model: mobilenet OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: Vulkan GPU-v3-v3-v3-v3-v3-v3-v3-v3-v3 - Model: mobilenet 3070 RTX 3070 Ti 4080 zzz 4090 4090 rep nv 4090 4080 4080 rep 4080 xxx 3090 rep 3090 4 8 12 16 20 SE +/- 0.25, N = 15 17.09 9.52 9.19 9.16 9.02 8.93 8.43 8.38 8.31 8.05 8.03 MIN: 7.89 / MAX: 121.53 MIN: 7.97 / MAX: 420.29 MIN: 8.51 / MAX: 11.04 MIN: 8.5 / MAX: 10.51 MIN: 8.42 / MAX: 11.17 MIN: 8.33 / MAX: 11.07 MIN: 8.03 / MAX: 9.64 MIN: 7.94 / MAX: 10.07 MIN: 7.85 / MAX: 10.21 MIN: 7.96 / MAX: 9.04 MIN: 7.96 / MAX: 8.83 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread
NCNN Target: Vulkan GPU-v3-v3-v3-v3-v3-v3-v3-v3-v3-v2-v2 - Model: mobilenet-v2 OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: Vulkan GPU-v3-v3-v3-v3-v3-v3-v3-v3-v3-v2-v2 - Model: mobilenet-v2 3070 RTX 3070 Ti 4090 nv 4090 4090 rep 4080 4080 zzz 4080 rep 3090 rep 4080 xxx 3090 1.2285 2.457 3.6855 4.914 6.1425 SE +/- 0.18, N = 15 5.46 3.66 3.48 3.42 3.36 3.29 3.28 3.27 3.17 3.14 3.12 MIN: 3.27 / MAX: 38.65 MIN: 2.73 / MAX: 398.42 MIN: 3.32 / MAX: 4.99 MIN: 3.15 / MAX: 25.1 MIN: 3.17 / MAX: 4.8 MIN: 3.12 / MAX: 3.99 MIN: 3.11 / MAX: 4.26 MIN: 3.08 / MAX: 4.68 MIN: 3.12 / MAX: 3.89 MIN: 3 / MAX: 3.85 MIN: 3.07 / MAX: 3.62 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread
NCNN Target: Vulkan GPU-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3 - Model: mobilenet-v3 OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: Vulkan GPU-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3 - Model: mobilenet-v3 3070 4090 RTX 3070 Ti 4090 rep 4080 rep 4080 zzz 4080 3090 rep nv 4090 3090 4080 xxx 1.3478 2.6956 4.0434 5.3912 6.739 SE +/- 0.13, N = 13 5.99 3.62 3.44 3.34 3.31 3.26 3.26 3.18 3.17 3.13 3.05 MIN: 3.05 / MAX: 26.81 MIN: 3.47 / MAX: 4.24 MIN: 2.65 / MAX: 361.91 MIN: 3.19 / MAX: 3.99 MIN: 3.16 / MAX: 3.93 MIN: 3.12 / MAX: 4.74 MIN: 3.13 / MAX: 4.7 MIN: 3.13 / MAX: 3.61 MIN: 3.04 / MAX: 4.3 MIN: 3.09 / MAX: 3.68 MIN: 2.94 / MAX: 3.56 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread
NCNN Target: Vulkan GPU-v3-v3-v3-v3-v3-v3-v3-v3-v3 - Model: shufflenet-v2 OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: Vulkan GPU-v3-v3-v3-v3-v3-v3-v3-v3-v3 - Model: shufflenet-v2 3070 RTX 3070 Ti 4090 nv 4090 4090 rep 4080 4080 zzz 4080 rep 3090 rep 4080 xxx 3090 1.2578 2.5156 3.7734 5.0312 6.289 SE +/- 0.20, N = 15 5.59 3.89 3.52 3.50 3.48 3.48 3.44 3.44 3.34 3.34 3.32 MIN: 3.32 / MAX: 42.33 MIN: 3.08 / MAX: 345.39 MIN: 3.38 / MAX: 4.23 MIN: 3.37 / MAX: 4.2 MIN: 3.34 / MAX: 4.1 MIN: 3.34 / MAX: 4.88 MIN: 3.31 / MAX: 4.85 MIN: 3.31 / MAX: 4.32 MIN: 3.3 / MAX: 3.79 MIN: 3.22 / MAX: 3.97 MIN: 3.29 / MAX: 3.79 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread
NCNN Target: Vulkan GPU-v3-v3-v3-v3-v3-v3-v3-v3-v3 - Model: mnasnet OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: Vulkan GPU-v3-v3-v3-v3-v3-v3-v3-v3-v3 - Model: mnasnet 3070 4090 rep 4090 RTX 3070 Ti 4080 4080 zzz nv 4090 4080 rep 4080 xxx 3090 rep 3090 2 4 6 8 10 SE +/- 0.04, N = 15 6.06 4.99 4.93 3.10 3.10 3.08 3.07 3.06 2.98 2.97 2.94 MIN: 2.96 / MAX: 42.7 MIN: 3.02 / MAX: 235.56 MIN: 2.97 / MAX: 124.96 MIN: 2.61 / MAX: 4.75 MIN: 2.95 / MAX: 4.05 MIN: 2.95 / MAX: 3.88 MIN: 2.93 / MAX: 4.52 MIN: 2.93 / MAX: 5.02 MIN: 2.86 / MAX: 4.47 MIN: 2.94 / MAX: 3.45 MIN: 2.9 / MAX: 3.34 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread
NCNN Target: Vulkan GPU-v3-v3-v3-v3-v3-v3-v3-v3-v3 - Model: efficientnet-b0 OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: Vulkan GPU-v3-v3-v3-v3-v3-v3-v3-v3-v3 - Model: efficientnet-b0 3070 4090 rep RTX 3070 Ti 4090 nv 4090 4080 zzz 4080 4080 rep 4080 xxx 3090 3090 rep 3 6 9 12 15 SE +/- 0.13, N = 15 9.81 4.41 4.37 4.15 4.10 4.05 4.04 4.01 3.97 3.88 3.85 MIN: 3.87 / MAX: 165.38 MIN: 4.21 / MAX: 5.82 MIN: 3.85 / MAX: 366.28 MIN: 3.93 / MAX: 5.94 MIN: 3.87 / MAX: 6.14 MIN: 3.83 / MAX: 5.42 MIN: 3.84 / MAX: 4.83 MIN: 3.81 / MAX: 6.04 MIN: 3.79 / MAX: 5.93 MIN: 3.83 / MAX: 4.72 MIN: 3.78 / MAX: 4.83 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread
NCNN Target: Vulkan GPU-v3-v3-v3-v3-v3-v3-v3-v3-v3 - Model: blazeface OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: Vulkan GPU-v3-v3-v3-v3-v3-v3-v3-v3-v3 - Model: blazeface 3070 4080 4090 4080 rep 4090 rep 4080 zzz 3090 3090 rep RTX 3070 Ti 4080 xxx nv 4090 0.6728 1.3456 2.0184 2.6912 3.364 SE +/- 0.03, N = 15 2.99 1.44 1.42 1.42 1.41 1.41 1.38 1.37 1.34 1.31 1.26 MIN: 1.22 / MAX: 149.55 MIN: 1.37 / MAX: 2.07 MIN: 1.36 / MAX: 1.92 MIN: 1.35 / MAX: 2.89 MIN: 1.35 / MAX: 1.91 MIN: 1.34 / MAX: 1.88 MIN: 1.36 / MAX: 1.53 MIN: 1.35 / MAX: 1.48 MIN: 1.06 / MAX: 2.66 MIN: 1.25 / MAX: 3.14 MIN: 1.2 / MAX: 1.76 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread
NCNN Target: Vulkan GPU-v3-v3-v3-v3-v3-v3-v3-v3-v3 - Model: googlenet OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: Vulkan GPU-v3-v3-v3-v3-v3-v3-v3-v3-v3 - Model: googlenet 3070 4090 rep RTX 3070 Ti 4090 4080 4080 zzz 4080 rep nv 4090 4080 xxx 3090 rep 3090 4 8 12 16 20 SE +/- 0.19, N = 15 16.97 10.39 9.90 9.05 8.79 8.55 8.42 8.35 8.26 7.86 7.84 MIN: 7.44 / MAX: 229.93 MIN: 7.87 / MAX: 391.66 MIN: 7.76 / MAX: 396.66 MIN: 8.26 / MAX: 13.34 MIN: 8.08 / MAX: 10.27 MIN: 7.86 / MAX: 10.08 MIN: 7.77 / MAX: 10.52 MIN: 7.7 / MAX: 10.46 MIN: 7.62 / MAX: 10.47 MIN: 7.75 / MAX: 8.71 MIN: 7.74 / MAX: 8.72 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread
NCNN Target: Vulkan GPU-v3-v3-v3-v3-v3-v3-v3-v3-v3 - Model: vgg16 OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: Vulkan GPU-v3-v3-v3-v3-v3-v3-v3-v3-v3 - Model: vgg16 3070 4090 rep RTX 3070 Ti nv 4090 4090 4080 zzz 4080 4080 rep 4080 xxx 3090 3090 rep 11 22 33 44 55 SE +/- 0.28, N = 15 49.70 29.17 28.53 28.14 27.44 26.09 25.67 25.56 25.33 23.51 23.38 MIN: 25.55 / MAX: 421.44 MIN: 24.61 / MAX: 264.85 MIN: 23.95 / MAX: 473.83 MIN: 24.24 / MAX: 221.5 MIN: 24.06 / MAX: 264.59 MIN: 24.58 / MAX: 30.18 MIN: 24.46 / MAX: 27.34 MIN: 24.24 / MAX: 27.92 MIN: 24.26 / MAX: 34.98 MIN: 23.27 / MAX: 24.38 MIN: 23.19 / MAX: 24.27 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread
NCNN Target: Vulkan GPU-v3-v3-v3-v3-v3-v3-v3-v3-v3 - Model: resnet18 OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: Vulkan GPU-v3-v3-v3-v3-v3-v3-v3-v3-v3 - Model: resnet18 3070 4090 nv 4090 RTX 3070 Ti 4080 4090 rep 4080 zzz 4080 rep 4080 xxx 3090 3090 rep 3 6 9 12 15 SE +/- 0.20, N = 15 11.30 7.52 7.38 6.40 5.92 5.90 5.74 5.67 5.65 5.21 5.20 MIN: 5.3 / MAX: 181.7 MIN: 5.45 / MAX: 290.49 MIN: 5.15 / MAX: 138.85 MIN: 5.1 / MAX: 457.07 MIN: 5.37 / MAX: 8.24 MIN: 5.43 / MAX: 7.49 MIN: 5.18 / MAX: 8.08 MIN: 5.19 / MAX: 7.38 MIN: 5.18 / MAX: 6.76 MIN: 5.09 / MAX: 6.13 MIN: 5.1 / MAX: 6.09 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread
NCNN Target: Vulkan GPU-v3-v3-v3-v3-v3-v3-v3-v3-v3 - Model: alexnet OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: Vulkan GPU-v3-v3-v3-v3-v3-v3-v3-v3-v3 - Model: alexnet 3070 RTX 3070 Ti 4090 rep 4080 4080 xxx 4080 zzz nv 4090 4090 4080 rep 3090 3090 rep 3 6 9 12 15 SE +/- 0.16, N = 15 11.89 5.34 5.25 4.98 4.71 4.70 4.69 4.67 4.64 4.32 4.30 MIN: 4.34 / MAX: 229.18 MIN: 4.25 / MAX: 221.78 MIN: 4.86 / MAX: 6.33 MIN: 4.59 / MAX: 7.15 MIN: 4.26 / MAX: 7.21 MIN: 4.28 / MAX: 5.92 MIN: 4.28 / MAX: 6.33 MIN: 4.28 / MAX: 6 MIN: 4.24 / MAX: 6 MIN: 4.25 / MAX: 5.33 MIN: 4.24 / MAX: 5.11 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread
NCNN Target: Vulkan GPU-v3-v3-v3-v3-v3-v3-v3-v3-v3 - Model: resnet50 OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: Vulkan GPU-v3-v3-v3-v3-v3-v3-v3-v3-v3 - Model: resnet50 3070 nv 4090 4080 zzz RTX 3070 Ti 4090 4090 rep 4080 4080 xxx 4080 rep 3090 3090 rep 6 12 18 24 30 SE +/- 0.25, N = 15 23.44 13.13 12.50 12.42 12.40 11.51 11.48 11.22 11.07 10.07 9.98 MIN: 10.17 / MAX: 219.36 MIN: 10.18 / MAX: 247.5 MIN: 11.47 / MAX: 14.56 MIN: 10.23 / MAX: 444.76 MIN: 11.44 / MAX: 14.43 MIN: 10.56 / MAX: 13.22 MIN: 10.56 / MAX: 12.93 MIN: 10.33 / MAX: 12.81 MIN: 10.16 / MAX: 13.16 MIN: 9.95 / MAX: 10.88 MIN: 9.85 / MAX: 11.35 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread
NCNN Target: Vulkan GPU-v3-v3-v3-v3-v3-v3-v3-v3-v3 - Model: yolov4-tiny OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: Vulkan GPU-v3-v3-v3-v3-v3-v3-v3-v3-v3 - Model: yolov4-tiny 3070 4090 nv 4090 4090 rep 4080 zzz RTX 3070 Ti 4080 4080 rep 4080 xxx 3090 3090 rep 7 14 21 28 35 SE +/- 0.19, N = 15 28.73 15.95 15.55 15.40 15.26 15.00 13.93 13.73 13.52 12.97 12.90 MIN: 12.83 / MAX: 264.49 MIN: 13.38 / MAX: 245.18 MIN: 12.87 / MAX: 342.3 MIN: 13 / MAX: 245.79 MIN: 14.19 / MAX: 17.06 MIN: 12.75 / MAX: 401.37 MIN: 13.08 / MAX: 15.68 MIN: 12.78 / MAX: 20.99 MIN: 12.72 / MAX: 21.19 MIN: 12.83 / MAX: 13.8 MIN: 12.77 / MAX: 13.92 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread
NCNN Target: Vulkan GPU-v3-v3-v3-v3-v3-v3-v3-v3-v3 - Model: squeezenet_ssd OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: Vulkan GPU-v3-v3-v3-v3-v3-v3-v3-v3-v3 - Model: squeezenet_ssd 3070 4090 4090 rep RTX 3070 Ti 4080 zzz 4080 4080 rep 4080 xxx 3090 rep 3090 nv 4090 5 10 15 20 25 SE +/- 0.24, N = 15 18.83 9.81 9.46 8.65 8.06 7.71 7.62 7.27 7.08 7.05 7.02 MIN: 6.71 / MAX: 206.11 MIN: 7.16 / MAX: 389.1 MIN: 7.03 / MAX: 160.39 MIN: 6.64 / MAX: 544.17 MIN: 7.42 / MAX: 9.25 MIN: 7.15 / MAX: 9.1 MIN: 7.01 / MAX: 14.37 MIN: 6.74 / MAX: 8.84 MIN: 7 / MAX: 7.94 MIN: 6.97 / MAX: 7.95 MIN: 6.38 / MAX: 9.36 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread
NCNN Target: Vulkan GPU-v3-v3-v3-v3-v3-v3-v3-v3-v3 - Model: regnety_400m OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: Vulkan GPU-v3-v3-v3-v3-v3-v3-v3-v3-v3 - Model: regnety_400m 3070 4090 rep nv 4090 4090 RTX 3070 Ti 4080 4080 zzz 4080 rep 3090 4080 xxx 3090 rep 4 8 12 16 20 SE +/- 0.24, N = 15 17.61 10.69 10.03 9.87 9.05 8.67 8.37 8.35 8.33 8.25 8.07 MIN: 7.85 / MAX: 165.34 MIN: 8.17 / MAX: 339.6 MIN: 7.81 / MAX: 171.2 MIN: 7.81 / MAX: 243.06 MIN: 7.52 / MAX: 417.33 MIN: 8.3 / MAX: 14.66 MIN: 8.04 / MAX: 10.13 MIN: 8.05 / MAX: 9.76 MIN: 8.25 / MAX: 9.32 MIN: 7.93 / MAX: 9.88 MIN: 7.99 / MAX: 8.88 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread
NCNN Target: Vulkan GPU-v3-v3-v3-v3-v3-v3-v3-v3-v3 - Model: vision_transformer OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: Vulkan GPU-v3-v3-v3-v3-v3-v3-v3-v3-v3 - Model: vision_transformer 3070 4090 rep 4090 nv 4090 RTX 3070 Ti 4080 4080 zzz 4080 rep 4080 xxx 3090 rep 3090 16 32 48 64 80 SE +/- 0.11, N = 15 70.29 39.03 38.62 38.46 38.27 35.60 35.36 33.93 33.90 31.97 31.94 MIN: 39.39 / MAX: 250.19 MIN: 33.61 / MAX: 343.67 MIN: 33.33 / MAX: 465 MIN: 32.39 / MAX: 435.46 MIN: 32.29 / MAX: 507.7 MIN: 34.13 / MAX: 38.49 MIN: 33.87 / MAX: 42.41 MIN: 32.77 / MAX: 36.2 MIN: 32.72 / MAX: 37.77 MIN: 31.71 / MAX: 33.78 MIN: 31.72 / MAX: 34.34 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread
NCNN Target: Vulkan GPU-v3-v3-v3-v3-v3-v3-v3-v3-v3 - Model: FastestDet OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: Vulkan GPU-v3-v3-v3-v3-v3-v3-v3-v3-v3 - Model: FastestDet 3070 4080 zzz 4090 RTX 3070 Ti 4080 4080 rep 4090 rep 3090 rep 3090 4080 xxx nv 4090 2 4 6 8 10 SE +/- 0.27, N = 15 6.71 4.61 4.45 4.32 4.19 4.17 4.11 4.07 3.83 3.75 2.64 MIN: 2.73 / MAX: 109.52 MIN: 4.45 / MAX: 5.92 MIN: 4.29 / MAX: 5.05 MIN: 2.51 / MAX: 398.91 MIN: 4.06 / MAX: 7.41 MIN: 4.02 / MAX: 4.75 MIN: 3.98 / MAX: 4.73 MIN: 4.03 / MAX: 4.18 MIN: 3.79 / MAX: 4.09 MIN: 3.63 / MAX: 5.24 MIN: 2.52 / MAX: 4.14 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread
NCNN Target: CPU-v3-v3-v3-v3-v3-v3 - Model: mobilenet OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: CPU-v3-v3-v3-v3-v3-v3 - Model: mobilenet 3070 nv 4090 4090 rep RTX 3070 Ti 4090 4080 rep 4080 xxx 4080 zzz 3090 3090 rep 4 8 12 16 20 SE +/- 0.14, N = 3 16.52 12.12 10.61 10.03 9.04 8.45 8.34 8.25 8.07 8.06 MIN: 7.9 / MAX: 82.53 MIN: 9.16 / MAX: 505.01 MIN: 8.34 / MAX: 225.97 MIN: 7.86 / MAX: 346.64 MIN: 8.49 / MAX: 10.96 MIN: 8.01 / MAX: 10.86 MIN: 7.89 / MAX: 9.42 MIN: 7.78 / MAX: 9.61 MIN: 8.01 / MAX: 8.62 MIN: 8 / MAX: 8.96 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread
NCNN Target: CPU-v3-v3-v3-v3-v3-v3-v2-v2 - Model: mobilenet-v2 OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: CPU-v3-v3-v3-v3-v3-v3-v2-v2 - Model: mobilenet-v2 3070 RTX 3070 Ti 4090 rep 4090 4080 rep nv 4090 4080 xxx 3090 rep 3090 4080 zzz 2 4 6 8 10 SE +/- 0.53, N = 3 7.22 3.91 3.44 3.36 3.30 3.29 3.20 3.16 3.16 3.16 MIN: 3.17 / MAX: 69.66 MIN: 3.04 / MAX: 394.66 MIN: 3.27 / MAX: 4.93 MIN: 3.21 / MAX: 4.78 MIN: 3.11 / MAX: 4.01 MIN: 3.13 / MAX: 4.29 MIN: 3.05 / MAX: 4.67 MIN: 3.09 / MAX: 4.06 MIN: 3.11 / MAX: 3.95 MIN: 3.01 / MAX: 5.17 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread
NCNN Target: CPU-v3-v3-v3-v3-v3-v3-v3-v3 - Model: mobilenet-v3 OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: CPU-v3-v3-v3-v3-v3-v3-v3-v3 - Model: mobilenet-v3 3070 nv 4090 RTX 3070 Ti 4090 4090 rep 4080 rep 3090 rep 4080 xxx 4080 zzz 2 4 6 8 10 SE +/- 0.53, N = 3 6.43 4.97 3.70 3.33 3.30 3.28 3.17 3.08 3.06 MIN: 2.85 / MAX: 164.91 MIN: 3.15 / MAX: 291.01 MIN: 2.98 / MAX: 261.6 MIN: 3.2 / MAX: 4.4 MIN: 3.15 / MAX: 3.91 MIN: 3.13 / MAX: 4.78 MIN: 3.12 / MAX: 3.75 MIN: 2.97 / MAX: 3.67 MIN: 2.94 / MAX: 3.94 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread
NCNN Target: CPU-v3-v3-v3-v3-v3-v3 - Model: shufflenet-v2 OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: CPU-v3-v3-v3-v3-v3-v3 - Model: shufflenet-v2 3070 RTX 3070 Ti 4090 4080 rep 4090 rep 4080 xxx 3090 4080 zzz 3090 rep nv 4090 2 4 6 8 10 SE +/- 0.60, N = 3 7.81 4.02 3.48 3.47 3.42 3.40 3.36 3.36 3.33 3.32 MIN: 3.3 / MAX: 131.26 MIN: 3.27 / MAX: 328.59 MIN: 3.35 / MAX: 4.05 MIN: 3.33 / MAX: 5.39 MIN: 3.29 / MAX: 3.94 MIN: 3.28 / MAX: 3.87 MIN: 3.32 / MAX: 3.66 MIN: 3.23 / MAX: 3.99 MIN: 3.3 / MAX: 3.78 MIN: 3.19 / MAX: 4.76 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread
NCNN Target: CPU-v3-v3-v3-v3-v3-v3 - Model: mnasnet OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: CPU-v3-v3-v3-v3-v3-v3 - Model: mnasnet 3070 4090 4090 rep RTX 3070 Ti nv 4090 4080 rep 4080 xxx 3090 3090 rep 4080 zzz 2 4 6 8 10 SE +/- 0.13, N = 3 6.07 5.19 5.11 3.24 3.12 3.09 3.00 2.98 2.97 2.96 MIN: 2.94 / MAX: 129.1 MIN: 3.04 / MAX: 436.91 MIN: 2.96 / MAX: 247.47 MIN: 2.9 / MAX: 5.34 MIN: 2.98 / MAX: 3.71 MIN: 2.96 / MAX: 4.98 MIN: 2.88 / MAX: 4.37 MIN: 2.95 / MAX: 3.9 MIN: 2.93 / MAX: 3.3 MIN: 2.85 / MAX: 3.82 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread
NCNN Target: CPU-v3-v3-v3-v3-v3-v3 - Model: efficientnet-b0 OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: CPU-v3-v3-v3-v3-v3-v3 - Model: efficientnet-b0 3070 nv 4090 RTX 3070 Ti 4090 4090 rep 4080 rep 4080 xxx 4080 zzz 3090 3090 rep 3 6 9 12 15 SE +/- 0.46, N = 3 9.19 5.94 4.74 4.47 4.35 4.07 4.01 3.95 3.88 3.85 MIN: 3.85 / MAX: 131.42 MIN: 3.97 / MAX: 208.59 MIN: 3.68 / MAX: 295.7 MIN: 4.23 / MAX: 5.82 MIN: 4.08 / MAX: 5.62 MIN: 3.85 / MAX: 4.79 MIN: 3.83 / MAX: 5.28 MIN: 3.79 / MAX: 4.59 MIN: 3.83 / MAX: 4.61 MIN: 3.81 / MAX: 4.75 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread
NCNN Target: CPU-v3-v3-v3-v3-v3-v3 - Model: blazeface OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: CPU-v3-v3-v3-v3-v3-v3 - Model: blazeface 3070 RTX 3070 Ti 4080 rep nv 4090 4090 rep 3090 3090 rep 4080 xxx 4080 zzz 4090 0.5693 1.1386 1.7079 2.2772 2.8465 SE +/- 0.48, N = 3 2.53 2.48 1.43 1.42 1.42 1.39 1.38 1.32 1.31 1.30 MIN: 1.08 / MAX: 118.73 MIN: 1.17 / MAX: 344.52 MIN: 1.36 / MAX: 2.02 MIN: 1.34 / MAX: 1.99 MIN: 1.34 / MAX: 2.37 MIN: 1.37 / MAX: 1.48 MIN: 1.35 / MAX: 1.64 MIN: 1.26 / MAX: 2.03 MIN: 1.25 / MAX: 1.76 MIN: 1.24 / MAX: 1.92 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread
NCNN Target: CPU-v3-v3-v3-v3-v3-v3 - Model: googlenet OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: CPU-v3-v3-v3-v3-v3-v3 - Model: googlenet 3070 nv 4090 RTX 3070 Ti 4090 rep 4080 rep 4090 4080 xxx 4080 zzz 3090 rep 3090 5 10 15 20 25 SE +/- 0.80, N = 3 19.20 10.75 9.68 8.97 8.52 8.38 8.32 8.29 7.91 7.90 MIN: 7.84 / MAX: 193.36 MIN: 7.92 / MAX: 447.83 MIN: 8.16 / MAX: 382.41 MIN: 8.22 / MAX: 10.51 MIN: 7.85 / MAX: 10.56 MIN: 7.78 / MAX: 10.43 MIN: 7.71 / MAX: 10.39 MIN: 7.63 / MAX: 9.87 MIN: 7.81 / MAX: 8.62 MIN: 7.8 / MAX: 8.73 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread
NCNN Target: CPU-v3-v3-v3-v3-v3-v3 - Model: vgg16 OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: CPU-v3-v3-v3-v3-v3-v3 - Model: vgg16 3070 4090 rep 4090 RTX 3070 Ti nv 4090 4080 xxx 4080 zzz 4080 rep 3090 3090 rep 11 22 33 44 55 SE +/- 0.54, N = 3 50.32 30.74 30.16 27.98 27.61 25.44 25.26 25.01 23.58 23.40 MIN: 25.92 / MAX: 281.06 MIN: 25.36 / MAX: 428.68 MIN: 24.66 / MAX: 332.49 MIN: 24.35 / MAX: 423.63 MIN: 24.67 / MAX: 401.29 MIN: 24.27 / MAX: 27.68 MIN: 24.29 / MAX: 27.75 MIN: 23.88 / MAX: 26.66 MIN: 23.35 / MAX: 24.43 MIN: 23.2 / MAX: 24.07 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread
NCNN Target: CPU-v3-v3-v3-v3-v3-v3 - Model: resnet18 OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: CPU-v3-v3-v3-v3-v3-v3 - Model: resnet18 3070 4090 rep 4090 RTX 3070 Ti nv 4090 4080 xxx 4080 zzz 4080 rep 3090 rep 3090 3 6 9 12 15 SE +/- 0.30, N = 3 12.64 8.14 7.74 6.22 6.07 5.78 5.77 5.64 5.30 5.20 MIN: 5.3 / MAX: 53.81 MIN: 5.39 / MAX: 122.47 MIN: 5.25 / MAX: 312.09 MIN: 5.3 / MAX: 8.22 MIN: 5.49 / MAX: 15.12 MIN: 5.21 / MAX: 6.97 MIN: 5.22 / MAX: 7.06 MIN: 5.11 / MAX: 7.51 MIN: 5.21 / MAX: 6.24 MIN: 5.1 / MAX: 6.16 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread
NCNN Target: CPU-v3-v3-v3-v3-v3-v3 - Model: alexnet OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: CPU-v3-v3-v3-v3-v3-v3 - Model: alexnet 3070 nv 4090 RTX 3070 Ti 4090 rep 4090 4080 xxx 4080 rep 4080 zzz 3090 3090 rep 3 6 9 12 15 SE +/- 0.43, N = 3 10.59 6.62 6.17 5.45 4.99 4.69 4.68 4.66 4.33 4.31 MIN: 4.3 / MAX: 177.68 MIN: 4.28 / MAX: 339.62 MIN: 4.5 / MAX: 261.75 MIN: 4.93 / MAX: 7.98 MIN: 4.56 / MAX: 6.91 MIN: 4.26 / MAX: 6.07 MIN: 4.27 / MAX: 6.08 MIN: 4.24 / MAX: 5.97 MIN: 4.26 / MAX: 5.19 MIN: 4.26 / MAX: 5.07 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread
NCNN Target: CPU-v3-v3-v3-v3-v3-v3 - Model: resnet50 OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: CPU-v3-v3-v3-v3-v3-v3 - Model: resnet50 3070 nv 4090 RTX 3070 Ti 4090 rep 4090 4080 xxx 4080 zzz 4080 rep 3090 rep 3090 5 10 15 20 25 SE +/- 0.04, N = 3 22.19 13.29 12.81 12.47 11.72 11.26 11.10 10.86 10.06 10.03 MIN: 10.16 / MAX: 181.74 MIN: 10.54 / MAX: 456.82 MIN: 10.06 / MAX: 349.03 MIN: 11.5 / MAX: 14.68 MIN: 10.8 / MAX: 12.8 MIN: 10.32 / MAX: 13.29 MIN: 10.19 / MAX: 18.3 MIN: 9.98 / MAX: 12.46 MIN: 9.86 / MAX: 11.9 MIN: 9.93 / MAX: 10.87 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread
NCNN Target: CPU-v3-v3-v3-v3-v3-v3 - Model: yolov4-tiny OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: CPU-v3-v3-v3-v3-v3-v3 - Model: yolov4-tiny 3070 nv 4090 4090 RTX 3070 Ti 4090 rep 4080 rep 4080 xxx 4080 zzz 3090 3090 rep 7 14 21 28 35 SE +/- 0.81, N = 3 28.41 17.67 15.85 14.57 13.88 13.71 13.63 13.42 12.82 12.81 MIN: 12.49 / MAX: 151.04 MIN: 14.92 / MAX: 343.93 MIN: 13.26 / MAX: 253.23 MIN: 12.33 / MAX: 312.42 MIN: 13.09 / MAX: 14.77 MIN: 12.78 / MAX: 15.62 MIN: 12.77 / MAX: 16.93 MIN: 12.65 / MAX: 16.19 MIN: 12.72 / MAX: 13.66 MIN: 12.7 / MAX: 13.69 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread
NCNN Target: CPU-v3-v3-v3-v3-v3-v3 - Model: squeezenet_ssd OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: CPU-v3-v3-v3-v3-v3-v3 - Model: squeezenet_ssd 3070 4090 4090 rep 4080 rep RTX 3070 Ti nv 4090 4080 xxx 4080 zzz 3090 3090 rep 4 8 12 16 20 SE +/- 0.23, N = 3 14.27 9.51 9.38 7.67 7.57 7.48 7.27 7.25 7.12 7.09 MIN: 7.01 / MAX: 51.13 MIN: 7.11 / MAX: 307.17 MIN: 6.77 / MAX: 224.11 MIN: 7.06 / MAX: 9.96 MIN: 6.69 / MAX: 10 MIN: 6.85 / MAX: 9.67 MIN: 6.73 / MAX: 8.77 MIN: 6.72 / MAX: 8.05 MIN: 7.04 / MAX: 7.97 MIN: 7.02 / MAX: 7.86 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread
NCNN Target: CPU-v3-v3-v3-v3-v3-v3 - Model: regnety_400m OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: CPU-v3-v3-v3-v3-v3-v3 - Model: regnety_400m 3070 4090 rep 4090 4080 rep RTX 3070 Ti 4080 xxx 4080 zzz nv 4090 3090 3090 rep 4 8 12 16 20 SE +/- 0.29, N = 3 18.25 10.23 10.09 8.72 8.42 8.38 8.34 8.25 8.22 8.03 MIN: 7.8 / MAX: 238.29 MIN: 8.22 / MAX: 197.1 MIN: 8.01 / MAX: 418.58 MIN: 8.32 / MAX: 10.48 MIN: 7.66 / MAX: 10.74 MIN: 8.04 / MAX: 9.63 MIN: 8.03 / MAX: 10.23 MIN: 7.87 / MAX: 10.07 MIN: 8.14 / MAX: 8.67 MIN: 7.97 / MAX: 8.65 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread
NCNN Target: CPU-v3-v3-v3-v3-v3-v3 - Model: vision_transformer OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: CPU-v3-v3-v3-v3-v3-v3 - Model: vision_transformer 3070 nv 4090 4090 rep 4090 RTX 3070 Ti 4080 zzz 4080 rep 4080 xxx 3090 3090 rep 15 30 45 60 75 SE +/- 0.11, N = 3 65.41 38.95 38.79 38.76 38.04 34.47 34.22 34.14 32.10 31.85 MIN: 39.08 / MAX: 230.59 MIN: 34.04 / MAX: 486.96 MIN: 34.02 / MAX: 460.15 MIN: 33.38 / MAX: 423.24 MIN: 33.11 / MAX: 346.94 MIN: 33.05 / MAX: 39.69 MIN: 33.01 / MAX: 37.09 MIN: 32.5 / MAX: 37.13 MIN: 31.9 / MAX: 33.03 MIN: 31.67 / MAX: 35.74 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread
NCNN Target: CPU-v3-v3-v3-v3-v3-v3 - Model: FastestDet OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: CPU-v3-v3-v3-v3-v3-v3 - Model: FastestDet 3070 4080 rep RTX 3070 Ti 3090 3090 rep nv 4090 4080 zzz 4080 xxx 4090 rep 4090 2 4 6 8 10 SE +/- 0.87, N = 3 7.12 4.20 4.18 4.10 4.07 3.93 3.82 3.80 3.12 2.85 MIN: 3.72 / MAX: 188.7 MIN: 4.04 / MAX: 5.63 MIN: 2.53 / MAX: 295.11 MIN: 4.07 / MAX: 4.34 MIN: 4.03 / MAX: 4.2 MIN: 3.76 / MAX: 11.77 MIN: 3.65 / MAX: 9.77 MIN: 3.65 / MAX: 6.08 MIN: 2.97 / MAX: 4.42 MIN: 2.74 / MAX: 4.36 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread
NCNN Target: CPU-v3-v3-v3-v3-v3-v3-v3-v3-v3 - Model: mobilenet OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: CPU-v3-v3-v3-v3-v3-v3-v3-v3-v3 - Model: mobilenet 3070 nv 4090 RTX 3070 Ti 4090 rep 4090 3090 rep 3090 5 10 15 20 25 SE +/- 0.24, N = 15 18.54 10.15 9.98 8.83 8.46 8.05 8.01 MIN: 8.01 / MAX: 164.45 MIN: 8.08 / MAX: 193.04 MIN: 7.79 / MAX: 434.9 MIN: 8.29 / MAX: 10.15 MIN: 8.12 / MAX: 10.14 MIN: 7.98 / MAX: 8.94 MIN: 7.96 / MAX: 8.47 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread
NCNN Target: CPU-v3-v3-v3-v3-v3-v3-v3-v3-v3-v2-v2 - Model: mobilenet-v2 OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: CPU-v3-v3-v3-v3-v3-v3-v3-v3-v3-v2-v2 - Model: mobilenet-v2 3070 4090 RTX 3070 Ti 4090 rep nv 4090 3090 rep 3090 1.2353 2.4706 3.7059 4.9412 6.1765 SE +/- 0.20, N = 15 5.49 5.25 3.69 3.60 3.27 3.17 3.15 MIN: 2.97 / MAX: 152.08 MIN: 3.11 / MAX: 367.53 MIN: 3.07 / MAX: 544.13 MIN: 3.44 / MAX: 4.27 MIN: 3.11 / MAX: 4.1 MIN: 3.12 / MAX: 3.78 MIN: 3.11 / MAX: 3.78 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread
NCNN Target: CPU-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3 - Model: mobilenet-v3 OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: CPU-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3 - Model: mobilenet-v3 3070 nv 4090 RTX 3070 Ti 4090 rep 4090 3090 1.3433 2.6866 4.0299 5.3732 6.7165 SE +/- 0.17, N = 15 5.97 4.81 3.52 3.44 3.36 3.16 MIN: 2.84 / MAX: 111.8 MIN: 3.13 / MAX: 149.75 MIN: 2.95 / MAX: 536.1 MIN: 3.3 / MAX: 4.34 MIN: 3.21 / MAX: 4.83 MIN: 3.12 / MAX: 3.67 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread
NCNN Target: CPU-v3-v3-v3-v3-v3-v3-v3-v3-v3 - Model: shufflenet-v2 OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: CPU-v3-v3-v3-v3-v3-v3-v3-v3-v3 - Model: shufflenet-v2 3070 4090 rep RTX 3070 Ti 4090 nv 4090 3090 rep 3090 2 4 6 8 10 SE +/- 0.20, N = 15 6.30 5.18 3.92 3.47 3.37 3.36 3.36 MIN: 3.28 / MAX: 147.57 MIN: 3.45 / MAX: 200.36 MIN: 3.12 / MAX: 496.78 MIN: 3.33 / MAX: 5.01 MIN: 3.25 / MAX: 5.26 MIN: 3.33 / MAX: 3.83 MIN: 3.32 / MAX: 3.82 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread
NCNN Target: CPU-v3-v3-v3-v3-v3-v3-v3-v3-v3 - Model: mnasnet OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: CPU-v3-v3-v3-v3-v3-v3-v3-v3-v3 - Model: mnasnet 3070 4090 rep RTX 3070 Ti 4090 nv 4090 3090 rep 3090 2 4 6 8 10 SE +/- 0.11, N = 15 8.15 3.28 3.25 3.19 3.10 2.98 2.97 MIN: 2.67 / MAX: 317.68 MIN: 3.15 / MAX: 4.32 MIN: 2.68 / MAX: 277.21 MIN: 3.04 / MAX: 3.98 MIN: 2.97 / MAX: 3.92 MIN: 2.94 / MAX: 3.36 MIN: 2.93 / MAX: 3.45 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread
NCNN Target: CPU-v3-v3-v3-v3-v3-v3-v3-v3-v3 - Model: efficientnet-b0 OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: CPU-v3-v3-v3-v3-v3-v3-v3-v3-v3 - Model: efficientnet-b0 3070 nv 4090 RTX 3070 Ti 4090 rep 4090 3090 3090 rep 3 6 9 12 15 SE +/- 0.16, N = 15 9.53 5.88 4.55 4.44 4.14 3.86 3.85 MIN: 3.77 / MAX: 182.53 MIN: 3.96 / MAX: 194.08 MIN: 3.84 / MAX: 379.07 MIN: 4.24 / MAX: 5.18 MIN: 3.93 / MAX: 5.94 MIN: 3.82 / MAX: 4.82 MIN: 3.81 / MAX: 4.6 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread
NCNN Target: CPU-v3-v3-v3-v3-v3-v3-v3-v3-v3 - Model: blazeface OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: CPU-v3-v3-v3-v3-v3-v3-v3-v3-v3 - Model: blazeface 3070 nv 4090 RTX 3070 Ti 4090 3090 4090 rep 3090 rep 0.8033 1.6066 2.4099 3.2132 4.0165 SE +/- 0.14, N = 15 3.57 2.91 1.51 1.45 1.39 1.38 1.38 MIN: 1.08 / MAX: 141.04 MIN: 1.29 / MAX: 113.97 MIN: 1.11 / MAX: 380.46 MIN: 1.38 / MAX: 2.98 MIN: 1.36 / MAX: 3.12 MIN: 1.33 / MAX: 1.98 MIN: 1.35 / MAX: 1.88 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread
NCNN Target: CPU-v3-v3-v3-v3-v3-v3-v3-v3-v3 - Model: googlenet OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: CPU-v3-v3-v3-v3-v3-v3-v3-v3-v3 - Model: googlenet 3070 4090 rep RTX 3070 Ti 4090 nv 4090 3090 3090 rep 5 10 15 20 25 SE +/- 0.22, N = 15 18.66 10.18 9.58 8.91 8.85 7.83 7.82 MIN: 7.42 / MAX: 326.73 MIN: 7.81 / MAX: 204.67 MIN: 7.62 / MAX: 396.9 MIN: 8.3 / MAX: 10.96 MIN: 8.16 / MAX: 10.25 MIN: 7.73 / MAX: 8.6 MIN: 7.72 / MAX: 8.6 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread
NCNN Target: CPU-v3-v3-v3-v3-v3-v3-v3-v3-v3 - Model: vgg16 OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: CPU-v3-v3-v3-v3-v3-v3-v3-v3-v3 - Model: vgg16 3070 nv 4090 4090 rep RTX 3070 Ti 4090 3090 3090 rep 12 24 36 48 60 SE +/- 0.26, N = 15 51.28 29.40 29.35 28.53 27.31 23.50 23.47 MIN: 24.83 / MAX: 242.12 MIN: 26.17 / MAX: 411.51 MIN: 24.55 / MAX: 485.35 MIN: 24.21 / MAX: 515.3 MIN: 24.27 / MAX: 230.86 MIN: 23.26 / MAX: 24.34 MIN: 23.25 / MAX: 24.24 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread
NCNN Target: CPU-v3-v3-v3-v3-v3-v3-v3-v3-v3 - Model: resnet18 OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: CPU-v3-v3-v3-v3-v3-v3-v3-v3-v3 - Model: resnet18 3070 4090 RTX 3070 Ti nv 4090 4090 rep 3090 rep 3090 3 6 9 12 15 SE +/- 0.24, N = 15 13.34 7.78 6.69 5.97 5.84 5.20 5.20 MIN: 5.43 / MAX: 279.86 MIN: 5.4 / MAX: 168.29 MIN: 5.06 / MAX: 462.37 MIN: 5.4 / MAX: 8.25 MIN: 5.35 / MAX: 8.28 MIN: 5.08 / MAX: 6.05 MIN: 5.1 / MAX: 6.05 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread
NCNN Target: CPU-v3-v3-v3-v3-v3-v3-v3-v3-v3 - Model: alexnet OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: CPU-v3-v3-v3-v3-v3-v3-v3-v3-v3 - Model: alexnet 3070 nv 4090 RTX 3070 Ti 4090 rep 4090 3090 rep 3090 3 6 9 12 15 SE +/- 0.21, N = 15 10.69 6.54 5.41 5.16 4.94 4.30 4.30 MIN: 4.32 / MAX: 148.92 MIN: 4.56 / MAX: 110.58 MIN: 4.23 / MAX: 364.66 MIN: 4.73 / MAX: 6.38 MIN: 4.52 / MAX: 6.23 MIN: 4.24 / MAX: 4.85 MIN: 4.25 / MAX: 4.63 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread
NCNN Target: CPU-v3-v3-v3-v3-v3-v3-v3-v3-v3 - Model: resnet50 OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: CPU-v3-v3-v3-v3-v3-v3-v3-v3-v3 - Model: resnet50 3070 nv 4090 4090 RTX 3070 Ti 4090 rep 3090 rep 3090 6 12 18 24 30 SE +/- 0.24, N = 15 23.54 13.46 12.98 12.52 11.24 10.04 10.03 MIN: 10.3 / MAX: 149.49 MIN: 10.6 / MAX: 340.67 MIN: 10.26 / MAX: 145.62 MIN: 9.95 / MAX: 459.05 MIN: 10.22 / MAX: 29.96 MIN: 9.94 / MAX: 10.91 MIN: 9.88 / MAX: 10.86 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread
NCNN Target: CPU-v3-v3-v3-v3-v3-v3-v3-v3-v3 - Model: yolov4-tiny OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: CPU-v3-v3-v3-v3-v3-v3-v3-v3-v3 - Model: yolov4-tiny 3070 4090 rep 4090 nv 4090 RTX 3070 Ti 3090 rep 3090 6 12 18 24 30 SE +/- 0.25, N = 15 26.33 16.60 15.69 15.67 15.56 12.89 12.86 MIN: 12.62 / MAX: 127.32 MIN: 12.98 / MAX: 103.04 MIN: 13.13 / MAX: 187.93 MIN: 12.91 / MAX: 334.44 MIN: 12.24 / MAX: 459.8 MIN: 12.79 / MAX: 13.77 MIN: 12.74 / MAX: 13.68 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread
NCNN Target: CPU-v3-v3-v3-v3-v3-v3-v3-v3-v3 - Model: squeezenet_ssd OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: CPU-v3-v3-v3-v3-v3-v3-v3-v3-v3 - Model: squeezenet_ssd 3070 4090 rep RTX 3070 Ti nv 4090 4090 3090 rep 3090 4 8 12 16 20 SE +/- 0.22, N = 15 15.46 9.34 8.31 7.72 7.40 7.07 7.05 MIN: 7.08 / MAX: 147.31 MIN: 6.88 / MAX: 268.7 MIN: 6.35 / MAX: 364.95 MIN: 7.13 / MAX: 8.97 MIN: 6.81 / MAX: 8.46 MIN: 6.99 / MAX: 7.81 MIN: 6.98 / MAX: 7.81 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread
NCNN Target: CPU-v3-v3-v3-v3-v3-v3-v3-v3-v3 - Model: regnety_400m OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: CPU-v3-v3-v3-v3-v3-v3-v3-v3-v3 - Model: regnety_400m 3070 4090 RTX 3070 Ti 4090 rep nv 4090 3090 3090 rep 4 8 12 16 20 SE +/- 0.20, N = 15 18.24 10.05 9.10 8.45 8.37 8.20 8.19 MIN: 7.5 / MAX: 201.09 MIN: 8.13 / MAX: 173.18 MIN: 7.61 / MAX: 454.62 MIN: 8.05 / MAX: 12.64 MIN: 8.08 / MAX: 10.1 MIN: 8.14 / MAX: 8.74 MIN: 8.12 / MAX: 8.98 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread
NCNN Target: CPU-v3-v3-v3-v3-v3-v3-v3-v3-v3 - Model: vision_transformer OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: CPU-v3-v3-v3-v3-v3-v3-v3-v3-v3 - Model: vision_transformer 3070 4090 4090 rep nv 4090 RTX 3070 Ti 3090 3090 rep 15 30 45 60 75 SE +/- 0.12, N = 15 69.48 38.82 38.69 38.58 38.32 32.16 32.13 MIN: 39.08 / MAX: 374.31 MIN: 33.83 / MAX: 435.6 MIN: 33.32 / MAX: 390.07 MIN: 33.06 / MAX: 464.16 MIN: 32.26 / MAX: 477.15 MIN: 31.94 / MAX: 33.7 MIN: 31.95 / MAX: 32.87 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread
NCNN Target: CPU-v3-v3-v3-v3-v3-v3-v3-v3-v3 - Model: FastestDet OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: CPU-v3-v3-v3-v3-v3-v3-v3-v3-v3 - Model: FastestDet 3070 RTX 3070 Ti 3090 rep 3090 nv 4090 4090 rep 4090 1.008 2.016 3.024 4.032 5.04 SE +/- 0.29, N = 15 4.48 4.25 4.10 4.08 4.01 3.91 2.82 MIN: 2.2 / MAX: 27.6 MIN: 2.46 / MAX: 526.3 MIN: 4.06 / MAX: 4.2 MIN: 4.04 / MAX: 4.2 MIN: 3.87 / MAX: 5.47 MIN: 3.77 / MAX: 5.87 MIN: 2.69 / MAX: 3.5 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread
NCNN Target: CPU-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3 - Model: mobilenet OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: CPU-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3 - Model: mobilenet 3070 nv 4090 4090 RTX 3070 Ti 4090 rep 3090 rep 4 8 12 16 20 SE +/- 0.13, N = 3 17.06 10.64 10.56 10.02 8.22 8.03 MIN: 8 / MAX: 101.45 MIN: 8.4 / MAX: 127.99 MIN: 8.32 / MAX: 239.95 MIN: 7.8 / MAX: 372.36 MIN: 7.75 / MAX: 9.41 MIN: 7.97 / MAX: 8.91 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread
NCNN Target: CPU-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v2-v2 - Model: mobilenet-v2 OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: CPU-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v2-v2 - Model: mobilenet-v2 3070 4090 RTX 3070 Ti 4090 rep nv 4090 3090 rep 1.332 2.664 3.996 5.328 6.66 SE +/- 0.53, N = 3 5.92 4.75 3.83 3.38 3.29 3.15 MIN: 3.16 / MAX: 103.24 MIN: 2.93 / MAX: 147.66 MIN: 3.11 / MAX: 343.21 MIN: 3.2 / MAX: 4 MIN: 3.12 / MAX: 4.27 MIN: 3.1 / MAX: 3.75 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread
NCNN Target: CPU-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3 - Model: mobilenet-v3 OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: CPU-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3 - Model: mobilenet-v3 3070 nv 4090 4090 4090 rep RTX 3070 Ti 3090 rep 2 4 6 8 10 SE +/- 0.04, N = 3 7.34 4.96 3.36 3.35 3.24 3.19 MIN: 3.09 / MAX: 155.33 MIN: 3.14 / MAX: 189.43 MIN: 3.22 / MAX: 4.62 MIN: 3.22 / MAX: 3.99 MIN: 3.05 / MAX: 5.14 MIN: 3.13 / MAX: 3.61 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread
NCNN Target: CPU-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3 - Model: shufflenet-v2 OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: CPU-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3 - Model: shufflenet-v2 3070 4090 rep 4090 RTX 3070 Ti nv 4090 3090 rep 1.3253 2.6506 3.9759 5.3012 6.6265 SE +/- 0.02, N = 3 5.89 5.23 3.56 3.48 3.43 3.32 MIN: 3.19 / MAX: 97.88 MIN: 3.34 / MAX: 185.57 MIN: 3.43 / MAX: 4.24 MIN: 3.33 / MAX: 5.22 MIN: 3.29 / MAX: 5.31 MIN: 3.29 / MAX: 3.62 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread
NCNN Target: CPU-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3 - Model: mnasnet OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: CPU-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3 - Model: mnasnet 3070 4090 4090 rep RTX 3070 Ti nv 4090 3090 rep 2 4 6 8 10 SE +/- 0.02, N = 3 8.55 3.23 3.12 3.12 3.10 2.96 MIN: 2.99 / MAX: 185.5 MIN: 3.08 / MAX: 4.73 MIN: 3 / MAX: 4.1 MIN: 2.97 / MAX: 4.65 MIN: 2.97 / MAX: 3.73 MIN: 2.92 / MAX: 3.27 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread
NCNN Target: CPU-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3 - Model: efficientnet-b0 OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: CPU-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3 - Model: efficientnet-b0 3070 nv 4090 4090 RTX 3070 Ti 4090 rep 3090 rep 2 4 6 8 10 SE +/- 0.08, N = 3 6.63 5.82 4.63 4.17 4.10 3.84 MIN: 3.75 / MAX: 22.34 MIN: 3.98 / MAX: 197.79 MIN: 4.38 / MAX: 6.01 MIN: 3.86 / MAX: 5.52 MIN: 3.88 / MAX: 5.04 MIN: 3.8 / MAX: 4.67 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread
NCNN Target: CPU-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3 - Model: blazeface OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: CPU-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3 - Model: blazeface 3070 4090 rep nv 4090 RTX 3070 Ti 3090 rep 4090 0.6053 1.2106 1.8159 2.4212 3.0265 SE +/- 0.04, N = 3 2.69 1.42 1.40 1.40 1.38 1.35 MIN: 1.35 / MAX: 48.81 MIN: 1.36 / MAX: 2.03 MIN: 1.34 / MAX: 1.86 MIN: 1.28 / MAX: 1.91 MIN: 1.36 / MAX: 1.73 MIN: 1.28 / MAX: 1.84 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread
NCNN Target: CPU-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3 - Model: googlenet OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: CPU-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3 - Model: googlenet 3070 4090 rep nv 4090 RTX 3070 Ti 4090 3090 rep 5 10 15 20 25 SE +/- 0.55, N = 3 18.80 10.47 10.14 9.97 8.55 7.86 MIN: 7.78 / MAX: 141.46 MIN: 7.86 / MAX: 191.94 MIN: 7.85 / MAX: 257.61 MIN: 8.16 / MAX: 381.49 MIN: 7.85 / MAX: 11.39 MIN: 7.75 / MAX: 8.57 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread
NCNN Target: CPU-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3 - Model: vgg16 OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: CPU-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3 - Model: vgg16 3070 4090 rep RTX 3070 Ti 4090 nv 4090 3090 rep 12 24 36 48 60 SE +/- 0.28, N = 3 53.48 29.85 27.86 27.32 27.25 23.72 MIN: 25.52 / MAX: 296.52 MIN: 24.25 / MAX: 400.86 MIN: 24.17 / MAX: 416.36 MIN: 24.36 / MAX: 262.38 MIN: 24.12 / MAX: 252.53 MIN: 23.56 / MAX: 24.59 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread
NCNN Target: CPU-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3 - Model: resnet18 OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: CPU-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3 - Model: resnet18 3070 4090 RTX 3070 Ti 4090 rep nv 4090 3090 rep 3 6 9 12 15 SE +/- 0.05, N = 3 12.13 6.96 5.94 5.87 5.58 5.27 MIN: 5.32 / MAX: 123.4 MIN: 5.3 / MAX: 242.18 MIN: 5.32 / MAX: 8.32 MIN: 5.41 / MAX: 7.58 MIN: 5.09 / MAX: 6.98 MIN: 5.15 / MAX: 6.11 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread
NCNN Target: CPU-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3 - Model: alexnet OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: CPU-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3 - Model: alexnet 3070 nv 4090 RTX 3070 Ti 4090 rep 4090 3090 rep 3 6 9 12 15 SE +/- 0.57, N = 3 11.43 6.32 6.25 5.34 5.14 4.31 MIN: 4.24 / MAX: 178.83 MIN: 4.26 / MAX: 195.95 MIN: 4.27 / MAX: 334.55 MIN: 4.87 / MAX: 6.57 MIN: 4.75 / MAX: 7.34 MIN: 4.26 / MAX: 4.83 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread
NCNN Target: CPU-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3 - Model: resnet50 OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: CPU-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3 - Model: resnet50 3070 nv 4090 RTX 3070 Ti 4090 4090 rep 3090 rep 5 10 15 20 25 SE +/- 0.30, N = 3 22.15 13.25 13.15 13.00 10.96 10.27 MIN: 10.11 / MAX: 123.04 MIN: 10.61 / MAX: 154.12 MIN: 10.26 / MAX: 349.93 MIN: 10.34 / MAX: 397.57 MIN: 10.09 / MAX: 12.99 MIN: 10.12 / MAX: 11.19 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread
NCNN Target: CPU-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3 - Model: yolov4-tiny OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: CPU-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3 - Model: yolov4-tiny 3070 nv 4090 4090 4090 rep RTX 3070 Ti 3090 rep 7 14 21 28 35 SE +/- 0.94, N = 3 29.38 16.30 16.05 15.41 14.64 12.92 MIN: 12.95 / MAX: 201.31 MIN: 14.11 / MAX: 184.46 MIN: 12.93 / MAX: 474.03 MIN: 12.75 / MAX: 226.87 MIN: 12.77 / MAX: 383.28 MIN: 12.79 / MAX: 18.5 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread
NCNN Target: CPU-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3 - Model: squeezenet_ssd OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: CPU-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3 - Model: squeezenet_ssd 3070 4090 rep nv 4090 RTX 3070 Ti 4090 3090 rep 4 8 12 16 20 SE +/- 0.14, N = 3 15.32 9.44 8.26 7.45 7.43 7.07 MIN: 6.66 / MAX: 139.17 MIN: 7.17 / MAX: 94.63 MIN: 7.64 / MAX: 11.08 MIN: 6.59 / MAX: 9.11 MIN: 6.84 / MAX: 8.82 MIN: 6.98 / MAX: 9.71 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread
NCNN Target: CPU-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3 - Model: regnety_400m OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: CPU-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3 - Model: regnety_400m 3070 4090 RTX 3070 Ti 4090 rep nv 4090 3090 rep 4 8 12 16 20 SE +/- 0.54, N = 3 17.02 10.11 9.14 8.70 8.34 8.06 MIN: 7.65 / MAX: 216.63 MIN: 8.03 / MAX: 259.38 MIN: 8.14 / MAX: 400.02 MIN: 8.29 / MAX: 12.6 MIN: 8.01 / MAX: 12.36 MIN: 7.98 / MAX: 8.6 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread
NCNN Target: CPU-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3 - Model: vision_transformer OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: CPU-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3 - Model: vision_transformer 3070 4090 4090 rep RTX 3070 Ti nv 4090 3090 rep 16 32 48 64 80 SE +/- 0.10, N = 3 70.53 39.35 38.65 38.50 37.13 31.94 MIN: 39.2 / MAX: 276.33 MIN: 34.22 / MAX: 466.65 MIN: 33.07 / MAX: 476.08 MIN: 33.7 / MAX: 418.06 MIN: 33.97 / MAX: 443.1 MIN: 31.73 / MAX: 32.75 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread
NCNN Target: CPU-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3 - Model: FastestDet OpenBenchmarking.org ms, Fewer Is Better NCNN 20230517 Target: CPU-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3-v3 - Model: FastestDet 3070 nv 4090 4090 4090 rep RTX 3070 Ti 3090 rep 2 4 6 8 10 SE +/- 0.15, N = 3 7.23 5.86 4.62 4.59 4.14 4.07 MIN: 3.75 / MAX: 121.71 MIN: 3.9 / MAX: 190.17 MIN: 4.48 / MAX: 5.16 MIN: 4.44 / MAX: 5.2 MIN: 3.73 / MAX: 5.07 MIN: 4.04 / MAX: 4.25 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread
Phoronix Test Suite v10.8.5