ncnn mnn 2022 AMD Ryzen 9 5950X 16-Core testing with a ASUS ROG CROSSHAIR VIII HERO (WI-FI) (4006 BIOS) and AMD Radeon RX 6700/6700 XT / 6800M on Ubuntu 22.04 via the Phoronix Test Suite.
HTML result view exported from: https://openbenchmarking.org/result/2208131-PTS-NCNNMNN216&sor&grs .
ncnn mnn 2022 Processor Motherboard Chipset Memory Disk Graphics Audio Monitor Network OS Kernel Desktop Display Server OpenGL Vulkan Compiler File-System Screen Resolution A B C D E AMD Ryzen 9 5950X 16-Core @ 3.40GHz (16 Cores / 32 Threads) ASUS ROG CROSSHAIR VIII HERO (WI-FI) (4006 BIOS) AMD Starship/Matisse 32GB 1000GB Sabrent Rocket 4.0 Plus AMD Radeon RX 6700/6700 XT / 6800M (2880/1124MHz) AMD Navi 21 HDMI Audio ASUS MG28U Realtek RTL8125 2.5GbE + Intel I211 + Intel Wi-Fi 6 AX200 Ubuntu 22.04 5.15.0-46-generic (x86_64) GNOME Shell 42.2 X Server 1.21.1.3 + Wayland 4.6 Mesa 22.0.1 (LLVM 13.0.1 DRM 3.42) 1.3.204 GCC 11.2.0 ext4 3840x2160 OpenBenchmarking.org Kernel Details - Transparent Huge Pages: madvise Compiler Details - --build=x86_64-linux-gnu --disable-vtable-verify --disable-werror --enable-bootstrap --enable-cet --enable-checking=release --enable-clocale=gnu --enable-default-pie --enable-gnu-unique-object --enable-languages=c,ada,c++,go,brig,d,fortran,objc,obj-c++,m2 --enable-libphobos-checking=release --enable-libstdcxx-debug --enable-libstdcxx-time=yes --enable-link-serialization=2 --enable-multiarch --enable-multilib --enable-nls --enable-objc-gc=auto --enable-offload-targets=nvptx-none=/build/gcc-11-gBFGDP/gcc-11-11.2.0/debian/tmp-nvptx/usr,amdgcn-amdhsa=/build/gcc-11-gBFGDP/gcc-11-11.2.0/debian/tmp-gcn/usr --enable-plugin --enable-shared --enable-threads=posix --host=x86_64-linux-gnu --program-prefix=x86_64-linux-gnu- --target=x86_64-linux-gnu --with-abi=m64 --with-arch-32=i686 --with-build-config=bootstrap-lto-lean --with-default-libstdcxx-abi=new --with-gcc-major-version-only --with-multilib-list=m32,m64,mx32 --with-target-system-zlib=auto --with-tune=generic --without-cuda-driver -v Processor Details - Scaling Governor: acpi-cpufreq schedutil (Boost: Enabled) - CPU Microcode: 0xa201016 Security Details - itlb_multihit: Not affected + l1tf: Not affected + mds: Not affected + meltdown: Not affected + mmio_stale_data: Not affected + retbleed: Not affected + spec_store_bypass: Mitigation of SSB disabled via prctl and seccomp + spectre_v1: Mitigation of usercopy/swapgs barriers and __user pointer sanitization + spectre_v2: Mitigation of Retpolines IBPB: conditional IBRS_FW STIBP: always-on RSB filling + srbds: Not affected + tsx_async_abort: Not affected
ncnn mnn 2022 ncnn: CPU - googlenet mnn: SqueezeNetV1.0 ncnn: CPU - blazeface mnn: resnet-v2-50 ncnn: Vulkan GPU - blazeface ncnn: Vulkan GPU - mnasnet ncnn: Vulkan GPU - resnet18 ncnn: CPU - resnet50 mnn: mobilenet-v1-1.0 ncnn: Vulkan GPU-v3-v3 - mobilenet-v3 mnn: mobilenetV3 mnn: squeezenetv1.1 ncnn: Vulkan GPU - vgg16 ncnn: CPU - squeezenet_ssd ncnn: CPU - mnasnet ncnn: CPU - efficientnet-b0 ncnn: Vulkan GPU - googlenet ncnn: CPU-v3-v3 - mobilenet-v3 mnn: MobileNetV2_224 ncnn: Vulkan GPU - efficientnet-b0 ncnn: CPU - yolov4-tiny ncnn: CPU - vgg16 ncnn: Vulkan GPU - resnet50 ncnn: CPU-v2-v2 - mobilenet-v2 ncnn: CPU - resnet18 mnn: inception-v3 ncnn: CPU - vision_transformer ncnn: CPU - shufflenet-v2 ncnn: CPU - regnety_400m ncnn: Vulkan GPU - vision_transformer ncnn: Vulkan GPU - FastestDet ncnn: Vulkan GPU - regnety_400m ncnn: Vulkan GPU - squeezenet_ssd ncnn: Vulkan GPU - yolov4-tiny ncnn: Vulkan GPU - alexnet ncnn: Vulkan GPU - shufflenet-v2 ncnn: Vulkan GPU-v2-v2 - mobilenet-v2 ncnn: Vulkan GPU - mobilenet ncnn: CPU - FastestDet ncnn: CPU - alexnet ncnn: CPU - mobilenet A B C D E 12.19 5.234 1.82 21.956 1.64 1.97 2.63 21.47 2.625 2.35 1.903 3.209 6.77 18.32 3.89 5.91 3.67 3.78 3.449 4.93 21.51 47.58 4.99 4.29 12.17 26.074 123.55 4.30 12.83 220.75 2.50 3.11 5.23 14.58 2.08 2.04 1.92 9.45 4.98 7.79 11.46 11.69 5.144 1.80 21.564 1.60 2.04 2.54 20.98 2.542 2.36 1.860 3.227 6.93 18.53 3.97 5.97 3.71 3.82 3.422 4.89 21.33 47.52 4.94 4.33 12.21 25.806 122.89 4.30 12.81 219.90 2.28 3.03 5.14 14.48 2.24 2.12 1.95 9.34 4.90 7.73 11.92 11.38 5.350 1.82 21.098 1.65 2.01 2.60 21.70 2.556 2.34 1.865 3.278 6.75 18.59 3.89 5.90 3.66 3.77 3.468 4.90 21.57 47.58 5.00 4.28 12.08 25.905 122.63 4.30 12.90 219.57 2.33 3.09 5.47 14.56 2.09 2.09 1.99 9.84 4.93 7.67 11.71 11.56 5.302 1.88 21.704 1.64 2.04 2.61 21.48 2.592 2.36 1.913 3.298 6.75 18.39 3.90 6.02 3.71 3.77 3.456 4.86 21.30 47.34 5.00 4.31 12.16 25.924 123.87 4.26 12.85 220.31 2.38 3.08 5.50 14.59 2.08 2.09 1.98 9.90 4.86 7.72 11.84 11.53 5.382 1.83 21.536 1.66 2.04 2.62 21.39 2.581 2.41 1.902 3.296 6.80 18.17 3.90 5.90 3.72 3.76 3.472 4.86 21.43 47.94 4.97 4.30 12.14 25.990 123.44 4.28 12.79 220.85 2.37 3.06 5.55 14.77 2.12 2.09 2.00 10.05 4.84 9.30 11.64 OpenBenchmarking.org
NCNN Target: CPU - Model: googlenet OpenBenchmarking.org ms, Fewer Is Better NCNN 20220729 Target: CPU - Model: googlenet C E D B A 3 6 9 12 15 SE +/- 0.01, N = 3 SE +/- 0.11, N = 3 SE +/- 0.14, N = 3 SE +/- 0.10, N = 15 SE +/- 0.31, N = 3 11.38 11.53 11.56 11.69 12.19 MIN: 10.66 / MAX: 20.39 MIN: 10.64 / MAX: 33.91 MIN: 10.58 / MAX: 20.15 MIN: 10.57 / MAX: 54.38 MIN: 10.69 / MAX: 42.59 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread
Mobile Neural Network Model: SqueezeNetV1.0 OpenBenchmarking.org ms, Fewer Is Better Mobile Neural Network 2.0 Model: SqueezeNetV1.0 B A D C E 1.211 2.422 3.633 4.844 6.055 SE +/- 0.059, N = 3 SE +/- 0.029, N = 3 SE +/- 0.047, N = 15 SE +/- 0.070, N = 3 SE +/- 0.042, N = 15 5.144 5.234 5.302 5.350 5.382 MIN: 4.8 / MAX: 17.58 MIN: 4.93 / MAX: 13.77 MIN: 4.58 / MAX: 14.4 MIN: 5.01 / MAX: 13.89 MIN: 4.9 / MAX: 40.68 1. (CXX) g++ options: -std=c++11 -O3 -fvisibility=hidden -fomit-frame-pointer -fstrict-aliasing -ffunction-sections -fdata-sections -ffast-math -fno-rtti -fno-exceptions -rdynamic -pthread -ldl
NCNN Target: CPU - Model: blazeface OpenBenchmarking.org ms, Fewer Is Better NCNN 20220729 Target: CPU - Model: blazeface B A C E D 0.423 0.846 1.269 1.692 2.115 SE +/- 0.01, N = 15 SE +/- 0.01, N = 3 SE +/- 0.00, N = 3 SE +/- 0.01, N = 3 SE +/- 0.04, N = 3 1.80 1.82 1.82 1.83 1.88 MIN: 1.63 / MAX: 9.64 MIN: 1.7 / MAX: 9.71 MIN: 1.71 / MAX: 4.41 MIN: 1.69 / MAX: 10.06 MIN: 1.68 / MAX: 49.69 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread
Mobile Neural Network Model: resnet-v2-50 OpenBenchmarking.org ms, Fewer Is Better Mobile Neural Network 2.0 Model: resnet-v2-50 C E B D A 5 10 15 20 25 SE +/- 0.14, N = 3 SE +/- 0.08, N = 15 SE +/- 0.49, N = 3 SE +/- 0.10, N = 15 SE +/- 0.06, N = 3 21.10 21.54 21.56 21.70 21.96 MIN: 19.73 / MAX: 46.88 MIN: 19.51 / MAX: 34.69 MIN: 19.55 / MAX: 34.8 MIN: 19.47 / MAX: 39.03 MIN: 19.8 / MAX: 82.59 1. (CXX) g++ options: -std=c++11 -O3 -fvisibility=hidden -fomit-frame-pointer -fstrict-aliasing -ffunction-sections -fdata-sections -ffast-math -fno-rtti -fno-exceptions -rdynamic -pthread -ldl
NCNN Target: Vulkan GPU - Model: blazeface OpenBenchmarking.org ms, Fewer Is Better NCNN 20220729 Target: Vulkan GPU - Model: blazeface B A D C E 0.3735 0.747 1.1205 1.494 1.8675 SE +/- 0.02, N = 3 SE +/- 0.04, N = 3 SE +/- 0.01, N = 15 SE +/- 0.01, N = 15 SE +/- 0.02, N = 15 1.60 1.64 1.64 1.65 1.66 MIN: 1.25 / MAX: 10.18 MIN: 1.18 / MAX: 11.1 MIN: 1.16 / MAX: 14.86 MIN: 1.15 / MAX: 12.54 MIN: 1.15 / MAX: 22.72 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread
NCNN Target: Vulkan GPU - Model: mnasnet OpenBenchmarking.org ms, Fewer Is Better NCNN 20220729 Target: Vulkan GPU - Model: mnasnet A C B D E 0.459 0.918 1.377 1.836 2.295 SE +/- 0.02, N = 3 SE +/- 0.01, N = 15 SE +/- 0.03, N = 3 SE +/- 0.02, N = 13 SE +/- 0.02, N = 14 1.97 2.01 2.04 2.04 2.04 MIN: 1.73 / MAX: 6.43 MIN: 1.73 / MAX: 9.57 MIN: 1.74 / MAX: 9.58 MIN: 1.73 / MAX: 14.09 MIN: 1.73 / MAX: 11.29 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread
NCNN Target: Vulkan GPU - Model: resnet18 OpenBenchmarking.org ms, Fewer Is Better NCNN 20220729 Target: Vulkan GPU - Model: resnet18 B C D E A 0.5918 1.1836 1.7754 2.3672 2.959 SE +/- 0.02, N = 3 SE +/- 0.02, N = 15 SE +/- 0.03, N = 15 SE +/- 0.03, N = 13 SE +/- 0.03, N = 3 2.54 2.60 2.61 2.62 2.63 MIN: 2.16 / MAX: 11.84 MIN: 2.15 / MAX: 15.22 MIN: 2.15 / MAX: 21.7 MIN: 2.15 / MAX: 20.98 MIN: 2.15 / MAX: 16.21 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread
NCNN Target: CPU - Model: resnet50 OpenBenchmarking.org ms, Fewer Is Better NCNN 20220729 Target: CPU - Model: resnet50 B E A D C 5 10 15 20 25 SE +/- 0.11, N = 15 SE +/- 0.19, N = 3 SE +/- 0.28, N = 3 SE +/- 0.10, N = 3 SE +/- 0.15, N = 3 20.98 21.39 21.47 21.48 21.70 MIN: 18.95 / MAX: 79.65 MIN: 19.66 / MAX: 37.46 MIN: 19.56 / MAX: 30.76 MIN: 19.75 / MAX: 39.98 MIN: 19.94 / MAX: 30.5 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread
Mobile Neural Network Model: mobilenet-v1-1.0 OpenBenchmarking.org ms, Fewer Is Better Mobile Neural Network 2.0 Model: mobilenet-v1-1.0 B C E D A 0.5906 1.1812 1.7718 2.3624 2.953 SE +/- 0.018, N = 3 SE +/- 0.031, N = 3 SE +/- 0.018, N = 15 SE +/- 0.025, N = 15 SE +/- 0.021, N = 3 2.542 2.556 2.581 2.592 2.625 MIN: 2.36 / MAX: 11.09 MIN: 2.32 / MAX: 11.13 MIN: 2.32 / MAX: 11.34 MIN: 2.32 / MAX: 11.27 MIN: 2.41 / MAX: 11.15 1. (CXX) g++ options: -std=c++11 -O3 -fvisibility=hidden -fomit-frame-pointer -fstrict-aliasing -ffunction-sections -fdata-sections -ffast-math -fno-rtti -fno-exceptions -rdynamic -pthread -ldl
NCNN Target: Vulkan GPU-v3-v3 - Model: mobilenet-v3 OpenBenchmarking.org ms, Fewer Is Better NCNN 20220729 Target: Vulkan GPU-v3-v3 - Model: mobilenet-v3 C A B D E 0.5423 1.0846 1.6269 2.1692 2.7115 SE +/- 0.01, N = 15 SE +/- 0.02, N = 3 SE +/- 0.02, N = 3 SE +/- 0.01, N = 15 SE +/- 0.03, N = 15 2.34 2.35 2.36 2.36 2.41 MIN: 2.07 / MAX: 14.62 MIN: 2.07 / MAX: 8.2 MIN: 2.09 / MAX: 9.49 MIN: 2.08 / MAX: 15.13 MIN: 2.07 / MAX: 19.36 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread
Mobile Neural Network Model: mobilenetV3 OpenBenchmarking.org ms, Fewer Is Better Mobile Neural Network 2.0 Model: mobilenetV3 B C E A D 0.4304 0.8608 1.2912 1.7216 2.152 SE +/- 0.010, N = 3 SE +/- 0.014, N = 3 SE +/- 0.015, N = 15 SE +/- 0.020, N = 3 SE +/- 0.015, N = 15 1.860 1.865 1.902 1.903 1.913 MIN: 1.74 / MAX: 11.33 MIN: 1.74 / MAX: 10.98 MIN: 1.69 / MAX: 11.39 MIN: 1.71 / MAX: 13.23 MIN: 1.69 / MAX: 11.76 1. (CXX) g++ options: -std=c++11 -O3 -fvisibility=hidden -fomit-frame-pointer -fstrict-aliasing -ffunction-sections -fdata-sections -ffast-math -fno-rtti -fno-exceptions -rdynamic -pthread -ldl
Mobile Neural Network Model: squeezenetv1.1 OpenBenchmarking.org ms, Fewer Is Better Mobile Neural Network 2.0 Model: squeezenetv1.1 A B C E D 0.7421 1.4842 2.2263 2.9684 3.7105 SE +/- 0.079, N = 3 SE +/- 0.058, N = 3 SE +/- 0.046, N = 3 SE +/- 0.028, N = 15 SE +/- 0.036, N = 15 3.209 3.227 3.278 3.296 3.298 MIN: 2.96 / MAX: 11.86 MIN: 2.95 / MAX: 11.74 MIN: 3.03 / MAX: 11.65 MIN: 2.88 / MAX: 12.56 MIN: 2.83 / MAX: 11.8 1. (CXX) g++ options: -std=c++11 -O3 -fvisibility=hidden -fomit-frame-pointer -fstrict-aliasing -ffunction-sections -fdata-sections -ffast-math -fno-rtti -fno-exceptions -rdynamic -pthread -ldl
NCNN Target: Vulkan GPU - Model: vgg16 OpenBenchmarking.org ms, Fewer Is Better NCNN 20220729 Target: Vulkan GPU - Model: vgg16 C D A E B 2 4 6 8 10 SE +/- 0.03, N = 15 SE +/- 0.02, N = 15 SE +/- 0.05, N = 3 SE +/- 0.02, N = 15 SE +/- 0.06, N = 3 6.75 6.75 6.77 6.80 6.93 MIN: 6.25 / MAX: 36.3 MIN: 6.25 / MAX: 28.85 MIN: 6.25 / MAX: 25.32 MIN: 6.24 / MAX: 30.4 MIN: 6.25 / MAX: 28.94 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread
NCNN Target: CPU - Model: squeezenet_ssd OpenBenchmarking.org ms, Fewer Is Better NCNN 20220729 Target: CPU - Model: squeezenet_ssd E A D B C 5 10 15 20 25 SE +/- 0.19, N = 3 SE +/- 0.19, N = 3 SE +/- 0.17, N = 3 SE +/- 0.13, N = 15 SE +/- 0.14, N = 3 18.17 18.32 18.39 18.53 18.59 MIN: 16.13 / MAX: 27.12 MIN: 15.5 / MAX: 34.97 MIN: 15.88 / MAX: 27.52 MIN: 15.19 / MAX: 66.5 MIN: 16.26 / MAX: 43.69 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread
NCNN Target: CPU - Model: mnasnet OpenBenchmarking.org ms, Fewer Is Better NCNN 20220729 Target: CPU - Model: mnasnet A C D E B 0.8933 1.7866 2.6799 3.5732 4.4665 SE +/- 0.01, N = 3 SE +/- 0.01, N = 3 SE +/- 0.02, N = 3 SE +/- 0.01, N = 3 SE +/- 0.05, N = 15 3.89 3.89 3.90 3.90 3.97 MIN: 3.68 / MAX: 11.81 MIN: 3.69 / MAX: 11.45 MIN: 3.68 / MAX: 11.86 MIN: 3.65 / MAX: 24.96 MIN: 3.69 / MAX: 14.19 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread
NCNN Target: CPU - Model: efficientnet-b0 OpenBenchmarking.org ms, Fewer Is Better NCNN 20220729 Target: CPU - Model: efficientnet-b0 C E A B D 2 4 6 8 10 SE +/- 0.01, N = 3 SE +/- 0.01, N = 3 SE +/- 0.02, N = 3 SE +/- 0.03, N = 15 SE +/- 0.08, N = 3 5.90 5.90 5.91 5.97 6.02 MIN: 5.59 / MAX: 14.02 MIN: 5.53 / MAX: 32.65 MIN: 5.57 / MAX: 17 MIN: 5.56 / MAX: 78.45 MIN: 5.56 / MAX: 47.25 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread
NCNN Target: Vulkan GPU - Model: googlenet OpenBenchmarking.org ms, Fewer Is Better NCNN 20220729 Target: Vulkan GPU - Model: googlenet C A B D E 0.837 1.674 2.511 3.348 4.185 SE +/- 0.03, N = 15 SE +/- 0.01, N = 3 SE +/- 0.08, N = 3 SE +/- 0.04, N = 15 SE +/- 0.03, N = 15 3.66 3.67 3.71 3.71 3.72 MIN: 3.24 / MAX: 16.69 MIN: 3.25 / MAX: 17.16 MIN: 3.26 / MAX: 16.59 MIN: 3.24 / MAX: 25.35 MIN: 3.24 / MAX: 18.21 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread
NCNN Target: CPU-v3-v3 - Model: mobilenet-v3 OpenBenchmarking.org ms, Fewer Is Better NCNN 20220729 Target: CPU-v3-v3 - Model: mobilenet-v3 E C D A B 0.8595 1.719 2.5785 3.438 4.2975 SE +/- 0.01, N = 3 SE +/- 0.01, N = 3 SE +/- 0.01, N = 3 SE +/- 0.00, N = 3 SE +/- 0.04, N = 15 3.76 3.77 3.77 3.78 3.82 MIN: 3.56 / MAX: 12.52 MIN: 3.56 / MAX: 11.78 MIN: 3.55 / MAX: 11.83 MIN: 3.56 / MAX: 11.69 MIN: 3.55 / MAX: 35.21 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread
Mobile Neural Network Model: MobileNetV2_224 OpenBenchmarking.org ms, Fewer Is Better Mobile Neural Network 2.0 Model: MobileNetV2_224 B A D C E 0.7812 1.5624 2.3436 3.1248 3.906 SE +/- 0.019, N = 3 SE +/- 0.011, N = 3 SE +/- 0.026, N = 15 SE +/- 0.008, N = 3 SE +/- 0.024, N = 15 3.422 3.449 3.456 3.468 3.472 MIN: 3.2 / MAX: 12.93 MIN: 3.24 / MAX: 13.32 MIN: 3.05 / MAX: 17.71 MIN: 3.25 / MAX: 13.17 MIN: 3.04 / MAX: 13.37 1. (CXX) g++ options: -std=c++11 -O3 -fvisibility=hidden -fomit-frame-pointer -fstrict-aliasing -ffunction-sections -fdata-sections -ffast-math -fno-rtti -fno-exceptions -rdynamic -pthread -ldl
NCNN Target: Vulkan GPU - Model: efficientnet-b0 OpenBenchmarking.org ms, Fewer Is Better NCNN 20220729 Target: Vulkan GPU - Model: efficientnet-b0 D E B C A 1.1093 2.2186 3.3279 4.4372 5.5465 SE +/- 0.01, N = 15 SE +/- 0.01, N = 15 SE +/- 0.05, N = 3 SE +/- 0.02, N = 15 SE +/- 0.04, N = 3 4.86 4.86 4.89 4.90 4.93 MIN: 4.49 / MAX: 18.62 MIN: 4.52 / MAX: 18.87 MIN: 4.53 / MAX: 27.46 MIN: 4.48 / MAX: 29.81 MIN: 4.52 / MAX: 29.24 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread
NCNN Target: CPU - Model: yolov4-tiny OpenBenchmarking.org ms, Fewer Is Better NCNN 20220729 Target: CPU - Model: yolov4-tiny D B E A C 5 10 15 20 25 SE +/- 0.04, N = 3 SE +/- 0.09, N = 15 SE +/- 0.14, N = 3 SE +/- 0.14, N = 3 SE +/- 0.14, N = 3 21.30 21.33 21.43 21.51 21.57 MIN: 19.44 / MAX: 33.91 MIN: 19.29 / MAX: 47.85 MIN: 19.88 / MAX: 29.43 MIN: 19.61 / MAX: 83.91 MIN: 19.56 / MAX: 36.88 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread
NCNN Target: CPU - Model: vgg16 OpenBenchmarking.org ms, Fewer Is Better NCNN 20220729 Target: CPU - Model: vgg16 D B A C E 11 22 33 44 55 SE +/- 0.08, N = 3 SE +/- 0.12, N = 15 SE +/- 0.13, N = 3 SE +/- 0.04, N = 3 SE +/- 0.40, N = 3 47.34 47.52 47.58 47.58 47.94 MIN: 43.83 / MAX: 71.03 MIN: 43.88 / MAX: 109.37 MIN: 44.3 / MAX: 69.79 MIN: 44.41 / MAX: 87.25 MIN: 44.37 / MAX: 85.57 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread
NCNN Target: Vulkan GPU - Model: resnet50 OpenBenchmarking.org ms, Fewer Is Better NCNN 20220729 Target: Vulkan GPU - Model: resnet50 B E A C D 1.125 2.25 3.375 4.5 5.625 SE +/- 0.03, N = 3 SE +/- 0.03, N = 15 SE +/- 0.02, N = 3 SE +/- 0.03, N = 15 SE +/- 0.04, N = 15 4.94 4.97 4.99 5.00 5.00 MIN: 4.5 / MAX: 26.23 MIN: 4.49 / MAX: 30.82 MIN: 4.5 / MAX: 26.58 MIN: 4.49 / MAX: 33.3 MIN: 4.5 / MAX: 30.89 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread
NCNN Target: CPU-v2-v2 - Model: mobilenet-v2 OpenBenchmarking.org ms, Fewer Is Better NCNN 20220729 Target: CPU-v2-v2 - Model: mobilenet-v2 C A E D B 0.9743 1.9486 2.9229 3.8972 4.8715 SE +/- 0.01, N = 3 SE +/- 0.01, N = 3 SE +/- 0.01, N = 3 SE +/- 0.02, N = 3 SE +/- 0.04, N = 15 4.28 4.29 4.30 4.31 4.33 MIN: 3.95 / MAX: 12.13 MIN: 3.94 / MAX: 12.29 MIN: 3.94 / MAX: 12.37 MIN: 3.94 / MAX: 16.66 MIN: 3.93 / MAX: 33.03 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread
NCNN Target: CPU - Model: resnet18 OpenBenchmarking.org ms, Fewer Is Better NCNN 20220729 Target: CPU - Model: resnet18 C E D A B 3 6 9 12 15 SE +/- 0.04, N = 3 SE +/- 0.08, N = 3 SE +/- 0.17, N = 3 SE +/- 0.02, N = 3 SE +/- 0.05, N = 15 12.08 12.14 12.16 12.17 12.21 MIN: 10.9 / MAX: 20.66 MIN: 10.93 / MAX: 29.46 MIN: 10.93 / MAX: 21.19 MIN: 11.06 / MAX: 20.42 MIN: 10.82 / MAX: 25.49 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread
Mobile Neural Network Model: inception-v3 OpenBenchmarking.org ms, Fewer Is Better Mobile Neural Network 2.0 Model: inception-v3 B C D E A 6 12 18 24 30 SE +/- 0.07, N = 3 SE +/- 0.05, N = 3 SE +/- 0.08, N = 15 SE +/- 0.06, N = 15 SE +/- 0.18, N = 3 25.81 25.91 25.92 25.99 26.07 MIN: 24.39 / MAX: 40.84 MIN: 24.43 / MAX: 37.78 MIN: 23.04 / MAX: 39.16 MIN: 23.91 / MAX: 39.7 MIN: 24.37 / MAX: 39.27 1. (CXX) g++ options: -std=c++11 -O3 -fvisibility=hidden -fomit-frame-pointer -fstrict-aliasing -ffunction-sections -fdata-sections -ffast-math -fno-rtti -fno-exceptions -rdynamic -pthread -ldl
NCNN Target: CPU - Model: vision_transformer OpenBenchmarking.org ms, Fewer Is Better NCNN 20220729 Target: CPU - Model: vision_transformer C B E A D 30 60 90 120 150 SE +/- 0.17, N = 3 SE +/- 0.11, N = 15 SE +/- 0.61, N = 3 SE +/- 0.06, N = 3 SE +/- 0.36, N = 3 122.63 122.89 123.44 123.55 123.87 MIN: 119.12 / MAX: 162.72 MIN: 119.06 / MAX: 171.55 MIN: 119.37 / MAX: 173.88 MIN: 119.55 / MAX: 191.98 MIN: 119.07 / MAX: 187.38 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread
NCNN Target: CPU - Model: shufflenet-v2 OpenBenchmarking.org ms, Fewer Is Better NCNN 20220729 Target: CPU - Model: shufflenet-v2 D E A B C 0.9675 1.935 2.9025 3.87 4.8375 SE +/- 0.01, N = 3 SE +/- 0.01, N = 3 SE +/- 0.03, N = 2 SE +/- 0.01, N = 14 SE +/- 0.01, N = 3 4.26 4.28 4.30 4.30 4.30 MIN: 4.01 / MAX: 12.04 MIN: 4.01 / MAX: 12.08 MIN: 4.03 / MAX: 12.41 MIN: 3.97 / MAX: 35.81 MIN: 4.07 / MAX: 12.2 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread
NCNN Target: CPU - Model: regnety_400m OpenBenchmarking.org ms, Fewer Is Better NCNN 20220729 Target: CPU - Model: regnety_400m E B A D C 3 6 9 12 15 SE +/- 0.10, N = 3 SE +/- 0.05, N = 15 SE +/- 0.04, N = 3 SE +/- 0.03, N = 3 SE +/- 0.04, N = 3 12.79 12.81 12.83 12.85 12.90 MIN: 11.97 / MAX: 21.29 MIN: 11.76 / MAX: 25.73 MIN: 12.11 / MAX: 26.51 MIN: 12.05 / MAX: 26.05 MIN: 12.14 / MAX: 20.46 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread
NCNN Target: Vulkan GPU - Model: vision_transformer OpenBenchmarking.org ms, Fewer Is Better NCNN 20220729 Target: Vulkan GPU - Model: vision_transformer C B D A E 50 100 150 200 250 SE +/- 0.33, N = 15 SE +/- 1.44, N = 3 SE +/- 0.34, N = 15 SE +/- 0.44, N = 3 SE +/- 0.36, N = 15 219.57 219.90 220.31 220.75 220.85 MIN: 204.22 / MAX: 967.44 MIN: 204.44 / MAX: 295.75 MIN: 204.86 / MAX: 914.53 MIN: 205.2 / MAX: 332.92 MIN: 204.72 / MAX: 1074.16 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread
NCNN Target: Vulkan GPU - Model: FastestDet OpenBenchmarking.org ms, Fewer Is Better NCNN 20220729 Target: Vulkan GPU - Model: FastestDet B C E D A 0.5625 1.125 1.6875 2.25 2.8125 SE +/- 0.03, N = 3 SE +/- 0.01, N = 14 SE +/- 0.03, N = 15 SE +/- 0.04, N = 15 SE +/- 0.17, N = 3 2.28 2.33 2.37 2.38 2.50 MIN: 1.85 / MAX: 8.13 MIN: 1.84 / MAX: 7.89 MIN: 1.85 / MAX: 17.06 MIN: 1.84 / MAX: 9.1 MIN: 1.86 / MAX: 8.64 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread
NCNN Target: Vulkan GPU - Model: regnety_400m OpenBenchmarking.org ms, Fewer Is Better NCNN 20220729 Target: Vulkan GPU - Model: regnety_400m B E D C A 0.6998 1.3996 2.0994 2.7992 3.499 SE +/- 0.12, N = 3 SE +/- 0.03, N = 15 SE +/- 0.03, N = 15 SE +/- 0.03, N = 15 SE +/- 0.13, N = 3 3.03 3.06 3.08 3.09 3.11 MIN: 2.67 / MAX: 13.33 MIN: 2.69 / MAX: 14.14 MIN: 2.69 / MAX: 25.76 MIN: 2.7 / MAX: 23.46 MIN: 2.71 / MAX: 12.9 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread
NCNN Target: Vulkan GPU - Model: squeezenet_ssd OpenBenchmarking.org ms, Fewer Is Better NCNN 20220729 Target: Vulkan GPU - Model: squeezenet_ssd B A C D E 1.2488 2.4976 3.7464 4.9952 6.244 SE +/- 0.10, N = 3 SE +/- 0.07, N = 3 SE +/- 0.13, N = 15 SE +/- 0.12, N = 15 SE +/- 0.14, N = 15 5.14 5.23 5.47 5.50 5.55 MIN: 3.67 / MAX: 16.01 MIN: 3.64 / MAX: 17.71 MIN: 3.7 / MAX: 27.5 MIN: 3.7 / MAX: 21.81 MIN: 3.7 / MAX: 22.31 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread
NCNN Target: Vulkan GPU - Model: yolov4-tiny OpenBenchmarking.org ms, Fewer Is Better NCNN 20220729 Target: Vulkan GPU - Model: yolov4-tiny B C A D E 4 8 12 16 20 SE +/- 0.09, N = 3 SE +/- 0.04, N = 15 SE +/- 0.04, N = 3 SE +/- 0.03, N = 15 SE +/- 0.25, N = 15 14.48 14.56 14.58 14.59 14.77 MIN: 13.05 / MAX: 21.86 MIN: 12.76 / MAX: 28.55 MIN: 13.18 / MAX: 22.42 MIN: 12.89 / MAX: 35.38 MIN: 12.94 / MAX: 50.63 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread
NCNN Target: Vulkan GPU - Model: alexnet OpenBenchmarking.org ms, Fewer Is Better NCNN 20220729 Target: Vulkan GPU - Model: alexnet A D C E B 0.504 1.008 1.512 2.016 2.52 SE +/- 0.06, N = 3 SE +/- 0.02, N = 13 SE +/- 0.03, N = 15 SE +/- 0.03, N = 12 SE +/- 0.08, N = 3 2.08 2.08 2.09 2.12 2.24 MIN: 1.68 / MAX: 11.23 MIN: 1.67 / MAX: 18.09 MIN: 1.67 / MAX: 18.2 MIN: 1.68 / MAX: 19.9 MIN: 1.68 / MAX: 18.33 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread
NCNN Target: Vulkan GPU - Model: shufflenet-v2 OpenBenchmarking.org ms, Fewer Is Better NCNN 20220729 Target: Vulkan GPU - Model: shufflenet-v2 A C D E B 0.477 0.954 1.431 1.908 2.385 SE +/- 0.07, N = 3 SE +/- 0.02, N = 15 SE +/- 0.02, N = 15 SE +/- 0.01, N = 15 SE +/- 0.02, N = 3 2.04 2.09 2.09 2.09 2.12 MIN: 1.65 / MAX: 11.74 MIN: 1.65 / MAX: 9.6 MIN: 1.65 / MAX: 13.26 MIN: 1.65 / MAX: 9.44 MIN: 1.67 / MAX: 7.91 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread
NCNN Target: Vulkan GPU-v2-v2 - Model: mobilenet-v2 OpenBenchmarking.org ms, Fewer Is Better NCNN 20220729 Target: Vulkan GPU-v2-v2 - Model: mobilenet-v2 A B D C E 0.45 0.9 1.35 1.8 2.25 SE +/- 0.04, N = 3 SE +/- 0.05, N = 3 SE +/- 0.01, N = 15 SE +/- 0.04, N = 15 SE +/- 0.02, N = 15 1.92 1.95 1.98 1.99 2.00 MIN: 1.7 / MAX: 6.92 MIN: 1.71 / MAX: 6.43 MIN: 1.69 / MAX: 9.54 MIN: 1.69 / MAX: 14.04 MIN: 1.7 / MAX: 12.73 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread
NCNN Target: Vulkan GPU - Model: mobilenet OpenBenchmarking.org ms, Fewer Is Better NCNN 20220729 Target: Vulkan GPU - Model: mobilenet B A C D E 3 6 9 12 15 SE +/- 0.10, N = 3 SE +/- 0.13, N = 3 SE +/- 0.22, N = 15 SE +/- 0.17, N = 15 SE +/- 0.23, N = 15 9.34 9.45 9.84 9.90 10.05 MIN: 5.97 / MAX: 18.47 MIN: 5.91 / MAX: 20.24 MIN: 5.61 / MAX: 25.64 MIN: 4.66 / MAX: 29.74 MIN: 4.72 / MAX: 26.35 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread
NCNN Target: CPU - Model: FastestDet OpenBenchmarking.org ms, Fewer Is Better NCNN 20220729 Target: CPU - Model: FastestDet E D B C A 1.1205 2.241 3.3615 4.482 5.6025 SE +/- 0.19, N = 3 SE +/- 0.02, N = 3 SE +/- 0.04, N = 15 SE +/- 0.08, N = 3 SE +/- 0.11, N = 3 4.84 4.86 4.90 4.93 4.98 MIN: 4.19 / MAX: 12.16 MIN: 4.61 / MAX: 12 MIN: 4.19 / MAX: 12.29 MIN: 4.52 / MAX: 11.93 MIN: 4.56 / MAX: 12.11 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread
NCNN Target: CPU - Model: alexnet OpenBenchmarking.org ms, Fewer Is Better NCNN 20220729 Target: CPU - Model: alexnet C D B A E 3 6 9 12 15 SE +/- 0.09, N = 3 SE +/- 0.03, N = 3 SE +/- 0.03, N = 15 SE +/- 0.05, N = 3 SE +/- 1.56, N = 3 7.67 7.72 7.73 7.79 9.30 MIN: 7.09 / MAX: 16.27 MIN: 7.14 / MAX: 16.18 MIN: 7.11 / MAX: 21.38 MIN: 7.1 / MAX: 16.44 MIN: 7.13 / MAX: 633.17 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread
NCNN Target: CPU - Model: mobilenet OpenBenchmarking.org ms, Fewer Is Better NCNN 20220729 Target: CPU - Model: mobilenet A E C D B 3 6 9 12 15 SE +/- 0.16, N = 3 SE +/- 0.05, N = 3 SE +/- 0.14, N = 3 SE +/- 0.14, N = 3 SE +/- 0.35, N = 15 11.46 11.64 11.71 11.84 11.92 MIN: 10.61 / MAX: 19.16 MIN: 10.9 / MAX: 20.48 MIN: 10.59 / MAX: 60.99 MIN: 10.94 / MAX: 29.92 MIN: 10.28 / MAX: 716.44 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread
Phoronix Test Suite v10.8.5