nn AMD Ryzen Threadripper PRO 5965WX 24-Cores testing with a ASUS Pro WS WRX80E-SAGE SE WIFI (1003 BIOS) and ASUS NVIDIA NV106 2GB on Ubuntu 22.04 via the Phoronix Test Suite.
HTML result view exported from: https://openbenchmarking.org/result/2208137-NE-NN282134066 .
nn Processor Motherboard Chipset Memory Disk Graphics Audio Monitor Network OS Kernel Desktop Display Server Display Driver OpenGL Vulkan Compiler File-System Screen Resolution a b c AMD Ryzen Threadripper PRO 5965WX 24-Cores @ 3.80GHz (24 Cores / 48 Threads) ASUS Pro WS WRX80E-SAGE SE WIFI (1003 BIOS) AMD Starship/Matisse 128GB 1000GB Western Digital WDS100T1X0E-00AFY0 ASUS NVIDIA NV106 2GB AMD Starship/Matisse VA2431 2 x Intel 10G X550T + Intel Wi-Fi 6 AX200 Ubuntu 22.04 5.19.0-051900daily20220809-generic (x86_64) GNOME Shell 42.2 X Server 1.21.1.3 + Wayland nouveau 4.3 Mesa 22.0.1 1.2.204 GCC 11.2.0 ext4 1920x1080 OpenBenchmarking.org Kernel Details - Transparent Huge Pages: madvise Compiler Details - --build=x86_64-linux-gnu --disable-vtable-verify --disable-werror --enable-bootstrap --enable-cet --enable-checking=release --enable-clocale=gnu --enable-default-pie --enable-gnu-unique-object --enable-languages=c,ada,c++,go,brig,d,fortran,objc,obj-c++,m2 --enable-libphobos-checking=release --enable-libstdcxx-debug --enable-libstdcxx-time=yes --enable-link-serialization=2 --enable-multiarch --enable-multilib --enable-nls --enable-objc-gc=auto --enable-offload-targets=nvptx-none=/build/gcc-11-gBFGDP/gcc-11-11.2.0/debian/tmp-nvptx/usr,amdgcn-amdhsa=/build/gcc-11-gBFGDP/gcc-11-11.2.0/debian/tmp-gcn/usr --enable-plugin --enable-shared --enable-threads=posix --host=x86_64-linux-gnu --program-prefix=x86_64-linux-gnu- --target=x86_64-linux-gnu --with-abi=m64 --with-arch-32=i686 --with-build-config=bootstrap-lto-lean --with-default-libstdcxx-abi=new --with-gcc-major-version-only --with-multilib-list=m32,m64,mx32 --with-target-system-zlib=auto --with-tune=generic --without-cuda-driver -v Processor Details - Scaling Governor: acpi-cpufreq schedutil (Boost: Enabled) - CPU Microcode: 0xa008203 Security Details - itlb_multihit: Not affected + l1tf: Not affected + mds: Not affected + meltdown: Not affected + mmio_stale_data: Not affected + retbleed: Not affected + spec_store_bypass: Mitigation of SSB disabled via prctl + spectre_v1: Mitigation of usercopy/swapgs barriers and __user pointer sanitization + spectre_v2: Mitigation of Retpolines IBPB: conditional IBRS_FW STIBP: always-on RSB filling + srbds: Not affected + tsx_async_abort: Not affected
nn mnn: mobilenetV3 mnn: squeezenetv1.1 mnn: resnet-v2-50 mnn: SqueezeNetV1.0 mnn: MobileNetV2_224 mnn: mobilenet-v1-1.0 mnn: inception-v3 ncnn: CPU - mobilenet ncnn: CPU-v2-v2 - mobilenet-v2 ncnn: CPU-v3-v3 - mobilenet-v3 ncnn: CPU - shufflenet-v2 ncnn: CPU - mnasnet ncnn: CPU - efficientnet-b0 ncnn: CPU - blazeface ncnn: CPU - googlenet ncnn: CPU - vgg16 ncnn: CPU - resnet18 ncnn: CPU - alexnet ncnn: CPU - resnet50 ncnn: CPU - yolov4-tiny ncnn: CPU - squeezenet_ssd ncnn: CPU - regnety_400m ncnn: CPU - vision_transformer ncnn: CPU - FastestDet a b c 2.257 3.648 18.433 5.946 3.820 2.700 22.134 13.12 6.17 5.69 7.25 5.69 8.28 3.09 14.16 23.28 9.25 5.96 17.10 19.49 15.51 23.84 110.26 8.76 2.281 3.789 18.815 5.983 3.901 2.720 22.399 12.93 5.94 5.45 7.04 5.46 7.98 2.99 13.84 23.25 9.13 5.89 16.81 19.16 15.24 22.46 110.27 8.59 2.038 3.291 17.115 5.658 3.545 2.729 19.854 12.84 5.91 5.42 6.96 5.46 7.9 2.96 13.8 23.32 9.11 5.98 16.72 18.99 15.23 22.26 110.27 8.43 OpenBenchmarking.org
Mobile Neural Network Model: mobilenetV3 OpenBenchmarking.org ms, Fewer Is Better Mobile Neural Network 2.0 Model: mobilenetV3 a b c 0.5132 1.0264 1.5396 2.0528 2.566 SE +/- 0.029, N = 15 SE +/- 0.026, N = 15 2.257 2.281 2.038 MIN: 2.09 / MAX: 3.98 MIN: 2.06 / MAX: 3.83 MIN: 2.01 / MAX: 3.61 1. (CXX) g++ options: -std=c++11 -O3 -fvisibility=hidden -fomit-frame-pointer -fstrict-aliasing -ffunction-sections -fdata-sections -ffast-math -fno-rtti -fno-exceptions -rdynamic -pthread -ldl
Mobile Neural Network Model: squeezenetv1.1 OpenBenchmarking.org ms, Fewer Is Better Mobile Neural Network 2.0 Model: squeezenetv1.1 a b c 0.8525 1.705 2.5575 3.41 4.2625 SE +/- 0.056, N = 15 SE +/- 0.046, N = 15 3.648 3.789 3.291 MIN: 3.33 / MAX: 5.24 MIN: 3.38 / MAX: 4.92 MIN: 3.26 / MAX: 4.16 1. (CXX) g++ options: -std=c++11 -O3 -fvisibility=hidden -fomit-frame-pointer -fstrict-aliasing -ffunction-sections -fdata-sections -ffast-math -fno-rtti -fno-exceptions -rdynamic -pthread -ldl
Mobile Neural Network Model: resnet-v2-50 OpenBenchmarking.org ms, Fewer Is Better Mobile Neural Network 2.0 Model: resnet-v2-50 a b c 5 10 15 20 25 SE +/- 0.26, N = 15 SE +/- 0.25, N = 15 18.43 18.82 17.12 MIN: 17.17 / MAX: 21.39 MIN: 17.52 / MAX: 21.68 MIN: 16.97 / MAX: 18.67 1. (CXX) g++ options: -std=c++11 -O3 -fvisibility=hidden -fomit-frame-pointer -fstrict-aliasing -ffunction-sections -fdata-sections -ffast-math -fno-rtti -fno-exceptions -rdynamic -pthread -ldl
Mobile Neural Network Model: SqueezeNetV1.0 OpenBenchmarking.org ms, Fewer Is Better Mobile Neural Network 2.0 Model: SqueezeNetV1.0 a b c 1.3462 2.6924 4.0386 5.3848 6.731 SE +/- 0.055, N = 15 SE +/- 0.052, N = 15 5.946 5.983 5.658 MIN: 5.24 / MAX: 7.59 MIN: 5.55 / MAX: 7.35 MIN: 5.59 / MAX: 7.09 1. (CXX) g++ options: -std=c++11 -O3 -fvisibility=hidden -fomit-frame-pointer -fstrict-aliasing -ffunction-sections -fdata-sections -ffast-math -fno-rtti -fno-exceptions -rdynamic -pthread -ldl
Mobile Neural Network Model: MobileNetV2_224 OpenBenchmarking.org ms, Fewer Is Better Mobile Neural Network 2.0 Model: MobileNetV2_224 a b c 0.8777 1.7554 2.6331 3.5108 4.3885 SE +/- 0.034, N = 15 SE +/- 0.047, N = 15 3.820 3.901 3.545 MIN: 3.54 / MAX: 5.26 MIN: 3.55 / MAX: 5.3 MIN: 3.51 / MAX: 4.49 1. (CXX) g++ options: -std=c++11 -O3 -fvisibility=hidden -fomit-frame-pointer -fstrict-aliasing -ffunction-sections -fdata-sections -ffast-math -fno-rtti -fno-exceptions -rdynamic -pthread -ldl
Mobile Neural Network Model: mobilenet-v1-1.0 OpenBenchmarking.org ms, Fewer Is Better Mobile Neural Network 2.0 Model: mobilenet-v1-1.0 a b c 0.614 1.228 1.842 2.456 3.07 SE +/- 0.019, N = 15 SE +/- 0.019, N = 15 2.700 2.720 2.729 MIN: 2.59 / MAX: 3.54 MIN: 2.57 / MAX: 4.14 MIN: 2.7 / MAX: 3.56 1. (CXX) g++ options: -std=c++11 -O3 -fvisibility=hidden -fomit-frame-pointer -fstrict-aliasing -ffunction-sections -fdata-sections -ffast-math -fno-rtti -fno-exceptions -rdynamic -pthread -ldl
Mobile Neural Network Model: inception-v3 OpenBenchmarking.org ms, Fewer Is Better Mobile Neural Network 2.0 Model: inception-v3 a b c 5 10 15 20 25 SE +/- 0.25, N = 15 SE +/- 0.30, N = 15 22.13 22.40 19.85 MIN: 20.33 / MAX: 24.82 MIN: 20.02 / MAX: 25.26 MIN: 19.7 / MAX: 21.4 1. (CXX) g++ options: -std=c++11 -O3 -fvisibility=hidden -fomit-frame-pointer -fstrict-aliasing -ffunction-sections -fdata-sections -ffast-math -fno-rtti -fno-exceptions -rdynamic -pthread -ldl
NCNN Target: CPU - Model: mobilenet OpenBenchmarking.org ms, Fewer Is Better NCNN 20220729 Target: CPU - Model: mobilenet a b c 3 6 9 12 15 SE +/- 0.00, N = 3 SE +/- 0.06, N = 3 13.12 12.93 12.84 MIN: 12.95 / MAX: 19.49 MIN: 12.71 / MAX: 15 MIN: 12.76 / MAX: 13.27 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread
NCNN Target: CPU-v2-v2 - Model: mobilenet-v2 OpenBenchmarking.org ms, Fewer Is Better NCNN 20220729 Target: CPU-v2-v2 - Model: mobilenet-v2 a b c 2 4 6 8 10 SE +/- 0.02, N = 3 SE +/- 0.04, N = 3 6.17 5.94 5.91 MIN: 6.04 / MAX: 10.12 MIN: 5.78 / MAX: 7.68 MIN: 5.82 / MAX: 7.49 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread
NCNN Target: CPU-v3-v3 - Model: mobilenet-v3 OpenBenchmarking.org ms, Fewer Is Better NCNN 20220729 Target: CPU-v3-v3 - Model: mobilenet-v3 a b c 1.2803 2.5606 3.8409 5.1212 6.4015 SE +/- 0.02, N = 3 SE +/- 0.05, N = 3 5.69 5.45 5.42 MIN: 5.55 / MAX: 9.2 MIN: 5.28 / MAX: 7.8 MIN: 5.35 / MAX: 6.75 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread
NCNN Target: CPU - Model: shufflenet-v2 OpenBenchmarking.org ms, Fewer Is Better NCNN 20220729 Target: CPU - Model: shufflenet-v2 a b c 2 4 6 8 10 SE +/- 0.03, N = 3 SE +/- 0.05, N = 3 7.25 7.04 6.96 MIN: 7.12 / MAX: 9.7 MIN: 6.85 / MAX: 8.14 MIN: 6.9 / MAX: 7.93 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread
NCNN Target: CPU - Model: mnasnet OpenBenchmarking.org ms, Fewer Is Better NCNN 20220729 Target: CPU - Model: mnasnet a b c 1.2803 2.5606 3.8409 5.1212 6.4015 SE +/- 0.01, N = 2 SE +/- 0.07, N = 2 5.69 5.46 5.46 MIN: 5.59 / MAX: 9.66 MIN: 5.33 / MAX: 7.09 MIN: 5.39 / MAX: 7.02 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread
NCNN Target: CPU - Model: efficientnet-b0 OpenBenchmarking.org ms, Fewer Is Better NCNN 20220729 Target: CPU - Model: efficientnet-b0 a b c 2 4 6 8 10 SE +/- 0.03, N = 3 SE +/- 0.07, N = 3 8.28 7.98 7.90 MIN: 8.13 / MAX: 13.55 MIN: 7.78 / MAX: 9.57 MIN: 7.84 / MAX: 9.08 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread
NCNN Target: CPU - Model: blazeface OpenBenchmarking.org ms, Fewer Is Better NCNN 20220729 Target: CPU - Model: blazeface a b c 0.6953 1.3906 2.0859 2.7812 3.4765 SE +/- 0.01, N = 3 SE +/- 0.03, N = 3 3.09 2.99 2.96 MIN: 3.02 / MAX: 5.92 MIN: 2.89 / MAX: 3.67 MIN: 2.92 / MAX: 3.42 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread
NCNN Target: CPU - Model: googlenet OpenBenchmarking.org ms, Fewer Is Better NCNN 20220729 Target: CPU - Model: googlenet a b c 4 8 12 16 20 SE +/- 0.04, N = 3 SE +/- 0.09, N = 3 14.16 13.84 13.80 MIN: 13.93 / MAX: 19.53 MIN: 13.5 / MAX: 17.24 MIN: 13.66 / MAX: 15.75 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread
NCNN Target: CPU - Model: vgg16 OpenBenchmarking.org ms, Fewer Is Better NCNN 20220729 Target: CPU - Model: vgg16 a b c 6 12 18 24 30 SE +/- 0.04, N = 3 SE +/- 0.04, N = 3 23.28 23.25 23.32 MIN: 22.77 / MAX: 28.58 MIN: 22.64 / MAX: 33.92 MIN: 22.65 / MAX: 28 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread
NCNN Target: CPU - Model: resnet18 OpenBenchmarking.org ms, Fewer Is Better NCNN 20220729 Target: CPU - Model: resnet18 a b c 3 6 9 12 15 SE +/- 0.01, N = 3 SE +/- 0.03, N = 3 9.25 9.13 9.11 MIN: 9.06 / MAX: 12.11 MIN: 8.93 / MAX: 13.18 MIN: 8.93 / MAX: 13.05 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread
NCNN Target: CPU - Model: alexnet OpenBenchmarking.org ms, Fewer Is Better NCNN 20220729 Target: CPU - Model: alexnet a b c 1.3455 2.691 4.0365 5.382 6.7275 SE +/- 0.07, N = 3 SE +/- 0.04, N = 3 5.96 5.89 5.98 MIN: 5.71 / MAX: 9.01 MIN: 5.69 / MAX: 9.06 MIN: 5.7 / MAX: 8.24 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread
NCNN Target: CPU - Model: resnet50 OpenBenchmarking.org ms, Fewer Is Better NCNN 20220729 Target: CPU - Model: resnet50 a b c 4 8 12 16 20 SE +/- 0.04, N = 3 SE +/- 0.04, N = 3 17.10 16.81 16.72 MIN: 16.78 / MAX: 21.76 MIN: 16.47 / MAX: 21.24 MIN: 16.54 / MAX: 22.07 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread
NCNN Target: CPU - Model: yolov4-tiny OpenBenchmarking.org ms, Fewer Is Better NCNN 20220729 Target: CPU - Model: yolov4-tiny a b c 5 10 15 20 25 SE +/- 0.17, N = 3 SE +/- 0.21, N = 3 19.49 19.16 18.99 MIN: 18.87 / MAX: 21.72 MIN: 18.61 / MAX: 28.33 MIN: 18.66 / MAX: 20.52 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread
NCNN Target: CPU - Model: squeezenet_ssd OpenBenchmarking.org ms, Fewer Is Better NCNN 20220729 Target: CPU - Model: squeezenet_ssd a b c 4 8 12 16 20 SE +/- 0.04, N = 3 SE +/- 0.09, N = 3 15.51 15.24 15.23 MIN: 15.09 / MAX: 18.17 MIN: 14.8 / MAX: 18.11 MIN: 14.9 / MAX: 23.96 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread
NCNN Target: CPU - Model: regnety_400m OpenBenchmarking.org ms, Fewer Is Better NCNN 20220729 Target: CPU - Model: regnety_400m a b c 6 12 18 24 30 SE +/- 0.13, N = 3 SE +/- 0.29, N = 3 23.84 22.46 22.26 MIN: 23.38 / MAX: 26.29 MIN: 21.61 / MAX: 28.98 MIN: 21.92 / MAX: 24.69 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread
NCNN Target: CPU - Model: vision_transformer OpenBenchmarking.org ms, Fewer Is Better NCNN 20220729 Target: CPU - Model: vision_transformer a b c 20 40 60 80 100 SE +/- 0.01, N = 3 SE +/- 0.01, N = 3 110.26 110.27 110.27 MIN: 109.56 / MAX: 114.27 MIN: 109.69 / MAX: 117.06 MIN: 109.27 / MAX: 117.21 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread
NCNN Target: CPU - Model: FastestDet OpenBenchmarking.org ms, Fewer Is Better NCNN 20220729 Target: CPU - Model: FastestDet a b c 2 4 6 8 10 SE +/- 0.04, N = 3 SE +/- 0.14, N = 3 8.76 8.59 8.43 MIN: 8.6 / MAX: 10.6 MIN: 8.26 / MAX: 10.5 MIN: 8.35 / MAX: 10.14 1. (CXX) g++ options: -O3 -rdynamic -lgomp -lpthread
Phoronix Test Suite v10.8.4