NCNN Benchmark - OpenBenchmarking.org

NCNN is a high performance neural network inference framework optimized for mobile and other platforms developed by Tencent.

To run this test with the Phoronix Test Suite, the basic command is: phoronix-test-suite benchmark ncnn.

Revision History

pts/ncnn-1.1.0 [View Source] Fri, 18 Dec 2020 08:06:41 GMT
Update against new upstream NCNN 20201218.

pts/ncnn-1.0.3 [View Source] Fri, 25 Sep 2020 06:36:39 GMT
Drop int8 tests per https://github.com/phoronix-test-suite/test-profiles/pull/167

pts/ncnn-1.0.2 [View Source] Thu, 24 Sep 2020 12:52:47 GMT
Expose Vulkan GPU support.

pts/ncnn-1.0.1 [View Source] Fri, 18 Sep 2020 12:28:10 GMT
Increase the run count.

pts/ncnn-1.0.0 [View Source] Fri, 18 Sep 2020 11:58:15 GMT
Initial commit of Tencent NCNN.

Performance Metrics

NCNN 20201218

Target: Vulkan GPU - Model: resnet18

OpenBenchmarking.org metrics for this test profile configuration based on 2,356 public results since 18 December 2020 with the latest data as of 9 April 2021.

Below is an overview of the generalized performance for components where there is sufficient statistically significant data based upon user-uploaded results. It is important to keep in mind particularly in the Linux/open-source space there can be vastly different OS configurations, with this overview intended to offer just general guidance as to the performance expectations.

Component

Percentile Rank

# Matching Public Results

ms (Average)

Gigabyte AMD Radeon 540

100th

XFX AMD Radeon R9 285

94th

Sapphire AMD Radeon RX 5600 OEM

94th

120

Gigabyte AMD Radeon RX 5500

85th

Sapphire AMD Radeon RX 5500

85th

NVIDIA GeForce RTX 3090

85th

14 ^{+/- 1}

ASUS Intel UHD 630

84th

14 ^{+/- 1}

ASUS NVIDIA GeForce RTX 2070

84th

NVIDIA GeForce RTX 2080 SUPER

84th

NVIDIA GeForce RTX 2070 SUPER

84th

Zotac NVIDIA GeForce RTX 2080

84th

NVIDIA GeForce RTX 2060 SUPER

83rd

Mid-Tier

75th

> 15

NVIDIA GeForce RTX 3080

71st

15 ^{+/- 2}

NVIDIA GeForce RTX 2080 Ti

71st

15 ^{+/- 1}

NVIDIA GeForce RTX 3060 Ti

71st

15 ^{+/- 2}

AMD Radeon RX 5600 OEM

71st

15 ^{+/- 2}

AMD Radeon RX 56

67th

MSI Intel HD 630

62nd

Zotac NVIDIA GeForce RTX 2060 SUPER

62nd

305

MSI Intel UHD 630

54th

NVIDIA Quadro RTX 5000 with Max-Q Design

53rd

MSI AMD Radeon RX 470

53rd

Median

50th

NVIDIA GeForce RTX 2060

46th

Intel HD P630

43rd

MSI Intel HD P530

30th

Gigabyte Intel UHD 630

30th

AMD Radeon RX 470

30th

22 ^{+/- 3}

Intel 8th Gen Core Gaussian Mixture Model

27th

Intel Iris Plus G7

27th

23 ^{+/- 2}

Low-Tier

25th

> 23

Intel UHD P630

21st

Gigabyte Intel HD 530

20th

Intel HD 530

17th

Intel Iris Pro 580

17th

109

Intel UHD 620

12th

30 ^{+/- 1}

Gigabyte Intel HD 4600

12th

Gigabyte Intel HD 630

11th

32 ^{+/- 1}

Intel HD 5500

5th

Sapphire AMD Radeon RX 550 640SP

5th

48 ^{+/- 6}

Gigabyte AMD Radeon RX 470

2nd

55 ^{+/- 2}

MSI NVIDIA GeForce GTX 1080 Ti

2nd

363

Based on OpenBenchmarking.org data, the selected test / test configuration (NCNN 20201218 - Target: Vulkan GPU - Model: resnet18) has an average run-time of 10 minutes. By default this test profile is set to run at least 3 times but may increase if the standard deviation exceeds pre-defined defaults or other calculations deem additional runs necessary for greater statistical accuracy of the result.

Based on public OpenBenchmarking.org results, the selected test / test configuration has an average standard deviation of 3.4%.

Notable Instruction Set Usage

Notable instruction set extensions supported by this test, based on an automatic analysis by the Phoronix Test Suite / OpenBenchmarking.org analytics engine.

Instruction Set

Support

Instructions Detected

SSE2 (SSE2)

Used by default on supported hardware.

MOVDQA MOVD PSHUFD PSHUFLW MOVDQU SUBSD MOVAPD MINSD MAXSD ADDSD CVTSI2SD DIVSD PUNPCKLQDQ CVTSS2SD MULSD MULPD PSRLDQ CVTSD2SS CVTDQ2PS CVTTPS2DQ PMULUDQ

Advanced Vector Extensions (AVX)

Used by default on supported hardware.
Found on Intel processors since Sandy Bridge (2011).
Found on AMD processors since Bulldozer (2011).

VZEROUPPER VINSERTF128 VEXTRACTF128 VBROADCASTSS VPERM2F128 VMASKMOVPS VPERMILPS VBROADCASTSD

Advanced Vector Extensions 2 (AVX2)

Used by default on supported hardware.
Found on Intel processors since Haswell (2013).
Found on AMD processors since Excavator (2016).

VINSERTI128 VPERM2I128 VEXTRACTI128 VPERMD VPERMQ VPBROADCASTD VPBROADCASTB VGATHERDPS VPBROADCASTW VPGATHERQQ VPSRLVQ VPBROADCASTQ

FMA (FMA)

Used by default on supported hardware.
Found on Intel processors since Haswell (2013).
Found on AMD processors since Bulldozer (2011).

VFMADD132PS VFMADD132SS VFMADD231SS VFMADD213PS VFMADD231PS VFNMADD231PS VFNMADD132PS VFMSUB231PS VFMSUB132PS VFMADD213SS VFMSUB213PS VFMSUB132SS VFMADD132SD VFNMADD213SS VFNMADD132SS VFNMADD231SS VFMADD132PD VFNMADD213PS

The test / benchmark does honor compiler flag changes.

Last automated analysis: 30 January 2021

This test profile binary relies on the shared libraries libgomp.so.1, libpthread.so.0, libm.so.6, libmvec.so.1, libc.so.6, libdl.so.2.

RTX 3070 Compute 3 Systems - 99 Benchmark Results	AMD Ryzen 9 5900X 12-Core - ASUS ROG CROSSHAIR VIII HERO - AMD Starship Ubuntu 20.04 - 5.8.0-48-generic - GNOME Shell 3.36.7
ngc rtx 3090 3 Systems - 100 Benchmark Results	AMD Ryzen 9 5900X 12-Core - ASUS ROG CROSSHAIR VIII HERO - AMD Starship Ubuntu 20.04 - 5.8.0-48-generic - GNOME Shell 3.36.7
Intel Core i5 11600K + Core i9 11900K Linux Performance 9 Systems - 442 Benchmark Results	Intel Core i9-10900K - Gigabyte Z490 AORUS MASTER - Intel Comet Lake PCH Ubuntu 21.04 - 5.12.0-051200rc3daily20210315-generic - GNOME Shell 3.38.3
installres1 1 System - 92 Benchmark Results	AMD Ryzen 5 5600X 6-Core - ASRock X570 Phantom Gaming-ITX/TB3 - AMD Device 1480 Ubuntu 18.04 - 5.4.0-70-generic - GNOME Shell 3.28.4
Core i9 11900K + ASUS MAXIMUS XIII HERO 0703 BIOS Linux Performance 1 System - 323 Benchmark Results	Intel Core i9-11900K - ASUS ROG MAXIMUS XIII HERO - Intel Tiger Lake-H Ubuntu 21.04 - 5.12.0-051200rc3daily20210315-generic - GNOME Shell 3.38.3
Intel Core i5 11600K + Core i9 11900K Linux Performance 8 Systems - 439 Benchmark Results	AMD Ryzen 9 5900X 12-Core - ASUS ROG CROSSHAIR VIII HERO - AMD Starship Ubuntu 21.04 - 5.12.0-051200rc3daily20210315-generic - GNOME Shell 3.38.3
GPU compute 1 System - 100 Benchmark Results	AMD Ryzen 9 5900X 12-Core - ASUS ROG CROSSHAIR VIII HERO - AMD Starship Ubuntu 20.04 - 5.8.0-48-generic - GNOME Shell 3.36.7
Intel Linux Rocket Lake Graphics 4 Systems - 138 Benchmark Results	Intel Core i9-10900K - Gigabyte Z490 AORUS MASTER - Intel Comet Lake PCH Ubuntu 21.04 - 5.12.0-051200rc3daily20210315-generic - GNOME Shell 3.38.3
ncnn-1 1 System - 30 Benchmark Results	AMD Ryzen 5 3600 6-Core - MSI B550M PRO-VDH WIFI - AMD Starship Linuxmint 20.1 - 5.11.0-051100-generic - MATE 1.24.0
stargazer-nvidia-test 1 System - 34 Benchmark Results	2 x Intel Xeon E5-2640 v4 - Supermicro X10DRG-O+-CPU v1.00 - Intel Xeon E7 v4 Ubuntu 18.04 - 5.4.0-67-generic - X Server
Ryzen 9 5950X Compiler Testing 1 System - 147 Benchmark Results	AMD Ryzen 9 5950X 16-Core - ASUS ROG CROSSHAIR VIII HERO - AMD Starship Ubuntu 20.10 - 5.11.6-051106-generic - GNOME Shell 3.38.2
090320211754-ryzen7 2 Systems - 23 Benchmark Results	AMD Ryzen 7 3800X 8-Core - ASUS ROG STRIX B550-F GAMING - AMD Starship Pop 20.10 - 5.8.0-7642-generic - GNOME Shell 3.38.3

Ubuntu 20.10 Kernels AMD Zen 3 6 Systems - 143 Benchmark Results	AMD Ryzen 9 5900X 12-Core - ASUS ROG CROSSHAIR VIII HERO - AMD Starship Ubuntu 20.10 - 5.10.7-051007-generic - GNOME Shell 3.38.1
Ryzen 3 2200G 2021 3 Systems - 191 Benchmark Results	AMD Ryzen 3 2200G - ASUS PRIME B350M-E - AMD Raven Ubuntu 20.10 - 5.8.0-38-generic - GNOME Shell 3.38.1
Ryzen 9 5900X LInux 5.11 Featured Kernel Comparison	AMD Ryzen 9 5900X 12-Core - ASUS ROG CROSSHAIR VIII HERO - AMD Starship Ubuntu 20.10 - 5.10.0-051000daily20201222-generic - GNOME Shell 3.38.1
Core i7 9750H EOY 4 Systems - 80 Benchmark Results	Intel Core i7-9750H - Notebook P95_96_97Ex Rx - Intel Cannon Lake PCH Ubuntu 20.04 - 5.7.0-999-generic - GNOME Shell 3.36.4
5600X New 2021 4 Systems - 91 Benchmark Results	AMD Ryzen 5 5600X 6-Core - ASUS TUF GAMING B550M-PLUS - AMD Starship Ubuntu 20.10 - 5.10.4-051004-generic - GNOME Shell 3.38.1
nvidia-linux-gpu-performance-20212 Featured Graphics Comparison	AMD Ryzen 9 5950X 16-Core - ASUS ROG CROSSHAIR VIII HERO - AMD Starship Ubuntu 20.10 - 5.8.0-41-generic - GNOME Shell 3.38.2
Clear Linux vs Other Distros - Tiger Lake 4 Systems - 110 Benchmark Results	Intel Core i7-1165G7 - Dell 0GG9PT - Intel Tiger Lake-LP ManjaroLinux 20.2 - 5.9.11-3-MANJARO - Xfce 4.14
Coder Radio XPS 13 ML Ubuntu Benchmark 2 Systems - 85 Benchmark Results	Intel Core i7-10510U - LENOVO 20U9CTO1WW - Intel Comet Lake PCH-LP Fedora 33 - 5.9.16-200.fc33.x86_64 - KDE Plasma 5.20.4
Kubuntu Focus 18.04 4 Systems - 145 Benchmark Results	Intel Core i7-9750H - Notebook P95_96_97Ex Rx - Intel Cannon Lake PCH Ubuntu 20.04 - 5.8.0-41-generic - KDE Plasma 5.18.5
rx580 1 System - 16 Benchmark Results	Intel Core i7-7700K - ASUS MAXIMUS IX CODE - Intel Xeon E3-1200 v6 Fedora 32 - 5.9.16-100.fc32.x86_64 - Openbox 3.6.1
Ryzen 9 5900X Clear Linux vs. Other Distributions 6 Systems - 107 Benchmark Results	AMD Ryzen 9 5900X 12-Core - ASUS ROG CROSSHAIR VIII HERO - AMD Starship Fedora 33 - 5.9.14-200.fc33.x86_64 - GNOME Shell 3.38.2
2021 Laptops 3 Systems - 158 Benchmark Results	Intel Core i7-1065G7 - Dell 06CDVY - Intel Device 34ef Ubuntu 20.10 - 5.8.0-41-generic - GNOME Shell 3.38.2

NCNN

Project Site

Test Created

Last Updated

Test Maintainer

Test Type

Average Install Time

Average Run Time

Test Dependencies

Accolades

Supported Platforms

Revision History

Suites Using This Test

Performance Metrics

NCNN 20201218

Target: Vulkan GPU - Model: resnet18

Notable Instruction Set Usage

Recent Test Results

Most Popular Test Results