NCNN

NCNN is a high performance neural network inference framework optimized for mobile and other platforms developed by Tencent.

To run this test with the Phoronix Test Suite, the basic command is: phoronix-test-suite benchmark ncnn.

Project Site

github.com

Test Created

18 September 2020

Last Updated

18 December 2020

Test Maintainer

Michael Larabel

Test Type

System

Average Install Time

57 Seconds

Average Run Time

20 Minutes, 48 Seconds

Supported Platforms


Public Result UploadsReported Installs*Test Completions*OpenBenchmarking.orgEventsNCNN Popularity Statisticspts/ncnn2020.092020.102020.112020.122021.012K4K6K8K10K
* Data based on those opting to upload their test results to OpenBenchmarking.org and users enabling the opt-in anonymous statistics reporting while running benchmarks from an Internet-connected platform.
Data current as of Sat, 23 Jan 2021 00:50:15 GMT.
Vulkan GPU40.3%CPU59.7%Target Option PopularityOpenBenchmarking.org
mnasnet6.7%efficientnet-b06.7%vgg166.7%yolov4-tiny6.7%squeezenet_ssd6.7%resnet186.7%regnety_400m6.7%shufflenet-v26.7%resnet506.7%mobilenet-v26.6%alexnet6.7%blazeface6.7%googlenet6.7%mobilenet6.7%mobilenet-v36.6%Model Option PopularityOpenBenchmarking.org

Revision History

pts/ncnn-1.1.0   [View Source]   Fri, 18 Dec 2020 08:06:41 GMT
Update against new upstream NCNN 20201218.

pts/ncnn-1.0.3   [View Source]   Fri, 25 Sep 2020 06:36:39 GMT
Drop int8 tests per https://github.com/phoronix-test-suite/test-profiles/pull/167

pts/ncnn-1.0.2   [View Source]   Thu, 24 Sep 2020 12:52:47 GMT
Expose Vulkan GPU support.

pts/ncnn-1.0.1   [View Source]   Fri, 18 Sep 2020 12:28:10 GMT
Increase the run count.

pts/ncnn-1.0.0   [View Source]   Fri, 18 Sep 2020 11:58:15 GMT
Initial commit of Tencent NCNN.

Suites Using This Test

Machine Learning

HPC - High Performance Computing

NVIDIA GPU Compute

Vulkan Compute


Performance Metrics

Analyze Test Configuration:

NCNN 20201218

Target: CPU - Model: mobilenet

OpenBenchmarking.org metrics for this test profile configuration based on 368 public results since 18 December 2020 with the latest data as of 23 January 2021.

Below is an overview of the generalized performance for components where there is sufficient statistically significant data based upon user-uploaded results. It is important to keep in mind particularly in the Linux/open-source space there can be vastly different OS configurations, with this overview intended to offer just general guidance as to the performance expectations.

Component
Percentile Rank
# Matching Public Results
ms (Average)
Mid-Tier
75th
> 18
Median
50th
28
48th
10
29 +/- 3
46th
15
30 +/- 1
Low-Tier
25th
> 38
25th
4
40 +/- 4
7th
4
101 +/- 1
3rd
6
177 +/- 11
OpenBenchmarking.orgDistribution Of Public Results - Target: CPU - Model: mobilenet368 Results Range From 11 To 508 ms113355779912114316518720923125327529731934136338540742945147349551750100150200250

Based on OpenBenchmarking.org data, the selected test / test configuration (NCNN 20201218 - Target: CPU - Model: mobilenet) has an average run-time of 10 minutes. By default this test profile is set to run at least 3 times but may increase if the standard deviation exceeds pre-defined defaults or other calculations deem additional runs necessary for greater statistical accuracy of the result.

OpenBenchmarking.orgMinutesTime Required To Complete BenchmarkTarget: CPU - Model: mobilenetRun-Time918273645Min: 3 / Avg: 10.07 / Max: 46

Based on public OpenBenchmarking.org results, the selected test / test configuration has an average standard deviation of 1.2%.

OpenBenchmarking.orgPercent, Fewer Is BetterAverage Deviation Between RunsTarget: CPU - Model: mobilenetDeviation3691215Min: 0 / Avg: 1.25 / Max: 12

Recent Test Results

OpenBenchmarking.org Results Compare

1 System - 466 Benchmark Results

2 x AMD EPYC 7F72 24-Core - Supermicro H11DSi-NT v2.00 - AMD Starship

Ubuntu 20.10 - 5.10.9-051009-generic - GNOME Shell 3.38.1

1 System - 94 Benchmark Results

Intel Core i7-1165G7 - Dell 08607K - Intel Device a0ef

Ubuntu 20.04 - 5.6.0-1042-oem - GNOME Shell 3.36.4

2 Systems - 129 Benchmark Results

AMD Ryzen 9 5950X 16-Core - ASUS ROG CROSSHAIR VIII HERO - AMD Starship

Ubuntu 20.10 - 5.11.0-051100rc2daily20210108-generic - GNOME Shell 3.38.1

1 System - 23 Benchmark Results

Intel Core i9-9900KS - Gigabyte Z390 DESIGNARE-CF - Intel Cannon Lake PCH

Linuxmint 20.1 - 5.9.16-mint1 - Cinnamon 4.8.6

1 System - 27 Benchmark Results

AMD Ryzen 5 3400G - ASUS Pro WS X570-ACE - AMD Raven

Gentoo 2.7 - 5.10.7-gentoo - Xfce 4.14

1 System - 20 Benchmark Results

AMD Ryzen 5 3600 6-Core - MSI B450 TOMAHAWK - AMD Starship

ManjaroLinux 20.2.1 - 5.10.2-2-MANJARO - Xfce 4.14

3 Systems - 191 Benchmark Results

AMD Ryzen 3 2200G - ASUS PRIME B350M-E - AMD Raven

Ubuntu 20.10 - 5.8.0-38-generic - GNOME Shell 3.38.1

4 Systems - 91 Benchmark Results

AMD Ryzen 5 5600X 6-Core - ASUS TUF GAMING B550M-PLUS - AMD Starship

Ubuntu 20.10 - 5.10.4-051004-generic - GNOME Shell 3.38.1

4 Systems - 104 Benchmark Results

AMD Ryzen 5 2400G - MSI B350M GAMING PRO - AMD Raven

Ubuntu 19.10 - 5.3.0-64-generic - GNOME Shell 3.34.1

1 System - 30 Benchmark Results

Intel Core i7-1165G7 - Dell 08607K - Intel Device a0ef

Ubuntu 20.04 - 5.6.0-1042-oem - GNOME Shell 3.36.4

1 System - 49 Benchmark Results

AMD Ryzen 7 3700X 8-Core - ASRock X570 Phantom Gaming-ITX/TB3 - AMD Starship

Ubuntu 20.10 - 5.8.0-36-lowlatency - X Server 1.20.9

3 Systems - 53 Benchmark Results

ARMv8 Cortex-A72 - BCM2835 Raspberry Pi 400 Rev 1.0 - 4096MB

Debian 10 - 5.4.51-v8+ - LXDE 0.10.0

Most Popular Test Results

OpenBenchmarking.org Results Compare

3 Systems - 191 Benchmark Results

AMD Ryzen 3 2200G - ASUS PRIME B350M-E - AMD Raven

Ubuntu 20.10 - 5.8.0-38-generic - GNOME Shell 3.38.1

3 Systems - 28 Benchmark Results

Intel Xeon Silver 4216 - TYAN S7100AG2NR - Intel Sky Lake-E DMI3 Registers

Debian 10 - 4.19.0-9-amd64 - GNOME Shell 3.30.2

4 Systems - 91 Benchmark Results

AMD Ryzen 5 5600X 6-Core - ASUS TUF GAMING B550M-PLUS - AMD Starship

Ubuntu 20.10 - 5.10.4-051004-generic - GNOME Shell 3.38.1

4 Systems - 104 Benchmark Results

AMD Ryzen 5 2400G - MSI B350M GAMING PRO - AMD Raven

Ubuntu 19.10 - 5.3.0-64-generic - GNOME Shell 3.34.1

6 Systems - 107 Benchmark Results

AMD Ryzen 9 5900X 12-Core - ASUS ROG CROSSHAIR VIII HERO - AMD Starship

Debian testing - 5.9.0-4-amd64 - GNOME Shell 3.38.2

3 Systems - 148 Benchmark Results

Intel Core i7-4790K - Gigabyte Z97-HD3P - Intel 4th Gen Core DRAM

Ubuntu 19.10 - 5.9.0-050900rc8daily20201009-generic - GNOME Shell 3.34.1

3 Systems - 53 Benchmark Results

ARMv8 Cortex-A72 - BCM2835 Raspberry Pi 400 Rev 1.0 - 4096MB

Debian 10 - 5.4.51-v8+ - LXDE 0.10.0

3 Systems - 124 Benchmark Results

Intel Core i9-9900KS - ASUS PRIME Z390-A - Intel Cannon Lake PCH

Ubuntu 20.04 - 5.9.0-050900rc8daily20201005-generic - GNOME Shell 3.36.2

8 Systems - 108 Benchmark Results

Intel Core i9-10900KF - Gigabyte Z490 VISION G - Intel Comet Lake PCH

Clear Linux OS 34100 - 5.9.9-1001.native - GNOME Shell 3.38.2

3 Systems - 126 Benchmark Results

Intel Core i7-1165G7 - Dell 0GG9PT - Intel Tiger Lake-LP

Fedora 33 - 5.8.15-301.fc33.x86_64 - GNOME Shell 3.38.2

Find More Test Results

OpenBenchmarking.org Community User Comments

Post A Comment