NVIDIA TensorRT Inference

This test profile uses any existing system installation of NVIDIA TensorRT for carrying out inference benchmarks with various neural networks.

This test profile requires you have already installed TensorRT to /usr/src/tensorrt.

To run this test with the Phoronix Test Suite, the basic command is: phoronix-test-suite benchmark system/tensorrt-inference.

Use with caution this test profile is currently marked Experimental.

Test Created

23 December 2018

Test Maintainer

Michael Larabel 

Test Type

Graphics

Average Install Time

3 Seconds

Average Run Time

7 Minutes, 34 Seconds

Accolades

5k+ Downloads

Supported Platforms


Public Result Uploads *Reported Installs **Reported Test Completions **Test Profile Page Views ***OpenBenchmarking.orgEventsNVIDIA TensorRT Inference Popularity Statisticssystem/tensorrt-inference2018.122019.012019.032019.042019.052019.062019.072019.082019.092019.112020.012020.042020.052020.082021.032021.042021.052021.062021.072021.082021.092021.10130260390520650
* Uploading of benchmark result data to OpenBenchmarking.org is always optional (opt-in) via the Phoronix Test Suite for users wishing to share their results publicly.
** Data based on those opting to upload their test results to OpenBenchmarking.org and users enabling the opt-in anonymous statistics reporting while running benchmarks from an Internet-connected platform.
*** Test profile page view reporting began March 2021.
Data current as of 14 October 2021.
ResNet15214.9%VGG197.7%VGG169.8%AlexNet22.6%ResNet5023.5%GoogleNet21.5%Neural Network Option PopularityOpenBenchmarking.org
FP1658.9%INT841.1%Precision Option PopularityOpenBenchmarking.org
115.2%819.9%425.2%3222.8%1616.8%Batch Size Option PopularityOpenBenchmarking.org

Revision History

system/tensorrt-inference-1.0.0   [View Source]   Sun, 23 Dec 2018 18:52:47 GMT
Initial commit of NVIDIA TensorRT benchmark from the Jetson Xavier refeenence guide.


Performance Metrics

Analyze Test Configuration:

NVIDIA TensorRT Inference

Neural Network: ResNet50 - Precision: FP16 - Batch Size: 32 - DLA Cores: Disabled

OpenBenchmarking.org metrics for this test profile configuration based on 723 public results since 23 December 2018 with the latest data as of 8 May 2020.

Additional benchmark metrics will come after OpenBenchmarking.org has collected a sufficient data-set.

OpenBenchmarking.orgDistribution Of Public Results - Neural Network: ResNet50 - Precision: FP16 - Batch Size: 32 - DLA Cores: Disabled695 Results Range From 32 To 5649 Images Per Second32145258371484597710823936104911621275138815011614172718401953206621792292240525182631274428572970308331963309342235353648376138743987410042134326443945524665477848915004511752305343545655695682110220330440550

Based on OpenBenchmarking.org data, the selected test / test configuration (NVIDIA TensorRT Inference - Neural Network: ResNet50 - Precision: FP16 - Batch Size: 32 - DLA Cores: Disabled) has an average run-time of 9 minutes. By default this test profile is set to run at least 3 times but may increase if the standard deviation exceeds pre-defined defaults or other calculations deem additional runs necessary for greater statistical accuracy of the result.

OpenBenchmarking.orgMinutesTime Required To Complete BenchmarkNeural Network: ResNet50 - Precision: FP16 - Batch Size: 32 - DLA Cores: DisabledRun-Time48121620Min: 1 / Avg: 8.54 / Max: 15

Recent Test Results

OpenBenchmarking.org Results Compare

1 System - 2 Benchmark Results

ARMv8 rev 0 - Jetson-AGX - 16GB

Ubuntu 18.04 - 4.9.140-tegra - GNOME Shell 3.28.4

1 System - 1 Benchmark Result

Intel Xeon Gold 6126 - Intel 440BX - Intel 440BX

Ubuntu 16.04 - 4.4.0-177-generic - X Server 1.18.4

9 Systems - 76 Benchmark Results

ARMv8 rev 3 - quill - 8192MB

Ubuntu 16.04 - 4.4.38-tegra - Unity 7.4.0

5 Systems - 71 Benchmark Results

ARMv8 rev 1 - NVIDIA Jetson Nano Developer Kit - 4096MB

Ubuntu 18.04 - 4.9.140-tegra - Unity 7.5.0

5 Systems - 71 Benchmark Results

ARMv8 rev 1 - NVIDIA Jetson Nano Developer Kit - 4096MB

Ubuntu 18.04 - 4.9.140-tegra - Unity 7.5.0

5 Systems - 71 Benchmark Results

ARMv8 rev 1 - NVIDIA Jetson Nano Developer Kit - 4096MB

Ubuntu 18.04 - 4.9.140-tegra - Unity 7.5.0

5 Systems - 71 Benchmark Results

ARMv8 rev 1 - jetson-nano - 4096MB

Ubuntu 18.04 - 4.9.140-tegra - Unity 7.5.0

4 Systems - 71 Benchmark Results

ARMv8 rev 1 - NVIDIA Jetson Nano Developer Kit - 4096MB

Ubuntu 18.04 - 4.9.140-tegra - Unity 7.5.0

3 Systems - 44 Benchmark Results

ARMv8 rev 1 - jetson-nano - 4096MB

Ubuntu 18.04 - 4.9.140-tegra - Unity 7.5.0

9 Systems - 77 Benchmark Results

ARMv7 rev 4 - BCM2835 Raspberry Pi 3 Model B Plus Rev 1.3 - 926MB

Raspbian 9.6 - 4.19.23-v7+ - LXDE

1 System - 71 Benchmark Results

ARMv8 rev 1 - jetson-nano - 4096MB

Ubuntu 18.04 - 4.9.140-tegra - Unity 7.5.0

7 Systems - 38 Benchmark Results

ARMv8 rev 1 - jetson_tx1 - 4096MB

Ubuntu 16.04 - 4.4.38-tegra - Unity 7.4.5

6 Systems - 38 Benchmark Results

ARMv7 rev 1 - Rockchip - 2048MB

Debian 9.0 - 4.4.16-00006-g4431f98-dirty - LXDE

5 Systems - 38 Benchmark Results

ARMv8 rev 3 - quill - 8192MB

Ubuntu 16.04 - 4.4.38-tegra - Unity 7.4.0

Most Popular Test Results

OpenBenchmarking.org Results Compare

8 Systems - 76 Benchmark Results

ARMv7 rev 4 - BCM2835 Raspberry Pi 3 Model B Plus Rev 1.3 - 926MB

Raspbian 9.6 - 4.19.23-v7+ - LXDE

1 System - 2 Benchmark Results

ARMv8 rev 0 - Jetson-AGX - 16GB

Ubuntu 18.04 - 4.9.140-tegra - GNOME Shell 3.28.4

9 Systems - 77 Benchmark Results

ARMv8 Cortex-A73 - Hardkernel ODROID-N2 - 4096MB

Ubuntu 18.04 - 4.9.156-14 - GCC 7.3.0

1 System - 71 Benchmark Results

ARMv8 rev 1 - jetson-nano - 4096MB

Ubuntu 18.04 - 4.9.140-tegra - Unity 7.5.0

4 Systems - 71 Benchmark Results

ARMv8 rev 1 - NVIDIA Jetson Nano Developer Kit - 4096MB

Ubuntu 18.04 - 4.9.140-tegra - Unity 7.5.0

1 System - 49 Benchmark Results

ARMv8 rev 0 - jetson-xavier - 16384MB

Ubuntu 18.04 - 4.9.108-tegra - Unity 7.5.0

5 Systems - 71 Benchmark Results

Intel Core i9-9820X - ASUS ROG STRIX X299-XE GAMING - Intel Sky Lake-E DMI3 Registers

Ubuntu 18.04 - 4.15.0-55-generic - X Server 1.19.6

10 Systems - 77 Benchmark Results

ARMv8 Cortex-A73 - Hardkernel ODROID-N2 - 4096MB

Ubuntu 18.04 - 4.9.156-14 - GCC 7.3.0

5 Systems - 71 Benchmark Results

ARMv8 rev 1 - NVIDIA Jetson Nano Developer Kit - 4096MB

Ubuntu 18.04 - 4.9.140-tegra - Unity 7.5.0

3 Systems - 44 Benchmark Results

ARMv8 rev 1 - jetson-nano - 4096MB

Ubuntu 18.04 - 4.9.140-tegra - Unity 7.5.0

5 Systems - 71 Benchmark Results

ARMv8 rev 1 - NVIDIA Jetson Nano Developer Kit - 4096MB

Ubuntu 18.04 - 4.9.140-tegra - Unity 7.5.0

5 Systems - 71 Benchmark Results

ARMv8 rev 1 - NVIDIA Jetson Nano Developer Kit - 4096MB

Ubuntu 18.04 - 4.9.140-tegra - Unity 7.5.0

5 Systems - 38 Benchmark Results

ARMv8 rev 0 - jetson-xavier - 16384MB

Ubuntu 18.04 - 4.9.108-tegra - Unity 7.5.0

9 Systems - 76 Benchmark Results

ARMv8 rev 3 - quill - 8192MB

Ubuntu 16.04 - 4.4.38-tegra - Unity 7.4.0

Find More Test Results