NVIDIA TensorRT Inference

This test profile uses any existing system installation of NVIDIA TensorRT for carrying out inference benchmarks with various neural networks.

This test profile requires you have already installed TensorRT to /usr/src/tensorrt.

To run this test with the Phoronix Test Suite, the basic command is: phoronix-test-suite benchmark system/tensorrt-inference.

Use with caution this test profile is currently marked Experimental.

Test Created

23 December 2018

Last Updated

23 December 2018

Test Maintainer

Michael Larabel 

Test Type

Graphics

Average Install Time

4 Seconds

Average Run Time

7 Minutes, 34 Seconds

Accolades

5k+ Downloads

Supported Platforms


Public Result UploadsReported Installs*Test Completions*OpenBenchmarking.orgEventsNVIDIA TensorRT Inference Popularity Statisticssystem/tensorrt-inference2018.122019.012019.032019.042019.052019.062019.072019.082019.092019.112020.012020.042020.052020.08130260390520650
* Data based on those opting to upload their test results to OpenBenchmarking.org and users enabling the opt-in anonymous statistics reporting while running benchmarks from an Internet-connected platform.
Data current as of Thu, 15 Apr 2021 23:19:24 GMT.
ResNet15214.9%VGG197.7%VGG169.8%AlexNet22.6%ResNet5023.5%GoogleNet21.5%Neural Network Option PopularityOpenBenchmarking.org
FP1658.9%INT841.1%Precision Option PopularityOpenBenchmarking.org
115.3%819.9%425.2%3222.7%1616.8%Batch Size Option PopularityOpenBenchmarking.org

Revision History

system/tensorrt-inference-1.0.0   [View Source]   Sun, 23 Dec 2018 18:52:47 GMT
Initial commit of NVIDIA TensorRT benchmark from the Jetson Xavier refeenence guide.


Performance Metrics

Analyze Test Configuration:

NVIDIA TensorRT Inference

Neural Network: ResNet50 - Precision: FP16 - Batch Size: 32 - DLA Cores: Disabled

OpenBenchmarking.org metrics for this test profile configuration based on 821 public results since 23 December 2018 with the latest data as of 8 May 2020.

Additional benchmark metrics will come after OpenBenchmarking.org has collected a sufficient data-set.

OpenBenchmarking.orgDistribution Of Public Results - Neural Network: ResNet50 - Precision: FP16 - Batch Size: 32 - DLA Cores: Disabled821 Results Range From 32 To 5649 Images Per Second32145258371484597710823936104911621275138815011614172718401953206621792292240525182631274428572970308331963309342235353648376138743987410042134326443945524665477848915004511752305343545655695682130260390520650

Based on OpenBenchmarking.org data, the selected test / test configuration (NVIDIA TensorRT Inference - Neural Network: ResNet50 - Precision: FP16 - Batch Size: 32 - DLA Cores: Disabled) has an average run-time of 9 minutes. By default this test profile is set to run at least 3 times but may increase if the standard deviation exceeds pre-defined defaults or other calculations deem additional runs necessary for greater statistical accuracy of the result.

OpenBenchmarking.orgMinutesTime Required To Complete BenchmarkNeural Network: ResNet50 - Precision: FP16 - Batch Size: 32 - DLA Cores: DisabledRun-Time48121620Min: 1 / Avg: 8.55 / Max: 15

Recent Test Results

OpenBenchmarking.org Results Compare

1 System - 2 Benchmark Results

ARMv8 rev 0 - Jetson-AGX - 16GB

Ubuntu 18.04 - 4.9.140-tegra - GNOME Shell 3.28.4

1 System - 1 Benchmark Result

Intel Xeon Gold 6126 - Intel 440BX - Intel 440BX

Ubuntu 16.04 - 4.4.0-177-generic - X Server 1.18.4

9 Systems - 76 Benchmark Results

ARMv8 rev 0 - jetson-xavier - 16384MB

Ubuntu 18.04 - 4.9.108-tegra - Unity 7.5.0

5 Systems - 71 Benchmark Results

ARMv8 rev 1 - NVIDIA Jetson Nano Developer Kit - 4096MB

Ubuntu 18.04 - 4.9.140-tegra - Unity 7.5.0

5 Systems - 71 Benchmark Results

ARMv8 rev 1 - NVIDIA Jetson Nano Developer Kit - 4096MB

Ubuntu 18.04 - 4.9.140-tegra - Unity 7.5.0

5 Systems - 71 Benchmark Results

ARMv8 rev 1 - NVIDIA Jetson Nano Developer Kit - 4096MB

Ubuntu 18.04 - 4.9.140-tegra - Unity 7.5.0

5 Systems - 71 Benchmark Results

Intel Core i9-9820X - ASUS ROG STRIX X299-XE GAMING - Intel Sky Lake-E DMI3 Registers

Ubuntu 18.04 - 4.15.0-55-generic - X Server 1.19.6

4 Systems - 71 Benchmark Results

ARMv8 rev 1 - NVIDIA Jetson Nano Developer Kit - 4096MB

Ubuntu 18.04 - 4.9.140-tegra - Unity 7.5.0

3 Systems - 44 Benchmark Results

ARMv8 rev 1 - jetson-nano - 4096MB

Ubuntu 18.04 - 4.9.140-tegra - Unity 7.5.0

9 Systems - 77 Benchmark Results

ARMv7 rev 4 - BCM2835 Raspberry Pi 3 Model B Plus Rev 1.3 - 926MB

Raspbian 9.6 - 4.19.23-v7+ - LXDE

1 System - 71 Benchmark Results

ARMv8 rev 1 - jetson-nano - 4096MB

Ubuntu 18.04 - 4.9.140-tegra - Unity 7.5.0

Most Popular Test Results

OpenBenchmarking.org Results Compare

8 Systems - 76 Benchmark Results

ARMv8 rev 3 - quill - 8192MB

Ubuntu 16.04 - 4.4.38-tegra - Unity 7.4.0

1 System - 71 Benchmark Results

ARMv8 rev 1 - jetson-nano - 4096MB

Ubuntu 18.04 - 4.9.140-tegra - Unity 7.5.0

9 Systems - 77 Benchmark Results

ARMv8 rev 3 - quill - 8192MB

Ubuntu 16.04 - 4.4.38-tegra - Unity 7.4.0

1 System - 49 Benchmark Results

ARMv8 rev 0 - jetson-xavier - 16384MB

Ubuntu 18.04 - 4.9.108-tegra - Unity 7.5.0

1 System - 2 Benchmark Results

ARMv8 rev 0 - Jetson-AGX - 16GB

Ubuntu 18.04 - 4.9.140-tegra - GNOME Shell 3.28.4

4 Systems - 71 Benchmark Results

ARMv8 rev 1 - NVIDIA Jetson Nano Developer Kit - 4096MB

Ubuntu 18.04 - 4.9.140-tegra - Unity 7.5.0

10 Systems - 77 Benchmark Results

ARMv8 rev 0 - jetson-xavier - 16384MB

Ubuntu 18.04 - 4.9.108-tegra - Unity 7.5.0

5 Systems - 71 Benchmark Results

ARMv8 rev 1 - NVIDIA Jetson Nano Developer Kit - 4096MB

Ubuntu 18.04 - 4.9.140-tegra - Unity 7.5.0

5 Systems - 71 Benchmark Results

ARMv8 rev 1 - jetson-nano - 4096MB

Ubuntu 18.04 - 4.9.140-tegra - Unity 7.5.0

5 Systems - 71 Benchmark Results

ARMv8 rev 1 - jetson-nano - 4096MB

Ubuntu 18.04 - 4.9.140-tegra - Unity 7.5.0

9 Systems - 76 Benchmark Results

ARMv8 rev 1 - jetson_tx1 - 4096MB

Ubuntu 16.04 - 4.4.38-tegra - Unity 7.4.5

Find More Test Results