NVIDIA Jetson TensorRT inference benchmarks by Michael Larabel for a future article on Phoronix.
Compare your own system(s) to this result file with the
Phoronix Test Suite by running the command:
phoronix-test-suite benchmark 1812240-SP-XAVIER80657
Jetson AGX Xavier vs. Jetson TX2 TensorRT
NVIDIA Jetson TensorRT inference benchmarks by Michael Larabel for a future article on Phoronix.
Jetson AGX Xavier:
Processor: ARMv8 rev 0 @ 2.27GHz (8 Cores), Motherboard: jetson-xavier, Memory: 16384MB, Disk: 31GB HBG4a2, Graphics: NVIDIA Tegra Xavier, Monitor: ASUS VP28U
OS: Ubuntu 18.04, Kernel: 4.9.108-tegra (aarch64), Desktop: Unity 7.5.0, Display Server: X Server 1.19.6, Display Driver: NVIDIA 31.0.2, OpenGL: 4.6.0, Vulkan: 1.1.76, Compiler: GCC 7.3.0 + CUDA 10.0, File-System: ext4, Screen Resolution: 1920x1080
Jetson TX2:
Processor: ARMv8 rev 3 @ 2.04GHz (4 Cores / 6 Threads), Motherboard: quill, Memory: 8192MB, Disk: 31GB 032G34, Graphics: NVIDIA Tegra X2, Monitor: VE228
OS: Ubuntu 16.04, Kernel: 4.4.38-tegra (aarch64), Desktop: Unity 7.4.0, Display Server: X Server 1.18.4, Display Driver: NVIDIA 28.2.1, OpenGL: 4.5.0, Compiler: GCC 5.4.0 20160609 + CUDA 9.0, File-System: ext4, Screen Resolution: 1920x1080
NVIDIA TensorRT Inference
Neural Network: ResNet152 - Precision: FP16 - Batch Size: 8
Images Per Second > Higher Is Better
Jetson AGX Xavier . 234.84 |===================================================
Jetson TX2 ........ 36.71 |========
NVIDIA TensorRT Inference
Neural Network: VGG16 - Precision: INT8 - Batch Size: 32
Images Per Second > Higher Is Better
Jetson AGX Xavier . 449.96 |===================================================
Jetson TX2 ........ 19.87 |==
NVIDIA TensorRT Inference
Neural Network: ResNet152 - Precision: INT8 - Batch Size: 32
Images Per Second > Higher Is Better
Jetson AGX Xavier . 485.22 |===================================================
Jetson TX2 ........ 22.05 |==
NVIDIA TensorRT Inference
Neural Network: ResNet152 - Precision: INT8 - Batch Size: 16
Images Per Second > Higher Is Better
Jetson AGX Xavier . 445.22 |===================================================
Jetson TX2 ........ 20.77 |==
NVIDIA TensorRT Inference
Neural Network: ResNet152 - Precision: FP16 - Batch Size: 16
Images Per Second > Higher Is Better
Jetson AGX Xavier . 224.60 |===================================================
Jetson TX2 ........ 40.19 |=========
NVIDIA TensorRT Inference
Neural Network: ResNet50 - Precision: INT8 - Batch Size: 16
Images Per Second > Higher Is Better
Jetson AGX Xavier . 1106.13 |==================================================
Jetson TX2 ........ 57.18 |===
NVIDIA TensorRT Inference
Neural Network: ResNet152 - Precision: INT8 - Batch Size: 8
Images Per Second > Higher Is Better
Jetson AGX Xavier . 407.01 |===================================================
Jetson TX2 ........ 19.48 |==
NVIDIA TensorRT Inference
Neural Network: ResNet50 - Precision: INT8 - Batch Size: 8
Images Per Second > Higher Is Better
Jetson AGX Xavier . 944.46 |===================================================
Jetson TX2 ........ 51.07 |===
NVIDIA TensorRT Inference
Neural Network: VGG16 - Precision: INT8 - Batch Size: 16
Images Per Second > Higher Is Better
Jetson AGX Xavier . 381.33 |===================================================
Jetson TX2 ........ 20.50 |===
NVIDIA TensorRT Inference
Neural Network: VGG19 - Precision: INT8 - Batch Size: 32
Images Per Second > Higher Is Better
Jetson AGX Xavier . 390.57 |===================================================
Jetson TX2 ........ 15.99 |==
NVIDIA TensorRT Inference
Neural Network: ResNet152 - Precision: INT8 - Batch Size: 4
Images Per Second > Higher Is Better
Jetson AGX Xavier . 350.28 |===================================================
Jetson TX2 ........ 17.97 |===
NVIDIA TensorRT Inference
Neural Network: GoogleNet - Precision: INT8 - Batch Size: 4
Images Per Second > Higher Is Better
Jetson AGX Xavier . 652 |======================================================
Jetson TX2 ........ 114 |=========
NVIDIA TensorRT Inference
Neural Network: ResNet152 - Precision: FP16 - Batch Size: 32
Images Per Second > Higher Is Better
Jetson AGX Xavier . 253.34 |===================================================
Jetson TX2 ........ 41.87 |========
NVIDIA TensorRT Inference
Neural Network: VGG19 - Precision: FP16 - Batch Size: 16
Images Per Second > Higher Is Better
Jetson AGX Xavier . 180.03 |===================================================
Jetson TX2 ........ 28.97 |========
NVIDIA TensorRT Inference
Neural Network: VGG19 - Precision: FP16 - Batch Size: 32
Images Per Second > Higher Is Better
Jetson AGX Xavier . 201.53 |===================================================
Jetson TX2 ........ 29.57 |=======
NVIDIA TensorRT Inference
Neural Network: GoogleNet - Precision: INT8 - Batch Size: 8
Images Per Second > Higher Is Better
Jetson AGX Xavier . 1049 |=====================================================
Jetson TX2 ........ 117 |======
NVIDIA TensorRT Inference
Neural Network: GoogleNet - Precision: INT8 - Batch Size: 16
Images Per Second > Higher Is Better
Jetson AGX Xavier . 1340 |=====================================================
Jetson TX2 ........ 125 |=====
NVIDIA TensorRT Inference
Neural Network: VGG16 - Precision: FP16 - Batch Size: 32
Images Per Second > Higher Is Better
Jetson AGX Xavier . 246.76 |===================================================
Jetson TX2 ........ 37.24 |========
NVIDIA TensorRT Inference
Neural Network: VGG19 - Precision: INT8 - Batch Size: 16
Images Per Second > Higher Is Better
Jetson AGX Xavier . 362.08 |===================================================
Jetson TX2 ........ 16.38 |==
NVIDIA TensorRT Inference
Neural Network: ResNet152 - Precision: FP16 - Batch Size: 4
Images Per Second > Higher Is Better
Jetson AGX Xavier . 219.08 |===================================================
Jetson TX2 ........ 35.60 |========
NVIDIA TensorRT Inference
Neural Network: VGG16 - Precision: FP16 - Batch Size: 4
Images Per Second > Higher Is Better
Jetson AGX Xavier . 195.45 |===================================================
Jetson TX2 ........ 32.30 |========
NVIDIA TensorRT Inference
Neural Network: GoogleNet - Precision: FP16 - Batch Size: 32
Images Per Second > Higher Is Better
Jetson AGX Xavier . 956 |======================================================
Jetson TX2 ........ 230 |=============
NVIDIA TensorRT Inference
Neural Network: ResNet50 - Precision: INT8 - Batch Size: 32
Images Per Second > Higher Is Better
Jetson AGX Xavier . 1184.50 |==================================================
Jetson TX2 ........ 59.45 |===
NVIDIA TensorRT Inference
Neural Network: AlexNet - Precision: FP16 - Batch Size: 16
Images Per Second > Higher Is Better
Jetson AGX Xavier . 1435 |=====================================================
Jetson TX2 ........ 370 |==============
NVIDIA TensorRT Inference
Neural Network: AlexNet - Precision: FP16 - Batch Size: 8
Images Per Second > Higher Is Better
Jetson AGX Xavier . 1247 |=====================================================
Jetson TX2 ........ 300 |=============
NVIDIA TensorRT Inference
Neural Network: VGG16 - Precision: FP16 - Batch Size: 16
Images Per Second > Higher Is Better
Jetson AGX Xavier . 228.75 |===================================================
Jetson TX2 ........ 36.44 |========
NVIDIA TensorRT Inference
Neural Network: VGG19 - Precision: INT8 - Batch Size: 8
Images Per Second > Higher Is Better
Jetson AGX Xavier . 296.94 |===================================================
Jetson TX2 ........ 16.02 |===
GLmark2
Resolution: 1920 x 1080
Score > Higher Is Better
Jetson AGX Xavier . 2861 |=====================================================
NVIDIA TensorRT Inference
Neural Network: AlexNet - Precision: FP16 - Batch Size: 4
Images Per Second > Higher Is Better
Jetson AGX Xavier . 799 |======================================================
Jetson TX2 ........ 261 |==================
NVIDIA TensorRT Inference
Neural Network: GoogleNet - Precision: INT8 - Batch Size: 32
Images Per Second > Higher Is Better
Jetson AGX Xavier . 1622 |=====================================================
Jetson TX2 ........ 130 |====
NVIDIA TensorRT Inference
Neural Network: VGG16 - Precision: INT8 - Batch Size: 8
Images Per Second > Higher Is Better
Jetson AGX Xavier . 341.20 |===================================================
Jetson TX2 ........ 19.89 |===
NVIDIA TensorRT Inference
Neural Network: VGG19 - Precision: INT8 - Batch Size: 4
Images Per Second > Higher Is Better
Jetson AGX Xavier . 262.17 |===================================================
Jetson TX2 ........ 14.62 |===
NVIDIA TensorRT Inference
Neural Network: VGG16 - Precision: FP16 - Batch Size: 8
Images Per Second > Higher Is Better
Jetson AGX Xavier . 215.68 |===================================================
Jetson TX2 ........ 33.68 |========
NVIDIA TensorRT Inference
Neural Network: VGG19 - Precision: FP16 - Batch Size: 8
Images Per Second > Higher Is Better
Jetson AGX Xavier . 184.43 |===================================================
Jetson TX2 ........ 27.02 |=======
NVIDIA TensorRT Inference
Neural Network: GoogleNet - Precision: FP16 - Batch Size: 8
Images Per Second > Higher Is Better
Jetson AGX Xavier . 863 |======================================================
Jetson TX2 ........ 198 |============
NVIDIA TensorRT Inference
Neural Network: VGG16 - Precision: INT8 - Batch Size: 4
Images Per Second > Higher Is Better
Jetson AGX Xavier . 286.64 |===================================================
Jetson TX2 ........ 17.98 |===
NVIDIA TensorRT Inference
Neural Network: GoogleNet - Precision: FP16 - Batch Size: 16
Images Per Second > Higher Is Better
Jetson AGX Xavier . 858 |======================================================
Jetson TX2 ........ 218 |==============
NVIDIA TensorRT Inference
Neural Network: ResNet50 - Precision: INT8 - Batch Size: 4
Images Per Second > Higher Is Better
Jetson AGX Xavier . 865.46 |===================================================
Jetson TX2 ........ 50.39 |===
NVIDIA TensorRT Inference
Neural Network: ResNet50 - Precision: FP16 - Batch Size: 32
Images Per Second > Higher Is Better
Jetson AGX Xavier . 613 |======================================================
Jetson TX2 ........ 110 |==========
NVIDIA TensorRT Inference
Neural Network: AlexNet - Precision: INT8 - Batch Size: 16
Images Per Second > Higher Is Better
Jetson AGX Xavier . 1879 |=====================================================
Jetson TX2 ........ 258 |=======
NVIDIA TensorRT Inference
Neural Network: VGG19 - Precision: FP16 - Batch Size: 4
Images Per Second > Higher Is Better
Jetson AGX Xavier . 172.15 |===================================================
Jetson TX2 ........ 26.49 |========
NVIDIA TensorRT Inference
Neural Network: AlexNet - Precision: INT8 - Batch Size: 4
Images Per Second > Higher Is Better
Jetson AGX Xavier . 975 |======================================================
Jetson TX2 ........ 179 |==========
NVIDIA TensorRT Inference
Neural Network: AlexNet - Precision: INT8 - Batch Size: 8
Images Per Second > Higher Is Better
Jetson AGX Xavier . 1237 |=====================================================
Jetson TX2 ........ 222 |==========
NVIDIA TensorRT Inference
Neural Network: GoogleNet - Precision: FP16 - Batch Size: 4
Images Per Second > Higher Is Better
Jetson AGX Xavier . 546 |======================================================
Jetson TX2 ........ 202 |====================
NVIDIA TensorRT Inference
Neural Network: AlexNet - Precision: INT8 - Batch Size: 32
Images Per Second > Higher Is Better
Jetson AGX Xavier . 2666 |=====================================================
Jetson TX2 ........ 307 |======
NVIDIA TensorRT Inference
Neural Network: ResNet50 - Precision: FP16 - Batch Size: 16
Images Per Second > Higher Is Better
Jetson AGX Xavier . 593 |======================================================
Jetson TX2 ........ 106 |==========
NVIDIA TensorRT Inference
Neural Network: ResNet50 - Precision: FP16 - Batch Size: 8
Images Per Second > Higher Is Better
Jetson AGX Xavier . 582.36 |===================================================
Jetson TX2 ........ 99.05 |=========
NVIDIA TensorRT Inference
Neural Network: ResNet50 - Precision: FP16 - Batch Size: 4
Images Per Second > Higher Is Better
Jetson AGX Xavier . 542.80 |===================================================
Jetson TX2 ........ 93.61 |=========
NVIDIA TensorRT Inference
Neural Network: AlexNet - Precision: FP16 - Batch Size: 32
Images Per Second > Higher Is Better
Jetson AGX Xavier . 1900 |=====================================================
Jetson TX2 ........ 472 |=============