Jetson AGX Xavier vs. Jetson TX2 TensorRT NVIDIA Jetson TensorRT inference benchmarks by Michael Larabel for a future article on Phoronix. Jetson AGX Xavier: Processor: ARMv8 rev 0 @ 2.27GHz (8 Cores), Motherboard: jetson-xavier, Memory: 16384MB, Disk: 31GB HBG4a2, Graphics: NVIDIA Tegra Xavier, Monitor: ASUS VP28U OS: Ubuntu 18.04, Kernel: 4.9.108-tegra (aarch64), Desktop: Unity 7.5.0, Display Server: X Server 1.19.6, Display Driver: NVIDIA 31.0.2, OpenGL: 4.6.0, Vulkan: 1.1.76, Compiler: GCC 7.3.0 + CUDA 10.0, File-System: ext4, Screen Resolution: 1920x1080 Jetson TX2: Processor: ARMv8 rev 3 @ 2.04GHz (4 Cores / 6 Threads), Motherboard: quill, Memory: 8192MB, Disk: 31GB 032G34, Graphics: NVIDIA Tegra X2, Monitor: VE228 OS: Ubuntu 16.04, Kernel: 4.4.38-tegra (aarch64), Desktop: Unity 7.4.0, Display Server: X Server 1.18.4, Display Driver: NVIDIA 28.2.1, OpenGL: 4.5.0, Compiler: GCC 5.4.0 20160609 + CUDA 9.0, File-System: ext4, Screen Resolution: 1920x1080 GLmark2 Resolution: 1920 x 1080 Score > Higher Is Better Jetson AGX Xavier . 2861 |===================================================== NVIDIA TensorRT Inference Neural Network: VGG16 - Precision: FP16 - Batch Size: 4 Images Per Second > Higher Is Better Jetson AGX Xavier . 195.45 |=================================================== Jetson TX2 ........ 32.30 |======== NVIDIA TensorRT Inference Neural Network: VGG16 - Precision: FP16 - Batch Size: 8 Images Per Second > Higher Is Better Jetson AGX Xavier . 215.68 |=================================================== Jetson TX2 ........ 33.68 |======== NVIDIA TensorRT Inference Neural Network: VGG16 - Precision: INT8 - Batch Size: 4 Images Per Second > Higher Is Better Jetson AGX Xavier . 286.64 |=================================================== Jetson TX2 ........ 17.98 |=== NVIDIA TensorRT Inference Neural Network: VGG16 - Precision: INT8 - Batch Size: 8 Images Per Second > Higher Is Better Jetson AGX Xavier . 341.20 |=================================================== Jetson TX2 ........ 19.89 |=== NVIDIA TensorRT Inference Neural Network: VGG19 - Precision: FP16 - Batch Size: 4 Images Per Second > Higher Is Better Jetson AGX Xavier . 172.15 |=================================================== Jetson TX2 ........ 26.49 |======== NVIDIA TensorRT Inference Neural Network: VGG19 - Precision: FP16 - Batch Size: 8 Images Per Second > Higher Is Better Jetson AGX Xavier . 184.43 |=================================================== Jetson TX2 ........ 27.02 |======= NVIDIA TensorRT Inference Neural Network: VGG19 - Precision: INT8 - Batch Size: 4 Images Per Second > Higher Is Better Jetson AGX Xavier . 262.17 |=================================================== Jetson TX2 ........ 14.62 |=== NVIDIA TensorRT Inference Neural Network: VGG19 - Precision: INT8 - Batch Size: 8 Images Per Second > Higher Is Better Jetson AGX Xavier . 296.94 |=================================================== Jetson TX2 ........ 16.02 |=== NVIDIA TensorRT Inference Neural Network: VGG16 - Precision: FP16 - Batch Size: 16 Images Per Second > Higher Is Better Jetson AGX Xavier . 228.75 |=================================================== Jetson TX2 ........ 36.44 |======== NVIDIA TensorRT Inference Neural Network: VGG16 - Precision: FP16 - Batch Size: 32 Images Per Second > Higher Is Better Jetson AGX Xavier . 246.76 |=================================================== Jetson TX2 ........ 37.24 |======== NVIDIA TensorRT Inference Neural Network: VGG16 - Precision: INT8 - Batch Size: 16 Images Per Second > Higher Is Better Jetson AGX Xavier . 381.33 |=================================================== Jetson TX2 ........ 20.50 |=== NVIDIA TensorRT Inference Neural Network: VGG16 - Precision: INT8 - Batch Size: 32 Images Per Second > Higher Is Better Jetson AGX Xavier . 449.96 |=================================================== Jetson TX2 ........ 19.87 |== NVIDIA TensorRT Inference Neural Network: VGG19 - Precision: FP16 - Batch Size: 16 Images Per Second > Higher Is Better Jetson AGX Xavier . 180.03 |=================================================== Jetson TX2 ........ 28.97 |======== NVIDIA TensorRT Inference Neural Network: VGG19 - Precision: FP16 - Batch Size: 32 Images Per Second > Higher Is Better Jetson AGX Xavier . 201.53 |=================================================== Jetson TX2 ........ 29.57 |======= NVIDIA TensorRT Inference Neural Network: VGG19 - Precision: INT8 - Batch Size: 16 Images Per Second > Higher Is Better Jetson AGX Xavier . 362.08 |=================================================== Jetson TX2 ........ 16.38 |== NVIDIA TensorRT Inference Neural Network: VGG19 - Precision: INT8 - Batch Size: 32 Images Per Second > Higher Is Better Jetson AGX Xavier . 390.57 |=================================================== Jetson TX2 ........ 15.99 |== NVIDIA TensorRT Inference Neural Network: AlexNet - Precision: FP16 - Batch Size: 4 Images Per Second > Higher Is Better Jetson AGX Xavier . 799 |====================================================== Jetson TX2 ........ 261 |================== NVIDIA TensorRT Inference Neural Network: AlexNet - Precision: FP16 - Batch Size: 8 Images Per Second > Higher Is Better Jetson AGX Xavier . 1247 |===================================================== Jetson TX2 ........ 300 |============= NVIDIA TensorRT Inference Neural Network: AlexNet - Precision: INT8 - Batch Size: 4 Images Per Second > Higher Is Better Jetson AGX Xavier . 975 |====================================================== Jetson TX2 ........ 179 |========== NVIDIA TensorRT Inference Neural Network: AlexNet - Precision: INT8 - Batch Size: 8 Images Per Second > Higher Is Better Jetson AGX Xavier . 1237 |===================================================== Jetson TX2 ........ 222 |========== NVIDIA TensorRT Inference Neural Network: AlexNet - Precision: FP16 - Batch Size: 16 Images Per Second > Higher Is Better Jetson AGX Xavier . 1435 |===================================================== Jetson TX2 ........ 370 |============== NVIDIA TensorRT Inference Neural Network: AlexNet - Precision: FP16 - Batch Size: 32 Images Per Second > Higher Is Better Jetson AGX Xavier . 1900 |===================================================== Jetson TX2 ........ 472 |============= NVIDIA TensorRT Inference Neural Network: AlexNet - Precision: INT8 - Batch Size: 16 Images Per Second > Higher Is Better Jetson AGX Xavier . 1879 |===================================================== Jetson TX2 ........ 258 |======= NVIDIA TensorRT Inference Neural Network: AlexNet - Precision: INT8 - Batch Size: 32 Images Per Second > Higher Is Better Jetson AGX Xavier . 2666 |===================================================== Jetson TX2 ........ 307 |====== NVIDIA TensorRT Inference Neural Network: ResNet50 - Precision: FP16 - Batch Size: 4 Images Per Second > Higher Is Better Jetson AGX Xavier . 542.80 |=================================================== Jetson TX2 ........ 93.61 |========= NVIDIA TensorRT Inference Neural Network: ResNet50 - Precision: FP16 - Batch Size: 8 Images Per Second > Higher Is Better Jetson AGX Xavier . 582.36 |=================================================== Jetson TX2 ........ 99.05 |========= NVIDIA TensorRT Inference Neural Network: ResNet50 - Precision: INT8 - Batch Size: 4 Images Per Second > Higher Is Better Jetson AGX Xavier . 865.46 |=================================================== Jetson TX2 ........ 50.39 |=== NVIDIA TensorRT Inference Neural Network: ResNet50 - Precision: INT8 - Batch Size: 8 Images Per Second > Higher Is Better Jetson AGX Xavier . 944.46 |=================================================== Jetson TX2 ........ 51.07 |=== NVIDIA TensorRT Inference Neural Network: GoogleNet - Precision: FP16 - Batch Size: 4 Images Per Second > Higher Is Better Jetson AGX Xavier . 546 |====================================================== Jetson TX2 ........ 202 |==================== NVIDIA TensorRT Inference Neural Network: GoogleNet - Precision: FP16 - Batch Size: 8 Images Per Second > Higher Is Better Jetson AGX Xavier . 863 |====================================================== Jetson TX2 ........ 198 |============ NVIDIA TensorRT Inference Neural Network: GoogleNet - Precision: INT8 - Batch Size: 4 Images Per Second > Higher Is Better Jetson AGX Xavier . 652 |====================================================== Jetson TX2 ........ 114 |========= NVIDIA TensorRT Inference Neural Network: GoogleNet - Precision: INT8 - Batch Size: 8 Images Per Second > Higher Is Better Jetson AGX Xavier . 1049 |===================================================== Jetson TX2 ........ 117 |====== NVIDIA TensorRT Inference Neural Network: ResNet152 - Precision: FP16 - Batch Size: 4 Images Per Second > Higher Is Better Jetson AGX Xavier . 219.08 |=================================================== Jetson TX2 ........ 35.60 |======== NVIDIA TensorRT Inference Neural Network: ResNet152 - Precision: FP16 - Batch Size: 8 Images Per Second > Higher Is Better Jetson AGX Xavier . 234.84 |=================================================== Jetson TX2 ........ 36.71 |======== NVIDIA TensorRT Inference Neural Network: ResNet152 - Precision: INT8 - Batch Size: 4 Images Per Second > Higher Is Better Jetson AGX Xavier . 350.28 |=================================================== Jetson TX2 ........ 17.97 |=== NVIDIA TensorRT Inference Neural Network: ResNet152 - Precision: INT8 - Batch Size: 8 Images Per Second > Higher Is Better Jetson AGX Xavier . 407.01 |=================================================== Jetson TX2 ........ 19.48 |== NVIDIA TensorRT Inference Neural Network: ResNet50 - Precision: FP16 - Batch Size: 16 Images Per Second > Higher Is Better Jetson AGX Xavier . 593 |====================================================== Jetson TX2 ........ 106 |========== NVIDIA TensorRT Inference Neural Network: ResNet50 - Precision: FP16 - Batch Size: 32 Images Per Second > Higher Is Better Jetson AGX Xavier . 613 |====================================================== Jetson TX2 ........ 110 |========== NVIDIA TensorRT Inference Neural Network: ResNet50 - Precision: INT8 - Batch Size: 16 Images Per Second > Higher Is Better Jetson AGX Xavier . 1106.13 |================================================== Jetson TX2 ........ 57.18 |=== NVIDIA TensorRT Inference Neural Network: ResNet50 - Precision: INT8 - Batch Size: 32 Images Per Second > Higher Is Better Jetson AGX Xavier . 1184.50 |================================================== Jetson TX2 ........ 59.45 |=== NVIDIA TensorRT Inference Neural Network: GoogleNet - Precision: FP16 - Batch Size: 16 Images Per Second > Higher Is Better Jetson AGX Xavier . 858 |====================================================== Jetson TX2 ........ 218 |============== NVIDIA TensorRT Inference Neural Network: GoogleNet - Precision: FP16 - Batch Size: 32 Images Per Second > Higher Is Better Jetson AGX Xavier . 956 |====================================================== Jetson TX2 ........ 230 |============= NVIDIA TensorRT Inference Neural Network: GoogleNet - Precision: INT8 - Batch Size: 16 Images Per Second > Higher Is Better Jetson AGX Xavier . 1340 |===================================================== Jetson TX2 ........ 125 |===== NVIDIA TensorRT Inference Neural Network: GoogleNet - Precision: INT8 - Batch Size: 32 Images Per Second > Higher Is Better Jetson AGX Xavier . 1622 |===================================================== Jetson TX2 ........ 130 |==== NVIDIA TensorRT Inference Neural Network: ResNet152 - Precision: FP16 - Batch Size: 16 Images Per Second > Higher Is Better Jetson AGX Xavier . 224.60 |=================================================== Jetson TX2 ........ 40.19 |========= NVIDIA TensorRT Inference Neural Network: ResNet152 - Precision: FP16 - Batch Size: 32 Images Per Second > Higher Is Better Jetson AGX Xavier . 253.34 |=================================================== Jetson TX2 ........ 41.87 |======== NVIDIA TensorRT Inference Neural Network: ResNet152 - Precision: INT8 - Batch Size: 16 Images Per Second > Higher Is Better Jetson AGX Xavier . 445.22 |=================================================== Jetson TX2 ........ 20.77 |== NVIDIA TensorRT Inference Neural Network: ResNet152 - Precision: INT8 - Batch Size: 32 Images Per Second > Higher Is Better Jetson AGX Xavier . 485.22 |=================================================== Jetson TX2 ........ 22.05 |==