NVIDIA Jetson TensorRT inference benchmarks by Michael Larabel for a future article on Phoronix.
Compare your own system(s) to this result file with the
Phoronix Test Suite by running the command:
phoronix-test-suite benchmark 1812240-SP-XAVIER80657
Jetson AGX Xavier vs. Jetson TX2 TensorRT,
"NVIDIA TensorRT Inference - Neural Network: ResNet152 - Precision: FP16 - Batch Size: 8",
Higher Results Are Better
"Jetson AGX Xavier",235.389,234.161,234.972
"Jetson TX2",38.2142,36.3171,33.0004,38.0637,34.2227,38.0447,36.2206,37.1718,39.1236
"NVIDIA TensorRT Inference - Neural Network: VGG16 - Precision: INT8 - Batch Size: 32",
Higher Results Are Better
"Jetson AGX Xavier",434.032,475.004,467.46,438.753,440.977,437.515,458.152,469.276,437.313,441.104
"Jetson TX2",19.8959,19.9036,19.8092
"NVIDIA TensorRT Inference - Neural Network: ResNet152 - Precision: INT8 - Batch Size: 32",
Higher Results Are Better
"Jetson AGX Xavier",484.001,483.498,488.146
"Jetson TX2",22.1179,22.0195,22.0243
"NVIDIA TensorRT Inference - Neural Network: ResNet152 - Precision: INT8 - Batch Size: 16",
Higher Results Are Better
"Jetson AGX Xavier",449.461,449.069,437.135
"Jetson TX2",20.9278,20.6184,20.7703
"NVIDIA TensorRT Inference - Neural Network: ResNet152 - Precision: FP16 - Batch Size: 16",
Higher Results Are Better
"Jetson AGX Xavier",100.72,240.808,237.779,241.928,240.817,241.604,240.345,235.361,242.033
"Jetson TX2",40.0309,40.0167,40.5266
"NVIDIA TensorRT Inference - Neural Network: ResNet50 - Precision: INT8 - Batch Size: 16",
Higher Results Are Better
"Jetson AGX Xavier",1128.64,1134.57,998.558,1117.7,1100.5,1121.85,1131.89,1060.92,1122.85,1127.03,1132.96,1096.09
"Jetson TX2",57.3849,57.087,57.0597
"NVIDIA TensorRT Inference - Neural Network: ResNet152 - Precision: INT8 - Batch Size: 8",
Higher Results Are Better
"Jetson AGX Xavier",393.461,410.88,416.699
"Jetson TX2",19.3195,20.0127,19.1011
"NVIDIA TensorRT Inference - Neural Network: ResNet50 - Precision: INT8 - Batch Size: 8",
Higher Results Are Better
"Jetson AGX Xavier",926.954,529.426,924.711,938.152,1027.2,937.073,1003.68,1044.82,1040.98,940.792,1047.41,972.272
"Jetson TX2",50.2667,50.8547,52.0955
"NVIDIA TensorRT Inference - Neural Network: VGG16 - Precision: INT8 - Batch Size: 16",
Higher Results Are Better
"Jetson AGX Xavier",361.233,432.868,362.923,332.538,425.394,354.801,402.875,414.478,419.963,350.124,363.122,355.584
"Jetson TX2",20.4675,20.5536,20.4761
"NVIDIA TensorRT Inference - Neural Network: VGG19 - Precision: INT8 - Batch Size: 32",
Higher Results Are Better
"Jetson AGX Xavier",390.986,387.493,393.237
"Jetson TX2",16.0187,15.9212,16.0268
"NVIDIA TensorRT Inference - Neural Network: ResNet152 - Precision: INT8 - Batch Size: 4",
Higher Results Are Better
"Jetson AGX Xavier",340.794,350.244,359.79
"Jetson TX2",18.0162,17.6172,18.2694
"NVIDIA TensorRT Inference - Neural Network: GoogleNet - Precision: INT8 - Batch Size: 4",
Higher Results Are Better
"Jetson AGX Xavier",973.293,104.545,103.953,103.884,1112.76,1110.67,1025.81,935.826,1013.44,103.359,103.125,1131.48
"Jetson TX2",116.357,115.479,109.957
"NVIDIA TensorRT Inference - Neural Network: ResNet152 - Precision: FP16 - Batch Size: 32",
Higher Results Are Better
"Jetson AGX Xavier",252.719,258.532,248.76
"Jetson TX2",41.8944,41.6108,42.0955
"NVIDIA TensorRT Inference - Neural Network: VGG19 - Precision: FP16 - Batch Size: 16",
Higher Results Are Better
"Jetson AGX Xavier",182.274,75.4806,192.08,192.702,193.329,192.633,193.777,194.769,190.895,192.396
"Jetson TX2",29.1521,28.9793,28.7746
"NVIDIA TensorRT Inference - Neural Network: VGG19 - Precision: FP16 - Batch Size: 32",
Higher Results Are Better
"Jetson AGX Xavier",202.27,198.313,203.996
"Jetson TX2",29.5727,29.7182,29.4134
"NVIDIA TensorRT Inference - Neural Network: GoogleNet - Precision: INT8 - Batch Size: 8",
Higher Results Are Better
"Jetson AGX Xavier",1173.05,126.903,1368.3,1082.89,1147.9,1210.38,1344.25,1042.97,656.561,1339.12
"Jetson TX2",114.113,115.897,121.168
"NVIDIA TensorRT Inference - Neural Network: GoogleNet - Precision: INT8 - Batch Size: 16",
Higher Results Are Better
"Jetson AGX Xavier",1481.18,131.593,1468.08,1485.28,1374.18,1560.03,1523.93,1462.3,1571.88
"Jetson TX2",125.602,127.01,123.031
"NVIDIA TensorRT Inference - Neural Network: VGG16 - Precision: FP16 - Batch Size: 32",
Higher Results Are Better
"Jetson AGX Xavier",247.095,246.546,246.63
"Jetson TX2",37.3087,36.967,37.4418
"NVIDIA TensorRT Inference - Neural Network: VGG19 - Precision: INT8 - Batch Size: 16",
Higher Results Are Better
"Jetson AGX Xavier",361.253,363.39,361.589
"Jetson TX2",16.3819,16.3484,16.4111
"NVIDIA TensorRT Inference - Neural Network: ResNet152 - Precision: FP16 - Batch Size: 4",
Higher Results Are Better
"Jetson AGX Xavier",224.088,219.966,213.188
"Jetson TX2",35.9141,34.7304,36.1487
"NVIDIA TensorRT Inference - Neural Network: VGG16 - Precision: FP16 - Batch Size: 4",
Higher Results Are Better
"Jetson AGX Xavier",199.848,183.101,206.277,203.436,202.327,173.046,203.366,187.448,203.926,205.962,186.056,190.551
"Jetson TX2",31.9091,32.8772,32.1007
"NVIDIA TensorRT Inference - Neural Network: GoogleNet - Precision: FP16 - Batch Size: 32",
Higher Results Are Better
"Jetson AGX Xavier",1006.48,894.504,887.88,918.013,1004.86,916.926,1004.9,1000.57,997.118,916.908,917.184,1002.52
"Jetson TX2",223.991,230.116,236.412
"NVIDIA TensorRT Inference - Neural Network: ResNet50 - Precision: INT8 - Batch Size: 32",
Higher Results Are Better
"Jetson AGX Xavier",1196.9,1174.68,1181.91
"Jetson TX2",59.0993,59.7263,59.5318
"NVIDIA TensorRT Inference - Neural Network: AlexNet - Precision: FP16 - Batch Size: 16",
Higher Results Are Better
"Jetson AGX Xavier",1504.59,1618.24,1376.69,1514.09,1505.85,1611.21,756.73,1410.08,1615.87
"Jetson TX2",372.568,343.41,408.088,353.56,407.076,344.286,350.415,359.981,365.717,390.785,370.763,368.561
"NVIDIA TensorRT Inference - Neural Network: AlexNet - Precision: FP16 - Batch Size: 8",
Higher Results Are Better
"Jetson AGX Xavier",1029.36,1506.02,1350.71,1221.31,1237.1,1334.58,1049.59,1458.15,1187.43,1059.42,1360.18,1166.49
"Jetson TX2",331.843,280.246,339.851,307.701,317.25,309.823,293.491,292.257,281.437,293.702,240.392,311.603
"NVIDIA TensorRT Inference - Neural Network: VGG16 - Precision: FP16 - Batch Size: 16",
Higher Results Are Better
"Jetson AGX Xavier",228.166,226.249,231.822
"Jetson TX2",36.2766,36.4053,36.6417
"NVIDIA TensorRT Inference - Neural Network: VGG19 - Precision: INT8 - Batch Size: 8",
Higher Results Are Better
"Jetson AGX Xavier",297.063,294.427,299.324
"Jetson TX2",16.1185,15.9237,16.0058
"GLmark2 - Resolution: 1920 x 1080",
Higher Results Are Better
"Jetson AGX Xavier",
"NVIDIA TensorRT Inference - Neural Network: AlexNet - Precision: FP16 - Batch Size: 4",
Higher Results Are Better
"Jetson AGX Xavier",723.385,945.139,832.234,1096.96,781.035,825.038,800.16,1089.22,101.611
"Jetson TX2",286.537,219.218,279.777,261.57,252.484,271.049,290.215,259.933,252.461,234.314,256.744,262.978
"NVIDIA TensorRT Inference - Neural Network: GoogleNet - Precision: INT8 - Batch Size: 32",
Higher Results Are Better
"Jetson AGX Xavier",1631.75,1621.22,1614.41
"Jetson TX2",131.09,128.455,131.251
"NVIDIA TensorRT Inference - Neural Network: VGG16 - Precision: INT8 - Batch Size: 8",
Higher Results Are Better
"Jetson AGX Xavier",339.161,341.597,342.855
"Jetson TX2",19.8901,19.9772,19.8075
"NVIDIA TensorRT Inference - Neural Network: VGG19 - Precision: INT8 - Batch Size: 4",
Higher Results Are Better
"Jetson AGX Xavier",263.586,260.342,262.583
"Jetson TX2",14.6955,14.4303,14.7395
"NVIDIA TensorRT Inference - Neural Network: VGG16 - Precision: FP16 - Batch Size: 8",
Higher Results Are Better
"Jetson AGX Xavier",206.899,221.274,220.632,208.026,221.589
"Jetson TX2",33.5417,34.1463,33.3392
"NVIDIA TensorRT Inference - Neural Network: VGG19 - Precision: FP16 - Batch Size: 8",
Higher Results Are Better
"Jetson AGX Xavier",179.886,185.603,187.815
"Jetson TX2",26.9805,26.8051,27.2717
"NVIDIA TensorRT Inference - Neural Network: GoogleNet - Precision: FP16 - Batch Size: 8",
Higher Results Are Better
"Jetson AGX Xavier",900.865,896.414,808.19,828.476,908.149,903.879,894.246,900.453,909.774,809.1,819.995,779.18
"Jetson TX2",197.856,191.304,204.104
"NVIDIA TensorRT Inference - Neural Network: VGG16 - Precision: INT8 - Batch Size: 4",
Higher Results Are Better
"Jetson AGX Xavier",287.27,293.199,279.441
"Jetson TX2",18.0354,18.039,17.866
"NVIDIA TensorRT Inference - Neural Network: GoogleNet - Precision: FP16 - Batch Size: 16",
Higher Results Are Better
"Jetson AGX Xavier",875.997,881.999,940.961,864.711,960.142,433.131,857.467,974.38,932.216
"Jetson TX2",215.98,212.494,224.61
"NVIDIA TensorRT Inference - Neural Network: ResNet50 - Precision: INT8 - Batch Size: 4",
Higher Results Are Better
"Jetson AGX Xavier",837.095,878.241,881.04
"Jetson TX2",49.1266,51.2072,50.8294
"NVIDIA TensorRT Inference - Neural Network: ResNet50 - Precision: FP16 - Batch Size: 32",
Higher Results Are Better
"Jetson AGX Xavier",604.297,631.286,603.571
"Jetson TX2",111.491,107.912,112.023
"NVIDIA TensorRT Inference - Neural Network: AlexNet - Precision: INT8 - Batch Size: 16",
Higher Results Are Better
"Jetson AGX Xavier",2045.1,1453,2041.63,1468.37,1391.55,2231.34,1886.79,2030.88,2198.44,2256.73,1620.91,1929.12
"Jetson TX2",265.177,255.757,254.117
"NVIDIA TensorRT Inference - Neural Network: VGG19 - Precision: FP16 - Batch Size: 4",
Higher Results Are Better
"Jetson AGX Xavier",174.083,169.808,172.553
"Jetson TX2",26.2968,26.403,26.7799
"NVIDIA TensorRT Inference - Neural Network: AlexNet - Precision: INT8 - Batch Size: 4",
Higher Results Are Better
"Jetson AGX Xavier",1058.01,549.609,730.372,1042.96,1088.5,1096.02,1108.83,953.487,1119.83,1111.5,1105.79,737.273
"Jetson TX2",175.026,187.036,176.535,178.341
"NVIDIA TensorRT Inference - Neural Network: AlexNet - Precision: INT8 - Batch Size: 8",
Higher Results Are Better
"Jetson AGX Xavier",847.316,1576.79,1183.28,1638.77,1273.91,369.208,1363.35,1423.09,1275.05,1428.5,1098.82,1362.73
"Jetson TX2",216.231,227.346,222.994
"NVIDIA TensorRT Inference - Neural Network: GoogleNet - Precision: FP16 - Batch Size: 4",
Higher Results Are Better
"Jetson AGX Xavier",714.235,702.123,433.934,66.7753,709.577,72.5876,785.572,789.196,640.884
"Jetson TX2",200.641,203.406,200.901
"NVIDIA TensorRT Inference - Neural Network: AlexNet - Precision: INT8 - Batch Size: 32",
Higher Results Are Better
"Jetson AGX Xavier",734.103,2990.18,2864.25,3136.33,2590.51,2974.06,2659.49,3086.03,2955.66
"Jetson TX2",306.44,305.055,308.113
"NVIDIA TensorRT Inference - Neural Network: ResNet50 - Precision: FP16 - Batch Size: 16",
Higher Results Are Better
"Jetson AGX Xavier",579.402,601.968,598.628
"Jetson TX2",106.753,105.093,106.968
"NVIDIA TensorRT Inference - Neural Network: ResNet50 - Precision: FP16 - Batch Size: 8",
Higher Results Are Better
"Jetson AGX Xavier",582.823,582.043,582.225
"Jetson TX2",97.7503,101.499,97.8917
"NVIDIA TensorRT Inference - Neural Network: ResNet50 - Precision: FP16 - Batch Size: 4",
Higher Results Are Better
"Jetson AGX Xavier",543.173,542.021,543.196
"Jetson TX2",92.9919,91.4618,96.3904
"NVIDIA TensorRT Inference - Neural Network: AlexNet - Precision: FP16 - Batch Size: 32",
Higher Results Are Better
"Jetson AGX Xavier",1939.78,1902.27,1859.02
"Jetson TX2",475.6,458.81,481.259