TensorFlow Lite

This is a benchmark of the TensorFlow Lite implementation focused on TensorFlow machine learning for mobile, IoT, edge, and other cases. The current Linux support is limited to running on CPUs. This test profile is measuring the average inference time.

To run this test with the Phoronix Test Suite, the basic command is: phoronix-test-suite benchmark tensorflow-lite.

Project Site

tensorflow.org

Source Repository

github.com

Test Created

23 August 2020

Last Updated

19 May 2022

Test Maintainer

Michael Larabel 

Test Type

System

Average Install Time

9 Seconds

Average Run Time

4 Minutes, 53 Seconds

Accolades

10k+ Downloads

Supported Platforms


Public Result Uploads *Reported Installs **Reported Test Completions **Test Profile Page Views ***OpenBenchmarking.orgEventsTensorFlow Lite Popularity Statisticspts/tensorflow-lite2020.082020.102020.122021.022021.042021.062021.082021.102021.122022.022022.042022.062022.082022.102022.122023.022023.042023.062023.082023.102023.122024.025K10K15K20K25K
* Uploading of benchmark result data to OpenBenchmarking.org is always optional (opt-in) via the Phoronix Test Suite for users wishing to share their results publicly.
** Data based on those opting to upload their test results to OpenBenchmarking.org and users enabling the opt-in anonymous statistics reporting while running benchmarks from an Internet-connected platform.
*** Test profile page view reporting began March 2021.
Data updated weekly as of 26 March 2024.
Mobilenet Float19.6%NASNet Mobile13.6%Mobilenet Quant16.8%SqueezeNet17.2%Inception ResNet V215.5%Inception V417.3%Model Option PopularityOpenBenchmarking.org

Revision History

pts/tensorflow-lite-1.1.0   [View Source]   Thu, 19 May 2022 09:57:39 GMT
Update against latest upstream nightly.

pts/tensorflow-lite-1.0.0   [View Source]   Sun, 23 Aug 2020 14:13:10 GMT
TensorFlow Lite initial commit.

Suites Using This Test

Machine Learning

HPC - High Performance Computing


Performance Metrics

Analyze Test Configuration:

TensorFlow Lite 2020-08-23

Model: Inception ResNet V2

OpenBenchmarking.org metrics for this test profile configuration based on 1,506 public results since 23 August 2020 with the latest data as of 3 October 2023.

Below is an overview of the generalized performance for components where there is sufficient statistically significant data based upon user-uploaded results. It is important to keep in mind particularly in the Linux/open-source space there can be vastly different OS configurations, with this overview intended to offer just general guidance as to the performance expectations.

Component
Percentile Rank
# Compatible Public Results
Microseconds (Average)
100th
20
413215 +/- 31056
95th
27
576673 +/- 8854
94th
14
590380 +/- 50390
93rd
12
616622 +/- 4985
93rd
3
629609 +/- 37273
91st
3
693216 +/- 25653
89th
3
710851 +/- 30209
89th
10
712514 +/- 59513
87th
25
738404 +/- 81495
87th
21
742157 +/- 10612
87th
15
748954 +/- 17152
87th
5
752316 +/- 16452
85th
3
771272 +/- 3699
85th
7
773867 +/- 6436
84th
4
783144 +/- 3102
84th
3
783761 +/- 74
83rd
45
808932 +/- 79114
82nd
4
823239 +/- 17126
81st
19
834802 +/- 8751
80th
6
839110 +/- 1510
80th
10
842083 +/- 10627
80th
8
845935 +/- 4924
78th
7
872338 +/- 9172
77th
10
893295 +/- 2101
76th
12
902071 +/- 3707
76th
6
903662 +/- 2408
Mid-Tier
75th
> 905167
74th
4
941970 +/- 8190
74th
10
948359 +/- 26294
73rd
20
960268 +/- 102717
72nd
7
1015399 +/- 27906
71st
10
1028452 +/- 9435
70th
10
1076625 +/- 1944
69th
10
1116116 +/- 6778
69th
3
1116922 +/- 708
69th
3
1130599 +/- 166
68th
16
1140713 +/- 4133
67th
4
1157025 +/- 3764
67th
15
1160719 +/- 14921
66th
6
1188341 +/- 9223
63rd
52
1218641 +/- 11514
63rd
12
1223279 +/- 17547
61st
11
1311599 +/- 18849
60th
6
1330458 +/- 74
60th
3
1337465 +/- 5070
59th
18
1348022 +/- 3910
59th
3
1354931 +/- 6254
58th
5
1374502 +/- 1665
58th
3
1384993 +/- 1241
58th
8
1385010 +/- 30463
56th
12
1571309 +/- 7910
55th
39
1607556 +/- 13376
53rd
23
1623994 +/- 45263
52nd
10
1639392 +/- 7060
Median
50th
1757590
50th
6
1766181 +/- 177311
49th
9
1797925 +/- 11104
49th
13
1799772 +/- 132080
48th
12
1832973 +/- 13324
47th
3
2073899 +/- 16742
46th
15
2122281 +/- 8536
44th
29
2147470 +/- 15534
43rd
5
2195060 +/- 3733
43rd
18
2198704 +/- 57067
42nd
8
2258168 +/- 16637
41st
6
2318708 +/- 9939
41st
8
2356646 +/- 77981
40th
3
2378487 +/- 9457
40th
14
2392200 +/- 29637
40th
8
2402267 +/- 1343
39th
8
2457248 +/- 11996
38th
3
2557136 +/- 5957
37th
17
2656161 +/- 65836
37th
7
2677517 +/- 389671
35th
4
2843560
35th
8
2923336 +/- 1405
34th
14
2941698 +/- 130486
34th
7
2960190 +/- 231889
33rd
23
2993074 +/- 47849
33rd
4
3013723 +/- 53347
32nd
7
3019218 +/- 2648
30th
12
3142142 +/- 6415
29th
3
3268110 +/- 1072
28th
8
3321318 +/- 39711
28th
4
3382000 +/- 29222
28th
7
3400194 +/- 282159
28th
11
3414240 +/- 197816
28th
3
3415448 +/- 3867
27th
3
3459362 +/- 10855
Low-Tier
25th
> 3774800
25th
3
3777707 +/- 2332
25th
3
3781611 +/- 1933
24th
15
3895272 +/- 110256
23rd
3
3959302 +/- 11505
23rd
7
4062308 +/- 225935
23rd
5
4074975 +/- 424
23rd
18
4161008 +/- 202064
22nd
5
4169994 +/- 437
21st
14
4403892 +/- 33005
20th
14
4662799 +/- 48414
20th
6
4667686 +/- 8700
19th
11
4776477 +/- 6706
18th
4
4914079 +/- 50617
17th
6
4997320 +/- 36672
17th
7
5044910 +/- 8223
17th
3
5195028 +/- 11376
16th
3
5211162 +/- 52
16th
11
5265602 +/- 503700
16th
4
5305553 +/- 207541
15th
4
5390148 +/- 102304
15th
3
5417777 +/- 12455
15th
4
5484275 +/- 34631
14th
3
5851123 +/- 1227
14th
4
5870767 +/- 8482
14th
5
6029606 +/- 278739
14th
3
6181699 +/- 1037
13th
3
6259003 +/- 6450
13th
3
6369685 +/- 23995
13th
4
6490798 +/- 1056
12th
3
6518117 +/- 687
12th
4
6619217 +/- 789
11th
3
6764393 +/- 36314
11th
3
7042845 +/- 6315
11th
3
7065428 +/- 997
11th
4
7073255 +/- 11170
10th
42
7235302 +/- 379077
8th
3
7504205 +/- 2901
8th
3
7575542 +/- 1773
7th
4
7811175 +/- 20167
6th
4
8485642 +/- 407
6th
15
8608708 +/- 336705
6th
3
8650855 +/- 2272
5th
3
8994871 +/- 29228
5th
9
9010380 +/- 753543
5th
5
9069778 +/- 659011
4th
6
9353880 +/- 9652
3rd
3
11292700 +/- 264657
3rd
4
11974967 +/- 849
3rd
3
13315667 +/- 15674
3rd
3
14999222 +/- 700
2nd
4
15930567 +/- 5075
2nd
5
17693280 +/- 18145
2nd
3
18852389 +/- 703
1st
3
19548311 +/- 4409
1st
4
21335358 +/- 2008
OpenBenchmarking.orgDistribution Of Public Results - Model: Inception ResNet V21506 Results Range From 382316 To 31032833 Microseconds38231699532716083382221349283436034473714060382467339352864045899415651242671254377738448835145989644709577481101904921080350311416514120295251264253613255547138685581448156915094580157075911632060216933613175466241815963518772646193856571999866820611679212246902183770122450712230637232367673424289745249027562551576726128778267417892735480027967811285808222919383329806844304198553103286690180270360450

Based on OpenBenchmarking.org data, the selected test / test configuration (TensorFlow Lite 2020-08-23 - Model: Inception ResNet V2) has an average run-time of 6 minutes. By default this test profile is set to run at least 3 times but may increase if the standard deviation exceeds pre-defined defaults or other calculations deem additional runs necessary for greater statistical accuracy of the result.

OpenBenchmarking.orgMinutesTime Required To Complete BenchmarkModel: Inception ResNet V2Run-Time612182430Min: 3 / Avg: 6.07 / Max: 25

Based on public OpenBenchmarking.org results, the selected test / test configuration has an average standard deviation of 0.2%.

OpenBenchmarking.orgPercent, Fewer Is BetterAverage Deviation Between RunsModel: Inception ResNet V2Deviation246810Min: 0 / Avg: 0.18 / Max: 4

Does It Scale Well With Increasing Cores?

Yes, based on the automated analysis of the collected public benchmark data, this test / test settings does generally scale well with increasing CPU core counts. Data based on publicly available results for this test / test settings, separated by vendor, result divided by the reference CPU clock speed, grouped by matching physical CPU core count, and normalized against the smallest core count tested from each vendor for each CPU having a sufficient number of test samples and statistically significant data.

IntelAMDOpenBenchmarking.orgRelative Core Scaling To BaseTensorFlow Lite CPU Core ScalingModel: Inception ResNet V224681216243248643691215

Notable Instruction Set Usage

Notable instruction set extensions supported by this test, based on an automatic analysis by the Phoronix Test Suite / OpenBenchmarking.org analytics engine.

Instruction Set
Support
Instructions Detected
SSE2 (SSE2)
Used by default on supported hardware.
 
MOVDQA PMULUDQ PSHUFD PSRLDQ MOVD CVTSI2SD ADDSD MULSD SUBSD DIVSD MOVAPD CVTSS2SD CVTTSD2SI SQRTSD UCOMISD XORPD CVTSD2SS CVTTPS2DQ CVTDQ2PS PADDQ MOVDQU PUNPCKLQDQ UNPCKLPD CVTDQ2PD MULPD CVTPD2PS ANDPD MAXSD PSUBQ CVTPS2PD MINSD MOVUPD UNPCKHPD ADDPD PUNPCKHQDQ CVTPS2DQ
Used by default on supported hardware.
Found on Intel processors since Sandy Bridge (2011).
Found on AMD processors since Bulldozer (2011).

 
VBROADCASTF128 VZEROUPPER VMASKMOVPS VEXTRACTF128 VBROADCASTSS VINSERTF128 VPERMILPS
Advanced Vector Extensions 512 (AVX512)
Used by default on supported hardware.
 
(ZMM REGISTER USE)
FMA (FMA)
Used by default on supported hardware.
Found on Intel processors since Haswell (2013).
Found on AMD processors since Bulldozer (2011).

 
VFMADD132PS VFMADD231PS VFMADD213PS
Last automated analysis: 18 January 2022

This test profile binary relies on the shared libraries libm.so.6, libpthread.so.0, libdl.so.2, librt.so.1, libc.so.6.

Tested CPU Architectures

This benchmark has been successfully tested on the below mentioned architectures. The CPU architectures listed is where successful OpenBenchmarking.org result uploads occurred, namely for helping to determine if a given test is compatible with various alternative CPU architectures.

CPU Architecture
Kernel Identifier
Verified On
Intel / AMD x86 64-bit
x86_64
(Many Processors)

Recent Test Results

OpenBenchmarking.org Results Compare

4 Systems - 16 Benchmark Results

Intel Core i7-4770 - ASUS H87M-PLUS - Intel 4th Gen Core DRAM

Ubuntu 22.04 - 5.15.0-46-generic - X Server 1.21.1.3

1 System - 6 Benchmark Results

Intel Core i5-4210U - Acer ZORO_BH - Intel Haswell-ULT DRAM

Ubuntu 20.04 - 5.15.0-83-generic - Xfce 4.14

1 System - 6 Benchmark Results

Intel Core i5-4210U - Acer ZORO_BH - Intel Haswell-ULT DRAM

Ubuntu 20.04 - 5.15.0-83-generic - Xfce 4.14

1 System - 6 Benchmark Results

Intel Core i5-4210U - Acer ZORO_BH - Intel Haswell-ULT DRAM

Ubuntu 20.04 - 5.15.0-83-generic - Xfce 4.14

1 System - 6 Benchmark Results

Intel Core i5-4210U - Acer ZORO_BH - Intel Haswell-ULT DRAM

Ubuntu 20.04 - 5.15.0-83-generic - Xfce 4.14

Find More Test Results