TensorFlow Lite

This is a benchmark of the TensorFlow Lite implementation focused on TensorFlow machine learning for mobile, IoT, edge, and other cases. The current Linux support is limited to running on CPUs. This test profile is measuring the average inference time.

To run this test with the Phoronix Test Suite, the basic command is: phoronix-test-suite benchmark tensorflow-lite.

Project Site

tensorflow.org

Source Repository

github.com

Test Created

23 August 2020

Last Updated

19 May 2022

Test Maintainer

Michael Larabel 

Test Type

System

Average Install Time

9 Seconds

Average Run Time

4 Minutes, 53 Seconds

Accolades

10k+ Downloads

Supported Platforms


Public Result Uploads *Reported Installs **Reported Test Completions **Test Profile Page Views ***OpenBenchmarking.orgEventsTensorFlow Lite Popularity Statisticspts/tensorflow-lite2020.082020.102020.122021.022021.042021.062021.082021.102021.122022.022022.042022.062022.082022.102022.122023.022023.042023.062023.082023.102023.122024.022024.045K10K15K20K25K
* Uploading of benchmark result data to OpenBenchmarking.org is always optional (opt-in) via the Phoronix Test Suite for users wishing to share their results publicly.
** Data based on those opting to upload their test results to OpenBenchmarking.org and users enabling the opt-in anonymous statistics reporting while running benchmarks from an Internet-connected platform.
*** Test profile page view reporting began March 2021.
Data updated weekly as of 7 May 2024.
Mobilenet Float19.8%NASNet Mobile13.6%Mobilenet Quant16.9%SqueezeNet17.1%Inception ResNet V215.4%Inception V417.2%Model Option PopularityOpenBenchmarking.org

Revision History

pts/tensorflow-lite-1.1.0   [View Source]   Thu, 19 May 2022 09:57:39 GMT
Update against latest upstream nightly.

pts/tensorflow-lite-1.0.0   [View Source]   Sun, 23 Aug 2020 14:13:10 GMT
TensorFlow Lite initial commit.

Suites Using This Test

Machine Learning

HPC - High Performance Computing


Performance Metrics

Analyze Test Configuration:

TensorFlow Lite 2020-08-23

Model: Mobilenet Float

OpenBenchmarking.org metrics for this test profile configuration based on 1,462 public results since 23 August 2020 with the latest data as of 3 October 2023.

Below is an overview of the generalized performance for components where there is sufficient statistically significant data based upon user-uploaded results. It is important to keep in mind particularly in the Linux/open-source space there can be vastly different OS configurations, with this overview intended to offer just general guidance as to the performance expectations.

Component
Percentile Rank
# Compatible Public Results
Microseconds (Average)
99th
3
29151 +/- 490
99th
18
29297 +/- 2857
98th
4
30239 +/- 164
98th
8
30771 +/- 1578
97th
7
31065 +/- 1333
95th
8
35720 +/- 3010
94th
6
37060 +/- 4302
94th
27
37128 +/- 4292
92nd
11
39087 +/- 1464
91st
23
40639 +/- 3772
91st
3
40793 +/- 62
90th
19
41220 +/- 1948
89th
3
41928 +/- 2673
88th
5
43626 +/- 970
87th
19
43845 +/- 858
87th
6
44023 +/- 35
85th
4
46222 +/- 952
84th
8
46568 +/- 937
84th
10
47115 +/- 60
83rd
4
48287 +/- 284
82nd
8
49480 +/- 806
82nd
8
49885 +/- 1491
79th
12
51910 +/- 1230
79th
7
53158 +/- 2657
78th
4
53571 +/- 6659
78th
9
54642 +/- 333
77th
7
56962 +/- 712
76th
15
59021 +/- 1123
Mid-Tier
75th
> 59041
75th
16
59230 +/- 256
75th
4
59655 +/- 204
73rd
6
60767 +/- 691
73rd
7
60932 +/- 362
72nd
10
62019 +/- 555
70th
52
62757 +/- 373
68th
3
66529 +/- 206
68th
5
67862 +/- 13
67th
12
67883 +/- 1136
66th
16
68764 +/- 204
66th
5
70466 +/- 181
65th
6
74310 +/- 2187
64th
10
80332 +/- 107
62nd
41
84014 +/- 491
60th
8
84336 +/- 321
59th
23
86653 +/- 2196
58th
6
92071 +/- 8672
57th
9
94588 +/- 527
56th
12
95796 +/- 652
53rd
15
110298 +/- 528
52nd
5
113030 +/- 112
51st
29
114707 +/- 839
Median
50th
114742
50th
22
116449 +/- 7671
49th
6
118677 +/- 1771
49th
7
121333 +/- 4454
48th
14
122793 +/- 1491
48th
3
123225 +/- 376
47th
8
126004 +/- 708
46th
8
126175 +/- 122
44th
3
133607 +/- 316
44th
15
136510 +/- 3620
43rd
3
143919 +/- 21323
41st
8
150236 +/- 42
41st
7
150945 +/- 10381
41st
14
152489 +/- 5989
41st
23
152640 +/- 2523
39th
7
153846 +/- 597
39th
4
155102 +/- 2980
37th
10
160841 +/- 183
35th
4
172373 +/- 15229
35th
4
172659 +/- 1568
35th
8
172864 +/- 1248
35th
3
173700 +/- 426
35th
11
174192 +/- 8585
34th
3
176492 +/- 1009
34th
3
176511 +/- 79
32nd
3
197030 +/- 65
32nd
3
197077 +/- 165
32nd
15
200346 +/- 5605
31st
3
202429 +/- 647
30th
7
209658 +/- 10382
30th
18
214587 +/- 9331
30th
5
214776 +/- 106
29th
3
216552 +/- 17
28th
5
219717 +/- 36
28th
12
222083 +/- 1630
26th
14
236681 +/- 2565
26th
6
237630 +/- 3084
Low-Tier
25th
> 239853
25th
11
246213 +/- 233
24th
4
255295 +/- 2341
24th
7
259567 +/- 86
23rd
6
260486 +/- 2756
23rd
3
261083 +/- 76
23rd
3
269343 +/- 13
22nd
11
271246 +/- 18112
22nd
4
274020 +/- 1059
22nd
4
275611 +/- 5565
22nd
4
275865 +/- 10078
21st
3
282718 +/- 739
21st
3
304775 +/- 42
20th
4
305678 +/- 566
20th
5
311378 +/- 13939
20th
3
319383 +/- 66
20th
3
325212 +/- 398
19th
3
332759 +/- 2119
19th
4
338524 +/- 85
19th
3
339406 +/- 31
18th
11
348389 +/- 10229
17th
4
351298 +/- 146
17th
3
352477 +/- 5025
17th
3
365322 +/- 37
16th
4
366219 +/- 709
15th
3
369899 +/- 225
15th
40
370557 +/- 18477
12th
63
383722 +/- 2442
9th
3
387869 +/- 89
9th
4
394458 +/- 5452
9th
3
394492 +/- 119
7th
15
438597 +/- 17004
7th
3
451547 +/- 222
6th
3
458252 +/- 4150
6th
5
464341 +/- 33449
6th
9
466068 +/- 33410
5th
6
489010 +/- 1403
5th
7
498786 +/- 13013
4th
3
585811 +/- 14584
4th
4
623697 +/- 379
3rd
3
690680 +/- 318
3rd
3
777976 +/- 47
3rd
4
827989 +/- 261
2nd
5
899236 +/- 652
2nd
3
976603 +/- 27
2nd
3
1011213 +/- 373
1st
4
1108977 +/- 603
OpenBenchmarking.orgDistribution Of Public Results - Model: Mobilenet Float1462 Results Range From 25734 To 1610230 Microseconds2573457424891141208041524941841842158742475642792543109443426343743244060144377044693945010845327745644645961546278446595346912247229147546047862948179848496748813649130549447449764341008124103981410715041103194113488411665741198264122995412616441293334132502413567141388404142009414517841483474151516415468541578544161023480160240320400

Based on OpenBenchmarking.org data, the selected test / test configuration (TensorFlow Lite 2020-08-23 - Model: Mobilenet Float) has an average run-time of 4 minutes. By default this test profile is set to run at least 3 times but may increase if the standard deviation exceeds pre-defined defaults or other calculations deem additional runs necessary for greater statistical accuracy of the result.

OpenBenchmarking.orgMinutesTime Required To Complete BenchmarkModel: Mobilenet FloatRun-Time3691215Min: 3 / Avg: 3.08 / Max: 11

Based on public OpenBenchmarking.org results, the selected test / test configuration has an average standard deviation of 0.3%.

OpenBenchmarking.orgPercent, Fewer Is BetterAverage Deviation Between RunsModel: Mobilenet FloatDeviation246810Min: 0 / Avg: 0.28 / Max: 4

Does It Scale Well With Increasing Cores?

Yes, based on the automated analysis of the collected public benchmark data, this test / test settings does generally scale well with increasing CPU core counts. Data based on publicly available results for this test / test settings, separated by vendor, result divided by the reference CPU clock speed, grouped by matching physical CPU core count, and normalized against the smallest core count tested from each vendor for each CPU having a sufficient number of test samples and statistically significant data.

IntelAMDOpenBenchmarking.orgRelative Core Scaling To BaseTensorFlow Lite CPU Core ScalingModel: Mobilenet Float24681216243248643691215

Notable Instruction Set Usage

Notable instruction set extensions supported by this test, based on an automatic analysis by the Phoronix Test Suite / OpenBenchmarking.org analytics engine.

Instruction Set
Support
Instructions Detected
SSE2 (SSE2)
Used by default on supported hardware.
 
MOVDQA PMULUDQ PSHUFD PSRLDQ MOVD CVTSI2SD ADDSD MULSD SUBSD DIVSD MOVAPD CVTSS2SD CVTTSD2SI SQRTSD UCOMISD XORPD CVTSD2SS CVTTPS2DQ CVTDQ2PS PADDQ MOVDQU PUNPCKLQDQ UNPCKLPD CVTDQ2PD MULPD CVTPD2PS ANDPD MAXSD PSUBQ CVTPS2PD MINSD MOVUPD UNPCKHPD ADDPD PUNPCKHQDQ CVTPS2DQ
Used by default on supported hardware.
Found on Intel processors since Sandy Bridge (2011).
Found on AMD processors since Bulldozer (2011).

 
VBROADCASTF128 VZEROUPPER VMASKMOVPS VEXTRACTF128 VBROADCASTSS VINSERTF128 VPERMILPS
Advanced Vector Extensions 512 (AVX512)
Used by default on supported hardware.
 
(ZMM REGISTER USE)
FMA (FMA)
Used by default on supported hardware.
Found on Intel processors since Haswell (2013).
Found on AMD processors since Bulldozer (2011).

 
VFMADD132PS VFMADD231PS VFMADD213PS
Last automated analysis: 18 January 2022

This test profile binary relies on the shared libraries libm.so.6, libpthread.so.0, libdl.so.2, librt.so.1, libc.so.6.

Tested CPU Architectures

This benchmark has been successfully tested on the below mentioned architectures. The CPU architectures listed is where successful OpenBenchmarking.org result uploads occurred, namely for helping to determine if a given test is compatible with various alternative CPU architectures.

CPU Architecture
Kernel Identifier
Verified On
Intel / AMD x86 64-bit
x86_64
(Many Processors)