TensorFlow Lite

This is a benchmark of the TensorFlow Lite implementation focused on TensorFlow machine learning for mobile, IoT, edge, and other cases. The current Linux support is limited to running on CPUs. This test profile is measuring the average inference time.

To run this test with the Phoronix Test Suite, the basic command is: phoronix-test-suite benchmark tensorflow-lite.

Project Site

tensorflow.org

Source Repository

github.com

Test Created

23 August 2020

Last Updated

19 May 2022

Test Maintainer

Michael Larabel 

Test Type

System

Average Install Time

9 Seconds

Average Run Time

4 Minutes, 53 Seconds

Accolades

10k+ Downloads

Supported Platforms


Public Result Uploads *Reported Installs **Reported Test Completions **Test Profile Page Views ***OpenBenchmarking.orgEventsTensorFlow Lite Popularity Statisticspts/tensorflow-lite2020.082020.102020.122021.022021.042021.062021.082021.102021.122022.022022.042022.062022.082022.102022.122023.022023.042023.062023.082023.102023.122024.022024.045K10K15K20K25K
* Uploading of benchmark result data to OpenBenchmarking.org is always optional (opt-in) via the Phoronix Test Suite for users wishing to share their results publicly.
** Data based on those opting to upload their test results to OpenBenchmarking.org and users enabling the opt-in anonymous statistics reporting while running benchmarks from an Internet-connected platform.
*** Test profile page view reporting began March 2021.
Data updated weekly as of 21 April 2024.
Mobilenet Float19.8%NASNet Mobile13.6%Mobilenet Quant16.9%SqueezeNet17.1%Inception ResNet V215.4%Inception V417.2%Model Option PopularityOpenBenchmarking.org

Revision History

pts/tensorflow-lite-1.1.0   [View Source]   Thu, 19 May 2022 09:57:39 GMT
Update against latest upstream nightly.

pts/tensorflow-lite-1.0.0   [View Source]   Sun, 23 Aug 2020 14:13:10 GMT
TensorFlow Lite initial commit.

Suites Using This Test

Machine Learning

HPC - High Performance Computing


Performance Metrics

Analyze Test Configuration:

TensorFlow Lite 2020-08-23

Model: Inception V4

OpenBenchmarking.org metrics for this test profile configuration based on 1,440 public results since 23 August 2020 with the latest data as of 3 October 2023.

Below is an overview of the generalized performance for components where there is sufficient statistically significant data based upon user-uploaded results. It is important to keep in mind particularly in the Linux/open-source space there can be vastly different OS configurations, with this overview intended to offer just general guidance as to the performance expectations.

Component
Percentile Rank
# Compatible Public Results
Microseconds (Average)
100th
17
531628 +/- 51003
97th
10
661029 +/- 38698
96th
9
668235 +/- 4988
94th
33
681815 +/- 15315
91st
3
782422 +/- 9279
91st
5
790549 +/- 2857
91st
19
800555 +/- 49557
89th
3
820592 +/- 11963
89th
8
826009 +/- 103656
88th
19
836569 +/- 26251
88th
12
841951 +/- 23111
87th
7
853117 +/- 13804
87th
3
863215 +/- 11420
87th
3
863314 +/- 16249
86th
3
875968 +/- 361
84th
4
905365 +/- 10546
84th
39
923952 +/- 84274
83rd
19
936514 +/- 10323
82nd
6
945673 +/- 1502
81st
6
951966 +/- 6767
81st
8
961202 +/- 18641
79th
7
979596 +/- 16355
79th
6
994889 +/- 3674
78th
8
1012202 +/- 27637
78th
10
1012840 +/- 5172
76th
15
1059602 +/- 115568
76th
4
1063555 +/- 11117
Mid-Tier
75th
> 1069320
75th
8
1073150 +/- 32115
75th
4
1087774 +/- 27392
73rd
10
1182810 +/- 11247
72nd
10
1220750 +/- 3314
72nd
3
1261178 +/- 949
71st
16
1266838 +/- 3753
71st
7
1267651 +/- 3250
70th
3
1272922 +/- 2025
69th
4
1301404 +/- 1089
69th
15
1323553 +/- 16184
68th
6
1337462 +/- 11159
66th
56
1355047 +/- 14000
64th
10
1401063 +/- 35026
63rd
11
1457031 +/- 20027
62nd
6
1484960 +/- 511
61st
16
1501519 +/- 3477
61st
3
1517958 +/- 10132
60th
5
1534562 +/- 886
60th
6
1535016 +/- 43018
60th
3
1541690 +/- 1270
58th
10
1746416 +/- 5087
56th
48
1791670 +/- 8455
54th
8
1818997 +/- 957
54th
23
1835360 +/- 51559
51st
13
1993321 +/- 120555
51st
6
1997966 +/- 223554
51st
9
2003503 +/- 8948
Median
50th
2015920
50th
10
2044059 +/- 13932
47th
15
2347085 +/- 9788
45th
29
2381827 +/- 17059
45th
5
2427632 +/- 2082
44th
18
2433795 +/- 61845
44th
6
2517422 +/- 25935
42nd
8
2614438 +/- 86377
42nd
15
2641227 +/- 34806
42nd
3
2645239 +/- 11937
42nd
8
2661069 +/- 1826
40th
8
2728458 +/- 12216
39th
3
2833567 +/- 2924
38th
7
2924914 +/- 349826
38th
15
2940211 +/- 65659
37th
4
3143810
36th
8
3230520 +/- 926
36th
14
3238735 +/- 125017
35th
7
3276896 +/- 257101
34th
23
3318278 +/- 56444
34th
4
3336966 +/- 60324
34th
7
3340748 +/- 4835
32nd
10
3476841 +/- 6255
30th
3
3622617 +/- 2512
29th
8
3674286 +/- 44625
29th
4
3739172 +/- 30097
29th
3
3755293 +/- 15369
29th
7
3771843 +/- 313765
29th
11
3781791 +/- 225796
28th
3
3829176 +/- 9336
27th
3
4177253 +/- 958
26th
3
4180822 +/- 3737
Low-Tier
25th
> 4293077
25th
15
4307112 +/- 122794
24th
3
4382592 +/- 18386
24th
7
4468804 +/- 219321
24th
5
4508480 +/- 271
24th
18
4566046 +/- 218453
23rd
5
4613280 +/- 1245
22nd
14
4883289 +/- 35126
22nd
6
4891576 +/- 670870
21st
14
5167313 +/- 47780
20th
11
5274770 +/- 7652
19th
4
5442567 +/- 32231
18th
6
5522806 +/- 31927
18th
7
5570668 +/- 6792
17th
3
5758377 +/- 384
17th
11
5778408 +/- 433355
17th
3
5785801 +/- 15088
17th
4
5862613 +/- 227290
16th
4
5956926 +/- 108509
16th
3
5972134 +/- 13379
15th
4
6415582 +/- 69824
15th
3
6467311 +/- 629
15th
4
6490153 +/- 11453
14th
5
6666829 +/- 311903
14th
3
6832660 +/- 679
14th
3
6924027 +/- 10096
14th
3
7025878 +/- 25406
13th
4
7166857 +/- 749
13th
3
7203782 +/- 760
12th
4
7346190 +/- 2925
12th
3
7471137 +/- 35283
12th
3
7780924 +/- 7562
11th
3
7807922 +/- 893
11th
4
7820482 +/- 12835
11th
41
7976029 +/- 429043
9th
3
8293764 +/- 3127
8th
3
8372534 +/- 1682
8th
4
8575365 +/- 28108
6th
4
9378049 +/- 1747
6th
15
9474898 +/- 346809
6th
3
9554590 +/- 2649
5th
3
10007035 +/- 85830
5th
5
10027546 +/- 724872
5th
9
10045373 +/- 806768
5th
6
10311522 +/- 880
4th
3
12494645 +/- 301685
3rd
4
13230717 +/- 661
3rd
3
14680022 +/- 88498
3rd
3
16580844 +/- 6996
3rd
4
17609975 +/- 5034
2nd
5
17668280 +/- 497696
2nd
5
19554807 +/- 24764
2nd
3
20841444 +/- 11702
1st
3
21619078 +/- 8720
1st
4
23573517 +/- 5391
OpenBenchmarking.orgDistribution Of Public Results - Model: Inception V41440 Results Range From 481340 To 34247700 Microseconds4813401156668183199625073243182652385798045333085208636588396465592927234620790994885852769260604993593210611260112865881196191612637244133125721398790014663228153385561601388416689212173645401803986818715196193905242006585220741180214165082209183622767164234424922411782024793148254684762614380426819132274944602816978828845116295204443019577230871100315464283222175632897084335724123424774080160240320400

Based on OpenBenchmarking.org data, the selected test / test configuration (TensorFlow Lite 2020-08-23 - Model: Inception V4) has an average run-time of 7 minutes. By default this test profile is set to run at least 3 times but may increase if the standard deviation exceeds pre-defined defaults or other calculations deem additional runs necessary for greater statistical accuracy of the result.

OpenBenchmarking.orgMinutesTime Required To Complete BenchmarkModel: Inception V4Run-Time48121620Min: 3 / Avg: 6.34 / Max: 16

Based on public OpenBenchmarking.org results, the selected test / test configuration has an average standard deviation of 0.2%.

OpenBenchmarking.orgPercent, Fewer Is BetterAverage Deviation Between RunsModel: Inception V4Deviation246810Min: 0 / Avg: 0.21 / Max: 4

Does It Scale Well With Increasing Cores?

Yes, based on the automated analysis of the collected public benchmark data, this test / test settings does generally scale well with increasing CPU core counts. Data based on publicly available results for this test / test settings, separated by vendor, result divided by the reference CPU clock speed, grouped by matching physical CPU core count, and normalized against the smallest core count tested from each vendor for each CPU having a sufficient number of test samples and statistically significant data.

IntelAMDOpenBenchmarking.orgRelative Core Scaling To BaseTensorFlow Lite CPU Core ScalingModel: Inception V424681216243248643691215

Notable Instruction Set Usage

Notable instruction set extensions supported by this test, based on an automatic analysis by the Phoronix Test Suite / OpenBenchmarking.org analytics engine.

Instruction Set
Support
Instructions Detected
SSE2 (SSE2)
Used by default on supported hardware.
 
MOVDQA PMULUDQ PSHUFD PSRLDQ MOVD CVTSI2SD ADDSD MULSD SUBSD DIVSD MOVAPD CVTSS2SD CVTTSD2SI SQRTSD UCOMISD XORPD CVTSD2SS CVTTPS2DQ CVTDQ2PS PADDQ MOVDQU PUNPCKLQDQ UNPCKLPD CVTDQ2PD MULPD CVTPD2PS ANDPD MAXSD PSUBQ CVTPS2PD MINSD MOVUPD UNPCKHPD ADDPD PUNPCKHQDQ CVTPS2DQ
Used by default on supported hardware.
Found on Intel processors since Sandy Bridge (2011).
Found on AMD processors since Bulldozer (2011).

 
VBROADCASTF128 VZEROUPPER VMASKMOVPS VEXTRACTF128 VBROADCASTSS VINSERTF128 VPERMILPS
Advanced Vector Extensions 512 (AVX512)
Used by default on supported hardware.
 
(ZMM REGISTER USE)
FMA (FMA)
Used by default on supported hardware.
Found on Intel processors since Haswell (2013).
Found on AMD processors since Bulldozer (2011).

 
VFMADD132PS VFMADD231PS VFMADD213PS
Last automated analysis: 18 January 2022

This test profile binary relies on the shared libraries libm.so.6, libpthread.so.0, libdl.so.2, librt.so.1, libc.so.6.

Tested CPU Architectures

This benchmark has been successfully tested on the below mentioned architectures. The CPU architectures listed is where successful OpenBenchmarking.org result uploads occurred, namely for helping to determine if a given test is compatible with various alternative CPU architectures.

CPU Architecture
Kernel Identifier
Verified On
Intel / AMD x86 64-bit
x86_64
(Many Processors)