TensorFlow Lite

This is a benchmark of the TensorFlow Lite implementation. The current Linux support is limited to running on CPUs. This test profile is measuring the average inference time.

To run this test with the Phoronix Test Suite, the basic command is: phoronix-test-suite benchmark tensorflow-lite.

Project Site

tensorflow.org

Test Created

23 August 2020

Test Maintainer

Michael Larabel 

Test Type

System

Average Install Time

10 Seconds

Average Run Time

4 Minutes, 51 Seconds

Supported Platforms


Public Result UploadsReported Installs*Test Completions*OpenBenchmarking.orgEventsTensorFlow Lite Popularity Statisticspts/tensorflow-lite2020.082020.092020.102020.112020.122021.012021.022021.032021.042021.0510002000300040005000
* Data based on those opting to upload their test results to OpenBenchmarking.org and users enabling the opt-in anonymous statistics reporting while running benchmarks from an Internet-connected platform.
Data current as of Wed, 05 May 2021 04:35:24 GMT.
Mobilenet Float16.2%Inception ResNet V218.0%SqueezeNet16.5%Inception V416.8%Mobilenet Quant16.6%NASNet Mobile16.0%Model Option PopularityOpenBenchmarking.org

Revision History

pts/tensorflow-lite-1.0.0   [View Source]   Sun, 23 Aug 2020 14:13:10 GMT
TensorFlow Lite initial commit.

Suites Using This Test

Machine Learning

HPC - High Performance Computing


Performance Metrics

Analyze Test Configuration:

TensorFlow Lite 2020-08-23

Model: Inception ResNet V2

OpenBenchmarking.org metrics for this test profile configuration based on 1,105 public results since 23 August 2020 with the latest data as of 3 May 2021.

Below is an overview of the generalized performance for components where there is sufficient statistically significant data based upon user-uploaded results. It is important to keep in mind particularly in the Linux/open-source space there can be vastly different OS configurations, with this overview intended to offer just general guidance as to the performance expectations.

Component
Percentile Rank
# Matching Public Results
Microseconds (Average)
100th
12
414063 +/- 26003
99th
16
483074 +/- 41759
98th
7
499289 +/- 29986
97th
3
572083 +/- 2178
97th
9
592443 +/- 64088
94th
10
678563 +/- 7509
92nd
10
712514 +/- 59513
89th
10
739878 +/- 6990
89th
21
742157 +/- 10612
88th
5
752316 +/- 16452
87th
3
773722 +/- 3052
86th
4
783144 +/- 3102
85th
45
808932 +/- 79114
83rd
5
838720 +/- 1308
83rd
10
842083 +/- 10627
82nd
8
845935 +/- 4924
79th
10
893295 +/- 2101
78th
16
901895 +/- 3202
Mid-Tier
75th
> 939492
75th
5
941316 +/- 7242
75th
10
948359 +/- 26294
75th
19
968716 +/- 98133
73rd
7
1015399 +/- 27906
72nd
5
1076488 +/- 94
71st
10
1116116 +/- 6778
70th
4
1157025 +/- 3764
70th
15
1160719 +/- 14921
68th
6
1188341 +/- 9223
65th
38
1215392 +/- 7006
64th
16
1220830 +/- 15671
63rd
8
1306424 +/- 19882
62nd
5
1330439 +/- 64
62nd
5
1336200 +/- 3981
61st
18
1348022 +/- 3910
60th
3
1354931 +/- 6254
59th
7
1395324 +/- 9478
57th
15
1568169 +/- 3906
56th
6
1584948 +/- 2624
55th
32
1606310 +/- 6788
52nd
10
1639392 +/- 7060
51st
21
1641106 +/- 2915
Median
50th
1658387
50th
6
1671634 +/- 1070
48th
13
1799772 +/- 132080
48th
6
1803461 +/- 9325
47th
3
2073899 +/- 16742
46th
11
2122639 +/- 7432
43rd
28
2147387 +/- 15250
42nd
11
2229494 +/- 45228
42nd
8
2258168 +/- 16637
41st
6
2318708 +/- 9939
40th
15
2389486 +/- 30432
40th
8
2402267 +/- 1343
38th
5
2456745 +/- 11248
37th
3
2557136 +/- 5957
36th
17
2656161 +/- 65836
34th
7
2960190 +/- 231889
34th
15
2999394 +/- 48980
33rd
10
3003644 +/- 98341
33rd
4
3013723 +/- 53347
31st
16
3140318 +/- 6414
30th
3
3268110 +/- 1072
29th
3
3311907
28th
11
3414240 +/- 197816
28th
3
3415448 +/- 3867
28th
4
3456231 +/- 10852
26th
3
3777707 +/- 2332
26th
3
3781611 +/- 1933
Low-Tier
25th
> 3783777
25th
4
3881257 +/- 177
24th
14
3937052 +/- 6822
23rd
3
3959302 +/- 11505
23rd
3
4075158 +/- 484
23rd
3
4170277 +/- 287
23rd
15
4205257 +/- 191481
21st
6
4667686 +/- 8700
21st
13
4668485 +/- 45809
20th
9
4779473 +/- 830
19th
4
4914079 +/- 50617
18th
6
4997320 +/- 36672
18th
6
5048004 +/- 835
17th
9
5110812 +/- 263186
17th
3
5195028 +/- 11376
17th
3
5201783 +/- 906
17th
3
5211162 +/- 52
16th
4
5390148 +/- 102304
16th
3
5417777 +/- 12455
15th
3
5491067 +/- 39015
14th
3
5851123 +/- 1227
14th
4
5870767 +/- 8482
14th
3
6181699 +/- 1037
14th
3
6233168 +/- 78
13th
3
6259003 +/- 6450
13th
3
6369685 +/- 23995
13th
3
6490521 +/- 1102
12th
3
6518117 +/- 687
12th
4
6619217 +/- 789
12th
3
7042845 +/- 6315
11th
3
7065428 +/- 997
11th
4
7073255 +/- 11170
10th
28
7345226 +/- 323894
9th
3
7504205 +/- 2901
8th
3
7575542 +/- 1773
7th
4
7811175 +/- 20167
6th
3
8485620 +/- 495
6th
12
8585390 +/- 373792
5th
3
8650855 +/- 2272
5th
7
8769820 +/- 742969
5th
3
8994871 +/- 29228
4th
7
9256685 +/- 656327
3rd
3
11974722 +/- 850
3rd
3
13315667 +/- 15674
3rd
3
14999222 +/- 700
2nd
4
15930567 +/- 5075
2nd
5
17693280 +/- 18145
1st
3
18852389 +/- 703
1st
3
19548311 +/- 4409
OpenBenchmarking.orgDistribution Of Public Results - Model: Inception ResNet V21115 Results Range From 389196 To 31032833 Microseconds389196100206916149422227815284068834535614066434467930752921805905053651792671307997743672835654589694189582291101951641080803711420910120337831264665613259529138724021448527515098148157110211632389416936767175496401816251318775386193882592000113220614005212268782183975122452624230654972367837024291243249041162551698926129862267427352735560827968481285813542919422729807100304199733103284670140210280350

Based on OpenBenchmarking.org data, the selected test / test configuration (TensorFlow Lite 2020-08-23 - Model: Inception ResNet V2) has an average run-time of 6 minutes. By default this test profile is set to run at least 3 times but may increase if the standard deviation exceeds pre-defined defaults or other calculations deem additional runs necessary for greater statistical accuracy of the result.

OpenBenchmarking.orgMinutesTime Required To Complete BenchmarkModel: Inception ResNet V2Run-Time3691215Min: 1 / Avg: 5.3 / Max: 12

Based on public OpenBenchmarking.org results, the selected test / test configuration has an average standard deviation of 0.2%.

OpenBenchmarking.orgPercent, Fewer Is BetterAverage Deviation Between RunsModel: Inception ResNet V2Deviation246810Min: 0 / Avg: 0.15 / Max: 3

Does It Scale Well With Increasing Cores?

Yes, based on the automated analysis of the collected public benchmark data, this test / test settings does generally scale well with increasing CPU core counts. Data based on publicly available results for this test / test settings, separated by vendor, result divided by the reference CPU clock speed, grouped by matching physical CPU core count, and normalized against the smallest core count tested from each vendor for each CPU having a sufficient number of test samples and statistically significant data.

IntelAMDOpenBenchmarking.orgRelative Core Scaling To BaseTensorFlow Lite CPU Core ScalingModel: Inception ResNet V2246812162432486412848121620

Notable Instruction Set Usage

Notable instruction set extensions supported by this test, based on an automatic analysis by the Phoronix Test Suite / OpenBenchmarking.org analytics engine.

Instruction Set
Support
Instructions Detected
SSE2 (SSE2)
Used by default on supported hardware.
 
MOVDQA PMULUDQ PSHUFD PSRLDQ MOVD CVTSI2SD ADDSD MULSD SUBSD DIVSD MOVAPD CVTSS2SD CVTTSD2SI SQRTSD UCOMISD XORPD CVTSD2SS CVTTPS2DQ CVTDQ2PS PADDQ MOVDQU PUNPCKLQDQ UNPCKLPD CVTDQ2PD MULPD CVTPD2PS ANDPD MAXSD PSUBQ CVTPS2PD MINSD MOVUPD UNPCKHPD ADDPD PUNPCKHQDQ CVTPS2DQ
Used by default on supported hardware.
Found on Intel processors since Sandy Bridge (2011).
Found on AMD processors since Bulldozer (2011).

 
VBROADCASTF128 VZEROUPPER VMASKMOVPS VEXTRACTF128 VBROADCASTSS VINSERTF128 VPERMILPS
FMA (FMA)
Used by default on supported hardware.
Found on Intel processors since Haswell (2013).
Found on AMD processors since Bulldozer (2011).

 
VFMADD132PS VFMADD231PS VFMADD213PS
Last automated analysis: 31 January 2021

This test profile binary relies on the shared libraries libm.so.6, libpthread.so.0, libdl.so.2, librt.so.1, libc.so.6.

Recent Test Results

OpenBenchmarking.org Results Compare

1 System - 732 Benchmark Results

2 x Intel Xeon Platinum 8380 - Intel M50CYP2SB2U - Intel Device 0998

Ubuntu 20.04 - 5.11.6-051106-generic - GNOME Shell 3.36.4

3 Systems - 101 Benchmark Results

Intel Core i9-10980XE - ASRock X299 Steel Legend - Intel Sky Lake-E DMI3 Registers

Ubuntu 20.10 - 5.8.0-50-generic - GNOME Shell 3.38.1

2 Systems - 434 Benchmark Results

AMD EPYC 72F3 8-Core - Supermicro H12SSL-i v1.01 - AMD Starship

Ubuntu 21.04 - 5.11.0-16-generic - GNOME Shell 3.38.4

2 Systems - 102 Benchmark Results

Intel Core i9-10980XE - ASRock X299 Steel Legend - Intel Sky Lake-E DMI3 Registers

Clear Linux OS 34530 - 5.10.31-1033.native - GNOME Shell 3.38.4

1 System - 103 Benchmark Results

Intel Core i9-10980XE - ASRock X299 Steel Legend - Intel Sky Lake-E DMI3 Registers

Clear Linux OS 34530 - 5.10.31-1033.native - GNOME Shell 3.38.4

10 Systems - 454 Benchmark Results

Intel Core i9-10900K - Gigabyte Z490 AORUS MASTER - Intel Comet Lake PCH

Ubuntu 21.04 - 5.12.0-051200rc3daily20210315-generic - GNOME Shell 3.38.3

1 System - 136 Benchmark Results

AMD EPYC 7763 64-Core - Supermicro H12SSL-i v1.01 - AMD Starship

Ubuntu 20.04 - 5.4.0-72-generic - GNOME Shell 3.36.7

1 System - 1 Benchmark Result

Intel Core i5-8250U - Dell 0RKTGR - Intel Xeon E3-1200 v6

Arch Linux - 5.11.13-arch1-1 - KDE Plasma 5.21.4

3 Systems - 95 Benchmark Results

2 x AMD EPYC 7551 32-Core - Microsoft Virtual Machine - 226GB

CentOS Linux 8 - 4.18.0-147.8.1.el8_1.x86_64 - GCC 8.3.1 20190507

2 Systems - 86 Benchmark Results

2 x AMD EPYC 7V12 64-Core - Microsoft Virtual Machine - 450GB

CentOS Linux 8 - 4.18.0-147.8.1.el8_1.x86_64 - GCC 8.3.1 20190507

Most Popular Test Results

OpenBenchmarking.org Results Compare

3 Systems - 268 Benchmark Results

Intel Core i5-2520M - HP 161C - Intel 2nd Generation Core DRAM

Ubuntu 18.04 - 4.18.0-20-generic - GNOME Shell 3.28.3

2 Systems - 535 Benchmark Results

AMD Ryzen 5 4500U - LENOVO LNVNB161216 - AMD Renoir Root Complex

Ubuntu 20.04 - 5.9.0-050900rc7daily20201002-generic - GNOME Shell 3.36.4

11 Systems - 217 Benchmark Results

AMD Ryzen 5 2600 Six-Core - ASUS ROG CROSSHAIR VIII HERO - AMD 17h

Ubuntu 20.04 - 5.9.0-050900-generic - GNOME Shell 3.36.4

3 Systems - 174 Benchmark Results

Intel Core i9-10900K - Gigabyte Z490 AORUS MASTER - Intel Comet Lake PCH

Fedora 32 - 5.8.11-200.fc32.x86_64 - GNOME Shell 3.36.6

2 Systems - 403 Benchmark Results

Intel Core i9-10900K - Gigabyte Z490 AORUS MASTER - Intel Comet Lake PCH

Ubuntu 20.04 - 5.4.0-48-generic - GNOME Shell 3.36.4

12 Systems - 229 Benchmark Results

AMD Ryzen 9 3900X 12-Core - ASUS ROG CROSSHAIR VIII HERO - AMD Starship

Ubuntu 20.04 - 5.9.0-050900-generic - GNOME Shell 3.36.4

3 Systems - 406 Benchmark Results

AMD Ryzen 9 3900XT 12-Core - MSI MEG X570 GODLIKE - AMD Starship

Ubuntu 20.04 - 5.4.0-48-generic - GNOME Shell 3.36.4

3 Systems - 11 Benchmark Results

AMD Ryzen 7 3800XT 8-Core - ASRock X570 Pro4 - AMD Starship

Ubuntu 20.04 - 5.4.0-42-generic - GNOME Shell 3.36.1

8 Systems - 32 Benchmark Results

Intel Core i7-1065G7 - Dell 06CDVY - Intel Device 34ef

Ubuntu 20.10 - 5.8.0-25-generic - GNOME Shell 3.38.1

2 Systems - 78 Benchmark Results

Intel Core i7-10700T - Logic Supply RXM-181 - Intel Comet Lake PCH

openSUSE 15.3 - 5.3.18-47-default - KDE Plasma 5.18.6

2 Systems - 81 Benchmark Results

2 x AMD EPYC 7742 64-Core - Supermicro H11DSi-NT v2.00 - AMD Starship

Ubuntu 20.04 - 5.8.0-44-generic - GNOME Shell 3.36.4

Find More Test Results