TensorFlow Lite

This is a benchmark of the TensorFlow Lite implementation. The current Linux support is limited to running on CPUs. This test profile is measuring the average inference time.

To run this test with the Phoronix Test Suite, the basic command is: phoronix-test-suite benchmark tensorflow-lite.

Project Site

tensorflow.org

Test Created

23 August 2020

Test Maintainer

Michael Larabel 

Test Type

System

Average Install Time

10 Seconds

Average Run Time

4 Minutes, 52 Seconds

Supported Platforms


Public Result UploadsReported Installs*Test Completions*OpenBenchmarking.orgEventsTensorFlow Lite Popularity Statisticspts/tensorflow-lite2020.082020.092020.102020.112020.122021.012021.022021.032021.042021.0510002000300040005000
* Data based on those opting to upload their test results to OpenBenchmarking.org and users enabling the opt-in anonymous statistics reporting while running benchmarks from an Internet-connected platform.
Data current as of Sat, 08 May 2021 16:43:27 GMT.
Mobilenet Float16.2%Inception ResNet V218.0%SqueezeNet16.5%Inception V416.8%Mobilenet Quant16.6%NASNet Mobile16.0%Model Option PopularityOpenBenchmarking.org

Revision History

pts/tensorflow-lite-1.0.0   [View Source]   Sun, 23 Aug 2020 14:13:10 GMT
TensorFlow Lite initial commit.

Suites Using This Test

Machine Learning

HPC - High Performance Computing


Performance Metrics

Analyze Test Configuration:

TensorFlow Lite 2020-08-23

Model: Inception V4

OpenBenchmarking.org metrics for this test profile configuration based on 1,028 public results since 23 August 2020 with the latest data as of 3 May 2021.

Below is an overview of the generalized performance for components where there is sufficient statistically significant data based upon user-uploaded results. It is important to keep in mind particularly in the Linux/open-source space there can be vastly different OS configurations, with this overview intended to offer just general guidance as to the performance expectations.

Component
Percentile Rank
# Matching Public Results
Microseconds (Average)
100th
7
504941 +/- 19381
99th
14
636039 +/- 93130
98th
5
682039 +/- 47370
96th
6
746056 +/- 11235
94th
5
790549 +/- 2857
92nd
8
826009 +/- 103656
92nd
7
826245 +/- 364
91st
19
836569 +/- 26251
91st
3
845095 +/- 12847
88th
39
923952 +/- 84274
87th
5
946060 +/- 1300
86th
6
951966 +/- 6767
86th
8
961202 +/- 18641
82nd
8
1012202 +/- 27637
82nd
14
1012313 +/- 4393
79th
5
1062883 +/- 9744
79th
14
1071665 +/- 109694
79th
8
1073150 +/- 32115
79th
4
1087774 +/- 27392
76th
5
1221675 +/- 1374
76th
7
1267651 +/- 3250
Mid-Tier
75th
> 1269323
75th
4
1301404 +/- 1089
74th
15
1323553 +/- 16184
73rd
6
1337462 +/- 11159
70th
40
1350275 +/- 7879
68th
14
1392197 +/- 32573
67th
8
1450834 +/- 20299
66th
5
1485092 +/- 443
65th
16
1501519 +/- 3477
64th
5
1513658 +/- 9274
63rd
5
1551351 +/- 17662
61st
13
1744837 +/- 5361
61st
6
1760707 +/- 423
59th
41
1789442 +/- 6994
56th
8
1818997 +/- 957
54th
21
1854732 +/- 6944
54th
6
1859435 +/- 323
51st
13
1993321 +/- 120555
51st
6
2007173 +/- 8923
Median
50th
2016277
49th
11
2347665 +/- 9113
46th
28
2381715 +/- 16751
45th
11
2466751 +/- 49803
45th
6
2517422 +/- 25935
43rd
16
2638703 +/- 35109
42nd
8
2661069 +/- 1826
41st
5
2728946 +/- 10947
40th
3
2833567 +/- 2924
39th
15
2940211 +/- 65659
39th
5
2945565 +/- 425896
37th
7
3276896 +/- 257101
36th
10
3300646 +/- 87576
36th
15
3325734 +/- 55303
36th
4
3336966 +/- 60324
33rd
14
3474440 +/- 6560
32nd
3
3622617 +/- 2512
31st
3
3661980
30th
3
3755293 +/- 15369
30th
11
3781791 +/- 225796
30th
4
3826542 +/- 9266
28th
3
4177253 +/- 958
28th
3
4180822 +/- 3737
27th
4
4293005 +/- 83
26th
14
4353398 +/- 5422
Low-Tier
25th
> 4371977
25th
3
4382592 +/- 18386
25th
3
4508289 +/- 101
25th
15
4609940 +/- 213154
25th
3
4614180 +/- 249
23rd
6
4891576 +/- 670870
22nd
13
5174104 +/- 42852
21st
9
5278125 +/- 1884
20th
4
5442567 +/- 32231
20th
6
5522806 +/- 31927
19th
6
5573186 +/- 1449
19th
9
5653801 +/- 292314
19th
3
5748968 +/- 566
18th
3
5758377 +/- 384
18th
3
5785801 +/- 15088
17th
4
5956926 +/- 108509
17th
3
5972134 +/- 13379
16th
3
6420592 +/- 84632
16th
3
6467311 +/- 629
16th
4
6490153 +/- 11453
15th
3
6832660 +/- 679
15th
3
6894610 +/- 1092
15th
3
6924027 +/- 10096
14th
3
7025878 +/- 25406
14th
3
7167092 +/- 714
14th
3
7203782 +/- 760
13th
4
7346190 +/- 2925
13th
3
7780924 +/- 7562
13th
3
7807922 +/- 893
12th
4
7820482 +/- 12835
11th
28
8101477 +/- 364988
9th
3
8293764 +/- 3127
9th
3
8372534 +/- 1682
8th
4
8575365 +/- 28108
6th
3
9377187 +/- 336
6th
12
9442529 +/- 379360
6th
3
9554590 +/- 2649
5th
3
10007035 +/- 85830
5th
5
10027546 +/- 724872
5th
7
10339239 +/- 631305
4th
3
13230678 +/- 805
4th
3
14680022 +/- 88498
3rd
3
16580844 +/- 6996
2nd
4
17609975 +/- 5034
2nd
5
17668280 +/- 497696
2nd
5
19554807 +/- 24764
2nd
3
20841444 +/- 11702
1st
3
21619078 +/- 8720
OpenBenchmarking.orgDistribution Of Public Results - Model: Inception V41028 Results Range From 493605 To 34247700 Microseconds4936051168687184376925188513193933386901545440975219179589426165693437244425791950785945899269671994475310619835112949171196999912645081133201631399524514670327153454091602049116695573173706551804573718720819193959012007098320746065214211472209622922771311234463932412147524796557254716392614672126821803274968852817196728847049295221313019721330872295315473773222245932897541335726233424770550100150200250

Based on OpenBenchmarking.org data, the selected test / test configuration (TensorFlow Lite 2020-08-23 - Model: Inception V4) has an average run-time of 6 minutes. By default this test profile is set to run at least 3 times but may increase if the standard deviation exceeds pre-defined defaults or other calculations deem additional runs necessary for greater statistical accuracy of the result.

OpenBenchmarking.orgMinutesTime Required To Complete BenchmarkModel: Inception V4Run-Time48121620Min: 1 / Avg: 5.78 / Max: 14

Based on public OpenBenchmarking.org results, the selected test / test configuration has an average standard deviation of 0.1%.

OpenBenchmarking.orgPercent, Fewer Is BetterAverage Deviation Between RunsModel: Inception V4Deviation246810Min: 0 / Avg: 0.15 / Max: 3

Does It Scale Well With Increasing Cores?

Yes, based on the automated analysis of the collected public benchmark data, this test / test settings does generally scale well with increasing CPU core counts. Data based on publicly available results for this test / test settings, separated by vendor, result divided by the reference CPU clock speed, grouped by matching physical CPU core count, and normalized against the smallest core count tested from each vendor for each CPU having a sufficient number of test samples and statistically significant data.

IntelAMDOpenBenchmarking.orgRelative Core Scaling To BaseTensorFlow Lite CPU Core ScalingModel: Inception V424681216243248643691215

Notable Instruction Set Usage

Notable instruction set extensions supported by this test, based on an automatic analysis by the Phoronix Test Suite / OpenBenchmarking.org analytics engine.

Instruction Set
Support
Instructions Detected
SSE2 (SSE2)
Used by default on supported hardware.
 
MOVDQA PMULUDQ PSHUFD PSRLDQ MOVD CVTSI2SD ADDSD MULSD SUBSD DIVSD MOVAPD CVTSS2SD CVTTSD2SI SQRTSD UCOMISD XORPD CVTSD2SS CVTTPS2DQ CVTDQ2PS PADDQ MOVDQU PUNPCKLQDQ UNPCKLPD CVTDQ2PD MULPD CVTPD2PS ANDPD MAXSD PSUBQ CVTPS2PD MINSD MOVUPD UNPCKHPD ADDPD PUNPCKHQDQ CVTPS2DQ
Used by default on supported hardware.
Found on Intel processors since Sandy Bridge (2011).
Found on AMD processors since Bulldozer (2011).

 
VBROADCASTF128 VZEROUPPER VMASKMOVPS VEXTRACTF128 VBROADCASTSS VINSERTF128 VPERMILPS
FMA (FMA)
Used by default on supported hardware.
Found on Intel processors since Haswell (2013).
Found on AMD processors since Bulldozer (2011).

 
VFMADD132PS VFMADD231PS VFMADD213PS
Last automated analysis: 31 January 2021

This test profile binary relies on the shared libraries libm.so.6, libpthread.so.0, libdl.so.2, librt.so.1, libc.so.6.

Recent Test Results

OpenBenchmarking.org Results Compare

1 System - 732 Benchmark Results

2 x Intel Xeon Platinum 8380 - Intel M50CYP2SB2U - Intel Device 0998

Ubuntu 20.04 - 5.11.6-051106-generic - GNOME Shell 3.36.4

3 Systems - 101 Benchmark Results

Intel Core i9-10980XE - ASRock X299 Steel Legend - Intel Sky Lake-E DMI3 Registers

Ubuntu 21.04 - 5.11.0-16-generic - GNOME Shell 3.38.4

2 Systems - 434 Benchmark Results

AMD EPYC 72F3 8-Core - Supermicro H12SSL-i v1.01 - AMD Starship

Ubuntu 21.04 - 5.11.0-16-generic - GNOME Shell 3.38.4

2 Systems - 102 Benchmark Results

Intel Core i9-10980XE - ASRock X299 Steel Legend - Intel Sky Lake-E DMI3 Registers

Clear Linux OS 34530 - 5.10.31-1033.native - GNOME Shell 3.38.4

1 System - 103 Benchmark Results

Intel Core i9-10980XE - ASRock X299 Steel Legend - Intel Sky Lake-E DMI3 Registers

Clear Linux OS 34530 - 5.10.31-1033.native - GNOME Shell 3.38.4

10 Systems - 454 Benchmark Results

AMD Ryzen 9 5950X 16-Core - ASUS ROG CROSSHAIR VIII HERO - AMD Starship

Ubuntu 21.04 - 5.12.0-051200rc3daily20210315-generic - GNOME Shell 3.38.3

1 System - 136 Benchmark Results

AMD EPYC 7763 64-Core - Supermicro H12SSL-i v1.01 - AMD Starship

Ubuntu 20.04 - 5.4.0-72-generic - GNOME Shell 3.36.7

12 Systems - 453 Benchmark Results

AMD Ryzen 5 5600X 6-Core - ASUS ROG CROSSHAIR VIII HERO - AMD Starship

Ubuntu 21.04 - 5.12.0-051200rc3daily20210315-generic - GNOME Shell 3.38.3

1 System - 6 Benchmark Results

Intel Core i3-3217U - Intel D33217GKE - Intel 3rd Gen Core DRAM

Debian 10 - 4.19.0-16-amd64 - ext4

1 System - 6 Benchmark Results

AMD Ryzen 7 5800X 8-Core - MSI B450-A PRO MAX - AMD Starship

Ubuntu 20.10 - 5.8.0-48-generic - GNOME Shell 3.38.2

Most Popular Test Results

OpenBenchmarking.org Results Compare

3 Systems - 268 Benchmark Results

Intel Core i5-2520M - HP 161C - Intel 2nd Generation Core DRAM

Ubuntu 18.04 - 4.18.0-20-generic - GNOME Shell 3.28.3

2 Systems - 535 Benchmark Results

AMD Ryzen 5 4500U - LENOVO LNVNB161216 - AMD Renoir Root Complex

Ubuntu 20.04 - 5.9.0-050900rc7daily20201002-generic - GNOME Shell 3.36.4

11 Systems - 217 Benchmark Results

Intel Core i5-10600K - Gigabyte Z490 AORUS MASTER - Intel Comet Lake PCH

Ubuntu 20.04 - 5.9.0-050900-generic - GNOME Shell 3.36.4

3 Systems - 174 Benchmark Results

Intel Core i9-10900K - Gigabyte Z490 AORUS MASTER - Intel Comet Lake PCH

Fedora 32 - 5.6.6-300.fc32.x86_64 - GNOME Shell 3.36.1

2 Systems - 403 Benchmark Results

Intel Core i9-10900K - Gigabyte Z490 AORUS MASTER - Intel Comet Lake PCH

Ubuntu 20.10 - 5.8.0-22-generic - GNOME Shell 3.38.0

12 Systems - 229 Benchmark Results

AMD Ryzen 9 3900X 12-Core - ASUS ROG CROSSHAIR VIII HERO - AMD Starship

Ubuntu 20.04 - 5.9.0-050900-generic - GNOME Shell 3.36.4

3 Systems - 406 Benchmark Results

Intel Core i9-10900K - Gigabyte Z490 AORUS MASTER - Intel Comet Lake PCH

Ubuntu 20.04 - 5.4.0-48-generic - GNOME Shell 3.36.4

2 Systems - 78 Benchmark Results

Intel Core i7-10700T - Logic Supply RXM-181 - Intel Comet Lake PCH

openSUSE 15.2 - 5.3.18-lp152.63-default - KDE Plasma 5.18.6

3 Systems - 11 Benchmark Results

AMD Ryzen 7 3800XT 8-Core - ASRock X570 Pro4 - AMD Starship

Ubuntu 20.04 - 5.4.0-42-generic - GNOME Shell 3.36.1

2 Systems - 81 Benchmark Results

2 x Intel Xeon Platinum 8280 - GIGABYTE MD61-SC2-00 v01000100 - Intel Sky Lake-E DMI3 Registers

Ubuntu 20.04 - 5.8.0-44-generic - GNOME Shell 3.36.4

8 Systems - 439 Benchmark Results

AMD Ryzen 9 5950X 16-Core - ASUS ROG CROSSHAIR VIII HERO - AMD Starship

Ubuntu 21.04 - 5.12.0-051200rc3daily20210315-generic - GNOME Shell 3.38.3

Find More Test Results