TensorFlow Lite

This is a benchmark of the TensorFlow Lite implementation focused on TensorFlow machine learning for mobile, IoT, edge, and other cases. The current Linux support is limited to running on CPUs. This test profile is measuring the average inference time.

To run this test with the Phoronix Test Suite, the basic command is: phoronix-test-suite benchmark tensorflow-lite.

Project Site

tensorflow.org

Source Repository

github.com

Test Created

23 August 2020

Last Updated

19 May 2022

Test Maintainer

Michael Larabel

Test Type

System

Average Install Time

8 Seconds

Average Run Time

4 Minutes, 31 Seconds

Accolades

10k+ Downloads

Supported Platforms

* Uploading of benchmark result data to OpenBenchmarking.org is always optional (opt-in) via the Phoronix Test Suite for users wishing to share their results publicly.
** Data based on those opting to upload their test results to OpenBenchmarking.org and users enabling the opt-in anonymous statistics reporting while running benchmarks from an Internet-connected platform.
*** Test profile page view reporting began March 2021.
Data updated weekly as of 3 October 2024.

Revision History

pts/tensorflow-lite-1.1.0 [View Source] Thu, 19 May 2022 09:57:39 GMT
Update against latest upstream nightly.

pts/tensorflow-lite-1.0.0 [View Source] Sun, 23 Aug 2020 14:13:10 GMT
TensorFlow Lite initial commit.

Suites Using This Test

Machine Learning

HPC - High Performance Computing

Performance Metrics

Analyze Test Configuration:

TensorFlow Lite 2022-05-18

Model: Mobilenet Float

OpenBenchmarking.org metrics for this test profile configuration based on 792 public results since 19 May 2022 with the latest data as of 10 September 2024.

Below is an overview of the generalized performance for components where there is sufficient statistically significant data based upon user-uploaded results. It is important to keep in mind particularly in the Linux/open-source space there can be vastly different OS configurations, with this overview intended to offer just general guidance as to the performance expectations.

Component

Percentile Rank

# Compatible Public Results

Microseconds (Average)

AMD Ryzen 7 7700X 8-Core

98th

1017 ^{+/- 8}

AMD Ryzen 9 7950X 16-Core

97th

1076 ^{+/- 49}

AMD Ryzen 9 7900X 12-Core

96th

1110 ^{+/- 9}

AMD Ryzen 9 7950X3D 16-Core

95th

1139 ^{+/- 76}

Intel Xeon Gold 6342

93rd

1297 ^{+/- 2}

AMD Ryzen 5 7600X 6-Core

92nd

1334 ^{+/- 6}

Intel Core i5-13600K

90th

1406 ^{+/- 30}

Intel Xeon Platinum 8380

90th

1411 ^{+/- 2}

Intel Core i9-11900K

89th

1434 ^{+/- 13}

Intel Xeon Gold 6226R

87th

1518 ^{+/- 6}

Intel Xeon Gold 6346

87th

1527 ^{+/- 5}

AMD Ryzen 9 5950X 16-Core

85th

1605 ^{+/- 2}

Intel Core i7-12700KF

85th

1606 ^{+/- 113}

AMD Ryzen 5 8600G

84th

1619

Intel Core i5-12600K

82nd

1706 ^{+/- 70}

AMD Ryzen 9 5900X 12-Core

82nd

1718 ^{+/- 14}

Intel Core i5-13400

80th

1796 ^{+/- 6}

AMD Ryzen 7 5800X3D 8-Core

79th

1806 ^{+/- 24}

2 x Intel Xeon Gold 6346

78th

1818 ^{+/- 10}

AMD Ryzen 7 5800X 8-Core

78th

1823 ^{+/- 142}

AMD Ryzen Threadripper 3960X 24-Core

76th

1888 ^{+/- 7}

AMD Ryzen Threadripper 3970X 32-Core

76th

1891 ^{+/- 8}

Mid-Tier

75th

> 1891

Intel Core i9-10900K

75th

1901 ^{+/- 39}

2 x Intel Xeon Gold 6342

74th

2084 ^{+/- 3}

2 x Intel Xeon Gold 5220R

73rd

2091 ^{+/- 6}

Intel Core i5-14500

71st

2165

Intel Core i5-12400

71st

2171 ^{+/- 81}

Intel Xeon Silver 4216

70th

2216 ^{+/- 18}

Intel Core i9-9900K

68th

2339 ^{+/- 38}

AMD Ryzen 7 5700G

68th

2345 ^{+/- 7}

AMD Ryzen 9 5900HX

67th

2396 ^{+/- 38}

AMD Ryzen 9 3900X 12-Core

65th

2455 ^{+/- 18}

AMD Ryzen 9 3950X 16-Core

64th

2480 ^{+/- 40}

AMD Ryzen 5 5600X 6-Core

63rd

2482 ^{+/- 108}

Intel Core i5-11600K

62nd

2542 ^{+/- 10}

Intel Core i5-10600K

60th

2655 ^{+/- 12}

2 x Intel Xeon Platinum 8362

60th

2667 ^{+/- 50}

Intel Core i7-5960X

60th

2692 ^{+/- 53}

AMD Ryzen 5 5600G

59th

2759 ^{+/- 23}

Intel Core i3-14100

58th

2766

Intel Core i7-8700K

58th

2788 ^{+/- 12}

AMD Ryzen 7 3800X 8-Core

57th

2814 ^{+/- 136}

AMD Ryzen 7 3800XT 8-Core

54th

2972 ^{+/- 278}

AMD Ryzen 7 3700X 8-Core

53rd

3033 ^{+/- 59}

AMD EPYC 7F32 8-Core

52nd

3156 ^{+/- 77}

AMD Ryzen 7 PRO 5850U

51st

3246 ^{+/- 65}

Median

50th

3296

2 x Intel Xeon Platinum 8280

49th

3392 ^{+/- 17}

2 x Intel Xeon Platinum 8380

48th

3428 ^{+/- 303}

Intel Core Ultra 7 155H

47th

3445 ^{+/- 258}

AMD Ryzen 7 PRO 6850U

46th

3526 ^{+/- 522}

AMD Ryzen Threadripper 2970WX 24-Core

45th

3666 ^{+/- 37}

AMD Ryzen Threadripper PRO 7995WX 96-Cores

44th

3778 ^{+/- 138}

Intel Core i7-1185G7E

37th

3939 ^{+/- 15}

AMD Ryzen 3 3300X 4-Core

35th

3953 ^{+/- 15}

Apple M1

35th

3954 ^{+/- 64}

Intel Core i7-1185G7

33rd

3999 ^{+/- 166}

Intel Core i7-1280P

33rd

4017 ^{+/- 257}

Intel Core i7-1165G7

32nd

4091 ^{+/- 50}

Intel Core i7-10700T

32nd

4110 ^{+/- 99}

AMD Ryzen Threadripper 3990X 64-Core

30th

4228 ^{+/- 64}

Intel Xeon E3-1275 v6

29th

4257 ^{+/- 8}

Intel Core i3-12100

27th

4470 ^{+/- 366}

AMD Ryzen 5 5500U

27th

4541 ^{+/- 219}

Intel Core i3-10100

27th

4547 ^{+/- 675}

Low-Tier

25th

> 4655

Intel Xeon E3-1245 v5

25th

4660 ^{+/- 5}

AMD Ryzen 7 4800U

23rd

4909 ^{+/- 23}

Apple M2

22nd

4963 ^{+/- 450}

AMD EPYC 7551 32-Core

22nd

4965 ^{+/- 398}

Intel Xeon E3-1260L v5

21st

5387 ^{+/- 53}

2 x AMD EPYC 75F3 32-Core

21st

5606 ^{+/- 173}

Intel Core i3-8100

18th

5955 ^{+/- 4}

Intel Xeon E3-1231 v3

17th

6049 ^{+/- 99}

Intel Core i7-1065G7

17th

6147 ^{+/- 270}

Intel Core i7-8565U

16th

6857 ^{+/- 528}

AMD Ryzen 5 3400G

15th

6917 ^{+/- 5}

AMD Ryzen 5 4500U

15th

6974 ^{+/- 145}

Intel Xeon E5-2609 v4

14th

7159 ^{+/- 10}

Intel Core i3-8100T

13th

7316 ^{+/- 37}

Intel Core i3-7100

11th

8069 ^{+/- 17}

2 x AMD EPYC 7601 32-Core

11th

8393 ^{+/- 520}

Intel Core i3-1005G1

9th

9975 ^{+/- 7}

2 x AMD EPYC 7773X 64-Core

9th

10045 ^{+/- 535}

AMD Ryzen 3 3200U

5th

23400 ^{+/- 1608}

Intel Core i3-3110M

4th

27422 ^{+/- 55}

ARMv8 Cortex-A72 4-Core

2nd

68651 ^{+/- 3119}

AmpereOne 192-Core

1st

277617 ^{+/- 152}

Based on OpenBenchmarking.org data, the selected test / test configuration (TensorFlow Lite 2022-05-18 - Model: Mobilenet Float) has an average run-time of 5 minutes. By default this test profile is set to run at least 3 times but may increase if the standard deviation exceeds pre-defined defaults or other calculations deem additional runs necessary for greater statistical accuracy of the result.

Based on public OpenBenchmarking.org results, the selected test / test configuration has an average standard deviation of 0.8%.

Does It Scale Well With Increasing Cores?

No, based on the automated analysis of the collected public benchmark data, this test / test settings does not generally scale well with increasing CPU core counts. Data based on publicly available results for this test / test settings, separated by vendor, result divided by the reference CPU clock speed, grouped by matching physical CPU core count, and normalized against the smallest core count tested from each vendor for each CPU having a sufficient number of test samples and statistically significant data.

Notable Instruction Set Usage

Notable instruction set extensions supported by this test, based on an automatic analysis by the Phoronix Test Suite / OpenBenchmarking.org analytics engine.

Instruction Set

Support

Instructions Detected

SSE2 (SSE2)

Used by default on supported hardware.

MOVDQA PMULUDQ PSHUFD PSRLDQ MOVD CVTSI2SD ADDSD MULSD SUBSD DIVSD MOVAPD CVTSS2SD CVTTSD2SI SQRTSD UCOMISD XORPD CVTSD2SS CVTTPS2DQ CVTDQ2PS PADDQ MOVDQU PUNPCKLQDQ UNPCKLPD CVTDQ2PD MULPD CVTPD2PS ANDPD MAXSD PSUBQ CVTPS2PD MINSD MOVUPD UNPCKHPD ADDPD PUNPCKHQDQ CVTPS2DQ

Advanced Vector Extensions (AVX)

Used by default on supported hardware.
Found on Intel processors since Sandy Bridge (2011).
Found on AMD processors since Bulldozer (2011).

VBROADCASTF128 VZEROUPPER VMASKMOVPS VEXTRACTF128 VBROADCASTSS VINSERTF128 VPERMILPS

Advanced Vector Extensions 512 (AVX512)

Used by default on supported hardware.

(ZMM REGISTER USE)

FMA (FMA)

Used by default on supported hardware.
Found on Intel processors since Haswell (2013).
Found on AMD processors since Bulldozer (2011).

VFMADD132PS VFMADD231PS VFMADD213PS

Last automated analysis: 18 January 2022

This test profile binary relies on the shared libraries libm.so.6, libpthread.so.0, libdl.so.2, librt.so.1, libc.so.6.

Tested CPU Architectures

This benchmark has been successfully tested on the below mentioned architectures. The CPU architectures listed is where successful OpenBenchmarking.org result uploads occurred, namely for helping to determine if a given test is compatible with various alternative CPU architectures.

CPU Architecture

Kernel Identifier

Verified On

Intel / AMD x86 64-bit

x86_64

(Many Processors)

ARMv8 64-bit

aarch64

ARMv8 9-Core, ARMv8 Cortex-A53 2-Core, ARMv8 Cortex-A53 4-Core, ARMv8 Cortex-A72, ARMv8 Cortex-A72 4-Core, ARMv8 Cortex-A76 4-Core, ARMv8 Cortex-A78E 12-Core, ARMv8 Cortex-A78E 6-Core, ARMv8 Cortex-A78E 8-Core, ARMv8 Neoverse-N1, ARMv8 Neoverse-V1, Ampere ARMv8 Neoverse-N1 160-Core, AmpereOne 192-Core, Apple, Apple M1, Apple M2, Qualcomm

tflite 1 System - 6 Benchmark Results	ARMv8 Cortex-A78E - NVIDIA Jetson Orin NX Engineering Developer Kit - 16GB Ubuntu 22.04 - 5.15.136-tegra - GNOME Shell 42.9
tflite 1 System - 6 Benchmark Results	ARMv8 Cortex-A78E - NVIDIA Jetson Orin Nano Developer Kit - 8GB Ubuntu 22.04 - 5.15.136-tegra - GNOME Shell 42.9
tflite 1 System - 6 Benchmark Results	ARMv8 Cortex-A78E - NVIDIA Jetson AGX Orin Developer Kit - 30GB Ubuntu 22.04 - 5.15.136-tegra - GNOME Shell 42.9
pytorch.txt 1 System - 6 Benchmark Results	ARMv8 Cortex-A78E - NVIDIA Jetson Orin NX Engineering Developer Kit - 16GB Ubuntu 22.04 - 5.15.136-tegra - GNOME Shell 42.9
Intel Core i7-1185G7E Performance 1 System - 4 Benchmark Results	Intel Core i7-1185G7E - Rockwell Automation/Allen-Bradley 6300B-JB1 MB1561 v1.1 - Intel Tiger Lake-LP Debian 12 - 6.1.0-23-amd64 - KDE Plasma 5.27.5
Intel Core i7-1185G7E Benchmarks 1 System - 7 Benchmark Results	Intel Core i7-1185G7E - Rockwell Automation/Allen-Bradley 6300B-JB1 MB1561 v1.1 - Intel Tiger Lake-LP Debian 12 - 6.1.0-23-amd64 - KDE Plasma 5.27.5
Intel Core i7-1185G7E On Debian 12 1 System - 3 Benchmark Results	Intel Core i7-1185G7E - Rockwell Automation/Allen-Bradley 6300B-JB1 MB1561 v1.1 - Intel Tiger Lake-LP Debian 12 - 6.1.0-23-amd64 - KDE Plasma 5.27.5
Intel Core i7-1185G7E On Debian 12 1 System - 5 Benchmark Results	Intel Core i7-1185G7E - Rockwell Automation/Allen-Bradley 6300B-JB1 MB1561 v1.1 - Intel Tiger Lake-LP Debian 12 - 6.1.0-23-amd64 - KDE Plasma 5.27.5
Debian 12 Performance 1 System - 8 Benchmark Results	Intel Core i7-1185G7E - Rockwell Automation/Allen-Bradley 6300B-JB1 MB1561 v1.1 - Intel Tiger Lake-LP Debian 12 - 6.1.0-23-amd64 - KDE Plasma 5.27.5
Intel Core i7-1185G7E On Debian 12 1 System - 6 Benchmark Results	Intel Core i7-1185G7E - Rockwell Automation/Allen-Bradley 6300B-JB1 MB1561 v1.1 - Intel Tiger Lake-LP Debian 12 - 6.1.0-23-amd64 - KDE Plasma 5.27.5
Intel Core i7-1185G7E Benchmarks 1 System - 9 Benchmark Results	Intel Core i7-1185G7E - Rockwell Automation/Allen-Bradley 6300B-JB1 MB1561 v1.1 - Intel Tiger Lake-LP Debian 12 - 6.1.0-23-amd64 - KDE Plasma 5.27.5
6.1.0-23-amd64 + Debian 12 Tests 1 System - 8 Benchmark Results	Intel Core i7-1185G7E - Rockwell Automation/Allen-Bradley 6300B-JB1 MB1561 v1.1 - Intel Tiger Lake-LP Debian 12 - 6.1.0-23-amd64 - KDE Plasma 5.27.5
Intel Core i7-1185G7E Performance 1 System - 7 Benchmark Results	Intel Core i7-1185G7E - Rockwell Automation/Allen-Bradley 6300B-JB1 MB1561 v1.1 - Intel Tiger Lake-LP Debian 12 - 6.1.0-23-amd64 - KDE Plasma 5.27.5
Debian 12 Performance 1 System - 8 Benchmark Results	Intel Core i7-1185G7E - Rockwell Automation/Allen-Bradley 6300B-JB1 MB1561 v1.1 - Intel Tiger Lake-LP Debian 12 - 6.1.0-23-amd64 - KDE Plasma 5.27.5
Intel Core i7-1185G7E Benchmarks 1 System - 5 Benchmark Results	Intel Core i7-1185G7E - Rockwell Automation/Allen-Bradley 6300B-JB1 MB1561 v1.1 - Intel Tiger Lake-LP Debian 12 - 6.1.0-23-amd64 - KDE Plasma 5.27.5

Intel Core i9 13900K Linux CPU Benchmarks 16 Systems - 333 Benchmark Results	Intel Core i9-13900K - ASUS PRIME Z790-P WIFI - Intel Device 7a27 Ubuntu 22.04 - 6.0.0-060000rc1daily20220820-generic - GNOME Shell 42.2
AMD Ryzen 7900 Series Linux CPU Benchmarks 13 Systems - 333 Benchmark Results	AMD Ryzen 9 7900X 12-Core - ASUS ROG CROSSHAIR X670E HERO - AMD Device 14d8 Ubuntu 22.04 - 6.0.0-060000rc1daily20220820-generic - GNOME Shell 42.2
AMD Ryzen 9 7950X3D Linux Review Benchmarks 5 Systems - 396 Benchmark Results	Intel Core i9-13900K - ASUS PRIME Z790-P WIFI - Intel Device 7a27 Ubuntu 23.04 - 6.2.0-060200rc8daily20230213-generic - GNOME Shell 43.2
HP Dev One Pop!_OS 22.04 Linux Laptop Benchmarks 7 Systems - 293 Benchmark Results	AMD Ryzen 5 4500U - LENOVO LNVNB161216 - AMD Renoir Pop 22.04 - 5.17.5-76051705-generic - GNOME Shell 42.1
Amazon EC2 Graviton3 Benchmark Comparison 5 Systems - 94 Benchmark Results	ARMv8 Neoverse-N1 - Amazon EC2 c6g.4xlarge - Amazon Device 0200 Ubuntu 22.04 - 5.15.0-1004-aws - GCC 11.2.0
gromacs ren 3 Systems - 18 Benchmark Results	AMD Ryzen Threadripper 3990X 64-Core - Gigabyte TRX40 AORUS PRO WIFI - AMD Starship Pop 22.04 - 5.17.5-76051705-generic - GNOME Shell 42.0
Intel 12th Gen Core Alder Lake Ubuntu Linux Benchmarks Featured Processor Comparison	Intel Celeron G6900 - ASRock B660M-HDV - Intel Device 7aa7 Ubuntu 22.04 - 5.18.0-051800-generic - GNOME Shell 42.1
AMD Ryzen 7 PRO 6850U Linux Laptop Benchmarks 11 Systems - 408 Benchmark Results	AMD Ryzen 7 PRO 5850U - HP 8A78 - AMD Renoir Ubuntu 22.04 - 5.18.8-051808-generic - GNOME Shell 42.2
xeon june 2 Systems - 86 Benchmark Results	Intel Xeon Silver 4216 - TYAN S7100AG2NR - Intel Sky Lake-E DMI3 Registers Debian 11 - 5.10.0-10-amd64 - X Server
10900K June Featured OpenGL Comparison	Intel Core i9-10900K - Gigabyte Z490 AORUS MASTER - Intel Comet Lake PCH Ubuntu 21.10 - 5.17.0-051700rc7daily20220309-generic - GNOME Shell 40.5
EPYC Milan-X Linux 5.19 Benchmarks Featured Kernel Comparison	2 x AMD EPYC 7773X 64-Core - AMD DAYTONA_X - AMD Starship Ubuntu 21.04 - 5.17.0-051700-generic - GNOME Shell 3.38.4
Apple M1 Mac Mini vs. Core i7 2 Systems - 218 Benchmark Results	Apple M1 - Apple Mac mini - 8GB Arch Linux ARM - 5.19.0-rc7-asahi-2-1-ARCH - KDE Plasma 5.25.4
AMD Ryzen 7600X 7700X series extra Featured Processor Comparison	AMD Ryzen 7 7700X 8-Core - ASUS ROG CROSSHAIR X670E HERO - AMD Device 14d8 Ubuntu 22.04 - 6.0.0-060000rc1daily20220820-generic - GNOME Shell 42.2
3300X June 2022 Featured OpenGL Comparison	AMD Ryzen 3 3300X 4-Core - MSI B350M GAMING PRO - AMD Starship Ubuntu 20.04 - 5.11.0-41-generic - GNOME Shell 3.36.4
Apple M1 Mac Mini 2 Systems - 219 Benchmark Results	AMD Ryzen 9 5900HX - ASUS G513QY v1.0 - AMD Renoir Arch rolling - 5.18.16-arch1-1 - KDE Plasma 5.25.4

TensorFlow Lite

Project Site

Source Repository

Test Created

Last Updated

Test Maintainer

Test Type

Average Install Time

Average Run Time

Accolades

Supported Platforms

Revision History

Suites Using This Test

Performance Metrics

TensorFlow Lite 2022-05-18

Model: Mobilenet Float

Does It Scale Well With Increasing Cores?

Notable Instruction Set Usage

Tested CPU Architectures

Recent Test Results

Most Popular Test Results