TensorFlow Lite

This is a benchmark of the TensorFlow Lite implementation focused on TensorFlow machine learning for mobile, IoT, edge, and other cases. The current Linux support is limited to running on CPUs. This test profile is measuring the average inference time.

To run this test with the Phoronix Test Suite, the basic command is: phoronix-test-suite benchmark tensorflow-lite.

Project Site

tensorflow.org

Source Repository

github.com

Test Created

23 August 2020

Last Updated

19 May 2022

Test Maintainer

Michael Larabel

Test Type

System

Average Install Time

9 Seconds

Average Run Time

4 Minutes, 53 Seconds

Accolades

10k+ Downloads

Supported Platforms

* Uploading of benchmark result data to OpenBenchmarking.org is always optional (opt-in) via the Phoronix Test Suite for users wishing to share their results publicly.
** Data based on those opting to upload their test results to OpenBenchmarking.org and users enabling the opt-in anonymous statistics reporting while running benchmarks from an Internet-connected platform.
*** Test profile page view reporting began March 2021.
Data updated weekly as of 13 April 2024.

Revision History

pts/tensorflow-lite-1.1.0 [View Source] Thu, 19 May 2022 09:57:39 GMT
Update against latest upstream nightly.

pts/tensorflow-lite-1.0.0 [View Source] Sun, 23 Aug 2020 14:13:10 GMT
TensorFlow Lite initial commit.

Suites Using This Test

Machine Learning

HPC - High Performance Computing

Performance Metrics

Analyze Test Configuration:

TensorFlow Lite 2022-05-18

Model: Mobilenet Float

OpenBenchmarking.org metrics for this test profile configuration based on 694 public results since 19 May 2022 with the latest data as of 3 April 2024.

Below is an overview of the generalized performance for components where there is sufficient statistically significant data based upon user-uploaded results. It is important to keep in mind particularly in the Linux/open-source space there can be vastly different OS configurations, with this overview intended to offer just general guidance as to the performance expectations.

Component

Percentile Rank

# Compatible Public Results

Microseconds (Average)

AMD Ryzen 7 7700X 8-Core

98th

1017 ^{+/- 8}

AMD Ryzen 9 7950X 16-Core

96th

1076 ^{+/- 49}

AMD Ryzen 9 7900X 12-Core

95th

1110 ^{+/- 9}

AMD Ryzen 9 7950X3D 16-Core

94th

1139 ^{+/- 76}

Intel Xeon Gold 6342

92nd

1297 ^{+/- 2}

AMD Ryzen 5 7600X 6-Core

91st

1334 ^{+/- 6}

Intel Core i5-13600K

89th

1406 ^{+/- 30}

Intel Xeon Platinum 8380

88th

1411 ^{+/- 2}

Intel Core i9-11900K

87th

1434 ^{+/- 13}

Intel Xeon Gold 6226R

85th

1518 ^{+/- 6}

Intel Xeon Gold 6346

85th

1527 ^{+/- 5}

AMD Ryzen 9 5950X 16-Core

83rd

1605 ^{+/- 2}

Intel Core i7-12700KF

83rd

1606 ^{+/- 113}

AMD Ryzen 5 8600G

82nd

1619

Intel Core i5-12600K

79th

1706 ^{+/- 70}

AMD Ryzen 9 5900X 12-Core

79th

1718 ^{+/- 14}

Intel Core i5-13400

77th

1796 ^{+/- 6}

AMD Ryzen 7 5800X3D 8-Core

76th

1806 ^{+/- 24}

Mid-Tier

75th

> 1817

2 x Intel Xeon Gold 6346

75th

1818 ^{+/- 10}

AMD Ryzen 7 5800X 8-Core

75th

1823 ^{+/- 142}

AMD Ryzen Threadripper 3960X 24-Core

73rd

1888 ^{+/- 7}

AMD Ryzen Threadripper 3970X 32-Core

72nd

1891 ^{+/- 8}

Intel Core i9-10900K

72nd

1901 ^{+/- 39}

2 x Intel Xeon Gold 6342

70th

2084 ^{+/- 3}

2 x Intel Xeon Gold 5220R

70th

2091 ^{+/- 6}

Intel Core i5-14500

67th

2165

Intel Core i5-12400

67th

2171 ^{+/- 81}

Intel Xeon Silver 4216

66th

2216 ^{+/- 18}

Intel Core i9-9900K

64th

2339 ^{+/- 38}

AMD Ryzen 7 5700G

64th

2345 ^{+/- 7}

AMD Ryzen 9 5900HX

63rd

2396 ^{+/- 38}

AMD Ryzen 9 3900X 12-Core

60th

2455 ^{+/- 18}

AMD Ryzen 9 3950X 16-Core

59th

2480 ^{+/- 40}

AMD Ryzen 5 5600X 6-Core

59th

2482 ^{+/- 108}

Intel Core i5-11600K

58th

2542 ^{+/- 10}

Intel Core i5-10600K

55th

2655 ^{+/- 12}

2 x Intel Xeon Platinum 8362

55th

2667 ^{+/- 50}

Intel Core i7-5960X

54th

2692 ^{+/- 53}

AMD Ryzen 5 5600G

53rd

2759 ^{+/- 23}

Intel Core i3-14100

53rd

2766

Intel Core i7-8700K

52nd

2788 ^{+/- 12}

AMD Ryzen 7 3800X 8-Core

52nd

2814 ^{+/- 136}

Median

50th

2880

AMD Ryzen 7 3800XT 8-Core

48th

2972 ^{+/- 278}

AMD Ryzen 7 3700X 8-Core

47th

3056 ^{+/- 30}

AMD EPYC 7F32 8-Core

46th

3156 ^{+/- 77}

AMD Ryzen 7 PRO 5850U

45th

3246 ^{+/- 65}

2 x Intel Xeon Platinum 8280

43rd

3392 ^{+/- 17}

2 x Intel Xeon Platinum 8380

42nd

3428 ^{+/- 303}

Intel Core Ultra 7 155H

41st

3445 ^{+/- 258}

AMD Ryzen 7 PRO 6850U

41st

3526 ^{+/- 522}

AMD Ryzen Threadripper 2970WX 24-Core

40th

3666 ^{+/- 37}

AMD Ryzen Threadripper PRO 7995WX 96-Cores

38th

3778 ^{+/- 138}

AMD Ryzen 3 3300X 4-Core

36th

3953 ^{+/- 15}

Apple M1

35th

3958 ^{+/- 66}

Intel Core i7-1185G7

35th

3999 ^{+/- 166}

Intel Core i7-1280P

35th

4017 ^{+/- 257}

Intel Core i7-1165G7

34th

4091 ^{+/- 50}

Intel Core i7-10700T

34th

4110 ^{+/- 99}

AMD Ryzen Threadripper 3990X 64-Core

32nd

4228 ^{+/- 64}

Intel Xeon E3-1275 v6

31st

4257 ^{+/- 8}

AMD Ryzen 5 5500U

29th

4541 ^{+/- 219}

Intel Core i3-10100

29th

4547 ^{+/- 675}

Intel Core i3-12100

28th

4598 ^{+/- 59}

Intel Xeon E3-1245 v5

27th

4660 ^{+/- 5}

Low-Tier

25th

> 4693

AMD Ryzen 7 4800U

24th

4909 ^{+/- 23}

Apple M2

23rd

4963 ^{+/- 450}

AMD EPYC 7551 32-Core

23rd

4965 ^{+/- 398}

Intel Xeon E3-1260L v5

22nd

5387 ^{+/- 53}

2 x AMD EPYC 75F3 32-Core

22nd

5606 ^{+/- 173}

Intel Core i3-8100

19th

5955 ^{+/- 4}

Intel Xeon E3-1231 v3

18th

6049 ^{+/- 99}

Intel Core i7-1065G7

18th

6147 ^{+/- 270}

Intel Core i7-8565U

16th

6857 ^{+/- 528}

AMD Ryzen 5 3400G

16th

6917 ^{+/- 5}

AMD Ryzen 5 4500U

15th

6974 ^{+/- 145}

Intel Xeon E5-2609 v4

14th

7159 ^{+/- 10}

Intel Core i3-8100T

13th

7316 ^{+/- 37}

Intel Core i3-7100

11th

8069 ^{+/- 17}

2 x AMD EPYC 7601 32-Core

11th

8393 ^{+/- 520}

Intel Core i3-1005G1

9th

9975 ^{+/- 7}

2 x AMD EPYC 7773X 64-Core

8th

10045 ^{+/- 535}

AMD Ryzen 3 3200U

5th

23400 ^{+/- 1608}

Intel Core i3-3110M

4th

27422 ^{+/- 55}

ARMv8 Cortex-A72 4-Core

2nd

68651 ^{+/- 3119}

Based on OpenBenchmarking.org data, the selected test / test configuration (TensorFlow Lite 2022-05-18 - Model: Mobilenet Float) has an average run-time of 5 minutes. By default this test profile is set to run at least 3 times but may increase if the standard deviation exceeds pre-defined defaults or other calculations deem additional runs necessary for greater statistical accuracy of the result.

Based on public OpenBenchmarking.org results, the selected test / test configuration has an average standard deviation of 0.5%.

Does It Scale Well With Increasing Cores?

No, based on the automated analysis of the collected public benchmark data, this test / test settings does not generally scale well with increasing CPU core counts. Data based on publicly available results for this test / test settings, separated by vendor, result divided by the reference CPU clock speed, grouped by matching physical CPU core count, and normalized against the smallest core count tested from each vendor for each CPU having a sufficient number of test samples and statistically significant data.

Notable Instruction Set Usage

Notable instruction set extensions supported by this test, based on an automatic analysis by the Phoronix Test Suite / OpenBenchmarking.org analytics engine.

Instruction Set

Support

Instructions Detected

SSE2 (SSE2)

Used by default on supported hardware.

MOVDQA PMULUDQ PSHUFD PSRLDQ MOVD CVTSI2SD ADDSD MULSD SUBSD DIVSD MOVAPD CVTSS2SD CVTTSD2SI SQRTSD UCOMISD XORPD CVTSD2SS CVTTPS2DQ CVTDQ2PS PADDQ MOVDQU PUNPCKLQDQ UNPCKLPD CVTDQ2PD MULPD CVTPD2PS ANDPD MAXSD PSUBQ CVTPS2PD MINSD MOVUPD UNPCKHPD ADDPD PUNPCKHQDQ CVTPS2DQ

Advanced Vector Extensions (AVX)

Used by default on supported hardware.
Found on Intel processors since Sandy Bridge (2011).
Found on AMD processors since Bulldozer (2011).

VBROADCASTF128 VZEROUPPER VMASKMOVPS VEXTRACTF128 VBROADCASTSS VINSERTF128 VPERMILPS

Advanced Vector Extensions 512 (AVX512)

Used by default on supported hardware.

(ZMM REGISTER USE)

FMA (FMA)

Used by default on supported hardware.
Found on Intel processors since Haswell (2013).
Found on AMD processors since Bulldozer (2011).

VFMADD132PS VFMADD231PS VFMADD213PS

Last automated analysis: 18 January 2022

This test profile binary relies on the shared libraries libm.so.6, libpthread.so.0, libdl.so.2, librt.so.1, libc.so.6.

Tested CPU Architectures

This benchmark has been successfully tested on the below mentioned architectures. The CPU architectures listed is where successful OpenBenchmarking.org result uploads occurred, namely for helping to determine if a given test is compatible with various alternative CPU architectures.

CPU Architecture

Kernel Identifier

Verified On

Intel / AMD x86 64-bit

x86_64

(Many Processors)

ARMv8 64-bit

aarch64

ARMv8 9-Core, ARMv8 Cortex-A53 2-Core, ARMv8 Cortex-A53 4-Core, ARMv8 Cortex-A72, ARMv8 Cortex-A72 4-Core, ARMv8 Cortex-A76 4-Core, ARMv8 Cortex-A78E 6-Core, ARMv8 Neoverse-N1, ARMv8 Neoverse-V1, Ampere ARMv8 Neoverse-N1 160-Core, Apple, Apple M1, Apple M2

pi5 1 System - 79 Benchmark Results	ARMv8 Cortex-A76 - Raspberry Pi 5 Model B Rev 1.0 - 4096MB Debian 12 - 6.6.20+rpt-rpi-2712 - X Server
edge2 1 System - 80 Benchmark Results	ARMv8 Cortex-A76 - Khadas Edge2 - 16GB Ubuntu 22.04 - 5.10.198 - 1.3.255
orin 1 System - 80 Benchmark Results	ARMv8 Cortex-A78E - EDK II 4.1-33958178 - 8GB Ubuntu 20.04 - 5.10.120-tegra - GNOME Shell 3.36.9
mixtile 1 System - 80 Benchmark Results	ARMv8 Cortex-A76 - Mixtile Core 3588E - 16GB Ubuntu 22.04 - 5.10.160-rockchip - GNOME Shell 42.9
fedora-beta-40 Featured Kernel Comparison	6.8.1-300.fc40.x86_64 - 6.8.2-300.fc40.x86_64
intel681 2 Systems - 56 Benchmark Results	Intel Core Ultra 7 155H - MSI MS-15A1 - Intel Device 7e7f EndeavourOS rolling - 6.8.1-arch1-1 - KDE Plasma 6.0.2
24.03.13.Pop.2204.ML.test1 1 System - 341 Benchmark Results	AMD Ryzen 9 7950X 16-Core - ASUS ProArt X670E-CREATOR WIFI - AMD Device 14d8 Pop 22.04 - 6.6.10-76060610-generic - GNOME Shell 42.5
ti-arm-2-core-tensorflow-lite-2-29-24 1 System - 6 Benchmark Results	ARMv8 Cortex-A72 - Texas Instruments J721E SK - 2560MB Ubuntu 22.04.4 LTS - 6.1.46-g5892b80d6b - overlayfs
c3708-tensorflow-lite 1 System - 183 Benchmark Results	Intel Atom C3708 - Kontron Europe COMe-bDV7 E2 v2.0.1 - 32GB Ubuntu 22.04.4 LTS - 4.18.0-448.el8.x86_64 - overlayfs
arm8-tensorflow-lite-monitor-all 1 System - 103 Benchmark Results	ARMv8 Cortex-A72 - Texas Instruments AM69 SK - 28GB Ubuntu 22.04.4 LTS - 6.1.46-g5892b80d6b - GCC 11.4.0
arm8-tensorflow-lite 1 System - 6 Benchmark Results	ARMv8 Cortex-A72 - Texas Instruments AM69 SK - 28GB Ubuntu 22.04.4 LTS - 6.1.46-g5892b80d6b - GCC 11.4.0
arm8-tensorflow-lite 1 System - 6 Benchmark Results	ARMv8 Cortex-A72 - Texas Instruments AM69 SK - 28GB Ubuntu 22.04.4 LTS - 6.1.46-g5892b80d6b - GCC 11.4.0
New CPU Benchmarks 5 Systems - 587 Benchmark Results	Intel Core i9-14900K - ASUS PRIME Z790-P WIFI - Intel Raptor Lake-S PCH Ubuntu 23.10 - 6.7.3-060703-generic - GNOME Shell 45.2
HDR3-A44000-1 1 System - 304 Benchmark Results	AMD A4-5300 APU - ASRock FM2A88M-HD+ R3.0 - AMD 15h Ubuntu 20.04 - 5.15.0-89-generic - GNOME Shell 3.36.9
New Tests 1 System - 1253 Benchmark Results

AMD Ryzen 7900 Series Linux CPU Benchmarks 13 Systems - 333 Benchmark Results	AMD Ryzen 5 5600X 6-Core - ASUS ROG CROSSHAIR VIII HERO - AMD Starship Ubuntu 22.04 - 6.0.0-060000rc1daily20220820-generic - GNOME Shell 42.2
Amazon EC2 Graviton3 Benchmark Comparison 5 Systems - 94 Benchmark Results	ARMv8 Neoverse-N1 - Amazon EC2 c6g.4xlarge - Amazon Device 0200 Ubuntu 22.04 - 5.15.0-1004-aws - GCC 11.2.0
HP Dev One Pop!_OS 22.04 Linux Laptop Benchmarks 7 Systems - 293 Benchmark Results	AMD Ryzen 9 5900HX - ASUS G513QY v1.0 - AMD Renoir Pop 22.04 - 5.17.5-76051705-generic - GNOME Shell 42.1
Intel Core i9 13900K Linux CPU Benchmarks 16 Systems - 333 Benchmark Results	AMD Ryzen 5 5600X 6-Core - ASUS ROG CROSSHAIR VIII HERO - AMD Starship Ubuntu 22.04 - 6.0.0-060000rc1daily20220820-generic - GNOME Shell 42.2
10900K June Featured OpenGL Comparison	Intel Core i9-10900K - Gigabyte Z490 AORUS MASTER - Intel Comet Lake PCH Ubuntu 21.10 - 5.17.0-051700rc7daily20220309-generic - GNOME Shell 40.5
AMD Ryzen 9 7950X3D Linux Review Benchmarks 5 Systems - 396 Benchmark Results	AMD Ryzen 9 7950X3D 16-Core - ASUS ROG CROSSHAIR X670E HERO - AMD Device 14d8 Ubuntu 23.04 - 6.2.0-060200rc8daily20230213-generic - GNOME Shell 43.2
gromacs ren 3 Systems - 18 Benchmark Results	AMD Ryzen Threadripper 3990X 64-Core - Gigabyte TRX40 AORUS PRO WIFI - AMD Starship Pop 22.04 - 5.17.5-76051705-generic - GNOME Shell 42.0
xeon june 2 Systems - 86 Benchmark Results	Intel Xeon Silver 4216 - TYAN S7100AG2NR - Intel Sky Lake-E DMI3 Registers Debian 11 - 5.10.0-10-amd64 - X Server
Intel 12th Gen Core Alder Lake Ubuntu Linux Benchmarks Featured Processor Comparison	Intel Core i5-12400 - ASRock B660M-HDV - Intel Device 7aa7 Ubuntu 22.04 - 5.18.0-051800-generic - GNOME Shell 42.1
3300X June 2022 Featured OpenGL Comparison	AMD Ryzen 3 3300X 4-Core - MSI B350M GAMING PRO - AMD Starship Ubuntu 20.04 - 5.11.0-41-generic - GNOME Shell 3.36.4
3960x-june-2022 4 Systems - 157 Benchmark Results	AMD Ryzen Threadripper 3960X 24-Core - MSI Creator TRX40 - AMD Starship Ubuntu 20.04 - 5.12.0-051200rc2daily20210307-generic - GNOME Shell 3.36.4
Apple M1 Mac Mini vs. Core i7 2 Systems - 218 Benchmark Results	Apple M1 - Apple Mac mini - 8GB Arch Linux ARM - 5.19.0-rc7-asahi-2-1-ARCH - KDE Plasma 5.25.4
AMD Ryzen 7600X 7700X series extra Featured Processor Comparison	AMD Ryzen 7 7700X 8-Core - ASUS ROG CROSSHAIR X670E HERO - AMD Device 14d8 Ubuntu 22.04 - 6.0.0-060000rc1daily20220820-generic - GNOME Shell 42.2
5600G Ryzen AMD Linux 4 Systems - 171 Benchmark Results	AMD Ryzen 5 5600G - Gigabyte A520M DS3H - AMD Renoir Ubuntu 22.04 - 5.15.0-41-generic - GNOME Shell 42.2
cascade lake summer 2022 3 Systems - 178 Benchmark Results	2 x Intel Xeon Platinum 8280 - GIGABYTE MD61-SC2-00 v01000100 - Intel Sky Lake-E DMI3 Registers Ubuntu 21.04 - 5.11.0-40-generic - GNOME Shell 3.38.4

TensorFlow Lite

Project Site

Source Repository

Test Created

Last Updated

Test Maintainer

Test Type

Average Install Time

Average Run Time

Accolades

Supported Platforms

Revision History

Suites Using This Test

Performance Metrics

TensorFlow Lite 2022-05-18

Model: Mobilenet Float

Does It Scale Well With Increasing Cores?

Notable Instruction Set Usage

Tested CPU Architectures

Recent Test Results

Most Popular Test Results