Caffe AlexNet

This is a benchmark of the Caffe deep learning framework and currently supports the AlexNet and Googlenet model and execution on both CPUs and NVIDIA GPUs.

To run this test with the Phoronix Test Suite, the basic command is: phoronix-test-suite benchmark caffe.

Test Created

14 November 2015

Last Updated

26 September 2020

Test Maintainer

Michael Larabel

Test Type

System

Average Install Time

32 Seconds

Average Run Time

2 Minutes, 1 Second

Accolades

100k+ Downloads

Supported Platforms


Public Result UploadsReported Installs*Test Completions*OpenBenchmarking.orgEventsCaffe AlexNet Popularity Statisticspts/caffe2015.112016.012016.032016.072016.092016.112017.012017.032017.052017.072017.092017.112018.012018.032018.052018.072018.092018.112019.012019.032019.052019.092020.012020.072020.102020.122K4K6K8K10K
* Data based on those opting to upload their test results to OpenBenchmarking.org and users enabling the opt-in anonymous statistics reporting while running benchmarks from an Internet-connected platform.
Data current as of Sat, 23 Jan 2021 09:53:30 GMT.
GoogleNet49.7%AlexNet50.3%Model Option PopularityOpenBenchmarking.org
NVIDIA CUDA 18.8%CPU81.2%Acceleration Option PopularityOpenBenchmarking.org
20043.3%100012.4%10044.3%Iterations Option PopularityOpenBenchmarking.org

Revision History

pts/caffe-1.5.0   [View Source]   Sat, 26 Sep 2020 21:35:45 GMT
Overhaul Caffe test profile with latest Git snapshot, switch to CMake build system, clean up test options, etc.

pts/caffe-1.4.0   [View Source]   Sat, 29 Dec 2018 11:15:41 GMT
Update Caffe to latest Git snapshot to hopefully workaround build problems on newer distros.

pts/caffe-1.3.3   [View Source]   Sun, 01 Apr 2018 18:50:19 GMT
Basic fix for OpenCV 3.4.

pts/caffe-1.3.2   [View Source]   Wed, 04 Jan 2017 11:07:36 GMT
Fix for OpenCV 3.2.

pts/caffe-1.3.1   [View Source]   Wed, 28 Dec 2016 20:36:42 GMT
Don't show title string of "Caffe AlexNet" but "Caffe" with recent test profile versions supporting more than just AlexNet.

pts/caffe-1.3.0   [View Source]   Wed, 28 Dec 2016 20:34:27 GMT
Update to latest Git snapshot to fix OpenCV compatibility.

pts/caffe-1.2.0   [View Source]   Mon, 15 Aug 2016 16:11:16 GMT
Add Googlenet support, decrease CPU only iteration count.

pts/caffe-1.1.1   [View Source]   Sun, 12 Jun 2016 18:32:44 GMT
Add OpenCV and OpenBLAS support.

pts/caffe-1.1.0   [View Source]   Sat, 11 Jun 2016 19:32:55 GMT
Update

pts/caffe-1.0.0   [View Source]   Sat, 14 Nov 2015 15:29:45 GMT
Initial commit of Caffe deep learning framework and with this benchmark using the AlexNet model for benchmarking.

Suites Using This Test

Machine Learning

HPC - High Performance Computing

NVIDIA GPU Compute


Performance Metrics

Analyze Test Configuration:

Caffe 2020-02-13

Model: AlexNet - Acceleration: CPU - Iterations: 100

OpenBenchmarking.org metrics for this test profile configuration based on 448 public results since 26 September 2020 with the latest data as of 21 January 2021.

Below is an overview of the generalized performance for components where there is sufficient statistically significant data based upon user-uploaded results. It is important to keep in mind particularly in the Linux/open-source space there can be vastly different OS configurations, with this overview intended to offer just general guidance as to the performance expectations.

Component
Percentile Rank
# Matching Public Results
Milli-Seconds (Average)
100th
3
31313 +/- 140
99th
3
32010 +/- 67
99th
3
34822 +/- 9
98th
5
35373 +/- 193
96th
7
36528 +/- 290
95th
3
37173 +/- 22
93rd
3
37955 +/- 132
92nd
3
38315 +/- 72
92nd
12
38664 +/- 1255
90th
6
39683 +/- 446
88th
3
40100 +/- 99
88th
9
40193 +/- 288
86th
3
40696 +/- 46
85th
7
41490 +/- 228
84th
3
42896 +/- 16
83rd
4
43703 +/- 1085
83rd
3
44032 +/- 123
81st
4
44786 +/- 117
80th
3
46908 +/- 38
79th
3
48030 +/- 65
78th
6
49030 +/- 55
76th
3
50195 +/- 92
Mid-Tier
75th
> 51290
75th
3
51637 +/- 187
74th
3
51775 +/- 77
74th
6
51822 +/- 2121
73rd
3
52433 +/- 1007
69th
3
53894 +/- 254
69th
5
53938 +/- 2643
68th
3
54246 +/- 97
68th
3
54528 +/- 194
66th
3
55743 +/- 343
63rd
3
57031 +/- 72
62nd
3
58792 +/- 45
60th
3
59244 +/- 55
59th
4
59847 +/- 31
59th
3
60055 +/- 341
58th
3
60383 +/- 85
56th
3
60847 +/- 97
54th
3
60956 +/- 44
54th
3
61442 +/- 484
51st
3
62756 +/- 661
Median
50th
63478
50th
7
63592 +/- 3987
49th
3
63959 +/- 372
48th
4
64167 +/- 112
47th
4
64360 +/- 4536
47th
6
64670 +/- 1103
46th
3
65462 +/- 825
46th
3
65525 +/- 14
44th
3
66966 +/- 92
43rd
3
67388 +/- 194
41st
3
68781 +/- 164
37th
9
69294 +/- 545
35th
9
70420 +/- 1606
34th
4
70911 +/- 127
33rd
3
71502 +/- 272
33rd
3
71788 +/- 91
32nd
3
73006 +/- 157
27th
8
75063 +/- 162
Low-Tier
25th
> 75192
22nd
4
76325 +/- 1670
20th
3
79454 +/- 240
17th
4
81789 +/- 576
16th
5
83498 +/- 14682
12th
9
85526 +/- 1817
11th
3
86210 +/- 644
8th
5
95251 +/- 13722
8th
3
96699 +/- 6657
7th
3
103280 +/- 3062
3rd
3
121367 +/- 4283
3rd
3
124930 +/- 1779
2nd
3
127752 +/- 382
1st
3
151504 +/- 48
OpenBenchmarking.orgDistribution Of Public Results - Model: AlexNet - Acceleration: CPU - Iterations: 100448 Results Range From 31153 To 568436 Milli-Seconds3115341899526456339174137848839562910637511712112786713861314935916010517085118159719234320308921383522458123532724607325681926756527831128905729980331054932129533204134278735353336427937502538577139651740726341800942875543950145024746099347173948248549323150397751472352546953621554696155770756845320406080100

Based on OpenBenchmarking.org data, the selected test / test configuration (Caffe 2020-02-13 - Model: AlexNet - Acceleration: CPU - Iterations: 100) has an average run-time of 5 minutes. By default this test profile is set to run at least 3 times but may increase if the standard deviation exceeds pre-defined defaults or other calculations deem additional runs necessary for greater statistical accuracy of the result.

OpenBenchmarking.orgMinutesTime Required To Complete BenchmarkModel: AlexNet - Acceleration: CPU - Iterations: 100Run-Time510152025Min: 2 / Avg: 4.23 / Max: 21

Based on public OpenBenchmarking.org results, the selected test / test configuration has an average standard deviation of 0.3%.

OpenBenchmarking.orgPercent, Fewer Is BetterAverage Deviation Between RunsModel: AlexNet - Acceleration: CPU - Iterations: 100Deviation246810Min: 0 / Avg: 0.31 / Max: 6

Recent Test Results

OpenBenchmarking.org Results Compare

1 System - 466 Benchmark Results

2 x AMD EPYC 7F72 24-Core - Supermicro H11DSi-NT v2.00 - AMD Starship

Ubuntu 20.10 - 5.10.9-051009-generic - GNOME Shell 3.38.1

3 Systems - 191 Benchmark Results

AMD Ryzen 3 2200G - ASUS PRIME B350M-E - AMD Raven

Ubuntu 20.10 - 5.8.0-38-generic - GNOME Shell 3.38.1

1 System - 2 Benchmark Results

Intel Core i7-1165G7 - Dell 08607K - Intel Device a0ef

Ubuntu 20.04 - 5.6.0-1042-oem - GNOME Shell 3.36.4

4 Systems - 210 Benchmark Results

POWER9 - PowerNV T2P9D01 REV 1.01 - 64GB

Ubuntu 20.10 - 5.9.10-050910-generic - X Server

2 Systems - 183 Benchmark Results

AMD FX-8370 Eight-Core - MSI 970 GAMING - AMD RD9x0

Ubuntu 20.10 - 5.8.0-33-generic - GNOME Shell 3.38.1

3 Systems - 313 Benchmark Results

Intel Core i7-5960X - ASRock X99 Extreme3 - Intel Xeon E7 v3

Ubuntu 20.04 - 5.4.0-58-generic - GNOME Shell 3.36.4

3 Systems - 122 Benchmark Results

Intel Core i7-5775C - MSI Z97-G45 GAMING - Intel Broadwell-U DMI

Ubuntu 18.10 - 5.0.0-999-generic - GNOME Shell 3.30.2

3 Systems - 142 Benchmark Results

Intel Core i7-7740K - Gigabyte X299 AORUS Gaming-CF - Intel Xeon E3-1200 v6

Ubuntu 18.10 - 4.18.0-25-generic - GNOME Shell 3.30.2

Most Popular Test Results

OpenBenchmarking.org Results Compare

3 Systems - 268 Benchmark Results

Intel Core i5-2520M - HP 161C - Intel 2nd Generation Core DRAM

Ubuntu 18.04 - 4.18.0-20-generic - GNOME Shell 3.28.3

3 Systems - 174 Benchmark Results

Intel Core i9-10900K - Gigabyte Z490 AORUS MASTER - Intel Comet Lake PCH

Fedora 32 - 5.6.6-300.fc32.x86_64 - GNOME Shell 3.36.1

3 Systems - 40 Benchmark Results

2 x Intel Xeon Gold 5220R - TYAN S7106 - Intel Sky Lake-E DMI3 Registers

Ubuntu 20.04 - 5.9.0-050900rc6-generic - GNOME Shell 3.36.4

2 Systems - 403 Benchmark Results

Intel Core i9-10900K - Gigabyte Z490 AORUS MASTER - Intel Comet Lake PCH

Ubuntu 20.04 - 5.4.0-48-generic - GNOME Shell 3.36.4

3 Systems - 406 Benchmark Results

AMD Ryzen 9 3900XT 12-Core - MSI MEG X570 GODLIKE - AMD Starship

Ubuntu 20.04 - 5.4.0-48-generic - GNOME Shell 3.36.4

3 Systems - 131 Benchmark Results

Intel Core i5-10600K - ASUS PRIME Z490M-PLUS - Intel Comet Lake PCH

Ubuntu 20.04 - 5.9.0-050900daily20201012-generic - GNOME Shell 3.36.4

3 Systems - 31 Benchmark Results

AMD Ryzen 5 3600XT 6-Core - MSI X470 GAMING M7 AC - AMD Starship

Ubuntu 20.04 - 5.8.0-050800-generic - GNOME Shell 3.36.4

3 Systems - 202 Benchmark Results

Intel Core i7-7700K - MSI Z270-A PRO - Intel Xeon E3-1200 v6

Ubuntu 20.04 - 5.4.0-28-generic - GNOME Shell 3.36.1

3 Systems - 100 Benchmark Results

AMD Ryzen Threadripper 3990X 64-Core - System76 Thelio Major - AMD Starship

Ubuntu 20.04 - 5.4.0-47-generic - GNOME Shell 3.36.4

3 Systems - 32 Benchmark Results

AMD Ryzen 9 3900X 12-Core - ASUS TUF GAMING X570-PLUS - AMD Starship

Ubuntu 20.04 - 5.9.0-050900rc6daily20200922-generic - GNOME Shell 3.36.4

3 Systems - 25 Benchmark Results

AMD Ryzen 7 1800X Eight-Core - MSI X370 XPOWER GAMING TITANIUM - AMD 17h

Debian 10 - 4.19.0-11-amd64 - X Server 1.20.4

Find More Test Results

OpenBenchmarking.org Community User Comments

Update for new CuDNN libraries?

I recently ran this test and found I had to use v5.1 or earlier to compile and couldn't get v6 or v7 to compile as there was an enumeration case not handled. Have I missed something?

v/r,
LabUserBen

Post by labuserben on 19 September 2017, 09.32.

Post A Comment