Apache Hadoop

This is a benchmark of the Apache Hadoop making use of its built-in name-node throughput benchmark (NNThroughputBenchmark).

To run this test with the Phoronix Test Suite, the basic command is: phoronix-test-suite benchmark hadoop.

Project Site

hadoop.apache.org

Source Repository

github.com

Test Created

7 September 2023

Test Maintainer

Michael Larabel 

Test Type

System

Average Install Time

1 Second

Average Run Time

6 Minutes, 19 Seconds

Test Dependencies

Java

Supported Platforms

Rename20.3%Delete21.9%Open20.9%Create23.5%File Status13.4%Operation Option PopularityOpenBenchmarking.org
2033.2%5008.6%10025.5%5032.7%Threads Option PopularityOpenBenchmarking.org
100000042.4%10000057.6%Files Option PopularityOpenBenchmarking.org

Revision History

pts/hadoop-1.0.0   [View Source]   Thu, 07 Sep 2023 14:37:09 GMT
Initial commit of Hadoop benchmark.


Performance Metrics

Analyze Test Configuration:

Apache Hadoop 3.3.6

Operation: Create - Threads: 100 - Files: 100000

OpenBenchmarking.org metrics for this test profile configuration based on 66 public results since 7 September 2023 with the latest data as of 19 September 2023.

Below is an overview of the generalized performance for components where there is sufficient statistically significant data based upon user-uploaded results. It is important to keep in mind particularly in the Linux/open-source space there can be vastly different OS configurations, with this overview intended to offer just general guidance as to the performance expectations.

Component
Percentile Rank
# Compatible Public Results
Ops per sec (Average)
Intel Core i9-13900K
98th
5
80509 +/- 1551
Intel Xeon E-2336
89th
4
50957 +/- 315
Intel Xeon Platinum 8490H
76th
10
48095 +/- 814
Mid-Tier
75th
< 47015
AMD Ryzen 7 PRO 5850U
68th
4
46295 +/- 755
2 x Intel Xeon Platinum 8490H
62nd
4
34470 +/- 487
AMD EPYC 7763 64-Core
59th
3
27594
Median
50th
23116
AMD EPYC 9684X 96-Core
44th
6
21003 +/- 834
AMD EPYC 9654 96-Core
39th
6
20242 +/- 630
Low-Tier
25th
< 15813
2 x AMD EPYC 9684X 96-Core
21st
6
15727 +/- 67
AMD EPYC 9754 128-Core
15th
7
15471 +/- 674
Intel Core i9-10980XE
15th
3
15401
AMD Ryzen Threadripper 3970X 32-Core
11th
3
15373
OpenBenchmarking.orgDistribution Of Public Results - Operation: Create - Threads: 100 - Files: 10000066 Results Range From 14759 To 82508 Ops per sec147591611417469188242017921534228892424425599269542830929664310193237433729350843643937794391494050441859432144456945924472794863449989513445269954054554095676458119594746082962184635396489466249676046895970314716697302474379757347708978444797998115482509510152025

Based on OpenBenchmarking.org data, the selected test / test configuration (Apache Hadoop 3.3.6 - Operation: Create - Threads: 100 - Files: 100000) has an average run-time of 8 minutes. By default this test profile is set to run at least 3 times but may increase if the standard deviation exceeds pre-defined defaults or other calculations deem additional runs necessary for greater statistical accuracy of the result.

OpenBenchmarking.orgMinutesTime Required To Complete BenchmarkOperation: Create - Threads: 100 - Files: 100000Run-Time3691215Min: 2 / Avg: 7.54 / Max: 11

Based on public OpenBenchmarking.org results, the selected test / test configuration has an average standard deviation of 2%.

OpenBenchmarking.orgPercent, Fewer Is BetterAverage Deviation Between RunsOperation: Create - Threads: 100 - Files: 100000Deviation48121620Min: 0 / Avg: 2 / Max: 14

Tested CPU Architectures

This benchmark has been successfully tested on the below mentioned architectures. The CPU architectures listed is where successful OpenBenchmarking.org result uploads occurred, namely for helping to determine if a given test is compatible with various alternative CPU architectures.

CPU Architecture
Kernel Identifier
Verified On
Intel / AMD x86 64-bit
x86_64
(Many Processors)

Recent Test Results

OpenBenchmarking.org Results Compare

3 Systems - 76 Benchmark Results

AMD Ryzen 7 7840U - PHX Ray_PEU - AMD Device 14e8

Ubuntu 23.10 - 6.5.0-with-patch2 - GNOME Shell

4 Systems - 183 Benchmark Results

1 System - 332 Benchmark Results

AMD Ryzen Threadripper 3970X 32-Core - ASUS ROG ZENITH II EXTREME - AMD Starship

Ubuntu 22.04 - 5.19.0-051900rc7-generic - GNOME Shell 42.2

2 Systems - 141 Benchmark Results

2 x AMD EPYC 9684X 96-Core - AMD Titanite_4G - AMD Device 14a4

Ubuntu 23.10 - 6.5.1-060501-generic - GNOME Shell

1 System - 95 Benchmark Results

AMD EPYC 7763 64-Core - AMD DAYTONA_X - AMD Starship

Ubuntu 22.04 - 5.15.0-47-generic - GNOME Shell 42.4

1 System - 188 Benchmark Results

AMD EPYC 9754 128-Core - AMD Titanite_4G - AMD Device 14a4

Ubuntu 23.10 - 6.5.1-060501-generic - GNOME Shell

3 Systems - 73 Benchmark Results

Intel Core i7-1165G7 - Dell 0GG9PT - Intel Tiger Lake-LP

Ubuntu 23.04 - 6.5.0-rc5-phx-tues - GNOME Shell 44.3

2 Systems - 191 Benchmark Results

AMD Ryzen 7 PRO 5850U - HP 8A78 - AMD Renoir

Pop 22.04 - 6.2.6-76060206-generic - GNOME Shell 42.5

2 Systems - 46 Benchmark Results

Intel Xeon E-2336 - ASRockRack E3C252D4U - Intel Tiger Lake-H

Ubuntu 22.04 - 6.5.1-060501-generic - GNOME Shell 42.9

5 Systems - 56 Benchmark Results

Intel Core i9-13900K - ASUS PRIME Z790-P WIFI - Intel Device 7a27

Ubuntu 22.04 - 6.4.6-060406-generic - GNOME Shell 42.9

2 Systems - 37 Benchmark Results

AMD Ryzen 9 7950X 16-Core - ASUS ROG STRIX X670E-E GAMING WIFI - AMD Device 14d8

Ubuntu 23.04 - 6.3.13-060313-generic - GNOME Shell 44.3

2 Systems - 37 Benchmark Results

AMD Ryzen 9 7950X 16-Core - ASUS ROG STRIX X670E-E GAMING WIFI - AMD Device 14d8

Ubuntu 23.04 - 6.3.13-060313-generic - GNOME Shell 44.3

5 Systems - 46 Benchmark Results

AMD Ryzen 9 5900HX - ASUS G513QY v1.0 - AMD Renoir

Ubuntu 22.10 - 5.19.0-46-generic - GNOME Shell 43.0

2 Systems - 46 Benchmark Results

AMD Ryzen 9 5900HX - ASUS G513QY v1.0 - AMD Renoir

Ubuntu 22.10 - 5.19.0-46-generic - GNOME Shell 43.0

1 System - 26 Benchmark Results

Intel Core i9-13900K - ASUS PRIME Z790-P WIFI - Intel Device 7a27

Ubuntu 22.04 - 6.4.6-060406-generic - GNOME Shell 42.9

Most Popular Test Results

OpenBenchmarking.org Results Compare

2 Systems - 46 Benchmark Results

Intel Xeon E-2336 - ASRockRack E3C252D4U - Intel Tiger Lake-H

Ubuntu 22.04 - 6.5.1-060501-generic - GNOME Shell 42.9

4 Systems - 137 Benchmark Results

AMD EPYC 9654 96-Core - AMD Titanite_4G - AMD Device 14a4

Ubuntu 23.10 - 6.5.1-060501-generic - GNOME Shell

2 Systems - 191 Benchmark Results

AMD Ryzen 7 PRO 5850U - HP 8A78 - AMD Renoir

Pop 22.04 - 6.2.6-76060206-generic - GNOME Shell 42.5

2 Systems - 141 Benchmark Results

2 x AMD EPYC 9684X 96-Core - AMD Titanite_4G - AMD Device 14a4

Ubuntu 23.10 - 6.5.1-060501-generic - GNOME Shell

1 System - 188 Benchmark Results

AMD EPYC 9754 128-Core - AMD Titanite_4G - AMD Device 14a4

Ubuntu 23.10 - 6.5.1-060501-generic - GNOME Shell

2 Systems - 46 Benchmark Results

Intel Xeon E-2336 - ASRockRack E3C252D4U - Intel Tiger Lake-H

Ubuntu 22.04 - 6.5.1-060501-generic - GNOME Shell 42.9

4 Systems - 183 Benchmark Results

2 Systems - 46 Benchmark Results

AMD Ryzen 9 5900HX - ASUS G513QY v1.0 - AMD Renoir

Ubuntu 22.10 - 5.19.0-46-generic - GNOME Shell 43.0

5 Systems - 46 Benchmark Results

AMD Ryzen 9 5900HX - ASUS G513QY v1.0 - AMD Renoir

Ubuntu 22.10 - 5.19.0-46-generic - GNOME Shell 43.0

6 Systems - 69 Benchmark Results

AMD EPYC 9684X 96-Core - AMD Titanite_4G - AMD Device 14a4

Ubuntu 23.10 - 6.6.0-060600rc1-generic - GNOME Shell

1 System - 332 Benchmark Results

AMD Ryzen Threadripper 3970X 32-Core - ASUS ROG ZENITH II EXTREME - AMD Starship

Ubuntu 22.04 - 5.19.0-051900rc7-generic - GNOME Shell 42.2

1 System - 95 Benchmark Results

AMD EPYC 7763 64-Core - AMD DAYTONA_X - AMD Starship

Ubuntu 22.04 - 5.15.0-47-generic - GNOME Shell 42.4

4 Systems - 183 Benchmark Results

Intel Xeon Platinum 8490H - Quanta Cloud S6Q-MB-MPS - Intel Device 1bce

Ubuntu 23.10 - 6.5.1-060501-generic - GNOME Shell

1 System - 304 Benchmark Results

AMD Ryzen Threadripper 3970X 32-Core - ASUS ROG ZENITH II EXTREME - AMD Starship

Ubuntu 22.04 - 5.19.0-051900rc7-generic - GNOME Shell 42.2

5 Systems - 56 Benchmark Results

Intel Core i9-13900K - ASUS PRIME Z790-P WIFI - Intel Device 7a27

Ubuntu 22.04 - 6.4.6-060406-generic - GNOME Shell 42.9
Find More Test Results