tr 2950X

AMD Ryzen Threadripper 2950X 16-Core testing with a MSI MEG X399 CREATION (MS-7B92) v1.0 (1.30 BIOS) and AMD FirePro V4900 1GB on Ubuntu 22.04 via the Phoronix Test Suite.

Compare your own system(s) to this result file with the Phoronix Test Suite by running the command: phoronix-test-suite benchmark 2208075-NE-TR2950X6958
Jump To Table - Results

View

Do Not Show Noisy Results
Do Not Show Results With Incomplete Data
Do Not Show Results With Little Change/Spread
List Notable Results
Show Result Confidence Charts

Statistics

Show Overall Harmonic Mean(s)
Show Overall Geometric Mean
Show Wins / Losses Counts (Pie Chart)
Normalize Results
Remove Outliers Before Calculating Averages

Graph Settings

Force Line Graphs Where Applicable
Convert To Scalar Where Applicable
Prefer Vertical Bar Graphs

Multi-Way Comparison

Condense Multi-Option Tests Into Single Result Graphs

Table

Show Detailed System Result Table

Run Management

Highlight
Result
Hide
Result
Result
Identifier
View Logs
Performance Per
Dollar
Date
Run
  Test
  Duration
A
August 06 2022
  6 Hours, 44 Minutes
BN
August 06 2022
  1 Day, 17 Hours, 45 Minutes
C
August 07 2022
  1 Day, 28 Minutes
Invert Hiding All Results Option
  1 Day, 19 Minutes

Only show results where is faster than
Only show results matching title/arguments (delimit multiple options with a comma):
Do not show results matching title/arguments (delimit multiple options with a comma):


tr 2950XOpenBenchmarking.orgPhoronix Test SuiteAMD Ryzen Threadripper 2950X 16-Core @ 3.50GHz (16 Cores / 32 Threads)MSI MEG X399 CREATION (MS-7B92) v1.0 (1.30 BIOS)AMD 17h32GBSamsung SSD 970 EVO 250GBAMD FirePro V4900 1GBRealtek ALC1220DELL S2409W2 x Intel I211 + Intel-AC 9260Ubuntu 22.045.19.0-051900rc7-generic (x86_64)GNOME Shell 42.2X Server + Wayland3.3 Mesa 22.0.1 (LLVM 13.0.1)1.2.204GCC 11.2.0ext41920x1080ProcessorMotherboardChipsetMemoryDiskGraphicsAudioMonitorNetworkOSKernelDesktopDisplay ServerOpenGLVulkanCompilerFile-SystemScreen ResolutionTr 2950X BenchmarksSystem Logs- Transparent Huge Pages: madvise- --build=x86_64-linux-gnu --disable-vtable-verify --disable-werror --enable-bootstrap --enable-cet --enable-checking=release --enable-clocale=gnu --enable-default-pie --enable-gnu-unique-object --enable-languages=c,ada,c++,go,brig,d,fortran,objc,obj-c++,m2 --enable-libphobos-checking=release --enable-libstdcxx-debug --enable-libstdcxx-time=yes --enable-link-serialization=2 --enable-multiarch --enable-multilib --enable-nls --enable-objc-gc=auto --enable-offload-targets=nvptx-none=/build/gcc-11-gBFGDP/gcc-11-11.2.0/debian/tmp-nvptx/usr,amdgcn-amdhsa=/build/gcc-11-gBFGDP/gcc-11-11.2.0/debian/tmp-gcn/usr --enable-plugin --enable-shared --enable-threads=posix --host=x86_64-linux-gnu --program-prefix=x86_64-linux-gnu- --target=x86_64-linux-gnu --with-abi=m64 --with-arch-32=i686 --with-build-config=bootstrap-lto-lean --with-default-libstdcxx-abi=new --with-gcc-major-version-only --with-multilib-list=m32,m64,mx32 --with-target-system-zlib=auto --with-tune=generic --without-cuda-driver -v - Scaling Governor: acpi-cpufreq schedutil (Boost: Enabled) - CPU Microcode: 0x800820d- OpenJDK Runtime Environment (build 11.0.15+10-Ubuntu-0ubuntu0.22.04.1)- Python 3.10.4- itlb_multihit: Not affected + l1tf: Not affected + mds: Not affected + meltdown: Not affected + mmio_stale_data: Not affected + retbleed: Mitigation of untrained return thunk; SMT vulnerable + spec_store_bypass: Mitigation of SSB disabled via prctl + spectre_v1: Mitigation of usercopy/swapgs barriers and __user pointer sanitization + spectre_v2: Mitigation of Retpolines IBPB: conditional STIBP: disabled RSB filling + srbds: Not affected + tsx_async_abort: Not affected

ABNCResult OverviewPhoronix Test Suite100%104%107%111%114%Apache SparkApache SparkApache SparkApache SparkApache SparkApache SparkApache SparkApache SparkApache SparkApache SparkApache SparkApache SparkApache SparkApache SparkApache SparkApache SparkApache SparkApache SparkApache SparkApache SparkApache SparkApache SparkSVT-AV1Apache SparkApache SparkApache SparkApache SparkApache SparkApache SparkApache SparkApache SparkApache SparkApache SparkApache SparkApache SparkApache SparkApache Spark1000000 - 2000 - R.T.T1000000 - 100 - I.J.T.T1000000 - 100 - B.I.J.T.T20000000 - 1000 - S.5.B.T40000000 - 1000 - S.5.B.T10000000 - 2000 - B.I.J.T.T10000000 - 500 - I.J.T.T1000000 - 1000 - S.5.B.T1000000 - 2000 - B.I.J.T.T20000000 - 1000 - B.I.J.T.T1000000 - 1000 - B.I.J.T.T40000000 - 2000 - Group By Test Time10000000 - 1000 - I.J.T.T1000000 - 500 - B.I.J.T.T20000000 - 1000 - R.T.T10000000 - 1000 - B.I.J.T.T1000000 - 1000 - R.T.T10000000 - 2000 - Group By Test Time1000000 - 500 - I.J.T.T1000000 - 100 - R.T.T1000000 - 500 - S.5.B.T40000000 - 1000 - B.I.J.T.TPreset 10 - Bosphorus 1080p40000000 - 100 - B.I.J.T.T40000000 - 100 - R.T.T40000000 - 100 - S.5.B.T40000000 - 2000 - I.J.T.T10000000 - 100 - I.J.T.T1000000 - 500 - R.T.T40000000 - 500 - B.I.J.T.T1000000 - 1000 - Group By Test Time10000000 - 1000 - Group By Test Time10000000 - 2000 - I.J.T.T40000000 - 2000 - S.5.B.T20000000 - 500 - R.T.T20000000 - 2000 - B.I.J.T.T40000000 - 500 - I.J.T.T

tr 2950Xspark: 1000000 - 2000 - Repartition Test Timespark: 1000000 - 100 - Inner Join Test Timespark: 20000000 - 1000 - SHA-512 Benchmark Timespark: 40000000 - 1000 - SHA-512 Benchmark Timespark: 10000000 - 500 - Inner Join Test Timespark: 1000000 - 1000 - SHA-512 Benchmark Timespark: 1000000 - 2000 - Broadcast Inner Join Test Timespark: 20000000 - 1000 - Broadcast Inner Join Test Timespark: 1000000 - 1000 - Broadcast Inner Join Test Timespark: 40000000 - 2000 - Group By Test Timespark: 10000000 - 1000 - Inner Join Test Timespark: 20000000 - 1000 - Repartition Test Timespark: 10000000 - 1000 - Broadcast Inner Join Test Timespark: 1000000 - 1000 - Repartition Test Timespark: 10000000 - 2000 - Group By Test Timespark: 1000000 - 100 - Repartition Test Timespark: 1000000 - 500 - SHA-512 Benchmark Timespark: 40000000 - 1000 - Broadcast Inner Join Test Timesvt-av1: Preset 10 - Bosphorus 1080pspark: 40000000 - 100 - Broadcast Inner Join Test Timespark: 40000000 - 100 - Repartition Test Timespark: 40000000 - 100 - SHA-512 Benchmark Timespark: 40000000 - 2000 - Inner Join Test Timespark: 10000000 - 100 - Inner Join Test Timespark: 1000000 - 500 - Repartition Test Timespark: 40000000 - 500 - Broadcast Inner Join Test Timespark: 1000000 - 1000 - Group By Test Timespark: 10000000 - 1000 - Group By Test Timespark: 10000000 - 2000 - Inner Join Test Timespark: 40000000 - 2000 - SHA-512 Benchmark Timespark: 20000000 - 500 - Repartition Test Timespark: 20000000 - 2000 - Broadcast Inner Join Test Timespark: 40000000 - 500 - Inner Join Test Timespark: 40000000 - 2000 - Repartition Test Timespark: 40000000 - 1000 - Inner Join Test Timespark: 10000000 - 500 - Broadcast Inner Join Test Timespark: 20000000 - 500 - Inner Join Test Timespark: 1000000 - 100 - SHA-512 Benchmark Timespark: 1000000 - 2000 - Inner Join Test Timespark: 40000000 - 2000 - Broadcast Inner Join Test Timespark: 10000000 - 1000 - Repartition Test Timespark: 10000000 - 100 - Group By Test Timespark: 20000000 - 2000 - Group By Test Timespark: 40000000 - 500 - Repartition Test Timespark: 20000000 - 100 - SHA-512 Benchmark Timespark: 10000000 - 1000 - SHA-512 Benchmark Timespark: 20000000 - 2000 - Repartition Test Timespark: 1000000 - 500 - Group By Test Timespark: 1000000 - 2000 - Group By Test Timespark: 1000000 - 100 - Group By Test Timespark: 10000000 - 2000 - Repartition Test Timespark: 20000000 - 100 - Broadcast Inner Join Test Timespark: 40000000 - 100 - Inner Join Test Timespark: 10000000 - 500 - Repartition Test Timespark: 40000000 - 500 - Calculate Pi Benchmark Using Dataframespark: 1000000 - 500 - Calculate Pi Benchmarkspark: 40000000 - 1000 - Calculate Pi Benchmark Using Dataframespark: 40000000 - 100 - Calculate Pi Benchmark Using Dataframespark: 20000000 - 2000 - SHA-512 Benchmark Timespark: 20000000 - 500 - Group By Test Timespark: 10000000 - 100 - SHA-512 Benchmark Timespark: 10000000 - 100 - Repartition Test Timesvt-av1: Preset 4 - Bosphorus 1080pspark: 40000000 - 500 - Group By Test Timesvt-av1: Preset 12 - Bosphorus 1080pspark: 20000000 - 500 - Broadcast Inner Join Test Timespark: 1000000 - 2000 - SHA-512 Benchmark Timespark: 10000000 - 1000 - Calculate Pi Benchmark Using Dataframespark: 20000000 - 1000 - Group By Test Timespark: 1000000 - 100 - Calculate Pi Benchmarkspark: 10000000 - 100 - Calculate Pi Benchmarkspark: 40000000 - 1000 - Repartition Test Timespark: 1000000 - 100 - Calculate Pi Benchmark Using Dataframespark: 10000000 - 500 - Group By Test Timespark: 40000000 - 100 - Calculate Pi Benchmarkspark: 10000000 - 500 - SHA-512 Benchmark Timespark: 40000000 - 2000 - Calculate Pi Benchmark Using Dataframespark: 1000000 - 2000 - Calculate Pi Benchmarkspark: 20000000 - 1000 - Inner Join Test Timespark: 20000000 - 100 - Repartition Test Timespark: 10000000 - 2000 - SHA-512 Benchmark Timespark: 20000000 - 2000 - Inner Join Test Timespark: 10000000 - 500 - Calculate Pi Benchmarkspark: 10000000 - 2000 - Calculate Pi Benchmark Using Dataframespark: 40000000 - 100 - Group By Test Timespark: 20000000 - 1000 - Calculate Pi Benchmark Using Dataframespark: 40000000 - 500 - Calculate Pi Benchmarkspark: 40000000 - 1000 - Group By Test Timesvt-av1: Preset 8 - Bosphorus 1080pspark: 40000000 - 500 - SHA-512 Benchmark Timespark: 10000000 - 500 - Calculate Pi Benchmark Using Dataframesvt-av1: Preset 10 - Bosphorus 4Kspark: 20000000 - 500 - Calculate Pi Benchmark Using Dataframespark: 1000000 - 1000 - Inner Join Test Timespark: 20000000 - 500 - SHA-512 Benchmark Timespark: 20000000 - 100 - Calculate Pi Benchmark Using Dataframespark: 1000000 - 2000 - Calculate Pi Benchmark Using Dataframespark: 10000000 - 100 - Calculate Pi Benchmark Using Dataframespark: 20000000 - 1000 - Calculate Pi Benchmarkspark: 1000000 - 1000 - Calculate Pi Benchmark Using Dataframespark: 10000000 - 100 - Broadcast Inner Join Test Timespark: 10000000 - 1000 - Calculate Pi Benchmarkspark: 20000000 - 100 - Group By Test Timespark: 1000000 - 500 - Calculate Pi Benchmark Using Dataframespark: 20000000 - 100 - Calculate Pi Benchmarkspark: 40000000 - 2000 - Calculate Pi Benchmarkspark: 40000000 - 1000 - Calculate Pi Benchmarkspark: 20000000 - 2000 - Calculate Pi Benchmarksvt-av1: Preset 12 - Bosphorus 4Ksvt-av1: Preset 8 - Bosphorus 4Ksvt-av1: Preset 4 - Bosphorus 4Kspark: 1000000 - 1000 - Calculate Pi Benchmarkspark: 20000000 - 500 - Calculate Pi Benchmarkspark: 10000000 - 2000 - Calculate Pi Benchmarkspark: 20000000 - 2000 - Calculate Pi Benchmark Using Dataframespark: 10000000 - 2000 - Broadcast Inner Join Test Timespark: 20000000 - 100 - Inner Join Test Timespark: 1000000 - 500 - Broadcast Inner Join Test Timespark: 1000000 - 500 - Inner Join Test Timespark: 1000000 - 100 - Broadcast Inner Join Test TimeABNC2.972.1023.23500652444.77757363410.4448530754.342.3219.2376279951.9225.81836339910.22690810315.89295816510.4121539082.879.802.7002640514.3736.818880624162.27940.08974714434.18555970546.52311235537.99194315510.7646975332.6537.860127725.379.5610.99707235743.85976886816.64063657719.05687729938.91663164430.64805419837.54260010210.01652585318.7858192354.5300637983.0936.6972444829.359.43975251613.2260123631.75970946224.42888620814.00452356516.4316364035.5046102136.175.8769422449.3020.75454159540.8352763839.096.72108.2259551586.866.8423.24735090911.89008500513.9311416639.473.59331.80822044221.10418.4286746864.9047595636.7013.180563735108.419781632108.68836702631.4483235166.7832715588.621025209110.52365756613.1988245676.86108.59728910819.68243866817.15439022814.13935093720.393716972109.2303116876.7632.4230433136.875336993109.23183516629.12648346475.96345.8154074386.75925895461.3886.8127446292.4322.7541114656.806.846.81110.7301282356.79919824710.662243237109.52235686513.3296263926.77685602109.108538715110.085285547109.896421862109.4397669189.75231.9951.384109.807174541109.612467287109.911051716.779.9620.6837346221.700073412.1709950551.9725177983.382.3724.8943.909.734.652.4119.612.0527.2210.7816.4010.662.949.892.814.5436.73158.92941.2732.9946.4038.8010.472.7238.995.489.6710.8343.4216.1619.2737.9431.5436.529.7519.294.623.1735.829.599.5912.9032.5624.6314.3416.695.576.276.019.4520.8039.988.936.86109.9074079896.876.7923.6911.9914.199.313.53031.26223.44218.374.996.8013.20110.166207147109.35703340731.516.898.49109.74699733113.096.84110.03983881719.9517.3814.0420.55109.3298560126.7832.116.80109.94045365029.3476.00546.276.7860.8466.802.4522.586.826.796.80110.2586475716.8210.69109.96819544813.406.81109.576906446110.551039063110.394615796109.89698440589.46032.1291.379109.680205103109.971942948110.2509737566.7910.8320.351.772.251.883.392.1325.4541.00312728610.194.642.4820.552.0327.26790608910.8016.6710.872.8210.212.734.4335.45156.49239.8133.2948.0539.3110.412.7439.135.559.8710.6644.7216.3319.6239.0631.2736.589.8819.064.503.1236.749.469.6813.0532.5525.0214.3016.305.446.315.979.5121.2240.129.126.84110.3578639466.746.9223.5511.7714.019.303.56631.45219.61618.694.926.8112.99109.870024098110.41137680231.026.868.50111.4313.296.76110.1219.8517.3714.2220.30110.5390732206.8432.496.88110.3929.4375.22546.146.8260.9426.862.4322.646.776.816.85109.9596909746.8410.72110.10528059013.356.81109.616776140110.588119919110.273490935109.9160597189.36632.0211.379110.057783068109.977979218109.966.7710.5420.601.792.261.75OpenBenchmarking.org

Apache Spark

This is a benchmark of Apache Spark with its PySpark interface. Apache Spark is an open-source unified analytics engine for large-scale data processing and dealing with big data. This test profile benchmars the Apache Spark in a single-system configuration using spark-submit. The test makes use of DIYBigData's pyspark-benchmark (https://github.com/DIYBigData/pyspark-benchmark/) for generating of test data and various Apache Spark operations. Learn more via the OpenBenchmarking.org test page.

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 1000000 - Partitions: 2000 - Repartition Test TimeCBNA0.76281.52562.28843.05123.814SE +/- 0.02, N = 3SE +/- 0.04, N = 63.393.382.97

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 1000000 - Partitions: 100 - Inner Join Test TimeCBNA0.53331.06661.59992.13322.6665SE +/- 0.05, N = 4SE +/- 0.07, N = 32.132.372.10

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 20000000 - Partitions: 1000 - SHA-512 Benchmark TimeCBNA612182430SE +/- 0.11, N = 325.4524.8923.24

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 40000000 - Partitions: 1000 - SHA-512 Benchmark TimeCBNA1020304050SE +/- 0.53, N = 941.0043.9044.78

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 10000000 - Partitions: 500 - Inner Join Test TimeCBNA3691215SE +/- 0.144346648, N = 6SE +/- 0.118870708, N = 310.1900000009.73000000010.444853075

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 1000000 - Partitions: 1000 - SHA-512 Benchmark TimeCBNA1.04632.09263.13894.18525.2315SE +/- 0.06, N = 3SE +/- 0.05, N = 34.644.654.34

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 1000000 - Partitions: 2000 - Broadcast Inner Join Test TimeCBNA0.5581.1161.6742.2322.79SE +/- 0.07, N = 3SE +/- 0.05, N = 62.482.412.32

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 20000000 - Partitions: 1000 - Broadcast Inner Join Test TimeCBNA510152025SE +/- 0.21, N = 320.5519.6119.24

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 1000000 - Partitions: 1000 - Broadcast Inner Join Test TimeCBNA0.46130.92261.38391.84522.3065SE +/- 0.06, N = 3SE +/- 0.06, N = 32.032.051.92

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 40000000 - Partitions: 2000 - Group By Test TimeCBNA612182430SE +/- 0.38, N = 927.2727.2225.82

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 10000000 - Partitions: 1000 - Inner Join Test TimeCBNA3691215SE +/- 0.13, N = 12SE +/- 0.13, N = 1210.8010.7810.23

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 20000000 - Partitions: 1000 - Repartition Test TimeCBNA48121620SE +/- 0.18, N = 316.6716.4015.89

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 10000000 - Partitions: 1000 - Broadcast Inner Join Test TimeCBNA3691215SE +/- 0.11, N = 12SE +/- 0.11, N = 1210.8710.6610.41

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 1000000 - Partitions: 1000 - Repartition Test TimeCBNA0.66151.3231.98452.6463.3075SE +/- 0.04, N = 3SE +/- 0.03, N = 32.822.942.87

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 10000000 - Partitions: 2000 - Group By Test TimeCBNA3691215SE +/- 0.19, N = 3SE +/- 0.02, N = 310.219.899.80

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 1000000 - Partitions: 100 - Repartition Test TimeCBNA0.63231.26461.89692.52923.1615SE +/- 0.052181616, N = 4SE +/- 0.018430805, N = 32.7300000002.8100000002.700264051

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 1000000 - Partitions: 500 - SHA-512 Benchmark TimeCBNA1.02152.0433.06454.0865.1075SE +/- 0.05, N = 3SE +/- 0.04, N = 84.434.544.37

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 40000000 - Partitions: 1000 - Broadcast Inner Join Test TimeCBNA816243240SE +/- 0.21, N = 935.4536.7336.82

SVT-AV1

OpenBenchmarking.orgFrames Per Second, More Is BetterSVT-AV1 1.2Encoder Mode: Preset 10 - Input: Bosphorus 1080pCBNA4080120160200SE +/- 1.66, N = 4SE +/- 1.56, N = 3156.49158.93162.281. (CXX) g++ options: -march=native -mno-avx -mavx2 -mavx512f -mavx512bw -mavx512dq

Apache Spark

This is a benchmark of Apache Spark with its PySpark interface. Apache Spark is an open-source unified analytics engine for large-scale data processing and dealing with big data. This test profile benchmars the Apache Spark in a single-system configuration using spark-submit. The test makes use of DIYBigData's pyspark-benchmark (https://github.com/DIYBigData/pyspark-benchmark/) for generating of test data and various Apache Spark operations. Learn more via the OpenBenchmarking.org test page.

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 40000000 - Partitions: 100 - Broadcast Inner Join Test TimeCBNA918273645SE +/- 0.52, N = 3SE +/- 0.37, N = 939.8141.2740.09

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 40000000 - Partitions: 100 - Repartition Test TimeCBNA816243240SE +/- 0.48, N = 3SE +/- 0.04, N = 933.2932.9934.19

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 40000000 - Partitions: 100 - SHA-512 Benchmark TimeCBNA1122334455SE +/- 0.33, N = 3SE +/- 0.41, N = 948.0546.4046.52

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 40000000 - Partitions: 2000 - Inner Join Test TimeCBNA918273645SE +/- 0.21, N = 939.3138.8037.99

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 10000000 - Partitions: 100 - Inner Join Test TimeCBNA3691215SE +/- 0.18, N = 3SE +/- 0.26, N = 310.4110.4710.76

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 1000000 - Partitions: 500 - Repartition Test TimeCBNA0.61651.2331.84952.4663.0825SE +/- 0.05, N = 3SE +/- 0.04, N = 82.742.722.65

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 40000000 - Partitions: 500 - Broadcast Inner Join Test TimeCBNA918273645SE +/- 0.24, N = 9SE +/- 0.14, N = 539.1338.9937.86

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 1000000 - Partitions: 1000 - Group By Test TimeCBNA1.24882.49763.74644.99526.244SE +/- 0.02, N = 3SE +/- 0.04, N = 35.555.485.37

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 10000000 - Partitions: 1000 - Group By Test TimeCBNA3691215SE +/- 0.06, N = 12SE +/- 0.05, N = 129.879.679.56

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 10000000 - Partitions: 2000 - Inner Join Test TimeCBNA3691215SE +/- 0.14, N = 3SE +/- 0.15, N = 310.6610.8311.00

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 40000000 - Partitions: 2000 - SHA-512 Benchmark TimeCBNA1020304050SE +/- 0.49, N = 944.7243.4243.86

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 20000000 - Partitions: 500 - Repartition Test TimeCBNA48121620SE +/- 0.17, N = 3SE +/- 0.26, N = 316.3316.1616.64

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 20000000 - Partitions: 2000 - Broadcast Inner Join Test TimeCBNA510152025SE +/- 0.17, N = 1219.6219.2719.06

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 40000000 - Partitions: 500 - Inner Join Test TimeCBNA918273645SE +/- 0.39, N = 9SE +/- 0.49, N = 539.0637.9438.92

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 40000000 - Partitions: 2000 - Repartition Test TimeCBNA714212835SE +/- 0.09, N = 931.2731.5430.65

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 40000000 - Partitions: 1000 - Inner Join Test TimeCBNA918273645SE +/- 0.26, N = 936.5836.5237.54

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 10000000 - Partitions: 500 - Broadcast Inner Join Test TimeCBNA3691215SE +/- 0.094306283, N = 6SE +/- 0.085017131, N = 39.8800000009.75000000010.016525853

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 20000000 - Partitions: 500 - Inner Join Test TimeCBNA510152025SE +/- 0.15, N = 3SE +/- 0.35, N = 319.0619.2918.79

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 1000000 - Partitions: 100 - SHA-512 Benchmark TimeCBNA1.03952.0793.11854.1585.1975SE +/- 0.055483981, N = 4SE +/- 0.034411464, N = 34.5000000004.6200000004.530063798

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 1000000 - Partitions: 2000 - Inner Join Test TimeCBNA0.71331.42662.13992.85323.5665SE +/- 0.09, N = 3SE +/- 0.04, N = 63.123.173.09

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 40000000 - Partitions: 2000 - Broadcast Inner Join Test TimeCBNA816243240SE +/- 0.15, N = 936.7435.8236.70

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 10000000 - Partitions: 1000 - Repartition Test TimeCBNA3691215SE +/- 0.06, N = 12SE +/- 0.07, N = 129.469.599.35

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 10000000 - Partitions: 100 - Group By Test TimeCBNA3691215SE +/- 0.055018199, N = 3SE +/- 0.023359039, N = 39.6800000009.5900000009.439752516

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 20000000 - Partitions: 2000 - Group By Test TimeCBNA3691215SE +/- 0.06, N = 1213.0512.9013.23

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 40000000 - Partitions: 500 - Repartition Test TimeCBNA816243240SE +/- 0.04, N = 9SE +/- 0.16, N = 532.5532.5631.76

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 20000000 - Partitions: 100 - SHA-512 Benchmark TimeCBNA612182430SE +/- 0.27, N = 3SE +/- 0.03, N = 325.0224.6324.43

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 10000000 - Partitions: 1000 - SHA-512 Benchmark TimeCBNA48121620SE +/- 0.11, N = 12SE +/- 0.14, N = 1214.3014.3414.00

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 20000000 - Partitions: 2000 - Repartition Test TimeCBNA48121620SE +/- 0.10, N = 1216.3016.6916.43

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 1000000 - Partitions: 500 - Group By Test TimeCBNA1.25332.50663.75995.01326.2665SE +/- 0.043659720, N = 3SE +/- 0.029315415, N = 85.4400000005.5700000005.504610213

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 1000000 - Partitions: 2000 - Group By Test TimeCBNA246810SE +/- 0.10, N = 3SE +/- 0.04, N = 66.316.276.17

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 1000000 - Partitions: 100 - Group By Test TimeCBNA246810SE +/- 0.080220571, N = 4SE +/- 0.060621777, N = 35.9700000006.0100000005.876942244

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 10000000 - Partitions: 2000 - Repartition Test TimeCBNA3691215SE +/- 0.06, N = 3SE +/- 0.06, N = 39.519.459.30

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 20000000 - Partitions: 100 - Broadcast Inner Join Test TimeCBNA510152025SE +/- 0.25, N = 3SE +/- 0.24, N = 321.2220.8020.75

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 40000000 - Partitions: 100 - Inner Join Test TimeCBNA918273645SE +/- 0.17, N = 3SE +/- 0.28, N = 940.1239.9840.84

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 10000000 - Partitions: 500 - Repartition Test TimeCBNA3691215SE +/- 0.10, N = 6SE +/- 0.19, N = 39.128.939.09

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 40000000 - Partitions: 500 - Calculate Pi Benchmark Using DataframeCBNA246810SE +/- 0.04, N = 9SE +/- 0.02, N = 56.846.866.72

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 1000000 - Partitions: 500 - Calculate Pi BenchmarkCBNA20406080100SE +/- 0.32, N = 3SE +/- 0.12, N = 8110.36109.91108.23

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 40000000 - Partitions: 1000 - Calculate Pi Benchmark Using DataframeCBNA246810SE +/- 0.03, N = 96.746.876.86

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 40000000 - Partitions: 100 - Calculate Pi Benchmark Using DataframeCBNA246810SE +/- 0.06, N = 3SE +/- 0.01, N = 96.926.796.84

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 20000000 - Partitions: 2000 - SHA-512 Benchmark TimeCBNA612182430SE +/- 0.16, N = 1223.5523.6923.25

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 20000000 - Partitions: 500 - Group By Test TimeCBNA3691215SE +/- 0.07, N = 3SE +/- 0.11, N = 311.7711.9911.89

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 10000000 - Partitions: 100 - SHA-512 Benchmark TimeCBNA48121620SE +/- 0.10, N = 3SE +/- 0.07, N = 314.0114.1913.93

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 10000000 - Partitions: 100 - Repartition Test TimeCBNA3691215SE +/- 0.09, N = 3SE +/- 0.16, N = 39.309.319.47

SVT-AV1

OpenBenchmarking.orgFrames Per Second, More Is BetterSVT-AV1 1.2Encoder Mode: Preset 4 - Input: Bosphorus 1080pCBNA0.80841.61682.42523.23364.042SE +/- 0.014, N = 3SE +/- 0.015, N = 33.5663.5303.5931. (CXX) g++ options: -march=native -mno-avx -mavx2 -mavx512f -mavx512bw -mavx512dq

Apache Spark

This is a benchmark of Apache Spark with its PySpark interface. Apache Spark is an open-source unified analytics engine for large-scale data processing and dealing with big data. This test profile benchmars the Apache Spark in a single-system configuration using spark-submit. The test makes use of DIYBigData's pyspark-benchmark (https://github.com/DIYBigData/pyspark-benchmark/) for generating of test data and various Apache Spark operations. Learn more via the OpenBenchmarking.org test page.

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 40000000 - Partitions: 500 - Group By Test TimeCBNA714212835SE +/- 0.11, N = 9SE +/- 0.10, N = 531.4531.2631.81

SVT-AV1

OpenBenchmarking.orgFrames Per Second, More Is BetterSVT-AV1 1.2Encoder Mode: Preset 12 - Input: Bosphorus 1080pCBNA50100150200250SE +/- 1.25, N = 3SE +/- 1.68, N = 3219.62223.44221.101. (CXX) g++ options: -march=native -mno-avx -mavx2 -mavx512f -mavx512bw -mavx512dq

Apache Spark

This is a benchmark of Apache Spark with its PySpark interface. Apache Spark is an open-source unified analytics engine for large-scale data processing and dealing with big data. This test profile benchmars the Apache Spark in a single-system configuration using spark-submit. The test makes use of DIYBigData's pyspark-benchmark (https://github.com/DIYBigData/pyspark-benchmark/) for generating of test data and various Apache Spark operations. Learn more via the OpenBenchmarking.org test page.

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 20000000 - Partitions: 500 - Broadcast Inner Join Test TimeCBNA510152025SE +/- 0.12, N = 3SE +/- 0.30, N = 318.6918.3718.43

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 1000000 - Partitions: 2000 - SHA-512 Benchmark TimeCBNA1.12282.24563.36844.49125.614SE +/- 0.049353750, N = 3SE +/- 0.048557793, N = 64.9200000004.9900000004.904759563

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 10000000 - Partitions: 1000 - Calculate Pi Benchmark Using DataframeCBNA246810SE +/- 0.01, N = 12SE +/- 0.01, N = 126.816.806.70

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 20000000 - Partitions: 1000 - Group By Test TimeCBNA3691215SE +/- 0.15, N = 312.9913.2013.18

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 1000000 - Partitions: 100 - Calculate Pi BenchmarkCBNA20406080100SE +/- 0.10, N = 4SE +/- 0.21, N = 3109.87110.17108.42

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 10000000 - Partitions: 100 - Calculate Pi BenchmarkCBNA20406080100SE +/- 0.10, N = 3SE +/- 0.17, N = 3110.41109.36108.69

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 40000000 - Partitions: 1000 - Repartition Test TimeCBNA714212835SE +/- 0.18, N = 931.0231.5131.45

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 1000000 - Partitions: 100 - Calculate Pi Benchmark Using DataframeCBNA246810SE +/- 0.022438723, N = 4SE +/- 0.068811394, N = 36.8600000006.8900000006.783271558

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 10000000 - Partitions: 500 - Group By Test TimeCBNA246810SE +/- 0.074190780, N = 6SE +/- 0.069704950, N = 38.5000000008.4900000008.621025209

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 40000000 - Partitions: 100 - Calculate Pi BenchmarkCBNA20406080100SE +/- 0.92, N = 3SE +/- 0.15, N = 9111.43109.75110.52

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 10000000 - Partitions: 500 - SHA-512 Benchmark TimeCBNA3691215SE +/- 0.13, N = 6SE +/- 0.14, N = 313.2913.0913.20

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 40000000 - Partitions: 2000 - Calculate Pi Benchmark Using DataframeCBNA246810SE +/- 0.02, N = 96.766.846.86

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 1000000 - Partitions: 2000 - Calculate Pi BenchmarkCBNA20406080100SE +/- 0.24, N = 3SE +/- 0.17, N = 6110.12110.04108.60

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 20000000 - Partitions: 1000 - Inner Join Test TimeCBNA510152025SE +/- 0.31, N = 319.8519.9519.68

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 20000000 - Partitions: 100 - Repartition Test TimeCBNA48121620SE +/- 0.07, N = 3SE +/- 0.11, N = 317.3717.3817.15

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 10000000 - Partitions: 2000 - SHA-512 Benchmark TimeCBNA48121620SE +/- 0.07, N = 3SE +/- 0.02, N = 314.2214.0414.14

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 20000000 - Partitions: 2000 - Inner Join Test TimeCBNA510152025SE +/- 0.19, N = 1220.3020.5520.39

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 10000000 - Partitions: 500 - Calculate Pi BenchmarkCBNA20406080100SE +/- 0.25, N = 6SE +/- 0.30, N = 3110.54109.33109.23

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 10000000 - Partitions: 2000 - Calculate Pi Benchmark Using DataframeCBNA246810SE +/- 0.07, N = 3SE +/- 0.02, N = 36.846.786.76

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 40000000 - Partitions: 100 - Group By Test TimeCBNA816243240SE +/- 0.61, N = 3SE +/- 0.07, N = 932.4932.1132.42

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 20000000 - Partitions: 1000 - Calculate Pi Benchmark Using DataframeCBNA246810SE +/- 0.046089856, N = 36.8800000006.8000000006.875336993

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 40000000 - Partitions: 500 - Calculate Pi BenchmarkCBNA20406080100SE +/- 0.09, N = 9SE +/- 0.12, N = 5110.39109.94109.23

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 40000000 - Partitions: 1000 - Group By Test TimeCBNA714212835SE +/- 0.28, N = 929.4329.3429.13

SVT-AV1

OpenBenchmarking.orgFrames Per Second, More Is BetterSVT-AV1 1.2Encoder Mode: Preset 8 - Input: Bosphorus 1080pCBNA20406080100SE +/- 0.10, N = 3SE +/- 0.37, N = 375.2376.0175.961. (CXX) g++ options: -march=native -mno-avx -mavx2 -mavx512f -mavx512bw -mavx512dq

Apache Spark

This is a benchmark of Apache Spark with its PySpark interface. Apache Spark is an open-source unified analytics engine for large-scale data processing and dealing with big data. This test profile benchmars the Apache Spark in a single-system configuration using spark-submit. The test makes use of DIYBigData's pyspark-benchmark (https://github.com/DIYBigData/pyspark-benchmark/) for generating of test data and various Apache Spark operations. Learn more via the OpenBenchmarking.org test page.

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 40000000 - Partitions: 500 - SHA-512 Benchmark TimeCBNA1020304050SE +/- 0.47, N = 9SE +/- 0.47, N = 546.1446.2745.82

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 10000000 - Partitions: 500 - Calculate Pi Benchmark Using DataframeCBNA246810SE +/- 0.025564809, N = 6SE +/- 0.003533182, N = 36.8200000006.7800000006.759258954

SVT-AV1

OpenBenchmarking.orgFrames Per Second, More Is BetterSVT-AV1 1.2Encoder Mode: Preset 10 - Input: Bosphorus 4KCBNA1428425670SE +/- 0.53, N = 3SE +/- 0.21, N = 360.9460.8561.391. (CXX) g++ options: -march=native -mno-avx -mavx2 -mavx512f -mavx512bw -mavx512dq

Apache Spark

This is a benchmark of Apache Spark with its PySpark interface. Apache Spark is an open-source unified analytics engine for large-scale data processing and dealing with big data. This test profile benchmars the Apache Spark in a single-system configuration using spark-submit. The test makes use of DIYBigData's pyspark-benchmark (https://github.com/DIYBigData/pyspark-benchmark/) for generating of test data and various Apache Spark operations. Learn more via the OpenBenchmarking.org test page.

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 20000000 - Partitions: 500 - Calculate Pi Benchmark Using DataframeCBNA246810SE +/- 0.012838149, N = 3SE +/- 0.034639869, N = 36.8600000006.8000000006.812744629

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 1000000 - Partitions: 1000 - Inner Join Test TimeCBNA0.55131.10261.65392.20522.7565SE +/- 0.04, N = 3SE +/- 0.05, N = 32.432.452.43

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 20000000 - Partitions: 500 - SHA-512 Benchmark TimeCBNA510152025SE +/- 0.27, N = 3SE +/- 0.30, N = 322.6422.5822.75

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 20000000 - Partitions: 100 - Calculate Pi Benchmark Using DataframeCBNA246810SE +/- 0.03, N = 3SE +/- 0.01, N = 36.776.826.80

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 1000000 - Partitions: 2000 - Calculate Pi Benchmark Using DataframeCBNA246810SE +/- 0.05, N = 3SE +/- 0.02, N = 66.816.796.84

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 10000000 - Partitions: 100 - Calculate Pi Benchmark Using DataframeCBNA246810SE +/- 0.03, N = 3SE +/- 0.00, N = 36.856.806.81

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 20000000 - Partitions: 1000 - Calculate Pi BenchmarkCBNA20406080100SE +/- 0.19, N = 3109.96110.26110.73

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 1000000 - Partitions: 1000 - Calculate Pi Benchmark Using DataframeCBNA246810SE +/- 0.025947993, N = 3SE +/- 0.031303906, N = 36.8400000006.8200000006.799198247

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 10000000 - Partitions: 100 - Broadcast Inner Join Test TimeCBNA3691215SE +/- 0.11, N = 3SE +/- 0.24, N = 310.7210.6910.66

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 10000000 - Partitions: 1000 - Calculate Pi BenchmarkCBNA20406080100SE +/- 0.15, N = 12SE +/- 0.13, N = 12110.11109.97109.52

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 20000000 - Partitions: 100 - Group By Test TimeCBNA3691215SE +/- 0.07, N = 3SE +/- 0.20, N = 313.3513.4013.33

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 1000000 - Partitions: 500 - Calculate Pi Benchmark Using DataframeCBNA246810SE +/- 0.02772713, N = 3SE +/- 0.02582423, N = 86.810000006.810000006.77685602

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 20000000 - Partitions: 100 - Calculate Pi BenchmarkCBNA20406080100SE +/- 0.14, N = 3SE +/- 0.21, N = 3109.62109.58109.11

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 40000000 - Partitions: 2000 - Calculate Pi BenchmarkCBNA20406080100SE +/- 0.21, N = 9110.59110.55110.09

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 40000000 - Partitions: 1000 - Calculate Pi BenchmarkCBNA20406080100SE +/- 0.19, N = 9110.27110.39109.90

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 20000000 - Partitions: 2000 - Calculate Pi BenchmarkCBNA20406080100SE +/- 0.12, N = 12109.92109.90109.44

SVT-AV1

OpenBenchmarking.orgFrames Per Second, More Is BetterSVT-AV1 1.2Encoder Mode: Preset 12 - Input: Bosphorus 4KCBNA20406080100SE +/- 0.47, N = 3SE +/- 0.46, N = 389.3789.4689.751. (CXX) g++ options: -march=native -mno-avx -mavx2 -mavx512f -mavx512bw -mavx512dq

OpenBenchmarking.orgFrames Per Second, More Is BetterSVT-AV1 1.2Encoder Mode: Preset 8 - Input: Bosphorus 4KCBNA714212835SE +/- 0.08, N = 3SE +/- 0.10, N = 332.0232.1332.001. (CXX) g++ options: -march=native -mno-avx -mavx2 -mavx512f -mavx512bw -mavx512dq

OpenBenchmarking.orgFrames Per Second, More Is BetterSVT-AV1 1.2Encoder Mode: Preset 4 - Input: Bosphorus 4KCBNA0.31140.62280.93421.24561.557SE +/- 0.001, N = 3SE +/- 0.004, N = 31.3791.3791.3841. (CXX) g++ options: -march=native -mno-avx -mavx2 -mavx512f -mavx512bw -mavx512dq

Apache Spark

This is a benchmark of Apache Spark with its PySpark interface. Apache Spark is an open-source unified analytics engine for large-scale data processing and dealing with big data. This test profile benchmars the Apache Spark in a single-system configuration using spark-submit. The test makes use of DIYBigData's pyspark-benchmark (https://github.com/DIYBigData/pyspark-benchmark/) for generating of test data and various Apache Spark operations. Learn more via the OpenBenchmarking.org test page.

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 1000000 - Partitions: 1000 - Calculate Pi BenchmarkCBNA20406080100SE +/- 0.15, N = 3SE +/- 0.38, N = 3110.06109.68109.81

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 20000000 - Partitions: 500 - Calculate Pi BenchmarkCBNA20406080100SE +/- 0.17, N = 3SE +/- 0.13, N = 3109.98109.97109.61

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 10000000 - Partitions: 2000 - Calculate Pi BenchmarkCBNA20406080100SE +/- 0.22, N = 3SE +/- 0.63, N = 3109.96110.25109.91

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 20000000 - Partitions: 2000 - Calculate Pi Benchmark Using DataframeCBNA246810SE +/- 0.01, N = 126.776.796.77

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 10000000 - Partitions: 2000 - Broadcast Inner Join Test TimeCBNA3691215SE +/- 0.50, N = 3SE +/- 0.35, N = 310.5410.839.96

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 20000000 - Partitions: 100 - Inner Join Test TimeCBNA510152025SE +/- 0.24, N = 3SE +/- 0.71, N = 320.6020.3520.68

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 1000000 - Partitions: 500 - Broadcast Inner Join Test TimeCBNA0.40280.80561.20841.61122.014SE +/- 0.04087612, N = 3SE +/- 0.05474021, N = 81.790000001.770000001.70007341

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 1000000 - Partitions: 500 - Inner Join Test TimeCBNA0.50851.0171.52552.0342.5425SE +/- 0.110292282, N = 3SE +/- 0.028445371, N = 82.2600000002.2500000002.170995055

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 1000000 - Partitions: 100 - Broadcast Inner Join Test TimeCBNA0.44380.88761.33141.77522.219SE +/- 0.055547846, N = 4SE +/- 0.059135782, N = 31.7500000001.8800000001.972517798

120 Results Shown

Apache Spark:
  1000000 - 2000 - Repartition Test Time
  1000000 - 100 - Inner Join Test Time
  20000000 - 1000 - SHA-512 Benchmark Time
  40000000 - 1000 - SHA-512 Benchmark Time
  10000000 - 500 - Inner Join Test Time
  1000000 - 1000 - SHA-512 Benchmark Time
  1000000 - 2000 - Broadcast Inner Join Test Time
  20000000 - 1000 - Broadcast Inner Join Test Time
  1000000 - 1000 - Broadcast Inner Join Test Time
  40000000 - 2000 - Group By Test Time
  10000000 - 1000 - Inner Join Test Time
  20000000 - 1000 - Repartition Test Time
  10000000 - 1000 - Broadcast Inner Join Test Time
  1000000 - 1000 - Repartition Test Time
  10000000 - 2000 - Group By Test Time
  1000000 - 100 - Repartition Test Time
  1000000 - 500 - SHA-512 Benchmark Time
  40000000 - 1000 - Broadcast Inner Join Test Time
SVT-AV1
Apache Spark:
  40000000 - 100 - Broadcast Inner Join Test Time
  40000000 - 100 - Repartition Test Time
  40000000 - 100 - SHA-512 Benchmark Time
  40000000 - 2000 - Inner Join Test Time
  10000000 - 100 - Inner Join Test Time
  1000000 - 500 - Repartition Test Time
  40000000 - 500 - Broadcast Inner Join Test Time
  1000000 - 1000 - Group By Test Time
  10000000 - 1000 - Group By Test Time
  10000000 - 2000 - Inner Join Test Time
  40000000 - 2000 - SHA-512 Benchmark Time
  20000000 - 500 - Repartition Test Time
  20000000 - 2000 - Broadcast Inner Join Test Time
  40000000 - 500 - Inner Join Test Time
  40000000 - 2000 - Repartition Test Time
  40000000 - 1000 - Inner Join Test Time
  10000000 - 500 - Broadcast Inner Join Test Time
  20000000 - 500 - Inner Join Test Time
  1000000 - 100 - SHA-512 Benchmark Time
  1000000 - 2000 - Inner Join Test Time
  40000000 - 2000 - Broadcast Inner Join Test Time
  10000000 - 1000 - Repartition Test Time
  10000000 - 100 - Group By Test Time
  20000000 - 2000 - Group By Test Time
  40000000 - 500 - Repartition Test Time
  20000000 - 100 - SHA-512 Benchmark Time
  10000000 - 1000 - SHA-512 Benchmark Time
  20000000 - 2000 - Repartition Test Time
  1000000 - 500 - Group By Test Time
  1000000 - 2000 - Group By Test Time
  1000000 - 100 - Group By Test Time
  10000000 - 2000 - Repartition Test Time
  20000000 - 100 - Broadcast Inner Join Test Time
  40000000 - 100 - Inner Join Test Time
  10000000 - 500 - Repartition Test Time
  40000000 - 500 - Calculate Pi Benchmark Using Dataframe
  1000000 - 500 - Calculate Pi Benchmark
  40000000 - 1000 - Calculate Pi Benchmark Using Dataframe
  40000000 - 100 - Calculate Pi Benchmark Using Dataframe
  20000000 - 2000 - SHA-512 Benchmark Time
  20000000 - 500 - Group By Test Time
  10000000 - 100 - SHA-512 Benchmark Time
  10000000 - 100 - Repartition Test Time
SVT-AV1
Apache Spark
SVT-AV1
Apache Spark:
  20000000 - 500 - Broadcast Inner Join Test Time
  1000000 - 2000 - SHA-512 Benchmark Time
  10000000 - 1000 - Calculate Pi Benchmark Using Dataframe
  20000000 - 1000 - Group By Test Time
  1000000 - 100 - Calculate Pi Benchmark
  10000000 - 100 - Calculate Pi Benchmark
  40000000 - 1000 - Repartition Test Time
  1000000 - 100 - Calculate Pi Benchmark Using Dataframe
  10000000 - 500 - Group By Test Time
  40000000 - 100 - Calculate Pi Benchmark
  10000000 - 500 - SHA-512 Benchmark Time
  40000000 - 2000 - Calculate Pi Benchmark Using Dataframe
  1000000 - 2000 - Calculate Pi Benchmark
  20000000 - 1000 - Inner Join Test Time
  20000000 - 100 - Repartition Test Time
  10000000 - 2000 - SHA-512 Benchmark Time
  20000000 - 2000 - Inner Join Test Time
  10000000 - 500 - Calculate Pi Benchmark
  10000000 - 2000 - Calculate Pi Benchmark Using Dataframe
  40000000 - 100 - Group By Test Time
  20000000 - 1000 - Calculate Pi Benchmark Using Dataframe
  40000000 - 500 - Calculate Pi Benchmark
  40000000 - 1000 - Group By Test Time
SVT-AV1
Apache Spark:
  40000000 - 500 - SHA-512 Benchmark Time
  10000000 - 500 - Calculate Pi Benchmark Using Dataframe
SVT-AV1
Apache Spark:
  20000000 - 500 - Calculate Pi Benchmark Using Dataframe
  1000000 - 1000 - Inner Join Test Time
  20000000 - 500 - SHA-512 Benchmark Time
  20000000 - 100 - Calculate Pi Benchmark Using Dataframe
  1000000 - 2000 - Calculate Pi Benchmark Using Dataframe
  10000000 - 100 - Calculate Pi Benchmark Using Dataframe
  20000000 - 1000 - Calculate Pi Benchmark
  1000000 - 1000 - Calculate Pi Benchmark Using Dataframe
  10000000 - 100 - Broadcast Inner Join Test Time
  10000000 - 1000 - Calculate Pi Benchmark
  20000000 - 100 - Group By Test Time
  1000000 - 500 - Calculate Pi Benchmark Using Dataframe
  20000000 - 100 - Calculate Pi Benchmark
  40000000 - 2000 - Calculate Pi Benchmark
  40000000 - 1000 - Calculate Pi Benchmark
  20000000 - 2000 - Calculate Pi Benchmark
SVT-AV1:
  Preset 12 - Bosphorus 4K
  Preset 8 - Bosphorus 4K
  Preset 4 - Bosphorus 4K
Apache Spark:
  1000000 - 1000 - Calculate Pi Benchmark
  20000000 - 500 - Calculate Pi Benchmark
  10000000 - 2000 - Calculate Pi Benchmark
  20000000 - 2000 - Calculate Pi Benchmark Using Dataframe
  10000000 - 2000 - Broadcast Inner Join Test Time
  20000000 - 100 - Inner Join Test Time
  1000000 - 500 - Broadcast Inner Join Test Time
  1000000 - 500 - Inner Join Test Time
  1000000 - 100 - Broadcast Inner Join Test Time