apache spark 5800X3D

AMD Ryzen 7 5800X3D 8-Core testing with a ASUS ROG CROSSHAIR VIII HERO (4201 BIOS) and Gigabyte AMD Radeon RX 6600 8GB on Ubuntu 22.04 via the Phoronix Test Suite.

HTML result view exported from: https://openbenchmarking.org/result/2208042-PTS-APACHESP16&grt.

apache spark 5800X3DProcessorMotherboardChipsetMemoryDiskGraphicsAudioMonitorNetworkOSKernelDesktopDisplay ServerOpenGLVulkanCompilerFile-SystemScreen ResolutionABCAMD Ryzen 7 5800X3D 8-Core @ 3.40GHz (8 Cores / 16 Threads)ASUS ROG CROSSHAIR VIII HERO (4201 BIOS)AMD Starship/Matisse32GB1000GB Western Digital WDS100T1X0E-00AFY0Gigabyte AMD Radeon RX 6600 8GB (2750/875MHz)AMD Navi 21 HDMI AudioASUS MG28URealtek RTL8125 2.5GbE + Intel I211Ubuntu 22.045.19.0-rc6-drm-next-28july (x86_64)GNOME Shell 42.2X Server 1.21.1.3 + Wayland4.6 Mesa 22.2.0-devel (git-c6ded153d1) (LLVM 12.0.0 DRM 3.48)1.3.211GCC 11.2.0ext43840x2160OpenBenchmarking.orgKernel Details- Transparent Huge Pages: madviseProcessor Details- Scaling Governor: acpi-cpufreq schedutil (Boost: Enabled) - CPU Microcode: 0xa20120aJava Details- OpenJDK Runtime Environment (build 11.0.15+10-Ubuntu-0ubuntu0.22.04.1)Python Details- Python 3.10.4Security Details- itlb_multihit: Not affected + l1tf: Not affected + mds: Not affected + meltdown: Not affected + mmio_stale_data: Not affected + spec_store_bypass: Mitigation of SSB disabled via prctl + spectre_v1: Mitigation of usercopy/swapgs barriers and __user pointer sanitization + spectre_v2: Mitigation of Retpolines IBPB: conditional IBRS_FW STIBP: always-on RSB filling + srbds: Not affected + tsx_async_abort: Not affected

apache spark 5800X3Dspark: 1000000 - 100 - SHA-512 Benchmark Timespark: 1000000 - 100 - Calculate Pi Benchmarkspark: 1000000 - 100 - Calculate Pi Benchmark Using Dataframespark: 1000000 - 100 - Group By Test Timespark: 1000000 - 100 - Repartition Test Timespark: 1000000 - 100 - Inner Join Test Timespark: 1000000 - 100 - Broadcast Inner Join Test Timespark: 1000000 - 500 - SHA-512 Benchmark Timespark: 1000000 - 500 - Calculate Pi Benchmarkspark: 1000000 - 500 - Calculate Pi Benchmark Using Dataframespark: 1000000 - 500 - Group By Test Timespark: 1000000 - 500 - Repartition Test Timespark: 1000000 - 500 - Inner Join Test Timespark: 1000000 - 500 - Broadcast Inner Join Test Timespark: 1000000 - 1000 - SHA-512 Benchmark Timespark: 1000000 - 1000 - Calculate Pi Benchmarkspark: 1000000 - 1000 - Calculate Pi Benchmark Using Dataframespark: 1000000 - 1000 - Group By Test Timespark: 1000000 - 1000 - Repartition Test Timespark: 1000000 - 1000 - Inner Join Test Timespark: 1000000 - 1000 - Broadcast Inner Join Test Timespark: 1000000 - 2000 - SHA-512 Benchmark Timespark: 1000000 - 2000 - Calculate Pi Benchmarkspark: 1000000 - 2000 - Calculate Pi Benchmark Using Dataframespark: 1000000 - 2000 - Group By Test Timespark: 1000000 - 2000 - Repartition Test Timespark: 1000000 - 2000 - Inner Join Test Timespark: 1000000 - 2000 - Broadcast Inner Join Test Timespark: 10000000 - 100 - SHA-512 Benchmark Timespark: 10000000 - 100 - Calculate Pi Benchmarkspark: 10000000 - 100 - Calculate Pi Benchmark Using Dataframespark: 10000000 - 100 - Group By Test Timespark: 10000000 - 100 - Repartition Test Timespark: 10000000 - 100 - Inner Join Test Timespark: 10000000 - 100 - Broadcast Inner Join Test Timespark: 10000000 - 500 - SHA-512 Benchmark Timespark: 10000000 - 500 - Calculate Pi Benchmarkspark: 10000000 - 500 - Calculate Pi Benchmark Using Dataframespark: 10000000 - 500 - Group By Test Timespark: 10000000 - 500 - Repartition Test Timespark: 10000000 - 500 - Inner Join Test Timespark: 10000000 - 500 - Broadcast Inner Join Test Timespark: 20000000 - 100 - SHA-512 Benchmark Timespark: 20000000 - 100 - Calculate Pi Benchmarkspark: 20000000 - 100 - Calculate Pi Benchmark Using Dataframespark: 20000000 - 100 - Group By Test Timespark: 20000000 - 100 - Repartition Test Timespark: 20000000 - 100 - Inner Join Test Timespark: 20000000 - 100 - Broadcast Inner Join Test Timespark: 20000000 - 500 - SHA-512 Benchmark Timespark: 20000000 - 500 - Calculate Pi Benchmarkspark: 20000000 - 500 - Calculate Pi Benchmark Using Dataframespark: 20000000 - 500 - Group By Test Timespark: 20000000 - 500 - Repartition Test Timespark: 20000000 - 500 - Inner Join Test Timespark: 20000000 - 500 - Broadcast Inner Join Test Timespark: 40000000 - 100 - SHA-512 Benchmark Timespark: 40000000 - 100 - Calculate Pi Benchmarkspark: 40000000 - 100 - Calculate Pi Benchmark Using Dataframespark: 40000000 - 100 - Group By Test Timespark: 40000000 - 100 - Repartition Test Timespark: 40000000 - 100 - Inner Join Test Timespark: 40000000 - 100 - Broadcast Inner Join Test Timespark: 40000000 - 500 - SHA-512 Benchmark Timespark: 40000000 - 500 - Calculate Pi Benchmarkspark: 40000000 - 500 - Calculate Pi Benchmark Using Dataframespark: 40000000 - 500 - Group By Test Timespark: 40000000 - 500 - Repartition Test Timespark: 40000000 - 500 - Inner Join Test Timespark: 40000000 - 500 - Broadcast Inner Join Test Timespark: 10000000 - 1000 - SHA-512 Benchmark Timespark: 10000000 - 1000 - Calculate Pi Benchmarkspark: 10000000 - 1000 - Calculate Pi Benchmark Using Dataframespark: 10000000 - 1000 - Group By Test Timespark: 10000000 - 1000 - Repartition Test Timespark: 10000000 - 1000 - Inner Join Test Timespark: 10000000 - 1000 - Broadcast Inner Join Test Timespark: 10000000 - 2000 - SHA-512 Benchmark Timespark: 10000000 - 2000 - Calculate Pi Benchmarkspark: 10000000 - 2000 - Calculate Pi Benchmark Using Dataframespark: 10000000 - 2000 - Group By Test Timespark: 10000000 - 2000 - Repartition Test Timespark: 10000000 - 2000 - Inner Join Test Timespark: 10000000 - 2000 - Broadcast Inner Join Test Timespark: 20000000 - 1000 - SHA-512 Benchmark Timespark: 20000000 - 1000 - Calculate Pi Benchmarkspark: 20000000 - 1000 - Calculate Pi Benchmark Using Dataframespark: 20000000 - 1000 - Group By Test Timespark: 20000000 - 1000 - Repartition Test Timespark: 20000000 - 1000 - Inner Join Test Timespark: 20000000 - 1000 - Broadcast Inner Join Test Timespark: 20000000 - 2000 - SHA-512 Benchmark Timespark: 20000000 - 2000 - Calculate Pi Benchmarkspark: 20000000 - 2000 - Calculate Pi Benchmark Using Dataframespark: 20000000 - 2000 - Group By Test Timespark: 20000000 - 2000 - Repartition Test Timespark: 20000000 - 2000 - Inner Join Test Timespark: 20000000 - 2000 - Broadcast Inner Join Test Timespark: 40000000 - 1000 - SHA-512 Benchmark Timespark: 40000000 - 1000 - Calculate Pi Benchmarkspark: 40000000 - 1000 - Calculate Pi Benchmark Using Dataframespark: 40000000 - 1000 - Group By Test Timespark: 40000000 - 1000 - Repartition Test Timespark: 40000000 - 1000 - Inner Join Test Timespark: 40000000 - 1000 - Broadcast Inner Join Test Timespark: 40000000 - 2000 - SHA-512 Benchmark Timespark: 40000000 - 2000 - Calculate Pi Benchmarkspark: 40000000 - 2000 - Calculate Pi Benchmark Using Dataframespark: 40000000 - 2000 - Group By Test Timespark: 40000000 - 2000 - Repartition Test Timespark: 40000000 - 2000 - Inner Join Test Timespark: 40000000 - 2000 - Broadcast Inner Join Test TimeABC2.91149.3487084148.672.891.901.451.313.06150.1846839248.693.151.991.631.483.17150.4094903338.683.422.201.921.583.46150.0316450928.683.742.502.322.0214.539789898149.7587099758.666.609.1611.34007182211.56624935613.94148.9820852318.696.758.9410.32010252610.58974872727.22149.3621860518.6710.6517.90807420021.8222.0825.771477402148.2244582688.6010.11860833916.9420.03552767020.3549.41149.2275087878.6327.36205143533.93305839340.0040.7850.13151.3309837938.5923.9733.54696650039.5540.3213.97148.5484723448.496.899.0510.7910.6514.15148.5485057058.587.039.3411.2010.6025.55149.2301684848.6210.2517.0319.9719.8725.70148.8535805598.6410.2517.5220.7920.0149.63149.4506584238.6122.8934.5039.6040.99309014250.33148.6018937418.5923.2634.5640.6041.552.88148.7353831318.612.881.931.401.363.02149.1346132808.673.201.991.631.453.19149.5204973648.613.432.201.841.663.47148.5830934158.613.842.462.362.0514.60149.5648260298.666.649.1511.0111.2813.97148.9157575358.686.829.0310.4810.1127.24148.7867671118.5910.5017.8122.0822.2725.98149.1399807358.6010.0517.0219.7720.0549.28148.5971756718.6327.2834.4139.8740.1449.86149.1286950608.6123.4933.8540.1440.76068007213.89148.5327314508.546.779.0111.0210.6014.12149.1833871538.577.149.2211.2110.5625.53147.9146059828.5810.1617.0920.3320.6625.76148.3005900128.6010.3217.8120.7219.8749.95148.6644341378.6223.2434.0940.5041.0151.35148.5027161958.5722.9433.9839.5840.712.84146.520975598.522.881.961.441.303.03148.5657660468.553.272.021.621.383.14149.5297730248.593.362.161.761.743.48149.717676978.633.792.542.501.9414.47149.067495028.646.558.9510.9311.5414.07148.0558918258.676.668.8310.1410.2427.29149.2458238998.7110.6018.0522.0922.65042046225.62148.9393673238.5710.0417.49321074521.2119.9449.53149.6108881798.6527.38216917234.7441.9839.62136240548.84147.7151803638.5823.0434.1140.1740.29138936414.07148.9885660838.616.868.9010.5110.0814.14150.2758108048.6878927257.049.2411.6610.41190633325.60149.2927512958.5410.0317.0120.4921.0825.72149.8470028688.5910.4117.4420.5019.3149.84151.9411348698.5623.1834.6239.89910031541.2050.12147.9813802848.5023.17078855933.9639.84819883641.18OpenBenchmarking.org

Apache Spark

Row Count: 1000000 - Partitions: 100 - SHA-512 Benchmark Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 1000000 - Partitions: 100 - SHA-512 Benchmark TimeABC0.65481.30961.96442.61923.274SE +/- 0.02, N = 12SE +/- 0.03, N = 32.912.882.84

Apache Spark

Row Count: 1000000 - Partitions: 100 - Calculate Pi Benchmark

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 1000000 - Partitions: 100 - Calculate Pi BenchmarkABC306090120150SE +/- 0.20, N = 12SE +/- 0.46, N = 3149.35148.74146.52

Apache Spark

Row Count: 1000000 - Partitions: 100 - Calculate Pi Benchmark Using Dataframe

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 1000000 - Partitions: 100 - Calculate Pi Benchmark Using DataframeABC246810SE +/- 0.01, N = 12SE +/- 0.02, N = 38.678.618.52

Apache Spark

Row Count: 1000000 - Partitions: 100 - Group By Test Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 1000000 - Partitions: 100 - Group By Test TimeABC0.65031.30061.95092.60123.2515SE +/- 0.02, N = 12SE +/- 0.01, N = 32.892.882.88

Apache Spark

Row Count: 1000000 - Partitions: 100 - Repartition Test Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 1000000 - Partitions: 100 - Repartition Test TimeABC0.4410.8821.3231.7642.205SE +/- 0.01, N = 12SE +/- 0.03, N = 31.901.931.96

Apache Spark

Row Count: 1000000 - Partitions: 100 - Inner Join Test Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 1000000 - Partitions: 100 - Inner Join Test TimeABC0.32630.65260.97891.30521.6315SE +/- 0.02, N = 12SE +/- 0.06, N = 31.451.401.44

Apache Spark

Row Count: 1000000 - Partitions: 100 - Broadcast Inner Join Test Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 1000000 - Partitions: 100 - Broadcast Inner Join Test TimeABC0.3060.6120.9181.2241.53SE +/- 0.02, N = 12SE +/- 0.07, N = 31.311.361.30

Apache Spark

Row Count: 1000000 - Partitions: 500 - SHA-512 Benchmark Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 1000000 - Partitions: 500 - SHA-512 Benchmark TimeABC0.68851.3772.06552.7543.4425SE +/- 0.03, N = 4SE +/- 0.02, N = 33.063.023.03

Apache Spark

Row Count: 1000000 - Partitions: 500 - Calculate Pi Benchmark

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 1000000 - Partitions: 500 - Calculate Pi BenchmarkABC306090120150SE +/- 0.65, N = 4SE +/- 0.44, N = 3150.18149.13148.57

Apache Spark

Row Count: 1000000 - Partitions: 500 - Calculate Pi Benchmark Using Dataframe

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 1000000 - Partitions: 500 - Calculate Pi Benchmark Using DataframeABC246810SE +/- 0.04, N = 4SE +/- 0.04, N = 38.698.678.55

Apache Spark

Row Count: 1000000 - Partitions: 500 - Group By Test Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 1000000 - Partitions: 500 - Group By Test TimeABC0.73581.47162.20742.94323.679SE +/- 0.03, N = 4SE +/- 0.03, N = 33.153.203.27

Apache Spark

Row Count: 1000000 - Partitions: 500 - Repartition Test Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 1000000 - Partitions: 500 - Repartition Test TimeABC0.45450.9091.36351.8182.2725SE +/- 0.02, N = 4SE +/- 0.02, N = 31.991.992.02

Apache Spark

Row Count: 1000000 - Partitions: 500 - Inner Join Test Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 1000000 - Partitions: 500 - Inner Join Test TimeABC0.36680.73361.10041.46721.834SE +/- 0.04, N = 4SE +/- 0.03, N = 31.631.631.62

Apache Spark

Row Count: 1000000 - Partitions: 500 - Broadcast Inner Join Test Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 1000000 - Partitions: 500 - Broadcast Inner Join Test TimeABC0.3330.6660.9991.3321.665SE +/- 0.07, N = 4SE +/- 0.02, N = 31.481.451.38

Apache Spark

Row Count: 1000000 - Partitions: 1000 - SHA-512 Benchmark Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 1000000 - Partitions: 1000 - SHA-512 Benchmark TimeABC0.71781.43562.15342.87123.589SE +/- 0.02, N = 3SE +/- 0.03, N = 123.173.193.14

Apache Spark

Row Count: 1000000 - Partitions: 1000 - Calculate Pi Benchmark

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 1000000 - Partitions: 1000 - Calculate Pi BenchmarkABC306090120150SE +/- 0.39, N = 3SE +/- 0.25, N = 12150.41149.52149.53

Apache Spark

Row Count: 1000000 - Partitions: 1000 - Calculate Pi Benchmark Using Dataframe

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 1000000 - Partitions: 1000 - Calculate Pi Benchmark Using DataframeABC246810SE +/- 0.04, N = 3SE +/- 0.02, N = 128.688.618.59

Apache Spark

Row Count: 1000000 - Partitions: 1000 - Group By Test Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 1000000 - Partitions: 1000 - Group By Test TimeABC0.77181.54362.31543.08723.859SE +/- 0.05, N = 3SE +/- 0.02, N = 123.423.433.36

Apache Spark

Row Count: 1000000 - Partitions: 1000 - Repartition Test Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 1000000 - Partitions: 1000 - Repartition Test TimeABC0.4950.991.4851.982.475SE +/- 0.01, N = 3SE +/- 0.01, N = 122.202.202.16

Apache Spark

Row Count: 1000000 - Partitions: 1000 - Inner Join Test Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 1000000 - Partitions: 1000 - Inner Join Test TimeABC0.4320.8641.2961.7282.16SE +/- 0.01, N = 3SE +/- 0.02, N = 121.921.841.76

Apache Spark

Row Count: 1000000 - Partitions: 1000 - Broadcast Inner Join Test Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 1000000 - Partitions: 1000 - Broadcast Inner Join Test TimeABC0.39150.7831.17451.5661.9575SE +/- 0.02, N = 3SE +/- 0.03, N = 121.581.661.74

Apache Spark

Row Count: 1000000 - Partitions: 2000 - SHA-512 Benchmark Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 1000000 - Partitions: 2000 - SHA-512 Benchmark TimeABC0.7831.5662.3493.1323.915SE +/- 0.04, N = 3SE +/- 0.03, N = 33.463.473.48

Apache Spark

Row Count: 1000000 - Partitions: 2000 - Calculate Pi Benchmark

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 1000000 - Partitions: 2000 - Calculate Pi BenchmarkABC306090120150SE +/- 0.94, N = 3SE +/- 0.29, N = 3150.03148.58149.72

Apache Spark

Row Count: 1000000 - Partitions: 2000 - Calculate Pi Benchmark Using Dataframe

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 1000000 - Partitions: 2000 - Calculate Pi Benchmark Using DataframeABC246810SE +/- 0.02, N = 3SE +/- 0.02, N = 38.688.618.63

Apache Spark

Row Count: 1000000 - Partitions: 2000 - Group By Test Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 1000000 - Partitions: 2000 - Group By Test TimeABC0.8641.7282.5923.4564.32SE +/- 0.04, N = 3SE +/- 0.04, N = 33.743.843.79

Apache Spark

Row Count: 1000000 - Partitions: 2000 - Repartition Test Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 1000000 - Partitions: 2000 - Repartition Test TimeABC0.57151.1431.71452.2862.8575SE +/- 0.01, N = 3SE +/- 0.05, N = 32.502.462.54

Apache Spark

Row Count: 1000000 - Partitions: 2000 - Inner Join Test Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 1000000 - Partitions: 2000 - Inner Join Test TimeABC0.56251.1251.68752.252.8125SE +/- 0.07, N = 3SE +/- 0.07, N = 32.322.362.50

Apache Spark

Row Count: 1000000 - Partitions: 2000 - Broadcast Inner Join Test Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 1000000 - Partitions: 2000 - Broadcast Inner Join Test TimeABC0.46130.92261.38391.84522.3065SE +/- 0.04, N = 3SE +/- 0.05, N = 32.022.051.94

Apache Spark

Row Count: 10000000 - Partitions: 100 - SHA-512 Benchmark Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 10000000 - Partitions: 100 - SHA-512 Benchmark TimeABC48121620SE +/- 0.10, N = 3SE +/- 0.05, N = 314.5414.6014.47

Apache Spark

Row Count: 10000000 - Partitions: 100 - Calculate Pi Benchmark

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 10000000 - Partitions: 100 - Calculate Pi BenchmarkABC306090120150SE +/- 0.68, N = 3SE +/- 0.44, N = 3149.76149.56149.07

Apache Spark

Row Count: 10000000 - Partitions: 100 - Calculate Pi Benchmark Using Dataframe

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 10000000 - Partitions: 100 - Calculate Pi Benchmark Using DataframeABC246810SE +/- 0.01, N = 3SE +/- 0.01, N = 38.668.668.64

Apache Spark

Row Count: 10000000 - Partitions: 100 - Group By Test Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 10000000 - Partitions: 100 - Group By Test TimeABC246810SE +/- 0.04, N = 3SE +/- 0.07, N = 36.606.646.55

Apache Spark

Row Count: 10000000 - Partitions: 100 - Repartition Test Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 10000000 - Partitions: 100 - Repartition Test TimeABC3691215SE +/- 0.12, N = 3SE +/- 0.11, N = 39.169.158.95

Apache Spark

Row Count: 10000000 - Partitions: 100 - Inner Join Test Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 10000000 - Partitions: 100 - Inner Join Test TimeABC3691215SE +/- 0.02, N = 3SE +/- 0.27, N = 311.3411.0110.93

Apache Spark

Row Count: 10000000 - Partitions: 100 - Broadcast Inner Join Test Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 10000000 - Partitions: 100 - Broadcast Inner Join Test TimeABC3691215SE +/- 0.04, N = 3SE +/- 0.22, N = 311.5711.2811.54

Apache Spark

Row Count: 10000000 - Partitions: 500 - SHA-512 Benchmark Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 10000000 - Partitions: 500 - SHA-512 Benchmark TimeABC48121620SE +/- 0.02, N = 3SE +/- 0.02, N = 313.9413.9714.07

Apache Spark

Row Count: 10000000 - Partitions: 500 - Calculate Pi Benchmark

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 10000000 - Partitions: 500 - Calculate Pi BenchmarkABC306090120150SE +/- 0.27, N = 3SE +/- 0.45, N = 3148.98148.92148.06

Apache Spark

Row Count: 10000000 - Partitions: 500 - Calculate Pi Benchmark Using Dataframe

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 10000000 - Partitions: 500 - Calculate Pi Benchmark Using DataframeABC246810SE +/- 0.02, N = 3SE +/- 0.03, N = 38.698.688.67

Apache Spark

Row Count: 10000000 - Partitions: 500 - Group By Test Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 10000000 - Partitions: 500 - Group By Test TimeABC246810SE +/- 0.08, N = 3SE +/- 0.03, N = 36.756.826.66

Apache Spark

Row Count: 10000000 - Partitions: 500 - Repartition Test Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 10000000 - Partitions: 500 - Repartition Test TimeABC3691215SE +/- 0.03, N = 3SE +/- 0.12, N = 38.949.038.83

Apache Spark

Row Count: 10000000 - Partitions: 500 - Inner Join Test Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 10000000 - Partitions: 500 - Inner Join Test TimeABC3691215SE +/- 0.11, N = 3SE +/- 0.19, N = 310.3210.4810.14

Apache Spark

Row Count: 10000000 - Partitions: 500 - Broadcast Inner Join Test Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 10000000 - Partitions: 500 - Broadcast Inner Join Test TimeABC3691215SE +/- 0.16, N = 3SE +/- 0.04, N = 310.5910.1110.24

Apache Spark

Row Count: 20000000 - Partitions: 100 - SHA-512 Benchmark Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 20000000 - Partitions: 100 - SHA-512 Benchmark TimeABC612182430SE +/- 0.12, N = 3SE +/- 0.12, N = 327.2227.2427.29

Apache Spark

Row Count: 20000000 - Partitions: 100 - Calculate Pi Benchmark

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 20000000 - Partitions: 100 - Calculate Pi BenchmarkABC306090120150SE +/- 0.30, N = 3SE +/- 0.50, N = 3149.36148.79149.25

Apache Spark

Row Count: 20000000 - Partitions: 100 - Calculate Pi Benchmark Using Dataframe

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 20000000 - Partitions: 100 - Calculate Pi Benchmark Using DataframeABC246810SE +/- 0.02, N = 3SE +/- 0.05, N = 38.678.598.71

Apache Spark

Row Count: 20000000 - Partitions: 100 - Group By Test Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 20000000 - Partitions: 100 - Group By Test TimeABC3691215SE +/- 0.06, N = 3SE +/- 0.09, N = 310.6510.5010.60

Apache Spark

Row Count: 20000000 - Partitions: 100 - Repartition Test Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 20000000 - Partitions: 100 - Repartition Test TimeABC48121620SE +/- 0.09, N = 3SE +/- 0.19, N = 317.9117.8118.05

Apache Spark

Row Count: 20000000 - Partitions: 100 - Inner Join Test Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 20000000 - Partitions: 100 - Inner Join Test TimeABC510152025SE +/- 0.26, N = 3SE +/- 0.15, N = 321.8222.0822.09

Apache Spark

Row Count: 20000000 - Partitions: 100 - Broadcast Inner Join Test Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 20000000 - Partitions: 100 - Broadcast Inner Join Test TimeABC510152025SE +/- 0.48, N = 3SE +/- 0.13, N = 322.0822.2722.65

Apache Spark

Row Count: 20000000 - Partitions: 500 - SHA-512 Benchmark Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 20000000 - Partitions: 500 - SHA-512 Benchmark TimeABC612182430SE +/- 0.12, N = 3SE +/- 0.31, N = 425.7725.9825.62

Apache Spark

Row Count: 20000000 - Partitions: 500 - Calculate Pi Benchmark

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 20000000 - Partitions: 500 - Calculate Pi BenchmarkABC306090120150SE +/- 0.25, N = 3SE +/- 0.53, N = 4148.22149.14148.94

Apache Spark

Row Count: 20000000 - Partitions: 500 - Calculate Pi Benchmark Using Dataframe

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 20000000 - Partitions: 500 - Calculate Pi Benchmark Using DataframeABC246810SE +/- 0.02, N = 3SE +/- 0.03, N = 48.608.608.57

Apache Spark

Row Count: 20000000 - Partitions: 500 - Group By Test Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 20000000 - Partitions: 500 - Group By Test TimeABC3691215SE +/- 0.08, N = 3SE +/- 0.06, N = 410.1210.0510.04

Apache Spark

Row Count: 20000000 - Partitions: 500 - Repartition Test Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 20000000 - Partitions: 500 - Repartition Test TimeABC48121620SE +/- 0.11, N = 3SE +/- 0.07, N = 416.9417.0217.49

Apache Spark

Row Count: 20000000 - Partitions: 500 - Inner Join Test Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 20000000 - Partitions: 500 - Inner Join Test TimeABC510152025SE +/- 0.21, N = 3SE +/- 0.18, N = 420.0419.7721.21

Apache Spark

Row Count: 20000000 - Partitions: 500 - Broadcast Inner Join Test Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 20000000 - Partitions: 500 - Broadcast Inner Join Test TimeABC510152025SE +/- 0.11, N = 3SE +/- 0.22, N = 420.3520.0519.94

Apache Spark

Row Count: 40000000 - Partitions: 100 - SHA-512 Benchmark Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 40000000 - Partitions: 100 - SHA-512 Benchmark TimeABC1122334455SE +/- 0.27, N = 3SE +/- 0.22, N = 349.4149.2849.53

Apache Spark

Row Count: 40000000 - Partitions: 100 - Calculate Pi Benchmark

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 40000000 - Partitions: 100 - Calculate Pi BenchmarkABC306090120150SE +/- 0.58, N = 3SE +/- 0.80, N = 3149.23148.60149.61

Apache Spark

Row Count: 40000000 - Partitions: 100 - Calculate Pi Benchmark Using Dataframe

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 40000000 - Partitions: 100 - Calculate Pi Benchmark Using DataframeABC246810SE +/- 0.02, N = 3SE +/- 0.01, N = 38.638.638.65

Apache Spark

Row Count: 40000000 - Partitions: 100 - Group By Test Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 40000000 - Partitions: 100 - Group By Test TimeABC612182430SE +/- 0.13, N = 3SE +/- 0.17, N = 327.3627.2827.38

Apache Spark

Row Count: 40000000 - Partitions: 100 - Repartition Test Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 40000000 - Partitions: 100 - Repartition Test TimeABC816243240SE +/- 0.38, N = 3SE +/- 0.20, N = 333.9334.4134.74

Apache Spark

Row Count: 40000000 - Partitions: 100 - Inner Join Test Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 40000000 - Partitions: 100 - Inner Join Test TimeABC1020304050SE +/- 0.25, N = 3SE +/- 0.42, N = 340.0039.8741.98

Apache Spark

Row Count: 40000000 - Partitions: 100 - Broadcast Inner Join Test Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 40000000 - Partitions: 100 - Broadcast Inner Join Test TimeABC918273645SE +/- 0.51, N = 3SE +/- 0.23, N = 340.7840.1439.62

Apache Spark

Row Count: 40000000 - Partitions: 500 - SHA-512 Benchmark Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 40000000 - Partitions: 500 - SHA-512 Benchmark TimeABC1122334455SE +/- 0.32, N = 3SE +/- 0.23, N = 350.1349.8648.84

Apache Spark

Row Count: 40000000 - Partitions: 500 - Calculate Pi Benchmark

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 40000000 - Partitions: 500 - Calculate Pi BenchmarkABC306090120150SE +/- 2.16, N = 3SE +/- 0.46, N = 3151.33149.13147.72

Apache Spark

Row Count: 40000000 - Partitions: 500 - Calculate Pi Benchmark Using Dataframe

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 40000000 - Partitions: 500 - Calculate Pi Benchmark Using DataframeABC246810SE +/- 0.03, N = 3SE +/- 0.05, N = 38.598.618.58

Apache Spark

Row Count: 40000000 - Partitions: 500 - Group By Test Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 40000000 - Partitions: 500 - Group By Test TimeABC612182430SE +/- 0.11, N = 3SE +/- 0.15, N = 323.9723.4923.04

Apache Spark

Row Count: 40000000 - Partitions: 500 - Repartition Test Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 40000000 - Partitions: 500 - Repartition Test TimeABC816243240SE +/- 0.37, N = 3SE +/- 0.28, N = 333.5533.8534.11

Apache Spark

Row Count: 40000000 - Partitions: 500 - Inner Join Test Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 40000000 - Partitions: 500 - Inner Join Test TimeABC918273645SE +/- 0.60, N = 3SE +/- 0.14, N = 339.5540.1440.17

Apache Spark

Row Count: 40000000 - Partitions: 500 - Broadcast Inner Join Test Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 40000000 - Partitions: 500 - Broadcast Inner Join Test TimeABC918273645SE +/- 0.39, N = 3SE +/- 0.53, N = 340.3240.7640.29

Apache Spark

Row Count: 10000000 - Partitions: 1000 - SHA-512 Benchmark Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 10000000 - Partitions: 1000 - SHA-512 Benchmark TimeABC48121620SE +/- 0.09, N = 3SE +/- 0.10, N = 313.9713.8914.07

Apache Spark

Row Count: 10000000 - Partitions: 1000 - Calculate Pi Benchmark

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 10000000 - Partitions: 1000 - Calculate Pi BenchmarkABC306090120150SE +/- 0.48, N = 3SE +/- 0.82, N = 3148.55148.53148.99

Apache Spark

Row Count: 10000000 - Partitions: 1000 - Calculate Pi Benchmark Using Dataframe

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 10000000 - Partitions: 1000 - Calculate Pi Benchmark Using DataframeABC246810SE +/- 0.03, N = 3SE +/- 0.01, N = 38.498.548.61

Apache Spark

Row Count: 10000000 - Partitions: 1000 - Group By Test Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 10000000 - Partitions: 1000 - Group By Test TimeABC246810SE +/- 0.05, N = 3SE +/- 0.09, N = 36.896.776.86

Apache Spark

Row Count: 10000000 - Partitions: 1000 - Repartition Test Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 10000000 - Partitions: 1000 - Repartition Test TimeABC3691215SE +/- 0.08, N = 3SE +/- 0.12, N = 39.059.018.90

Apache Spark

Row Count: 10000000 - Partitions: 1000 - Inner Join Test Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 10000000 - Partitions: 1000 - Inner Join Test TimeABC3691215SE +/- 0.41, N = 3SE +/- 0.17, N = 310.7911.0210.51

Apache Spark

Row Count: 10000000 - Partitions: 1000 - Broadcast Inner Join Test Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 10000000 - Partitions: 1000 - Broadcast Inner Join Test TimeABC3691215SE +/- 0.43, N = 3SE +/- 0.41, N = 310.6510.6010.08

Apache Spark

Row Count: 10000000 - Partitions: 2000 - SHA-512 Benchmark Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 10000000 - Partitions: 2000 - SHA-512 Benchmark TimeABC48121620SE +/- 0.06, N = 3SE +/- 0.10, N = 314.1514.1214.14

Apache Spark

Row Count: 10000000 - Partitions: 2000 - Calculate Pi Benchmark

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 10000000 - Partitions: 2000 - Calculate Pi BenchmarkABC306090120150SE +/- 0.22, N = 3SE +/- 0.57, N = 3148.55149.18150.28

Apache Spark

Row Count: 10000000 - Partitions: 2000 - Calculate Pi Benchmark Using Dataframe

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 10000000 - Partitions: 2000 - Calculate Pi Benchmark Using DataframeABC246810SE +/- 0.032542604, N = 3SE +/- 0.034828757, N = 38.5800000008.5700000008.687892725

Apache Spark

Row Count: 10000000 - Partitions: 2000 - Group By Test Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 10000000 - Partitions: 2000 - Group By Test TimeABC246810SE +/- 0.08, N = 3SE +/- 0.02, N = 37.037.147.04

Apache Spark

Row Count: 10000000 - Partitions: 2000 - Repartition Test Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 10000000 - Partitions: 2000 - Repartition Test TimeABC3691215SE +/- 0.07, N = 3SE +/- 0.03, N = 39.349.229.24

Apache Spark

Row Count: 10000000 - Partitions: 2000 - Inner Join Test Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 10000000 - Partitions: 2000 - Inner Join Test TimeABC3691215SE +/- 0.18, N = 3SE +/- 0.16, N = 311.2011.2111.66

Apache Spark

Row Count: 10000000 - Partitions: 2000 - Broadcast Inner Join Test Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 10000000 - Partitions: 2000 - Broadcast Inner Join Test TimeABC3691215SE +/- 0.31, N = 3SE +/- 0.15, N = 310.6010.5610.41

Apache Spark

Row Count: 20000000 - Partitions: 1000 - SHA-512 Benchmark Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 20000000 - Partitions: 1000 - SHA-512 Benchmark TimeABC612182430SE +/- 0.03, N = 3SE +/- 0.08, N = 325.5525.5325.60

Apache Spark

Row Count: 20000000 - Partitions: 1000 - Calculate Pi Benchmark

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 20000000 - Partitions: 1000 - Calculate Pi BenchmarkABC306090120150SE +/- 0.51, N = 3SE +/- 0.38, N = 3149.23147.91149.29

Apache Spark

Row Count: 20000000 - Partitions: 1000 - Calculate Pi Benchmark Using Dataframe

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 20000000 - Partitions: 1000 - Calculate Pi Benchmark Using DataframeABC246810SE +/- 0.01, N = 3SE +/- 0.03, N = 38.628.588.54

Apache Spark

Row Count: 20000000 - Partitions: 1000 - Group By Test Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 20000000 - Partitions: 1000 - Group By Test TimeABC3691215SE +/- 0.09, N = 3SE +/- 0.04, N = 310.2510.1610.03

Apache Spark

Row Count: 20000000 - Partitions: 1000 - Repartition Test Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 20000000 - Partitions: 1000 - Repartition Test TimeABC48121620SE +/- 0.08, N = 3SE +/- 0.04, N = 317.0317.0917.01

Apache Spark

Row Count: 20000000 - Partitions: 1000 - Inner Join Test Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 20000000 - Partitions: 1000 - Inner Join Test TimeABC510152025SE +/- 0.19, N = 3SE +/- 0.10, N = 319.9720.3320.49

Apache Spark

Row Count: 20000000 - Partitions: 1000 - Broadcast Inner Join Test Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 20000000 - Partitions: 1000 - Broadcast Inner Join Test TimeABC510152025SE +/- 0.23, N = 3SE +/- 0.37, N = 319.8720.6621.08

Apache Spark

Row Count: 20000000 - Partitions: 2000 - SHA-512 Benchmark Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 20000000 - Partitions: 2000 - SHA-512 Benchmark TimeABC612182430SE +/- 0.02, N = 3SE +/- 0.02, N = 325.7025.7625.72

Apache Spark

Row Count: 20000000 - Partitions: 2000 - Calculate Pi Benchmark

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 20000000 - Partitions: 2000 - Calculate Pi BenchmarkABC306090120150SE +/- 0.53, N = 3SE +/- 0.23, N = 3148.85148.30149.85

Apache Spark

Row Count: 20000000 - Partitions: 2000 - Calculate Pi Benchmark Using Dataframe

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 20000000 - Partitions: 2000 - Calculate Pi Benchmark Using DataframeABC246810SE +/- 0.02, N = 3SE +/- 0.04, N = 38.648.608.59

Apache Spark

Row Count: 20000000 - Partitions: 2000 - Group By Test Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 20000000 - Partitions: 2000 - Group By Test TimeABC3691215SE +/- 0.08, N = 3SE +/- 0.04, N = 310.2510.3210.41

Apache Spark

Row Count: 20000000 - Partitions: 2000 - Repartition Test Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 20000000 - Partitions: 2000 - Repartition Test TimeABC48121620SE +/- 0.23, N = 3SE +/- 0.29, N = 317.5217.8117.44

Apache Spark

Row Count: 20000000 - Partitions: 2000 - Inner Join Test Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 20000000 - Partitions: 2000 - Inner Join Test TimeABC510152025SE +/- 0.36, N = 3SE +/- 0.17, N = 320.7920.7220.50

Apache Spark

Row Count: 20000000 - Partitions: 2000 - Broadcast Inner Join Test Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 20000000 - Partitions: 2000 - Broadcast Inner Join Test TimeABC510152025SE +/- 0.31, N = 3SE +/- 0.18, N = 320.0119.8719.31

Apache Spark

Row Count: 40000000 - Partitions: 1000 - SHA-512 Benchmark Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 40000000 - Partitions: 1000 - SHA-512 Benchmark TimeABC1122334455SE +/- 0.04, N = 3SE +/- 0.28, N = 349.6349.9549.84

Apache Spark

Row Count: 40000000 - Partitions: 1000 - Calculate Pi Benchmark

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 40000000 - Partitions: 1000 - Calculate Pi BenchmarkABC306090120150SE +/- 0.40, N = 3SE +/- 0.29, N = 3149.45148.66151.94

Apache Spark

Row Count: 40000000 - Partitions: 1000 - Calculate Pi Benchmark Using Dataframe

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 40000000 - Partitions: 1000 - Calculate Pi Benchmark Using DataframeABC246810SE +/- 0.02, N = 3SE +/- 0.03, N = 38.618.628.56

Apache Spark

Row Count: 40000000 - Partitions: 1000 - Group By Test Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 40000000 - Partitions: 1000 - Group By Test TimeABC612182430SE +/- 0.20, N = 3SE +/- 0.23, N = 322.8923.2423.18

Apache Spark

Row Count: 40000000 - Partitions: 1000 - Repartition Test Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 40000000 - Partitions: 1000 - Repartition Test TimeABC816243240SE +/- 0.28, N = 3SE +/- 0.43, N = 334.5034.0934.62

Apache Spark

Row Count: 40000000 - Partitions: 1000 - Inner Join Test Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 40000000 - Partitions: 1000 - Inner Join Test TimeABC918273645SE +/- 0.57, N = 3SE +/- 0.28, N = 339.6040.5039.90

Apache Spark

Row Count: 40000000 - Partitions: 1000 - Broadcast Inner Join Test Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 40000000 - Partitions: 1000 - Broadcast Inner Join Test TimeABC918273645SE +/- 0.09, N = 3SE +/- 1.12, N = 340.9941.0141.20

Apache Spark

Row Count: 40000000 - Partitions: 2000 - SHA-512 Benchmark Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 40000000 - Partitions: 2000 - SHA-512 Benchmark TimeABC1224364860SE +/- 0.17, N = 3SE +/- 0.68, N = 350.3351.3550.12

Apache Spark

Row Count: 40000000 - Partitions: 2000 - Calculate Pi Benchmark

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 40000000 - Partitions: 2000 - Calculate Pi BenchmarkABC306090120150SE +/- 0.43, N = 3SE +/- 0.45, N = 3148.60148.50147.98

Apache Spark

Row Count: 40000000 - Partitions: 2000 - Calculate Pi Benchmark Using Dataframe

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 40000000 - Partitions: 2000 - Calculate Pi Benchmark Using DataframeABC246810SE +/- 0.03, N = 3SE +/- 0.04, N = 38.598.578.50

Apache Spark

Row Count: 40000000 - Partitions: 2000 - Group By Test Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 40000000 - Partitions: 2000 - Group By Test TimeABC612182430SE +/- 0.21, N = 3SE +/- 0.07, N = 323.2622.9423.17

Apache Spark

Row Count: 40000000 - Partitions: 2000 - Repartition Test Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 40000000 - Partitions: 2000 - Repartition Test TimeABC816243240SE +/- 0.21, N = 3SE +/- 0.42, N = 334.5633.9833.96

Apache Spark

Row Count: 40000000 - Partitions: 2000 - Inner Join Test Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 40000000 - Partitions: 2000 - Inner Join Test TimeABC918273645SE +/- 0.65, N = 3SE +/- 0.48, N = 340.6039.5839.85

Apache Spark

Row Count: 40000000 - Partitions: 2000 - Broadcast Inner Join Test Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 40000000 - Partitions: 2000 - Broadcast Inner Join Test TimeABC918273645SE +/- 0.68, N = 3SE +/- 0.38, N = 341.5540.7141.18


Phoronix Test Suite v10.8.4