spark rembrandty

AMD Ryzen 7 PRO 6850U testing with a LENOVO 21CM0001US (R22ET46W 1.16 BIOS) and AMD Rembrandt 1GB on Ubuntu 22.04 via the Phoronix Test Suite.

HTML result view exported from: https://openbenchmarking.org/result/2208040-NE-SPARKREMB86&sor&grr.

spark rembrandtyProcessorMotherboardChipsetMemoryDiskGraphicsAudioNetworkOSKernelDesktopDisplay ServerOpenGLVulkanCompilerFile-SystemScreen ResolutionABAMD Ryzen 7 PRO 6850U @ 4.77GHz (8 Cores / 16 Threads)LENOVO 21CM0001US (R22ET46W 1.16 BIOS)AMD Device 14b516GB512GB Micron MTFDKBA512TFKAMD Rembrandt 1GBAMD Rembrandt Radeon HD AudioQualcomm Atheros QCNFA765Ubuntu 22.045.15.0-41-generic (x86_64)GNOME Shell 42.2X Server + Wayland4.6 Mesa 22.2.0-devel (git-ba461f8 2022-07-26 jammy-oibaf-ppa) (LLVM 14.0.6 DRM 3.42)1.3.219GCC 11.2.0ext41920x1200OpenBenchmarking.orgKernel Details- Transparent Huge Pages: madviseProcessor Details- Scaling Governor: amd-pstate schedutil (Boost: Enabled) - Platform Profile: balanced - CPU Microcode: 0xa404102 - ACPI Profile: balanced Java Details- OpenJDK Runtime Environment (build 11.0.15+10-Ubuntu-0ubuntu0.22.04.1)Python Details- Python 3.10.4Security Details- itlb_multihit: Not affected + l1tf: Not affected + mds: Not affected + meltdown: Not affected + mmio_stale_data: Not affected + spec_store_bypass: Mitigation of SSB disabled via prctl and seccomp + spectre_v1: Mitigation of usercopy/swapgs barriers and __user pointer sanitization + spectre_v2: Mitigation of Retpolines IBPB: conditional IBRS_FW STIBP: always-on RSB filling + srbds: Not affected + tsx_async_abort: Not affected

spark rembrandtyspark: 1000000 - 100 - Broadcast Inner Join Test Timespark: 1000000 - 100 - Inner Join Test Timespark: 1000000 - 100 - Repartition Test Timespark: 1000000 - 100 - Group By Test Timespark: 1000000 - 100 - Calculate Pi Benchmark Using Dataframespark: 1000000 - 100 - Calculate Pi Benchmarkspark: 1000000 - 100 - SHA-512 Benchmark Timespark: 40000000 - 100 - Broadcast Inner Join Test Timespark: 40000000 - 100 - Inner Join Test Timespark: 40000000 - 100 - Repartition Test Timespark: 40000000 - 100 - Group By Test Timespark: 40000000 - 100 - Calculate Pi Benchmark Using Dataframespark: 40000000 - 100 - Calculate Pi Benchmarkspark: 40000000 - 100 - SHA-512 Benchmark Timespark: 40000000 - 2000 - Broadcast Inner Join Test Timespark: 40000000 - 2000 - Inner Join Test Timespark: 40000000 - 2000 - Repartition Test Timespark: 40000000 - 2000 - Group By Test Timespark: 40000000 - 2000 - Calculate Pi Benchmark Using Dataframespark: 40000000 - 2000 - Calculate Pi Benchmarkspark: 40000000 - 2000 - SHA-512 Benchmark Timespark: 40000000 - 500 - Broadcast Inner Join Test Timespark: 40000000 - 500 - Inner Join Test Timespark: 40000000 - 500 - Repartition Test Timespark: 40000000 - 500 - Group By Test Timespark: 40000000 - 500 - Calculate Pi Benchmark Using Dataframespark: 40000000 - 500 - Calculate Pi Benchmarkspark: 40000000 - 500 - SHA-512 Benchmark Timespark: 40000000 - 1000 - Broadcast Inner Join Test Timespark: 40000000 - 1000 - Inner Join Test Timespark: 40000000 - 1000 - Repartition Test Timespark: 40000000 - 1000 - Group By Test Timespark: 40000000 - 1000 - Calculate Pi Benchmark Using Dataframespark: 40000000 - 1000 - Calculate Pi Benchmarkspark: 40000000 - 1000 - SHA-512 Benchmark Timespark: 1000000 - 2000 - Broadcast Inner Join Test Timespark: 1000000 - 2000 - Inner Join Test Timespark: 1000000 - 2000 - Repartition Test Timespark: 1000000 - 2000 - Group By Test Timespark: 1000000 - 2000 - Calculate Pi Benchmark Using Dataframespark: 1000000 - 2000 - Calculate Pi Benchmarkspark: 1000000 - 2000 - SHA-512 Benchmark Timespark: 20000000 - 100 - Broadcast Inner Join Test Timespark: 20000000 - 100 - Inner Join Test Timespark: 20000000 - 100 - Repartition Test Timespark: 20000000 - 100 - Group By Test Timespark: 20000000 - 100 - Calculate Pi Benchmark Using Dataframespark: 20000000 - 100 - Calculate Pi Benchmarkspark: 20000000 - 100 - SHA-512 Benchmark Timespark: 20000000 - 1000 - Broadcast Inner Join Test Timespark: 20000000 - 1000 - Inner Join Test Timespark: 20000000 - 1000 - Repartition Test Timespark: 20000000 - 1000 - Group By Test Timespark: 20000000 - 1000 - Calculate Pi Benchmark Using Dataframespark: 20000000 - 1000 - Calculate Pi Benchmarkspark: 20000000 - 1000 - SHA-512 Benchmark Timespark: 20000000 - 2000 - Broadcast Inner Join Test Timespark: 20000000 - 2000 - Inner Join Test Timespark: 20000000 - 2000 - Repartition Test Timespark: 20000000 - 2000 - Group By Test Timespark: 20000000 - 2000 - Calculate Pi Benchmark Using Dataframespark: 20000000 - 2000 - Calculate Pi Benchmarkspark: 20000000 - 2000 - SHA-512 Benchmark Timespark: 20000000 - 500 - Broadcast Inner Join Test Timespark: 20000000 - 500 - Inner Join Test Timespark: 20000000 - 500 - Repartition Test Timespark: 20000000 - 500 - Group By Test Timespark: 20000000 - 500 - Calculate Pi Benchmark Using Dataframespark: 20000000 - 500 - Calculate Pi Benchmarkspark: 20000000 - 500 - SHA-512 Benchmark Timespark: 10000000 - 2000 - Broadcast Inner Join Test Timespark: 10000000 - 2000 - Inner Join Test Timespark: 10000000 - 2000 - Repartition Test Timespark: 10000000 - 2000 - Group By Test Timespark: 10000000 - 2000 - Calculate Pi Benchmark Using Dataframespark: 10000000 - 2000 - Calculate Pi Benchmarkspark: 10000000 - 2000 - SHA-512 Benchmark Timespark: 10000000 - 1000 - Broadcast Inner Join Test Timespark: 10000000 - 1000 - Inner Join Test Timespark: 10000000 - 1000 - Repartition Test Timespark: 10000000 - 1000 - Group By Test Timespark: 10000000 - 1000 - Calculate Pi Benchmark Using Dataframespark: 10000000 - 1000 - Calculate Pi Benchmarkspark: 10000000 - 1000 - SHA-512 Benchmark Timespark: 10000000 - 100 - Broadcast Inner Join Test Timespark: 10000000 - 100 - Inner Join Test Timespark: 10000000 - 100 - Repartition Test Timespark: 10000000 - 100 - Group By Test Timespark: 10000000 - 100 - Calculate Pi Benchmark Using Dataframespark: 10000000 - 100 - Calculate Pi Benchmarkspark: 10000000 - 100 - SHA-512 Benchmark Timespark: 10000000 - 500 - Broadcast Inner Join Test Timespark: 10000000 - 500 - Inner Join Test Timespark: 10000000 - 500 - Repartition Test Timespark: 10000000 - 500 - Group By Test Timespark: 10000000 - 500 - Calculate Pi Benchmark Using Dataframespark: 10000000 - 500 - Calculate Pi Benchmarkspark: 10000000 - 500 - SHA-512 Benchmark Timespark: 1000000 - 1000 - Broadcast Inner Join Test Timespark: 1000000 - 1000 - Inner Join Test Timespark: 1000000 - 1000 - Repartition Test Timespark: 1000000 - 1000 - Group By Test Timespark: 1000000 - 1000 - Calculate Pi Benchmark Using Dataframespark: 1000000 - 1000 - Calculate Pi Benchmarkspark: 1000000 - 1000 - SHA-512 Benchmark Timespark: 1000000 - 500 - Broadcast Inner Join Test Timespark: 1000000 - 500 - Inner Join Test Timespark: 1000000 - 500 - Repartition Test Timespark: 1000000 - 500 - Group By Test Timespark: 1000000 - 500 - Calculate Pi Benchmark Using Dataframespark: 1000000 - 500 - Calculate Pi Benchmarkspark: 1000000 - 500 - SHA-512 Benchmark TimeAB2.142.443.374.5713.62241.054.5460.3061.5952.3940.6013.83249.7469.0661.8561.6150.8134.7113.83246.5470.4464.3361.3550.6835.3513.66249.4469.5159.4159.2550.0533.9513.58247.4069.983.314.014.156.0913.59241.475.6332.1032.4827.4515.3713.51239.8436.5228.7730.1525.6514.9513.60241.0035.7329.4430.3626.4615.7813.58242.0936.2828.6430.2726.2714.9213.50241.4035.0015.6316.8014.4410.9413.54241.9420.4014.9215.5813.9610.4413.56241.8219.9916.2216.7114.4510.0013.60238.6519.5815.0515.1114.079.8613.56239.0519.703.213.173.685.3213.50239.975.113.082.653.404.9213.66240.044.751.892.393.244.4813.49241.0241388544.6058.2556.6849.8240.4313.70240.9966.0361.7961.5550.2434.9113.73250.1570.4358.9257.4349.2035.6413.43244.7468.0859.0659.7450.2734.8413.65246.2869.513.043.934.096.1013.54242.205.4631.8231.1327.0214.9013.67240.0336.4830.2930.5526.9915.5813.53244.0136.0329.5630.8326.5415.7913.48243.7736.3628.3029.1225.5514.8413.45239.4034.9815.3416.1714.1710.5013.60241.1120.1115.0315.6714.1210.6513.58239.8420.0515.1315.7614.639.7213.54240.0119.6215.1715.4113.9410.0213.47240.2519.792.623.173.705.5513.46239.655.082.292.533.435.0313.44245.154.74OpenBenchmarking.org

Apache Spark

Row Count: 1000000 - Partitions: 100 - Broadcast Inner Join Test Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 1000000 - Partitions: 100 - Broadcast Inner Join Test TimeBA0.48150.9631.44451.9262.4075SE +/- 0.08, N = 91.892.14

Apache Spark

Row Count: 1000000 - Partitions: 100 - Inner Join Test Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 1000000 - Partitions: 100 - Inner Join Test TimeBA0.5491.0981.6472.1962.745SE +/- 0.02, N = 92.392.44

Apache Spark

Row Count: 1000000 - Partitions: 100 - Repartition Test Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 1000000 - Partitions: 100 - Repartition Test TimeBA0.75831.51662.27493.03323.7915SE +/- 0.03, N = 93.243.37

Apache Spark

Row Count: 1000000 - Partitions: 100 - Group By Test Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 1000000 - Partitions: 100 - Group By Test TimeBA1.02832.05663.08494.11325.1415SE +/- 0.05, N = 94.484.57

Apache Spark

Row Count: 1000000 - Partitions: 100 - Calculate Pi Benchmark Using Dataframe

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 1000000 - Partitions: 100 - Calculate Pi Benchmark Using DataframeBA48121620SE +/- 0.03, N = 913.4913.62

Apache Spark

Row Count: 1000000 - Partitions: 100 - Calculate Pi Benchmark

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 1000000 - Partitions: 100 - Calculate Pi BenchmarkBA50100150200250SE +/- 0.82, N = 9241.02241.05

Apache Spark

Row Count: 1000000 - Partitions: 100 - SHA-512 Benchmark Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 1000000 - Partitions: 100 - SHA-512 Benchmark TimeAB1.0352.073.1054.145.175SE +/- 0.07, N = 94.544.60

Apache Spark

Row Count: 40000000 - Partitions: 100 - Broadcast Inner Join Test Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 40000000 - Partitions: 100 - Broadcast Inner Join Test TimeBA1326395265SE +/- 1.43, N = 358.2560.30

Apache Spark

Row Count: 40000000 - Partitions: 100 - Inner Join Test Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 40000000 - Partitions: 100 - Inner Join Test TimeBA1428425670SE +/- 1.79, N = 356.6861.59

Apache Spark

Row Count: 40000000 - Partitions: 100 - Repartition Test Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 40000000 - Partitions: 100 - Repartition Test TimeBA1224364860SE +/- 1.35, N = 349.8252.39

Apache Spark

Row Count: 40000000 - Partitions: 100 - Group By Test Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 40000000 - Partitions: 100 - Group By Test TimeBA918273645SE +/- 0.27, N = 340.4340.60

Apache Spark

Row Count: 40000000 - Partitions: 100 - Calculate Pi Benchmark Using Dataframe

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 40000000 - Partitions: 100 - Calculate Pi Benchmark Using DataframeBA48121620SE +/- 0.05, N = 313.7013.83

Apache Spark

Row Count: 40000000 - Partitions: 100 - Calculate Pi Benchmark

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 40000000 - Partitions: 100 - Calculate Pi BenchmarkBA50100150200250SE +/- 0.92, N = 3240.99249.74

Apache Spark

Row Count: 40000000 - Partitions: 100 - SHA-512 Benchmark Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 40000000 - Partitions: 100 - SHA-512 Benchmark TimeBA1530456075SE +/- 0.74, N = 366.0369.06

Apache Spark

Row Count: 40000000 - Partitions: 2000 - Broadcast Inner Join Test Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 40000000 - Partitions: 2000 - Broadcast Inner Join Test TimeBA1428425670SE +/- 0.87, N = 361.7961.85

Apache Spark

Row Count: 40000000 - Partitions: 2000 - Inner Join Test Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 40000000 - Partitions: 2000 - Inner Join Test TimeBA1428425670SE +/- 0.47, N = 361.5561.61

Apache Spark

Row Count: 40000000 - Partitions: 2000 - Repartition Test Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 40000000 - Partitions: 2000 - Repartition Test TimeBA1122334455SE +/- 0.38, N = 350.2450.81

Apache Spark

Row Count: 40000000 - Partitions: 2000 - Group By Test Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 40000000 - Partitions: 2000 - Group By Test TimeAB816243240SE +/- 0.44, N = 334.7134.91

Apache Spark

Row Count: 40000000 - Partitions: 2000 - Calculate Pi Benchmark Using Dataframe

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 40000000 - Partitions: 2000 - Calculate Pi Benchmark Using DataframeBA48121620SE +/- 0.01, N = 313.7313.83

Apache Spark

Row Count: 40000000 - Partitions: 2000 - Calculate Pi Benchmark

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 40000000 - Partitions: 2000 - Calculate Pi BenchmarkAB50100150200250SE +/- 0.74, N = 3246.54250.15

Apache Spark

Row Count: 40000000 - Partitions: 2000 - SHA-512 Benchmark Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 40000000 - Partitions: 2000 - SHA-512 Benchmark TimeBA1632486480SE +/- 0.14, N = 370.4370.44

Apache Spark

Row Count: 40000000 - Partitions: 500 - Broadcast Inner Join Test Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 40000000 - Partitions: 500 - Broadcast Inner Join Test TimeBA1428425670SE +/- 1.68, N = 358.9264.33

Apache Spark

Row Count: 40000000 - Partitions: 500 - Inner Join Test Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 40000000 - Partitions: 500 - Inner Join Test TimeBA1428425670SE +/- 1.61, N = 357.4361.35

Apache Spark

Row Count: 40000000 - Partitions: 500 - Repartition Test Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 40000000 - Partitions: 500 - Repartition Test TimeBA1122334455SE +/- 0.25, N = 349.2050.68

Apache Spark

Row Count: 40000000 - Partitions: 500 - Group By Test Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 40000000 - Partitions: 500 - Group By Test TimeAB816243240SE +/- 0.43, N = 335.3535.64

Apache Spark

Row Count: 40000000 - Partitions: 500 - Calculate Pi Benchmark Using Dataframe

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 40000000 - Partitions: 500 - Calculate Pi Benchmark Using DataframeBA48121620SE +/- 0.06, N = 313.4313.66

Apache Spark

Row Count: 40000000 - Partitions: 500 - Calculate Pi Benchmark

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 40000000 - Partitions: 500 - Calculate Pi BenchmarkBA50100150200250SE +/- 3.48, N = 3244.74249.44

Apache Spark

Row Count: 40000000 - Partitions: 500 - SHA-512 Benchmark Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 40000000 - Partitions: 500 - SHA-512 Benchmark TimeBA1530456075SE +/- 0.35, N = 368.0869.51

Apache Spark

Row Count: 40000000 - Partitions: 1000 - Broadcast Inner Join Test Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 40000000 - Partitions: 1000 - Broadcast Inner Join Test TimeBA1326395265SE +/- 0.72, N = 359.0659.41

Apache Spark

Row Count: 40000000 - Partitions: 1000 - Inner Join Test Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 40000000 - Partitions: 1000 - Inner Join Test TimeAB1326395265SE +/- 0.85, N = 359.2559.74

Apache Spark

Row Count: 40000000 - Partitions: 1000 - Repartition Test Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 40000000 - Partitions: 1000 - Repartition Test TimeAB1122334455SE +/- 0.62, N = 350.0550.27

Apache Spark

Row Count: 40000000 - Partitions: 1000 - Group By Test Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 40000000 - Partitions: 1000 - Group By Test TimeAB816243240SE +/- 0.28, N = 333.9534.84

Apache Spark

Row Count: 40000000 - Partitions: 1000 - Calculate Pi Benchmark Using Dataframe

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 40000000 - Partitions: 1000 - Calculate Pi Benchmark Using DataframeAB48121620SE +/- 0.04, N = 313.5813.65

Apache Spark

Row Count: 40000000 - Partitions: 1000 - Calculate Pi Benchmark

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 40000000 - Partitions: 1000 - Calculate Pi BenchmarkBA50100150200250SE +/- 0.40, N = 3246.28247.40

Apache Spark

Row Count: 40000000 - Partitions: 1000 - SHA-512 Benchmark Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 40000000 - Partitions: 1000 - SHA-512 Benchmark TimeBA1632486480SE +/- 0.51, N = 369.5169.98

Apache Spark

Row Count: 1000000 - Partitions: 2000 - Broadcast Inner Join Test Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 1000000 - Partitions: 2000 - Broadcast Inner Join Test TimeBA0.74481.48962.23442.97923.724SE +/- 0.10, N = 53.043.31

Apache Spark

Row Count: 1000000 - Partitions: 2000 - Inner Join Test Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 1000000 - Partitions: 2000 - Inner Join Test TimeBA0.90231.80462.70693.60924.5115SE +/- 0.04, N = 53.934.01

Apache Spark

Row Count: 1000000 - Partitions: 2000 - Repartition Test Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 1000000 - Partitions: 2000 - Repartition Test TimeBA0.93381.86762.80143.73524.669SE +/- 0.04, N = 54.094.15

Apache Spark

Row Count: 1000000 - Partitions: 2000 - Group By Test Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 1000000 - Partitions: 2000 - Group By Test TimeAB246810SE +/- 0.07, N = 56.096.10

Apache Spark

Row Count: 1000000 - Partitions: 2000 - Calculate Pi Benchmark Using Dataframe

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 1000000 - Partitions: 2000 - Calculate Pi Benchmark Using DataframeBA3691215SE +/- 0.05, N = 513.5413.59

Apache Spark

Row Count: 1000000 - Partitions: 2000 - Calculate Pi Benchmark

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 1000000 - Partitions: 2000 - Calculate Pi BenchmarkAB50100150200250SE +/- 0.77, N = 5241.47242.20

Apache Spark

Row Count: 1000000 - Partitions: 2000 - SHA-512 Benchmark Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 1000000 - Partitions: 2000 - SHA-512 Benchmark TimeBA1.26682.53363.80045.06726.334SE +/- 0.06, N = 55.465.63

Apache Spark

Row Count: 20000000 - Partitions: 100 - Broadcast Inner Join Test Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 20000000 - Partitions: 100 - Broadcast Inner Join Test TimeBA714212835SE +/- 0.30, N = 331.8232.10

Apache Spark

Row Count: 20000000 - Partitions: 100 - Inner Join Test Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 20000000 - Partitions: 100 - Inner Join Test TimeBA816243240SE +/- 0.13, N = 331.1332.48

Apache Spark

Row Count: 20000000 - Partitions: 100 - Repartition Test Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 20000000 - Partitions: 100 - Repartition Test TimeBA612182430SE +/- 0.14, N = 327.0227.45

Apache Spark

Row Count: 20000000 - Partitions: 100 - Group By Test Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 20000000 - Partitions: 100 - Group By Test TimeBA48121620SE +/- 0.21, N = 314.9015.37

Apache Spark

Row Count: 20000000 - Partitions: 100 - Calculate Pi Benchmark Using Dataframe

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 20000000 - Partitions: 100 - Calculate Pi Benchmark Using DataframeAB48121620SE +/- 0.02, N = 313.5113.67

Apache Spark

Row Count: 20000000 - Partitions: 100 - Calculate Pi Benchmark

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 20000000 - Partitions: 100 - Calculate Pi BenchmarkAB50100150200250SE +/- 0.71, N = 3239.84240.03

Apache Spark

Row Count: 20000000 - Partitions: 100 - SHA-512 Benchmark Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 20000000 - Partitions: 100 - SHA-512 Benchmark TimeBA816243240SE +/- 0.20, N = 336.4836.52

Apache Spark

Row Count: 20000000 - Partitions: 1000 - Broadcast Inner Join Test Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 20000000 - Partitions: 1000 - Broadcast Inner Join Test TimeAB714212835SE +/- 0.11, N = 328.7730.29

Apache Spark

Row Count: 20000000 - Partitions: 1000 - Inner Join Test Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 20000000 - Partitions: 1000 - Inner Join Test TimeAB714212835SE +/- 0.42, N = 330.1530.55

Apache Spark

Row Count: 20000000 - Partitions: 1000 - Repartition Test Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 20000000 - Partitions: 1000 - Repartition Test TimeAB612182430SE +/- 0.12, N = 325.6526.99

Apache Spark

Row Count: 20000000 - Partitions: 1000 - Group By Test Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 20000000 - Partitions: 1000 - Group By Test TimeAB48121620SE +/- 0.07, N = 314.9515.58

Apache Spark

Row Count: 20000000 - Partitions: 1000 - Calculate Pi Benchmark Using Dataframe

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 20000000 - Partitions: 1000 - Calculate Pi Benchmark Using DataframeBA3691215SE +/- 0.01, N = 313.5313.60

Apache Spark

Row Count: 20000000 - Partitions: 1000 - Calculate Pi Benchmark

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 20000000 - Partitions: 1000 - Calculate Pi BenchmarkAB50100150200250SE +/- 0.76, N = 3241.00244.01

Apache Spark

Row Count: 20000000 - Partitions: 1000 - SHA-512 Benchmark Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 20000000 - Partitions: 1000 - SHA-512 Benchmark TimeAB816243240SE +/- 0.44, N = 335.7336.03

Apache Spark

Row Count: 20000000 - Partitions: 2000 - Broadcast Inner Join Test Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 20000000 - Partitions: 2000 - Broadcast Inner Join Test TimeAB714212835SE +/- 0.18, N = 329.4429.56

Apache Spark

Row Count: 20000000 - Partitions: 2000 - Inner Join Test Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 20000000 - Partitions: 2000 - Inner Join Test TimeAB714212835SE +/- 0.07, N = 330.3630.83

Apache Spark

Row Count: 20000000 - Partitions: 2000 - Repartition Test Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 20000000 - Partitions: 2000 - Repartition Test TimeAB612182430SE +/- 0.48, N = 326.4626.54

Apache Spark

Row Count: 20000000 - Partitions: 2000 - Group By Test Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 20000000 - Partitions: 2000 - Group By Test TimeAB48121620SE +/- 0.19, N = 315.7815.79

Apache Spark

Row Count: 20000000 - Partitions: 2000 - Calculate Pi Benchmark Using Dataframe

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 20000000 - Partitions: 2000 - Calculate Pi Benchmark Using DataframeBA3691215SE +/- 0.04, N = 313.4813.58

Apache Spark

Row Count: 20000000 - Partitions: 2000 - Calculate Pi Benchmark

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 20000000 - Partitions: 2000 - Calculate Pi BenchmarkAB50100150200250SE +/- 1.02, N = 3242.09243.77

Apache Spark

Row Count: 20000000 - Partitions: 2000 - SHA-512 Benchmark Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 20000000 - Partitions: 2000 - SHA-512 Benchmark TimeAB816243240SE +/- 0.22, N = 336.2836.36

Apache Spark

Row Count: 20000000 - Partitions: 500 - Broadcast Inner Join Test Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 20000000 - Partitions: 500 - Broadcast Inner Join Test TimeBA714212835SE +/- 0.25, N = 328.3028.64

Apache Spark

Row Count: 20000000 - Partitions: 500 - Inner Join Test Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 20000000 - Partitions: 500 - Inner Join Test TimeBA714212835SE +/- 0.94, N = 329.1230.27

Apache Spark

Row Count: 20000000 - Partitions: 500 - Repartition Test Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 20000000 - Partitions: 500 - Repartition Test TimeBA612182430SE +/- 0.44, N = 325.5526.27

Apache Spark

Row Count: 20000000 - Partitions: 500 - Group By Test Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 20000000 - Partitions: 500 - Group By Test TimeBA48121620SE +/- 0.10, N = 314.8414.92

Apache Spark

Row Count: 20000000 - Partitions: 500 - Calculate Pi Benchmark Using Dataframe

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 20000000 - Partitions: 500 - Calculate Pi Benchmark Using DataframeBA3691215SE +/- 0.06, N = 313.4513.50

Apache Spark

Row Count: 20000000 - Partitions: 500 - Calculate Pi Benchmark

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 20000000 - Partitions: 500 - Calculate Pi BenchmarkBA50100150200250SE +/- 0.69, N = 3239.40241.40

Apache Spark

Row Count: 20000000 - Partitions: 500 - SHA-512 Benchmark Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 20000000 - Partitions: 500 - SHA-512 Benchmark TimeBA816243240SE +/- 0.18, N = 334.9835.00

Apache Spark

Row Count: 10000000 - Partitions: 2000 - Broadcast Inner Join Test Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 10000000 - Partitions: 2000 - Broadcast Inner Join Test TimeBA48121620SE +/- 0.11, N = 315.3415.63

Apache Spark

Row Count: 10000000 - Partitions: 2000 - Inner Join Test Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 10000000 - Partitions: 2000 - Inner Join Test TimeBA48121620SE +/- 0.16, N = 316.1716.80

Apache Spark

Row Count: 10000000 - Partitions: 2000 - Repartition Test Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 10000000 - Partitions: 2000 - Repartition Test TimeBA48121620SE +/- 0.02, N = 314.1714.44

Apache Spark

Row Count: 10000000 - Partitions: 2000 - Group By Test Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 10000000 - Partitions: 2000 - Group By Test TimeBA3691215SE +/- 0.13, N = 310.5010.94

Apache Spark

Row Count: 10000000 - Partitions: 2000 - Calculate Pi Benchmark Using Dataframe

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 10000000 - Partitions: 2000 - Calculate Pi Benchmark Using DataframeAB3691215SE +/- 0.04, N = 313.5413.60

Apache Spark

Row Count: 10000000 - Partitions: 2000 - Calculate Pi Benchmark

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 10000000 - Partitions: 2000 - Calculate Pi BenchmarkBA50100150200250SE +/- 0.88, N = 3241.11241.94

Apache Spark

Row Count: 10000000 - Partitions: 2000 - SHA-512 Benchmark Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 10000000 - Partitions: 2000 - SHA-512 Benchmark TimeBA510152025SE +/- 0.08, N = 320.1120.40

Apache Spark

Row Count: 10000000 - Partitions: 1000 - Broadcast Inner Join Test Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 10000000 - Partitions: 1000 - Broadcast Inner Join Test TimeAB48121620SE +/- 0.11, N = 314.9215.03

Apache Spark

Row Count: 10000000 - Partitions: 1000 - Inner Join Test Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 10000000 - Partitions: 1000 - Inner Join Test TimeAB48121620SE +/- 0.06, N = 315.5815.67

Apache Spark

Row Count: 10000000 - Partitions: 1000 - Repartition Test Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 10000000 - Partitions: 1000 - Repartition Test TimeAB48121620SE +/- 0.07, N = 313.9614.12

Apache Spark

Row Count: 10000000 - Partitions: 1000 - Group By Test Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 10000000 - Partitions: 1000 - Group By Test TimeAB3691215SE +/- 0.12, N = 310.4410.65

Apache Spark

Row Count: 10000000 - Partitions: 1000 - Calculate Pi Benchmark Using Dataframe

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 10000000 - Partitions: 1000 - Calculate Pi Benchmark Using DataframeAB3691215SE +/- 0.03, N = 313.5613.58

Apache Spark

Row Count: 10000000 - Partitions: 1000 - Calculate Pi Benchmark

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 10000000 - Partitions: 1000 - Calculate Pi BenchmarkBA50100150200250SE +/- 1.29, N = 3239.84241.82

Apache Spark

Row Count: 10000000 - Partitions: 1000 - SHA-512 Benchmark Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 10000000 - Partitions: 1000 - SHA-512 Benchmark TimeAB510152025SE +/- 0.08, N = 319.9920.05

Apache Spark

Row Count: 10000000 - Partitions: 100 - Broadcast Inner Join Test Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 10000000 - Partitions: 100 - Broadcast Inner Join Test TimeBA48121620SE +/- 0.51, N = 315.1316.22

Apache Spark

Row Count: 10000000 - Partitions: 100 - Inner Join Test Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 10000000 - Partitions: 100 - Inner Join Test TimeBA48121620SE +/- 0.29, N = 315.7616.71

Apache Spark

Row Count: 10000000 - Partitions: 100 - Repartition Test Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 10000000 - Partitions: 100 - Repartition Test TimeAB48121620SE +/- 0.17, N = 314.4514.63

Apache Spark

Row Count: 10000000 - Partitions: 100 - Group By Test Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 10000000 - Partitions: 100 - Group By Test TimeBA3691215SE +/- 0.11, N = 39.7210.00

Apache Spark

Row Count: 10000000 - Partitions: 100 - Calculate Pi Benchmark Using Dataframe

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 10000000 - Partitions: 100 - Calculate Pi Benchmark Using DataframeBA3691215SE +/- 0.04, N = 313.5413.60

Apache Spark

Row Count: 10000000 - Partitions: 100 - Calculate Pi Benchmark

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 10000000 - Partitions: 100 - Calculate Pi BenchmarkAB50100150200250SE +/- 0.22, N = 3238.65240.01

Apache Spark

Row Count: 10000000 - Partitions: 100 - SHA-512 Benchmark Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 10000000 - Partitions: 100 - SHA-512 Benchmark TimeAB510152025SE +/- 0.10, N = 319.5819.62

Apache Spark

Row Count: 10000000 - Partitions: 500 - Broadcast Inner Join Test Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 10000000 - Partitions: 500 - Broadcast Inner Join Test TimeAB48121620SE +/- 0.06, N = 315.0515.17

Apache Spark

Row Count: 10000000 - Partitions: 500 - Inner Join Test Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 10000000 - Partitions: 500 - Inner Join Test TimeAB48121620SE +/- 0.13, N = 315.1115.41

Apache Spark

Row Count: 10000000 - Partitions: 500 - Repartition Test Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 10000000 - Partitions: 500 - Repartition Test TimeBA48121620SE +/- 0.11, N = 313.9414.07

Apache Spark

Row Count: 10000000 - Partitions: 500 - Group By Test Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 10000000 - Partitions: 500 - Group By Test TimeAB3691215SE +/- 0.06, N = 39.8610.02

Apache Spark

Row Count: 10000000 - Partitions: 500 - Calculate Pi Benchmark Using Dataframe

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 10000000 - Partitions: 500 - Calculate Pi Benchmark Using DataframeBA3691215SE +/- 0.05, N = 313.4713.56

Apache Spark

Row Count: 10000000 - Partitions: 500 - Calculate Pi Benchmark

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 10000000 - Partitions: 500 - Calculate Pi BenchmarkAB50100150200250SE +/- 0.46, N = 3239.05240.25

Apache Spark

Row Count: 10000000 - Partitions: 500 - SHA-512 Benchmark Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 10000000 - Partitions: 500 - SHA-512 Benchmark TimeAB510152025SE +/- 0.06, N = 319.7019.79

Apache Spark

Row Count: 1000000 - Partitions: 1000 - Broadcast Inner Join Test Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 1000000 - Partitions: 1000 - Broadcast Inner Join Test TimeBA0.72231.44462.16692.88923.6115SE +/- 0.09, N = 32.623.21

Apache Spark

Row Count: 1000000 - Partitions: 1000 - Inner Join Test Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 1000000 - Partitions: 1000 - Inner Join Test TimeAB0.71331.42662.13992.85323.5665SE +/- 0.05, N = 33.173.17

Apache Spark

Row Count: 1000000 - Partitions: 1000 - Repartition Test Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 1000000 - Partitions: 1000 - Repartition Test TimeAB0.83251.6652.49753.334.1625SE +/- 0.05, N = 33.683.70

Apache Spark

Row Count: 1000000 - Partitions: 1000 - Group By Test Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 1000000 - Partitions: 1000 - Group By Test TimeAB1.24882.49763.74644.99526.244SE +/- 0.02, N = 35.325.55

Apache Spark

Row Count: 1000000 - Partitions: 1000 - Calculate Pi Benchmark Using Dataframe

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 1000000 - Partitions: 1000 - Calculate Pi Benchmark Using DataframeBA3691215SE +/- 0.05, N = 313.4613.50

Apache Spark

Row Count: 1000000 - Partitions: 1000 - Calculate Pi Benchmark

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 1000000 - Partitions: 1000 - Calculate Pi BenchmarkBA50100150200250SE +/- 0.83, N = 3239.65239.97

Apache Spark

Row Count: 1000000 - Partitions: 1000 - SHA-512 Benchmark Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 1000000 - Partitions: 1000 - SHA-512 Benchmark TimeBA1.14982.29963.44944.59925.749SE +/- 0.07, N = 35.085.11

Apache Spark

Row Count: 1000000 - Partitions: 500 - Broadcast Inner Join Test Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 1000000 - Partitions: 500 - Broadcast Inner Join Test TimeBA0.6931.3862.0792.7723.465SE +/- 0.64, N = 32.293.08

Apache Spark

Row Count: 1000000 - Partitions: 500 - Inner Join Test Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 1000000 - Partitions: 500 - Inner Join Test TimeBA0.59631.19261.78892.38522.9815SE +/- 0.01, N = 32.532.65

Apache Spark

Row Count: 1000000 - Partitions: 500 - Repartition Test Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 1000000 - Partitions: 500 - Repartition Test TimeAB0.77181.54362.31543.08723.859SE +/- 0.02, N = 33.403.43

Apache Spark

Row Count: 1000000 - Partitions: 500 - Group By Test Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 1000000 - Partitions: 500 - Group By Test TimeAB1.13182.26363.39544.52725.659SE +/- 0.09, N = 34.925.03

Apache Spark

Row Count: 1000000 - Partitions: 500 - Calculate Pi Benchmark Using Dataframe

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 1000000 - Partitions: 500 - Calculate Pi Benchmark Using DataframeBA48121620SE +/- 0.02, N = 313.4413.66

Apache Spark

Row Count: 1000000 - Partitions: 500 - Calculate Pi Benchmark

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 1000000 - Partitions: 500 - Calculate Pi BenchmarkAB50100150200250SE +/- 0.34, N = 3240.04245.15

Apache Spark

Row Count: 1000000 - Partitions: 500 - SHA-512 Benchmark Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 1000000 - Partitions: 500 - SHA-512 Benchmark TimeBA1.06882.13763.20644.27525.344SE +/- 0.06, N = 34.744.75

Phoronix Test Suite v10.8.4