spark rembrandty

AMD Ryzen 7 PRO 6850U testing with a LENOVO 21CM0001US (R22ET46W 1.16 BIOS) and AMD Rembrandt 1GB on Ubuntu 22.04 via the Phoronix Test Suite.

HTML result view exported from: https://openbenchmarking.org/result/2208040-NE-SPARKREMB86&grs.

spark rembrandtyProcessorMotherboardChipsetMemoryDiskGraphicsAudioNetworkOSKernelDesktopDisplay ServerOpenGLVulkanCompilerFile-SystemScreen ResolutionABAMD Ryzen 7 PRO 6850U @ 4.77GHz (8 Cores / 16 Threads)LENOVO 21CM0001US (R22ET46W 1.16 BIOS)AMD Device 14b516GB512GB Micron MTFDKBA512TFKAMD Rembrandt 1GBAMD Rembrandt Radeon HD AudioQualcomm Atheros QCNFA765Ubuntu 22.045.15.0-41-generic (x86_64)GNOME Shell 42.2X Server + Wayland4.6 Mesa 22.2.0-devel (git-ba461f8 2022-07-26 jammy-oibaf-ppa) (LLVM 14.0.6 DRM 3.42)1.3.219GCC 11.2.0ext41920x1200OpenBenchmarking.orgKernel Details- Transparent Huge Pages: madviseProcessor Details- Scaling Governor: amd-pstate schedutil (Boost: Enabled) - Platform Profile: balanced - CPU Microcode: 0xa404102 - ACPI Profile: balanced Java Details- OpenJDK Runtime Environment (build 11.0.15+10-Ubuntu-0ubuntu0.22.04.1)Python Details- Python 3.10.4Security Details- itlb_multihit: Not affected + l1tf: Not affected + mds: Not affected + meltdown: Not affected + mmio_stale_data: Not affected + spec_store_bypass: Mitigation of SSB disabled via prctl and seccomp + spectre_v1: Mitigation of usercopy/swapgs barriers and __user pointer sanitization + spectre_v2: Mitigation of Retpolines IBPB: conditional IBRS_FW STIBP: always-on RSB filling + srbds: Not affected + tsx_async_abort: Not affected

spark rembrandtyspark: 1000000 - 1000 - Broadcast Inner Join Test Timespark: 40000000 - 500 - Broadcast Inner Join Test Timespark: 40000000 - 100 - Inner Join Test Timespark: 10000000 - 100 - Broadcast Inner Join Test Timespark: 40000000 - 500 - Inner Join Test Timespark: 10000000 - 100 - Inner Join Test Timespark: 20000000 - 1000 - Broadcast Inner Join Test Timespark: 20000000 - 1000 - Repartition Test Timespark: 40000000 - 100 - Repartition Test Timespark: 1000000 - 500 - Inner Join Test Timespark: 40000000 - 100 - SHA-512 Benchmark Timespark: 20000000 - 100 - Inner Join Test Timespark: 1000000 - 1000 - Group By Test Timespark: 20000000 - 1000 - Group By Test Timespark: 10000000 - 2000 - Group By Test Timespark: 1000000 - 100 - Repartition Test Timespark: 20000000 - 500 - Inner Join Test Timespark: 10000000 - 2000 - Inner Join Test Timespark: 40000000 - 100 - Calculate Pi Benchmarkspark: 40000000 - 100 - Broadcast Inner Join Test Timespark: 20000000 - 100 - Group By Test Timespark: 1000000 - 2000 - SHA-512 Benchmark Timespark: 40000000 - 500 - Repartition Test Timespark: 10000000 - 100 - Group By Test Timespark: 20000000 - 500 - Repartition Test Timespark: 40000000 - 1000 - Group By Test Timespark: 1000000 - 500 - Group By Test Timespark: 1000000 - 500 - Calculate Pi Benchmarkspark: 40000000 - 500 - SHA-512 Benchmark Timespark: 1000000 - 100 - Inner Join Test Timespark: 1000000 - 2000 - Inner Join Test Timespark: 10000000 - 1000 - Group By Test Timespark: 1000000 - 100 - Group By Test Timespark: 10000000 - 500 - Inner Join Test Timespark: 40000000 - 500 - Calculate Pi Benchmarkspark: 10000000 - 2000 - Repartition Test Timespark: 10000000 - 2000 - Broadcast Inner Join Test Timespark: 40000000 - 500 - Calculate Pi Benchmark Using Dataframespark: 1000000 - 500 - Calculate Pi Benchmark Using Dataframespark: 10000000 - 500 - Group By Test Timespark: 20000000 - 100 - Repartition Test Timespark: 20000000 - 2000 - Inner Join Test Timespark: 1000000 - 2000 - Repartition Test Timespark: 40000000 - 2000 - Calculate Pi Benchmarkspark: 10000000 - 2000 - SHA-512 Benchmark Timespark: 20000000 - 1000 - Inner Join Test Timespark: 1000000 - 100 - SHA-512 Benchmark Timespark: 20000000 - 1000 - Calculate Pi Benchmarkspark: 10000000 - 100 - Repartition Test Timespark: 20000000 - 500 - Broadcast Inner Join Test Timespark: 20000000 - 100 - Calculate Pi Benchmark Using Dataframespark: 10000000 - 1000 - Repartition Test Timespark: 40000000 - 2000 - Repartition Test Timespark: 1000000 - 100 - Calculate Pi Benchmark Using Dataframespark: 40000000 - 100 - Calculate Pi Benchmark Using Dataframespark: 10000000 - 500 - Repartition Test Timespark: 1000000 - 500 - Repartition Test Timespark: 20000000 - 100 - Broadcast Inner Join Test Timespark: 20000000 - 1000 - SHA-512 Benchmark Timespark: 20000000 - 500 - Calculate Pi Benchmarkspark: 40000000 - 1000 - Inner Join Test Timespark: 10000000 - 1000 - Calculate Pi Benchmarkspark: 40000000 - 500 - Group By Test Timespark: 10000000 - 500 - Broadcast Inner Join Test Timespark: 20000000 - 2000 - Calculate Pi Benchmark Using Dataframespark: 10000000 - 1000 - Broadcast Inner Join Test Timespark: 40000000 - 2000 - Calculate Pi Benchmark Using Dataframespark: 20000000 - 2000 - Calculate Pi Benchmarkspark: 40000000 - 1000 - SHA-512 Benchmark Timespark: 10000000 - 500 - Calculate Pi Benchmark Using Dataframespark: 40000000 - 1000 - Broadcast Inner Join Test Timespark: 1000000 - 1000 - SHA-512 Benchmark Timespark: 10000000 - 1000 - Inner Join Test Timespark: 40000000 - 2000 - Group By Test Timespark: 10000000 - 100 - Calculate Pi Benchmarkspark: 1000000 - 1000 - Repartition Test Timespark: 20000000 - 500 - Group By Test Timespark: 20000000 - 1000 - Calculate Pi Benchmark Using Dataframespark: 40000000 - 1000 - Calculate Pi Benchmark Using Dataframespark: 10000000 - 500 - Calculate Pi Benchmarkspark: 10000000 - 500 - SHA-512 Benchmark Timespark: 40000000 - 1000 - Calculate Pi Benchmarkspark: 10000000 - 2000 - Calculate Pi Benchmark Using Dataframespark: 10000000 - 100 - Calculate Pi Benchmark Using Dataframespark: 40000000 - 1000 - Repartition Test Timespark: 40000000 - 100 - Group By Test Timespark: 20000000 - 2000 - Broadcast Inner Join Test Timespark: 20000000 - 500 - Calculate Pi Benchmark Using Dataframespark: 1000000 - 2000 - Calculate Pi Benchmark Using Dataframespark: 10000000 - 2000 - Calculate Pi Benchmarkspark: 20000000 - 2000 - Repartition Test Timespark: 1000000 - 2000 - Calculate Pi Benchmarkspark: 10000000 - 1000 - SHA-512 Benchmark Timespark: 1000000 - 1000 - Calculate Pi Benchmark Using Dataframespark: 20000000 - 2000 - SHA-512 Benchmark Timespark: 1000000 - 500 - SHA-512 Benchmark Timespark: 10000000 - 100 - SHA-512 Benchmark Timespark: 1000000 - 2000 - Group By Test Timespark: 10000000 - 1000 - Calculate Pi Benchmark Using Dataframespark: 1000000 - 1000 - Calculate Pi Benchmarkspark: 20000000 - 100 - SHA-512 Benchmark Timespark: 40000000 - 2000 - Inner Join Test Timespark: 40000000 - 2000 - Broadcast Inner Join Test Timespark: 20000000 - 100 - Calculate Pi Benchmarkspark: 20000000 - 2000 - Group By Test Timespark: 20000000 - 500 - SHA-512 Benchmark Timespark: 40000000 - 2000 - SHA-512 Benchmark Timespark: 1000000 - 100 - Calculate Pi Benchmarkspark: 1000000 - 1000 - Inner Join Test Timespark: 1000000 - 2000 - Broadcast Inner Join Test Timespark: 1000000 - 500 - Broadcast Inner Join Test Timespark: 1000000 - 100 - Broadcast Inner Join Test TimeAB3.2164.3361.5916.2261.3516.7128.7725.6552.392.6569.0632.485.3214.9510.943.3730.2716.80249.7460.3015.375.6350.6810.0026.2733.954.92240.0469.512.444.0110.444.5715.11249.4414.4415.6313.6613.669.8627.4530.364.15246.5420.4030.154.54241.0014.4528.6413.5113.9650.8113.6213.8314.073.4032.1035.73241.4059.25241.8235.3515.0513.5814.9213.83242.0969.9813.5659.415.1115.5834.71238.653.6814.9213.6013.58239.0519.70247.4013.5413.6050.0540.6029.4413.5013.59241.9426.46241.4719.9913.5036.284.7519.586.0913.56239.9736.5261.6161.85239.8415.7835.0070.44241.053.173.313.082.142.6258.9256.6815.1357.4315.7630.2926.9949.822.5366.0331.135.5515.5810.503.2429.1216.17240.9958.2514.905.4649.209.7225.5534.845.03245.1568.082.393.9310.654.4815.41244.7414.1715.3413.4313.4410.0227.0230.834.09250.1520.1130.554.60244.0114.6328.3013.6714.1250.2413.4913.7013.943.4331.8236.03239.4059.74239.8435.6415.1713.4815.0313.73243.7769.5113.4759.065.0815.6734.91240.013.7014.8413.5313.65240.2519.79246.2813.6013.5450.2740.4329.5613.4513.54241.1126.54242.2020.0513.4636.364.7419.626.1013.58239.6536.4861.5561.79240.0315.7934.9870.43241.0241388543.173.042.291.89OpenBenchmarking.org

Apache Spark

Row Count: 1000000 - Partitions: 1000 - Broadcast Inner Join Test Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 1000000 - Partitions: 1000 - Broadcast Inner Join Test TimeAB0.72231.44462.16692.88923.6115SE +/- 0.09, N = 33.212.62

Apache Spark

Row Count: 40000000 - Partitions: 500 - Broadcast Inner Join Test Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 40000000 - Partitions: 500 - Broadcast Inner Join Test TimeAB1428425670SE +/- 1.68, N = 364.3358.92

Apache Spark

Row Count: 40000000 - Partitions: 100 - Inner Join Test Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 40000000 - Partitions: 100 - Inner Join Test TimeAB1428425670SE +/- 1.79, N = 361.5956.68

Apache Spark

Row Count: 10000000 - Partitions: 100 - Broadcast Inner Join Test Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 10000000 - Partitions: 100 - Broadcast Inner Join Test TimeAB48121620SE +/- 0.51, N = 316.2215.13

Apache Spark

Row Count: 40000000 - Partitions: 500 - Inner Join Test Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 40000000 - Partitions: 500 - Inner Join Test TimeAB1428425670SE +/- 1.61, N = 361.3557.43

Apache Spark

Row Count: 10000000 - Partitions: 100 - Inner Join Test Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 10000000 - Partitions: 100 - Inner Join Test TimeAB48121620SE +/- 0.29, N = 316.7115.76

Apache Spark

Row Count: 20000000 - Partitions: 1000 - Broadcast Inner Join Test Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 20000000 - Partitions: 1000 - Broadcast Inner Join Test TimeAB714212835SE +/- 0.11, N = 328.7730.29

Apache Spark

Row Count: 20000000 - Partitions: 1000 - Repartition Test Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 20000000 - Partitions: 1000 - Repartition Test TimeAB612182430SE +/- 0.12, N = 325.6526.99

Apache Spark

Row Count: 40000000 - Partitions: 100 - Repartition Test Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 40000000 - Partitions: 100 - Repartition Test TimeAB1224364860SE +/- 1.35, N = 352.3949.82

Apache Spark

Row Count: 1000000 - Partitions: 500 - Inner Join Test Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 1000000 - Partitions: 500 - Inner Join Test TimeAB0.59631.19261.78892.38522.9815SE +/- 0.01, N = 32.652.53

Apache Spark

Row Count: 40000000 - Partitions: 100 - SHA-512 Benchmark Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 40000000 - Partitions: 100 - SHA-512 Benchmark TimeAB1530456075SE +/- 0.74, N = 369.0666.03

Apache Spark

Row Count: 20000000 - Partitions: 100 - Inner Join Test Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 20000000 - Partitions: 100 - Inner Join Test TimeAB816243240SE +/- 0.13, N = 332.4831.13

Apache Spark

Row Count: 1000000 - Partitions: 1000 - Group By Test Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 1000000 - Partitions: 1000 - Group By Test TimeAB1.24882.49763.74644.99526.244SE +/- 0.02, N = 35.325.55

Apache Spark

Row Count: 20000000 - Partitions: 1000 - Group By Test Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 20000000 - Partitions: 1000 - Group By Test TimeAB48121620SE +/- 0.07, N = 314.9515.58

Apache Spark

Row Count: 10000000 - Partitions: 2000 - Group By Test Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 10000000 - Partitions: 2000 - Group By Test TimeAB3691215SE +/- 0.13, N = 310.9410.50

Apache Spark

Row Count: 1000000 - Partitions: 100 - Repartition Test Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 1000000 - Partitions: 100 - Repartition Test TimeAB0.75831.51662.27493.03323.7915SE +/- 0.03, N = 93.373.24

Apache Spark

Row Count: 20000000 - Partitions: 500 - Inner Join Test Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 20000000 - Partitions: 500 - Inner Join Test TimeAB714212835SE +/- 0.94, N = 330.2729.12

Apache Spark

Row Count: 10000000 - Partitions: 2000 - Inner Join Test Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 10000000 - Partitions: 2000 - Inner Join Test TimeAB48121620SE +/- 0.16, N = 316.8016.17

Apache Spark

Row Count: 40000000 - Partitions: 100 - Calculate Pi Benchmark

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 40000000 - Partitions: 100 - Calculate Pi BenchmarkAB50100150200250SE +/- 0.92, N = 3249.74240.99

Apache Spark

Row Count: 40000000 - Partitions: 100 - Broadcast Inner Join Test Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 40000000 - Partitions: 100 - Broadcast Inner Join Test TimeAB1326395265SE +/- 1.43, N = 360.3058.25

Apache Spark

Row Count: 20000000 - Partitions: 100 - Group By Test Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 20000000 - Partitions: 100 - Group By Test TimeAB48121620SE +/- 0.21, N = 315.3714.90

Apache Spark

Row Count: 1000000 - Partitions: 2000 - SHA-512 Benchmark Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 1000000 - Partitions: 2000 - SHA-512 Benchmark TimeAB1.26682.53363.80045.06726.334SE +/- 0.06, N = 55.635.46

Apache Spark

Row Count: 40000000 - Partitions: 500 - Repartition Test Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 40000000 - Partitions: 500 - Repartition Test TimeAB1122334455SE +/- 0.25, N = 350.6849.20

Apache Spark

Row Count: 10000000 - Partitions: 100 - Group By Test Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 10000000 - Partitions: 100 - Group By Test TimeAB3691215SE +/- 0.11, N = 310.009.72

Apache Spark

Row Count: 20000000 - Partitions: 500 - Repartition Test Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 20000000 - Partitions: 500 - Repartition Test TimeAB612182430SE +/- 0.44, N = 326.2725.55

Apache Spark

Row Count: 40000000 - Partitions: 1000 - Group By Test Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 40000000 - Partitions: 1000 - Group By Test TimeAB816243240SE +/- 0.28, N = 333.9534.84

Apache Spark

Row Count: 1000000 - Partitions: 500 - Group By Test Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 1000000 - Partitions: 500 - Group By Test TimeAB1.13182.26363.39544.52725.659SE +/- 0.09, N = 34.925.03

Apache Spark

Row Count: 1000000 - Partitions: 500 - Calculate Pi Benchmark

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 1000000 - Partitions: 500 - Calculate Pi BenchmarkAB50100150200250SE +/- 0.34, N = 3240.04245.15

Apache Spark

Row Count: 40000000 - Partitions: 500 - SHA-512 Benchmark Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 40000000 - Partitions: 500 - SHA-512 Benchmark TimeAB1530456075SE +/- 0.35, N = 369.5168.08

Apache Spark

Row Count: 1000000 - Partitions: 100 - Inner Join Test Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 1000000 - Partitions: 100 - Inner Join Test TimeAB0.5491.0981.6472.1962.745SE +/- 0.02, N = 92.442.39

Apache Spark

Row Count: 1000000 - Partitions: 2000 - Inner Join Test Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 1000000 - Partitions: 2000 - Inner Join Test TimeAB0.90231.80462.70693.60924.5115SE +/- 0.04, N = 54.013.93

Apache Spark

Row Count: 10000000 - Partitions: 1000 - Group By Test Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 10000000 - Partitions: 1000 - Group By Test TimeAB3691215SE +/- 0.12, N = 310.4410.65

Apache Spark

Row Count: 1000000 - Partitions: 100 - Group By Test Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 1000000 - Partitions: 100 - Group By Test TimeAB1.02832.05663.08494.11325.1415SE +/- 0.05, N = 94.574.48

Apache Spark

Row Count: 10000000 - Partitions: 500 - Inner Join Test Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 10000000 - Partitions: 500 - Inner Join Test TimeAB48121620SE +/- 0.13, N = 315.1115.41

Apache Spark

Row Count: 40000000 - Partitions: 500 - Calculate Pi Benchmark

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 40000000 - Partitions: 500 - Calculate Pi BenchmarkAB50100150200250SE +/- 3.48, N = 3249.44244.74

Apache Spark

Row Count: 10000000 - Partitions: 2000 - Repartition Test Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 10000000 - Partitions: 2000 - Repartition Test TimeAB48121620SE +/- 0.02, N = 314.4414.17

Apache Spark

Row Count: 10000000 - Partitions: 2000 - Broadcast Inner Join Test Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 10000000 - Partitions: 2000 - Broadcast Inner Join Test TimeAB48121620SE +/- 0.11, N = 315.6315.34

Apache Spark

Row Count: 40000000 - Partitions: 500 - Calculate Pi Benchmark Using Dataframe

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 40000000 - Partitions: 500 - Calculate Pi Benchmark Using DataframeAB48121620SE +/- 0.06, N = 313.6613.43

Apache Spark

Row Count: 1000000 - Partitions: 500 - Calculate Pi Benchmark Using Dataframe

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 1000000 - Partitions: 500 - Calculate Pi Benchmark Using DataframeAB48121620SE +/- 0.02, N = 313.6613.44

Apache Spark

Row Count: 10000000 - Partitions: 500 - Group By Test Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 10000000 - Partitions: 500 - Group By Test TimeAB3691215SE +/- 0.06, N = 39.8610.02

Apache Spark

Row Count: 20000000 - Partitions: 100 - Repartition Test Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 20000000 - Partitions: 100 - Repartition Test TimeAB612182430SE +/- 0.14, N = 327.4527.02

Apache Spark

Row Count: 20000000 - Partitions: 2000 - Inner Join Test Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 20000000 - Partitions: 2000 - Inner Join Test TimeAB714212835SE +/- 0.07, N = 330.3630.83

Apache Spark

Row Count: 1000000 - Partitions: 2000 - Repartition Test Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 1000000 - Partitions: 2000 - Repartition Test TimeAB0.93381.86762.80143.73524.669SE +/- 0.04, N = 54.154.09

Apache Spark

Row Count: 40000000 - Partitions: 2000 - Calculate Pi Benchmark

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 40000000 - Partitions: 2000 - Calculate Pi BenchmarkAB50100150200250SE +/- 0.74, N = 3246.54250.15

Apache Spark

Row Count: 10000000 - Partitions: 2000 - SHA-512 Benchmark Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 10000000 - Partitions: 2000 - SHA-512 Benchmark TimeAB510152025SE +/- 0.08, N = 320.4020.11

Apache Spark

Row Count: 20000000 - Partitions: 1000 - Inner Join Test Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 20000000 - Partitions: 1000 - Inner Join Test TimeAB714212835SE +/- 0.42, N = 330.1530.55

Apache Spark

Row Count: 1000000 - Partitions: 100 - SHA-512 Benchmark Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 1000000 - Partitions: 100 - SHA-512 Benchmark TimeAB1.0352.073.1054.145.175SE +/- 0.07, N = 94.544.60

Apache Spark

Row Count: 20000000 - Partitions: 1000 - Calculate Pi Benchmark

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 20000000 - Partitions: 1000 - Calculate Pi BenchmarkAB50100150200250SE +/- 0.76, N = 3241.00244.01

Apache Spark

Row Count: 10000000 - Partitions: 100 - Repartition Test Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 10000000 - Partitions: 100 - Repartition Test TimeAB48121620SE +/- 0.17, N = 314.4514.63

Apache Spark

Row Count: 20000000 - Partitions: 500 - Broadcast Inner Join Test Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 20000000 - Partitions: 500 - Broadcast Inner Join Test TimeAB714212835SE +/- 0.25, N = 328.6428.30

Apache Spark

Row Count: 20000000 - Partitions: 100 - Calculate Pi Benchmark Using Dataframe

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 20000000 - Partitions: 100 - Calculate Pi Benchmark Using DataframeAB48121620SE +/- 0.02, N = 313.5113.67

Apache Spark

Row Count: 10000000 - Partitions: 1000 - Repartition Test Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 10000000 - Partitions: 1000 - Repartition Test TimeAB48121620SE +/- 0.07, N = 313.9614.12

Apache Spark

Row Count: 40000000 - Partitions: 2000 - Repartition Test Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 40000000 - Partitions: 2000 - Repartition Test TimeAB1122334455SE +/- 0.38, N = 350.8150.24

Apache Spark

Row Count: 1000000 - Partitions: 100 - Calculate Pi Benchmark Using Dataframe

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 1000000 - Partitions: 100 - Calculate Pi Benchmark Using DataframeAB48121620SE +/- 0.03, N = 913.6213.49

Apache Spark

Row Count: 40000000 - Partitions: 100 - Calculate Pi Benchmark Using Dataframe

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 40000000 - Partitions: 100 - Calculate Pi Benchmark Using DataframeAB48121620SE +/- 0.05, N = 313.8313.70

Apache Spark

Row Count: 10000000 - Partitions: 500 - Repartition Test Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 10000000 - Partitions: 500 - Repartition Test TimeAB48121620SE +/- 0.11, N = 314.0713.94

Apache Spark

Row Count: 1000000 - Partitions: 500 - Repartition Test Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 1000000 - Partitions: 500 - Repartition Test TimeAB0.77181.54362.31543.08723.859SE +/- 0.02, N = 33.403.43

Apache Spark

Row Count: 20000000 - Partitions: 100 - Broadcast Inner Join Test Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 20000000 - Partitions: 100 - Broadcast Inner Join Test TimeAB714212835SE +/- 0.30, N = 332.1031.82

Apache Spark

Row Count: 20000000 - Partitions: 1000 - SHA-512 Benchmark Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 20000000 - Partitions: 1000 - SHA-512 Benchmark TimeAB816243240SE +/- 0.44, N = 335.7336.03

Apache Spark

Row Count: 20000000 - Partitions: 500 - Calculate Pi Benchmark

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 20000000 - Partitions: 500 - Calculate Pi BenchmarkAB50100150200250SE +/- 0.69, N = 3241.40239.40

Apache Spark

Row Count: 40000000 - Partitions: 1000 - Inner Join Test Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 40000000 - Partitions: 1000 - Inner Join Test TimeAB1326395265SE +/- 0.85, N = 359.2559.74

Apache Spark

Row Count: 10000000 - Partitions: 1000 - Calculate Pi Benchmark

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 10000000 - Partitions: 1000 - Calculate Pi BenchmarkAB50100150200250SE +/- 1.29, N = 3241.82239.84

Apache Spark

Row Count: 40000000 - Partitions: 500 - Group By Test Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 40000000 - Partitions: 500 - Group By Test TimeAB816243240SE +/- 0.43, N = 335.3535.64

Apache Spark

Row Count: 10000000 - Partitions: 500 - Broadcast Inner Join Test Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 10000000 - Partitions: 500 - Broadcast Inner Join Test TimeAB48121620SE +/- 0.06, N = 315.0515.17

Apache Spark

Row Count: 20000000 - Partitions: 2000 - Calculate Pi Benchmark Using Dataframe

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 20000000 - Partitions: 2000 - Calculate Pi Benchmark Using DataframeAB3691215SE +/- 0.04, N = 313.5813.48

Apache Spark

Row Count: 10000000 - Partitions: 1000 - Broadcast Inner Join Test Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 10000000 - Partitions: 1000 - Broadcast Inner Join Test TimeAB48121620SE +/- 0.11, N = 314.9215.03

Apache Spark

Row Count: 40000000 - Partitions: 2000 - Calculate Pi Benchmark Using Dataframe

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 40000000 - Partitions: 2000 - Calculate Pi Benchmark Using DataframeAB48121620SE +/- 0.01, N = 313.8313.73

Apache Spark

Row Count: 20000000 - Partitions: 2000 - Calculate Pi Benchmark

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 20000000 - Partitions: 2000 - Calculate Pi BenchmarkAB50100150200250SE +/- 1.02, N = 3242.09243.77

Apache Spark

Row Count: 40000000 - Partitions: 1000 - SHA-512 Benchmark Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 40000000 - Partitions: 1000 - SHA-512 Benchmark TimeAB1632486480SE +/- 0.51, N = 369.9869.51

Apache Spark

Row Count: 10000000 - Partitions: 500 - Calculate Pi Benchmark Using Dataframe

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 10000000 - Partitions: 500 - Calculate Pi Benchmark Using DataframeAB3691215SE +/- 0.05, N = 313.5613.47

Apache Spark

Row Count: 40000000 - Partitions: 1000 - Broadcast Inner Join Test Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 40000000 - Partitions: 1000 - Broadcast Inner Join Test TimeAB1326395265SE +/- 0.72, N = 359.4159.06

Apache Spark

Row Count: 1000000 - Partitions: 1000 - SHA-512 Benchmark Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 1000000 - Partitions: 1000 - SHA-512 Benchmark TimeAB1.14982.29963.44944.59925.749SE +/- 0.07, N = 35.115.08

Apache Spark

Row Count: 10000000 - Partitions: 1000 - Inner Join Test Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 10000000 - Partitions: 1000 - Inner Join Test TimeAB48121620SE +/- 0.06, N = 315.5815.67

Apache Spark

Row Count: 40000000 - Partitions: 2000 - Group By Test Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 40000000 - Partitions: 2000 - Group By Test TimeAB816243240SE +/- 0.44, N = 334.7134.91

Apache Spark

Row Count: 10000000 - Partitions: 100 - Calculate Pi Benchmark

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 10000000 - Partitions: 100 - Calculate Pi BenchmarkAB50100150200250SE +/- 0.22, N = 3238.65240.01

Apache Spark

Row Count: 1000000 - Partitions: 1000 - Repartition Test Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 1000000 - Partitions: 1000 - Repartition Test TimeAB0.83251.6652.49753.334.1625SE +/- 0.05, N = 33.683.70

Apache Spark

Row Count: 20000000 - Partitions: 500 - Group By Test Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 20000000 - Partitions: 500 - Group By Test TimeAB48121620SE +/- 0.10, N = 314.9214.84

Apache Spark

Row Count: 20000000 - Partitions: 1000 - Calculate Pi Benchmark Using Dataframe

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 20000000 - Partitions: 1000 - Calculate Pi Benchmark Using DataframeAB3691215SE +/- 0.01, N = 313.6013.53

Apache Spark

Row Count: 40000000 - Partitions: 1000 - Calculate Pi Benchmark Using Dataframe

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 40000000 - Partitions: 1000 - Calculate Pi Benchmark Using DataframeAB48121620SE +/- 0.04, N = 313.5813.65

Apache Spark

Row Count: 10000000 - Partitions: 500 - Calculate Pi Benchmark

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 10000000 - Partitions: 500 - Calculate Pi BenchmarkAB50100150200250SE +/- 0.46, N = 3239.05240.25

Apache Spark

Row Count: 10000000 - Partitions: 500 - SHA-512 Benchmark Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 10000000 - Partitions: 500 - SHA-512 Benchmark TimeAB510152025SE +/- 0.06, N = 319.7019.79

Apache Spark

Row Count: 40000000 - Partitions: 1000 - Calculate Pi Benchmark

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 40000000 - Partitions: 1000 - Calculate Pi BenchmarkAB50100150200250SE +/- 0.40, N = 3247.40246.28

Apache Spark

Row Count: 10000000 - Partitions: 2000 - Calculate Pi Benchmark Using Dataframe

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 10000000 - Partitions: 2000 - Calculate Pi Benchmark Using DataframeAB3691215SE +/- 0.04, N = 313.5413.60

Apache Spark

Row Count: 10000000 - Partitions: 100 - Calculate Pi Benchmark Using Dataframe

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 10000000 - Partitions: 100 - Calculate Pi Benchmark Using DataframeAB3691215SE +/- 0.04, N = 313.6013.54

Apache Spark

Row Count: 40000000 - Partitions: 1000 - Repartition Test Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 40000000 - Partitions: 1000 - Repartition Test TimeAB1122334455SE +/- 0.62, N = 350.0550.27

Apache Spark

Row Count: 40000000 - Partitions: 100 - Group By Test Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 40000000 - Partitions: 100 - Group By Test TimeAB918273645SE +/- 0.27, N = 340.6040.43

Apache Spark

Row Count: 20000000 - Partitions: 2000 - Broadcast Inner Join Test Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 20000000 - Partitions: 2000 - Broadcast Inner Join Test TimeAB714212835SE +/- 0.18, N = 329.4429.56

Apache Spark

Row Count: 20000000 - Partitions: 500 - Calculate Pi Benchmark Using Dataframe

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 20000000 - Partitions: 500 - Calculate Pi Benchmark Using DataframeAB3691215SE +/- 0.06, N = 313.5013.45

Apache Spark

Row Count: 1000000 - Partitions: 2000 - Calculate Pi Benchmark Using Dataframe

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 1000000 - Partitions: 2000 - Calculate Pi Benchmark Using DataframeAB3691215SE +/- 0.05, N = 513.5913.54

Apache Spark

Row Count: 10000000 - Partitions: 2000 - Calculate Pi Benchmark

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 10000000 - Partitions: 2000 - Calculate Pi BenchmarkAB50100150200250SE +/- 0.88, N = 3241.94241.11

Apache Spark

Row Count: 20000000 - Partitions: 2000 - Repartition Test Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 20000000 - Partitions: 2000 - Repartition Test TimeAB612182430SE +/- 0.48, N = 326.4626.54

Apache Spark

Row Count: 1000000 - Partitions: 2000 - Calculate Pi Benchmark

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 1000000 - Partitions: 2000 - Calculate Pi BenchmarkAB50100150200250SE +/- 0.77, N = 5241.47242.20

Apache Spark

Row Count: 10000000 - Partitions: 1000 - SHA-512 Benchmark Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 10000000 - Partitions: 1000 - SHA-512 Benchmark TimeAB510152025SE +/- 0.08, N = 319.9920.05

Apache Spark

Row Count: 1000000 - Partitions: 1000 - Calculate Pi Benchmark Using Dataframe

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 1000000 - Partitions: 1000 - Calculate Pi Benchmark Using DataframeAB3691215SE +/- 0.05, N = 313.5013.46

Apache Spark

Row Count: 20000000 - Partitions: 2000 - SHA-512 Benchmark Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 20000000 - Partitions: 2000 - SHA-512 Benchmark TimeAB816243240SE +/- 0.22, N = 336.2836.36

Apache Spark

Row Count: 1000000 - Partitions: 500 - SHA-512 Benchmark Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 1000000 - Partitions: 500 - SHA-512 Benchmark TimeAB1.06882.13763.20644.27525.344SE +/- 0.06, N = 34.754.74

Apache Spark

Row Count: 10000000 - Partitions: 100 - SHA-512 Benchmark Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 10000000 - Partitions: 100 - SHA-512 Benchmark TimeAB510152025SE +/- 0.10, N = 319.5819.62

Apache Spark

Row Count: 1000000 - Partitions: 2000 - Group By Test Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 1000000 - Partitions: 2000 - Group By Test TimeAB246810SE +/- 0.07, N = 56.096.10

Apache Spark

Row Count: 10000000 - Partitions: 1000 - Calculate Pi Benchmark Using Dataframe

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 10000000 - Partitions: 1000 - Calculate Pi Benchmark Using DataframeAB3691215SE +/- 0.03, N = 313.5613.58

Apache Spark

Row Count: 1000000 - Partitions: 1000 - Calculate Pi Benchmark

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 1000000 - Partitions: 1000 - Calculate Pi BenchmarkAB50100150200250SE +/- 0.83, N = 3239.97239.65

Apache Spark

Row Count: 20000000 - Partitions: 100 - SHA-512 Benchmark Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 20000000 - Partitions: 100 - SHA-512 Benchmark TimeAB816243240SE +/- 0.20, N = 336.5236.48

Apache Spark

Row Count: 40000000 - Partitions: 2000 - Inner Join Test Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 40000000 - Partitions: 2000 - Inner Join Test TimeAB1428425670SE +/- 0.47, N = 361.6161.55

Apache Spark

Row Count: 40000000 - Partitions: 2000 - Broadcast Inner Join Test Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 40000000 - Partitions: 2000 - Broadcast Inner Join Test TimeAB1428425670SE +/- 0.87, N = 361.8561.79

Apache Spark

Row Count: 20000000 - Partitions: 100 - Calculate Pi Benchmark

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 20000000 - Partitions: 100 - Calculate Pi BenchmarkAB50100150200250SE +/- 0.71, N = 3239.84240.03

Apache Spark

Row Count: 20000000 - Partitions: 2000 - Group By Test Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 20000000 - Partitions: 2000 - Group By Test TimeAB48121620SE +/- 0.19, N = 315.7815.79

Apache Spark

Row Count: 20000000 - Partitions: 500 - SHA-512 Benchmark Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 20000000 - Partitions: 500 - SHA-512 Benchmark TimeAB816243240SE +/- 0.18, N = 335.0034.98

Apache Spark

Row Count: 40000000 - Partitions: 2000 - SHA-512 Benchmark Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 40000000 - Partitions: 2000 - SHA-512 Benchmark TimeAB1632486480SE +/- 0.14, N = 370.4470.43

Apache Spark

Row Count: 1000000 - Partitions: 100 - Calculate Pi Benchmark

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 1000000 - Partitions: 100 - Calculate Pi BenchmarkAB50100150200250SE +/- 0.82, N = 9241.05241.02

Apache Spark

Row Count: 1000000 - Partitions: 1000 - Inner Join Test Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 1000000 - Partitions: 1000 - Inner Join Test TimeAB0.71331.42662.13992.85323.5665SE +/- 0.05, N = 33.173.17

Apache Spark

Row Count: 1000000 - Partitions: 2000 - Broadcast Inner Join Test Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 1000000 - Partitions: 2000 - Broadcast Inner Join Test TimeAB0.74481.48962.23442.97923.724SE +/- 0.10, N = 53.313.04

Apache Spark

Row Count: 1000000 - Partitions: 500 - Broadcast Inner Join Test Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 1000000 - Partitions: 500 - Broadcast Inner Join Test TimeAB0.6931.3862.0792.7723.465SE +/- 0.64, N = 33.082.29

Apache Spark

Row Count: 1000000 - Partitions: 100 - Broadcast Inner Join Test Time

OpenBenchmarking.orgSeconds, Fewer Is BetterApache Spark 3.3Row Count: 1000000 - Partitions: 100 - Broadcast Inner Join Test TimeAB0.48150.9631.44451.9262.4075SE +/- 0.08, N = 92.141.89


Phoronix Test Suite v10.8.4